Skip to content

hubPattern — Ingest → Carve → Reason

The chain for when the source graph is larger than one backend can reason over. Ingest it all in parallel, carve the slice you actually need, park the rest, and reason over the right-sized slice. This is scale meets hardware.

Green is parallel and shipped; amber is single-threaded and shipped; purple is the carve waist — on the roadmap.

When to use it

The source graph exceeds what a single backend can close over — up to the 8.2-billion-triple extreme. You still ingest it in full (that scales), but you do not reason over the whole thing.

Reading the chain

PositionVerbStatus
HeadImport + Seal (full graph)Shipped — scales to 8.2 B
WaistCarve (slice) + Seal (slice) + Unload (source)Roadmap — C1/C2/C4; manual path today
TailReasonValidateQuery (slice)Shipped — single-threaded, over the sized slice

A worked scenario — reason over a neighborhood

You have a large source graph and want to reason only over the neighborhood of one seed entity. Each step below is a shipped primitive; the one-call Carve / Unload verbs that will fold the waist together are the roadmap (C1–C4).

Head — ingest the full source (shipped, scales to 8.2 B)

sql
SELECT pgrdf.add_graph(1);
SELECT pgrdf.load_turtle_staged_run('/data/source.nt', 1, 0);
--  → {"ok": true, "quads": ..., "phase_ms": { ... }}

The staged loader fans across the worker pool — this is the half that scales, proven on the full 8.2 B graph.

Waist (manual today) — carve the slice, park the source

construct the neighborhood you need and reload it into a fresh, small graph with its own dictionary:

sql
-- carve: a query-defined slice → /tmp/slice.nt
SELECT pgrdf.construct(
  'PREFIX ex: <http://example.org/>
   CONSTRUCT { ?s ?p ?o }
   WHERE { ex:seed (ex:related){1,2} ?s . ?s ?p ?o }');

-- reload the slice (fresh small dictionary), then park the source
SELECT pgrdf.add_graph(900);
SELECT pgrdf.load_turtle_staged_run('/tmp/slice.nt', 900, 0);
ALTER TABLE pgrdf._pgrdf_quads DETACH PARTITION pgrdf._pgrdf_quads_g1;  -- park the source

CONSTRUCT, the staged loader, and partition DETACH are each covered in the test suite; the one-call verbs will replace this hand-assembly and re-encode the slice in place.

Tail — reason and validate the right-sized slice (shipped)

sql
SELECT pgrdf.materialize(900, 'owl-rl');   -- single-threaded, but over the small slice
SELECT pgrdf.validate(900, 901);

Reasoning now runs over a graph sized to your box, not the full source — see Load → Reason → Query for the reasoning step on its own.

On v0.6.14 today

The head and tail are shipped and verified; the waist — the one-call Carve and Unload verbs — is the carving line on the roadmap (C1–C4, v0.6.15–v0.6.18). The manual path above runs on v0.6.14 today.

See also

MIT licensed. Documentation for pgRDF — built with VitePress, served via GitHub Pages.