Case study — claim-level rag knowledge base
Claim-Level RAG Knowledge Base
Most RAG retrieves paragraphs and hopes the model reads them right. This one retrieves claims. Every document my agency ingests is shredded by an LLM into atomic facts (each typed, scored for confidence, and tagged with how trustworthy its source is), then embedded and stored in Postgres with pgvector. When an agent asks a question, it gets back a ranked set of facts with their receipts attached, not a wall of text. It's the live knowledge base my agent fleet actually reads from.
in practice
What it's actually like to use
Sources flow in on a schedule: industry digests, competitor reports, internal research docs. Each one is embedded and stored, and then the interesting step: an LLM reads it and pulls out the 5–15 claims it actually makes. One fact per claim, each classified by type (a trend, a platform change, a benchmark) and scored for confidence and how authoritative the source is. Today that's ~734 documents broken into ~8,240 claims.
When my research agent needs to know something, it doesn't get handed three long documents to skim. It gets the specific claims that answer the question, ranked by semantic similarity, each carrying its confidence and source authority. That's the difference between "here are some pages that mention this" and "here are the six things we actually know, and how much to trust each."
This isn't a demo store. It's the production knowledge layer behind the agent platform, wired into the research agent's tools and queried live. Verifiable both ways: the retrieval code and the schema are real, and the row counts above are what's in the database right now.
The practical outcome: agency research compounds instead of rotting. 734 documents are now 8,240 queryable facts.
architecture
How it's put together
write path — building the base
a source lands
A research digest, a report, a Notion doc — pulled in on a schedule.
Upsert keys on (source_type, source_id), so re-ingesting the same source updates in place instead of duplicating.
claim extraction — the signature move
Claude shreds each document into 5–15 atomic claims — one fact each, under 200 characters, classified by type and scored for confidence and source authority. Then the claims are batch-embedded and stored.
PostgreSQL + pgvector (Supabase)
~734 documents · ~8,240 claims · HNSW cosine index on 1024-d vectors
read path — retrieval over facts
a question — from an agent or a human
- 1Embed the query — Voyage voyage-3-lite → a 1024-d vector
- 2Cosine match over claims — search_knowledge_claims RPC — HNSW index, similarity ≥ 0.3, filter by type / authority
- 3Text fallback — if embedding is unavailable, ilike search so it never returns nothing
Ranked claims come back — each with its type, confidence, and source authority — so the answer is a set of scored facts, not a wall of text to re-read.
Retrieval over facts, not chunks
The core decision: don't retrieve document chunks, retrieve claims. An LLM extracts atomic assertions from every document into a knowledge_claims table: each row one fact, with a type, a confidence score, and a source-authority rating. Retrieval then works at the granularity of individual facts.
pgvector, not a separate vector DB
Documents and claims both live in Postgres with 1024-dimension embeddings in a vector column, indexed with HNSW for fast approximate nearest-neighbor. One database holds the text, the metadata, and the vectors, with no separate store to keep in sync.
Semantic search as a SQL function
Search is a Postgres function: embed the query with Voyage, then rank by cosine distance (<=>) with a similarity floor and optional filters on claim type and source authority. The agent calls it like any other RPC and gets scored rows back.
Idempotent, hash-guarded ingestion
Ingestion upserts on (source_type, source_id) and skips anything whose content_hash hasn't changed, so the scheduled jobs can re-scan sources freely without duplicating documents or re-spending on embeddings.
Graded before it counts
Documents carry an LLM grader's verdict: a score, a pass/fail, and feedback. The same quality-gate discipline that governs the agent fleet's output governs what's allowed to become long-term memory.
A read surface for other systems
The whole thing exists to be queried by software, not clicked through: the research agent's searchClaims / searchDocumentstools call it directly. Building it as a service is what makes it shared infrastructure the rest of the platform leans on.
engineering highlights
The parts I'm proud of
The unit of retrieval is a claim, not a chunk
Chunk-based RAG returns passages and makes the model re-derive the point. Here an extraction prompt turns each document into typed, scored facts up front, so retrieval returns assertions an agent can act on directly, each with a confidence and a source-authority tag it can weigh.
# LLM extracts atomic claims from each document
claims = callClaudeJSON(EXTRACTION_PROMPT, content)
# → [{ "claim": "...", "claim_type": "trend", "confidence": 0.5 },
# { "claim": "...", "claim_type": "platform_update", "confidence": 0.8 }, ...]
# each claim embedded (batched) + stored with a source_authority rating,
# linked back to its document (ON DELETE CASCADE keeps the graph clean)Semantic search lives in the database
No bolt-on vector service. The embeddings sit next to the data and search is a Postgres function over an HNSW index. Cosine similarity, a threshold, and metadata filters all resolve in one query the agent calls as an RPC.
SELECT kc.id, kc.claim, kc.claim_type, kc.confidence, kc.source_authority,
1 - (kc.embedding <=> query_embedding) AS similarity
FROM knowledge_claims kc
WHERE kc.embedding IS NOT NULL
AND 1 - (kc.embedding <=> query_embedding) > similarity_threshold
AND (filter_claim_type IS NULL OR kc.claim_type = filter_claim_type)
AND (filter_source_authority IS NULL OR kc.source_authority = filter_source_authority)
ORDER BY kc.embedding <=> query_embedding -- HNSW-backed
LIMIT match_count;It degrades instead of failing
Re-ingesting is cheap and safe
stack
Built with
Supabase Postgres + pgvector (HNSW cosine index) · Voyage AI voyage-3-lite embeddings (1024-d) · Claude for claim extraction and document grading · Node (ESM) ingestion jobs · PostgREST / RPC read surface consumed by the agent fleet.
The live production knowledge base behind the agent platform; schema, retrieval function, and row counts are shown as they are in the database. A separate self-hosted R&D prototype explored a tri-store variant (adding a vector DB and a Neo4j knowledge graph); this is the version that shipped.