Reference architecture

Production RAG Reference Architecture

A source-to-answer architecture for governed ingestion, access-aware retrieval, reranking, grounded generation, evaluation, and index operations.

18 minVerified 2026-07-214 primary sources
A governed production AI reference architecture with observable, secured service boundaries.

Architecture

A source-to-answer architecture for governed ingestion, access-aware retrieval, reranking, grounded generation, evaluation, and index operations.

Intent and policy
Validated request
Controlled execution
Independent verification
Telemetry and response
Conceptual teaching model synthesized from:Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksIntroduction to Information Retrievalpgvector DocumentationPROV Overview

The architecture separates intent, policy, state, execution, evidence, and response. Each boundary has one accountable owner and an observable contract. Probabilistic components may propose or classify; deterministic software owns permissions, budgets, state transitions, and release conditions.

Architecture claim
Representative test
Production signal
Decision gate
Review or rollback
Conceptual teaching model synthesized from:Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksIntroduction to Information Retrievalpgvector DocumentationPROV Overview

Decisions

| Decision | Required evidence | Review trigger | |---|---|---| | Choose authoritative sources and lineage before selecting a vector store. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Enforce authorization before candidates enter ranking or model context. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Version ingestion, embeddings, indexes, prompts, and evaluation sets together. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Separate retrieval sufficiency, generation support, and product outcome metrics. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change |

Alternatives and trade-offs

A centralized control plane simplifies policy and evidence collection but can become a latency and availability dependency. Distributed enforcement reduces central bottlenecks but requires consistent identity, versions, and audit semantics. Prefer the smallest boundary that can enforce the invariant and prove the outcome.

Failure modes

  • Stale or undeleted chunks remain retrievable.
  • Global caches or indexes cross tenant boundaries.
  • Generation is graded without retrieval-stage evidence.
  • Index cutover has no shadow comparison or rollback.

Operational checklist

  • [ ] Every chunk has source revision and ACL provenance.
  • [ ] Negative authorization tests run against retrieval and caches.
  • [ ] Golden queries include rare, conflicting, and unanswerable cases.
  • [ ] Index rebuild and rollback are rehearsed.
  • [ ] Citations are checked for both validity and claim support.

Connected practice

Sources

  • rag-paper
  • information-retrieval-book
  • pgvector-docs
  • w3c-prov-overview