Enterprise Retrieval-Augmented Generation Architecture: Design, Deployment, and Security

Overview

Retrieval-augmented generation (RAG) is an architecture pattern that combines a large language model (LLM) with a retrieval system that fetches relevant enterprise data at query time. Its core purpose is to improve answer accuracy, reduce hallucinations, and ground model outputs in governed sources such as knowledge bases, ticketing systems, document repositories, and internal APIs.

Enterprises adopt RAG because fine-tuning alone is slow, expensive, and poorly suited to frequently changing knowledge. With RAG, teams can keep data in existing systems, index approved content into a vector or hybrid search layer, and enforce policy at retrieval time. This makes RAG useful for internal copilots, support assistants, engineering knowledge search, and regulated document workflows.

Architecture

A typical enterprise RAG stack includes:

• Ingestion pipeline for SharePoint, Confluence, S3, file shares, and databases
• Preprocessing layer for parsing, chunking, metadata extraction, PII tagging, and deduplication
• Embedding service to convert chunks into vectors
• Vector or hybrid index such as OpenSearch, Azure AI Search, or Pinecone
• Retriever and re-ranker to select the most relevant passages
• Prompt orchestration layer to assemble context, citations, and system instructions
• LLM inference endpoint hosted privately or via managed API
• Observability and policy controls for tracing, redaction, and access decisions

[User App] ---- 1. query ----> [API Gateway] ---- 2. authz ----> [RAG Orchestrator]
[RAG Orchestrator] ---- 3. embed query ----> [Embedding Model]
[RAG Orchestrator] ---- 4. search ----> [Vector Index]
[Vector Index] ---- 5. top-k chunks ----> [RAG Orchestrator]
[RAG Orchestrator] ---- 6. prompt + context ----> [LLM Endpoint]
[LLM Endpoint] ---- 7. grounded answer ----> [RAG Orchestrator]
[RAG Orchestrator] ---- 8. response + citations ----> [User App]
[Content Sources] |
v
[Ingestion Jobs] ---- 9. parse/chunk/tag ----> [Embedding Model] ---- 10. vectors ----> [Vector Index]

Deployment models include fully managed cloud RAG, self-managed Kubernetes, and hybrid designs where documents remain on-premises while embeddings and inference run in cloud VPCs. The most common enterprise pattern is hybrid retrieval with private networking, customer-managed keys, and source-aware authorization.

Implementation Guide

1. Create an OpenSearch vector index and enable k-NN.
2. Deploy an embedding service and LLM endpoint behind private networking.
3. Ingest documents, chunk them, and attach metadata such as source, owner, classification, and ACLs.
4. Implement query-time filtering so retrieval respects user entitlements.
5. Add observability for latency, retrieval quality, token use, and prompt traces.

CLI commands

aws opensearchserverless create-collection --name rag-prod --type VECTORSEARCH
kubectl create namespace rag
helm repo add opensearch https://opensearch-project.github.io/helm-charts/
helm upgrade --install rag-api oci://ghcr.io/open-webui/open-webui --namespace rag
python ingest.py --source s3://corp-docs/policies --index rag-knowledge --chunk-size 800 --chunk-overlap 120
curl -X POST https://rag-api.internal/v1/query -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"query":"What is the retention policy for audit logs?","top_k":5}'

Retrieval policy config

rag:
  index: rag-knowledge
  retrieval:
    strategy: hybrid
    top_k: 8
    rerank: true
    min_score: 0.72
  security:
    acl_filter: true
    pii_redaction: true
    encryption: kms
  llm:
    provider: azure-openai
    model: gpt-4o-mini
    temperature: 0.1

Code Examples

curl -X PUT "https://opensearch.internal:9200/rag-knowledge" -u admin:$PASS -H "Content-Type: application/json" -d '{"settings":{"index":{"knn":true}},"mappings":{"properties":{"content":{"type":"text"},"embedding":{"type":"knn_vector","dimension":1536},"source":{"type":"keyword"},"acl":{"type":"keyword"}}}}'

from opensearchpy import OpenSearch
client = OpenSearch(hosts=[{"host": "opensearch.internal", "port": 9200}], http_auth=("svc_rag", "secret"), use_ssl=True, verify_certs=True)
def search(query_vector, groups):
    body = {"size": 5, "query": {"bool": {"filter": [{"terms": {"acl": groups}}], "must": [{"knn": {"embedding": {"vector": query_vector, "k": 5}}}]}}}
    return client.search(index="rag-knowledge", body=body)

{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"rag-app-config","namespace":"rag"},"data":{"APP_ENV":"prod","ENABLE_CITATIONS":"true","OTEL_EXPORTER_OTLP_ENDPOINT":"http://otel-collector.observability:4317","AUTHZ_MODE":"jwt-groups","MAX_CONTEXT_TOKENS":"12000"}}

Security Hardening

• Encrypt everywhere: TLS 1.2+ in transit, AES-256 at rest, and customer-managed KMS keys for indexes, object storage, and model caches.
• Enforce least privilege: separate service accounts for ingestion, retrieval, and administration; map user JWT groups to metadata filters.
• Protect sensitive data: classify documents during ingestion, redact PII before embedding when required, and avoid storing raw secrets in prompts.
• Constrain model behavior: low temperature, citation-required prompts, output filtering, and allowlisted tools only.
• Audit everything: log source documents retrieved, policy decisions, prompt hashes, and model IDs for forensic traceability.

Comparison

Troubleshooting

1. Low relevance from retrieval
   • Log sample: WARN retriever score below threshold query_id=8a12 top_score=0.41 min_score=0.72
   • Fix: reduce chunk size variance, enable hybrid search, and add re-ranking.
2. Authorization filter returns zero documents
   • Log sample: ERROR authz filter denied all results user=jdoe groups=[] index=rag-knowledge
   • Fix: verify JWT group claims, group-to-ACL mapping, and metadata normalization in ingestion.
3. Embedding dimension mismatch
   • Log sample: opensearchpy.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Vector dimension mismatch. Expected: 1536, Given: 1024')
   • Fix: recreate the index with the correct dimension or align the embedding model to the index schema.

Best Practices

Do

• Use hybrid retrieval combining keyword and vector search for policy documents and technical manuals.
• Store rich metadata such as business unit, sensitivity, owner, and retention class.
• Measure groundedness with citation coverage, answer faithfulness, and retrieval precision.
• Separate dev and prod indexes to avoid accidental exposure of draft content.

Don't

• Do not embed everything blindly; exclude secrets, credentials, and low-value duplicate content.
• Do not bypass source ACLs just because the vector index is internal.
• Do not rely on prompt instructions alone for data protection; enforce controls in retrieval and serving layers.
• Do not ignore cost telemetry; long contexts and excessive top-k values can materially increase token spend.