The open-source platform
for AI chat agents.
Deploy intelligent RAG chat agents powered by your own data. Start for free on our managed cloud, or self-host natively inside your own AWS account.
Live demo — RAG chat with document upload, SQL querying, and streaming responses
How VegaRAG Actually Works
A production-grade RAG platform built on AWS Fargate, LangGraph StateGraph agents, Pinecone vector search, PostgreSQL Row-Level Security, and Amazon Bedrock Nova — guarded by Microsoft Presidio PII redaction and dual-LLM hallucination checks.
Layer 1 — Cloud Infrastructure
Route 53 + ACM
Custom domain with HTTPS. SSL certificate auto-renewed via ACM. All traffic encrypted in transit end-to-end.
Application Load Balancer
Single ALB handles all traffic via priority rules: /api/* → Backend, /chat/* → Chat UI, /* → Frontend.
ECS Fargate Cluster
Serverless containers — no EC2 to manage. Auto-scales on demand, zero idle cost. Each service in its own task definition with separate IAM task roles.
VPC + Security Groups
All services share one SG. Inbound: 80, 443, 3000, 3001, 8000. No hardcoded keys — IAM Instance Profile auth only.
Layer 2 — Three Fargate Microservices
Frontend Dashboard
- AWS Cognito authentication (JWT + Refresh tokens)
- Agent CRUD — create, configure, deploy agents
- Data ingestion: URLs, PDFs, CSVs, plain text
- Workflow Studio (ReactFlow visual canvas)
- Analytics charts, chat logs, deploy embed codes
- Settings: system prompt, brand color, chat UI branding (title + logo per agent)
Backend AI Engine
- LangGraph StateGraph orchestrates every agent run end-to-end
- Hybrid Retrieval Engine: Vector (BM25+Pinecone+RRF), Structural (LLM tree walking), or blended — chosen per query
- Corrective RAG (CRAG): scores retrieved chunks and self-corrects with query rewriting
- Semantic Caching (Pinecone-backed <50ms exact-match replies)
- Microsoft Presidio PII Redaction (SSN/Email anonymisation)
- Token Bucket Rate Limiting (Multi-tenant noisy-neighbor protection)
- Output Guardrails (Dual-LLM entailment checks to block hallucinations)
Chat UI Service
- LangGraph SDK React — full streaming state management
- Next.js API proxy at /chat/api/langgraph/* (server-side)
- Translates LangGraph SDK wire calls → VegaRAG REST API
- Thread history sidebar grouped by assistantId
- Per-agent branding: custom name + logo from backend config
- Artifact renderer for structured outputs, tool call display
- SSE event reconstruction with stable message IDs
Layer 3 — LangGraph StateGraph Agent Topology
Nova Micro classifies query as casual, rag, or sql via strict JSON structured output. Zero hallucination risk.
For RAG queries, LLM picks the best retrieval strategy per query: vector, structural, or hybrid blend.
Vector: BM25+Pinecone+RRF fusion. Structural: LLM-guided tree walking. Hybrid: both + dedupe + rerank.
Scores each chunk's relevance 0-10. If avg below threshold → rewrites query + re-retrieves. Self-healing retrieval.
Bedrock Nova streams tokens. FastAPI yields each chunk as SSE to the Chat UI proxy and then to the browser.
Layer 3.5 — Hybrid Retrieval Engine (New)
3-Mode AdaptiveRetrieval Strategy — Chosen Per Query at Runtime
The Retrieval Router uses LLM classification to pick the optimal strategy for each query. Admins can also pin a strategy per agent in Settings.
- Embeds query with Amazon Titan v2 → Pinecone top-15 ANN search
- BM25 keyword scoring catches exact-match entity names and IDs
- Reciprocal Rank Fusion merges both rankings → top-5 final chunks
- Best for: semantic/conversational queries, paraphrase-style lookups
- At ingestion, Nova Micro builds a 3-level hierarchical TOC tree per document
- At query time, LLM scores section titles/summaries → descends best branch
- Returns raw text from char ranges — no embeddings needed
- Best for: section-oriented queries like 'what does chapter 3 say?'
- Runs both vector and structural retrievers in sequence
- Deduplicates overlapping passages by SHA-256 content hash
- Nova Micro reranks the combined set with 0-10 relevance scoring
- Best for: ambiguous queries or documents with mixed structure
After retrieval, CRAG scores each chunk's relevance to the query on a 0-10 scale. If the average score drops below the quality threshold, it automatically rewrites the query to be more specific and re-retrieves — preventing the LLM from hallucinating based on irrelevant context. This self-healing loop runs transparently on every RAG request.
Hybrid retrieval uses the exact same Bedrock models (Nova Micro), DynamoDB table, and Pinecone index already provisioned. Document trees are stored as additional items in the existing DynamoDB single-table design. No new AWS resources, no new services, no new bills. The structural retriever adds ~2 LLM calls per query — pennies at Nova Micro pricing.
Layer 4 — Persistent Data Stores
DynamoDB (Single Table)
- ▸ PK: USER#{email} → agent list per user
- ▸ PK: AGENT#{id} / SK: CONFIG → prompt, brand, chat title/logo
- ▸ PK: AGENT#{id} / SK: SOURCE#* → data sources
- ▸ PK: ACTIVITY#{id} / SK: ENTRY#* → full chat logs
- ▸ PK: STATS#{id} / SK: DAY#* → daily query metrics
Pinecone Vector DB
- ▸ Namespace-per-agent isolation (no cross-contamination)
- ▸ Amazon Titan v2 embeddings (1536-dim)
- ▸ Top-5 cosine similarity ANN retrieval
- ▸ Semantic Caching: Sub-50ms cache hits bypassing LLM
- ▸ Sub-50ms query latency at scale
PostgreSQL Data Warehouse
- ▸ Mandatory Row-Level Security (SET LOCAL app.current_tenant)
- ▸ Text-to-SQL via Bedrock Nova structured output
- ▸ Read-only session enforcement (blocks DROP/DELETE)
- ▸ Cross-tenant data leakage structurally impossible
- ▸ Enterprise-scale persistence and execution
OpenTelemetry + CloudWatch
- ▸ Structured JSON Logging (TraceID/SpanID injection)
- ▸ AWS X-Ray Distributed Waterfall Tracing
- ▸ CloudWatch Logs: /ecs/vegarag-* log streams
- ▸ CloudWatch Metrics: task CPU + memory graphs
- ▸ ALB access logs for traffic + error analysis
Complete Request Lifecycle — A to Z
Every single hop from the browser to Bedrock and back, in exact order.
User visits vegarag.com/chat?assistantId=bot_xxx
Browser hits Route 53 → resolves to ALB → HTTPS listener on port 443
ALB matches /chat/* priority rule → forwards to TG-ChatUI on port 3001
Chat UI Next.js container (basePath=/chat) receives the request and serves the React app
React app boots, LangGraph SDK reads assistantId from URL query param
SDK fetches /chat/api/langgraph/info → Next.js proxy returns {version} confirming proxy is alive
SDK calls POST /chat/api/langgraph/threads/search to load thread history
Next.js proxy hits backend GET /api/agents/{id}/activity → groups by session_id → returns thread list
User types a message and submits the chat form
SDK fires POST /chat/api/langgraph/threads/{thread_id}/runs/stream with message payload
Next.js proxy receives request, extracts query text + bot_id
Forwards to backend POST /api/chat as {bot_id, session_id, query} JSON body
FastAPI receives request, starts LangGraph StateGraph run
Entry node loads conversation history from DynamoDB, initializes GraphState
Intent Router node calls Bedrock Nova Micro with structured schema
LLM returns JSON: {intent: 'rag'|'sql'|'casual'} — conditional edge dispatches to correct branch
RAG branch: Retrieval Router picks strategy → Vector, Structural, or Hybrid
Vector: Titan v2 embed + Pinecone + BM25+RRF. Structural: LLM walks document tree. Hybrid: both + rerank.
CRAG node scores chunk relevance 0-10. Low scores → query rewrite + re-retrieve
Self-correcting retrieval prevents hallucination from irrelevant context. Context injected as <context>XML</context>
Next.js proxy re-wraps backend SSE as valid LangGraph values events
Browser LangGraph SDK React hook receives events, updates message state, renders tokens progressively
Stream ends. SDK fires GET /chat/api/langgraph/threads/{id}/state
Proxy fetches full session from DynamoDB, reconstructs LangGraph message format with stable IDs
Deploy exactly how you want.
Choose between zero-config managed hosting up to 250k free tokens, or own your data completely by deploying the open-source platform into your AWS account.
The Free SaaS Trial
Test your agents instantly without touching AWS. We provide Amazon Bedrock compute and Pinecone vector hosting free for up to 250,000 tokens per month and 100MB of storage.
Self-Host Open Source
Use our exhaustive GitHub deployment instructions to provision 100% of the infrastructure inside your own AWS VPC using strict ECS Fargate zero-key Task IAM boundaries.