RAG Template for Bult.ai

​

Overview

​

Architecture

​

Components

​

Features

• Multi-model LLM support — switch between OpenAI, Anthropic, Google AI, or local Ollama models via environment variable
• Advanced RAG pipeline — hybrid search (BM25 + vector), cross-encoder reranking, HyDE query transformation, multi-query retrieval, query decomposition
• Document processing — PDF, DOCX, PPTX, TXT, MD, CSV, JSON, HTML; automatic OCR for scanned PDFs via Tesseract
• Inline citations — every response cites source documents with relevance scores
• Authentication — JWT login/register + optional Google OAuth
• Analytics dashboard — usage metrics, cost tracking, query latency, top projects
• Conversation export — Markdown, JSON, and PDF export with full Unicode support
• Background processing — async job queue with progress tracking and retry logic
• Single-page frontend — clean UI with streaming responses, markdown rendering, dark mode

​

Deploy RAG on bult.ai

​

Prerequisites

• GitHub account
• OpenAI API key (get one here)
• Bult.ai account

​

Step 1: Fork or clone this repository

​

Step 2: Create the app service (GitHub)

1. On bult.ai, click Create > GitHub
2. Select your forked repository
3. Go to the Git tab and change build settings from Nixpacks to Dockerfile. Set Dockerfile Path to Dockerfile and Dockerfile Context to ”.”
4. Inside the service settings, set the port to 8002
5. Go to Environment Variables and add the variables from .env.example.

​

Step 3: Create the PostgreSQL database

1. Click Create > in the list of services, go to Databases > Postgres
2. This creates a Postgres instance from a built-in template — it’s automated
3. The only thing you need to configure is the environment variables:

​

Step 4: Create the pgvector service (Docker)

1. Click Create > Docker
2. Docker image: ankane/pgvector:latest
3. Name this service to match the hostname in your PG_CONN (e.g., pgvector)
4. Deploy the service
5. After it’s running, add a volume mounted at /var/lib/postgresql/data for persistent storage
6. Add internal port 5432

​

Step 5: Deploy and verify

1. All three services should show as running

2. Check the app service logs — you should see database migrations and the worker starting
3. Open the public URL for your app service. Register a user and start chatting.

​

Troubleshooting

​

Optional: Google Sign-In

1. Go to Google Cloud Console and create a project (or use an existing one)
2. Navigate to APIs & Services > OAuth Consent Screen and configure it
3. Go to Credentials > Create OAuth Client ID > Web Application
4. Under Authorized redirect URIs, add your bult.ai callback URL. The format is:

https://<your-project>.<region>.bult.app/api/auth/google/callback

5. Copy the Client ID and Client Secret, then add these environment variables to your app service:

6. Redeploy. The login page will show a “Sign in with Google” button.

​

Local Development

​

Prerequisites

• Python 3.10+
• PostgreSQL 14+ with pgvector extension
• OpenAI API key

​

Setup

# Clone your fork
git clone https://github.com/YOUR_USERNAME/rag-template-multimodel.git
cd rag-template-multimodel

# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: .\venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Set up database
createdb ragdb
psql -d ragdb -c "CREATE EXTENSION IF NOT EXISTS vector;"

# Schema is created automatically on first app startup

# Configure environment
cp .env.example .env

# Edit .env -- set OPENAI_API_KEY and PG_CONN at minimum

# Run
python app.py

python core/worker.py

​

How It Works

​

Document processing pipeline

1. User uploads a file through the web UI
2. A background job is created (status: queued)
3. The worker picks up the job and processes the document:

• Loads the file with PyMuPDF (PDFs) or LlamaIndex readers (other formats)
• Detects if the PDF has a text layer — if not, runs OCR (Tesseract)
• Optionally extracts tables with pdfplumber
• Splits text into chunks (semantic or fixed-size, configurable)
• Generates embeddings via OpenAI API
• Stores chunks + embeddings in PostgreSQL with pgvector

4. Document status updates in real time: queued > processing > ready

​

Query pipeline

1. User asks a question in a conversation
2. The query goes through optional transformations: HyDE, multi-query expansion, decomposition
3. Hybrid search combines BM25 keyword matching + vector similarity (configurable alpha balance)
4. Cross-encoder reranks the top candidates for precision
5. Top chunks are sent to the LLM with a system prompt that enforces inline citations
6. Response streams back to the UI character by character

​

Configuration

​

Required

​

LLM provider

​

RAG tuning

​

Project Structure

rag-template-multimodel/
|-- app.py # FastAPI entry point, all endpoints, DB bootstrap
|-- core/
| |-- config.py # Environment-driven configuration
| |-- db.py # PostgreSQL connection pool
| |-- auth.py # JWT authentication, user registration/login
| |-- oauth.py # Google OAuth2 integration
| |-- tasks.py # Document processing: load, chunk, embed, OCR
| |-- retrieval.py # RAG pipeline: hybrid search, reranking, HyDE
| |-- export.py # Conversation export (Markdown, JSON, PDF)
| |-- worker.py # Background worker logic
| +-- logging_config.py # Structured logging setup
|-- providers/
| |-- base.py # Abstract LLM/embedding provider interface
| |-- registry.py # Provider auto-detection and registration
| |-- llm_openai.py # OpenAI LLM provider
| |-- llm_anthropic.py # Anthropic LLM provider
| |-- llm_ollama.py # Ollama LLM provider
| +-- embedder_openai.py # OpenAI embedding provider
|-- scripts/
| +-- create_admin.py # Create admin user from CLI
|-- static/
| +-- index.html # Single-page frontend
|-- Dockerfile # Docker image with OCR dependencies
|-- nixpacks.toml # Nixpacks config (used by bult.ai)
|-- requirements.txt # Python dependencies
+-- .env.example # Environment variable template

​

API Reference

​

Authentication

POST /api/auth/register -- Create account (email + password)
POST /api/auth/login -- Get JWT token
GET /api/auth/me -- Get current user info
GET /api/auth/google -- Start Google OAuth flow (if configured)

​

Projects & Documents

GET /api/projects -- List user's projects
POST /api/projects -- Create project
DELETE /api/projects/{id} -- Delete project and all its data
POST /api/upload -- Upload document to project
DELETE /api/documents/batch -- Delete multiple documents
GET /api/projects/{id}/documents -- List documents in project

​

Chat

GET /api/projects/{id}/conversations -- List conversations
POST /api/projects/{id}/conversations -- Create conversation
POST /api/chat -- Send message (SSE streaming response)
GET /api/conversations/{id}/export?format=md -- Export conversation (md|json|pdf)

​

Analytics

GET /api/analytics -- Usage stats, query history, top documents

​

System

GET /api/health -- Health check (no auth required)
GET /api/jobs/{id}/progress -- Job processing progress

​

Example: Send Chat Message

​

Customization

​

Change LLM provider

LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-your-key
LLM_MODEL=claude-sonnet-4-20250514

​

Customize the system prompt

​

Change the UI theme

​

Default Retrieval Configuration

• Embedding model: text-embedding-3-large
• Embedding dimension: 3072
• Chunk size: 1024 tokens
• Chunk overlap: 200 tokens
• Top K before reranking: 50
• Top K after reranking: 8
• Hybrid search alpha: 0.5