bigRAG
Getting Started

Quickstart

Configure Turbopuffer, then upload and search your first document with bigRAG.

Follow these steps to go from zero to Turbopuffer-backed search over your documents.

Start bigRAG

docker compose up -d

Wait for all services to be healthy, then verify:

curl http://localhost:4000/health

Create the first admin

A fresh install has no users. Create the first admin — this endpoint is only available while needs_setup: true.

export BASE="http://localhost:4000"

curl -X POST $BASE/v1/auth/setup \
  -H "Content-Type: application/json" \
  -d '{
    "email": "admin@example.com",
    "password": "a-strong-password",
    "display_name": "Admin"
  }' \
  -c cookies.txt

The response sets a bigrag_session cookie. The admin UI exposes the same flow at http://localhost:3000/setup, then redirects to /onboarding where you create one verified embedding preset and use Save and Finish on the Turbopuffer step before collection work.

Complete provider onboarding

In the admin UI, finish /onboarding by adding one embedding preset and saving Turbopuffer connection details. A successful Turbopuffer save sends you to /overview.

You can skip Turbopuffer during onboarding for a read-only tour, but collections cannot ingest or query until the backend has a working Turbopuffer connection.

Mint an API key

For SDKs, CI jobs, and backend services, mint a long-lived API key. The plaintext key is returned once:

curl -X POST $BASE/v1/admin/api-keys \
  -b cookies.txt \
  -H "Content-Type: application/json" \
  -d '{"name": "quickstart"}' \
  | tee /tmp/api-key.json

export BIGRAG_API_KEY=$(jq -r .key /tmp/api-key.json)

All further examples use $BIGRAG_API_KEY as a bearer token. See Authentication for scopes.

Create a collection

A collection groups documents that share the same embedding configuration and Turbopuffer namespace.

curl -X POST $BASE/v1/collections \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "knowledge_base",
    "description": "Company knowledge base",
    "embedding_provider": "openai",
    "embedding_model": "text-embedding-3-small",
    "embedding_api_key": "'"$OPENAI_API_KEY"'",
    "chunk_size": 512,
    "chunk_overlap": 50
  }'

No managed embedding key handy? Use embedding_provider: "openai_compatible" with embedding_base_url pointing at Ollama, vLLM, TEI, or any OpenAI-compatible endpoint. For local gateways without auth, set embedding_api_key to any non-empty placeholder. See Embeddings.

Upload a document

curl -X POST $BASE/v1/collections/knowledge_base/documents \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -F "file=@handbook.pdf" \
  -F 'metadata={"department": "engineering"}' \
  | tee /tmp/document.json

export DOC_ID=$(jq -r .id /tmp/document.json)

The document starts as pending and transitions to processing then ready.

Watch ingestion status

while true; do
  doc=$(curl -s "$BASE/v1/collections/knowledge_base/documents/$DOC_ID" \
    -H "Authorization: Bearer $BIGRAG_API_KEY")
  status=$(jq -r '.status' <<< "$doc")
  jq -r '.progress.message // .status' <<< "$doc"
  [[ "$status" == "ready" || "$status" == "failed" ]] && break
  sleep 2
done

Query the collection

Once documents are ready, you can query Turbopuffer-backed semantic search:

curl -X POST $BASE/v1/collections/knowledge_base/query \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is the PTO policy?",
    "top_k": 5
  }'

Next Steps

  • Open the admin UI at http://localhost:3000 to review collections, documents, and query analytics.
  • Learn about Collections and per-collection chunking, reranking, metadata schemas, and tenant-aware filtering.
  • Explore Search modes backed by Turbopuffer — semantic, keyword, and hybrid.
  • Set up Webhooks for collection and connector sync notifications.
  • Read the full API Reference.

On this page