Collections

A collection is a logical grouping of documents that share the same embedding configuration and Turbopuffer namespace. The namespace holds dense vectors, chunk text, payload metadata, and full-text search indexes for that collection.

What is a Collection?

When you create a collection, you define:

• Embedding provider and model — how documents will be embedded (e.g., OpenAI text-embedding-3-small)
• Chunk size and overlap — how documents are split into searchable segments
• Default query settings — top_k, min_score, and search_mode defaults
• Turbopuffer namespace — the backend search object used for writes, deletes, exports, filters, and retrieval
• Multimodal element handling — whether ingestion stores headings, tables, equations, images, page bounds, and optional LLM summaries

All documents within a collection use the same embedding model, ensuring consistent vector dimensions for search.

Creating a Collection

curl -X POST http://localhost:4000/v1/collections \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "research_papers",
    "description": "Academic research papers",
    "embedding_provider": "openai",
    "embedding_model": "text-embedding-3-small",
    "embedding_api_key": "sk-...",
    "dimension": 1536,
    "chunk_size": 512,
    "chunk_overlap": 50
  }'

Collection Properties

Collection creation validates the saved Turbopuffer connection and any existing namespace schema for the collection name. The namespace is created on the first document or vector write, and chunk text is stored with full-text search enabled so semantic, keyword, and hybrid retrieval use the same backend.

Collection Stats

curl http://localhost:4000/v1/collections/research_papers/stats \
  -H "Authorization: Bearer $BIGRAG_API_KEY"

Returns document_count, total_chunks, total_tokens, total_size_bytes, and per-status counts.

Default Query Settings

Each collection can define default query settings that apply when not specified per-query:

curl -X POST http://localhost:4000/v1/collections \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "docs",
    "default_top_k": 20,
    "default_min_score": 0.3,
    "default_search_mode": "hybrid"
  }'

When querying, if top_k, min_score, or search_mode are omitted, the collection's defaults are used.

Reranking

Collections can enable server-side reranking using the Cohere Rerank API:

curl -X POST http://localhost:4000/v1/collections \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "docs",
    "reranking_enabled": true,
    "reranking_model": "rerank-v3.5",
    "reranking_api_key": "your-cohere-key"
  }'

You can override reranking per query by passing "rerank": true or "rerank": false.

Multimodal Elements

Set multimodal_enabled: true to preserve document structure alongside chunk vectors. Ingestion stores first-class element records for headings, tables, equations, images, page numbers, bounding boxes, captions, and nearby context. It does not retain binary image/table assets after temporary staging cleanup. Query and chat responses remain backward compatible; multimodal provenance appears in multimodal_elements when a retrieved chunk overlaps stored elements.

Set multimodal_enrichment_enabled: true to queue asynchronous LLM summaries for image, table, and equation elements. Text-only or heading-only element sets are stored without queuing an enrichment job. Enrichment reuses the collection's own embedding key and runs only for collections on the openai provider; collections using openai_compatible, Cohere, or Voyage are not enriched. Core document readiness is still based on parse/chunk/embed completion; enrichment can finish later or fail independently.

Updating a Collection

curl -X PUT http://localhost:4000/v1/collections/research_papers \
  -H "Authorization: Bearer $BIGRAG_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "description": "Updated description",
    "metadata": { "team": "engineering" },
    "reranking_enabled": true,
    "reranking_model": "rerank-v3.5",
    "default_top_k": 20,
    "default_min_score": 0.3,
    "default_search_mode": "hybrid"
  }'

All fields are optional — only provided fields are updated.

Deleting a Collection

Deleting a collection removes all its documents and vectors:

curl -X DELETE http://localhost:4000/v1/collections/research_papers \
  -H "Authorization: Bearer $BIGRAG_API_KEY"