Skip to main content

Recovery & Reindexing

Procedures for recovering from bad state and choosing the right reindex strategy. The guiding principle: zero-downtime whenever possible, destructive only when necessary.

Reindex Modes​

ModeTool callDowntimeWhen to use
Incrementalindex_codebase (no flags)NoneDefault after code changes. index_codebase auto-detects an existing collection and processes only added/modified/deleted files via snapshot diff — no separate tool needed.
Seeded first indexindex_codebase on a new git worktree (no flags)NoneAutomatic when another working tree of the repository is indexed with the same stamps: its index is cloned, only differing files are embedded, git signals are rebuilt in the background. Opt out with seedFromWorktree: false / --no-worktree-seed. See Worktree Indexes.
Recompute enrichmenttea-rags index-codebase --force-enrichments <scope>NoneAdditive drift in enrichment-owned payload (git.*, codegraph.*). Rewrites payload in place — no re-embedding, chunk ids unchanged, minutes rather than hours.
Scoped force (in place)tea-rags index-codebase --force --test-file only (or --path-pattern, --languages, --file-extension, --files); MCP forceReindex: true plus testFile / pathPattern / languages / fileExtension / filesNoneA chunk-set change that touched only some files — e.g. a test-chunking revision. Re-chunks and re-embeds just the selected files on the live collection; every other point keeps its id and payload. See Scoped force.
Force (zero-downtime)index_codebase with forceReindex: trueNoneModel change, chunk-set drift (grammar, chunking, chunk size), suspected index corruption. New collection built alongside the old one; alias swaps on success.
Destructiveclear_index then index_codebaseYes — search unavailable during rebuildOnly when force reindex isn't enough (e.g. embedding provider unreachable and you want to start clean). Requires explicit user confirmation.

For the plugin-first workflow, the equivalents are:

  • /tea-rags:index — handles both not-indexed and incremental cases
  • /tea-rags:force-reindex — zero-downtime rebuild (search stays up)

Decision Tree​

Symptom: stale results / missing new files
  → index_codebase  (incremental by default — diffs the snapshot)

Symptom: drift report names a command
  → run the command the report names (it is already the cheapest one)
    additive, enrichment-owned keys → --force-enrichments <scope>
    chunk-set drift in a declared file set → --force --test-file only (etc.)
    chunk-set drift / wrong embedding model → index_codebase forceReindex=true
  (force is zero-downtime; old alias stays live during rebuild)

Symptom: indexing itself is stuck / corrupted marker
  → check get_index_status — is indexingInProgress=true?
    YES → wait OR restart MCP server (marker clears on restart)
    NO  → index_codebase forceReindex=true

Symptom: INGEST_SNAPSHOT_CORRUPTED
  → rm ~/.tea-rags/snapshots/{collection}.json
  → index_codebase forceReindex=true

Symptom: INFRA_QDRANT_UNAVAILABLE persistent
  → fix Qdrant first (see Failure Model)
  → index not touched; retry index_codebase after

Symptom: everything broken, want fresh start
  → clear_index (ASK USER FIRST — irreversible)
  → index_codebase

Scoped force — re-chunk a file set in place​

--force with any file filter does not build a new collection. It runs the incremental pipeline with the selected files forced into its work set: their points are deleted and replaced by freshly chunked and embedded ones, git and codegraph enrichment reruns for those files only, and every other point keeps its id, vector and payload.

# Re-chunk every test file (the same classification as the search testFile filter)
tea-rags index-codebase --project myapp --force --test-file only

# Re-chunk Ruby specs only
tea-rags index-codebase --project myapp --force --languages ruby --path-pattern 'spec/**'

# Re-chunk two named files
tea-rags index-codebase --project myapp --force --files lib/a.rb,lib/b.rb
Flag (CLI)MCP paramSelects
--test-file only|excludetestFiletest files only, or everything but test files
--path-pattern <glob>pathPatternpicomatch over the relative path; a leading ! negates
--languages <list>languagesfiles of these languages
--file-extension <list>fileExtensionfiles with these extensions (.rb or rb)
--files <list>filesexactly these project-relative paths

Filters combine as a conjunction. Only files already in the index are selected; new and deleted files are handled by the same run the way an incremental run handles them. The filters require --force and cannot be combined with --force-enrichments. The project must already be indexed.

The run is crash-safe. The selected files are marked stale in the snapshot before any point is deleted, so if the run dies part-way, the next run (a plain incremental or the auto-update watcher) re-chunks whatever it did not finish.

After a successful scoped run, the language-version stamps advance only for the chunk-set bumps the selection fully covered. A run narrower than the bump — or one that names explicit --files — claims nothing, and the drift report keeps naming the remaining work. See Drift Detection.

Changed in this release

--force --languages <lang> used to build a new collection containing only those languages. It now re-chunks those languages' files in place and leaves everything else untouched.

Zero-Downtime Force Reindex — How It Works​

When you run index_codebase with forceReindex: true:

  1. A new versioned collection is created with the target embedding model + current schema version ({name}_{model}_{schemaV}).
  2. Indexing populates the new collection while the alias still points at the old one — search traffic is unaffected.
  3. On successful completion, the alias atomically switches to the new collection.
  4. The old collection is kept for a grace period (cleanup via alias-cleanup.ts), then deleted.
  5. A fresh snapshot is written for future incremental runs.

If the rebuild fails, the alias stays on the old collection and the new one is cleaned up. Your search never saw a broken index.

Resuming Interrupted Indexing​

Indexing is checkpoint-based. If the process dies mid-run:

  1. Do nothing at first — when you run index_codebase again, ReindexPipeline#checkForCheckpoint (at src/core/domains/ingest/reindexing.ts) detects the partial marker and resumes from the last committed batch.
  2. Chunks already embedded and upserted are not re-processed. Embedding cost = what's left, not the whole codebase.
  3. If resume fails (corrupted marker, older version mismatch), the error code will direct you to forceReindex: true.

The same mechanism handles "I accidentally cancelled" scenarios — the CLI returning doesn't mean indexing stopped. Check get_index_status to confirm background progress.

A run that was killed outright (SIGKILL, OOM) rather than exiting cleanly is detected as soon as the recorded process is confirmed dead, so get_index_status reports stale_indexing immediately instead of waiting out the heartbeat window. The next index_codebase call is admitted right away — it takes over the dead run's lock and reclaims its unfinished, never-promoted build version in the same pass, instead of refusing with INGEST_INDEXING_IN_PROGRESS against a marker whose writer no longer exists.

Schema Drift Recovery​

Payload-key drift is one axis of a broader check that also watches language versions, the indexing environment and the commit the index was built from, and the report it produces already names the single cheapest command that repairs every finding — run that rather than reaching for forceReindex: true. What each axis compares, how to read a report and what each remedy costs: Drift Detection.

Drift never triggers a reindex on its own, so you pick when to pay the cost. Breaking drift is the exception that needs nothing from you: when a new version requires new Qdrant indexes, SchemaManager detects it on startup and the migration runs during the next indexing call, with no data loss.

Snapshot Hygiene​

Snapshots live at ~/.tea-rags/snapshots/ (sharded for large repos). They store file-content hashes used for incremental diff. Safe operations:

  • Delete a snapshot — forces the next run to behave as full index (no incremental diff). Use after INGEST_SNAPSHOT_CORRUPTED.
  • Move to new machine — copy the matching snapshot alongside the Qdrant data to skip reindexing on the new host.

Unsafe operations:

  • Do not edit snapshot JSON manually — the next run will mis-diff and miss real changes.
  • Do not mix snapshots across different embedding models — the dimensions won't match; force reindex first.

Destructive Operations — Explicit Confirmation Required​

Three tools irreversibly delete data. Agents must always ask before calling them:

ToolWhat it kills
clear_indexAll chunks, git enrichment, snapshots for this codebase
delete_collectionA named Qdrant collection and all its points
delete_documentsSpecific points from a collection

The system enforces the protocol via the resource tea-rags://schema/overview, which carries the warning. Agents should:

  1. Surface the exact operation and scope to the user ("This will delete the code_a3f82b91 collection — 82k chunks, irreversible.")
  2. Wait for explicit confirmation.
  3. Only then execute.

Re-indexing a million-LOC codebase takes 10–60 minutes depending on provider. Don't destroy without a reason.

Cross-Session Safety​

When multiple sessions work on the same repo (e.g. two Claude Code windows), the indexing marker prevents races:

  • First session starts indexing → writes marker.
  • Second session calls index_codebase → reads marker, returns early saying another indexer is running.
  • First session finishes → marker cleared → second session's next call proceeds as incremental.

Never clear_index to "break through" a conflict. That's the scenario the marker exists to prevent. Instead, get_index_status to see who's indexing, wait, or restart the MCP server if you believe the marker is stale (e.g. the first session crashed).

Cache & Storage Layout​

All TeaRAGs state lives under ~/.tea-rags/:

~/.tea-rags/
├── qdrant/                    # Embedded Qdrant data (if used)
│   ├── storage/               # Vector data
│   ├── daemon.pid             # Current daemon PID
│   ├── daemon.port            # HTTP port
│   └── daemon.refs            # Active reference count
├── snapshots/                 # File-hash snapshots per collection
│   └── {collection}.json
├── git-cache/                 # L2 disk cache for git enrichment (HEAD-keyed)
├── onnx-calibration.json      # GPU batch-size calibration for ONNX
└── logs/                      # DEBUG=1 logs

When in doubt, deleting ~/.tea-rags/ and re-running index_codebase is always safe — you'll lose the index and start from scratch, but nothing in this directory is irrecoverable.