Reindex a Vector Store After an Embedding Model Change With No Downtime

TL;DR — When you change embedding models, do not overwrite vectors in place. Build a second index keyed by the new model, backfill it from source content, validate with dual reads, then switch your application to the new index through an alias or config flag so rollback is one change. Reading time: ~5 min

Goal

When you finish, your application will serve production traffic from a new vector index built with the new embedding model, with no query downtime, and you will still be able to roll back to the old index immediately if relevance or latency regresses.

Prerequisites

• Production access to your vector store and the application config that selects the active index/collection/namespace
• A canonical source of truth for documents/chunks outside the vector store, such as Postgres, object storage, or your document DB
• A way to write to a second index/collection in the same vector store cluster
• python >= 3.11 — check with python3 --version
• jq >= 1.6 — check with jq --version
• curl >= 8 — check with curl --version
• If your app uses feature flags or env-based deploys, access to that system
• The exact old and new embedding model identifiers, for example text-embedding-3-large replacing text-embedding-3-small
• The current vector dimension for the old model and the required dimension for the new model; verify from your embedding provider docs or one test call before creating the new index
• Enough temporary capacity for two indexes at once: storage for duplicated vectors and write throughput for backfill

Steps

Step 1: Record the current production index and model

Run these commands and save the output in your change ticket or deploy notes.

export APP_ENV=production
export ACTIVE_INDEX=$(printenv VECTOR_INDEX_NAME)
export ACTIVE_MODEL=$(printenv EMBEDDING_MODEL)
printf 'ACTIVE_INDEX=%s\nACTIVE_MODEL=%s\n' "$ACTIVE_INDEX" "$ACTIVE_MODEL"

If your app stores config in Kubernetes:

kubectl -n prod get deploy api -o json | jq -r '.spec.template.spec.containers[0].env[] | select(.name=="VECTOR_INDEX_NAME" or .name=="EMBEDDING_MODEL") | "\(.name)=\(.value)"'

You should see the currently serving index name and the old embedding model identifier.

Step 2: Create a new index for the new model

⚠️ Do not reuse the existing index. If the new model has a different vector dimension, in-place writes will fail. If the dimension matches, mixed-model vectors in one index will silently degrade retrieval quality.

Pick a new index name that includes the model and date.

export NEW_MODEL=text-embedding-3-large
export NEW_DIM=3072
export NEW_INDEX=docs_te3large_2026_08_10
printf 'NEW_INDEX=%s\nNEW_MODEL=%s\nNEW_DIM=%s\n' "$NEW_INDEX" "$NEW_MODEL" "$NEW_DIM"

Create the index using your store’s API or CLI. Generic HTTP shape:

curl -sS -X POST "$VECTOR_API/indexes" \
  -H "Authorization: Bearer $VECTOR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"name\":\"$NEW_INDEX\",\"dimension\":$NEW_DIM,\"metric\":\"cosine\"}" | jq .

Typical success output shape:

{
  "name": "docs_te3large_2026_08_10",
  "dimension": 3072,
  "metric": "cosine",
  "status": "ready"
}

You should see the new index in ready or equivalent active state.

Step 3: Turn on dual-write for new and updated content

Update your ingest path so every new or changed document writes to both the old and new indexes. Use the old index for reads until cutover.

{
  "embedding": {
    "read_index": "docs_te3small_prod",
    "write_indexes": ["docs_te3small_prod", "docs_te3large_2026_08_10"],
    "models": {
      "docs_te3small_prod": "text-embedding-3-small",
      "docs_te3large_2026_08_10": "text-embedding-3-large"
    }
  }
}

If you deploy by env vars:

kubectl -n prod set env deploy/api \
  VECTOR_READ_INDEX=docs_te3small_prod \
  VECTOR_WRITE_INDEXES=docs_te3small_prod,docs_te3large_2026_08_10 \
  EMBEDDING_MODEL_OLD=text-embedding-3-small \
  EMBEDDING_MODEL_NEW=text-embedding-3-large
kubectl -n prod rollout status deploy/api

You should see the deploy complete and new writes landing in both indexes.

Step 4: Backfill the new index from canonical content

Run a backfill from source documents, not from old vectors. Re-embedding old vectors into a new index is wrong; you must regenerate embeddings from text.

python3 backfill_embeddings.py \
  --source postgres \
  --dsn "$PG_DSN" \
  --select "SELECT id, chunk_text, updated_at FROM document_chunks ORDER BY id" \
  --target-index "$NEW_INDEX" \
  --model "$NEW_MODEL" \
  --batch-size 128 \
  --concurrency 8 \
  --id-field id \
  --text-field chunk_text \
  --resume-file .backfill-${NEW_INDEX}.json

Typical progress output shape:

[2026-08-10T12:00:01Z] scanned=128 embedded=128 upserted=128 failed=0 rate=42.1 docs/s
[2026-08-10T12:00:04Z] scanned=1024 embedded=1024 upserted=1024 failed=0 rate=51.7 docs/s
[2026-08-10T12:15:44Z] scanned=250000 embedded=249984 upserted=249984 failed=16 rate=278.4 docs/s

You should see the backfill complete with failed=0 or only retryable failures that are re-run to zero.

Step 5: Check counts and dimension before any read traffic hits the new index

Compare source row count to vector count and run one known query against both indexes.

psql "$PG_DSN" -Atc "SELECT count(*) FROM document_chunks;"
curl -sS "$VECTOR_API/indexes/$NEW_INDEX/stats" -H "Authorization: Bearer $VECTOR_API_TOKEN" | jq .

Expected stats output shape:

{
  "name": "docs_te3large_2026_08_10",
  "vector_count": 250000,
  "dimension": 3072,
  "status": "ready"
}

Smoke-test one query against old and new.

curl -sS -X POST "$APP_URL/internal/search-debug" \
  -H "Authorization: Bearer $INTERNAL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"reset a user password", "top_k":5, "indexes":["docs_te3small_prod","docs_te3large_2026_08_10"]}' | jq .

You should see both result sets returned, with no 4xx/5xx errors.

Step 6: Enable dual-read comparison on a small slice of traffic

Send a small percentage of requests to query both indexes, but continue serving results from the old one. Log overlap, latency, and empty-result rate.

{
  "vector_search": {
    "primary_read_index": "docs_te3small_prod",
    "shadow_read_index": "docs_te3large_2026_08_10",
    "shadow_read_percent": 5,
    "log_topk_overlap": true,
    "log_empty_results": true,
    "log_p95_latency_ms": true
  }
}

If you use env vars:

kubectl -n prod set env deploy/api \
  VECTOR_PRIMARY_READ_INDEX=docs_te3small_prod \
  VECTOR_SHADOW_READ_INDEX=docs_te3large_2026_08_10 \
  VECTOR_SHADOW_READ_PERCENT=5
kubectl -n prod rollout status deploy/api

You should see logs for both indexes and stable application latency.

Step 7: Cut over reads to the new index

Switch the read alias or app config in one change. Prefer an alias if your store supports it; otherwise change one env var and redeploy.

Alias update via generic API shape:

curl -sS -X POST "$VECTOR_API/aliases/docs_prod" \
  -H "Authorization: Bearer $VECTOR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"index\":\"$NEW_INDEX\"}" | jq .

Or app config cutover:

kubectl -n prod set env deploy/api VECTOR_READ_INDEX="$NEW_INDEX"
kubectl -n prod rollout status deploy/api

You should see new queries served from the new index with no increase in error rate.

Step 8: Keep dual-write briefly, then retire the old index

Keep dual-write on for one release window so rollback stays instant. After that, stop writing the old index and delete it only after a final snapshot/export if your store supports one.

⚠️ Deleting the old index removes your fastest rollback path. Do not delete it until relevance, latency, and count checks have been stable for your normal traffic cycle.

kubectl -n prod set env deploy/api VECTOR_WRITE_INDEXES="$NEW_INDEX"
kubectl -n prod rollout status deploy/api
curl -sS -X DELETE "$VECTOR_API/indexes/docs_te3small_prod" -H "Authorization: Bearer $VECTOR_API_TOKEN" -i

Typical delete response shape:

HTTP/1.1 204 No Content

You should see writes only to the new index, and the old index removed only after you are done with rollback risk.

Verify it works

Run these checks end to end.

curl -sS -X POST "$APP_URL/internal/search-debug" \
  -H "Authorization: Bearer $INTERNAL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"reset a user password", "top_k":5}' | jq .active_index

Expected output shape:

"docs_te3large_2026_08_10"

curl -sS "$VECTOR_API/indexes/$NEW_INDEX/stats" -H "Authorization: Bearer $VECTOR_API_TOKEN" | jq '{name,vector_count,dimension,status}'

Expected output shape:

{
  "name": "docs_te3large_2026_08_10",
  "vector_count": 250000,
  "dimension": 3072,
  "status": "ready"
}

kubectl -n prod logs deploy/api --since=10m | grep -E 'shadow_read|active_index|vector_search' | tail -20

Expected result: no dimension mismatch errors, no spike in empty-result logs, and the active index shown as the new index.

Common pitfalls

Writing new-model vectors into the old index

Mistake: reusing the existing index because the name is already wired into production. Symptom: upsert failures like dimension mismatch: got 3072 expected 1536 or silent quality regression if dimensions happen to match. Fix: create a second index with the new model’s dimension and cut over reads later.

Backfilling from old vectors instead of source text

Mistake: copying vector payloads from old index to new index. Symptom: search quality is obviously wrong even though counts match and no errors appear. Fix: regenerate embeddings from the original text chunks, not from stored vectors.

Forgetting dual-write during the backfill window

Mistake: starting a long backfill while production keeps accepting document updates. Symptom: the new index is missing the newest content at cutover even though the backfill completed cleanly. Fix: enable dual-write before backfill starts, then backfill historical rows.

Cutting over before count parity and shadow-read checks

Mistake: switching reads immediately after the backfill job exits 0. Symptom: users see empty or low-quality results for some queries, often due to skipped rows, provider rate-limit retries, or bad chunk filters. Fix: compare source count to index count, run known queries, and shadow-read a small traffic slice first.

Deleting the old index too early

Mistake: removing the old index right after cutover to save storage. Symptom: rollback requires a full rebuild when relevance complaints arrive later. Fix: keep the old index read-ready for at least one normal traffic cycle, then delete it after metrics stay stable.

Ignoring provider rate limits during backfill

Mistake: running too much concurrency against the embedding API. Symptom: logs show bursts of 429 Too Many Requests, throughput collapses, and the job appears stuck. Fix: lower --concurrency, keep --batch-size moderate, and rerun with the same --resume-file so only failed or remaining rows are retried.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.