You have a REST API with keyword search, clients depend on it, and the mandate arrives: make search smarter. The temptation is a v2 endpoint and a migration project. There's a quieter path that ships semantic search without breaking a single client.
Step one: dual-write embeddings
Add an embedding step to your existing write path. Every create and update also computes a vector and stores it next to the row (pgvector makes this a column, not a new database). Backfill the corpus with a batch job. Nothing about your API has changed yet; you're just accumulating a shadow index.
Step two: shadow-query and measure
Run vector search in parallel with the existing keyword search on real traffic, log both result sets, and compare offline. This answers the question that matters before any client sees a change: on your data, for your queries, does semantic ranking actually beat what you have? Sometimes it doesn't, and you just saved a quarter.
Step three: blend behind the same contract
The endpoint keeps its shape: same path, same params, same response schema. Inside, retrieval becomes hybrid: take the union of keyword and vector candidates, rescore with reciprocal rank fusion or a small reranker, return the same JSON as before. Clients get better results without knowing why. Relevance improvements are invisible API changes, which is the best kind.
Step four: expose semantics deliberately
Only after the blend is stable do you add opt-in surface area: a mode=semantic param, a similarity threshold, maybe a "more like this" endpoint. These are additive, documented, and versioned. The default behaviour never broke.
The traps
- Embedding drift: changing embedding models invalidates the whole index. Version your vectors and reindex offline.
- Latency: vector search plus rerank must fit the existing latency budget, or the blend needs a timeout that falls back to keywords.
- Filters: your API's WHERE clauses must apply to vector results too. Pre-filter in the vector store, not post-filter in memory, or pagination lies.
- Cost: embed on write, not on read. Reads outnumber writes; don't pay per query.
The whole migration is a sequence of invisible steps followed by optional visible ones. That ordering, not the vector database, is what keeps your API intact.