Vector databases: search by meaning, not just keywords

You type “refund for wrong size” into a help-center search. A keyword engine looks for those words. A semantic system can also surface articles titled “exchanges when the fit is off,” even if the word refund never appears. Under the hood, both the query and the documents became lists of numbers (vectors). A vector database is specialized storage and indexing for those lists so you can find nearest neighbors fast.

This post is a plain-English briefing for analysts, PMs, and data engineers who keep hearing “just put it in a vector DB” in AI meetings. You will learn what vectors are doing, why databases for them exist, common use cases, how they differ from keyword search, and a practical “should we bother?” checklist. For LLM vocabulary, see Key Terms and our post on checking AI-written SQL for a related “trust but verify” mindset.

What you’ll learn

What an embedding vector is in everyday terms
Why vector search is different from SQL filters and keyword search
Popular use cases (RAG, recommendations, moderation, multimodal)
How product categories differ: libraries vs managed vector DBs
A decision checklist before you add another system to the stack

Vectors in one coffee chat

An embedding model turns text (or image, audio) into a point in a high-dimensional space. Similar meanings land near each other. “King – man + woman ≈ queen” was the old word2vec party trick; modern models do this for whole sentences and documents with more nuance.

Once everything is a point, “find similar” becomes “find nearest points.” That is vector search. Doing it with a naive scan over millions of points is slow, so vector systems use approximate nearest neighbor (ANN) indexes to trade a bit of recall for a lot of speed.

Toy 2D sketch. Real embeddings have hundreds or thousands of dimensions.

What a vector database adds

You can store vectors in many places. People use FAISS as a library, pgvector inside Postgres, or purpose-built databases. A vector database product typically combines:

Persistent storage for vectors + metadata (tenant id, URL, timestamp)
ANN indexes (HNSW, IVF, etc.) tuned for recall/latency
Filtering (“only docs for tenant 18 and product = shoes”)
CRUD APIs, backups, sharding, and sometimes hybrid keyword + vector search

If your corpus is small and static, an in-memory library might be enough. If many tenants write continuously and you need filters, ops tooling, and SLAs, a managed vector database starts to make sense.

Why the hype now

Large language models are great at language and uneven at facts. Retrieval-augmented generation (RAG) fixes a chunk of that by fetching relevant passages and stuffing them into the prompt. Fetching those passages at scale is a vector search problem. Recommendation, fraud similarity, and multimodal search also rode the same wave.

Hype side effect: teams embed everything, store it twice, and never measure whether retrieval quality beats a well-tuned keyword baseline. Always keep a baseline.

Use cases that are real

Use case	What you embed	What you retrieve
Help center / RAG	Article chunks	Top passages for a user question
Semantic support routing	Ticket text	Similar past tickets or macros
Recommendations	Item or user history text	Nearby items
Moderation	User content	Near-duplicate known bad examples
Image search	Image embeddings	Visually similar images

Keyword vs vector vs hybrid

Style	Strength	Weakness
Keyword (BM25 etc.)	Exact tokens, SKUs, error codes	Misses paraphrases
Vector	Paraphrase and fuzzy intent	Can miss exact IDs; needs good chunks
Hybrid	Best of both when tuned	More moving parts

Practical rule: if users search with order ids and SKUs, keep keyword. If they search with messy natural language, add vectors. Many production systems hybridize.

Chunking and metadata (where projects actually fail)

Garbage chunks in, garbage answers out. A 50-page PDF dumped as one vector is almost useless. Common practice: split into 200-800 token chunks with overlap, store source URL and section title as metadata, and filter by product or permission before ranking.

Also log retrieval: what was fetched, what the user asked, whether the answer was accepted. Without evaluation sets, you are tuning by vibes.

Platform landscape (examples, not ads)

You will hear names like Pinecone, Weaviate, Milvus, Qdrant, and libraries like FAISS or Annoy. Postgres with pgvector is a solid “start simple” path for many teams. Redis and other data stores also grew vector features. Pick based on where your data already lives, ops skill, filter needs, and cost at your QPS, not based on Twitter threads from 2023.

Should you add a vector database?

Do users ask natural-language questions over a corpus larger than a keyword search handles well?
Do you have a permission model that must filter before retrieval?
Can you build a 50-100 question evaluation set with expected docs?
Have you tried a strong keyword baseline on that set?
Who owns chunking quality and re-embed jobs when content changes?
What is the cost of being wrong (wrong medical advice vs wrong hoodie size FAQ)?

If you cannot answer evaluation and ownership, pause. A vector DB will not invent process.

Minimal architecture sketch

documents -> clean/split chunks -> embedding model -> vector index + metadata
user query -> embed query -> ANN search (+ filters) -> top chunks -> LLM or UI

Re-embed when content changes. Version your embedding model name next to the index so you never mix incompatible spaces.

Common mistakes

Embedding entire sites without access control filters
No hybrid search when users type SKUs and error codes
Chunk sizes that split tables and code mid-thought
Never measuring recall@k on a fixed question set
Treating the vector DB as a general system of record for business facts (it is not your warehouse)

Quick recap

Vectors encode meaning as coordinates; vector DBs find neighbors fast
RAG and semantic search are the main drivers, not magic AI dust
Chunking, metadata filters, and evaluation matter more than brand names
Compare against keyword baselines before migrating the stack
Own re-embedding and permissions or the system will rot quietly

For analytics folks, a helpful bridge is to treat retrieval quality like any other model metric. Build a small offline set. Measure recall of the right document in the top 5. Track it when you change chunk size or embedding models. If you cannot measure, you cannot tell whether the new vector database helped or only added cost.

Security note: embeddings can still leak information in clever attacks, and metadata often contains sensitive fields. Apply the same access control you would to the source documents. A public vector index of internal wikis is an incident report waiting for a date.

When stakeholders say “we need a vector database,” translate the request: “we need better retrieval for X corpus for Y users with Z latency.” Sometimes the answer is better documentation structure, not a new database. Sometimes it is both. Your job is to keep the problem statement honest.

Write the messy edge cases in the open. Hidden footnotes become tribal knowledge and then become outages.

If two teams need different definitions, name both clearly instead of forcing a fake compromise that satisfies nobody.

Ship the smallest useful artifact this week: a definition card, a quality check, or a retired vanity chart. Momentum beats manifesto.

Teach newcomers where the source of truth lives. Onboarding is a governance surface whether you designed it or not.

When something fails, prefer a short postmortem over a new committee. Fix the rule or the test that should have caught it.