Vector Index (ANN)
Also known as: approximate nearest neighbour, ANN index, HNSW
A vector index accelerates nearest-neighbour search over high-dimensional embeddings by trading exactness for speed. Structures such as HNSW and IVF return approximate results in logarithmic rather than linear time, with a recall/latency trade-off controlled by build and query parameters.
Last reviewed · Part of the Architecture Glossary
In practice
Exact search over a million 1,536-dimension vectors is a million dot products per query — hundreds of milliseconds. ANN indexes cut that to single-digit milliseconds by not looking everywhere.
| Index | Build | Query | Memory | Notes |
|---|---|---|---|---|
| Flat (exact) | none | O(n) | vectors only | Correct baseline; fine below ~50k vectors |
| IVF | fast | fast | low | Clusters; recall depends on nprobe |
| HNSW | slow | very fast | high (graph) | The usual production default |
| Product quantisation | medium | fast | very low | Compresses vectors; recall cost |
The parameters that matter for HNSW: M (graph degree, memory and recall), ef_construction (build quality), ef_search (query-time recall/latency dial — the one to tune per endpoint).
Filtering is the hard part. "Nearest neighbours where tenant_id = X" either pre-filters (correct, slow) or post-filters (fast, and may return nothing when the filter is selective). Check which one your store does before designing a multi-tenant retrieval path around it.
When it matters
RAG retrieval, semantic search, deduplication, recommendation.
Common mistake
Reporting latency without recall. An ANN index is only meaningful as a pair — 5 ms at 70% recall is a worse system than 12 ms at 95%, and only one of those numbers usually makes it into the benchmark slide.
See also
- EmbeddingAn embedding is a dense vector representation of text, an image or other content, positioned so that semantically similar items sit close together under a distance measure such as cosine similarity.
- RAG (Retrieval-Augmented Generation)RAG is the pattern of retrieving relevant documents at query time and placing them in the model's context so the answer is grounded in your data rather than in the model's parameters.
- OLAP (Online Analytical Processing)OLAP describes workloads made of a small number of large queries that scan and aggregate many rows over few columns — revenue by region by month.