Vector search at scale: HNSW without the maths degree
Recall, latency and memory in plain language: what the graph parameters actually trade away, and how to pick them for a real corpus.
What the index is doing
Exact nearest-neighbour search compares your query against every vector. Fine at ten thousand rows, painful at ten million. HNSW builds a layered graph where each node links to a few close neighbours, and search walks greedily towards the query.
It is approximate: you give up the guarantee of the true top result in exchange for speed. The whole game is deciding how much recall you are willing to trade.
The three knobs that matter
M is how many neighbours each node keeps. Higher M means a better connected graph, better recall, and more memory.
efConstruction is how hard the builder works while inserting. Higher values make a better graph and a slower build. This is a one-time cost, so be generous here.
efSearch is how hard the search works at query time. This is your runtime dial: raise it when recall is short, lower it when latency is long.
The usual mistake is tuning efSearch when the graph itself was built too cheaply. If recall is stuck no matter how high you push it, rebuild with a higher M instead.
Memory is the real budget
Vectors are dense and the graph adds overhead on top, so index size is roughly your raw vector footprint plus twenty to fifty percent. Plan capacity from that number, not from the row count, and remember that filters change things too.
Filtering is the gotcha
Post-filtering (search, then filter) silently gives you fewer results than you asked for when the filter is selective. Pre-filtering can destroy the graph's navigability if it is implemented as a brute-force scan. Most engines now handle this better than they did, but it is worth testing with your own filters rather than trusting the default.
How I pick settings
Build with generous M and efConstruction. Tune efSearch against a labelled query set until recall hits the bar the product needs, then stop and measure p95 latency. Keep the query set as a regression test, because an index that loses recall after a reindex is a bug that nobody notices until answers get worse.
