AI engineering · 22 of 42

Vector Index

Close enough, much faster

Scroll

Close enough, much faster

Once your text is a million points, finding the nearest one by comparing all of them is too slow. A vector index builds a navigable structure — usually a layered graph — and walks it instead.

Search drops from checking everything to a handful of hops, which is the difference between a demo on a laptop and a service.

The word people skim past is approximate. The index can miss a true nearest neighbor, and how often it does is a dial you set against speed and memory. If your retrieval quality is mysteriously capped, this parameter is worth looking at before the embedding model.

Retrieval
CLOSE ENOUGH, MUCH FASTER exact search compares every point — correct, and O(n) approximate index L2 L1 L0 hops down a few layers — log(n), and usually right A million vectors is too many to compare one at a time, so the index builds a graph of shortcuts and walks it instead. The word approximate is doing real work: it can miss a true nearest neighbor. Recall is a dial you set against speed and memory, not a given.
An exact scan comparing every point, against a layered graph that hops to the answer in a few steps.