AI engineering · 07 of 42

BM25

Rank documents by keyword relevance

Scroll

Rank documents by keyword relevance

BM25 scores documents by which of your query's words they contain, how often, and how rare those words are across the whole collection. Rare matching terms count for much more than common ones.

It also discounts long documents, so a page cannot rank highly just by being big enough to contain everything.

It is fast, cheap, needs no training, and you can always explain exactly why something ranked. Its weakness is the obvious one: it matches words, not meaning, so a page describing your exact problem in different vocabulary scores zero. That is precisely the gap the next concept exists to close.

Retrieval
RANK BY WORD OVERLAP, NOT MEANING ERR_CONNECTION _RESET scores 1 ERR_CONNECTION_RESET fix 2 browser connection errors 3 network timeout config ranked documents It rewards rare terms and repeated matches, and discounts long documents so they cannot win on length alone. Fast, cheap and explainable — you can always say why something ranked. It will miss a page that says the same thing differently.
An exact error code ranking the document that contains it first, by word overlap alone.