Math: BM25
Embed text as a BM25 sparse vector (indices + values) suitable for hybrid search in vector databases like Qdrant. Pair this with a dense embedding upstream to power a hybrid (dense + sparse) retrieval pipeline.
{
"indices": [12, 84, 132, ...],
"values": [0.42, 0.31, 0.18, ...]
}
Parameters
Average document length used by the BM25 length-normalisation term. Tune
to the typical token count of your corpus. Defaults to 256.
Length-normalisation parameter — 0.0 disables length normalisation,
1.0 applies it fully. Defaults to 0.0.
Term-frequency saturation parameter. Higher values let very frequent terms
dominate; typical range 1.2–2.0. Defaults to 1.2.
Language used for tokenisation and stop-word handling, chosen from a
dropdown of full language names (english, french, german, spanish,
italian, portuguese, dutch, swedish, norwegian, danish,
russian, arabic, greek, hungarian, romanian, tamil, turkish)
plus detect, which infers the language from the input text. The editor
preselects english for new nodes (the engine itself falls back to
detect only when no language is stored). Any unrecognised value silently
falls back to english.
Input
Output
JSON object holding the sparse vector (indices + values arrays).
Entries are sorted by index, and the values of duplicate indices are
summed. Empty input text yields empty indices / values arrays.