Chooses which parts of a long page a model gets to read.
A page within the budget passes through unchanged. A longer page keeps
its opening, for context, then the chunks that best match the query terms,
scored like BM25 (term frequency weighted by how rare the term is across
the page's chunks), joined in page order with […] between them. When no
chunk matches, the page is cut to the budget from the top.
Summary
Functions
Up to budget characters of text, favoring passages that match the
terms in queries (the question and any search queries).
Functions
@spec select(String.t(), [String.t()], pos_integer()) :: String.t()
Up to budget characters of text, favoring passages that match the
terms in queries (the question and any search queries).