# `Perplexed.Search.SearXNG`
[🔗](https://github.com/agoodway/perplexed/blob/v0.2.0/lib/perplexed/search/searxng.ex#L1)

Keyless web search through SearXNG metasearch instances, as a
`Perplexed.Search` provider (`:searxng`). Free: price 0.

You give it the instances (`searxng_instances:` in the settings, or
`:instances` per call); `Perplexed.Search.SearXNG.Directory` can pick
healthy public ones from searx.space, filtered by country. Public
instances are volunteers' servers: most close the JSON API and run a bot
limiter, so this sends one HTML search at a time and backs off from any
instance that refuses. Prefer your own instances, or ones whose operators
welcome the traffic, for anything beyond experiments.

## The waterfall

`Perplexed.Search.SearXNG.Pool` ranks the instances by what this node has
measured: response-time and failure averages, plus a cooldown after a
refusal. A search asks the best `:merge` instances concurrently; each one
that fails (refused, challenged, timed out, empty) is replaced by the next
in the ranking, until `:merge` have answered or `:attempts` instances
have been tried. Every outcome feeds back into the pool, so the ranking
follows the instances as they speed up, slow down or start refusing.

With `merge: 1` (the default) the hits are the one instance's, re-ranked;
with more, the instances' lists are fused by `Perplexed.Search.Fusion`
(reciprocal rank, engine agreement, query-term coverage).

## Options

  * `:instances` - base URLs, or `:auto` for the live public instances
    `Perplexed.Search.SearXNG.Instances` discovers and keeps fresh; default
    the `searxng_instances` setting.
  * `:merge` - instances to combine, default 1.
  * `:attempts` - instances to try at most, default 4.
  * `:timeout_ms` - per-instance wait, default 6000.
  * `:limit` - hits to return, default 8.
  * `:format` - `:html` (default) or `:json` for instances with the JSON
    API open.
  * `:language` (default `"en-US"`), `:categories` (default `"general"`),
    `:time_range` (`"day"`, `"month"`, `"year"`) and `:safesearch` (0-2).
  * `:proxy` - `"http://user:pass@host:port"`, sent through CloakedReq's
    proxy support, for a residential or rotating proxy.
  * `:impersonate` - the CloakedReq browser profile: a profile atom, a
    list to pick from at random per request, or `:random` (the default),
    which picks from a set of current Chrome, Edge, Firefox and Safari
    desktop profiles. A fresh fingerprint per request spreads us across
    what fingerprint-based limiters see; it does nothing against limits
    on the IP, which is what refused us most in testing.
  * `:min_coverage` - lowest average share of query terms in an
    instance's top 5 hits, default 0.4; below it the
    answer counts as `:irrelevant` and the waterfall moves on. Instances
    pass on results from engines that localize to the server's country
    or answer bots with unrelated pages, and fusion can only reorder a
    list, not rescue one that is all off-topic. 0 turns the gate off.
  * `:fusion` - options for `Perplexed.Search.Fusion.fuse/2`.
  * `:jitter` - random spread on each instance's score, 0 to 1, default
    0.25, so searches share the fastest few instances instead of piling
    onto one.
  * `:min_interval_ms` - an instance used this recently is tried only
    after every rested one, default 15000. Public instances refuse a client
    that searches them again too soon (SearXNG's limiter redirects it to
    the start page), so spreading the load is also what keeps them
    answering.
  * `:pool` - the `Pool` to rank with, default the application's.

Every instance URL passes `Perplexed.UrlGuard` before each search (DNS can
change after an instance is listed), so an instance on a private network
is refused as `:unsafe_url`; pass `url_validator:` to allow a self-hosted
one there.

Each attempt emits `[:perplexed, :searxng, :attempt]` with `duration`
(ms) and `hits`, and metadata `instance`, `outcome` and `retry_after_ms`
(nil unless the instance sent a retry header).

# `collect`

```elixir
@spec collect(String.t(), keyword()) ::
  {:ok, {[{String.t(), [Perplexed.Search.Hit.t()]}], [{String.t(), atom()}]}}
  | {:error, Perplexed.error()}
```

Runs the waterfall without fusing: `{:ok, {answers, failures}}`, where
`answers` are `{instance, hits}` in the order the instances answered and
`failures` are `{instance, reason}`. For custom fusion or experiments;
`search/2` is `collect/2` followed by `Perplexed.Search.Fusion.fuse/2`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*