# `Perplexed.Search.SearXNG.Directory`
[🔗](https://github.com/agoodway/perplexed/blob/v0.2.0/lib/perplexed/search/searxng/directory.ex#L1)

Picks public SearXNG instances from searx.space's daily health check
(`https://searx.space/data/instances.json`).

Keeps instances on the open web (not Tor or I2P) that answered HTTP 200
and passed at least `:min_success` percent of searx.space's test searches,
optionally only those hosted in `:countries` (ISO codes such as `"US"` or
`"DE"`, from the network searx.space resolved each instance's IPs to).
Each entry carries searx.space's median search time, which `seed/1` hands
to `Perplexed.Search.SearXNG.Pool` as a prior until this node has measured
the instance itself. searx.space measures from Germany, so the median is a
starting point, not your latency.

`probe/2` checks from here which instances are up right now, with one
light request to each start page (never a search), and `discover/1`
does the whole job: fetch, filter, probe, keep the live ones and seed the
pool. `Perplexed.Search.SearXNG.Instances` runs `discover/1` on a timer,
and `mix perplexed.searxng.instances` runs it from the shell. A live start
page does not promise the instance will search for you, since SearXNG's
limiter acts on `/search`; the pool's circuit breaker covers that.

# `entry`

```elixir
@type entry() :: %{
  :url => String.t(),
  :countries => [String.t()],
  :median_ms => non_neg_integer() | nil,
  :engines => [String.t()],
  optional(:probe) => probe()
}
```

# `probe`

```elixir
@type probe() :: %{
  status: :alive | :challenge | :blocked | :down,
  latency_ms: non_neg_integer()
}
```

# `alive?`

```elixir
@spec alive?(entry()) :: boolean()
```

Whether `entry` is worth using: probed alive, or not probed at all.

# `discover`

```elixir
@spec discover(keyword()) :: {:ok, [entry()]} | {:error, term()}
```

Fetches the directory, probes what it lists, seeds the pool with the
live instances and returns them, fastest first.

Takes `fetch/1`'s and `probe/2`'s options, plus `probe: false` to skip
probing (every listed instance is kept) and `:pool`.

# `entries`

```elixir
@spec entries(map(), keyword()) :: [entry()]
```

Filters a decoded `instances.json` body; see `fetch/1` for the options.

# `fetch`

```elixir
@spec fetch(keyword()) :: {:ok, [entry()]} | {:error, term()}
```

Fetches and filters the directory.

## Options

  * `:countries` - ISO country codes to keep, default all.
  * `:min_success` - lowest test-search success percentage, default 90.
  * `:max_median_ms` - slowest median search time to keep, default none.
  * `:req_options` - merged into the Req call (tests pass `plug:`).

# `probe`

```elixir
@spec probe([entry()], keyword()) :: [entry()]
```

Checks each entry's start page concurrently and adds its `:probe`:
`:alive` (a SearXNG page), `:challenge` (a bot check in front of it),
`:blocked` (refused or redirected) or `:down`, with the round trip in ms.
Live entries come back fastest first by searx.space median plus that
round trip; the rest follow.

Options: `:timeout_ms` (default 5000), `:max_concurrency`
(default 8), and the request options SearXNG searches
take (`:proxy`, `:impersonate`, `:req_options`).

# `seed`

```elixir
@spec seed([entry()], keyword()) :: :ok
```

Seeds the pool with each entry's prior: searx.space's median search time,
plus the probe's round trip from here when the entry was probed.

# `urls`

```elixir
@spec urls([entry()]) :: [String.t()]
```

The entries' URLs, for the `:instances` option.

---

*Consult [api-reference.md](api-reference.md) for complete listing*