searxng-mcp-go/CONTEXT.md
XLion 1afa9612c3
docs: align documentation with code behavior (issues #547-#551) (#554)
* docs: align documentation with code behavior (issues #547-#551)

- INSTALL.md: drop SEARXNG_MAX_RETRIES=-1 from the unparsable-value
  example list; cross-reference the hard-error semantic path
- PROMPT_INJECTION_SAFETY.md: mark Option A (warning header + warning
  field) as implemented, referencing #448 and #50
- MCP_TESTING.md: E2E binary is built once into a package-level cache,
  not per test; e2eMCPEnv lives in e2e_helper_test.go; goroutine-leak
  guard is suite-wide, not stress-file-specific
- e2e-tests.md: fix workflow line citations (:87, :104)
- verification.md: drop misleading '(except E2E)' qualifier; narrow the
  E2E-workflow coverage claim for exit-code tests
- test-server.md: add missing test-server-pid-helper recipe row
- .env.example: document SEARXNG_ALLOW_GET_FALLBACK opt-in
- MCP_TOOLS.md: include 'failed to execute search request' segment in
  the network-failure error format
- OUTPUT_FORMAT.md: sync Example 1 count line with the (showing N) format
- adr/014: suppression was relocated to searchWithRetries, not removed
- CONTEXT.md: note empty-text answers are dropped before the engine gate

* docs: address review findings on doc accuracy (P2 items)

- INSTALL.md: SEARXNG_MAX_RETRIES format is an integer in the range
  0-20, not 'non-negative integer' — -1 parses and is rejected by
  SetMaxRetries, so the old wording conflated syntax and semantics
- MCP_TESTING.md: example and E2E_MCP_BINARY section no longer claim a
  per-test build; list e2e_mockserver_test.go among the ungated CLI
  tests that build per-test via buildTestBinary
- CONTEXT.md: dedup description notes the early return when no
  non-empty infobox text exists (original answers returned unchanged)

* docs: fix remaining dedup and E2E_MCP_BINARY wording (P2)

- CONTEXT.md: example dialogue no longer claims non-DuckDuckGo
  answers are never touched; empty answers are dropped before the
  engine check when dedup runs, matching the Search Behavior entry
- test-server.md, verification.md: E2E_MCP_BINARY skips the MCP E2E
  package-level build, not the ungated CLI per-test builds

* docs: scope dedup pass-through claim to non-empty answers

SEARXNG_ANSWER_DEDUP.md and the DeduplicateAnswers doc comment both
claimed non-DuckDuckGo answers always pass through. In fact, when dedup
runs (non-empty infobox content exists), empty-text answers are dropped
before the engine gate; only non-empty non-DuckDuckGo answers pass
through. Aligns both with the implementation in deduplicate.go.

* docs: fix dedup algorithm order in SEARXNG_ANSWER_DEDUP.md

Document the empty-text filter running before the engine gate (matching
deduplicate.go) and add the early-return precondition; sync the Edge
Cases table so retain behavior is scoped to non-empty answers.

* docs: qualify empty-answer drop with early-return precondition

SEARXNG_ANSWER_DEDUP.md edge-case row and the deduplicateAnswers
wrapper comment both stated empty-text answers are always filtered.
When no non-empty infobox exists the implementation early-returns the
original answers unchanged; scope the drop to dedup runs only.
2026-08-11 05:20:47 +08:00

13 KiB
Raw Permalink Blame History

SearXNG MCP Server (searxng-mcp-go)

A Model Context Protocol (MCP) server and CLI tool that proxies web search requests to a SearXNG meta-search engine instance and returns formatted results.

Language

Core Types

SearXNGSearcher: The HTTP client that holds a base URL and an *http.Client and communicates with a SearXNG instance to execute search queries. Its Close() method releases owned idle HTTP connections and cancels in-flight Search() calls; see ADR-012. Avoid: Searcher (ambiguous — appears only in docstrings, not as a separate exported type)

Config: The connection parameters for a SearXNG instance — a base URL, a per-request HTTP timeout duration, retry configuration (MaxRetries, RetryDelay, MaxRetryDelay), an optional custom HTTP client, and the opt-in GET Fallback flag.

SearchArgs: All input parameters for a search operation — the search query, language code, SafeSearch level, time range, categories, engines, page number, and result limit. Avoid: SearchParams

SearchResponse: The complete, normalized response from SearXNG, containing the query string, direct answers, result count, infoboxes, web results, suggestions, and unresponsive-engine diagnostics.

SearchResult: A single web result returned by a search engine — title, URL, content summary, engine name, and optional publication date.

Response Components

Answer: A direct instant answer from SearXNG (e.g., IP address, hash digest, calculator result, timezone), carrying an answer string, engine name, and optional template name. Avoid: Instant Answer (SearXNG internal term; not used in Go types)

Infobox: A structured knowledge panel from SearXNG containing a title, body content, key-value attribute pairs, and related URLs — typically derived from Wikipedia or Wikidata for entity queries (people, companies, locations).

InfoboxAttribute: A single key–value attribute pair within an infobox (e.g., Founded: April 01, 1976).

InfoboxURL: A titled URL entry within an infobox (e.g., Wikipedia: https://en.wikipedia.org/wiki/Apple_Inc.).

Suggestion: An alternative search query suggested by the upstream search engines when the original query could be refined.

UnresponsiveEngines: A list of [engine_name, error_message] pairs for search engines that failed during a query — only exposed in JSON output when Debug Mode is active. Avoid: DeadEngines, FailedEngines

Operation Modes

CLI Mode: An operation mode where the program accepts command-line arguments, executes a one-shot search, and prints formatted text (or JSON with --json) to stdout, then exits.

MCP Mode: An operation mode where the program starts a stdio JSON-RPC server implementing the Model Context Protocol, listening for incoming search tool-call requests and returning JSON responses; uses no CLI args, only environment variables.

Debug Mode: An operational mode activated by --debug or DEBUG=1 that logs HTTP request/response details and raw JSON body previews to stderr and gates exposure of UnresponsiveEngines in JSON output.

CLIFlags: The parsed command-line flag values that include search parameters (query, language, safesearch, etc.) plus meta-flags for output format (--json), help, version, debug, and custom SearXNG URL.

Error Types

ValidationError: An error raised when user-supplied search arguments fail validation rules (e.g., empty query, excessive length, invalid time range, control characters).

SearXNGError: An error originating from the SearXNG service layer, carrying the HTTP status code, response content type, a truncated body preview, and the underlying cause.

HTMLResponseError: A specialized error for when the SearXNG server returns an HTML page instead of the expected JSON — signals that format=json is likely not enabled on the instance.

Search Behavior

Deduplicate: The process of filtering out DuckDuckGo Answer entries whose text overlaps with Infobox content (specifically, DuckDuckGo returning the same Wikipedia summary in both answers and infoboxes), using prefix matching on the first 200 characters with a lowercase fallback. Dedup runs only when at least one non-empty infobox text exists — otherwise the original answers are returned unchanged. When it runs, answers with empty Answer text are dropped before the engine gate; the remaining answers are checked only when Engine == "duckduckgo", and non-DuckDuckGo answers with non-empty text pass through. The implementation lives in internal/searxng/answer. Avoid: Dedup (internal function name; prefer the full term in docs)

setBrowserHeaders: The function that applies Chrome-like HTTP headers (User-Agent, Accept, Sec-* family, Priority) to every search request to bypass SearXNG's limiter / bot-detection mechanism. Avoid: BotHeaders, StealthHeaders

GET Fallback: The opt-in compatibility mechanism that re-issues a failed POST search as a GET request when the SearXNG instance returns HTTP 405 (Method Not Allowed) or 501 (Not Implemented). It is disabled by default and enabled only with SEARXNG_ALLOW_GET_FALLBACK=1 because GET sends search parameters in the URL. Avoid: POSTtoGETFallback (internal test function name)

Private Host Detection: The RFC-grounded classification that suppresses the HTTP warning when the configured SearXNG URL points to a private/internal destination. A host is "private" iff the literal name is localhost or ends in .localhost (RFC 6761 §6.3, Special-Use Domain Names), or the literal address falls inside one of the published private/loopback/link-local/ULA/CGNAT/multicast/broadcast ranges enumerated in docs/adr/003-http-warning-for-non-private-hosts.md. No DNS resolution is performed, and the contract intentionally accepts that names like printer.local or nas.lan will now trigger the warning because no cited RFC reserves those suffixes for "private network" use.

prepareMCPStdin: The function that peeks at the first line of stdin and applies a fixed 1 MiB transport bound to the JSON first-message wire bytes, excluding an optional trailing newline delimiter, plus a safe structural MCP gate. It accepts a legacy JSON-RPC 2.0 initialize request, a server/discover request, or a stateless request carrying params._meta["io.modelcontextprotocol/protocolVersion"]; complete protocol metadata validation remains with the MCP SDK. This prevents the MCP server from hanging when piped non-MCP input.

Constants

MaxContentRunes: The truncation limit of 4000 Unicode runes applied by the CLI text formatter (formatResults in format.go) to the content field of search results and infoboxes. It is purely a rendering budget for terminal output — JSON mode and MCP mode return the full un-truncated normalized response, so downstream JSON/MCP consumers always see the complete upstream text. The rune-safe truncation itself is shared with the searxng deduplication prefix match through the searxng.TruncateRunes helper in internal/searxng/truncate.go. See docs/adr/011-max-content-runes-cli-only.md for the scope decision.

Relationships

  • SearXNGSearcher is configured by Config (URL + timeout + optional HTTP client) and executes a search via SearchArgs, returning a SearchResponse.
  • SearchResponse contains zero or more SearchResults, Answers, Infoboxes, Suggestions, and UnresponsiveEngines entries.
  • Answers are Deduplicated against Infobox content to remove overlapping DuckDuckGo Wikipedia summaries; answers from non-DuckDuckGo engines are not checked.
  • setBrowserHeaders is applied by SearXNGSearcher to every HTTP request made during search execution.
  • SearXNGSearcher falls back to a GET Fallback only when the SearXNG instance rejects the initial POST with 405 or 501 and Config enables AllowGETFallback.
  • SearXNGSearcher.Close cancels active SearchArgs executions through an internal shutdown signal, so shutdown can return in-flight searches with context.Canceled instead of waiting for upstream completion.
  • CLI Mode and MCP Mode are the two mutually exclusive operation modes — the program runs CLI mode when any arguments are present, MCP mode otherwise.
  • Debug Mode gates the exposure of UnresponsiveEngines in JSON output and enables verbose HTTP logging.
  • CLIFlags maps to SearchArgs fields plus mode-selection flags (--json, --help, --version, --debug, --searxng-url).
  • ValidationError is returned by ValidateSearchArgs, which pre-checks all SearchArgs before any HTTP request.
  • SearXNGError wraps HTTP-level failures from the SearXNG service; HTMLResponseError is a specific case returned when HTML is received instead of JSON.
  • Config reads the SearXNG URL from CLIFlags or the SEARXNG_URL environment variable, returning an error (SEARXNG_URL is required) if neither is set. It also reads SEARXNG_ALLOW_GET_FALLBACK=1 as an explicit opt-in for GET Fallback.

Example dialogue

Dev: "Why did the CLI mode search return HTML instead of JSON?"

Domain expert: "That's an HTMLResponseError — the SearXNG instance at that Config URL doesn't have format=json enabled. Check the server's settings. The SearXNGSearcher tries a POST first. If POST is rejected with 405 or 501, fix the reverse proxy or explicitly enable GET Fallback with SEARXNG_ALLOW_GET_FALLBACK=1; an HTML response is not a method issue — it's the SearXNG config."

Dev: "And I see duplicate answers in the output for DuckDuckGo queries. What's that about?"

Domain expert: "The Deduplicate function handles that. DuckDuckGo puts the same Wikipedia snippet in both the Answer and the Infobox. When non-empty infobox content exists, empty answers are dropped first; remaining answers are checked only if they come from the duckduckgo engine — if the answer text is a prefix of any infobox content, it's filtered out. Non-empty non-DuckDuckGo answers pass through."

Dev: "I ran a query with --debug and saw my search query logged in plain text. Is that safe?"

Domain expert: "Debug Mode logs the full HTTP request body, so yes — avoid sensitive queries. Also note that --debug gates the UnresponsiveEngines field in JSON output; without it you won't see which engines failed."

Dev: "The server at 192.168.1.50 uses plain HTTP. There's no warning, but the same instance at search.example.com does warn. And I just noticed that printer.local also warns now — it used to be quiet. Why?"

Domain expert: "That's Private Host Detection. The HTTP warning is suppressed only for RFC-grounded private destinations: literal localhost / *.localhost (RFC 6761) and literal addresses inside the private/loopback/link-local/ULA/CGNAT/multicast/broadcast ranges listed in ADR-003. There is no DNS resolution and no private-TLD allowlist. 192.168.1.50 matches RFC 1918 and is silent; search.example.com is a public name and warns. printer.local is not considered private by this check — RFC 6762 defines .local as a multicast-DNS link-local namespace, not a private-network indicator — so it now triggers the warning too. If you don't want the warning for a LAN-only host, either bind SearXNG to a literal RFC 1918 address or put TLS in front of it."

Flagged ambiguities

  1. "Answer" ambiguity: The term Answer refers both to the Go struct (with Answer, Engine, Template fields) and to the top-level answers array in the SearXNG JSON response. Additionally, SearXNG documentation mentions a legacy {"answer": "..."} result format that is a flat string, which is distinct from the typed struct used in this project.

  2. "Searcher" vs "SearXNGSearcher": The broader term "Searcher" appears in function-level docstrings and developer discussion but is not a separate exported type — only SearXNGSearcher exists in code. Search is the public entry point for executing a search.

  3. NumberOfResults post-processing: SearXNG may return number_of_results = 0 even when results exist. The code silently corrects this by setting it to len(results) when the field is zero but results are non-empty. Consumers cannot rely on this field being an unmodified pass-through from SearXNG.

  4. corrections explicitly excluded: The SearXNG response includes a corrections string array (spelling corrections from search engines), but this field is intentionally absent from SearchResponse per ADR-005 — it is not exposed in any output mode.

  5. UnresponsiveEngines conditional exposure: This field is always captured in the Go struct but is only serialized to JSON when Debug is true. In CLI text mode, it is never printed to stdout — it is only logged via slog.Debug (visible in stderr with debug logging).

  6. Unknown engines/categories silently ignored by SearXNG: When engines or categories contains unknown names, SearXNG silently falls back to defaults (wikipedia for engines, general for categories) and returns results with warnings: [] — no error or warning is emitted. The MCP server has no way to detect this, so consumers should not treat the presence of results as confirmation that a requested engine or category was used.