searxng-mcp-go/docs/OUTPUT_FORMAT.md
XLion 1afa9612c3
docs: align documentation with code behavior (issues #547-#551) (#554)
* docs: align documentation with code behavior (issues #547-#551)

- INSTALL.md: drop SEARXNG_MAX_RETRIES=-1 from the unparsable-value
  example list; cross-reference the hard-error semantic path
- PROMPT_INJECTION_SAFETY.md: mark Option A (warning header + warning
  field) as implemented, referencing #448 and #50
- MCP_TESTING.md: E2E binary is built once into a package-level cache,
  not per test; e2eMCPEnv lives in e2e_helper_test.go; goroutine-leak
  guard is suite-wide, not stress-file-specific
- e2e-tests.md: fix workflow line citations (:87, :104)
- verification.md: drop misleading '(except E2E)' qualifier; narrow the
  E2E-workflow coverage claim for exit-code tests
- test-server.md: add missing test-server-pid-helper recipe row
- .env.example: document SEARXNG_ALLOW_GET_FALLBACK opt-in
- MCP_TOOLS.md: include 'failed to execute search request' segment in
  the network-failure error format
- OUTPUT_FORMAT.md: sync Example 1 count line with the (showing N) format
- adr/014: suppression was relocated to searchWithRetries, not removed
- CONTEXT.md: note empty-text answers are dropped before the engine gate

* docs: address review findings on doc accuracy (P2 items)

- INSTALL.md: SEARXNG_MAX_RETRIES format is an integer in the range
  0-20, not 'non-negative integer' — -1 parses and is rejected by
  SetMaxRetries, so the old wording conflated syntax and semantics
- MCP_TESTING.md: example and E2E_MCP_BINARY section no longer claim a
  per-test build; list e2e_mockserver_test.go among the ungated CLI
  tests that build per-test via buildTestBinary
- CONTEXT.md: dedup description notes the early return when no
  non-empty infobox text exists (original answers returned unchanged)

* docs: fix remaining dedup and E2E_MCP_BINARY wording (P2)

- CONTEXT.md: example dialogue no longer claims non-DuckDuckGo
  answers are never touched; empty answers are dropped before the
  engine check when dedup runs, matching the Search Behavior entry
- test-server.md, verification.md: E2E_MCP_BINARY skips the MCP E2E
  package-level build, not the ungated CLI per-test builds

* docs: scope dedup pass-through claim to non-empty answers

SEARXNG_ANSWER_DEDUP.md and the DeduplicateAnswers doc comment both
claimed non-DuckDuckGo answers always pass through. In fact, when dedup
runs (non-empty infobox content exists), empty-text answers are dropped
before the engine gate; only non-empty non-DuckDuckGo answers pass
through. Aligns both with the implementation in deduplicate.go.

* docs: fix dedup algorithm order in SEARXNG_ANSWER_DEDUP.md

Document the empty-text filter running before the engine gate (matching
deduplicate.go) and add the early-return precondition; sync the Edge
Cases table so retain behavior is scoped to non-empty answers.

* docs: qualify empty-answer drop with early-return precondition

SEARXNG_ANSWER_DEDUP.md edge-case row and the deduplicateAnswers
wrapper comment both stated empty-text answers are always filtered.
When no non-empty infobox exists the implementation early-returns the
original answers unchanged; scope the drop to dedup runs only.
2026-08-11 05:20:47 +08:00

16 KiB
Raw Permalink Blame History

Output Format

The SearXNG MCP Server provides two output modes: CLI text mode (default) and JSON mode (--json).

# CLI text mode (default)
./searxng-mcp-go "query"

# JSON mode
./searxng-mcp-go "query" --json

Example 1: Full Output (All Fields Populated)

Query: apple inc — this query triggers Infoboxes, Results, and Suggestions simultaneously. Answers from DuckDuckGo are deduplicated against infobox content and do not appear.

CLI Text Mode

$ ./searxng-mcp-go "apple inc"

=== Web Search Results ===
Warning: Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.

=== Infoboxes ===

[1] Apple Inc.
    Apple Inc. is an American multinational technology company headquartered in Cupertino, California, in Silicon Valley, best known for its consumer electronics, software and online services. ...
    Attributes:
      - Formerly called: Apple Computer Company (1976–1977), Apple Computer, Inc. (1977–2007)
      - Type: Public
      - Traded as: NASDAQ: AAPL, Nasdaq-100 component, DJIA component, S&P 100 component, S&P 500 component
      - Industry: Consumer electronics, Software services, Online services
      - Founded: April 01, 1976, in Los Altos, California, US
      - Founders: Steve Jobs, Steve Wozniak, Ronald Wayne
      - Key people: Arthur Levinson (chairman), Tim Cook (CEO)
      - Products: AirPods, AirTag, Apple TV, Apple Vision Pro, Apple Watch, HomePod, iPad, iPhone, Mac
      ...
    URLs:
      - Wikipedia: https://en.wikipedia.org/wiki/Apple_Inc.
      - Official site: https://www.apple.com/
      ...

=== Results ===

Found 16 total (showing 10) results for 'apple inc':

1. Apple Inc.
   URL: https://www.apple.com/
   Summary: Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.
   Engine: ddg definitions

2. Apple Inc. (AAPL) Stock Price, News, Quote & History - Yahoo Finance
   URL: https://finance.yahoo.com/quote/AAPL/
   Summary: Find the latest Apple Inc. (AAPL) stock quote, history, news and other vital information to help you with your stock trading and investing.
   Engine: google

...

=== Search Suggestions ===
  - Apple app
  - Retail companies of the United States
  - Apple Store locations
  - Apple Inc stock
  - Steve Jobs
  - ...

JSON Mode

{
  "query": "apple inc",
  "warning": "Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.",
  "number_of_results": 16,
  "infoboxes": [
    {
      "infobox": "Apple Inc.",
      "content": "Apple Inc. is an American multinational technology company headquartered in Cupertino, California...",
      "attributes": [
        {
          "label": "Type",
          "value": "Public"
        }
      ],
      "urls": [
        {
          "title": "Official site",
          "url": "https://www.apple.com/"
        }
      ]
    }
  ],
  "results": [
    {
      "title": "Apple Inc.",
      "url": "https://www.apple.com/",
      "content": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV...",
      "engine": "ddg definitions"
    },
    {
      "title": "Apple Inc. (AAPL) Stock Price, News, Quote & History - Yahoo Finance",
      "url": "https://finance.yahoo.com/quote/AAPL/",
      "content": "Find the latest Apple Inc. (AAPL) stock quote, history, news...",
      "engine": "google"
    }
  ],
  "suggestions": [
    "Apple app",
    "Retail companies of the United States",
    "Apple Store locations",
    "Apple Inc stock"
  ]
}

Example 2: Simple Output (Results and Suggestions Only)

Query: golang tutorial — this query returns no Answers or Infoboxes.

CLI Text Mode

$ ./searxng-mcp-go "golang tutorial"

=== Web Search Results ===
Warning: Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.

=== Results ===

Found 17 results for 'golang tutorial':

1. Tutorials - The Go Programming Language
   URL: https://go.dev/doc/tutorial/
   Summary: Learn Go with tutorials on various topics, such as modules, databases, APIs, generics, fuzzing, and vulnerabilities.
   Engine: google

2. Go Tutorial - GeeksforGeeks
   URL: https://www.geeksforgeeks.org/go-language/go/
   Summary: Go (or Golang) is a modern programming language developed by Google, designed for building fast and reliable applications...
   Engine: google

3. Go by Example
   URL: https://gobyexample.com/
   Summary: Learn Go by doing with annotated example programs. Go by Example covers topics such as variables, functions, channels...
   Engine: google

...

=== Search Suggestions ===
  - Best Golang tutorial
  - Golang tutorial interactive
  - Golang tutorial w3schools
  - Golang tutorial youtube
  - Golang tutorial for beginners
  - Golang tutorial free

JSON Mode

{
  "query": "golang tutorial",
  "warning": "Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.",
  "number_of_results": 17,
  "results": [
    {
      "title": "Tutorials - The Go Programming Language",
      "url": "https://go.dev/doc/tutorial/",
      "content": "Learn Go with tutorials on various topics, such as modules, databases, APIs, generics, fuzzing, and vulnerabilities.",
      "engine": "google"
    },
    {
      "title": "Go Tutorial - GeeksforGeeks",
      "url": "https://www.geeksforgeeks.org/go-language/go/",
      "content": "Go (or Golang) is a modern programming language developed by Google...",
      "engine": "google"
    }
  ],
  "suggestions": [
    "Best Golang tutorial",
    "Golang tutorial interactive",
    "Golang tutorial w3schools",
    "Golang tutorial for beginners",
    "Golang tutorial free"
  ]
}

Note: answers and infoboxes are omitted from this output because they are empty (uses omitempty). In contrast, results and suggestions always appear — they are forced to [] (empty array) rather than omitted.


Empty Field Handling

When a field in the query results has no value, the behavior is as follows:

Mode Behavior
CLI text mode The entire section is omitted — nothing is printed. For example, if there are no Answers, the === Answers === heading will not appear.
JSON mode warning, results, and suggestions are always present for successful searches. answers and infoboxes use omitempty and are omitted when empty. results and suggestions are forced to [] (empty array) when empty, never omitted or null. unresponsive_engines is omitted unless debug mode is enabled.

corrections is intentionally excluded from all output modes per ADR-005.

Specific Rules

CLI text mode:

  • successful output includes the === Web Search Results === warning header
  • answers is empty → omit the entire === Answers === section
  • infoboxes is empty → omit the entire === Infoboxes === section
  • infobox.attributes is empty → omit the Attributes: subsection for that infobox
  • infobox.urls is empty → omit the URLs: subsection for that infobox
  • results is empty → omit the entire === Results === section
  • When the server-reported numberOfResults exceeds the actual rendered count (e.g., due to truncation or filtering), the result count line shows Found N total (showing M) results for '...' instead of Found N results for '...'
  • result.content is empty → omit the Summary: line for that result
  • result.publishedDate is empty → omit the Published date: line for that result
  • suggestions is empty → omit the entire === Search Suggestions === section

JSON mode:

  • warning is present for successful search responses
  • answers is empty → no answers key in the JSON (omitempty)
  • infoboxes is empty → no infoboxes key in the JSON (omitempty)
  • results is empty → "results": [] (always present, forced to empty array)
  • suggestions is empty → "suggestions": [] (always present, forced to empty array)
  • unresponsive_engines is empty and debug mode is on → "unresponsive_engines": [] (always present in debug JSON)
  • debug mode is off → no unresponsive_engines key in the JSON
  • result.publishedDate is empty → no publishedDate key in that result object
  • infobox.attributes is empty → no attributes key in that infobox object
  • infobox.urls is empty → no urls key in that infobox object
  • answer.template is empty → no template key in that answer object

Field Ordering

JSON Mode Field Order

  1. query — Search query echoed in the response
  2. warning — External-content warning for AI agents and JSON consumers
  3. answers — Direct answers (e.g., IP, hash, timezone, etc.)
  4. number_of_results — Total result count reported in JSON output
  5. infoboxes — Knowledge panels
  6. results — Search result list
  7. suggestions — Related search suggestions
  8. unresponsive_engines — Debug-only array of [engine, error] pairs (omitted unless debug mode is enabled; see ADR-006)

answers, infoboxes, and unresponsive_engines may be omitted when empty. results and suggestions remain present as arrays. unresponsive_engines is additionally gated on debug mode.

Output Order Diagram

query
  ↓
warning
  ↓
answers
  ↓
number_of_results
  ↓
infoboxes
  ↓
results
  ↓
suggestions
{
  "query": "string",
  "warning": "string",
  "answers": [],
  "number_of_results": "int",
  "infoboxes": [],
  "results": [],
  "suggestions": [],
  "unresponsive_engines": []  // debug mode only
}

Note: JSON does not guarantee field order, but Go's encoding/json.Marshal serializes struct fields in declaration order. The order above is enforced by the Go struct definition and its MarshalJSON override. warning, results, and suggestions are always present for successful searches; answers and infoboxes use omitempty (omitted when empty). corrections is intentionally excluded per ADR-005. unresponsive_engines is debug-only and omitted when debug mode is off.


Field Trigger Conditions

Whether each field is populated depends on the results returned by the SearXNG backend engines:

Field Trigger Condition
answers SearXNG's answerer module is triggered (e.g., DuckDuckGo Instant Answer, calculator, IP lookup, hash lookup, timezone conversion, etc.). Entity queries (e.g., company names, celebrity names) tend to trigger this.
infoboxes The query targets a well-known entity (person, company, location, concept) and SearXNG engines (e.g., Wikipedia, Wikidata) have corresponding knowledge panel data.
results Nearly all queries return results. The count depends on the number of SearXNG engines configured and their responses.
suggestions SearXNG engines return related search suggestions. Most queries will have suggestions, but not 100%.

Result Sub-field Trigger Conditions

Sub-field Trigger Condition
title Always populated (required field)
url Always populated (required field)
content The engine returned a summary/description; some results may lack this field
engine Always populated; indicates which search engine produced this result
publishedDate SearXNG provided a publication date for the result

CLI Text Content Truncation: In CLI text mode only, formatResults truncates result summaries (content field) and infobox content to 4000 Unicode characters (runes) to keep terminal output manageable for LLM context windows. JSON mode and MCP mode return the full normalized response without this formatting truncation.


CLI Terminal Control Sanitization

CLI text mode neutralizes terminal control sequences in upstream search content before writing to stdout. This defends against a malicious or compromised SearXNG instance that returns ANSI / OSC / DCS bytes in result titles, summaries, suggestions, query echo, infobox titles, attributes, or URLs.

formatResults runs every user-controlled text field through sanitizeTerminalControl (in format.go), which rewrites the following byte and codepoint classes:

Class Bytes / codepoints Treatment
C0 controls 0x00–0x1F except \t, \n replaced with \xNN
DEL 0x7F replaced with \x7f
C1 controls U+0080–U+009F replaced with \xNN
Invalid UTF-8 bytes any byte that fails utf8.DecodeRuneInString replaced with \xNN
Whitespace \t, \n preserved (used for layout)
All other codepoints printable ASCII, Latin diacritics, CJK, emoji preserved unchanged

The sanitizer visibly encodes control bytes rather than silently stripping them, so a downstream operator reviewing a captured transcript can still see the original payload (e.g. \x1b[31mred) instead of a mystery gap. Unicode is preserved: CJK characters, emoji, and accented Latin all round-trip byte-for-byte.

JSON mode and MCP mode are not affected by this text-mode sanitizer: they continue to return the full normalized response. Go's encoding/json escapes JSON control characters such as C0 controls, but does not rewrite every terminal-significant Unicode codepoint such as C1 controls; consumers should treat returned strings as untrusted data, not terminal-ready text. Tests covering ANSI, OSC 52 clipboard payloads, HTML-entity-encoded ESC, C1 controls, carriage returns, and Unicode preservation live in format_sanitize_test.go and format_test.go.


Limits

The server applies a small set of size limits to keep responses manageable and to protect consumers from a misbehaving SearXNG upstream.

Limit Value Applies to Behavior on overflow
Response body size 2 MB (MaxResponseBodySize) All modes (CLI, JSON, MCP) The response is rejected with a SearXNGError whose message contains "response body exceeded maximum size limit". This is a guardrail against a malformed or misconfigured upstream; a normal SearXNG JSON response is well under 2 MB.
Per-result content 4000 Unicode characters (MaxContentRunes) CLI text mode only formatResults truncates result summaries (content field) and infobox content to 4000 runes. JSON mode and MCP mode return the full normalized response without this formatting truncation.
Answers count 100 (MaxAnswers) All modes (CLI, JSON, MCP) normalizeResponse trims Answers to the first 100 entries before answer/infobox deduplication and logs a warning. Bounds the O(n*m) dedup work even when a compact JSON body fits many entries below the body cap. See docs/SEARXNG_ANSWER_DEDUP.md for details.
Infoboxes count 100 (MaxInfoboxes) All modes (CLI, JSON, MCP) normalizeResponse trims Infoboxes to the first 100 entries before answer/infobox deduplication and logs a warning. Same rationale as MaxAnswers.

The response-body-size cap is checked before the JSON is parsed, so the result is a hard error rather than a partial response. MCP consumers receive the SearXNGError as a Search error: ... reply; CLI consumers see it in stderr/exit code. The error's ResponseBody field carries up to 200 bytes of the truncated body for debugging (MaxErrorDisplayBytes); the truncation walks back to a valid UTF-8 rune boundary so multi-byte sequences are never split.


Quick Reference

# CLI text mode (default)
./searxng-mcp-go "apple inc"

# JSON mode
./searxng-mcp-go "apple inc" --json

# With additional parameters
./searxng-mcp-go "apple inc" --json --language en --safesearch 1 --time_range month

# Custom SearXNG server
./searxng-mcp-go "query" --searxng-url=https://your-searxng.example.com

# Debug mode
./searxng-mcp-go "query" --debug