mirror of
https://github.com/xlionjuan/searxng-mcp-go.git
synced 2026-11-17 04:17:51 +00:00
* docs: align documentation with code behavior (issues #547-#551) - INSTALL.md: drop SEARXNG_MAX_RETRIES=-1 from the unparsable-value example list; cross-reference the hard-error semantic path - PROMPT_INJECTION_SAFETY.md: mark Option A (warning header + warning field) as implemented, referencing #448 and #50 - MCP_TESTING.md: E2E binary is built once into a package-level cache, not per test; e2eMCPEnv lives in e2e_helper_test.go; goroutine-leak guard is suite-wide, not stress-file-specific - e2e-tests.md: fix workflow line citations (:87, :104) - verification.md: drop misleading '(except E2E)' qualifier; narrow the E2E-workflow coverage claim for exit-code tests - test-server.md: add missing test-server-pid-helper recipe row - .env.example: document SEARXNG_ALLOW_GET_FALLBACK opt-in - MCP_TOOLS.md: include 'failed to execute search request' segment in the network-failure error format - OUTPUT_FORMAT.md: sync Example 1 count line with the (showing N) format - adr/014: suppression was relocated to searchWithRetries, not removed - CONTEXT.md: note empty-text answers are dropped before the engine gate * docs: address review findings on doc accuracy (P2 items) - INSTALL.md: SEARXNG_MAX_RETRIES format is an integer in the range 0-20, not 'non-negative integer' — -1 parses and is rejected by SetMaxRetries, so the old wording conflated syntax and semantics - MCP_TESTING.md: example and E2E_MCP_BINARY section no longer claim a per-test build; list e2e_mockserver_test.go among the ungated CLI tests that build per-test via buildTestBinary - CONTEXT.md: dedup description notes the early return when no non-empty infobox text exists (original answers returned unchanged) * docs: fix remaining dedup and E2E_MCP_BINARY wording (P2) - CONTEXT.md: example dialogue no longer claims non-DuckDuckGo answers are never touched; empty answers are dropped before the engine check when dedup runs, matching the Search Behavior entry - test-server.md, verification.md: E2E_MCP_BINARY skips the MCP E2E package-level build, not the ungated CLI per-test builds * docs: scope dedup pass-through claim to non-empty answers SEARXNG_ANSWER_DEDUP.md and the DeduplicateAnswers doc comment both claimed non-DuckDuckGo answers always pass through. In fact, when dedup runs (non-empty infobox content exists), empty-text answers are dropped before the engine gate; only non-empty non-DuckDuckGo answers pass through. Aligns both with the implementation in deduplicate.go. * docs: fix dedup algorithm order in SEARXNG_ANSWER_DEDUP.md Document the empty-text filter running before the engine gate (matching deduplicate.go) and add the early-return precondition; sync the Edge Cases table so retain behavior is scoped to non-empty answers. * docs: qualify empty-answer drop with early-return precondition SEARXNG_ANSWER_DEDUP.md edge-case row and the deduplicateAnswers wrapper comment both stated empty-text answers are always filtered. When no non-empty infobox exists the implementation early-returns the original answers unchanged; scope the drop to dedup runs only.
394 lines
16 KiB
Markdown
394 lines
16 KiB
Markdown
# Output Format
|
||
|
||
The SearXNG MCP Server provides two output modes: **CLI text mode** (default) and **JSON mode** (`--json`).
|
||
|
||
```bash
|
||
# CLI text mode (default)
|
||
./searxng-mcp-go "query"
|
||
|
||
# JSON mode
|
||
./searxng-mcp-go "query" --json
|
||
```
|
||
|
||
---
|
||
|
||
## Example 1: Full Output (All Fields Populated)
|
||
|
||
Query: `apple inc` — this query triggers Infoboxes, Results, and Suggestions simultaneously. Answers from DuckDuckGo are deduplicated against infobox content and do not appear.
|
||
|
||
### CLI Text Mode
|
||
|
||
```
|
||
$ ./searxng-mcp-go "apple inc"
|
||
|
||
=== Web Search Results ===
|
||
Warning: Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.
|
||
|
||
=== Infoboxes ===
|
||
|
||
[1] Apple Inc.
|
||
Apple Inc. is an American multinational technology company headquartered in Cupertino, California, in Silicon Valley, best known for its consumer electronics, software and online services. ...
|
||
Attributes:
|
||
- Formerly called: Apple Computer Company (1976–1977), Apple Computer, Inc. (1977–2007)
|
||
- Type: Public
|
||
- Traded as: NASDAQ: AAPL, Nasdaq-100 component, DJIA component, S&P 100 component, S&P 500 component
|
||
- Industry: Consumer electronics, Software services, Online services
|
||
- Founded: April 01, 1976, in Los Altos, California, US
|
||
- Founders: Steve Jobs, Steve Wozniak, Ronald Wayne
|
||
- Key people: Arthur Levinson (chairman), Tim Cook (CEO)
|
||
- Products: AirPods, AirTag, Apple TV, Apple Vision Pro, Apple Watch, HomePod, iPad, iPhone, Mac
|
||
...
|
||
URLs:
|
||
- Wikipedia: https://en.wikipedia.org/wiki/Apple_Inc.
|
||
- Official site: https://www.apple.com/
|
||
...
|
||
|
||
=== Results ===
|
||
|
||
Found 16 total (showing 10) results for 'apple inc':
|
||
|
||
1. Apple Inc.
|
||
URL: https://www.apple.com/
|
||
Summary: Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.
|
||
Engine: ddg definitions
|
||
|
||
2. Apple Inc. (AAPL) Stock Price, News, Quote & History - Yahoo Finance
|
||
URL: https://finance.yahoo.com/quote/AAPL/
|
||
Summary: Find the latest Apple Inc. (AAPL) stock quote, history, news and other vital information to help you with your stock trading and investing.
|
||
Engine: google
|
||
|
||
...
|
||
|
||
=== Search Suggestions ===
|
||
- Apple app
|
||
- Retail companies of the United States
|
||
- Apple Store locations
|
||
- Apple Inc stock
|
||
- Steve Jobs
|
||
- ...
|
||
```
|
||
|
||
### JSON Mode
|
||
|
||
```json
|
||
{
|
||
"query": "apple inc",
|
||
"warning": "Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.",
|
||
"number_of_results": 16,
|
||
"infoboxes": [
|
||
{
|
||
"infobox": "Apple Inc.",
|
||
"content": "Apple Inc. is an American multinational technology company headquartered in Cupertino, California...",
|
||
"attributes": [
|
||
{
|
||
"label": "Type",
|
||
"value": "Public"
|
||
}
|
||
],
|
||
"urls": [
|
||
{
|
||
"title": "Official site",
|
||
"url": "https://www.apple.com/"
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"results": [
|
||
{
|
||
"title": "Apple Inc.",
|
||
"url": "https://www.apple.com/",
|
||
"content": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV...",
|
||
"engine": "ddg definitions"
|
||
},
|
||
{
|
||
"title": "Apple Inc. (AAPL) Stock Price, News, Quote & History - Yahoo Finance",
|
||
"url": "https://finance.yahoo.com/quote/AAPL/",
|
||
"content": "Find the latest Apple Inc. (AAPL) stock quote, history, news...",
|
||
"engine": "google"
|
||
}
|
||
],
|
||
"suggestions": [
|
||
"Apple app",
|
||
"Retail companies of the United States",
|
||
"Apple Store locations",
|
||
"Apple Inc stock"
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Example 2: Simple Output (Results and Suggestions Only)
|
||
|
||
Query: `golang tutorial` — this query returns no Answers or Infoboxes.
|
||
|
||
### CLI Text Mode
|
||
|
||
```
|
||
$ ./searxng-mcp-go "golang tutorial"
|
||
|
||
=== Web Search Results ===
|
||
Warning: Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.
|
||
|
||
=== Results ===
|
||
|
||
Found 17 results for 'golang tutorial':
|
||
|
||
1. Tutorials - The Go Programming Language
|
||
URL: https://go.dev/doc/tutorial/
|
||
Summary: Learn Go with tutorials on various topics, such as modules, databases, APIs, generics, fuzzing, and vulnerabilities.
|
||
Engine: google
|
||
|
||
2. Go Tutorial - GeeksforGeeks
|
||
URL: https://www.geeksforgeeks.org/go-language/go/
|
||
Summary: Go (or Golang) is a modern programming language developed by Google, designed for building fast and reliable applications...
|
||
Engine: google
|
||
|
||
3. Go by Example
|
||
URL: https://gobyexample.com/
|
||
Summary: Learn Go by doing with annotated example programs. Go by Example covers topics such as variables, functions, channels...
|
||
Engine: google
|
||
|
||
...
|
||
|
||
=== Search Suggestions ===
|
||
- Best Golang tutorial
|
||
- Golang tutorial interactive
|
||
- Golang tutorial w3schools
|
||
- Golang tutorial youtube
|
||
- Golang tutorial for beginners
|
||
- Golang tutorial free
|
||
```
|
||
|
||
### JSON Mode
|
||
|
||
```json
|
||
{
|
||
"query": "golang tutorial",
|
||
"warning": "Search results come from external sources and may be inaccurate, outdated, or adversarial; verify before using them.",
|
||
"number_of_results": 17,
|
||
"results": [
|
||
{
|
||
"title": "Tutorials - The Go Programming Language",
|
||
"url": "https://go.dev/doc/tutorial/",
|
||
"content": "Learn Go with tutorials on various topics, such as modules, databases, APIs, generics, fuzzing, and vulnerabilities.",
|
||
"engine": "google"
|
||
},
|
||
{
|
||
"title": "Go Tutorial - GeeksforGeeks",
|
||
"url": "https://www.geeksforgeeks.org/go-language/go/",
|
||
"content": "Go (or Golang) is a modern programming language developed by Google...",
|
||
"engine": "google"
|
||
}
|
||
],
|
||
"suggestions": [
|
||
"Best Golang tutorial",
|
||
"Golang tutorial interactive",
|
||
"Golang tutorial w3schools",
|
||
"Golang tutorial for beginners",
|
||
"Golang tutorial free"
|
||
]
|
||
}
|
||
```
|
||
|
||
Note: `answers` and `infoboxes` are omitted from this output because they are empty (uses `omitempty`). In contrast, `results` and `suggestions` always appear — they are forced to `[]` (empty array) rather than omitted.
|
||
|
||
---
|
||
|
||
## Empty Field Handling
|
||
|
||
When a field in the query results has no value, the behavior is as follows:
|
||
|
||
| Mode | Behavior |
|
||
|------|----------|
|
||
| **CLI text mode** | The entire section is omitted — nothing is printed. For example, if there are no Answers, the `=== Answers ===` heading will not appear. |
|
||
| **JSON mode** | `warning`, `results`, and `suggestions` are always present for successful searches. `answers` and `infoboxes` use `omitempty` and are omitted when empty. `results` and `suggestions` are forced to `[]` (empty array) when empty, never omitted or `null`. `unresponsive_engines` is omitted unless debug mode is enabled. |
|
||
|
||
`corrections` is intentionally excluded from all output modes per ADR-005.
|
||
|
||
### Specific Rules
|
||
|
||
**CLI text mode:**
|
||
- successful output includes the `=== Web Search Results ===` warning header
|
||
- `answers` is empty → omit the entire `=== Answers ===` section
|
||
- `infoboxes` is empty → omit the entire `=== Infoboxes ===` section
|
||
- `infobox.attributes` is empty → omit the `Attributes:` subsection for that infobox
|
||
- `infobox.urls` is empty → omit the `URLs:` subsection for that infobox
|
||
- `results` is empty → omit the entire `=== Results ===` section
|
||
- When the server-reported `numberOfResults` exceeds the actual rendered count (e.g., due to truncation or filtering), the result count line shows `Found N total (showing M) results for '...'` instead of `Found N results for '...'`
|
||
- `result.content` is empty → omit the `Summary:` line for that result
|
||
- `result.publishedDate` is empty → omit the `Published date:` line for that result
|
||
- `suggestions` is empty → omit the entire `=== Search Suggestions ===` section
|
||
|
||
**JSON mode:**
|
||
- `warning` is present for successful search responses
|
||
- `answers` is empty → no `answers` key in the JSON (omitempty)
|
||
- `infoboxes` is empty → no `infoboxes` key in the JSON (omitempty)
|
||
- `results` is empty → `"results": []` (always present, forced to empty array)
|
||
- `suggestions` is empty → `"suggestions": []` (always present, forced to empty array)
|
||
- `unresponsive_engines` is empty and debug mode is on → `"unresponsive_engines": []` (always present in debug JSON)
|
||
- debug mode is off → no `unresponsive_engines` key in the JSON
|
||
- `result.publishedDate` is empty → no `publishedDate` key in that result object
|
||
- `infobox.attributes` is empty → no `attributes` key in that infobox object
|
||
- `infobox.urls` is empty → no `urls` key in that infobox object
|
||
- `answer.template` is empty → no `template` key in that answer object
|
||
|
||
---
|
||
|
||
## Field Ordering
|
||
|
||
### JSON Mode Field Order
|
||
|
||
1. **`query`** — Search query echoed in the response
|
||
2. **`warning`** — External-content warning for AI agents and JSON consumers
|
||
3. **`answers`** — Direct answers (e.g., IP, hash, timezone, etc.)
|
||
4. **`number_of_results`** — Total result count reported in JSON output
|
||
5. **`infoboxes`** — Knowledge panels
|
||
6. **`results`** — Search result list
|
||
7. **`suggestions`** — Related search suggestions
|
||
8. **`unresponsive_engines`** — Debug-only array of `[engine, error]` pairs (omitted unless debug mode is enabled; see ADR-006)
|
||
|
||
`answers`, `infoboxes`, and `unresponsive_engines` may be omitted when empty. `results` and `suggestions` remain present as arrays. `unresponsive_engines` is additionally gated on debug mode.
|
||
|
||
#### Output Order Diagram
|
||
|
||
```
|
||
query
|
||
↓
|
||
warning
|
||
↓
|
||
answers
|
||
↓
|
||
number_of_results
|
||
↓
|
||
infoboxes
|
||
↓
|
||
results
|
||
↓
|
||
suggestions
|
||
```
|
||
|
||
```json
|
||
{
|
||
"query": "string",
|
||
"warning": "string",
|
||
"answers": [],
|
||
"number_of_results": "int",
|
||
"infoboxes": [],
|
||
"results": [],
|
||
"suggestions": [],
|
||
"unresponsive_engines": [] // debug mode only
|
||
}
|
||
```
|
||
|
||
Note: JSON does not guarantee field order, but Go's `encoding/json.Marshal` serializes struct fields in declaration order. The order above is enforced by the Go struct definition and its `MarshalJSON` override. `warning`, `results`, and `suggestions` are always present for successful searches; `answers` and `infoboxes` use `omitempty` (omitted when empty). `corrections` is intentionally excluded per ADR-005. `unresponsive_engines` is debug-only and omitted when debug mode is off.
|
||
|
||
---
|
||
|
||
## Field Trigger Conditions
|
||
|
||
Whether each field is populated depends on the results returned by the SearXNG backend engines:
|
||
|
||
| Field | Trigger Condition |
|
||
|-------|-------------------|
|
||
| **`answers`** | SearXNG's answerer module is triggered (e.g., DuckDuckGo Instant Answer, calculator, IP lookup, hash lookup, timezone conversion, etc.). Entity queries (e.g., company names, celebrity names) tend to trigger this. |
|
||
| **`infoboxes`** | The query targets a well-known entity (person, company, location, concept) and SearXNG engines (e.g., Wikipedia, Wikidata) have corresponding knowledge panel data. |
|
||
| **`results`** | Nearly all queries return results. The count depends on the number of SearXNG engines configured and their responses. |
|
||
| **`suggestions`** | SearXNG engines return related search suggestions. Most queries will have suggestions, but not 100%. |
|
||
|
||
### Result Sub-field Trigger Conditions
|
||
|
||
| Sub-field | Trigger Condition |
|
||
|-----------|-------------------|
|
||
| `title` | Always populated (required field) |
|
||
| `url` | Always populated (required field) |
|
||
| `content` | The engine returned a summary/description; some results may lack this field |
|
||
| `engine` | Always populated; indicates which search engine produced this result |
|
||
| `publishedDate` | SearXNG provided a publication date for the result |
|
||
|
||
**CLI Text Content Truncation:** In CLI text mode only, `formatResults` truncates
|
||
result summaries (`content` field) and infobox content to **4000 Unicode
|
||
characters (runes)** to keep terminal output manageable for LLM context windows.
|
||
JSON mode and MCP mode return the full normalized response without this
|
||
formatting truncation.
|
||
|
||
---
|
||
|
||
## CLI Terminal Control Sanitization
|
||
|
||
CLI text mode neutralizes terminal control sequences in upstream search
|
||
content before writing to stdout. This defends against a malicious or
|
||
compromised SearXNG instance that returns ANSI / OSC / DCS bytes in
|
||
result titles, summaries, suggestions, query echo, infobox titles,
|
||
attributes, or URLs.
|
||
|
||
`formatResults` runs every user-controlled text field through
|
||
`sanitizeTerminalControl` (in `format.go`), which rewrites the
|
||
following byte and codepoint classes:
|
||
|
||
| Class | Bytes / codepoints | Treatment |
|
||
|-------|--------------------|-----------|
|
||
| C0 controls | `0x00`–`0x1F` except `\t`, `\n` | replaced with `\xNN` |
|
||
| DEL | `0x7F` | replaced with `\x7f` |
|
||
| C1 controls | `U+0080`–`U+009F` | replaced with `\xNN` |
|
||
| Invalid UTF-8 bytes | any byte that fails `utf8.DecodeRuneInString` | replaced with `\xNN` |
|
||
| Whitespace | `\t`, `\n` | preserved (used for layout) |
|
||
| All other codepoints | printable ASCII, Latin diacritics, CJK, emoji | preserved unchanged |
|
||
|
||
The sanitizer visibly encodes control bytes rather than silently
|
||
stripping them, so a downstream operator reviewing a captured transcript
|
||
can still see the original payload (e.g. `\x1b[31mred`) instead of a
|
||
mystery gap. Unicode is preserved: CJK characters, emoji, and accented
|
||
Latin all round-trip byte-for-byte.
|
||
|
||
JSON mode and MCP mode are not affected by this text-mode sanitizer:
|
||
they continue to return the full normalized response. Go's
|
||
`encoding/json` escapes JSON control characters such as C0 controls, but
|
||
does not rewrite every terminal-significant Unicode codepoint such as
|
||
C1 controls; consumers should treat returned strings as untrusted data,
|
||
not terminal-ready text. Tests covering ANSI, OSC 52 clipboard payloads,
|
||
HTML-entity-encoded ESC, C1 controls, carriage returns, and Unicode
|
||
preservation live in `format_sanitize_test.go` and `format_test.go`.
|
||
|
||
---
|
||
|
||
## Limits
|
||
|
||
The server applies a small set of size limits to keep responses manageable
|
||
and to protect consumers from a misbehaving SearXNG upstream.
|
||
|
||
| Limit | Value | Applies to | Behavior on overflow |
|
||
|-------|-------|------------|----------------------|
|
||
| Response body size | 2 MB (`MaxResponseBodySize`) | All modes (CLI, JSON, MCP) | The response is rejected with a `SearXNGError` whose message contains `"response body exceeded maximum size limit"`. This is a guardrail against a malformed or misconfigured upstream; a normal SearXNG JSON response is well under 2 MB. |
|
||
| Per-result content | 4000 Unicode characters (`MaxContentRunes`) | CLI text mode only | `formatResults` truncates result summaries (`content` field) and infobox content to 4000 runes. JSON mode and MCP mode return the full normalized response without this formatting truncation. |
|
||
| Answers count | 100 (`MaxAnswers`) | All modes (CLI, JSON, MCP) | `normalizeResponse` trims `Answers` to the first 100 entries before answer/infobox deduplication and logs a warning. Bounds the O(n*m) dedup work even when a compact JSON body fits many entries below the body cap. See `docs/SEARXNG_ANSWER_DEDUP.md` for details. |
|
||
| Infoboxes count | 100 (`MaxInfoboxes`) | All modes (CLI, JSON, MCP) | `normalizeResponse` trims `Infoboxes` to the first 100 entries before answer/infobox deduplication and logs a warning. Same rationale as `MaxAnswers`. |
|
||
|
||
The response-body-size cap is checked before the JSON is parsed, so the
|
||
result is a hard error rather than a partial response. MCP consumers receive
|
||
the `SearXNGError` as a `Search error: ...` reply; CLI consumers see it in
|
||
stderr/exit code. The error's `ResponseBody` field carries up to 200
|
||
bytes of the truncated body for debugging (`MaxErrorDisplayBytes`); the
|
||
truncation walks back to a valid UTF-8 rune boundary so multi-byte
|
||
sequences are never split.
|
||
|
||
---
|
||
|
||
## Quick Reference
|
||
|
||
```bash
|
||
# CLI text mode (default)
|
||
./searxng-mcp-go "apple inc"
|
||
|
||
# JSON mode
|
||
./searxng-mcp-go "apple inc" --json
|
||
|
||
# With additional parameters
|
||
./searxng-mcp-go "apple inc" --json --language en --safesearch 1 --time_range month
|
||
|
||
# Custom SearXNG server
|
||
./searxng-mcp-go "query" --searxng-url=https://your-searxng.example.com
|
||
|
||
# Debug mode
|
||
./searxng-mcp-go "query" --debug
|
||
```
|