mirror of
https://github.com/xlionjuan/searxng-mcp-go.git
synced 2026-11-17 04:17:51 +00:00
* docs: align documentation with code behavior (issues #547-#551) - INSTALL.md: drop SEARXNG_MAX_RETRIES=-1 from the unparsable-value example list; cross-reference the hard-error semantic path - PROMPT_INJECTION_SAFETY.md: mark Option A (warning header + warning field) as implemented, referencing #448 and #50 - MCP_TESTING.md: E2E binary is built once into a package-level cache, not per test; e2eMCPEnv lives in e2e_helper_test.go; goroutine-leak guard is suite-wide, not stress-file-specific - e2e-tests.md: fix workflow line citations (:87, :104) - verification.md: drop misleading '(except E2E)' qualifier; narrow the E2E-workflow coverage claim for exit-code tests - test-server.md: add missing test-server-pid-helper recipe row - .env.example: document SEARXNG_ALLOW_GET_FALLBACK opt-in - MCP_TOOLS.md: include 'failed to execute search request' segment in the network-failure error format - OUTPUT_FORMAT.md: sync Example 1 count line with the (showing N) format - adr/014: suppression was relocated to searchWithRetries, not removed - CONTEXT.md: note empty-text answers are dropped before the engine gate * docs: address review findings on doc accuracy (P2 items) - INSTALL.md: SEARXNG_MAX_RETRIES format is an integer in the range 0-20, not 'non-negative integer' — -1 parses and is rejected by SetMaxRetries, so the old wording conflated syntax and semantics - MCP_TESTING.md: example and E2E_MCP_BINARY section no longer claim a per-test build; list e2e_mockserver_test.go among the ungated CLI tests that build per-test via buildTestBinary - CONTEXT.md: dedup description notes the early return when no non-empty infobox text exists (original answers returned unchanged) * docs: fix remaining dedup and E2E_MCP_BINARY wording (P2) - CONTEXT.md: example dialogue no longer claims non-DuckDuckGo answers are never touched; empty answers are dropped before the engine check when dedup runs, matching the Search Behavior entry - test-server.md, verification.md: E2E_MCP_BINARY skips the MCP E2E package-level build, not the ungated CLI per-test builds * docs: scope dedup pass-through claim to non-empty answers SEARXNG_ANSWER_DEDUP.md and the DeduplicateAnswers doc comment both claimed non-DuckDuckGo answers always pass through. In fact, when dedup runs (non-empty infobox content exists), empty-text answers are dropped before the engine gate; only non-empty non-DuckDuckGo answers pass through. Aligns both with the implementation in deduplicate.go. * docs: fix dedup algorithm order in SEARXNG_ANSWER_DEDUP.md Document the empty-text filter running before the engine gate (matching deduplicate.go) and add the early-return precondition; sync the Edge Cases table so retain behavior is scoped to non-empty answers. * docs: qualify empty-answer drop with early-return precondition SEARXNG_ANSWER_DEDUP.md edge-case row and the deduplicateAnswers wrapper comment both stated empty-text answers are always filtered. When no non-empty infobox exists the implementation early-returns the original answers unchanged; scope the drop to dedup runs only.
55 lines
3.2 KiB
Markdown
55 lines
3.2 KiB
Markdown
# E2E Tests
|
||
|
||
- Core functional tests assert non-zero results. If the live server returns only
|
||
an infobox with 0 web results, tests log a warning instead of failing, and
|
||
warnings are collected in a `WARNING SUMMARY` block at the end of the test
|
||
group (`TestMCPFunctional`, `TestMCPStdioE2E`, `TestCLISmoke`).
|
||
- Exceptions are owner-approved and documented. Known limitations of the CI
|
||
SearXNG instance — for example, the `files` category being excluded from
|
||
`TestMCPFunctional`'s category loop, or the `time_range=""` subtest
|
||
downgrading zero-result outcomes to a `WARNING SUMMARY` entry — must not be
|
||
broadened without approval. (The `time_range=""` subtest is named `"all"` in
|
||
`TestMCPFunctional`'s `"all time ranges"` subtest.) The `time_range="day"`,
|
||
`"month"`, and `"year"` subtests are expected to potentially return zero
|
||
results and log the outcome as `(persistent, expected)`; they are not
|
||
downgraded to warnings and are not part of the exception list.
|
||
- Adding strict assertions is preferred when a test can meaningfully assert
|
||
non-zero results with a better query or parameter combination.
|
||
- Live-server assertions belong in Go E2E tests (`TestMCPFunctional`,
|
||
`TestMCPStdioE2E`, `TestCLISmoke`) so they can use the shared warning-summary
|
||
path. Shell smoke in `.github/workflows/e2e.yml` must stay limited to
|
||
deterministic exit-code or structural checks and must not use `|| echo`,
|
||
`grep -Eq "A|B"`, swallowed `python3 -c` assertions, or similar patterns that
|
||
mask missing live-server evidence.
|
||
- For local server setup, use `just test-server-start`; never run
|
||
`searxng-server-test/01-start-fg.sh` from agents or CI. See
|
||
`docs/agents/test-server.md`.
|
||
|
||
## CI Retry Wrapper
|
||
|
||
`.github/workflows/e2e.yml` wraps all E2E test suites — those matching
|
||
`-run 'TestMCP'` (which includes `TestMCPFunctional`, `TestMCPStdioE2E`,
|
||
`TestMCPErrors_InvalidInputs`, `TestMCPLifecycle_SIGINTGracefulShutdown`,
|
||
`TestMCPLifecycle_SIGTERMGracefulShutdown`, and
|
||
`TestMCPLifecycle_InvalidJSONAfterInitialize`) and `-run 'TestCLISmoke'` — in
|
||
`Wandalen/wretry.action` with `attempt_limit=3` using
|
||
`-v -tags=e2e -run 'TestMCP' -race -count=1 -timeout=600s .` (see
|
||
`.github/workflows/e2e.yml:87` and `:104` for the exact commands).
|
||
This is because upstream search engines probabilistically rate-limit or ban
|
||
the CI SearXNG instance during automated runs — a characteristic inherent to
|
||
meta-search against live third-party engines, not a code or SearXNG bug.
|
||
|
||
Library-level retry (`SEARXNG_MAX_RETRIES=2`) covers single-request flakes;
|
||
the CI retry wrapper recovers from whole-instance throttling by retrying the
|
||
entire test run to land on a clean window. See the inline comment in
|
||
`e2e.yml` for the worst-case budget.
|
||
|
||
## Race Detector
|
||
|
||
E2E workflows (`e2e.yml`, `e2e-stress.yml`) run with `-race`. The race
|
||
detector adds non-trivial wall-clock overhead (typically 2–5× for a given
|
||
test binary), but this is the expected cost of catching data races at the
|
||
live-server layer — the only layer that exercises the real subprocess stdio
|
||
boundary, real SearXNG integration, and real MCP `CommandTransport` lifecycle.
|
||
The existing `-timeout` values include sufficient headroom for this overhead;
|
||
if runs become timeout-limited, increase them as needed.
|