hermes-agent/tests/fakes/providers
Repository files (latest commit first)
Filename Latest commit message Latest commit date
teknium1 7088ba3b94 test: provider-catalog E2E: merge-order-safe KNOWN gates, exact-path fake, bounded vendor stalls
Review fixes on the provider-catalog matrix:

- KNOWN bugs go through tests.e2e.core._pending_fixes.known_failure again (strict_known and the
  "now green -> drop KNOWN" asserts are gone), so a fix PR landing first simply turns its cells
  green. Each KNOWN is gated on the bug's OWN observed signature via a dedicated CatalogGap raised
  only at the gated assertion: xai = no inference at the fake + api.x.ai CONNECT; listing = vendor
  host hit with no probe error; nebius switch = vendor-host validation error; fallback = fallback
  fake untouched + its vendor host CONNECTed; OAuth patterns anchored. Unlisted red cells still fail.
- CatalogFake answers only exact routes (configured base path + dialect endpoint / listing path),
  404 otherwise; reached_own_endpoint and the listing cells assert the exact path.
- Turns run with agent.auto_recovery_cycles: 0 (the documented post-exhaustion ladder parked the
  xai/fallback rows for minutes: 14 CONNECTs over 118 s, bounded by design, not a retry bug) and a
  watchdog kills a child 8 s after a vendor-host CONNECT with no inference at its fake.
- An unknown api_mode fails the row instead of skipping; only explicit auth types and the named
  keyless provider skip.
- Usage cell asserts the exact sum the fake reported for answered main-turn calls.
- Listing 404 / hang degradation cells for the rows that already list from the configured endpoint.
- Shard body moved into the helper; fallback key check uses the dialect's auth header; the three
  unconditional-skip OAuth params dropped (kept under NOT COVERED).
2026-09-24 05:41:14 -07:00
..
__init__.py test: OpenAI Responses fake (SDK-oracle validated) + Responses replay/parallel-call/soft-failure E2E 2026-09-24 04:32:16 -07:00
anthropic_messages.py test: SDK oracle checks every content block against its request Param (assistant blocks were unchecked) 2026-09-24 04:32:48 -07:00
bedrock_converse.py test: Bedrock Converse/ConverseStream wire-conformance E2E suite (SigV4-verified eventstream fake, tools, signed reasoning, resume, compaction, faults) 2026-09-24 05:01:07 -07:00
catalog_fake.py test: provider-catalog E2E: merge-order-safe KNOWN gates, exact-path fake, bounded vendor stalls 2026-09-24 05:41:14 -07:00
catalog_oauth.py test: provider-catalog OAuth E2E (device-code cadence, refresh rotation) + multi-dialect loopback fake 2026-09-24 05:41:14 -07:00
chat_variants.py test: providers-openai E2E review fixes: decoys on the custom: slug path, fail-fast strict endpoint, Responses faults, KNOWN hygiene 2026-09-24 04:32:16 -07:00
codex_app_server.py test: native provider suites: KNOWN xfails fail only on their symptom; codex orphan released without a foreign kill 2026-09-24 05:01:07 -07:00
copilot_acp.py test: copilot-acp auto compaction mid-turn keeps the next ACP prompt valid and grounded 2026-09-24 05:01:07 -07:00
gemini_native.py test(e2e): Gemini native generateContent wire-conformance suite (fake Google TLS endpoint + tools/resume, schema, errors, compaction) 2026-09-24 05:01:07 -07:00
oauth_token_server.py test: Anthropic OAuth mid-turn refresh E2E via TLS-intercepting token proxy (#120815 strict xfail) 2026-09-24 04:32:48 -07:00
openai_responses.py test: providers-openai E2E review fixes: decoys on the custom: slug path, fail-fast strict endpoint, Responses faults, KNOWN hygiene 2026-09-24 04:32:16 -07:00
vertex.py test: Vertex AI wire-conformance E2E suite (OAuth JWT mint, TLS proxy fake, tools, signatures, errors, compaction, stream drop) 2026-09-24 05:01:07 -07:00