hermes-agent/evals/gateway_failure_ownership
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Teknium 3114916ee4 fix(gateway): carry accepted-input ownership through persistence
Namespace delivery markers and assign fresh keyless turn identities instead
of inferring ownership from IDs or process-local row baselines. Query only
marker existence on the canonical live compression continuation and ancestors.
Preserve raw reply IDs and exclude metadata from provider wire messages.

Expand the two existing invariants with resumed cross-chat ID collisions,
a real independent SQLite writer, reaped siblings, and archived-history
allocation controls. All 20 full-handler checkpoints and 63 targeted tests pass.
2026-09-07 14:11:18 -07:00
..
foreign_writer.py fix(gateway): carry accepted-input ownership through persistence 2026-09-07 14:11:18 -07:00
probe.py fix(gateway): carry accepted-input ownership through persistence 2026-09-07 14:11:18 -07:00
README.md fix(gateway): carry accepted-input ownership through persistence 2026-09-07 14:11:18 -07:00

Gateway failure-writer ownership

Offline integration: _handle_message → session preparation and lease → production _run_agent / TurnRunner → real AIAgent → loopback HTTP/SSE → on-disk SQLite → gateway persistence/delivery → exception fallback.

.venv/bin/python evals/gateway_failure_ownership/probe.py "$PWD" /tmp/ownership.json
scripts/run_tests.sh -j 1 tests/gateway/test_failure_writer_ownership.py

The first argument selects the production checkout, so the same fixture can A/B another checkout without editing it. The receipt records its temporary HERMES_HOME. Inherited credentials are cleared, tools are disabled, and socket connections are restricted to loopback. No live messaging account or paid provider is contacted.

Fault boundaries and controls

  • Real HTTP 400 from the loopback peer; successful replies use HTTP/SSE.
  • Controlled AIAgent constructor failure, or voice-policy failure after persistence. The handler and exception writer remain production code.
  • Chat A succeeds with ID 100; chat B resumes that session and fails construction with a different input and the same ID. Both inputs must remain, with unmodified platform IDs for quote/reply lookup.
  • A separate Python process commits an unrelated, nonobserved input during a keyless constructor failure. Both writers' inputs must survive; the gateway lease is not treated as universal writer authority.
  • Separately accepted identical keyed/keyless inputs, identical timestamps, same-delivery pre-agent retry, provider failure/recovery, and healthy follow-up.
  • A normal history read fails closed before agent construction. There is no longer a raw baseline to read or an extra keyless admission query.
  • 5,000 archived 4-KiB rows remain in SQLite. The complete keyless failure handler must stay below 8 MiB traced peak allocation, with zero raw transcript scans.
  • Durable owner markers are unique across accepted inputs and absent from provider wire messages. The foreign writer supplies its own independent marker.

The second invariant uses real SQLite archives and a compression tree containing an earlier ws_orphan_reap sibling and a live successor. It checks both published reroutes and map-free restart routing; root/middle ancestors and successor writes can establish ownership, while reaped, undone, observed, foreign-marker, and unmarked rows cannot. These are storage-boundary checks, not provider-driven compaction.

Receipts

The expanded full-handler matrix has 20 checkpoints (19 deliveries plus marker propagation). Base 869228cab4a8276d3b4c78da9d9939670c47bd0f: 7/20; reviewed intermediate 653cd72ef63b013798748774a2509f453b566a77: 1/20; fixed: 20/20. All fault boundaries were reached. Base and fixed runs made 10 loopback requests and recorded zero external connection attempts. Counts are cumulative exact-row sequence checks, not independent defect counts.

The reviewed intermediate lost chat B's input (1 row instead of 2), also lost the keyless input beside the independent writer, and allocated 28,034,300 bytes in the archived-history handler. The fixed run retained all 18 expected active user rows (including the independent writer) and used 107,679 bytes in that handler. The memory measurement excludes fixture seeding and receipt serialization.

Exactly two invariant tests are retained. Eleven targeted files passed 63 tests; broader directory suites and CI are not claimed. No historical rows are rewritten. Existing rows without a marker cannot establish delivery ownership, so historical redelivery is not deduplicated by this mechanism. This is exception-path input arbitration, not global exactly-once delivery or content deduplication.