mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-11-17 04:11:35 +00:00
* feat(i18n): layered catalogs — plugin packs and user overlay over bundled locales
* feat(tui): i18n layer — en catalog, nanostore runtime, RPC pack loader, _keys.tui.json emitter
ui-tui/src/i18n/: en.ts (facade over topical siblings under en/), types.ts
(Translations + dotted TranslationKey derived from en), runtime.ts ($locale/
$catalog atoms, translateFrom active→en→key, pack merge with string→fn
wrapping for {0}/{1} placeholders), loader.ts (display.language →
i18n.catalog {lang, surface:'tui'}, English when the method is missing),
useT()/useLocale() hooks, t() for non-React code. useConfigSync feeds the
loader from the existing config.get full hydration. `npm run i18n:keys`
writes locales/_keys.tui.json (sorted flat key list) and runs before build.
* feat(plugins): provides_locales manifest field, ctx.register_locale/register_locale_dir, manifest-only language packs
* chore(tui): split en catalog siblings by lane (slash sibling)
* feat(plugins): validate language packs — parse, text-only, key-subset WARN against en / _keys exports
* feat(tui_gateway): i18n.languages / i18n.catalog RPC + regenerated contracts
* feat(config): display.language accepts any supported_languages() id, refuses unknown ids with the list
* docs(i18n): language packs user guide, pluggable display.language, plugin developer section, AGENTS notes
* feat(plugins): report language-pack layers in the mid-run activation summary
* feat(tui): wire status bar, composer placeholders, hotkey help and approval/clarify/confirm prompts through i18n
StatusRule maps compared state values (ready/running…/summoning) to catalog
text at render via displayStatus(); hotkeys()/placeholder() resolve lazily so
a pack that arrives after boot applies. Catalog grows to 81 keys.
* feat(desktop): pluggable app locales — registry, host.i18n.registerAppLocale, backend packs, keys emitter
- Locale widens to string (BundledLocale keeps the union); TRANSLATIONS stays
the bundled record and every consumer resolves through the registry.
- src/i18n/registry.ts: registerAppLocale(id, {endonym, rtl, translations})
layers partial packs (nested or flat dotted) over bundled/en via
mergeTranslations; a string over a function-valued en entry becomes a
positional {0}/{1} formatter; $appLocaleVersion bumps so translators
re-render; per-source disposers + replaceAppLocaleSource for atomic swaps.
- Backend packs: i18n.languages + i18n.catalog {surface:'desktop'} feed the
registry as source 'backend' (method-not-found is silent); re-synced on
socket open, display.language change and profile switch. A saved pack-only
language is promoted once its pack registers.
- SDK: host.i18n.registerAppLocale / languageOptions; ctx.i18n.registerAppLocale
tracked for unload. Docs in the desktop plugin SDK guide + skill reference.
- Language switcher lists bundled ∪ registered ∪ backend, endonym-only; RTL
from the registry (applyDocumentLocale takes rtl).
- npm run i18n:keys emits locales/_keys.desktop.json (wired into build).
* i18n(cli): route /topup + /subscription copy through t() (cli.billing.*, cli.subscription.*)
Module-level copy tables and modal choice tuples in cli_billing_mixin.py froze
English at import, before display.language was known. They are now key tables /
builder functions evaluated at call time; every user-facing line in the /usage
balance block, /subscription and the five /topup screens reads the catalog.
Choice VALUES stay English identifiers. Fragment-assembled status lines
(Plan: … → cancels · $x left · renews …) become full templates.
* i18n(gateway): exec-approval card contract + base/run/run_busy/run_inbound replies through t()
- base_exec_approval: EA_* English constants stay; add ea_header_text()/ea_reason_label_text()/
ea_smart_deny_line_text()/ea_default_reason_text()/ea_action_labels()/approval_timed_out_notice()
accessors; deadline + timed-out notice resolve via gateway.exec_approval.*
- BasePlatformAdapter._EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/_EA_ACTION_LABELS become
properties (adapters still shadow them with markup class attrs)
- run.py: provider error replies table holds catalog keys; _CONTEXT_OVERFLOW_REPLY -> _context_overflow_reply()
- run_busy/run_inbound: typed approval + slash-confirm matchers accept English ∪ approval.inputs.* (t())
- locales/en.yaml: gateway.exec_approval/busy/errors/... namespaces
* i18n(cli): wire modal, loops, agent-setup mixins through t() (cli.* keys)
* i18n(platforms): route Slack, Matrix and Feishu user-facing text through t()
Exec-approval markup overrides (_EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/
_EA_ACTION_LABELS) become per-call properties over the shared
gateway.exec_approval.* contract keys, so Slack's 3000-char section budget
measures the resolved template. Slack _APPROVAL_DECISIONS/_CONFIRM_DECISIONS,
Feishu _APPROVAL_LABEL_MAP and Matrix _EA_LEGEND/_EA_TYPED_HINT turn into
key tables resolved at click time; the Matrix typed hints become whole
sentences per offered tier instead of spliced fragments. Slack button labels
are cut to 75 chars and select placeholders to 150 after translation; the
model-facing clarify fallback answer ('choice N') stays English while the
card copy localizes.
locales/en.yaml gains the gateway.exec_approval.* contract keys plus the
platform.shared.* / platform.slack.* / platform.matrix.* / platform.feishu.*
namespaces (and the keys for the other adapters wired in follow-up commits).
* i18n(gateway): run_turn / run_turn_runner / approval-settle copy through t()
- status hints, proxy errors, background task notices, progress heartbeats, session info lines
- tool progress chrome (tool_head/tool_pending/tool_preview/tool_verbose) shared by base.format_tool_event
- run_turn_runner:1406 Chinese clarify placeholder -> gateway.clarify.native_stream_placeholder (zh text kept in zh.yaml)
- _UNEXPECTED_SILENCE_REPLY/_CLARIFY_EXPIRED_NOTICE -> accessor functions
* i18n(platforms): route Google Chat and Teams user-facing text through t()
Google Chat clarify card, typing placeholder, orphan-card labels and the whole
/setup-files reply set (module constants become platform.google_chat.setup_files.*
keys resolved at reply time). The attachment-fallback notice that shipped
hardcoded in Spanish is keyed with an English en value; es.yaml carries the
original Spanish text for those four keys.
Teams approval card header/reason use the gateway.exec_approval.* contract,
_APPROVAL_LABELS becomes a key table resolved at click time, and the meeting
summary writer resolves its section headings/fallbacks per render.
* i18n(platforms): route LINE, WeCom, email, DingTalk, IRC and Home Assistant text through t()
LINE default copy constants become catalog keys resolved in __init__ (the
LINE_*_TEXT / extra.* operator overrides still win); the busy-ack bypass
matcher keys on the leading emoji marker only, so it keeps firing once the
gateway busy heads are localized. WeCom media size/format notices that shipped
hardcoded in Chinese are keyed with English en values and zh.yaml carries the
original Chinese text. DingTalk emotion bubbles resolve per send.
* i18n(cli): route /model switch output and -q status lines through t() (cli.model.*, cli.single_query.*)
Switch-summary labels shared with the gateway reuse gateway.model.* keys
(provider/context/max-output/capabilities/prompt-caching); CLI-only variants
(glyph or no-backtick forms) live under cli.model.*. The hand-padded /model usage
block becomes a (form, description-key) table padded at render time so the
command syntax stays fixed while descriptions translate. -q 'Error:' reuses
gateway.model.error_prefix.
* i18n(cli): route TUI panel/hint/placeholder copy through t() (cli.tui.*)
_APPROVAL_CHOICE_LABELS and _TUI_MODAL_HINTS become key tables resolved at
render time; vault/sudo panel bodies are one catalog value per panel split on
newline; inline plurals use <key>_one/<key>_other. Adds the cli.* namespace
(shared/tui/voice/render/subagents/dock) to locales/en.yaml.
* i18n(cli): voice/wake-word CLI copy through t() (cli.voice.*)
RuntimeError texts raised in _voice_start_recording are human copy (callers
print {e}) and are keyed; the Termux requirement-check match stays English.
Wake state ids stay internal, only their labels localize.
* i18n(cli): live-work dock, subagent monitor and render copy through t()
cli.subagents.* / cli.dock.* / cli.render.*; count fragments pluralize via
_one/_other keys, verdict table holds keys resolved at paint time so width
clipping measures the translated text.
* i18n(gateway): unauthorized/pairing, voice, topics, shutdown, startup, notifications, kanban pings through t()
* i18n(cli): move tips + composer placeholders into the catalog (tips.tNNN / tips.placeholder.pNN)
get_random_tip()/get_random_composer_placeholder() pick a key from the English catalog
(the parity baseline, probed once per process) and resolve it through t() for the active
language, so language packs translate tips like any other string. Also lands the cli.*
en.yaml namespace consumed by the CLI info/help/error-copy wiring in the next commit.
* i18n(cli): wire chat-turn + session mixins through t(); kanban log trimmer matches t() output
* test(cli): assert TUI/dock/voice copy via t(key); prove labels resolve at render time
Pinned-English assertions in the approval-UI, live-work dock and voice tests now
go through the catalog. New test swaps the catalog after import and checks the
approval panel + hint row follow it (the reason _APPROVAL_CHOICE_LABELS and
_TUI_MODAL_HINTS became key tables).
* i18n(cli): route CLI info/help/error copy through t() (cli.* namespace)
cli_info_mixin: /help consumes CommandDef.describe() (added to commands.py: slash.<name>.description
with fallback to .description), section titles/skill/quick-command headers, /tools, /toolsets,
/usage labels, /context, /whoami, /insights, /gateway status, tool-progress labels, bang-shell
denials, MCP config-watch + /reload-mcp confirm/reload lines, /reload-skills, and the session-store
warning all read the catalog at call time (module-level label tables became functions so the
active language is honoured after startup). cli.py: worktree cleanup, tirith warning, show_config
(labels re-padded at print time), quick/plugin/skill slash-command errors, ambiguous-command hint,
stdin error, gateway start, profile warning. cli_chat_error_copy / cli_unknown_command /
cli_output / cli_init_mixin: chat error panel copy, did-you-mean lines, n-more / yes-no prompt
(localized affirmative initial alongside 'y'), unknown-toolsets warning.
* i18n(w1a): wire agent display/explainers/approval + slash registry/help through t()
- hermes_cli/commands.py: CommandDef.describe() resolves slash.<name>.description
at call time; category labels via slash.category.*; help/alias/usage suffixes
via slash.shared.*; gateway_help_lines and commands_platforms/slash_exec use them.
- agent/display.py: display.verb.* resolved at call time via get_tool_verb();
bridge/spinner/thinking-verb/cute-row/failure/preview/diff text via display.*.
- agent/turn_explainers.py: exit-reason / persistence-cause tables become call-time
lookups (explainer.exit.*, explainer.persistence.*, explainer.file_mutation.*).
- agent/background_review.py, session_activity.py, context_breakdown.py,
status_output.py: review summaries, iteration progress, context notices.
- tools/approval.py, approval_context.py: approval.summary.*, approval.noun.*,
approval.window.* pluralized keys.
- gateway/slash_commands*.py: remaining raw strings (busy, whoami, platform,
bundles, memory, skills, approvals, set_home, diff, update, debug, profile,
heartbeat, refine, review, subgoal, loop, retry, compress codex path, save,
sessions, model guard/errors, agents rows, topup, login). HISTORY_UNREADABLE
keeps its English constant; callers use history_unreadable() ->
gateway.shared.history_unreadable.
- locales/en.yaml: new approval/display/explainer/slash blocks + gateway leftovers.
* i18n(telegram): route adapter chat copy through t()
Approval card (header/reason/smart-deny as HTML-escaped properties), inline
button labels, callback toasts (cut at Telegram's 200-char cap), model/choice
pickers, clarify/update/slash-confirm prompts, gmail-triage labels and the
inbound-media failure notice now come from the catalog. _UNAUTHORIZED is a
lazy _unauthorized() so the import no longer binds a language. The command
menu carries a language+payload fingerprint (forum scopes re-register on
change) and BotCommand descriptions are cut at 256.
Adds gateway.exec_approval.* (WAVE2 contract), platform.telegram.*,
platform.discord.* and the slash.*.description keys the Discord table shares
with the CLI registry to locales/en.yaml.
* i18n(gateway/platforms): whatsapp_cloud, yuanbao, weixin, signal, api_server copy through t()
- whatsapp_cloud: clarify list/buttons, approve/deny + slash-confirm labels via platform.whatsapp.* (t()-then-truncate at 20/24/72 caps); _EA_HEADER becomes a property wrapping ea_header_text()
- yuanbao: SLOW_RESPONSE_MESSAGE -> slow_response_message() (platform.yuanbao.slow_response_notice; zh keeps the original text); cron-wrapper markers centralized as module constants for strip_cron_wrapper
- api_server: PROVIDER_AUTH_FAILED_LABEL/PROVIDER_RATE_LIMITED_LABEL stay English for run.py matchers; user_text() renders via t()
- signal/_format_wait, weixin voice caption, openai_routes transformed notice
- run_turn: second _UNEXPECTED_SILENCE_REPLY consumer -> accessor
* i18n(discord): route adapter chat copy through t()
Native slash-command table becomes _NATIVE_SLASH_COMMAND_SPECS holding catalog
keys; _native_slash_commands() resolves descriptions, parameter descriptions
and Choice names for the active language, each cut at Discord's 100-char cap,
and the app-command sync fingerprint now includes get_language() so a
display.language change re-syncs. Exec-approval card (gateway.exec_approval.*
contract), slash-confirm / clarify / update views, model+choice pickers,
thread creation, forum titles, voice acks, the response-truncation notice,
the unauthorized-slash security alert and the media upload-size notices all
read from platform.discord.*. Decorator-declared button labels are relabelled
in __init__ (80-char cap); embed titles cut at 256, select placeholders at
150, option label/description at 100. _UNAUTHORIZED is a lazy _unauthorized().
* i18n(cli): wire status-bar, stream, terminal mixins + terminal_input through t(); rename kwargs that shadow t(key)
* i18n: wire hermes_cli/cli_commands_mixin.py slash-command copy through t()
- 431 new leaves under cli.commands.<cmd>.* in locales/en.yaml; 12 rows reuse
existing gateway.* keys (rollback, diff, resume, branch, btw, model, reasoning)
via a _gt() helper so CLI and gateway replies stay identical.
- Module-level English tables (_BUSY_MODE_*, _REASONING_TOGGLES, _HATCH_PROGRESS,
_DIFF_LABELS, _LOCAL_ENGINE_LINES) become call-time catalog lookups keyed by id.
- Verb tables (Enabling/Disabling, Paused/Resumed/Triggered, planned/done,
Updating/Generating) are one full template per variant; plurals use
<key>_one/<key>_other via _tn(); hand-padded column labels (/snapshot list)
translate the value and re-pad at the call site.
- Multi-line usage blocks are single catalog values split with _lines().
- Model-facing system notes and DB-stored reasons stay English (EXCLUDED).
* tests: assert /handoff, /worktree, /login CLI copy via t(key) instead of pinned English
* test(i18n): pin Telegram/Discord adapter catalog wiring
Lazy unauthorized notice, exec-approval contract keys, HTML escaping before
Telegram <b> wrapping, 200-char toast / 256-char BotCommand caps, Discord
100-char app-command text and 80-char button caps, and language-bearing
command-menu fingerprints on both platforms.
* i18n: reconcile cli.shared on/off vs enabled/disabled after lane merge
* i18n: describe() in TUI-gateway slash listings; localize TUI exit resume hint
* i18n(tr): translate bundled catalog + tui pack
* i18n(ja): translate bundled catalog + tui pack
* i18n(ko): translate bundled catalog + tui pack
* i18n(zh): translate bundled catalog + tui pack
* i18n(fr): translate bundled catalog + tui pack
* i18n(af): translate bundled catalog + tui pack
* i18n(uk): translate bundled catalog + tui pack
* i18n(ar): translate bundled catalog + tui pack
* i18n(pt): translate bundled catalog + tui pack
* i18n(it): translate bundled catalog + tui pack
* i18n(es): translate bundled catalog + tui pack
* i18n(zh-hant): translate bundled catalog + tui pack
* i18n(ru): translate bundled catalog + tui pack
* i18n(hu): translate bundled catalog + tui pack
* i18n(hu): translate pre-existing English-valued leftovers (kanban wake, /context, /status, fast labels)
* i18n(de): translate bundled catalog + tui pack
* i18n(ga): translate bundled catalog + tui pack
* test(i18n): fixture matches _normalize_lang(lang, home) signature
* i18n(tui): scaffold userMessages/slashCmd en siblings
* i18n(tui): wire secure prompts + content tables
* feat(tui): i18n — wire billing, subscription, connection-setup and journey overlays
Adds en siblings billing.ts / subscription.ts / connection.ts (namespaces
billing, subscription, connection, journey) and routes every user-facing
literal in billingOverlay, subscriptionOverlay, connectionSetupOverlay and
journey through useT()/messages(). Module-level label tables became lazy
(scopeStillDeniedResult(), verbOf(T, action)); auto-reload rows dispatch on
stable ids instead of label text. Regenerates locales/_keys.tui.json.
* i18n(tui): wire slash ops/wake replies
* i18n(tui): wire pickers (modelPicker, activeSessionSwitcher, petPicker)
* i18n(tui): wire slash core/debug/setup replies
* i18n(tui): wire hubs (agents overlay/panel/controls, skills, plugins)
* i18n(tui): wire slash session/topup/subscription replies
* i18n(tui): wire chat bits (branding, thinking, messageLine, loaders, todo, queued, banner, entry)
* i18n(tui): register t3 siblings (pickers, hubs, secure, content, chatBits) and regenerate keys
* i18n(tui): wire userMessages copy through the userMessages namespace
* i18n(tui): lazy-copy test for userMessages, regenerate _keys.tui.json
* i18n(tui): wire session/gateway/lib text through the TUI catalog (lane t2)
Adds en siblings session.ts, gatewayMsg.ts, libText.ts (namespaces session,
gatewayMsg, libText) and routes user-facing literals in app/{useMainApp,
useSessionLifecycle,useInputHandlers,turnController,createServerRequestHandler,
setupHandoff,createGatewayEventHandler}.ts, gatewayClient displayed reasons,
lib/*, domain/*, hooks/* through t()/messages(). Status-bar state values that
code compares against, backend-matched strings, log lines, model-bound text,
and machine 'error:' prefixes stay literal. Regenerates locales/_keys.tui.json
(232 keys).
* i18n: serve bundled locales/<lang>.tui.yaml under overlay/packs; TUI pack parity test; regen _keys.tui.json (1250)
* i18n: translate pre-existing English stubs in bundled locales (424 leaves, 14 locales)
* tui: i18n-export-en script (English templates for pack translators)
* docs(i18n): bundled TUI packs are the bottom layer of the tui surface
* i18n(ru): translate TUI pack
* i18n(ar): translate TUI pack
* i18n(es): translate TUI pack
* i18n(pt): translate TUI pack
* i18n(ko): translate TUI pack
* i18n(de): translate TUI pack
* i18n(ja): translate TUI pack
* i18n(fr): translate TUI pack
* i18n(tr): translate TUI pack
* i18n(it): translate TUI pack
* i18n(zh): translate TUI pack
* i18n(zh-hant): translate TUI pack
* i18n(hu): translate TUI pack
* i18n(uk): translate TUI pack
1,169 missing keys translated; 81 pre-existing kept byte-identical. Parity OK missing=0 extra=0 placeholder_mismatch=0 empty=0.
Deliberately identical to en: chatBits.branding.mcpSummary ({0} MCP), chatBits.thinking.agentsHint ((/agents)), session.main.voiceStt (◉ STT), session.main.voiceTtsSuffix ( [tts]), slashCmd.core.help.tuiSection (TUI), slashCmd.core.history.hermesTag (Hermes #{0}), slashCmd.debug.heapdump.heapPath (heapdump: {0}), slashCmd.debug.mem.rss (rss), subscription.stepUp.title (Remote Spending — product feature name, as in core catalog), content.faces.* (glyph-only kaomoji).
* i18n(ga): translate TUI pack
* i18n(af): translate TUI pack
* plugin_guard: locale catalogs in language packs step down the agent-config family
A translated status line such as "Updating AGENTS.md" in locales/<lang>.yaml is UI text the loader
reads as a string leaf; it cannot edit a file. The bundled en.yaml itself tripped agent_config_mod
at critical, making any faithful language pack uninstallable. Injection shapes keep full severity.
* plugin_validate_locales: read key exports with utf-8-sig (Windows footgun lint)
* i18n(relay): route relay adapter prompt copy through t(); drop dead import-bound approval header
Adds platform.relay.* (5 keys) to en and all 16 bundled locales, reusing the sibling platform
translations for the confirm buttons and the Other option.
* ci: fix TUI import order, MDX table pipe, main's overflow-warning wording in all locales; fresh-install fixture carries the i18n kernel
- ui-tui/src/i18n/en.ts: perfectionist/sort-imports (slash before slashCmd)
- docs plugins/index.md: escape the | inside the provides_locales table cell (MDX parsed <id> as JSX)
- display.notice.uncompressed_context_overflow: adopt main's wording (names compression.enabled: false
and /compact) in en + 16 locales; the guardrail test pins that phrase
- tests/scripts/test_fresh_source_install.py: the installer tail now resolves CLI text through
agent.i18n, so the fixture tree carries the i18n kernel + en.yaml (not the agent runtime)
* docs(desktop-plugin-sdk): double-backtick the template-literal example (MDX evaluated ${n})
* test(e2e): display.language is validated against the live language set; exclude it from the arbitrary-string set property
* commands: keep the localized COMMANDS/COMMANDS_BY_CATEGORY module __getattr__ after the compat block removal
* build: never write locales/_keys.*.json from the desktop/TUI builds; regenerate the committed desktop key export
The desktop build regenerated locales/_keys.desktop.json in the checkout, so a
hermes update that rebuilt the app left the tree dirty (Desktop update E2E:
'M locales/_keys.desktop.json'). The key exports are committed artifacts pinned
to en.ts by apps/desktop/scripts/i18n-keys.test.mjs and ui-tui i18n:keys:check;
builds read them, never write them. Regenerated after main's new desktop strings.
* test: unbreak two main-red timing tests the PR merge-ref inherits
- test_local_runtime racing fake publishes the modern state record (legacy pid-only
records are rejected since 65ff3ad353; main has been red on this test since)
- test_run_progress_topics ManyProgressLinesAgent waits for the first bubble instead of
a fixed 0.35s, which a loaded CI runner does not always meet
* chore(i18n): regenerate desktop key catalog for main's new strings (model pricing, copy changelog)
* test(e2e): torture-chamber fd monitor confirms a deleted sidecar is still held before calling it a leak
SQLite's WAL last-close unlinks -shm before closing its descriptor (unixShmUnmap, then
unixShmPurge), so a healthy close shows a (deleted) -shm for microseconds; the 20ms poll
occasionally caught that window on the short-lived opener and failed the episode.
* chore(i18n): regenerate desktop key catalog for main's telemetry/consent strings
* chore(i18n): regenerate desktop key catalog after main sync
---------
Co-authored-by: Teknium <[email protected]>
808 lines
42 KiB
Python
808 lines
42 KiB
Python
"""Kanban notifier: claim terminal task events per subscription and deliver them.
|
|
|
|
``GatewayKanbanWatchersMixin._kanban_notifier_watcher`` owns the loop and
|
|
the GC cadence; the per-tick claim (``_notifier_collect``) and the
|
|
per-subscription delivery (``_KanbanNotification``) live here.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import contextlib
|
|
import re
|
|
from functools import partial
|
|
from pathlib import Path
|
|
import weakref
|
|
from typing import Any, Callable, Optional
|
|
|
|
from agent.i18n import t
|
|
|
|
from gateway.kanban_watchers_common import _list_boards, _to_thread_process_service, logger
|
|
from gateway.wake import session_owned_by_profile
|
|
|
|
|
|
def _kbc():
|
|
from hermes_cli import kanban_db_connect
|
|
return kanban_db_connect
|
|
|
|
|
|
def _kbn():
|
|
from hermes_cli import kanban_db_notify
|
|
return kanban_db_notify
|
|
|
|
# "status" covers dashboard drag-drop and `_set_status_direct()`.
|
|
# ``review_requested`` wakes the origin like a block but is not one;
|
|
# the task is not archived so later review cycles keep notifying.
|
|
TERMINAL_KINDS = ("completed", "blocked", "gave_up", "crashed", "timed_out", "status", "archived", "unblocked", "block_loop_detected", "review_requested", "changes_requested")
|
|
# Kinds that hand a decision back to the origin, which must take a turn.
|
|
# status/archived/unblocked are bookkeeping.
|
|
_WAKE_KINDS = ("completed", "gave_up", "crashed", "timed_out", "blocked", "review_requested", "changes_requested", "block_loop_detected")
|
|
|
|
|
|
def diagnostic_event(ev) -> bool:
|
|
"""Infrastructure attention is distinct from an explicit owner decision."""
|
|
if ev.kind in {"crashed", "timed_out", "gave_up"}:
|
|
return True
|
|
if ev.kind in {"blocked", "block_loop_detected"}:
|
|
return (ev.payload or {}).get("kind") != "needs_input"
|
|
return ev.kind == "status" and (ev.payload or {}).get("status") in {"blocked", "triage"}
|
|
# Consecutive send failures (adapter raised OR reported SendResult(success=False))
|
|
# before a sub is dropped as a dead chat. 12 ≈ 60s at the 5s cadence: a transient
|
|
# API outage must not permanently unsubscribe a live review-gate channel.
|
|
# Subscriptions are removed only when the task reaches the irreversible archived status. ``done`` is
|
|
# reversible in review/controller flows, so removing its subscription would silence a later reopen. We used
|
|
# to also unsub on any terminal event kind (gave_up / crashed / timed_out / blocked), but that silently
|
|
# dropped the user out of the loop whenever the dispatcher respawned the task: a worker that crashes, gets
|
|
# reclaimed, runs again, and crashes a second time would only notify on the first crash because the
|
|
# subscription was deleted after the first event. Same shape as the reblock-after-unblock cycle that PR
|
|
# #22941 fixed for `blocked`. Keeping the subscription alive until the task is archived lets the cursor
|
|
# (advanced atomically by claim_unseen_events_for_sub) handle dedup, and any retry-loop event reaches the
|
|
# user. Per-subscription send-failure counter. Adapter.send raising means the chat is dead (deleted, bot
|
|
# kicked, etc.) — after N consecutive send failures the sub is dropped so we don't spin against a dead chat
|
|
# every 5 seconds forever. A genuinely dead chat still drops, just ~60s later — a fine trade for an
|
|
# unattended gate where a false drop means silent work pileup.
|
|
MAX_SEND_FAILURES = 12
|
|
|
|
_LOCAL_PATH_RE = re.compile(r"(?<![\w:/])(?:/(?:Users|home|private|tmp|var|etc|workspace)/[^\s,;]+|" r"[A-Za-z]:\\[^\s,;]+)")
|
|
|
|
|
|
def _safe_review_reason(value: Any, limit: int = 160) -> str:
|
|
"""Return a mobile-friendly review reason safe for external delivery."""
|
|
from agent.redact import redact_sensitive_text
|
|
|
|
reason = redact_sensitive_text("" if value is None else str(value), force=True, redact_url_credentials=True)
|
|
reason = " ".join(_LOCAL_PATH_RE.sub("[local path]", reason).split())
|
|
if len(reason) > limit:
|
|
reason = reason[: limit - 1].rstrip() + "…"
|
|
return reason
|
|
|
|
|
|
def _wake_scope_id(adapter: Any, sub: dict) -> Optional[str]:
|
|
"""Return the tenant scope (Slack workspace) a subscription's wake keys to.
|
|
|
|
``build_session_key()`` includes ``scope_id`` on multi-tenant platforms,
|
|
so the wake must carry the same scope as inbound messages. Persisted
|
|
``delivery_metadata`` wins (it records the creating scope); the adapter's
|
|
live chat → scope map only covers rows without metadata. ``None`` means
|
|
unscoped, matching an unscoped platform's key.
|
|
"""
|
|
delivery_meta = sub.get("delivery_metadata")
|
|
if isinstance(delivery_meta, dict):
|
|
for key in ("scope_id", "guild_id", "slack_team_id", "team_id"):
|
|
value = delivery_meta.get(key)
|
|
if value:
|
|
return str(value)
|
|
resolver = getattr(adapter, "scope_id_for_chat", None)
|
|
if not callable(resolver):
|
|
return None
|
|
try:
|
|
resolved = resolver(str(sub.get("chat_id") or ""))
|
|
except Exception as exc:
|
|
# An adapter-side lookup failure yields no scope, never an error.
|
|
logger.debug("kanban notifier: scope lookup failed for chat %s: %s", sub.get("chat_id"), exc, exc_info=True)
|
|
return None
|
|
return str(resolved) if resolved else None
|
|
|
|
|
|
_ANCHORLESS_WARNED: set[tuple] = set()
|
|
|
|
|
|
def _warn_anchorless_thread_sub_once(sub: dict, platform: str) -> None:
|
|
"""A thread-shaped subscription without ``parent_chat_id`` cannot match a channel-level
|
|
``profile_routes`` entry, so the fail-closed route gate skips it on every tick. Say so ONCE per
|
|
row at WARNING — a subscription that can never deliver was invisible below DEBUG (#110919)."""
|
|
metadata = sub.get("delivery_metadata") or {}
|
|
thread_like = bool(sub.get("thread_id")) or (sub.get("chat_type") or metadata.get("chat_type")) in {
|
|
"thread", "forum", "forum_post", "forum-post", "topic"}
|
|
if not thread_like or metadata.get("parent_chat_id"):
|
|
return
|
|
key = (sub.get("task_id"), platform, sub.get("chat_id"), sub.get("thread_id") or "")
|
|
if key in _ANCHORLESS_WARNED:
|
|
return
|
|
_ANCHORLESS_WARNED.add(key)
|
|
logger.warning(
|
|
"kanban notifier: subscription for %s on %s thread %s has no parent_chat_id anchor and matched no "
|
|
"profile route; it will not be delivered. Re-subscribe with `hermes kanban notify-subscribe ... "
|
|
"--parent-chat-id <channel id> [--guild-id <guild id>]`.",
|
|
sub.get("task_id"), platform, sub.get("chat_id"),
|
|
)
|
|
|
|
|
|
_UNROUTABLE_WARNED: set[tuple] = set()
|
|
|
|
|
|
def _warn_unroutable_sub_once(sub: dict, platform: Any, message: str, *extra_args: Any) -> None:
|
|
"""A routed subscription the credential gate fail-closes is a permanent dead-end: delivery
|
|
rewinds every tick with only a DEBUG line. Say so ONCE per row at WARNING, mirroring
|
|
``_warn_anchorless_thread_sub_once`` (#115460)."""
|
|
key = (sub.get("task_id"), platform, sub.get("chat_id"), sub.get("thread_id") or "")
|
|
if key in _UNROUTABLE_WARNED:
|
|
return
|
|
_UNROUTABLE_WARNED.add(key)
|
|
logger.warning(message, sub.get("task_id"), getattr(platform, "value", platform),
|
|
sub.get("chat_id"), *extra_args)
|
|
|
|
|
|
def _platform_names(mapping: Any) -> set[str]:
|
|
"""Lower-cased platform names of an adapters mapping (Platform enums or strings)."""
|
|
return {getattr(platform, "value", str(platform)).lower() for platform in mapping}
|
|
|
|
|
|
def _adapter_for_subscription(runner: Any, platform: Any, sub: dict, owner_profile: Optional[str]) -> Any:
|
|
"""Resolve a durable route without turning a missing secondary bot into primary authority."""
|
|
adapter = runner._authorization_adapter(platform, owner_profile)
|
|
config = getattr(runner, "config", None)
|
|
if not getattr(config, "multiplex_profiles", False):
|
|
return adapter
|
|
primary = runner.adapters.get(platform)
|
|
if adapter is not None and adapter is not primary:
|
|
return adapter
|
|
profile = owner_profile or getattr(runner, "_kanban_notifier_profile", None)
|
|
primary_profile = getattr(runner, "_primary_profile_name", None) or runner._active_profile_name()
|
|
profile = profile or primary_profile
|
|
# A profile holding its OWN adapter for this platform is an independent credential boundary —
|
|
# ``_authorization_adapter`` already answered for it, so the primary never stands in. Adapters
|
|
# on OTHER platforms do not gate this one: the primary bot is the only credential serving the
|
|
# pinned chat, for inbound turns and for these notifications alike (#115460).
|
|
own_adapters = (getattr(runner, "_profile_adapters", {}) or {}).get(profile) or {}
|
|
if getattr(platform, "value", str(platform)).lower() in _platform_names(own_adapters):
|
|
return None
|
|
metadata = sub.get("delivery_metadata") or {}
|
|
guild = metadata.get("scope_id") or metadata.get("guild_id")
|
|
parent = metadata.get("parent_chat_id")
|
|
chat, thread = sub.get("chat_id"), sub.get("thread_id") or None
|
|
user_id = sub.get("user_id") or None
|
|
thread_like = bool(thread) or (sub.get("chat_type") or metadata.get("chat_type")) in {
|
|
"thread", "forum", "forum_post", "forum-post", "topic",
|
|
}
|
|
# Preserve canonical route order, including equal-specificity ties. An older
|
|
# row missing an anchor must not skip a potentially winning route. Reuse the
|
|
# route's matcher (including platform-specific identity aliases), not a second
|
|
# hand-maintained equality implementation.
|
|
for route in getattr(config, "profile_routes", None) or []:
|
|
if route.matches(platform.value, guild_id=guild, chat_id=chat,
|
|
thread_id=thread, parent_chat_id=parent, user_id=user_id):
|
|
if route.profile != profile:
|
|
_warn_unroutable_sub_once(
|
|
sub, platform,
|
|
"kanban notifier: subscription for %s on %s chat %s is stamped with profile %s but a "
|
|
"profile_routes entry pins that chat to profile %s; it will not be delivered. "
|
|
"Re-subscribe with `hermes kanban notify-subscribe ... --notifier-profile %s`.",
|
|
profile, route.profile, route.profile)
|
|
return None
|
|
from gateway.run import _multiplex_profile_homes
|
|
served = {name for name, _home in _multiplex_profile_homes(config)}
|
|
return primary if profile in served else None
|
|
if route.matches(platform.value, guild_id=guild or route.guild_id, chat_id=chat,
|
|
thread_id=thread, parent_chat_id=parent or (route.chat_id if thread_like else None),
|
|
user_id=user_id or route.user_id):
|
|
return None
|
|
# A stateless (api_server) subscription carries a RAW session id, not a routable chat, so no
|
|
# profile_routes entry can anchor it — and a platform-wide api_server route would deny the
|
|
# default profile's own api_server destinations. The shared listener mirrors /p/<profile>/ for
|
|
# every served profile, so the owner's own session store is the proof: authorize exactly the
|
|
# session that lives in the served profile's state.db, never the platform. The default profile
|
|
# keeps the historical fallthrough below (no store read).
|
|
if profile != primary_profile and getattr(platform, "value", platform) == "api_server" \
|
|
and session_owned_by_profile(config, profile, chat):
|
|
return primary
|
|
return primary if profile == primary_profile else None
|
|
|
|
|
|
# --- Collection (runs in a worker thread) ---
|
|
|
|
|
|
class _Collector:
|
|
"""One tick's claim state: which profiles/platforms this gateway serves and the GC gate."""
|
|
|
|
def __init__(self, runner: Any, kb: Any, *, notifier_profile: Optional[str], gc_due: bool, gc_retention_days: int) -> None:
|
|
self.runner = runner
|
|
self.kb = kb
|
|
self.notifier_profile = notifier_profile
|
|
self.gc_due = gc_due
|
|
self.gc_retention_days = gc_retention_days
|
|
self.deliveries: list[dict] = []
|
|
self.include_unowned = runner._owns_kanban_dispatcher_lock()
|
|
self.profile_adapters = getattr(runner, "_profile_adapters", {})
|
|
self.notifier_profiles = {notifier_profile}
|
|
self.notifier_profiles.update(str(p).strip() for p in self.profile_adapters if str(p).strip())
|
|
config = getattr(runner, "config", None)
|
|
if getattr(config, "multiplex_profiles", False):
|
|
self.notifier_profiles.update(
|
|
route.profile for route in config.profile_routes
|
|
if route.enabled and route.platform in _platform_names(runner.adapters)
|
|
)
|
|
# Include every platform any secondary profile has live. This is only a
|
|
# coarse pre-filter; exact destination authorization runs before claim
|
|
# and again at delivery, rewinding if the route or adapter changed.
|
|
self.active_platforms = _platform_names(runner.adapters).union(
|
|
*(_platform_names(m) for m in self.profile_adapters.values()))
|
|
|
|
def collect(self) -> list[dict]:
|
|
if not self.active_platforms:
|
|
logger.debug("kanban notifier: no connected adapters; skipping tick")
|
|
return self.deliveries
|
|
# Poll each resolved DB path once: several slugs can map to one DB when
|
|
# HERMES_KANBAN_DB pins the board path.
|
|
kb = self.kb
|
|
seen_db_paths: set[str] = set()
|
|
for board_meta in _list_boards(kb):
|
|
slug = board_meta.get("slug") or kb.DEFAULT_BOARD
|
|
db_path = board_meta.get("db_path")
|
|
try:
|
|
resolved_db_path = str(Path(db_path).expanduser().resolve()) if db_path else str(kb.kanban_db_path(slug).resolve())
|
|
except Exception:
|
|
resolved_db_path = f"slug:{slug}"
|
|
if resolved_db_path in seen_db_paths:
|
|
logger.debug("kanban notifier: skipping duplicate board slug %s for DB %s", slug, resolved_db_path)
|
|
continue
|
|
seen_db_paths.add(resolved_db_path)
|
|
self.collect_board(slug)
|
|
return self.deliveries
|
|
|
|
def _board_has_subs(self, slug: str) -> bool:
|
|
"""Cheap read-only probe before the writable connect() (schema init, WAL
|
|
sidecars, checkpoints); a probe failure falls back to the writable open."""
|
|
try:
|
|
count = _kbn().count_notify_subs(
|
|
board=slug, notifier_profiles=self.notifier_profiles, include_unowned=self.include_unowned)
|
|
except Exception as exc:
|
|
logger.debug("kanban notifier: read-only subscription probe failed "
|
|
"for board %s (%s); falling back to writable open", slug, exc)
|
|
return True
|
|
if count == 0:
|
|
logger.debug("kanban notifier: board %s has no subscriptions owned by %s; skipping open",
|
|
slug, sorted(self.notifier_profiles))
|
|
return count != 0
|
|
|
|
def _gc_stale_subs(self, conn: Any, slug: str) -> None:
|
|
"""Best-effort stale-sub sweep: a failed sweep never blocks delivery; the next hourly gate retries."""
|
|
try:
|
|
_purged = _kbn().purge_stale_done_notify_subs(conn, max_age_days=self.gc_retention_days)
|
|
if _purged:
|
|
logger.info("kanban notifier: purged %d stale done/blocked-task subscription(s) on board %s (retention %dd)",
|
|
_purged, slug, self.gc_retention_days)
|
|
except Exception as _gc_exc:
|
|
logger.debug("kanban notifier: stale-sub GC failed for board %s: %s", slug, _gc_exc)
|
|
|
|
def _claim_for_sub(self, conn: Any, slug: str, sub: dict) -> Optional[dict]:
|
|
"""Claim one subscription's unseen events; None when skipped or nothing new."""
|
|
owner_profile = sub.get("notifier_profile") or None
|
|
platform = (sub.get("platform") or "").lower()
|
|
if platform not in self.active_platforms:
|
|
logger.debug("kanban notifier: subscription for %s on %s skipped; adapter not connected",
|
|
sub.get("task_id"), platform or "<missing>")
|
|
return None
|
|
from gateway.config import Platform
|
|
if _adapter_for_subscription(self.runner, Platform(platform), sub, owner_profile or self.notifier_profile) is None:
|
|
_warn_anchorless_thread_sub_once(sub, platform)
|
|
return None
|
|
old_cursor, cursor, events = _kbn().claim_unseen_events_for_sub(
|
|
conn, task_id=sub["task_id"], platform=sub["platform"], chat_id=sub["chat_id"],
|
|
thread_id=sub.get("thread_id") or "", kinds=TERMINAL_KINDS,
|
|
)
|
|
if not events:
|
|
return None
|
|
task = self.kb.get_task(conn, sub["task_id"])
|
|
logger.debug("kanban notifier: claimed %d event(s) for %s on board %s cursor %s→%s",
|
|
len(events), sub["task_id"], slug, old_cursor, cursor)
|
|
return {"sub": sub, "old_cursor": old_cursor, "cursor": cursor, "events": events, "task": task, "board": slug}
|
|
|
|
def collect_board(self, slug: str) -> None:
|
|
"""Claim events on one board, appending delivery dicts to ``deliveries``."""
|
|
if not self._board_has_subs(slug):
|
|
return
|
|
kb = self.kb
|
|
try:
|
|
conn = _kbc().connect(board=slug)
|
|
except Exception as exc:
|
|
logger.debug("kanban notifier: cannot open board %s: %s", slug, exc)
|
|
return
|
|
try:
|
|
if self.gc_due:
|
|
self._gc_stale_subs(conn, slug)
|
|
# No explicit init_db(): connect() already runs the migration once per
|
|
# process, and init_db() would re-run it on a second connection racing
|
|
# the first.
|
|
subs = _kbn().list_notify_subs(conn, notifier_profiles=self.notifier_profiles, include_unowned=self.include_unowned)
|
|
if not subs:
|
|
logger.debug("kanban notifier: board %s has no subscriptions", slug)
|
|
for sub in subs:
|
|
try:
|
|
claimed = self._claim_for_sub(conn, slug, sub)
|
|
if claimed is not None:
|
|
self.deliveries.append(claimed)
|
|
except Exception as sub_exc:
|
|
# One bad subscription must not block the rest of the tick.
|
|
logger.warning("kanban notifier: subscription for %s on board %s failed: %s",
|
|
sub.get("task_id"), slug, sub_exc)
|
|
finally:
|
|
conn.close()
|
|
|
|
|
|
def _notifier_collect(runner: Any, kb: Any, *, notifier_profile: Optional[str], gc_due: bool, gc_retention_days: int) -> list[dict]:
|
|
"""Claim unseen terminal events for every owned subscription on every board.
|
|
|
|
Each gateway polls only subscriptions owned by profiles whose adapters it
|
|
hosts; legacy rows without a profile stamp are visible only to the process
|
|
holding the singleton dispatcher lock.
|
|
"""
|
|
return _Collector(
|
|
runner, kb, notifier_profile=notifier_profile, gc_due=gc_due, gc_retention_days=gc_retention_days,
|
|
).collect()
|
|
|
|
|
|
# --- Per-event message formatting: kind -> (msg, wake_handoff, wake_review_detail) ---
|
|
# ``None`` for handoff / review_detail leaves the accumulated wake value untouched.
|
|
|
|
|
|
def _payload(ev: Any, key: str) -> Any:
|
|
"""Shared "payload present and truthy" read."""
|
|
return ev.payload.get(key) if ev.payload and ev.payload.get(key) else None
|
|
|
|
|
|
def _clip(ev: Any, key: str, msg_key: str, limit: int) -> str:
|
|
"""Catalog message ``msg_key`` (``{value}`` placeholder) rendered with the truncated payload
|
|
value, or ``""`` when absent."""
|
|
value = _payload(ev, key)
|
|
return t(msg_key, value=str(value)[:limit]) if value else ""
|
|
|
|
|
|
_NL = "\n{}"
|
|
|
|
|
|
def _first_line(text: str, limit: int) -> str:
|
|
lines = text.strip().splitlines()
|
|
return lines[0][:limit] if lines else text[:limit]
|
|
|
|
|
|
def _fmt_completed(ev, n) -> tuple:
|
|
# Prefer the run summary from the event payload; fall back to task.result for legacy rows.
|
|
wake_handoff = None
|
|
payload_summary = _payload(ev, "summary")
|
|
if payload_summary:
|
|
wake_handoff = _first_line(str(payload_summary), 200)
|
|
elif n.task and n.task.result:
|
|
wake_handoff = _first_line(n.task.result, 160)
|
|
handoff = f"\n{wake_handoff}" if wake_handoff is not None else ""
|
|
return t("gateway.kanban.ping.completed", head=n.head, title=n.title, handoff=handoff), wake_handoff, None
|
|
|
|
|
|
def _fmt_review_requested(ev, n) -> tuple:
|
|
# Implementation done; task moved to the review lane. Carry the handoff
|
|
# into the wake turn like ``completed`` so the reviewer needn't re-read the board.
|
|
handoff = ""
|
|
wake_handoff = None
|
|
summary = _payload(ev, "summary")
|
|
if summary:
|
|
summary = str(summary)
|
|
handoff = f"\n{summary[:200]}"
|
|
wake_handoff = _first_line(summary, 200)
|
|
return t("gateway.kanban.ping.review_requested", head=n.head, title=n.title, handoff=handoff), wake_handoff, None
|
|
|
|
|
|
def _fmt_changes_requested(ev, n) -> tuple:
|
|
payload = ev.payload or {}
|
|
reason = _safe_review_reason(payload.get("reason"))
|
|
reviewer = _safe_review_reason(payload.get("reviewer"), 48)
|
|
implementer = _safe_review_reason(payload.get("implementer"), 48)
|
|
reason_text = reason or t("gateway.kanban.ping.changes_default_reason")
|
|
provenance = t("gateway.kanban.ping.reviewer_suffix", reviewer=reviewer) if reviewer else ""
|
|
if implementer:
|
|
provenance += t("gateway.kanban.ping.implementer_suffix", implementer=implementer)
|
|
msg = t("gateway.kanban.ping.changes_requested",
|
|
board_tag=n.board_tag, task_id=n.task_id, reason=reason_text, provenance=provenance)
|
|
return msg, None, reason_text
|
|
|
|
|
|
def _fmt_block_loop_detected(ev, n) -> tuple:
|
|
"""Re-blocked for the same cause past the limit and routed to `triage`.
|
|
|
|
It emits no blocked/status event, so ping loudly here. A repeated-block
|
|
circuit breaker establishes that orchestration attention is needed; it
|
|
does NOT establish that a human decision or owner input exists. Use
|
|
neutral orchestration wording unless the block was typed as a genuine
|
|
owner-input request (`needs_input`, the only kind that carries a concrete
|
|
question for the owner).
|
|
"""
|
|
kind = _payload(ev, "kind")
|
|
decision = kind == "needs_input"
|
|
msg = t(
|
|
"gateway.kanban.ping.triage", head=n.head,
|
|
why=t("gateway.kanban.ping.triage_decision" if decision else "gateway.kanban.ping.triage_attention"),
|
|
recurrences=_clip(ev, "recurrences", "gateway.kanban.ping.triage_recurrences", 200),
|
|
reason=_clip(ev, "reason", "gateway.kanban.ping.reason_suffix", 160),
|
|
)
|
|
return msg, None, None
|
|
|
|
|
|
def _fmt_gave_up(ev, n) -> tuple:
|
|
# The dispatcher auto-blocked the task after ``failures`` consecutive non-success attempts
|
|
# (spawn failure, crash, or timeout alike): it is now Blocked and waiting for a human.
|
|
failures = _payload(ev, "failures")
|
|
count = (t("gateway.kanban.ping.failed_n_times", count=int(failures)) if failures
|
|
else t("gateway.kanban.ping.kept_failing"))
|
|
last = _clip(ev, "error", "gateway.kanban.ping.last_error", 160)
|
|
return t("gateway.kanban.ping.gave_up", head=n.head, count=count, last=last, task_id=n.task_id), None, None
|
|
|
|
|
|
def _fmt_timed_out(ev, n) -> tuple:
|
|
limit = int(_payload(ev, "limit_seconds") or 0)
|
|
minutes = max(1, round(limit / 60)) if limit else 0
|
|
span = t("gateway.kanban.ping.limit_minutes", minutes=minutes) if minutes else t("gateway.kanban.ping.limit_generic")
|
|
return t("gateway.kanban.ping.timed_out", head=n.head, span=span), None, None
|
|
|
|
|
|
# archived / unblocked are claimed (so the cursor advances past them) but
|
|
# intentionally silent (no formatter), and excluded from _WAKE_KINDS so they
|
|
# never wake the creator.
|
|
_EVENT_FORMATTERS: dict[str, Callable[[Any, "_KanbanNotification"], tuple]] = {
|
|
"completed": _fmt_completed,
|
|
"blocked": lambda ev, n: (
|
|
t("gateway.kanban.ping.blocked", head=n.head, reason=_clip(ev, "reason", "gateway.kanban.ping.reason_suffix", 160)),
|
|
None, None,
|
|
),
|
|
"gave_up": _fmt_gave_up,
|
|
"crashed": lambda ev, n: (t("gateway.kanban.ping.crashed", head=n.head), None, None),
|
|
"timed_out": _fmt_timed_out,
|
|
"status": lambda ev, n: (t("gateway.kanban.ping.status", head=n.head, status=_payload(ev, "status") or ""), None, None),
|
|
"review_requested": _fmt_review_requested,
|
|
"changes_requested": _fmt_changes_requested,
|
|
"block_loop_detected": _fmt_block_loop_detected,
|
|
}
|
|
|
|
|
|
# --- Delivery of one claimed batch (one subscription, N events) ---
|
|
|
|
|
|
class _KanbanNotification:
|
|
"""Deliver one subscription's claimed events, then settle the cursor.
|
|
|
|
Both legs of notify+wake must succeed before settling. Sent pings have a
|
|
separate durable checkpoint so wake retries do not resend them. Admission
|
|
is at-least-once queueing, not an execution or final-response receipt.
|
|
"""
|
|
|
|
def __init__(self, runner: Any, d: dict, *, platform_cls: Any, sub_fail_counts: dict) -> None:
|
|
self.runner = runner
|
|
self.d = d
|
|
self.platform_cls = platform_cls
|
|
self.sub_fail_counts = sub_fail_counts
|
|
self.sub = sub = d["sub"]
|
|
self.task = task = d["task"]
|
|
self.board_slug = d.get("board")
|
|
self.platform_str = (sub["platform"] or "").lower()
|
|
self.task_id = sub["task_id"]
|
|
self.sub_profile = sub.get("notifier_profile") or ""
|
|
self.title = (task.title if task else sub["task_id"])[:120]
|
|
self.board_tag = t("gateway.kanban.ping.board_tag", board=self.board_slug) if self.board_slug else ""
|
|
# Attribute the ping to the worker that did the work.
|
|
tag = t("gateway.kanban.ping.assignee_tag", assignee=task.assignee) if task and task.assignee else ""
|
|
self.head = t("gateway.kanban.ping.head", board_tag=self.board_tag, assignee_tag=tag, task_id=self.task_id)
|
|
# The wake self-post path needs the key even when every event was skipped.
|
|
self.sub_key = (sub["task_id"], sub["platform"], sub["chat_id"], sub.get("thread_id") or "")
|
|
mode = sub.get("delivery_mode") or "notify"
|
|
self.wake_agent = mode in ("notify+wake", "wake")
|
|
self.send_passive = mode != "wake"
|
|
# Worker handoff carried into the synthetic wake turn so the woken
|
|
# creator doesn't re-decompose work already on the board.
|
|
self.wake_handoff = self.wake_review_detail = self.session_key = self.synth = ""
|
|
self.plat: Any = None
|
|
self.adapter: Any = None
|
|
self.is_push_adapter = True
|
|
self.wake_kinds: set = set()
|
|
|
|
# -- cursor / subscription ops (blocking, run in a fresh-context thread) --
|
|
|
|
async def rewind(self) -> None:
|
|
await _to_thread_process_service(
|
|
self.runner._kanban_rewind, self.sub, self.d["cursor"], self.d.get("old_cursor", 0), self.board_slug,
|
|
)
|
|
|
|
async def advance(self) -> None:
|
|
await _to_thread_process_service(self.runner._kanban_advance, self.sub, self.d["cursor"], self.board_slug)
|
|
|
|
async def unsub(self) -> None:
|
|
await _to_thread_process_service(self.runner._kanban_unsub, self.sub, self.board_slug)
|
|
|
|
def clear_failures(self) -> None:
|
|
self.sub_fail_counts.pop(self.sub_key, None)
|
|
|
|
async def delivery_failed(self, fmt: str, prefix: tuple, drop_fmt: str, exc: Exception, exc_info: bool) -> None:
|
|
"""Bump the failure counter; drop the sub past the limit, else rewind the claim so the next tick retries."""
|
|
fails = self.sub_fail_counts.get(self.sub_key, 0) + 1
|
|
self.sub_fail_counts[self.sub_key] = fails
|
|
logger.warning(fmt, *prefix, fails, MAX_SEND_FAILURES, exc, exc_info=exc_info)
|
|
if fails >= MAX_SEND_FAILURES:
|
|
logger.warning(drop_fmt, self.task_id, self.platform_str, fails)
|
|
await self.unsub()
|
|
self.clear_failures()
|
|
else:
|
|
await self.rewind()
|
|
|
|
async def _wake_failed(self, fmt: str, exc: Exception) -> None:
|
|
drop_fmt = "kanban notifier: dropping subscription %s on %s after %d consecutive wake failures"
|
|
await self.delivery_failed(fmt, (self.task_id,), drop_fmt, exc, True)
|
|
|
|
# -- formatting --
|
|
|
|
def format_event(self, ev: Any) -> Optional[str]:
|
|
"""Render one event; accumulates wake handoff/review detail. None → silent kind."""
|
|
formatter = _EVENT_FORMATTERS.get(ev.kind)
|
|
if formatter is None:
|
|
return None
|
|
msg, handoff, review_detail = formatter(ev, self)
|
|
if handoff is not None:
|
|
self.wake_handoff = handoff
|
|
if review_detail is not None:
|
|
self.wake_review_detail = review_detail
|
|
return msg
|
|
|
|
def build_wake_text(self) -> None:
|
|
"""Set ``wake_kinds`` / ``session_key`` / ``synth`` for the wake paths."""
|
|
task, sub = self.task, self.sub
|
|
self.wake_kinds = {ev.kind for ev in self.d["events"] if ev.kind in _WAKE_KINDS} if self.wake_agent else set()
|
|
self.wake_diagnostic = all(diagnostic_event(ev) for ev in self.d["events"] if ev.kind in self.wake_kinds)
|
|
if not self.wake_kinds:
|
|
return
|
|
if self.is_push_adapter:
|
|
self.session_key = getattr(task, "session_id", None) or ""
|
|
else:
|
|
# Non-push wakes target sub["chat_id"] (the raw session id the
|
|
# subscriber registered). task.session_id may be a WORKER session
|
|
# for child tasks; use it only for legacy rows.
|
|
self.session_key = sub["chat_id"] or getattr(task, "session_id", None) or ""
|
|
# i18n keys: gateway.kanban.wake.<kind> for each _WAKE_KINDS entry.
|
|
_parts = [t(f"gateway.kanban.wake.{k}") for k in _WAKE_KINDS if k in self.wake_kinds]
|
|
_status = t("gateway.kanban.wake.status_joiner").join(_parts) or t("gateway.kanban.wake.status_default")
|
|
synth = t(
|
|
"gateway.kanban.wake.message",
|
|
task_id=sub["task_id"], status=_status, title=self.title,
|
|
assignee=task.assignee if task else "", board=self.board_slug,
|
|
)
|
|
# Label as an automatic notification and carry the handoff so the
|
|
# creator inspects the board instead of re-decomposing.
|
|
if self.wake_handoff:
|
|
synth += "\n" + t("gateway.kanban.wake.handoff", summary=self.wake_handoff)
|
|
if self.wake_review_detail:
|
|
synth += "\n" + t("gateway.kanban.wake.review_detail", reason=self.wake_review_detail)
|
|
self.synth = synth + "\n\n" + t("gateway.kanban.wake.guidance")
|
|
|
|
def _log_woke(self) -> None:
|
|
logger.info("kanban notifier: woke agent for %s on %s/%s profile=%s events=%s",
|
|
self.task_id, self.platform_str, self.sub["chat_id"], self.sub_profile or "default", self.wake_kinds)
|
|
|
|
def _served_wake_profile(self) -> Optional[str]:
|
|
"""The subscription's profile when THIS gateway is a multiplexer serving it, else ``None``.
|
|
|
|
``None`` keeps the historical path: a standalone ``hermes -p <name>`` gateway owns its own
|
|
listener and key, so its api_server wakes keep using the HTTP self-post.
|
|
"""
|
|
if not self.sub_profile:
|
|
return None
|
|
if not getattr(getattr(self.runner, "config", None), "multiplex_profiles", False):
|
|
return None
|
|
return self.sub_profile
|
|
|
|
def _owner_scope(self):
|
|
"""Runtime scope of the subscription's profile under multiplex, else a no-op context."""
|
|
runner = self.runner
|
|
served_profile = self._served_wake_profile()
|
|
if not served_profile:
|
|
return contextlib.nullcontext()
|
|
from gateway.run import _async_profile_runtime_scope
|
|
from gateway.session import SessionSource
|
|
source = SessionSource(platform=self.plat, chat_id=self.sub["chat_id"], profile=served_profile)
|
|
return _async_profile_runtime_scope(runner._resolve_profile_home_for_source(source))
|
|
|
|
async def wake(self) -> None:
|
|
"""Wake the creator session (raises on failure): push adapters get a full SessionSource, non-push a raw self-post."""
|
|
from gateway.wake import deliver_wake
|
|
sub = self.sub
|
|
if not self.is_push_adapter:
|
|
# A served profile's raw-session wake runs in THAT profile's scope, in-process: the
|
|
# shared listener's /p/<profile>/ self-post would need the profile's own
|
|
# API_SERVER_KEY, which a route-only profile legitimately does not have, and an
|
|
# unprefixed self-post would resume the session in the DEFAULT profile's store.
|
|
async with self._owner_scope():
|
|
await deliver_wake(self.adapter, text=self.synth, session_id=self.session_key,
|
|
profile=self._served_wake_profile(),
|
|
notification_category="diagnostic" if self.wake_diagnostic else "result")
|
|
self._log_woke()
|
|
return
|
|
from gateway.session import SessionSource
|
|
# Rebuild the creator's real session scope from the persisted chat_type:
|
|
# build_session_key() keys DMs differently from group/thread, so a
|
|
# hardcoded "group" mis-routed DM/thread creators into a fresh session.
|
|
# Legacy rows may carry chat_type in delivery_metadata; last resort is
|
|
# "group". A mismatch only degrades to a fresh session.
|
|
# Legacy rows written before the column existed may still carry chat_type in delivery_metadata
|
|
# (#60600 rows) — fall back to that, then to "group" (the historical default that suits the
|
|
# dashboard/group flows). handle_message() get_or_create_session's the target, so a mismatch only
|
|
# ever degrades to a fresh session, never an exception.
|
|
_delivery_meta = sub.get("delivery_metadata") or {}
|
|
_chat_type = str(sub.get("chat_type") or _delivery_meta.get("chat_type") or "").strip()
|
|
_source = SessionSource(
|
|
platform=self.plat, chat_id=sub["chat_id"], chat_type=_chat_type or "group",
|
|
thread_id=sub.get("thread_id") or None, user_id=sub.get("user_id"), user_id_alt=sub.get("user_id_alt"),
|
|
profile=self.sub_profile or None, scope_id=_wake_scope_id(self.adapter, sub),
|
|
parent_chat_id=_delivery_meta.get("parent_chat_id"),
|
|
)
|
|
_source._transport_adapter_ref = weakref.ref(self.adapter)
|
|
from gateway.run import _async_profile_runtime_scope
|
|
if self.sub_profile and getattr(getattr(self.runner, "config", None), "multiplex_profiles", False):
|
|
from hermes_cli.profiles import profile_exists
|
|
if not profile_exists(self.sub_profile):
|
|
raise RuntimeError(f"Kanban wake profile {self.sub_profile!r} no longer exists")
|
|
async with _async_profile_runtime_scope(self.runner._resolve_profile_home_for_source(_source)):
|
|
await deliver_wake(self.adapter, text=self.synth, session_id=self.session_key, source=_source,
|
|
notification_category="diagnostic" if self.wake_diagnostic else "result")
|
|
self._log_woke()
|
|
|
|
async def _send_event(self, ev: Any, msg: str) -> bool:
|
|
"""Send one text ping; raises on adapter exception or SendResult(success=False)."""
|
|
from gateway.warning_notifications import present_notification
|
|
sub, adapter = self.sub, self.adapter
|
|
delivery_metadata = sub.get("delivery_metadata")
|
|
metadata: dict[str, Any] = dict(delivery_metadata) if isinstance(delivery_metadata, dict) else {}
|
|
if sub.get("thread_id") and not metadata.get("thread_id"):
|
|
metadata["thread_id"] = sub["thread_id"]
|
|
_send_res = None
|
|
async def send_ping():
|
|
nonlocal _send_res
|
|
_send_res = await adapter.send(sub["chat_id"], msg, metadata=metadata)
|
|
if not await present_notification(send_ping, platform=self.platform_str, diagnostic=diagnostic_event(ev)):
|
|
return False
|
|
# SendResult(success=False) without an exception is a FAILED delivery
|
|
# (else the event is lost); None / non-SendResult keeps the
|
|
# "no exception == delivered" contract.
|
|
if getattr(_send_res, "success", True) is False:
|
|
raise RuntimeError(f"adapter send() reported failure: {getattr(_send_res, 'error', None) or 'unknown error'}")
|
|
logger.debug("kanban notifier: delivered %s event for %s to %s/%s on board %s",
|
|
ev.kind, self.task_id, self.platform_str, sub["chat_id"], self.board_slug)
|
|
# Upload artifact paths from the handoff payload / legacy result as
|
|
# native files. Both handoff kinds stage files for exactly this: a
|
|
# review-bound card's files exist precisely so the human sees them at
|
|
# handoff time. Retry exposure matches ``completed`` (the sub cursor is
|
|
# rewound only when a send failed).
|
|
if ev.kind in ("completed", "review_requested"):
|
|
try:
|
|
await self.runner._deliver_kanban_artifacts(
|
|
adapter=adapter, chat_id=sub["chat_id"], metadata=metadata,
|
|
event_payload=getattr(ev, "payload", None), task=self.task,
|
|
)
|
|
except Exception as art_exc:
|
|
logger.debug("kanban notifier: artifact delivery for %s failed: %s", self.task_id, art_exc)
|
|
return True
|
|
|
|
async def _send_pings(self) -> bool:
|
|
"""Send every text ping; False when a send failed (claim already rewound/dropped)."""
|
|
for ev in self.d["events"]:
|
|
msg = self.format_event(ev)
|
|
if msg is None:
|
|
continue
|
|
# Non-push adapters (api_server) always report SendResult(success=False)
|
|
# from send(); treating that as failure would drop the sub forever and
|
|
# make the wake path unreachable. Skip the doomed send; the self-post
|
|
# IS the delivery and resolves the failure counter.
|
|
if not self.is_push_adapter and self.wake_agent:
|
|
logger.debug(
|
|
"kanban notifier: adapter %s has no push channel; skipping text ping for %s, relying "
|
|
"on wake self-post instead", self.platform_str, self.task_id,
|
|
)
|
|
continue
|
|
if not self.send_passive:
|
|
# Wake-only: the wake path is the sole delivery and resolves the counter.
|
|
continue
|
|
if ev.id <= self.sub.get("last_ping_event_id", 0):
|
|
continue
|
|
try:
|
|
if await self._send_event(ev, msg) is False:
|
|
continue
|
|
await _to_thread_process_service(partial(
|
|
self.runner._kanban_sub_op, self.board_slug, "record_notify_ping", self.sub,
|
|
event_id=ev.id,
|
|
))
|
|
self.clear_failures()
|
|
except Exception as exc:
|
|
await self.delivery_failed(
|
|
"kanban notifier: send failed for %s on %s (attempt %d/%d): %s", (self.task_id, self.platform_str),
|
|
"kanban notifier: dropping subscription %s on %s after %d consecutive send failures", exc, False,
|
|
)
|
|
return False
|
|
return True
|
|
|
|
async def deliver(self) -> None:
|
|
try:
|
|
self.plat = self.platform_cls(self.platform_str)
|
|
except ValueError:
|
|
await self.advance()
|
|
return
|
|
# Recheck the exact route after claiming: config/adapters can change between ticks. The
|
|
# recheck reads the served profile's session store for a stateless destination, so it runs
|
|
# off the event loop (the claim path already collects in a worker thread).
|
|
adapter = await asyncio.to_thread(
|
|
_adapter_for_subscription, self.runner, self.plat, self.sub, self.sub_profile or None)
|
|
if adapter is None:
|
|
logger.debug("kanban notifier: adapter %s disconnected before delivery for %s; rewinding claim",
|
|
self.platform_str, self.task_id)
|
|
await self.rewind()
|
|
return
|
|
self.adapter = adapter
|
|
from gateway.wake import adapter_supports_push
|
|
self.is_push_adapter = adapter_supports_push(adapter)
|
|
|
|
# Pings, artifact uploads (media policy) and the wake text (display.language) all read the
|
|
# SUBSCRIBER profile's config; the notifier thread itself runs in the launch profile's scope.
|
|
async with self._owner_scope():
|
|
if not await self._send_pings():
|
|
return
|
|
# All text pings delivered (or skipped for non-push / wake-only).
|
|
original_events = self.d["events"]
|
|
from gateway.warning_notifications import warning_notifications_enabled
|
|
split = not warning_notifications_enabled(self.platform_str)
|
|
wake_groups = ([original_events] if not split else [
|
|
[ev for ev in original_events if diagnostic_event(ev)],
|
|
[ev for ev in original_events if not diagnostic_event(ev)],
|
|
])
|
|
wake_payloads = []
|
|
for events in wake_groups:
|
|
if not events:
|
|
continue
|
|
self.d = {**self.d, "events": events}
|
|
self.wake_handoff = self.wake_review_detail = ""
|
|
for ev in events:
|
|
self.format_event(ev)
|
|
self.build_wake_text()
|
|
if self.wake_kinds:
|
|
wake_payloads.append((self.synth, self.wake_diagnostic, self.wake_kinds))
|
|
self.d = {**self.d, "events": original_events}
|
|
wake_kinds, is_push = self.wake_kinds, self.is_push_adapter
|
|
from gateway.wake import WakeNotAccepted
|
|
|
|
# A requested wake is required even when its passive ping already landed.
|
|
if wake_payloads:
|
|
try:
|
|
for self.synth, self.wake_diagnostic, self.wake_kinds in wake_payloads:
|
|
await self.wake()
|
|
self.clear_failures()
|
|
except WakeNotAccepted:
|
|
# Startup / full queue is not a dead destination. Keep the durable
|
|
# subscription alive regardless of how long admission takes.
|
|
await self.rewind()
|
|
return
|
|
except Exception as _wk_err:
|
|
await self._wake_failed(
|
|
"kanban notifier: wake-only delivery failed for %s (attempt %d/%d): %s" if is_push
|
|
else "kanban notifier: wake self-post failed for %s (attempt %d/%d): %s",
|
|
_wk_err,
|
|
)
|
|
return
|
|
|
|
# Delivery complete: advance the cursor (the dedup mechanism).
|
|
await self.advance()
|
|
if not is_push:
|
|
self.clear_failures()
|
|
# Unsubscribe only on archive; ``done`` is reversible.
|
|
if self.task and self.task.status == "archived":
|
|
await self.unsub()
|