hermes-agent/gateway/platforms/signal.py
Teknium 9bcbe7b5df
feat(i18n): pluggable, layered language packs across core, Desktop and TUI (#126296)
* feat(i18n): layered catalogs — plugin packs and user overlay over bundled locales

* feat(tui): i18n layer — en catalog, nanostore runtime, RPC pack loader, _keys.tui.json emitter

ui-tui/src/i18n/: en.ts (facade over topical siblings under en/), types.ts
(Translations + dotted TranslationKey derived from en), runtime.ts ($locale/
$catalog atoms, translateFrom active→en→key, pack merge with string→fn
wrapping for {0}/{1} placeholders), loader.ts (display.language →
i18n.catalog {lang, surface:'tui'}, English when the method is missing),
useT()/useLocale() hooks, t() for non-React code. useConfigSync feeds the
loader from the existing config.get full hydration. `npm run i18n:keys`
writes locales/_keys.tui.json (sorted flat key list) and runs before build.

* feat(plugins): provides_locales manifest field, ctx.register_locale/register_locale_dir, manifest-only language packs

* chore(tui): split en catalog siblings by lane (slash sibling)

* feat(plugins): validate language packs — parse, text-only, key-subset WARN against en / _keys exports

* feat(tui_gateway): i18n.languages / i18n.catalog RPC + regenerated contracts

* feat(config): display.language accepts any supported_languages() id, refuses unknown ids with the list

* docs(i18n): language packs user guide, pluggable display.language, plugin developer section, AGENTS notes

* feat(plugins): report language-pack layers in the mid-run activation summary

* feat(tui): wire status bar, composer placeholders, hotkey help and approval/clarify/confirm prompts through i18n

StatusRule maps compared state values (ready/running…/summoning) to catalog
text at render via displayStatus(); hotkeys()/placeholder() resolve lazily so
a pack that arrives after boot applies. Catalog grows to 81 keys.

* feat(desktop): pluggable app locales — registry, host.i18n.registerAppLocale, backend packs, keys emitter

- Locale widens to string (BundledLocale keeps the union); TRANSLATIONS stays
  the bundled record and every consumer resolves through the registry.
- src/i18n/registry.ts: registerAppLocale(id, {endonym, rtl, translations})
  layers partial packs (nested or flat dotted) over bundled/en via
  mergeTranslations; a string over a function-valued en entry becomes a
  positional {0}/{1} formatter; $appLocaleVersion bumps so translators
  re-render; per-source disposers + replaceAppLocaleSource for atomic swaps.
- Backend packs: i18n.languages + i18n.catalog {surface:'desktop'} feed the
  registry as source 'backend' (method-not-found is silent); re-synced on
  socket open, display.language change and profile switch. A saved pack-only
  language is promoted once its pack registers.
- SDK: host.i18n.registerAppLocale / languageOptions; ctx.i18n.registerAppLocale
  tracked for unload. Docs in the desktop plugin SDK guide + skill reference.
- Language switcher lists bundled ∪ registered ∪ backend, endonym-only; RTL
  from the registry (applyDocumentLocale takes rtl).
- npm run i18n:keys emits locales/_keys.desktop.json (wired into build).

* i18n(cli): route /topup + /subscription copy through t() (cli.billing.*, cli.subscription.*)

Module-level copy tables and modal choice tuples in cli_billing_mixin.py froze
English at import, before display.language was known. They are now key tables /
builder functions evaluated at call time; every user-facing line in the /usage
balance block, /subscription and the five /topup screens reads the catalog.
Choice VALUES stay English identifiers. Fragment-assembled status lines
(Plan: … → cancels · $x left · renews …) become full templates.

* i18n(gateway): exec-approval card contract + base/run/run_busy/run_inbound replies through t()

- base_exec_approval: EA_* English constants stay; add ea_header_text()/ea_reason_label_text()/
  ea_smart_deny_line_text()/ea_default_reason_text()/ea_action_labels()/approval_timed_out_notice()
  accessors; deadline + timed-out notice resolve via gateway.exec_approval.*
- BasePlatformAdapter._EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/_EA_ACTION_LABELS become
  properties (adapters still shadow them with markup class attrs)
- run.py: provider error replies table holds catalog keys; _CONTEXT_OVERFLOW_REPLY -> _context_overflow_reply()
- run_busy/run_inbound: typed approval + slash-confirm matchers accept English ∪ approval.inputs.* (t())
- locales/en.yaml: gateway.exec_approval/busy/errors/... namespaces

* i18n(cli): wire modal, loops, agent-setup mixins through t() (cli.* keys)

* i18n(platforms): route Slack, Matrix and Feishu user-facing text through t()

Exec-approval markup overrides (_EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/
_EA_ACTION_LABELS) become per-call properties over the shared
gateway.exec_approval.* contract keys, so Slack's 3000-char section budget
measures the resolved template. Slack _APPROVAL_DECISIONS/_CONFIRM_DECISIONS,
Feishu _APPROVAL_LABEL_MAP and Matrix _EA_LEGEND/_EA_TYPED_HINT turn into
key tables resolved at click time; the Matrix typed hints become whole
sentences per offered tier instead of spliced fragments. Slack button labels
are cut to 75 chars and select placeholders to 150 after translation; the
model-facing clarify fallback answer ('choice N') stays English while the
card copy localizes.

locales/en.yaml gains the gateway.exec_approval.* contract keys plus the
platform.shared.* / platform.slack.* / platform.matrix.* / platform.feishu.*
namespaces (and the keys for the other adapters wired in follow-up commits).

* i18n(gateway): run_turn / run_turn_runner / approval-settle copy through t()

- status hints, proxy errors, background task notices, progress heartbeats, session info lines
- tool progress chrome (tool_head/tool_pending/tool_preview/tool_verbose) shared by base.format_tool_event
- run_turn_runner:1406 Chinese clarify placeholder -> gateway.clarify.native_stream_placeholder (zh text kept in zh.yaml)
- _UNEXPECTED_SILENCE_REPLY/_CLARIFY_EXPIRED_NOTICE -> accessor functions

* i18n(platforms): route Google Chat and Teams user-facing text through t()

Google Chat clarify card, typing placeholder, orphan-card labels and the whole
/setup-files reply set (module constants become platform.google_chat.setup_files.*
keys resolved at reply time). The attachment-fallback notice that shipped
hardcoded in Spanish is keyed with an English en value; es.yaml carries the
original Spanish text for those four keys.

Teams approval card header/reason use the gateway.exec_approval.* contract,
_APPROVAL_LABELS becomes a key table resolved at click time, and the meeting
summary writer resolves its section headings/fallbacks per render.

* i18n(platforms): route LINE, WeCom, email, DingTalk, IRC and Home Assistant text through t()

LINE default copy constants become catalog keys resolved in __init__ (the
LINE_*_TEXT / extra.* operator overrides still win); the busy-ack bypass
matcher keys on the leading emoji marker only, so it keeps firing once the
gateway busy heads are localized. WeCom media size/format notices that shipped
hardcoded in Chinese are keyed with English en values and zh.yaml carries the
original Chinese text. DingTalk emotion bubbles resolve per send.

* i18n(cli): route /model switch output and -q status lines through t() (cli.model.*, cli.single_query.*)

Switch-summary labels shared with the gateway reuse gateway.model.* keys
(provider/context/max-output/capabilities/prompt-caching); CLI-only variants
(glyph or no-backtick forms) live under cli.model.*. The hand-padded /model usage
block becomes a (form, description-key) table padded at render time so the
command syntax stays fixed while descriptions translate. -q 'Error:' reuses
gateway.model.error_prefix.

* i18n(cli): route TUI panel/hint/placeholder copy through t() (cli.tui.*)

_APPROVAL_CHOICE_LABELS and _TUI_MODAL_HINTS become key tables resolved at
render time; vault/sudo panel bodies are one catalog value per panel split on
newline; inline plurals use <key>_one/<key>_other. Adds the cli.* namespace
(shared/tui/voice/render/subagents/dock) to locales/en.yaml.

* i18n(cli): voice/wake-word CLI copy through t() (cli.voice.*)

RuntimeError texts raised in _voice_start_recording are human copy (callers
print {e}) and are keyed; the Termux requirement-check match stays English.
Wake state ids stay internal, only their labels localize.

* i18n(cli): live-work dock, subagent monitor and render copy through t()

cli.subagents.* / cli.dock.* / cli.render.*; count fragments pluralize via
_one/_other keys, verdict table holds keys resolved at paint time so width
clipping measures the translated text.

* i18n(gateway): unauthorized/pairing, voice, topics, shutdown, startup, notifications, kanban pings through t()

* i18n(cli): move tips + composer placeholders into the catalog (tips.tNNN / tips.placeholder.pNN)

get_random_tip()/get_random_composer_placeholder() pick a key from the English catalog
(the parity baseline, probed once per process) and resolve it through t() for the active
language, so language packs translate tips like any other string. Also lands the cli.*
en.yaml namespace consumed by the CLI info/help/error-copy wiring in the next commit.

* i18n(cli): wire chat-turn + session mixins through t(); kanban log trimmer matches t() output

* test(cli): assert TUI/dock/voice copy via t(key); prove labels resolve at render time

Pinned-English assertions in the approval-UI, live-work dock and voice tests now
go through the catalog. New test swaps the catalog after import and checks the
approval panel + hint row follow it (the reason _APPROVAL_CHOICE_LABELS and
_TUI_MODAL_HINTS became key tables).

* i18n(cli): route CLI info/help/error copy through t() (cli.* namespace)

cli_info_mixin: /help consumes CommandDef.describe() (added to commands.py: slash.<name>.description
with fallback to .description), section titles/skill/quick-command headers, /tools, /toolsets,
/usage labels, /context, /whoami, /insights, /gateway status, tool-progress labels, bang-shell
denials, MCP config-watch + /reload-mcp confirm/reload lines, /reload-skills, and the session-store
warning all read the catalog at call time (module-level label tables became functions so the
active language is honoured after startup). cli.py: worktree cleanup, tirith warning, show_config
(labels re-padded at print time), quick/plugin/skill slash-command errors, ambiguous-command hint,
stdin error, gateway start, profile warning. cli_chat_error_copy / cli_unknown_command /
cli_output / cli_init_mixin: chat error panel copy, did-you-mean lines, n-more / yes-no prompt
(localized affirmative initial alongside 'y'), unknown-toolsets warning.

* i18n(w1a): wire agent display/explainers/approval + slash registry/help through t()

- hermes_cli/commands.py: CommandDef.describe() resolves slash.<name>.description
  at call time; category labels via slash.category.*; help/alias/usage suffixes
  via slash.shared.*; gateway_help_lines and commands_platforms/slash_exec use them.
- agent/display.py: display.verb.* resolved at call time via get_tool_verb();
  bridge/spinner/thinking-verb/cute-row/failure/preview/diff text via display.*.
- agent/turn_explainers.py: exit-reason / persistence-cause tables become call-time
  lookups (explainer.exit.*, explainer.persistence.*, explainer.file_mutation.*).
- agent/background_review.py, session_activity.py, context_breakdown.py,
  status_output.py: review summaries, iteration progress, context notices.
- tools/approval.py, approval_context.py: approval.summary.*, approval.noun.*,
  approval.window.* pluralized keys.
- gateway/slash_commands*.py: remaining raw strings (busy, whoami, platform,
  bundles, memory, skills, approvals, set_home, diff, update, debug, profile,
  heartbeat, refine, review, subgoal, loop, retry, compress codex path, save,
  sessions, model guard/errors, agents rows, topup, login). HISTORY_UNREADABLE
  keeps its English constant; callers use history_unreadable() ->
  gateway.shared.history_unreadable.
- locales/en.yaml: new approval/display/explainer/slash blocks + gateway leftovers.

* i18n(telegram): route adapter chat copy through t()

Approval card (header/reason/smart-deny as HTML-escaped properties), inline
button labels, callback toasts (cut at Telegram's 200-char cap), model/choice
pickers, clarify/update/slash-confirm prompts, gmail-triage labels and the
inbound-media failure notice now come from the catalog. _UNAUTHORIZED is a
lazy _unauthorized() so the import no longer binds a language. The command
menu carries a language+payload fingerprint (forum scopes re-register on
change) and BotCommand descriptions are cut at 256.

Adds gateway.exec_approval.* (WAVE2 contract), platform.telegram.*,
platform.discord.* and the slash.*.description keys the Discord table shares
with the CLI registry to locales/en.yaml.

* i18n(gateway/platforms): whatsapp_cloud, yuanbao, weixin, signal, api_server copy through t()

- whatsapp_cloud: clarify list/buttons, approve/deny + slash-confirm labels via platform.whatsapp.* (t()-then-truncate at 20/24/72 caps); _EA_HEADER becomes a property wrapping ea_header_text()
- yuanbao: SLOW_RESPONSE_MESSAGE -> slow_response_message() (platform.yuanbao.slow_response_notice; zh keeps the original text); cron-wrapper markers centralized as module constants for strip_cron_wrapper
- api_server: PROVIDER_AUTH_FAILED_LABEL/PROVIDER_RATE_LIMITED_LABEL stay English for run.py matchers; user_text() renders via t()
- signal/_format_wait, weixin voice caption, openai_routes transformed notice
- run_turn: second _UNEXPECTED_SILENCE_REPLY consumer -> accessor

* i18n(discord): route adapter chat copy through t()

Native slash-command table becomes _NATIVE_SLASH_COMMAND_SPECS holding catalog
keys; _native_slash_commands() resolves descriptions, parameter descriptions
and Choice names for the active language, each cut at Discord's 100-char cap,
and the app-command sync fingerprint now includes get_language() so a
display.language change re-syncs. Exec-approval card (gateway.exec_approval.*
contract), slash-confirm / clarify / update views, model+choice pickers,
thread creation, forum titles, voice acks, the response-truncation notice,
the unauthorized-slash security alert and the media upload-size notices all
read from platform.discord.*. Decorator-declared button labels are relabelled
in __init__ (80-char cap); embed titles cut at 256, select placeholders at
150, option label/description at 100. _UNAUTHORIZED is a lazy _unauthorized().

* i18n(cli): wire status-bar, stream, terminal mixins + terminal_input through t(); rename kwargs that shadow t(key)

* i18n: wire hermes_cli/cli_commands_mixin.py slash-command copy through t()

- 431 new leaves under cli.commands.<cmd>.* in locales/en.yaml; 12 rows reuse
  existing gateway.* keys (rollback, diff, resume, branch, btw, model, reasoning)
  via a _gt() helper so CLI and gateway replies stay identical.
- Module-level English tables (_BUSY_MODE_*, _REASONING_TOGGLES, _HATCH_PROGRESS,
  _DIFF_LABELS, _LOCAL_ENGINE_LINES) become call-time catalog lookups keyed by id.
- Verb tables (Enabling/Disabling, Paused/Resumed/Triggered, planned/done,
  Updating/Generating) are one full template per variant; plurals use
  <key>_one/<key>_other via _tn(); hand-padded column labels (/snapshot list)
  translate the value and re-pad at the call site.
- Multi-line usage blocks are single catalog values split with _lines().
- Model-facing system notes and DB-stored reasons stay English (EXCLUDED).

* tests: assert /handoff, /worktree, /login CLI copy via t(key) instead of pinned English

* test(i18n): pin Telegram/Discord adapter catalog wiring

Lazy unauthorized notice, exec-approval contract keys, HTML escaping before
Telegram <b> wrapping, 200-char toast / 256-char BotCommand caps, Discord
100-char app-command text and 80-char button caps, and language-bearing
command-menu fingerprints on both platforms.

* i18n: reconcile cli.shared on/off vs enabled/disabled after lane merge

* i18n: describe() in TUI-gateway slash listings; localize TUI exit resume hint

* i18n(tr): translate bundled catalog + tui pack

* i18n(ja): translate bundled catalog + tui pack

* i18n(ko): translate bundled catalog + tui pack

* i18n(zh): translate bundled catalog + tui pack

* i18n(fr): translate bundled catalog + tui pack

* i18n(af): translate bundled catalog + tui pack

* i18n(uk): translate bundled catalog + tui pack

* i18n(ar): translate bundled catalog + tui pack

* i18n(pt): translate bundled catalog + tui pack

* i18n(it): translate bundled catalog + tui pack

* i18n(es): translate bundled catalog + tui pack

* i18n(zh-hant): translate bundled catalog + tui pack

* i18n(ru): translate bundled catalog + tui pack

* i18n(hu): translate bundled catalog + tui pack

* i18n(hu): translate pre-existing English-valued leftovers (kanban wake, /context, /status, fast labels)

* i18n(de): translate bundled catalog + tui pack

* i18n(ga): translate bundled catalog + tui pack

* test(i18n): fixture matches _normalize_lang(lang, home) signature

* i18n(tui): scaffold userMessages/slashCmd en siblings

* i18n(tui): wire secure prompts + content tables

* feat(tui): i18n — wire billing, subscription, connection-setup and journey overlays

Adds en siblings billing.ts / subscription.ts / connection.ts (namespaces
billing, subscription, connection, journey) and routes every user-facing
literal in billingOverlay, subscriptionOverlay, connectionSetupOverlay and
journey through useT()/messages(). Module-level label tables became lazy
(scopeStillDeniedResult(), verbOf(T, action)); auto-reload rows dispatch on
stable ids instead of label text. Regenerates locales/_keys.tui.json.

* i18n(tui): wire slash ops/wake replies

* i18n(tui): wire pickers (modelPicker, activeSessionSwitcher, petPicker)

* i18n(tui): wire slash core/debug/setup replies

* i18n(tui): wire hubs (agents overlay/panel/controls, skills, plugins)

* i18n(tui): wire slash session/topup/subscription replies

* i18n(tui): wire chat bits (branding, thinking, messageLine, loaders, todo, queued, banner, entry)

* i18n(tui): register t3 siblings (pickers, hubs, secure, content, chatBits) and regenerate keys

* i18n(tui): wire userMessages copy through the userMessages namespace

* i18n(tui): lazy-copy test for userMessages, regenerate _keys.tui.json

* i18n(tui): wire session/gateway/lib text through the TUI catalog (lane t2)

Adds en siblings session.ts, gatewayMsg.ts, libText.ts (namespaces session,
gatewayMsg, libText) and routes user-facing literals in app/{useMainApp,
useSessionLifecycle,useInputHandlers,turnController,createServerRequestHandler,
setupHandoff,createGatewayEventHandler}.ts, gatewayClient displayed reasons,
lib/*, domain/*, hooks/* through t()/messages(). Status-bar state values that
code compares against, backend-matched strings, log lines, model-bound text,
and machine 'error:' prefixes stay literal. Regenerates locales/_keys.tui.json
(232 keys).

* i18n: serve bundled locales/<lang>.tui.yaml under overlay/packs; TUI pack parity test; regen _keys.tui.json (1250)

* i18n: translate pre-existing English stubs in bundled locales (424 leaves, 14 locales)

* tui: i18n-export-en script (English templates for pack translators)

* docs(i18n): bundled TUI packs are the bottom layer of the tui surface

* i18n(ru): translate TUI pack

* i18n(ar): translate TUI pack

* i18n(es): translate TUI pack

* i18n(pt): translate TUI pack

* i18n(ko): translate TUI pack

* i18n(de): translate TUI pack

* i18n(ja): translate TUI pack

* i18n(fr): translate TUI pack

* i18n(tr): translate TUI pack

* i18n(it): translate TUI pack

* i18n(zh): translate TUI pack

* i18n(zh-hant): translate TUI pack

* i18n(hu): translate TUI pack

* i18n(uk): translate TUI pack

1,169 missing keys translated; 81 pre-existing kept byte-identical. Parity OK missing=0 extra=0 placeholder_mismatch=0 empty=0.

Deliberately identical to en: chatBits.branding.mcpSummary ({0} MCP), chatBits.thinking.agentsHint ((/agents)), session.main.voiceStt (◉ STT), session.main.voiceTtsSuffix ( [tts]), slashCmd.core.help.tuiSection (TUI), slashCmd.core.history.hermesTag (Hermes #{0}), slashCmd.debug.heapdump.heapPath (heapdump: {0}), slashCmd.debug.mem.rss (rss), subscription.stepUp.title (Remote Spending — product feature name, as in core catalog), content.faces.* (glyph-only kaomoji).

* i18n(ga): translate TUI pack

* i18n(af): translate TUI pack

* plugin_guard: locale catalogs in language packs step down the agent-config family

A translated status line such as "Updating AGENTS.md" in locales/<lang>.yaml is UI text the loader
reads as a string leaf; it cannot edit a file. The bundled en.yaml itself tripped agent_config_mod
at critical, making any faithful language pack uninstallable. Injection shapes keep full severity.

* plugin_validate_locales: read key exports with utf-8-sig (Windows footgun lint)

* i18n(relay): route relay adapter prompt copy through t(); drop dead import-bound approval header

Adds platform.relay.* (5 keys) to en and all 16 bundled locales, reusing the sibling platform
translations for the confirm buttons and the Other option.

* ci: fix TUI import order, MDX table pipe, main's overflow-warning wording in all locales; fresh-install fixture carries the i18n kernel

- ui-tui/src/i18n/en.ts: perfectionist/sort-imports (slash before slashCmd)
- docs plugins/index.md: escape the | inside the provides_locales table cell (MDX parsed <id> as JSX)
- display.notice.uncompressed_context_overflow: adopt main's wording (names compression.enabled: false
  and /compact) in en + 16 locales; the guardrail test pins that phrase
- tests/scripts/test_fresh_source_install.py: the installer tail now resolves CLI text through
  agent.i18n, so the fixture tree carries the i18n kernel + en.yaml (not the agent runtime)

* docs(desktop-plugin-sdk): double-backtick the template-literal example (MDX evaluated ${n})

* test(e2e): display.language is validated against the live language set; exclude it from the arbitrary-string set property

* commands: keep the localized COMMANDS/COMMANDS_BY_CATEGORY module __getattr__ after the compat block removal

* build: never write locales/_keys.*.json from the desktop/TUI builds; regenerate the committed desktop key export

The desktop build regenerated locales/_keys.desktop.json in the checkout, so a
hermes update that rebuilt the app left the tree dirty (Desktop update E2E:
'M locales/_keys.desktop.json'). The key exports are committed artifacts pinned
to en.ts by apps/desktop/scripts/i18n-keys.test.mjs and ui-tui i18n:keys:check;
builds read them, never write them. Regenerated after main's new desktop strings.

* test: unbreak two main-red timing tests the PR merge-ref inherits

- test_local_runtime racing fake publishes the modern state record (legacy pid-only
  records are rejected since 65ff3ad353; main has been red on this test since)
- test_run_progress_topics ManyProgressLinesAgent waits for the first bubble instead of
  a fixed 0.35s, which a loaded CI runner does not always meet

* chore(i18n): regenerate desktop key catalog for main's new strings (model pricing, copy changelog)

* test(e2e): torture-chamber fd monitor confirms a deleted sidecar is still held before calling it a leak

SQLite's WAL last-close unlinks -shm before closing its descriptor (unixShmUnmap, then
unixShmPurge), so a healthy close shows a (deleted) -shm for microseconds; the 20ms poll
occasionally caught that window on the short-lived opener and failed the episode.

* chore(i18n): regenerate desktop key catalog for main's telemetry/consent strings

* chore(i18n): regenerate desktop key catalog after main sync

---------

Co-authored-by: Teknium <[email protected]>
2026-09-28 14:16:18 -07:00

982 lines
53 KiB
Python

"""Signal messenger platform adapter for a signal-cli daemon in HTTP mode (``signal-cli daemon
--http 127.0.0.1:8080``): inbound via SSE, outbound via JSON-RPC 2.0. Requires SIGNAL_HTTP_URL
and SIGNAL_ACCOUNT."""
import asyncio
import base64
import itertools
import json
import logging
import os
import random
import shutil
import subprocess
import tempfile
import time
import uuid
from collections import OrderedDict
from contextlib import suppress
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
from urllib.parse import quote, unquote
import httpx
from gateway.config import Platform, PlatformConfig
from agent.i18n import t
from gateway.platforms.base import (
BasePlatformAdapter, SendResult, cache_image_from_bytes_async,
cache_audio_from_bytes_async, cache_document_from_bytes_async, cache_image_from_url, utf16_len,
)
from gateway.platforms.event import MessageEvent, MessageType, ProcessingOutcome
from gateway.platforms.helpers import redact_phone
from gateway.platforms.helpers import cancel_task
from gateway.platforms.media_cache import mime_for_ext
from tools.audio_container import CONTAINER_TO_EXT, sniff_container
from gateway.platforms.signal_format import markdown_to_signal
from gateway.platforms.signal_rate_limit import (
SIGNAL_BATCH_PACING_NOTICE_THRESHOLD, SIGNAL_MAX_ATTACHMENTS_PER_MSG, SIGNAL_RATE_LIMIT_MAX_ATTEMPTS,
SignalRateLimitError, _extract_retry_after_seconds, _format_wait, _is_signal_rate_limit_error,
_signal_send_timeout, get_scheduler)
from gateway.platforms._shared import get_scoped_secret as _sig_secret
from utils import TRUTHY_STRINGS
logger = logging.getLogger(__name__)
SIGNAL_MAX_ATTACHMENT_SIZE = 100 * 1024 * 1024 # 100 MB
MAX_MESSAGE_LENGTH = 8000 # Signal message size limit
SSE_RETRY_DELAY_INITIAL = 2.0
SSE_RETRY_DELAY_MAX = 60.0
HEALTH_CHECK_INTERVAL = 30.0 # seconds between health checks
HEALTH_CHECK_STALE_THRESHOLD = 120.0 # seconds without SSE activity before concern
# Magic-byte prefixes checked before delegating to the shared audio/AV sniffer.
_MAGIC_EXTENSIONS = ((b"\x89PNG", ".png"), (b"\xff\xd8", ".jpg"), (b"GIF8", ".gif"), (b"%PDF", ".pdf"))
_MEDIA_TYPE_BY_MIME_PREFIX = (
("audio/", MessageType.VOICE), ("image/", MessageType.PHOTO), ("video/", MessageType.VIDEO))
_OUTCOME_REACTION = {ProcessingOutcome.SUCCESS: "✅", ProcessingOutcome.FAILURE: "❌"}
# send_multiple_images skip reasons → logger.warning args (url, detail).
_SKIP_IMAGE_LOG = {
"download": lambda url, detail: ("Signal: failed to download image %s: %s", url, detail),
"missing": lambda url, detail: ("Signal: image file not found for %s", url),
"oversize": lambda url, detail: ("Signal: image too large (%d bytes), skipping %s", detail, url)}
_QUOTE_AUTHOR_KEYS = (
"author", "authorNumber", "authorUuid", "authorAci", "authorServiceId", "authorServiceIdString")
def _parse_comma_list(value: str) -> List[str]:
"""Split a comma-separated string into a list, stripping whitespace."""
return [v.strip() for v in value.split(",") if v.strip()]
def _guess_extension(data: bytes) -> str:
"""Guess file extension from magic bytes. WEBP is claimed before the shared audio/AV sniffer
(it shares RIFF with WAVE); tools/audio_container.py owns MP3-vs-ADTS-AAC disambiguation."""
for magic, ext in _MAGIC_EXTENSIONS:
if data.startswith(magic):
return ext
if len(data) >= 12 and data[:4] == b"RIFF" and data[8:12] == b"WEBP":
return ".webp"
if (container := sniff_container(data)) is not None:
return CONTAINER_TO_EXT[container]
return ".zip" if data[:2] == b"PK" else ".bin"
def _is_image_ext(ext: str) -> bool:
return ext.lower() in {".jpg", ".jpeg", ".png", ".gif", ".webp"}
def _is_audio_ext(ext: str) -> bool:
return ext.lower() in {".mp3", ".wav", ".ogg", ".m4a", ".aac"}
def _ext_to_mime(ext: str) -> str:
"""Map file extension to MIME type (shared table matches Signal's historical map)."""
return mime_for_ext(ext, fallback="application/octet-stream")
def _remux_aac_to_m4a(aac_data: bytes) -> Optional[Tuple[bytes, str]]:
"""Losslessly remux raw ADTS AAC (Android voice notes, rejected by most STT APIs) to .m4a.
Returns ``(m4a_bytes, ".m4a")``, or ``None`` when ffmpeg is missing/fails (caller keeps the input)."""
# Fall back to common Homebrew/local prefixes on macOS dev hosts.
ffmpeg = shutil.which("ffmpeg") or next(
(p for p in ("/opt/homebrew/bin/ffmpeg", "/usr/local/bin/ffmpeg")
if os.path.isfile(p) and os.access(p, os.X_OK)), None)
if not ffmpeg:
logger.debug("Signal: ffmpeg not found, skipping AAC→M4A remux")
return None
try:
with tempfile.NamedTemporaryFile(suffix=".aac", delete=False) as src:
src.write(aac_data)
src_path = src.name
dst_path = src_path[:-4] + ".m4a"
try:
proc = subprocess.run([ffmpeg, "-y", "-loglevel", "error", "-i", src_path, "-c:a", "copy", "-movflags",
"+faststart", dst_path], capture_output=True, timeout=10)
if proc.returncode != 0:
logger.warning("Signal: AAC→M4A remux failed (ffmpeg exit %d): %s",
proc.returncode, proc.stderr.decode("utf-8", "replace")[:300])
return None
with open(dst_path, "rb") as f:
return f.read(), ".m4a"
finally:
for p in (src_path, dst_path):
with suppress(OSError):
os.unlink(p)
except subprocess.TimeoutExpired:
logger.warning("Signal: AAC→M4A remux timed out (>10s)")
return None
except Exception:
logger.exception("Signal: AAC→M4A remux error")
return None
def _render_mentions(text: str, mentions: list) -> str:
"""Replace \\uFFFC mention placeholders with readable @identifiers (from the end so indices hold)."""
if not mentions or "\uFFFC" not in text:
return text
for mention in sorted(mentions, key=lambda m: m.get("start", 0), reverse=True):
start, length = mention.get("start", 0), mention.get("length", 1)
identifier = mention.get("number") or mention.get("uuid") or "user"
text = text[:start] + f"@{identifier}" + text[start + length:]
return text
def _is_signal_service_id(value: str) -> bool:
"""Return True if *value* already looks like a Signal service identifier (PNI:/u: prefix or UUID)."""
if not value:
return False
if value.startswith(("PNI:", "u:")):
return True
with suppress(ValueError, AttributeError, TypeError):
uuid.UUID(value)
return True
return False
def _looks_like_e164_number(value: str) -> bool:
"""Return True for a plausible E.164 phone number."""
return bool(value) and value.startswith("+") and value[1:].isdigit() and 7 <= len(value) - 1 <= 15
def check_signal_requirements() -> bool:
"""Check if Signal runtime dependencies are available."""
return True
def validate_signal_config(config: PlatformConfig) -> bool:
"""Check if Signal has enough config to connect."""
extra = getattr(config, "extra", {}) or {}
http_url = (extra.get("http_url", "") or _sig_secret("SIGNAL_HTTP_URL", "")).strip()
account = (extra.get("account", "") or _sig_secret("SIGNAL_ACCOUNT", "")).strip()
return bool(http_url and account)
class SignalAdapter(BasePlatformAdapter):
"""Signal messenger adapter using signal-cli HTTP daemon."""
platform = Platform.SIGNAL
MAX_MESSAGE_LENGTH = MAX_MESSAGE_LENGTH
splits_long_messages = True # send() chunks after markdown → Signal formatting conversion
# No real edit API; declaring it lets streaming suppress the cursor instead of a stale tofu square.
SUPPORTS_MESSAGE_EDITING = False
def __init__(self, config: PlatformConfig):
super().__init__(config, Platform.SIGNAL)
extra = config.extra or {}
self.http_url = extra.get("http_url", "http://127.0.0.1:8080").rstrip("/")
self.account = extra.get("account", "")
self.ignore_stories = extra.get("ignore_stories", True)
# Allowlists are per-profile (scoped reads); group policy derives from the group allowlist's
# presence. The DM allowlist mirrors run.py's SIGNAL_ALLOWED_USERS so reaction hooks (which
# fire before run.py's auth gate) can skip unauthorized senders; "*" = open.
self.group_allow_from = set(_parse_comma_list(_sig_secret("SIGNAL_GROUP_ALLOWED_USERS", "")))
_rm_cfg = extra.get("require_mention")
self.require_mention = (bool(_rm_cfg) if _rm_cfg is not None
else (_sig_secret("SIGNAL_REQUIRE_MENTION", "false") or "false").lower() in TRUTHY_STRINGS)
self.dm_allow_from = set(_parse_comma_list(_sig_secret("SIGNAL_ALLOWED_USERS", "*")))
self.client: Optional[httpx.AsyncClient] = None
self._sse_task: Optional[asyncio.Task] = None
self._health_monitor_task: Optional[asyncio.Task] = None
self._typing_tasks: Dict[str, asyncio.Task] = {}
# Per-chat typing backoff: on NETWORK_FAILURE base.py's _keep_typing would hammer sendTyping every ~2s.
self._typing_failures: Dict[str, int] = {}
self._typing_skip_until: Dict[str, float] = {}
self._running = False
self._last_sse_activity = 0.0
self._sse_response: Optional[httpx.Response] = None
self._account_normalized = self.account.strip()
# Recently sent timestamps filter echo-backs (Note to Self / linked-device sync-sents); LRU + TTL
# so a pending echo in a chatty group isn't evicted by many outbounds.
self._recent_sent_timestamps: "OrderedDict[int, float]" = OrderedDict()
self._max_recent_timestamps = 512
self._recent_sent_ttl_seconds = 300.0
# Separate FIFO of outbound timestamps: Signal quote.id is the quoted message's timestamp, so
# replies to this bot are recognised after the echo was consumed.
self._sent_message_timestamps: "OrderedDict[str, None]" = OrderedDict()
self._max_sent_message_timestamps = 500
# Best-effort number↔ACI/PNI UUID mapping so sends can upgrade a number to the UUID signal-cli prefers.
self._recipient_uuid_by_number: Dict[str, str] = {}
self._recipient_number_by_uuid: Dict[str, str] = {}
self._recipient_cache_lock = asyncio.Lock()
logger.info("Signal adapter initialized: url=%s account=%s groups=%s", self.http_url,
redact_phone(self.account), "enabled" if self.group_allow_from else "disabled")
async def connect(self, *, is_reconnect: bool = False) -> bool:
"""Connect to signal-cli daemon and start SSE listener."""
if not self.http_url or not self.account:
logger.error("Signal: SIGNAL_HTTP_URL and SIGNAL_ACCOUNT are required")
return False
lock_acquired = False # scoped lock prevents duplicate Signal listeners for the same phone
try:
if not self._acquire_platform_lock('signal-phone', self.account, 'Signal account'):
return False
lock_acquired = True
except Exception as e:
logger.warning("Signal: Could not acquire phone lock (non-fatal): %s", e)
# Tighter keepalive so idle CLOSE_WAIT drains promptly.
# See #18451.
from gateway.platforms._http_client_limits import platform_httpx_limits
self.client = httpx.AsyncClient(timeout=30.0, limits=platform_httpx_limits())
try:
try:
resp = await self.client.get(f"{self.http_url}/api/v1/check", timeout=10.0)
except Exception as e:
logger.error("Signal: cannot reach signal-cli at %s: %s", self.http_url, e)
return False
if resp.status_code != 200:
logger.error("Signal: health check failed (status %d)", resp.status_code)
return False
self._running = True
self._last_sse_activity = time.time()
self._sse_task = asyncio.create_task(self._sse_listener())
self._health_monitor_task = asyncio.create_task(self._health_monitor())
logger.info("Signal: connected to %s", self.http_url)
# Plugin-registered native handlers (ctx.register_platform_handler).
self._wire_plugin_handlers(None)
return True
finally:
if not self._running:
await self._close_client()
if lock_acquired:
self._release_platform_lock()
async def _close_client(self) -> None:
if self.client:
await self.client.aclose()
self.client = None
async def disconnect(self) -> None:
"""Stop SSE listener and clean up."""
self._running = False
for task in (self._sse_task, self._health_monitor_task):
await cancel_task(task)
for task in self._typing_tasks.values():
task.cancel()
self._typing_tasks.clear()
await self._close_client()
self._release_platform_lock()
logger.info("Signal: disconnected")
async def _handle_sse_line(self, line: str) -> None:
"""Dispatch one SSE line; keepalive comments (":") count as activity so the health monitor stays quiet."""
if line.startswith(":"):
self._last_sse_activity = time.time()
elif line.startswith("data:") and (data_str := line[5:].strip()):
self._last_sse_activity = time.time()
try:
await self._handle_envelope(json.loads(data_str))
except json.JSONDecodeError:
logger.debug("Signal SSE: invalid JSON: %s", data_str[:100])
except Exception:
logger.exception("Signal SSE: error handling event")
async def _sse_listener(self) -> None:
"""Listen for SSE events from signal-cli daemon, reconnecting with jittered backoff."""
url = f"{self.http_url}/api/v1/events?account={quote(self.account, safe='')}"
backoff = SSE_RETRY_DELAY_INITIAL
while self._running:
try:
logger.debug("Signal SSE: connecting to %s", url)
async with self.client.stream("GET", url, headers={"Accept": "text/event-stream"},
timeout=None) as response:
self._sse_response = response
backoff = SSE_RETRY_DELAY_INITIAL # Reset on successful connection
self._last_sse_activity = time.time()
logger.info("Signal SSE: connected")
buffer = ""
async for chunk in response.aiter_text():
if not self._running:
break
buffer += chunk
while "\n" in buffer:
line, buffer = buffer.split("\n", 1)
if line := line.strip():
await self._handle_sse_line(line)
except asyncio.CancelledError:
break
except httpx.HTTPError as e:
if self._running:
logger.warning("Signal SSE: HTTP error: %s (reconnecting in %.0fs)", e, backoff)
except Exception as e:
if self._running:
logger.warning("Signal SSE: error: %s (reconnecting in %.0fs)", e, backoff)
if self._running:
await asyncio.sleep(backoff + backoff * 0.2 * random.random()) # 20% jitter vs thundering herd
backoff = min(backoff * 2, SSE_RETRY_DELAY_MAX)
self._sse_response = None
async def _health_monitor(self) -> None:
"""Monitor SSE connection health and force reconnect if stale."""
while self._running:
await asyncio.sleep(HEALTH_CHECK_INTERVAL)
if not self._running:
break
if (elapsed := time.time() - self._last_sse_activity) <= HEALTH_CHECK_STALE_THRESHOLD:
continue
logger.warning("Signal: SSE idle for %.0fs, checking daemon health", elapsed)
try:
resp = await self.client.get(f"{self.http_url}/api/v1/check", timeout=10.0)
except Exception as e:
logger.warning("Signal: health check error: %s, forcing reconnect", e)
self._force_reconnect()
continue
if resp.status_code == 200: # daemon alive but SSE quiet — reset activity to avoid repeated warnings
self._last_sse_activity = time.time()
logger.debug("Signal: daemon healthy, SSE idle")
else:
logger.warning("Signal: health check failed (%d), forcing reconnect", resp.status_code)
self._force_reconnect()
def _force_reconnect(self) -> None:
"""Force SSE reconnection by closing the current response."""
if self._sse_response and not self._sse_response.is_stream_consumed:
with suppress(Exception):
task = asyncio.create_task(self._sse_response.aclose())
self._background_tasks.add(task)
task.add_done_callback(self._background_tasks.discard)
self._sse_response = None
def _unwrap_sync_message(self, envelope_data: dict) -> Optional[dict]:
"""Promote a "Note to Self" / group sync-sent to a dataMessage envelope; None for other
sync events (read receipts, typing, our own outbound echoes)."""
sync_msg = envelope_data.get("syncMessage")
sent_msg = sync_msg.get("sentMessage") if isinstance(sync_msg, dict) else None
if not sent_msg or not isinstance(sent_msg, dict):
return None
dest = sent_msg.get("destinationNumber") or sent_msg.get("destination")
if dest != self._account_normalized and not (sent_msg.get("groupInfo") or {}).get("groupId"):
return None
if self._consume_sent_timestamp(sent_msg.get("timestamp")):
return None # echo of our own outbound reply
return {**envelope_data, "dataMessage": sent_msg}
def _apply_group_mention_rules(self, text: str, data_message: dict) -> Tuple[bool, str]:
"""Gate on require_mention (False = drop) and strip the bot's own @mention from every group
message, so the agent doesn't read "@+155****4567 say hello" as a directive to contact that number."""
account_norm = self._account_normalized
if self.require_mention:
mentioned_in_text = account_norm and (f"@{account_norm}" in (text or ""))
mentioned_in_metadata = any(account_norm in (m.get("number"), m.get("uuid"))
for m in (data_message.get("mentions") or []))
if not mentioned_in_text and not mentioned_in_metadata:
logger.debug("Signal: ignoring group message (require_mention=true, bot not mentioned)")
return False, text
if text and account_norm:
text = text.replace(f"@{account_norm}", "")
if bot_uuid := self._recipient_uuid_by_number.get(account_norm):
text = text.replace(f"@{bot_uuid}", "")
text = text.replace(" ", " ").strip() # collapse only the doubled space; newlines preserved
return True, text
async def _collect_attachments(self, attachments_data: list) -> Tuple[List[str], List[str]]:
"""Fetch + cache inbound attachments; returns (media_urls, media_types)."""
media_urls: List[str] = []
media_types: List[str] = []
for att in attachments_data:
att_id, att_size = att.get("id"), att.get("size", 0)
if not att_id:
continue
if att_size > SIGNAL_MAX_ATTACHMENT_SIZE:
logger.warning("Signal: attachment too large (%d bytes), skipping", att_size)
continue
try:
cached_path, ext = await self._fetch_attachment(att_id)
if cached_path:
media_urls.append(cached_path)
media_types.append(att.get("contentType") or _ext_to_mime(ext))
except Exception:
logger.exception("Signal: failed to fetch attachment %s", att_id)
return media_urls, media_types
async def _handle_envelope(self, envelope: dict) -> None:
"""Process an incoming signal-cli envelope."""
envelope_data = envelope.get("envelope", envelope)
is_note_to_self = "syncMessage" in envelope_data
if is_note_to_self:
envelope_data = self._unwrap_sync_message(envelope_data)
if envelope_data is None:
return
sender = (envelope_data.get("sourceNumber") or envelope_data.get("sourceUuid")
or envelope_data.get("source"))
sender_name = envelope_data.get("sourceName", "")
sender_uuid = envelope_data.get("sourceUuid", "")
self._remember_recipient_identifiers(sender, sender_uuid)
if not sender:
logger.debug("Signal: ignoring envelope with no sender")
return
# Self-message filtering prevents reply loops (Note to Self is allowed)
if self._account_normalized and sender == self._account_normalized and not is_note_to_self:
return
if self.ignore_stories and envelope_data.get("storyMessage"):
return
# Edited messages carry their updated dataMessage inside editMessage
data_message = (envelope_data.get("dataMessage")
or (envelope_data.get("editMessage") or {}).get("dataMessage"))
if not data_message:
return
group_info = data_message.get("groupInfo")
group_id = group_info.get("groupId") if group_info else None
is_group = bool(group_id)
if is_group and not self._group_allowed(group_id):
return
chat_id = f"group:{group_id}" if is_group else sender
text = data_message.get("message", "")
if text and (mentions := data_message.get("mentions", [])):
text = _render_mentions(text, mentions)
if is_group:
mentioned, text = self._apply_group_mention_rules(text, data_message)
if not mentioned:
return
# quote.id is the quoted message's timestamp, quote.author the quoted sender —
# both are preserved so the gateway can tell the agent which message was replied to.
quote_data = data_message.get("quote") or {}
reply_to_id = str(quote_data.get("id")) if quote_data.get("id") else None
reply_to_author = self._extract_quote_author(quote_data)
attachments_data = data_message.get("attachments", [])
media_urls, media_types = [], []
if attachments_data and not getattr(self, "ignore_attachments", False):
media_urls, media_types = await self._collect_attachments(attachments_data)
# Skip contentless envelopes (profile key updates, empty messages) that still carry a
# dataMessage wrapper — otherwise msg='' triggers a full agent turn.
if (not text or not text.strip()) and not media_urls:
logger.debug("Signal: skipping contentless envelope from %s (%d attachments)", redact_phone(sender),
len(media_urls) if media_urls else 0)
return
source = self.build_source(
chat_id=chat_id, chat_name=group_info.get("groupName") if group_info else sender_name,
chat_type="group" if is_group else "dm", user_id=sender,
user_name=sender_name or sender, user_id_alt=sender_uuid if sender_uuid else None,
chat_id_alt=group_id if is_group else None)
# First matching MIME prefix wins; everything else (application/*, text/*, unknown) is a DOCUMENT
# so run.py's document-context injection surfaces the cached path.
msg_type = MessageType.TEXT if not media_types else next(
(mt for prefix, mt in _MEDIA_TYPE_BY_MIME_PREFIX if any(m.startswith(prefix) for m in media_types)),
MessageType.DOCUMENT)
ts_ms = envelope_data.get("timestamp", 0) # milliseconds since epoch
timestamp = datetime.now(tz=timezone.utc)
if ts_ms:
with suppress(ValueError, OSError):
timestamp = datetime.fromtimestamp(ts_ms / 1000, tz=timezone.utc)
# raw_message keeps sender + timestamp_ms so processing hooks can build sendReaction targets.
event = MessageEvent(
source=source, text=text or "", message_type=msg_type, media_urls=media_urls,
media_types=media_types, timestamp=timestamp,
raw_message={"sender": sender, "timestamp_ms": ts_ms, "quote": quote_data if quote_data else None},
reply_to_message_id=reply_to_id, reply_to_text=quote_data.get("text"),
reply_to_author_id=reply_to_author,
reply_to_author_name=quote_data.get("authorName") or quote_data.get("authorProfileName"),
reply_to_is_own_message=self._quote_references_own_message(reply_to_id, reply_to_author),
)
logger.debug("Signal: message from %s in %s: %s", redact_phone(sender), chat_id[:20], (text or "")[:50])
await self.handle_message(event)
def _group_allowed(self, group_id: str) -> bool:
"""Group policy from SIGNAL_GROUP_ALLOWED_USERS: unset → groups disabled; IDs → only those
groups; "*" → all. DM auth is run.py's (_is_user_authorized)."""
if not self.group_allow_from:
logger.debug("Signal: ignoring group message (no SIGNAL_GROUP_ALLOWED_USERS)")
return False
if "*" not in self.group_allow_from and group_id not in self.group_allow_from:
logger.debug("Signal: group %s not in allowlist", group_id[:8] if group_id else "?")
return False
return True
def _remember_recipient_identifiers(self, number: Optional[str], service_id: Optional[str]) -> None:
"""Cache any number↔UUID mapping observed from Signal envelopes."""
if number and service_id and _is_signal_service_id(service_id):
self._recipient_uuid_by_number[number] = service_id
self._recipient_number_by_uuid[service_id] = number
@staticmethod
def _extract_quote_author(quote_data: Any) -> Optional[str]:
"""Return the best available Signal sender identifier from quote metadata."""
keys = _QUOTE_AUTHOR_KEYS if isinstance(quote_data, dict) else ()
return next((str(quote_data[k]) for k in keys if quote_data.get(k)), None)
def _quote_references_own_message(self, reply_to_id: Optional[str], reply_to_author: Optional[str]) -> bool:
"""True when a Signal quote points at this adapter's outbound message."""
if reply_to_id and str(reply_to_id) in self._sent_message_timestamps:
return True
if not reply_to_author:
return False
author, acct = str(reply_to_author).strip(), self._account_normalized
# Cached number↔UUID mappings are only ever stored with truthy keys and values.
return bool(acct) and (author == acct or author == self._recipient_uuid_by_number.get(acct)
or self._recipient_number_by_uuid.get(author) == acct)
def _remember_sent_message_timestamp(self, timestamp: Any) -> None:
"""Keep a bounded cache of outbound Signal timestamps for quote matching."""
if timestamp is None:
return
key = str(timestamp)
self._sent_message_timestamps.pop(key, None) # re-insert as most-recently-used so eviction drops old ones
self._sent_message_timestamps[key] = None
while len(self._sent_message_timestamps) > self._max_sent_message_timestamps:
self._sent_message_timestamps.popitem(last=False)
def _extract_contact_uuid(self, contact: Any, phone_number: str) -> Optional[str]:
"""Best-effort extraction of a Signal service ID from listContacts output."""
if not isinstance(contact, dict):
return None
service_id = contact.get("uuid") or contact.get("serviceId")
profile = contact.get("profile")
if not service_id and isinstance(profile, dict):
service_id = profile.get("serviceId") or profile.get("uuid")
if service_id and _is_signal_service_id(service_id) and phone_number in (
contact.get("number"), contact.get("recipient")):
return service_id
return None
async def _resolve_recipient(self, chat_id: str) -> str:
"""Return the preferred Signal recipient identifier for a direct chat."""
if not chat_id or chat_id.startswith("group:") or not _looks_like_e164_number(chat_id):
return chat_id
if cached := self._recipient_uuid_by_number.get(chat_id):
return cached
async with self._recipient_cache_lock:
if cached := self._recipient_uuid_by_number.get(chat_id):
return cached
contacts = await self._rpc("listContacts", {"account": self.account, "allRecipients": True})
for contact in contacts if isinstance(contacts, list) else ():
number = contact.get("number") if isinstance(contact, dict) else None
service_id = self._extract_contact_uuid(contact, chat_id)
if number and service_id:
self._remember_recipient_identifiers(number, service_id)
return self._recipient_uuid_by_number.get(chat_id, chat_id)
async def _with_target(self, params: Dict[str, Any], chat_id: str, *, resolve: bool = True) -> Dict[str, Any]:
"""Add the groupId / recipient routing key for *chat_id* to *params* (in place)."""
if chat_id.startswith("group:"):
params["groupId"] = chat_id[6:]
else:
params["recipient"] = [await self._resolve_recipient(chat_id) if resolve else chat_id]
return params
async def _fetch_attachment(self, attachment_id: str) -> tuple:
"""Fetch an attachment via JSON-RPC and cache it. Returns (path, ext)."""
result = await self._rpc("getAttachment", {"account": self.account, "id": attachment_id})
if not result:
return None, ""
if isinstance(result, dict): # signal-cli returns {"data": "base64..."}
result = result.get("data")
if not result:
logger.warning("Signal: attachment response missing 'data' key")
return None, ""
raw_data = base64.b64decode(result)
ext = _guess_extension(raw_data)
# Android voice notes are raw ADTS AAC, which Whisper-style STT rejects; remux losslessly
# to .m4a. Without ffmpeg the raw file is cached as-is (no downstream remux fallback).
if ext == ".aac":
raw_data, ext = (await asyncio.to_thread(_remux_aac_to_m4a, raw_data)) or (raw_data, ext)
cache = (cache_image_from_bytes_async if _is_image_ext(ext)
else cache_audio_from_bytes_async if _is_audio_ext(ext) else cache_document_from_bytes_async)
return await cache(raw_data, ext), ext
async def _rpc(self, method: str, params: dict, rpc_id: str = None, *, log_failures: bool = True,
raise_on_rate_limit: bool = False, timeout: float = 30.0) -> Any:
"""Send a JSON-RPC 2.0 request to signal-cli. ``log_failures=False`` logs failures at DEBUG (typing
path: silence NETWORK_FAILURE spam); ``raise_on_rate_limit=True`` raises ``SignalRateLimitError``
on a 429 / RateLimitException instead of swallowing it."""
if not self.client:
logger.warning("Signal: RPC called but client not connected")
return None
payload = {"jsonrpc": "2.0", "method": method, "params": params,
"id": rpc_id if rpc_id is not None else f"{method}_{int(time.time() * 1000)}"}
fail_level = logging.WARNING if log_failures else logging.DEBUG
try:
resp = await self.client.post(f"{self.http_url}/api/v1/rpc", json=payload, timeout=timeout)
resp.raise_for_status()
data = resp.json()
if "error" in data:
err = data["error"]
if raise_on_rate_limit and _is_signal_rate_limit_error(err):
err_msg = str(err.get("message", "")) if isinstance(err, dict) else str(err)
raise SignalRateLimitError(err_msg, retry_after=_extract_retry_after_seconds(err))
logger.log(fail_level, "Signal RPC error (%s): %s", method, err)
return None
result = data.get("result")
if isinstance(result, dict) and raise_on_rate_limit:
for r in result.get("results") if isinstance(result.get("results"), list) else ():
if isinstance(r, dict) and r.get("type") == "RATE_LIMIT_FAILURE":
raise SignalRateLimitError("Rate limit exceeded for recipient",
retry_after=r.get("retryAfterSeconds"))
return result
except SignalRateLimitError:
raise
except Exception as e:
logger.log(fail_level, "Signal RPC %s failed: %s", method, e)
return None
def format_message(self, content: str) -> str:
"""Plain-text fallback for the base-class send path; send() applies rich styles itself."""
return content
def _validate_send_result(self, result: Any) -> tuple[bool, Optional[str]]:
"""Validate signal-cli send response results. Returns (success, error_message)."""
results = result.get("results") if isinstance(result, dict) else None
for r in results if isinstance(results, list) else ():
if not isinstance(r, dict):
continue
rtype = r.get("type")
if rtype and rtype != "SUCCESS":
return False, str(rtype)
if "success" in r and not r.get("success"):
return False, str(r.get("failure") or "Recipient delivery failed")
return True, None
@staticmethod
def _utf16_offsets(text: str) -> list[int]:
"""Return cumulative UTF-16 offsets for every Python character boundary."""
return [0, *itertools.accumulate(utf16_len(char) for char in text)]
@staticmethod
def _styles_for_chunk(text_styles: list[str], chunk_start: int, chunk_end: int) -> list[str]:
"""Translate full-message Signal styles into a chunk-local range list."""
adjusted: list[str] = []
for style_string in text_styles:
try:
start_s, length_s, style_type = style_string.split(":", 2)
style_start, style_end = int(start_s), int(start_s) + int(length_s)
except (TypeError, ValueError):
logger.debug("[Signal] Ignoring malformed textStyle range: %r", style_string)
continue
overlap_start, overlap_end = max(style_start, chunk_start), min(style_end, chunk_end)
if overlap_start < overlap_end:
adjusted.append(f"{overlap_start - chunk_start}:{overlap_end - overlap_start}:{style_type}")
return adjusted
@classmethod
def _split_signal_formatted_message(
cls, plain_text: str, text_styles: list[str], max_length: int) -> list[tuple[str, list[str]]]:
"""Split converted Signal text into chunks, translating body ranges per chunk. Splitting after
conversion keeps styles that cross a chunk boundary intact instead of leaking Markdown markers."""
if utf16_len(plain_text) <= max_length:
return [(plain_text, text_styles)]
body_limit = max(1, max_length - 10) # 10 = indicator reserve, mirrors truncate_message().
offsets = cls._utf16_offsets(plain_text)
chunks: list[tuple[str, list[str]]] = []
start_idx, total_u16 = 0, offsets[-1]
while offsets[start_idx] < total_u16:
end_budget = min(total_u16, offsets[start_idx] + body_limit)
end_idx = start_idx + 1
while end_idx < len(offsets) and offsets[end_idx] <= end_budget:
end_idx += 1
end_idx = max(end_idx - 1, start_idx + 1)
chunk_styles = cls._styles_for_chunk(text_styles, offsets[start_idx], offsets[end_idx])
chunks.append((plain_text[start_idx:end_idx], chunk_styles))
start_idx = end_idx
if len(chunks) == 1:
return chunks
return [(f"{txt} ({idx}/{len(chunks)})", st) for idx, (txt, st) in enumerate(chunks, start=1)]
async def _rpc_send(self, params: Dict[str, Any], fail_error: str) -> Tuple[Any, Optional[SendResult]]:
"""Run a ``send`` RPC, validate and track it; ``(result, None)`` or ``(None, failed SendResult)``."""
if (result := await self._rpc("send", params)) is None:
return None, SendResult(success=False, error=fail_error)
success, err_msg = self._validate_send_result(result)
if not success:
return None, SendResult(success=False, error=err_msg, raw_response=result)
self._track_sent_timestamp(result)
return result, None
async def send(self, chat_id: str, content: str, reply_to: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None) -> SendResult:
"""Send a text message with native Signal formatting."""
await self._stop_typing_indicator(chat_id)
if not content or not content.strip():
return SendResult(success=True, message_id=None)
base_params = await self._with_target({"account": self.account}, chat_id)
chunks = self._split_signal_formatted_message(*markdown_to_signal(content), self.MAX_MESSAGE_LENGTH)
last_result = None
for idx, (plain_text, text_styles) in enumerate(chunks, start=1):
params: Dict[str, Any] = dict(base_params, message=plain_text)
if len(text_styles) == 1:
params["textStyle"] = text_styles[0]
elif text_styles:
params["textStyles"] = text_styles
logger.info("[Signal] Sending response chunk %d/%d (%d chars) to %s", idx, len(chunks), len(plain_text),
chat_id)
last_result, err = await self._rpc_send(params, "RPC send failed")
if err:
return err
# No editable message identifier; message_id=None keeps the stream consumer on the non-edit path.
return SendResult(success=True, message_id=None, raw_response=last_result)
def _track_sent_timestamp(self, rpc_result) -> None:
"""Record outbound message timestamp for echo-back filtering."""
ts = rpc_result.get("timestamp") if isinstance(rpc_result, dict) else None
if not ts:
return
self._remember_sent_message_timestamp(ts)
now, recent = time.monotonic(), self._recent_sent_timestamps
recent.pop(ts, None) # re-insert to mark as most-recently-used
recent[ts] = now
# Drop entries older than TTL first, then enforce the hard cap.
cutoff = now - self._recent_sent_ttl_seconds
while recent and next(iter(recent.values())) < cutoff:
recent.popitem(last=False)
while len(recent) > self._max_recent_timestamps:
recent.popitem(last=False)
def _consume_sent_timestamp(self, ts) -> bool:
"""Pop a timestamp if it matches one we sent. Returns True on echo."""
return bool(ts) and self._recent_sent_timestamps.pop(ts, None) is not None
async def send_typing(self, chat_id: str, metadata=None) -> None:
"""Typing indicator (called every ~2s by base.py's ``_keep_typing``). Only the first consecutive
failure logs at WARNING; after three, the RPC is skipped for an exponential cooldown; success resets."""
now = time.monotonic()
if now < self._typing_skip_until.get(chat_id, 0.0):
return
params = await self._with_target({"account": self.account}, chat_id)
fails = self._typing_failures.get(chat_id, 0)
if await self._rpc("sendTyping", params, rpc_id="typing", log_failures=(fails == 0)) is not None:
self._typing_failures.pop(chat_id, None)
self._typing_skip_until.pop(chat_id, None)
return
fails = self._typing_failures[chat_id] = fails + 1
if fails >= 3: # exponential backoff: 16s, 32s, 60s cap
self._typing_skip_until[chat_id] = now + min(60.0, 16.0 * (2 ** (fails - 3)))
async def _resolve_image_path(self, image_url: str) -> Tuple[Optional[str], Optional[str], Any]:
"""``(path, None, None)`` or ``(None, reason, detail)``: reason download (exc) / missing / oversize (size)."""
if image_url.startswith("file://"):
file_path = unquote(image_url[7:])
else:
try:
file_path = await cache_image_from_url(image_url)
except Exception as e:
return None, "download", e
if not file_path or not Path(file_path).exists():
return None, "missing", None
if (file_size := Path(file_path).stat().st_size) > SIGNAL_MAX_ATTACHMENT_SIZE:
return None, "oversize", file_size
return file_path, None, None
async def send_multiple_images(self, chat_id: str, images: List[Tuple[str, str]],
metadata: Optional[Dict[str, Any]] = None, human_delay: float = 0.0) -> SendResult:
"""Send a batch of images via chunked Signal RPC calls. Alt texts are dropped (one shared body
per send); bad images are skipped with a warning; ``human_delay`` is ignored (scheduler paces).
Returns success when at least one batch was accepted, so media-only turns report SUCCESS."""
if not images:
return SendResult(success=False, error="no images to send")
scheduler = get_scheduler()
logger.info("Signal send_multiple_images: received %d image(s) for %s — scheduler state: %s", len(images),
chat_id[:30], scheduler.state())
await self._stop_typing_indicator(chat_id)
attachments: List[str] = []
skipped = {"download": 0, "missing": 0, "oversize": 0}
for image_url, _alt_text in images:
file_path, reason, detail = await self._resolve_image_path(image_url)
if not reason:
attachments.append(file_path)
continue
skipped[reason] += 1
logger.warning(*_SKIP_IMAGE_LOG[reason](image_url, detail))
if not attachments:
logger.error("Signal: no valid images in batch of %d (download=%d missing=%d oversize=%d)", len(images),
skipped["download"], skipped["missing"], skipped["oversize"])
return SendResult(success=False, error="no valid images in batch")
logger.info("Signal send_multiple_images: %d/%d images valid, sending in chunks", len(attachments), len(images))
base_params = await self._with_target({"account": self.account, "message": ""}, chat_id)
per = SIGNAL_MAX_ATTACHMENTS_PER_MSG
att_batches = [attachments[i:i + per] for i in range(0, len(attachments), per)]
n_batches = len(att_batches)
delivered = False
for idx, att_batch in enumerate(att_batches, start=1):
n = len(att_batch)
estimated = scheduler.estimate_wait(n)
logger.debug("Signal batch %d/%d: %d attachments, estimated wait=%.1fs", idx, n_batches, n, estimated)
if estimated >= SIGNAL_BATCH_PACING_NOTICE_THRESHOLD:
await self._notify_batch_pacing(chat_id, idx, n_batches, estimated)
if await self._send_attachment_batch(scheduler, dict(base_params, attachments=att_batch), n,
f"{idx}/{n_batches}"):
delivered = True
return SendResult(
success=delivered,
error=None if delivered else "all Signal attachment batches failed")
async def _send_attachment_batch(self, scheduler, params: Dict[str, Any], n: int, label: str) -> bool:
"""Send one attachment batch with rate-limit pacing and a single transient retry. Tokens are
deducted only on validated success (None = server never accepted it); 429s feed the scheduler.
Returns True when the server accepted the batch."""
send_timeout, max_attempts = _signal_send_timeout(n), SIGNAL_RATE_LIMIT_MAX_ATTEMPTS
for attempt in range(1, max_attempts + 1):
await scheduler.acquire(n)
t0 = time.monotonic()
try:
result = await self._rpc("send", params, raise_on_rate_limit=True, timeout=send_timeout)
except SignalRateLimitError as e:
scheduler.feedback(e.retry_after, n)
retry_after = f"{e.retry_after:.0f}s" if e.retry_after else "unknown"
if attempt >= max_attempts:
logger.error("Signal: rate-limit retries exhausted on batch %s (%d attachments lost, "
"server retry_after=%s)", label, n, retry_after)
return False
logger.warning("Signal: rate-limited on batch %s (attempt %d/%d, server retry_after=%s); "
"scheduler will pace the retry", label, attempt, max_attempts, retry_after)
continue
duration = time.monotonic() - t0
success, err_msg = self._validate_send_result(result) if result is not None else (False, None)
if success:
self._track_sent_timestamp(result)
await scheduler.report_rpc_duration(duration, n)
logger.info("Signal batch %s: %d attachments sent in %.1fs (attempt %d/%d)", label, n, duration,
attempt, max_attempts)
return True
logger.error("Signal: RPC send failed for batch %s (%d attachments, attempt %d/%d, rpc_duration=%.1fs)%s",
label, n, attempt, max_attempts, duration, f": {err_msg}" if result is not None else "")
if attempt >= max_attempts:
return False
logger.info("Signal: retrying batch %s after %.1fs backoff", label, 2.0 ** attempt)
await asyncio.sleep(2.0 ** attempt)
return False
async def _notify_batch_pacing(self, chat_id: str, next_batch_idx: int, total_batches: int, wait_s: float) -> None:
"""Tell the user about an inter-batch pacing wait over the notice threshold (best-effort)."""
try:
await self.emit_warning(chat_id, t(
"platform.signal.batch_pacing_notice",
wait=_format_wait(wait_s), batch=next_batch_idx, total=total_batches))
except Exception as e:
logger.warning("Signal: failed to send pacing notice: %s", e)
async def send_image(self, chat_id: str, image_url: str, caption: Optional[str] = None, **kwargs) -> SendResult:
"""Send an image. Supports http(s):// and file:// URLs."""
await self._stop_typing_indicator(chat_id)
file_path, reason, detail = await self._resolve_image_path(image_url)
if reason == "download":
logger.warning("Signal: failed to download image: %s", detail)
if reason:
return SendResult(success=False, error={
"download": str(detail), "missing": "Image file not found",
"oversize": f"Image too large ({detail} bytes)"}[reason])
return await self._send_file(chat_id, file_path, caption, "RPC send with attachment failed")
async def _send_file(self, chat_id: str, file_path: str, caption: Optional[str], fail_error: str) -> SendResult:
"""Send one local file as a Signal attachment via the ``send`` RPC."""
params = await self._with_target(
{"account": self.account, "message": caption or "", "attachments": [file_path]}, chat_id)
_, err = await self._rpc_send(params, fail_error)
return err or SendResult(success=True)
async def _send_attachment(self, chat_id: str, file_path: str, media_label: str,
caption: Optional[str] = None) -> SendResult:
"""Send any local file as a Signal attachment (shared by send_document/image_file/voice/video)."""
await self._stop_typing_indicator(chat_id)
try:
file_size = Path(file_path).stat().st_size
except FileNotFoundError:
return SendResult(success=False, error=f"{media_label} file not found: {file_path}")
if file_size > SIGNAL_MAX_ATTACHMENT_SIZE:
return SendResult(success=False, error=f"{media_label} too large ({file_size} bytes)")
return await self._send_file(chat_id, file_path, caption, f"RPC send {media_label.lower()} failed")
async def send_document(self, chat_id, file_path, caption=None, filename=None, **kwargs) -> SendResult:
return await self._send_attachment(chat_id, file_path, "File", caption)
async def send_image_file(self, chat_id, image_path, caption=None, reply_to=None, **kwargs) -> SendResult:
"""Native Signal attachment for the gateway MEDIA: delivery path."""
return await self._send_attachment(chat_id, image_path, "Image", caption)
async def send_voice(self, chat_id, audio_path, caption=None, reply_to=None, **kwargs) -> SendResult:
"""Audio attachment — Signal has no distinct voice-message API."""
return await self._send_attachment(chat_id, audio_path, "Audio", caption)
async def send_video(self, chat_id, video_path, caption=None, reply_to=None, **kwargs) -> SendResult:
return await self._send_attachment(chat_id, video_path, "Video", caption)
async def _stop_typing_indicator(self, chat_id: str) -> None:
"""Stop a typing indicator loop for a chat."""
await cancel_task(self._typing_tasks.pop(chat_id, None))
# Explicit stop-typing RPC so the recipient drops the indicator now instead of after
# Signal's ~5s timeout. Best-effort: failures must not prevent the backoff cleanup below.
with suppress(Exception):
params = await self._with_target({"account": self.account}, chat_id)
params["stop"] = True
await self._rpc("sendTyping", params, rpc_id="typing-stop", log_failures=False)
self._typing_failures.pop(chat_id, None)
self._typing_skip_until.pop(chat_id, None)
async def stop_typing(self, chat_id: str) -> None:
"""Public stop-typing hook called from the base adapter's _keep_typing finally block."""
await self._stop_typing_indicator(chat_id)
async def _send_reaction_rpc(self, chat_id: str, params: Dict[str, Any]) -> bool:
"""Route a ``sendReaction`` RPC to *chat_id* (no UUID upgrade — author IDs come from the envelope)."""
await self._with_target(params, chat_id, resolve=False)
return await self._rpc("sendReaction", params) is not None
async def send_reaction(self, chat_id: str, emoji: str, target_author: str, target_timestamp: int) -> bool:
"""React to the message (author number/UUID, Signal ms timestamp) via signal-cli RPC."""
ok = await self._send_reaction_rpc(chat_id, {
"account": self.account, "emoji": emoji, "targetAuthor": target_author,
"targetTimestamp": target_timestamp})
if not ok:
logger.debug("Signal: sendReaction failed (chat=%s, emoji=%s)", chat_id[:20], emoji)
return ok
async def remove_reaction(self, chat_id: str, target_author: str, target_timestamp: int) -> bool:
"""Remove a reaction by sending an empty-string emoji."""
return await self._send_reaction_rpc(chat_id, {
"account": self.account, "emoji": "", "targetAuthor": target_author, "targetTimestamp": target_timestamp,
"remove": True})
def _extract_reaction_target(self, event: MessageEvent) -> Optional[tuple]:
"""Extract (target_author, target_timestamp) from a MessageEvent, or None."""
raw = event.raw_message
ok = isinstance(raw, dict) and raw.get("sender") and raw.get("timestamp_ms")
return (raw["sender"], raw["timestamp_ms"]) if ok else None
def _reactions_enabled(self, event: "MessageEvent" = None) -> bool:
"""SIGNAL_REACTIONS env gate, then the DM allowlist: reactions fire before run.py's auth gate,
so an unauthorized contact's 👀 would otherwise reveal a listening bot."""
if str(_sig_secret("SIGNAL_REACTIONS", "true")).lower() in {"false", "0", "no"}:
return False
sender = getattr(getattr(event, "source", None), "user_id", None) if event is not None else None
return not (sender and "*" not in self.dm_allow_from and sender not in self.dm_allow_from)
async def on_processing_start(self, event: MessageEvent) -> None:
"""React with 👀 when processing begins."""
if self._reactions_enabled(event) and (target := self._extract_reaction_target(event)):
await self.send_reaction(event.source.chat_id, "👀", *target)
async def on_processing_complete(self, event: MessageEvent, outcome: "ProcessingOutcome") -> None:
"""Swap 👀 for ✅/❌; on CANCELLED the 👀 stays to keep reflecting "in progress" (matches Telegram)."""
if outcome == ProcessingOutcome.CANCELLED or not self._reactions_enabled(event):
return
if not (target := self._extract_reaction_target(event)):
return
await self.remove_reaction(event.source.chat_id, *target)
if emoji := _OUTCOME_REACTION.get(outcome):
await self.send_reaction(event.source.chat_id, emoji, *target)
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
"""Get information about a chat/contact."""
if chat_id.startswith("group:"):
return {"name": chat_id, "type": "group", "chat_id": chat_id}
result = await self._rpc("getContact", {"account": self.account, "contactAddress": chat_id})
name = (result.get("name") or result.get("profileName")) if isinstance(result, dict) else None
return {"name": name or chat_id, "type": "dm", "chat_id": chat_id}