hermes-agent/tests/gateway/test_email.py
Teknium 9bcbe7b5df
feat(i18n): pluggable, layered language packs across core, Desktop and TUI (#126296)
* feat(i18n): layered catalogs — plugin packs and user overlay over bundled locales

* feat(tui): i18n layer — en catalog, nanostore runtime, RPC pack loader, _keys.tui.json emitter

ui-tui/src/i18n/: en.ts (facade over topical siblings under en/), types.ts
(Translations + dotted TranslationKey derived from en), runtime.ts ($locale/
$catalog atoms, translateFrom active→en→key, pack merge with string→fn
wrapping for {0}/{1} placeholders), loader.ts (display.language →
i18n.catalog {lang, surface:'tui'}, English when the method is missing),
useT()/useLocale() hooks, t() for non-React code. useConfigSync feeds the
loader from the existing config.get full hydration. `npm run i18n:keys`
writes locales/_keys.tui.json (sorted flat key list) and runs before build.

* feat(plugins): provides_locales manifest field, ctx.register_locale/register_locale_dir, manifest-only language packs

* chore(tui): split en catalog siblings by lane (slash sibling)

* feat(plugins): validate language packs — parse, text-only, key-subset WARN against en / _keys exports

* feat(tui_gateway): i18n.languages / i18n.catalog RPC + regenerated contracts

* feat(config): display.language accepts any supported_languages() id, refuses unknown ids with the list

* docs(i18n): language packs user guide, pluggable display.language, plugin developer section, AGENTS notes

* feat(plugins): report language-pack layers in the mid-run activation summary

* feat(tui): wire status bar, composer placeholders, hotkey help and approval/clarify/confirm prompts through i18n

StatusRule maps compared state values (ready/running…/summoning) to catalog
text at render via displayStatus(); hotkeys()/placeholder() resolve lazily so
a pack that arrives after boot applies. Catalog grows to 81 keys.

* feat(desktop): pluggable app locales — registry, host.i18n.registerAppLocale, backend packs, keys emitter

- Locale widens to string (BundledLocale keeps the union); TRANSLATIONS stays
  the bundled record and every consumer resolves through the registry.
- src/i18n/registry.ts: registerAppLocale(id, {endonym, rtl, translations})
  layers partial packs (nested or flat dotted) over bundled/en via
  mergeTranslations; a string over a function-valued en entry becomes a
  positional {0}/{1} formatter; $appLocaleVersion bumps so translators
  re-render; per-source disposers + replaceAppLocaleSource for atomic swaps.
- Backend packs: i18n.languages + i18n.catalog {surface:'desktop'} feed the
  registry as source 'backend' (method-not-found is silent); re-synced on
  socket open, display.language change and profile switch. A saved pack-only
  language is promoted once its pack registers.
- SDK: host.i18n.registerAppLocale / languageOptions; ctx.i18n.registerAppLocale
  tracked for unload. Docs in the desktop plugin SDK guide + skill reference.
- Language switcher lists bundled ∪ registered ∪ backend, endonym-only; RTL
  from the registry (applyDocumentLocale takes rtl).
- npm run i18n:keys emits locales/_keys.desktop.json (wired into build).

* i18n(cli): route /topup + /subscription copy through t() (cli.billing.*, cli.subscription.*)

Module-level copy tables and modal choice tuples in cli_billing_mixin.py froze
English at import, before display.language was known. They are now key tables /
builder functions evaluated at call time; every user-facing line in the /usage
balance block, /subscription and the five /topup screens reads the catalog.
Choice VALUES stay English identifiers. Fragment-assembled status lines
(Plan: … → cancels · $x left · renews …) become full templates.

* i18n(gateway): exec-approval card contract + base/run/run_busy/run_inbound replies through t()

- base_exec_approval: EA_* English constants stay; add ea_header_text()/ea_reason_label_text()/
  ea_smart_deny_line_text()/ea_default_reason_text()/ea_action_labels()/approval_timed_out_notice()
  accessors; deadline + timed-out notice resolve via gateway.exec_approval.*
- BasePlatformAdapter._EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/_EA_ACTION_LABELS become
  properties (adapters still shadow them with markup class attrs)
- run.py: provider error replies table holds catalog keys; _CONTEXT_OVERFLOW_REPLY -> _context_overflow_reply()
- run_busy/run_inbound: typed approval + slash-confirm matchers accept English ∪ approval.inputs.* (t())
- locales/en.yaml: gateway.exec_approval/busy/errors/... namespaces

* i18n(cli): wire modal, loops, agent-setup mixins through t() (cli.* keys)

* i18n(platforms): route Slack, Matrix and Feishu user-facing text through t()

Exec-approval markup overrides (_EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/
_EA_ACTION_LABELS) become per-call properties over the shared
gateway.exec_approval.* contract keys, so Slack's 3000-char section budget
measures the resolved template. Slack _APPROVAL_DECISIONS/_CONFIRM_DECISIONS,
Feishu _APPROVAL_LABEL_MAP and Matrix _EA_LEGEND/_EA_TYPED_HINT turn into
key tables resolved at click time; the Matrix typed hints become whole
sentences per offered tier instead of spliced fragments. Slack button labels
are cut to 75 chars and select placeholders to 150 after translation; the
model-facing clarify fallback answer ('choice N') stays English while the
card copy localizes.

locales/en.yaml gains the gateway.exec_approval.* contract keys plus the
platform.shared.* / platform.slack.* / platform.matrix.* / platform.feishu.*
namespaces (and the keys for the other adapters wired in follow-up commits).

* i18n(gateway): run_turn / run_turn_runner / approval-settle copy through t()

- status hints, proxy errors, background task notices, progress heartbeats, session info lines
- tool progress chrome (tool_head/tool_pending/tool_preview/tool_verbose) shared by base.format_tool_event
- run_turn_runner:1406 Chinese clarify placeholder -> gateway.clarify.native_stream_placeholder (zh text kept in zh.yaml)
- _UNEXPECTED_SILENCE_REPLY/_CLARIFY_EXPIRED_NOTICE -> accessor functions

* i18n(platforms): route Google Chat and Teams user-facing text through t()

Google Chat clarify card, typing placeholder, orphan-card labels and the whole
/setup-files reply set (module constants become platform.google_chat.setup_files.*
keys resolved at reply time). The attachment-fallback notice that shipped
hardcoded in Spanish is keyed with an English en value; es.yaml carries the
original Spanish text for those four keys.

Teams approval card header/reason use the gateway.exec_approval.* contract,
_APPROVAL_LABELS becomes a key table resolved at click time, and the meeting
summary writer resolves its section headings/fallbacks per render.

* i18n(platforms): route LINE, WeCom, email, DingTalk, IRC and Home Assistant text through t()

LINE default copy constants become catalog keys resolved in __init__ (the
LINE_*_TEXT / extra.* operator overrides still win); the busy-ack bypass
matcher keys on the leading emoji marker only, so it keeps firing once the
gateway busy heads are localized. WeCom media size/format notices that shipped
hardcoded in Chinese are keyed with English en values and zh.yaml carries the
original Chinese text. DingTalk emotion bubbles resolve per send.

* i18n(cli): route /model switch output and -q status lines through t() (cli.model.*, cli.single_query.*)

Switch-summary labels shared with the gateway reuse gateway.model.* keys
(provider/context/max-output/capabilities/prompt-caching); CLI-only variants
(glyph or no-backtick forms) live under cli.model.*. The hand-padded /model usage
block becomes a (form, description-key) table padded at render time so the
command syntax stays fixed while descriptions translate. -q 'Error:' reuses
gateway.model.error_prefix.

* i18n(cli): route TUI panel/hint/placeholder copy through t() (cli.tui.*)

_APPROVAL_CHOICE_LABELS and _TUI_MODAL_HINTS become key tables resolved at
render time; vault/sudo panel bodies are one catalog value per panel split on
newline; inline plurals use <key>_one/<key>_other. Adds the cli.* namespace
(shared/tui/voice/render/subagents/dock) to locales/en.yaml.

* i18n(cli): voice/wake-word CLI copy through t() (cli.voice.*)

RuntimeError texts raised in _voice_start_recording are human copy (callers
print {e}) and are keyed; the Termux requirement-check match stays English.
Wake state ids stay internal, only their labels localize.

* i18n(cli): live-work dock, subagent monitor and render copy through t()

cli.subagents.* / cli.dock.* / cli.render.*; count fragments pluralize via
_one/_other keys, verdict table holds keys resolved at paint time so width
clipping measures the translated text.

* i18n(gateway): unauthorized/pairing, voice, topics, shutdown, startup, notifications, kanban pings through t()

* i18n(cli): move tips + composer placeholders into the catalog (tips.tNNN / tips.placeholder.pNN)

get_random_tip()/get_random_composer_placeholder() pick a key from the English catalog
(the parity baseline, probed once per process) and resolve it through t() for the active
language, so language packs translate tips like any other string. Also lands the cli.*
en.yaml namespace consumed by the CLI info/help/error-copy wiring in the next commit.

* i18n(cli): wire chat-turn + session mixins through t(); kanban log trimmer matches t() output

* test(cli): assert TUI/dock/voice copy via t(key); prove labels resolve at render time

Pinned-English assertions in the approval-UI, live-work dock and voice tests now
go through the catalog. New test swaps the catalog after import and checks the
approval panel + hint row follow it (the reason _APPROVAL_CHOICE_LABELS and
_TUI_MODAL_HINTS became key tables).

* i18n(cli): route CLI info/help/error copy through t() (cli.* namespace)

cli_info_mixin: /help consumes CommandDef.describe() (added to commands.py: slash.<name>.description
with fallback to .description), section titles/skill/quick-command headers, /tools, /toolsets,
/usage labels, /context, /whoami, /insights, /gateway status, tool-progress labels, bang-shell
denials, MCP config-watch + /reload-mcp confirm/reload lines, /reload-skills, and the session-store
warning all read the catalog at call time (module-level label tables became functions so the
active language is honoured after startup). cli.py: worktree cleanup, tirith warning, show_config
(labels re-padded at print time), quick/plugin/skill slash-command errors, ambiguous-command hint,
stdin error, gateway start, profile warning. cli_chat_error_copy / cli_unknown_command /
cli_output / cli_init_mixin: chat error panel copy, did-you-mean lines, n-more / yes-no prompt
(localized affirmative initial alongside 'y'), unknown-toolsets warning.

* i18n(w1a): wire agent display/explainers/approval + slash registry/help through t()

- hermes_cli/commands.py: CommandDef.describe() resolves slash.<name>.description
  at call time; category labels via slash.category.*; help/alias/usage suffixes
  via slash.shared.*; gateway_help_lines and commands_platforms/slash_exec use them.
- agent/display.py: display.verb.* resolved at call time via get_tool_verb();
  bridge/spinner/thinking-verb/cute-row/failure/preview/diff text via display.*.
- agent/turn_explainers.py: exit-reason / persistence-cause tables become call-time
  lookups (explainer.exit.*, explainer.persistence.*, explainer.file_mutation.*).
- agent/background_review.py, session_activity.py, context_breakdown.py,
  status_output.py: review summaries, iteration progress, context notices.
- tools/approval.py, approval_context.py: approval.summary.*, approval.noun.*,
  approval.window.* pluralized keys.
- gateway/slash_commands*.py: remaining raw strings (busy, whoami, platform,
  bundles, memory, skills, approvals, set_home, diff, update, debug, profile,
  heartbeat, refine, review, subgoal, loop, retry, compress codex path, save,
  sessions, model guard/errors, agents rows, topup, login). HISTORY_UNREADABLE
  keeps its English constant; callers use history_unreadable() ->
  gateway.shared.history_unreadable.
- locales/en.yaml: new approval/display/explainer/slash blocks + gateway leftovers.

* i18n(telegram): route adapter chat copy through t()

Approval card (header/reason/smart-deny as HTML-escaped properties), inline
button labels, callback toasts (cut at Telegram's 200-char cap), model/choice
pickers, clarify/update/slash-confirm prompts, gmail-triage labels and the
inbound-media failure notice now come from the catalog. _UNAUTHORIZED is a
lazy _unauthorized() so the import no longer binds a language. The command
menu carries a language+payload fingerprint (forum scopes re-register on
change) and BotCommand descriptions are cut at 256.

Adds gateway.exec_approval.* (WAVE2 contract), platform.telegram.*,
platform.discord.* and the slash.*.description keys the Discord table shares
with the CLI registry to locales/en.yaml.

* i18n(gateway/platforms): whatsapp_cloud, yuanbao, weixin, signal, api_server copy through t()

- whatsapp_cloud: clarify list/buttons, approve/deny + slash-confirm labels via platform.whatsapp.* (t()-then-truncate at 20/24/72 caps); _EA_HEADER becomes a property wrapping ea_header_text()
- yuanbao: SLOW_RESPONSE_MESSAGE -> slow_response_message() (platform.yuanbao.slow_response_notice; zh keeps the original text); cron-wrapper markers centralized as module constants for strip_cron_wrapper
- api_server: PROVIDER_AUTH_FAILED_LABEL/PROVIDER_RATE_LIMITED_LABEL stay English for run.py matchers; user_text() renders via t()
- signal/_format_wait, weixin voice caption, openai_routes transformed notice
- run_turn: second _UNEXPECTED_SILENCE_REPLY consumer -> accessor

* i18n(discord): route adapter chat copy through t()

Native slash-command table becomes _NATIVE_SLASH_COMMAND_SPECS holding catalog
keys; _native_slash_commands() resolves descriptions, parameter descriptions
and Choice names for the active language, each cut at Discord's 100-char cap,
and the app-command sync fingerprint now includes get_language() so a
display.language change re-syncs. Exec-approval card (gateway.exec_approval.*
contract), slash-confirm / clarify / update views, model+choice pickers,
thread creation, forum titles, voice acks, the response-truncation notice,
the unauthorized-slash security alert and the media upload-size notices all
read from platform.discord.*. Decorator-declared button labels are relabelled
in __init__ (80-char cap); embed titles cut at 256, select placeholders at
150, option label/description at 100. _UNAUTHORIZED is a lazy _unauthorized().

* i18n(cli): wire status-bar, stream, terminal mixins + terminal_input through t(); rename kwargs that shadow t(key)

* i18n: wire hermes_cli/cli_commands_mixin.py slash-command copy through t()

- 431 new leaves under cli.commands.<cmd>.* in locales/en.yaml; 12 rows reuse
  existing gateway.* keys (rollback, diff, resume, branch, btw, model, reasoning)
  via a _gt() helper so CLI and gateway replies stay identical.
- Module-level English tables (_BUSY_MODE_*, _REASONING_TOGGLES, _HATCH_PROGRESS,
  _DIFF_LABELS, _LOCAL_ENGINE_LINES) become call-time catalog lookups keyed by id.
- Verb tables (Enabling/Disabling, Paused/Resumed/Triggered, planned/done,
  Updating/Generating) are one full template per variant; plurals use
  <key>_one/<key>_other via _tn(); hand-padded column labels (/snapshot list)
  translate the value and re-pad at the call site.
- Multi-line usage blocks are single catalog values split with _lines().
- Model-facing system notes and DB-stored reasons stay English (EXCLUDED).

* tests: assert /handoff, /worktree, /login CLI copy via t(key) instead of pinned English

* test(i18n): pin Telegram/Discord adapter catalog wiring

Lazy unauthorized notice, exec-approval contract keys, HTML escaping before
Telegram <b> wrapping, 200-char toast / 256-char BotCommand caps, Discord
100-char app-command text and 80-char button caps, and language-bearing
command-menu fingerprints on both platforms.

* i18n: reconcile cli.shared on/off vs enabled/disabled after lane merge

* i18n: describe() in TUI-gateway slash listings; localize TUI exit resume hint

* i18n(tr): translate bundled catalog + tui pack

* i18n(ja): translate bundled catalog + tui pack

* i18n(ko): translate bundled catalog + tui pack

* i18n(zh): translate bundled catalog + tui pack

* i18n(fr): translate bundled catalog + tui pack

* i18n(af): translate bundled catalog + tui pack

* i18n(uk): translate bundled catalog + tui pack

* i18n(ar): translate bundled catalog + tui pack

* i18n(pt): translate bundled catalog + tui pack

* i18n(it): translate bundled catalog + tui pack

* i18n(es): translate bundled catalog + tui pack

* i18n(zh-hant): translate bundled catalog + tui pack

* i18n(ru): translate bundled catalog + tui pack

* i18n(hu): translate bundled catalog + tui pack

* i18n(hu): translate pre-existing English-valued leftovers (kanban wake, /context, /status, fast labels)

* i18n(de): translate bundled catalog + tui pack

* i18n(ga): translate bundled catalog + tui pack

* test(i18n): fixture matches _normalize_lang(lang, home) signature

* i18n(tui): scaffold userMessages/slashCmd en siblings

* i18n(tui): wire secure prompts + content tables

* feat(tui): i18n — wire billing, subscription, connection-setup and journey overlays

Adds en siblings billing.ts / subscription.ts / connection.ts (namespaces
billing, subscription, connection, journey) and routes every user-facing
literal in billingOverlay, subscriptionOverlay, connectionSetupOverlay and
journey through useT()/messages(). Module-level label tables became lazy
(scopeStillDeniedResult(), verbOf(T, action)); auto-reload rows dispatch on
stable ids instead of label text. Regenerates locales/_keys.tui.json.

* i18n(tui): wire slash ops/wake replies

* i18n(tui): wire pickers (modelPicker, activeSessionSwitcher, petPicker)

* i18n(tui): wire slash core/debug/setup replies

* i18n(tui): wire hubs (agents overlay/panel/controls, skills, plugins)

* i18n(tui): wire slash session/topup/subscription replies

* i18n(tui): wire chat bits (branding, thinking, messageLine, loaders, todo, queued, banner, entry)

* i18n(tui): register t3 siblings (pickers, hubs, secure, content, chatBits) and regenerate keys

* i18n(tui): wire userMessages copy through the userMessages namespace

* i18n(tui): lazy-copy test for userMessages, regenerate _keys.tui.json

* i18n(tui): wire session/gateway/lib text through the TUI catalog (lane t2)

Adds en siblings session.ts, gatewayMsg.ts, libText.ts (namespaces session,
gatewayMsg, libText) and routes user-facing literals in app/{useMainApp,
useSessionLifecycle,useInputHandlers,turnController,createServerRequestHandler,
setupHandoff,createGatewayEventHandler}.ts, gatewayClient displayed reasons,
lib/*, domain/*, hooks/* through t()/messages(). Status-bar state values that
code compares against, backend-matched strings, log lines, model-bound text,
and machine 'error:' prefixes stay literal. Regenerates locales/_keys.tui.json
(232 keys).

* i18n: serve bundled locales/<lang>.tui.yaml under overlay/packs; TUI pack parity test; regen _keys.tui.json (1250)

* i18n: translate pre-existing English stubs in bundled locales (424 leaves, 14 locales)

* tui: i18n-export-en script (English templates for pack translators)

* docs(i18n): bundled TUI packs are the bottom layer of the tui surface

* i18n(ru): translate TUI pack

* i18n(ar): translate TUI pack

* i18n(es): translate TUI pack

* i18n(pt): translate TUI pack

* i18n(ko): translate TUI pack

* i18n(de): translate TUI pack

* i18n(ja): translate TUI pack

* i18n(fr): translate TUI pack

* i18n(tr): translate TUI pack

* i18n(it): translate TUI pack

* i18n(zh): translate TUI pack

* i18n(zh-hant): translate TUI pack

* i18n(hu): translate TUI pack

* i18n(uk): translate TUI pack

1,169 missing keys translated; 81 pre-existing kept byte-identical. Parity OK missing=0 extra=0 placeholder_mismatch=0 empty=0.

Deliberately identical to en: chatBits.branding.mcpSummary ({0} MCP), chatBits.thinking.agentsHint ((/agents)), session.main.voiceStt (◉ STT), session.main.voiceTtsSuffix ( [tts]), slashCmd.core.help.tuiSection (TUI), slashCmd.core.history.hermesTag (Hermes #{0}), slashCmd.debug.heapdump.heapPath (heapdump: {0}), slashCmd.debug.mem.rss (rss), subscription.stepUp.title (Remote Spending — product feature name, as in core catalog), content.faces.* (glyph-only kaomoji).

* i18n(ga): translate TUI pack

* i18n(af): translate TUI pack

* plugin_guard: locale catalogs in language packs step down the agent-config family

A translated status line such as "Updating AGENTS.md" in locales/<lang>.yaml is UI text the loader
reads as a string leaf; it cannot edit a file. The bundled en.yaml itself tripped agent_config_mod
at critical, making any faithful language pack uninstallable. Injection shapes keep full severity.

* plugin_validate_locales: read key exports with utf-8-sig (Windows footgun lint)

* i18n(relay): route relay adapter prompt copy through t(); drop dead import-bound approval header

Adds platform.relay.* (5 keys) to en and all 16 bundled locales, reusing the sibling platform
translations for the confirm buttons and the Other option.

* ci: fix TUI import order, MDX table pipe, main's overflow-warning wording in all locales; fresh-install fixture carries the i18n kernel

- ui-tui/src/i18n/en.ts: perfectionist/sort-imports (slash before slashCmd)
- docs plugins/index.md: escape the | inside the provides_locales table cell (MDX parsed <id> as JSX)
- display.notice.uncompressed_context_overflow: adopt main's wording (names compression.enabled: false
  and /compact) in en + 16 locales; the guardrail test pins that phrase
- tests/scripts/test_fresh_source_install.py: the installer tail now resolves CLI text through
  agent.i18n, so the fixture tree carries the i18n kernel + en.yaml (not the agent runtime)

* docs(desktop-plugin-sdk): double-backtick the template-literal example (MDX evaluated ${n})

* test(e2e): display.language is validated against the live language set; exclude it from the arbitrary-string set property

* commands: keep the localized COMMANDS/COMMANDS_BY_CATEGORY module __getattr__ after the compat block removal

* build: never write locales/_keys.*.json from the desktop/TUI builds; regenerate the committed desktop key export

The desktop build regenerated locales/_keys.desktop.json in the checkout, so a
hermes update that rebuilt the app left the tree dirty (Desktop update E2E:
'M locales/_keys.desktop.json'). The key exports are committed artifacts pinned
to en.ts by apps/desktop/scripts/i18n-keys.test.mjs and ui-tui i18n:keys:check;
builds read them, never write them. Regenerated after main's new desktop strings.

* test: unbreak two main-red timing tests the PR merge-ref inherits

- test_local_runtime racing fake publishes the modern state record (legacy pid-only
  records are rejected since 65ff3ad353; main has been red on this test since)
- test_run_progress_topics ManyProgressLinesAgent waits for the first bubble instead of
  a fixed 0.35s, which a loaded CI runner does not always meet

* chore(i18n): regenerate desktop key catalog for main's new strings (model pricing, copy changelog)

* test(e2e): torture-chamber fd monitor confirms a deleted sidecar is still held before calling it a leak

SQLite's WAL last-close unlinks -shm before closing its descriptor (unixShmUnmap, then
unixShmPurge), so a healthy close shows a (deleted) -shm for microseconds; the 20ms poll
occasionally caught that window on the short-lived opener and failed the episode.

* chore(i18n): regenerate desktop key catalog for main's telemetry/consent strings

* chore(i18n): regenerate desktop key catalog after main sync

---------

Co-authored-by: Teknium <[email protected]>
2026-09-28 14:16:18 -07:00

1304 lines
55 KiB
Python

"""Tests for the Email gateway platform adapter.
Covers:
1. Platform enum exists with correct value
2. Config loading from env vars via _apply_env_overrides
3. Adapter init and config parsing
4. Helper functions (header decoding, body extraction, address extraction, HTML stripping)
5. Authorization integration (platform in allowlist maps)
6. Send message tool routing (platform in platform_map)
7. check_email_requirements function
8. Attachment extraction and caching
9. Message dispatch and threading
"""
import os
import unittest
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from unittest.mock import patch, MagicMock, ANY
class TestConfigEnvOverrides(unittest.TestCase):
"""Verify email config is loaded from environment variables."""
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_HOME_ADDRESS": "[email protected]",
}, clear=False)
def test_email_home_channel_loaded(self):
from gateway.config import GatewayConfig, Platform, _apply_env_overrides
config = GatewayConfig()
_apply_env_overrides(config)
home = config.platforms[Platform.EMAIL].home_channel
self.assertIsNotNone(home)
self.assertEqual(home.chat_id, "[email protected]")
class TestCheckRequirements(unittest.TestCase):
"""Verify check_email_requirements function."""
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "pw",
"EMAIL_IMAP_HOST": "imap.b.com",
"EMAIL_SMTP_HOST": "smtp.b.com",
}, clear=False)
def test_requirements_met(self):
from plugins.platforms.email.adapter import check_email_requirements
self.assertTrue(check_email_requirements())
class TestHelperFunctions(unittest.TestCase):
"""Test email parsing helper functions."""
def test_decode_header_encoded(self):
from plugins.platforms.email.adapter import _decode_header_value
# RFC 2047 encoded subject
encoded = "=?utf-8?B?TWVyaGFiYQ==?=" # "Merhaba" in base64
result = _decode_header_value(encoded)
self.assertEqual(result, "Merhaba")
def test_extract_email_address_with_name(self):
from plugins.platforms.email.adapter import _extract_email_address
self.assertEqual(
_extract_email_address("John Doe <[email protected]>"),
"[email protected]"
)
# Unquoted forms strict parseaddr rejects still resolve to their single bracketed address.
for raw in ("[email protected] <[email protected]>", "Doe, John <[email protected]>",
"Doe, John (CEO) <[email protected]>"):
self.assertEqual(_extract_email_address(raw), "[email protected]", raw)
# ...but never when the display part could hold another mailbox, group or comment.
for raw in ("[email protected], [email protected]", "Group: [email protected], [email protected];", "<>", "[email protected] <[email protected]>",
"[email protected], <victim@x>", "[email protected],\r\n <victim@x>",
"[email protected] (c) <victim@x>", "[email protected]; <victim@x>",
"Grp: [email protected]; <victim@x>", "undisclosed-recipients:; <victim@x>",
"John", 'a\\"b <victim@x>',
# over _MAX_FROM_LEN (uncapped parseaddr would return [email protected])
"x" * 3000 + " <[email protected]>",
# >=500 nested comments make stdlib parseaddr raise RecursionError
"Doe, " + "(" * 500 + ")" * 500 + " <[email protected]>"):
self.assertEqual(_extract_email_address(raw), "", raw)
# A From with no usable address is dropped at parse time, before dispatch.
from plugins.platforms.email.adapter import EmailAdapter
self.assertIsNone(EmailAdapter._parse_fetched_message(
object.__new__(EmailAdapter), b"2", b"From: [email protected], [email protected]\r\nSubject: x\r\n\r\nbody"))
def test_extract_email_address_ignores_angle_brackets_in_display_name(self):
from plugins.platforms.email.adapter import _extract_email_address
self.assertEqual(
_extract_email_address('"Victim <[email protected]>" <[email protected]>'),
"[email protected]",
)
def test_extract_email_address_unfolds_display_name_before_parsing(self):
from plugins.platforms.email.adapter import _extract_email_address
self.assertEqual(
_extract_email_address(
'"Some Very Long Display Name That Exceeds The Line\r\n Limit" <[email protected]>'
),
"[email protected]",
)
# Ordinary forms must keep resolving the same way.
self.assertEqual(_extract_email_address("Plain <[email protected]>"), "[email protected]")
self.assertEqual(_extract_email_address("[email protected]"), "[email protected]")
def test_strip_html_basic(self):
from plugins.platforms.email.adapter import _strip_html
html = "<p>Hello <b>world</b></p>"
result = _strip_html(html)
self.assertIn("Hello", result)
self.assertIn("world", result)
self.assertNotIn("<p>", result)
self.assertNotIn("<b>", result)
class TestExtractTextBody(unittest.TestCase):
"""Test email body extraction from different message formats."""
def test_plain_text_body(self):
from plugins.platforms.email.adapter import _extract_text_body
msg = MIMEText("Hello, this is a test.", "plain", "utf-8")
result = _extract_text_body(msg)
self.assertEqual(result, "Hello, this is a test.")
def test_multipart_prefers_plain(self):
from plugins.platforms.email.adapter import _extract_text_body
msg = MIMEMultipart("alternative")
msg.attach(MIMEText("<p>HTML version</p>", "html", "utf-8"))
msg.attach(MIMEText("Plain version", "plain", "utf-8"))
result = _extract_text_body(msg)
self.assertEqual(result, "Plain version")
class TestDispatchMessage(unittest.TestCase):
"""Test email message dispatch logic."""
def setUp(self):
# These tests exercise dispatch mechanics (subject formatting,
# attachment typing, source building), not the authorization gate.
# The adapter now fails closed at dispatch when no allowlist / allow-all
# is configured (SECURITY.md 2.6), so opt into allow-all here to keep
# exercising the dispatch path. Auth-contract tests below override this.
self._prev_allow_all = os.environ.get("EMAIL_ALLOW_ALL_USERS")
os.environ["EMAIL_ALLOW_ALL_USERS"] = "true"
def tearDown(self):
if self._prev_allow_all is None:
os.environ.pop("EMAIL_ALLOW_ALL_USERS", None)
else:
os.environ["EMAIL_ALLOW_ALL_USERS"] = self._prev_allow_all
def _make_adapter(self):
"""Create an EmailAdapter with mocked env vars."""
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_IMAP_PORT": "993",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_SMTP_PORT": "587",
"EMAIL_POLL_INTERVAL": "15",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_self_message_filtered(self):
"""Messages from the agent's own address should be skipped."""
import asyncio
adapter = self._make_adapter()
adapter._message_handler = MagicMock()
msg_data = {
"uid": b"1",
"sender_addr": "[email protected]",
"sender_name": "Hermes",
"subject": "Test",
"message_id": "<[email protected]>",
"in_reply_to": "",
"body": "Self message",
"attachments": [],
"date": "",
}
asyncio.run(adapter._dispatch_message(msg_data))
adapter._message_handler.assert_not_called()
def test_subject_included_in_text(self):
"""Subject should be prepended to body for non-reply emails."""
import asyncio
adapter = self._make_adapter()
captured_events = []
async def mock_handler(event):
captured_events.append(event)
return None
adapter._message_handler = mock_handler
# Override handle_message to capture the event directly
async def capture_handle(event):
captured_events.append(event)
adapter.handle_message = capture_handle
msg_data = {
"uid": b"2",
"sender_addr": "[email protected]",
"sender_name": "User",
"subject": "Help with Python",
"message_id": "<[email protected]>",
"in_reply_to": "",
"body": "How do I use lists?",
"attachments": [],
"date": "",
}
asyncio.run(adapter._dispatch_message(msg_data))
self.assertEqual(len(captured_events), 1)
self.assertIn("[Subject: Help with Python]", captured_events[0].text)
self.assertIn("How do I use lists?", captured_events[0].text)
def test_reply_subject_not_duplicated(self):
"""Re: subjects should not be prepended to body."""
import asyncio
adapter = self._make_adapter()
captured_events = []
async def capture_handle(event):
captured_events.append(event)
adapter.handle_message = capture_handle
msg_data = {
"uid": b"3",
"sender_addr": "[email protected]",
"sender_name": "User",
"subject": "Re: Help with Python",
"message_id": "<[email protected]>",
"in_reply_to": "<[email protected]>",
"body": "Thanks for the help!",
"attachments": [],
"date": "",
}
asyncio.run(adapter._dispatch_message(msg_data))
self.assertEqual(len(captured_events), 1)
self.assertNotIn("[Subject:", captured_events[0].text)
self.assertEqual(captured_events[0].text, "Thanks for the help!")
def test_image_attachment_sets_photo_type(self):
"""Email with image attachment should set message type to PHOTO."""
import asyncio
from gateway.platforms.event import MessageType
adapter = self._make_adapter()
captured_events = []
async def capture_handle(event):
captured_events.append(event)
adapter.handle_message = capture_handle
msg_data = {
"uid": b"5",
"sender_addr": "[email protected]",
"sender_name": "User",
"subject": "Re: photo",
"message_id": "<[email protected]>",
"in_reply_to": "",
"body": "Check this photo",
"attachments": [{"path": "/tmp/img.jpg", "filename": "img.jpg", "type": "image", "media_type": "image/jpeg"}],
"date": "",
}
asyncio.run(adapter._dispatch_message(msg_data))
self.assertEqual(len(captured_events), 1)
self.assertEqual(captured_events[0].message_type, MessageType.PHOTO)
self.assertEqual(captured_events[0].media_urls, ["/tmp/img.jpg"])
def test_empty_allowlist_denies_without_optin(self):
"""No allowlist and no allow-all opt-in → adapter fails closed (2.6)."""
import asyncio
with patch.dict(os.environ, {}, clear=False):
# No allowlist, and explicitly no allow-all opt-in.
for k in ("EMAIL_ALLOWED_USERS", "EMAIL_ALLOW_ALL_USERS",
"GATEWAY_ALLOW_ALL_USERS"):
os.environ.pop(k, None)
adapter = self._make_adapter()
adapter._message_handler = MagicMock()
msg_data = {
"uid": b"101",
"sender_addr": "[email protected]",
"sender_name": "Anyone",
"subject": "Hey",
"message_id": "<[email protected]>",
"in_reply_to": "",
"body": "Hi",
"attachments": [],
"date": "",
}
asyncio.run(adapter._dispatch_message(msg_data))
# Fail closed: an unset allowlist without allow-all drops the sender.
adapter._message_handler.assert_not_called()
def test_unauthenticated_allowed_with_allow_all(self):
"""EMAIL_ALLOW_ALL_USERS=true makes sender identity moot — gate skipped.
With allow-all and no restrictive allowlist, an unauthenticated sender
is forwarded: the operator has explicitly chosen to accept anyone.
"""
import asyncio
with patch.dict(os.environ, {
"EMAIL_ALLOW_ALL_USERS": "true",
}):
os.environ.pop("EMAIL_ALLOWED_USERS", None)
os.environ.pop("GATEWAY_ALLOWED_USERS", None)
adapter = self._make_adapter()
captured = []
async def capture_handle(event):
captured.append(event)
adapter.handle_message = capture_handle
msg_data = {
"uid": b"203",
"sender_addr": "[email protected]",
"sender_name": "Stranger",
"subject": "Hi",
"message_id": "<[email protected]>",
"in_reply_to": "",
"body": "Hello",
"attachments": [],
"date": "",
"sender_authenticated": False,
"auth_reason": "no Authentication-Results header",
}
asyncio.run(adapter._dispatch_message(msg_data))
self.assertEqual(len(captured), 1)
class TestDispatchDefersToGatewayAuthorization(unittest.TestCase):
"""The pre-dispatch gate must not drop mail the gateway would authorize (GATEWAY_ALLOWED_USERS,
an approved pairing) or answer itself (an explicit pair/decline unauthorized_dm_behavior)."""
STRANGER = "[email protected]"
def setUp(self):
self._env = patch.dict(os.environ, {}, clear=False)
self._env.start()
for key in ("EMAIL_ALLOWED_USERS", "EMAIL_ALLOW_ALL_USERS", "GATEWAY_ALLOWED_USERS",
"GATEWAY_ALLOW_ALL_USERS", "EMAIL_TRUST_FROM_HEADER"):
os.environ.pop(key, None)
def tearDown(self):
self._env.stop()
def _reached_gateway(self, *, extra=None, env=None, paired=False, authenticated=True):
"""Dispatch one mail from STRANGER with the real GatewayRunner auth callback wired, as startup does;
return the events handed to the gateway. Each call gets its own pairing store."""
import asyncio
import tempfile
from pathlib import Path
from gateway.config import GatewayConfig, Platform, PlatformConfig
from gateway.pairing import PairingStore
from gateway.run import GatewayRunner
from plugins.platforms.email.adapter import EmailAdapter
with tempfile.TemporaryDirectory() as pairing_dir, \
patch("gateway.pairing.PAIRING_DIR", Path(pairing_dir)), \
patch.dict(os.environ, {"EMAIL_ADDRESS": "[email protected]", "EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com", "EMAIL_SMTP_HOST": "smtp.test.com",
**(env or {})}):
adapter = EmailAdapter(PlatformConfig(enabled=True, extra=dict(extra or {})))
runner = object.__new__(GatewayRunner)
runner.config = GatewayConfig(platforms={Platform.EMAIL: adapter.config})
runner.adapters = {Platform.EMAIL: adapter}
runner.pairing_store = PairingStore()
adapter.set_authorization_check(runner._make_adapter_auth_check(Platform.EMAIL))
if paired:
code = runner.pairing_store.generate_code("email", self.STRANGER, "Stranger")
self.assertIsNotNone(runner.pairing_store.approve_code("email", code))
captured = []
async def capture(event):
captured.append(event)
adapter.handle_message = capture
asyncio.run(adapter._dispatch_message({
"uid": b"301", "sender_addr": self.STRANGER, "sender_name": "Stranger", "subject": "Hello",
"message_id": "<[email protected]>", "in_reply_to": "", "body": "Hi there", "attachments": [],
"date": "", "sender_authenticated": authenticated,
"auth_reason": "dmarc=pass" if authenticated else "no Authentication-Results header"}))
return captured
def test_mail_the_gateway_admits_or_answers_reaches_it(self):
cases = {
"pair opt-in": {"extra": {"unauthorized_dm_behavior": "pair"}},
"decline opt-in": {"extra": {"unauthorized_dm_behavior": "decline"}},
"GATEWAY_ALLOWED_USERS": {"env": {"GATEWAY_ALLOWED_USERS": self.STRANGER}},
"EMAIL_ALLOWED_USERS JSON list literal": {"env": {"EMAIL_ALLOWED_USERS": f'["{self.STRANGER}"]'}},
"approved pairing": {"paired": True},
}
for label, kwargs in cases.items():
with self.subTest(label):
self.assertEqual(len(self._reached_gateway(**kwargs)), 1)
def test_mail_the_gateway_would_ignore_or_that_forges_from_is_dropped(self):
cases = {
"default ignore": {},
"pair opt-in, unauthenticated From": {"extra": {"unauthorized_dm_behavior": "pair"}, "authenticated": False},
"approved pairing, unauthenticated From": {"paired": True, "authenticated": False},
# Open access grants a stranger nothing beside a list, so a pairing code must not go to a forged From:.
"pair opt-in, allow-all beside EMAIL list, unauthenticated From": {
"extra": {"unauthorized_dm_behavior": "pair"}, "authenticated": False,
"env": {"GATEWAY_ALLOW_ALL_USERS": "true", "EMAIL_ALLOWED_USERS": "[email protected]"}},
# GATEWAY_ALLOW_ALL_USERS is inert beside a list, so a listed address still has to authenticate its From:.
"listed sender, GATEWAY allow-all beside the list, unauthenticated From": {
"authenticated": False, "env": {"GATEWAY_ALLOW_ALL_USERS": "true", "EMAIL_ALLOWED_USERS": self.STRANGER}},
"pair opt-in, allow-all beside GATEWAY list, unauthenticated From": {
"extra": {"unauthorized_dm_behavior": "pair"}, "authenticated": False,
"env": {"GATEWAY_ALLOW_ALL_USERS": "true", "GATEWAY_ALLOWED_USERS": "[email protected]"}},
# A bare entry (a chat username, say) names one principal, never stranger@<any domain>: the
# domain is the sender's to choose, so such mail is dropped rather than admitted or paired.
"GATEWAY_ALLOWED_USERS bare entry": {"env": {"GATEWAY_ALLOWED_USERS": "stranger"}},
"EMAIL_ALLOWED_USERS bare entry, JSON list literal": {"env": {"EMAIL_ALLOWED_USERS": '["stranger"]'}},
"bare entry, pair opt-in": {"env": {"GATEWAY_ALLOWED_USERS": "stranger"},
"extra": {"unauthorized_dm_behavior": "pair"}},
}
for label, kwargs in cases.items():
with self.subTest(label):
self.assertEqual(self._reached_gateway(**kwargs), [])
class TestThreadContext(unittest.TestCase):
"""Test email reply threading logic."""
def setUp(self):
# Thread-context storage is a dispatch-mechanics test, not an auth test.
# The adapter fails closed at dispatch without allow-all (SECURITY.md 2.6),
# so opt into allow-all to keep exercising the threading path.
self._prev_allow_all = os.environ.get("EMAIL_ALLOW_ALL_USERS")
os.environ["EMAIL_ALLOW_ALL_USERS"] = "true"
def tearDown(self):
if self._prev_allow_all is None:
os.environ.pop("EMAIL_ALLOW_ALL_USERS", None)
else:
os.environ["EMAIL_ALLOW_ALL_USERS"] = self._prev_allow_all
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_reply_uses_re_prefix(self):
"""Reply subject should have Re: prefix."""
adapter = self._make_adapter()
adapter._thread_context["[email protected]"] = {
"subject": "Project question",
"message_id": "<[email protected]>",
}
with patch("smtplib.SMTP") as mock_smtp:
mock_server = MagicMock()
mock_smtp.return_value = mock_server
adapter._send_email("[email protected]", "Here is the answer.", None)
# Check the sent message
send_call = mock_server.send_message.call_args[0][0]
self.assertEqual(send_call["Subject"], "Re: Project question")
self.assertEqual(send_call["In-Reply-To"], "<[email protected]>")
self.assertEqual(send_call["References"], "<[email protected]>")
self.assertIn("Date", send_call)
class TestSendMethods(unittest.TestCase):
"""Test email send methods."""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_send_document_with_attachment(self):
"""send_document should send email with file attachment."""
import asyncio
import tempfile
adapter = self._make_adapter()
with tempfile.NamedTemporaryFile(suffix=".txt", delete=False) as f:
f.write(b"Test document content")
tmp_path = f.name
try:
with patch("smtplib.SMTP") as mock_smtp:
mock_server = MagicMock()
mock_smtp.return_value = mock_server
result = asyncio.run(
adapter.send_document("[email protected]", tmp_path, "Here is the file")
)
self.assertTrue(result.success)
mock_server.send_message.assert_called_once()
sent_msg = mock_server.send_message.call_args[0][0]
# Should be multipart with attachment
parts = list(sent_msg.walk())
has_attachment = any(
"attachment" in str(p.get("Content-Disposition", ""))
for p in parts
)
self.assertTrue(has_attachment)
finally:
os.unlink(tmp_path)
def test_send_document_threads_on_explicit_reply_to(self):
"""An explicit reply_to wins over the cached thread context for attachment sends (#10131)."""
import asyncio
import tempfile
adapter = self._make_adapter()
adapter._thread_context["[email protected]"] = {"subject": "Old", "message_id": "<[email protected]>"}
with tempfile.NamedTemporaryFile(suffix=".txt", delete=False) as f:
f.write(b"doc")
tmp_path = f.name
try:
with patch("smtplib.SMTP") as mock_smtp:
mock_server = MagicMock()
mock_smtp.return_value = mock_server
result = asyncio.run(adapter.send_document("[email protected]", tmp_path, reply_to="<[email protected]>"))
self.assertTrue(result.success)
sent_msg = mock_server.send_message.call_args[0][0]
self.assertEqual(sent_msg["In-Reply-To"], "<[email protected]>")
self.assertEqual(sent_msg["References"], "<[email protected]>")
finally:
os.unlink(tmp_path)
class TestConnectDisconnect(unittest.TestCase):
"""Test IMAP/SMTP connection lifecycle."""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_connect_success(self):
"""Successful IMAP + SMTP connection returns True."""
import asyncio
adapter = self._make_adapter()
mock_imap = MagicMock()
mock_imap.uid.return_value = ("OK", [b"1 2 3"])
with patch("imaplib.IMAP4_SSL", return_value=mock_imap), \
patch("smtplib.SMTP") as mock_smtp:
mock_server = MagicMock()
mock_smtp.return_value = mock_server
result = asyncio.run(adapter.connect())
self.assertTrue(result)
self.assertTrue(adapter._running)
# Should have skipped existing messages
self.assertEqual(len(adapter._seen_uids), 3)
# Cleanup
adapter._running = False
if adapter._poll_task:
adapter._poll_task.cancel()
class TestFetchNewMessages(unittest.TestCase):
"""Test IMAP message fetching logic."""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_fetch_skips_seen_uids(self):
"""Already-seen UIDs should not be fetched again."""
adapter = self._make_adapter()
adapter._seen_uids = {b"1", b"2"}
raw_email = MIMEText("Hello", "plain", "utf-8")
raw_email["From"] = "[email protected]"
raw_email["Subject"] = "Test"
raw_email["Message-ID"] = "<[email protected]>"
mock_imap = MagicMock()
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1 2 3"])
if command == "fetch":
return ("OK", [(b"3", raw_email.as_bytes())])
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
results = adapter._fetch_new_messages()
# Only UID 3 should be fetched (1 and 2 already seen)
self.assertEqual(len(results), 1)
self.assertEqual(results[0]["sender_addr"], "[email protected]")
self.assertIn(b"3", adapter._seen_uids)
class TestPollLoop(unittest.TestCase):
"""Test the async polling loop."""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_POLL_INTERVAL": "1",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_check_inbox_dispatches_messages(self):
"""_check_inbox should fetch and dispatch new messages."""
import asyncio
adapter = self._make_adapter()
dispatched = []
async def mock_dispatch(msg_data):
dispatched.append(msg_data)
adapter._dispatch_message = mock_dispatch
raw_email = MIMEText("Test body", "plain", "utf-8")
raw_email["From"] = "[email protected]"
raw_email["Subject"] = "Inbox Test"
raw_email["Message-ID"] = "<[email protected]>"
mock_imap = MagicMock()
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1"])
if command == "fetch":
return ("OK", [(b"1", raw_email.as_bytes())])
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
asyncio.run(adapter._check_inbox())
self.assertEqual(len(dispatched), 1)
self.assertEqual(dispatched[0]["subject"], "Inbox Test")
def test_check_inbox_notifies_fatal_error_on_fetch_failure(self):
"""A failed IMAP check must surface through the fatal-error hook so
the gateway's reconnect/backoff machinery learns email is unhealthy
instead of silently treating the failed check as an empty inbox
(#80016)."""
import asyncio
adapter = self._make_adapter()
notified = []
async def mock_fatal_handler(adapter):
notified.append(adapter)
adapter.set_fatal_error_handler(mock_fatal_handler)
mock_imap = MagicMock()
mock_imap.login.side_effect = Exception("read operation timed out")
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
asyncio.run(adapter._check_inbox())
self.assertEqual(len(notified), 1)
self.assertEqual(adapter.fatal_error_code, "email_imap_fetch_failed")
self.assertTrue(adapter.fatal_error_retryable)
self.assertIn("read operation timed out", adapter.fatal_error_message)
def test_partial_batch_dispatched_before_escalation(self):
"""A mid-batch IMAP failure must dispatch the messages already
fetched BEFORE escalating — dropping them would lose mail, since
their UIDs are marked seen (#80032 review)."""
import asyncio
adapter = self._make_adapter()
dispatched, notified = [], []
async def mock_dispatch(msg_data):
dispatched.append(msg_data)
async def mock_fatal_handler(a):
notified.append(a)
adapter._dispatch_message = mock_dispatch
adapter.set_fatal_error_handler(mock_fatal_handler)
raw_email = MIMEText("Body", "plain", "utf-8")
raw_email["From"] = "[email protected]"
raw_email["Subject"] = "First of batch"
raw_email["Message-ID"] = "<[email protected]>"
mock_imap = MagicMock()
fetches = []
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1 2"])
if command == "fetch":
fetches.append(args)
if len(fetches) == 1:
return ("OK", [(b"1", raw_email.as_bytes())])
raise OSError("connection dropped mid-batch")
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
asyncio.run(adapter._check_inbox())
# The successfully fetched message was dispatched, not dropped.
self.assertEqual(len(dispatched), 1)
self.assertEqual(dispatched[0]["subject"], "First of batch")
# The failure still escalated through the fatal-error hook.
self.assertEqual(len(notified), 1)
self.assertEqual(adapter.fatal_error_code, "email_imap_fetch_failed")
def test_mid_batch_failure_leaves_unfetched_uids_eligible(self):
"""UIDs are marked seen only after their fetch returns — a
connection failure mid-batch must leave the remaining UIDs eligible
for the next poll instead of permanently skipping them."""
adapter = self._make_adapter()
raw_email = MIMEText("Body", "plain", "utf-8")
raw_email["From"] = "[email protected]"
raw_email["Subject"] = "ok"
raw_email["Message-ID"] = "<[email protected]>"
mock_imap = MagicMock()
fetches = []
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1 2 3"])
if command == "fetch":
fetches.append(args)
if len(fetches) == 1:
return ("OK", [(b"1", raw_email.as_bytes())])
raise OSError("connection dropped")
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
results = adapter._fetch_new_messages()
self.assertEqual(len(results), 1)
self.assertIn(b"1", adapter._seen_uids) # fetched → seen
self.assertNotIn(b"2", adapter._seen_uids) # fetch raised → retry next poll
self.assertNotIn(b"3", adapter._seen_uids) # never reached → retry next poll
self.assertTrue(adapter._last_fetch_failed)
def test_poison_message_skipped_once_without_escalation(self):
"""A message whose processing raises is marked seen and skipped —
it must not abort the batch, escalate to a reconnect, or be
retried forever (#80032 review)."""
adapter = self._make_adapter()
good_email = MIMEText("Body", "plain", "utf-8")
good_email["From"] = "[email protected]"
good_email["Subject"] = "good"
good_email["Message-ID"] = "<[email protected]>"
mock_imap = MagicMock()
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1 2"])
if command == "fetch":
uid = args[0]
if uid == b"1":
return ("OK", [(b"1", b"poison")])
return ("OK", [(b"2", good_email.as_bytes())])
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap), patch(
"plugins.platforms.email.adapter.EmailAdapter._parse_fetched_message",
side_effect=[ValueError("unparseable"), {"subject": "good"}],
):
results = adapter._fetch_new_messages()
# Poison message consumed (seen, skipped); good message survived.
self.assertEqual(len(results), 1)
self.assertIn(b"1", adapter._seen_uids)
self.assertIn(b"2", adapter._seen_uids)
self.assertFalse(adapter._last_fetch_failed)
class TestReconnectSeenUidsRestore(unittest.TestCase):
"""connect(is_reconnect=True) must not re-mark the whole mailbox seen."""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def setUp(self):
from plugins.platforms.email.adapter import EmailAdapter
EmailAdapter._seen_uids_snapshot.clear()
tearDown = setUp
def _run_connect(self, adapter, mailbox_uids, *, is_reconnect):
import asyncio
mock_imap = MagicMock()
def uid_handler(command, *args):
if command == "search":
return ("OK", [mailbox_uids])
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
smtp = MagicMock()
with patch("imaplib.IMAP4_SSL", return_value=mock_imap), patch.object(
adapter, "_connect_smtp", return_value=smtp
):
return asyncio.run(adapter.connect(is_reconnect=is_reconnect))
def test_reconnect_restores_snapshot_instead_of_marking_all_seen(self):
# First adapter connects with UIDs 1-2 in the mailbox.
first = self._make_adapter()
self.assertTrue(self._run_connect(first, b"1 2", is_reconnect=False))
self.assertEqual(first._seen_uids, {b"1", b"2"})
import asyncio
asyncio.run(first.disconnect())
# Outage: UID 3 arrives. The reconnect watcher builds a FRESH adapter
# and connects with is_reconnect=True.
second = self._make_adapter()
self.assertTrue(self._run_connect(second, b"1 2 3", is_reconnect=True))
# Baseline restored from the snapshot — UID 3 stays eligible.
self.assertEqual(second._seen_uids, {b"1", b"2"})
asyncio.run(second.disconnect())
def test_first_connect_still_marks_all_seen(self):
adapter = self._make_adapter()
self.assertTrue(self._run_connect(adapter, b"7 8 9", is_reconnect=False))
self.assertEqual(adapter._seen_uids, {b"7", b"8", b"9"})
import asyncio
asyncio.run(adapter.disconnect())
def test_reconnect_without_snapshot_falls_back_to_mark_all_seen(self):
# e.g. gateway restarted: no in-process snapshot exists.
adapter = self._make_adapter()
self.assertTrue(self._run_connect(adapter, b"4 5", is_reconnect=True))
self.assertEqual(adapter._seen_uids, {b"4", b"5"})
import asyncio
asyncio.run(adapter.disconnect())
class TestSendEmailStandalone(unittest.TestCase):
"""Test the standalone _send_email function in send_message_tool."""
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_SMTP_PORT": "587",
})
def test_send_email_tool_success(self):
"""_send_email should use verified STARTTLS when sending."""
import asyncio
import ssl
from plugins.platforms.email.adapter import _standalone_send as _email_send
from types import SimpleNamespace
async def _send_email(extra, chat_id, message):
return await _email_send(SimpleNamespace(token=None, api_key=None, extra=extra or {}), chat_id, message)
with patch("smtplib.SMTP") as mock_smtp:
mock_server = MagicMock()
mock_smtp.return_value = mock_server
result = asyncio.run(
_send_email({"address": "[email protected]", "smtp_host": "smtp.test.com"}, "[email protected]", "Hello")
)
self.assertTrue(result["success"])
self.assertEqual(result["platform"], "email")
_, kwargs = mock_server.starttls.call_args
self.assertIsInstance(kwargs["context"], ssl.SSLContext)
send_call = mock_server.send_message.call_args[0][0]
from agent.i18n import t
self.assertEqual(send_call["Subject"], t("platform.email.standalone_subject"))
self.assertIn("Date", send_call)
self.assertEqual(send_call["To"], "[email protected]")
self.assertEqual(send_call["From"], "[email protected]")
class TestSmtpConnectionCleanup(unittest.TestCase):
"""Verify SMTP connections are closed even when send_message raises."""
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_SMTP_PORT": "587",
}, clear=False)
def _make_adapter(self):
from gateway.config import PlatformConfig
from plugins.platforms.email.adapter import EmailAdapter
return EmailAdapter(PlatformConfig(enabled=True))
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_SMTP_PORT": "587",
}, clear=False)
def test_smtp_close_called_when_quit_also_fails(self):
"""If both send_message() and quit() fail, close() is the fallback."""
adapter = self._make_adapter()
mock_smtp = MagicMock()
mock_smtp.send_message.side_effect = Exception("send failed")
mock_smtp.quit.side_effect = Exception("quit failed")
with patch("smtplib.SMTP", return_value=mock_smtp):
with self.assertRaises(Exception):
adapter._send_email("[email protected]", "Hello")
mock_smtp.close.assert_called_once()
class TestImapConnectionCleanup(unittest.TestCase):
"""Verify IMAP connections are closed even when fetch raises."""
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_IMAP_PORT": "993",
"EMAIL_SMTP_HOST": "smtp.test.com",
}, clear=False)
def _make_adapter(self):
from gateway.config import PlatformConfig
from plugins.platforms.email.adapter import EmailAdapter
return EmailAdapter(PlatformConfig(enabled=True))
@patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_IMAP_PORT": "993",
"EMAIL_SMTP_HOST": "smtp.test.com",
}, clear=False)
def test_imap_logout_called_on_uid_fetch_failure(self):
"""IMAP logout() must be called even when uid fetch raises."""
adapter = self._make_adapter()
mock_imap = MagicMock()
def uid_handler(command, *args):
if command == "search":
return ("OK", [b"1"])
if command == "fetch":
raise Exception("fetch failed")
return ("NO", [])
mock_imap.uid.side_effect = uid_handler
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
results = adapter._fetch_new_messages()
self.assertEqual(results, [])
mock_imap.logout.assert_called_once()
class TestImapIdExtensionForNetEase(unittest.TestCase):
"""Regression for #22271: 163/NetEase mailbox requires the RFC 2971
IMAP ID command after LOGIN, otherwise it returns ``BYE Unsafe Login``
on every UID SEARCH. We send ID best-effort after every login so that
163 works while non-supporting servers stay unaffected.
"""
def _make_adapter(self):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.163.com",
"EMAIL_SMTP_HOST": "smtp.163.com",
}):
from plugins.platforms.email.adapter import EmailAdapter
adapter = EmailAdapter(PlatformConfig(enabled=True))
return adapter
def test_connect_sends_imap_id_after_login(self):
"""connect() must call xatom('ID', ...) after LOGIN for 163 support."""
import asyncio
adapter = self._make_adapter()
mock_imap = MagicMock()
mock_imap.capabilities = ("IMAP4REV1", "ID", "UIDPLUS")
mock_imap.uid.return_value = ("OK", [b""])
with patch("imaplib.IMAP4_SSL", return_value=mock_imap), \
patch("smtplib.SMTP") as mock_smtp:
mock_smtp.return_value = MagicMock()
asyncio.run(adapter.connect())
adapter._running = False
if adapter._poll_task:
adapter._poll_task.cancel()
id_calls = [c for c in mock_imap.xatom.call_args_list if c.args and c.args[0] == "ID"]
self.assertTrue(
id_calls,
"EmailAdapter.connect() must call imap.xatom('ID', ...) after "
"LOGIN so 163/NetEase mailbox does not return 'Unsafe Login'.",
)
payload = id_calls[0].args[1]
self.assertIn("hermes-agent", payload)
names = [c[0] for c in mock_imap.method_calls]
self.assertIn("login", names)
self.assertLess(names.index("login"), names.index("xatom"))
class TestConnectSmtp(unittest.TestCase):
"""Test _connect_smtp() helper: protocol selection and IPv6 fallback."""
def _make_adapter(self, port="587"):
from gateway.config import PlatformConfig
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "imap.test.com",
"EMAIL_SMTP_HOST": "smtp.test.com",
"EMAIL_SMTP_PORT": port,
}):
from plugins.platforms.email.adapter import EmailAdapter
return EmailAdapter(PlatformConfig(enabled=True))
def test_ipv6_timeout_falls_back_to_ipv4(self):
"""When default connection times out, retry with an IPv4-only SMTP path."""
import socket as _socket
import plugins.platforms.email.adapter as email_mod
adapter = self._make_adapter("587")
with patch("smtplib.SMTP", side_effect=_socket.timeout("timed out")), \
patch.object(email_mod, "_IPv4SMTP") as mock_ipv4_smtp:
mock_server = MagicMock()
mock_ipv4_smtp.return_value = mock_server
result = adapter._connect_smtp()
self.assertIs(result, mock_server)
mock_ipv4_smtp.assert_called_once_with("smtp.test.com", 587, timeout=30)
mock_server.starttls.assert_called_once()
def test_port_465_ipv6_fallback(self):
"""Port 465 IPv6 timeout falls back to IPv4 with SMTP_SSL."""
import socket as _socket
import plugins.platforms.email.adapter as email_mod
adapter = self._make_adapter("465")
with patch("smtplib.SMTP_SSL", side_effect=_socket.timeout("timed out")), \
patch.object(email_mod, "_IPv4SMTP_SSL") as mock_ipv4_smtp_ssl:
mock_server = MagicMock()
mock_ipv4_smtp_ssl.return_value = mock_server
result = adapter._connect_smtp()
self.assertIs(result, mock_server)
mock_ipv4_smtp_ssl.assert_called_once_with(
"smtp.test.com", 465, timeout=30, context=ANY,
)
class TestConnectionConfigResolution(unittest.TestCase):
"""Host/address resolution and pre-connect validation (#49736)."""
def test_connect_aborts_without_attempting_imap_when_host_missing(self):
"""A missing host returns False without the cryptic DNS error, and marks
the failure non-retryable so the gateway stops reconnecting (#40715)."""
import asyncio
from gateway.config import PlatformConfig
from plugins.platforms.email.adapter import EmailAdapter
with patch.dict(os.environ, {
"EMAIL_ADDRESS": "[email protected]",
"EMAIL_PASSWORD": "secret",
"EMAIL_IMAP_HOST": "",
"EMAIL_SMTP_HOST": "smtp.test.com",
}, clear=False):
adapter = EmailAdapter(PlatformConfig(enabled=True))
with patch("imaplib.IMAP4_SSL") as mock_imap:
result = asyncio.run(adapter.connect())
self.assertFalse(result)
mock_imap.assert_not_called()
# The OOM fix (#40715): a blank host must NOT leave the platform in the
# retryable reconnect loop — it is a permanent config error.
self.assertTrue(adapter.has_fatal_error)
self.assertEqual(adapter.fatal_error_code, "email_missing_configuration")
self.assertFalse(adapter.fatal_error_retryable)
self.assertIn("EMAIL_IMAP_HOST", adapter.fatal_error_message or "")
def test_blank_present_env_vars_are_not_required(self):
"""Blank/whitespace EMAIL_* values must read as missing (#40715) — an
abandoned setup with empty keys must not enable the platform."""
from plugins.platforms.email.adapter import check_email_requirements
for blank in ("", " ", "\n"):
with patch.dict(os.environ, {
"EMAIL_ADDRESS": blank, "EMAIL_PASSWORD": blank,
"EMAIL_IMAP_HOST": blank, "EMAIL_SMTP_HOST": blank,
}, clear=False):
self.assertFalse(check_email_requirements())
class TestSenderAuthentication(unittest.TestCase):
"""Verify _verify_sender_authentication parses Authentication-Results
correctly and resists From: spoofing (GHSA-rxqh-5572-8m77)."""
def _msg(self, from_addr, auth_results=None):
"""Build an email.message.Message with the given From: and
zero or more Authentication-Results headers (first = topmost/trusted)."""
msg = MIMEText("body")
msg["From"] = from_addr
for ar in auth_results or []:
msg["Authentication-Results"] = ar
return msg
def _verify(self, from_addr, auth_results=None, authserv_id=""):
from plugins.platforms.email.adapter import (
_verify_sender_authentication,
_extract_email_address,
)
msg = self._msg(from_addr, auth_results)
addr = _extract_email_address(from_addr)
return _verify_sender_authentication(msg, addr, authserv_id=authserv_id)
def test_auth_results_verdicts(self):
ok, reason = self._verify(
"Admin <[email protected]>",
["mx.google.com; dmarc=pass header.from=example.com; spf=pass"],
)
self.assertTrue(ok, reason)
# A dmarc=pass issued for another domain must not vouch for this From,
# even when a later dkim clause carries an aligned header.from.
# Verdict and header.from are read from the one dmarc clause, with (comments) stripped first.
for ar in ("mx.google.com; dmarc=pass header.from=evil.test",
"mx.google.com; dmarc=pass header.from=evil.test; dkim=pass header.d=x.test header.from=example.com",
"mx.google.com; dmarc=pass (p=none; sp=none) header.from=evil.test",
"mx.google.com; dmarc=fail header.from=example.com; dmarc=pass header.from=evil.test",
# every header.from in the dmarc clause must align, not just one
"mx.google.com; dmarc=pass header.from=evil.test header.from=example.com",
# ';' inside quoted-strings / nested comments must not split or smuggle a dmarc clause
'mx.google.com; spf=pass smtp.mailfrom="x;dmarc=pass header.from=example.com x"@evil.test; '
"dmarc=fail header.from=example.com",
"mx.google.com; dmarc=pass (a (b) ; header.from=example.com) header.from=evil.test",
'mx.google.com; dmarc=pass reason="a;b" header.from=evil.test',
"mx.google.com; dmarc=pass a) ; header.from=evil.test", # stray ')' is unbalanced
"mx.google.com; dmarc=pass header.from=example.com; dmarc=pass header.from=evil.test",
"mx.google.com; dmarc=pass (a ; header.from=evil.test",
r'mx.google.com; spf=pass smtp.mailfrom="x\\";dmarc=pass header.from=example.com;x="y"; '
"dmarc=fail header.from=example.com",
# spf/dkim verdicts and domains come only from their own clause, never quoted text or comments
'mx.google.com; spf=fail smtp.mailfrom="x spf=pass smtp.mailfrom=example.com "@evil.test; '
"dmarc=fail header.from=example.com",
"mx.google.com; spf=fail (spf=pass) [email protected]",
"mx.google.com; spf=fail [email protected]; dmarc=fail header.from=example.com",
'mx.google.com; dkim=pass header.d=evil.test header.i="x header.d=example.com y"@evil.test',
"mx.google.com; spf=pass smtp.mailfrom=example.com; spf=fail smtp.mailfrom=evil.test",
"mx.google.com; spf=fail smtp.mailfrom=evil.test; spf=pass smtp.mailfrom=example.com",
"mx.google.com; dkim=pass header.d=evil.test; dkim=fail header.d=example.com",
'mx.google.com; dkim=pass header.i="x header.d=example.com"@evil.test',
# an escaped quote keeps the quoted-string open, so no dmarc clause is smuggled out of it
r'mx.google.com; spf=fail smtp.mailfrom="a\";dmarc=pass header.from=example.com;x=\""@evil.test'):
ok, reason = self._verify("Admin <[email protected]>", [ar])
self.assertFalse(ok, ar)
# Real MTA headers (multi-signature DKIM, comments, quoted values) keep authenticating.
for ar in ("mx.google.com; arc=pass (dmarc=fail header.from=evil.test); dmarc=pass header.from=example.com",
'mx.google.com; dmarc=pass reason="a;b" header.from="example.com"',
'mx.google.com; dmarc=pass reason="header.from=evil.test" header.from=example.com',
"mx.google.com; dkim=pass [email protected] header.s=s1 header.b=AbC; spf=pass (google.com: "
"domain of [email protected] designates 1.2.3.4 as permitted sender) [email protected]; "
"dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=example.com",
"spf=pass (sender IP is 1.2.3.4) smtp.mailfrom=example.com; dkim=pass (signature was verified) "
"header.d=example.com;dmarc=pass action=none header.from=example.com;compauth=pass reason=100",
"mail.example.org; dmarc=pass (p=none dis=none) header.from=example.com",
'mail.example.org; dkim=pass (2048-bit key; unprotected) header.d=example.com [email protected] '
'header.b="AbC+/1"; spf=pass smtp.mailfrom=example.com',
"mx.example.org; dkim=pass (1024-bit key) header.d=esp.test [email protected]; "
"dkim=pass (2048-bit key) header.d=example.com [email protected]; spf=softfail "
"[email protected]"):
ok, reason = self._verify("Admin <[email protected]>", [ar])
self.assertTrue(ok, (ar, reason))
def test_dkim_pass_aligned_authenticates(self):
ok, reason = self._verify(
"[email protected]",
["mx.google.com; dkim=pass header.d=example.com"],
)
self.assertTrue(ok, reason)
def test_spf_pass_misaligned_rejected(self):
# SPF passes for the envelope domain, but it doesn't match From: domain.
ok, reason = self._verify(
"[email protected]",
["mx.google.com; spf=pass [email protected]"],
)
self.assertFalse(ok, reason)
def test_injected_header_below_trusted_does_not_authenticate(self):
"""An attacker-injected Authentication-Results sorts BELOW the receiving
server's. With authserv-id pinning, only the trusted (first) header is
consulted, so a forged 'dmarc=pass' lower in the stack is ignored."""
ok, reason = self._verify(
"[email protected]",
[
# Trusted: stamped by our server, real verdict = fail
"mx.ourserver.com; dmarc=fail header.from=example.com",
# Forged by attacker, claims pass
"mx.ourserver.com; dmarc=pass header.from=example.com",
],
authserv_id="mx.ourserver.com",
)
self.assertFalse(ok, reason)
if __name__ == "__main__":
unittest.main()