hermes-agent/tests/agent/test_provider_history_parity.py
teknium1 13fe9c7171 feat(providers): external-process provider support for standalone model-provider plugins (from #105863)
The provider-agnostic half of PR #105863, so a CLI-driven subscription provider can ship as a
standalone `kind: model-provider` plugin instead of a bundled one:

- ProviderProfile: `native_reasoning_details_type`, `model_aliases`, `get_model_context_length`,
  `get_usage_cost`, `setup_status`, `discover_models` hooks (all default None / no-op).
- Chat Completions transport: provider-native `reasoning_details` carriers follow only their
  declaring profile; standard records still replay on OpenRouter-style routes, strict routes
  drop the field wholesale (#70233). Relay/stream accumulate `delta.reasoning_details` verbatim.
- `hermes model`: the generic plugin flow gates an external-process row on the CLI's own login
  status (inline `login_command` on a TTY), offers `discover_models()` rows with per-row notes,
  and never writes config when the executable is missing.
- `/model` and the pickers: process providers list their live catalog merged with the pinned
  one, declared aliases/ids resolve inside the provider, and validation accepts a listed id
  without probing `process://`.
- Delegation keeps the selected external-process provider and protocol for the child.
- Model metadata / usage pricing consult the profile's bound and cost hooks first.
- Desktop: `[1m]` renders as a "1M" tag and hyphenated Anthropic versions read "Haiku 4.5".

The bespoke `_model_flow_external_process` and hard-coded `hermes_cli/main.py` paths from the
PR were dropped in favour of main's `_model_flow_plugin_provider`.

Co-authored-by: unsupportedpastels <[email protected]>
2026-09-20 14:29:39 -07:00

36 lines
2.1 KiB
Python

"""Provider-native replay carriers stay private to their owning provider."""
from copy import deepcopy
from agent.transports.chat_completions import ChatCompletionsTransport
from providers.base import ProviderProfile
OPENROUTER = "https://openrouter.ai/api/v1"
def test_native_carriers_follow_only_their_owner_on_each_request():
owner = ProviderProfile(name="native-owner", base_url="process://native-owner")
owner.native_reasoning_details_type = "native-owner.native_assistant"
carrier = {"type": owner.native_reasoning_details_type, "messages": [{"text": "private"}]}
standard = {"type": "reasoning.encrypted", "data": "opaque-signature"}
history = [{"role": "assistant", "content": "answer", "reasoning_details": [carrier, standard]}]
original = deepcopy(history)
transport = ChatCompletionsTransport()
# The declaring profile gets its carrier on its own (non-HTTP) route, on every request,
# including after another provider served a turn in between; the other provider on a
# replaying route sees standard records only.
for profile in (owner, ProviderProfile(name="other"), owner):
base_url = owner.base_url if profile is owner else OPENROUTER
wire = transport.build_kwargs("test", history, provider_profile=profile, base_url=base_url)["messages"]
expected = [carrier, standard] if profile is owner else [standard]
assert wire[0]["reasoning_details"] == expected
assert history == original
# No declaring profile: a replaying route keeps standard records, a strict route drops the
# field wholesale (#70233) — never the stored history.
assert transport.build_kwargs("test", history, base_url=OPENROUTER)["messages"][0]["reasoning_details"] == [standard]
assert "reasoning_details" not in transport.build_kwargs("test", history, base_url="https://api.groq.com/openai/v1")["messages"][0]
assert history == original
only_native = [{"role": "assistant", "content": "answer", "reasoning_details": [carrier]}]
assert "reasoning_details" not in transport.convert_messages(only_native, base_url=OPENROUTER)[0]