8 KiB
Hermes Plugin System — Research Notes for hermes-trace
Findings from reading the Hermes Agent source code and official documentation.
Correct hook choice: pre_api_request / post_api_request
These are the official hooks for per-LLM-call telemetry. Not middleware, not pre_llm_call.
Evidence: Hermes Agent's own bundled Langfuse plugin at plugins/observability/langfuse/__init__.py uses exactly these hooks.
From register() in the Langfuse plugin (L1026-1035):
def register(ctx) -> None:
# Register for both hook name variants so the plugin works across
# Hermes versions. pre_api_request / post_api_request fire per API
# call (preferred); pre_llm_call / post_llm_call fire once per turn.
ctx.register_hook("pre_api_request", on_pre_llm_request)
ctx.register_hook("post_api_request", on_post_llm_call)
ctx.register_hook("pre_llm_call", on_pre_llm_call)
ctx.register_hook("post_llm_call", on_post_llm_call)
ctx.register_hook("pre_tool_call", on_pre_tool_call)
ctx.register_hook("post_tool_call", on_post_tool_call)
Key insight: pre_api_request / post_api_request fire per individual LLM API call — every time the agent calls the provider, including retries and interim calls inside the tool loop. This is exactly what we need for telemetry, and it's the preferred mechanism per the code comments.
Why not middleware
ctx.register_middleware("llm_execution", callback) technically exists in the code (hermes_cli/plugins.py) but is not documented in any official Hermes documentation (website/docs/). Using it would be relying on an undocumented internal mechanism.
Callback signatures — from source code
pre_api_request
Called at agent/conversation_loop.py L1012-1058, every API call, before the call is made.
def on_pre_api_request(
*,
task_id: str,
turn_id: str,
api_request_id: str,
session_id: str,
user_message: str,
conversation_history: list, # full conversation so far
platform: str,
model: str,
provider: str,
base_url: str,
api_mode: str,
api_call_count: int, # which call in this turn (0, 1, 2...)
request_messages: list, # the actual messages sent to the LLM
message_count: int, # total messages in request
tool_count: int,
approx_input_tokens: int,
request_char_count: int,
max_tokens: int,
started_at: float, # unix timestamp
**_: Any,
) -> None:
The request_messages list contains the full prompt sent to the LLM, including system prompt + conversation history + current user message. This is where we can detect background review forks by scanning for the review prompt text.
post_api_request
Called at agent/conversation_loop.py L3553-3588, every API call, after the response is received.
def on_post_api_request(
*,
task_id: str,
turn_id: str,
api_request_id: str,
session_id: str,
platform: str,
model: str,
provider: str,
base_url: str,
api_mode: str,
api_call_count: int,
api_duration: float, # seconds
started_at: float,
ended_at: float,
finish_reason: str, # "stop", "tool_calls", "length", ...
message_count: int,
response_model: str,
response: dict, # sanitized response payload (via _api_response_payload_for_hook)
usage: dict, # token usage summary (via _usage_summary_for_api_request_hook)
assistant_message: object, # the message object — has .content, .tool_calls, .reasoning
assistant_content_chars: int,
assistant_tool_call_count: int,
**_: Any,
) -> None:
The usage dict contains (from _usage_summary_for_api_request_hook):
input_tokens/prompt_tokensoutput_tokens/completion_tokenstotal_tokenscache_read_tokenscache_write_tokensreasoning_tokens
The response dict (from _api_response_payload_for_hook) contains a sanitized view with:
- Content / tool calls from the choice
- Finish reason
- But NOT raw usage (that's in the separate
usageparam)
The assistant_message object has:
.content— the text response.reasoning— reasoning content (deepseek-v4, etc.).tool_calls— list of tool call objects with.id,.function.name,.function.arguments
pre_tool_call
Called before each tool executes. Gets tool_name, args, task_id, session_id, tool_call_id.
post_tool_call
Called after each tool returns. Gets tool_name, args, result, task_id, session_id, tool_call_id, duration_ms.
Detecting background review forks
In pre_api_request, scan request_messages for the _SKILL_REVIEW_PROMPT text:
def on_pre_api_request(*, request_messages, session_id, model, **kw):
for msg in request_messages:
content = msg.get("content", "") or ""
if (
isinstance(content, str)
and "Review the conversation above and update the skill library" in content
):
# This is a background review fork!
turn_type = "background_review"
break
Langfuse plugin reference
The canonical reference implementation is at:
~/.hermes/hermes-agent/plugins/observability/langfuse/__init__.py
This is Hermes Agent's bundled Langfuse observability plugin. Key sections:
| Lines | Content |
|---|---|
| 1-22 | Module docstring + env var docs |
| 43-56 | TraceState dataclass — trace tree state management |
| 712-750 | on_pre_llm_call — turn-scoped hook handler |
| 752-821 | on_pre_llm_request — per-API-call hook handler (our model) |
| 824-949 | on_post_llm_call — per-API-call post hook handler (our model) |
| 952-976 | on_pre_tool_call — per-tool pre hook |
| 978-1023 | on_post_tool_call — per-tool post hook |
| 1026-1035 | register() — hook registration |
Note: the Langfuse plugin has the naming confusing where on_pre_llm_request (per call) and on_pre_llm_call (per turn) are different things. We should follow the naming more clearly for hermes-trace.
Hermes plugin structure (hermes-trace)
~/.hermes/plugins/hermes-trace/
├── plugin.yaml # Manifest: name, version, description
├── PLAN.md # Design document
├── HERMES_PLUGIN_RESEARCH.md # this file
├── __init__.py # register(ctx): wire hooks
├── db.py # SQLite schema + write/read
├── app.py # Granian + FastAPI + Jinja2 frontend
└── templates/ # Jinja2 HTML templates
Plugin lifecycle notes
register(ctx)is called exactly once at Hermes startup (gateway process or CLI).- Daemon threads (
threading.Thread(daemon=True)) spawned insideregister()live as long as the Hermes process. - This is safe for: background cleanup loops, the Granian frontend server, or any async housekeeping.
- If the plugin fails to load (exception in
register()), Hermes logs a warning and continues; the plugin is skipped.
Source files consulted
| File | Lines | Content |
|---|---|---|
plugins/observability/langfuse/__init__.py |
1-1035 | Bundled Langfuse plugin — our reference implementation |
plugins/observability/nemo_relay/__init__.py |
1-500+ | Another observability plugin using same hooks |
agent/conversation_loop.py |
1008-1058 | pre_api_request hook call site |
agent/conversation_loop.py |
3545-3588 | post_api_request hook call site |
hermes_cli/plugins.py |
290-550 | PluginContext class — all ctx.register_* methods |
hermes_cli/plugins.py |
997-1030 | register_hook implementation |
agent/background_review.py |
428-710 | Fork mechanism |
agent/turn_finalizer.py |
375-401 | Review trigger |
website/docs/guides/build-a-hermes-plugin.md |
1-1254 | Official plugin guide |
Frontend Research (added 2026-06-18)
See AGENTS.md for current frontend architecture (flat-table + overlay trace viewer with HTMX + Alpine patterns).