hermes-trace/docs/hooks.md

133 lines
5.2 KiB
Markdown

# hermes-trace Hook Implementation
## Capture Mechanism
Uses **3 official Hermes plugin hooks** — the same mechanism as the bundled Langfuse plugin.
### Hook Registration (in `__init__.py`)
```python
# __init__.py
import threading
from .hooks import on_post_api_request, on_post_tool_call, on_pre_api_request
from .launcher import _cleanup_loop, _run_frontend
def register(ctx):
# Configuration and database initialization happen before this wiring.
ctx.register_hook("pre_api_request", on_pre_api_request)
ctx.register_hook("post_api_request", on_post_api_request)
ctx.register_hook("post_tool_call", on_post_tool_call)
threading.Thread(
target=_cleanup_loop, daemon=True, name="hermes-trace-cleanup"
).start()
threading.Thread(
target=_run_frontend, daemon=True, name="hermes-trace-frontend"
).start()
```
The handlers are implemented in `hooks.py`; the daemon targets are provided by
`launcher.py`. `__init__.py` keeps `register()` and re-exports the package
surface used by the plugin and its tests.
### Why hooks, not middleware
| Aspect | Hooks (`pre/post_api_request`) | Middleware (`llm_execution`) |
|--------|-------------------------------|------------------------------|
| Documented | ✅ Yes (official plugin docs) | ❌ No (internal, undocumented) |
| Used by | ✅ Langfuse plugin | ❌ No official plugin uses it |
| Signature | Stable, typed kwargs | May change without notice |
## Hook Callbacks
### `pre_api_request`
Fires **before** every LLM API call. Used to capture:
- Full request messages (conversation history + current prompt)
- Request preview (first 300 chars of the last non-system message)
> **Note:** Background review fork detection (scanning for `_SKILL_REVIEW_PROMPT`) is **not yet implemented** — all turns are currently stored without turn_type classification. This is tracked in PLAN.md as deferred.
### `post_api_request`
Fires **after** every LLM API call. Used to capture:
- Response content and tool calls
- Token usage (input, output, total, cache_read, cache_write, reasoning)
- Latency (api_duration)
- Finish reason
- Full response payload
### `post_tool_call`
Fires after each tool returns. Captures tool name, arguments, result, and duration.
> **⚠️ Error parameter limitation:** The Hermes hook does **not** provide an `error` parameter — tool failure information is encoded in the `result` field. The `tool_calls.error` column in the database schema exists but is **always `None`** because there is no error data available from the hook payload. Inspect the `result` field to detect tool failures.
## Daemon Threads / Subprocess
Spawning pattern (in `register()`):
```python
threading.Thread(target=_cleanup_loop, daemon=True, name="hermes-trace-cleanup").start()
threading.Thread(
target=_run_frontend, daemon=True, name="hermes-trace-frontend"
).start()
```
Both threads are daemon — they live as long as the Hermes process.
### Cleanup Thread
- Runs every 60 minutes
- Deletes turns older than `retention_days` (default: 7)
- Tool calls deleted via `ON DELETE CASCADE`
### Frontend Subprocess
`_run_frontend()` is a daemon thread that:
1. **Acquires a singleton lock** via `fcntl.flock(LOCK_EX | LOCK_NB)` — only one
Hermes process across all profiles runs the frontend.
2. **Resolves the runtime** — prefers the plugin's own `.venv/bin/python`
(verified via a subprocess preflight that checks `granian`, `fastapi`, and
`jinja2` are importable). If unavailable, logs an error and exits
without retrying.
3. **Kills orphan processes** — reads a PID file + `/proc/<pid>/cmdline` to
identify and terminate any lingering Granian from a previous crash.
4. **Launches Granian** via `subprocess.Popen()` as a **separate OS process**.
5. **Monitors and respawns** — if Granian exits, sleeps 3 seconds and retries.
The loop is bounded by the `finally` block which releases the lock,
closes the file descriptor, and cleans up PID and lock files on exit.
Reason for subprocess: Granian's `.serve()` calls `signal.signal()` which only
works in the main thread of the main interpreter. Running it inside the Hermes
process would require blocking the Hermes event loop.
## Plugin Configuration
Configuration is read from **environment variables** (not `plugins_config`):
| Env var | Default | Description |
|---------|---------|-------------|
| `HERMES_TRACE_DB_PATH` | `$HERMES_HOME/telemetry/telemetry.db`¹ | SQLite database path |
| `HERMES_TRACE_HOST` | `127.0.0.1` | Granian bind address |
| `HERMES_TRACE_PORT` | `7788` | Frontend server port |
| `HERMES_TRACE_RETENTION_DAYS` | `7` | Days to keep turn data |
> ¹ `$HERMES_HOME` 預設為 `~/.hermes`(可透過環境變數覆蓋)。使用 `HERMES_HOME` 讓 database 自動跟隨 Hermes profile 切換。
In `~/.hermes/config.yaml`, the plugin can be enabled/disabled but runtime config uses env vars:
```yaml
plugins:
enabled:
- hermes-trace
```
## Full Payload Storage
All fields stored unconditionally:
- `full_request`: Normalized conversation transcript assembled from provider-bound request data (JSON)
- `full_response`: Sanitized response snapshot exposed by the Hermes observability hook (JSON)
- Tool call args and result (JSON)
No conditional capture toggle. If storage space is a concern, lower `retention_days`.