A plugin for Observability + Budget Guardrails built with Hermes Agent

# hermesagentchallenge# devchallenge# agents
A plugin for Observability + Budget Guardrails built with Hermes Agentnujovich

Challenge Entry for the Hermes Agent Challenge ๐Ÿš€ What Problem Does This Solve? AI agent...

Challenge Entry for the Hermes Agent Challenge

๐Ÿš€ What Problem Does This Solve?

AI agent deployments often suffer from two critical blind spots:

  1. Cost visibility โ€” you discover a $500 OpenAI bill at the end of the month with no clue which cron jobs or sessions caused it
  2. Budget control โ€” runaway loops or expensive model choices can drain your account before you notice

hermes-telemetry solves both by giving you real-time observability and automatic budget enforcement for Hermes Agent.

Hermes Agent

๐ŸŽฏ Why This Plugin Matters

Every production AI system needs observability and cost control. This isn't just a nice-to-have โ€” it's essential infrastructure.

Before this plugin, Hermes users had no way to:

  • Track spending per cron job or messaging platform
  • Set budget limits that actually pause runaway processes
  • Compare cost efficiency between different models
  • Get real-time cost alerts before hitting billing limits

Now they can manage their AI spend like a modern SaaS โ€” with dashboards, alerts, and automatic circuit breakers.

โœจ Key Features

Real Usage Data (Not Estimates)

Captures actual token counts and costs returned by providers like OpenRouter, OpenAI, and Anthropic. No guesswork.

Multi-Level Budget Enforcement

  • Soft warnings at 80% of budget
  • Hard tool blocks at 100% (prevents new API calls)
  • Cron job pauses for automated workflows
  • Scope-specific limits (global, per-cron-job, per-platform)

Rich Analytics via Slash Commands

  • /stats โ€” session performance, tool usage, cost breakdowns
  • /stats cron week โ€” cron job cost comparison across time
  • /stats providers โ€” which providers return real vs estimated data
  • /budget โ€” current spending vs limits with visual indicators

Zero Model Awareness

Pure observability layer โ€” captures everything through hooks without affecting model behavior or adding latency.

๐Ÿ“Š Screenshots

Session Analytics (/stats)

Stats output

Budget Status (/budget)

Budget output

Cron Job Cost Comparison (/stats cron week)

Cron output

Provider Analysis (/stats providers)

Providers output

๐Ÿงช Proof of Concept: Real Data

I tested the plugin with three different models to validate pricing accuracy and budget enforcement:

Model Cost per Test Run Budget Behavior
owl-alpha (free) $0.00 No limits triggered
claude-sonnet-4-6 $0.31 Soft warning at $0.001 limit
claude-opus-4-7 $2.23 Hard pause enforced โœ…

Budget enforcement works. When I set a $0.001 daily limit and ran a cron job, it correctly paused at $0.18 spending. When I raised the limit to $2.00, jobs resumed normally.

Real provider data. OpenRouter returned actual token counts (Est% = 0%), not estimates. The plugin correctly captured and priced these.

๐Ÿ—๏ธ Technical Implementation

Hook Pipeline Architecture

on_session_start โ†’ pre_api_request โ†’ โ˜… post_api_request โ†’ post_tool_call
                                     โ”‚
                                     โ–ผ
                               [capture usage]
                                     โ”‚
                                     โ–ผ
pre_llm_call (budget check) โ†’ pre_tool_call (tool gate) โ†’ SQLite storage
Enter fullscreen mode Exit fullscreen mode

Data Layer

  • SQLite WAL database โ€” efficient, local, no external deps
  • Custom pricing.yaml โ€” override provider rates for accurate cost calculation
  • budget.yaml configuration โ€” flexible limits (daily/monthly, global/scoped)
  • 94 comprehensive tests โ€” full coverage of edge cases and enforcement logic

Provider Compatibility

Works with any provider that follows the Hermes Agent provider interface:

  • โœ… OpenRouter (tested with real usage data)
  • โœ… OpenAI (pricing table included)
  • โœ… Anthropic (pricing table included)
  • โœ… Custom providers (via pricing.yaml overrides)

๐ŸŽฏ Production Ready

This isn't a demo โ€” it's production infrastructure. The plugin includes:

  • Error handling โ€” graceful fallbacks when providers return no usage data
  • Hot-reload โ€” update budgets via /budget set without restart
  • Concurrent safety โ€” SQLite WAL mode handles multiple sessions
  • Memory efficiency โ€” hook pipeline adds negligible overhead
  • Comprehensive logging โ€” debug telemetry issues with structured logs

๐Ÿš€ Installation & Usage

1. Install

cd ~/.hermes/plugins
git clone https://github.com/nujovich/hermes-telemetry.git
# Add 'hermes-telemetry' to plugins.enabled in config.yaml
# Restart gateway: hermes gateway restart
Enter fullscreen mode Exit fullscreen mode

2. Configure Budget (Optional)

# Set daily budget
hermes> /budget set global daily 5.00

# Check status  
hermes> /budget
Enter fullscreen mode Exit fullscreen mode

3. Monitor Usage

# Session stats
hermes> /stats

# Cron job breakdown
hermes> /stats cron week

# Provider analysis
hermes> /stats providers
Enter fullscreen mode Exit fullscreen mode

That's it. The plugin immediately starts capturing usage data for all sessions and cron jobs.

๐Ÿ† Why this is a win-win

This plugin solves a universal need in AI systems โ€” cost visibility and control. Every Hermes Agent deployment, from personal automation to enterprise cron jobs, benefits from this infrastructure.

It's not just useful, it's essential. Without budget controls, a misconfigured cron job with an expensive model can cost hundreds of dollars overnight. This plugin prevents that.

Real-world tested. I built, deployed, and validated this with actual usage data across multiple providers and models. It's not a concept โ€” it's working infrastructure that saves money and provides operational insight.

Community impact. This sets a standard for observability in the Hermes ecosystem. Other plugin authors can build on these patterns, and users get immediate operational confidence.

๐Ÿ“‹ Technical Details

  • Repository: https://github.com/nujovich/hermes-telemetry
  • Documentation: Complete README with architecture, configuration, and troubleshooting
  • Tests: 94 passing tests covering all major functionality
  • License: MIT
  • Dependencies: PyYAML only (for config files)

๐Ÿ‘จโ€๐Ÿ’ป About the Author

I'm Nadia Ujovich.

I understand the operational challenges of running AI systems at scale, and I built this plugin to solve the observability gap I see in every deployment.


This plugin makes Hermes Agent production-ready for cost-conscious deployments. It's the infrastructure piece that every serious AI system needs, but few teams build themselves.

Give your agents the observability they deserve. Try hermes-telemetry today.

Made with โ˜• for the Hermes Agent ecosystem