Per-Agent Policy Enforcement and Budget Management at the AI Inference Proxy Layer

This disclosure describes a system and method for governing AI agent inference access through a proxy layer that enforces per-agent model allowlists, budget caps with configurable reset periods, and agent self-service introspection APIs—all without requiring modifications to the agents themselves.

The system enables platform operators to govern heterogeneous fleets of AI agents through a single configuration surface, shifting governance enforcement from distributed application code to a centralized infrastructure layer positioned between agents and upstream model providers.

Organizations deploying multiple AI agents face a compound governance challenge spanning three domains: multi-agent policy enforcement, budget control, and agent operational awareness. As of March 2026, no standardized mechanism exists to address these at the infrastructure layer.

1.1 The Multi-Agent Governance Problem

Each agent may require different model access, different spending limits, and different levels of operational visibility. Current approaches embed governance logic within each agent’s application code, creating compounding problems:

1.2 The Budget Enforcement Problem

AI inference APIs charge per-token, with costs varying by orders of magnitude across models—from $0.25/million tokens for small models to $75/million tokens for frontier reasoning models. An agent with unrestricted access can accumulate significant costs through:

Existing cloud provider billing operates at the account level with monthly invoices. There is no mechanism to enforce spending limits per-agent, per-period, at the point of inference. Organizations discover overspend after the fact, not before the request is made.

1.3 The Agent Self-Awareness Problem

AI agents operating autonomously benefit from awareness of their own operational constraints. An agent that knows it has consumed 80% of its budget can proactively switch to cheaper models or defer non-urgent tasks. An agent that knows which models are available to it can select appropriately without trial-and-error requests that fail with authorization errors. No existing inference infrastructure provides agents with self-service APIs for querying their own policies, budgets, and usage.

The system operates as a proxy layer—implemented as a serverless function, edge worker, or reverse proxy—positioned between AI agents and upstream model provider APIs. The proxy intercepts all inference requests and applies the following pipeline:

Each agent is identified by a long-lived authentication token (e.g., a 256-bit random hex string). The token maps to a configuration record in a key-value store containing:

This single record is the complete governance configuration for one agent. Fleet management reduces to CRUD operations on these records. The configuration scales from a solo developer with one agent to an enterprise with thousands of agents without architectural changes.

Model identifiers in AI inference APIs use two conventions: bare names (e.g., “claude-sonnet-4”) and provider-prefixed names (e.g., “anthropic/claude-sonnet-4”). The proxy’s allowlist matching handles both by comparing the bare portion of the requested model against the bare portion of each allowed model.

This means an allowlist entry of “claude-sonnet-4” permits requests for both “claude-sonnet-4” and “anthropic/claude-sonnet-4”, preventing policy circumvention through name format variation.

When an inference request arrives without a model field in the request body, the proxy injects the agent’s configured default model before forwarding to the upstream provider, enabling agents to operate without hardcoded model names.

Budget state per agent is stored as three values: cumulative spend in the current period, timestamp when the current period began, and timestamp of the most recent spend recording. On each request, the proxy executes:

Concurrency Model

Budget tracking uses a read-then-write pattern against a key-value store (not a transactional database). Concurrent requests from the same agent may result in slight overspend. This mirrors the soft-limit model used by cloud infrastructure providers—an acceptable trade-off for the performance benefit of key-value store latency versus database transactions.

For organizations requiring exact budget enforcement, the system can be extended with serialized access via durable objects or distributed locks, at the cost of increased latency. Key-value store entries are configured with TTL equal to the remaining period plus a buffer, ensuring automatic cleanup without manual garbage collection.

The proxy exposes authenticated endpoints that agents can call to query their own operational state, using the same token they use for inference:

These endpoints do not count against the agent’s rate limit, enabling agents to check their status without consuming inference quota. Budget warnings are actionable (e.g., “Budget 80% used—2,000 sats remaining”).

Every inference response includes headers communicating governance state to the agent, enabling informed decisions about subsequent requests without requiring a separate API call:

The proxy emits machine-parseable structured log entries for every request at every governance decision point. Each entry records:

This creates a complete audit trail suitable for compliance reporting without logging prompt content.

Agent identities are associated with both a configured trust tier (determining rate limits) and a trust score from an external scoring system. The proxy uses the higher of the two—configured tier or score-derived tier—enabling agents to “earn” higher rate limits through demonstrated trustworthiness while maintaining a minimum floor configured by the platform operator.

Platform administration APIs, separately authenticated from agent credentials using a platform-level secret, enable programmatic fleet management: creating, reading, updating, and deleting agent identity configurations without direct access to the underlying key-value store.

The following aspects are believed to be novel as of the filing date:

1. Enforcement of per-agent model access policies at an inference proxy layer rather than within agent application code, using configurable allowlists with bare/prefixed model name matching
2. Per-agent budget caps with configurable reset periods enforced at the point of inference, with two-phase enforcement (pre-check before upstream call, spend recording after)
3. Agent self-service introspection APIs co-located with the inference proxy, enabling agents to query their own governance state through the same endpoint they use for inference
4. A universal agent configuration surface wherein all governance parameters for an agent are stored as a single serialized record, scaling from single-agent to enterprise-fleet without architectural changes
5. Structured audit logging at every governance decision point in the inference proxy pipeline, recording governance metadata without prompt content
6. Governance state communicated via response headers on every inference response, enabling agents to adapt behavior without separate API calls
7. Platform administration APIs for programmatic fleet management of agent governance configurations, separately authenticated from agent credentials
8. Integration of trust-tier systems with per-agent policy enforcement, using the higher of configured and earned trust levels