AI agents and LLM APIs

An API built for agents is called by code that was not written by the customer's engineers, at a rate they did not plan, with a credential they pasted into a config file. Three things go wrong more often than with a human integrator: the agent explores with a live key, the agent retries a mutation it already made, and the agent loops.

Give the agent a test key first

Every tenant gets a kr_test_ key that hits the same endpoint as the live one and reaches only test resources. An agent can explore your whole surface, hit every error path and mint a thousand fake orders without a single real side effect, and the test-mode usage tells you what it did. Test traffic is free.

app.use(
  keyring({ resources: { store: { live: liveStore, test: testStore } } }),
);

Make retries safe

Agents retry. A tool call that timed out is called again with the same arguments, and a mutation that already happened happens twice. Tell your customers to send Idempotency-Key, and the replay returns the stored response instead of charging them again, scoped to their tenant so it survives a key rotation. The idempotency page has the semantics; they are Stripe's, which an agent's authors already know.

Bound the loop

A per-key sliding limit per second stops a runaway loop; a per-tenant fixed limit per day is the budget the customer agreed to. Both travel on the key, and RateLimit-Remaining in every response is what a well-behaved agent reads before it decides to call again.

{
  "rate_limits": [
    {
      "id": "burst",
      "limit": 20,
      "window_ms": 1000,
      "algorithm": "sliding",
      "scope": "key"
    },
    {
      "id": "daily",
      "limit": 50000,
      "window_ms": 86400000,
      "algorithm": "fixed",
      "scope": "tenant"
    }
  ]
}

For a route whose every call costs you money, an LLM call proxied on the customer's behalf, set rateLimitOnUnavailable: 'closed' so an outage of the counter store refuses rather than admits unmetered traffic, and onUnavailable: 'closed' so an unknown key is never let through unverified.

Scope the key to the tool

Mint keys with scopes named after your tools, read:documents, write:documents, run:agent, and require them per route. A key given to a read-only research agent cannot call the route that spends.

Let the customer manage it themselves

The embeddable component puts key creation, rotation and revocation inside your own dashboard, so the customer's own engineer mints a scoped key for each agent and revokes the one that misbehaves, without a support ticket.

Not yet

There is no per-request explorer, so "show me every call this agent made" is your own access log beside req.keyring.displayPrefix until that ships. Per-key requests and errors per day are on the key's usage chart.