@keyring/cache

npm install @keyring/cache

Functions

fromDiskKeys

export function fromDiskKeys(snapshot: DiskSnapshot): KeyPolicy[];

generateNodeId

Identifies one SDK process to the control plane's watermark table, so that revoke?wait=true can answer "revoked on 4/4 servers" (report section 2.6).

Host and pid make it legible in that answer -- an operator looking at a stalled rollout wants to know which box has not caught up -- and the random suffix keeps two processes on one host, or two hosts that share a name in a container fleet, from colliding onto one watermark and each reporting the other's progress.

Shaped to policy_node_id_shape in migration 0005; anything the hostname contributes that does not fit is dropped rather than escaped.

export function generateNodeId(): string;

toDiskKeys

export function toDiskKeys(policies: Iterable<KeyPolicy>): DiskSnapshot['keys'];

toKeyPolicy

export function toKeyPolicy(wire: WirePolicyKey, workspaceId: string, projectId: string): KeyPolicy;

Classes

DiskSnapshotStore

export class DiskSnapshotStore {
    constructor(options: {
        cacheDir?: string;
        baseUrl: string;
        projectId: string;
        env: EnvFilter;
    });
    get path(): string;
    read(): DiskSnapshot | null;
    write(snapshot: DiskSnapshot): void;
    async writeAsync(snapshot: DiskSnapshot): Promise<void>;
    remove(): void;
}

LruPolicyStore

A bounded, least-recently-used policy cache.

Report section 6.1's fix 1 ("cache by tenant, not by project") and fix 3 ("bound it hard and say so"). Fix 2 -- packing each record into a Uint8Array with an interned scope table -- is deliberately not implemented: the report marks its 582 B -> ~140 B figure an estimate rather than a measurement, and says it is worth doing only if fix 1 proves insufficient.

The insertion order of a JS Map is the LRU list. A get that hits deletes and re-inserts, which moves the entry to the end in O(1); eviction takes the first key the iterator yields, which is the least recently used. No linked list, no second index, nothing else per entry.

export class LruPolicyStore implements PolicyStore {
    constructor(options: LruPolicyStoreOptions = {});
    get(lookupHashKey: string): KeyPolicy | undefined;
    getCached(lookupHashKey: string): CachedPolicy | undefined;
    status(): PolicyStoreStatus;
    stats(): PolicyStoreStats;
    loadSnapshot(policies: Iterable<KeyPolicy>, meta: {
        version: number;
        fetchedAt: number;
        source: PolicyStoreSource;
        scopeComplete: boolean;
        narrowed?: boolean;
    }): void;
    applyUpsert(policy: KeyPolicy, lookupHashKey: string): boolean;
    applyTombstone(lookupHashKey: string): boolean;
    markSynced(version: number, fetchedAt: number): void;
    raiseMaxCachedKeys(next: number): boolean;
    entries(): IterableIterator<KeyPolicy>;
    get size(): number;
}

PolicyCacheAutoRaised

Reported through onError when the snapshot walk raised maxCachedKeys by itself. It is a notice and not a failure -- the poll it came from succeeded -- and it is on the error channel because that is the SDK's one operator channel. An auto-raise that no one can see is the silent behaviour change this whole decision exists to avoid; stats().policyCache.autoRaisedTo carries the same fact for a scrape.

export class PolicyCacheAutoRaised extends Error {
    override readonly name: "PolicyCacheAutoRaised";
    readonly projectId: string;
    readonly from: number;
    readonly to: number;
    constructor(options: {
        projectId: string;
        from: number;
        to: number;
    });
}

PolicyClient

The only part of the SDK that touches the network, and it is deliberately not reachable from verify(). Report section 2.7 is the whole argument: Unkey's verifyKey is a network call to Unkey, and no amount of caching in front of that changes the shape. Keyring ships the policy here instead, on a timer, out of band.

export class PolicyClient {
    constructor(options: PolicyClientOptions);
    get tenantIds(): readonly string[];
    addTenants(tenantIds: readonly string[]): readonly string[];
    removeTenants(tenantIds: readonly string[]): void;
    async snapshot(options: {
        cursor?: string;
        maxKeys: number;
    }): Promise<WireSnapshot>;
    async delta(since: number, options: {
        maxKeys: number;
    }): Promise<WireDeltaAnswer | null>;
}

PolicyPoller

Keeps an LruPolicyStore fed from the versioned snapshot and the delta feed, and keeps a copy on disk so a restart of the same process/filesystem is not a cold start.

This is the wedge, in one class. Report section 2.7: Unkey's verifyKey is a network call to Unkey, and their co-founder's summary of years of latency work is "zero network requests are always faster than one network request." The answer is not a better cache in front of that call -- it is to ship the policy here, on this timer, so the call does not exist.

export class PolicyPoller {
    readonly nodeId: string;
    constructor(options: PolicyPollerOptions);
    get store(): LruPolicyStore;
    get consecutiveFailures(): number;
    get tenantIdsRejected(): number;
    get tenantIds(): readonly string[];
    get suspectsTracked(): number;
    get tenantsHeldOut(): number;
    loadFromDisk(): DiskSnapshot | null;
    async refreshOnce(): Promise<PollOutcome>;
    start(): void;
    stop(): void;
    admitTenants(tenantIds: readonly string[]): boolean;
    get snapshotPages(): number;
    get snapshotTruncated(): boolean;
    get autoRaisedMaxCachedKeys(): number | null;
    get maxAutoCachedKeys(): number;
    get scopeTooLarge(): PolicyScopeTooLargeError | null;
    get diskSnapshotScopeMismatch(): PolicySnapshotScopeMismatch | null;
    async flushPersist(): Promise<void>;
}

PolicyRequestError

export class PolicyRequestError extends Error {
    readonly status: number;
    constructor(status: number, message: string);
}

PolicyScopeTooLargeError

The project has more live keys than this node will hold, and no tenantIds narrows it.

Two different events, told apart by refusedToStart, because "refuse to start" and "kill a process that is answering requests" are different actions with different blast radii and the captain only answered the first.

• Cold (refusedToStart: true): the node has never served. Thrown out of the snapshot walk, so the store is never loaded and the poller stops; @keyring/sdk rethrows it out of a microtask and the deploy dies. That is the captain's decision, faithfully.
• Warm (refusedToStart: false): the node is already serving. Recorded and reported on every poll, and nothing is stopped. Throwing here made one ordinary PATCH /v1/projects/:id -- which raises policy_change_floor, so every node re-seeds on its next poll -- take a whole fleet down inside 31 ms, taking the customer's own API with it. And stop()ping such a node is worse than not stopping it: it freezes the store, so no revocation ever reaches that node again.

The message names what an operator needs and a generic error would not: how many keys were walked, what the ceiling is, what it costs to raise it, and -- tenantIds being the answer that reads like a fix and is not -- what narrowing would cost, in the same sentence that offers it.

export class PolicyScopeTooLargeError extends Error {
    override readonly name: "PolicyScopeTooLargeError";
    readonly projectId: string;
    readonly keysWalked: number;
    readonly ceiling: number;
    readonly refusedToStart: boolean;
    constructor(options: {
        projectId: string;
        keysWalked: number;
        ceiling: number;
        refusedToStart: boolean;
    });
}

PolicySnapshotScopeMismatch

Reported through onError when a disk snapshot was refused because the process that wrote it was running under a different scope than this one.

A notice, like PolicyCacheAutoRaised: the node is not broken, it is cold, and the cold path is one it already has. But an operator staring at a node that keeps cold-starting has no other way to see why -- the file is there, it is valid, it is this project and this environment, and it is still not being read. So the message names both scopes and the file.

export class PolicySnapshotScopeMismatch extends Error {
    override readonly name: "PolicySnapshotScopeMismatch";
    readonly path: string;
    readonly writer: SnapshotScope | undefined;
    readonly reader: SnapshotScope;
    constructor(options: {
        path: string;
        writer: SnapshotScope | undefined;
        reader: SnapshotScope;
    });
}

Interfaces

CachedPolicy

export interface CachedPolicy {
  readonly policy: KeyPolicy;
  /**
   * Epoch ms. Derived rather than stored per record: every successful poll
   * confirms the whole resident set, so a record's freshness is the later of
   * when it was admitted and when the store last synced. Storing an absolute
   * value per record and rewriting all of them on each poll would be O(n) every
   * five seconds for no additional truth.
   */
  readonly cachedUntil: number;
}

DiskSnapshot

export interface DiskSnapshot {
  readonly format: typeof SNAPSHOT_FORMAT;
  readonly workspaceId: string;
  readonly projectId: string;
  readonly env: EnvFilter;
  readonly version: number;
  readonly fetchedAt: number;
  readonly scopeComplete: boolean;
  /**
   * Absent in a file written by an SDK older than this field. That is an
   * *unknown* scope, not a matching one: the file it comes from is exactly the
   * file a mixed-version rollout leaves behind, which is the deployment this
   * check exists for. A tenant-narrowed reader refuses it and comes up cold --
   * one poll's worth of cost, against a poisoned store that never repairs
   * itself. A project-scoped reader loads it but does **not** believe its
   * `scopeComplete`, because such a file may have been written by a node that
   * was `complete: true` while holding one tenant's keys.
   *
   * The field is additive rather than a `SNAPSHOT_FORMAT` bump, deliberately:
   * a bump cold-starts every node in every fleet once, which is the cost this
   * decision exists to avoid. The price of that is a one-directional
   * mixed-version window -- an older SDK ignores this field and keeps
   * poisoning itself until it is replaced -- and `packages/cache/README.md`
   * says so out loud rather than leaving a customer to find it.
   */
  readonly scope?: SnapshotScope;
  readonly keys: ReadonlyArray<{
    keyId: string;
    tenantId: string;
    env: Environment;
    lookupHash: string;
    displayPrefix: string;
    scopes: string[];
    expiresAt: number | null;
    /** Absent in a file written by an SDK older than week 5. */
    rateLimits?: WireRateLimit[];
  }>;
}

LruPolicyStoreOptions

export interface LruPolicyStoreOptions {
  /** Report section 6.1: `= 640 bytes x maxCachedKeys`, so 10,000 ~ 6.1 MiB. */
  readonly maxCachedKeys?: number;
  readonly cachedTtlMs?: number;
  readonly now?: () => number;
}

PolicyClientOptions

export interface PolicyClientOptions {
  readonly baseUrl: string;
  /** A `krsk_` vendor secret key. Its environment decides what the feed shows. */
  readonly secretKey: string;
  readonly projectId: string;
  readonly env?: EnvFilter;
  /** Report section 6.1's fix 1, done server-side. */
  readonly tenantIds?: readonly string[];
  readonly nodeId?: string;
  readonly requestTimeoutMs?: number;
  readonly fetch?: FetchLike;
}

PolicyPollerOptions

export interface PolicyPollerOptions extends LruPolicyStoreOptions {
  readonly baseUrl: string;
  readonly secretKey: string;
  readonly projectId: string;
  readonly env?: EnvFilter;
  readonly tenantIds?: readonly string[];
  readonly store?: LruPolicyStore;
  /**
   * How far the snapshot walk may raise `maxCachedKeys` on its own.
   * `DEFAULT_MAX_AUTO_CACHED_KEYS` unless set, and never below `maxCachedKeys`
   * -- an operator who picked a larger bound already chose it.
   *
   * Setting it *equal* to `maxCachedKeys` turns the automatic step off, which
   * leaves the two honest answers to a project that does not fit and not the
   * silent third one: a node that has never served refuses to start, and one
   * that is already serving alarms on every poll and keeps serving. Raising
   * the bound is the fix either way; `tenantIds` lets the node run without
   * closing the hole, because a narrowed store is never `complete` either.
   */
  readonly maxAutoCachedKeys?: number;
  readonly policyRefreshMs?: number;
  readonly pollJitter?: number;
  readonly requestTimeoutMs?: number;
  readonly nodeId?: string;
  /** Report section 9.5's day-0 requirement. Set false to opt out entirely. */
  readonly persist?: boolean;
  readonly cacheDir?: string;
  readonly fetch?: FetchLike;
  readonly now?: () => number;
  readonly onError?: (error: unknown) => void;
  readonly onPoll?: (outcome: PollOutcome) => void;
}

PolicyStoreStats

export interface PolicyStoreStats {
  readonly size: number;
  /** The bound in force now, which `raiseMaxCachedKeys` may have widened. */
  readonly maxCachedKeys: number;
  /** The bound this store was constructed with, whatever it is now. */
  readonly configuredMaxCachedKeys: number;
  readonly hits: number;
  readonly misses: number;
  /**
   * Report section 6.1, fix 3: "bound it hard and say so". An SDK that silently
   * grows unbounded inside a customer's process is how you get uninstalled --
   * and one that silently *evicts* is how a fail-open default stops being safe,
   * because the argument for it is that the cache is complete for the project.
   */
  readonly evictions: number;
  /** Evictions per second since the store was created. */
  readonly evictionRate: number;
  /** Upserts dropped because the key was not resident. See `applyUpsert`. */
  readonly skippedUpserts: number;
}

PollOutcome

export interface PollOutcome {
  readonly kind: PollKind;
  readonly version: number | null;
  /** How many cached records the poll added, replaced or dropped. */
  readonly applied: number;
  readonly error?: unknown;
}

WireDelta

export interface WireDelta {
  readonly object: 'policy_delta';
  readonly workspace_id: string;
  readonly project_id: string;
  readonly env: EnvFilter;
  readonly since: number;
  readonly version: number;
  readonly project_deleted: boolean;
  readonly generated_at: string;
  readonly changes: WireChange[];
}

WirePolicyKey

One key as GET /v1/policy/snapshot sends it.

export interface WirePolicyKey {
  readonly key_id: string;
  readonly tenant_id: string;
  readonly env: Environment;
  /** base64 SHA-256 of the whole key: the cache index, and not a secret. */
  readonly lookup_hash: string;
  readonly display_prefix: string;
  readonly scopes: string[];
  readonly expires_at: string | null;
  /**
   * The key's effective limits, resolved against the project default by the
   * control plane. Absent on a snapshot from a control plane older than week 5,
   * which reads as "no limits" -- the same answer that plane would have given.
   */
  readonly rate_limits?: WireRateLimit[];
}

WireSnapshot

export interface WireSnapshot {
  readonly object: 'policy_snapshot';
  readonly workspace_id: string;
  readonly project_id: string;
  readonly env: EnvFilter;
  readonly version: number;
  readonly project_deleted: boolean;
  readonly generated_at: string;
  readonly keys: WirePolicyKey[];
  /**
   * The token that fetches the next page of this snapshot, or `null`/absent for
   * the last one.
   *
   * Optional on this side and not on the control plane's, because a snapshot
   * also arrives from disk and from a control plane older than the paging
   * protocol. Absent is read as "there is no next page", which is safe in both
   * directions: an old control plane never truncates -- it refuses or it sends
   * everything -- and a truncation is only ever something this client asked for
   * by sending `max_keys`.
   */
  readonly next_cursor?: string | null;
}

Types

EnvFilter

null is "both environments", which is what a live credential gets.

export type EnvFilter = Environment | null;

FetchLike

export type FetchLike = (
  input: string,
  init?: {
    method?: string;
    headers?: Record<string, string>;
    signal?: AbortSignal;
  },
) => Promise<{
  status: number;
  headers: { get(name: string): string | null };
  text(): Promise<string>;
}>;

PollKind

export type PollKind = 'snapshot' | 'delta' | 'unchanged' | 'error';

SnapshotScope

The scope of the process that wrote the file: the whole project, or a specific tenant filter.

Nothing recorded it before, so PolicyPoller.loadFromDisk built half the loaded state from the file (scopeComplete) and half from this node's options (narrowed). Two differently-scoped processes sharing one cacheDir -- several workers in one container is the ordinary shape of that -- then reached complete: true together with narrowed: true, which no network walk can produce: applyUpsert skips every non-resident delta upsert while status() still says the store can prove a miss, so a key minted for a tenant that node serves is answered 401 for the life of the process. Recording the writer's scope is what lets the loader refuse.

The filter is normalised -- deduplicated and sorted -- so the comparison is about the set and not about the order an integrator listed it in, and the empty filter is spelled project rather than tenants: [] so there is exactly one representation of "the whole project".

The rejected alternative is putting tenantIds into the filename fingerprint, so differently-scoped processes never share a file at all. It makes the mismatch invisible rather than diagnosable, it multiplies files in a shared directory, and a node whose filter widens at runtime would write under a name its own restart no longer looks for.

export type SnapshotScope =
  | { readonly kind: 'project' }
  | { readonly kind: 'tenants'; readonly tenantIds: readonly string[] };

WireChange

export type WireChange =
  | ({ readonly change: 'upsert' } & WirePolicyKey)
  | {
      readonly change: 'tombstone';
      readonly key_id: string;
      readonly env: Environment;
      readonly lookup_hash: string;
    };

WireDeltaAnswer

A delta request can come back as a snapshot. That is not an error: it is the control plane saying the caller's since is not one it can answer completely (pruned, ahead of the current version, or so far behind that the delta would cost more than a re-seed). A partial delta would be a silently wrong cache, so the protocol has no way to express one.

export type WireDeltaAnswer = WireDelta | WireSnapshot;

Constants

DEFAULT_CACHE_DIR

os.tmpdir() is the report's default and it is world-writable on every Unix. That means the directory is the exposure, not the file: another local user can pre-create /tmp/keyring and own it, and then every 0600 file we write lands somewhere they control.

So the directory is created 0700 and, if it already exists, checked before anything is written to it: a symlink, a non-directory, or permissions that let group or other write to it are all refused. Refusing means the SDK runs without a disk cache and says so -- degraded, and loudly, rather than silently writing the customer's policy where someone else can read or replace it.

A customer sharing a host with untrusted local users should set cacheDir to a private path they own. The README says so.

const DEFAULT_CACHE_DIR: string;

DEFAULT_CACHED_TTL_MS

Report section 2.6's belt and braces: every cached record also carries cachedUntil = now + 60 s. It is not the invalidation mechanism -- the delta feed is, at 5 s -- it is the bound on how long a node that has stopped being able to reach us keeps serving before the configured fail-open/fail-closed policy takes over.

const DEFAULT_CACHED_TTL_MS: 60000;

DEFAULT_MAX_AUTO_CACHED_KEYS

How far one node will raise its own maxCachedKeys to finish a snapshot walk, before it refuses to start (cold) or alarms and keeps serving (warm).

The captain settled the large-project fail-open call on option b with c as an automatic first step: raise the bound automatically to here, and only then insist the operator raise it further and pay for it. What was rejected matters as much as what was chosen. Leaving the walk silently truncated makes the largest customers carry a permanent complete: false, which under stale-then-open admits any well-formed key, unverified, with no tenant, no scopes and no scope check, for the life of the process; the key's checksum is CRC-32, so forging a well-formed key costs an attacker nothing. Defaulting a truncated store to stale-then-closed instead would close that hole by rejecting legitimate keys for ever: a key outside the cached slice can never arrive, because there is no per-key remote lookup and the async refresh re-walks into the same wall.

The bound the captain approved is a memory cost, not a key count, and this number is the key count that delivers it. He approved "50,000 keys at 31.2 MiB". The 31.2 came from bench/policy-memory.mjs, which built its wire records as JS object literals: all 50,000 of its keys shared one 'read:orders' string object and one set of rate_limits ids, and JSON.parse -- the path PolicyClient actually takes -- allocates a string per occurrence. Re-measured on the wire path by bench/policy-heap.mjs, a key costs 855 B at three scopes and the two-rule set migration 0008 bounds, so 50,000 of them is 40.3 MiB and not 31.2. 38,000 x 855 B is 31.0 MiB, which is the number that was approved.

What that buys and what it does not, measured on the same run:

A re-seed costs another ~1.4x of that while the page in flight is still uncollected, and one PATCH /v1/projects/:id makes a whole fleet re-seed at once -- so eight Node workers on the 2 GB container DEFAULT_MAX_CACHED_KEYS is written against spend 248 MiB steady at this ceiling and ~340 MiB through a simultaneous re-seed. At 50,000 the same fleet spends 322 MiB and ~450 MiB. Bounding on measured bytes rather than on a key count would be the better invariant and is deliberately not this change: it moves the configuration surface, which is the captain's call and not one he was asked.

const DEFAULT_MAX_AUTO_CACHED_KEYS: 38000;

DEFAULT_MAX_CACHED_KEYS

Report section 6.1's recommendation, and the number the memory formula is written against. The measurement is the finding, not the speed: 582-679 bytes per cached key in the V8 heap means 1 M keys is 555 MiB inside someone else's process, and a customer running eight Node workers on a 2 GB container cannot afford that.

const DEFAULT_MAX_CACHED_KEYS: 10000;

DEFAULT_POLICY_REFRESH_MS

Report section 2.6. Section 2.6 also prices 1 s as a real cost with a name.

const DEFAULT_POLICY_REFRESH_MS: 5000;

DEFAULT_POLL_JITTER

10,000 nodes on an unjittered 5 s poll is 10,000 requests arriving in the same millisecond every five seconds, because they all started when the same deploy rolled out. Report section 10.2 sizes the feed for 2,000 req/s spread evenly; in lockstep it is a 10,000-request spike against our own delta feed twelve times a minute.

The jitter is subtracted, never added: the delay is drawn from [interval * (1 - jitter), interval]. Adding it would spread the herd just as well and would push worst-case propagation past the interval, and report section 2.6's "<= 5 seconds (p100)" is a number we quote to customers.

const DEFAULT_POLL_JITTER: 0.2;

MAX_SNAPSHOT_PAGES

How many pages of one snapshot this node will walk before it stops and says the store is incomplete.

The real bound is maxCachedKeys: everything past it would be evicted on load anyway, so paging beyond it is bandwidth spent to throw the result away. This is the second bound, and it is about a control plane that misbehaves rather than one that is large -- a page that answers with a cursor and no keys would otherwise be an infinite loop inside the customer's process, which is not a failure mode an SDK gets to have.

const MAX_SNAPSHOT_PAGES: 512;

SNAPSHOT_FORMAT

Report section 9.5 calls the cold start "the genuinely dangerous case": a restarted process with no cache to be stale from. Without a snapshot on disk, a customer restarting the same process/filesystem during a Keyring outage goes down -- the worst possible first impression -- so this is a day-0 requirement rather than a nice-to-have. It does not cover a fresh container in an immutable-container rollout, which has no file to hydrate regardless; see packages/cache/README.md's "Immutable containers and rollouts".

What lands on disk, and why that is acceptable:

• lookup_hash is the unpeppered SHA-256 of the key. That is not a concession: it has to be computable inside the customer's process, which is what makes verification local, and the authoritative peppered hash never leaves the control plane. A key carries 192 bits of entropy, so a hash is not invertible and there is no dictionary to attack -- the same argument that lets api_key.lookup_hash be unpeppered in our own database. (Which is also why report section 2.2 is wrong to imply the pepper could substitute for that entropy; see docs/architecture.md.)
• key ids, tenant ids, display prefixes, scopes and expiries. Metadata, and already in the customer's own heap and their own request logs.
• never a plaintext key, never the pepper, never the peppered hash. None of the three exists in this process at all.

So the file adds no class of secret the process was not already holding; what it adds is a persistence window, and that is what the permissions below are for.

const SNAPSHOT_FORMAT: "keyring.policy-snapshot.v1";