feat: harden the capability-token lifecycle — rotation, TTL, invoke-time enforcement

[dgenio]

Closes #170, closes #183, closes #185, closes #200, closes #203.
Investigates #224 (ADR — no code).

Note on base branch: this PR is stacked on #238 (still open) because both touch kernel/_invoke.py's perform_invoke. Basing here avoids a guaranteed merge conflict; once #238 merges to main, please retarget this PR's base to main (GitHub usually offers to do this automatically).

Why this group

These five issues were selected as a coherent group: they share one implementation path — the lifecycle of a CapabilityToken, from issuance through invoke-time enforcement — and one code area (tokens.py, kernel/_grant.py, kernel/_invoke.py). #170 was explicitly flagged as a deliberate follow-up in #238's own description ("a distinct rate-limiting/policy concern... recommend it as a separate follow-up"), which is what this PR is.

What changed

#185 — Signing-key rotation

• HMACTokenProvider accepts a {key_id: secret} KeyRing + active_key_id, so WEAVER_KERNEL_SECRET can rotate without invalidating every outstanding token at once.
• A token signed under a previous key still verifies during the overlap window; an unknown key_id fails closed as TokenInvalid.
• New optional WEAVER_KERNEL_SECRETS env var (JSON {key_id: secret}), taking precedence over the legacy WEAVER_KERNEL_SECRET.
• Verifying a non-active-key token logs token_verified_non_active_key (key id only, never the secret) so operators can tell when it's safe to retire a key.

#200 — Typed errors from CapabilityToken.from_dict

• Malformed input (missing field, wrong type, invalid timestamp, non-object constraints) now raises TokenInvalid with a descriptive message instead of a bare KeyError/ValueError. Valid round-trips unchanged.

#203 — Per-grant TTL

• Kernel.grant_capability(..., ttl_s=...) sets a token's lifetime per grant.
• DefaultPolicyEngine(max_ttl_s={SafetyClass: seconds}) validates it via a new, Protocol-optional resolve_ttl() method — an excessive or non-positive ttl_s is denied, never silently clamped.
• resolve_ttl is deliberately not added to the PolicyEngine Protocol — the same non-breaking pattern already used to split off ExplainingPolicyEngine (see CHANGELOG.md's 0.7.0 entry) — so no third-party PolicyEngine implementation breaks. A test (test_grant_capability_ttl_s_passthrough_without_resolve_ttl) proves the graceful degradation.

#183 — Signed argument-level constraints

• A token's constraints["args"] (allowed_keys, pinned, prefix) is enforced by Kernel.invoke() before the driver runs and before budget is reserved, raising TokenScopeError with a stable reason code and an audited "deny" trace on violation.
• Dry-run predicts the identical outcome (same check, no rate-limit consumption).
• DeclarativePolicyEngine needed zero code changes — constraints is already a free-form mapping that flows straight into the issued token. New example rule in examples/policies/default.{yaml,toml}.

#170 — Opt-in per-invocation rate limiting

• Kernel(invoke_rate_limits={SafetyClass: (limit, window_s)}) adds a second, invoke-time sliding-window limit, independent of and in addition to the existing grant-time limit. Default off — behavior is unchanged unless configured.
• Raises the new RateLimitExceeded with a stable reason code; dry_run=True never consumes it.
• A concurrency test (test_invoke_rate_limit_concurrent_calls_do_not_exceed_limit) proves 10 concurrent invoke() calls against a limit of 3 admit exactly 3 — the check-then-record pair runs with no await in between, so there's no over-admission race.

#224 — Token signing format evolution (investigation only)

• docs/adr/0001-token-signing-evolution.md evaluates HMAC (status quo), macaroon-style caveat chaining, and Biscuit against this kernel's actual invariants (I-06, minimal-dependency policy, determinism, revocation model, explain() transparency) — including real measured numbers for the HMAC baseline (401-byte token, ~12.5µs verify).
• Recommendation: status quo + re-issuance now; macaroon-style chaining as the documented future path if an offline-delegation need materializes ([Feature] Delegated, attenuated grants for multi-agent delegation chains #129); Biscuit explicitly deferred (mandatory dependency, weakened default revocation posture, no current consumer).
• No code or dependency changes.

Architecture note

tokens.py, kernel/__init__.py, and kernel/_invoke.py were all already at their line-budget ratchet ceilings before this change. Rather than raise every ceiling, this PR:

• Extracted grant_capability's body into a new kernel/_grant.py (mirrors the existing _invoke.py/_dry_run.py split).
• Extracted trace-recording helpers from kernel/_invoke.py into kernel/_trace_record.py (shared with _stream.py/_audit.py, which already imported them from _invoke.py).
• Extracted token signing/verification/deserialization into a new _token_signing.py, and TTL validation into policy_ttl.py — mirroring the existing rate_limit.py/kernel/_driver_exec.py split pattern.

Net effect: kernel/__init__.py and tokens.py both end up smaller than before despite the new features. policy.py's ceiling needed a modest, justified raise (+47 lines, documented inline) since further splitting would mean refactoring evaluate()/explain() — that's issue #219's job, out of scope here.

Compatibility

Deliberate breaking change: CapabilityToken's signable payload now includes key_id, so a token issued by pre-upgrade code fails verification after this deploys — even under the same secret, since the signed payload shape differs. Documented in CHANGELOG.md under ### Changed. Given tokens default to a 1-hour TTL and this project is pre-1.0 alpha (see the 0.10.0 import-rename precedent already in the CHANGELOG), this is treated as an accepted break, not a compatibility shim.

No PolicyEngine/ExplainingPolicyEngine Protocol changes — resolve_ttl is additive and optional, so no existing engine implementation is broken.

Scope notes

Deliberately out of scope:

• max_uses token-use-counter from Add per-invocation enforcement of rate limits (not only at grant time) #170 — the issue itself marks it "Optionally," and deferring it kept tokens.py comfortably under its line budget. Good follow-up issue.
• General RateLimiter thread-safety hardening — covered by existing issue [Testing] Document the concurrency model and add asyncio stress tests #142 (asyncio concurrency stress tests). This PR's own invoke-time check/record is written to avoid the specific race (no await between check and record), proven by the concurrency test above, but broader hardening belongs in [Testing] Document the concurrency model and add asyncio stress tests #142.
• DefaultPolicyEngine-level auto-attachment of args constraints per safety class — the declarative engine already supports this today with zero code changes (see Support signed argument-level constraints enforced at invoke time #183 above); a code-level equivalent on DefaultPolicyEngine is a reasonable, separate follow-up if wanted.

How verified

Full repo gate (make ci → fmt-check → lint → type → test → example) in a clean venv — exit 0:

• ruff format --check — 124 files already formatted
• ruff check — All checks passed
• mypy src/ (strict) — no issues in 67 source files
• pytest — 839 passed, 1 skipped, branch coverage 94.10% (floor 90%)
• make example — all 13 examples complete

New/updated tests:

• tests/test_tokens.py — key rotation + overlap window, unknown-key-id fails closed, legacy single-secret compat unaffected, active_key_id misconfiguration errors, non-active-key verification logging (never the secret), malformed from_dict parametrized over every failure class, valid round-trip unchanged.
• tests/test_kernel.py — invoke-time rate limit on/off/window-reset/per-principal-scoping/concurrency-race/dry-run-non-consumption, arg-constraint violation + dry-run parity + tamper-invalidates-signature + never-reaches-driver + deny-trace-recorded.
• tests/test_policy.py — resolve_ttl denial (non-positive, over-max)/pass-through/no-op-without-implementation, grant_capability(ttl_s=...) end-to-end, declarative args-constraint round-trip.

🤖 Generated with Claude Code

---

Generated by Claude Code

[cursor]

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.