Skip to content

Threat model, production wiring, and boundaries ​

This page covers the security model the chain defends, the composition-root wiring that brings the substrate online in a real deployment, and the explicit boundary against downstream consumers. For the storage and chain mechanics that back the threat model see Storage topology and Hash chain and residency model; for the entry point see the index.

Threat model ​

The Platform Audit Log defends against post-incident attempts to rewrite history. The substrate is forensic, not preventive: an attacker who has already executed a privileged action cannot make the action un-happen, but they MUST NOT be able to make it un-traceable. Four attacker shapes drive the design:

  1. Hostile DBA. An operator with direct PostgreSQL access who tries to UPDATE a chain row to alter what was recorded. Mitigation: entry_hash is sha256(prev_hash ‖ sha256(canonical_bytes)) over fields the DBA cannot regenerate without the per-Domain pepper, so any UPDATE forces the row's entry_hash to disagree with sha256(prev_hash ‖ sha256(canonical_bytes)). The verifier names the divergent seq and writes a quarantine row; the integration test tests/integration/audit_chain_tamper_detection_test.go exercises this exact attack path.
  2. Panic dump. A core-dump from a crashed plexsphere binary that contains audit-relevant memory. Mitigation: the per-Domain pepper is resolved through the Pepper port. The OpenBao resolver caches each Domain's pepper in process memory after the first fetch, while the audit package itself holds no pepper; chain bytes reference only the pseudonym, so a panic dump containing chain rows reveals nothing about the underlying subjects. PII (plaintext, email, IP) lands in the sibling audit_subject_pii table and is purged by the right-to-erasure workflow when the data subject requests it.
  3. Backup tamper. An attacker who can substitute a malicious PostgreSQL dump or object-store mirror at restore time. Mitigation: the boot-time RegisterAuditReconcileProbe (internal/platform/bootstrap/audit_reconcile.go) re-verifies every Domain's chain end-to-end before /readyz returns 200; a substituted dump fails the verifier within seconds of boot. The Chainsaw E2E tests/e2e/security/audit-tamper-detection/ pins the ≤ 90 s detection budget and the named-divergence error path.
  4. Archive-mirror divergence. An attacker who corrupts the object-store mirror in the hope a future restore loads the tampered version. Mitigation: the archiver's idempotency contract preserves the archive_etag; the Chainsaw E2E tests/e2e/security/audit-archive-restore/ restores from the object store and re-runs the verifier against the restored chain — equivalence with the pre-drop state is the pass condition; abort-on-divergence is the fail condition.

The threat model deliberately excludes:

  • Pepper compromise. A hostile operator with both the per-Domain pepper AND DB write access can mint pseudonyms at will. The protection here is depth (OpenBao + DB ACLs + audit-of-audit) not cryptographic; pepper rotation as a defence belongs to the downstream tenant-side erasure UX work (see Boundary below).
  • Side-channel leaks. Wall-clock timing of Sink.Record could in theory reveal pepper-presence; we accept the channel because the audit emit path is on the slow path (every request emits at least one row, and timing variance is dominated by network latency).

Production wiring status ​

The current surface is the substrate: the Postgres schema, the canonical-byte encoder, the hash-chained Sink, the per-Domain reconciler, the four read handlers, and the archiver drain worker. Several wiring legs at the composition root deliberately scope to the early-boot deployment shape — the bounded context's invariants hold, but a production deployment must satisfy the gates documented below before the surface goes live.

The composition root for the audit surface lives at cmd/plexsphere/audit_factory_prod.go, split across four files for SRP:

  • audit_factory_prod.go — AuditFactory type, AuditWiring bundle, env loader, BuildProductionAuditFactory closure, and the audit registry surface with its bundle and Export.
  • audit_pepper_prod.go — staticPepper deterministic fallback, the buildAuditPepper boot build, and the requireAuditPepper factory gate.
  • audit_authz_prod.go — authzAuditAdapter and the auditAuthzCheck adapter over Runtime.AuthzCheck (both delegate to a SpiceDB-backed *internal/authz.Authorizer). The earlier staticAuditAuthorizer fail-closed default has been removed: the factory now refuses to build without a non-nil AuthzCheck closure (ErrAuditAuthzCheckRequired) so a misconfigured rollout cannot silently degrade every audit handler to 403.
  • audit_archive_prod.go — blobstoreArchiveUploader and buildArchiveWorker.

The OpenBao adapter behind the Pepper port is its own package, internal/audit/pepper: OpenBaoResolver in openbao.go, and the storage contract of the pepper secrets in doc.go.

Wiring legProduction bindingBoot gate
Hash-chained audit.Sinkaudit.NewChainedSink (internal/audit/service.go) over *repo.Repository, audit.Pepper, and defaultDomainResolver.Always wired when PLEXSPHERE_DSN is set.
Read-side query.EntryReader*repo.Repository.ListEntries (internal/audit/repo/read.go).Always wired when PLEXSPHERE_DSN is set.
Archiver archiver.EntryReader*repo.Repository.ListUnarchivedEntries + MarkArchived (internal/audit/repo/read.go).Wired when PLEXSPHERE_AUDIT_ARCHIVE_BUCKET is set.
Per-Domain reconcileraudit.NewReconciler (internal/audit/reconcile.go) iterates Repository.ListChainHeads, calls chain.VerifyChain over each Domain's tail, writes one audit_tamper_quarantine row per detected divergence.Always wired when PLEXSPHERE_DSN is set.
Per-Domain pepperpepper.OpenBaoResolver (internal/audit/pepper/openbao.go), built once by buildAuditPepper in cmd/plexsphere/main.go from the PLEXSPHERE_AUDIT_PEPPER_* family and read off Runtime.Pepper by every surface that pseudonymises or writes audit rows. staticPepper (deterministic sha256(domainID), NOT a secret) replaces it only when PLEXSPHERE_AUDIT_ALLOW_INSECURE_PEPPER=true is set and no mount is.Boot-time refusal: ErrAuditPepperNotProvisioned when neither PLEXSPHERE_AUDIT_PEPPER_MOUNT nor the opt-in is set, ErrAuditPepperModeConflict when both are, ErrAuditPepperEndpointRequired for a mount without PLEXSPHERE_AUDIT_PEPPER_ENDPOINT, ErrAuditPepperAuthRequired for a mount without an AppRole role id or Kubernetes role, and ErrAuditPepperUnreachable when every dial of the store fails. A factory handed no pepper refuses to build with ErrAuditPepperRequired.
Audit handlers.AuditAuthorizerauthzAuditAdapter over a function-shaped AuditAuthzCheckFunc closure the audit surface builder derives from Runtime.AuthzCheck through auditAuthzCheck; that closure delegates to the SpiceDB-backed *internal/authz.Authorizer.Check cmd/plexsphere/main.go assembles after the SpiceDB dial.Build-time refusal: BuildProductionAuditFactory returns ErrAuditAuthzCheckRequired when AuthzCheck is nil so a misconfigured rollout cannot silently degrade every audit handler to 403.
Domain CRUD services.AuditSink / domains.AuditSink*domainsAuditAdapter (cmd/plexsphere/domains_factory_prod.go) wraps the canonical chained audit.Sink so every Domain Create / Read / Update / Delete row lands on the per-Domain hash chain. The chained Sink reaches the adapter via Runtime.AuditSink, which the audit surface publishes through its Export; the registry lists it ahead of every surface that writes an audit row. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the domains factory closure returns ErrDomainsAuditSinkRequired when Runtime.AuditSink is nil so a misconfigured rollout cannot silently degrade /v1/domains to slog-only emission. TestRun_SurfaceRegistry_BuildSeesChainedAuditSink (cmd/plexsphere/app_surfaces_test.go) pins that a surface builds with the chained Sink on Runtime.AuditSink.
Project CRUD services.AuditSink / projects.AuditSink*projectsAuditAdapter (cmd/plexsphere/projects_factory_prod.go) wraps the canonical chained audit.Sink. Project Create already emits domain:<id> and forwards verbatim; Project Update / Delete emit project:<id>, pre-resolved to the owning Domain's domain:<hex> anchor via the shared projectDomainLookup (ProjectRepo.Get), with the original aggregate-shaped Object preserved on Entry.RelationPath. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the projects factory closure returns ErrProjectsAuditSinkRequired when Runtime.AuditSink is nil. A project whose owning Domain cannot be resolved fails closed with audit.ErrDomainUnresolved.
Cloud CRUD auditport.Sink (service) / clouds.AuditSink (transport)Two adapters (cmd/plexsphere/clouds_factory_prod.go) wrap the canonical chained audit.Sink: newCloudsServiceAuditAdapter for the application service's canonical auditport.Entry (Create / Update / Delete), and newCloudsAuditAdapter for the transport-local outcome-string entry (Get / List / denial reads). Clouds is entirely platform-scoped — the clouds table has no domain_id. Cloud Create emits the platform anchor object; Cloud Update / Delete emit cloud:<id>. Both shapes forward verbatim onto the chained Sink; the defaultDomainResolver routes platform: and cloud: onto the reserved platform anchor so every Cloud row lands on the single platform-residency chain read through /v1/platform/audit/.... The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the clouds factory closure returns ErrCloudsAuditSinkRequired when Runtime.AuditSink is nil.
Blueprint Catalog auditport.Sink (service) / blueprints.AuditSink + blueprintcatalogs.AuditSink (transport)Two seams onto the canonical chained audit.Sink: newBlueprintsServiceAuditAdapter (cmd/plexsphere/blueprints_factory_prod.go, shared by both the /v1/blueprints and /v1/blueprint-catalogs factories) forwards the application service's canonical auditport.Entry — Register / PublishVersion / Retire and the composed catalog-source RegisterSource / DeregisterSource / Import success rows — while newBlueprintsTransportAuditAdapter and newBlueprintCatalogsTransportAuditAdapter forward the two transport-local outcome-string entries (the blueprint.list / blueprint.get / catalog_source.* reads and the denial / refusal rows). The Blueprint Catalog is platform-managed configuration: Register and the listings emit the platform anchor object, PublishVersion / Retire / Get emit blueprint:<id>, and the catalog-source flows emit the platform anchor or domain:<id>. Every shape forwards verbatim onto the chained Sink; the defaultDomainResolver routes platform:, blueprint:, and domain: onto their residency anchor so each row lands on the correct chain. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the blueprints and blueprint-catalogs factory closures return ErrBlueprintsAuditSinkRequired / ErrBlueprintCatalogsAuditSinkRequired when Runtime.AuditSink is nil so a misconfigured rollout cannot silently degrade the authorship / catalog-source surfaces to slog-only emission.
Invitation invservices.AuditSink / invitationstx.AuditSink*invitationsAuditAdapter (cmd/plexsphere/invitations_factory_prod.go) wraps the canonical chained audit.Sink. Invitation Create / List emit domain:<id> and forward verbatim; Read / Revoke / Accept emit invitation:<id>, pre-resolved to the owning Domain via invitationDomainLookup (InvitationRepo.GetByID), with the original Object preserved on Entry.RelationPath; the ExpirePending sweep emits the platform anchor object and forwards verbatim onto the platform-residency chain. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the invitations factory closure returns ErrInvitationsAuditSinkRequired when the chained audit.Sink is nil. The sink reaches the closure through Runtime.AuditSink, which the audit surface publishes through its Export before the invitations surface builds. An invitation whose owning Domain cannot be resolved fails closed with audit.ErrDomainUnresolved.
Identity listing identities.AuditSink*identitiesAuditAdapter (cmd/plexsphere/identities_factory_prod.go) wraps the canonical chained audit.Sink. The list / permission-denied paths emit domain:<id> and forward verbatim; the per-principal Get path emits user:<id> or serviceaccount:<id>, pre-resolved to the owning Domain via identityDomainLookup (UserRepo.GetByID / ServiceIdentityRepo.Get), with the original Object preserved on Entry.RelationPath. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the identities factory closure returns ErrIdentitiesAuditSinkRequired when Runtime.AuditSink is nil, which cmd/plexsphere/identities_audit_adapter_test.go pins. The surface registry builds the surface only after Run has threaded the chained sink onto the Runtime. A principal whose owning Domain cannot be resolved fails closed with audit.ErrDomainUnresolved.
Admin surface adminhttp.AuditSinkchainedAdminAuditAdapter (cmd/plexsphere/admin_factory_prod.go) wraps the canonical chained audit.Sink so every /v1/admin/* refusal, accepted mutation, and successful read lands on a hash chain. domain:<id> and platform:plexsphere Objects forward verbatim, so Domain-scoped Group, IdP-binding, and token-list rows land on that Domain's chain and shared IdP-binding rows on the platform-residency chain. The membership (group:<id>) and API-token (token:<id>) Objects do not name a Domain: the adapter anchors them on AuditEntry.DomainID, rewriting the Object to domain:<id> and preserving the original on Entry.RelationPath. API-token rows reside on the token owner's Domain, the Domain the manage gate checked, so a platform operator's token rows land on the reserved platform-operators Domain. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the admin factory closure returns ErrAdminAuditSinkRequired when Runtime.AuditSink is nil so a misconfigured rollout cannot silently drop the admin rows.
Label Registry auditport.Sink*labelsAuditAdapter (cmd/plexsphere/labels_factory_prod.go) wraps the canonical chained audit.Sink. The label services now emit the canonical auditport.Reason directly, so the adapter carries Reason through verbatim — the former labels-local AuditReason mirror and its boot-time ordinal-lockstep assertion are gone. Domain-scope definitions emit domain:<uuid> and forward verbatim; project-scope definitions emit project:<uuid>, pre-resolved to the owning Domain; labeldefinition:<id> (definition delete + assignment) is pre-resolved via the definition's scope; the platform-scope sentinel plexsphere:platform is rewritten to the canonical platform anchor object so platform-scope label rows land on the platform-residency chain. The Zedtoken and existing RelationPath are carried through; the original Object is appended on rewrite. The slog INFO breadcrumb is preserved as a defence-in-depth side emission.Closure-time refusal: the labels factory closure returns ErrLabelsAuditSinkRequired when Runtime.AuditSink is nil. TestRun_SurfaceRegistry_BuildSeesChainedAuditSink (cmd/plexsphere/app_surfaces_test.go) pins that a surface built after audit sees the chained Sink on Runtime.AuditSink. An object whose residency cannot be resolved — including the arbitrary <kind>:<id> assignment objectRef on a maintainer-relation denial — fails closed with audit.ErrDomainUnresolved.
Reachability evaluator AuditSink*chainedReachabilityAuditSink (cmd/plexsphere/evaluator_factory_prod.go) wraps the canonical chained audit.Sink so every Healthy → Stale → Unreachable transition lands on the per-Domain hash chain with Relation = "node_reachability.transition", Object = "domain:<hex16>", Reason = audit.ReasonGranted, and RelationPath = [from_state, to_state, reason]. The chained Sink reaches the adapter via Runtime.AuditSink, which the audit surface publishes before the evaluator surface builds. The companion reachabilityWriterAdapter folds the tenancy.NodeReachabilityChanged outbox row into the same transaction as UpdateReachabilityState via NodeRepo.TransitionAndEmit. The slog INFO breadcrumbs on both legs are preserved as defence-in-depth side emissions.Closure-time refusal: the evaluator factory closure returns ErrEvaluatorAuditSinkRequired when Runtime.AuditSink is nil so a misconfigured rollout cannot silently degrade reachability transitions to slog-only emission.

Deferred wiring legs ​

  • SpiceDB endpoint plumbing in cmd/plexsphere/main.go.cmd/plexsphere/main.go dials SpiceDB, builds a shared *internal/authz.Authorizer, and puts its Check closure on Config.AuthzCheck. The audit surface builder adapts that closure onto productionAuditConfig.AuthzCheck before BuildProductionAuditFactory runs. The factory refuses to build without a non-nil AuthzCheck (ErrAuditAuthzCheckRequired), collapsing the previous fail-closed staticAuditAuthorizer-as-default posture into a build-time gate. TestProductionSurfaces_Order (cmd/plexsphere/surfaces_prod_test.go) fails when the audit surface leaves ProductionSurfaces(), so the factory is never silently dropped.
  • Cross-Domain DomainResolver. The defaultDomainResolver extracts the Domain UUID from the Authorizer Entry's Object field (domain:<hex16> shape); multi-Domain effects (cross-Domain System-admin actions) emit one row in the primary Domain only. A follow-up extends the resolver with the secondary Domain enumeration once the cross-Domain Authorizer Entry shape is finalised.
  • Per-aggregate audit emitters that still drop into slog only. Closed: every per-aggregate CRUD audit emitter now wraps the canonical chained audit.Sink. The Domain, Project, Cloud, Invitation, Identity, Label, and Admin adapters in cmd/plexsphere/*_factory_prod.go each forward their rows onto the chain following the dual residency model. A Domain-owned object segment (project:, invitation:, user:, serviceaccount:, and domain/project-scope labels) is pre-resolved to its owning Domain inside the adapter using the aggregate's own repo, with the original aggregate-shaped Object preserved on Entry.RelationPath. The Admin adapter anchors its group: and token: Objects on the Domain the handler carries on the entry instead of looking it up. A platform-scoped object segment that no Domain owns — Cloud lifecycle in full (the clouds table has no domain_id), platform-scope label definitions, and the invitation ExpirePending sweep — is routed to the platform-residency chain at the reserved platform anchor and read through /v1/platform/audit/.... Each adapter fails closed with audit.ErrDomainUnresolved when an object's residency cannot be determined, and each factory closure refuses to boot its surface (returning its per-surface Err…AuditSinkRequired sentinel) when the chained Sink was not threaded onto cfg.AuditSink. The slog INFO breadcrumb is preserved on every adapter as a defence-in-depth side emission. See the per-surface rows in the "Wiring leg" table above.
  • Per-aggregate cursor caller binding. Closed: the six per-aggregate cursor codecs (domains, projects, clouds, invitations, identities, nodesList) now sign <payload>||<version>||<caller_pseudonym> where the pseudonym is audit.Pseudonymise(ctx, pepper, principal.DomainID().UUID(), principalSubject(principal)). The wire-format version byte bumped from 0x01 to 0x02 in lockstep — pre-binding cursors decode as errXxxCursorInvalid ("unknown version byte" → 400 invalid_cursor) rather than as silently bind-free cursors. The transport boundary maps the new per-package errXxxCursorBindingMismatch sentinel onto 403 cursor_binding_mismatch so audit triage can distinguish a tampered-payload 400 from a cross-caller replay 403. The invariant is pinned by the table-driven contract in cmd/plexsphere/cursor_binding_contract_test.go (round-trip same-A, cross-caller A→B errors.Is mismatch, 0x01 envelope rejected, hmac.Equal discipline). See the "Per-aggregate cursor caller binding" paragraph under ## Invariants below.
  • Reachability evaluator outbox + chained audit emission. Closed: the Node aggregate now carries a non-zero projectID, NodeRepo.TransitionAndEmit folds the tenancy.NodeReachabilityChanged outbox append into the same transaction as UpdateReachabilityState, and the evaluator's AuditSink is *chainedReachabilityAuditSink wrapping the canonical chained audit.Sink. See the "Wiring leg" table row immediately above. The slog INFO breadcrumb is preserved on both legs as a defence-in-depth side emission.

A fresh production deployment that sets PLEXSPHERE_DSN, PLEXSPHERE_AUDIT_CURSOR_HMAC_KEY, PLEXSPHERE_AUDIT_PEPPER_MOUNT, PLEXSPHERE_AUDIT_PEPPER_ENDPOINT with an AppRole (PLEXSPHERE_AUDIT_PEPPER_APPROLE_ROLE_ID and PLEXSPHERE_AUDIT_PEPPER_APPROLE_SECRET_ID) or Kubernetes (PLEXSPHERE_AUDIT_PEPPER_K8S_ROLE) auth strategy, PLEXSPHERE_AUDIT_ARCHIVE_BUCKET, and the PLEXSPHERE_S3_* family observes:

  • the platform pepper created in OpenBao at boot, and each Domain's pepper created on first use;
  • the hash-chained Sink writing every privileged action to its per-Domain chain;
  • the four read handlers serving 403 (until the SpiceDB plumbing lands) and otherwise 200 with live Postgres rows;
  • the per-Domain reconciler red-lighting /readyz on a chain divergence and writing one audit_tamper_quarantine row per detected (domain, seq) tuple;
  • the archiver draining audit_log_entry rows out to the bucket at the canonical audit/<domain-uuid>/<seq:020d>.json.zst key shape (archiver.KeyFor).

Invariants ​

Canonical SpiceDB subject form ​

Every audit-side call that issues a (subject, relation, object) triple against the SpiceDB-backed authorizer (Service.List for the page read, the auditor gate on GET /v1/domains/{domain_id}/audit/entries/{seq} and POST .../audit/verify, and the manage gate on POST .../audit/erase-identity) MUST project the internal/identity/authn.Principal through authn.Subject so the resulting subject string carries the canonical type:id form:

  • KindUser → user:<uuid>
  • KindService → serviceaccount:<uuid>
  • KindAPIToken → apitoken:<uuid>
  • KindUnknown → unknown:<uuid>

Calls that bypass authn.Subject and pass principal.ID().String() directly surface as is not of the form type:id inside the authz adapter's parser, which collapses every audit list / get / verify / erase call onto an Internal Server Error before any persistence read happens. The shared helper lives in internal/identity/authn/subject.go; the per-aggregate transport packages (domains, projects, bootstraptokens, invitations, identities, labels, clouds, admin, authz) wrap it from their local helpers.go so call sites stay idiomatic and the canonical form is a single drift seam.

The chain-row AuditEntry.Subject on audit_list / audit_get / audit_verify / audit_erase self-audit rows uses the SAME canonical form so a forensic correlation across audit.<surface> rows by Subject joins cleanly across every surface. The unit pin lives at internal/identity/authn/subject_test.go; the audit-side regression on the query gate is TestList_SubjectIsCanonicalSpiceDBForm in internal/audit/query/service_test.go, mirrored at the transport boundary by TestGetAuditEntry_AuthzSubjectCanonicalForm in internal/transport/http/v1/handlers/audit_get_test.go, TestVerifyAuditChain_AuthzSubjectCanonicalForm in audit_verify_test.go, TestEraseIdentityFromAudit_AuthzSubjectCanonicalForm in audit_erase_test.go, and TestListAuditEntries_AuditRowSubjectCanonicalForm in audit_list_test.go.

Per-aggregate cursor caller binding ​

The six per-aggregate cursor codecs (domains, projects, clouds, identities, invitations, nodesList) bind the HMAC envelope to a per-(caller, pepper) projection so a cursor minted by user A cannot be replayed by user B. The MAC is computed over payload || version_byte || caller_pseudonym, where:

  • payload is the surface-specific opaque continuation token (a slug for the slug-paginated surfaces, a JSON-then-base64-wrapped ListCursor tuple for the keyset-paginated ones);
  • version_byte is 0x02 after this binding leg landed (0x01 was the pre-binding wire format and now decodes as errXxxCursorInvalid → 400 invalid_cursor);
  • caller_pseudonym is audit.Pseudonymise(ctx, pepper, principal.DomainID().UUID(), authn.Subject(principal)) — the same primitive the audit context uses for forensic-stream subject pseudonymisation, with the per-Domain pepper coming from the shared resolver cmd/plexsphere/main.go threads into each factory, which checks it with requireAuditPepper. The subject projection lives in internal/identity/authn/subject.go and is the same canonical form documented under Canonical SpiceDB subject form above.

Decode-time taxonomy:

  • A malformed base64 envelope, a payload-too-short envelope, or an unknown version byte routes to errXxxCursorInvalid and the transport layer maps it to 400 invalid_cursor.
  • A failed constant-time MAC equality (hmac.Equal) check under a valid version byte routes to the new errXxxCursorBindingMismatch sentinel and the transport layer maps it to 403 cursor_binding_mismatch. Audit triage uses the distinct status code to differentiate a tampered-payload 400 from a cross-caller cursor replay 403.

Rotation posture: a per-Domain pepper rotation invalidates outstanding cursors loudly (decode → 403) and clients re-paginate from page 1. This is inherited from the audit.Pseudonymise contract — pepper rotation is out of scope for both the audit and the cursor-binding surfaces and is tracked separately as part of the tenant-side erasure UX work.

DECISION: per-codec implementations rather than a shared package. The six codecs are independently rotatable via independent PLEXSPHERE_<SURFACE>_CURSOR_HMAC_KEY env vars (the per-factory DECISION blocks in cmd/plexsphere/*_factory_prod.go pin that rationale). The per-(caller, pepper) projection is shared through the cmd/plexsphere/cursor_binding.go helper so the binding contribution to the MAC is byte-for-byte identical across the six codecs, but the rotation surfaces stay independent.

Platform residency chain ​

plexsphere runs a dual residency model. A Domain-owned privileged action lands on that Domain's chain; a platform-scoped action owned by no Domain lands on a single platform-residency chain. The two classes are disjoint and both are hash-chained with the identical machinery (head, advisory lock, verify, reconcile, pepper, pseudonym, erasure) — the platform chain is not a fallback that absorbs unresolved entries.

  • Reserved anchor. The platform chain is anchored at the fixed identifier 00000000-0000-0000-0000-706c6174666d. It is structurally outside the generated-UUID space: a Domain id is a UUIDv7, whose version nibble and RFC 4122 variant bits are set, whereas the anchor zeroes both. Disjointness from any present or future Domain id is therefore a property of the id space, not an allocation-time exclusion list. The anchor is non-zero, which the chained Sink requires (a zero primary anchor is rejected as ErrDomainUnresolved).

  • Routing. The resolver maps every platform:<...> Object (canonically platform:plexsphere) and every cloud:<...> Object to the reserved anchor. The clouds aggregate has no domain_id, so Cloud Create (platform:plexsphere), Cloud Update / Delete (cloud:<id>), platform-scope Label definitions, and the Invitation ExpirePending sweep all reside here. The resolver stays persistence-free: Domain-owned non-domain: shapes (project: / invitation: / user: / serviceaccount: / domain-scope label) are pre-resolved to domain:<hex> inside their own per-aggregate adapter before the Entry reaches the Sink, so the resolver never has to enumerate or query a Domain.

  • Fail-closed boundary. An Object that is neither a recognised platform shape nor a domain:<uuid> shape still surfaces ErrDomainUnresolved. Platform-scoped is a resolved state, not an unresolved one; the catch-all-suppression risk the former no-system-chain invariant guarded against is preserved by keeping the unresolvable case fail-closed.

  • Read surface and authorization. Platform-chain rows are read through the dedicated /v1/platform/audit/... endpoints, gated by the platform-level auditor relation (manage for erasure), not the per-Domain domain#auditor. A per-Domain auditor cannot read the platform chain and vice versa, so the residency boundary is also an authorization boundary.

  • Right-to-erasure. The platform chain is erasure-capable on the same mechanics as a per-Domain chain: the subject is pseudonymised and the erasable plaintext mapping lives in the subject-PII sibling table keyed under the reserved anchor.

OpenBao policy ​

The API's OpenBao credential for the audit pepper store needs create and read on the pepper paths and nothing else:

hcl
path "<mount>/data/audit/pepper/*" {
  capabilities = ["create", "read"]
}
  • update is withheld, so the API cannot rotate or overwrite a pepper. The resolver creates a pepper with check-and-set 0, which succeeds only while the path is empty.
  • Two replicas that race on a fresh Domain converge on one pepper. The loser's create is refused, by a check-and-set mismatch or a denied write, and its single re-read adopts the winner's value.
  • The API creates a pepper only for the platform anchor or a Domain that exists, and only while that anchor's audit chain holds no rows. A request that names a made-up Domain id writes no secret.
  • A pepper that is missing for a chain that holds rows is never replaced, whether it was deleted, the mount is empty, or the store was restored from an older backup. A new pepper would give every recorded subject a second pseudonym, and erasure would no longer find the rows written before. Audit writes and erasure requests for that chain fail with pepper.ErrPepperLost until an operator restores the secret, and a missing platform pepper stops the boot.
  • A stored value that is not the base64 encoding of 32 bytes is rejected as pepper.ErrMalformed and never repaired.

Boundary ​

The Platform Audit Log is the substrate. Several downstream work items build on it; getting the boundary right matters because the substrate's invariants are load-bearing for every consumer.

The cleanest boundary cases — the ones a future contributor is most likely to confuse — are the node-side observability ingestion path and the approval-workflow correlation-ID lifecycle. Both touch "audit", but neither belongs in this context.

  • Observability ingestion (node-side audit).POST /v1/nodes/{id}/audit ingests batched JSON Lines from Linux auditd (via AF_AUDIT Netlink) or Kubernetes audit log tailing. That stream lands in Grafana Loki with a separate retention class and a distinct SIEM route; it does NOT flow through the hash-chained Postgres surface this context owns. The depguard rule no-node-audit-on-platform-chain keeps the two streams structurally separate; a future contributor cannot silently route the node-side stream through the operator-action chain. SIEM forwarder lifecycle, retry semantics, and per-sink schema mapping belong with the observability ingestion surface, not this context.
  • Approval workflow correlation IDs. The Platform Audit Log records every correlation_id that arrives on an Entry, but the dual-control state machine that issues correlation IDs spanning the proposal / approval / rejection / break-glass transitions lives in the approval workflow context. The chain is the evidence stream; the approval workflow produces the IDs the evidence groups by.

The other deferred items, recorded for completeness:

  • Dashboard audit-log viewer. The browser-facing audit-log viewer and its end-to-end coverage live in the separate dashboard project; this context ships the API substrate that viewer consumes.
  • plexctl audit CLI client. This context ships the typed Go OpenAPI client and the integration tests that exercise the API directly. CLI ergonomics (filter flags, output formats, paging glue) belong with the rest of plexctl's initial surface.
  • Tenant-side erasure UX. The EraseIdentityFromAudit API ships here; the Dashboard confirmation flow, downstream-impact preview, and operator ergonomics around the erasure event belong with the tenant-side erasure UX work.
  • Pepper rotation. This context fixes the pepper at Domain creation and rejects rotation with a typed error (ErrPepperUnavailable). Pepper rotation requires a re-pseudonymise-and-rechain workflow that the tenant-side erasure UX work owns.
  • Sovereign archive export. The promise — per-Domain audit export in JSON / CSV — is an operator-initiated flow that lands with the tenant-side erasure UX work. The substrate is here; the export surface is downstream.