plexsphere

Appearance

Policy Compiler — per-Node projection of a Policy revision ​

This document is the authoritative bounded-context reference for the Policy Compiler that ships alongside the Network Policy Engine under ../../../internal/policy/compiler/ and ../../../internal/policy/services/compile_service.go. It covers the ubiquitous language, the single Postgres table that backs the per-Node projection, the four-stage projection algorithm, the SHA-256 fingerprint contract that powers plexd's hash-and-skip reconcile, the outbox-consumer wiring that drives the compile pass in lockstep with the existing SpiceDB tuple-write arm, the PolicySource seam onto the reconciliation-pull snapshot, the relationship to the future wire-side fan-out story, the closed two-event guarantee the compiler relies on, and the depguard rules that keep this surface isolated from its siblings.

The Policy Compiler is the per-Node materialisation surface for L3/L4 network policy — and only that. It consumes policy_revision_created and policy_deleted outbox rows the Network Policy Engine writes, projects the published revision onto every Node whose label snapshot satisfies the head revision's selector, and persists one plexsphere.policy_compiled_ruleset row per (Node, Policy) tuple inside the same transaction as the SpiceDB tuple-write arm. plexd ingests the row through the reconciliation-pull snapshot at GET /v1/nodes/{id}/state until the future wire-side fan-out story ships the per-Node policy_updated SSE event.

This is NOT the Policy aggregate. The operator-authored Policy aggregate, the append-only Revision history, the L3/L4 Rule grammar, the dry-run preview, and the ReBAC ownership matrix live in model.md — a reader looking for "how do I author a Policy?" belongs there. This document covers the downstream materialisation arm that turns a published Revision into per-Node-applied bytes.

For the bounded-context siblings and upstream platform references see:

  • ./model.md — Network Policy Engine, the upstream authoring surface that writes the policy_revision_created and policy_deleted events this compiler consumes.
  • ../labels/index.md — Label Registry and the SelectorPort seam the compile service consumes through an anti-corruption adapter to enumerate matched Nodes.
  • ../mesh/peers.md — Key & Peer Manager, the source of the per-Domain Node projection the compile service intersects with the selector match.
  • ../mesh/reconciliation-pull.md — the NodeStateSnapshot envelope that carries the per-Node compiled ruleset in its policy block.
  • ../identity/rebac.md — ReBAC schema; the compiler does not gate any HTTP surface, but the outbox row carries the same project / policy ids the upstream ReBAC tuples reference.
  • ../../architecture/storage-topology.md — the shared Postgres node that backs plexsphere.policy_compiled_ruleset and the Down-refusal rule.
  • ../../contributing/layout.md — the bounded-context map row for internal/policy enumerating the CompiledRulesetRepo and LabelSnapshotPort outbound ports this document pins.

Ubiquitous language ​

Six terms travel verbatim across the Go code, the SQL schema, the OpenAPI surface (NodeStatePolicy / NodeStateCompiledRule), the outbox event payloads, and the operator-facing reconciliation snapshot. Internal code never paraphrases them; documentation and error messages adopt the exact spelling below.

TermDefinitionCode anchor
Compiled RulesetThe per-(Node, Policy) projection of a Policy revision's canonical rule list. Carries the byte-stable rule_payload plexd applies, the 32-byte SHA-256 fingerprint plexd uses for hash-and-skip, the source RevisionID, and the wall-clock compiled_at timestamp. Value object with structural equality.CompiledRuleset in ../../../internal/policy/ports.go
Per-Node ProjectionThe four-stage (canonical sort → per-Node selector filter → JSON marshal → SHA-256) pipeline the compiler runs when a policy_revision_created event arrives. Two replicas running the same code against the same inputs MUST emit byte-identical bytes.Compiler.Compile in ../../../internal/policy/compiler/compiler.go
Fingerprintsha256.Sum256(json.Marshal(canonical_rules)) — the 32-byte digest stored in the fingerprint bytea column. plexd compares the row's fingerprint against its locally-applied ruleset's hash to short-circuit the apply when the bytes already match. The empty-projection fingerprint equals sha256.Sum256([]byte("[]")).Compiler.Compile stage 4 in ../../../internal/policy/compiler/compiler.go
Rule PayloadThe byte-stable JSON bytes the fingerprint covers, stored verbatim in the rule_payload JSONB column. Carries the canonical-sorted rule list encoded through a shadow struct whose field order is pinned by Go's encoding/json declaration-order rule, so the JSON ordering depends on the shadow shape rather than the input slice's address-space layout. A caller that needs the bytes to re-derive the fingerprint MUST use RulePayload (not a re-marshal of Rules).RulePayload on CompiledRuleset in ../../../internal/policy/ports.go
Determinism ContractThe guarantee that two replicas of cmd/plexsphere processing the same outbox event insert byte-identical rule_payload + fingerprint columns. The Go compiler is the primary enforcer; the SQL CHECK length(fingerprint) = 32 is defence-in-depth. The contract is what makes plexd's hash-and-skip reconcile path correct under replicated control planes.determinism_property_test.go in ../../../internal/policy/compiler/
Compile ArmThe sibling outbox-consumer arm at internal/authz/sync/consumer.go that runs CompileService.HandleEvent AFTER the SpiceDB tuple-write succeeds. The two arms share one retry boundary: the row is marked applied_at only when both succeed, so a transient compile failure replays both arms — the upsert is idempotent on (node_id, policy_id), so a re-fire that lands the same fingerprint twice is harmless.Consumer.dispatch in ../../../internal/authz/sync/consumer.go

The future story that closes the wire-side gap is named verbatim in internal/policy/compiler/compiler.go and in the package-level comment of internal/policy/services/compile_service.go; there is no rename in flight. A new term added to this table requires the same triple-update discipline the Network Policy Engine follows: code, schema, and documentation in one commit.

Schema reference ​

The Policy Compiler attaches to a single table in the plexsphere schema. The migration that creates it is ../../../internal/platform/db/migrations/0034_policy_compiled_rulesets.sql. Every CHECK constraint, partial-unique index, and ON DELETE behaviour described below is sourced from that file.

TablePurposeNotable constraints
plexsphere.policy_compiled_rulesetOne row per (Node, Policy) tuple whose label snapshot satisfies the Policy's selector. Carries the source revision id, the SHA-256 fingerprint, the byte-stable rule payload, and the wall-clock compile timestamp.UNIQUE (node_id, policy_id) — the natural key; a Node whose labels match multiple Policies receives one row per Policy. FK policy_id REFERENCES plexsphere.policy ON DELETE CASCADE — a deleted Policy's compiled rows are purged when the parent aggregate disappears, with the explicit policy_deleted consumer arm covering the case where the cascade has not yet fired. CHECK length(fingerprint) = 32 pins the SHA-256 length at the SQL layer.

Why (node_id, policy_id) is the natural key ​

A Node whose labels match multiple Policies receives one compiled row per Policy. The DECISION block at the top of the migration enumerates the rejected alternative — a UNIQUE (node_id) constraint that collapses every matching Policy into a single per-Node row — and spells out why it loses load-bearing semantics: a per-Node compaction would force the application service to merge rule lists from independent Policies before the upsert, which violates the operator-visible separation between Policies in the Revision history and breaks the per-Policy purge semantics of policy_deleted.

Down-refusal — SQLSTATE 0A000 ​

The migration's Down block deliberately refuses the downgrade and raises SQL exception code 0A000 (feature_not_supported). The compiled-ruleset rows are the in-cluster forwarding-table source of truth a plexd Node ingests through the reconciliation-pull snapshot; dropping the table would force every Node to re-converge from an empty starting state and would silently lose the per-Node hash-and- skip optimisation until the next outbox event re-populates each row. The Down-refusal pattern mirrors the convention 0033_policies.sql established for the upstream Policy aggregate and the seven prior migrations that protect security- and audit-critical material; see ../identity/registration.md for the broader Down-refusal taxonomy.

Projection algorithm ​

The compiler runs a four-stage projection against a single Policy revision and a single addressed Node. Every stage is deterministic; the package-level doc on internal/policy/compiler/compiler.go pins the contract:

  1. Canonical sort. canon.Canonicalise pins the rule order over the stable key (SourceCIDR, DestinationCIDR, Protocol, Ports.From, Ports.To, Action). Two input slices that hold the same rule set in different orders produce identical canonical-sorted output. The canon sub-package is shared with the dry-run service and the diff sub-package so all three surfaces agree on the rule-ordering definition.
  2. Per-Node selector filter. The canonical list is included verbatim when the Node's labels satisfy at least one side of the Policy selector — source OR destination — and the projected slice is empty otherwise. Empty is a legitimate, fingerprint-equal state (see Fingerprint shape below). The selector evaluation lives in the caller (CompileService) so the compiler package stays free of the internal/labels dependency the depguard rule denies.
  3. JSON encoding. The projected slice is marshalled through a compiledRule shadow struct whose field order is fixed by Go's encoding/json declaration-order rule, yielding byte-stable bytes regardless of the input slice's address-space layout. The DECISION block on the shadow struct names the rejected alternative of exporting canon.canonicalRule and explains why the six-line duplication is cheaper than promoting a private encoder shape to a public coupling point.
  4. SHA-256 fingerprint. sha256.Sum256(json.Marshal(projected)) produces the 32-byte digest plexd's hash-and-skip reconcile reads to short-circuit the apply when the locally-applied ruleset already matches.

The compiler is stateless: a single compiler.Compiler instance is shared across goroutines and across consumer arms without coordination. The struct exists so a future v2 can carry configuration (a slog.Logger, a metrics sink) without breaking the public signature.

Fingerprint shape ​

The fingerprint is the load-bearing primitive that makes the hash-and-skip reconcile path correct. Three invariants pin the per-row (per-(Node, Policy)) shape — the layer the internal/policy/compiler/compiler.go package owns:

  • Exactly 32 bytes. sha256.Sum256 returns [32]byte; the SQL CHECK length(fingerprint) = 32 is the defence-in-depth that catches a future caller persisting a truncated or mis-typed digest.
  • Derived from RulePayload, not Rules. A caller that needs the bytes to re-derive the per-row fingerprint MUST use RulePayload and not re-marshal Rules. The JSON ordering depends on the compiledRule shadow shape rather than the public policy.Rule field order, so a naive json.Marshal(Rules) would produce byte-different output and a non-matching fingerprint.
  • Empty-projection fingerprint is stable. A Node whose label snapshot does NOT match the Policy selector receives a row whose Rules is the zero-length slice and whose RulePayload is the two-byte string []. The fingerprint equals sha256.Sum256([]byte("[]")) and is byte-identical for every such Node. The empty-match property test in internal/policy/compiler/compiler_test.go pins the value.

Multi-policy merge — the snapshot composer's reduction ​

The persistence layer stores one row per (Node, Policy), but the reconciliation-pull NodeStateSnapshot.Policy block carries a single merged shape — one revision_id, one fingerprint, one rules array. The merge runs at the composition root, in cmd/plexsphere/policy_source_adapter.go, and obeys a deterministic three-part contract:

  1. Sort by policy_id ascending. The adapter performs a defensive re-sort on a local copy of the rows so a future repo implementation that breaks the persisted-order invariant cannot silently flip the merged fingerprint.
  2. Concatenate the raw rule_payload byte streams. The merged byte stream is payload_0 || payload_1 || ... || payload_n where payload_i is the canonical RulePayload of the i-th row in the sorted order. The operation is on raw bytes — no re-marshal, no re-canonicalisation.
  3. Hash the merged stream and project the wire fields. The block's fingerprint equals sha256.Sum256(payload_0 || payload_1 || ... || payload_n). The block's rules is the per-Policy concatenation of each row's decoded canonical rule list in the same order. The block's revision_id is drawn from the policy-id-ascending first row, so a single uuid remains on the wire — a Node matching a single Policy carries that Policy's revision id directly; a Node matching multiple Policies carries the first one (see the NodeStatePolicy.revision_id field description for the wire contract).

The merge formula is pinned by the composition-root unit test cmd/plexsphere/policy_source_adapter_test.go (TestPolicySourceAdapter_MultiRowMergesDeterministically) and mirrored by the integration-tier test adapter at tests/integration/state_snapshot_policy_block_test.go (policySourceTestAdapter.CompiledRulesetForNode). Both adapters re-sort the rows and concatenate-then-hash in identical order so a drift in either side surfaces as a test failure.

Plexd treats the wire fingerprint as an opaque comparison key. It compares the bytes against its locally-stored digest from the previous reconcile snapshot byte-for-byte; it does NOT re-derive the fingerprint from the rules array (the wire shape uses the NodeStateCompiledRule field naming, while the persisted rule_payload uses the shadow shape — a re-derivation would not match without a shape conversion the wire does not document).

DECISION — merge at the composition root ​

The merge lives in the composition-root adapter rather than in the policy aggregate or the snapshot composer. Hoisting it into internal/policy would couple the policy aggregate to internal/mesh/state's wire shape; hoisting it into internal/mesh/state would require importing internal/policy and would trip the no-cross-context-imports depguard rule. The composition-root adapter is the canonical seam for inter-context translation — same reasoning the peerSourceAdapter cites for living next to the pgx pool. The DECISION block at the top of cmd/plexsphere/policy_source_adapter.go carries the rationale.

Determinism contract ​

The 50-iteration × 5-permutation property test internal/policy/compiler/determinism_property_test.go enumerates deterministic (revision, peer-set, label-snapshot) triples and asserts bytes.Equal on both Fingerprint and json.Marshal(Rules) across every permutation. The test uses no math/rand — every input is deterministically derived from a seed counter so a CI re-run reproduces the exact same byte streams. A regression in any stage of the projection (a non-stable sort, a shadow-struct field re-order, a JSON marshaller change) trips the test.

Outbox-consumer wiring ​

The compiler runs as a sibling arm of the existing SpiceDB tuple-write consumer at internal/authz/sync/consumer.go. The two arms share one retry boundary: the outbox row is marked applied_at only when both succeed. A transient failure in either arm replays both — the SpiceDB writes are idempotent on the relation tuple shape, and the compiled-ruleset upsert is idempotent on (node_id, policy_id), so a re-fire that lands the same fingerprint twice is harmless (the same-fingerprint upsert flips zero rows).

Per-event dispatch ​

The compile service's HandleEvent dispatches on the outbox row's event type:

Event typeCompile-arm behaviour
policy_revision_createdLoad the Policy + head revision via policy.Repo.Get. List matched Nodes for source and destination selectors via SelectorPort.List. Union the two sets, remembering which side each Node matched. Call Compiler.Compile per Node with the correct (sourceMatches, destMatches) booleans. Upsert each CompiledRuleset via CompiledRulesetRepo.UpsertForNode.
policy_deletedIssue a single CompiledRulesetRepo.DeleteByPolicy(policyID) call. The repo's delete is idempotent: a re-fire on an already-purged Policy returns nil.
any other typeNo-op. The same payload reaches the SpiceDB arm and the compile arm; each arm filters on its own discriminator so an unrelated event (e.g. domain_created) flows past the compile arm without contention.

DECISION — SpiceDB writes run BEFORE the compile arm ​

The DECISION block above Consumer.dispatch names the ordering and the rejected alternative. SpiceDB writes run first because the SpiceDB write surface is the request-authorisation seam — a compile arm that succeeded against a Policy whose ReBAC tuples had not yet landed would let a downstream reader observe an applied ruleset for a Policy the ReBAC graph still says they cannot see, which violates the visibility invariant the dual-check policy ownership matrix in model.md pins. The compile arm runs after the SpiceDB write so a visibility-leak window cannot open.

Head-revision drift ​

The compile service always works against the head revision of the addressed Policy, not the event's RevisionID. If a faster PATCH has already advanced the head between the outbox emit and the consumer pickup, the head compile is the correct outcome — a later event for the new head will be a no-op because the Upsert lands the same fingerprint and flips zero rows. The DECISION block in handleRevisionCreated pins the rationale.

PolicySource seam ​

The compiled rows reach plexd through the reconciliation-pull snapshot envelope NodeStateSnapshot.Policy. The seam between the policy bounded context (which owns the persisted row) and the mesh bounded context (which owns the snapshot envelope) is the PolicySource port declared at internal/mesh/state/policy_source.go:

go
type PolicySource interface {
    CompiledRulesetForNode(ctx context.Context, nodeID uuid.UUID) (*Policy, error)
}

Domain-shaped boundary types ​

The port is intentionally declared with package-local uuid.UUID and *state.Policy instead of forwarding policy.CompiledRulesetRepo's tenancy.ID + policy.CompiledRuleset shapes. The no-cross-context-imports depguard rule denies internal/policy and internal/identity imports from internal/mesh/state; mirroring the PeerSource and ReachabilitySource posture keeps the rule satisfied without widening any exception. The translation lives in the composition root at ../../../cmd/plexsphere/policy_source_adapter.go, which wraps the production policy.CompiledRulesetRepo and maps the tenancy.ID + policy.CompiledRuleset row into the uuid.UUID + state.Policy shape the composer reads.

Nil-policy steady state ​

A (nil, nil) return is the canonical "no compiled row for this Node" steady state. The composer threads the nil onto NodeStateSnapshot.Policy unchanged; the transport layer's JSON encoder maps it to JSON null. A Node that has not yet been matched by any Policy carries policy: null on the snapshot and plexd treats this as "no L3/L4 forwarding rules to apply" — additive evolution: a populated policy block is a strict superset of the nil case, so a plexd build that predates the policy block ignores it and a build that knows about it tolerates the nil case.

Relationship to the wire-side fan-out story ​

The compiler ships the reconciliation-pull half of the per-Node policy convergence. The SSE wire-side half — the per-Node policy_updated event that fans the change out over the Signed Event Bus so a Node receives a push rather than waiting for the next pull tick — is now live alongside the compiler. See ./events.md for the full explanation of the producer-side closure (translation seam, fan-out algorithm, payload schema, Nats-Msg-Id dedup, Last-Event-ID resume, and the deferred relay-loop posture).

The compile service consumes the policy.WirePublisher port declared in ../../../internal/policy/ports.go to emit per-Node fan-out as a sibling of the per-(Node, Policy) upsert:

  • PublishCompiledRulesetUpdated(ctx, cr) runs once per matched Node immediately after CompiledRulesetRepo.UpsertForNode succeeds for that Node. The adapter at ../../../cmd/plexsphere/policy_wire_publisher_adapter.go constructs an outbox-shaped envelope carrying the row's RulePayload and EventType: "policy_revision_created" so the publisher's wireTypeFor dispatch maps it onto the wire literal policy_updated.
  • PublishPolicyDeleted(ctx, policyID, nodeIDs) runs once per delete with the slice of previously compiled Nodes; the adapter fans out one publish per Node carrying the canonical empty-rules payload []byte("[]") and EventType: "policy_deleted" so the same wireTypeFor dispatch maps each onto policy_updated. The compile service's handleDeleted MUST list the matched Nodes via CompiledRulesetRepo.ListNodesByPolicy before calling DeleteByPolicy; the lookup-before-cascade order is pinned by the DECISION block on the function.

The port is nil-tolerant: a composition root that has not yet threaded the adapter passes nil, the compile service skips the publish, and the per-Node compiled row remains the authoritative source of truth for the next reconciliation-pull tick. Three properties keep the producer-side closure correct under replicated control planes:

  • The compiler row is the source of truth for the Node-applied ruleset; the wire event carries the same fingerprint as the row, so a Node that receives the wire event and re-fetches the pull snapshot observes identical bytes.
  • The wire emission runs in the same application-service layer as the per-Node upsert, so the matched-Node identity and the canonical bytes are already in hand — no extra Postgres round-trip, no widening of the outbox envelope shape.
  • The wire-side literal mapping (policy_revision_created / policy_deleted → policy_updated) lives in wireTypeFor on ../../../internal/mesh/sse/publisher.go, so the policy context's closed two-event outbox set (see Closed-event-set guarantee below) stays unchanged while the wire-side enum carries the single literal the consumer needs.

The producer-side closure is complete; the remaining deferral is the relay-loop side (the RelayStore adapter, the EnsureNodeEventsStream call, and the RunRelay goroutine) owned by the Signed Event Bus epic per ../../architecture/mesh-event-bus-roadmap.md. The end-to-end fixture at tests/e2e/policy/policy-updated/chainsaw-test.yaml exercises the producer-side closure against the SSE stub; the companion fixture at tests/e2e/policy/policy-compiler/chainsaw-test.yaml continues to cover the reconciliation-pull half.

Closed-event-set guarantee ​

The Policy Compiler relies on the upstream Network Policy Engine's closed two-event outbox set: policy_revision_created and policy_deleted. The set is enforced by the AST-walking gate tests/workspace/policy_event_type_set_test.go. The compile service's HandleEvent dispatch is therefore a closed two-arm switch — a future event type that escapes the gate would surface here as a missing branch at consumer-test time.

Adding a third event type is a coordinated change across:

The four moves land together in one commit; the gate, the dispatch table, and the consumer wiring stay in lockstep.

Depguard rules ​

The compiler's bounded-context isolation extends the rules pinned by the upstream Policy aggregate. The full rule definitions live in ../../../.golangci.yml; the bounded-context map row for internal/policy in ../../contributing/layout.md enumerates them in operator-facing form.

RuleWhat it denies (compiler-relevant)Carve-outs
no-cross-context-imports-policyImports of internal/labels/**, internal/mesh/**, and internal/identity/** (except internal/identity/tenancy and internal/identity/tenancy/events) from any file under internal/policy/**, including the compiler sub-package and the compile service. The compiler reaches the label selector evaluator exclusively through SelectorPort; the labels selector AST is NEVER imported directly. The label snapshot is reached through LabelSnapshotPort — currently unused by the happy path but wired onto CompileService for the future per-Node re-evaluation pass named in the DECISION block in HandleEvent.internal/policy/repo/** is excluded from the deny-list so the persistence adapter at internal/policy/repo/compiled_ruleset.go can import the narrow pgx-shaped helpers it needs.
no-cross-context-imports (the global rule)Imports of internal/policy/** from any other bounded context, including internal/mesh/state (which reads the compiled rows through the PolicySource port instead).The transport layer (internal/transport/http/**) and the composition root (cmd/plexsphere/**) are the two allowed callers — the composition root binds the production policy.CompiledRulesetRepo against the mesh/state.PolicySource port via the adapter at cmd/plexsphere/policy_source_adapter.go.
no-direct-persistence-from-contexts (the global rule)Imports of github.com/jackc/pgx/** from any internal/<context>/ directory, including internal/policy/compiler/** and internal/policy/services/**.internal/policy/repo/** is on the allow-list — the persistence adapter for the compiled rows lives there and is the single seam under internal/policy that may speak pgx.

A new bounded context that needs to read the compiled-ruleset projection MUST declare a port in its own package and reach the projection through it — direct imports of internal/policy are denied. A new consumer of the closed outbox event set is a tuple-change in the AST gate and a new arm under internal/authz/sync or internal/policy/services; the moves stay in lockstep.

What this context is not ​

To keep the boundary sharp, the Policy Compiler is deliberately NOT:

  • The Policy aggregate. The operator-authored Policy, the append-only Revision history, the L3/L4 Rule grammar, the dry-run preview, and the ReBAC ownership matrix are owned by the Network Policy Engine documented in model.md.
  • The wire-side publisher. The internal/policy/compiler/ package emits no wire-side event and publishes nothing onto the Node-scoped SSE subject. The wire fan-out lives one layer up in the compile service, which reaches the SSE publisher exclusively through the policy.WirePublisher port; the production binding is the adapter at ../../../cmd/plexsphere/policy_wire_publisher_adapter.go. The wire literal policy_updated does NOT appear in this context's closed outbox event set — it is produced by the publisher-side dispatch table documented in ./events.md.
  • A per-Node label evaluator. The compiler accepts the (sourceMatches, destMatches) booleans from the orchestrator and does not evaluate label selectors itself. The label evaluation reaches the compile service exclusively through SelectorPort so the compiler sub-package stays free of the internal/labels dependency the depguard rule denies.
  • The plexd-side applier. plexd reads the compiled row through GET /v1/nodes/{id}/state, compares the row's fingerprint against its locally-applied ruleset's hash, and applies the rules to its forwarding table when the hashes differ. That applier is a plexd concern; the compiler ships only the source-of-truth row.

Cross-references ​