plexsphere

Appearance

Discovered PlexdHooks — Kubernetes custom resources projected, persisted, and diffed ​

This document is the authoritative bounded-context reference for the discovery-only PlexdHook surface — the seam between a cluster-hosted plexd agent's observation of upstream Kubernetes PlexdHook custom resources in its customer cluster and the durable projection plexsphere persists alongside the rest of a Node's capability manifest. It covers the ubiquitous-language pins, the Kubernetes-side projection that turns an unstructured custom resource into a five-field spec, the on-the-wire shape the agent carries on the capability-manifest PUT, the exhaustive catalogue of Problem codes the discovered-hook invariants emit, the set-keyed diff that folds into the existing capabilities outbox event without a new event type, the Postgres column migration 0038_plexd_hooks.sql adds, the trust-on-first-use hook-integrity gate that consumes the persisted catalog to refuse a drifted action dispatch, and the managed-push concern this slice deliberately leaves out of scope.

The PlexdHook surface is a read-only discovery projection — and only that. A plexd agent watches PlexdHook objects in its own cluster and advertises them on its capability manifest; plexsphere projects each object onto a stable five-field shape, persists the list on the per-Node manifest row, and folds a change to the list into the same NodeCapabilitiesUpdated outbox event the rest of the manifest already emits. The discovery surface deliberately does NOT write PlexdHook objects back into the customer cluster, does NOT resolve the image digest against a registry, and does NOT itself gate action dispatch. The trust-on-first-use hook-integrity gate (see Hook-integrity gating) is a separate consumer that pins a known-good baseline from this catalog and refuses a drifted or uncatalogued dispatch; the remaining managed-push and operator-read concerns live in named follow-up stories enumerated under Out of scope.

This surface is a sibling of the capability manifest ingest reference: the declared-script-hook list (declared_hooks) and the discovered-PlexdHook list (plexd_hooks) travel on the same PUT /v1/nodes/{id}/capabilities envelope, land on the same manifest row, and diff into the same outbox event. The two hook collections are independent — never conflated — and this document covers only the discovered-PlexdHook half; read the capabilities reference for the manifest envelope, the NSK authentication model, and the single-transaction ingest contract the two halves share.

Ubiquitous language ​

The terms below travel verbatim across the Kubernetes-side hooks.PlexdHookSpec projection, the tenancy.PlexdHook value object, the plexd_hooks array on the PUT /v1/nodes/{id}/capabilities envelope, and the plexsphere.node_capability_manifest.plexd_hooks column. Internal code never paraphrases them; documentation and Problem detail strings adopt the exact spelling below.

TermDefinitionCode anchor
PlexdHook (custom resource)The upstream Kubernetes custom resource a cluster-hosted plexd agent watches in its customer cluster (group/version/kind hooks.plexd.io / v1alpha1 / PlexdHook). plexsphere reads objects of this kind through the discovery projection and never writes them back. The single PlexdHookGVK declaration is shared by the parser, the CRD test fixture, and the envtest acceptance object so the projected kind cannot drift.internal/policy/hooks/plexdhook.go (PlexdHookGVK)
PlexdHookSpecThe discovery-only projection of one custom resource: the exact five-field set plexsphere persists (name, image_digest, parameters, timeout_seconds, sandbox). ParsePlexdHook fails closed — a missing required field or a wrong-typed field is rejected with an error wrapping ErrInvalidPlexdHook rather than yielding a half-formed spec. The json tags are byte-equivalent to the persisted shape.internal/policy/hooks/plexdhook.go (PlexdHookSpec, ParsePlexdHook)
PlexdHook (value object)The application-layer value object carried inside a CapabilityManifest. Distinct from DeclaredHook (the declared script hook): a PlexdHook carries an OCI image digest, a free-form parameter map, an execution timeout (modelled as a time.Duration, non-negative), and a sandbox flag rather than a payload checksum. The constructor enforces every per-entry invariant.internal/identity/tenancy/capability_manifest.go (PlexdHook, NewPlexdHook)
plexd_hooks (manifest field)The list of discovered PlexdHooks on a CapabilityManifest, independent of declared_hooks. Capped at CapabilitiesMaxPlexdHooks (128), no duplicate names (case-sensitive). It is one of the fields DiffManifests compares; a change names plexd_hooks in the diff's FieldsChanged.internal/identity/tenancy/capability_manifest.go (CapabilityManifest, plexdHooksEqual)
NodeCapabilitiesUpdatedThe existing capability-manifest outbox event a non-empty diff emits. The discovered-hook surface reuses this event verbatim — a discovered-hook change surfaces as plexd_hooks inside the event's FieldsChanged list; there is no PlexdHook-specific event type.internal/identity/tenancy/events/events.go (NodeCapabilitiesUpdated)

The translation is one-directional: plexd observes a PlexdHook object, the agent advertises its projection on the next capability PUT, the handler canonicalises it through the value object, and the repository folds a change into the plexd_hooks arm of the existing diff. plexsphere never writes a PlexdHook object back to the cluster.

Kubernetes-boundary projection ​

internal/policy/hooks is the only package in the policy bounded context aware of the Kubernetes client libraries. It imports k8s.io/apimachinery's unstructured accessors to read a custom resource without binding to a generated client-go type; every other policy package stays Kubernetes-free. This mirrors how the broker context confines controller-runtime and the k8s.io libraries to its render and reconcile subpackages.

ParsePlexdHook (internal/policy/hooks/plexdhook.go) projects one *unstructured.Unstructured onto a PlexdHookSpec, failing closed on a malformed object. The field-path mapping mirrors the custom resource's openAPIV3Schema:

CRD field pathSpec fieldRule
metadata.nameNameRequired; non-empty after trimming whitespace.
spec.imageDigestImageDigestRequired; present and non-empty (the value object enforces the canonical digest shape downstream).
spec.parametersParametersOptional; absent yields nil.
spec.timeoutSecondsTimeoutSecondsOptional; absent yields 0. Read as an int64 — the CRD schema types the field integer / format: int64 so the apiserver decodes it as an integer rather than a float.
spec.sandboxSandboxOptional; absent yields false.

Every extraction uses the unstructured (value, found, err) triplet: a non-nil err means the field is present but wrong-typed and the object is rejected; a !found on a required field is rejected; a !found on an optional field yields the zero value. All rejections wrap ErrInvalidPlexdHook. The function never resolves the image digest against a registry and never mutates the object.

PlexdHookGVK is the single GroupVersionKind declaration the parser, the CRD fixture (internal/policy/hooks/testdata/crds/plexdhook.yaml), and the envtest acceptance object all share, so the projected kind cannot drift between them. The DECISION block on PlexdHookGVK pins why the group/version/kind are declared here rather than imported from plexd's published CRD Go types: vendoring plexd's apis module would pull a transitive dependency on its controller scaffolding for a read-only five-field projection, so the GVK is pinned locally and the conformance test pins the wire shape instead.

Wire shape ​

Discovered PlexdHooks travel on the existing capability-manifest operation: PUT /v1/nodes/{id}/capabilities (operationId PutNodeCapabilities). The envelope, the NSK authentication model, the 32 KiB body cap, and the single-transaction ingest are documented in the capability manifest ingest reference; this section covers only the plexd_hooks addition.

Request ​

The plexd_hooks array is an optional sibling of declared_hooks on the CapabilityManifestRequest envelope. The handler decodes it into PlexdHookRequest entries with DisallowUnknownFields, then canonicalises the whole envelope through tenancy.NewCapabilityManifest.

jsonc
{
  "binary_version":  "plexd-v0.4.2-ge5f3a1c",
  "binary_checksum": "<base64 of 32-byte SHA-256>",
  "declared_hooks":  [ /* see capabilities.md */ ],
  "plexd_hooks": [
    {
      "name":            "nightly-backup",
      "image_digest":    "sha256:<64 lowercase hex>",
      "parameters":      { "retention": "7d" },  // optional
      "timeout_seconds": 30,                      // optional, whole seconds
      "sandbox":         true                      // optional
    }
  ]
}

Field rules:

  • name (string, required) — the discovered hook's name. Empty and whitespace-only values are rejected with 422 plexd_hook_invalid.
  • image_digest (string, required) — the OCI image digest in canonical sha256:<64 lowercase hex> form. Validation is format-only — plexsphere never resolves the digest against a registry. A value not matching the pattern is rejected with 422 plexd_hook_invalid.
  • parameters (object, optional) — a free-form string-to-string map copied verbatim from the discovered spec. Absent yields no parameters.
  • timeout_seconds (integer, optional) — the execution timeout in whole seconds. Non-negative; a negative value is rejected with 422 plexd_hook_invalid. The domain owns this as a time.Duration — the _seconds suffix is the wire encoding only, and the handler converts seconds to a time.Duration at the transport boundary.
  • sandbox (boolean, optional) — whether the discovered hook requests sandboxed execution. Absent is treated as false.

The list as a whole rejects duplicate name values with 422 plexd_hook_duplicate and more than CapabilitiesMaxPlexdHooks (128) entries with 422 plexd_hooks_too_many.

Response ​

The plexd_hooks list participates in the existing CapabilityManifestResponse: a change to the discovered-hook list adds the literal plexd_hooks to the alphabetised fields_changed array. There is no PlexdHook-specific field on the response envelope.

jsonc
{
  "accepted_at":      "2026-04-27T10:15:30.123Z",
  "fields_changed":   ["binary_version", "plexd_hooks"],
  "host_key_changed": false
}

The valid fields_changed enum gains plexd_hooks as a fifth value alongside the four documented in the capabilities reference:

  • binary_version
  • binary_checksum
  • ssh_host_key_fingerprint
  • declared_hooks
  • plexd_hooks

host_key_changed is unaffected by discovered hooks — a discovered PlexdHook change is never security-load-bearing in the SSH-host-key sense, so it flows only through the fields_changed list.

Error code catalog ​

The discovered-hook invariants surface as 422 Unprocessable Entity, deliberately distinct from the 400 Bad Request arms the declared-hook invariants use. The envelope is structurally well-formed (it decoded, and DisallowUnknownFields passed), but a discovered-hook value is semantically invalid — so the surface returns 422 rather than 400. Generated TypeScript / Go clients can exhaustively switch on code without a fall-through arm.

HTTP statuscode literalTrigger
422plexd_hook_invalidA plexd_hooks entry violates a per-entry invariant — empty name, an image_digest that is not canonical sha256:<64 lowercase hex>, or a negative timeout_seconds (tenancy.ErrCapabilityManifestPlexdHookInvalid).
422plexd_hook_duplicateTwo plexd_hooks entries carry the same name (tenancy.ErrCapabilityManifestPlexdHookDuplicate).
422plexd_hooks_too_manyplexd_hooks exceeds CapabilitiesMaxPlexdHooks (128) (tenancy.ErrCapabilityManifestPlexdHooksTooMany).

The handler maps these from both the invariant path (handleCapabilitiesInvariantError, the canonicalisation gate) and the recorder path (handleCapabilitiesRecordError, a defensive re-validation arm) so a downstream re-check does not collapse onto the catch-all 500 arm. Every other Problem code on the operation — the 400 envelope/declared-hook arms, 401 unauthorized, 403 node_id_mismatch, 404 capabilities_node_not_found, 413 capabilities_body_too_large, 501 capabilities_not_provisioned — is shared with the capability ingest surface and is catalogued in the capabilities reference.

Like the rest of the surface, the discovered-hook rejection arms emit an audit row through the shared AuditSink stamped with the node_capabilities.record relation; the defense-in-depth path-id gate stamps node_capabilities.path_gate.

Value-object invariants ​

tenancy.NewPlexdHook (internal/identity/tenancy/capability_manifest.go) is the single seam that enforces every per-hook invariant; the parent tenancy.NewCapabilityManifest enforces the list-level rules. The invariants are:

  • Name is non-empty after trimming whitespace.
  • ImageDigest matches the canonical sha256:<64 lowercase hex> form — format only, no registry lookup. The DECISION block on imageDigestPattern pins why only the sha256 algorithm is accepted rather than the general OCI <algorithm>:<hex> grammar: plexd projects every discovered image to a sha256 digest today, the persisted shape and the later integrity-gating concern assume sha256, and a narrow pattern fails closed on an unexpected algorithm at the value-object boundary.
  • Timeout is non-negative. The DECISION block on PlexdHook pins why the timeout is a time.Duration rather than the raw integer seconds carried on the wire: the domain concept is a duration, and modelling it as a time.Duration lets the aggregate enforce the >= 0 invariant once while keeping the seconds-to-Duration conversion a transport concern next to the timeout_seconds field name.

The list-level invariants live on NewCapabilityManifest: no duplicate name values (case-sensitive) and at most CapabilitiesMaxPlexdHooks (128) entries. The CapabilitiesMaxPlexdHooks cap mirrors CapabilitiesMaxDeclaredHooks — discovered hooks are a separate collection, but the same unbounded-growth protection applies and a shared limit keeps the two collections symmetric for operators.

The PlexdHook value object exposes defensive-copy accessors (Name, ImageDigest, Parameters, Timeout, Sandbox) and an IsZero predicate; Parameters returns a clone so a caller cannot reach into the value object's private map and mutate the invariant.

Diff & event contract ​

A change to the discovered-hook list folds into the existing capability-manifest diff and outbox event — there is no PlexdHook-specific event type. DiffManifests (internal/identity/tenancy/capability_manifest.go) compares the prior persisted manifest against the freshly-PUT one and names plexd_hooks in FieldsChanged when the discovered-hook list moved. The repository then appends one NodeCapabilitiesUpdated event carrying that FieldsChanged list — the same single-transaction SELECT + diff + UPSERT + outbox-append contract the capability ingest surface already runs (see the diff & event contract in the capabilities reference).

Set-keyed comparison ​

plexdHooksEqual treats the discovered-hook list as a set keyed by name: a same-named pair is equal only when its full content matches — the image digest, the parameter map (compared with maps.Equal), the timeout, and the sandbox flag. A re-order does NOT register as a diff. The DECISION block on DiffManifests pins the rationale: the ordering plexd reports for discovered hooks is not load-bearing, so a re-order must not emit a spurious "agent capabilities changed" signal that the dashboard projector would render as an alert. Two empty lists — including nil versus nil — compare equal, so a manifest carrying no discovered hooks never registers a spurious plexd_hooks diff.

Reusing the existing diff and event contract (no new event type) is itself a DECISION: it lets the later read-API, dispatch-gating, and managed-push stories consume the discovery signal for free off the same event the rest of the manifest already emits.

Persistence ​

The discovered-hook list is stored as a jsonb column on the existing per-Node manifest row. Migration internal/platform/db/migrations/0038_plexd_hooks.sqlALTERs the table plexsphere.node_capability_manifest (created by 0035_node_capabilities.sql) to add one column:

ColumnTypeNotes
plexd_hooksjsonb NOT NULL DEFAULT '[]'::jsonbThe discovered Kubernetes PlexdHook list the agent advertised on its most recent PUT, stored as a jsonb array of objects whose shape matches the wire shape. DEFAULT '[]'::jsonb keeps the "no discovered hooks" case round-trip-safe and backfills cleanly onto existing manifest rows. The domain value object is the authoritative parser; the SQL layer treats the blob as opaque.

The ADD COLUMN uses IF NOT EXISTS and the Down arm DROPs the column with IF EXISTS, so a repeat Up stays idempotent.

Column-versus-sibling-table decision ​

The migration's DECISION block pins why the discovered-hook list is an ADD COLUMN plexd_hooks jsonb on the existing manifest row rather than a sibling node_plexd_hooks(node_id, name, …) table. Every PUT overwrites the whole discovered-hook list; the list is small (capped at 128) and is never the subject of a relational join, so a sibling table would force a DELETE+INSERT-per-hook flow inside the UPSERT transaction for no observable gain. The column mirrors the declared_hooks jsonb decision already recorded on 0035_node_capabilities.sql, keeping the two advertised lists co-located under one manifest row.

Down policy ​

The Down arm DROPs the column rather than RAISE-ing on the downgrade. The DECISION block pins the rationale: the discovered-hook list is per-Node current state the agent re-advertises on every PUT — the next capability handoff after a downgrade-and-reup cycle hands the full list back unprompted, so dropping the column does NOT lose retention-critical material. The NodeCapabilitiesUpdated event chain lives in plexsphere.outbox_events, anchored to its own aggregate identifier, and survives the DROP. This DROP-on-Down posture matches 0035_node_capabilities.sql rather than the retention-bearing tables.

Persisted JSON shape ​

The repository serialises each discovered hook through the plexdHookJSON shape in internal/identity/tenancy/repo/node_capability_repo.go: {"name", "image_digest", "parameters", "timeout_seconds", "sandbox"}, every field always present (no omitempty), mirroring declaredHookJSON. The DECISION block pins why timeout_seconds is a plain integer rather than a Go duration string — it matches the wire field name and the value object's seconds-to-Duration boundary, so the application layer marshals once on write and scans once on read without a custom duration codec. An empty list marshals to [] (not null) so the column's NOT NULL DEFAULT '[]' invariant holds.

Hook-integrity gating ​

The advertised-hook catalog is not only persisted and diffed — it is the trust anchor a hook-integrity gate consults before the Action Orchestrator dispatches a hook-based action to a Node. The gate pins a trust-on-first-use (TOFU) known-good digest the first time the control plane observes each advertised hook — across BOTH the declared script hooks (declared_hooks, keyed script_hook) and the discovered PlexdHooks (plexd_hooks, keyed plexd_hook) — raises an integrity alert when a later capability PUT advertises a divergent digest for either, and refuses an action dispatch whose target hook is not byte-for-byte the pinned known-good. The hook kind is part of the key so the same hook name under the two advertisement channels pins two independent baselines.

The baseline table ​

Migration internal/platform/db/migrations/0049_node_hook_baseline.sql creates plexsphere.node_hook_baseline, one row per advertised hook on one Node:

ColumnTypeNotes
node_iduuidFOREIGN KEY to plexsphere.nodes(id) ON DELETE CASCADE — deleting a Node atomically clears its pinned baselines.
hook_nametextThe advertised hook's name.
hook_kindtextThe closed discriminator {script_hook, plexd_hook} (CHECK-constrained) — part of the composite key so a declared hook and a discovered hook of the same name pin independently.
known_good_digesttext NOT NULLThe opaque TOFU digest: base64(checksum) for a script_hook, the sha256:<hex> image digest for a plexd_hook. The SQL layer treats it as an opaque token; the application layer is the authoritative producer and comparer.
first_seen_attimestamptzThe instant the baseline was pinned.

The composite PRIMARY KEY (node_id, hook_name, hook_kind) is the dispatch gate's lookup key and the conflict target the TOFU pin writes against. The baseline is write-once per key — the pin never advances an existing row, so a drift is always measured against the first-seen digest. The single-table composite-key shape mirrors the per-Node current-state posture of plexsphere.node_capability_manifest; the migration's DECISION blocks record the rejected surrogate-key and per-kind-sibling-table alternatives, and the DROP-on-Down posture (the baseline is reconstructible from the next capability PUT, so dropping it loses no retention-critical material — the alert evidence chain lives in plexsphere.outbox_events).

Trust-on-first-use pin at capability ingest ​

The pin runs inside the same single transaction as the capability ingest. RecordCapabilities (internal/identity/tenancy/repo/node_capability_repo.go) calls pinHookBaselines (internal/identity/tenancy/repo/node_hook_baseline_repo.go) after it UPSERTs the manifest, reconciling every advertised hook against its pinned baseline:

  • For each declared hook the known-good is base64(checksum); for each discovered PlexdHook it is the OCI image_digest.
  • It reads the pinned baseline. Absent → it pins the first-seen digest with an ON CONFLICT (node_id, hook_name, hook_kind) DO NOTHING insert, so a concurrent first-pin from another replica is a silent no-op rather than a unique-constraint violation.
  • Present → the baseline is NEVER advanced. A live digest that diverges from the pinned known-good is collected as a drifted hook the caller surfaces; the stored baseline stays at the first-seen value.

Drift raises an integrity alert ​

When pinHookBaselines returns one or more drifted hooks, RecordCapabilities appends — inside the same transaction, independently of whether the manifest diff was empty — one IntegrityAlert outbox event built by tenancy.events.NewIntegrityAlert (internal/identity/tenancy/events/events.go). The alert carries the count of drifted hooks, the violation kinds ["hook_checksum"], and the recommended action quarantine_node, and fans out per-Domain through the denormalised (Node, Resource, Project, Domain) quartet exactly as the agent-reported integrity surface does (see integrity.md for the alert's payload shape and per-Domain fan-out). A subsequent PUT that re-advertises the pinned known-good digest raises no alert — the hook has self-healed back to its baseline. Because the baseline is never advanced to the drifted value, the comparison is always against the first-seen digest, so a PUT whose digest still diverges raises the alert again until the hook returns to its pinned digest.

Dispatch verdict ​

At dispatch time the gate reduces the Node's live hook digest, the pinned known-good, and whether a baseline exists into a three-way verdict via tenancy.ClassifyHookIntegrity (internal/identity/tenancy/hook_baseline.go):

VerdictConditionDispatch
VerifiedA baseline is pinned and the live digest matches it byte-for-byte.Admitted — the ONLY admitting verdict.
DriftA baseline is pinned but the live digest diverges from it.Refused — the hook changed since first observation.
UncataloguedNo baseline is pinned for the hook.Refused — the control plane has never vouched for it (fail-closed).

The classification is total and fail-closed: a missing baseline is Uncatalogued (not Verified), and only a byte-for-byte match admits. The DECISION block on ClassifyHookIntegrity records why the baseline is per-Node TOFU rather than a per-Domain operator-curated catalogue, and why an uncatalogued hook is refused rather than admitted.

The Action Orchestrator consults the verdict through its CapabilityReader.HookIntegrity port (internal/actions/ports.go); the composition-root adapter (cmd/plexsphere/actions_factory_prod.go) reads the named action's live checksum from the manifest's declared_hooks, fetches the script_hook baseline through NodeHookBaselineRepo.GetHookBaseline, runs ClassifyHookIntegrity, and translates the status into the actions-local verdict — a missing manifest, an undeclared hook, or no pinned baseline all map to Uncatalogued (fail-closed, never a 5xx); only corruption of the control plane's own persisted state surfaces as a 5xx. A builtin action ships with the Node agent and carries no operator-declared hook, so it skips the gate entirely.

The 409 contract ​

The gate sits in the dispatch service (internal/actions/services/dispatch.go) as hookIntegrityGate, and the two dispatch paths treat a non-Verified verdict differently:

  • DispatchByNode (a single explicit Node) is a hard refusal: a Drift or Uncatalogued hook returns ErrHookIntegrityViolation and persists nothing.
  • DispatchBySelector (a label-selected cohort) drops each failing Node and stamps one denial audit row with the hook_integrity_violation relation. In the no-survivor precedence an integrity failure outranks action_not_declared and an empty cohort, so an operator whose whole cohort tripped the gate sees the integrity refusal rather than a capability or selector error.

Both surface at the transport boundary as 409 Conflict with code: hook_integrity_violation on the DispatchExecution operation (POST /v1/projects/{project_id}/executions:dispatch) — the live hook conflicts with the control plane's pinned record of it. The mapping is pinned in internal/transport/http/v1/actions/errors.go and documented on the operation in ../../../api/openapi/plexsphere-v1.yaml.

Out of scope ​

The discovery surface itself deliberately does NOT write, does NOT resolve, and does NOT schedule. Of the concerns below, managed push has since shipped as its own opt-in surface (linked from its bullet); the operator-facing read surface is still owned by a named follow-up story. This slice is the producer side of the data their consumer arms read.

  • Managed push (in-cluster writes) — now shipped; see the managed-push context doc. It owns the opt-in, per-Domain capability for plexsphere to write PlexdHook CRDs back into a customer cluster, turning the catalogue into a publish target. The discovery surface remains the read-only default: plexsphere records what plexd observed and never holds kubectl credentials unless a Domain opts in. The managed-push doc covers the sealed credential, the Server-Side-Apply write path, and the GitOps-versus-managed authority model.
  • Operator-facing read surface — owns the dashboard view of "what discovered hooks is each Node advertising right now?". A read API and the dashboard projection live in that follow-up; the discovery surface is intentionally write-only on the agent path and consumes no read of its own. The dashboard projector reads the same NodeCapabilitiesUpdated outbox event the rest of the manifest emits.

Testing ​

The four-aspect coverage for this surface is held across the test pyramid:

Hook-integrity gating coverage ​

The gating surface layered on top of the catalog carries its own four-aspect coverage:

Cross-references ​