KeyProvider contract and adapters

This page covers the outbound port every signing back-end implements, the adapters that ship today, and the wiring operators reach for in production. Three adapters carry production key custody: PKCS #11 against a vendor HSM, AWS KMS, and GCP Cloud KMS. The software provider is dev-only and the test double never leaves the test tree. For the rotation lifecycle that sits above this port see Deployment scoping and rotation; for the entry point and ubiquitous language see the index.

The KeyProvider port

KeyProvider is the outbound port every signing back-end implements. Implementations are the only code in the system that ever touches private key material; the plexsphere-signer process speaks to them exclusively through this interface so the "private halves never leave the custodian process" invariant stays enforceable at the type level.

type KeyProvider interface {
    Generate(ctx context.Context, scope Scope, keyID KeyID) (handleID string, publicKey [32]byte, err error)
    Sign(ctx context.Context, handleID string, digest [sha512.Size]byte) (signature []byte, err error)
    PublicKey(ctx context.Context, handleID string) (publicKey [32]byte, err error)
    Retire(ctx context.Context, handleID string) error
}

Every method returns domain sentinels — ErrKeyNotFound, ErrProviderUnavailable, ErrKeyRetired — so application services never couple to PKCS #11 return codes or cloud-provider SDK error types. Retire is idempotent: retiring an already-retired handle must not error. The port's full contract lives in internal/signing/ports.go.

Adapter: software (dev-only)

internal/signing/providers/software/ is a dev-only crypto/ed25519 provider backed by a single 32-byte seed mounted from a Kubernetes Secret. Per-KeyID Ed25519 material is deterministically derived via HKDF-SHA512 keyed by the seed with an info string of scope.String() + "|" + keyID.String(). The constructor emits exactly one slog.Warn line at boot carrying a dev-only substring; the /livez handler surfaces a matching warning flag so operators never learn after the fact that a production cluster was running the software provider.

Beyond the warning, the signer binary refuses to boot with --key-provider=software unless PLEXSPHERE_ENV=dev. Any other value, an unset PLEXSPHERE_ENV included, exits non-zero with a message directing the operator at a PKCS #11 or cloud KMS provider. A misconfigured production rollout therefore fails to start instead of running quietly on the software provider.

Because HKDF is pure, every process holding the same seed derives byte-identical keypairs under the same (scope, KeyID) pair. Sign and PublicKey rely on that property: when the provider sees a handle ID it has not previously Generate()-d — for example a rotation opened by a sibling Pod such as the cmd/signer-e2e-demo fixture's rotate-open job — it parses the (scope, KeyID) back out of the software:<scope>|<key_id> handle layout and re-runs the HKDF expansion on demand. An unparseable handle still surfaces ErrKeyNotFound, which keeps the port-level contract "no such handle" indistinguishable from "wrong provider address" at the call site.

Retire stays strict — an unknown handle returns ErrKeyNotFound even when the layout would parse — because the retired flag on the provider is a LOCAL defence-in-depth check. The authoritative "retired" signal lives on signing_key.state, which SignerService.guardSignState inspects before the provider is consulted; keeping Retire's strict contract lets operator tooling distinguish "never existed" from "already retired" inside one process without conflating the two.

Adapter: PKCS #11 (production HSM)

internal/signing/providers/pkcs11/ is the reference production trust anchor: a thin wrapper over github.com/miekg/pkcs11 targeting the PKCS #11 v3.0 EDDSA mechanism set. It is compiled only under the pkcs11 build tag; a sibling stub file with the inverse !pkcs11 tag keeps the package importable without dragging the CGO binding into the default build. Operator wiring (vendor module paths, token-label and slot addressing, the PIN file, the per-key CKA_LABEL) is documented in the HSM backing section below.

Adapter: cloud KMS (AWS KMS, GCP Cloud KMS)

Two sibling adapters plug the signer into a managed key service, each gated by its own build tag so the default build never pulls the cloud SDK graphs in. Neither creates key material: the operator provisions the key out of band and hands the adapter its one reference. A signer installation serves the single scope --scope pins it to, so a deployment provisions one KMS key per scope.

AWS KMS

internal/signing/providers/kms/aws/ is compiled under the aws_kms build tag. The key spec is ECC_NIST_EDWARDS25519, the one spec that yields an Ed25519 key:

aws kms create-key --key-usage SIGN_VERIFY \
  --key-spec ECC_NIST_EDWARDS25519

Operators hand the resulting ARN to Config.KeyARN. The adapter uses that ARN as the provider handle, because KMS already dereferences it on every call. Generate resolves the configured key and returns its public half rather than calling CreateKey, which keeps the IAM scoping and the quota cost of a new key an operator decision.

Sign asks for pure Ed25519: MessageType=RAW, SigningAlgorithm=ED25519_SHA_512, and the 64 SHA-512 digest bytes from internal/signing/envelope as the message. The signature that comes back verifies with ed25519.Verify(pub, digest[:], sig), the call every verifier in the fleet already makes against the other providers. Ed25519ph (ED25519_PH_SHA_512 over MessageType=DIGEST) hashes the message a second time under a domain-separated prefix, so its signatures never satisfy that call; asking KMS for it would be a wire-format change for every node rather than an adapter detail.

GCP Cloud KMS

internal/signing/providers/kms/gcp/ is compiled under the gcp_kms build tag and signs with an EC_SIGN_ED25519_2019 CryptoKeyVersion. Cloud KMS signs against a version rather than against the logical key, so the adapter takes the CryptoKeyVersion resource name gcloud kms keys versions list prints, projects/<p>/locations/<l>/keyRings/<r>/cryptoKeys/<k>/cryptoKeyVersions/<v>. gcp.ParseKeyVersionName splits that one value into the five Config fields, so the operator copies a single string out of the CLI instead of filling five settings by hand. A name of any other shape is rejected with ErrProviderUnavailable.

Retire disables the key rather than deleting it

Retire disables on both clouds: the AWS adapter calls DisableKey, the GCP adapter sends UpdateCryptoKeyVersion with state DISABLED and state as the update mask. Neither adapter deletes key material. A disabled key refuses every Sign with ErrKeyRetired, and an operator who retired the wrong key re-enables it.

PublicKey keeps answering for a key the adapters disabled, which is what keeps every signature that key produced verifiable. Both clouds refuse the read on a key that is not enabled — AWS with DisabledException, Cloud KMS with FAILED_PRECONDITION — and both adapters map that refusal to ErrProviderUnavailable, never to ErrKeyRetired. SignerService.PublicKey falls back to the public bytes cached on the signing_key row for that sentinel alone, so the public half stays readable. Only Sign reads those same cloud codes as a retirement.

Retire reports success only against a state the key no longer signs in. Neither cloud's refusal code is specific to a retired key: AWS raises KMSInvalidStateException for Creating, PendingImport, Updating and for a key whose custom key store is disconnected, and Cloud KMS raises FAILED_PRECONDITION for PENDING_GENERATION and IMPORT_FAILED. The key stays enabled in every one of those and resumes signing once the state clears. Both adapters therefore read the state back and return success only for a terminal one: Disabled, PendingDeletion or PendingReplicaDeletion on AWS, DISABLED, DESTROYED or DESTROY_SCHEDULED on GCP. Anything else — including a read-back that itself fails — surfaces the original refusal, so RotationService.Close logs it instead of leaving the row marked retired against a key that keeps signing.

The permission set follows from that. The signer's cloud identity needs exactly four rights:

• Sign with the key: kms:Sign on AWS, cloudkms.cryptoKeyVersions.useToSign on GCP.
• Read the public half: kms:GetPublicKey on AWS, cloudkms.cryptoKeyVersions.viewPublicKey on GCP.
• Disable the key: kms:DisableKey on AWS, cloudkms.cryptoKeyVersions.update on GCP.
• Read the key state back: kms:DescribeKey on AWS, cloudkms.cryptoKeyVersions.get on GCP. This is the read-back above; both are metadata reads. Withholding either does not let a wrong retirement through — the read-back fails and Retire surfaces the refusal — but a replayed retire against an already-retired key then reports a failure instead of the intended no-op.

No delete or destroy right belongs on that identity: kms:ScheduleKeyDeletion and cloudkms.cryptoKeyVersions.destroy stay off it. The process that signs with the key then cannot destroy it, and a mistaken retire stays recoverable. Deleting a retired key is an operator decision, taken on the operator's own schedule with an identity that carries the delete permission.

Adapter: testdouble (unit + integration fixture)

internal/signing/providers/testdouble/ is an in-memory, concurrency-safe KeyProvider for unit tests, integration tests, and Chainsaw E2E fixtures. It records every Generate / Sign / PublicKey / Retire invocation on an ordered Call log and exposes InjectFailure / ClearFailure so tests exercise the ErrProviderUnavailable and ErrRotationInProgress paths deterministically. The double is never wired into a running plexsphere binary — no production package imports it.

Selecting a provider on the signer binary

--key-provider picks the adapter. It accepts software, pkcs11, aws-kms and gcp-kms; any other value is rejected at boot with --key-provider must be one of software, pkcs11, aws-kms, gcp-kms. Each provider carries its own settings. Every setting reads its PLEXSPHERE_SIGNER_* variable when the flag is absent, and the flag wins when both are present.

Flag	Environment variable	Provider	Value
--aws-kms-region	PLEXSPHERE_SIGNER_AWS_KMS_REGION	aws-kms	Region the KMS key lives in, for example eu-central-1.
--aws-kms-key-arn	PLEXSPHERE_SIGNER_AWS_KMS_KEY_ARN	aws-kms	ARN of the pre-provisioned Ed25519 key.
--gcp-kms-key-version	PLEXSPHERE_SIGNER_GCP_KMS_KEY_VERSION	gcp-kms	CryptoKeyVersion resource name, projects/<p>/locations/<l>/keyRings/<r>/cryptoKeys/<k>/cryptoKeyVersions/<v>.
--pkcs11-module	PLEXSPHERE_SIGNER_PKCS11_MODULE	pkcs11	Absolute path to the vendor's PKCS #11 shared object.
--pkcs11-token-label	PLEXSPHERE_SIGNER_PKCS11_TOKEN_LABEL	pkcs11	Label of the token whose slot the adapter opens.
--pkcs11-slot	PLEXSPHERE_SIGNER_PKCS11_SLOT	pkcs11	Numeric id of the slot the adapter opens, carried as a string so slot 0 stays expressible.
--pkcs11-pin-file	PLEXSPHERE_SIGNER_PKCS11_PIN_FILE	pkcs11	Path to the file holding the user-level PIN.
--pkcs11-label	PLEXSPHERE_SIGNER_PKCS11_LABEL	pkcs11	CKA_LABEL prefix for plexsphere-owned objects. Defaults to plexsphere.

The PIN only ever comes from a file

There is no --pkcs11-pin flag and no PLEXSPHERE_SIGNER_PKCS11_PIN variable. The signer reads the PIN from the file --pkcs11-pin-file points at, which in production is a mounted Kubernetes Secret. A flag value is readable by every process on the node through ps and is stored verbatim in the Pod spec; an environment variable is stored in the Pod spec too and is readable through kubectl describe pod and /proc/<pid>/environ. A path discloses nothing: the PIN stays in the Secret and is read once, at boot.

The signer trims trailing carriage returns and newlines from the file, so a PIN written with echo works unchanged. Everything else is passed to C_Login byte for byte, spaces included. A file that holds nothing but line breaks is refused with its own error, distinct from the one a missing file produces.

A tagged build per production provider

pkcs11, aws-kms and gcp-kms reach their adapter only in a binary built with the matching tag:

go build -tags pkcs11 ./cmd/plexsphere-signer
go build -tags aws_kms ./cmd/plexsphere-signer
go build -tags gcp_kms ./cmd/plexsphere-signer

Without the tag the binary refuses at boot and prints the command that produces one with the adapter linked. It never falls back to the software provider: a silent swap would turn a production wiring mistake into an invisible dev-mode deployment.

The AWS and GCP adapters are pure Go, so CGO_ENABLED=0 builds both. The PKCS #11 adapter binds the vendor module through cgo and needs a C toolchain. The default build links none of the three, which is what keeps make build working on a machine that has no C compiler and no cloud SDK in its module cache.

The published plexsphere-signer image is the exception: it is built with cgo and with all three tags set, so one image drives a PKCS #11 HSM, AWS KMS and GCP Cloud KMS. It carries no vendor PKCS #11 module of its own. Under keyProvider: pkcs11 the chart's extra* lists bring the module in, or you publish an image that already holds it.

Validation at boot

The binary validates the settings of the selected provider before it opens a listener. A missing setting exits 2 with the flag named on stderr. Settings belonging to another provider are ignored, so a deployment that keeps --aws-kms-region in place while running --key-provider=software still boots.

• aws-kms requires --aws-kms-region and --aws-kms-key-arn.
• gcp-kms requires --gcp-kms-key-version.
• pkcs11 requires --pkcs11-module and --pkcs11-pin-file, plus exactly one of --pkcs11-token-label and --pkcs11-slot. Setting both is rejected, and so is setting neither. A --pkcs11-slot value must be an unsigned 32-bit integer. Slot 0 is a legal address, which is why the setting is carried as a string rather than as a number whose zero value would read as "unset".

HSM backing

The internal/signing/providers/pkcs11/ adapter is a reference implementation against the PKCS #11 v3.0 API, guarded by the pkcs11 build tag.

Vendor slot wiring

The adapter's Config fields map onto the --pkcs11-* flags listed in Selecting a provider on the signer binary:

• ModulePath — absolute path to the vendor's PKCS #11 .so (SoftHSM dev: /usr/lib/softhsm/libsofthsm2.so; AWS CloudHSM: /opt/cloudhsm/lib/libcloudhsm_pkcs11.so; Thales Luna: vendor-provided).
• Slot — the numeric slot the vendor assigns to the plexsphere application. One slot per Domain; SaaS deployments create one slot per Domain at onboarding.
• TokenLabel — the label the token carries in its CK_TOKEN_INFO record. When set it takes precedence over Slot: the adapter walks the token-present slots, compares each label with surrounding spaces trimmed (PKCS #11 pads the label field), and opens the slot whose token matches. A label that matches no token, or that matches more than one, fails startup with ErrProviderUnavailable. SoftHSM assigns a random slot number at softhsm2-util --init-token, so a numeric slot cannot be written into deployment values ahead of time; the label is chosen by whoever initialises the token. The CLI admits exactly one of the two, so the precedence applies only to callers that build the Config directly.
• Pin: the user-level PIN, read from the file --pkcs11-pin-file points at. There is no PIN flag and no PIN environment variable, so the PIN itself never reaches a process listing or a Pod spec.
• Label — the CKA_LABEL prefix under which plexsphere stores keypairs in the slot. The full label format is `<Label>:<scope>:<key_id>` so an operator with pkcs11-tool can grep for all plexsphere-owned objects.

Supported mechanisms

Only modules advertising CKM_EDDSA (PKCS #11 v3.0 §6.25) are supported. Run pkcs11-tool --module <ModulePath> --list-mechanisms and confirm CKM_EDDSA is listed before provisioning the slot.

CI gate

The adapter's tests live behind //go:build pkcs11 && integration_pkcs11, so neither the untagged unit run nor make test-integration compiles them. The integration-pkcs11 job in .github/workflows/ci.yaml is their only execution path, and it runs on every code-touching pull request.

The job invokes one target, which is also the local reproducer and needs nothing on the machine beyond Docker:

make test-integration-pkcs11

The target starts a container, installs SoftHSM, initialises three tokens (one under the label the passing cases open, two sharing a second label so the ambiguous-label refusal has a slot pair to fail against), exports the four variables below, and runs the tagged suite against them.

Against a token you have already provisioned, point the suite at it directly:

PKCS11_MODULE=/usr/lib/softhsm/libsofthsm2.so \
  PKCS11_TOKEN_LABEL=<token label> \
  PKCS11_PIN=<user pin> \
  PKCS11_DUPLICATE_LABEL=<label two tokens share> \
  go test -tags 'pkcs11 integration_pkcs11' \
  ./internal/signing/providers/pkcs11/...

The suite fails, rather than skips, when PKCS11_MODULE is unset: a skip would let the job report green having loaded no module at all.