Telemetry Sinks HTTP API

This is the reference for the sinks OpenAPI tag. It maps each of the fifteen operations to its ReBAC gate, audit relation, outbox event, and the closed Problem.code taxonomy. The wire-contract origin is api/openapi/plexsphere-v1.yaml; this doc is a map, not a duplicate contract.

The tag covers three aggregates that compose into one operator flow. A Sink is a named telemetry destination. A Domain declares its own otlp and syslog destinations, and the platform seeds its three backends (mimir, loki, siem) as built-in sinks that belong to no Domain. A sink enablement is the grant that makes one Domain's tenant sink usable inside one Project of that Domain. A Telemetry Route states where one Project sends one signal, naming the sinks it may reach. A route may target a built-in sink without any grant.

The bounded-context narratives behind the three aggregates are ../../contexts/observability/sinks.md, ../../contexts/observability/sink-enablement.md, and ../../contexts/observability/routes.md. The matching CLI families are ../cli/plexctl/sink.md and ../cli/plexctl/route.md.

Operations

Method	Path	Operation ID	ReBAC gate	Notes
POST	/v1/domains/{id}/sinks	CreateSink	domain#manage on the path Domain	Declares a tenant sink. A created sink derives its own manage from parent->manage, so the Domain check is the authority the sink answers to afterwards. id is optional; a caller storing connection material mints it first.
GET	/v1/domains/{id}/sinks	ListDomainSinks	domain#read on the path Domain	The Domain's own tenant sinks, ordered by slug. Every row belongs to the one path Domain, so the single gate authorises the whole list and no per-row filter runs. Not paginated.
GET	/v1/sinks/built-in	ListBuiltInSinks	authenticated principal only	The platform's own backends. They carry no authorization tuples, so there is no object to check a relation against, and they are legitimate route targets for every Project. Not paginated.
GET	/v1/sinks/{id}	GetSink	sink#observe; a built-in row answers on the authenticated principal alone	The row is read before the gate, because the path id alone does not say which of the two shapes it addresses. observe resolves for the sink's owner, an assigner, a Domain manager, and a reader of a Project holding a live grant.
PUT	/v1/sinks/{id}	UpdateSink	sink#manage	Full post-image: a field absent from the body is cleared. expected_updated_at is required and guards the write; a stale one answers 409 sink_cas_conflict. A built-in row answers 409 sink_conflict before the gate runs.
DELETE	/v1/sinks/{id}	DeleteSink	sink#manage	A built-in row answers 409 sink_conflict before the gate runs.
POST	/v1/projects/{id}/sink-enablements	RequestSinkEnablement	project#deploy, checked in the application service	Opens the enablement in requested. A request grants nothing; the approvals queue decides it.
GET	/v1/projects/{id}/sink-enablements	ListSinkEnablements	project#read, checked in the application service	Cursor-paginated page of the Project's enablements in every lifecycle state, in creation order.
POST	/v1/sinks/{id}/sink-enablements	GrantSinkEnablement	sink#assign, checked in the application service	The owner push. The enablement lands already approved and the uses edge is written. May answer 201 with sync_pending: true.
POST	/v1/sink-enablements/{id}/revoke	RevokeSinkEnablement	sink#assign, and on its denial project#deploy, both checked in the application service	Legal from approved only; any other source state answers 409 illegal_transition. May answer 502 revocation_sync_pending.
POST	/v1/projects/{id}/telemetry-routes	CreateTelemetryRoute	project#deploy on the path Project	One signal, at most one predicate, between one and sixteen targets. Every target is checked at authoring time.
GET	/v1/projects/{id}/telemetry-routes	ListTelemetryRoutes	project#read on the path Project	Creation-ordered. A Project holds a bounded number of routes, so the listing is not paginated.
GET	/v1/telemetry-routes/{id}	GetTelemetryRoute	project#read on the Project the STORED route belongs to	The route id does not encode its Project, so the row is read before the gate.
PUT	/v1/telemetry-routes/{id}	UpdateTelemetryRoute	project#deploy on the Project the STORED route belongs to	Full post-image. The stored project_id is immutable; a body naming another Project is refused with 422 route_project_mismatch.
DELETE	/v1/telemetry-routes/{id}	DeleteTelemetryRoute	project#deploy on the Project the STORED route belongs to	Deleting the last route for a signal does not stop delivery: the signal falls back to the platform's own backends.

• Body caps. CreateSink and UpdateSink cap the request body at 256 KiB (MaxSinkRequestBodyBytes in internal/transport/http/v1/sinks/wiring.go), because a sink may carry a PEM CA bundle. The enablement and route bodies cap at 8 KiB (MaxSinkEnablementRequestBodyBytes, MaxTelemetryRouteRequestBodyBytes). An over-cap body surfaces as 413 request_body_too_large.
• Count ceilings. Three bounds keep the unpaginated reads and the delivery fan-out finite, and each is refused at the write that would exceed it. A Domain holds at most MaxSinksPerDomain tenant sinks (409 sink_limit_reached), a Project at most MaxRoutesPerProject routes (409 route_limit_reached), and one route names at most sixteen targets (422 telemetry_route_invalid). The last two multiply: the cost of a buffered batch is routes x targets x records, so bounding only the route count would leave the fan-out open.
• Audit relations. The sink operations record under sinks.create, sinks.update, sinks.delete, sinks.list and sinks.get; the enablement operations under sinkenablement.request, .grant, .revoke and .list, with .approve and .reject recorded by the decisions taken on the queue; the route operations under routes.create, routes.update, routes.delete, routes.list and routes.get. Every row is names-only: it carries the field names the write is about and no value of any of them, so no endpoint address and no credential coordinate reaches the audit chain.
• Outbox events. SinkCreated, SinkUpdated and SinkDeleted for the sink; SinkEnablementRequested, Granted, Approved, Rejected and Revoked for the grant; RouteCreated, RouteUpdated and RouteDeleted for the route. Of these only SinkCreated, SinkDeleted, SinkEnablementGranted, SinkEnablementApproved and SinkEnablementRevoked change the authorization graph. The built-in seed writes no event, which is why built-in rows carry no tuples.
• Compare-and-swap on the sink update. UpdateSink takes a required expected_updated_at — the updated_at the client read the sink at — and applies the write only if the stored row still carries it. The body is a full post-image, so without the precondition two admins editing from the same listing resolve last-writer-wins and the loser's body silently restores the three security-relevant fields it never touched: tls.insecure_skip_verify, tls.ca_pem and credential. The audit row would read as an ordinary sinks.update. A mismatch answers 409 sink_cas_conflict and writes nothing. This mirrors expected_version on POST /v1/credentials/{id}/rotate; the sink compares a timestamp rather than a counter because the sinks table carries no version column and updated_at is already projected on every read.
• Read ordering. GetSink, UpdateSink, DeleteSink, GetTelemetryRoute, UpdateTelemetryRoute and DeleteTelemetryRoute read the row before the ReBAC check, because the path id does not carry the object the gate needs (the sink's shape, or the route's Project). The narrow existence side channel that ordering opens is accepted: the denial goes out through the audit-first path, so the probe is recorded, and a UUIDv7 id is not guessable.

Credential references

A sink states where its connection material lives, never the material itself. SinkCreateRequest.credential and SinkUpdateRequest.credential carry the KV mount and, optionally, the KV version to read. There is no path field and no material field on the wire.

The path is derived server-side from the owning Domain and the sink id:

domains/<domain-id>/sinks/<sink-id>

No caller can point a sink at material belonging to another tenant, because no caller states the path. The projection every read returns carries the coordinates (mount, version, derived path) and never the material stored at them.

That derivation fixes the authoring order. Write the secret first, then declare the sink under the same id:

1. Mint the sink id client-side (uuidgen, or any canonical UUID; the platform mints UUIDv7 so creation order sorts).
2. Write the material at the derived path, for example bao kv put secret/domains/<domain-id>/sinks/<sink-id> token='<token>'.
3. POST /v1/domains/{id}/sinks with that id and a credential naming the mount.

Omitting id lets the server mint one, which is what a sink needing no credential does. Rotating the material is a write at the same derived path: the token is read on every export and never cached, so the next batch carries the new value and the sink row does not change. A sink that pins kv_version keeps reading the pinned version, so a rotation reaches it only once that field is updated.

Grant synchronisation

The enablement lifecycle writes a row and a sink:<id>#uses@project:<id> tuple. The two can diverge for one request, and the two directions report the divergence differently, because their risk is not symmetric.

A grant reports success with a flag. GrantSinkEnablement may answer 201 with sync_pending: true. The transition COMMITTED: the row moved and its event is durable. What has not completed is the arm that mirrors it into the authorization graph, so the grant is not yet effective. The caller must NOT retry. A retry is refused by the live-unique index and would report a conflict for work that already happened. The committed event drives the same mutation through the authz-sync arm.

An approve taken on the approvals queue commits under the same fail-safe rule and also answers success, but the Approval projection carries no sync_pending field, so the pending sync is not distinguishable there. Read the enablement back through ListSinkEnablements when the caller needs the effective state rather than the committed one.

A revocation reports failure with a status.RevokeSinkEnablement answers 502 revocation_sync_pending when the row moved to revoked and its event is durable but the edge delete did not complete. The Project still holds the grant and keeps delivering, which is a fail-open state and therefore reported as a failure rather than as a completed revocation. Retrying is refused (revoked is terminal); the removal is left to the committed event replaying through the authz-sync arm.

Built-in sinks

The platform seeds mimir, loki and siem at boot. A built-in sink carries no Domain, no endpoint, no credential and no TLS posture, and is addressed by the slug that equals its type. Three consequences show on this surface:

• ListBuiltInSinks and a GetSink addressing a built-in row gate on the authenticated principal alone. The rows carry no tuples, so there is no object to check a relation against, and the roster discloses no tenant state.
• UpdateSink and DeleteSink answer 409 sink_conflict before their gate runs. A manage check against a built-in row would deny for want of tuples, telling the caller it lacks a permission when the truth is that the row is an immutable platform fixture no grant could make writable. The 409 is identical for a Domain admin and for a stranger, so the answer leaks nothing about who holds what.
• A Telemetry Route reaches a built-in sink with no enablement. Naming one in RequestSinkEnablement or GrantSinkEnablement is refused with 422 sink_not_enableable.

Deciding an enablement request

RequestSinkEnablement opens the grant; it does not settle it. The decision is taken on the dual-control queue, where the enablement surfaces as an Approval row of kind sink_enablement: POST /v1/approvals/{id}/approve and POST /v1/approvals/{id}/reject dispatch onto the same application service this surface calls. Listing and deciding a sink_enablement row both gate on sink#assign over the sink the row addresses, so the kind does not vary its gate by path. See approvals.md for the queue's own contract. On the CLI the equivalent is plexctl approval list --kind sink_enablement, followed by plexctl approval approve or plexctl approval reject; the Console carries the same rows in its approvals view.

The four-eyes rule is enforced by the enablement service: the principal that filed the request may not decide it, and the comparison runs on principal UUIDs, so a re-spelled subject kind does not dodge it. A rejection carries no such guard, because it grants nothing. GrantSinkEnablement bypasses the rule by design and is why it exists: requesting and then approving one's own request is refused, so without the owner push a sink's owner could not place it in a Project at all.

Pagination

ListSinkEnablements is the one paginated operation on this surface. limit is validated against [1, 200] with default 50; a value outside the window is refused with 400 invalid_limit rather than clamped. cursor is opaque, HMAC-signed, and bound to the per-(caller, pepper) pseudonym: a tampered envelope or unknown version byte surfaces as 400 invalid_cursor, and a cursor minted by one principal and replayed by another as 403 cursor_binding_mismatch. The page renders in creation order, and the single top-level project#read gate authorises the whole page because every row belongs to the one path Project.

ListDomainSinks, ListBuiltInSinks and ListTelemetryRoutes return bounded collections and take no cursor.

Error taxonomy

All error responses use the shared Problem envelope (application/problem+json). The 403 path uses the richer PermissionDenied shape carrying the ReBAC denial reason and the request correlation_id.

Code	Status	Where	Meaning
sink_invalid	422	CreateSink / UpdateSink	The Sink aggregate refused the body: a malformed slug, an oversized display name, a type no tenant may declare, an endpoint invalid for the stated type or naming a literal address inside the platform's own network, a CA bundle holding no certificate, or a dataset on a type other than otlp.
sink_not_found	404	GetSink / UpdateSink / DeleteSink / Request / Grant	The addressed sink does not exist, or a body named a sink that does not.
sink_slug_taken	409	CreateSink / UpdateSink	The write would take a slug another sink already holds in the same scope.
sink_conflict	409	UpdateSink / DeleteSink	The write cannot be applied against the stored sink for a reason other than the slug: it would move the sink to another Domain or flip its built-in discriminator, or it addresses a built-in sink, which is neither updatable nor deletable.
sink_cas_conflict	409	UpdateSink	expected_updated_at did not match the stored sink's updated_at.
sink_limit_reached	409	CreateSink	The Domain already holds the maximum number of tenant sinks. The Domain listing is unpaginated and every route picks its targets from the same roster, so the ceiling is what keeps both bounded.
sinks_not_provisioned	501	every sink operation	The composition root has not wired the sink dependency bundle.
sink_enablement_not_found	404	RevokeSinkEnablement	No enablement with the given {id}.
duplicate_live_sink_enablement	409	Request / Grant	A live (requested or approved) enablement already exists for the same (Project, sink) pair.
sink_not_enableable	422	Request / Grant	The named sink is built in. A built-in sink is reachable from a route without a grant, so there is nothing to enable.
sink_domain_mismatch	422	Request / Grant	The sink and the Project belong to different Domains, so the grant would cross a tenancy boundary.
revocation_sync_pending	502	RevokeSinkEnablement	The revocation committed but the uses edge delete did not complete, so the Project still holds the grant. See Grant synchronisation.
sink_enablements_not_provisioned	501	every enablement operation	The composition root has not wired the sink-enablement dependency bundle.
telemetry_route_invalid	422	CreateTelemetryRoute / UpdateTelemetryRoute	The Route aggregate refused the body: an unknown signal, a severity floor on a non-logs route, a name prefix on a non-metrics route, an oversized prefix, an empty sink_ids, more than sixteen sink ids, a zero sink id, or a sink named twice.
telemetry_route_not_found	404	GetTelemetryRoute / UpdateTelemetryRoute / DeleteTelemetryRoute	No Telemetry Route with the given {id}.
route_sink_not_found	422	CreateTelemetryRoute / UpdateTelemetryRoute	A sink_ids entry names no sink.
route_sink_not_usable	422	CreateTelemetryRoute / UpdateTelemetryRoute	A sink_ids entry names a tenant sink the Project holds no live approved enablement for, so the route would deliver nowhere.
route_signal_not_accepted	422	CreateTelemetryRoute / UpdateTelemetryRoute	A target sink's type cannot receive the route's signal, so the route would only ever go dark.
route_project_mismatch	422	UpdateTelemetryRoute	The post-image names a different Project than the stored route belongs to; the persisted project_id is immutable.
route_limit_reached	409	CreateTelemetryRoute	The Project already holds the maximum number of routes. The delivery engine walks a Project's routes for every buffered batch, so the ceiling is what keeps that walk bounded.
telemetry_routes_not_provisioned	501	every route operation	The composition root has not wired the Telemetry Route dependency bundle.

The surface reuses the shared invalid_domain_id, invalid_sink_id, invalid_project_id, invalid_sink_enablement_id, invalid_telemetry_route_id, invalid_body, invalid_limit, invalid_cursor, invalid_decision_reason, domain_not_found, project_not_found, illegal_transition, cursor_binding_mismatch, request_body_too_large, permission_denied, unauthenticated, authz_unavailable, and internal codes with the semantics they carry on the sibling surfaces.

A 403 permission-denied response carries the extended PermissionDenied fields documented in authz.md.

Cross-references

• ../../contexts/observability/sinks.md — the Sink aggregate, its two shapes, the built-in seed, and the sink ReBAC definition with its event-to-tuple mapping.
• ../../contexts/observability/sink-enablement.md — the four-eyes grant lifecycle and the two sync arms.
• ../../contexts/observability/routes.md — the replace-per-signal rule, the two predicates, and the capability table that decides which signals a sink type receives.
• ./approvals.md — the queue that carries the approve and reject decisions for the sink_enablement kind.
• ./observability.md — the ingest front door that admits the telemetry these routes then place, and the operator-facing query, alert-rule and incident surfaces.
• ../cli/plexctl/sink.md and ../cli/plexctl/route.md — the matching CLI families.
• ../../how-to/observability/deliver-telemetry-to-dash0.md and ../../how-to/observability/deliver-logs-to-a-syslog-collector.md — the two operator walkthroughs that drive this surface end to end.
• ./index.md — platform-wide /v1 HTTP surface map and the tag table this surface sits in.
• ../../../api/openapi/plexsphere-v1.yaml — authoritative OpenAPI contract; this doc is a map, not a duplicate.