Deliver logs to a syslog collector

A syslog collector speaks RFC 5424 over TLS, which is the second tenant protocol the platform ships an exporter adapter for. Such a collector is an ordinary tenant sink of type syslog: a Domain declares it, a Project is granted the use of it, and a route sends that Project's logs and audit records there instead of to the platform's own backends.

This guide takes the local dev stack as the running installation. The wire mapping the adapter produces is on the routing page.

What a syslog sink needs

Two values, and no credential:

Value	Where it goes
the collector's host and port	the sink's --endpoint
the CA that signed the collector's certificate	the sink's --ca-pem-file

The endpoint is a host and port pair, not a URL: logs.example.net:6514, with a port between 1 and 65535. The transport is server-authenticated TLS. The collector proves itself with a certificate that chains to the sink's CA bundle, and the platform proves nothing, so a sink stating no bundle verifies against the system trust store and a privately signed certificate needs its CA passed with --ca-pem-file.

A syslog sink takes no credential. A sink that states one is refused when its exporter is built: the client-side convention a collector speaks is mutual TLS, which needs a certificate and a private key rather than the token the credential contract carries. A refused sink is left without an exporter, the server logs the reason, and no consumer is provisioned for it, so the destination receives nothing at all rather than receiving batches anonymously.

The certificate has to name the endpoint's host. The adapter dials raw TCP and wraps the connection itself, so it sets the host the endpoint names as the name the certificate is verified against. A certificate whose subject alternative name does not carry that exact host fails the handshake on every batch.

Before you start

• A running dev stack (make dev). Its overlay sets both PLEXSPHERE_OBS_NATS_URL and PLEXSPHERE_DSN, which is the routing pipeline's opt-in gate, so the pipeline is live whether or not a platform backend is configured.
• No secret store. Unlike the Dash0 path, this sink reads nothing out of OpenBao, so the PLEXSPHERE_SECRETS_OPENBAO_* family may stay unset.
• kubectl pointed at the dev cluster (kind-plexsphere). The dev overlay applies into the default namespace, so no -n flag is needed below.
• Permissions on the three objects the steps touch: manage on the Domain to declare the sink, assign on the sink to grant it, and deploy on the Project to state the route. The seeded admin operator holds all three.

Stand up a collector

The collector has to sit outside the platform's own network. Every connection the adapter opens is re-checked after the name resolves against the address set the Sink aggregate refuses: loopback, unspecified, private, link-local, interface-local, the 100.64.0.0/10 mesh range, 192.0.0.0/24, 198.18.0.0/15 and fec0::/10. A collector on the laptop that runs the dev stack, or one inside the kind cluster, resolves into one of those ranges and the dial is refused. The refusal is retryable, so the batch waits on the stream while retries_total climbs and the server log names the address. Run the collector on a host with a routable address, and point the endpoint at the name that resolves to it.

Learning the flow rather than running it for real? The lesson Route logs to your own collector does exactly this against the dev stack, with a collector on your own machine. It works there because that stack states PLEXSPHERE_OBS_ALLOW_INTERNAL_SINK_DESTINATIONS, which waives the rule above — a dev-stack setting this guide assumes you have not made.

Give it a certificate for the host the endpoint will name. A self-signed one is enough for a first delivery, and it is its own CA:

mkdir -p tls
openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout tls/collector.key -out tls/collector.pem \
  -subj "/CN=<collector-host>" \
  -addext "subjectAltName=DNS:<collector-host>"
cp tls/collector.pem tls/ca.pem

The rsyslog collector image ships a TLS input on port 6514 that reads the octet-counted framing the adapter writes. It is off by default and switched on with environment variables, so there is no configuration file to write:

docker run -d --name syslog-collector -p 6514:6514 \
  -v "$PWD/tls:/etc/rsyslog-tls:ro" \
  -e ENABLE_TLS=on \
  -e ENABLE_TCP=off -e ENABLE_UDP=off -e ENABLE_RELP=off \
  -e WRITE_ALL_FILE=on -e WRITE_JSON_FILE=on \
  -e TLS_AUTH_MODE=anon \
  -e TLS_CA_FILE=/etc/rsyslog-tls/ca.pem \
  -e TLS_CERT_FILE=/etc/rsyslog-tls/collector.pem \
  -e TLS_KEY_FILE=/etc/rsyslog-tls/collector.key \
  rsyslog/rsyslog-collector:2026-04

TLS_AUTH_MODE=anon asks the client for no certificate, which is what the platform's server-authenticated posture needs. The four ENABLE_* switches are all read, so set the three unused inputs to off rather than leaving them unset. The image publishes an amd64 tag only; on an arm64 host add --platform linux/amd64 and expect emulation.

The container writes what it receives to two files: /var/log/all.log carries the timestamp, the hostname, the app name and the message, and /var/log/all-json.log carries the numeric facility and severity of each message, which is where the two streams read apart. Neither of the templates the image ships renders structured data, so the plexsphere@32473 element travels on the wire without appearing in those files.

Read the ids you will substitute

export PATH="$PWD/bin:$PATH"
export PLEXSPHERE_URL=http://localhost:8080

eval "$(make -s dev-ids)"   # exports DOMAIN_ID for the seeded acme-corp Domain
plexctl login --domain-id "$DOMAIN_ID"

plexctl project list --domain "$DOMAIN_ID"

Every <placeholder> below is one of these: the domain id, the project id, and the collector host with its port. The sink, the enablement and the route all take server-minted ids: nothing here needs a client-minted one, because this sink stores no connection material and so derives no KV path.

Declare the sink

--ca-pem-file reads the bundle client-side and puts the PEM bytes in the request body, so the path is the local file the collector's CA sits in:

plexctl sink create \
  --domain-id    "$DOMAIN_ID" \
  --slug         central-syslog \
  --display-name "Central syslog" \
  --type         syslog \
  --endpoint     <collector-host>:6514 \
  --ca-pem-file  tls/ca.pem

Omit --ca-pem-file entirely when the collector's certificate chains to a public CA; verification then runs against the system trust store. State no credential flag: a syslog sink that names a KV mount is left without an exporter, for the reason above. --dataset belongs to otlp and is refused here.

The aggregate checks the body before anything is stored, and the refusal names what it refused. A slug another sink of the Domain already holds fails with 409 sink_slug_taken. 422 sink_invalid covers the rest: a slug that is not kebab-case or is longer than 63 characters, an empty display name, an endpoint that is not a host and port pair, one whose host is a literal address inside the platform's own network, and a CA bundle that decodes to no certificate. The Sink context page has the full set.

Read the id the server minted back off the Domain's roster:

plexctl sink list --domain-id "$DOMAIN_ID"

Grant the sink to the Project

Route resolution narrows a Project's targets to the built-in sinks plus the tenant sinks it holds an approved enablement for. A requested enablement grants nothing.

The sink's owner places the grant in one step:

plexctl sink enablement grant \
  --sink-id    <sink-id> \
  --project-id <project-id>

The enablement lands already approved, so the sink is usable in the Project the moment the command returns. The single step is deliberate: requesting and then approving one's own request is refused by the four-eyes guard, so without this route a sink's owner could not place it at all.

The other direction is the consuming Project asking, and a second principal deciding:

plexctl sink enablement request \
  --project-id <project-id> \
  --sink-id    <sink-id>

plexctl approval list --kind sink_enablement
plexctl approval approve <enablement-id>

Either way, read the result back and confirm the state is approved:

plexctl sink enablement list --project-id <project-id> --all

A grant response carrying sync_pending: true is a committed grant whose authorization edge has not landed yet. Do not retry it: a second grant for the same pair is refused with 409 duplicate_live_sink_enablement while the first is live, and the committed event drives the same edge through the sync arm on its own.

State the route

A route governs exactly one signal, and it replaces the platform's default for that signal. After the command below the Project's logs go to the collector and stop reaching Loki, unless the route names the loki sink too.

plexctl route create \
  --project-id <project-id> \
  --signal     logs \
  --sink-id    <sink-id>

A syslog sink receives logs and audit, so repeat the command with --signal audit for the audit trail. Metrics have no syslog mapping, and the route surface refuses that pairing where it is written: --signal metrics against this sink fails with 422 route_signal_not_accepted rather than being stored as a route that delivers nothing. --severity-floor narrows a logs route to a syslog severity and above; unstated, the route carries every record on its signal.

To keep Loki carrying the same logs alongside the collector, name both targets. Read the built-in id off the roster:

plexctl sink list --built-in

plexctl route create \
  --project-id <project-id> \
  --signal     logs \
  --sink-id    <sink-id> \
  --sink-id    <loki-sink-id>

A built-in sink needs no enablement, so it passes the usability check by construction. On a route that already exists, restate both targets with plexctl route update <route-id>: the body is the full post-image, so a target the command does not name is dropped.

Read the delivery metrics

Two 30-second windows stand between the rows and the first delivery: the consumer lifecycle reconciles the desired durables every 30 seconds, and the route resolver caches a Project's routes for 30 seconds. Give it a minute, then sample the counters. The dev stack serves them unauthenticated:

curl -s http://localhost:8080/metrics | grep '<sink-id>'

The sink label of a tenant destination is the sink id, not the slug, because a slug is unique per Domain only and two Domains naming their destination central-syslog would otherwise share one time series.

Metric	Reads
plexsphere_observability_routing_batches_total	batches driven to a terminal disposition, by signal and outcome (exported, dropped, skipped)
plexsphere_observability_routing_records_total	records exported, by signal
plexsphere_observability_routing_retries_total	retryable outcomes that scheduled a redelivery
plexsphere_observability_routing_resolution_failures_total	deliveries whose route resolution failed
plexsphere_observability_routing_lag_seconds	age of the telemetry at export time

A climbing outcome="exported" is the delivery working, and the messages land in the collector:

docker exec syslog-collector tail -2 /var/log/all.log

2026-08-19T14:00:00.123456Z web-01 nginx disk quota exceeded
2026-08-19T14:00:01Z - identity {"source":"identity","action":"user.login","outcome":"success"}

The audit line names no host and carries its event as raw JSON, and its facility in /var/log/all-json.log reads 13 where a log line reads 16. The ways the delivery goes wrong read differently:

• retries_total climbs and nothing is exported. Every transport failure is retryable here, because syslog over TLS has no status channel to classify: a refused dial, a failed handshake, a broken write. The batch stays on the stream and the server log names the reason. A collector on a private address, an expired certificate and a certificate that does not name the endpoint's host all read this way.
• outcome="skipped" climbs. The routes selected none of this sink's records. A missing route, or an enablement that is not approved, looks like this.
• No series at all for the sink id. No consumer was provisioned, because the exporter could not be built. A sink that states a credential reads this way. The endpoint grammar and the CA bundle are checked earlier, at plexctl sink create, so those two cannot reach this state. The server log names the sink and the reason on every reconcile.
• outcome="dropped" climbs. Nothing in the batch could be encoded: every record was undecodable, or every log record carried a severity keyword outside the roster the ingest front door admits. No retry repairs that, so the batch is acked and gone. The warning line the export writes carries the tally per reason.

Rotate the collector's certificate

The sink carries the CA that signs the collector's certificate, not the certificate itself, so re-issuing the leaf under the same CA needs no change here. Replacing the CA does:

plexctl sink get <sink-id> --output json | jq -r '.updated_at'

plexctl sink update <sink-id> \
  --slug                central-syslog \
  --display-name        "Central syslog" \
  --type                syslog \
  --endpoint            <collector-host>:6514 \
  --ca-pem-file         tls/new-ca.pem \
  --expected-updated-at <updated-at-from-the-read>

The body is the full post-image, so restate every field the sink holds. An update that omits --ca-pem-file drops the stored bundle and falls back to the system trust store, which fails the handshake on every batch for a privately signed collector.

--expected-updated-at is the compare-and-swap token, which is why the read above comes first. If somebody else changed the sink between your read and your write, the update fails with 409 sink_cas_conflict and nothing is written — rather than silently reverting their change, which a full post-image otherwise would.

Remove the destination

Removal runs in reference order: delete the routes that target the sink, revoke the enablement, then delete the sink. A sink a route still targets is not deletable, and the refusal is what keeps a route from silently shrinking to no target.

plexctl route delete <route-id> --yes
plexctl sink enablement revoke <enablement-id> --yes \
  --reason "Collector decommissioned"
plexctl sink delete <sink-id> --yes

A 502 revocation_sync_pending on the revoke is not a completed revocation: the row moved to revoked while the grant is still live in the authorization graph, so the Project keeps delivering. Retrying is refused, and the removal replays through the committed event.

Related

• Syslog mapping: the framing, the facility split, the header projections, the structured-data element and the classification the adapter applies.
• Telemetry Sinks HTTP API: the fifteen operations these commands drive, their ReBAC gates, and the closed Problem.code taxonomy behind every refusal quoted above.
• plexctl sink and plexctl route: the full flag reference for both command families.
• Telemetry sinks: the aggregate the sink hydrates into and every invariant it enforces.
• Sink enablement: the four-eyes grant lifecycle behind the grant and the request paths.
• Telemetry routes: the replace-per-signal rule and the two predicates.
• Deliver telemetry to Dash0: the same flow for an OTLP destination, with the credential the KV store holds.