plexsphere

Appearance

🛡️ adminadmin@example.com · Domain admin in Acme Corp

Chapter 12 — Watch it ​

Query metrics through the control plane ​

bash
plexctl metrics query \
  --domain "$DOMAIN_ID" \
  --query  'vector(1)' \
  --time   "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
text
STATUS_CODE  BODY
200          {"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1782051335,"1"]}]}}

Plain PromQL. You never talk to the metrics backend — the control plane proxies the query and stamps your Domain as the upstream tenant server-side.

There is no query you can write that reads another Domain's series.

Range queries, same path ​

bash
NOW=$(date -u +%Y-%m-%dT%H:%M:%SZ)
START=$(date -u -v-10M +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -d '-10 min' +%Y-%m-%dT%H:%M:%SZ)

plexctl metrics query \
  --domain "$DOMAIN_ID" \
  --query  'vector(1)' \
  --start  "$START" \
  --end    "$NOW" \
  --step   2m \
  --output json | jq -r '.body | fromjson | .data.result[0].values'
text
[
  [1782050843, "1"],
  [1782050963, "1"],
  [1782051083, "1"],
  [1782051203, "1"],
  [1782051443, "1"]
]

vector(1) is synthetic — it proves the round trip without needing ingested series. On a real fleet you would swap in up or rate(node_cpu_seconds_total[5m]). The mechanism is identical.

A different kind of read ​

bash
plexctl metrics capacity --domain "$DOMAIN_ID"

Not PromQL at all. The control plane samples this from its own datastore, and it answers what PromQL cannot: how close is this Domain to the ceilings the platform enforces on it?

text
DIMENSION             USED  TARGET       RATIO   UNIT
nodes                 2     10000        0.0002  count
sse_fanout            0     1000         0       events_per_second

On an idle Domain this returns capacity_snapshot_unavailable — the collector only snapshots dimensions that carry a measurable level. The chapter 11 Node is what makes it answer.

Logs, with LogQL ​

bash
FROM=$(date -u -v-1H +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -d '-1 hour' +%Y-%m-%dT%H:%M:%SZ)
TO=$(date -u +%Y-%m-%dT%H:%M:%SZ)

plexctl logs query \
  --domain "$DOMAIN_ID" \
  --query  '{app="plexd"}' \
  --from   "$FROM" \
  --to     "$TO" \
  --output json \
  | jq -r '.body | fromjson | .data.result[].values[] | "\(.[0][:10] | tonumber | todate)  \(.[1])"'

Explicit --from / --to rather than a relative duration, so the window is never ambiguous on the wire.

This probably prints nothing — a valid 200 with an empty envelope.

Empty is not broken ​

The proxy ran. The Domain scoping applied. The LogQL evaluated. There was simply nothing to match.

Put a line where a Node's pipeline would land it:

bash
kubectl run loki-push --rm -i --quiet --restart=Never \
  --image=curlimages/curl:8.10.1 -- -sS -XPOST http://loki:3100/loki/api/v1/push \
  -H 'Content-Type: application/json' \
  -d "{\"streams\":[{\"stream\":{\"app\":\"plexd\",\"level\":\"error\"},\"values\":[[\"$(date +%s)000000000\",\"demo enrolment failed: bootstrap token expired\"]]}]}"

And read it back ​

bash
plexctl logs query \
  --domain "$DOMAIN_ID" \
  --query  '{app="plexd"}' \
  --from   "$FROM" \
  --to     "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --output json \
  | jq -r '.body | fromjson | .data.result[].values[] | "\(.[0][:10] | tonumber | todate)  \(.[1])"'
text
2026-07-31T15:12:47Z  demo enrolment failed: bootstrap token expired

Through the mediated proxy, scoped to your Domain, audited.

Chapter 14 does this for real — a line typed into a file on a host, collected by the agent, and read back out through this same query.

Store a rule instead of watching by hand ​

bash
ALERT_ID=$(plexctl alert create \
  --domain     "$DOMAIN_ID" \
  --name       high-cpu \
  --signal     'avg(rate(node_cpu_seconds_total[5m]))' \
  --comparator gt \
  --threshold  0.9 \
  --severity   warning \
  --output json | jq -r '.id')
echo "$ALERT_ID"
bash
plexctl alert list --domain "$DOMAIN_ID"
text
NAME      SIGNAL                                 COMPARATOR  THRESHOLD  SEVERITY  ENABLED  ID
high-cpu  avg(rate(node_cpu_seconds_total[5m]))  gt          0.9        warning   true     019eea8b-6dd9-7e4a-bd06-7a988ea37218

--comparator and --severity are validated against their enums before the request leaves your machine.

Updates are patches ​

bash
plexctl alert update --domain "$DOMAIN_ID" --id "$ALERT_ID" --threshold 0.95
text
NAME      SIGNAL                                 COMPARATOR  THRESHOLD  SEVERITY  ENABLED  ID
high-cpu  avg(rate(node_cpu_seconds_total[5m]))  gt          0.95       warning   true     019eea8b-6dd9-7e4a-bd06-7a988ea37218

Only the field you named moved. The signal, comparator, and severity are untouched — no read-modify-write race between two operators editing the same rule.