Monitoring stack
What is collected, where it lands, which alerts exist, and how to read them.
victoriametrics
grafana
zitadelCollection
| signal | component | storage |
|---|---|---|
| metrics | VictoriaMetrics, single node | ceph-block, 30 days |
| logs | Vector, node and pod sources | 7 days |
| traces | services that instrument themselves | best effort |
VictoriaMetrics keeps PromQL and costs less per series than Prometheus at this size, so one store answers cross-signal questions without a second query surface.
Alerts
Every alert here has an action attached. An alert without one gets deleted rather than tuned.
- a certificate expiring inside seven days
- a Ceph pool or a volume above 85 percent
- a Postgres replica lagging more than 30 seconds
- a controller failing to reconcile for more than 15 minutes
- a node NotReady for more than 5 minutes
CPU sitting high is not an alert. A cluster using its capacity is doing its job.
Dashboards
Three, deliberately: cluster (nodes, pods, restarts), storage (pools, OSDs, volume usage), databases (replication, connections, WAL). A dashboard nobody opens is a maintenance bill disguised as observability.
Access
Grafana is served at grafana.kubexa.tech behind the gateway with Zitadel as the
identity provider, so there is no separate password to rotate.