Runbooks

One page per Ready-condition Reason the operator sets, plus cross-cutting incident guides. Each page covers the symptom, the cause, how to diagnose it, and how to remediate.

The operator automatically appends a link to the matching page on each actionable Ready-condition message — (runbook: https://jaas.projects.metio.wtf/runbooks/<reason>/) — so kubectl describe points straight at the remediation page. Healthy reasons (Synced, Suspended, Pending) get no link.

• ArtifactTooLarge

  The snippet's rendered output exceeds the operator's per-artifact byte cap

• CRD watch engagement failing

  A runtime watch on a Flux source CRD failed to engage, so snippets referencing that kind no longer re-render on upstream changes

• CrossNamespaceRefRejected

  The snippet references a library or Flux source in a different namespace and cross-namespace refs are disabled

• DependencyCycle

  The snippet's sourceRef or library chain loops back to itself, which would cause infinite re-render

• Eval-concurrency saturation

  The global concurrent-eval cap is full and the operator is shedding new evaluations, typically because a runaway snippet is holding slots past its deadline

• EvaluationFailed

  The Jsonnet snippet failed to evaluate due to a syntax error, runtime error, or unresolved import

• EvaluationTimeout

  The snippet's evaluation exceeded the operator's wall-clock deadline and was abandoned

• ExternalVariableConflict

  The snippet declares an external variable key already claimed by the operator via --ext-var

• High reconcile latency

  Individual reconcile calls are taking longer than the configured p99 threshold, indicating slow source fetches, heavy evaluation, or a sluggish storage backend

• InvalidSpec

  The snippet's spec contains a field combination the reconciler cannot process, such as a missing entryFile or conflicting source fields

• LibraryNotFound

  A JsonnetLibrary CR referenced in spec.libraries does not exist or is unreachable by the tenant ServiceAccount

• Operator pod not ready

  At least one jaas pod has been Ready=False for the configured alert window, so new snippets are not being reconciled

• Pending

  The snippet has been observed by the operator but its first reconcile pass has not yet completed

• RBACDenied

  The apiserver returned Forbidden on a call the reconciler made with the tenant ServiceAccount's impersonated identity

• Self-signed webhook cert renewal failing

  The Renewer goroutine cannot rotate the webhook TLS material; if uncorrected the cert will expire and every JsonnetSnippet admission will fail

• ServiceAccountMissing

  The snippet specifies no ServiceAccount and the operator has no --default-service-account configured

• SourceFetchFailed

  The operator resolved the source CR but the artifact download failed due to an HTTP error, digest mismatch, or oversized tarball

• SourceNotReady

  The referenced Flux source CR exists but has not yet reported Ready=True or has no published artifact

• SourceRefNotYetSupported

  The snippet uses spec.sourceRef but the running binary was built without a Fetcher wired in

• Storage backend recovery

  The artifact store is degraded (PVC lost, S3 endpoint down, or storage HTTP server unreachable) and downstream Flux consumers can no longer fetch tarballs

• Suspended

  Reconciliation is intentionally paused because spec.suspend is true; the last published artifact remains intact

• Synced

  The snippet reconciled end-to-end and its ExternalArtifact is current; no action is required

• Watch-layer silent failure

  The operator's own ClusterRole is missing a verb on a watched kind, so controller-runtime's informer retries silently and no snippet status reflects the problem

• Workqueue saturation

  The reconciler's workqueue depth exceeds the threshold because the operator is dequeuing reconciles slower than the apiserver enqueues them