CrashLoopBackOff: A Complete Diagnostic Flowchart

The short answer

CrashLoopBackOff means your container starts, exits, and Kubernetes keeps restarting it with a growing delay. It is a symptom, not a cause. Run these three commands in order:

kubectl describe pod <pod>                   # Last State, then Exit Code
kubectl logs <pod> --previous                # logs from the run that died
kubectl get events --sort-by=.lastTimestamp  # cluster level context

The exit code narrows it immediately. 1 means your app threw. 137 means OOMKilled or SIGKILL. 139 is a segfault. 143 is SIGTERM. 127 is command not found. 0 means your process finished normally and the restart policy is wrong for this workload.

Tested on Kubernetes 1.32. Behaviour is stable across 1.28 and later.

What the state actually means

The backoff is the important part. Kubernetes restarts a failed container after 10 seconds, then 20, 40, 80, 160, capping at 5 minutes. CrashLoopBackOff is the status shown while the kubelet is waiting. The pod is not crashing at that instant, it is sitting in a penalty box.

Two practical consequences people miss. Once a pod is deep into backoff, your fix will appear not to work for up to five minutes, so delete the pod after applying a change rather than waiting. And kubectl logs without --previous will often return nothing at all, because the current container has not started yet.

The counter resets after a container runs successfully for 10 minutes.

Step one: get the exit code

kubectl describe pod <pod> | grep -A6 "Last State"

Last State:     Terminated
  Reason:       Error
  Exit Code:    1
  Started:      Wed, 11 Mar 2026 08:51:02 +0000
  Finished:     Wed, 11 Mar 2026 08:51:03 +0000

Two things to read here. The exit code, and the gap between Started and Finished. One second means the process died during startup, so config, a missing file, or a failed connection. Forty seconds means it started, did some work, and then hit something. That distinction saves a lot of time.

The causes, ranked

1. The application throws on startup, exit 1

The most common by a wide margin and the most boring. Your app cannot start because something it needs is missing or wrong.

kubectl logs <pod> --previous

The --previous flag is essential. Without it you get the container that is about to start, not the one that died.

Typically you find a missing environment variable, an unparseable config file, a failed database connection, a port binding failure, or a migration that cannot run.

The subtle version is a missing key in an existing ConfigMap or Secret. The pod will not start at all if a key referenced through valueFrom does not exist, and that produces CreateContainerConfigError rather than CrashLoopBackOff. So if you see that status instead, check your key names:

kubectl get secret app-secrets -o jsonpath='{.data}' | jq 'keys'

The other subtle version: the app connects to a dependency that is not ready yet. Your API starts before Postgres has finished its own startup, throws, exits, and by the third restart Postgres is up and everything is fine. If your pod crashes twice and then stabilises, that is this. It is not harmful but it is noise, and the fix is a proper readiness gate or an init container that waits.

2. OOMKilled, exit 137

Last State:  Terminated
  Reason:    OOMKilled
  Exit Code: 137

The container exceeded its memory limit and the kernel killed it. Note that Reason: OOMKilled appears explicitly. If you see 137 without it, something else sent SIGKILL, such as a forced delete, a node drain, or a liveness probe failure with a hard kill.

The trap is that your app's own memory reporting will show it was nowhere near the limit. JVM heap says 400MB, limit is 512MB, and it still died, because the limit counts heap plus metaspace, thread stacks, direct buffers, and JVM overhead. Same story for Node with heap versus RSS versus native modules, and for Python with heap versus allocator arenas that never return to the OS.

Fix by raising the limit, or by setting the runtime's internal limits well below the container limit with something like -XX:MaxRAMPercentage=75 or --max-old-space-size.

3. Command not found or not executable, 127 and 126

exec /app/server: no such file or directory

This message lies to you in a specific way. It very often does not mean the file is missing. It means the dynamic linker the binary needs is missing.

The classic case: you build a Go or Rust binary on a Debian builder image and copy it into Alpine. Alpine uses musl, not glibc. The binary exists, ls shows it, and it still reports "no such file or directory" because the interpreter it references at /lib64/ld-linux-x86-64.so.2 is not there.

kubectl run debug --rm -it --image=<your-image> --command -- sh
ls -la /app/server
ldd /app/server           # "not a dynamic executable" is good for a static binary

Fix by building with CGO_ENABLED=0 for a static Go binary, adding gcompat on Alpine, or matching your builder and runtime base images.

The other flavour of 127 is a shell form CMD referencing a binary that is not in the image, such as curl or bash, since distroless and Alpine do not ship bash. 126 is usually a missing execute bit, fixed with COPY --chmod=755.

4. Architecture mismatch

exec /app/server: exec format error

Different message, different cause. The binary was built for a different CPU architecture. Built on an Apple Silicon laptop, deployed to amd64 nodes. Very common and worth its own post. The short version is docker buildx build --platform linux/amd64.

5. A liveness probe killing a healthy container

Nasty, because the app is fine and Kubernetes is the problem.

Warning  Unhealthy  kubelet  Liveness probe failed: HTTP probe failed with statuscode: 503
Normal   Killing    kubelet  Container app failed liveness probe, will be restarted

If you see Killing events in kubectl describe, the container did not crash, it was executed.

The usual cause is an initialDelaySeconds shorter than your app's real startup time. The app needs 45 seconds to warm caches and run migrations, the probe starts checking at 10 seconds, fails three times, kubelet kills it, repeat forever. The app never gets far enough to become healthy.

The correct fix in modern Kubernetes is a startup probe, which suspends the liveness probe until the app has come up once:

startupProbe:
  httpGet: { path: /healthz, port: 8080 }
  failureThreshold: 30
  periodSeconds: 10        # allows up to 300s to start
livenessProbe:
  httpGet: { path: /healthz, port: 8080 }
  periodSeconds: 10
  failureThreshold: 3

The second cause here is a liveness endpoint that checks dependencies. If /healthz returns 503 because the database is briefly unavailable, Kubernetes restarts every replica of your service simultaneously during a database blip, turning a small incident into an outage. Liveness should answer one question, is this process wedged, and nothing else. Dependency checks belong in readiness.

6. SIGTERM during shutdown, exit 143

Exit 143 means the process received SIGTERM. During a normal rolling update that is expected. In a crash loop it means something is repeatedly telling the container to stop, often a failing preStop hook or a node under pressure evicting the pod.

Also check whether your process is PID 1 and ignoring signals. If you start your app through a shell, as in CMD npm start, the shell is PID 1 and may not forward SIGTERM. The app never shuts down cleanly, the grace period expires, and it gets SIGKILL and exit 137 instead. Use exec form, CMD ["node", "server.js"], or an init such as tini.

7. Segfault, exit 139

Native code crashed. Usually a native dependency compiled against different library versions than the runtime provides, or genuine memory corruption in a C extension.

Look for sharp, node-canvas, psycopg2 binary wheels, grpcio, or anything shipping a shared object. Try rebuilding native modules inside the target image rather than copying node_modules from the host.

8. Exit code 0 with restarts

The container did what it was told and finished. Kubernetes restarts it because restartPolicy defaults to Always.

That means you have deployed a batch task as a Deployment. Use a Job or CronJob.

The other version is a CMD that is not a long running process at all: a shell script that ends, a server started with & in the background, a docker compose habit carried over. Whatever runs as PID 1 has to block for the lifetime of the container.

9. Read only filesystem or permission denied

Hardened pod security contexts break applications that expect to write somewhere.

EACCES: permission denied, open '/app/tmp/cache'

With readOnlyRootFilesystem: true you have to mount an emptyDir at every path the app writes to, including /tmp, which more libraries use than you would guess. With runAsNonRoot: true, files copied in a Dockerfile are owned by root and may not be readable by the runtime UID.

The flow, compressed

CrashLoopBackOff
|
+- describe pod, read Last State, get Exit Code
|
+- 0 ......... wrong workload type (use a Job) or CMD does not block
+- 1 ......... kubectl logs --previous, read the app error
+- 126/127 ... binary or interpreter missing, check base image, ldd, chmod
+- 137 ....... Reason OOMKilled?  yes: raise limit or tune runtime heap
|              no reason given?   check liveness probe events
+- 139 ....... native extension segfault, rebuild in target image
+- 143 ....... SIGTERM, check PID 1 signal handling, preStop, evictions
|
+- no Last State, or Started and Finished within 1s
   +- describe, read Events, look for Killing or Unhealthy
      +- yes: probe misconfiguration, add a startupProbe
      +- no:  CreateContainerConfigError? missing ConfigMap or Secret key

When you cannot get logs at all

Sometimes the container dies too fast to log anything. Two techniques.

Override the entrypoint and look around inside the real image:

kubectl run debug --rm -it --image=<image> --command -- sh

Or use an ephemeral debug container attached to the live pod's namespaces:

kubectl debug -it <pod> --image=busybox --target=app

That gets you into a distroless container's network and process namespace without rebuilding anything, which is the main reason kubectl debug exists.

Prevention

Startup probes on anything with a warm up period. This eliminates the single most confusing cause.

Liveness probes must not check dependencies. Put it in your platform's golden path template so nobody has to remember.

Put kubectl logs --previous in your runbook. Most people do not know the flag exists and conclude the pod produces no logs.

Pin --platform in CI if any developer builds images locally on a different architecture.

Alert on restart count, not just pod status. A pod that crashes twice then stabilises never shows as unhealthy, but it is telling you about a missing readiness gate.