Kubernetes for Backend Engineers: Deploying Java Services the Right Way

It was a Tuesday deploy. Nothing unusual — a Spring Boot service, three new endpoints, a minor library bump. It ran clean locally, passed CI, passed the staging smoke tests. We merged, the pipeline built the image, and kubectl rollout status sat there for a minute before the pods started cycling: Running → Running → CrashLoopBackOff → Running → CrashLoopBackOff.

Locally the JVM had 8 GB of laptop RAM to stretch into. In the cluster, the container had a 512 MB memory limit that nobody had touched since the manifest was copy-pasted from another team eighteen months earlier. The JVM didn't know or care about that limit — it sized its heap off the node's visible memory, allocated more than the cgroup allowed, and the kernel OOM-killed the process a few seconds after startup, every time, forever, in a loop.

That's the moment most backend engineers get formally introduced to Kubernetes. Not through a tutorial — through a pager. This post is the explanation we wish we'd had before that: what the core objects are actually for, why the JVM and containers have an uneasy relationship around memory, how probes decide whether your pod lives or dies, and a debugging sequence that doesn't start with "have you tried checking the logs."

We're writing this from the point of view of the person who owns the Spring Boot service, not the platform team who owns the cluster. You don't need to know how etcd achieves consensus. You need to know why your pod won't start, why it restarts every ninety seconds, and what to put in your YAML so it stops doing that.

That distinction matters more than it sounds like it should. A platform-team article about Kubernetes will spend paragraphs on scheduler internals, CNI plugins, and cluster autoscaling policy — all real, all mostly irrelevant to the person whose job is "make the orders service respond to HTTP requests reliably." The failures that actually eat a backend engineer's evening are almost always at the boundary between "how the JVM manages memory and threads" and "how Kubernetes decides a container is healthy." That boundary is where this post lives.

The objects you actually touch

Kubernetes has dozens of resource types. As a backend engineer shipping a Java service, you'll interact with six of them regularly, and understanding what each one is for — not its formal definition — saves a lot of confusion.

Pod. The smallest deployable unit — one or more containers that share a network namespace and are always scheduled together. In practice, for a typical Spring Boot service, a pod is one container: your app. You almost never create pods directly. Something else creates them for you.

Deployment. The thing that actually manages pods. You declare "I want 3 replicas of this container image, with these resource limits and these probes," and the Deployment controller reconciles reality toward that declaration — creating pods, replacing crashed ones, and rolling out new versions when you change the image tag. When people say "deploy the service," they mean "update the Deployment's pod template," and Kubernetes takes it from there.

Service. A stable network identity in front of a set of pods that come and go. Pods get new IPs every time they're recreated; a Service gives you one DNS name (my-service.namespace.svc.cluster.local) and load-balances across whichever pods currently match its label selector. Without a Service, every pod restart would mean re-discovering IPs, which is unworkable.

Ingress. The entry point from outside the cluster into a Service. It's where hostname and path routing live — api.example.com/orders goes to one Service, api.example.com/payments goes to another. It's also usually where TLS termination happens. Think of it as the reverse proxy layer sitting in front of your internal Services.

ConfigMap. Non-secret configuration — property files, environment-specific values, feature flags — injected into pods as environment variables or mounted files, decoupled from the container image. The same image gets different application.yml overrides in dev versus prod purely by which ConfigMap it's paired with.

Secret. Structurally identical to a ConfigMap but intended for sensitive values — database passwords, API keys, signing certs. Base64-encoded by default, not encrypted, which surprises people the first time they kubectl get secret -o yaml and realize anyone with cluster read access can decode it in one line. Real secret hygiene means RBAC restrictions and, ideally, an external secrets manager — Kubernetes Secrets alone are a weak boundary.

Every one of those six objects shows up in a normal deploy of a single service, and none of them is optional in practice — skip the Ingress and nothing outside the cluster can reach you, skip the ConfigMap and your environment-specific settings end up hardcoded in the image, skip a properly scoped Service and every pod restart breaks whoever was talking to the old IP. The rest of this post is really about getting the Deployment's pod template right, because that's the one object that encodes everything the JVM specifically needs to run well: resource sizing, probe timing, and shutdown behavior.

Here's how those pieces connect for a request coming in from a browser or another service:

The Ingress picks the right Service by hostname/path. The Service picks a healthy pod by label selector and round-robins (roughly — kube-proxy's actual algorithm depends on mode). The pod runs your Spring Boot JAR. Everything downstream of "Service" is where the JVM-specific problems live, and that's most of this post.

Memory limits and the JVM: where OOMKilled comes from

This is the failure mode that opened this post, and it's worth understanding precisely, because "just bump the memory limit" is a band-aid, not a fix.

A container's memory limit is enforced by a Linux cgroup. When the total memory used by all processes in that cgroup exceeds the limit, the kernel's OOM killer terminates a process — usually your JVM — and Kubernetes reports the pod's last state as OOMKilled. This has nothing to do with Java's own OutOfMemoryError; the kernel doesn't ask the JVM nicely, it sends SIGKILL.

Modern JVMs (Java 10+) are "container-aware" — they read the cgroup limit and size the default max heap (-Xmx) as a fraction of it (historically 25%, tunable via -XX:MaxRAMPercentage). That sounds like it should just work. The problem is heap is not the only thing living in that container's memory budget. A running JVM process also needs:

• Heap — your objects, sized by -Xmx / MaxRAMPercentage
• Metaspace — class metadata, unbounded by default unless you cap it with -XX:MaxMetaspaceSize
• Thread stacks — each thread reserves stack space (-Xss, default ~1 MB); a Spring Boot app with a Tomcat thread pool of 200 can reserve 200 MB in stacks alone under load
• Off-heap / direct buffers — Netty (used by WebFlux and many reactive clients), JDBC connection pool buffers, compression buffers
• JIT code cache, GC bookkeeping structures, native libraries

If you set -Xmx to consume the entire container memory limit, everything else — metaspace, threads, off-heap — has nowhere to live, and the kernel kills the container the moment total usage crosses the cgroup boundary. This is why a service that "worked fine" under light load OOM-kills the instant real traffic hits it: more concurrent requests means more threads means more stack memory means the JVM tips over a ceiling that had no headroom to begin with.

The fix is to explicitly budget the container limit across all of these, rather than letting the JVM's automatic heap sizing assume it owns the whole container:

resources:
  requests:
    cpu: "500m"
    memory: "768Mi"
  limits:
    cpu: "1"
    memory: "1Gi"

And on the JVM side, be explicit rather than relying purely on percentage-based defaults:

env:
  - name: JAVA_TOOL_OPTIONS
    value: >-
      -XX:MaxRAMPercentage=65.0
      -XX:MaxMetaspaceSize=256m
      -XX:ReservedCodeCacheSize=96m
      -Xss512k
      -XX:+ExitOnOutOfMemoryError

The rough budget for a 1 GB limit: ~650 MB heap, ~256 MB metaspace ceiling, thread stacks and code cache and Netty buffers absorbing the remaining ~100–150 MB. Leave headroom — don't spend right up to the limit. -XX:+ExitOnOutOfMemoryError is worth calling out specifically: without it, a JVM that hits a real OutOfMemoryError can end up in a zombie state — alive but unable to make progress, still holding the port, still failing readiness checks — instead of dying cleanly and letting Kubernetes restart it. Fail fast and let the orchestrator do its job.

Requests versus limits matter for a different reason: requests are what the scheduler uses to decide which node has room for your pod, and they're what you're guaranteed. Limits are the hard ceiling before throttling (CPU) or killing (memory) kicks in. Setting requests too low packs more pods onto a node than it can comfortably serve under load — everyone gets starved together during a traffic spike, which is a worse failure mode than one pod restarting.

Three probes, three different jobs

Kubernetes gives you three probe types, and conflating them is one of the most common sources of self-inflicted outages.

Liveness probe — "is this process still functioning, or is it stuck?" A failed liveness probe causes Kubernetes to kill and restart the container. This is your safety net against deadlocks and unrecoverable hangs, not against slow responses.

Readiness probe — "should this pod currently receive traffic?" A failed readiness probe doesn't restart anything — it just pulls the pod out of the Service's endpoint list until it passes again. This is what you want during a temporary dependency outage (say, the database connection pool is exhausted): stop sending traffic, don't kill the process.

Startup probe — "has this process finished starting up yet?" While the startup probe hasn't succeeded, liveness and readiness probes are suspended entirely. This exists specifically for slow-booting applications, and a Spring Boot service with a large context, Flyway migrations, and cache warming on startup is exactly the case it was built for.

Here's the failure this prevents: without a startup probe, a liveness probe with, say, a 10-second initial delay and a 3-second timeout will start firing at 10 seconds. If your Spring context takes 40 seconds to come up under a real dependency set (JPA repositories, Kafka consumers, a warm cache load), the liveness probe fails repeatedly during startup, Kubernetes concludes the app is unhealthy, and kills it — before it ever had a chance to finish booting. The pod restarts, starts booting again, gets killed again at the 10-second mark again. That loop is CrashLoopBackOff, and the process was never actually broken — it just needed more time than the probe gave it.

A full deployment manifest with all three probes and sane resource sizing:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-service
  labels:
    app: orders-service
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  selector:
    matchLabels:
      app: orders-service
  template:
    metadata:
      labels:
        app: orders-service
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: orders-service
          image: registry.internal/orders-service:1.14.2
          ports:
            - containerPort: 8080
          envFrom:
            - configMapRef:
                name: orders-service-config
            - secretRef:
                name: orders-service-secrets
          resources:
            requests:
              cpu: "500m"
              memory: "768Mi"
            limits:
              cpu: "1"
              memory: "1Gi"
          startupProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            failureThreshold: 30
            periodSeconds: 2
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 0
            periodSeconds: 10
            timeoutSeconds: 3
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 0
            periodSeconds: 5
            timeoutSeconds: 2
            failureThreshold: 3

Notice the startup probe's math: failureThreshold: 30 × periodSeconds: 2 gives the app up to 60 seconds to come up before Kubernetes gives up and restarts it. Once the startup probe succeeds once, it stops being checked, and liveness/readiness take over on their own schedules. Spring Boot Actuator's /actuator/health/liveness and /actuator/health/readiness groups (enabled via management.endpoint.health.probes.enabled=true) map directly onto Kubernetes' liveness/readiness split — use them rather than hitting the general /actuator/health endpoint, which bundles in dependency checks (DB, message broker) that belong on the readiness side, not liveness. A database being briefly unreachable should make a pod stop receiving traffic, not get killed.

Rolling updates: what maxUnavailable and maxSurge actually do

The default RollingUpdate strategy replaces old pods with new ones incrementally rather than all at once. Two fields control the pace:

• maxUnavailable — how many pods (count or percentage) below the desired replica count the Deployment is allowed to drop to during the rollout. maxUnavailable: 1 on 3 replicas means at least 2 must always be up and ready.
• maxSurge — how many extra pods (above the desired count) can be created temporarily while old ones are still terminating. maxSurge: 1 on 3 replicas means the Deployment may briefly run 4 pods.

With both set to 1 on a 3-replica Deployment, a rollout looks roughly like: spin up 1 new pod (now at 4 total) → wait for it to pass its readiness probe → terminate 1 old pod (back to 3) → repeat until all 3 are new. Readiness gating is the important part here — a new pod counts as "up" for rollout purposes only once it passes readiness, which is exactly why getting the readiness probe right (see above) directly affects rollout safety. A readiness probe that reports "ready" before the app can actually serve traffic will make Kubernetes route real requests to a pod that isn't prepared for them, producing a burst of errors mid-deploy that looks like nothing changed in kubectl rollout status — because from the controller's point of view, nothing went wrong.

For services where even one dropped replica during deploys matters, maxUnavailable: 0, maxSurge: 1 guarantees full capacity throughout the rollout at the cost of momentarily running one extra pod (and consuming its resource requests on some node).

Horizontal Pod Autoscaling and the JVM warm-up problem

The HorizontalPodAutoscaler (HPA) watches a metric — most commonly CPU utilization — and adjusts replica count to keep it near a target. The default setup looks simple:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-service
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

For a JVM service, plain CPU-based scaling has a timing problem worth knowing about before you rely on it. The HPA checks metrics on an interval (default every 15 seconds) and needs several consecutive high readings before it scales up — by design, to avoid flapping. That's fine for stateless, fast-starting workloads. It's a poor fit for a Spring Boot service where a freshly scheduled pod needs 20–40 seconds just to pass its startup probe, and then several more seconds of JIT warm-up before its steady-state latency is representative. By the time the HPA notices sustained CPU pressure, decides to scale, and the new pod finishes starting and warming up, the traffic spike that triggered the scale-up may already be over — or worse, still climbing, because the new capacity arrived too late to help and the existing pods are now under even more load.

Two practical mitigations: use custom metrics (request latency, queue depth, requests-in-flight from something like Prometheus Adapter) instead of raw CPU, since those track user-facing pain more directly than CPU percentage does; and set minReplicas high enough that normal traffic variance never triggers a cold scale-up in the first place — treat the HPA as a safety valve for sustained, unusual load, not as your primary capacity mechanism for daily traffic curves. Pre-provisioned baseline capacity beats reactive autoscaling for any workload where "slow to start" is a real cost.

Graceful shutdown: don't drop in-flight requests

When a Deployment scales down, rolls out a new version, or a node gets drained, Kubernetes doesn't just kill pods — it sends SIGTERM, waits up to terminationGracePeriodSeconds, and only then sends SIGKILL if the process hasn't exited. That window is your chance to finish in-flight work cleanly, and by default a plain Spring Boot app doesn't use it well: Tomcat's default shutdown behavior can drop connections mid-request when the JVM receives SIGTERM.

The fix is one property:

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

With server.shutdown: graceful, on SIGTERM the embedded server stops accepting new connections but lets existing in-flight requests complete, up to the configured timeout. There's a coordination detail that catches people out here: there's a gap between when a pod is marked "terminating" and removed from Service endpoints, and when it actually receives SIGTERM and stops accepting new work — during that gap, some load balancers may still route a trickle of new requests to a pod that's on its way out. A short preStop hook that just sleeps a couple of seconds before the container's main process gets SIGTERM closes most of that gap in practice:

lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "sleep 5"]

And terminationGracePeriodSeconds needs to comfortably exceed your graceful shutdown timeout plus the preStop sleep — if it's shorter, Kubernetes SIGKILLs the process before your graceful drain finishes, which defeats the entire point. In the manifest above we set it to 45 seconds against a 30-second shutdown phase timeout and a 5-second preStop sleep, leaving headroom rather than cutting it exactly to the wire.

Config across dev, staging, and prod without manifest duplication

The pattern that scales is: one Deployment manifest per service (parameterized, not duplicated per environment), paired with an environment-specific ConfigMap and Secret that get swapped at deploy time — typically via Helm values files, Kustomize overlays, or a templating step in CI, rather than hand-copied YAML per environment.

apiVersion: v1
kind: ConfigMap
metadata:
  name: orders-service-config
data:
  SPRING_PROFILES_ACTIVE: "prod"
  DB_POOL_SIZE: "20"
  FEATURE_NEW_PRICING: "true"
  LOG_LEVEL: "INFO"

The Deployment references the ConfigMap and Secret by name only (envFrom, as in the manifest earlier) — it never contains environment-specific values itself. Swap the ConfigMap and Secret, keep the Deployment identical, and the same container image gets promoted unchanged from dev through staging to prod, with only its surrounding configuration differing. This also gets you the property Spring Boot's application-{profile}.yml layering was designed for: the image is truly the same artifact everywhere, which is the whole point of promoting a build rather than rebuilding per environment.

One gotcha: updating a ConfigMap doesn't automatically restart pods that already read its values as environment variables at container start — they keep the old values until the pod is recreated. If you need config changes to take effect without a full redeploy, mount the ConfigMap as a volume instead of environment variables (mounted files do update live, and Spring Cloud Kubernetes or a file-watching config mechanism can pick up the change), or accept that env-var-based config requires a rollout to apply, and treat that as a feature, not a bug — it keeps what's running observably tied to what's declared.

Debugging CrashLoopBackOff without guessing

CrashLoopBackOff is not itself an error — it's Kubernetes telling you a container keeps exiting and it's backing off between restart attempts. The actual cause is always somewhere else, and there's a fixed order worth checking in, rather than jumping straight to logs:

1. kubectl get pods — confirm it's actually CrashLoopBackOff and not ImagePullBackOff (those two get confused constantly, and the fix is completely different — see below).

2. kubectl describe pod <pod-name> — read the Events section at the bottom first. This tells you why the last restart happened before you even look at application logs: Last State: Terminated, Reason: OOMKilled means the memory conversation above is your answer, full stop — no amount of log reading will show a Java stack trace for a SIGKILL. Reason: Error with a non-zero exit code means the process exited on its own — that's an application-level failure. Failed probe events (Liveness probe failed, Readiness probe failed) point at startup timing, not a crash at all.

3. kubectl logs <pod-name> --previous — the --previous flag is the part people forget. Once a container has restarted, plain kubectl logs shows the new attempt's (often empty) logs, not the one that actually crashed. --previous pulls logs from the terminated instance.

4. Check the exit code and reason together: kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].lastState.terminated}' gives you the exact exitCode and reason in one shot — 137 is SIGKILL (OOM or manual kill), 143 is SIGTERM (often a probe-triggered restart or graceful shutdown timeout being exceeded), and application-thrown exit codes vary by what your main() or Spring's SpringApplication.exit() returns.

5. Cross-reference probe timing against actual startup time. Run the image locally (or kubectl exec into a working replica and hit the health endpoint with curl at intervals right after a restart) to measure real cold-start time, then compare against your startup probe's failureThreshold × periodSeconds. If real startup consistently exceeds that budget, the probe — not the app — is the bug.

6. Check resource pressure at the node level with kubectl top pod and kubectl describe node <node-name> — a pod can also be evicted or throttled because the node itself is under memory pressure from other pods, which shows up looking like your service is unhealthy when the actual problem is a noisy neighbor.

ImagePullBackOff is a different animal entirely and worth distinguishing explicitly because the panic response ("check the app logs") wastes time here — there are no app logs, because the container never started. It means the kubelet can't pull the image: wrong tag, private registry auth missing (imagePullSecrets not configured or expired), or a typo in the image name. kubectl describe pod again gives you the exact pull error in Events — "manifest not found" means the tag doesn't exist, "unauthorized" means a registry credentials problem, not a Kubernetes problem.

Worth internalizing as a general rule: describe, not logs, is almost always the right first move for anything that looks like a scheduling or lifecycle problem rather than an application bug. logs tells you what your code did. describe tells you what Kubernetes did to your pod and why — which container image it tried to pull, which probe it ran and what response it got back, which node it landed on, and whether it was evicted for resource pressure rather than crashing on its own. Reaching for logs first, out of habit, is usually why a ten-second diagnosis turns into a twenty-minute one.

Common mistakes

No memory limit set at all. Omitting limits.memory doesn't mean "unlimited and safe" — it means one runaway pod (a memory leak, an unbounded cache, a bad request pattern) can consume enough of a shared node's memory to degrade or evict every other pod scheduled there, turning a single service's bug into a multi-tenant outage. Always set a limit, even a generous one.

Liveness probe timing that kills a pod mid-GC-pause. A timeoutSeconds: 1 liveness probe against a service using a stop-the-world collector (or a G1/ZGC instance under memory pressure triggering a longer-than-usual pause) can register as a failure during a GC pause that's slow but recoverable, restarting a pod that would have been fine two seconds later. Give liveness probes reasonable timeouts (2–3 seconds) and a failureThreshold above 1, so a single slow response doesn't trigger a restart.

No startup probe on a slow-booting Spring Boot app. Covered above, but worth repeating as its own line item because it's the single most common cause of CrashLoopBackOff on Java services specifically — the app isn't broken, the liveness probe just started grading it before it had a chance to finish booting.

Readiness probe hitting a path that depends on a downstream system with no fallback. If /actuator/health/readiness includes a check against a third-party API with a long timeout and no circuit breaker, one slow external dependency takes your entire fleet out of rotation simultaneously — the opposite of what readiness checks are supposed to protect against.

Relying on kubectl logs without --previous and concluding "there's nothing in the logs." This one costs more debugging time than almost any other single habit gap — the logs exist, they're just attached to the terminated container instance, not the freshly restarted one you're looking at by default.

Copy-pasting a manifest from another service and never revisiting its numbers. The 512 MB limit that started this post wasn't chosen for the orders service — it was inherited from whichever manifest got cloned first, for a much lighter service, and nobody circled back once real traffic and a heavier dependency set arrived. Resource requests and limits are not "infrastructure boilerplate" you set once and forget; they're a claim about how much memory and CPU this specific JVM, running this specific code, actually needs, and that claim needs revisiting whenever the workload changes meaningfully — a new cache, a bigger connection pool, a heavier library.

Final takeaway

None of this is exotic Kubernetes knowledge — it's the specific intersection of container resource management and JVM memory behavior, plus a handful of timing parameters that decide whether a slow-starting, well-behaved Spring Boot service looks healthy or looks broken to an orchestrator that doesn't know anything about the JVM underneath it. The pattern behind almost every one of these failure modes is the same: Kubernetes makes a decision (kill this pod, route traffic here, scale up now) based on a signal you configured, and when that signal doesn't match what your application is actually doing, the orchestrator does exactly what you told it to and it looks like a bug. Get the resource budget, the three probes, and the shutdown sequence right, and most of what looks like "Kubernetes is unstable" turns out to have been a JVM that was never given an honest description of the container it was running in.

Next in this DevOps series: Production Observability with OpenTelemetry, Prometheus, Grafana, and Logs, where we'll cover how to actually see what's happening inside the pods this post just got running reliably.

Subscribe to get system design and backend engineering posts in your inbox every two weeks.