Production issues are where senior Java developers are really tested.
A Spring Boot application may work perfectly in development and testing, but production introduces real traffic, database contention, network failures, memory pressure, slow dependencies, Kubernetes resource limits, and many other unexpected problems.
At senior level, troubleshooting is not about randomly changing configuration values. It is about collecting evidence, identifying the bottleneck, finding the root cause, fixing the problem, and preventing it from happening again.
In a production incident, a strong Java developer should be able to investigate multiple layers:
This guide covers 30 real-world Spring Boot production troubleshooting scenarios that can be asked in senior Java, Spring Boot, Microservices, Kubernetes, AWS, and backend engineering interviews.
I would not immediately increase CPU or memory. First, I would determine where the additional 4.8 seconds are being spent.
I would check:
Distributed tracing is especially useful because it can show whether the request spent most of its time inside the application, database, or a downstream service.
Senior-level approach: Start with the symptom, break the request into components, and identify which component introduced the latency.
Because the problem started after a deployment, I would first compare the new version with the previous version.
I would check:
Java Flight Recorder or a profiler can help identify which methods or threads are consuming CPU.
I would also check whether the new deployment changed JVM options or Kubernetes CPU limits.
If memory continues increasing after GC, I would investigate whether objects are still reachable from GC Roots.
Possible causes include:
A heap dump can help identify which objects are consuming memory and what is retaining them.
The important distinction is that Garbage Collection cannot remove an object that is still reachable.
First, I would determine which type of OutOfMemoryError occurred.
For example, the cause could be:
I would check application logs, JVM metrics, GC logs, heap usage, container memory usage, and recent changes.
If heap exhaustion is suspected, I would capture and analyze a heap dump using tools such as Eclipse MAT or VisualVM.
Increasing heap size may temporarily reduce the frequency of failures, but it does not necessarily fix a memory leak.
I would first determine whether connections are genuinely being consumed by legitimate workload or whether they are being held unnecessarily.
I would investigate:
I would also compare application connection pool metrics with database-side connection metrics.
Increasing the pool size without understanding the database capacity can make the problem worse.
A completely occupied HikariCP pool usually means application requests are holding database connections for too long or the workload requires more connections than the configured pool can provide.
Common causes include:
I would examine HikariCP metrics such as active, idle, pending, and maximum connections along with database query latency.
Important: Connection pool tuning should be based on measured database capacity, not simply increased whenever connections are exhausted.
This is often a sign of an inefficient data-access pattern such as the N+1 query problem.
I would enable appropriate SQL logging or database query monitoring and inspect the queries generated for a single API request.
Possible solutions include:
The goal is not simply to reduce the number of queries blindly, but to retrieve the required data efficiently.
Frequent Full GC indicates significant memory pressure and requires investigation.
I would check:
If the heap remains heavily occupied after Full GC, I would investigate possible memory leaks or excessive object retention.
If the application is simply processing a much larger workload, heap sizing and allocation behavior should also be evaluated.
I would determine what the threads are waiting for.
A thread dump is one of the first tools I would use.
I would look for threads:
I would correlate thread behavior with database pool metrics, HTTP client metrics, CPU usage, and distributed traces.
A growing thread count is a symptom. The important question is why are those threads not completing?
The main goal is to prevent slow dependencies from consuming all resources in the calling service.
I would consider:
For example, if every request waits 30 seconds for a slow downstream service, application threads and HTTP connections can become exhausted.
Timeouts are a reliability mechanism, not just an HTTP configuration setting.
I would introduce an idempotency mechanism so that processing the same logical request multiple times does not produce multiple side effects.
A common approach is to require an Idempotency-Key from the client.
The service stores the key along with the result or processing state and ensures that repeated requests with the same key do not create duplicate transactions.
Database constraints can also provide an important second layer of protection.
I would first determine whether the producer rate increased or the consumer processing rate decreased.
I would check:
If consumers are spending too much time processing each message, simply adding consumers may not solve the underlying bottleneck.
I would also check whether the number of consumers is appropriate for the number of partitions.
In distributed systems, duplicate message delivery can happen even when the application is designed carefully.
I would make message processing idempotent.
Possible techniques include:
The consumer should be designed so that processing the same message twice does not create an incorrect business result.
I would compare the local and Kubernetes environments systematically.
I would check:
I would also inspect Pod events and application startup logs.
Many deployment problems are configuration or environment problems rather than Java code problems.
I would determine why Kubernetes is restarting the container.
I would check:
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous
The key question is whether the restart is caused by the JVM, application failure, memory pressure, or an incorrectly configured health probe.
The container's memory usage includes more than the Java heap.
Possible sources include:
Therefore, a normal heap graph does not automatically mean that the container is safe from OOMKilled.
I would compare JVM memory metrics with the container's total memory usage.
First determine what causes the readiness endpoint to fail.
I would check:
If the readiness probe is unnecessarily dependent on a slow external service, temporary dependency problems could cause healthy Pods to be removed from traffic.
I would compare system behavior during normal traffic and peak traffic.
I would monitor:
The most important question is:
Which resource reaches saturation first?
That usually gives a strong clue about the real bottleneck.
I would not assume the query itself changed.
I would compare:
A query that runs against a small local dataset can behave very differently against millions of production records.
I would first determine whether the application is generating different keys, whether cached entries are expiring too quickly, or whether the cache itself is unavailable or unhealthy.
I would investigate:
Cache metrics should be correlated with database load because a cache miss can directly increase database traffic.
This is primarily a cache invalidation problem.
Possible approaches include:
The correct approach depends on how strongly the application requires consistency.
There is no universal cache invalidation strategy.
I would configure sensible connection and read timeouts instead of allowing requests to wait indefinitely.
I would also consider:
If the business operation does not need an immediate response, asynchronous processing can prevent the user's request from being tied to a slow external dependency.
Retries can amplify an existing failure.
Suppose 1,000 requests are sent to a failing service and every request retries three times.
The downstream service could receive thousands of additional requests while it is already unhealthy.
This is sometimes called a retry storm.
Retries should therefore use appropriate:
Retries should also generally be limited to operations where retrying is safe.
This is where distributed tracing and correlation IDs become extremely useful.
A request may travel through:
API Gateway → Order Service → Payment Service → Inventory Service → Database
If every service logs a correlation or trace identifier, the complete request path can be reconstructed.
Distributed tracing can additionally show which service or operation introduced the error or latency.
Micrometer provides application-level metrics from the Spring Boot application.
Prometheus can collect and store metrics, while Grafana can visualize them through dashboards.
I would monitor metrics such as:
The goal is not to create hundreds of dashboards. The goal is to identify a small set of signals that quickly show where the system is becoming unhealthy.
Distributed tracing provides visibility into the individual operations that make up a request.
For example:
API = 5 seconds
The trace immediately suggests that the Payment Service is responsible for most of the latency.
This is much more useful than looking at the overall API latency alone.
I would compare the performance of the old and new versions using the same or similar traffic conditions.
I would investigate:
Canary deployments and gradual rollouts make this type of comparison much easier in production.
If only the new version shows the degradation, the deployment becomes a high-priority suspect.
I would use multiple application instances or Pods and ensure that new instances become ready before receiving traffic.
A typical Kubernetes deployment would use:
The old version should continue serving traffic while the new version becomes healthy.
An application can be technically healthy while individual requests are failing.
I would first identify the pattern of failures.
I would check:
If only one Pod is returning errors, I would compare it with healthy Pods.
If failures happen only for a particular endpoint or business operation, I would investigate that code path specifically.
This scenario requires correlation rather than focusing on one metric.
I would establish the timeline and determine which resource became saturated first.
For example:
Traffic spike → CPU increases → request processing slows → threads remain busy → database connections remain occupied → latency increases → requests accumulate.
Or the problem could be:
Traffic spike → database becomes saturated → queries become slow → connections remain occupied → application threads block → API latency increases.
I would compare:
The first metric to reach saturation is an important clue, but it should be validated against the rest of the telemetry.
A production Spring Boot application has:
You have 15 minutes to investigate.
What would you check first, what tools would you use, and how would you identify the root cause?
First determine when the incident started.
Compare it with:
Look at request rate, latency, error rate, CPU, memory, thread count, and connection pool utilization.
p95 and p99 latency are particularly useful because average latency can hide a significant number of slow requests.
Investigate heap usage, GC activity, thread behavior, CPU consumption, and allocation patterns.
Use tools such as Java Flight Recorder, thread dumps, heap dumps, and JVM metrics when appropriate.
Investigate database connection usage, slow queries, locks, database CPU, and connection wait time.
If the database is slow, increasing the application connection pool may actually increase database pressure.
Use distributed tracing to determine whether requests are spending significant time waiting for external services or other microservices.
The goal is to determine the first resource that became constrained.
For example:
Slow database → exhausted connections → blocked application threads → increased latency.
Or:
CPU saturation → slower request processing → more concurrent requests → connection exhaustion → increased latency.
Once the root cause is confirmed, apply the appropriate fix.
That might mean:
The important part is that scaling should be based on evidence.
| Symptom | What to Investigate First |
|---|---|
| High API latency | Tracing, database, CPU, threads, downstream calls |
| High CPU | Hot threads, GC, traffic, recent deployment |
| Increasing memory | Heap, GC, heap dump, native memory |
| OutOfMemoryError | Heap, metaspace, native memory, object retention |
| Connection exhaustion | HikariCP, slow queries, transactions, leaks |
| N+1 queries | SQL logs, Hibernate statistics, query analysis |
| Thread count increasing | Thread dump, blocked threads, downstream calls |
| Kafka lag | Consumer throughput, processing time, partitions |
| Pod restarting | Events, previous logs, probes, OOMKilled |
| Intermittent 500 errors | Exceptions, Pods, dependencies, request patterns |
| Cache misses | Keys, TTL, eviction, Redis health |
| Slow database | Execution plan, indexes, locks, database load |
A senior Java developer should be comfortable using both application-level and infrastructure-level tools.
When an interviewer asks:
"Your Spring Boot application is slow in production. What would you do?"
They are usually not looking for an answer such as:
"I will increase the server CPU."
They want to understand how you think.
A strong senior-level troubleshooting approach is:
Symptom → Metrics → Logs → Traces → JVM → Application → Database → Network → Dependencies → Root Cause → Fix → Prevention
For example, if database connections are exhausted, don't immediately increase the HikariCP pool size.
First determine whether connections are being held because of slow queries, long transactions, database locks, connection leaks, or simply increased workload.
That difference demonstrates production-level engineering experience.
Production troubleshooting is one of the most important skills for a senior Java developer.
Spring Boot gives you a powerful application framework, but production performance depends on the entire system.
A slow API could be caused by Java code.
It could be caused by Garbage Collection.
It could be a database problem.
It could be thread or connection pool exhaustion.
It could be Kubernetes resource throttling.
It could even be a completely separate downstream service.
That's why senior engineers should avoid guessing and instead follow the evidence.
Measure → Correlate → Isolate → Fix → Verify → Prevent.
That mindset is more valuable than memorizing a list of production commands.
If you are preparing for a senior Java, Spring Boot, Microservices, Kubernetes, AWS, or Backend Engineering interview, keep these 30 scenarios as a practical troubleshooting checklist.
Production problems rarely come with a clear root-cause message. The skill is finding the signal inside the noise.
Common issues include high API latency, CPU spikes, memory leaks, OutOfMemoryError, database connection exhaustion, slow queries, thread pool exhaustion, Kafka consumer lag, cache problems, downstream service failures, and Kubernetes-related issues.
Start with metrics to understand the symptoms, then use logs and distributed traces to identify where the time is being spent. Investigate CPU, JVM, threads, database connections, SQL queries, network calls, and downstream services before applying a fix.
Check active connections, idle connections, pending connection requests, connection acquisition time, slow SQL queries, long-running transactions, and possible connection leaks. Increasing the pool size should only be done after understanding the database capacity.
Identify the type of memory failure first. Check heap usage, GC activity, container memory, metaspace, direct memory, and thread count. A heap dump can help identify objects that are consuming or retaining excessive memory.
Check whether CPU is being consumed by application code, garbage collection, excessive logging, or another process. Thread dumps, Java Flight Recorder, and profiling can help identify CPU-intensive threads and methods.
Use appropriate timeouts, circuit breakers, bulkheads, bounded concurrency, carefully configured retries, rate limiting, and graceful degradation. The objective is to prevent a slow or unavailable dependency from exhausting resources in the calling service.
The most important skill is systematic root-cause analysis. Instead of changing configuration blindly, collect metrics, logs, and traces, correlate the evidence, isolate the bottleneck, apply the appropriate fix, and verify the result.
0 Comments