Coroot already does eBPF-based CPU profiling for Java. It catches CPU hotspots well, but that’s all it can do. Every time we looked at a GC pressure issue or a latency spike caused by lock contention, we could see something was wrong but not what.
We wanted memory allocation and lock contention profiling. So we decided to add <a href="https://github.com/async-profiler/async-profiler" target="_blank" rel="noopener nofollow">async-profiler</a> support to <a href="https://github.com/coroot/coroot" target="_blank" rel="noopener nofollow">coroot-node-agent.</a> The goal: memory allocation and lock contention profiles for any HotSpot JVM, with zero code changes. Here’s how we got there.
<h1>Why async-profiler</h1>
We went with async-profiler. It’s a native JVMTI agent used by pretty much everyone in the Java profiling space (Pyroscope, IntelliJ, Datadog). It can be loaded into a running JVM dynamically, supports CPU, allocation, and lock contention profiling in a single session, and works in unprivileged containers with no JVM flags. It outputs JFR format, which we parse using Grafana’s <a href="https://github.com/grafana/jfr-parser" target="_blank" rel="noopener nofollow">jfr-parser.</a>
<h1>How we integrated it</h1>
The integration follows the same pattern as our <a href="https://coroot.com/blog/java-tls-instrumentation-with-ebpf" target="_blank" rel="noopener nofollow">Java TLS agent:</a>
The node agent detects Java processes by checking if the binary name ends with java, then confirms it’s a HotSpot JVM by scanning <code>/proc/&lt;pid&gt;/maps.</code> It deploys <code>libasyncProfiler.so</code> (~600KB) into the container’s filesystem at <code>/tmp/coroot/</code> and loads the library into the JVM via the Attach API. async-profiler starts with <code>event=itimer</code>,<code>interval=10ms,alloc,lock,jfr,</code> capturing CPU, allocation, and lock events in a single session.
For data collection, every 60 seconds the agent sends a stop command (async-profiler finalizes the JFR file), reads the file from the host via <code>/proc/&lt;pid&gt;/root/,</code> and immediately sends a start command to begin a new recording.
The gap between stop and start is ~4ms. We considered using dump (which doesn’t stop the profiler), but JFR output requires proper chunk finalization, a dump writes incomplete metadata that parsers reject. The stop/start approach guarantees valid output every time.
Each command goes through the JVM Attach protocol. It’s one command per connection, HotSpot closes the socket after each response. After the first attach (which triggers the attach listener via SIGQUIT), subsequent connections just hit the existing Unix socket. Total overhead: ~2ms per command.
If another tool (Pyroscope Java agent, Datadog, etc.) already loaded async-profiler into the JVM, we detect it by scanning <code>/proc/&lt;pid&gt;/maps</code> and skip that process to avoid conflicts.
<h1>Enabling it</h1>
Set the <code>ENABLEJAVAASYNC_PROFILER=true</code> environment variable on the node agent. In the Coroot custom resource:
<pre><code>apiVersion: coroot.com/v1
kind: Coroot
spec:
 nodeAgent:
 env:
 - name: ENABLE_JAVA_ASYNC_PROFILER
 value: &quot;true&quot;
</code></pre>
No JVM flags, no application restarts, no agent JARs. The node agent handles everything automatically for all HotSpot JVMs it discovers. If you haven’t enabled it yet, the JVM report shows a hint with a link to the docs.
<h1>What you get</h1>
Once enabled, Coroot adds new charts to the JVM report: allocation rate (bytes/s and objects/s) and lock contention (contentions/s and delay). Each chart has a profile button that opens the corresponding flamegraph, so you can go from “allocation rate spiked” to “this function is allocating all the memory” in one click.
We also export Prometheus metrics from the profiling data. These are monotonic counters accumulated from the parsed profiles, so <code>rate()</code> gives you allocation rate and contention rate over time. We initially tried getting allocation metrics from <code>hsperfdata (sun.gc.tlab.alloc)</code>, but those are per-GC-cycle snapshots that reset every collection. The async-profiler data is the real thing.
<h1>Seeing it in action</h1>
Enough theory. Let’s break something and see how the profiling data helps us find the root cause.
We have a demo environment with several microservices. The one we’ll focus on is order-service, a Spring Boot app running on JDK 21, backed by MySQL. It handles order creation, listing, and payment processing. Normal latency is under 10ms.
The demo has a built-in chaos controller that lets us inject failures via a REST API. We’ll use two scenarios: lock contention and memory allocation pressure.
<h1>Lock contention</h1>
For this scenario, the chaos controller spawns background threads that repeatedly acquire a shared lock and hold it for 5ms:
<pre><code>private void startLockContention() {
 int count = Runtime.getRuntime().availableProcessors();
 for (int i = 0; i &lt; count; i++) {
 Thread t = new Thread(() -&gt; {
 while (!Thread.currentThread().isInterrupted()) {
 synchronized (CONTENTION_LOCK) {
 try {
 Thread.sleep(5);
 } catch (InterruptedException e) {
 Thread.currentThread().interrupt();
 break;
 }
 }
 }
 }, &quot;chaos-lock-holder-&quot; + i);
 t.setDaemon(true);
 t.start();
 }
}
</code></pre>
Meanwhile, every incoming request also tries to acquire the same lock in the request interceptor:
<pre><code>if (chaosConfig.isLockContention()) {
 synchronized (ChaosController.CONTENTION_LOCK) {
 // Request thread blocks here while holder threads occupy the lock
 }
}
</code></pre>
After enabling this scenario, we can immediately see the impact on the order-service SLIs. The latency heatmap shows a clear shift, requests that used to complete in under 10ms are now taking 100ms+, with some exceeding a second:
<img src="https://media.daily.dev/image/fetch/s--sD2TnUfM--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u47w26cliy0d9j2brund.png?_a=BAMAMiWQ0" alt="Image description">
The request rate chart confirms the degradation, you can see the latency distribution shifting from green (fast) to red (slow):
<img src="https://media.daily.dev/image/fetch/s--ab8OChVU--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xpjf9wxgucrbi1qleah3.png?_a=BAMAMiWQ0" alt="Image description">
Now let’s look at the JVM report. The lock contention chart shows a clear spike, the lock wait time jumps from near zero to significant values:
<img src="https://media.daily.dev/image/fetch/s--1Kb1Xc1M--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8coord8qsamof65wqho0.png?_a=BAMAMiWQ0" alt="Image description">
Let’s click the profile button on the lock contention chart to open the flamegraph:
<img src="https://media.daily.dev/image/fetch/s--ZWkx-rnX--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fe7t50u0wplkjbryejtl.png?_a=BAMAMiWQ0" alt="Image description">
The flamegraph shows the Java Lock (delay) profile in comparison mode. Red means “more time waiting for locks than before.” Reading from top to bottom, we can see the Spring request processing chain, and at the bottom of the flamegraph, our ‘ChaosInterceptor.preHandle’ method, the one that tries to acquire the shared lock. That’s the bottleneck.
Without profiling, all we’d know is “requests are slow.” With the lock profile, we can point at the exact monitor and the exact code paths waiting for it.
<h1>Memory allocation pressure</h1>
The demo also supports a GC pressure scenario. It starts a background thread that continuously allocates and discards 256 MB byte arrays:
<pre><code>private void startGcPressure(int megabytes) {
 Thread t = new Thread(() -&gt; {
 while (!Thread.currentThread().isInterrupted()) {
 byte[] garbage = new byte[megabytes * 1024 * 1024];
 garbage[0] = 1; // prevent dead-code elimination
 }
 }, &quot;chaos-gc-pressure&quot;);
 t.setDaemon(true);
 t.start();
}
</code></pre>
The JVM is configured with <code>-Xmx=512m</code> so allocating 256 MB chunks means the GC runs after almost every allocation.
After enabling this scenario, the JVM report tells the story. The allocation rate chart spikes from near zero to ~3 GB/s. GC time jumps in lockstep, the young collection pauses go from occasional to constant:
<img src="https://media.daily.dev/image/fetch/s--1AWAvZPg--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r5e128uro59vpwhaur6l.png?_a=BAMAMiWQ0" alt="Image description">
Now let’s click the profile button on the allocation rate chart to see what is allocating all this memory:
<img src="https://media.daily.dev/image/fetch/s--NcvYxjhm--/f_auto,q_auto/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/taivr103q3f1jugfa4yu.png?_a=BAMAMiWQ0" alt="Image description">
The flamegraph shows the Java Memory (alloc_space) profile in comparison mode. At the bottom we can see <code>ChaosController$$Lambda.run</code> and <code>startGcPressure</code> marked as +100%, they didn’t exist in the baseline period. The top-level <code>Thread.run</code> frames confirm this is a background thread, not request processing.
Without profiling, all we’d see is GC time going up. With the allocation profile, we know exactly which code is responsible.
Enable it with a single environment variable and you get flamegraphs, time-series metrics, and a direct link between “something changed” and “here’s the code.” You can install it open source on your system <a href="https://github.com/coroot/coroot" target="_blank" rel="noopener nofollow">here. </a>

The squad for Kubernetes afficionados. 
All things Kubernetes: best practices, tools, tricks, new features, operator building. 
For experienced Kubernetes administrators and Kube-newbies alike.


KubeSquad

Coroot extended its eBPF-based Java profiling with async-profiler support to capture memory allocation and lock contention profiles. The integration works by detecting HotSpot JVMs, deploying libasyncProfiler.so into the container filesystem, and loading it via the JVM Attach API — requiring zero code changes or JVM restarts. Every 60 seconds, the agent stops the profiler, reads the JFR file via /proc, and restarts recording. The post demonstrates the feature with two chaos scenarios on a Spring Boot app: injected lock contention causing 100ms+ latency spikes, and a GC pressure scenario allocating 256MB arrays at ~3GB/s. In both cases, flamegraphs pinpoint the exact code responsible, going from 'something is slow' to 'this specific method is the culprit' in one click.

Profiling Java apps: breaking things to prove it works