JFR as the Default: Enabling Continuous Flight Recording Safely

Why continuous recording

A safe baseline configuration

-XX:FlightRecorderOptions=repository=/var/tmp/jfr
-XX:StartFlightRecording=name=continuous,delay=2m,disk=true,maxage=30m,maxsize=250m,settings=default,dumponexit=true,filename=/var/tmp/jfr/exit.jfr

• delay=2m waits until the application has started and warmed up before recording, so startup noise does not fill your window.
• disk=true writes data to disk, which allows long windows without holding everything in memory.
• maxage=30m and maxsize=250m bound the recording. The data is discarded when either limit is hit, which is what keeps disk use predictable.
• settings=default selects the standard event template, which is designed to be low overhead. Use the heavier profile template only for short, targeted sessions.
• dumponexit=true saves the recording when the JVM exits, which helps with crashes and deliberate restarts.

Capturing data during an incident

jcmd <pid> JFR.dump name=continuous filename=/var/tmp/jfr/incident-$(date +%s).jfr

Defaults per service class

Analyze a recording in Java

import jdk.jfr.consumer.RecordedEvent;
import jdk.jfr.consumer.RecordedFrame;
import jdk.jfr.consumer.RecordedStackTrace;
import jdk.jfr.consumer.RecordingFile;

import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.Path;
import java.util.*;

public final class JfrAllocationFolder {

    public static void main(String[] args) throws IOException {
        Path jfr = Path.of(args[0]);
        Path out = Path.of(args[1]);

        Map<String, Long> bytesByStack = new HashMap<>();

        for (RecordedEvent event : RecordingFile.readAllEvents(jfr)) {
            if (!"jdk.ObjectAllocationSample".equals(event.getEventType().getName())) {
                continue;
            }
            RecordedStackTrace trace = event.getStackTrace();
            if (trace == null) {
                continue;
            }
            List<RecordedFrame> frames = trace.getFrames();
            List<String> names = new ArrayList<>();
            for (int i = frames.size() - 1; i >= 0; i--) {         // root first
                var method = frames.get(i).getMethod();
                names.add(method.getType().getName() + "." + method.getName());
            }
            String key = String.join(";", names);
            bytesByStack.merge(key, event.getLong("weight"), Long::sum);
        }

        try (PrintWriter w = new PrintWriter(out.toFile())) {
            bytesByStack.forEach((stack, bytes) -> w.println(stack + " " + bytes));
        }
    }
}

Gate your visualizations

Failure modes

• Oversized recordings. Always set maxage and maxsize.
• Mismatched JDK versions. A recording from one JDK may not parse with a very different tool version. Keep your analysis tools at or above the JDK you run.
• Using the heavy template by default. Reserve profile for short investigations.
• No access plan. Decide in advance who can dump recordings and where they are stored, because recordings can contain sensitive data such as stack traces and thread names.

Demo to build

Takeaways

• Continuous JFR is safe when it is bounded by age and size.
• Dump on demand with jcmd and analyze with the consumer API or a flame graph.
• Verify that data exists before visualizing it.