Looking at Java 21: Generational ZGC

2023-09-12 · 9 min

Java is (in)famous for its Garbage Collection. It’s one of its main strengths, but it can also be a source of many headaches. An additional garbage collector was introduced in Java 11 (JEP 333), a low latency/high scalability GC called ZGC. And now, with Java 21, it has evolved into a generational GC (JEP 439).

This quite technical article ended up being more about garbage collection in than the generational ZGC. You can skip all the other content and go right to ZGC if you like.

Table of Contents

Taking Out the Trash

The concept of “garbage collection” is, in essence, about automatic memory management. Each time we create a new data structure, the runtime requires memory to do so. If we no longer need an object or leave a stackframe and its variables go out of scope, someone has to clean up the related memory, the “garbage”, or we will soon learn about Java’s OutOfMemoryError.

The general downside of GC is that it takes time to clean and rearrange memory. This introduces certain runtime overhead outside of our direct control, as the actual point of the GC runs is usually non-deterministic and not triggered manually. Especially high-throughput large and memory-hungry applications scaled over multiple threads can suffer from long “GC pauses”. Ironically, the non-deterministic nature of GCs is also one of their upsides, as we don’t have to worry about when or how the memory is freed, it will just happen.

There are many different options on how to manage memory, and most times, a language forces you to adhere to its preferred way of doing things. Looking at different languages will give us a better understanding of the available approaches.

How Languages Manage Memory

C/C++ uses manual memory management, giving you complete control over everything, including blowing up your app anytime or leaking all your memory over time.

Objective-C and Swift don’t use a garbage collector, either. However, the compiler uses “Automatic Reference Counting” to manage objects’ lifecycles. No GC means no background process and no GC pauses. It also means that the lifecycles must be clear at compile-time, which requires marking weak references yourself or risking deadlocks and memory leaks.

Rust uses an affine type system instead of a GC. That means a concept of explicit “ownership” defines the lifetimes of any value. It boils down to each value requiring an owner and only one of them. If a value goes out of scope, it’s discarded. As the type system represents the ownerships, the so-called borrow checker can validate the correctness of your lifetimes. This creates a safe and performant environment, which led to Rust’s gain in popularity for many types of apps. However, “fighting” the borrow checker isn’t much fun at all, and the concept of ownership, borrowing references, etc., is quite unfamiliar to many developers, making the initial commitment needed to learn Rust an uphill battle.

Java, Python, Go, C#, JavaScript, and many more languages use a GC to manage their memory. What kind of GC actually differs in many aspects.

.NET-based languages uses a “three-level generational mark and compact collection” algorithm.

Python combines reference counting and “three-level generation collection”.

The HotSpot JVM Garbage Collectors

As you can see, there are many different approaches to memory management, and there’s no “best” way to do it. Even in a single language/runtime, there can be more than one approach to garbage collection, and the JVM is a great example of that.

Instead of a singular GC, the HotSpot JVM has 5 to choose from:

Garbage-First (G1) Collector (the default since Java 9)
Serial Collector
Parallel Collector
Concurrent Mark Sweep (CMS) Collector (deprecated in Java 9)
Shenandoah GC (Java 12+)
Z Garbage Collector (available for production since Java 15)

Also, don’t forget that there are other JDK implementations out there!

Eclipse OpenJ9 uses a generational concurrent GC with multiple collection policies.
GraalVM has Epsilon GC, which is a No-Op GC, doing no memory cleanup at all.
and many more…

How to Choose Your JVM GC

Many languages provide only a single GC with a few configuration and optimization options, so why does Java have 5 of them? Well, it all depends on your application’s tolerance to “stop-the-world” events and overall pause times.

Stopping the Whole World

For a GC to do its job, it needs to check the heap for trash. To make sure that no new objects are allocated or existing ones become unreachable while the GC is running. Freeing memory can lead to fragmentation, which is why the heap needs to get compacted by moving around an object’s memory and updating its reference. If you update the reference first and then move, the wrong memory is referenced until the data is actually moved, and vice versa. So, stopping the world ensures that no one accesses or changes memory during a GC run.

This is a quite simplified view of garbage collection, as it is a quite extensive and complicated topic where the inner details of each of them is out of the scope of this article. If you want to find out more about GCs in general, check out this WikiWikiWeb page that offers a lot of information and further reading.

How often such an STW event is required depends on the GC algorithm.

How To Choose Your JVM GC

GC algorithms mainly focus on three metrics:

Throughput: percentage of total time not spent GC-ing over long periods.
Latency: overall responsiveness of the application, which is affected by GC pauses.
Footprint: working set of a process, measured in pages and cache lines.

Like with many problems, you can’t optimize for all of them, so every GC needs to find a balance between them Here are some scenarios and their matching GCs as a starting point:

Garbage Collectors	Scenarios
Serial	Small data sets (~100 MB max) Limited resources (e.g., single core) Low pause times
Parallel	Peak performance on multi-core systems Well suited for high computational loads > 1-second pauses are acceptable
G1 CMS	Response time > throughput Large heap Pauses < 1 sec
Shenandoah	Minimize pause times Predicatable latencies
ZGC	Response time is high-priority, and/or Very large heap
Epsilon GC	Performance testing and troubleshooting

Each of them has its own advantages and disadvantages, depending highly on your application’s requirements and the available resources.

Well, that was quite a lot of words before coming to the actual topic of the article…

Generational ZGC

The generational hypothesis is the observation that younger objects are much more likely to “die young” than older objects, at least in most cases. That’s why handling objects differently based on their age can be beneficial. Improving the collection of younger objects requires fewer resources and yields more memory.

Even without handling generations, ZGC is quite an improvement in GC pause times, reclaiming memory faster than concurrent threads consume them, at least if enough resources are available. However, all objects are stored regardless of their age, and all of them have to be checked when the GC runs.

With Java 21, the ZGC splits the heap into two logical generations: one for recently allocated objects and another for long-lived objects. The GC can focus on collecting younger and more promising objects more often without increasing pause time, keeping them under 1 millisecond.

The generational ZGC should give us

Lower risks of allocation stalls
Lower required heap overhead
Lower GC CPU impact

All these benefits should come without a significant decrease in throughput compared to non-generational ZGC. Also, there shouldn’t be a need to configure the size of the generations, threads used by the GC, or the age limit ourselves.

How to Use Generational ZGC

In typical Java fashion, the new ZGC isn’t immediately forced up on us as soon as it’s available. Instead, it will be available alongside its non-generational predecessor. You can configure which to use with java arguments:

shell

# Enable ZGC (defaults to non-generational)
$ java -XX:+UseZGC

# Use Generational ZGC
$ java -XX:+UseZGC -XX:+ZGenerational

Be aware that the Generation ZGC is supposed to replace its predecessor over time and become the default ZGC. At this point, you can turn it off with the antagonistic argument to previously enabling it, by replacing + (plus) with - (minus/dash):

shell

# Do not use Generational ZGC
$ java -XX:+UseZGC -XX:-ZGenerational

It’s also planned to remove the non-generational ZGC completely in an even later release.

Conclusion

Even though I’m personally quite excited about such technical details of the JVM, I also know that most Java developers won’t have to think much about the inner workings of a GC or fine-tune it to match their application’s profile. Nevertheless, the steady improvements will bring benefits to each Java developer.

The ZGC improves its already excellent “out-of-the-box” experience with each release and drives the platform’s performance forward in significant steps. The addition of generations will make it even more versatile, and benchmarks are pretty impressive so far. For example, Apache Cassandra, an open-source NoSQL database, achieves 4 times the throughput with only a quarter of the heap size, compared to the non-generational ZGC, and still under 1 millisecond pause-times.

However, it also introduces more complexity and even performance degradation in certain applications. If your workload is “non-generational” by nature, a “generational” GC can’t utilize its full potential. However, the JDK team is confident that only a small set of workloads would be affected by this, and does not justify keeping the non-generational ZGC around forever. And looking at the history of the ZGC and its constant stream of GC improvements, it can and will only become better over time.

Resources

Looking at Java 21

#java #looking-at-java-21

Support Me on Ko-fi