Developers and system owners usually take for granted that there are some intrinsic tradeoffs in the Java design. For example, it is commonly accepted that if you aim at reducing resource usage (e.g. CPU), you must accept some performance degradation. Or that, if you optimize for maximum throughput and lowest latency, then the resource footprint of your Java application (e.g. memory usage) grows.
The general assumption is that you cannot improve throughput, latency, and footprint of a Java application at the same time. The common belief is that the best you can achieve is to improve only two of these three metrics. This rule, which we refer to as the “JVM short blanket”, is considered to be always valid.
In practice, that assumption is wrong: with the right JVM configuration you can improve throughput, latency, and footprint at the same time. In this post we show the results of real-world JVM optimization studies we have conducted, and how the short-blanket rule breaks once you explore the full JVM configuration space.
The JVM short blanket: throughput, response time and cost.
To describe the “JVM short blanket”, we will use the following statement by Charlie Hunt, the world-renowned Java performance expert, taken from his presentation on “The Fundamentals of GC Performance” at the GOTO Chicago conference in 2014:
Improving one or two of these performance attributes, (throughput, latency or footprint) results in sacrificing some performance in the other. Improving all three performance attributes usually requires a lot of non-trivial development work.”
Charlie Hunt, JVM Engineer at Oracle
It is worth noting that he used a “three-legged stool” metaphor, while others have used the “performance triangle” metaphor to illustrate the same concept.
Before further analyzing this statement, let’s review the three metrics that represent the apexes of the “JVM short blanket”:
- Footprint: the amount of resources (memory and CPU) the JVM needs to run.
- Throughput: the number of operations that your application can complete in a certain period of time.
- Latency: the time a single operation takes to be processed.
Intuitively, if you consider how Garbage Collection (GC) operates, this makes perfect sense. If you want high throughput and low latency, the GC will need more memory (and possibly CPU cycles) to make room for application allocations without stopping it too often. On the other hand, if you want a low footprint and low latency, you must accept lower throughput. The GC then has to stop the application very frequently, which hits throughput hard.
Therefore, it may seem that there is no way to improve throughput, latency, and footprint at the same time, at least without “some non-trivial development work”. However, we will show that it is actually possible to improve all three main Java performance metrics when a different approach to performance tuning is taken.
Achieving higher throughput, lower response time, and smaller footprint
As stated by another leading Java performance expert, Scott Oaks, author of the well-known “Java Performance: The Definitive Guide” book:
The JVM is highly configurable with literally hundreds of command-line options and switches. These switches provide performance engineers a gold mine of possibilities to explore in the pursuit of the optimal configuration for a given workload on a given platform.”
Scott Oaks, Architect at Oracle
The key to unlocking JVM performance and efficiency lies in exploiting the incredibly vast optimization space that modern JVMs offer.
A striking example of improving throughput, latency, and footprint all at the same time comes from an enterprise Java application supporting the CRM service of a leading Telco.
We performed an optimization study with Akamas, our AI-powered solution, by setting the goal of minimizing the footprint of the application (JVM heap size). The customer did not want to make the application slower, so we included constraints to maintain the current throughput and response time, with a maximum degradation of 5% allowed. We used our Java OpenJDK Optimization Pack to quickly tune the key parameters among the hundreds the JVM offers.
Akamas identified an optimal configuration in just 28 fully automated experiments run in about 20 hours, cutting the heap size by 85%: from 1.7 GB to 252 MB (see Figure 1).
The result of this optimization study was considered very satisfying by the customer, as it provided the opportunity to safely reduce the heap size while preserving the defined SLOs.
Besides reducing the memory footprint, this configuration is also an interesting outcome for the “JVM short blanket” discussion. The same optimal configuration increased throughput by 23% and reduced response time by 22% on average and 17% at the 90th percentile (see Figures 2 to 4). All these numbers come from the same Akamas optimization study on the Telco CRM application.
Let’s summarize the result of this optimization study. The JVM configuration identified by Akamas made the application run faster while also reducing the heap size by more than 85%. All three metrics (throughput, latency, footprint) improved at once, just by tuning JVM parameters well: an opportunity that would otherwise be left on the table.
Conclusions
In the Java performance world, the general rule is that you cannot improve application latency, throughput, and footprint at the same time without significant development work. This is what we call the “JVM short blanket”.
Modern JVMs offer a goldmine of 700+ configuration options that software and performance engineers can tune to reach optimal performance and efficiency. As we have seen, it is possible to improve all three key metrics at the same time.
Finding configurations that improve throughput, latency, and footprint together requires the ability to intelligently explore the vast space of application configurations. This is only made possible by AI techniques, such as the patented Reinforcement Learning behind Akamas, designed to solve optimization problems that are otherwise unmanageable even for the best human experts. The same approach extends beyond the JVM to other application runtimes and to Java workloads running on Kubernetes.
This is the second entry in a series on JVM performance tuning and the AI-powered approach to performance optimization. You can read the first entry here. Keep on reading us!
FAQs
Can you improve Java throughput, latency, and memory footprint at the same time?
Yes. The tradeoff is not a hard law of the JVM. In one Akamas study on a Telco CRM application, tuning JVM parameters cut heap size 85% while raising throughput 23% and lowering average response time 22%.
What is Java performance tuning?
Java performance tuning is the practice of adjusting JVM and application settings, such as heap sizing, garbage collector choice, and GC parameters, to improve throughput, response time, and resource usage for a given workload.
How do you improve the performance of a Java application?
Start by defining clear goals and SLOs, then explore the JVM configuration space rather than changing one flag at a time. Modern JVMs expose 700+ options, so an automated, AI-driven search finds better combinations faster than manual tuning.
What is the “JVM short blanket”?
The “JVM short blanket” is the common assumption that improving one or two of throughput, latency, and footprint always sacrifices the third. Real optimization studies show this assumption breaks once the full configuration space is explored.
How many configuration options does the JVM have?
Modern JVMs expose more than 700 command-line options and switches. Most are never touched, which is exactly why so much performance and efficiency is left unclaimed.
