Several cloud cost optimization solutions are today available both by Cloud Providers, such as AWS Compute Optimizer or Google machine type recommendations, and by specialized COTS vendors. These tools may help you choosing the right cloud instance and volume sizes, allocating resources in order to minimize costs over time, and identify the most cost-effective option among multiple Cloud Providers.
Unfortunately, these tools do not ensure that your end-to-end application performance and resilience will be positively affected by recommended changes. They only rely on historical infrastructure metrics (like CPU utilization) and predefined best practices which do not take into account your specific application architecture and scalability. Such characteristics play an important role on how the real application performance will be impacted by changes in the underlying infrastructure, such as compute (e.g. moving to an instance type with a new cheap but lower power AMD processor vs a costly Intel) or storage (e.g. AWS gp2/gp3 vs io1/io2).
Balancing application performance and cost on the cloud is a daunting task. Today even more so, in light of the growing number of instance types available by Cloud Providers and faster release cycles which may prevent application teams from properly optimizing cloud configurations for their applications and thus ensuring that SLOs are matched. However, as we show below, achieving both maximum application performance while also minimizing cloud costs is not an impossible dream.
The approach we advocate for does not simply provide some general recommendations based on historical utilization, but delivers optimal cloud configurations that reduce costs and have been validated with real performance experiments against your actual application.
Choosing an EC2 instance type: a world of opportunities, and unexpected costs
Today’s cloud offerings provide plenty of choices. For example, the following picture from an AWS presentation lists 300+ available EC2 instance types, each one tailored “for virtually every workload or business need”.
Thus, choosing the right instance translates into selecting the underlying processors and architectures (e.g. Intel Xeon, AMD EPYC, AWS Graviton, etc), how much CPU and memory is needed, which storage (e.g. HDD – either cold or throughput optimized, SSD either general-purpose or provisioned IOPS), not Choosing the right instance therefore means selecting the underlying processors and architectures (for example, Intel Xeon, AMD EPYC, or AWS Graviton), how much CPU and memory you need, and which storage to use (for example, HDD, either cold or throughput-optimized; SSD, either general-purpose or provisioned IOPS), not counting special-purpose hardware and networking such as GPUs and FPGAs.
This choice is further complicated because compute instance and storage types are interdependent. If you need higher storage performance, buying a more powerful volume may not be enough, because your IOPS or bandwidth can be capped by your instance size. You may need a bigger instance even when the extra compute is not required.
So cloud teams have plenty of choices when building their infrastructure. The impact of those choices on the cloud bill can be significant. For example, the cost of EC2 instances ranges from a few dollars to tens of thousands of dollars per month on demand, while EBS ranges from a dozen dollars to hundreds of dollars per 1 TB a month for provisioned IOPS.
The driving factor in selecting the best infrastructure is your applications and their specific characteristics, like architecture and scalability, as well as the required performance (throughput and latency) and availability. For example: what is the cheapest EC2 instance that can sustain a peak load of 1,000 payments per second while still processing each payment within one second? Should you go with blazing-fast (and expensive) Intel Xeon processors, or are cheaper AMD alternatives enough? And how much memory and IOPS do you really need?
Current tools cannot answer these questions, because their recommendations rely only on measured resource utilization and on more or less sophisticated models. For example, Amazon recently released an enhanced version of AWS Compute Optimizer that recommends EC2 instance types using machine learning to analyze historical utilization metrics such as CPU, memory, network, and disk.
Do these modeling approaches work when application performance is at stake? Generally, no. Even the most sophisticated modeling techniques cannot identify hidden bottlenecks or capture unique application and workload behavior.
The cloud providers confirm this themselves. See, for example, the official AWS EC2 documentation:
There is no substitute for measuring the performance of your entire application, because application performance can be impacted by the underlying infrastructure or by software and architectural limitations. We recommend application-level testing, including the use of application profiling and load testing tools and services.
So when cloud cost and service quality are both critical, performance tuning is a must. Yet the sheer complexity of cloud environments and the accelerated pace of application releases make any manual approach infeasible. That is why an entirely different approach is required.
Performance-aware EC2 cost optimization is not a dream
To illustrate how AI-driven cloud optimization works, we take an application based on a MongoDB database hosted on AWS. Here the customer goal was to maximize cost efficiency: find the AWS compute and storage types that minimize the price/performance ratio, where
- Price is the monthly cost of the cloud resources;
- Performance is the maximum application throughput (queries/sec) achievable with a given AWS EC2 and EBS configuration.
To meet this requirement, we used the Akamas platform to automatically drive a series of experiments guided by AI-based optimization against custom-defined goals. Akamas native integration with AWS pricing APIs also lets the optimization goal be formulated directly in terms of the cost associated with each instance type and each EC2 and EBS option. This is the same performance-validated approach behind the Akamas Cloud Instances solution, driven by autonomous performance experiments.
In each experiment, a specific AWS EC2 and EBS configuration, identified by the AI engine as a good candidate for improving cost efficiency, was automatically provisioned and tested. The real application throughput (queries/sec) was measured from the load testing tool.
The initial (baseline) configuration for the database was an r4.large EC2 instance with a gp2 70 GB EBS volume, a common choice for memory-intensive database workloads (see AWS recommendations).
The optimization study produced good results quickly. After only 18 experiments, in approximately 22 hours, it identified a configuration providing a 68% improvement in cost/performance over the baseline.
This configuration has a similar cost to the baseline (it is just 3% cheaper) but triples the performance (+205% throughput). It also reduces latency (-90%), an interesting and unexpected result, since latency was not part of the original goal.
To understand these results, let us compare the best configuration with the baseline.
The instance type in the best configuration is m5d.large, which is surprising: M5 is a general-purpose family, while the baseline r4.large is memory-optimized and should, in theory, suit a database better.
But there is more. The best configuration uses only half the memory, which shows that extra memory was not helping database throughput.
What about storage? The best configuration uses locally attached storage, which helps bring the cost down. However, that may not be acceptable, because losing the instance would also mean losing the database and all its data. EBS storage, which is replicated across Availability Zones, is typically recommended for databases. This constraint could have been set from the start, but in this first study we chose not to restrict anything, so we could evaluate every option.
Even when we restrict the study to EBS storage only, we still beat the baseline. An m5a.large configuration is both cheaper (-24%, thanks to an AMD processor) and faster (+12% throughput).
The following table summarizes the results of the optimization study.
This case shows that it is possible to reduce the cloud bill while maintaining, and even improving, application performance.
Conclusions
Cloud optimization should not focus only on reducing the bill. It should also account for end-to-end application performance.
Most available cloud optimization tools cannot support this. They rely on historical infrastructure utilization and on modeling techniques that may fail to find hidden bottlenecks or capture unique application and workload behavior.
The approach shown here uses specialized AI to explore thousands of configurations, including the hundreds tied to cloud instance types, in just a few hours. With Akamas, performance engineers can identify the optimal configuration for their specific cost and performance goals and constraints, such as SLOs.
Keep reading our blog for more real-world stories on how to optimize cloud-based and other applications.
FAQs
What is EC2 cost optimization?
EC2 cost optimization is the practice of running each workload on the lowest-cost instance and storage that still meet its performance and availability targets. It spans instance family, size, processor architecture, and EBS volume type. Done well, it lowers the bill without breaching SLOs.
Can you cut EC2 costs without hurting application performance?
Yes. In the AWS study above, Akamas found a configuration with a cost similar to the baseline that tripled throughput (+205%). A second, EBS-only configuration was 24% cheaper and still 12% faster. Performance-aware optimization can improve cost and performance together.
Why don’t tools like AWS Compute Optimizer protect application performance?
Tools such as AWS Compute Optimizer recommend instances from historical utilization metrics like CPU and memory. They do not run your application, so they miss hidden bottlenecks and workload-specific behavior. AWS itself recommends application-level load testing to confirm real performance.
Is a memory-optimized instance always best for a database?
No. In this study, a general-purpose m5d.large beat the memory-optimized r4.large baseline, using half the memory while delivering higher throughput. The right instance depends on how your workload actually uses resources, not on the instance category label.
