In the August 2026 edition: pushing Qwen 3.5 to over a million tokens per second and the tuning lessons behind it, how OpenTelemetry now installs in a single command, and why graduated autonomy beats confidence in self-healing infrastructure. The Optimal is the Akamas newsletter where DevRel Engineer Graziano Casto breaks down Kubernetes performance, reliability, and cost optimization.
Hello everyone! Graziano here.
If you’re reading this, we’re officially closing out August 2026, which means half of you just got back from a summer that was 90% vacation and 10% checking Slack because a node pool decided to scale to infinity while you were at the beach. We’ve been busy optimizing clusters in the lab in the meantime. Grab a coffee and let’s look at the logs for this month.
The Latest Replica
From 9,500 to 1.1 Million Tokens Per Second
Federico Iezzi’s writeup of pushing Qwen 3.5 27B to over a million tokens per second on GKE with B200 GPUs is one of the most honest performance engineering pieces you’ll read this year, because it documents every wrong turn along the way. The first instinct, tensor parallelism across all 8 GPUs, was wrong: each GPU finished its slice of the model instantly and then sat waiting for the others to sync, the GPU equivalent of eight people ordering coffee and standing around while the espresso machine catches up. Switching to data parallelism helped, but then the KV cache stole the show: with a default 131K context window against 1,500-token prompts, the setup was over-allocating memory by roughly 85x. Tuning the context window down and switching to FP8 KV cache with speculative decoding is what actually got GPU utilization off the floor. The takeaway that matters beyond the benchmark: throwing more GPUs at a workload without touching the right knobs first, in this case parallelism strategy and context window sizing, just buys you idle silicon at a higher price point.
Read the full article here.
One Command, and OpenTelemetry Stpos Being a Chore
If you’ve ever had to instrument a Java, .NET, Node.js, or Python app running directly on a Linux host, you know the drill: download the agent by hand, wire up environment variables, hope you got the port right. OpenTelemetry’s new Packaging SIG just killed that ritual. You can now run apt install opentelemetry or dnf install opentelemetry and get the OpenTelemetry Injector plus auto-instrumentation for all four languages in one shot, no code changes, no redeploys. The Injector hooks into process startup and activates the matching instrumentation automatically, which sounds small until you remember how many “temporary” manual agent installs are quietly running in production right now because nobody wanted to touch a legacy host. It’s not signed yet and it’s hosted on GitHub Pages rather than production-grade infrastructure, so this is very much a “try it, don’t bet prod on it yet” moment. But the direction is the right one: observability should be a system dependency, not a snowflake install script someone wrote three years ago.
Read the full article here.
Confidence Is Not a Substitute for Safety
If you’ve ever watched an “AI-powered” remediation script make an incident worse, this piece on self-healing infrastructure will feel less like theory and more like therapy. The author’s framework for graduated autonomy splits remediation into three tiers: fully automated for boring, reversible, narrow-blast-radius actions like restarting one bad node; recommend-and-approve for anything riskier like a deployment rollback or database failover; and human-led for anything novel or ambiguous. The part that’s easy to skip past but shouldn’t be is the policy gate sitting between detection and execution, checking confidence, blast radius, redundancy, and recent failure history before anything gets to run unattended. The model proposes, the policy disposes, and only after months of shadow-mode agreement between the system’s recommendations and what engineers actually did does an action graduate into full automation. A good read for anyone who’s been asked to “just let the AI handle it” this year.
Commits From The Lab
The articles above all land on the same theme: tuning one layer while ignoring the rest just moves the waste somewhere else, whether that’s idle GPUs, a manual install script, or an autoscaler firing on bad signals. That’s exactly the gap we keep building Akamas to close.
Node pool recommendations just got smarter. Instead of comparing instance types on hourly price alone, Akamas now factors in how much work each core actually gets done, so a pricier instance that finishes the same job on fewer nodes can beat a “cheaper” one that just looks good on paper. You get a real performance-aware cost, not just a sticker price.
And on the .NET side, Akamas now catches GC stress and memory over-provisioning that container-level metrics alone would miss, and recommends right-sizing your container memory limits with the .NET runtime’s actual behavior factored in, not just what’s set at the container boundary.
Try Akamas Insights (Free Trial) to see what your own clusters are quietly overpaying for.
The Diff
Numbers beat opinions, so here is what optimization looked like in production this month.
A Fortune 500 European group, two Node.js microservices behind an internal workflow application: paying for an autoscaler that never stopped scaling.
The setup was a gateway service fronting a backend service, both tuned for cost with a simple constraint: don’t break anything that’s currently working. The instinct going in was that the fix would be smaller container limits. It wasn’t. The real problem was HPA configuration that kept both services scaling far past what traffic actually demanded, one workload was regularly running 80+ replicas when demand justified a fraction of that. Akamas found a configuration in under 10 experiments that already cut the cost score by 17%, and converged on a full solution across 39 experiments: HPA usage down 77%, with peak replica counts on the heaviest workload dropping from the high double digits into single digits. Global CPU allocation for the workload came down 14% and memory allocation dropped 56%, all while keeping used resources safely under requested ones, the opposite of where things started. Total cloud cost for the two services: down 29.8%. Setup took about three hours, the full study took four days, and the recommendations went straight to production without a rewrite.
The takeaway: the autoscaler wasn’t broken, it was just never told what “enough” looked like.
Catch Us
Want to meet the Akamas team in person? We’re regularly at conferences and meetups across the Kubernetes and Java ecosystems. See where we’ll be next on our Events page, and follow me on LinkedIn for the talks and sessions I’ll be at.
Keep those CPUs cool and your latencies low.
Stay optimized,
Graziano Casto, DevRel @ Akamas
