Load Testing and Tuning Inference with Gatling and Akamas
Serving an LLM is not like serving a web API. A single request holds a GPU for seconds, the response arrives token by token, and the same deployment looks fast or slow depending on prompt length and concurrency. Averages hide all of it.
In this webinar with Heather Thacker, Developer Advocate at Gatling, and Stefano Cereda, Head of Research at Akamas, we show a practical workflow to measure and improve LLM serving performance.
Gatling covers how to design a realistic load test for a streaming endpoint: modeling prompt and output length, shaping concurrency, and capturing the metrics that matter, such as time to first token, inter-token latency, and throughput under load.
Akamas covers what to do with those results. A serving stack exposes dozens of interacting parameters, from batching and KV cache settings to quantization and instance type. We show how to search that space automatically, targeting a clear goal such as the lowest cost per million tokens that still meets your latency SLO.
Register for the webinar
Registration is free. Fill out the form to save your seat.