Throughput vs latency: how they interact under load

Throughput is how much work a system finishes per unit of time, such as requests per second. Latency is how long one piece of work takes, such as 120 milliseconds for one request. Under light load the two barely affect each other; as load approaches the system’s capacity, throughput stops rising and latency starts to climb, and finding that point is what a load test is for.

The worked example below is arithmetic with no test run behind it. Tool definitions come from each project’s documentation, checked on 16 October 2026.

What is the difference between throughput and latency?

Throughput counts completed work over time; latency measures the time one unit of work takes. A system can have low latency and low throughput, or high throughput and high latency, so you need both numbers to describe it.

JMeter’s glossary calculates throughput as the number of requests divided by the total time, from the start of the first sample to the end of the last, including the gaps between samples, “as it is supposed to represent the load on the server” (checked on 16 October 2026). That makes throughput a property of the whole run, while latency belongs to each request and is best read as a distribution: median, p95, p99.

Is bandwidth the same as throughput?

No. Bandwidth is the most data a network link could carry; throughput is what actually gets through. In a load test, throughput usually means completed requests, and bandwidth matters only when large payloads fill the link before the application runs out of capacity.

AWS’s comparison page puts the network view simply: latency is the delay in network communication, throughput is the average volume of data that actually passes over a period, and bandwidth is the theoretical maximum (checked on 16 October 2026). Much of what you find online uses that network sense. Load testing uses the same words for requests rather than packets, which is why it pays to say which one a report means.

What does “latency” mean in a load testing tool?

It depends on the tool. In JMeter, Latency is the time to the first part of the response, while the full response time is called Elapsed time. In k6, http_req_duration covers sending, waiting and receiving, and the time to first byte is a separate metric.

The JMeter glossary measures elapsed time “from just before sending the request to just after the last response has been received” and latency to “just after the first response has been received”. The k6 metrics reference defines http_req_duration as http_req_sending + http_req_waiting + http_req_receiving, without DNS lookup or connection time, and calls http_req_waiting the time to first byte (both checked on 16 October 2026). So JMeter’s Latency lines up with k6’s http_req_waiting, not with http_req_duration. Compare a JMeter “latency” with a k6 “duration” and the JMeter number will look better without being better.

How do throughput and latency change as load grows?

At first, more load means more throughput and latency stays roughly flat. Once the system reaches its capacity, throughput stops growing and every extra user only adds waiting, so latency rises with the load.

A worked example, arithmetic only. A service handles at most 200 requests per second and answers in 0.1 seconds when it is not busy. Each virtual user sends one request, then pauses for 1 second. With a fixed number of users, the closed-model form of Little’s law ties them together: users = requests per second × (latency + think time), as our Little’s law article shows step by step.

Up to 220 users, throughput grows in step with them. After that it is pinned at 200, so latency has to absorb the rest: 300 users need 300 ÷ 200 = 1.5 seconds per loop, which leaves 0.5 seconds of latency. Real systems typically bend more gradually, because queues start to form before the exact ceiling. With an arrival-rate test the picture is harsher: if requests arrive faster than 200 per second, the backlog keeps growing for as long as the excess lasts, which is the difference the open vs closed workload model article explains.

How do you find the knee in your own results?

Raise the load in steps, hold each step long enough to settle, and record throughput and a latency percentile for each one. The knee is the first step where throughput stops rising with the load while latency starts to climb.

A few things make that reading reliable:

• Hold each step for a few minutes so throughput is measured on a steady stretch, not a ramp.
• Read a percentile, not only the average; an average can stay calm while the slowest few percent of requests get much slower.
• Check errors at every step. Fast failures count as completed requests with short times, so a failing system can show rising throughput and falling latency.
• Watch the load generator. If its CPU is saturated, the ceiling you found may be its own.

LoadFocus’s response time vs virtual users chart plots average and maximum response time against active virtual users, one point per user level, which is where the knee shows up in a stepped test.

Can you raise throughput without raising latency?

Sometimes. Adding capacity moves the ceiling, so latency stays flat for longer. Batching usually trades the other way: it raises throughput by making individual requests wait.

TCP’s Nagle algorithm is the classic example of that trade. RFC 896 holds back new segments while earlier data on the connection is still unacknowledged, so the network carries fewer, fuller packets and each small write may wait (checked on 16 October 2026). Request batching and write buffering work the same way: more done per second, more time per item. When you change one of them, rerun the stepped test and compare both curves, not just one number.

Which one belongs in your pass/fail criteria?

Both, tied together. A latency target means little without the throughput it was measured at, and a throughput target means little if latency doubled to reach it. State them as a pair, such as p95 under 300 milliseconds at 200 requests per second with under 1% errors.

Our guide to load test pass/fail criteria shows how to write and automate that pair. If your tool waits for slow responses before sending the next request, its latency figures can understate the problem; coordinated omission explains how that happens and how to avoid it. LoadFocus’s glossary entry on latency breaks down where latency comes from and why percentiles matter.

FAQ

Does low latency mean high throughput? Not necessarily. A system can answer each request quickly but handle few at once, or handle many at once while each one waits. You only know both after measuring both under the same load, ideally across several load levels.

What is the difference between latency and response time? In everyday use they often mean the same thing. In JMeter, Latency is the time to the first part of the response and Elapsed time is the full response time, so check which one a report shows before comparing tools.

Why does throughput stop increasing in a load test? Something has reached its limit: CPU, a database, a connection pool, a rate limit, or the load generator itself. From that point extra users only queue, which shows up as rising latency at flat throughput.

Related reading

• Little’s law in load testing: size and check your tests
• Open vs closed workload model: which one should you use?
• Coordinated omission: what it is and how to avoid it
• Load test pass/fail criteria: how to set them
• Tail latency: what it is and how to measure it
• Apdex score: what it is and how to calculate it
• Amdahl’s law: formula, examples and scaling limits

The technical claims in this article have not been independently reviewed.