Load test pass/fail criteria: how to set them

Load test pass/fail criteria are the few limits a run must meet to count as a pass: a latency percentile for the requests users wait on, an error rate and a minimum request rate, all at a stated load. Take the limits from a service-level objective (SLO), add regression limits from earlier runs inside them, and let the tool’s exit code fail the run only on those hard limits. Everything else is a warning that a person reads.

Tool facts were checked in each project’s own docs on 9 October 2026, and every limit shown is an example.

What makes a load test pass or fail?

A run passes when every hard limit holds at the planned load and fails when one is missed. It is invalid when the test itself did not do its job, whatever the numbers say.

A saturated load generator, a load that never reached its target or a failing script all produce numbers about the test setup. LoadFocus, for example, shows a load generator saturation warning when an engine peaked at or above 80% CPU. k6’s exit codes separate the cases for CI: 99 means thresholds failed, 107 a script exception (checked on 9 October 2026).

How do you turn an SLO into load test criteria?

Restate each SLO that load can break, usually latency and availability, as a limit on what the test measures, and make it stricter than the SLO so a pass leaves room.

Google’s SRE book defines an SLO as a target value or range for a service level measured by an indicator (SLI), with examples such as 99% of Get calls completing in under 100 ms. It wants an SLO to state how it is measured and when it is valid, and recommends a safety margin: an internal target tighter than the one users see.

For a load test, “when it is valid” means the load and request mix: “95% of checkout calls under 500 ms” might become “p95 of checkout under 400 ms at 300 requests per second for 15 minutes”, as an example. With no SLO yet, agree a provisional one with the product owner.

Which metrics should the criteria cover?

Latency for the requests users wait on, an error rate, the request rate achieved and, mostly as warnings, server resources.

• Latency per request type. A global percentile mixes a fast health check with a slow search. In k6, tagged requests create sub-metrics with their own limits; Gatling’s details() targets one request or group. A test that slowed its own load down reports flattering percentiles (coordinated omission).
• Errors, counted broadly. A fast 200 with an error page is still a failure, so assert on content. A failed JMeter Response Assertion marks its sample unsuccessful; k6 checks reach the exit code only through a threshold.
• A request-rate floor. In a closed workload model a slow system receives fewer requests, so a run can meet its latency limits at a fraction of the planned load (open vs closed workload model).
• Server resources. CPU, memory and pool usage come from your monitoring. Treat them as warnings unless a capacity plan sets a ceiling.

How do you pick the numbers?

Take the ceiling from the SLO and the regression limit from earlier runs of the same test. The SLO says what users tolerate; earlier runs say what this build normally does.

The SRE book warns against setting an SLO from current performance. A regression limit exists to notice change, so current performance is exactly its input. Run the unchanged build several times with the same environment and load, note how far each metric moves, and set the limit above the worst run with some headroom, still under the SLO-derived ceiling where there is one.

If runs of one build vary by more than the regression you want to catch, the test is too noisy to gate on: lengthen the steady phase or isolate the environment first. Record new reference runs after any deliberate change.

Which criteria should fail the build, and which should only warn?

Fail the build on limits that come from an SLO or check correctness. Warn on tail percentiles, resource use and slow drifts, which need judgment; a gate that fails on noise soon gets switched off.

Artillery’s ensure plugin has a warning tier: a condition with strict: false is optional, so a miss leaves the exit code alone (checked on 9 October 2026). In k6, a check without a threshold never fails the run. Elsewhere, keep warnings out of the assertions.

config:
  plugins:
    ensure:
      thresholds:
        - "http.response_time.p95": 400
      conditions:
        - expression: "http.response_time.p99 < 1200"
          strict: false

When should a load test stop early?

When carrying on can only do harm or prove nothing: errors far past the limit, a system already down or a generator that cannot keep up. Give the rule a grace period so start-up noise cannot trip it.

Make abort rules looser than pass limits: they protect a shared environment. k6’s abortOnFail stops the test when a threshold fails, after an optional delayAbortEval. Locust’s extension docs show a background greenlet that quits once the failure ratio passes a threshold. Gatling’s crashLoadGeneratorIf stops the generator with a failure code when a condition in a user’s session holds. A JMeter Thread Group can stop the test on a sampler error, which suits a correctness check (all checked on 9 October 2026).

How does each tool encode the criteria?

k6 and Artillery turn a missed hard limit into a non-zero exit code, Locust does so for any failed request or for limits you code yourself, and Gatling’s assertions fail the build. JMeter leaves the verdict to a step after the run.

k6 thresholds sit in the script’s options. This example fails on the p95 of requests tagged name: checkout or a 0.5% error rate, and aborts after a minute if errors pass 5%:

export const options = {
  thresholds: {
    "http_req_duration{name:checkout}": ["p(95)<400"],
    http_req_failed: [
      "rate<0.005",
      { threshold: "rate<0.05", abortOnFail: true,
        delayAbortEval: "1m" },
    ],
  },
};

Gatling assertions combine a scope (global, forAll, details), a statistic such as a percentile or requestsPerSec, and a condition. They are evaluated after the run, and one failure fails the simulation.

Locust exits with code 1 when any request failed, and --exit-code-on-error changes that code. For a ratio or a percentile, set environment.process_exit_code in a quitting listener, which Locust’s source honours first.

JMeter assertions judge single samples, and its command-line options include no run-level limit. CI decides: a short script over the results file, as in our GitHub Actions guide, or the third-party jmeter-maven-plugin, whose results goal fails the build on any failed sample unless errorRateThresholdInPercent sets a rate (checked on 9 October 2026).

LoadFocus grades runs against pass/fail thresholds saved on the test: maximum p95 and p99 response times, a maximum error rate and a minimum request rate, each optional. Each run gets a PASS or FAIL badge with actual and target values, and an API hands the verdict to a pipeline. LoadFocus runs k6 arrival-rate scenarios with a fixed number of users, so there a rate floor checks what those users achieved.

FAQ

Should the average response time be a pass/fail criterion? Rarely. The SRE book prefers percentiles: an average hides the long tail and can stay flat while the slowest requests get much slower.

How many pass/fail criteria should a load test have? A handful of hard limits, plus warnings. The SRE book advises having as few SLOs as possible and says that if quoting one can never win a conversation about priorities, it is probably not worth having. Apply the same rule to each load test limit.

Can a load test pass with some failed requests? Yes, if the error rate stays under its limit. Check your tool’s default: Locust fails the run on any failed request.

Related reading

• Load testing in GitHub Actions: a practical guide
• Coordinated omission: what it is and how to avoid it
• How to run a load test: a step-by-step guide
• Apdex score: what it is and how to calculate it
• SLO vs SLA: the difference, SLIs and error budgets

The technical claims in this article have not been independently reviewed.