p95 latency, the value below which 95% of requests complete, is the default latency metric behind most automatic rollback setups, and it earns that default. It captures broad degradation without reacting to the isolated slow request every service has occasionally. That's exactly why it's the right starting point, and exactly why it's insufficient on its own for some services; more on that below.
Setting it up
A p95 latency guardrail needs a metric definition whose value represents that percentile. It's computed upstream (in your APM, your load balancer's own histogram, or a script that percentiles a raw sample) and pushed or pulled into rollfuse, plus a guardrail referencing it:
POST /v1/rollouts/{rollout_id}/guardrails
{
"metric_definition_id": "<p95-latency-ms>",
"comparison_operator": "gt",
"threshold": 500,
"evaluation_window_seconds": 300
}
The threshold should be set against the service's own historical p95, with headroom, not an arbitrary "under 500ms is fine" number pulled from a general best-practices list. A service whose normal p95 is 480ms and one whose normal p95 is 80ms need very different thresholds even if both "feel fine" today.
The blind spot p95 alone has
p95 is a median-of-the-tail metric: it tells you about the worse half of the upper half of your distribution, but a genuine 1% tail, the requests actually hitting a slow dependency, a cold cache, a retry storm, can sit entirely outside it. On a service with a homogeneous dependency graph, that's an acceptable trade-off. On a service where the tail specifically represents disproportionate risk (a payment gateway retrying against a slow issuer, for instance), a p95-only guardrail can stay green for the entire duration of a regression that a p99 guardrail would have caught in minutes.
The fix isn't replacing p95 with p99. It's carrying both, as two separate guardrails on the same rollout, each referencing its own metric definition. Either one breaching aborts the rollout independently; neither has to wait for the other to agree.
What "automatic" buys you here specifically
Latency regressions are usually gradual, not a cliff. That's exactly the shape a human staring at a dashboard is worst at catching in real time, and exactly the shape a scheduled evaluation against a fixed threshold catches reliably. The guardrail doesn't get bored, doesn't get distracted, and doesn't wait for someone to notice the line creeping upward before it acts.