Why the average hides the long wait.

An average combines fast and slow requests into one value. That can help describe a workload, but it does not show which users waited longest or where those waits occurred.

Understand the distribution

The median describes the middle of the ordered measurements. A 95th percentile describes a threshold near which 95% of the measured values fall at or below, with the exact calculation depending on the chosen percentile method. Neither number explains the cause of a slow request by itself.

Keep the comparison fair

Compare the same operation, measurement boundary and time window. Separate materially different workloads: a short classification request and a long generated response should not automatically share one latency target. Always consider sample size when interpreting a percentile.

Move from the aggregate to a trace

After finding a slow group, inspect representative requests. A long retrieval span suggests a different investigation from a long model span. The Tracefield slow-response example changes the model duration while leaving the surrounding operations unchanged.

Distinguish first output from completion

For streaming applications, time to first output and time to complete response answer different questions. Record both when they matter. Do not compare one metric against another under a single generic label.

Use a clear experiment

Change one part of the path, then compare equivalent workloads. Report the measurement window and trade-offs as well as the result. The figures in this template are illustrative and do not claim a performance improvement for a real customer.