Sampling is the practice of recording only a share of events, such as 5% of requests, to cut data volume and cost. It is widely used for traces and sometimes for logs, because keeping every event from a busy system can be expensive. A well-chosen sample can still show typical behavior.
Two common approaches are head-based and tail-based sampling. Head-based sampling decides at the start of a request whether to keep it, which is simple and cheap. Tail-based sampling waits until the request finishes, so it can keep the unusual ones, such as errors and slow requests, though it needs more resources to run.
Sampling can hide what you most need to see. Rare failures may fall outside a small random sample, and the sample may not represent all users or paths. Counts derived from sampled data must be scaled by the sampling rate, or they will understate real volumes, and different rates across services can produce misleading comparisons.
To use it carefully, record the sampling rate, keep errors and slow requests where possible, avoid sampling the metrics that drive alerts and check from time to time that the sample still answers your questions.