The measure pass^k is the probability that an AI agent succeeds on a task in all k attempts. It is used to judge consistency: an agent that solves a task three times out of four has a good chance of succeeding once, but a much lower chance of succeeding every time.
It matters because many real uses need reliability, not luck. An agent that handles customer requests or runs automated jobs must work each time, so a measure that rewards a single success can make an unreliable agent look better than it is. As k increases, the score usually falls.
It is a statistic estimated from a limited number of trials, so it has uncertainty, and it depends on how success is defined and graded. Different studies may use different definitions or values of k, and it should not be confused with pass@k, which measures the chance of at least one success.
When you see it, check the value of k, how many trials were run and how success was graded, and consider the confidence interval if one is given.