The measure pass@k is the probability that an AI agent succeeds on a task at least once in k attempts. It is commonly used for tasks such as code generation, where several attempts can be made and a checker can tell which one works.

It matters because it shows what an agent can do when it is allowed multiple tries, and it is useful when a person or an automated test can pick the correct answer afterwards. A higher value of k usually gives a higher score.

It does not show reliability. A high pass@k can be reached by an agent that fails most of the time, as long as one attempt in k succeeds, so it can overstate how well the agent works when each attempt matters. It is also often estimated from sampled attempts, so it has statistical uncertainty, and different papers may define the details differently.

When you read one, check the value of k, how success is verified and whether the use case allows several attempts, and compare with pass^k if consistency matters.