For each task the model gets one try, and the score is the share of tasks that it solves. Because it shows what you get when you run the model once, it is the most realistic measure of everyday use. In practice it is often estimated from several samples per task, to reduce the effect of chance.
The notation comes from code generation research, where the Codex paper in 2021 defined pass@k for the HumanEval benchmark. The idea has since spread to many other kinds of task.
A high pass@1 means reliability. A model that solves a task only sometimes will score lower, even if it can eventually succeed with more tries.
When you read a score, check how many samples were used, how success was judged and whether tools or retries were allowed, and compare it with pass@k figures only knowing that they answer different questions.