Parameters are the numbers inside a neural network that training adjusts. They include the weights that set the strength of connections between units, and other values such as biases. The model's behavior, including its knowledge and skills, is stored in them. Model sizes are described by their count, such as 7 billion or 70 billion parameters.
The count gives a rough idea of capacity and cost. Memory needed is approximately the number of parameters multiplied by the bytes used per parameter. A 70-billion-parameter model stored at 16-bit precision, which is 2 bytes each, needs about 140 gigabytes just for the weights, and about 35 gigabytes at 4-bit precision, before overhead.
Size is not quality. Training data, training methods and architecture matter a great deal, and smaller well-trained models can outperform larger ones. In mixture-of-experts models, the total number is much larger than the number used for each token.
Use the parameter count to estimate hardware needs, and compare models by their results on your tasks.