A weight is a learned number inside an AI model. A model is made up of a very large number of them, and they are set during training so that the model's calculations produce useful outputs. A model's weights are what it reads from memory to produce each output.
It matters for on-device AI because the weights have to be stored and read quickly. The total size of the weights decides how much memory a model needs, and the speed at which they can be read from memory is often what limits how fast the model generates text.
The word has several meanings in other fields, so context is important. A model's weights are not the whole of a system, since prompts, tools and surrounding software also shape what it does. Sharing or publishing the weights has implications for how a model can be used, and the details depend on the licence under which it is released.
When estimating whether a model will run on a device, multiply the number of parameters by the size of each at the chosen precision, and compare the result with the available memory and its bandwidth.