Parameters are the tiny numerical settings inside a model that get tuned during training. Think of them as billions of little dials that slowly turn until the model gets good at predicting language. More parameters can mean a more capable model, but also a bigger, slower, and more expensive one, so bigger is not always better.
For example, A model described as '70 billion parameters' has 70 billion of these little dials that were adjusted as it learned.