Basics
Parameters
The internal dials a model adjusts while learning, numbering in the billions.
Parameters are the tiny numerical settings inside a model that get tuned during training. Think of them as billions of little dials that slowly turn until the model gets good at predicting language. More parameters can mean a more capable model, but also a bigger, slower, and more expensive one, so bigger is not always better.
For example
A model described as '70 billion parameters' has 70 billion of these little dials that were adjusted as it learned.
Related
Large language modelA program that learned to use language by reading enormous amounts of text.TokenThe small chunk of text an AI reads and writes, often part of a word.Context windowHow much text an AI can keep in mind at one time.Training vs inferenceTraining is teaching the model once; inference is using it every day.