Basics
Training vs inference
Training is teaching the model once; inference is using it every day.
Training is the long, expensive process of building a model by feeding it mountains of text until it learns. Inference is what happens every time you actually use the finished model to get an answer. Training is like the years a doctor spends in medical school, while inference is a single appointment where they put that learning to work.
For example
A company might spend months training a model, then run inference millions of times a day as people chat with it.
Related
Large language modelA program that learned to use language by reading enormous amounts of text.TokenThe small chunk of text an AI reads and writes, often part of a word.Context windowHow much text an AI can keep in mind at one time.ParametersThe internal dials a model adjusts while learning, numbering in the billions.