Frontier models are the biggest and most capable models at the leading edge of what AI can do. They tend to be the most expensive to build and run, and they are the ones that spark headlines and safety debates. What counts as the frontier keeps moving as new models leapfrog the old.
For example, The flagship models from the top AI labs each year are considered frontier models.