Early build · scores computed by the deterministic engine from real, sourced developments (30 days) · weights v1.0-prior
LatestLearn › Models
Vikshy Learn · Models

Mixture of experts

A big model split into specialists, only a few of which handle each question.

A mixture of experts model is built from many smaller sub-models, each a bit of a specialist, plus a traffic director that routes each question to just the right few. Because only a slice of the full model activates for any given task, you get the smarts of a very large model without paying to run all of it every time. It is a clever way to make big models faster and cheaper.

For example, A coding question might be routed to the parts of the model best at programming, leaving the rest idle.

← All 42 concepts in the glossary