Large Language Models
Mixture of Experts
MoE
Large Language Models· Advanced
Definition
An architecture where different expert sub-networks specialize in different input types, with a gating mechanism routing each token to the most relevant experts. MoE models achieve high capacity while activating only a fraction of parameters per forward pass. Used in GPT-4 and Gemini.
Tags
#architecture#efficiency#routing#sparsity#scaling
MS
Maxx Stacks Editorial
Reviewed by enterprise AI practitioners
Maxx University
Keep learning. Keep building.
250+ terms. 5 learning paths. AI maturity assessment. Jargon translator. All free, always.