Neural Networks
Mixture of Depths
MoD
Neural Networks· Advanced
Definition
A transformer architecture that dynamically allocates compute to different tokens — processing some tokens through all layers while routing others through fewer layers. Complementary to Mixture of Experts, MoD reduces computation on less informative tokens while maintaining model capacity.
Enterprise Context
Part of the emerging class of adaptive compute architectures that will define the next generation of efficient enterprise AI — reducing inference cost without sacrificing quality.
Tags
#architecture#efficiency#transformer
MS
Maxx Stacks Editorial
Reviewed by enterprise AI practitioners
Maxx University
Keep learning. Keep building.
250+ terms. 5 learning paths. AI maturity assessment. Jargon translator. All free, always.