Neural Networks
Attention Mechanism
Neural Networks· Advanced
Definition
A component allowing a model to focus on the most relevant parts of input when producing each output element. In Transformers, self-attention computes relationships between all token pairs simultaneously. Multi-head attention runs multiple operations in parallel — capturing different relationship types.
Tags
#transformers#self-attention#context#relevance
MS
Maxx Stacks Editorial
Reviewed by enterprise AI practitioners
Maxx University
Keep learning. Keep building.
250+ terms. 5 learning paths. AI maturity assessment. Jargon translator. All free, always.