Attention Mechanism
Definition
The mathematical layer in transformers calculating query-key relationships to weight token dependencies in text.
Example Case
Self-attention linking the word "it" back to the subject "dog" in a previous sentence.