Self-Attention
Definition
The core mechanism in transformers calculating relative token importances.
Example Case
Calculating attention weights for each word in a query.
The core mechanism in transformers calculating relative token importances.