Multi-Head Attention
Definition
Running multiple attention heads in parallel to capture distinct features.
Example Case
Transformer layer routing signals to 8 attention heads.
Running multiple attention heads in parallel to capture distinct features.