Top-K
Definition
Sampling from the top K highest probability tokens during inference, restricting output vocabulary scope.
Example Case
Setting Top-K to 40 to sample from the top 40 candidate tokens.
Sampling from the top K highest probability tokens during inference, restricting output vocabulary scope.