Top-P
Definition
Nucleus sampling. Filter tokens during inference based on the cumulative probability threshold P.
Example Case
Setting Top-P to 0.9 to sample from the top 90% of tokens.
Nucleus sampling. Filter tokens during inference based on the cumulative probability threshold P.