apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-05 08:37:06 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Feature] support top_k_top_p sampling (#2753)

Browse Source 
* support top_k_top_p sampling

* fix

* add api param

* add api para

* fix

* fix

* fix

* fix

* fix

* fix

* fix
This commit is contained in:
Sunny-bot1
2025-07-10 11:58:58 +08:00
committed by GitHub
parent b0f525955c
commit e45050cae3
15 changed files with 501 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/zh/offline_inference.md
View File

@@ -178,6 +178,7 @@ for output in outputs:
* repetition_penalty(float): 直接对重复生成的token进行惩罚的系数（>1时惩罚重复，<1时鼓励重复）
* temperature(float): 控制生成随机性的参数，值越高结果越随机，值越低结果越确定
* top_p(float): 概率累积分布截断阈值，仅考虑累计概率达到此阈值的最可能token集合
* top_k(int): 采样概率最高的的token数量，考虑概率最高的k个token进行采样
* max_tokens(int): 限制模型生成的最大token数量（包括输入和输出）
* min_tokens(int): 强制模型生成的最少token数量，避免过早结束