Juncai
36822fa49c
[PD Disaggregation] remove splitwise deployment on single node and refine the code ( #4891 )
...
* remove splitwise deployment on single node and refine the code
* up
* up
* up
* add test
* up
2025-11-14 09:56:53 +08:00
kxz2002
9703108c28
[BugFix] adjust max_tokens and min_tokens when continue to generate tokens ( #5010 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
* fix max and min tokens initial commit
* fix double subtraction
* add unit tests
2025-11-13 23:52:54 +08:00
yangjianfengo1
ae7bee8122
【New Feature】W4afp8 supports per group quantization ( #4987 )
...
* w4afp8 支持per group
* code style
* fix transpose
* revert fast hardmard
---------
Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com >
Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com >
2025-11-13 19:17:27 +08:00
zccjjj
88da9d9788
[XPU] [CI] Change CI ep test from offline to online ( #4885 )
...
* change CI ep test from offline to online
* add ep all2all ci's changes, from offline to online
* change env var in ep-all2all ci test
* add expected response for ep8tp8 all2all
* Adapt to CI refactoring and support dual-concurrent code execution
* Adapt to CI refactoring and support dual-concurrent, second
* Explicitly specify the #port
* change the startup method of all2all
* Modify the command of all2all
* Update assertion to check multiple keywords
* Update assertion to check multiple keywords
* Update run_w4a8.py
* Update run_w4a8.py
---------
Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com >
2025-11-13 16:15:45 +08:00
ltd0924
303c986cc7
[FDConfig] add block number verfied ( #4983 )
...
* Update config.py
* fix
* update unit test
---------
Co-authored-by: ltd0924 <luotingdan@baidu.com >
2025-11-13 09:48:44 +08:00
YuBaoku
1c0b0b08b7
[CI] set DG_NVCC_OVERRIDE_CPP_STANDARD in test_quantized_linear ( #4995 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
2025-11-12 23:03:21 +08:00
bukejiyu
f0189292df
[CI] fix test_model_cache ( #4982 )
...
* ci
* update
2025-11-12 20:26:49 +08:00
qwes5s5
a2d06118e1
[Logprobs]Support prompt_logprobs and max_logprobs ( #4897 )
...
* add prompt logprobs
* trigger ci
* fix unitest
* Update fastdeploy/config.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/entrypoints/llm.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/engine/sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/engine/test_sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/engine/test_sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix max_logprobs
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-11-12 19:29:48 +08:00
ltd0924
5bf48de999
[KVCache] support unified cache backend ( #4903 )
...
* [Feature] support unified cache backend
* fix
* fix
* fix
* fix
* Update metax_model_runner.py
* fix
* update
* Update test_moba_attention_backend.py
---------
Co-authored-by: ltd0924 <luotingdan@baidu.com >
2025-11-12 14:54:52 +08:00
yzwu
76e60e98f8
[Iluvatar][CI] fix safetensors_rust.SafetensorError: framework paddle is invalid ( #4972 )
2025-11-12 14:13:40 +08:00
Sunny-bot1
35bd2afab3
[Benchmark] Add GEMM & MoE kernel bench ( #4809 )
2025-11-12 11:56:40 +08:00
YuBaoku
8a96944a0a
[CI] Update PORT range to avoid conflict with system ports ( #4953 )
2025-11-12 11:17:49 +08:00
Echo-Nie
ff653503ff
[Docs] Add License in Unittest ( #4957 )
...
* add copyright
* add CopyRight
2025-11-12 10:44:09 +08:00
Echo-Nie
2aabaecbc2
[CI] Add five unittest ( #4958 )
...
* add unittest
* Update test_logger.py
2025-11-12 10:43:33 +08:00
bukejiyu
b09ebb2813
refactor pt loading ( #4532 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
2025-11-11 21:30:39 +08:00
周周周
da6b4c10e5
[ATTENTION] make buffer alloc as a function ( #4945 )
2025-11-11 19:17:08 +08:00
chen
896ef565cc
[Others] Add Tests for GPU Model Runner and Logprobs Output ( #4913 )
2025-11-11 18:37:33 +08:00
kxz2002
a83250ae3f
[CI] Update test_api_key.py ( #4948 )
...
* fix test_api_key
* fix test_api_key
2025-11-11 16:49:54 +08:00
yzwu
3707af7a4f
[Iluvatar] add vl into ci and support v1 loader ( #4774 )
2025-11-11 10:50:17 +08:00
Yuanle Liu
3dc0ffa46d
[TSP] Support qwen3 moe tsp + cudagraph ( #4871 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
* support qwen3_moe tsp mode
* fix
* fix
* update
* update
* update
* fix
* support external_rmsnorm
* update
* fix
2025-11-10 23:37:51 +08:00
plusNew001
3665c283b5
[XPU] [CI]Change CI to multi-concurrency ( #4866 )
...
* Refactor GPU ID logic in CI workflow
Updated GPU ID assignment logic and removed unused port calculations.
* Refactor GPU device and port configuration
* Update engine_worker_queue_port calculation logic
* Refactor XPU_VISIBLE_DEVICES export logic
* Adjust service port based on GPU ID
* Adjust service HTTP port based on GPU ID
* Adjust service_http_port based on GPU_ID
* Add import for os module in run_45T.py
* Update run_45vl.py
* Import os module in run_w4a8.py
Added import for os module to use environment variables.
* Remove duplicate import of os module
* Remove duplicate import of os module
* Update run_45T.py
* Update run_w4a8.py
* fix bug
* fix bug
* Update run_w4a8.py
* Fix directory change command in run_ci_xpu.sh
2025-11-10 21:09:48 +08:00
Sunny-bot1
59d2edde29
[BugFix] Add support for weight shape constraints and group size selection in Machete ( #4911 )
2025-11-10 20:57:35 +08:00
Echo-Nie
112623e33e
init version, exist some bugs, waiting fix ( #4906 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-10 14:16:09 +08:00
luukunn
41c0bef964
[BugFix] When the value of "temperature" is 0, adjust it to 1e-06 ( #4900 )
...
* add default temperature value
* add unit test
* update
* update
* add unit test
* update
* fix unit test
2025-11-10 13:24:33 +08:00
plusNew001
0a3bc84f71
[XPU][CI]Update test assertion and base response value ( #4907 )
2025-11-10 11:44:54 +08:00
kxz2002
87911b7cf1
[Feature] Enable FastDeploy to support adding the “--api-key” authentication parameter. ( #4806 )
...
* add api key initial commit
* add unit test
* modify unit test
* move middleware to a single file and add unit tests
2025-11-08 18:24:02 +08:00
plusNew001
fa098383f6
[XPU][CI] Ci bug fix ( #4889 )
...
* Refactor test_45t by commenting out responses
Comment out base response variables and update assertion.
* Update run_w4a8.py
* Fix assertion syntax in run_45T.py
2025-11-07 17:50:11 +08:00
ming1753
cba185f1fe
[Feature] Optim PaddleOCR-VL ( #4873 )
...
* [Feature] Optim PaddleOCR-VL
* fix bug
2025-11-07 14:56:44 +08:00
YuBaoku
fa28745f19
[CI] Update ERNIE-4.5-VL baseline to adapt to MoE changes ( #4867 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-06 22:02:10 +08:00
kevin
cc34487810
[Feature] support mm disable_chunked ( #4803 )
...
* support mm disable_chunked
* update code
* update code
* update code
2025-11-06 21:32:25 +08:00
YuBaoku
a139f8f3cb
[CI] Optimize port cleanup logic ( #4860 )
2025-11-06 19:13:48 +08:00
Zhang Yulong
5aa73d32f4
Update deploy.py ( #4850 )
2025-11-06 19:09:28 +08:00
YuBaoku
819b2dbbae
Revert "【New Feature】W4afp8 supports per group quantization ( #4272 )" ( #4854 )
...
This reverts commit 93fcf7e4ec
2025-11-06 17:48:28 +08:00
Juncai
08ca0f6aea
[Feature] [PD] add simple router and refine splitwise deployment ( #4709 )
...
* add simple router and refine splitwise deployment
* fix
2025-11-06 14:56:02 +08:00
plusNew001
fc8bef2c95
[XPU][CI]Change ci vl model to 28 b ( #4764 )
...
* Update XPU_VISIBLE_DEVICES and model parameters
* Update base response and adjust max tokens
* Implement process cleanup in CI workflow
Add process cleanup commands to prevent port conflicts
* Remove process cleanup commands from CI workflow
Removed old process cleanup commands to prevent port conflicts.
2025-11-06 14:12:23 +08:00
Echo-Nie
354ddc8bc5
[CI] Add unittest for activation, native_paddle_backend, w4a8, w4afp8, platforms/utils ( #4812 )
...
* add unnitest for activation, native_paddle_backend, w4a8, w4afp8, platforms/utils
* Remove activation function retrieval tests
Removed tests for valid and unsupported activation function retrieval.
* move w4a8, w4afp8 to quantization
* fix code style
2025-11-06 14:08:00 +08:00
SunLei
782818c031
fix: ci port conflict ( #4840 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-06 11:56:17 +08:00
kxz2002
5bdd40da5d
[BugFix] Fix ernie_vl_reasoning_parsers.py 'end_token' to 'think_end_token' ( #4805 )
...
* fix ernie_vl_reasoning_parsers.py 'end_token' to 'think_end_token'
* add unit tests
2025-11-06 11:28:55 +08:00
yangjianfengo1
93fcf7e4ec
【New Feature】W4afp8 supports per group quantization ( #4272 )
...
* w4afp8 支持per group
* code style
* 精度完成
* revert append attn utils
* ffn1 动态量化
* ffn2 支持动态量化
* code style
* code style
* 修改单测
* 修改单测
* fix bug
* Implement conditional parameter creation for layers
Add parameter creation for up_gate_proj_in_scale when ep_size > 1.
* code style
* fix conflict
* code style
* code style
* 修复w4aint8 精度
* fix ci
---------
Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com >
2025-11-05 21:00:23 +08:00
chenjian
cc8f5312f5
[Feature] Add timestamp for profiler ( #4726 )
...
* [Feature] Add timestamp for profiler
* fix bug for offine inference
* fix for ci
* fix
* fix ci
2025-11-05 12:04:59 +08:00
周周周
876e4a8935
remove input_ids from ForwardMeta ( #4793 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-05 11:55:51 +08:00
kxz2002
9676cc87d6
fix parser register name ( #4795 )
...
Co-authored-by: luukunn <83932082+luukunn@users.noreply.github.com >
2025-11-05 11:27:30 +08:00
zhupengyang
2fd254e5b7
support ep+tp at op layer ( #4688 )
2025-11-05 11:15:57 +08:00
周周周
937eb3c6ed
[get_padding_offset.] clean get_padding_offset.cu ( #4777 )
...
[get_padding_offset.] clean get_padding_offset.cu (#4777 )
2025-11-05 10:47:40 +08:00
Haonan Luo
2c281e617c
Update Unit Test for PaddleOCR-VL ( #4802 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
* fix paddleocr prefix cache bug
* add test for paddleocr_vl
* disable prefix-caching in ocr
* add test for paddleocr_vl
* Fix top_p for rejection sampling
* add test for ocr processor; fix top_p for rejection sampling
* add test for ocr processor; fix top_p for rejection sampling
* add test for ocr processor; fix top_p for rejection sampling
* add test for ocr processor; fix top_p for rejection sampling
* add test for ocr processor; fix top_p for rejection sampling
---------
Co-authored-by: ming1753 <ideaminghp@163.com >
Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com >
2025-11-04 22:40:15 +08:00
YuBaoku
722110a952
[CI] Refactor CE wheel upload for multiple target paths ( #4790 )
...
* [CI] Refactor CE wheel upload for multiple target paths
* [CI] fix test_streaming_with_stop_str error
2025-11-04 18:56:38 +08:00
kxz2002
8a40374bfe
[BugFix] Fix ernie4_5_vl_processor.py and qwen_vl_processor.py can not disable thinking ( #4762 )
...
* fix ernie4_5_vl_processor.py and qwen_vl_processor.py
* add unit test
2025-11-04 16:00:32 +08:00
bukejiyu
41bfa1090d
[CI]delete test_common_model ( #4794 )
...
* fix
* update
* update
2025-11-04 13:57:55 +08:00
plusNew001
9887025926
Update run_w4a8.py ( #4783 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-03 21:41:00 +08:00
bukejiyu
69c2f3cda1
[CI]test common model ( #4697 )
...
* ut
* update
2025-11-03 16:48:36 +08:00
