Daci
eab8384da6
[Feature] ThreadPoolExecutor async fill_token_bitmask ( #5083 )
...
* ThreadPoolExecutor async fill_token_bitmask
* ThreadPoolExecutor async fill_token_bitmask logging
* fix test_guided_decoding
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* add fill_bitmask_parallel_batch_size ENV
* FD_FILL_BITMASK_BATCH fastdeploy.envs
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-11-19 10:04:16 +08:00
kxz2002
97189079b9
[BugFix] unify max_tokens ( #4968 )
...
* unify max tokens
* modify and add unit test
* modify and add unit test
* modify and add unit tests
---------
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com >
2025-11-18 20:01:33 +08:00
周周周
6584ee90e8
[unitest]clean code ( #5094 )
2025-11-18 17:21:35 +08:00
lizhenyun01
d11235333e
format flash_mask_attn
2025-11-18 17:18:12 +08:00
Echo-Nie
abc9fd31c7
【Hackathon 9th No.76】supplementary unit test for XGrammarChecker ( #4075 )
...
* supplementary unit test for XGrammarChecker
* mock the xgrammer,torch
2025-11-17 22:05:53 +08:00
FocusLuo
c2c1942db9
[INTEL_HPU] [CI] enabled fastdeploy PR testing ( #4596 )
...
* [INTEL HPU] added hpu ci work flow support
Signed-off-by: Luo, Focus <focus.luo@intel.com >
* [INTEL HPU] added run ci hpu test scripts
Signed-off-by: Luo, Focus <focus.luo@intel.com >
* [INTEL HPU] enabled HPU ernie test case
Signed-off-by: Luo, Focus <focus.luo@intel.com >
* [INTEL HPU] updated Intel Gaudi Readme with Warmup disable cmdline
Signed-off-by: Luo, Focus <focus.luo@intel.com >
* Modify paddlepaddle installation command
Updated paddlepaddle installation command to use a specific index URL.
* Update run_ci_hpu.sh
* Rename json directory to nlohmann_json
Rename extracted json directory to nlohmann_json.
* Update ci_hpu.yml
* Set pip global index URL to Tsinghua mirror
* Update CI workflow to use self-hosted runner and paths
* Update Docker image in CI workflow
* Modify HPU installation URLs in run_ci_hpu.sh
Updated the installation URL for paddle_intel_hpu and added paddlenlp_ops installation.
* Fix paddle_intel_hpu installation URL
Corrected the URL for paddle_intel_hpu wheel installation.
---------
Signed-off-by: Luo, Focus <focus.luo@intel.com >
Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com >
2025-11-17 19:24:41 +08:00
周周周
b23e684b67
revert group size 3 ( #5079 )
2025-11-17 18:54:13 +08:00
plusNew001
7f94d77e08
[XPU][CI] fix ci case bug ( #5084 )
...
* Ignore markdown and text files in CI workflow
* Change GPU_ID to XPU_ID in run_ci_xpu.sh
* Change GPU_ID to XPU_ID in test configuration
* Change GPU_ID to XPU_ID for service port calculation
* Change GPU_ID to XPU_ID for device identification
* Change GPU_ID to XPU_ID in test_ep function
* Update run_w4a8.py
* Redirect stop_processes output to kill.log
Redirect output of stop_processes to kill.log to capture logs.
* Log server output for failed test cases
Added logging of server.log for failed tests.
* Add '-s' option to pytest commands in run_ci_xpu.sh
* Refactor assertion to validate multiple keywords
Updated assertion to check for multiple keywords in response.
* Fix assertany to assert any in run_45vl.py
2025-11-17 16:01:27 +08:00
LiqinruiG
33f96ff93a
[BugFix] rollback max_tokens and min_tokens when continue to infer ( #5052 )
...
Co-authored-by: liqinrui <liqinrui@baidu.com >
2025-11-17 14:31:26 +08:00
Winters Montagne
ff26158f20
Add unit tests for triton_utils_v2 ( #5073 )
2025-11-17 11:46:38 +08:00
Winters Montagne
02c83d65db
[CI]【Hackathon 9th Sprint No.13】NO.13 功能模块 fastdeploy/model_executor/ops/triton_ops/triton_utils.py 单测补充 ( #5035 )
...
* Add unit tests for triton_utils.py
* update name
* update
* update
* update
2025-11-17 11:43:31 +08:00
qwes5s5
36216e62f0
[Log] Add trace log and add loggingInstrumentor tool ( #4692 )
...
* add trace logger and trace print
* trigger ci
* fix unittest
* translate notes and add copyright
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com >
2025-11-17 11:08:57 +08:00
zhouchong
5444af6ff6
[APIServer] metrics use port the same as api_port ( #5016 )
...
* metrics use port the same as api_port
* Be tolerant to tests that monkeypatch/partially mock args.
* Reduce code redundancy
---------
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com >
2025-11-17 10:42:45 +08:00
plusNew001
0e819cd596
[CI][XPU] Optimize CI logs and variable names ( #5025 )
...
* Ignore markdown and text files in CI workflow
* Change GPU_ID to XPU_ID in run_ci_xpu.sh
* Change GPU_ID to XPU_ID in test configuration
* Change GPU_ID to XPU_ID for service port calculation
* Change GPU_ID to XPU_ID for device identification
* Change GPU_ID to XPU_ID in test_ep function
* Update run_w4a8.py
* Redirect stop_processes output to kill.log
Redirect output of stop_processes to kill.log to capture logs.
* Log server output for failed test cases
Added logging of server.log for failed tests.
* Add '-s' option to pytest commands in run_ci_xpu.sh
2025-11-14 19:35:35 +08:00
Daci
5fc12eddfe
[Optimization] xgrammar async compile, multi thread, speed up ( #4835 )
...
* xgrammar async compile, multi thread, speed up
* fix test_sampler.py & pre-commit err
* add redis version check && fix request.llm_engine_recv_req_timestamp
* xgrammar prefill & decode & v0
* fix test_gpu_prompt_logprobs.py
* add test_guided_decoding.py
* Update fastdeploy/scheduler/splitwise_scheduler.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/model_executor/guided_decoding/xgrammar_backend.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/model_executor/guided_decoding/xgrammar_backend.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix torch xgrammar unittest env
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-11-14 18:05:26 +08:00
Winters Montagne
b925533051
add test_process_video.py ( #5011 )
2025-11-14 17:23:30 +08:00
周周周
c0a4393d72
[ATTENTION] unitest ( #4962 )
2025-11-14 13:45:53 +08:00
essos
191a597d9f
[CI]【Hackathon 9th Sprint No.56】NO.56 功能模块 fastdeploy/multimodal/utils.py 单测补充 ( #4954 )
...
* update test utils
* update test utils code
* update test file name
2025-11-14 10:37:27 +08:00
Juncai
36822fa49c
[PD Disaggregation] remove splitwise deployment on single node and refine the code ( #4891 )
...
* remove splitwise deployment on single node and refine the code
* up
* up
* up
* add test
* up
2025-11-14 09:56:53 +08:00
kxz2002
9703108c28
[BugFix] adjust max_tokens and min_tokens when continue to generate tokens ( #5010 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
* fix max and min tokens initial commit
* fix double subtraction
* add unit tests
2025-11-13 23:52:54 +08:00
yangjianfengo1
ae7bee8122
【New Feature】W4afp8 supports per group quantization ( #4987 )
...
* w4afp8 支持per group
* code style
* fix transpose
* revert fast hardmard
---------
Co-authored-by: yuanxiaolan <yuanxiaolan01@baidu.com >
Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com >
2025-11-13 19:17:27 +08:00
zccjjj
88da9d9788
[XPU] [CI] Change CI ep test from offline to online ( #4885 )
...
* change CI ep test from offline to online
* add ep all2all ci's changes, from offline to online
* change env var in ep-all2all ci test
* add expected response for ep8tp8 all2all
* Adapt to CI refactoring and support dual-concurrent code execution
* Adapt to CI refactoring and support dual-concurrent, second
* Explicitly specify the #port
* change the startup method of all2all
* Modify the command of all2all
* Update assertion to check multiple keywords
* Update assertion to check multiple keywords
* Update run_w4a8.py
* Update run_w4a8.py
---------
Co-authored-by: plusNew001 <95567040+plusNew001@users.noreply.github.com >
2025-11-13 16:15:45 +08:00
ltd0924
303c986cc7
[FDConfig] add block number verfied ( #4983 )
...
* Update config.py
* fix
* update unit test
---------
Co-authored-by: ltd0924 <luotingdan@baidu.com >
2025-11-13 09:48:44 +08:00
YuBaoku
1c0b0b08b7
[CI] set DG_NVCC_OVERRIDE_CPP_STANDARD in test_quantized_linear ( #4995 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
2025-11-12 23:03:21 +08:00
bukejiyu
f0189292df
[CI] fix test_model_cache ( #4982 )
...
* ci
* update
2025-11-12 20:26:49 +08:00
qwes5s5
a2d06118e1
[Logprobs]Support prompt_logprobs and max_logprobs ( #4897 )
...
* add prompt logprobs
* trigger ci
* fix unitest
* Update fastdeploy/config.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/entrypoints/llm.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/engine/sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/engine/test_sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/engine/test_sampling_params.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix max_logprobs
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-11-12 19:29:48 +08:00
ltd0924
5bf48de999
[KVCache] support unified cache backend ( #4903 )
...
* [Feature] support unified cache backend
* fix
* fix
* fix
* fix
* Update metax_model_runner.py
* fix
* update
* Update test_moba_attention_backend.py
---------
Co-authored-by: ltd0924 <luotingdan@baidu.com >
2025-11-12 14:54:52 +08:00
yzwu
76e60e98f8
[Iluvatar][CI] fix safetensors_rust.SafetensorError: framework paddle is invalid ( #4972 )
2025-11-12 14:13:40 +08:00
Sunny-bot1
35bd2afab3
[Benchmark] Add GEMM & MoE kernel bench ( #4809 )
2025-11-12 11:56:40 +08:00
YuBaoku
8a96944a0a
[CI] Update PORT range to avoid conflict with system ports ( #4953 )
2025-11-12 11:17:49 +08:00
Echo-Nie
ff653503ff
[Docs] Add License in Unittest ( #4957 )
...
* add copyright
* add CopyRight
2025-11-12 10:44:09 +08:00
Echo-Nie
2aabaecbc2
[CI] Add five unittest ( #4958 )
...
* add unittest
* Update test_logger.py
2025-11-12 10:43:33 +08:00
bukejiyu
b09ebb2813
refactor pt loading ( #4532 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FD Image Build (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
2025-11-11 21:30:39 +08:00
周周周
da6b4c10e5
[ATTENTION] make buffer alloc as a function ( #4945 )
2025-11-11 19:17:08 +08:00
chen
896ef565cc
[Others] Add Tests for GPU Model Runner and Logprobs Output ( #4913 )
2025-11-11 18:37:33 +08:00
kxz2002
a83250ae3f
[CI] Update test_api_key.py ( #4948 )
...
* fix test_api_key
* fix test_api_key
2025-11-11 16:49:54 +08:00
yzwu
3707af7a4f
[Iluvatar] add vl into ci and support v1 loader ( #4774 )
2025-11-11 10:50:17 +08:00
Yuanle Liu
3dc0ffa46d
[TSP] Support qwen3 moe tsp + cudagraph ( #4871 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
* support qwen3_moe tsp mode
* fix
* fix
* update
* update
* update
* fix
* support external_rmsnorm
* update
* fix
2025-11-10 23:37:51 +08:00
plusNew001
3665c283b5
[XPU] [CI]Change CI to multi-concurrency ( #4866 )
...
* Refactor GPU ID logic in CI workflow
Updated GPU ID assignment logic and removed unused port calculations.
* Refactor GPU device and port configuration
* Update engine_worker_queue_port calculation logic
* Refactor XPU_VISIBLE_DEVICES export logic
* Adjust service port based on GPU ID
* Adjust service HTTP port based on GPU ID
* Adjust service_http_port based on GPU_ID
* Add import for os module in run_45T.py
* Update run_45vl.py
* Import os module in run_w4a8.py
Added import for os module to use environment variables.
* Remove duplicate import of os module
* Remove duplicate import of os module
* Update run_45T.py
* Update run_w4a8.py
* fix bug
* fix bug
* Update run_w4a8.py
* Fix directory change command in run_ci_xpu.sh
2025-11-10 21:09:48 +08:00
Sunny-bot1
59d2edde29
[BugFix] Add support for weight shape constraints and group size selection in Machete ( #4911 )
2025-11-10 20:57:35 +08:00
Echo-Nie
112623e33e
init version, exist some bugs, waiting fix ( #4906 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-10 14:16:09 +08:00
luukunn
41c0bef964
[BugFix] When the value of "temperature" is 0, adjust it to 1e-06 ( #4900 )
...
* add default temperature value
* add unit test
* update
* update
* add unit test
* update
* fix unit test
2025-11-10 13:24:33 +08:00
plusNew001
0a3bc84f71
[XPU][CI]Update test assertion and base response value ( #4907 )
2025-11-10 11:44:54 +08:00
kxz2002
87911b7cf1
[Feature] Enable FastDeploy to support adding the “--api-key” authentication parameter. ( #4806 )
...
* add api key initial commit
* add unit test
* modify unit test
* move middleware to a single file and add unit tests
2025-11-08 18:24:02 +08:00
plusNew001
fa098383f6
[XPU][CI] Ci bug fix ( #4889 )
...
* Refactor test_45t by commenting out responses
Comment out base response variables and update assertion.
* Update run_w4a8.py
* Fix assertion syntax in run_45T.py
2025-11-07 17:50:11 +08:00
ming1753
cba185f1fe
[Feature] Optim PaddleOCR-VL ( #4873 )
...
* [Feature] Optim PaddleOCR-VL
* fix bug
2025-11-07 14:56:44 +08:00
YuBaoku
fa28745f19
[CI] Update ERNIE-4.5-VL baseline to adapt to MoE changes ( #4867 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-11-06 22:02:10 +08:00
kevin
cc34487810
[Feature] support mm disable_chunked ( #4803 )
...
* support mm disable_chunked
* update code
* update code
* update code
2025-11-06 21:32:25 +08:00
YuBaoku
a139f8f3cb
[CI] Optimize port cleanup logic ( #4860 )
2025-11-06 19:13:48 +08:00
Zhang Yulong
5aa73d32f4
Update deploy.py ( #4850 )
2025-11-06 19:09:28 +08:00
