chenjian
41cd3e24c9
[Feature] Enable prefix caching as default ( #3816 )
...
* [Feature] Enable prefix caching as default
* [Feature] Enable prefix caching as default
* Set prefix caching as default
* skip dynamic load
* fix kill bug
* fix kill bug
* fix kill bug
* fix ci
* fix
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
2025-09-06 09:51:34 +08:00
Zhang Yulong
11b18e5ef0
add cache queue port ( #3904 ) ( #3926 )
...
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
* add cache queue port
* add cache queue port
* add cache queue port
2025-09-06 00:00:12 +08:00
Divano
17731a8acd
add concurrency cases ( #3689 )
2025-08-28 18:30:19 +08:00
ltd0924
3d92fb09f7
[BugFix] fix parameter is 0 ( #3592 )
...
* Update engine_client.py
* fix
* Update common_engine.py
2025-08-28 09:52:36 +08:00
xjkmfa
afb9f327ef
【CI case】for echo finish_reason text_after_process and raw_prediction check ( #3630 )
...
* Add ci case for min token and max token
* 【CI case】include total_tokens in the last packet of completion interface stream output
* echo&finish_reason&text_after_process&raw_prediction check
* echo&finish_reason&text_after_process&raw_prediction check
* echo&finish_reason&text_after_process&raw_prediction check
* echo&finish_reason&text_after_process&raw_prediction check
* echo&finish_reason&text_after_process&raw_prediction check
---------
Co-authored-by: xujing43 <xujing43@baidu.com >
2025-08-27 15:21:16 +08:00
Ryan
a5b4866ff1
[CudaGraph][SOT] Add unit tests for splitting the static graph into piecewise graphs that support cuda_graph ( #3590 )
...
* add unitest
* change sot_warmup_sizes
* wtf; add missed commit
2025-08-26 11:25:04 +08:00
chen
9cab3f47ff
[Feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing ( #3552 )
...
* [feature] Add temp_scaled_logprobs and top_p_normalized_logprobs parameters for logits and logprobs post processing
* infer engine support temp_scaled_logprobs and top_p_normalized_logprobs
* delete some code
* code check
* code check and add doc
* fix tokenizer.decoder(-1), return 'Invalid Token'
* add ci for temp_scaled and top_p logprobs
* check test
* check seq len time shape
* logprob clip inf
---------
Co-authored-by: sunlei1024 <sunlei5788@gmail.com >
2025-08-25 14:11:49 +08:00
YuBaoku
7821534ff5
[CI] add sot test ( #3579 )
...
* [CI] add sot test
* [CI] add sot test
2025-08-25 10:14:50 +08:00
Zhang Yulong
3cc182236a
update ci ( #3519 )
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
2025-08-21 20:05:50 +08:00
Yzc216
466cbb5a99
[Feature] Models api ( #3073 )
...
* add v1/models interface related
* add model parameters
* default model verification
* unit test
* check model err_msg
* unit test
* type annotation
* model parameter in response
* modify document description
* modify document description
* unit test
* verification
* verification update
* model_name
* pre-commit
* update test case
* update test case
* Update tests/entrypoints/openai/test_serving_models.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/entrypoints/openai/test_serving_models.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/entrypoints/openai/test_serving_models.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/entrypoints/openai/test_serving_models.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/entrypoints/openai/serving_models.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: LiqinruiG <37392159+LiqinruiG@users.noreply.github.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-08-21 17:02:56 +08:00
Zhang Yulong
b7eee3aec1
Update CI ( #3474 )
...
* update CI cases
* update CI cases
* update CI cases
* update CI cases
* Merge upstream/develop and resolve directory rename conflict
* Merge upstream/develop and resolve directory rename conflict
* Merge upstream/develop and resolve directory rename conflict
* update deploy
* update deploy
* update deploy
* update deploy
* update deploy
2025-08-21 16:49:20 +08:00
YUNSHEN XIE
3a6058e445
Add stable ci ( #3460 )
...
* add stable ci
* fix
* update
* fix
* rename tests dir;fix stable ci bug
* add timeout limit
* update
2025-08-20 08:57:17 +08:00
