apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-12-24 13:28:13 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[xpu] support mtp for xpu(mix) (#5274)

Browse Source 
* [XPU] support kernel for mtp(base)

* [XPU] support kernel for mtp(base)

* format

* format

* format

* fix gather next token

* fix step && add test

* fix

* mv pre/post process

* add adjust batch / gather next token for mtp

* fix code style

* fix mtp kenrel name

* fix mtp kernel test

* mv xpu pre/post process

* mv xpu pre/post process

* [xpu] support mtp

* fix code style
This commit is contained in:
cmcamdy
2025-12-01 11:03:14 +08:00
committed by GitHub
parent 8aec3acc8c
commit 9f4977eb74
8 changed files with 691 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
fastdeploy/spec_decode/__init__.py
View File

@@ -14,9 +14,12 @@
"""
speculative decoding module
"""
from fastdeploy.platforms import current_platform
from .base import Proposer
from .mtp import MTPProposer
from .ngram import NgramProposer
# XPU is not support ngram proposer now
if not current_platform.is_xpu():
from .ngram import NgramProposer
__all__ = ["Proposer", "MTPProposer", "NgramProposer"]