apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2025-10-06 09:07:10 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[plugin] Custom model_runner/model support (#3186)

Browse Source 
* support custom model&&model_runner

* fix merge

* add test && update doc

* fix codestyle

* fix unittest

* load model in rl
This commit is contained in:
lizhenyun01
2025-08-05 09:52:39 +08:00
committed by GitHub
parent 72ef5a9c93
commit fe540f6caa
15 changed files with 150 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/features/plugins.md
View File

@@ -20,12 +20,23 @@ Assuming you have a custom model class `MyModelForCasualLM` and a pretrained cla
# File: fd_add_dummy_model/__init__.py or fd_add_dummy_model/register.py
from fastdeploy.model_registry import ModelRegistry
from my_custom_model import MyModelForCasualLM, MyPretrainedModel
from fastdeploy.config import ErnieArchitectures
def register():
if "MyModelForCasualLM" not in ModelRegistry.get_supported_archs():
if MyModelForCasualLM.name().startswith("Ernie"):
ErnieArchitectures.register_ernie_model_arch(MyModelForCasualLM)
ModelRegistry.register_model_class(MyModelForCasualLM)
ModelRegistry.register_pretrained_model(MyPretrainedModel)
```
Assuming you have a custom model_runner class `MyModelRunner`, you can write the following registration function:
```python
# File: fd_add_dummy_model_runner/__init__.py
from .my_model_runner import MyModelRunner
def get_runner():
return MyModelRunner
```
#### 2. Register Plugin in `setup.py`
@@ -36,11 +47,14 @@ from setuptools import setup
setup(
name="fastdeploy-plugins",
version="0.1",
packages=["fd_add_dummy_model"],
packages=["fd_add_dummy_model", "fd_add_dummy_model_runner"],
entry_points={
"fastdeploy.model_register_plugins": [
"fd_add_dummy_model = fd_add_dummy_model:register",
],
"fastdeploy.model_runner_plugins": [
"model_runner = fd_add_dummy_model:get_runner"
],
},
)
```