[Other] faster_tokenizer->fast_tokenizer (#636)

* faster_tokenizer->fast_tokenizer

* ErnieFasterTokenizer->ErnieFastTokenizer

* update the fastdeploy_init

Co-authored-by: Jason <jiangjiajun@baidu.com>
This commit is contained in:
Jack Zhou
2022-11-21 13:45:00 +08:00
committed by GitHub
parent 3e1fc69a0c
commit eeae48deff
14 changed files with 170 additions and 175 deletions

View File

@@ -15,7 +15,7 @@ import os
import distutils.util
import numpy as np
import faster_tokenizer
import fast_tokenizer
from paddlenlp.transformers import AutoTokenizer
import fastdeploy as fd