mirror of
https://github.com/PaddlePaddle/FastDeploy.git
synced 2025-10-28 02:42:21 +08:00
79c2c52756
Some checks failed
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
Publish Job / publish_pre_check (push) Has been cancelled
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Publish Job / FD-Clone-Linux (push) Has been cancelled
Publish Job / Show Code Archive Output (push) Has been cancelled
Publish Job / BUILD_SM8090 (push) Has been cancelled
Publish Job / BUILD_SM8689 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Publish Job / Run Base Tests (push) Has been cancelled
Publish Job / Run Accuracy Tests (push) Has been cancelled
Publish Job / Run Stable Tests (push) Has been cancelled
CI Images Build / FD-Clone-Linux (push) Has been cancelled
CI Images Build / Show Code Archive Output (push) Has been cancelled
CI Images Build / CI Images Build (push) Has been cancelled
CI Images Build / BUILD_SM8090 (push) Has been cancelled
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
CI Images Build / Run Base Tests (push) Has been cancelled
CI Images Build / Run Accuracy Tests (push) Has been cancelled
CI Images Build / Run Stable Tests (push) Has been cancelled
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
DeepGEMM Pre-compilation Tool
This tool provides pre-compilation functionality for DeepGEMM kernels to optimize performance.
Usage
1. Using Shell Script (Recommended)
bash pre_compile.sh \
[MODEL_PATH] \
[TP_SIZE] \
[EP_SIZE] \
[HAS_SHARED_EXPERTS] \
[OUTPUT_FILE]
The script will:
- Generate configurations
- Pre-compile all kernels
2. Alternative: Manual Steps
If you need more control, you can run the steps manually:
Generate Configuration
python generate_config.py \
--model /path/to/model \
--tensor-parallel-size [TP_SIZE] \
--expert-parallel-size [EP_SIZE] \
--has-shared-experts [True/False] \
--output [CONFIG_FILE]
Arguments:
--model: Path to model directory containing config.json--tensor-parallel-size: Tensor parallel size (default: 1)--expert-parallel-size: Expert parallel size (default: 8)--has-shared-experts: Whether model has shared experts (default: False)--output: Output config file path (default: ./deep_gemm_pre_compile_config.jsonl)
Pre-compile Kernels
python pre_compile.py \
--config-file [CONFIG_FILE] \
--expert-parallel-size [EP_SIZE] \
--num-threads [NUM_THREADS]
Arguments:
--config-file: Path to config file generated in step 1--expert-parallel-size: Expert parallel size (must match step 1)--num-threads: Number of compilation threads (default: CPU cores)
Environment Variables
PRE_COMPILE_LOG_LEVEL: Set log level (DEBUG/INFO/WARNING/ERROR)DG_CACHE_DIR: Cache directory for compiled kernels (default: ./deep_gemm_cache)
Notes
- For best performance, set
--num-threadsto the number of available CPU cores - The compilation process may take significant time depending on configuration size
- Compiled kernels will be cached in
DG_CACHE_DIR