English | 简体中文 | हिन्दी | 日本語 | 한국인 | Pу́сский язы́к
安装 | 使用文档 | 快速开始 | API文档 | 更新日志
⚡️FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具, 支持云边端部署。提供超过 🔥160+ Text,Vision, Speech和跨模态模型📦开箱即用的部署体验,并实现🔚端到端的推理性能优化。包括 物体检测、字符识别(OCR)、人脸、人像扣图、多目标跟踪系统、NLP、Stable Diffusion文图生成、TTS 等几十种任务场景,满足开发者多场景、多硬件、多平台的产业部署需求。
FastDeploy系列直播课程回放
2023.01.17 发布 YOLOv8 在FastDeploy系列硬件的部署支持。 其中包括 Paddle YOLOv8 以及 社区 ultralytics YOLOv8
服务化部署结合VisualDL新增支持可视化部署。在FastDeploy容器中启动VDL服务后,即可在VDL界面修改模型配置、启动/管理模型服务、查看性能数据、发送请求等,详细操作可参考相关文档
✨👥✨ 社区交流
Slack:Join our Slack community and chat with other community members about ideas
微信:扫描二维码,填写问卷加入技术社区,与社区开发者交流部署产业落地痛点问题
X86_64 CPU | |
||||||
NVDIA GPU | |||||||
飞腾 CPU | |||||||
昆仑芯 XPU | |||||||
华为昇腾 NPU | |||||||
Graphcore IPU | |||||||
算能 | |||||||
Intel 显卡 | |||||||
Jetson | |||||||
ARM CPU | |||||||
RK3588等 | |||||||
RV1126等 | |||||||
晶晨 | |||||||
恩智浦 |
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
更多部署案例请参考模型部署示例 .
符号说明: (1) ✅ : 已经支持; (2) ❔: 正在进行中; (3) N/A : 暂不支持.
任务场景 | 模型 | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | X86 CPU | NVIDIA GPU | X86 CPU | NVIDIA GPU | X86 CPU | Arm CPU | AArch64 CPU | 飞腾D2000 aarch64 | NVIDIA Jetson | Graphcore IPU | 昆仑芯 XPU | 华为 昇腾 | Serving |
Classification | PaddleClas/ResNet50 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | TorchVison/ResNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Classification | ultralytics/YOLOv5Cls | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNetv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/EfficientNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/GhostNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/ShuffleNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SqueeezeNetV1.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/Inceptionv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-HGNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | 🔥🔥PaddleDetection/PP-YOLOE+ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | 🔥PaddleDetection/YOLOv8 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | 🔥ultralytics/YOLOv8 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | PaddleDetection/PicoDet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/YOLOv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLO | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLOv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/Faster-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Mask-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | Megvii-BaseDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOv7 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_ort | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | meituan/YOLOv6 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ |
Detection | ultralytics/YOLOv5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | WongKinYiu/YOLOR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ✅ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ? | ❔ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Perception | Paddle3D/Smoke | ❔ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
KeyPoint | PaddleDetection/TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
HeadPose | omasaht/headpose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-LiteSeg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/HRNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Unet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
FaceDetection | biubug6/RetinaFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | insightface/SCRFD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | Single430/FaceLandmark1000 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | jhb86253817/PIPNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/VPL | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Matting | PaddleSeg/ModNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/BasicVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/EDVR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/PP-MSVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | |
NLP | PaddleNLP/ERNIE-3.0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ✅ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -- | ❔ | ❔ | ✅ |
任务场景 | 模型 | 大小(MB) | Linux | Android | Linux | Linux | Linux | Linux | Linux | 更新中... |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | 瑞芯微NPU RK3588/RK3568/RK3566 |
瑞芯微NPU RV1109/RV1126/RK1808 |
晶晨NPU A311D/S905D/C308X |
恩智浦NPU i.MX 8M Plus |
更新中... | |
Classification | PaddleClas/ResNet50 | 98 | ✅ | ✅ | ✅ | ✅ | ||||
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | -- | |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Detection | PaddleDetection/PicoDet_s | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -- | |
Detection | YOLOv5 | ❔ | ❔ | ✅ | ❔ | ❔ | ❔ | -- | ||
Face Detection | deepinsight/SCRFD | 2.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | -- | |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ✅ | ✅ | |||||
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ❔ | -- | -- | -- | -- | |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
任务场景 | 模型 | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
本项目中SDK生成和下载使用了EasyEdge中的免费开放能力,在此表示感谢。
FastDeploy遵循Apache-2.0开源协议。
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
1. 开源生态
2. 协作、人、软件
3. 评估模型