Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
徐永飞 2e991ff9ed | 5 months ago | |
---|---|---|
.gitee | 3 years ago | |
.jenkins/test/config | 5 months ago | |
cmake | 5 months ago | |
docs | 6 months ago | |
example | 1 year ago | |
mindspore_serving | 6 months ago | |
scripts | 3 years ago | |
tests | 8 months ago | |
third_party | 5 months ago | |
.clang-format | 3 years ago | |
.gitignore | 2 years ago | |
.gitmodules | 3 years ago | |
CMakeLists.txt | 2 years ago | |
LICENSE | 3 years ago | |
NOTICE | 3 years ago | |
OWNERS | 2 years ago | |
README.md | 6 months ago | |
README_CN.md | 6 months ago | |
RELEASE.md | 5 months ago | |
RELEASE_CN.md | 5 months ago | |
Third_Party_Open_Source_Software_Notice | 2 years ago | |
build.sh | 2 years ago | |
requirements_test.txt | 1 year ago | |
setup.py | 5 months ago |
MindSpore Serving is a lightweight and high-performance service module that helps MindSpore developers efficiently
deploy online inference services in the production environment. After completing model training on MindSpore, you can
export the MindSpore model and use MindSpore Serving to create an inference service for the model.
MindSpore Serving architecture:
MindSpore Serving includes two parts: Client
and Server
. On a Client
node, you can deliver inference service
commands through the gRPC or RESTful API. The Server
consists of a Main
node and one or more Worker
nodes.
The Main
node manages all Worker
nodes and their model information, accepts user requests from Client
s, and
distributes the requests to Worker
nodes. Servable
is deployed on a worker node, indicates a single model or a
combination of multiple models and can provide different services in various methods. `
On the server side, when MindSpore is used as the inference backend,, MindSpore Serving
supports the Ascend 910 and Nvidia GPU environments. When MindSpore Lite is
used as the inference backend, MindSpore Serving supports Ascend 310/310P, Nvidia GPU and CPU environments. Client` does
not depend on specific hardware platforms.
MindSpore Serving provides the following functions:
batch size
requirement of the model.For details about how to install and configure MindSpore Serving, see the MindSpore Serving installation page.
MindSpore-based Inference Service Deployment is
used to demonstrate how to use MindSpore Serving.
For more details about the installation guide, tutorials, and APIs,
see MindSpore Python API.
Welcome to MindSpore contribution.
A lightweight and high-performance service module that helps MindSpore developers efficiently deploy online inference services in the production environment.
C++ Python CMake Shell Markdown other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》