Bolt supports ARM GPU inference with OpenCL.
But building OpenCL kernel function from source code and selecting optimal algorithm takes up a lot of time.
They can be optimized by preparing the OpenCL binary function library and algorithm file in advance.
Inference can directly use prepared files.
Bolt provides offline tool preprocess_ocl to reduce GPU prepare time.
We have test mobilenet_v1 model on ARM MALI G76 GPU. Prepare time can be reduced from 2-3s to 60ms after building OpenCL binary kernel and algorithm file.
Here we give an example:
<1> Connect target device by using Andriod ADB;
<2> Convert your models to xxx.bolt by using X2bolt;
<3> Create a directory on target device, copy all your needed xxx.bolt models into it, E.g:
adb shell "mkdir /data/local/tmp/preprocess_bolt_models"
adb shell "cp ${boltModelDir}/*.bolt /data/local/tmp/preprocess_bolt_models"
<4> Set essential command line arguments for shell script preprocess_ocl:
adb devices
for example:
./build_preprocess_ocl.sh --device 435bc850 --target android-aarch64 -d /data/local/tmp/preprocess_bolt_models
<5> Run build_preprocess_ocl.sh on host;
After running build_preprocess_ocl.sh successfully, OpenCL binary kernel shared library libxxx_map.so will be produced.
All needed kernels for your models has been compiled from sources to bins,
and packaged into libxxx_map.so, such as ${BOLT_ROOT}/inference/engine/tools/preprocess_ocl/lib/libMali_G76p_map.so
adb shell "mkdir /data/local/tmp/kits"
adb push install_arm_llvm/kits/classification /data/local/tmp/kits
adb push tools/preprocess_ocl/lib/libMali_G76p_map.so /data/local/tmp/kits
adb shell "cd /data/local/tmp/kits && export LD_LIBRARY_PATH=./ && ./classification -m ./mobilenet_v1_f16.bolt -a GPU"
Package kernel binary dynamic library into your project, and put it in libbolt.so directory.
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》