Note: 1.Inference time is tested using MNN official Test Tool with scorethreshold 0.2 And 0.7849 is the original tensorflow result. 2.All MAP results are evaluated using the first 300 testing images in order to save time. 3.-quant model is quantized using official MNN tool. The poor inference speed is due to arm-specified optimization. Check this.
Model
InputSize
Thread
Inference(ms)
Params
MAP(VOC)
Yolov3
544
2/4
112/75.1
26M
0.7803(0.7849)
Yolov3
320
2/4
38.6/24.2
26M
0.7127(0.7249)
Yolov3-quant
320
2/4
316.2/225.2
6.7M
0.7082(0.7249)
Important Notes during model converting
Replace v3/model/head/build_nework with build_nework_MNN, which replaces tf.shape with static inputshape and replace
[:, tf.newaxis] -> tf.expand_dims // currently strided_slice op is not very well supported in MNN.
2. Following this issue to remove/replace some op. 3. Remove condition op which is related to BatchNormalization and training Flag. Otherwise it will cause MNN converting failure.Identity's input node num. != 1
Update: 2019-9-24 Don't bother to adjust op carefully. Just follow this
to replace nn.batch_normalization with nn.fused_batch_norm. After this
modification we can also merge BN,Relu into convolution directly in MNN.
Qualitative Comparison
Testing Result in Tensorflow(top), MNN(middle), and Android phone(bottom).
TODO
Speed analyse.
Model Quantization.
Op Integration. (BN,Relu->Convolution)
Android Support.
Channel Pruning/ Weight Sparsification ... (Update: 2019-10-26 see stronger-yolo-pytorch for more detail)