An error is reported when using MKLDNN to run a quantize model

[yeliang2258]

bug描述 Describe the Bug

When I deploy a quantize yolov6 model use MKLDNN，An error is reported.

My script and model: https://github.com/PaddlePaddle/PaddleTest/tree/develop/inference/python_api_test/test_int8_model
You can download model and data use following commend:

sh prepare.sh

Open the test_yolo_series_infer.py file and modify the configuration to enable_mkldnn_int8(), then run YoloV6 model inference:

python test_yolo_series_infer.py --model_path=models/yolov6s_quant --device=CPU --use_mkldnn=True --cpu_threads=10 --arch=YOLOv6 --precision=int8

Open the test_segmentation_infer.py file and modify the configuration to enable_mkldnn_int8(), then run PP-Liteseg model inference:

python test_segmentation_infer.py --model_path=models/pp_liteseg_qat --dataset='cityscape' --dataset_config=configs/cityscapes_1024x512_scale1.0.yml --device=CPU --use_mkldnn=True --precision=int8

After reading onednn's documentation, I found that, in addition to concat, the OP with multiple inputs should have the problem of reporting an error when the input type is inconsistent.

其他补充信息 Additional Supplementary Information

No response

[paddle-bot]

您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue、AI社区来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ，Github Issue and AI community to get the answer.Have a nice day!

[wozna]

This PR #46378 fixes the error with the concat OneDNN exception. But unfortunately, now this test throws a different error that is caused because no detection boxes were found.

Traceback (most recent call last):
  File "test_yolo_series_infer.py", line 493, in <module>
    main()
  File "test_yolo_series_infer.py", line 479, in the main
    eval(predictor, val_loader, anno_file, rerun_flag=rerun_flag)
  File "test_yolo_series_infer.py", line 390, in eval
    map_res = coco_metric(anno_file, bboxes_list, bbox_nums_list, image_id_list)
  File "/home/wozna/PaddleTest/inference/python_api_test/test_int8_model/utils/post_process.py", line 190, in coco_metric
    coco_dt = coco_gt.loadRes(output)
  File "/home/wozna/.local/lib/python3.8/site-packages/pycocotools/coco.py", line 329, in loadRes
    if 'caption' in anns[0]:
IndexError: list index out of range

After excluding the 4 last concats from the quantization test works fine with good accuracy.
I'll continue work to find why the quantization of last concats decreased acc so much.

[yaomichael]

@wozna does #48872 fix the error you mentioned in #45970 (comment) ?

@yeliang2258 are you ok to close this issue?

[wozna]

@wozna does #48872 fix the error you mentioned in #45970 (comment) ?

@yeliang2258 are you ok to close this issue?

Yes #48872 fixed this error