SROIE Competition Results

ICDAR 2019 Robust Reading Competition on Scanned Receipts OCR and Information Extraction

In the following, we give the ranking table of each task using results evaluated under the given evaluation protocol. For a fair ranking, we make sure that every team will only present the latest result in the case of multiple submissions. We also filter out some informal submissions that have insufficient or meaningless method descriptions. You could contact us if you have other questions. We encourage the participants to submit sufficient information about your Team Name, Team Members and Insititute to us. This allows interested peers to recognize your efforts.

Hmean Ranking Table

Rank Team Name Team Members Insititute Recall Precision Hmean
1 SCUT-DLVC-Lab-Refinement Jiapeng Wang*, Yan Li*, Tianwei Wang, Jiaxin Zhang, Yichao Huang, Canjie Luo, Kai Ding, Lianwen Jin (*equal contribution) South China University of Technology， INTSIG Information Co. Ltd 98.64% 98.53% 98.59%
2 Ping An Property & Casualty Insurance Company Xianbiao Qi, Ning Lu, Yuan Gao, Yihao Chen, Shaoqiong Chen, Wenwen Yu, Rong Xiao Ping An Property & Casualty Insurance Company 98.60% 98.40% 98.50%
3 H&H Lab HUST_VLRGROUP(Mengde Xu, Zhen Zhu, Hui Zhang, Mingkun Yang, Jiehua Yang) & HUAWEI_CLOUD_EI(Jing Wang, Yibin Ye, Shenggao Zhu, Dandan Tu) Huazhong University of Science and Technology & Huawei Technologies Co. Ltd Joint Laboratory 97.93% 97.95% 97.94%
4 Psenet_dcn Xiufeng Jiang - 96.62% 96.21% 96.42%
5 BOE_IOT_AIBD v5 BOE_IOT_AIBD - 95.95% 95.99% 95.97%
6 EM_ocr Hao Wu, Na He, Zhou Shen, Dan Meng, Qingfeng Wang - 95.85% 96.08% 95.97%
7 Clova OCR Seung Shin, Sungrae Park, Seonghyeon Kim, Jaeheung Surh, Junyeop Lee, Hwalsuk Lee Clova AI Research, NAVER Corp 96.04% 95.79% 95.92%
8 IFLYTEK-textDet_v3 IFLYTEK IFLYTEK 93.77% 95.89% 94.81%
9 A Single-Shot Model for Robust Text Localization Hanqin Wang, Jie Qin, Fan Zhu, Li Liu, and Ling Shao Inception Institute of Artificial Intelligence 93.93% 94.80% 94.37%
10 SituTech_OCR Kui Lyu, Tianhao Tang, Minghao Wang SituTech 93.81% 94.18% 94.00%
11 SROIE Fourth Submission Yangliu - 92.98% 94.99% 93.97%
12 HeReceipt-Rotation Yan Li*, Jiapeng Wang*, Yichao Huang, Tianwei Wang, Jiaxin Zhang, Canjie Luo, Kai Ding, Lianwen Jin (*equal contribution) INTSIG Information Co. Ltd, South China University of Technology 93.87% 93.47% 93.67%
13 Pixellink multi-scale Detection PINGAN Tech PINGAN Tech 93.07% 92.84% 92.95%
14 BiLSTM Based on CTPN Zhuang Liu, Yuanping Zhu - 91.40% 94.03% 92.69%
15 CITlab Argus Textline Detection Tobias Grüning, Gundram Leifert, Jochen Zöllner, Tobias Strauß, Roger Labahn CITlab 92.02% 91.34% 91.68%
16 Textline detection Anh Duc Le, Dung Van Pham, Tuan Anh Nguyen - 89.85% 92.72% 91.26%
17 A Text Localization Method Based on CTPN Songyi Yang, Shengjie Xiu, Niansong Zhang - 85.23% 88.73% 86.94%
18 MCTPN2 Chao He, Zhen Zhu, Shaohui Ruan, Fengming Xie, Kai Dai, Haoqi Shi, Junguo Lu - 85.57% 87.65% 86.59%
19 Vsdnu Xiaohan Zhang - 85.07% 87.17% 86.11%
20 CTPN-SROIE Gordon - 81.14% 87.23% 84.07%
21 Yolov3_Autohome Liu Rong, Xu Chengshen, Huang Xiao, Li lin - 82.93% 83.67% 83.30%
22 ICA-IVA Xiangcheng Du, Xingjiao Wu, Shufan Wu, Shuchen Kong, Hao Ye, Yingbin Zheng, Liang He - 81.86% 76.43% 79.05%
23 YOLO Text Detector Zhaohanyue, Litong, Yuanzhibo, Daipengwen - 77.29% 79.32% 78.29%
24 Improved yolov3 model WangXiong，Yanqinqin，Chenyifan State Key Laboratory of Digital Publishing Technology，Beijing，PRC 68.52% 78.23% 73.06%
25 Task 1 - Scanned Receipt Text Localisation Homa Foroughi, Chang Liu, Tharathorn Joy Rimchala, Terrence J. Torres Intuit Inc. 71.14% 63.76% 67.25%
26 scene text detection weapon Zhangfei - 49.61% 64.75% 56.18%
27 Unet Segmentation and Watershed Roberto Lotufo, Israel Campiotti, Ramon Pires, Rubens Machado, Luis Serrano, Giovanni Garuffi - 56.31% 53.46% 54.85%
28 Receipt Info Extracting Task1 zone-dividing Shi Jiachen, Liang Yuhao, Ma Pingchuan - 32.62% 46.48% 38.33%

Hmean Ranking Table

Rank Team Name Team Members Insititute Recall Precision Hmean
1 H&H Lab HUST_VLRGROUP(Hui Zhang, Mingkun Yang, Mengde Xu, Zhen Zhu, Jiehua Yang) & HUAWEI_CLOUD_EI(Jing Wang, Yibin Ye, Shenggao Zhu, Dandan Tu) Huazhong University of Science and Technology & Huawei Technologies Co. Ltd Joint Laboratory 96.35% 96.52% 96.43%
2 HeReceipt-Ensemble Yichao Huang*, Tianwei Wang*, Jiaxin Zhang*, Yan Li, Jiapeng Wang, Canjie Luo, Kai Ding, Lianwen Jin (*equal contribution) INTSIG Information Co. Ltd, South China University of Technology 94.56% 95.10% 94.82%
3 Ping An Property & Casualty Insurance Company Xianbiao Qi, Yihao Chen, Shaoqiong Chen, Ning Lu, Yuan Gao, Wenwen Yu, Rong Xiao Ping An Property & Casualty Insurance Company 94.48% 94.86% 94.67%
4 CLOVA OCR Sungrae Park, Seung Shin, Seonghyeon Kim, Jaeheung Surh, Junyeop Lee, Hwalsuk Lee Clova AI Research, NAVER Corp 94.30% 94.88% 94.59%
5 SCUT-DLVC-Lab-Lexicon Tianwei Wang*, Jiaxin Zhang*, Yichao Huang*, Jiapeng Wang, Yan Li, Canjie Luo, Kai Ding, Lianwen Jin (*equal contribution) South China University of Technology， INTSIG Information Co. Ltd 94.18% 94.88% 94.53%
6 DenseNet-Attention Recognition PINGAN Tech PINGAN Tech 94.29% 94.58% 94.44%
7 CITlab Argus Text Recognition Tobias Grüning, Gundram Leifert, Jochen Zöllner, Tobias Strauß, Roger Labahn CITlab 93.55% 93.61% 93.58%
8 Unet followed by CRNN with CTC Roberto Lotufo, Ramon Pires - 88.58% 87.30% 87.93%
9 BOE_IOT_AIBD T2 V5 BOE_IOT_AIBD - 87.84% 86.66% 87.24%
10 CRNN after UNet Segmentation Roberto Lotufo, Ramon Pires, Israel Campiotti, Rubens Machado, Luis Serrano, Giovanni Garuffi - 85.77% 86.48% 86.12%
11 BiLstmCtcIgnoreSpaces-Segment Shen He, Meng Wu - 83.38% 87.37% 85.33%
12 IFLYTEK-textRec_v4 IFLYTEK IFLYTEK 80.63% 81.72% 81.17%
13 SituTech_OCR Tianhao Tang, Kui Lyu, Minghao Wang SituTech 77.81% 80.10% 78.94%
14 Attention based OCR Anh Duc Le, Dung Van Pham, Tuan Anh Nguyen - 70.87% 72.27% 71.57%
15 CTPN_CRNN caisiqi - 35.75% 63.89% 45.85%
16 A Text Extraction Method Based on Modified CRNN Songyi Yang, Shengjie Xiu, Niansong Zhang - 26.33% 72.53% 38.63%
17 BiLSTM+ctc Zhuang Liu, Yuanping Zhu - 28.75% 49.69% 36.42%
18 CRNN_Pytorch_BiCTC jzw - 21.43% 45.65% 29.16%
19 ICA-IVA Xiangcheng Du, Xingjiao Wu, Shufan Wu, Shuchen Kong, Hao Ye, Yingbin Zheng, Liang He - 19.48% 40.32% 26.27%
20 Task 2 - Scanned Receipt OCR Homa Foroughi, Chang Liu, Tharathorn Joy Rimchala, Terrence J. Torres Intuit Inc. 1.05% 3.96% 1.66%
21 Receipt Info Extracting Task2 letter-recognizing Shi Jiachen, Liang Yuhao, Ma Pingchuan - 0.24% 0.97% 0.38%
22 CRNN_Autohome Liu Rong, Xu Chengshen, Huang Xiao, Li lin - 0.00% 0.00% 0.00%
23 VIL Xiaohan Zhang - 0.00% 0.00% 0.00%

Hmean Ranking Table

Rank Team Name Team Members Insititute Recall Precision Hmean
1 Ping An Property & Casualty Insurance Company Xianbiao Qi, Wenwen Yu, Ning Lu, Yihao Chen, Shaoqiong Chen, Yuan Gao, Rong Xiao Ping An Property & Casualty Insurance Company 90.49% 90.49% 90.49%
2 EAST det + Multi-class classification liyulin, v_huangju, xiequnyi, qinxiameng Baidu 89.70% 89.70% 89.70%
3 H&H Lab HUST_VLRGROUP(Hui Zhang, Mengde Xu, Mingkun Yang, Zhen Zhu, Jiehua Yang) & HUAWEI_CLOUD_EI(Jing Wang, Yibin Ye, Shenggao Zhu, Dandan Tu) Huazhong University of Science and Technology & Huawei Technologies Co. Ltd Joint Laboratory 89.63% 89.63% 89.63%
4 CLOVA OCR Sungrae Park, Seonghyeon Kim, Seung Shin, Jaeheung Surh, Junyeop Lee, Hwalsuk Lee Clova AI Research, NAVER Corp 89.05% 89.05% 89.05%
5 NiuBiHongHong Ge daye - 87.61% 87.61% 87.61%
6 HeReceipt-withoutRM Hanmin Duan, Zhiqin Lu, Yang Chang, Yan Li, Yichao Huang, Kai Ding INTSIG Information Co. Ltd, South China University of Technology 83.00% 83.24% 83.12%
7 BOE_IOT_AIBD_v3 BOE_IOT_AIBD - 82.71% 82.71% 82.71%
8 PATECH_CHENGDU_OCR JunKun Zhou, Ming Guan, ZhengNan Luo, MingTao Wang, YuBin Xiao, MingBin Hou - 81.70% 82.29% 82.00%
9 NER with spaCy model Roberto Lotufo, Giovani Garuffi, Israel Campiotti, Ramon Pires, Rubens Machado, Luis Serrano - 78.96% 79.02% 78.99%
10 CITlab Argus Information Extraction (positional & line features, enhanced gt) Tobias Strauß, Tobias Grüning, Gundram Leifert, Jochen Zöllner, Roger Labahn CITlab 77.38% 77.38% 77.38%
11 A Simple Method for Key Information Extraction as Character-wise Classification with LSTM Songyi Yang, Shengjie Xiu, Niansong Zhang - 75.58% 75.58% 75.58%
12 Location-aware BERT model for Text Information Extraction Hanqin Wang, Jie Qin, Fan Zhu, Li Liu, and Ling Shao Inception Institute of Artificial Intelligence 74.42% 74.42% 74.42%
13 BERT with Multi-task Confidence Prediction T.J. Torres, Chang Liu, Homa Foroughi, Tharathorn Joy Rimchala - 66.14% 66.14% 66.14%
14 With receipt framing veryfi team - 63.04% 63.54% 63.29%
15 IFLYTEK-textNLP_v2 IFLYTEK IFLYTEK 61.24% 61.24% 61.24%
16 SituTech_OCR Kui Lyu, Tianhao Tang, Minghao Wang SituTech 59.01% 62.38% 60.64%
17 Key Information Extraction from Scanned Receipts Liu Rong, Xu Chengshen, Huang Xiao, Li Lin - 28.75% 36.31% 32.09%

Organizers


onlyyou hust cvc nayang IIIT Aston Sjtu csig