Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

Jiaqi Chen; Lujiao Shao; Haibin Zhou; Jianghong Ma; Weizhi Meng; Zenghui Wang; Haijun Zhang

doi:10.1007/978-981-19-6142-7_29

Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

Jiaqi Chen
, Lujiao Shao
, Haibin Zhou
, Jianghong Ma
, Weizhi Meng
, Zenghui Wang
, Haijun Zhang^*

^*Corresponding author for this work

University of South Africa
Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Abstract

This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

Original language	English
Title of host publication	Neural Computing for Advanced Applications
Editors	Haijun Zhang, Yuehui Chen, Xianghua Chu, Zhao Zhang, Tianyong Hao, Zhou Wu, Yimin Yang
Publisher	Springer
Publication date	2022
Pages	378-393
ISBN (Print)	9789811961410
DOIs	https://doi.org/10.1007/978-981-19-6142-7_29
Publication status	Published - 2022
Event	Third International Conference on Neural Computing for Advanced Applications - Jinan, China Duration: 8 Jul 2022 → 10 Jul 2022 Conference number: 3

Conference

Conference	Third International Conference on Neural Computing for Advanced Applications
Number	3
Country/Territory	China
City	Jinan
Period	08/07/2022 → 10/07/2022

Series	Communications in Computer and Information Science
Volume	1637 CCIS
ISSN	1865-0929

Keywords

Bayesian deep learning
Information extraction
Multi-modal feature fusion
Shopping receipt

Access to Document

10.1007/978-981-19-6142-7_29

OpenUrl availability

Check availability in DTU Findit

Cite this

Chen, J., Shao, L., Zhou, H., Ma, J., Meng, W., Wang, Z., & Zhang, H. (2022). Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. In H. Zhang, Y. Chen, X. Chu, Z. Zhang, T. Hao, Z. Wu, & Y. Yang (Eds.), Neural Computing for Advanced Applications (pp. 378-393). Springer. https://doi.org/10.1007/978-981-19-6142-7_29

Chen, Jiaqi ; Shao, Lujiao ; Zhou, Haibin et al. / Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. Neural Computing for Advanced Applications. editor / Haijun Zhang ; Yuehui Chen ; Xianghua Chu ; Zhao Zhang ; Tianyong Hao ; Zhou Wu ; Yimin Yang. Springer, 2022. pp. 378-393 (Communications in Computer and Information Science, Vol. 1637 CCIS).

@inproceedings{3565073691a542e2ba899a02246a4dcc,

title = "Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features",

abstract = "This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.",

keywords = "Bayesian deep learning, Information extraction, Multi-modal feature fusion, Shopping receipt",

author = "Jiaqi Chen and Lujiao Shao and Haibin Zhou and Jianghong Ma and Weizhi Meng and Zenghui Wang and Haijun Zhang",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; Third International Conference on Neural Computing for Advanced Applications, NCAA 2022 ; Conference date: 08-07-2022 Through 10-07-2022",

year = "2022",

doi = "10.1007/978-981-19-6142-7\_29",

language = "English",

isbn = "9789811961410",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "378--393",

editor = "Haijun Zhang and Yuehui Chen and Xianghua Chu and Zhao Zhang and Tianyong Hao and Zhou Wu and Yimin Yang",

booktitle = "Neural Computing for Advanced Applications",

}

Chen, J, Shao, L, Zhou, H, Ma, J, Meng, W, Wang, Z & Zhang, H 2022, Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. in H Zhang, Y Chen, X Chu, Z Zhang, T Hao, Z Wu & Y Yang (eds), Neural Computing for Advanced Applications. Springer, Communications in Computer and Information Science, vol. 1637 CCIS, pp. 378-393, Third International Conference on Neural Computing for Advanced Applications, Jinan, China, 08/07/2022. https://doi.org/10.1007/978-981-19-6142-7_29

Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. / Chen, Jiaqi; Shao, Lujiao; Zhou, Haibin et al.
Neural Computing for Advanced Applications. ed. / Haijun Zhang; Yuehui Chen; Xianghua Chu; Zhao Zhang; Tianyong Hao; Zhou Wu; Yimin Yang. Springer, 2022. p. 378-393 (Communications in Computer and Information Science, Vol. 1637 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

TY - GEN

T1 - Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

AU - Chen, Jiaqi

AU - Shao, Lujiao

AU - Zhou, Haibin

AU - Ma, Jianghong

AU - Meng, Weizhi

AU - Wang, Zenghui

AU - Zhang, Haijun

N1 - Conference code: 3

PY - 2022

Y1 - 2022

N2 - This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

AB - This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.

KW - Bayesian deep learning

KW - Information extraction

KW - Multi-modal feature fusion

KW - Shopping receipt

U2 - 10.1007/978-981-19-6142-7_29

DO - 10.1007/978-981-19-6142-7_29

M3 - Article in proceedings

AN - SCOPUS:85142766959

SN - 9789811961410

T3 - Communications in Computer and Information Science

SP - 378

EP - 393

BT - Neural Computing for Advanced Applications

A2 - Zhang, Haijun

A2 - Chen, Yuehui

A2 - Chu, Xianghua

A2 - Zhang, Zhao

A2 - Hao, Tianyong

A2 - Wu, Zhou

A2 - Yang, Yimin

PB - Springer

T2 - Third International Conference on Neural Computing for Advanced Applications

Y2 - 8 July 2022 through 10 July 2022

ER -

Chen J, Shao L, Zhou H, Ma J, Meng W, Wang Z et al. Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features. In Zhang H, Chen Y, Chu X, Zhang Z, Hao T, Wu Z, Yang Y, editors, Neural Computing for Advanced Applications. Springer. 2022. p. 378-393. (Communications in Computer and Information Science, Vol. 1637 CCIS). doi: 10.1007/978-981-19-6142-7_29