机构:[1]Research Center for Graphic Communication, Printing and Packaging, and Institute of Artificial Intelligence,Wuhan University,Wuhan 430072, China[2]School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering,Wuhan University,Wuhan 430072, China[3]JD Explore Academy at JD.com, Beijing 100101, China[4]Affiliated Hospital, Kunming University of Science and Technology, Kunming 650032, China[5]School of Computer Science, Faculty of Engineering, The University of Sydney, Sydney, NSW 2006, Australia
The goal of aspect-based sentiment analysis (ABSA) is to determine the sentiment polarity towards an aspect. Because of the expensive and limited amounts of labelled data, the pretraining strategy has become the de facto standard for ABSA. However, there always exists a severe domain shift between the pretraining and downstream ABSA datasets, which hinders effective knowledge transfer when directly fine-tuning, making the downstream task suboptimal. To mitigate this domain shift, we introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline, that has both instance- and knowledge-level alignments. Specifically, we first devise a novel coarse-to-fine retrieval sampling approach to select target domain-related instances from the large-scale pretraining dataset, thus aligning the instances between pretraining and the target domains (First Stage). Then, we introduce a knowledge guidance-based strategy to further bridge the domain gap at the knowledge level. In practice, we formulate the model pretrained on the sampled instances into a knowledge guidance model and a learner model. On the target dataset, we design an on-the-fly teacher-student joint fine-tuning approach to progressively transfer the knowledge from the knowledge guidance model to the learner model (Second Stage). Therefore, the learner model can maintain more domain-invariant knowledge when learning new knowledge from the target dataset. In the Third Stage, the learner model is finetuned to better adapt its learned knowledge to the target dataset. Extensive experiments and analyses on several ABSA benchmarks demonstrate the effectiveness and universality of our proposed pretraining framework.
基金:
National Natural Science Foundation of China [62225113, 62076186]; National Key Research and Development Program of China [2021YFB2401302]
第一作者机构:[1]Research Center for Graphic Communication, Printing and Packaging, and Institute of Artificial Intelligence,Wuhan University,Wuhan 430072, China
共同第一作者:
通讯作者:
推荐引用方式(GB/T 7714):
Liu Juhua,Zhong Qihuang,Ding Liang,et al.Unified Instance and Knowledge Alignment Pretraining for Aspect-Based Sentiment Analysis[J].IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING.2023,31:2629-2642.doi:10.1109/TASLP.2023.3290431.
APA:
Liu, Juhua,Zhong, Qihuang,Ding, Liang,Jin, Hua,Du, Bo&Tao, Dacheng.(2023).Unified Instance and Knowledge Alignment Pretraining for Aspect-Based Sentiment Analysis.IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,31,
MLA:
Liu, Juhua,et al."Unified Instance and Knowledge Alignment Pretraining for Aspect-Based Sentiment Analysis".IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 31.(2023):2629-2642