| 234 | 1 | 30 |
| 下载次数 | 被引频次 | 阅读次数 |
针对现有命名实体识别模型缺乏对局部信息的关注，同时在处理长距离依赖关系和复杂序列数据时存在不足的问题，提出一种结合对抗训练的双通道特征融合的命名实体识别模型。首先，使用预训练模型提取特征，通过对抗训练增强模型的鲁棒性和泛化性。其次，在双向长短期记忆网络后引入多头注意力和时间步构成的双通道模块，在捕捉全局和局部信息的同时提升模型对长距离依赖和复杂序列数据的处理能力。最后，利用GlobalPointer和旋转位置编码识别实体的头尾信息，在减少计算成本的同时进一步增强对全局信息的捕捉能力。实验结果表明，模型在五个公共数据集上的F1值相较于其他模型均有一定的提升，验证了其在获取全局和局部信息及处理复杂序列数据的能力。
Abstract：Existing named entity recognition(NER) models often lack focus on local information and face challenges in handling long-range dependencies and complex sequence data. To address these issues, a dual-channel feature fusion NER model incorporating adversarial training was proposed. First, features were extracted using a pretrained model, with adversarial training enhancing robustness and generalization. Then, a dual-channel module, comprising multi-head attention and time steps, was introduced after a bidirectional long short-term memory network to capture both global and local information, improving the model′s ability to manage long-range dependencies and complex sequences. Finally, GlobalPointer and rotary position encoding were employed to identify the head and tail information of entities, reducing computational costs while further enhancing the model′s ability to capture global information. Experimental results showed that the proposed model achieved improvements in F1 scores across five public datasets, validating its effectiveness in capturing both global and local information and handling complex sequence data.
[1] HOSSAIN M S,NAYLA N,RASSEL A A.Product market demand analysis using NLP in Banglish text with sentiment analysis and named entity recognition[C]//2022 56th Annual Conference on Information Sciences and Systems.Piscataway:IEEE Press,2022:166-171.
[2] NASSIRI K,AKHLOUFI M.Transformer models used for text-based question answering systems[J].Applied intelligence,2023,53(9):10602-10635.
[3] ABDOUN N,CHAMI M.Automatic text classification of PDF documents using NLP techniques[J].INCOSE international symposium,2022,32(1):1320-1331.
[4] SCHÄFER H,IDRISSI-YAGHIR A,HORN P,et al.Cross-language transfer of high-quality annotations:combining neural machine translation with cross-linguistic span alignment to apply NER to clinical texts in a low-resource language[C]//Proceedings of the 4th Clinical Natural Language Processing Workshop.Stroudsburg:Association for Computational Linguistics,2022:53-62.
[5] WESTON L,TSHITOYAN V,DAGDELEN J,et al.Named entity recognition and normalization applied to large-scale information extraction from the materials science literature[J].Journal of chemical information and modeling,2019,59(9):3692-3702.
[6] LI J Y,FEI H,LIU J,et al.Unified named entity recognition as word-word relation classification[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2022:10965-10973.
[7] 李明键，李卫军，王海荣.融合关联信息与CNN的实体识别研究[J].郑州大学学报(理学版),2023,55(5):53-59.LI M J,LI W J,WANG H R.Research on entity recognition based on related information and CNN[J].Journal of Zhengzhou university (natural science edition),2023,55(5):53-59.
[8] ZHANG Y N,CHEN Q C.A neural span-based continual named entity recognition model[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2023:13993-14001.
[9] CUI Y M,CHE W X,LIU T,et al.Pre-training with whole word masking for Chinese BERT[J].IEEE/ACM transactions on audio,speech,and language processing,2021,29:3504-3514.
[10] DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Cambridge:MIT Press,2019:4171-4186.
[11] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[EB/OL].(2023-08-02)[2024-07-05].https://doi.org/10.48550/arXiv.1706.03762.
[12] MIYATO T,DAI A M,GOODFELLOW I.Adversarial training methods for semi-supervised text classification[EB/OL].(2016-05-25)[2024-07-16].https://arxiv.org/abs/1605.07725v4.
[13] MADRY A,MAKELOV A,SCHMIDT L,et al.Towards deep learning models resistant to adversarial attacks[EB/OL].(2019-09-04)[2024-07-19].https://arxiv.org/abs/1706.06083v4.
[14] GOODFELLOW I J,SHLENS J,SZEGEDY C,et al.Explaining and harnessing adversarial examples[EB/OL].(2015-03-20)[2024-07-17].https://arxiv.org/abs/1412.6572v3.
[15] SU J L,MURTADHA A,PAN S F,et al.Global pointer:novel efficient span-based approach for named entity recognition[EB/OL].(2022-08-05)[2024-07-19].https://arxiv.org/abs/2208.03054v1.
[16] WANG T B,HUANG R Y,HU N,et al.Chinese named entity recognition method based on dictionary semantic knowledge enhancement[J].IEICE transactions on information and systems,2023,106(5):1010-1017.
[17] SHAO Q,XIAO B,CHEN Q,et al.Chinese name entity recognition with label semantics[C]//IEEE International Conference on Network Intelligence and Digital Content.Piscataway:IEEE Press,2023:1-5.
[18] 林令德，刘纳，徐贞顺，等.基于多层动态融合的中文医疗命名实体识别[J].计算机工程与应用，2024,60(15):161-169.LIN L D,LIU N,XU Z S,et al.Chinese medical named entity recognition based on multi-layer dynamic fusion[J].Computer engineering and applications,2024,60(15):161-169.
[19] 李明键，李卫军，王海荣.融合词汇信息与GlobalPointer的实体识别[J].山东大学学报(工学版),2024,54(1):91-99.LI M J,LI W J,WANG H R.Entity recognition based on lexicon information and GlobalPointer[J].Journal of Shandong university (engineering science),2024,54(1):91-99.
基本信息:
DOI：10.13705/j.issn.1671-6841.2024175
中图分类号:TP391.1
引用信息:
[1]李卫军,丁建平,刘雪洋,等.结合对抗训练的双通道特征融合命名实体识别[J].郑州大学学报(理学版),2026,58(04):44-51.DOI:10.13705/j.issn.1671-6841.2024175.
基金信息:
宁夏高等学校科学研究项目(NYG2024086); 中央高校基本科研业务费(2022PT＿S04); 国家自然科学基金项目(62066038,61962001)
2025-02-27
2025-02-27
2025-02-27