| 235 | 0 | 67 |
| 下载次数 | 被引频次 | 阅读次数 |
领域自适应面临现实场景复杂多变的问题，且现有的方法大多注重优化分类的一致性，而忽略了分类的确定性。针对上述问题，提出一种结合对比语言-图像预训练(constrastive language-image pre-training, CLIP)与分类确定性最大化的网络模型。CLIP作为一个多模态预训练模型，通过对大规模的图像-文本对进行预训练，具有强大的跨域泛化能力。通过提示学习和对比学习获取CLIP模型的知识，使模型适应更多的复杂现实场景。通过分类确定性最大化的方法，采用双分类器评估分类的一致性，减少模型在推理过程中的混淆。在Office-31、Office-Home和MiniDomainNet三个领域自适应基准数据集上进行实验，结果表明，与现有的先进方法相比，所提模型在三个数据集上的图像分类精确度均有提升。
Abstract：Domain adaptation faced the issue of complex and variable real-world scenarios, and existing methods mostly focused on optimizing classification consistency while neglecting classification certainty. To address these issues, a network model combining constrastive language-image pre-training(CLIP) with classification certainty maximization was proposed. CLIP, as a multimodal pre-trained model, was pre-trained on a large scale of image-text pairs and possessed strong cross-domain generalization capabilities. By leveraging prompt learning and contrastive learning, the knowledge of the CLIP model was acquired, enabling the model to adapt more complex real-world scenarios. Through the method of classification certainty maximization, a dual-classifier was employed to assess classification consistency and reduce confusion during the model′s inference process. Experiments were conducted on three domain adaptation benchmark datasets: Office-31, Office-Home, and MiniDomainNet. The experimental results indicated that compared with existing advanced methods, the proposed model showed improvements in image classification accuracy across all three datasets.
[1] RADFORD A,KIM J W,HALLACY C,et al.Learning transferable visual models from natural language supervision[C]//Proceedings of the 38th International Conference on Machine Learning.New York:ACM Press,2021:8748-8763.
[2] 黄露，曾庆山.基于平衡概率分布和实例的迁移学习算法[J].郑州大学学报(理学版),2020,52(3):55-61.HUANG L,ZENG Q S.Balanced distribution adaptation and instance based transfer learning algorithm[J].Journal of Zhengzhou university (natural science edition),2020,52(3):55-61.
[3] TZENG E,HOFFMAN J,ZHANG N,et al.Deep domain confusion:maximizing for domain invariance[EB/OL].(2014-12-10) [2024-06-01].https://doi.org/10.48550/arXiv.1412.3474.
[4] GAO Z Q,ZHANG S F,HUANG K Z,et al.Gradient distribution alignment certificates better adversarial domain adaptation[C]//IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE Press,2021:8917-8926.
[5] PETRONI F,ROCKTÄSCHEL T,RIEDEL S,et al.Language models as knowledge bases?[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2019:2463-2473.
[6] LIU P F,YUAN W Z,FU J L,et al.Pre-train,prompt,and predict:a systematic survey of prompting methods in natural language processing[J].ACM computing surveys,2023,55(9):1-35.
[7] POERNER N,WALTINGER U,SCHÜTZE H.E-BERT:efficient-yet-effective entity embeddings for BERT[EB/OL].(2019-11-09) [2024-06-01].https://doi.org/10.48550/arXiv.1911.03681.
[8] JIANG Z B,XU F F,ARAKI J,et al.How can we know what language models know?[J].Transactions of the association for computational linguistics,2020,8:423-438.
[9] SHIN T,RAZEGHI Y,LOGAN R L,et al.AutoPrompt:eliciting knowledge from language models with automatically generated prompts[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2020:4222-4235.
[10] ZHOU K Y,YANG J K,LOY C C,et al.Learning to prompt for vision-language models[J].International journal of computer vision,2022,130(9):2337-2348.
[11] GE C J,HUANG R,XIE M X,et al.Domain adaptation via prompt learning[J].IEEE transactions on neural networks and learning systems,2023:1-11.
[12] LUO Y W,ZHENG L,GUAN T,et al.Taking a closer look at domain shift:category-level adversaries for semantics consistent domain adaptation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2019:2502-2511.
[13] LI S,LV F R,XIE B H,et al.Bi-classifier determinacy maximization for unsupervised domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2021:8455-8464.
[14] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:770-778.
[15] VASWANI A,SHAZEER N,PAEMAR N,et al.Attention is all you need[C]//Proceedings of IEEE Conference on Neural Information Processing Systems.Piscataway:IEEE Press,2017:5998-6008.
[16] VAPNIK V N.The nature of statistical learning theory[M].Berlin:Springer Press,1999.
[17] TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial discriminative domain adaptation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:2962-2971.
[18] LONG M,CAO Z,WANG J,et al.Conditional adversarial domain adaptation[EB/OL].(2018-12-29)[2024-06-01].https://doi.org/10.48550/arXiv.1705.10667.
[19] JIN Y,WANG X M,LONG M S,et al.Minimum class confusion for versatile domain adaptation[M]//Lecture Notes in Computer Science.Cham:Springer International Publishing,2020:464-480.
[20] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[M]//Advances in Computer Vision and Pattern Recognition.Cham:Springer International Publishing,2017:189-209.
[21] ZHANG Y F,WANG X,LIANG J,et al.Free lunch for domain adversarial training:environment label smoothing[EB/OL].(2023-02-01)[2024-06-18].https://arxiv.org/pdf/2302.00194.
[22] RANGWANI H,AITHAL S K,MISHRA M,et al.A closer look at smoothness in domain adversarial training[C]//Proceedings of the International Conference on Machine Learning.New York:ACM Press,2022:18378-18399.
[23] ZHANG Y,LIU T,LONG M,et al.Bridging theory and algorithm for domain adaptation[C]//Proceedings of the International Conference on Machine Learning.New York:ACM Press,2019:7404-7413.
[24] ACUNA D,ZHANG G,LAW M T,et al.f-Domain adversarial learning:theory and algorithms[C]//Proceedings of the International Conference on Machine Learning.New York:ACM Press,2021:66-75.
[25] WESTFECHTEL T,YEH H W,MENG Q E,et al.Backprop induced feature weighting for adversarial domain adaptation with iterative label distribution alignment[C]//IEEE/CVF Winter Conference on Applications of Computer Vision.Piscataway:IEEE Press,2023:392-401.
[26] XU R J,CHEN Z L,ZUO W M,et al.Deep cocktail network:multi-source unsupervised domain adaptation with category shift[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2018:3964-3973.
[27] SAITO K,WATANABE K,USHIKU Y,et al.Maximum classifier discrepancy for unsupervised domain adaptation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2018:3723-3732.
[28] SAITO K,KIM D,SCLAROFF S,et al.Semi-supervised domain adaptation via minimax entropy[C]//IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE Press,2019:8049-8057.
[29] ZHOU K Y,YANG Y X,QIAO Y,et al.Domain adaptive ensemble learning[J].IEEE transactions on image processing,2021,30:8008-8018.
[30] ZHANG J,QI L,SHI Y H,et al.DomainAdaptor:a novel approach to test-time adaptation[C]//IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE Press,2023:18925-18935.
基本信息:
DOI：10.13705/j.issn.1671-6841.2024147
中图分类号:TP391.41;TP18
引用信息:
[1]丁美荣,卓金鑫,刘庆龙,等.融合提示学习与分类确定性最大化的领域自适应[J].郑州大学学报(理学版),2026,58(02):25-32.DOI:10.13705/j.issn.1671-6841.2024147.
基金信息:
国家自然科学基金面上项目(62176162); 广东省自然科学基金项目(2022A1515140099,2023A1515012875)
2025-01-13
2025-01-13
2025-01-13