| 120 | 3 | 278 |
| 下载次数 | 被引频次 | 阅读次数 |
基于同义词词林提出一种语义空间变换算法,并将其应用于平滑BLEU中,提出一种改进的基于同义词词林的BLEUS评测方法,该方法针对候选译文中短译文或英文缩写可能导致一元语法零匹配的情况,对传统BLEUS的n元语法均进行了平滑处理,并且以参考译文的一元语法为标准,对候选译文进行语义空间变换.在俄汉双语句子数据集上对谷歌、百度、必应、有道在线翻译系统的俄汉翻译输出译文进行评测,改进方法与传统BLEUS的评测结果一致;基于同义词词林的BLEUS提升传统BLEUS的评测性能,使得百度的NBLEUS值提高了3.99%,谷歌提高了7.66%,必应提高了11.15%,有道提高了4.65%.与此同时,验证了基于同一类型评测方法的纵向比较方法的有效性.
Abstract：A new algorithm based on thesaurus of Cilin was put forward with the name statistical space transformation( SST). And then it was applied into traditional smoothed BLEU( BLEUS). And an improved smoothed BLEU was got based on thesaurus of Cilin. As many cases of short translations or English abbreviations in candidate translations might cause unigram without matches,this new evaluation metric smoothed the traditional BLEUS n-gram and made the candidate translation unigram "synonymy match"based on thesaurus of Cilin,and took the reference translations unigrams as standard. Exact match and synonymy match were applied in unigram matching. Experiments were performed in Russian and Chinese bilingual sentence data set,and it evaluated the output translations of online translation system such as Google,Baidu,Bing and Youdao. The evaluation results of Cilin-based BLEUS and traditional BLEUS were proved to be consistent. Cilin-based BLEUS could greatly enhance the traditional BLEUS evaluation performance. NBLEUSvalue of the Baidu improveed 3. 99 percent,Google improved 7. 66 percent,Bing improved 11. 15 percent,and Youdao improved 4. 65 percent. Experiments were performed on the longitudinal comparisons to evaluate the metrics with different parameter settings based on the same measurement. And the results were consistent with the results of the human evaluation.
[1]中华人民共和国教育部国家语言文字工作委员会.机器翻译系统评测规范:GF-2006[S].2006.
[2]YU S.Automatic evaluation of output quality for machine translation systems[J].Machine translation,1993,8(1):117-126.
[3]PAPINEN K,ROUKOS S,WARD T,et al.BLEU:a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.Philadelphia,2002:311-318.
[4]LIN C Y,OCH F J.Orange:method for evaluating automatic evaluation metrics for machine translation[C]//Proceedings of the International Committee on Computational Linguistics 2004.Barcelona,2004.
[5]LIN C Y.Rouge:package for automatic evaluation of summaries[C]//Proceedings of Workshop on Text Summarization Branches out,Post-conference Workshop of Association for Computational Linguistics 2004.Barcelona,2004.
[6]BANERJEE S,LAVIE A.Meteor:an automatic metric for MT evaluation with improved correlation with human judgments[C]//ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization.Michigan,2005.
[7]梅家驹,竺一鸣,高蕴琦,等.同义词词林[M].2版.上海:上海辞书出版社,1996.
[8]CHEN B,CHERRY C.A systematic comparison of smoothing techniques for sentence-level BLEU[C]//Proceedings of the 9th Workshop on Statistical Machine Translation on Association for Computational Linguistics 2014.Baltimore,2014:362-367.
[9]张钹.自然语言处理的计算模型[J].中文信息学报,2007,21(3):3-7.
[10]田久乐,赵蔚.基于同义词词林的词语相似度计算方法[J].吉林大学学报(信息科学版),2010,28(6):602-608.
[11]徐建民,刘清江.基于同义词关系的局部查询扩展[J].郑州大学学报(理学版),2010,42(1):45-48.
[12]DU W,LIU W,YU J,et al.Russian-Chinese sentence-level aligned news corpus[C]//Proceedings of the 18th Annual Conference of the European Association for Machine Translation.Antalya,2015:213.
[13]姚建民,周明,赵铁军,等.基于句子相似度的机器翻译评价方法及其有效性分析[J].计算机研究与发展,2004,41(7):1258-1265.
基本信息:
DOI：10.13705/j.issn.1671-6841.2016307
中图分类号:TP391.2
引用信息:
[1]于俊婷,何宏业,刘伍颖,等.基于同义词词林的平滑BLEU研究[J],2017,49(02):55-60.DOI:10.13705/j.issn.1671-6841.2016307.
基金信息:
国家语言文字工作委员会重点项目(ZDI135-26);; 广东省高校特色创新项目(2015KTSCX035)
2016-11-10
2016
2016-12-05
2016
1