- 无标题文档
查看论文信息

中文题名:

 基于自然语言处理的作文自动评分系统研究     

姓名:

 王川    

学号:

 1049721203161    

保密级别:

 公开    

论文语种:

 chi    

学科代码:

 0810    

学科名称:

 信息与通信工程    

学生类型:

 硕士    

学位:

 工学硕士    

学校:

 武汉理工大学    

获奖论文:

 校优秀硕士学位论文    

院系:

 信息工程学院    

专业:

 信息与通信工程    

研究方向:

 信息采集、传输与处理    

第一导师姓名:

 杨杰    

第一导师院系:

 武汉理工大学    

完成日期:

 2015-05-02    

答辩日期:

 2015-05-14    

中文关键词:

 

作文自动评分 ; 语法错误检测 ; 语言模型 ; 最大熵分类器

    

中文摘要:

作文自动评分系统是应用计算机技术对英文作文进行评分的系统。作文自动评分系统综合应用了统计学、自然语言处理、语言学及信息检索等领域的技术。目前国外作文评分系统(如E-rater)都得到了广泛运用。但是目前国内学者对英语作文自动评分系统的研究仍然处于初始阶段。随着在线教育的兴起,学生对知识掌握程度的衡量也需要自动化评测工具的帮助,因为传统的人工批改方式不再适用于拥有大量学生的在线教育。相比于人工评分,作文自动评分系统的评分更加快速,更加公正,更加经济。

本文首先基于EDX平台的开源项目研发了基础的作文评分系统。该系统将作文评分过程看作是文本分类过程,采用的分类器是梯度提升决策树分类器。但这个评分系统并不完美,一方面系统特征不能充分反映作文特征,另一方面可扩展性不好,为了添加新的作文题目,需要新的训练集和测试集来重新训练评分模型。国外优秀的作文自动评分系统综合考虑了作文的语法表达、语义内容和篇章结构等评分因素。语法错误数量是衡量语法表达好坏的重要标准,所以论文将作文语法错误检测作为研究重点。

本文随后研究了基于语言模型的语法检测方法。在该系统中,用SRILM工具搭建语言模型服务器。语言模型服务器可以用来查询N-gram词组的概率。根据单词词干为单词生成候选集,然后根据维特比算法求取最优词汇组合。若该词汇组合与原始句子不同,则认为检测出语法错误。但该方法也有缺陷,只能检测出替换类型错误,而无法检测出插入型错误和删除型错误。

本文还研究了冠词和介词错误检测,这两种错误是英语学习者最常犯的语法错误。模型训练集提取自大不列颠国家语料库。因为该语料库可以认为是没有语法错误的,所以初始训练过程无任何错误样本。为了使训练样本更接近英语学习者语言表达,本文在训练过程中添加了人工制造的错误样本,从而引入错误语法信息,提高了分类器对错误信息的敏感度。本文将语法错误检测过程视作分类任务,选取的分类器是对稀疏特征有着强大分类能力的最大熵分类器。从实验结果来看,冠词和介词错误检测系统性取得了与国外大学研究成果相当的结果。论文最后展望了以后的研究重点:语义分析与更多种类的语法错误检测。

参考文献:

[1]韩宁.几个英语自动评分系统的原理与评述[J].中国考试,2009,2009(3):38-44.

[2]梁茂成、文秋芳国外作文自动评分系统评述及启示[J],外语电化教学,2007,2007(5):18-24

[3]Page E B. Project Essay Grade: PEG [C]. In: Automated essay scoring: A cross-disciplinary perspective. Mahwah, United States, 2003, 43-54.

[4]Page E B. Grading Essays by Computer: Progress Report [C]. In: Notes from the 1966 Invitational Conference on Testing Problems, Princeton, United States, 1966, 87-100.

[5]Page, E. B. Computer grading of student prose [J], using modern concepts and software. Journal of Experimental Education, ?62, 127–142.

[6]Thomask. Launder,DarrellLatham, Peter Foltz, Automatic essay assessment [J], Assessment in Education, 2003,2003(10), 295-308

[7]Valenti S, Neri Fand Cucchiarelli A. An Overviewof Current Research on Automated Essay Grading [J]. Journal of Information Technology Education, 2003, 2: 319-330.

[8]LandauerTK,LahamD,andFoltzPW.AutomatedEssayScoring:ACrossDisciplinary Perspective[C].In: Automated Essay Scoring and Annotation of Essays with the Intelligent Essay Assessor, Mahwah, 2003,83-112.

[9]Foltz P W, Laham D. The Intelligent Essay Assessor: Application to Education Technology[C]. In: Interactive Multimedia Electronic Journal of Computer-Enhanced learning, Wake forest University, Unite State, 1999, 1(2). Retrieved from http://imej.wfu.edu./articles/1999/2/04

[10]Burstein K, Kukich S, Wolff C, Lu M, Chodorow L, Braden-Harder, and Harris M D. Automated Scoring Using A Hybrid Feature Identification Technique[C]. In: Proceedings of COLING-ACL'98, Montreal, Canada, 1998, 206-210.

[11]Attali Yigal, and Jill Burstein. Automated essay scoring with e-rater @ v.2.0 [J]. ETS Research Report Series 2004, no. 2 (2004): i-21.

[12]Wolfe M. B, Schreiner M.E, Rehder B et al.Learning from text: matching readers and text by Latent Semantic Analysis, Discourse Processes, 1998,25, 309–336.

[13]刘雷. 英语作文智能批改中语法检查的研究与实现[D]. 北京:北京邮电大学. 2013

[14]梁茂成.大规模考试英语作文自动评分系统的研制[M].北京:高等教育出版社.2011

[15]Ng H T, Wu S M, Ted Briscoe, et al. The conll-2014 shared task on grammatical error correction[C]. Eighteenth Conference on Computational Natural Language LearningProceedings of the Shared Task. 2014, 1-14.

[16]Naber Daniel. A rule-based style and grammar checker. 2003.

[17]Mariano Felice, Zheng Yuan, et al. Grammatical error correction using hybrid systems and type filtering[C] Eighteenth Conference on Computational Natural Language LearningProceedings of the Shared Task. 2014, 15-24.

[18]Rozovskaya, Alla, and Dan Roth. Building a State-of-the-Art Grammatical Error Correction System [J].Transactions of the Association for Computational Linguistics 2 (2014): 419-434.

[19]Han, Na-Rae, Martin Chodorow, and Claudia Leacock. Detecting Errors in English Article Usage with a Maximum Entropy Classifier Trained on a Large, Diverse Corpus.LREC. 2004.

[20]Izumi, Emi, Kiyotaka Uchimoto, and Hitoshi Isahara. SST speech corpus of Japanese learners’ English and automatic detection of learners’ errors[C]. ICAME, 2004. 28:31–48.

[21]Han, Na-Rae, Martin Chodorow, and Claudia Leacock. Detecting errors in English article usage by non-native speakers[J]. Natural Language Engineering, 2006. 12(1): 115–129.

[22]Chodorow, Martin, Joel R. Tetreault, and Na-Rae Han. Detection of grammatical errors involving prepositions[C]. In Proceedings of the fourth ACL-SIGSEM workshop on prepositions, Association for Computational Linguistics, 2007, 25-30.

[23]Gamon, Michael, Jianfeng Gao, Chris Brockett, Alexandre Klementiev, William B. Dolan, Dmitriy Belenko, and Lucy Vanderwende. Using Contextual Speller Techniques and Language Modeling for ESL Error Correction[C]. In IJCNLP, 2008. vol. 8, pp. 449-456.

[24]Yi, Xing, Jianfeng Gao, and William Dolan. A web-based English proofing system for ESL users[C]. In Proceedings of IJCNLP. 2008

[25]Wang, Longkai Zhang Houfeng. A Unified Framework for Grammar Error Correction[C]. CoNLL-2014. 2014. 96-102.

[26]Felice, Mariano, Zheng Yuan, ?istein E. Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. Grammatical error correction using hybrid systems and type filtering[C].CoNLL-2014 .2014. 15-24.

[27]Murata, M. and Nagao, M. Determination of referential property and number of nouns in Japanese sentences for machine translation into English[C]. In Proceedings of the 5th International Conference on Theoretical and Methodological Issues in Machine Translation, 1993. 218-225.

[28]Knight, K., and Chander, I. Automated postediting of documents. In Proceedings of the Twelfth National Conference on Artificial Intelligence[C]. AAAI Press, Menlo Park, CA. 1994. 779-784.

[29]Chodorow, Martin, and Claudia Leacock. An unsupervised method for detectinggrammatical errors[C]. In Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference. Association for Computational Linguistics, 2000.140-147

[30]Eeg-Olofsson, Jens, and Ola Knutsson. Automatic grammar checking for second language learners-the use of prepositions[C].Proceedings of NODALIDA. 2003.

[31]宗成庆. 统计自然语言处理. 2008.

[32]Agresti, Alan. A survey of exact inference for contingency tables[J]. Statistical Science (1992): 131-153.

[33]Atwell, Eric S., and Stephen Elliot. Dealing with ill-formed English text [J]. The Computational Analysis of English: A Corpus-Based Approach (1987): 120-138.

[34]Friedman, Jerome H. Greedy function approximation: a gradient boosting machine[J]. Annals of statistics (2001): 1189-1232.

[35]Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze: An Introduction to Information Retrieval [M]. Cambridge University Press, 2009.237-240

[36]R. Kneser and H. Ney, Improved backing-off for $m$-gram language modeling [C], Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, pp.181 -184 1995

[37]Stolcke, Andreas. SRILM-an extensible language modeling toolkit [J].INTERSPEECH. 2002.

[38]Porter, Martin. The Porter stemming algorithm[DB], 2005. See http://www. tartarus. org/~ martin/PorterStemmer.

[39]Forney Jr, G. David. The viterbi algorithm[C].Proceedings of the IEEE 61.3 1973: 268-278.

[40]Marshall, Robert C., and J. M. Gillespie. The tryptophan-rich keratin protein fraction of claws of the lizard Varanus gouldii [J]. Comparative Biochemistry and Physiology Part B: Comparative Biochemistry 71.4 (1982): 623-628.

[41]Bird, Steven. NLTK: the natural language toolkit [C]. Proceedings of the COLING/ACL on Interactive presentation sessions. Association for Computational Linguistics, 2006.

[42]Berger, Adam L., Vincent J. Della Pietra, and Stephen A. Della Pietra. A maximum entropy approach to natural language processing [J]. Computational linguistics 22.1 (1996): 39-71.

[43]Le, Zhang. Maximum entropy modeling toolkit for Python and C++. Natural Language Processing Lab, Northeastern University, China (2004).

[44]Leech, Geoffrey. 100 million words of English: the British National Corpus (BNC). Language Research 28.1 (1992): 1-13.

[45]Ng, Hwee Tou, et al. The conll-2013 shared task on grammatical error correction. Proceedings of CoNLL. 2013.

[46]Dale, R., I. Anisimoff, and G. Narroway. A report on the preposition and determiner error correction shared task [C]. Proc. of the NAACL HLT 2012 Seventh Workshop on Innovative Use of NLP for Building Educational Applications, Montreal, Canada, June. Association for Computational Linguistics. 2012.

[47]Tetreault, Joel R., and Martin Chodorow. The ups and downs of preposition error detection in ESL writing [C]. Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1. Association for Computational Linguistics, 2008.

[48]Chodorow, Martin, Joel R. Tetreault, and Na-Rae Han. Detection of grammatical errors involving prepositions[C]. In Proceedings of the fourth ACL-SIGSEM workshop on prepositions, pp. 25-30. Association for Computational Linguistics, 2007.

[49]Tetreault, Joel R., and Martin Chodorow. Native judgments of non-native usage: Experiments in preposition error detection [C]. Proceedings of the Workshop on Human Judgements in Computational Linguistics. Association for Computational Linguistics, 2008.

[50]Miller, George A. WordNet: a lexical database for English [J]. Communications of the ACM 38.11 (1995): 39-41.

[51]Miller, George A., et al. Introduction to wordnet: An on-line lexical database* [J].International journal of lexicography 3.4 (1990): 235-244.

中图分类号:

 TP311.52    

馆藏号:

 TP311.52/3161/2015    

备注:

 403-西院分馆博硕论文库;203-余家头分馆博硕论文库    

无标题文档

   建议浏览器: 谷歌 火狐 360请用极速模式,双核浏览器请用极速模式