Questions play a central role as functional contexts for language use. As such they are relevant in a number of contexts: Questions support the interpretation of answers in a concrete language-based context. They make it possible to test knowledge, to verify whether someone has read a given text, or to explore the interpretations drawn from a given text. Questions can foster learning and they are central to assessment. In computational linguistics, the automatic generation of questions is an attractive challenge given the mix of function, meaning and grammatical characteristics that it involves. In this seminar, we survey different techniques for generating questions and their use cases.

Nature of course and our expectations: This is a research-oriented Hauptseminar, in which we jointly explore perspectives and approaches on complexity in linguistics, psycholinguistics, and computational linguistics. You are expected to

Academic conduct and misconduct: Research is driven by discussion and free exchange of ideas, motivations, and perspectives. So you are encouraged to work in groups, discuss, and exchange ideas. At the same time, the foundation of the free exchange of ideas is that everyone is open about where they obtained which information. Concretely, this means you are expected to always make explicit when you’ve worked on something as a team – and keep in mind that being part of a team always means sharing the work.

For text you write, you always have to provide explicit references for any ideas or passages you reuse from somewhere else. Note that this includes text “found” on the web, where you should cite the url of the web site in case no more official publication is available.

Class etiquette: Please do not read or work on materials for other classes in our seminar. All portable electronic devices such as cell phones and laptops should be switched off for the entire length of the flight, oops, class.

Note that the following session plan is subject to change; it only constitutes the current state of our planning as the semester unfolds.

References

Agarwal, M., R. Shah & P. Mannem (2011). Automatic question generation using discourse cues. In Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications. Portland, OR: Association for Computational Linguistics, pp. 1–9. URL http://aclweb.org/anthology/W11-1401.

Aldabe, I., M. Lopez de Lacalle, M. Maritxalar, E. Martínez & L. Uria (2006). ArikIturri: An Automatic Question Generator Based on Corpora and NLP Techniques. In M. Ikeda, K. Ashley & T.-W. Chan (eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS’06), Jhongli (Taiwan). Springer-Verlag, pp. 584–594. URL http://siuc01.si.ehu.es/~jibalari/.

Aldabe, I., M. Maritxalar & A. Soraluze (2011). Question Generation Based on Numerical Entities in Basque. In Proceedings of AAAI Symposium on Question Generation. URL http://siuc01.si.ehu.es/~jibalari/. To appear.

Araki, J., D. Rajagopal, S. Sankaranarayanan, S. Holm, Y. Yamakawa & T. Mitamura (2016). Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan: The COLING 2016 Organizing Committee, pp. 1125–1136. URL http://aclweb.org/anthology/C16-1107.

Asghar, N., P. Poupart, X. Jiang & H. Li (2017). Deep Active Learning for Dialogue Generation. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017). Vancouver, Canada: Association for Computational Linguistics, pp. 78–83. URL http://www.aclweb.org/anthology/S17-1008.

Becker, L., S. Basu & L. Vanderwende (2012). Mind the Gap: Learning to Choose Gaps for Question Generation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA, USA: Association for Computational Linguistics, NAACL HLT ’12, pp. 742–751. URL http://dl.acm.org/citation.cfm?id=2382029.2382150.

Becker, L., R. D. Nielsen & W. H. Ward (2009). What a pilot study says about running a question generation challenge. In Proceedings of the Second Workshop on Question Generation, AIED 2009. URL https://research.csc.ncsu.edu/intellimedia/AIED09-QG/Proceedings.QG-09.Rus-Lester.FullVersion.pdf.

Bernhard, D., L. D. Viron, V. Moriceau & X. Tannier (2012). Question Generation for French: Collating Parsers and Paraphrasing Questions. Dialogue & Discourse 3(2), 43–74. URL http://dad.uni-bielefeld.de/index.php/dad/article/view/2151/2833.

Brown, J., G. Frishkoff & M. Eskenazi (2005). Automatic Question Generation for Vocabulary Assessment. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing. Vancouver, British Columbia, Canada: Association for Computational Linguistics, pp. 819–826. URL http://aclweb.org/anthology/H05-1103.

Chali, Y. & S. A. Hasan (2012). Towards Automatic Topical Question Generation. In COLING. Mumbia, India, pp. 475–492.

Chali, Y. & S. A. Hasan (2015). Towards topic-to-question generation. Computational Linguistics URL http://aclanthology.coli.uni-saarland.de/pdf/J/J15/J15-1001.pdf.

Curto, S., A. C. Mendes & L. Coheur (2011). Exploring linguistically-rich patterns for question generation. In Proceedings of the UCNLG+Eval: Language Generation and Evaluation Workshop. Edinburgh, Scotland: Association for Computational Linguistics, pp. 33–38. URL http://www.aclweb.org/anthology/W11-2705.

Curto, S., A. C. Mendes & L. Coheur (2012). Question generation based on lexico-syntactic patterns learned from the web. Dialogue & Discourse 3(2), 147–175. URL https://doi.org/10.5087/dad.2012.207.

Du, X., J. Shao & C. Cardie (2017). Learning to Ask: Neural Question Generation for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vancouver, Canada: Association for Computational Linguistics, pp. 1342–1352. URL http://aclweb.org/anthology/P17-1123.

Garg, P. & E. C. S. Bedi (2013). Automatic question generation system from Punjabi text using hybrid approach. International Journal of Computer Trends and Technology (IJCTT) 21(3), 130–133.

Goyal, V., S. Garg & U. Singh (2013). System for Generating Questions Automatically from Given Punjabi Text. In LTC 2013: Human Language Technology. Challenges for Computer Science and Linguistics. Springer, vol. 9561 of Lecture Notes in Computer Science book series (LNCS).

Gütl, C., K. Lankmayr, J. Weinhofer & M. Hofler (2011). Enhanced Automatic Question Creator–EAQC: Concept, Development and Evaluation of an Automatic Test Item Creation Tool to Foster Modern e-Education. Electronic Journal of e-Learning 9(1), 23–38.

Heilman, M. (2011). Automatic factual question generation from text. Ph.D. thesis, Carnegie Mellon University. URL http://www.cs.cmu.edu/~ark/mheilman/questions/papers/heilman-question-generation-dissertation.pdf.

Heilman, M. & N. A. Smith (2010a). Extracting Simplified Statements for Factual Question Generation. In In Proceedings of the Third Workshop on Question Generation. URL https://core.ac.uk/download/pdf/18877.pdf#page=16.

Heilman, M. & N. A. Smith (2010b). Good Question! Statistical Ranking for Question Generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, HLT ’10, pp. 609–617. URL http://dl.acm.org/citation.cfm?id=1857999.1858085.

Hoshino, A. & H. Nakagawa (2005). WebExperimenter for Multiple-Choice Question Generation. In Proceedings of HLT/EMNLP 2005 Interactive Demonstrations. Vancouver, British Columbia, Canada: Association for Computational Linguistics, pp. 18–19. URL http://www.aclweb.org/anthology/H/H05/H05-2010.

Jouault, C. & K. Seta (2013). Building a semantic open learning space with adaptive question generation support. In Proceedings of the 21st International Conference on Computers in Education. Inderscience Publishers, pp. 41–50.

Jouault, C. & K. Seta (2014). Content-Dependent Question Generation for History Learning in Semantic Open Learning Space. In S. Trausan-Matu, K. E. Boyer, M. Crosby & K. Panourgia (eds.), Intelligent Tutoring Systems: 12th International Conference, ITS 2014, Honolulu, HI, USA, June 5-9, 2014. Proceedings, Cham: Springer International Publishing, pp. 300–305. URL https://doi.org/10.1007/978-3-319-07221-0_37.

Jouault, C., K. Seta & Y. Hayashi (2015). Quality of LOD Based Semantically Generated Questions. In C. Conati, N. Heffernan, A. Mitrovic & M. F. Verdejo (eds.), Artificial Intelligence in Education: 17th International Conference, AIED 2015, Madrid, Spain, June 22-26, 2015. Proceedings, Cham: Springer International Publishing, pp. 662–665. URL https://doi.org/10.1007/978-3-319-19773-9_86.

Kalady, S., A. Elikkottil & R. Das (2010). Natural language question generation using syntax and keywords. In Proceedings of QG2010: The Third Workshop on Question Generation. pp. 1–10. URL http://oro.open.ac.uk/22343/1/QG2010-Proceedings.pdf#page=5.

Kaur, J. & A. K. Bathla (2015). A review on automatic question generation system from a given hindi text. International Journal of Research in Computer Applications and Robotics IJRCAR 3(6).

Kolditz, T. (2015). Generating Questions for German Text. Master thesis in computational linguistics, Department of Linguistics, University of Tübingen.

Kumar, G., R. Banchs & L. F. D’Haro (2015). RevUP: Automatic Gap-Fill Question Generation from Educational Texts. In Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications. Denver, Colorado: Association for Computational Linguistics, pp. 154–161. URL http://www.aclweb.org/anthology/W15-0618.

Kunichika, H., T. Katayama, T. Hirashima & A. Takeuchi (2004). Automated question generation methods for intelligent English learning systems and its evaluation. In Proc. of ICCE. URL https://pdfs.semanticscholar.org/3147/93e319bd152b3648d0e23e99a05060b0de40.pdf.

Labutov, I., S. Basu & L. Vanderwende (2015). Deep Questions without Deep Understanding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Beijing, China: Association for Computational Linguistics, pp. 889–898. URL http://www.aclweb.org/anthology/P15-1086.

Le, N.-T. & N. Pinkwart (2015). Evaluation of a question generation approach using semantic web for supporting argumentation. Research and Practice in Technology Enhanced Learning 10(1), 3.

Lindberg, D., F. Popowich, J. Nesbit & P. Winne (2013). Generating Natural Language Questions to Support Learning On-Line. In Proceedings of the 14th European Workshop on Natural Language Generatio, Sofia, Bulgaria: ACL, p. 105–114.

Liu, M., R. A. Calvo & V. Rus (2010). Automatic question generation for literature review writing support. In International Conference on Intelligent Tutoring Systems. Springer, pp. 45–54. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.651.8729$\&$rep=rep1$\&$type=pdf.

Liu, M., R. A. Calvo & V. Rus (2012). G-Asks: An intelligent automatic question generation system for academic writing support. Dialogue & Discourse 3(2), 101–124. URL http://journals.linguisticsociety.org/elanguage/dad/article/download/1463/1463-5845-1-PB.pdf.

Liu, M., V. Rus & L. Liu (2017). Automatic Chinese factual question generation. IEEE Transactions on Learning Technologies 10(2), 194–204. URL http://doi.ieeecomputersociety.org/10.1109/TLT.2016.2565477.

Mannem, P., R. Prasad & A. Joshi (2010). Question generation from paragraphs at UPenn: QGSTEC system description. In Proceedings of QG2010: The Third Workshop on Question Generation. Pittsburgh, PA, pp. 84–91. URL http://oro.open.ac.uk/22343/1/QG2010-Proceedings.pdf.

Mazidi, K. & R. D. Nielsen (2014a). Linguistic Considerations in Automatic Question Generation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA: ACL, p. 321–326.

Mazidi, K. & R. D. Nielsen (2014b). Pedagogical evaluation of automatically generated questions. In International Conference on Intelligent Tutoring Systems. Springer, pp. 294–299.

Mazidi, K. & R. D. Nielsen (2015). Leveraging Multiple Views of Text for Automatic Question Generation. In Proceedings of the 17th International Conferenece on Artificial Intelligence in Education (AIED 2015). Cham, Switzerland: Springer International Publishing, pp. 257–266.

Mazidi, K. & P. Tarau (2016). Infusing NLU into Automatic Question Generation. In Proceedings of the 9th International Natural Language Generation conference. Edinburgh, UK: Association for Computational Linguistics, pp. 51–60. URL http://anthology.aclweb.org/W16-6609.

Mitkov, R., L. An Ha & N. Karamanis (2006). A computer-aided environment for generating multiple-choice test items. Natural Language Engineering 12(2), 177–194.

Mostafazadeh, N., I. Misra, J. Devlin, M. Mitchell, X. He & L. Vanderwende (2016a). Generating natural questions about an image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL. URL https://www.aclweb.org/anthology/P16-1170.

Mostafazadeh, N., I. Misra, J. Devlin, L. Zitnick, M. Mitchell, X. He & L. Vanderwende (2016b). Generating Natural Questions About an Image. CoRR abs/1603.06059. URL http://arxiv.org/abs/1603.06059.

Mostow, J. & W. Chen (2009). Generating Instruction Automatically for the Reading Strategy of Self-Questioning. In AIED. pp. 465–472. URL http://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/listen/pdfs/AIED2009-self-question-final-A4.pdf.

Olney, A., W. Cade & C. Williams (2011). Generating Concept Map Exercises from Textbooks. In Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications. Portland, Oregon: Association for Computational Linguistics, pp. 111–119. URL http://www.aclweb.org/anthology/W11-1414.

Olney, A. M., A. C. Graesser & N. K. Person (2012). Question Generation from Concept Maps. Dialogue & Discourse 3(2), 75–99. URL http://dad.uni-bielefeld.de/index.php/dad/article/view/1480/2832.

Pal, S., T. Mondal, P. Pakray, D. Das & S. Bandyopadhyay (2010). QGSTEC system description – JUQGG: A rule based approach. In Proceedings of QG2010: The Third Workshop on Question Generation. pp. 76–79. URL http://oro.open.ac.uk/22343/1/QG2010-Proceedings.pdf.

Piwek, P. & K. E. Boyer (2012). Varieties of Question Generation: Introduction to this Special Issue. Dialogue and Discourse 3(2), 1–9. URL http://dad.uni-bielefeld.de/index.php/dad/article/view/2886.

Piwek, P. & S. Stoyanchev (2010a). Question generation in the CODA project. In Proceeding of the Third Workshop on Question Generation. URL http://oro.open.ac.uk/22324/1/PiwekStoyanchevQG2010.pdf.

Piwek, P. & S. Stoyanchev (2010b). Generating expository dialogue from monologue: motivation, corpus and preliminary rules. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pp. 333–336. URL http://oro.open.ac.uk/20922/1/NAACL-CODA-final.pdf.

Ravichandran, D. & E. Hovy (2002). Learning Surface Text Patterns for a Question Answering System. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, ACL ’02, pp. 41–47. URL https://doi.org/10.3115/1073083.1073092.

Reddy, S., D. Raghu, M. M. Khapra & S. Joshi (2017). Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Valencia, Spain: Association for Computational Linguistics, pp. 376–385. URL http://www.aclweb.org/anthology/E17-1036.

Rus, V., B. Wyse, P. Piwek, M. Lintean, S. Stoyanchev & C. Moldovan (2010). The first question generation shared task evaluation challenge. In Proceedings of the 6th International Natural Language Generation Conference. Association for Computational Linguistics, pp. 251–257. URL https://www.aclweb.org/anthology/W10-4234.

Rus, V., B. Wyse, P. Piwek, M. Lintean, S. Stoyanchev & C. Moldovan (2012). A Detailed Account of The First Question Generation Shared Task Evaluation Challenge. Dialogue & Discourse 3(2), 177–204. URL http://dad.uni-bielefeld.de/index.php/dad/article/view/1474/2830.

Serban, I. V., A. García-Durán, C. Gulcehre, S. Ahn, S. Chandar, A. Courville & Y. Bengio (2016). Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, pp. 588–598. URL http://www.aclweb.org/anthology/P16-1056.

Skalban, Y., L. A. Ha, L. Specia & R. Mitkov (2012). Automatic Question Generation in Multimedia-Based Learning. In Proceedings of COLING 2012: Posters. Mumbai, India: The COLING 2012 Organizing Committee, pp. 1151–1160. URL http://www.aclweb.org/anthology/C12-2112.

Soleymanzadeh, K. (2017). Domain Specific Automatic Question Generation from Text. In Proceedings of ACL 2017, Student Research Workshop. Vancouver, Canada: Association for Computational Linguistics, pp. 82–88. URL http://aclweb.org/anthology/P17-3014.

Susanti, Y., R. Iida & T. Tokunaga (2015). Automatic Generation of English Vocabulary Tests. In CSEDU (1). pp. 77–87. URL http://www.cl.cs.titech.ac.jp/_media/publication/csedu_2015_72_cr.pdf.

Varga, A. & L. A. Ha (2010). WLV: A question generation system for the QGSTEC 2010 task b. In Proceedings of QG2010: The Third Workshop on Question Generation. pp. 80–83. URL http://oro.open.ac.uk/22343/1/QG2010-Proceedings.pdf.

Wyse, B. & P. Piwek (2009). Generating Questions from OpenLearn study units. In Proceedings of the 2nd Workshop on Question Generation, AIED. URL http://mcs.open.ac.uk/pp2464/inpress/Wyse-Piwek-QG-09.pdf.

Yao, X., G. Bouma & Y. Zhang (2012a). Semantics-based question generation and implementation. Dialogue & Discourse 3(2), 11–42. URL http://journals.linguisticsociety.org/elanguage/dad/article/download/1439/1439-5842-1-PB.pdf.

Yao, X., E. Tosch, G. Chen, E. Nouri, R. Artstein, A. Leuski, K. Sagae & D. Traum (2012b). Creating Conversational Characters Using Question Generation Tools. Dialogue & Discourse 3(2), 125–146. URL http://dad.uni-bielefeld.de/index.php/dad/article/view/1476/2831.

Yao, X. & Y. Zhang (2010). Question generation with minimal recursion semantics. In Proceedings of QG2010: The Third Workshop on Question Generation. Pittsburgh, PA: Citeseer, pp. 68–75. URL http://oro.open.ac.uk/22343/1/QG2010-Proceedings.pdf#page=5.

Zhou, Q., N. Yang, F. Wei, C. Tan, H. Bao & M. Zhou (2017). Neural Question Generation from Text: A Preliminary Study. arXiv preprint arXiv:1704.01792 URL https://arxiv.org/pdf/1704.01792.pdf.