Abstract:

Language data collected in electronic corpora can in principle provide important empirical insights for theoretical and computational linguistics. For theoretical linguistics, corpus examples can be used to validate or falsify linguistic generalizations. In computational linguistics, language models and classifiers can be trained on corpus data to learn how to predict or classify previously unseen data on that basis.

Effective querying of corpora for specific phenomena and the development of computational tools for the automatic analysis of language often requires reference to annotations. Annotations essentially function as an index to classes of data which cannot easily be identified based on the surface form alone. For example, finding all sentences containing modal verbs using only the surface forms is possible, but would require a long list of all forms of the modal verbs. Even so, sentences where, for example, “can” is not actually a modal verb (as in “Pass me a can of beer” or “I can tuna for a living”) would be wrongly identified. Other search patterns, such as a query for all sentences containing past participle verbs, cannot even be specified in finite form using the surface string alone. The annotation of corpora thus serves an important function in providing abstractions which make it possible to access or generalize over large sets of examples.

This seminar will provide an overview of the creation and use of linguistically annotated corpora in theoretical and computational linguistics. It will include basic questions such as how to tokenize or sentence segment a corpus as well as conceptual considerations relevant to the creation of annotation schemes, and will then explore different types of corpora (from newspaper to learner corpora) and different types of annotations (morphological, constituency, dependency, semantic and formal pragmatic).

Instructor: Prof. Dr. Detmar Meurers

• Office: Room 1.28, Blochbau (Wilhelmstr. 19)
• Email: dm@sfs.uni-tuebingen.de
• Office hours: Wednesdays 10:00–11:00 (please arrange slot by email beforehand)

Course meets: 4 SWS in Seminarraum 1.13, Blochbau (Wilhelmstr. 19)

• Wednesdays, 8:30-10:00
• Fridays, 8:30-10:00

Credit Points:

• Core Computatonal Linguistics Hauptseminar Schein with 9 CP

Online syllabus: http://purl.org/dm/14/ws/hs

Moodle page: https://moodle02.zdv.uni-tuebingen.de/course/view.php?id=980

If you have not already used this Moodle installation for another course, please log onto it asap and create an account for yourself using your ordinary ZDV university login, then enroll into our course.

Nature of course and my expectations: This is a Hauptseminar intended to provide an overview of the key issues and annotation schemes in this active research area. Each participant is expected to

1. regularly and actively participate in class, read the papers assigned by any of the presenters and post a question on Moodle to the“Reading Discussion Forum” on each reading at the latest the day before it is discussed in class. (30% of grade)

   Note: According to the rules of the Fakultät missing more than two meetings unexcused, automatically results in failing the class.

2. explore and present a topic (30% of grade):
   • select one of the sub-topics (by October 27)
   • thoroughly research the topic, taking my literature pointers as a starting point
   • prepare the presentation with slides and send me a copy during the week before the presentation
   • start a new Moodle thread on the “Reading Discussion Forum” specifying what every course participant should read to prepare for your presentation a week before your presentation
   • Present the topic in class. Presentations should include a short (10 minute) in-class activity, e.g., something of the form “Analyze these five sentences using the annotation scheme presented” or a unix-based exercise (for which you should ask everyone to bring laptops or reserve the department lab)
3. for a Hauptseminar Schein, work out a term paper (40% of grade for Hauptseminar):
   • select a topic and submit a one page abstract by Friday, Feb 6.
     • The term paper topic must be related to the seminar topic.
     • Comparisons of two linguistic annotation schemes make good term paper topics (e.g., comparing TIGER and TüBA-D/Z annotation).
   • email me the term paper (as pdf) by the beginning of the next semester.
     • Note for ISCL students: The term paper must be produced in LaTeX, using the Computational Linguistics journal style (http://cljournal.org/style.html) and BibTex must be used for the bibliography.

Academic conduct and misconduct: Research is driven by discussion and free exchange of ideas, motivations, and perspectives. So you are encouraged to work in groups, discuss, and exchange ideas. At the same time, the foundation of the free exchange of ideas is that everyone is open about where they obtained which information. Concretely, this means you are expected to always make explicit when you’ve worked on something as a team – and keep in mind that being part of a team always means sharing the work.

For text you write, you always have to provide explicit references for any ideas or passages you reuse from somewhere else. Note that this includes text “found” on the web, where you should cite the url of the web site in case no more official publication is available.

Class etiquette: Please do not read or work on materials for other classes in our seminar. Come to class on time and do not pack up early. When our seminar meets in the computer lab, only use the computers when you are asked to do a specific activity – do not read email or browse the web. All portable electronic devices such as cell phones should be switched off for the entire length of the flight – oops – class. If for some reason, you must leave early or you have to miss class for an important reason, please let me know before class.

Session plan:

• 24.10. Overview and historical development [Detmar Meurers]
  • Reading: Garside et al. (1997, ch 1), McEnery & Wilson (1996, ch. 1 & ch 2)
• 29.10. On the use of corpora
  • Reading: Meurers (2005); Meurers & Müller (2009)
• 31.10. Text processing with the command line interface [Veit Rückert]
  • Reading: (Lu 2014, ch. 2)
• 5.11. Tokenization and sentence segmantation [Andreia Rauber]
• 7.11. → SfS colloq fest in the afternoon
• 12.11. POS and morphological annotation overview and hands-on exploration [Spyridoula Georgatou]
  • Reading: (Lu 2014, ch. 3)
• 14.11. no meeting (Detmar in Uppsala)
• 19.11. lexical analysis (Lu 2014, ch. 4) [Anastasia Didkovska]
• 21.11. Which and how many tags? (Elworthy 1995; Brants 1995; Kübler & Wagner 2000; Dienes & Oravecz 2000; Tufiş et al. 2000; Teufel et al. 1996) [Kathrin Adlung]
• 26.11. A POS Tagset for German (STTS) and the challenge of tagging Computer-Mediated Communication [Sabrina Galasso]
  • Reading: (Schiller et al. 1995; Thielen & Schiller 1996; Zinsmeister & und Kathrin Beck 2013)
• 28.11. Part-of-Speech Analysis of learner language (Díaz Negrillo et al. 2010) [Eduard Schaf]
• 3.12.
  • Tagset comparision: a case study for Brazilian Portuguese [Björn Rudzewitz]
• 5.12.
  • pos-annotation of Greek [Spyridoula Georgatou]
  • tokenization and pos-annotation for Japanese [Stanislav Reichert]
    • Reading: related work for Chinese (Lu 2006)
• 10.12. Syntax overview and hands-on exploration [Johanna Heininger]
  • Reading: (Lu 2014, ch. 5)
• 12.12. → Treebanks and Linguistic Theory Workshop
• 17.12.
  • Annotation scheme (re)design to maximize quality (Voutilainen & Järvinen 1995; Sampson & Babarczy 2003) [Alina Ladygina]
• 19.12.
  • Inter-annotator agreement (Brants & Skut 1998; Artstein & Poesio 2009) [Julia Koch]
  • Detecting Errors in Corpus Annotation: The variation n-gram approach (Dickinson & Meurers 2003a,b, 2005a,b; Boyd et al. 2008) [Thomas Fecker]
• 7.1. Using nltk for corpus annotation and the TIMIT corpus [Holger Muth-Hellebrandt]
  • Lopes and Perdigão
  • NLTK book, esp. ch. 11 http://www.nltk.org/book/ch11.html
• 9.1.
  • Penn Treebank [Roshanak Hamidi]
  • Dependency treebanks for English (PARC 700) (and possibly pointers to German, TiGerDB) [Ben Campbell]
• 14.1. Topological field-based annotation of German [Zarah Weiss, David Bausch]
  • German TüBa-D/Z (Telljohann et al. 2005; Stegmann et al. 2000; Cheung & Penn 2009)
• 16.1. Syntactic treebanks for other languages
  • Turkish treebank [Leyla Adiguezel]
  • French treebank [Anastasia Gorbunova]
• 21.1.
  • Annotating historical corpora [Kameliya Hristova]
• 23.1. ⇒Division of Labor Workshop
• 28.1.
  • Persian treebank [Shahrzad Kananizadeh]
  • Querying syntactically parsed corpora [Yevgen Karpenko]
    • Reading: (Lu 2014, ch. 6)
• 30.1.
  • Querying syntactically parsed corpora (continued)
  • A case study using Tündra and TübaD/Z [Eyal Schejter]
• 4.2. Annotating spoken language corpora
  • Overview, segmental and suprasegmental annotation, using Praat [Vivian Fresen, Sabine Koban]
• 6.2. Semantic, pragmatic, and discourse analysis
  • Lu (2014, ch. 7) [Daniela Stier]
• 11.2. A case study using syntactic and semantic annotation for paraphrase analysis [Ido Freeman]
• 13.2.
  • Propbank (Kingsbury et al. 2002; Palmer et al. 2005) and RST annotation [Alla Münch]

Topics we can chose from

We focus on the conceptual issues, in particular questions relating to linguistic modeling. Which properties and insights can be and have been identified and annotated in corpora?

• Historical and overview Garside et al. (1997, ch1), McEnery & Wilson (1996, ch.1 & ch2)
• On the use of corpora (Meurers 2005; Meurers & Müller 2009)
• Tokenization and Sentence Segmentation (Grefenstette & Tapanainen 1994; Grefenstette 1999; Palmer 2000)
• Text Processing with the Command Line Interface (Lu 2014, ch.2)
• Annotation types:
  • Part-of-Speech (and morphological analysis): (Leech 1997; Cloeren 1999; Brill 2000)
    • overview and hands-on exploration (Lu 2014, ch.3)
    • Which and how many tags? (Elworthy 1995; Brants 1995; Kübler & Wagner 2000; Dienes & Oravecz 2000; Tufiş et al. 2000; Teufel et al. 1996)
    • tagset comparison: (Teufel 1995; Déjean 2000; Atwell et al. 1994, 2000a,b)
    • tagsets for different languages
      • English: WSJ Tagset (Santorini 1990), CLAWS 5/7, Susanne
      • German: STTS (Schiller et al. 1995; Thielen & Schiller 1996)
      • Eastern European: Hungarian (Váradi & Oravecz 1999), Slovene (Džeroski et al. 2000), Czech PDT Tagset and morphological analysis (Hajič & Hladká 1998; Hana & Zeman 2005), MULTEXT-East
      • different sources of evidence (Díaz Negrillo et al. 2010)
  • Syntax
    • overview and hands-on exploration (Lu 2014, ch.5)
    • constituency-based, with function labels
      • English: Penn Treebank (Marcus et al. 1993, 1994; Bies et al. 1995; Gaizauskas 1995; Taylor et al. 2003)
      • German: TiGer/Negra
      • other languages: Chinese, Arabic, French (Abeillé et al. 2003), Turkish (Oflazer et al. 1999, 2003; Atalay et al. 2003)…
    • dependency treebanks:
      • German TiGerDB (Forst et al. 2004)
      • Swedish Talbanken (Nivre et al. 2006)
      • Prague Dependency Treebank, PDT (Hajič et al. 2003, 2001; Hajičová et al. 2000; Hajič et al. 1999)
      • English PARC 700 Dependency Bank (King et al. 2003)
    • topological field-based
      • German TüBa-D/Z (Telljohann et al. 2005; Stegmann et al. 2000; Cheung & Penn 2009)
  • semantics (e.g., Propbank Kingsbury et al. 2002; Palmer et al. 2005)
    coreference annotation, word sense annotation
  • information and discourse structure (Potsdam SFB, RST annotation)
  • spoken language corpora: phonetic transcription, intonation
• How to query a corpus: grep (regular expressions), cqp, TigerSearch, Annis, Linguist’s search engine, Tündra
  • querying syntactically parsed corpora (Lu 2014, ch.6)
  • lexical analysis using unix tools (frequency lists, etc.) (Lu 2014, ch.5)
• Ensuring and measuring annotation quality:
  • Inter-annotator agreement (Brants & Skut 1998; Artstein & Poesio 2009)
  • Annotation scheme (re)design to maximize quality (Voutilainen & Järvinen 1995; Sampson & Babarczy 2003)
  • Variation n-gram approach (Dickinson & Meurers 2003a,b, 2005a,b; Boyd et al. 2008)
• Other relevant issues which we will only touch in passing:
  • compiling a corpus (representativeness, legal aspect, etc.)
  • corpus encoding formats (vertical format, XML, etc.)
  • corpus encoding standards (TEI, XCES, TUSNELDA, etc.)
  • determining the relevant corpus properties for a given machine learning task (feature selection)

References