Class GoogleDocs Folder
Assignment for April 24
1. Review the papers below
- Figure out how the sections map to the general outline I gave in class (and below). Note that it won't be exact.
- Read the abstract, into and conclusion for each and any other part that captures your interest. You don't have to read the whole paper unless it particularly interests you.
- Be prepared to talk about the mapping.
Outline
- abstract
- 1. Intro
- 2. Background/ Methodology / Related work / Bigger picture
- 3. Description of the work itself (often multiple sections)
- 4. Results (with numbers)
- 5. Conclusion
- References
Papers
Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory
Lynn Carlson, Daniel Marcu,Mary Ellen Okurowsk, 2004 ACL Workshop on Discourse Annotation
Image Annotation with ISO-Space: Distinguishing Content from Structure
James Pustejovsky and Zachary Yochum
Annotating Discourse Connectives And Their Arguments
Eleni Miltsakaki, Rashmi Prasad, Aravind Joshi, Bonnie Webber
Anaphoric Annotation in the ARRAU Corpus
Massimo Poesio, Ron Artstein
Discourse Annotation and Semantic Annotation in the GNOME Corpus
Massimo Poesio
Sentiment Analysis of Twitter Data
Apoorv Agarwal, Boyi Xie, Ilia Vovsha, Owen Rambow, Rebecca Passonneau
2. Find another ACL-like annotation paper that is related to your project
- Understand how its sections map into the broad outline
- Be prepared to give the "elevator pitch" of what it's about and why you thought it was interesting
- NOTE: No two people in a group can have the same paper (though they can be from the same author/group
3. Read the paper following paper
Cheap and Fast — But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks
Be prepared to discuss how the methods described might work (or not) on your project.
MAE
- MAE : https://code.google.com/p/mae-annotation/
- MAI : https://code.google.com/p/mai-adjudication/
- Keigh's mae and mai
- https://www.github.com/keighrim/mae-annotation
- https://www.github.com/keighrim/mai-adjudication
From Zach on Kappa:
1. An implementation of Fleiss’ kappa in Python I found on Google Code: https://code.google.com/p/hydrat/source/browse/src/hydrat/common/fleiss.py
IAA Starter code from Zach
- https://github.com/zyocum/IAA
2. My implementation of Cohen’s kappa in Python: https://github.com/zyocum/cohens_kappa
Additionally, I have a couple spreadsheets with the worked out kappa calculation examples from NLAML up on Google Docs. I can put these up in ‘view only’ mode on the class Google Drive as well.
Fleiss: https://docs.google.com/a/brandeis.edu/spreadsheet/ccc?key=0AhnYYMZRL22ZdG9odlprM3E5ZFE0QVo2RGctTkFVbXc&usp=sharing
Cohen: https://docs.google.com/a/brandeis.edu/spreadsheet/ccc?key=0AhnYYMZRL22ZdF81YXJBVExtb29LdUhDVnhHUzE0SXc&usp=sharing