ACL 2005 CD | WORKSHOP ON CD | ACL 2005 ONLINE | ACL ONLINE


Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

WORKSHOP PROGRAM

Wednesday, June 29, 2005

8:45–8:50Opening Remarks
 Session 1: Summarization Metrics I
8:50–9:15A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate?
Bonnie Dorr, Christof Monz, Stacy President, Richard Schwartz and David Zajic
9:15–9:40On the Subjectivity of Human Authored Summaries
BalaKrishna Kolluru and Yoshihiko Gotoh
 Session 2: MT Metrics I
9:40–10:05Preprocessing and Normalization for Automatic Evaluation of Machine Translation
Gregor Leusch, Nicola Ueffing, David Vilar and Hermann Ney
10:05–10:30Syntactic Features for Evaluation of Machine Translation
Ding Liu and Daniel Gildea
10:30–11:00Break
 Session 3: Invited Talk
11:00–12:00Kathy McKeown on Results of the Multilingual Summarization Evaluation
 Session 4: Student Session - Work in Progress
12:00–12:15Evaluation of Sentence Selection on Spoken Dialogue by Xiaodan Zhu
12:15–12:30Toward a Predictive Statistical Model of Task-based Performance by Callandra R. Tate
12:30–2:15Lunch
 Session 5: Summarization Metrics II
2:15–2:40Evaluating Automatic Summaries of Meeting Recordings
Gabriel Murray, Steve Renals, Jean Carletta and Johanna Moore
2:40–3:05Evaluating Summaries and Answers: Two Sides of the Same Coin?
Jimmy Lin and Dina Demner-Fushman
3:05–3:30Evaluating DUC 2004 Tasks with the QARLA Framework
Enrique Amigó, Julio Gonzalo, Anselmo Peñas and Felisa Verdejo
 Session 6: MT Metrics II
4:00–4:25On Some Pitfalls in Automatic Evaluation and Significance Testing for MT
Stefan Riezler and John T. Maxwell
4:25–4:50METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments
Satanjeev Banerjee and Alon Lavie
 Session 7: Panel Discussion and Open Forum on Future Plans
4:50–5:50Panel Discussion
5:50–6:00Future Plans