The Fourth Linguistic Annotation Workshop (LAW IV)

Held in conjunction with ACL-2010 

Uppsala, Sweden

15-16 July 2010


The Linguistic Annotation Workshop (The LAW)

Call for Papers (law4-cfp.pdf)

Linguistic annotation of natural language corpora is the backbone of supervised methods for statistical natural language processing. The Fourth LAW will provide a forum for presentation and discussion of innovative research on all aspects of linguistic annotation, including the creation/evaluation of annotation schemes, methods for automatic and manual annotation, use and evaluation of annotation software and frameworks, representation of linguistic data and annotations, etc. As in the past, the LAW will serve as a venue for annotation researchers to work towards standardization, best practices, and interoperability of annotation information and software. Specifically, the goals of this workshop include:

(1) The exchange and propagation of research results with respect to the annotation, manipulation and exploitation of corpora, taking into account different applications and theoretical investigations in the field of language technology and research;

(2) Working towards the harmonization and interoperability from the perspective of the increasingly large number of tools and frameworks that support the creation, instantiation, manipulation, querying, and exploitation of annotated resources;

(3) Pushing the frontier of linguistic annotation and of human language technology by extending the range of linguistic phenomena for which reliable annotation techniques exist.

(4)  Working towards a consensus on all issues crucial to the advancement of the field of corpus annotation.

We invite submissions of long papers and posters, and demonstrations relating to any aspect of the linguistic annotation. Long papers should reflect work in an advanced state, but posters may describe more preliminary work and pilot studies. Posters and proposals for a system demonstration are to be submitted in the form of a short paper. A demonstration proposal should provide an overview of the system to be demonstrated, including functionality, supported input/output formats or structures, supported languages and modalities, etc. Accepted proposals will also appear in the proceedings and are intended to provide background for the demonstration. Papers are invited to address issues in all aspects of linguistic annotation, including but not limited to:

Annotation schemes:

  1. New and innovative annotation schemes

  2. Comparison of annotation schemes

Annotation procedures:

  1. Innovative automated and manual strategies for annotation

  2. Creation, maintenance, and interactive exploration of annotation structures and annotated data

Annotation software and frameworks:

  1. Machine learning and knowledge-based methods for automation of corpus annotation

  2. Development, evaluation and/or innovative use of annotation software frameworks

  3. Using games and “human computation” (e.g., Mechanical Turk) for annotation

Annotation evaluation:

  1. Inter-annotator agreement and other evaluation metrics and strategies

  2. Qualitative evaluation of linguistic representation

Annotation access and use:

  1. Representation formats/structures for merged annotations of different phenomena, and means to explore/manipulate them

  2. Linguistic considerations for merging annotations of distinct phenomena

Annotation guidelines and standards:

  1. Best practices for annotation procedures and/or development and documentation of annotation schemes

  2. Interoperability of annotation formats and/or frameworks among different systems as well as different tasks, frameworks, modalities, and languages

The special themes for LAW IV are:

  1. Demands on annotation for machine-learning purposes, such as the size and composition of annotated corpora, the granularity of the linguistic categories that are amenable to supervised machine learning

  2. Annotation of text transcripts of informal modalities: spoken language, blogs, correspondence, etc.

  3. Annotation of figurative language (metaphor, metonymy, etc.)

ACL mentoring service:

ACL is providing a mentoring (coaching) service for authors from regions of the world where English is less emphasized as a language of scientific exchange. Many authors from these regions, although able to read the scientific literature in English, have little or no experience in writing papers in English for conferences such as the ACL meetings. The service will be arranged as follows. A set of potential mentors will be identified by Mentoring Service Chairs Björn Gambäck (SICS, Sweden and NTNU, Norway) and Diana McCarthy (Lexical Computing Ltd., UK), who will organize this service for ACL 2010. If you would like to take advantage of the service for a submission to this workshop, please upload your paper in PDF format using the paper submission software for the mentoring service available at:

The deadline for the mentoring service is six weeks before the workshop submission deadline. An appropriate mentor will be assigned to your paper and the mentor will get back to you no later than two weeks before the submission deadline.

Please note that this service is for the benefit of the authors as described above. It is not a general mentoring service for authors to improve the technical content of their papers.

Questions about the mentoring service should be referred to


The reviewing of the papers will be blind. The paper should not include the authors' names and affiliations. Furthermore, self-citations and other references (e.g. to projects, corpora, or software) that could reveal the author's identity should be avoided. For example, instead of "We previously showed (Smith, 1991) ...", write "Smith previously showed (Smith, 1991) ...".

Important Dates

Papers due: April 5, 2010 April 10, 2010
Acceptance/rejection notification:
May 6, 2010 May 9, 2010
Camera-ready final version due: May 16, 2010
Workshop Dates: July 15-16, 2010