Complex Linguistic Annotation – No Easy Way Out! A Case from Bengali and Hindi POS Labeling Tasks

Alternative paths to linguistic annotation, such as those utilizing games or exploiting the web users, are becoming popular in recent times owing to their very high benefit-to-cost

ratios. In this paper, however, we report a case study on POS annotation for Bangla and

Hindi, where we observe that reliable linguistic annotation requires not only expert annotators, but also a great deal of supervision. For our hierarchical POS annotation scheme, we find that close supervision and training is necessary at every level of the hierarchy, or equivalently, complexity of the tagset. Nevertheless, an intelligent annotation tool can significantly accelerate the annotation process and increase the inter-annotator agreement for both expert and non-expert annotators. These findings lead us to believe that reliable annotation requiring deep linguistic knowledge (e.g., POS, chunking, Treebank, semantic role labeling) requires expertise and supervision. The focus, therefore, should be on design and development of appropriate annotation tools equipped with machine learning

based predictive modules that can significantly boost the productivity of the annotators.

In  Proceedings of the Third Linguistic Annotation Workshop

Publisher  Association for Computational Linguistics
All copyrights reserved by ACL 2007

Details

TypeInproceedings
URLhttp://www.aclweb.org/anthology/W/W09/W09-3002.pdf
Pages10 - 18
> Publications > Complex Linguistic Annotation – No Easy Way Out! A Case from Bengali and Hindi POS Labeling Tasks