Textual Entailment Recognition was proposed recently as a generic task that captures major semantic inference needs across many natural language processing applications, such as Question Answering (QA), Information Retrieval (IR), Information Extraction (IE), and (multi) document summarization. This task requires a system to recognize, given two text fragments, whether the meaning of one text is entailed (can be inferred) from the other text.
The first challenge on Recognizing Textual Entailment (RTE) was held in April 2005. The accuracy of competing systems ranged from 49.5% to 60.60% as against a coin-toss baseline of 50%. Microsoft Research did not compete in this challenge, but instead submitted a manually-conducted oracle experiment that suggested that a combination of syntactic analysis and a thesaurus could potentially score as high as 75%. Capitalizing on these insights we built a system that used syntactic analysis to recognize, in particular, false entailments. This system was entered in the second Recognizing Textual Entailment challenge held in April 2006.
The Logical Forms for the RTE2005 development and test sets used in the Effectively using syntax for recognizing false entailment paper are available for download here. These structures are computed by NLPWIN, a rule-based English parser developed by the NLP group at MSR.
- Rion Snow, Lucy Vanderwende, and Arul Menezes, Effectively using syntax for recognizing false entailment, Association for Computational Linguistics, May 2006
- Lucy Vanderwende and William B. Dolan, What syntax can contribute in entailment task, Springer-Verlag, June 2006
- Chris Brockett, ALIGNING THE RTE 2006 CORPUS, no. MSR-TR-2007-77, June 2007