Computer-Assisted Grading at Microsoft Research

 

This site contains further information about our work on computer-assisted grading at Microsoft Research. We invite you to read the papers below, and then if interested download the corpus we developed for use in your own experiments.

Papers

Videos and Demos

The 5-minute video below gives an overview of our power-assisted grading research approach (Paper 1) as well as a demonstration of our user interface, including results from a study of teachers using the system (Paper 2) .  For further details please read the papers above.

Powergrading-1.0 Short Answer Grading Corpus

The Powergrading-1.0 short answer corpus contains the original data analyzed in our TACL paper. It consists of responses from 100+698 Mechanical Turk workers to each of 20 questions from the 100 questions published by the United States Citizenship and Immigration Services as preparation for the citizenship test.

Please be aware that this data contains raw, unfiltered answers from Mechanical Turk Workers; as such some may contain profanity or be otherwise offensive.  

The corpus is in several files, each in tab-separated format (TSV), where the first row contains the column headings.  The files are as follows:

Corpus Download and License

The corpus and associated licensing information are available at this link. If you do end up using the corpus in your research, please refer to it as the "Powergrading-1.0" corpus and cite the paper above, as we may have later versions with more data.

Contributors and Contact Information

If you're interested in finding out more, please feel free to contact us:

Sumit Basu (sumitb at microsoft dot com), Chuck Jacobs (cjacobs at microsoft dot com), and Lucy Vanderwende (lucyv at microsoft dot com) are all with Microsoft Research. Michael Brooks (mjbrooks at uw dot edu) was an intern at Microsoft Research from the University of Washington, working with us in 2013.