An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge

Objective:
To create an end-to-end system to identify temporal relation in discharge summaries for the 2012 i2b2 challenge. The challenge includes event extraction, timex extraction and temporal relation identification.

Design:
An end-to-end temporal relation system was developed. It includes three subsystems: event extraction system (CRF name entity extraction and their corresponding attribute classifiers), temporal extraction system (CRF name entity extraction, their corresponding attribute classifiers, and context-free grammar based normalization system), and temporal relation system (ten multi-SVM classifiers and a markov logic networks inference system) using labeled sequential pattern mining, syntactic structures based on parse trees, and results from a coordination classifier.

Measurements:
Micro-averaged precision (P), recall (R), averaged P&R (P&R) and F-measure (F) were used to evaluate results.

Results:
For event extraction, the system achieved 0.9415 (P), 0.8930 (R), 0.9166 (P&R) and 0.9166 (F), respectively. For timex extraction, it achieved 0.8818, 0.9489, 0.9141 and 0.9141, respectively. For temporal relation, it achieved 0.6589, 0.7129, 0.6767 and 0.6849, respectively. For end-to-end temporal relation, it achieved 0.5904, 0.5944, 0.5921 and 0.5924, respectively. As for F-measure as evaluation, we were ranked as the first out of 14 competing teams (event extraction), the first out of 14 teams (timex extraction), the third out of 12 teams (temporal relation), and the second out of 7 teams (end-to-end temporal relation).

Conclusions:
The system achieved encouraging results, demonstrating the feasibility of the tasks defined by the i2b2 organizer. The experiment result demonstrates that both global and local information is useful in the 2012 challenge.