Targeted Feature Dropout for Robust Slot Filling in Natural Language Understanding

  • Puyang Xu ,
  • Ruhi Sarikaya

Published by ISCA - International Speech Communication Association

In slot filling with conditional random field (CRF), the strong current word and dictionary features tend to swamp the effect of contextual features, a phenomenon also known as feature undertraining. This is a dangerous tradeoff especially when training data is small and dictionaries are limited in its coverage of the entities observed during testing. In this paper, we propose a simple and effective solution that extends the feature dropout algorithm, directly aiming at boosting the contribution from entity context. We show with extensive experiments that the proposed technique can significantly improve the robustness against unseen entities, without degrading performance on entities that are either seen or exist in the dictionary.