Partitioned Logistic Regression for Spam Filtering

Ming-Wei Chang, Wen-tau Yih, and Christopher Meek

Abstract

Naive Bayes and logistic regression perform well in different regimes. While the former is a very simple generative model which is efficient to train and performs well empirically in many applications, the latter is a discriminative model which often achieves better accuracy and can be shown to outperform naive Bayes asymptotically. In this paper, we propose a novel hybrid model, partitioned logistic regression, which has several advantages over both naive Bayes and logistic regression. This model separates the original feature space into several disjoint feature groups. Individual models on these groups of features are learned using logistic regression and their predictions are combined using the naive Bayes principle to produce a robust final estimation. We show that our model is better both theoretically and empirically. In addition, when applying it in a practical application, email spam filtering, it improves the normalized AUC score at 10% false-positive rate by 28.8% and 23.6% compared to naive Bayes and logistic regression, when using the exact same training examples.

Details

Publication typeInproceedings
Published inProceedings of The 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
PublisherChoose...
> Publications > Partitioned Logistic Regression for Spam Filtering