Conditional Independence Assumption

The Bayes classifier depends on a conditional independence assumption, meaning that the model it predicts assumes that the probability of a given word is independent of the other words. Clearly, this assumption does not hold. Nevertheless, the Bayes classifier functions well, in part because the positive and negative correlations between features tend to cancel each other out [9].

We found a huge difference between results of Naive Bayes and Maximum Entropy for positive testing accuracy and negative testing accuracy. Maximum Entropy, which makes no unfounded assumptions about the data, gave very similar results for positive tests and negative tests with a 0.2% difference on average. On the other hand, positive and negative results from Naive Bayes, which assumes conditional independence, varies by 27.5% on average, with the worst cases performing on test configurations using frequency, averaging 40% difference. These disparities suggest evidence that the movie dataset does not satisfy the conditional independence assumption.