I came across this paper which brought out an interesting point that an ensemble of bayesian classifiers (called base classifiers) could predict an hypothesis sometimes more accurately than an individual bayesian classifier. Something that reminds me of James Surowiecki’s ‘The Wisdom Of Crowds’. There seems to be an analogy between the two both assume that the individual (base classifier) makes the decision independent of the others.

An excerpt from the paper:

A popular method for creating an accurate classifier from a set of training instances is to train several different classifiers, and then to combine their predictions. Previous theoretical and empirical research has shown that an ensemble is often more accurate than any of the single classifiers in the ensemble. To create an ensemble, one generally must focus on two aspects: (1) which classifiers to use as components of the ensemble (generation of the base classifiers); and (2) how to combine their individual predictions into one (the integration procedure).

Get hold of the paper here.