The bootstrapped aggregation of classifiers, also referred to as bagging, is a classic meta-classification algorithm. We extend it to a two-stage architecture consisting of an initial voting amongst one-versus-all classifiers or single-class recognizers, and a second stage of one-versus-one classifiers or two-class discriminators used for disambiguation. Since our method constructs an ensemble of elementary classifiers, it lends itself very well to parallelization. We describe a static workload balancing strategy for embarrassingly parallel classifier construction as well as a parallelization of the classification process with the message passing interface. We evaluate our approach both in terms of classification performance and speed-up and demonstrate the utility of our approach.