Ensemble Competition

Winner is stacking at .95 accuracy 

2nd place was boosting with .943 accuracy 

3rd place was bagging with .942 accuracy 

 

Bagging 

Call:

summary.resamples(object = bagging_results)

 

Models: treebag, rf 

Number of resamples: 30 

 

Accuracy 

             Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's

treebag 0.8000000 0.8857143 0.9166667 0.9211407 0.9640523 0.9722222    0

rf      0.8333333 0.8946078 0.9420168 0.9280781 0.9712185 1.0000000    0

 

Kappa 

             Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's

treebag 0.5648313 0.7552448 0.8208228 0.8265147 0.9216460 0.9407895    0

rf      0.6504854 0.7801430 0.8722003 0.8412055 0.9350695 1.0000000    0
 

Boosting

Call:

summary.resamples(object = boosting_results)

 

Models: c5.0, gbm 

Number of resamples: 30 

 

Accuracy 

          Min.  1st Qu.    Median      Mean   3rd Qu. Max. NA's

c5.0 0.8857143 0.914881 0.9428571 0.9437877 0.9712185    1    0

gbm  0.8571429 0.914881 0.9436508 0.9440881 0.9714286    1    0

 

Kappa 

          Min.   1st Qu.    Median      Mean   3rd Qu. Max. NA's

c5.0 0.7388060 0.8173731 0.8745394 0.8750652 0.9353050    1    0

gbm  0.6788991 0.8142060 0.8776224 0.8753566 0.9375755    1    0

 

 

 

Stacking 

 

A glm ensemble of 5 base models: lda, rpart, glm, knn, svmRadial

 

Ensemble results:

Generalized Linear Model 

 

1053 samples

   5 predictor

   2 classes: 'bad', 'good' 

 

No pre-processing

Resampling: Cross-Validated (10 fold, repeated 3 times) 

Summary of sample sizes: 947, 947, 948, 948, 947, 948, ... 

Resampling results:

 

  Accuracy   Kappa    

  0.9509604  0.8935122