1 Decision Tree Classifier

1.1 G-mean

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9604 0.9437 0.9448 0.9387 0.9583 0.9505 0.9530 0.9468 0.9493 0.9586 0.9612
art2 0.7257 0.7170 0.7238 0.6576 0.6663 0.6523 0.6973 0.6475 0.6604 0.6868 0.6465
art3 0.5176 0.4820 0.4360 0.4351 0.4446 0.4609 0.4639 0.4422 0.4693 0.5412 0.4363
art4 0.8022 0.8073 0.7769 0.7278 0.7249 0.7503 0.7429 0.7267 0.7342 0.7731 0.7563
cleveland 0.2944 0.1742 0.0592 0.1817 0.1346 0.1739 0.2274 0.1531 0.2205 0.2441 0.1546
cleveland_2 0.3454 0.3228 0.0692 0.2548 0.2010 0.1951 0.2251 0.2093 0.2911 0.3773 0.2518
contraceptive 0.5106 0.4638 0.4664 0.4599 0.4752 0.4707 0.4694 0.4657 0.4582 0.4889 0.4930
glass 0.6698 0.4512 0.3263 0.6620 0.6445 0.6340 0.5388 0.6794 0.7195 0.7363 0.7251
hayes-roth 0.8778 0.8436 0.5822 0.8622 0.8637 0.8637 0.8637 0.8638 0.8697 0.8205 0.8810
ecoli 0.7192 0.6400 0.6769 0.6392 0.6893 0.7001 0.7251 0.6733 0.6897 0.7315 0.7421
vehicle 0.9005 0.8883 0.9045 0.8901 0.9086 0.8915 0.8866 0.9082 0.9014 0.9057 0.9022
wine quality 0.3705 0.3367 0.3242 0.2692 0.2354 0.3638 0.3056 0.2464 0.3297 0.2993 0.3163
yeast 0.4377 0.3590 0.2498 0.2847 0.2731 0.2967 0.3507 0.3086 0.2751 0.4370 0.4215
pageblocks 0.5200 0.4774 0.1539 0.3710 0.5724 0.3404 0.1804 0.3171 0.3971 0.3722 0.3832
newthyroid 0.9512 0.8946 0.9052 0.9035 0.8954 0.9101 0.9288 0.8647 0.9403 0.9554 0.9138

Friedman rank sum test
Friedman chi-squared = 48.847, df = 10, p-value = 4.345e-07

GMMSampler SOUP MultiClassCCR StaticSMOTE MDO RandomUnderSampler KDESampler KMeansSMOTE GlobalCS AllKNN RandomOverSampler
2.33 3.47 4.67 5.53 6.13 6.27 6.6 6.87 7.73 8.07 8.33

1.2 Balanced Accuracy

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9597 0.9341 0.9482 0.9488 0.9601 0.9514 0.9565 0.9530 0.9488 0.9532 0.9637
art2 0.7530 0.7387 0.7464 0.7034 0.7129 0.6982 0.7302 0.6950 0.7062 0.7127 0.6935
art3 0.5385 0.5258 0.5244 0.5026 0.5298 0.5371 0.5331 0.5117 0.5056 0.5516 0.5069
art4 0.8228 0.8240 0.8018 0.7708 0.7683 0.7907 0.7935 0.7681 0.7770 0.7978 0.7901
cleveland 0.3572 0.2548 0.2528 0.2751 0.2884 0.2827 0.2764 0.2181 0.2726 0.2808 0.2513
cleveland_2 0.3938 0.3387 0.3468 0.3143 0.3358 0.4048 0.3573 0.3502 0.3487 0.3874 0.3457
contraceptive 0.5125 0.4680 0.4764 0.4777 0.4879 0.4877 0.4850 0.4707 0.4747 0.4872 0.4961
glass 0.7454 0.5860 0.5498 0.7202 0.5805 0.6704 0.5906 0.6610 0.7362 0.7428 0.7468
hayes-roth 0.8749 0.8338 0.6608 0.8740 0.8742 0.8742 0.8742 0.8745 0.8795 0.8437 0.8797
ecoli 0.7236 0.6709 0.7281 0.6786 0.6966 0.7190 0.7442 0.6813 0.7115 0.7334 0.7656
vehicle 0.9002 0.8880 0.9099 0.8922 0.9041 0.8896 0.8862 0.9050 0.9046 0.9031 0.8986
wine quality 0.3966 0.3592 0.3747 0.3533 0.3800 0.3839 0.3923 0.3589 0.3533 0.3697 0.3690
yeast 0.4754 0.4244 0.4685 0.4313 0.4266 0.4330 0.4555 0.4113 0.4146 0.4732 0.4723
pageblocks 0.7583 0.6547 0.6428 0.5944 0.6302 0.6795 0.6968 0.6246 0.7416 0.6918 0.7014
newthyroid 0.9442 0.8928 0.8993 0.9105 0.9061 0.9206 0.9273 0.8712 0.9210 0.9273 0.9095

Friedman rank sum test
Friedman chi-squared = 50.34, df = 10, p-value = 2.311e-07

GMMSampler SOUP MDO MultiClassCCR KDESampler KMeansSMOTE StaticSMOTE AllKNN RandomUnderSampler GlobalCS RandomOverSampler
2.13 4.23 4.83 5.2 5.33 6.27 6.57 6.6 8.2 8.27 8.37

1.3 F-score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9292 0.8947 0.9395 0.9373 0.9471 0.9338 0.9466 0.9531 0.9469 0.9281 0.9566
art2 0.6991 0.6830 0.7023 0.6745 0.6855 0.6624 0.6842 0.6709 0.6802 0.6639 0.6669
art3 0.5079 0.4790 0.4994 0.5000 0.5122 0.5147 0.5120 0.5135 0.4868 0.5211 0.5069
art4 0.7507 0.7555 0.7458 0.7508 0.7377 0.7590 0.7473 0.7456 0.7437 0.7366 0.7538
cleveland 0.2736 0.2289 0.2302 0.2547 0.2764 0.2509 0.2891 0.2537 0.2577 0.2693 0.2510
cleveland_2 0.3234 0.2334 0.2848 0.3569 0.3368 0.3682 0.3303 0.3753 0.3402 0.2991 0.3546
contraceptive 0.5047 0.4586 0.4629 0.4591 0.4922 0.4803 0.4804 0.4709 0.4723 0.4808 0.4925
glass 0.6742 0.5666 0.5626 0.6607 0.5964 0.6714 0.6062 0.6447 0.7304 0.6357 0.7095
hayes-roth 0.8851 0.8534 0.6091 0.8732 0.8736 0.8736 0.8555 0.8737 0.8789 0.8450 0.8904
ecoli 0.7218 0.6424 0.7214 0.6877 0.7213 0.7146 0.7339 0.6892 0.6754 0.7114 0.7184
vehicle 0.8766 0.8650 0.8927 0.8879 0.9080 0.8842 0.8853 0.8880 0.8981 0.8840 0.9048
wine quality 0.3587 0.3248 0.3492 0.3607 0.3602 0.3697 0.3711 0.3526 0.3475 0.3414 0.3704
yeast 0.4596 0.3705 0.4597 0.4215 0.4268 0.4426 0.4481 0.4034 0.3974 0.4126 0.4331
pageblocks 0.7526 0.4995 0.5653 0.6293 0.7285 0.6330 0.6197 0.7253 0.6735 0.4157 0.6921
newthyroid 0.9158 0.8248 0.8865 0.9018 0.9012 0.9154 0.9126 0.8820 0.9217 0.9150 0.8953

Friedman rank sum test
Friedman chi-squared = 38.845, df = 10, p-value = 2.702e-05

GMMSampler MultiClassCCR KMeansSMOTE MDO KDESampler StaticSMOTE GlobalCS RandomOverSampler AllKNN SOUP RandomUnderSampler
4.13 4.2 4.7 4.93 5.23 5.87 5.93 6.4 7.2 7.67 9.73

1.4 Average Precision Score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8868 0.8335 0.8889 0.8976 0.9236 0.8899 0.9059 0.9153 0.9146 0.8761 0.9168
art2 0.6389 0.6394 0.6666 0.5860 0.6507 0.5866 0.6121 0.5692 0.5934 0.5681 0.5684
art3 0.4410 0.4229 0.4235 0.4189 0.4391 0.4667 0.4615 0.4292 0.4116 0.4381 0.4257
art4 0.6615 0.6819 0.6829 0.6787 0.7152 0.7274 0.7037 0.6806 0.6845 0.6526 0.6854
cleveland 0.2492 0.2468 0.2496 0.2390 0.2474 0.2421 0.2481 0.2242 0.2239 0.2332 0.2277
cleveland_2 0.3024 0.2829 0.3267 0.2808 0.2942 0.3091 0.2969 0.2817 0.2899 0.2908 0.2810
contraceptive 0.4883 0.4411 0.4109 0.4023 0.4576 0.4782 0.4551 0.4029 0.4072 0.4582 0.4444
glass 0.6191 0.4777 0.4621 0.5486 0.5849 0.5576 0.4905 0.5800 0.6654 0.5667 0.6044
hayes-roth 0.8965 0.8418 0.5464 0.9205 0.9196 0.9196 0.9196 0.9225 0.9039 0.8234 0.8473
ecoli 0.6586 0.4940 0.5837 0.5258 0.6009 0.5961 0.6431 0.5533 0.5679 0.5918 0.6525
vehicle 0.8222 0.8189 0.8374 0.8301 0.8602 0.8339 0.8520 0.8524 0.8458 0.8506 0.8435
wine quality 0.3067 0.3115 0.3040 0.3013 0.3007 0.3273 0.3236 0.2925 0.3019 0.2964 0.3000
yeast 0.3729 0.2504 0.2095 0.2824 0.3362 0.3342 0.3120 0.2570 0.2635 0.2871 0.2869
pageblocks 0.6201 0.2968 0.5643 0.5814 0.6718 0.6146 0.5386 0.5629 0.6053 0.3613 0.5944
newthyroid 0.8830 0.7529 0.8488 0.8604 0.8551 0.8544 0.8798 0.8290 0.8945 0.8810 0.8514

Friedman rank sum test
Friedman chi-squared = 39.029, df = 10, p-value = 2.509e-05

KMeansSMOTE GMMSampler KDESampler MDO StaticSMOTE MultiClassCCR AllKNN SOUP GlobalCS RandomOverSampler RandomUnderSampler
3.47 4.07 4.27 4.33 6.13 6.33 6.73 7.2 7.33 7.73 8.4

1.5 Matthews Correlation Coefficient

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8973 0.8394 0.8992 0.9151 0.9229 0.9012 0.9143 0.9147 0.9148 0.8837 0.9261
art2 0.6150 0.6045 0.6318 0.6026 0.6232 0.5799 0.6038 0.5949 0.6131 0.5782 0.5921
art3 0.3524 0.2731 0.3432 0.3150 0.3593 0.3457 0.3365 0.3294 0.2866 0.3401 0.3391
art4 0.6538 0.6514 0.6594 0.6571 0.6519 0.6744 0.6639 0.6571 0.6561 0.6315 0.6651
cleveland 0.1818 0.0881 0.1544 0.1022 0.1579 0.0917 0.1070 0.0486 0.1025 0.1211 0.0988
cleveland_2 0.1917 0.1601 0.2040 0.1030 0.2225 0.2461 0.2029 0.1623 0.1973 0.1852 0.1631
contraceptive 0.2756 0.2185 0.2205 0.1951 0.2425 0.2396 0.2407 0.2025 0.2175 0.2282 0.2487
glass 0.5446 0.3599 0.4969 0.6044 0.4860 0.5922 0.5240 0.4898 0.6188 0.5347 0.5792
hayes-roth 0.7790 0.6918 0.3894 0.7537 0.7743 0.7743 0.7561 0.7753 0.7835 0.7307 0.8033
ecoli 0.7107 0.5814 0.7134 0.6374 0.6948 0.7213 0.7116 0.6524 0.6942 0.6427 0.6892
vehicle 0.8172 0.7943 0.8461 0.8315 0.8455 0.8241 0.7989 0.8521 0.8510 0.8227 0.8416
wine quality 0.2075 0.1430 0.1837 0.1779 0.1654 0.1826 0.1911 0.1610 0.1565 0.1335 0.1827
yeast 0.3917 0.2441 0.3999 0.3417 0.3360 0.3517 0.3503 0.3267 0.3187 0.3715 0.3297
pageblocks 0.7261 0.3077 0.6857 0.7222 0.6851 0.7430 0.6875 0.7123 0.7250 0.3294 0.7180
newthyroid 0.8636 0.6751 0.8232 0.8656 0.8402 0.8847 0.8650 0.8308 0.8781 0.8816 0.8427

Friedman rank sum test
Friedman chi-squared = 41.967, df = 10, p-value = 7.6e-06

GMMSampler KDESampler AllKNN KMeansSMOTE MultiClassCCR MDO StaticSMOTE RandomOverSampler GlobalCS SOUP RandomUnderSampler
4.13 4.23 5.07 5.1 5.13 5.4 5.4 6.6 7.4 7.47 10.07

2 K Nearest Neighbours Classifier

2.1 G-mean

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9708 0.9409 0.9788 0.9765 0.9769 0.9842 0.9851 0.9765 0.9711 0.9619 0.9772
art2 0.7214 0.7288 0.7196 0.6527 0.6669 0.6938 0.7340 0.6545 0.6500 0.7367 0.7370
art3 0.5712 0.6300 0.4391 0.4373 0.3725 0.4529 0.5032 0.4361 0.4253 0.4926 0.4467
art4 0.8543 0.8371 0.8248 0.7550 0.7468 0.8065 0.7805 0.7541 0.7230 0.8384 0.7441
cleveland 0.1097 0.0581 0.0153 0.1175 0.0477 0.0709 0.0327 0.1179 0.1978 0.1470 0.1181
cleveland_2 0.1731 0.1318 0.0145 0.1377 0.0056 0.0832 0.0106 0.1377 0.2650 0.1416 0.1377
contraceptive 0.5169 0.5114 0.4621 0.4539 0.4790 0.5148 0.5176 0.4957 0.4600 0.5236 0.5136
glass 0.6299 0.4157 0.3228 0.6368 0.4414 0.5580 0.4740 0.6588 0.6799 0.6123 0.6693
hayes-roth 0.6474 0.5119 0.4499 0.6532 0.6626 0.6421 0.6125 0.7063 0.6753 0.6135 0.6451
ecoli 0.8177 0.7751 0.7482 0.7659 0.7491 0.7628 0.7951 0.7455 0.7500 0.8124 0.7652
vehicle 0.8371 0.8262 0.8217 0.8515 0.8436 0.8470 0.8551 0.8465 0.8610 0.8128 0.8538
wine quality 0.2867 0.3226 0.2477 0.2530 0.1200 0.1782 0.1299 0.2600 0.2659 0.2980 0.2898
yeast 0.4041 0.3265 0.2576 0.3980 0.2696 0.2693 0.2569 0.4067 0.4007 0.4316 0.4203
pageblocks 0.1116 0.0691 0.0902 0.1488 0.1116 0.1116 0.1215 0.1488 0.2696 0.1646 0.1488
newthyroid 0.8596 0.8531 0.7728 0.8637 0.8289 0.8125 0.8233 0.8842 0.9067 0.8784 0.8842

Friedman rank sum test
Friedman chi-squared = 32.054, df = 10, p-value = 0.0003922

SOUP MultiClassCCR GMMSampler StaticSMOTE GlobalCS RandomOverSampler MDO RandomUnderSampler KDESampler KMeansSMOTE AllKNN
4 4.17 4.6 5.07 5.67 6.1 6.2 6.6 6.67 8.27 8.67

2.2 Balanced Accuracy

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9718 0.9444 0.9792 0.9768 0.9762 0.9845 0.9853 0.9768 0.9712 0.9635 0.9776
art2 0.7438 0.7425 0.7504 0.6994 0.7202 0.7363 0.7665 0.7014 0.7077 0.7498 0.7534
art3 0.6119 0.6204 0.5306 0.4853 0.5306 0.5355 0.5815 0.4849 0.4853 0.5071 0.4895
art4 0.8633 0.8510 0.8375 0.7901 0.7895 0.8260 0.8089 0.7893 0.7746 0.8490 0.7815
cleveland 0.2847 0.1946 0.2025 0.2311 0.2210 0.2279 0.2041 0.2321 0.2499 0.2051 0.2333
cleveland_2 0.3515 0.2711 0.2737 0.2784 0.2443 0.2719 0.2434 0.2784 0.3231 0.2657 0.2784
contraceptive 0.5166 0.5167 0.4784 0.4611 0.4941 0.5202 0.5247 0.4988 0.4692 0.5188 0.5215
glass 0.6584 0.5548 0.5652 0.6833 0.5939 0.6171 0.6044 0.6914 0.7103 0.6321 0.7004
hayes-roth 0.6691 0.5446 0.5387 0.6553 0.6487 0.6442 0.6291 0.6652 0.6946 0.6344 0.6508
ecoli 0.8254 0.7930 0.7828 0.7773 0.7688 0.7794 0.8247 0.7600 0.7662 0.8234 0.7817
vehicle 0.8412 0.8308 0.8244 0.8540 0.8451 0.8489 0.8568 0.8483 0.8616 0.8172 0.8608
wine quality 0.3413 0.3296 0.3117 0.3215 0.3108 0.3307 0.3175 0.3292 0.3284 0.3430 0.3306
yeast 0.5192 0.4902 0.4886 0.4576 0.5014 0.5008 0.5101 0.4559 0.4465 0.4763 0.4652
pageblocks 0.4479 0.4312 0.4697 0.5584 0.4479 0.4479 0.4697 0.5584 0.5758 0.5960 0.5584
newthyroid 0.8665 0.8601 0.7910 0.8702 0.8412 0.8249 0.8360 0.8905 0.9109 0.8816 0.8905

Friedman rank sum test
Friedman chi-squared = 19.196, df = 10, p-value = 0.03784

GMMSampler MultiClassCCR MDO KDESampler StaticSMOTE SOUP GlobalCS RandomOverSampler RandomUnderSampler AllKNN KMeansSMOTE
3.87 4.43 5.37 5.73 5.73 5.73 6.4 6.63 7.07 7.3 7.73

2.3 F-score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9249 0.8535 0.9438 0.9515 0.9619 0.9579 0.9551 0.9515 0.9576 0.9030 0.9495
art2 0.7096 0.6379 0.7114 0.6571 0.7036 0.7035 0.7114 0.6585 0.6626 0.6916 0.6998
art3 0.5858 0.5502 0.5262 0.4652 0.5243 0.5360 0.5487 0.4651 0.4663 0.4548 0.4550
art4 0.7791 0.7439 0.7743 0.7424 0.7435 0.7707 0.7554 0.7409 0.7291 0.7620 0.7155
cleveland 0.2221 0.2101 0.1639 0.2173 0.2157 0.2388 0.1983 0.2173 0.2294 0.1965 0.2187
cleveland_2 0.2751 0.2590 0.2369 0.2581 0.2077 0.2667 0.2120 0.2581 0.2900 0.2344 0.2581
contraceptive 0.5004 0.5009 0.4642 0.4565 0.4948 0.5134 0.5122 0.4873 0.4701 0.5074 0.5028
glass 0.6330 0.5048 0.5722 0.6359 0.6195 0.6391 0.5789 0.6485 0.6854 0.5565 0.6590
hayes-roth 0.6579 0.5256 0.5234 0.6688 0.6905 0.6612 0.6335 0.6768 0.6978 0.6208 0.6552
ecoli 0.8240 0.7907 0.7796 0.7680 0.7702 0.7988 0.8150 0.7503 0.7588 0.8060 0.7693
vehicle 0.8123 0.8035 0.8130 0.8331 0.8349 0.8477 0.8537 0.8284 0.8510 0.7890 0.8322
wine quality 0.3273 0.3029 0.3132 0.3093 0.3119 0.3325 0.3117 0.3156 0.3247 0.3012 0.3138
yeast 0.5055 0.4454 0.5046 0.4336 0.5024 0.5019 0.4929 0.4293 0.4179 0.3971 0.4141
pageblocks 0.4797 0.2860 0.4896 0.5158 0.4797 0.4797 0.4548 0.5158 0.5324 0.3867 0.5158
newthyroid 0.8852 0.8738 0.8359 0.8766 0.8808 0.8731 0.8775 0.8991 0.9135 0.8633 0.8991

Friedman rank sum test
Friedman chi-squared = 32.03, df = 10, p-value = 0.0003959

KDESampler GMMSampler StaticSMOTE MDO KMeansSMOTE MultiClassCCR GlobalCS RandomOverSampler AllKNN RandomUnderSampler SOUP
3.73 4.07 4.53 5.4 5.67 6.1 6.27 6.83 6.93 7.87 8.6

2.4 Average Precision Score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9648 0.8954 0.9763 0.9568 0.9835 0.9968 0.9803 0.9506 0.9615 0.9103 0.9558
art2 0.7368 0.7310 0.7362 0.6036 0.7220 0.8168 0.8073 0.5976 0.6374 0.6130 0.6810
art3 0.5486 0.5637 0.5261 0.4289 0.5005 0.5756 0.5507 0.4264 0.4350 0.4198 0.4524
art4 0.7990 0.7986 0.7745 0.7247 0.7707 0.9250 0.8587 0.7156 0.7198 0.7175 0.8109
cleveland 0.2553 0.2269 0.2495 0.2260 0.2398 0.2519 0.2365 0.2267 0.2224 0.2523 0.2271
cleveland_2 0.3121 0.2801 0.3078 0.2761 0.2914 0.2960 0.2881 0.2760 0.2763 0.2811 0.2764
contraceptive 0.5274 0.5110 0.4499 0.4448 0.5034 0.5275 0.5205 0.4962 0.4377 0.5155 0.5249
glass 0.6120 0.4813 0.5228 0.6036 0.6187 0.6269 0.6111 0.6034 0.6320 0.5318 0.6131
hayes-roth 0.6978 0.5878 0.5559 0.6954 0.7420 0.7031 0.7129 0.7401 0.7737 0.6043 0.7327
ecoli 0.8185 0.8199 0.7720 0.6839 0.7185 0.8162 0.7977 0.6741 0.6968 0.8153 0.7340
vehicle 0.8286 0.8410 0.8189 0.8425 0.8600 0.8790 0.8765 0.8737 0.8358 0.7923 0.8819
wine quality 0.3263 0.3101 0.2890 0.2921 0.2991 0.3078 0.3107 0.2942 0.2992 0.2968 0.3009
yeast 0.4577 0.4291 0.2881 0.3144 0.4861 0.4840 0.4587 0.3244 0.2844 0.3253 0.3296
pageblocks 0.6013 0.3469 0.5343 0.4424 0.6013 0.6013 0.6035 0.4424 0.4763 0.4672 0.4473
newthyroid 0.9084 0.9050 0.8526 0.8844 0.9033 0.8987 0.9011 0.8773 0.8881 0.8413 0.8925

Friedman rank sum test
Friedman chi-squared = 65.982, df = 10, p-value = 2.629e-10

KDESampler GMMSampler MDO KMeansSMOTE MultiClassCCR RandomUnderSampler AllKNN StaticSMOTE SOUP GlobalCS RandomOverSampler
2.53 3.47 3.6 4.6 5.6 6.27 7.13 7.4 7.93 8.7 8.77

2.5 Matthews Correlation Coefficient

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8878 0.7982 0.9165 0.9238 0.9386 0.9404 0.9377 0.9238 0.9349 0.8587 0.9198
art2 0.6436 0.5433 0.6509 0.5808 0.6463 0.6409 0.6458 0.5817 0.5930 0.6134 0.6197
art3 0.4485 0.3868 0.4028 0.2667 0.4069 0.4160 0.4114 0.2688 0.2712 0.2688 0.2659
art4 0.6876 0.6374 0.6865 0.6531 0.6617 0.6866 0.6713 0.6508 0.6390 0.6695 0.6081
cleveland 0.1492 0.0654 0.0482 0.0610 0.1075 0.1035 0.0914 0.0620 0.0566 0.0418 0.0654
cleveland_2 0.1599 0.1019 0.0020 0.0570 -0.0066 0.0458 -0.0068 0.0570 0.0913 0.0734 0.0570
contraceptive 0.2702 0.2727 0.2332 0.1864 0.2568 0.2823 0.2846 0.2469 0.2076 0.2728 0.2694
glass 0.5388 0.3807 0.5338 0.5541 0.5354 0.5496 0.5093 0.5662 0.6157 0.4611 0.5709
hayes-roth 0.4690 0.2846 0.2439 0.4483 0.4896 0.4592 0.4238 0.4780 0.5402 0.4178 0.4371
ecoli 0.7926 0.7692 0.7695 0.7401 0.7499 0.7902 0.7924 0.7098 0.7267 0.7770 0.7422
vehicle 0.7214 0.7037 0.7153 0.7465 0.7467 0.7647 0.7747 0.7425 0.7729 0.6853 0.7530
wine quality 0.1442 0.0696 0.1038 0.0967 0.1192 0.1102 0.1163 0.1046 0.1198 0.0982 0.1023
yeast 0.4443 0.3484 0.4302 0.3704 0.4321 0.4321 0.4287 0.3574 0.3297 0.3479 0.3461
pageblocks 0.5705 0.1521 0.5995 0.5602 0.5705 0.5705 0.5915 0.5602 0.5859 0.2808 0.5602
newthyroid 0.8177 0.8208 0.7614 0.8099 0.8281 0.8156 0.8270 0.8490 0.8704 0.7839 0.8490

Friedman rank sum test
Friedman chi-squared = 38.204, df = 10, p-value = 3.497e-05

GMMSampler KDESampler MDO KMeansSMOTE StaticSMOTE GlobalCS AllKNN MultiClassCCR RandomOverSampler SOUP RandomUnderSampler
3.67 3.97 4.33 4.57 5.4 6.67 6.73 6.83 7.7 8 8.13

3 Naive Bayes Classifier

3.1 G-mean

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8738 0.9231 0.7897 0.9316 0.8663 0.8652 0.9022 0.9330 0.9364 0.9334 0.9321
art2 0.6885 0.7758 0.4503 0.7907 0.6654 0.6240 0.6605 0.7883 0.7321 0.7856 0.7888
art3 0.2382 0.6311 0.0927 0.6188 0.1735 0.1687 0.2243 0.6267 0.6509 0.6249 0.6355
art4 0.7890 0.8432 0.6991 0.8489 0.8318 0.7621 0.7872 0.8364 0.8304 0.8473 0.8363
cleveland 0.2463 0.1920 0.0412 0.2223 0.1025 0.1233 0.1010 0.1731 0.2688 0.1955 0.2012
cleveland_2 0.3421 0.2217 0.0685 0.2494 0.1211 0.2021 0.1257 0.2267 0.2517 0.2644 0.2432
contraceptive 0.4876 0.4837 0.4740 0.4919 0.4938 0.4830 0.4833 0.4836 0.4909 0.4865 0.4641
glass 0.2795 0.2017 0.1783 0.2261 0.1697 0.1707 0.2270 0.1516 0.3217 0.4247 0.2250
hayes-roth 0.6603 0.6401 0.5520 0.6546 0.6450 0.6546 0.6546 0.6421 0.2453 0.5461 0.6474
ecoli 0.5077 0.3527 0.6755 0.2898 0.4418 0.3275 0.1032 0.3031 0.4573 0.3767 0.5508
vehicle 0.5321 0.5200 0.4465 0.4595 0.5976 0.4597 0.6198 0.5069 0.5386 0.5219 0.5058
wine quality 0.4403 0.4296 0.3387 0.4377 0.3629 0.4126 0.3654 0.4497 0.4319 0.4466 0.4341
yeast 0.1492 0.2687 0.1238 0.0689 0.0686 0.0686 0.0365 0.0644 0.0906 0.1874 0.1710
pageblocks 0.1923 0.5241 0.1060 0.2064 0.1923 0.1923 0.1769 0.1405 0.1997 0.2164 0.1404
newthyroid 0.9479 0.9559 0.9405 0.9458 0.9344 0.9405 0.9405 0.9437 0.9559 0.9416 0.9458

Friedman rank sum test
Friedman chi-squared = 50.962, df = 10, p-value = 1.775e-07

StaticSMOTE SOUP GMMSampler RandomOverSampler RandomUnderSampler MultiClassCCR GlobalCS KMeansSMOTE MDO KDESampler AllKNN
3.97 4 4.27 4.53 4.97 5.13 6.27 7.37 7.73 8.37 9.4

3.2 Balanced Accuracy

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8810 0.9272 0.8077 0.9351 0.8687 0.8690 0.9052 0.9363 0.9367 0.9371 0.9351
art2 0.7538 0.7905 0.5611 0.8018 0.7071 0.6907 0.7411 0.7992 0.7659 0.7966 0.8038
art3 0.5802 0.6429 0.3708 0.6325 0.3986 0.4058 0.5492 0.6403 0.6665 0.6395 0.6472
art4 0.8206 0.8524 0.7563 0.8575 0.8435 0.7935 0.8177 0.8466 0.8421 0.8569 0.8464
cleveland 0.3300 0.2456 0.3160 0.2885 0.2827 0.3019 0.2908 0.2596 0.3128 0.2484 0.2886
cleveland_2 0.4219 0.3320 0.3710 0.3532 0.3470 0.3586 0.3749 0.3325 0.3573 0.3395 0.3487
contraceptive 0.5011 0.4974 0.5003 0.5033 0.4989 0.4942 0.4995 0.4963 0.4997 0.4989 0.4921
glass 0.5456 0.4501 0.5272 0.4904 0.4766 0.4812 0.5115 0.4623 0.4962 0.6067 0.4867
hayes-roth 0.7039 0.6948 0.6086 0.7143 0.6921 0.7143 0.7143 0.6989 0.6275 0.6364 0.7042
ecoli 0.7266 0.6163 0.7229 0.5548 0.6205 0.5462 0.4633 0.5546 0.6374 0.6590 0.6723
vehicle 0.6041 0.6112 0.5798 0.5894 0.6063 0.5848 0.6371 0.6042 0.6209 0.6074 0.6051
wine quality 0.4740 0.4705 0.3872 0.4749 0.4367 0.4580 0.4181 0.4762 0.4596 0.4669 0.4643
yeast 0.4118 0.4398 0.3996 0.3172 0.3204 0.3204 0.2302 0.3165 0.3323 0.4255 0.3938
pageblocks 0.6145 0.7391 0.5725 0.6404 0.6145 0.6145 0.5595 0.5567 0.6327 0.6913 0.5559
newthyroid 0.9511 0.9578 0.9432 0.9489 0.9400 0.9432 0.9432 0.9467 0.9578 0.9444 0.9489

Friedman rank sum test
Friedman chi-squared = 24.886, df = 10, p-value = 0.005566

GMMSampler StaticSMOTE RandomOverSampler SOUP RandomUnderSampler MultiClassCCR MDO GlobalCS AllKNN KDESampler KMeansSMOTE
4.33 4.57 4.7 4.8 5.5 5.57 6.73 6.73 7.4 7.77 7.9

3.3 F-score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9027 0.8307 0.8597 0.8430 0.8518 0.9001 0.8777 0.8455 0.9035 0.8433 0.8497
art2 0.7458 0.6881 0.5996 0.7070 0.7191 0.7010 0.6961 0.7066 0.7148 0.7003 0.7119
art3 0.5271 0.5236 0.3360 0.5128 0.3560 0.3745 0.4567 0.5199 0.6152 0.5147 0.5331
art4 0.7899 0.7586 0.7479 0.7613 0.7876 0.7761 0.7824 0.7483 0.7823 0.7546 0.7549
cleveland 0.2995 0.2256 0.2207 0.2817 0.2774 0.2954 0.2651 0.2483 0.2909 0.2318 0.2721
cleveland_2 0.3589 0.3103 0.3124 0.3347 0.3519 0.3526 0.3464 0.3146 0.3414 0.3022 0.3284
contraceptive 0.4744 0.4702 0.4668 0.4775 0.4812 0.4714 0.4717 0.4694 0.4821 0.4726 0.4562
glass 0.4814 0.3862 0.4774 0.4107 0.4164 0.4213 0.4490 0.3871 0.4408 0.5221 0.4409
hayes-roth 0.6510 0.6357 0.5447 0.6724 0.6493 0.6672 0.6714 0.6462 0.5036 0.5612 0.6515
ecoli 0.7040 0.5781 0.7272 0.4616 0.5493 0.4632 0.3678 0.4712 0.5611 0.6307 0.6581
vehicle 0.5570 0.5634 0.5180 0.5273 0.5962 0.5269 0.6367 0.5550 0.5758 0.5554 0.5550
wine quality 0.4390 0.4026 0.3828 0.3882 0.4359 0.4364 0.3843 0.4086 0.3909 0.4025 0.4263
yeast 0.3314 0.4035 0.3521 0.2078 0.2127 0.2127 0.1572 0.2022 0.2518 0.3786 0.3427
pageblocks 0.5574 0.4938 0.4754 0.5031 0.5574 0.5574 0.5201 0.4587 0.5287 0.5233 0.4560
newthyroid 0.9591 0.9550 0.9583 0.9531 0.9507 0.9583 0.9583 0.9476 0.9550 0.9417 0.9532

Friedman rank sum test
Friedman chi-squared = 38.395, df = 10, p-value = 3.239e-05

GMMSampler StaticSMOTE KMeansSMOTE KDESampler MDO MultiClassCCR SOUP RandomOverSampler RandomUnderSampler AllKNN GlobalCS
2.4 4.5 4.97 5.23 5.93 6 6.8 6.87 7.23 7.93 8.13

3.4 Average Precision Score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9803 0.9706 0.9781 0.9765 0.9357 0.9749 0.9762 0.9760 0.9788 0.9760 0.9764
art2 0.8464 0.8135 0.8462 0.8206 0.8464 0.8497 0.8460 0.8225 0.8381 0.8227 0.8265
art3 0.6229 0.5795 0.6599 0.5800 0.6612 0.6350 0.6303 0.5745 0.6307 0.5708 0.5771
art4 0.9191 0.9126 0.9196 0.9172 0.8834 0.9152 0.9217 0.9166 0.9186 0.9173 0.9169
cleveland 0.3444 0.3114 0.3556 0.3406 0.3414 0.3474 0.3278 0.3392 0.3544 0.3134 0.3397
cleveland_2 0.3974 0.3455 0.4430 0.3958 0.4116 0.4128 0.3932 0.3897 0.4078 0.3710 0.3995
contraceptive 0.4987 0.5085 0.5121 0.4992 0.5099 0.5051 0.5020 0.4981 0.5057 0.4947 0.4988
glass 0.5830 0.5191 0.5503 0.5690 0.5557 0.5615 0.5786 0.5637 0.5869 0.6181 0.5677
hayes-roth 0.8280 0.7703 0.6951 0.8414 0.8148 0.8504 0.8479 0.8463 0.8243 0.7636 0.8488
ecoli 0.8490 0.8190 0.8144 0.7945 0.7922 0.7976 0.7854 0.7894 0.7986 0.8374 0.8103
vehicle 0.6736 0.7014 0.6883 0.6797 0.6532 0.7024 0.7142 0.7007 0.7150 0.6824 0.6981
wine quality 0.4352 0.4149 0.3914 0.4241 0.4274 0.4256 0.3962 0.4223 0.4184 0.4189 0.4311
yeast 0.4291 0.4288 0.3046 0.3493 0.3648 0.3645 0.4015 0.3495 0.3726 0.4608 0.4854
pageblocks 0.6169 0.5703 0.4921 0.6034 0.6169 0.6169 0.5801 0.6063 0.6669 0.6112 0.6061
newthyroid 0.9912 0.9885 0.9909 0.9897 0.9894 0.9913 0.9914 0.9892 0.9897 0.9904 0.9892

Friedman rank sum test
Friedman chi-squared = 26.833, df = 10, p-value = 0.002768

GMMSampler StaticSMOTE KDESampler AllKNN MDO MultiClassCCR KMeansSMOTE SOUP RandomOverSampler GlobalCS RandomUnderSampler
4.13 4.27 4.53 5.53 5.67 5.73 6.2 6.8 6.87 7.93 8.33

3.5 Matthews Correlation Coefficient

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8613 0.7659 0.8169 0.7850 0.7822 0.8460 0.8136 0.7883 0.8534 0.7845 0.7923
art2 0.6967 0.5993 0.4808 0.6243 0.6497 0.6060 0.6018 0.6243 0.6405 0.6172 0.6289
art3 0.4804 0.3780 0.1268 0.3661 0.1331 0.1363 0.3022 0.3777 0.4699 0.3695 0.3914
art4 0.7128 0.6644 0.6731 0.6658 0.6974 0.6853 0.6852 0.6479 0.6894 0.6594 0.6551
cleveland 0.2206 0.0716 0.1951 0.1791 0.2089 0.2106 0.2133 0.1457 0.1850 0.1185 0.1809
cleveland_2 0.2695 0.2227 0.2538 0.2462 0.2682 0.2545 0.3113 0.2225 0.2521 0.1956 0.2347
contraceptive 0.2389 0.2336 0.2414 0.2415 0.2332 0.2284 0.2355 0.2309 0.2375 0.2354 0.2270
glass 0.3570 0.2254 0.3958 0.2584 0.2975 0.3106 0.3111 0.2479 0.2952 0.4161 0.3349
hayes-roth 0.4775 0.4645 0.3512 0.4969 0.4757 0.5024 0.4998 0.4686 0.3948 0.3749 0.4795
ecoli 0.7216 0.6049 0.7494 0.4688 0.5134 0.4616 0.4258 0.4714 0.5438 0.7024 0.6092
vehicle 0.4478 0.4682 0.4222 0.4338 0.4152 0.4318 0.4706 0.4558 0.4837 0.4554 0.4587
wine quality 0.2895 0.2531 0.2052 0.2402 0.2935 0.2980 0.2419 0.2688 0.2368 0.2337 0.2845
yeast 0.2924 0.3129 0.2598 0.1255 0.1277 0.1277 0.1160 0.1225 0.1577 0.3096 0.2632
pageblocks 0.6246 0.4060 0.6835 0.5696 0.6246 0.6246 0.5995 0.5442 0.6233 0.6130 0.5388
newthyroid 0.9403 0.9309 0.9394 0.9300 0.9301 0.9394 0.9394 0.9219 0.9309 0.9115 0.9308

Friedman rank sum test
Friedman chi-squared = 30.827, df = 10, p-value = 0.0006266

GMMSampler StaticSMOTE KDESampler MDO AllKNN MultiClassCCR KMeansSMOTE RandomOverSampler SOUP RandomUnderSampler GlobalCS
2.47 5.03 5.43 5.53 5.87 6 6.03 7.07 7.13 7.37 8.07

4 Random Forest

4.1 G-mean

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9626 0.9608 0.9649 0.9582 0.9590 0.9675 0.9613 0.9675 0.9567 0.9639 0.9688
art2 0.7655 0.7032 0.7065 0.6618 0.6870 0.6703 0.7090 0.6574 0.6631 0.6920 0.6818
art3 0.6296 0.5751 0.4391 0.4422 0.4204 0.4725 0.5040 0.4649 0.4968 0.5544 0.4667
art4 0.8352 0.7714 0.7702 0.7495 0.7568 0.7651 0.7641 0.7361 0.7404 0.7746 0.7597
cleveland 0.2631 0.1453 0.0232 0.1481 0.0848 0.1516 0.0668 0.1889 0.2100 0.2381 0.1835
cleveland_2 0.3564 0.2777 0.0369 0.1028 0.1090 0.1115 0.0956 0.1831 0.1340 0.2749 0.2332
contraceptive 0.5461 0.5239 0.5007 0.4901 0.5120 0.5243 0.5160 0.4953 0.4928 0.5477 0.5211
glass 0.7480 0.6241 0.3651 0.7252 0.5363 0.6353 0.6058 0.7017 0.7590 0.6961 0.7363
hayes-roth 0.8474 0.8708 0.5766 0.8572 0.8532 0.8545 0.8533 0.8552 0.8677 0.8124 0.8642
ecoli 0.8167 0.7985 0.7544 0.7362 0.7908 0.7798 0.8239 0.7566 0.7812 0.8234 0.7988
vehicle 0.9228 0.9475 0.9453 0.9601 0.9684 0.9584 0.9563 0.9664 0.9667 0.9193 0.9680
wine quality 0.4462 0.4324 0.3067 0.2311 0.1662 0.4193 0.3398 0.3011 0.2752 0.3785 0.3786
yeast 0.3556 0.3621 0.2513 0.2791 0.2622 0.3379 0.2815 0.2675 0.3348 0.4313 0.3487
pageblocks 0.4777 0.5187 0.1574 0.3435 0.3661 0.3788 0.4852 0.3398 0.3543 0.6138 0.4960
newthyroid 0.9819 0.9469 0.8811 0.8897 0.9188 0.9135 0.9525 0.9093 0.8796 0.9473 0.9355

Friedman rank sum test
Friedman chi-squared = 50.024, df = 10, p-value = 2.642e-07

GMMSampler SOUP RandomUnderSampler MultiClassCCR KDESampler MDO StaticSMOTE GlobalCS KMeansSMOTE RandomOverSampler AllKNN
3.13 4 4.2 4.27 5.73 5.87 6.73 7.2 7.8 8.33 8.73

4.2 Balanced Accuracy

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9675 0.9625 0.9647 0.9575 0.9536 0.9665 0.9619 0.9581 0.9625 0.9631 0.9724
art2 0.7859 0.7254 0.7470 0.7079 0.7214 0.7206 0.7429 0.7012 0.7103 0.7171 0.7147
art3 0.6444 0.5986 0.5391 0.5248 0.5435 0.5591 0.5708 0.5244 0.5413 0.5679 0.5492
art4 0.8597 0.8044 0.7974 0.7887 0.7901 0.7962 0.7974 0.7950 0.7891 0.7998 0.7921
cleveland 0.3050 0.2332 0.2595 0.2595 0.2860 0.3146 0.2756 0.2627 0.2687 0.2984 0.2661
cleveland_2 0.3954 0.3073 0.3390 0.3320 0.3220 0.3527 0.3007 0.3459 0.3118 0.3454 0.3471
contraceptive 0.5557 0.5168 0.5157 0.4884 0.5215 0.5338 0.5274 0.4970 0.4989 0.5439 0.5276
glass 0.7925 0.6583 0.6808 0.7656 0.7090 0.7096 0.6898 0.7580 0.7449 0.7434 0.7808
hayes-roth 0.8644 0.8555 0.6984 0.8745 0.8696 0.8696 0.8699 0.8485 0.8694 0.8283 0.8841
ecoli 0.8293 0.8137 0.7750 0.8075 0.7899 0.8126 0.8052 0.7867 0.7850 0.8245 0.7982
vehicle 0.9285 0.9498 0.9484 0.9603 0.9588 0.9577 0.9556 0.9643 0.9691 0.9198 0.9676
wine quality 0.4682 0.4356 0.3601 0.4219 0.4028 0.4585 0.4358 0.4311 0.4019 0.4452 0.4410
yeast 0.5520 0.4872 0.5110 0.5193 0.5157 0.5340 0.5325 0.5191 0.5159 0.5190 0.5290
pageblocks 0.7099 0.7062 0.6541 0.6880 0.6876 0.5926 0.5785 0.7103 0.7215 0.7949 0.6861
newthyroid 0.9711 0.9333 0.9067 0.9035 0.9176 0.9406 0.9394 0.9051 0.8794 0.9505 0.9489

Friedman rank sum test
Friedman chi-squared = 43.715, df = 10, p-value = 3.702e-06

GMMSampler KDESampler MultiClassCCR SOUP MDO RandomUnderSampler KMeansSMOTE GlobalCS StaticSMOTE RandomOverSampler AllKNN
2.33 4.37 4.53 4.73 5.9 6.7 7.17 7.33 7.37 7.4 8.17

4.3 F-score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9353 0.8893 0.9472 0.9535 0.9494 0.9463 0.9526 0.9473 0.9578 0.9324 0.9579
art2 0.7268 0.6611 0.6959 0.6621 0.7118 0.6887 0.7105 0.6708 0.6708 0.6722 0.6864
art3 0.5760 0.5480 0.5415 0.5294 0.5338 0.5560 0.5383 0.5346 0.5196 0.5312 0.5371
art4 0.7583 0.7419 0.7596 0.7561 0.7550 0.7559 0.7627 0.7541 0.7567 0.7416 0.7482
cleveland 0.2715 0.2440 0.1832 0.2941 0.2783 0.2996 0.2454 0.2466 0.2637 0.2171 0.2660
cleveland_2 0.3439 0.2845 0.2886 0.3103 0.3093 0.3130 0.3051 0.3483 0.3349 0.3002 0.3528
contraceptive 0.5347 0.5082 0.5029 0.5006 0.5369 0.5285 0.5317 0.4985 0.5026 0.5431 0.5315
glass 0.7308 0.5919 0.6194 0.7147 0.6946 0.7091 0.6980 0.7138 0.7760 0.6513 0.7454
hayes-roth 0.8577 0.8014 0.6470 0.8685 0.8745 0.8696 0.8598 0.8678 0.8748 0.7628 0.8832
ecoli 0.8112 0.7969 0.7921 0.7937 0.8051 0.8073 0.8079 0.8057 0.7958 0.8309 0.8164
vehicle 0.8983 0.9129 0.9394 0.9600 0.9538 0.9576 0.9473 0.9574 0.9607 0.8862 0.9584
wine quality 0.4294 0.3839 0.3678 0.4009 0.3914 0.4460 0.4193 0.4170 0.4041 0.4169 0.4441
yeast 0.5394 0.4337 0.5249 0.5233 0.5464 0.5339 0.5434 0.5164 0.5172 0.4947 0.5143
pageblocks 0.6805 0.4718 0.5652 0.7723 0.6006 0.6265 0.6541 0.5681 0.6030 0.5607 0.7164
newthyroid 0.9512 0.9035 0.8909 0.9273 0.9295 0.9398 0.9496 0.9221 0.9166 0.9548 0.9508

Friedman rank sum test
Friedman chi-squared = 44.509, df = 10, p-value = 2.666e-06

MultiClassCCR GMMSampler KDESampler MDO KMeansSMOTE StaticSMOTE RandomOverSampler GlobalCS SOUP AllKNN RandomUnderSampler
3.8 3.93 4.47 4.8 5.4 5.8 5.93 6.67 7.67 8.13 9.4

4.4 Average Precision Score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9846 0.9630 0.9895 0.9707 0.9894 0.9915 0.9933 0.9754 0.9940 0.9730 0.9864
art2 0.8032 0.7813 0.7677 0.7136 0.7178 0.7877 0.7955 0.7024 0.7602 0.7463 0.7429
art3 0.5820 0.5580 0.5518 0.5084 0.5801 0.5771 0.5873 0.5167 0.5153 0.5244 0.5369
art4 0.8766 0.8973 0.8811 0.7772 0.8751 0.8967 0.8878 0.7910 0.8405 0.7826 0.8540
cleveland 0.3176 0.3279 0.3353 0.3111 0.3040 0.3105 0.3174 0.3131 0.3078 0.3029 0.3165
cleveland_2 0.3734 0.3599 0.3991 0.3761 0.4000 0.3703 0.4021 0.3671 0.3758 0.3595 0.3897
contraceptive 0.5789 0.5776 0.5549 0.5323 0.5763 0.5838 0.5811 0.5193 0.5290 0.5756 0.5755
glass 0.8077 0.6842 0.7451 0.8461 0.8334 0.8341 0.8160 0.8374 0.8451 0.7852 0.8315
hayes-roth 0.9203 0.8457 0.7470 0.9465 0.9328 0.9366 0.9429 0.9514 0.9456 0.8678 0.9283
ecoli 0.8776 0.8367 0.8476 0.8385 0.8571 0.8745 0.8742 0.8517 0.8514 0.8536 0.8657
vehicle 0.9753 0.9858 0.9798 0.9898 0.9858 0.9913 0.9893 0.9919 0.9890 0.9708 0.9917
wine quality 0.4607 0.4120 0.3796 0.4212 0.4333 0.4445 0.4370 0.4394 0.4316 0.4444 0.4461
yeast 0.5552 0.4855 0.3545 0.5278 0.5314 0.5482 0.5338 0.5312 0.5290 0.5214 0.5413
pageblocks 0.8083 0.6328 0.7461 0.8195 0.8664 0.7767 0.8263 0.7986 0.8126 0.8072 0.8092
newthyroid 0.9924 0.9887 0.9687 0.9785 0.9885 0.9771 0.9927 0.9907 0.9872 0.9940 0.9958

Friedman rank sum test
Friedman chi-squared = 33.139, df = 10, p-value = 0.0002581

MDO GMMSampler KDESampler MultiClassCCR KMeansSMOTE StaticSMOTE GlobalCS RandomOverSampler AllKNN RandomUnderSampler SOUP
3.27 4.47 4.47 4.93 5.53 6.4 6.6 7.27 7.53 7.6 7.93

4.5 Matthews Correlation Coefficient

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9074 0.8580 0.9061 0.9255 0.9321 0.9254 0.9241 0.9174 0.9406 0.8934 0.9369
art2 0.6476 0.5575 0.6364 0.6094 0.6446 0.6204 0.6277 0.6046 0.6046 0.5806 0.6152
art3 0.4274 0.3564 0.4092 0.3804 0.4364 0.4087 0.4039 0.3900 0.3572 0.3558 0.3821
art4 0.6694 0.6528 0.6642 0.6624 0.6666 0.6786 0.6862 0.6545 0.6761 0.6294 0.6664
cleveland 0.1798 0.0702 0.1377 0.1468 0.1814 0.1943 0.2177 0.1486 0.1542 0.1046 0.1430
cleveland_2 0.2117 0.1888 0.1616 0.1790 0.1958 0.2359 0.2087 0.2228 0.1703 0.1875 0.2128
contraceptive 0.3209 0.2835 0.2717 0.2598 0.2969 0.3145 0.3051 0.2651 0.2512 0.3283 0.2963
glass 0.6406 0.4729 0.5908 0.6963 0.6650 0.6469 0.6408 0.6866 0.6845 0.5407 0.6389
hayes-roth 0.7645 0.6470 0.4338 0.7618 0.7634 0.7628 0.7829 0.7349 0.7665 0.6489 0.7929
ecoli 0.7933 0.7509 0.7817 0.7978 0.8033 0.7947 0.7920 0.7817 0.7870 0.8030 0.8274
vehicle 0.8402 0.9062 0.9039 0.9311 0.9275 0.9324 0.9295 0.9426 0.9400 0.8368 0.9354
wine quality 0.2824 0.2377 0.2313 0.2737 0.3020 0.2955 0.2962 0.2993 0.2743 0.2706 0.3061
yeast 0.4745 0.3228 0.4754 0.4512 0.4792 0.4745 0.4844 0.4622 0.4483 0.4349 0.4761
pageblocks 0.7776 0.4256 0.6874 0.7278 0.7258 0.7882 0.7496 0.6710 0.7523 0.5349 0.7434
newthyroid 0.9332 0.8189 0.8681 0.8922 0.9003 0.9114 0.9213 0.8995 0.8941 0.9159 0.9203

Friedman rank sum test
Friedman chi-squared = 62.17, df = 10, p-value = 1.405e-09

MDO KDESampler KMeansSMOTE MultiClassCCR GMMSampler StaticSMOTE GlobalCS RandomOverSampler AllKNN SOUP RandomUnderSampler
3.87 3.93 4 4.2 4.4 6.07 6.27 6.73 8.07 8.47 10

5 Gradient Boosting Trees

5.1 G-mean

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9605 0.9581 0.9554 0.9477 0.9635 0.9620 0.9664 0.9497 0.9677 0.9581 0.9680
art2 0.7651 0.6930 0.7162 0.6985 0.7199 0.7159 0.7166 0.6881 0.6821 0.7070 0.6979
art3 0.6383 0.5571 0.4575 0.4465 0.4611 0.5061 0.5089 0.4473 0.5032 0.5579 0.4970
art4 0.8453 0.7838 0.7801 0.7282 0.7550 0.7631 0.7663 0.7315 0.7331 0.7786 0.7635
cleveland 0.2781 0.1717 0.1049 0.1370 0.0593 0.1789 0.0599 0.1219 0.2244 0.1797 0.1340
cleveland_2 0.3471 0.2275 0.0352 0.1366 0.0534 0.2026 0.0594 0.0451 0.2020 0.2442 0.1580
contraceptive 0.5570 0.5458 0.4953 0.4979 0.5116 0.5222 0.5314 0.4919 0.4947 0.5402 0.5494
glass 0.7545 0.5927 0.2291 0.5048 0.4827 0.5133 0.6030 0.6387 0.7064 0.6793 0.6949
hayes-roth 0.8636 0.8465 0.7043 0.8432 0.8533 0.8572 0.8441 0.8618 0.8555 0.8791 0.8777
ecoli 0.7774 0.7377 0.7388 0.7739 0.7553 0.7426 0.8007 0.7589 0.7630 0.7694 0.7847
vehicle 0.9455 0.9500 0.9440 0.9627 0.9648 0.9539 0.9578 0.9610 0.9690 0.9413 0.9650
wine quality 0.4209 0.4544 0.3269 0.2259 0.3172 0.4052 0.3633 0.3041 0.3018 0.3776 0.3206
yeast 0.4792 0.3424 0.2627 0.3193 0.3086 0.2608 0.3370 0.3028 0.3713 0.3852 0.3737
pageblocks 0.4870 0.8607 0.3240 0.5214 0.3590 0.1884 0.1884 0.5271 0.3583 0.6517 0.5749
newthyroid 0.9703 0.9111 0.9080 0.9052 0.9018 0.9277 0.9282 0.8841 0.9146 0.9469 0.9136

Friedman rank sum test
Friedman chi-squared = 50.194, df = 10, p-value = 2.459e-07

GMMSampler SOUP MultiClassCCR RandomUnderSampler MDO StaticSMOTE KDESampler KMeansSMOTE RandomOverSampler GlobalCS AllKNN
2.53 4 4.33 5.33 5.67 5.8 6.2 7.33 8 8.2 8.6

5.2 Balanced Accuracy

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9589 0.9589 0.9558 0.9492 0.9565 0.9603 0.9615 0.9506 0.9679 0.9599 0.9651
art2 0.7782 0.7173 0.7446 0.7353 0.7558 0.7649 0.7504 0.7258 0.7276 0.7367 0.7325
art3 0.6502 0.5702 0.5361 0.5294 0.5718 0.5716 0.5833 0.5194 0.5470 0.5792 0.5349
art4 0.8560 0.8081 0.7986 0.7702 0.7911 0.7954 0.7996 0.7708 0.7853 0.8010 0.7952
cleveland 0.2879 0.2205 0.2830 0.2428 0.2305 0.2721 0.2513 0.2523 0.2911 0.2343 0.2673
cleveland_2 0.4093 0.3153 0.3051 0.2863 0.2794 0.3210 0.2812 0.2661 0.3586 0.2652 0.3069
contraceptive 0.5582 0.5473 0.5084 0.4981 0.5225 0.5248 0.5389 0.5001 0.4982 0.5470 0.5609
glass 0.7710 0.5788 0.5869 0.6921 0.6407 0.6394 0.6564 0.6668 0.7187 0.7144 0.7380
hayes-roth 0.8747 0.8537 0.7561 0.8588 0.8598 0.8639 0.8540 0.8696 0.8641 0.8860 0.8894
ecoli 0.7845 0.7380 0.7559 0.7758 0.7766 0.7586 0.7984 0.7611 0.7821 0.7875 0.7867
vehicle 0.9449 0.9452 0.9412 0.9622 0.9594 0.9515 0.9594 0.9594 0.9684 0.9449 0.9648
wine quality 0.4450 0.4737 0.3728 0.3958 0.4199 0.4286 0.4351 0.3851 0.3958 0.4268 0.4046
yeast 0.5423 0.4378 0.4836 0.4603 0.4864 0.4885 0.5158 0.4729 0.4796 0.5140 0.4908
pageblocks 0.6485 0.8838 0.6988 0.7212 0.6981 0.6738 0.6103 0.7147 0.6878 0.8181 0.7704
newthyroid 0.9610 0.9130 0.9108 0.9118 0.9097 0.9265 0.9318 0.8912 0.9188 0.9473 0.9198

Friedman rank sum test
Friedman chi-squared = 38.863, df = 10, p-value = 2.683e-05

GMMSampler MultiClassCCR SOUP MDO KDESampler StaticSMOTE KMeansSMOTE RandomUnderSampler RandomOverSampler AllKNN GlobalCS
3.1 4.27 4.67 5 5.53 5.6 6.87 6.9 7.87 7.93 8.27

5.3 F-score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9199 0.9046 0.9311 0.9468 0.9510 0.9336 0.9472 0.9427 0.9577 0.9204 0.9512
art2 0.7116 0.6731 0.7029 0.7075 0.7277 0.7268 0.7148 0.6973 0.6845 0.6712 0.6979
art3 0.5863 0.5313 0.5260 0.5261 0.5748 0.5851 0.5596 0.5160 0.5255 0.5334 0.5367
art4 0.7534 0.7345 0.7556 0.7330 0.7574 0.7615 0.7665 0.7420 0.7422 0.7332 0.7493
cleveland 0.2778 0.1809 0.2275 0.2800 0.2536 0.2521 0.2483 0.2310 0.3081 0.2203 0.2561
cleveland_2 0.3333 0.2504 0.2659 0.2685 0.2699 0.3181 0.2841 0.3181 0.3694 0.2814 0.3262
contraceptive 0.5453 0.5345 0.4956 0.5067 0.5244 0.5288 0.5359 0.4941 0.4990 0.5368 0.5519
glass 0.7237 0.5512 0.5408 0.6970 0.6544 0.6884 0.6545 0.6905 0.7327 0.6161 0.7336
hayes-roth 0.8739 0.8583 0.7436 0.8533 0.8599 0.8663 0.8533 0.8741 0.8738 0.8852 0.8886
ecoli 0.7716 0.7262 0.7694 0.7958 0.7766 0.7721 0.7947 0.7704 0.7764 0.7761 0.7962
vehicle 0.9224 0.9382 0.9370 0.9558 0.9573 0.9507 0.9558 0.9507 0.9607 0.9193 0.9617
wine quality 0.4147 0.3820 0.3648 0.3975 0.4261 0.4289 0.4328 0.3970 0.3839 0.4147 0.4013
yeast 0.5172 0.3638 0.5010 0.4672 0.4948 0.4959 0.5231 0.4762 0.4796 0.4524 0.4787
pageblocks 0.6860 0.5760 0.7233 0.7026 0.6923 0.6305 0.5628 0.7205 0.6985 0.6572 0.7445
newthyroid 0.9417 0.8643 0.8991 0.9148 0.9070 0.9270 0.9252 0.8956 0.9142 0.9228 0.9151

Friedman rank sum test
Friedman chi-squared = 47.684, df = 10, p-value = 7.088e-07

MultiClassCCR GMMSampler MDO KDESampler KMeansSMOTE StaticSMOTE RandomOverSampler GlobalCS SOUP AllKNN RandomUnderSampler
3.4 4.4 4.63 4.8 4.93 5.27 6.23 7.27 7.4 8.13 9.53

5.4 Average Precision Score

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.9838 0.9658 0.9873 0.9871 0.9908 0.9874 0.9921 0.9872 0.9920 0.9783 0.9921
art2 0.7956 0.7761 0.7951 0.7853 0.7973 0.8195 0.8156 0.7890 0.8083 0.7241 0.7950
art3 0.6218 0.5714 0.5800 0.5526 0.5638 0.6132 0.6383 0.5374 0.5563 0.5342 0.5472
art4 0.9036 0.8488 0.8746 0.8542 0.8825 0.8936 0.9011 0.8357 0.8896 0.8391 0.8817
cleveland 0.3110 0.2657 0.3604 0.2973 0.2867 0.3011 0.2887 0.2899 0.3167 0.3007 0.2907
cleveland_2 0.3834 0.3078 0.4005 0.3572 0.3603 0.3655 0.3668 0.3361 0.3923 0.3454 0.3795
contraceptive 0.5825 0.5751 0.5392 0.5325 0.5755 0.5842 0.5827 0.5228 0.5311 0.5766 0.5715
glass 0.7927 0.5877 0.6546 0.7511 0.7191 0.7265 0.7364 0.7808 0.7902 0.6900 0.7820
hayes-roth 0.9534 0.8863 0.7886 0.9540 0.9536 0.9552 0.9544 0.9546 0.9513 0.9184 0.9480
ecoli 0.8555 0.8014 0.8241 0.8266 0.8322 0.8432 0.8332 0.8275 0.8311 0.8494 0.8387
vehicle 0.9746 0.9906 0.9836 0.9905 0.9917 0.9922 0.9930 0.9922 0.9919 0.9702 0.9929
wine quality 0.4269 0.3785 0.3735 0.4175 0.4207 0.4303 0.4406 0.4302 0.4025 0.4216 0.4232
yeast 0.5243 0.4185 0.3251 0.5086 0.4932 0.5041 0.5215 0.5007 0.4983 0.4816 0.5063
pageblocks 0.7677 0.7151 0.7723 0.8086 0.7888 0.7701 0.7911 0.8588 0.7996 0.7332 0.8217
newthyroid 0.9943 0.8941 0.8876 0.9256 0.9693 0.9770 0.9796 0.9457 0.9726 0.9934 0.9738

Friedman rank sum test
Friedman chi-squared = 52.085, df = 10, p-value = 1.101e-07

MDO KDESampler GMMSampler MultiClassCCR StaticSMOTE KMeansSMOTE GlobalCS RandomOverSampler AllKNN SOUP RandomUnderSampler
3.27 3.8 3.93 4.93 5.2 6.27 6.67 7 7.53 7.87 9.53

5.5 Matthews Correlation Coefficient

Dataset GMMSampler RandomUnderSampler AllKNN RandomOverSampler KMeansSMOTE KDESampler MDO GlobalCS StaticSMOTE SOUP MultiClassCCR
art1 0.8838 0.8511 0.8959 0.9162 0.9267 0.9028 0.9159 0.9125 0.9344 0.8881 0.9222
art2 0.6278 0.5633 0.6343 0.6362 0.6689 0.6732 0.6447 0.6196 0.6114 0.5882 0.6237
art3 0.4389 0.3205 0.3865 0.3689 0.4825 0.4569 0.4222 0.3607 0.3584 0.3683 0.3763
art4 0.6614 0.6361 0.6718 0.6372 0.6729 0.6735 0.6785 0.6445 0.6662 0.6290 0.6622
cleveland 0.1451 0.0635 0.1642 0.1449 0.1599 0.1676 0.1714 0.1325 0.1779 0.0687 0.1285
cleveland_2 0.2535 0.0759 0.1500 0.1138 0.1502 0.1604 0.1442 0.1211 0.1829 0.1062 0.1415
contraceptive 0.3329 0.3161 0.2707 0.2578 0.3226 0.2994 0.3107 0.2591 0.2499 0.3241 0.3361
glass 0.6475 0.3664 0.5526 0.6358 0.6142 0.6156 0.6231 0.6620 0.6720 0.4981 0.6775
hayes-roth 0.7803 0.7507 0.5599 0.7314 0.7365 0.7602 0.7314 0.7736 0.7746 0.7502 0.8088
ecoli 0.7289 0.7208 0.7790 0.7669 0.7754 0.7768 0.7791 0.7482 0.7582 0.7538 0.7801
vehicle 0.8811 0.9038 0.9040 0.9344 0.9360 0.9264 0.9287 0.9308 0.9358 0.8782 0.9454
wine quality 0.2582 0.2210 0.1941 0.2572 0.3180 0.2605 0.2936 0.2478 0.2484 0.2080 0.2528
yeast 0.4296 0.3041 0.4402 0.4128 0.4529 0.4500 0.4418 0.4104 0.4042 0.4126 0.4110
pageblocks 0.7435 0.5146 0.7358 0.7617 0.7205 0.6779 0.7278 0.7506 0.7604 0.5223 0.7972
newthyroid 0.9078 0.8049 0.8424 0.8734 0.8501 0.8930 0.8868 0.8447 0.8696 0.8780 0.8765

Friedman rank sum test
Friedman chi-squared = 51.006, df = 10, p-value = 1.742e-07

KMeansSMOTE MultiClassCCR KDESampler MDO GMMSampler StaticSMOTE RandomOverSampler AllKNN GlobalCS SOUP RandomUnderSampler
4.13 4.2 4.27 4.37 5.07 5.4 6.23 6.73 7.13 8.53 9.93