1 Comparing the related approaches

1.1 G-mean for J48 classifier

Dataset	baseline	Global CS	OVO ROS	OVO RUS	OVO NCR	OVA ROS	OVA RUS	OVA NCR	Static-SMOTE
balance_scale	0.000	0.340	0.526	0.602	0.474	0.302	0.297	0.000	0.080
car	0.847	0.940	0.939	0.876	0.919	0.112	0.184	0.130	0.897
cleveland_1	0.227	0.000	0.255	0.287	0.262	0.254	0.259	0.000	0.052
cleveland_2	0.000	0.000	0.288	0.285	0.000	0.280	0.287	0.000	0.037
cmc_2	0.483	0.478	0.509	0.514	0.526	0.510	0.511	0.529	0.452
dermatology	0.945	0.952	0.921	0.929	0.948	0.000	0.000	0.000	0.927
ecoli	0.728	0.710	0.805	0.767	0.000	0.000	0.000	0.000	0.738
flare	0.446	0.570	0.544	0.568	0.522	0.000	0.000	0.000	0.421
glass	0.625	0.715	0.699	0.697	0.691	0.000	0.000	0.000	0.322
hayes_roth	0.843	0.832	0.843	0.843	0.838	0.000	0.000	0.000	0.835
led7digit	0.786	0.770	0.771	0.779	0.722	0.120	0.162	0.156	0.756
thyroid	0.889	0.922	0.922	0.886	0.913	0.904	0.927	0.898	0.879
vehicle	0.912	0.912	0.916	0.923	0.915	0.133	0.141	0.164	0.915
wine_quality	0.432	0.464	0.492	0.476	0.434	0.459	0.489	0.356	0.356
yeast	0.000	0.406	0.442	0.479	0.000	0.000	0.000	0.000	0.184
art1	0.945	0.961	0.958	0.949	0.949	0.039	0.000	0.039	0.947
art2	0.686	0.734	0.758	0.777	0.762	0.250	0.253	0.244	0.741
art3	0.410	0.534	0.615	0.612	0.559	0.307	0.304	0.236	0.535
art4	0.785	0.829	0.840	0.872	0.839	0.000	0.000	0.000	0.856

Friedman rank sum test
Friedman chi-squared = 67.913, df = 8, p-value = 1.278e-11

OVO ROS	OVO RUS	Global CS	OVO NCR	Static-SMOTE	baseline	OVA RUS	OVA ROS	OVA NCR
2.55	2.55	4.11	4.13	5.32	5.55	6.24	7	7.55

1.2 Avg Minority for J48 classifier

Dataset	baseline	OVO ROS	OVO RUS	OVO NCR	OVA ROS	OVA RUS	OVA NCR	Global CS
balance scale	0.000	0.245	0.653	0.163	0.041	0.041	0.000	0.061
car	0.777	0.963	0.986	0.926	0.090	0.296	0.127	0.955
cleveland 1	0.171	0.213	0.263	0.226	0.217	0.252	0.188	0.197
cleveland 2	0.103	0.251	0.269	0.236	0.199	0.240	0.160	0.169
cmc 2	0.360	0.562	0.619	0.595	0.535	0.643	0.562	0.420
dermatology	0.949	0.903	0.933	0.940	0.323	0.313	0.340	0.963
ecoli	0.647	0.774	0.807	0.645	0.028	0.059	0.038	0.669
flare	0.169	0.321	0.432	0.224	0.000	0.000	0.000	0.396
glass	0.551	0.704	0.896	0.749	0.045	0.104	0.039	0.684
hayes roth	1.000	1.000	1.000	1.000	0.000	0.000	0.000	1.000
led7digit	0.750	0.721	0.751	0.752	0.040	0.129	0.069	0.739
thyroid	0.857	0.910	0.893	0.893	0.893	0.955	0.879	0.910
vehicle	0.905	0.932	0.942	0.934	0.059	0.063	0.078	0.909
wine quality red	0.345	0.401	0.525	0.324	0.361	0.552	0.330	0.373
yeast	0.401	0.448	0.498	0.173	0.011	0.090	0.000	0.406
1czysty	0.938	0.954	0.979	0.948	0.008	0.004	0.008	0.960
2delikatne	0.606	0.715	0.765	0.717	0.144	0.156	0.146	0.681
3mocniej	0.340	0.558	0.573	0.452	0.200	0.200	0.146	0.463
4delikatne bezover	0.740	0.844	0.940	0.827	0.015	0.015	0.019	0.827

Friedman rank sum test
Friedman chi-squared = 76.684, df = 7, p-value = 6.521e-14

OVO RUS	OVO ROS	OVO NCR	Global CS	OVA RUS	baseline	OVA ROS	OVA NCR
1.65	3.05	3.48	3.8	5.28	5.47	6.6	6.67

1.3 F1 measure for J48 classifier

Dataset	baseline	OVO ROS	OVO RUS	OVO NCR	OVA ROS	OVA RUS	OVA NCR	Global CS
balance scale	0.000	0.128	0.202	0.133	0.033	0.031	0.000	0.038
car	0.769	0.895	0.532	0.855	0.081	0.211	0.112	0.831
cleveland 1	0.175	0.189	0.186	0.192	0.177	0.169	0.180	0.179
cleveland 2	0.117	0.220	0.141	0.171	0.174	0.175	0.148	0.148
cmc 2	0.382	0.441	0.458	0.458	0.442	0.460	0.461	0.391
dermatology	0.946	0.878	0.907	0.954	0.308	0.263	0.324	0.946
ecoli	0.675	0.735	0.674	0.452	0.028	0.050	0.038	0.673
flare	0.226	0.232	0.272	0.276	0.000	0.000	0.000	0.318
glass	0.577	0.593	0.572	0.600	0.042	0.050	0.034	0.675
hayes roth	1.000	1.000	1.000	0.954	0.000	0.000	0.000	1.000
led7digit	0.734	0.682	0.697	0.654	0.035	0.092	0.068	0.684
thyroid	0.878	0.894	0.818	0.891	0.853	0.844	0.863	0.894
vehicle	0.906	0.904	0.909	0.896	0.059	0.062	0.076	0.902
wine quality red	0.366	0.365	0.297	0.308	0.356	0.316	0.314	0.362
yeast	0.405	0.415	0.325	0.120	0.011	0.026	0.000	0.377
1czysty	0.922	0.938	0.862	0.917	0.008	0.004	0.008	0.937
2delikatne	0.649	0.642	0.622	0.633	0.133	0.134	0.135	0.628
3mocniej	0.400	0.465	0.437	0.444	0.181	0.168	0.139	0.408
4delikatne bezover	0.748	0.751	0.727	0.757	0.014	0.013	0.017	0.743

Friedman rank sum test
Friedman chi-squared = 55.975, df = 7, p-value = 9.554e-10

OVO ROS	OVO NCR	Global CS	baseline	OVO RUS	OVA NCR	OVA RUS	OVA ROS
2.42	3.17	3.55	4.03	4.1	6.05	6.2	6.47

2 Parametrization of SOUP algorithm on G-mean and J48 classifier

Dataset	SOUP SIM1	SOUP SIM2	SOUP SIM3	SOUP SIM4	SOUP SIM5	SOUP SIM6	SOUP Heur
balance_scale	0.598	0.614	0.598	0.576	0.598	0.598	0.585
car	0.938	0.938	0.938	0.932	0.938	0.940	0.941
cleveland_1	0.272	0.285	0.285	0.208	0.275	0.208	0.266
cleveland_2	0.256	0.256	0.000	0.305	0.000	0.305	0.303
cmc_2	0.520	0.520	0.528	0.518	0.531	0.528	0.535
dermatology	0.960	0.960	0.960	0.960	0.960	0.960	0.962
ecoli	0.717	0.721	0.762	0.745	0.710	0.727	0.735
flare	0.586	0.575	0.573	0.560	0.568	0.571	0.566
glass	0.662	0.667	0.672	0.715	0.671	0.672	0.667
hayes_roth	0.818	0.817	0.828	0.825	0.833	0.828	0.835
led7digit	0.780	0.790	0.772	0.772	0.774	0.777	0.778
thyroid	0.922	0.922	0.922	0.922	0.922	0.922	0.922
vehicle	0.909	0.909	0.909	0.910	0.909	0.910	0.915
wine_quality	0.449	0.448	0.465	0.454	0.469	0.482	0.472
yeast	0.371	0.398	0.407	0.464	0.434	0.410	0.451
art1	0.960	0.960	0.960	0.960	0.960	0.960	0.960
art2	0.777	0.777	0.777	0.793	0.779	0.793	0.777
art3	0.609	0.594	0.605	0.605	0.602	0.605	0.608
art4	0.899	0.899	0.899	0.899	0.899	0.899	0.899

Friedman rank sum test
Friedman chi-squared = 6.5599, df = 6, p-value = 0.3635

SOUP Heur	SOUP SIM6	SOUP SIM3	SOUP SIM5	SOUP SIM4	SOUP SIM2	SOUP SIM1
3.34	3.37	4	4.16	4.24	4.42	4.47

3 Comparing SOUP vs. the best-performing related

3.1 Comparision for J48 classifier on G-mean

Dataset	OVO ROS	OVO RUS	OVO NCR	OVO SO	OVO SU	SOUP	MRBB
balance_scale	0.526	0.602	0.474	0.542	0.547	0.585	0.683
car	0.939	0.876	0.919	0.940	0.794	0.941	0.907
cleveland_1	0.255	0.287	0.262	0.268	0.302	0.266	0.021
cleveland_2	0.288	0.285	0.000	0.284	0.312	0.303	0.055
cmc_2	0.509	0.514	0.526	0.522	0.524	0.535	0.517
dermatology	0.921	0.929	0.948	0.925	0.939	0.962	0.959
ecoli	0.805	0.767	0.000	0.791	0.739	0.735	0.768
flare	0.544	0.568	0.522	0.582	0.506	0.566	0.542
glass	0.699	0.697	0.691	0.701	0.697	0.667	0.400
hayes_roth	0.843	0.843	0.838	0.843	0.775	0.835	0.823
led7digit	0.771	0.779	0.722	0.765	0.704	0.778	0.778
thyroid	0.922	0.886	0.913	0.897	0.896	0.922	0.932
vehicle	0.916	0.923	0.915	0.904	0.880	0.915	0.943
wine_quality	0.492	0.476	0.434	0.524	0.490	0.472	0.525
yeast	0.442	0.479	0.000	0.000	0.484	0.451	0.201
art1	0.958	0.949	0.949	0.959	0.951	0.960	0.960
art2	0.758	0.777	0.762	0.754	0.804	0.777	0.808
art3	0.615	0.612	0.559	0.627	0.634	0.608	0.631
art4	0.840	0.872	0.839	0.831	0.878	0.899	0.893

Friedman rank sum test
Friedman chi-squared = 12.205, df = 6, p-value = 0.05756

SOUP	MRBB	OVO RUS	OVO SO	OVO ROS	OVO SU	OVO NCR
3.29	3.37	3.82	3.97	4	4.13	5.42

3.2 Comparision for kNN classifier on G-mean

Dataset	OVO ROS	OVO RUS	OVO NCR	OVO SO	OVO SU	SOUP	MRBB
balance scale	0.000	0.582	0.531	0.000	0.668	0.692	0.727
car	0.788	0.780	0.649	0.775	0.778	0.755	0.709
cleveland 1	0.234	0.229	0.226	0.234	0.272	0.263	0.046
cleveland 2	0.287	0.350	0.299	0.287	0.293	0.339	0.072
cmc 2	0.439	0.434	0.476	0.436	0.468	0.461	0.463
dermatology	0.954	0.947	0.954	0.954	0.939	0.952	0.957
ecoli	0.757	0.754	0.000	0.757	0.743	0.725	0.809
flare	0.515	0.527	0.484	0.516	0.503	0.534	0.527
glass	0.633	0.601	0.644	0.633	0.630	0.637	0.325
hayes roth	0.736	0.742	0.758	0.736	0.781	0.661	0.394
led7digit	0.754	0.766	0.575	0.787	0.677	0.769	0.784
thyroid	0.952	0.941	0.943	0.952	0.932	0.960	0.917
vehicle	0.909	0.900	0.911	0.909	0.870	0.900	0.917
wine quality red	0.458	0.471	0.468	0.458	0.461	0.484	0.444
yeast	0.406	0.431	0.000	0.406	0.430	0.432	0.341
art1	0.962	0.959	0.974	0.962	0.958	0.973	0.949
art2	0.690	0.765	0.744	0.690	0.784	0.781	0.805
art3	0.456	0.521	0.517	0.456	0.569	0.540	0.567
art4	0.751	0.849	0.826	0.751	0.877	0.880	0.896

Friedman rank sum test
Friedman chi-squared = 6.7514, df = 6, p-value = 0.3445

SOUP	OVO SU	MRBB	OVO RUS	OVO NCR	OVO SO	OVO ROS
3	3.85	3.95	4.1	4.1	4.45	4.55

3.3 Comparision for PART classifier on G-mean

Dataset	OVO ROS	OVO RUS	OVO NCR	OVO SO	OVO SU	SOUP	MRBB
balance scale	0.678	0.637	0.596	0.663	0.563	0.588	0.754
car	0.923	0.864	0.952	0.927	0.807	0.940	0.937
cleveland 1	0.181	0.309	0.277	0.259	0.300	0.238	0.063
cleveland 2	0.000	0.345	0.365	0.000	0.304	0.000	0.092
cmc 2	0.508	0.480	0.511	0.502	0.509	0.510	0.505
dermatology	0.899	0.914	0.952	0.924	0.922	0.928	0.961
ecoli	0.788	0.768	0.000	0.752	0.736	0.769	0.794
flare	0.514	0.584	0.538	0.566	0.498	0.557	0.544
glass	0.683	0.689	0.689	0.651	0.720	0.640	0.445
hayes roth	0.855	0.855	0.849	0.855	0.787	0.840	0.836
led7digit	0.778	0.777	0.631	0.784	0.659	0.791	0.778
thyroid	0.922	0.886	0.913	0.887	0.896	0.923	0.926
vehicle	0.916	0.926	0.928	0.924	0.892	0.913	0.954
wine quality red	0.429	0.454	0.426	0.479	0.452	0.505	0.536
yeast	0.000	0.490	0.000	0.000	0.450	0.458	0.349
art1	0.960	0.940	0.942	0.954	0.948	0.943	0.958
art2	0.781	0.783	0.756	0.805	0.807	0.788	0.805
art3	0.620	0.625	0.559	0.603	0.641	0.615	0.632
art4	0.880	0.873	0.834	0.874	0.882	0.896	0.891

Friedman rank sum test
Friedman chi-squared = 6.0217, df = 6, p-value = 0.4208

MRBB	SOUP	OVO SO	OVO RUS	OVO SU	OVO ROS	OVO NCR
3.05	3.8	4	4.05	4.2	4.45	4.45

Using Information on Class Interrelations to Improve Classification of Multi-class Imbalanced Data: A New Re-sampling Algorithm

Małgorzata Janicka, Mateusz Lango, Jerzy Stefanowski

2 Parametrization of SOUP algorithm on G-mean and J48 classifier

3 Comparing SOUP vs. the best-performing related

3.1 Comparision for J48 classifier on G-mean

3.2 Comparision for kNN classifier on G-mean

3.3 Comparision for PART classifier on G-mean