Difference between revisions of "Statistical Analysis for More than Two Levels"
(Created page with "It is not straight-forward to run permutation (M)ANOVAs. This would of course be the best way to treat data with more than two levels per variable. We are working on it with a...") |
m |
||
| (4 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | {{BESAInfobox | |
| + | |title = Module information | ||
| + | |module = BESA Statistics | ||
| + | |version = BESA Statistics 1.0 or higher | ||
| + | }} | ||
| − | The same principle holds true if one has more than two levels per factor. Let us assume you have 3 factor levels. The difficulty here is that the variance needs to be comparable in all factor levels. If it is not, the sphericity assumption is violated and the results are not valid (one might achieve significant results, although in truth they are not, and more rarely the other way round). So, in order to use the difference approach for more than two factor levels, you would need to compare the variance between the levels. It is not so easy to do this in time-series EEG data, as it is not clear, which time-window and sensor group to choose. So you would have to first calculate the differences (e.g. level 1 minus level 2 and level 1 minus level 3), compare the differences with a permutation test, and then use the cluster results for the sphericity test (you can run Levene’s test for homogeneous variances). The same would need to be repeated for the differences level 2 minus level | + | The easiest way to treat data with more than two levels per variable is to work with differences. |
| + | |||
| + | Consider this example for a 2×2 ANOVA: You have two groups, ''"Patients"'' and ''"Controls"''. Each have two measurements, ''"Time 1"'' and ''"Time 2"''. The question is, whether patients and controls change differently from time 1 to time 2. In this case one would subtract condition 1 from condition 2 per group and compare the differences. If this comparison becomes significant, it is identical to the interaction group × time. Now it is necessary to investigate the source of the interaction, as it is not clear from the differing differences (sorry!), if the groups differ in time 1 and not time 2, or if one group changes from time 1 to time 2 and the other group does not, etc. This could be done by planned comparisons in any statistical program (usually simple paired or unpaired t-tests), i.e. making only meaningful comparisons (e.g. controls time 1 vs. patients time 1). To do so, you should use the data (i.e. the mean) of the clusters that became significant in the difference comparison. If you want to be 100% strict, you would need to adjust the p-value of the planned comparisons, as this again means running multiple tests. A good way to do this is the Bonferroni-Holm correction, which is conservative, but not as conservative as the original Bonferroni correction. It is very easy to apply: [https://en.wikipedia.org/wiki/Holm%E2%80%93Bonferroni_method Holm–Bonferroni method] | ||
| + | |||
| + | The same principle holds true if one has more than two levels per factor. Let us assume you have 3 factor levels. The difficulty here is that the variance needs to be comparable in all factor levels. If it is not, the sphericity assumption is violated and the results are not valid (one might achieve significant results, although in truth they are not, and more rarely the other way round). So, in order to use the difference approach for more than two factor levels, you would need to compare the variance between the levels. It is not so easy to do this in time-series EEG data, as it is not clear, which time-window and sensor group to choose. So you would have to first calculate the differences (e.g. level 1 minus level 2 and level 1 minus level 3), compare the differences with a permutation test, and then use the cluster results for the sphericity test (you can run Levene’s test for homogeneous variances). The same would need to be repeated for the differences level 2 minus level 1 and level 2 minus level 3. | ||
| + | |||
| + | For additional information visit the FAQ page of FieldTrip: [http://www.fieldtriptoolbox.org/faq/how_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests How to test an interaction effect using cluster-based permutation tests?] | ||
| + | |||
| + | [[Category:Statistics]] | ||
Latest revision as of 14:32, 5 May 2021
| Module information | |
| Modules | BESA Statistics |
| Version | BESA Statistics 1.0 or higher |
The easiest way to treat data with more than two levels per variable is to work with differences.
Consider this example for a 2×2 ANOVA: You have two groups, "Patients" and "Controls". Each have two measurements, "Time 1" and "Time 2". The question is, whether patients and controls change differently from time 1 to time 2. In this case one would subtract condition 1 from condition 2 per group and compare the differences. If this comparison becomes significant, it is identical to the interaction group × time. Now it is necessary to investigate the source of the interaction, as it is not clear from the differing differences (sorry!), if the groups differ in time 1 and not time 2, or if one group changes from time 1 to time 2 and the other group does not, etc. This could be done by planned comparisons in any statistical program (usually simple paired or unpaired t-tests), i.e. making only meaningful comparisons (e.g. controls time 1 vs. patients time 1). To do so, you should use the data (i.e. the mean) of the clusters that became significant in the difference comparison. If you want to be 100% strict, you would need to adjust the p-value of the planned comparisons, as this again means running multiple tests. A good way to do this is the Bonferroni-Holm correction, which is conservative, but not as conservative as the original Bonferroni correction. It is very easy to apply: Holm–Bonferroni method
The same principle holds true if one has more than two levels per factor. Let us assume you have 3 factor levels. The difficulty here is that the variance needs to be comparable in all factor levels. If it is not, the sphericity assumption is violated and the results are not valid (one might achieve significant results, although in truth they are not, and more rarely the other way round). So, in order to use the difference approach for more than two factor levels, you would need to compare the variance between the levels. It is not so easy to do this in time-series EEG data, as it is not clear, which time-window and sensor group to choose. So you would have to first calculate the differences (e.g. level 1 minus level 2 and level 1 minus level 3), compare the differences with a permutation test, and then use the cluster results for the sphericity test (you can run Levene’s test for homogeneous variances). The same would need to be repeated for the differences level 2 minus level 1 and level 2 minus level 3.
For additional information visit the FAQ page of FieldTrip: How to test an interaction effect using cluster-based permutation tests?
