[FieldTrip] Cluster-based permutation tests

[Schoffelen, J.M. (Jan Mathijs)]

Hi Zhipeng,

 I think that the documentation on the website is clear enough about this, but to reiterate:

When you want to do non-parametric statistical inference, using permutation tests you specify cfg.method = ‘montecarlo’. This does not specify yet how to address the multiple comparisons problem.

The correction scheme for the multiple comparisons problem is specified by cfg.correctm. This can be set to ‘cluster’, which means that you use a clustering heuristic to reduce the number of comparisons to 1 (or 2, if two-sided testing is involved).

The test statistic that you boil out of your data - and that is going to be used to construct the permutation distribution under the null hypothesis of exchangeability - is specified by cfg.statistic. The value for this option should refer to an existing matlab m-file that converts the input data into the required test statistic. Fieldtrip comes shipped with a few pre-cooked ones, e.g. the (in)dependent samples T-statistic. If you want some more adventurous test statistics, such as the one you mentioned, you need to write your own ’statfun’. Feel free to contribute your own statfuns as a pull request to the fieldtrip code base.


Best wishes and good luck,

Jan-Mathijs


> On 20 Oct 2022, at 10:23, zhipeng li via fieldtrip <fieldtrip at science.ru.nl> wrote:
> 
> Dear community,
> 
> 1. When I make multiple comparisons based on cluster, does cfg.statistic  = 'ft_statfun_depsamplesT' refer to the test of each sample in the first step using  paired t-test? And cfg.method  = 'montecarlo' refers to the second cluster test using a permutation test?
> 
> 2. If that's the case, and we didn't adjust for multiple comparisons in example 2 (cfg.correctm   = 'no'), then cfg.statistic   = 'ft_statfun_depsamplesT' is a paired t-test, and what is cfg.method  = 'montecarlo'?
> 
> 3. If this is the case, I would like to use non-parametric tests such as the Wilcoxon test and Kruskal-Wallis test in the first step of each sample test, which parameter in the cfg should be modified?
> 
> Example 1：Cluster-based permutation in FieldTrip
> 
> cfg = [];
> cfg.channel     = 'MEG';
> cfg.neighbours  = neighbours; % defined as above
> cfg.latency     = [0.3 0.7];
> cfg.avgovertime = 'yes';
> cfg.parameter   = 'avg';
> cfg.method      = 'montecarlo';
> cfg.statistic   = 'ft_statfun_depsamplesT';
> cfg.alpha       = 0.05;
> cfg.correctm    = 'cluster';
> cfg.correcttail = 'prob';
> cfg.numrandomization = 'all';  
> 
> Example 2：Permutation test based on t statistics in FieldTrip
> 
> cfg = [];
> cfg.channel     = 'MEG';
> cfg.latency     = [0.3 0.7];
> cfg.avgovertime = 'yes';
> cfg.parameter   = 'avg';
> cfg.method      = 'montecarlo';
> cfg.statistic   = 'ft_statfun_depsamplesT';
> cfg.alpha       = 0.05;
> cfg.correctm    = 'no';
> cfg.correcttail = 'prob';
> cfg.numrandomization = 'all'; 
> 
> Best,
> Zhipeng
> 
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!6plwN9AGqY-PwZFmuMuA5NLPiL9zZ6oQM3C3vFAhrWI6RWLh-D5Xvwq1S5O7Io0ezYDxf2HbJQi9p8eVwRtO4OlMvYpJ-668T202uA$