[FieldTrip] General matlab script for cluster based permutation testing?

[Robert Oostenveld]

Hi Markus

On 09 Sep 2015, at 13:37, Markus Gschwind <markus.gschwind at gmail.com> wrote:

> => So is it right to choose the threshold for the positive T-values and the negative T-values separately?

It is allowed, but not required. If you know that your massinve univariate statistical values form a symetrical distribution, you could also have a single threshold (and use that plus/minus).

But is is good that you raise this point. In the FieldTrip implementation we are in general performing seperate tests for the clusters that exceed the positive and the negative threshold. I.e. we make separate histograms of cluster mass, and separately decide the probability of observing the clusters in the data (given H0).

See http://www.fieldtriptoolbox.org/faq/why_should_i_use_the_cfg.correcttail_option_when_using_statistics_montecarlo

If you choose to use a single threshold, and assume symmetry, you can also concatenate the positive and negative cluster mass histograms (after correcting for the sign) and do a single test (again correcting for sign in the observed clusters). This basically boils down to using abs(t) as the test statistic and only doing a single sided test.

> => Would you then choose the outer 97.5 percentile as the threshold  (i.e. T>7.2 for the positive and T< -6.3 for the negative values)? 

The choice of the threshold for clustering does not influence the validity of the statistical test. It will affect the sensitivity of the test. Choosing it too small or too large will decrease sensitivity. But note that this is not a parameter you are allowed to play with, i.e. trying out many values for the threshold, and then reporting only the outcomes that you like. This would constitute multiple testing, and you would have to (Bonferroni) correct for it. 

> However this is drastically more restrictive than a T-value of >2.0097 in case of a single two-sided paired T-test with df=49 (N=50), which would be applied without any correction.
> 
> The choice of this initial threshold defines the cluster size!

Yes. There are many more choices in the analysis that will determine the cluster size, such as temporal filtering, or spectral smoothing. As long as the choice is consistently applied assuming that H0 holds, it does not affect the validity of the test. But it does affect the sensitivity (as previously mentioned).

best
Robert



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150909/287b6182/attachment-0002.html>