[FieldTrip] time windows and number of channels in cluster based permutation tests

Stephen Politzer-Ahles politzerahless at gmail.com
Fri Feb 17 12:49:06 CET 2017

Dear Michele,

The general idea is, when you get a p-value out of the cluster test, it's
evaluated against a permutation distribution---the 'real' cluster is
getting compared against all the fake clusters that pop up when you shuffle
the data randomly. Roughly speaking, when you analyze more timepoints and
more channels, there's more chances for random nonsense clusters to pop up
in each permutation of randomly shuffled data; thus, there's more chances
to get random nonsense clusters that have a bigger test statistic than your
real cluster. Since the p-value is just the proportion of random clusters
that have a bigger test statistic than the real cluster, testing more
timepoints and/or more channels means you might end up with a higher

Anyway, the selection of channels and time ranges has to be chosen *a
priori*, not based on the data (choosing which window to analyze after
you've seen where the effect is would be a non-independent / double-dipping
analysis), so since you seem to have found a significant effect based on
the samples you had identified a priori, I don't think you have to worry
about the whole-head analysis.


Stephen Politzer-Ahles
The Hong Kong Polytechnic University
Department of Chinese and Bilingual Studies

On Fri, Feb 17, 2017 at 7:00 PM, <fieldtrip-request at science.ru.nl> wrote:

> Send fieldtrip mailing list submissions to
>         fieldtrip at science.ru.nl
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> or, via email, send a message with subject or body 'help' to
>         fieldtrip-request at science.ru.nl
> You can reach the person managing the list at
>         fieldtrip-owner at science.ru.nl

> Message: 3
> Date: Fri, 17 Feb 2017 10:33:52 +0100
> From: Michele Scaltritti <michele.scaltritti at gmail.com>
> To: fieldtrip at science.ru.nl
> Subject: [FieldTrip] time windows and number of channels in cluster
>         based   permutation tests
> Message-ID:
>         <CABGEo=TM-TGqh3bQubOD_29z-cxQ93grvXQHNMo70zqLcpjjEg@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> Dear FieldTrip users,
> I have a rather general question regarding cluster-based permutation test.
> In the tutorial for Cluster-based permutation tests on event related
> fields, I read that sensitivity of the analysis depends on the length of
> the time interval that is analyzed. Elsewhere, I found a similar reasoning
> regarding channels, in the sense that it the exclusion of the sensors where
> no effect is likely to be present should yield a more sensitive analysis.
> In my experiment, I analyzed a limited set of central electrodes (where
> previous evidence suggests that the difference should be located) and found
> a significant positive cluster, and a (smaller) significant negative one.
> When doing the same analysis considering all the electrodes, no significant
> negative cluster is found, just a positive one. Among other things, this
> may reflect a different sensitivity as a function of the number of channels
> considered in the analysis. I explored a bit the different parameters (such
> as neighbors channels definition), but I did not identify the source of the
> difference between the analyses.
> Leaving aside my specific case, in more general terms I cannot understand
> how sensitivity is increased by focusing on a smaller time-windows, or on a
> specific set of electrodes. After all, the number of comparisons we make
> should not affect the number of samples surpassing the threshold, nor the
> clustering as a function of the spatial and temporal adjacency of these
> supra-theshold samples. I understand that the same cluster may vary when
> other samples are considered in the analyses (for example, samples from
> other electrodes may get included), but again, I am failing to see how this
> may be related to the sensitivity of the analysis. The cluster-level
> statistics, for example, should not be weakened by this.
> I apologize if this message underlies any gross misunderstanding.
> Michele Scaltritti
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/
> attachments/20170217/c376010b/attachment-0001.html>
> ------------------------------
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> End of fieldtrip Digest, Vol 75, Issue 19
> *****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20170217/baec7eda/attachment-0002.html>

More information about the fieldtrip mailing list