Freqstatistics Yields Zero Significant Clusters?

[Eric Maris]

Dear Erick,


> Following up on the discussion, we have also tried to use the cluster
> option for multiple comparisons correction in a previous study and
> thought that our results were not significant. Now, we tried again with
> 1000 randomizations and cfg.correctm = 'cluster', to no avail. Since we
> are interested in p<0.05, 1000 iterations seem large enough, but most of
> the probabilities calculated are near or equal to 100%. Further testing
> with 10000 permutations yielded similar results.
> 
> We are fairly sure that there is something there, because calculating
> analytically we see big clusters that extend in a consistent way to
> neighboring frequency and time bins, even at low p thresholds (<10^-4).
> Since Bonferroni is over-strict, Montecarlo is the option of choice for
> a corrected result that we will trust. But now something seems off.


Can you post the configuration that you used for these analyses?


dr. Eric Maris
Donders Institute for Brain, Cognition and Behavior
Center for Cognition and F.C. Donders Center for Cognitive Neuroimaging
Radboud University
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
T:+31 24 3612651
F:+31 24 3616066
E: e.maris at donders.ru.nl
 
MSc Cognitive Neuroscience: www.ru.nl/master/cns/


> 
> 
> Thank you in advance,
> Erick Britis Ortiz
> MEG-Zentrum, University of Tübingen
> 
> 
> Michael Wibral wrote:
> > Hi Charles,
> >
> > from your output: ... computing statistic 100 from 100 performing
> > Bonferoni correction for multiple comparisons ...
> >
> > it seems that you're only computing 100 randomizations. It follows
> > that the best p-value you could EVER get is 0.01. You then do
> > bonferroni correction (not the cluster based correction you
> > intended!). So if you set an alpha of 0.9 and divide this by - say -
> > 2000 for your bonferoni corrcetion you alpha is 0.9/2000=0.00045. You
> > see that you'll never reach this limit given that you do only 100
> > randomizations and by defibition cannot get below p=0.01. In addition
> > you won't get clusters when you use Bonferroni.
> >
> > I suggest using:
> >
> > cfg.numrandomization = 5000; cfg.corectm='cluster'; % or 'fdr'
> >
> >
> > Good Luck! Michael
> 
> 
> 
> 
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the
FieldTrip
> toolbox, to share experiences and to discuss new ideas for MEG and EEG
analysis.
> See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.