Freqstatistics Yields Zero Significant Clusters?

[Eric Maris]

Dear Erick,


The configuration that you posted seems OK. I am a bit puzzled by the
following lines that you posted previously:

>> We are fairly sure that there is something there, because calculating
>> analytically we see big clusters that extend in a consistent way to
>> neighboring frequency and time bins, even at low p thresholds (<10^-4).
>> Since Bonferroni is over-strict, Montecarlo is the option of choice
>> for a corrected result that we will trust. But now something seems off.

I'm trying to find out what might be off. One way to perform a check on the
cluster-based permutation test is trying to find the biggest cluster that
you identified using other code than the one in freqstatistics. This biggest
positive cluster should show up in statout.posclusters(1) and the biggest
negative cluster in statout.negclusters(1) (statout is the output of
freqstatistics). With cfg.clusteralpha, you can control the size of the
clusters (the smaller cfg.clusteralpha, the smaller the clusters).

Another way to get some confidence in freqstatistics is reducing the
dimensionality of data arrays. I understood that you now compare
three-dimensional data arrays (channels X frequency X time). By selecting
two- or one-dimensional slices from this three-dimensional array, the Monte
Carlo p-values should become smaller, at least if you select channels,
frequencies, or time points that show differences between the two
conditions. At the extreme, using cluster-based permutation to test a single
(channel,frequency,time)-triplet, the Monte Carlo p-value should be
approximately equal to the analytical (T-distribution-based) p-value.

Good luck,

Eric




dr. Eric Maris
Donders Institute for Brain, Cognition and Behavior
Center for Cognition and F.C. Donders Center for Cognitive Neuroimaging
Radboud University
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
T:+31 24 3612651
F:+31 24 3616066
E: e.maris at donders.ru.nl

MSc Cognitive Neuroscience: www.ru.nl/master/cns/

> -----Oorspronkelijk bericht-----
> Van: FieldTrip discussion list [mailto:FIELDTRIP at NIC.SURFNET.NL] Namens
Erick
> Britis Ortiz
> Verzonden: maandag 8 juni 2009 15:06
> Aan: FIELDTRIP at NIC.SURFNET.NL
> Onderwerp: Re: [FIELDTRIP] Freqstatistics Yields Zero Significant
Clusters?
>
> Dear Eric,
>
> this is the configuration used for the analysis.
>
> % Configuration for statistics
> cfg = [];
> cfg.channel     = 'MEG';
> cfg.latency     = [ 0.0 0.6 ];
> cfg.avgovertime = 'no';
> cfg.method      = 'montecarlo';
> cfg.numrandomization = 1000;
> cfg.statistic   = 'depsamplesT';
> cfg.alpha       = 0.05;
> cfg.tail        = 0;
> cfg.correctm = 'cluster'; cfg.grad = grad;
> cfg.clusteralpha = 0.05;
> cfg.clusterstatistic = 'maxsum';
> cfg.minnbchan = 2;
> cfg.clustertail = 0;
> cfg.design(1,1:2*numsubj)  = [ones(1,numsubj) 2*ones(1,numsubj)];
> cfg.design(2,1:2*numsubj)  = [1:numsubj 1:numsubj];
> cfg.ivar                = 1; % the 1st row in cfg.design contains the
> independent variable
> cfg.uvar                = 2; % the 2nd row in cfg.design contains the
> subject number
>
> Cordially,
> Erick
>
>
> Eric Maris wrote:
> > Dear Erick,
> >
> >
> >> Following up on the discussion, we have also tried to use the cluster
> >> option for multiple comparisons correction in a previous study and
> >> thought that our results were not significant. Now, we tried again with
> >> 1000 randomizations and cfg.correctm = 'cluster', to no avail. Since we
> >> are interested in p<0.05, 1000 iterations seem large enough, but most
of
> >> the probabilities calculated are near or equal to 100%. Further testing
> >> with 10000 permutations yielded similar results.
> >>
> >> We are fairly sure that there is something there, because calculating
> >> analytically we see big clusters that extend in a consistent way to
> >> neighboring frequency and time bins, even at low p thresholds (<10^-4).
> >> Since Bonferroni is over-strict, Montecarlo is the option of choice for
> >> a corrected result that we will trust. But now something seems off.
> >
> >
> > Can you post the configuration that you used for these analyses?
> >
> >
> > dr. Eric Maris
> > Donders Institute for Brain, Cognition and Behavior
> > Center for Cognition and F.C. Donders Center for Cognitive Neuroimaging
> > Radboud University
> > P.O. Box 9104
> > 6500 HE Nijmegen
> > The Netherlands
> > T:+31 24 3612651
> > F:+31 24 3616066
> > E: e.maris at donders.ru.nl
> >
> > MSc Cognitive Neuroscience: www.ru.nl/master/cns/
>
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the
FieldTrip
> toolbox, to share experiences and to discuss new ideas for MEG and EEG
analysis.
> See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.