Update: Freqstatistics Now Yields (Som e) Significant Clusters
Eric Maris
e.maris at DONDERS.RU.NL
Thu Jun 25 21:27:43 CEST 2009
Hi Michael,
> thanks for the update. Cluster based statistics is exactly what the name
says: A
> statistics telling you whether you have spatially and temporally
contiguous effects
> that cross a certain threshold - in sum over the cluster. It is sometimes
worth
> considering, whether this is what you want to test after all. E.g.
extended effects of
> small effect size per electrode but large time/frequency extent and
effects of large
> effect size but small time/frequency extent may have similar cluster
statistics. The
> even compete in the sense that randomizations of the larger of the two (in
total
> cluster sum) may still have larger cluster statistics thatn the smaller of
the two,
> thus effectively rendering in non-significant. Bear in mind that the only
thing really
> tested is the exchangeability of the data (which is the null hypothesis).
That may
> sometimes make your results more difficult to interpret. You clould also
try
> cfg.correctm = 'fdr', to get classical FDR correction, but you may loose
sensitivity in
> some cases.
This is a fair assessment of cluster-based statistics.
>
> One last thing: Check carefully that there is no factor that has been
balanced over
> subjects (e.g. response hand) that may be resorted in the randomizations.
For
> example: half of the subjects report match with the right hand and
non-match with
> the left hand, the other half responding with an inverted assignment. This
analysis
> setup:
> 1. violates the exchangeability hypothesis from the start (and you know!),
but not in
> the way you wanted to test it - this is a serious error in applying
randomization
> testing...
> 2. Consequentaly, it renders all other effects insignificant because the
sorted
> response hand effects in the randomizations most likely exceed any other
effect in
> the unrandomized data.
The theory of permutation tests also applies to statistical testing problems
that involve control variables (e.g., response hand). In this case, the
mechanics of the permutation test involves randomly permuting the data sets
(single trials or subject averages) within each of the levels of the control
variable (conditional permutation). To increase statistical sensitivity in
the presence of control variable, it also is good to use a special test
statistic that partials out the variance explained by the control variable.
This is implemented in Fieldtrip for a couple of statfuns, but not for all
of them. Conditional permutation is implemented via cfg.cvar.
dr. Eric Maris
Donders Institute for Brain, Cognition and Behavior
Center for Cognition and F.C. Donders Center for Cognitive Neuroimaging
Radboud University
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
T:+31 24 3612651
F:+31 24 3616066
E: e.maris at donders.ru.nl
MSc Cognitive Neuroscience: www.ru.nl/master/cns/
>
>
> Michael
>
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: "Charles Cook" <charles.cook at ULETH.CA>
> > Gesendet: 24.06.09 16:23:39
> > An: FIELDTRIP at NIC.SURFNET.NL
> > Betreff: Re: [FIELDTRIP] Update: Freqstatistics Now Yields (Some)
Significant
> Clusters
>
>
> > Hi Michael,
> >
> > I would say the single most important variable was getting the scale
> > correct, something you suggested in your last post. Thus, getting this
> > correct in our code:
> >
> > %Read in the electrode locations for the Std81 montage
> > cfg = [];
> > elec = read_fcdc_elec('EGI-BESA_Standard_81_prime.sfp');
> > elec.pnt = 1000*elec.pnt;
> >
> > This really set us on the right track. Also, having the correct
neighbour
> > distance was quite important. That took a little bit of tweaking to get
right.
> >
> > Having said all of this, the cluster analysis is proceeding on our data.
One
> > point we're not completely satisfied with is the within subjects
analysis
> > using the dependent t-test. In our study, we are interested in how male
> > participants were performing in task1 vs. task2. Behaviourally we do see
a
> > significant RT difference between task1 vs. task2, but when we run the
> > analysis, we are unable to find any significant clusters (<0.05). It may
be
> > possible that this method simply might be too conservative, so if anyone
has
> > any suggestions, I'd be very interested and appreciative in some
feedback.
> >
> > Cheers,
> >
> > Charles Cook
> > ==============================================
> >
> > % perform the statistical test using randomization and a clustering
approach
> > % using the NEW freqstatistics function
> > cfg = [];
> > cfg.neighbourdist = 45;
> > cfg.elec = elec;
> > cfg.statistic = 'depsamplesT';
> > cfg.minnbchan = 0;
> > cfg.clusteralpha = 0.05;
> > cfg.alpha = 0.05;
> > cfg.clustertail = 0;
> > cfg.numrandomization = 10000;
> >
> > cfg.latency = [0 1000];
> > cfg.frequency = [8 10];
> > cfg.avgovertime = 'no';
> > cfg.avgoverfreq = 'yes';
> > cfg.avgoverchan = 'no';
> >
> > cfg.correctm = 'cluster';
> > cfg.method = 'montecarlo';
> > cfg.feedback = 'gui';
> > %cfg.design = [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20
> > 21 22; % subject number
> > %1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2
2
> > 2 2]; % condition number
> >
> > cfg.design = [1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8
9
> > 10 11; % subject number
> > 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2
2 2
> > 2]; % condition number
> >
> >
> > cfg.uvar = 1; % "subject" is unit of
> > observation
> > cfg.ivar = 2; % "condition" is the
> > independent variable
> > %stat = freqstatistics(cfg, maleloc_all, femloc_all);
> > %stat = freqstatistics(cfg, femfeat_all, malefeat_all);
> > stat = freqstatistics(cfg, maleloc_all, malefeat_all);
> > %stat = freqstatistics(cfg, femfeat_all, femloc_all);
> >
> > cfg = [];
> > cfg.elec = elec;
> > cfg.rotate = 0;
> > %cfg.zlim = [-6 6];
> > cfg.alpha = 0.05;
> > cfg.label = stat.label;
> > cfg.electrodes = 'labels';
> > cfg.showxlim = 'yes';
> > cfg.showzlim = 'yes';
> > cfg.showylim = 'yes';
> > clusterplot (cfg, stat);
> > ===========================
> > Obtaining the electrode configuration from the configuration.
> > there are on average 4.7 neighbours per channel
> > using "statistics_montecarlo" for the statistical testing
> > using "statfun_depsamplesT" for the single-sample statistics
> > constructing randomized design
> > total number of measurements = 22
> > total number of variables = 2
> > number of independent variables = 1
> > number of unit variables = 1
> > number of within-cell variables = 0
> > number of control variables = 0
> > using a permutation resampling approach
> > repeated measurement in variable 1 over 11 levels
> > number of repeated measurements in each level is 2 2 2 2 2 2 2 2 2 2 2
> > computing a parmetric threshold for clustering
> > estimated time per randomization is 0 seconds
> > found 5 positive clusters in observed data
> > found 12 negative clusters in observed data
> > using a cluster-based method for multiple comparison correction
> > the returned probabilities and the thresholded mask are corrected for
> > multiple comparisons
> > There are 0 clusters smaller than alpha (0.05)
> >
> > On Wed, 17 Jun 2009 09:24:41 +0200, Michael Wibral
> > <wibral at BIC.UNI-FRANKFURT.DE> wrote:
> >
> > >Hi Charles,
> > >
> > >the plots look OK to me. Could you let us know what finally made your
> > analysis work (if there was a single most important thing)
> > >- that would be most helpful.
> > >
> > >Thanks,
> > >Michael
> > >
> >
> > ----------------------------------
> > The aim of this list is to facilitate the discussion between users of
the FieldTrip
> toolbox, to share experiences and to discuss new ideas for MEG and EEG
analysis.
> See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.
> >
>
>
>
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the
FieldTrip
> toolbox, to share experiences and to discuss new ideas for MEG and EEG
analysis.
> See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.
----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
More information about the fieldtrip
mailing list