Update: Freqstatistics Now Yields (Som e) Significant Clusters

Michael Wibral wibral at BIC.UNI-FRANKFURT.DE
Mon Jun 29 10:44:13 CEST 2009


Hi Eric,

thanks for the update on the statfuns. "cfg.cvar" is indeed very helpful.
Again I see it always pays to update to the latest version :-).

Michael


> -----Ursprüngliche Nachricht-----
> Von: "Eric Maris" <e.maris at DONDERS.RU.NL>
> Gesendet: 25.06.09 21:30:40
> An: FIELDTRIP at NIC.SURFNET.NL
> Betreff: Re: [FIELDTRIP] Update: Freqstatistics Now Yields (Som e) Significant Clusters


> Hi Michael,
>  
> > thanks for the update. Cluster based statistics is exactly what the name
> says: A
> > statistics telling you whether you have spatially and temporally
> contiguous effects
> > that cross a certain threshold - in sum over the cluster. It is sometimes
> worth
> > considering, whether this is what you want to test after all. E.g.
> extended effects of
> > small effect size per electrode but large time/frequency extent and
> effects of large
> > effect size but small time/frequency extent may have similar cluster
> statistics. The
> > even compete in the sense that randomizations of the larger of the two (in
> total
> > cluster sum) may still have larger cluster statistics thatn the smaller of
> the two,
> > thus effectively rendering in non-significant. Bear in mind that the only
> thing really
> > tested is the exchangeability of the data (which is the null hypothesis).
> That may
> > sometimes make your results more difficult to interpret. You clould also
> try
> > cfg.correctm = 'fdr', to get classical FDR correction, but you may loose
> sensitivity in
> > some cases.
> 
> This is a fair assessment of cluster-based statistics. 
> 
> > 
> > One last thing: Check carefully that there is no factor that has been
> balanced over
> > subjects (e.g. response hand) that may be resorted in the randomizations.
> For
> > example: half of the subjects report match with the right hand and
> non-match with
> > the left hand, the other half responding with an inverted assignment. This
> analysis
> > setup:
> > 1. violates the exchangeability hypothesis from the start (and you know!),
> but not in
> > the way you wanted to test it - this is a serious error in applying
> randomization
> > testing...
> > 2. Consequentaly, it renders all other effects insignificant because the
> sorted
> > response hand effects in the randomizations most likely exceed any other
> effect in
> > the unrandomized data.
> 
> The theory of permutation tests also applies to statistical testing problems
> that involve control variables (e.g., response hand). In this case, the
> mechanics of the permutation test involves randomly permuting the data sets
> (single trials or subject averages) within each of the levels of the control
> variable (conditional permutation). To increase statistical sensitivity in
> the presence of control variable, it also is good to use a special test
> statistic that partials out the variance explained by the control variable.
> This is implemented in Fieldtrip for a couple of statfuns, but not for all
> of them. Conditional permutation is implemented via cfg.cvar.
> 
> 
> dr. Eric Maris
> Donders Institute for Brain, Cognition and Behavior
> 
> Center for Cognition and F.C. Donders Center for Cognitive Neuroimaging
> 
> Radboud University
> P.O. Box 9104
> 6500 HE Nijmegen
> The Netherlands
> T:+31 24 3612651
> F:+31 24 3616066
> E: e.maris at donders.ru.nl
> 
>  
> 
> MSc Cognitive Neuroscience: www.ru.nl/master/cns/
> 
> 
> 
> > 
> > 
> > Michael
> > 
> > 
> > 
> > > -----Ursprüngliche Nachricht-----
> > > Von: "Charles Cook" <charles.cook at ULETH.CA>
> > > Gesendet: 24.06.09 16:23:39
> > > An: FIELDTRIP at NIC.SURFNET.NL
> > > Betreff: Re: [FIELDTRIP] Update: Freqstatistics Now Yields (Some)
> Significant
> > Clusters
> > 
> > 
> > > Hi Michael,
> > >
> > > I would say the single most important variable was getting the scale
> > > correct, something you suggested in your last post. Thus, getting this
> > > correct in our code:
> > >
> > > %Read in the electrode locations for the Std81 montage
> > > cfg = [];
> > > elec = read_fcdc_elec('EGI-BESA_Standard_81_prime.sfp');
> > > elec.pnt = 1000*elec.pnt;
> > >
> > > This really set us on the right track. Also, having the correct
> neighbour
> > > distance was quite important. That took a little bit of tweaking to get
> right.
> > >
> > > Having said all of this, the cluster analysis is proceeding on our data.
> One
> > > point we're not completely satisfied with is the within subjects
> analysis
> > > using the dependent t-test. In our study, we are interested in how male
> > > participants were performing in task1 vs. task2. Behaviourally we do see
> a
> > > significant RT difference between task1 vs. task2, but when we run the
> > > analysis, we are unable to find any significant clusters (<0.05). It may
> be
> > > possible that this method simply might be too conservative, so if anyone
> has
> > > any suggestions, I'd be very interested and appreciative in some
> feedback.
> > >
> > > Cheers,
> > >
> > > Charles Cook
> > > ==============================================
> > >
> > > % perform the statistical test using randomization and a clustering
> approach
> > > % using the NEW freqstatistics function
> > > cfg = [];
> > > cfg.neighbourdist    = 45;
> > > cfg.elec             = elec;
> > > cfg.statistic        = 'depsamplesT';
> > > cfg.minnbchan        = 0;
> > > cfg.clusteralpha     = 0.05;
> > > cfg.alpha            = 0.05;
> > > cfg.clustertail      = 0;
> > > cfg.numrandomization = 10000;
> > >
> > > cfg.latency          = [0 1000];
> > > cfg.frequency        = [8 10];
> > > cfg.avgovertime      = 'no';
> > > cfg.avgoverfreq      = 'yes';
> > > cfg.avgoverchan      = 'no';
> > >
> > > cfg.correctm         = 'cluster';
> > > cfg.method           = 'montecarlo';
> > > cfg.feedback         = 'gui';
> > > %cfg.design           = [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
> 20
> > > 21 22;   % subject number
> > >                         %1 1 1 1 1 1 1 1 1 1  1  2  2  2  2  2  2  2  2
> 2
> > > 2  2]; % condition number
> > >
> > > cfg.design           = [1 2 3 4 5 6 7 8 9 10 11 1  2  3  4  5  6  7  8
> 9
> > > 10 11;   % subject number
> > >                         1 1 1 1 1 1 1 1 1 1  1  2  2  2  2  2  2  2  2
> 2  2
> > >  2]; % condition number
> > >
> > >
> > > cfg.uvar = 1;                                   % "subject" is unit of
> > > observation
> > > cfg.ivar = 2;                                   % "condition" is the
> > > independent variable
> > > %stat = freqstatistics(cfg, maleloc_all, femloc_all);
> > > %stat = freqstatistics(cfg, femfeat_all, malefeat_all);
> > > stat = freqstatistics(cfg, maleloc_all, malefeat_all);
> > > %stat = freqstatistics(cfg, femfeat_all, femloc_all);
> > >
> > > cfg = [];
> > > cfg.elec  = elec;
> > > cfg.rotate = 0;
> > > %cfg.zlim = [-6 6];
> > > cfg.alpha = 0.05;
> > > cfg.label = stat.label;
> > > cfg.electrodes = 'labels';
> > > cfg.showxlim = 'yes';
> > > cfg.showzlim = 'yes';
> > > cfg.showylim = 'yes';
> > > clusterplot (cfg, stat);
> > > ===========================
> > > Obtaining the electrode configuration from the configuration.
> > > there are on average 4.7 neighbours per channel
> > > using "statistics_montecarlo" for the statistical testing
> > > using "statfun_depsamplesT" for the single-sample statistics
> > > constructing randomized design
> > > total number of measurements     = 22
> > > total number of variables        = 2
> > > number of independent variables  = 1
> > > number of unit variables         = 1
> > > number of within-cell variables = 0
> > > number of control variables      = 0
> > > using a permutation resampling approach
> > > repeated measurement in variable 1 over 11 levels
> > > number of repeated measurements in each level is 2 2 2 2 2 2 2 2 2 2 2
> > > computing a parmetric threshold for clustering
> > > estimated time per randomization is 0 seconds
> > > found 5 positive clusters in observed data
> > > found 12 negative clusters in observed data
> > > using a cluster-based method for multiple comparison correction
> > > the returned probabilities and the thresholded mask are corrected for
> > > multiple comparisons
> > > There are 0 clusters smaller than alpha (0.05)
> > >
> > > On Wed, 17 Jun 2009 09:24:41 +0200, Michael Wibral
> > > <wibral at BIC.UNI-FRANKFURT.DE> wrote:
> > >
> > > >Hi Charles,
> > > >
> > > >the plots look OK to me. Could you let us know what finally made your
> > > analysis work (if there was a single most important thing)
> > > >- that would be most helpful.
> > > >
> > > >Thanks,
> > > >Michael
> > > >
> > >
> > > ----------------------------------
> > > The aim of this list is to facilitate the discussion between users of
> the FieldTrip
> > toolbox, to share experiences and to discuss new ideas for MEG and EEG
> analysis.
> > See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> > http://www.ru.nl/neuroimaging/fieldtrip.
> > >
> > 
> > 
> > 
> > ----------------------------------
> > The aim of this list is to facilitate the discussion between users of the
> FieldTrip
> > toolbox, to share experiences and to discuss new ideas for MEG and EEG
> analysis.
> > See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> > http://www.ru.nl/neuroimaging/fieldtrip.
> 
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
> 



----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Michael Wibral.vcf
Type: text/x-vcard
Size: 344 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20090629/543a6013/attachment-0002.vcf>


More information about the fieldtrip mailing list