Update: Freqstatistics Now Yields (Som e) Significant Clusters
Michael Wibral
wibral at BIC.UNI-FRANKFURT.DE
Thu Jun 25 12:00:31 CEST 2009
Hi Charles,
thanks for the update. Cluster based statistics is exactly what the name says: A statistics telling you whether you have spatially and temporally contiguous effects that cross a certain threshold - in sum over the cluster. It is sometimes worth considering, whether this is what you want to test after all. E.g. extended effects of small effect size per electrode but large time/frequency extent and effects of large effect size but small time/frequency extent may have similar cluster statistics. The even compete in the sense that randomizations of the larger of the two (in total cluster sum) may still have larger cluster statistics thatn the smaller of the two, thus effectively rendering in non-significant. Bear in mind that the only thing really tested is the exchangeability of the data (which is the null hypothesis). That may sometimes make your results more difficult to interpret. You clould also try cfg.correctm = 'fdr', to get classical FDR correction, but you may loose sensitivity in some cases.
One last thing: Check carefully that there is no factor that has been balanced over subjects (e.g. response hand) that may be resorted in the randomizations. For example: half of the subjects report match with the right hand and non-match with the left hand, the other half responding with an inverted assignment. This analysis setup:
1. violates the exchangeability hypothesis from the start (and you know!), but not in the way you wanted to test it - this is a serious error in applying randomization testing...
2. Consequentaly, it renders all other effects insignificant because the sorted response hand effects in the randomizations most likely exceed any other effect in the unrandomized data.
Michael
> -----Ursprüngliche Nachricht-----
> Von: "Charles Cook" <charles.cook at ULETH.CA>
> Gesendet: 24.06.09 16:23:39
> An: FIELDTRIP at NIC.SURFNET.NL
> Betreff: Re: [FIELDTRIP] Update: Freqstatistics Now Yields (Some) Significant Clusters
> Hi Michael,
>
> I would say the single most important variable was getting the scale
> correct, something you suggested in your last post. Thus, getting this
> correct in our code:
>
> %Read in the electrode locations for the Std81 montage
> cfg = [];
> elec = read_fcdc_elec('EGI-BESA_Standard_81_prime.sfp');
> elec.pnt = 1000*elec.pnt;
>
> This really set us on the right track. Also, having the correct neighbour
> distance was quite important. That took a little bit of tweaking to get right.
>
> Having said all of this, the cluster analysis is proceeding on our data. One
> point we're not completely satisfied with is the within subjects analysis
> using the dependent t-test. In our study, we are interested in how male
> participants were performing in task1 vs. task2. Behaviourally we do see a
> significant RT difference between task1 vs. task2, but when we run the
> analysis, we are unable to find any significant clusters (<0.05). It may be
> possible that this method simply might be too conservative, so if anyone has
> any suggestions, I'd be very interested and appreciative in some feedback.
>
> Cheers,
>
> Charles Cook
> ==============================================
>
> % perform the statistical test using randomization and a clustering approach
> % using the NEW freqstatistics function
> cfg = [];
> cfg.neighbourdist = 45;
> cfg.elec = elec;
> cfg.statistic = 'depsamplesT';
> cfg.minnbchan = 0;
> cfg.clusteralpha = 0.05;
> cfg.alpha = 0.05;
> cfg.clustertail = 0;
> cfg.numrandomization = 10000;
>
> cfg.latency = [0 1000];
> cfg.frequency = [8 10];
> cfg.avgovertime = 'no';
> cfg.avgoverfreq = 'yes';
> cfg.avgoverchan = 'no';
>
> cfg.correctm = 'cluster';
> cfg.method = 'montecarlo';
> cfg.feedback = 'gui';
> %cfg.design = [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
> 21 22; % subject number
> %1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2
> 2 2]; % condition number
>
> cfg.design = [1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9
> 10 11; % subject number
> 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2
> 2]; % condition number
>
>
> cfg.uvar = 1; % "subject" is unit of
> observation
> cfg.ivar = 2; % "condition" is the
> independent variable
> %stat = freqstatistics(cfg, maleloc_all, femloc_all);
> %stat = freqstatistics(cfg, femfeat_all, malefeat_all);
> stat = freqstatistics(cfg, maleloc_all, malefeat_all);
> %stat = freqstatistics(cfg, femfeat_all, femloc_all);
>
> cfg = [];
> cfg.elec = elec;
> cfg.rotate = 0;
> %cfg.zlim = [-6 6];
> cfg.alpha = 0.05;
> cfg.label = stat.label;
> cfg.electrodes = 'labels';
> cfg.showxlim = 'yes';
> cfg.showzlim = 'yes';
> cfg.showylim = 'yes';
> clusterplot (cfg, stat);
> ===========================
> Obtaining the electrode configuration from the configuration.
> there are on average 4.7 neighbours per channel
> using "statistics_montecarlo" for the statistical testing
> using "statfun_depsamplesT" for the single-sample statistics
> constructing randomized design
> total number of measurements = 22
> total number of variables = 2
> number of independent variables = 1
> number of unit variables = 1
> number of within-cell variables = 0
> number of control variables = 0
> using a permutation resampling approach
> repeated measurement in variable 1 over 11 levels
> number of repeated measurements in each level is 2 2 2 2 2 2 2 2 2 2 2
> computing a parmetric threshold for clustering
> estimated time per randomization is 0 seconds
> found 5 positive clusters in observed data
> found 12 negative clusters in observed data
> using a cluster-based method for multiple comparison correction
> the returned probabilities and the thresholded mask are corrected for
> multiple comparisons
> There are 0 clusters smaller than alpha (0.05)
>
> On Wed, 17 Jun 2009 09:24:41 +0200, Michael Wibral
> <wibral at BIC.UNI-FRANKFURT.DE> wrote:
>
> >Hi Charles,
> >
> >the plots look OK to me. Could you let us know what finally made your
> analysis work (if there was a single most important thing)
> >- that would be most helpful.
> >
> >Thanks,
> >Michael
> >
>
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
>
----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Michael Wibral.vcf
Type: text/x-vcard
Size: 344 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20090625/c35b6649/attachment-0001.vcf>
More information about the fieldtrip
mailing list