Thanks! / Question regarding test statistics
Roemer van der Meij
r.vandermeij at DONDERS.RU.NL
Thu Jan 28 11:07:15 CET 2010
Hi Andrea,
Sorry for my late reply (compared to my previous replies), I was a bit
too busy. I can't seem to find any reference on cfg.testtail, are you
referring to cfg.clustertail instead?
We are thinking at the moment on changing how the correcting of
p-values/alpha-values work, but for the moment you are correct in
assuming that you need to divide the cfg.alpha manually by 2 in case of
a two-sided test. This will most likely change in the near-future and be
made more explicit in the documentation, but for now, the tutorials are
correct.
It is very important to make the distinction between cfg.clusteralpha
and cfg.alpha. Cfg.clusteralpha only refers to when certain values where
you want to cluster over are able to be selected for clustering. When
clustering over t-values, cfg.clusteralpha only sets the threshold on
/when /to cluster. A cfg.clusteralpha = 0.05 would allow more t-values
to be selected for clustering, most likely producing bigger clusters,
then a cfg.clusteralpha = 0.01. This /does not/ affect the false alarm
rate of the statistical test at the cluster-level, this is the cfg.alpha
you mentioned. So, to control your false-alarm rate when doing a
two-sided cluster-level test, cfg.alpha is the only one that is
important. The selection of values to cluster over however, does change
when you do a two-sided test.
You are correct in stating that, when using the defaults, the overall
alpha-level that you are testing against is 0.10, whereas the
alpha-level per tail is 0.05. I guess it's a matter of preference which
way you look at it. But it is something that can be made more explicit
in the documentation/tutorials, thanks for bringing this up.
Thanks for noticing the old alphathresh field, the tutorial is updated now.
I hope this helps!
Best,
Roemer
On 1/21/2010 1:33 PM, Andrea Ostendorf wrote:
>
> Dear Roemer,
>
> Thanks a lot. My previous version worked with your proposed
> corrections and I downloaded the new version today.
>
> Sorry, I have yet another question, this time regarding the parameters
> controlling statistical analysis for the cluster-based test statistic.
>
> When asking myself whether it made sense to set cfg.testtail and
> cfg.tail independently, I found the error message in clusterstat.m,
> l.32, which states that the parameters should be equal. Could you
> perhaps mention this requirement in the tutorials?
>
>
> *Also, regarding cfg.alpha and cfg.clusteralpha -- just to make sure
> that I have got things right: For a two-sided test, I have to split
> cfg.alpha manually so as to equal half the desired significance
> level but I do not have to change cfg.clusteralpha explicitly, is
> this correct? I gathered so from the tutorials and it looks so from
> the code (in clusterstat and the statfun functions, the alpha
> parameter is halved automatically). If nothing is specified, both
> parameters are set to 0.05. As a two-sided test is the default, does
> this mean that the false alarm rate is set at the 10% level if the
> user does not specify anything, or have I overlooked something? (I
> have to admit that I am impressed with the intricacies but that I
> have not really grasped the details...)*
>
>
> *By the way, the tutorial on c*luster-based permutation tests on
> event related fields mentions a parameter cfg.alphathresh, which I
> only found in the clusterrand functions and which I took to mean
> cfg.clusteralpha.
>
> All the best to all of you
>
> Andrea
>
> ------------------------------------------------------------------------
>
> *From:* FieldTrip discussion list [mailto:FIELDTRIP at NIC.SURFNET.NL]
> *On Behalf Of *Roemer van der Meij
> *Sent:* Mittwoch, 20. Januar 2010 14:08
> *To:* FIELDTRIP at NIC.SURFNET.NL
> *Subject:* Re: [FIELDTRIP] Warning in ft_topoplotER
>
> Hi Andrea,
>
> Thanks for the data/script, it was exactly what I needed. There was a
> silly bug in ft_clusterplot where several calls to cfg.highlight were
> not changed to cfg.highlightchannel during the big update, and I
> failed to notice them during browsing. Things should be fixed now
> (will be on the ftp-server tonight). You can fix it in your own
> version easily by changing in line 267 and line 285 cfg.highlight >>>
> cfg.highlightchannel. Clusterplot should work as intended afterwards.
>
> I also realised now that any additional options that go into
> clusterplot will not be used (like cfg.marker), these are fixed now as
> well (on ftp-server tonight).
>
> Thanks for pointing out these bugs!
>
> Best,
> Roemer
>
>
>
> Andrea Ostendorf wrote:
>
> Dear Roemer,
>
> thanks a lot for your help and the time you are taking to assist me!
> Your remarks have been really helpful.
>
> The warning is from topoplotER, which I call indirectly through
> ft_clusterplot. Thanks for pointing out that I am not to use the
> options directly. I saw some options in the help of ft_clusterplot
> together with the hint that more options were to be found in topoplotER.
>
> Since I keep having trouble with uploading files to your server, I
> attach a test MATLAB file (stripped down to the basics) and my data. I
> hope that is okay, it is really quite a small amount of data.
> Otherwise, I would try my private Ubuntu computer.
>
> The warning is
>
> Warning: cfg.highlight is now used for specifing highlighting-mode,
> use cfg.highlightchannel instead of cfg.highlight for specifiying channels
>
> > In ft_topoplotER at 214
>
> In topoplotER at 17
>
> In ft_clusterplot at 277
>
> In test_clusterplot at 5
>
> Thanks a lot!
>
> Best regards
>
> Andrea
>
> ------------------------------------------------------------------------
>
> *From:* FieldTrip discussion list [mailto:FIELDTRIP at NIC.SURFNET.NL]
> *On Behalf Of *Roemer van der Meij
> *Sent:* Mittwoch, 20. Januar 2010 12:52
> *To:* FIELDTRIP at NIC.SURFNET.NL <mailto:FIELDTRIP at NIC.SURFNET.NL>
> *Subject:* Re: [FIELDTRIP] Thanks / warning in ft_topoplotER
>
> Hi Andrea,
>
> Always happy to help. And thank you for helping us squashing bugs!
>
> Are you sure you are using the most recent fieldtrip version? There
> was a bug several weeks ago that gave the warning (which used to be on
> line 214, but is now on line 215) even when cfg.highlight was not
> numerical. (if it's numerical, it is assumed cfg.highlight is used in
> the old way). If you have a version that is newer than 19-12-2009
> (date of previous bug fix), could you post the exact code you use to
> call ft_topoplotER? This could give me a better idea on where in the
> code the problem lies. Could you then also post the exact warning
> matlab spits out? (if it's not a version problem, something
> complicated is going on and all information is helpful). Thanks!
>
> Concerning ft_clusterplot, you're not actually supposed to use the
> highlighting options directly. As clusterplot tries to determine the
> clustering automatically, this could possibly interfere with it's
> normal functioning. Thanks for pointing this out. I now added a check
> for these 'forbidden' options and they will be removed automatically.
> The updated clusterplot will be uploaded to the ftp-server somewhere
> late in the evening today.
>
> However, if you want to manage you're own clustering directly, you can
> also do this using topoplotER/TFR. It might be a bit difficult, but
> you have to put all the highlighting options in its own cell
> (including the ones that were already in cells). There is 'hidden'
> documentation about this in topoplotER (you won't see it with the
> help/doc command, but you will if you edit the function), you can find
> it just below the normal documentation.
> I copy-pasted it here for easy reference:
> ***********
> % It is possible to use multiple highlight-selections (e.g.: multiple
> statistical clusters of channels)
> % To do this, all the content of the highlight-options (including
> cfg.highlight) should be placed in a cell-array
> % (even if the normal content was already in a cell-array). Specific
> marker settings (e.g. color, size) are defaulted when
> % not present.
> % Example (3 selections):
> % cfg.highlight = {'labels', 'labels', 'numbers'}
> % cfg.highlightchannel = {{'MZF03','MZC01','MRT54'}, [1:5], 'C*'}
> % cfg.highlightsymbol = {'o',[],'+'} % the empty option
> will be defaulted
> % cfg.highlightcolor = {'r',[0 0 1]}; % the missing option
> will be defaulted
> % cfg.highlightsize = []; % will be set
> to default, as will the missing cfg.highlightfontsize
> ***********
>
> Using these options you should be able to get the same individual
> plots clusterplot can give you. If you do not want to see the other
> channels, set cfg.marker = 'off';. On the use of the different
> routines, topoplotTFR is identical to topoplotER, but is just present
> because of our present naming-scheme. Clusterplot however, is a
> wrapper around topoplotER/TFR designed to quickly and without much
> configuring plot your statistical clusters (which requires statistics
> output as input). By using the 'hidden' options above, you can make
> topoplotER/TFR do exactly the same as clusterplot, but then you can
> specify your own clusters.
>
>
> We are currently looking into your question number 3, as it appears
> things go a little deeper there. There are at least some situations
> where cfg.neighbours = [] apparently doesn't work, but we are still
> figuring out why. For the moment, a really ugly and time consuming
> workaround would be to make a for-loop over single channels, but it
> would prevent the clustering over channels and should provide (per
> channel) the exact same output as cfg.neighbours = [];
>
>
> I hope all of the above helps. Thanks for bringing these bugs to our
> attention!
>
> Kind regards,
>
> Roemer
>
>
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4788 (20100120) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
> ----------------------------------
>
> The aim of this list is to facilitate the discussion between users of
> the FieldTrip toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis.
>
> http://listserv.surfnet.nl/archives/fieldtrip.html
>
> http://www.ru.nl/fcdonders/fieldtrip/
>
> ----------------------------------
>
> The aim of this list is to facilitate the discussion between users of
> the FieldTrip toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis.
>
> http://listserv.surfnet.nl/archives/fieldtrip.html
>
> http://www.ru.nl/fcdonders/fieldtrip/
>
>
>
> --
> Roemer van der Meij MSc
> Scientific Programmer& Data-Analyst
> Donders Institute for Brain, Cognition and Behaviour
> Centre for Cognition
> P.O. Box 9104
> 6500 HE Nijmegen
> The Netherlands
> Tel: +31(0)24 3612631
> E-mail:r.vandermeij at donders.ru.nl <mailto:r.vandermeij at donders.ru.nl>
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4788 (20100120) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4791 (20100120) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
> ----------------------------------
>
> The aim of this list is to facilitate the discussion between users of
> the FieldTrip toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis.
>
> http://listserv.surfnet.nl/archives/fieldtrip.html
>
> http://www.ru.nl/fcdonders/fieldtrip/
>
> ----------------------------------
>
> The aim of this list is to facilitate the discussion between users of
> the FieldTrip toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis.
>
> http://listserv.surfnet.nl/archives/fieldtrip.html
>
> http://www.ru.nl/fcdonders/fieldtrip/
>
--
Roemer van der Meij MSc
Scientific Programmer& Data-Analyst
Donders Institute for Brain, Cognition and Behaviour
Centre for Cognition
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
Tel: +31(0)24 3612631
E-mail: r.vandermeij at donders.ru.nl
----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20100128/823cbd09/attachment-0002.html>
More information about the fieldtrip
mailing list