[FieldTrip] One-sided versus two-sided cluster statistics

[Eric Maris]

Hi Nina,

 

 

I have a question concerning cluster based permutation statistics.

It is more a basic question on the difference between one- and two-sided
testings. I recall from simple t-tests that as an approach to your two-sided
p-value you can multiply your one-sided p-value by 2. I hope this is
correct? And I am assuming this to also hold in the other direction, thus,
approximating your one-sided p-value by dividing your two-sided p-value by
2?

 

I propose that you calculate your p-values always one-sided. In fact, this
is what the FT permutation statistics functions also do. The difference
between a one- and a two-sided test is that you compare this one-sided
p-value either with your desired type-I error level (for a one-sided test)
or with half your desired type-I error level (for a two-sided test). 

 

 

That is why I am expecting something similar when calculating my statistics
with a cluster based permutation approach.

So what I am actually doing is comparing two conditions at one sensor,
comparing time-frequency data. 

 

When I calculate a two-sided dependent samples t-test 

(cfg = [];

cfg.channel = 'all';                              

cfg.latency = [0.5 1.5];                               

cfg.avgoverchan = 'no';

cfg.avgovertime = 'no';

cfg.parameter = 'powspctrm';                      

cfg.method = 'montecarlo';                          

cfg.statistic = 'depsamplesT';

cfg.correctm = 'cluster';

cfg.clusteralpha = 0.05;

cfg.clusterstatistic = 'maxsum';                    

cfg.tail = 0;                                       

cfg.clustertail = 0;                                

cfg.alpha = 0.025;

cfg.numrandomization = 1000;  

)

, I find one positive cluster with a p-value of 0.036, thus not significant
(see fig 1).

 

When I then change my settings to a one-sided test

(cfg.tail = 1;                                       

cfg.clustertail = 1;                                

cfg.alpha = 0.05;)

, the positive cluster gets a p-value of 0.056, again not significant (see
fig 2). 

 

I think that the difference between 0.036 and 0.056 is due to the fact that
these are random quantities. If expect that, if you would increase
cfg.numrandomization to 100,000, you would find two p-values that are much
closer. In any case, the p-values are one-sided, and their calculation is
independent of the value that you choose for cfg.alpha.

 

 

Best,

 

Eric Maris

 

 

 

 

 

 

 

 

With the assumption described above, however, I would expect this to become
significant when using a one-sided test. Is my assumption correct? If not,
could anybody comment on what is wrong about my assumption?

 

I would be very grateful for any advice!

 

Kind regards and thank you very much in advance! 

Nina

 

 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - -

 

Nina Kahlbrock

Institute of Clinical Neuroscience and Medical Psychology 

Heinrich Heine University Duesseldorf

Universitaetsstr.  1

40225  Düsseldorf

 

Tel.:      +49 211 81 18075

Fax. .:   +49 211 81 19916

 

Mail:      <mailto:Nina.Kahlbrock at med.uni-duesseldorf.de>
Nina.Kahlbrock at med.uni-duesseldorf.de

 <http://www.uniklinik-duesseldorf.de/medpsychologie>
http://www.uniklinik-duesseldorf.de/medpsychologie

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20110221/c1207a70/attachment-0001.html>