[FieldTrip] permutation test: multiple comparison correction for the p-values

[Steinmann, Iris]

Hey Julian,

thanks, you are right! I thought instead of correcting the threshold on which I consider significance, it would be possible to fix the threshold at 0.05 and adapt the p-values. Maybe that's really not possible... so, thanks for your comprehensive explanation. I will go with your first advice and multiply the p-values with the mask.

Thanks a lot and have a nice weekend!

Iris

From: fieldtrip-bounces at science.ru.nl [mailto:fieldtrip-bounces at science.ru.nl] On Behalf Of Julian Keil
Sent: Freitag, 13. November 2015 17:00
To: FieldTrip discussion list
Subject: Re: [FieldTrip] permutation test: multiple comparison correction for the p-values

Hi Iris,

maybe I completely miss your point, could you describe what you want to do with your p-values?
Please keep in mind, that a correction for multiple comparisons does not change your p-values, it changes the threshold after which a p-value is significant.
Think about the very simple Bonferroni-correction: If you have 5 tests, you set your significance-level to 0.05/5 = 0.01, then compute your 5 t-tests and see which p-value is below 0.01 (instead of below 0.05). This does not affect the t-tests, or the p-value or t-value of the tests themselves, but only the level you consider significant.

tl;dr: You don't correct your p-values, you correct the level after which you consider them significant.

Hope this helps

Julian

Am 13.11.2015 um 16:41 schrieb Steinmann, Iris:


Hi Julian,

Thanks a lot for your advice! But I wane keep all the p-values. Maybe to check what will happen when I would go with a more liberal threshold (maybe p = 0.06 instead of p = 0.05). So, is there any possibility to correct my p-values directly instead of only the mask?

Best wishes!
Iris



From: fieldtrip-bounces at science.ru.nl<mailto:fieldtrip-bounces at science.ru.nl> [mailto:fieldtrip-bounces at science.ru.nl] On Behalf Of Julian Keil
Sent: Freitag, 13. November 2015 09:51
To: FieldTrip discussion list
Subject: Re: [FieldTrip] permutation test: multiple comparison correction for the p-values

Hi Iris,

I'm not sure what you want to do with the p-values in the following, but you can multiply your (FDR-corrected) mask with the prob.
This would look like this: stat_TF.prob_new = stat_TF.prob .* stat_TF.mask;

Now you only have the significant p-values left in your prob_new-field, everything else is set to 0.

Hope this helps

Julian

********************
Dr. Julian Keil

AG Multisensorische Integration
Psychiatrische Universitätsklinik
der Charité im St. Hedwig-Krankenhaus
Große Hamburger Straße 5-11, Raum A007
10115 Berlin

Telefon: +49-30-2311-1879
Fax:        +49-30-2311-2209
http://psy-ccm.charite.de/forschung/bildgebung/ag_multisensorische_integration

Am 13.11.2015 um 09:41 schrieb Steinmann, Iris:



Hi everybody,

I' m using the 'ft_freqstatistics' function to find significant differences between two time-frequency spectra
(see the used options down below).
While fieldtrip calculates the permutation test it throws the following information:

"performing FDR correction for multiple comparisons
the returned probabilities are uncorrected, the thresholded mask is corrected"

Since I'm using the p-values for the following analysis and not the thresholded mask, I was wondering how I
get my p-values corrected for multiple comparison?

Does anyone has an idea? Thanks in advance!


I used the following options:

cfg = [];
cfg.channel     = 'all';
cfg.latency     = [2.3 2.8];
cfg.avgoverchan = 'yes';
cfg.avgovertime = 'no';
cfg.frequency = [9 14];
cfg.parameter   = 'powspctrm';
cfg.alpha = 0.05;
cfg.tail = 0;
cfg.correctm = 'fdr';
cfg.correcttail = 'prob';
cfg.ivar = 1;
cfg.statistic = 'ft_statfun_indepsamplesT';
cfg.method = 'montecarlo';
cfg.design = design; % defined in at the beginning of the function
cfg.numrandomization = 1000;

stat_TF = ft_freqstatistics(cfg, TF_remove, TF_noremove)
_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151113/2e3e6dfa/attachment-0002.html>