[FieldTrip] permutation test: multiple comparison correction for the p-values

[Julian Keil]

Hi Iris,

maybe I completely miss your point, could you describe what you want to do with your p-values?
Please keep in mind, that a correction for multiple comparisons does not change your p-values, it changes the threshold after which a p-value is significant.
Think about the very simple Bonferroni-correction: If you have 5 tests, you set your significance-level to 0.05/5 = 0.01, then compute your 5 t-tests and see which p-value is below 0.01 (instead of below 0.05). This does not affect the t-tests, or the p-value or t-value of the tests themselves, but only the level you consider significant.

tl;dr: You don't correct your p-values, you correct the level after which you consider them significant.

Hope this helps

Julian

Am 13.11.2015 um 16:41 schrieb Steinmann, Iris:

> Hi Julian,
> 
> Thanks a lot for your advice! But I wane keep all the p-values. Maybe to check what will happen when I would go with a more liberal threshold (maybe p = 0.06 instead of p = 0.05). So, is there any possibility to correct my p-values directly instead of only the mask?
> 
> Best wishes!
> Iris
> 
> 
> 
> From: fieldtrip-bounces at science.ru.nl [mailto:fieldtrip-bounces at science.ru.nl] On Behalf Of Julian Keil
> Sent: Freitag, 13. November 2015 09:51
> To: FieldTrip discussion list
> Subject: Re: [FieldTrip] permutation test: multiple comparison correction for the p-values
> 
> Hi Iris,
> 
> I'm not sure what you want to do with the p-values in the following, but you can multiply your (FDR-corrected) mask with the prob.
> This would look like this: stat_TF.prob_new = stat_TF.prob .* stat_TF.mask;
> 
> Now you only have the significant p-values left in your prob_new-field, everything else is set to 0.
> 
> Hope this helps
> 
> Julian
> 
> ********************
> Dr. Julian Keil
> 
> AG Multisensorische Integration
> Psychiatrische Universitätsklinik
> der Charité im St. Hedwig-Krankenhaus
> Große Hamburger Straße 5-11, Raum A007
> 10115 Berlin
> 
> Telefon: +49-30-2311-1879
> Fax:        +49-30-2311-2209
> http://psy-ccm.charite.de/forschung/bildgebung/ag_multisensorische_integration
> 
> Am 13.11.2015 um 09:41 schrieb Steinmann, Iris:
> 
> 
> Hi everybody,
> 
> I' m using the 'ft_freqstatistics' function to find significant differences between two time-frequency spectra
> (see the used options down below).
> While fieldtrip calculates the permutation test it throws the following information:
> 
> "performing FDR correction for multiple comparisons
> the returned probabilities are uncorrected, the thresholded mask is corrected"
> 
> Since I'm using the p-values for the following analysis and not the thresholded mask, I was wondering how I
> get my p-values corrected for multiple comparison?
> 
> Does anyone has an idea? Thanks in advance!
> 
> 
> I used the following options:
> 
> cfg = [];
> cfg.channel     = 'all';
> cfg.latency     = [2.3 2.8];
> cfg.avgoverchan = 'yes';
> cfg.avgovertime = 'no';
> cfg.frequency = [9 14];
> cfg.parameter   = 'powspctrm';
> cfg.alpha = 0.05;
> cfg.tail = 0;
> cfg.correctm = 'fdr';
> cfg.correcttail = 'prob';
> cfg.ivar = 1;
> cfg.statistic = 'ft_statfun_indepsamplesT';
> cfg.method = 'montecarlo';
> cfg.design = design; % defined in at the beginning of the function
> cfg.numrandomization = 1000;
> 
> stat_TF = ft_freqstatistics(cfg, TF_remove, TF_noremove)
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> 
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151113/85f76765/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151113/85f76765/attachment-0002.sig>