[FieldTrip] bug in FDR script

[David Groppe]

Kathrin is correct about the bug in FieldTrip's FDR function.
You can verify it yourself with Matlab's built in mafdr.m or this File
Exchange Function
https://www.mathworks.com/matlabcentral/fileexchange/27418-benjamini---hochberg-yekutieli-false-discovery-rate--fdr--control-procedure--fdr-bh-m-

For example, if you run:
raw_p=[.01 .01 .01 .6 .6 .6]; %uncorrected p-values

% Matlab's built-in function
sig_p=mafdr(p,'BHFDR',.05)<.05; % p-values signficant after Benjamini &
Hochberg FDR adjustment
fprintf('mafdr significant p-values:\n');
disp(sig_p)

% FieldTrip's function
sig_pFT=fdr(p,.05);
fprintf('FieldTrip FDR significant p-values:\n');
disp(sig_pFT)

The first three p-values should be significant but FieldTrip's function
only declares the 3rd p-value significant.

If you've used FieldTrip's FDR function you should fix the bug and re-run
your analysis. You may have missed significant effects (though in practice
missed p-values might actually be rare).
   -David



On Thu, Nov 19, 2015 at 12:46 PM, Kathrin Müsch <kathrin.muesch at gmail.com>
wrote:

> Hi,
>
> I realized that the fdr.m file gives a wrong output for the case that the smallest p values are above the FDR threshold. This means that the smallest p-values might be rejected, whereas lower p-values could pass the test. I have filed it as a bug (#3008).
>
> The current script only declares the p-values below or equal to the threshold as significant, whereas in fact all p-values that are smaller or equal to the largest p-value below or equal to this threshold should be declared significant according to the original paper ("find the largest P value that is below the line [threshold]. All voxels with P values less than or equal to this are declared active.”).
>
> I assume that the following line is incorrect and should be replaced by the one below:
>
> Incorrect: h = (ps<=pi);
>
> Correct: h = ps<=(max(ps(ps<=pi)));
>
> Cheers,
> Kathrin
>
> --
>
> Kathrin Müsch, Ph.D.
>
> Department of Psychology
> University of Toronto
> Toronto, Canada
> www.honeylab.org
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151122/bbc6a1ac/attachment-0002.html>