Hi Fieldtrippers,<br><br>I have questions similar to the ones asked a while back in the discussion archive. (See discussion pasted below my email if you're interested)<br><br>We have ~200 intracranial EEG electrodes, which we assume are independent, and the question is how to apply<br>
cluster statistics to time frequency data while accounting for the multiple comparisons problem.<br><br>In the archived discussion below, it was suggested that cluster statistics be applied to each channel separately, and then, to account for MCP, apply <br>
Bonferroni correction on the resulting p-values. However, as was pointed out in the discussion below, with ~200 electrodes Bonferroni<br>correction is overly conservative. Another possibility suggested in the archived discussion was FDR correction on the results of cluster statistics, but it isn't clear to me <br>
how this would be done. If anyone can explain how FDR can be applied to the results of cluster statistics, please let me know.<br><br>I think a third possible solution to the problem is to get a 'global' null distribution as follows:<br>
<br>
Repeatedly:<br>1. Randomly partition data <br>2. Find time frequency clusters and the associated sum of t-statistics for each TFR cluster (do this WITHOUT clustering over electrodes/space, thereby treating each channel independently) <br>
3. Record maximum t-statistic sum in a 'global' null distribution on each iteration. (Search over all electrodes for this maximum)<br>
<br>Then one could compare the clusters as observed in the actual data (as determined independently at each electrode) to the global null distribution to get the false alarm rate<br>associated with any cluster. I think the above should account for the MCP.<br>
<br>Is there anyway to implement the above procedure in Fieldtrip without modifying the underlying functions? I know how to do a for loop around <br>
freqstatistics to treat each channel separately and then get the null distribution of tstat maxima and cluster statistics for each channel separately, but I am not sure<br>if it is possible to treat each channel independently as I have outlined above and also get a global null distribution over all channels (without making some<br>
changes to the underlying FT functions, that is).<br><br>Thanks in advance for any help!<br>Sangi<br> <br> <br><br><br><br><br><br><br><br><br><br><br><br><div class="gmail_quote">On Tue, Jul 6, 2010 at 12:47 PM, Matthew Davidson <span dir="ltr"><<a href="mailto:sunyata@gmail.com" target="_blank">sunyata@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Jan-Mathis, thanks for the response.<br>
<br>
Unfortunately, we tend to have a lot of channels (~120-200), and once<br>
we start using microelectrodes in the patients, it'll only get worse.<br>
<br>
If we were to divide our alpha by 120-200, wouldn't we have to run<br>
120-200 times as many permutations in order to get p-values low enough<br>
to survive Bonferroni correction? That's a large jump; we might have<br>
to run 100,000 permutations!<br>
<br>
What do you think about something like FDR correction instead?<br>
<font color="#888888"><br>
Matthew<br>
</font><div><div></div><div><br>
On Tue, Jul 6, 2010 at 7:21 AM, jan-mathijs schoffelen<br>
<<a href="mailto:jan.schoffelen@donders.ru.nl" target="_blank">jan.schoffelen@donders.ru.nl</a>> wrote:<br>
> Dear Matthew,<br>
><br>
> Your sensitivity problem is a known issue when using cluster-based test<br>
> statistics, in which it is difficult to get small clusters significant in<br>
> the presence of large clusters. This could also occur within a single<br>
> channel (for example with a time-frequency decomposition, in which the<br>
> summed spectro-temporal extent of an alpha-band effect could be much bigger<br>
> than a gamma-band effect).<br>
> In your case I think it would be statistically valid to do the cluster-based<br>
> permutation test on each channel separately (which will involve a for loop<br>
> around ft_freqstatistics, because it is not implemented in the fieldtrip<br>
> code) and doing a post-hoc Bonferroni correction on the resulting p-values.<br>
> If the number of channels is not too big, this might work.<br>
><br>
> Good luck,<br>
><br>
> Jan-Mathijs<br>
><br>
><br>
> On Jul 6, 2010, at 3:26 AM, Matthew Davidson wrote:<br>
><br>
>> Hi, this is Matthew Davidson. I recently took the Fieldtrip EEG/MEG<br>
>> Toolkit (Hi Robert and Jan-Mathis!), and have been diving into using<br>
>> Fieldtrip more directly.<br>
>><br>
>> My question pertains to cluster-based correction when channels are<br>
>> independent. My data is primarily intracranial EEG, and due to the<br>
>> 1/f^2 power drop-off, electrodes directly on the brain reflect local<br>
>> activity much more strongly than sensors further away. As a result, we<br>
>> treat them as independent. Now, I can force the Fieldtrip clustering<br>
>> algorithm to not cluster across channels by setting:<br>
>><br>
>> cfg.neighbours = [];<br>
>> cfg.minnbchan = 0;<br>
>><br>
>> but it still computes the maximum cluster size for a particular<br>
>> permutation based on *all* the data. This seems... less sensitive<br>
>> somehow, as if large clusters in one channel negatively impact the<br>
>> significance of clusters in another channel.<br>
>><br>
>> Is there a better way to do this and still solve the MCP? E.g.,<br>
>> compute the maxsum on each channel separately, and then use something<br>
>> like FDR or Bonferroni correction on the maxsums across channels?<br>
>><br>
>> Thanks for any advice you may have, and thanks for producing fieldtrip!<br>
>> Matthew<br>
>><br>
>> ----------------------------------<br>
>> The aim of this list is to facilitate the discussion between users of the<br>
>> FieldTrip toolbox, to share experiences and to discuss new ideas for MEG<br>
>> and EEG analysis. See also<br>
>> <a href="http://listserv.surfnet.nl/archives/fieldtrip.html" target="_blank">http://listserv.surfnet.nl/archives/fieldtrip.html</a> and<br>
>> <a href="http://www.ru.nl/neuroimaging/fieldtrip" target="_blank">http://www.ru.nl/neuroimaging/fieldtrip</a>.<br>
>><br>
><br>
> Dr. J.M. (Jan-Mathijs) Schoffelen<br>
> Donders Institute for Brain, Cognition and Behaviour,<br>
> Centre for Cognitive Neuroimaging,<br>
> Radboud University Nijmegen, The Netherlands<br>
> <a href="mailto:J.Schoffelen@donders.ru.nl" target="_blank">J.Schoffelen@donders.ru.nl</a><br>
> Telephone: 0031-24-3668063<br>
><br>
> ----------------------------------<br>
> The aim of this list is to facilitate the discussion between users of the<br>
> FieldTrip toolbox, to share experiences and to discuss new ideas for MEG<br>
> and EEG analysis. See also<br>
> <a href="http://listserv.surfnet.nl/archives/fieldtrip.html" target="_blank">http://listserv.surfnet.nl/archives/fieldtrip.html</a> and<br>
> <a href="http://www.ru.nl/neuroimaging/fieldtrip" target="_blank">http://www.ru.nl/neuroimaging/fieldtrip</a>.<br>
><br>
<br>
----------------------------------<br>
The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also <a href="http://listserv.surfnet.nl/archives/fieldtrip.html" target="_blank">http://listserv.surfnet.nl/archives/fieldtrip.html</a> and <a href="http://www.ru.nl/neuroimaging/fieldtrip" target="_blank">http://www.ru.nl/neuroimaging/fieldtrip</a>.<br>
</div></div></blockquote></div><br>