[FieldTrip] [FIELDTRIP] Independent channels stats question

Tue Oct 25 20:43:22 CEST 2011

Dear Sangi,

Eric might correct me if I'm wrong but when you do cluster-based stats
and your cfg.neighbors structure is empty what happens is exactly what
you described. You are still effectively correcting for 200 channels
just not for all the pixels so it should be quite close to Bonferoni
across channels.

Best,

Vladimir

On Tue, Oct 25, 2011 at 6:55 PM, Sangita Dandekar
<sangita.dandekar at gmail.com> wrote:
> Hi Fieldtrippers,
>
> I have questions similar to the ones asked a while back in the discussion
> archive. (See discussion pasted below my email if you're interested)
>
> We have ~200  intracranial EEG electrodes, which we assume are independent,
> and the question is how to apply
> cluster statistics to time frequency data while accounting for the multiple
> comparisons problem.
>
> In the archived discussion below, it was suggested that cluster statistics
> be applied to each  channel separately, and then, to account for MCP, apply
> Bonferroni correction on the resulting p-values.  However, as was pointed
> out in the discussion below, with ~200 electrodes Bonferroni
> correction is overly conservative.  Another possibility suggested in the
> archived discussion was  FDR correction on the results of cluster
> statistics, but it isn't clear to me
> how this would be done.  If anyone can explain how FDR can be applied to the
> results of cluster statistics, please let me know.
>
> I think a third possible solution to the problem is to get a 'global' null
> distribution as follows:
>
> Repeatedly:
> 1.  Randomly partition data
> 2.  Find time frequency clusters and the associated sum of t-statistics for
> each TFR cluster (do this WITHOUT clustering over electrodes/space, thereby
> treating each channel independently)
> 3.  Record maximum t-statistic sum in a 'global' null distribution on each
> iteration.   (Search over all electrodes for this maximum)
>
> Then one could compare the clusters as observed in the actual data (as
> determined independently at each electrode) to the global null distribution
> to get the false alarm rate
> associated with any cluster.  I think the above should account for the MCP.
>
> Is there anyway to implement the above procedure in Fieldtrip without
> modifying the underlying functions?  I know how to do a for loop around
> freqstatistics to treat each channel separately and then get the null
> distribution of tstat maxima and cluster statistics for each channel
> separately, but I am not sure
> if it is possible to treat each channel independently as I have outlined
> above and also get a global null distribution over all channels  (without
> making some
> changes to the underlying FT functions, that is).
>
> Thanks in advance for any help!
> Sangi
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Jul 6, 2010 at 12:47 PM, Matthew Davidson <sunyata at gmail.com> wrote:
>>
>> Jan-Mathis, thanks for the response.
>>
>> Unfortunately, we tend to have a lot of channels (~120-200), and once
>> we start using microelectrodes in the patients, it'll only get worse.
>>
>> If we were to divide our alpha by 120-200, wouldn't we have to run
>> 120-200 times as many permutations in order to get p-values low enough
>> to survive Bonferroni correction? That's a large jump; we might have
>> to run 100,000 permutations!
>>
>> What do you think about something like FDR correction instead?
>>
>> Matthew
>>
>> On Tue, Jul 6, 2010 at 7:21 AM, jan-mathijs schoffelen
>> <jan.schoffelen at donders.ru.nl> wrote:
>> > Dear Matthew,
>> >
>> > Your sensitivity problem is a known issue when using cluster-based test
>> > statistics, in which it is difficult to get small clusters significant
>> > in
>> > the presence of large clusters. This could also occur within a single
>> > channel (for example with a time-frequency decomposition, in which the
>> > summed spectro-temporal extent of an alpha-band effect could be much
>> > bigger
>> > than a gamma-band effect).
>> > In your case I think it would be statistically valid to do the
>> > cluster-based
>> > permutation test on each channel separately (which will involve a for
>> > loop
>> > around ft_freqstatistics, because it is not implemented in the fieldtrip
>> > code) and doing a post-hoc Bonferroni correction on the resulting
>> > p-values.
>> > If the number of channels is not too big, this might work.
>> >
>> > Good luck,
>> >
>> > Jan-Mathijs
>> >
>> >
>> > On Jul 6, 2010, at 3:26 AM, Matthew Davidson wrote:
>> >
>> >> Hi, this is Matthew Davidson. I recently took the Fieldtrip EEG/MEG
>> >> Toolkit (Hi Robert and Jan-Mathis!), and have been diving into using
>> >> Fieldtrip more directly.
>> >>
>> >> My question pertains to cluster-based correction when channels are
>> >> independent. My data is primarily intracranial EEG, and due to the
>> >> 1/f^2 power drop-off, electrodes directly on the brain reflect local
>> >> activity much more strongly than sensors further away. As a result, we
>> >> treat them as independent. Now, I can force the Fieldtrip clustering
>> >> algorithm to not cluster across channels by setting:
>> >>
>> >> cfg.neighbours = [];
>> >> cfg.minnbchan = 0;
>> >>
>> >> but it still computes the maximum cluster size for a particular
>> >> permutation based on *all* the data. This seems... less sensitive
>> >> somehow, as if large clusters in one channel negatively impact the
>> >> significance of clusters in another channel.
>> >>
>> >> Is there a better way to do this and still solve the MCP? E.g.,
>> >> compute the maxsum on each channel separately, and then use something
>> >> like FDR or Bonferroni correction on the maxsums across channels?
>> >>
>> >> Thanks for any advice you may have, and thanks for producing fieldtrip!
>> >> Matthew
>> >>
>> >> ----------------------------------
>> >> The aim of this list is to facilitate the discussion between users of
>> >> the
>> >> FieldTrip  toolbox, to share experiences and to discuss new ideas for
>> >> MEG
>> >> and EEG analysis. See also
>> >> http://listserv.surfnet.nl/archives/fieldtrip.html and
>> >> http://www.ru.nl/neuroimaging/fieldtrip.
>> >>
>> >
>> > Dr. J.M. (Jan-Mathijs) Schoffelen
>> > Donders Institute for Brain, Cognition and Behaviour,
>> > Centre for Cognitive Neuroimaging,
>> > Radboud University Nijmegen, The Netherlands
>> > J.Schoffelen at donders.ru.nl
>> > Telephone: 0031-24-3668063
>> >
>> > ----------------------------------
>> > The aim of this list is to facilitate the discussion between users of
>> > the
>> > FieldTrip  toolbox, to share experiences and to discuss new ideas for
>> > MEG
>> > and EEG analysis. See also
>> > http://listserv.surfnet.nl/archives/fieldtrip.html and
>> > http://www.ru.nl/neuroimaging/fieldtrip.
>> >
>>
>> ----------------------------------
>> The aim of this list is to facilitate the discussion between users of the
>> FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG
>> and EEG analysis. See also
>> http://listserv.surfnet.nl/archives/fieldtrip.html and
>> http://www.ru.nl/neuroimaging/fieldtrip.
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>