[FieldTrip] Cluster-based permutation tests on time-frequency and size of conditions

Wed Mar 18 19:43:01 CET 2015

You can also check out this video of Robert. Apologies for the quality -
not of the talk, but of the recording :-)

At 14:45 he actually mentions unequal number of trials between conditions.

https://www.youtube.com/watch?v=vOSfabsDUNg

Cheers,
Stephen

On 18 March 2015 at 19:33, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
wrote:

> I should add that (1) is typically done *within *subjects, and (2) *over *
> subjects.
>
> Cheers,
> Stephen
>
> On 18 March 2015 at 19:20, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
> wrote:
>
>> Dear Martina,
>>
>> It might help to distinguish two aspects of cluster-based statistic.
>>
>> 1) the statistical approuch that you will use to determine whether a
>> time-channel-datapoint / time-frequency-channel-datapoint /
>> time-frequency-voxel-datapoint is considered significant different between
>> conditions.
>> 2) the statistical approuch that you will use to determine whether *a
>> cluster of* time-channel-datapoints / time-frequency-channel-datapoints
>> / time-frequency-voxel-datapoints is considered significantly different
>> between conditions.
>>
>> When you talk about *cluster statistics*, you probably think about the
>> second part. But this might not be what you should initially be concerned
>> with when thinking about e.g. different numbers of trials between
>> conditions. Rather, consider what statistical tests you (can) use to
>> compare your time-frequency values between conditions *(within
>> subjects).* This can be, e.g., a t-test, a nonparametric (e.g.
>> montecarlo) test, or any test, for that matter. As far as my limited
>> knowledge of statistics goes, in most simple and non-extreme cases, unequal
>> number of trials that does not have to increase your chance of type I
>> errors, rather that of type 2 (you'll be insensitive to differences if you
>> don't have enough observations in one condition due to noisy estimate of
>> means/distribution). But in any case it's a simple question to google or
>> ask a statistician.
>>
>> Now, after you are happy with and confident about the between conditions
>> statistical test, consider how the cluster statistics might help you.
>> First of all, how does it determine whether a cluster is significantly
>> different between conditions? There are different options, but the gist is
>> that it takes your significant statistical numbers of step (1), adds them
>> up when they belong to the same cluster (based on whether they are
>> neigbourings in time/freq/space with other significant numbers), takes the
>> maximum of these summed up clusters (there might be more than one cluster),
>> and then compares this one value to the same taken from a*(non-parametric)
>> monte-carlo distribution *of the null hypothesis based on permuting the
>> values over conditions (and then calculating the maximum sum). The Maris
>> and Oostenveld paper explains it in more detail.
>>
>> The reason for doing cluster-statistics is that its a smart way of
>> dealing with multiple comparisons over many time x frequency x channels (or
>> space). The method is blind for your decisions about how its computed for
>> each point in time x frequency x channels (or space).
>>
>> I find the FieldTrip statistics functions, their configurations etc., and
>> the way they interact confusing at times, but I hope this helps to clear it
>> up a bit.
>>
>> Long story short - I think your question does not limit itself to cluster
>> statistics and at the same time is much simpler. It's all about (1).
>>
>> Best wishes,
>> Stephen
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> There are two separate steps cluster statistics (as implemented in
>> FieldTrip, but in general as well).
>>
>>
>>
>>
>> On 18 March 2015 at 14:51, Joram van Driel <joramvandriel at gmail.com>
>> wrote:
>>
>>> Hi Martina,
>>>
>>> In general, I'd advice to do some kind of trial-selection procedure when
>>> comparing error versus correct trials, in order to trial-count-match the
>>> two conditions. Otherwise you run into problems considering: SNR (higher
>>> for the correct condition), and RT (errors are usually faster, resulting in
>>> a time-on-task confound). What I always do is pick from the correct
>>> condition a similar number of trials that are close to the RT distribution
>>> of the error trials (i.e. the faster correct trials). That way you solve
>>> both problems at once (and probably the cluster-based permutation test in
>>> field trip will work as well, as a bonus ;)).
>>>
>>> Best,
>>> Joram
>>>
>>> On Wed, Mar 18, 2015 at 2:31 PM, Martina Rossi <martina.rossi76 at yahoo.it
>>> > wrote:
>>>
>>>> Dear All,
>>>>
>>>> I would like to get some feedback from the community about a
>>>> statistical analysis problem I need to tackle with my study.
>>>> I want to apply the cluster-based permutation tests on time-frequency
>>>> data considering two conditions (correct vs error).
>>>> Unfortunately, these two conditions have different sizes (correct >>
>>>> error).
>>>> Right now, I am only considering subjects having a ratio
>>>> "error/correct" bigger than 1/5, yet this is only an arbitrary threshold I
>>>> set.
>>>> The question is the following:
>>>> is there a formal way to identify a threshold by which two conditions
>>>> can be realiably compared with the cluster-based permutation tests?
>>>> If the cluster-based approach is not suitable in this scenario, is
>>>> there any other approach you would suggest?
>>>> I shall perhaps point out that I am working on EEG data recorded with a
>>>> 32 channel system (impedance levels < 10 kΩ).
>>>>
>>>> Looking forward to hear your feedback :)
>>>>
>>>> Kind Regards,
>>>> Martina Rossi
>>>>
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>
>>>
>>>
>>>
>>> --
>>> Joram van Driel
>>> Postdoc @ Vrije Universiteit Amsterdam
>>> Cognitive Psychology
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150318/2f8017c2/attachment-0001.html>