[FieldTrip] Cluster-based permutation tests on time-frequency and size of conditions

[Stephen Whitmarsh]

I should add that (1) is typically done *within *subjects, and (2) *over *
subjects.

Cheers,
Stephen

On 18 March 2015 at 19:20, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
wrote:

> Dear Martina,
>
> It might help to distinguish two aspects of cluster-based statistic.
>
> 1) the statistical approuch that you will use to determine whether a
> time-channel-datapoint / time-frequency-channel-datapoint /
> time-frequency-voxel-datapoint is considered significant different between
> conditions.
> 2) the statistical approuch that you will use to determine whether *a
> cluster of* time-channel-datapoints / time-frequency-channel-datapoints /
> time-frequency-voxel-datapoints is considered significantly different
> between conditions.
>
> When you talk about *cluster statistics*, you probably think about the
> second part. But this might not be what you should initially be concerned
> with when thinking about e.g. different numbers of trials between
> conditions. Rather, consider what statistical tests you (can) use to
> compare your time-frequency values between conditions *(within subjects).*
> This can be, e.g., a t-test, a nonparametric (e.g. montecarlo) test, or any
> test, for that matter. As far as my limited knowledge of statistics goes,
> in most simple and non-extreme cases, unequal number of trials that does
> not have to increase your chance of type I errors, rather that of type 2
> (you'll be insensitive to differences if you don't have enough observations
> in one condition due to noisy estimate of means/distribution). But in any
> case it's a simple question to google or ask a statistician.
>
> Now, after you are happy with and confident about the between conditions
> statistical test, consider how the cluster statistics might help you.
> First of all, how does it determine whether a cluster is significantly
> different between conditions? There are different options, but the gist is
> that it takes your significant statistical numbers of step (1), adds them
> up when they belong to the same cluster (based on whether they are
> neigbourings in time/freq/space with other significant numbers), takes the
> maximum of these summed up clusters (there might be more than one cluster),
> and then compares this one value to the same taken from a*(non-parametric)
> monte-carlo distribution *of the null hypothesis based on permuting the
> values over conditions (and then calculating the maximum sum). The Maris
> and Oostenveld paper explains it in more detail.
>
> The reason for doing cluster-statistics is that its a smart way of dealing
> with multiple comparisons over many time x frequency x channels (or space).
> The method is blind for your decisions about how its computed for each
> point in time x frequency x channels (or space).
>
> I find the FieldTrip statistics functions, their configurations etc., and
> the way they interact confusing at times, but I hope this helps to clear it
> up a bit.
>
> Long story short - I think your question does not limit itself to cluster
> statistics and at the same time is much simpler. It's all about (1).
>
> Best wishes,
> Stephen
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> There are two separate steps cluster statistics (as implemented in
> FieldTrip, but in general as well).
>
>
>
>
> On 18 March 2015 at 14:51, Joram van Driel <joramvandriel at gmail.com>
> wrote:
>
>> Hi Martina,
>>
>> In general, I'd advice to do some kind of trial-selection procedure when
>> comparing error versus correct trials, in order to trial-count-match the
>> two conditions. Otherwise you run into problems considering: SNR (higher
>> for the correct condition), and RT (errors are usually faster, resulting in
>> a time-on-task confound). What I always do is pick from the correct
>> condition a similar number of trials that are close to the RT distribution
>> of the error trials (i.e. the faster correct trials). That way you solve
>> both problems at once (and probably the cluster-based permutation test in
>> field trip will work as well, as a bonus ;)).
>>
>> Best,
>> Joram
>>
>> On Wed, Mar 18, 2015 at 2:31 PM, Martina Rossi <martina.rossi76 at yahoo.it>
>> wrote:
>>
>>> Dear All,
>>>
>>> I would like to get some feedback from the community about a statistical
>>> analysis problem I need to tackle with my study.
>>> I want to apply the cluster-based permutation tests on time-frequency
>>> data considering two conditions (correct vs error).
>>> Unfortunately, these two conditions have different sizes (correct >>
>>> error).
>>> Right now, I am only considering subjects having a ratio "error/correct"
>>> bigger than 1/5, yet this is only an arbitrary threshold I set.
>>> The question is the following:
>>> is there a formal way to identify a threshold by which two conditions
>>> can be realiably compared with the cluster-based permutation tests?
>>> If the cluster-based approach is not suitable in this scenario, is there
>>> any other approach you would suggest?
>>> I shall perhaps point out that I am working on EEG data recorded with a
>>> 32 channel system (impedance levels < 10 kΩ).
>>>
>>> Looking forward to hear your feedback :)
>>>
>>> Kind Regards,
>>> Martina Rossi
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>
>>
>>
>> --
>> Joram van Driel
>> Postdoc @ Vrije Universiteit Amsterdam
>> Cognitive Psychology
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150318/68c7ffa3/attachment-0001.html>