[FieldTrip] Cluster-based permutation tests on time-frequency and size of conditions

[Stephen Whitmarsh]

Dear Martina,

It might help to distinguish two aspects of cluster-based statistic.

1) the statistical approuch that you will use to determine whether a
time-channel-datapoint / time-frequency-channel-datapoint /
time-frequency-voxel-datapoint is considered significant different between
conditions.
2) the statistical approuch that you will use to determine whether *a
cluster of* time-channel-datapoints / time-frequency-channel-datapoints /
time-frequency-voxel-datapoints is considered significantly different
between conditions.

When you talk about *cluster statistics*, you probably think about the
second part. But this might not be what you should initially be concerned
with when thinking about e.g. different numbers of trials between
conditions. Rather, consider what statistical tests you (can) use to
compare your time-frequency values between conditions *(within subjects).*
This can be, e.g., a t-test, a nonparametric (e.g. montecarlo) test, or any
test, for that matter. As far as my limited knowledge of statistics goes,
in most simple and non-extreme cases, unequal number of trials that does
not have to increase your chance of type I errors, rather that of type 2
(you'll be insensitive to differences if you don't have enough observations
in one condition due to noisy estimate of means/distribution). But in any
case it's a simple question to google or ask a statistician.

Now, after you are happy with and confident about the between conditions
statistical test, consider how the cluster statistics might help you.
First of all, how does it determine whether a cluster is significantly
different between conditions? There are different options, but the gist is
that it takes your significant statistical numbers of step (1), adds them
up when they belong to the same cluster (based on whether they are
neigbourings in time/freq/space with other significant numbers), takes the
maximum of these summed up clusters (there might be more than one cluster),
and then compares this one value to the same taken from a*(non-parametric)
monte-carlo distribution *of the null hypothesis based on permuting the
values over conditions (and then calculating the maximum sum). The Maris
and Oostenveld paper explains it in more detail.

The reason for doing cluster-statistics is that its a smart way of dealing
with multiple comparisons over many time x frequency x channels (or space).
The method is blind for your decisions about how its computed for each
point in time x frequency x channels (or space).

I find the FieldTrip statistics functions, their configurations etc., and
the way they interact confusing at times, but I hope this helps to clear it
up a bit.

Long story short - I think your question does not limit itself to cluster
statistics and at the same time is much simpler. It's all about (1).

Best wishes,
Stephen















There are two separate steps cluster statistics (as implemented in
FieldTrip, but in general as well).




On 18 March 2015 at 14:51, Joram van Driel <joramvandriel at gmail.com> wrote:

> Hi Martina,
>
> In general, I'd advice to do some kind of trial-selection procedure when
> comparing error versus correct trials, in order to trial-count-match the
> two conditions. Otherwise you run into problems considering: SNR (higher
> for the correct condition), and RT (errors are usually faster, resulting in
> a time-on-task confound). What I always do is pick from the correct
> condition a similar number of trials that are close to the RT distribution
> of the error trials (i.e. the faster correct trials). That way you solve
> both problems at once (and probably the cluster-based permutation test in
> field trip will work as well, as a bonus ;)).
>
> Best,
> Joram
>
> On Wed, Mar 18, 2015 at 2:31 PM, Martina Rossi <martina.rossi76 at yahoo.it>
> wrote:
>
>> Dear All,
>>
>> I would like to get some feedback from the community about a statistical
>> analysis problem I need to tackle with my study.
>> I want to apply the cluster-based permutation tests on time-frequency
>> data considering two conditions (correct vs error).
>> Unfortunately, these two conditions have different sizes (correct >>
>> error).
>> Right now, I am only considering subjects having a ratio "error/correct"
>> bigger than 1/5, yet this is only an arbitrary threshold I set.
>> The question is the following:
>> is there a formal way to identify a threshold by which two conditions can
>> be realiably compared with the cluster-based permutation tests?
>> If the cluster-based approach is not suitable in this scenario, is there
>> any other approach you would suggest?
>> I shall perhaps point out that I am working on EEG data recorded with a
>> 32 channel system (impedance levels < 10 kΩ).
>>
>> Looking forward to hear your feedback :)
>>
>> Kind Regards,
>> Martina Rossi
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>
>
>
> --
> Joram van Driel
> Postdoc @ Vrije Universiteit Amsterdam
> Cognitive Psychology
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150318/ea7c328b/attachment-0001.html>