[FieldTrip] Question about cluster-based statistical testing (sum of t-stats or suprathreshold t-stats?)

[Eric Maris]

Dear Artemy,

 

Keep in mind that in the "Bullmore-style" cluster-mass statistic

 

\sum_i (t_i - c_i) where t_i > c_i

 

the thresholds c_i are constants (i.e., independent of the data).
Importantly, these constants also enter in the permutation distribution
that is used to evaluated the significance of the maximum cluster-mass
statistic, to the effect that the Bullmore-style and the Fieldtrip-style
permutation distributions are shifted versions of each other. As a result,
the p-values that roll out of the two approaches are identical.

 

Best,

 

Eric Maris

 

 

From: Artemy Kolchinsky [mailto:akolchin at indiana.edu] 
Sent: vrijdag 14 december 2012 14:31
To: fieldtrip at science.ru.nl
Subject: [FieldTrip] Question about cluster-based statistical testing (sum
of t-stats or suprathreshold t-stats?)

 

Hi all,

I have a question about the cluster-based statistical testing in
Fieldtrip, and also as described in Maris & Oostenveld's "Nonparametric
statistical testing of EEG-and MEG-data".  As far as I understand, the
'maxsum' statistic (as implemented in
https://code.google.com/p/fieldtrip/source/browse/trunk/private/clustersta
t.m) does the following:

1) Thresholds the t-statistic image (I just refer to t-statistics here for
simplicity, I realize it can be other things also) above a 'non-corrected'
threshold
2) Sums the t-statistics in each contiguous cluster (the 'cluster-mass')
of values that exceed the threshold
3) Does a random-permutation-based null distribution of the 'maximum
cluster-mass' statistic to test the significance of actually observed
cluster-masses

However, from reading about cluster-mass statistics in other places (such
as fMRI-based literature, e.g. Bullmore's
http://www.ai.mit.edu/events/talks/fMRI/papers/permutation_tests2.pdf [see
Section G] , recent work on Random Field Theory of cluster masses
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2739659/ , etc.), each
cluster's mass is actually taken to be the sum of the statistics *above*
the non-corrected threshold (ie, the 'sum of suprathreshold statistics').
In other words, let's say our t-statistic for electrode i is t_i and the
non-corrected threshold is c_i .  Then, if I am correct, Fieldtrip does:

\sum_i t_i where t_i > c_i

whereas other literature suggests

\sum_i (t_i - c_i) where t_i > c_i

It seems like these would be testing different things --- for example, I
suspect Fieldtrip's method, versus 'Bullmore's method', would disadvantage
clusters that have small spatial support.  To further complicate matters,
from scanning Fieldtrip's code it seems that using the 'wcm'
clusterstatistic option with wcm_weight = 1 would implement suprathreshold
summing, but this option is not documented.  Basically, my question is if
there is a particular reason for these discrepancies, and also if there is
any opinion on which cluster-based test 'works better' (for lack of a
better criterion).

Thanks greatly for any help,
-a

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121217/cd64450e/attachment.html>