[FieldTrip] Question about cluster-based statistical testing (sum of t-stats or suprathreshold t-stats?)

[Eric Maris]

Dear Artemy,

 

 


Thank you greatly for the response!

 

Keep in mind that in the "Bullmore-style" cluster-mass statistic

 

\sum_i (t_i - c_i) where t_i > c_i

 

the thresholds c_i are constants (i.e., independent of the data). 

 

I think I see what you mean.  In my case, t_i are t-statistics, and I use
a p < 0.05 cutoff on both tails for the t distribution.  So yes, here all
the c_i would be the same for all voxels and independent of the data.

 

Importantly, these constants also enter in the permutation distribution
that is used to evaluated the significance of the maximum cluster-mass
statistic, to the effect that the Bullmore-style and the Fieldtrip-style
permutation distributions are shifted versions of each other. As a result,
the p-values that roll out of the two approaches are identical.

 

If I understand correctly, having the same resulting p-values could only
be if the two methods assign the same rank-ordering to a given a set of
clusters.  But I don't think that is the case. Let's imagine that the
t-statistic cutoff 'c' is equal to 1, and the data contains two
suprathreshold clusters (let's say this is a spatial test and the clusters
are composed of electrodes):

 

- The first cluster has 10 electrodes, each one with a t-statistic equal
to 1.1

- The second cluster has 2 electrodes, both with a t-statistic equal to 3

 

As I understand, Bullmore's method would assign cluster 1 a mass of
10*(1.1-1) = 1 and cluster 2 a mass of 2*(3-1)=4 , while your method would
assign cluster 1 a mass of 10*1.1 = 11 and cluster 2 a mass of 2*3 = 6.
Hence, given a null distribution, it should be possible to choose a
cluster-based threshold that indicates as significant only cluster 1 under
Bullmore's method, and only cluster 2 under yours.

 

 

I think your reasoning is correct: when the data contain more than one
suprathreshold cluster, my argument does not apply anymore. Your example
shows that the Bullmore- and Fieldtrip-style cluster statistics have
different sensitivities. Thank you for pointing this out. For every test
statistic, the decisions based on the permutation p-value controls the
type-I error rate, but the type-II error rate (the complement of
sensitivity) depends on the exact test statistic.

 

Best,

 

Eric Maris

 

 

 

 

 

 

 

 

Thanks again,

-Artemy

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121229/65d5f6d8/attachment.html>