about cluster randomization analysis

[Eric Maris]

Dear Marco,


> The procedure I am following now is a sort of two-steps method: in the
> first place, I choose a wide time interval and a low minimum number of
> channels. I end up with many clusters that are far from being
> significative. I then shorten the time interval to include just one
> cluster (starting from the most significant one), and increase the minimum
> number of channels, and run the analysis again. In this case, I eventually
> got a significative cluster where I was expecting it from a simple
> observation of the t-test. Do you think this procedure is right or am I
> doing something wrong? Is it correct to temporally focus on a cluster to
> check its significance?


Clusterrandanalysis only controls the false alarm (type I error) rate if you
choose the "tuning parameters" (latency interval, channel subset, the
minnbchan-parameter; and if you use on TFRs, also the frequency interval)
independent of the data. Instead, if you play around with these tuning
parameters until you find a cluster whose p-value exceeds the critical
alpha-level, you are not controlling the false alarm rate. In this case, the
chosen tuning parameters depend on the data.

An extreme example illustrates this even better. Assume you calculate
T-statistics for all (channel, time point)-pairs and you select the pair
with the largest T-statistic. Then, you select the latency interval that
only contains this time point and the channel subset that only contains this
channel. With these tuning parameters, you reduce your data to a single cell
in the spatiotemporal matrix, and clusterrrandanalysis will produce a
p-value that is very close to the p-value of a T-test. Since you have
selected this (channel, time point)-pair on the basis of its T-statistic,
this p-value is strongly biased.


> Another couple of questions:
> 1) Minnbchan. I understood it is the minimum number of significative
> neighbor (channel,time) points for a  (channel,time) point to enter a
> cluster, no matter if adjacency is more in channel space or time
> direction. Am I right? Since time and channel space are quite different
> dimension, would it be better to set a minimum channel number separately
> for the two?

Minnbchan should also be chosen independent of the data. I introduced this
tuning parameter because it turned out that in 3-dimensional analyses on
TFRs (involving the dimensions time, space (i.e., sensors) and frequency),
sometimes a cluster appeared that consisted of two or more 3-dimensional
"blobs" that were connected by a single (channel, time, frequency)-element.
>>From a physiological perspective, such a cluster does not make sense. To
remove these physiologically implausible (and therefore probably random)
connections, I introduced the minnbchan parameter. Because of this
physiological rationale, I apply the minimum number criterium to the
spatial, and not to the temporal dimension. Short-lived phenomena are very
well possible from a physiological perspective, whereas effects at spatially
isolated sensors are not.


> 2) Maybe because my data are average-referenced, I often end up with a
> positive and negative cluster emerging almost at the same time. Have you
> thought about any way to include the search of dipole-like configurations?

I have not thought about it, but it certainly makes sense to incorporate
biophysical constraints (such dipolar patterns) in the test statistic.

One should be aware of the fact that different hypotheses are tested before
and after rereferencing. This is physical and not a statistical issue. As
you most certainly know, EEG-signals are potential DIFFERENCES and therefore
the underlying physiological events that are measured by EEG depend on the
reference channel(s). If the experimental manipulation affects the current
reference channel, then rereferencing to another channel (or set of
channels) that is not affected by the experimental manipulation makes a
difference for the result of the statistical test.


greetings,

Eric Maris