about cluster randomization analysis
Eric Maris
maris at NICI.RU.NL
Tue Nov 8 13:17:50 CET 2005
Hi Marco,
thank you for your accurate responses. I fully understand from your arguments that temporally zooming on clusters is definitely wrong. Still, I wonder whether and how it is possible to use cluster randomization analysis cases in which it is difficult to formulate a precise hypothesis about when to expect an effect (for example, in infants), or cases in which an unexpected effect arises from a t-test. Do you think it would be correct to slide a relatively large (width of 200ms? 400ms? to be chosen a priori of course) window through the epochs and compute cluster randomization analysis for each latency to explore dubious significant t-test clusters?
If you have no hypothesis about where to expect an effect, you should use the complete latency window in which it may occur. Of course, this will reduce the sensitivity (statistical power) of your test (in comparison with the situation in which you do know when the effect can occur). As a rule, prior knowledge increases sensitivity.
Another related question: I computed a post-hoc non kosher tuning of the window around the most significative cluster in my data, and I saw that it is significative (p<0.05) if the window edges exceed of about 50 ms the cluster edges (since the cluster is about 70 ms long, the whole window is about 170 ms long); but if I take longer windows, the p-value increases quite rapidly (I'm running at least 500 random draws for each window, and checking that the result does not depend on the number of draws). Do you have such instabilities in your data or should I think that the effect relative to my cluster is definitely too weak? Or maybe my data are not clean enough?
This phenomenon is not an instability, it is what I would expect. Imagine your trials are 10 seconds long and there is an effect in the latency window between 1.3 and 1.35 seconds (i.e., less than 1 percent of trial length). If you ask clusterrandanalysis to compare the conditions over the complete trial length, it may very well miss the effect in the window between 1.3 and 1.35 seconds, because it has to use a large critical value in order to control for false positives in the time window where there is no effect (i.e., 99 percent of the 10 second trial).
greetings,
Eric Maris
On 10/28/05, Eric Maris <maris at nici.ru.nl> wrote:
Dear Marco,
> The procedure I am following now is a sort of two-steps method: in the
> first place, I choose a wide time interval and a low minimum number of
> channels. I end up with many clusters that are far from being
> significative. I then shorten the time interval to include just one
> cluster (starting from the most significant one), and increase the minimum
> number of channels, and run the analysis again. In this case, I eventually
> got a significative cluster where I was expecting it from a simple
> observation of the t-test. Do you think this procedure is right or am I
> doing something wrong? Is it correct to temporally focus on a cluster to
> check its significance?
Clusterrandanalysis only controls the false alarm (type I error) rate if you
choose the "tuning parameters" (latency interval, channel subset, the
minnbchan-parameter; and if you use on TFRs, also the frequency interval)
independent of the data. Instead, if you play around with these tuning
parameters until you find a cluster whose p-value exceeds the critical
alpha-level, you are not controlling the false alarm rate. In this case, the
chosen tuning parameters depend on the data.
An extreme example illustrates this even better. Assume you calculate
T-statistics for all (channel, time point)-pairs and you select the pair
with the largest T-statistic. Then, you select the latency interval that
only contains this time point and the channel subset that only contains this
channel. With these tuning parameters, you reduce your data to a single cell
in the spatiotemporal matrix, and clusterrrandanalysis will produce a
p-value that is very close to the p-value of a T-test. Since you have
selected this (channel, time point)-pair on the basis of its T-statistic,
this p-value is strongly biased.
> Another couple of questions:
> 1) Minnbchan. I understood it is the minimum number of significative
> neighbor (channel,time) points for a (channel,time) point to enter a
> cluster, no matter if adjacency is more in channel space or time
> direction. Am I right? Since time and channel space are quite different
> dimension, would it be better to set a minimum channel number separately
> for the two?
Minnbchan should also be chosen independent of the data. I introduced this
tuning parameter because it turned out that in 3-dimensional analyses on
TFRs (involving the dimensions time, space ( i.e., sensors) and frequency),
sometimes a cluster appeared that consisted of two or more 3-dimensional
"blobs" that were connected by a single (channel, time, frequency)-element.
From a physiological perspective, such a cluster does not make sense. To
remove these physiologically implausible (and therefore probably random)
connections, I introduced the minnbchan parameter. Because of this
physiological rationale, I apply the minimum number criterium to the
spatial, and not to the temporal dimension. Short-lived phenomena are very
well possible from a physiological perspective, whereas effects at spatially
isolated sensors are not.
> 2) Maybe because my data are average-referenced, I often end up with a
> positive and negative cluster emerging almost at the same time. Have you
> thought about any way to include the search of dipole-like configurations?
I have not thought about it, but it certainly makes sense to incorporate
biophysical constraints (such dipolar patterns) in the test statistic.
One should be aware of the fact that different hypotheses are tested before
and after rereferencing. This is physical and not a statistical issue. As
you most certainly know, EEG-signals are potential DIFFERENCES and therefore
the underlying physiological events that are measured by EEG depend on the
reference channel(s). If the experimental manipulation affects the current
reference channel, then rereferencing to another channel (or set of
channels) that is not affected by the experimental manipulation makes a
difference for the result of the statistical test.
greetings,
Eric Maris
--
Marco Buiatti - Post Doc
**************************************************************
Cognitive Neuroimaging Unit - INSERM U562
Service Hospitalier Frederic Joliot, CEA/DRM/DSV
4 Place du general Leclerc, 91401 Orsay cedex, France
Telephone: +33 1 69 86 77 65 Fax: +33 1 69 86 78 16
E-mail: marco.buiatti at gmail.com Web: www.unicog.org
***************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20051108/34ca3ab1/attachment-0001.html>
More information about the fieldtrip
mailing list