about cluster randomization analysis

[Marco Buiatti]

Dear Vladimir and Eric,
 thank you for your accurate responses. I fully understand from your
arguments that temporally zooming on clusters is definitely wrong. Still, I
wonder whether and how it is possible to use cluster randomization analysis
cases in which it is difficult to formulate a precise hypothesis about when
to expect an effect (for example, in infants), or cases in which an
unexpected effect arises from a t-test. Do you think it would be correct to
slide a relatively large (width of 200ms? 400ms? to be chosen a priori of
course) window through the epochs and compute cluster randomization analysis
for each latency to explore dubious significant t-test clusters?
 Another related question: I computed a post-hoc non kosher tuning of the
window around the most significative cluster in my data, and I saw that it
is significative (p<0.05) if the window edges exceed of about 50 ms the
cluster edges (since the cluster is about 70 ms long, the whole window is
about 170 ms long); but if I take longer windows, the p-value increases
quite rapidly (I'm running at least 500 random draws for each window, and
checking that the result does not depend on the number of draws). Do you
have such instabilities in your data or should I think that the effect
relative to my cluster is definitely too weak? Or maybe my data are not
clean enough?
 About the minimum number of channels: I understand and agree that it is set
only in space. Maybe it would help saying it explicitly on the tutorial.
 About the reference: my non kosher approach does not include changing the
reference to chase a significant effect! My previous e-mail was probably
misleading about that.
 Thank you and have a good day,
 Marco
   On 10/28/05, Eric Maris <maris at nici.ru.nl> wrote:
>
> Dear Marco,
>
>
> > The procedure I am following now is a sort of two-steps method: in the
> > first place, I choose a wide time interval and a low minimum number of
> > channels. I end up with many clusters that are far from being
> > significative. I then shorten the time interval to include just one
> > cluster (starting from the most significant one), and increase the
> minimum
> > number of channels, and run the analysis again. In this case, I
> eventually
> > got a significative cluster where I was expecting it from a simple
> > observation of the t-test. Do you think this procedure is right or am I
> > doing something wrong? Is it correct to temporally focus on a cluster to
> > check its significance?
>
>
> Clusterrandanalysis only controls the false alarm (type I error) rate if
> you
> choose the "tuning parameters" (latency interval, channel subset, the
> minnbchan-parameter; and if you use on TFRs, also the frequency interval)
> independent of the data. Instead, if you play around with these tuning
> parameters until you find a cluster whose p-value exceeds the critical
> alpha-level, you are not controlling the false alarm rate. In this case,
> the
> chosen tuning parameters depend on the data.
>
> An extreme example illustrates this even better. Assume you calculate
> T-statistics for all (channel, time point)-pairs and you select the pair
> with the largest T-statistic. Then, you select the latency interval that
> only contains this time point and the channel subset that only contains
> this
> channel. With these tuning parameters, you reduce your data to a single
> cell
> in the spatiotemporal matrix, and clusterrrandanalysis will produce a
> p-value that is very close to the p-value of a T-test. Since you have
> selected this (channel, time point)-pair on the basis of its T-statistic,
> this p-value is strongly biased.
>
>
> > Another couple of questions:
> > 1) Minnbchan. I understood it is the minimum number of significative
> > neighbor (channel,time) points for a (channel,time) point to enter a
> > cluster, no matter if adjacency is more in channel space or time
> > direction. Am I right? Since time and channel space are quite different
> > dimension, would it be better to set a minimum channel number separately
> > for the two?
>
> Minnbchan should also be chosen independent of the data. I introduced this
> tuning parameter because it turned out that in 3-dimensional analyses on
> TFRs (involving the dimensions time, space (i.e., sensors) and frequency),
> sometimes a cluster appeared that consisted of two or more 3-dimensional
> "blobs" that were connected by a single (channel, time,
> frequency)-element.
> From a physiological perspective, such a cluster does not make sense. To
> remove these physiologically implausible (and therefore probably random)
> connections, I introduced the minnbchan parameter. Because of this
> physiological rationale, I apply the minimum number criterium to the
> spatial, and not to the temporal dimension. Short-lived phenomena are very
> well possible from a physiological perspective, whereas effects at
> spatially
> isolated sensors are not.
>
>
> > 2) Maybe because my data are average-referenced, I often end up with a
> > positive and negative cluster emerging almost at the same time. Have you
> > thought about any way to include the search of dipole-like
> configurations?
>
> I have not thought about it, but it certainly makes sense to incorporate
> biophysical constraints (such dipolar patterns) in the test statistic.
>
> One should be aware of the fact that different hypotheses are tested
> before
> and after rereferencing. This is physical and not a statistical issue. As
> you most certainly know, EEG-signals are potential DIFFERENCES and
> therefore
> the underlying physiological events that are measured by EEG depend on the
> reference channel(s). If the experimental manipulation affects the current
> reference channel, then rereferencing to another channel (or set of
> channels) that is not affected by the experimental manipulation makes a
> difference for the result of the statistical test.
>
>
> greetings,
>
> Eric Maris
>



--
Marco Buiatti - Post Doc

**************************************************************
Cognitive Neuroimaging Unit - INSERM U562
Service Hospitalier Frederic Joliot, CEA/DRM/DSV
4 Place du general Leclerc, 91401 Orsay cedex, France
Telephone: +33 1 69 86 77 65 Fax: +33 1 69 86 78 16
E-mail: marco.buiatti at gmail.com Web: www.unicog.org <http://www.unicog.org>
***************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20051108/1dd2cc5c/attachment.html>