warning: bug discovered in new cluster-based randomization statistics

[Robert Oostenveld]

Dear FT users,

last Friday we found a bug in the newly implemented cluster-based
randomization statistics. The problem occurred when using either
TIMELOCKSTATISTICS, FREQSTATISTICS or SOURCESTATISTICS with
cfg.method='montecarlo' in combination with cfg.correctm='cluster'
and cfg.clusterthreshold='parametric' (which is the default for
clusterthreshold).

This bug only affected the new statistics implementations and NOT the
old CLUSTERRANDANALYSIS function, which most of you are probably
still using for your channel data.

The problem was that the samples belonging to the time-frequency-
spatial clusters were determined in the observed data, but that the
same samples were also re-used to compute the cluster-statistic in
each random reshuffling of the data. I.e., the cluster statistics for
the randomizations were computed not based on the thresholded data
for that randomization, but instead based on the original thresholded
data. It is likely that this resulted in values for each random
cluster statistic that were smaller that what they should have been
(*see note below*), resulting in a randomization distribution of the
cluster-statistic that was too narrow (i.e. the histogram was too
narrow). Therefore, effects in the observed may have been indicated
as significant (in the tail of the randomization distribution) where
they should not have been. The bug has not always been in the code,
but was recently introduced (copy and paste problem). I suggest that
you redo your recent analyses.

Sorry about the inconvenience. If you have questions, you can contact
us through the mailing list or with a direct mail.

best regards,
Robert

(*) due to the bug, the cluster-statistic values in each observation
may also have been larger than what they should have been. This is
most likely the case when you have very strong effects. I.e. in
strong effects, it is most likely that you have underestimated your
significance and in small effects it is likely that you have
overestimated your significance.