Question about nonparametric statistical testing of coherence differences

[Matthew Nelson]

My question involves a situation in which one is comparing the coherence
between two signals in two different conditions, and the coherence is
generally high in both conditions, but we would like to say that the
coherence is higher in one condition than the other. After some
thought about this issue, I wonder if shuffling trials between the two
conditions allows for a fair comparison to be made to the coherence in
the raw data? For example, if a pair of signals has different relative
phases in the two conditions, the overall coherence that exists in
both conditions will be decreased by the trial shuffling. It’s also
conceivable, though perhaps less likely, that the condition-shuffled
data could have more coherence than either condition alone. In either
case, the distribution of the cluster statistics of the coherence
differences in the condition-shuffled data would certainly provide
some sort of estimate of a null distribution of the coherence
difference that does uses the actual data, but since the overall
coherence values of the condition-shuffled and raw data could be so
different, is it fair to say that this is a null distribution of the
difference in coherence between the unshuffled conditions?

If you have the time to respond, I’m curious to know what your
thoughts are regarding this issue and specifically in a situation in
which it is necessary to assess significance on one session.
Performing the clustering and shuffling of coherence values across
sessions obviously avoids this problem. For a test within a session, I
wonder if it would be possible to use a bootstrapping or jackknife
method combined with clustering to get a fair estimate of the
variability of the cluster level statistics and perform hypothesis
tests using that.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.