Question about nonparametric statistical testing of coherence differences

[Eric Maris]

Dear Matthew,


> > It is important to always keep in mind which null hypothesis is tested
using
> > permutation inference. This null hypothesis is exchangeability. For the
case
> > that you describe, exchangeability involves that the probability
> > distributions of bivariate time series (or, equivalently, their Fourier
> > coefficients) in the two experimental conditions are identical. You
describe
> > a situation in which this null hypothesis does not hold, and the reason
for
> > this is not a single but two parameters: coherence and relative phase.
Both
> > parameters differ across the two conditions that you describe. Now, if
you
> > want to test whether coherence differs across the two conditions and you
> > don't want your inference to be affected by a difference in relative
phase,
> > then you have to come up with a test statistic that has the appropriate
> > sensitivity.  Such a test statistic might be the following: calculate
the
> > coherence difference between the two conditions using Fourier
coefficients
> > of which the phases were adjusted such that relative phase in the two
> > conditions are both equal to zero. (This is a simple operation,
involving a
> > condition-specific phase shift applied to all Fourier coefficients of
one of
> > the two channels.)
>
> It seems to me that this indeed is the test that I would be interested in
> for this situation. To reiterate the suggestion just to be clear that I
> understand it completely (and please let me know if I am misunderstanding
> it), this suggestion would be to adjust the phase of one signal in one
> condition so that the average relative phase between the two signals is
the
> same for the two conditions, not to eliminate the phase information on
each
> and every Fourier component. Thanks again for sharing your thoughts on
that.

This is correct. Don't forget to apply this phase shift for every
permutation, such that it becomes part of your test statistic.


>
> >
> > By the way, I did not understand the following sentence in your email:
"It's
> > also
> > conceivable, though perhaps less likely, that the condition-shuffled
> > data could have more coherence than either condition alone." For the
> > situation that you describe, the coherence difference between the
conditions
> > (which is the relevant quantity for the phenomen in which you are
> > interested) will on average be less for the permuted data than for the
> > original data.
>
> Sorry for the confusion. For the situation that I specifically described
in
> my first message, indeed the overall coherence in the permuted data would
be
> less. What I was referring to in that sentence though was that in other
> situations, which I did not describe, it could be possible to have an
> increase in overall coherence in the shuffled data. For an example of such
a
> situation, it seems to me that this could occur if the relative phases of
> the two signals are the same in both conditions but the overall spectral
> power differed between the two conditions for both signals such both
signals
> had more power in condition one than in condition two. I haven't tested
this
> with simulations, but it seems that in the case the coherence could be
> higher in the trial shuffled case because of the induced correlation in
> spectral power between the signals resulting from mixing the two
conditions.

In case you decide to run such a simulation, you may also include the
phase-locking factor (Lachaux et al, 1999) in your calculation. The PLF does
not depend on amplitude correlation over trials.

>
> So following the line of thought from your suggestion above, in such a
> situation in order to prevent the magnitude differences from affecting the
> inference regarding the coherence difference between the two conditions,
> would you say that it would be appropriate to also adjust the overall
> magnitude of the pair of Fourier components in one condition to match the
> overall magnitude of the components in the other conditions when doing the
> permutation?

This may help. You can include this test statistic in your simulation,
together with the PLF.

>
> >
> >> sessions obviously avoids this problem. For a test within a session, I
> >> wonder if it would be possible to use a bootstrapping or jackknife
> >> method combined with clustering to get a fair estimate of the
> >> variability of the cluster level statistics and perform hypothesis
> >> tests using that.
> >
> > Here the answer is simple. Permutation inference can only be used for
> > comparing experimental conditions. However, I want to argue that, for
> > investigating coherence, comparing experimental conditions is the only
> > sensible thing one can do. The reason is that, for simple biophysical
> > reasons, the null hypothesis of zero coherence within a single
condition,
> > will never be true, at least not in electrophysiological studies. This
is
> > because the potentials that are recorded in the two channels will always
be
> > affected by the physiological activity of common sources, simply because
all
> > potentials are volume conducted.
> >
>
> I agree completely. Just to clarify, I was suggesting using bootstrapping
or
> jackknifing to estimate the variance of the coherence of each measure
> independently, and then use that to test the null hypothesis of no
coherence
> difference between the two conditions rather than the null hypothesis of
> zero coherence.

I would not advise this. The permutation test is to be preferred over
bootstrap- and jacknife-based statistical test, because we can prove that
the permutation test controls the false alarm rate. We cannot prove this for
bootstrap- and jacknife-based tests.

Best,

Eric



>
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of the
FieldTrip
> toolbox, to share experiences and to discuss new ideas for MEG and EEG
analysis.
> See also http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.