Question about nonparametric statistical testing of coherence differences

[Matthew Nelson]

Thanks very much for your response Eric, I found it to be very helpful.

On Tue, Nov 17, 2009 at 10:36 AM, Eric Maris <e.maris at donders.ru.nl> wrote:
>
> It is important to always keep in mind which null hypothesis is tested using
> permutation inference. This null hypothesis is exchangeability. For the case
> that you describe, exchangeability involves that the probability
> distributions of bivariate time series (or, equivalently, their Fourier
> coefficients) in the two experimental conditions are identical. You describe
> a situation in which this null hypothesis does not hold, and the reason for
> this is not a single but two parameters: coherence and relative phase. Both
> parameters differ across the two conditions that you describe. Now, if you
> want to test whether coherence differs across the two conditions and you
> don't want your inference to be affected by a difference in relative phase,
> then you have to come up with a test statistic that has the appropriate
> sensitivity.  Such a test statistic might be the following: calculate the
> coherence difference between the two conditions using Fourier coefficients
> of which the phases were adjusted such that relative phase in the two
> conditions are both equal to zero. (This is a simple operation, involving a
> condition-specific phase shift applied to all Fourier coefficients of one of
> the two channels.)

It seems to me that this indeed is the test that I would be interested in
for this situation. To reiterate the suggestion just to be clear that I
understand it completely (and please let me know if I am misunderstanding
it), this suggestion would be to adjust the phase of one signal in one
condition so that the average relative phase between the two signals is the
same for the two conditions, not to eliminate the phase information on each
and every Fourier component. Thanks again for sharing your thoughts on that.

>
> By the way, I did not understand the following sentence in your email: "It's
> also
> conceivable, though perhaps less likely, that the condition-shuffled
> data could have more coherence than either condition alone." For the
> situation that you describe, the coherence difference between the conditions
> (which is the relevant quantity for the phenomen in which you are
> interested) will on average be less for the permuted data than for the
> original data.

Sorry for the confusion. For the situation that I specifically described in
my first message, indeed the overall coherence in the permuted data would be
less. What I was referring to in that sentence though was that in other
situations, which I did not describe, it could be possible to have an
increase in overall coherence in the shuffled data. For an example of such a
situation, it seems to me that this could occur if the relative phases of
the two signals are the same in both conditions but the overall spectral
power differed between the two conditions for both signals such both signals
had more power in condition one than in condition two. I haven’t tested this
with simulations, but it seems that in the case the coherence could be
higher in the trial shuffled case because of the induced correlation in
spectral power between the signals resulting from mixing the two conditions.

So following the line of thought from your suggestion above, in such a
situation in order to prevent the magnitude differences from affecting the
inference regarding the coherence difference between the two conditions,
would you say that it would be appropriate to also adjust the overall
magnitude of the pair of Fourier components in one condition to match the
overall magnitude of the components in the other conditions when doing the
permutation?

>
>> sessions obviously avoids this problem. For a test within a session, I
>> wonder if it would be possible to use a bootstrapping or jackknife
>> method combined with clustering to get a fair estimate of the
>> variability of the cluster level statistics and perform hypothesis
>> tests using that.
>
> Here the answer is simple. Permutation inference can only be used for
> comparing experimental conditions. However, I want to argue that, for
> investigating coherence, comparing experimental conditions is the only
> sensible thing one can do. The reason is that, for simple biophysical
> reasons, the null hypothesis of zero coherence within a single condition,
> will never be true, at least not in electrophysiological studies. This is
> because the potentials that are recorded in the two channels will always be
> affected by the physiological activity of common sources, simply because all
> potentials are volume conducted.
>

I agree completely. Just to clarify, I was suggesting using bootstrapping or
jackknifing to estimate the variance of the coherence of each measure
independently, and then use that to test the null hypothesis of no coherence
difference between the two conditions rather than the null hypothesis of
zero coherence.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.