[FieldTrip] Single-subject Monte Carlo PLV or WPLI test?

[Eric Maris]

Dear Matthew,


> I'd already tried clustering across only time/frequency and not across
> channel, but what I found was that the strongest channels "set the
> bar" for all the others, so to speak. I would see 2-3 strong channels
> with long significant durations reaching significance, and everything
> else would be silenced. Whereas with parametric stats, I had enough
> strong signals to detect changes under FDR and Bonferroni correction
> across a wide range of times and channels. Would z-scoring to
> compensate for electrode sensitivity differences have helped?

Are the channel-time-frequency-specific parametric p-values smaller than the
corresponding permutation p-values (for the same test statistic, of course)?
If that is the case, then I would be suspicious wrt to the validity of the
parametric test (i.c., its false alarm rate control). (This does not mean,
of course, that a smaller parametric p-value implies poor false alarm
control.)

It's not clear to me what you mean by z-scoring. Does this amount to a
linear transform of the dependent variable (the same one for the conditions
that are compared)?

> 
> I considered doing Monte Carlo stats for each channel independently,
> and then adjusting critical p-vals via FDR or Bonferroni, but for 100
> channels, I would need at least 20000 permutations just to have the
> Monte Carlo p-val resolution *approach* the adjusted Bonferroni
> p-vals, and would probably need more to be safe. Factor in several
> subjs and contrasts, and I computed my analysis would take a few weeks
> to run.
> 
> This was why I asked about a parametric method; while I'd prefer
> permutation methods, I fear the same problem will occur with my
> connectivity analysis. I know your focus is on permutation stats, but
> do you have any insight into how to proceed? Think I could generate a
> permutation distribution of the WPLI differences from a random
> sampling of electrodes and contrasts, and then, if they look
> sufficiently close to normal (or transformable via something like a
> log transform), use that as an argument for using t-tests if I have
> to?

If a parametric test solves my problem, then I will definitely use it.
However, for statistical tests outside the normal theory parametric
framework, it is typically a big challenge to come up with an appropriate
parametric reference distribution. I expect this to hold for the WPLI too.

Statistical testing of differences at the level of channel pairs (e.g.,
differences in coupling strength) is a big methodological challenge, for
many reasons (a huge multiple comparison problem, lack of specificity of the
coupling measure that is used for testing, difficulty of clustering in the
space formed by channel pairs). A discussion of these issues is being the
scope of the FT discussion list.

It is not clear to me what you mean by "generate a permutation distribution
of the WPLI differences from a random sampling of electrodes and contrasts".
Constructing a permutation distribution in a single subject study (which I
assume you are conducting, because you have ECoG data) involves random
partitioning of trials (and not electrodes and contrasts).


Best,

Eric



dr. Eric Maris
Donders Institute for Brain, Cognition and Behavior
Radboud University
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
T:+31 24 3612651
Mobile: 06 39584581
F:+31 24 3616066
mailto:e.maris at donders.ru.nl
http://www.nphyscog.com/



> 
> Thanks again,
> Matthew
> 
> 
> On Fri, Feb 18, 2011 at 4:36 AM, Eric Maris <e.maris at donders.ru.nl>
> wrote:
> > Hi Matthew,
> >
> >
> > Permutation inference is valid for comparing two experimental
> conditions
> > using ANY statistic. If your channels are more or less independent
> (no
> > common pick-up via volume conduction), then don't use the cluster-
> based
> > statistics (at least not for the spatial dimension; clustering along
> the
> > spectral dimension may still be wise).
> >
> > Best,
> >
> > Eric Maris
> >
> >
> >
> >
> >> -----Original Message-----
> >> From: fieldtrip-bounces at donders.ru.nl [mailto:fieldtrip-
> >> bounces at donders.ru.nl] On Behalf Of Matthew Davidson
> >> Sent: vrijdag 18 februari 2011 2:32
> >> To: fieldtrip at donders.ru.nl
> >> Subject: [FieldTrip] Single-subject Monte Carlo PLV or WPLI test?
> >>
> >> Hi everyone, I'm looking to see if there's an equivalent to the
> >> statfun_indepsamplesZcoh function, but for other connectivity
> >> measures, like PLV or WPLI. I need to do several single-subject,
> >> between-trials analyses of differences between two conditions. Since
> >> my data are intracranial EEG, there's no meaningful group test I
> could
> >> use, which I gather is how many people make inferences on
> connectivity
> >> measures. So, has anyone implemented this, or something like it? Am
> I
> >> missing something obvious in how to do this?
> >>
> >> If I implement it myself, I guess I should randomly partition the
> >> trials, compute the WPLIs of the two groups, take the difference,
> >> compute the max cluster size, and build a permutation distribution
> of
> >> the max cluster WPLI difference. Is that generally correct? Should I
> >> use jackknife variance to transform them into Z-scores for
> >> thresholding?
> >>
> >> Alternatively, if I wanted to do this parametrically, how should I
> do
> >> that? (I ask because Monte Carlo methods w/clustering haven't worked
> >> as well as analytic methods on intracranial data where the
> electrodes
> >> are more independent than in MEG or scalp EEG.) What's the proper
> >> reference distribution of differences in these bounded connectivity
> >> metrics? Do I just compute the jackknife variance, and then do a
> mass
> >> univariate t-test on the connectivity measures (for a single
> electrode
> >> pair and freq bin)?
> >>
> >> Thanks for any insight or advice you might have,
> >> Matthew Davidson
> >> _______________________________________________
> >> fieldtrip mailing list
> >> fieldtrip at donders.ru.nl
> >> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> >
> > _______________________________________________
> > fieldtrip mailing list
> > fieldtrip at donders.ru.nl
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> >
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip