Permutaion (of residuals) and factorial designs

[Eric Maris]

Dear Michael,


> this might sound like nitpicking but, we all routinely seem to analyse
> the interaction of a factorial design using permutation testing. The
> example is this: we have two experimental conditions (that we want to
> compare) and record task and baseline intervals in each. Clearly this
> is a 2x2 design (task/base and cond1/cond2 are the respective levels of
> the two factors). What we all do to deal with this is that we compute
> residuals - either by subtracting the baseline values or normalizing to
> them and then do a (restricted) permutation between the conditions on
> these task-base residuals. We are interested in the interaction between
> the task/base factor and the cond factor.
> 
> Anything wrong here or anything particular about this case that saves
> us from the fundamental difficulties of interaction testing?


This is a very sensible remark that forces me to be explicit about when interaction effect null hypotheses are problematic for permutation tests and when not.

What you describe is a mixed between-within unit-of-observation (UO) design. The UOs are trials and there is one between-UO independent variable (the two task conditions) and within-UO independent variable (baseline-versus-activation). In this type of design, permutation tests can be used without problems to test the interaction between the independent variables. The way you do this is exactly as you have described: perform trial-wise subtraction/normalization to construct a new dependent variable that is subsequently compared between the two task conditions, as in a regular between-UO study.

This approach does not work anymore in a two-factorial design in which both independent variables are manipulated between-UO. For example, this would be the case in a single subject study with the following independent variables: (1) attend left versus attend right (SIDE), and (2) attend visual versus attend auditory (MODALITY). It cannot be ruled out that there is an interest in the null hypothesis of no interaction between SIDE and MODALITY. (For this example, I find it hard to produce a convincing physiological story that produces this null hypothesis, but this does not have to be always the case.) I do not see how to test this null hypothesis using a permutation test that involves random permutation over the four cells in this two-factorial design.


Best,

Eric 







> 
> Michael
> 
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: "Eric Maris" <e.maris at DONDERS.RU.NL>
> Gesendet: Sep 29, 2010 11:18:06 AM
> An: FIELDTRIP at NIC.SURFNET.NL
> Betreff: Re: [FIELDTRIP] Coherence differences and factorial designs
> 
> >Dear Suresh,
> >
> >
> >
> >>     In a fixed effects context I have been obtaining coherence
> >> estimates. I have been reading Maris et al 2007 and the theory there
> >> describes how to test between two different conditions I would like
> >> to extend the theory in that paper (2.7.1) to k sample (one factor
> eg
> >> 1 x 3 ANOVA) and two-way (e.,g. 2 x 2 designs). I was wondering if
> >> anyone had attempted such a thing if it can be done, and in
> >> particular how one might go about constructing an apprppriate test
> >> statistic and surrogate distribution? Prior implementation in
> >> fieldtrip isnt needed its more the theory behind it I am asking
> about
> >
> >Statistical comparison of coherence estimates in k samples is
> discussed
> >by Amjad et al (2007) in J. Neurosc. Methods.
> >
> >In the permutation framework there is no analogue of the factorial
> >ANOVA (involving both main and interaction effects) for the simple
> >reason that the interaction null hypothesis cannot be tested in the
> permutation framework.
> >There is at least one thread in the Fieldtrip Discussion list that
> >deals with this issue. However, it is possible to test multiple
> >conditional null hypotheses (main effect of one factor separately for
> >each of the levels of another factor) and this comes close to an
> interaction effect test.
> >
> >
> >Good luck,
> >
> >Eric Maris
> >
> >
> >
> >
> >
> >
> >> Thanks for your help,
> >> Dr Suresh Muthukumaraswamy
> >>
> >> Suresh Muthukumaraswamy, PhD
> >> CUBRIC
> >> Cardiff University
> >> Park Place
> >> Cardiff, CF10 3AT
> >> United Kingdom
> >> email: sdmuthu at cardiff.ac.uk
> >> Phone: +44 (0)29 2087 0354
> >>
> http://www.cf.ac.uk/psych/contactsandpeople/researchstaff/muthukumara
> >> sw
> >> amy-suresh-dr-overview_new.html
> >>
> >> ----------------------------------
> >> The aim of this list is to facilitate the discussion between users
> of
> >> the FieldTrip  toolbox, to share experiences and to discuss new
> ideas
> >> for MEG and EEG analysis. See also
> >> http://listserv.surfnet.nl/archives/fieldtrip.html and
> >> http://www.ru.nl/neuroimaging/fieldtrip.
> >
> >----------------------------------
> >The aim of this list is to facilitate the discussion between users of
> the FieldTrip  toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis. See also
> http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.
> 
> ----------------------------------
> The aim of this list is to facilitate the discussion between users of
> the FieldTrip  toolbox, to share experiences and to discuss new ideas
> for MEG and EEG analysis. See also
> http://listserv.surfnet.nl/archives/fieldtrip.html and
> http://www.ru.nl/neuroimaging/fieldtrip.

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.