[FieldTrip] interactions

Eric Maris e.maris at psych.ru.nl
Sun Jan 26 10:08:35 CET 2014

Hi Steve and Josh,

Josh writes

> > labels. I'm sure there's a proof somewhere for why this doesn't work,
> > and it would be great to see it.

In general, questions like these are very hard to answer satisfactorily on a 
discussion list. It is dealt with much more easily in person, say at one of 
the Fieldtrip courses. However, let me give it a try.

To prove that something does not work it suffices to produces a single 
example that shows the contrary.

Try the following:

Generate random data in a 2-by-2 between-subjects design (say, normally 
distributed within every cell). Add large main effects (relative to the 
within-cell variance; say, MS_beween 50 times larger than MS_within) and no 
interaction effect. Take a small number of subjects (say, 5 per cell). Now, 
calculate a permutation p-value for the interaction-effect F-statistic by 
permuting across all 4 cells. Do this for a large number of simulated data 
set. My prediction is that, on average, the F-statistic p-value is less than 
0.05, which it should be (because there is no interaction effect).

I have not run this simulation study myself. Let me know if it does not 
produce the predicted result. (I cannot guarantee that I'm not missing 
something when producing this recipe.)



> -----Original Message-----
> From: Stephen Politzer-Ahles [mailto:politzerahless at gmail.com]
> Sent: zondag 26 januari 2014 8:25
> To: fieldtrip at science.ru.nl
> Subject: Re: [FieldTrip] interactions
> Hi Josh,
> Have you seen this [admittedly pretty old now] message from the
> archives: http://mailman.science.ru.nl/pipermail/fieldtrip/2011-
> January/003447.html
> ? My understanding was that it is ok to test interactions in within-
> subjects designs, and that you could do it by faking a dataset that
> represents the interaction (step 3 in that message) and then doing a
> dependent samples t-test. I had never heard before that interactions
> can't be tested in a within-subjects design, but also it's been a long
> time since I've looked at this issue--I'd definitely be interested to
> hear if this is no longer the recommended way to test interactions. I
> have seen messages saying that it doesn't work for between-subjects
> designs (e.g.
> http://mailman.science.ru.nl/pipermail/fieldtrip/2011-
> September/004244.html),
> but I'm not sure if that's still current. Hopefully someone on the list
> can offer more insight about the second question.
> Best,
> Steve
> >
> > Message: 2
> > Date: Fri, 24 Jan 2014 10:54:10 -0500
> > From: Joshua Hartshorne <jkhartshorne at gmail.com>
> > To: fieldtrip at science.ru.nl
> > Subject: [FieldTrip] interactions
> > Message-ID:
> >
> > <CA+3amhe+x4+TNUY1tf0aXe-cf-AB1kTE+ZHTpuRJxNQ=bNioUQ at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hi List!
> >
> > I have seen around a dozen comments in the archives that interactions
> > can't be tested by permutation for within-subject designs. I haven't
> > been able to find a thread that explains why not. It seems like in a
> > 2x2 design, you could still pick one of the conditions and permute
> the
> > labels. I'm sure there's a proof somewhere for why this doesn't work,
> > and it would be great to see it.
> >
> > Similarly, for the mixed design, why permute the between-subject
> labels?
> > Why not permute the within-subject labels instead? Actually, why not
> > do both? I follow the reasoning why permuting both is overkill, but
> > not why it's wrong.
> >
> > If someone could explain, it would be much appreciated. Knowing what
> > to do is good, but it would be even better to understand why.
> >
> > Thanks,
> > Josh
> > -------------- next part -------------- An HTML attachment was
> > scrubbed...
> > URL:
> >
> <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20140124/
> b885cb4a/attachment-0001.html>
> >

More information about the fieldtrip mailing list