[FieldTrip] Interactions

[Joshua Hartshorne]

Hi Eric,

I tried effect sizes of 20 and 4 subjects per condition. With 200
iterations, the false alarm rate was *lower* for permutations (.02) than
for a standard ANOVA (.025). For good measure, I also tried with effect
sizes of 10, with the same result (.02 vs. .03).

I still don't see what the problem would be. I always understood that the
main thing permutation tests need is exchangeability, and it doesn't get
much more exchangeable than in a between/between design. I'm reading the
original work now, though, so maybe I'll discover something.

Josh

On Tue, Feb 4, 2014 at 6:16 AM, Eric Maris <e.maris at psych.ru.nl> wrote:

> Hi Josh,
>
>
>
> You may have found something that is worth exploring. However, it will be
> challenge to provide a formal proof of the fact that the permutation
> analysis provides false alarm rate control under the null hypothesis of no
> interaction effect. The crucial design here is the full between-subjects
> design, because for the other designs a valid permutation analysis exists
> (as demonstrated by formal proof).
>
>
>
> I would embark on a testing-the-limits simulation study:
>
>
>
> *between/between*. Normally-distributed data in each of 4 cells. The
> effect size for e1 and e2 were each set to 1, with a SD of 1. The
> interaction (if present) was .75 with an SD of 1. There were 40 subjects
> per cell.
>
>
>
> Set the effect sizes to 20, reduce the number of subjects to 5 per cell,
> and simulate data without an interaction effect. I'm curious how the
> simulated false alarm rate of the permutation test looks like.
>
>
>
> (After this email, I will not continue this discussion any further. It is
> becoming a scientific project; interesting though ...)
>
>
>
> Best,
>
>
>
> Eric
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20140204/9b4bdc73/attachment-0002.html>