[FieldTrip] Cluster-Based Permutation Tests with Mixed Design 2x2
Diego Lozano-Soldevilla
dlozanosoldevilla at gmail.com
Thu Oct 8 17:19:00 CEST 2020
Hi Ascen,
Take a look at this tutorial
https://www.fieldtriptoolbox.org/workshop/madrid2019/tutorial_stats/#4-compute-a-2x2-interaction
I hope that helps
Diego
On Thu, 8 Oct 2020 at 17:14, Pagan, Ascension (Dr.) <appc1 at leicester.ac.uk>
wrote:
> Hi Arno,
>
> Thank you for your answer. I was aware of the fieldtrip webpage about the
> interaction effect using cluster-based permutation tests, but I'm still
> unsure if I should run a single cfg.statistic with a mixed design or two
> separate tests? I'd be grateful for any suggestion or advice.
>
> Best wishes,
> Ascen
>
> -----Original Message-----
> From: fieldtrip <fieldtrip-bounces at science.ru.nl> On Behalf Of Arnaud
> Delorme
> Sent: 06 October 2020 01:48
> To: FieldTrip discussion list <fieldtrip at science.ru.nl>
> Subject: Re: [FieldTrip] Cluster-Based Permutation Tests with Mixed Design
> 2x2
>
> Hi Ascen,
>
> Robert (Oostenveld) pointed me recently to the page below. You should test
> for differences in the differences.
>
>
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.fieldtriptoolbox.org%2Ffaq%2Fhow_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests%2F&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=vCzK44xxjvPKwwuGtSMoJ1BUOiFaFSAThEA86cBmvhc%3D&reserved=0
>
> If it does not fulfill your need, then the only alternative I know of is
> to use LIMO (
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FLIMO-EEG-Toolbox%2Flimo_tools%2Fwiki&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=6poH22QgPXRegqzOXTBTx6zXt8Qg4BaAHotERCYUqpc%3D&reserved=0)
> which allows you to create arbitrarily complex designs and perform cluster
> based permutation tests.
>
> Best wishes,
>
> Arno
>
> > On Oct 5, 2020, at 2:12 AM, Pagan, Ascension (Dr.) <
> appc1 at leicester.ac.uk> wrote:
> >
> >
> > Dear All,
> > I have a mixed design (2 conditions x 2 groups), with 44 participants in
> group1, and 30 participants in group2, and I am planning to run
> cluster-based permutation tests on ERP data to test:
> > • The main effect of condition (cond1 vs. cond2)
> > • The main effect of group (group1 vs. group2)
> > • The effect of condition in group1 (group1 cond1 vs. group1 cond2)
> > • The effect of condition in group2 (group2 cond1 vs. group2 cond2)
> > • The interaction effect between condition and group (group1 cond1
> -
> > group1 cond2) vs. (group2 cond1 - group2 cond2)
> >
> > I have used ft_timelockanalysis to create data structures for COND1_ft,
> COND2_ft, group1_ft, group2_ft; COND1group1_ft, COND2group1_ft;
> COND1group2_ft, COND2group2_ft but I am having some doubts regarding design
> matrix and cfg.statistic. I have read the suggestions and comments in the
> discussion list, but I’m still not sure if I’m setting them up correctly.
> Below, I specify what I’m trying to do and how I have done it so far:
> >
> > First, my understanding from a message dated some years ago
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fpipermail%2Ffieldtrip%2F2008-March%2F014371.html&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=e2vl%2FmYSae0D6GKCckXqL7JpKWoHOUcLxfifEGuJI%2FE%3D&reserved=0
> is that to perform permutation tests on a mixed design we should have a
> design matrix with Row 1 (condition), Row 2 (subject number), and Row 3
> (group). In my case:
> >
> > sbj = 74 %(i.e., 44 sbjs in group1, 30 subjs in group2); design =
> > zeros(3, 2*sbj);
> >
> > %Set up Row 1
> > design(1, 1:sbj) = 1; %(Row 1 with cond1 from 1:74 column)
> > design(1,sbj+1:2*sbj) = 2; %(Row 1 with cond2 from 75:148 column)
> >
> > %Set up Row 2
> > for i= 1:sbj
> > design(2,i) = i; %(Row 2 with sbj number from 1:74 column)
> > end for i=1:sbj
> > design(2,sbj+i)=i; %(Row 2 with sbj number from 75:148
> > column) end
> >
> > %Set up Row 3
> > design(3, [1:44 75:118]) = 1; %(Row 3 with group1 from 1:44 and 75:118
> > column) design(3, [45:74 119:sbj*2]) = 2; %(Row 3 with group2 from
> > 45:74 and 119:148 column)
> >
> > cfg.design = design;
> >
> > With the following configuration:
> > cfg.method = 'montecarlo';
> > cfg.ivar = 1 %(row with independent variable, i.e., condition number)
> > cfg.uvar = 2 %(row with unit variables, i.e., subject number) cfg.wvar
> > = 3 %(row with within-cell variable, i.e., group number)
> >
> > And, with the rest of the configuration set up as follows:
> > cfg.minnbchan = 2;
> > cfg.neighbours = neighbours;
> > cfg.alpha = 0.05;
> > cfg.tail = 0; %two-sided test
> > cfg.correcttail = 'alpha'; %conceptually equivalent to performing a
> > Bonferroni correction for the two tails cfg.numrandomization = 10000;
> > cfg.correctm = 'cluster'; cfg.clusterstatistic = 'maxsum';
> > cfg.clusteralpha = 0.05; cfg.clustertail = 0;
> >
> > However, I am not sure about the design matrix because when I try to run
> the previous code, I get the following error messages:
> > Error using resampledesign (line 153)
> > the design matrix variables should be constant within a block
> >
> > Error in ft_statistics_montecarlo (line 215) resample =
> > resampledesign(cfg, design);
> >
> > Error in ft_timelockstatistics (line 182)
> > [stat, cfg] = statmethod(cfg, dat, design);
> >
> > Second, I am not sure which cfg.statistic I should be using in the mixed
> design. If I use subject number as my unit of observation (UO), then, in
> one case, UO is assigned to only one of two groups (between UO-design;
> independent samples), but at the same time, every UO is assigned to two
> experimental conditions (within UO-design; dependent samples) (
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.fieldtriptoolbox.org%2Ftutorial%2Fcluster_permutation_timelock%2F%3Fs&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=j4S64%2BAYOtFy%2BGB9RiHFGWP7GebXQTses84rSjO2xdk%3D&reserved=0).
>
> >
> > So, my question is: Is there any way to use a single cfg.statistic with
> a mixed design or should I run two separate tests: one with cfg.statistic =
> 'depsamplesT'; to test the main effect of condition (cond1 vs. cond2), the
> effect of condition in group1 (group1 cond1 vs. group1 cond2), and the
> effect of condition in group2 (group2 cond1 vs. group2 cond2), and another
> test cfg.statistic = 'indepsamplesT'; to test the main effect of group
> (group1 vs. group2)?
> >
> > After this, I would then test:
> > %Main effect of Condition
> > [statCOND] = ft_timelockstatistics(cfg, COND1_ft{:}, COND2_ft{:});
> >
> > %Main effect of Group
> > [statGROUP] = ft_timelockstatistics(cfg, group1_ft{:}, group2_ft{:});
> >
> > %Main Effect of Condition in Group1
> > [statCONDgroup1] = ft_timelockstatistics(cfg,
> > COND1group1_ft{:},COND2group1_ft{:});
> >
> > %Main Effect of Condition in Group2
> > [statCONDgroup2] = ft_timelockstatistics(cfg,
> > COND1group2_ft{:},COND2group2_ft{:});
> >
> > Finally, to test the interaction effect ((group1 cond1 - group1 cond2)
> vs. (group2 cond1 - group2 cond2)), I was planning to use cfg.statistic =
> 'indepsamplesT’
> > (as in (
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.fieldtriptoolbox.org%2Ffaq%2Fhow_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests%2F&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=vCzK44xxjvPKwwuGtSMoJ1BUOiFaFSAThEA86cBmvhc%3D&reserved=0)
> and then, first, calculate the differences as follows:
> > cfg = [];
> > cfg.operation = 'subtract';
> > cfg.parameter = 'avg';
> > for i =1:length(COND1group1_ft)
> > CONDgroup1DIFF_ft{i} = ft_math(cfg, COND1group1_ft{i},
> > COND2group1_ft{i}); end for i = 1:length(COND1group2_ft)
> > CONDgroup2DIFF_ft{i} = ft_math(cfg, COND1group2_ft{i},
> > COND2group2_ft{i}); end
> >
> > Second, set up the configuration as above and run the following:
> > %Test Interaction Effect between Condition and Group [statCONDbyGROUP]
> > =
> > ft_timelockstatistics(cfg,CONDgroup1DIFF_ft{:},CONDgroup2DIFF_ft{:});
> >
> >
> > Apologise for the long email, but I hope it makes sense. I’d be grateful
> to any advice or suggestion about the design matrix and cfg.statistic.
> >
> > Best wishes,
> > Ascen
> >
> >
> > --------------------------------
> > Dr Ascensión Pagán
> > Lecturer, Department of Psychology
> > Faculty of Health and Life Sciences
> > Northumbria University
> > Newcastle upon Tyne
> > NE1 8ST
> > UK
> > ascension.pagan at northumbria.ac.uk
> >
> >
> > This message is intended solely for the addressee and may contain
> confidential and/or legally privileged information. Any use, disclosure or
> reproduction without the sender’s explicit consent is unauthorised and may
> be unlawful. If you have received this message in error, please notify
> Northumbria University immediately and permanently delete it. Any views or
> opinions expressed in this message are solely those of the author and do
> not necessarily represent those of the University. Northumbria University
> email is provided by Microsoft Office365 and is hosted within the EEA,
> although some information may be replicated globally for backup purposes.
> The University cannot guarantee that this message or any attachment is
> virus free or has not been intercepted and/or amended.
> > _______________________________________________
> > fieldtrip mailing list
> > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail
> > man.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=02%7C01%7C
> > appc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a3
> > 1d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=7F2iqHI
> > 2Kd7kFVBIPEUCKBZywOrz4Y19gqgc7rphP3Q%3D&reserved=0
> > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.
> > org%2F10.1371%2Fjournal.pcbi.1002202&data=02%7C01%7Cappc1%40leices
> > ter.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce827
> > 4afe853d9%7C0%7C1%7C637375427464343217&sdata=voQgeeTGZkpUbMa%2FWL2
> > JcPzYdxSJuTV0VNApCUXTiJo%3D&reserved=0
>
>
> _______________________________________________
> fieldtrip mailing list
>
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=7F2iqHI2Kd7kFVBIPEUCKBZywOrz4Y19gqgc7rphP3Q%3D&reserved=0
>
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=02%7C01%7Cappc1%40leicester.ac.uk%7Cc8053ca84f11430452a908d8699305a3%7Caebecd6a31d44b0195ce8274afe853d9%7C0%7C1%7C637375427464343217&sdata=voQgeeTGZkpUbMa%2FWL2JcPzYdxSJuTV0VNApCUXTiJo%3D&reserved=0
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20201008/84e4b21d/attachment.htm>
More information about the fieldtrip
mailing list