[FieldTrip] Question of cluster based permutation test

[Stephen Politzer-Ahles]

I don't believe you automatically need to apply a correction just because
you're making several comparisons against the same standard. The important
issue is whether these multiple comparisons are testing the same research
question. If all of these are testing the same question, and getting a
significant result on any one of them would allow you to say "yes, my
theory was right!", then you would need to correct accordingly. (This is
the reason correction for multiple comparisons is needed with ERP data in
the first place; if you perform a t-test at every millisecond and getting a
significant result at any particular sample would let you say "look, I got
an MMN!", then that would cause a big multiple comparisons problem.)
However, if they are testing distinct questions then, as far as I
understand, correction isn't necessary.

---
Stephen Politzer-Ahles
The Hong Kong Polytechnic University
Department of Chinese and Bilingual Studies
http://www.polyu.edu.hk/cbs/sjpolit/




>
> Message: 1
> Date: Fri, 1 Jan 2021 16:34:58 +1100
> From: Arti Abhishek <mailtome.2113 at gmail.com>
> To: FieldTrip discussion list <fieldtrip at science.ru.nl>
> Subject: [FieldTrip] Question of cluster based permutation test
> Message-ID:
>         <CA+N_CJqXU+d70-CfUxLMai+=
> FaHCxds2hhZw+wS0SXrrps-ubQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear fieldtrip community,
>
> I have a question on cluster based permutation tests where one condition is
> compared with multiple other conditions. My experiment is a MMN
> multi-deviant paradigm where I have 4 deviants and one standard. I want to
> compare each deviant with the standard using the cluster stats to check the
> presence of MMN. Do I have to apply any correction as the same standard
> condition is compared multiple times?  My script is below:
>
> cfg                             = [];
>
> cfg.channel                     = {'EEG'};
>
> cfg.spmversion                  = 'spm12';
>
> cfg.latency                     = [-100 600];
>
> cfg.method                      = 'montecarlo';
>
> cfg.statistic                   = 'depsamplesT';
>
> cfg.correctm                    = 'cluster';
>
> cfg.clusteralpha                = 0.05;
>
> cfg.clusterstatistic            = 'maxsum';
>
> cfg.minnbchan                   = 2;
>
> cfg.tail                        = 0;
>
> cfg.clustertail                 = 0;
>
> cfg.alpha                       = 0.025;
>
> cfg.numrandomization            = 5000;
>
> cfg_neighb.layout               = lay;
>
> cfg_neighb.method               = 'triangulation';
>
> cfg.neighbours                  = ft_prepare_neighbours(cfg_neighb);
>
> %cfg.neighbours                 = neighbours;
>
> cfg.uvar                        = 1;
>
> cfg.ivar                        = 2;
>
> cfg.design                      = [1:19 1:19; ones(1,19) 2*ones(1,19)];
>
> Stat_MMN1                       = ft_timelockstatistics(cfg,
> All_Dev1{:},All_Std{:});
>
> Stat_MMN2                       = ft_timelockstatistics(cfg,
> All_Dev2{:},All_Std{:});
>
> Stat_MMN3                       = ft_timelockstatistics(cfg,
> All_Dev3{:},All_Std{:});
>
> Stat_MMN4                       = ft_timelockstatistics(cfg,
> All_Dev4{:},All_Std{:});
>
> Kind regards,
> Arti
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20210101/73de8f98/attachment.htm>