[FieldTrip] montecarlo statistic inversion

[Eelke Spaak]

Dear Jac,

The cfg.tail option indicates which tail of the monte carlo
distribution the observed clusterstat should be compared to, in order
to produce a p-value. With cfg.tail = 1, a right-tailed test is done;
i.e. the p-value reflects the proportion of permuted
"pseudo-observations" that *exceed* the observed clusterstat. With
cfg.tail = -1, a left-tailed test is done; i.e. the p-value reflects
the proportion of permuted "pseudo-observations" that are *lower than*
the observed clusterstat. cfg.tail = 0 basically does both a left- and
a right-tailed test, and returns the lowest p-value of the two (by
default corrected for the double test).

Left-tailed (or two-tailed) tests make sense when e.g. you are
comparing two conditions using a direct contrast (e.g. t, or raw
difference). When you are using an F-statistic (or something else that
quantifies evidence, like r^2 or so), only right-tailed tests make
sense. The question of whether your model explains the data *worse*
than random (the left-tailed question) is usually not of interest.

Cheers,
Eelke

On Tue, 23 Jul 2019 at 07:28, Jac Billington <J.Billington at leeds.ac.uk> wrote:
>
> Dear experts,
>
> I’d be grateful if someone could explain to me why there is an inversion in the statistics for the dependant sample montecarlo option - depending on the tail chosen
>
> I’m running the code below- a 0 or -1 tail seems to result in the lowest stat.stat value having the most significant p. A tail of ‘1’  flips this relationship (and is more in line with my result if I run a stats anova1 option). I’ve also extracted all the electrode results and placed them in another software (SPSS) and these results reflect the anova1  / montecarlo tail of ‘1’. See dropbox link below.
>
> Happy with suggested readings, but I really can’t place why this may be the case, or the best option to choose.
>
> Thank you. Jac
>
>
> https://www.dropbox.com/s/liq79s2ppzqpht2/m_carlo_comp.jpg?dl=0
>
> cfg = [];
> cfg.channel = {'all'};
> cfg.latency = time;
> cfg.frequency = frq;
> cfg.method = 'montecarlo' ; %%%%'stats' %
> cfg.statistic = 'depsamplesFmultivariate' %%%%%'anova1';
> cfg.correctm = 'no';
> % cfg.clusteralpha = 0.05;
> % cfg.clusterstatistic = 'maxsum';
> % cfg.minnbchan = 2;
> % prepare_neighbours determines what sensors may form clusters
> cfg_neighb.method = 'distance';
> cfg.neighbours = ft_prepare_neighbours(cfg_neighb, dataEEG);
> cfg.tail = 0; %%% (-1 1)
> cfg.clustertail = 0;
> cfg.alpha = 0.05;
> cfg.numrandomization = 5000;
> cfg.avgoverchan = 'no'
> cfg.avgovertime = 'yes'
> cfg.avgoverfreq = 'yes'
> subj = length(selsubs);
> design = zeros(2,4*subj);
> for i = 1:subj
> design(1,i) = i;
> end
> for i = 1:subj
> design(1,subj+i) = i;
> end
> for i = 1:subj
> design(1,subj+subj+i) = i;
> end
> for i = 1:subj
> design(1,subj+subj+subj+i) = i;
> end
>
> design(2,1:subj) = 1;
> design(2,subj+1:2*subj) = 2;
> design(2,subj+subj+1:3*subj) = 3;
> design(2,subj+subj+subj+1:4*subj) = 4;
> cfg.design = design
> cfg.uvar = 1;
> cfg.ivar = 2;
>
> %%%%stats option
> %cfg.design = design(2,:) %% stats option
> %cfg.ivar = 1;
>
> [stat] = ft_freqstatistics(cfg, frequencyR{1}{:}, frequencyR{2}{:},frequencyR{3}{:},frequencyR{4}{:})
>
>
>
> Dr Jac Billington
> Associate Professor - Perceptual Neuroscience
> School of Psychology, Rm G.06A
> University of Leeds
> Leeds, LS2 9JT
> Tel: +44(0)113 343 6686
>
> || Orcid Profile || Twitter ||
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://doi.org/10.1371/journal.pcbi.1002202