[FieldTrip] Bayesian "follow-up" on Cluster-Based Permutation Tests

Eugenio Abela abela.eugenio at gmail.com
Mon Mar 12 11:10:55 CET 2018

Hi Christine, 

If you want to try something Bayesian and only  if you have strong a priori hypotheses for a particular effect in a specific data window of interest, you could extract values from that window and do Bayesian analyses in JASP (https://jasp-stats.org), which is really easy to use and has  a range Bayesian alternatives for classical F- and t-tests available. 

Alternatively, but this is probably much more complicated and time-consuming, you could do full Bayesian analyses with SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12/), which allows you to calculate posterior probabilities for scalp, scalp x frequency or scalp x time maps. This is a great way of assessing evidence in your data, but I think it’s less popular than permutation tests. You can convert between Fieldtrip and SPM formats using spm_eeg_ft2spm.m

I totally agree with Eelke that doing Bayesian tests post-hoc, after you obtained convincing results with permutations, seems like unnecessary methodological flourish. Yet it’s often hard to argue with reviewers without compromising a bit. You might want to use one of the above e.g. for your most important analysis and show in the rebuttal that it doesn’t alter your conclusions (hopefully!). This should convince them that results are robust to methodological choices, and exempt you from having to recalculate the whole thing again.

Hope that helps


On 12 Mar 2018, at 09:19, Eelke Spaak <e.spaak at donders.ru.nl> wrote:

Dear Christine,

Bayes factors etc. are computed from the posterior distribution over
some model parameters (e.g. means of Gaussians in the case analogous
to the t-test). As the cluster-based permutation approach is
inherently non-parametric (i.e. it tests the exchangeability of data
beween conditions), I think it would be quite esoteric to try
something Bayesian with the cluster test. I think your best bet would
be to figure out *why* the reviewer wants this, and then come up with
an alternative answer that does not depend on Bayesian measures.

Of course, one could "zoom in" on the effect you found and compute
parametric Bayesian stats for that region of interest, but that would
constitute "double dipping" if you don't have an independent contrast.
In case you find evidence in favour of a null effect (one circumstance
under which reviewers might ask for Bayesian evidence), this approach
and result might still be valid (as it goes against the bias
introduced by the preselection).


On 11 March 2018 at 18:21, Blume Christine <christine.blume at sbg.ac.at> wrote:
> Dear FT-Community,
> In the analysis of high-density EEG data for a recent manuscript
> (https://www.biorxiv.org/content/early/2017/12/06/187195) we have used the
> cluster-based permutation approach. While the reviewers commended the choice
> of this approach, one reviewer would like us to calculate a Bayesian measure
> in addition to the Monte Carlo p values. Does anyone have a recommendation
> how to best approach this, any "best practice" to share?
> It is quite easy to calculate a Bayes factor as a follow-up on classic
> t-tests for example (e.g. see here
> http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm).
> However, even though the permutation approach uses a t-value as a test
> statistic, it is not a "t-test"...
> Best,
> Christine
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
fieldtrip mailing list
fieldtrip at donders.ru.nl

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20180312/ee9fd608/attachment-0002.html>

More information about the fieldtrip mailing list