[FieldTrip] Bayesian "follow-up" on Cluster-Based Permutation Tests

[Eelke Spaak]

Dear Christine,

Bayes factors etc. are computed from the posterior distribution over
some model parameters (e.g. means of Gaussians in the case analogous
to the t-test). As the cluster-based permutation approach is
inherently non-parametric (i.e. it tests the exchangeability of data
beween conditions), I think it would be quite esoteric to try
something Bayesian with the cluster test. I think your best bet would
be to figure out *why* the reviewer wants this, and then come up with
an alternative answer that does not depend on Bayesian measures.

Of course, one could "zoom in" on the effect you found and compute
parametric Bayesian stats for that region of interest, but that would
constitute "double dipping" if you don't have an independent contrast.
In case you find evidence in favour of a null effect (one circumstance
under which reviewers might ask for Bayesian evidence), this approach
and result might still be valid (as it goes against the bias
introduced by the preselection).

Best,
Eelke

On 11 March 2018 at 18:21, Blume Christine <christine.blume at sbg.ac.at> wrote:
> Dear FT-Community,
>
> In the analysis of high-density EEG data for a recent manuscript
> (https://www.biorxiv.org/content/early/2017/12/06/187195) we have used the
> cluster-based permutation approach. While the reviewers commended the choice
> of this approach, one reviewer would like us to calculate a Bayesian measure
> in addition to the Monte Carlo p values. Does anyone have a recommendation
> how to best approach this, any "best practice" to share?
>
> It is quite easy to calculate a Bayes factor as a follow-up on classic
> t-tests for example (e.g. see here
> http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm).
> However, even though the permutation approach uses a t-value as a test
> statistic, it is not a "t-test"...
>
> Best,
> Christine
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip