[FieldTrip] 2x3 anova in cluster analysis

[David Groppe]

Hi Marcin,
  You could use a permutation test based on a rank correlation
statistic (e.g., Kendall's tau or Spearman's rho) to test for such a
monotonic relationship.  I do not know if FieldTrip currently
implements this or not though.
      cheers,
          -D

On Thu, Dec 8, 2011 at 11:33 AM, Marcin Leszczynski
<m.leszczynski.m at googlemail.com> wrote:
> Hi Jan-Mathijs and Fieldtrippers,
>
> I have a follow up question refering to statistical inference about
> monotonic trends in the permutation framework.
>
> I have four conditions A, B, C, D and I would like to test whether there is
> a monotonic change in the ERP/TF among the conditions (i.e. A<B<C<D but
> possibly also A < B <= C < D). I guess one way would be to run multiple t
> tests. However it seems a bit unconvenient as one needs to correct for
> multiple comparisons. Furtheremore, the second case (A < B <= C < D) might
> not be well captured.
>
> I found a thread from 31 Aug 2011 ("[FieldTrip] depsamplesregrT help" by
> Jonas Obleser) suggesting that depsamplesregrT will be the way to go.
> However, your reply to Nico somehow destroyed my confidence. If I got the
> reply correctly this is not that easy as I would be testing a particular
> linear model which is a bit of a problem. Thus, the question:
>
> How would you suggest testing for such a monotonic effect in the permutation
> framework?
>
> Best,
> Marcin
>
>>
>>
>> 2011/12/5 jan-mathijs schoffelen <jan.schoffelen at donders.ru.nl>
>>>
>>> Hi Stan et al,
>>>
>>> Actually FieldTrip allows in a very straightforward way to plug-in one's
>>> favourite statistic. The idea is the following: when specifying
>>> cfg.statistic you need to specify a string that under the hood points to a
>>> function, e.g. 'indepsamplesT' eventually causes a function to be called,
>>> answering to the aptly chosen name: 'statfun_indepsamplesT' which is doing
>>> the legwork. As long as the statfun_younameit has properly defined input and
>>> output arguments, you can let it do whatever you want. FieldTrip contains a
>>> bunch of 'statfuns' in the statfun directory but there's no reason not to
>>> implement any yourself (and ideally contribute them to the repository).
>>> Another discussion altogether is the question whether the test statistic
>>> of interest is an appropriate one that allows for statistical inference
>>> using the permutation framework. Over the past years there have been various
>>> threads on this mailing list regarding the possibility to test for
>>> interactions using a permutation test. You can look this up in the archive.
>>> Strictly speaking this is statistically not possible this way. The
>>> permutation-framework tests the null-hypothesis of exchangeability of data
>>> across allocated conditions, whereas when one wants to test for an
>>> interaction effect tests one tests for a particular linear (parametric)
>>> model of the dependent variables explaining variance in the dependent
>>> variable. Inferring that exchangeability across conditions is unlikely (i.e.
>>> obtaining a small p-value through permutation) does not necessarily lead to
>>> the conclusion that there is an actual interaction effect. Though opinions
>>> among the statisticians about this seem to vary, there may be a way out: one
>>> could either use bootstrapping to obtain confidence intervals for the
>>> interaction F under the null-hypothesis. The recipe for this would be to do
>>> a cell-specific demeaning of the data to impose the null-hypothesis of no
>>> interaction, and then through bootstrap resampling obtain a reference
>>> distribution of the interaction F. Alternatively, although I haven't thought
>>> this one through for any case other than a 2x2 anova, one could reduce the
>>> interaction effect to a two-condition contrast where observations belonging
>>> to the 'main diagonal' of the 2x2 conditions design are labeled as condition
>>> 1, and the observations belonging to the other diagonal are labeled as
>>> condition 2. One could then proceed with using a T-statistic as a
>>> descriptive statistic across these two 'conditions' on which inference can
>>> be done.
>>>
>>> Cheers and a happy Sinterklaas to you all,
>>>
>>> Jan-Mathijs
>>>
>>>
>>> On Dec 5, 2011, at 8:17 PM, Stanley Klein wrote:
>>>
>>> Nico,
>>>
>>> Your question gives me the chance to ask something I've been curious
>>> about. It seems to me that one of the nifty things about permutation cluster
>>> analysis is that it allows you to do anything that seems reasonable as long
>>> you decide on the process ahead of time and don't do any future tweaking
>>> other than fixing coding errors. I'm curious whether the Fieldtrip code make
>>> it easy to insert new statistics modules.
>>>
>>> I presume your actual question had to do with whether Fieldtrip already
>>> has the possibility of doing that 2x3 ANOVA.  I suspect it doesn't have that
>>> capability. However, it would be nice if the permutation cluster analysis
>>> could be made sufficiently modular that one could make use of all the
>>> complicated clustering and permuting and all, but enable the user to
>>> substitute their own statistical method.  I haven't actually watched my
>>> students doing the nitty-gritty use of Fieldtrip's version of the
>>> permutation test (other than knowing that is is very nice and well
>>> documented) so I don't know how easy it is to stick in one's own favorite
>>> statistic, but I'm hoping it is easy.
>>>
>>> Stan
>>>
>>> On Mon, Dec 5, 2011 at 4:58 AM, Nico Alexander Willi Kremers
>>> <nkremers at uni-bonn.de> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I followed the discussion about implementing a 2x2 within subjects
>>>> design into a cluster statistic with much interest. I want to implement a
>>>> 2x3 repeated measure anova in a cluster analysis using fieldtrip. I think
>>>> for this type of analysis subracting two conditions from each other and then
>>>> calculate a ttest is not appropriate. Is there another way of calculating
>>>> the interaction effect between the two factors and generate clusters for
>>>> real and permutation data? What specifications must be set and how?
>>>>
>>>> Thank you very much for your answer,
>>>>
>>>> Nico
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>>
>>> Jan-Mathijs Schoffelen, MD PhD
>>>
>>> Donders Institute for Brain, Cognition and Behaviour,
>>> Centre for Cognitive Neuroimaging,
>>> Radboud University Nijmegen, The Netherlands
>>>
>>> Max Planck Institute for Psycholinguistics,
>>> Nijmegen, The Netherlands
>>>
>>> J.Schoffelen at donders.ru.nl
>>> Telephone: +31-24-3614793
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>
>>
>>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip



-- 
David Groppe, Ph.D.
Postdoctoral Researcher
North Shore LIJ Health System
New Hyde Park, New York
http://www.cogsci.ucsd.edu/~dgroppe/