[FieldTrip] One-sample monte-carlo permutation statistics in Fieldtrip?

Wed May 8 12:04:03 CEST 2019

Dear all,
I don't want to hijack the discussion but I have a related problem to 
the situation Lars and Julian described (just that my actual data is n=1).

I have a single time series that I derive from thousands of trials (a 
rather complex measure requiring many samples) and a large number of 
surrogate time series I created to perform statistics. I estimate power 
at different frequencies in both the actual and surrogate data using 
mtmfft. Now I would like to test for power differences between actual 
and surrogate data.

Seems to me this is equivalent to a one-sample t-test in which you test 
the surrogate data against a different value for each frequency (or in 
Lars' case: test your data against an array of zeros).

I tried to use indepsamplesT and do a cluster permutation test with my 
actual time series as one condition and the 1000 surrogate time series 
as the other condition, but it does not execute properly. The reason is 
the way variance is calculated, which leads to NaNs (due to division / 
0) for the single-sample condition. Using Matlab's var function instead 
(ft_statfun_indepsamplesT, line 92 vs. 94) yields 0 as the variance for 
that condition and the end result seems to be correct (at least it finds 
the clusters I expected to find).

Or is there a statistical problem with that solution?

Thanks and all the best,
Jens

Lars Costers wrote on 02/05/2019 15:42:
> Hi,
>
> Thanks for your suggestion Julian but I¿m afraid that your approach is 
> not applicable for my data/hypotheses...
>
> I¿m wondering whether I could generate zero data and use that in 
> Fieldtrip's dependent t-test (so cfg.statistics = ¿depsamplesT¿) 
> together with my real data in my permutations. I think this will (in 
> theory) be the same as doing a one-sample t-test and flipping the sign 
> of subjects because the real data ("condition 1") and zero data 
> ("condition 2") will be randomly shuffled within subjects. So for some 
> subjects 'data - 0¿ (equal to multiplying by 1) will be tested and for 
> other subjects it will be ¿0 - data¿ ( equal to multiplying by -1)).
>
> In Nichols & Holmes (2001, 
> https://www.fil.ion.ucl.ac.uk/spm/doc/papers/NicholsHolmes.pdf) they 
> state the following for testing H0 = distribution centred to zero:
>
>     We consider subject labels of ¿+1¿ and ¿-1¿ indicating an
>     unflipped or flipped sign of the data. Under the null hypothesis,
>     we have data symmetric about zero, and hence for a particular
>     subject the sign of the observed data can be flipped without
>     altering its distribution. With exchangeable subjects, we can flip
>     the signs of any or all subjects¿ data and the joint distribution
>     of all of the data will remain unchanged. 
>
>
> Therefore, I think this can be done, but I'm not 100% sure..
>
> Does anybody have advice on this matter?
>
> Best,
> Lars
>
>> On 2 May 2019, at 12:31, Julian Keil <julian.keil at gmail.com 
>> <mailto:julian.keil at gmail.com>> wrote:
>>
>> Hi Lars,
>>
>> we recently faced a similar problem (see here: 
>> https://www.nature.com/articles/s41598-019-42380-x). Our solution was 
>> to create ¿dummy¿ data based on the actual data and use these in the 
>> comparison.
>> In short, this relates to Eric Maris¿ statement about the null 
>> hypothesis: In our case, the H0 was ¿the regression weights are 
>> independent of the stimulus category¿, so we test our empirically 
>> found regression weights against random regression weights. Please 
>> note that we don¿t have a formal proof that his is a valid approach.
>>
>> Unfortunately, I don¿t think this is directly applicable to your 
>> situation, as it sounds like you only have one category, but maybe 
>> there is a way for you to come up with dummy data?
>> Again, please note that there might be problems with our approach we 
>> didn¿t consider.
>>
>> Best,
>>
>> Julian
>>
>>
>>> Am 02.05.2019 um 12:05 schrieb Lars Costers <larscosters at gmail.com>:
>>>
>>> Hi all,
>>>
>>> I was wondering whether there are any one-sample permutation 
>>> statistics to test for a difference from zero implemented in 
>>> Fieldtrip, ideally to work with monte-carlo permutations? Didn¿t 
>>> find any standard of 'statfun¿ options in the documentation.
>>>
>>> My goal is to do maxstat correction on ERF MEG data over voxel space.
>>> cfg.latency = [-0.2 0.8];
>>> cfg.parameter = 'avg';
>>> cfg.method = 'montecarlo';
>>> cfg.numrandomization = 2000;
>>> cfg.correctm = 'max'; % MaxStat correction
>>> cfg.design    = ones(1,nsub);
>>> cfg.ivar = 1;  % independent variable
>>> cfg.statistic = ???;
>>> [stat] = ft_timelockstatistics (cfg, timelock{:,1});
>>>
>>>  I read the discussions about one-sample cluster-based permutation 
>>> tests (e.g. 
>>> https://mailman.science.ru.nl/pipermail/fieldtrip/2018-August/012314.html). 
>>> However, for me it's not possible to permute the baseline and post 
>>> stimulus condition because I have expectation reactions in the 
>>> baseline.
>>> Anybody could suggest me how I could validly test whether my ERF is 
>>> signficantly different from zero at every time-frame and electrode?
>>>
>>> Thanks,
>>>
>>> Lars
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> https://doi.org/10.1371/journal.pcbi.1002202
>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://doi.org/10.1371/journal.pcbi.1002202
>
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190508/fb577600/attachment-0002.html>
-------------- next part --------------
_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://doi.org/10.1371/journal.pcbi.1002202