[FieldTrip] Regression test and t-test providing almost identical results

[Baptiste Gauthier]

Dear Fieldtrippers,

I recently observed a strange behavior with the results provided with the
regression statistics provided by “ft_statfun_depsamplesregrT”:

I did a regression test on 3 conditions: cond1, cond2 and cond3:

Here are the results:


[image: Images intégrées 1]



*Output for group-level regression test:*

the call to "ft_timelockgrandaverage" took 36 seconds and required the
additional allocation of an estimated 0 MB
reading layout from file
/neurospin/meg/meg_tmp/MTT_MEG_Baptiste/From_laptop/fieldtrip-20130901/template/layout/NM306mag.lay
the call to "ft_prepare_layout" took 0 seconds and required the additional
allocation of an estimated 0 MB
the call to "ft_selectdata" took 0 seconds and required the additional
allocation of an estimated 0 MB
using "ft_statistics_montecarlo" for the statistical testing
using "ft_statfun_depsamplesregrT" for the single-sample statistics
constructing randomized design
total number of measurements     = 57
total number of variables        = 2
number of independent variables  = 1
number of unit variables         = 1
number of within-cell variables  = 0
number of control variables      = 0
using a permutation resampling approach
repeated measurement in variable 1 over 19 levels
number of repeated measurements in each level is 3 3 3 3 3 3 3 3 3 3 3 3 3
3 3 3 3 3 3
computing a parametric threshold for clustering
computing statistic
estimated time per randomization is 0.26 seconds
computing statistic 1000 from 1000

Warning: adding
/neurospin/meg/meg_tmp/MTT_MEG_Baptiste/fieldtrip-20160719/external/spm8
toolbox to your MATLAB path
found 13 positive clusters in observed data
found 9 negative clusters in observed data
computing clusters in randomization
computing clusters in randomization 1000 from 1000

using a cluster-based method for multiple comparison correction
the returned probabilities and the thresholded mask are corrected for
multiple comparisons
the call to "ft_timelockstatistics" took 55 seconds and required the
additional allocation of an estimated 12 MB



Afterwards, for some other reasons I performed directly a T-test between
condition 1 and condition 3. Here is the result:


[image: Images intégrées 2]


*Output for group-level T-test:*

the call to "ft_timelockgrandaverage" took 36 seconds and required the
additional allocation of an estimated 0 MB
reading layout from file
/neurospin/meg/meg_tmp/MTT_MEG_Baptiste/From_laptop/fieldtrip-20130901/template/layout/NM306mag.lay
the call to "ft_prepare_layout" took 0 seconds and required the additional
allocation of an estimated 0 MB
the call to "ft_selectdata" took 0 seconds and required the additional
allocation of an estimated 0 MB
using "ft_statistics_montecarlo" for the statistical testing
using "ft_statfun_depsamplesT" for the single-sample statistics
constructing randomized design
total number of measurements     = 38
total number of variables        = 2
number of independent variables  = 1
number of unit variables         = 1
number of within-cell variables  = 0
number of control variables      = 0
using a permutation resampling approach
repeated measurement in variable 1 over 19 levels
number of repeated measurements in each level is 2 2 2 2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2
computing a parametric threshold for clustering
computing statistic
estimated time per randomization is 0.04 seconds
computing statistic 1000 from 1000

found 9 positive clusters in observed data
found 13 negative clusters in observed data
computing clusters in randomization
computing clusters in randomization 1000 from 1000

using a cluster-based method for multiple comparison correction
the returned probabilities and the thresholded mask are corrected for
multiple comparisons
the call to "ft_timelockstatistics" took 46 seconds and required the
additional allocation of an estimated 0 MB



I checked that I actually provided the good data and parameters to the
function, but I am puzzled by the resemblance (if you invert the sign) of
the results. To doublecheck, I directly compared the stat.stat field from
the two tests. Here are the results:


[image: Images intégrées 3]


So basically, it is the same if you round to the 3rd digits after the
floating point. It sounds really weird to me.

Has someone ever noticed something similar when performing regression
tests? I there any simple mathematical tricks that could explain that?


Best regards,



Baptiste Gauthier, PhD

-- 
Baptiste Gauthier
Postdoctoral Research Fellow

INSERM-CEA Cognitive Neuroimaging unit
CEA/SAC/DSV/DRM/Neurospin center
Bât 145, Point Courier 156
F-91191 Gif-sur-Yvette Cedex FRANCE
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160720/dc2d72df/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 102935 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160720/dc2d72df/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 87834 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160720/dc2d72df/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 30501 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160720/dc2d72df/attachment-0005.png>