[FieldTrip] ft_sourcestatistics startup memory problem

Akiko Ikkai akiko.ikkai at gmail.com
Thu Oct 18 22:29:37 CEST 2012


Hi Everyone,

Thank you very much for your advise. I just came back from SfN, and just
found your replies, so I will test these options and report back what I
find.

Thank you again! Akiko

On Mon, Oct 8, 2012 at 6:14 AM, Stephen Whitmarsh <
stephen.whitmarsh at gmail.com> wrote:

> Hi Akiko!
>
> Just to chime in - your source grid is very, very large indeed! ALthough I
> started with a very fine grid, at one point I also had to downside it
> (going to 5mm), and had to stop using interpolated source data for stats
> and the like. Johanna's suggestion will certainly do the trick and it will
> speed up your analysis enormously as well. You then only need to do
> interpolate for plotting purposes.
>
> all the best,
> Stephen
>
>
> On 8 October 2012 12:02, Johanna Zumer <johanna.zumer at donders.ru.nl>wrote:
>
>> Hi Akiko,
>>
>> In addition to Saskia's comment (which is very useful!) remember also
>> that if you create the subject's grid from the warped MNI template grid
>> (explained here
>> http://fieldtrip.fcdonders.nl/example/create_single-subject_grids_in_individual_head_space_that_are_all_aligned_in_mni_space?s[]=template&s[]=grid)
>>  then you can keep each subject's source data in a 'source' structure with
>> .pos field and still do group level statistics, without need to convert to
>> 'volume' structure which upsamples the spatial resolution perhaps
>> artificially too high.
>>
>> Cheers,
>> Johanna
>>
>>
>> 2012/10/7 Akiko Ikkai <akiko.ikkai at gmail.com>
>>
>>> Hi Saskia,
>>>
>>> Thank you for the quick advise! Removing cfg definitely works well. Each
>>> of the group data is now 1.3G (I also used "single" to convert everything
>>> into single precision), which is definitely manageable. Computation time
>>> has also been reduced to 3 times less now.
>>>
>>> Thanks! Akiko
>>>
>>>
>>> On Sun, Oct 7, 2012 at 2:16 PM, Saskia Haegens <shaegens at gmail.com>wrote:
>>>
>>>> Hi Akiko,
>>>>
>>>> In my experience with grandavg source structs, sometimes the cfg
>>>> (that's attached to the data struct) becomes very large and can
>>>> consume a considerable amount of memory. I'm not sure if that's the
>>>> case/problem here, but might be worth checking and removing the cfg.
>>>> You could even use checkconfig to cleanup your cfg with:
>>>> data.cfg = ft_checkconfig(data.cfg, 'checksize', 'yes')
>>>> Hope this helps!
>>>>
>>>> Best,
>>>> Saskia
>>>>
>>>> On Sun, Oct 7, 2012 at 11:36 AM, Akiko Ikkai <akiko.ikkai at gmail.com>
>>>> wrote:
>>>> > Dear Fieldtrip users,
>>>> >
>>>> > I've been trying to run group stats on my EEG source data, which
>>>> contains 14
>>>> > subjects' normalized beamformer data, and having serious swap memory
>>>> issue
>>>> > (not Matlab memory issue, but OS swap memory).
>>>> >
>>>> > I'm trying to contrast 2 conditions (within subject design). Each
>>>> subject's
>>>> > normalized beamformer data (1 condition) is
>>>> >
>>>> > source_lTMI_intNorm =
>>>> >
>>>> >       anatomy: [181x217x181 double]
>>>> >
>>>> >        inside: [181x217x181 logical]
>>>> >
>>>> >           avg: [1x1 struct]
>>>> >
>>>> >     transform: [4x4 double]
>>>> >
>>>> >           dim: [181 217 181]
>>>> >
>>>> >           cfg: [1x1 struct]
>>>> >
>>>> >
>>>> >>whos
>>>> >
>>>> >
>>>> >
>>>> > Name                     Size                Bytes  Class
>>>> Attributes
>>>> >
>>>> >   source_lTMI_intNorm      1x1             520114791  struct
>>>> >
>>>> >
>>>> > therefore, when I open all subjects' data ("data1group" and
>>>> "data2group"),
>>>> > it's huge...
>>>> >
>>>> > Name            Size                 Bytes  Class     Attributes
>>>> >
>>>> >   data1group      1x14            5909429438  cell
>>>> >
>>>> >   data2group      1x14            6705652782  cell
>>>> >
>>>> >
>>>> > data1group & data2group are both 1x14 struct (1 cell/subject).
>>>> Therefore,
>>>> >
>>>> >>data1group{1}
>>>> >
>>>> > anatomy: [181x217x181 double]
>>>> >
>>>> >        inside: [181x217x181 logical]
>>>> >
>>>> >           avg: [1x1 struct]
>>>> >
>>>> >     transform: [4x4 double]
>>>> >
>>>> >           dim: [181 217 181]
>>>> >
>>>> >           cfg: [1x1 struct]
>>>> >
>>>> > So, when I try to run
>>>> >
>>>> > cfg=[];
>>>> >
>>>> >  cfg.dim         = data1group{1}.dim;
>>>> >
>>>> >  cfg.method      = 'montecarlo';
>>>> >
>>>> >  cfg.statistic   = 'depsamplesT';
>>>> >
>>>> >  cfg.parameter   = 'avg.pow';
>>>> >
>>>> >  cfg.correctm    = 'cluster';
>>>> >
>>>> >  cfg.numrandomization = 100;
>>>> >
>>>> >  cfg.alpha       = 0.05;
>>>> >
>>>> >  cfg.tail        = 0;
>>>> >
>>>> >  nsubj=length(data1group);
>>>> >
>>>> >  cfg.design(1,:) = [1:nsubj 1:nsubj];
>>>> >
>>>> >  cfg.design(2,:) = [ones(1,nsubj) ones(1,nsubj)*2];
>>>> >
>>>> >  cfg.uvar        = 1;
>>>> >
>>>> >  cfg.ivar        = 2;
>>>> >
>>>> >  stat = ft_sourcestatistics(cfg, data1group{:}, data2group{:});
>>>> >
>>>> >  stat.anatomy = data1group{1}.anatomy;
>>>> >
>>>> >
>>>> > my computer (os 10.6.8, 6G memory) runs out of swap memory (startup
>>>> > memory?), which forces me to quit Matlab. I'm running above processes
>>>> in a
>>>> > function, so I'm not running into Matlab memory error.
>>>> >
>>>> > Could someone help me how it could run more efficiently? I guess
>>>> > cfg.inputfile is not available for ft_sourcestatistics, so I have to
>>>> > eventually load 2 group data in Matlab workspace...?
>>>> >
>>>> > Thank you in advance! Akiko
>>>> >
>>>> > --
>>>> > Akiko Ikkai, Ph.D.
>>>> > Postdoctoral Fellow
>>>> > Department of Psychological and Brain Sciences
>>>> > Johns Hopkins University
>>>> > Ames Hall, 3400 N. Charles St.
>>>> > Baltimore, MD 21218
>>>> >
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > fieldtrip mailing list
>>>> > fieldtrip at donders.ru.nl
>>>> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>
>>>
>>>
>>>
>>> --
>>> Akiko Ikkai, Ph.D.
>>> Postdoctoral Fellow
>>> Department of Psychological and Brain Sciences
>>> Johns Hopkins University
>>> Ames Hall, 3400 N. Charles St.
>>> Baltimore, MD 21218
>>>
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>



-- 
Akiko Ikkai, Ph.D.
Postdoctoral Fellow
Department of Psychological and Brain Sciences
Johns Hopkins University
Ames Hall, 3400 N. Charles St.
Baltimore, MD 21218
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121018/87332c4c/attachment.html>


More information about the fieldtrip mailing list