[FieldTrip] ft_sourcestatistics startup memory problem

[Johanna Zumer]

Hi Akiko,

In addition to Saskia's comment (which is very useful!) remember also that
if you create the subject's grid from the warped MNI template grid
(explained here
http://fieldtrip.fcdonders.nl/example/create_single-subject_grids_in_individual_head_space_that_are_all_aligned_in_mni_space?s[]=template&s[]=grid)
 then you can keep each subject's source data in a 'source' structure with
.pos field and still do group level statistics, without need to convert to
'volume' structure which upsamples the spatial resolution perhaps
artificially too high.

Cheers,
Johanna

2012/10/7 Akiko Ikkai <akiko.ikkai at gmail.com>

> Hi Saskia,
>
> Thank you for the quick advise! Removing cfg definitely works well. Each
> of the group data is now 1.3G (I also used "single" to convert everything
> into single precision), which is definitely manageable. Computation time
> has also been reduced to 3 times less now.
>
> Thanks! Akiko
>
>
> On Sun, Oct 7, 2012 at 2:16 PM, Saskia Haegens <shaegens at gmail.com> wrote:
>
>> Hi Akiko,
>>
>> In my experience with grandavg source structs, sometimes the cfg
>> (that's attached to the data struct) becomes very large and can
>> consume a considerable amount of memory. I'm not sure if that's the
>> case/problem here, but might be worth checking and removing the cfg.
>> You could even use checkconfig to cleanup your cfg with:
>> data.cfg = ft_checkconfig(data.cfg, 'checksize', 'yes')
>> Hope this helps!
>>
>> Best,
>> Saskia
>>
>> On Sun, Oct 7, 2012 at 11:36 AM, Akiko Ikkai <akiko.ikkai at gmail.com>
>> wrote:
>> > Dear Fieldtrip users,
>> >
>> > I've been trying to run group stats on my EEG source data, which
>> contains 14
>> > subjects' normalized beamformer data, and having serious swap memory
>> issue
>> > (not Matlab memory issue, but OS swap memory).
>> >
>> > I'm trying to contrast 2 conditions (within subject design). Each
>> subject's
>> > normalized beamformer data (1 condition) is
>> >
>> > source_lTMI_intNorm =
>> >
>> >       anatomy: [181x217x181 double]
>> >
>> >        inside: [181x217x181 logical]
>> >
>> >           avg: [1x1 struct]
>> >
>> >     transform: [4x4 double]
>> >
>> >           dim: [181 217 181]
>> >
>> >           cfg: [1x1 struct]
>> >
>> >
>> >>whos
>> >
>> >
>> >
>> > Name                     Size                Bytes  Class     Attributes
>> >
>> >   source_lTMI_intNorm      1x1             520114791  struct
>> >
>> >
>> > therefore, when I open all subjects' data ("data1group" and
>> "data2group"),
>> > it's huge...
>> >
>> > Name            Size                 Bytes  Class     Attributes
>> >
>> >   data1group      1x14            5909429438  cell
>> >
>> >   data2group      1x14            6705652782  cell
>> >
>> >
>> > data1group & data2group are both 1x14 struct (1 cell/subject).
>> Therefore,
>> >
>> >>data1group{1}
>> >
>> > anatomy: [181x217x181 double]
>> >
>> >        inside: [181x217x181 logical]
>> >
>> >           avg: [1x1 struct]
>> >
>> >     transform: [4x4 double]
>> >
>> >           dim: [181 217 181]
>> >
>> >           cfg: [1x1 struct]
>> >
>> > So, when I try to run
>> >
>> > cfg=[];
>> >
>> >  cfg.dim         = data1group{1}.dim;
>> >
>> >  cfg.method      = 'montecarlo';
>> >
>> >  cfg.statistic   = 'depsamplesT';
>> >
>> >  cfg.parameter   = 'avg.pow';
>> >
>> >  cfg.correctm    = 'cluster';
>> >
>> >  cfg.numrandomization = 100;
>> >
>> >  cfg.alpha       = 0.05;
>> >
>> >  cfg.tail        = 0;
>> >
>> >  nsubj=length(data1group);
>> >
>> >  cfg.design(1,:) = [1:nsubj 1:nsubj];
>> >
>> >  cfg.design(2,:) = [ones(1,nsubj) ones(1,nsubj)*2];
>> >
>> >  cfg.uvar        = 1;
>> >
>> >  cfg.ivar        = 2;
>> >
>> >  stat = ft_sourcestatistics(cfg, data1group{:}, data2group{:});
>> >
>> >  stat.anatomy = data1group{1}.anatomy;
>> >
>> >
>> > my computer (os 10.6.8, 6G memory) runs out of swap memory (startup
>> > memory?), which forces me to quit Matlab. I'm running above processes
>> in a
>> > function, so I'm not running into Matlab memory error.
>> >
>> > Could someone help me how it could run more efficiently? I guess
>> > cfg.inputfile is not available for ft_sourcestatistics, so I have to
>> > eventually load 2 group data in Matlab workspace...?
>> >
>> > Thank you in advance! Akiko
>> >
>> > --
>> > Akiko Ikkai, Ph.D.
>> > Postdoctoral Fellow
>> > Department of Psychological and Brain Sciences
>> > Johns Hopkins University
>> > Ames Hall, 3400 N. Charles St.
>> > Baltimore, MD 21218
>> >
>> >
>> >
>> > _______________________________________________
>> > fieldtrip mailing list
>> > fieldtrip at donders.ru.nl
>> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>
>
>
> --
> Akiko Ikkai, Ph.D.
> Postdoctoral Fellow
> Department of Psychological and Brain Sciences
> Johns Hopkins University
> Ames Hall, 3400 N. Charles St.
> Baltimore, MD 21218
>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121008/f3f0942e/attachment-0002.html>