[FieldTrip] ft_sourcestatistics startup memory problem
Akiko Ikkai
akiko.ikkai at gmail.com
Thu Oct 18 22:29:37 CEST 2012
Hi Everyone,
Thank you very much for your advise. I just came back from SfN, and just
found your replies, so I will test these options and report back what I
find.
Thank you again! Akiko
On Mon, Oct 8, 2012 at 6:14 AM, Stephen Whitmarsh <
stephen.whitmarsh at gmail.com> wrote:
> Hi Akiko!
>
> Just to chime in - your source grid is very, very large indeed! ALthough I
> started with a very fine grid, at one point I also had to downside it
> (going to 5mm), and had to stop using interpolated source data for stats
> and the like. Johanna's suggestion will certainly do the trick and it will
> speed up your analysis enormously as well. You then only need to do
> interpolate for plotting purposes.
>
> all the best,
> Stephen
>
>
> On 8 October 2012 12:02, Johanna Zumer <johanna.zumer at donders.ru.nl>wrote:
>
>> Hi Akiko,
>>
>> In addition to Saskia's comment (which is very useful!) remember also
>> that if you create the subject's grid from the warped MNI template grid
>> (explained here
>> http://fieldtrip.fcdonders.nl/example/create_single-subject_grids_in_individual_head_space_that_are_all_aligned_in_mni_space?s[]=template&s[]=grid)
>> then you can keep each subject's source data in a 'source' structure with
>> .pos field and still do group level statistics, without need to convert to
>> 'volume' structure which upsamples the spatial resolution perhaps
>> artificially too high.
>>
>> Cheers,
>> Johanna
>>
>>
>> 2012/10/7 Akiko Ikkai <akiko.ikkai at gmail.com>
>>
>>> Hi Saskia,
>>>
>>> Thank you for the quick advise! Removing cfg definitely works well. Each
>>> of the group data is now 1.3G (I also used "single" to convert everything
>>> into single precision), which is definitely manageable. Computation time
>>> has also been reduced to 3 times less now.
>>>
>>> Thanks! Akiko
>>>
>>>
>>> On Sun, Oct 7, 2012 at 2:16 PM, Saskia Haegens <shaegens at gmail.com>wrote:
>>>
>>>> Hi Akiko,
>>>>
>>>> In my experience with grandavg source structs, sometimes the cfg
>>>> (that's attached to the data struct) becomes very large and can
>>>> consume a considerable amount of memory. I'm not sure if that's the
>>>> case/problem here, but might be worth checking and removing the cfg.
>>>> You could even use checkconfig to cleanup your cfg with:
>>>> data.cfg = ft_checkconfig(data.cfg, 'checksize', 'yes')
>>>> Hope this helps!
>>>>
>>>> Best,
>>>> Saskia
>>>>
>>>> On Sun, Oct 7, 2012 at 11:36 AM, Akiko Ikkai <akiko.ikkai at gmail.com>
>>>> wrote:
>>>> > Dear Fieldtrip users,
>>>> >
>>>> > I've been trying to run group stats on my EEG source data, which
>>>> contains 14
>>>> > subjects' normalized beamformer data, and having serious swap memory
>>>> issue
>>>> > (not Matlab memory issue, but OS swap memory).
>>>> >
>>>> > I'm trying to contrast 2 conditions (within subject design). Each
>>>> subject's
>>>> > normalized beamformer data (1 condition) is
>>>> >
>>>> > source_lTMI_intNorm =
>>>> >
>>>> > anatomy: [181x217x181 double]
>>>> >
>>>> > inside: [181x217x181 logical]
>>>> >
>>>> > avg: [1x1 struct]
>>>> >
>>>> > transform: [4x4 double]
>>>> >
>>>> > dim: [181 217 181]
>>>> >
>>>> > cfg: [1x1 struct]
>>>> >
>>>> >
>>>> >>whos
>>>> >
>>>> >
>>>> >
>>>> > Name Size Bytes Class
>>>> Attributes
>>>> >
>>>> > source_lTMI_intNorm 1x1 520114791 struct
>>>> >
>>>> >
>>>> > therefore, when I open all subjects' data ("data1group" and
>>>> "data2group"),
>>>> > it's huge...
>>>> >
>>>> > Name Size Bytes Class Attributes
>>>> >
>>>> > data1group 1x14 5909429438 cell
>>>> >
>>>> > data2group 1x14 6705652782 cell
>>>> >
>>>> >
>>>> > data1group & data2group are both 1x14 struct (1 cell/subject).
>>>> Therefore,
>>>> >
>>>> >>data1group{1}
>>>> >
>>>> > anatomy: [181x217x181 double]
>>>> >
>>>> > inside: [181x217x181 logical]
>>>> >
>>>> > avg: [1x1 struct]
>>>> >
>>>> > transform: [4x4 double]
>>>> >
>>>> > dim: [181 217 181]
>>>> >
>>>> > cfg: [1x1 struct]
>>>> >
>>>> > So, when I try to run
>>>> >
>>>> > cfg=[];
>>>> >
>>>> > cfg.dim = data1group{1}.dim;
>>>> >
>>>> > cfg.method = 'montecarlo';
>>>> >
>>>> > cfg.statistic = 'depsamplesT';
>>>> >
>>>> > cfg.parameter = 'avg.pow';
>>>> >
>>>> > cfg.correctm = 'cluster';
>>>> >
>>>> > cfg.numrandomization = 100;
>>>> >
>>>> > cfg.alpha = 0.05;
>>>> >
>>>> > cfg.tail = 0;
>>>> >
>>>> > nsubj=length(data1group);
>>>> >
>>>> > cfg.design(1,:) = [1:nsubj 1:nsubj];
>>>> >
>>>> > cfg.design(2,:) = [ones(1,nsubj) ones(1,nsubj)*2];
>>>> >
>>>> > cfg.uvar = 1;
>>>> >
>>>> > cfg.ivar = 2;
>>>> >
>>>> > stat = ft_sourcestatistics(cfg, data1group{:}, data2group{:});
>>>> >
>>>> > stat.anatomy = data1group{1}.anatomy;
>>>> >
>>>> >
>>>> > my computer (os 10.6.8, 6G memory) runs out of swap memory (startup
>>>> > memory?), which forces me to quit Matlab. I'm running above processes
>>>> in a
>>>> > function, so I'm not running into Matlab memory error.
>>>> >
>>>> > Could someone help me how it could run more efficiently? I guess
>>>> > cfg.inputfile is not available for ft_sourcestatistics, so I have to
>>>> > eventually load 2 group data in Matlab workspace...?
>>>> >
>>>> > Thank you in advance! Akiko
>>>> >
>>>> > --
>>>> > Akiko Ikkai, Ph.D.
>>>> > Postdoctoral Fellow
>>>> > Department of Psychological and Brain Sciences
>>>> > Johns Hopkins University
>>>> > Ames Hall, 3400 N. Charles St.
>>>> > Baltimore, MD 21218
>>>> >
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > fieldtrip mailing list
>>>> > fieldtrip at donders.ru.nl
>>>> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>
>>>
>>>
>>>
>>> --
>>> Akiko Ikkai, Ph.D.
>>> Postdoctoral Fellow
>>> Department of Psychological and Brain Sciences
>>> Johns Hopkins University
>>> Ames Hall, 3400 N. Charles St.
>>> Baltimore, MD 21218
>>>
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
--
Akiko Ikkai, Ph.D.
Postdoctoral Fellow
Department of Psychological and Brain Sciences
Johns Hopkins University
Ames Hall, 3400 N. Charles St.
Baltimore, MD 21218
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121018/87332c4c/attachment-0002.html>
More information about the fieldtrip
mailing list