[FieldTrip] ft_sourcestatistics startup memory problem
Johanna Zumer
johanna.zumer at donders.ru.nl
Sun Oct 21 17:32:37 CEST 2012
Hi Akiko,
Can you type 'dbstop if error' before running the last step, so that
it goes to debug mode when it crashes there? What are the sizes of
'filt' and 'Cf' for the 40th position?
Another thing to try is, during your last call to ft_sourceanalysis,
cfg = [];
cfg.grid = grid;
cfg.grid.filter = source_common.avg.filter;
so that all the .inside and .outside information is transfered as well.
A note to all users: you can skip the first step by loading the
precomuted template grids in */fieldtrip/template/sourcemodel.
Cheers,
Johanna
2012/10/20 Akiko Ikkai <akiko.ikkai at gmail.com>:
> Hi,
>
> So I have followed Johanna and Stephan's advise to create template_grid and
> each subject's grid from MNI image, which is now 1cm spacing instead of
> .5cm. These steps seem to run, and I'm able to create a common spatial
> filter to run beamformer using DICS. However, I get an error when I apply
> the commom filter to each condition.
>
> I had to tweak a few things on example on the page
> http://fieldtrip.fcdonders.nl/example/create_single-subject_grids_in_individual_head_space_that_are_all_aligned_in_mni_space?s[]=template&s[]=grid
> , since my data is EEG and electrode locations vary between subjects, such
> that:
>
> cfg = [];
> cfg.grid.xgrid = -20:1:20;
> cfg.grid.ygrid = -20:1:20;
> cfg.grid.zgrid = -20:1:20;
> cfg.grid.tight = 'yes';
> cfg.inwardshift = -1.5;
> cfg.vol = template_vol;
> cfg.elec = ft_data.elec; % individual sub's electrode locations: had to add
> this (got error otherwise)
> template_grid = ft_prepare_sourcemodel(cfg);
>
> then
> cfg = [];
> cfg.grid.warpmni = 'yes';
> cfg.grid.template = template_grid;
> cfg.grid.nonlinear = 'yes';
> cfg.mri = mri_aligned;
> cfg.elec = ft_data.elec; % individual sub's electrode locations: had to add
> this (got error otherwise)
> grid = ft_prepare_sourcemodel(cfg);
>
> These create grids in MNI space, both for template and individual fine. To
> create a common spatial filter, I run the following command, which runs:
> cfg = [];
> cfg.grid.pos = grid.pos;
> cfg.grid.dim = grid.dim;
> cfg.grid.inside = grid.inside;
> cfg.grid.outside = grid.outside;
> cfg.inwardshift = -1.5;
> cfg.vol = vol_cm;
> cfg.channel = {'all'};
> cfg.frequency = 10.5;
> cfg.method = 'dics';
> cfg.dics.projectnoise = 'yes';
> cfg.dics.keepfilter = 'yes';
> cfg.dics.feedback = 'no';
> source_common = ft_sourceanalysis(cfg, freq_common);
>
> however, when I apply this spatial filter to a task condition, I get an
> error message
> cfg = [];
> cfg.elec = freq_common.elec;
> cfg.grid.pos = source_common.pos;
> cfg.grid.filter = source_common.avg.filter;
> cfg.inwardshift = -1.5;
> cfg.vol = vol_cm;
> cfg.channel = {'all'};
> cfg.frequency = 10.5;
> cfg.method = 'dics';
> cfg.dics.projectnoise = 'yes';
> cfg.dics.keepfilter = 'yes';
> cfg.dics.feedback = 'no';
> source_itemL = ft_sourceanalysis(cfg, freq_itemL);
> source_itemL.unit = 'cm';
>
> ??? Error using ==> mtimes
> Inner matrix dimensions must agree.
>
> Error in ==> beamformer_dics at 316
> csd = filt * Cf * ctranspose(filt); % Gross
> eqn. 4
> and 5
>
> Error in ==> ft_sourceanalysis at 588
> dip(i) = beamformer_dics(grid, sens, vol, [], squeeze(Cf(i,:,:)),
> optarg{:});
>
> The error occurs when dip.pos = 40 (i.e. it runs fine 1-39). I can't spot
> where exactly the error is originating from. Am I missing some steps when I
> create template grids that I feed into ft_sourceanalysis?
>
> Thank you in advance for your help! Akiko
>
>
> On Thu, Oct 18, 2012 at 4:29 PM, Akiko Ikkai <akiko.ikkai at gmail.com> wrote:
>>
>> Hi Everyone,
>>
>> Thank you very much for your advise. I just came back from SfN, and just
>> found your replies, so I will test these options and report back what I
>> find.
>>
>> Thank you again! Akiko
>>
>>
>> On Mon, Oct 8, 2012 at 6:14 AM, Stephen Whitmarsh
>> <stephen.whitmarsh at gmail.com> wrote:
>>>
>>> Hi Akiko!
>>>
>>> Just to chime in - your source grid is very, very large indeed! ALthough
>>> I started with a very fine grid, at one point I also had to downside it
>>> (going to 5mm), and had to stop using interpolated source data for stats and
>>> the like. Johanna's suggestion will certainly do the trick and it will speed
>>> up your analysis enormously as well. You then only need to do interpolate
>>> for plotting purposes.
>>>
>>> all the best,
>>> Stephen
>>>
>>>
>>> On 8 October 2012 12:02, Johanna Zumer <johanna.zumer at donders.ru.nl>
>>> wrote:
>>>>
>>>> Hi Akiko,
>>>>
>>>> In addition to Saskia's comment (which is very useful!) remember also
>>>> that if you create the subject's grid from the warped MNI template grid
>>>> (explained here
>>>> http://fieldtrip.fcdonders.nl/example/create_single-subject_grids_in_individual_head_space_that_are_all_aligned_in_mni_space?s[]=template&s[]=grid)
>>>> then you can keep each subject's source data in a 'source' structure with
>>>> .pos field and still do group level statistics, without need to convert to
>>>> 'volume' structure which upsamples the spatial resolution perhaps
>>>> artificially too high.
>>>>
>>>> Cheers,
>>>> Johanna
>>>>
>>>>
>>>> 2012/10/7 Akiko Ikkai <akiko.ikkai at gmail.com>
>>>>>
>>>>> Hi Saskia,
>>>>>
>>>>> Thank you for the quick advise! Removing cfg definitely works well.
>>>>> Each of the group data is now 1.3G (I also used "single" to convert
>>>>> everything into single precision), which is definitely manageable.
>>>>> Computation time has also been reduced to 3 times less now.
>>>>>
>>>>> Thanks! Akiko
>>>>>
>>>>>
>>>>> On Sun, Oct 7, 2012 at 2:16 PM, Saskia Haegens <shaegens at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi Akiko,
>>>>>>
>>>>>> In my experience with grandavg source structs, sometimes the cfg
>>>>>> (that's attached to the data struct) becomes very large and can
>>>>>> consume a considerable amount of memory. I'm not sure if that's the
>>>>>> case/problem here, but might be worth checking and removing the cfg.
>>>>>> You could even use checkconfig to cleanup your cfg with:
>>>>>> data.cfg = ft_checkconfig(data.cfg, 'checksize', 'yes')
>>>>>> Hope this helps!
>>>>>>
>>>>>> Best,
>>>>>> Saskia
>>>>>>
>>>>>> On Sun, Oct 7, 2012 at 11:36 AM, Akiko Ikkai <akiko.ikkai at gmail.com>
>>>>>> wrote:
>>>>>> > Dear Fieldtrip users,
>>>>>> >
>>>>>> > I've been trying to run group stats on my EEG source data, which
>>>>>> > contains 14
>>>>>> > subjects' normalized beamformer data, and having serious swap memory
>>>>>> > issue
>>>>>> > (not Matlab memory issue, but OS swap memory).
>>>>>> >
>>>>>> > I'm trying to contrast 2 conditions (within subject design). Each
>>>>>> > subject's
>>>>>> > normalized beamformer data (1 condition) is
>>>>>> >
>>>>>> > source_lTMI_intNorm =
>>>>>> >
>>>>>> > anatomy: [181x217x181 double]
>>>>>> >
>>>>>> > inside: [181x217x181 logical]
>>>>>> >
>>>>>> > avg: [1x1 struct]
>>>>>> >
>>>>>> > transform: [4x4 double]
>>>>>> >
>>>>>> > dim: [181 217 181]
>>>>>> >
>>>>>> > cfg: [1x1 struct]
>>>>>> >
>>>>>> >
>>>>>> >>whos
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > Name Size Bytes Class
>>>>>> > Attributes
>>>>>> >
>>>>>> > source_lTMI_intNorm 1x1 520114791 struct
>>>>>> >
>>>>>> >
>>>>>> > therefore, when I open all subjects' data ("data1group" and
>>>>>> > "data2group"),
>>>>>> > it's huge...
>>>>>> >
>>>>>> > Name Size Bytes Class Attributes
>>>>>> >
>>>>>> > data1group 1x14 5909429438 cell
>>>>>> >
>>>>>> > data2group 1x14 6705652782 cell
>>>>>> >
>>>>>> >
>>>>>> > data1group & data2group are both 1x14 struct (1 cell/subject).
>>>>>> > Therefore,
>>>>>> >
>>>>>> >>data1group{1}
>>>>>> >
>>>>>> > anatomy: [181x217x181 double]
>>>>>> >
>>>>>> > inside: [181x217x181 logical]
>>>>>> >
>>>>>> > avg: [1x1 struct]
>>>>>> >
>>>>>> > transform: [4x4 double]
>>>>>> >
>>>>>> > dim: [181 217 181]
>>>>>> >
>>>>>> > cfg: [1x1 struct]
>>>>>> >
>>>>>> > So, when I try to run
>>>>>> >
>>>>>> > cfg=[];
>>>>>> >
>>>>>> > cfg.dim = data1group{1}.dim;
>>>>>> >
>>>>>> > cfg.method = 'montecarlo';
>>>>>> >
>>>>>> > cfg.statistic = 'depsamplesT';
>>>>>> >
>>>>>> > cfg.parameter = 'avg.pow';
>>>>>> >
>>>>>> > cfg.correctm = 'cluster';
>>>>>> >
>>>>>> > cfg.numrandomization = 100;
>>>>>> >
>>>>>> > cfg.alpha = 0.05;
>>>>>> >
>>>>>> > cfg.tail = 0;
>>>>>> >
>>>>>> > nsubj=length(data1group);
>>>>>> >
>>>>>> > cfg.design(1,:) = [1:nsubj 1:nsubj];
>>>>>> >
>>>>>> > cfg.design(2,:) = [ones(1,nsubj) ones(1,nsubj)*2];
>>>>>> >
>>>>>> > cfg.uvar = 1;
>>>>>> >
>>>>>> > cfg.ivar = 2;
>>>>>> >
>>>>>> > stat = ft_sourcestatistics(cfg, data1group{:}, data2group{:});
>>>>>> >
>>>>>> > stat.anatomy = data1group{1}.anatomy;
>>>>>> >
>>>>>> >
>>>>>> > my computer (os 10.6.8, 6G memory) runs out of swap memory (startup
>>>>>> > memory?), which forces me to quit Matlab. I'm running above
>>>>>> > processes in a
>>>>>> > function, so I'm not running into Matlab memory error.
>>>>>> >
>>>>>> > Could someone help me how it could run more efficiently? I guess
>>>>>> > cfg.inputfile is not available for ft_sourcestatistics, so I have to
>>>>>> > eventually load 2 group data in Matlab workspace...?
>>>>>> >
>>>>>> > Thank you in advance! Akiko
>>>>>> >
>>>>>> > --
>>>>>> > Akiko Ikkai, Ph.D.
>>>>>> > Postdoctoral Fellow
>>>>>> > Department of Psychological and Brain Sciences
>>>>>> > Johns Hopkins University
>>>>>> > Ames Hall, 3400 N. Charles St.
>>>>>> > Baltimore, MD 21218
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > fieldtrip mailing list
>>>>>> > fieldtrip at donders.ru.nl
>>>>>> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>> _______________________________________________
>>>>>> fieldtrip mailing list
>>>>>> fieldtrip at donders.ru.nl
>>>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Akiko Ikkai, Ph.D.
>>>>> Postdoctoral Fellow
>>>>> Department of Psychological and Brain Sciences
>>>>> Johns Hopkins University
>>>>> Ames Hall, 3400 N. Charles St.
>>>>> Baltimore, MD 21218
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> fieldtrip mailing list
>>>>> fieldtrip at donders.ru.nl
>>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>
>>
>>
>> --
>> Akiko Ikkai, Ph.D.
>> Postdoctoral Fellow
>> Department of Psychological and Brain Sciences
>> Johns Hopkins University
>> Ames Hall, 3400 N. Charles St.
>> Baltimore, MD 21218
>>
>>
>
>
>
> --
> Akiko Ikkai, Ph.D.
> Postdoctoral Fellow
> Department of Psychological and Brain Sciences
> Johns Hopkins University
> Ames Hall, 3400 N. Charles St.
> Baltimore, MD 21218
>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
More information about the fieldtrip
mailing list