Thank you all for this interesting advices. I am not using Fieldtrip in this case to read the raw data (for this I am using a function from BCI2000), just for signal processing. <br>The qsubcellfun function is new for me. I will implement it, this will solve the problem of memory as Giovanni mentioned.<br>
<br>Thanks again,<br><br>Eliana<br><br><div class="gmail_quote">On Thu, Jul 12, 2012 at 8:17 AM, Gio Piantoni <span dir="ltr"><<a href="mailto:g.piantoni@nin.knaw.nl" target="_blank">g.piantoni@nin.knaw.nl</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Please, note that "clear all" clears variables, but does not free up<br>
memory, especially for files read from disk. You can check the memory<br>
usage with the command "top" in Linux command line. You'll see that<br>
nothing changes even if you use "clear all".<br>
I agree with Stephen that you need to write smaller functions for each<br>
subjects. Something like:<br>
<br>
for i = 1:numel(subj)<br>
do_analysis(subj)<br>
end<br>
Where "do_analysis.m" is a function you wrote to do all the analysis.<br>
In this way, Matlab takes care of memory management by itself (it<br>
frees the memory once the function has been executed).<br>
<br>
A more elaborate solution is to use the "qsub" functions included in<br>
fieldtrip. Try this:<br>
addpath /path/to/fieldtrip/qsub<br>
fname = 'do_analysis';<br>
subj = {1, 2, 3, 4, 5};<br>
qsubcellfun(fname, x1, 'memreq', 1024^3, 'timreq', 60*60, 'backend', 'local');<br>
<br>
This will start a new Matlab session (and use a new Matlab license)<br>
and it will close it when the function has been executed. There should<br>
be no memory leaks at all. See "help qsubcellfun" for more info.<br>
Remember to use 'backend' 'local' unless you have access to Oracle<br>
Grid Engine or Torque.<br>
<br>
Hope this helps.<br>
<span class="HOEnZb"><font color="#888888">Gio<br>
<br>
--<br>
Giovanni Piantoni, MSc<br>
Dept. Sleep & Cognition<br>
Netherlands Institute for Neuroscience<br>
Meibergdreef 47<br>
1105 BA Amsterdam (NL)<br>
<br>
<a href="tel:%2B31%2020%205665492" value="+31205665492">+31 20 5665492</a><br>
<a href="mailto:gio@gpiantoni.com">gio@gpiantoni.com</a><br>
<a href="http://www.gpiantoni.com" target="_blank">www.gpiantoni.com</a><br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Thu, Jul 12, 2012 at 1:33 AM, Bill Budd <<a href="mailto:Bill.Budd@newcastle.edu.au">Bill.Budd@newcastle.edu.au</a>> wrote:<br>
> Hi Eliana<br>
><br>
><br>
><br>
> I see something similar when using SPM8 to process data. I’ve heard that<br>
> this might be because SPM8 uses fieldtrip functions to import my continuous<br>
> EEG files (Biosemi/bdf) and these functions may not release memory in matlab<br>
> after closing. When looping my preprocessing functions over multiple<br>
> subjects I have the exactly same problem in SPM that you see in fieldtrip.<br>
> This may not be causing your problem (or mine) but wondered if the<br>
> processing your refer to also involves importing raw MEG data?<br>
><br>
><br>
><br>
> The PCs I’m using have plenty of RAM/disk space (Win7/matlab 2011b).<br>
><br>
><br>
><br>
> Be great to find a solution to this as it slows done processing multi<br>
> subject data significantly!<br>
><br>
><br>
><br>
> Cheers<br>
><br>
><br>
><br>
> -Bill<br>
><br>
><br>
><br>
><br>
><br>
> From: <a href="mailto:fieldtrip-bounces@science.ru.nl">fieldtrip-bounces@science.ru.nl</a><br>
> [mailto:<a href="mailto:fieldtrip-bounces@science.ru.nl">fieldtrip-bounces@science.ru.nl</a>] On Behalf Of Eliana Garcia<br>
> Sent: Wednesday, 11 July 2012 2:40 AM<br>
> To: FieldTrip discussion list<br>
> Subject: Re: [FieldTrip] problems with speed of processing<br>
><br>
><br>
><br>
> Dear Stephen,<br>
><br>
><br>
><br>
> Thanks for the suggestion. I have already one function that is doing all the<br>
> preprocessing and saving as well. But this seams not to solve the issue.<br>
><br>
><br>
><br>
> Thanks,<br>
><br>
> Eliana<br>
><br>
><br>
><br>
> On Tue, Jul 10, 2012 at 6:01 PM, Stephen Whitmarsh<br>
> <<a href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>> wrote:<br>
><br>
> Dear Eliana,<br>
><br>
><br>
><br>
> You could try creating a function that does all this preprocessing and then<br>
> writes the output to file.<br>
><br>
> If you then call this function in a (subject) loop you are sure that Matlab<br>
> will free up the memory it used for that function.<br>
><br>
><br>
><br>
> Perhaps it helps,<br>
><br>
><br>
><br>
> Stephen<br>
><br>
><br>
><br>
> On 10 July 2012 16:57, Eliana Garcia <<a href="mailto:elilife@gmail.com">elilife@gmail.com</a>> wrote:<br>
><br>
> Hello Dear Fieldtrip Community,<br>
><br>
><br>
><br>
> I am having now problems with processing my data. I am analyzing a quite big<br>
> data set with 400 trials that are composed by 10 sec (re-sampled at 150Hz).<br>
> I am doing first a demean, then planar gradient transformation,<br>
> time-frequency analysis using multitapers and then combine the planar<br>
> gradient again. For the first subject always runs normally, but for the<br>
> second one matlab gets really slow. I am taking care in deleting the<br>
> variables that I create with big amount of data after I don't need them any<br>
> more, so I am using just one big file that is being safe at the end (average<br>
> time-frequency across trials) for each of the subjects. Every time matlab<br>
> starts to analyze a new subject I am using clear all (with exceptions like<br>
> directory and name of subject) but stills with every new subject it gets<br>
> slower and slower.<br>
><br>
><br>
><br>
> Do you have any advises to make the analysis faster? Sometimes is too slow<br>
> and the computer normally crashes after 3 subjects.<br>
><br>
><br>
><br>
> Thank you very much for the attention.<br>
><br>
><br>
><br>
> Best,<br>
><br>
><br>
><br>
> --<br>
><br>
> Eliana García Cossio<br>
> Applied Neurotechnology Lab.<br>
><br>
> Graduate School of Neural and Behavioural Science - Max Planck Research<br>
> School<br>
> Institute of Medical Psychology and Behavioural Neurobiology<br>
> Universität Tübingen<br>
><br>
> (+0049) 01 577-8587604<br>
> Otfried-Müller-Str. 47, 72076<br>
> Tübingen- Germany<br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> fieldtrip mailing list<br>
> <a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
> <a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> fieldtrip mailing list<br>
> <a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
> <a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
><br>
><br>
><br>
><br>
><br>
> --<br>
><br>
> Eliana García Cossio<br>
> Applied Neurotechnology Lab.<br>
><br>
> Graduate School of Neural and Behavioural Science - Max Planck Research<br>
> School<br>
> Institute of Medical Psychology and Behavioural Neurobiology<br>
> Universität Tübingen<br>
><br>
> (+0049) 01 577-8587604<br>
> Otfried-Müller-Str. 47, 72076<br>
> Tübingen- Germany<br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> fieldtrip mailing list<br>
> <a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
> <a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<br>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
<a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a></div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div><span style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse;color:rgb(136,136,136)"><div>
Eliana García Cossio<br>Applied Neurotechnology Lab.<br></div><div>Graduate School of Neural and Behavioural Science - Max Planck Research School<br>Institute of Medical Psychology and Behavioural Neurobiology<br>Universität Tübingen</div>
<div>(+0049) 01 577-8587604 <br><span lang="EN-US">Otfried-Müller-Str. 47</span><span lang="EN-US"></span><span lang="EN-US">, 72076 <br></span>Tübingen- Germany</div></span></div><span style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse;color:rgb(136,136,136)"></span></div>
<br>