<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi Casper,<div><br></div><div>Biosemi files are often problematic. The files themselves are 24 bit, which makes them efficient on disk (although though they are still huge on disk), but once in memory they take 64 bit per sample. So your 3GB becomes 8GB in memory, not accounting for any overhead. Depending on the analysis pipeline, it might well be that two copies of the data are needed in memory (so 16GB), plus further overhead.</div><div><br></div><div><div>Note that downampling requires that a low-pass filter is applied prior to downsampling to avoid aliassing (<a href="http://en.wikipedia.org/wiki/Aliasing">http://en.wikipedia.org/wiki/Aliasing</a>). This happens automatically in ft_resampledata (look for cfg.resamplemethod and related comments in the code).</div></div><div><br></div><div>You can use a strategy like this</div><div><span style="font-size: 11px; font-family: 'Andale Mono';"><br></span></div><div><span style="font-size: 11px; font-family: 'Andale Mono';">cfg1 = [];</span></div><div><div><font face="Andale Mono"><span style="font-size: 11px;">cfg1.dataset = yourfilename;</span></font></div></div><div><font face="Andale Mono"><span style="font-size: 11px;">cfg1 = ...</span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;">cfg1 = ft_definetrial(cfg1); % this part is optional, without it it results in continuous data in memory </span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;"><br></span></font></div><div><div><font face="Andale Mono"><span style="font-size: 11px;">cfg2 = [];</span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;">cfg2.resamplefs = 500;</span></font></div></div><div><font face="Andale Mono"><span style="font-size: 11px;"><br></span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;">for i=1:nchan</span></font></div><div><span style="font-size: 11px; font-family: 'Andale Mono';"> cfg1.channel = i; % you can use a number as well as a string</span></div><div><font face="Andale Mono"><span style="font-size: 11px;"> temp = ft_preprocessing(cfg1);</span></font></div><div><span style="font-size: 11px; font-family: 'Andale Mono';"> singlechan{i} = ft_resampledata(cfg2, temp);</span></div><div><font face="Andale Mono"><span style="font-size: 11px;"> clear temp;</span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;">end % for all channels</span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;"><br></span></font></div><div><font face="Andale Mono"><span style="font-size: 11px;">data = ft_appenddata([], singlechan{:});</span></font></div><div><br></div><div>This reads and downsamples the data one channel at a time. </div><div><br></div><div><br></div><div>best regards,</div><div>Robert</div><div><br></div><div><br></div><div><br><div><div>On 05 Jan 2015, at 17:02, Casper van Heck <<a href="mailto:caspervanheck@gmail.com">caspervanheck@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div dir="ltr">Dear Emilie,<div><br></div><div>I'm using a Windows-based wreck with 8GB ram and 1.5GB files, which never pops over an usage of 4GB, so I am a bit surprised that you're getting issues with your data. Also; reducing the sampling rate that much should reduce the memory footprint to something close to 400mb, at least, which to my mind should not produce issues of any kind. Could you post a bit more of your code?</div><div><br></div><div>I've been lowering the sampling rate too (5000 to 500), but I'm also cutting the data into smaller pieces, based on markers, effectively splitting the data into four parts, and throwing away more than 80%. Cutting the data into pieces can provide a workaround for memory issues. Detail: while I do filter (and some other details) before the resampling, and I'm only resampling due to time constraints, not crashing behaviour.</div><div><br></div><div>Also check the memory tutorial: <a href="http://fieldtrip.fcdonders.nl/tutorial/memory">fieldtrip.fcdonders.nl/tutorial/memory</a></div><div><br></div><div>Does this help?</div><div><br></div><div>Casper</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 12, 2014 at 11:24 PM, Caspar, Emilie <span dir="ltr"><<a href="mailto:e.caspar@ucl.ac.uk" target="_blank">e.caspar@ucl.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>
<div>
<div style="word-wrap:break-word"><p class="MsoNormal">Dear Fieltrippers, </p>
</div>
</div>
</div>
<div>I did a pilot study on one participant today. Now that I'm trying to analyze my data, I realize that the size file is too big for my computer (size = 3Gb). Even after one hour, the filters (high pass + low pass) were not yet achieved. </div>
<div><br>
</div>
<div>So I would like to see how to reduce the size of my sample BEFORE the filters.</div>
<div><br>
</div>
<div>I know that there is "ft_resampledata", and I did it to reduce the actual sample rate (= 2048) to 256. However, even with this procedure my computer is crashing (even with 16 Go RAM). In addition, I'm not sure that I can resample before filtering (I read
different informations). </div>
<div><br>
</div>
<div>Another way I was thinking about was to pre-select electrodes that I need (only 6 electrodes on 64). But here I have two questions:</div>
<div>- Can I pre-process only some electrodes? Does it really reduce size for next preprocesses?</div>
<div>- Is this the correct way to ask? As it crashes, not sure it works.</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:10px;font-family:Courier"> cfg = [];</div>
<div style="margin:0px;font-size:10px;font-family:Courier"> cfg.dataset = [ <a href="http://file.name/" target="_blank">file.name</a>];</div>
<div style="margin:0px;font-size:10px;font-family:Courier"> cfg.channel =
<span style="color:#b245f3">'B5'</span>, <span style="color:#b245f3">'B6'</span>,
<span style="color:#b245f3">'B15'</span>, <span style="color:#b245f3">'B16'</span>;</div>
<div style="margin:0px;font-size:10px;font-family:Courier"> allData_prepross = ft_preprocessing(cfg);</div>
<div style="margin:0px;font-size:10px;font-family:Courier"> cfg.resamplefs = 256;</div>
<div style="margin:0px;font-size:10px;font-family:Courier"> DataSample = ft_resampledata(cfg, allData_prepross)</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I would appreciate pieces of advice!</div>
<div><br>
</div>
<div>Many thanks :)</div><span class="HOEnZb"><font color="#888888">
<div><br>
</div>
<div>Emilie</div>
<div><br>
</div>
<div>
<div>
<div style="word-wrap:break-word"><p class="MsoNormal"><br>
</p>
</div>
</div>
</div>
</font></span></div>
<br>_______________________________________________<br>
fieldtrip mailing list<br>
<a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
<a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br></blockquote></div><br></div>
_______________________________________________<br>fieldtrip mailing list<br><a href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</blockquote></div><br></div></body></html>