[FieldTrip] Question about how to reduce the file size

[Casper van Heck]

Dear Emilie,

I'm using a Windows-based wreck with 8GB ram and 1.5GB files, which never
pops over an usage of 4GB, so I am a bit surprised that you're getting
issues with your data. Also; reducing the sampling rate that much should
reduce the memory footprint to something close to 400mb, at least, which to
my mind should not produce issues of any kind. Could you post a bit more of
your code?

I've been lowering the sampling rate too (5000 to 500), but I'm also
cutting the data into smaller pieces, based on markers, effectively
splitting the data into four parts, and throwing away more than
80%. Cutting the data into pieces can provide a workaround for memory
issues. Detail: while I do filter (and some other details) before the
resampling, and I'm only resampling due to time constraints, not crashing
behaviour.

Also check the memory tutorial: fieldtrip.fcdonders.nl/tutorial/memory

Does this help?

Casper

On Fri, Dec 12, 2014 at 11:24 PM, Caspar, Emilie <e.caspar at ucl.ac.uk> wrote:

>   Dear Fieltrippers,
>   I did a pilot study on one participant today. Now that I'm trying to
> analyze my data, I realize that the size file is too big for my computer
> (size = 3Gb). Even after one hour, the filters (high pass + low pass) were
> not yet achieved.
>
>  So I would like to see how to reduce the size of my sample BEFORE the
> filters.
>
>  I know that there is "ft_resampledata", and I did it to reduce the
> actual sample rate (= 2048) to 256. However, even with this procedure my
> computer is crashing (even with 16 Go RAM). In addition, I'm not sure that
> I can resample before filtering (I read different informations).
>
>  Another way I was thinking about was to pre-select electrodes that I
> need (only 6 electrodes on 64). But here I have two questions:
> - Can I pre-process only some electrodes? Does it really reduce size for
> next preprocesses?
> - Is this the correct way to ask? As it crashes, not sure it works.
>
>      cfg             = [];
>     cfg.dataset     = [ file.name];
>     cfg.channel     = 'B5', 'B6', 'B15', 'B16';
>     allData_prepross = ft_preprocessing(cfg);
>     cfg.resamplefs = 256;
>     DataSample = ft_resampledata(cfg, allData_prepross)
>
>
>  I would appreciate pieces of advice!
>
>  Many thanks :)
>
>  Emilie
>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150105/699ea8f9/attachment-0001.html>