[FieldTrip] Question about how to reduce the file size (Caspar, Emilie)

Arjen Stolk a.stolk8 at gmail.com
Sun Dec 14 22:13:23 CET 2014


Hi Emilie,

It seems from the code pasted that you're still trying to read all channels
in at once:
cfgp.channel = 'all';

Try reading in & downsampling one channel at a time, as Haiteng suggested
with the following link:
http://fieldtrip.fcdonders.nl/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory

Hope that solves it for you,
Arjen

2014-12-14 21:44 GMT+01:00 Caspar, Emilie <e.caspar at ucl.ac.uk>:
>
>  Dear Haiteng
>
>  Thank you for your suggestion. I tried but again, the downsample with
> this procedure takes more than 40 minutes (I gave up after this time period
> and stopped the script, the computer was crashing).
>
>  Perhaps one could tell if the following is correct? I'm maybe wrong
> somewhere.
>
>   chansAll    = 1:68;%{'CPz', 'Cz', 'FCz', 'Fz'}
> chansEEG    = 1:64;
> nchans=length(chansEEG);
>
>   display('downsample each channel')
>
>  for i=1:nchans
>   cfgp         = [];
>   cfgp.dataset     = [ file.name];
>   cfgp.channel = 'all';
>   datp         = ft_preprocessing(cfgp);
>
>
>    cfgr            = [];
>   cfgr.resamplefs = 512;
>   datr{i}         = ft_resampledata(cfgr, datp);
>
>
>    clear datp
>
>  cfg = [];
> datall = ft_appenddata(cfg, datr{:});
>
>
>  display('Filters')
>
>
>      cfg = [];
>     cfg.bpfilter    = 'yes';
>     cfg.bpfreq      = bpfilterRange;
>     cfg.bpfiltord   = bpfilterOrder;
>     cfg.reref       = 'yes';
>     cfg.refchannel  = chansRef;
>     allData_preprosses = ft_preprocessing(cfg, datall);
>
>  Thanks again!
>
>  Emilie
>
>  On 13 déc. 2014, at 14:49, Haiteng Jiang <haiteng.jiang at gmail.com> wrote:
>
>  Hi  Emilie,
>      You can read  each channel and then  down sample.   In the end , you
> append all the channel. For more information and how to , you can refer to
> the FAQ:
> http://fieldtrip.fcdonders.nl/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory
> .
>
>        Hope this helps,
>             Haiteng
>
> On 13 December 2014 at 12:00, <fieldtrip-request at science.ru.nl> wrote:
>>
>> Send fieldtrip mailing list submissions to
>>         fieldtrip at science.ru.nl
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> or, via email, send a message with subject or body 'help' to
>>         fieldtrip-request at science.ru.nl
>>
>> You can reach the person managing the list at
>>         fieldtrip-owner at science.ru.nl
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of fieldtrip digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Correction baseline : removing the mean value or high-pass
>>       filtering ? (Kristensen Emmanuelle)
>>    2. Question about how to reduce the file size (Caspar, Emilie)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 12 Dec 2014 15:49:10 +0100
>> From: Kristensen Emmanuelle
>>         <emmanuelle.kristensen at gipsa-lab.grenoble-inp.fr>
>> To: FieldTrip discussion list <fieldtrip at science.ru.nl>
>> Subject: [FieldTrip] Correction baseline : removing the mean value or
>>         high-pass filtering ?
>> Message-ID: <548B0066.3090309 at gipsa-lab.grenoble-inp.fr>
>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>>
>> Hi,
>> I would need your opinion, please.
>> Generally, the baseline correction means to remove a mean value from
>> each epoch. I am wondering if a high pass filter, applied on the whole
>> recorded signal, can perform the baseline correction.
>> What do you think?
>> Regards
>> Emmanuelle
>>
>> --
>> Emmanuelle KRISTENSEN
>> Doctorante au Gipsa-Lab
>> Equipe VIBS
>>
>>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141212/1ceca823/attachment-0001.html
>> >
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Fri, 12 Dec 2014 22:24:17 +0000
>> From: "Caspar, Emilie" <e.caspar at ucl.ac.uk>
>> To: "fieldtrip at science.ru.nl" <fieldtrip at science.ru.nl>
>> Subject: [FieldTrip] Question about how to reduce the file size
>> Message-ID: <484FAA32-F84A-4BD2-8928-C07265183751 at live.ucl.ac.uk>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Dear Fieltrippers,
>> I did a pilot study on one participant today. Now that I'm trying to
>> analyze my data, I realize that the size file is too big for my computer
>> (size = 3Gb). Even after one hour, the filters (high pass + low pass) were
>> not yet achieved.
>>
>> So I would like to see how to reduce the size of my sample BEFORE the
>> filters.
>>
>> I know that there is "ft_resampledata", and I did it to reduce the actual
>> sample rate (= 2048) to 256. However, even with this procedure my computer
>> is crashing (even with 16 Go RAM). In addition, I'm not sure that I can
>> resample before filtering (I read different informations).
>>
>> Another way I was thinking about was to pre-select electrodes that I need
>> (only 6 electrodes on 64). But here I have two questions:
>> - Can I pre-process only some electrodes? Does it really reduce size for
>> next preprocesses?
>> - Is this the correct way to ask? As it crashes, not sure it works.
>>
>>     cfg             = [];
>>     cfg.dataset     = [ file.name];
>>     cfg.channel     = 'B5', 'B6', 'B15', 'B16';
>>     allData_prepross = ft_preprocessing(cfg);
>>     cfg.resamplefs = 256;
>>     DataSample = ft_resampledata(cfg, allData_prepross)
>>
>>
>> I would appreciate pieces of advice!
>>
>> Many thanks :)
>>
>> Emilie
>>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141212/1fa62a62/attachment-0001.html
>> >
>>
>> ------------------------------
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>> End of fieldtrip Digest, Vol 49, Issue 12
>> *****************************************
>>
>
>
>  --
>   Haiteng Jiang
> PhD candidate
> Donders Institute for Brain, Cognition and Behaviour
> Neuronal Oscillations Group
> Computational Cognitive Neuroscience Lab
>  https://sites.google.com/site/haitengjiang/
>    _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141214/b2de5877/attachment.html>


More information about the fieldtrip mailing list