[FieldTrip] Question about how to reduce the file size (Caspar, Emilie)

Caspar, Emilie e.caspar at ucl.ac.uk
Mon Dec 15 10:03:38 CET 2014


It works indeed, I have understood my mistake.
Thank you!




On 14 déc. 2014, at 22:13, Arjen Stolk <a.stolk8 at gmail.com<mailto:a.stolk8 at gmail.com>> wrote:

Hi Emilie,

It seems from the code pasted that you're still trying to read all channels in at once:
cfgp.channel = 'all';

Try reading in & downsampling one channel at a time, as Haiteng suggested with the following link:
http://fieldtrip.fcdonders.nl/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory

Hope that solves it for you,
Arjen

2014-12-14 21:44 GMT+01:00 Caspar, Emilie <e.caspar at ucl.ac.uk<mailto:e.caspar at ucl.ac.uk>>:
Dear Haiteng

Thank you for your suggestion. I tried but again, the downsample with this procedure takes more than 40 minutes (I gave up after this time period and stopped the script, the computer was crashing).

Perhaps one could tell if the following is correct? I'm maybe wrong somewhere.

chansAll    = 1:68;%{'CPz', 'Cz', 'FCz', 'Fz'}
chansEEG    = 1:64;
nchans=length(chansEEG);

display('downsample each channel')

for i=1:nchans
  cfgp         = [];
  cfgp.dataset     = [ file.name<http://file.name/>];
  cfgp.channel = 'all';
  datp         = ft_preprocessing(cfgp);

  cfgr            = [];
  cfgr.resamplefs = 512;
  datr{i}         = ft_resampledata(cfgr, datp);

  clear datp

cfg = [];
datall = ft_appenddata(cfg, datr{:});

display('Filters')

    cfg = [];
    cfg.bpfilter    = 'yes';
    cfg.bpfreq      = bpfilterRange;
    cfg.bpfiltord   = bpfilterOrder;
    cfg.reref       = 'yes';
    cfg.refchannel  = chansRef;
    allData_preprosses = ft_preprocessing(cfg, datall);

Thanks again!

Emilie

On 13 déc. 2014, at 14:49, Haiteng Jiang <haiteng.jiang at gmail.com<mailto:haiteng.jiang at gmail.com>> wrote:

Hi  Emilie,
     You can read  each channel and then  down sample.   In the end , you append all the channel. For more information and how to , you can refer to the FAQ: http://fieldtrip.fcdonders.nl/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory.

       Hope this helps,
            Haiteng

On 13 December 2014 at 12:00, <fieldtrip-request at science.ru.nl<mailto:fieldtrip-request at science.ru.nl>> wrote:
Send fieldtrip mailing list submissions to
        fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>

To subscribe or unsubscribe via the World Wide Web, visit
        http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
or, via email, send a message with subject or body 'help' to
        fieldtrip-request at science.ru.nl<mailto:fieldtrip-request at science.ru.nl>

You can reach the person managing the list at
        fieldtrip-owner at science.ru.nl<mailto:fieldtrip-owner at science.ru.nl>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of fieldtrip digest..."


Today's Topics:

   1. Correction baseline : removing the mean value or high-pass
      filtering ? (Kristensen Emmanuelle)
   2. Question about how to reduce the file size (Caspar, Emilie)


----------------------------------------------------------------------

Message: 1
Date: Fri, 12 Dec 2014 15:49:10 +0100
From: Kristensen Emmanuelle
        <emmanuelle.kristensen at gipsa-lab.grenoble-inp.fr<mailto:emmanuelle.kristensen at gipsa-lab.grenoble-inp.fr>>
To: FieldTrip discussion list <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
Subject: [FieldTrip] Correction baseline : removing the mean value or
        high-pass filtering ?
Message-ID: <548B0066.3090309 at gipsa-lab.grenoble-inp.fr<mailto:548B0066.3090309 at gipsa-lab.grenoble-inp.fr>>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Hi,
I would need your opinion, please.
Generally, the baseline correction means to remove a mean value from
each epoch. I am wondering if a high pass filter, applied on the whole
recorded signal, can perform the baseline correction.
What do you think?
Regards
Emmanuelle

--
Emmanuelle KRISTENSEN
Doctorante au Gipsa-Lab
Equipe VIBS



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141212/1ceca823/attachment-0001.html>

------------------------------

Message: 2
Date: Fri, 12 Dec 2014 22:24:17 +0000
From: "Caspar, Emilie" <e.caspar at ucl.ac.uk<mailto:e.caspar at ucl.ac.uk>>
To: "fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>" <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
Subject: [FieldTrip] Question about how to reduce the file size
Message-ID: <484FAA32-F84A-4BD2-8928-C07265183751 at live.ucl.ac.uk<mailto:484FAA32-F84A-4BD2-8928-C07265183751 at live.ucl.ac.uk>>
Content-Type: text/plain; charset="us-ascii"

Dear Fieltrippers,
I did a pilot study on one participant today. Now that I'm trying to analyze my data, I realize that the size file is too big for my computer (size = 3Gb). Even after one hour, the filters (high pass + low pass) were not yet achieved.

So I would like to see how to reduce the size of my sample BEFORE the filters.

I know that there is "ft_resampledata", and I did it to reduce the actual sample rate (= 2048) to 256. However, even with this procedure my computer is crashing (even with 16 Go RAM). In addition, I'm not sure that I can resample before filtering (I read different informations).

Another way I was thinking about was to pre-select electrodes that I need (only 6 electrodes on 64). But here I have two questions:
- Can I pre-process only some electrodes? Does it really reduce size for next preprocesses?
- Is this the correct way to ask? As it crashes, not sure it works.

    cfg             = [];
    cfg.dataset     = [ file.name<http://file.name/>];
    cfg.channel     = 'B5', 'B6', 'B15', 'B16';
    allData_prepross = ft_preprocessing(cfg);
    cfg.resamplefs = 256;
    DataSample = ft_resampledata(cfg, allData_prepross)


I would appreciate pieces of advice!

Many thanks :)

Emilie


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141212/1fa62a62/attachment-0001.html>

------------------------------

_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

End of fieldtrip Digest, Vol 49, Issue 12
*****************************************


--
Haiteng Jiang
PhD candidate
Donders Institute for Brain, Cognition and Behaviour
Neuronal Oscillations Group
Computational Cognitive Neuroscience Lab
https://sites.google.com/site/haitengjiang/
_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip


_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
_______________________________________________
fieldtrip mailing list
fieldtrip at donders.ru.nl<mailto:fieldtrip at donders.ru.nl>
http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20141215/7acc58fe/attachment-0002.html>


More information about the fieldtrip mailing list