[FieldTrip] slow reading with ft_preprocessing of a fcdc_matbin file?

[jan-mathijs schoffelen]

Hi Ben,

I guess MATLAB needs to read in the whole datafile each time (for each channel). What about creating separate files for each channel?

JM


On Aug 29, 2013, at 1:53 PM, van Lier Ben wrote:

> Hi all,
> 
> I have a pretty big data file of 120 channels for 20min continuous at 20khz. clearly i need to downsample. to do so, i convert the original file to fcdc_matbin, creating a 22gb bin file with all the data.
> 
> then its time for downsampling as shown on http://fieldtrip.fcdonders.nl/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory
> 
> this is very slow. ft_preprocessing takes 230 seconds per channel (using 380 mb), doing nothing but just reading 1 channel. the subsequent downsampling with ft_resample to 1khz is surprisingly fast with only 4 seconds.
> 
> does that make any sense? it seems like a very long time to read in 380mb... :(
> 
> Best,
> Ben
> 
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

Jan-Mathijs Schoffelen, MD PhD 

Donders Institute for Brain, Cognition and Behaviour, 
Centre for Cognitive Neuroimaging,
Radboud University Nijmegen, The Netherlands

Max Planck Institute for Psycholinguistics,
Nijmegen, The Netherlands

J.Schoffelen at donders.ru.nl
Telephone: +31-24-3614793

http://www.hettaligebrein.nl

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20130829/c326a421/attachment-0002.html>