<html><head></head><body><div style="color:#000; background-color:#fff; font-family:garamond, new york, times, serif;font-size:16px"><div id="yui_3_16_0_1_1450430167445_28271"><span></span></div><div id="yui_3_16_0_1_1450430167445_28272"><div id="yui_3_16_0_1_1450430167445_28685"> Thanks Steve for your reply.</div><div dir="ltr" id="yui_3_16_0_1_1450430167445_28684">I think your code for segmenting the data is the way to go for me ! :) And I do appreciate you gave me an example. Thanks</div><div id="yui_3_16_0_1_1450430167445_28918"><br></div><div dir="ltr" id="yui_3_16_0_1_1450430167445_28925">However I tried it with two different cfg.trl = [1 1000 0]; and cfg.trl = [1 10000 0]; to see the results, and it looked the same! <br></div><div id="yui_3_16_0_1_1450430167445_28919"><br></div><div id="yui_3_16_0_1_1450430167445_28703">So this is what I get: <br></div><div id="yui_3_16_0_1_1450430167445_28704"><br></div><div id="yui_3_16_0_1_1450430167445_28689"><br></div><div id="yui_3_16_0_1_1450430167445_28705" dir="ltr"> hdr: [1x1 struct]<br class="" id="yui_3_16_0_1_1450430167445_28691"> label: {51x1 cell}<br class="" id="yui_3_16_0_1_1450430167445_28693"> time: {1x50710 cell}<br class="" id="yui_3_16_0_1_1450430167445_28695"> trial: {1x50710 cell}<br class="" id="yui_3_16_0_1_1450430167445_28697"> fsample: 1024<br class="" id="yui_3_16_0_1_1450430167445_28699"> sampleinfo: [50710x2 double]<br class="" id="yui_3_16_0_1_1450430167445_28701"> cfg: [1x1 struct]</div><div id="yui_3_16_0_1_1450430167445_29431" dir="ltr"><br></div><div id="yui_3_16_0_1_1450430167445_29477" dir="ltr"><br></div><div id="yui_3_16_0_1_1450430167445_29476" dir="ltr">so the actual data which is in data.trial is this in both cases! <br></div><div id="yui_3_16_0_1_1450430167445_29611" dir="ltr">size(data.trial)<br class="" id="yui_3_16_0_1_1450430167445_29613"><br class="" id="yui_3_16_0_1_1450430167445_29615">ans =<br class="" id="yui_3_16_0_1_1450430167445_29617"><br class="" id="yui_3_16_0_1_1450430167445_29619"> 1 50710</div><div id="yui_3_16_0_1_1450430167445_29784" dir="ltr"><br></div><div id="yui_3_16_0_1_1450430167445_35274" dir="ltr"><br></div><div dir="ltr">How could that happen if you are segmenting it in two different chunks?</div><div id="yui_3_16_0_1_1450430167445_35277" dir="ltr"><br></div><div id="yui_3_16_0_1_1450430167445_35280" dir="ltr">Best regards<br></div><div id="yui_3_16_0_1_1450430167445_35276" dir="ltr"><br></div></div><div id="yui_3_16_0_1_1450430167445_28279" class="signature"><div id="yui_3_16_0_1_1450430167445_28278"><font id="yui_3_16_0_1_1450430167445_28277" color="#000000"><div id="yui_3_16_0_1_1450430167445_28276" style="font-family:'bookman old style', 'new york', times, serif;margin-top:0cm;margin-right:0cm;margin-bottom:0pt;margin-left:0cm;" class="MsoPlainText"><span id="yui_3_16_0_1_1450430167445_28275" style=""><font id="yui_3_16_0_1_1450430167445_28274" face="times new roman, new york, times, serif"><font id="yui_3_16_0_1_1450430167445_28273" size="2">___________________________________________________</font></font></span></div> <div id="yui_3_16_0_1_1450430167445_28280" style="font-family:'bookman old style', 'new york', times, serif;margin-top:0cm;margin-right:0cm;margin-bottom:0pt;margin-left:0cm;" class="MsoPlainText"><span id="yui_3_16_0_1_1450430167445_28283" style=""><font id="yui_3_16_0_1_1450430167445_28282" face="times new roman, new york, times, serif"><font id="yui_3_16_0_1_1450430167445_28281" size="2">Leila Ayoubian</font></font></span></div> <div id="yui_3_16_0_1_1450430167445_28284" style="margin-top:0cm;margin-right:0cm;margin-bottom:0pt;margin-left:0cm;" class="MsoPlainText"><font class="Apple-style-span" size="2" face="'times new roman', 'new york', times, serif"><br></font></div></font></div><div style="MARGIN:0cm 0cm 0pt;" class="MsoPlainText"><font color="#000000"><font style="" face="bookman old style, new york, times, serif"><font face="bookman old style, new york, times, serif"><font color="#000000"><span style="color:rgb(84, 140, 214);font-size:9pt;font-family:Arial;" lang="EN-US"><span style="font-size:10pt;font-family:sans-serif;"><font face="bookman old style, new york, times, serif"></font></span></span></font></font></font></font></div></div><br> <div id="yui_3_16_0_1_1450430167445_28285" class="qtdSeparateBR"><br><br></div><div style="display: block;" id="yui_3_16_0_1_1450430167445_28289" class="yahoo_quoted"> <div id="yui_3_16_0_1_1450430167445_28288" style="font-family: garamond, new york, times, serif; font-size: 16px;"> <div id="yui_3_16_0_1_1450430167445_28287" style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div id="yui_3_16_0_1_1450430167445_28286" dir="ltr"> <font id="yui_3_16_0_1_1450430167445_28290" size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Stephen Politzer-Ahles <stephen.politzer-ahles@ling-phil.ox.ac.uk><br> <b><span style="font-weight: bold;">To:</span></b> fieldtrip@science.ru.nl <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, December 18, 2015 12:37 PM<br> <b id="yui_3_16_0_1_1450430167445_28923"><span id="yui_3_16_0_1_1450430167445_28922" style="font-weight: bold;">Subject:</span></b> Re: [FieldTrip] importing large edf data<br> </font> </div> <div id="yui_3_16_0_1_1450430167445_28449" class="y_msg_container"><br><div id="yiv6266347124"><div id="yui_3_16_0_1_1450430167445_28455" dir="ltr"><div id="yui_3_16_0_1_1450430167445_28454"><div id="yui_3_16_0_1_1450430167445_28453"><div id="yui_3_16_0_1_1450430167445_28452"><div id="yui_3_16_0_1_1450430167445_28451"><div id="yui_3_16_0_1_1450430167445_28450">Hello Leila,<br><br></div>I think there are a few possible options for you. If you're not interested in analyzing high frequencies, then you could import the data one sample at a time, downsampling each channel, and then concatenate the channels into one structure, as described in <a id="yui_3_16_0_1_1450430167445_28460" rel="nofollow" target="_blank" href="http://www.fieldtriptoolbox.org/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory">http://www.fieldtriptoolbox.org/faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory</a>. I don't know what your sampling rate was, but if your sampling rate was extremely high then this can save you a lot of space. For example, if you're only interested in frequencies up to high gamma (around 90-100 Hz), then your data only need to be sampled at around 250 Hz (per the Nyquist theorem, plus a little extra wiggle room), so if you sampled at e.g. 2500 Hz (which our EDF system does by default) or 5000 Hz then you have way more samples than necessary; downsampling like this will greatly reduce your array size.<br><br><br><br></div>If that is not an option, then you can read in a segment of data as a 'trial'. ft_preprocessing understands trial start and end times from cfg.trl, which minimally is an Nx3 matrix (N is the number of trials to read, and the columns are the start sample for each trial, the end sample for each trial, and the trigger value associated with the trial; you could just put 0 if there's no trigger). So let's say you want to read the first through thousandth sample of the recording, then you would set up your cfg as normal, then add:<br><br></div>cfg.trl = [1 1000 0];<br><br></div>and then use ft_preprocessing. <br><br></div>By way of example, here is code that reads in raw data as a series of 1-second "trials":<br><br>cfg = [];<br>cfg.dataset = 'C:\Users\Steve\Desktop\subj2.vhdr';<br>cfg.trialdef.triallength = 1; % see documentation for ft_definetrial for an explanation of these parameters<br>cfg.trialdef.ntrials = Inf;<br>cfg = ft_definetrial( cfg );<br><br>cfg.continuous = 'yes';<br>data = ft_preprocessing(cfg);<br><div id="yui_3_16_0_1_1450430167445_28546"><div id="yui_3_16_0_1_1450430167445_28545"><div id="yui_3_16_0_1_1450430167445_28544"><div id="yui_3_16_0_1_1450430167445_28543"><div id="yui_3_16_0_1_1450430167445_28542"><div id="yui_3_16_0_1_1450430167445_28541"><div id="yui_3_16_0_1_1450430167445_28540" class="yiv6266347124gmail_extra"><br clear="all"><div id="yui_3_16_0_1_1450430167445_28539"><div id="yui_3_16_0_1_1450430167445_28538" class="yiv6266347124gmail_signature"><div id="yui_3_16_0_1_1450430167445_28537" dir="ltr"><div id="yui_3_16_0_1_1450430167445_28536"><div id="yui_3_16_0_1_1450430167445_28535" dir="ltr"><div id="yui_3_16_0_1_1450430167445_28534"><div id="yui_3_16_0_1_1450430167445_28533" dir="ltr"><span id="yui_3_16_0_1_1450430167445_28532"><div id="yui_3_16_0_1_1450430167445_28531">After doing whatever you need to these trials, you could concatenate them back into continuous data (see <a rel="nofollow" target="_blank" href="http://mailman.science.ru.nl/pipermail/fieldtrip/2011-June/003971.html">http://mailman.science.ru.nl/pipermail/fieldtrip/2011-June/003971.html</a> for details).<br><br></div><div>Best,<br></div><div>Steve<br></div><div id="yui_3_16_0_1_1450430167445_31523"><br>---<br></div>Stephen Politzer-Ahles<br>University of Oxford<br>Language and Brain Lab<br>Faculty of Linguistics, Phonetics & Philology<br><a rel="nofollow" target="_blank" href="http://users.ox.ac.uk/~cpgl0080/">http://users.ox.ac.uk/~cpgl0080/</a></span></div></div></div></div></div></div></div>
<br><div id="yui_3_16_0_1_1450430167445_31486" class="yiv6266347124gmail_quote">On Fri, Dec 18, 2015 at 11:00 AM, <span id="yui_3_16_0_1_1450430167445_31521" dir="ltr"><<a id="yui_3_16_0_1_1450430167445_31520" rel="nofollow" ymailto="mailto:fieldtrip-request@science.ru.nl" target="_blank" href="mailto:fieldtrip-request@science.ru.nl">fieldtrip-request@science.ru.nl</a>></span> wrote:<br><blockquote id="yui_3_16_0_1_1450430167445_31488" class="yiv6266347124gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">Send fieldtrip mailing list submissions to<br>
<a rel="nofollow" ymailto="mailto:fieldtrip@science.ru.nl" target="_blank" href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a rel="nofollow" target="_blank" href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a rel="nofollow" ymailto="mailto:fieldtrip-request@science.ru.nl" target="_blank" href="mailto:fieldtrip-request@science.ru.nl">fieldtrip-request@science.ru.nl</a><br>
<br>
You can reach the person managing the list at<br>
<a rel="nofollow" ymailto="mailto:fieldtrip-owner@science.ru.nl" target="_blank" href="mailto:fieldtrip-owner@science.ru.nl">fieldtrip-owner@science.ru.nl</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of fieldtrip digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. importing large edf data (Leila Ayoubian)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 18 Dec 2015 09:43:43 +0000 (UTC)<br>
From: Leila Ayoubian <<a rel="nofollow" ymailto="mailto:leilayou_54@yahoo.com" target="_blank" href="mailto:leilayou_54@yahoo.com">leilayou_54@yahoo.com</a>><br>
To: "<a rel="nofollow" ymailto="mailto:fieldtrip@science.ru.nl" target="_blank" href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a>" <<a rel="nofollow" ymailto="mailto:fieldtrip@science.ru.nl" target="_blank" href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a>><br>
Subject: [FieldTrip] importing large edf data<br>
Message-ID:<br>
<<a rel="nofollow" ymailto="mailto:807603762.476165.1450431823359.JavaMail.yahoo@mail.yahoo.com" target="_blank" href="mailto:807603762.476165.1450431823359.JavaMail.yahoo@mail.yahoo.com">807603762.476165.1450431823359.JavaMail.yahoo@mail.yahoo.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi:Thanks for providing us with this amazing toolbox.<br>
I am new to fieldtrip. I am trying to import EEG data in the format of *.edf for the purpose of seizure detection for epileptic patients. I can't break up the data as you understand the detection has to have continuous data. However I could load section of the data at a time and clear what is already used and reload again.?<br>
So trying to load the data like this:<br>
cfg.dataset= ('mydata.edf');<br>
rawdata=ft_preprocessing(cfg);<br>
This is the error message I get :<br>
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%<br>
Error using zeros<br>
Requested 51x51927040 (19.7GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time<br>
and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.<br>
<br>
Error in read_edf (line 403)<br>
? dat = zeros(length(chanindx),nepochs*epochlength);<br>
<br>
Error in ft_read_data (line 622)<br>
??? dat = read_edf(filename, hdr, begsample, endsample, chanindx);<br>
<br>
Error in ft_preprocessing (line 566)<br>
????? dat = ft_read_data(cfg.datafile, 'header', hdr, 'begsample', begsample, 'endsample', endsample, 'chanindx', rawindx, 'checkboundary',<br>
????? strcmp(cfg.continuous, 'no'), 'dataformat', cfg.dataformat);<br>
<br>
Error in ReadFedfiles (line 8)<br>
rawdata=ft_preprocessing(cfg);%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%<br>
I tried the ft_redefinetrial, but since the input to this file is the output from ft-PREPROCESSING it gives me the same error.<br>
<br>
<br>
I also tried to use ft_read_data to choose the samples I want to load :?filename???? = 'mydata.edf'<br>
?hdr?? = ft_read_header(filename);<br>
?sample.start=1<br>
?sample.end=100<br>
dat = ft_read_data(filename,'sample.start','1','sample.end','100')<br>
and here is again the error message:<br>
Error using zeros<br>
Requested 51x51927040 (19.7GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a<br>
long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.<br>
<br>
Error in read_edf (line 403)<br>
? dat = zeros(length(chanindx),nepochs*epochlength);<br>
<br>
Error in ft_read_data (line 622)<br>
??? dat = read_edf(filename, hdr, begsample, endsample, chanindx);<br>
<br>
Error in ReadFedfiles (line 31)<br>
dat = ft_read_data(filename,'sample.start','1','sample.end','100')<br>
?<br>
>><br>
<br>
Could you please direct me as to which functions and in which order I should be using the functions to load data in smaller samples.Some examples would be useful.<br>
Thanks again for your support and assistance.<br>
We appreciate your effort.<br>
Kind regards<br>
<br>
___________________________________________________ Dr. Leila Ayoubian<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a id="yui_3_16_0_1_1450430167445_31519" rel="nofollow" target="_blank" href="http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151218/6dfa7ce1/attachment-0001.html">http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20151218/6dfa7ce1/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
fieldtrip mailing list<br>
<a rel="nofollow" ymailto="mailto:fieldtrip@donders.ru.nl" target="_blank" href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br>
<a rel="nofollow" target="_blank" href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<br>
End of fieldtrip Digest, Vol 61, Issue 14<br>
*****************************************<br>
</blockquote></div><br></div></div></div></div></div></div></div></div></div><br>_______________________________________________<br>fieldtrip mailing list<br><a ymailto="mailto:fieldtrip@donders.ru.nl" href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br><a href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip" target="_blank">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br><br></div> </div> </div> </div></div></body></html>