<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div dir="ltr"><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Sadly, no that doesn't work. I am still getting approximately 1800 trials with an epoch length of 64 samples and 200 trials with an epoch length of 63. Additionally, I've run into extra complications because some trials after downsampling now have the same start and/or end sample indices; this raises error messages later on.</div><div><br></div><div>I might just bite the bullet and epoch first then downsample. But if you have any other solutions I would love to try them!</div><div><br></div><div>Thanks!</div></div></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 19, 2019 at 12:00 PM Stephen Whitmarsh <<a href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div>Hi Jeremy,</div><div dir="auto"><br></div><div dir="auto">A quick reply on your first question during lunch: round(trl/8+0.5)?</div><div dir="auto"><br></div><div dir="auto">Cheers,</div><div dir="auto">s<br><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">On Fri, 19 Jul 2019, 12:35 Jeremy Pulmano, <<a href="mailto:jpulmano@princeton.edu" target="_blank">jpulmano@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Stephen,<div><br></div><div>Thank you, this is very helpful! However, I am encountering an issue when attempting to divide the cfg.trl by the same ratio (1024/128 = 8) and rounding them off as integers; this results in different sized epochs across trials. In my case, some are 63 samples in length and others are 64 samples in length because of the rounding. Any ideas on how to avoid this? I was thinking that I can simply add one sample to those that become 63 in length but I'm worried that it will skew the data.</div><div><br></div><div>In terms of the trial function, I have been doing some experimenting upon writing my own, but unfortunately, I've encountered a few roadblocks. First, the function ft_read_event only works with the original dataset and therefore only identifies the epochs in the <b>raw</b> <b>data</b>. Thus, I cannot seem to find a way to identify the triggers in the downsampled data.</div><div><br></div><div>In terms of your second point, yes it is definitely possible to resample after epoching. In my case, however, I have been instructed that downsampling <i>before</i> epoching is the best practice for the dataset I am working with.</div><div><br></div><div>Any further suggestions? Thanks again for your help.</div><div>J</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 18, 2019 at 4:54 PM Stephen Whitmarsh <<a href="mailto:stephen.whitmarsh@gmail.com" rel="noreferrer" target="_blank">stephen.whitmarsh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Jeremy,</div><div><br></div><div>Yes, it seems likely that that would be the problem.</div><div><br></div><div>I see a couple of solutions:</div><div><br></div><div>1) Take your cfg.trl (trialdefinition which contains the start, end and offset sample) which you get out of ft_definetrial, and divide them with the same ratio as you did your data (and round them off to integers).</div><div><br></div><div>2) Resample your trial data after epoching. Honestly not 100% sure this is supported, but only one way to find out :-)</div><div><br></div><div>3a) Make your own trialfun*, i.e. function to create your own trl (trial definition), and that uses the appropriate samplerate (to be found in the datastructure field .fsample). An example on making your own trialfun can be found here: <a href="http://www.fieldtriptoolbox.org/tutorial/preprocessing/" rel="noreferrer" target="_blank">http://www.fieldtriptoolbox.org/tutorial/preprocessing/</a>. You can also check <a href="http://www.fieldtriptoolbox.org/walkthrough/" rel="noreferrer" target="_blank">http://www.fieldtriptoolbox.org/walkthrough/</a></div><div><br></div><div>3b) Use your fancy corrected trl in combination with ft_redefinetrial to epoch your resampled data.<br></div><div><br></div><div>*) IMHO, creating your own trialfun pays off in the long run, as it allows you to be much flexible, and e.g. enter response times and those kind of things in extra columns in the .trl, which will then be carried trhough your data in a .trialinfo field.</div><div><br></div><div>Good fieldtripping!</div><div>Stephen<br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 18 Jul 2019 at 16:59, Jeremy Pulmano <<a href="mailto:jpulmano@princeton.edu" rel="noreferrer" target="_blank">jpulmano@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Stephen,<div><br></div><div>Thanks so much for your reply. Apologies for the lack of specificity; what I meant to say is that out of 2000 trials, around 1500 of them contain NaNs after epoching. Thus, when I try visual artifact rejection, I only see 500 trials and the other 1500 are omitted completely.</div><div><br></div><div>The NaNs appear after epoching. I think you're correct in identifying a mistake with the sampling rates. How can I specify the new sampling frequency in the trial definition? I think there might also be an issue because FieldTrip reads the old sampling frequency from the headerfile (I am using Biosemi .bdf files).</div><div><br></div><div>Here is a snippet of the code. I am also attaching it to this email for easier reference. Thanks again.</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">%% Setup code (abridged)<br><br>filename = 'example';<br>header = ft_read_header(filename);<br>new_fs = 128;<br>prestim = 0.1;<br>poststim = 0.4;<br>trigger_codes = [ 1 2 3 ];<br><br>%% High pass filter<br><br>% Define trial for all the continuous data<br>cfg = [];<br>cfg.dataset = filename;<br>cfg.headerfile = header;<br>cfg.trialdef.triallength = Inf;<br>cfg.trialdef.ntrials = 1; <br><br>cfg = ft_definetrial(cfg);<br><br>% Preprocess with high pass filter<br>order = 100;<br>cfg.hpfilter = 'yes';<br>cfg.hpfilttype = 'fir';<br>cfg.hpfreq = 0.1;<br>cfg.hpfiltord = order;<br><br>data = ft_preprocessing(cfg);<br>data_hp = data;<br><br>%% Downsampling<br><br>fprintf('\nDownsampling\n');<br><br>cfg = [];<br>cfg.resamplefs = new_fs;<br><br>data = ft_resampledata(cfg, data);<br>data_res = data;<br><br>%% Epoch (NaNs appear after this block)<br><br>% Now define the ACTUAL trial<br>cfg = [];<br>cfg.continuous = 'yes';<br>cfg.dataset = filename; <br>cfg.trialdef.prestim = prestim;<br>cfg.trialdef.poststim = poststim;<br>cfg.trialdef.eventtype = 'STATUS';<br>cfg.trialdef.eventvalue = trigger_codes<br><br>cfg = ft_definetrial(cfg);<br><br>data = ft_redefinetrial(cfg, data);<br>data_ep = data;<br></blockquote><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 18, 2019 at 3:06 PM Stephen Whitmarsh <<a href="mailto:stephen.whitmarsh@gmail.com" rel="noreferrer" target="_blank">stephen.whitmarsh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Dear Jeremy,</div><div><br></div><div>Welcome to the FieldTrip community!</div><div><br></div><div>Could you be more specific about "the data looks incorrect in the end.", and perhaps share a screenshot if informative? Sharing the relevant part of your script would be helpful too. For more info about how to get the best response to your questions, please take a look at: <a href="http://www.fieldtriptoolbox.org/faq/how_to_ask_good_questions_to_the_community/" rel="noreferrer" target="_blank">http://www.fieldtriptoolbox.org/faq/how_to_ask_good_questions_to_the_community/</a>
</div><div><br></div><div>Have you checked where the NaN's in the data are, and whether they appear before or after downsampling?</div><div></div><div></div><div><br></div><div>Just an shot in the dark, but have you perhaps defined your trials
for epoching
(i.e. the samples in the .trl) based on the old samplerate?</div><div><div><br></div><div>Cheers,</div><div>Stephen<br></div><div><br></div><div><br></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 18 Jul 2019 at 12:35, Jeremy Pulmano <<a href="mailto:jpulmano@princeton.edu" rel="noreferrer" target="_blank">jpulmano@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div dir="ltr" class="gmail-m_-3316590610200037212m_4542011861123379288gmail-m_8950339527473855572gmail-m_-4913603565702056737gmail-m_-5584369551286859102gmail-m_-2691078732228537235gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"></div></div></div></div></div></div></div></div></div></div>Hi,<div><br></div><div>I've realized that the warning message often comes right after epoching (using ft_definetrial and then ft_redefinetrial). It seems like it successfully breaks the data into segments but still contains NaNs for whatever reason. Here are more details about the pipeline so far:</div><div><ol><li style="margin-left:15px">High pass filtering (0.1 Hz / FIR / order = 100)</li><li style="margin-left:15px">Downsampling (1024 to 128 Hz)</li><li style="margin-left:15px">Epoching (prestim 0.1, poststim 0.4)</li><li style="margin-left:15px">Base-line correction [ -0.1 0 ]</li><li style="margin-left:15px">Re-referencing (common avg.)</li><li style="margin-left:15px">ICA / component rejection </li><li style="margin-left:15px">Visual artifact removal</li><li style="margin-left:15px">Low pass filter (30 Hz)</li></ol><div>Any further help would be greatly appreciated.</div></div><div><br></div><div>Thanks,</div><div>Jeremy</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 17, 2019 at 4:57 PM Jeremy Pulmano <<a href="mailto:jpulmano@princeton.edu" rel="noreferrer" target="_blank">jpulmano@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto">Hi Erika,</div></div><div dir="auto"><br></div><div dir="auto">The original sampling rate is 1024 Hz and the new sampling rate is 128 Hz.</div><div dir="auto"><br></div><div dir="auto">Thanks,</div><div dir="auto">J</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 17, 2019 at 4:49 PM Erika Puiutta <<a href="mailto:erika.puiutta@uni-oldenburg.de" rel="noreferrer" target="_blank">erika.puiutta@uni-oldenburg.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div></div><div>Hey J,</div><div><br></div><div>this is just an educated guess since I don't really know how fieldtrip does the downsampling, but what is your sampling frequency and the sampling frequency you sample down to? Is it possible that your downsampling frequency is not a harmonic of your original sampling frequency (say 1000Hz to 300Hz)? Maybe there are NaNs for the new samples where fieldtrip can't find a corresponding one from the old sampling frequency?</div><div><br></div><div>Best of luck,</div><div>Erika</div></div><div dir="auto"><div><br>Am 17.07.2019 um 17:26 schrieb Jeremy Pulmano <<a href="mailto:jpulmano@princeton.edu" rel="noreferrer" target="_blank">jpulmano@princeton.edu</a>>:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div>Hi all,</div><div><br></div><div>I am new to both neuroscience and FieldTrip and am trying to debug my preprocessing pipeline of EEG data (biosemi .bdf files). These are the current steps:</div><div><ol><li style="margin-left:15px">High pass filtering</li><li style="margin-left:15px">Downsampling</li><li style="margin-left:15px">Epoching</li><li style="margin-left:15px">Base-line correction</li><li style="margin-left:15px">Re-referencing</li><li style="margin-left:15px">ICA / component rejection</li><li style="margin-left:15px">Visual artifact removal</li><li style="margin-left:15px">Low pass filter (I save the low pass filter for the end so that ICA yields better time course plots).</li></ol><div>However, after the first three steps (high pass, downsample, epoch), I continue to get the warning message: "Warning: data contains NaN values, no filtering or preprocessing applied," starting at baseline correction. It repeats itself over and over again. If I ignore these errors, the data looks incorrect in the end. Why does this happen, and how can I resolve this?</div></div><div><br></div><div>Any tips would be appreciated.</div><div><br></div><div>Thanks,</div><div>J</div></div>
</div></blockquote></div><div dir="auto"><blockquote type="cite"><div><span>_______________________________________________</span><br><span>fieldtrip mailing list</span><br><span><a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a></span><br><span><a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a></span><br></div></blockquote></div>_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div></div>-- <br><div dir="ltr" class="gmail-m_-3316590610200037212m_4542011861123379288gmail-m_8950339527473855572gmail-m_-4913603565702056737gmail-m_-5584369551286859102gmail-m_-2691078732228537235gmail-m_1422488924757768724gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br><table style="font-family:arial,sans-serif;color:rgb(80,0,80)" cellspacing="0" cellpadding="0" border="0"><tbody><tr><td style="padding-right:15px;padding-left:7px;border-right:1px dotted rgb(135,127,116)" valign="top"><img src="https://docs.google.com/uc?export=download&id=1-cTSY4Ow0kUKbhb45irGhGCGli8wFwgZ&revid=0B4aqG7IAPQqtWVpyeU8vUFBIS1A3RzdaeUh2NzBncThkVFdrPQ" style="margin-right: 0px;" width="56" height="71"></td><td style="padding-left:15px" width="400" valign="top"><font face="garamond, serif"><font color="#222222"><b>Jeremy Pulmano</b></font><br><span style="color:rgb(255,153,0)"><b>Princeton University Class of 2021</b></span><font color="#000000"><br>B.S.E. Computer Science<br></font><font color="#000000">e: <a href="mailto:jpulmano@princeton.edu" style="color:rgb(17,85,204)" rel="noreferrer" target="_blank">jpulmano@princeton.edu</a> | c: (862) 220-5903</font></font></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div></div></div>
_______________________________________________<br>
fieldtrip mailing list<br>
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" rel="noreferrer" target="_blank">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br>
<a href="https://doi.org/10.1371/journal.pcbi.1002202" rel="noreferrer" target="_blank">https://doi.org/10.1371/journal.pcbi.1002202</a><br>
</blockquote></div>