[FieldTrip] Dealing with warning message "data contians NaN values"

Stephen Whitmarsh stephen.whitmarsh at gmail.com
Thu Jul 18 17:51:37 CEST 2019 

Hi Jeremy,

Yes, it seems likely that that would be the problem.

I see a couple of solutions:

1) Take your cfg.trl (trialdefinition which contains the start, end and
offset sample) which you get out of ft_definetrial, and divide them with
the same ratio as you did your data (and round them off to integers).

2) Resample your trial data after epoching. Honestly not 100% sure this is
supported, but only one way to find out :-)

3a) Make your own trialfun*, i.e. function to create your own trl (trial
definition), and that uses the appropriate samplerate (to be found in the
datastructure field .fsample). An example on making your own trialfun can
be found here: http://www.fieldtriptoolbox.org/tutorial/preprocessing/. You
can also check http://www.fieldtriptoolbox.org/walkthrough/

3b) Use your fancy corrected trl in combination with ft_redefinetrial to
epoch your resampled data.

*) IMHO, creating your own trialfun pays off in the long run, as it allows
you to be much flexible, and e.g. enter response times and those kind of
things in extra columns in the .trl, which will then be carried trhough
your data in a .trialinfo field.

Good fieldtripping!
Stephen



On Thu, 18 Jul 2019 at 16:59, Jeremy Pulmano <jpulmano at princeton.edu> wrote:

> Hi Stephen,
>
> Thanks so much for your reply. Apologies for the lack of specificity; what
> I meant to say is that out of 2000 trials, around 1500 of them contain NaNs
> after epoching. Thus, when I try visual artifact rejection, I only see 500
> trials and the other 1500 are omitted completely.
>
> The NaNs appear after epoching. I think you're correct in identifying a
> mistake with the sampling rates. How can I specify the new sampling
> frequency in the trial definition? I think there might also be an issue
> because FieldTrip reads the old sampling frequency from the headerfile (I
> am using Biosemi .bdf files).
>
> Here is a snippet of the code. I am also attaching it to this email for
> easier reference. Thanks again.
>
> %% Setup code (abridged)
>>
>> filename = 'example';
>> header = ft_read_header(filename);
>> new_fs = 128;
>> prestim = 0.1;
>> poststim = 0.4;
>> trigger_codes = [ 1 2 3 ];
>>
>> %% High pass filter
>>
>> % Define trial for all the continuous data
>> cfg = [];
>> cfg.dataset = filename;
>> cfg.headerfile = header;
>> cfg.trialdef.triallength = Inf;
>> cfg.trialdef.ntrials = 1;
>>
>> cfg = ft_definetrial(cfg);
>>
>> % Preprocess with high pass filter
>> order = 100;
>> cfg.hpfilter = 'yes';
>> cfg.hpfilttype = 'fir';
>> cfg.hpfreq = 0.1;
>> cfg.hpfiltord = order;
>>
>> data = ft_preprocessing(cfg);
>> data_hp = data;
>>
>> %% Downsampling
>>
>> fprintf('\nDownsampling\n');
>>
>> cfg = [];
>> cfg.resamplefs = new_fs;
>>
>> data = ft_resampledata(cfg, data);
>> data_res = data;
>>
>> %% Epoch (NaNs appear after this block)
>>
>> % Now define the ACTUAL trial
>> cfg = [];
>> cfg.continuous = 'yes';
>> cfg.dataset = filename;
>> cfg.trialdef.prestim = prestim;
>> cfg.trialdef.poststim = poststim;
>> cfg.trialdef.eventtype = 'STATUS';
>> cfg.trialdef.eventvalue = trigger_codes
>>
>> cfg = ft_definetrial(cfg);
>>
>> data = ft_redefinetrial(cfg, data);
>> data_ep = data;
>>
>
>
> On Thu, Jul 18, 2019 at 3:06 PM Stephen Whitmarsh <
> stephen.whitmarsh at gmail.com> wrote:
>
>> Dear Jeremy,
>>
>> Welcome to the FieldTrip community!
>>
>> Could you be more specific about "the data looks incorrect in the end.",
>> and perhaps share a screenshot if informative? Sharing the relevant part of
>> your script would be helpful too. For more info about how to get the best
>> response to your questions, please take a look at:
>> http://www.fieldtriptoolbox.org/faq/how_to_ask_good_questions_to_the_community/
>>
>> Have you checked where the NaN's in the data are, and whether they appear
>> before or after downsampling?
>>
>> Just an shot in the dark, but have you perhaps defined your trials for
>> epoching (i.e. the samples in the .trl) based on the old samplerate?
>>
>> Cheers,
>> Stephen
>>
>>
>>
>>
>>
>> On Thu, 18 Jul 2019 at 12:35, Jeremy Pulmano <jpulmano at princeton.edu>
>> wrote:
>>
>>> Hi,
>>>
>>> I've realized that the warning message often comes right after epoching
>>> (using ft_definetrial and then ft_redefinetrial). It seems like it
>>> successfully breaks the data into segments but still contains NaNs for
>>> whatever reason. Here are more details about the pipeline so far:
>>>
>>>    1. High pass filtering (0.1 Hz / FIR / order = 100)
>>>    2. Downsampling (1024 to 128 Hz)
>>>    3. Epoching (prestim 0.1, poststim 0.4)
>>>    4. Base-line correction [ -0.1 0 ]
>>>    5. Re-referencing (common avg.)
>>>    6. ICA / component rejection
>>>    7. Visual artifact removal
>>>    8. Low pass filter (30 Hz)
>>>
>>> Any further help would be greatly appreciated.
>>>
>>> Thanks,
>>> Jeremy
>>>
>>> On Wed, Jul 17, 2019 at 4:57 PM Jeremy Pulmano <jpulmano at princeton.edu>
>>> wrote:
>>>
>>>> Hi Erika,
>>>>
>>>> The original sampling rate is 1024 Hz and the new sampling rate is 128
>>>> Hz.
>>>>
>>>> Thanks,
>>>> J
>>>>
>>>> On Wed, Jul 17, 2019 at 4:49 PM Erika Puiutta <
>>>> erika.puiutta at uni-oldenburg.de> wrote:
>>>>
>>>>> Hey J,
>>>>>
>>>>> this is just an educated guess since I don't really know how fieldtrip
>>>>> does the downsampling, but what is your sampling frequency and the sampling
>>>>> frequency you sample down to? Is it possible that your downsampling
>>>>> frequency is not a harmonic of your original sampling frequency (say 1000Hz
>>>>> to 300Hz)? Maybe there are NaNs for the new samples where fieldtrip can't
>>>>> find a corresponding one from the old sampling frequency?
>>>>>
>>>>> Best of luck,
>>>>> Erika
>>>>>
>>>>> Am 17.07.2019 um 17:26 schrieb Jeremy Pulmano <jpulmano at princeton.edu
>>>>> >:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I am new to both neuroscience and FieldTrip and am trying to debug my
>>>>> preprocessing pipeline of EEG data (biosemi .bdf files). These are the
>>>>> current steps:
>>>>>
>>>>>    1. High pass filtering
>>>>>    2. Downsampling
>>>>>    3. Epoching
>>>>>    4. Base-line correction
>>>>>    5. Re-referencing
>>>>>    6. ICA / component rejection
>>>>>    7. Visual artifact removal
>>>>>    8. Low pass filter (I save the low pass filter for the end so that
>>>>>    ICA yields better time course plots).
>>>>>
>>>>> However, after the first three steps (high pass, downsample, epoch), I
>>>>> continue to get the warning message: "Warning: data contains NaN values, no
>>>>> filtering or preprocessing applied," starting at baseline correction. It
>>>>> repeats itself over and over again. If I ignore these errors, the data
>>>>> looks incorrect in the end. Why does this happen, and how can I resolve
>>>>> this?
>>>>>
>>>>> Any tips would be appreciated.
>>>>>
>>>>> Thanks,
>>>>> J
>>>>>
>>>>> _______________________________________________
>>>>> fieldtrip mailing list
>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>>>
>>>>> _______________________________________________
>>>>> fieldtrip mailing list
>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>>>
>>>> --
>>>>
>>>> *Jeremy Pulmano*
>>>> *Princeton University Class of 2021*
>>>> B.S.E. Computer Science
>>>> e: jpulmano at princeton.edu | c: (862) 220-5903
>>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://doi.org/10.1371/journal.pcbi.1002202
>>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190718/00ca340a/attachment-0002.html>
-------------- next part --------------
_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://doi.org/10.1371/journal.pcbi.1002202

More information about the fieldtrip mailing list