[FieldTrip] Dealing with warning message "data contians NaN values"

[Jeremy Pulmano]

Hi Stephen,

Thanks so much for your reply. Apologies for the lack of specificity; what
I meant to say is that out of 2000 trials, around 1500 of them contain NaNs
after epoching. Thus, when I try visual artifact rejection, I only see 500
trials and the other 1500 are omitted completely.

The NaNs appear after epoching. I think you're correct in identifying a
mistake with the sampling rates. How can I specify the new sampling
frequency in the trial definition? I think there might also be an issue
because FieldTrip reads the old sampling frequency from the headerfile (I
am using Biosemi .bdf files).

Here is a snippet of the code. I am also attaching it to this email for
easier reference. Thanks again.

%% Setup code (abridged)
>
> filename = 'example';
> header = ft_read_header(filename);
> new_fs = 128;
> prestim = 0.1;
> poststim = 0.4;
> trigger_codes = [ 1 2 3 ];
>
> %% High pass filter
>
> % Define trial for all the continuous data
> cfg = [];
> cfg.dataset = filename;
> cfg.headerfile = header;
> cfg.trialdef.triallength = Inf;
> cfg.trialdef.ntrials = 1;
>
> cfg = ft_definetrial(cfg);
>
> % Preprocess with high pass filter
> order = 100;
> cfg.hpfilter = 'yes';
> cfg.hpfilttype = 'fir';
> cfg.hpfreq = 0.1;
> cfg.hpfiltord = order;
>
> data = ft_preprocessing(cfg);
> data_hp = data;
>
> %% Downsampling
>
> fprintf('\nDownsampling\n');
>
> cfg = [];
> cfg.resamplefs = new_fs;
>
> data = ft_resampledata(cfg, data);
> data_res = data;
>
> %% Epoch (NaNs appear after this block)
>
> % Now define the ACTUAL trial
> cfg = [];
> cfg.continuous = 'yes';
> cfg.dataset = filename;
> cfg.trialdef.prestim = prestim;
> cfg.trialdef.poststim = poststim;
> cfg.trialdef.eventtype = 'STATUS';
> cfg.trialdef.eventvalue = trigger_codes
>
> cfg = ft_definetrial(cfg);
>
> data = ft_redefinetrial(cfg, data);
> data_ep = data;
>


On Thu, Jul 18, 2019 at 3:06 PM Stephen Whitmarsh <
stephen.whitmarsh at gmail.com> wrote:

> Dear Jeremy,
>
> Welcome to the FieldTrip community!
>
> Could you be more specific about "the data looks incorrect in the end.",
> and perhaps share a screenshot if informative? Sharing the relevant part of
> your script would be helpful too. For more info about how to get the best
> response to your questions, please take a look at:
> http://www.fieldtriptoolbox.org/faq/how_to_ask_good_questions_to_the_community/
>
> Have you checked where the NaN's in the data are, and whether they appear
> before or after downsampling?
>
> Just an shot in the dark, but have you perhaps defined your trials for
> epoching (i.e. the samples in the .trl) based on the old samplerate?
>
> Cheers,
> Stephen
>
>
>
>
>
> On Thu, 18 Jul 2019 at 12:35, Jeremy Pulmano <jpulmano at princeton.edu>
> wrote:
>
>> Hi,
>>
>> I've realized that the warning message often comes right after epoching
>> (using ft_definetrial and then ft_redefinetrial). It seems like it
>> successfully breaks the data into segments but still contains NaNs for
>> whatever reason. Here are more details about the pipeline so far:
>>
>>    1. High pass filtering (0.1 Hz / FIR / order = 100)
>>    2. Downsampling (1024 to 128 Hz)
>>    3. Epoching (prestim 0.1, poststim 0.4)
>>    4. Base-line correction [ -0.1 0 ]
>>    5. Re-referencing (common avg.)
>>    6. ICA / component rejection
>>    7. Visual artifact removal
>>    8. Low pass filter (30 Hz)
>>
>> Any further help would be greatly appreciated.
>>
>> Thanks,
>> Jeremy
>>
>> On Wed, Jul 17, 2019 at 4:57 PM Jeremy Pulmano <jpulmano at princeton.edu>
>> wrote:
>>
>>> Hi Erika,
>>>
>>> The original sampling rate is 1024 Hz and the new sampling rate is 128
>>> Hz.
>>>
>>> Thanks,
>>> J
>>>
>>> On Wed, Jul 17, 2019 at 4:49 PM Erika Puiutta <
>>> erika.puiutta at uni-oldenburg.de> wrote:
>>>
>>>> Hey J,
>>>>
>>>> this is just an educated guess since I don't really know how fieldtrip
>>>> does the downsampling, but what is your sampling frequency and the sampling
>>>> frequency you sample down to? Is it possible that your downsampling
>>>> frequency is not a harmonic of your original sampling frequency (say 1000Hz
>>>> to 300Hz)? Maybe there are NaNs for the new samples where fieldtrip can't
>>>> find a corresponding one from the old sampling frequency?
>>>>
>>>> Best of luck,
>>>> Erika
>>>>
>>>> Am 17.07.2019 um 17:26 schrieb Jeremy Pulmano <jpulmano at princeton.edu>:
>>>>
>>>> Hi all,
>>>>
>>>> I am new to both neuroscience and FieldTrip and am trying to debug my
>>>> preprocessing pipeline of EEG data (biosemi .bdf files). These are the
>>>> current steps:
>>>>
>>>>    1. High pass filtering
>>>>    2. Downsampling
>>>>    3. Epoching
>>>>    4. Base-line correction
>>>>    5. Re-referencing
>>>>    6. ICA / component rejection
>>>>    7. Visual artifact removal
>>>>    8. Low pass filter (I save the low pass filter for the end so that
>>>>    ICA yields better time course plots).
>>>>
>>>> However, after the first three steps (high pass, downsample, epoch), I
>>>> continue to get the warning message: "Warning: data contains NaN values, no
>>>> filtering or preprocessing applied," starting at baseline correction. It
>>>> repeats itself over and over again. If I ignore these errors, the data
>>>> looks incorrect in the end. Why does this happen, and how can I resolve
>>>> this?
>>>>
>>>> Any tips would be appreciated.
>>>>
>>>> Thanks,
>>>> J
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>>
>>> --
>>>
>>> *Jeremy Pulmano*
>>> *Princeton University Class of 2021*
>>> B.S.E. Computer Science
>>> e: jpulmano at princeton.edu | c: (862) 220-5903
>>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://doi.org/10.1371/journal.pcbi.1002202
>>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190718/6b2a673f/attachment-0002.html>
-------------- next part --------------
%% Setup code (abridged)

filename = 'example';
header = ft_read_header(filename);
new_fs = 128;
prestim = 0.1;
poststim = 0.4;
trigger_codes = [ 1 2 3 ];

%% High pass filter

% Define trial for all the continuous data
cfg = [];
cfg.dataset = filename;
cfg.headerfile = header;
cfg.trialdef.triallength = Inf;
cfg.trialdef.ntrials = 1; 

cfg = ft_definetrial(cfg);

% Preprocess with high pass filter
order = 100;
cfg.hpfilter = 'yes';
cfg.hpfilttype = 'fir';
cfg.hpfreq = 0.1;
cfg.hpfiltord = order;

data = ft_preprocessing(cfg);
data_hp = data;

%% Downsampling

fprintf('\nDownsampling\n');

cfg = [];
cfg.resamplefs = new_fs;

data = ft_resampledata(cfg, data);
data_res = data;

%% Epoch (NaNs appear after this block)

% Now define the ACTUAL trial
cfg = [];
cfg.continuous = 'yes';
cfg.dataset = filename;
cfg.trialdef.prestim = prestim;
cfg.trialdef.poststim = poststim;
cfg.trialdef.eventtype = 'STATUS';
cfg.trialdef.eventvalue = trigger_codes

cfg = ft_definetrial(cfg);

data = ft_redefinetrial(cfg, data);
data_ep = data;
-------------- next part --------------
_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://doi.org/10.1371/journal.pcbi.1002202