[FieldTrip] Issue with NaN values using ft_artifact_zvalue

[Schubert, E.S. (Sophie)]

Dear Konstantinos,
Thank you for your suggestions regarding cfg.artfctdef.zvalue.fltpadding and for your advice on optimizing the filtering process and addressing potential gaps between trials. I have implemented the following steps to tackle the issue:

  *   Updated the segmentation function to ensure data segments without gaps
  *   Added 0.1 seconds (and also tried 0.2, 0.3, 0.5, 1) at the beginning and end of each trial for cfg.artfctdef.zvalue.fltpadding to work properly
Despite these adjustments, I am still encountering NaN values during the filtering process. Below is a snippet of my code:
%% Read in events and segment the data
cfg = [];
cfg.dataset = data_path;
cfg.eventfile = event_file_path;
cfg.trialfun = 'trialfun_sleep';
cfg.trialdef.eventtype = events;
cfg.trialdef.pre = 0.1;  % Change accordingly
cfg.trialdef.post = 30.1; % Change accordingly

% Call ft_definetrial function in order to define the trials/segments based on the event file
cfg_trl_def = ft_definetrial(cfg);

% Segment data according to the trial definition
data_segmented = ft_redefinetrial(cfg_trl_def, data_filt);
Upon inspecting data_segmented.cfg.trl, it appears there are no gaps between samples as defined in the trial definition (if my interpretation is correct). Could there be something overlooked in my approach, or are the NaN values originating elsewhere in the preprocessing pipeline?
Snippet of my trl matrix:
65981       72021         -20           1
71981       78021         -20           1
77981       84021         -20           1
83981       90021         -20           1
I would greatly appreciate any further insights or guidance you could provide on resolving this specific issue.
Thank you in advance!
Kind regards,
Sophie
function [trl, event] = trialfun_sleep(cfg)
    % Lees de EDF file met de PSG signalen
    hdr = ft_read_header(cfg.dataset);

    % Lees de events uit het CSV bestand
    csvfile = cfg.eventfile;
    events = readtable(csvfile);

    % Initialiseer de event structure
    event = struct();

    % Definieer de trials
    trl = [];
    pre  = -round(cfg.trialdef.pre  * hdr.Fs);
    post = round(cfg.trialdef.post * hdr.Fs);

    % Controleer of eventtypes of eventvalues zijn gespecificeerd
    eventtype_specified = isfield(cfg.trialdef, 'eventtype') && ~isempty(cfg.trialdef.eventtype);
    eventvalue_specified = isfield(cfg.trialdef, 'eventvalue') && ~isempty(cfg.trialdef.eventvalue);

    % Vul de event structure in en bereken trials
    for i = 1:height(events)
        event(i).type     = events.event_type{i};
        event(i).sample   = round(events.start_time(i) * hdr.Fs) + 1;
        event(i).value    = events.event_code(i);
        event(i).offset   = 0; % Initialiseer offset
        event(i).duration = []; % Voeg duur toe indien beschikbaar

        % Filter de gewenste event types en waarden
        type_match = ~eventtype_specified || any(ismember(event(i).type, cfg.trialdef.eventtype));
        value_match = ~eventvalue_specified || any(ismember(event(i).value, cfg.trialdef.eventvalue));

        if type_match && value_match
            % Bereken de begin- en eindpunten van de trial
            trlbegin = event(i).sample + pre;
            trlend   = event(i).sample + post;
            offset   = pre;  % Offset is het aantal samples voor het event

            % Controleer of de trial binnen de grenzen van het dataset valt
            if trlbegin > 0 && trlend <= hdr.nSamples
                newtrl = [trlbegin trlend offset event(i).value];
                trl = [trl; newtrl];
            end
        end
    end
end

Van: fieldtrip <fieldtrip-bounces at science.ru.nl> namens Konstantinos Tsilimparis via fieldtrip <fieldtrip at science.ru.nl>
Datum: donderdag, 4 juli 2024 om 14:37
Aan: FieldTrip discussion list <fieldtrip at science.ru.nl>
CC: Konstantinos Tsilimparis <konstantinos.tsilimparis at outlook.com>
Onderwerp: Re: [FieldTrip] Issue with NaN values using ft_artifact_zvalue
Dear Sophie,
You have correctly identified that the error arises from the filter padding process. From my experience, if there are gaps (i.e., no available data points) between your trials, this will lead to NaN values during filter padding. While I am not fully aware of your data preprocessing and trial selection methods that leads to the data_segment, it believe that your trials have gaps in between. For instance, inspecting data_segment.samplenfo might reveal that the trials do not consist of continuous, uncut segments.
One option is to set the filter padding to 0, but this won’t take care of the filter-induced edge effects. Another option is to go back in your analysis and change the way you segment your data. When you split your data into trials do not leave any gaps between trials. For example, split your signal in 30 seconds trials without gaps in between. Or if you leave gaps between trials, make sure to keep 0.1 sec more in the beginning and the end of each trial for cfg.artfctdef.zvalue.fltpadding = 0.1 to work properly.

Best regards,
Konstantinos Tsilimparis

From: fieldtrip <fieldtrip-bounces at science.ru.nl> On Behalf Of Schubert, E.S. (Sophie) via fieldtrip
Sent: Wednesday, July 3, 2024 4:14 PM
To: fieldtrip at science.ru.nl
Cc: Schubert, E.S. (Sophie) <E.S.Schubert at umcutrecht.nl>
Subject: [FieldTrip] Issue with NaN values using ft_artifact_zvalue

Dear FieldTrip Support Team,

I am writing to seek assistance with an issue I am encountering while using the ft_artifact_zvalue function in FieldTrip for artifact rejection. Despite my data containing no NaN values initially, I receive an error during the bandpass filtering process indicating that the input is expected to be finite.

Here is a detailed description of the problem:
searching for artifacts in 2 channels
Warning: data contains NaNs, not all processing methods are robust to NaNs, so the NaNs
might spread
Warning: data contains NaN values
Error using filtfilt>efiltfilt
Error using filtfilt
Expected input to be finite.

Error in filtfilt (line 102)
        y = efiltfilt(b,a,x);

Error in filter_with_correction (line 73)
    filt = filtfilt(B, A, dat')';

Error in ft_preproc_bandpassfilter (line 300)
    filt = filter_with_correction(B,A,dat,dir,usefftfilt);

Error in preproc (line 405)
if strcmp(cfg.bpfilter, 'yes'),     dat = ft_preproc_bandpassfilter(dat, fsample, cfg.bpfreq, cfg.bpfiltord, cfg.bpfilttype, cfg.bpfiltdir, cfg.bpinstabilityfix, cfg.bpfiltdf, cfg.bpfiltwintype, cfg.bpfiltdev, cfg.plotfiltresp, cfg.usefftfilt); end

Error in ft_artifact_zvalue (line 315)
  thisdat = preproc(thisdat, cfg.artfctdef.zvalue.channel, offset2time(0, hdr.Fs, size(thisdat,2)), cfg.artfctdef.zvalue, fltpadding, fltpadding);

The code that I use:
frontalEye = {'EOG LOC-A2', 'EOG ROC-A2'}; % for artifact rejection
cfg = [];
cfg.artfctdef.zvalue.channel = frontalEye;
cfg.artfctdef.zvalue.cutoff = 2.5;
cfg.artfctdef.zvalue.trlpadding = 0;
cfg.artfctdef.zvalue.artpadding = 0.4;
cfg.artfctdef.zvalue.fltpadding = 0.1;
cfg.artfctdef.zvalue.bpfilter = 'yes';
cfg.artfctdef.zvalue.bpfreq = [0.5 2];
cfg.artfctdef.zvalue.bpfiltord = 4;
cfg.artfctdef.zvalue.bpfilttype = 'but';
cfg.artfctdef.zvalue.hilbert = 'yes';
cfg.artfctdef.zvalue.interactive = 'no';
[cfg, artifact_EOG_eye] = ft_artifact_zvalue(cfg, data_segmented);

Important notes:

  1.  I have checked and verified that the data does not contain any NaN values before the filtering process.
  2.  I have tried different configurations, including setting the padding to 0, which prevents the error

It appears that NaN values are being introduced during the filtering process, but I am unable to determine the cause. I understand that the filterpadding specifies how much extra data around each trial's segment is read from the file. This extra data (padding) is used to mitigate edge artifacts that can occur after applying filters to the data. However, given the encountered error, I am uncertain if I am utilizing this function correctly. Could you please provide guidance on how to resolve this issue?

Thank you for your support!

Kind regards,

Sophie Schubert

________________________________

De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct te informeren door het bericht te retourneren. Het Universitair Medisch Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197.

Denk s.v.p aan het milieu voor u deze e-mail afdrukt.

________________________________

This message may contain confidential information and is intended exclusively for the addressee. If you receive this message unintentionally, please do not use the contents but notify the sender immediately by return e-mail. University Medical Center Utrecht is a legal person by public law and is registered at the Chamber of Commerce for Midden-Nederland under no. 30244197.

Please consider the environment before printing this e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20240710/0d027b55/attachment.htm>