[FieldTrip] Issue with NaN values using ft_artifact_zvalue

Mon Jul 22 12:23:23 CEST 2024

Hi Sophie,

Next to Konstantinos’ excellent suggestions, let me try and straighten a few things out here.

As soon as you see the warning ‘data contains NaNs’ it means that FieldTrip has detected NaNs in your data. I haven’t looked into the relevant code for a while, but I am pretty sure that FieldTrip does not silently add any NaNs to a well-defined data object without NaNs, unless there is something I am not aware of (or forgot about) in the functional behavior of the code. This being said, it could be that we are looking at an inappropriately defined cfg is used in combination with a data argument. Specifically, you seem to call ft_artifact_zvalue with both a cfg and a data object (i.e. data_segmented). In this case, I would have expected the fltpadding and trlpadding options to not have any functional consequences, because you supply the function with fixed data, rather than with a combination of a filename + a trl-matrix to be used to read in the data on-the-fly, and to search for artifacts in the as of yet implicitly defined segments. It could be the case, that FieldTrip (with non-zero valuse for cfg…..fltpadding/trlpadding) adds NaNs as the requested padded segments, which would explain your observations. To be sure about this, you could try and step through the code line by line (you can use the matlab debugger for this) and I suspect that NaNs may be added to the variable ’thisdat’ around line 311 in ft_artifact_zvalue. If this is the case, then the solution to the issue would be to either specify fltpadding to be 0, OR to call ft_artifact_zvalue without a data argument, but with a previously defined trl matrix in the cfg + a filename to a data file.

As a side note: filtering data with NaNs is not super-problematic if the following 2 conditions are met:
- the type of filter used is FIR, rather than IIR (as in your case, a ‘but’ filter is IIR and causes NaNs to spread to the full range of the data, rather than being constrained to the length of the impulse response of the FIR filter)
- you manage to get FieldTrip to use the so-called ‘compat’ version of the filtfilt function. This is a bit advanced, but would require you to remove the path of matlab’s filtfilt from the matlab search path, and then do something like:

global ft_default;
ft_default.toolbox.signal = ‘compat’;
ft_hastoolbox(’signal’, 1);

Best wishes,

Jan-Mathijs

PS: I could not help noticing that you very nicely address the list with ‘Dear FieldTrip Support team’. Just to  avoid any unrealistic expectations with you and the rest of the newer readership: this e-mail discussion list is explicitly not a ‘helpdesk’ type of list, there is no team behind the scenes that is charged with providing solutions to problems. We’re all here discussing with each other on a voluntary basis :).

On 3 Jul 2024, at 16:13, Schubert, E.S. (Sophie) via fieldtrip <fieldtrip at science.ru.nl> wrote:

Dear FieldTrip Support Team,

I am writing to seek assistance with an issue I am encountering while using the ft_artifact_zvalue function in FieldTrip for artifact rejection. Despite my data containing no NaN values initially, I receive an error during the bandpass filtering process indicating that the input is expected to be finite.

Here is a detailed description of the problem:
searching for artifacts in 2 channels
Warning: data contains NaNs, not all processing methods are robust to NaNs, so the NaNs
might spread
Warning: data contains NaN values
Error using filtfilt>efiltfilt
Error using filtfilt
Expected input to be finite.

Error in filtfilt (line 102)
        y = efiltfilt(b,a,x);

Error in filter_with_correction (line 73)
    filt = filtfilt(B, A, dat')';

Error in ft_preproc_bandpassfilter (line 300)
    filt = filter_with_correction(B,A,dat,dir,usefftfilt);

Error in preproc (line 405)
if strcmp(cfg.bpfilter, 'yes'),     dat = ft_preproc_bandpassfilter(dat, fsample, cfg.bpfreq, cfg.bpfiltord, cfg.bpfilttype, cfg.bpfiltdir, cfg.bpinstabilityfix, cfg.bpfiltdf, cfg.bpfiltwintype, cfg.bpfiltdev, cfg.plotfiltresp, cfg.usefftfilt); end

Error in ft_artifact_zvalue (line 315)
  thisdat = preproc(thisdat, cfg.artfctdef.zvalue.channel, offset2time(0, hdr.Fs, size(thisdat,2)), cfg.artfctdef.zvalue, fltpadding, fltpadding);

The code that I use:
frontalEye = {'EOG LOC-A2', 'EOG ROC-A2'}; % for artifact rejection
cfg = [];
cfg.artfctdef.zvalue.channel = frontalEye;
cfg.artfctdef.zvalue.cutoff = 2.5;
cfg.artfctdef.zvalue.trlpadding = 0;
cfg.artfctdef.zvalue.artpadding = 0.4;
cfg.artfctdef.zvalue.fltpadding = 0.1;
cfg.artfctdef.zvalue.bpfilter = 'yes';
cfg.artfctdef.zvalue.bpfreq = [0.5 2];
cfg.artfctdef.zvalue.bpfiltord = 4;
cfg.artfctdef.zvalue.bpfilttype = 'but';
cfg.artfctdef.zvalue.hilbert = 'yes';
cfg.artfctdef.zvalue.interactive = 'no';
[cfg, artifact_EOG_eye] = ft_artifact_zvalue(cfg, data_segmented);

Important notes:

  1.  I have checked and verified that the data does not contain any NaN values before the filtering process.
  2.  I have tried different configurations, including setting the padding to 0, which prevents the error

It appears that NaN values are being introduced during the filtering process, but I am unable to determine the cause. I understand that the filterpadding specifies how much extra data around each trial's segment is read from the file. This extra data (padding) is used to mitigate edge artifacts that can occur after applying filters to the data. However, given the encountered error, I am uncertain if I am utilizing this function correctly. Could you please provide guidance on how to resolve this issue?

Thank you for your support!

Kind regards,

Sophie Schubert

________________________________

De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct te informeren door het bericht te retourneren. Het Universitair Medisch Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197.

Denk s.v.p aan het milieu voor u deze e-mail afdrukt.

________________________________

This message may contain confidential information and is intended exclusively for the addressee. If you receive this message unintentionally, please do not use the contents but notify the sender immediately by return e-mail. University Medical Center Utrecht is a legal person by public law and is registered at the Chamber of Commerce for Midden-Nederland under no. 30244197.

Please consider the environment before printing this e-mail.

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C4cc85df59efe482e9fcb08dcaa385163%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638572406058210588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=e2PiOjIUqoIIapmpcX0u%2FasFKR3Myto3zg8sZgiHrU8%3D&reserved=0>
https://doi.org/10.1371/journal.pcbi.1002202<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C4cc85df59efe482e9fcb08dcaa385163%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638572406058210588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=lvt4KXSqk2KDmbDOs5vARlHF69JVPddya5t9GLQsPf8%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20240722/8177f79b/attachment.htm>