[FieldTrip] What exactly does filterpadding do?

Tue May 3 11:29:27 CEST 2016

Dear Julia,

The filtering not being applied might be related to the data samples you're
providing the bandpass filtering. Several things:

Keep in mind:
You're creating a bandpass butterworth filter [1-3hz] with filter order 2.
This means you'll need enough data to filter so low frequencies. Briefly,
the butterworth filter is an infinite impulse response filter (IIR) that
filters the data recursively (not by convolution) and the filter order
means the amount of samples from input and output the filter will use
(please go http://www.dspguide.com/CH14.PDF if this is not clear). The
higher the filter order, the sharper the frequency response, the stronger
the "ringing". The finite impulse response (FIR) filters, by the contrary,
are implemented by convolution and the filter order means the length of the
filter-kernel. The higher the filter order, the higher the frequency
resolution, the lower the temporal resolution, the longer the filter kernel.

Then you'll have to make sure your filter order is appropriate given the
length of your data, which is not intuitive for IIR filters. I'm not an
expert on filtering and I find the Widmann, Schröger & Maess recent paper
an excellent guide:
http://www.ncbi.nlm.nih.gov/pubmed/25128257

To dig into this issue, you can apply the very same filter settings with
ft_preprocessing separately and check what's happening before and after.
Then you can discard if the filtering is what is creating the problem.

Check:
- Make sure the epoching you're using in cfg.trl = temptrl; is not too
short for your filtering. My hunch is that the NaNs might be related to this
- I don't know what kind of artifacts are you chasing filtering so low
(drifts?) but the artifact padding you're using might be too short
(artpadding  = 0.1;)

Regarding the very large effects in those two trials, I'd be pragmatic and
I would delete them or I'd cut the edges that are problematic for you...

I hope that helps!

Diego

On 2 May 2016 at 16:31, Brehm, J. (Julia) <j.brehm at uu.nl> wrote:

> Dear Diego,
>
> yes, thank you, I've seen that already. Unfortunately this does not really
> help me.
>
> Apparently filtering is not even applied in my case (as the warning says).
> It seems that the NaNs are introduced in ft_fetch_data, shouldn't these be
> replaced by zeros where it's not filled by actual data when applying
> filterpadding? I assume it does that at some other point I couldn't find!?
>
> Furthermore, in the interactive mode one can see that the z-values mimic
> the data in the first and last trial of the dataset. This leads to the
> detection of artifacts which are none and missing real artifacts (due to
> very large effects in those two trials). I have attached a screenshot to
> illustrate what I mean.
>
> As far as I understand filterpadding uses zero-padding, meaning I can use
> it on segmented (not continuous) data, correct? So why does this have such
> a large effect on my whole data instead of simply getting rid of edge
> effects?
>
> Sorry if I am missing something obvious.
>
> Best,
> Julia
>
>
> ------------------------------
> *From:* fieldtrip-bounces at science.ru.nl [fieldtrip-bounces at science.ru.nl]
> on behalf of Diego Lozano-Soldevilla [dlozanosoldevilla at gmail.com]
> *Sent:* Monday, May 02, 2016 3:24 PM
> *To:* FieldTrip discussion list
> *Subject:* Re: [FieldTrip] What exactly does filterpadding do?
>
> Dear Julia,
>
> From the FAQ section:
>
>
> http://www.fieldtriptoolbox.org/faq/how_does_the_filter_padding_in_preprocessing_work
> <http://redir.aspx?REF=0gkAGAqPbCJ2dB292COe_fT8oQ4uUuB0lAE5ZjlGPIMMbbmbkXLTCAFodHRwOi8vd3d3LmZpZWxkdHJpcHRvb2xib3gub3JnL2ZhcS9ob3dfZG9lc190aGVfZmlsdGVyX3BhZGRpbmdfaW5fcHJlcHJvY2Vzc2luZ193b3Jr>
>
> http://www.fieldtriptoolbox.org/faq/how_can_i_interpret_the_different_types_of_padding_that_i_find_when_dealing_with_artifacts
> <http://redir.aspx?REF=JxY36va1p9AZr0dc60zsrQZfgm_me-O_jx77ehMz3x4MbbmbkXLTCAFodHRwOi8vd3d3LmZpZWxkdHJpcHRvb2xib3gub3JnL2ZhcS9ob3dfY2FuX2lfaW50ZXJwcmV0X3RoZV9kaWZmZXJlbnRfdHlwZXNfb2ZfcGFkZGluZ190aGF0X2lfZmluZF93aGVuX2RlYWxpbmdfd2l0aF9hcnRpZmFjdHM.>
>
> I hope that helps,
>
> Diego
>
>
>
> On 2 May 2016 at 15:08, Brehm, J. (Julia) <j.brehm at uu.nl
> <http://redir.aspx?REF=bbm3PKzTSZZ7WAp2IZa3r2IscZbmOgYQdLrtX5WQsp0MbbmbkXLTCAFtYWlsdG86ai5icmVobUB1dS5ubA..>
> > wrote:
>
>> Dear list,
>>
>> I am having problems applying filterpadding when
>> using ft_artifact_zvalue.
>>
>> Could someone please explain what exactly the effect of filterpadding is,
>> and in which cases I can apply it and in which not?
>>
>> I intended to use it to avoid edge effects in the artifact detection.
>> However, I get the following warning (only when applying fltpadding, my
>> data does not actually contain NaNs):
>>
>> Warning: data contains NaNs, no filtering or preprocessing applied
>> > In ft_warning (line 181)
>>   In preproc (line 282)
>>   In ft_artifact_zvalue (line 292)
>>
>> These are my settings:
>> cfg            = [];
>> cfg.trl        = temptrl; % all trials
>>
>> cfg.artfctdef.zvalue.channel     = channel{ch};  % single channel
>> cfg.artfctdef.zvalue.cutoff      = 3;
>> cfg.artfctdef.zvalue.fltpadding  = 0.1;
>>
>> cfg.artfctdef.zvalue.bpfilter   = 'yes';
>> cfg.artfctdef.zvalue.bpfilttype = 'but';
>> cfg.artfctdef.zvalue.bpfreq     = [1 3];
>> cfg.artfctdef.zvalue.bpfiltord  = 2;
>> cfg.artfctdef.zvalue.hilbert    = 'yes';
>>
>> Thanks!
>>
>> Best,
>> Julia
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> <http://redir.aspx?REF=G8XJ7tCtpbYE8RHR4kVgxIqdcT0l_J9AFDcMhGt9sDcMbbmbkXLTCAFtYWlsdG86ZmllbGR0cmlwQGRvbmRlcnMucnUubmw.>
>> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> <http://redir.aspx?REF=BpGfhXBN8ygpF5iDmBXoUMkjlLaKCCmwRMHwC-J7PnsMbbmbkXLTCAFodHRwOi8vbWFpbG1hbi5zY2llbmNlLnJ1Lm5sL21haWxtYW4vbGlzdGluZm8vZmllbGR0cmlw>
>>
>
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160503/1833e142/attachment-0002.html>