<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Dear Vadim,
<div class=""><br class="">
</div>
<div class="">Indeed in practice - if you want to highpass filter your data at a low cutoff frequency, as you want - it is perhaps better to filter the ‘continuous’ data. Or, at least to filter the data in longer segments than the ones you want to subsequently
analyse. At low highpass cutoff frequencies, the edge artifacts of the filter will be quite large, particularly if the data segments are short, and an IIR filter is used (FT’s firws implementation is a bit better-behaved, but I still wouldn’t use it in practice). </div>
<div class=""><br class="">
</div>
<div class="">Once upon a time, the available RAM in computers did not allow for reading in an entire recording at once, so that’s why in a lot of tutorials the filtering is still done on the smaller segments. This is in many cases not a big deal, as long as
the data is more or less well behaved, and the requested filter settings are not too adventurous.</div>
<div class=""><br class="">
</div>
<div class="">Note, that, if you apply the filtering at the ft_preprocessing step, combining it with reading in epochs from the raw data file, the cfg.padding option can be used to alleviate edge effects of the filter in the data segments of interest. The underlying
idea is that ft_preprocessing reads in a specified longer segment of data, applies the filter, and then cuts off the edges to return the smaller epoch-of-interest. The potential drawback is that artifacts outside your scope-of-interest bleed into the epochs,
but this would also happen if you were to filter the continuous data, and cut into epochs later on.</div>
<div class=""><br class="">
</div>
<div class="">Best wishes,</div>
<div class=""><br class="">
</div>
<div class="">Jan-Mathijs</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 10 Apr 2022, at 22:13, Vadim Axelrod via fieldtrip <<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Dear fieldrip forum,
<div class=""><br class="">
</div>
<div class="">First, as someone originally from Ukraine it was heartwarming for me to see your yellow-blue logo. Thank you.</div>
<div class=""><br class="">
</div>
<div class="">I am confused with regard to the stage to filter the data of the ERP experiment (hpfilter and lpfilter). According to tutorials, the hpfilter and lpfilter are done at trials level (i.e., not for continuous data):</div>
<div class=""><a href="https://urldefense.com/v3/__https://www.fieldtriptoolbox.org/workshop/baci2017/preprocessing/__;!!HJOPV4FYYWzcc1jazlU!5-jSpu_vkn7LSyzYFlV5CW8TvBrlg29_lb0rY6RJQqkVwRoRbK9uJ_PMNJYOEwC1dek8qeop0hdnl_necuB2qFMQS5vTAXdZxfrYRw$" class="">https://www.fieldtriptoolbox.org/workshop/baci2017/preprocessing/</a><br class="">
</div>
<div class=""><a href="https://urldefense.com/v3/__https://www.fieldtriptoolbox.org/workshop/oslo2019/introduction/__;!!HJOPV4FYYWzcc1jazlU!5-jSpu_vkn7LSyzYFlV5CW8TvBrlg29_lb0rY6RJQqkVwRoRbK9uJ_PMNJYOEwC1dek8qeop0hdnl_necuB2qFMQS5vTAXfyyFfjlg$" class="">https://www.fieldtriptoolbox.org/workshop/oslo2019/introduction/</a><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">However, this clearly resulted in an artifact in my case. In the attached graph I compared this result to filtering at the stage of continuous data and just having a notch electricity line filter (without fieldtrip). For some reason neither bsfilter
nor dftfilter work for me (this is a separate issue). Below is the code I used. I am using the fieldtrip version that I downloaded several weeks ago.</div>
<div class=""><br class="">
</div>
<div class="">Thank you for the help.</div>
<div class="">Vadim</div>
<div class=""><br class="">
</div>
<div class="">function FilterTimeLockedData(fileName, condNameArr)<br class="">
</div>
<div class=""><br class="">
</div>
<div class=""> cfg = [];<br class="">
cfg.trialdef.prestim = 0.5;% in seconds<br class="">
cfg.trialdef.poststim = 1; %in seconds<br class="">
cfg.trialdef.eventtype = condNameArr;<br class="">
cfg.dataset = fileName; <br class="">
cfg_tr_def = ft_definetrial(cfg); <br class="">
<br class="">
cfg = [];<br class="">
cfg.dataset = fileName;<br class="">
data_continous = ft_preprocessing(cfg); % read raw data<br class="">
<br class="">
data = ft_redefinetrial(cfg_tr_def, data_continous); <br class="">
<br class="">
cfg = [];<br class="">
cfg.hpfilter = 'yes'; % enable high-pass filtering<br class="">
cfg.lpfilter = 'yes'; % enable low-pass filtering<br class="">
cfg.hpfreq = 0.2; % set up the frequency for high-pass filter<br class="">
cfg.lpfreq = 30; % set up the frequency for low-pass filter</div>
<div class=""><br class="">
data = ft_preprocessing(cfg,data); <br class="">
<br class="">
function FilterContinousData(fileName, condNameArr)<br class="">
<br class="">
cfg = [];<br class="">
cfg.trialdef.prestim = 0.5;% in seconds<br class="">
cfg.trialdef.poststim = 1; %in seconds<br class="">
cfg.trialdef.eventtype = condNameArr;<br class="">
cfg.dataset = fileName; <br class="">
cfg_tr_def = ft_definetrial(cfg); <br class="">
<br class="">
cfg = [];<br class="">
cfg.dataset = fileName;<br class="">
cfg.hpfilter = 'yes'; % enable high-pass filtering<br class="">
cfg.lpfilter = 'yes'; % enable low-pass filtering<br class="">
cfg.hpfreq = 0.2; % set up the frequency for high-pass filter<br class="">
cfg.lpfreq = 30; % set up the frequency for low-pass filter<br class="">
data_continous = ft_preprocessing(cfg); % read raw data<br class="">
<br class="">
data = ft_redefinetrial(cfg_tr_def, data_continous); <br class="">
<br class="">
</div>
</div>
<span id="cid:f_l1tpxc510"><filtering_comparison.jpg></span>_______________________________________________<br class="">
fieldtrip mailing list<br class="">
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" class="">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br class="">
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!5-jSpu_vkn7LSyzYFlV5CW8TvBrlg29_lb0rY6RJQqkVwRoRbK9uJ_PMNJYOEwC1dek8qeop0hdnl_necuB2qFMQS5vTAXfOobgA4Q$
<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>