[FieldTrip] High pass filtering before epoching

[Schoffelen, J.M. (Jan Mathijs)]

Hi Frans,

Historically, due to RAM constraints etc. a tradition FieldTrip pipeline started with epoching, then filtering. However, currently there is no reason to load in the continuous data first, then do the filtering, and then do the epoching. Although I didn’t read Julian’s reply in detail, I suspect that his code is doing exactly that.

There is just still a lot of documentation online which reverses this optimal order.

Note, in addition, that if one uses a filter with a finite impulse response, the filtered signal will not depend on the overall length of the data segment that you supply to the filter, once you get sufficiently far away from the edges. In other words, for instance, the center piece a one minute data segment that has been filtered will look exactly the same as a filtered version of the exact same center piece obtained from the 1-hour continuous recording.

Fieldtrip allows for cfg.padding in ft_preprocessing, which will allow you to obtain epoched and filtered data in a single call to ft_preprocessing, without suffering from the edge effects of the filter. Once the value for cfg.padding exceeds the maximum epoch length + the filter order (and provided you use a finite impulse response filter) then you should be good.

Best wishes,
Jan-Mathijs


On 15 Feb 2023, at 11:33, Frans Nord via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:

Hello mailing-list,

I have a question regarding the comment about Fieldtrip in the recently published paper "EEG is better left alone" [1]. While I don't agree with the conclusions of the paper there is a specific comment regarding Fieldtrip that I would like some comments about:

"We realized that the FieldTrip preprocessing function extracts data epochs before filtering the data when provided with both filter settings and epoch information. After consulting with FieldTrip developers, for all analyses using FieldTrip, we used an alternate multi-step implementation allowing us to extract epochs after the raw data had been filtered."

Looking through the code of ft_preprocessing and how the preproc function is used in my own preprocessing pipeline i realized that I seem to do the filtering on raw data epochs with this code:

cfg = [];
cfg.continuous ='yes';
cfg.trialfun = 'ft_trialfun_general';
cfg.trialdef.eventtype = 'STATUS';
cfg.trialdef.eventvalue = {1,2,3,4}; % Trigger values
cfg.trialdef.prestim =.5; % -500 ms before the stimulation
cfg.trialdef.poststim = 2; % +2000ms after the stimulation
cfg.dataset = path_to_dataset; % set the name of the dataset
cfg = ft_definetrial(cfg); % read the list of the specific stimuluscfg.hpfilter = 'yes';
cfg.hpfreq = 1;[data] =ft_preprocessing(cfg);

I did however think that the code above would apply the high pass filter on the continuous data and not on the epochs but it seems that I was wrong. Or am I? So I'm now consulting the E-mail list to get some clarity about this and a potential recommendation on what to do instead. Basically, how to implement the "multi-step implementation" within Fieldtrip.

Thank you for providing the Fieldtrip toolbox
Best
Frans

[1] Delorme, A. EEG is better left alone. Sci Rep 13, 2372 (2023). https://urldefense.com/v3/__https://doi.org/10.1038/s41598-023-27528-0__;!!HJOPV4FYYWzcc1jazlU!6kEZP86f4bgt2ARcX7863-yoptmy-dkOs3laiNe30BiJKGGueB0Mz8rLMIPNpn3t_8nhurJluaLVRRMdZLJYVmsPDhI_ssl-pt5LhA$ <https://urldefense.com/v3/__https://doi.org/10.1038/s41598-023-27528-0__;!!HJOPV4FYYWzcc1jazlU!4hjKg111dQ7s9-L3G_ddKy01SDzvgiEvo8gLx_R-WDx6Qxj6JJS0HuEwuUx1atexqoHmVc8kvd5ouM4p4smQ$>
_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-_6L05NY4jlsfrP8C3I8I2jFQwOMSiDKS1tqSQ21aLsqyTB_rndzUnLg-8WiMQbKsVsuFCrx5lJzqbdD1tXsy2Tu8f4-RQxPapetqA$

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20230623/4effe204/attachment.htm>