[FieldTrip] High pass filtering before epoching

[Arnaud Delorme]

Hi Jan and Alain,

>> You are right that with proper precautions one can epoch then filter with no ill effects. 

In my testing in the paper “EEG is better left alone” the decrease in significance associated with high pass filtering data epochs was large (only half of the electrodes are significant when filtering data epochs instead of the raw data for the oddball dataset in figure S2). Using 3-second epochs instead of 1-second epochs did not fix the issue. 10-second might do it, but I think that would be overkill.

Unfortunately, the ERP tutorial of Fieldtrip still uses the one-step preprocessing approach of epoching first and then filtering

https://urldefense.com/v3/__https://www.fieldtriptoolbox.org/tutorial/preprocessing_erp/__;!!HJOPV4FYYWzcc1jazlU!-kwcDOKyZcbjid93QHIVl1OP_hwvIe18FTa-pbZGJ_z_5BurltCT28oHIPU10dFHM6ih8xQcDXhJVpNsA3tr2CqcQA$ 

The continuous data tutorial uses the ft_redefinetrial method, which is a two-step approach of filtering continuous data and then epoching.

https://urldefense.com/v3/__https://www.fieldtriptoolbox.org/tutorial/continuous/__;!!HJOPV4FYYWzcc1jazlU!-kwcDOKyZcbjid93QHIVl1OP_hwvIe18FTa-pbZGJ_z_5BurltCT28oHIPU10dFHM6ih8xQcDXhJVpNsA3vynCHXUA$ 

Cheers,

Arno

>> Alain
>> 
>>> On 23 Jun 2023, at 07:54, Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl> wrote:
>>> 
>>> Hi Frans,
>>> 
>>> Historically, due to RAM constraints etc. a tradition FieldTrip pipeline started with epoching, then filtering. However, currently there is no reason to load in the continuous data first, then do the filtering, and then do the epoching. Although I didn’t read Julian’s reply in detail, I suspect that his code is doing exactly that.
>>> 
>>> There is just still a lot of documentation online which reverses this optimal order.
>>> 
>>> Note, in addition, that if one uses a filter with a finite impulse response, the filtered signal will not depend on the overall length of the data segment that you supply to the filter, once you get sufficiently far away from the edges. In other words, for instance, the center piece a one minute data segment that has been filtered will look exactly the same as a filtered version of the exact same center piece obtained from the 1-hour continuous recording. 
>>> 
>>> Fieldtrip allows for cfg.padding in ft_preprocessing, which will allow you to obtain epoched and filtered data in a single call to ft_preprocessing, without suffering from the edge effects of the filter. Once the value for cfg.padding exceeds the maximum epoch length + the filter order (and provided you use a finite impulse response filter) then you should be good.
>>> 
>>> Best wishes,
>>> Jan-Mathijs
>>> 
>>> 
>>>> On 15 Feb 2023, at 11:33, Frans Nord via fieldtrip <fieldtrip at science.ru.nl> wrote:
>>>> 
>>>> Hello mailing-list,
>>>> 
>>>> I have a question regarding the comment about Fieldtrip in the recently published paper "EEG is better left alone" [1]. While I don't agree with the conclusions of the paper there is a specific comment regarding Fieldtrip that I would like some comments about:
>>>> 
>>>> "We realized that the FieldTrip preprocessing function extracts data epochs before filtering the data when provided with both filter settings and epoch information. After consulting with FieldTrip developers, for all analyses using FieldTrip, we used an alternate multi-step implementation allowing us to extract epochs after the raw data had been filtered."
>>>> 
>>>> Looking through the code of ft_preprocessing and how the preproc function is used in my own preprocessing pipeline i realized that I seem to do the filtering on raw data epochs with this code:
>>>> 
>>>> cfg = [];
>>>> cfg.continuous ='yes';
>>>> cfg.trialfun = 
>>>> 
>>>> 'ft_trialfun_general'
>>>> ;
>>>> cfg.trialdef.eventtype = 
>>>> 
>>>> 'STATUS'
>>>> ;
>>>> cfg.trialdef.eventvalue = {1,2,3,4}; 
>>>> 
>>>> % Trigger values
>>>> 
>>>> cfg.trialdef.prestim =.5; 
>>>> 
>>>> % -500 ms before the stimulation
>>>> 
>>>> cfg.trialdef.poststim = 2; 
>>>> 
>>>> % +2000ms after the stimulation
>>>> 
>>>> cfg.dataset = path_to_dataset; 
>>>> 
>>>> % set the name of the dataset
>>>> 
>>>> cfg = ft_definetrial(cfg); 
>>>> 
>>>> % read the list of the specific stimulus
>>>> cfg.hpfilter
>>>> =
>>>> 
>>>> 'yes';
>>>> cfg.hpfreq = 1;[data] =ft_preprocessing(cfg);
>>>> 
>>>> 
>>>> I did however think that the code above would apply the high pass filter on the continuous data and not on the epochs but it seems that I was wrong. Or am I? So I'm now consulting the E-mail list to get some clarity about this and a potential recommendation on what to do instead. Basically, how to implement the "multi-step implementation" within Fieldtrip.
>>>> 
>>>> Thank you for providing the Fieldtrip toolbox
>>>> Best
>>>> Frans
>>>> 
>>>> [1] Delorme, A. EEG is better left alone. Sci Rep 13, 2372 (2023). https://urldefense.com/v3/__https://doi.org/10.1038/s41598-023-27528-0__;!!HJOPV4FYYWzcc1jazlU!49ARQZhij07QcXbznLgyQvIp4PNcdVSoZ2EsDDB043TbF0wd09QCDkgHP7DO1BXEz9jwCWU0q6yBAjm91FJ0WtPuFdr4qia6CZo$ 
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-_6L05NY4jlsfrP8C3I8I2jFQwOMSiDKS1tqSQ21aLsqyTB_rndzUnLg-8WiMQbKsVsuFCrx5lJzqbdD1tXsy2Tu8f4-RQxPapetqA$ 
>>> 
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!49ARQZhij07QcXbznLgyQvIp4PNcdVSoZ2EsDDB043TbF0wd09QCDkgHP7DO1BXEz9jwCWU0q6yBAjm91FJ0WtPuFdr4zPBfgKQ$ 
>> 
>> 
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!61hGn_iqSQTqOUy5TR8ASuo3th2U8mj2LiIj1gmQhqZt2NtnKDt3WqoEVWCmSIHMFnolsu2de5rRpkJnjxYJ4TTv0myofG2Rlu5zVg$
> 
> _______________________________________________
> fieldtrip mailing list
> https://urldefense.com/v3/__https://mailman.science.ru.nl/mailman/listinfo/fieldtrip__;!!Mih3wA!AVtHBHat3jUEF6ka5zCGkFNZBaQyq5gtgZHurR6WTN74wSjFfJMyVNeSvKoJUTCbcpE-ay-MQwAQIHEs78xR$ 
> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!Mih3wA!AVtHBHat3jUEF6ka5zCGkFNZBaQyq5gtgZHurR6WTN74wSjFfJMyVNeSvKoJUTCbcpE-ay-MQwAQIKYOBOjn$