[FieldTrip] Removing power line

Alain de Cheveigne alain.de.cheveigne at ens.psl.eu
Mon Oct 4 15:29:52 CEST 2021


Hi Emilie, 

If you don't care about the spectral region ~50Hz and above (and you don't need temporal resolution better than ~20ms) the simplest and most effective is to filter by convolution with a 20ms square window. For example, if the sampling rate is 512ms: 

	B=ones(floor(512/50),1); 
	B=[B; 512/50-floor(512/50)]; % interpolate to approximate exactly 20ms
	clean_data=filter(B,1,raw_data);

This should be perfectly effective, but you lose the high frequency region and fine temporal resolution. If you care about those, then ZapLine is your best bet.  You may need to play with the nremove parameter, depending on the complexity of the 50Hz subspace, and possibly also the p.nkeep parameter if there are many channels. Note that ZapLine works less well if the sampling rate is low (e.g. 128 Hz).  Best acquire at a faster rate, apply Zapline, then downsample.

As a final remark, high line noise is often symptomatic of other problems, such as loose leads.  If that's the case there may not be much you can do...

Alain



> On 4 Oct 2021, at 12:46, Emilie Caspar via fieldtrip <fieldtrip at science.ru.nl> wrote:
> 
> Dear all, 
> 
> Thank you all very much for your answers. 
> 
> We have used a 64-electrode Biosemi system with left and right mastoids as references. The epochs lasts 1.5 s and we will look at averaged data, not single trial data. Regarding the preprocessing of the data, a filter from 0.1 to 40Hz was already applied as follows:
> 
> bpfilterOrder = 1; % ft default
> bpfilterRange = [.1 40];
> 
>  cfg = [];
>     cfg.dataset    = [ file.name];
>     cfg.bpfilter    = 'yes';
>     cfg.bpfreq      = bpfilterRange;
>     cfg.bpfiltord   = bpfilterOrder;
>     cfg.reref       = 'yes';
>     cfg.refchannel  = chansRef;
>     allData_preprosses = ft_preprocessing(cfg);
> 
> Following your suggestions, I performed some additional graphs and analyses, which actually made me a bit confused.
> 
> I run different analyses with or without the spectrum interpolation (SI) on two participants whose data where acquired in similar testing conditions: P122 and P123 (graphs in attachement).
> - For P122, we clearly see a peak at 50Hz without SI, but when I apply the usual SI at 50Hz with 100 and 150 as harmonics, the noise is clearly removed. This works for roughly 70% of the data.
> - for P123, the pattern is a bit more complex. Again, without SI, we see a peak at 50Hz. But for this participant, when I apply the same SI as for P122 at 50Hz and 100 150 as harmonics, then the peak appears to have been increased. I made a second test with a SI for 50HZ and 60Hz, and now on the graph I have two peaks which appeared, one at 50Hz and one at 60hz. And actually the noise is higher in the data after the SI for this participant. This pattern is replicated for the remaining 30% of the data. I am not I understand why the SI works for some participants, but not for others by actually increasing the power noise.  Because when we compare P122 and P123 without SI, it looks like the 50Hz peak is the same between the two participants.
> 
> I also tried with broader neighbors for P123 (for instance, 5 instead of 2). But then the peak at 50Hz remains higher in amplitude and in addition becomes larger. 
> 
> Regarding ZAPline, we started looking at it but it did not remove entirely the noise. But we are still learning to manipulate it to see if it can work with those data.
> 
> Thank you!
> 
> Emilie
> 
> 
> <PowerLineNoise.001.jpeg>
> ---------------------------------------------
> Emilie CASPAR
> 
> Associate Professor 
> 
> Department of Experimental Psychology, Ghent University
> office: Henri Dunantlaan, 2 - Floor 2, Room 94
> 
> lab’s website: https://moralsocialbrain.com/ 
> personal website: https://emiliecaspar.home.blog/
> 
> 
> Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95
> 
> 
>> Le 4 oct. 2021 à 12:43, Sarang S. Dalal via fieldtrip <fieldtrip at science.ru.nl> a écrit :
>> 
>> Hi Emilie,
>> 
>> I agree with Stephen -- absolutely, looking at the power spectrum will help guide your strategy here. It's normally unlikely to see both 50 Hz and 60 Hz in the same place... but I could imagine this may happen if the village gets power from a local electricity grid (likely 50 Hz) but with some buildings using potentially mismatched power inverters on solar panels or diesel generators (can be either 50 Hz or 60 Hz depending on manufacturer). Fluorescent lighting and perhaps some appliances may also generate noise frequencies that are totally unrelated to 50/60 Hz.
>> 
>> Just to clarify, cfg.dftreplace = 'neighbour' uses the spectrum interpolation method that Sabine Leske and I developed. It's actually designed for powerline noise frequencies that are *less* sharp than the default dftfilter method. So while it sounds like you have an extreme situation, it might indeed be an appropriate strategy for your data, depending on what your power spectrum actually looks like.
>> 
>> The design and test scenario was 50 Hz line noise in a typical home environment in Germany, which was actually a precise 50 Hz but of moderately fluctuating amplitude. These sorts of common fluctuations effectively broaden the noise frequency peak from a sharp 50 Hz to, for example, 49-51 Hz. For such a case, you would indeed define cfg.dftbandwidth of 2. The "neighbor" bandwidth is more tricky, but this should represent "clean" frequencies since their amplitude is used to replace the powerline corruption. In this example, a neighbourbandwidth of 1 would imply that the amplitude at 48-49 Hz and 51-52 Hz would be used to "patch" the noise peak at 49-51 Hz.
>> 
>> You may have a more extreme scenario, but nevertheless looking at the power spectrum will help you see how broad each powerline noise peak is (use that to set "dftbandwidth"), and ensure that you pick clean neighbor frequencies (guiding "neighbourbandwidth").
>> 
>> Our paper describing the method has an example of real data and some tips on usage that might be helpful:
>> https://doi.org/10.1016/j.neuroimage.2019.01.026
>> 
>> Hope this helps!
>> 
>> Cheers,
>> Sarang
>> 
>> 
>> On Sun, 2021-10-03 at 14:45 +0200, Stephen Whitmarsh via fieldtrip wrote:
>>> Dear Emilie,
>>> 
>>> Those sound indeed like challenging, and interesting, circumstances.
>>> I don't have a direct answer to your question, that would depend on having more information about the data, but here are my 2 cents;
>>> 
>>> You could first make an FFT plot (e.g. using cfg.method = mtmfft in ft_freqanalysis) to get a better picture of the frequencies in your data (on both 'successful' and 'unsuccessful' data), for several reasons:
>>> 	• You are also trying to remove both 50 and 60Hz, and their harmonics, of which probably only one of those would reflect the noise in the power lines.
>>> 	• You are using cfg.dftreplace = 'neighbour'; which might only work if the frequency band of the noise is sharp enough, or can be estimated precise enough (depending on the duration of the data you are filtering).
>>> 	• It would confirm whether it is line noise at all (which is probable, but more might be at play, e.g. some non-linear combinations and strange harmonics).
>>> 	• You could look for whether the degree of noise is stationary, by e.g. plotting the FFT over different periods, but it might be more important that the frequency is accurately chosen and stable over time.
>>> Depending on your data, the duration of data, whether or not your are looking at trials or the whole data file at once (*), etc., you might also want to use a 'welch method' for estimating the power in the data, i.e. by averaging over several (sliding) windows. Using cfg.method = 'mtmconvol' in ft_freqanalysis, then averaging over time with ft_selectdata (cfg.avgovertime), would be an easy way to do that. Once you have a good picture of the noise, it will be easier to both choose the right filters and to check their efficacy (by comparing the FFT before and after filtering, on both 'successful' and 'unsuccessful' data). In fact, since you are dealing with EEG in such noisy conditions, and assuming you are interested in either ERPs or slow oscillations (<40Hz), you could consider just using a lowpass filter at e.g. 40 Hz. 
>>> 
>>> Good luck, I hope this helps,
>>> Stephen 
>>> 
>>> *) The behaviour of filters depends a lot on the structure/size of the data, so to better understand its behaviour you would need to describe your data(structures) in more detail, i.e. trial length, nr. of trials/channels, and any processing steps until filtering. 
>>> 
>>> 
>>>  
>>> 
>>> Op zo 3 okt. 2021 om 13:12 schreef Emilie Caspar via fieldtrip <fieldtrip at science.ru.nl>:
>>>> Dear Fieldtrippers,
>>>> 
>>>> I recently acquired EEG data in Rwanda in complicated testing conditions, especially for the power line : we were testing in rural villages with a single electrical system for the whole village. Our signal is thus parasited by what the villagers were plugging on the electrical system (so probably nonstationary noise) during the testing. The electrical system was also very rudimentary and not correctly grounded.
>>>> 
>>>> Usually, I use the following command in Fieldtrip to remove the 50H or 60 Hz power line and it works quite well. Here, it worked for roughly 70% of our acquired data, but it does not work for all the data acquired.
>>>> 
>>>>     cfg= [];
>>>>     cfg.dftfilter = 'yes';
>>>>     cfg.dftfreq = [50 60 100 120 150 180];
>>>>     cfg.dftreplace = 'neighbour'; 
>>>>     cfg.dftbandwidth = [2 2 2 2 2 2];
>>>>     cfg.dftneighbourwidth = [2 2 2 2 2 2];
>>>>     data_intpl = ft_preprocessing(cfg, allData_preprosses);
>>>> 
>>>> 
>>>> When I look at the graphs (see figure attached), it really looks like it’s a 60Hz noise, but it seems that the dftfilter function does not remove it. We are certain it’s power noise because when the electricity was cut off because of a storm or else, and we were thus only relying on the batteries to collect our data, the signal was perfect. So in theory we should be able to remove it but we have no cue of what other possibilities to try. Perhaps it’s because the noise is non stationary and dftfilter does not account for that? I know some residual power noise can stay after a dftfiltre, but here it does not remove anything.
>>>> 
>>>> Thanks a lot for the help, 
>>>> 
>>>> Emilie
>>>> 
>>>> <PowerLine.jpg>
>>>> ---------------------------------------------
>>>> Prof. Dr. Emilie Caspar
>>>> 
>>>> Associate Professor 
>>>> 
>>>> Department of Experimental Psychology, Ghent University
>>>> office: Henri Dunantlaan, 2 - Floor 2, Room 94
>>>> 
>>>> lab’s website: https://moralsocialbrain.com/ 
>>>> personal website: https://emiliecaspar.home.blog/
>>>> 
>>>> 
>>>> Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95
>>>> 
>>>> 
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> https://doi.org/10.1371/journal.pcbi.1002202
>> 
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://doi.org/10.1371/journal.pcbi.1002202
> 
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202




More information about the fieldtrip mailing list