[FieldTrip] Removing power line

Mon Oct 4 13:11:58 CEST 2021

Dear Sarang,

Cheers! Thanks for clarifying the 'neightbour' dft method (and developing
it in the first place :)).
Indeed, I was not aware of the paper/method and assumed a 'standard' use;
but as they say - assuming makes an ... ;-)
Looking forward to reading the paper and seeing how the data will improve.

Best wishes,
Stephen

Op ma 4 okt. 2021 om 12:49 schreef Sarang S. Dalal via fieldtrip <
fieldtrip at science.ru.nl>:

> Hi Emilie,
>
> I agree with Stephen -- absolutely, looking at the power spectrum will
> help guide your strategy here. It's normally unlikely to see both 50 Hz and
> 60 Hz in the same place... but I could imagine this may happen if the
> village gets power from a local electricity grid (likely 50 Hz) but with
> some buildings using potentially mismatched power inverters on solar panels
> or diesel generators (can be either 50 Hz or 60 Hz depending on
> manufacturer). Fluorescent lighting and perhaps some appliances may also
> generate noise frequencies that are totally unrelated to 50/60 Hz.
>
> Just to clarify, cfg.dftreplace = 'neighbour' uses the spectrum
> interpolation method that Sabine Leske and I developed. It's actually
> designed for powerline noise frequencies that are *less* sharp than the
> default dftfilter method. So while it sounds like you have an extreme
> situation, it might indeed be an appropriate strategy for your data,
> depending on what your power spectrum actually looks like.
>
> The design and test scenario was 50 Hz line noise in a typical home
> environment in Germany, which was actually a precise 50 Hz but of
> moderately fluctuating amplitude. These sorts of common fluctuations
> effectively broaden the noise frequency peak from a sharp 50 Hz to, for
> example, 49-51 Hz. For such a case, you would indeed define
> cfg.dftbandwidth of 2. The "neighbor" bandwidth is more tricky, but this
> should represent "clean" frequencies since their amplitude is used to
> replace the powerline corruption. In this example, a neighbourbandwidth of
> 1 would imply that the amplitude at 48-49 Hz and 51-52 Hz would be used to
> "patch" the noise peak at 49-51 Hz.
>
> You may have a more extreme scenario, but nevertheless looking at the
> power spectrum will help you see how broad each powerline noise peak is
> (use that to set "dftbandwidth"), and ensure that you pick clean neighbor
> frequencies (guiding "neighbourbandwidth").
>
> Our paper describing the method has an example of real data and some tips
> on usage that might be helpful:
> https://doi.org/10.1016/j.neuroimage.2019.01.026
>
> Hope this helps!
>
> Cheers,
> Sarang
>
>
> On Sun, 2021-10-03 at 14:45 +0200, Stephen Whitmarsh via fieldtrip wrote:
>
> Dear Emilie,
>
> Those sound indeed like challenging, and interesting, circumstances.
> I don't have a direct answer to your question, that would depend on having
> more information about the data, but here are my 2 cents;
>
> You could first make an FFT plot (e.g. using cfg.method = mtmfft in
> ft_freqanalysis) to get a better picture of the frequencies in your data
> (on both 'successful' and 'unsuccessful' data), for several reasons:
>
>    - You are also trying to remove both 50 and 60Hz, and their harmonics,
>    of which probably only one of those would reflect the noise in the power
>    lines.
>    - You are using cfg.dftreplace = 'neighbour'; which might only work if
>    the frequency band of the noise is sharp enough, or can be estimated
>    precise enough (depending on the duration of the data you are filtering).
>    - It would confirm whether it is line noise at all (which is probable,
>    but more might be at play, e.g. some non-linear combinations and strange
>    harmonics).
>    - You could look for whether the degree of noise is stationary, by
>    e.g. plotting the FFT over different periods, but it might be more
>    important that the frequency is accurately chosen and stable over time.
>
> Depending on your data, the duration of data, whether or not your are
> looking at trials or the whole data file at once (*), etc., you might also
> want to use a 'welch method' for estimating the power in the data, i.e. by
> averaging over several (sliding) windows. Using cfg.method = 'mtmconvol' in
> ft_freqanalysis, then averaging over time with ft_selectdata
> (cfg.avgovertime), would be an easy way to do that. Once you have a good
> picture of the noise, it will be easier to both choose the right filters
> and to check their efficacy (by comparing the FFT before and after
> filtering, on both 'successful' and 'unsuccessful' data). In fact, since
> you are dealing with EEG in such noisy conditions, and assuming you are
> interested in either ERPs or slow oscillations (<40Hz), you could consider
> just using a lowpass filter at e.g. 40 Hz.
>
> Good luck, I hope this helps,
> Stephen
>
> *) The behaviour of filters depends a lot on the structure/size of the
> data, so to better understand its behaviour you would need to describe your
> data(structures) in more detail, i.e. trial length, nr. of trials/channels,
> and any processing steps until filtering.
>
>
>
>
> Op zo 3 okt. 2021 om 13:12 schreef Emilie Caspar via fieldtrip <
> fieldtrip at science.ru.nl>:
>
> Dear Fieldtrippers,
>
> I recently acquired EEG data in Rwanda in complicated testing conditions,
> especially for the power line : we were testing in rural villages with a
> single electrical system for the whole village. Our signal is thus
> parasited by what the villagers were plugging on the electrical system (so
> probably nonstationary noise) during the testing. The electrical system was
> also very rudimentary and not correctly grounded.
>
> Usually, I use the following command in Fieldtrip to remove the 50H or 60
> Hz power line and it works quite well. Here, it worked for roughly 70% of
> our acquired data, but it does not work for all the data acquired.
>
>     cfg= [];
>     cfg.dftfilter = 'yes';
>     cfg.dftfreq = [50 60 100 120 150 180];
>     cfg.dftreplace = 'neighbour';
>     cfg.dftbandwidth = [2 2 2 2 2 2];
>     cfg.dftneighbourwidth = [2 2 2 2 2 2];
>     data_intpl = ft_preprocessing(cfg, allData_preprosses);
>
>
> When I look at the graphs (see figure attached), it really looks like it’s
> a 60Hz noise, but it seems that the dftfilter function does not remove it.
> We are certain it’s power noise because when the electricity was cut off
> because of a storm or else, and we were thus only relying on the batteries
> to collect our data, the signal was perfect. So in theory we should be able
> to remove it but we have no cue of what other possibilities to try. Perhaps
> it’s because the noise is non stationary and dftfilter does not account for
> that? I know some residual power noise can stay after a dftfiltre, but here
> it does not remove anything.
>
> Thanks a lot for the help,
>
> Emilie
>
> ---------------------------------------------
> *Prof. Dr. Emilie Caspar*
>
> *Associate Professor *
>
> Department of Experimental Psychology, Ghent University
> office: Henri Dunantlaan, 2 - Floor 2, Room 94
>
> lab’s website: https://moralsocialbrain.com/
> personal website: https://emiliecaspar.home.blog/
>
> Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32
> 95
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/678d1915/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PowerLine.jpg
Type: image/jpeg
Size: 37857 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/678d1915/attachment.jpg>