<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Emilie,
<div class=""><br class="">
</div>
<div class="">Regarding Zapline, we are currently working on a wrapper software called zapline-plus that (1) removes line noise using Zapline on segments of data, allowing it to deal with noise non-stationarities over time, (2) is fully automatic, so it requires
no manual setting of N components to remove or the exact line noise frequency, and (3) produces detailed plots of the denoising process, including power spectra so you can see how it performs. It works very well also on mobile EEG data sets that we have tested
- we would be curious to know how it works on your data.</div>
<div class=""><br class="">
</div>
<div class="">See <a href="https://github.com/MariusKlug/zapline-plus" class="">https://github.com/MariusKlug/zapline-plus</a> </div>
<div class=""><br class="">
<div class="">
<div>Niels</div>
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 4. Oct 2021, at 13:46, <a href="mailto:fieldtrip-request@science.ru.nl" class="">
fieldtrip-request@science.ru.nl</a> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Send fieldtrip mailing list submissions to<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a><br class="">
<br class="">
To subscribe or unsubscribe via the World Wide Web, visit<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
or, via email, send a message with subject or body 'help' to<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>fieldtrip-request@science.ru.nl<br class="">
<br class="">
You can reach the person managing the list at<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>fieldtrip-owner@science.ru.nl<br class="">
<br class="">
When replying, please edit your Subject line so it is more specific<br class="">
than "Re: Contents of fieldtrip digest..."<br class="">
<br class="">
<br class="">
Today's Topics:<br class="">
<br class="">
1. Re: Removing power line (Sarang S. Dalal)<br class="">
2. Re: Removing power line (Stephen Whitmarsh)<br class="">
3. Re: Removing power line (Alain de Cheveigne)<br class="">
4. Re: Removing power line (Emilie Caspar)<br class="">
<br class="">
<br class="">
----------------------------------------------------------------------<br class="">
<br class="">
Message: 1<br class="">
Date: Mon, 4 Oct 2021 10:43:01 +0000<br class="">
From: "Sarang S. Dalal" <sarang@cfin.au.dk><br class="">
To: "fieldtrip@science.ru.nl" <fieldtrip@science.ru.nl><br class="">
Subject: Re: [FieldTrip] Removing power line<br class="">
Message-ID:<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><95cba4971f21d1e42a6ec3d99c7f9db1da62661b.camel@cfin.au.dk><br class="">
Content-Type: text/plain; charset="utf-8"<br class="">
<br class="">
Hi Emilie,<br class="">
<br class="">
I agree with Stephen -- absolutely, looking at the power spectrum will help guide your strategy here. It's normally unlikely to see both 50 Hz and 60 Hz in the same place... but I could imagine this may happen if the village gets power from a local electricity
grid (likely 50 Hz) but with some buildings using potentially mismatched power inverters on solar panels or diesel generators (can be either 50 Hz or 60 Hz depending on manufacturer). Fluorescent lighting and perhaps some appliances may also generate noise
frequencies that are totally unrelated to 50/60 Hz.<br class="">
<br class="">
Just to clarify, cfg.dftreplace = 'neighbour' uses the spectrum interpolation method that Sabine Leske and I developed. It's actually designed for powerline noise frequencies that are *less* sharp than the default dftfilter method. So while it sounds like you
have an extreme situation, it might indeed be an appropriate strategy for your data, depending on what your power spectrum actually looks like.<br class="">
<br class="">
The design and test scenario was 50 Hz line noise in a typical home environment in Germany, which was actually a precise 50 Hz but of moderately fluctuating amplitude. These sorts of common fluctuations effectively broaden the noise frequency peak from a sharp
50 Hz to, for example, 49-51 Hz. For such a case, you would indeed define cfg.dftbandwidth of 2. The "neighbor" bandwidth is more tricky, but this should represent "clean" frequencies since their amplitude is used to replace the powerline corruption. In this
example, a neighbourbandwidth of 1 would imply that the amplitude at 48-49 Hz and 51-52 Hz would be used to "patch" the noise peak at 49-51 Hz.<br class="">
<br class="">
You may have a more extreme scenario, but nevertheless looking at the power spectrum will help you see how broad each powerline noise peak is (use that to set "dftbandwidth"), and ensure that you pick clean neighbor frequencies (guiding "neighbourbandwidth").<br class="">
<br class="">
Our paper describing the method has an example of real data and some tips on usage that might be helpful:<br class="">
https://doi.org/10.1016/j.neuroimage.2019.01.026<br class="">
<br class="">
Hope this helps!<br class="">
<br class="">
Cheers,<br class="">
Sarang<br class="">
<br class="">
<br class="">
On Sun, 2021-10-03 at 14:45 +0200, Stephen Whitmarsh via fieldtrip wrote:<br class="">
Dear Emilie,<br class="">
<br class="">
Those sound indeed like challenging, and interesting, circumstances.<br class="">
I don't have a direct answer to your question, that would depend on having more information about the data, but here are my 2 cents;<br class="">
<br class="">
You could first make an FFT plot (e.g. using cfg.method = mtmfft in ft_freqanalysis) to get a better picture of the frequencies in your data (on both 'successful' and 'unsuccessful' data), for several reasons:<br class="">
<br class="">
* You are also trying to remove both 50 and 60Hz, and their harmonics, of which probably only one of those would reflect the noise in the power lines.<br class="">
* You are using cfg.dftreplace = 'neighbour'; which might only work if the frequency band of the noise is sharp enough, or can be estimated precise enough (depending on the duration of the data you are filtering).<br class="">
* It would confirm whether it is line noise at all (which is probable, but more might be at play, e.g. some non-linear combinations and strange harmonics).<br class="">
* You could look for whether the degree of noise is stationary, by e.g. plotting the FFT over different periods, but it might be more important that the frequency is accurately chosen and stable over time.<br class="">
<br class="">
Depending on your data, the duration of data, whether or not your are looking at trials or the whole data file at once (*), etc., you might also want to use a 'welch method' for estimating the power in the data, i.e. by averaging over several (sliding) windows.
Using cfg.method = 'mtmconvol' in ft_freqanalysis, then averaging over time with ft_selectdata (cfg.avgovertime), would be an easy way to do that. Once you have a good picture of the noise, it will be easier to both choose the right filters and to check their
efficacy (by comparing the FFT before and after filtering, on both 'successful' and 'unsuccessful' data). In fact, since you are dealing with EEG in such noisy conditions, and assuming you are interested in either ERPs or slow oscillations (<40Hz), you could
consider just using a lowpass filter at e.g. 40 Hz.<br class="">
<br class="">
Good luck, I hope this helps,<br class="">
Stephen<br class="">
<br class="">
*) The behaviour of filters depends a lot on the structure/size of the data, so to better understand its behaviour you would need to describe your data(structures) in more detail, i.e. trial length, nr. of trials/channels, and any processing steps until filtering.<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Op zo 3 okt. 2021 om 13:12 schreef Emilie Caspar via fieldtrip <fieldtrip@science.ru.nl<mailto:fieldtrip@science.ru.nl>>:<br class="">
Dear Fieldtrippers,<br class="">
<br class="">
I recently acquired EEG data in Rwanda in complicated testing conditions, especially for the power line : we were testing in rural villages with a single electrical system for the whole village. Our signal is thus parasited by what the villagers were plugging
on the electrical system (so probably nonstationary noise) during the testing. The electrical system was also very rudimentary and not correctly grounded.<br class="">
<br class="">
Usually, I use the following command in Fieldtrip to remove the 50H or 60 Hz power line and it works quite well. Here, it worked for roughly 70% of our acquired data, but it does not work for all the data acquired.<br class="">
<br class="">
cfg= [];<br class="">
cfg.dftfilter = 'yes';<br class="">
cfg.dftfreq = [50 60 100 120 150 180];<br class="">
cfg.dftreplace = 'neighbour';<br class="">
cfg.dftbandwidth = [2 2 2 2 2 2];<br class="">
cfg.dftneighbourwidth = [2 2 2 2 2 2];<br class="">
data_intpl = ft_preprocessing(cfg, allData_preprosses);<br class="">
<br class="">
<br class="">
When I look at the graphs (see figure attached), it really looks like it’s a 60Hz noise, but it seems that the dftfilter function does not remove it. We are certain it’s power noise because when the electricity was cut off because of a storm or else, and we
were thus only relying on the batteries to collect our data, the signal was perfect. So in theory we should be able to remove it but we have no cue of what other possibilities to try. Perhaps it’s because the noise is non stationary and dftfilter does not
account for that? I know some residual power noise can stay after a dftfiltre, but here it does not remove anything.<br class="">
<br class="">
Thanks a lot for the help,<br class="">
<br class="">
Emilie<br class="">
<br class="">
[cid:17c461876c4c75d62ec1]<br class="">
---------------------------------------------<br class="">
Prof. Dr. Emilie Caspar<br class="">
Associate Professor<br class="">
Department of Experimental Psychology, Ghent University<br class="">
office: Henri Dunantlaan, 2 - Floor 2, Room 94<br class="">
lab’s website: https://moralsocialbrain.com/<br class="">
personal website: https://emiliecaspar.home.blog/<br class="">
Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
<br class="">
-------------- next part --------------<br class="">
An HTML attachment was scrubbed...<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/9a599f3e/attachment-0001.htm><br class="">
-------------- next part --------------<br class="">
A non-text attachment was scrubbed...<br class="">
Name: PowerLine.jpg<br class="">
Type: image/jpeg<br class="">
Size: 37857 bytes<br class="">
Desc: PowerLine.jpg<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/9a599f3e/attachment-0001.jpg><br class="">
<br class="">
------------------------------<br class="">
<br class="">
Message: 2<br class="">
Date: Mon, 4 Oct 2021 13:11:58 +0200<br class="">
From: Stephen Whitmarsh <stephen.whitmarsh@gmail.com><br class="">
To: FieldTrip discussion list <fieldtrip@science.ru.nl><br class="">
Subject: Re: [FieldTrip] Removing power line<br class="">
Message-ID:<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><CAFrxm=wMHSY_2VLCON=f6V2+9nP3kum=zuuo=6d+ypb+mfNBvw@mail.gmail.com><br class="">
Content-Type: text/plain; charset="utf-8"<br class="">
<br class="">
Dear Sarang,<br class="">
<br class="">
Cheers! Thanks for clarifying the 'neightbour' dft method (and developing<br class="">
it in the first place :)).<br class="">
Indeed, I was not aware of the paper/method and assumed a 'standard' use;<br class="">
but as they say - assuming makes an ... ;-)<br class="">
Looking forward to reading the paper and seeing how the data will improve.<br class="">
<br class="">
Best wishes,<br class="">
Stephen<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Op ma 4 okt. 2021 om 12:49 schreef Sarang S. Dalal via fieldtrip <<br class="">
fieldtrip@science.ru.nl>:<br class="">
<br class="">
<blockquote type="cite" class="">Hi Emilie,<br class="">
<br class="">
I agree with Stephen -- absolutely, looking at the power spectrum will<br class="">
help guide your strategy here. It's normally unlikely to see both 50 Hz and<br class="">
60 Hz in the same place... but I could imagine this may happen if the<br class="">
village gets power from a local electricity grid (likely 50 Hz) but with<br class="">
some buildings using potentially mismatched power inverters on solar panels<br class="">
or diesel generators (can be either 50 Hz or 60 Hz depending on<br class="">
manufacturer). Fluorescent lighting and perhaps some appliances may also<br class="">
generate noise frequencies that are totally unrelated to 50/60 Hz.<br class="">
<br class="">
Just to clarify, cfg.dftreplace = 'neighbour' uses the spectrum<br class="">
interpolation method that Sabine Leske and I developed. It's actually<br class="">
designed for powerline noise frequencies that are *less* sharp than the<br class="">
default dftfilter method. So while it sounds like you have an extreme<br class="">
situation, it might indeed be an appropriate strategy for your data,<br class="">
depending on what your power spectrum actually looks like.<br class="">
<br class="">
The design and test scenario was 50 Hz line noise in a typical home<br class="">
environment in Germany, which was actually a precise 50 Hz but of<br class="">
moderately fluctuating amplitude. These sorts of common fluctuations<br class="">
effectively broaden the noise frequency peak from a sharp 50 Hz to, for<br class="">
example, 49-51 Hz. For such a case, you would indeed define<br class="">
cfg.dftbandwidth of 2. The "neighbor" bandwidth is more tricky, but this<br class="">
should represent "clean" frequencies since their amplitude is used to<br class="">
replace the powerline corruption. In this example, a neighbourbandwidth of<br class="">
1 would imply that the amplitude at 48-49 Hz and 51-52 Hz would be used to<br class="">
"patch" the noise peak at 49-51 Hz.<br class="">
<br class="">
You may have a more extreme scenario, but nevertheless looking at the<br class="">
power spectrum will help you see how broad each powerline noise peak is<br class="">
(use that to set "dftbandwidth"), and ensure that you pick clean neighbor<br class="">
frequencies (guiding "neighbourbandwidth").<br class="">
<br class="">
Our paper describing the method has an example of real data and some tips<br class="">
on usage that might be helpful:<br class="">
https://doi.org/10.1016/j.neuroimage.2019.01.026<br class="">
<br class="">
Hope this helps!<br class="">
<br class="">
Cheers,<br class="">
Sarang<br class="">
<br class="">
<br class="">
On Sun, 2021-10-03 at 14:45 +0200, Stephen Whitmarsh via fieldtrip wrote:<br class="">
<br class="">
Dear Emilie,<br class="">
<br class="">
Those sound indeed like challenging, and interesting, circumstances.<br class="">
I don't have a direct answer to your question, that would depend on having<br class="">
more information about the data, but here are my 2 cents;<br class="">
<br class="">
You could first make an FFT plot (e.g. using cfg.method = mtmfft in<br class="">
ft_freqanalysis) to get a better picture of the frequencies in your data<br class="">
(on both 'successful' and 'unsuccessful' data), for several reasons:<br class="">
<br class="">
- You are also trying to remove both 50 and 60Hz, and their harmonics,<br class="">
of which probably only one of those would reflect the noise in the power<br class="">
lines.<br class="">
- You are using cfg.dftreplace = 'neighbour'; which might only work if<br class="">
the frequency band of the noise is sharp enough, or can be estimated<br class="">
precise enough (depending on the duration of the data you are filtering).<br class="">
- It would confirm whether it is line noise at all (which is probable,<br class="">
but more might be at play, e.g. some non-linear combinations and strange<br class="">
harmonics).<br class="">
- You could look for whether the degree of noise is stationary, by<br class="">
e.g. plotting the FFT over different periods, but it might be more<br class="">
important that the frequency is accurately chosen and stable over time.<br class="">
<br class="">
Depending on your data, the duration of data, whether or not your are<br class="">
looking at trials or the whole data file at once (*), etc., you might also<br class="">
want to use a 'welch method' for estimating the power in the data, i.e. by<br class="">
averaging over several (sliding) windows. Using cfg.method = 'mtmconvol' in<br class="">
ft_freqanalysis, then averaging over time with ft_selectdata<br class="">
(cfg.avgovertime), would be an easy way to do that. Once you have a good<br class="">
picture of the noise, it will be easier to both choose the right filters<br class="">
and to check their efficacy (by comparing the FFT before and after<br class="">
filtering, on both 'successful' and 'unsuccessful' data). In fact, since<br class="">
you are dealing with EEG in such noisy conditions, and assuming you are<br class="">
interested in either ERPs or slow oscillations (<40Hz), you could consider<br class="">
just using a lowpass filter at e.g. 40 Hz.<br class="">
<br class="">
Good luck, I hope this helps,<br class="">
Stephen<br class="">
<br class="">
*) The behaviour of filters depends a lot on the structure/size of the<br class="">
data, so to better understand its behaviour you would need to describe your<br class="">
data(structures) in more detail, i.e. trial length, nr. of trials/channels,<br class="">
and any processing steps until filtering.<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Op zo 3 okt. 2021 om 13:12 schreef Emilie Caspar via fieldtrip <<br class="">
fieldtrip@science.ru.nl>:<br class="">
<br class="">
Dear Fieldtrippers,<br class="">
<br class="">
I recently acquired EEG data in Rwanda in complicated testing conditions,<br class="">
especially for the power line : we were testing in rural villages with a<br class="">
single electrical system for the whole village. Our signal is thus<br class="">
parasited by what the villagers were plugging on the electrical system (so<br class="">
probably nonstationary noise) during the testing. The electrical system was<br class="">
also very rudimentary and not correctly grounded.<br class="">
<br class="">
Usually, I use the following command in Fieldtrip to remove the 50H or 60<br class="">
Hz power line and it works quite well. Here, it worked for roughly 70% of<br class="">
our acquired data, but it does not work for all the data acquired.<br class="">
<br class="">
cfg= [];<br class="">
cfg.dftfilter = 'yes';<br class="">
cfg.dftfreq = [50 60 100 120 150 180];<br class="">
cfg.dftreplace = 'neighbour';<br class="">
cfg.dftbandwidth = [2 2 2 2 2 2];<br class="">
cfg.dftneighbourwidth = [2 2 2 2 2 2];<br class="">
data_intpl = ft_preprocessing(cfg, allData_preprosses);<br class="">
<br class="">
<br class="">
When I look at the graphs (see figure attached), it really looks like it’s<br class="">
a 60Hz noise, but it seems that the dftfilter function does not remove it.<br class="">
We are certain it’s power noise because when the electricity was cut off<br class="">
because of a storm or else, and we were thus only relying on the batteries<br class="">
to collect our data, the signal was perfect. So in theory we should be able<br class="">
to remove it but we have no cue of what other possibilities to try. Perhaps<br class="">
it’s because the noise is non stationary and dftfilter does not account for<br class="">
that? I know some residual power noise can stay after a dftfiltre, but here<br class="">
it does not remove anything.<br class="">
<br class="">
Thanks a lot for the help,<br class="">
<br class="">
Emilie<br class="">
<br class="">
---------------------------------------------<br class="">
*Prof. Dr. Emilie Caspar*<br class="">
<br class="">
*Associate Professor *<br class="">
<br class="">
Department of Experimental Psychology, Ghent University<br class="">
office: Henri Dunantlaan, 2 - Floor 2, Room 94<br class="">
<br class="">
lab’s website: https://moralsocialbrain.com/<br class="">
personal website: https://emiliecaspar.home.blog/<br class="">
<br class="">
Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32<br class="">
95<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
<br class="">
</blockquote>
-------------- next part --------------<br class="">
An HTML attachment was scrubbed...<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/678d1915/attachment-0001.htm><br class="">
-------------- next part --------------<br class="">
A non-text attachment was scrubbed...<br class="">
Name: PowerLine.jpg<br class="">
Type: image/jpeg<br class="">
Size: 37857 bytes<br class="">
Desc: not available<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/678d1915/attachment-0001.jpg><br class="">
<br class="">
------------------------------<br class="">
<br class="">
Message: 3<br class="">
Date: Mon, 4 Oct 2021 12:29:04 +0100<br class="">
From: Alain de Cheveigne <alain.de.cheveigne@ens.psl.eu><br class="">
To: FieldTrip discussion list <fieldtrip@science.ru.nl><br class="">
Subject: Re: [FieldTrip] Removing power line<br class="">
Message-ID: <3F456298-66A9-4DB4-9857-ED728A674554@ens.psl.eu><br class="">
Content-Type: text/plain;<span class="Apple-tab-span" style="white-space:pre"> </span>
charset=utf-8<br class="">
<br class="">
Try ZapLine from the NoiseTools toolbox: http://audition.ens.fr/adc/NoiseTools/.<br class="">
<br class="">
Alain<br class="">
<br class="">
<blockquote type="cite" class="">On 3 Oct 2021, at 12:03, Emilie Caspar via fieldtrip <fieldtrip@science.ru.nl> wrote:<br class="">
<br class="">
Dear Fieldtrippers,<br class="">
<br class="">
I recently acquired EEG data in Rwanda in complicated testing conditions, especially for the power line : we were testing in rural villages with a single electrical system for the whole village. Our signal is thus parasited by what the villagers were plugging
on the electrical system (so probably nonstationary noise) during the testing. The electrical system was also very rudimentary and not correctly grounded.<br class="">
<br class="">
Usually, I use the following command in Fieldtrip to remove the 50H or 60 Hz power line and it works quite well. Here, it worked for roughly 70% of our acquired data, but it does not work for all the data acquired.<br class="">
<br class="">
cfg= [];<br class="">
cfg.dftfilter = 'yes';<br class="">
cfg.dftfreq = [50 60 100 120 150 180];<br class="">
cfg.dftreplace = 'neighbour'; <br class="">
cfg.dftbandwidth = [2 2 2 2 2 2];<br class="">
cfg.dftneighbourwidth = [2 2 2 2 2 2];<br class="">
data_intpl = ft_preprocessing(cfg, allData_preprosses);<br class="">
<br class="">
<br class="">
When I look at the graphs (see figure attached), it really looks like it’s a 60Hz noise, but it seems that the dftfilter function does not remove it. We are certain it’s power noise because when the electricity was cut off because of a storm or else, and we
were thus only relying on the batteries to collect our data, the signal was perfect. So in theory we should be able to remove it but we have no cue of what other possibilities to try. Perhaps it’s because the noise is non stationary and dftfilter does not
account for that? I know some residual power noise can stay after a dftfiltre, but here it does not remove anything.<br class="">
<br class="">
Thanks a lot for the help, <br class="">
<br class="">
Emilie<br class="">
<br class="">
<PowerLine.jpg><br class="">
---------------------------------------------<br class="">
Prof. Dr. Emilie Caspar<br class="">
<br class="">
Associate Professor <br class="">
<br class="">
Department of Experimental Psychology, Ghent University<br class="">
office: Henri Dunantlaan, 2 - Floor 2, Room 94<br class="">
<br class="">
lab’s website: https://moralsocialbrain.com/ <br class="">
personal website: https://emiliecaspar.home.blog/<br class="">
<br class="">
<br class="">
Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
</blockquote>
<br class="">
<br class="">
<br class="">
------------------------------<br class="">
<br class="">
Message: 4<br class="">
Date: Mon, 4 Oct 2021 13:46:42 +0200<br class="">
From: Emilie Caspar <ecaspar@ulb.ac.be><br class="">
To: FieldTrip discussion list <fieldtrip@science.ru.nl><br class="">
Subject: Re: [FieldTrip] Removing power line<br class="">
Message-ID: <E6C52CE1-11C6-4DCD-8E55-7A42A9D27A1D@ulb.ac.be><br class="">
Content-Type: text/plain; charset="utf-8"<br class="">
<br class="">
Dear all, <br class="">
<br class="">
Thank you all very much for your answers. <br class="">
<br class="">
We have used a 64-electrode Biosemi system with left and right mastoids as references. The epochs lasts 1.5 s and we will look at averaged data, not single trial data. Regarding the preprocessing of the data, a filter from 0.1 to 40Hz was already applied as
follows:<br class="">
<br class="">
bpfilterOrder = 1; % ft default<br class="">
bpfilterRange = [.1 40];<br class="">
<br class="">
cfg = [];<br class="">
cfg.dataset = [ file.name];<br class="">
cfg.bpfilter = 'yes';<br class="">
cfg.bpfreq = bpfilterRange;<br class="">
cfg.bpfiltord = bpfilterOrder;<br class="">
cfg.reref = 'yes';<br class="">
cfg.refchannel = chansRef;<br class="">
allData_preprosses = ft_preprocessing(cfg);<br class="">
<br class="">
Following your suggestions, I performed some additional graphs and analyses, which actually made me a bit confused.<br class="">
<br class="">
I run different analyses with or without the spectrum interpolation (SI) on two participants whose data where acquired in similar testing conditions: P122 and P123 (graphs in attachement).<br class="">
- For P122, we clearly see a peak at 50Hz without SI, but when I apply the usual SI at 50Hz with 100 and 150 as harmonics, the noise is clearly removed. This works for roughly 70% of the data.<br class="">
- for P123, the pattern is a bit more complex. Again, without SI, we see a peak at 50Hz. But for this participant, when I apply the same SI as for P122 at 50Hz and 100 150 as harmonics, then the peak appears to have been increased. I made a second test with
a SI for 50HZ and 60Hz, and now on the graph I have two peaks which appeared, one at 50Hz and one at 60hz. And actually the noise is higher in the data after the SI for this participant. This pattern is replicated for the remaining 30% of the data. I am not
I understand why the SI works for some participants, but not for others by actually increasing the power noise. Because when we compare P122 and P123 without SI, it looks like the 50Hz peak is the same between the two participants.<br class="">
<br class="">
I also tried with broader neighbors for P123 (for instance, 5 instead of 2). But then the peak at 50Hz remains higher in amplitude and in addition becomes larger.
<br class="">
<br class="">
Regarding ZAPline, we started looking at it but it did not remove entirely the noise. But we are still learning to manipulate it to see if it can work with those data.<br class="">
<br class="">
Thank you!<br class="">
<br class="">
Emilie<br class="">
<br class="">
<br class="">
<br class="">
---------------------------------------------<br class="">
Emilie CASPAR<br class="">
<br class="">
Associate Professor <br class="">
<br class="">
Department of Experimental Psychology, Ghent University<br class="">
office: Henri Dunantlaan, 2 - Floor 2, Room 94<br class="">
<br class="">
lab’s website: https://moralsocialbrain.com/ <https://moralsocialbrain.com/> <br class="">
personal website: https://emiliecaspar.home.blog/ <https://emiliecaspar.home.blog/><br class="">
Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95<br class="">
<br class="">
<br class="">
<blockquote type="cite" class="">Le 4 oct. 2021 à 12:43, Sarang S. Dalal via fieldtrip <fieldtrip@science.ru.nl> a écrit :<br class="">
<br class="">
Hi Emilie,<br class="">
<br class="">
I agree with Stephen -- absolutely, looking at the power spectrum will help guide your strategy here. It's normally unlikely to see both 50 Hz and 60 Hz in the same place... but I could imagine this may happen if the village gets power from a local electricity
grid (likely 50 Hz) but with some buildings using potentially mismatched power inverters on solar panels or diesel generators (can be either 50 Hz or 60 Hz depending on manufacturer). Fluorescent lighting and perhaps some appliances may also generate noise
frequencies that are totally unrelated to 50/60 Hz.<br class="">
<br class="">
Just to clarify, cfg.dftreplace = 'neighbour' uses the spectrum interpolation method that Sabine Leske and I developed. It's actually designed for powerline noise frequencies that are *less* sharp than the default dftfilter method. So while it sounds like you
have an extreme situation, it might indeed be an appropriate strategy for your data, depending on what your power spectrum actually looks like.<br class="">
<br class="">
The design and test scenario was 50 Hz line noise in a typical home environment in Germany, which was actually a precise 50 Hz but of moderately fluctuating amplitude. These sorts of common fluctuations effectively broaden the noise frequency peak from a sharp
50 Hz to, for example, 49-51 Hz. For such a case, you would indeed define cfg.dftbandwidth of 2. The "neighbor" bandwidth is more tricky, but this should represent "clean" frequencies since their amplitude is used to replace the powerline corruption. In this
example, a neighbourbandwidth of 1 would imply that the amplitude at 48-49 Hz and 51-52 Hz would be used to "patch" the noise peak at 49-51 Hz.<br class="">
<br class="">
You may have a more extreme scenario, but nevertheless looking at the power spectrum will help you see how broad each powerline noise peak is (use that to set "dftbandwidth"), and ensure that you pick clean neighbor frequencies (guiding "neighbourbandwidth").<br class="">
<br class="">
Our paper describing the method has an example of real data and some tips on usage that might be helpful:<br class="">
https://doi.org/10.1016/j.neuroimage.2019.01.026 <https://doi.org/10.1016/j.neuroimage.2019.01.026><br class="">
<br class="">
Hope this helps!<br class="">
<br class="">
Cheers,<br class="">
Sarang<br class="">
<br class="">
<br class="">
On Sun, 2021-10-03 at 14:45 +0200, Stephen Whitmarsh via fieldtrip wrote:<br class="">
<blockquote type="cite" class="">Dear Emilie,<br class="">
<br class="">
Those sound indeed like challenging, and interesting, circumstances.<br class="">
I don't have a direct answer to your question, that would depend on having more information about the data, but here are my 2 cents;<br class="">
<br class="">
You could first make an FFT plot (e.g. using cfg.method = mtmfft in ft_freqanalysis) to get a better picture of the frequencies in your data (on both 'successful' and 'unsuccessful' data), for several reasons:<br class="">
You are also trying to remove both 50 and 60Hz, and their harmonics, of which probably only one of those would reflect the noise in the power lines.<br class="">
You are using cfg.dftreplace = 'neighbour'; which might only work if the frequency band of the noise is sharp enough, or can be estimated precise enough (depending on the duration of the data you are filtering).<br class="">
It would confirm whether it is line noise at all (which is probable, but more might be at play, e.g. some non-linear combinations and strange harmonics).<br class="">
You could look for whether the degree of noise is stationary, by e.g. plotting the FFT over different periods, but it might be more important that the frequency is accurately chosen and stable over time.<br class="">
Depending on your data, the duration of data, whether or not your are looking at trials or the whole data file at once (*), etc., you might also want to use a 'welch method' for estimating the power in the data, i.e. by averaging over several (sliding) windows.
Using cfg.method = 'mtmconvol' in ft_freqanalysis, then averaging over time with ft_selectdata (cfg.avgovertime), would be an easy way to do that. Once you have a good picture of the noise, it will be easier to both choose the right filters and to check their
efficacy (by comparing the FFT before and after filtering, on both 'successful' and 'unsuccessful' data). In fact, since you are dealing with EEG in such noisy conditions, and assuming you are interested in either ERPs or slow oscillations (<40Hz), you could
consider just using a lowpass filter at e.g. 40 Hz. <br class="">
<br class="">
Good luck, I hope this helps,<br class="">
Stephen <br class="">
<br class="">
*) The behaviour of filters depends a lot on the structure/size of the data, so to better understand its behaviour you would need to describe your data(structures) in more detail, i.e. trial length, nr. of trials/channels, and any processing steps until filtering.
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Op zo 3 okt. 2021 om 13:12 schreef Emilie Caspar via fieldtrip <fieldtrip@science.ru.nl <mailto:fieldtrip@science.ru.nl>>:<br class="">
<blockquote type="cite" class="">Dear Fieldtrippers,<br class="">
<br class="">
I recently acquired EEG data in Rwanda in complicated testing conditions, especially for the power line : we were testing in rural villages with a single electrical system for the whole village. Our signal is thus parasited by what the villagers were plugging
on the electrical system (so probably nonstationary noise) during the testing. The electrical system was also very rudimentary and not correctly grounded.<br class="">
<br class="">
Usually, I use the following command in Fieldtrip to remove the 50H or 60 Hz power line and it works quite well. Here, it worked for roughly 70% of our acquired data, but it does not work for all the data acquired.<br class="">
<br class="">
cfg= [];<br class="">
cfg.dftfilter = 'yes';<br class="">
cfg.dftfreq = [50 60 100 120 150 180];<br class="">
cfg.dftreplace = 'neighbour'; <br class="">
cfg.dftbandwidth = [2 2 2 2 2 2];<br class="">
cfg.dftneighbourwidth = [2 2 2 2 2 2];<br class="">
data_intpl = ft_preprocessing(cfg, allData_preprosses);<br class="">
<br class="">
<br class="">
When I look at the graphs (see figure attached), it really looks like it’s a 60Hz noise, but it seems that the dftfilter function does not remove it. We are certain it’s power noise because when the electricity was cut off because of a storm or else, and we
were thus only relying on the batteries to collect our data, the signal was perfect. So in theory we should be able to remove it but we have no cue of what other possibilities to try. Perhaps it’s because the noise is non stationary and dftfilter does not
account for that? I know some residual power noise can stay after a dftfiltre, but here it does not remove anything.<br class="">
<br class="">
Thanks a lot for the help, <br class="">
<br class="">
Emilie<br class="">
<br class="">
<PowerLine.jpg><br class="">
---------------------------------------------<br class="">
Prof. Dr. Emilie Caspar<br class="">
<br class="">
Associate Professor <br class="">
<br class="">
Department of Experimental Psychology, Ghent University<br class="">
office: Henri Dunantlaan, 2 - Floor 2, Room 94<br class="">
<br class="">
lab’s website: https://moralsocialbrain.com/ <https://moralsocialbrain.com/> <br class="">
personal website: https://emiliecaspar.home.blog/ <https://emiliecaspar.home.blog/><br class="">
<br class="">
Université libre de Bruxelles (office & contact): DB10.138 / +32 2 650 32 95<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip <https://mailman.science.ru.nl/mailman/listinfo/fieldtrip><br class="">
https://doi.org/10.1371/journal.pcbi.1002202 <https://doi.org/10.1371/journal.pcbi.1002202><br class="">
</blockquote>
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip <https://mailman.science.ru.nl/mailman/listinfo/fieldtrip><br class="">
https://doi.org/10.1371/journal.pcbi.1002202 <https://doi.org/10.1371/journal.pcbi.1002202><br class="">
</blockquote>
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
</blockquote>
<br class="">
-------------- next part --------------<br class="">
An HTML attachment was scrubbed...<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/30b3ca57/attachment.htm><br class="">
-------------- next part --------------<br class="">
A non-text attachment was scrubbed...<br class="">
Name: PowerLineNoise.001.jpeg<br class="">
Type: image/jpeg<br class="">
Size: 565683 bytes<br class="">
Desc: not available<br class="">
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20211004/30b3ca57/attachment.jpeg><br class="">
<br class="">
------------------------------<br class="">
<br class="">
Subject: Digest Footer<br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<br class="">
https://doi.org/10.1371/journal.pcbi.1002202<br class="">
<br class="">
<br class="">
------------------------------<br class="">
<br class="">
End of fieldtrip Digest, Vol 131, Issue 4<br class="">
*****************************************<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>