<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">Hi Kris,</span>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class="">
</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">I think that Eelke meant that the Z-scoring leads to the threshold becoming independent of the signals’ physical units.</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">I agree that the signals have a non-Gaussian distribution, but this is equally true for non-envelope signals.</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Personally I think that normalisation with an estimate of the median or mad at this stage of an analysis pipeline is not going to change the outcome of the science much (i.e. the inferential
decisions), but if you have a good implementation of this, we could certainly build this in as an option. I would however welcome much more improvements to the way that we compute our test-statistics that we use for inferential decision making, e.g. using
Yuen-Welch T-statistics based on trimmed means and variances, rather than the good old T-statistic.</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class="">
</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Feel free to submit your version of the code as a PR to the github repository.</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class="">
</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class="">
</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Best wishes,</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class="">
</div>
<div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Jan-Mathijs</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 19 Feb 2020, at 13:13, De Meyer, Kris <<a href="mailto:kris.de_meyer@kcl.ac.uk" class="">kris.de_meyer@kcl.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Thank you for your response, Eelke.</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
I guess it makes sense but it does make the selection of a threshold more arbitrary. </div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
If your underlying distribution is normal, then a z-score of 3, 4 or 5 has a clear meaning in terms of outlier detection. </div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
For a non-Gaussian distribution, that interpretation no longer holds, and rejection becomes less automated and more open to subjective interpretation.</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Isn't it better to z-transform the Hilbert-envelope values with the mean/std or the median/mad of the original filtered EEG distribution rather than the mean/std or median/mad of the hilbert values distribution? That way, the meaning of the threshold magnitude
would remain valid. Or is that what is happening in the code already?</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Best,</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Kris</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<font face="Calibri, Arial, Helvetica, sans-serif" class=""><br class="">
</font></div>
<div id="appendonsend" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
</div>
<hr tabindex="-1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline-block; width: 1088.765625px;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""></span>
<div id="divRplyFwdMsg" dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>fieldtrip <<a href="mailto:fieldtrip-bounces@science.ru.nl" class="">fieldtrip-bounces@science.ru.nl</a>> on behalf of Eelke
Spaak <<a href="mailto:e.spaak@donders.ru.nl" class="">e.spaak@donders.ru.nl</a>><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>18 February 2020 10:45<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>FieldTrip discussion list <<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [FieldTrip] question about automatic artefact rejection</font>
<div class=""> </div>
</div>
<div class="BodyFragment" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<font size="2" class=""><span style="font-size: 11pt;" class="">
<div class="PlainText">Dear Kris,<br class="">
<br class="">
Since there is no statistical test involved, I see no problems with<br class="">
computing a z-score here. Think of it more as bringing the data into a<br class="">
sensible range and allowing yourself to forget about details like<br class="">
measurement unit etc. You could also not do the z-transform, and<br class="">
simply specify a threshold based on raw amplitude, this would be<br class="">
equivalent (but might make it potentially more difficult to come up<br class="">
with a good threshold).<br class="">
<br class="">
Best,<br class="">
Eelke<br class="">
<br class="">
On Tue, 18 Feb 2020 at 09:47, De Meyer, Kris <<a href="mailto:kris.de_meyer@kcl.ac.uk" class="">kris.de_meyer@kcl.ac.uk</a>> wrote:<br class="">
><br class="">
> Dear Fieldtrippers,<br class="">
><br class="">
> In ft_artifact_muscle, an EEG signal is first filtered - usually with a bandpass filter with cutoff frequencies of [110,140]. The documentation then says that the hilbert envelope is calculated, after which the envelope values are z-transformed before they
are thresholded.<br class="">
><br class="">
> However, is this sound? Whereas the actual filtered EEG values (the result of the bandpass filter) are usually normally distributed, the hilbert envelope values are far from.<br class="">
><br class="">
> What is the rationale for calculating the z-transform of a non-Gaussian distribution? Doesn't that introduce problems?<br class="">
><br class="">
> Best,<br class="">
><br class="">
> Kris<br class="">
><br class="">
><br class="">
><br class="">
><br class="">
><br class="">
> _______________________________________________<br class="">
> fieldtrip mailing list<br class="">
><span class="Apple-converted-space"> </span><a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=miiXPIhNF0zd23xrFrXakUxAUldGY6DZLnhoajNtLlg%3D&reserved=0" class="">https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=miiXPIhNF0zd23xrFrXakUxAUldGY6DZLnhoajNtLlg%3D&reserved=0</a><br class="">
><span class="Apple-converted-space"> </span><a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=DNmcynQGcA4%2FAmDi8PSk8UE%2FBfy0Ce8yGRPr3Cv%2BvZ8%3D&reserved=0" class="">https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=DNmcynQGcA4%2FAmDi8PSk8UE%2FBfy0Ce8yGRPr3Cv%2BvZ8%3D&reserved=0</a><br class="">
<br class="">
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
<a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=miiXPIhNF0zd23xrFrXakUxAUldGY6DZLnhoajNtLlg%3D&reserved=0" class="">https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=miiXPIhNF0zd23xrFrXakUxAUldGY6DZLnhoajNtLlg%3D&reserved=0</a><br class="">
<a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=DNmcynQGcA4%2FAmDi8PSk8UE%2FBfy0Ce8yGRPr3Cv%2BvZ8%3D&reserved=0" class="">https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=01%7C01%7Ckris.de_meyer%40kcl.ac.uk%7C87c72918f62a46527c2408d7b4676f15%7C8370cf1416f34c16b83c724071654356%7C0&sdata=DNmcynQGcA4%2FAmDi8PSk8UE%2FBfy0Ce8yGRPr3Cv%2BvZ8%3D&reserved=0</a><br class="">
</div>
</span></font></div>
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">fieldtrip
mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<a href="https://doi.org/10.1371/journal.pcbi.1002202" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://doi.org/10.1371/journal.pcbi.1002202</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
</div>
</blockquote>
</div>
<br class="">
</body>
</html>