<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Hans,
<div class=""><br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 29 Nov 2022, at 14:17, Hans Revers via fieldtrip <<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dear FieldTrip community,<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
My name is Hans Revers. I am a PhD student in the Cognitive Neuropsychology department at Tilburg University, working on decoding emotions from EEG patterns.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I would appreciate your insight on how to handle NaNs in TFR power spectra resulting from partial artifact rejection.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
We have trials of 90 seconds duration, with artifacts marked from pre-processing in Brain Vision Analyzer. We wish to do a time-frequency analysis (montecarlo cluster analysis) and a classification (mvpa-light). We would want to exclude only the artefact parts
per trial while retaining the non-artifact parts and the temporal information. Partial rejection of artifacts appears to be the way to achieve this (<a href="https://mailman.science.ru.nl/pipermail/fieldtrip/2018-May/012138.html" style="color: rgb(149, 79, 114); text-decoration: underline;" class="">https://mailman.science.ru.nl/pipermail/fieldtrip/2018-May/012138.html</a>).<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
% reject artefact cfg<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
cfg_artifact = [];<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
cfg_artifact.artfctdef.reject = 'partial';<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
% remove artifacts<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
cfg_artifact.artfctdef.xxx.artifact = formatArtifactList(Dairy_per_pp(pp_i).cfg);<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
% The formatArtifactList() function returns an Nx2 matrix containing<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
% the start/end samples of each ‘Bad Interval’.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
ConditionA_per_pp(pp_i) = ft_rejectartifact(cfg_artifact, ConditionA _per_pp(pp_i));<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This works as expected. Artifact intervals are stored in the cfg of the data in artfcdef.xxx.artifact. Next, ft_freqanalysis results in NaNs in the power spectrum for these intervals with a half taper width before and after. Baseline normalization with ft_freqbaseline
seems to handle these NaNs just fine. So far, so good. (I guess we need to make sure that the baseline does not consist of solely NaNs.)<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
However, when we average over participants with ft_freqgrandaverage, the GA results in a NaN for each time bin that has a NaN for one (or more) of the participants. We had expected a nanmean result.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This could be easily fixed by calculating our own nanmean GA powerspectrum, but I am hesitant to alter the intended workings of FieldTrip and I have the following concerns:<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Question 1) Are there reasons not to use a nanmean GA?<span class="Apple-converted-space"> </span></div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
No, there is not reason not to use a nanmean. Particularly, if the average is ‘just’ used for visualization or so. Note that the ft_freqgrandaverage is historically an old beast. I can imagine that it does not deal gracefully with NaNs, if they happen to pop
up in different time points for different subjects (which - in itself - I find a bit surprising, because if ft_freqanalysis deals with the NaNs gracefully, I would expect consistency across subjects). As an alternative to ft_freqgrandaverage, you could try
a combo call to ft_appendfreq, followed by a call to ft_selectdata (with cfg.avgoverrpt = ‘yes’). Ft_selectdata has functionality to use nanmean, or mean for the averaging.</div>
<div>
<div><br class="">
</div>
<blockquote type="cite" class="">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
ft_freqstatistics appears to run unaffected by the presence of NaNs in the per-pp power spectra. We are using non-parametric montecarlo cluster analysis with depsamplesT test. ft_statfun_depsamplesT seems to use a nanmean in the calculation of the statistics.<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Question 2) Can we assume that the statistics are reliable? Or do the NaNs compromise the statistics?</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>I’m not sure what you mean with ‘reliable’, but the inferential procedure (+ its conclusion) will still be valid. I guess that as long as the percentage of NaNs for a given data point is low given the total number of subjects, there may be some more noise
on the test statistic for those data points (compared to the data points to which all subjects have contributed).</div>
<div><br class="">
</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
And similarly<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Question 3) Will mvpa classification results be reliable?</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>Well, I’d say that mvpa classification is done at the first (subject) level, and group level statistics are performed on the performance scores. So this sounds to me as a slightly different issue. I don’t know whether (and how) the mvpa works with missing
data (partial artifact rejection).</div>
<div><br class="">
</div>
<div>Best wishes,</div>
<div>Jan-Mathijs</div>
<div><br class="">
</div>
<div><br class="">
</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Any help would be appreciated.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kind regards,<span style="font-size: 10pt; font-family: Arial, sans-serif;" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 10pt; font-family: Arial, sans-serif;" class="">Hans Revers<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<a href="https://urldefense.com/v3/__https://www.tilburguniversity.edu/about/schools/socialsciences/organization/departments/cognitive-neuropsychology.htm__;!!HJOPV4FYYWzcc1jazlU!-brl_sNsdOMz0Oj0QzbxpMbDjqgae5VDMPPesqlreMz47FFf2RBY8SMXg_fyY2jCzLSvy2MESGigE9NMDBvLoMD7knjPjBCSgvu0og$" style="color: rgb(149, 79, 114); text-decoration: underline;" class=""><span style="font-size: 10pt; font-family: Arial, sans-serif;" class="">Department
of Cognitive Neuropsychology</span></a><span style="font-size: 10pt; font-family: Arial, sans-serif;" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 10pt; font-family: Arial, sans-serif;" class="">Tilburg University<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">fieldtrip
mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" style="color: rgb(149, 79, 114); text-decoration: underline; font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<a href="https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-brl_sNsdOMz0Oj0QzbxpMbDjqgae5VDMPPesqlreMz47FFf2RBY8SMXg_fyY2jCzLSvy2MESGigE9NMDBvLoMD7knjPjBAhIwf6NQ$" style="color: rgb(149, 79, 114); text-decoration: underline; font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-brl_sNsdOMz0Oj0QzbxpMbDjqgae5VDMPPesqlreMz47FFf2RBY8SMXg_fyY2jCzLSvy2MESGigE9NMDBvLoMD7knjPjBAhIwf6NQ$</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""></span></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>