<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="en-DE" link="blue" vlink="purple" style="word-wrap:break-word;-webkit-nbsp-mode: space;line-break:after-white-space">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Hi Jan-Mathjis,
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thank you for your reply, these are very helpful notes.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Indeed, since my pipeline runs through without these warnings when I do not redefine the trials, the issue should be with the steps that attempt to create longer trials from the shorter
resting-state trials. Yet, the redefined data does not (at least explicitly) include NaN values, and the outputted structs for these steps look as they should, with the relevant fields and no explicit NaNs.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Exactly as you suggested, when digging into the issue a bit further, it looks like NaN values are first introduced when computing the covariance in ft_timelockanalysis. When running
the ft_timelockanalysis function without the covariance configuration I do not receive a warning about NaNs, but I need the covariance for later steps so I of course cannot omit this configuration nor would this solve the issue.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Exactly as you pointed out again, the NaNs in the covariance seem to then affect the rest of the steps downstream including computing the spatial filters and the ft_virtualchannel function.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">To answer your question: I want to create longer 10-second trials from the 2-second trials because I need a longer trial length for the analyses I later conduct. I’m running power spectral
density analyses, fitting oscillations and one-over-f (FOOOF) in Python, and later entropy analyses, for which 2-second trials simply aren’t long enough to generate “decent” results or to visualize results appropriately.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I know there are multiple ways to redefine trial length, and I have tried several approaches (such as defining cfg.begsample/endsample, cfg.continuous, and reshaping the original data
by hand before passing it to ft_redefinetrial). My main question would therefore be how to ensure that the time axes of the original small snippets match up without gaps when redefining, as I’m sure there must be a way. I check the trial, label, and time fields
for the outputs when redefining the trials and, again, all looks as it should, without any NaNs that seem to “fill the gaps”.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">And an important final note would be then that the NaNs are exactly introduced in the covariance step rather than anywhere earlier (no NaNs in the data itself before or after timelocking),
so something seems to be going wrong when computing the covariance.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thanks a lot again,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Marta<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">fieldtrip <fieldtrip-bounces@science.ru.nl> on behalf of "Schoffelen, J.M. (Jan Mathijs) via fieldtrip" <fieldtrip@science.ru.nl><br>
<b>Reply to: </b>FieldTrip discussion list <fieldtrip@science.ru.nl><br>
<b>Date: </b>Thursday, 9. March 2023 at 09:17<br>
<b>To: </b>FieldTrip discussion list <fieldtrip@science.ru.nl><br>
<b>Cc: </b>"Schoffelen, J.M. (Jan Mathijs)" <janmathijs.schoffelen@donders.ru.nl><br>
<b>Subject: </b>Re: [FieldTrip] Warning with redefining data<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Hi Marta, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">You say: "I’ve run a loop over the trial field in the output of the ft_timelockanalysis as well as ft_preprocessing to check for NaNs in my 10-second data structs, but I didn’t find any when running this loop.”<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">This is at odds with your earlier statement that you got a warning that the data contains NaNs. </span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The NaNs have probably been introduced in your attempt to ‘glue together’ the 2-second epochs into longer snippets of 10 seconds. If the time axes of the original small snippets does not entirely match up (i.e. if there were gaps) then
ft_redefinetrial might have filled those gaps with NaNs, while creating the 10 second snippets.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">FieldTrip functions that operate on data containing NaNs often gracefully ignore them, but in many occasions the NaNs will affect the output. For instance (as is likely in your case), I think that the computation of the covariance in ft_timelockanalysis
is not NaN-aware. This may cause downstream problems, when you use an all-NaN covariance matrix for the computation of spatial filters, which consequently will by construction also be all-NaN, as will be the virtual channel data that you try to derive from
it. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The relevant question to ask here is: why do you want to stitch together the 2 second snippets of data?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Good luck and best wishes,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Jan-Mathijs<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On 7 Mar 2023, at 21:09, Marta Stojanović via fieldtrip <<a href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi Fieldtrip community, <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I’m reaching out with a question about a warning I encountered when redefining trial length from resting-state MEG data using ft_definetrial and ft_redefine trial.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Namely, I redefined the length of input data from a resting-state MEG file from two-second trials to longer trials of 10-second length. I successfully used the ft_definetrial and ft_redefinetrial functions to do so, using the ‘trl’ field from the
struct where I define the trial to input as a configuration for the ft_redefinetrial function.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">With the two-second length I have 144 trials from the data, while with the ten-second length there are 28 trials.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The pipeline and structs look like the following: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">%% define trials with 10-second length <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">clear ten_seconds;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg = [];<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.dataset = restfile; % refers to resting-state file of a single subject<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.trialdef.beginning = 0; % start at the beginning of the recording<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.trialdef.length = 10; % 10-second trial length<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.trialdef.overlap = 0; % no overlap between trials<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ten_seconds = ft_definetrial(cfg);<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">%% redefine trials to 10 seconds<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">clear redef_data;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg = [];<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.trl = ten_seconds.trl; % use trl from defined trial<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">redef_data = ft_redefinetrial(cfg,data);<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">%% preprocess redefined 10-second trials<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">clear ten_sec_trls;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg = [];<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.demean = 'yes';<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg.baselinewindow = 'all';<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">% cfg.baselinewindow = [0 1018];<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ten_sec_trls = ft_preprocessing(cfg,redef_data);<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">These are the associated structs with their fields:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">redef_data = <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">fsample: 508.6275<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">grad: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">label: {244×1 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">hdr: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">trial: {1×28 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">time: {1×28 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">trialinfo: [28×2 double]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">sampleinfo: [28×2 double]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ten_sec_trls = <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">fsample: 508.6275<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">grad: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">hdr: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">trialinfo: [28×2 double]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">trial: {1×28 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">time: {1×28 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">label: {244×1 cell}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">cfg: [1×1 struct]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">When I plug the redefined data (i.e., with redefined length, ten_sec_trls) into ft_timelockanalysis, I receive the<span class="apple-converted-space"> </span><span lang="EN-US">following<span class="apple-converted-space"> </span></span>warning:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Warning: data contains NaN values <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_preproc_polyremoval at line 75<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_preproc_baselinecorrect at line 52<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_timelockanalysis at line 174<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">the call to "ft_selectdata" took 0 seconds and required the additional allocation of an estimated 22 MB<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Warning: inconsistent sampleinfo <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In makessense at line 115<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_datatype_timelock at line 72<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_checkdata at line 479<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> In ft_timelockanalysis at line 207<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I’ve run a loop over the trial field in the output of the ft_timelockanalysis as well as ft_preprocessing to check for NaNs in my 10-second data structs, but I didn’t find any when running this loop.<span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">I’ve
checked the relevant lines from the Fieldtrip functions but couldn’t find a way to solve the warning from there.<span class="apple-converted-space"> </span></span>I’ve also checked the sampleinfo, but I sense the warning may be due to the sampleinfo in the
configuration being inconsistent with the actual data due to the removal of non-MEG channels from the sampleinfo in the resting file.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The warning message might influence later analyses, so I’m wondering if anyone has faced the same and/or has an explanation. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks a lot in advance!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">All the best,<br>
Marta<o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Helvetica">_______________________________________________<br>
fieldtrip mailing list<br>
</span><a href="https://urldefense.com/v3/__https://eur04.safelinks.protection.outlook.com/?url=https*3A*2F*2Fmailman.science.ru.nl*2Fmailman*2Flistinfo*2Ffieldtrip&data=05*7C01*7Cmarta.stojanovic*40student.uva.nl*7Ca176944561f24559cdde08db2076b219*7Ca0f1cacd618c4403b94576fb3d6874e5*7C0*7C0*7C638139466402430326*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C&sdata=OsJDvoBFn3F3sGMfA0o9ij70Gs0uIamb0T4nX*2FurMJk*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!HJOPV4FYYWzcc1jazlU!-2PB3F72YfAhGO8MRm7edBkmsxtIIj1iHidGe-fn3vBb24hJrp61lSH--nnDL8k83-HbawlwfBwoEkjHHo0Q-aac2JFeVJcIuijxiFw$"><span style="font-size:10.5pt;font-family:Helvetica">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</span></a><span style="font-size:10.5pt;font-family:Helvetica"><br>
</span><a href="https://urldefense.com/v3/__https://eur04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fdoi.org*2F10.1371*2Fjournal.pcbi.1002202__*3B!!HJOPV4FYYWzcc1jazlU!44dPKLWAx3oGspX-Q-IZx8b_gfPmQbTd45nbdyM61ROZgOiO2314lg4aY5eGcvDpKtQW3t1ilWOQcQ56Yd02ltXfOme3MNrQw0Ahfg*24&data=05*7C01*7Cmarta.stojanovic*40student.uva.nl*7Ca176944561f24559cdde08db2076b219*7Ca0f1cacd618c4403b94576fb3d6874e5*7C0*7C0*7C638139466402430326*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C&sdata=SSG4qTaXpWJjWwF6LzGTFa1t*2BJwucoKKkAla2GnPI8o*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!HJOPV4FYYWzcc1jazlU!-2PB3F72YfAhGO8MRm7edBkmsxtIIj1iHidGe-fn3vBb24hJrp61lSH--nnDL8k83-HbawlwfBwoEkjHHo0Q-aac2JFeVJcIW-6Dbs4$"><span style="font-size:10.5pt;font-family:Helvetica">https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!44dPKLWAx3oGspX-Q-IZx8b_gfPmQbTd45nbdyM61ROZgOiO2314lg4aY5eGcvDpKtQW3t1ilWOQcQ56Yd02ltXfOme3MNrQw0Ahfg$</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>