<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Dear Jan-Mathijs,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Thank you for your reply.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
The error is indeed thrown by ft_artifact_threshold. Unfortunately, specifying cfg.continuous = 'yes' does not solve the problem. The error message after this change is the same as before:</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof ContentPasted2" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Error using ft_fetch_data
<div class="ContentPasted2">some of the requested samples occur twice in the data and have conflicting values</div>
<div><br class="ContentPasted2">
</div>
<div class="ContentPasted2">Error in ft_artifact_threshold (line 186)</div>
<div class="ContentPasted2"> dat = ft_fetch_data(data, 'header', hdr, 'begsample', trl(trlop,1), 'endsample', trl(trlop,2), 'chanindx', chanindx, 'checkboundary', strcmp(cfg.continuous, 'no'));</div>
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Is there anything else I can try?</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Best,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Maria</div>
<div class="elementToProof">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div class="WordSection1">
<p class="MsoNormal" style="margin-top: 0px; margin-bottom: 0px;"><span style="font-size: 10pt; color: black;">------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-top: 0px; margin-bottom: 0px;"><br>
</p>
<div class="MsoAutoSig" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; margin-top: 0px; margin-bottom: 0px; color: rgb(0, 0, 0);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Date: Fri, 27 Oct 2023 08:04:42 +0000</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">From: "Schoffelen, J.M. (Jan Mathijs)"</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> <</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">janmathijs.schoffelen@donders.ru.nl</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">></span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">To: FieldTrip discussion list <</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">fieldtrip@science.ru.nl</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">></span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Subject: Re: [FieldTrip] Artifact rejection problem due to samples</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> occurring twice in the data</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Message-ID: <</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">244FB7A1-D7BD-4DDE-9B5A-486E940B4259@donders.ru.nl</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">></span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Content-Type: text/plain; charset="utf-8"</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Dear Maria,</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Thanks for your clear message and diagnostics so far. Just for the clear picture: the error is thrown by ft_artifact_threshold, isn’t it? If I read through
the code of this function, I get the impression that you might get yourself back into the game, if you specify cfg.continuous = ‘yes’ for the cfg into ft_artifact_threshold. Although, this feels a bit at odds with the representation of the data that you give
as an input to the function, because the time-axes of the individual trials clearly is not continuous. However, the sampelinfo field of the data object is still consistent with a continuous data representation (on disk). It’s the sampleinfo field that ft_fetch_data
uses under the hood to determine the overlapping samples. I think that the error will convert into a warning with cfg.continuous = ‘yes’; (unless you have already tried this and can disconfirm my hunch). So, long story short, I’d recommend cfg.continuous =
‘yes’.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Best wishes,</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Jan-Mathijs</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br>
<span class="ContentPasted0" style="font-size: 13.3333px; display: inline !important; background-color: rgb(255, 255, 255);">------------------------------------------------------------------------</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">On 26 Oct 2023, at 11:31, Hartog, M.M.J. den (Maria) via fieldtrip <</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">fieldtrip@science.ru.nl</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">mailto:fieldtrip@science.ru.nl</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">>>
wrote:</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Dear community,</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">My name is Maria den Hartog and I am a PhD candidate at the Centre for Language Studies (CLS) at Radboud University in Nijmegen. I am currently preprocessing
data from an ERP reading experiment about the processing of pronouns of address (e.g. he, she, they) within a text.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">I want to use automatic artifact rejection to remove full trials with signal exceeding +-100 µV as part of my preprocessing pipeline, but I get an error
message indicating that 'some of the requested samples occur twice in the data and have conflicting values'. This is true, and it is a consequence of the naturalistic reading paradigm I used. My trials run from -200 ms prestimulus to 1000 ms poststimulus.
The t = 0 of each trial is the onset of a pronoun within a text, and the pronouns can occur less than 1000 ms after each other (e.g., the stimuli are he and his in 'He looked at his cat.'). I have baselined my trials to the 200 ms window up to the stimulus,
meaning the same sample can have different values in different trials.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">The code I use for artifact rejection is:</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> cfg = [];</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> cfg.continuous = 'no';</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> cfg.artfctdef.threshold.min = -100;</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> cfg.artfctdef.threshold.max = 100;</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> art = ft_artifact_threshold(cfg,data);</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> cfg = [];</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> <span> </span></span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">cfg.artfctdef.xxx</span><span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">.artifact
= art.artfctdef.threshold.artifact;</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> data_forERP = ft_rejectartifact(cfg,data);</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
</div>
<div class="MsoAutoSig" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; margin-top: 0px; margin-bottom: 0px; color: rgb(0, 0, 0);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><br>
</span></div>
<div class="MsoAutoSig" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; margin-top: 0px; margin-bottom: 0px; color: rgb(0, 0, 0);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">The error I get:</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
</div>
<div class="MsoAutoSig" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; margin-top: 0px; margin-bottom: 0px; color: rgb(0, 0, 0);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Error using ft_fetch_data</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">some of the requested samples occur twice in the data and have conflicting values</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Error in ft_artifact_threshold (line 186)</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> dat = ft_fetch_data(data, 'header', hdr, 'begsample', trl(trlop,1), 'endsample', trl(trlop,2), 'chanindx', chanindx, 'checkboundary', strcmp(cfg.continuous,
'no'));</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Is there a way to adjust the cfg for ft_rejectartifact so that the function will accept that samples do indeed occur twice in the data and that the same
sample can have different values in different trials (and data from the correct trial will be fetched)?</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Here are a few solutions that I have tried based on other discussion list questions, that I have found to be unsatisfactory or ineffective:</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> * For ft_databrowser, it is possible to use cfg.allowoverlap = 'yes' to avoid errors due to conflicting sample values, but this setting is not available
for ft_rejectartifact.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> * I do not want to remove or adjust the sampleinfo from the data, because I want to be able to relate trials to the original samples if necessary.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"> * I have found that downsampling my signal will lead ft_rejectartifact to reconstruct the sampleinfo assuming that the trials are consecutive segments
of a continuous recording, but I do not want to 'hack' my way around the problem in this way.</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Hope you can help me out!</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Best,</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<span style="display: inline !important; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">Maria</span><br style="color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
<br>
</div>
</div>
</div>
</div>
</body>
</html>