<div dir="ltr">Dear Stephen,<div><br></div><div>Yes, I now understand that the 3rd column needs to be adjusted too, and I think I've done that now, but still I am not able to visualise the markers with <i>ft_databrowser</i>. </div><div>I think I'm missing a very simple thing at this point, so I'm sorry to stretch out this thread. All of the information seems to be present in the redefined data, including in cfg.trl:</div><div><br></div><div>data_To_base_resp_E = <br><br> struct with fields:<br><br> hdr: [1×1 struct]<br> trial: {1×359 cell}<br> time: {1×359 cell}<br> fsample: 500<br> label: {29×1 cell}<br> trialinfo: [359×3 double]<br> sampleinfo: [359×2 double]<br> cfg: [1×1 struct]<br></div><div><br></div><div>data_To_base_resp_E.cfg = <br><br> struct with fields:<br><br> trl: [359×6 double]<br> checkpath: 'pedantic'<br> outputfilepresent: 'overwrite'<br> toolbox: [1×1 struct]<br> callinfo: [1×1 struct]<br> version: [1×1 struct]<br> offset: []<br> toilim: []<br> begsample: []<br> endsample: []<br> minlength: []<br> trials: 'all'<br> feedback: 'yes'<br> length: []<br> overlap: 0<br> previous: [1×1 struct]<br></div><div><br></div><div>data_To_base_resp_E.cfg.trl<br><br>ans = % begsample, endsample, offset, marker, trial number, accuracy code (last 2 are my additions to the trialinfo a few steps prior)<br><br> 8849 10098 -750 221 1 11<br> 10950 12199 -750 213 2 11<br> 12451 13700 -750 212 3 12<br> 14578 15827 -750 223 4 12<br> 16719 17968 -750 213 5 11<br></div><div><br></div><div>I suspected perhaps the empty fields offset: [] toilim: [] begsample: [] endsample: [] were the issue but manipulating them didn't help.</div><div>Below is my code:</div><div> u = repelem(-500,359)';<br>% epoch your trial data<br>For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,... % sampling rate is 500 hz, so begsample+750 in a -2.5 s onset gives me -1 s<br> data_resp_E.cfg.trl(:,2)-250,... % it was +1.5 s before, now is +1 s<br> u,... %offset is now 1 sec<br> data_resp_E.cfg.trl(:,4:6)); % the marker informations<br>cfg = [];<br>cfg.trl = For_trl;<br>data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E); <br></div><div><br></div><div><br></div><div>Thank you again for the help. </div><div><br></div><div>Best, </div><div><br></div><div>Duru</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 23 Sep 2019 at 10:51, Stephen Whitmarsh <<a href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="auto">Dear Duru,</div><div dir="auto"><br><div dir="auto">Great to hear you are getting along.</div><div dir="auto"><br></div><div dir="auto">1) yes, you are right, you should only average over time, i.e the second dimension (also see your .dimord field)</div><div dir="auto"><br></div><div dir="auto">2) We should really make a .trl FAQ, you are not the only one confused! The .trl goes like this: [start sample, end sample, offset]. Offset determines your t=0, <i>not </i>by setting a sample at t=0 (which you probably think intuitively), but by telling FieldTrip where the trial starts with <i>respect to the begin sample</i>. So, in a typical scenario where you have a baseline period <i>before </i>your marker, your start sample sets the beginning of your period you will be extracting from the data, i.e some time <i>before </i>the marker, e.g. for 500ms at 1000hz samplerate that will be 500 samples before your marker. Now your offset then tells FieldTrip that at that sample you are 500 samples <i>before </i>the start of the trial, so your offset will be <i>minus </i>500.</div><div>For example: say your samplerate is 1000Hz, your data starts at sample 1, but your first marker is at samplenr. 12000, i.e. after 12 seconds. Now if you want a trial with a 500ms baseline before that marker, and lasting until 2 seconds after that marker, your trl will be: [12000-500, 12000+2000, -500]; <br></div><div><br></div><div>I hope this helps,</div><div>Stephen<br></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 23 Sep 2019, 10:14 Duru Gun Ozkan, <<a href="mailto:durugun.ozkan@uniroma1.it" target="_blank">durugun.ozkan@uniroma1.it</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Stephen,<div><br></div><div>Thank you so much for your detailed response. I think I'm mostly there, I just wanted to clarify a few points.</div><div><br></div><div>I followed your second suggestion, in your first loop, when you get the mean of the baseline period for each trial, I also specified the dimension, because it was resulting in an array rather than a single number, which was problematic later on. I wanted to double check this was correct, and that it is supposed to be a single number, otherwise the next loop doesn't compute, because there is 2000 data points per trial in the original data (4 seconds) and 101 data points in the baseline (200 ms). (There is a chance my understanding of the math isn't sufficient for this issue, hence the doublechecking)</div><div><br>for itrial = 1 : size(data_resp_E_baseline.trial,2)<br> baseline{itrial} = mean(data_resp_E_baseline.trial{itrial}<b>,2);</b><br>end<br><br>The thing is, I still seem to be losing marker information and marker time when I redefine the trial to be shorter, as it doesn't respect the time 0 anymore:</div><div><br></div><div>% epoch trial data: Here I add 1.5 second to the first column, (start time), and subtract .5 s from the second (end), and keep the rest of the trl matrix, then use it to redefine trial to be -1 1 : 2 s long<br>For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,...<br> data_resp_E.cfg.trl(:,2)-250,...<br> data_resp_E.cfg.trl(:,2:6));<br>cfg = [];<br>cfg.trl = For_trl;<br>data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E); <br><br>data_To_base_resp_E.trl = For_trl;<br><br>% epoch the baseline <br>For_cfg_trl = horzcat(data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,1)-750,...<br> data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,2)-750,...<br> data_resp_E.cfg.trl(:,3:6));<br>cfg = [];<br>cfg.trl = For_cfg_trl; <br>data_resp_E_baseline = ft_redefinetrial (cfg,data_resp_E); <br></div><div><br></div><div>I suspect this time I'm not getting redefine trial to work correctly. I think the baseline routine works perfectly though, so thank you for that! If you have any further suggestions, I would appreciate it!</div><div><br></div><div>Best,</div><div><br></div><div>Duru</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 20 Sep 2019 at 11:53, Stephen Whitmarsh <<a href="mailto:stephen.whitmarsh@gmail.com" rel="noreferrer" target="_blank">stephen.whitmarsh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Dear Duru,</div><div><br></div><div>1) Yes, cfg.trials does not work on ft_preprocessing, so you will end up concatinating ntrials x ntrials :-). I thought you would be using ft_timelockbaseline (but even there isn't a cfg.trials option. My bad. I'll make up for it below :-))<br></div><div>2) After appending data, the .sampleinfo field is removed, and no reference to the original data is made, as it is done by ft_databrowser to plot the events from the original data (who gets some info from the .cfg field as well, i.e. the original file). This is because typically appending data goes over several datasets, and references to 1 original file doesn't make sense anymore.</div><div>3) About baseline correcting with other data, i.e,. outside of the trial duration; It seems in general you would benefit from just making your own baseline correction, which will solve all your issues and is pretty simple.</div><div><br></div><div>For example (there are many ways, but this would be my way) you could do the following in which you preprocess and epoch both the baseline period, and trail period (including any other 'baseline' pre-stim period) separately<br></div><div><br></div><div>% preprocess data as you did before<br></div><div>cfg = [];</div><div>cfg.trl = 'your data trl'<br></div><div>
data_base_end_E = ft_preprocessing(cfg);</div><div><br></div><div>% preprocess only your baseline periods</div><div>cfg = [];</div><div>cfg.trial = 'your "BaselineWindow" however you want to define it, including an arbitrary offset, say 0; Of course the same amount of trials as your data'<br></div><div>data_baseline = ft_preprocessing(cfg);</div><div><br></div><div>% Calculate baseline value, i.e. the mean of the baseline period</div><div>for itrial = 1 : size(data_baseline.trial,2)</div><div> baseline{itrial} = mean(data_baseline.trial{itrial})</div><div>end<br>
</div><div><br></div><div>% now just divide your data with the baseline value, or subtract for absolute baseline, or make a ratio, whatever you want.</div><div>
data_base_end_E_baseline =
data_base_end_E;
</div><div>for itrial = 1 : size(
data_base_end_E.trial,2) <br></div><div>
data_base_end_E_baseline.trial{itrial} =
data_base_end_E.trial{itrial} / baseline{itrial};</div><div>end</div><div><br></div><div>Doing it like this, also allows you even to extract baselines of unequal duration is you want to. I now realize though that by preprocessing your data separately, you might introduce differences in e.g. filtering between the baseline period and the trial data, especially when using short and variable trial lengths for either. So, even better would be to preprocess you data first as one trial, and then use ft_redefine trials to separate it in either baseline and baseline+data. Let's write it out:</div><div><div><br></div><div>----------</div><div><br></div><div>% preprocess all your data as one trial, i.e. without epoching:<br></div><div>cfg = [];</div><div>cfg.dataset = 'your datafile';</div><div>cfg.hpfilter = ... etc...</div><div>...<br></div><div>all_data = ft_preprocessing(cfg);</div>
</div><div>
</div>
<div><br></div><div>% epoch your trial data<br></div><div>cfg = [];</div><div>cfg.trl = 'your data trl'<br></div><div>
data_base_end_E = ft_redefinetrial(cfg,all_data);</div><div><br></div><div>% preprocess only your baseline periods</div><div>cfg = [];</div><div>
cfg.trl = your "BaselineWindow" trl, however you want to define it, including an arbitrary offset, say 0; Of course the same amount of trials as your data<br></div><div>data_baseline =
ft_redefinetrial
(cfg,all_data);</div><div><br></div><div>% The rest is the same as above:</div><div><br></div><div>% Calculate baseline value, i.e. the mean of the baseline period</div><div>for itrial = 1 : size(data_baseline.trial,2)</div><div> baseline{itrial} = mean(data_baseline.trial{itrial})</div><div>end<br>
</div><div><br></div><div>% now just divide your data with the baseline value, or subtract for absolute baseline, or make a ratio, whatever you want.</div><div>
data_base_end_E_baseline =
data_base_end_E;
</div><div>for itrial = 1 : size(
data_base_end_E.trial,2) <br></div><div>
data_base_end_E_baseline.trial{itrial} =
data_base_end_E.trial{itrial} / baseline{itrial};</div><div>end</div><div><br></div>
<div>----------</div><div><br></div><div>This should take care of treating all data the same, and especially allows you e.g. to define a nice high-pass filter on all your data at once. There might be a typo in the code, but I how you get my idea.<br></div><div><br></div><div>Hope this helps,</div><div>Stephen<br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Op vr 20 sep. 2019 om 08:35 schreef Duru Gun Ozkan <<a href="mailto:durugun.ozkan@uniroma1.it" rel="noreferrer" target="_blank">durugun.ozkan@uniroma1.it</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Stephen,<div><br></div><div>Thank you for your response. It seemed like your suggestion helped, but I thave another potential issue now.</div><div>The problem is, when I use <i>ft_databrowser </i>to visualise the trials after baseline corection loop, I can no longer see the markers, even if I add <i> cfg.plotevents = 'yes'</i>. I can see that this information exists in the data structure, so I'm not sure if this is a problem per se. But the bigger issue is, after running these loops, the data becomes very large (even after being appended), impossible to save, and slows down the rest of my analysis, where regular baseline correction doesn't cause this. </div><div><br></div><div> Any ideas where I might be going wrong? I suspect I might have used the wrong kinds of brackets, but the curly brackets as in your suggestion caused an error about cell structures, and when I run it as I wrote, the data structures actually look fine.Or it might be the way that I made a loop to append the data, instead of spelling it out for all of the 360 trials.</div><div><br></div><div>Another somewhat related question, is it possible to have the baseline window outside of the defined trial length? </div><div><br></div><div> Here is what I did:</div><div><br></div><div> for trial_index = 1:360<br> <br> cfg = [];<br> cfg.demean = 'yes';<br> cfg.baselinewindow = [BaselineWindow(trial_index,1) BaselineWindow(trial_index,2)];<br> cfg.trials = trial_index;<br> bl_end_E(trial_index) = ft_preprocessing(cfg, data_end_E);<br> <br> end <br> </div><div>%% This results in the struct:</div><div> bl_end_E = 1×360 struct array with 8 fields:<br> hdr<br> fsample<br> trialinfo<br> sampleinfo<br> trial<br> time<br> label<br> cfg</div><div><br></div><div>%% Then I append the data:<br> data_base_end_E = bl_end_E(1);<br> <br> for trial_ind = 2:360<br> <br> cfg = [];<br> cfg.keepsampleinfo = 'yes';<br> data_base_end_E = ft_appenddata(cfg,data_base_end_E,bl_end_E(trial_ind));<br><br> end<br> </div><div>%% This results in losing hdr and fsample fields: data_base_end_E = struct with fields:<br><br> label: {29×1 cell}<br> trialinfo: [360×4 double]<br> sampleinfo: [360×2 double]<br> trial: {1×360 cell}<br> time: {1×360 cell}<br> cfg: [1×1 struct]<br>%% So I add them back:</div><div> base_end_E.hdr = data_end_E.hdr;<br> base_end_E.fsample = data_end_E.fsample;<br><br></div><div> %% Visual data inspection <br> cfg = [];<br> cfg.viewmode = 'vertical'; <br> cfg.continuous = 'no'; <br> cfg.blocksize = 1.5;<br> cfg.channel = {'all'}; <br> cfg = ft_databrowser(cfg,data_base_end_E);<br></div><div><div><br></div><div><br></div><div>Thanks again for the help!</div><div><br></div><div>Duru</div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><i style="font-size:12.8px">Duru G</i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>ü</i></span><i style="font-size:12.8px">n </i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>Ö</i></span><i style="font-size:12.8px">zkan</i><div style="font-size:12.8px"><span style="font-size:12.8px"><i><br></i></span></div><div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Ph.D. student in Cognitive Social and Affective Neuroscience.<br></b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Department of Psychology.<br>University of Rome "La Sapienza".<br>Via dei Marsi 78 - 00185 - Roma.</b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b> </b></font><b style="font-family:"trebuchet ms",sans-serif;font-size:x-small">e-mail: <a href="mailto:vanessa.era@uniroma1.it" style="color:rgb(17,85,204)" rel="noreferrer" target="_blank">durugun.ozkan@uniroma1.it</a></b></div><div><font size="1" color="#1155cc"><u><a href="https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor" rel="noreferrer" target="_blank">https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor</a></u></font><br></div></div><div><font size="1" color="#1155cc"><br></font></div><div>> Hi Duru,<br><br>> Use cfg.trials = trial_index<br>> Put the output in a struct, i.e. bl{trial_index}<br>> Then combine with ft_append_data([],bl{:});<br><br>> Hope this helps,<br>> Stephen<br><br><br>> On Tue, 17 Sep 2019, 19:30 Duru Gun Ozkan, <<a href="mailto:durugun.ozkan@uniroma1.it" rel="noreferrer" target="_blank">durugun.ozkan@uniroma1.it</a>><br>> wrote:<br><br>> Hi everyone,<br>><br>> I have a question regarding specifying a different baseline correction<br>> time for each trial in ft_preprocessing.<br>> I had 360 audio stimuli all of which had different lengths. My trigger is<br>> located at the end of each stimuli, and I would like to place the baseline<br>> correction between 200 ms before stimulus start and stimulus start.<br>> I have a matrix for cfg.baselinewindow called BaselineWindow such as this:<br>> -1.83018 -1.630182<br>> -1.69807 -1.498071<br>> -0.58653 -0.38653<br>> -1.07604 -0.876039<br>> -2.2608 -2.060803<br>> -0.80863 -0.608632<br>> -1.55663 -1.356629<br>> -0.94261 -0.742605<br>> -1.20845 -1.008448 ...<br>> I tried to create a loop:<br>><br>> for trial_index = 1:360<br>><br>> cfg = [];<br>> cfg.demean = 'yes';<br>> cfg.baselinewindow = [BaselineWindow(trial_index,1)<br>> BaselineWindow(trial_index,2)];<br>><br>> data_baselined_end_E = ft_preprocessing(cfg, data_end_E);<br>><br>> end<br>><br>> This didn't work because it kept running the preprocessing with all the<br>> baselines with all trials, instead of keeping to its specific trial.<br>><br>> My question is, is there another way to specify individual baselines for<br>> individual trials? Or can anyone suggest to improve this loop to have the<br>> data preprocessed with its specific baseline windows?<br>><br>> Thank you in advance.<br>><br>> Best,<br>><br>> <br><font size="1" color="#1155cc"><br></font></div></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><i style="font-size:12.8px">Duru G</i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>ü</i></span><i style="font-size:12.8px">n </i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>Ö</i></span><i style="font-size:12.8px">zkan</i><div style="font-size:12.8px"><span style="font-size:12.8px"><i><br></i></span></div><div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Ph.D. student in Cognitive Social and Affective Neuroscience.<br></b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Department of Psychology.<br>University of Rome "La Sapienza".<br>Via dei Marsi 78 - 00185 - Roma.</b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b> </b></font><b style="font-family:"trebuchet ms",sans-serif;font-size:x-small">e-mail: <a href="mailto:vanessa.era@uniroma1.it" style="color:rgb(17,85,204)" rel="noreferrer" target="_blank">durugun.ozkan@uniroma1.it</a></b></div><div><font size="1" color="#1155cc"><u><a href="https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor" rel="noreferrer" target="_blank">https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor</a></u></font><br></div></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><i style="font-size:12.8px">Duru G</i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>ü</i></span><i style="font-size:12.8px">n </i><span style="font-family:Calibri,sans-serif;font-size:14.6667px;text-align:justify"><i>Ö</i></span><i style="font-size:12.8px">zkan</i><div style="font-size:12.8px"><span style="font-size:12.8px"><i><br></i></span></div><div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Ph.D. student in Cognitive Social and Affective Neuroscience.<br></b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b>Department of Psychology.<br>University of Rome "La Sapienza".<br>Via dei Marsi 78 - 00185 - Roma.</b></font></div><div style="font-size:12.8px;color:rgb(136,136,136)"><font size="1" face="trebuchet ms, sans-serif"><b> </b></font><b style="font-family:"trebuchet ms",sans-serif;font-size:x-small">e-mail: <a href="mailto:vanessa.era@uniroma1.it" style="color:rgb(17,85,204)" target="_blank">durugun.ozkan@uniroma1.it</a></b></div><div><font color="#1155cc" size="1"><u><a href="https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor" target="_blank">https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor</a></u></font><br></div></div></div></div></div></div></div></div></div>