[FieldTrip] Single trial baseline correction in ft_preprocessing
Duru Gun Ozkan
durugun.ozkan at uniroma1.it
Mon Sep 23 15:39:19 CEST 2019
Dear Stephen,
Yes, I now understand that the 3rd column needs to be adjusted too, and I
think I've done that now, but still I am not able to visualise the markers
with *ft_databrowser*.
I think I'm missing a very simple thing at this point, so I'm sorry to
stretch out this thread. All of the information seems to be present in the
redefined data, including in cfg.trl:
data_To_base_resp_E =
struct with fields:
hdr: [1×1 struct]
trial: {1×359 cell}
time: {1×359 cell}
fsample: 500
label: {29×1 cell}
trialinfo: [359×3 double]
sampleinfo: [359×2 double]
cfg: [1×1 struct]
data_To_base_resp_E.cfg =
struct with fields:
trl: [359×6 double]
checkpath: 'pedantic'
outputfilepresent: 'overwrite'
toolbox: [1×1 struct]
callinfo: [1×1 struct]
version: [1×1 struct]
offset: []
toilim: []
begsample: []
endsample: []
minlength: []
trials: 'all'
feedback: 'yes'
length: []
overlap: 0
previous: [1×1 struct]
data_To_base_resp_E.cfg.trl
ans = % begsample, endsample, offset, marker, trial number, accuracy code
(last 2 are my additions to the trialinfo a few steps prior)
8849 10098 -750 221 1 11
10950 12199 -750 213 2 11
12451 13700 -750 212 3 12
14578 15827 -750 223 4 12
16719 17968 -750 213 5 11
I suspected perhaps the empty fields offset: [] toilim: []
begsample: [] endsample: [] were the issue but manipulating them didn't
help.
Below is my code:
u = repelem(-500,359)';
% epoch your trial data
For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,... % sampling rate is 500
hz, so begsample+750 in a -2.5 s onset gives me -1 s
data_resp_E.cfg.trl(:,2)-250,... % it was +1.5 s before,
now is +1 s
u,... %offset is now 1 sec
data_resp_E.cfg.trl(:,4:6)); % the marker informations
cfg = [];
cfg.trl = For_trl;
data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
Thank you again for the help.
Best,
Duru
On Mon, 23 Sep 2019 at 10:51, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
wrote:
> Dear Duru,
>
> Great to hear you are getting along.
>
> 1) yes, you are right, you should only average over time, i.e the second
> dimension (also see your .dimord field)
>
> 2) We should really make a .trl FAQ, you are not the only one confused!
> The .trl goes like this: [start sample, end sample, offset]. Offset
> determines your t=0, *not *by setting a sample at t=0 (which you probably
> think intuitively), but by telling FieldTrip where the trial starts with *respect
> to the begin sample*. So, in a typical scenario where you have a baseline
> period *before *your marker, your start sample sets the beginning of your
> period you will be extracting from the data, i.e some time *before *the
> marker, e.g. for 500ms at 1000hz samplerate that will be 500 samples before
> your marker. Now your offset then tells FieldTrip that at that sample you
> are 500 samples *before *the start of the trial, so your offset will be *minus
> *500.
> For example: say your samplerate is 1000Hz, your data starts at sample 1,
> but your first marker is at samplenr. 12000, i.e. after 12 seconds. Now if
> you want a trial with a 500ms baseline before that marker, and lasting
> until 2 seconds after that marker, your trl will be: [12000-500,
> 12000+2000, -500];
>
> I hope this helps,
> Stephen
>
>
> On Mon, 23 Sep 2019, 10:14 Duru Gun Ozkan, <durugun.ozkan at uniroma1.it>
> wrote:
>
>> Dear Stephen,
>>
>> Thank you so much for your detailed response. I think I'm mostly there, I
>> just wanted to clarify a few points.
>>
>> I followed your second suggestion, in your first loop, when you get the
>> mean of the baseline period for each trial, I also specified the dimension,
>> because it was resulting in an array rather than a single number, which was
>> problematic later on. I wanted to double check this was correct, and that
>> it is supposed to be a single number, otherwise the next loop doesn't
>> compute, because there is 2000 data points per trial in the original data
>> (4 seconds) and 101 data points in the baseline (200 ms). (There is a
>> chance my understanding of the math isn't sufficient for this issue, hence
>> the doublechecking)
>>
>> for itrial = 1 : size(data_resp_E_baseline.trial,2)
>> baseline{itrial} = mean(data_resp_E_baseline.trial{itrial}*,2);*
>> end
>>
>> The thing is, I still seem to be losing marker information and marker
>> time when I redefine the trial to be shorter, as it doesn't respect the
>> time 0 anymore:
>>
>> % epoch trial data: Here I add 1.5 second to the first column, (start
>> time), and subtract .5 s from the second (end), and keep the rest of the
>> trl matrix, then use it to redefine trial to be -1 1 : 2 s long
>> For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,...
>> data_resp_E.cfg.trl(:,2)-250,...
>> data_resp_E.cfg.trl(:,2:6));
>> cfg = [];
>> cfg.trl = For_trl;
>> data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
>>
>> data_To_base_resp_E.trl = For_trl;
>>
>> % epoch the baseline
>> For_cfg_trl =
>> horzcat(data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,1)-750,...
>>
>> data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,2)-750,...
>> data_resp_E.cfg.trl(:,3:6));
>> cfg = [];
>> cfg.trl = For_cfg_trl;
>> data_resp_E_baseline = ft_redefinetrial (cfg,data_resp_E);
>>
>> I suspect this time I'm not getting redefine trial to work correctly. I
>> think the baseline routine works perfectly though, so thank you for that!
>> If you have any further suggestions, I would appreciate it!
>>
>> Best,
>>
>> Duru
>>
>> On Fri, 20 Sep 2019 at 11:53, Stephen Whitmarsh <
>> stephen.whitmarsh at gmail.com> wrote:
>>
>>> Dear Duru,
>>>
>>> 1) Yes, cfg.trials does not work on ft_preprocessing, so you will end up
>>> concatinating ntrials x ntrials :-). I thought you would be using
>>> ft_timelockbaseline (but even there isn't a cfg.trials option. My bad. I'll
>>> make up for it below :-))
>>> 2) After appending data, the .sampleinfo field is removed, and no
>>> reference to the original data is made, as it is done by ft_databrowser to
>>> plot the events from the original data (who gets some info from the .cfg
>>> field as well, i.e. the original file). This is because typically appending
>>> data goes over several datasets, and references to 1 original file doesn't
>>> make sense anymore.
>>> 3) About baseline correcting with other data, i.e,. outside of the trial
>>> duration; It seems in general you would benefit from just making your own
>>> baseline correction, which will solve all your issues and is pretty simple.
>>>
>>> For example (there are many ways, but this would be my way) you could do
>>> the following in which you preprocess and epoch both the baseline period,
>>> and trail period (including any other 'baseline' pre-stim period) separately
>>>
>>> % preprocess data as you did before
>>> cfg = [];
>>> cfg.trl = 'your data trl'
>>> data_base_end_E = ft_preprocessing(cfg);
>>>
>>> % preprocess only your baseline periods
>>> cfg = [];
>>> cfg.trial = 'your "BaselineWindow" however you want to define it,
>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>> as your data'
>>> data_baseline = ft_preprocessing(cfg);
>>>
>>> % Calculate baseline value, i.e. the mean of the baseline period
>>> for itrial = 1 : size(data_baseline.trial,2)
>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>> end
>>>
>>> % now just divide your data with the baseline value, or subtract for
>>> absolute baseline, or make a ratio, whatever you want.
>>> data_base_end_E_baseline = data_base_end_E;
>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>> data_base_end_E_baseline.trial{itrial} =
>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>> end
>>>
>>> Doing it like this, also allows you even to extract baselines of unequal
>>> duration is you want to. I now realize though that by preprocessing your
>>> data separately, you might introduce differences in e.g. filtering between
>>> the baseline period and the trial data, especially when using short and
>>> variable trial lengths for either. So, even better would be to preprocess
>>> you data first as one trial, and then use ft_redefine trials to separate it
>>> in either baseline and baseline+data. Let's write it out:
>>>
>>> ----------
>>>
>>> % preprocess all your data as one trial, i.e. without epoching:
>>> cfg = [];
>>> cfg.dataset = 'your datafile';
>>> cfg.hpfilter = ... etc...
>>> ...
>>> all_data = ft_preprocessing(cfg);
>>>
>>> % epoch your trial data
>>> cfg = [];
>>> cfg.trl = 'your data trl'
>>> data_base_end_E = ft_redefinetrial(cfg,all_data);
>>>
>>> % preprocess only your baseline periods
>>> cfg = [];
>>> cfg.trl = your "BaselineWindow" trl, however you want to define it,
>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>> as your data
>>> data_baseline = ft_redefinetrial (cfg,all_data);
>>>
>>> % The rest is the same as above:
>>>
>>> % Calculate baseline value, i.e. the mean of the baseline period
>>> for itrial = 1 : size(data_baseline.trial,2)
>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>> end
>>>
>>> % now just divide your data with the baseline value, or subtract for
>>> absolute baseline, or make a ratio, whatever you want.
>>> data_base_end_E_baseline = data_base_end_E;
>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>> data_base_end_E_baseline.trial{itrial} =
>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>> end
>>>
>>> ----------
>>>
>>> This should take care of treating all data the same, and especially
>>> allows you e.g. to define a nice high-pass filter on all your data at once.
>>> There might be a typo in the code, but I how you get my idea.
>>>
>>> Hope this helps,
>>> Stephen
>>>
>>>
>>>
>>>
>>>
>>> Op vr 20 sep. 2019 om 08:35 schreef Duru Gun Ozkan <
>>> durugun.ozkan at uniroma1.it>:
>>>
>>>> Dear Stephen,
>>>>
>>>> Thank you for your response. It seemed like your suggestion helped, but
>>>> I thave another potential issue now.
>>>> The problem is, when I use *ft_databrowser *to visualise the trials
>>>> after baseline corection loop, I can no longer see the markers, even if I
>>>> add * cfg.plotevents = 'yes'*. I can see that this information exists
>>>> in the data structure, so I'm not sure if this is a problem per se. But the
>>>> bigger issue is, after running these loops, the data becomes very large
>>>> (even after being appended), impossible to save, and slows down the rest of
>>>> my analysis, where regular baseline correction doesn't cause this.
>>>>
>>>> Any ideas where I might be going wrong? I suspect I might have used
>>>> the wrong kinds of brackets, but the curly brackets as in your suggestion
>>>> caused an error about cell structures, and when I run it as I wrote, the
>>>> data structures actually look fine.Or it might be the way that I made a
>>>> loop to append the data, instead of spelling it out for all of the 360
>>>> trials.
>>>>
>>>> Another somewhat related question, is it possible to have the baseline
>>>> window outside of the defined trial length?
>>>>
>>>> Here is what I did:
>>>>
>>>> for trial_index = 1:360
>>>>
>>>> cfg = [];
>>>> cfg.demean = 'yes';
>>>> cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>> BaselineWindow(trial_index,2)];
>>>> cfg.trials = trial_index;
>>>> bl_end_E(trial_index) = ft_preprocessing(cfg, data_end_E);
>>>>
>>>> end
>>>>
>>>> %% This results in the struct:
>>>> bl_end_E = 1×360 struct array with 8 fields:
>>>> hdr
>>>> fsample
>>>> trialinfo
>>>> sampleinfo
>>>> trial
>>>> time
>>>> label
>>>> cfg
>>>>
>>>> %% Then I append the data:
>>>> data_base_end_E = bl_end_E(1);
>>>>
>>>> for trial_ind = 2:360
>>>>
>>>> cfg = [];
>>>> cfg.keepsampleinfo = 'yes';
>>>> data_base_end_E =
>>>> ft_appenddata(cfg,data_base_end_E,bl_end_E(trial_ind));
>>>>
>>>> end
>>>>
>>>> %% This results in losing hdr and fsample fields: data_base_end_E =
>>>> struct with fields:
>>>>
>>>> label: {29×1 cell}
>>>> trialinfo: [360×4 double]
>>>> sampleinfo: [360×2 double]
>>>> trial: {1×360 cell}
>>>> time: {1×360 cell}
>>>> cfg: [1×1 struct]
>>>> %% So I add them back:
>>>> base_end_E.hdr = data_end_E.hdr;
>>>> base_end_E.fsample = data_end_E.fsample;
>>>>
>>>> %% Visual data inspection
>>>> cfg = [];
>>>> cfg.viewmode = 'vertical';
>>>> cfg.continuous = 'no';
>>>> cfg.blocksize = 1.5;
>>>> cfg.channel = {'all'};
>>>> cfg = ft_databrowser(cfg,data_base_end_E);
>>>>
>>>>
>>>> Thanks again for the help!
>>>>
>>>> Duru
>>>>
>>>> --
>>>> *Duru G**ü**n **Ö**zkan*
>>>>
>>>>
>>>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>>>
>>>>
>>>> *Department of Psychology.University of Rome "La Sapienza".Via dei
>>>> Marsi 78 - 00185 - Roma.*
>>>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>>>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>>>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>>>
>>>> > Hi Duru,
>>>>
>>>> > Use cfg.trials = trial_index
>>>> > Put the output in a struct, i.e. bl{trial_index}
>>>> > Then combine with ft_append_data([],bl{:});
>>>>
>>>> > Hope this helps,
>>>> > Stephen
>>>>
>>>>
>>>> > On Tue, 17 Sep 2019, 19:30 Duru Gun Ozkan, <
>>>> durugun.ozkan at uniroma1.it>
>>>> > wrote:
>>>>
>>>> > Hi everyone,
>>>> >
>>>> > I have a question regarding specifying a different baseline correction
>>>> > time for each trial in ft_preprocessing.
>>>> > I had 360 audio stimuli all of which had different lengths. My
>>>> trigger is
>>>> > located at the end of each stimuli, and I would like to place the
>>>> baseline
>>>> > correction between 200 ms before stimulus start and stimulus start.
>>>> > I have a matrix for cfg.baselinewindow called BaselineWindow such as
>>>> this:
>>>> > -1.83018 -1.630182
>>>> > -1.69807 -1.498071
>>>> > -0.58653 -0.38653
>>>> > -1.07604 -0.876039
>>>> > -2.2608 -2.060803
>>>> > -0.80863 -0.608632
>>>> > -1.55663 -1.356629
>>>> > -0.94261 -0.742605
>>>> > -1.20845 -1.008448 ...
>>>> > I tried to create a loop:
>>>> >
>>>> > for trial_index = 1:360
>>>> >
>>>> > cfg = [];
>>>> > cfg.demean = 'yes';
>>>> > cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>> > BaselineWindow(trial_index,2)];
>>>> >
>>>> > data_baselined_end_E = ft_preprocessing(cfg, data_end_E);
>>>> >
>>>> > end
>>>> >
>>>> > This didn't work because it kept running the preprocessing with all
>>>> the
>>>> > baselines with all trials, instead of keeping to its specific trial.
>>>> >
>>>> > My question is, is there another way to specify individual baselines
>>>> for
>>>> > individual trials? Or can anyone suggest to improve this loop to have
>>>> the
>>>> > data preprocessed with its specific baseline windows?
>>>> >
>>>> > Thank you in advance.
>>>> >
>>>> > Best,
>>>> >
>>>> >
>>>>
>>>>
>>
>> --
>> *Duru G**ü**n **Ö**zkan*
>>
>>
>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>
>>
>> *Department of Psychology.University of Rome "La Sapienza".Via dei Marsi
>> 78 - 00185 - Roma.*
>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>
>
--
*Duru G**ü**n **Ö**zkan*
*Ph.D. student in Cognitive Social and Affective Neuroscience.*
*Department of Psychology.University of Rome "La Sapienza".Via dei Marsi 78
- 00185 - Roma.*
*e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
*https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
<https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190923/aa734fbe/attachment.html>
More information about the fieldtrip
mailing list