[FieldTrip] Single trial baseline correction in ft_preprocessing
Stephen Whitmarsh
stephen.whitmarsh at gmail.com
Mon Sep 23 15:55:36 CEST 2019
Dear Duru,
I am not sure what you want databrowser to plot. After segmenting the data
with your trl, the data consists of trials that have t = 0 at your marker.
In other words, from that point on, *other *markers don't mean anything
with regard to that data representation's timing. There is some intelligent
behaviour is databrowser, extracting markers from the original data and
data in .cfg, even after segmentation, but I don't know the details or how
far that goes. I never used ft_databrowser to plot events after
segmentation myself, so I'm not sure I can help. Perhaps someone else can
jump in here.
In any case, you should be able to debug/understand it pretty easily
looking at the code. E.g. in ft_databrowser at line 287:
---
if isfield(data, 'cfg') && ~isempty(ft_findcfg(data.cfg, 'origfs'))
% don't use the events in case the data has been resampled
ft_warning('the data has been resampled, not showing the events');
event = [];
elseif isfield(data, 'cfg') && isfield(data.cfg, 'event')
% use the event structure from the data as per bug #2501
event = data.cfg.event;
elseif ~isempty(cfg.event)
% use the events that the user passed in the configuration
event = cfg.event;
else
% fetch the events from the data structure in memory
%event = ft_fetch_event(data);
event = [];
end
---
So I would put a breakpoint there, see if it catches your events (from the
cfg I imagine, so perhaps check first whether they are in your data.cfg)
and then work back or forth from there, i.e. find where you lost them, or
why they don't show up. Sorry to not be of much more help here.
Cheers,
Stephen
Op ma 23 sep. 2019 om 15:39 schreef Duru Gun Ozkan <
durugun.ozkan at uniroma1.it>:
> Dear Stephen,
>
> Yes, I now understand that the 3rd column needs to be adjusted too, and I
> think I've done that now, but still I am not able to visualise the markers
> with *ft_databrowser*.
> I think I'm missing a very simple thing at this point, so I'm sorry to
> stretch out this thread. All of the information seems to be present in the
> redefined data, including in cfg.trl:
>
> data_To_base_resp_E =
>
> struct with fields:
>
> hdr: [1×1 struct]
> trial: {1×359 cell}
> time: {1×359 cell}
> fsample: 500
> label: {29×1 cell}
> trialinfo: [359×3 double]
> sampleinfo: [359×2 double]
> cfg: [1×1 struct]
>
> data_To_base_resp_E.cfg =
>
> struct with fields:
>
> trl: [359×6 double]
> checkpath: 'pedantic'
> outputfilepresent: 'overwrite'
> toolbox: [1×1 struct]
> callinfo: [1×1 struct]
> version: [1×1 struct]
> offset: []
> toilim: []
> begsample: []
> endsample: []
> minlength: []
> trials: 'all'
> feedback: 'yes'
> length: []
> overlap: 0
> previous: [1×1 struct]
>
> data_To_base_resp_E.cfg.trl
>
> ans = % begsample, endsample, offset, marker, trial number, accuracy code
> (last 2 are my additions to the trialinfo a few steps prior)
>
> 8849 10098 -750 221 1 11
> 10950 12199 -750 213 2 11
> 12451 13700 -750 212 3 12
> 14578 15827 -750 223 4 12
> 16719 17968 -750 213 5 11
>
> I suspected perhaps the empty fields offset: [] toilim: []
> begsample: [] endsample: [] were the issue but manipulating them didn't
> help.
> Below is my code:
> u = repelem(-500,359)';
> % epoch your trial data
> For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,... % sampling rate is 500
> hz, so begsample+750 in a -2.5 s onset gives me -1 s
> data_resp_E.cfg.trl(:,2)-250,... % it was +1.5 s before,
> now is +1 s
> u,... %offset is now 1 sec
> data_resp_E.cfg.trl(:,4:6)); % the marker informations
> cfg = [];
> cfg.trl = For_trl;
> data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
>
>
> Thank you again for the help.
>
> Best,
>
> Duru
>
> On Mon, 23 Sep 2019 at 10:51, Stephen Whitmarsh <
> stephen.whitmarsh at gmail.com> wrote:
>
>> Dear Duru,
>>
>> Great to hear you are getting along.
>>
>> 1) yes, you are right, you should only average over time, i.e the second
>> dimension (also see your .dimord field)
>>
>> 2) We should really make a .trl FAQ, you are not the only one confused!
>> The .trl goes like this: [start sample, end sample, offset]. Offset
>> determines your t=0, *not *by setting a sample at t=0 (which you
>> probably think intuitively), but by telling FieldTrip where the trial
>> starts with *respect to the begin sample*. So, in a typical scenario
>> where you have a baseline period *before *your marker, your start sample
>> sets the beginning of your period you will be extracting from the data, i.e
>> some time *before *the marker, e.g. for 500ms at 1000hz samplerate that
>> will be 500 samples before your marker. Now your offset then tells
>> FieldTrip that at that sample you are 500 samples *before *the start of
>> the trial, so your offset will be *minus *500.
>> For example: say your samplerate is 1000Hz, your data starts at sample 1,
>> but your first marker is at samplenr. 12000, i.e. after 12 seconds. Now if
>> you want a trial with a 500ms baseline before that marker, and lasting
>> until 2 seconds after that marker, your trl will be: [12000-500,
>> 12000+2000, -500];
>>
>> I hope this helps,
>> Stephen
>>
>>
>> On Mon, 23 Sep 2019, 10:14 Duru Gun Ozkan, <durugun.ozkan at uniroma1.it>
>> wrote:
>>
>>> Dear Stephen,
>>>
>>> Thank you so much for your detailed response. I think I'm mostly there,
>>> I just wanted to clarify a few points.
>>>
>>> I followed your second suggestion, in your first loop, when you get the
>>> mean of the baseline period for each trial, I also specified the dimension,
>>> because it was resulting in an array rather than a single number, which was
>>> problematic later on. I wanted to double check this was correct, and that
>>> it is supposed to be a single number, otherwise the next loop doesn't
>>> compute, because there is 2000 data points per trial in the original data
>>> (4 seconds) and 101 data points in the baseline (200 ms). (There is a
>>> chance my understanding of the math isn't sufficient for this issue, hence
>>> the doublechecking)
>>>
>>> for itrial = 1 : size(data_resp_E_baseline.trial,2)
>>> baseline{itrial} = mean(data_resp_E_baseline.trial{itrial}*,2);*
>>> end
>>>
>>> The thing is, I still seem to be losing marker information and marker
>>> time when I redefine the trial to be shorter, as it doesn't respect the
>>> time 0 anymore:
>>>
>>> % epoch trial data: Here I add 1.5 second to the first column, (start
>>> time), and subtract .5 s from the second (end), and keep the rest of the
>>> trl matrix, then use it to redefine trial to be -1 1 : 2 s long
>>> For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,...
>>> data_resp_E.cfg.trl(:,2)-250,...
>>> data_resp_E.cfg.trl(:,2:6));
>>> cfg = [];
>>> cfg.trl = For_trl;
>>> data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
>>>
>>> data_To_base_resp_E.trl = For_trl;
>>>
>>> % epoch the baseline
>>> For_cfg_trl =
>>> horzcat(data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,1)-750,...
>>>
>>> data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,2)-750,...
>>> data_resp_E.cfg.trl(:,3:6));
>>> cfg = [];
>>> cfg.trl = For_cfg_trl;
>>> data_resp_E_baseline = ft_redefinetrial (cfg,data_resp_E);
>>>
>>> I suspect this time I'm not getting redefine trial to work correctly. I
>>> think the baseline routine works perfectly though, so thank you for that!
>>> If you have any further suggestions, I would appreciate it!
>>>
>>> Best,
>>>
>>> Duru
>>>
>>> On Fri, 20 Sep 2019 at 11:53, Stephen Whitmarsh <
>>> stephen.whitmarsh at gmail.com> wrote:
>>>
>>>> Dear Duru,
>>>>
>>>> 1) Yes, cfg.trials does not work on ft_preprocessing, so you will end
>>>> up concatinating ntrials x ntrials :-). I thought you would be using
>>>> ft_timelockbaseline (but even there isn't a cfg.trials option. My bad. I'll
>>>> make up for it below :-))
>>>> 2) After appending data, the .sampleinfo field is removed, and no
>>>> reference to the original data is made, as it is done by ft_databrowser to
>>>> plot the events from the original data (who gets some info from the .cfg
>>>> field as well, i.e. the original file). This is because typically appending
>>>> data goes over several datasets, and references to 1 original file doesn't
>>>> make sense anymore.
>>>> 3) About baseline correcting with other data, i.e,. outside of the
>>>> trial duration; It seems in general you would benefit from just making your
>>>> own baseline correction, which will solve all your issues and is pretty
>>>> simple.
>>>>
>>>> For example (there are many ways, but this would be my way) you could
>>>> do the following in which you preprocess and epoch both the baseline
>>>> period, and trail period (including any other 'baseline' pre-stim period)
>>>> separately
>>>>
>>>> % preprocess data as you did before
>>>> cfg = [];
>>>> cfg.trl = 'your data trl'
>>>> data_base_end_E = ft_preprocessing(cfg);
>>>>
>>>> % preprocess only your baseline periods
>>>> cfg = [];
>>>> cfg.trial = 'your "BaselineWindow" however you want to define it,
>>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>>> as your data'
>>>> data_baseline = ft_preprocessing(cfg);
>>>>
>>>> % Calculate baseline value, i.e. the mean of the baseline period
>>>> for itrial = 1 : size(data_baseline.trial,2)
>>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>>> end
>>>>
>>>> % now just divide your data with the baseline value, or subtract for
>>>> absolute baseline, or make a ratio, whatever you want.
>>>> data_base_end_E_baseline = data_base_end_E;
>>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>>> data_base_end_E_baseline.trial{itrial} =
>>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>>> end
>>>>
>>>> Doing it like this, also allows you even to extract baselines of
>>>> unequal duration is you want to. I now realize though that by preprocessing
>>>> your data separately, you might introduce differences in e.g. filtering
>>>> between the baseline period and the trial data, especially when using short
>>>> and variable trial lengths for either. So, even better would be to
>>>> preprocess you data first as one trial, and then use ft_redefine trials to
>>>> separate it in either baseline and baseline+data. Let's write it out:
>>>>
>>>> ----------
>>>>
>>>> % preprocess all your data as one trial, i.e. without epoching:
>>>> cfg = [];
>>>> cfg.dataset = 'your datafile';
>>>> cfg.hpfilter = ... etc...
>>>> ...
>>>> all_data = ft_preprocessing(cfg);
>>>>
>>>> % epoch your trial data
>>>> cfg = [];
>>>> cfg.trl = 'your data trl'
>>>> data_base_end_E = ft_redefinetrial(cfg,all_data);
>>>>
>>>> % preprocess only your baseline periods
>>>> cfg = [];
>>>> cfg.trl = your "BaselineWindow" trl, however you want to define it,
>>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>>> as your data
>>>> data_baseline = ft_redefinetrial (cfg,all_data);
>>>>
>>>> % The rest is the same as above:
>>>>
>>>> % Calculate baseline value, i.e. the mean of the baseline period
>>>> for itrial = 1 : size(data_baseline.trial,2)
>>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>>> end
>>>>
>>>> % now just divide your data with the baseline value, or subtract for
>>>> absolute baseline, or make a ratio, whatever you want.
>>>> data_base_end_E_baseline = data_base_end_E;
>>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>>> data_base_end_E_baseline.trial{itrial} =
>>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>>> end
>>>>
>>>> ----------
>>>>
>>>> This should take care of treating all data the same, and especially
>>>> allows you e.g. to define a nice high-pass filter on all your data at once.
>>>> There might be a typo in the code, but I how you get my idea.
>>>>
>>>> Hope this helps,
>>>> Stephen
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Op vr 20 sep. 2019 om 08:35 schreef Duru Gun Ozkan <
>>>> durugun.ozkan at uniroma1.it>:
>>>>
>>>>> Dear Stephen,
>>>>>
>>>>> Thank you for your response. It seemed like your suggestion helped,
>>>>> but I thave another potential issue now.
>>>>> The problem is, when I use *ft_databrowser *to visualise the trials
>>>>> after baseline corection loop, I can no longer see the markers, even if I
>>>>> add * cfg.plotevents = 'yes'*. I can see that this information
>>>>> exists in the data structure, so I'm not sure if this is a problem per se.
>>>>> But the bigger issue is, after running these loops, the data becomes very
>>>>> large (even after being appended), impossible to save, and slows down the
>>>>> rest of my analysis, where regular baseline correction doesn't cause this.
>>>>>
>>>>> Any ideas where I might be going wrong? I suspect I might have used
>>>>> the wrong kinds of brackets, but the curly brackets as in your suggestion
>>>>> caused an error about cell structures, and when I run it as I wrote, the
>>>>> data structures actually look fine.Or it might be the way that I made a
>>>>> loop to append the data, instead of spelling it out for all of the 360
>>>>> trials.
>>>>>
>>>>> Another somewhat related question, is it possible to have the baseline
>>>>> window outside of the defined trial length?
>>>>>
>>>>> Here is what I did:
>>>>>
>>>>> for trial_index = 1:360
>>>>>
>>>>> cfg = [];
>>>>> cfg.demean = 'yes';
>>>>> cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>>> BaselineWindow(trial_index,2)];
>>>>> cfg.trials = trial_index;
>>>>> bl_end_E(trial_index) = ft_preprocessing(cfg, data_end_E);
>>>>>
>>>>> end
>>>>>
>>>>> %% This results in the struct:
>>>>> bl_end_E = 1×360 struct array with 8 fields:
>>>>> hdr
>>>>> fsample
>>>>> trialinfo
>>>>> sampleinfo
>>>>> trial
>>>>> time
>>>>> label
>>>>> cfg
>>>>>
>>>>> %% Then I append the data:
>>>>> data_base_end_E = bl_end_E(1);
>>>>>
>>>>> for trial_ind = 2:360
>>>>>
>>>>> cfg = [];
>>>>> cfg.keepsampleinfo = 'yes';
>>>>> data_base_end_E =
>>>>> ft_appenddata(cfg,data_base_end_E,bl_end_E(trial_ind));
>>>>>
>>>>> end
>>>>>
>>>>> %% This results in losing hdr and fsample fields: data_base_end_E =
>>>>> struct with fields:
>>>>>
>>>>> label: {29×1 cell}
>>>>> trialinfo: [360×4 double]
>>>>> sampleinfo: [360×2 double]
>>>>> trial: {1×360 cell}
>>>>> time: {1×360 cell}
>>>>> cfg: [1×1 struct]
>>>>> %% So I add them back:
>>>>> base_end_E.hdr = data_end_E.hdr;
>>>>> base_end_E.fsample = data_end_E.fsample;
>>>>>
>>>>> %% Visual data inspection
>>>>> cfg = [];
>>>>> cfg.viewmode = 'vertical';
>>>>> cfg.continuous = 'no';
>>>>> cfg.blocksize = 1.5;
>>>>> cfg.channel = {'all'};
>>>>> cfg = ft_databrowser(cfg,data_base_end_E);
>>>>>
>>>>>
>>>>> Thanks again for the help!
>>>>>
>>>>> Duru
>>>>>
>>>>> --
>>>>> *Duru G**ü**n **Ö**zkan*
>>>>>
>>>>>
>>>>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>>>>
>>>>>
>>>>> *Department of Psychology.University of Rome "La Sapienza".Via dei
>>>>> Marsi 78 - 00185 - Roma.*
>>>>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>>>>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>>>>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>>>>
>>>>> > Hi Duru,
>>>>>
>>>>> > Use cfg.trials = trial_index
>>>>> > Put the output in a struct, i.e. bl{trial_index}
>>>>> > Then combine with ft_append_data([],bl{:});
>>>>>
>>>>> > Hope this helps,
>>>>> > Stephen
>>>>>
>>>>>
>>>>> > On Tue, 17 Sep 2019, 19:30 Duru Gun Ozkan, <
>>>>> durugun.ozkan at uniroma1.it>
>>>>> > wrote:
>>>>>
>>>>> > Hi everyone,
>>>>> >
>>>>> > I have a question regarding specifying a different baseline
>>>>> correction
>>>>> > time for each trial in ft_preprocessing.
>>>>> > I had 360 audio stimuli all of which had different lengths. My
>>>>> trigger is
>>>>> > located at the end of each stimuli, and I would like to place the
>>>>> baseline
>>>>> > correction between 200 ms before stimulus start and stimulus start.
>>>>> > I have a matrix for cfg.baselinewindow called BaselineWindow such as
>>>>> this:
>>>>> > -1.83018 -1.630182
>>>>> > -1.69807 -1.498071
>>>>> > -0.58653 -0.38653
>>>>> > -1.07604 -0.876039
>>>>> > -2.2608 -2.060803
>>>>> > -0.80863 -0.608632
>>>>> > -1.55663 -1.356629
>>>>> > -0.94261 -0.742605
>>>>> > -1.20845 -1.008448 ...
>>>>> > I tried to create a loop:
>>>>> >
>>>>> > for trial_index = 1:360
>>>>> >
>>>>> > cfg = [];
>>>>> > cfg.demean = 'yes';
>>>>> > cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>>> > BaselineWindow(trial_index,2)];
>>>>> >
>>>>> > data_baselined_end_E = ft_preprocessing(cfg, data_end_E);
>>>>> >
>>>>> > end
>>>>> >
>>>>> > This didn't work because it kept running the preprocessing with all
>>>>> the
>>>>> > baselines with all trials, instead of keeping to its specific trial.
>>>>> >
>>>>> > My question is, is there another way to specify individual baselines
>>>>> for
>>>>> > individual trials? Or can anyone suggest to improve this loop to
>>>>> have the
>>>>> > data preprocessed with its specific baseline windows?
>>>>> >
>>>>> > Thank you in advance.
>>>>> >
>>>>> > Best,
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>
>>> --
>>> *Duru G**ü**n **Ö**zkan*
>>>
>>>
>>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>>
>>>
>>> *Department of Psychology.University of Rome "La Sapienza".Via dei Marsi
>>> 78 - 00185 - Roma.*
>>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>>
>>
>
> --
> *Duru G**ü**n **Ö**zkan*
>
>
> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>
>
> *Department of Psychology.University of Rome "La Sapienza".Via dei Marsi
> 78 - 00185 - Roma.*
> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190923/3971bba6/attachment.html>
More information about the fieldtrip
mailing list