[FieldTrip] Single trial baseline correction in ft_preprocessing
Duru Gun Ozkan
durugun.ozkan at uniroma1.it
Mon Sep 23 15:57:26 CEST 2019
Ah I see, I was expecting for the markers to be present still but there is
no problems after all.
Thank you so much!
Best,
Duru
On Mon, 23 Sep 2019 at 15:55, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
wrote:
> Dear Duru,
>
> I am not sure what you want databrowser to plot. After segmenting the data
> with your trl, the data consists of trials that have t = 0 at your marker.
> In other words, from that point on, *other *markers don't mean anything
> with regard to that data representation's timing. There is some intelligent
> behaviour is databrowser, extracting markers from the original data and
> data in .cfg, even after segmentation, but I don't know the details or how
> far that goes. I never used ft_databrowser to plot events after
> segmentation myself, so I'm not sure I can help. Perhaps someone else can
> jump in here.
>
> In any case, you should be able to debug/understand it pretty easily
> looking at the code. E.g. in ft_databrowser at line 287:
>
> ---
>
> if isfield(data, 'cfg') && ~isempty(ft_findcfg(data.cfg, 'origfs'))
> % don't use the events in case the data has been resampled
> ft_warning('the data has been resampled, not showing the events');
> event = [];
> elseif isfield(data, 'cfg') && isfield(data.cfg, 'event')
> % use the event structure from the data as per bug #2501
> event = data.cfg.event;
> elseif ~isempty(cfg.event)
> % use the events that the user passed in the configuration
> event = cfg.event;
> else
> % fetch the events from the data structure in memory
> %event = ft_fetch_event(data);
> event = [];
> end
>
> ---
>
> So I would put a breakpoint there, see if it catches your events (from the
> cfg I imagine, so perhaps check first whether they are in your data.cfg)
> and then work back or forth from there, i.e. find where you lost them, or
> why they don't show up. Sorry to not be of much more help here.
>
> Cheers,
> Stephen
>
> Op ma 23 sep. 2019 om 15:39 schreef Duru Gun Ozkan <
> durugun.ozkan at uniroma1.it>:
>
>> Dear Stephen,
>>
>> Yes, I now understand that the 3rd column needs to be adjusted too, and I
>> think I've done that now, but still I am not able to visualise the markers
>> with *ft_databrowser*.
>> I think I'm missing a very simple thing at this point, so I'm sorry to
>> stretch out this thread. All of the information seems to be present in the
>> redefined data, including in cfg.trl:
>>
>> data_To_base_resp_E =
>>
>> struct with fields:
>>
>> hdr: [1×1 struct]
>> trial: {1×359 cell}
>> time: {1×359 cell}
>> fsample: 500
>> label: {29×1 cell}
>> trialinfo: [359×3 double]
>> sampleinfo: [359×2 double]
>> cfg: [1×1 struct]
>>
>> data_To_base_resp_E.cfg =
>>
>> struct with fields:
>>
>> trl: [359×6 double]
>> checkpath: 'pedantic'
>> outputfilepresent: 'overwrite'
>> toolbox: [1×1 struct]
>> callinfo: [1×1 struct]
>> version: [1×1 struct]
>> offset: []
>> toilim: []
>> begsample: []
>> endsample: []
>> minlength: []
>> trials: 'all'
>> feedback: 'yes'
>> length: []
>> overlap: 0
>> previous: [1×1 struct]
>>
>> data_To_base_resp_E.cfg.trl
>>
>> ans = % begsample, endsample, offset, marker, trial number, accuracy code
>> (last 2 are my additions to the trialinfo a few steps prior)
>>
>> 8849 10098 -750 221 1 11
>> 10950 12199 -750 213 2 11
>> 12451 13700 -750 212 3 12
>> 14578 15827 -750 223 4 12
>> 16719 17968 -750 213 5 11
>>
>> I suspected perhaps the empty fields offset: [] toilim: []
>> begsample: [] endsample: [] were the issue but manipulating them didn't
>> help.
>> Below is my code:
>> u = repelem(-500,359)';
>> % epoch your trial data
>> For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,... % sampling rate is 500
>> hz, so begsample+750 in a -2.5 s onset gives me -1 s
>> data_resp_E.cfg.trl(:,2)-250,... % it was +1.5 s
>> before, now is +1 s
>> u,... %offset is now 1 sec
>> data_resp_E.cfg.trl(:,4:6)); % the marker informations
>> cfg = [];
>> cfg.trl = For_trl;
>> data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
>>
>>
>> Thank you again for the help.
>>
>> Best,
>>
>> Duru
>>
>> On Mon, 23 Sep 2019 at 10:51, Stephen Whitmarsh <
>> stephen.whitmarsh at gmail.com> wrote:
>>
>>> Dear Duru,
>>>
>>> Great to hear you are getting along.
>>>
>>> 1) yes, you are right, you should only average over time, i.e the second
>>> dimension (also see your .dimord field)
>>>
>>> 2) We should really make a .trl FAQ, you are not the only one confused!
>>> The .trl goes like this: [start sample, end sample, offset]. Offset
>>> determines your t=0, *not *by setting a sample at t=0 (which you
>>> probably think intuitively), but by telling FieldTrip where the trial
>>> starts with *respect to the begin sample*. So, in a typical scenario
>>> where you have a baseline period *before *your marker, your start
>>> sample sets the beginning of your period you will be extracting from the
>>> data, i.e some time *before *the marker, e.g. for 500ms at 1000hz
>>> samplerate that will be 500 samples before your marker. Now your offset
>>> then tells FieldTrip that at that sample you are 500 samples *before *the
>>> start of the trial, so your offset will be *minus *500.
>>> For example: say your samplerate is 1000Hz, your data starts at sample
>>> 1, but your first marker is at samplenr. 12000, i.e. after 12 seconds. Now
>>> if you want a trial with a 500ms baseline before that marker, and lasting
>>> until 2 seconds after that marker, your trl will be: [12000-500,
>>> 12000+2000, -500];
>>>
>>> I hope this helps,
>>> Stephen
>>>
>>>
>>> On Mon, 23 Sep 2019, 10:14 Duru Gun Ozkan, <durugun.ozkan at uniroma1.it>
>>> wrote:
>>>
>>>> Dear Stephen,
>>>>
>>>> Thank you so much for your detailed response. I think I'm mostly there,
>>>> I just wanted to clarify a few points.
>>>>
>>>> I followed your second suggestion, in your first loop, when you get the
>>>> mean of the baseline period for each trial, I also specified the dimension,
>>>> because it was resulting in an array rather than a single number, which was
>>>> problematic later on. I wanted to double check this was correct, and that
>>>> it is supposed to be a single number, otherwise the next loop doesn't
>>>> compute, because there is 2000 data points per trial in the original data
>>>> (4 seconds) and 101 data points in the baseline (200 ms). (There is a
>>>> chance my understanding of the math isn't sufficient for this issue, hence
>>>> the doublechecking)
>>>>
>>>> for itrial = 1 : size(data_resp_E_baseline.trial,2)
>>>> baseline{itrial} = mean(data_resp_E_baseline.trial{itrial}*,2);*
>>>> end
>>>>
>>>> The thing is, I still seem to be losing marker information and marker
>>>> time when I redefine the trial to be shorter, as it doesn't respect the
>>>> time 0 anymore:
>>>>
>>>> % epoch trial data: Here I add 1.5 second to the first column, (start
>>>> time), and subtract .5 s from the second (end), and keep the rest of the
>>>> trl matrix, then use it to redefine trial to be -1 1 : 2 s long
>>>> For_trl = horzcat(data_resp_E.cfg.trl(:,1)+750,...
>>>> data_resp_E.cfg.trl(:,2)-250,...
>>>> data_resp_E.cfg.trl(:,2:6));
>>>> cfg = [];
>>>> cfg.trl = For_trl;
>>>> data_To_base_resp_E = ft_redefinetrial(cfg,data_resp_E);
>>>>
>>>> data_To_base_resp_E.trl = For_trl;
>>>>
>>>> % epoch the baseline
>>>> For_cfg_trl =
>>>> horzcat(data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,1)-750,...
>>>>
>>>> data_resp_E.cfg.trl(:,2)+BaseRespWindow3(:,2)-750,...
>>>> data_resp_E.cfg.trl(:,3:6));
>>>> cfg = [];
>>>> cfg.trl = For_cfg_trl;
>>>> data_resp_E_baseline = ft_redefinetrial (cfg,data_resp_E);
>>>>
>>>> I suspect this time I'm not getting redefine trial to work correctly. I
>>>> think the baseline routine works perfectly though, so thank you for that!
>>>> If you have any further suggestions, I would appreciate it!
>>>>
>>>> Best,
>>>>
>>>> Duru
>>>>
>>>> On Fri, 20 Sep 2019 at 11:53, Stephen Whitmarsh <
>>>> stephen.whitmarsh at gmail.com> wrote:
>>>>
>>>>> Dear Duru,
>>>>>
>>>>> 1) Yes, cfg.trials does not work on ft_preprocessing, so you will end
>>>>> up concatinating ntrials x ntrials :-). I thought you would be using
>>>>> ft_timelockbaseline (but even there isn't a cfg.trials option. My bad. I'll
>>>>> make up for it below :-))
>>>>> 2) After appending data, the .sampleinfo field is removed, and no
>>>>> reference to the original data is made, as it is done by ft_databrowser to
>>>>> plot the events from the original data (who gets some info from the .cfg
>>>>> field as well, i.e. the original file). This is because typically appending
>>>>> data goes over several datasets, and references to 1 original file doesn't
>>>>> make sense anymore.
>>>>> 3) About baseline correcting with other data, i.e,. outside of the
>>>>> trial duration; It seems in general you would benefit from just making your
>>>>> own baseline correction, which will solve all your issues and is pretty
>>>>> simple.
>>>>>
>>>>> For example (there are many ways, but this would be my way) you could
>>>>> do the following in which you preprocess and epoch both the baseline
>>>>> period, and trail period (including any other 'baseline' pre-stim period)
>>>>> separately
>>>>>
>>>>> % preprocess data as you did before
>>>>> cfg = [];
>>>>> cfg.trl = 'your data trl'
>>>>> data_base_end_E = ft_preprocessing(cfg);
>>>>>
>>>>> % preprocess only your baseline periods
>>>>> cfg = [];
>>>>> cfg.trial = 'your "BaselineWindow" however you want to define it,
>>>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>>>> as your data'
>>>>> data_baseline = ft_preprocessing(cfg);
>>>>>
>>>>> % Calculate baseline value, i.e. the mean of the baseline period
>>>>> for itrial = 1 : size(data_baseline.trial,2)
>>>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>>>> end
>>>>>
>>>>> % now just divide your data with the baseline value, or subtract for
>>>>> absolute baseline, or make a ratio, whatever you want.
>>>>> data_base_end_E_baseline = data_base_end_E;
>>>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>>>> data_base_end_E_baseline.trial{itrial} =
>>>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>>>> end
>>>>>
>>>>> Doing it like this, also allows you even to extract baselines of
>>>>> unequal duration is you want to. I now realize though that by preprocessing
>>>>> your data separately, you might introduce differences in e.g. filtering
>>>>> between the baseline period and the trial data, especially when using short
>>>>> and variable trial lengths for either. So, even better would be to
>>>>> preprocess you data first as one trial, and then use ft_redefine trials to
>>>>> separate it in either baseline and baseline+data. Let's write it out:
>>>>>
>>>>> ----------
>>>>>
>>>>> % preprocess all your data as one trial, i.e. without epoching:
>>>>> cfg = [];
>>>>> cfg.dataset = 'your datafile';
>>>>> cfg.hpfilter = ... etc...
>>>>> ...
>>>>> all_data = ft_preprocessing(cfg);
>>>>>
>>>>> % epoch your trial data
>>>>> cfg = [];
>>>>> cfg.trl = 'your data trl'
>>>>> data_base_end_E = ft_redefinetrial(cfg,all_data);
>>>>>
>>>>> % preprocess only your baseline periods
>>>>> cfg = [];
>>>>> cfg.trl = your "BaselineWindow" trl, however you want to define it,
>>>>> including an arbitrary offset, say 0; Of course the same amount of trials
>>>>> as your data
>>>>> data_baseline = ft_redefinetrial (cfg,all_data);
>>>>>
>>>>> % The rest is the same as above:
>>>>>
>>>>> % Calculate baseline value, i.e. the mean of the baseline period
>>>>> for itrial = 1 : size(data_baseline.trial,2)
>>>>> baseline{itrial} = mean(data_baseline.trial{itrial})
>>>>> end
>>>>>
>>>>> % now just divide your data with the baseline value, or subtract for
>>>>> absolute baseline, or make a ratio, whatever you want.
>>>>> data_base_end_E_baseline = data_base_end_E;
>>>>> for itrial = 1 : size( data_base_end_E.trial,2)
>>>>> data_base_end_E_baseline.trial{itrial} =
>>>>> data_base_end_E.trial{itrial} / baseline{itrial};
>>>>> end
>>>>>
>>>>> ----------
>>>>>
>>>>> This should take care of treating all data the same, and especially
>>>>> allows you e.g. to define a nice high-pass filter on all your data at once.
>>>>> There might be a typo in the code, but I how you get my idea.
>>>>>
>>>>> Hope this helps,
>>>>> Stephen
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Op vr 20 sep. 2019 om 08:35 schreef Duru Gun Ozkan <
>>>>> durugun.ozkan at uniroma1.it>:
>>>>>
>>>>>> Dear Stephen,
>>>>>>
>>>>>> Thank you for your response. It seemed like your suggestion helped,
>>>>>> but I thave another potential issue now.
>>>>>> The problem is, when I use *ft_databrowser *to visualise the trials
>>>>>> after baseline corection loop, I can no longer see the markers, even if I
>>>>>> add * cfg.plotevents = 'yes'*. I can see that this information
>>>>>> exists in the data structure, so I'm not sure if this is a problem per se.
>>>>>> But the bigger issue is, after running these loops, the data becomes very
>>>>>> large (even after being appended), impossible to save, and slows down the
>>>>>> rest of my analysis, where regular baseline correction doesn't cause this.
>>>>>>
>>>>>> Any ideas where I might be going wrong? I suspect I might have used
>>>>>> the wrong kinds of brackets, but the curly brackets as in your suggestion
>>>>>> caused an error about cell structures, and when I run it as I wrote, the
>>>>>> data structures actually look fine.Or it might be the way that I made a
>>>>>> loop to append the data, instead of spelling it out for all of the 360
>>>>>> trials.
>>>>>>
>>>>>> Another somewhat related question, is it possible to have the
>>>>>> baseline window outside of the defined trial length?
>>>>>>
>>>>>> Here is what I did:
>>>>>>
>>>>>> for trial_index = 1:360
>>>>>>
>>>>>> cfg = [];
>>>>>> cfg.demean = 'yes';
>>>>>> cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>>>> BaselineWindow(trial_index,2)];
>>>>>> cfg.trials = trial_index;
>>>>>> bl_end_E(trial_index) = ft_preprocessing(cfg, data_end_E);
>>>>>>
>>>>>> end
>>>>>>
>>>>>> %% This results in the struct:
>>>>>> bl_end_E = 1×360 struct array with 8 fields:
>>>>>> hdr
>>>>>> fsample
>>>>>> trialinfo
>>>>>> sampleinfo
>>>>>> trial
>>>>>> time
>>>>>> label
>>>>>> cfg
>>>>>>
>>>>>> %% Then I append the data:
>>>>>> data_base_end_E = bl_end_E(1);
>>>>>>
>>>>>> for trial_ind = 2:360
>>>>>>
>>>>>> cfg = [];
>>>>>> cfg.keepsampleinfo = 'yes';
>>>>>> data_base_end_E =
>>>>>> ft_appenddata(cfg,data_base_end_E,bl_end_E(trial_ind));
>>>>>>
>>>>>> end
>>>>>>
>>>>>> %% This results in losing hdr and fsample fields: data_base_end_E =
>>>>>> struct with fields:
>>>>>>
>>>>>> label: {29×1 cell}
>>>>>> trialinfo: [360×4 double]
>>>>>> sampleinfo: [360×2 double]
>>>>>> trial: {1×360 cell}
>>>>>> time: {1×360 cell}
>>>>>> cfg: [1×1 struct]
>>>>>> %% So I add them back:
>>>>>> base_end_E.hdr = data_end_E.hdr;
>>>>>> base_end_E.fsample = data_end_E.fsample;
>>>>>>
>>>>>> %% Visual data inspection
>>>>>> cfg = [];
>>>>>> cfg.viewmode = 'vertical';
>>>>>> cfg.continuous = 'no';
>>>>>> cfg.blocksize = 1.5;
>>>>>> cfg.channel = {'all'};
>>>>>> cfg = ft_databrowser(cfg,data_base_end_E);
>>>>>>
>>>>>>
>>>>>> Thanks again for the help!
>>>>>>
>>>>>> Duru
>>>>>>
>>>>>> --
>>>>>> *Duru G**ü**n **Ö**zkan*
>>>>>>
>>>>>>
>>>>>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>>>>>
>>>>>>
>>>>>> *Department of Psychology.University of Rome "La Sapienza".Via dei
>>>>>> Marsi 78 - 00185 - Roma.*
>>>>>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>>>>>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>>>>>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>>>>>
>>>>>> > Hi Duru,
>>>>>>
>>>>>> > Use cfg.trials = trial_index
>>>>>> > Put the output in a struct, i.e. bl{trial_index}
>>>>>> > Then combine with ft_append_data([],bl{:});
>>>>>>
>>>>>> > Hope this helps,
>>>>>> > Stephen
>>>>>>
>>>>>>
>>>>>> > On Tue, 17 Sep 2019, 19:30 Duru Gun Ozkan, <
>>>>>> durugun.ozkan at uniroma1.it>
>>>>>> > wrote:
>>>>>>
>>>>>> > Hi everyone,
>>>>>> >
>>>>>> > I have a question regarding specifying a different baseline
>>>>>> correction
>>>>>> > time for each trial in ft_preprocessing.
>>>>>> > I had 360 audio stimuli all of which had different lengths. My
>>>>>> trigger is
>>>>>> > located at the end of each stimuli, and I would like to place the
>>>>>> baseline
>>>>>> > correction between 200 ms before stimulus start and stimulus start.
>>>>>> > I have a matrix for cfg.baselinewindow called BaselineWindow such
>>>>>> as this:
>>>>>> > -1.83018 -1.630182
>>>>>> > -1.69807 -1.498071
>>>>>> > -0.58653 -0.38653
>>>>>> > -1.07604 -0.876039
>>>>>> > -2.2608 -2.060803
>>>>>> > -0.80863 -0.608632
>>>>>> > -1.55663 -1.356629
>>>>>> > -0.94261 -0.742605
>>>>>> > -1.20845 -1.008448 ...
>>>>>> > I tried to create a loop:
>>>>>> >
>>>>>> > for trial_index = 1:360
>>>>>> >
>>>>>> > cfg = [];
>>>>>> > cfg.demean = 'yes';
>>>>>> > cfg.baselinewindow = [BaselineWindow(trial_index,1)
>>>>>> > BaselineWindow(trial_index,2)];
>>>>>> >
>>>>>> > data_baselined_end_E = ft_preprocessing(cfg, data_end_E);
>>>>>> >
>>>>>> > end
>>>>>> >
>>>>>> > This didn't work because it kept running the preprocessing with all
>>>>>> the
>>>>>> > baselines with all trials, instead of keeping to its specific trial.
>>>>>> >
>>>>>> > My question is, is there another way to specify individual
>>>>>> baselines for
>>>>>> > individual trials? Or can anyone suggest to improve this loop to
>>>>>> have the
>>>>>> > data preprocessed with its specific baseline windows?
>>>>>> >
>>>>>> > Thank you in advance.
>>>>>> >
>>>>>> > Best,
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> *Duru G**ü**n **Ö**zkan*
>>>>
>>>>
>>>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>>>
>>>>
>>>> *Department of Psychology.University of Rome "La Sapienza".Via dei
>>>> Marsi 78 - 00185 - Roma.*
>>>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>>>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>>>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>>>
>>>
>>
>> --
>> *Duru G**ü**n **Ö**zkan*
>>
>>
>> *Ph.D. student in Cognitive Social and Affective Neuroscience.*
>>
>>
>> *Department of Psychology.University of Rome "La Sapienza".Via dei Marsi
>> 78 - 00185 - Roma.*
>> *e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
>> *https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
>> <https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
>>
>
--
*Duru G**ü**n **Ö**zkan*
*Ph.D. student in Cognitive Social and Affective Neuroscience.*
*Department of Psychology.University of Rome "La Sapienza".Via dei Marsi 78
- 00185 - Roma.*
*e-mail: durugun.ozkan at uniroma1.it <vanessa.era at uniroma1.it>*
*https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor
<https://agliotilab.org/lab-staff/phd-students/2nd-year#anchor>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20190923/c929a703/attachment.html>
More information about the fieldtrip
mailing list