[FieldTrip] Different cfg.poststim values in ft_preprocessing alters signal
Stephen Whitmarsh
stephen.whitmarsh at gmail.com
Thu Feb 4 14:00:23 CET 2021
Hi Benjy,
Good to see you are hitting something when digging deeper.
So it seems then that the problem occurs on those channels that do not have
signal. Perhaps there is some numerical issue going on, about which Jan
Mathijs might have something more specific to say.
However, while I don't want to demean the issue (pun intended), it seems
your first problem would be the lack of signal on those channels, or
perhaps even on others?
Do you have any indication, e.g. after some quick timelocking of your
trials, that your data generally behaves as you would expect? Do you have
colleagues that have preprocessed data similarly as you that might be able
to take a look?
I am a bit worried that something more upstream might be the issue here as
those are a lot/very noisy channels for Neuromag data.
Cheers,
Stephen
Op do 4 feb. 2021 om 13:11 schreef Barnett, Benjy <
benjy.barnett.20 at ucl.ac.uk>:
> Hey Jan-Mathijs, Stephen,
>
> Thanks for your help!
>
> Unfortunately that did not solve the issue. After some more digging, it
> looks like the problem is indeed caused by downsampling or demeaning. When
> I don’t use any downsampling, I have signal in different channels with
> different orders of magnitude (see attached visual reject output). The
> channels with the huge black rectangles as signal are the very same that
> show this huge spike at the trial start after I downsample. However, when I
> don’t use any demeaning, the signals just look like flat lines. Please see
> a subplot of 9 random GRAD channels too (created without demeaning).
>
> The issue must lie somewhere in this I imagine, although I am a bit
> clueless as to what it could be. Thanks for your time!
>
> Cheers,
> Benjy
>
> On Feb 4, 2021, at 10:47 AM, Schoffelen, J.M. (Jan Mathijs) <
> jan.schoffelen at donders.ru.nl> wrote:
>
> Hi Benjy,
>
> You are probably looking at a downsampling artifact. I don’t understand
> why this happens, because the FT-code should be dealing with this ‘under
> the hood’.
>
> Could you - after the call to ft_redefinetrial - explicitly demean the
> epochs, prior to resampling, and see whether your problem disappears? i.e.
> you should do something like: cfg = [];
> cfg.demean = ‘yes’; data = ft_preprocessing(cfg, data); (as mentioned on
> the data structure after re-epoching, but before resampling)
>
>
> I hope this helps.
>
> Best wishes,
> Jan-Mathijs
>
>
>
> On 4 Feb 2021, at 10:57, Stephen Whitmarsh <stephen.whitmarsh at gmail.com>
> wrote:
>
> Hi Benjy,
>
> Strange indeed. Does this still occur if you don't resample at all?
>
> If so, I don't know what else it could be.
>
> Unless someone else has an idea, perhaps you can create a small dataset
> and script with which we can reproduce the problem (on the latest FieldTrip
> version), then upload it to dropbox/google drive or similar,
>
> Cheers,
> Stephen
>
>
>
>
>
>
>
>
> Op do 4 feb. 2021 om 10:42 schreef Barnett, Benjy <
> benjy.barnett.20 at ucl.ac.uk>:
>
>> Hey Stephen,
>>
>> I believe I am plotting just single channels here, averaging over all the
>> trials. After I call ft_timelockanalysis my dimord field is ‘rpt_chan_time’
>> and the dimensions of my trial field as 228 x 204 x 169, which makes sense
>> as I am looking only at the gradiometers, of which there are 204 in the
>> neuromas set up.
>>
>> Then when I plot the signal I am using this command:
>>
>> plot(data_TL.time,squeeze(mean(data_TL.trial(:,39,:),1)))
>>
>> Looking at, in this example, channel 39 on its own. As there are just 204
>> channels in the dimord field I don’t believe there are any reference
>> channels added, and this jump occurs on ~20 different channels, and these
>> vary across subjects. Like I say, I don’t think the issue is with my data
>> since the plot of all the data in one long trial looks quite normal.
>> Indeed, when I call ft_rejectvisual, you get an idea of the scope of the
>> problem as multiple different trials have these large spikes (image
>> attached).
>>
>> All the best,
>> Benjy
>>
>> <sub02.png>
>>
>> On Feb 3, 2021, at 6:05 PM, Stephen Whitmarsh <
>> stephen.whitmarsh at gmail.com> wrote:
>>
>> Hi Benjy,
>>
>> Just a though - but am I right that you are plotting an average over
>> channels? Can you look at the channels individually and see on which
>> channels the jumps occur? Might you accidentally be adding a
>> reference/external/trigger channel to the average?
>>
>> Cheers,
>> Stephen
>>
>> Op wo 3 feb. 2021 om 18:42 schreef Barnett, Benjy <
>> benjy.barnett.20 at ucl.ac.uk>:
>>
>>> Hey Stephen,
>>>
>>> Thanks for your help! Whilst this certainly answers my question about
>>> the slight difference between different trial lengths, the huge spikes at
>>> the beginning of trials remains regardless of whether I filter the data as
>>> a whole first, or if I segment into trials and then filter. Here’s my code
>>> for doing the filtering first (my code for the alternative way is in my
>>> initial post):
>>>
>>> cfg = [];
>>> cfg.dataset = full_path;
>>> cfg.continuous = 'yes';
>>> %cfg.demean = 'yes';
>>> %cfg.baselinewindow = baselinewindow;
>>> cfg.lpfilter = 'no';
>>> cfg.hpfilter = 'no';
>>> cfg.dftfilter = 'yes';
>>>
>>> data = ft_preprocessing(cfg);
>>>
>>> cfg=[];
>>> cfg.dataset=full_path;
>>> cfg.trialdef.prestim=0.175;
>>> cfg.trialdef.poststim=0.600;
>>> cfg.trialdef.eventvalue=[6 7];
>>> cfg.trialdef.eventtype='STI101';
>>> cfg = ft_definetrial(cfg);
>>> trl = cfg.trl(:,1:3);
>>>
>>> cfg = [];
>>> cfg.trl = trl;
>>> data_segmented = ft_redefinetrial(cfg,data);
>>>
>>> cfg = [];
>>> cfg.resamplefs = 250;
>>> data_ds = ft_resampledata(cfg, data_segmented);
>>>
>>> cfg = [];
>>> cfg.channel = 'MEGGRAD';
>>> cfg.keeptrials = 'yes';
>>> data_TL = ft_timelockanalysis(cfg,data_ds);
>>>
>>> plot(data_TL.time,squeeze(mean(data_TL.trial(:,39,:),1))) %17, 23 and
>>> 39 are examples of spikes in channels
>>> <PastedGraphic-1.png><PastedGraphic-2.png>
>>>
>>>
>>> I’m still seeing spikes as in the images attached. Something else
>>> strange: when I change the poststim parameter to 0.075 instead of 0.175
>>> (keeping everything else equal), the direction of the spike completely
>>> switches (see third image attached). This happens regardless or
>>> ft_definetrial/preprocessing order. Moreover, when I look at the signal
>>> across the single trial encapsulating all the data (i.e. before
>>> segmentation), there are none of these huge spikes where every event would
>>> be, only a few spikes that I assume are SQUID jumps.
>>> <Screenshot 2021-02-03 at 11.29.20 AM.png>
>>> <long_trial_single_channel.png>
>>> This makes me think the issue is coming at the ft_definetrial() step.
>>> But since this does not read in the actual MEG data, I can’t work out what
>>> is causing this effect, it certainly doesn’t seem to be an issue with my
>>> data. I’ve added a couple of lines to the ft_trialfun_general function, so
>>> to be sure I’ve removed them and am just using the default version yet the
>>> problem is persisting.
>>>
>>> Any assistance is of course greatly appreciated. Thanks!
>>>
>>> Benjy
>>>
>>> On Feb 2, 2021, at 4:14 PM, Stephen Whitmarsh <
>>> stephen.whitmarsh at gmail.com> wrote:
>>>
>>> Hey Benjy,
>>>
>>> I would suspect this has to do with your (dft) filtering, which depends
>>> on the amount of data, and perhaps some small numerical differences due to
>>> resampling. Strange jumps at onset/offset of data segments also often
>>> indicate something to do with filtering.
>>>
>>> To avoid this, and to create a fair comparison between your different
>>> trial duration segmentation, I would do the following:
>>>
>>> - preprocess all your data as one long 'trial'. I.e. without
>>> segmentation. Do all your filtering at this step, i.e. at one go over all
>>> your data. This gives you the maximum frequency resolution & ability to
>>> filter low frequencies.
>>> - segment your filtered data into trials using ft_redefinetrial.
>>> - do your resampling
>>> - check the different triallengths.
>>>
>>> If anything stays slightly different, I would imagine it has to do with
>>> the resampling, because samples will end up slightly different on your
>>> axis, creating numerical differences. You could resample before segmenting,
>>> which would be more efficient, but then you'll have to deal with changing
>>> timing of your markers which would still be in the original samplerate.
>>> Dealing with time-axis and samples is a pain so personally I therefore
>>> always let FT deal with the resampling after trial segmentation.
>>>
>>> Have fun,
>>> Stephen
>>>
>>> Op di 2 feb. 2021 om 16:36 schreef Barnett, Benjy <
>>> benjy.barnett.20 at ucl.ac.uk>:
>>>
>>>> Hey. When I am preprocessing my MEG neuromag data, I’m noticing that
>>>> the signal of my channels are altered depending on how long after the
>>>> stimulus I include in the trial, even in the portion of the trial that is
>>>> overlapping between the two cases. For instance, if I use cfg.poststim =
>>>> 2.5 in one run and cfg.poststim = 0.6 in a second, the two signals are not
>>>> identical up until 0.6 seconds. Please see my code and attached plots of a
>>>> Grad sensor in these two cases up to 0.6 seconds.
>>>>
>>>> % define trials
>>>> cfg = [];
>>>> cfg.dataset = full_path;
>>>> cfg.trialdef.prestim = 0.175;
>>>> cfg.trialdef.poststim = 0.6; % compare with 2.5 seconds
>>>> cfg.trialdef.eventvalue = [6 7 9];
>>>>
>>>> cfg.trialdef.eventtype = 'STI101';
>>>> cfg = ft_definetrial(cfg);
>>>>
>>>> % preprocess
>>>>
>>>> cfg.demean = 'yes';
>>>> cfg.baselinewindow = [-0.175 0.025];
>>>> cfg.lpfilter = 'no';
>>>> cfg.hpfilter = 'no';
>>>> cfg.dftfilter = 'yes';
>>>> split_files{1} = ft_preprocessing(cfg); %preprocess
>>>>
>>>>
>>>> %%%%%%%%%%%Downsampling%%%%%%%%%
>>>> cfg = [];
>>>> cfg.resamplefs = 250;
>>>> data = ft_resampledata(cfg, split_files{1});
>>>>
>>>>
>>>> %%Visualisation
>>>> cfg = [];
>>>> cfg.channel = 'MEGGRAD';
>>>> cfg.keeptrials = 'yes';
>>>> data = ft_timelockanalysis(cfg,data);
>>>> Xlim([-0.2 0.6])
>>>> plot(data.time,squeeze(mean(data.trial(:,39,:),1)))
>>>>
>>>> As you can see from the plots attached, these signals are not identical
>>>> when they should be (shouldn’t they?). The first plot is when I Use
>>>> 2.5 seconds post stim and the second is using 0.6. I’ve altered the xlim to
>>>> show only up to 600ms on both. Also, can anyone explain these huge spikes
>>>> at the beginning of trials, that’s my next problem to solve.
>>>>
>>>> Thanks
>>>> <2_5_ch39_full.png><600_ch39_full.png>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761390305%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3vNV1onz5YIaWv9EINg%2FfOWD3pkypJg0HyXE3SpR%2Bwk%3D&reserved=0>
>>>> https://doi.org/10.1371/journal.pcbi.1002202
>>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761400260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=65oDGrQ1pjqdcjVaQ5SQZ%2BYX1KZprK9CLDiQacN9JAI%3D&reserved=0>
>>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761400260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iTCmEsyoQkyNa58dG%2FcxUh3gT%2Fq7pZZfKmL%2FuZJETzY%3D&reserved=0>
>>> https://doi.org/10.1371/journal.pcbi.1002202
>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761410222%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=19Yi4LmrjaVo7yt8DZNhrSNwIAW2u%2FsIUm6C7N%2Flyp0%3D&reserved=0>
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761410222%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4%2BA08oCtdbTYd5kCLwBPL%2FEV8rwi%2Fech%2FQUgi5nQo68%3D&reserved=0>
>>> https://doi.org/10.1371/journal.pcbi.1002202
>>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761410222%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=19Yi4LmrjaVo7yt8DZNhrSNwIAW2u%2FsIUm6C7N%2Flyp0%3D&reserved=0>
>>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761420177%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=w6qp2qVtPFLm9DzIA5tRYM254me8o%2F%2FOlOow3%2FRZsQw%3D&reserved=0>
>> https://doi.org/10.1371/journal.pcbi.1002202
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761420177%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=OAWtCnVCY2qRLSHeTRNky9TYYJxSZBAbYVby7DKrYyo%3D&reserved=0>
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761430133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jEQvVxzuoEf7b6j1KQJNGIwuIWcjjwLJ638ltBLuJVE%3D&reserved=0>
>> https://doi.org/10.1371/journal.pcbi.1002202
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761430133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=B8zEmFCcbbbDApHVjkkx6rG9Q%2FpDCuE6isQU1hEGL%2Fw%3D&reserved=0>
>>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=04%7C01%7C%7Cac3574fb81c84972efe008d8c8fe0d64%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637480340761430133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jEQvVxzuoEf7b6j1KQJNGIwuIWcjjwLJ638ltBLuJVE%3D&reserved=0>
> https://doi.org/10.1371/journal.pcbi.1002202
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20210204/20b00a31/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2021-02-04 at 10.54.17 AM.png
Type: image/png
Size: 45162 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20210204/20b00a31/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2021-02-04 at 11.13.27 AM.png
Type: image/png
Size: 36414 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20210204/20b00a31/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2021-02-04 at 12.02.08 PM.png
Type: image/png
Size: 61166 bytes
Desc: not available
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20210204/20b00a31/attachment-0005.png>
More information about the fieldtrip
mailing list