[FieldTrip] Downsampling Issue

Mon May 16 11:58:16 CEST 2022

Hi all,

To follow up on Alain’s previous cautionary remark - that matlab’s under-the-hood anti aliasing lowpassfilter is not so good - I wrote some test code to illustrate this. See the attached figure.

I simulated some simple data (at a 500 Hz sampling rate), which I resampled down to a new sampling rate of 200 Hz. Those data was either white noise (blue line, left panel), white noise with an added broad band component (which ramps up starting at ~ 100 Hz, middle panel), or white noise with an added narrow band component (which ramps up starting at ~ 100 Hz, right panel).

Using the native matlab implementation of resample.m highlights the aliasing in the red lines. Note, that the high frequency noise can be problematic, but also note that it really needs to be substantial (few orders of magnitude larger than the signal in the pass band).

The long story short of it is that I implemented an optional explicit lowpass filtering step in ft_resampledata - which currently is still in the development branch of fieldtrip - which would allow the user to specify a better behaved (e.g. by using a ‘firws’ filter, and also at a slightly lower cut off frequency than the new Nyquist) anti-aliasing filter. The behavior of this new functionality is highlighted in the yellow and purple lines.

I will write some documentation on the FT website (probably in a FAQ) and then will merge the associated PR https://github.com/fieldtrip/fieldtrip/pull/2028, which will also fix the problem of missing functions for matlab installations that lack the signal processing toolbox.

Stay tuned, and keep an eye on the new release messages on the website/twitter.

Best wishes,

Jan-Mathijs

[cid:AE2756C4-7FBB-4D1B-8A05-F7B419550E63]
On 14 May 2022, at 11:06, Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:

Aiai,

I was hoping that Mathworks eût ses affaires en ordre.

Thanks for the heads up, we’ll think of a way to address this issue in FieldTrip (if only by throwing a warning, and/or to recommend that the user should lowpassfilter in a separate step).

Best wishes,

Jan-Mathijs

On 13 May 2022, at 16:21, Alain de Cheveigne via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:

Hi Jan-Mathjis,

While you're at it you might consider fixing an issue with resample which is that the antialiasing filter is too high.

The -6dB cutoff is at Nyquist, implying that power just beyond Nyquist is not attenuated much, which can lead to aliased components that can be confusing.

The simple fix is to set it a bit lower, as in the version attached.

Best,
Alain

> On 13 May 2022, at 13:44, Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:
>
> Thanks Eleanor,
>
> Coincidentally just now one of our toolkit participants ran in exactly the same error.
>
> It is caused by the fact that FieldTrip expects that a resample.m function exists in the signal processing toolbox. Your installation of Matlab does not contain this toolbox, so Matlab falls back the the version of resample.m in the timeseries toolbox. This file, however, is incompatible with FieldTrip’s expectations unfortunately.
>
> A workaround for now could be to get yourself a copy of the octave (i.e. open source) version of the resample version (https://urldefense.com/v3/__https://sourceforge.net/p/octave/signal/ci/default/tree/inst/resample.m__;!!HJOPV4FYYWzcc1jazlU!4iA_J0mR_M16uk22L_TdfeVZnJ3ANPs5dq9Vb-ce8TWs_yBVnLxOlqHlzWUt_C1iyLuWhyUh54jImrIOUj_zQ_3OLfyOgQkIHnbzEA$ ), and adjust it a little bit so that it works in matlab.
>
> This is what I will do (and include it in FieldTrip), to avoid this problem in the future.
>
> I will create an issue on  https://urldefense.com/v3/__https://github.com/fieldtrip/fieldtrip/issues/2027__;!!HJOPV4FYYWzcc1jazlU!4iA_J0mR_M16uk22L_TdfeVZnJ3ANPs5dq9Vb-ce8TWs_yBVnLxOlqHlzWUt_C1iyLuWhyUh54jImrIOUj_zQ_3OLfyOgQkWYrur6Q$   where you can track the progress.
>
> Best wishes,
>
> Jan-Mathijs
>
>
>
>> On 13 May 2022, at 10:25, Eleanor Moses via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:
>>
>> Hi Jan-Mathijs,
>>
>> sorry for my misunderstanding, this is what I get using both resampledata and resample.
>>
>> >> which resampledata
>> 'resampledata' not found.
>> >> which resample
>> C:\Program Files\MATLAB\R2021b\toolbox\matlab\timeseries\@timeseries\resample.m  % timeseries method
>> >>
>>
>> Do you have any suggestions on how to ensure that I end up in the correct .m file when I call this function?
>>
>> Best wishes,
>>
>> Eleanor
>>
>> Eleanor Moses BPsych(Hons)
>> PhD Candidate
>> School of Psychology
>> The University of Queensland
>> Brisbane QLD 4072 Australia
>> Email: eleanor.moses at uqconnect.edu.au<mailto:eleanor.moses at uqconnect.edu.au>
>>
>> From: fieldtrip <fieldtrip-bounces at science.ru.nl<mailto:fieldtrip-bounces at science.ru.nl>> on behalf of Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
>> Sent: 13 May 2022 17:50
>> To: FieldTrip discussion list <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
>> Cc: Schoffelen, J.M. (Jan Mathijs) <janmathijs.schoffelen at donders.ru.nl<mailto:janmathijs.schoffelen at donders.ru.nl>>
>> Subject: Re: [FieldTrip] Downsampling Issue
>>
>> Sorry Eleanor, I was unclear. I meant to ask about the ‘resample’ function. The fact that you get an error internally when this function is called, makes me suspect that you end up in a wrong (overloaded but with different functionality) m-file.
>>
>> Best wishes,
>> Jan-Mathijs
>>
>>
>>> On 13 May 2022, at 04:18, Eleanor Moses via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:
>>>
>>> Hi Jan-Mathijs,
>>>
>>> thankyou for your reply. This is what I get (just the one version).
>>>
>>> >> which ft_resampledata
>>> C:\Users\Ellie\AppData\Roaming\MathWorks\MATLAB Add-Ons\Collections\fieldtrip-20220426\ft_resampledata.m
>>>
>>> I have also added the relevant fieldtrip directories using ft_defaults.
>>>
>>> Thankyou
>>>
>>> Eleanor Moses BPsych(Hons)
>>> PhD Candidate
>>> School of Psychology
>>> The University of Queensland
>>> Brisbane QLD 4072 Australia
>>> Email: eleanor.moses at uqconnect.edu.au<mailto:eleanor.moses at uqconnect.edu.au>
>>>
>>> From: fieldtrip <fieldtrip-bounces at science.ru.nl<mailto:fieldtrip-bounces at science.ru.nl>> on behalf of Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
>>> Sent: 12 May 2022 00:36
>>> To: FieldTrip discussion list <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
>>> Cc: Schoffelen, J.M. (Jan Mathijs) <janmathijs.schoffelen at donders.ru.nl<mailto:janmathijs.schoffelen at donders.ru.nl>>
>>> Subject: Re: [FieldTrip] Downsampling Issue
>>>
>>> Hi Eleanor,
>>>
>>> Could it be that you have an unexpected version of resampledata on your matlab path? What do you get if you type which resampledata on the matlab command line?
>>>
>>> Best wishes,
>>> Jan-Mathijs
>>>
>>>> On 11 May 2022, at 10:45, Eleanor Moses via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:
>>>>
>>>> Dear community,
>>>>
>>>> I am new to fieldtrip and am trying to preprocess my EEG data so that it was be compatible with later analysis using cosmomvpa.
>>>> However, I keep running into the same error with the ft_resampledata function.
>>>>
>>>> When I attempt to downsample, this is the error I get
>>>>
>>>> Check for incorrect argument data type or missing argument in call to function 'resample'.
>>>>
>>>> Error in ft_resampledata (line 231)
>>>>         newdat = transpose(resample(transpose(olddat),fsres,fsorig));
>>>>
>>>> Error in Fieldtrip_Test (line 27)
>>>> data_eeg = ft_resampledata(cfg, data_eeg);
>>>>
>>>>
>>>> I have checked previous posts to the community, to check that I am not encourting previously posted errors (I am downsampling from 1024Hz to 512, or have tried to 256, so this is a neat integer division which was a problem previously discussed that produced this error with ft_resampledata).
>>>> I have ensured that I have specified all necessary configurations (cfg.resamplefs = 512; cfg.detrend = 'no';) and aside from this am using the default configurations.
>>>> When I remove the resampling section from my preprocessing pipeline script, it otherwise works fine, and works with mvpa functions from cosmomvpa.
>>>> I tried to apply the ft_resampledata function after the ft_preprocessing function as the help ft_resampledata notes indicate that data should be
>>>> resampled as obtained from the ft_preprocessing function, but encounter the same error even if I rearrange when I use ft_resampledata (before pre-proc, right after importing data).
>>>>
>>>> I also encounter this error when I try to run tutorial fieldtrip preprocessing scripts that I have downloaded from the cosmomvpa website - so the error isn't unique to my script.
>>>>
>>>>
>>>> I have included the data structure and cfg's for the function below.
>>>>
>>>> % Raw data configuration
>>>> %            hdr: [1×1 struct]
>>>> %          label: {73×1 cell}
>>>> %           time: {[0 9.7656e-04 0.0020 0.0029 0.0039 0.0049 0.0059 0.0068 0.0078 0.0088 0.0098 0.0107 … ]}
>>>> %          trial: {[73×391168 double]}
>>>> %        fsample: 1024
>>>> %     sampleinfo: [1 391168]
>>>> %            cfg: [1×1 struct]
>>>>
>>>>
>>>> cfg = [];
>>>> cfg.resamplefs = 256;
>>>> cfg.detrend = 'no';
>>>> data_eeg = ft_resampledata(cfg, data_eeg);
>>>>
>>>> Any help would be greatly appreciated, kind regards
>>>>
>>>> Eleanor Moses
>>>>
>>>>
>>>>
>>>>
>>>> Eleanor Moses BPsych(Hons)
>>>> PhD Candidate
>>>> School of Psychology
>>>> The University of Queensland
>>>> Brisbane QLD 4072 Australia
>>>> Email: eleanor.moses at uqconnect.edu.au<mailto:eleanor.moses at uqconnect.edu.au>
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!6_puilsIXBSl-VOPdSiXILpOMezgn_ng1hMy0gaxjp0fc3pv8jiBwG1L_o4YBZTBM2aSjSABeEueTp4obfl-YYHVJS2isgYDW59U3Q$
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!5vIRskZ8q-baSkTYVKHdrn-tPB6p06Sv84lJkNyIKzG_beWjwUTiOQ6UhsjasLhUt88MSABz0VqCVPID_otcJjv5FbnjGvV39_0KNA$
>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!4Iz5S1yy8Ou8y0y1U0BAziooHkywyx4mU2t3DfZq7lbeZLAwvMlidcY8wfjXpWlrmeCjqdTNdrcXj7JXFCWee4khcfTNxaB_ZKJZCQ$
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!4iA_J0mR_M16uk22L_TdfeVZnJ3ANPs5dq9Vb-ce8TWs_yBVnLxOlqHlzWUt_C1iyLuWhyUh54jImrIOUj_zQ_3OLfyOgQnsfX4cEg$

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!4iA_J0mR_M16uk22L_TdfeVZnJ3ANPs5dq9Vb-ce8TWs_yBVnLxOlqHlzWUt_C1iyLuWhyUh54jImrIOUj_zQ_3OLfyOgQnsfX4cEg$
<nt_resample.m>

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!9UMguds6-MoZKyLm-qmC4OE0w4c9ubJEep7wmUbQdMkpW1buPvHXuRd_j-IydUw0RKrNS7x6E3fbqWTOnV5ns_4KVNkPIj88lSn0IQ$

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20220516/66b9c1ba/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2022-05-16 at 11.50.54.png
Type: image/png
Size: 136101 bytes
Desc: Screenshot 2022-05-16 at 11.50.54.png
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20220516/66b9c1ba/attachment-0001.png>