Questions on High pass filter and Padding in Preprocessing

[Robert Oostenveld]

Dear Feng,

On 12 Mar 2010, at 11:07, gao zai wrote:
> I am now working on the MEG data in 4D format. Since it was
> collected in DC, so I want to first use a bandpass filter [0.1 80]
> to filter my data. However, I found that the latter part of averaged
> results drifted away relative to the  results when no filter or only
> a low-pass filter is used (see the attached Fig.1). I thought it may
> be related to the end effect, so I used cfg.padding=1, it didn't work.

the padding option in ft_preprocessing pads the data _to_ the
specified length, not _with_ the indicated length. In your case
according to the figure the trial was ~2.5 seconds long, i.e. already
longer than 1 second, so cfg.padding=1 did not have an efefct (whereas
cfg.padding=10 does).

> Latter I turned to use cfg.padding=10 (1/.1=10) it works and no
> obvious drift was found.
>
> However, when tried to only apply a high pass filter to the raw
> data, regardless of using cfg.padding=10, the results always showed
> a square wave (see Fig.2). And there is a warning:"Matrix is close
> to singular or badly scaled. Results may be inaccurate. RCOND =
> 6.669292e-17."I don't understand why this happened. Does anybody
> know what is the problem with it? How can I get a right result in
> this case? Thank you very much.

You specified cfg.hpfreq=0.1, i.e. that frequencies below 0.1 Hz
should be removed. These frequencies correspond to dripfts in the data
of about 10 seconds. The data segment you are working with is only 2.5
seconds, so removing the 10 second drift fails. You can increase the
cfg.hpfreq to around 1/2.5 = 0.4 Hz, which would be the lowest
frequency that can be estimated from the 2.5 second data segment.

Alternatively, you could use the following pipeline

cfg=[];
cfg.dataset ='c,rfDC';
cfg.continuous = 'yes';
cfg.hpfilter = 'yes';
cfg.hpfreq = 0.1;
datacnt = ft_preprocessing(cfg);

to read it into memory as one continuous segment, and apply the filter
on the complete data.

Then you can do

cfg=[];
cfg.dataset ='c,rfDC';
cfg.trialdef.eventtype  = 'TRIGGER';
cfg.trialdef.eventvalue =42;
cfg.trialdef.prestim    = 1;
cfg.trialdef.poststim   = 1.5;
cfg.trialdef.offset     = 1;
cfg.trialfun            = 'mytrialfun';
cfg = definetrial(cfg);

followed by

datatrl = ft_redefinetrial(cfg, datacnt);

Reading all data (also the inter-trial interval, the breaks, and the
instruction periods) into memory requires a lot of RAM. If it does not
fit all at once, you could do the above section for small groups of
channels

cfg.channel = 1:10;
% cfg.channel = 11:20;
% cfg.channel = 21:30;
% ...

and then call ft_appenddata([], datatrl_changroup1,
datatrl_changroup2, ...)

best,
Robert











----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.