<div dir="ltr">Dear Robert,<br><br>Thanks very much for your great answer. As you mentioned, I tried to read the continuous data into memory, it was slow. I will try to read them in group of channels. <br><br>BTW, what do you mean by "the padding option in ft_preprocessing pads the data _to_ the specified
length, not _with_ the indicated length. " I checked the script about ft-preprocessing, it seems to me that if the padding length is shorter than the specified length (in my case, 2.5s), as you point it out, it will not work. If the padding is longer than that, then fieldtrip will read a longer epoch than the specified length, so if I use a cfg.padding=10, it will input a 10s epoch (and the middle part of 2.5s is the trial I am focused) into memory, am I right? If I am right, can I just use the padding here (e.g., cfg.padding=10) instead of filtering the continuous data, since it is equal to filtering a longer trial and then drop the padding part after completion? <br>
<br>Millions of thanks.<br><br>Best,<br>Feng<br><br><br><br><div class="gmail_quote">On Fri, Mar 19, 2010 at 12:21 PM, Robert Oostenveld <span dir="ltr"><<a href="mailto:r.oostenveld@fcdonders.ru.nl">r.oostenveld@fcdonders.ru.nl</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Dear Feng,<div class="im"><br>
<br>
On 12 Mar 2010, at 11:07, gao zai wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
I am now working on the MEG data in 4D format. Since it was collected in DC, so I want to first use a bandpass filter [0.1 80] to filter my data. However, I found that the latter part of averaged results drifted away relative to the results when no filter or only a low-pass filter is used (see the attached Fig.1). I thought it may be related to the end effect, so I used cfg.padding=1, it didn't work.<br>
</blockquote>
<br></div>
the padding option in ft_preprocessing pads the data _to_ the specified length, not _with_ the indicated length. In your case according to the figure the trial was ~2.5 seconds long, i.e. already longer than 1 second, so cfg.padding=1 did not have an efefct (whereas cfg.padding=10 does).<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Latter I turned to use cfg.padding=10 (1/.1=10) it works and no obvious drift was found.<br>
<br>
However, when tried to only apply a high pass filter to the raw data, regardless of using cfg.padding=10, the results always showed a square wave (see Fig.2). And there is a warning:"Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 6.669292e-17."I don't understand why this happened. Does anybody know what is the problem with it? How can I get a right result in this case? Thank you very much.<br>
</blockquote>
<br></div>
You specified cfg.hpfreq=0.1, i.e. that frequencies below 0.1 Hz should be removed. These frequencies correspond to dripfts in the data of about 10 seconds. The data segment you are working with is only 2.5 seconds, so removing the 10 second drift fails. You can increase the cfg.hpfreq to around 1/2.5 = 0.4 Hz, which would be the lowest frequency that can be estimated from the 2.5 second data segment.<br>
<br>
Alternatively, you could use the following pipeline<div class="im"><br>
<br>
cfg=[];<br>
cfg.dataset ='c,rfDC';<br></div>
cfg.continuous = 'yes';<br>
cfg.hpfilter = 'yes';<br>
cfg.hpfreq = 0.1;<br>
datacnt = ft_preprocessing(cfg);<br>
<br>
to read it into memory as one continuous segment, and apply the filter on the complete data.<br>
<br>
Then you can do<div class="im"><br>
<br>
cfg=[];<br>
cfg.dataset ='c,rfDC';<br>
cfg.trialdef.eventtype = 'TRIGGER';<br>
cfg.trialdef.eventvalue =42;<br>
cfg.trialdef.prestim = 1;<br>
cfg.trialdef.poststim = 1.5;<br>
cfg.trialdef.offset = 1;<br>
cfg.trialfun = 'mytrialfun';<br>
cfg = definetrial(cfg);<br>
<br></div>
followed by<br>
<br>
datatrl = ft_redefinetrial(cfg, datacnt);<br>
<br>
Reading all data (also the inter-trial interval, the breaks, and the instruction periods) into memory requires a lot of RAM. If it does not fit all at once, you could do the above section for small groups of channels<br>
<br>
cfg.channel = 1:10;<br>
% cfg.channel = 11:20;<br>
% cfg.channel = 21:30;<br>
% ...<br>
<br>
and then call ft_appenddata([], datatrl_changroup1, datatrl_changroup2, ...)<br>
<br>
best,<br>
Robert<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
----------------------------------<br>
The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also <a href="http://listserv.surfnet.nl/archives/fieldtrip.html" target="_blank">http://listserv.surfnet.nl/archives/fieldtrip.html</a> and <a href="http://www.ru.nl/neuroimaging/fieldtrip" target="_blank">http://www.ru.nl/neuroimaging/fieldtrip</a>.<br>
</blockquote></div><br></div>
<p>----------------------------------</p>
<p>The aim of this list is to facilitate the discussion between users of the FieldTrip toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis.</p>
<p> http://listserv.surfnet.nl/archives/fieldtrip.html</p>
<p> http://www.ru.nl/fcdonders/fieldtrip/</p>