Some suggestion for improving performance in reading buffer event in ft_read_event

[teris tam]

Dear all,

When I am profiling the performance of reading buffer event in
ft_read_event, in around line 710, I see the following code causing a
performance neck:

  for i=1:length(evt)
      % convert the field "type" into the Matlab representation
      this_type = type{evt(i).type_type+1};
      this_size = wordsize{evt(i).type_type+1} * evt(i).type_numel;
      sel = 1:this_size;
      if strcmp(this_type, 'char')
        event(i).type = char(evt(i).buf(sel));
      else
        event(i).type = typecast(evt(i).buf(sel), this_type);
      end
      .
      .
      .
end


If I understand correctly, what the code is currently doing is to read a
maximum of 100 events (the number 100 was obtained from my observation when
the code was running) from the buffer and then typcast all of them.
Filtering of the event is then done by ft_filter_event. However, event(i) is
growing inside a loop and has significant negative impact on performance.
The impact is even larger in real-time processing because very often we need
to keep polling the event information from the buffer. In most of the case,
reading such a large number of event to search for the required event type
is not necessary because there is only a handful of states in the
experimental paradigm. So I suggest we add a "maxsearch" option to let the
user set how many total events they want to read from the buffer to search
for the required event:

*In ft_read_event, after:*

flt_mintimestamp = keyval('mintimestamp', varargin);
flt_maxtimestamp = keyval('maxtimestamp', varargin);
flt_minnumber    = keyval('minnumber', varargin);
flt_maxnumber    = keyval('maxnumber', varargin);

*add:*
flt_maxsearch     = keyval('maxsearch',varargin);

*then around line 710, before:*

  for i=1:length(evt)
      % convert the field "type" into the Matlab representation
      this_type = type{evt(i).type_type+1};
      this_size = wordsize{evt(i).type_type+1} * evt(i).type_numel;
      sel = 1:this_size;
      if strcmp(this_type, 'char')
        event(i).type = char(evt(i).buf(sel));
      else
        event(i).type = typecast(evt(i).buf(sel), this_type);
      end
      .
      .
      .
end


*add:*
if(~isempty(flt_maxsearch))
        evt=evt((end-flt_maxsearch+1):end);
end


I have done some profiling and it does improve the performance. Hope that
helps.

Best regards,
Teris

----------------------------------
The aim of this list is to facilitate the discussion between users of the FieldTrip  toolbox, to share experiences and to discuss new ideas for MEG and EEG analysis. See also http://listserv.surfnet.nl/archives/fieldtrip.html and http://www.ru.nl/neuroimaging/fieldtrip.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20100519/7be22073/attachment-0001.html>