[FieldTrip] Trying to merge blocks of data using ft_appenddata

[Schoffelen, J.M. (Jan Mathijs)]

Hi Éva,

I don’t completely follow your exposition, but I don’t understand well why you think that you should use cfg.keepsampleinfo = ‘yes’ for the reason that you mentioned:  (QUOTE): 'Even though the files are separated, they should contain the same events, so I kept the option cfg.keepsampleinfo = 'yes’.’

If you retain the sampleinfo in the data structures that come from different files, the mapping from recorded samples to samples present in the data structure - which is what the sampleinfo reflects - results in conflicting information. Let’s say that you have two recordings (with a length of 100 samples) that are read in into a fieldtrip structure. If you retain the sampleinfo, both datastructures will  have a sampleinfo [1 100]. The later check (in ft_fetch_data, called by a subfunction of ft_databrowser) investigates (in case of a sampleinfo being present, and in that case based on the sampleinfo), whether the data segments are overlapping - under the assumption that the segments are derived from the same recording. In your case it fails.

Now, you also mention ft_appenddata in passing. I don’t know what this has to do with it, but I guess that you tried this function to append the two data structures. I don’t know how by heart how clever this function is, so it could be that it did not work in your case. I suspect that if you get rid of the sampleinfo, things will work out fine.

Best wishes,
Jan-Mathijs



On 26 Jan 2023, at 22:49, Eva Masson via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:

Dear FieldTrip community,

I am trying to merge two blocks of (so far) continuous data. The recordings are the same in the sense that they have the same number of channels. The files contain events, but I have not defined trials yet. (as a side note, I tried merging with defined trials in both blocks, but the issue is the same)

To do so, after I transformed my .set files into a fieldtrip structure, I tried using ft_appenddata.
Even though the files are separated, they should contain the same events, so I kept the option cfg.keepsampleinfo = 'yes'.

In appearances, the obtained structure is what I expected (twice as much data as a single file). But when I try to visualize the raw data with ft_databrowser, I get the following error:


Error using ft_fetch_data
some of the requested samples occur twice in the data and have conflicting values

Error in ft_databrowser>redraw_cb (line 1807)
  dat = ft_fetch_data(opt.orgdata, 'header', opt.hdr, 'begsample', begsample, 'endsample', endsample, 'chanindx', chanindx, 'allowoverlap', cfg.allowoverlap, 'skipcheckdata', true);

Error in ft_databrowser (line 810)
redraw_cb(h);


From my understanding ('the trials will be concatenated', from documentation), I expect ft_appenddata to merge the data as data1 followed by data2 as a big continuous data file. Is there a misunderstanding of how ft_appenddata actually works? Or is it a sign for a design issue of my experiment?

Thank you very much for your help.
Best regards,
Éva

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!6lL40jy3LPXM6ebwJoPRKwNXe8x0-9B5MXorSFIBF23I7oazUFZ1XuiaRSP5lI2xV7y8DFyXjr8iVytp-ARtO48Wgmu5EoPogJd-0Q$

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20230127/fc349a4d/attachment.htm>