[FieldTrip] ft_write_data error for .edf format

[Gio Piantoni]

Hi Iris,

Thanks for sharing the data and the clear description of the problem.
The issue is not the sampling rate but the physical range of some of
your channels. And you're right that it's a problem in writing EDF
files, not reading them in (i.e. ft_write_data fails in this case and
'exampledata.edf' is not a legal EDF file).

If you want to track down the issue, check "write_edf.m", where
physMin and physMax should be 960 characters long (120 channels x 8
bytes) but in your case, they are 966 and 963. So, there will be a
small offset in the header and read_edf.m (or any other EDF reader)
cannot make sense of it. F.e. your channel 117 has a physical minimum
of -2038879.22647786, which then becomes:

sprintf('%-8g', -2038879.22647786)
'-2.03888e+06'
(which is longer than 8 characters).

The scaling number "-2038879.22647786" is computed by write_edf.m
following a heuristic approach. In this case, the range is too large
to be represented with sprintf.

Some of the options are:
1) fix the sprintf syntax
2) use a more robust approach to estimating the scaling
3) use brainvision analyzer format

I wouldn't know how to solve 1) and 2) in a reliable way, so I suggest
you go for 3)...

Hope this helps,
-g

PS: I'm surprised you said that it used to work fine a few months ago.
I tested some older commits that affected edf functions
(d2b408d9ab6e1baf64659de2c2a6132d743fe34e
b51f1c13d30fc8ecd7359f54ec2e4be5841aa6bf
998cda7bef2a9c4506c8b8292095e364e5eca9ec
885e6ec0a6da3c193c506114812ec8a4cb494e9a) and your MWE always throws
some kind of error.



On Mon, Sep 24, 2018 at 6:33 PM Iris Groen <ig24 at nyu.edu> wrote:
>
> Hello FieldTrippers,
>
> I am working on an iEEG dataset. As part of our processing pipeline I convert the data to BIDS format, and I would like to use the basic Fieldtrip functions ft_read_data and ft_write_data to read in my data file, split it up in separate runs, and then write the data out again in the same format (.edf), without applying any processing on the data itself.
>
> When I used these functions a few months ago they worked just fine. However, when I use them now I run into a problem. When I read in the original data, edfread throws a warning, but not an error. However, when I read in that data after it has been written out by ft_write_data, it results in an error. This happens even when I read in and write out the exact same file, suggesting there might be something going wrong in the data writing process.
>
> Please see below for example code and data. The same error does not appear when I write the data out to brainvision analyzer format, so I could simply switch that format. But I still would like to understand why writing to .edf is resulting in a problem.
>
> Many thanks for your help!
>
> Best,
> Iris
>
> fileName = 'examplerecording.edf’;
>
> % read in the data
> data_in = ft_read_data(fileName);
> Warning: channels with different sampling rate not supported, selecting subset of 120 channels at 512.000000 Hz
> hdr_in = ft_read_header(fileName);
> Warning: channels with different sampling rate not supported, selecting subset of 120 channels at 512.000000 Hz
>
> % write out the data
> ft_write_data(‘exampledata.edf', data_in, 'header', hdr_in, 'dataformat', 'edf’);
>
> % read the data in again
> data_out = ft_read_data(‘exampledata.edf’);
>
> Warning OPENEDF: Failing Physical Minimum, taking Digital Minimum instead
> Warning OPENEDF: Failing Physical Maximum, taking Digital Maximum instead
> Index exceeds matrix dimensions.
> Error in read_edf (line 209)
>     if EDF.Chan_Select(k)
> Error in ft_read_header (line 775)
>       hdr = read_edf(filename);
> Error in ft_read_data (line 201)
>   hdr = ft_read_header(filename, 'headerformat', headerformat, 'chanindx', chanindx, 'checkmaxfilter', checkmaxfilter);
> 209     if EDF.Chan_Select(k)
>
>
> Please find the data file from the example above here:
> https://nyu.box.com/s/eleoyaiowyfxutrxl2v1j8as6v985rw7
>
> Note on these data: One problem seems to be the different sampling rates of the channels as indicated by the first warning. However, this warning was present before and then it didn’t result in an error when reading in the same data after it had been written out: the two channels with deviating sampling rates were simply dropped (as indicated by the warning), resulting in 120 channels instead of the original 122. But it seems that now when reading in the same data results in another channel being dropped, creating a discrepancy between the number of channels in EDF.Chan_Select (119 in this case) and the number of channels listed in EDF.NS (120). So the issue appears to be with the header channel info - should I perhaps adapt the header info in  hdr.orig when writing out the file using ft_write_data? Alternatively, coudl the warning about the physical and digital minima/maxima be an indication of what’s causing this problem? (there is no such warning when reading in the same data after first writing it to brain vision analyzer format).
>
> —
>
> Iris Groen
> Postdoctoral Associate
> New York University
> 6 Washington Place
> New York, NY, 10003
> iris.groen at nyu.edu
>
>
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202