[FieldTrip] Unsupported data format
Arjen Stolk
a.stolk8 at gmail.com
Sat Jul 16 04:26:32 CEST 2016
Great to hear!
> On Jul 15, 2016, at 6:54 PM, Nakyung Lee <rleese12 at berkeley.edu> wrote:
>
> Hooray, it works!
> Thanks for being patient with me :)
>
> >> cfg = [];
> >> cfg.demean = 'yes';
> >> data = signal_raw
>
> data =
>
> label: {176x1 cell}
> time: {[1x721186 double]}
> trial: {[176x721186 double]}
> fsample: 1000
>
> >> data2 = ft_preprocessing(cfg, data);
> Warning: discarding non-unique channel names
> > In ft_channelselection (line 102)
> In ft_selectdata>getselection_chan (line 649)
> In ft_selectdata (line 279)
> In ft_preprocessing (line 317)
> the call to "ft_selectdata" took 2 seconds
> preprocessing
> preprocessing trial 1 from 1
>
> the call to "ft_preprocessing" took 4 seconds
>
> And thanks again.
>
> Best,
> Rachel
>
>> On Fri, Jul 15, 2016 at 10:47 AM, Arjen Stolk <a.stolk8 at gmail.com> wrote:
>> Almost there, Rachel. :) Now ensure your data.label field contains strings and not numbers, i.e.
>>
>> data.label = {'1';'2'} and not {1;2}
>>
>> 2016-07-15 10:33 GMT-07:00 Nakyung Lee <rleese12 at berkeley.edu>:
>>> Hi Arjen,
>>>
>>> Yes, that is exactly the structure of 'signal_raw' I'm using.
>>> I tried your code (without running 'ft_definetrial' beforehand) and here's the log:
>>>
>>> >> cfg = [];
>>> >> cfg.demean = 'yes';
>>> >> data = signal_raw
>>>
>>> data =
>>>
>>> label: {176x1 cell}
>>> time: {[1x721186 double]}
>>> trial: {[176x721186 double]}
>>> fsample: 1000
>>>
>>> >> data2 = ft_preprocessing(cfg, data);
>>> Warning: the data does not contain a trial definition
>>> > In ft_warning (line 184)
>>> In fixsampleinfo (line 66)
>>> In ft_datatype_raw (line 175)
>>> In ft_checkdata (line 219)
>>> In ft_preprocessing (line 274)
>>> Warning: reconstructing sampleinfo by assuming that the trials are consecutive segments of a continuous recording
>>> > In ft_warning (line 184)
>>> In fixsampleinfo (line 79)
>>> In ft_datatype_raw (line 175)
>>> In ft_checkdata (line 219)
>>> In ft_preprocessing (line 274)
>>> Error using cell/ismember (line 34)
>>> Input A of class cell and input B of class cell must be cell arrays of strings, unless one is a string.
>>>
>>> Error in ft_senstype (line 303)
>>> if (mean(ismember(ft_senslabel('ant128'), sens.label)) > 0.8)
>>>
>>> Error in ft_channelselection (line 83)
>>> senstype = ft_senstype(datachannel);
>>>
>>> Error in ft_selectdata>getselection_chan (line 649)
>>> selchannel = ft_channelselection(cfg.channel, varargin{k}.label);
>>>
>>> Error in ft_selectdata (line 279)
>>> if haschan, [selchan, cfg] = getselection_chan (cfg, varargin{:}, cfg.select); end
>>>
>>> Error in ft_preprocessing (line 317)
>>> data = ft_selectdata(tmpcfg, data);
>>>
>>> This data is not segmented yet so there's only one trial (and hence time: {[1x721186 double]} and trial: {[176x721186 double]}) with 'Ntime' equal to '721186'.
>>> There were warning signs but seems like those are ignore-able ones.
>>> Again, let me know if you need more information and thank you.
>>>
>>> Best,
>>> Rachel
>>>
>>>> On Fri, Jul 15, 2016 at 10:01 AM, Arjen Stolk <a.stolk8 at gmail.com> wrote:
>>>> Hi Rachel,
>>>>
>>>> Given that you already have the data imported into matlab and organized into trials, try creating a structure 'data' with the following fields (i.e. data.label, data.time, data.trial) and the following dimensions:
>>>>
>>>> label: {151x1 cell} the channel labels (e.g. 'MRC13')
>>>> time: {1x266 cell} the timeaxis [1*Ntime double] per trial
>>>> trial: {1x266 cell} the numeric data [151*Ntime double] per trial
>>>> Then, for a proof of principle, try:
>>>>
>>>> cfg = [];
>>>> cfg.demean = 'yes';
>>>> data2 = ft_preprocessing(cfg, data);
>>>>
>>>> Let us know if this worked for you.
>>>> Arjen
>>>>
>>>>
>>>> 2016-07-15 9:51 GMT-07:00 Nakyung Lee <rleese12 at berkeley.edu>:
>>>>> Hi Jan-Mathijs,
>>>>>
>>>>> Thanks for your advice.
>>>>> Yes, I used 'ft_=definetrial' to segment the data.
>>>>> My data is in the fieldtrip style data format not because it's been segmented and preprocessed through fieldtrip but because our original dataformat is unsupported by fieldtrip (.Tbk, .Tdx, .tev, .tsq to be exact).
>>>>> So my coworker transformed those data into a .mat file and I transformed into a fieldtrip style raw data but it still needs some segmenting and some other preprocessing done on it (for example high-gamma filtering).
>>>>> But if 'ft_preprocessing' is only useable when reading proprietary data from disk, there's just no way we can preprocess the above data using fieldtrip?
>>>>>
>>>>> Thank you,
>>>>> Rachel
>>>>>
>>>>>> On Fri, Jul 15, 2016 at 1:10 AM, Schoffelen, J.M. (Jan Mathijs) <jan.schoffelen at donders.ru.nl> wrote:
>>>>>> Hi Rachel,
>>>>>>
>>>>>> Once your data is represented in a fieldtrip-style structure, there’s no need anymore to call ft_definetrial with a specified named dataset in the cfg.
>>>>>>
>>>>>> If you want to segment your data, you may still want to call ft_definetrial to generate a so-called ‘trl’ matrix, and subsequently use ft_redefinetrial to re-segment, i.e.:
>>>>>>
>>>>>> load(’signal_raw.mat’); % I assume that this files contains the fieldtrip-style data structure
>>>>>>
>>>>>> cfg = [];
>>>>>> cfg.trl = trl;
>>>>>> data_epoched = ft_redefinetrial(cfg, data);
>>>>>>
>>>>>> So, note that ft_preprocessing shouldn’t be called, this is onlyl useable when reading proprietary data from disk.
>>>>>>
>>>>>> It seems that you don’t need the cfg.dataset argument to begin with, and if you manage to titrate your ft_trialfun_test such that it converts the timestamps you provide it with into the correct trl-matrix, I don’t see a reason why it shouldn’t work.
>>>>>>
>>>>>> Best,
>>>>>> Jan-Mathijs
>>>>>>
>>>>>>
>>>>>>> On 15 Jul 2016, at 08:49, Nakyung Lee <rleese12 at berkeley.edu> wrote:
>>>>>>>
>>>>>>> Hi Arjen,
>>>>>>>
>>>>>>> Thank you for your reply.
>>>>>>> Your email got forwarded successfully.
>>>>>>>
>>>>>>> So currently I tried using 'ft_definetrial' and 'ft_preprocessing'.
>>>>>>> 'ft_definetrial' returns an error when used with 'ft_trialfun' set to default.
>>>>>>> Below is the code I used (and 'signal_raw.mat' is a fieldtrip raw data and it has all three fields you mentioned).
>>>>>>>
>>>>>>> cfg.dataset = 'signal_raw.mat';
>>>>>>> cfg.trialdef.event = subj_globals.buttonpress_times;
>>>>>>> cfg = ft_definetrial(cfg);
>>>>>>>
>>>>>>> So I simply call ft_definetrial with minimal configuration and here's the error I got:
>>>>>>>
>>>>>>> Error using ft_read_header (line 2194)
>>>>>>> unsupported header format "matlab"
>>>>>>>
>>>>>>> Error in ft_trialfun_general (line 78)
>>>>>>> hdr = ft_read_header(cfg.headerfile, 'headerformat', cfg.headerformat);
>>>>>>>
>>>>>>> Error in ft_definetrial (line 177)
>>>>>>> [trl, event] = feval(cfg.trialfun, cfg);
>>>>>>>
>>>>>>> Error in trialdef_alsotest (line 12)
>>>>>>> cfg = ft_definetrial(cfg);
>>>>>>>
>>>>>>> I circumvented this issue by using my own trialfun ('ft_trialfun_test' in the code).
>>>>>>> We already have all the necessary timestamps so I could build a very simple one.
>>>>>>> Here's a new code (I ran the code with minimal cfg for 'ft_preprocessing' to isolate the problem):
>>>>>>>
>>>>>>> cfg.dataset = 'signal_raw.mat';
>>>>>>> cfg.trialfun = 'ft_trialfun_test';
>>>>>>> cfg.trialdef.event = subj_globals.buttonpress_times;
>>>>>>>
>>>>>>> cfg = ft_definetrial(cfg);
>>>>>>>
>>>>>>> cfg.continuous = 'yes';
>>>>>>> buttonpress_hg = ft_preprocessing(cfg);
>>>>>>>
>>>>>>> And here's the error I got:
>>>>>>>
>>>>>>> Error using ft_read_header (line 2194)
>>>>>>> unsupported header format "matlab"
>>>>>>>
>>>>>>> Error in ft_preprocessing (line 397)
>>>>>>> hdr = ft_read_header(cfg.headerfile, 'headerformat', cfg.headerformat, 'coordsys', cfg.coordsys, 'coilaccuracy',
>>>>>>> cfg.coilaccuracy);
>>>>>>>
>>>>>>> Error in test_trialdef (line 18)
>>>>>>> buttonpress_hg = ft_preprocessing(cfg);
>>>>>>>
>>>>>>> So both error come from 'ft_read_header'.
>>>>>>> I've been looking at the code but it seemed quite hard to debug as 'ft_read_header' uses a lower level function that directly reads from the file (hence it seemed like a simple fix like passing in a fake header wouldn't work).
>>>>>>> Am I doing something wrong here?
>>>>>>>
>>>>>>> Again, thank you for your reply and let me know if you need more information.
>>>>>>>
>>>>>>> Best,
>>>>>>> Rachel
>>>>>>>
>>>>>>>> On Thu, Jul 14, 2016 at 12:10 AM, Arjen Stolk <a.stolk8 at gmail.com> wrote:
>>>>>>>> Hi Rachel,
>>>>>>>>
>>>>>>>> What is the error you're receiving, and how are you calling ft_preprocessing?
>>>>>>>>
>>>>>>>> If the data is in raw format (incl a trial, time and label field), it should theoretically be possible to use ft_preprocessing on it, eg;
>>>>>>>> cfg = []
>>>>>>>> cfg ..
>>>>>>>> data = ft_preprocessing(cfg, raw)
>>>>>>>>
>>>>>>>> where raw is your data as described above.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Arjen
>>>>>>>>
>>>>>>>> > On Jul 13, 2016, at 11:50 PM, Nakyung Lee <rleese12 at berkeley.edu> wrote:
>>>>>>>> >
>>>>>>>> > Dear FieldTrip community,
>>>>>>>> >
>>>>>>>> > My name is Rachel Lee and I'm a research assistant in Prof. Ming Hsu's lab in UC Berkeley.
>>>>>>>> >
>>>>>>>> > We currently have a .Tbk file, a .Tdx file, .tev file, .tsq file to preprocess and analyze but it seems like those data formats are not yet supported by FieldTrip.
>>>>>>>> > So, instead of using those raw data files, we've been trying to find a workaround by transforming them into a .mat file (and casting the .mat file into a raw datatype in FieldTrip by manually assigning fields to it).
>>>>>>>> > This hasn't been so successful; we've been unable to call ft_preprocessing because of the lack of a proper header format (which is needed in ft_read_header, one of the lower level functions that ft_preprocessing calls).
>>>>>>>> > I've been looking at the codes but been unable to come up with a solution.
>>>>>>>> >
>>>>>>>> > Does anyone have a similar experience?
>>>>>>>> > Is there any way to circumvent this issue?
>>>>>>>> > Any type of help would be appreciated.
>>>>>>>> >
>>>>>>>> > Thank you,
>>>>>>>> > Rachel Lee
>>>>>>>> > _______________________________________________
>>>>>>>> > fieldtrip mailing list
>>>>>>>> > fieldtrip at donders.ru.nl
>>>>>>>> > https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> fieldtrip mailing list
>>>>>>>> fieldtrip at donders.ru.nl
>>>>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> fieldtrip mailing list
>>>>>>> fieldtrip at donders.ru.nl
>>>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> fieldtrip mailing list
>>>>>> fieldtrip at donders.ru.nl
>>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> fieldtrip mailing list
>>>>> fieldtrip at donders.ru.nl
>>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>>
>>>>
>>>> _______________________________________________
>>>> fieldtrip mailing list
>>>> fieldtrip at donders.ru.nl
>>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>>
>>>
>>> _______________________________________________
>>> fieldtrip mailing list
>>> fieldtrip at donders.ru.nl
>>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>
>> _______________________________________________
>> fieldtrip mailing list
>> fieldtrip at donders.ru.nl
>> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20160715/ceafcb89/attachment-0002.html>
More information about the fieldtrip
mailing list