[FieldTrip] ft_channelrepair increases the size of the data file

[Stephen Whitmarsh]

Hi Vitória,

To add on Eelke's advice: you can also just temporarily save it in a
different variable, and put it back in your datastructure when done with
your channelrepair.

Cheers,
Stephen

On 21 March 2015 at 02:52, Eelke Spaak <eelke.spaak at donders.ru.nl> wrote:

> Hi Vitória,
>
> It should be perfectly safe to delete cfg.previous (or data.cfg); no
> FieldTrip functions depend on it. Note that you won't be able to run
> ft_analysisprotocol on those data anymore, but that should be the only
> limitation.
>
> Best,
> Eelke
> Op 21 mrt. 2015 01:38 schreef "Vitoria Piai" <v.piai.research at gmail.com>:
>
>  Hi FT-ers,
>>
>> I think I won't be able to figure out this problem by diving into the
>> ft_channelrepair code. So if you can shed any light to what could be going
>> on, I'd be *very *thankful!!
>>
>> I followed Eelke's and JM's advice to split the data, fix some channels
>> only for certain trials, then put the data together. All fine, except for
>> the following. If I apply this procedure a large number of times (haven't
>> figured out what the threshold is yet), the file size increases by a
>> considerable amount. For example, when I fixed one channel for all trials,
>> and then 7 channels for a couple of trials, the file went from 1GB to
>> almost 3GB. Then Matlab slows down and eventually hangs, and the data
>> (saved with '-v7.3') is "corrupted" once I try to load it to my workspace
>> again.
>>
>> It seems to me that ft_channelrepair is keeping information in a
>> 'previous' field such that every call to it will just keep adding bytes, is
>> that the case? I'm planning to delete everything that is in cfg.previous
>> and just keep the .trl field for later. Can you foresee I will run into
>> unrepairable trouble later because of this? Is there any other field within
>> the cfg.previous I should keep besides the .trl?
>>
>> Thanks a lot again!!
>> Have a nice weekend,
>> Vitória
>>
>> On 3/13/2015 9:08 AM, Schoffelen, J.M. (Jan Mathijs) wrote:
>>
>> Hi V,
>> Note that you can use the trialinfo field in your data for bookkeeping, e.g. (assuming there is already such field in your data)
>>
>> data.trialinfo(:,end+1) = (1:numel(data.trial))’;
>> cfg = [];
>> cfg.trials = sel_to_be_repaired;
>> cfg.etc
>> data1 = ft_channelrepair(cfg,data);
>>
>> cfg = [];
>> cfg.trials = sel_to_be_unrepaired
>> data2 = ft_selectdata(cfg, data);
>>
>> data = ft_appenddata([],data1,data2);
>>
>> Although the order has now been changed, you’ll see that the last column of the trialinfo field still pertains to your original trial indices.
>>
>> JM
>>
>>
>>
>> On Mar 13, 2015, at 4:35 PM, Vitória Piai <v.piai.research at gmail.com> <v.piai.research at gmail.com> wrote:
>>
>>
>>  Thanks, Eelke!
>>
>> I had already done what you suggested. The issue is that ft_appenddata appends (;-) ) the data rather than restoring it back to normal. Since I defined all trials to repair at the very beginning, none of the indexes for those trials make sense anymore after the first call to ft_appenddata. I guess I'll work around it by keeping values rather than indices for the trials I need to repair.
>>
>> Thanks a lot,
>> Hope all is well in Nijmegen!!
>>
>> On 3/11/2015 12:32 AM, Eelke Spaak wrote:
>>
>>  Hi Vitoria,
>>
>> Yes, that is the expected behaviour of ft_channelrepair, because that
>> is consistent with all other functions supporting a cfg.trials-option
>> (i.e. select first, then do the work on what remains).
>>
>> It should be quite straightforward to do something like:
>>
>> cfg = [];
>> cfg.trials = goodtrials;
>> dat1 = ft_selectdata(cfg, data);
>>
>> cfg = [];
>> ...
>> cfg.trials = badtrials;
>> dat2 = ft_channelrepair(cfg, data);
>>
>> datcmb = ft_appenddata([], dat1, dat2);
>>
>> to achieve what you want.
>>
>> Groetjes,
>> Eelke
>>
>> On 10 March 2015 at 23:58, Vitoria Piai <v.piai.research at gmail.com> <v.piai.research at gmail.com> wrote:
>>
>>  Hi all,
>>
>> I'm using ft_channelrepair, ($Id: ft_channelrepair.m 9520 2014-05-14
>> 09:33:28Z) to repair bad channels. I wanted to do it only for certain
>> trials, and when I saw the option cfg.trials, I was really happy.
>> However, if I pass cfg.trials with a vector rather than 'all', then the
>> function does:
>> ft_selectdata (lines 104, 108),
>> then from line 352 onwards, things are converted back to normal.
>> But if the cfg contained only some trials (as I'm trying to use it for),
>> these are not restored back, and so I'm left with a data structure that
>> only has as many trials as my cfg had.
>>
>> I was just wondering whether that is the expected behaviour of
>> ft_channelrepair and that I should restore things myself (presumably
>> with ft_append-like functions). If so, it would be nice if the help on
>> this function would say that...
>> If I'm simply using a too old version, my apologies. (I should start
>> working with github, I know!!)
>>
>> Thanks a lot,
>> Vitoria
>>
>>
>>
>> _______________________________________________
>> fieldtrip mailing listfieldtrip at donders.ru.nlhttp://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>  _______________________________________________
>> fieldtrip mailing listfieldtrip at donders.ru.nlhttp://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>  _______________________________________________
>> fieldtrip mailing listfieldtrip at donders.ru.nlhttp://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>
>> _______________________________________________
>> fieldtrip mailing listfieldtrip at donders.ru.nlhttp://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>>
>>
>>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20150325/eebfd907/attachment-0002.html>