[FieldTrip] regressconfound and statistics

[Stolk, A. (Arjen)]

Hi Raghavan, Could you provide some more details on what statistical comparisons you are trying to make? It seems you have 4 blocks of recordings, each encompassing 2 task conditions, which you want to compare in terms of evoked/induced activity. If this is correct, you could 1) append data of the 4 blocks (subject-level) 2) regress out contributions from head movement using ft_regressconfound (subject-level) 3) split the trials into the 2 task conditions (subject-level) 4) compute t-scores using an independent samples t-test (subject-level) 5) non-parametric testing of effects using a dummy variable with zeros (group-level) From your descriptions, I got the impression you might have reversed the order of step 1 and 2. "However, the problem is, whatever significance I found earlier (i.e. by comparing means rather than t-statistic) doesn’t test significant now. I thought it might get better by doing this, but seems to get worse. Please correct me if I am doing anything wrong." It's hard to tell from this information what might have caused this difference. For instance, the incorrect order of the steps above, or differences in neural processes you may be capturing through using a mean vs. a t-score, or maybe even head positions that are positively correlated with your effects, possibly because they're driving them (e.g. differential distances to the MEG sensors between the task conditions may cause the amplitude of the same ERP be more pronounced in one condition than another). I'd recommend looking into the (average) head positions of the different blocks/conditions, and see what is going on there (e.g. http://fieldtrip.fcdonders.nl/example/how_to_incorporate_head_movements_in_meg_analysis? ). Arjen ----- Oorspronkelijk bericht -----
> Van: "Raghavan Gopalakrishnan" <gopalar.ccf at gmail.com>
> Aan: fieldtrip at science.ru.nl
> Verzonden: Dinsdag 25 februari 2014 23:01:17
> Onderwerp: Re: [FieldTrip] regressconfound and statistics
> Arjen et al,
> I have the following output data from regressconfound pertaining to
> one block of data.
> hpicompfiltdata =
> avg: [306x1500 double]
> var: [306x1500 double]
> time: [1x1500 double]
> dof: [306x1500 double]
> label: {306x1 cell}
> trial: [59x306x1500 double]
> dimord: 'rpt_chan_time'
> trialinfo: [59x1 double]
> cfg: [1x1 struct]
> beta: [37x306x1500 double]
> Since I have 4 blocks for each subject, I appended them using
> ft_appenddata, and using ft_timelockanalysis (mentioning appropriate
> cfg.trial and cfg.keeptrials=‘yes’; ) I split the data into 2
> conditions timelock_cond1 (shown below) and timelock_cond2, retaining
> only 204 gradiometers ( i suppose I am not supposed to use
> ft_combineplanar at this level since it would distort the results).
> timelock_cond1 =
> avg: [204x1500 double]
> var: [204x1500 double]
> time: [1x1500 double]
> dof: [204x1500 double]
> label: {204x1 cell}
> trial: [143x204x1500 double]
> dimord: 'rpt_chan_time'
> trialinfo: [143x1 double]
> cfg: [1x1 struct]
> Then these are the steps I did, per your suggestion.
> 1. I gathered all the time lock (from above step) for all subjects in
> a cell matrix, timelock_cond1{:} and timelock_cond2{:}
> 3. Then I ran first level stats for each subject, with cfg.statistic=
> indepsamplesT and cfg.method=‘analytic’; cfg.design as you suggested,
> and rest of cfg.
> for sj = 1: size(subj,2)
> stat1{sj}=first_level_stats(timelock_cond1{sj},timelock_cond2{:});
> end
> 4. Then, I did timelockgrandaverage
> cfg.keepindividual = 'yes';
> cfg.parameter = 'stat';
> stat2=ft_timelockgrandaverage(cfg,stat1{:});
> 5. Created a dummy variable just like stat2 in the above step and
> replaced the field ‘individual’ with zeros
> dummy2=stat2;
> dummy2.individual=zeros(size(stat2.individual));
> 6. Finally, ran the second level stats, with cfg. statistic =
> ‘depsamplesT’; cfg.method=‘montecarlo’; and rest of cfg.
> stat = ft_timelockstatistics(cfg, stat2, dummy2);
> However, the problem is, whatever significance I found earlier (i.e.
> by comparing means rather than t-statistic) doesn’t test significant
> now. I thought it might get better by doing this, but seems to get
> worse. Please correct me if I am doing anything wrong.
> Thanks,
> Raghavan
> What you described in your reply to me just now, calling
> ft_timelockstatistics at the subject level to calculate descriptives,
> still
> effectively reduces each subject to one "datapoint" -- the difference
> between conditions of the selected t-descriptive for that subject. I
> believe I understand how to do that, and how to then do nonparametric
> shuffle testing at the between-subjects level for the signfiicance of
> that
> within-subjects descriptive/difference.
> >> Ok. That's indeed the procedure I tried to outline. In your earlier
> >> reply to Raghavan, it sounded like you were suggesting
> pooling the trials themselves between subjects (that is, all subjects'
> trials go into one big "bucket"), with the trial-to-trial variance
> having
> been removed via ft_regressconfound, *then* calling the ft_statistics
> functions on this single large "bucket" of trials. Have I
> misunderstood
> your suggestion?
> >> I might have not been clear there then. I was meant to say that one
> >> could use ft_regressconfound on trials of two (or more) conditions
> >> combined
> of a single subject, prior to calculating the single subject
> t-descriptive
> (using ft_xxxstatistics). This allows one to compute a 'cleaner'
> t-score,
> or at least provide a good case that head movement did not
> differentially
> influence activity levels in the separate conditions. It was not
> related to
> pooling trials between subjects.
> 2014-02-21 16:14 GMT+01:00 Alik Widge < alik.widge at gmail.com >:
> > I don't *think* I'm asking about how to do second-order analysis,
> > unless I > have missed something important. I did read the link/FAQ
> > when I was working > on this last week, and that seems different
> > than the question I am asking. > > What you described in your reply
> > to me just now, calling > ft_timelockstatistics at the subject level
> > to calculate descriptives, still > effectively reduces each subject
> > to one "datapoint" -- the difference > between conditions of the
> > selected t-descriptive for that subject. I > believe I understand
> > how to do that, and how to then do nonparametric > shuffle testing
> > at the between-subjects level for the signfiicance of that >
> > within-subjects descriptive/difference. > > In your earlier reply to
> > Raghavan, it sounded like you were suggesting > pooling the trials
> > themselves between subjects (that is, all subjects' > trials go into
> > one big "bucket"), with the trial-to-trial variance having > been
> > removed via ft_regressconfound, *then* calling the ft_statistics >
> > functions on this single large "bucket" of trials. Have I
> > misunderstood > your suggestion? > > Alik > > Alik Widge >
> > alik.widge at gmail.com > (206) 866-5435 > > > > On Fri, Feb 21,
> > 2014 at 9:53 AM, Stolk, A. (Arjen) < > a.stolk at fcdonders.ru.nl >
> > wrote: > >> Dear Alik, >> >> If I am correct, you're asking how to
> > create subject-level >> t-descriptives, mentioned in the previous
> > post? A quick answer would be to >> perform an indepedent t-test (
> > cfg.statistic = indepsamplesT; ) using >> ft_timelockstatistics
> > (although you're not really 'testing' here, but only >> interested
> > in that t-descriptive), in which trials are the unit of >>
> > observations ( cfg.design = [ones(1, ntrls_conditionA) 2*ones(1, >>
> > ntrls_conditionB)]; ). Note that since an indepedent t-test tests
> > for >> differences of the means of two conditions, it won't fail on
> > unequal number >> of observations across the conditions (unlike a
> > dependent/paired t-test). >> >> Hope this gets you kickstarted for
> > the moment. I found a previous >> mail-conversation that goes more
> > into detail, providing more overview of >> the steps involved: >>
> > http://mailman.science.ru.nl/pipermail/fieldtrip/2011-November/004539.html
> > >> More useful documentation: >> >>
> > http://fieldtrip.fcdonders.nl/faq/what_is_the_idea_behind_statistical_inference_at_the_second-level
> > >> >> Yours, >> Arjen >> >> ------------------------------ >> >>
> > *Van: *"Alik Widge" < alik.widge at gmail.com > >> *Aan: *"FieldTrip
> > discussion list" < fieldtrip at science.ru.nl > >> *Verzonden:
> > *Vrijdag 21 februari 2014 11:38:10 >> >> *Onderwerp: *Re:
> > [FieldTrip] regressconfound and frequency domain >> >> Arjen, what
> > you just described is more or less what I struggled to do >> last
> > week and ultimately gave up as I was unable to figure out how to get
> > >> FT to do it despite much meditation over tutorials and source
> > files. Can >> you elaborate a bit more on what you are saying below
> > -- not the >> ft_regressconfound bit, but the bit about how to get
> > >> ft_statistics_montecarlo and its wrappers to do a trials-level
> > analysis and >> permutation at the whole-group level? Especially,
> > what does one put in >> cfg.design and how does one call the
> > function? Everything I could find in >> the tutorials described the
> > case of doing means at the subject level and >> then permutation of
> > means at the group level, which as you point out is >> underpowered
> > for subtle effects. >> >> My particular situation was
> > timelock-analyzed trials (with >> keeptrials='yes'), but I could not
> > find a way to set up cfg.design that >> did not throw error
> > messages. The thing that really seemed to bother it was >> that
> > there were different numbers of trials in the 2-3 conditions of >>
> > interest, since some had to be removed for excessive artifact. >> >>
> > Thanks for any help, >> Alik >> >> Alik Widge >> alik.widge at
> > gmail.com >> (206) 866-5435 >> >> >> >> On Fri, Feb 21, 2014 at 3:23
> > AM, Stolk, A. (Arjen) < >> a.stolk at fcdonders.ru.nl > wrote: >>
> > >>> Dear Raghavan, >>> >>> To compute a t-descriptive on subject
> > level freq data, you'll need to >>> use ft_freqstatistics. Have a
> > look here for instance: >>>
> > http://fieldtrip.fcdonders.nl/tutorial/cluster_permutation_freq >>>
> > >>> At the subject level, you do not need (non-parametric) cluster
> > >>> permutation testing (Maris & Oostenveld), as you're taking each
> > subject's >>> t-descriptives to the group level. At the group level,
> > you can then test >>> the hypothesis that there's a difference
> > between tasks/conditions (H1) vs. >>> no difference (H0). In order
> > to do so, you'll need to create a dummy >>> variable at the group
> > level, that has the same number of 'subjects', but >>> with zeros in
> > all fields (in your case this will be a .stat field). At the >>>
> > group level, you thus call ft_freqstatistics again. This approach
> > has the >>> advantage that you're more sensitive (as compared to
> > taking each subject's >>> mean to the group level) to effects that
> > are small but consistent over >>> trials in each subject. >>> >>>
> > Arjen >>> >>> >>> ------------------------------ >>> >>> *Van:
> > *"Raghavan Gopalakrishnan" < gopalar.ccf at gmail.com > >>> *Aan: *
> > fieldtrip at science.ru.nl >>> *Verzonden: *Donderdag 20 februari
> > 2014 23:33:18 >>> >>> *Onderwerp: *Re: [FieldTrip] regressconfound
> > and frequency domain >>> >>> Thanks Arjen, >>> Should I use
> > ft_freqdescriptives to compute t descriptives for >>> individual
> > subjects, and then take that to group level instead of mean? If >>>
> > not, what are the other alternatives? >>> Thanks, >>> Raghavan >>>
> > >>> Hi Raghavan, ft_regressconfound run on timelock data seems to
> > return output with avg field. However, ft_regressconfound run on
> > frequency data, does not return average. I see the avg field being
> > removed. Is there a reason? >> Not intentionally, but not an issue
> > either. You could still use ft_freqdescriptives to compute the
> > average for you, but see my comment below. Question - Since
> > ft_regressconfound outputs power spectrum of individual trials - 4D
> > matrix (instead of average), can I simply re-average the power
> > spectrum over trials to see the average power for that subject.
> > Also, I need to run grand average (over subjects) before running
> > statistics. I hope these steps does not distort the data. Please
> > advise. >> Remember that the mean over trials is not affected by
> > your clean-up of trial-by-trial variance due to head movement.
> > Taking each subject's mean (unaffected) to the group level is an
> > approach that will not benefit from your clean-up. In order to
> > benefit from reduced trial-by-trial variance, you'll need a measure
> > that depends on it, e.g. t-descriptive, neural activity-behavior
> > correlation (for taking to the group level). Hope this helps, Arjen
> > ----- Oorspronkelijk bericht ----- >>> >* Van: "Raghavan
> > Gopalakrishnan" <gopalar.ccf at gmail.com <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip >> >>> *>*
> > Aan: fieldtrip at science.ru.nl <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > >>> *>*
> > Verzonden: Donderdag 20 februari 2014 22:12:28 >>> *>* Onderwerp:
> > Re: [FieldTrip] regressconfound and frequency domain >>> *>* Arjen,
> > >>> *>* Thanks, I reduced down the time resolution so computation
> > can go >>> *>* faster. Now, m y matrix looks like this >>> *>*
> > hpicomptimefreq = >>> *>* label: {204x1 cell} >>> *>* dimord:
> > 'rpt_chan_freq_time' >>> *>* freq: [1x56 double] >>> *>* time:
> > [1x375 double] >>> *>* powspctrm: [4-D double] >>> *>* cumtapcnt:
> > [59x56 double] >>> *>* cfg: [1x1 struct] >>> *>* trialinfo: [59x1
> > double] >>> *>* beta: [4-D double] >>> *>* ft_regressconfound run on
> > timelock data seems to return output with >>> *>* avg field.
> > However, ft_regressconfound run on frequency data, does not >>> *>*
> > return average. I see the avg field being removed. Is there a
> > reason? >>> *>* Question - Since ft_regressconfound outputs power
> > spectrum of >>> *>* individual trials - 4D matrix (instead of
> > average), can I simply >>> *>* re-average the power spectrum over
> > trials to see the average power for >>> *>* that subject. Also, I
> > need to run grand average (over subjects) before >>> *>* running
> > statistics. I hope these steps does not distort the data. >>> *>*
> > Please advise. >>> *>* Thanks, >>> *>* Raghavan >>> *>* Date: Wed,
> > 19 Feb 2014 22:58:38 +0100 (CET) >>> *>* From: "Stolk, A. (Arjen)" <
> > a.stolk at fcdonders.ru.nl <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > > >>> *>*
> > To: FieldTrip discussion list < fieldtrip at science.ru.nl <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > > >>> *>*
> > Subject: Re: [FieldTrip] regressconfound and frequency domain >>>
> > *>* Message-ID: >>> *>* <
> > 2108167665.5423215.1392847118322.JavaMail.root at
> > sculptor.zimbra.ru.nl <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > >>> *>* >
> > >>> *>* Content-Type: text/plain; charset="utf-8" >>> *>* Dear
> > Raghavan, Good to hear it's working out for you. A short answer >>>
> > *>* would be 'no'. Reducing the size of your data matrix is likely
> > going >>> *>* to speed up computations. Your time resolution seems
> > pretty high (1500 >>> *>* frequency estimations per single trial);
> > do you need that many? Yours, >>> *>* Arjen ----- Oorspronkelijk
> > bericht ----- >>> *>* > Van: "Raghavan Gopalakrishnan" < gopalar.ccf
> > at gmail.com <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > > >>> *>*
> > > Aan: fieldtrip at science.ru.nl <
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip > >>> *>* >
> > Verzonden: Woensdag 19 februari 2014 22:01:00 >>> *>* > Onderwerp:
> > [FieldTrip] regressconfound and frequency domain >>> *>* > Arjen,
> > >>> *>* > Thanks for answering all my previous questions. I was
> > successfully >>> *>* > able to incorporate head movements to my erf
> > data. As I understand I >>> *>* > have to do this separately for the
> > time frequency data after keeping >>> *>* > individual trials. I am
> > interested in both beta and gamma bands >>> *>* > [15:1:70]. my time
> > frequency looks like this using wavelets, >>> *>* > timefreq = >>>
> > *>* > label: {204x1 cell} >>> *>* > dimord: 'rpt_chan_freq_time' >>>
> > *>* > freq: [1x56 double] >>> *>* > time: [1x1500 double] >>> *>* >
> > powspctrm: [4-D double] >>> *>* > cumtapcnt: [55x56 double] >>> *>*
> > > grad: [1x1 struct] >>> *>* > elec: [1x1 struct] >>> *>* > cfg:
> > [1x1 struct] >>> *>* > trialinfo: [55x1 double] >>> *>* > After
> > regressconfound >>> *>* > hpicomptimefreq = >>> *>* > label: {204x1
> > cell} >>> *>* > dimord: 'rpt_chan_freq_time' >>> *>* > freq: [1x56
> > double] >>> *>* > time: [1x1500 double] >>> *>* > powspctrm: [4-D
> > double] >>> *>* > cumtapcnt: [55x56 double] >>> *>* > cfg: [1x1
> > struct] >>> *>* > trialinfo: [55x1 double] >>> *>* > beta: [4-D
> > double] >>> *>* > Regressconfound took about 1 hr and 30 mins, since
> > its a huge matrix >>> *>* > [55x204x56x1500]. I have 25 such blocks
> > of data for 20 subjects. It >>> *>* > will take an enoumous amount
> > of time to process the data through >>> *>* > regressconfound. Is
> > there a workaround to make the processing faster >>> *>* > or am I
> > missing something. Any help would be of great help. >>> *>* >
> > Thanks, >>> *>* > Raghavan* >>> >>> >>> >>>
> > _______________________________________________ >>> fieldtrip
> > mailing list >>> fieldtrip at donders.ru.nl >>>
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip >>> >>> >>>
> > >>> >>> -- >>> Donders Institute for Brain, Cognition and Behaviour
> > >>> Centre for Cognitive Neuroimaging >>> Radboud University
> > Nijmegen >>> >>> Email: a.stolk at donders.ru.nl >>> Phone:
> > +31(0)243 68294 >>> Web: www.arjenstolk.nl >>> >>>
> > _______________________________________________ >>> fieldtrip
> > mailing list >>> fieldtrip at donders.ru.nl >>>
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip >>> >> >> >>
> > _______________________________________________ >> fieldtrip mailing
> > list >> fieldtrip at donders.ru.nl >>
> > http://mailman.science.ru.nl/mailman/listinfo/fieldtrip >> >> >> >>
> > >> -- >> Donders Institute for Brain, Cognition and Behaviour >>
> > Centre for Cognitive Neuroimaging >> Radboud University Nijmegen >>
> > >> Email: a.stolk at donders.ru.nl >> Phone: +31(0)243 68294 >> Web:
> > www.arjenstolk.nl >>
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
-- Donders Institute for Brain, Cognition and Behaviour Centre for Cognitive Neuroimaging Radboud University Nijmegen Email: a.stolk at donders.ru.nl Phone: +31(0)243 68294 Web: www.arjenstolk.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20140226/4359be35/attachment-0002.html>