[FieldTrip] Threshold free cluster enhancement for a Repeated Measures design

Fri Apr 26 13:28:52 CEST 2024

Hi Aishwarya,

I think it is fine to do fine-grained pairwise comparisons, but typically in a 2x3 factorial within subject design, one would typically compute F-statistics in an omnibus test (for which fieldtrip does not have a ready made ’statfun’ -> there is a ft_statfun_depsamplesF, but this only works for 1xN factorial designs), or do pairwise tests for the main effect, (e.g.by averaging across R/L/H conditions and comparing vis versus invis). Of course, in these types of designs, people typically are interested in statistically evaluating the interactions. The ancient 2011 post you refer to in your e-mail is still valid in this respect, and the take home message still is that it is quite complicated to test for an interaction effect using permutation tests (particularly if the design gets fancy).

Another piece of potentially relevant documentation is here: https://www.fieldtriptoolbox.org/faq/how_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.fieldtriptoolbox.org%2Ffaq%2Fhow_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests%2F&data=05%7C02%7Cfieldtrip%40science.ru.nl%7Ce52d6a3df3894485616308dc65e40cf5%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638497277344262717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=5M8ngveogdRghRRGnHJsR%2FHFLaK7TZbqbDSl%2BPheiQc%3D&reserved=0>. The take home message from that page is that 2x2 designs are relatively straightforward to handle, once you explicitly want to go to 2x3, you need to do some additional coding to write an appropriate statfun for the test statistic of interest.

With respect to your question regarding tfce: by definition this is a ’threshold-free’ heuristic, so you will not get clusters. Rather, the test statistic + p-value is obtained at each data point. If you use correctm=‘cluster’, the clusters are a necessary by-product of the clustering heuristic that is used for the multipe-comparison correction. Note - by the way - that it’s incorrect to talk about ’significant clusters’: the statistical test is significant, not the cluster. The decision about significance is based on the ‘p-value’ of the largest cluster. The fact that your test is not signicant if you don’t a priori constrain the search range based in prior knowledge is too bad, but that’s the reality unfortunately. If however you can justify the a priori selection of electrodes, and frequency bands based on earlier work, it’s perfectly fine to report those results (and motivate your choice for the prior selection), it is not mandatory to show that there’s a significant effect using the whole search space.

Best wishes,
Jan-Mathijs

On 25 Apr 2024, at 13:26, Bhonsle, Aishwarya via fieldtrip <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:

Hi all,
I am a little lost with cluster-based analyses for my data. My experiment is a within-subjects design, with 30 subjects performing a perceptual task where they report on whether they see a given target or not (factor, perception; 2 levels – visible, invisible), and they perform the task either at rest or while cycling at low-resistance or at high-resistance (factor, cycling; 3 levels: rest, low-resistance, high-resistance).
I am using Fieldtrip to analyse alpha (8-12 Hz) amplitudes in a specific time window of interest. I do this by calculating the average absolute value of the 8-12 Hz filtered Hilbert transform for each channel separately. I am trying to find time points and channels that differ between my experimental conditions in my time window of interest by running a threshold free cluster enhancement, requiring a minimum number of two neighbouring channels and consecutive time points and employing a Monte Carlo permutation with 5000 iterations in order to obtain cluster-corrected p-values as follows:
cfg = [];
cfg.channel     = 'EEG';
cfg.latency     = [2 3];
cfg.avgovertime = 'no';
cfg.parameter   = 'alpha';
cfg.method      = 'montecarlo';
cfg.statistic   = 'ft_statfun_depsamplesT';
cfg.alpha       = 0.05;
cfg.correctm    = 'tfce';
cfg.correcttail = 'prob';
cfg.numrandomization = 5000;
cfg.minnbchan        = 2;      % minimal number of neighbouring channels

Nsub = 30;
cfg.design(1,1:2*Nsub)  = [ones(1,Nsub) 2*ones(1,Nsub)];
cfg.design(2,1:2*Nsub)  = [1:Nsub 1:Nsub];
cfg.ivar                = 1; % the 1st row in cfg.design contains the independent variable
cfg.uvar                = 2; % the 2nd row in cfg.design contains the subject number

%Effect of perception
R_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, R_alpha_vis{:}, R_alpha_invis{:});
L_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, L_alpha_vis{:}, L_alpha_invis{:});
H_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, H_alpha_vis{:}, H_alpha_invis{:});

% Effect of cycling
RLvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, R_alpha_vis{:}, L_alpha_vis{:});
RHvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, R_alpha_vis{:}, H_alpha_vis{:});
LHvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, L_alpha_vis{:}, H_alpha_vis{:});
RLinvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, R_alpha_invis{:}, L_alpha_invis{:});
RHinvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, R_alpha_invis{:}, H_alpha_invis{:});
LHinvis_stat_cluster_preRDM_noavg = ft_timelockstatistics(cfg, L_alpha_invis{:}, H_alpha_invis{:});

I am not sure of the following:
1.     The design of test – am I doing this right? I am comparing two conditions at a time, choosing the comparisons that would allow me to check the main effects of cycling and of perception if this were a Repeated Measures ANOVA. But I am not sure if this is the appropriate way to go about this, as I feel this might not be adequately capturing the repeated measures design of the experiment? How would I go about this? How would I have to change the cfg.design? Also, is 'ft_statfun_depsamplesT' the appropriate statistic?
On the other hand, I am not sure there is a way to capture the repeated measures design while doing a permutation test? For instance, I have seen some threads on the mailing about the possibility of testing for interaction effects using a permutation test and about how this is not possible since that is testing for a linear (parametric) model to explain the variance in the dependent variable.
I have read this thread (https://mailman.science.ru.nl/pipermail/fieldtrip/2011-December/017428.html<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fpipermail%2Ffieldtrip%2F2011-December%2F017428.html&data=05%7C02%7Cfieldtrip%40science.ru.nl%7Ce52d6a3df3894485616308dc65e40cf5%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638497277344262717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=CpAnTgWpLpt%2BETComAFvI8SqOLv6Xyn0S6%2FsCtEwo%2Bo%3D&reserved=0>) and though they seem relevant to my analysis, I am not sure how to implement Jan-Mathijs’s suggestions.

2.     When I run the analysis described above, I don’t find any significant clusters (or any clusters at all, although when I run a cluster based analysis without tfce, some clusters are identified, however, they are also not significant). Looking at the visualisation of my data, it doesn’t seem like that should be the case (although, I do understand that visual inspection is not the best estimate of statistical significance of a cluster). Furthermore, when I compare the mean alpha amplitudes of my various conditions for my time window of interest and limited to parieto-occipital channels (chosen based on a priori assumptions from previous studies and papers on this experimental paradigm), I do find significant effects. Again, I suppose this doesn’t have to translate to finding significant clusters? But I am a little confused thinking about the fact that if I had no a priori knowledge to base my analysis on, and I ran a cluster based analysis to find time windows/channels of interest and found no significant clusters, I would probably not have dug any deeper and missed the significant effects we do find when considering alpha amplitude at the parieto-occipital channels?

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=05%7C02%7Cfieldtrip%40science.ru.nl%7Ce52d6a3df3894485616308dc65e40cf5%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638497277344262717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=JpR5TZRYECl4wRK55SNFHvUTqAzPXnXY8y0GPB95Jhc%3D&reserved=0>
https://doi.org/10.1371/journal.pcbi.1002202<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=05%7C02%7Cfieldtrip%40science.ru.nl%7Ce52d6a3df3894485616308dc65e40cf5%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638497277344262717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=JDBRnBDI3WP8nGuL1YrFUe%2FCQb4UCc7MFeAeAbzQjU4%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20240426/a5b71a3f/attachment.htm>