<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Note: this is the second time I post this reply, and the reason is that I forgot to add an appropriate Subject (for findability) to my email (shame on me…(-;)</div>
<div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
</div>
<div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<blockquote type="cite" class="">
<div class="">
<div class="" style="margin: 0px;"><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif; color: rgb(127, 127, 127);"><b class="">From: </b></span><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;">Elisabeth
May <<a href="mailto:elisabethsusanne.may@gmail.com" class="">elisabethsusanne.may@gmail.com</a>><br class="">
</span></div>
<div class="" style="margin: 0px;"><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif; color: rgb(127, 127, 127);"><b class="">Subject: </b></span><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;"><b class="">[FieldTrip]
Question about cluster-based permutation tests on linear mixed models</b><br class="">
</span></div>
<div class="" style="margin: 0px;"><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif; color: rgb(127, 127, 127);"><b class="">Date: </b></span><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;">27
September 2016 at 14:46:55 GMT+2<br class="">
</span></div>
<div class="" style="margin: 0px;"><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif; color: rgb(127, 127, 127);"><b class="">To: </b></span><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;"><<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>><br class="">
</span></div>
<div class="" style="margin: 0px;"><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif; color: rgb(127, 127, 127);"><b class="">Reply-To: </b></span><span class="" style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;">FieldTrip
discussion list <<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>><br class="">
</span></div>
<br class="">
<br class="">
<div dir="ltr" class=""><span class="" style="font-size: 12.8px;">Dear FieldTripers, </span>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">I have a question about the potential use of cluster-based permutation tests for results obtained using linear mixed models. </div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">We are working with data from a 10 min EEG experiment on source level with the aim to quantify the relationship of brain activity in different frequency bands with continous perceptual ratings across 20 subjects in different
experimental conditions. Thus, we have 10 min time courses of brain activity and ratings for each voxel for different conditions and want to test a) if there are significant relationships in the single conditions and b) if these relationships differ between
two conditions. To this end, I have calculated linear mixed models in R using the lme4 toolbox. For both the single condition relationships and the condition contrasts, they result in a single t-value (and a corresponding p-value), which is based on information
on both the single subject and the group level (i.e. we perform a multi-level analysis). However, with more than 2000 voxels, we have a lot of t-values and are wondering if there is a way to apply cluster-based tests to correct for multiple comparisons. </div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">The main problem I see is that I only have one multilevel t-value for the effect across all subjects, i.e. I don't have single subjects values, which I could then e.g. randomize between conditions as normally done in
cluster-based permutation tests. (Or rather, I would be able to extract single subject values but would then loose the advantage of the multi-level analysis.)</div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">I found an old thread in the mailinglist archive where it was suggested to flip the signs of the t-statistic for cluster-level correction (<a href="https://mailman.science.ru.nl/pipermail/fieldtrip/2012-July/005375.html" target="_blank" class="">https://mailman.science.ru.<wbr class="">nl/pipermail/fieldtrip/2012-<wbr class="">July/005375.html</a>).
I understand that, in our case, I would do this randomly for all voxels in each randomization and then build spatial clusters on the resulting (partly flipped) t-values. However, I am not sure if that is a valid approach based on the null hypothesis that there
are no significant relations in my single conditions (a) or no significant relationship differences in my condition contrasts (b). </div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">For the condition contrasts, I would be able to permute the condition labels as normally done in cluster-based permutation tests,I think, but would then have to recalculate the linear mixed models for all voxels in every
permutation. This would result in a very high computational load. </div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
<div class="" style="font-size: 12.8px;">Does anyone have any experience with this kind of analysis? Would the flipping of t-values be a valid approach (and if yes, is there anything to keep in mind in particular)? Can you think of other ways to combine linear
mixed models with a multiple comparison correction on the cluster level?</div>
<div class="" style="font-size: 12.8px;"><br class="">
</div>
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">Hi Elisabeth,
<div class=""><br class="">
</div>
<div class="">I’m not an expert on linear mixed modelling, at least not with respect to the different ways in which they can be used to deal with correlated observations (typically, time series). However, from a theoretical point of view, I do not see how these
models could be combined with permutation-based inference; they are just different statistical frameworks. However, it IS possible to answer your questions ("<span class="" style="white-space: pre-wrap;">we have 10 min time courses of brain activity and ratings
for each voxel for different conditions and wan to test a) if there are significant relationships in the single conditions and b) if these relationships differ between two conditions.”) within the framework of cluster-based permutation tests. Question b) is
the most straightforward because it amounts to a cluster-based permutation test using the depsamplesT statfun applied to the regression coefficients in each of the two conditions. Answering question a) requires that you bin your ratings in a number of categories,
calculate the trial-averaged EEG data for each of the categoreies, and test the difference between them using a cluster-based permutation test using the depsamplesregrT statfun. Both of these approaches have been described previously on this discussion list,
and for the depsamplesregrT statfun (your question a), it was Vladimir Litvak who used it first (actually, I implemented it for him). The approach for question b) is actually a variant on the general approach for testing interactions using cluster-based permutation
tests. </span></div>
<div class=""><span class="" style="white-space: pre-wrap;"><br class="">
</span></div>
<div class=""><span class="" style="white-space: pre-wrap;">Have a look here:</span></div>
<div class=""><span class="" style="white-space: pre-wrap;"><a href="http://www.fieldtriptoolbox.org/faq/how_can_i_test_for_correlations_between_neuronal_data_and_quantitative_stimulus_and_behavioural_variables" class="">http://www.fieldtriptoolbox.org/faq/how_can_i_test_for_correlations_between_neuronal_data_and_quantitative_stimulus_and_behavioural_variables</a></span></div>
<div class=""><span class="" style="white-space: pre-wrap;">and </span></div>
<div class=""><span class="" style="white-space: pre-wrap;"><a href="http://www.fieldtriptoolbox.org/faq/how_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests" class="">http://www.fieldtriptoolbox.org/faq/how_can_i_test_an_interaction_effect_using_cluster-based_permutation_tests</a></span></div>
<div class=""><span class="" style="white-space: pre-wrap;"><br class="">
</span></div>
<div class=""><span class="" style="white-space: pre-wrap;">These tutorials provide all the necessary concepts, although they do not answer your question in a recipe-like fashion.</span></div>
<div class=""><span class="" style="white-space: pre-wrap;"><br class="">
</span></div>
<div class=""><span class="" style="white-space: pre-wrap;">best,</span></div>
<div class=""><span class="" style="white-space: pre-wrap;">Eric Maris</span></div>
</div>
</div>
<div><br class="">
</div>
</div>
</body>
</html>