<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:"Helvetica Neue";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
p.elementtoproof, li.elementtoproof, div.elementtoproof
{mso-style-name:elementtoproof;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Dear Jan-Mathijs,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Thank you for your response. Yes, that makes sense and is consistent with applying a Bonferroni Correction (/2) to the final p-value.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Oliver<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">-------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Oliver Hermann<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">PhD Researcher<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Department of Psychology<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">University of Bath<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> fieldtrip <fieldtrip-bounces@science.ru.nl>
<b>On Behalf Of </b>Schoffelen, J.M. (Jan Mathijs) via fieldtrip<br>
<b>Sent:</b> 13 June 2025 10:19<br>
<b>To:</b> FieldTrip discussion list <fieldtrip@science.ru.nl><br>
<b>Cc:</b> Schoffelen, J.M. (Jan Mathijs) <janmathijs.schoffelen@donders.ru.nl><br>
<b>Subject:</b> Re: [FieldTrip] Evaluation under the permutation distribution positive/negative for two-tailed analysis<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left" width="100%" style="width:100.0%">
<tbody>
<tr>
<td width="0" style="width:.3pt;background:#A6A6A6;padding:5.25pt 1.5pt 5.25pt 1.5pt">
</td>
<td width="100%" style="width:100.0%;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 11.25pt;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important">
<div>
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<span style="font-size:9.0pt;font-family:"Segoe UI",sans-serif;color:#212121">You don't often get email from
<a href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a>. <a href="https://aka.ms/LearnAboutSenderIdentification">
Learn why this is important</a> <o:p></o:p></span></p>
</div>
</td>
<td width="75" style="width:56.25pt;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 3.75pt;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important;align: left !important">
</td>
</tr>
</tbody>
</table>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" style="background:#FFF698;border-collapse:collapse;border:none">
<tbody>
<tr>
<td width="100%" valign="top" style="width:100.0%;border:none;border-left:solid #FEC811 6.0pt;padding:5.65pt 5.65pt 5.65pt 5.65pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><strong><span style="font-family:"Helvetica Neue",serif;color:black">CAUTION:
</span></strong><span style="font-family:"Helvetica Neue",serif;color:black"> This email came from outside of the University. To keep your account safe, only click on links and open attachments if you know the person who sent the email, or you expected to receive
this communication.</span><span style="font-family:"Helvetica Neue",serif"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p> <o:p></o:p></p>
<div>
<p class="MsoNormal">Dear Oliver, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Let me give it a shot: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think the underlying idea of the statement in the referred paper is that the permutation distribution of the test-statistic (in this case a T-statistic) is symmetric around 0. Thus, under the assumption that a complete permutation distribution
can be created (i.e. one can take all possible random splits of the individual observations to one of the conditions, and subsequently compute the T-statistics, followed by thresholding and clustering), the permutation distribution obtained by retaining (for
each permutation) the largest positive cluster-based test statistic, will be exactly mirrored by a permutation distribution of the largest negative cluster-based test statistics. I would assume that this justifies the idea to use the absolute cluster statistics
and test these against its permutation distribution.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">However, in reality these permutation distributions will not necessarily be based on test statistics that ‘behave nicely (i.e. symmetrically) around 0’. For that reason, and this is how it is done in the code, a two-sided (cluster-based)
permutation test actually compares the observed test statistics with 2 permutation distributions, obtained by taking the most extreme (positive and negative) permuted test statistics.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best wishes,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Jan-Mathijs<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On 12 Jun 2025, at 16:08, Oliver Hermann via fieldtrip <<a href="mailto:fieldtrip@science.ru.nl">fieldtrip@science.ru.nl</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi all,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I wonder if anyone can clear up some confusion regarding evaluation of summed-cluster statistics following permutation. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Specifically, as of Maris and Oostenveld (2007), evaluation is performed for all clusters, based on the permutation distribution of the max (absolute) cluster statistic, see "It is important to note that the<span class="apple-converted-space"> </span><i>p</i>-values
for all eight clusters are calculated under the permutation distribution of the maximum (absolute value) cluster-level statistic and not under the permutation distribution of the second largest, third largest, etc."<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">However, when I look at the guide at<span class="apple-converted-space"> </span><a href="https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.fieldtriptoolbox.org%2Ftutorial%2Fstats%2Fcluster_permutation_timelock%2F&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C14fb2dd67cc443c1d01208ddaca9eec6%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638856568050729790%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=mxVdIE7Zs6nbq8HjGWlbkbZqrdwQXKT%2FPOeuykrznYg%3D&reserved=0" originalSrc="https://www.fieldtriptoolbox.org/tutorial/stats/cluster_permutation_timelock/">https://www.fieldtriptoolbox.org/tutorial/stats/cluster_permutation_timelock/</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">the following recommendation is given: And third, the critical value for the cluster-level test statistic (determined by cfg.alpha; see further) is now two-sided: negative cluster-level statistics must be compared with the negative critical
value, and positive cluster-level statistics must be compared with the positive critical value.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">To me, these two statements seem contradictory. Please could someone clarify the correct approach (I assume the latter) and why the discrepancy?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Oliver<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div id="Signature">
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="elementtoproof" style="margin:0cm"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">-------------------<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">Oliver Hermann<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">PhD Researcher<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">Department of Psychology<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">University of Bath<o:p></o:p></span></p>
</div>
<p class="elementtoproof" style="margin:0cm"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">_______________________________________________<br>
fieldtrip mailing list<br>
</span><a href="https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C14fb2dd67cc443c1d01208ddaca9eec6%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638856568050760344%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=uEhCyGSOC6gsILBgw7mSSPejiMTjSVaIevqyRWoN0Zs%3D&reserved=0" originalSrc="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</span></a><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif"><br>
</span><a href="https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C14fb2dd67cc443c1d01208ddaca9eec6%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638856568050778718%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ss744dIKk%2BKrGdaAen%2BeTjPZLnp4809swdmSYN4oOwM%3D&reserved=0" originalSrc="https://doi.org/10.1371/journal.pcbi.1002202"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif">https://doi.org/10.1371/journal.pcbi.1002202</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>