<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Elif,
<div class=""><br class="">
</div>
<div class="">Cluster-based permutation tests can be done using ft_timelock/freq/sourcestatistics, using ‘montecarlo’ as a method. If you use ‘mvpa’ as a method, you won’t be able to get a multiple comparison correct p-value AND a classification result in a
single shot.</div>
<div class=""><br class="">
</div>
<div class="">What you can do, though, is to obtain some performance metric per subject, using the mvpa functionality, and then come up with a meaningful second-level test that performs statistical inference of the performance metric across subjects, using
the clustering philosophy. </div>
<div class=""><br class="">
</div>
<div class="">Best wishes,</div>
<div class="">Jan-Mathijs</div>
<div class=""><br class="">
</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 23 May 2022, at 16:29, Elif Tülay via fieldtrip <<a href="mailto:fieldtrip@science.ru.nl" class="">fieldtrip@science.ru.nl</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size:12pt;font-family:"Times New Roman",serif" class="">Dear fieldtrip community</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 8pt; line-height: 107%; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class=""><span class=""> </span></span><br class="webkit-block-placeholder">
</div>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">I have a question about how I can perform statistical testing for selected metrics (such as for accuracy). Actually I want to obtain p-values for my accuracy results
via cluster-based permutation test.<span class=""></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">When I check the link below<span class=""></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class=""><a href="https://urldefense.com/v3/__https://github.com/fieldtrip/fieldtrip/blob/master/ft_statistics_mvpa.m__;!!HJOPV4FYYWzcc1jazlU!-mRuDeeFEAj91C2mt6bRmziZJszE2ZSUUxwWZMjrNc18GwC67yM1N3gOeAylbpXClko1JVWMsaJ-YpsA3ul_-5-14ZbjtY_lMkrg1w$" style="color:rgb(5,99,193);text-decoration:underline" class="">https://github.com/fieldtrip/fieldtrip/blob/master/ft_statistics_mvpa.m</a><span class=""></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">I can not see anything about it. But it says<span class=""></span></span></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<i class=""><span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">The function supports cross-validation, searchlight analysis, generalization, nested preprocessing, a variety of classification and regression metrics,
<b class="">as well as statistical testing of these metrics</b></span></i><span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">.<span class=""></span></span></p>
<div style="margin: 0cm 0cm 8pt; line-height: 107%; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class=""><span class=""> </span></span><br class="webkit-block-placeholder">
</div>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">Here is some part of my script;<span class=""></span></span></p>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.method<span class="">
</span>= </span><span style="font-size:12pt;font-family:"Times New Roman",serif;color:rgb(160,32,240)" class="">'mvpa'</span><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">;</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.design<span class="">
</span>= [ones(length(GROUP1),1); 2*ones(length(GROUP2),1)];</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.features<span class="">
</span>= </span><span style="font-size:12pt;font-family:"Times New Roman",serif;color:rgb(160,32,240)" class="">'chan'</span><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">;<span class="">
</span></span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.mvpa<span class="">
</span>= [];</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.mvpa.classifier =
</span><span style="font-size:12pt;font-family:"Times New Roman",serif;color:rgb(160,32,240)" class="">'lda'</span><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">;<span class="">
</span></span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.mvpa.metric<span class="">
</span>= </span><span style="font-size:12pt;font-family:"Times New Roman",serif;color:rgb(160,32,240)" class="">'accuracy'</span><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">;</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 0.0001pt; line-height: normal; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">cfg.mvpa.k<span class="">
</span>= 10;</span><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></div>
<div style="margin: 0cm 0cm 8pt; line-height: 107%; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class=""><span class=""> </span></span><br class="webkit-block-placeholder">
</div>
<p class="MsoNormal" style="margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:"Calibri",sans-serif">
<span class="gmail-pl-c"><span style="font-size:12pt;line-height:107%;font-family:"Times New Roman",serif" class="">stat = ft_freqstatistics (cfg, data1, data2, data3, ...) where the data obtained from ft_freqanalysis for each subject.<span class=""></span></span></span></p>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""> </span></span></pre>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class="">Thank you in advance for your consideration. I am looking forward to hearing from you.<span class=""></span></span></pre>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""> </span></span></pre>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class="">Best regards,<br class=""><br class=""></span></pre>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class="">Elif Tülay<br class=""></span></pre>
<pre style="margin:0cm 0cm 0.0001pt;font-size:10pt;font-family:"Courier New"" class=""><span style="font-size:12pt;font-family:"Times New Roman",serif" class=""><span class=""></span></span></pre>
</div>
_______________________________________________<br class="">
fieldtrip mailing list<br class="">
<a href="https://mailman.science.ru.nl/mailman/listinfo/fieldtrip" class="">https://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br class="">
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-mRuDeeFEAj91C2mt6bRmziZJszE2ZSUUxwWZMjrNc18GwC67yM1N3gOeAylbpXClko1JVWMsaJ-YpsA3ul_-5-14ZbjtY8VExj9YA$
<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>