<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Martin,<div class=""><br class=""></div><div class="">Let me CC this to the FieldTrip mailing list, since people there might want to chip in.</div><div class=""><br class=""></div><div class="">I am not aware of a way to calculate sample sizes for a non-parametric test in general, although for specific tests there are estimates (see e.g. <a href="http://www.stat.purdue.edu/~bacraig/notes582/noether.pdf" class="">http://www.stat.purdue.edu/~bacraig/notes582/noether.pdf</a>). The challenge with the cluster-based permutation test is that the effect is not a-priori specified, and hence the effect size is not quantified. If you don’t know the effect size, you cannot estimate the likelihood of observing that effect (given the number of subjects), or number of subjects (given the desired likelihood of observing the effect). Actually, to be more precise: with the non-parametric permutation test you are not even making an inference about a specific effect, but you are making an inference on the data being exchangeable in general (see also this <a href="http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test/" class="">FAQ</a> on the FieldTrip website). </div><div class=""><br class=""></div><div class="">If you do know the effect (as you seem to do), then a non-parametric test might actually not be the most optimal. Your follow up study would have a better power if you were not to ask the question "is there a difference in the distribution of the data in general” (i.e. H0: there is no difference, versus H1 there is a difference), but rather “is there a difference in the channel-time-frequency range (ie.. cluster) that I previously identified”. In the 2nd case it might be possible to average over the region of interest. Still, you may have the issue that the distribution of the statistic-of-interest is not according to a known distribution (or cannot be assumed to have normally distributed error terms), so a non-parametric test might still be in place. </div><div class=""><br class=""></div><div class="">Pragmatically I think that you can indeed use a resampling strategy to estimate the required sample size from previously acquired data, asssuming that your previous study is sufficiently overpowered (and of course assuming that your previous study is not a false alarm, but a true rejection of the H0). Another approach (although not 100% addressing your question) is to specify in the new study design to specify that a certain number of subjects are going to be set aside to optimize the analysis pipeline and determine the effect of interest, after which that optimized pipeline will be used to evaluate the effect in all other subjects. However, that does not allow a-priori specification of the number of subjects; it just replaces the effect size estimate from the previous study with an effect size estimate from the pilot group.</div><div class=""><br class=""></div><div class="">Many studies actually benefit from having specific hypotheses amd being able to specifying the effect of interest prior to testing that effect. E.g. that is also what is done <a href="http://www.fieldtriptoolbox.org/tutorial/beamformer/#preprocessing" class="">here</a>. In general, initiating new neuroscience studies again-and-again as if we don’t know anything about the data is simply not efficient. So please do use your existing data in the analysis plan of your new study.</div><div class=""><br class=""></div><div class="">best</div><div class="">Robert</div><div class=""> <br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 15 Feb 2019, at 01:22, Gallagher, Martin J <<a href="mailto:martin.j.gallagher@vumc.org" class="">martin.j.gallagher@vumc.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hello Dr. Oostenveld,<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thank you for developing the nonparametric permutation tests in Fieldtrip and your reviews of the subjects.<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">For a grant, my study section is asking for sample size calculations. Is there a way to estimate this? I know it’s probably simple-minded, but could I use permutation tests I’ve run in the past and keep scaling back the number of trials until I get to a N value that still has a p value < 0.025?<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thank you for considering!<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">_______________________________<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Martin J. Gallagher MD, PhD<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Associate Professor<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Dept. of Neurology, Epilepsy Division<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Vanderbilt University School of Medicine<o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 10pt; font-family: Georgia, serif;" class="">The information contained in this message and any attachments is intended solely for the individual or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this email in error, please contact the sender and delete the material from your system.</span></div></div></div></blockquote></div><br class=""></div></body></html>