<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_1_1427355858287_8468">Hi David,</div><div id="yui_3_16_0_1_1427355858287_8469"><br></div><div id="yui_3_16_0_1_1427355858287_8470" dir="ltr">Thank so much for the reference and all the useful details, indeed very helpful :)<br></div><div id="yui_3_16_0_1_1427355858287_8471" dir="ltr"><br></div><div id="yui_3_16_0_1_1427355858287_8472" dir="ltr">Best,</div><div id="yui_3_16_0_1_1427355858287_8474" dir="ltr">Martina<br></div><div id="yui_3_16_0_1_1427355858287_4350"><span></span></div> <br><div class="qtdSeparateBR"><br><br></div><div style="display: block;" class="yahoo_quoted"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"> <font face="Arial" size="2"> Il Lunedì 23 Marzo 2015 5:03, David Groppe <david.m.groppe@gmail.com> ha scritto:<br> </font> </div> <br><br> <div class="y_msg_container"><div id="yiv1464813987"><div><div dir="ltr"><div><div>Hi Martina,<br clear="none"></div> Balanced sample sizes are typically recommended for conventional parametric independent samples tests (e.g., t-tests, ANOVAs) because it makes the tests less sensitive to differences in variation between the populations being compared. If the populations being compared differ in variance, having more observations from the population with less variability will make these tests overly permissive (i.e., the true false positive rate will be greater than your nominal alpha level). If you have more observations from the population with greater variability, the tests become overly conservative.<br clear="none"></div><div> A few years ago, my colleagues and I simulated some EEG data and found that permutation tests exhibit a qualitatively similar sensitivity to differences in variance between populations (see below). If you're concerned about such a difference in your data you could do as has already been suggested and use a subset of data so that the number of observations between samples is the same. Alternatively you could use a permutation test based on variants of the t-statistic that are less sensitive to differences in variance. In our paper below, we investigated two variants, Welch's t and t_dif. Welch's t proved a bit less sensitive to differences in variance and was only slightly less powerful than the conventional t-statistic. t_dif was markedly insensitive to differences in variance but was significantly less powerful. However, I would guess that using t_dif or Welch's t are likely more powerful than discarding trials (though we didn't investigate that option in the paper).<br clear="none"></div><div> cheers,<br clear="none"></div><div> -David<br clear="none"><br clear="none"><span style="">Groppe, D.M., Urbach, T.P., & Kutas,
M. (2011) <i style="">Mass univariate analysis
of event-related brain potentials/fields II: Simulation studies</i>. <b style="">Psychophysiology</b>, 48(12) pp.
1726-1737, DOI: 10.1111/j.1469-8986.2011.01272.x.<br clear="none"><a rel="nofollow" shape="rect" target="_blank" href="http://www.cogsci.ucsd.edu/~dgroppe/PUBLICATIONS/mass_uni_preprint2.pdf">www.cogsci.ucsd.edu/~dgroppe/PUBLICATIONS/mass_uni_preprint2.pdf</a><br clear="none"></span><br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none"></div></div><div class="yiv1464813987gmail_extra"><br clear="none"><div class="yiv1464813987gmail_quote">On Thu, Mar 19, 2015 at 4:06 AM, Martina Rossi <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:martina.rossi76@yahoo.it" target="_blank" href="mailto:martina.rossi76@yahoo.it">martina.rossi76@yahoo.it</a>></span> wrote:<br clear="none"><blockquote class="yiv1464813987gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="yiv1464813987yqt8747217792" id="yiv1464813987yqt69829"><div><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><div dir="ltr"><span>Dear Stephen and Joram,</span></div><div dir="ltr"><br clear="none"><span></span></div><div dir="ltr"><span>Thank so much for your feedback,</span></div><div dir="ltr"><span>I will check out the suggested material,</span></div><div dir="ltr"><br clear="none"><span></span></div><div dir="ltr"><span>Best,</span></div><div dir="ltr"><span>Martina</span></div> <div><br clear="none"><br clear="none"></div><div style="display:block;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div dir="ltr"> <font face="Arial"> Il Mercoledì 18 Marzo 2015 20:43, Stephen Whitmarsh <<a rel="nofollow" shape="rect" ymailto="mailto:stephen.whitmarsh@gmail.com" target="_blank" href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>> ha scritto:<br clear="none"> </font> </div><div><div class="yiv1464813987h5"> <br clear="none"><br clear="none"> <div><div><div><div dir="ltr"><div>You can also check out this video of Robert. Apologies for the quality - not of the talk, but of the recording :-)<br clear="none"></div><div><br clear="none">At 14:45 he actually mentions unequal number of trials between conditions.<br clear="none"></div><div><br clear="none"><a rel="nofollow" shape="rect" target="_blank" href="https://www.youtube.com/watch?v=vOSfabsDUNg">https://www.youtube.com/watch?v=vOSfabsDUNg</a><br clear="none"><br clear="none"></div><div>Cheers,<br clear="none"></div><div>Stephen<br clear="none"></div><br clear="none"><br clear="none"><br clear="none"><div><div><div><br clear="none"><div><div>On 18 March 2015 at 19:33, Stephen Whitmarsh <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:stephen.whitmarsh@gmail.com" target="_blank" href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div dir="ltr"><div><div>I should add that (1) is typically done <i>within </i>subjects, and (2) <i>over </i>subjects.<br clear="none"><br clear="none"></div>Cheers,<br clear="none"></div>Stephen <br clear="none"></div><div><div><div><br clear="none"><div>On 18 March 2015 at 19:20, Stephen Whitmarsh <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:stephen.whitmarsh@gmail.com" target="_blank" href="mailto:stephen.whitmarsh@gmail.com">stephen.whitmarsh@gmail.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div dir="ltr"><div><div>Dear Martina,<br clear="none"><br clear="none"></div>It might help to distinguish two aspects of cluster-based statistic.<br clear="none"></div><div><br clear="none">1) the statistical approuch that you will use to determine whether a time-channel-datapoint / time-frequency-channel-datapoint / time-frequency-voxel-datapoint is considered significant different between conditions.<br clear="none"></div><div>2) the statistical approuch that you will use to determine whether <b>a cluster of</b> time-channel-datapoints / time-frequency-channel-datapoints / time-frequency-voxel-datapoints is considered significantly different between conditions.</div><div><br clear="none"></div><div>When you talk about <b>cluster statistics</b>, you probably think about the second part. But this might not be what you should initially be concerned with when thinking about e.g. different numbers of trials between conditions. Rather, consider what statistical tests you (can) use to compare your time-frequency values between conditions <i>(within subjects).</i> This can be, e.g., a t-test, a nonparametric (e.g. montecarlo) test, or any test, for that matter. As far as my limited knowledge of statistics goes, in most simple and non-extreme cases, unequal number of trials that does not have to increase your chance of type I errors, rather that of type 2 (you'll be insensitive to differences if you don't have enough observations in one condition due to noisy estimate of means/distribution). But in any case it's a simple question to google or ask a statistician.<br clear="none"><br clear="none"></div><div>Now, after you are happy with and confident about the between conditions statistical test, consider how the cluster statistics might help you. <br clear="none">First of all, how does it determine whether a cluster is significantly different between conditions? There are different options, but the gist is that it takes your significant statistical numbers of step (1), adds them up when they belong to the same cluster (based on whether they are neigbourings in time/freq/space with other significant numbers), takes the maximum of these summed up clusters (there might be more than one cluster), and then compares this one value to the same taken from a<b>(non-parametric) monte-carlo distribution </b>of the null hypothesis based on permuting the values over conditions (and then calculating the maximum sum). The Maris and Oostenveld paper explains it in more detail. <br clear="none"><br clear="none">The reason for doing cluster-statistics is that its a smart way of dealing with multiple comparisons over many time x frequency x channels (or space). The method is blind for your decisions about how its computed for each point in time x frequency x channels (or space). <br clear="none"><br clear="none">I find the FieldTrip statistics functions, their configurations etc., and the way they interact confusing at times, but I hope this helps to clear it up a bit. <br clear="none"><br clear="none"></div><div>Long story short - I think your question does not limit itself to cluster statistics and at the same time is much simpler. It's all about (1). <br clear="none"><br clear="none"></div><div>Best wishes,<br clear="none"></div><div>Stephen<br clear="none"></div><div><br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none"><br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none"><br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none">There are two separate steps cluster statistics (as implemented in FieldTrip, but in general as well).<br clear="none"><br clear="none"><br clear="none"></div><div><br clear="none"></div></div><div><div><div><br clear="none"><div>On 18 March 2015 at 14:51, Joram van Driel <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:joramvandriel@gmail.com" target="_blank" href="mailto:joramvandriel@gmail.com">joramvandriel@gmail.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div dir="ltr">Hi Martina,<div><br clear="none"></div><div>In general, I'd advice to do some kind of trial-selection procedure when comparing error versus correct trials, in order to trial-count-match the two conditions. Otherwise you run into problems considering: SNR (higher for the correct condition), and RT (errors are usually faster, resulting in a time-on-task confound). What I always do is pick from the correct condition a similar number of trials that are close to the RT distribution of the error trials (i.e. the faster correct trials). That way you solve both problems at once (and probably the cluster-based permutation test in field trip will work as well, as a bonus ;)).</div><div><br clear="none"></div><div>Best,</div><div>Joram</div></div><div><br clear="none"><div><div><div>On Wed, Mar 18, 2015 at 2:31 PM, Martina Rossi <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:martina.rossi76@yahoo.it" target="_blank" href="mailto:martina.rossi76@yahoo.it">martina.rossi76@yahoo.it</a>></span> wrote:<br clear="none"></div></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div><div><div><div style="color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><div><div style="color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><div dir="ltr">Dear All,</div><div dir="ltr"><br clear="none"></div><div dir="ltr">I would like to get some feedback from the community about a statistical analysis problem I need to tackle with my study.</div><div dir="ltr">I want to apply the cluster-based permutation tests on time-frequency data considering two conditions (correct vs error).</div><div dir="ltr">Unfortunately, these two conditions have different sizes (correct >> error).</div><div dir="ltr">Right
now, I am only considering subjects having a ratio "error/correct"
bigger than 1/5, yet this is only an arbitrary threshold I set.</div><div dir="ltr">The question is the following:</div><div dir="ltr">is
there a formal way to identify a threshold by which two conditions can
be realiably compared with the cluster-based permutation tests?</div><div dir="ltr">If the cluster-based approach is not suitable in this scenario, is there any other approach you would suggest?</div><div dir="ltr">I shall perhaps point out that I am working on EEG data recorded with a 32 channel system (impedance levels < 10 k<span>Ω).</span></div><div dir="ltr"><br clear="none"></div><div dir="ltr">Looking forward to hear your feedback :)</div><div dir="ltr"><br clear="none"></div><div dir="ltr">Kind Regards,</div>Martina Rossi<div dir="ltr"><br clear="none"></div></div></div></div></div><br clear="none"></div></div>_______________________________________________<br clear="none">
fieldtrip mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:fieldtrip@donders.ru.nl" target="_blank" href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><span><font color="#888888"><br clear="none"></font></span></blockquote></div><span><font color="#888888"><br clear="none"><br clear="all"></font></span><div><br clear="none"></div>-- <br clear="none"><div><div dir="ltr"><div><div dir="ltr">Joram van Driel<div>Postdoc @ Vrije Universiteit Amsterdam</div><div>Cognitive Psychology</div><div><br clear="none"></div></div></div></div></div>
</div>
<br clear="none">_______________________________________________<br clear="none">
fieldtrip mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:fieldtrip@donders.ru.nl" target="_blank" href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br clear="none"></blockquote></div><br clear="none"></div>
</div></div></blockquote></div><br clear="none"></div>
</div></div></blockquote></div><br clear="none"></div></div></div></div></div></div></div><br clear="none"><br clear="none"></div> </div></div></div> </div> </div> </div></div></div><br clear="none">_______________________________________________<br clear="none">
fieldtrip mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:fieldtrip@donders.ru.nl" target="_blank" href="mailto:fieldtrip@donders.ru.nl">fieldtrip@donders.ru.nl</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://mailman.science.ru.nl/mailman/listinfo/fieldtrip">http://mailman.science.ru.nl/mailman/listinfo/fieldtrip</a><br clear="none"></blockquote></div><br clear="none"></div></div></div><br><br></div> </div> </div> </div></div></body></html>