<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.formatvorlagearial10ptvor5ptnach5pt, li.formatvorlagearial10ptvor5ptnach5pt, div.formatvorlagearial10ptvor5ptnach5pt
{mso-style-name:formatvorlagearial10ptvor5ptnach5pt;
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Arial","sans-serif";}
p.formatvorlagearial10ptvor5ptnach5pt1, li.formatvorlagearial10ptvor5ptnach5pt1, div.formatvorlagearial10ptvor5ptnach5pt1
{mso-style-name:formatvorlagearial10ptvor5ptnach5pt1;
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Arial","sans-serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Arial","sans-serif";
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Verdana","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=NL link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Hi Nina,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I have a question concerning cluster based permutation statistics.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>It is more a basic question on the difference between one- and two-sided testings. I recall from simple t-tests that as an approach to your two-sided p-value you can multiply your one-sided p-value by 2. I hope this is correct? And I am assuming this to also hold in the other direction, thus, approximating your one-sided p-value by dividing your two-sided p-value by 2?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>I propose that you calculate your p-values always one-sided. In fact, this is what the FT permutation statistics functions also do. The difference between a one- and a two-sided test is that you compare this one-sided p-value either with your desired type-I error level (for a one-sided test) or with half your desired type-I error level (for a two-sided test). <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>That is why I am expecting something similar when calculating my statistics with a cluster based permutation approach.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>So what I am actually doing is comparing two conditions at one sensor, comparing time-frequency data. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>When I calculate a two-sided dependent samples t-test <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>(cfg = [];<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.channel = 'all'; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.latency = [0.5 1.5]; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.avgoverchan = 'no';<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.avgovertime = 'no';<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.parameter = 'powspctrm'; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.method = 'montecarlo'; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.statistic = 'depsamplesT';<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.correctm = 'cluster';<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.clusteralpha = 0.05;<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.clusterstatistic = 'maxsum'; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.tail = 0; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.clustertail = 0; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.alpha = 0.025;<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.numrandomization = 1000; <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>, I find one positive cluster with a p-value of 0.036, thus not significant (see fig 1).<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>When I then change my settings to a one-sided test<o:p></o:p></span></p><p class=MsoNormal><span lang=FR style='font-size:10.0pt;font-family:"Arial","sans-serif"'>(cfg.tail = 1; <o:p></o:p></span></p><p class=MsoNormal><span lang=FR style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.clustertail = 1; <o:p></o:p></span></p><p class=MsoNormal><span lang=FR style='font-size:10.0pt;font-family:"Arial","sans-serif"'>cfg.alpha = 0.05;)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>, the positive cluster gets a p-value of 0.056, again not significant (see fig 2). <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>I think that the difference between 0.036 and 0.056 is due to the fact that these are random quantities. If expect that, if you would increase cfg.numrandomization to 100,000, you would find two p-values that are much closer. In any case, the p-values are one-sided, and their calculation is independent of the value that you choose for cfg.alpha.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Best,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Eric Maris<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>With the assumption described above, however, I would expect this to become significant when using a one-sided test. Is my assumption correct? If not, could anybody comment on what is wrong about my assumption?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I would be very grateful for any advice!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Kind regards and thank you very much in advance! <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Nina<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=EN-GB>- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=EN-GB><o:p> </o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=EN-GB>Nina Kahlbrock<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=EN-GB>Institute of Clinical Neuroscience and Medical Psychology <o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>Heinrich Heine University Duesseldorf<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>Universitaetsstr. 1<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>40225 Düsseldorf<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE><o:p> </o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>Tel.: +49 211 81 18075<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>Fax. .: +49 211 81 19916<o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE><o:p> </o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=DE>Mail: </span><span lang=FR><a href="mailto:Nina.Kahlbrock@med.uni-duesseldorf.de"><span lang=DE style='color:windowtext;text-decoration:none'>Nina.Kahlbrock@med.uni-duesseldorf.de</span></a></span><span lang=DE><o:p></o:p></span></p><p class=formatvorlagearial10ptvor5ptnach5pt><span lang=FR><a href="http://www.uniklinik-duesseldorf.de/medpsychologie"><span lang=DE style='color:windowtext;text-decoration:none'>http://www.uniklinik-duesseldorf.de/medpsychologie</span></a></span><span lang=DE><o:p></o:p></span></p><p class=MsoNormal><span lang=DE><o:p> </o:p></span></p></div></div></body></html>