[FieldTrip] classification group-level stats?

Irina Simanova irina.simanova at mpi.nl
Mon Dec 10 10:30:39 CET 2012


Dear Akiko,

I think the t-test against 0.5 is still valid. The null hypothesis there will be that the classification accuracy across subjects has the mean of 0.5. If there is no class information in the data, you should see  the accuracy of 0.44 as often as 0.56. Then the null hypothesis is not rejected. Many fMRI searchlight studies use this logic.
Alternatively, you can use a binomial test. You first apply the individual-subject threshold (say, p less than 0.05), and then count in how many subjects out of your sample the classification is significant (say, 8 out of 10). You then test this number with the binomial test. So you compute the probability of this amount of "coin flips" to return 1, given that the probability of such an event is 0.05. 

Hope this helps, 
Best, 
Irina


Op 09.12.2012 om 23:48 heeft Akiko Ikkai <akiko.ikkai at gmail.com> het volgende geschreven:

> Dear FieldTrip users,
> 
> I'm running multivariate classification on my EEG data (subset of electrodes), following this page (http://fieldtrip.fcdonders.nl/tutorial/multivariateanalysis?s[]=classification). I have a question regarding a group-level stats.
> 
> I have classification results (accuracy and binomial p) for each of 16 subs' selected sensors (averaged across time and freq). I don't think I can simply run one-sample t-test on accuracy against .5 (chance) at group level, since at individual level, accuracy of .56 (for example) could be associated with p of .3 (for example), depending on number of trials (across subjects, number of trials that goes into SVM is more or less constant). This could be significantly higher than chance (.5) if I simply run one-sample t-test across subjects with just accuracy. 
> 
> Does anyone have suggestions on how to run group stats (comparing with chance-level) considering these factors? 
> 
> Thanks in advance! Akiko
> 
> -- 
> Akiko Ikkai, Ph.D. 
> Postdoctoral Fellow
> Department of Psychological and Brain Sciences
> Johns Hopkins University
> Ames Hall, 3400 N. Charles St.
> Baltimore, MD 21218
> 
> 
> _______________________________________________
> fieldtrip mailing list
> fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20121210/959a386a/attachment-0002.html>


More information about the fieldtrip mailing list