[FieldTrip] Cluster based permutation test interpretation

[Yvonne Visser]

Dear Eric,
Thank you for your extensive reply, we will take it into account for the interpretation of our results!
Best,
Yvonne
________________________________
From: fieldtrip <fieldtrip-bounces at science.ru.nl> on behalf of Maris, E.G.G. (Eric) <e.maris at donders.ru.nl>
Sent: Wednesday, December 5, 2018 2:40 PM
To: fieldtrip at science.ru.nl
Subject: Re: [FieldTrip] Cluster based permutation test interpretation

Dear Yvonne,

I advise you to make a distinction between statistical inference and interpretation. Statistical inference pertains to a particular null hypothesis, which is either rejected or not. Interpretation, on the other hand, pertains to the likely cause of a rejection (e.g., alpha-band power modulations in the -0.75 to 0 interval). The p-value pertains to statistical inference but not to the interpretation. To make a case for a particular interpretation you are free to use all arguments at your disposal. One of these arguments is the fact that the largest cluster in in the alpha-band in the —0.75 to 0 interval. But you can also refer to published results. For instance, the alpha-band is one of the two bands of the sensorimotor rhythm, whose spatio-spectral structure has been described in numerous papers (among which several ones to which I contributed as a senior author), and whose amplitude is inversely related to movement and reaction time. In your case, I would look in the post-stimulation period to characterise the sensorimotor rhythm in space and frequency. I have advised this procedure in this paper: https://doi.org/10.1111/j.1469-8986.2011.01320.x.

In general, I think that we should rely less on p-values for localisation in space, frequency and time. Instead, we should use the spatio-spectro-temporal extent of the largest clusters to provide a mechanistic account of the effect. For that, it is necessary to relate your results to the published findings in the literature. I have argued for this approach in this paper: https://www.sciencedirect.com/science/article/pii/S1364661316300481

best,
Eric


From: Yvonne Visser <y.visser at hotmail.com<mailto:y.visser at hotmail.com>>
Subject: [FieldTrip] Cluster based permutation test interpretation
Date: 30 November 2018 at 10:47:03 CET
To: "fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>" <fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>>
Cc: "aaron.schurger at gmail.com<mailto:aaron.schurger at gmail.com>" <aaron.schurger at gmail.com<mailto:aaron.schurger at gmail.com>>


Dear all,

Thank you for welcoming me to the discussion list, my name is Yvonne Visser and I currently work as a research assistant with dr. Aaron Schurger at Neurospin. During my masters program I learned about cluster based permutation tests for electrophysiological data and distinctly remember how from this type of test one can not conclude that a particular cluster is significant (in line with what is said on the fieldtrip website here, http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test)<http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test>

We are currently using the cluster based permutation test in the analysis of our experiment, but we are a bit confused on
how to interpret the results from our test.
To give you a short introduction to our experiment: we are looking for a relationship between a behavioural variable and our collected EEG data. So we computed the grand average time frequency spectrum in a single channel of the time bins of interest. Then, we correlated each time/frequency point in this 2d matrix with the behavioural variable in that trial. This resulted in a correlation matrix like you can see in attachment1_correlationmatrix. As you can see, we also computed clusters of time/frequency points with p<0.05. After computing the permutations, we found that the biggest "real" cluster is bigger than any of the permuted clusters.

Now, we would like to conclude something from this result about which frequency band at what time is correlated to our behavioural variable. We found a fieldtrip function called ft_clusterplot that does seem to suggest that you can highlight a specific cluster it if it survives the test, but isn't that exactly what my lectures and the webpage say we should not do? Can we say that activity in the alpha band around -0.75 to 0 (where the biggest cluster is located) is correlated to the size of the movement? Or should we not conclude something about which cluster is significant and can we only say that some time frequency power is correlated to our behavioural variable? If the second is true, do you have any advice for us to make the interpretation more specific?

Thank you so much in advance, and please let us know if anything is unclear.
Kind regards,
Yvonne & Aaron.

<attachment1_correlationmatrix.jpg>

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://doi.org/10.1371/journal.pcbi.1002202

Eric Maris | Donders Institute for Brain, Cognition, and Behaviour & Faculty of Social Sciences | Radboud University | PO Box 9104, 6500 HE Nijmegen | (024) 3612651 | www.ru.nl<http://www.ru.nl>

This message (and any attachments) is intended solely for the addressee(s) and may contain confidential information. If you are not the addressee, do not copy this message (and any attachments), forward or share this message with third parties. You are requested to notify the sender immediately and delete this message.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20181207/5cab8472/attachment-0002.html>