[FieldTrip] Statistics question alpha cutoff adjustment

[Paul Steinfath]

Dear Jan-Mathijs,

Thanks a lot for the answer, I appreciate the confirmation.

Currently I am only working with EEG data, so I think the ft_combineplanar does not apply there.
I will hopefully come back to it in the future if I get the chance to work with MEG data :)

Thank you and all the best,
Paul

----- Ursprüngliche Mail -----
> Von: "Schoffelen, J.M. (Jan Mathijs) via fieldtrip" <fieldtrip at science.ru.nl>
> An: "FieldTrip discussion list" <fieldtrip at science.ru.nl>
> CC: "Schoffelen, J.M. (Jan Mathijs)" <janmathijs.schoffelen at donders.ru.nl>
> Gesendet: Donnerstag, 8. August 2024 09:24:23
> Betreff: Re: [FieldTrip] Statistics question alpha cutoff adjustment

> Hi Paul,
> 
> You asked:
> 
>> Should my alpha cutoff in this final comparison of the real sum(t) HEP effect
>> against surrogate sum(t) effects stay at 5%, or should I adjust it because I
>> test both, positive and negative directions?
> 
> The answer: yes, you should adjust the alpha level for a two-sided test.
> 
> The fact that a positive/negative going effect may have the same underlying
> dipolar source is physiological knowledge/assumption, which the statistical
> machinery does not know/care about.
> If you want to optimally pool information across sensors (assuming you are
> working with MEG magnetometer data (or axial gradiometers) so that you can
> optimally leverage the spatial clustering heuristic, you may consider to do a
> synthetic planar gradient transform (followed by ft_combineplanar).
> 
> Best wishes,
> Jan-Mathijs
> 
> 
>> On 6 Aug 2024, at 07:37, Paul Steinfath via fieldtrip <fieldtrip at science.ru.nl>
>> wrote:
>>
>> Dear Fieldtrippers,
>>
>> I have a question regarding multiple comparison correction in a case of
>> permutation based statistics.
>> In the field of Heartbeat Evoked Potential (HEP) research, it is a common
>> practice to perform a “surrogate heartbeat analysis” to assess if effects are
>> truly locked to the heartbeat, or merely a result of fluctuations in ongoing
>> brain activity.
>>
>> The procedure can work like this:
>> 1. Compare HEPs between condition A and condition B using cluster based
>> permutation t-test and identify any significant clusters.
>>
>> 2. Create surrogate data by shuffling the HEP onset triggers per condition and
>> repeating the cluster based permutation test >100 times.
>> Each time, keeping the sum(t) value of the largest clusters.
>>
>> 3. Compare the sum(t) value of the original comparison using real HEPs to the
>> distribution of maximum sum(t) values obtained from surrogate data.
>>
>> For reference, here is an example study describing the approach:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F30738205%2F&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C4356c9c7bd4f455c9c9008dcb77b204b%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638586986652184682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=5JhmZcNU3Zee1tZrDZzE07DfKjYka8NPJwxhdmVNu2U%3D&reserved=0
>>
>> Now, let's say I initially find two HEP effects, one in positive and one in
>> negative direction (could be two directions of the same dipole).
>> I would compare the real positive sum(t) against the surrogate positive max
>> sum(t) and vice versa for the negative direction.
>>
>> Should my alpha cutoff in this final comparison of the real sum(t) HEP effect
>> against surrogate sum(t) effects stay at 5%, or should I adjust it because I
>> test both, positive and negative directions?
>>
>> I am getting a bit confused in this multi-level statistic approach…
>>
>> Thank you very much and all the best
>> Paul
>>
>> _______________________________________________
>> fieldtrip mailing list
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C4356c9c7bd4f455c9c9008dcb77b204b%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638586986652184682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=wts9TIBygPETlD%2BNWjlR6nfp4wwJbaF8s2qeu9%2Bubqc%3D&reserved=0
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C4356c9c7bd4f455c9c9008dcb77b204b%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638586986652184682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=vjQm0O7dhUvU2jUGe270amRvphJRDHtu4ItcqwRRmT8%3D&reserved=0
> 
> 
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202