[FieldTrip] Query regarding stat.prob and permutation tests

[cxd425 at student.bham.ac.uk]

Dear Jan,

Thank you for the response, this is helpful.

May I ask then, if cfg.method=‘montecarlo’ and cfg.computeprob=True the p-values, are much lower in stat.prob compared to when cfg.method=‘montecarlo’ and cfg.computeprob=False – they are mostly 1s when computeprob is False. I’m not referring to the cluster p-values (2nd level), but more so the sample (1st) level p-values.

Thank you in advance for the help,
Cihan

________________________________
From: fieldtrip <fieldtrip-bounces at science.ru.nl> on behalf of Schoffelen, J.M. (Jan Mathijs) via fieldtrip <fieldtrip at science.ru.nl>
Sent: Wednesday, August 6, 2025 12:26 pm
To: FieldTrip discussion list <fieldtrip at science.ru.nl>
Cc: Schoffelen, J.M. (Jan Mathijs) <janmathijs.schoffelen at donders.ru.nl>
Subject: Re: [FieldTrip] Query regarding stat.prob and permutation tests

Hi Cihan,

cfg.computeprob is an option that is used by (some of) the ’statfuns’, to convert a test-statistic with a known shape into a p-value, based on the shape of the parameter, (e.g. a T or F statistic).

The computational algorithm with which the p-value is computed (+returned in the output of a high level function call to ft_timelock/freq/sourcestatistics) depends on a combination of the contents of cfg.method (indeed if you would use cfg.method=‘montecarlo’, then the returned p-values are based on a permutation test, if you would  use cfg.method=‘analytic’ the p-values will be based on the known distributions), and cfg.correctm.

I  hope that this helps,

best wishes,
Jan-Mathijs


On 31 Jul 2025, at 17:03, cxd425--- via fieldtrip <fieldtrip at science.ru.nl> wrote:

Hi FieldTrip community,

I hope you’re doing well.

I was just curious how the p-values are computed at the first level - I couldn’t find any documentation explaining the behaviour of stat.prob when cfg.computeprob is set to true or false. When I set computeprob = false, the resulting stat object still contains p-values. Could it be that when cfg.computeprob = true, the p-values at the first level are computed using a permutation test, whereas when it’s set to false, a simpler non-permutation statistical test (e.g., a two-sample t-test) is used instead?

Any clarification would be greatly appreciated.

Best regards,
Cihan

_______________________________________________
fieldtrip mailing list
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.science.ru.nl%2Fmailman%2Flistinfo%2Ffieldtrip&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C0079f1328ab346ca8f9d08ddd659008b%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638902399949432080%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=uvTskqyTb1jNYFoOMuOv9B%2B9A2o4t%2FKbLoC2cpP3fjc%3D&reserved=0
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202&data=05%7C02%7Cfieldtrip%40science.ru.nl%7C0079f1328ab346ca8f9d08ddd659008b%7C084578d9400d4a5aa7c7e76ca47af400%7C1%7C0%7C638902399949443487%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kgFGnO356dOAc3B44RQAA4XuxVsa5UwGRmD%2BKwnEgew%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20250808/0b888c97/attachment.htm>