Problem with data from BESA

[Eric Maris]

Hi Michael,


> thank you for the explanation, things are much clearer now. In the
> meantime I have encountered another problem with clusterrandanalysis:
> When using maxsum as a test statistic everything works fine, but using
> maxsumtminclustersize with the same specified (maximum) alpha the fields
> clusrand.posclusters and clusrand.negclusters stay empty - although there
> seem to be large enough clusters in posclusterslabelmat and
> negclusterslabelmat (I used cfg.smallestcluster=2..) . Is this a bug or
> does the use of  maxsumtminclustersize somehow reduce sensitivity (- from
> the description in the 2005 tutorial I thought it is similar to using FDR
> or 'Holmes' method for computing critical p values?). Maybe I am also
> missing something on a conceptual level that makes the information in
> posclusters invalid if I use maxsumtminclustersize as a test statistic ??


Yes, there is a bug in Clusterrandanalysis when the option
cfg.clusterteststat = 'maxsumtminclustersize' is used. I know where it is,
but I need some time to fix it (due to dependencies in the code). Give me a
week to fix it.


greetings,

Eric



> Below I pasted the code that produced this behaviour.
>
> %Clusterrandomization analysis
> cfg=[];
> cfg.elec  =elec;
> cfg.statistic = 'depsamplesT';
> cfg.alphathresh = 0.05;
> cfg.makeclusters = 'yes';
> cfg.minnbchan = 1; %1 neighbour i.e. 2 channels
> cfg.smallestcluster = 2;
> cfg.clusterteststat = 'maxsumtminclustersize'; % replace with maxsum to
> get lots of entries in clusrand.posclusters
> cfg.onetwo = 'twosided';
> cfg.alpha = 0.05;
> cfg.nranddraws = 1000;
> cfg.latency = [0.40 0.65];
> [clusrand] = clusterrandanalysis(cfg, AMdata, vAMdata);
> clusrand.elec=elec;
>
>
> Best,
> Michael
>
> Eric Maris schrieb:
>
>> Hi Michael,
>>
>>
>>> I have however another question regarding the interpretation of
>>> clusteranalysis results. Am I correct in saying that the family wise
>>> error rate (alpha) tells me the risk in obtaining a false positive
>>> statement of the type that I specify previously with alphatresh? For
>>> example if I specify alphathresh of 0.1 (lets calls this trend for
>>> abbreviation) in the first pass of the analysis (multiple testing)
>>> before clustering then the clusterrandomization using alpha =0.05 tells
>>> me that I run a risk of 5% of identifying wrongly at least one of these
>>> 'trend clusters'.
>>> (Or else, if the above is incorrect what is the reason not to use a very
>>> lenient criterion in the first pass to feed the clusterrandomization
>>> with as many clusters as possible?)
>>
>>
>>
>> The issue is statistical power (sensitivity). If you use a very lenient
>> criterion (say, alphathresh=0.2) to select candidate cluster members,
>> this will result in large clusters purely by chance. If the effect in
>> your data is strong but confined to a small number of sensors and
>> timepoints, clusterrandanalysis may not pick it up. This is because the
>> reference distribution is dominated by these weak but large "chance
>> clusters". You will not encounter this problem if you put alphathresh
>> higher. On the other hand, a high aphathresh will miss weak but
>> widespread effects.
>>
>> To sum up, alphathresh determines the relative sensitivity to "strong but
>> small" and "weak but large" clusters.
>>
>>
>> greetings,
>>
>> Eric Maris
>>
>> .
>>