Comparison of effect size estimates
Effect size estimates describe the proportion of variation in the dependent variable that can be explained by variation in the independent variable. Although probability values will tend to decline with increasing effect sizes, effect size estimates are not tests of statistical significance (these are χ2for Kendall’s coefficient of concordance and F-ratio for ANOVA). It is therefore possible to obtain a test-statistic associated with a significant probability value, indicating that the independent variable has an effect on the dependent variable, associated with a low effect size estimate, indicating thatthe magnitude of the significant effect is low. In the case of Kendall’s coefficient of concordance ‘W’ gives the proportion of ‘agreement’ in the ranking of responses between trials, such that when W = 0 there is no agreement, indicating that ranks are completely random, whereas if W = 1 there is complete consistency between trials. Thus, W gives a scaled measure of the effect size of consistency in ranks between trials. This may be compared directly with equivalent scaled measures of effect size used in other types of statistical test such as the Etasquared values associated with ANOVA tests. Etasquared is given by
η2= SSeffect / SStotal,
where SS = sum of squares. This measure estimates the proportion of variance in responses that is due to the effect of treatment group (e.g. ‘trial type’ in this study). If η2= 0, then none of the variance in response is due to the effect of treatment group, whereas if η2 = 1 then all of the variance in response is due to the effect of treatment group. In one way ANOVA, the partial Etasquared values,
η2p =SSeffect / (SSeffect + SSerror),
normally provided by statistical software (e.g. SPSS) also scale to 1, giving the same value as ‘classical’ η2.Thus, our reports of η2p for one-way tests as used for comparison with Kendall’s W (see Table 1) are identical to η2. In multivariate models, however, η2p values may not scale to a maximum of 1. It is therefore important to notethat in studies relying on comparisons between multivariate tests, η2should be used instead of η2p (see Levine & Hullett 2002 for a discussion of this point).
Note that although we are interested in ‘consistent individual variation in behaviour’ the statistical tests and effect size estimates used here and in previous studies must relate to samples containing multiple individuals. In the case of behavioural plasticity it is intuitive that variation between situations can only be statistically tested for by comparing average responses, generated from a sufficient number of individuals to provide adequate replication. Similarly, consistency in behavioural differences between individuals is a property of a sample of individuals, where tests of correlation or concordance between situations indicate the stability of individual responses relative to one another.