The Use of Multiple t Tests in Language Research.
Author: James Dean Brown
Source: TESOL Quarterly, Vol. 24, No. 4 (Winter, 1990), pp. 770-773.
Published by: Teachers of English to Speakers of Other Languages, Inc. (TESOL)
Stable URL: http://www.jstor.org/stable/3587135
Accessed: 29/09/2008 10:24
The central problem with multiple t tests is that there is a certain probability that one or more significant differences will be found by chance alone. As the number of comparisons increases in a study, there is an increasingly high probability that one or more spuriously significant differences will be found. The severity of this problem depends on whether the means are independent (i.e., based on different groups of subjects) or nonindependent (i.e., based on sets of observations of the same group of subjects). For independent means, the probability of one or more t tests being spuriously significant (i.e., the probability of committing a Type I error) can be calculated using 1- (1-α)c, where c represents the number of independent comparisons. For instance, with α set at .05, the probability of a Type I error for one comparison is 5%, for six comparisons it is 26%, for ten comparisons it is 40%, for fifteen comparisons it is 54%, for twenty comparisons it is 64%, etc. For nonindependent means, the probabilities are even higher: for six comparisons the probability has been estimated to be approximately 40%, for ten comparisons about 60%, for twenty comparisons 90%, etc. (Cochran & Cox, 1957). Whether the means are independent or nonindependent, the general problem is that any one (or more) of the observed significant differences in the study may have occurred by chance alone. Since it is not possible to determine which of the significant differences might be spurious, interpretation of the results becomes difficult, if not impossible.

附件下载