Repeatability or test–retest reliability[1] is the closeness of the agreement between the results of successive measurements of the same measure, when carried out under the same conditions of measurement. [2] In other words, the measurements are taken by a single person or instrument on the same item, under the same conditions, and in a short period of time. A less-than-perfect test–retest reliability causes test–retest variability. Such variability can be caused by, for example, intra-individual variability and inter-observer variability. A measurement may be said to be repeatable when this variation is smaller than a predetermined acceptance criterion.
Test–retest variability is practically used, for example, in medical monitoring of conditions. In these situations, there is often a predetermined "critical difference", and for differences in monitored values that are smaller than this critical difference, the possibility of variability as a sole cause of the difference may be considered in addition to, for example, changes in diseases or treatments.[3]
Conditions
The following conditions need to be fulfilled in the establishment of repeatability: [2][4]
- the same experimental tools
- the same observer
- the same measuring instrument, used under the same conditions
- the same location
- repetition over a short period of time.
- same objectives
Repeatability methods were developed by Bland and Altman (1986).[5]
If the correlation between separate administrations of the test is high (e.g. 0.7 or higher as in this Cronbach's alpha-internal consistency-table[6]), then it has good test–retest reliability.
The repeatability coefficient is a precision measure which represents the value below which the absolute difference between two repeated test results may be expected to lie with a probability of 95%.
The standard deviation under repeatability conditions is part of precision and accuracy.
Attribute agreement analysis for defect databases
An attribute agreement analysis is designed to simultaneously evaluate the impact of repeatability and reproducibility on accuracy. It allows the analyst to examine the responses from multiple reviewers as they look at several scenarios multiple times. It produces statistics that evaluate the ability of the appraisers to agree with themselves (repeatability), with each other (reproducibility), and with a known master or correct value (overall accuracy) for each characteristic – over and over again.[7]
Psychological testing
Because the same test is administered twice and every test is parallel with itself, differences between scores on the test and scores on the retest should be due solely to measurement error. This sort of argument is quite probably true for many physical measurements. However, this argument is often inappropriate for psychological measurement, because it is often impossible to consider the second administration of a test a parallel measure to the first.[8]
The second administration of a psychological test might yield systematically different scores than the first administration due to the following reasons:[8]
- The attribute that is being measured may change between the first test and the retest. For example, a reading test that is administered in September to a third grade class may yield different results when retaken in June. One would expect some change in children's reading ability over that span of time, a low test–retest correlation might reflect real changes in the attribute itself.
- The experience of taking the test itself can change a person's true score. For example, completing an anxiety inventory could serve to increase a person's level of anxiety.
- Carryover effect, particularly if the interval between test and retest is short. When retested, people may remember their original answer, which could affect answers on the second administration.
See also
References
- ^ Types of Reliability Archived 2018-06-06 at the Wayback Machine The Research Methods Knowledge Base. Last Revised: 20 October 2006
- ^ a b JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in measurement (PDF), Joint Committee for Guides in Metrology, 2008, archived (PDF) from the original on 2009-10-01, retrieved 2018-04-11
- ^ Fraser, C. G.; Fogarty, Y. (1989). "Interpreting laboratory results". BMJ (Clinical Research Ed.). 298 (6689): 1659–1660. doi:10.1136/bmj.298.6689.1659. PMC 1836738. PMID 2503170.
- ^ Taylor, Barry N.; Kuyatt, Chris E. (1994), NIST Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results Cover, Gaithersburg, MD, USA: National Institute of Standards and Technology, archived from the original on 2019-09-30, retrieved 2018-04-11
- ^ "Statistical methods for assessing agreement between two methods of clinical measurement". Archived from the original on 2018-07-06. Retrieved 2010-09-30.
- ^ George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Boston: Allyn & Bacon.
- ^ "Attribute Agreement Analysis for Defect Databases | iSixSigma". 26 February 2010. Archived from the original on 22 March 2016. Retrieved 7 February 2013.
- ^ a b Davidshofer, Kevin R. Murphy, Charles O. (2005). Psychological testing : principles and applications (6th ed.). Upper Saddle River, N.J.: Pearson/Prentice Hall. ISBN 978-0-13-189172-2.
{{cite book}}: CS1 maint: multiple names: authors list (link)