Reliability concerns in the repeated computerized assessment of attention in children.

TitleReliability concerns in the repeated computerized assessment of attention in children.
Publication TypeJournal Article
Year of Publication2009
AuthorsZabel AT, von Thomsen C, Cole C, Martin R, Mahone ME
JournalThe Clinical neuropsychologist
Date Published2009 Sep

Assessment of attentional processes via computerized assessment is frequently used to quantify intra-individual cognitive improvement or decline in response to treatment. However, assessment of intra-individual change is highly dependent on sufficient test reliability. We examined the test-retest reliability of selected variables from one popular computerized continuous performance test (CPT)-i.e., the Conners' CPT - Second Edition (CPT-II). Participants were 39 healthy children (20 girls) ages 6-18 without intellectual impairment (mean PPVT-III SS = 102.6), LD, or psychiatric disorders (DICA-IV). Test-retest reliability over the 3-8 month interval (mean = 6 months) was acceptable (Intraclass Correlations [ICC] =.82 to.92) on comparison measures (Beery Test of Visual Perception, WISC-IV Block Design, PPVT-III). In contrast, test-retest reliability was only modest for CPT-II raw scores (ICCs ranging from.62 to.82) and T-scores (ICCs ranging from.33 to.65) for variables of interest (Omissions, Commissions, Variability, Hit Reaction Time, and Attentiveness). Using test-retest reliability information published in the CPT-II manual, 90% confidence intervals based on reliable change index (RCI) methodology were constructed to examine the significance of test-retest difference/change scores. Of the participants in this sample of typically developing youth, 30% generated intra-individual changes in T-scores on the Omissions and Attentiveness variables that exceeded the 90% confidence intervals and qualified as "statistically rare" changes in score. These results suggest a considerable degree of normal variability in CPT-II test scores over extended test-retest intervals, and suggest a need for caution when interpreting test score changes in neurologically unstable clinical populations.

Alternate JournalClin Neuropsychol