SRCLD Presentation Details
  Title  
       
    Accuracy, agreement, and reliability of language samples reconsidered.  
Author(s)
John Heilmann - University of Wisconsin-Madison
Jon Miller - University of Wisconsin-Madison
Ann Nockerts - University of Wisconsin-Madison
Karen Andriacchi - University of Wisconsin-Madison

SRCLD Info
SRCLD Year: 2004
Presentation Type: Poster Presentation
Poster Number:
Presentation Time: (na)
Categories
- Assessment
Abstract
The reliability of language sample analysis (LSA) has been debated for the past 50 years. The major criticisms of this assessment technique relate to child variability relative to speaking conditions, repeatability, transcription, and analysis. This paper takes a fresh look at the reliability of LSA in the context of computerized transcription and analysis for Spanish-English bilingual children. Accuracy of transcription, reflected by percent agreement for adhering to SALT conventions, ranged from 98.5%-100%. Differences in transcription were noted when two teams of transcribers independently transcribed the same narrative, with differences ranging from 0.6%-30% for five perceptual categories. Statistical analyses (ANOVA) revealed no significant differences for standard LSA measures across transcribers. High levels of inter-judge agreement were noted for the coding schemes used for additional subjective measures. Finally, six measures of test-retest reliability revealed significant correlations of moderate strength.