Item Response Theory (IRT) Models
IRT (or “latent trait theory”) is an example of how statistical innovations in one field can be slow to cross disciplinary boundaries. Although IRT models have been used in education for decades, they have only been applied in other domains more recently. Embretson (1996) and Borsboom (2006) offer many reasons why these models are not widely used, such as statistical complexities and lack of IRT algorithms in popular software programs.
Another reason may be that early IRT models were more readily applied to academic constructs than they were to developmental phenomenon. Early models focused on measuring (not explaining) unidimensional constructs (e.g., math ability) at one point in time. These models assumed a monotonically increasing function (e.g., gifted students have a higher probability of a correct response than average students) and an upper asymptote of 1 (e.g., gifted students will not miss an easy item). Such models may not be useful for describing and explaining complex behavior. Recent statistical advances, however, have made IRT models more appropriate for non-educational constructs and allowed these models to be applied more widely.
In particular, IRT may be useful for studying problem behavior. The distribution of problem behavior is often highly skewed, which violates the assumptions of many classical test theory (CTT) statistical tests. In addition, items on measures of problem behavior are often summed, even though intuition suggests that some items should be given differential weight (e.g., “I skip class often” and “I set things on fire” are not equally likely to be endorsed.) Finally, some measures provide more precise measurement for individuals on one end of a latent trait, and therefore the CTT assumption that standard errors are identical for everyone is flawed. I am exploring how IRT offers potential solutions to these problems and how it may improve the assessment of change in problem behavior over time.
For an overview of Item Response Theory, you can access a PowerPoint lecture that I used in a graduate measurement course on my Teaching Samples page.
Introductory resources for those interested in Item Response Theory (IRT)
Overview of IRT:
Baker, F. B., & Kim, S. (2004). Item Response Theory: Parameter Estimation Techniques (2nd ed.). New York: Marcel
Dekker, Inc.
De Boeck, P., & Wilson, M. (2004). Explanatory Item Response Models. New York: Springer.
Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341-349.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Multivariate Applications Books Series.
Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers, 371.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory (Vol. 2). Thousand
Oaks, CA: Sage Publications.
Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications, and promise in
psychological research. Current Directions in Psychological Science, 14(2), 95-101.
Applications:
Lanza, S. T., Foster, M., Taylor, T. K., & Burns, L. (2002). Assessing the Impact of Measurement Specificity in a Behavior
Problems Checklist: An IRT Analysis. Technical Report.
Osgood, D. W., McMorris, B. J., & Potenza, M. T. (2002). Analyzing multiple-item measures of crime and deviance I: Item
response theory scaling. Journal of Quantitative Criminology, 18(3), 267-296.
Reise, S. P., & Waller, N. G. (2003). How many IRT parameters does it take to model psychopathology items?
Psychological Methods, 8(2), 164-184.