| Natasha Beretvas
University of Texas at Austin
Extension of the Hierarchical Generalized Linear Model for validation of test scores' psychometric functioning
FINAL REPORT:
The implementation of Bush's No Child Left Behind (NCLB) policy encourages renewed focus on the validation of test scores and on the accuracy and precision of the estimates that result from the models used to assess test scores' psychometric functioning. Psychometricians have a plethora of tools at their disposal for investigating the technical validity of test scores. However, most of these tools are designed to assess single psychometric attributes of the data. Yet, the possible parameterizations of the hierarchical generalized linear model (HGLM) permit simultaneous evaluation of multiple factors affecting the validity of test and item scores including differential item functioning and parameter drift. The flexibility of HGLM allows incorporation of test designs that include multidimensional subscales, as well as equating of test forms across samples and across time. Item characteristics, latent trait initial status and trajectories can each be estimated. Lastly, the model permits investigation of possible differences in latent trait intercept and slope estimates that might be related to person-level characteristics.
Use of HGLM in the psychometric context for rating-scale data has not been sufficiently studied. In addition, the use of HGLM for assessing dichotomous and polytomous item scores used to assess growth over time (of crucial importance under the NCLB policy) has also been insufficiently evaluated. This study was comprised of three studies designed to illustrate how HGLM can be used in these psychometric contexts. The first study extended the multidimensional formulation of the HGLM model (for dichotomous items) for use with polytomous items. The second study extended use of the multilevel measurement model for use with longitudinal data to estimate item and person parameters (including starting point and growth parameters). The third study extended the multilevel measurement model to include modeling of cross-classified random effects for appropriately modeling of the cross-classified data structures that are commonly found in educational data. The Early Childhood Longitudinal Study (ECLS) data set provided real test data useful for the illustration of the specific HGLM applications. These models provide opportunities to appropriately model the complexities commonly encountered in real educational datasets. It is hoped that this study will contribute to the field of measurement and measurement practice through its demonstration and evaluation of the use of HGLM for the validation of test scores' psychometric functioning.
Back to Funded Research Grants Page |