The Use of Data Imputation when Investigating Dimensionality in Sparse Data from Computerized Adaptive Tests

Abstract

The development of a Computerized Adaptive Test (CAT) for operational use begins with several important steps, such as creating a large-size item bank, piloting the items on a sizable and representative sample of examinees, dimensionality assessment of the item bank, and estimation of item parameters. Among these steps, testing the dimensionality of the item bank is particularly important because the subsequent analyses depend on the confirmation of the hypothesized factor structure (e.g., unidimensionality). After the CAT becomes operational, it is still important to periodically assess the dimensionality of the item bank because both the examinee population and the item bank may change over time. However, extreme sparseness of the response data returned from the CAT makes the test of dimensionality very difficult. This study investigated whether data imputation can be a feasible solution to the sparseness problem when examining test dimensionality in sparse data returned from CATs. Sparse data with unidimensional, multidimensional, and bi-factor test structures were simulated based on real data from a large-scale, operational CAT. Two-way imputation and Multivariate Imputation with Chain Equations (MICE) methods were used to replace missing responses in the data. Using confirmatory factor analysis, imputed datasets were analyzed to examine whether the true test structure was retained after imputations. Results indicated that MICE with classification and regression trees (MICE-CART) produced highly accurate results in retaining the true structure, whereas the performances of other imputation methods were quite poor. Data imputation with MICE-CART appears to be promising solution to data sparsity when examining test dimensionality for CATs.