Identifying Statistically Actionable Collusion in Remote Proctored Exams

Abstract

The rise of online proctoring potentially provides more opportunities for item harvesting and consequent brain dumping and shared “study guides†based on stolen content. This has increased the need for rapid approaches for evaluating and acting on suspicious test responses in every delivery modality. Both hiring proxy test takers and studying unauthorized test content (e.g., “study guides†or brain dumps) result in characteristic patterns of responses, many of which are detectable through collusion analysis. The ability to identify and rapidly revoke test results are one component of stopping test takers from engaging in these behaviors, both in online proctored and test center testing. Existing collusion analyses have typically taken the approach of evaluating all response pairs sequentially, potentially requiring several days to evaluate a set of test results. This paper demonstrates matrix-based methods for quickly calculating exact overlap counts for large data sets, as well as approaches for determining criteria for flagging suspicious results or invalidating results. We discuss and compare the results for simulations and probability calculations and discuss the operational implications of these decisions.