A Simple Sorting Selection Method for Picking Results from an Automatic Item Generator

Abstract

Automatic Item Generation (AIG) is an extremely useful tool to construct many high-quality exam items more efficiently than traditional item writing methods. A large pool of items, however, presents challenges like identifying a particular item to meet a specific need. For example, when making a fixed form exam, best practices forbid item stems that are too similar, have identical keys, and have too many similar or identical distractors. This paper introduces the Simple Sorting Selection algorithm for selecting a subset of dissimilar AIG items. This algorithm will select a set of items with the maximum possible variety of features as defined by the user. The method is labelled as simple because it does not require advanced syntactical techniques such as language processing and can be implemented in numerous software platforms. Results of a trial using the output of an AIG model template demonstrate that the algorithm can find a smaller set of the most unique items in an AIG set. Simple Sorting Selection's performance in finding the most unique items was notably better than individual trials of random selection. Performance was somewhat better when selecting a small number of items and notably better when selecting numerous items compared to a maximum dissimilarity index calculated from dummy-coded representations of the AIG-produced items.