Developing Authentic Digital Math Assessments

Developing Authentic Digital Math Assessments

Authors

  • Laurie Laughlin Davis
     Curriculum Associates, Billerica, MA
  • Kristin Morrison
     Curriculum Associates, Billerica, MA
  • Joyce Zhou-Yile Schnieders
     ACT, Inc., Iowa City, IA
  • Benjamin Marsh
     State Farm, Champaign, IL

Keywords:

Computer-based Testing, Digital Assessment, Technology Enhanced Items, Construct Fidelity, Math Assessment

Abstract

With the shift to next generation digital assessments, increased attention has focused on Technology-Enhanced Assessments and Items (TEIs). This study evaluated the feasibility of a high-fidelity digital assessment item response format, which allows students to solve mathematics questions on a tablet using a digital pen. This digital ink approach allows students to hand write their responses while showing their work in a digital format. Responses obtained using the digital pen were compared to both paper- and type-written response formats in a repeated measures design. Results showed that students liked the digital pen format and preferred it to the type-written format, but continued to rank the paper format as their first preference. Statistical comparisons showed no significant differences in student performance across response modalities, but means fell in the predicted order favoring digital pen over keyboard for the digital response formats. Results are discussed in light of software usability, technical issues during data collection, and statistical power for detecting effects in the study.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2021-02-10

How to Cite

Davis, L. L., Morrison, K., Schnieders, J. Z.-Y., & Marsh, B. (2021). Developing Authentic Digital Math Assessments. Journal of Applied Testing Technology, 22(1), 1–11. Retrieved from http://jattjournal.net/index.php/atp/article/view/155879

Issue

Volume 22, Issue 1, 2021

Section

Articles

References

Bobeck, E., & Tversky B. (2014). Creating visual explanations improves learning. Proceedings of the 36th Annual Conference on the Cognitive Science Society, Austin, TX. https://doi.org/10.1037/e528942014-239

Bower, M. (2008). Affordance analysis-matching learning tasks with learning technologies. Educational Media International, 45(1), 3–15. https://doi.org/10.1080/09523980701847115

Hammond, T., Valentine, S., Adler, A., & Payton, M. (Eds). (2015). The impact of pen and Touch technology on education. Springer. https://doi.org/10.1007/978-3-319-15594-4

Heiten (Loewus), L. (2014a, September). Will common core testing platforms impede math tasks? Education Week. http:// www.edweek.org/ew/articles/2014/09/24/05math.h34.html?r=194816188&print=1

Heiten (Loewus), L. (2014b, September). Common-core math testing: Using an equation editor. Education Week. http:// blogs.edweek.org/edweek/curriculum/2014/09/equation_ editors.html

Keng, L., McClarty, K. L., & Davis, L. L. (2008). Item-level comparative analysis of online and paper administrations of the Texas assessment of knowledge and skills. Applied Measurement in Education, 21(3), 207–226. https://doi.org/10.1080/08957340802161774

Koile, K., & Rubin, A. (2015). Machine interpretation of students’ hand-drawn mathematical representations. The Impact of Pen and Touch Technology on Education (pp. 49–56). Springer. Cham. https://doi.org/10.1007/978-3319-15594-4_5

Koile, K., & Rubin, A. (2016). Tablet-based technology to support students’ understanding of division. Revolutionizing Education with Digital Ink (pp. 71-89). Springer, Cham.

https://doi.org/10.1007/978-3-319-31193-7_5

Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. https:// doi.org/10.1177/0956797614524581. PMid:24760141

Norman, D.A. (1988). The psychology of everyday things. New York, NY: Basic Books. Oviatt, S., Arthur, A., & Cohen, J. (Eds.). (2006). Quiet interfaces that help students think. Proceedings of the 19th annual ACM symposium on User interface software and technology (pp. 191-200). ACM. https://doi.org/10.1145/1166253.1166284

Oviatt, S., Arthur, A., & Cohen, J. (2006, October). Quiet interfaces that help students think. In Proceedings of the 19th annual ACM symposium on User interface software and technology (pp. 191-200). ACM.

Parshall, C. G., Davey, T., & Pashley, P. J. (2002). Innovating item types for computerized testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp.129-148). Norwell, MA: Kluwer. https:// doi.org/10.1007/0-306-47531-6_7

Piper, A. M., & Hollan, J. D. (2009, April). Tabletop displays for small group study: Affordances of paper and digital materials. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1227-1236). ACM. https://doi.org/10.1145/1518701.1518885

Russell, M. (2016). A framework for examining the utility of technology-enhanced items. Journal of Applied Testing Technology, 17(1), 20–32.

Schilit, B. N., Golovchinsky, G., & Price, M. N. (1998, January). Beyond paper: supporting active reading with free form digital ink annotations. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 249–256). ACM Press/Addison-Wesley Publishing Co..

https://doi.org/10.1145/274644.274680

Smarter Balanced Assessment Consortium (2012). Smarter Balanced Assessment Consortium: Technology enhanced item guidelines. https://www.measuredprogress.org/wp content/uploads/2015/08/SBAC-Technology-EnhancedItems-Guidelines.pdf

Tversky, B. (2015). Keynote address: Tools for thinking. In The impact of pen and touch technology on education, HumanComputer Interaction Series. In T. Hammond, S. Valentine, A. Addler, & M. Payton (Eds). (pp. 1-3). Springer, Cham. https://doi.org/10.1007/978-3-319-15594-4_1

Wang, G., Bowditch, N., Zeleznik, R., Kwon, M., & LaViola, J. J. (2016). A tablet-based math tutor for beginning algebra. Revolutionizing Education with Digital Ink (pp. 91-102). Springer, Cham. https://doi.org/10.1007/978-3-319-311937_6

Way, W. D., Davis, L. L., Keng, L., & Strain-Seymour, E. (2015). From standardization to personalization: The comparability of scores based on different testing conditions, modes, and devices. Technology in testing: Measurement issues, F.Drasgow (Ed). Vol 2 of the NCME book series.

Wachsmuth, B. (2015). Statistics in the classroom on touchbased smart phones. In The impact of pen and touch technology on education, Human-Computer Interaction Series. T. Hammond, S. Valentine, A. Addler, & M. Payton (Eds). (pp. 289–296). Springer, Cham. https://doi.org/10.1007/978-3-319-15594-4_30

Williams, L. (2016). The integration of inking touch, and flipping within the mathematics middle school classroom. Revolutionizing Education with Digital Ink (pp. 329–334). Springer, Cham. https://doi.org/10.1007/978-3-319-311937_23