7 June 2023: Authentic and online assessment in an AI enabled world

A joint session with Assessment in Higher Education Network (UK) and Transforming Assessment. 

Session chair: Fabio Arico (University of East Anglia, UK)

Featuring two presentations:

1) "On Chat-GPT and Multiple-Choice Assessment: Outsmarting the model" Chahna Gonsalves (King's College London, UK)

Multiple-choice quizzes (MCQs) are a popular form of assessment. A rapid shift to online assessment during the Covid-19 pandemic in 2020, drove the uptake of MCQs, yet limited invigilation and wide access to material on the internet allow students to solve the questions via internet search. ChatGPT, an artificial intelligence (AI) agent trained on a large language model, exacerbates this challenge as it responds to information retrieval questions with speed and a good level of accuracy. I contend that while the place of MCQ in summative assessment may be uncertain, current shortcomings of ChatGPT offer opportunities for continued formative use. In this presentation, I outline how ChatGPT’s limitations can inform effective question design. I provide tips for effective multiple-choice question design and outline implications for academics and learning developers, and on learning support.

2) "Is authentic assessment preparing students for the digital world?" Margaret Bearman with Rola Ajjawi and Juuuson Nieminen (Deakin University, Australia)

Authentic assessment in higher education seeks to develop students’ employability and future work capabilities. Several frameworks exist that define key characteristics of authentic assessment and guide its design. However, none have engaged critically (or at all) with the digital world where artificial intelligence is profoundly shaping practice. We use the terminology of preparing students for a digital world purposefully to acknowledge that this goes beyond digital literacies (although they matter) to also include aspects of adaptability and sense of self that will enable students to work and adapt to the everchanging technological innovations. The term ‘the digital’ refers to both a digital technology and a social practice: in the digital world we now live with technology rather than only use it. We developed a conceptual framework based on our expertise and knowledge of the literature. We then conducted a critical scoping review of authentic assessment literature to determine the role of digital technology and to test the framework against empirical practice in higher education. We analysed the task design of each of the studies to determine how they made use of technology to further refine the framework. Findings indicate that authentic assessment design in the literature has mostly used digital technology for mechanical and procedural purposes, such as gaining efficiency. Most studies reported using technology largely to enhance assessment design and to ensure that students learned certain predetermined digital literacy skills, for example uploading videos. Much less attention was given to developing authentic capabilities in the context of the digital world, such as developing evaluative judgement for future activities working alongside big data or how authentic assessment could support the growth of students' (digital) identities. We argue that future authentic assessment research and practice needs to better integrate the digital dimensions of authenticity to be fit for purpose. We advance an organising framework for digital assessment design to guide future research and practice on authentic assessment.

Further resources:

Session Recording

Tags: