Not All Silence is Golden

Traditional university assessments face urgent challenges in the face of universally available Generative Artificial Intelligence (GenAI). The robust ability of GenAI to produce essays, lab reports, code, and other media, thwart our attempts to accurately assess student capabilities. Recent works highlight the need for structural reform; for academics to imagine assessment formats which are both secure and retain their the validity.

Such a call might harken images of silent exam halls, with rows of desks patrolled by stern invigilators. However, we argue increasing our reliance on high stakes exam assessments is not a satisfactory answer. While familiar in many disciplines due to historical, financial or administrative factors, there is poor pedagogical evidence to support its use. The traditional end of semester written exam is therefore unlikely to be a suitable way to assess student capabilities across many types of learning outcomes. Furthermore, one of the TEQSA proposition for Assessment Reform in the Age of AI states that assessment should emphasise the “process of learning”. We must therefore experiment and innovate with assessment methods that holistically evaluate the process of student learning over time, and in context.

Why Conversation Beats Silence

We argue that the answer rests in conversations. Regular, structured, conversations between students and educators. By designing assessment around regular conversation, we can both provide security and validity of assessment. We have embraced this assessment philosophy by redesigning our biochemistry classes to follow a form of problem-based learning (PBL). In PBL, complex problems are tackled in small student groups (traditionally 8-12) under academic supervision. This style of teaching is common in medical schools and is particularly well suited to subjects delivered using a flipped approach (where lecture material is delivered asynchronously prior to each PBL).

We scaled PBLs to a second-year biochemistry subject with large PBL class sizes of 40-50 students, where students work through the problem in small groups of 4-6. The relative contributions of students to PBL discussions are used as the mode of assessment. Key to this approach, we do not grade students based on giving “correct responses” but on their reasoning, their efforts to help their groupmates and on researching the problem. Both student peers and the supervising academic use a shared rubric to assess these contributions, which accumulate through the semester. This assessment format has notable advantages in that the PBL can be tailored to content across many different disciplines and captures student learning in situ. In this environment, GenAI usage should not invalidate the mode of assessment. It may even be desirable to encourage GenAI use in ways that foster critical and nuanced use of this technology. We have experimented with this approach by including AI-driven chatbots to role-play characters within some of the PBL scenarios.

Now for the hard part

There are challenges in adopting the PBL approach however, principally in accurately assessing student contributions during PBL classes. Tutors must effectively “hover” inconspicuously to monitor authentic student participation and have sufficient time to regularly check in with each PBL group. Also, while this form of assessment may suit many students, group discussion can be challenging to many students, including neurodiverse students, English as a second language, or students with anxieties about working in groups. Creating a supportive, inclusive, and safe environment for students in PBLs is essential to making this assessment equitable for diverse student cohorts. Students may also prefer to contribute non-verbally through shared documents (e.g., Google Docs). This also poses a challenge for academics to identify where and how these students are interacting within their PBL groups. We have found that adopting a more holistic assessment scheme for assessing these contributions is necessary, given the diverse ways that students may choose to interact and contribute to their PBL discussions. We explored this format during the arduous COVID lockdowns in Melbourne in 2020- 2021, we found it difficult to sustain strong student engagement when teaching online as we often encountered dreaded black wall of Zoom effect. While technically not impossible to use this assessment format in online contexts, doing so may increase the risk of the format being made invalid through GenAI usage.

We believe that this style of assessment of shows value by not only tracing the process of student learning but also leans into the relational nature of teaching and helps foster a sense of community in the subject. These conversational modes of assessment may even continue further, as many educators experiment with other oral forms of assessment such as vivas and interactive orals. Interactive orals may be a suitable form of summative assessment for subjects that use PBLs, as each PBL effectively scaffolds student learning and builds their confidence in using oral assessment formats. We encourage anyone looking for inspiration to consider this format as an engaging, fulfilling and, shockingly, even a fun way to teach and assess.

Blog contributors:

• Dr James Tsatsaronis - The University of Sydney
• Dr Katrina Binger - Latrobe University

Banner image Brodie Vissers from Freerange Stock