Higher Education Research and Development Society of Australasia
Many of us in higher education have spent the past year mapping degree-level learning outcomes to secured assessments; so, we can be confident the student and not a friend, someone who was paid, or an LLM did the work.
The challenge I've faced is that the more I tried to engage with potential solutions, the less sure I became about what I could do. Every conversation seemed to circle back to the same answers: supervised exams and interactive oral assessments.
Both are legitimate and have a lot of evidence supporting them. They're also both designed for students who can turn up somewhere, usually in person, at a scheduled time. That isn't who most of my students are, and I think a lot of other educators are grappling with this problem as well.
I teach postgraduate health management and public health. Most of my students study online, in their own time, while working for health services, government, or community organisations. I can't assume synchronous engagement - students are spread across time zones and fitting study around shift work, clinical, and caring responsibilities. I can't assume staged progression - students enter in any term, move at whatever pace their working life allows, and there's no shared cohort moving through the degree together. This means that end-of-year hurdle assessments often don't map onto how people actually progress. And I can't require in-person attendance - many of my students have never been to campus (which itself makes me a bit sad, but means many more people are able to study).
The degrees that have built the more developed inbuilt assurance systems, like nursing, engineering, accounting or medicine, tend to be structured, mostly full-time, mostly in-person, cohort-based programs where programmatic assessment can be built in from the start. Online educators are trying to assure learning with a smaller and distinct set of tools. The frustration I keep hearing from colleagues, and feeling myself, is that the standard advice seems like it’s mostly written for someone else's students.
So, before simply setting online interactive orals, as many universities seem to be doing, I wanted more clarity about what it actually means to secure an assessment. What's actually being secured, against what?
Phil Dawson wrote a book on assessment security in 2021 (ahead of the game on that one). In it he separates assessment security from academic integrity. Security is about hardening assessment against cheating, usually by making it harder to pass off work that isn't your own. Integrity is about students' values and their motivation to do honest work. Security addresses opportunity; integrity addresses motivation. Most institutional policy bundles them together, which can result in systems that are either surveillance-heavy and adversarial, or naive and dependent on goodwill. Neither works on its own.
The second distinction Dawson makes, between securing and assuring, took me a while to wrap my head around. Securing is something you do to an individual task. Assuring is the degree-level claim we can make about what people know and can do, based on those secured tasks. A secured task is an input. Assurance is the argument you build from that body of evidence.
With those distinctions in hand, I had a look for what securing mechanisms actually exist, beyond the usual ones. I identified five or six distinct clusters (depending on how you bound it) that I've sorted into a working typology. You can [download the full draft document here], which sets out mechanisms, descriptions, forms, strengths and limitations. Briefly:
The first five clusters address opportunity. Only the sixth focuses on motivation. A coherent approach probably needs both, designed at degree level rather than bolted onto individual subjects.
One caution about the cluster I've leaned on most heavily in the past: authentic assessment alone isn't the answer (and it may be more accurately regarded as assessment validity than security as such) - you have to layer it with approaches from other clusters.
Developing the working typology led me to two conclusions. The first is that securing the inference, through programmatic assessment and aggregation via portfolios or capstones, is probably the single mechanism that fits a non-standard cohort best. Mapping degree outcomes across multiple subjects and many at least partially secured assessments gives more assurance than a single high-stakes exam or viva voce hurdle. It's more defensible and doesn't require everyone to be in the same place at the same time.
The second is that the clusters can be combined to make assessments more interesting as well as more secure. Kelly Webb-Davies and colleagues propose an example they call Voice-First Written Assessment: a first stage captures a student's thinking under observed conditions, then a second stage develops that into a final submission with AI use permitted. The final product is assessed as a plausible workup of the secured baseline discussion, with a follow-up interview only where that link is unclear. It's a single task that draws on multiple clusters at once - identity, observation, construct design, and a portfolio - and it permits AI use rather than thinking it can be controlled or proscribed.
The typology doesn't solve my problem - the mechanisms that seem most robust are hard to implement at scale within existing resources. But it makes the issues more comprehensible, at least to me. Most conversations about securing assessments start with the format and work backwards. Starting with what's being secured instead may open up more options for non-traditional student cohorts, particularly if we combine mechanisms.
Nobody I know who teaches online wants to dodge securing and assuring learning. But we're trying to do it with fewer tools and none of the conditions the advice seems to assume. And my students overwhelmingly aren't seeking to cheat. For them, securing assessment is less about deterring calculated cheating than about designing assessment well, so that doing honest work is straightforward. Creative assessment design, and resisting the temptation to run to old default settings, is probably our only way forward.
This post is adapted from a longer version first published on Ben's blog.
Banner image source: “Day 23 - Exam hall” by Jack Hynes, CC BY-NC-SA 2.0
The HERDSA Connect Blog offers comment and discussion on higher education issues; provides information about relevant publications, programs and research and celebrates the achievements of our HERDSA members.
HERDSA Connect links members of the HERDSA community in Australasia and beyond by sharing branch activities, member perspectives and achievements, book reviews, comments on contemporary issues in higher education, and conference reflections.
Members are encouraged to respond to articles and engage in ongoing discussion relevant to higher education and aligned to HERDSA’s values and mission. Contact Daniel Andrews Daniel.Andrews@herdsa.org.au to propose a blog post for the HERDSA Connect blog.
HERDSA members can login to comment and subscribe.