Assessment reform and transforming higher education

It is argued that the procedures that higher education institutions use to assess student learning are revealing indicators of quality, which is itself taken to depend upon the extent to which the undergraduate experience transforms students.

Authentic learning, authentic assessment

These gaps are caused by many things, not all of them amenable to technicist treatment. In this section, attention will be focused on the part played by the ways in which student learning is assessed.

Tutors, like schoolteachers, have an interest in seeing that students do as well on a course as possible. This is often judged in terms of student performance on graded assessments. It is therefore in their interests to gear their teaching to helping students to succeed on graded assessments. Of course, if the assessments are fair and reputable reflections of the well-conceived aims of the course, there is nothing wrong with this. However, for a variety of reasons, not least that some aims are hard to assess reputably (that is to say with reliability) and fairly (which is to say with validity), graded assessments are not normally like that. Typically, they give priority to assessing student performance against a limited - and arguably limiting - subset of course aims.

Unless the assessment of student learning is carefully considered at the time when programmes and modules are developed, there is a strong likelihood that assessment procedures will fail to represent the full range of programme and module goals. As a result, what is to be assessed is what gets priority in teaching and a planned-delivered gap is put in place.

Learners wish to get the best grades that are consistent with their understandings, motivation and the effort they think it reasonable to make. Assessment procedures will therefore signal to them what it is that they ought to concentrate upon, suggesting what the real aims of the module or programme are and how they might try to show that they are meeting those 'real' aims. Learners will tend to focus upon elements of a programme that are subject to high status, usually summative, grade-bearing assessment. In some cases, then, they will take a strategic approach, even if they appreciate that understanding the material might be better served by a different learning approach. The system of assessment can encourage surface approaches to learning (Entwistle, 1994) and be a source of the delivered-received gap.

Authentic learning needs authentic assessment. Where there is a mismatch between programme goals, teaching arrangements and assessment procedures, it will be the assessment procedures that exert the defining force. The assessment arrangements will define the curriculum in the eyes of the learners.

A view of quality in higher education

Harvey and Knight (1996) argue that higher education must concern itself with transforming the life-experiences of students, by enhancing or empowering them. Only this version of quality is directly connected to the key business of higher education, which is student learning. The key question becomes one about how higher education might support the process of student transformation. The subtext, explored here, is the role that assessment plays in transformation.

Transformation and assessment

There is a body of research into what stakeholders in higher education value that suggests that an emphasis on 'low level' cognitive achievements is undesirable (Harvey and Knight, 1996). Employers, to take just one group of stakeholders, are not looking for 'trainees' when they employ graduates but people equipped to learn and to deal with change. They want rounded, intelligent people, with a broad grounding in a subject and an understanding of core principles. Employers want graduate recruits who are adaptable and flexible; who can communicate well and relate to a wide range of people; who are aware of, but not indoctrinated into, the world of work and the culture of organizations; who have inquiring minds, are willing and quick to learn; and who are critical, can synthesize, and are innovative. In terms of outcomes, there is little here that is contrary to traditional academic values and expectations, nor that is conflict with a transformative view of quality.

It follows, then, that a programme that exhibited quality in the sense of being fit for employers' purposes would have a wide range of cognitive and non-cognitive goals. Student learning would be assessed according to the degree to which they met the specific criteria that were derived from such goals. Consequently,

1. Assessment would become openly criterion-referenced.

2. A test of the quality of a programme is the degree to which the assessment arrangements document learners' achievements on the full range of avowed goals.

3. Since those goals will usually pervade the whole programme, there should be signs that they are assessed throughout the programme. In a quality programme, several assessments of group work, for example, across the duration of the programme, will tell students that skill in group work is seen as an important goal, not as a marginal concern.

4. Repeated assessments of any competence enhance the reliability of the assessment.

5. A range of goals implies that there will be a range of assessment techniques in place because many of the things valued by employers, for example, cannot be satisfactorily measured by paper-and-pencil assessments of individual performance. Some demand self-assessment, others co-assessment, and a set of written, oral and performance assessments will be necessary to document achievement across the range of programme goals (Brown and Knight, 1994).

Summative, criterion-referenced assessment

Some resolution of these problems may be achieved by saying that shared understandings of criteria have to be constructed. This has often happened tacitly, where the operational meanings of criteria have become known to faculty and students by inference from the tasks set and the standards applied. This is neither fair nor efficient. It is preferable that both staff and students analyse samples of work using the criteria and in this way construct shared understandings of what the criteria are to mean. In a quality programme, where summative assessments have high status, occasions for constructing this meaning will be inbuilt and prominent. The distinction between assessment and learning, a distinction that is both appealing and misleading, may then prove less easy to make (Birenbaum, 1996, Harvey and Knight, 1996).

However, some of the problems can be attenuated by placing formative assessment above summative assessment, thereby reducing the imperative to guarantee reliability to outsiders.

Formative assessment

• it involves giving learners feedback that can be used to improve the next performance;
• it may not, and need not, involve grading;
• it is therefore directly connected to the transformational notion of quality, since it is about improving student learning;
• there is no reason why tutors have to conduct this assessment; co-, peer- and self-assessment are possible and, arguably, necessary;
• it tends not to be high stakes assessment;
• where it is associated with a dialogue between learner and teacher, the initial reliability of the assessment need not be high: what matters is that a basis is laid for a conversation;
• however, if there is to be communication between assessor and assessed, there needs to be an understanding of the meaning of the relevant criteria. This is not the same as making assessment criteria available to learners: the operational meaning of the criteria must be understood with formative assessment, as with summative assessment.

There seem to be several solutions to this problem. One is to assess a group's understanding or competence, perhaps by using a method such as Classroom Assessment. This is the term given by Angelo and Cross (1993, pp. 4-5) to a set of activities 'to help teachers find out what students are learning in the classroom and how well they are learning it ... the teacher is not obliged to share the results of Classroom Assessment with anyone outside the classroom ... Classroom Assessments ... are almost never graded and are almost always anonymous'. In other words, here there is a direct connection between one form of formative assessment and the improvement of learning quality.

Arguably more important is the idea of encouraging [formative] self-assessment (Boud, 1995) and [formative] peer-assessment (Brown and Knight, 1994). Again, there is a close link between these approaches and the enhancement of learning. Interestingly, the student who has learned something of self-assessment has not only been transformed in some measure but has also gained a powerful tool for future transformation.

Computer-based learning systems also have the power to reduce the cost of formative assessment.

Conclusion - assessment systems

These include:

• consistency between programme goals, teaching, learning and assessment procedures;
• the use of a wide range of assessment modes and methods;
• provision of opportunities for the development of shared meanings about assessment criteria;
• plentiful formative assessment, both at the classroom level and at the individual level;
• evidence of self-, peer- and co-assessment;
• a concern for depth of understanding in preference to breadth of knowledge;
• provision of opportunities for all faculty to learn about ways of teaching that are consistent with an emphasis on student learning and on the tranformative potential of higher education.

Conclusion - quality and transformation

How we assess students says far more about us and what we value than we might wish.

References

Biggs, J. B. (1993). From theory to practice: A cognitive systems approach. Higher Education Research and Development, 12(1), 73-85.

Birenbaum, M. (1996). Assessment 2000. In M. Birenbaum and F. Dochy (Eds), Alternatives in Assessment of Achievement, Learning Processes and Prior Knowledge. Norwell MA: Kluwer.

Boud, D. (1995). Enhancing Learning through Self Assessment. London: Kogan Page.

Brown, S. and Knight, P. (1994). Assessing Learners in Higher Education. London: Kogan Page.

Entwistle, N. (1994). Recent Research on Student Learning and the Learning Environment. Paper presented to the International Symposium Independent Study and Flexible Learning. Cambridge, 6 September.

Erwin, T. D. and Knight, P. T. (1995). A transatlantic view of assessment and higher education. Quality in Higher Education, 1(2), 179-188.

Fullan, M. (1991). The New Meaning of Educational Change. London: Cassell.

Harvey, L. and Knight, P. (1996). Transforming Higher Education. Buckingham: Open University Press.

Orton, J. D. and Weick, D. (1990). Loosely-coupled systems: A reconceptualization. Academy of Management Review, 15(2), 203-220.

Trow, M. (1993). Managerialism and the academic profession: The case of England. Paper presented to the Quality Debate Conference, Milton Keynes, 24 September.

Please cite as: Knight, P. (1996). Assessment reform and transforming higher education. Different Approaches: Theory and Practice in Higher Education. Proceedings HERDSA Conference 1996. Perth, Western Australia, 8-12 July. http://www.herdsa.org.au/confs/1996/knightp.html