|
| [ HERDSA ]
[ Proceedings Contents ] |
A series of three studies designed to minimise problems of involving students in peer assessment and feedback are described. In Study I, feedback related to oral presentation skills. Positive feedback was found to be more forthcoming than hints for improvement. Study II found that lecturers and students tended to provide different kinds of feedback, and an "hierarchy of feedback" was proposed. Study III involved peer criticism of student writing. Feedback was given to writers in stages, to enable students to act on, and benefit from, advice. Short reflective statements submitted by students indicated that nearly 90% rated the exercise as useful. Both the quality of advice, and the extent to which students acted upon it, are examined, and recommendations made for future implementations.
Comparisons of mean peer and lecturer mark indicated little difference between the two, and evaluation suggested that the benefits of previous systems of peer assessment had been preserved.
Feedback to peers reflected the agreed criteria, but more strengths than weaknesses were identified.
Marks awarded by peers and teachers were again found to be similar, supporting the prevailing view that peer assessment is a useful and valid methodology.
Feedback statements by both lecturers and students were compared for numbers of "positive" and "negative" statements. Both lecturers and students provided less helpful critical feedback or suggestions for improvement than positive features, particularly in "good" presentations, although constructively critical feedback is thought to be more useful. Many similarities were found between student and teacher feedback, but some interesting differences emerged. Students focused more than lecturers on practical issues such as preparation and delivery, and on the amount and quality of information, whereas teachers emphasised understanding and methodological issues. An "hierarchy" of feedback was proposed.
Studies I and II were unable to explore longer term benefits of peer feedback, due to the very short length and limited assessment opportunities of the modules of which they formed a part. Thus, study III was designed to assess longer term usefulness of peer feedback by examining modifications to students' written work as a result of a feedback exercise.
In this study, a primary concern was the quality of feedback, and the differences between numbers of positive and negative statements. An attempt was made to assess the extent to which students acted on advice, using the reflective statements and a comparison of peer and lecturer feedback. Reflective statements also provided an evaluation of the scheme.
"1. There are many errors which you make when writing an essay, but it takes someone else to actually notice them.A male student described a rather different reaction.
2. The input from someone else can only serve to help. Two heads really are better than one.
3. The exercise spurs you on to write a good essay and not leave it until the last minute."
"The fact that a peer is going to read your essay and be invited to rip it to shreds makes demands upon you to take extra care. There is nothing worse than somebody quietly smirking to themselves at your expense, and an attempt to avoid this becomes a priority when writing the essay."Two students made comments which suggested their view of education included dependence on the lecturer or an over emphasis on the acquisition of information.
"It may have proved useful if a lecturer had read your essay - even to give a few pointers without giving too much away. A provisional mark may also have showed the individual students if they had incorporated the right pieces of information in their essay." (female)Another female student speculated whether it would have been better if the criticism had been supplied by the lecturer, but concluded:"A bad point may be that it wasted a few hours of lecture time which may have been spent discussing other topics." (female)
"I understand that the purpose of the exercise was to extend your own critical techniques as well as aiding in peer assessment. Therefore, it would have defeated the purpose if the lecturer had taken this role."Participation in the exercise may have played some part in this move towards independence.
In the 36 submissions where feedback was included, students supplied 140 positive statements, 119 negative statements and 26 prompts to reflection. Lecturers, on the other hand, generated only 56 positive, 93 negative and 10 neutral prompts and suggestion statements for the same corpus. Thus, students provided both more, and more positive, feedback than lecturers, together with more prompts and suggestions. Overall, the most frequently used feedback category was "Content", followed by "Research and Evidence", "Structure", "Communication", "Answers the question", "Quality", "Spelling and Grammar" and "Style", "Production quality", "Length", "Global statements" and "Preparation". Spearman rank order correlations were calculated to compare frequency of use of categories in four groups (student positive, student negative, lecturer positive and lecturer negative). Two correlations reached significance: that between student positive and student negative feedback (rho = 0.7929, significance p = 0.002), and between lecturer positive and student negative feedback (rho = 0.6320, significance p = 0.027). The correlation between lecturer positive and lecturer negative feedback just failed to reach significance. Thus, positive and negative student feedback tends to make use of similar categories, whereas lecturer positive or negative feedback is characterised by particular categories. For example, lecturer negative feedback made less use of the "Content" category than any other group, stressing instead the "Quality" category. Lecturers overall made greater use of "Quality" than students.
"Throughout the evaluation there was constructive criticism, but (the reviewer) also explained the good points of the essay. This gave me more confidence in my essay as a whole, and allowed me to concentrate on the aspects that needed to be developed." (female)Identification of weaknesses and hints for improvement were rated as essential by those students whose feedback was limited to identification of strengths.
"The feedback which I received from a peer was generally very limited in terms of its usefulness. The feedback only contained positive remarks. For the evaluative criticism to have been of worth, the statement should obviously have contained some ideas for improvement of the essay - even if I had though these ideas to be unsuitable, they would still at least have been food for thought / consideration." (male)The solitary female student who found the exercise unhelpful exemplifies the importance of receiving a balance of types of feedback which includes positive feedback. She wrote:
"I found the feedback very unhelpful. ..... There was no feedback on structure ..... no comments on communication. ..... To say that about half an essay should be changed and not suggest any ideas is very unhelpful."In fact, the reviewer had not failed to provide any helpful feedback as the recipient perceived, and there were a number of criticisms and suggestions for remedy, some of which were very useful. What was missing from the peer criticism, however, was any positive feedback.
In summary, PFM III seems to have achieved the aim of stimulating reflection. Students and lecturers provided similar kinds of feedback, though students supplied more, and more positive, feedback than lecturers. However, students had more time in which to complete the task than lecturers. The importance of providing a balance of positive and negative feedback was highlighted. Most students found the exercise useful, and appeared to have acted on some advice given.
Positive versus negative feedback formed a theme running through the studies. In studies I and II more strengths than weaknesses were identified by both students and teachers. The results of study III also suggested that students (but not lecturers) provided more positive statements than negative. Positive feedback brings both advantages and disadvantages to recipients. From personal experience, and from some reflective statements, it seems that positive feedback may be very desirable, particularly for female students who lack confidence. (This may also apply to male students, but the present study involved a very small number of males, none of whom gave any indication of a lack of confidence.) Conversely, an over emphasis on positive feedback may carry disadvantages, as "negative" criticism and suggestions for improvement are also necessary for stimulating reflection and change.
Future implementations will devote even more time to feedback exercises. In addition, more than one peer acting as reviewer for each author may help ensure that each student receives balanced feedback. Differences of opinion between reviewers, though potentially unsettling for the author, might stimulate further reflection. Practice may also help improve reviewer performance.
A number of unresolved issues remain concerning the choice of reviewers. Should they be friends or allocated anonymously? Should the reviewer be a topic expert or not? In study III, students were given the choice in relation to the first question, but were instructed to review an essay on a different topic from their own in order to prevent plagiarism.
Student evaluations of the exercise suggested that future researchers would be advised to be aware of the possibility of gender effects.
This series of studies indicate that students are capable of providing useful and timely feedback to their peers. Even though the teacher may supply sound advice, typically, it is not offered in time for students to benefit immediately. In any case, reflective statements in study III suggested that some students do not act on advice given by the teacher after submission of a piece of work.
Finally, the last words come from two female student participants in study III.
"In conclusion, the evaluative criticism and feedback is an excellent learning experience, and I have certainly gained from it.""This is the first time in which I have participated in a system such as this ..... and to sum the experience up, 'Why wasn't it implemented before now?'"
Bruffee, K.A. (1978) The Brooklyn Plan: Attaining intellectual growth through peer-group tutoring. Liberal Education, 64(4), 447-468.
Falchikov, N. (1986). Product comparisons and process benefits of collaborative self and peer group assessments. Assessment and Evaluation in Higher Education, 11(2), 146-166.
Falchikov, N. (1994). Learning from peer feedback marking: student and teacher perspectives. In Group and Interactive Learning (Eds Foote, H.C., Howe, C.J., Anderson, A., Tolmie, A.K. & Warden, D.A.), Computational Mechanics Publications: Southampton & Boston.
Falchikov, N. (1995a). Peer feedback marking: developing peer assessment. Innovations in Education and Training International, 32(2), 175-187.
Falchikov, N. (1995b). Improving feedback to and from students. In Assessment for learning in higher education, (Ed. Peter Knight), Kogan Page: London.
Farmer, B. and Eastcott, D. (1995). Making assessment a positive experience. In Assessment for learning in higher education, (Ed. Peter Knight), Kogan Page: London.
Gray, T.G.F. (1987). An exercise in improving the potential of exams for learning, European Journal of Engineering Education, 12(4), 311-323.
Magin, D.J. and Churches, A.E. (1989). What do students learn from self and peer assessment? Australian Journal of Educational Technology.
| Author: Nancy Falchikov, Department of Psychology and Sociology, Napier University, Edinburgh, EH10 5DT, UK. Tel (0)131 455 2296/2522 Fax (0)131 455 2295 Email n.falchikov@central.napier.ac.uk
Please cite as: Falchikov, N. (1996). Improving learning through critical peer feedback and reflection. Different Approaches: Theory and Practice in Higher Education. Proceedings HERDSA Conference 1996. Perth, Western Australia, 8-12 July. http://www.herdsa.org.au/confs/1996/falchikov.html |