HERDSA 1996: Hicks - initial task representation of an essay question during composing processes

A description of the initial task representation of an essay question during the composing processes of a group of tertiary students

Margaret Hicks
Flexible Learning Centre
University of South Australia

Over the last twenty years there has been a greater emphasis on writing research. This has been prolific particularly in America with the work of such researchers as Emig, Flower, Hayes, Bereiter and Scardamalia. Using a psychological perspective or a cognitive approach they have concentrated on the composing processes of writers. Recursive models of the writing processes of competent writers have been developed from laboratory research and intervention studies. These have made expert/novice comparisons across different age levels. More recently some general studies have been conducted on the writing strategies of tertiary students in an academic context; however there has not been a combination of the two approaches. Detailed studies of the writing process of tertiary students in relation to competence have not been conducted.
My research provides a descriptive insight into what students do when they analyse an essay question in the area of social science. To develop a more detailed understanding of how a group of students of varying abilities analyse and represent the question/topic when writing an essay, it was necessary to describe, interpret and analyse this initial stage in the composing processes of individual students. The aim of this study was to describe detail students' cognitive processes in some detail. "Think aloud" protocols were the main form of data used to access this information. This was supplemented with data collected from interviews and the assessment by the lecturer of the students' final essays. A preliminary analysis of the data has indicated that there are two factors which are significant in the way students make an initial representation of the essay question to themselves and that these factors have an impact on performance. These factors are the amount of attention a student pays to discourse knowledge and the amount of time spent directly representing the essay question.

Introduction

In most areas of tertiary study, students need to have highly developed written skills (Nightingale, 1988; Reid, 1994). Students at this level are required to complete a significant amount of writing and a large percentage of their assessment is written. To succeed at such tasks there is a need for students to have adequate topic knowledge, a knowledge of discourse conventions, the ability to generate new information and to restructure current knowledge (Flower, Schriver, Carey, Haas and Hayes, 1989). There are many stages involved in writing and academic writing in particular is complex and places complicated cognitive demands on the writer.

It is well documented that many students find this kind of writing difficult and they do not complete a writing exercise successfully. As a study adviser in a university I am required to support student learning and assist students in becoming more effective learners. In this role I see many students who are experiencing problems with different aspects of essay writing. Writing problems are faced not by a minority but a majority of students ( Scardamalia and Bereiter, 1986, Norton, 1990; Taylor; 1989).

Over the last two decades the process of writing has been studied in depth as researchers attempt to understand in more detail the nature of this process. In particular the composing processes of writing have been studied from the perspective of cognitive psychology (Hayes and Flower, 1986; Bereiter and Scardamalia, 1987). As a result of this research, linear or stage models of the writing process have been discounted in favour of recursive models and Scardamalia and Bereiter have shown that for experts, writing is a 'knowledge transforming' process rather than a 'knowledge telling' process. Within the knowledge transforming model they emphasise the importance of integrating content, discourse and procedural knowledge.

Flower and Hayes (in Faigley, Cherry, Jolliffe and Skinner, 1985, p 8) state that composing begins when a writer is given a 'rhetorical problem'. A writer needs to make his/her own initial representation of the problem and Flower and Hayes in their research have found that how a person represents the rhetorical problem is a critical element in the composing process and that there is a significant difference in the way that experienced and inexperienced writers do this. Much of this research has been completed through laboratory/intervention studies focussing particularly on expert/novice comparisons across different age levels where subjects have been given varying writing tasks from different genres. There has been little research on academic writing from this perspective.

Despite the large importance placed on academic writing at the tertiary level, the amount of research in this area is limited and what research has been completed has been approached from different perspectives. Hounsell (1984) in a study on essay writing focussed on extended planning. Norton (1990) and Mahalski (1992) in two separate studies on essay writing looked at the relationship between students' strategies and lecturers' expected strategies. While these studies and others make some valuable comments about the planning stage of writing at tertiary level and about strategies in relationship to essay writing, they have not looked in detail at the process involved with this stage of composing, the initial representation of the essay question.

The study reported in this paper investigated how a group of students all completing the same essay question in a first year university subject, represented the essay question to themselves and what effect the way they represented the problem had on their performance.

Method

The aim of this study was to describe in more detail students' cognitive processes as they made their initial representation of an essay question in the social sciences. Seventeen students, studying in the second semester of a first year general studies subject, participated in the study. All students were completing the same essay topic and the essay was assessed by the same lecturer. Data was collected in three parts. Think aloud protocols were the main form of data used to access the information. This was supplemented with data collected from an interview, which also included a separate task to assess discourse knowledge. A more detailed assessment of the essay by the subject lecturer was used in the final analysis.

Part 1: Initial Interview - Think Aloud Protocols

The main methodological approach chosen for this study was thinking aloud protocol analyses. The composing process of writing is difficult to research as it is such a complex process and much of it happens subconsciously. Think aloud or concurrent verbal protocol methodology (Ericsson and Simon, 1993) developed from cognitive psychology was used to try and capture as fully a description of this process. Students were first instructed and trained in using a think aloud procedure. When they had mastered this procedure they were asked to think aloud as they analysed the essay question. The task was completed when students had exhausted all possibilities of explaining their analysis and felt that their next step would be either library research, reading or writing.

The recorded think aloud protocols were transcribed into scripts that were then standardised in their presentation so that line lengths could be counted and compared between subjects. A coding scheme was devised under the four main headings of 'content', 'discourse', and 'procedural' knowledge, and 'other comments'. Content knowledge related to the identification of content which was a direct result of the essay question and to factual knowledge about the subject. Discourse knowledge was about the type of writing expected within a certain context. In this particular case students were being asked to write an academic discussion paper and to develop an academic argument. Procedural knowledge referred to how students strategically completed a process. Within this coding area, students were concerned with problem solving knowledge which was often articulated in the form of goals and sub-goals. The fourth area, 'other comments' included information that could not be categorised within the three major code areas. Sub categories were developed under these headings to assist in describing these transcripts in more detail and are explained with examples in Table 1.

Table 1: Coding Categories

Code No. Description of code Examples from protocols

1. Content Knowledge

1.1 reading stated information related to the content ...showing how your understanding of the point or points made, has been challenged or increased by reading selected articles

1.2 direct interpretation/representation of the essay question ...that is the key element in this whole paper is- increased understanding of the idea chosen for discussion

1.3 talking about the need to be able to represent the essay question I would take that instead of [a] question without being given a clear question at this stage

1.4 identifying specific sections relating to the question well the other three parts of the lecture are out of the question now

1.5 generating topic content I'm going to focus on sort of the racist language and sort of using that in everyday, sort of, life...

1.6 justification/purpose in relation to content because in the lecture we were told all books telling the truth

1.7 reading notes from lecture or quotations 'we all view the world through the lens of our own culture'

1.8 underlining for content key words would be 'encourage' and 'develop your understanding'

2. Discourse Knowledge

2.1 identifying stated information given by the lecturer about the type of writing expected you are not required to summarise the articles

2.2 interpreting/representing what kind of writing is expected it's a little bit different than an essay, because it's more of a discussion

2.3 commenting on the end product I'm not sure what my finished product is going to be at all but I will definitely get there

2.4 justifying/stating a purpose in relation to discourse but that is good because I really like working without any structure

2.5 underlining discourse information well better underline that too 'you're not required to summarise the articles'

2.6 generating discourse information well obviously I'd have my introduction paragraph and what that would do is basically outline what my essay is going to say

3. Procedural Knowledge

3.1 identifying stated information given by the lecturer find at least 3 articles

3.2 interpreting and representing how to complete the task or of increasing it to five if I find another article in the interim

3.3 verbalising what they are doing now I'm now looking at the next part of the piece of paper

3.4 planning future actions I'll probably be looking at finding three articles, three to five...obviously going to the library and just having a look under human rights

3.5 verbalising past procedures I've read that, some of that...

3.6 commenting on task (eg how clearly it is specified) It's fairly well laid out...telling me what we've done so far and what we're expected to do

3.7 underlining indicating key words I'll underline that as well

3.8 justifying /stating the purpose for a particular action because I might have to change it [choice of article]

4. Other Comments

4.1 comments about the reader/audience no comments were made in this category

4.2 comments about the voice/persona no comments were made in this category

4.3 statements about self and how the student feels in relation to the task but I don't like that statement

4.4 non-coding comments unrelated comments which were deleted from the final analysis

4.5 comments about the question designer/lecture ...[names lecturer] who set the paper is trying to stimulate my thought activity in some way

Part 2: Interview and Discourse Task

After the essays had been completed and handed into the lecturer for assessment, an interview was conducted to collect further data. There were two parts to this interview: questions on the composing process and a task which assessed students' discourse knowledge. To assess discourse knowledge, the students were given an example of a non-cohesive piece of writing. The paragraphs in an essay were reordered at random and students were asked to reorganise them into a cohesive piece of writing. They were then asked questions about this task to ascertain what they knew about academic discourse. Some general questions relating to discourse about the specific essay writing were also asked. Responses from the interview and discourse task were collated and scored.

Part 3: Lecturer's Assessment of the essay

The grade given by the lecturer to each of the completed essays was collected and was one of the main performance indicators used to rank students within the group. The requirements of the assignment were outlined in more detail and the lecturer was asked to comment on these areas. These included rating the suitability of the answer to the set question, the accuracy of the content knowledge, the structure/organisation of the writing, and how well an academic argument had been constructed. Information from this separate assessment was scored and used as another performance indicator.

Results and Discussion

At this stage a preliminary analysis of the data has been completed in two ways: first by looking at the results of the whole group (seventeen students) and secondly by dividing the group into two and making a comparison between the five highest and five lowest essay scorers.

Table 2 gives an indication of the performance results for the whole group and the two sub groups. For the seventeen students within this study there was an average spread of ability and performance in the tasks required. Correlations were completed between the students' GPAs and the three different performance indicators (essay score, lecturer's detailed assessment of the essay and discourse task performance - Table 2). There were strong correlations between all of these tasks.

Table 2: Performance results for the Whole group and the High and Low groups

Whole group n=17 High Group n=5 Low Group n=5

Mean SD Actual
range Mean SD Actual
range Mean SD Actual
range

GPA (possible range 0-7.00) 4.77 0.85 3.11 - 7.00 5.49 0.89 4.75 - 7.00 3.96 0.76 3.11 - 5.12

Discourse task (range 0-18) 7.77 2.22 4 - 11 9.40 2.30 6 - 11 6.8 1.48 5 - 9

Essay score (%) 62.06 10.22 48% - 90% 74 9.41 68% - 90% 53 3.74 48% - 56%

Lecturer's assessment (possible range 0-33) 23.59 4.86 11 - 32 28 3 25 - 32 18.60 4.56 11 - 22

Each transcript was coded initially into four macro codes: content , discourse, procedural and other knowledge (Table 3). The attention given by all groups to content knowledge was proportionally similar and this supports research and observations which have been made by others (Scardamalia and Bereiter, 1986; Jackson, 1991).

Table 3: Breakdown of the amount of macro knowledge codes (%)
for the whole group and the High and Low groups

Whole group High Group Low Group

Mean SD Mean SD Mean SD

Content 55.82 18.34 47.42 10.16 54.00 24.03

Discourse 4.47 5.43 8.58 6.53 3.60 4.96

Procedural 33.51 15.07 38.66 13.42 35.68 20.84

Other 6.14 5.47 5.14 2.16 6.62 4.72

The significance of the results in Table 3 is the small amount of time given to discourse knowledge and the proportionally higher amount of attention given to content and procedural knowledge. The difference is even greater for the low group in comparison to the whole group or the high group. Knowledge of discourse conventions is very important to success ( Flower et al, 1989) yet for the whole group of students less than 5% of the time was allocated as attention to this aspect. For the low group, this was even lower, and six students out of the seventeen who took part in the study paid no attention at all to this area of knowledge. The two highest scoring students spent 13.1% and 16.7% of their time respectively attending to discourse knowledge.

For each of the four macro code areas a further breakdown was completed to describe more fully what the students did in the representation (see Table 1 for micro coding categories). For each group the greatest amount of time within content knowledge was spent on generating content (code no.1.5). Within discourse knowledge little attention was given to any of the breakdown areas. The result of significance here was the greater attention given by the high group to generating discourse knowledge (code no.2.6). The results for the three groups regarding procedural knowledge were quite consistent, with all groups giving the greatest amount of attention to future planning (code no.3.4).

Within each of the three major code areas there were similarities between the micro codes which could be looked at across the macro codes. These included the areas of reading out the question, representing the question, generating knowledge, planning, underlining the question and justifying comments.

Table 4: Micro Codes (%) across the Macro Code Areas

Whole group High Group Low Group

Mean SD Mean SD Mean SD

Reading 11.46 8.92 15.64 8.15 10.43 9.82

Representing 7.31 6.87 11.76 1.46 3.39 1.45

Generating 28.75 23.03 21.69 11.24 29.95 25.72

Planning 28.56 14.56 32.12 15.66 31.72 18.60

Underlining 0.99 1.44 1.46 1.70 0.61 0.95

Justifying 3.52 2.15 4.84 1.48 3.19 1.99

Once again for all groups, generating knowledge (content and discourse) was high as was time devoted to planning (past, present and future). The significance of these results was the greater attention given by the high essay scorers to representing the essay question, 11.76% in comparison to 3.39% by the low group. The higher essay scorers paid greater attention to the two areas of reading the essay question and representing it to themselves where, by comparison, the low essay scorers paid less attention to the essay question. This result indicates that the ability to focus on the question, whether through reading or re-reading it or by representing it in your own words, does have a clear relationship with success.

Conclusions and future directions

The preliminary analysis of data from this study has indicated that a student's initial representation of the essay question/task does have a relationship with performance and assessment. This study has shown that there are two factors in the way students make an initial representation which relate to success - the amount of attention students pay to discourse knowledge and the students' abilities to be able to represent the question in their own words and terms.

A student's attention to discourse knowledge during the initial representation of an essay question is significant and this study has identified the need to investigate further discourse knowledge and its role and accessibility during the composing process of academic writing. Methodologically, this is a difficult area to access and information could be used from this study to develop more detailed and appropriate methodologies for investigating this area.

References

Bereiter, C. and Scardamalia, M. 1987. The Psychology of Written Composition. Lawrence Erlbaum, New Jersey.

Ericsson K. and Simon H. 1993, Protocol Analysis, (revised edition), MIT, Cambridge.

Faigley L., Cherry R., Jolliffe D. and Skinner A. 1985, Assessing Writers' Knowledge and Processes of Composing. Ablex, Norwood, New Jersey.

Flower L., Schriver K., Carey L., Haas C. and Hayes J. 1989. Planning in Writing: The Cognition of a Constructive Process. Technical Report No. 34, Centre for the Study of Writing, Carnegie Mellon University, Pittsburgh.

Hayes, J and Flower, L, 1986. 'Writing Research and the Writer', American Psychologist, October, p1106-1113.

Hounsell, D, 1984. 'Essay Planning and Essay Writing', Higher Education Research and Development, Vol 3, No 1, P13-31.

Jackson, M. W. 1991. 'Writing as Learning: Reflections on Developing Students' Writing Strategies', Higher Education Research and Development, Vol 10, No 1, p41-52.

Mahalski, P. 1992. 'Essay-writing: do study manuals give relevant advice?' Higher Education, Vol 24, No 1, July, p113-132.

Nightingale, P. 1988, 'Understanding Processes and Problems in Student Writing' Studies in Higher Education,Vol 13, no 3, p263-283.

Norton, L. 1990. 'Essay-Writing: what really counts?' Higher Education, 20 p411-442.

Reid, I. 1994. 'Literacy requirements need a rethink' Campus Review, June 23-29, p8.

Scardamalia, M. and Bereiter, C. 1986. 'Research on Written Composition' in Wittrock M. (Ed), Handbook of Research on Teaching (3rd ed), MacMillan, New York.

Author: Margaret Hicks, Flexible Learning Centre, University of South Australia. Email: margaret.j.hicks@unisa.edu.au

Please cite as: Hicks, M. (1996). A description of the initial task representation of an essay question during the composing processes of a group of tertiary students. Different Approaches: Theory and Practice in Higher Education. Proceedings HERDSA Conference 1996. Perth, Western Australia, 8-12 July. http://www.herdsa.org.au/confs/1996/hicksm.html

[ HERDSA ] [ Proceedings Contents ]
This URL: http://www.herdsa.org.au/confs/1996/hicksm.html
Created 25 Apr 2002. Last revision: 24 May 2002.
© Higher Education Research and Development Society of Australasia Inc

Code No.	Description of code	Examples from protocols
1.	Content Knowledge
1.1	reading stated information related to the content	...showing how your understanding of the point or points made, has been challenged or increased by reading selected articles
1.2	direct interpretation/representation of the essay question	...that is the key element in this whole paper is- increased understanding of the idea chosen for discussion
1.3	talking about the need to be able to represent the essay question	I would take that instead of [a] question without being given a clear question at this stage
1.4	identifying specific sections relating to the question	well the other three parts of the lecture are out of the question now
1.5	generating topic content	I'm going to focus on sort of the racist language and sort of using that in everyday, sort of, life...
1.6	justification/purpose in relation to content	because in the lecture we were told all books telling the truth
1.7	reading notes from lecture or quotations	'we all view the world through the lens of our own culture'
1.8	underlining for content	key words would be 'encourage' and 'develop your understanding'
2.	Discourse Knowledge
2.1	identifying stated information given by the lecturer about the type of writing expected	you are not required to summarise the articles
2.2	interpreting/representing what kind of writing is expected	it's a little bit different than an essay, because it's more of a discussion
2.3	commenting on the end product	I'm not sure what my finished product is going to be at all but I will definitely get there
2.4	justifying/stating a purpose in relation to discourse	but that is good because I really like working without any structure
2.5	underlining discourse information	well better underline that too 'you're not required to summarise the articles'
2.6	generating discourse information	well obviously I'd have my introduction paragraph and what that would do is basically outline what my essay is going to say
3.	Procedural Knowledge
3.1	identifying stated information given by the lecturer	find at least 3 articles
3.2	interpreting and representing how to complete the task	or of increasing it to five if I find another article in the interim
3.3	verbalising what they are doing now	I'm now looking at the next part of the piece of paper
3.4	planning future actions	I'll probably be looking at finding three articles, three to five...obviously going to the library and just having a look under human rights
3.5	verbalising past procedures	I've read that, some of that...
3.6	commenting on task (eg how clearly it is specified)	It's fairly well laid out...telling me what we've done so far and what we're expected to do
3.7	underlining indicating key words	I'll underline that as well
3.8	justifying /stating the purpose for a particular action	because I might have to change it [choice of article]
4.	Other Comments
4.1	comments about the reader/audience	no comments were made in this category
4.2	comments about the voice/persona	no comments were made in this category
4.3	statements about self and how the student feels in relation to the task	but I don't like that statement
4.4	non-coding comments	unrelated comments which were deleted from the final analysis
4.5	comments about the question designer/lecture	...[names lecturer] who set the paper is trying to stimulate my thought activity in some way

	Whole group n=17			High Group n=5			Low Group n=5
	Mean	SD	Actual range	Mean	SD	Actual range	Mean	SD	Actual range
GPA (possible range 0-7.00)	4.77	0.85	3.11 - 7.00	5.49	0.89	4.75 - 7.00	3.96	0.76	3.11 - 5.12
Discourse task (range 0-18)	7.77	2.22	4 - 11	9.40	2.30	6 - 11	6.8	1.48	5 - 9
Essay score (%)	62.06	10.22	48% - 90%	74	9.41	68% - 90%	53	3.74	48% - 56%
Lecturer's assessment (possible range 0-33)	23.59	4.86	11 - 32	28	3	25 - 32	18.60	4.56	11 - 22

	Whole group		High Group		Low Group
	Mean	SD	Mean	SD	Mean	SD
Content	55.82	18.34	47.42	10.16	54.00	24.03
Discourse	4.47	5.43	8.58	6.53	3.60	4.96
Procedural	33.51	15.07	38.66	13.42	35.68	20.84
Other	6.14	5.47	5.14	2.16	6.62	4.72