Recording events – using a research diary

Data Collection

201

Planning the data-collection process

Having decided on your approach and the data you want to collect, it is a good idea 

to plan each stage carefully before you start. The example in Table 7.1 illustrates how 

Nicky, a secondary science teacher, went about planning her research design. She asked 

the question What are students’ attitudes to writing in science lessons? She refined this 

question further and decided to try out a range of strategies in her class and look for 

changes in the students’ attitudes. She developed three questions and choose a range of 

data collection methods which she believed would help her answer these questions. Her 

overall research design is shown in Table 5.1. 

Table 5.1

 Research design of a secondary science MEd student

4 Hunches and thoughts.

5 Diagrams, drawings and mind-maps.

6 Observations.

7 Reflections on what you saw.

8 Reflections on re-reading the diary.

9 Plans for future action or research.

Data ColleCtion 

105

Planning the data-collection process

Having decided on your approach and the data you want to collect, it is a good 

idea to plan each stage carefully before you start. The example in Table 7.1 illus-

trates how Nicky, a secondary science teacher, went about planning her research 

design. She asked the question What are students’ attitudes to writing in science 

lessons? She refined this question further and decided to try out a range of strat-

egies in her class and look for changes in the students’ attitudes. She developed 

three questions and choose a range of data collection methods which she 

believed would help her answer these questions. Her overall research design is 

shown in Table 7.1. 

Table 7.1  Research design of a secondary science MEd student

What are students attitudes to writing in science lessons? 

Question

Data source

Data source

Data source

Data source

RQ1 – What attitudes do 

students have about 

science? 

Attitude 

questionnaire (Kind 

et al., 2007)

Semi-structured 

interviews (individual 

and/or group) with 

pupils

Sub question 1 – Can 

writing to an authentic 

audience other than the 

teacher improve student 

attitude to science?

Attitude 

questionnaire, before 

and after (Kind  

et al., 2007)

Semi-structured 

interviews (individual 

and/or group) with 

pupils

Analysis of written 

comments in student 

learning journals

RQ2 – What would help 

them to be better writers?

Analysis of students’ 

written work 

Semi-structured 

interviews (individual 

and/or group) with 

pupils

Analysis of written 

comments in student 

learning journals

RQ3 – Can writing to an 

authentic audience other 

than the teacher improve 

students’ learning of 

scientific concepts?

Concept map before, 

and at the end for 

each topic

Semi-structured 

interviews (individual 

and/or group) with 

pupils

Analysis of student 

written work and 

comments in student 

learning journals

Summative 

assessment 

results

Nicky carried out a survey of her class first and then selected a subsample for followup 

interview using the survey data. This method of merging data is sequential. Nicky also 

concurrently collected three other sources of qualitative data: the students’ written 

work, concept maps and learning journals. Her final data source was summative assess-

ment of students’ work (Figure 7.1). 

08-Wilson-Ch-07.indd   105

8/31/2012   5:48:00 PM

Data Collection

202

Nicky carried out a survey of her class first and then selected a subsample for followup 

interview using the survey data. This method of merging data is sequential. Nicky also 

concurrently collected three other sources of qualitative data: the students’ written work, 

concept maps and learning journals. Her final data source was summative assessment of 

students’ work (Figure 5.1).

Figure 5.1

 Meeting multiple methods

The grids in Figure 7.2 will help you to structure your research design and plan the

project timescales.

Figure 5.2 

Planning data collection

SCHOOL-BASED RESEARCH

106

The grids in Figure 7.2 will help you to structure your research design and plan the 

project timescales. 

Quantitative Data

1. Attitudes

  survey

Qualitative Data

2. Student

  interviews

Qualitative Data 

3. Pre and post

  concept maps

Qualitative Data 

4. Students’

  written work

Qualitative Data 

5. Students’

  learning journals

Quantitative Data 

6. Summative

  assessment 

Figure 7.1  Meeting multiple methods

Title:

Research question:

Sub-questions 

Data Source 

Data Source 

Data Source 

Data 

Source 

1. 

2. 

3. 

When do I need to collect the data? 

Pre-intervention?

During 

Post-intervention 

Sub-Q 1 

Sub-Q 2 

An example timescale 

Jan. – Apr. 

Apr. – Jun. 

Jul. – Aug. 

Sept. 

Review 

literature 

Essay 1 

Methods/pilot 

study 

Writing up 

Essay 2 

Data collection 

Lit. rev. chapter 

Data analysis 

Method 

chapter 

Thesis 

Figure 7.2  Planning data collection

08-Wilson-Ch-07.indd   106

8/31/2012   5:48:10 PM

SCHOOL-BASED RESEARCH

106

The grids in Figure 7.2 will help you to structure your research design and plan the 

project timescales. 

Quantitative Data

1. Attitudes

  survey

Qualitative Data

2. Student

  interviews

Qualitative Data 

3. Pre and post

  concept maps

Qualitative Data 

4. Students’

  written work

Qualitative Data 

5. Students’

  learning journals

Quantitative Data 

6. Summative

  assessment 

Figure 7.1  Meeting multiple methods

Title:

Research question:

Sub-questions 

Data Source 

Data Source 

Data Source 

Data 

Source 

1. 

2. 

3. 

When do I need to collect the data? 

Pre-intervention?

During 

Post-intervention 

Sub-Q 1 

Sub-Q 2 

An example timescale 

Jan. – Apr. 

Apr. – Jun. 

Jul. – Aug. 

Sept. 

Review 

literature 

Essay 1 

Methods/pilot 

study 

Writing up 

Essay 2 

Data collection 

Lit. rev. chapter 

Data analysis 

Method 

chapter 

Thesis 

Figure 7.2  Planning data collection

08-Wilson-Ch-07.indd   106

8/31/2012   5:48:10 PM

Data Collection

203

Figure 5.3 

Data-collection methods

REDUCING THREATS TO THE VALIDITY AND RELIABILITY OF YOUR DATA-

COLLECTION METHODS

Whichever methods you choose will involve selecting what data you are going to collect. 

During the process of becoming data, either by being written down, as in an observation 

transcript or recorded during an interview, some element of selection will have taken 

place, (Figure 7.3). Indeed, focusing on a particular approach means that you have already 

selected some aspect of reality as important while at the same time ignoring other parts. 

To some extent, you will have made a conscious decision to ignore some aspects of the 

classroom dynamic as part of interpreting the research question or choosing a particular 

methodology. However, it may also be the case that you have done this accidentally, as 

a result of your own unconscious prejudices, or some known or unknown bias of the 

methods chosen. There may also be some form of restriction in the research context, such 

as a timetabling issue which has made it impossible for you to interview a particular pupil, 

thus affecting your sample strategy, which will also introduce further bias to the process.

Furthermore, having collected the data, you will still need to be vigilant to the fact that 

selecting the source of data is in itself an interpretative process and that the next stages 

of making meaning from the data will extend this interpretation process even further, 

particularly if you intend to transform your own personal experiences into data. So there 

are many pitfalls along the way which will compromise the reliability and validity of your 

data-collection process.

The next sections will set out the advantages and disadvantages of the various 

datacollection

methods which you could use in school-based research. The nature of your

Data ColleCtion 

107

Reducing threats to the validity and reliability of your  

data-collection methods

Whichever methods you choose will involve selecting what data you are going to collect. 

During the process of becoming data, either by being written down, as in an observation 

transcript or recorded during an interview, some element of selection will have taken 

place, (Figure 7.3). Indeed, focusing on a particular approach means that you have 

already selected some aspect of reality as important while at the same time ignoring 

other parts. To some extent, you will have made a conscious decision to ignore some 

aspects of the classroom dynamic as part of interpreting the research question or choos-

ing a particular methodology. However, it may also be the case that you have done this 

accidentally, as a result of your own unconscious prejudices, or some known or 

unknown bias of the methods chosen. There may also be some form of restriction in the 

research context, such as a timetabling issue which has made it impossible for you to 

interview a particular pupil, thus affecting your sample strategy, which will also introduce 

further bias to the process.

Furthermore, having collected the data, you will still need to be vigilant to the fact that 

selecting the source of data is in itself an interpretative process and that the next stages 

of making meaning from the data will extend this interpretation process even further, 

particularly if you intend to transform your own personal experiences into data. So there 

are many pitfalls along the way which will compromise the reliability and validity of your 

data-collection process.

Record what is

happening

Direct

observation

Video or audio

tape

Ask about what

is happening

Interview

participants

Use surveys and

questionnaires

Use standardized

tests

Other sources

of evidence

Visual images

Documentary

analysis

Figure 7.3  Data-collection methods

08-Wilson-Ch-07.indd   107

8/31/2012   5:48:21 PM

Data Collection

204

research question will determine what the most appropriate data is to collect. However,

one thing to avoid is the tendency of collecting too much data. It will probably be just

you who will have to analyse all the data collected in a finite amount of time. Activity 5.2

will help guide the planning stage.

Activity 5.2 

Data-collection checklist

Planning: Who, where, when, how and what?

Have you considered the following?

1.  Who or what are you going to include in the data-collection process and 

why?

2.  Do you have a clear plan for where and when you will collect data?

3.  Will you collect data in your own classroom or that of a colleague? Have 

you established what your role will be during the data-collection process? 

Are you a teacher or researcher or both?

4.  Have you identified and set in place measures to help you control your own 

possible subjectivities?

5.  Have you consulted the BERA ethical guidelines? Are you aware of any 

potential ethical dilemmas?

6.  Have you read the relevant sections in this chapter on how to collect each 

data source?

7.  Have you considered how you will minimize the threats to the validity of 

your data-collection methods?

8.  Are you sure that the data you intend to collect will help you answer your 

research question?

9.  Do you have a back-up plan in case there are last-minute changes in school?

Preparing and piloting the data-collection tools 

10.   Can you use an existing standardized test, survey or questionnaire?

11.   Have you prepared your interview schedule, if applicable?

12.   Have you checked that your audio-visual equipment is working, if you    

intend to use this approach?

13.  Have you tried out your survey or interview questions on a small pilot     

 group?

14.  Do you need to modify your data-collection plan in light of your pilot?

Data Collection

205

RECORDING WHAT IS HAPPENING 

Observations 

Observation is a normal process for teachers. Indeed, teachers are continually monitoring 

what is going on in classrooms and constantly questioning what is happening. Reactions 

and responses are often informed by teachers’ tacit experiential knowledge which helps to 

interpret what is going on. This tacit experiential knowledge is constantly being developed, 

and indeed this may well be why you are embarking on a classroom-based research project. 

Consequently, if you want to research your own classroom, you will need to be able to stand 

back and open your mind to other ways of interpreting what is going on in your classroom. 

In other words, you will need to be systematic in how you collect evidence about what is 

happening in your classroom.

Evidence collected through observation will rely on your ability to gather data through using 

all your senses, and to do this you might use checklists, inventories or narrative descriptions 

as tools to help your analysis.

Classrooms are very complex social settings and there are many things going on at once. 

You need to have a clear purpose when you come to observe the classroom, otherwise you 

might end up with a very superficial analysis if you try to record everything. On the other 

hand, if you limit your focus too much, there is a danger that you only see what you want 

to see. Remember that when you observe a classroom you are selecting from an ongoing 

sequence of events, so plan carefully for this opportunity. Activity 5.3 provides a checklist of 

points to consider before you carry out an observation.

Activity 5.3 

Preparing to observe

Planning: Who, where, when, how and what?

1.  What are you going to observe? Will you focus on a particular aspect of 

practice? Will this allow you to collect enough data to answer your question?

2.  Why are you carrying out the observation? What are the assumptions and 

expectations on which it is based? Are you sure your biases are not ruling 

out another interpretation of the situation?

3.   When will the observation be carried out, and how long will it take? Will 

this fit in with the timetabling schedule and other whole school events?

Data Collection

206

Recording observations

Decide before the observation event what your approach to recording what you observe 

will be.

Figure 7.4 

Classroom Observation schedule

Lesson aims:

 

Year gr

oup:

 

Number of students:

Teac

her–student interaction 

K

ey:

 Mgt = management or admin, Inst = Giving instr

uctions

, A, B

, C 

or D type dialogue

, Demo =demonstr

ating, Explan = e

xplaining,

 

Quest = questioning

What students are doing 

Ho

w students are 

gr

ouped 

Time 

Whole class:

Teacher addressing the

 

whole class 

Teacher inter

acting

 

with small g

roups of students 

or individuals 

0–5 

mins 

Mgt 

Inst 

A 

B 

C 

D 

Demo 

Mgt 

Inst 

Explan 

Quest 

O 

Wr

ite 

Listen 

Talk 

Read 

Pr

actical 

Other 

individual 

P

air 

Group 

Figur

e 

7.4

 

Classr

oom 

Observ

ation 

sc

hedule

08-Wilson-Ch-07.indd   110

8/31/2012   5:48:21 PM

Data Collection

207

Classroom observation methods include a wide range of approaches: checklists, inventories, 

timed  interval  ratings,  holistic  ratings,  narrative  descriptions,  logs,  questionnaires,  rubrics, 

matrices, models, conceptual grids, and open-ended questions.

Another important consideration is to agree what time period you will observe and why. 

Will you observe an entire lesson? A segment? An entire day?

Figure 5.4 provides an example of a structured observation schedule used to analyse talk in 

science lessons. 

Challenges

Constructing a classroom observation schedule is quite a difficult process, especially if you are 

going to ask someone else to use your schedule. This is because each observer will approach 

the classroom with his or her own experiences and biases. As a result, two observers may 

interpret what is going on differently and may then record different responses to the same 

lesson.

Moreover, the presence of one or more observers may interrupt the normal class 

environment which can lead to lessons that do not represent the norm. Table 7.2summarizes 

the advantages and disadvantages of observing lessons. 

Table 5.2

 Advantages and disadvantages of using observation and audio/video recording

Also, Read the thought experiment too in Box 7.1, which challenges the notion that

observation can reliably assess how well students are learning in classrooms.

Data ColleCtion 

111

Classroom observation methods include a wide range of approaches: checklists, 

inventories, timed interval ratings, holistic ratings, narrative descriptions, logs, question-

naires, rubrics, matrices, models, conceptual grids, and open-ended questions.

Another important consideration is to agree what time period you will observe and 

why. Will you observe an entire lesson? A segment? An entire day? 

Figure 7.4 provides an example of a structured observation schedule used to analyse 

talk in science lessons. 

Challenges

Constructing a classroom observation schedule is quite a difficult process, especially if 

you are going to ask someone else to use your schedule. This is because each observer 

will approach the classroom with his or her own experiences and biases. As a result, two 

observers may interpret what is going on differently and may then record different 

responses to the same lesson.

Moreover, the presence of one or more observers may interrupt the normal class 

environment which can lead to lessons that do not represent the norm. Table 7.2  sum-

marizes the advantages and disadvantages of observing lessons.

Table 7.2  Advantages and disadvantages of using observation and audio/video recording

Advantages of this approach 

Disadvantages of this approach 

Written observations 

Immediate account available; observer 

actually sees what is happening; account 

can be available for discussion immediately 

after observation; full view of the classroom 

or school setting available to the observer 

at time of observation 

Observer must make immediate decisions 

about what to record, so may be superficial 

or unreliable account; no chance of ‘action 

replay’; some effects on class behaviour 

because of observer’s presence 

Video recording 

Both visual and sound record of the lesson 

which can be replayed several times; no 

pressure to make instant decisions; focus 

can be on teacher only or on individual or 

group of pupils; lesson can be discussed 

with participants 

Loss of information such as room 

temperature, smells, events out of camera 

shot; effects on class of presence of camera; 

time-consuming analysis; one camera 

provides one perspective 

Audio recording 

Sound recording can be replayed several 

times for discussion, analysis, or 

corroboration of written account; radio 

microphone can be used to obtain 

high-quality record of what the teacher 

says; observer’s comments can be recorded 

simultaneously on twin-track tape; allows 

lesson to be subsequently transcribed 

Loss of important visual cues such as facial 

expressions, gesture, body language, 

movement; sound quality can be poor 

without radio microphone, especially if 

acoustics are poor; difficult to identify 

individual children who speak; analysis time 

substantially increased 

08-Wilson-Ch-07.indd   111

8/31/2012   5:48:21 PM

Data Collection

208

жүктеу/скачать 32,4 Mb.

Достарыңызбен бөлісу: