Interpreting and using the scales
124. Just as students can be ordered from the least proficient to the highly skilled on a single
scale, reading literacy tasks are arranged along a scale that indicates progressively the level of
difficulty for students and the level of skill required to answer each item correctly. By comparing the
position of students and items on these scales, we can summarise both the proficiency of a person
in terms of his or her ability and the complexity of an item in terms of its difficulty.
125. Reading literacy tasks used in PISA vary widely in situation, text format and task
requirements, and they also vary in difficulty. This range is captured through what is known as an
item map. The item map provides a visual representation of the reading literacy skills
demonstrated by students at different points along the scale.
126. Tasks at the lower end of the reading scale and subscales differ from those at the higher end.
Difficulty is in part determined by the length, structure and complexity of the text itself. However,
184
while the structure of a text contributes to the difficulty of an item, what the reader has to do with
that text, as defined by the question or instruction, interacts with the text and affects the overall
difficulty. A number of variables that can influence the difficulty of any reading literacy task have
been identified, including the complexity and sophistication of the mental processes integral to the
aspect of the task (retrieving, interpreting or reflecting), the amount of information to be assimilated
by the reader and the familiarity or specificity of the knowledge that the reader must draw on both
from within and from outside the text.
Defining levels of reading literacy proficiency
127. In an attempt to capture this progression of complexity and difficulty in PISA 2000, the
composite reading literacy scale and each of the subscales were divided into six levels (Below
level 1, 1, 2, 3, 4, 5). These levels as they were defined for PISA 2000 were kept for the composite
scale used to measure trends in PISA 2009 and 2015. However, newly constructed items helped to
improve descriptions of the existing levels of performance and to furnish descriptions of levels of
performance above and below those established in PISA 2000. Thus, the scales were extended to
level 6, and level 1b was introduced at the bottom of the scale (OECD, 2012).
128. The levels provide a useful way to explore the progression of reading literacy demands within
the composite scale and each subscale. The scale summarises both the proficiency of a person in
terms of his or her ability and the complexity of an item in terms of its difficulty. The mapping of
students and items on one scale represents the idea that students are more likely to be able to
successfully complete tasks mapped at the same level on the scale (or lower), and less likely to be
able to successfully complete tasks mapped at a higher level on the scale.
129. As an example, the reading proficiency scale for the PISA 2012 study is represented in Table
4. The left-hand column shows the level number, the lower score limit, and the percentage of
students who are able to perform tasks at each level or above (OECD average). The right-hand
column describes what students can do at each level (adapted from OECD, 2013).
|