180
Efficiency and Comprehension (TOSREC) (Wagner, Torgesen, Rashotte, & Pearson, 2010). It is
also the task type used in the PIAAC Reading Components task set (OECD, 2013a; Sabatini &
Bruce, 2009), and in two PISA countries (Bruce & Sabatini, 2013) with success. A similar task has
been used in the Austrian PISA 2000 assessment and showed high correlations (
r
= .64) with
students’ final test score (Landerl & Reiter, 2002). This task design thus has a proven empirical
foundation as an index of reading ease and efficiency in international study contexts. Task 1 in
Appendix B shows a sample item taken from the PIAAC Reading Components task.
109. While it may be possible in future cycles of PISA to use log-file data based on complex
reading literacy tasks as the sole source for measuring ease and efficiency, this option is not
recommended for the current cycle. In order to ensure that students complete tasks under
conditions that yield a valid indicator of efficiency, the design and instructions accompanying the
task should target the desired construct. The texts need to be simple and short in order to
maximize reading efficiency versus strategic or compensatory processes. In addition, the task
demands should take minimal reasoning so as to not confound individual differences in decision
time with basic reading rate information. It will therefore be difficult to ensure that the reading rates
and accuracy observed in tasks that were designed for different measurement purposes that are
executed by students under these constraints. The more complex the task, the more likely that
students will deploy strategic or compensatory processes that interfere with measuring ease and
efficiency of basic understanding.
110. Thus, it is recommended that the log files from this cycle be analysed to evaluate whether
there are indicators within the new PISA Reading Literacy task set that are strongly correlated with
the sentence level efficiency task proposed. The probability is low that there is sufficient valid
evidence in the field test log files
– essentially psychometric equivalence with the sentence task –
from initial item trials of the new reading literacy tasks. On the other hand, such log file
correlational evidence would serve as cross-validation evidence for the ease and efficiency task.
Достарыңызбен бөлісу: