KS2 Reading SAT

6 Jun 2020

Reading Strands are Meaningless

Teachwell Assessment, Education Policy, Teaching and Learning KS2 Reading SAT, Reading, Reading Skills, Reading Strands 0

This is the fourth blog in the series – the first covers the overall issues with the KS2 Reading SAT, the second whether it tests reading and the third how it embeds the teaching of generic comprehension reading skills.

In this blog, I will examine what the KS2 Reading SAT is testing and whether this is useful to support the teaching of reading.

The Reading Strands

Here’s a short quiz to begin with.

If you are struggling with spotting the difference, it’s because there really isn’t one.

The reading strands are reinterpretations of the attainments targets in the National Curriculum (see blog 2). This enabled the Standard and Testing Agency to continue using their test item database which they had been building up since 2012.

I’ll hold my hands up, both as a SATs marker and a teacher. I knew the questions were familiar and noticed the anomalies with question stems but didn’t look at the coherence of the reading strands overall.

What is the KS2 Reading SAT really measuring?

Analysing the questions from the SAT from 2016 to 2019 it’s clear that they come mainly from the old Assessment Focuses 2 and 3 from the previous testing regime.¹ Few new types of questions have been formulated to test the newer parts of the reinterpreted attainment targets.²

2a – Vocabulary:

The vocabulary questions involve retrieval, inference or deduction. The concern here is the deduction questions. Contextual clues were only available for 44% of the deduction questions asked. Clearly they are not constructed with consideration as to whether deduction is possible.

It is not clear how data in this strand could be used by teachers when only 9% of the questions require knowledge from the National Curriculum. Also there is a focus on asking about more obscure meanings of vocabulary (e.g. spat) or for words they are unlikely to have encountered (e.g. illegal, recesses). The only purpose for this is to see who can be counted as a greater depth reader – a meaningless term which I will look at in my next blog.

2b – Retrieval:

Retrieval, in so far as it is a skill, simply involves knowing that the text contains the answers and extracting what is needed to answer a question. This takes little time to teach (my Year 2’s picked this up in the space of 4-6 weeks and over the course of the year they didn’t need reminding to use the text to answer questions). Beyond that the ability to answer retrieval questions depends on whether the pupil has understood the question and made meaning from the text. This is all dependent on knowledge.

To be fair this is the only strand that has some coherence in terms of the questions asked. 87% of the questions are retrieval while the rest are inference. 34% of the questions involve some paraphrasing of the sentence to support pupils to find the answer. While paraphrased questions have their issues (easier for pupils to retrieve the word/phrase for the answer), the use of synonyms used to avoid paraphrasing can also cause issues. It is difficult to know in such cases whether it was the wording of the question that was not understood or the text.

2c – Summary:

This strand is a missed opportunity. Instead of asking the children to summarise in their own words, they are asked to select the correct summary. This has only occurred twice. Puzzling there are more sequencing questions – which are actually retrieval questions – in this strand.

2d – Inference:

As discussed in the last blog, inference is not a skill that can be applied to any text, it requires background and domain specific knowledge.

Writers don’t write for the convenience of teachers or for test creators who want fortyish percent of the marks for an inference reading strand.

This insistence on weighting would appear to be the reason for reinterpreting the National Curriculum attainment target to include finding evidence for inferences that are made in the question. These are not inference questions and are no different to non-paraphrased questions for retrieval. While there is merit in finding evidence from the text to support an assertion, the ability to do so for an inference already made tells us nothing about whether a pupil can make an inference.

Only 43% of the questions in this strand give any information about the pupils ability to make inferences. The rest are retrieval questions. The inclusion of fact or opinion questions in this category makes little sense. This is about knowledge and knowing

Why this strand can’t just be weighted less and more coherent is a question for the design makers. I can only guess that the need to make the test appear challenging and therefore in line with the expectations of the National Curriculum, they feel they have to maintain this strand as the one with the greatest weighting.

2e to 2h:

2e – Prediction: Only one prediction question has ever been asked. There is no merit in testing it as a skill (see blog 3).

2f – Meaning of the Text: These are ‘understand’ questions relating to the whole text.

2g – Choice of Language: These are either retrieval or inference questions. The former have involved finding evidence and the latter explaining the meaning of quotes.

2h – Comparison within Texts: To say this is making comparisons within the text is a stretch and while the questions do provide useful information for retrieval and inference – they don’t really provide any information about the comparison within longer texts or between them which was the aim of the attainment target in the National Curriculum.

Breakdown of the strands by Generic Reading Comprehension Skills (2016-2019):

What this data highlights is that the reading strands are meaningless. Extracting data from the KS2 Reading SAT or indeed Reading style SATs for other year groups requires deeper analysis of the questions and answers to be of any use to teachers.

Implications for Teaching Reading

There are no generic reading comprehension skills and organising teaching around reading strands is pointless. The focus should be on teaching pupils the knowledge they need to enable them to extract meaning from the text.

In terms of reading lessons, there is evidence that there is some benefit to teaching reading strategies – e.g. skimming, re-reading, summarising to support pupils comprehension.³ But as Willingham points out, even the reading strategies that do work to support comprehension are just a ‘bag of tricks’. The one time boost they deliver is not a substitute for teaching the knowledge pupils require to extract meaning from the text.⁴

The questions stems and answer formats themselves are useful and can be adapted for use in general teaching.

While we have the KS2 Reading SAT I think that we need to adopt a pragmatic stance to it.

The following gives a breakdown of questions and answers types.

The green highlights those question and answer formats that come up the most frequently and give teachers the most information. The orange are encountered less frequently but are useful to include from Year 5 onward as children will have the knowledge to be able to do this usefully. The light red do not require much focus and should be left to test practice.

The bright red is purely for test strategy of last resort. While many teachers would ask questions and point out contextual clues that would help to infer an unknown word as part of teaching a text, there is little to be gained from teaching it as a strategy or a skill. The pupils who are picking up on this, do so without much teaching and those who aren’t lack knowledge (see blog 3).

Teaching Reading

We have removed the guessing component from word reading. It’s time to do this with reading comprehension too. The following diagram is an attempt to represent what the focus for each stage of teaching reading should be:

We need to check progress at each stage and not assume. Years 2 to 3 are really crucial for many children. This is the point where they are expected to move from phonics to comprehension. One of the main reasons many struggle is due to lack of reading fluency.

This model has implications for internal assessment too.

What aspect of reading are we testing and when?
Are we tracking progress using SAT style reading tests across year groups?
Can teachers use test results to improve teaching or decide on suitable interventions for pupils?
When we know that background and domain knowledge matters then what is the purpose of using tests containing random extracts unrelated to the National Curriculum?

The last question is going to be the focus of the next blog – the use of unseen extracts with mostly unknown content in reading assessments.

References:

^{1. Assessment Focus 2 (understand, describe, select or retrieve information, events or ideas from texts and use quotation and reference to text) and Assessment Focus 3 (deduce, infer or interpret information, events or ideas from texts)}

^{2. My analysis is qualitative and will suffer to some degree from subjectivity but I am happy to share the data analysis of the questions. While select was a category, in effect it broke down into retrieval and inference so I assigned those categories.}

3. Quigley, A, (2020), The Reading Gap, Oxon, Routledge.

^{4. Willingham, D.T (2006/2007) The Usefulness of Brief Instruction in Reading Comprehension Strategies, https://www.aft.org/sites/default/files/periodicals/CogSci.pdf}

23 May 2020

Perverse Incentives: The KS2 Reading SAT and the Myth of Generic Reading Comprehension Skills

Teachwell Assessment, Education Policy, Teaching and Learning Deduction, Generic Reading Comprehension Skills, Inference, KS2 Reading SAT, Myth, Prediction, Question Types, Reading, Reading Comprehension 0

This is the third blog in the series on the Key Stage 2 Reading SAT. The first one outlining the series is here and the second on whether the KS2 Reading SAT is testing reading is here.

The Reading SAT and Generic Reading Comprehension Skills

Reading comprehension is ultimately making meaning of what is being read.

For most of my teaching career, reading was synonymous with teaching generic reading comprehension skills to pupils and expecting them to ‘master’ them. This is based on the idea that readers make meaning by applying a set of skills to a text.

In the past this was driven by the assessment focuses and the Reading SATs in KS1 and 2 were a reflection of these.

While the KS1 Reading SAT is being replaced by the baseline,the KS2 Reading SAT will remain. Unreformed, it will continue to be the driving force behind the practice of teaching generic reading comprehension skills in primary schools.

There are three reasons why the KS2 Reading SAT can and does lead to poor practice in teaching reading:

1. It is underpinned by a belief that generic reading comprehension skills exist and they can be measured.

2. The weighting given to different assessment focuses in the past and reading strands now skews the teaching of reading toward generic reading comprehension skills.

3. The inclusion of unseen extracts with (varying levels of) unknown content is a direct result of the belief that application of reading skills – not the content of the text – results in comprehension. (This will be tackled in a later blog.)

The National Curriculum and Generic Reading Comprehension Skills

There are attainment targets in the National Curriculum that state pupils should be able to demonstrate their understanding of texts in particular ways such as making predictions from what is stated and implied. While their inclusion was, with hindsight, a mistake, they are not stated as skills to be mastered.¹

As outlined in the previous blog, it is the Standard and Testing Agency (STA) that has selected attainment targets from the National Curriculum, interpreted them as skills to be measured and converted them into reading strands. The test design divides the reading strands into generic reading comprehension skills (reading strands 2a – 2e) and analysis of the text (reading strands 2f – 2h).

Weightings

The attainment targets in the National Curriculum have equal weighting but their reading strand equivalents do not.

Under the old testing system the weightings were more equally distributed between generic reading comprehension skills and analysis of the text.

Average Percentage for Assessment Focuses (2003-2015)

AF2

Understand, describe, select or retrieve information, events or ideas from texts and use quotation and reference to text

AF3

Deduce, infer or interpret information, events or ideas from texts

AF4

Identify and comment on the structure and organisation of texts, including grammatical and presentational features at text level

AF5

Explain and comment on writers’ uses of language, including grammatical and literary features at word and sentence level

AF6

Identify and comment on writers’ purposes and viewpoints and the overall effect of the text on the reader

AF7

Relate texts to their social, cultural and historical contexts and literary traditions

23%

40%

12%

13%

63%

37%

Unlike the reading strands, the AFs relating to generic reading comprehension skills, consisted of several skills that needed to be taught. AF3, which included deduction and inference questions, was given the greatest weighting. These were assumed to be the most important ‘skills’ good readers demonstrated.

The combination of AF2 and AF3 were enough to enable a pupil to gain a Level 4 in the old assessment system. This is not to say the other assessment focuses were not taught, but, in my experience, the weighting impacted on the time devoted to each in guided reading lessons. It also informed the content of interventions for those struggling to read.

Average Percentage for Reading Strands (2016-2019)

Give/explain the meaning of words in context.

Retrieve and record information /identify key details from fiction and non-fiction.

Summarise main ideas from more than one paragraph.

Make inferences from the text/explain and justify inferences with evidence from the text.

Predict what might happen from details stated and implied.

Identify/explain how information/ narrative content is related and contributes to meaning as a whole.

Identify/explain how meaning is enhanced through choice of words and phrases.

Make comparisons within the text.

18%

31.5%

3.5%

40%

1.5%

94.5%

5.5%

The main difference between the old and new Reading SAT is that the current one is even more of a generic reading skills test than the previous one. Again inference questions are given the greatest weighting. The reading strands are meaningless due to the shoehorning of questions from the test item bank – thus half the vocabulary questions involve deduction or inference (this will be examined in the next blog).

Typically, the weightings of the reading strands drive what is considered to be important when teaching reading comprehension throughout Key Stage 2.

Moreover, in order to track progress, SAT style tests with similar weightings are used across other year groups. This shapes the priorities in teaching reading comprehension in all year groups despite the research and evidence in this field. Thus, schools are inexorably wedded to the teaching of generic reading comprehension skills as a direct result of the Reading SAT.

But why are generic reading comprehension skills being taught and assessed?

The Myth of Generic Reading Comprehension Skills

The belief that generic reading comprehension skills exist first arose in the 1950s.²

Identifying what ‘good’ readers could do which ‘poor’ readers couldn’t does seem like a logical starting point to bridge the gap. Claims about what good readers could do include:

They make predictions as they read.
They deduce the meaning of unknown words either by word substitution or using context cues.
They are able to make inferences.

But do they?

Prediction

The idea that good readers predict or guess what might happen next appears to have no research base at all and only appeared as a skill in the 1980s.³

Deduction

This ‘skill’ is the epitome of a lethal mutation. In an era which prized pupils discovering learning for themselves above explicit instruction, the idea of pupils deducing unknown words was a seductive one.

Good readers can deduce/infer around 15% of the unknown words they encounter.⁴ Despite the low percentage, the increase in vocabulary acquisition can be substantial depending on the number of books a child reads.

It does not, however, constitute a skill good readers have mastered and can apply regardless of the word or text they encounter.

This ability to deduce the meaning of some unknown words mutated into teaching strategies that would lead to the acquiring of the ‘skill’ of deduction.

Word Substitution

The idea that good readers substitute words for unknown words, like predicting, appears as a reading strategy but, in so far as my investigation took me, lacks an evidence base. It also doesn’t seem to bear any resemblance to reality.

As skilled readers, we do not waste time substituting unknown words with alternatives, we look the word up! It’s only in a test situation that substitution might be a useful. Even then its use is limited; it depends on extracting meaning from the rest of the sentence and having the vocabulary knowledge that would enable a substitution.

In normal reading circumstances, substituting words is a poor strategy to teach struggling readers. It stalls the reader and wastes their time. It can lead to wild guessing which is counter-productive to the goal of making meaning from what they are reading. Worse still, it can embed the idea that reading is guessing in the very pupils who are struggling the most.

Reading on, asking the meaning of the word or looking it up are clearly better strategies.

Context Clues

Good readers can use context clues such as definitions, examples and synonyms to deduce unknown word meanings.⁵ But only if they are included, which is not always the case.

All writers make assumptions about the vocabulary and background knowledge the reader possesses. Most of the time writers don’t write with the purpose of supporting the reader to deduce the meaning of potentially unknown words. Even when they do, for example by including glossaries, the exact words that are unknown to the reader will vary from person to person.

Whether the context clues are incidental or intentional, using them involves having the vocabulary and background knowledge to understand the context clue in the first place.

The evidence shows teaching poor comprehenders to use context cues to understand unknown words does not result in increased vocabulary acquisition. This is due to two factors: they lack vocabulary and because their understanding of the vocabulary they have is less well developed.⁶

The truth is that the vast majority of the time, good readers, like poor readers, don’t deduce the meaning of unknown words either! I doubt anyone would advocate turning this into a skill and teaching it.

Inference

The teaching of inference as a skill is also questionable.

Good readers can make inferences if they have made meaning from the sentences written and if they have the background knowledge to be able to fill in what the writer has omitted. Thus it is not the skill but the knowledge that needs to be taught.

Knowledge not Skills

Once we stop thinking of readers in absolute categories of good and poor readers, the comprehension process makes more sense. ‘Good’ readers understand more of what they are reading more of the time. They, like poor readers, will have vocabulary and knowledge gaps that need addressing too.

Ultimately, comprehension is about making meaning from the writing being read, which is determined by the reader’s pre-existing vocabulary and knowledge, not the ability to apply a set of skills to a text.

At best, what is thought of as generic reading comprehension skills should just be seen as types of questions, some of which can be used to ascertain if a pupil comprehends what they’ve read.⁷ At worst, generic reading comprehension skills are a barrier to the effective teaching of reading.

Yet, many will claim, correctly, that teaching generic reading comprehension skills does lead to improvements in reading comprehension tests

Generic Reading Comprehension Skills and Test Scores

If the skills are actually types of questions then clearly teaching them would improve scores in a test that contains these questions and measures the ability to answer them.

Teaching generic reading comprehension skills can improve test scores in the following ways:

1) Pupils are taught to read the question carefully to avoid mistakes such as ticking two boxes instead of one.

2) Teaching involves examining the vocabulary and concepts contained in questions. If a pupil does not know what ‘retrieve’ means, has no concept of what ‘a reason’ is or does understand what a ‘prediction’ involves, then teaching them this means they can at least have a go at answering the question.

3) It gives children exposure to the kinds of answers they are expected to give and what to write e.g. the different formats such as short and long answers, multiple choice, numbering when ordering or sequencing. Again the outcome is that pupils are more likely to attempt to answer the questions.

These gains in tests should not, however, be mistaken for improvements in comprehension. Teaching generic reading comprehension skills in reading lessons is just teaching to the test.

Such lessons can include some benefits such as exposure to reading different texts and therefore new vocabulary. However, the extent to which this leads to long term vocabulary acquisition, development of word knowledge or improved comprehension when reading other texts is severely limited because the focus remains on acquiring and practising skills, not knowledge.

Once the issue of knowing the question type and answer format are put to one side, the inability of pupils to answer these questions is dependent on vocabulary and background or domain knowledge needed to comprehend the text.⁸ Repeated practice does not lead to gains in test scores or comprehension.

Measuring Question Types is not Measuring Comprehension

We cannot equate the ability to answer such questions with reading comprehension because it is only one way of assessing comprehension. It is also possible to assess comprehension by asking pupils to draw and/or annotate a picture or diagram for example.

Measuring the ability of pupils to answer different types of questions, as the KS2 Reading SAT does, is measuring one of the means of assessing comprehension, not an assessment of comprehension itself.

Reading comprehension is a test of knowledge not of skills. Pupils’ ability to demonstrate their comprehension depends on their ability to understand the meaning of what they read. This is, in turn, dependent on the vocabulary, domain knowledge and background knowledge of the content that they already possess, not their ability to apply generic skills to an unseen text.

In the next blog I will delve into the ‘skills’/question types the KS2 Reading SAT is actually testing (spoiler – they don’t match the reading strands!) and comparing the idea of generic reading comprehension skills (which don’t improve comprehension) and reading strategies which might be of some, albeit limited, use when teaching reading.

¹In my opinion, The National Curriculum programmes of study for English need to be reframed into basic English which is required to become literate and then English Literature and Language. This distinction is important in primary as it separates what is generic and what is subject specific. This would allow for greater alignment with secondary.

²https://shanahanonliteracy.com/blog/comprehension-skills-or-strategies-is-there-a-difference-and-does-it-matter

³The Reading Ape does a thorough analysis of the research field in this blog: https://www.thereadingape.com/single-post/2020/05/02/I-predicta-riot

^{4 Swanborn, M.S.L. and de Glopper, K. (1999) Incidental Word Learning While Reading: A Meta-Analysis. https://journals.sagepub.com/doi/10.3102/00346543069003261}

^{5 Dechant, E. (1991) Understanding and Teaching Reading: An Interactive Model: An Interactive Approach. New Jersey, Routledge.}

^{6 Shelfbine, J.L.(1990) Student Factors Related To Variability In Learning Word Meanings From Context https://journals.sagepub.com/doi/10.1080/10862969009547695}

^{7 https://shanahanonliteracy.com/blog/comprehension-skills-or-strategies-is-there-a-difference-and-does-it-matter}

^{8 The role of knowledge in comprehension has been highlighted by Hirsch, E.D., (2016), Why Knowledge Matters: Rescuing Our Children from Failed Educational Theories. Massachusetts: Harvard Education Press and Willingham, D.T., (2009), Why Don’t Students Like School? A Cognitive Scientist Answers Questions About How the Mind Works and What It Means for Your Classroom. San Francisco: Jossey-Bass, among others.}

9 May 2020

Is the KS2 Reading SAT testing reading?

Teachwell Assessment, Education Policy, Knowledge-Based, Teaching and Learning attainment targets, English, KS2 Reading SAT, National Curriculum, Standard and Testing Agency, validity 0

I started this series with this a blog outlining the main issues with the Key Stage 2 (KS2) Reading SAT, why it needs reforming and possibly how here.

For the sake of needing to start somewhere, let’s look at the issue of validity. Kime states a valid assessment a) tests what it claims to and b) provides information that is valuable or appropriate to its purposes.¹

It would seem obvious that the KS2 Reading SAT is a test of reading.

What is Reading?

Reading has 5 key components:²

1) Phonemic Awareness

2) Phonics

3) Fluency

4) Vocabulary

5) Comprehension

Comprehension requires the other four components and the necessary domain or background knowledge.³

The National Curriculum English programmes of study for KS1 and KS2 cover the component parts but do not always make them statutory.

The teaching of reading in the National Curriculum is split into word reading, which covers phonemic awareness, phonics and fluency and comprehension, which covers vocabulary and comprehension.

Does the KS2 Reading SAT cover these component parts?

What is the KS2 Reading SAT testing?

According to the STA test framework and handbooks, it is testing the attainment targets in the English programme of study in the National Curriculum.⁴ However, it only refers to the Year 5 and 6 attainment targets.

One would assume, in so far as possible, it is testing the objectives we are expected to teach for reading comprehension.

But it’s not that simple.

To be fair to the test designers, this is not a task that has been made straightforward. The programme of study is wooly in places and the role of knowledge in comprehension is not as clearly outlined as it should be.

The specific knowledge in the comprehension part of the programme of study was kept to a minimum (unlike for word reading). This was to allow schools the freedom to develop their own school curriculum.

How a national test could be devised to assess comprehension based on locally based curricula was not addressed.

While the STA’s job has undoubtedly been made difficult, they have not attempted to design a test based on the evidence on generic reading comprehension skills or the role knowledge plays in comprehension. This should have guided their rationale when selecting attainment targets to assess but it didn’t.

Instead, as Ofqual point out:

“Whereas the Test Framework for mathematics is essentially a re-presentation of the national curriculum framework, the Test Framework for reading is more of a re-interpretation. Indeed the eight strands also share some continuity with the seven assessment focuses previously used by STA to represent the content domain for earlier national curriculum reading tests, to support test development and marking.”⁵

In short, the STA have recreated the old assessment in so far as it can and continues to use questions from the item bank they set up in 2012. This has allowed the belief that comprehension is composed of a set of generic reading comprehension skills that can be transferred to different texts to continue to underpin test design and text selection. (I will return to the issue of the Reading SAT being untethered to the curriculum as well as the issues of breadth vs depth in a later blog).

The old assessment focuses (AF’s) were a mixture of generic reading comprehension skills (e.g. retrieval and inference) and evaluation of the text (e.g. writer’s choice of language, grammatical features and context of the text).⁶The latter are more closely linked to secondary objectives for English as a subject (a mixture of English Literature and Language).

The revised KS2 Reading Test Framework includes five reading strands measuring reading comprehension skills compared to the previous two AF’s. There are three reading strands assessing evaluation of the text compared to previous four AF’s.⁷

How were the attainment targets that are measured selected and reinterpreted?

The validity frameworks for the Reading SAT include their rationale behind the attainment targets they don’t assess but not the ones they do.⁸

Clearly attainment targets such as ‘recommending books they (i.e.pupils) have read to their peers, giving reasons for their choices’ can’t be selected.

However, it is less clear why identifying themes and conventions can’t. They give no reason for themes, while they argue assessing conventions would require the teaching of particular genres of texts.

This ignores the fact that particular genres are expected to be taught e.g. myths. However it’s true that it’s not possible to assess conventions for other categories such as fiction from our literary heritage as these do not constitute a genre.

To digress for a short while, they claim that it’s too subjective to test the understanding of a wider range of fiction, poetry, plays, non-fiction and reference or textbooks. Yet the selection of texts for the Reading SAT is made from fiction, poetry and non-fiction. When attainment targets for both poetry and plays relates to performance not analysis, it is not clear why poetry is included and not plays other than their overriding concern to ensure continuity with previous tests.⁹

This concern explains their contradictory rationale – they don’t want to assess conventions because it would mean telling schools what genres to teach, yet the inclusion of poetry texts de facto means schools have to teach analysis of it even though it’s not an attainment target.

It also explains the arbitrary nature of the part of the attainment target they focus on. When they excluded themes and conventions they decided the attainment target was about maintaining positive attitudes.

Yet they divorce ‘explore the meaning of words’ from the overall target which involves (pupils’) checking the book makes sense to them and discussing their understanding.

From analysis of the vocabulary questions on the Reading SAT it’s clear that they thought they could make this attainment target fit their existing item bank. In doing so they have distorted the attainment target from exploring the meaning of words to knowing and giving the meaning of words which involves retrieving a word or deducing the meaning of a word (essentially guessing the meaning of the word from the context. This will be examined further in the next blog).

Why does it matter?

The attainment targets selected matter, as does the reinterpretation of attainment targets, as it determines the nature and scope of the assessment and its validity.

From 2003 to 2015, on average 63% of the marks were awarded for AF’s assessing reading comprehension skills in the Reading SAT.¹⁰ This has sky-rocketed since 2016. From 2016 to 2019, on average 94.5% were awarded for reading strands assessing reading comprehension skills.¹¹

Clearly the Reading SAT is not valid, the selection and reinterpretation of attainment targets means it does not assess what it claims to. By ignoring the role knowledge plays in comprehension and focusing so heavily on measuring reading comprehension skills it does not provide valuable or useful information about the attainment targets or reading.

In the next blog I will focus on how the Reading SAT continues to drive the teaching and practice of generic reading comprehension skills at the expense of fluency, comprehension strategies and ultimately the goal of raising achievement in reading.

Footnotes:

¹ Kime, S., (2017), Four Pillars of Assessment Validity (https://evidencebased.education/pillars-assessment-purpose-validity/)

^{2 https://fivefromfive.com.au/}

³The role of domain and background knowledge has been identified by multiple researchers including Hirsch, E.D., (2016), Why Knowledge Matters: Rescuing Our Children from Failed Educational Theories. Massachusetts: Harvard Education Press and Willingham, D.T., (2009), Why Don’t Students Like School? A Cognitive Scientist Answers Questions About How the Mind Works and What It Means for Your Classroom. San Francisco: Jossey-Bass.

⁴2016 Key Stage 2 English Reading Test Framework – https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/628816/2016_KS2_Englishreading_framework_PDFA_V3.pdf

National curriculum test handbook: 2016 and 2017 Key stage 1 and key stage 2 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/670132/NCT_Handbook_final.pdf

National curriculum test handbook: 2018 Key stage 1 and key stage 2 https://www.gov.uk/government/publications/2018-national-curriculum-test-handbook

National curriculum test handbook: 2019 Key stage 1 and key stage 2 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/868425/2019_national_curriculum_test_handbook.pdf

⁵ Content validation study: 2016 key stage 2 reading and mathematics tests https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/649573/Content_validation_for_mathematics_and_reading.pdf

⁶Assessment Focus 1 – Use a range of strategies including accurate decoding of text, to read for meaning – was not assessed in the KS2 Reading Test. https://webarchive.nationalarchives.gov.uk/20090327072110/http://www.standards.dfes.gov.uk/primaryframework/assessment/app/waaf

⁷

⁸Appendix F: 2019 Validity framework Key stage 2 English reading https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/868437/Appendix_F_2019_validity_framework_key_stage_2_English_reading.pdf (pp. 5-7)

⁹ So far only poem has only been included in the 2018 Reading SAT paper.

¹⁰The average of the percentage of marks awarded to AF2 and AF3.

¹¹The average of the percentage of marks awarded to Reading Strands 2a to 2e (vocabulary, retrieval, summarisation, inference and prediction). Vocabulary is not a generic reading comprehension skill but the kind of vocabulary questions asked expect pupils to retrieve or deduct meaning of the word.