Sharing Standards 2016-17: The results

In July, I will be leaving my role at Ark Schools to work for No More Marking as Director of Education. 

Over the last 6 months, No More Marking have been working with primary schools in England on a pilot of comparative judgement for year 6 writing called Sharing Standards. Comparative judgement is a quick and reliable method of marking open tasks like essays and stories. The easiest way to understand it is to try out the demo on the No More Marking website, but you can also read my explanation of it on this blog here.

The results of this pilot were published last Tuesday, and you can read the full report here.

Overall, 199 schools participated in the pilot, and a total of 8,512 writing portfolios were judged. 1,649 teachers in those schools did the judging, and the reliability of their judgements was 0.84.  This allowed for the creation of a measurement scale featuring every portfolio, and then for the application of a national gradeset: Working Towards, Expected Standard and Greater Depth. The overview report on the No More Marking website features exemplars of the portfolios at each threshold. Here’s a piece from the portfolio that was judged as the best.

80% of the judgements teachers made were comparisons of pupils in their own school. 20% were comparisons of pupils from other schools. This allowed for the creation of a national scale, but it also meant that it wasn’t possible for teachers to favour pupils from their own schools, as they were never asked to directly compare their pupils with pupils from other schools.

The other nice thing about this structure was that it allowed teachers to see tasks and pupil work from other schools. I particularly noted the popularity of tasks that asked pupils to write from the point of view of a character in a novel, and the variety of novels selected as the basis for this task. And in discussions with teachers after, it was interesting to try and pick out the aspects that made such types of writing more or less successful. Very often it was subtle uses of syntax or vocabulary that made the difference. For example, some pupils trying to capture the voice of Bruno in ‘Boy in the Striped Pyjamas’ would use the same very precise and measured sentence structure of Bruno. Others would get this right, but then fall down by using modern slang terms that just didn’t ring true.

And this brings me to the most exciting next step for comparative judgement. As Jon Brunskill writes here, once you have the fascinating data set of accurately graded portfolios, you can then ask: now what? Why are some pieces of writing better than others?  What aspects of writing matter, and how can we teach them? Of course, good teachers have always been doing this, but it’s also always been made harder by the way that traditional methods of marking writing lead to disagreement and disputes. If you can’t get reliable agreement on what good writing is, it’s obviously going to be much harder to teach good writing.

Take a look at the exemplar portfolios here and start this process yourself! Next year, No More Marking will be running similar national assessment windows for all primary year groups. See here for more details about how to participate.