Why did Johnny and Jenny's exam grades yo-yo over the summer? Here's some of the code behind UK results chaos

UK exams body Ofqual has published the code behind the summer's results fiasco. Sort of. Like a 1980s hobbyist keenly keying in source from Home Computing Weekly and hunting for that sneaky bug or typo, you too can now peer at reams of code and wonder just how it all went so wrong. Assuming the statistical language R is your …

[Anonymous Coward]

"High-performing students in poorly performing schools were at risk of losing university places after they received A Level grades far lower than they expected, whereas private schools saw a significant increase in their student's estimated grades."

But from what I understood, the algorithm was not completely abandoned. IIRC no-one was downgraded from the grades that went up....

Does anyone know any different?

[Dan 55]

There's an explanation here, it seems every step in the calculation used variables which favoured small private schools and students' real ability had nothing to do with their final grades.

[sad_loser]

well done for publishing

I work in public services and I applaud the fact they have published this.

If they had published this in the summer they might not have endured the needless backlash of a poorly informed public who have now fuelled gradepoint inflation on an industrial scale.

I always used to take pride in the fact that the UK had been quite immune to grade point inflation. No longer.

All must have prizes.

[I can understand this approach in primary school but by secondary school, they should have pulled on their big boy pants and be ready for the fact that the world is difficult and based on assessment of merit that may not always be fair or just.

[Anonymous Coward]

Re: well done for publishing

"I work in public services and I applaud the fact they have published this."

Indeed.

"If they had published this in the summer they might not have endured the needless backlash of a poorly informed public who have now fuelled gradepoint inflation on an industrial scale."

They did ask the Royal Society of Statisticians to look at it, but on the condition of it being under NDA. The RSS smelled a trap and refused (it'd have been easy for the RSS to say "it's awful", the government ignore it, and then when there's a problem, say "but the RSS said it was fine"), but wrote the Government a 6-page letter listing all the possible issues with their approach. Guess what? The Government ignored the advice, and what the RSS said would probably happen, did.

"I always used to take pride in the fact that the UK had been quite immune to grade point inflation. No longer. All must have prizes."

Complete and utter nonsense.

[Dan 55]

Re: well done for publishing

Pulled on their big boy pants and be ready for the fact that which school they go to is the only deciding factor? There was no assessment of individual merit. Very... British.

[Naselus]

Re: well done for publishing

if (ETON =1)

{ Grade++}

else

{Grade/2}

[Howard Sway]

it would have been interesting to see what was in those "final codes"

For each examResult in examResults

{

if examResult.candidate.school == "Eton"

{

examResult.setGrade("A");

}

}

[b0llchit]

Re: it would have been interesting to see what was in those "final codes"

Piece of hidden code added by local entities:

if(QuestionsAsked()) {

        
        
database.Substitude("true", "false", NULL); // Constraint free update
        
        
} else {
        
        
database.Select("goodOnes where friends in (select boyscouts)");
        
        
}

[Anonymous Coward]

Re: it would have been interesting to see what was in those "final codes"

No bias, said Ofqual, but

if (school has large classes)

{

grade = grade predicted by teacher /fudge factor

}

else

{

grade = grade predicted by teacher

}

So small class private schools....vs large comprehensives.....

[maffski]

Re: it would have been interesting to see what was in those "final codes"

Unless, of course, they have previous data of predicted vs actual grades that show teachers of larger classes are more likely to overestimate predicted grades.

[chivo243]

Of what?

How about Ofput?

[trevorde]

Warning!

Don't run the code - it's a mutant algorithm (https://www.bbc.co.uk/news/education-53923279)

[Anonymous Coward]

Is this really the code?

Is this really the code that they were using?

I was expecting to see a lot more usage of the rand() function ...

[TDog]

Re: Is this really the code?

You can't do that - Rand is non predictable.

However

{

if (rnd && classSize<25)

{

// not biased allocating test here

If (schoolAge (//scholls over 150 yeares o9d)

>150)

{

// add reputation and previously 'one of us' factor 'cos it worked historically

score += rnd() * .5 * raw_score

// using randoms to normalise for historical accuracy

}

// use noise to show missing peaks hidden in the expected results

else

{

// iterate through noise sufficiently to get expected results

// some sort of algorithm that reduces results by 0.2%

// until such time that the small number of peaks show through

// justifying while there are as always small numbers of high achievers (HA)

// And Brian if you can tie the HA' via your NLP analysis to ensure we get at least

// some of the BAME's that would be brilliant

// but don't worry too much as we can always fudge this in later

// You are shure that the pre-processeror (sorry about that - not quite sure how that works)

// Strips the comments aren't you. PS what is the symbol for comment - ? (?)

// just put some BS xode in here and we'll worry about oh what the xwdaadsfadfdf!

}

//hoe do hids ntrackets mork asnd does it muttr?

[TRT]

Re: Is this really the code?

What programming language do pirates use?

Arrrrr!

[Anonymous Coward]

How long before

the code or portions of it end on http://thedailywtf.com/ ?

[Long John Silver]

What else should anyone have expected?

We have a government which by inept handling of Covid-19 has run our economy into the ground, accrued massive debt for useless technological 'solutions', and soon shall preside over a deluge of unintended consequences. Incidentally, I don't suggest that any other grouping drawn from the present day dismal Commons would have done better.

Lives of school pupils and university students have unnecessarily been disrupted at critical stages of their development. They cannot socialise with their peers and develop related skills for adult life. Education in schools now takes place in an extraordinarily regulated environment. Fearful teachers, often too ignorant to question the basis of diktat from government and downstream petty officials, preside over classes of physically separated mask wearing pupils. They all have been inducted into what doom-sayers declare the 'new normal'.

University students are imprisoned in their residences, punished when they gather to do things natural for their intellectual and emotional development, these encouraged by hormones, and subject to considerable restraint upon learning imposed by inherent inadequacies of solely electronic communication.

Young adult students are at minute risk of harm from Covid-19. Those and older students believed vulnerable should have been isolated in planned comfortable circumstances along with, but separate from, grannies. Even so, despite edict from the shambles of government and its ill-chosen advisors, risk free existence can never be guaranteed. Students, just like the bulk of the ovine adult population, have been introduced to a world of anxiety centred upon useless viral testing and ridiculous isolation of asymptomatic people in a manner better suited to smallpox.

Controlling this at policy level is a person who even within low standards expected of present day parliamentarians hits rock bottom: ex-fireplace salesman Gavin Williamson. When placed in charge of defence by Mrs May he made a pigs ear of the job and became laughing stock among international opposite numbers and diplomats. His utterances then and further ones concerning the epidemic (e.g. lauding the UK's 'success' in being first in the world to fast-track a vaccine) are risible.

One must conclude that Johnson put Williamson in charge of education because intellects even lesser than Johnson's own pose no threat. Williamson barely has any education of his own unless one believes a degree in sociology passes muster (it's obviously superior to one in, say, 'Tourism' or 'Gender Studies'). Even should Williamson's accomplishment be considered adequate in a general sense that still leaves him with a mind, much like his master's, unlikely to have been honed into rigorous thought or capable of basic quantitative thinking concerning risk and its balance against deleterious consequences from attempting to negate it.

Even should conventional examinations have been impossible (they were not) Williamson managed to mess up and cause anxiety (and receive ridicule) for grossly mishandling the alternative.

Nobody expects Williamson to read computer code or have deep knowledge of the basis for assumptions underlying the code. Yet as a government minister he should have had sufficient general nous and reasoning ability to pose well-directed questions to his advisors. He failed dismally as did his master when the latter fixated upon snake-oil salesmen from Imperial College and put naive faith in 'science' to quickly pull rabbits out of a hat like at the ending of a Hollywood movie about an epidemic and zombie plague whilst ignoring the extant huge body of knowledge based upon consolidated science and long experience among medical experts in epidemiology and communicable disease control.

I suppose it is onwards and upwards to the 'Lords' for Williamson, thus enabling him to share insights with other great minds such as that of Kinnoch.

[Anonymous Coward]

Re: What else should anyone have expected?

Bravo.

[TDog]

Re: What else should anyone have expected? Gender studies

To be fair - when I was one or two, playing with my sister some 62 years ago I didn't even realise that gender studies were dual option. I understand that it is a bit more difficult now so the (can't say lad here) thing might have problems with even binary choices, let alone the harsh reality of others votes and political opinions.

Go for it son, (to the wee small laddie), all you have to lose is your image of your dick. Or whatever you currently see yourself as.

[Anonymous Coward]

Re: What else should anyone have expected?

"Young adult students are at minute risk of harm from Covid-19. Those and older students believed vulnerable should have been isolated in planned comfortable circumstances along with, but separate from, grannies"

Someone doesn't understand the subject matter.

[Muscleguy]

No UK

Ofqual is just for England and Wales, we have our own regulator here in Scotland, where we don’t have exam boards queering the pitch either.

The problem with using historical data to normalise is that it doesn’t allow formerly poorly performing schools to up their game. The algorithm will knock their successes back. Meanwhile the top of the tables will get upgrades if they slip a bit or get a bigger proportion of dullards. Which is why buying into the right school zone works more often than not.

[Martin an gof]

Re: No UK

We have our own in Wales too - Estyn.

Not certain they've done a fantasic job themselves, but at least they don't seem to have been quite as disorganised as Ofqual.

M.

[Anonymous Coward]

Re: No UK

It also doesn't allow for the fact that a school that historically gets low results will occassionally have a gifted pupil. When you have data that says a school gets an 'A' grade in a particular subject once ever four years on average, how is an algorithm supposed to know whether this year is "that" year when faced with data from the school that predicts an 'A' for someone.

I suspect - not having reviewed the code myself - that the algorithm produces "statisically correct" results. But in reality, any one year is unlikely to conform to the norm - just as if I roll a die five times and get the numbers 1-5, I am not more likely to get a 6 on my next roll.