Exam body’s chair describes non-disclosure clause as ‘normal and ethical’ while RSS chief accuses regulator of ‘desperate’ behaviour

Credit: Pxhere

The chief executive of the Royal Statistical Society has accused Ofqual of a “desperate” attempt to downplay concerns about a non-disclosure agreement experts were asked to sign if they were to advise the exam regulator on the controversial algorithm it used to award A-Level grades.

Ofqual chair Roger Taylor has said the royal society caused “widespread misunderstanding and suspicion of Ofqual’s process” when it revealed that the exam regulator had turned down an offer to provide independent expertise on the statistical modelling several weeks before.

The RSS wrote to Ofqual in April setting out some of the risks involved in estimating student grades – which were then borne out as students received lower than predicted results and lost out on university and job offers – and putting forward fellows who they said could provide independent expertise on the algorithm.

But the RSS was told statisticians would have to sign an NDA to be considered. The agreement would have prevented the fellows from “commenting in any way on the final choice of the model for some years after this year’s results were released”, Sharon Witherspoon, RSS vice president, education and statistical literacy said in a second letter to the regulator earlier this month.

Related content

• Government scraps all algorithmic grades for A-levels and GCSEs
• Sturgeon apologises after outcry over exam results downgraded by ‘deprivation algorithm’
• No plans for ‘right to challenge’ laws for algorithm decisions

Responding to Witherspoon‘s letter, Taylor said she had been “incorrect both in terms of your reading of the confidentiality agreement and in relation to the correspondence with Ofqual”.

Taylor, who is also chair of the Centre for Data Ethics and Innovation, which advises government on the benefits and risks of artificial intelligence, said the NDA was a “normal and entirely ethical mechanism” to protect confidential information.

He said the agreement “does not preclude anyone from commenting on the model” but “only precludes the disclosure of confidential information shared within the group”.

“Those in the group are not precluded from making general comments about the process publicly so long as they do not disclose confidential information about those discussions. The moment information is published – and we published details of the model on the day results were released – there is no restriction on any member of the group commenting on it,” he added.

But RSS chief executive Stian Westlake has said Ofqual’s criticism of Witherspoon’s letter is “baseless”.

“Frankly, this is pretty desperate stuff from Ofqual at this stage,” he told BBC Radio 4’s Today programme. “The confidentiality agreement that they asked our expert statisticians to sign before they would consider their help was what one lawyer I showed it to called ‘very widely drafted’. It would have prevented us, in our view, from commenting on any information that was discussed in private with Ofqual that Ofqual didn’t choose to publish.”

Westlake said when they were asked to sign the clause, RSS fellows did not know what information Ofqual would have eventually chosen to make public. 

“In the end, [Ofqual] published a methodology document but the decisions that went into it, the decisions about the trade-offs, key issues like grade inflation versus individual accounts, that would have been up to Ofqual whether we could have discussed that,” he said.

“To really understand whether the right model was chosen and to have an open discussion about that, all of that stuff we would like people to be transparent about,” he said – adding that the watchdog had not published any minutes from the meetings.

The NDA, published alongside Taylor’s letter, states that confidential information includes “all business, strategic, financial, technical, editorial, marketing or operational information of whatever kind (whether oral, written, or in any other form)” supplied by or on behalf of Ofqual about the regulator.

Westlake said the RSS had told Ofqual it was “perfectly happy” to agree participants would not leak discussions from the meetings while they were ongoing. But he said the five-year ban on disclosing information was a “real problem”.