Digital director John Sheridan tells PublicTechnology how new technology is changing not only how the UK’s official archive operates, but is altering the very nature of archiving

The National Archives has about 600 staff – one in six of whom are focused on digital  Credit: PA

There can be few, if any, public bodies for whom the concept of ‘digital transformation’ will be quite so transformative as The National Archives.

The organisation, a non-ministerial government department sponsored by the Department for Digital, Culture, Media and Sport, manages hundreds of millions of documents spanning 1,000 years of British history. Its headquarters in Kew, in west London, requires well over 100 miles of shelving to store all these records – with another mile needed each year.

All of which adds up to a pretty hefty digitisation project.

Of course, not all of those miles and miles of historic documents can be digitised, and the organisation is not attempting to put a millennium of data online in one fell swoop.

But, in the last few years, the organisation has redoubled its focus on digital platforms and services, beginning in 2015, when it laid out a four-year plan to “become a digital archive by design”. The same year, head of legislation services John Sheridan was promoted to become The National Archives’ first digital director, taking his place in a six-strong senior-management team.

This year brought the publication of an updated digital strategy for the period up until 2019. The plan encompasses three phases on the way to achieving its digital ambitions: Reshape; Grow; and Accelerate. The Reshape stage has been successfully completed, and the Grow programme will run until late next year, before 2019 brings the Accelerate agenda.

The organisation’s main digital service for the public is Discovery, which allows users to search descriptions of 32 million records held not only by The National Archives, but by a further 2,500 archives across the UK. Nine million of these records are already available for online download.

Related content

• The National Archives sets out two-year plan to ensure digital future
• The National Archives commits to apprenticeships as part of plan to ramp up digital headcount
• National Archives on the hunt for £65k head of digital services

In the 2016/17 year – across Discovery and the archives’ other services – almost 230 million individual documents were delivered to the public online, compared with a little under 600,000 that were supplied in physical form.

The work to digitise records has been taking place for about 15 years. And, although hundreds of millions of documents are now available digitally, only “between 5% and 6%” of The National Archives’ records have already been digitised, Sheridan tells PublicTechnology.

The process has been partly funded by grants, but has primarily come as a result of members of the public – or, more commonly, the genealogy websites that have risen to popularity in recent years – paying the archives to create digital copies of records.

“We have a really successful model for digitisation, which is commercially led – that may well be by a genealogy company, or a company that is developing a product,” Sheridan says. “We will grant you the right to digitise the collection, we will facilitate that, and you will give us the [copy].” 

He adds: “We are then free to do what we like in terms of [usage], and we get royalty payments as well. It creates a really sustainable model of digitising the records –  and we are not spending any of our money on digitising the collection. We have a model that works particularly well for large-scale digitisation – we are doing that through our partners.”

Being digital by design means rethinking some big ideas that have been part of the archival process for a century or more

While digitisation brings opportunities to open up and disseminate archival information in a much broader way, it also presents challenges in how collections are maintained and presented, Sheridan says. The way in which paper archives are kept has, in essence, changed little for a couple of centuries. But the means of storing and protecting digital items changes every few years. 

“Being a digital archive, as opposed to being a physical archive, is a big deal, and a different deal,” he says. “In a physical archive, records are provided to us in a form that we can preserve and present. With digital, we cannot easily or readily present them. And we have to keep doing that preservation work with each generation of technological change. We have to make decisions to maximise the benefits and the capability [for years to come].”

Sheridan adds: “The big thing to understand with digital records is that they are going to outlive the digital-preservation system. [We need to consider] how do I get the records out this thing, and onto the next thing? We are very clear: the record is the thing that will outlive the system.”

Context and caution
At the moment, The National Archives is a “first-generation digital archive”, Sheridan says, meaning that so far it has solely been digitising existing physical records. The next stage of the organisation’s evolution will bring much more elemental change.

“We are a first-generation digital archive; we are implementing, in a digital way, the archival practices that were designed for managing physical records,” he says. “But a big part of our strategy is not just being a first-generation digital archive, but being a second-generation digital archive that is designed as digital.”

Sheridan adds: “It means rethinking some big ideas that have been part of the archival process for a century or more, and how we understand the evidence of the record, and how we evidence the life of the record.”

5-6%
Percentage of The National Archives’ records that have been digitised to date

229.8 million
Number of documents supplied to the UK public online last year – compared with fewer than 600,000 supplied in physical form

1786
Year to which The National Archives can trace its history, when the Her Majesty’s Stationery Office was founded

Kew
West London location of The National Archives headquarters, and its physical archive

Four years
Duration of its plan to become ‘digital by design’, spanning the period from 2015 to 2019

The key to understanding a record’s worth and meaning has long been the context in which it is archived and presented, according to Sheridan.

“Context has always been the archivist’s special sauce – how do we display a record so that its evidential value can be understood? That is something that the [commercial] marketplace is not addressing,” he says.

And, so, The National Archives – alongside the University of Surrey, the Open Data Institute, and tech companies Methods, Guardtime, and Voetek – is currently working to try and develop distributed-ledger technology that ensures the historic sustainability of digital archives. The 18-month project, which is dubbed ARCHANGEL, has been funded by the Engineering and Physical Sciences Research Council.

Alongside its investments in technology, the organisation is also significantly boosting its digital headcount. Currently about 100 staff – one in six of its total – are focused on digital. Over the next two years this figure is expected to grow to 150, Sheridan says. 

A key part of this growth will be developing its own talent via an apprenticeship scheme. The National Archives has already taken on five apprentices, and is the only public-sector body currently taking part in the programme run by the Ada National College for Digital Skills, which was founded last year.

This influx of young minds and new ideas will, perhaps, be particularly beneficial in a profession that Sheridan says can tend towards staidness.

“Archivists are steady Eddies,” he adds. “But, with the challenge of digital archives, they have no choice but to evolve their practice, and to move from relatively slow evolution, to much more rapid development.”