Using AI to process digital records

All UK government departments must keep the files they produce, and then release them to a public archive, usually after about 15 years. Before it’s made public, each file must be checked to make sure it does not contain information that is still sensitive.

We review all Foreign, Commonwealth and Development Office (FCDO) files before they’re moved to The National Archives. Normally, one of our Sensitivity Reviewers will look at and clear every document for publication.

However, the huge growth in the use of digital technologies has massively increased the number of files that we need to review.

We therefore faced a problem: how could we continue to make FCDO documents public at our normal rate without also massively increasing our number of Sensitivity Reviewers?

Designing the solution

In 2018 our team of experts began creating a digital system to review and transfer digital files. From the start, we knew artificial intelligence (AI) was the best system to use.

Our goal was to design an AI system that would help our Sensitivity Reviewers work more quickly. This meant designing new hardware as well as software.

The system needed to:

• be robust
• be future-proof
• have an effective interface for the Sensitivity Reviewers to operate
• meet a rigorous set of government security protocols

The AI system we designed met all of these criteria and the first version was launched in April 2019.

Improving efficiency

To meet our goal of helping Sensitivity Reviewers to work more quickly and efficiently, we trained our AI system to find relevant information.

The system used our reviewers’ experience and decisions they’ve made in the past. Applying this learning to current records helps answer questions and speed up the review process.

Fast-moving technology

We’ve continued to improve the AI system in the year since its launch. Our team has introduced applications that:

• simplify complex tasks
• standardise common processes, to reduce human error
• provide relevant insights, so that Sensitivity Reviewers can spend less time analysing material

We constantly review the way our system works, and work with a range of experts and interested parties to find ways to improve it. We are continuously developing new tools.

To take advantage of any new technology, our AI system is built on a modular basis. This allows us to add new elements as needed.

Use for other government departments

We’re the first government organisation to develop this kind of AI system. One of our aims is to iterate the system so that other government departments can use it to make their reviewing process more efficient. In the last year, we’ve started working with HM Treasury to see whether the AI system could work for their records.