Sensitivity reviewing for His Majesty’s (HM) Treasury

With vast amounts of annual records and numerous different file types, HM Treasury needed a bespoke solution to sensitivity review its files.

His Majesty’s (HM) Treasury, as the government’s economic and finance ministry, sits at the centre of much government activity, amassing vast amounts of data without even trying.

Sitting in the intersection of government, the Treasury are often copied into emails from departments across His Majesty’s Government (HMG) with a wide variety of file types attached.

This means there is a huge amount of information that needs reviewing for sensitivity each year to ensure that in complying with the 20 year rule, HM Treasury sends the right information to The National Archives.   

Handling different files

With far reaching connections across HMG, the HM Treasury digital legacy records collection is comprised of more than 247 different groups of file types, that all require opening and checking for sensitivities prior to transfer. This wide range of content types provided a significant challenge for the DSR team. 

The DSR team reviews documents for our main customer the Foreign, Commonwealth and Development Office (FCDO), whose file plan is very structured. A file plan is a tool for organising records that makes them easy to store, retrieve and dispose of. The process was originally set up to review just three different file types, so the team had to design a unique solution to address both the quantity of documents held by HM Treasury, and the large number of different file types identified. 

Every version of Excel, PDF and Word must be processed slightly differently, so the DSR team had to adapt. This complexity was made more difficult due to lockdown restrictions in place at the time, with our teams forced to build relationships and manage technical developments virtually. 

Developing a new approach

To address the quantity of data and different file types, the DSR team built on the investment made by the FCDO. They added extensions to the system to process the HM Treasury file types and researched Artificial Intelligence (AI) and machine learning technologies. The team created efficiencies to manage the hundreds of thousands of records to be reviewed each year, to ensure HM Treasury could achieve compliance with the Act.   

The newly developed system is designed to support the rendering of documents and speed up the removal of duplicate and non-historic data to identify the original needed for transfer. Due to the design of the DSR system, our customers at HM Treasury are reassured that the correct documents are redacted with consistency across all versions of the file. AI helps to maintain consistency and guide our reviewers, but does not make the decisions on their behalf.   

What’s next?

Developed specifically for the unique requirements of HM Treasury, the use of AI to support the work of sensitivity reviewers has been a success. The project is now moving into the service stage, and HM Treasury is optimistic about the future of the programme.   

Through the use of this system across HMG to improve the coherence of inter-departmental decision making, the risk of “mosaicking” would be greatly reduced. Mosaicking is the piecing together of information by finding apparently innocent elements that combine to reveal a sensitive whole. If sensitivity reviews are not consistent across government departments therefore, there is a risk that redactions can be unpicked. The consistency that AI provides in the sensitivity review process should help HMG develop a joined-up approach.  

The DSR team are developing a system which I am confident will have strategic benefits across government in the longer term and will reduce risks, enable coherent decision making, and enable departments to manage their digital legacy records to meet their obligations against the Public Records Act.”
Departmental Records Officer and Head of Knowledge and Information Management, HM Treasury