TDR: the hard drive forward

Abstract

The National Archives (UK) has archived born-digital records for over 20 years. However, our records transfer process (using hard drives) has historically been slow and error-prone. Depositors struggle with technically demanding requirements for preparation and characterisation of digital records. We offer guidance and training but we need to do more. In 2020 we embarked on the development of a cloud-based service for digital transfer. The new service, ‘Transfer Digital Records’ or TDR, helps UK Government departments upload, prepare and transfer custody of digital public records to TNA for permanent preservation and access. TDR addresses pain points for depositors, removing the need to install digital preservation tools client-side, eliminating metadata spreadsheets, and reducing back-and-forth of records. TDR delivers a more complete, better-structured and more timely record to the archive. TDR allows transferring departments to upload records via a browser, it performs automated checks including checksums (compared before and after upload), metadata extraction, file format identification and virus scanning. The interface lets users add custom metadata and describe sensitivities and access conditions. TDR is being built following Agile principles. Our Users are central to the design of the service. We conducted a thorough Discovery (proof-of-concept) followed by a fast Alpha (functional prototype), testing with users all the way. TDR is being built in the open - our code is available for re-use. We’re happy to discuss how others might integrate elements of TDR into their workflows. The initial service was released in public Beta in April 2023 and has already made it easier and faster to transfer records. We are seeing daily small transfers of digital records to the archive, allowing us to move away from large batches towards a ‘little and often’ principle which brings records into the safety of the archive much earlier. We are working to widen our scope to accept records directly from cloud ecosystems such as SharePoint or Google. We also wish to increase automated processing to help our users, for example, by detecting (near-)duplicates of records already in the archive or flagging potentially sensitive content. This poster showcases the TDR journey and explores future challenges.

Details

Creators
Kirsten Arnold
Institutions
Date
2024-09-17 11:00:00 +0100
Keywords
information technology for dp; start 2 preserve
Publication Type
poster
License
UK Open Government Licence v3
Download
(unknown) bytes

View This Publication