Providing observability and transparency in a distributed digital preservation network

Abstract

To effectively manage their distributed digital preservation network, digital archivists should be able to answer the following questions at any time: - Are my collections intact and replicated at the intended sites? - Is the preservation network working properly? (Are all nodes operational and communicating correctly with each other?) - Have any copies ever needed repair ? Moreover, archivists also need to keep a complete history of their preservation network, ensuring that every action can be traced and used for auditing purposes. In this lightning talk, I will explain how we implemented our dashboard solution to ensure enduring observability in a distributed preservation network, highlighting its key features. While the solution is applicable to any digital preservation network, I will illustrate it on the specific use case of SAFE PLN, a LOCKSS-based preservation network operated by seven university libraries, which celebrates its 10th anniversary this year. In SAFE PLN, content is automatically collected from the institutional repositories of the 7 participating universities and distributed to preservation nodes across Europe and Canada. Copies are then regularly verified and automatically repaired using the LOCKSS technology developed at Stanford University. This open-source software was specifically designed to secure many distributed copies of digital archives and guarantee their integrity over the very long term. By design, the peer-to-peer architecture of a LOCKSS network makes it truly decentralized and highly resistant to internal and external attacks. But for the same reason, it's also difficult to monitor the status of the preservation network. Our open source dashboard solution based on Grafana provides each network member with global and detailed views of the network status, enabling each participant to track the status of their archived copies in real time. Problems in the network are also detected and reported instantly, enabling us to react quickly. By ensuring observability and transparency, our dashboard contributes to making preservation networks more efficient, and brings greater peace of mind to all those concerned with protecting digital assets. Our solution has already been adopted by the Alabama Digital Preservation Network and is currently being deployed at the Michigan Digital Preservation Network.

Details

Creators

Anthony Leroy

Institutions

Date

2024-09-17 11:05:00 +0100

Keywords

approaches to preservation; start 2 preserve

Publication Type

lightning talk

License

Creative Commons Attribution 4.0 (CC-BY-4.0)

Download

(unknown) bytes

Slides

here

Video Stream

here

Collaborative Notes

here

View This Publication