The Global List of Digitally Endangered Species (2019)

The Global List of Digitally Endangered Species (2019)

7 November 2019

Non-standard public records, open source Intelligence, unpublished research data, and web interfaces have joined the list of critically endangered digital materials in 2019.

The Global List of Digitally Endangered Species – The BitList – has been released today in a new edition as part of the Digital Preservation Coalition’s international campaign to raise awareness of the need to preserve digital materials.

Co-ordinated and published by the Digital Preservation Coalition (DPC), the Bit List identifies a range of digital materials, software and storage types which, if appropriate action is not taken, could become ‘practically extinct.’

“We first published the Bit List in 2017,” explains Executive Director of the DPC, Dr. William Kilbride. “Although there are some important achievements to celebrate, the digital preservation community has to work very hard to stay still, such is the pace of change and growth in digital content. So in spite of efforts, and in spite of considerable public attention to the uses and misuses of data, it’s hard to see any significant trend towards improvement.”

The Bit List 2019 provide a much deeper dive into challenges of long-term access to the digital estate than in previous reports. The result is a much longer list offering with 74 entries which provides a much more specific comments on the risks associated with digital content. It also outline the good practice which the global digital preservation community believes will be necessary to address these challenges.

Dr. Kilbride adds, “the BitList 2019 shows that digital objects are ‘born vulnerable’ and that risks expand over time if not addressed.  We depend on digital tools and data and services in every aspect of our lives. So in the digital age, all our interactions become vulnerable by default”

“We know that professionals in agencies around the world have reported their own use of The BitList to support advocacy and target resources to greater effect,” says Jane Winters, Professor of Digital Humanities at the University of London School of Advanced Study and Chair of the international panel of judges who evaluated the Bit List before its publication.

“Technological obsolescence is still a challenge, but we know how to address that now, what is more of a problem is human behaviour. If the Bit List can help persuade us all of our need to take responsibility for preservation, that is an excellent outcome.”

The two yearly cycle echoes the annual headlines about new words that have made it into the dictionary, reflecting something of the concerns that have arisen since the last edition. There are 18 completely new submissions and it is not hard to trace the social, economic and political trends which underlie some of them.

For example, ‘Open Source Intelligence’ sources (or OSInt) have been added as a new entry to the list. These sources have been used very effectively by investigative journalists, as well as law enforcement agencies, around the world to test and disprove propaganda.  Investigations at the time of the Arab Spring, as well as the wars in Syria and Yemen have all used OSInt to test claims made, while focussed investigations into individual outrages like the downing of the Malaysian Airline MH17 and the poisoning of the Skripals have depended on access to large quantities of data published on the open web.

Such investigations often depend on claims made by terrorists via social media channels. Perpetrators are quick to delete or alter evidence that defeats their malign purposes, while technology companies have responded quickly and algorithmically to remove propaganda and fake news. But that creates a preservation challenge, especially for older investigations where the content was not secured for investigative purposes. Digital preservation has become a necessary tool for investigation and prosecution with a chain of evidence which would otherwise easily be destroyed.

The BitList also points to what might be achieved. For example, although web archiving represents a significant and dynamic challenge for digital preservation, the BitList suggests that it is regulation rather than technology which is the main enabler or hindrance to success. The single most explicit reference to web archiving is in relation to domains where legal deposit laws are weak or unenforceable; and by implication where regulation is modernized, significant risks to important content can be eliminated. The digital preservation community is ready. Small but obvious changes in law could make a profound difference.

Juan Bicarregui, Chair of the DPC Board and Head of Data Division for UKRI-STFC observes that “by compiling, monitoring and maintaining the Bit List over the coming years, the DPC can begin to track the trend for loss, and for conversely for preservation. Our hope is that we will be able to celebrate great digital preservation endeavours as entries become less of a ‘concern,’ whilst still highlighting the need for efforts to safeguard those still considered ‘critically endangered.’”

In response to the Bit List, the DPC wants action to be taken, and in some cases urgently. They observe that in all cases assessment is needed quickly as the scale of the challenge gets bigger and bigger, as the importance, scale and complexity of data grows.

In some cases, they are calling for industry regulators to become involved to impose more onerous stipulations for the preservation of digital material. Large corporations and providers on web-based interfaces and services will certainly be asked to take responsibility for ensuring that simple preservation functions can be built into infrastructure, so that objects and code are robust at the point of creation rather than having to be reconstructed afterwards.

Regulatory reform is also required. While there is a very active and very capable global community of digital preservation expertise, their efforts to preserve digital materials are often thwarted by copyright laws. There are some exceptions to these laws to enable copies to be made for preservation purposes, but these have not always kept pace with technological advancements or apply universally to all preservation activities.

The Bit List is published as part of World Digital Preservation Day which aims to raise awareness of the strategic, cultural and technological issues which make up the digital preservation challenge.