Every British Website Is Going into the British Library: A Q&A on How & Why

Photo via Flickr: Steve Cadman

Even though the digital world is functionally permanent, it somehow still feels fleeting—content is continually replaced or it disappears behind paywalls, websites are redesigned, the original Hampster Dance was replaced with one that’s not nearly as good.

Physical media definitely has its own set of limitations—thieves, moths, bookburners—but there are institutions, like libraries, that have a long history devoted to their preservation. As our media changes, libraries are changing too, even the very old ones.

In the United Kingdom, a law from 1662 has been applied to “new media,” with the goal of collecting every website with a British domain, archiving “digital ephemera” like Tweets and blurry pictures of Kate Middleton changing out of her swimsuit in perpetuity, giving future generations the opportunity to see both how we tweeted and were titillated.

America’s Library of Congress is recording Tweets too, but at 213 years old, it’s young enough that we expect this kind of dynamism. The British libraries are not only twice as old, they’re using laws that originated under James I.

In 1610, the Stationer’s Guild made an agreement with a wealthy diplomat named Thomas Bodley, who was looking to expand Oxford’s library: Give Oxford a copy of everything published, and it will be preserved for the public. This agreement evolved into the law called legal deposit, which it states that a copy of every book, magazine and newspaper printed in the UK must be given to the British Library, and five other major libraries as requested.

In April 2013, legal deposit was expanded to cover material published digitally and online, “so that the Legal Deposit Libraries can provide a national archive of the UK’s non-print published material, such as websites, blogs, e-journals and CD-ROMs.”

That’s an estimated 4.8 million websites with over a billion pages, to say nothing of the CD-ROMs.

To get the particulars on what sounds like a positively Sisyphean project, I corresponded with Richard Gibby, the legal deposit project officer at the British Library, to find out what’s being collected, how and where patrons can get to it.  

Motherboard: First off, what do you think the criteria will be for what gets recorded?

Gibby: Our goal is a national collection of the whole UK domain and we are collecting as comprehensively as possible, not selectively. The main constraints on our collecting are legal, operational and financial in nature rather than qualitative criteria. The legislation sets parameters that govern what we can collect; technical and operational issues may, in some cases, affect how a website is archived; and, although we have made a substantial investment and are budgeting for future digital storage requirements, we also have to budget for maintaining, preserving and providing access to our print and other collections.

Will someone from the library be seeking out worthy additions? Will people submit works? Or will it be automatic program collecting things?

We will continue to work with “traditional” publishers individually, many of whom will deposit their e-books, e-journals and other such publications separately. Collecting websites is largely an automated process; however curatorial expertise is required to determine the most appropriate intervals between capturing a website and going back to capture changes and new material and for developing “special collections” in response to major events or themes. Here’s an overview.

Will physical copies be made and stored, or are they digitally made, therefore just digitally stored?

They are digital only. We have invested in a secure, resilient and self-checking preservation system that stores copies of the content across four locations – in London, Yorkshire, Aberystwyth and Edinburgh. We are confident that the collection will be preserved forever.

Will the digital copies be updated?

We will continue taking additional copies – every UK website will be captured at least once a year – but our aim is to preserve the entire history so that a future researcher can see what the UK domain looked like at any point in time.

Is there a physical space being set aside? How big is it?

This collection is digital rather than physical. It will be accessible to library users in our reading rooms, alongside our other collections both physical and digital.

Has the British Library needed to get more server space in preparation?

Yes, we and our partner libraries have invested several million pounds over the last few years, in technical systems, server equipment, data storage disks, operational processes and resources, in order to prepare for this.

Said server space, via the British Library

Is there anything in the library’s collection already that’s equivalent to a ‘Tweet’?

Today’s researchers use newspapers (including personal ads), the personal archives of notable individuals and other material that is of great interest to social historians. But the web and tools such as Blogger, Facebook or Twitter are perhaps unique in that they have “democratised” and made it possible for almost anyone to give public expression to their views and thoughts. We believe that social historians of the future will find this an incredibly rich and fascinating resource.

Elsewhere you’ve said this program will provide “evidence of the way life was like in 2013 – what people cared about, what made us laugh,” but you’re still bound to digital material made in the UK. Wouldn’t a portrayal of how we live today have to be fairly international to be accurate?

The new law allowed us to only archive the UK domain and so we’re unable to archive anything from other countries, although there are similar initiatives going on in the US, France and other countries. For this reason what we’ll be able to provide will never be comprehensive, but it will be a compelling snapshot of what life was like for future generations.