There is a good story in today’s Herald Tribune on how costly digital loss can be:

“JUNEAU, Alaska: Perhaps you know that sinking feeling when a single keystroke accidentally destroys hours of work. Now imagine wiping out a disk drive containing information for an account worth $38 billion (€29 billion).
That is what happened to a computer technician reformatting a disk drive at the Alaska Department of Revenue. While doing routine maintenance work, the technician accidentally deleted applicant information for an oil-funded account — one of Alaska residents’ biggest perks — and mistakenly reformatted the backup drive, as well.
There was still hope, until the department discovered its third line of defense, backup tapes, were unreadable.” -AP

This whole article brings up an interesting issue however. How should we store our public data as a civilization?
Some of the details of this article made my antenna perk up… (starting with the fact that the qualified 800,000 Alaskan residents are getting $38 billion dollars). But it is quotes like this one;

“Over the next few days, as the department, the division and consultants from Microsoft Corp. and Dell Inc. labored to retrieve the data,…”

That really bring up another interesting point of where we are entrusting our public data. Data like tax records, property documents, census, birth, death and marriage announcements are being stored all over the country at city, county, state, and federal levels on proprietary systems. In other words we have public data who’s future is resting entirely on the hope that companies like Microsoft will both stay in business, but also make all their software backwards compatable — forever.

We are also seeing some governments, like Venezuela,  really understanding this issue before the rest of the world. It will be interesting when the future tries to look back on this time, our early digital history, only to find a bunch degraded mag tapes with proprietary file formats. They will likely know more about ancient Egypt than us.

Share This ▼

From The Prelinger Library:

Now comes word that the SAA Council has decided that the archives of its own listserv are no longer worth saving and will be “disposed of” at the end of this month. After an appraisal of their value, they’ve determined the cost of keeping these bits is higher than their “evidential or informational value.”

Way to archive, Archivists! We here in the Digital Dark Age Corner salute you.

We note that the meeting on the listserv archive’s fate was held via email.

One can only hope that by this precedent they will find it perfectly acceptable to expunge records of other past digital events that they deem too difficult to maintain, such as the evidence that archivists removed the archives of their archivism miscellanea.

The whole article, including where to complain, can be found here.

Share This ▼

Interesting and quite long article in the Times Business Section on Sunday, beginning page about one of the downsides of digitization of books and similar printed resources. The thesis is that as we come to expect sources to be available digitally, and thus to rely on what we can find and search that way, those sources that are NOT digitized are lost to cultural memory. And, despite what seems to be the vast quantity or material being scanned for inclusion in digital libraries, much more cannot be scanned because it is the wrong size, or because it is not deemed economically feasible to scan them….

-Paul Alan Levy via Farber List

New York Times Article (login required)

Share This ▼

As reported by Engaget, Sandisk is rumored to be developing 100 year archival memory cards… These may be just the thing for those family photos languishing on your ever detiorating CDRs.

Share This ▼

On February 24th the Rosetta spacecraft successfully completed its somewhat hairy slingshot around Mars with a copy of our Rosetta Disk on board. More here: http://www.esa.int/SPECIALS/Rosetta/

Here is agreat photo forwarded by Stewart Brand shot from Rosetta as it swung by Mars.

Share This ▼

A piece in Forbes Magazine

Publish And Perish
Elisabeth Eaves, 12.01.06, 12:00 PM ET

Nothing is safe. Not your e-mails, digital photos or Word files. Not old newspapers or books. When it comes to storing information, everything will disappear into digital obsolescence or crumble to dust.

Even White House e-mails, important blueprints and influential works of 20th-century literature–the very artifacts that you’d expect would be carefully preserved–are at risk of being lost forever.

The National Archives and Records Administration, the agency responsible for preserving the federal government’s documents, realized in the 1990s that it couldn’t cope with the digital era using its old electronic storage system of magnetic tapes. The White House under the Bush Administration alone will generate as many as 100 million e-mails. Copying them would take years. NARA has contracted Lockheed Martin (nyse: LMT - news - people ) to build a federal digital archive, but the system won’t be ready until at least September of 2007.
The Library of Congress, meanwhile, ditched most of its original newspaper collection after transferring the content to microform, which uses a machine to read film. But Nicholson Baker, in his book Double Fold: Libraries and the Assault on Paper, says the medium is at least as iffy as paper: Some early acetate films “shrink, buckle, bubble or stick together in a solid illegible lump,” he writes. In the ’80s libraries switched to polyester-based films. But some types of polyester films are prone to spots, others attract fungus and another suffered “complete image loss” when exposed to the high heat of common microform readers.

(more…)

Share This ▼

 
A good article on the digital preservation problem in Popular Mechanics:
http://www.popularmechanics.com/technology/industry/4201645.html?page=1

When the aircraft carrier USS Nimitz takes to sea, it carries more than a half-million files with diagrams of the propulsion, electrical and other systems critical to operation. Because this is the 21st century, these are not unwieldy paper scrolls of engineering drawings, but digital files on the ship’s computers. The shift to digital technology, which enables Navy engineers anywhere in the world to access the diagrams, makes maintenance and repair more efficient. In theory. Several years ago, the Navy noticed a problem when older files were opened on newer versions of computer-aided design (CAD) software.

“We would open up these drawings and be like, ‘Wow, this doesn’t look exactly like the drawing did before,’” says Brad Cumming, head of the aircraft carrier planning yard division at Norfolk Navy Shipyard.

The changes were subtle — a dotted line instead of dashes or minor dimension changes — but significant enough to worry the Navy’s engineers. Even the tiniest discrepancy might be mission critical on a ship powered by two nuclear reactors and carrying up to 85 aircraft.

The challenge of retrieving digital files isn’t an issue just for the U.S. Navy. In fact, the threat of lost or corrupted data faces anyone who relies on digital media to store documents — and these days, that’s practically everyone. Digital information is so simple to create and store, we naturally think it will be easily and accurately preserved for the future. Nothing could be further from the truth. In fact, our digital information — everything from photos of loved ones to diagrams of Navy ships — is at risk of degrading, becoming unreadable or disappearing altogether.

(more…)

Share This ▼

Storing information is easier than ever, but it’s also never been so easy to lose it — forever. We could end up with a modern history gap.

By Charles Piller, LA Times Staff Writer
September 13, 2006

Carter G. Walker remembers the day her memories vanished.

After sending an e-mail to her aunt, the Montana freelance writer stepped away from the computer to make a grilled-cheese sandwich. She returned a few minutes later to a black screen. Data recovery experts did what they could, but the hard drive was beyond saving — as were the precious moments Walker had entrusted to it.

“All my pregnancy pictures are gone. The video from my first daughter’s first couple of days is gone,” Walker said. “It was like a piece of my brain was cut out.”

Walker’s digital amnesia has become a frustratingly common part of life. Computers make storing personal letters, family pictures and home movies more convenient than ever. But those captured moments can disappear with a few errant mouse clicks — or for no apparent reason at all.

(more…)

Share This ▼

Original recordings of Apollo moon missions are missing

WASHINGTON (Reuters) — The U.S. government has misplaced the original recording of the first moon landing, including astronaut Neil Armstrong’s famous “one small step for man, one giant leap for mankind,” a NASA spokesman said on Monday.

Armstrong’s famous space walk, seen by millions of viewers on July 20, 1969, is among transmissions that NASA has failed to turn up in a year of searching, spokesman Grey Hautaloma said.

“We haven’t seen them for quite a while. We’ve been looking for over a year and they haven’t turned up,” Hautaloma said.

The tapes also contain data about the health of the astronauts and the condition of the spacecraft. In all, some 700 boxes of transmissions from the Apollo lunar missions are missing, he said.

(more…)

Share This ▼

Long Now collects stories of the “digital dark age”.  Originally these were kept in discussion boards, but we have moved this to our blog as of 02007 (Acknowledging that the present will in fact be known as the digital dark age, since all our digital data has no forward migration path.)

I thought that the first post in this category should be the story of some of our own data loss here at Long Now including the first version f this forum. We have learned a lot over the last decade, and are working now on some new architectures based on this learning.

Our problems stemmed from the fact that Long Now is a project-based institution that survives soley on on project grants. This means that we have little to no carryover infrastructure when there is a lull between projects. (we now have a full time sys-admin however)

So when we had a lull about 2 years ago we did not have any sysadmin staff for several months. This resulted in our servers getting compromised by hackers, age, and heat.

We had hackers using one of our open wikis for file trading gobbling up bandwidth, and we had a drive or two fail from excessive heat in a server (due to the fan dying).

The wiki problem was solved easily enough and no permanent damage was done. We learned our lesson about allowing public file uploads to our wikis though.

The drive problem was much more problematic. The drive was mirrored to allow for two copies of all the data should a problem occur. The problem was that the mirroring actually mirrored corrupted data leaving us with two drives with exactly the same worthless data on it. One of the things we lost was the first version of this forum.

We now do incremental backups on separate machines that allow us to revert to several different snapshots of the data moving backward in time. This way if we get corrupt data we can keep moving backward until we have a clean sample.

We also now backup much of our cultural resources (like our Rosetta linguistic database) to the Stanford servers thanks to Stanford librarian Mike Keller.

For the future we are now moving away from a server room all-together. We are moving all publically served data to a collocation facility. For the in house data we are now looking at distributing the in house server architecture under peoples desks. Each project workgroup at Long Now will get a Network Attached Server with 1Tb or more storage in it. Each workgroup will administer permissions on their NAS box. But we will only use half of the storage of each box for each workgroup. The other half will be a distributed RAID of sorts - backing up the rest of the offices data. This way no one box has all the data, and no one box can lose all the data.

We now have begun these efforts using Infrant’s ReadyNAS standalone RAID storage servers..

Share This ▼