One of our favorite rogue digital archivists, Jason Scott, has just posted a video of his talk at DefCon 19 about The Archive Team exploits. This is perhaps the most eloquent (and freely peppered with profanity) explanations of the problems inherent with preserving our digital cultural heritage. He also describes in a fair amount of detail what he and The Archive Team have been doing to help remedy the problem.

I am going to take a moment here and say that THIS is what I was talking about a few weeks ago. Jason Scott and The Archive Team exemplify the type of community activity that we need to be happening in order to save our shi.. stuff.

Earlier this month author Charles Stross gave a lecture in San Francisco for the USENIX Security Symposium. He called his talk “Network Security in the Medium Term, 2061-2561 AD” and in it he took the concept far beyond keeping your email password private or your WiFi from being hacked.

Network security, according to Stross, will slowly work its way down to a basic need for everyone until it resembles the right to personal safety.

With increasingly pervasive networked sensors, knowledgeable genetic tests, and falling data storage costs, our online identities become more and more just our identities. Trade-offs and double-edged swords abound:

Is losing your genomic privacy an excessive price to pay for surviving cancer and evading plagues?

Is compromising your sensory privacy through lifelogging a reasonable price to pay for preventing malicious impersonation and apprehending criminals?

Is letting your insurance company know exactly how you steer and hit the gas and brake pedals, and where you drive, an acceptable price to pay for cheaper insurance?

But the value in storing and selectively sharing this data is there, as anyone who’s searched for an old email to absolve themselves of some minor (or not so minor) blame can attest. A short story, Nanolaw with Daughter, by Paul Ford hints at this same issue:

Then would come the game. Cameras in the phone of every parent. Sensors on the goals; sensors in the ref’s whistle; in the ball; in the lamps that light the field. Yellow cards, goals, offsides, all recorded from many angles and tagged with time, location, temperature, whether for the memories or to limit liability—the motion of 22 bobbing ponytails transformed into lines of light.

And so, if one is compelled to record as much of their life as possible, even just as a means of refuting those who would accuse them, network security becomes a highly personal long-term archiving project:

But some forms of personal data – medical records, for example, or land title deeds – need to remain accessible over periods of decades to centuries. Lifelogs will be similar; if you want at age ninety to recall events from age nine, then a stable platform for storing your memory is essential, and it needs to be one that isn’t trivially crackable in less than eighty-one years and counting.

Your very assertion of who you are will become dependent on the reliable and secure functioning of a vast infrastructure: “Robustness and durability are going to be at a premium in the future,” Stross emphasizes.

You can view video of the talk or read the full text.

*An old VisiCalc ad from the early 80′s.

The Digital Dark Age beacon has been flashing lately with some renewed frequency. It seems that articles on the pitfalls and challenges of preserving our digital “stuff” are starting to find their way back into the mainstream media. Most recent and notable of these is Kari Kraus’ op-ed piece in the New York Times, “When Data Disappears.” The most salient thing that Kraus points to in this piece is the formation of specialist communities and their role in the preservation of video games.

When I first met Kevin Kelly, he told me of his notion that no technology will ever become obsolete because there will always be someone or some enthusiast community that will put energy toward the preservation of even the most obscure thing. He famously told Robert Krulwich of NPR that, “there is no species of technology that has ever gone globally extinct on this planet.”

What he is saying is that there will always be some force of human compulsion or need that emerges to buoy the inventions of our race. This is important. It is this notion of emergence that will help save us from our dreaded digital dark age. What I find myself doing now is trying to envision the existence of Visicalc enthusiast clubs or a group for any of the tens of thousands of digital file formats that have surfaced over the years. I can almost see it. It doesn’t seem totally infeasible to me, but part of me worries that some of these technologies just aren’t sexy enough to be embraced in the same way that old video games are. I wonder, too, about the scalability of Kelly’s idea. As the production of new technology gets faster, will there be enough human interest to sustain the preservation of ALL of it?

Time will tell and I am certainly betting on the hope that there will.

Phillip Mendonça-Vieira captured the front page of the website of the New York Times every few hours from September 2010 to July 2011 and made a video of all those images. As far as historical documents go, it’s a hypnotic view into a particular period of time.

On what we might learn from this he says:

Having worked with and developed on a number of content management systems I can tell you that as a rule of thumb no one is storing their frontpage layout data. It’s all gone, and once newspapers shutter their physical distribution operations I get this feeling that we’re no longer going to have a comprehensive archive of how our news-sources of note looked on a daily basis. Archive.orgcomes close, but there are too many gaps to my liking.

This, in my humble opinion, is a tragedy because in many ways our frontpages are summaries of our perspectives and our preconceptions. They store what we thought was important, in a way that is easy and quick to parse and extremely valuable for any future generations wishing to study our time period.

He also did one for the BBC!

After selling only 800 models last year – down from over 10,000 as recently as 02009 – the last typewriter factory in the world (according to The Daily Mail) has closed its doors and halted production. The factory was run by Godrej and Boyce and was based in Mumbai, India. The majority of the typewriters being produced in the last few years were for writers of Arabic in countries where modern PCs have yet to fully penetrate the market.

Despite the demise of industrial production, it seems likely the typewriter will live on. Kevin Kelly once said technologies are immortal and if the collector/maintainer culture that’s already going strong around typewriters is any indication, manually thwacking ink onto paper has a good run ahead of it yet:

• For the real, restored thing there’s myTypewriter.com
• Rarotype Inc. in Sunrise, Florida still manufactures printwheels for use in manual typewriters of many different brands.
• Mr. T pities the early-adopting fool: “With a typewriter from MrTypewriter.com you can rest assured that you have invested in a quality machine that has been fully tested and restored to once again provide many years of reliable service for today’s wordsmiths. “
• You can live in a typewriter factory in Emerson, IL.
• Check out these USB Typewriters for giving your iPad that vintage feel.
• The Early Office Museum explores the history of typewriters.
• IBM has an online timeline describing their contributions to typewriter technology.

(Sent in by Katie Malone – thanks!)

I imagine a library atop a remote mountain that collects the essential information needed to re-learn practical knowledge essential to civilization. This depot, open to anyone who journeys there, is the cultural equivalent of the Svalbard seed bank, a vault on the Arctic Circle that holds frozen seeds of crop plants from around the world. The utilitarian documents in this vault would be the seeds of culture, able to sprout again if needed. It would be the Library of Utility, and it would serve as civilization’s backup.

Most great libraries of today have a broad mandate to be very inclusive. They contain “everything.” This everything is being duplicated in digital form by Google and others as the long-desired Universal Library. But the library at the top of the mountain would be different. It would be a very selective library. It would not contain the world’s great literature, or varied accounts of history, or deep knowledge of ethnic wonders, or speculations about the future. It has no records of past news, no children’s books, no tomes on philosophy. It contains only seeds. Seeds of utilitarian know-how. How to recreate the infrastructure and technology of civilization so far. The library would gather the knowledge needed to recreate itself — all the mechanical structures of brick, mortar, glass — the library itself. One could think of it as a manual for making a physical library with books and paper. Or a manual for reconstruction the infrastructure of civilization. A civilization reboot manual, which has also been discussed at the Long Now Foundation and in various science fiction stories. From the seeds of know-how archived here you could regrow the arts of printing, or metalworking, or plastics, or plywood, or laser discs.

This information is not usually found in libraries, or in books, or even on the web in text. These days much instructional and utilitarian information is conveyed in YouTube clips. Partly because video is a good way to show how something is done, but also because it is much easier to record a video that put things into words and diagrams. But often that ease lowers the quality of instruction. If you had to rely on a university library to find instructions on how to make sheet metal from ore, or even to find and extract the ore, or to make plastic from oil, or to grow silicon to make make a chip, it would be very difficult. Usually such utilitarian knowledge is missing from books, but even when it is present in the library, it is dilute and spread throughout many books or journals. A lot of this utilitarian knowledge is implicit knowledge and passed along outside of written documentation. And when written down, these documents are often not the type to find their way into libraries.

It need not be a giant library. It may be possible to fit all the essential information needed to bootstrap the infrastructure of civilization into 10,000 books or so. And unlike the Universal Library of Google, it would be on paper. In a century or so, paper-based books will be rare. But paper books will outlast any digital platform and paper requires the least amount of technology to access. Paper will be universally readable at any period. You can’t say that about floppy disks, CD-Roms, and PDFs.

But rather than containing merely shelves of books, this Library of Utility would contain sequences of books. Depending on where you wanted to start, you would visit different documents. If you already knew how to make glue, you could immediately start the instructions on making plywood. But if you did not know how to make water-proof glue, you would begin at a different point. Or if you knew glue and wood spinning, but did not know about hydraulic presses, you’d get a different set of instructions. That multi-forking seems pretty hypertext; would not digital be better for this? Yes, it would be better, but would be done in paper as a back up.

Perhaps the Library of Utility is usually sealed airtight, say through the winter, and it is opened a few times, or a few months, a year for adding books and research. This is a 10,000-year Library, encased in an impermeable shell that could last for hundreds of years without human attention if it came to that. So the Library of Utility would be built to house the most essential 10,000 books for 10,000 years, a library of practical knowledge that could be bootstrapped to restart civilization at any point it might be needed.

There is no need to wait for the Library to be built at the top of the mountain. It could be started now, in any garage. What books would you bring to it if you could?

(The image on top is of small monastery in the Himalayas, near Paro, Bhutan. There were only a few books in it. The second image is of the Svalbard seed bank. No books, only seeds.)

This article was cross posted from The Technium.

The second season of FUTURESTATES has been released, a film series featuring visions and stories of the “not-too-distant future.” Participants imagined narratives based on scenarios such as extreme climate change with environmental refugees, gated communities that regulate the genetic makeup of their offspring, and the proliferation of software that charts our likes and dislikes, “creeping into the human heart and soul.” J.P. Chan’s “Digital Antiquities” tells the tale of a man with a cryptic old device (a CD) that his mother left him and the woman who helps him retrieve its data. The story takes place in a time when all information is constantly uploaded to ‘the cloud,’ rendering nearly all of our present media obsolete. Interestingly, this time is fast approaching: the year is 2036. Chan writes:

My own experience with data loss made me think about how easy it is to lose digital memories and what it might mean for our culture — and ourselves — when that loss happens billions of times over. What memories will be preserved of our era, when the media itself is so fragile? Stone tablets survive millenia to tell us stories of civilizations that left few other traces. If the far-more-frail hard drive is the stone tablet of our times, we’re in big trouble.

In the future, virtually all of our lives will be recorded and presumably stored safely online somewhere. Recovering data from personal media like floppy disks, hard drives, optical discs, and memory chips will be an extinct business. But right now, we’re creating lots of digital memories on these media but only haphazardly preserving them. How will we feel about this in a few decades when much of it is gone?

You can watch “Digital Antiquities” here, and also check out FUTURESTATES’ Predict-o-Meter where you can weigh in on the future and see other users’ predictions.

Two days ago, we learned that a Rosetta Disk made its way into the Special Collections of the University of Colorado Boulder library, and was on public display there. One of our members, Zane Selvans paid a visit, and had an extraordinary experience. He took fantastic pictures and wrote it up on his blog Amateur Earthling – we repost it here with his permission. It is a great illustration of the challenge in keeping information alive over time, place, and people.

Human Language in the Palm of my Hand

by Zane Selvans

One of the Rosetta discs was recently bequeathed to the University of Colorado libraries, and the Long Now put out a request for pictures of it in its new home.  I eagerly responded by heading to the special collections in Norlin yesterday.  It didn’t seem to be on display anywhere, so when the librarian made eye contact, I said I was here to see the Rosetta disc, and she sent someone off to get it.  And they took it out of its Pelican case, and set it on the table in front of me (after I’d filled out a reader card and agreed only to take notes in pencil… or by digital means — no pens are allowed near the old books)  At first I was hesitant to touch it, and asked if it was okay, and she said “Oh it doesn’t look like the kind of thing that requires any special handling.”  So I picked it up.

I was amazed at the weight of the thing.  The tungsten hemisphere (I think it’s tungsten anyway… but maybe I’m thinking of one of the clock parts) [ed. note - it is actually stainless steel]  is much denser than most everyday objects.  That, plus the iridescent sheet of the etched words and the distortion of the lens makes it into a strange kind of artifact.  It’s obviously a weird thing.  I couldn’t help but think of Michael Madsen’s Into Eternity, and the difficulty of attempting to ensure that we communicate anything tens of thousands of years into the future.  His one way conversation with those who inherit our histories.  These spheres are beautiful art and elegant thought experiments today, but holding one made me envision the world in which they were actually needed, where they’ve been used for their intended purpose.  Far seeing, informational time machines.  Linguistic Palantír.  It’s both horrifying and hopeful to think about what could come to pass in our deep futures.

If this thing has been used, then darkness fell one day.

If this thing has been used, then someone made it through, and they want to know again.

I couldn’t help myself.  I had to open it up.  Gingerly.  It’s a hard thing to handle, so smooth and round and heavy enough that it’s challenging to control it with one hand.  The lockring tinkled down and the librarian looked over a little surprised.  “Oh, I didn’t know you could open it.”

Have you looked at it.  Do you know what it is?  Something to do with languages.  Mmm.  Yes.

With only a single change of custody, all information about the thing had already apparently been lost.  They said that when it was checked in to the collections, it hadn’t come with any accompanying documentation.  Just a strange heavy sphere in a padded box.  The box was labeled, saying who it had come from, and naming it a Rosetta disc, but that was about it.  It’s supposed to be usable even without any documentation — that’s kind of the point — but it certainly does highlight the fragility of information.  I tweeted to the Long Now afterward, and they’ve sent “Care and Feeding” documentation to the curator.  Somehow it feels good to have participated, even peripherally, in the smuggling of this information into the future.

The figure shows the projected increase in global climate data holdings for climate models, remotely sensed data, and in situ.

On February 11 Science magazine published a special issue dedicated to the challenges that research communities face as they produce increasing quantities and types of data. One of the articles tells the story of particle physicist Siegfried Bethke, who wanted to reanalyze the data from an experiment conducted twenty years earlier. He discovered that no organized effort had been made to preserve the data, and it took him, his secretary and a graduate student two years to find it all and rewrite the now-obsolete code necessary to read it.

“The problem starts when the experiment is over, and the data used by one group of people is only understood by those people,” [Cristinel] Diaconu says. “When they go off and do other things, the data is orphaned; it has no parents anymore.” The orphan metaphor only goes so far: After a certain point, orphaned data can’t be adopted by later researchers who weren’t part of the original team. Even given the raw data, only someone intimately involved in the original experiment can make sense of it.

A particle physics study group has recommended that every large experiment hire a ‘data archivist,’ a sort of Receiver of Memory who would be responsible for making sure that data remains intelligible and accessible long after ‘the end’ of a project.

A data archivist would be a mix of librarian, IT expert, and physicist, with the computing skills to keep porting data to new formats but savvy enough about the physics to be able to crosscheck old results on new computer systems.

As indicated by another article in the special issue, “Climate Data Challenges in the 21st Century,” scientists not only need to make their data accessible to colleagues and to researchers of the future, but also to non-researchers of the present. As managers and policy-makers move to address time-sensitive issues such as climate change, the long-term soundness of their decisions will depend at least partly on the information available to them.

Increased support from the funding agencies is needed to enhance data access, manipulation, and modeling tools; improve climate system understanding; articulate model limitations; and ensure that the observations necessary to underpin it all are made. Otherwise, climate science will suffer, and the climate information needed by society—climate assessment, services, and adaptation capability—will not only fall short of its potential to reduce the vulnerability of human and natural systems to climate variability and change, but will also cause society to miss out on opportunities that will inevitably arise in the face of changing conditions.

If you plan to be in San Francisco next week, there is still time to register for the Personal Digital Archiving Conference. The conference is being held at the Internet Archive on February 24 & 25, 2011. The event blurb says it all:

From family photographs and personal papers to health and financial information, vital personal records are becoming digital. Creation and capture of new digital information has become a part of the daily routine for hundreds of millions of people. But what are the long-term prospects for this data? The combination of new capture devices (more than 1 billion camera phones will be sold in 2010) with the move from older forms of media is reshaping both our personal and collective memories. The size and complexity of personal collections growing, these collections are spread across different media (including film and paper!), and the lines between personal and professional, published and unpublished are being redrawn.

For individuals, institutions, investors, entrepreneurs, and funding agencies thinking about how best to address these issues, Personal Digital Archiving 2011 will include a variety of examples that may be replicated, and will clarify the technical, social, economic questions around personal archiving.

Long Now’s Laura Welcher will be presenting “An Archive Model with Long Term Benefits” Thursday evening.

Check out the full, provisional schedule HERE. As you will see from this line-up, this is a wonderful opportunity to see and talk to the leading researchers in this area.

You can register HERE. [Deadline: February 24]

Travel and location  information is on the conference website.