The usual “digital dark age” stories we see are the ones where people lose data because a platform obsolesces.  Business Week is running an interesting story about a computer platform that has refused to obsolesce, and it is the people who are leaving it behind – The Mainframe.  It turns out that there are still over 10,000 Mainframe computers out there churning away at major companies – representing a $3.4 billion dollar market segment.  Who knew right?

One part of the story that is poorly addressed is why these companies have not ported the functionality they are getting out of these mainframes to a more modern computer system.  Wikipedia answers that question this way:

Modern mainframe computers have abilities not so much defined by their single task computational speed (usually defined as MIPS — Millions of Instructions Per Second) as by their redundant internal engineering and resulting high reliability and security, extensive input-output facilities, strict backward compatibility with older software, and high utilization rates to support massive throughput. These machines often run for years without interruption, with repairs and hardware upgrades taking place during normal operation.

…[IBM's modern] mainframe processors such as 2008′s 4.4 GHz quad-core z10 mainframe microprocessor. IBM is rapidly expanding its software business, including its mainframe software portfolio…

So I guess we still need mainframes and they have been modernized somewhat, but it seems to me this would be better handled by cloud or cluster computing that would be more hardware and software agnostic.  My bet is that most of these systems are actually emulating other emulations several layers deep – in some cases all the way back to punch card programming.  I assume no one actually wants to unravel that spaghetti out of fear of losing some critical legacy functionality.  I welcome comments here from anyone who actually uses mainframes (and if that story is to be believed, your skill set is in high demand, congrats!)

Jeff Rothenberg once said “Digital information lasts forever – or five years, whichever comes first.”  This is basis of an interesting debate between New York Times writer Jeffrey Rosen who recently published “The End of Forgetting,” and Scott Rosenberg’s rebuttal on his blog. (Excerpt from Rosenberg below)

But Rosen is too busy hatching plans for “expire dates” on social-network postings and other artificial-forgetting schemes to give his head the Janus-turn his subject demands. The idea that the Web has a long memory is hardly new (here’s J.D. Lasica’s piece on how “The Web Never Forgets” from 1998). But there is a flipside to this notion: Information online can be fragile and fleeting, as well. There is an entropic quality to everything that is shared online. Data gets lost; servers die; databases are corrupted; formats fall into disuse; storage media deteriorate; backups fail.

Rosen’s piece along with new projects such as the Electronic Frontier Foundation’s HTTPS Everywhere project are reactions to a feeling that we are losing privacy in the digital age.  These reactions have an unfortunate side effect however – if we encrypt or auto delete our data, we will lose it forever.

Privacy and security concerns generally have a short half life.  While you might not want your drunk college photos to be a part of a future employers decision making criteria, you will certainly lament losing all your chilhood photos by the time you are 60.  If we lost the treasure trove of human to human interactions that is now being recorded on the web it would truly be a tragedy.  Imagine how much more we would know about ancient Rome, Egypt, or the Mayan culture if we could sift through their Facebook logs…

In order to demonstrate digital impermanence, scientists from the European Planets Project deposited a time capsule containing five of today’s most common types of digital objects into the Swiss Fort Knox data center. The time capsule contains a JPEG photograph, a message in Java source code, a short film in .MOV format, a web-page in HTML and a brochure in PDF.

According to the Planets website, the deposited box also included..

…conversion tools that were used to migrate the objects as well as software to open and view/use these objects and supporting software all the way down to an operating system; descriptions of the file formats, of the file systems and encodings used on the storage media; and description of all these objects and their relationship to supporting technology and recognised standards.

The TimeCapsule will be available to researchers in the future to investigate how much of its content will still be or can be made accessible and usable with the information provided.  An online version will make it possible to see the contents of the TimeCapsule and experiment with technology to preserve them. Replicas will be available to libraries, archives, science museums and others for research and public exhibit.

The Planets TimeCapsule will demonstrate in ten, 20, 30, 50 and hundreds of years the fragility of digital data and the ability of technology to overcome it.

Long Now has compiled a record of all of the drawings made to create the first 10,000 Year Clock Prototype into a new book. Geared towards the mechanically inclined, this book has the technical drawing of every part used in the first prototype. It also includes several math notebooks and spreadsheets that Danny Hillis used to make the underlying calculations for parts of the Clock.

Long Now hopes that the widespread distribution of these plans will ensure that the knowledge and work that went into building our first Clock prototype is not lost and that this will help the survival of the Clock itself and the long-term thinking it represents.

The paperback book is available through Amazon (we do get a % of sales) and is $19.95. The drawings are also available to download for free on our site.

Trees on the second story of the abandoned Carnegie Library in Camden NJ. Photo: Camilo Jose Vergara.

Today we received another email about creating a record of humanity and technology that would help restart civilization.  The latest one is inspired by an essay that James Lovelock published in Science over 12 years ago called A Book For All Seasons (excerpt):

We have confidence in our science-based civilization and think it has tenure. In so doing, I think we fail to distinguish between the life-span of civilizations and that of our species. In fact, civilizations are ephemeral compared with species. Humans have lasted at least a million years, but there have been 30 civilizations in the past 5000 years. Humans are tough and will survive; civilizations are fragile. It seems clear to me that we are not evolving in intelligence, not becoming true Homo sapiens. Indeed there is little evidence that our individual intelligence has improved through the 5000 years of recorded history.

Over the years these proposals have been in different forms; create a book, set of books, stone tablets, micro-etched metal disk, or a constantly updated wiki.  I really like the idea of creating such a record, in fact the Rosetta Disk project was our first effort in this direction.  These Doomsday Manuals are a positive step in the direction of making a softer landing for a collapse, and the people creating them (like ourselves) are certainly out to help people.  It took millennia for the world to regain the technology and levels of societal organization attained by the Romans, so maybe a book like this would help that.

However it also seems that these efforts tap a romantic notion that we would all love to find something like this book from a past or otherwise alien civilization.  My worry is that it also feeds off a (likely incorrect) feeling that somehow collapse might be a fun challenge to live through, and that everyone kind of wants to be the monk in A Canticle For Leibowitz or Mel Gibson in Road Warrior.

My bet is that the reality of watching your civilization (and population) collapse is likely one of the worst things anyone could experience.  I am also not so sure the problem is just knowing how to remake a technology.  For instance after the fall of the great Egyptian, Mayan, and Roman empires we had evidence and examples of their engineering achievements all around us.  But aqueducts or senate buildings are worthless without a society around them to maintain, contextualize and protect them.

It is also worth pointing out that there are likely well over a billion people on earth who currently don’t interact with formal economies or technological society at all.  They will be very well adapted to a post collapse world, you should find some and make friends.  They will likely be far more helpful than a manual on restarting the internet, because they know how to gut a deer.

In any case I thought I would create this blog post which I will try and keep updated as these proposals and efforts come to me (and hopefully come to fruition).  I will also list some of the resources that I usually refer to when I get these inquiries.   Please note these resources are extremely biased toward the English language, the United States and Western culture.  Also note that one of the first things that comes up when creating any compendium style work is the issue of copyright.  It might sound ridiculous that you might worry about copyright in a doomsday manual, but if you want to publish it and get it into peoples hands before the apocalypse, you are going to have to deal with it in some way. Please feel free to use the comments field to make suggestions and pointers and I will integrate them here as well.

Projects that are attempts in this direction:

• The Rosetta Project: A multi-millennial micro-etched disk with a record of thousands of the worlds languages.
• Westinghouse Time Capsules: Two time capsules (they actually coined the term for this project) by Westinghouse buried at Worlds Fair sites, one in 01939 and the other 01965 to be recovered in 5000 years.  They also did the very smart thing of making a “Book of Record” and an above ground duplicate of the contents on display.
• The Human Document Project: A German project to create a record of humanity that will last one million years.
• Crypt of Civilization: A airtight chamber located at Oglethorpe University in Atlanta, Georgia. The crypt consists of preserved artifacts scheduled to be opened in the year 8113 AD.
• The Voyager Record: The Voyager Golden Record are phonograph records which were included aboard both Voyager spacecraft, which were launched in 1977. They contain sounds and images selected to portray the diversity of life and culture on Earth, and are intended for any intelligent extraterrestrial life form, or far future humans, who may find them.
• Georgia Guidestones: The four granite Guidestones are covered in inscriptions written in 8 major languages that describe the tenets of their imagined Age of Reason.
• (added) Doomsday Chests by Noah Raford
• (added) The Forever Book an idea by Kevin Kelly

Content that has been discussed to be used for these projects:

• The Gingery books always seemed to me to be a great first pass on how to re-start manufacturing technology
• (added) Wiki How has a lot of great info and it is continuously updated.  The entry on how to deliver a baby seems like a particularly handy one…
• (added) The Foxfire Books on homespun technology seem to have a slightly less industrial take than the Gingery books, and are pretty comprehensive
• (added) The Lets Say Youve Gone Back in Time poster to help you restart civilization by Ryan North the creator of the awesome Dinosaur Comics
• (added) The Way Things Work by David Macaulay.  This is a fantastic book, but it might leave people thinking that all technology is powered by woolly mammoths and angels.
• The Harvard Classic‘s originally known as Dr. Elliots Five Foot Shelf are often referred to as an item that should go into a record like this.
• Encyclopedia Britannica People often suggest using the latest version that is now out of copyright.  I believe this is the 13th edition but so far I have only found digital copies of the 11th.
• The Domesday book: The Domesday Book is the record of the great survey of England completed in 1086.  It would be interesting to find surveys and census’ from around the world
• The Mormon Genealogical Data:  This is also held in a bunker outside Salt Lake City Utah, but it might be nice to have a record of gene lines for a future civilization to better understand its past.
• The Top 100 Project Gutenberg books: If you are concerned with archiving works in copyright this is a great source to find texts that are free to use.
• The Internet Archive: An archive of complete snapshots of the web as well as thousands of books and videos.  Incidentally you would also get all of our scanned page content from the Rosetta Project with this.
• Wikipedia: The text only version of this is actually not that large, and could be archived fairly easily.  Also one of the few sources that is beginning to get filled out in many languages and is also not held under a copyright.
• How to field dress a deer: PDF pocket version from Penn State College of Agricultural Science (living in Northern California, I think this one will be especially handy).

Researchers at NASA’s Ames Research Center recently were able to recover some very old and useful data.  The Nimbus II satellite created a detailed mosaic of the earth’s cloud cover and heat radiation in 1966.  Such old and detailed climate data is a boon to today’s researchers, but it wasn’t easy to come by.  Indeed, the data was lost for quite some time due to the tapes on which it was recorded – the secret to their longevity was whale-oil, but it became unattainable in the 1980′s due to the cessation of commercial whaling.  Since they couldn’t get more long-lasting tapes, NASA chose not to keep the old data, but rather to rewrite the tapes with newer data that they decided needed to be preserved for the long-term more than the old data that, when it was new, needed to be preserved for the long-term but, once it was old, did not.  Some 200,000 tapes endured this fate.

What about the machines used to read the tapes?  Perhaps it was a form of penance: many of them ended up being dumped into the ocean to create coral reefs. Fortunately, a team of techno-archaeologists working as the Lunar Orbiter Image Recovery Project have been working to track down old and lost copies of tapes like these, as well as restore the machines required to read them.  Luckily, some of the Ampex tape drives made their way into the garage of a woman named Nancy Evans, an engineer from Jet Propulsion Laboratory.

The LOIRP team obtained $750,000 from NASA and private enterprise and enlisted the assistance of a retired Ampex engineer. They cleaned, rebuilt, and reassembled one drive, then designed and built equipment to convert the analog signals into an exact 16-bit digital copy. “It was like dumpster diving for science,” says Cowing, co-team leader at LOIRP.

- NASA Lunar Science Institute

(via Metamodern)

Death Bear visits clients in their homes and accepts love letters, old photos, anything they can't just throw away. The man behind the mask, Nate Hill, says he wants to create art that helps people. (Carolyn Cole / Los Angeles Times / March 18, 2010)

At The Long Now Foundation we spend a lot of time trying to figure out how to preserve information and artifacts from our increasingly ephemeral culture.  A piece in the LA TImes sent in this morning by board member Paul Saffo reminded me of a point that Brian Eno brought up at our first conference on digital preservation: the case for forgetting.

If we were able to save and recall absolutely everything, we have to remember that sometimes the past can be as stifling as it is informative.  Many great inventions for instance may never have been created if the inventors actually knew how many great minds failed before them.  But aside from innovation there is also the emotional side to memory.  This story about the Death Bear project reminds us that there is plenty that we may want to forget, and that by doing so we can liberate our future. (excerpt below)

And while most of his calls are from the lovelorn, others hint at tragedies greater than being dateless on Valentine’s Day.

One man gave Hill a photo of himself and his ex-girlfriend on a beach and said they had served in the Army together. Then he gave Hill his military dog tags. Finally, he handed Hill a bullet.

“He almost started to cry,” said Hill, whose clients know him only as Death Bear and never see his face. “I started walking away and started to break down. I thought maybe something happened to her. Maybe she got shot, maybe she killed herself.”

But Hill never presses clients for details. As a bear, his job is not to make conversation. (read the full article)

Salman Rushdie at Emory University in Atlanta, which is currently exhibiting his personal archive, including personal papers, and electronically produced drafts of his novels.

Stewart Brand sends in this excellent piece in the The New York Times on what I am sure is to be an oft repeated story.  As museums ingest invaluable intellectual material from authors and thinkers that increasingly will have never used paper, they are finding that preserving this data is a many layered problem.  Huge Kudos goes out to Emory University for pulling together a full emulated environment of Rushdie’s word processor to recreate the digital “environment” for others to see into his process.  I suspect this emulation strategy will be used more and more…

Electronically produced drafts, correspondence and editorial comments, sweated over by contemporary poets, novelists and nonfiction authors, are ultimately just a series of digits — 0’s and 1’s — written on floppy disks, CDs and hard drives, all of which degrade much faster than old-fashioned acid-free paper. Even if those storage media do survive, the relentless march of technology can mean that the older equipment and software that can make sense of all those 0’s and 1’s simply don’t exist anymore.

Imagine having a record but no record player.

All of which means that archivists are finding themselves trying to fend off digital extinction at the same time that they are puzzling through questions about what to save, how to save it and how to make that material accessible.

“If you’re interested in primary materials, you’re interested in the context as well as the content, the authentic artifact,” Ms. Farr said. “Fifty years from now, people may be researching how the impact of word processing affected literary output,” she added, which would require seeing the original computer images.

(…continue reading at NYT)

It seems unavoidable that most of the data in our future will be digital, so it behooves us to understand how to manage and preserve digital data so we can avoid what some have called the “digital dark age.” This is the idea—or fear!—that if we cannot learn to explicitly save our digital data, we will lose that data and, with it, the record that future generations might use to remember and understand us.

It’s a fairly long and comprehensive piece with lots of good advice and a good description of how the Rosetta Disk tries to address some of these problems.

Read the full article at New Scientist.

King’s College London president Rick Trainor announced recently that the university would be closing the chair of paleography, the UK’s only one.  Held by Professor David Ganz, the chair of paleography is the position that overseas a discipline many consider to be a vital component of historical research.  Paleography is the study of ancient manuscripts and has pieced together and deciphered many of the texts that have provided the basis for our knowledge of history.

Budget cuts are the precipitating factor, or rather “strategic disinvestment” as the official announcement goes, but they’re being met with some resistance.

“Palaeography is not simply an arcane auxiliary science,” says Professor Jeffrey Hamburger, chair of medieval studies at Harvard University. “It is as basic to the training and practice of ­historians as mastery of Dos or Unix might be to a computer scientist.”

-from the Guardian

Rosetta Project Director Laura Welcher recently took part in a segment on The History Channel’s Life After People series.

In an episode titled “Crypt of Civilization,” Laura discusses the Rosetta Disk and The 10,000 Year Clock.   

The central question of the series is “How long would it last?” The series explores various materials, systems and structures built by humans to determine their durability sans maintenance as well as natural systems and how they might flourish or decline without human intervention.

“Crypt of Civilization” focuses on time capsules, vaults and other attempts to create long-lasting caches of materials or data.  Laura explores some of the unique challenges in designing artifacts like the Disk and Clock to last thousands of years while the show’s producers vividly illustrate them.

You can watch the series on its website (though the “Crypt of Civilization” episode isn’t available yet).

]]> http://blog.longnow.org/2010/02/04/rosetta-and-long-now-on-life-after-people/feed/ 1 http://blog.longnow.org/2009/12/15/1000-years-of-forgetting/ http://blog.longnow.org/2009/12/15/1000-years-of-forgetting/#comments Wed, 16 Dec 2009 00:50:32 +0000 Kevin Kelly http://blog.longnow.org/?p=1331
One thousand years from now, much of what we know will be forgotten. That’s been true in the past. We have only a fragmentary cultural memory of what happened 1,000 years ago. And what we think we know about 1000 may in fact be quite garbled. In a very witty demo of this, this youtube clip, the Beatles 3000, imagines how corrupted our current ideas of “what everone knows” will most likely be in 10 centuries. Ever heard of the Beatles?  (Thanks, Mark)

Long Now friend and supporter Ken Wilson sends in this awesome concept for the Stockholm Library.  This design seems like it would lend itself well to a 10,000 year library…

The image above is a rendering by a team of students at the Architecture School of Paris La Seine. You can see the un-textured model below and read how the design was generated over at CG Society.

The European Space Agency’s Rosetta probe made its final flyby of the Earth on Friday in order to fling itself off towards its target: Comet 67P/Churyumov-Gerasimenko.

Launched in 02004, Rosetta has made several planetary flybys in order to gain the velocity necessary to approach and eventually orbit the comet so that a small landing craft can touchdown upon and sample some of the comet’s material.  Scientists hope that a better understanding of the make-up of a comet will be like a key that will unlock many secrets about the formation of the planets and the development of our solar system.

Included on the craft is one of the early Rosetta Disks produced by Long Now.  The highly durable, format-independent linguistic archive will survive as long as the craft continues to orbit Comet 67P.  Unlike the Voyager Disks, this terrestrial artifact will remain in our solar system orbiting the comet, which is orbiting the Sun and will continue to do so until it runs into something (which could be quite a while).

You can see lots of great photos and amazing animations on the Rosetta blog, run by the ESA.  In addition, there was a lovely little piece in the Guardian highlighting the mission’s long-term nature:

The scientific pay-off from Rosetta could be huge. But contemplate the generosity of vision that made the mission possible. Some of those who lobbied for Rosetta will have died by the time the first results are delivered. Some young scientists who will build their careers on the data from Rosetta were not born when the mission was conceived. If, as Harold Wilson famously observed, a week is a long time in politics, Rosetta is a reminder that we can also think on a celestial timescale.

What seems to be the first real optical archival digital tech is now shipping. The Millenniata product is a type of DVD storage that uses a mechanical scratching process, instead of a thermal process, making the media vastly more stable.  The disks are in the current DVD standard and the company claims they are therefore backwards compatable to normal players.  To write your own disk however you will need the $1700 writer and one of the special blank disks that range from $16-$25ea depending on qty.

If the companies claims on life-span of the media are true this is a major milestone in commodity level archival media.  I do think however that they really need some sort of marking on the tops of all the blank media that explains what the DVD data stadard is and how to read it.  Otherwise in a 100 years, I cant imagine that many people will remember…