Daniel Suarez, “Daemon: Bot-Mediated Reality”
August 19th, 02008 by Paul Saffo
[Daniel Suarez, originally published as Leinad Zaurus, delivered a talk on the themes developed in his (originally self published) book Daemon. The book is now scheduled to be released in hard cover in January 02009 by Dutton. Here is Paul Saffo’s write up of the evening.]
Forget about HAL-like robots enslaving humankind a few decades from now, the takeover is already underway. The agents of this unwelcome revolution aren’t strong AIs, but “bots”– autonomous programs that have insinuated themselves into the internet and thus into every corner of our lives. Apply for a mortgage lately? A bot determined your FICA score and thus whether you got the loan. Call 411? A bot gave you the number and connected the call. Highway-bots collect your tolls, read your license plate and report you if you have an outstanding violation.
Bots are proliferating because they are so very useful. Businesses rely on them to automate essential processes, and of course bots running on zombie computers are responsible for the tsunami of spam and malware plaguing Internet users worldwide. At current growth rates, bots will be the majority users of the Net by 2010.
We are visible to bots even when we are not at our computers. Next time you are on a downtown street, contemplate the bot-controlled video cameras watching you, or the bots tracking your cellphone and sniffing at your Bluetooth-enabled gizmos. We walk through a gauntlet of bot-controlled sensors every time we step into a public space and the sensors are proliferating.
Bots are at best narrow AI, nothing that would make a cleric remotely nervous. But they would scare the hell out of epidemiologists who understand that parasites don’t need to be smart to be dangerous. Meanwhile, the Internet and the complex of processing, storage and sensors linked to it is growing exponentially, creating a vast new ecology for bots to roam in. Bots aren’t evolving on their own — yet.
Left unchecked, bots will trap the human race because the automation they enable will make it possible for a few people to run humanity while the rest of us are unable to make decisions of any consequence. Bots are thus vectors for despotism, with the potential to create a world where only a small group of people understand how society works. In the worst case, the controls over bots disappear — for example, the only person who knows the password to a corporate bot dies– and the bots become autonomous.
We are in a Darwinian struggle with narrow AI, and so far at least the bots are winning. But there is a solution: build a new Internet hard-coded with democratic values. Start with an encrypted Darknet into which only verifiably human users can enter. Create augmented reality tools to identify bots in the physical world. Enlist the aid of a few tame bots to help forge a symbiotic relationship with narrow AI. Stir in some luck, and perhaps we can avoid the fate of the Sorcerer’s Apprentice who rashly enchants a broom to do his tedious chores and ends up terrorized by his imperfect creations. We had better succeed, for unlike the fable, there is no Master Sorcerer ready to return to break the spell and save us from our folly.
-Paul Saffo
August 19th, 2008 at 12:18 pm
These ‘bots’ you speak of? The ones calculating our FICA scores? The ones connecting our calls? They are conventionally called programs. Calling them ‘bots’ misleadingly grants them autonomy. Programs don’t “roam” on their own, they don’t evolve, and to call them (even narrow) AI is completely wrong - unless you want to define intelligence as “doing what you have been programmed to do”. In that case, my laptop, my phone, and my MP3 player are intelligent. Narrow AI, much less general AI as required by your bots, is still a fair bit away.
This doesn’t address your concerns, however, applied to buggy or mis-designed programs, instead of bots - those are still valid. The solutions, however, is not to dream up some magical “human-only internet” (how would such a thing be enforced, anyway? See: Turing Test). Enlisting the “aid” of “bots” (programs) doesn’t make any sense in today’s world.
What we can (and should) focus on is writing and auditing our programs and systems to avoid errors and handle unexpected situations gracefully. This is the realistic option. Finally, we can build in emergency procedures that are less destructive than simply pulling the plug.
There are real issues around quality control, safety procedures, and emergency situations with our computer systems, but coloring them as a distributed and unmanageable plot by a bunch of self-aware “bots” to eliminate those pesky humans is unhelpful and irresponsible.
August 23rd, 2008 at 10:42 am
Hi Ben,
I would agree that portraying bots as a plot to ‘eliminate those pesky humans’ would be unhelpful and irresponsible. Fortunately that’s not what my talk was about, nor did I imply this. In fact, I clearly stated that bots and the systems which support them are not ever going away and that they provide significant benefits to humanity. What I have a problem with is the reckless pursuit of efficiency (by human actors) at the expense of most everything else. This results in centralization and monoculture which greatly decreases the flexibility and robustness of civilization as a whole. Bots (a subset of programs that search for, retrieve, and/or react to external data) are the main agent of this consolidation, and we as a society should increase the transparency of systems which have a direct affect on the populace. At present, the public has little or no right to inquire into the logic of proprietary algorithms that have significant power over their lives (e.g., FICO-scoring algorithms, etc.)
Incidentally, ‘Narrow AI’ consists mainly of pattern-recognition systems, and is clearly delineated from “Strong AI”. At no point in my talk did I conflate “AI” with “Narrow AI”. They are two distinctly different things.
`
Thanks for taking an interest and responding to what you considered to be irresponsible, though. I’m always glad to see people not putting up with what they view as BS…
Best,
D.S.
August 24th, 2008 at 3:10 pm
It seems to me that main quote I would take issue with, and what the entire talk turns on is this:
“reckless pursuit of efficiency…” which “results in centralization and monoculture which greatly decreases the flexibility and robustness of civilization as a whole.”
I disagree.
In fact, the whole wisdom of the crowds concept that has been gaining such cache in the past year or so undermines it as well. For example, the efficiency of wikipedia comes from the *decentralization* of the work. By embracing this emergent aspect of intelligence, and designing tools to support it, humans are rapidly increasing the mean intelligence of the race.
But we’re still “stupid, panicy animals”… for now
August 26th, 2008 at 10:14 am
Dear Daniel,
A fundamental problem with our computer infrastructure is that machines are vulnerable to take-over by bits of code delivered in any of many different ways. We build fragile systems in which small bugs in a single part can give attackers control of the whole.
There is a widespread perception that this is inevitable, inherent in the nature of computation. This is false: There are known ways to structure an object-oriented language such that, if the kernel language itself is implemented correctly, no object written in that language can gain more authority (to read files, to communicate, to continue running…) than it is given by the objects that created it or called it. This turns out to be very powerful: Instead of relying on firewalls for protection, using these methods is like building with incombustible materials. No single part, however corrupt it may be, can take control of the whole. The amount of code that must be correct is small, where today it is unlimited.
The same abstract structure can be built into the microkernel of an operating system, embedding all running programs, regardless of language, in an environment where they can see only what they are shown, and can affect only what is placed within reach.
Without this structure, I know of no way to implement open, distributed systems that we can trust.
Fortunately, after long incubation, the language-level approach is getting a firm foothold in the world, through Google and social networking sites (search “Caja programming language”). At the OS level, the technology is established, but dissemination is further away (search “CapROS: The Capability-based Reliable Operating System”; “Coyotos Secure Operating System”).
The underlying ideas, by the way, had for many years been obscured by a fog of academic confusion stemming from the 1970s. That fog is now lifting, but there are still confused experts wandering around. They are building firewall-style security, trying to find every last bug in every piece of code, etc., etc., and practitioners of these failed approaches dominate the discussion space by their sheer number. Understanding what is going on at a deeper level, and what it can mean, requires a sensitive ear.
August 26th, 2008 at 10:31 pm
Wow, astounding talk. This is really innovative, creative stuff. I wonder to what extent these bluescanners and car-whisperers are already being used on the public? The technology is there to do it, but is there data out there about how many of these systems are in operation, or how many people may be subject or victimized by these? Even the thought of spam through my car radio frightens me.
Daniel, to what extent are you going to lobby for - or even engineer - this new human-centered, bot-mediated reality network (this New-Internet)? Do you have plans to work on this infrastructure yourself?
Also, if anyone can recommend where i can buy a copy of Daemon now, I’m too interested to wait until January!
Thanks!
Trevor
coopstah@gmail.com
September 14th, 2008 at 10:13 pm
Benjamin said…
These ‘bots’ you speak of? The ones calculating our FICA scores? The ones connecting our calls? They are conventionally called programs.
Yes, but they are programs that inject intelligence into it. Do you want to re-define what it means by that in the computing literature? I recommend starting with Peter Norvig’s excellent book on the subject (Google’s head of research), with title: Artificial Intelligence - A Modern Approach and after that book, then you can move on to other advanced ones.
Benjamin said…
Calling them ‘bots’ misleadingly grants them autonomy.
Again, do you want to re-define jargon used in the computing literatures? Are you familiar with autonomous software agent systems? If not, then again, Peter Norving’s book is a good start in learning about autonomous software agents, and after that then you can go on to other advanced books on the subject. There are also open source software available in autonomous agents (such as JADE - a Java Multi-agents system). The meaning of autonomous is just that. To mimic human capability of autonomy in terms of software. Well, can you re-define that definition which has already been defined by pioneers in the field for over the last 40 years or so. I think that your argument is either based on the semantics of the words, which you’re unaware that indeed there is a formal branch of computing that exactly meant what the word says, ie, autonomous. I guess that in your case , it is the latter.
Benjamin said…
Programs don’t “roam” on their own, they don’t evolve, and to call them (even narrow) AI is completely wrong - unless you want to define intelligence as “doing what you have been programmed to do”.
Again, that is what autonomous multi-agent software is about, ie, roaming. So, don’t get hung up on word semantic, because it exposes your lack of knowledge of the field. Learning agents accumulate knowledge that its designers never anticipated it would, well that is what evolving learning agent is all about. This exactly how human learns his/her external world. Do you want to know more, then do a Google on the term Machine Learning (a sub-branch of AI).
Daniel Suarez, I interpret your definition of Strong AI to mean evolving/learning based AI system, and Weak AI non-evolving/non-learning based AI system. These systems are already available today, and that’s exactly what they are.
You might be interested in a similar view expressed by Sun MicroSystem’s former chief scientist, Bill Joy in his article in Wired Magazine published in 2001.
Why the future doesn’t need us (Our most powerful 21st-century technologies - robotics, genetic engineering, and nanotech - are threatening to make humans an endangered species).
His view is that the next revolution in IT comes from Physicist and not from computer scientists, electrical engineers, software architect, etc… This is true as physicists are starting to make breakthroughs in the domain of “Quantum Computing” which will make classical computing redundant at some stage. Programmers will need to be familiar with Quantum Physics (which is something very difficult) in order to write software, when the age of Quantum computing arrives.
September 17th, 2008 at 9:28 am
[…] of the lectures I’m particularly looking forward to downloading is the recent Daemon: Bot-Mediated Reality lecture by Daniel Suarez, which I think has particular relevance given the recent and rather large […]
October 20th, 2008 at 7:19 pm
[…] and influential guests, including Craig Venter, Jimmy Wales, Francis Fukuyama, and Ray Kurzweil, Daemon: Bot-Mediated Reality by author and software engineer Daniel Suarez was one of the most interesting and thought-provoking […]
November 7th, 2008 at 11:55 pm
[…] The Long Now Blog » Blog Archive » Daniel Suarez, “Daemon: Bot-Mediated Reality” We are visible to bots even when we are not at our computers. Next time you are on a downtown street, contemplate the bot-controlled video cameras watching you, or the bots tracking your cellphone and sniffing at your Bluetooth-enabled gizmos. We walk through a gauntlet of bot-controlled sensors every time we step into a public space and the sensors are proliferating. (tags: privacy future ai security) Evangelize:These icons link to social bookmarking sites where readers can share and discover new web pages. […]
December 3rd, 2008 at 10:39 am
[…] Suarez gave a talk about his book at the Long Now Foundation. There is a video of the event at Fora […]
December 30th, 2008 at 6:20 am
I like the idea of an Internet that only humans can enter. Even better would be the impossible dream of an Internet with no anonymity for those of us that have nothing to fear but those full of fear.
Technology will continue to do more good than harm. I get amused by those who pretend to have a long view of history by looking forward several hundred years. The sobering reality is the impending lifelessness of our planet no more than 4.5 billion years from now. One could argue that the greatest increase in human well-being, efficiency, output, growth, and wealth will have been the 20th century. A 100-year period that saw the internal combustion engine gain mass acceptance, the computer, and the Internet. What did we do with this incredible period of increased wealth? We spent most of it on devices used to kill one another, and the rest on too-large houses.
Take those trillions and trillions of dollars and imagine using them to prepare for a long future for humanity. A settlement on the moon. A space station that means something. People on Mars. Ideas for reaching other stars. All of that potential was blown, like watching an immature youth churn through an early inheritance, unable to forego immediate self-gratification.
Global warming isn’t a problem, but global warring is. And no, I do not blame the United States as is popular, I blame every culture that attempts to solve their problems with violence. Cuba, N. Korea, Iraq, Iran, Israel, Palestine, Sudan, Russia, abusive husbands, petty thieves, we each play our part to some degree. The waste is enough to sicken my optimistic heart.
December 31st, 2008 at 6:49 pm
[…] HERE for the podcast, HERE for more on the speaker and HERE for more information on this […]
January 6th, 2009 at 7:12 am
Great review with Daniel here:
http://www.crimecritics.com/2009/01/interrogating-daniel-suarez-author-of-daemon/
January 9th, 2009 at 5:57 pm
[…] The Long Now Blog » Blog Archive » Daniel Suarez, “Daemon: Bot-Mediated Reality” Forget about HAL-like robots enslaving humankind a few decades from now, the takeover is already underway. The agents of this unwelcome revolution aren’t strong AIs, but “bots”– autonomous programs that have insinuated themselves into the internet and thus into every corner of our lives. Apply for a mortgage lately? A bot determined your FICA score and thus whether you got the loan. Call 411? A bot gave you the number and connected the call. Highway-bots collect your tolls, read your license plate and report you if you have an outstanding violation. (tags: technology scifi web psychology society security robots) […]
January 22nd, 2009 at 11:40 am
My wife just heard Mr. suarez on Glenn Beck’s talk radio show and she told me things about computers that I did not know.
She found the interview very interesting and informative.
Thanks,
Troy
January 23rd, 2009 at 6:24 pm
[…] http://blog.longnow.org/2008/08/19/daniel-suarez-daemon-bot-mediated-reality/ […]
January 27th, 2009 at 4:28 pm
[…] Long Now is presenting a reading and book signing of DAEMON by author Daniel Suarez, who gave a Seminar last August on “Bot-mediated Reality”. […]
February 1st, 2009 at 4:44 pm
The Daemon-Haunted World…
This very blog post is being propagated across the internet by daemons responding to trackback calls
……
February 9th, 2009 at 2:53 am
[…] for 2009–02–09 9. February 2009 Kommentare Einen Kommentar schreiben Daniel Suarez, “Daemon: Bot-Mediated Reality” (tags: daemon bot ai internet societzy democracy technology […]
May 15th, 2009 at 5:48 pm
[…] The Long Now Blog » Blog Archive » Daniel Suarez, “Daemon: Bot-Mediated Reality” […]
May 30th, 2009 at 3:25 pm
So, is anyone working on the darknet/op system project. (I realize I’m a bit behind the most recent talks.)