Daniel Suarez, “Daemon: Bot-Mediated Reality”

Hi Ben,

I would agree that portraying bots as a plot to ‘eliminate those pesky humans’ would be unhelpful and irresponsible. Fortunately that’s not what my talk was about, nor did I imply this. In fact, I clearly stated that bots and the systems which support them are not ever going away and that they provide significant benefits to humanity. What I have a problem with is the reckless pursuit of efficiency (by human actors) at the expense of most everything else. This results in centralization and monoculture which greatly decreases the flexibility and robustness of civilization as a whole. Bots (a subset of programs that search for, retrieve, and/or react to external data) are the main agent of this consolidation, and we as a society should increase the transparency of systems which have a direct affect on the populace. At present, the public has little or no right to inquire into the logic of proprietary algorithms that have significant power over their lives (e.g., FICO-scoring algorithms, etc.)

Incidentally, ‘Narrow AI’ consists mainly of pattern-recognition systems, and is clearly delineated from “Strong AI”. At no point in my talk did I conflate “AI” with “Narrow AI”. They are two distinctly different things.
`
Thanks for taking an interest and responding to what you considered to be irresponsible, though. I’m always glad to see people not putting up with what they view as BS…

Best,

D.S.

Dear Daniel,

A fundamental problem with our computer infrastructure is that machines are vulnerable to take-over by bits of code delivered in any of many different ways. We build fragile systems in which small bugs in a single part can give attackers control of the whole.

There is a widespread perception that this is inevitable, inherent in the nature of computation. This is false: There are known ways to structure an object-oriented language such that, if the kernel language itself is implemented correctly, no object written in that language can gain more authority (to read files, to communicate, to continue running…) than it is given by the objects that created it or called it. This turns out to be very powerful: Instead of relying on firewalls for protection, using these methods is like building with incombustible materials. No single part, however corrupt it may be, can take control of the whole. The amount of code that must be correct is small, where today it is unlimited.

The same abstract structure can be built into the microkernel of an operating system, embedding all running programs, regardless of language, in an environment where they can see only what they are shown, and can affect only what is placed within reach.

Without this structure, I know of no way to implement open, distributed systems that we can trust.

Fortunately, after long incubation, the language-level approach is getting a firm foothold in the world, through Google and social networking sites (search “Caja programming language”). At the OS level, the technology is established, but dissemination is further away (search “CapROS: The Capability-based Reliable Operating System”; “Coyotos Secure Operating System”).

The underlying ideas, by the way, had for many years been obscured by a fog of academic confusion stemming from the 1970s. That fog is now lifting, but there are still confused experts wandering around. They are building firewall-style security, trying to find every last bug in every piece of code, etc., etc., and practitioners of these failed approaches dominate the discussion space by their sheer number. Understanding what is going on at a deeper level, and what it can mean, requires a sensitive ear.

Benjamin said…
These ‘bots’ you speak of? The ones calculating our FICA scores? The ones connecting our calls? They are conventionally called programs.

Yes, but they are programs that inject intelligence into it. Do you want to re-define what it means by that in the computing literature? I recommend starting with Peter Norvig’s excellent book on the subject (Google’s head of research), with title: Artificial Intelligence – A Modern Approach and after that book, then you can move on to other advanced ones.

Benjamin said…
Calling them ‘bots’ misleadingly grants them autonomy.

Again, do you want to re-define jargon used in the computing literatures? Are you familiar with autonomous software agent systems? If not, then again, Peter Norving’s book is a good start in learning about autonomous software agents, and after that then you can go on to other advanced books on the subject. There are also open source software available in autonomous agents (such as JADE – a Java Multi-agents system). The meaning of autonomous is just that. To mimic human capability of autonomy in terms of software. Well, can you re-define that definition which has already been defined by pioneers in the field for over the last 40 years or so. I think that your argument is either based on the semantics of the words, which you’re unaware that indeed there is a formal branch of computing that exactly meant what the word says, ie, autonomous. I guess that in your case , it is the latter.

Benjamin said…
Programs don’t “roam” on their own, they don’t evolve, and to call them (even narrow) AI is completely wrong – unless you want to define intelligence as “doing what you have been programmed to do”.

Again, that is what autonomous multi-agent software is about, ie, roaming. So, don’t get hung up on word semantic, because it exposes your lack of knowledge of the field. Learning agents accumulate knowledge that its designers never anticipated it would, well that is what evolving learning agent is all about. This exactly how human learns his/her external world. Do you want to know more, then do a Google on the term Machine Learning (a sub-branch of AI).

Daniel Suarez, I interpret your definition of Strong AI to mean evolving/learning based AI system, and Weak AI non-evolving/non-learning based AI system. These systems are already available today, and that’s exactly what they are.

You might be interested in a similar view expressed by Sun MicroSystem’s former chief scientist, Bill Joy in his article in Wired Magazine published in 2001.

Why the future doesn’t need us (Our most powerful 21st-century technologies – robotics, genetic engineering, and nanotech – are threatening to make humans an endangered species).

His view is that the next revolution in IT comes from Physicist and not from computer scientists, electrical engineers, software architect, etc… This is true as physicists are starting to make breakthroughs in the domain of “Quantum Computing” which will make classical computing redundant at some stage. Programmers will need to be familiar with Quantum Physics (which is something very difficult) in order to write software, when the age of Quantum computing arrives.

Great review with Daniel here:

http://www.crimecritics.com/2009/01/interrogating-daniel-suarez-author-of-daemon/

So, is anyone working on the darknet/op system project. (I realize I’m a bit behind the most recent talks.)