Singularities and Nightmares

"Options for a coming singularity include self-destruction of civilization, a positive singularity, a negative singularity (machines take over), and retreat into tradition. Our urgent goal: find (and avoid) failure modes, using anticipation (thought experiments) and resiliency -- establishing robust systems that can deal with almost any problem as it arises."

David Brin KurzweilAI.net:

In order to give you pleasant dreams tonight, let me offer a few possibilities about the days that lie ahead—changes that may occur within the next twenty or so years, roughly a single human generation. Possibilities that are taken seriously by some of today's best minds. Potential transformations of human life on Earth and, perhaps, even what it means to be human.

For example, what if biologists and organic chemists manage to do to their laboratories the same thing that cyberneticists did to computers? Shrinking their vast biochemical labs from building-sized behemoths down to units that are utterly compact, making them smaller, cheaper, and more powerful than anyone imagined. Isn't that what happened to those gigantic computers of yesteryear? Until, today, your pocket cell phone contains as much processing power and sophistication as NASA owned during the moon shots. People who foresaw this change were able to ride this technological wave. Some of them made a lot of money.

More here.