[Dean's World] Dean: The Friendliness Problem

[notify at powerblogs.com]

Posted by Dean:
The Friendliness Problem
http://www.deanesmay.com/posts/1152311639.shtml


   Artificial Intelligence researchers often discuss what they call the
   "friendliness problem." If you develop an AI (Artificial
   Intelligence), what happens if you design one that is not friendly to
   humans? To researchers, this is not a SciFi question: what happens if
   you develop AIs that have certain capabilities, but they are supremely
   indifferent to humans and human values?

   The usual answer to this is to develop AIs that mimick humans as much
   as possible, and have as their central focus making people happy.
   Matoko-chan [1]has a meditation on that question, with a pretty good
   [2]response by Phil Bowermaster.

   Most people assume that we're generations away from having to think
   about these questions, but they're wrong. AI gets better all the time,
   and growth in this field is exponential just like most other
   technological fields. Indeed, what's most noticeable about AI is that
   whenever AI does something that was previously considered
   impossible--compose a pretty song, improvise music with a live
   musician, play master's level chess, pilot an airplane from takeoff to
   landing, identify a specific building and blow it up, drive a car
   through an obstacle course, diagnose a sick patient and recommend
   viable treatment, speak intelligibly, respond to verbal commands,
   recognize a specific human's face (all of which AIs are now capable
   of)--people tend to immediately say, "okay, but that's not real
   intelligence, that's not real creativity." Which is question-begging
   at best: so what is real intelligence, what is real creativity? If you
   can't tell the difference between something an AI produced and
   something a human produced, what is your justification for saying that
   the AI didn't "really" do anything intelligent?

   Computers still double in power every couple of years, with no end in
   sight. Software tends to lag behind hardware, although that's getting
   faster too because an increasing amount of software writes code and
   designs hardware by itself--at the direction of humans, yes, but
   what's the difference?

   This is not a matter of one day a supercomputer "waking up" like
   SkyNet in the Terminator movies. Rather, it's a matter of different
   technologies to do different things all gradually growing both more
   intelligent and more autonomous in their functioning. And how to
   safeguard against them--not against their becoming malevolent, but
   against their doing things that are hideously destructive because
   there's nothing in them to stop them from doing so.

References

   1. http://quantumghosts.blogspot.com/2006/07/dr-yes-and-friendliness-problem.html
   2. http://www.blog.speculist.com/archives/000900.html