Superintelligence: Paths, Dangers, Strategies

Just a few annotations:

“A big breakthrough in artificial intelligence. It seems somewhat likely that it will happen sometime in this century, but we don't know for sure”.

-I don't think so, but I truly hope to see that with my eyes.

“The idea of a coming technological singularity has been popularized, starting with Vernor Vinge's seminal essay and continuing with the writings of Ray Kurzweil and others. The term ‘Singularity,' however, has been used confusedly in many disparate senses and has accreted an unholy aura of techno-utopian connotations”.

-Agree.

“Machines matching humans in general intelligence have been expected since the invention of computers in the 1940s”

-Add another century to it.

“The fact that the best performance at one time is attained through a complicated mechanism does not mean that no simple mechanism could do the job as well or better. It might simply be that nobody has yet found the simpler alternative”.

-That's my bet for developing a SI.

“How far are we currently from achieving a human whole brain emulation?. One recent assesment presented a technical roadmap and concluded that the prerequisite capabilities might be available around mid-century, though with a large uncertainty level”.

- I also doubt a WBE in the near future. Probably in 2120?.

“Final goal: ‘make us smile'.
Perverse instantiation: ‘paralyze human facial musculatures into constant beaming smiles”.

- I laughed on this because it makes sense.

“...These observations make it plausible that any type of entity that developed a much greater than human level intelligence would be potentially extremely powerful. Such entities could accumulate content much faster than us and invent new technologies on a much shorter timescale. They could also use their intelligence to strategize more effectively than we can”.

- I liked this book a lot specially because of the Realistic Pessimism about an SI. I consider there must be a real concern to think about that level of intelligence.