(Photo by David Orban.)

(Michael Vassar, Peter de Blanc, Nick Hay, Michael Anissimov. Photo by Sarah Butler.)

(Singularity Summit covered on the front page of the San Francisco Chronicle.)

Rodney Brooks, CTO of iRobot Corp, former Director (until June 30, 2007) of the MIT Computer Science and Artificial Intelligence Laboratory
The Singularity: A Period Not An Event
9:30AM – 10:30AM

- slight Australian accent, I forgot he was from Australia
- I wonder why he left MIT
- opened with a lot of too-frequently-quoted quotes – Yogi Berra etc.
- says that techno-salvation and techno-holocaust are not likely, too extreme
- presented some seemingly superfluous demographic charts
- “The Future – Exponentials”
- presented his iRobot stuff, PackBot for Iraq, mentioned the bonding thing
- FCS Future Combat Systems, 1/3 of military missions unmanned?
- “Send in the SUGV”, military putting a lot of money into robotics as well as AI
- he’s also involved with MIT, Kismet, (Kismet may be made obsolete by Hanson robotics)
- played a bunch of Kismet videos, Cynthia Brazil
- Kismet is highly uncanny – uncanny valley in action
- it reminds me of how easy it is to anthropomorphize robots, seeing complexity which isn’t there
- Kismet is able to look where people are looking
- robot “Domo” — Aaron Edsinger
- showed some interesting videos with robots-eye view of Domo
- safer to interact with physically than automobile-building robots
- aware of external forces
- dark video that was hard to see
- shows us a few optical illusions, says human vision is sophisticated
- recently, we didn’t know about Archaea
- 50% of the world’s biomass we didn’t even know about
- maybe AI is already out there (why does he think this is even possible?)
- before we have 100% AI we’ll have 99% AI
- not likely to happen accidentally
- large scale, unexplained oscillations in the Internet may be a sign of AI
- “an annoying alternative” – maybe AI is created and it ignores us
- “another alternative future” – a virus spreads among robots that take care of baby boomers, five million aging Rolling Stones fans die in one evening – heh
- robotkind sent back 50 years, postponing the Singularity
- another alternative – maybe we’re just not smart enough to create AI
- direct neural implants become elective
- drug enhancement becomes accepted
- genetic enhancement becomes normal
- neural enhancement catches on
- we and our world won’t be us anymore
- may lose distinction between us and them, this could all happen before 2029
- he likes the movie iRobot
- believes that humans and robots will become indistinguishable
- question: what was your inspiration for the user interface for the combat robot?
- answer – similar to game controller, 19-year old soldiers can use it without training
- question: will we pay an emotional price as we become symbiotic with robots??
- answer: not just going to be connections w/ machines, also drugs, a wide range, wouldn’t say that it’s damping down our emotions
- there’s a third-party industry making clothes and skins for Roombas
- “we’ll have Facebook for robots”
- question: are there robots with machine guns in the military?
- answer:no, but we need to think about that soon
- question: you’ve helped developed robotics for the US government, who has disobeyed the Geneva conventions. Why?
- answer: other scientists have been funded by the military, scientists do need to enforce control, Geneva conventions do work, governments can change, people do change, it’s an ongoing question

Eliezer Yudkowsky, co-founder of the Singularity Institute
Introducing the “Singularity”: Three Major Schools of Thought
10:30AM-11:30AM

- wrote Creating Friendly AI in 2001
- Introducing the “Singularity”: Three Major Schools
- word means a different thing today than when the Singularity Institute got started
- Ray Kurzweil: Accelerating Change, Vernor Vinge: event horizon, I.J. Good: intelligence explosion
- accelerating change: summarizes the idea, that technology accelerates, may even be a smooth exponential
- techno-juju increasing exponentially over time
- Threshold of Big Juju will be crossed at a specific time in the future, perhaps on March 4th 2015 at 9:30AM in the morning
- criticisms of exponentiality don’t hit the proposal that acceleration is still happening
- any positive derivative shows that acceleration is happening
- Event Horizon: formulated by Vernor Vinge in the 1970s
- The original “Singularity”
- realized that he couldn’t write stories with people smarter than him
- note that it’s the model of the future that breaks down, not the future itself
- something happens, we just don’t know what it is
- transhuman minds imply a weirder future than flying cars and gadgets
- brains are the source of all dangerous, beautiful and impressive things
- the brain: the trick that does all other tricks at the same time
- intelligence is not book smarts, it’s also persuasiveness, enthusiasm, rationality, musical talent, thinking on your feet, etc.
- intelligence is the foundation of human power, strength that fuels our other arts
- scale of intelligent minds – village idiot to Einstein? no.
- when he talks about intelligence, he means trans-species – a rock to flatworm to insects to chimps to humans and beyond
- if you want to see the true shape of the future, look to cognitive technologies, not gadgets with blinking lights
- bold version of Event Horizon: to predict anything a transhuman mind would do, we’d have to be at least that smart ourselves
- the Event Horizon argues against the bold version of accelerating change, that smooth exponentials will allow indefinite predictions
- this is why it’s important to disentangle all these concepts!
- bold accelerationism can be wrong and it’s not an argument against the Event Horizon
- we could reach the threshold following some totally weird trajectory, even one where technological progress even reverses
- third school: intelligence explosion, goes back to the 1960s, pre-invented by John Campbell in the 1930s, editor of Astounding Stories
- intelligence explosion: closing the loop of intelligence and technology
- what might enhanced humans use their smartness for? to design the next generation of intelligence enhancements.
- core thesis of intelligence explosion; minds making technology to improve minds is a positive feedback cycle, like a pencil falling on its side
- most extreme version of the thesis is an AI improving its own source code
- 18 years cycle for genetic engineering, 18 months for brain-computer interfaces, AI could be 18 seconds!
- comparison between neuron speed and silicon chips
- neuron speed: ~20 spikes/second, 150 meters/second
- 1,000,000-fold speedup physically possible
- just like a skyscraper is OOM taller than a human, and a jet plane OOM faster, there could be minds that think OOM faster
- all known AIs today are dumber than a village idiot
- shouldn’t use the human scale of intelligence to judge AIs
- shows an animation of the AI creeping along the intelligence scale
- if AI thinks 1000 times faster than human programmers, wouldn’t it improve itself quite faster than before?
- what does the global economy look like once humans come along? comparatively, it explodes overnight. AI could be the same.
- bold claim of Intelligence Explosion: minds making technology to improve minds rapidly improves to superintelligence
- contradicts both accelerationism and Strong Event horizon
- graph of the intelligence explosion goes almost vertical due to criticality of recursive self-improvement
- calculating the exact time of the Singularity is a popular pastime among accelerationists
- how long to factor a 75-digit number?
- Geordie Rose calculated that it’s better to use a 2007 algorithm with a 1977 computer than vice versa, by a factor of 10:3
- to factor that huge number, he’d rather have a modern theory and Apple II than a 70s theory and Blue Gene
- this shows that algorithms are improving
- more brute force lets you get away with a less clever design
- Moore’s Law of Mad Science: every year the IQ required to destroy the world drops by one point
- summarizes the three core theses
- the three schools are logically distinct but can support or contradict each other’s core or bold claims
- these claims are often smashed together into Singularity paste
- people coin new meanings of Singularity all the time: if you do so, for the love of cute kittens, tell us what you mean?
- question: is AI parallelizable?
- answer: I usually say “the true way of AI is as pure as moonlight reflecting off a pool of water”, computing power might not matter, but AI
- question: how does biology and other wet stuff play into the Singularity?
- answer: I tend to be sort of skeptical, first flying machine was neither an artificial or scaled-up bird
- it will be easier to make it from scratch than hack the spaghetti code, undocumented human brain
- question: what do people donating to SIAI expect?
- answer: we’re supposed to build an intelligence explosion grenade, carefully shape it to be Friendly, and then pull the pin!
- question: is there any other system that recursively self-improves
- answer: closest analogy are humans thinking about how to think, this may be the most important open problem in AI, no AIs getting mileage out of thinking about thinking
- question: what’s your thought about evolutionary computing?
- answer: a major skeptic, amazing that evolution even works at all, zero intelligence stumbling around in the dark, requires hundreds of thousands of generations of complex machinery that a human could create in an afternoon
- our thinking is vastly more efficient, no need to create millions of generations
- question: would you debate John Koza on this?
- answer: yes but we might not totally disagree
- question: can we estimate the threshold for RSI?
- answer: we probably can’t do it, just as we can’t predict math breakthroughs
- question: how much computing power necessary to make an intelligence explosion grenade?
- answer: a billion ops/sec might do it if you were clever enough with another 100 years worth of AI science
- Networks of Plausible Inference by Judea Pearl, says exactly what goes wrong if you try to use 1st-order logic to describe reality

Session I: What are the Pathways and Major Challenges?

Barney Pell, Co-founder and CEO of Powerset
Pathways to Advanced General Intelligence: Architecture, Development, and Funding
10:45AM – 12:15PM

- AGI scenario: competing for jobs
- concrete scenario: a humanoid robot that competes with High School graduates for jobs
- often get the job: general, trainable, robust, social, affordable
- he asked a bunch of people at a AI conference, broad consensus this would happen in 100 years
- what are the milestones? people said they had no idea.
- is your work going to be a critical milestone? Answer: probably not.
- little consensus or even debate on what the milestones are
- most of AI has not worked on AGI, it’s too hard, there’s a preference for specialist systems instead of general ones
- focus on incremental research and low-hanging fruit
- no roadmap that emphasizes generality, and lack of funding is incredibly daunting
- even the DoD wants turnaround in 5 years
- shows a graph with two axes: development and architecture, either similar or different than human beings
- same same: same as human architecture, same developmental process, like a human baby
- same architecture, different development: neuromorphic AI, thanks to advances in brain imaging
- different architecture, same development: might be certain types of baby-like robots that grow like humans but are not neuromorphic at all
- different architecture, different development goes off into totally new and exotic areas, with their own languages and social structures
- he agrees with Brooks that AI may have already happened, probably not, but it is possible
- may be possible to create AI without understanding how brains actually work, maybe we can leave that hard task to future scientists or to the AIs themselves
- fundamental risk of top-down is that the theory might not actually work
- he believes that a compromise between top-down and bottom-up may actually create true AI
- virtuous cycle: better AGI -> better products -> more revenue -> more funding -> better AI
- historically, this has not been the case, AI has only been valued as specialized, non-general systems
- within 10 years we’ll have cars driving us around, will lead to a major transformation in urban arrangements
- once that happens, there will be more drive towards AI
- there’s an increasing emphasis on general purpose for military robots
- elderly care robotics is another area which could drive financial resources to general AI
- virtual worlds, video games: they’re close to passing the point where the realism is not as important as the gameplay, the characters, that is, the AIs
- natural language search is a huge revolution in 5-20 years
- introduction of linguistically capable software into everyday activities
- 411 calls are being increasingly handled automatically
- progress made so far is impressive considering how little funding
- next 10 years there will be outstanding increases in funding and focus on generality
- Powerset: building a conversational interface and natural language search engine
- licensed tech from Xerox PARC, it’s been in development for decades
- they knew the conversational interface was a long-term project, although the GUI from PARC changed the world
- they were given the freedom to keep working on the problem
- in the last five years, it seems the fundamental problems have been solved
- goal is to read every sentence on the web, read out meanings, create a database and use it in amazing ways
- the idea is a better and transformative search experience
- he believes Powerset is not alone, everyone (Google, Yahoo, etc.) are all going to do this
- these systems are becoming economically central
- their knowledge will increas rapidly: engineering, platform effects, ecosystems, machine reading
- there may be a milestone where top-down systems like Powerset combined with adaptive, predictive, associative systems
- conclusions; AGI has received only limited research and computational resources to date
- these conditions are changing quickly
- multiple pathways exist
- a combination of approaches is likely
- the problems are very hard…
- but the Singularity may be nearer
- Q: are these systems going to help us learn?
- A: Yes, and we already have these systems, like Google, which has transformed education
- factual knowledge extracted from Wikipedia and organized automatically is one possible
- Ron Caplan at ACC03 said he predicted dialogue and ambiguity-savvy interfaces in 5 years, we’re on track to that
- there’s going to be a time when no one thinks huge changes are coming around the corner, but they are
- eventually people in the future will thank us for thinking about this!

Sam S. Adams, IBM Distinguished Engineer, IBM Research Division
Superstition and Forgetfulness – Two Essentials for Artificial General Intelligence

- our experience is highly tied to our temporality
- speed up a record by a couple times, and we can’t even understand it!
- the Joshua Blue Project; develop a computer system, pattern after the human mind, capable of autonomously learning to successfully function in any number of embodiments and environments
- stop at age 3, because they can already do all sorts of things most AIs have never been able to do
- not able to read or write, but able to do all sorts of cool stuff
- pass a “Toddler Turing Test”
- semantic processing, autonomous common sense knowledge acquisition
- quantitative measurement of cognitive machine capabilities
- general purpose “mind”, universal cognitive engine
- self-bootstrapping with minimal a priori knowledge
- graph: architectural clues from human development
- shows neuronal density, eye focus and tracking, neuronal plasticity, synaptic density in visual cortex, neocortex, other qualities graphed over time
- Y axis: % adult level, X axis: age in years and months
- superstition and forgetfulness: the source of all knowledge?
- everything we ever learn starts off as a first time experience, we have no idea whether or not it’s going to repeat
- “superstition’? yes, but grounded by experience
- aggressive forgetfulness is the way to get rid of superstitions that aren’t useful
- evidence that forgetfulness is happening in a continual way in the human brain
- superstition + forgetfulness = general intelligence?
- project has been in full swing since 2002
- needs better hardware to do the algorithms justice
- where’s the pathway to AI? Follow the child.

Day Two

Peter Norvig, Director of Search Quality, Google
The History and Future of Technological Change
9:00AM

- emphasizes findings that on political issues, educated laypeople can be just as effective as “expert” theorists
- shows that GDP has increased at a steady rate without large jumps due to techology
- life expectancy steadily increasing, no marked acceleration though
- field he made up: AGS: artificial general space exploration
- talks about his “former life” at NASA Ames
- they sent people to a crater in the Arctic to understand what it would like to be an astronaut
- using it to simulate Mars exploration
- another field that’s made up – AGMS – artificial general materials science
- no one is saying “we need one material that can have all these special properties”
- he talked to the Nobelist of RNA interference who said he was extremely excited about the future of AI – not biotech!
- AGC: artificial general culture
- what makes humans distinct is not our individual intelligence but our collective intelligence
- he believes the Singularity will be a period, not an event

Session IV: What Risks May We Face?

J. Storrs Hall, author of Beyond AI, 10:00 – 10:35AM
Asimov’s Laws of Robotics – Revised

- he used to make tin can robots with bicycle reflectors for eyes, wrote down Asimov’s laws on a piece of paper and put it into its head
- Asimov’s robots were not self-improving
- law #1: superintelligent AIs should understand memetic evolution
- we need to select evolutionarily stable strategies that are also beneficial
- start AIs out in an evolutionarily stable strategy attractor that took us millions of years to find
- ESC (evolutionary stable conscience)
- an AI with an ESC who “knows” that if it moves off the attractor it will just go back
- will have guidelines to extend the spirit and morality of our ethics into new situations
- law #2: robots should be open source
- we live in a world largely run by artificial information processing structures which have no conscience: corporations and governments
- law #3: a robot shall be economically sentient
- in other words, at least a part of its utility function should be a consideration of the values placed on things by others
- this turns out to be an evolutionary stable conscience
- the more transparent the government, the less likely it is to commit atrocities
- law #4: a robot shall be trustworthy, loyal, helpful, friendly, courteous, kind, obedient, cheerful, thrifty
- summary: he thinks robots should be boy scouts

Peter Thiel, venture capitalist
Financial Markets and the Singularity

- investing in general: do something fundamentally true, and that no one else is doing
- how would markets be different if the Singularity were to happen?
- in a world where there’s a possibility of things going extraordinary good or bad
- the Singularity could either cause a tremendous boom or a world where there’s nothing else to invest in
- this causes an unusual investment environment
- in a world where everything blows up, there’d be nothing left to buy or sell
- you can’t invest in the bad scenarios of the Singularity
- he thinks the volatility of markets is suggestive of accelerating change
- as we get more information, things should smooth out, but they don’t
- if you were to chart all the booms and busts in history of the modern world, you’d see the recent amplitudes are the largest
- first hyper-boom took place in Japan in the 1980s
- Japanese stock market in 1989 was worth more than all other markets combined
- people said the entire future of the world would be based in Tokyo
- there was eventually a massive bust there
- another hyper boom: the dot com era in Silicon Valley
- post-mortem: march 2000 – people were completely crazy, he could say a lot of anecdotes that support that, but…
- what if that “peak of insanity” was just a peak of sanity?
- what if they were right that the old ways of business would all stop working
- what if you had to bet on the one way that would be the way out?
- people thought it was the Internet, but it turned out some companies were totally useless
- bust in 02/03 was extraordinary
- in the last three years, there have been a series of new booms
- you have to choose – which area is going to rapidly accelerate?
- enormous boom in hedge funds, applying computer science to Wall Street
- making them allocate money perfectly, maybe Wall Street, finance will kickstart a Singularity
- Web 2.0 boom is happening, obviously
- macro thesis: even though most of these massive moves may be fake, it can’t the case that they’re all fake
- as an investor, you have to try to figure out which ones are real
- people may be betting on the Singularity, or parts of it, without even knowing it
- much of Buffett portfolio in the last decade has been about catastrophic insurance products
- different scenarios: you just get premiums, a disaster happens and it benefits the company, a disaster so big happens that millions die and the fabric of society suffers
- as an investor, it’s important to look at incipient booms
- in the next 20 years, booms and busts will get even more extreme

Charles L. Harper Jr., John Templeton Foundation, 10:45 – 11:30AM
Superintelligence, the Dilemma of Power, and the Transformation of Desire

- remarks he doesn’t know anything about AI
- first question: what does a slug know of Mozart
- we tend to think that superintelligence is just von Neumann, only smarter – continuity
- or there could be radical discontinuities, like phase changes in matter
- to a slug, all of Mozart are primitive perceptions of vibrations
- limitations are based on the slug’s ontology
- slug not well matched to understand Mozart, doesn’t have the ontology
- Gork is speglic (based on orphensic “ontology”)
- he made these words up, making the point that these words mean nothing to us but could mean a lot to superintelligence
- are we like slugs with respect to some higher complexity level?
- could major discontinuities that we know nothing about? (and can know nothing about?)
- (slugs are not) but are humans at the top?
- we tend to think that we’re on the top
- question #2: how serious is the “dilemma of power”?
- shows image Fritz Haber and his nitrogen fixation apparatus
- this is the fact that science and tech create new forms of power rapidly, but cultures don’t easily create parallel capacities of stewardship requires to utility newly created powers for benevolent use and restrain them from being used to serve malevolent ends
- does the dilemma of power matter?
- science/tech production of new powers is rapid and relatively easy
- civilizational production of new capacities of appropraite stewardship of these new expanding human powers is hard
- suggests a reasonable prospect for future megadisruptive megadisasters
- also implies a vital challenge for innovation to advance our stewardship capabilities
- Christmas Eve, 1938: the idea of fission
- the power was immediately taken over by governments
- another excellent Haber image
- reminds us that Haber invented Zyklon B
- we are creating “mixing problems” – like giving kindergartners loaded AK 47s
- could New York be nuked in the next few decades?
- does the dilemma of power matter a lot? he says probably yes
- what can be done on the stewardship side?
- question #3: how important is the “transformation of desire”
- shows images of young and old Michael Jackson
- demonstrates that the transformation of desire can be problematic
- fasting in all traditions is the direct effect to transform desire for hunger into something “higher”
- Bhuddist “superintelligence” – Four Noble Truths
- Nirvana is the extinction of desire
- humanist version: Library of Celsus in Ephesus
- Sophia (wisdom), Arete (virtue/excellence), Ennoia (thought/intelligence/intention), Episteme (knowledge)
- Christian humanism ads agapic love to the conversation
- in addition to IQ, what is it “intelligent” to advance?
- wisdom, empathy, compassion, agapic love, altruistic vision and purpose, courage, fortitude, etc.
- how does this relate to computational science?
- Martin Nowak’s agenda – develop a formal mathematical understanding of human cooperation
- could the games industry, like SimCity, have some overlap with his deep theoretical work?
- connection between games industry and learning of virtues?
- Olaf Stapledon – Starmaker
- might it be possible to develop win-win synergy between evolutionary logic of virtues and gaming?
- the success of modernity follows a productivity logic involving the division of labor
- there is some opposition to integration in universities because they are so used to these divisions
- Bacon, Boyle, Newton, Hooke, etc all pursued integrated visions
- this meeting is fascinating, specifically because people care about

Lunch: Special Presentation by X PRIZE Foundation

- they are playing the standard X-Prize promotion video (this was also showed by Diamandis at TV07)
- Genomics X-Prize: $10 million for 100 human genomes in 10 days
- Automotive X-Prize: $25 million for 100 mpg and proven market demand for their design
- “Revolution Through Competition”
- target categories that are “stuck”
- looking at educational software for Education X-Prize
- trying to develop educational software that can improve student achievement by as much as 2 sigma

Steve Jurvetson, Draper Fischer Jurvetson, 1:30 – 2:00PM
Dichotomy of Designed and Evolutionary Paths to AI Futures

- asked people whether they think AI will be designed or evolved
- 1/5 said designed, about 2/3 said evolved
- DFJ invests about $5 billion
- he talks very fast, very impressive
- nanotech futures: top-down vs. bottom-up
- progress in nanotech using bio-components
- he finds bottom-up more interesting and more near-term potential
- reminds us that life sciences have just as much interesting accelerating change and activity as IT
- shows us an exponential graph of genome base pairs sequenced
- Craig Venter’s Socrerer II Expedition (Moore Foundation program)
- there’s a renaissance in genomics and it may relate to AI
- building robust complex systems is the similarity
- evolved information systems
- biology is existence proof of something that can create products more complex than its antecedents
- emergent layerse of abstraction
- subsystem inscrutability
- no simple shortcuts across the iterations
- design: control, brittle, simple problems, subsystem clarity, artifact learning, modular reuse and portability
- evolution: out of control, robust resilient, adaptive, complex, transcendent, inscrutability, hierarchial subsumption
- implications: co-evolutionary islands, path dependence, uploads will definitely require bodies as well
- mentions the company Genomatica, knocks out a bunch of genes in the metabolic pathway of microbes to make custom-wanted chemicals
- bio-factories out of e-coli using artificial selection and selective gene repression
- “we will not engineer an AI, rather we will set up the right conditions under which an intelligence can emerge.”

Christine L. Peterson, Foresight Institute, 1:45 – 2:30PM
Preparing for Bizarreness: Open Source Physical Security

- encourages us to support life extension, SENS
- reminds us we may need to stick around if AGI takes a while
- preparing for risks: we need both info and physical tech to be secure
- three stages of physical risk
- stage 1: explosives, chemical, nuclear
- stage 2: bio
- stage 3: nano
- each one gets harder
- very scary world ahead
- current approaches to risk mitigation are too top-down and short-term
- the top-down way: centralized, mandatory, monolithic, limited in participation, secretive
- the bottom-up way: decentralized, voluntary to extent possible, experimental, collaborative, open, transparent
- general goals: make voluntary/privatize wherever possible, decentralize remaining gov’t tasks to lowest levels possible
- how would open source-style security principles apply to these sample situations?
- immediate: airline seucrity
- mid-term: sensing for bioweapons
- long-term: sensing for advanced nanotech weapons
- sousveillance: surveillance from below
- role of sims and games: from Engines of Creation, “We can turn loose a horde of…”
- role of openness, reciprocal transparency
- a lot of what bad guys can do has to do with what they can do in secret before they’re found out
- long-term security goal: automated system able to protect without being threatening
- role of alpha geeks: just like Franklin and Jefferson, when necessary they dealt with politics superbly well
- it’s like we’re living in a Heinlein novel
- “What — who, me?”
- call for action: many talented people exist here, we have to participate, and start early!
- EoC free at e-drexler.com
- Foresight Open Source Physical Security Project

James Hughes, Institute for Ethics and Emerging Technologies
Waiting for the Great Leap… Forward?, 2:30 – 3:15PM

- lefty transhumanist, wants to build a bridge between us fringe futurists and mainstream
- AGI is likely, “sentient, self-willed, greater-than-human machine minds are very likely in the coming fifty years”
- AGI Probably Very Dangerous, so steps must be taken to ensure its safety
- he agrees with Singularitarians on those two points
- AGI will be radically alien; if you can’t communicate with a gecko with 60% genetic similarity, how will we communicate with AI?
- attempt Friendly AI, but it is ultimately futile
- how to ensure that human-derived superintelligences are accountable?
- we could design a perfectly utilitarian computer
- wouldn’t necessarily want a perfect utilitarian AGI
- believes motivations can drift, be edited
- Millennialist Cognitive Biases in Singularitarianism?
- yes, apocalypse and rapture both possible
- but we shouldn’t assume either
- we have some ability to determine outcomes
- “positive assumptions that what comes out of this would be good”
- the AI that comes out of the box will just be like Jesus Christ
- have to say “I’m going to engage in these public policy actions”
- need to have serious engagement, not magical thinking
- emergent A-Life may evolve from primitive designed AI in infosphere ecosystem
- connecting S^ with cyber-security initiatives
- Storm Worm botnet acting in a defensive matter, launches DDoS against researchers
- most cybersecurity ignores AGI
- global tech regulation: techs of mass destruction require transnational regulation
- AGI Police Infrastructure: detection and counter-measures may require machine minds as well
- with AGIs that have loyalty tied to human institutions somehow?
- human intelligence augmentation is integral to keep up with AI
- perhaps all AGI should be driven by mammal-origin brains
- CogAug and uploads as safe AGI
- toaster caricature