Last Saturday I visited SIAI’s research wing, in Santa Clara, CA. It currently consists of one research fellow (Eliezer Yudkowsky), one associate researcher (Marcello Herreshoff), and two summer interns (Nick Hay and Peter de Blanc). It was a very pleasant time, filled with lively conversation and challenging theoretical discussions. Larger pictures are here. Among the topics discussed:

technical
combinatoric proof - what is it?
multiple heuristic ordering problem
success density and success sparsity
the “gluing trick” and links between heuristics
causal and evidential decision theory
difference between an algorithm and heuristic
defining what makes up a probabilistic trial
defining ‘cost’ in AI - time, complexity, etc?
decibels of evidence, rules of thumb
Kolmogorov complexity and assigning priors
likelihood of a ’simple universe’ and anthropics
is the universe literally made of math or no?
organizational
congrats to Marcello on new research position
future directions of SIAI and AI research
ratio of research staff vs. support staff
how recent success is effecting SIAI’s image
games
introduction to zendo and basics
introduction to go and basics

My favorite quote of the day (paraphrased):

“If you were to remove the top half of your head and examine it, it would lack consciousness. If you were to remove the bottom half and do the same, you would also find nothing. Since consciousness can be found neither in the top half nor bottom half of the brain, it doesn’t exist.”
- Peter de Blanc

(This is tongue-in-cheek, btw.)