If Artificial General Intelligence has an okay outcome, what will be the reason? — AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.

On Manifold Markets, "If Artificial General Intelligence has an okay outcome, what will be the reason? — AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled." has a probability of 1.0%.