Godfather of AI: How To Make Safe Superintelligent AI – Yoshua Bengio
Tech1 of 34
Bengio: Honesty as the Foundation for AI Safety
Yoshua Bengio proposes that baking honesty into AI systems is the key to achieving safety, reducing the problem to training a system to be honest through modified training objectives and data processing. This 'Scientist AI' is envisioned as a predictor, not an agent, with inherent honesty guarantees.
Transforming Data for Scientist AI
The training data remains largely the same, but its presentation is syntactically altered. Statements are tagged as 'communication acts' or 'factual/hypothesis' syntax. This allows the model to learn the joint distribution of variables, including latent ones, to explain observed data and infer underlying truths.
Scientist AI as an Oracle and Agent
Initially, the honest predictor can serve as a 'guardrail' for existing AI agents. However, Bengio's research extends this to an 'agentic Scientist AI' by using the predictor's probabilities to construct a policy, aiming to retain safety guarantees while enabling goal-directed behavior.
