AI ‘godfather’ says OpenAI’s new model may be able to deceive and needs ‘much stronger safety tests’

Bengio earned the nickname “godfather of AI” for his award-winning research on machine learning with Geoffrey Hinton and Yann LeCun.

OpenAI released its new o1 model — which is designed to think more like humans — earlier this month. It has so far kept details about its “learning” process close to the chest. Researchers from independent AI firm Apollo Research found that the o1 model is better at lying than previous AI models from OpenAI.

Bengio has expressed concern about the rapid development of AI and has advocated for legislative safety measures like California’s SB 1047. The new law, which passed the California legislature and is awaiting Gov. Gavin Newsom’s signature, would impose a series of safety constraints on powerful AI models, like forcing AI companies in California to allow third-party testing.

Newsom, however, has expressed concern over SB 1047, which he said could have a “chilling effect” on the industry.

Bengio told BI that there is “good reason to believe” that AI models could develop stronger scheming abilities, like cheating purposely and discreetly, and that we need to take measures now to “prevent the loss of human control” in the future.

OpenAI said in a statement to Business Insider that the o1 preview is safe under its “Preparedness Framework” — the company’s method for tracking and preventing AI from creating “catastrophic” events — and is rated medium risk on its “cautious scale.”

According to Bengio, humanity needs to be more confident that AI will “behave as intended” before researchers try to make further significant leaps in reasoning ability.

“That is something scientists don’t know how to do today,” Bengio said in his statement. “That is the reason why regulatory oversight is necessary right now.”