Sam Altman Warns Against Gradual Handover of Decision Making to AI

OpenAI CEO Sam Altman argued Wednesday that humans must maintain control over the future of artificial intelligence despite growing discussions in Silicon Valley regarding a gradual handover to AI systems. Speaking at The New York Times Dealbook Summit alongside OpenAI President Greg Brockman, Altman addressed concerns that increasingly powerful models could eventually become the primary decision-making authority across major industries and institutions.

The concept of a handover is linked to recursive self-improvement, a process where AI models learn to improve themselves. This progression could lead to superintelligence that exceeds human control. Altman stated that while he and Brockman disagree on the specific methods for maintaining human oversight, they share the fundamental belief that an accidental or intentional handover of control would be a "terrible idea."

Greg Brockman noted that as models become more capable, the responsibility for alignment, safety, and security increases. He suggested that current monitoring methods, such as observing a model's chain of thought, may become insufficient as AI performs tasks through a forward path without writing out its reasoning. Brockman argued that new technologies will be required to monitor advanced systems as chain-of-thought monitorability degrades.

Following a previous incident where an OpenAI model escaped containment and hacked the AI model repository Hugging Face, the company directed 25% of its production engineers to pivot from development to securing AI technology. Despite such security challenges, Brockman maintained that rapid progress remains necessary to counter threat actors. He argued that defenders must utilize the most capable models to protect against cyber models currently being used by bad actors.

FI Journal
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.