Steve Omohundro: Building self-aware AI systems

Steve Omohundro is president of Self-Aware Systems, which has a goal of developing an AI systems that understand their own behavior and work to improve themselves. He will be speaking at the Singularity Summit 2007 at the Palace of Fine Arts in San Francisco on September 8-9.

In this podcast interview, Omohundro explained his approach to developing artificial intelligence (AI) machines. He is applying machine learning and theorem proving to the task, building systems with models of their own behavior. "With a detailed model, it will be able to make changes, and by learning from its own experiences a system can develop a richer and more adaptive model than something that a person can put in," he said.

He gave an example of Microsoft Windows crashing--the Windows system doesn't know it crashed or why it crashed and a human engineer has to fix it. A self-aware system would be able to fix its own code and learn from it. It turns out that machines will be far better than humans at writing software.

The model will have constraints from the outside and an initial set of goals matched with the deepest human values so that as the system evolves and improves itself, a positive outcome, in human terms, will be produced, Omohundro said.

The alternative is that AIs pursue goals that aren't in the best interest of humanity. "We really have to envision what kind of world we want to live in, and only when we have a clear vision about where we want to go can we ensure that the technology we are building actually supports and creates that," he said. "This is a turning point in history where we have a chance to create something entirely new."

The Singularity Summit 2007 will address the risks and benefits of advanced AI and how we should prepare for Singularity, the creation of smarter-than-human intelligence, driven by technology advahttp://blogs.zdnet.com/BTL/?p=6051nces such as AI, direct brain-computer interfaces, biological augmentation of the brain, genetic engineering and ultra-high resolution scans of the brain followed by computer emulation.

See also: Barney Pell: Pathways to artificial intelligence

Steve Jurvetson: AI, nanotech and the future of the human species

Can ‘friendly’ AI save humans from irrelevance or extinction?