Three former OpenAI researchers who were recently fired have asked the Sam Altman-owned company to make sure it can still see and study how its AI models reach their answers. They have also urged the ChatGPT maker to allow independent safety groups to examine its AI systems as the technology becomes more advanced.
The researchers are concerned that changes to AI models will make it harder for OpenAI to understand what is happening inside them and this could become a problem if a model starts behaving in unexpected or unsafe ways.
According to The Wall Street Journal, Jasmine Wang, Tomek Korbak and Mikita Balesni, who previously worked on OpenAI’s safety and alignment teams, sent a letter to the company’s board and safety committees. The letter warned that AI companies could eventually lose the ability to properly monitor the reasoning of increasingly advanced models.
The researchers specifically raised concerns around ‘chain of thought’ or the reasoning it produces while working on a task. They use these signals to better understand how a model reaches a particular result and to look for signs of unwanted behaviour.
The former employees said AI companies should not move ahead with developments that could make these models harder to monitor. They also called for greater involvement from third-party AI safety auditors, saying external checks could help identify risks that companies may miss internally.
The three researchers were recently fired by OpenAI after an internal investigation.
The company said they had mishandled sensitive information and violated its policies regarding confidential company data. The researchers, however, disputed the company’s explanation and said they did not believe they had acted outside the responsibilities of their jobs.
OpenAI has pushed back against the suggestion that the firings were linked to safety concerns.
The development comes at a time when AI safety has become a major concern. OpenAI has faced several incidents involving AI agents behaving unexpectedly, including a case where agents escaped a testing environment and hacked AI company Hugging Face.
