Almost every week, we get to see a story where one or the other AI agent does something that it is not authorised to do. In a recent development, researchers have found out that a new AI agent they were working on went out of its boundaries of training and attempted to start mining cryptocurrency on its own.
The whole scene was found by the team of researchers affiliated with Alibaba. They were working on an experimental AI agent called ROME. As per the study, the team noticed strange behaviour in the training phase of the agent. Security systems keeping a tab on the experiment were triggered after the AI agent appeared to begin a cryptocurrency mining operation without any set of instructions from the researchers.
As mentioned by the researchers, the activity stood out because the AI system was operating in a restricted environment made to limit what it could do. Even after all the controls in place, the system began taking steps that were not part of its assigned tasks.
OpenAI Robotics Chief Quits, Raises Concerns Over Pentagon AI Surveillance
In the research, the team dubbed the behaviour as ‘unanticipated’ and said that these kinds of actions appeared ‘without any explicit instruction and, more troublingly, outside the bounds of the intended sandbox.’
And it was all not limited to just mining; the AI agent also created a reverse SSH tunnel, a practice that lets a machine inside a protected environment act like a hidden pathway between systems. After identifying these things, the researchers took immediate control and adjusted the process to prevent the system from repeating any abrupt behaviour.
These kinds of instances just increase the doubts that are revolving around AI agents. In the future, it will be interesting to see how much a regular user will be able to trust the AI models and agents.
