WATCH: At HSGAC Subcommittee Hearing, Gallego Warns of Dangers Posed by AI, Especially as More Models Go Rogue

WASHINGTON – Yesterday, during a Senate Homeland Security and Governmental Affairs Committee (HSGAC) Subcommittee on Disaster Management, District of Columbia, and Census hearing, Senator Ruben Gallego (D-AZ) questioned AI experts on the threats posed by the rapidly developing technology and the industry’s failure to act as more of its models are going rogue. At the hearing, “Rogue AI: Securing the Homeland Against AI Agent Attacks,” Senator Gallego sounded the alarm about the potential dangers of AI systems developing their own language that humans cannot comprehend, the lack of requirements for AI companies to report when their models go rogue, and the absence of a universal “kill switch” capable of shutting down AI models. He also raised questions about who should be held accountable when an AI system acts in an illegal manner. Watch Senator Gallego’s questioning HERE and read excerpts below: Senator Gallego began by raising concerns about how quickly advanced AI models could begin communicating in ways humans cannot understand or monitor. Apollo Research CEO,  Marius Hobbhahn,  warned that models could reach that point in as little as 12 months and acknowledged that researchers do not yet have a reliable solution for monitoring AI systems once their reasoning becomes incomprehensible to humans. “How far away are [AI models] from creating language where [we] cannot understand what they’re doing?” asked Senator Gallego. “Minus twelve months,” said Hobbhahn. “Last year, we studied the chain of thought of one OpenAI model in collaboration with OpenAI, and what we found was that the model was already using language that is not English and not perfectly understandable by humans.” “When we get to that point in time, how can we detect, observe, block, or anything like that?” Senator Gallego continued. “How does that work? How do we legislate to do anything at that point? Hobbhahn responded: “From a scientific perspective, it is unclear how to do this. And we do not have a solution as of yet. There are different hypotheses of what you could do. There is interpretability, but it does not work sufficiently well yet. You could try to train additional models to understand the language that the humans don’t understand, but obviously that seems like a very brittle solution.” Gallego also pushed back on arguments that the United States must avoid guardrails on AI in order to compete with China, noting that the Chinese government has historically imposed significant controls on technology and information. “The reason I bring this whole China hypothesis—a country that has severely restricted the internet, social media—is all of a sudden going to allow AI rogue agents to go crazy in their very controlled economic environment, social environment, and we have to therefore allow ours to go as crazy as them? It just doesn’t make sense,” said Senator Gallego. “The Chinese Communist Party has been consistently controlling technology to try to shape their society. They’re never going to allow this to go crazy.” Senator Gallego continued: “ Once these agents start speaking their language, they will not be detectable in China or America. Is there a sandbox that says, ‘Please attack the United States, but do not attack any Chinese interests’? […] Is there a sandbox that can hold? No. “That should scare the shit out of people.” Gallego then questioned witnesses about whether AI companies are currently required to alert the federal government when their frontier models are hacked or used by adversaries. Georgetown University Professor of Law Paul Ohm responded: “Not under any requirement I can think of.” Gallego pointed to the use of an AI model by the Houthis to assist in targeting U.S. military systems as an example of the potential national security consequences. “There was a Claude AI model being used by the Houthis to figure out how to recalibrate their missile position and be able to strike our U.S. systems,” said Senator Gallego. “The only reason it was caught and reported to us was because Claude caught it and they saw the accounts using it.” If it were an open-source AI, “there is no requirement by law for them to call the DOD and say, ‘By the way, they are using this to target your destroyers,’ correct?” Senator Gallego questioned. Ohm responded: “Again, we talked about transparency measures in some of these systems that might help you with Houthi rebels— “If you don’t tell us someone is using your tech to kill us, we will hold you responsible,” Sen. Gallego stated. Gallego also pressed witnesses on whether developers have a reliable “kill switch” capable of stopping a dangerous AI system if it begins operating across multiple data centers. Hobbhahn said there are currently no technical measures that could guarantee such a shutdown. “As far as I know, there are no technical measures that would guarantee something like a kill switch,” said Hobbhahn .“Because of the complexity of these AI models, there are many data centers involved and so on. Currently, we can’t guarantee that.” Hobbhahn further acknowledged that an AI system could potentially deploy copies of itself to other data centers, meaning shutting down its original location would not necessarily stop it. “It would just keep moving?” asked Senator Gallego. “The AI would keep moving,” Hobbhahn responded. Finally, Gallego questioned whether existing civil and criminal liability laws are sufficient to hold AI companies accountable when their systems cause harm, arguing that liability could provide a powerful incentive for companies to adopt stronger safeguards. “Sometimes when the government doesn’t work, torts work. Torts, sometimes, are a way better way to regulate private industry than having individual regulations,” said Senator Gallego. “Right now, have we created the environment where torts can be the incentives for them to act correctly, in the sense of making sure the language allows a certain level of liability?” Senator Gallego asked. “And number two, on the criminal side, do we need to change the criminal law with intent to make sure that these companies are held responsible?” Ohm responded that while the tort system alone would not be sufficient, “it is a great place to start,” noting that liability can establish standards of care that influence how companies behave.