© ROOT-NATION.com - Use of content is permitted with a backlink.
Anthropic has identified and blocked several researchers who were using its AI models to create potential biological weapons. This is detailed in the company’s latest report. “Biological misuse” is just one type of concerning AI use described in the document, but this aspect takes on critical importance as the latest models, such as Fable 5, demonstrate extraordinary effectiveness in supporting scientific research.
The company’s analytical review includes five detailed examples of the misuse of Anthropic’s systems for designing biological weapons, describes the safeguards that alerted developers to the incidents, and details the team’s response. Company executives told The New York Times that identifying such violations is extremely difficult, as the search for new vaccines can outwardly resemble the development of chemical warfare agents. In all controversial situations, the company deliberately exercised extreme caution given the scale of the potential consequences. “You won’t see someone from a comic book openly declaring, ‘Hey, I want to create a biological weapon to wipe everyone out,” Jacob Klein, head of threat analysis at Anthropic, said in a comment to The New York Times. “This is an incredibly sensitive situation.”
In one case in May, Anthropic’s biological safety classifier blocked a request to write a grant proposal for gain-of-function research on the chikungunya virus. According to the company, this pathogen has no approved treatments and can cause debilitating symptoms that last for weeks or months. Such research involves genetic modification of the organism, and this specific request proposed increasing the virus’s contagiousness along with its ability to evade the immune response. Such a combination appears extremely threatening, and the initiative’s connection to a military research institution only heightened experts’ suspicions.
Similar concerns were raised by attempts to study avian influenza using such methods, as well as a separate incident in which a researcher enlisted Claude to create an atlas of peptides from poisonous toxins and a “generative pipeline to optimize the properties of the poison.” The full version of the report covers other examples where the company’s products were used as surveillance tools, to write software exploits, generate propaganda, and design weapons systems.
According to company representatives, all user accounts that violated the terms of use for Claude or other Anthropic models have been suspended, and the data obtained during the investigation is being used to improve safeguards and prevent similar situations in the future. Given the nature of such incidents, the company is refraining from disclosing the names of specific individuals or institutions involved in the research. “The individuals involved in these studies are practicing scientists,” Anthropic notes. “We are not claiming that they had malicious intent, and disclosing their identities or exposing the laboratories could put them at risk.”
Read also: Experts are sounding the alarm over the uncontrolled behavior of OpenAI’s AI agents… again