OpenAI, Anthropic enter AI agreements with US AI Safety Institute

Two of the biggest artificial intelligence providers have signed agreements to formally collaborate with the U.S. AI Safety Institute on research, testing and evaluation of their models. 

The agreements, known as memorandums of understanding, were announced Thursday by the AI Safety Institute, which called them “first-of-their-kind” government and industry partnerships. Under those agreements, the institute, which is housed at the Commerce Department’s National Institute of Standards and Technology, will “receive access to major new models from each company prior to and following their public release” and collaborate with the companies on evaluation and risk mitigation, according to a news release.

“With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety,” Elizabeth Kelly, director of the U.S. AI Safety Institute, said in a written statement. “These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI.”

While the agreements with the AI Safety Institute are a first, the Biden administration has previously used voluntary commitments with AI companies as a mechanism for working with the booming industry. So far, 16 companies — including Anthropic and OpenAI — have agreed to those safety commitments for AI technology. The AI Safety Institute agreements, however, go deeper, allowing for the review of models and collaboration between government and companies.

That collaboration isn’t limited to the United States. According to the release, the institute will also work with the U.K. AI Safety Institute to provide them with feedback on potential safety improvements. The U.S. and U.K. institutes similarly have their own memorandum of understanding to work together on research and guidance in an effort to create a common approach to testing AI systems.  

“Our collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment,” Jack Clark, co-founder and head of policy at Anthropic, said in an emailed statement. “This strengthens our ability to identify and mitigate risks, advancing responsible AI development. We’re proud to contribute to this vital work, setting new benchmarks for safe and trustworthy AI.” 

Anthropic previously worked with the U.K. AI Safety Institute ahead of its release of Claude 3.5 Sonnet in June. According to a June blog post about the launch of the model, the U.K. AISI tested it and shared those results with the U.S. institute as a result of the collaborative agreement between the safety organizations.