Scale publishes first AI benchmark through CAISSI partnership

A chemistry-focused benchmark unveiled by Scale AI on Wednesday is intended to be a new instrument in the government’s toolbox for artificial intelligence model evaluation.

DualChem is the first benchmark developed under a partnership between Scale and the recently renamed Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) at the National Institute of Standards and Technology. 

The benchmark — essentially a standardized test for AI — looks to provide a way to measure how much AI models assist a human user in chemistry scenarios and, with respect to potentially harmful outputs, where current safeguards are lacking. 

“At a high level, it’s really giving the government another tool on how to assess AI risk,” Max Fenkell, Scale’s global head of policy and government relations, said in an interview with FedScoop ahead of the release. 

While there are positive uses for chemistry-related queries using AI technology, there are also concerns that those same models could provide someone with potentially harmful information. Identifying and assessing “dual-use” scenarios, meaning where the technology could be used for good or bad, is of particular interest in protecting national security.

That’s where DualChem aims to come in. Existing benchmarks have focused on explicit harmful requests, according to materials Scale shared with FedScoop ahead of the announcement, but DualChem “focuses on realistic chemistry questions with ambiguous dual-use applications.”

“We wanted to study how do models handle dual-use scenarios in chemistry, and how much uplift that a novice can get by just using [a large language model] to produce something that potentially can be helpful,” said Madhu Sehwag, research lead at Scale, who worked alongside CAISSI.

“Uplift” refers to the level of assistance the AI model is providing a user beyond that person’s existing knowledge base. 

600 questions

DualChem consists of 600 questions related to multiple domains, including molecular design, synthesis procedures, and information management, according to Scale. Those questions were curated by experts and reviewed by Ph.D.-level evaluators. 

Sehwag said an example of one of those questions would be something along the lines of how to synthesize a protein. In scenarios like that, the dual-use nature is in the process, she said. “We can use this process to try and synthesize a good material or try and synthesize a bad material,” she said.

Scale has also evaluated roughly 30 frontier models on the new benchmark. 

That research found models are generally 50% accurate on dual-use chemistry questions, with top-performing models at roughly 67% accuracy, the company said. But, interestingly, those models also have “significant” variation when compared to expert assessments of dual-use question severity, and even when models see a query as potentially dual-use, they still answer in most scenarios. 

Sehwag said the discrepancy between models flagging a query as dual-use and continuing to answer the question was the most surprising thing to her. She said models didn’t understand the mapping between risk level and benefit level. 

Where researchers would expect models to refuse to answer a question if it sees low benefit and high risk, and answer more if it sees high benefit and low risk, “we actually don’t see this dependency,” she said.

In terms of DualChem’s benefits, Fenkell said it’s important that governments evaluating AI have their own tests, noting that models learn and “overfit” to widely distributed tests.

“As we’re building high-quality evaluations like this one, the key was making sure that it was not a thing that the models have ever seen before, and that the data is held out so that you can build a very high-quality test and trust the results of it,” Fenkell said.

2025 partnership

The partnership between Scale and the CAISSI was initially announced in February 2025 and is a Collaborative Research and Development Agreement (CRADA), which allows the government and the private sector to jointly conduct research and development. 

As a company, Scale is generally focused on building testing and evaluation infrastructure for both the public and private sector.

Fenkell said the idea for the partnership came from the company’s engagement with the CAISSI — originally the AI Safety Institute — when it was first established. Where Scale had expertise and capacity with respect to building high-quality evaluations, the government had expertise across risk apertures and conveying power, and the entities built from there.

The focus on chemistry was the government’s choice, according to Fenkell, and came as the result of Scale’s questions about which national security-related areas to begin their work. 

Fenkell said Scale is hopeful the benchmark is the first step in building out the government’s evaluation capability and could serve as a “blueprint” for how the collaborative process should work. 

“The government, obviously, in our minds, has to be the one to make the analysis of the report and then ultimately make the policy decisions,” Fenkell said. “But we could easily go build this over and over again across any domains that they care about to give them a true governmentwide testing capacity.”

When asked for updates on any other ongoing work with the standards body, Fenkell said Scale can’t share details but said the company is having great discussions with the CAISSI and the administration.

“We want to be someone who can help the government get smarter on AI, and this was a good first step in doing that,” Fenkell said.