DOD to get ‘entire spectrum’ of AI testing in upcoming JAIC contract

The Department of Defense‘s Joint Artificial Intelligence Center plans to issue a multiple-award contract for testing and evaluation services to support all of the department’s AI testing needs.

The contract, expected to be awarded sometime in February, will not just be for the JAIC but rather a mechanism for all of the DOD offices working on AI to access private sector support when testing their systems.

“What we are looking for is the entire spectrum of support,” Jane Pinelis, the JAIC’s head of AI testing and evaluation, said Tuesday during an AFCEA DC event. That could mean testing an algorithm for bias or checking that it can integrate across systems, Pinelis said.

JAIC leaders have said the center should be a leader in testing, given that many of its AI systems will be applied to lethal missions. Pinelis said the JAIC has some of its own internal tools for testing models, but it needs help automating parts of the process, like updating algorithms and software interoperability. Other areas of need include human integration testing, human-machine trust and simulating real-world environments that AI could be deployed in.

The contract’s design for whole-of-DOD use comes after the JAIC recently shifted its core mission away from building products and fielding AI to be an “enabling force” in support of the many offices across the DOD working on AI. Part of that “JAIC 2.0” mindset is to ensure the services it procures from industry can also be leveraged by other parts of the DOD’s AI ecosystem.

This multiple-award contract will be one way for the JAIC to make a connection with industry that others across the department can later use, Pinelis said.

“That was a very important setup for us,” she said of the larger department’s benefit from the contract. “The JAIC is about enabling the department to achieve scale, and the only way we can do that is if we come up with these contract vehicles that others can use as well.”