Homeland Security Investigations needs auditable AI, official says

Immigration and Customs Enforcement’s investigative arm has more guardrails on artificial intelligence than other units in the Department of Homeland Security, according to Peter Hatch, the principal assistant director for Homeland Security Investigations’ Office of Intelligence. 

HSI is responsible for conducting federal criminal investigations into drug and weapon smuggling, cyber and financial crime, human trafficking, illegal technology exports and intellectual property crime. 

Because of its mission, HSI cannot rely on technology tools with outputs that cannot be verified. It’s a sizable sticking point in the federal government’s push for broader AI adoption. While inroads have been made in terms of citations and technical methods to improve explainability, the general black box nature of large language models and agentic solutions leaves questions unanswered. 

“It is different working with investigative analysis or working intel in an investigative agency than it is if you’re working with our counterparts in [Customs and Border Protection] or Coast Guard, who have special authorities that don’t need probable cause,” Hatch said onstage during a GDIT Emerge event Thursday in Washington, D.C.

HSI’s work is subject to discovery, from emails and notes to AI inputs and outputs. Anything used to build a case will be used in court, per Hatch. When acquiring an AI tool, the investigative unit looks for whether it’s possible to audit outputs and access source documents. 

“If you use AI to support us, we’re going to ask you these questions,” Hatch said. “Every step of the way, we have to know and validate that it was correct.”

When AI is used in building a case, investigators have to document the use and publicize it in reports. 

HSI has nearly a dozen deployed AI use cases, according to analysis of DHS’s latest AI inventory. It is already using AI to summarize and categorize tips, classify media evidence and automatically redact audio and video used in investigations or legal proceedings. 

One of the most impactful use cases for AI is identity resolution, per Hatch. 

“We have so much more information, not just the forms you fill out when you travel, but also social media,” Hatch said. “We have just a tremendous amount of publicly available information … [so] it’s not a case of finding a needle in a haystack. It’s finding 100 needles in a haystack and knowing which one is the actual person you’re looking for.” 

The goal isn’t to buy more AI tools and models, Hatch said, but to use what’s already available in the right way.

The investigation unit has around five use cases in pre-deployment phases, which signal that they are in development or still being acquired. HSI is looking to use AI for lead generation, analysis of translated text, digital records review and fragmented maritime data synthesis. 

For some of the use cases that HSI is pursuing, they are characterized as “presumed high-impact but determined not high-impact.” In its justifications, the investigative unit credits the categorization to the outputs serving in support — rather than as a principal basis — for decisions, and that outputs are reviewed before any action is taken. 

Experts have said that the distinction is a “pretty high definitional bar” and have raised concerns about these use cases not needing to meet the same standards as high-impact ones. 

But no matter the use case, HSI wants to keep workers, not AI, in charge. 

“AI makes a good agent faster,” Hatch said. “It does not make a bad criminal analyst into a great criminal analyst.”

“I need help building a typology into a workflow in a way that is auditable, in a way that has humans in the loop, as I described, in a way that we can cite the source material,” Hatch added. “If you’re in that industry and you do that kind of work, that is something that we and other investigative agencies are all interested in.”