Report: Pre-flight screening programs need better metrics, baselines
U.S. Customs and Border Protection may have a data-driven process for flagging high-risk travelers before they board airliners, but the agency doesn’t have a data-driven procedure for measuring whether the system is working, a congressional watchdog said in a new report.
CBP’s National Targeting Center checks traveler information against government databases and lists, and uses rules-based targeting to try to flag people like “foreign fighters and potential terrorists, human traffickers, drug smugglers and otherwise inadmissible persons” before they get on a U.S.-bound flight. CBP officers at various different pre-departure programs told the GAO they work on “targets” the NTC has identified.
But the agency doesn’t have an established set of metrics and baselines to measure the performance of its several pre-flight programs, the Government Accountability Office said in a public version published Tuesday of a sensitive report issued in Sept. 2016.
CBP’s data showed that through pre-departure programs it identified and “ultimately interdicted” about 22,000 high-risk air travelers in fiscal year 2015. And in that same year the pre-departure programs helped the agency identify 1,500 previously unknown-high risk people to nominate for inclusion into the Terrorist Screening Database.
While the CBP tracks some data, the GAO recommended the CBP commissioner “develop and implement a system of performance measures and baselines to evaluate the effectiveness of CBP’s pre-departure programs and assess whether the programs are achieving their stated goals.”
The report seems to imply that some review occurs for various aspects of the programs. For example, when it comes to the rules-based targeting, officials told the GAO “CBP reviews the targeting rules on a periodic basis to assess whether the rules remain relevant in light of evolving threats” and that “CBP also conducts constant rule maintenance and development as needed in response to new or changing threats.”
But Rebecca Gambler, director of homeland security and justice issues at GAO, told FedScoop that the agency could be using data it already collects to better measure the overall performance of of its pre-departure programs.
“It certainly can be difficult to measure programs like this but what we found in the report is that they could look at some of the data that they are already collecting, and use, for example data from fiscal year 2015 to develop initial baselines and then they can measure program performance over a time relative to the baseline,” Gambler said.
CBP officials told the GAO that they collect a lot of data on their pre-departure programs, but it was collected inconsistently due to operational focus shifts and technology updates. But now the agency has “updated and uniform data collection systems that are consistent across all predeparture programs,” officials told the GAO.
“They have these different sets of data like traveler volume or the number of invalid travel documents that they seized, but just looking at you know increases and decreases doesn’t really allow them to fully evaluate the programs,” Gambler said. “And so being able to measure against baselines over time would help them kind of understand what increases or decreases in that program data mean, as an example.”
CBP is planning to expand its pre-departure programs, the GAO found in its report, and Gambler said “as they do continue to move forward with their plans it is important for them to establish these performance metrics and use them going forward.”
The Department of Homeland Security agreed with the watchdog’s recommendation, and said it plans to create a working group to develop and implement a system for measuring performance.
The GAO said that “if implemented effectively,” the agency’s plan should address the watchdog’s recommendation.