Improving federal solicitations’ ‘scary low’ Section 508 accessibility compliance
The General Services Administration plans to begin final testing of its artificial intelligence-enabled Solicitation Review Tool in July before launching in September, said Marina Fox, DotGov domain services manager.
Every federal agency has its own process for writing solicitations, and oftentimes Section 508 of the Rehabilitation Act is an afterthought, Fox said at the Government Analytics Breakfast Forum on Thursday.
Section 508 requires electronic and information technology procured by the federal government be accessible to the approximately 60 million people with disabilities in the U.S. When the rule is overlooked in solicitations, products wind up being built that aren’t accessible to, say, a person with blindness.
“The best case, it gets caught, it gets corrected,” Fox said. “The worst-case scenario, an advocacy group finds it, files a complaint, there is a class-action suit, Department of Justice has to go defend the agency. It’s happened to GSA before.”
In 2012, DOJ had to defend federal agencies in about 140 Section 508 lawsuits, Fox said, and about 500 new solicitations for information and communications technology are posted to FedBizOpps.gov daily.
At the suggestion of a data scientist on staff, GSA started work on the open-source AI tool in 2016. Using 10 years of manual solicitation reviews, GSA took data from 4,000 of them and built a predictive model with Python—a “training data set,” Fox said.
AI classifies and predicts whether a solicitation is compliant or not based on control vocabulary.
“This is not just a word search,” Fox said. “We are using natural language processing, but this is algorithm based.”
At one point the project was defunded and had to be archived in GitHub, Fox said, but eventually, SRT evolved into an enterprisewide data application that will scan solicitations every 24 hours and flag suspected noncompliant ones for review by contract officers.
Last year, GSA began SRT’s pilot phase and determined the app needed to live in cloud.gov to scale. In scanning a sampling of solicitations, only 10 percent held up under review—meaning nine out of every 10 solicitations contained insufficient Section 508 language.
“The compliance rate was extremely low—scary low—to the point where some said don’t release the statistic,” Fox said. “But that’s the reality.”
Developers built a feedback loop into SRT’s first release where, even though the AI predicts noncompliance with 90 percent accuracy, contract officers who review flagged solicitations are afterward asked if the machine made the right call. That way the model is continuously retrained, Fox said.
GSA has already been contacted by several teams, including FedRAMP, about using SRT, Fox said.
At least four weeks of “pretty extensive” user acceptance testing are scheduled to start in July, Fox said, and GSA is considering a soft launch within 10 agencies that tend to be more compliant before the end of the fiscal year.
With solicitations scanned daily, GSA will be watching how long it takes agencies to fix the noncompliant ones.
SRT version 2.0 will make use of AI neural networks to offer suggestions for fixing solicitations in the tool itself, Fox said.
GSA is also considering running scans before solicitations are even posted, she added.
