Schumer, Markey call on agencies to set up civil rights offices to mitigate AI bias

All federal agencies using artificial intelligence for “consequential decisions” should be required to have a civil rights office to prevent things like algorithmic discrimination, Senate Majority Leader Chuck Schumer, D-N.Y., and Sen. Ed Markey, D-Mass., said Monday in a letter to the White House.

In the letter to Office of Management and Budget Director Shalanda Young, the lawmakers noted that President Joe Biden had taken steps when it comes to addressing AI bias — including his Blueprint for an AI Bill of Rights and October 2023 executive order on the technology — but said that more oversight is needed. 

“Without new protections, today’s supercharged, AI-powered algorithms risk reinforcing and magnifying the discrimination that marginalized communities already experience due to poorly-trained and -tested algorithms,” the Democratic senators said in the letter first published by Axios.

The letter comes as the executive branch and Congress work to address the potential harms and benefits of AI as it booms in popularity. The Biden administration, for example, has made applying special risk management practices to use cases that are rights- and safety-impacting a central part of its governance strategy. 

While Schumer and Markey commended that work, they argued that civil rights offices staffed with technologists and algorithmic discrimination experts are also needed to prevent harms, and they urged OMB to require such an office in all agencies using AI for “consequential decisions.” 

A White House spokesperson confirmed OMB had received the letter but didn’t provide additional comment.

According to the senators, examples of consequential decisions include usage during the job application process, hospital treatment, educational institution admission, or mortgage qualification. The stakes in those instances are “especially high,” the lawmakers said. 

“Over the past decade, biased algorithms have increasingly been used to make or influence decisions, imposing real harm on Black, Brown, immigrant, and other marginalized communities,” they wrote.

Once established, senators said the civil rights offices should work together on risk mitigation and information sharing, and consistent with the executive order on AI, the White House should “ensure that the U.S. Department of Justice’s Civil Rights Division is effectively coordinating civil rights enforcement on AI among federal agencies,” they said.

The senators also asked OMB to consider directing “key federal agencies” to report on the state of efforts to mitigate civil rights harms in AI on an annual basis.

“By establishing civil rights offices at every federal agency with the proper resources and expertise, the Biden administration can ensure that identifying and mitigating algorithmic discrimination is a priority across the government,” they wrote.

They also recognized the Biden administration’s requirement that agencies have a chief AI officer, calling it “a good start,” but they argued that “OMB should also ensure that the CAIOs have the resources and expertise to address civil rights harm created by AI-driven algorithms.”

Beyond the civil rights offices, the senators asked the White House to “encourage agencies,” where possible, to take additional steps to address algorithmic discrimination. 

Those steps included requiring entities that receive federal funds and use, develop, procure or fund AI to complete certain bias assessments and requiring those same entities to allow people to opt out of AI-powered algorithm usage. Other steps included funding the development of resources for auditing bias in algorithms and developing guidance on best practices for mitigating bias in the creation of AI algorithms.

Grant Fergusson, counsel with the Electronic Privacy Information Center focused on the government’s use and procurement of AI, told FedScoop his organization was pleased with the letter, since much of the federal government’s work has focused on what the agencies are doing themselves and has been limited in terms of harms it prioritizes mitigating.

Fergusson said what the letter is trying to achieve is twofold: The first is calling on agencies to bolster staff and expertise in civil rights offices to address AI bias, and the second is extending obligations to mitigate bias and discrimination to federal fund recipients and contractors.

“It’s extending what has currently been a really limited federal AI oversight regime — one that has only been used to dictate what federal agencies themselves are doing to the broader landscape of private contractors and vendors and AI developers,” Fergusson said.

This story was updated Sept. 16, 2024, with a response from the White House and comments from EPIC’s Fergusson.