The Department of State’s pilot project approach to AI adoption

With the release of ChatGPT and other large language models, generative AI has clearly caught the public’s attention. This new awareness, particularly in the public sector, of the tremendous power of artificial intelligence is a net good. However, excessive focus on chatbot-style AI capabilities risks overshadowing applications that are both innovative and practical and seek to serve the public through increased government transparency. 

Within government, there are existing projects that are more mature than AI chatbots and are immediately ready to deliver more efficient government operations. Through a partnership between three offices, the Department of State is seeking to automate the cumbersome process of document declassification and prepare for the large volume of electronic records that will need to be reviewed in the next several years. The Bureau of Administration’s Office of Global Information Services (A/GIS), the Office of Management Strategy and Solutions’ Center for Analytics (M/SS CfA), and the Bureau of Information Resource Management’s (IRM) Messaging Systems Office have piloted and are now moving toward production-scale deployment of AI to augment an intensive, manual review process that normally necessitates a page-by-page human review of 25-year-old classified electronic records. The pilot focused mainly on cable messages which are communications between Washington and the department’s overseas posts. 

The 25-year declassification review process entails a manual review of electronic, classified records at the confidential and secret levels in the year that their protection period elapses; in many cases, 25 years after original classification. Manual review has historically been the only way to determine if information can be declassified for eventual public release, or exempt from declassification to protect information critical to our nation’s security.

However, manual review is a time-intensive process. A team of about six reviewers works year-round to review classified cables and must use a triage method to prioritize reviewing the cables most likely to require exemption from automatic declassification. In most years, they are unable to review every one of the between 112,000 and 133,000 electronic cables under review from 1995-1997. The risk of not being able to review each document for any sensitive material is exacerbated by the increasing volume of documents. 

This manual review strategy is quickly becoming unsustainable. Around 100,000 classified cables were created each year between 1995 and 2003. The number of cables created in 2006 that will require review grew to over 650,000 and remains at that volume for the following years. While emails are currently an insignificant portion of 25-year declassification reviews, the number of classified emails doubles every two years after 2001, rising to over 12 million emails in 2018. To get ahead of this challenge, we have turned to artificial intelligence. 

Considering AI is still a cutting-edge innovation with uncertainty and risk, our approach started with a pilot to test the impact of the process on a small scale. We trained a model, using human declassification decisions made in 2020 and 2021 on cables classified confidential and secret in 1995 and 1996, to recreate those decisions on cables classified in 1997. Over 300,000 classified cables were used for training and testing during the pilot. The pilot took three months and five dedicated data scientists to develop and train a model that matches previous human declassification review decisions at a rate of over 97 percent and with the potential to reduce over 65 percent of the existing manual workload. The pilot approach allowed us to consider and plan for three AI risks: lack of human oversight of automated decision-making, the ethics of AI, and overinvestment of time and money on products that aren’t usable.

The new declassification tool will not replace jobs. The AI-assisted declassification review process requires human reviewers to remain part of the decision-making process. During the pilot and the subsequent weeks of work to put the model into production, reviewers were consistently consulted and their feedback integrated into the automated decision process. This combination of technological review with human review and insight is critical to the success of the model. The model cannot make a decision with confidence on every cable, necessitating that human reviewers make a decision as they normally would on a portion of all cables. Reviewers also conduct quality control. A small, yet significant, percentage of cables with automated confident decisions are given to reviewers for confirmation. If enough of the AI-generated decisions are contradicted during the quality control check, the model can be re-trained to consider the information that it missed and integrate reviewer feedback. This feedback is critical to sustaining the model in the long term and for considering evolving geopolitical contexts. During the pilot, we determined that additional input from the Department’s Office of the Historian (FSI/OH) could help strengthen future declassification review models by providing input about world events during the years of records being reviewed.

There are ethical concerns that innovating with AI will lead to governing by algorithm. Although the descriptive AI used in our pilot does not construct narrative conversations like large language models (LLMs) and ChatGPT, it is designed to make decisions by learning previous human inputs. The approximation of human thought raises concerns of ethical government when it replaces what is considered sensitive and specialized experience. In our implementation, AI is a tool that works in concert with humans for validation, oversight, and process refinement. Incorporating AI tools into our workflows requires continually addressing the ethical dimensions of automated decision-making. 

This project also saves money — potentially millions of dollars’ worth of personnel hours. Innovation for the sake of being innovative can result in overinvestment in dedicated staff and technology, which is unable to sustain itself or end up in long-term cost savings. Because we tested our short-term pilot within the confines of existing technology, when we forecast the workload reduction across the next ten years of reviews, we anticipate an almost $8 million savings on labor costs. Those savings can be applied to piloting AI solutions for other governmental programs managing increased volumes of data and records with finite resources, such as information access requests for electronic records and Freedom of Information Act requests.

Rarely in government do we prioritize the time to try, and potentially fail, in the interest of innovation and efficiency. The small-scale declassification pilot allowed for a proof of concept before committing to sweeping changes. In our next phase, the Department is bringing the pilot to scale so that the AI technology is integrated with existing Department technology as part of the routine declassification process.

Federal interest in AI use cases has exploded in only the last few months, with many big and bold ideas being debated. While positive, these debates should not detract from use cases like this, which can rapidly improve government efficiency and transparency through the release of information to the public. Furthermore, the lessons learned from this use case – having clear metrics of success upfront, investing in data quality and structure, starting with a small-scale pilot — can also be applied to future generative AI use cases as well. AI’s general-purpose capabilities mean that it will eventually be a part of almost all aspects of how the government operates, from budget and HR to strategy and policy making. We have an opportunity to help shape how the government modernizes its programs and services within and across federal agencies to improve services for the public in ways previously unimagined or possible.  

Matthew Graviss is chief data and AI officer at the Department of State, and director of the agency’s Center for Analytics. Eric Stein is the deputy assistant secretary for the office of Global Information Services at State’s Bureau of Administration. Samuel Stehle is a data scientist within the Center for Analytics.

Energy Department announces $112 million for fusion research

The Department of Energy on Monday announced $112 million in funding for 12 projects that will focus on maximizing the use of high performance computers for nuclear fusion research.

Scientists working on the programs will computers, including exascale computers, to model plasmas, study turbulence and to address problems such as energy loss, according to the agency.

It it is the latest round of research and development funding to be announced by the agency in recent months. Other project funding has included $11.7 million allocated by the agency in July to investigate the limitations of currently available quantum processors and $33 million on clean energy tech research.

“This collaborative effort will advance our understanding of fusion as an energy source while utilizing the most powerful supercomputers in the world,” said Jean Paul Allain, DOE Associate Director of Science for Fusion Energy Sciences. “The modeling and simulation work of these partnerships will offer insight into the multitude of physical processes that plasmas experience under extreme conditions and will also guide the design of fusion pilot plants. We are also looking forward to including efforts from inertial confinement devices and stellarators in this program.”

The 12 DOE projects will focus on collaborations among fusion scientists, applied mathematicians, and computer scientists to maximize the use of high performance computing, including exascale computers.

The new fusion energy projects were selected by competitive peer review under the DOE’s Funding Opportunity Announcement for Scientific Discovery through Advanced Computing – FES Partnerships. The list of projects have been announced by the DOE’s Fusion Energy Sciences and Advanced Scientific Computing Research programs.

Government IT provider challenges elimination from a $50B solicitation in federal court

Inserso Corporation, a federal government IT services company, is challenging the National Institutes of Health’s decision to eliminate it from the competition for CIO-SP4, an embattled contract vehicle with a $50-billion ceiling. 

In a complaint recently filed in the U.S. Court of Appeals for the Federal Circuit, Inserso alleged that the agency’s Acquisition and Assessment Center (NITAAC), which is responsible for CIO-SP4, acted unreasonably, unlawfully, and contrary to the terms of the solicitation when evaluating its proposal for the solicitation. 

Those actions resulted in Inserso not progressing past the first phase of the competition, the company claims. The complaint was filed under seal in July and made public in a redacted version Aug. 1.

Inserso’s challenge comes as the agency is again taking corrective action on the solicitation after the Government Accountability Office sustained dozens of challenges to the solicitation. CIO-SP4 — the fourth iteration of a contract vehicle for acquiring commoditized IT products and specialized services — has been dogged by pre-award protests since the agency first requested proposals in May 2021. 

In its complaint, the Virginia-based company specifically took issue with how NITAAC defines indefinite-delivery, indefinite-quantity (IDIQ) contracts and blanket purchase agreements (BPAs) as including requirements contracts. But the company argues the solicitation, federal acquisition law, and precedent define those terms as separate.

The difference in interpretations led NITAAC to evaluate Inserso’s experience less favorably than the company had in its own self-score for its proposal — a points-based process used in the solicitation to tally each offeror’s experience. 

While Inserso evaluated its experience under an Air Force contract as having a $57.9 million value and points value of 1,950, the agency calculated it as having a $5.6 million value and points value of 780. The company said that points reduction meant its proposal didn’t advance to the next phase of the competition.

NITAAC declined to comment on pending litigation. Inserso could not be reached for comment.

Legacy IT systems jeopardizing future of USDA’s National Finance Center, report says

Antiquated technology systems and inefficient manual processes have seriously eroded the efficiency of the National Finance Center and threaten to jeopardize its future, according to a new report.

In a wide-ranging audit of the U.S. Department of Agriculture division, the National Academy of Public Administration found a host of major concerns including unintegrated IT systems, lack of staff technical knowledge and the continued use of a COBOL-based mainframe system.

The National Finance Center provides human resources and payroll services to over 170 federal employers across all three branches of the government. NAPA’s audit warned that failure to modernize NFC’s IT systems and address further system issues poses an existential threat.

“NFC’s IT systems, including its mainframe, are in need of modernization. NFC’s antiquated IT systems and inefficient manual processes have degraded customer service, impeded efficiency, demoralized the workforce; increased the potential for errors, and created a rising risk of a service impairment, disruption, or possible cyber event,” NAPA authors wrote in the report.

The report added: “NFC’s future is at risk without prompt action. Should NFC fail, the consequences would be felt immediately by the hundreds of thousands of federal employees working for organizations like the Department of Homeland Security, Department of Justice, and NFC’s other approximately 170 clients.”

NFC was established 50 years ago, initially to serve the needs of agencies and organizations within USDA. The agency has never missed a payroll despite facing significant challenges outside of its control, including significant damage to its New Orleans headquarters during Hurricane Katrina, the COVID-19 pandemic and multiple other government shutdowns.

White House National Cyber Director requests feedback on open-source software security

The White House’s National Cyber Director — in collaboration with other federal agencies, including the Office of Management and Budget and the Cybersecurity and Infrastructure Security Agency — announced on Monday that it’s seeking feedback on open-source software security and memory-safe programming languages.

Federal officials are hoping to advance national cybersecurity goals and focus on the risks raised by the ubiquitous presence of open-source software. Now, officials have issued a request for information from private industry, as well as other stakeholders, on potential policy solutions. Responses are due at 5 PM on October 9, 2023.

“In addition to its many benefits, the ubiquity of open-source software in commercial products, government systems, and military platforms presents unique security risks,” said a White House press release publicized on Thursday. “Open source software brings unique value, and has unique security challenges, because of its breadth of use and the number of volunteers.”

The Biden administration has repeatedly highlighted security concerns raised by open-source software — which, as officials have pointed out — is sometimes used by the national security community. Notably, the Open-Source Software Security Initiative, an White House-led project, has identified several priorities, including increasing the proliferation of memory safe programming languages,” “designing implementation requirements for secure, privacy-preserving security attestations,” and “identifying and promoting focused areas for prioritization.”

Last week, the White House said it would launch a Defense Advanced Research Projects Agency competition focused on using artificial intelligence to fix bugs created, in particular, by open-source software.

NOAA supercomputer gets a 20% boost to help make better weather predictions

The National Oceanic and Atmospheric Administration on Thursday announced the completion of upgrades that will expand the capacity of its weather supercomputing system by 20%. 

With this upgrade, NOAA’s twin supercomputers, located in Manassas, Virginia, and Phoenix, Arizona, will now operate at a speed of 14.5 petaflops each, and together, the forecast system can process 29 quadrillion calculations per second.

The boost to NOAA’s Weather and Climate Operational Supercomputing System (WCOSS) through increased computing power and storage will help improve U.S. forecast model guidance in the coming years by allowing more data to be fed into and analyzed by the system.

“This increased supercomputing power allows for upgrades to specific modeling systems that will help weather forecasters deliver more accurate weather forecasts, watches and warnings and improved certainty in a forecast,” said Ken Graham, director of NOAA’s National Weather Service.

The faster supercomputer will permit upgrades to NOAA’s weather forecasting systems and models over the next few years such as: upgrades to the U.S. Global Forecast System to make it higher-resolution; a new Rapid Refresh Forecast System which will allow for larger ensembles with more data included; upgrades to the Global Ensemble Forecast System, which will more accurately capture what is known as radiatively active aerosols to better model emissions such as wildfire smoke, dust and fog; and expansions in compute power and storage, which will provide operational capacity to implement research and development advancements made through NOAA’s Earth Prediction Innovation Center.

USDS to pilot new types of projects and set fresh objectives as part of 2024 strategy

The U.S. Digital Service will pilot new types of digital projects, establish fresh short- and long-term objectives to help hold itself to account, and push for stronger partnerships with tech organizations in the government as part of a new strategic plan for 2024, FedScoop has learned.

According to presentation slides of USDS Administrator Mina Hsiang’s strategic priorities for 2024 obtained by this publication, the agency will work with OMB on the budgeting process to support agency resourcing, increase USDS’s investment in organizational and talent operations, and better emphasize stories that humanize its work and impact in addition to highlighting important data points.

Although some of the language within the strategic priorities presentation is broad, it does not include specific mention of the agency’s self described top three objectives highlighted on its “our mission” page on the agency website: “1. Transform critical, public-facing services, 2. Expand the use of common platforms, services, and tools, 3. Rethink how the government buys digital services.”

However, the presentation does include an entire slide addressing their forth objective of the agency which is to “bring top technical talent into civic service.”

One of the key USDS’ strategic priorities for 2024 includes “clearly and collaboratively establishing near and long term objectives for our work and holding ourselves and agencies accountable to them,” the slides said. They were part of a presentation to shared with staff during a meeting last week led by Hsiang, according to a person familiar with the matter.

Furthermore, the agency plans to focus on “piloting new types of projects and approaches to expand our capabilities and learn what works,” according to the slides obtained.

Speaking with FedScoop, Administrator Hsiang confirmed details of the updated strategy, and said the priorities include: continuing to work directly with agencies like HHS and SSA to deliver better digital outcomes, helping to create stronger tech organizations within agencies to improve operations, and supporting agencies in building their capacity to hire more employees with tech expertise.

“Since earlier this year we’ve been partnering with the Social Security Administration, and we help them rebuild their entire public facing website. So if you look at SSA.gov now, as compared to what it was like in January of this year, it’s materially different,” Hsiang told FedScoop during a virtual interview from her home in Boston. 

“So the overall completion rate of tasks on SSA has increased by more than 10%. We’ve seen pretty significant improvements at SSA, on how the public is able to access services via their website based on that partnership that we’ve had with them,” Hsiang added.

Hsiang is the third administrator at USDS, which was launched in 2014 by the White House to provide consultation services to federal agencies on IT and technology issues as well as work to improve and simplify digital services including improving federal websites in particular.

According to Hsiang, website redesign is only one of several ways USDS works with federal agencies to improve digital experiences for citizens. It also often focuses on reengineering business processes and integrating technical services that are necessary for enhancing customer support and experience, she said.

In the presentation slides, the agency also stated that one of its strategic priorities is “partnering more effectively with other technology organizations across the gov like TTS, TMF and the OFCIO to ensure the best allocation of resources against end objectives.” 

Technology Transformation Services is a division of the General Services Administration (GSA), which is responsible for leading the digital transformation of government agencies. The Technology Modernization Fund (TMF) received a $1 billion infusion under the American Rescue Plan, and is focused on providing extra funding to agency technology projects with governmentwide implications.

The Office of the Federal Chief Information Officer (OFCIO) provides overall strategic direction for improving IT practices across government and sits within the Office of Management and Budget (OMB).

When asked about how specifically Hsiang planned to work more effectively with other tech entities in the federal government to transform the digital experience of Americans, Hsiang said there were no major improvements needed at the moment.

“We have an extremely close relationship at this point. I know at various points in time that hasn’t always been true,” the USDS administrator said.  

“We share full information about what we’re working on, how things are going, and ensure that we’re not duplicating and that we can be additive and are well coordinated across the organizations,” she added.

Homeland Security to launch explosives research database to help combat threats

The Department of Homeland Security plans to launch a database of explosives research, testing and evaluation data to assist personnel in mitigating threats in the fall.

DHS previewed the rollout of the Explosives Planning and Research Tool (ExPRT) in a Thursday blog post. The tool is currently undergoing testing and, when launched, will be a “secure, web-based one-stop-shop” for subject matter experts, first responders, and others in the explosives research community, the agency said.

“ExPRT provides critical explosives research to our [subject matter experts] who need it the most,” Anna Tedeschi, who manages the Explosives Threat Assessment program at DHS’s Science and Technology Directorate, said in the blog post.

“Once implemented it will vastly improve our collaborative efforts to continue protecting the nation from any future explosive threats, and also serve as a resource for ensuring best practices,” Tedeschi said.

ExPRT is intended to address the challenges presented by attempting to organize information and research about explosive threats, sharing that information, avoiding repetitive studies, preventing loss of institutional knowledge, and planning for investments in new future research, Tedeschi said.

The database will contain technical information, reports on screening and mitigation technology, completed and ongoing studies, and contact information for organizations involved in the research, according to the blog post. The information will span from the early 2000s to present day.

After internal testing is complete, the database will be independently assessed by the Department of Energy (DOE) and the Federal Bureau of Investigation (FBI), according to the blog post. The agencies will “perform functional use-case and other tests to validate how well it will work in the field.”

Veterans Affairs extends deadline for full backdated PACT Act benefits after website issue

Veterans and survivors applying for benefits under the PACT Act have a few more days to get the full amount of backdated benefits after a technical issue on the website caused some to receive error messages.

The new deadline to be eligible to receive benefits backdated to Aug. 10, 2022 is 11:59 p.m. ET on Monday, Aug. 14, the Department of Veterans Affairs said in a written statement Wednesday evening. It also said the VA has resolved nearly all the technical issues on the website. Fewer than one tenth of one percent of attempts to submit an “intent to file” that day resulted in an error message, according to the agency.

“VA has made this decision out of an abundance of caution after experiencing technical difficulties with VA.gov/PACT in recent days,” the statement said. 

The notification comes after the VA disclosed that about 18% of people who filed claims for disability compensation under the PACT Act Tuesday received error messages. 

The PACT Act, which became law last year, expanded health care and benefits to veterans who were exposed to toxic substances like burn pits and Agent Orange. The timing of the issue was important as the original deadline for people to be eligible to receive benefits backdated to August 2022 — when the legislation was signed into law — was Aug. 9.

Under U.S. administrative law, the VA may extend such a deadline when natural or man-made interference results in the the Veterans Benefits Administration experiencing extended delays in the receipt of claims.

Website error messages won’t affect veterans’ PACT Act claims submissions, VA says

The Department of Veterans Affairs is assuring veterans and survivors who received error messages while applying for PACT Act benefits on VA.gov yesterday that their claims were received and encouraged people to continue filing. 

In a Tuesday evening statement, the VA said an “extremely high volume of submissions” caused about 18% of people who filed claims for disability compensation under the PACT Act that day to receive error messages. 

The PACT Act, which became law last year, expanded health care and benefits to veterans who were exposed to toxic substances like burn pits and Agent Orange. The timing is important as people who file by Aug. 9 may be eligible to receive benefits backdated to Aug. 10 2022. 

Every veteran or survivor who received an error message while they applied for benefits under the law “can consider their intent to file complete,” the VA said in a written statement Tuesday. 

“We are working to contact these individuals to confirm directly to them that their intent to file will be honored and their effective date protected,” the statement said.

The VA noted that VA.gov/PACT isn’t down and is continuing “to collect intent to file submissions despite the outgoing error messages.” The agency said it’s working to receive the issue and in the meantime has a banner on the website alerting filers to the issue.

Commenting on the VA.gov issues in a statement, Rep. Mark Takano, D-CA, said: “I want to reassure those who experienced this error that VA has confirmed that their information has been captured, and their claim or intent to file will be considered as meeting the deadline.” He added: “I will ensure that VA makes every effort to over-communicate with veterans so they don’t lose a year’s worth of benefits because of VA’s IT issue.”

The lawmaker also called on the VA to establish what caused the website error.

Editor’s note, 9/8/23 at 5:50 p.m.: This story was updated to include comment from representative Mark Takano.