From translation to email drafting, State Department turns to AI to assist workforce

Over the past year, employees at the State Department have gotten something of an artificial intelligence upgrade. 

At this very moment, State Department workers stationed across the globe could use the internal AI chatbot, StateChat, to help them draft an email, translate a document or brainstorm policy. They can turn to Northstar to summarize and analyze news stories from multiple countries. And they might even query another chatbot, FAM Search, to look up department practices for booking a flight or cybersecurity protocols.

“What we are focused on is AI technologies that benefit the workforce right now, and in a lot of ways, that’s by gaining them efficiency in their day-to-day work,” Matthew Graviss, the chief data and AI officer at the department, said in an interview with FedScoop. 

Those AI deployments, all of which were launched in 2024, are part of a push from the highest levels of the State Department to harness the technology for its diplomatic mission. Secretary Antony Blinken has been vocal about AI’s potential to help employees free up time for more strategic work and improve the department’s analysis capabilities. That strategy involves moving quickly and putting the technology directly into the hands of department workers.

A central component of that work is StateChat, a ChatGPT-like interface launched in August that leverages Palantir and Azure OpenAI, a spokesperson confirmed, and is currently available to the more than 75,000 State Department employees around the world. 

“What I think is really cool is we’re seeing diplomats use StateChat to do their core job and allow them more time to do value-added work that only humans can do,” Kelly Fletcher, chief information officer of the State Department, said in an interview. 

That could mean meeting with colleagues for coffee and thinking creatively about solving the set of problems in the country they’re stationed in, Fletcher said.

And there are plans to put even more tools at employees’ fingertips. The department also expects to launch an AI Marketplace in 2025 focused on accelerating the adoption of the technology across the department by easing the connection between data and AI tools.

“Basically, three words to help guide us: Just get started,” Blinken told employees at a June event on the department’s use of AI. By November, he noted that AI tools, such as the StateChat, had saved department workers “tens of thousands of hours.” 

Working in 190 countries

The idea behind StateChat was to provide employees with the type of generative AI technology they’re able to access in their personal lives but in a sensitive unclassified environment where data is accessible only to them, Fletcher said.

That has the effect of preventing employees “from going to that other technology in cases where maybe it’s appropriate, but it’s not ideal,” she said.

The development of that project began last spring and included both the innovation and security sides of the department, Graviss said. Over the summer, leaders began onboarding general users who provided feedback on the platform, and that eventually led to the launch across the department.

According to a spokesperson, the combination of Palantir — which has driven the user interface development and integration with the large language model — and Azure OpenAI ensures the tool is “robust and secure.” 

Building the chatbot had to meet the department’s unique needs. The cloud-based tool had to be something that could work in a wide variety of environments to accommodate the fact that the department operates in countries where the government will shut off internet access, Fletcher said.

The ability to have that tool work in Washington, D.C., is one thing, she said, “but it needs to work in 190 countries.” So far, Fletcher said no one has come back to them with complaints that the tool isn’t working because of a connectivity issue.

An AI intern

Since its launch, State leaders have been helping workers use the tool based on the needs of their specific role within the department.

Economic, political, diplomatic, and technology officers all have different ways they go about using the technology, Graviss said. As a result, the department has a prompt analytics team examining how superusers are approaching the chatbot and using those findings to inform training so other people in the same roles can adopt those techniques.

“It’s one thing for technologists to communicate, ‘hey, this is great and use it.’ It’s another for diplomats themselves, who are superusers, to communicate to their peer group, ‘hey, this is what we’ve learned, and this is the value that we’ve seen,’” Graviss said.

Paula Osborn, deputy chief data and AI officer at the department, told FedScoop in an interview that her message to employees to help them understand how to use the tool is to “view StateChat as your intern or your assistant.” 

“You wouldn’t give something that your intern wrote to your ambassador,” Osborn said, noting that an employee would likely want to edit first in that scenario. But, like an intern, StateChat is really helpful for research and drafting. Osborn said she uses StateChat almost every day for tasks such as email drafting and summarization.

How users approach the chatbot is driving the path forward for the department. Graviss said  he and Fletcher didn’t want to assume what the right AI use cases were from the beginning, and opted to deploy a general purpose capability, observe how it’s being used, and then let that drive the department’s roadmap and focus areas.

Foreign service officers, for example, are using the tool to help them get up to speed when coming into a new role. 

Ideally, foreign service officers — who rotate every two to three years — overlap with their predecessor by a month or so to onboard, Graviss said. But when that isn’t the case, there’s a knowledge-transfer issue that the department has already been trying to address.

StateChat allows those officials coming into new roles to input documents, notes, cables, and other written communications and summarize them quickly. That cuts down what could have been a month-long process to synthesize the information to “a matter of minutes,” Graviss said.

Beyond general job efficiencies like summarization, Graviss said workers are also using the tool to generate modernization ideas and brainstorming policy. “There is a substantial efficiency play, but it’s not just that,” he said.

Improved reference manual

Some of the department’s other AI-powered tools, like FAM Search, are making what could have been time-consuming searches accessible to employees. That tool was launched in October to facilitate queries about information in the Foreign Affairs Manual, a comprehensive document with information about how the State Department conducts its business.

Before FAM Search, “the status quo was, if you wanted to find something in the FAM — you’re a new employee, you want to know how to book your flight — you would word search it,” Fletcher said. 

The FAM, however, is roughly 25,000 pages long and searching that way is “sort of lunacy,” Fletcher said. Now the document is available in a chatbot format that uses the same backend as StateChat. Through that tool, department workers can ask a question in common language and get a response with a summary of the relevant section and that section in full for reference.

In addition to making searches easier, deploying tools like FAM Search has the added benefit of providing technology leaders at the department with an opportunity to interrogate the data fueling the AI.

The FAM is an organic document and though the department has been careful with it over the years, Fletcher said it does have some inconsistent and outdated information. She was initially worried those inconsistencies in the FAM could be exacerbated by a chatbot and expressed that doubt to industry colleagues. 

But her colleagues pointed out that’s actually a good thing. “In some cases, we’re finding that deploying technology helps us to clean up data,” Fletcher said.

Similarly, an AI-powered resource called the Civil Service Career Pathing Tool is helping employees figure out what experience and training they might need to advance their career at the department. 

Through that program, which is loaded with 19,000 position descriptions, employees self-assess their competencies and can ask questions about things like what else they need to do to get to the next level in their field, Marcia S. Bernicat, State’s director general and director of global talent management, told FedScoop. 

An employee might get suggestions for training courses available at the Foreign Service Institute, and there’s plans to expand that database to courses outside the department as well, Bernicat said. 

Employees could also use it to discover other career paths available to them, she said. Someone could ask the program about other fields they might be competitive in, she said, “and like any AI program, it will get better the more people who use it, and more times an individual employee uses it.”

Institutional support 

Graviss attributes the State Department’s progress in rolling out AI capabilities to the approach outlined in the department’s strategy on the technology last year.

That document acknowledged that innovation, policy, infrastructure and culture aren’t linear goals, Graviss said; they all have to be done at the same time. Taking an iterative approach to AI development and following that strategy has been key for the department’s success, he said.

He said he knows people “in other agencies or in other organizations are struggling with what policy is necessary for adopting more artificial intelligence. Part of the way you do that is developing it as you start incrementally building the technology,” Graviss said. 

Support from leadership has been essential in allowing the department to grow its capabilities at a rapid pace, Osborn said. The fact that Blinken did an event with employees on generative AI “signals to everyone the importance of this and that everybody needs to be using it,” Osborn said. 

“I don’t know if all federal agencies have that, but it’s so critically important,” she said. 

Next on the docket for the department? The AI Marketplace. That tool is currently in development, according to a spokesperson, and will be aimed at helping people throughout the agency connect their data and workflows to approved AI tools more easily.

“The vision for the marketplace is ‘how do we accelerate AI adoption across the various technology organizations and across our consumer base?’” Graviss said. 

He said they’re thinking about that marketplace in both a business-to-business-like model — serving CDOs and CIOs within the agency — and as a business-to-consumer-like model through which anyone at the department has “the ability to sync their data with the benefits of generative AI.”

Laying that out and onboarding consumers and bureaus throughout the department is “what we’re focused on in this fiscal year,” Graviss said.