Mobile app, cable queries among planned expansions of State Department’s AI chatbot

The Department of State is continuing to expand its artificial intelligence chatbot known as StateChat, including working toward a mobile version and the ability to query internal messages called cables, a department official involved in that effort told FedScoop.

John Silson, director of analytics in the State Department’s Center for Analytics, said the agency is “working on building out StateChat as a mobile application that our team members can use on their government phones.” 

Silson, who was speaking on a panel at Scoop News Group’s SNG Live event Thursday, said those efforts will continue to ensure security, but will also put the technology into “diplomats’ hands as they’re walking around, and taking meetings, and working with individuals.”

The department is also working on ingesting cables — internal dispatches department officials send back from the field — into StateChat, so teams can query them for information such as the last time the agency met with someone or what officials have said around the world. Silson called that “a particularly powerful use case.”

“I am excited about using these kinds of tools in the future because I think it can help us to manage our knowledge and information much more effectively in the State Department, and make our teams just really much more able to engage our partners overseas and work on policy here in Washington,” Silson said.

StateChat has been the centerpiece of the department’s recent efforts to put AI tools in the hands of its workforce. The chatbot, which leverages Palantir and Azure OpenAI, was rolled out last year. According to Silson, since that launch, the tool has amassed roughly 3 million prompts in its system and averages about 10,000 users a week. 

During the panel, which was about AI training and tailoring, Silson also discussed the process behind getting StateChat ready for use by officials and focused specifically on the importance of providing the models with context. 

When using a model like ChatGPT, it can sometimes take a couple of prompts to get a right answer, Silson said. So to address that and provide better results, the department creates “sophisticated prompting” that provides the large language model with information about how the agency operates and how to do different tasks.

The prompt the model gets around each inquiry might give parameters like “you are a mid-career professional at the State Department, you are an expert in all things diplomatic, etc., etc., please produce a summary in the following format,” Silson said.

That fine-tuning helps with the overall adoption of the tool, he said. 

“The trick is, if it doesn’t work once, then people will walk away,” Silson said. “Our goal is to reduce the number of times that that happens.”

Silson said the department monitors StateChat continuously and finds “it to be extraordinarily effective at responding” to questions from users. And ultimately, it’s saving people time. The department estimates that StateChat saves between 20,000 to 30,000 hours per week, he said.

That efficiency “gives our team both an edge in negotiations and information management that’s helping them to be more effective in the field,” Silson said.