New Jersey shares materials on AI translation tools with other states

Working with civic tech nonprofit US Digital Response and Google.org, the New Jersey Office of Innovation announced Thursday that it will provide training materials for other states to help them build their own artificial intelligence-based systems Can use translation assistants for unemployment insurance systems.

The training materials are based on New Jersey’s new AI translation assistant, which the state deployed in April to improve language access, particularly for Spanish speakers, who account for 95% of language access requests in the state’s unemployment cases. To support the AI ​​Assistant, the state, with the help of USDR experts and labor from Google.org grantees, spent more than a year designing a new glossary of unemployment terms in Spanish, reviewed by bilingual call center agents to train the AI ​​assistant.

New Jersey officials said the AI ​​assistant has already tripled the speed of translation for the state’s call center workers, with quality comparable to that of human translators. Employment agencies in other states can use the materials to support the development of their own AI translation assistants. According to New Jersey, the resources are compatible with any commercially available major language model such as ChatGPT, Claude or Gemini.

Gillian Gutierrez, senior adviser and director of the New Jersey Unemployment Insurance Modernization Project, said the effort began shortly after Gov. Phil Murphy took office in 2018, when officials began discussing how to improve access to benefit programs .

“Whether it’s unemployment, whether it’s paid family medical leave, whether it’s an employee right like paid sick days, language has always been a part of it, but one of the challenges we’ve faced is: How do we make sure we do that?’ Feel confident (in that translation),” Gutierrez told StateScoop. “The government is always risk-averse, right? It’s worried about its legal responsibilities, and it should be, right? But how do we deal with the risk of offering a translated document that represents the state?”

The COVID-19 pandemic exacerbated the need to improve access to services, as did a legislative mandate last January. The new law says the state must improve language access through interpretation and in written documents, including for unemployment insurance.

Guiterrez said that in 2022, her department began using the specialized knowledge of the state’s bilingual UI call center agents to translate the “very technical jargon” and terms related to unemployment used by some Spanish-speaking residents.

“How can we use the agents’ knowledge and the time they’ve spent developing this type of explanation of difficult concepts to speed up the process so that we’re putting real, trustworthy translations in front of people and giving them greater access , better understanding, better knowledge of what rights they have, what responsibilities they have and what we ask of them. So they can end up getting the benefits they deserve,” she said.

Guiterrez said the Labor Department has been studying how to expand and improve the state’s glossary of terms on its unemployment website. She said this was done with the help of call center experts and required a lot of testing to ensure the descriptions were accurate.

Translations of the glossary terms were also tested against the knowledge of bilingual call center experts as well as “other unfounded translations” results from Google and ChatGPT. This testing and research phase took the longest, she said.

US Digital Response spent eight months helping New Jersey translate the glossary into plain language. Plain Language is a set of federal communications guidelines developed by the Plain Language Action and Information Network and underlying the Plain Language Act of 2010. The style includes grammatically correct and generally understandable language that includes complete sentence structure and accurate word usage.

“When we completed this project and then published the glossary in the translation guide, generative AI was commercialized. And suddenly ChatGPT blew up,” said Marcie Chin, product delivery manager for voice access at USDR. “We started researching all these things, like research around translation, how well these large language models actually work when it comes to language translation. And we thought, ‘Well, we’ve just developed all this very human-focused training data.'”

“These AI models are only as good as the data they are trained on. And so we were really excited that we happened to have all this really human-centered, really high-quality, accurate, verified data again that we can use to start training these large language models,” Chin continued.

Gutierrez said the state is testing additional uses for its AI assistant, such as integration with other state services, applications, websites and communications beyond the unemployment system. She said this could include other benefit programs, health portals or educational resources.

Krista Canellakis, director of USDR’s digital delivery program, told StateScoop that New Jersey’s solution offers a model for states still thinking about how best to use AI, and that the materials can help them develop their to make approaches “lower risk”.

“Many governments are quite hesitant to use these AI technologies for translations because there is no telling how accurate these translations are, especially when you look at existing technologies like the Google Translate widget found on most government websites. “This technology translates in a literal sense rather than a contextual one, so the quality of those translations is not always entirely accurate to you,” Canellakis said. “In this case, we’re really excited that we’re reducing translation risk for governments as they’re co-developed with policy experts and bilingual front-line workers, and then we’re also reducing risk for the end user.” well for Spanish-speaking residents or Workers in New Jersey.”