Advancing Language AI: How a Mixture of Experts Empowers Translation for Low-Resource Languages

Language AI and translation technology are constantly improving, but when are advances in technologies like natural language processing (NLP) truly significant? With a steady stream of news forthcoming, keeping a view of the bigger picture can be a challenge, while the vocabulary of algorithmic concepts that developers are tinkering with is more elaborate at each turn. If anything can concise be said for what makes a tweak in, say, GPT-3 a meaningful advance over the baseline model, it tends to revolve not around the brawn of computational resources that a licensed partner can dedicate to the model, but rather the subtleties of tailoring, training, and layering within its general framework that give machine learning a fighting chance at realistic uses of language. The goal in all of this is to replace what has so far been very credible mimicry of that with truly conversational artificial intelligence – something that may never see completion. This week, though, one such effort appears to be inching closer, with an algorithm that specifically advances translation by extending some of the more impressive achievements demonstrated between common language pairs to include lesser-resourced languages.

As VentureBeat reports, Microsoft’s Azure Translator service, a language AI widely deployed to power call centers, chatbots, and other real-time MT scenarios, is improving by roughly 15% in overall translation accuracy thanks to an update that delivers new components to its algorithmic framework. To this end, Microsoft has targeted a layer of Translator’s programming called its language-focused Z code, which forms one part of an “XYX” series of modules that together enable insights from machine learning vision models, for example, to inform machine learning speech models. Specifically, the addition of a Mixture of Experts (MoE) functionality in its Z code is helping a model with proven capabilities translating between English and other major languages extend some of its general knowledge to make better inferences about languages it has not learned as well. In short, based on a first reading of the linguistic information it receives, the Z code level of Translator will engage some of the many “expert” algorithms that it can call on to give candidate translations informed by specific linguistic contexts; based on this domain knowledge, these artificial experts are able to figure things out about a text in, say, Georgian that aren’t immediately apparent at the semantic level, forming an accurate idea of meaning based on what should realistically make sense regardless of dialect.

MoE models are a fascinating innovation not limited to Microsoft’s Translator tool, reflecting a growing shift to pair one kind of artificial intelligence with another where potentially complementary. These advances, however, demonstrate how even the best-resourced technology groups need to grapple with the limitations of computer knowledge when it comes to languages that don’t have equal representation one to the next in the human data footprint. It also underscores something at the very core of language services that helps ensure machine translation is a means to effective localization, rather than flawed outputs: subject matter expertise. By combining the power of powerful algorithms that can slice and dice language with the nuanced knowledge of experts, translation invariably improves its relevance and impact. For the time being, there is simply no substitute for the human linguists who ensure MT projects reach their mark, and that persistent reality is central to every effort to further automate translation.

For companies with growing needs for global content, CSOFT offers technology-driven translation solutions that leverage the power of language AI in combination with our global network of 10,000+ linguists and subject matter experts. Learn more at csoftintl.com!