Knowledge · Guide
Bilingual retrieval is not the same as translating the answer
Test whether questions and evidence can cross languages before polishing the interface.
Separate the language layers
A Chinese interface can still fail to find an English procedure. Separate interface language, query language, source language, and output language in the test plan. Microsoft’s multilingual-index guidance offers technical patterns, while the business acceptance question remains simple: can the authorized employee find the correct evidence in the language they naturally use? Translating an unsupported answer merely makes the same mistake easier to read.
Build paired questions
For an illustrative U.S. service business, ask the same operational question in English, Simplified Chinese, and Traditional Chinese. Include mixed-language abbreviations that staff actually use. Keep the expected source passage constant and compare retrieval before generation. A bilingual reviewer should assess meaning rather than insist on literal translation. Variations are useful when they expose genuine differences in vocabulary rather than artificial test difficulty.
Workflow at a glance
Preserve original evidence
Show the original passage alongside an optional translated explanation. Mark which text is a translation and let the user open the source. A translated document can become outdated independently of the original, so track their relationship and approval state. Do not let the newest translation silently override an older authoritative source without review. For important instructions, define who checks terminology and who resolves disagreement.
Measure service effort by language
Track unresolved questions and reviewer corrections by language pair. If Chinese-to-English retrieval is weak while English-only questions succeed, changing the final writing prompt is unlikely to be the first fix. Compare analyzers, vocabulary mapping, and embedding candidates using the same local cases. The MMTEB research demonstrates the breadth of multilingual evaluation, but a public benchmark cannot replace your own company-specific questions and approved documents.
A bilingual review session
Have bilingual employees ask paired questions without seeing each other’s wording. Compare which evidence appears and where clarification becomes necessary. Preserve natural phrasing in the test set; editing every question into formal documentation language hides the problem the assistant is supposed to solve.
Sources and further reading
- Create an index for multiple languages in Azure AI SearchCreate an index for multiple languages in Azure AI SearchMicrosoft · Documentation ·
- MMTEB: Massive Multilingual Text Embedding BenchmarkMMTEB: Massive Multilingual Text Embedding BenchmarkMMTEB authors / arXiv · Research source ·
Editorial approach
AI-assisted original writing, checked against linked public sources. Practical recommendations and illustrative scenarios are editorial analysis, not reported client results. Source dates and product details may differ from this collection’s preparation date.