Chinese neural machine translation can speed up work on technical documents, product content, and large localization projects. But fluent output can still get terminology, numbers, or market-specific language wrong. Simplified and Traditional Chinese also require separate checks.
Evaluate output on a representative sample of your own content. Check meaning, terminology, numbers, and the conventions of each target market. Use the findings to decide whether the material is suitable for raw machine translation, machine translation post-editing (MTPE), or translation from scratch.
1. How to Evaluate Chinese Neural Machine Translation Quality
Machine translation quality should be evaluated using actual documents and consistent criteria rather than simply assessing how fluently the translation reads.
1.1. Building a Sample Dataset for Testing
The test dataset should reflect the content businesses intend to translate, including industry-specific terminology, technical specifications, negation, complex sentence structures, and passages that require contextual understanding.
For example, a sample dataset of 100 sentences could include:
- 30 sentences containing industry-specific terminology.
- 25 sentences containing numerical data and operating conditions.
- 20 sentences involving complex contexts.
- 15 sentences requiring Simplified and Traditional Chinese variant checks.
- 10 sentences related to safety instructions.
This is an illustrative distribution, not a required ratio. The sample dataset should be adjusted based on the document type and actual level of risk.
1.2. Evaluation Criteria and Error Classification
Chinese NMT output quality should be assessed using the following criteria:
| Criteria | Evaluation Focus |
|---|---|
| Semantics | Preserving the original meaning without omitting information |
| Terminology | Accuracy and consistency with the glossary |
| Numerical data | Correct values, units, and conditions |
| Style | Suitability for the intended purpose |
| Chinese language variants | Correct terminology and market-specific conventions |
Errors can be classified into three severity levels: minor (affecting expression), major (distorting meaning or important terminology), and critical (potentially leading to incorrect decisions or unsafe operations).
Businesses should establish acceptance criteria before testing. For example, safety instruction documents may be required to contain no critical errors.
2. Common Errors in Chinese Machine Translation
Chinese machine translations may be grammatically correct but still contain errors involving terminology, numerical data, meaning, or expressions specific to the target market. These errors need to be identified and addressed during testing before the translated content is used.
2.1. Terminology, Numerical Data, and Contextual Errors
Terminology errors occur when a machine translation system selects words or terms that are inappropriate for a particular industry or field. For example, the term “device driver” in technical documentation should be translated into Chinese as “设备驱动程序” rather than “设备驾驶员” (a person who drives equipment).
Machine translation output should therefore be checked against an industry-specific terminology glossary to ensure that the correct terms are used.
Errors involving numerical data and operating conditions can change the intended meaning of instructions, particularly in technical documents and safety-related materials.
Example of Machine Translation Testing:
Source sentence: Do not operate the equipment when the temperature exceeds 80°C.
Hypothetical machine translation: 温度低于 80°C 时，请勿操作设备。
Corrected translation: 温度超过 80°C 时，请勿操作设备。
The hypothetical machine translation incorrectly changes the operating condition from “the temperature exceeds 80°C” to “the temperature is below 80°C,” reversing the meaning of the original warning.
This is a serious semantic error because the operating condition has been reversed, even though the numerical value of 80°C remains unchanged.
During machine translation testing, numerical values, measurement units, negation, and operating conditions must be carefully compared with the original source content. These checks help identify errors that could compromise the accuracy of the final translation.
2.2. Simplified and Traditional Chinese Translation Errors
Errors involving Simplified and Traditional Chinese go beyond differences in written characters. They can also involve terminology choices and expressions that vary by target market. The same concept may be expressed using different terms in Mainland China and Taiwan.
| Concept | Mainland China | Taiwan |
|---|---|---|
| Software | 软件 | 軟體 |
| Information | 信息 | 資訊 |
| Server | 服务器 | 伺服器 |
A translation that uses Traditional Chinese characters but retains terminology commonly used in Mainland China may not be appropriate for the Taiwanese market.
Therefore, businesses need to check not only the differences between Simplified and Traditional Chinese characters but also the terminology and expressions used in the translation to ensure they are appropriate for the intended market.
3. Terminology Control in Chinese Neural Machine Translation
Terminology control helps ensure translation accuracy and consistency, particularly for technical documents or product-related content containing specialized terms and industry-specific concepts.
Businesses should establish a terminology glossary before beginning the translation process. The glossary should include the original source terms, their approved translations, and the specific contexts in which each term should be used.
| Term | Simplified Chinese | Traditional Chinese |
|---|---|---|
| Controller | 控制器 | 控制器 |
| Temperature sensor | 温度传感器 | 溫度感測器 |
| Operating system | 操作系统 | 作業系統 |
These terminology examples are for illustrative purposes and should be verified based on the specific industry or field in which they will be used.
If the machine translation system supports terminology control, businesses can use the established glossary to guide translation output and help ensure that appropriate terms are used.
However, post-translation checks are still necessary to identify cases where terminology has been used incorrectly or inconsistently throughout the content.
Key areas to check include whether the terminology matches the approved glossary, whether each term is appropriate for its specific context, whether terms are used consistently throughout the translation, and whether they follow Simplified and Traditional Chinese language conventions.
Recording recurring terminology errors also helps businesses update and refine their glossaries and improve the translation process for future projects.
4. Chinese Machine Translation Post-Editing and Choosing the Right Translation Approach
Chinese machine translation post-editing (Chinese MTPE) is the process of having a human translator review, check, and correct machine translation output to ensure the content meets the required quality standards.
This process includes correcting errors involving meaning, terminology, numerical data, and writing style.
4.1. When Is Machine Translation Post-Editing Necessary?
Light post-editing focuses on correcting translation errors that affect the reader’s ability to understand the information. This approach is generally more suitable for low-risk reference documents.
Full post-editing involves a comprehensive review against the agreed publication requirements. It does not, by itself, certify a translation or satisfy a particular authority’s submission rules.
Illustrative Example:
Source sentence: Disconnect the power supply before removing the protective cover.
Hypothetical machine translation: 拆下防护盖后，请断开电源。
After post-editing: 拆下防护盖前，请断开电源。
The hypothetical machine translation incorrectly changes the sequence of operations from disconnecting the power supply before removing the protective cover to disconnecting it after removing the cover.
Machine translation post-editing helps identify and correct these types of errors, which could affect safety during equipment operation or use.
However, if the machine translation output contains too many serious errors or requires most of the content to be rewritten, businesses should consider translating the content from scratch rather than continuing with post-editing.
4.2. Should Businesses Choose Machine Translation, Post-Editing, or Human Translation?
Choosing between machine translation, machine translation post-editing, and human translation should depend on the level of risk associated with the documents, the quality of the machine translation output, and the amount of editing required.
| Document Type | Raw Machine Translation | Machine Translation with Post-Editing | Human Translation |
|---|---|---|---|
| Low-risk internal documents | Can be considered | Suitable | When necessary |
| Product descriptions | Limited use | Often suitable | When extensive localization is required |
| Technical documents | Should not be used by default | With appropriate quality control | Preferred for high-risk documents |
| Legal contracts | Should not be used directly | Requires thorough review | Preferred |
| Medical safety instructions | Should not be used directly | Requires strict quality control | Translation by specialists is preferred |
The table above provides general guidance for choosing a translation approach and does not represent actual test results from any specific machine translation system.
Businesses can combine multiple translation approaches within the same project based on machine translation test results.
Raw machine translation is more suitable for low-risk content that meets established quality acceptance criteria. Machine translation post-editing is an appropriate option when errors in the output can be corrected efficiently.
If the machine translation contains numerous serious errors, requires most of the content to be rewritten, or involves specialized professional requirements, businesses should consider translating the content from scratch and having the final translation reviewed by an appropriately qualified specialist.
5. Chinese Machine Translation Testing Process Before Implementation
Businesses can evaluate Chinese machine translation quality by following six steps:
- Define the requirements: Clearly identify the intended purpose of the translation, the target market where the content will be used, and the level of risk associated with the documents.
- Select a sample dataset: Include specialized terminology, numerical data, complex sentences, and cases where translation errors are likely to occur.
- Prepare translation rules: Establish a terminology glossary and language guidelines to be followed throughout the translation process.
- Conduct test translations: Save both the original source text and the machine translation output so they can be compared and checked for accuracy.
- Evaluate and post-edit: Record the types of translation errors identified, their severity levels, and the time required to correct them through post-editing.
- Make an implementation decision: Choose between raw machine translation, machine translation post-editing (MTPE), or translating the content from scratch based on the quality assessment results.
Key metrics to monitor include the percentage of sentences containing translation errors, the occurrence of serious errors, terminology consistency throughout the translated content, and the effort required for post-editing.
The decision to implement Chinese machine translation should be based on the quality acceptance criteria established before testing begins.
For high-risk documents, even a single translation error that changes the meaning of a safety instruction may be enough for the translation to fail to meet the required quality standards.
6. Frequently Asked Questions About Chinese Neural Machine Translation
Q1. Is Chinese neural machine translation accurate enough for business documents?
It can be, but accuracy depends on the industry, the quality of the machine translation system, and the intended purpose of the translated content. Businesses need to test translation quality before using the content, especially for technical, legal, or medical documents.
Q2. How can businesses evaluate the effectiveness of Chinese machine translation post-editing?
Businesses should assess the quality of the final translation, the number of errors remaining after post-editing, and the time required to make corrections.
If the effort required for Chinese MTPE is nearly equal to or greater than the effort needed to translate the content from scratch, businesses should reconsider their chosen translation approach.
Q3. Is it necessary to test Simplified Chinese and Traditional Chinese separately?
Yes. Testing Simplified Chinese and Traditional Chinese separately helps identify terminology errors, language convention issues, and expressions that may not be appropriate for each target market.
Q4. When should businesses choose to retranslate content instead of post-editing?
Businesses should consider translating the content from scratch when the machine translation contains numerous serious semantic errors, omits important information, or requires extensive revisions to most of the translated content.
Q5. Which metrics should businesses monitor when evaluating Chinese neural machine translation quality?
Businesses should monitor the percentage of sentences containing errors, error severity, terminology accuracy, and the time required for post-editing.
These metrics help businesses evaluate machine translation output quality and select an appropriate approach for handling the translated content. For high-risk documents, particular attention should be paid to errors that change the intended meaning or affect safety.
7. Conclusion
Chinese neural machine translation can help businesses process large volumes of documents, but translation quality must be carefully controlled before the content is used.
Conducting translation tests with representative sample data, checking terminology accuracy and consistency, and assessing potential risks help businesses determine the most appropriate translation approach for their specific needs.
Rather than applying a single translation method to every document type, businesses should choose between raw machine translation, machine translation post-editing (MTPE), and human translation based on actual translation quality test results.
Need to Evaluate Chinese Machine Translation Quality Before Implementation?
Testing machine translation output using sample documents helps businesses identify terminology errors and numerical inaccuracies, as well as assess the extent of post-editing required. Based on these testing results, businesses can select a translation approach that meets the quality requirements and risk level of their specific projects.
Contact Green Sun Japan for consultation on evaluating sample translations and selecting an appropriate translation solution.
