Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendix 14 Translation Versus Postediting of Machine Translation This study reports the results of a small experiment done for the purpose of obtaining some facts regarding the process of postedit- ing machine-translation output as compared with the process of ordinary translation. In particular, information was desired con- cerning the relative speed and ease (or difficulty) of postediting as compared with those of translation. A variety of translators (i.e., commercial free-lance translators, government in-house translators, government contract translators, and bilingual persons who did not ordinarily engage in translation work) were sent a packet containing (~) a 1,135-word excerpt from a Russian book on cybernetics, Machina i Mysl', which they were to translate and provide typed copy of their translations; (2) a 765-word excerpt from the same book; (3) a print-out of the machine transla- tion of (2), which was to be postedited and typed; and (4) a question- naire (Exhibit I, page 99~. The translators were to keep a careful record of time spent in translating, editing, postediting, and (for some) typing. Those responding were: (a) three translators employed by commercial translation agencies (Numbers 2, 14, and 23~; (b) eleven translators who held contracts with the U.S. Joint Publications Research Service (Numbers 1, 3, 6, 7, 11, 13, 15, 16, 17, 18, and 22~; (c) six full-time translators employed, in-house, by an agency of the U.S. Government (Numbers 4, 9, 10, 12, 19, and 21~; and (d) three members of the faculty of the Russian department at the Defense Language Institute (Numbers 5, 8, and 20~. These three are language instructors and not primarily translators. EASE OF POSTEDITING Eight translators found postediting to be more difficult than ordi- nary translation. Six found it to be about the same, and eight found 91
it easier. (One translator indicated that he found the degree of difficulty to lie between "easier" and "the same.") Thus, from the answers received, it can be seen that the trans- lators were almost evenly divided in their opinions on the difficulty of postediting. The point of interest is that the more adept (rapid) translators found postediting more difficult than did the slower translators (see Exhibit 2, page 100~. The apparent paradox that those people who thought postediting was more difficult were more proficient at it than those who found it to be "the same" or "easier" is explained by the fact that those who found it more difficult are the same people who are the most adept at translation. From Exhibit 2 one may see that six of the eight translators who found postediting to be more difficult than translating were among the faster half, and that six of the eight translators who found postediting to be easier than translating were in the slower half . The average translation speeds of translators were as follows: those who found postediting more difficult, 11.9 wpm; those who found postediting easier, 6.5 wpm; and those who found postediting about the same, 7.9 wpm. The average postediting speeds of translators were as follows: those who found postediting more difficult, 9.4 wpm; those who found postediting easier, 8.6 wpm; and those who found postediting about the same, 8.0 wpm. RELIANCE ON THE ORIGINAL Only one translator (number 2) indicated that he seldom had to refer to the original (8a) in order to postedit machine translation. Eight translators indicated that it was almost necessary to translate the original (8b), and 14 translators answered that the degree of reli- ance fell between answers (8a) and (8b). It is of interest to note that most of those who said they had to translate the original were the fastest translators (and perhaps the best at translation). POSTEDITING AND TRANSLATION SPEED Translation Speed The fastest translation speed was 19.5 wpm by translator number 1 and the slowest was 4.2 wpm by translator number 23. The differ- ence between the translation rates of the fastest and slowest was 15.3 wpm; the mean speed was 8.7 wpm; the median was 7.6 wpm; the mode was 6.3 wpm (Figure 2~. 92
Postediting Speed The fastest posteditor was translator number 5, with a rate of 12.7 wpm. The slowest was translator number 23, with a rate of 3.9 wpm. The difference between the postediting rates of the fastest and slowest translators was 8.8 wpm; the mean postediting speed was 8.7 wpm; the median postediting speed was 9.2 wpm; the mode was 10.2 wpm (Figure 2~. ~ TRANSLATION POSTEDiTING 20 15 10 TRANS LATOR FIGURE 2. Speed (in words per minute) of translation arid postediting. OBSERVATIONS 13 (a) The mean speed for both translation and postediting was 8.7 wpm. (b) Although the fastest translator could translate almost five times as fast as the slowest translator, the fastest translator could postedit only about three times as fast as the slowest posteditor. (c) Of the 23 respondents, ten (3, 6, 7, 11, 13, 14, 15, 16, 17, and 22) indicated that they had had previous experience at postedit- ing machine-translation output (one translator said that he had postedited 93,000 words). Of this group, half had slower rates for postediting than for ordinary translation. Almost exactly the same ratio (number slower:number faster) held overall (11/23 slower: 12/23 faster). (d) The mean postediting speed of the experienced posteditors was 8.6 wpm. The mean postediting speed of those who did not indicate having experience at postediting was 8.8 wpm. 93
(e) 1. The four fastest posteditors had an average postediting rate of 11.8 and an average translation rate of 11.5. 2. The four slowest posteditors had an average postediting rate of 5.3 and an average translation rate of 6.1. 3. The four fastest translators had an average postediting rate of 10.4 and an average translation rate of 16.3. 4. The four slowest translators had an average postediting rate of 8.5 and an average translation rate of 5.3. Thus the differ- ence between the faster and slower of these two groups was only 1.9 wpm for postediting but 11 wpm for translation. 5. The fastest translator's postediting rate was the median for postediting (9.2 wpm). 6. The slowest translator was also the slowest posteditor. IMPACT OF POSTEDITING ON OUTPUT RATES Figure 3 indicates for each translator his speeds for postediting and translation. It is fairly obvious from a glance at this chart that fast translators will lose productivity if given postediting to do, whereas slow translators will gain. If translators are given postediting to do, then, contrasted with their translation rates: Translators 1-4 will show an aggregate loss of 23.6 wpm or 34 percent in output. Translators 5-8 will show an aggregate gain of 1.7 wpm or 5 percent in output. Translators 9-12 will show an aggregate gain of 2.1 wpm or 3 percent in output. Translators 13-15 will show an aggregate gain of 0.6 wpm or 3 percent in output. Translators 16-19 will show an aggregate gain of 6.3 wpm or 20 percent in output. Translators 20-23 will show an aggregate gain of 12.6 wpm or 37 percent in output. Thus, it may be seen that postediting machine translation tends to impede the rapid translators and assist the slow translators. 94
GAI N LOSS _ _ ~ ~ ~ ~ ~ . l ~ ~ ~ ~ ~ ~ ~ 30 40 _ rTRANSLATORS 1-4 1 5-8 1 9-12 13-15116-19l20-23 FIGURE 3. Percentage gain or loss in output from postediting. TIME SPENT PREPARING THE COPY Practice varied in producing typed translations. Some respondents combined various processes. Ten translators performed transla- tion, editing, and typing as separate operations. The total amount of time these 10 spent on the various processes was as follows: Translation 1, 69 7 min or 63 percent Editing 365 min or 13 percent Typing 645 min or 24 percent Average typing speed of translators was only 18 wpm. Not all translators produced a typed copy. WILLINGNESS TO POSTEDIT MACHINE TRANSLATION Twenty translators answered question 9a. Of the 20 replies, eight were negative, 11 were affirmative, and one was a qualified affirma- tive (yes, only if straight translation is not available). Of those who would do postediting at a lower rate than that received for translation, over half (6/11) would be willing to postedit for one half or less than the rate paid for translation. 95
Translators 4 1 1 2 Rate 1/3 1/3- 1/2 1/2 2/3 2/3- 3/4 3/4 4/5 It is of considerable interest (especially in a society that is alleg- edly materialistic) to compare the willingness to postedit at reduced rates with the respondents' speeds of translation and postediting (see Exhibit 2~. For example, although translator number 13 indi- cated that he would accept a rate of I/3 for postediting, his post- editing speed (7.0 wpm) is actually lower than his translation speed (7.3 wpm). Only one translator, number 22, would have broken even. The other 10 would be willing in effect, to do the same number of hours of work for less pay. Of those translators who indicated their willingness to postedit at reduced rates, one out of three were commercial translators, three out of six were government in-house translators. Seven out of 11 were government-contract translators (an eighth gave a qualified "yes"~. TRANSLATORS' REACTIONS TO POSTEDITING Twenty respondents took the time to give their reactions to the process of postediting machine-translation output. Although their remarks make interesting reading, for the purpose of this study we will only summarize some of the opinions expressed: Most of the translators found postediting tedious and even frus- trating. In particular, they complained of the contorted syntax produced by the machine. Other complaints concerned the excessive number of lexical alternatives provided and the amount of time re- quired to make purely mechanical revisions. A number of the ex- perienced posteditors remarked that, although the material in this study had been carefully keypunched, they had found in their previous experience that careless keypunching was a considerable detriment. Although no translator commented that he really liked to work with the machine output, a number stated that they found the output 96
served as an aid in the translation process, particularly with regard to technical terms. (The difficulty in trying to reflect accurately the opinions of the translators may be appreciated when one reads the following com- ment made by translator number 23~: "In conclusion, the MT was an aid and made translation easier, but when all the time used is figured up, was not as fast or profitable." TRANSLATORS' RECOMMENDATIONS Several of the respondents were moved to suggest possible improve- ments in the machine output: Number21 "I believe it might do well to scan the copy to be translated and provide a translator with a vocabulary and then allow him to translate it directly." Number 15 "Syntax-wise, some time in postediting might be reduced if the editor has knowledge of the degree of dissemination to be given the end product." Number 3 "A major improvement would be a much bolder programming of word-blocks which have a single or at most dual word English equivalent." Number 9 "More space for corrections would be a welcome format modifica- tion and would, incidently, help assure accuracy if the text is to be retyped after editing." CONCLUSIONS In view of the small sample that formed the basis for this study, any conclusions must be tentative. With this in mind, one might draw the following conclusions from this study: i. An adept translator's skills will probably be wasted on postediting. 2. The slower the translator, the greater the likelihood that his output can be increased by having him postedit machine translation. 97
3. Machine tr~sl~ion is not yet of such allay as to blow postedidng to be done Without ~ copy of the original in the hands of Me translator. 4. Iransl~ors me apt to be rather mediocre typists. 5. Either tr~nsl~ors do not consider their bme Id effort to be overly dead or our responders were exaggerating the bme neces- s~y to perform postedibug, since hag indicted their ~ilUngness to do the same work for less pay. 98
- - - Exhibit 1. QUESTIONNAIRE 1. Exactly how much time (hours and minutes) was required to translate document number 2? 2. Exactly how much time (hours and minutes) was required to edit the translation? 3. Exactly how much time (hours and minutes) was required to type this translation? 4. How much time was required to edit document number 3? 5. How much time was required to edit the edited copy (if this was necessary) ? 6. How much time was required to type document number 3? 7. How did you find the postediting process to be compared to the process of full translation from the original? Easier? O More Difficult? O About the Same ? 5 8. Check the appropriate box: O a. "It was necessary almost to translate the original in order to properly edit the machine output. " O b. "I seldom had to refer to the original." O c. "I placed not so great reliance on the original as question number 8, but greater than indicated by question number 9. " 9.a. Would you be willing to regularly postedit similar machine- translation output if you were to be paid at a lower rate than you earn for translating from a document in the original language ? Yes ~ No O 9.b. If yes, what is the lowest rate you would accept? Circle. 4/5 2/3 3/4 1/2 1/3 1/4 1/5 of the conventional translation rate. 10. Your candid comments and your reactions to the experience of postediting the machine output are invited below. 99
Exhibit 2. Data Compiled from Questionnaires Translator Number 1 2 3 4 5 6 7 8 9 I. Time (minutes) re- 58 65 73 87 120 120 120 125 134 quired to translate II. Time (minutes) re- 83 180 75 68 60 90 100 75 75 quired to postedit III. Postediting was found MD MD MD MD S S MD E E to be more difficult (MD) than transla- tion, about the same (S), or easier (E) IV. For postediting (A) it was necessary to translate, (B) seldom had to refer to the original, or (C) be- tween (A) and (B) C B A A A A C C C V. Willingness to regu- No No Yes No Yesa No Yes larly postedit MT output if paid at lower rate VI. Amount lower 2/3 1/2 VII. Translation speed 19.5 17.4 15.5 13.0 9.4 9.4 9.4 9.1 8.5 (wpm) VIII. Postediting speed 9.2 11.1 10.2 11.3 12.7 8.5 7.6 10.2 10.2 (wpm) IX. Editing speed (wpm) Com 25 Com 227 ND 19 56 ND 113 X. Typing speed (wpm) Com 19 Com ND ND 19 19 ND ND Com: Done in combination with other processes. ND: Not done. Yes, only if straight translation i Easier, but not much. 1/2 if typed copy not required, otherwise 3/4 to 4/5. Between easier and same. is not available. 100
10 11 12 13 14 15 16 17 _18 19 20 21 22 23 135 150 150 155 170 177 180 180 180 180 190 190 210 270 75 90 140 110 120 100 105 60 125 130 80 70 195 MD S S S Eb E E E S MD MD Ed E E A C A C C C C C C A C C C A No No Yes Yes Yes Yes Yes Yes Yes Yes No Yes No 3/4 1/3 2/3- 3/4c 1/2 1/3-1/2 4/5 4/5 1/2 8.5 7.6 7.6 7.3 6.7 6.4 6.3 6.3 6.3 6.3 5.9 5.9 5.4 4.2 10.2 8.5 5.4 7.0 6.4 7.6 7.3 12.2 6.1 5.9 9.6 9.6 10.9 3.9 ND Com 28 56 37 37 74 113 74 74 ND 56 32 15 ND 10 37 17 15 15 23 Com 14 ND ND ND 16 14 101
