Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 91
Appendix 14
Translation Versus
Postediting of Machine Translation
This study reports the results of a small experiment done for the
purpose of obtaining some facts regarding the process of postedit-
ing machine-translation output as compared with the process of
ordinary translation. In particular, information was desired con-
cerning the relative speed and ease (or difficulty) of postediting as
compared with those of translation.
A variety of translators (i.e., commercial free-lance translators,
government in-house translators, government contract translators,
and bilingual persons who did not ordinarily engage in translation
work) were sent a packet containing (~) a 1,135-word excerpt from
a Russian book on cybernetics, Machina i Mysl', which they were to
translate and provide typed copy of their translations; (2) a 765-word
excerpt from the same book; (3) a print-out of the machine transla-
tion of (2), which was to be postedited and typed; and (4) a question-
naire (Exhibit I, page 99~.
The translators were to keep a careful record of time spent in
translating, editing, postediting, and (for some) typing.
Those responding were:
(a) three translators employed by commercial translation
agencies (Numbers 2, 14, and 23~;
(b) eleven translators who held contracts with the U.S. Joint
Publications Research Service (Numbers 1, 3, 6, 7, 11, 13, 15, 16,
17, 18, and 22~;
(c) six full-time translators employed, in-house, by an agency
of the U.S. Government (Numbers 4, 9, 10, 12, 19, and 21~; and
(d) three members of the faculty of the Russian department at
the Defense Language Institute (Numbers 5, 8, and 20~. These three
are language instructors and not primarily translators.
EASE OF POSTEDITING
Eight translators found postediting to be more difficult than ordi-
nary translation. Six found it to be about the same, and eight found
91
OCR for page 92
it easier. (One translator indicated that he found the degree of
difficulty to lie between "easier" and "the same.")
Thus, from the answers received, it can be seen that the trans-
lators were almost evenly divided in their opinions on the difficulty
of postediting.
The point of interest is that the more adept (rapid) translators
found postediting more difficult than did the slower translators (see
Exhibit 2, page 100~. The apparent paradox that those people who
thought postediting was more difficult were more proficient at it
than those who found it to be "the same" or "easier" is explained
by the fact that those who found it more difficult are the same
people who are the most adept at translation.
From Exhibit 2 one may see that six of the eight translators
who found postediting to be more difficult than translating were
among the faster half, and that six of the eight translators who
found postediting to be easier than translating were in the slower
half .
The average translation speeds of translators were as follows:
those who found postediting more difficult, 11.9 wpm; those who
found postediting easier, 6.5 wpm; and those who found postediting
about the same, 7.9 wpm.
The average postediting speeds of translators were as follows:
those who found postediting more difficult, 9.4 wpm; those who
found postediting easier, 8.6 wpm; and those who found postediting
about the same, 8.0 wpm.
RELIANCE ON THE ORIGINAL
Only one translator (number 2) indicated that he seldom had to refer
to the original (8a) in order to postedit machine translation. Eight
translators indicated that it was almost necessary to translate the
original (8b), and 14 translators answered that the degree of reli-
ance fell between answers (8a) and (8b). It is of interest to note
that most of those who said they had to translate the original were
the fastest translators (and perhaps the best at translation).
POSTEDITING AND TRANSLATION SPEED
Translation Speed
The fastest translation speed was 19.5 wpm by translator number 1
and the slowest was 4.2 wpm by translator number 23. The differ-
ence between the translation rates of the fastest and slowest was
15.3 wpm; the mean speed was 8.7 wpm; the median was 7.6 wpm;
the mode was 6.3 wpm (Figure 2~.
92
OCR for page 93
Postediting Speed
The fastest posteditor was translator number 5, with a rate of 12.7
wpm. The slowest was translator number 23, with a rate of 3.9 wpm.
The difference between the postediting rates of the fastest and
slowest translators was 8.8 wpm; the mean postediting speed was
8.7 wpm; the median postediting speed was 9.2 wpm; the mode was
10.2 wpm (Figure 2~.
~ TRANSLATION POSTEDiTING
20 15 10
TRANS LATOR
FIGURE 2. Speed (in words per minute) of translation arid
postediting.
OBSERVATIONS
13
(a) The mean speed for both translation and postediting was
8.7 wpm.
(b) Although the fastest translator could translate almost five
times as fast as the slowest translator, the fastest translator could
postedit only about three times as fast as the slowest posteditor.
(c) Of the 23 respondents, ten (3, 6, 7, 11, 13, 14, 15, 16, 17,
and 22) indicated that they had had previous experience at postedit-
ing machine-translation output (one translator said that he had
postedited 93,000 words). Of this group, half had slower rates for
postediting than for ordinary translation. Almost exactly the same
ratio (number slower:number faster) held overall (11/23 slower:
12/23 faster).
(d) The mean postediting speed of the experienced posteditors
was 8.6 wpm. The mean postediting speed of those who did not
indicate having experience at postediting was 8.8 wpm.
93
OCR for page 94
(e) 1. The four fastest posteditors had an average postediting
rate of 11.8 and an average translation rate of 11.5.
2. The four slowest posteditors had an average postediting
rate of 5.3 and an average translation rate of 6.1.
3. The four fastest translators had an average postediting
rate of 10.4 and an average translation rate of 16.3.
4. The four slowest translators had an average postediting
rate of 8.5 and an average translation rate of 5.3. Thus the differ-
ence between the faster and slower of these two groups was only
1.9 wpm for postediting but 11 wpm for translation.
5. The fastest translator's postediting rate was the median
for postediting (9.2 wpm).
6. The slowest translator was also the slowest posteditor.
IMPACT OF POSTEDITING ON
OUTPUT RATES
Figure 3 indicates for each translator his speeds for postediting
and translation. It is fairly obvious from a glance at this chart
that fast translators will lose productivity if given postediting to
do, whereas slow translators will gain.
If translators are given postediting to do, then, contrasted with
their translation rates:
Translators 1-4 will show an aggregate loss of 23.6 wpm or
34 percent in output.
Translators 5-8 will show an aggregate gain of 1.7 wpm or
5 percent in output.
Translators 9-12 will show an aggregate gain of 2.1 wpm or
3 percent in output.
Translators 13-15 will show an aggregate gain of 0.6 wpm or
3 percent in output.
Translators 16-19 will show an aggregate gain of 6.3 wpm or
20 percent in output.
Translators 20-23 will show an aggregate gain of 12.6 wpm
or 37 percent in output.
Thus, it may be seen that postediting machine translation tends
to impede the rapid translators and assist the slow translators.
94
OCR for page 95
GAI N
LOSS
_ _ ~ ~ ~ ~ ~ . l ~ ~ ~ ~ ~ ~ ~
30
40 _
rTRANSLATORS 1-4 1 5-8 1 9-12 13-15116-19l20-23
FIGURE 3. Percentage gain or loss in output
from postediting.
TIME SPENT PREPARING THE COPY
Practice varied in producing typed translations. Some respondents
combined various processes. Ten translators performed transla-
tion, editing, and typing as separate operations. The total amount
of time these 10 spent on the various processes was as follows:
Translation 1, 69 7 min or 63 percent
Editing 365 min or 13 percent
Typing 645 min or 24 percent
Average typing speed of translators was only 18 wpm. Not all
translators produced a typed copy.
WILLINGNESS TO POSTEDIT
MACHINE TRANSLATION
Twenty translators answered question 9a. Of the 20 replies, eight
were negative, 11 were affirmative, and one was a qualified affirma-
tive (yes, only if straight translation is not available). Of those
who would do postediting at a lower rate than that received for
translation, over half (6/11) would be willing to postedit for one
half or less than the rate paid for translation.
95
OCR for page 96
Translators
4
1
1
2
Rate
1/3
1/3- 1/2
1/2
2/3
2/3- 3/4
3/4
4/5
It is of considerable interest (especially in a society that is alleg-
edly materialistic) to compare the willingness to postedit at reduced
rates with the respondents' speeds of translation and postediting
(see Exhibit 2~. For example, although translator number 13 indi-
cated that he would accept a rate of I/3 for postediting, his post-
editing speed (7.0 wpm) is actually lower than his translation speed
(7.3 wpm). Only one translator, number 22, would have broken even.
The other 10 would be willing in effect, to do the same number of
hours of work for less pay.
Of those translators who indicated their willingness to postedit
at reduced rates, one out of three were commercial translators,
three out of six were government in-house translators. Seven out
of 11 were government-contract translators (an eighth gave a
qualified "yes"~.
TRANSLATORS' REACTIONS TO POSTEDITING
Twenty respondents took the time to give their reactions to the
process of postediting machine-translation output. Although their
remarks make interesting reading, for the purpose of this study we
will only summarize some of the opinions expressed:
Most of the translators found postediting tedious and even frus-
trating. In particular, they complained of the contorted syntax
produced by the machine. Other complaints concerned the excessive
number of lexical alternatives provided and the amount of time re-
quired to make purely mechanical revisions. A number of the ex-
perienced posteditors remarked that, although the material in this
study had been carefully keypunched, they had found in their previous
experience that careless keypunching was a considerable detriment.
Although no translator commented that he really liked to work
with the machine output, a number stated that they found the output
96
OCR for page 97
served as an aid in the translation process, particularly with
regard to technical terms.
(The difficulty in trying to reflect accurately the opinions of the
translators may be appreciated when one reads the following com-
ment made by translator number 23~: "In conclusion, the MT was
an aid and made translation easier, but when all the time used is
figured up, was not as fast or profitable."
TRANSLATORS' RECOMMENDATIONS
Several of the respondents were moved to suggest possible improve-
ments in the machine output:
Number21
"I believe it might do well to scan the copy to be translated and
provide a translator with a vocabulary and then allow him to
translate it directly."
Number 15
"Syntax-wise, some time in postediting might be reduced if the
editor has knowledge of the degree of dissemination to be given
the end product."
Number 3
"A major improvement would be a much bolder programming of
word-blocks which have a single or at most dual word English
equivalent."
Number 9
"More space for corrections would be a welcome format modifica-
tion and would, incidently, help assure accuracy if the text is to be
retyped after editing."
CONCLUSIONS
In view of the small sample that formed the basis for this study,
any conclusions must be tentative. With this in mind, one might
draw the following conclusions from this study:
i. An adept translator's skills will probably be wasted on
postediting.
2. The slower the translator, the greater the likelihood that
his output can be increased by having him postedit machine
translation.
97
OCR for page 98
3. Machine tr~sl~ion is not yet of such allay as to blow
postedidng to be done Without ~ copy of the original in the hands
of Me translator.
4. Iransl~ors me apt to be rather mediocre typists.
5. Either tr~nsl~ors do not consider their bme Id effort to be
overly dead or our responders were exaggerating the bme neces-
s~y to perform postedibug, since hag indicted their ~ilUngness
to do the same work for less pay.
98
OCR for page 99
- - -
Exhibit 1.
QUESTIONNAIRE
1. Exactly how much time (hours and minutes) was required to
translate document number 2?
2. Exactly how much time (hours and minutes) was required to
edit the translation?
3. Exactly how much time (hours and minutes) was required to
type this translation?
4. How much time was required to edit document number 3?
5. How much time was required to edit the edited copy (if this was
necessary) ?
6. How much time was required to type document number 3?
7. How did you find the postediting process to be compared to the
process of full translation from the original?
Easier? O
More Difficult? O
About the Same ? 5
8. Check the appropriate box:
O a. "It was necessary almost to translate the original
in order to properly edit the machine output. "
O b. "I seldom had to refer to the original."
O c. "I placed not so great reliance on the original as
question number 8, but greater than indicated by
question number 9. "
9.a. Would you be willing to regularly postedit similar machine-
translation output if you were to be paid at a lower rate than
you earn for translating from a document in the original
language ?
Yes ~ No O
9.b. If yes, what is the lowest rate you would accept?
Circle.
4/5 2/3 3/4 1/2 1/3 1/4 1/5 of the conventional
translation rate.
10. Your candid comments and your reactions to the experience of
postediting the machine output are invited below.
99
OCR for page 100
Exhibit 2. Data Compiled from Questionnaires
Translator Number 1 2 3 4 5 6 7 8 9
I. Time (minutes) re- 58 65 73 87 120 120 120 125 134
quired to translate
II. Time (minutes) re- 83 180 75 68 60 90 100 75 75
quired to postedit
III. Postediting was found MD MD MD MD S S MD E E
to be more difficult
(MD) than transla-
tion, about the same
(S), or easier (E)
IV. For postediting (A)
it was necessary to
translate, (B) seldom
had to refer to the
original, or (C) be-
tween (A) and (B)
C B A A A A C
C C
V. Willingness to regu- No No Yes No — Yesa No — Yes
larly postedit MT
output if paid at
lower rate
VI. Amount lower 2/3 1/2
VII. Translation speed 19.5 17.4 15.5 13.0 9.4 9.4 9.4 9.1 8.5
(wpm)
VIII. Postediting speed 9.2 11.1 10.2 11.3 12.7 8.5 7.6 10.2 10.2
(wpm)
IX. Editing speed (wpm) Com 25 Com 227 ND 19 56 ND 113
X. Typing speed (wpm) Com 19 Com ND ND 19 19 ND ND
Com: Done in combination with other processes.
ND: Not done.
Yes, only if straight translation i
Easier, but not much.
1/2 if typed copy not required, otherwise 3/4 to 4/5.
Between easier and same.
is not available.
100
OCR for page 101
10 11 12 13 14 15 16 17 _18 19 20 21 22 23
135 150 150 155 170 177 180 180 180 180 190 190 210 270
75 90 140 110 120 100 105 60 125 130 80 70 195
MD S S S Eb E E E S MD MD Ed E E
A C A C C C C C C A C C C A
No No Yes Yes Yes Yes Yes Yes Yes Yes — No Yes No
3/4 1/3 2/3- 3/4c 1/2 1/3-1/2 4/5 4/5 1/2
8.5 7.6 7.6 7.3 6.7 6.4 6.3 6.3 6.3 6.3 5.9 5.9 5.4 4.2
10.2 8.5 5.4 7.0 6.4 7.6 7.3 12.2 6.1 5.9 9.6 9.6 10.9 3.9
ND Com 28 56 37 37 74 113 74 74 ND 56 32 15
ND 10 37 17 15 15 23 Com 14 ND ND ND 16 14
101
Representative terms from entire chapter:
found postediting
