PaperlayerView exact revision on arXiv

Source-linked AI summary

AI translation of literary texts is "fine", but readers still prefer human translations

Yves Ferstler, Adam Podoxin, Ty Brassington, Roman Grundkiewicz, Maite Taboada, Marzena Karpinska

arXiv:2606.26040v1cs.CL

TL;DR

Existing evaluations miss how readers experience literary machine translation in terms of immersion and literary effect. This paper compares professional human and agentic LLM translations with readers, finding consistent human-translation preference despite readers’ difficulty identifying machine translation.

  • Problem

    Current literary-translation evaluations insufficiently capture readers’ immersive experience, literary effect, rhythm, emotion, voice, and deliberate word choices.

    Source: §1 · p. 1

  • Method

    Fifteen readers compare uninterrupted 8K-word excerpts and aligned 300-word chunks from 15 French, Polish, and Japanese novels translated by professionals and an agentic LLM pipeline.

    Source: §1 · pp. 1–2

  • Results

    Readers consistently prefer human translation, especially under close reading (522/772), while finding machine translation reasonable and failing to identify it reliably (17/30).

    Source: §1 · p. 2, §5 · p. 14

  • Takeaways & Limitations

    Reader-centered evaluation reveals preferences and within-book quality variation that automatic metrics miss, supporting further research on literary machine translation.

    Source: §1 · p. 2, §5 · p. 14

    The main evaluation covers only French, Polish, and Japanese translated into English, so broader language-pair generalization requires larger studies.

    Source: Limitations · p. 15

Abstract

from arXiv · show

AI translation of literary works is increasingly common. While the content may be rendered adequately, we do not know enough about how readers experience it in terms of immersiveness and literary effect, aspects poorly captured by automatic machine translation metrics or human evaluation targeting fluency and adequacy. We ask 15 avid readers to compare recently published human translations (HT) to machine translations (MT) generated with an agentic large language model (LLM)-based pipeline, for 15 recent novels in French, Polish, and Japanese and translated into English. Readers evaluated approximately 8K-word excerpts in two conditions: immersive reading of the whole excerpt (30 comparisons) and close reading of 386 aligned HT-MT chunk pairs (772 comparisons), with two readers per book and in alternating order of presentation. Overall, readers find MT "fine", but prefer HT (slightly at excerpt-level 19/30, more clearly at chunk-level 522/772) for its ease, clarity, and immersive nature. Readers' highlights show that MT's quality varies more within one book than HT's does. Crucially, readers cannot reliably tell the two apart (17/30 guess correctly) and tend to prefer the version they believe to be human. Automatic metrics, including LLM-as-a-judge approaches, fail to recover reader preferences and favor MT. We release LAIT (Literary AI Translation), a reader-centered evaluation dataset with 1K reader comments, 2K judgments and preference ratings, and 7.2K span-level annotations, along with our evaluation protocol and supporting interface.

1 Introduction

Readers slightly but consistently prefer professional human translations over agentic LLM-based machine translations, especially under close reading, although machine translations remain readable and difficult to identify. The paper addresses this gap with a reader-centered evaluation of literary experience and releases the LAIT dataset and reproducible protocol.

Motivation

  • Existing translation evaluations miss readers’ responses to rhythm, emotion, voice, and deliberate word choice because they rely on small units, segment ratings, or error annotations.The paper therefore asks readers directly about literary immersion and perceptions of the author’s work.Source: §1 · p. 1

Methodology

  • The study compares professional HT and agentic LLM-based MT through uninterrupted 8K-word reading and side-by-side 300-word chunk evaluation by 15 avid readers.Readers also provided preference strengths, justifications, suspected-MT explanations, and span-level highlights; presentation order was counterbalanced.Source: §1 · pp. 1–2, Figure 1 · p. 2

Results

  • 522/772 close-reading judgments preferred HT, versus 19/30 immersive-reading judgments, while MT remained readable and sometimes won.MT quality varied more within individual books than HT, and success depended more on the book than the source language.Source: §1 · p. 2
  • 17/30 readers correctly identified MT after reading both excerpts, while automatic metrics, including LLM-as-a-judge methods, favored MT over reader preferences.Readers were misled by common presumed AI markers such as em-dashes and assumptions about swear words.Source: §1 · p. 2

Contributions

  • LAIT contains 15 FR/PL/JA→EN novel openings with 2K ratings and preferences, 1K comments, and 7.2K span-level annotations, plus development data.The paper also releases the reader-centered evaluation guidelines and annotation software for reproducibility and future research.Source: §1 · p. 2

2 Data & Methods

The study constructs LAIT to compare professional human translations with agentic machine translations of literary excerpts from French, Polish, and Japanese into English. It combines immersive whole-excerpt reading with side-by-side close reading of aligned chunks.

2.1 Constructing

  • LAIT contains 31 opening excerpts of approximately 8K words from recently published novels, including 15 books for human evaluation and 16 for MT-pipeline selection.The evaluation set includes five books per source language, with excerpts paired to published human translations and corresponding machine translations.Source: §2 · p. 2, §2.1 · p. 3
  • The selected French, Polish, and Japanese novels span distinct language families, cultures, and literary traditions, while English probes translation into a language where modern models are especially strong.Books were recently published and critically acclaimed, with professional English translations from 2025–2026 to mitigate data contamination (Jacovi et al., 2023; Kocyigit et al., 2025).Source: §2.1 · p. 3
  • P3, an AutoFiction-inspired agentic pipeline using Claude Code and Codex, was selected from five configurations after a blind five-way preference study on 16 development books.P3 received 5/16 first-choice votes, ahead of Gemini-based P1 with 4/16, while GPT-5.4 P1 received 1/16; the authors treat this as a practical rather than decisive choice.Source: §2.1 · p. 3
  • The adopted pipeline preprocesses sources into 1K-token chunks, translates and reviews them for faithfulness and literary quality, then performs excerpt-level consistency review with gated revisions.It uses Claude Code and Codex, allows up to three local cycles and two global cycles, and triggers revision when automatic reviewers find major problems or thresholds are unmet.Source: §2.1 · pp. 3–4, Figure 2 · p. 4

2.2 Human Evaluation Protocol

  • The human evaluation covers 15 excerpts, each assessed by two readers through immersive reading and 772 side-by-side comparisons of 386 aligned approximately 300-word HT–MT chunk pairs.Fifteen avid English-reading participants each evaluated two books, with within-subject comparisons and a one-day break between immersive and close-reading tasks.Source: §2.2 · pp. 4–5

3 Results & Analysis

This section analyzes reader preferences, perceptions, and identification of human and machine literary translations across immersive and close-reading evaluations, alongside automatic metrics and a multilingual case study. It examines how these assessments compare and where they diverge.

3.1 Readers’ ratings and preferences

  • 522/772 close-reading judgments favored HT versus 250 for MT, while excerpt-level preferences were 19/30 for HT versus 11/30 for MT; only the chunk-level preference was significant.The order-adjusted HT probability was 76.2% at chunk level (p=.011) versus 63.4% at excerpt level (p=.148).Source: §3.1 · pp. 6–7, Figure 4 · p. 6
  • HT received higher odds of better acceptability and smoothness ratings than MT, with 4.0× and 4.3× higher odds respectively, while immersion and willingness to continue showed smaller nonsignificant HT advantages.MT still won 32% of chunk-level choices and received moderate-to-strong willingness-to-continue ratings in 54% of responses versus 66% for HT.Source: Figure 5 · p. 6, §3.1 · p. 7
  • MT’s preference share was stable across languages but varied sharply by book, ranging from 4% to 88%, while poor-wording variability was higher for MT in 13 of 15 books.Close-reading MT preferences were 32% for French, 34% for Japanese, and 31% for Polish; book identity strongly predicted preference (p<.001).Source: §3.1 · pp. 7–8
  • Span evidence tracked explicit preferences closely, matching stated choices in 91.5% of non-tied judgments by highlighted-word scores and 95.8% by span counts.HT received more positive and fewer negative highlights than MT, while MT negative evidence was concentrated in 41.7% of chunks versus 11.9% for HT.Source: §3.1 · pp. 7–8
  • Reader agreement was higher for local, concrete quality judgments than for global literary preferences, with excerpt-level choice agreement at 40.0% versus 72.2% for non-tied chunk choices.Agreement was strongest when both readers marked one version significantly better (79.5%), while quality-rating agreement was stronger for HT than MT (AC2 = 0.616 vs. 0.216).Source: §3.1 · pp. 8–9

3.2 What do we learn from readers’ comments?

  • Readers praised both systems for readability, but credited HT more often with smoothness, comprehension, engagement, immersion, and vividness, whereas MT sometimes won through immediate readability or sharper local wording.HT was praised for smoothness in 19/30 comments versus 11/30 for MT and engagement/immersion in 12/30 versus 5/30; MT preference comments emphasized local wording and contextual fit.Source: §3.2 · pp. 9–10

3.3 Can readers detect machine translation?

  • Readers could not reliably distinguish MT from HT: they identified the actual MT in only 17/30 comparative pairs, and in 28/30 comparisons they judged their preferred translation to be human.Word choice was nondiscriminative, while literal rendering, formatting, and poor sentence flow were more informative than folk theories such as em-dashes as AI markers.Source: §3.3 · pp. 11–12, What cues do readers use to identify MT? · pp. 12–13

3.4 Can automatic metrics capture readers’ preferences?

  • All automatic metrics prefer MT, while their chunk-level Kendall τ correlations with human judgments are weak or negative.LITRANSPROQA favors MT (0.996 vs. 0.920), and correlations range from −0.075 to −0.318 across metric variants.Source: §3.4 · p. 13

3.5 Case study: How about translations into languages other than English?

  • The multilingual comparison used recent professional translations of two novels into Spanish, French, Polish, and Japanese to test whether MT’s English performance generalized.Readers compared translations of My Grandfather, the Master Detective and The Witcher: Crossroads of Ravens in a small-scale case study.Source: §3.5 · p. 13
  • In the multilingual case study, readers preferred HT in 103/111 chunks (92.8%), with excerpt-level preferences favoring HT except for one French reader.The French reader still rated HT only one point below MT and later shifted toward HT during close reading.Source: §3.5 · pp. 13–14
  • Unlike the English-facing study, 4 of 5 multilingual readers correctly identified MT, with mean confidence of 4.4/5.Readers cited translationese, grammatical and structural problems, calques, and nonstandard or nonidiomatic vocabulary as concrete cues.Source: §3.5 · p. 14

4 Related Work

Prior literary machine-translation evaluation relies heavily on surface-level automatic metrics and short or local human judgments. This work instead centers readers’ experience of sustained immersive reading and close comparison of long-fiction human and machine translations.

Automatic evaluation

  • Automatic metrics, including neural and LLM-as-a-judge approaches, provide weak signals of literary quality and do not directly measure reader experience, especially over longer dependencies.LLM-as-a-judge protocols have mainly been tested on short passages, while document-level evaluations often combine them with automatic metrics without directly measuring reading experience.Source: §4 · p. 14

Human evaluation

  • Human evaluation commonly uses error annotations, crowdsourced ratings, or short local judgments that miss literary translation’s purpose as sustained reading.Prior literary MT and reception research is narrower than a reader-facing comparison of professional HT and recent MT across long fiction excerpts.Source: §4 · p. 14

Research gap

  • The study targets readers’ experience of long literary MT versus HT through immersive reading and close reading, addressing gaps in prior literary MT evaluation.Its framing connects to research on narrative transportation, AI perception, literary translator voice, and ethics in machine-assisted literary translation.Source: §4 · p. 14

5 Discussion & Conclusion

Across 15 French, Polish, and Japanese novels, readers found agentic-LLM machine translations reasonable and difficult to detect, yet consistently preferred professional human translations, especially during close reading. Preferences varied mainly by book, automatic metrics favored machine translation, and the authors release LAIT and their evaluation pipeline.

Discussion & Conclusion

  • Readers consistently preferred human translations over reasonable machine translations, especially in close reading, while failing to detect the translation type reliably.The comparison covered professional human translations and a strong agentic LLM pipeline across 15 recent novels.Source: §5 · p. 14
  • Preferences varied mainly by book rather than source language, and readers often preferred the version they believed was human.The novels were translated from French, Polish, and Japanese into English.Source: §5 · p. 14
  • Automatic metrics favored machine translation despite readers’ preferences for human translation.Source: §5 · p. 14
  • The study releases LAIT and its evaluation pipeline to support future research on literary machine translation.Source: §5 · p. 14

Limitations

The study evaluates 8K-word excerpts rather than full books, and its curated corpus, language directions, exploratory multilingual case study, and small reader sample limit generalizability and statistical power.

Excerpt scope

  • The evaluation covers novel openings rather than full books, so longer-term literary quality, pacing, narrative voice, and book-spanning context may be missed.Financial constraints limited the study to excerpts, although 8K words is already comparable to short-story length.Source: Limitations · pp. 14–15

Corpus selection

  • The recently published, critically acclaimed corpus with professional English translations reduces training-data contamination but limits literary diversity and representativeness.The books were published in 2025–2026.Source: Limitations · p. 15

Languages and directions

  • The main evaluation includes only French, Polish, and Japanese translated into English, while the multilingual case study is exploratory and should not be generalized.The main sample has five books per source language; the case study uses two additional books, four target languages, and a smaller, mixed-recruitment sample.Source: Limitations · p. 15

Sample size and agreement

  • Only two Upwork-recruited readers evaluate each book, limiting statistical power and leaving chunk-level inter-reader agreement modest in places.The within-subject design reduces individual variability and requires fewer annotators than a between-subjects design.Source: Limitations · p. 15

Ethics Statement

The study reports ethics approval, informed consent, fair participant compensation, anonymization, and restricted research-only data sharing, while acknowledging corpus limits and risks of literary machine translation.

Human participants

  • The study received ethics approval, obtained informed consent, allowed withdrawal with prorated compensation, paid participants fairly, and anonymized their identities.Participants were paid $110 USD per book, approximately $27.50/hour for roughly four hours, plus completion bonuses.Source: Ethics Statement · p. 15

Copyright and data sharing

  • Source books were purchased for research, no more than 10% of any individual work is retained, and the dataset will be released only for research.These practices follow fair-dealing/fair-use principles for scholarly research.Source: Ethics Statement · p. 15

Corpus and societal impact

  • The corpus overrepresents internationally published, commercially successful fiction, and the study aims to improve reader-centered evaluation and publishing transparency rather than replace human translators or deceive readers.The authors emphasize that readers often prefer human-authored translations and hope evaluation supports fair labor practices and informed reader choices.Source: Ethics Statement · p. 15

Risks of literary machine translation

  • Literary MT can appear acceptable while altering readers’ impressions through problems with phrasing, context, voice, or biased and harmful wording, especially with little human review.These risks may affect perceptions of both a book and its author.Source: Ethics Statement · p. 15

AI use disclosure

  • The authors used coding agents for plots and table-layout editing and language models as writing assistants, but no part of the paper was generated from scratch.Source: Ethics Statement · p. 15

A Dataset · B Machine Translation · C Agentic pipeline runs

The appendix documents the evaluation dataset, five machine-translation setups culminating in an agentic pipeline, and the resulting run artifacts and acceptance outcomes. It reports how the pipeline was selected, implemented, gated, and executed across development and evaluation books.

A Dataset

B Machine Translation

  • The study compared three pipelines across five setups: full-excerpt translation, chunk-level translation with excerpt-level post-editing, and an agentic Claude Code–Codex pipeline.GPT-5.4 and Gemini 3.1 Pro were selected for strong WMT 2025 performance, while coding agents supported long-context tasks and multiple reference documents.Source: B Machine Translation · p. 19
  • P1 translated each complete excerpt in one call, whereas P2 translated paragraph-preserving chunks of at most 1K source tokens before excerpt-level revision.Both procedures used prompts targeting meaning, tone, voice, register, rhythm, stylistic effect, ambiguity, characterization, and narrative voice.Source: B Machine Translation · p. 19
  • P3 combined style analysis, chunk-level translation, and excerpt-level review with automatic metrics and guard gates that triggered revisions when issues appeared.Chunk acceptance required no quality-estimation NO judgments and at most five MAYBEs, plus no MEDIUM-or-higher literary findings; final acceptance required no MEDIUM-or-higher issue in either review.Source: B Machine Translation · pp. 19–20
  • 5/16 development excerpts preferred P3, which was adopted over Gemini- and GPT-based alternatives by a small margin for more idiomatic vocabulary.The remaining preferences were P1 Gemini 4/16, P2 Gemini 3/16, P2 GPT 3/16, and P1 GPT 1/16.Source: B Machine Translation · p. 20

C Agentic pipeline runs

  • 35 completed runs comprised 15 evaluation-book, 16 development-book, and 4 multilingual target-language runs, processing 441 source chunks and producing 1,059 translation or revision artifacts.Job logs covered 31 runs and recorded 2,930 jobs, including 19 failures; four development runs lacked job-metric logs.Source: C Agentic pipeline runs · p. 20
  • 13 runs passed the automatic final acceptance gate, while 22 retained flagged issues across 74 chunks.Even when translations failed within enforced limits, increasing those limits did not yield justifiable improvements, and failed outputs were often acceptable.Source: C Agentic pipeline runs · p. 20

D Human evaluation

The human evaluation recruited avid, native-level English readers through Upwork and used consent, demographic, immersive-reading, comparison, and close-reading questionnaires in a counterbalanced interface.

Participant recruitment

  • Participants were recruited through Upwork, limited to avid readers who read at least one English book monthly and had native-level English proficiency.They completed an entry questionnaire, with demographics and AI-use frequency reported separately.Source: D Human evaluation · p. 20

Procedure

  • The study collected consent and demographic information before evaluation through Microsoft Forms and recruitment-stage reading-habit questions.Source: D Human evaluation · pp. 20–21

Questions and interface

  • Readers completed separate immersive-reading questionnaires, holistic comparison questionnaires, and chunk-level close-reading comparisons for each translation pair.Immersive-reading order was counterbalanced, while close-reading display order was randomized for each chunk pair.Source: D Human evaluation · p. 21

Display normalization

  • Display normalization standardized quotation marks, guillemets, apostrophes, and ellipses for presentation and annotation without modifying source files.Span-highlight offsets referred to the same normalized text shown in the evaluation interface.Source: D Human evaluation · p. 21

E Human evaluation results

The appendix expands the human evaluation with book- and chunk-level preferences, agreement diagnostics, comment analyses, rating breakdowns, detection analyses, and post-study responses. Readers’ evidence overlapped more when they agreed on the preferred translation, while AI experience did not improve detection and participants generally favored human-edited publication.

Overall results

  • Additional figures break down book-level and chunk-level preferences, source-language patterns, single-reading ratings by presentation order and final preference, and book-level AI guesses.These analyses include highlighted-span counts and lengths, preference direction and strength, and ratings for acceptability, smoothness, immersiveness, and willingness to continue reading.Source: E Human evaluation results · pp. 21–22

Agreement

  • The appendix reports Gwet’s AC2 for ordinal single-reading ratings, Gwet’s AC1 for binary judgments, and book-level agreement analyses across preference conditions.Agreement covers acceptability, smoothness, immersiveness, would-continue ratings, MT identification, excerpt preference, chunk preference, and book-level variation.Source: E Human evaluation results · p. 21
  • Span overlap was 0.307 overall and 0.217 when labels also matched, increasing to 0.322/0.263 among readers agreeing on preference versus 0.280/0.137 among disagreeing readers.Overlap was measured both independently of and conditioned on matching good/poor labels.Source: E Human evaluation results · p. 21

Comment analysis

  • Comment analyses categorize why readers preferred HT or MT at excerpt and chunk levels and summarize positive and negative aspects attributed to both translations.Tables 23–28 and Figures 23–25 organize quality labels, AI-generation reasons, preference reasons, and comments about the competing translation.Source: E Human evaluation results · pp. 21–22

AI-detection guesses

  • Readers who frequently use AI were not more accurate at detecting AI translation, with no apparent correlation between AI experience and detection accuracy.Detection guesses were also compared across single-reading and final-comparison responses, with additional correctness and confidence flows.Source: E Human evaluation results · p. 22

Post-study interviews

  • Participants’ post-study consensus was that agentic LLM translation is not ready for standalone publication but could provide drafts for expert human editing.Readers were often pleasantly surprised by MT quality, yet cited literalness, missed context, mistranslated idioms, and flattened authorial voice; some also criticized HT excerpts.Source: E Human evaluation results · p. 22

F Case study results

This appendix case study examines literary translation into Spanish, French, Polish, and Japanese. Across target languages, readers strongly preferred human translations in direct comparisons, while identification accuracy and span-level evaluations varied by translation type and language.

MT identification

  • Readers reliably identified MT for Polish, Spanish, and Japanese, but the French reader was incorrect in the final excerpt-level comparison.The Spanish reader initially misidentified MT during single-reading but distinguished both translations during comparison, suggesting higher translation quality for French and Spanish than Polish and Japanese.Source: F Case study results · p. 23

Preference judgments

  • 4 out of 5 readers preferred HT at book level, and all 5 preferred it in close-reading judgments across target languages.Preferences shifted strongly toward HT when readers directly compared translations and evaluated aligned chunks.Source: F Case study results · p. 23

Span annotations

  • Readers marked mostly positive evidence in HT and mostly negative evidence in MT across target languages, with median span lengths indicating judgment localization.These span-level patterns are reported alongside Figure 20 and Figure 19.Source: F Case study results · p. 23

G Statistical analysis

The appendix matches statistical models to response types, using mixed-effects analyses with reader and book random intercepts whenever estimable. Singular random-effects structures trigger simpler alternatives, including fixed-effect, intercept-only, or exact-binomial analyses.

Binary preferences

  • Analyses used logistic GLMMs for binary preferences, coding HT as 1 and MT as 0, with reader and book random intercepts when estimable.These models estimate whether readers are more likely to choose HT than MT.Source: G Statistical analysis · pp. 23–24

Attribute comparisons

  • Signed dialogue and word-choice preferences were analyzed with order-adjusted fixed-effect linear models because crossed reader-book mixed-effects models were singular.HT was coded +1, no difference 0, and MT −1.Source: G Statistical analysis · p. 23

Ratings and preference strength

  • Preference-strength scores used linear mixed-effects models, while 5-point ordered ratings used cumulative-link mixed models with reader and book effects when estimable.Ratings covered acceptability, smoothness, immersion, and willingness to continue reading; cumulative-link models avoid assuming equal distances between adjacent scale points.Source: G Statistical analysis · p. 23

AI detection

  • AI-detection correctness used logistic mixed-effects models, or an intercept-only logistic model and exact binomial test when direct-comparison random effects were singular.Confidence was analyzed with linear mixed-effects models among correct direct-comparison guesses.Source: G Statistical analysis · pp. 23–24

Highlight counts and implementation

  • Highlighted evidence spans were modeled with negative-binomial mixed-effects count models, appropriate when count variance exceeds Poisson expectations.Analyses were conducted in R using lme4, lmerTest, emmeans, and ordinal; formulas and outputs appear in Tables 35–38.Source: G Statistical analysis · pp. 23–24

H Automatic Evaluation · 5 What, if anything, stood out positively in the wording or style of this version?

The appendix evaluates automatic translation metrics at paragraph and chunk levels, comparing five MT setups and testing whether metrics recover human preferences between professional HT and agentic P3 MT. It also documents the human-evaluation questions used to assess acceptability, fluency, immersion, and willingness to continue reading.

H Automatic Evaluation

  • Automatic evaluation compares five MT setups across 16-book development and 15-book evaluation sets, then tests metric recovery of readers’ preferences between professional HT and agentic P3 MT.The five setups are P1 and P2 with GPT-5.4, P1 and P2 with Gemini 3.1 Pro, and agentic P3.Source: H Automatic Evaluation · p. 24
  • Paragraph-level evaluation uses aligned source paragraphs, filters likely misalignments with length and lexical-overlap heuristics, and manually inspects flagged cases.Length issues are flagged when one paragraph exceeds three times the other, while lexical issues use pairwise BLEU below 1.5 against most aligned MT outputs.Source: H Automatic Evaluation · p. 24
  • The paragraph-level metrics comprise reference-based COMET-22 and METRICX and reference-free COMETKIWI and METRICX-QE, with model-specific context limits and skipped overlong pairs.COMET scores are higher-is-better, whereas METRICX scores are lower-is-better; reference-based metrics use professional HT as reference and are not reported for HT itself.Source: H Automatic Evaluation · p. 24
  • Table 39 summarizes paragraph-level par3 automatic-evaluation results for all systems.The supplied passage identifies the table’s scope but does not provide its cell values or a supported winner.Source: H Automatic Evaluation · p. 24
  • Chunk-level comparisons reuse the approximately 300-word human-evaluation chunks and score HT versus agentic P3 MT with METRICX-QE, LITRANSPROQA, and mapped paragraph-level metrics.Oriented HT–MT deltas are positive when they favor HT and negative when they favor agentic/P3 MT; the comparison is summarized in Table 40.Source: H Automatic Evaluation · p. 24

5 What, if anything, stood out positively in the wording or style of this version?

  • The evaluation questionnaires ask readers about overall acceptability, smoothness, immersion, and willingness to continue reading a translation.The response scales distinguish degrees of acceptability and smoothness, immersion interference or support, and willingness ranging from definitely not to definitely yes.Source: H Automatic Evaluation · p. 36

6 What, if anything, stood out negatively in the wording or style of this version?

This section defines how readers’ negative comments were annotated and provides contextual examples of judgments about translation wording and style. The supplied material does not report specific negative aspects or their frequencies.

Annotation scheme

  • Negative comments were coded separately from positive comments, including negative aspects of the non-preferred translation in comparison and chunk-level judgments.The annotation scheme applies only negative codes to Single Q6 comments and to the non-preferred translation in comparative judgments.Source: Table 23 · p. 45

Contextual judgments

  • Contextual examples show readers evaluating wording against genre, period, cultural setting, character voice, and reading effort.The supplied caption identifies these factors but does not provide the individual negative examples or their frequencies.Source: Table 34 · p. 54

Related papers

Loading 2606.26040v1…