The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers

1 INTRODUCTION

• Parallel phrase suggestions may offer composition ideas but take time to read, creating a trade-off between efficiency and ideation.The authors use ideation to mean coming up with what to write next and how to write it.Source: §1 · p. 2
• The study asks how different numbers of parallel suggestions affect input and text composition by native and non-native English writers.Source: §1 · p. 2
• The number of parallel suggestions had not yet been empirically studied, and language proficiency had not been examined for phrase suggestions.Prior work identified suggestion count as a design factor.Source: §1 · p. 2
• The authors built a GPT-2 email editor, refined it in a 30-person prestudy, and logged interactions in an online email-writing study with 156 participants.Source: §1 · p. 2
• The analyses report benefits and costs of multiple phrase suggestions, differences in suggestion use by language background, and further behavioral insights.Source: §1 · p. 2
• The authors release the model, data, and study materials to support research toward augmenting human writers with AI rather than replacing them.Source: §1 · p. 2

2.1 Interpreting and Correcting Typing Input

• Language models have long been used to interpret and correct typing input, whereas this paper focuses on neural-model phrase prediction during desktop typing.Prior examples include T9, stylus input, smartphone keyboards, and autocorrect.Source: §2.1 · p. 3

2.2 Predicting Text Input

• Text prediction ranges from single-word suggestions to phrase or reply suggestions, while this study examines how such features affect input and composition.The authors compare native and non-native writers because composition and effort may differ by language proficiency.Source: §2.2 · p. 3

2.3 Perception and Impact of Suggestions

• Prior work found that users read phrases as content or expression suggestions, unlike single-word predictions; this motivates investigating efficiency versus inspiration.This paper studies user behavior and perception in desktop email writing, rather than mobile use.Source: §2.3 · p. 3

2.4 Generating Text and Interactive NLP

• Interactive NLP research includes artistic text-generation tools, but empirical analyses of language-model use and interface design remain limited.The authors position their work as an HCI analysis of neural language models as suggestion providers and their effects on user behavior.Source: §2.4 · p. 4

2.5 UI Design Factors for Text Suggestions

• Suggestion count is a recognized interface design factor, yet prior systems and simulations generally fixed their counts and empirical study of the factor was lacking.The paper investigates this factor in relation to the trade-off that reading more suggestions takes time but may provide more potentially inspiring content.Source: §2.5 · p. 4

2.6 Writing Research

• Writing research on cognition and HCI text-entry research have few cross-references, although text generators may affect idea generation, expression, and revision.The authors study native and non-native English writers and analyze interaction logs, given differences in writing processes and cognitive effort.Source: §2.6 · p. 4

3.1.1 Main GUI.

• The email composition web app presents a task scenario and email field, with greetings and headers prefilled and a nickname-based signature.Source: §3 · p. 5, §3.1.1 · p. 5

3.1.2 Interactions.

• Users can type, select, edit, delete, and move the caret, and can select suggestions by mouse or keyboard.Keyboard controls use Tab for the first suggestion, arrows to change selection, and Enter to accept.Source: §3.1.2 · p. 5

3.1.3 Suggestions.

• Suggestion lists appear at the caret 250 ms after a keypress; users accept a suggestion by selecting it to add it to the text.The popup sits below the current line to distinguish suggestions from the writer’s text.Source: §3.1.3 · p. 5

3.2 Backend and Deployment

• The prototype logs data through a Python backend, with the logs stored in ElasticSearch and deployment on AWS ECS.Source: §3.2 · p. 5

3.3 Language Model

• The model is pretrained GPT-2, finetuned for email on a filtered ENRON dataset of 236,206 messages with names and dates replaced by placeholders.Source: §3.3 · p. 5
• Beam search generates suggestions at twice the requested number of beams, with a 1.5 repetition penalty; suggestions are ordered by likelihood, not optimized as a set.Source: §3.3 · p. 5

4.1 Study Design and Apparatus

• The prestudy used three business-email scenarios designed to allow participants to include their own ideas, while collecting interaction logs and task and final questionnaire responses.Questionnaires covered preferred suggestion settings, interaction preferences, and perceptions of authorship, phrase use, and daily use.Source: §4.1 · p. 6

4.2 Participants and Procedure

• Thirty participants were recruited online and completed two study parts on their own laptop or desktop in about 10 to 15 minutes.Their mean age was 26 years, and 23 reported English proficiency of C1 or higher.Source: §4.2 · p. 6

4.2.1 Part 1: Fixed Settings.

• In the fixed-settings part, participants used four suggestions of up to four words, then completed a task questionnaire.Participants were randomly assigned to one scenario and then another different scenario.Source: §4.2.1 · p. 6, §4.2.2 · p. 6

4.2.2 Part 2: Customisable Settings.

• In the customizable part, participants could explore up to 20 suggestions and 15 words per suggestion, with settings available during the task.Source: §4.2.2 · p. 6

4.3.1 Logging.

• Emails averaged 49.22 words, including 14.53 words from suggestions; participants triggered 3,042 lists and accepted 455 suggestions.Mean suggestion loading time was 1,142 ms, and mean selection time was 4,344 ms.Source: §4.3.1 · p. 7

4.3.2 Subjective Feedback.

• After the fixed-settings part, preferred counts were a median of three suggestions and three words; after customization, preferences were four suggestions and three words.Source: §4.3.2 · p. 7
• Twelve of 30 participants said suggestions gave them ideas, while common criticisms were long loading times (13) and insufficient suggestion quality (11).Participants generally rated authorship and use of other phrases positively, with daily use rated between neutral and agree.Source: §4.3.2 · p. 7

4.3.3 Summary and Learnings.

• The authors interpreted the feedback as encouraging for ideation, while roughly four-second selection times suggested users were not primarily optimizing efficiency.After the prestudy, they moved the model to AWS SageMaker to speed suggestions and added the email subject as model context to improve relevance.Source: §4.3.3 · p. 7, §4.4 · p. 7

5.1 Study Design and Apparatus

• The within-subject study varied parallel suggestions across 0, 1, 3, and 6, and compared native with non-native English speakers.The study logged a range of measures, and the four suggestion conditions were crossed with proficiency as the between-subject factor.Source: §5.1 · p. 7

5.1.1 Web App and Scenarios.

• The web app used the improved prototype, four-word suggestions, four email scenarios, and a ten-word minimum, with no minimum writing time.The three prestudy scenarios were reused and a fourth was added to match the four-condition design.Source: §5.1.1 · p. 8

5.1.2 Questionnaires.

• Questionnaires followed each email task and asked about suggestion effects on email content and wording, alongside prestudy questions and open feedback.A final questionnaire was also included.Source: §5.1.2 · p. 8

5.2 Participants and Procedure

• Of 162 recruits, six were excluded for composing externally, leaving 156 participants who completed four counterbalanced email tasks and questionnaires.Scenarios and suggestion conditions were counterbalanced with two orthogonal 4 × 4 Latin squares, covering all combinations and task positions.Source: §5.2 · p. 8

6.1 Dataset Overview

• Emails with suggestions averaged 45.95 words and took 3.50 minutes to write; participants accepted at least one suggestion in 411 of 468 such emails (87.82%).The prototype's mean suggestion-list loading time was 361 ms, and 3,920 suggestions were accepted overall.Source: §6.1 · p. 9

6.2.1 Suggestion Acceptance Rate.

• Showing three or six suggestions significantly increased acceptance chance by 49% in each condition; language proficiency alone was nonsignificant, but non-native speakers had larger increases than native speakers (+36% at three; +92% at six).Source: §6.2.1 · p. 9

6.2.2 List Selection.

• Selection probability declined lower in the list: for three suggestions, selection rates were 7.3%, 5.6%, and 4.0% from first to third position.For six suggestions, rates were 5.8%, 4.1%, 2.8%, 2.2%, and 1.6% for positions one through six, with positions five and six sharing the same rate.Source: §6.2.2 · p. 9

6.2.3 Suggestion Modification Rate.

• Six suggestions reduced the chance of modifying an accepted suggestion by 65% compared with one; the three-suggestion estimate was not significant.Modification rates descriptively decreased from 0.26 with one suggestion to 0.20 with three and 0.18 with six.Source: §6.2.3 · p. 9

6.2.4 Used Suggestions Word Count.

• Three and six suggestions significantly increased suggested words in email text by 51% and 27%, respectively.Non-native proficiency was also associated with an estimated 51% increase in suggested words; interactions were not significant.Source: §6.2.4 · p. 10

6.3.1 Task Time.

• Task time was significantly longer with six suggestions than with none or one; mean times were 196 s, 188 s, 209 s, and 247 s for 0, 1, 3, and 6 suggestions.The six-versus-zero and six-versus-one comparisons were significant; interactions were not significant.Source: §6.3.1 · p. 10

6.3.2 Search & Selection Time.

• Search and selection time increased with list size: the model estimates were 1,543 ms for three suggestions and 2,510 ms for six.Language proficiency and its interaction with suggestion count were not significant for this measure.Source: §6.3.2 · p. 10

6.4.1 HMM Analysis.

• The analysis grouped keyboard activity into five actions and used an HMM to infer higher-level behaviour states from action sequences.Eleven states were selected using log-likelihood and BIC, then the HMM was fitted to all action sequences.Source: §6.4 · p. 10, §6.4.1 · pp. 10–11

6.4.2 Intepreting Behaviour Patterns.

• The fitted model summarized sequential behaviour, and the analysis described nine patterns spanning typing, suggestion use, navigation, and revision.The figure shows action emission distributions, state transitions, and decoded state frequencies; the table lists the states involved in each pattern and the interpretation derived from state parameters.Source: Figure 3 · p. 11, Table 2 · p. 11, §6.4.2 · p. 12
• The behaviour patterns are exploratory rather than confirmatory, although similar behaviours appeared across repeated fits with random initializations.The authors checked interpretations against sequences assigned to states and compared repeated HMM fits for local minima.Source: §6.4.2 · p. 12

6.4.3 Summary of Patterns.

• Patterns ranged from focused manual typing, where suggestions were mostly ignored or not triggered, to sporadic or heavy use, including accepting multiple suggestions in a row.Composing varied both between users and within users over the course of writing; dismissals did not always mean users attended to the list.Source: §6.4.3 · p. 12

6.5 Suggestions and Text Content

• Birthday emails used “lovely” 14 times as often without suggestions, while “hope you are doing well” 4–12 times as often with them.Source: §6.5 · p. 12
• Interview emails used prior-contact phrases 10–12 times as often without suggestions, while “let me know if you have any questions” appeared 3–10 times as often with them.Source: §6.5 · p. 12
• “Please let me know” was 10–20 times more common with suggestions in the ID Card and Reference scenarios.Source: §6.5 · p. 12
• Suggestion-associated patterns suggest that suggested phrases might replace other wordings.The examples are study-specific.Source: §6.5 · p. 12

6.6 Perception and Subjective Feedback

• Many perception questions received mixed responses, suggesting individual differences in use, perceptions, and preferences.Source: §6.6.1 · p. 13

6.6.1 Perception of Self, Task and Suggestions (Q1-7).

• Non-native speakers were more likely to report using other phrases or words due to suggestions, though responses overall leaned toward disagreement.Source: §6.6.1 · p. 13
• People who accepted more suggestions rated perceived authorship lower (Kendall’s τ=-.28, p<.0001), although authorship ratings were “agree”/4 for both groups.Source: §6.6.1 · p. 13
• Native speakers were more likely to perceive writing as easier, while both groups rated it as rather easy; non-native speakers were less likely to feel authorship with suggestions.Source: §6.6.1 · p. 13

6.6.2 Opinion on Suggestion Settings (Q8-9).

• Three suggestions were “just right” for 62% of people; one was more often rated too few and six too many, though many still rated those settings just right.Source: §6.6.2 · p. 13
• Suggestion length was generally rated “just right,” with a slight indication that some users wanted a single suggestion to be longer.Source: §6.6.2 · p. 13

6.6.3 Open Feedback.

• Positive feedback most often cited good suggestions (38.5%), helpfulness (24.4%), and inspiration (22.4%).Usability (20.5%) and efficiency (15.4%) were also mentioned; nine people (5.8%) stated nothing positive.Source: §6.6.3 · p. 13, §7.1 · p. 14
• Suggestion quality was the most-mentioned improvement area (25%), alongside better attention to email context and topic (12.8%).Twelve people (7.7%) wanted shorter delays or faster loading, and eight (5.1%) mentioned distraction; 36.5% stated there was nothing to improve.Source: §6.6.3 · p. 13

7.1 Multiple Suggestions Help to Find Phrases

• More suggestions increased acceptance, especially for non-native speakers, and reduced manual modification; choosing multiple suggestions (>1) added suggestion words to emails without increasing email length.Subjective feedback showed 62% found three suggestions just right, compared with 44% for six.Source: §7.1 · pp. 13–14

7.2 User Engagement with Suggestions Varies

• Users ranged from typing without suggestions to chaining them, edited 21.3% of accepted suggestions, and displayed nine previously undescribed behavior patterns.The patterns provide a starting point for replication and motivate support for revision as well as suggestion use.Source: §7.2 · p. 14

7.3 Suggestions Cost Time and Actions

• Suggestions cost time and actions: each additional four-word phrase beyond the first added 610 ms, while focused typing fell from 80% to about 50% of keyboard actions.The time estimate was fitted to mean search and selection times (R^2=.98); other actions included navigating, selecting, reading, or dismissing suggestions.Source: §7.3 · p. 14

7.4 Users Consider More than Efficiency

• Beyond efficiency, 22.4% praised inspiration, 9.0% reported help when stuck, and 49.3% and 41.3% perceived effects on wording and content, respectively.Open comments viewed those effects positively, and the study provides evidence of perceived and actual use of multiple phrase suggestions for desktop email ideation.Source: §7.4 · pp. 14–15

7.5 Language Proficiency Matters

• Non-native speakers accepted and used more suggestions, gained relatively more from parallel options, faced less temporal overhead, and were slightly more positive about helpfulness and influence.The authors offer this comparison and dataset as a first motivating basis for further study of native and non-native phrase-suggestion use.Source: §7.5 · p. 15

7.6 Implications for Research and Design

• Design explorations include making phrase sets easier to compare, offering semantic categories, testing direct or filtered navigation, and comparing utility-based or explicit suggestion triggers.Possible navigation methods include number- or gaze-based selection; explicit triggers could use a key such as Tab.Source: §7.6.1 · p. 15, §7.6.2 · p. 15, §7.6.3 · p. 15
• The authors motivate adaptive interfaces that respond to current behavior or user control, and phrase generation that optimizes variety and utility across a set rather than only the most likely phrase.Some participants explicitly wanted more variety in meaning, not just wording.Source: §7.6.4 · p. 16, §7.6.5 · p. 16

7.7 Limitations & Reflections on Methodology

• The online composition study used typical business cases, but novelty and practice may have affected behavior; writing and suggestion-selection times fell across four tasks.Writing time decreased from M=260 s on task one to M=181 s on task four; suggestion-selection time decreased from M=4309 ms to M=3656 ms. The authors call for longer-term, in-the-wild comparisons.Source: §7.7 · p. 16
• The rich dataset supports text composition as a useful study task beyond text-copy tasks.The discussion also points to linguistic measures such as pauses and bursts as a possible research interest.Source: §7.7 · p. 16

7.8 Broader Reflections on Text Suggestions

• Despite prestudy and finetuning, some suggestions were unsuitable.People identified grammar, topic, and formality as areas for improvement.Source: §7.8.1 · p. 16

7.8.1 Quality.

• Quality improvements could compare GPT-3 [12] with this study’s results and assess communication quality as perceived by email receivers.The discussion notes that emailing may be more goal-directed than chat messaging, potentially shaping users’ needs and preferences.Source: §7.8.1 · p. 16

7.8.2 Bias.

• An efficiency-focused interface may itself introduce bias by limiting supporting effects for users with lower language proficiency.The study found that at least two participants saw inappropriate words, motivating further work on language-model output and its presentation to end users.Source: §7.8.2 · pp. 16–17

7.8.3 Learning vs Deskilling.

• Suggestions may help foreign-language writers with phrasing, wording, and spelling, but could also harm writing skills or limit creativity.Long-term studies could test whether any proficiency gains persist when writing without suggestions; the discussion frames these tensions as a reason to develop interactive AI interdisciplinarily.Source: §7.8.3 · p. 17

• The study was the first to compare different numbers of parallel phrase suggestions for email writing by native and non-native English writers.Source: §8 · p. 17
• Multiple phrase suggestions supported ideation at an efficiency cost for both writer groups, with non-native speakers seeming to benefit more.Source: §8 · p. 17
• The findings motivate exploring a wider variety of system and UI parameters and developing AI systems that augment rather than replace human writers.Source: §8 · p. 17
• The study identified nine fundamental behaviour patterns related to text input, navigation, revision, and suggestion use.Source: §8 · p. 17