PaperlayerView exact revision on arXiv

Source-linked AI summary

Academic Writing with GPT-3.5: Reflections on Practices, Efficacy and Transparency

Oğuz 'Oz' Buruk

arXiv:2304.11079v1cs.CLcs.AIcs.HCcs.LG

TL;DR

The paper addresses limited evidence on how GPT-3.5 can support academic writing effectively and transparently. Through a personal experiment with text-davinci-003, it compares five writing methods and reflects on their ethical implications. Chunk Stylist and Talk Textualizer were used most successfully for retaining the author’s ideas and tone, while more generative methods required caution and substantial editing.

  • Problem

    The paper addresses limited evidence on GPT-3.5’s efficacy, writing applications, and transparent use in academic writing.

    Source: GPT-3 IN ACADEMIC WRITING · p. 2

  • Method

    The author conducts a personal experience experiment using text-davinci-003 across academic-writing tasks and documents prompts, outputs, edits, and revisions.

    Source: PROCEDURE FOLLOWED · p. 2, DISCUSSION · p. 7

  • Results

    Chunk Stylist and Talk Textualizer were used predominantly and were judged more effective for retaining the author’s tone and ideas than more generative methods.

    Source: DISCUSSION · p. 7, CONCLUSION · p. 8

  • Takeaways & Limitations

    Transparent disclosure of prompts, outputs, and transformations is important when GPT-3.5 strongly influences academic content, language, and tone.

    Source: DISCUSSION · p. 7

    The reflections are not generalizable across applications or disciplines, and the trial was limited in duration.

    Source: LIMITATIONS · p. 8

Abstract

from arXiv · show

The debate around the use of GPT 3.5 has been a popular topic among academics since the release of ChatGPT. Whilst some have argued for the advantages of GPT 3.5 in enhancing academic writing, others have raised concerns such as plagiarism, the spread of false information, and ecological issues. The need for finding ways to use GPT 3.5 models transparently has been voiced, and suggestions have been made on social media as to how to use GPT 3.5 models in a smart way. Nevertheless, to date, there is a lack of literature which clearly outlines how to use GPT 3.5 models in academic writing, how effective they are, and how to use them transparently. To address this, I conducted a personal experience experiment with GPT 3.5, specifically by using OpenAI text davinci 003 model, for writing this article. I identified five ways of using GPT 3.5: Chunk Stylist, Bullet to Paragraph, Talk Textualizer, Research Buddy, and Polisher. I reflected on their efficacy, and commented on their potential impact on writing ethics. Additionally, I provided a comprehensive document which shows the prompts I used, results I got from GPT 3.5, the final edits and visually compares those by showing the differences in percentages.

INTRODUCTION

GPT-3.5 is presented as a potentially useful aid for academic writing, especially for expressing ideas, but its use raises ethical concerns requiring responsible and transparent practices.

  • Academic interest in GPT-3.5 has grown rapidly since ChatGPT’s launch, including claims that it could be used in authorship roles.Source: INTRODUCTION · p. 1
  • GPT-3.5 may help researchers communicate ideas more fluently, including those with limited writing skills or whose first language is not English.Source: INTRODUCTION · p. 1
  • Researchers and journals have raised concerns about ownership, plagiarism, misinformation, and the need for human checking.Source: INTRODUCTION · p. 1
  • The experiment aims to clarify effective uses of GPT-3.5 and provide reflections on ethical use and transparency.Source: INTRODUCTION · p. 1

GPT-3 IN ACADEMIC WRITING

GPT-3.5 is a transformer-based language model used for generating and transforming text, but its academic applications remain contested because of accuracy, authorship, bias, and transparency concerns.

  • GPT-3.5 is a transformer-based NLP model trained on extensive text data to generate contextually coherent language.Source: GPT-3 IN ACADEMIC WRITING · pp. 1–2
  • Researchers have explored GPT models for scientific communication, literature navigation, concept definitions, and academic writing.Source: GPT-3 IN ACADEMIC WRITING · p. 2
  • Concerns include plagiarism, unclear responsibility and ownership, embedded bias, training-data permissions, labor exploitation, and environmental costs.Source: GPT-3 IN ACADEMIC WRITING · p. 2
  • The literature lacks studies showing GPT-3.5’s efficacy, practical writing uses, and transparent workflows for reviewers.Source: GPT-3 IN ACADEMIC WRITING · p. 2

PROCEDURE FOLLOWED

The paper uses a personal, non-generalizable experiment to document how GPT-3.5 supports academic writing and how its use can be made transparent.

  • The study examines GPT-3.5’s effects on writing speed, style, motivation, academic integrity, and transparency without claiming generalizability.Source: PROCEDURE FOLLOWED · p. 2
  • The author used text-davinci-003 in OpenAI Playground to generate material from prompts based on paragraphs, questions, bullets, and voice recordings.Source: PROCEDURE FOLLOWED · p. 2
  • Prompts, model outputs, edits, visual comparisons, trial notes, and an affinity diagram were documented before final manual copy-editing.Source: PROCEDURE FOLLOWED · p. 2

WAYS OF USING GPT-3.5 FOR ACADEMIC WRITING

The paper demonstrates five GPT-3.5 writing methods, ranging from transforming researcher-generated material to generating or polishing text, with varying degrees of editing and control.

Chunk Stylist

Research Buddy

Bullet-to-Paragraph

Talk Textualizer

Polisher

DISCUSSION

The author's experience suggests GPT-3.5 can improve motivation, focus, and writing flow, but its efficiency, output quality, agency, ecological effects, and transparency depend on how it is used and reviewed.

Writing experience

  • GPT-3.5 made writing smoother and increased motivation, but the author became less time-efficient because substantial revisions and reporting were required.The introduction took about 10 minutes to draft but about 90 minutes to revise, and documenting comparisons required roughly five additional hours.Source: DISCUSSION · p. 6

Writing quality

  • Output quality varied across sections, and several additional revision rounds were needed to achieve consistent language style and narrative flow.The manuscript was intentionally left after limited editing to demonstrate what could be produced in about a day.Source: DISCUSSION · p. 7

Ways of Using Methods

  • Chunk Stylist and Talk Textualizer were preferred because they formalized the author's ideas, whereas Research Buddy and Bullet to Paragraph required repeated attempts and substantial factual editing.The author removed most of a generated GPT-3 description after finding insufficient supporting information for its claims.Source: DISCUSSION · p. 7

Transparency, Agency and Biases

  • Differences between source prompts, GPT-3.5 outputs, and final edits revealed variable authorial agency, making transparent documentation of the writing process important.Across the full text, the difference from original prompts to outputs was 70.27%, while the difference from outputs to final edits was 29.13%; individual chunks ranged from 3.3% to 98.54% change.Source: DISCUSSION · p. 7

Concerns of Ecological Sustainability

  • The author reports a tension between GPT-3.5's potential ecological costs and possible resource and wellbeing benefits from a more focused writing process.The article involved approximately 500 requests and an estimated cost of about $2.5, while the carbon footprint was unknown; the author also links reduced distraction and writing pressure to possible benefits.Source: DISCUSSION · pp. 7–8
  • Responsible use should minimize unnecessary prompts and extend beyond efficiency to consider resource use, academic wellbeing, and ecological responsibility.The author argues that tools could compare a GPT-assisted text's carbon footprint with a non-GPT alternative.Source: DISCUSSION · p. 8

Additional tools that can help with GPT-based academic writing

  • Proposed future tools should document prompts, outputs, edits, carbon effects, and factual reliability rather than merely watermarking GPT-generated text.The author suggests transparent editors, carbon-footprint comparisons, and systems that verify facts and references because current GPTs are not capable of reliably providing them.Source: DISCUSSION · p. 8

LIMITATIONS

The study’s reflections are shaped by a single researcher’s perspective, a short trial, and relatively practical writing content, limiting generalizability across disciplines and contexts.

  • The reflections are not generalizable across the board because they derive from one design researcher’s perspective and positionality.The author considers them potentially more relevant to applied-science fields where expressing ideas is important.Source: LIMITATIONS · p. 8
  • The trial was limited in duration, so longer engagement might reveal additional ideas, practices, and opinions.Source: LIMITATIONS · p. 8
  • GPT-3.5 may be less useful for fields requiring complex thoughts expressed in precise or abstract language because the paper’s content was relatively practical.Source: LIMITATIONS · p. 8

CONCLUSION

The paper reflects on GPT-3.5 methods for academic writing, transparency, and ethics. It reports especially positive experiences with Chunk Stylist and Talk Textualizer while urging caution with more spontaneous methods.

  • Chunk Stylist and Talk Textualizer enabled more effective writing while retaining the author’s tone and ideas.Source: CONCLUSION · p. 8
  • The other methods require caution because greater spontaneity and randomness could contribute to plagiarism, false information, or hateful rhetoric.Source: CONCLUSION · p. 8
  • Documenting prompts, outcomes, and final edits in a supplementary document provides an example of transparent GPT-3.5 use.Source: CONCLUSION · p. 8
  • The paper aims to support researchers considering GPT-3.5 and encourage tools and methods for more responsible AI-supported academic writing.Source: CONCLUSION · p. 8

Appendix-1

The appendix documents the paper’s GPT-3.5 writing workflow, including prompts, generated text, revisions, and visual comparisons. It presents motivations, ethical concerns, five use methods, and the author’s stated contributions.

  • The study addresses a literature gap concerning how GPT-3.5 can be used in academic writing, how effective it is, and how to use it transparently.Source: Appendix-1 · p. 11
  • The appendix identifies five GPT-3.5 uses: Chunk Stylist, Bullet-to-Paragraph, Talk Textualizer, Research Buddy, and Polisher.Source: Appendix-1 · pp. 11–12
  • The author presents a personal experience experiment using OpenAI’s text-davinci-003 model to examine writing practices, efficacy, and ethical implications.Source: INTRODUCTION · p. 1, Appendix-1 · p. 12
  • The appendix records prompts, GPT-3.5 results, final edits, and percentage-based visual comparisons between versions.Source: Appendix-1 · pp. 12–13
  • GPT-3.5 is motivated as a possible aid for researchers facing writing barriers, including limited writing skills or limited English fluency.Source: Appendix-1 · pp. 14–16
  • The appendix also documents concerns about content ownership, plagiarism, misinformation, and the need for human checking and responsible use.Source: Appendix-1 · pp. 15–16

4 Some writers added ChatGPT as a co-author to papers

The appendix notes that GPT-3.5 became popular among writers and researchers after ChatGPT’s release and that some writers added ChatGPT as a paper co-author.

  • GPT-3.5 attracted academic-writing interest for varied purposes after ChatGPT’s release, and some writers added ChatGPT as a paper co-author.Source: §4 · p. 17

8 The ability to communicatinge scientific material in a paper

GPT-3 is presented as potentially helping researchers communicate scientific ideas, particularly when writing skill, English fluency, hesitation, or speed create barriers. However, its use raises concerns about content ownership, misinformation, and the need for effective and responsible guidance.

8 The ability to communicatinge scientific material in a paper

  • Writing difficulties, hesitation, and slow processes can prevent researchers from formalizing their research ideas and projects.Source: §8 · p. 17
  • GPT-3 may help researchers with limited writing skills or English fluency express scientific ideas more clearly.The passages frame this as a potential way to reduce communication barriers and equalize differences in writing ability.Source: §8 · p. 17
  • GPT-3 has raised concerns among scientists, journals, and institutions, which have issued principles or guidelines for its use.The concerns are presented alongside broader questions about how such tools should be employed in research and education.Source: §8 · pp. 17–18
  • Ethical issues include content ownership and the possibility that unchecked tool output could introduce wrong information into scientific writing.The passages associate these risks with inattentive or careless use and insufficient human checking.Source: §8 · p. 18
  • The paper responds to the need for responsible GPT-3 use by reflecting on a personal experience experiment and describing ways to incorporate the tool into writing.It aims to discuss efficacy and methods for using GPT-3 efficiently, ethically, and responsibly.Source: §8 · p. 18

30 P, possible different ways tof using it

This perspective piece reports a personal, short-term experiment using GPT-3.5 across several academic-writing methods, documenting prompts, outputs, edits, and reflections to examine efficacy, applications, and transparency. It considers the methods’ practical value alongside unresolved ethical and ecological concerns, while emphasizing substantial variability, the need for human revision and agency, and limited generalizability.

  • GPT-3.5 may help researchers overcome writing barriers and reduce disadvantages for those with limited writing skills or who are not native English speakers.The motivation identifies communication difficulties, hesitation, slow writing, and language background as barriers that the tool may help address.Source: §30 · p. 18
  • The discussion situates responsible use within concerns about content ownership, plagiarism, misinformation, embedded bias, exploited data-labeling workers, training-data permissions, and environmental costs.The author explicitly notes that the piece does not provide definitive answers to bias, worker exploitation, or ecological sustainability concerns.Source: §30 · pp. 19, 23, 26
  • The experiment used OpenAI’s Playground rather than ChatGPT, specifically text-davinci-003, to provide greater control over output randomness through temperature settings.The author states that both interfaces use the same training models but selected the Playground for its additional control.Source: §30 · p. 19
  • The piece aims to clarify how GPT-3.5 can facilitate effective academic writing, identify possible uses, and support ethical use and transparency.It presents a personal experience experiment reflecting on efficacy, writing-process integration, and responsible methods.Source: §30 · p. 19
  • The work responds to a lack of studies demonstrating GPT-3.5’s efficacy in writing, different ways of incorporating it, and workflows that reviewers can consider transparent.The stated gap concerns practical efficacy and transparent workflow demonstrations rather than only debates about GPT-3.5’s benefits and harms.Source: §30 · pp. 23, 24, 26
  • The study addresses a gap in evidence about GPT-3.5’s writing efficacy, possible uses, and transparent workflows for academic reviewers.It does not claim to resolve broader concerns about bias, worker exploitation, or ecological sustainability.Source: §30 · pp. 28–30
  • GPT-3 use remains controversial because reported concerns include plagiarism, opaque authorship, embedded bias, exploited data-labeling workers, training-data permissions, and environmental costs.The cited discussion also questions private-company control and notes that models can be prompted to produce radicalized content despite safeguards.Source: §30 · pp. 28–29
  • The personal experience study explores using GPT-3.5 to produce an academic article efficiently while maintaining academic integrity and transparency.It reflects on effects on writing speed, style, and motivation, without aiming to generalize across all applications or ethical considerations.Source: §30 · pp. 31–32
  • The workflow used OpenAI’s text-davinci-003 model across paper sections, with prompts based on rough paragraphs, questions, bullet points, commands, or voice recordings.The author also consulted prompt-writing guidance and performed a final manual copy-edit to support narrative flow.Source: §30 · p. 34
  • Transparency was documented by preserving prompts, GPT-3 outputs, edits, trial notes, and an affinity diagram, with supplementary material showing the writing process.Recorded trial notes included time spent selecting outputs, retries, feelings while writing, and reflections on results.Source: §30 · pp. 33–34

Writing process and transparency

  • The author stored prompts and outputs, recorded retries and reflections, compared revisions visually, organized the data thematically, and completed a final manual copy-edit.This documentation included supplementary materials showing the original prompt, generated output, and final edit.Source: §30 · pp. 36–37

Chunk Stylist

  • Chunk Stylist generated text from rough passages, allowing faster drafting while temperature settings controlled predictability versus improvisation.The method was used for the introduction, background, and this section.Source: §30 · pp. 39–40

Research Buddy

  • Research Buddy helped generate ideas and arguments, but produced inaccurate or overly broad content that required critical evaluation and manual correction.The method was used for the first paragraph of Section 2 and this section.Source: §30 · p. 41

Bullet-to-Paragraph

  • Bullet-to-Paragraph converted listed principles or procedural steps into flowing academic prose and was used for this section and the second Method paragraph.The approach was described as similar to Chunk Stylist but began with bullet points.Source: §30 · p. 43

Talk Textualizer

  • Talk Textualizer transformed monologues, including pauses and stuttering, into readable text before converting it into academic writing.It formed the basis of the discussion, limitations, and conclusion.Source: §30 · p. 45

Polisher

  • Polisher treated complete, well-structured material as copy-editing input, correcting grammar and spelling while preserving the original text more closely.Lower temperature settings were suggested when minimal changes were desired, and the method was also used to polish edited outputs.Source: §30 · p. 48
  • Polisher uses GPT-3 like a copy editor to correct grammar and spelling and improve clarity in already complete text with a clear narrative flow.Lower temperature settings can produce changes closer to the original text, and the method was also used to polish edited outputs elsewhere in the piece.Source: §30 · p. 49

Reflections on using GPT-3.5 for Writing Articles

  • GPT-3 produced a largely positive effect on motivation and productivity, making writing and thinking more fluid, faster, and efficient than usual.The experience involved an approximately three-week incubation period before writing, during which attention shifted from writing mechanics toward representing ideas.Source: §30 · p. 51
  • For a design-trained academic, GPT-3.5 provided freedom to focus on ideas rather than the burden of manual writing, while basic writing skills remained necessary for coherent narratives.The author compared GPT-3.5 with CAD software, 3D printers, and other prototyping tools used to realize and conceptualize artifacts.Source: §30 · pp. 56, 58, 59
  • The writing process became more manageable and less distracting, although producing a flowing, understandable narrative still required multiple writing practices and editing.This qualification is especially relevant to the author’s account of using GPT-3.5 to textualize ideas and streamline manual writing.Source: §30 · pp. 56, 59, 60
  • GPT-3.5 lightened the burden of manual writing and enabled greater focus on ideas, but the author still needed basic writing skills and experienced fewer distractions rather than a fully automated process.The author compared GPT-3.5 with computational design tools and reported that it helped streamline writing.Source: §30 · p. 61
  • 10 minutes of drafting an introduction was followed by approximately 90 minutes of GPT-3 revisions, showing that GPT-3.5 was not a quick writing solution despite reducing distractions.The author also spent around 5 more hours creating a comparison report and described GPT-3.5 as a tool for more effective writing and new writing techniques.Source: §30 · pp. 63–64
  • The manuscript’s quality varied across sections, with some passages plain and unengaging and others easier to read, so several further revisions were needed for consistent tone and narrative flow.The manuscript was intentionally left as a demonstration of what GPT-3.5 could produce in a short time with minimal editing.Source: §30 · pp. 66–67

Ways of Using Methods

  • The author primarily used Chunk Stylist and Talk Textualizer to formalize personal ideas, while Bullet to Paragraph and Research Buddy required several attempts and substantial editing to avoid factually unsupported sentences.The author used Polisher mainly to refine edited results, though proficient writers could also use it for copy-editing.Source: §30 · p. 69

Transparency, Agency and Biases

  • Minor copyediting may require only an acknowledgement, but methods that substantially alter content, language, or tone require more substantial reporting, such as an external link or appendix.The author documented prompts, results, and final edited versions and called for tools that show heavy modifications and the sources used to generate text.Source: §30 · pp. 72–73
  • 70.45% was the difference between original prompted texts and GPT-3.5 outcomes, while 28% was the difference between outcomes and final edited versions.The biggest original-to-outcome difference was 98.54%, while outcome-to-final-edit differences ranged from 3.3% to 71.44%.Source: §30 · p. 73

Transparency

  • Authors should document prompts, outputs, and final edits, using acknowledgements for minor copyediting but appendices or external links for heavier methods such as Talk Textualizer or Bullet to Paragraph.Such reporting should help reviewers assess modifications and ensure that generated content does not unintentionally consolidate racial biases or radicalized political ideas.Source: §30 · pp. 73–75

Visual Comparisons of Ways of Using Methods / Whole Section

  • GPT-3.5 changed original prompted text by 70.45% overall; outcome-to-final-edit differences were 28%, with maximum differences of 98.54% and 71.44%, and a minimum of 3.3%.These comparisons are documented in the supplementary report.Source: §30 · p. 75

Ecological Sustainability

  • Approximately 500 requests and repeated reprompting cost about $3, while GPT-3.5 may increase energy consumption but also reduce distraction, resource use, overwork, and potentially improve academic wellbeing.The author recommends using as few prompts as possible while considering benefits beyond writing efficiency.Source: §30 · pp. 76–77

Additional Tools That Can Help with GPT-Based Academic Writing

  • Future tools should visualize prompts, outputs, and edits; compare writing carbon footprints; detect faulty information and authoritative tone; and generate reliable references rather than relying on binary watermarks.Watermarks may help classroom assignments but may be less meaningful for academic writing, where current GPT systems can improvise titles or URLs.Source: §30 · pp. 82–83

Limitations

  • Because this perspective reflects one design researcher’s short-term, subjective trial, its proposed uses may not generalize across disciplines, especially fields requiring highly precise or abstract language.The writing was relatively practical and did not require substantial philosophical scrutiny.Source: §30 · p. 85
  • Because the trial was personal, short-term, practical, and shaped by a design-researcher perspective, its reflections are not generalizable and may be less useful for precise or abstract fields.Source: §30 · pp. 86–87
  • Five methods were identified: Chunk Stylist, Research Buddy, Talk Textualizer, Bullet-to-Paragraph, and Polisher.Source: §30 · p. 91
  • Chunk Stylist and Talk Textualizer were used predominantly because they improved writing effectiveness while retaining the author’s tone and ideas.Source: §30 · p. 91
  • The other methods require caution because their spontaneity and randomness could contribute to plagiarism, false information, or hateful rhetoric.Source: §30 · p. 91
  • The paper documents prompts, outcomes, and final edits to demonstrate transparent GPT-3.5 use and support more responsible academic-writing practices.Source: §30 · p. 89
Loading 2304.11079v1…