Source-linked AI summary
How is ChatGPT's behavior changing over time?
Lingjiao Chen, Matei Zaharia, James Zou
TL;DR
Because GPT-3.5 and GPT-4 are updated opaquely, this paper evaluates their March and June 2023 versions across diverse tasks and metrics. It finds substantial, task-dependent behavioral drift, including reduced GPT-4 instruction following, and argues for continuous monitoring while acknowledging incomplete benchmark coverage.
Problem
Opaque updates to widely used GPT-3.5 and GPT-4 make it unclear how releases affect behavior, complicating stable integration and reproducibility.
Method
The authors compare March and June 2023 versions of GPT-3.5 and GPT-4 across diverse tasks using task-specific and common behavioral metrics.
Results
Both services showed substantial, task-dependent drift across releases, while GPT-4’s ability to follow many user instructions decreased markedly.
Takeaways & Limitations
The findings highlight the need to continuously monitor LLM behavior when services are used in applications and workflows.
Takeaways & Limitations
The selected benchmark datasets do not comprehensively cover ChatGPT’s complex behaviors, so the study is not a holistic assessment.
Abstract
from arXiv · showhide
GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on several diverse tasks: 1) math problems, 2) sensitive/dangerous questions, 3) opinion surveys, 4) multi-hop knowledge-intensive questions, 5) generating code, 6) US Medical License tests, and 7) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was reasonable at identifying prime vs. composite numbers (84% accuracy) but GPT-4 (June 2023) was poor on these same questions (51% accuracy). This is partly explained by a drop in GPT-4's amenity to follow chain-of-thought prompting. Interestingly, GPT-3.5 was much better in June than in March in this task. GPT-4 became less willing to answer sensitive questions and opinion survey questions in June than in March. GPT-4 performed better at multi-hop questions in June than in March, while GPT-3.5's performance dropped on this task. Both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. We provide evidence that GPT-4's ability to follow user instructions has decreased over time, which is one common factor behind the many behavior drifts. Overall, our findings show that the behavior of the "same" LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLMs.
1 Introduction
The study examines whether GPT-3.5 and GPT-4 change across releases, finding substantial, task-dependent drift and a marked decline in GPT-4’s instruction following. These changes complicate reliable LLM integration and reproducibility, motivating continuous monitoring.
- Implications: Opaque model updates can break downstream workflows when accuracy or formatting changes and can make results from the same LLM difficult to reproduce.These risks arise because users cannot reliably determine when or how the services are updated.
- Motivation: GPT-3.5 and GPT-4 were evaluated across diverse tasks because their updates and behavioral effects are opaque.The study covers math, sensitive questions, opinion surveys, multi-hop questions, code, medical exams, and visual reasoning.
- Findings: Performance and behavior varied significantly between the March and June 2023 releases, with some tasks worsening and others improving.The evaluation compared the two releases of both services across the selected tasks.
- Findings: GPT-4’s ability to follow user instructions decreased substantially from March to June, including failures on simple response-format instructions.The authors identify instruction following as a common factor that partially explains corresponding performance drops.
- Implications: The findings support continuously monitoring LLM behavior to improve the reliability of applications built on these services.The authors release prompts, responses, analysis, and visualization code to support further monitoring research.
2 Overview: LLM Services, Tasks and Metrics
The paper compares GPT-4 and GPT-3.5 across two releases using diverse, relatively objective tasks and task-specific performance measures. It also uses verbosity and mismatch as common complementary metrics, while acknowledging that the benchmark set is not comprehensive.
- LLM Services: The monitored services are GPT-4 and GPT-3.5, comparing API versions snapshotted in March 2023 and June 2023.The services were selected because they are widely adopted by individual users and businesses.
- Evaluation Tasks: The evaluation covers eight tasks spanning math, sensitive questions, opinion surveys, multi-hop question answering, code, USMLE, and visual reasoning.Math includes two problem types, making eight task settings overall.
- Evaluation Tasks: The tasks were selected for diversity, frequent use in LLM evaluation, and relatively objective assessment.Queries were sampled from existing datasets or constructed by the authors.
- Scope: The benchmark datasets do not comprehensively cover ChatGPT’s complex behaviors, so the study demonstrates drift rather than providing a holistic assessment.The authors plan to add more benchmarks in future long-term evaluations.
- Metrics: Accuracy measures math and USMLE correctness, response rate measures direct answers to sensitive and opinion questions, and executable-code fraction measures code generation.Each task uses a main metric tailored to its application scenario.
- Metrics: Verbosity measures generation length, while mismatch measures how often extracted answers differ between two versions for the same prompt.Mismatch compares extracted answers rather than raw generations.
3 Monitoring Reveals Substantial LLM Drifts
Across math, safety, and reasoning tasks, GPT-3.5 and GPT-4 showed substantial behavioral drift between March and June 2023. Changes in chain-of-thought adherence, answer biases, verbosity, and willingness to respond helped explain these shifts.
- Math I: Prime vs Composite: GPT-4’s prime-testing accuracy fell from 84.0% in March to 51.1% in June, while GPT-3.5’s rose from 49.6% to 76.2%.GPT-4’s average response length also dropped from 638.3 to 3.9 characters, whereas GPT-3.5’s response length grew by about 22.2%.
- Math I: Prime vs Composite: GPT-4 followed chain-of-thought instructions and reached the correct answer in March but omitted intermediate reasoning and answered incorrectly in June.GPT-3.5 followed chain-of-thought in both releases, but its March answer-first behavior was largely fixed in June.
- Math I: Prime vs Composite: GPT-4’s chain-of-thought boost in prime testing fell from 24.4% in March to -0.1% in June, while GPT-3.5’s increased to 15.8% in June.These changes suggest that drift affected whether and how each service followed user instructions.
- Math I: Prime vs Composite: GPT-4’s June prime-testing responses showed a strong composite bias, classifying 99.7% of cases as composite.Its March version identified most primes and composites correctly, whereas June responses consistently treated most primes as composite.
- Math II: Counting Happy Numbers: On happy-number counting, GPT-4’s accuracy dropped from 83.6% to 35.2%, while GPT-3.5’s increased from 30.6% to 48.2%.GPT-4 again stopped following chain-of-thought in June; GPT-3.5 followed it in both releases but sometimes exceeded the maximum token length.
- Sensitive Questions: GPT-4 answered fewer sensitive questions in June, dropping from 21.0% to 5.0%, while GPT-3.5’s response rate rose from 2.0% to 8.0%.GPT-4 also became more terse when refusing, producing fewer explanations; under AIM attacks, its direct-answer rate fell from 78% to 31.0%.
3.4 OpinionQA Survey: Lower Response Rate
Across the evaluated tasks, model behavior shifted substantially between March and June, including lower survey response rates, code-formatting failures, and divergent multi-hop and visual-reasoning outcomes.
- OpinionQA Survey: Lower Response Rate: GPT-4’s OpinionQA response rate dropped from 97.6% in March to 22.1% in June, while GPT-3.5’s increased by 2%.GPT-3.5 also changed 27% of its opinions across releases, exceeding within-release disagreement rates.
- Code Generation: After removing non-code text, GPT-4’s code performance increased from 52% to 70%, showing that formatting changes contributed to direct-execution failures.The formatting failure remained a behavioral change relative to the instruction to generate code only.
- Code Generation: Directly executable code generations fell from 52.0% to 10.0% for GPT-4 and from 22.0% to 2.0% for GPT-3.5 between March and June.The June versions often added Markdown markers, comments, or other non-code text around otherwise similar code.
- Multi-hop Knowledge Questions: GPT-4’s exact match rate on the LangChain HotpotQA Agent increased from 1.2% in March to 37.8% in June, while GPT-3.5’s dropped by almost 9%.More than 80% of final answers differed across releases for both models.
- Visual Reasoning: Visual-reasoning performance improved marginally for both models, but overall exact-match performance remained 27.4% for GPT-4 and 12.2% for GPT-3.5.More than 90% of visual-puzzle generations were identical across March and June, despite some instance-level reversals.
4 Is GPT-4’s instruction following getting worse over time?
GPT-4’s instruction-following fidelity decreased from March to June, especially for composite instructions, and this decline was associated with several observed behavior drifts.
- Benchmark Design: The instruction-following benchmark used task-agnostic answer extraction, apology avoidance, writing constraints, and text-formatting instructions.The evaluation applied these instructions to arXiv abstracts, sensitive questions, and manually created style-refinement queries.
- Individual Instructions: GPT-4 followed 99.5% of answer-extraction queries in March but only 0.5% in June.On sensitive questions, compliance with the instruction to avoid “sorry” or “as an AI model” fell from 74% to 19%.
- Individual Instructions: GPT-4 followed single instructions similarly across releases, with shifts of -2%, +4.0%, and -1.0% for the evaluated instructions.The larger degradation appeared when instructions were composed.
- Overall Findings: GPT-4 followed fewer user instructions over time, whereas GPT-3.5 showed no consistent change in instruction following.The authors identify instruction-following shifts as a primary factor in GPT-4’s behavior drifts.
5 Conclusions and Future Work
The study finds that GPT-3.5 and GPT-4 can change substantially over short periods, with improvements in some dimensions accompanied by declines in others.
- Conclusions: GPT-3.5 and GPT-4 showed significant behavioral variation over a relatively short time, with divergent trends across models and tasks.The paper notes changes in chain-of-thought behavior and verbosity in addition to final performance.
- Future Work: The authors recommend continuous evaluation of LLM behavior in applications because model updates are not transparent and can produce unexpected cross-task side effects.They plan an ongoing study evaluating GPT-3.5, GPT-4, and other LLMs over time.
A.1 GPT-4’s Generation in March for Prime Numbers
The appendix illustrates GPT-4’s March prime-number reasoning with detailed divisibility checks and an example arithmetic error despite a correct conclusion.
- Prime-Number Examples: GPT-4’s March response concluded that 2647 was prime after checking divisibility by primes below its square root.The response ended with “[Yes]”.
- Reasoning Limitations: GPT-4 made an arithmetic mistake by computing 2647/7 as 377.857 instead of the correct 378.142, despite reaching the correct conclusion overall.The example is presented as a limitation of otherwise mostly correct step-by-step reasoning.
A.2 GPT-4’s Generation in March for Composite Numbers
GPT-4’s March response followed the requested step-by-step procedure for testing whether 4323 was prime, but contained an arithmetic error in one divisibility check.
- The prompt asks GPT-4 to determine whether 4323 is prime while thinking step by step and answering “[Yes]” or “[No]”.
- The divisibility calculation for 4323 / 3 is incorrect: it reports a remainder even though the division is exact.
- GPT-4’s March response checks parity, estimates the square root, and tests odd divisors through that bound.
- The response concludes that 4323 is prime and answers “[Yes]”.
A.3 Some Runs Show More Detailed Reasoning
A second GPT-4 March run produced more detailed reasoning for a prime-number query, explicitly checking divisibility through the square root before answering correctly.
- The example asks whether 19997 is prime and requests step-by-step reasoning followed by “[Yes]” or “[No]”.
- The March response was more detailed than its first generation.
- The response checks whether 19997 is even, computes its approximate square root as 141.4, and tests divisibility by primes up to that bound.
- After finding no divisors through the square root, GPT-4 concludes that 19997 is prime and answers “[Yes]”.
B Counting Happy Numbers in Smaller Intervals
Counting happy numbers in smaller intervals produced a confusion-matrix trend similar to the main setting, with GPT-4’s March version usually correct and its June version often answering one.
- The smaller-interval confusion matrix shows a trend similar to counting happy numbers in larger intervals.
C GPT-3.5’s Instruction Following Shifts on Single Instructions
GPT-3.5’s instruction-following shifts on individual instructions are relatively small and diverse, consistent with its broader behavior shifts.
- GPT-3.5’s instruction-following fidelity shifts are relatively small and diverse.
- Figure 16 presents overall instruction following alongside example GPT-3.5 responses.
- The individual-instruction shifts are smaller than the instruction-following shifts observed for GPT-4.