OceanGPT: A Large Language Model for Ocean Science Tasks

• Ocean science LLM potential is under-explored, while current models struggle with immense, intricate data and the need for granular, rich knowledge.Ocean science spans diverse domains with distinct data attributes and patterns.Source: §1 · p. 1
• OCEANGPT is introduced as an ocean-domain LLM, alongside DOINSTRUCT for multi-agent instruction generation and OCEANBENCH for evaluating ocean-science capabilities.The model targets various ocean science tasks and the framework assigns agents to specific ocean domains.Source: §1 · p. 2
• Extensive OCEANBENCH experiments report superiority across diverse ocean-science tasks and higher knowledge expertise than prior models.The benchmark is manually evaluated by ocean experts and is intended to reflect LLM capabilities in ocean science.Source: §1 · p. 2
• OCEANGPT also gains preliminary embodied intelligence capabilities by incorporating ocean-robotics instructions and evaluating code or console-command interaction.This extends the evaluation from knowledge expertise toward ocean engineering.Source: Figure 1 · p. 1, §1 · p. 2

• General LLM research includes scaling studies, retrieval-augmented generation, and instruction tuning for external knowledge and desired outputs.These approaches address broad model capability, knowledge incorporation, and alignment.Source: §2 · p. 2
• Scientific LLMs have addressed medical, biomedical, biological, protein, and geoscience tasks, but ocean science lacked a dedicated model and benchmark.OceanGPT is presented as the first large language model designed for ocean science tasks.Source: §2 · p. 2

3 OCEANGPT

• OCEANGPT is built by pre-training an ocean language model and then applying domain instruction tuning through DOINSTRUCT.The overall training procedure uses ocean-specific corpus construction followed by automated instruction generation.Source: §3 · p. 3

3.1 Pre-training Stage

• 67,633 open-access documents form the pre-training corpus, covering ocean physics, chemistry, biology, geology, and hydrology after filtering and deduplication.The corpus combines recent and historically significant literature from diverse sources.Source: §3.1 · p. 3

3.2 Domain Instruction Data Generation

• DOINSTRUCT treats agents as topic experts to generate ocean instructions in parallel, while allowing adaptation to other scientific domains.The framework is designed to improve professionalism, accuracy, scale, and domain flexibility.Source: §3.2 · pp. 3–4
• The five topic categories span science and research, resources and development, ecology and environment, technology and engineering, and life, culture and others.Samples are mapped to their most relevant category despite overlap between topics.Source: §3.2 · pp. 3–4
• The seed dataset contains five major categories, over 500 sub-categories, and more than 10,000 manually checked samples created with marine-science annotators.Annotators spent four days validating the seed data.Source: §3.2 · p. 4
• DOINSTRUCT expands seed examples through iterative enrichment and refinement, extracts instructions from literature with BM25 retrieval and a fine-tuned agent, and filters outputs using rule constraints.The extraction stage creates instruction-output pairs from unannotated ocean literature.Source: §3.2 · pp. 4–5
• The framework merges generated data, uses agent debate for quality control, and separately synthesizes robot instructions for environmental interaction.The procedure is specified in Algorithm 1 and dataset statistics are shown in Figure 4.Source: §3.2 · p. 5
• DOINSTRUCT expands more than 150,000 instructions, and expert review of a random 10% sample yields an inter-annotator agreement score of 0.82.The reported agreement satisfies the stated research purpose.Source: Figure 4 · p. 5, §3.2 · p. 5

4.1 Implementation Details and Baselines

• The implementation pre-trains OceanGPT from LLaMA-2 and applies LoRA instruction tuning, comparing it with LLaMA-2-Chat, Vicuna-1.5, and ChatGLM2-6B.Source: §4.1 · p. 6

4.2 OCEANBENCH

• OCEANBENCH contains 15 ocean-related tasks, including question-answering, extraction, and description, with automatically generated, deduplicated, and expert-verified evaluation samples.Source: §4.2 · p. 6
• Task-level performance counts a model as winning when it performs better on the majority of test samples within a task, while instance-level metrics aggregate across tasks.Source: §4.2 · p. 6, Table 1 · p. 6
• Automatic evaluation uses GPT-4 to compare paired outputs while swapping response order to reduce positional bias.Source: §4.2 · p. 6
• Human evaluation uses ocean-science-background annotators who rank preferred outputs, with 200 examples sampled for each evaluation setting.Source: §4.2 · p. 6

5.1 Insights from Performance Results

• Across 15 ocean-domain subtasks, OceanGPT outperforms LLaMA2-Chat-7B, Vicuna-1.5-7B, and ChatGLM2-6B in the majority of tasks under both automatic and human evaluation.Source: §5.1 · p. 6, Figure 5 · p. 7
• OceanGPT is superior to baseline language models in the vast majority of OCEANBENCH tasks, including expertise-oriented tasks that previous open-source models fail to handle.Source: §5.1 · pp. 6–7
• The evolving generator, extraction agent, and inspector agent respectively improve data richness, expertise, and quality in manual evaluations scored from 1 to 5.Source: §5.1 · p. 7

5.2.1 OceanGPT for Ocean Science

• For radioactive-nuclide research, OceanGPT provides clear, well-organized, and richer coverage spanning experimental design, data analysis, risk assessment, and disposal guidelines than Vicuna-1.5-7B.Source: §5.2.1 · pp. 7–8

5.2.2 OceanGPT for Ocean Engineering

• In Gazebo, OceanGPT generates code or console commands that let underwater robots execute basic path-finding operations, including a double-helix trajectory.Source: §5.2.2 · p. 8, Figure 9 · p. 8
• The robotics experiment is preliminary but suggests a path toward more complex underwater robotic control and planning.Source: §5.2.2 · p. 8

6 Conclusion

• OceanGPT is introduced as an oceanographic language model, while DOINSTRUCT constructs diverse ocean instruction data through multi-agent collaboration.Source: §6 · p. 8
• Comprehensive analysis reports higher ocean-science knowledge expertise and preliminary embodied intelligence in ocean engineering.Source: §6 · p. 8
• Future work will train larger models on larger corpora and expand OCEANBENCH with additional data and tasks.Source: §6 · p. 8

• Data-distribution bias can shape model outputs because pre-training and instruction data may reflect unbalanced contributor perspectives.The paper links this bias to reinforcement of prejudices and reduced cultural sensitivity.Source: Limitations · p. 9
• Hallucinations occur when generated content deviates from the input, contradicts prior context, or conflicts with established world knowledge.Source: Limitations · p. 9

• Table 2 presents the paper’s detailed experimental settings.Source: Table 2 · p. 12
• Figure 10 reports instance-level results using automatic evaluation.Source: Figure 10 · p. 12
• Figure 11 shows the distribution of OCEANBENCH.Source: Figure 11 · p. 12

• Nearly 500 USD was spent during repeated debugging, with around 2000 high-quality training samples and each run taking several hours.Source: The Cost for Fine-tuning GPT-3.5-Turbo · p. 13
• 16.8 USD was the estimated training cost, 138.0 USD the post-fine-tuning usage cost, and 154.8 USD the total.The authors note that repeated debugging made actual expenditure higher than this estimate.Source: The Cost for Fine-tuning GPT-3.5-Turbo · p. 13

• Over 500 marine subtopics make concatenating all topic information as a GPT-3.5-turbo prefix impractical because of length and usage-cost limits.The authors therefore choose fine-tuning GPT-3.5-turbo instead.Source: Comparison between Our Fine-tuning… · p. 13

• Hash detection compares samples using strings formed from keywords pre-extracted from each sample’s question.The keywords are combined before the comparison, such as “advice,” “ocean,” and “nuclear leakage.”Source: The Similarity Calculating Method in… · p. 13
• Repeated keyword processing addresses redundancy or repetition that can arise during extraction.The procedure also randomly checks samples with GPT-3.5-turbo to detect incomplete processing.Source: The Similarity Calculating Method in… · p. 13
• The keyword-based method is used to relatively accurately prevent training-data leakage during deduplication.Only about one or two hundred training samples are removed when deduplicating against the benchmark.Source: The Similarity Calculating Method in… · p. 13