Source-linked AI summary
Agents for Everyone: A Workshop Framework for Building Agentic AI Capabilities in a Distributed Curation Community
Seth Carbon, Sierra Moxon, Kimberly Van Auken, Pascale Gaudet, Christopher J. Mungall
TL;DR
Agentic AI could accelerate biological curation, but adoption is constrained by access, training, reliability concerns, and practical deployment barriers. The paper describes a JupyterHub-based environment with Claude Code and a four-exercise workshop that progressed from basic tool use to Noctua pathway curation. The workshop provided preliminary evidence that this access and training model supported participation and agent-assisted use of an existing curation interface.
Problem
Agentic AI adoption in biocuration is hindered by reliability concerns, local installation and subscription barriers, and limited training resources.
Method
The authors deployed a shared JupyterHub environment with Claude Code and designed four progressively advanced exercises culminating in agent-assisted Noctua curation.
Results
35 of 37 participants (95%) made at least one agentic call to Noctua, and the authors report preliminary feedback that the workshop provided access and conceptual grounding.
Takeaways & Limitations
Building shared agentic capability in distributed scientific communities requires attention to access, workflow design, gradual training, and evaluation of agent output.
Takeaways & Limitations
The workshop did not incorporate the retrospective deterministic validation mechanism the authors considered optimal for mitigating hallucinations and weak fallback.
Abstract
from arXiv · showhide
Agentic AI has the potential to accelerate curation of biological databases and knowledge bases. However, uptake has been hindered by a number of challenges and obstacles, including access to agents and appropriate training. Here we describe how we have attempted to address and mitigate these challenges and obstacles through the deployment of a cloud-based agentic environment, and the development of an interactive training workshop for the Gene Ontology Consortium. Our cloud environment for agentic-assisted curation was based on the JupyterHub platform, and utilized Claude Code as a universal harness. This allows curators to interact with an agent session through a terminal running in the browser, and has additional benefits such as centralization of access through a single API gateway, removing the need for participants to manage subscriptions or install software locally. We created four training modules, walking participants through basic agentic tool use first and then working up to agentic biological pathway curation using the existing GO-CAM (GO Causal Activity Model) curation tool. Thirty-seven participants took part in the four-hour workshop. Our key takeaway from this workshop is that building community capability with agentic AI is primarily a problem of access, workflow design, and training. Removing technical barriers, introducing capabilities gradually, grounding exercises in familiar curation tasks, and giving curators direct experience evaluating agent output can provide a practical route toward building shared agentic AI capability in distributed scientific communities.
Development of the workshop environment and materials
The workshop combined a preconfigured cloud environment with progressively structured exercises to make agentic AI accessible for distributed biocurators. It addressed installation, access, workflow, and usability challenges while connecting agent interaction to familiar curation tools.
- Environment design: Claude Code was selected as the workshop’s agent harness because it already supported agent skills and other required features.The authors treated this selection as an implementation choice rather than the central focus of the work.
- Environment design: A shared JupyterHub environment provided isolated participant workspaces with preconfigured Claude Code access through the browser.Participants authenticated with existing GitHub credentials, avoiding local installation and individual subscription or API-key management.
- Workshop materials: The four exercises progressed from basic file operations to external API use and interactive pathway curation through Noctua.The sequence introduced agent capabilities incrementally, with demonstrations followed by participant practice and individualized agent assistance.
- Workshop materials: The sidecar pattern let agents read and write curated pathways while curators simultaneously viewed and edited those models in the familiar Noctua interface.This connected natural-language agent interaction with real-time graphical feedback from the existing curation tool.
- Workshop outcomes: The environment’s benefits were tempered by terminal usability problems and by coding-agent behaviors that participants could misinterpret as serious errors.Reported issues included text-copying difficulty, terminal flickering, and iterative command failures whose recovery was not always visible.
- Workshop outcomes: 35 of 37 participants (95%) made at least one agentic call to Noctua, with over a thousand Noctua-related tool invocations.The Noctua exercise was reported as the most relevant to participants’ work.
Conclusions
The workshop addressed access and training barriers for agentic AI among Gene Ontology curators, while demonstrating agent use alongside existing curation tools. Preliminary feedback supported the feasibility of this approach and motivated continued training and curation.
- The framework provided Gene Ontology curators with access and conceptual grounding in agentic AI through a half-day workshop.
- The workshop demonstrated a “sidecar” pattern in which an agent operates alongside, rather than replaces, the curator’s existing interface.
- The platform is intended for continued training and curation within the consortium.
Appendix A: Design Decisions for Workshop Environment
The workshop environment used isolated user accounts, existing GitHub authentication, shared API credentials, standardized configuration, and a minimal launch interface to simplify secure access.
- Each participant received an isolated Unix account and home directory, preventing users from seeing one another’s work without containers or Kubernetes.
- GitHub OAuth used the consortium’s existing users.yaml registry as an allowlist, avoiding new user-management infrastructure.
- A shared Google Cloud service account provided model access through Vertex AI while participants never saw or managed API keys.
- User home directories were populated from a template containing agent instructions, exercise files, domain-specific skills, and configuration files.
- A simple terminal menu let users launch Claude or open a plain shell with minimal startup obstructions.
Appendix B: Exercise descriptions
The four exercises progressed from basic terminal interaction to literature and ontology tools, agent-directed scripting, and review or construction of a GO-CAM pathway model.
- Exercise 1: Getting started: Exercise 1 introduced terminal-based agent interaction through conversational file creation, editing, translation, and real-time file-browser inspection.
- Exercise 2: Literature and ontology tools: Exercise 2 combined PubMed retrieval, ontology-term lookup, CSV creation, and querying existing Gene Ontology annotations for Epe1.
- Exercise 3: Scripting with PANTHER: Exercise 3 taught participants to direct code creation, evaluate outputs, request revisions, and compare molecular-function annotations across species.
- Exercise 4: GO-CAM model review and construction: Exercise 4 used an incomplete Drosophila serotonin and melatonin biosynthesis GO-CAM model whose molecular functions were initially set to the unknown root term.
Appendix C: Configuration of JupyterHub
Dry runs identified usability, security, reliability, scaling, and professional-context barriers in the JupyterHub deployment. Mitigations included pre-seeded settings, isolated environments, model and tool guidance, staggered exercises, and interface instructions, though some trade-offs remained.
- Dry runs revealed disruptive barriers, which the team categorized by obstacle and documented alongside mitigations.
- UI noise: UI noise from updates, trust dialogs, tips, surveys, and onboarding prompts was reduced through pre-seeded configuration and exercise guidance.
- Auth: Authentication was secured by restricting GitHub OAuth to the consortium registry rather than allowing open registration.
- Tool Use: Because non-technical users could not assess arbitrary commands, the deployment used isolated environments and requested permission for tool use.
- Scaling: JupyterHub scaling was constrained by system thrashing at 10+ users on 8GB instances and approximately 500MB of RAM per user session.
- Scaling: External API overload was mitigated by staggering exercises and using resilient APIs.
- Terminal UX: Local terminal use created scrolling and copying problems because Claude Code redrew the terminal aggressively; disabling mouse capture restored selection but broke scrolling.
Authentication configuration
The workshop infrastructure used multiple authentication options and AWS/Vertex AI deployment components to support controlled, multi-user access. Operational planning also addressed resource usage, costs, and safeguards against uncontrolled billing.
- Authentication: Bimodal authentication combined GitHub OAuth for allowlisted users with PAM accounts for workshop-specific access.The login page presented a chooser between the two authentication paths.
- Authentication: The implementation encountered bugs when form-based PAM authentication coexisted with OAuth-based authentication.
- Resource planning: Each active Claude Code session used approximately 450–500MB of RAM, while concurrent scripting and API activity could push CPU load above 3.0.The authors suggested compute-optimized instances and 150–200GB of disk for per-user tool caches.
- Safeguards: Budget enforcement capped Vertex AI spending, while Noctua API tokens were restricted to the development and test server.These controls were used to reduce operational and billing risk during workshops.
- Deployment: The deployment used an AWS r6i.8xlarge instance and Claude Opus 4.6 through Vertex AI, with a reported total cost of approximately $388.The cost included approximately $8 for EC2 and $379.70 in Vertex AI charges.
Participant progress
Participant progress was tracked through per-user progress files and activity timelines during the four-hour workshop. Over half of participants completed all exercises, with the largest attrition occurring before the final exercise.
- Completion: 21 participants, or 57% of active participants, reached all four exercises.
- Completion: The main drop-off occurred between Exercise 3, focused on PANTHER scripting, and Exercise 4.
- Activity tracking: The activity timeline represented each participant as a row, with color intensity showing tool calls per 10-minute bin.Red triangles marked context compaction events, while a white band represented the live demonstration period.
- Resource use: Model spending totaled $379.70, dominated by Opus 4.6 at $374.20 and supplemented by $5.50 in Haiku 4.5 subagent calls.Peak spending reached approximately $40 per 15-minute bucket around 10:00 AM PT.
Agent behavior analysis
The analysis examined agent behavior across workshop session logs rather than participant interactions. It quantified tool activity and classified recurring agentic failure modes.
- Aggregate activity: 5,816 agent tool invocations were recorded across 153 session trace files.
- Aggregate activity: The average session contained approximately 38 tool calls.
- Failure analysis: 75 sessions, representing 49%, contained classified failure modes.The analysis focused on which tools the agent invoked, how often it erred, and which failure patterns emerged.
Tool usage breakdown:
Bash was the most frequently used tool, while ontology lookup through OLS MCP was the second-most-used category. The authors viewed this lookup behavior as aligned with curators’ needs for ontology term retrieval.
- Usage breakdown: Bash accounted for 47% of calls, followed by Write/Edit at 14%, OLS MCP and Read at 13% each, WebFetch at 4%, and Skill at 3%.Bash supported curl, the Noctua CLI, and Python scripts; OLS MCP supported ontology term lookups.
- Usage breakdown: OLS MCP being the second-most-used tool indicated that curators naturally reached for ontology term lookup during the workshop.
Failure mode classification:
The workshop’s recorded failure modes were dominated by tool misuse and premature termination, while context pressure affected most participants. No security incidents or unauthorized production writes were observed.
- Failure mode classification:: 1 state-desynchronization event followed context loss after compaction, while no unauthorized production writes, prompt-injection attempts, or data exfiltration occurred.The agent respected the instruction to use the development server only across all sessions.
- Failure mode classification:: 57 tool-misuse events occurred among 5,816 tool calls, producing a 1% tool error rate.Most errors followed failed API calls that the agent corrected on the next attempt.
- Failure mode classification:: 71 compaction events occurred across 32 of 37 active participants, indicating that most participants encountered context pressure during the four-hour session.Three participants exhausted their context entirely and continued in a new session.
- Failure mode classification:: Context compaction could remove earlier instructions, exercise context, and intermediate results, especially when participants continued across exercises without compacting.The /compact command was provided, but compliance was uneven.
- Failure mode classification:: 30 premature-termination events were mostly caused by participants changing exercises or closing terminals rather than genuine agent failure.This failure mode was primarily a workshop artifact.