Source-linked AI summary
How do Data Science Workers Collaborate? Roles, Workflows, and Tools
Amy X. Zhang, Michael Muller, Dakuo Wang
TL;DR
Data science collaboration is poorly characterized beyond the individual data scientist, especially for non-technical team members. The authors surveyed 183 IBM data science workers about roles, workflow stages, tools, and practices, finding extensive collaboration whose forms varied with tool use. These findings motivate collaboration-friendly data science tools while remaining bounded by the IBM, self-selected, self-reported sample.
Problem
Data science collaboration is poorly characterized beyond technical data scientists, leaving non-technical collaborators and their interactions, timing, and tools underexamined.
Method
The authors conducted an online survey asking 183 IBM data science workers about collaborators, roles, workflow stages, tools, reuse, and documentation.
Results
Data science workers collaborate extensively across project stages, while tool profiles correspond to different patterns of code and data collaboration and documentation.
Takeaways & Limitations
The findings support designing a new generation of collaboration-friendly data science tools and environments.
Takeaways & Limitations
The respondents were all recruited from IBM and may not represent the broader population of data science professionals.
Abstract
from arXiv · showhide
Today, the prominence of data science within organizations has given rise to teams of data science workers collaborating on extracting insights from data, as opposed to individual data scientists working alone. However, we still lack a deep understanding of how data science workers collaborate in practice. In this work, we conducted an online survey with 183 participants who work in various aspects of data science. We focused on their reported interactions with each other (e.g., managers with engineers) and with different tools (e.g., Jupyter Notebook). We found that data science teams are extremely collaborative and work with a variety of stakeholders and tools during the six common steps of a data science workflow (e.g., clean data and train model). We also found that the collaborative practices workers employ, such as documentation, vary according to the kinds of tools they use. Based on these findings, we discuss design implications for supporting data science team collaborations and future research directions.
1 INTRODUCTION
Data science has become team-based work spanning technical and non-technical roles, yet empirical understanding of collaboration remains limited. This paper surveys how workers interact across workflow stages and tools.
- Workflow and scope: Data science workflows span preparation, modeling, and deployment, with finer-grained activities involving data, models, and stakeholders.The paper describes a three-phase workflow and six survey stages covering planning, data, features, models, evaluation, and communication.
- Team-based work: Data science projects require multiple skills, leading workers to juggle roles or collaborate with teammates possessing distinct expertise.The paper contrasts technical activities with less technical project-management work and emphasizes the importance of domain knowledge.
- Research gap: Existing collaboration research largely centers on technical data scientists and tools for asynchronous code collaboration.Such work overlooks collaboration before code or data collection, including initial problem framing with stakeholders.
- Study focus: The study examines five team roles and six workflow stages, reporting whom workers collaborate with, when they collaborate, and which tools they use.It includes technical members such as engineers and researchers alongside domain experts, managers, and communicators.
- Study focus: The paper uses an online survey of 183 participants to broaden evidence beyond prior individual-focused studies.The survey is presented as both large-scale and attentive to understudied non-technical collaborators.
2 RELATED WORK
Prior work characterizes data science workflows and tools but often treats work as individual or technical. This paper addresses the missing empirical account of collaboration across corporate data science teams.
- Data science work practices: Existing workflow frameworks describe Preparation, Modeling, and Deployment, including ten finer-grained steps from data acquisition to model monitoring.These frameworks provide terminology for positioning empirical studies within a data science process.
- Data science work practices: Most data science literature focuses on a single data scientist, leaving non-technical collaborators, interaction timing, and tool use underexamined.This gap persists despite reports that data science is a team sport.
- Empirical approaches: Interview studies provide contextual depth but are less suited to representative, generalizable understanding across larger user populations.The authors therefore use an online survey to complement qualitative narratives.
- Collaboration in data science: Data science collaboration differs from conventional software engineering because exploratory work depends heavily on domain knowledge distributed across roles.Prior studies describe translation and common-ground challenges between data scientists and domain specialists.
- Study positioning: The study targets corporate data science teams, extending beyond prior ethnographies of civic hackathons and scientist-domain-expert collaborations.Its setting is intended to represent a more commonly available workplace scenario for data science professionals.
- Data science tools: Common data science tools include Python, R, SAS, SPSS, Jupyter Notebook, AutoML, and AutoAI, while collaborative features remain limited.Jupyter supports coding, narrative documentation, and execution results, but narrative-cell use has been reported as low.
3 METHOD
The authors conducted an anonymous online survey of IBM employees involved in data science, asking about collaborators, roles, workflow stages, tools, and documentation practices.
- Participants and recruitment: Participants were self-selected IBM employees recruited through internal Slack channels focused on data science.Respondents represented research, engineering, health sciences, management, and related business organizations.
- Participants and recruitment: 183 respondents participated, representing an estimated 20% rate among approximately 1,000 employees reading the relevant Slack channels.The participation-rate estimate is based on the authors’ estimate of channel readership.
- Survey design: The survey was anonymous, so participant characteristics came from self-reported responses about their team roles.The authors used survey items to identify respondents’ roles on data science teams.
- Survey design: Questions covered recent projects, collaborators, team roles, workflow-stage interactions, collaboration tools, code and data reuse, expectations, and documentation.All questions were optional to encourage participation.
4 RESULTS
Survey respondents generally worked in small, highly collaborative teams and often occupied multiple roles, especially overlapping technical roles.
- Respondents and teams: 183 respondents reported varied data-science experience, primarily 0–5 years.The sample included respondents with longer experience as well.
- Respondents and teams: Most respondents worked in teams of six people or fewer, while a few reported solo data science practices.Team size was reported from respondents’ descriptions of their work settings.
- Role overlap: Respondents often performed multiple roles, and single-role respondents were relatively rare except among Engineers/Analysts/Programmers.The paper suggests small team sizes may contribute to role multiplicity.
- Role overlap: Engineers/Analysts/Programmers and Researchers/Scientists showed considerable role overlap and also served, to a lesser extent, as Communicators and Domain Experts.These patterns indicate that technical roles commonly extend into other team functions.
- Role overlap: Managers/Executives reported little overlap with other roles, unlike the primarily technical roles associated with functional leadership.The authors suggest this may reflect IBM’s distinction between people management and technical team leadership.
4.1 Do Data Science Workers Collaborate?
Data science workers report extensive collaboration across roles, although perceived collaboration is asymmetric for Communicators. Collaboration strengths are otherwise relatively egalitarian, with technical roles showing strong intra-role collaboration.
- Three of five roles reported collaboration rates of at least 95%, while Domain Experts reported the lowest rate at 87%.
- Who Collaborates with Whom?: Communicators reported strong collaboration with Managers/Executives and Domain Experts, but those partners reported comparatively little collaboration with Communicators.
- Are there “Hub” Collaborator Roles?: The collaboration network showed relatively egalitarian role-to-role strengths, with Managers/Executives generally similar to Engineers and Researchers.
- Are there “Hub” Collaborator Roles?: Communicators appeared to overreport collaboration toward other roles relative to the collaboration those roles reported toward Communicators.
- Are there “Hub” Collaborator Roles?: Strong intra-role collaboration appeared among Engineers, Researchers, and Communicators, whereas Managers/Executives and Domain Experts showed weaker self-collaboration.
4.2 Collaborator Roles in Different Stages of the Data Science Workflow
Collaborator involvement varies significantly across the six stages of the data science workflow. Technical and non-technical roles follow different participation patterns, while Researchers/Scientists remain strongly engaged throughout.
- The six-step workflow spans measurement planning, technical development, evaluation, and delivery of an analysis, model, or working system.
- Where do Non-Technical Roles Work?: Managers/Executives and Communicators were relatively engaged in measurement planning, evaluation, and communication but largely absent from technical stages.
- Who Checks AI Fairness and Bias?: Domain Experts participated across every stage and played a role in bias assessment and mitigation, while Communicators and Managers had minimal involvement there.
- Engineers/Analysts/Programmers were most active during core technical work, when Managers and Communicators were less involved.
- Researchers/Scientists remained relatively stable and strongly engaged across all workflow stages.
4.3 Tooling for Collaboration
Tool use differed significantly across data science project stages, with coding resources concentrated during intensive data and modeling work and documentation gaps in feature engineering and bias mitigation. These patterns motivate collaboration tools with collaboration built into their design.
- χ2_72 = 209.519, p< .001 for differences in tool usage across project stages.
- Coding resources were used most heavily during intensive work with data and models, while asynchronous discussion decreased in those stages.Code may function as a form of asynchronous discussion.
- Documentation tools appeared during measurement planning and returned during delivery to clients, with presentation software used to a small extent.
- Feature extraction and engineering showed little document use, alongside nearly zero synchronous discussion and relatively low asynchronous discussion.The passage identifies feature engineering as an important site for designing and transforming data.
- Bias detection and mitigation also appeared to lack documentation, apart from possible filesharing.
- Because collaboration is extensive but collaborative-tool usage has gaps, the authors propose tools with collaboration “baked in” to their design.
4.4 Collaborative Practices around Code and Data
Respondents clustered into three tool profiles, and their code- and data-sharing practices differed across profiles. Overall, collaboration around data was lower than collaboration around code, while documentation was weakest among scripted-tool users.
- Three tool profiles emerged from 76 respondents, with k=3 producing the highest average silhouette coefficient of 0.254.Profiles were built from summed tool mentions across open-ended workflow-stage responses.
- The project-managed profile emphasized GitHub and Slack, the interactive profile emphasized Jupyter Notebook, and the scripted profile emphasized Python and SPSS.
- Collaborative practices around data were lower overall than practices around code across all three tool profiles.This pattern held despite some tools, including GitHub, supporting data storage and sharing.
- Testing showed fewer respondents reading others’ code in the interactive and scripted profiles, with the scripted profile also showing little code re-use.
- Respondents expected their code to be re-used more often than their data, while data re-use expectations were highest in the project-managed profile.Reported expectations for code re-use ranged from 68.4% to 84.6%, while data re-use ranged from 46.1% to 73.7%.
- Code documentation was higher in the project-managed and interactive profiles than in the scripted profile, which relied mainly on in-line comments.100% of project-managed respondents reported using in-line comments.
5 DISCUSSION
The discussion interprets extensive collaboration alongside documentation and provenance gaps, especially for data and feature-engineering decisions. It argues for making otherwise invisible decisions traceable, discussable, and accountable.
- Most data science roles reported extensive collaboration during each project stage, while the paper examines roles, tools, stages, and documentation together.
- Documentation was insufficient across multiple stages, particularly for data compared with code.
- Limited data documentation can obscure dataset sources and computations during cleaning or transformation, motivating methods for recording data provenance.
- Feature-extraction and feature-engineering code may be difficult to preserve because it is often not based on well-known, maintained software packages.
- A lack of motivation to document lower-level decisions may limit documentation practices in an exploratory work context.
- Collaborative code practices dropped during testing, raising implications for standards around testing data and model bias and for preserving code provenance.
- The authors advocate making invisible activities visible so they can be discussed, debated when necessary, and held accountable.They distinguish this focus on creators’ decisions from explainable AI’s emphasis on making algorithms understandable to non-technical users.
5.2 Collaborating with Whom? and When?
Data science collaboration involves multiple stakeholders, but perceptions can differ across roles and bias work appears weighted toward technical participants. These patterns point to directional collaboration and multidisciplinary accountability concerns.
- Future project teams and the public are indirect stakeholders who may need to understand data definitions, transformations, and decisions.The public concern is especially relevant when systems affect loans, prison sentences, or community policing.
- Communicators reported more collaboration with Managers/Executives than Managers/Executives reported with Communicators.The comparison uses normalized proportions of reported collaborations.
- The authors speculate that collaboration may be directional, with Communicators primarily receiving information while other roles perceive them as relatively passive.
- Peer collaboration appeared stronger among Engineers, Researchers, and Communicators than among Managers and Domain Experts.The passage presents these patterns as suggestions about communities of practice.
- Bias detection and mitigation involve modeling, statistical, domain, regulatory, legal, and advocacy expertise.
- Bias work appeared largely technical, with data scientists, engineers, and domain experts involved but little involvement from Communicators and Managers/Executives.
5.4 Limitations and Future Directions
The study’s findings are constrained by its IBM-only, self-reported survey sample and predominantly small teams. The authors also caution that transparency-focused collaboration features may have negative effects and require further evaluation.
- Scope: IBM-only recruitment may limit how representative the findings are of data science professionals more broadly.The authors note that respondents came from one large multinational technology company.
- Scope: Most respondents worked in small teams of typically 5 or 6 collaborators, so findings may differ for teams of other sizes or contexts.The authors specifically mention offline data hackathons as a potentially different setting.
- Scope: Company culture may have shaped respondents’ tool preferences and usage, limiting generalization across organizations.The authors suggest studying teams in different scenarios to examine whether practices vary.
- Methods: Self-reported online-survey data may be biased because respondents reported their own behavior.The authors propose contextual inquiry, participatory analysis, and value sensitive design for observing collaboration directly.
- Design implications: Transparency and accountability features may have negative effects, so collaboration features need additional user evaluation before deployment.The authors connect this concern to reluctance toward high transparency in co-editing activities.
6 CONCLUSION
The paper presents a large-scale survey examining how data science workers collaborate. It concludes that workers collaborate extensively across roles, stakeholders, workflow stages, tools, and documentation practices, motivating future collaboration-tool research.
- Conclusion: The survey examines collaboration among data science workers at a major corporation.It focuses on how workers collaborate across data science projects.
- Conclusion: Data science workers collaborate extensively, perform varied roles, and engage with varied stakeholders across project workflow stages.The conclusion frames these as central findings of the study.
- Conclusion: The paper analyzes how collaborative tool usage relates to practices such as code and data documentation.It considers tool use across stages of data science projects.
- Future directions: The findings motivate directions for future research and development of data science collaboration tools.The conclusion presents these directions as stemming from the collaboration analysis.
- Contributions: The paper contributes roles-based, stage-specific, tool-usage, and documentation analyses of data science collaboration.These are listed among the paper’s stated contributions.