Source-linked AI summary
Data Science in Statistics Curricula: Preparing Students to "Think with Data"
Johanna Hardin, Roger Hoerl, Nicholas J. Horton, Deborah Nolan
TL;DR
Undergraduate statistics programs have not traditionally emphasized the computing, data-handling, and algorithmic skills expected of analysts. The paper presents case studies, curricular examples, and resources from seven institutions, reporting popular and rewarding courses while identifying workload and student-background challenges.
Problem
Statistics graduates entering analyst roles are expected to use databases, scrape Internet data, program in multiple languages, and think algorithmically, but these topics have not traditionally been major components of undergraduate statistics programs.
Method
The paper details data-science curricular approaches from seven institutions and provides syllabi, course materials, assignments, and implementation examples for instructors.
Results
Faculty at the participating institutions report that data-science courses are popular and rewarding to teach.
Takeaways & Limitations
Data-science skills can be integrated through dedicated courses and reinforced across the statistics curriculum, including with examples such as SQL queries in introductory statistics.
Takeaways & Limitations
Introductory data-science courses face heterogeneous student backgrounds and heavy workloads, and topics may need to be reduced to avoid discouraging newer or overburdened students.
Abstract
from arXiv · showhide
A growing number of students are completing undergraduate degrees in statistics and entering the workforce as data analysts. In these positions, they are expected to understand how to utilize databases and other data warehouses, scrape data from Internet sources, program solutions to complex problems in multiple languages, and think algorithmically as well as statistically. These data science topics have not traditionally been a major component of undergraduate programs in statistics. Consequently, a curricular shift is needed to address additional learning outcomes. The goal of this paper is to motivate the importance of data science proficiency and to provide examples and resources for instructors to implement data science in their own statistics curricula. We provide case studies from seven institutions. These varied approaches to teaching data science demonstrate curricular innovations to address new needs. Also included here are examples of assignments designed for courses that foster engagement of undergraduates with data and data science.
INTRODUCTION
Statistics bachelor’s degrees more than doubled from 2008 to 2013 and continue to grow, while graduates increasingly need computing, data-technology, statistical, and communication skills for analyst roles.
- More than twice as many bachelor's degrees in statistics were awarded in 2013 than in 2008, and the number continues to grow.
- Statistics graduates working as analysts need statistical methods, statistical thinking and practice, theoretical foundations, computing and data-related technologies, and communication.
- The paper presents approaches from seven institutions to incorporate computational and authentic data experiences into undergraduate statistics coursework.
- These approaches aim to prepare students for industry analyst roles, graduate study in statistics, or scientific work in allied fields.
WHAT IS DATA SCIENCE?
Data science combines statistics with large-scale computing, while requiring deep statistical reasoning about uncertainty, questions, data quality, context, and domain knowledge.
- Data science is described as a discipline involving some mixture of statistics and large-scale computing.
- The paper identifies an increased need for statistical thinking because data volume and complexity have grown alongside technologies for processing and analysis.
- Core statistical reasoning includes understanding randomness, variability, and uncertainty in the problem.
- Data scientists should formulate clear research questions, acquire high-quality data, understand how data were produced, and use domain knowledge in collection and analysis.
Why does data science belong in the undergraduate curriculum?
Traditional statistical designs, models, and analyses remain necessary but are insufficient for data-rich problems, which also require computational and algorithmic problem-solving skills.
- Existing statistics curricula already address variability, sampling bias, causation, responsible statistical use, and the need for domain understanding.
- Traditional undergraduate statistical designs, models, and analyses are necessary but not sufficient for today’s data problems.
- Statistics students need enhanced problem-solving skills that incorporate practical computational skills to address new and future data-rich problems.
- As questions, data structures, and algorithms become more complicated, computational and algorithmic aspects become integral to principled statistical solutions.
- Computing skills are presented as essential for scientific research involving statistical analyses and increasingly important in the future.
Integrating Data Science into the Curriculum: Seven Prototypes
The prototypes integrate computational tools, authentic data, and programming into statistics education through varied course designs. Reported benefits include student confidence, perceived career relevance, and stronger connections to research, while implementation challenges center on assessment, resource development, workload, and heterogeneous preparation.
- Course content: A key learning outcome is recognizing that different programming languages can express the same underlying data operation despite differing syntax.R and SQL are compared through operations such as merging and aggregating, including SQL GROUP BY.
- Reported outcomes: Students and faculty described the courses as rewarding, and St. Olaf statistics concentrators commonly recommended requiring CS125 early in the concentration.Students particularly praised the computing and R insights gained beyond what methods-focused statistics courses could cover.
- Implementation challenges: Implementation challenges included paper-based assessment, difficulty finding suitably unprocessed project data, and substantial variation in students’ programming backgrounds.Instructors also faced workload concerns and difficulty designing assignments that kept beginners and advanced students engaged.
- Course reach: The courses attracted broad participation, including more than 600 Berkeley students and over 200 Davis students in 2014-15, while St. Olaf’s CS125 drew diverse audiences across academic backgrounds.CS125 served statistics students, computer science students, and students interested in scientific research or general education requirements.
- Reported outcomes: Students reported that computational courses were useful for employment, research participation, learning new technologies, and confidence in tackling increasingly challenging problems.At Berkeley, students linked projects to jobs and research, while Purdue students connected course skills directly to employer interview questions.
CURRICULAR TOPICS
The example curricula integrate programming, data technologies, formats, and statistical topics across varied course designs. They aim to prepare students to work through data problems computationally and statistically, while topic balance varies by course and assignment type.
- Curricular scope: The example courses differ in level and audience but repeatedly address programming, data technologies, formats, and statistical topics.Local curricular constraints influence which topics are included, how much time they receive, and their sequence.
- Programming: Programming is treated as essential, including structured programming, modular code, control flow, efficiency, and sometimes high-performance computing.The courses generally integrate programming with data handling and analysis rather than requiring a separate programming prerequisite.
- Data technologies and formats: Data technologies and formats include databases, SQL, text data, regular expressions, XML, shell commands, and other tools for accessing and managing data.These topics support tasks such as programmatic handling of files and extracting information for analysis.
- Statistical topics: Statistical topics remain central because understanding analysis affects how data are processed and enables statistical thinking throughout the data analysis process.The courses are not limited to data wrangling; they connect data technologies with statistical analysis in realistic contexts.
- Assignment balance: At St. Olaf, nearly 75% of daily assignments focused on programming in R and Python, while projects allocated about 50% to programming, 20% to modern methods, and 15% to visualization.The balance differed between daily assignments and projects.
- Assignment balance: At Berkeley, programming accounted for about 35% of lab work and homework, while projects divided effort between programming and visualization at 30% each.Berkeley’s lecture time also allocated 35% to programming and 15% each to visualization and modern methods.
Examples of Assignments and Projects
The paper presents assignments that engage students with the full data analysis process, combining research questions, data acquisition and cleaning, computation, statistical analysis, visualization, and communication. The examples use authentic settings such as film networks, email spam, energy use, and interactive data products.
- Assignment resources: The assignment examples are provided as concrete resources for bringing data science topics into statistics curricula.Resources for the assignments are also available through a git repository.
- Network analysis: The Hollywood network assignment has students formulate a research question, use SQL to organize IMDb data, and apply centrality measures such as betweenness and eigenvector centrality.The assignment uses the Kevin Bacon six-degrees claim as its motivating question.
- Text processing: The spam-filter assignment uses over 9,000 classified email messages and requires students to transform raw text files into data suitable for statistical analysis.Students use regular expressions to separate message components and extract header information.
- Data analysis cycle: The Residence Hall Energy Use project emphasizes the complete analysis process by having students visualize energy data and write functions to clean and transform it.Students then apply the techniques to energy usage on a global scale.
- Communication: The Developing Data Products course teaches students to present statistical analyses through interactive Web interfaces using the Shiny application in R.This example emphasizes communication aspects of statistics and data science.
SUMMARY DISCUSSION
The paper presents concrete ways to integrate data science into statistics curricula, emphasizing repeated authentic data experiences, curricular flexibility, and practical resources for instructors.
- Student engagement: Faculty report that data science courses are popular and rewarding, and conjecture that students perceive them as relevant and exciting.The paper also describes competitions as student-led investigations of real data problems, with students able to display their skills publicly.
- Curricular approaches: Data science courses repeatedly engage students in the full data analysis cycle, from forming questions and obtaining data through cleaning, analysis, and communication.This data-centric approach addresses early analytical stages often skipped in traditional courses.
- Teaching and assessment: Data science assignments require both computing aptitude and insight into data analysis, creating challenges for grading technical writing, coding, open-ended analyses, and code tests.Peer grading and systems for evaluating code can reduce grading burdens in large courses, but faculty preparation remains an issue.
- Teaching and assessment: Courses with few prerequisites attract students with heterogeneous backgrounds, so faculty use adaptable assignments, sequenced material, peer learning, and modular designs to maintain engagement without overwhelming students.Faculty attribute the variation partly to limited exposure to computing concepts before college.
- Curricular approaches: Data science topics can be introduced, reiterated, and reinforced across statistics courses rather than confined to a standalone class.Suggested extensions include SQL examples in introductory statistics and later capstones, summer experiences, and competitions.
- Curricular approaches: Seven institutions provide varied implementations, syllabi, course materials, and assignments that instructors can compare, adapt, and incorporate into undergraduate programs.The implementations include required courses, electives, and collaborations between statistics and computer science programs.