PaperlayerView exact revision on arXiv

Source-linked AI summary

A Data Science Course for Undergraduates: Thinking with Data

Ben Baumer

arXiv:1503.05570v1stat.OTcs.CYstat.CO

TL;DR

Modern data are often complex, messy, and unlike the small, clean datasets in introductory statistics, while students will need skills to work with them. The paper describes an integrated liberal-arts course spanning the modern data-analysis cycle. Student feedback was nearly universally positive, and students who took the course were part of a team that won Best In Show at a local DataFest.

  • Problem

    Students often encounter live, large, complex, or messy data unlike the small, clean datasets typical of introductory statistics, despite strong demand for people who can analyze data.

    Source: §1 · pp. 1–2

  • Method

    The course develops foundational skills across the modern data-analysis cycle, including visualization and data manipulation, within an integrated liberal-arts data-science curriculum.

    Source: §3 · p. 3, §3.2 · p. 5, §3.3 · p. 6

  • Results

    Feedback from 42 students was nearly universally positive, and four course takers were on the team that won Best In Show at the Five College DataFest.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The course aims to build confidence and foundational skills—not expertise—for the modern analysis cycle, which the authors say are transitioning from desirable to necessary.

    Source: §3 · p. 3, §7 · p. 11

Abstract

from arXiv · show

Data science is an emerging interdisciplinary field that combines elements of mathematics, statistics, computer science, and knowledge in a particular application domain for the purpose of extracting meaningful information from the increasingly sophisticated array of data available in many settings. These data tend to be non-traditional, in the sense that they are often live, large, complex, and/or messy. A first course in statistics at the undergraduate level typically introduces students with a variety of techniques to analyze small, neat, and clean data sets. However, whether they pursue more formal training in statistics or not, many of these students will end up working with data that is considerably more complex, and will need facility with statistical computing techniques. More importantly, these students require a framework for thinking structurally about data. We describe an undergraduate course in a liberal arts environment that provides students with the tools necessary to apply data science. The course emphasizes modern, practical, and useful skills that cover the full data analysis spectrum, from asking an interesting question to acquiring, managing, manipulating, processing, querying, analyzing, and visualizing data, as well communicating findings in written, graphical, and oral forms.

1 Introduction

Growing data production and strong labor demand have increased student interest in statistics, while modern data streams differ from the tidy datasets common in introductory courses. The paper describes an integrated liberal-arts course intended to prepare students to work with these data.

1 Introduction

  • Demand for people who can turn data into actionable information is reported to exceed supply as students invest in data-analysis skills.The authors cite a strong job market for this work.Source: §1 · p. 1
  • Modern data may be observational, unstructured, live, nonnumeric, or too large for traditional inferential techniques to be practical.Examples include databases, XML, web-scraped material, Twitter streams, text, images, and video.Source: §1 · pp. 1–2
  • The course integrates statistics and computer science to prepare students for modern data streams rather than simply assembling topics from existing courses.The authors identify visualization, database querying, and computational concerns as topics statistics curricula should include.Source: §1 · p. 2

2 Background and Related Work

Related work argues for computing and data-wrangling skills in statistical education, while framing data science as a broader practice that crosses traditional disciplinary boundaries.

2 Background and Related Work

  • Statistical programming is presented as distinct from menu-driven computation, with command-line tools such as grep useful for cleaning and manipulating documents.Nolan and Temple Lang (2010) and the 2014 ASA undergraduate guidelines support a larger role for computing in statistics curricula.Source: §2 · p. 2
  • Working with data can build a “data habit of mind,” including planning data formats and storage in light of later analysis.This is distinguished from Chance (2002)’s account of statistical thinking, which the passage says does not mention computing.Source: §2 · p. 2
  • A footnote contrasts data science as “the extraction of knowledge from data” with statistics as the study of data collection, analysis, interpretation, presentation, and organization.It asks whether writing an SQL query belongs to both fields.Source: §2 · p. 2
  • Data manipulation is framed as an intellectual discipline, with practical data “verbs” connected to computer-science ideas such as normal forms and relational algebra.The passage associates this structuring of data with Wickham (2014) and Wickham and Francois (2014).Source: §2 · p. 3
  • The passage describes abundant data as shifting the challenge from strong inference on scarce data toward extracting meaning, amid evolving ties between statistics and computing.It contrasts statistical data models with algorithmic models and describes data science as belonging exclusively to neither traditional departments nor disciplines.Source: §2 · p. 3

3 The Course

MTH 292 is an experimental course taught in 2013 and 2014 for students with introductory statistics and some programming experience. Its short modules aim to build foundational confidence across the modern data-analysis cycle.

3 The Course

  • The course aims to give students foundational skills and confidence for each step of the modern data-analysis cycle, beyond the streamlined process taught in introductory statistics.The course addresses data challenges, computational methods, and visualization and presentation, which are typically not covered in introductory statistics.Source: §3 · p. 3

3.1 Day One

  • On day one, students analyze a data-science study and consider what reproducing it requires, from accessing Twitter data to storing it, mapping locations, and improving graphics.The course is presented as giving students tools to carry out a similar study.Source: §3.1 · pp. 4–5

3.2 Data Visualization

  • The first two weeks teach data visualization as an engaging introduction and a way for students with less programming experience to become comfortable in R.Readings and exercises develop graphic-design awareness, critique skills, and practical R annotation techniques.Source: §3.2 · p. 5

3.3 Data Manipulation/Data Wrangling

  • Students learn fundamental data operations in R and SQL, think about what those operations do, and choose software in light of data size and hardware.R stores data in memory and offers many operations, while SQL stores data on disk and is better suited to large data storage.Source: §3.3 · p. 6

3.4 Computational Statistics

  • Computational statistics covers resampling-based interval estimates and regression, including multiple and logistic regression, with regression also preparing students for machine learning.The course emphasizes understanding variation and presenting interval estimates rather than point estimates alone.Source: §3.4 · p. 8

3.5 Machine Learning

  • The machine-learning unit introduces algorithmic prediction, evaluation with cross-validation and confusion matrices, and classification methods using a large breast-cancer dataset.The dataset has 117 nameless attributes, making manual exploration impractical.Source: §3.5 · pp. 8–9

3.6 Additional Topics

  • Instructors can add spatial analysis, text mining and regular expressions, Data Expo, network science, or big-data topics.These are presented as optional additions to the course’s four pillars.Source: §3.6 · p. 9

4 Computing

The course uses practical coding to prepare students to work with data, using R and SQL while considering Python as a possible addition.

4 Computing

  • Students write their own code in class using R, with annotated computational notes and active laptop participation; despite programming struggles, students reported enthusiasm as they gained comfort.Several students subsequently took computer-science courses in data structures or algorithms.Source: §4 · p. 9
  • Programming is conducted in R and SQL, with Python identified as a possible addition in future offerings.The authors describe SQL as suited to storage, R as designed for statistical computing, and Python as a general-purpose language with data-analysis capabilities.Source: §4 · p. 9
  • At Smith, introductory computer-science students learn Python and introductory statistics students learn R, but the size of the overlap between those groups is unclear.The authors suggest it may be easier for Python users to learn R than the reverse.Source: §4 · p. 9

5 Assignments

Assignments combine conceptual and computational work, two exams, and a group project that culminates in public presentation and writing.

5 Assignments

  • Students read selections from textbooks and articles, complete written and coding homework about every two weeks, and take two exams with in-class and take-home components.The first exam focuses on visualization and manipulation; the second centers on breast-cancer classification.Source: §5 · p. 10
  • Groups of three complete an open-ended project that must center on data, communicate something, and go beyond class material for an A.The project is submitted for proposal approval before groups proceed.Source: §5 · p. 10
  • The project ends with a 10-minute in-class presentation and a written blog post created in R Markdown.Examples of successful projects are provided in supplementary materials.Source: §5 · p. 10

6 Feedback

Feedback was nearly universally positive among 42 students, and some students soon applied course skills in careers and a data-analysis competition. The course used R rather than Python in this instance.

6 Feedback

  • Students reported learning “useful things,” and feedback from the 42 students was nearly universally positive.Source: §6 · p. 10
  • Two students interned at NIST, one accepted a full-time MIT Lincoln Laboratory offer, and a team with four course-takers won DataFest Best In Show.The team tokenized string-encoded dates in R, letting them move past a hurdle that slowed other groups.Source: §6 · p. 10
  • The course chose R over Python for this instance, while other instructors could make a different choice without disruption.Source: §6 · p. 10

7 Discussion

The discussion frames data science as both an opportunity and a challenge for statistics, given modern data and the need to use statistical models appropriately. It presents data science courses as a way to build skills for working with modern data.

7 Discussion

  • Computer scientists often distinguish data science from statistics by the heterogeneity and lack of structure of data, while statisticians continue to debate their relationship.Source: §7 · p. 11
  • Statisticians must teach appropriate model use and the hazards of ignoring assumptions, although strict adherence to idealized conditions can leave practitioners dissatisfied.Source: §7 · p. 11
  • Failing to embrace data science could cost statistics interested students and leave analysts without sufficient appreciation of statistics.The paper describes the latter risk as potentially catastrophic beyond the profession.Source: §7 · p. 11
  • Data science courses develop students’ abilities to work with modern data, with these skills shifting from desirable toward necessary.Source: §7 · p. 11

A Additional Topics

The additional topics extend the course’s practical work to spatial data, text, networks, and Big Data concepts. Students responded positively to spatial analysis and left with interest in further Big Data exploration, though the course did not cover Big Data in depth.

Spatial Analysis

  • Spatial analysis combined instruction on maps, shapefiles, projections, and choropleths with a project mapping Smith students’ hometowns.Students responded positively; one became interested enough in ArcGIS to frequent the Spatial Analysis Lab.Source: A Additional Topics · p. 15

Text Mining & Regular Expressions

  • Regular expressions let students use pattern matching to locate text patterns computers can scan quickly but do not understand unaided.A few class periods can establish basic understanding, not expertise, while giving students confidence to tackle similar problems later.Source: A Additional Topics · p. 15

Data Expo

  • The Data Expo brought community members’ data, questions, and hurdles to students as they began considering term projects.Speakers reinforced the value of combining technical analysis with communication for non-technical audiences, energizing students about employable skills.Source: A Additional Topics · pp. 15–16

Network Science

  • Network projects combined graph structure, shortest paths, SQL queries, and R graph libraries, including a “Kevin Bacon Oracle” built from IMDB data.Source: A Additional Topics · p. 16

Big Data

  • MTH 292 served as a precursor to Big Data, giving students experience with large data sets and practical tools but not absolute Big Data exposure.The final class introduced parallel processing, non-SQL storage, Hadoop, and MapReduce; the paper notes there was not enough time for depth.Source: A Additional Topics · p. 16

B Projects

The student projects applied data science to questions involving weather and social media, and networks of Smith College alumnae in mathematics. The Smathies project also aimed to provide undergraduates with relevant information about institutions and advisors.

Projects

  • The weathR project examined connections between Twitter posts and external weather conditions.Source: B Projects · p. 17
  • The Smathies project explored how Smith alumnae who earned PhDs in mathematics or related fields connected to advisors, advisees, and co-authors.Source: B Projects · p. 17
  • The Smathies network project aimed to help undergraduates retrieve relevant information while exploring how connected alumnae are to the wider mathematics world.The paper presents two visual depictions of the network.Source: B Projects · p. 17

C Feedback

Students reported that the course taught useful skills and helped them appreciate the challenges of working with data. They also valued the Data Expo’s real-world speakers and job-related perspective.

Feedback

  • All students agreed they were learning useful things, including new ways to think, ask questions, and approach project problems.Students also cited the variety of material and examples.Source: C Feedback · p. 17
  • The Data Expo helped students appreciate the challenges people face when working with data.Source: C Feedback · p. 17
  • All students enjoyed the Data Expo and praised its guest speakers; one group also valued the job opportunities and career examples.Source: C Feedback · p. 17
  • The project received an Honorable Mention in the 2014 Undergraduate Statistics Class Project Competition.Source: C Feedback · p. 17

D A Note to Prospective Instructors

Prospective instructors need willingness to learn and comfort troubleshooting in R, while students need prior programming experience and time for demanding assignments. The course’s pace may be adjusted, and support mechanisms can help prevent frustration from becoming helplessness.

Instructor preparation

  • Teaching requires willingness to learn alongside students and enough comfort with R to troubleshoot; student teaching assistants and office hours can provide support.A lack of support when students hit immovable roadblocks can allow frustration to become helplessness.Source: D A Note to Prospective Instructors · p. 18
  • Visualization and machine-learning material can be learned from books, while SQL basics can be learned through tutorials or self-study, with practice important.The passage notes SQL subtleties beyond the course are unlikely to arise.Source: D A Note to Prospective Instructors · p. 18

Student preparation

  • Prior programming experience is essential for students, although prior R experience is not required.The instructor reports that computer science majors with weaker statistics backgrounds usually fared better than students with stronger statistics backgrounds but less programming experience.Source: D A Note to Prospective Instructors · p. 18

Course demands

E Sample Exam Questions

The sample exam asks students to reason about data management, compare SQL with R, and use a traffic simulator to plan confidence intervals. Its questions emphasize choosing and specifying operations for concrete tasks.

Data management

  • The address-book problem asks students to combine student and house data, retain every student using an appropriate join, and filter the result to the Class of 2014.Students must describe the operations and how they are specified, without writing code.Source: E Sample Exam Questions · p. 19

SQL and R

Simulation

  • The traffic-simulator question asks how to find confidence intervals for mean waiting time under specified conditions and for an average day.The examples include Mondays at 56 degrees and a generic day regardless of weekday or temperature.Source: E Sample Exam Questions · p. 21
Loading 1503.05570v1…