PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Nature and Types of Anomalies: A Review of Deviations in Data

Ralph Foorthuis

arXiv:2007.15634v5cs.DBcs.AIcs.LGstat.OT

TL;DR

Anomaly definitions are often vague and domain-dependent, and comprehensive, concrete overviews of anomaly types have been lacking. This literature review proposes a data-centric typology organized by five dimensions and presents a comprehensive account of anomaly types. The typology offers a basis for evaluating detection algorithms and understanding anomaly-analysis results.

  • Problem

    Anomaly definitions are often vague and domain-dependent, and existing work lacks a comprehensive, concrete overview of anomaly types.

    Source: §1 · p. 2

  • Method

    The study reviews the literature and develops a qualitative, data-centric typology organized around five dimensions of anomalies.

    Source: §1 · pp. 2–3

  • Results

    The study presents an integrative overview of concrete anomaly types and subtypes using the typology’s five fundamental dimensions.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The typology supports systematic evaluation of which anomaly types detection algorithms can identify and helps make analysis results more understandable post hoc.

    Source: §1 · p. 3

    The typology defines anomaly types from intrinsic data properties, without reference to external causes, domain knowledge, or analyst decisions.

    Source: §1 · p. 3

Abstract

from arXiv · show

Anomalies are occurrences in a dataset that are in some way unusual and do not fit the general patterns. The concept of the anomaly is typically ill-defined and perceived as vague and domain-dependent. Moreover, despite some 250 years of publications on the topic, no comprehensive and concrete overviews of the different types of anomalies have hitherto been published. By means of an extensive literature review this study therefore offers the first theoretically principled and domain-independent typology of data anomalies and presents a full overview of anomaly types and subtypes. To concretely define the concept of the anomaly and its different manifestations, the typology employs five dimensions: data type, cardinality of relationship, anomaly level, data structure, and data distribution. These fundamental and data-centric dimensions naturally yield 3 broad groups, 9 basic types, and 63 subtypes of anomalies. The typology facilitates the evaluation of the functional capabilities of anomaly detection algorithms, contributes to explainable data science, and provides insights into relevant topics such as local versus global anomalies.

1 Introduction

The paper addresses the lack of a concrete, integrative account of anomaly manifestations by proposing a data-centric typology organized around five dimensions.

1 Introduction

  • Anomalies may be noise that hinders analysis or signals that analysts seek, and they appear in many forms.The paper describes occurrences ranging from static entities to time-related events and from atomic cases to groups.Source: §1 · p. 1, Figure 1 · p. 2
  • The study presents an integrative, data-centric typology that describes anomaly types through five fundamental dimensions.It offers a qualitative overview of the different anomaly types and subtypes.Source: §1 · pp. 2–3
  • Anomaly detection lacks a comprehensive, concrete framework, while general anomaly definitions remain vague and domain-dependent.Existing conceptualizations can be situation-specific or too abstract to support tangible understanding or algorithm evaluation.Source: §1 · p. 2
  • The typology defines anomaly types using intrinsic data properties, without relying on external factors or domain knowledge.The paper states that these anomalies can in principle be detected by unsupervised methods based on the data at hand.Source: §1 · p. 3
  • The typology supports systematic evaluation of detection algorithms and can make anomaly-analysis results more understandable post hoc.It also helps determine whether detected cases constitute true deviations and supports testing systems with anomalous data.Source: §1 · p. 3

2 Theory

The theory section establishes the terms and data structures used to discuss anomalies, then reviews existing classifications and their limits relative to the paper’s typology.

2.1 Key terms and concepts

  • An anomaly in unsupervised data analysis is a case or group that is unusual and does not fit the general patterns of most data.Cases may also be called data points, rows, records, or observations; attributes may be variables, columns, fields, dimensions, or features.Source: §2.1 · p. 4
  • Dependencies may link attributes or individual cases, with time series, spatial, graph, and hierarchical data as examples of dependent data.Independent data, such as an i.i.d. cross-sectional sample, treats each row as a stand-alone observation.Source: §2.1 · p. 4
  • The section distinguishes data structures including cross-sectional, sequence, graph, spatial, and spatio-temporal data.An aggregate treats multiple rows as a group, such as a subsequence or subgraph, while a typology uses two or more classificatory principles to distinguish concept types.Source: §2.1 · pp. 4–5

2.2 Related work

  • Prior classifications include broad categories such as point, contextual, and collective anomalies, alongside purpose-specific classes for sequences, regression, trajectories, and graphs.Examples include sequence level shifts, regression high-leverage points, trajectory positional outliers, and anomalous graph vertices, edges, or subgraphs.Source: §2.2 · pp. 6–7
  • Existing overviews are either too abstract for concrete understanding or limited to particular purposes, and generally lack principled partitions of the anomaly space.The paper characterizes its framework and predecessors as the first overall typology providing a comprehensive overview of concrete anomaly types.Source: §2.2 · p. 7
  • The typology distinguishes anomalies through intrinsic data properties rather than algorithm choices, analyst decisions, or unknown external causes.The paper argues that external causes may be difficult to ascertain and can make distinctions subjective; it therefore keeps detection techniques outside its core focus.Source: §2.2 · p. 7

3 A Typology of Anomalies

This section presents a data-oriented typology that classifies anomalies into broad groups, basic types, and subtypes. It covers atomic multivariate anomalies and aggregate deviations involving sequences, subgraphs, spatial regions, and groups of mixed-type cases.

3.1 Classificatory principles

  • Five dimensions—data type, attribute relationship cardinality, anomaly level, data structure, and data distribution—organize nine basic anomaly types and further subtypes.Data type distinguishes quantitative, qualitative, and mixed attributes; cardinality distinguishes univariate from multivariate relationships; anomaly level distinguishes atomic from aggregate cases. Structure and distribution further subdivide types, with distribution focusing on density and dispersion.Source: §3.1 · pp. 7–9
  • The framework makes atomic and aggregate anomalies explicit, refining an earlier typology whose multivariate types were too broad and yielding nine basic types.The classificatory dimensions matter for analysis and storage: for example, dependent data may require accounting for autocorrelation, while multidimensional pattern discovery must address inter-variable relationships and scaling.Source: §3.1 · p. 9

3.2 Overview of anomaly types and subtypes

  • The three groups are atomic univariate cases with individually deviant values, atomic multivariate cases with deviant relationships, and aggregates that deviate collectively.Aggregate anomalies are generally the most difficult to describe and detect because relationships between attributes and cases help define the group; a deviant subsequence is one example.Source: §3.2 · p. 10
  • The typology assumes parsimonious data and the original dataset, includes relevant management and dependency attributes, and favors the simplest type that captures a deviation.Transformations may be applied, but typification then concerns the derived dataset or remains agnostic about transformations performed within the detection algorithm.Source: §3.2 · p. 10

3.2.1 Atomic univariate anomalies

  • Atomic univariate numerical anomalies include extreme tail values and isolated intermediate values, which traditional methods for extremes may fail to detect.Skewed, leptokurtic, or heavy-tailed distributions can produce more extreme cases than normal distributions, and masking and swamping are also distribution-related concerns.Source: §3.2.1 · p. 12
  • Atomic univariate categorical anomalies include rare or unique classes and frequent classes that are unusual under a skewed distribution; mixed anomalies combine individually deviant numeric and categorical values.Rarity may require an arbitrary threshold, while mixed anomalies do not require joint analysis: each contributing attribute value is anomalous in its own right.Source: §3.2.1 · p. 13

3.2.2 Atomic multivariate anomalies

  • Atomic multivariate numerical anomalies combine individually ordinary values unusually, appearing either at a cloud’s periphery or enclosed by normal data.Some methods can detect peripheral points but are not geared toward enclosed points.Source: §3.2.2 · p. 13
  • Atomic multivariate anomalies include locally isolated cases, abrupt sequence spikes relative to temporal context, and unusual quantitative spatial or spatio-temporal points.A local density anomaly is isolated within its neighborhood rather than globally; a single-observation additive anomaly is a short-lived spike, while slower events may be aggregates. Spatial examples include unusual pixels or voxels, though such anomalies are usually regions.Source: §3.2.2 · p. 14
  • Type V anomalies are uncommon combinations of categorical values or unusual categorical relationships among vertices and edges in graphs.A combination can be anomalous even when its individual values are common; text and market-basket cases can be represented without word order, while graph cases can depend on topology or a vertex’s position in its community.Source: §3.2.2 · pp. 14–15
  • Type VI anomalies jointly involve quantitative and categorical attributes, including locally incongruous labels, weighted graph vertices or edges, dynamic graph events, and spatial or spatio-temporal cases.Examples include graph vertices or edges that are unusual relative to neighbors, changes in graph membership or structure over time, and locations whose properties are unexpected in their spatial or temporal neighborhood.Source: §3.2.2 · pp. 16–18

3.2.3 Aggregate anomalies

  • Type VII numerical aggregate anomalies include deviant cycles, temporary changes, level shifts, innovational outliers, trend changes, and changes in variation.These group-level deviations are often subsequences of time series; a temporary change may start or recover gradually, while a level shift is a permanent step spanning at least two consecutive observations.Source: §3.2.3 · pp. 18–19
  • Type VII also covers unusual spatial or spatio-temporal regions, collective motion patterns, point-based aggregates, and clusters with deviant group-level distributions.Spatial anomalies may be locally unusual but globally normal, while i.i.d. data can contain either micro-clusters of individually deviant points or groups distinguished by variance, covariance, mean, frequency, density, or shape.Source: §3.2.3 · pp. 19–20
  • Type VIII categorical aggregates include deviant text segments, unusual graph substructures, and relational-database collectives whose categorical relationships depart from expected patterns.Text aggregates can differ in topic, content, style, or tone; graph examples include unusual paths or splits, and relational cases can reflect unexpected links or orphaned records.Source: §3.2.3 · pp. 20–21
  • Type IX mixed aggregates include anomalous symbolic subsequences, such as class changes or cycles, whose detection relies on linking consecutive cases.Class changes can reflect a new symbolic value or pattern, including changes in real-estate categories, market-basket co-occurrence, or news topics; these may coincide with changes in the dataset’s underlying distribution.Source: §3.2.3 · pp. 21–22
  • The typology describes aggregate sequence anomalies ranging from unusual symbolic subsequences to deviant complete sequences, including anomalous process traces and text whose word order changes its meaning.Examples include repeated login-password events that may indicate hacking, DNA from another organism, and text deviations relevant to fake-review detection.Source: §3.2.3 · p. 22
  • Panel time-series anomalies include isolated or sustained deviations in shift, shape, amplitude, trend, or variation, with seasonality captured by shape and amplitude subtypes.The same subtypes can describe trajectory anomalies, such as ships deviating from routes or taxis taking unusual turns.Source: §3.2.3 · pp. 22–23
  • Graph aggregate anomalies include deviant subgraphs and communities whose relationships, weights, or attributes differ, as well as evolutionary paths or clades that depart from expected properties.In molecular-clock trees, unusually short or long root-to-leaf paths can be anomalous; unstable clades or conspicuous branch lengths may indicate tree-building errors.Source: §3.2.3 · p. 23
  • Dynamic graph anomalies include changing subgraph patterns such as appearance, disappearance, merging, splitting, growth, or eccentric behavior, which can reveal network faults or fraudulent trading rings.The described fraud group trades exclusively among itself before selling the increased-price stocks to outsiders.Source: §3.2.3 · pp. 23–24
  • Spatial and spatio-temporal aggregate anomalies include unusual regions or flows, identified relative to nearby areas, spatial neighbors, and temporal patterns.Examples include unusually dense counties, a rapidly increasing local disease risk, and river-sensor flow patterns that may indicate floods or chemical spills.Source: §3.2.3 · p. 24
  • Mixed-data aggregate anomalies comprise groups of cases with unusual individual categorical and numerical values or unusual group-level characteristics and distributions.A cluster of neighboring points with an anomalous distribution of categorical values illustrates the distribution-based subtype.Source: §3.2.3 · p. 24
  • The typology introduces nine basic anomaly types through tangible subtypes and examples, with stable basic types and an extensible set of subtypes.Source: §3.2.3 · p. 24

4 Discussion

The discussion recommends classifying deviations by the simplest applicable subtype, while allowing additional classifications for other attribute subsets. It also presents the typology as a basis for transparent algorithm evaluation and more systematic test sets.

Deciding on the anomaly type

  • Classify a deviation using the simplest applicable subtype, while allowing it to qualify as another type for a different attribute subset.The typology ranks univariate over multivariate, single-data-type over mixed, independent over dependent, and atomic over aggregate definitions as simpler.Source: §4 · pp. 24–25

Evaluation of algorithms

  • The typology can clarify which anomaly subtypes an algorithm detects and guide test sets that systematically include relevant subtypes.The authors recommend injecting relevant subtypes into real-world or simulated data, rather than relying only on existing benchmarks or minority classes.Source: §4 · p. 25

I II III IV V VI Remarks

The discussion connects the typology to algorithm evaluation, explainable analysis, and several understandings of locality. It also considers how data distributions and domain-specific patterns shape anomaly classifications and detection.

Remarks

  • Implementation notes flag that ST-IIIa cannot be directly distinguished from ST-Ia using quantitative data alone, while other entries note rescaling and dimensionality concerns.The supplied remarks also state that ST-IIb cases receive high rather than low anomaly scores.Source: I II III IV V VI Remarks · p. 25

Evaluation of algorithms

  • The typology supports subtype-focused algorithm evaluation and systematic test sets; Table 3 organizes evaluation characteristics by anomaly type.Relevant characteristics can include subtype detection capability and metrics such as AUC, F1, and sensitivity.Source: I II III IV V VI Remarks · p. 26, Table 3 · p. 26

Explainable anomaly detection

  • Subtype labels and the five dimensions clarify how detected cases differ from normal data, and can inform different follow-up actions.The discussion contrasts threshold rules for extreme values with combined rules for multivariate cases, and notes that some subtle deviations may be ignored.Source: I II III IV V VI Remarks · pp. 26–27

Local anomalies

  • Locality can mean conditional deviation by attribute relationships, relative isolation within a neighborhood, or deviation within a dependent-data context.For dependent data, spatial positions or other linking attributes can define neighborhoods in which a case is unusual despite being normal elsewhere.Source: I II III IV V VI Remarks · p. 27

Other classifications

  • The typology distinguishes global extreme time-series values from local deviations and excludes classifications grounded in external causes rather than observed data properties.It contrasts its 63 concrete subtypes with a broader classification whose classes may overlap and lack classificatory principles.Source: I II III IV V VI Remarks · p. 28

Conceptual variations

  • Anomalies need not occupy low-density regions: small clusters, high-density regions, and repeated categorical values can be anomalous in some settings.High-density anomalies may hide among otherwise normal cases, making their identification a balance between anomalousness and normalness.Source: I II III IV V VI Remarks · p. 29

Domain-specific anomalies

  • Domain-specific deviations may require supervised or rule-based detection when they depend on patterns relevant only to a particular domain or problem.The discussion gives maritime, stock-market, and network-traffic examples.Source: I II III IV V VI Remarks · p. 29

5 Conclusion

The conclusion presents a data-centric, domain-independent typology built from five dimensions and organized into three groups, nine basic types, and 63 subtypes. It describes uses in analysis and algorithm evaluation, while identifying a scope centered on intrinsic data properties and unsupervised detection.

5 Conclusion

  • The study presents a theoretically principled, data-centric, domain-independent typology grounded in an extensive literature review and five data-oriented dimensions.The dimensions are data type, cardinality of relationship, anomaly level, data structure, and data distribution.Source: §5 · p. 29
  • The typology has three broad groups, nine basic types, and 63 subtypes, with the subtype set open to future extension.Subtypes are positioned within the framework's nine main type cells and described using data structure and distribution.Source: §5 · p. 30
  • The typology can support explanation of analysis results, transparent algorithm evaluation, subtype-based test sets, and the organization of research and teaching.The conclusion also argues for attending to understanding and explaining anomalies, not only detecting them.Source: §5 · p. 30
  • The study focuses on anomalies characterized by intrinsic data properties and detectable with unsupervised methods; extending scope to rare classes requiring supervised methods is future work.Further work could also examine distribution-based distinctions and anomalies in graphs and other complex data structures.Source: §5 · p. 30
Loading 2007.15634v5…