PaperlayerView exact revision on arXiv

Source-linked AI summary

The Future is Big Graphs! A Community View on Graph Processing Systems

Sherif Sakr, Angela Bonifati, Hannes Voigt, Alexandru Iosup, Khaled Ammar, Renzo Angles, Walid Aref, Marcelo Arenas, Maciej Besta, Peter A. Boncz, Khuzaima Daudjee, Emanuele Della Valle, Stefania Dumbrava, Olaf Hartig, Bernhard Haslhofer, Tim Hegeman, Jan Hidders, Katja Hose, Adriana Iamnitchi, Vasiliki Kalavri, Hugo Kapp, Wim Martens, M. Tamer Özsu, Eric Peukert, Stefan Plantikow, Mohamed Ragab, Matei R. Ripeanu, Semih Salihoglu, Christian Schulz, Petra Selmer, Juan F. Sequeda, Joshua Shinavier, Gábor Szárnyas, Riccardo Tommasini, Antonino Tumeo, Alexandru Uta, Ana Lucia Varbanescu, Hsiang-Yun Wu, Nikolay Yakovets, Da Yan, Eiko Yoneki

arXiv:2012.06171v1cs.DCcs.DB

TL;DR

Graph-processing systems cannot yet support complex pipelines, and no single application can unite the diverse community around one use case. The paper develops a community vision across abstractions, ecosystems, and performance; its proposed directions remain predictions whose value only time can establish.

  • Problem

    Graph-processing systems cannot support complex pipelines, and there is no single most-common application around which the community can unite.

    Source: §1 · p. 2

  • Method

    The paper takes a community approach to framing next-decade graph-processing directions across abstractions, ecosystems, and performance.

    Source: §5 · p. 11

  • Results

    The paper’s vision covers abstractions, ecosystems, and performance rather than centering on one application.

    Source: §5 · p. 11

  • Takeaways & Limitations

    The paper presents its proposed directions as predictions whose value only time can establish.

    Source: §5 · p. 11

    Whether the proposed predictions provide worthwhile directions to the community remains unknown.

    Source: §5 · p. 11

Abstract

from arXiv · show

Graphs are by nature unifying abstractions that can leverage interconnectedness to represent, explore, predict, and explain real- and digital-world phenomena. Although real users and consumers of graph instances and graph workloads understand these abstractions, future problems will require new abstractions and systems. What needs to happen in the next decade for big graph processing to continue to succeed?

1 INTRODUCTION

Current graph-processing technology cannot support the complex pipelines needed to process multiple graph datasets. The paper frames a next-decade vision around abstractions, ecosystem coordination, and performance.

1 INTRODUCTION

  • The diversity of graph models and systems fragmented the market, while the absence of a killer application leaves no single use case to unite the community.The authors point to efforts to bring together programming languages, ecosystem structure, and performance benchmarks.Source: §1 · p. 2
  • Current graph-processing technology cannot support a full pipeline across multiple graph datasets.The proposed pipeline spans data sources, OLTP and OLAP operations, and downstream consumers, with intermediate results feeding earlier steps.Source: §1 · pp. 2–3
  • The paper presents a lattice of expected data models and query languages to clarify principles for graph data extraction, exchange, processing, and analysis.The lattice is intended to accommodate future graph data models and query languages.Source: §1 · p. 3
  • The proposed ecosystem would govern big-graph systems and enable tuning across OLAP/OLTP operations, workloads, standards, and performance needs.The paper treats future graph-processing systems as more complex than those of the prior decade.Source: §1 · pp. 3–4
  • Performance work must address both meaningful, tractable, reproducible experiments and the trade-off between scalability, portability, and interoperability.The paper identifies these as methodological and practical challenges, respectively.Source: §1 · p. 4

2 ABSTRACTIONS

The paper frames graph processing abstractions as a choice among models with different expressiveness, costs, and uses, and identifies open questions about understanding, comparing, and connecting them. It also proposes directions for combining logical and statistical approaches and developing a composable graph algebra.

2 ABSTRACTIONS

  • The “abstraction soup” leaves users to choose among similar graph models with differing expressiveness, cost, and intended uses.The paper identifies choosing a model per use case and mastering interoperability as key challenges.Source: §2 · p. 4, §2.1 · p. 4

2.1 Understanding data models

  • Research should examine how people conceptualize data and compare models’ expressive power, costs, and benefits across graph, relational, and document families.A lattice can represent how added characteristics yield richer graph-model expressiveness.Source: §2.1 · p. 4

2.2 Logic-based and declarative formalisms

  • The paper calls for combining statistical learning with graph processing and reasoning, including using learned graph representations in indexing, querying, and logical reasoning.It also raises questions about the formalisms for integration and how to weigh the two mechanisms.Source: §2.2 · p. 6

2.3 Algebraic operators for graph processing

  • A graph algebra could begin with relational-like operators for first-order queries and graph pattern matching, while supporting composition across relations and graphs.Current engines combine algebraic operators with ad-hoc graph algorithms, complicating implementation and affecting performance.Source: §2.3 · p. 6

Sidebar B: In-Depth: Known properties of graph-processing workloads

Graph-processing workloads vary across domains, methods, programming models, scale, and pipeline tasks; their irregularity reflects interactions among hardware, platform, algorithm, and dataset. The sidebar also describes tractability constraints on query-language expressiveness and calls for algebraic operators usable across environments and methods.

Known properties of graph-processing workloads

  • Workloads span diverse domains, graph methods, programming models, sizes up to trillion-edge graphs, and query or process selectivities.They can mix data-intensive and compute-intensive phases, with performance affected by dataset, algorithm, and platform interdependencies captured by the HPAD Law.Source: Sidebar B: In-Depth: Known properties… · p. 7
  • Graph pipelines combine OLTP and OLAP with tasks such as ETL, visualization, cleaning, mining, debugging, testing, and synthetic graph generation.Scalability, interactivity, and usability affect how users construct workloads.Source: Sidebar B: In-Depth: Known properties… · p. 7
  • A fully Turing-capable query language is considered impractical because it entails tractability and feasibility problems.The paper identifies as desirable algebraic operators that work centrally and distributively and can serve graph algorithms and machine-learning models.Source: Sidebar B: In-Depth: Known properties… · p. 7

3 ECOSYSTEMS

Graph-processing ecosystems couple systems built for different purposes and processes, with workloads spanning complex pipelines and domain-specific tools. The paper identifies directions involving standards, a reference architecture, workload-aware scalability, and dynamic and streaming graph support.

3.1 Workloads in graph processing ecosystems

  • Workloads combine heterogeneous queries and algorithms over diverse datasets, linking graph processing to domain-specific and general processing ecosystems.Examples include science and engineering, business analytics, social media, and machine learning.Source: §3.1 · p. 7

3.2 Standards for data models and query languages

  • Standards should cover OLTP and OLAP data models, manipulation and definition languages, and exchange formats, while reflecting industry practice and enabling adoption.The paper identifies graph algorithms, analytical APIs, workflow definitions, embeddings, and benchmarks as worthwhile longer-term standardization targets.Source: §3.2 · pp. 7–8
  • GQL is described as supporting composable queries over potentially overlapping graphs, with graph updates, enhanced path queries, ranking, and path aggregation.Its syntax combines SQL-style constructs with visual graph patterns.Source: §3.2 · p. 8

3.3 Reference architecture

  • The proposed reference architecture draws on a datacenter architecture, with resource infrastructure, cross-resource services and management, and specialized back-end and front-end layers.The authors identify capturing the evolving graph-processing field as a challenge.Source: Figure 3 · p. 8, §3.3 · p. 9

3.4 Beyond scale-up vs. scale-out

  • A scalability continuum would let an ecosystem automatically choose how and where to run diverse workloads on heterogeneous infrastructure while meeting Service Level objectives.Possible techniques include data and work partitioning, migration, offloading, replication, and elastic scaling.Source: §3.4 · p. 9

3.5 Dynamic and streaming aspects

  • Future ecosystems should handle dynamic and streaming graphs, including updates and unbounded streams that may not fit entirely in system state.The paper calls for complex graph data streams, advanced analytics and ML operators, and query operators that account for recursion.Source: §3.5 · p. 9
  • As graph ecosystems become embedded in larger pipelines, discovering, understanding, and controlling their dynamics remains an open challenge.The paper asks what new phenomena may emerge alongside performance variability, cascading failures, and autoscaling resources.Source: §3.5 · p. 9

4 PERFORMANCE

The paper identifies performance challenges in measurement, comparison, portability, and reproducibility, made harder by complex graph workloads and runtimes. It proposes reproducible experimental methods, better portability and interoperability, and a Big Graph Memex for operational information.

4 PERFORMANCE

  • Graph processing lacks widely used performance metrics beyond response time, and comparison across architectures, tuning processes, and reproducibility remains challenging.These challenges are compounded by complex workloads and pipelines.Source: §4 · p. 9

4.1 Benchmarks, performance measurement, methodological aspects

  • Graph benchmarks must account for complex workloads and data pipelines, since even graph degree-distribution variations can significantly affect performance and lack of interoperability hinders fair comparisons.The paper notes that indexing and sampling might help improve benchmarking.Source: §4.1 · p. 10
  • The paper calls for reproducible experimental methodologies, faithful metrics, combined-operation workloads, and benchmarks for pipelines that include machine learning and simulation.Large-scale experiments can be difficult to implement, deploy, and run within reasonable time and cost; LDBC could curate benchmark sharing and audit practice.Source: §4.1 · pp. 9–10

4.2 Specialization vs. portability and interoperability

  • The paper highlights tension between specialization for performance and portability across heterogeneous hardware, noting that no standard graph library or query language currently exists.It also identifies integration with machine learning, data mining, simulation, and decision-making tools as important but unsupported by existing frameworks.Source: §4.2 · p. 10

4.3 A memex for big graph processing systems

  • A Big Graph Memex could collect, archive, and retrieve operational information about graph systems.The paper identifies potential uses including addressing performance issues, enabling creative designs and automation, and supporting meaningful, reproducible testing.Source: §4.3 · p. 11

5 CONCLUSION

The conclusion asks how future big graph processing and database systems can provide the scalable, efficient, and diversified querying and analytical capabilities users demand. The authors describe a community-shaped view of what comes next, organized around abstractions, ecosystems, and performance, while noting that the value of their predictions remains to be seen.

5 CONCLUSION

  • The conclusion asks how future big graph processing and database systems can meet demands for scalable, efficient, and diversified querying and analytical capabilities.Graphs are described as a mainstay abstraction in current data-processing pipelines.Source: §5 · p. 11
  • The authors took a community approach, beginning with a Dagstuhl Seminar and then shaping structured connections.The article focuses on three interrelated elements: abstractions, ecosystems, and performance.Source: §5 · p. 11
  • The article offers a view of what comes next across abstractions, ecosystems, and performance.The authors say only time can tell whether their predictions provide worthwhile directions for the community.Source: §5 · p. 11
Loading 2012.06171v1…