Source-linked AI summary
Practical Whole-System Provenance Capture
Thomas Pasquier, Xueyuan Han, Mark Goldstein, Thomas Moyer, David Eyers, Margo Seltzer, Jean Bacon
TL;DR
Existing whole-system provenance systems faced adoption barriers involving maintainability, data volume, portability, and system scope. CamFlow provides a modular, maintainable capture mechanism for PaaS clouds with application-specific policies and distributed streaming, and demonstrates its use in provenance applications. The authors report maintainability across Linux kernel revisions, while noting that formal completeness and expressiveness comparisons remain unevaluated.
Problem
Prior whole-system provenance systems faced adoption barriers, including difficult portability, excessive data generation, and limited applicability across systems.
Method
CamFlow captures whole-system provenance through a modular Linux-based mechanism, tunes capture policies to application requirements, and streams provenance to tenant applications.
Results
CamFlow was maintained across multiple Linux kernel major revisions with only a few hours of effort, and the paper demonstrates provenance applications including application-level log integration.
Takeaways & Limitations
CamFlow provides a standards-compliant, deployable approach for capturing provenance in PaaS environments and supporting varied provenance applications.
Takeaways & Limitations
The paper does not formally evaluate whether CamFlow’s provenance graph is as complete or expressive as those produced by alternative implementations.
Abstract
from arXiv · showhide
Data provenance describes how data came to be in its present form. It includes data sources and the transformations that have been applied to them. Data provenance has many uses, from forensics and security to aiding the reproducibility of scientific experiments. We present CamFlow, a whole-system provenance capture mechanism that integrates easily into a PaaS offering. While there have been several prior whole-system provenance systems that captured a comprehensive, systemic and ubiquitous record of a system's behavior, none have been widely adopted. They either A) impose too much overhead, B) are designed for long-outdated kernel releases and are hard to port to current systems, C) generate too much data, or D) are designed for a single system. CamFlow addresses these shortcoming by: 1) leveraging the latest kernel design advances to achieve efficiency; 2) using a self-contained, easily maintainable implementation relying on a Linux Security Module, NetFilter, and other existing kernel facilities; 3) providing a mechanism to tailor the captured provenance data to the needs of the application; and 4) making it easy to integrate provenance across distributed systems. The provenance we capture is streamed and consumed by tenant-built auditor applications. We illustrate the usability of our implementation by describing three such applications: demonstrating compliance with data regulations; performing fault/intrusion detection; and implementing data loss prevention. We also show how CamFlow can be leveraged to capture meaningful provenance without modifying existing applications.
1 INTRODUCTION
CamFlow captures whole-system provenance in PaaS clouds, recording interactions among processes and kernel objects while streaming provenance to tenant applications. It targets maintainability, application-specific capture, distributed integration, and provenance-aware analysis.
- Motivation: Data provenance records relationships among data entities, transformations, and associated agents, supporting reproducibility and regulatory compliance.Whole-system provenance extends this scope to system objects and interactions across execution.
- Architecture: CamFlow embeds a provenance capture module in the cloud-managed OS to record process–kernel-object interactions and network packet information.Captured data is streamed across machines to tenant-provided applications that process or store the provenance.
- Implementation: CamFlow uses a self-contained Linux kernel module built on LSM and NetFilters to improve maintainability across current Linux kernel releases.The authors report maintaining CamFlow across multiple major revisions, unlike PASS and LPM, which were not ported to newer releases.
- Selective capture: Application-specific policies finely tune which provenance data CamFlow captures, addressing the volume of data generated by whole-system capture.The policy approach extends selective capture beyond security-sensitive objects to the requirements of individual provenance applications.
- Applications: The paper demonstrates provenance applications and extensions for containers, shared memory, and application-level logs without requiring existing applications to be modified.The system can integrate logs into the provenance graph and stream binary or PROV-JSON data through messaging middleware.
- Provenance model: CamFlow represents kernel objects as state-versioned provenance nodes, preserving an acyclic graph as objects change state.Nodes sharing an underlying object retain common identifiers, while attributes can change with incoming information.
4 CAPTURING SYSTEM PROVENANCE
CamFlow captures whole-system provenance in the kernel, transfers it to user space, and streams it to tenant applications. Its architecture also supports application-level annotations and integration with existing logs.
- Design goals: CamFlow’s implementation strategy emphasizes maintainability, reuse of existing kernel mechanisms, avoidance of duplication, and possible mainline Linux integration.The design builds on earlier systems while addressing their maintenance difficulties through LSM and NetFilter support.
- Architecture: CamFlow uses LSM and NetFilter hooks to capture provenance, relayfs to transfer records to user space, and middleware to stream them to applications.The kernel writes records into a relayfs-managed ring buffer; a user-space library can serialize binary records into PROV-JSON.
- Capture mechanism: The LSM hook ordering lets CamFlow record flows only after other access-control checks have allowed the system call.CamFlow is arranged to run last so blocked flows are not recorded.
- Cross-host capture: CamFlow represents packets as provenance entities and captures incoming packets through LSM hooks and outgoing packets through NetFilter.Packet identifiers use immutable IP and protocol fields, with optional payload capture.
- Application-level provenance: CamFlow provides an API that associates application provenance with system objects while maintaining system-wide identifier uniqueness and preventing infrastructure-induced cycles.Applications can disclose provenance when they have a system-level descriptor such as a file descriptor.
- Application-level provenance: Existing application logs can be integrated into system provenance graphs, helping explain interactions such as client request/response exchanges.The paper uses this infrastructure to insert httpd logs into a system provenance graph.
5 TAILORING PROVENANCE CAPTURE
Whole-system provenance can become much larger than the primary data and make provenance applications slower. CamFlow therefore supports tailoring capture to an application’s specific query needs.
- 5 TAILORING PROVENANCE CAPTURE: CamFlow addresses provenance volume by allowing detailed capture policies tailored to the exact requirements of each provenance application.This extends selective capture beyond security-sensitive objects to application-specific needs.
1 [provenance]
The capture policy enables provenance capture and configures machine identity, object coverage, and exchanged-data recording.
- 1 [provenance]: The policy enables provenance capture and configures a machine identifier, kernel-object coverage, and recording of exchanges with a local server.The supplied policy fragments show these settings as separate entries.
31 [group]
CamFlow policies can be specified through a configuration file and can target, propagate, filter, or taint provenance according to object and user-defined criteria.
- 31 [group]: Users specify capture policies through a command-line interface or policy configuration file.The paper illustrates this with a capture-policy listing.
- 31 [group]: Selective policies track specified targets and objects that interact with them or subsequently receive propagated information.Propagation resembles taint tracking by automatically tracking objects receiving information from tracked objects.
- 31 [group]: Policies can identify opaque or tracked objects by pathname, network address, LSM security context, control group, or user and group ID.The available criteria are expanded based on user feedback.
- 31 [group]: CamFlow can exclude selected node or edge types, such as directory operations or read-permission checks, during event capture.The filter compares endpoint types for each recorded edge and discards matching edges.
- 31 [group]: Taints are arbitrary 64-bit integers that can be attached to kernel objects and used for rapid policy-enforcement decisions.Kernel objects may hold an arbitrary number of taints under policy-specified restrictions.
- 31 [group]: Tailoring makes provenance incomplete, so applications must identify the minimum sufficient information required for correctness.The paper explicitly warns that tailoring excludes part of the provenance record.
6 EVALUATION
CamFlow’s evaluation examines maintainability, performance overhead, selective-capture data reduction, and provenance completeness. Results indicate easier kernel maintenance, modest benchmark overhead, configurable data volume, and an unresolved formal comparison of completeness and expressiveness.
- 6.1 Maintainability: PASS’s second version required over two years and several dozen person months to transition across two minor Linux revisions.The effort included re-architecting kernel interaction, adding features, and changing kernel releases.
- 6.1 Maintainability: CamFlow was maintained across multiple Linux kernel major revisions with only a few hours of effort, whereas PASS and LPM were not ported to newer releases.The authors report modifying eight lines in /security/security.c during the 4.4.38-to-4.9 upgrade, with most patches applying without conflict.
- 6.2 Performance: Whole-system capture added minimal overhead to kernel unpack and build benchmarks and up to 11% overhead to Postmark.The authors state these results compare favorably with prior systems, while selective capture provides a better-performing option.
- 6.2 Performance: Cloud-based benchmarks incurred larger overheads, reaching a maximum of 22% for redis LPOP under whole-system capture.The comparison used an off configuration that included CamFlow in the kernel but captured no data, versus whole capture.
- 6.3 Generated Data Volume: CamFlow does not formally evaluate provenance storage strategy, and the authors identify full comparison of graph completeness and expressiveness against alternatives as an unresolved challenge.They note that quantitatively measuring provenance completeness or expressivity remains complex and ongoing.
7 EXAMPLE APPLICATIONS
CamFlow supports several provenance applications in PaaS clouds, including compliance monitoring, intrusion detection, and data loss prevention. Its selective capture and taint mechanisms reduce data or query costs while preserving application-specific forensic capabilities.
- Architecture: CamFlow streams captured provenance to tenant-provided applications that process or store it.The capture mechanism records process–kernel-object interactions and network packet information across machines.
- Compliance: Compliance auditors can monitor distributed provenance streams against information-flow constraints representing data-protection regulations.Auditors use label propagation to verify that streamed provenance respects specified constraints.
- Intrusion detection: FRAPpuccino builds per-application normal-behavior models during training and compares deployment-time provenance streams against them.Detected anomalies can trigger intrusion or fault analysis, while confirmed incidents support forensic root-cause investigation.
- Intrusion detection: The prototype detected known vulnerabilities involving arbitrary file upload, malicious packets, and a Ruby server out-of-memory exception.These examples cover privilege escalation, a tcpdump crash, and disrupted garbage collection.
- Intrusion detection: The intrusion-detection prototype remains ongoing work, with planned scalability improvements and further evaluation for discovering new vulnerabilities.Planned work includes streamed analysis on systems such as Apache Spark and retraining support.
- Data loss prevention: Selective capture restricts provenance to application-relevant edge types, while taint checks replace costly graph queries for data-loss prevention.The taint-based approach can preserve whole or selective provenance as forensic evidence while reducing computational overhead.
8 RELATED WORK
Related systems capture provenance at application, workflow, system-call, or kernel levels, each with different completeness, compatibility, and maintenance trade-offs. CamFlow uses standard Linux facilities and selective capture to provide a maintainable, extensible whole-system implementation.
- System-level provenance: Application- and workflow-level systems provide incomplete records, whereas system-level provenance records interactions between programs for more complex use cases.Earlier system-level approaches were difficult to maintain because capture functionality was distributed across kernel components.
- CamFlow architecture: CamFlow is self-contained and uses standard kernel features, making it easier to extend and maintain than earlier distributed kernel implementations.Its architecture adopts LSM and NetFilter-related facilities and supports modular integration with existing software.
- CamFlow architecture: CamFlow leverages newer LSM capabilities to stack security modules without duplicating features, improving maintainability and extensibility.This addresses compatibility limitations associated with earlier LSM-based provenance systems.
- Completeness: System-call approaches such as ProTracer combine logging and taint tracking but may lack guarantees that the provenance record is complete.CamFlow instead relies on system-level kernel integration while acknowledging that LSM coverage may not capture every information flow.
- Selective capture: Selective capture addresses the large volume of system-level provenance while remaining consistent with the provenance data model.This reduction is intended to support contexts where security policies do not identify all data of interest.
- Cloud deployment: The work presents an open-source maintained cloud implementation with demonstrated use cases, while application-level provenance for MapReduce or Spark remains complementary.Those frameworks could be incorporated through CamFlow’s integration mechanism.
9 CONCLUSION
CamFlow is presented as a flexible, efficient, and maintainable Linux provenance mechanism suitable for inclusion in PaaS cloud operating systems. Its tunable capture and standards-based modularity support scalability, applicability, and adoption.
- Conclusion: CamFlow uses a modular LSM-based architecture to include provenance in cloud operating systems for PaaS offerings.The mechanism is designed for Linux, which is widely used in cloud service offerings.
- Conclusion: Fine-grained capture policies help create a scalable system by tuning provenance to each application’s requirements.The standards-compliant approach is described as enhancing broad applicability and ease of adoption.
- Conclusion: The evaluation demonstrated low performance impact and broad applicability for the approach.
A.1 Running CamFlow
CamFlow can be run through a Vagrant-based Fedora demo or installed and updated through a Fedora package manager.
- Running CamFlow: The Vagrant demo requires cloning the repository, entering the Fedora directory, installing the vagrant-vbguest plugin, and running vagrant up.
- Running CamFlow: After starting the demo, users connect with vagrant ssh and check the installed version against the CamFlow head.
- Running CamFlow: The demo exposes audit and configuration logs at /tmp/audit.log and /tmp/camflow.clg.
- Running CamFlow: Alternatively, Fedora users can add the packagecloud repository and install CamFlow with the package manager.
A.2 Repeating Evaluation Results
The evaluation results can be reproduced by installing CamFlow, cloning the benchmark repository, preparing it, selecting a configuration, and running the tests. More accurate results may require running tests individually and rebooting after each test.
- Setup: Reproducing the evaluation requires a Linux kernel running CamFlow and the benchmark repository.The instructions begin by assuming CamFlow is installed, then direct users to clone the benchmark repository.
- Setup: Run `make prepare` before selecting the test configuration.The listed commands place preparation before configuration selection.
- Test execution: Choose the `whole`, `selective`, or `off` configuration when running the tests.The instructions expose three configuration choices through the make command.
- Test execution: For more accurate results, run tests individually and reboot after each test.The instructions present individual execution and rebooting as an optional accuracy measure.
- Test execution: The procedure is provided as Listing 6, titled “Reproducing evaluation results.”The listing labels the surrounding commands as the reproduction procedure.