Source-linked AI summary
ShellVis: Sandboxed Live Programming for Shell Scripts
Joshua Horowitz, Jeffrey Heer
TL;DR
Live programming becomes risky when repeated execution can produce harmful side effects. ShellVis addresses this by sandboxing live shell-script execution and providing detailed runtime feedback; participants found it helpful and safer, while identifying limits around novice support and sandbox completeness.
Problem
Live programming needs to make programs with side effects repeatable and safe because automatic re-execution can destabilize state or cause disastrous irreversible results.
Method
ShellVis combines sandboxed live shell scripting with focus+context visualizations and automatic, in-context execution feedback.
Results
Participants unanimously found ShellVis helpful, comfortable, and desirable, reporting that its feedback and sandboxing replaced tedious practices and increased safety and confidence.
Takeaways & Limitations
Sandboxing can extend live-programming techniques to shell scripting and other real-world programming contexts where side effects matter.
Takeaways & Limitations
The study used simple feature-oriented tasks, a small narrow sample, and participants with at least one year of shell experience, limiting conclusions about novices.
Abstract
from arXiv · showhide
Live programming provides visibility to programmers by running and tracing programs as they are edited. However, for programs with potentially harmful side effects, liveness can turn mistakes into disasters. We propose enabling live programming in environments with side effects via sandboxing: confining effects to a simulation of the true environment. We apply sandboxed live programming in the challenging context of shell scripting: a ubiquitous and powerful---yet notoriously opaque and error-prone---tool. ShellVis provides line-by-line feedback on a shell script's run-time behavior, with file operations sandboxed via a safe overlay of the file system. A qualitative user evaluation finds ShellVis to be helpful to participants, replacing tedious existing practices and instilling confidence. Participant responses also reveal areas for future research, particularly bridging the gulf of execution alongside the gulf of evaluation. ShellVis serves as a case study of how sandboxing can bring live-programming techniques into the many real-world programming contexts where side effects are important.
1 INTRODUCTION
Shell scripting offers powerful, reusable automation but remains difficult and risky because of opaque semantics and potentially destructive side effects. ShellVis addresses this challenge by combining live, fine-grained feedback with sandboxed execution, and a user study found participants helpful, safer, and more confident using it.
- Shell scripts provide precise, reusable automation, extending from one-off commands to parameterized programs and loops.
- Opaque syntax, unfamiliar semantics, and high-stakes side effects make shell scripts difficult even for experienced programmers.
- Live programming can expose runtime data, detect problems quickly, and help diagnose root causes while code is edited.
- ShellVis runs edited Bash scripts repeatedly and displays fine-grained output, shell-state changes, and file-system modifications while protecting the real file system.
- In a study of seven shell users completing four tasks, participants unanimously found ShellVis helpful, comfortable, and desirable for their work.
- The paper contributes sandboxed live programming, the ShellVis prototype, and a user evaluation showing helpfulness while identifying future research directions.
2 BACKGROUND AND RELATED WORK
ShellVis builds on shell scripting, live-programming environments, and prior sandboxing tools to address the difficulty of safely gaining runtime visibility into side-effectful scripts. Its distinguishing combination is fine-grained, within-code feedback with sandboxed effects.
- 2.1 Shell Scripts: Shell scripts combine command-line control, pipes, variables, and loops to support automation and reusable file-oriented workflows.
- 2.1 Shell Scripts: Shell scripting remains difficult because of opaque syntax, limited visibility into shell state, and the danger of destructive mistakes.
- 2.2 Programming Environments: Within-code live-programming environments augment textual editors with in-context runtime displays while preserving existing editors, tools, and revision-control workflows.
- 2.3 Live Programming with Side Effects: Live programming with side effects requires both repeatability, because reruns otherwise shift the starting world, and safety against irreversible results.
- 2.4 Tools for Shell Scripting: ShellVis applies sandboxing to file-system access, building on overlay filesystems while extending prior tools with command-level traces, output, and shell-variable changes.
- 2.4 Tools for Shell Scripting: Compared with pipeline-focused live tools, ShellVis visualizes entire scripts and sandboxes side effects; the paper identifies this combination as an unexplored synergy.
3 SANDBOXED LIVE PROGRAMMING IN SHELLVIS
ShellVis implements a four-step live-programming loop—editing, running, tracing, and viewing—for shell scripts. It uses sandboxed file access and focus-plus-context visualization to present detailed runtime behavior without requiring a custom editor.
- ShellVis follows four stages: users edit code, the system runs it, execution is traced, and results are presented for inspection.
- Editing: Users edit scripts in their preferred text editor, preserving familiar tools while ShellVis focuses on supplying runtime feedback.
- Running: ShellVis currently focuses on sandboxing file-system access, leaving network, process, device, and other system effects for future work.
- Tracing: Sandbox machinery traces which files each command adds, removes, or modifies, and this external-effect information is supplemented with shell-state and output traces.
- Viewing: A focus-plus-context fisheye view manages large traces by adjusting detail around the user’s focus and supporting inspection of loop iterations.
4 USAGE SCENARIO
ShellVis visualizes a shell script’s execution as it is edited, helping users inspect loop data, command output, and variable changes while safely testing file operations. In the scenario, sandboxing exposes and contains a destructive move mistake before the corrected script is run for real.
- Usage scenario: ShellVis automatically reruns edited scripts and shows execution feedback, including loop iterations, command output, and variable changes.The visualizer displays the script alongside annotations, while a loop slider exposes values that would otherwise require manual echo statements.
- Usage scenario: A loop slider lets users inspect each file and confirm that extracted metadata changes as expected across iterations.Users can scrub through loop iterations to inspect filenames and verify the target date field returned by exiftool.
- Usage scenario: ShellVis displays large command output in scrollable bubbles and marks changed shell variables with a distinct icon.These annotations let users inspect relevant metadata and follow the effects of piping and assignment steps.
- Usage scenario: The mistaken mv command renames and overwrites photos in the simulated file system, while the original files remain untouched.Because each execution starts from the same state, the user can recognize the error and revise the script without damaging real files.
- Usage scenario: Adding mkdir $year before mv fixes the script, after which the move succeeds and the completed script can be run in the terminal.The corrected command sequence creates the destination directory before moving each photo.
5 INTERFACE DESIGN
ShellVis is designed as an incrementally adoptable, within-code live-programming environment that works with standard shell scripts and existing editors. Its interface uses trace-linked bubbles, fisheye expansion, and iteration sliders to balance source-code correspondence with detailed runtime feedback.
- 5.1 Scope: ShellVis supports standard POSIX-family scripts and works alongside existing editors rather than requiring a new language or custom editor.This preserves access to existing scripts and familiar editing practices while keeping the system focused on runtime feedback.
- 5.1 Scope: The visualizer streams a program trace into information bubbles attached to executed commands, showing generic runtime feedback.Supported annotations include output streams, shell-variable changes, and file-system modifications as described across the design.
- 5.2 Fisheye Information Bubbles: Abbreviated bubbles provide stable overviews and expand into focused pop-ups, preserving a predictable layout while exposing detailed information.The fisheye design avoids pushing other bubbles off-screen when the current line changes.
- 5.3 Visualizing Loops: ShellVis represents loop execution with an iteration slider that updates the loop variable or condition and the body annotations for the selected iteration.This interaction preserves correspondence with source code while letting users inspect behavior across iterations.
- 5.3 Visualizing Loops: The current design handles moderate data volumes, including hundreds of loop iterations and large outputs through scrubbing and scrolling.The authors do not optimize the design or implementation for big data.
6 IMPLEMENTATION
ShellVis combines a browser-based live trace with sandboxed, line-by-line shell execution. Its prototype uses layered union filesystems and script instrumentation, but remains limited by macOS sandboxing gaps, tracing edge cases, performance overhead, and rerun costs for slow scripts.
- Architecture: ShellVis runs scripts in a sandboxed environment, traces their activities, and streams a live-updating program trace to an interactive browser client.The server executes and traces the script; the client displays and supports exploration of the trace.
- Sandboxing: Layered union filesystems isolate each command’s writes, scan its upper delta directory for changes, and commit those changes to the lower filesystem.This supports more precise attribution of filesystem modifications to individual commands.
- Sandboxing: On macOS, the sandbox is leaky because absolute paths can access original files outside the union filesystem, disqualifying the prototype for unaware users in the wild.The issue stems from difficulties using chroot and the lack of namespaces on macOS; future work may move to Linux.
- Limitations: Concurrent filesystem writes may be attributed to the wrong command, while the prototype’s final script takes around 3.5 seconds versus 0.5 seconds when run directly.Most reported overhead comes from communication between the instrumented script and the server.
- Tracing: ShellVis transforms scripts into instrumented code that reports command events and data over one-off HTTP connections, using AST parsing to insert tracing instructions.The prototype instruments calls, for loops, and while loops; future versions could cover additional constructs.
- Limitations: Re-running scripts from scratch becomes less practical for larger or slower workloads because of the intrinsic cost of executing those scripts.Suggested future approaches include limiting loop iterations during testing or caching long-running subcommands.
7 USER STUDY DESIGN
The user study examined shell programmers’ challenges and their use of live feedback and sandboxing through four ShellVis tasks. Seven participants completed the study procedure, followed by a survey and interview, within a study design whose sample and tasks impose important limits.
- Research questions: The evaluation addressed challenges in shell scripting and how programmers use live feedback and sandboxing while working with shell scripts.These were formalized as three research questions covering shell-script challenges, live feedback, and sandboxing.
- Participants: Seven participants with 1 to 20 years of shell-scripting experience completed the evaluation.All participants were men aged 28 to 42 and were recruited through social media channels.
- Procedure: Participants first discussed their prior shell-scripting experiences, then learned ShellVis through a photo-sorting scenario and completed four shell tasks.The tasks covered renaming files, deleting user directories, reading a log-processing pipeline, and multiplying file line-counts.
- Procedure: After the tasks, participants completed a five-point Likert survey and a post-study interview about ShellVis’s fit with their real-world workflows.The survey used 1 for “Disagree” and 5 for “Agree”; sessions lasted one to two hours.
- Limitations: The study used simple feature-oriented tasks, a small and demographically narrow sample, and social-network recruitment that may have favored participants sympathetic to the goals.Because all participants had at least one year of shell experience, the study does not support conclusions about ShellVis for novices.
8 USER STUDY RESULTS
Participants found ShellVis helpful for understanding and writing shell scripts, with line-by-line feedback and sandboxing replacing tedious safety and debugging practices. They also reported increased confidence, while noting that the incomplete file-system sandbox limits trust.
- 8.2 Using Live Feedback (RQ2): ShellVis helped participants understand unfamiliar scripts and inspect intermediate artifacts while planning or diagnosing subsequent steps.Participants valued seeing output and data flow attached to execution, without rerunning earlier steps.
- 8.2.1 ShellVis vs. Echo Logging: Automatic, contextual annotations improved on echo logging by removing manual log management and showing outputs alongside the responsible code and loop iterations.Participants also said ShellVis’s “show everything” approach avoided guessing where to place diagnostic echoes.
- 8.2.2 ShellVis vs. Debuggers: Unlike single-step debugging, ShellVis let participants run the script and then navigate through its execution trace visually.This distinction was described as supporting visualization of what the shell script was doing rather than requiring delicate step-by-step inspection.
- 8.3 Using Sandboxing (RQ3): Sandboxing replaced error-prone workarounds such as dry-run commands, backups, and testing on one file, enabling participants to run complete scripts more confidently.Participants unanimously rated sandboxing helpful and described it as reducing the effort required to protect against destructive file operations.
9 DISCUSSION & FUTURE WORK
ShellVis bridges the gulf of evaluation by showing what shell scripts do, but participants identified a remaining gap in expressing intended behavior as shell code. Discussion also identifies interface, visualization, feature, side-effect, and audience limitations that motivate future work.
- 9.1 Gulf of Evaluation vs. Gulf of Execution: ShellVis helps with the larger task of writing shell scripts, but it provides limited assistance when programmers do not know the required syntax or commands.Participants described ShellVis as useful for understanding script behavior, while noting that it cannot determine what users should type initially.
- 9.1 Gulf of Evaluation vs. Gulf of Execution: Code assistants could pair with ShellVis’s sandboxed feedback to verify generated shell code without destructive consequences, but this combination remains untested.Participants specifically valued immediate verification and protection from costly mistakes; the paper calls for further study.
- 9.2 System improvements: ShellVis’s separate visualizer window consumes screen space and can be confused with the editor, motivating tighter editor integration.Survey responses averaged 3.4/5 for the editor arrangement, and participants reported practical workspace and interaction problems.
- 9.2 System improvements: Loop scrubbing can hide anomalies in nonvisible iterations, while piped commands separate corresponding inputs and outputs because Unix tools rarely expose line mappings.Participants requested aggregate loop views and traceability across pipeline stages; the latter may be difficult because mappings vary across tools.
- 9.2 System improvements: Future work includes domain-specific visualizers, broader shell-feature support, remote use, and sandboxing side effects beyond the file system.Network requests and other effects may require quarantine rather than high-fidelity simulation, and these approaches have not yet been tested.
- 9.3 ShellVis for novice programmers: Because the study involved proficient programmers, ShellVis’s usefulness for novices remains uncertain, especially given its limited support for the gulf of execution.The authors hope it can support novices alongside other learning resources but present this as speculation.
10 CONCLUSION
The conclusion presents sandboxed live programming as a general approach for bringing live-programming techniques to side-effectful contexts, illustrated through ShellVis for shell scripting.
- 10 CONCLUSION: ShellVis combines overlay file systems with focus+context visualizations to support live shell scripting while accommodating side effects.The conclusion frames ShellVis as a prototype demonstrating sandboxed live shell scripting.
A STUDY TASKS
The appendix introduces the written prompts and starter code used for the study’s four shell-scripting tasks.
- A STUDY TASKS: The study appendix provides the written prompts for four tasks, including any starter code supplied to participants.This passage identifies the appendix’s contents without describing the individual tasks.
A.1 titles
The titles task asked participants to rename speech files using the title in each file’s first line, exposing a quoting issue when titles contained spaces.
- A.1 titles: Participants wrote a script to rename each speech file to the title listed on its first line.The task used the “docs” directory and required processing multiple files.
- A.1 titles: Titles containing spaces required appropriate quoting to avoid trouble during renaming.Participants discovered this issue while performing the task.
A.2 users
Shell scripting tasks expose how small input or command mistakes can produce unintended file-system effects. The examples include destructive deletion, log-processing pipelines, and incomplete arithmetic over file counts.
- A.2 users: A spurious blank line causes the deletion script to remove the entire users folder.The loop interprets the blank line as an empty username and executes rm -rf on users/ with no suffix.
- A.2 users: The log-processing task asks users to filter, transform, sort, and count matching sandbox-restriction requests.
- A.2 users: The line-count task provides a loop that counts files but leaves the requested multiplication unfinished.The intended result is the product of the line counts across files.