PaperlayerView exact revision on arXiv

Source-linked AI summary

A Multi-Month Study of Git Commit Signing

Abubakar Sadiq Shittu, John Sadik, Scott Ruoti

arXiv:2608.29283v1cs.CRcs.SE

TL;DR

Developer-controlled Git commit signing is rarely studied as a lived adoption experience, especially across setup, sustained use, verification, and key management. This three-month study followed 22 student proxies for junior developers through those activities and found that routine signing success did not ensure effective security use. The findings support tools and education that help developers interpret verification states, authorized identities, and key-lifecycle operations.

  • Problem

    Prior work documents rare and inconsistent developer-controlled signing, but does not show what developers experience during adoption or whether sustained signing supports effective verification and security understanding.

  • Method

    A three-month study followed 22 senior and graduate computer science students through independent setup, signing across four projects, second-device configuration, anomalous-repository verification, and security-reasoning prompts.

  • Results

    Sustained signing and positive usability did not ensure effective security use: participants identified 4.77 of 13 problematic commits on average, 27.3% identified none, and 45% reported at least one misconception.

  • Takeaways & Limitations

    Secure adoption requires more than easier signing setup; tools and education must support authorized-identity verification, interpretation of verification states, workflow exceptions, and key-lifecycle reasoning.

  • Takeaways & Limitations

    The verification task used SSH-signed commits and an unusually high anomaly density with fingerprints rather than a preconfigured trusted-signer list, which may have affected task difficulty.

Abstract

from arXiv · show

Git commit signing, introduced in 2012, is one mechanism for establishing commit provenance in software supply chains, yet developer-controlled adoption remains rare and developers' experiences using it are understudied. To examine this experience, we conducted a three-month study with senior undergraduate and graduate computer science students (n = 22), whom we treat as proxies for junior developers. Participants configured commit signing independently, used it across four coursework projects, extended it to a second device, examined an external repository containing anomalous commits, and answered security-reasoning prompts. We found that while almost all participants successfully signed every commit and rated routine signing positively, many faced friction during setup, multi-device configuration, and repository verification. Despite signing all semester, they struggled to spot anomalous commits during verification, with over a quarter finding none. Additionally, nearly half expressed at least one misconception regarding signing guarantees or key management. Without isolating whether these difficulties stemmed from tooling, education, or understanding, we conclude that making signing easier is not enough to ensure effective security use. Rather, secure adoption also requires tools and education that support signature verification against authorized identities, interpretation of missing signatures and unknown keys, and correct reasoning about key-lifecycle operations.

1 Introduction

The paper examines how developers experience Git commit signing and whether sustained signing translates into effective security use. Participants signed consistently and liked routine signing, but many struggled with verification, key management, and the boundaries of signing guarantees.

  • Cryptographic commit signing can detect changes made without the developer’s signing key, but valid signatures do not prove identity, authorization, or benign code.Attackers may use unauthorized keys or spoof Git identity information.
  • Developers must distinguish unsigned commits, invalid signatures, unlinked signatures, and valid signatures from unauthorized keys.Effective verification also depends on consistent signing, key protection, appropriate authorization policies, and understanding verification states.
  • Prior studies show that developers rarely sign commits and often use signing inconsistently across projects.Those measurement studies do not reveal developers’ adoption experiences or whether sustained signing supports effective verification and security understanding.
  • 81.8% of participants signed every observed commit, and end-of-semester usability was positive with SUS M=77.61.The study followed senior undergraduate and graduate computer science students through independent setup and signing across coursework projects.
  • 4.77 of 13 problematic commits were identified on average, 27.3% identified none, and 45% reported at least one misconception about signing or key lifecycle operations.No participant identified every problematic commit, despite sustained signing throughout the semester.
  • Secure adoption requires tools and training that support verification against authorized identities, interpretation of verification states, workflow exceptions, and cryptographic guarantees.The study does not isolate whether the observed challenges arose from confusing tools, poor training, or workflow understanding.

2 Related Work

Related work frames commit signing as only one component of software provenance and shows persistent challenges in adoption, identity binding, verification, and key management. The paper addresses the still-underexamined practical experience of handling these responsibilities in Git.

  • Repository studies report low and uneven signing adoption, including approximately 10% verified commits and 95.4% of users never signing a commit.Estimates vary with sampling, domains, Git clients, and treatment of platform-generated signatures.
  • Fewer than 6% of developers signed locally, signing rarely persisted, and key-upload or expiration problems often undermined verification.Among signing developers, approximately one in eight signatures failed because the public key was not uploaded, and more than one-quarter had an expired or unusable key.
  • A valid signature establishes integrity and possession of the corresponding private key, but not signer identity, authorization, or code safety.Repository trust therefore also requires identity binding, contributor policies, and signature verification.
  • Software provenance systems extend signing across source, build, test, packaging, and release steps, while newer systems can reduce dependence on long-lived developer-managed keys.In-toto signs metadata across pipeline stages; Sigstore uses short-lived identity-linked certificates and transparency logs.
  • Key management spans generation, storage, identity verification, multi-device use, recovery, rotation, revocation, and retirement.Users face tradeoffs among availability, recovery, protection from unauthorized access, provider trust, and exposure when keys are shared across devices.
  • The specific practical challenges developers encounter when managing Git signing keys remain largely unexamined.Prior work documents broader usability and organizational problems, while student-proxy research supports task-specific rather than general conclusions about developers.

3 Gap and Research Questions

Prior work documents low adoption, identity-binding requirements, and key-management problems, but has not followed the same users across the full signing lifecycle. This study asks about sustained-use experiences and whether sustained use brings the verification ability and security understanding needed for effective use.

  • No prior study had followed the same Git users through setup, sustained use, multi-device configuration, verification, and security reasoning over several months.The study uses advanced computer science students as scoped proxies for junior developers.
  • RQ1 asks what developers experience when adopting and sustaining Git commit signing over time.
  • RQ2 asks whether sustained Git commit signing is accompanied by the verification ability and security understanding needed to use it effectively as a security mechanism.

4 Methodology

The study embedded a three-month, semester-long investigation of Git signing in required coursework with technically prepared senior and graduate students. It combined independent setup, sustained signing, cross-device configuration, verification of anomalous commits, and written security reasoning, while limiting claims beyond the student-proxy and study-period scope.

  • Participants and setting: The study was embedded in an IRB-approved semester-long course study of senior and graduate computer science students at a public U.S. university.The coursework setting enabled observation during normal activities rather than only in a laboratory.
  • Participants and setting: The 22 participants were treated as scoped proxies for junior developers, not developers broadly.They had version-control fluency, cryptography coursework, and practical implementation skills, but were not yet uniformly professional developers.
  • Scope and limitations: The study does not generalize to professional developers or long-term key-lifecycle events such as expiration, rotation, revocation, and recovery.The authors identify replication with professional populations as necessary future work and note the three-month window as a scope boundary.
  • Participants and setting: Only 22 of 70 enrolled students, or 31%, consented to research participation.All students completed the coursework, but research participation was opt-in.
  • Sustained use: Signing was observed across four coursework projects over three months rather than treated as a single assignment.Submitted repositories and Git histories were used to assess whether signing continued and how consistently it was applied.
  • Initial setup: Participants independently configured keys and commit signing, submitted repositories and public keys, and provided written self-reports of their setup processes.Institutional guidance was withheld initially to preserve ecological validity, with tiered support available after independent effort.
  • Multi-device configuration: The study included a second-device portability task in which participants either transferred an existing private key or generated and managed a new keypair.Self-reports and post-task ASQ ratings captured choices, rationale, failures, effort, and support documentation.
  • Verification task: A trust-assessment task used a 20-commit repository containing 13 injected anomalous commits to evaluate signature-based detection.The task measured how many and which categories of anomalies participants could detect.

5 Findings: Initial Setup

Participants successfully configured commit signing, but setup commonly required external guidance and troubleshooting across tools, platforms, and configuration formats. Despite these hurdles, perceived usability remained generally positive, while longer completion experiences were associated with lower usability ratings.

  • Setup completion: All 22 participants completed setup, but 36% recorded multiple attempts, indicating trial-and-error rework during configuration.Participants with technical hurdles took longer to finish.
  • Setup completion: 73% used GnuPG and all participants consulted external sources, showing that commit signing was not easily self-discoverable.GitHub’s official documentation was the most commonly used source and was generally rated positively for clarity and completeness.
  • Configuration friction: SSH users encountered allowedSignersFile schema mismatches that Git often reported only through generic verification failures or no specific error message.Participants engaged in misdirected debugging before identifying the required file structure.
  • Configuration friction: macOS participants experienced signing failures because GPG could not prompt for passphrases without the pinentry-mac helper, which primary instructions did not include.Four participants encountered this platform-specific failure.
  • Configuration friction: Six participants encountered Git configuration problems involving omitted signing-key settings or unconfigured SSH signing format.Some incorrectly assumed Git would automatically associate a newly generated key with matching name and email fields.
  • Perceived usability: Participants generally rated setup positively, but longer completion times were associated with lower perceived usability.Reported setup scores were ASQ M=6.12 and SUS M=76.62.

6 Findings: Semester-Long Usage

After initial setup, routine commit signing became largely effortless and was evaluated positively across the semester. However, sustained use did not ensure consistent per-commit signing, deliberate key management, or deeper understanding of the signing security model.

  • Perceived usability: SUS was 77.61 at semester-end, with no statistically significant change from initial setup among participants measured at both time points.The semester-end mean corresponded to a Good adjective rating.
  • Routine signing: 18 participants signed every observed commit, while four signed inconsistently at rates ranging from 57–87%.The authors suggest signing was not always intuitively understood as a per-commit practice.
  • Routine signing: 21 participants described setup as harder than daily signing, while 20 characterized routine signing as seamless or automatic once configured.This pattern held across operating systems, key types, and SUS scores.
  • Routine signing: 16 participants reported no changes to their signing process during the semester, and only one discovered automatic signing after initially using the -S flag manually.This indicates limited subsequent exploration of the toolchain.
  • Key management: 14 participants relied on default private-key storage, five could not recall the key location, and only three demonstrated proactive key management beyond defaults.Eleven participants reported using a passphrase, usually as part of setup rather than a deliberate security decision.
  • Future adoption: 13 participants expressed unconditional intent to continue signing, seven expressed conditional intent, and one expressed no intent to continue.The lowest SUS scorers still reported conditional or continued intent, so usability alone did not determine adoption intent in this cohort.
  • Improvement requests: Participants most often requested consolidated documentation and graphical or integrated setup support, alongside clearer errors and reduced dependence on external tools.The most frequent requests were consolidated documentation (n=7) and graphical or integrated setup support (n=6).

7 Findings: Cross-Device Setup

Extending signing to a second device was achievable but required substantial reconfiguration and troubleshooting. Copying an existing key and generating a new one produced similar outcomes, while each path introduced distinct friction.

  • Task outcome: All 22 participants produced and verified a signed commit on a second device, with a geometric-mean completion time of 51.04 minutes.Among 20 participants with valid times at both points, second-device setup did not differ significantly from initial setup.
  • Key strategy: Participants chose either generating a new key for security or copying an existing key for convenience and identity consistency.Neither rationale guaranteed a smoother workflow.
  • Key strategy: The two key-management paths produced similar satisfaction and completion-time outcomes, with exploratory comparisons detecting no statistically significant differences.Satisfaction remained positive in both groups.
  • Configuration friction: 14 participants encountered at least one Git or signing configuration error during second-device setup.Friction centered on key-transfer mechanics and persistent Git reconfiguration gaps.
  • Configuration friction: Six participants encountered omitted user.signingkey settings or allowedSignersFile schema errors while reconfiguring the new machine.These issues persisted beyond initial setup rather than appearing only once.
  • Key transfer: Among 12 participants who copied an existing key, nine needed external guidance for export, and eight needed external sources for import.GitHub’s main signing documentation did not cover private-key transfer between machines.
  • Key transfer: Imported keys required explicit trust configuration before use without warnings, and the warning appeared during verification rather than at import.Participants who skipped manual verification might not encounter the issue until downstream use.
  • Overall implication: The study concludes that adding devices remains manageable but does not become easier because the toolchain retains no cross-machine configuration state.Users managing their own keys bear the reconfiguration cost for each additional device.

8 Findings: Identifying Malicious Commits

Participants showed mixed ability to detect anomalous commits: many found few or none, and verification difficulties were linked to configuration failures, mechanism confusion, and poor tooling feedback.

  • Detection performance: Timestamp manipulation was the least frequently detected category, with 9 participants identifying it.Participants noticed future-dated commits more readily than a backdated commit.
  • Detection performance: 27% of participants identified no anomalous commits, and none identified all 13 problematic commits.Overall performance was mixed despite participants approaching the repository with suspicion.
  • Detection performance: More time spent on the task correlated with identifying more anomalous commits, whereas satisfaction did not predict detection accuracy.Time-on-task correlated with performance, but perceived ease did not.
  • Verification strategies: CLI, web, and combined strategies produced similar mean detection scores, while allowedSignersFile configuration was a more consequential difference.Mean scores were 5.00 for CLI, 5.11 for CLI+web, and 4.50 for web alone.
  • Verification strategies: 55% attempted allowedSignersFile configuration, but 4 of 12 active users failed and 7 participants confused fingerprints with full public keys.Participants often did not discover that full public keys had to be retrieved from GitHub’s .keys endpoint.
  • Tooling and mechanism barriers: Fourteen participants failed to configure verification successfully, and six abandoned cryptographic verification entirely, often relying on surface-level cues.Two participants gave up after confusing SSH and GPG mechanisms despite successfully configuring allowedSignersFile.
  • Tooling and mechanism barriers: Git’s support for GPG, SSH, and S/MIME lacked a unified verification workflow, while unclear errors made mechanism confusion harder to resolve.Participants described errors as confusing when prior GPG experience did not transfer to SSH signatures.
  • Tooling and mechanism barriers: Manually reviewing code did not improve detection: content reviewers averaged 4.60 flagged commits versus 4.82 for non-reviewers.One participant incorrectly treated benign-looking code as evidence that commits were not problematic, even though signing concerns authorship and authorization.

9 Findings: Security Understanding

Nearly half of participants demonstrated misconceptions about commit-signing guarantees or key management, despite correctly using signing procedurally. These misconceptions involved confusing verification with authentication, mishandling key lifecycle reasoning, and transferring rules from other security contexts.

  • Overall security understanding: 45% of participants gave at least one misconception or wrong response to the security-reasoning prompts.Seven participants had misconceptions without wrong responses, while three had at least one wrong response.
  • Key lifecycle reasoning: Three participants incorrectly believed that losing a private key makes previously signed commits unverifiable.Existing signatures remain verifiable with the public key because signing and verification use deliberately independent operations.
  • Security value and usability: One participant confused SSH push authentication with commit signing, although push authentication does not authenticate commit content.Without signing, Git commit metadata remains trivially spoofable regardless of who holds the SSH push key.
  • Key lifecycle reasoning: Two participants proposed moving repositories or creating new accounts after key theft, neither of which revokes the compromised signing key.These responses reflected account-lifecycle thinking rather than cryptographic key-lifecycle thinking.
  • Security value and usability: Two participants argued that poor key management justified abandoning commit signing, confusing implementation risk with the security value of signing.The paper distinguishes better tooling as the response to usability problems from leaving commits unsigned.
  • Security education: One participant incorrectly applied a do-not-reuse-keys rule to signing keys, which are designed for repeated use across many commits.The error illustrates how security rules learned in one context can transfer incorrectly to another.
  • Security education: Procedural fluency did not consistently produce correct security reasoning: participants transferred unrelated rules, substituted other authentication mechanisms, or proposed account-level responses to key compromise.The study frames this mismatch as a gap between routine signing and understanding signing’s security boundaries.

10 Study Limitations

The study’s findings are bounded by the verification task design, its short duration, its student proxy sample, and its required-course setting. These constraints limit generalization to typical real-world verification, long-term key management, professional developers, and voluntary adoption.

  • Verification task: The SSH-based verification task differed from many participants’ earlier GPG experience, potentially adding difficulty, although no significant performance difference was observed between groups.The authors also note that SSH may increasingly be encountered because GitHub recommends it as a simpler signing mechanism.
  • Verification task: The task used 13 anomalous commits among 20 and provided fingerprints instead of a preconfigured trusted-signer list, limiting representativeness of ordinary verification scenarios.These choices were deliberate to obtain sufficient behavioral observations and model high-stakes verification of unfamiliar signers.
  • Study duration: The three-month study did not measure performance during long-term key expiration, rotation, revocation, or recovery after loss or compromise.The study examined reasoning about key loss, theft, and revocation, but the authors caution against generalizing to long-term lifecycle events.
  • Sample: The single-university opt-in sample was only a scoped proxy for junior developers, limiting external validity regarding professional incentives and organizational support.The authors call for testing whether the observed verification and security-understanding gaps persist among industry practitioners.
  • Study setting: The required-course setting measured sustained use under an external requirement rather than voluntary adoption and may have encouraged unusually positive reports or careful behavior.Completion-only grading, post-grade consent, open-ended prompts, and .git-log cross-checks reduced but did not eliminate these concerns.

11 Discussion and Conclusion

The discussion argues that signed-commit counts do not establish repository security: participants made signing routine but still struggled with verification and key-management reasoning. Effective adoption therefore requires usable verification tools, practical education, and contexts where developers understand signing’s relevance.

  • Discussion and Conclusion: Counting signed commits cannot measure repository security because signing logs do not reveal whether developers can verify signatures, interpret results, or handle key-management exceptions.Within this cohort, routine signing coexisted with missed anomalous commits and misunderstandings about lost, stolen, or revoked keys.
  • Barriers to Effective Verification: The study does not identify tooling, education, or security understanding as the single underlying barrier; technical friction and conceptual ambiguity appeared together during verification.The authors frame the interaction between these barriers as the reason isolated interventions may be insufficient.
  • Barriers to Effective Verification: Existing verification workflows impose manual public-key discovery, allowedSignersFile formatting, and navigation across GPG and SSH standards.These burdens make verification an advanced, optional task rather than a core security primitive in the current Git architecture.
  • Barriers to Effective Verification: Developers may conflate access control, signature validity, signer authorization, and code safety when evaluating repository trust.The discussion gives manual malware inspection and treating platform push authentication as a substitute for commit signing as examples.
  • Adoption and perceived value: Ease of use alone may not support voluntary adoption when developers perceive no practical reason to sign.Participants emphasizing security threats, provenance, or collaboration more often intended to continue, whereas personal or low-stakes repositories reduced perceived value.
  • Implications: The paper recommends combining tools that clarify identity binding and verification with education about guarantees, limitations, unknown signers, key loss, theft, rotation, and revocation.The proposed goal is correct trust decisions across signing, verification, and exception handling rather than maximizing signed-commit counts.
  • Open Science: The study’s artifacts include instructions, a codebook, redacted forms, and analysis scripts, but participant repositories, public keys, and raw qualitative data cannot be released because of privacy and human-subjects constraints.The unreleased materials contain identifying metadata or potentially identifying free-text responses collected under IRB oversight.

B.4 Harms and Mitigations

The study addresses participant and ecosystem risks through privacy protections, defensive reporting, and safeguards against coercion or self-censorship. Residual self-report bias remains acknowledged but uneliminated.

  • Harms and Mitigations: Self-censorship in written self-reports could not be fully eliminated despite completion-only grading, open-ended prompts, post-grade consent, and .git-log cross-checks.Students may have moderated responses to please the instructor.
  • Harms and Mitigations: Reputational harm from signing patterns and self-reports was mitigated through anonymization, aggregate reporting, exclusion of identifying quotes, and deletion of identifying information.The reported residual risk was low.
  • Harms and Mitigations: Potential adversarial targeting from identified failure modes was mitigated by conceptual-level reporting, omission of operational details, and defensive framing.The mitigation limits how findings could be used to operationalize attacks.
  • Harms and Mitigations: The authors conclude that the research burden was minimal, residual harms were low after mitigation, and the expected ecosystem-security benefit supported publication.The ethical assessment also states that informed-consent and privacy rights were protected.

C Supplementary Results

The supplementary materials provide participant demographics, per-participant security-reasoning classifications, problematic-commit identification counts, consulted information sources, and descriptive and inferential outcome statistics.

  • Table 5 summarizes the demographics of the 22 participants.
  • Table 6 records each participant’s classifications across five security-reasoning prompts using correct, misconception, wrong, or unanswered codes.Response counts vary because P2, P10, P18, and P22 provided incomplete responses.
  • Table 7 reports the number of problematic commits identified by each participant.
  • Table 8 summarizes external information sources consulted by participants, allowing participants to appear in multiple categories.
  • Tables 9–11 report descriptive statistics and exploratory inferential analyses for the principal outcomes, setup tasks, and commit-verification performance.Table 10 covers initial, sustained, and second-device setup; Table 11 defines detection performance as problematic commits identified out of 13.
Loading 2608.29283v1…