FoC: Figure out the Cryptographic Functions in Stripped Binaries with LLMs

1 INTRODUCTION

• Cryptography-oriented heuristics can be undermined by hidden constants and compiler optimizations, and generally provide only primitive classes rather than deeper semantics.Source: §1 · p. 2
• Binary-code similarity methods can reflect minor code changes, but cryptographic applications remain underexplored and retrieval is limited by the known-function library.Source: §1 · p. 3
• FoC combines FoC-BinLLM’s natural-language semantic summaries with FoC-Sim, which uses semantic, structural, and cryptographic features to retrieve homologous functions.The framework targets unknown cryptographic functions in its database.Source: §1 · p. 3
• The authors construct a cryptographic binary dataset from popular libraries and automate semantic labeling for large-scale binary code.Source: §1 · p. 3
• FoC-BinLLM beats ChatGPT by 14.61% on ROUGE-L, while FoC-Sim achieves 52% higher Recall@1 than previous best methods.Source: §1 · p. 3
• FoC is also reported to show practical utility in cryptographic-virus analysis and identification of vulnerable cryptographic implementations in real-world firmware.Source: §1 · p. 4

2.1 Problem Definition

• The analysis objective is to generate a natural-language summary of a cryptographic function’s behavior and purpose, plus an embedding for binary code similarity detection.The function is analyzed in the context of its binary file.Source: §2.1 · p. 4

2.2 Challenges

• A diverse public dataset is difficult to assemble because implementations vary across algorithms, standards, developers, compilation environments, optimizations, and architectures.The paper identifies the lack of comprehensive cryptographic binary datasets as a barrier to research.Source: §2.2 · pp. 4–5
• High-quality semantic labels must describe binary-function behavior, but names and primitive classes are brief, comments may be missing, and manual annotation is impractical at scale.The paper identifies automated, accurate label generation as necessary for effective training and evaluation.Source: §2.2 · p. 5
• Cryptographic updates can be tiny, such as a single-line vulnerability patch, making sensitivity to small code changes a key challenge for generative models.The section names this requirement cross-version awareness.Source: §2.2 · p. 5

2.3 Existing Techniques

• Existing techniques leave the identified challenges unresolved: heuristic methods depend on patterns or known implementations, while binary summarizers target general rather than cryptographic code.BCSD methods may help with cross-version awareness, but the cited approaches are not tailored to cryptography.Source: §2.3 · pp. 5–6

2.4 Large Language Model and Our Motivation

• FoC motivates using a binary-specific LLM for semantic descriptions and adding a BCSD module based on that model to produce cross-version-aware embeddings.The motivation also connects automated semantic descriptions with building a rich cryptographic binary dataset.Source: §2.4 · p. 7

3 DATASET CONSTRUCTION

• FoC's dataset construction addresses the shortage of publicly available, implementation-diverse cryptographic binary datasets and supports method development and further research.The collection focuses on widely used cryptographic libraries, implementations, and modules in large C projects.Source: §3.1 · p. 7, §3 · p. 7

3.1 Cryptographic Dataset Collection

• Projects are cross-compiled with two compilers, four optimization levels, and six target architectures; binaries are stripped, decompiled, and deduplicated by function MD5.Projects with overlapping compiled code are kept from appearing in both training and test sets.Source: §3.1 · p. 8

3.2 Automated Semantic Labels Creation

• Because comments occur in fewer than 20% of functions and often lack useful descriptions or contain indirect or redundant information, the authors generate summaries from source metadata.The prompt requests one-sentence summaries to limit information not present in the pseudo-code.Source: §3.2 · pp. 8–10

3.3 Keyword-based Discriminator

• More than 85% of generated summaries passed the keyword-based discriminator, and developer-comment/model-summary consistency scored 43.55% ROUGE-L.The discriminator accepts a label only when the class found in the summary matches the class in the source code.Source: §3.3 · p. 10

• FoC-BinLLM interprets lifted pseudo-code, predicts stripped function names, and generates concise function summaries.It is trained to interpret the behavior of cryptographic functions in stripped binaries.Source: §4 · pp. 11–12
• The models share one encoder: FoC-Sim reuses the trained encoder and trains only a lightweight GCN and MLP, while both models run in parallel after encoding at inference.A single input can yield both a semantic summary and retrieval of homologous similar functions.Source: §4 · pp. 11–12
• FoC-Sim embeds functions using FoC-BinLLM's semantic encoder together with control-flow graph structure and cryptographic features.Its representations are change-sensitive, addressing the generative model's insensitivity to minor code changes.Source: §4 · p. 12
• FoC-Sim can retrieve source-level information when a homologous implementation is present in its database and can be used to inspect binaries for vulnerable cryptographic implementations.Source: §4 · pp. 12–13

5 DETAILED DESIGN

• FoC's detailed design reuses FoC-BinLLM's encoder to create semantic representations for FoC-Sim.Source: §5 · p. 13, §5.2 · p. 15

5.1.1 Golden Model Initialization.

• FoC-BinLLM starts from pretrained CodeT5+, whose encoder-decoder architecture supports both understanding and generation tasks.CodeT5+ is initialized from previous pretrained LLM weights and trained on large source-code datasets.Source: §5.1.1 · p. 13

5.1.2 Multi-Task & Frozen-Decoder Training.

• FoC-BinLLM uses function-name prediction, binary-code summarization, and binary-source contrastive learning; the contrastive task brings source and binary embeddings closer.Name prediction gives a brief functional cue, while summaries describe behavior more fully; contrastive learning adapts the source-code-pretrained model to binary code.Source: §5.1.2 · pp. 14–15
• The decoder is frozen while the encoder and cross-attention layer remain trainable, reducing the number of trainable parameters.The decoder has more layers than the encoder because the authors regard it as critical to complex causal-generation tasks.Source: §5.1.2 · p. 15

5.2.1 Code Semantic & Control Structure Encoding.

• FoC-Sim represents pseudo-code with the FoC-BinLLM encoder and summarizes an attributed control-flow graph with a five-layer GCN.The GCN aggregates neighboring-node information across up to five graph steps, then sums node states into a function-structure embedding.Source: §5.2 · p. 15, §5.2.1 · p. 16

5.2.2 Cryptographic Features.

• Cryptographic features combine basic-block arithmetic and logic opcode statistics with function-level keyword and graph statistics in a 65-dimensional vector.The function-level features include cryptographic-class keyword counts, basic blocks, edges, and callees.Source: §5.2.2 · pp. 16–17

5.2.3 Embedding Fusion & Model Training.

• FoC-Sim fuses semantic, control-structure, and handcrafted cryptographic-feature embeddings with a single-layer MLP.Training pairs are sampled from the dataset; functions with the same name in the same file and project are treated as similar, and MultipleNegativesRankingLoss is used.Source: §5.2.3 · p. 17

6.1 Research Questions

• The evaluation asks whether FoC-BinLLM summarizes cryptographic binaries, FoC-Sim detects similarity, FoC works in virus and vulnerability analyses, and its components contribute to performance.Source: §6.1 · p. 17

6.2.1 Dataset.

• The training set combines the cryptographic dataset with binaries from GNU repositories, then uses MD5 deduplication and MinHash filtering to reduce leakage and textual similarity.Five percent of the training data is held out for validation.Source: §6.2.1 · p. 18

6.2.2 Evaluation Metrics.

• Summarization is evaluated with BLEU-4, METEOR, and ROUGE-L; similarity detection uses AUC, Recall@K, and MRR@K across pairwise comparison and search.The setup also evaluates similarity through ROC curves and reports TPR and FPR across thresholds.Source: §6.2.2 · pp. 18–19

6.3.1 Environment.

• Experiments run on Ubuntu 20.04 with a 48-core Intel Xeon Gold 5220 CPU, 256 GB RAM, and NVIDIA RTX 3090 GPUs.IDA Pro decompiles binary functions, and the models are implemented with PyTorch and Transformers and accelerated with DeepSpeed ZeRO2.Source: §6.3.1 · pp. 19–20

6.3.2 Model & Training Setting.

• FoC-BinLLM uses a 12-layer encoder and decoder with 38.11% of parameters trainable; FoC-Sim uses that model's encoder, a five-layer GCN, and a 256-dimensional MLP.Source: §6.3.2 · p. 20
• FoC-BinLLM trains for four epochs, with one for contrastive learning and three for name prediction and summarization; FoC-Sim trains for 110,000 steps.Source: §6.3.2 · p. 20

7.1 Answer to RQ1: Performance in Cryptographic Binary Code Summarization

• 30.2% Precision, 31.7% Recall, and 32.6% F1-score: FoC-BinLLM significantly outperforms the function-name prediction baselines.Its F1-score is 19.2% and 15.3% higher than SymLM and XFL, respectively, and 14.0% and 13.1% higher than Mixtral and ChatGPT.Source: Table 9 · p. 21, §7.1 · pp. 21–22
• 41.34% ROUGE-L, 14.47% BLEU-4, and 40.20% METEOR: FoC-BinLLM outperforms all existing summarization baselines.It exceeds general LLMs Mixtral and ChatGPT by 3.28%, 2.15%, and 5.59% on average across the three metrics, respectively.Source: §7.1 · pp. 21–22
• 0.1533 seconds per binary function: FoC-BinLLM processes functions faster than Mixtral’s 9.0232 seconds, while ChatGPT timing is not evaluated.ChatGPT is accessed through an API, so its time depends on network conditions.Source: §7.1 · p. 21
• 46 of 70 primitive-class instances are correctly identified by FoC-BinLLM, the most among compared methods, with 5 false positives.Other methods produced no false positives but identified substantially fewer instances.Source: §7.1 · p. 22

7.2.1 Performance on the General Dataset .

• For general-dataset one-to-one comparison, FoC-Sim reaches 99%, 98%, 97%, and 99% AUC on XC, XC+XB, XA, and XM, respectively, exceeding GMN by 14%, 12%, 11%, and 13%.The reported sub-tasks vary compiler/optimization settings, bitness, and architecture.Source: §7.2.1 · p. 23, §7.2 · p. 23
• On general-dataset XM one-to-many search, FoC-Sim scores 83% MRR@10, 78% Recall@1, and 95% Recall@10, exceeding GMN by 30%, 33%, and 37%.Zeek, SAFE, and Asm2Vec reach only 0.13, 0.16, and 0.07 Recall@1; Gemini and GMN reach 0.28 and 0.45.Source: §7.2.1 · p. 24

7.2.2 Performance on the Cryptographic Dataset.

• On the cryptographic dataset, FoC-Sim exceeds 99% AUC on all four one-to-one sub-tasks, beating RCFG2Vec by 2.6%, 2.1%, 6.4%, and 5.2%.For XM one-to-many search, it scores 94.0% MRR@10, 91.0% Recall@1, and 99.0% Recall@10, versus RCFG2Vec’s 77.5%, 69.4%, and 89.1%.Source: Table 12 · p. 24, §7.2.2 · p. 25
• FoC-Sim’s larger advantage on cryptographic data is attributed to cryptographic-related features in its embeddings; pseudo-code also provides cross-architecture capabilities.The authors contrast its pseudo-code basis with baselines that generate semantic embeddings directly from assembly.Source: §7.2.2 · p. 25
• FoC-Sim completes 100 cryptographic-function comparisons in an average of 62.26ms, comparable to SAFE and PalmTree and faster than several other baselines, though slower than Zeek’s 28.14ms.The authors characterize the extra time relative to Zeek as a reasonable trade-off given FoC-Sim’s performance improvements.Source: §7.2.2 · p. 25
• As the cryptographic one-to-many search pool grows from 10 to 10,000, Recall@1 declines for all methods, but FoC-Sim remains ahead and degrades relatively little.At pool size 10,000, FoC-Sim maintains high performance; no exact Recall@1 value is supplied here.Source: §7.2.2 · p. 25, Figure 8 · p. 25

7.3.2 Cryptographic Implementation Detection in Firmware.

• FoC-Sim detects 91/97 vulnerable functions in NETGEAR firmware, 168/181 in TP-LINK, and 319/330 in Xiaomi.The firmware database covers OpenSSL, mbedTLS, and Libgcrypt libraries from three IoT vendors; vulnerability presence is determined using library version numbers.Source: Table 13 · p. 27, §7.3.2 · p. 28

7.3.1 Cryptographic Function Analysis in Virus.

• FoC-BinLLM analyzes the virus’s encrypted-communication call chain with generally accurate names and summaries, including a behaviorally informative handshake-verification summary despite a mismatched predicted name for mbedtls_ssl_write.The example starts from main and follows callees such as SSL-context initialization, certificate parsing, and key parsing.Source: §7.3.1 · pp. 26–27

7.3.2 Cryptographic Implementation Detection in Firmware.

• The practical evaluation covers three tasks: finding vulnerable functions, distinguishing vulnerable from patched versions, and retrieving cryptographic functions in real-world stripped binaries.The first task counts a find as successful when the vulnerable function appears among the top 10 most similar functions in the suspicious file.Source: §7.3.2 · p. 27
• In fully stripped firmware binaries, FoC-Sim achieved Recall@1 0.714 and Recall@100 0.955 on 2,000 annotated cryptographic-related functions, each having at least one homologous Database Server function.Among removed components, code semantics was the most critical factor affecting detection performance.Source: §7.3.2 · p. 29
• FoC-Sim correctly distinguishes vulnerable from patched functions in 70/97 NETGEAR, 142/181 TP-Link, and 289/330 Xiaomi cases.A Libgcrypt failure involved substantially different vulnerable-function versions in firmware and only one version represented in the vulnerability database.Source: §7.3.2 · pp. 27–28

7.4.1 Ablation Studies of FoC-BinLLM.

• 2B FoC-BinLLM achieves a ROUGE-L score of 43.96%, compared to 41.34% for the 220M model.The 2B model also has a lower loss convergence bound and faster convergence speed.Source: §7.4.1 · p. 30, Table 15 · p. 30
• With Task3, Binary-Source Contrastive Learning, FoC-BinLLM has lower training loss at the same steps than without Task3.The authors describe Task3 as facilitating rapid adaptation to the binary domain; they did not ablate Tasks 1 and 2.Source: §7.4.1 · p. 30

7.4.2 Ablation Studies of FoC-Sim.

• Removing code semantics, control structures, and cryptographic features lowers FoC-Sim Recall@1 by 18%, 4%, and 1%, respectively, especially in One-to-many Search.Code semantics has the largest impact.Source: §7.4.2 · p. 30, Table 15 · p. 30

Quality of Summaries

• Semantic-label quality depends largely on the LLM used; the discriminator can ensure key cryptographic semantics, while higher-quality labels in general domains remain future work.The authors identify automated semantic-summary generation as the source of labels.Source: §8 · p. 31

Primitive Classes of Discriminator

• The collected primitive classes may omit algorithms, especially in extreme cases or special applications.The authors propose systematically expanding the range of primitives.Source: §8 · p. 31

Function Context

• FoC analyzes binary functions independently, and the authors report occasional wrong predictions in virus analysis that may stem from missing context.They suggest incorporating dependencies and cross-references between functions.Source: §8 · p. 31

Obfuscated Binaries

• FoC does not consider obfuscated binaries, which the authors treat as orthogonal work and identify as important in applications such as malware analysis.They suggest training with obfuscated samples to improve robustness.Source: §8 · p. 31

9.1 Binary Analysis

• Traditional static and dynamic binary-analysis tools are labor-intensive, expertise-dependent, and require substantial manual intervention.Stripped binaries also lack symbolic information such as function names and variable labels, limiting readability.Source: §9.1 · pp. 31–32
• Deep-learning methods address binary tasks including recovering names, types, and debug information, summarizing code, and detecting similarity.The discussion includes both data-driven binary-analysis methods and LLM-based tools for decompilation and analysis.Source: §9.1 · p. 32
• FoC focuses on binary-code summarization and similarity detection within the broader binary-analysis landscape.The discussion identifies both as areas where deep learning and LLMs have been applied.Source: §9.1 · p. 32

9.2 LLM for SE

• LLMs for software engineering span tasks across the software-development life cycle, while source-code models perform code generation, summarization, search, and semantic understanding.The paper argues these capabilities motivate exploring LLMs for binary-code tasks.Source: §9.2 · pp. 32–33

10 CONCLUSION

• FoC-BinLLM exceeds ChatGPT by 14.61% on ROUGE-L, and FoC-Sim achieves 52% higher Recall@1 than previous methods on the cryptographic dataset.The authors also report practical ability in cryptographic virus analysis and 1-day vulnerability detection.Source: §10 · p. 33
• The paper provides a public cryptographic dataset for future research on cryptographic functions in stripped binaries.The authors state that it addresses challenges not handled by existing works.Source: §10 · p. 33