Source-linked AI summary
Scalable PII Discovery in Mobile App Databases via Hypothesis-Driven Search
Jeel Piyushkumar Khatiwala, Samad Afolabi, Ruoyao Xiao, Yu Luo, Dianxiang Xu, Weifeng Xu
TL;DR
The paper addresses the difficulty of locating PII across heterogeneous mobile SQLite databases and semi-structured fields. It introduces a hypothesis-driven, memory-guided search that validates candidate regions before targeted extraction. Across the evaluated corpus, the framework maintains strong discovery effectiveness while reducing the extraction search space, with performance varying substantially by model backend.
Problem
PII localization is difficult because relevant regions are unknown across heterogeneous SQLite schemas and values may be embedded in free-text or semi-structured fields.
Method
An agent ranks table-column hypotheses, probes sampled values, stores evidence and decisions in memory, and extracts only from validated regions.
Results
Across five PII types, recall ranges from 86.7% for email addresses to 96.1% for phone numbers, while Gemini-2.5-Pro achieves 94.47% F1.
Takeaways & Limitations
Sampled validation substantially reduces the extraction search space while maintaining strong discovery effectiveness.
Takeaways & Limitations
The evaluation uses 25 databases from a controlled Cellebrite CTF corpus and one run per backend, limiting evidence about population-level generalizability and repeated-run stability.
Abstract
from arXiv · showhide
Discovering personally identifiable information (PII) in mobile forensic databases is difficult because the relevant table-column regions are unknown, distributed across heterogeneous SQLite schemas, and may contain values embedded in free-text or semi-structured fields. We present a hypothesis-driven framework that treats PII localization as bounded, adaptive search under uncertainty. An agent ranks candidate table-column regions, probes sampled values, and maintains a memory of prior evidence, confidence scores, and decisions to refine subsequent hypotheses. The framework separates lightweight PII exploration from targeted extraction, normalization, and deduplication over validated regions, thereby limiting exhaustive inspection to regions supported by sampled evidence. We evaluate the framework on 25 SQLite databases from 10 Android and iOS applications in the Cellebrite CTF corpus, targeting email addresses, phone numbers, domain names, person names, and postal addresses. Against a corpus-level distinct ground-truth set of 3,751 entities, Gemini 2.5 Pro achieves 94.5% F1 while reducing the effective extraction search space by 79.9% on average. Results across 12 model backends show strong performance among several frontier models, but substantial sensitivity to model capability.
1. Introduction
Mobile forensic databases distribute PII across heterogeneous, poorly documented SQLite schemas, with relevant values often hidden in misleading or semi-structured regions. The paper addresses this localization problem with bounded, adaptive search that guides targeted extraction.
- Motivation: PII localization is difficult because relevant table-column regions are unknown across changing, app-specific SQLite databases.Values may occur in messages, serialized objects, logs, caches, and auxiliary tables.
- Motivation: Broad scanning and application-specific methods either create substantial noise or depend on known schemas and artifact locations.Pattern-based extraction also offers limited contextual interpretation for names and postal addresses.
- Approach: The framework treats PII localization as bounded, adaptive search under uncertainty, using an agent to generate and refine region hypotheses.Exploration results, evidence, confidence scores, and decisions guide subsequent searches.
- Approach: A two-stage design separates lightweight exploration from targeted extraction, normalization, and deduplication over validated regions.This limits exhaustive inspection to regions supported by sampled evidence.
- Evaluation: The evaluation covers 25 SQLite databases from 10 Android and iOS applications across 12 language-model backends.It measures discovery effectiveness, extraction-space reduction, and comparisons with bulk extractor and Microsoft Presidio.
2. Related Work
Prior mobile-forensics methods use application-specific schemas, static code inference, bulk scanning, or text-oriented PII detectors. The paper positions its planning layer upstream, locating database regions before dispatching content to LLM or conventional extraction systems.
- Existing forensic methods: Application-specific forensic studies and tools rely on known schemas or artifact locations, limiting their portability across changing databases.Static approaches such as EviHunter are sensitive to application updates and obfuscation.
- Extraction approaches: Bulk extractor scans efficiently but produces noise and offers limited support for semantically complex entities such as person names and postal addresses.LLM-based extraction improves semantic interpretation but generally assumes relevant content has already been localized.
- PII detection systems: Commercial PII detectors combine regular expressions, validation rules, and named-entity recognition on text streams or preselected fields.They do not plan over database schemas.
- Positioning: The proposed planning layer operates upstream and can dispatch localized column content to either an LLM or a conventional extractor.This connects schema-level localization with downstream text-oriented detection systems.
3. Problem Formulation and System Architecture
The framework models PII discovery as adaptive, budgeted search over table-column regions, using schema metadata, target specifications, sampled evidence, and memory to refine hypotheses. Validated regions then feed targeted extraction under fixed exploration and confidence rules.
- Problem formulation: A region is a table-column pair r = (T,C), while σ(r) contains schema metadata and φ(E) specifies the target PII semantics and formats.The specification also includes examples and exclusion criteria.
- Problem formulation: Search must handle uncertainty about whether a PII type exists, where it is stored, and how it is represented.Regions are ranked and probed adaptively using sampled values rather than schema names or fixed patterns alone.
- Problem formulation: The objective is to identify validated regions R⋆ containing instances of E with at most B exploratory probes.The final discovery output unions entities extracted from all validated regions.
- Adaptive planning: The planner combines φ(E), σ(r), and memory M_t to generate relevance hints, rank regions, and produce decisions after sampling.Hints are hypotheses rather than evidence; exploration returns observed evidence, confidence, and reject, confirm, or validate decisions.
- Adaptive planning: Memory stores explored regions, evidence, confidence scores, decisions, rejected hypotheses, and remaining budget so ranking evolves with accumulated evidence.This allows revised hypotheses and new hints for subsequent probes.
- Workflow: Figure 1 links planning, exploration, memory updates, and targeted extraction into one adaptive discovery workflow.Validated regions are passed onward for extraction.
- Implementation: The implementation uses B = 2 probes per database–PII-type pair, with each probe sampling at most 15 rows from one candidate region.A confident positive or negative judgment uses c_t ≥ 0.6; otherwise the planner reprobes while budget remains.
- Implementation: Model-generated confidence scores are uncalibrated, while the budget, threshold, and sampling limit were fixed empirically to bound exploration cost.Type-specific regular-expression prefilters restrict which text-column values enter samples.
4. Evaluation
The evaluation measures PII discovery across 25 heterogeneous SQLite databases, five entity types, and 12 language-model backends. Gemini 2.5 Pro delivers strong corpus-level performance and substantial extraction-space reduction, while results vary markedly by entity type and backend.
- Robustness Across PII Types: 86.7%–96.1% recall was maintained across email, phone, domain, person-name, and postal-address categories.Phone numbers and person names achieved both high recall and high precision, whereas precision was more sensitive for ambiguous boundaries and small categories.
- Robustness Across PII Types: 50.0% precision was obtained for postal addresses, despite 91.7% recall, with only 12 postal addresses in ground truth.Domain-name precision was 68.7%, reflecting boundary and canonicalization differences such as s.whatsapp.net versus whatsapp.net.
- Model Sensitivity: 94.47% F1 was achieved by Gemini 2.5 Pro across the corpus-level distinct PII evaluation.The framework was assessed against 3,751 distinct ground-truth entities, with recall prioritized because missed identifiers are costly in forensic triage.
- Discovery Yield and Search-Space Reduction: 79.9% effective extraction-space reduction was combined with 94.24% recall by Gemini 2.5 Pro.Reduction measures column-level pruning before exhaustive extraction, not runtime, token consumption, or computational cost.
5. Discussion and Conclusion
The framework reduces extraction to regions supported by sampled evidence, but its findings remain bounded by evaluation scope, model dependence, and unsupported data formats. The authors recommend broader evaluation and analyst verification before investigative or legal use.
- Scope and safeguards: The reported reduction measures column-level pruning, not runtime, token cost, API cost, or analyst effort.Structured outputs, read-only queries, and type-specific prefilters address misleading schemas and adversarial content, but a five-record test does not establish adaptive-attack robustness.
- Scope and safeguards: Budgeted exploration may miss low-salience regions, and performance depends on confidence thresholds and model capability.The implementation also has limited support for encoded, deleted, cloud-backed, non-U.S. phone, cross-application, and unevaluated PII settings.
- Implications and future work: Future work will broaden format and PII coverage and evaluate stability, cost, latency, and analyst effort.Discovered identifiers may seed open-source intelligence enrichment, but recovered entities require analyst verification before investigative or legal use.
- Reproducibility: Code, ground truth, prompts, configurations, logs, and evaluation scripts are publicly available, while the source data is available through NIST CFReDS.