PaperlayerView exact revision on arXiv

Source-linked AI summary

AirLog: Store-Level Indoor Life Logging Made Easy

Zihui Yun, Jiaying Du, Yue Yu, Zhewei Liu, Zhen Xiang, Longfei Shangguan, Zhenlin An

arXiv:2609.31864v1cs.NIcs.AI

TL;DR

Store-level indoor life logging is difficult because existing localization often depends on site-specific surveys or infrastructure, while indoor maps are not readily queryable. AirLog combines directory-derived region maps with Wi-Fi and motion sensing to reconstruct visits and generate journals. Its evaluation reports improvements across store-level recovery, semantic matching, trajectory reconstruction, and journal quality.

  • Problem

    Store-level indoor logging lacks readily queryable region maps, and conventional localization often requires site-specific surveys or infrastructure.

    Source: §4 · p. 4

  • Method

    AirLog converts directory images into labeled region maps and combines Wi-Fi semantic anchors with inertial motion and map constraints to reconstruct store visits.

    Source: §1 · p. 2, §4.1 · p. 5

  • Results

    AirLog reports improvements over baselines in store-level region recovery, semantic matching, trajectory reconstruction, and journal quality.

    Source: §1 · p. 3

  • Takeaways & Limitations

    AirLog demonstrates store-level indoor logging without dedicated localization infrastructure by grounding everyday signals in spatial artifacts already present in commercial environments.

    Source: §9 · p. 13

Abstract

from arXiv · show

This paper presents AirLog, a smartphone-based life journaling system that automatically reconstructs users' store visits in shopping malls and summarizes them into human-readable journals. Unlike conventional indoor localization systems, AirLog avoids labor-intensive radio-map construction and dedicated wireless localization infrastructure and algorithm calibrations. Instead, it repurposes two cues already available in commercial spaces: semantic information exposed by ambient Wi-Fi SSIDs and indoor directory images. AirLog converts directory images into spatial maps and fuses Wi-Fi semantic anchors with inertial dead reckoning to recover store-level trajectories, which are then summarized into journals by an LLM. Such store-level life logs can support applications such as personal memory recall, activity reflection, and automated diary generation without requiring users to manually record where they have been. We implement AirLog on commodity smartphones and evaluate it on both a large-scale public dataset and a self-collected dataset. The results demonstrate that AirLog substantially improves store-level region recovery, semantic matching, trajectory reconstruction, and journal quality over existing baselines. A human evaluation further shows that the generated journals are coherent and faithful to users' visits.

1 Introduction

AirLog targets store-level indoor life logging, where existing localization approaches make recovering store visits difficult to scale. It turns passive Wi-Fi and motion sensing plus directory images into store trajectories and human-readable journals.

Motivation

  • Store-level logging asks which mall stores a user visited, in what order, and for how long.Such records can support memory recall, visit retrospection, and context-aware assistants.Source: §1 · p. 1

Challenge

  • GPS is unreliable indoors, while conventional Wi-Fi localization often requires site surveys, fingerprint databases, or dedicated pipelines.These requirements make store-level logging difficult to scale across large, changing commercial spaces.Source: §1 · p. 1

Opportunity and challenge

  • Ambient Wi-Fi SSIDs and directory images offer semantic clues and region locations, but the paper asks whether they can recover trajectories without traditional localization infrastructure.An SSID can be ambiguous about whether a user entered a nearby store or passed it.Source: §1 · pp. 1–2

AirLog

  • AirLog combines passive Wi-Fi scans, inertial recordings, and a crowd-accessible floor plan to infer visited stores, their order, and dwell time.It structures the task into map construction, trajectory reconstruction, and journal generation.Source: §1 · p. 2
  • Its geometry-first parser converts directory images into labeled GeoJSON regions, while trajectory inference treats SSIDs as probabilistic anchors and combines them with PDR and map connectivity.The parser leaves corridor and connectivity derivation to a separate deterministic map-preparation stage.Source: §1 · p. 2

Evaluation and journaling

  • The journal layer derives supported behavior events from dwell time, POI type, motion cues, and trajectory confidence before an LLM summarizes them.Evaluation uses public indoor traces and a self-collected corpus; local processing takes longer than cloud execution but keeps raw data and journals on-device with comparable accuracy from smaller models.Source: §1 · p. 3

2 Related Works

Prior work addresses lifelogging, indoor localization, and map understanding, but AirLog focuses on grounding indoor store visits in map structure before generating journals.

2.1 Life Journaling

  • 2.1 Life Journaling: LLM-based systems describe heterogeneous sensor data, but often use prompt-level context and target outdoor mobility or coarse activities.AirLog grounds Wi-Fi and motion evidence on indoor map structure to recover store-level trajectories before journal generation.Source: §2.1 · p. 3

2.2 Indoor Localization

  • 2.2 Indoor Localization: Fingerprinting and calibrated ranging or AoA methods require deployment-specific measurements or infrastructure, while survey-free systems mainly target metric coordinates.AirLog instead infers visited stores from a single visit without site-specific fingerprints or accumulated crowdsourced observations.Source: §2.2 · p. 3

2.3 Map Understanding with VLMs

  • 2.3 Map Understanding with VLMs: Existing systems struggle to jointly handle OCR, topology, and route planning, and public indoor maps rarely expose queryable floor-level geometry.AirLog converts directory images into labeled GeoJSON region maps for reasoning over explicit polygons and labels.Source: §2.3 · p. 3

3 A Glimpse of AirLog

AirLog turns a mall directory and passive phone sensing into a store-level visit record and journal.

Map preparation

  • A directory photo is converted into labeled store polygons because trajectory agents need explicit geometry and POI labels rather than raw pixels.A public map may identify the mall without providing queryable floor-level geometry.Source: §3 · p. 4

Trajectory reconstruction

  • Wi-Fi SSIDs suggest nearby stores, PDR estimates movement, and floor-plan topology constrains plausible paths and helps distinguish passing from sustained visits.Seeing a store’s SSID alone does not establish that the user entered it.Source: §3 · p. 4

Journal generation

  • AirLog aggregates the reconstructed path into store visits using dwell time and motion cues, then summarizes the validated sequence into a human-readable journal.The example distinguishes walking through an area from spending about 25 minutes at a store.Source: §3 · p. 4

4 Constructing Labeled Region Maps from Floor-Plan Images

AirLog converts human-facing directory images into labeled region maps that downstream modules can query. Its geometry-first pipeline recovers polygons before assigning labels and exporting the result.

4 Constructing Labeled Region Maps from Floor-Plan Images

  • Directory images expose store regions and names as pixels, while trajectory reconstruction needs an explicit mapping from each POI label to its polygon.Geometry and label association are coupled: an error in either can produce an incorrect map and mislead trajectory reconstruction.Source: §4 · p. 4
  • AirLog addresses heterogeneous directories by canonicalizing region geometry and assigning semantics directly to each recovered region.The approach avoids relying on OCR-readable room identifiers that must be matched back to the directory.Source: §4 · p. 4

4.1 Geometry Pass: Recovering Regions

  • GPT Image 2 generates a clean region mask, then deterministic processing aligns it and extracts polygon features.Alignment permits uniform scaling and horizontal and vertical translation, excludes rotation and non-rigid deformation, and falls back to identity if reliable alignment fails.Source: §4.1 · p. 5

4.2 Per-Region Semantic Subagent

  • AirLog constructs one polygon-masked crop per recovered region, preserving its source pixels while suppressing neighboring labels and decorations.A per-region semantic subagent returns readable labels for that crop, which are written to the corresponding feature.Source: §4.2 · p. 6

4.3 Generate GeoJSON

  • The resulting GeoJSON FeatureCollection stores each region’s Polygon or MultiPolygon geometry and readable strings in properties.labels.Corridors and navigability graphs are derived separately rather than included in the parser’s GeoJSON output.Source: §4.3 · p. 6

5 Semantic Trajectory Reconstruction

AirLog combines Wi-Fi semantic anchors, inertial motion, and map constraints to reconstruct store-level trajectories that are semantically plausible and physically feasible. Its pipeline treats Wi-Fi as uncertain evidence, bridges gaps with PDR, and uses map-constrained fusion and review to recover a consistent path.

5 Semantic Trajectory Reconstruction

  • A multi-agent pipeline coordinates Wi-Fi semantic anchoring, PDR motion estimation, and map-constrained inference to construct semantically plausible, physically feasible trajectories.The context-fusion and review stages integrate location candidates, motion, and walkable-map constraints, then detect failures and trigger targeted replanning.Source: §5 · p. 6, §5.1 · p. 6, Figure 6 · p. 6

5.2 Wi-Fi Semantic Agent

  • Across six cities, 185,140 candidate SSID–POI pairs show SSID semantics remain observable at scale, while lexical matching alone is too sparse and noisy as a direct signal.Simple SSID-set Jaccard matching achieves 87.88% accuracy in identifying the correct site–floor pair; SSID distributions are venue- and floor-specific.Source: §5.2 · pp. 6–7

5.2.1 SSID Semantic-noise filtering.

  • AirLog removes generic infrastructure SSIDs, aggregates scans into sliding windows, and passes top-ranked SSID–RSSI pairs as soft evidence rather than treating the strongest signal as the location.Gaps longer than 600 s are flagged to prevent incorrect PDR integration across disconnected sessions.Source: §5.2.1 · p. 7

5.2.2 LLM-based semantic anchoring.

  • For each Wi-Fi window, a structured prompt combines filtered SSID–RSSI pairs with POI names and map-derived spatial descriptions to produce nearby-store hypotheses with confidence scores.The model can return multiple plausible stores rather than forcing a single location; spatial relations help disambiguate lexically similar or overlapping POIs.Source: §5.2.2 · p. 7

5.3 Motion Context via PDR

  • PDR integrates detected steps and rotation-vector-derived headings into relative displacement and heading change, providing continuous motion between sparse Wi-Fi anchors.PDR remains motion context rather than a store-level trajectory because its errors accumulate over time.Source: §5.3 · p. 7

5.4 Context Fusion & Review Agent

  • A semantic-priority Viterbi decoder scores Wi-Fi consistency and IMU distance and heading consistency while restricting transitions to adjacent nodes in the walkable GeoJSON graph.An anchor-based heading-calibration bonus applies when consecutive high-confidence semantic anchors imply a reliable direction.Source: §5.4 · p. 8
  • When diagnostics indicate a low-quality decode, AirLog retries Viterbi over preset parameter configurations and may take a bounded corrective action before accepting a result.The review process uses flagged windows and read-only Wi-Fi, anchor, spatial, and PDR context.Source: §5.4 · p. 8, Figure 8 · p. 8

6 Journal Generation

AirLog turns a corrected trajectory into a Life Journal through behavior inference, motion calibration, and narrative generation. It derives candidate visits from store contacts, refines activity labels with motion and location context, and narrates validated behaviors chronologically.

6 Journal Generation

  • AirLog transforms a corrected trajectory into a diary through behavior inference, motion calibration, and journal generation.The stages convert the localization sequence into a human-readable account.Source: §6 · pp. 8–9

6.1 Behavior Inference

  • Behavior inference merges consecutive contacts near each POI into visits with enter and exit times, dwell duration, semantic confidence, and local motion summaries.Short contacts count as passing behavior, longer dwell periods form candidate visits, and structural POIs are excluded regardless of dwell time.Source: §6.1 · p. 9

6.2 Journal Generation

  • For each candidate visit, the inference agent uses store name and category, dwell duration, neighboring POIs, and an IMU activity cue to infer behavior.Motion calibration refines activity labels using PDR features and location context; the diary agent outlines chronological flow and salient visit characteristics before writing a fluent third-person narrative.Source: §6.1 · p. 9, §6.2 · p. 9

7 Implementation

The implementation is a Python prototype with structured session preprocessing and configurable LLM backends. Evaluation uses public indoor traces and a self-collected corpus, while the map-parsing benchmark compares semantic-labeling configurations with shared region geometry.

7 Implementation

  • Table 1 reports map-parsing results on the complete benchmark, with two AirLog rows sharing region geometry and differing only in semantic-labeling configuration.The caption states that higher values are better but does not provide the result values.Source: Table 1 · p. 9
  • A preprocessing module normalizes timestamps, converts Wi-Fi records into SSID–RSSI snapshots, resamples IMU streams for PDR, and serializes sessions as structured JSON.The JSON contains Wi-Fi windows, PDR motion primitives, and map context including POI names, polygon centroids, and graph-based spatial descriptors.Source: §7 · p. 9
  • GPT-4o is the default LLM backbone, with GPT-5.4, Claude Sonnet 4.6, gpt-oss-120b, Gemma 4, Qwen 3, and DeepSeek-R1 used for cross-model evaluation.Preprocessing, graph construction, Viterbi parameters, prompts, and post-processing remain fixed across backbones.Source: §7 · p. 9
  • The public dataset contains 1,071 traces, 19,712 Wi-Fi scans, and 8,704 surveyed waypoints across two shopping malls; the self-collected corpus spans 603.0 hours.The public traces support quantitative trajectory evaluation, while self-collected data support Wi-Fi semantics and journaling validation.Source: §7 · pp. 9–10

8 Evaluation

Across eight evaluations, AirLog improves map-region recovery, store-visit reconstruction, and journal quality over baselines; ablations identify Wi-Fi semantics as the strongest recovery component. User ratings support journal clarity and faithfulness, while cost and local-runtime measurements describe deployment trade-offs.

8.1 Map Parsing Evaluation

  • 0.705 Area F1 and 0.767 Region Purity for AirLog exceed Color Filling’s 0.277 and 0.510; GPT-4o region-crop labeling reaches 0.814 Matched Label F1 versus 0.734 for full-map GPT-5.5.The labeling comparison changes both backend model and input scope, so it is operational rather than a controlled ablation.Source: §8.1 · p. 10, Table 2 · p. 10

8.2 Store-Level Trajectory & Visit Recovery

  • 24.8% higher Subset Hit and 24.6% higher Avg Hit Ratio than AutoLife accompany 84.6% Recall and 0.778 SeqSim, versus AutoLife’s 63.6% and 0.327.Remaining errors mainly occur in anchor-sparse corridors and areas with overlapping Wi-Fi semantics.Source: §8.2 · p. 11, Figure 12 · p. 11

8.3 Journalling Recovery

  • 0.000 hallucination rate, 26.9% higher chrF, and 53.3% higher BERTScore F1 than SenLLM make AirLog best across the reported journal metrics.The improvements over AutoLife are 10.2% for chrF and 23.7% for BERTScore F1.Source: §8.3 · pp. 11–12, Figure 13 · p. 11, Table 3 · p. 12

8.4 Ablation Study

  • Removing the Wi-Fi Semantic Agent drops Subset Hit and Avg Hit Ratio from 71.2%/75.5% to 38.3%/42.3%, and Recall from 84.6% to 36.0%.Map Agent and Context Fusion also reduce point-level recovery; Motion Agent mainly improves visit-level consistency, while Planning Agent has the smallest point-level effect.Source: §8.4 · p. 12, Figure 14 · p. 13

8.5 Cross-Model Evaluation

  • Trajectory recovery remains relatively stable across LLM backbones, but diary generation is more sensitive; Qwen3 (32B) has 91.7% Recall alongside a 0.313 hallucination rate.GPT-5.4 has the strongest overall trajectory performance: 73.9% Subset Hit, 77.3% Avg Hit Ratio, 83.3% Recall, and 0.833 SeqSim.Source: §8.5 · p. 12, Figure 15 · p. 13

8.6 Human Evaluation Study

  • Human ratings describe journals as clear, relevant, and faithful, with correctness a strength and conciseness needing improvement.Third-party ratings align with the authors’ self-assessment, and raters show substantial agreement.Source: §8.6 · p. 12, Figure 16 · p. 13

8.7 Token Cost

  • 208.7k input and 43.4k output tokens per trace-hour correspond to an estimated $0.96 per hour, dominated by Wi-Fi semantic anchoring and trajectory validation.The dollar figure is a nominal estimate, not a verified gateway invoice.Source: §8.7 · p. 12, Table 4 · p. 12

8.8 Mobile and Edge Deployment

  • 153 mAh (3% battery) and 70 MB of local data per sensing hour indicate modest passive-sensing overhead; local processing takes roughly 75 minutes per trace-hour.The tested local setup is suited to deferred analysis rather than interactive use; map parsing can be done offline once and reused at the same venue.Source: §8.8 · p. 12, Figure 16 · p. 13

9 Conclusion

AirLog turns human-facing floor plans into machine-readable spatial structures that ambient Wi-Fi can be anchored to, enabling store-level indoor logging without dedicated infrastructure. The paper points to interpreting built-environment artifacts and grounding everyday signals against them as a broader direction.

9 Conclusion

  • AirLog converts a floor plan intended for human interpretation into a machine-readable spatial structure.The structure provides a map to which ambient Wi-Fi can be anchored.Source: §9 · p. 13
  • This makes store-level indoor logging possible without dedicated infrastructure.Source: §9 · p. 13
  • The authors identify interpreting spatial artifacts in built environments and grounding everyday signals against them as a broader direction for agents.Source: §9 · p. 13

A Map Parsing Evaluation and Failure Analysis

The appendix expands the map-parsing claims and results, explaining the metric choices and scores and presenting qualitative cases related to design choices and limitations.

A Map Parsing Evaluation and Failure Analysis

A.1 Metric Design and Interpretation

The metrics separate tolerant region recovery, polygon cleanliness, and label accuracy after geometry is matched. Together, they assess whether the parser produces queryable regions with useful labels.

A.1 Metric Design and Interpretation

  • Area F1 rises from 0.277 to 0.705, indicating more successful one-to-one region recovery rather than 70.5% pixel overlap.The tolerant criterion prioritizes missing, split, or duplicated regions over small boundary deviations.Source: §A.1 · p. 16
  • Region Purity rises from 0.510 to 0.767, indicating predictions absorb less unrelated content.Purity complements Area F1 by measuring overlap with annotated foreground; image-level means are macro-averaged.Source: §A.1 · p. 16
  • Matched Label F1 evaluates canonicalized label sets after label-independent geometry matching at IoU ≥0.5.Unmatched polygons remain part of geometry metrics, avoiding counting geometry failures again as semantic errors.Source: §A.1 · p. 16
  • Under standard one-to-one matching at IoU ≥0.5, the method ordering remains unchanged for Region F1@0.5 and PQ@0.5.Table 5 reports these metrics macro-averaged across 24 maps.Source: §A.1 · p. 16, Table 5 · p. 16

A.2 Benchmark and Comparison Protocol

The appendix documents the benchmark and protocol, clarifies the boundary between map parsing and downstream preparation, and illustrates remaining map and Wi-Fi evidence failures. These materials provide context and qualitative examples rather than additional quantitative comparisons in several subsections.

A.2 Benchmark and Comparison Protocol

  • The benchmark spans 24 maps, including 20 public mall directories and four in-the-wild photographs; ground-truth geometry and labels were manually maintained or reviewed.Baseline settings were fixed across maps, and the two AirLog labeling configurations differ in both model and input scope, so their comparison isolates neither factor.Source: §A.2 · pp. 16–17
  • The permissive SAM 3 setup produced 24,939 polygons for 1,482 annotated polygons, with 68.05% of predictions lacking annotated-foreground overlap.A more conservative threshold yielded Region F1@0.5 of 0.042 and PQ@0.5 of 0.033, with predictions on only one of 24 maps; this sensitivity check does not indicate robust benchmark performance.Source: §A.2 · p. 17

A.3 Downstream Map Preparation

  • The parser outputs labeled GeoJSON regions, while a separate deterministic stage derives corridor masks and the navigability graph for trajectory fusion.The map-parsing metrics assess region geometry and labels, not corridor or graph construction.Source: §A.3 · p. 17, Figure 17 · p. 18

A.4 Qualitative Evidence and Remaining Failures

  • Exact-region crops do not prevent OCR orientation or glyph errors, and inaccurate polygons can remove readable text needed by the semantic subagent.A full-map example also places a recognized VOGUE label in the adjacent UBERFONE polygon; these are qualitative failure mechanisms, not additional quantitative comparisons.Source: §A.4 · p. 18, Figure 19 · p. 18

B Wi-Fi Semantic Evidence and Collection Context

  • The Wi-Fi evidence appendix covers collection context, sparse and noisy lexical SSID–POI matching, and RSSI ambiguity that motivates soft semantic anchoring.The collection figures document coverage; indoor reconstruction remains grounded in Wi-Fi observations, PDR continuity, and indoor topology.Source: B Wi-Fi Semantic Evidence and… · p. 18, §B.1 · p. 18

B.1 Collection Coverage

  • Figure 20 summarizes two urban collection areas and session-level mobility coverage in Hong Kong.The figures document collection context rather than a standalone store-reconstruction result.Source: §B.1 · p. 18

B.2 Lexical Similarity and Representative Failure Cases

Lexical overlap between SSIDs and POI names is sparse, and surface similarity can still produce misleading matches. The appendix also illustrates RSSI ambiguity and provides supplementary journal examples and prompt designs for spatial context and Wi-Fi anchoring.

B.2 Lexical Similarity and Representative Failure Cases

  • Only 647 of 185,140 SSID–POI pairs exceed 0.8 character-level similarity, at 52.2% precision; high-similarity word-level and TF-IDF matches cover only 11 and 40 pairs.The passages characterize exact token matches as extremely sparse, while representative cases show that surface similarity can rank semantically misleading pairs highly.Source: §B.2 · pp. 18–19, Table 7 · p. 19

B.3 RSSI Ambiguity

  • Overlapping nearby access points can leave one signal window compatible with multiple neighboring stores, motivating top-K SSIDs as soft evidence rather than strongest-signal location selection.Figure 21 illustrates the signal profiles and spatial layout behind this ambiguity.Source: §B.3 · p. 19, Figure 21 · p. 20

C Additional Journal Examples

D.1 Spatial Description Generation

  • The spatial-description prompt turns GeoJSON store features into normalized names, neighbor relations, and natural-language position descriptions for later prompts.This compact context avoids direct reasoning over raw polygon coordinates.Source: §D.1 · p. 21, Figure 24 · p. 21

D.2 Wi-Fi Semantic Anchoring

  • The Wi-Fi anchoring prompt maps SSIDs to candidate store locations using semantic similarity, relative signal strength, and spatial context.The prompt template is shown in Figure 25.Source: §D.2 · p. 21, Figure 25 · p. 21

Related papers

Loading 2609.31864v1…