Automated Extraction of Personal Knowledge from Smartphone Push Notifications

1 INTRODUCTION

• Personal knowledge extraction matters for services such as recommenders, virtual assistants, and authentication, but direct access to smartphone data conflicts with privacy priorities.The paper represents extracted knowledge as <user, relation, entity> triples.Source: §1 · p. 1
• Push notifications summarize profiles, relationships, locations, shopping, and everyday events while offering a uniform interface across otherwise siloed apps.Their fixed-template structure makes personal information easier to identify than arbitrary notification text.Source: §1 · p. 1
• Sequence labeling is poorly suited to arbitrary entity phrases in notifications, motivating template-based extraction of parameter values and their meanings.An example template yields facts such as <user, name, David> and <user, profession, software engineer>.Source: §1 · p. 2
• The system discovers templates on devices, uploads only templates for offline semantic-model training, and extracts knowledge locally using the trained model.This avoids sending specific user information to the server.Source: §1 · p. 2
• 86.8% template-discovery precision and 83%/91% F1 on unseen templates show accurate semantics for new-app and existing-app templates, respectively.The evaluation used around 120 million notifications from 100,000 smartphone users.Source: §1 · p. 2
• Around 5.7% of notifications contain personal knowledge, and the method extracts it with high accuracy in an Android prototype.The prototype supports personalized services such as recommender systems and virtual personal assistants.Source: §1 · p. 2

2.1 Push Notifications

• Notifications deliver varied events, including messages, purchases, recommendations, updates, and device events, without requiring apps’ normal interfaces.This makes them a prompt channel for information that can contain valuable personal content.Source: §2.1 · p. 2
• Most push notifications use fixed templates with user-specific parameters, making personal information extractable once the templates are known.Remote notifications are more dynamic and less structured than locally generated notifications.Source: §2.1 · p. 2

2.2 Knowledge representation

• Relational knowledge bases represent information as subject-predicate-object triples, with subjects and objects as entities and predicates as relations.The W3C RDF abstract syntax provides this triple-based representation.Source: §2.2 · p. 3
• Personal knowledge is modeled as user-centered <user, relation, entity> triples, using relations suited to information appearing in push notifications.Examples include birthplace and parent relations in prior formulations.Source: §2.2 · p. 3
• The paper’s ontology covers 11 relations across four categories: user profile, social relationship, location, and shopping.Additional knowledge categories can be added in future work.Source: §2.2 · p. 3, Table 1 · p. 3

3 PERSONAL KNOWLEDGE EXTRACTION FROM PUSH NOTIFICATIONS

• The approach targets automated extraction of <user, relation, entity> knowledge triples from smartphone notifications with little-to-none manual effort.Source: §3 · p. 3

3.2 Template Discovering

• Automated discovery is needed because notification templates vary across apps and versions, new templates continually appear, and manual enumeration is impractical.Uploading all privacy-sensitive notifications for joint server analysis is also undesirable.Source: §3 · p. 3, §3.2 · p. 4

3.1 Approach Overview

• The pipeline has three phases: server-side template learning, on-device template understanding, and on-device knowledge extraction.The learned templates and inferred semantics identify parameters and generate personal knowledge triples.Source: §3.1 · p. 3, Figure 1 · p. 4
• Template parameters are treated as potential personal knowledge entities because they vary across notifications generated from the same template.For example, repeated order-shipping notifications yield a template such as “Your order $param has been shipped”.Source: §3.1 · p. 3
• A server-trained semantic model performs multi-label classification to map notification templates to knowledge triple templates, such as purchases relations.The resulting semantic rules are applied to notification content to produce concrete triples such as <user, purchases, iPhone X>.Source: §3.1 · pp. 3–4

3.2 Template Discovering

• Local template discovery filters notifications, clusters similar messages, and extracts common patterns to separate template text from personalized parameters.The method assumes same-template notifications are close in edit distance while differing in their parameter values.Source: §3.2 · p. 4

3.2.1 Notification Filtering

• Filtering removes duplicated and unstructured notifications because they usually lack personal knowledge and can reduce clustering accuracy or increase computational load.Global duplicate counting uses hashed notifications before server upload to preserve privacy.Source: §3.2.1 · p. 4

1. Notification Filtering

• The examples include personalized order confirmations and deliveries for Alice, with product names varying across notifications.Source: §1 · p. 5
• Recommendation notifications vary by product category, including T-shirts, sports shoes, and smart watches.Source: §1 · p. 5
• The examples are organized as distinct notification messages that can be compared for repeated or varying content.Source: §1 · p. 5

2. Notification Clustering

• Four order notifications share a Dear Alice structure while varying in product and delivery status.Source: §2 · p. 5
• The order examples contain products including iPhone X, milk powder, Canon EOS, and Google Pixel 2.Source: §2 · p. 5
• Recommendation notifications share a common recommendation format while naming T-shirts, sports shoes, and smart watches.Source: §2 · p. 5

3. Template Extracting

• Template discovery filters duplicates and unstructured notifications, clusters similar messages, and extracts templates from each cluster.DBSCAN groups notifications using edit distance before pattern mining identifies common template text and parameter slots.Source: Figure 2 · p. 5, §3 · p. 5, §3.2.2 · pp. 5–6

3.2.2 Notification Clustering

• DBSCAN uses relative edit distance because same-template notifications differ mainly through parameter substitutions, with δ selected between two distance peaks.The paper sets δ = 0.5 after inspecting distance distributions; excessively large or small thresholds respectively merge distinct templates or split matching ones.Source: §3.2.2 · pp. 5–6, Figure 3 · p. 5
• Clustering groups notifications generated by one or several similar templates and removes notifications that do not belong to any cluster.This step also filters a substantial amount of noisy data before template extraction.Source: Figure 2 · p. 5, §3.2.2 · p. 6

3.2.3 Template Extracting

• Longest-common-subsequence mining identifies template bodies and parameter slots, but can confuse stable personal values with template text and variable status words with entities.In the example, “Alice” is missed as a parameter while “delivered” and “confirmed” are incorrectly treated as parameter values.Source: §3.2.3 · p. 6
• Global word frequency corrects these errors by treating user-specific low-frequency words as parameters and common words as template text, splitting mixed templates when needed.The method corrects the example into separate shipped and delivered templates.Source: §3.2.3 · p. 6, Figure 4 · p. 6
• Word frequencies are computed from hashed words across users, allowing devices to identify parameters without uploading the original words.Templates are further filtered by parameter count, non-parameter length, and user coverage before semantic analysis.Source: §3.2.3 · p. 6

3.3 Template Semantic Rules

• Template semantic rules map notification templates to 0-parameter or 1-parameter knowledge triple templates, enabling parameter values to become personal knowledge entities.For example, a job-opportunity template yields name, profession, and job-hunt knowledge templates, while a notification match fills concrete triples.Source: §3.3 · p. 7, Table 2 · p. 7

3.4 Automated Template Semantic Rule Generation

• A Bi-LSTM predicts semantic rules for unseen templates from word embeddings and contextual representations of the whole template and each parameter.The template vector predicts fixed-value KTTs, while parameter vectors predict KTTs whose entity values come from individual parameters.Source: §3.4 · pp. 7–8, Figure 5 · p. 8

4.1 Dataset Overview

• The evaluation dataset contains 119,289,901 notifications from 100,000 smartphone users, generated by 2,658 apps over 30 days, with anonymized user identities and notifications kept on the provider’s servers.Notification content comprises the title followed by the text body, with all numbers elided for security and privacy reasons.Source: §4.1 · pp. 8–9

4.2 Accuracy of Notification Template Discovering

• 86.8% overall correctness was achieved when discovering 2,788 templates from 409 apps, including 92.7% for personal-knowledge templates and 66.2% for non-personal templates.The correctness for non-personal templates is relatively low because those notifications are usually less-structured.Source: §4.2 · p. 9
• 5.7% of 119,289,901 notifications contained personal information, yielding at least 68 personal notifications per user on average.The dataset captures only a subset of smartphone notifications, so the paper expects actual devices to contain more personal notifications.Source: §4.2 · p. 9

4.3 Accuracy of Template Semantic Analysis

• 92.19% recall and 89.41% precision were obtained for new templates from existing apps, whose templates are usually similar to their earlier templates.The semantic model predicts knowledge-triple labels for unseen notification templates.Source: §4.3 · p. 10, Table 4 · p. 11
• 83.62% precision and 82.56% recall were obtained for templates from unseen apps, where different applications may express the same knowledge using different vocabularies.The paper characterizes this accuracy as acceptable for common uses such as recommender systems and conversational bots.Source: §4.3 · p. 10

4.4 Client Overhead

• 5.8 seconds were required to extract knowledge triples from 10,000 notifications on a Nexus 5 running Android 7.1.2.Template discovery took 2.8, 38.7, and 770.8 seconds for 100, 1,000, and 10,000 notifications, respectively, and both phases run only occasionally.Source: §4.4 · p. 11

• Uploaded notification templates may contain personal information when discovered templates are incorrect, weakening the system’s privacy guarantee.The authors suggest scanning potential templates for sensitive information before uploading, for example with Named Entity Recognition.Source: §5 · p. 11
• The evaluation uses remotely generated notifications from a small subset of apps, so performance in a real smartphone-wide deployment remains untested.The authors propose deploying the system and evaluating it in a real-world scenario.Source: §5 · p. 11
• The implementation covers three common personal-knowledge categories, while other categories such as work and travel information remain outside its ontology.The authors call for a more complete and formal ontology of personal knowledge.Source: §5 · p. 11
• Template learning requires some users to upload discovered templates, which may be difficult when the system lacks enough users.Possible alternatives include static analysis of application code or automatically testing apps to generate notifications.Source: §5 · p. 11

6.1 Knowledge Base Population

• Public knowledge-base approaches extract structured or open information, use fixed ontologies, or build taxonomies, whereas this work targets personal knowledge for individuals.Source: §6.1 · p. 11
• Unlike conventional sequence-labeling approaches for natural-language slot filling, this method is tailored to templated mobile notifications through template learning and understanding.Source: §6.1 · pp. 11–12

6.2 Personal Knowledge Extraction

• Personal-knowledge research mines search, dialogue, communication, sensor, and interface data, while privacy-preserving systems commonly process data locally in two phases.Source: §6.2 · p. 12

6.3 Notification Analysis

• Prior notification studies examine disruption, meta-learning, and self-logging, whereas this work uses notification content to build a personal knowledge base for personalized services.Source: §6.3 · p. 12

6.4 Wrapper Generation

• Traditional wrappers extract relational content from semi-structured sources; this work discovers push-notification templates unsupervised and learns their semantic rules with supervised machine learning.Source: §6.4 · p. 12

• The approach automatically identifies notification templates with pattern mining and learns their semantics through supervised machine learning.Only templates may be uploaded to the server for labeling and learning.Source: §7 · p. 12
• Experiments on about 120 million push notifications from 100,000 smartphone users show accurate template discovery and understanding while successfully extracting personal knowledge.The prototype was implemented on Android and evaluated on three common personal-knowledge categories.Source: §7 · p. 12