Validating FKG.in: Soundness Assessment in LLM-Augmented Indian Food Knowledge

Introduction

• FKG.in addresses fragmented Indian food knowledge by structuring machine-readable information from diverse culinary sources, including blogs, cookbooks, wikis, and videos.These often informal and inconsistent sources contain culturally grounded culinary knowledge.Source: §1 · p. 1
• Regional, linguistic, and culinary variation makes Indian food extraction and named entity recognition difficult, as names, quantities, and recipe formats vary widely.These variations complicate identifying, extracting, and standardizing food-related entities at scale.Source: §1 · p. 1
• LLM outputs can contain inaccuracies, inconsistencies, omissions, and hallucinations; even plausible ingredients absent from the source can alter recipes and affect downstream applications.Potential downstream uses include nutritional analysis, dietary recommendations, recipe retrieval, and knowledge graph construction.Source: §1 · p. 2
• The workflow validates LLM-extracted information for factual correctness, internal consistency, semantic plausibility, and source fidelity before incorporation into food knowledge infrastructures.It combines rule-based checks, statistical heuristics, semantic consistency modeling, retrieval-based verification, and human-in-the-loop review.Source: §1 · p. 2

• General-purpose factuality work spans claim decomposition and entailment [4], retrieval-verifier pipelines [5], evidence generation, proof-based verification, and knowledge-graph-oriented verification and assessment.Source: §2 · p. 2
• Many general-purpose methods assume well-formed claims and evidence, complicating their application to noisy, heterogeneous culinary data.Source: §2 · p. 2
• Food-domain research has studied hallucination risks in nutrition and ingredients [14], culinary language, recipe generation, and substitution [15], retrieval and adapted models [18] [19], and structured representations.Source: §2 · p. 2
• Existing food approaches emphasize generation quality, personalization, or domain adaptation rather than verifying recipe structure, semantic plausibility, and fidelity to source text.Source: §2 · p. 2

Structural and Numerical Inconsistency

• Structural and numerical errors include schema violations and implausible or ambiguous quantities, addressed with schema validation, grammars, rules, distribution models, and normalization heuristics.Informal and culturally specific measures complicate normalization; examples include malformed ingredient-quantity pairs and implausible amounts.Source: §3 · p. 3

Semantic and Logical Incoherence

• Structurally valid ingredients may still be semantically implausible or culturally inappropriate, so vocabulary checks, co-occurrence statistics, and Set Transformer compatibility modeling assess contextual fit.Unfamiliar or region-specific ingredients are flagged for human review to distinguish culinary variation and vernacular terms from errors.Source: §3 · p. 3

Fidelity to the Source Text

• Valid and coherent outputs can still omit, distort, or oversimplify source details; source-grounded parsing, retrieval-based evidence matching, and fine-grained culinary NER check extracted facts against the recipe.These methods target invented, omitted, or misrepresented facts in multilingual, code-switched, vernacular, and transliterated content while reducing reliance on manual review.Source: §3 · p. 3

Cross-cutting errors and validation scope

• Errors can span categories through preparation-state confusion, transliteration ambiguity, culturally mismatched substitutions, and code-switched instructions, motivating multi-stage validation before knowledge-graph ingestion.The detection-resolution architecture and source-grounded verification are broadly reusable, while vocabularies, co-occurrence distributions, substitution rules, and cultural compatibility models require cuisine- and language-specific adaptation.Source: §3 · pp. 3–4

4.1. Multi-Stage Soundness Assessment Workflow

• The seven-stage workflow integrates extraction, preprocessing, iterative soundness assessment, post-processing, and ingestion into FKG.in.The pipeline combines rule-based, statistical, and learning-based validation to address structural, semantic, and source-fidelity issues before ingestion.Source: §4.1 · pp. 4–5
• Recipe-card parsers provide reference data, while GPT-3.5 Turbo extracts recipe details and latent culinary attributes from free text into a standardized JSON schema.Preprocessing normalizes names, quantities, units, and preparation states, handles vernacular and script variants, and resolves synonyms, substitutions, and duplicates.Source: §4.1 · pp. 4–5

4.2. Inconsistency Detection Agent

• The Inconsistency Detection Agent applies five checks to ingredients, while other recipe attributes receive schema validation and source-fidelity checks only.The workflow iterates corrections and re-evaluation until critical inconsistencies are resolved or confidence thresholds are met.Source: §4.1 · p. 5, §4.2 · p. 6
• Schema validation enforces essential fields and data types, and vocabulary checks compare ingredient names against over 23,000 canonical, synonymous, and vernacular terms.Vocabulary checks use near-exact RapidFuzz matching and corpus co-occurrences to flag improbable substitutions and insertions.Source: §4.2 · p. 6
• Ingredient quantity anomalies are tested against serving-normalized IQR bounds, while a Set Transformer flags recipe-relative semantic discordance using GTE-Large embeddings [26].For ingredient-unit pairs with fewer than three observations, bounds expand by 40%; the 12-layer, 16-head model uses 1024-dimensional embeddings and an adaptive IQR upper bound.Source: §4.2 · p. 6
• Dual-judge 435M-parameter DeBERTa-v3-large models [27] check extracted facts against source text, using non-overlapping ingredient-quantity pairs and mapping Neutral NLI predictions to Contradiction.A Staple Guard exempts ambient ingredients such as salt, water, and oil from fidelity rejections.Source: §4.2 · pp. 6–7

4.3. Inconsistency Resolution Agent

• Resolution progresses from rule-based repairs and canonicalization to patch-based semantic correction using similar human-validated cases and context-aware constraints.When patch similarity exceeds 0.85, Gemini 1.5 Flash adapts a validated correction to the recipe context; uncertain or incompatible cases may be flagged for review.Source: §4.3 · p. 7
• Source grounding rolls back unsupported extracted facts, while human adjudication handles ambiguous cases through an interface that logs decisions to refine correction resources.Post-processing consolidates canonical ingredients, generates embeddings, retains source-fidelity metadata, clusters near-duplicates, and aggregates co-occurrence data before validated records are ingested into FKG.in.Source: §4.1 · p. 5, §4.3 · pp. 7–8

5.1. Evaluation of the Inconsistency Detection Agent

• Structural violations affected 91.6% of documents, versus 48.0% for vocabulary enforcement, 36.6% for statistical grounding, 68.8% for semantic coherence, and 96.4% for source fidelity.These results show that errors would largely remain undetected through schema validation alone; source-fidelity checks found contradictions even in structurally valid and semantically plausible extractions.Source: §5.1 · p. 9, Table 2 · p. 9
• Quantitative hallucinations comprised 49.0% of source-fidelity errors (1,201 instances), existential hallucinations 29.7% (728), and morphological contradictions 4.0% (100).The findings indicate that many ingredient-level failures reflect semantic drift and unsupported inference during structured generation, rather than entity recognition alone.Source: §5.1 · p. 9

5.2. Evaluation of the Inconsistency Resolution Agent

• Retrieval covered 53.48% of unseen contradictions at cosine similarity ≥0.88, while conservative automated correction resolved 311 errors, or 20.8% of all unseen errors.The cross-recipe evaluation used 1,494 contradictions and 42 human-validated corrections; retrieval weighted the hallucinated entity at 80% and recipe context at 20%.Source: §5.2 · pp. 9–10

5.3. Ingredient NER Evaluation

• Ingredient NER achieved F1 = 84.2% on simple expressions and 75.0% on complex constructions, but scored 50.3% on longer compositional sentences and 65.1% on Hinglish or vernacular expressions.Evaluation suites were designed as diagnostic stress tests for realistic extraction conditions, including coordination, vernacular terminology, and complex ingredient descriptions.Source: §5.3 · p. 10

Limitations

• Evaluation beyond patch-based resolution remains ongoing; culturally nuanced concepts need human oversight, detection-resolution feedback is incomplete, and conservative thresholds limit autonomous handling of novel inconsistencies.Resolution uses similarity thresholds of ≥0.88 and strict error archetypes, prioritizing knowledge-graph integrity over autonomous handling without human-validated correction seeds.Source: §6 · pp. 10–11
• The framework focuses primarily on text-based recipes; video and image-centric sources will require more sophisticated alignment and grounding mechanisms.Source: §6 · p. 11

Future Directions

• Planned work will evaluate detection and resolution stage by stage on expanded curated benchmarks and strengthen their interaction through iterative feedback.The evaluation will assess structural validity, semantic coherence, source fidelity, and resolution quality.Source: §6 · p. 11
• Future work will extend multilingual coverage beyond English and Devanagari, improve retrieval-augmented reasoning, and expand culturally nuanced substitutions and preparation practices.These efforts aim to reduce manual review while preserving transparency, traceability, and culinary fidelity, and may inform soundness workflows for other domain-specific knowledge graphs.Source: §6 · p. 11

7. Conclusion

• Across 500 recipe documents, extraction failures spanned all validation dimensions, and many were undetectable through schema validation alone.The findings underscore the need for layered verification before recipe data enters a knowledge graph.Source: §7 · p. 11
• The semi-automated FKG.in workflow combines schema validation, statistical grounding, lexical and semantic checks, source-grounded verification, retrieval-assisted correction, and human review.It targets structural inconsistencies, numerical anomalies, semantic incoherence, and source-fidelity violations in LLM-extracted recipe data.Source: §7 · p. 11
• Although developed for Indian culinary data, the workflow offers a practical template for validating structured knowledge in other domains.It is intended for extracted information that must remain faithful, coherent, traceable, and auditable, while offering a path toward reducing manual curation effort.Source: §7 · p. 11

• The authors used ChatGPT (OpenAI) for language refinement and proofreading, reviewed all AI-assisted text, and take full responsibility for its content.Source: Declaration on Generative AI · p. 11

A. Inconsistency Detection and Resolution: Proof of Concept on a Sample Recipe

• The proof of concept walks through both agents using a representative dum aloo blog recipe and its LLM-generated structured JSON, manually annotated for representative failure modes.The recipe document was selected from the evaluation corpus, and the walkthrough assesses how the validation pipeline handles the annotated failures.Source: A. Inconsistency Detection and… · p. 13
• The dum aloo extraction contained five source-level ingredient errors spanning quantities and forms, plus an unsupported serving-size claim.Baby potatoes were reported as 12.5 rather than 13.5 and whole rather than boiled, peeled, and pricked; oil as 5 rather than 2.5; coriander seeds as “na 2” rather than crushed; and onion as 3 rather than 1. Serving size was reported as 6 despite being absent from the source.Source: A. Inconsistency Detection and… · p. 13
• Two inconsistent ingredients were added to the extracted JSON to further demonstrate the framework’s coherence capabilities.The passage introduces these additions as part of the stage-by-stage demonstration but does not report their resolution outcome.Source: A. Inconsistency Detection and… · p. 13

A.1. Inconsistency Detection Agent Walkthrough

• Detection validates schema fields, flags the dish name dum aloo as an ingredient, identifies quantity outliers, and assigns injected dum aloo and bread discard probabilities of 34% and 54%.The Set Transformer assigned valid ingredients probabilities near 0.1%; source-fidelity verification is the fifth detection stage.Source: §A.1 · p. 14
• Source-fidelity checks flagged the “whole” versus “boiled, peeled, and pricked” contradiction and an unsupported serving size of 6, while detecting quantitative drift in Errors 1, 3, and 5.The system also mapped extracted ingredient components against textual bounds.Source: §A.1 · p. 14

A.2. Inconsistency Resolution Agent Walkthrough

• The resolution agent processes flagged nodes through a tiered correction loop, activating only the stages relevant to the detected inconsistencies in this example.The walkthrough illustrates rule-based auto-correction, patch-based semantic correction, and source grounding with rollback.Source: §A.2 · p. 14
• Rule-based correction fills missing non-essential fields such as knife_cuts and ferment_time with the canonical na value, avoiding an LLM call.This schema-driven repair supports database interoperability.Source: §A.2 · p. 14
• For Error (2)’s morphological contradiction, KNN search across 42 human corrections retrieved a patch at cosine similarity ≥0.88, which Gemini 1.5 Flash applied to replace the default whole state with specified preparation modifiers.Source: §A.2 · p. 14
• The resolution loop autonomously repairs structural omissions, semantic contradictions, and unsupported serving-size claims, while escalating numerical errors that miss its similarity threshold to human review.Errors 1 and 3 are escalated when they fail the ≥0.88 similarity threshold for autonomous patching.Source: §A.2 · p. 15

B. Error Detection Performance (N=500)

• Across 500 recipes, Stages 1–4 detected structural violations, unfamiliar ingredient expressions, numerical outliers, and contextually anomalous ingredients.Examples included invalid data types in sparse arrays, vernacular variants and synonyms, and implausible weight-to-unit ratios.Source: B. Error Detection Performance (N=500) · p. 15, Table 4 · p. 15
• Stage 5 identified source-fidelity errors, including ingredient-level contradictions and metadata discrepancies between extracted information and source text.Table 5 summarizes the distribution of fidelity errors by category.Source: B. Error Detection Performance (N=500) · p. 15, Table 5 · p. 15