PaperlayerView exact revision on arXiv

Source-linked AI summary

A Conservative OCR-Enabled Workflow for R214 Sodium Screening of South African Packaged Foods

Mayimunah Nagayi, Alice Scaria Khan, Tamryn Frank, Rina Swart, Clement Nyirenda

arXiv:2609.15427v2cs.CVcs.AI

TL;DR

Screening package images against South Africa’s R214 sodium limits requires product identity, sodium evidence, reporting basis, and category-specific thresholds. This paper evaluates a conservative image-based workflow alongside an independent vision-language-model workflow and manual verification. The workflows agreed more often on category assignment and regulated status than on final screening outcome.

  • Problem

    R214 screening requires product categorisation, comparable sodium evidence, and reporting-basis checks, while package images make relevant evidence difficult to recover and link across views.

    Source: §I · p. 1

  • Method

    The paper combines package-region detection, OCR, evidence extraction, R214 category assignment, basis checks, and deterministic decisions, alongside independent Qwen2.5-VL 7B screening and manual verification.

    Source: §I · p. 2, §IV · p. 3

  • Results

    Category agreement was 415 of 442 products (93.9%) and regulated status agreement was 416 of 442 (94.1%), versus final decision group agreement of 307 of 442 (69.5%).

    Source: B. Workflow Outcomes and Automated… · p. 6

  • Takeaways & Limitations

    The workflow organises package evidence, identifies clear screening cases, and assigns uncertain products to REVIEW rather than forcing pass or fail decisions.

    Source: §VII · pp. 6–7

Abstract

from arXiv · show

Using food package images to monitor sodium and salt content against South Africa's R214 sodium limits is challenging when screening decisions require product identity, nutrition facts panel evidence, reporting basis, and category-specific thresholds. This study presents a conservative image-based workflow that combines region detection, optical character recognition (OCR), product identity and sodium evidence extraction, R214 category assignment, deterministic threshold comparison, and independent vision language model comparison. The evaluation used 442 packaged food products and 3 929 full package images from a real-world South African food packaging dataset. A YOLO26s small detector generated 4 195 region crops, and strict post-processing produced one sodium evidence row per product. The integrated workflow produced 290 OUTSIDE R214 SCOPE, 139 REVIEW, seven SCREEN-PASS, and six SCREEN-FAIL outcomes. The independent Qwen2.5-VL 7B vision language model workflow produced 387 OUTSIDE R214 SCOPE, 31 REVIEW, twenty SCREEN-PASS, and four SCREEN-FAIL outcomes. The workflows agreed on exact R214 category assignment for 415 of 442 products (93.9%) and on whether the assigned category was within R214 scope for 416 of 442 products (94.1%). Final screening outcome agreement was 307 out of 442 products, or 69.5%. Manual verification on 60 products showed lower strict outcome agreement than regulated status agreement, while all manual INSUFFICIENT DATA cases were kept out of SCREEN-PASS and SCREEN-FAIL by both automated workflows. The findings show that conservative image-based screening can organise package evidence, identify clear cases, and assign uncertain cases to REVIEW rather than forcing SCREEN-PASS or SCREEN-FAIL decisions.

I. INTRODUCTION

R214 screening from package images requires linking product identity and sodium evidence to a reporting basis and category-specific threshold, not merely extracting text. The paper proposes a conservative workflow and evaluates it alongside an independent vision-language-model workflow and manual checking.

I. INTRODUCTION

  • R214 sets category-specific maximum sodium values per 100 g, so screening must establish product type, category, sodium value, and reporting basis before comparing a threshold.The paper frames this as a structured evidence problem rather than sodium-value reading alone.Source: §I · p. 1
  • Package-image screening is difficult to scale because relevant evidence may span views and be affected by small text, glare, curved surfaces, dense tables, multilingual text, and occlusion.Prior monitoring approaches include manual, laboratory-based, and database-based assessment.Source: §I · p. 1
  • When identity, category, sodium or salt evidence, or reporting basis is unclear, the proposed conservative workflow assigns REVIEW instead of forcing SCREEN-PASS or SCREEN-FAIL.The workflow links package evidence extraction, category classification, basis checks, and deterministic decisions, with an independent Qwen2.5-VL 7B comparison.Source: §I · p. 2
  • The evaluation covers 442 real-world South African packaged food products and 3 929 package images, with evidence recovery, category assignment, screening outcomes, model comparison, and targeted manual checking.The dataset size was determined by three selected 2023 dataset folders.Source: §I · p. 2

II. RELATED WORK

South African R214 monitoring relies on category-appropriate, comparable sodium evidence and expert checking. Existing monitoring and label-data systems provide relevant evidence, but the reviewed approaches are mainly structured, manual, or laboratory-supported rather than automated from package images.

A. Sodium Regulation and Food Label Monitoring

  • R214 sets maximum total sodium values per 100 g for thirteen regulated processed-food categories, with limits applied according to product type.The relevant screening context is category-specific rather than based on sodium values alone.Source: A. Sodium Regulation and Food Label… · p. 2
  • Prior South African monitoring used processed-food data, label-declared values, nutrition panels, laboratory analysis, and regulated-food monitoring, with correct categorisation and expert checking needed.Comparable sodium values and interpretation of category, panel values, and measurement basis are also required.Source: A. Sodium Regulation and Food Label… · p. 2
  • Food databases, package-based systems, and package-photograph studies support nutrition and regulation monitoring, but reviewed approaches rely mainly on structured, manual, or laboratory-supported data.Source: A. Sodium Regulation and Food Label… · p. 2

B. Food Package Text Extraction and Product Evidence

Package-image text recovery and multimodal nutrition analysis address important parts of food-label processing, but the reviewed work does not combine them into a traceable R214 screening workflow. The paper identifies that gap as motivation for its workflow.

B. Food Package Text Extraction and Product Evidence

  • South African package OCR can recover useful label text, but dense panels, multilingual content, glare, curved surfaces, and varied layouts challenge recognition.Recovered text still needs linking to product identity, sodium or salt evidence, reporting basis, and the applicable R214 category.Source: B. Food Package Text Extraction and… · p. 2
  • Prior work studied bilingual nutrition extraction with large language models and fine-grained grocery recognition using product images and OCR text, without combining R214 category, basis, threshold, and conservative-review logic.Source: B. Food Package Text Extraction and… · p. 2

C. Multimodal Models and Related Work Synthesis

  • The reviewed multimodal nutrition work supports faster processing, but the paper identifies a gap in traceable workflows connecting real package images to category assignment, basis checks, category-specific thresholds, and conservative outcomes.The stated gap motivates the proposed workflow.Source: C. Multimodal Models and Related Work… · pp. 2–3

III. DATASET AND REGULATION CONTEXT

The study uses a selected multi-view dataset of South African packaged foods and the 2019 R214 limits, then applies integrated, independent-model, and manual-checking workflows. The package views supply evidence that may appear on different parts of a product.

A. Dataset Source and Package Views

  • The selected 2023 dataset folders contained 442 products and 3 929 images, with three to twenty-seven views per product and an average of 8.9 images.Folders were selected without formal random or category-based sampling, and all products and associated images in them were included.Source: A. Dataset Source and Package Views · p. 3

B. R214 Sodium Screening Context

  • The workflow uses 2019 R214 limits across thirteen official categories; operational code 14 denotes evidence outside those categories, not an official category or absence of sodium.Unclear identity or category evidence routes products to REVIEW rather than code 14.Source: B. R214 Sodium Screening Context · p. 3

IV. METHODOLOGY

  • The integrated workflow combines detection, crops, OCR, evidence construction, identity and sodium or salt extraction, category assignment, and deterministic threshold comparison.A separate Qwen2.5-VL 7B workflow screens full images, while manual verification checks agreement patterns and error cases on 60 products.Source: §IV · p. 3

A. Region Detection and Crop Generation

The workflow detects four package evidence regions, then uses OCR and region-specific scoring to construct product-level evidence, including a strict sodium evidence row.

A. Region Detection and Crop Generation

  • YOLO26s detected nutrition facts panels, ingredients, product names, and quantities in study images, with training images kept separate from the 442-product study set.The detector was trained on 176 products and 718 images, then applied to 3,929 study images.Source: A. Region Detection and Crop Generation · p. 3

B. Text Recognition and Evidence Construction

  • PaddleOCR outputs were cleaned and weak crop-level results retried with an enhanced crop, retaining the retry only when its candidate score improved.Crop outputs were grouped by product and region to build product-level evidence.Source: B. Text Recognition and Evidence… · p. 3
  • The overview links the integrated workflow's product-level outcomes with an independent comparison and manual verification of a 60-product subset.Manual verification checked agreement and error analysis.Source: Figure 1 · p. 4
  • Nutrition-panel candidates were scored using nutrition and sodium or salt cues, numeric content, per-100-g or per-100-ml cues, and table-like content.The strict evidence step selected one sodium evidence row per product.Source: B. Text Recognition and Evidence… · p. 4

C. Product Identity, Sodium Evidence, and Category Assignment

Product identity, sodium evidence, and category assignment are assembled using image-based extraction and conservative rules, with unclear cases routed for review.

Product identity

  • Qwen2.5-VL 7B extracted product identity from full package images to capture stylised names, front-of-pack text, visual context, and multiple package views.The highest-scoring image-level candidate was selected; low-scoring or incomplete outputs were flagged for review.Source: C. Product Identity, Sodium Evidence,… · p. 4

Sodium evidence

  • Automatic sodium evidence was limited to direct sodium in mg per 100 g or salt in g per 100 g with clear column order and no quality-assurance flag.Other conversions, unclear cases, and products without usable comparable values or a clear per-100-g basis were assigned to REVIEW.Source: C. Product Identity, Sodium Evidence,… · p. 4

Category assignment

  • R214 category assignment used strict rules first, then a temperature-0 Groq-assisted fallback for uncertain brand-specific or non-standard wording.Unclear category evidence went to category review rather than being forced into operational code 14.Source: C. Product Identity, Sodium Evidence,… · p. 4

D. Screening Outcomes, Decision Rules, and Evaluation Measures

The decision rules distinguish clear threshold comparisons from insufficient or out-of-scope evidence, while evaluation measures assess detector performance and workflow agreement.

Screening outcomes

  • REVIEW covered incomplete, unclear, conflicting, or unreliable evidence, including uncertain identity or category, weak extraction, unclear reporting basis, and noncomparable units.The four outcomes were for screening and prioritisation, not final legal determination.Source: D. Screening Outcomes, Decision Rules,… · p. 4
  • OUTSIDE R214 SCOPE meant available evidence did not support an applicable R214 sodium category, not that the product was outside all food regulations.Category review or sodium evidence that was not READY led to REVIEW.Source: D. Screening Outcomes, Decision Rules,… · pp. 4–5

Decision rules

  • A product with an applicable R214 category and READY sodium evidence was SCREEN-PASS at or below its 2019 limit and SCREEN-FAIL above it.Safety checks could retain provisional labels for audit while reporting the product as REVIEW when further checking was needed.Source: D. Screening Outcomes, Decision Rules,… · p. 5

Evaluation measures

  • Evaluation used precision, recall, and mAP for detector performance, and category, regulated-status, final-decision-group, and direct pass-versus-fail conflict measures for workflow agreement.Final-decision-group agreement compared the four reported screening outcomes directly.Source: D. Screening Outcomes, Decision Rules,… · p. 5

E. Independent Workflow and Manual Verification

The independent workflow processes full package images separately from integrated-workflow outputs, while manual verification checks selected products against package evidence.

E. Independent Workflow and Manual Verification

  • The independent Qwen2.5-VL 7B workflow used full package images only and did not use integrated-workflow crops, OCR text, selected sodium evidence, categories, or decisions.Python handled execution, structured-output validation, retries, and logging; pass or fail required internally valid, clearly comparable per-100-g evidence.Source: E. Independent Workflow and Manual… · p. 5
  • Manual verification checked a fixed, product-level subset of 60 products matched across the manual file and both automated workflow outputs.Selection focused on regulated or priority products, uncertain cases, and disagreements between workflows.Source: E. Independent Workflow and Manual… · p. 5

V. RESULTS AND EVALUATION

  • The results section evaluated detector performance with precision, recall, and mAP, and workflow agreement with product-level measures defined in the methodology.It reports processing outputs, screening outcomes, automated agreement, and manual verification results.Source: §V · p. 5

A. Processing and Evidence Recovery

The workflows showed stronger agreement on R214 category and regulated status than on final screening outcome. Processing produced one strict sodium evidence row per product, while many products still required review.

A. Processing and Evidence Recovery

  • Across 3 929 package images, the detector generated 4 195 region crops; product-level processing produced one strict sodium evidence row per product, with 207 READY and 235 assigned to REVIEW.Source: A. Processing and Evidence Recovery · p. 5
  • On the held-out test split, the detector achieved 0.897 precision, 0.877 recall, 0.887 mAP at 0.5, and 0.638 mAP at 0.5:0.95.Nutrition facts panel had the strongest class result at 0.995 mAP at 0.5; quantity was weakest at 0.778 mAP at 0.5.Source: Table I · p. 5, A. Processing and Evidence Recovery · p. 5

B. Workflow Outcomes and Automated Agreement

C. Manual Verification

Manual verification covered a targeted 60-product subset and showed higher agreement on regulated status than on exact outcome. Both automated workflows kept all manual INSUFFICIENT DATA cases out of SCREEN-PASS and SCREEN-FAIL.

  • Manual review assigned 26 products to SCREEN-PASS, 21 to SCREEN-FAIL, eight to INSUFFICIENT DATA, and five to OUTSIDE R214 SCOPE.Source: C. Manual Verification · p. 6
  • Both automated workflows kept all eight manual INSUFFICIENT DATA cases out of SCREEN-PASS and SCREEN-FAIL.Of the 13 pass-or-fail decisions retained by the integrated workflow after conservative review, 12 agreed with the manual screening direction.Source: C. Manual Verification · p. 6
  • Strict outcome agreement with manual review was 12/60 (20.0%) for the integrated workflow and 20/60 (33.3%) for the independent workflow; regulated status agreement was 55/60 (91.7%) and 47/60 (78.3%), respectively.Source: Table III · p. 6, C. Manual Verification · p. 6

VI. DISCUSSION

The discussion finds that broad R214 relevance was more stable across workflows than exact screening outcome, with uncertain products directed to review. The conclusion frames the workflow as a way to organise package evidence and prioritise manual checking, not to replace official assessment.

VI. DISCUSSION

  • Package evidence could span views and needed linking to the correct reporting basis and regulatory category, leaving uncertainty after recognition, parsing, and assignment.Source: §VI · p. 6
  • Broad R214 relevance was more stable than exact screening outcome; the integrated workflow assigned more products to REVIEW, and 12 of its 13 retained pass-or-fail decisions agreed with manual screening direction.Source: §VI · p. 6

VII. CONCLUSION

  • The workflow's practical role is to organise package-image evidence, identify clear screening cases, and prioritise uncertain products for manual review.Source: §VII · pp. 6–7
  • The workflow is not intended to replace human judgement or official compliance assessment; future work should expand manual verification and extend it to additional labelling regulations.Source: §VII · pp. 6–7
Loading 2609.15427v2…