Source-linked AI summary
KinyaMed: Seeds, Not Rows -- What a Corpus Requirement Written in the Wrong Unit Fails to Constrain
Marius Bayizere
TL;DR
A row-count corpus requirement can be satisfied without securing independently authored clinical language, and the available evaluation set cannot support a verdict. The paper develops gates and checks while attempting a patient-voice Kinyarwanda urgency classifier, then reports the failures they expose. The result is an apparatus and negative findings, not a model that has passed its gate.
Problem
Rows can be generated from a small set of phrases, while independently authored patient-voice clinical sentences remain the constrained corpus quantity.
Source: §6.1 · p. 15Method
The paper builds a corpus gate checker, a power derivation for held-out sets, a deployment gate that refuses unsupported verdicts, and probes for model and serving failures.
Source: §9 · p. 26Results
At 165 seeds, no row count passes the binding seed gate; 17,942 evaluation rows from nine sentences support no verdict, and the model’s predicted urgency changes for 31.5% of inputs under capitalisation and 21.0% under a single typo.
Source: §9 · pp. 25–26Takeaways & Limitations
The contribution is an apparatus that refuses to report when it cannot measure, rather than a model result; no model has passed the gate.
Source: §9 · p. 26No clinician has signed off on the corpus, and the planned clinician session has not been held.
Source: §7.6 · p. 24
Abstract
from arXiv · showhide
Triage decides who is seen first. Building an urgency classifier for patient-voice Kinyarwanda, we found our specification could be met without producing anything it was meant to secure. We report that, and the instruments that detect it, instead of a classifier. Designed for the four languages a Rwandan health centre receives, with every instrument per-language: sentences are authored in all four arms and rows generate in one, because the frame slots those three need do not exist. Our requirement asked for one million examples; generation produced them in 130 seconds. It fails four of the nine quality gates in that specification, six when rows are attributed to their source sentence. The binding gate counts distinct authored seed phrases, not rows: at our 165, no corpus passes at any row count. Two shortfalls follow and differ: 2,835 further sentences to pass the seed-count gate, 19,835 to reach the stated million rows, because a separate gate caps a seed at 50 rows. Rows come from a machine at 7,700 per second; seeds from clinicians. A row count constrains the cheap quantity, leaves the expensive one free, and so does not constrain quality at all. Two further negatives follow. An evaluation set of 17,942 rows built from nine distinct sentences supports no verdict: our gate, which counts distinct sentences, refuses 38 of its cells and reports nothing. A model trained on a corpus of uniform surface form changes its predicted urgency for 31.5% of inputs under capitalisation and 21.0% under a single typo, reported as measurement and not attribution. A model directory shipped without its tokenizer loads without error and answers its class prior on input it cannot read, with well-formed probabilities. No figure here is evidence of model quality; the contribution is the apparatus and the negative results it produced, reproducible from a clean clone except where marked NOT REPRODUCIBLE.
1 Introduction
The paper reports negative results from building a patient-voice Kinyarwanda urgency classifier: a row-count requirement can be met without securing corpus quality, while the available evaluation set cannot support a verdict. It presents the gates and checks that expose these failures, alongside limitations on the project’s scope.
Language scope
- The design covers Kinyarwanda, English, French, and Kiswahili per language, but only Kinyarwanda rows exist because the other arms lack the needed frame slots.Sentences are authored in all four arms; rows are generated in one.Source: §1 · p. 1
Corpus requirement and seed count
- A corpus can meet a one-million-row requirement in 130 seconds yet fail four of nine quality gates, or six when rows are attributed to source sentences.The binding gate counts distinct authored seeds: with 165, no row count passes; 2,835 more seeds pass that gate, while reaching one million rows requires 19,835 more because each seed is capped at 50 rows.Source: §1 · p. 2
Language and authorship
- The first corpus, generated by a non-speaker from English concepts and machine translation, produced grammatical Kinyarwanda that did not sound like patient speech.The authors report that English syntax and clinical register carried over, and that authoring phrases rather than translating them was the remedy they found.Source: §1 · p. 2
Evaluation set
- 17,942 evaluation rows from nine distinct sentences support no verdict; the gate refuses 38 cells rather than reporting a number.Rows derived from a shared sentence are not independent for a gate that counts distinct sentences.Source: §1 · p. 2
Acceptance gate
- A recall-only safety gate certified a configuration that nearly assigned the critical label to every urgent case, with precision below merging the classes.The gate had no term in which over-triage could be expressed.Source: §1 · p. 2
Scope and claims
- No clinician has validated labels, no second speaker has reviewed the corpus, and nothing has been used with patients.The trained model is an audit artefact, not a model comparison; the authors state they do not claim priority and did not run a systematic field search.Source: §1 · p. 2
Silent failure modes
- The tokenizer can be absent from a shipped model directory, while surface variation can change urgency predictions; output-side signals alone reveal neither failure.The checks used properties outside the model rather than probabilities, accuracy, or label order.Source: §1 · p. 3
Recorded but unread
- Thirteen recorded facts went unread until something unrelated prompted inspection, including a required gloss field left as a placeholder in 13 of 15 phrases underlying the figures.The authors distinguish these process failures from measurement failures.Source: §1 · p. 3
2 Related work
Related work sets the corpus constraint against existing Kinyarwanda resources and explains why observed-sign triage instruments are not valid comparators for patient-written text. It also treats the existence of a patient-voice triage reference standard as an open question.
2.1 Kinyarwanda in current African-language model work
- The cited African-language model work reports 481M raw Kinyarwanda tokens, upsampled to 1.07B, with 13M synthetic tokens; this paper’s constraint is independently authored clinical sentences.The passage contrasts general Kinyarwanda text scale with the need for patient-voice clinical material.Source: §2.1 · p. 3
2.2 Why we make no comparison with a triage instrument
- ETAT is defined around examining children in person, so comparing it with a system that receives unexamined patient text would compare incommensurable procedures.Its categories also differ from the critical/urgent/routine categories in the specification.Source: §2.2 · pp. 3–4
- No located instrument was validated for assigning urgency from a patient’s own written free text without examination or clinician follow-up.The authors leave open whether such an instrument exists and treat corpus urgency labels as unvalidated until a clinician rules otherwise.Source: §2.2 · p. 4
2.3 Clinical guidance consulted during construction
- Clinical guidance was consulted while concepts were written, but no guidance text was reproduced.The authors do not name or cite the documents because the assignments that would justify doing so have not been checked against the sources.Source: §2.3 · p. 4
2.4 Kinyarwanda language resources
- The authors assessed Kinyarwanda resources in three groups and used none of the news-domain resources described here for pre-training.They cite licensing concerns for MasakhaNER and an unstated KINNEWS licence as reasons for non-use.Source: §2.4 · pp. 4–5
2.5 Where this work sits
- Kinyarwanda text resources include news corpora and named-entity data, but the authors identify no prior patient-voice medical triage dataset in their audit.The clinical resource they found records community health worker questions and clinician answers, not patient statements to a health worker.Source: §2.4 · pp. 4–5, §2.5 · p. 5
3 Method
The method replaces a machine-generated, unverified corpus with phrase-level authoring and audits how language arms, taxonomy, model setup, and evaluation design constrain what the resulting corpus can support. It uses grouped splits and per-language instruments while documenting limits in clinical review and measurement.
3.1 Corpus v1, and why it was replaced
- The v1 Kinyarwanda phrases were grammatical but not what patients would say, and its family holdout was not a holdout according to its own manifest.A Kinyarwanda speaker identified English-calqued syntax and clinical register; rewriting phrases as full utterances moved grammatical person into the phrase.Source: §3.1 · p. 5
3.2 Corpus v2
- v2 contains 330,000 Kinyarwanda rows generated from 165 distinct phrases, while a frame-permutation example shows 446 rows can represent one observation.The v1 and v2 results are not the same experiment and should not be combined in one table.Source: §3.2 · pp. 5–6
3.3 Four language arms, one corpus
- All four language arms contain authored sentences, but only Kinyarwanda has all four frame slots needed to generate rows.The other three arms have authored phrases but lack openers, contexts, and closers; each therefore generates zero rows.Source: §3.3 · pp. 6–7, Table 2 · p. 7
- Cue-list duplication produced exact sentence duplicates in English, French, and Kiswahili, while some distinct cues also collapse when the target language cannot express their distinctions.The three arms share 100 duplicate cues; separately, Kiswahili renders some gender- or age-distinguished cues identically.Source: §3.3 · p. 7
- Independent English and French reviews found the same reasoning for six held concepts because the blocks concerned Kinyarwanda words, not the underlying concepts.A seventh ruling was mirrored rather than reached independently, and another shared hold was also propagated from English.Source: §3.3 · p. 8
3.4 Concept taxonomy and clinical anchoring
- The taxonomy’s concept total and clinical anchoring remain unverified, and no clinician or second annotator has reviewed the urgency labels.The paper reports anchor-record counts rather than a concept total; no inter-rater agreement statistic exists.Source: §3.4 · p. 9
3.5 Model, training configuration and environment
- The reported model configuration freezes the embedding module and eight lower transformer layers, while its 96-token limit can truncate inputs at an unmeasured rate.The reported checkpoint was selected on stopping-set loss; the truncation rate is not yet measured.Source: §3.5 · pp. 9–10
3.6 Why the evaluation set must be the size it is
- The exact-binomial sample-size minima assume independent items, but generated rows share seed phrases, making the reported minima optimistic floors.No intra-cluster correlation coefficient was estimated, so no design effect was applied.Source: §3.6 · p. 10
4 Results
The results show that row-count requirements can be met while corpus gates fail, and that the available evaluation set cannot support a verdict. Two further audits find tokenizer and surface-variation failures; none of the reported model numbers is offered as evidence of model quality.
4.2 Cluster structure, not row count, decides what a held-out set can show
- 17,942 evaluation rows from nine distinct sentences leave the gate unable to report a verdict, so it refuses 38 cells rather than computing metrics.The required sample sizes are 710 sentences for overall accuracy, 365 gold critical items per pure language for critical recall, and 720 for the critical-to-routine rate.Source: §4.2 · pp. 11–12
4.1 A corpus requirement stated in rows does not constrain quality
- Four of nine corpus-quality gates fail on the generated corpus, and six fail when rows are attributed to source sentences; its 165 seeds cannot pass the binding seed gate at any row count.The specification requires 3,000 distinct seeds per language and caps each seed at 50 rows; the million-row target took 130 seconds to generate.Source: §4.1 · p. 11, Figure 1 · p. 11, Table 3 · p. 12
4.3 Two failure modes that leave every output-side signal healthy
- Without tokenizer files, a model directory still loads, maps every word to unknown, and returns its class prior with well-formed probabilities.The probe now checks the loaded tokenizer’s vocabulary size and unknown-token rate.Source: §4.3 · p. 13
- 31.5% of sampled inputs change predicted urgency under capitalisation and 21.0% under a single realistic typo, reported as measurements rather than attributed effects.The probe used 200 inputs from nine source sentences and reported no interval because nine clusters cannot support one; the authors do not claim the corpus caused the flips.Source: §4.3 · p. 13
4.4 What we do not claim
- No figure is evidence of model quality, and the existing trained artefact is for auditing the serving path and gate refusal, not comparison or retraining.The gate refuses before reading the training corpus because no evaluation set meets the specification.Source: §4.4 · p. 13
5 The apparatus
The apparatus is designed to fail closed when a model is absent, unclassifiable, or unsuitable, while the proposed invariance check remains unimplemented and unevaluated. The system has not been used with patients or shown to meet its latency target.
5.1 Safety properties of the apparatus
- When the model is absent, cannot classify, or times out, the triage endpoint returns a manual-triage instruction and writes no report, urgency, or queue entry.The earlier keyword fallback was removed after it sent most critical cases to the routine queue; the supporting figures are marked NOT REPRODUCIBLE.Source: §5.1 · p. 14
- Unclassifiable cases sort above urgent and routine, patient-facing messages always include escalation instructions, and the rules layer can only escalate but currently has an empty term table.Startup also refuses configuration records it cannot apply exactly, including mismatched label order or configured sequence length.Source: §5.1 · p. 14
5.2 An invariance check the fragility finding implies
- The proposed invariance check compares model predictions on original and perturbed copies, serving only agreements and routing disagreements to human review; it has not been implemented or evaluated.It would require two forward passes, roughly doubling latency and halving throughput, and could still pass a stable but wrong prediction or a shared missing-tokenizer failure.Source: §5.2 · pp. 14–15
5.3 What the apparatus has not been shown to do
- The apparatus has not been used with patients, and its latency target is unchecked; on the measured machine, meeting the specified figure would require roughly 26 times its throughput.None of the six named ordered language pairs currently generates a code-switched row because the inventory is too thin for the claimed mixed-language output.Source: §5.3 · p. 15
6 Discussion
The discussion argues that requirements and evaluation must count scarce independent items rather than cheap generated rows. It also explains when the seed-count argument may not transfer and distinguishes the evidential status of the replacement acceptance-gate conditions.
6.1 Why a row count cannot constrain a corpus
- A row target can be met through cheap expansion without constraining the scarce authored seeds; the proposed remedy is to set a floor on distinct authored items per language and cell.The argument applies when frames permute invariant seeds, but may not transfer when frames add genuine linguistic variation or surface form itself carries label information.Source: §6.1 · pp. 15–16
6.2 Why cluster structure, not row count, governs an evaluation set
- 17,942 rows from nine sentences provide only nine clusters’ worth of evaluation evidence, so the gate resamples distinct sentences and refuses cells below their required counts.Row resampling answers how estimates vary across more permutations of those sentences, while sentence resampling addresses variation across different sentences.Source: §6.2 · p. 16
6.3 Why a single-metric gate fails, and what replaced ours
- The replacement gate requires all three conditions: critical recall ≥0.95, macro F1 > 2/3, and critical precision at least the merge-strategy precision plus a margin.The recall threshold is inherited and unverified, the strict macro-F1 floor is derived exactly but permits a nearly dead class, and the margin is not derived.Source: §6.3 · pp. 16–17
nCRIT nCRIT +nURG
A single-recall safety gate accepted a model that abandoned an urgency class, while rejecting a configuration with all classes active. The section also documents recorded-but-unread facts and silent serving failures that required checks beyond model outputs to detect.
nCRIT nCRIT +nURG
- 0.9749 CRITICAL recall passed the original gate while URGENT recall was 0.0083, because the single-metric condition could not express over-triage.The model assigned the CRITICAL label to 7,633 of 7,793 URGENT rows.Source: nCRIT nCRIT +nURG · p. 17
- 0.5337 CRITICAL precision sat 0.0012 below the set’s 0.5349 merge-strategy precision, while a configuration with macro F1 0.7724 and all three classes alive failed the gate.The merge-strategy precision is a class prior computed from gold support counts, not a model output.Source: nCRIT nCRIT +nURG · pp. 17–18
- The leakage guarantee rests on authoring metadata because phrase-group closure from strings alone was insufficient.Its supporting figure is marked NOT REPRODUCIBLE.Source: nCRIT nCRIT +nURG · p. 18
6.4 Recorded but unread
- Thirteen recorded facts went unread until unrelated changes prompted attention, including a gate that read no column and a checker that passed a table it could not parse.The authors argue that verification needs a third outcome for unparsed inputs and should treat them as failure.Source: §6.4 · pp. 18–20
- A stable SHA-256 digest confirmed that repeated archives were identical, not that they were complete; the bibliography was absent from two shipped archives.The digest was recomputed on each rebuild, reproducing the same absence.Source: §6.4 · p. 19
- The project’s own checks missed recurring mismatches between recorded facts and what consumers read, and naming the pattern did not prevent another instance.The authors identify tests that fail when recorded and consumed values disagree as the reliable automatic check.Source: §6.4 · pp. 20–21
6.5 Why the two silent failure modes matter to anyone deploying a low-resource clinical model
- A missing tokenizer can leave a model loading normally and returning its class prior on unreadable input, with well-formed probabilities and deterministic predictions.The authors recommend checking vocabulary size and unknown-token rate on a fixed probe string at start-up.Source: §6.5 · p. 21
- A model trained on uniform surface forms changed its answer under surface perturbations, but the measured flip rates do not identify whether the corpus, tokenizer, or decision-boundary proximity explains them.The authors say that the evaluated corpus was equally uniform and could not reveal this fragility.Source: §6.5 · p. 21
7 Limitations
The paper’s limits concern the evaluation’s small number of independent sentences, unvalidated labels, and narrow project scope. Its gates and proposed seed-count generalization are also not validated, and several planned studies remain incomplete.
7.1 The three that bound everything else
- Nine distinct sentences underlie 17,942 evaluation rows, so the gate refuses 38 cells and reports no verdict on model quality.The paper states that urgency labels are unvalidated and no instrument establishes ground truth for urgency assigned from unexamined patients’ text.Source: §7.1 · pp. 21–22, §7.2 · p. 22
- One author, no clinical collaborator, and one language bound the work; no clinician reviewed the taxonomy, corpus, or outputs, and no second speaker reviewed the Kinyarwanda phrases.Inter-rater agreement is unmeasured.Source: §7.1 · p. 22
7.2 How many distinct sentences each figure rests on
- A single sentence can shift CRITICAL and URGENT recall by about 25 percentage points, making the nine-sentence evaluation too coarse for a class-level verdict.The configuration search was inconclusive on the CRITICAL/URGENT boundary.Source: §7.2 · p. 22
7.3 The paper’s critique applies to the paper’s own specification
- The specification’s numerical requirements lacked stated sources, derivations, or checks of measurability, including its one-million-example target.The authors count six instances across five bullets, including two latency requirements.Source: §7.3 · p. 22
7.4 The gates are engineering defaults, not validated instruments
- Most corpus-gate thresholds are engineering defaults rather than validated instruments, and some remain placeholders pending a real pilot.The authors say the structural finding that authored seeds bind remains regardless of threshold choice.Source: §7.4 · p. 23
7.5 One generator, one language, one project
- The seed-count argument is demonstrated on one generator, one language, and one codebase; its proposed sweep was not completed and yielded no measurement.The attempt stopped at step 600 of 940 after a machine restart, and the harness could not resume within an arm.Source: §7.5 · p. 23
- Some Phase 0 accuracy, recall, confidence-interval, calibration, and latency figures are marked NOT REPRODUCIBLE because their scripts were never committed and no longer exist.The authors replaced figures needed for arguments with repository-derived quantities where possible.Source: §7.5 · p. 24
7.6 Further limitations
- The corpus’s phrase provenance is mixed, a second-speaker review has not been run, and clinical questions and thresholds still await clinician judgment.Roughly half the phrases were authored directly and the remainder were machine drafts under the speaker’s approval; the paper gives no reproducible percentage.Source: §7.6 · p. 24
8 What the apparatus makes possible, and what it cannot settle
The paper presents an apparatus whose use is constrained by the absence of authored seeds and clinical validation. It identifies what the instruments could support once those inputs exist, while leaving ground truth, diversity measurement, and calibration questions unresolved.
8 What the apparatus makes possible, and what it cannot settle
- The apparatus is built, but its first real uses are blocked by the scarce inputs the project lacks.The paper says this section is not a dated plan and does not assert that the system will work.Source: §8 · p. 24, §8.1 · p. 24
8.1 What becomes possible once seeds exist
- Corpus gates could check returned authoring sheets as they are produced, making corpus properties visible while there is still time to address them.The authoring sheet includes seed, origin, author, validation, reporter, age-group, and surface-variation fields.Source: §8.1 · p. 24
- The power derivation could become an authoring quota for the distinct sentences needed per language and class before a verdict is possible.The paper describes these counts as the basis it knows for costing the work.Source: §8.1 · pp. 24–25
- A comparison between surface-invariance rates for models trained with and without surface variation would test whether authored diversity had its intended effect.The paper describes this as a baseline comparison, not an established result.Source: §8.1 · p. 25
8.2 Open questions we cannot answer from this project
- Whether urgency labels from unexamined patients’ text have defensible ground truth remains a clinical and methodological question, not an engineering one.The gap between an annotator’s label and what a patient needed is unmeasured.Source: §8.2 · p. 25
- Distinct authored seeds are a proxy for effective diversity: different seeds may describe the same presentation, and the chosen lexical-diversity floor cannot yet be calibrated.The authors leave open whether any single number can represent diversity in this kind of corpus.Source: §8.2 · p. 25
- The probed artefact never exceeded 0.55 confidence on the critical class, so the stated referral threshold would refer every case and calibration would be practically inert.The paper leaves open whether calibration should be reported in this regime and what review threshold would be useful.Source: §8.2 · p. 25
The diagnostic that costs one inference pass
- The next diagnostic is per-sentence and requires one inference pass rather than another training run.Source: The diagnostic that costs one… · p. 25
9 Conclusion
The conclusion presents the work as an apparatus and negative results rather than a classifier. It argues that row counts cannot substitute for authored seeds and identifies the additional authored data and clinical input still needed.
9 Conclusion
- A row-count requirement can be met quickly without satisfying the seed-count gate that determines whether any corpus passes.The one-million-example target took 130 seconds; with 165 seeds, no row count passes every gate.Source: §9 · p. 25
- 2,835 further authored sentences are needed to pass the seed-count gate, while 19,835 are needed to reach one million rows under the separate 50-rows-per-seed cap.Source: §9 · p. 26
- 17,942 evaluation rows built from nine distinct sentences support no verdict, and the gate refuses rather than reporting a number.Source: §9 · p. 26
- The work provides a generator, corpus checker, power derivation, refusal gates, malfunction probe, training pipeline, and fail-closed service, but no model has passed the gate.The trained model is an audit artefact, not a baseline.Source: §9 · p. 26
- Further authored sentences, a clinical lead to define and ratify categories, and a sufficiently large distinct-sentence evaluation set remain necessary for a verdict.The paper says its honest output until then is the apparatus and the negative results it produced.Source: §9 · p. 26
- The paper says its numbers re-derive from a clean repository clone except those marked NOT REPRODUCIBLE.Source: §9 · p. 26
Ethics statement
No patient data was used, and the system has not been used with patients or in a health centre. The paper distinguishes this work from future human-participant studies, which require prior review and approvals.
- No real, identifiable, or de-identified patient record was collected, processed, or stored; every corpus row is synthetic.Rows were generated from speaker-authored seed phrases with no provenance in a clinical encounter.Source: Ethics statement · p. 26
- No ethics review was sought because the work involved no human-subjects data; this does not clear prospective studies.A nurse-baseline study, clinician review, community health worker consultation, or health-centre deployment requires approval before it begins.Source: Ethics statement · p. 26
- The system has never been used with a patient or in a health centre, is not cleared for clinical use, and fails its acceptance gate.Source: Ethics statement · p. 26
Data statement
The corpus is synthetic, generated from speaker-authored seeds, and its size follows from that inventory rather than a target. Its released materials exclude trained weights and licensed clinical source documents.
- The corpus is a synthetic multilingual urgency-classification dataset generated from a small inventory of speaker-authored seed phrases.Source: Data statement · p. 26
- Its size follows from the seed inventory rather than a target, making row count an inadequate unit for corpus requirements.Source: Data statement · p. 27
- The generator, split manifests, evaluation apparatus, and number-producing emitters re-derive from a clean clone, but trained weights are withheld because the model fails its acceptance gate.Source: Data statement · p. 27
- Clinical source documents, including WHO ETAT, are cited but not redistributed; an ETAT participant-manual blob remains reachable in earlier repository history after its removal from the working tree.The paper records the history and the decision not to rewrite published history.Source: Data statement · p. 27
- Every phrase has per-row provenance, and no Kinyarwanda text was machine-translated from English; the other three language arms are machine-reviewed but not native-speaker-reviewed.Source: Data statement · p. 27
Appendix A: Person rulings, concept collapse and relation restrictions
The appendices document how person, concept, and relation rulings shape the authored inventory, then explain why the generator emits no code-switched rows. The inventory reconciliation distinguishes unwritten phrases from rows excluded by rulings.
Appendix A: Person rulings, concept collapse and relation restrictions
- The authoring record cannot be converted into a concept total by dividing rows by two, because the repository contains five conflicting concept counts.Source: Appendix A · p. 28
- 15 concepts were collapsed into another across 30 rows, but only when the absorbing phrase carried the whole sign; two proposed collapses failed that test.Source: Appendix A · p. 28, Table 5 · p. 29
- First-person phrases were removed when the patient could not report a sign, when it was an examination finding, or when a paediatric phrase merely duplicated an adult concept.Third-person phrases were also removed for concepts not plausibly presented by another person; relation options were ruled per concept rather than defaulted.Source: Appendix A · p. 28
- 19 applicable, unheld rows have no phrase yet, while one row contributes a second phrasing, so row totals and phrase totals reconcile differently.The earlier attribution of the discrepancy to the overlap between flagged and held rows was incorrect; the counts for inapplicable and held rows are disjoint.Source: Appendix A · p. 28
Appendix B: Code-switching, pair by pair
- None of the six ordered language pairs generates a row because the available inventory is too thin and some candidate terms are borrowed rather than switched.A speaker reviewed fifteen terms; five were established Kinyarwanda borrowings for monolingual rows and four were cleanly switched.Source: Appendix B · p. 29
- Mixed-language generation inserts one content morpheme into an intact matrix utterance and requires language-specific noun-class integration, so the generator declines pairs it cannot integrate.Five terms had both switched and borrowed dispositions, and two had competing candidate noun classes.Source: Appendix B · p. 29
Appendix C: Leakage: closure, and the declaration that replaced it
Text-based closure misses leakage between first- and third-person phrasings of the same concept, so the split uses authoring metadata to group them. The paper also records how unchecked pipeline contracts let leakage and generated tables go unnoticed.
Appendix C: Leakage: closure, and the declaration that replaced it
- 60 of 61 concepts with both persons authored had their phrasings in separate phrase groups, so a holdout could place the same presentation in training and evaluation.The count is marked NOT REPRODUCIBLE because no committed script recomputes it; the leakage mechanism does not depend on the exact count.Source: Appendix C · p. 29
- Phrase-string closure cannot catch the person-pair leakage: first-person phrases begin with a letter, third-person phrases with a relation placeholder, and their verb morphology differs.The exceptional mid-phrase placeholder case shares 42 characters, but does not resolve the structural problem.Source: Appendix C · pp. 29–30
- The splitter now unions every phrase of a concept using the authoring brief’s mapping, rejecting declarations for phrases absent from the inventory.The guarantee depends on speaker metadata; unioning concepts reduces the number of independent evaluation units.Source: Appendix C · p. 30
- A flat phrase list would make text-only closure report zero overlap while splitting 60 of 61 concepts across the holdout.The paper says this metadata-dependent guarantee cannot be recovered by analyzing corpus text alone.Source: Appendix C · pp. 29–30
- The split manifest recorded complete leakage in a machine-readable field that nobody read, while the two fields being checked both showed zero.The paper identifies the table/emitter mismatch as another instance of a contract that remained unchecked.Source: Appendix C · p. 30
- Generated evaluation tables were omitted from the document until 2026-09-18 even though the emitter kept writing them, because the build included a nonexistent table-input path without raising an error.Source: Appendix C · p. 30
Appendix D: The v1 family holdout, in full
Appendix D documents that v1’s family holdout evaluated phrases already present in training, despite zero exact-text and family overlap. The appendix also reports an inconclusive three-configuration search, the reported run’s class-boundary errors, and the unfinished language arms.
Appendix D: The v1 family holdout, in full
- 114,321 of 114,321 v1 family-evaluation rows used phrases also present in training, so the family split was not a phrase holdout.Exact-text overlap and family overlap were both zero; the phrase, which carried the label, was the relevant unit.Source: Appendix D · pp. 30–31
- 89.2% of family-evaluation rows appeared verbatim in phrase-split training; v2 fixed within-split family leakage but had 100.0% cross-split overlap in both directions.The two v2 splits are disjoint partitions of one corpus, so comparing their results as different holdout difficulty levels would be misleading.Source: Appendix D · p. 31
Appendix E: The cost of the configuration search
- Three configurations were trained in under five hours, but the search was inconclusive: three runs do not characterize a response surface or resolve the CRITICAL/URGENT boundary.The search stopped by judgement rather than convergence.Source: Appendix E · p. 31
- The task was 150 distinct training phrases mapped to three labels; a small multilingual encoder had its embedding module frozen, leaving 3.7–10.8 million trainable parameters.The paper describes a few hundred optimization steps as appropriate to the task, rather than a hardware-imposed compromise.Source: Appendix E · p. 31
Appendix F: The reported run in full
- The run tables are evidence about the gate, not model quality; Table 8’s 17,942 rows come from nine distinct sentences, so its support is not an independent sample size.Table 9 gives the confusion matrix used to compute the per-class figures.Source: Appendix F · pp. 31–32
- All three configurations failed the acceptance gate, and the middle configuration had 0.0083 URGENT recall and the lowest macro F1.It had the highest CRITICAL recall of the three, so a gate using CRITICAL recall alone would have selected it.Source: Appendix F · p. 32
Appendix G: The other three language arms; G.1 As built
- For Swahili, a Kiswahili speaker authored relation wording, while English and French relation wording came from v1 machine drafts; set membership was mirrored from the Kinyarwanda speaker’s ruling.One Kinyarwanda relation has no Swahili counterpart, so its slot remains empty rather than using terms with a different referent.Source: §G.1 · p. 32
- No phrases are authored in the other three arms, their frame fragments are Kinyarwanda-only, and no rating session has measured phrase authenticity.Without those frame fragments, the other arms cannot generate rows regardless of their phrase inventory.Source: §G.1 · p. 32
G.2 As they stand for future work
The three non-Kinyarwanda arms contribute no rows: English and French are unverified machine-drafted review briefs, and Swahili is awaiting authored phrases. Missing frame slots prevent any of them from generating rows, while a proposed per-sentence diagnostic remains contingent on a sufficiently large set.
G.2 As they stand for future work
- The corpus is Kinyarwanda-only; English and French are unverified machine-drafted briefs, Swahili has no draft, and none contributes rows to reported results.No English or French speaker has reviewed the briefs.Source: §G.2 · p. 32
- Swahili authoring is to begin without a machine draft, following the Kinyarwanda finding that a fluent draft anchors speakers to its errors.The existing machine-translated Swahili corpus is withheld until authored phrases exist, so comparisons can be meaningful.Source: §G.2 · p. 32
- Table 8 reports performance on 17,942 frame-permutation rows from nine distinct phrases in five groups; the rows are not an independent sample size.Its final column gives the count that governs recall granularity.Source: Table 8 · p. 33
Appendix H: The per-sentence diagnostic
- Every trained configuration separated ROUTINE without error, but none resolved the CRITICAL/URGENT boundary; Table 9 shows the reporting-set confusion matrix.The proposed next step is a per-sentence diagnostic to distinguish ambiguous concepts, mislabels, and split artifacts; it awaits a sufficiently large set.Source: Table 9 · p. 33, Appendix H · p. 33
G.2 As they stand for future work
- The three other arms have no authored phrases, and Kinyarwanda-only frame fragments mean none can generate a row, whatever its phrase count.The missing frame slots, not the phrase inventory, are the binding constraint on a four-language corpus.Source: §G.2 · p. 33
Appendix I: Further limitations
The paper’s evidence is bounded by a Kinyarwanda-only corpus, an evaluation set too small for its gate, and unvalidated labels and phrases. No deployable model or baseline comparison is reported, and the system has not been used with patients.
Appendix I: Further limitations
- The corpus is Kinyarwanda-only, so the system has not been evaluated in its intended multilingual setting.English and French are unverified machine-drafted briefs, while Swahili is out for authoring.Source: Appendix I · p. 33
- The phrase and family holdouts are disjoint partitions of the same monolingual corpus, making cross-split evaluation 100.0% overlapping in both directions and leaving one honest evaluation per model.They are not two difficulty levels; only the within-split evaluation is a valid holdout.Source: Appendix I · p. 33
- No model meets the acceptance gate, and the pipeline refuses before training because the evaluation set is below its minimum; nothing is in use with patients.The existing artefact is not a deployable model.Source: Appendix I · p. 33
- No baseline comparison was run: multilingual BERT and AfriBERTa were scoped but not tested, so relative performance remains unanswered.The reported work covers one model family across three configurations.Source: Appendix I · p. 34
- No clinician reviewed the labels, and no second annotator exists to provide an agreement statistic.The paper reports no agreement value, rather than a low one.Source: Appendix I · p. 34
- Patient-facing text is authored or available only in Kinyarwanda and Swahili; English and French return an explicit pending state.The system declines to provide patient-facing text in a language no speaker has written for.Source: Appendix I · p. 34
- Intermittent-connectivity settings exceed the system’s current design: classification is server-side, disconnected submissions are lost, and a stale queue can appear current.The paper identifies capture-and-sync, server-side idempotency, and replay conflict handling as absent requirements for an offline mode.Source: Appendix I · p. 34
- The configuration search stopped after three runs by judgement, without convergence or the per-sentence diagnostic needed to investigate the unresolved boundary.The paper describes the search as inconclusive.Source: Appendix I · p. 34