Source-linked AI summary
TSWAP: A Multilingual Retrieval-Augmented Thai Wellness Advisor
Pornthep Ukosaramig, Kobkrit Viriyayudhakorn
TL;DR
Thai wellness and traditional-medicine information lacked a trustworthy, verifiable, multilingual source and a public retrieval benchmark. TSWAP deploys an eight-language advisor using an unmodified multilingual LLM, retrieval grounding, forced retrieval for entity lookups, and rule-based safety. The released benchmark achieved Recall@5 = 0.88, while production QA recorded 91.1% test–retest pass over 259 cases and probing exposed distinct contributions of safety and knowledge grounding.
Problem
Thai wellness and traditional-medicine information is fragmented across channels, lacks a trustworthy multilingual source, and has no public Thai traditional-medicine or herbal benchmark.
Method
TSWAP deploys an eight-language conversational advisor that grounds an unmodified multilingual LLM in verified provider and Thai traditional-medicine data using retrieval, forced entity-lookup routing, and rule-based safety.
Results
Recall@5 = 0.88 on 50 benchmark questions, and 91.1% of 259 production QA cases reproduced a pass on test–retest evaluation.
Takeaways & Limitations
A grounding stack around an unmodified multilingual LLM can deliver a deployed wellness advisor in a culturally specific, low-resource domain without fine-tuning.
Takeaways & Limitations
Per-language quality depends on pretraining; safety lacks a trained classifier or interaction checker; retrieval is weakest for herbal content, and evaluation covers one deployment.
Abstract
from arXiv · showhide
We present TSWAP, a deployed eight-language conversational wellness advisor grounded, via retrieval-augmented generation, in a verified knowledge base of Thai traditional medicine and certified wellness providers. An unmodified open-weight LLM (Qwen3.6-35B-A3B on vLLM) is grounded on a ~30.6K-chunk Thai index by a hybrid dense-sparse retriever with cross-encoder reranking; a first-turn query classifier forces tool-based retrieval for entity lookups; a rule-based safety layer enforces medical scope and Thai emergency routing; and all eight languages are served zero-shot with translate-then-retrieve. We release the first Thai traditional-medicine/wellness retrieval benchmark (50 questions with gold document IDs; Recall@5 = 0.88), production QA logs (91.1% test-retest pass over 259 cases), and a 71-question frontier no-retrieval probe showing what each grounding pillar contributes: without the safety prompt the backend model family produced a full drug-dosing schedule and complied with out-of-scope requests, and without the knowledge base it produced zero verifiable provider recommendations. We further report two transferable deployment findings: English-calibrated 4-bit AWQ quantization corrupts Thai tone marks, and forced-retrieval routing is necessary for reliable grounding.
1 Introduction
TSWAP addresses fragmented, multilingual Thai wellness information by deploying a conversational advisor grounded in verified provider and traditional-medicine data. It fills an underdeveloped Thai NLP niche with a benchmarked, grounding-first system.
- Motivation: US$40.5 billion valued Thailand’s wellness economy in 2023, while provider information and traditional-medicine knowledge remained fragmented and difficult to verify across languages.Wellness-tourism spending was US$12.3 billion, and foreign visitors faced language barriers and uncertainty about provider standards.
- Contribution: TSWAP consolidates verified provider data and Thai traditional-medicine knowledge into a multilingual conversational advisor across web, mobile, and chatbot interfaces.The paper focuses on the platform’s conversational AI engine.
- Research gap: No public benchmark or dataset previously covered Thai traditional or herbal medicine, and only one prior Thai traditional-medicine language model existed.AppHerb generated treatment and recipe text from two textbooks but lacked deployment, multilingual support, retrieval grounding, and a reusable benchmark.
- Contribution: TSWAP released the first Thai traditional-medicine/wellness retrieval benchmark with 50 questions, gold document IDs, and Recall@5 = 0.88.The release also includes production QA logs and frontier-probe logs.
- System design: A query classifier forces retrieval for entity lookups, while the system uses prompt-and-routing safety rather than fine-tuning.The paper presents forced retrieval as necessary to prevent silent hallucination and reports serving findings involving Thai output and hidden-reasoning budgets.
2 Related Work
Prior Thai and Southeast-Asian language models and clinical NLP resources are mostly general-purpose or clinically focused, leaving Thai traditional and herbal wellness advisory underrepresented. TSWAP’s related-work position is shaped by analogues in Traditional Chinese Medicine and multilingual health-assistant evaluation.
- Thai and SEA LLMs: Thai and Southeast-Asian foundation models such as Typhoon, OpenThaiGPT, SEA-LION, Sailor, and Chinda are general-purpose rather than traditional-medicine focused.TSWAP uses an unmodified multilingual base model and invests in grounding rather than pretraining.
- Thai medical NLP: Thai clinical NLP includes Eir and medical-exam evaluations, but existing general Thai benchmarks target exams or broad domains.These resources do not evaluate open-ended, culturally specific herbal or wellness advisory quality.
- Traditional-medicine benchmarks: Traditional Chinese Medicine provides multiple benchmarking precedents that the paper treats as templates for a future Thai herbal benchmark.The cited analogues are MTCMB and TCM-Ladder.
- Multilingual health evaluation: Cross-lingual health-QA studies report substantial non-English quality degradation, while prior multilingual health assistants address other low-resource settings.These studies inform the paper’s evaluation protocol alongside faithfulness, relevance, context, and rubric-based safety measures.
3 The TSWAP Engine
TSWAP is a production conversational AI engine that combines a verified Thai wellness knowledge base, multilingual serving, hybrid retrieval, forced retrieval routing, and rule-based safety controls.
- System scope: TSWAP’s AI engine receives a data-minimized wellness profile and owns retrieval tools, multilingual behavior, and the safety prompt.The platform is available on web, mobile, and chatbot interfaces, while the paper focuses on its conversational AI engine.
- Knowledge base: The live Milvus index contains eight collections covering providers, venues, packages, catalogs, and Thai herbal medicine.As of 2026-07-15, it holds approximately 30,625 chunks; venue records are entity-level, while herbs and attractions are multi-chunk.
- Knowledge base: Herb monographs encode contraindications and scientific and research references, using DTAM-sourced materials with deterministic ingestion and change detection.The contraindications support vulnerable-group cautions, while reference fields are surfaced as citations.
- Retrieval pipeline: Hybrid retrieval fuses multilingual dense embeddings with BM25, reranks candidates with a cross-encoder, and returns a final top_k of 3.The system retrieves 10 candidates, reranks the top 7, and may skip reranking when top similarity reaches 0.85; structured and geospatial filters supplement semantic search.
- Grounding and multilingual behavior: A first-turn classifier routes intents and forces retrieval for wellness entity lookups, preventing the silent retrieval skips observed with automatic tool selection.The forced tool call applies to herbs, providers, spas, packages, hotels, restaurants, and places.
- Grounding and multilingual behavior: TSWAP serves eight languages zero-shot through translate-then-retrieve, while a rule-based safety layer restricts scope, avoids diagnosis and dosing, and routes emergencies.A Thai quality guard addresses corrupted output, but it is explicitly a quality filter rather than a safety filter; herb–drug interactions rely on deferral and contraindications rather than a dedicated checker.
4 Evaluation
The evaluation measures retrieval quality, production consistency, safety behavior, provider grounding, and deployment effects. Results show strong retrieval and test–retest performance, while failures concentrate in colloquial geospatial queries and serving configuration remains consequential.
- Production QA campaign: 91.1% of 259 round-one-passing cases reproduced pass in the second-round production QA campaign.The reexecution yielded 236 reproduced passes, 21 odd outcomes, and 2 failures, exposing sampling nondeterminism.
- Production QA campaign: 84% of the 56 problematic cases fell in colloquial geospatial search, with false proximity claims, unresolved landmarks and zones, and non-reproducible answers.End-to-end failures therefore concentrated where retrieval was structurally hardest.
- Frontier no-retrieval probe: 0/22 clear safety violations occurred with the safety prompt, compared with 3/22 without it in the frontier no-retrieval probe.Without the prompt, the model produced a complete paracetamol dosing schedule and complied with out-of-scope requests; with it, these requests were refused or redirected.
- Frontier no-retrieval probe: 0/12 no-retrieval answers provided verifiable provider recommendations for colloquial geospatial queries.Eight answers named national spa chains without location-specific verification, while four deferred users to external services.
- Serving findings: English-calibrated 4-bit AWQ corrupted Thai vowel and tone-mark generation, while classifier-forced retrieval restored reliable grounding after automatic tool selection skipped short-query retrieval.An initial run also showed that hidden-reasoning budgets could silently truncate answers.
5 Discussion and Limitations
TSWAP demonstrates a deployed grounding stack for culturally specific wellness advising without fine-tuning, while identifying important gaps in language coverage, safety checking, retrieval, reproducibility, and evaluation scope.
- Discussion: TSWAP delivers a deployed wellness advisor in a culturally specific, low-resource domain using an unmodified multilingual LLM without fine-tuning.The paper identifies future work in answer-quality benchmarking, engine-side ablations, FP8 migration, herbal retrieval, geospatial queries, and red-teaming.
- Limitations: Per-language quality is inherited from pretraining, the safety layer lacks a trained classifier and interaction checker, and evaluation covers a single deployment.The paper also reports that engine-side ablations and per-language answer-quality results do not yet exist.
- Limitations: Retrieval is weakest on the herbal collection, while end-to-end failures concentrate on colloquial geospatial queries and answers are not fully reproducible across runs.These limitations define concrete targets for retrieval improvements and structured red-teaming.
6 Ethics and Data Statement
TSWAP limits data sent to the LLM, frames responses as general wellness guidance rather than medical advice, routes emergencies to Thai hotlines, and reports consent and release boundaries.
- Data handling: Only data-minimized, wellness-relevant profile fields are forwarded to the LLM for PDPA compliance.The system does not forward direct identifiers.
- Safety scope: TSWAP is positioned as general wellness guidance rather than medical advice and routes emergencies to Thai hotlines 1669 and 1323.The statement defines the platform’s intended advisory and emergency-routing scope.
- User evaluation: Informed consent was obtained from all 120 UAT participants.The UAT participant count is reported alongside the consent statement.
- Data release: The paper releases the 50-question retrieval golden set, production QA logs, and frontier-probe logs, but not the herbal knowledge base itself.The release boundary distinguishes evaluation artifacts from the underlying herbal corpus.
7 Conclusion
TSWAP fills an unmet Thai NLP niche with a deployed, multilingual, retrieval-grounded traditional-medicine advisor and releases a benchmark, production logs, and transferable grounding and serving findings.
- Conclusion: TSWAP fills an empty Thai NLP niche because no prior deployed, multilingual, retrieval-grounded Thai traditional-medicine advisor or public Thai herbal benchmark existed.The paper positions the benchmark and logs as accompanying releases.
- Conclusion: The released contributions include the first Thai traditional-medicine/wellness retrieval benchmark with production evaluation logs.The paper also contributes grounding and serving findings intended to transfer to other tool-using RAG assistants in low-resource languages.