Source-linked AI summary
Chinese-Jev: Bringing System One Model to Chinese-Language Tasks
Zexiao Wang, Zihao Zhang, Xudong Wang, Pan Wang, Ziyi Ye, Haoyu Zhao, Zuxuan Wu, Shuicheng Yan
TL;DR
Chinese-language decision tasks need an approach that supports varied bounded outputs without repeatedly relying on costly large-model calls. Chinese-Jev converts heterogeneous annotations into shared candidate-probability supervision and trains general and domain-specific models; it improves on Jev in general tasks and medicine while reducing hosted-API latency.
Problem
Bounded tasks such as routing, relevance judgments, and answer selection need labels or scores, while repeatedly invoking large models for these decisions can be costly.
Source: §1 · p. 1Method
Chinese-Jev converts heterogeneous Chinese annotations into candidate-option probability targets, then trains a general model and separately fine-tunes medical, legal, and financial specialists.
Source: §1 · p. 2Results
Chinese-Jev surpasses Jev on general tasks and in medicine, reaches 92% of Jev’s average specialized-domain accuracy, and measures 20.3×/17× lower hosted-API latency.
Source: §5 · p. 10Takeaways & Limitations
An INT8 browser deployment demonstrates local mobile inference at approximately 1 second per decision.
Source: §1 · p. 3
Abstract
from arXiv · showhide
System One models such as Jev offer an efficient alternative to generative language models for tasks that require decisions rather than open-ended responses. However, existing Jev models exhibit limited Chinese-language decision accuracy, restricting their utility in both general and specialized settings. In this paper, we introduce Chinese-Jev, a System One model that addresses this gap through a unified data processing and training pipeline. Our data processing protocol converts heterogeneous Chinese-language annotations into probability targets over candidate options, enabling a shared training formulation across domains and question formats. To enable efficient inference, Chinese-Jev adopts a lightweight encoder-only backbone for text encoding and learns to score candidate answers through decision-oriented training. To address the misalignment between the pre-training distribution and downstream Chinese-language scenarios, we first train the model on a general-purpose corpus of 10 million examples, then fine-tune it separately for the medical, legal, and financial domains. To evaluate decision accuracy and calibration in both general and domain-specific Chinese-language settings, we introduce Chinese-Jev Bench (CJ-Bench). After first-stage pre-training, Chinese-Jev exceeds the accuracy of the closed-source Jev model by 1.24% on general-domain tasks while achieving a 20.3x speedup. Subsequent domain-specific fine-tuning yields a 4.0% accuracy improvement over Jev in medicine and achieves 92% of Jev's average accuracy across specialized domains, with a 17x speedup and an average latency of only 15 ms per example. We further demonstrate on-device deployment of an INT8-quantized model on mobile devices, achieving an inference latency of approximately 1.0 second per decision. The project is available at https://gulucaptain.github.io/Chinese-Jev/.
1 Introduction
Chinese-Jev addresses Chinese-language decision tasks with shared candidate-probability supervision and general-to-domain training. CJ-Bench reports accuracy gains over Jev in general tasks and medicine, alongside lower latency.
1 Introduction
- The data pipeline converts heterogeneous Chinese annotations into candidate-option probability targets and yields a 10-million-decision corpus across eight task categories.It also produces dedicated medical, legal, and financial fine-tuning corpora and groups decisions sharing source material into the same data partition.Source: §1 · pp. 2–3
- A lightweight encoder-only model is pre-trained on general Chinese decisions, then independently fine-tuned for medicine, law, and finance without changing its architecture or decision interface.The shared interface supports option selection, proposition judgment, and ordinal rating in a single forward pass.Source: §1 · pp. 2–3
- 1.24% relative general-accuracy gain over Jev and 4.0% medical-accuracy gain accompany 92% of Jev’s specialized-domain average, with 20.3×/17× lower latency.The latency comparisons are relative to the hosted Jev API.Source: §1 · pp. 2–3
- ECE falls from 11.45% to 3.78% after general pre-training; CJ-Bench evaluates 307,900 held-out decisions for accuracy, calibration, and latency.The benchmark spans general, medical, legal, and financial tasks.Source: §1 · p. 2
- An INT8-quantized deployment enables local mobile inference at approximately 1 second per decision.This demonstrates on-device Chinese-language decision-making.Source: §1 · p. 3
2 Related Work
Related work covers generalist text classification, typed Jev decision models, and multitask supervision and Chinese resources. Chinese-Jev connects these areas through candidate scoring and converted probability targets.
2.1 Generalist Text Classification
- Generalist classifiers transfer across tasks using label descriptions, text–label compatibility, or premise–hypothesis formulations; GLiClass jointly encodes inputs and candidate labels in one pass (Stepanov et al., 2025).Chinese-Jev also jointly encodes the input and candidate answers, with separate readouts for choices, proposition judgments, and ordered ratings.Source: §2.1 · p. 3
2.2 Jev and Typed Decision Models
- Jev exposes choice, noul, and score probabilities, while open implementations use encoder representations or language-model decision heads; Chinese-Jev adapts an encoder separately to three domains.Chinese-oriented releases cited include a football-domain language-model system and a bilingual encoder for local decisions.Source: §2.2 · p. 3
2.3 Multitask Supervision and Chinese Resources
- Multitask instruction tuning combines tasks through shared formats, while Chinese resources span general understanding, knowledge, instruction following, and specialized domains; Chinese-Jev converts annotations into candidate-probability targets.The conversion preserves categorical labels, grade order, and human-rating distributions.Source: §2.3 · pp. 3–4
3 Methods
Chinese-Jev uses a shared decision format to turn varied Chinese-language annotations into training targets, then trains compact models for general and specialized tasks. Its pipeline combines source selection, annotation conversion, deduplication, and mixture construction with a common architecture and training objective.
3.1 Data Construction Pipeline
- The general training mixture spans eight task categories, with approximately balanced budgets across choice, noul, and score decisions.The pipeline selects mixtures after source discovery, annotation conversion, and deduplication.Source: Figure 2 · p. 4, §3.1 · p. 4
- Source selection retains Chinese datasets with clear usage terms and candidate-based or ordered-score annotations, and adds 10,700 rule-based decisions.Examples include C3, T2Ranking, ASAP, USTS, CMB, LeCaRDv2, and FinRE.Source: §3.1 · p. 4
- Annotations become context, instruction, decision type, candidates, and a target distribution; single-answer and categorical tasks assign all target mass to the annotated answer or class.Binary proposition labels map to false/true targets, while continuous ratings distribute mass between adjacent levels and average across raters.Source: §3.1 · pp. 4–5
- Multiple-answer questions become separate option-membership noul decisions, each retaining the full question and alternatives while predicting one option’s membership.Complete multi-label annotations support the same conversion.Source: §3.1 · p. 5
- Deduplication fingerprints decisions without their target distributions, makes choice fingerprints invariant to candidate permutations, and removes matching groups with conflicting targets.Decisions with agreeing targets are aligned by candidate text and reduced to one; related decisions from the same source material may remain distinct but share a data partition.Source: §3.1 · pp. 5–6
3.2 Chinese-Jev Training
- A lightweight encoder jointly represents the input and candidates, then a type-conditioned decision Transformer and shared scorer produce candidate probabilities in one forward pass.The general model is independently fine-tuned for medical, legal, and financial domains using the same architecture and objective.Source: §3.2 · p. 6, Figure 4 · p. 7
- Choice returns the highest-probability candidate, noul returns the true probability, and score returns an expected level value.The score readout uses a zero-based level index, with source-scale ratings using their numerical level values.Source: §3.2 · p. 7
- Training matches predicted and target candidate distributions with supervised cross-entropy and an RLCD policy-gradient term over perturbed candidate logits.RLCD rewards agreement with the target distribution and adds ranked probability score for ordered score decisions.Source: §3.2 · p. 7
4 Experiments
Chinese-Jev improves general and domain-specific decision accuracy while reducing measured latency, and an INT8 browser prototype demonstrates local smartphone inference. The experiments evaluate accuracy, calibration, and latency across general and specialized tasks.
4.1 Experimental Setup
- The evaluation uses CJ-Bench, shared eligible examples, and ACC, ECE, and median end-to-end latency; hosted Jev timing includes network round trips.Models are evaluated on the same retained context and candidates. Latency is measured at batch size one after warm-up; ECE uses 15 equal-width bins, and soft targets are excluded from ACC and ECE.Source: §4.1 · pp. 7–8
4.2 Experimental Results
- General pre-training reaches 69.20% accuracy and 3.78% ECE, exceeding Jev’s accuracy by 1.24% relatively; measured latency is 20.3× lower than Jev’s hosted API.Against Laya Multilingual, accuracy rises from 40.22% to 69.20% and ECE falls from 23.04% to 3.78%, while both models have 14 ms median latency.Source: §4.2 · p. 8, Table 1 · p. 8
- Domain fine-tuning raises average accuracy from 37.97% to 70.82%, though general pre-training alone scores below multilingual initialization on Finance.Relative to Chinese-Jev General, specialists gain 47.88, 21.34, and 29.33 percentage points on Medical, Legal, and Finance; both Legal and Finance specialists remain below Jev.Source: §4.2 · p. 8, Table 2 · p. 9
- Chinese-Jev exceeds Jev’s general-task accuracy by 0.85 percentage points and its Medical accuracy by 3.36 points; specialists reach 92.0% of Jev’s average accuracy at 15 ms per decision.The general result is a 1.24% relative accuracy improvement; Medical’s gain is 4.0% relative. Hosted-API latency comparisons include Jev’s network round-trip time.Source: §4.2 · pp. 8–9, Table 2 · p. 9
4.3 Application: Interactive Decisions on Mobile Devices
- The iPhone 15 Pro browser prototype scores editable candidate answers locally with an INT8 model in approximately one second per decision.Tokenization and inference run locally in the browser; subsequent queries are processed on-device without transmitting input text to a remote inference service.Source: §4.3 · p. 9, Figure 5 · p. 9
5 Conclusion
Chinese-Jev combines a unified data pipeline, a shared decision interface, and general pre-training followed by domain specialization. The paper reports accuracy gains over Jev in general tasks and medicine, but remaining accuracy and calibration gaps motivate further work.
5 Conclusion
- Chinese-Jev surpasses Jev on general tasks and in medicine, reaches 92% of Jev’s average specialized-domain accuracy, and reduces measured latency by about 20.3× and 17×, respectively.The hosted Jev API latency comparisons include network overhead; INT8 mobile inference takes approximately one second per decision.Source: §5 · p. 10
- The unified data construction pipeline supports large-scale Chinese pre-training and subsequent fine-tuning in medicine, law, and finance.CJ-Bench evaluates decision accuracy, calibration, and inference latency.Source: §5 · p. 10
- Remaining accuracy gaps in law and finance and uneven calibration gains after specialization motivate further work on domain adaptation and confidence calibration.The paper states that the models, training data, benchmark, and data construction code will be released to support reproducible research.Source: §5 · p. 10
A.1 Dataset Composition
The appendix describes the corpus partitions and benchmark composition, including how the General benchmark subset is selected and the input limits used across domains.
A.1 Dataset Composition
- Chinese-Jev Benchmark’s General component contains 100,000 decisions drawn from the full General test pool, and its partitions are counted in decisions.The general training corpus combines 20 public data sources and a programmatic rule supplement.Source: Table 3 · p. 15, §A.1 · p. 15, Table 4 · p. 16
- Full-input limits are 1,024 tokens for General and Finance, 2,048 for Medical, and 8,192 for Legal.Medical also has a 256-token decision-header limit.Source: §A.1 · p. 15
- The General benchmark subset balances choice, noul, and score budgets, stratifies within tasks, and caps each source at 15% where possible.Smaller tasks contribute all available examples, unused capacity is redistributed, and category shares reflect source-task counts and available test data rather than equal category quotas.Source: §A.1 · p. 16
- Table 5 compares task composition in the 100,000-decision General benchmark component and the 10 million-decision general training corpus.Its categories are listed alphabetically, and counts include hard and soft targets.Source: Table 5 · p. 16
A.2 Deduplication and Shared Material
The appendix specifies exact deduplication rules, special handling for candidate order, and split safeguards for shared source material and near-duplicates.
A.2 Deduplication and Shared Material
- General decisions are exactly deduplicated after task-specific conversion, with Unicode NFC and whitespace normalization while preserving numbers and negation.Matching inputs merge only when aligned targets agree; conflicting groups are discarded, and equality is checked within a task definition.Source: §A.2 · p. 16, Table 6 · p. 17
- Choice targets are matched as normalized candidate–probability pairs, so reordering alone does not create a label conflict.The general converter rejects unresolved position-dependent references and repeated candidate text; the medical pipeline instead uses order-preserving fingerprints.Source: §A.2 · p. 17
- Related examples derived from shared material remain in one partition, and official held-out material takes priority over training candidates.A separate screen excludes unresolved near-duplicates against indexed held-out text while preserving numerical and negation differences.Source: §A.2 · p. 17
- The near-duplicate screen covers only held-out text available when each candidate is checked; paraphrases across the full training corpus are not exhaustively merged.Source: §A.2 · p. 17
A.3 Rule-Based Synthetic Data
The project supplements public datasets with template- and rule-generated Chinese exercises, handling unknown facts differently across question types.
A.3 Rule-Based Synthetic Data
- 10,700 synthetic decisions (0.107% of the general corpus) cover table conditions and counts, plus rule-based grades.The total includes 10,526 table-condition and count decisions and 174 rule-based grade decisions.Source: §A.3 · p. 17
- Table exercises select among three to six plans using stated prices, distances, service availability, and selection conditions.The settings include accommodation, dining, and service packages; grading exercises use fictional registration or document-submission procedures.Source: §A.3 · p. 17
- When facts are unknown, score questions are retained only if the exact count or grade is determined, while choice questions may offer insufficient information.A noul question is retained only if its proposition has a determined truth value; whether a unique grade is determined can itself be asked as a noul question.Source: §A.3 · p. 17
- Figure 6 illustrates three decisions from one accommodation table and an unknown optional receipt that yields an insufficient-information choice, but no exact-grade score target.The grading rule is fictional; the noul judgment about whether the grade is uniquely determined is false.Source: §A.3 · p. 17, Figure 6 · p. 18
A.4 Domain Data Sources
The domain corpora combine converted public sources with selected examples and preserve separate evaluation partitions.
A.4 Domain Data Sources
- Medical training contains 3,000,000 decisions, while law and finance each contain 24,000, with 8,000 decisions of each type in each latter corpus.Medical data have a larger noul share because of matching and multiple-label tasks.Source: §A.4 · p. 18, Table 8 · p. 19
- Medical data draw on 30 source families spanning examinations, biomedical understanding, clinical text, and query matching.CMB and CMExam supply examination questions, while CBLUE contributes biomedical language-understanding tasks; multiple-answer examinations become option-membership judgments.Source: §A.4 · p. 18
- Medical selection limits complete inputs to 2,048 tokens and decision headers to 256, excluding tasks whose candidate sets exceed the header budget.The 13,281 medical score examples all use the QTR query–title relevance scale; the original test partition supplies 200,000 benchmark decisions, while validation and calibration selections come from disjoint validation groups.Source: §A.4 · p. 18
- Legal sources cover classification, claim support, case relevance, legal-element overlap, conflicts between provisions, and privacy-policy annotations.The resulting legal tasks use choice, noul, and score decisions.Source: §A.4 · p. 19
- Financial sources support examination questions, entity-relation labels, customer-service strategies, and relevance, answer-relevance, and readability judgments.These annotations yield choice and noul judgments, plus score targets that retain original quality scales.Source: §A.4 · p. 19
- Legal and financial supplements add decisions governed by fictional rules, including finance calculations for interest, fees, and repayment.Source licenses are retained, including noncommercial terms for ClaimGen-CN, AppealCase, and CFLUE.Source: §A.4 · p. 19
A.5 RLCD Objective
RLCD combines cross-entropy with a policy-gradient term that scores perturbed candidate distributions against the same targets.
A.5 RLCD Objective
- RLCD combines supervised cross-entropy with a policy-gradient term evaluating perturbed candidate distributions against the same targets.Source: §A.5 · p. 19
- RLCD draws G = 4 centered Gaussian perturbations per decision, adds them to detached logits, and converts each sample into a candidate distribution using one model forward pass.Source: §A.5 · p. 19
- Rewards combine log and spherical scores, adding an ordering-sensitive ranked probability score only for score decisions.The ranked term penalizes errors according to the ordering of levels, and the log score floors log probabilities at −9.21.Source: §A.5 · pp. 19–20
- The policy-gradient update uses rewards centered within each decision’s perturbation group and batch-standardized advantages, favoring above-average-reward perturbations.Source: §A.5 · p. 20