PaperlayerView exact revision on arXiv

Source-linked AI summary

VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text

Trieu Hai Nguyen, Van-Dung Hoang

arXiv:2608.25478v1cs.CL

TL;DR

AI-generated and human-written Vietnamese text can be difficult to distinguish, while existing detection research has limited coverage of Vietnamese. VietAIDetector addresses this gap with an open-source, zero-shot tool built on Vietnamese-specific models and evaluated on out-of-domain data, where it achieves state-of-the-art performance according to previous research.

  • Problem

    Distinguishing AI-generated from human-written text remains challenging, and Vietnamese AI-text detection research is limited.

  • Method

    VietAIDetector uses a VietBinoculars-based zero-shot approach with Vietnamese-specific language models, configurable thresholds, and sliding-window processing for long documents.

  • Results

    VietAIDetector achieves state-of-the-art performance on out-of-domain Vietnamese datasets, as reported in previous research.

  • Takeaways & Limitations

    The open-source tool provides Vietnamese AI-text detection with support for scanned PDFs, long documents, chunk-level results, configurable thresholds, and downloadable PDF reports.

  • Takeaways & Limitations

    Performance depends on the underlying language models and may vary across domains, while optimal chunking remains an open research problem and diverse real-world evaluation is limited.

Abstract

from arXiv · show

In recent years, distinguishing between AI-generated text and human-written text has remained a challenge. In this paper, we introduce VietAIDetector, an open-source tool designed specifically for detecting Vietnamese AI-generated text. It allows users to interact through a Gradio web interface with inputs ranging from raw Vietnamese text to common text file formats, including scanned documents and exceptionally long texts that exceed the context size of the employed Large Language Models (LLMs). The core component of the tool employs a Zero-Shot approach to detect AI-generated text without requiring domain-specific training data, building upon the previous VietBinoculars and Binoculars research. The tool is built upon a Vietnamese-specific language model and has been evaluated on out-of-domain datasets, demonstrating superior performance compared to existing methods primarily developed for English. Additionally, users can select optimal detection thresholds based on F1 score, accuracy, or TPR@0.05FPR requirements. The results are presented through the web interface, allowing users to easily review and verify suspicious texts or download them as a PDF report. The tool is publicly available at https://github.com/trieuntu/VietAIDetector

1 Motivation and significance

VietAIDetector addresses the limited availability of Vietnamese AI-text detection by combining zero-shot detection with an accessible, feature-rich open-source tool.

  • Research on AI-generated text detection has focused mainly on widely spoken languages, while Vietnamese remains comparatively under-studied.
  • VietAIDetector is an open-source tool specifically designed to detect Vietnamese AI-generated text.
  • Its VietBinoculars-based Zero-Shot approach detects AI-generated text without domain-specific training data, reducing development time and cost.
  • The Gradio interface accepts raw text, common text files, scanned documents, and long texts exceeding employed LLM context sizes.
  • Users can select thresholds based on F1 score, accuracy, or TPR at 0.05 FPR to match specific detection requirements.

2 Software description

VietAIDetector computes Vietnamese-specific zero-shot detection scores and supports configurable decision thresholds, while reporting state-of-the-art performance on out-of-domain Vietnamese datasets.

  • PhoGPT-4B serves as the observer and PhoGPT-4B-Chat as the performer in the VietBinoculars detection technique.
  • The method computes log(perplexity), cross-perplexity, and their ratio before classifying text with a selected threshold.
  • TPR at 0.05 FPR is available as a threshold option to minimize false alarms in higher-education use cases.
  • VietAIDetector achieves state-of-the-art performance on out-of-domain Vietnamese datasets, according to previous research.

1 Part 1: Compute the VietBinoculars score

The VietBinoculars score computation uses paired observer and performer models with shared tokenization to produce a score for AI-versus-human classification.

  • The scoring function takes a raw string, observer model M1, performer model M2, and shared BPE tokenizer as inputs.
  • It outputs the VietBinoculars score BM1,M2(s) for the input string.
  • The observer model produces next-token prediction distributions Y for the tokenized input.
  • The performer model produces next-token prediction distributions Z for the same input.

11 Part 2: Classify the text as Human-written or AI-generated

VietAIDetector processes text through configurable ingestion, preprocessing, sliding-window chunking, VietBinoculars scoring, thresholding, and document-level aggregation. Its interface supports multiple input formats, long documents, reporting, reliability safeguards, and structured integration outputs.

  • Input and preprocessing: The classifier accepts raw text and file uploads, including .txt, .docx, native PDFs, and scanned PDFs routed through OCR.Extracted text is normalized before downstream detection.
  • Sliding-window chunking: Long documents are divided into overlapping token chunks using configurable window and stride settings, keeping each chunk within the LLM context limit.The method preserves contextual continuity rather than truncating the input.
  • Scoring: Each retained chunk receives a VietBinoculars score computed as the ratio of performer-model log perplexity to observer-performer cross-perplexity.PhoGPT-4B serves as observer and PhoGPT-4B-Chat as performer; inference uses gradient-free execution, bfloat16 precision, capped lengths, and fixed-size batching.
  • Thresholding: Users can select Youden’s J statistic, the Closest Point approach, or TPR@0.05FPR to set the chunk-level decision threshold.The selected threshold determines whether each chunk is labeled AI or Human.
  • Aggregation: Document-level classification aggregates retained chunk labels by majority voting and labels the document AI-generated when PAI > 50%.The percentage of AI-classified chunks is computed before applying the document-level rule.
  • Interface and reporting: The Gradio interface exposes uploads, thresholds, chunking parameters, progress feedback, PDF reports, and hierarchical JSON outputs for inspection and integration.Reliability controls include minimum-token constraints, extraction verification, and warnings for unsuitable inputs.

3 Illustrative examples

VietAIDetector supports raw-text and file-based detection, including scanned and long documents, with configurable processing parameters and benchmark reporting. Its interface provides document- and chunk-level results, while comparisons with GPTZero show comparable performance on newly generated out-of-domain datasets.

  • 3.1 Examples using text input: Users can analyze raw Vietnamese text and obtain examples from out-of-domain news and literary datasets.The interface starts detection through an Input Text field and an Analyze button.
  • 3.2 Detection results: The completed analysis displays a document-level prediction alongside chunk index, detection score, label, token count, and chunk content.Users can export the complete detection report as a PDF.
  • 3.3 File upload and configuration: File processing accepts text documents, native PDFs, and scanned PDFs through drag-and-drop or an upload area.Scanned PDFs are included among the example files supplied with the source code.
  • 3.3 File upload and configuration: The configuration interface exposes detection-threshold and chunking parameters, and these settings are included in downloaded PDF reports.This preserves the parameters used during each detection process for later review.
  • 3.3 Short benchmark: VietAIDetector achieved performance comparable to GPTZero on three newly generated out-of-domain news datasets.The datasets contain documents exceeding the detection models’ context window and were generated by GPT-5.6 Luna, Gemini 3.6 Flash, and Claude Sonnet 4.6.
  • 3.3 Short benchmark: On the Gemini 3.6 Flash dataset, VietAIDetector’s average AI score was 0.81 versus GPTZero’s 0.7, although GPTZero achieved higher accuracy.The difference reflects GPTZero’s additional decision conditions beyond an AI-probability threshold, whereas VietAIDetector uses the AI-score threshold.

4 Impact

VietAIDetector extends zero-shot Vietnamese AI-text detection into an open-source, practical document-analysis tool. Its configurable processing, long-document support, OCR integration, reporting, and open interfaces support research, education, and downstream applications.

  • Open-source practical impact: VietAIDetector integrates zero-shot Vietnamese AI-text detection with multi-format processing, an interactive interface, and downloadable PDF reports.It is presented as an open-source alternative to existing open-source and commercial tools.
  • Open-source practical impact: The tool lowers deployment barriers for Vietnamese AI-generated-text detection in research and educational settings.The paper specifically connects this integration with the VietBinoculars algorithm to practical software deployment.
  • Research directions: Configurable thresholds support comparative studies of threshold calibration across deployment scenarios.The threshold options include performance-oriented operating requirements described elsewhere in the paper.
  • Research directions: OCR based on Vintern-1B-v2 enables investigation of AI-text detection in OCR-degraded Vietnamese documents.The paper identifies this as a setting that has received little attention.
  • Research directions: Sliding-window chunking enables detection on documents exceeding employed language models’ context windows, while JSON output supports downstream NLP integration.Embedding configuration parameters in PDF reports also supports reproduction and verification of results.
  • User impact: Educators, fact-checkers, content reviewers, and the public can use the tool to screen content and generate structured reports.The paper identifies higher education and social-media content verification as primary user contexts.

5 Limitations and future work

The tool’s performance depends on its underlying language models and can vary across domains. Long-document parameter selection, OCR-degraded or complex documents, computational cost, evolving LLMs, and limited real-world evaluation remain important boundaries.

  • Limitations: Detection performance depends on the quality of the underlying language models and may vary across domains.This is stated as a central limitation of VietAIDetector.
  • Limitations: Selecting optimal sliding-window chunking parameters remains an open research problem despite enabling long-document processing.The limitation concerns parameter choice rather than the availability of long-document processing itself.
  • Limitations: OCR-degraded documents and documents containing tables, images, or multimedia content remain challenging.The paper groups these document conditions among areas requiring further improvement.
  • Limitations: Improving detection accuracy can require larger language models and therefore higher computational costs.The paper also notes that evolving LLMs require continuous adaptation of detection methods.
  • Limitations: The tool has not yet been extensively evaluated across diverse real-world deployment scenarios and is not a legally authoritative decision-making tool.Final authenticity decisions should be made by human evaluators.
  • Future work: Future work targets complex document structures, multiple forms of AI-generated content, and real-world evaluation through a planned university pilot.The pilot is intended to guide improvements in accuracy and adaptability.

6 Conclusions

VietAIDetector is an open-source Vietnamese AI-text detector built on VietBinoculars and Binoculars frameworks. It combines zero-shot detection with broad input support, configurable thresholds, chunk-level results, reports, modular processing, and JSON integration.

  • Conclusion: VietAIDetector detects Vietnamese AI-generated text without model retraining through a zero-shot approach based on VietBinoculars and Binoculars.The tool supports scanned PDFs and documents exceeding the employed language models’ context limits.
  • Conclusion: The tool accepts multiple input formats and provides chunk-level results, configurable thresholds, and downloadable PDF reports.These features are presented as part of the paper’s practical detection platform.
  • Conclusion: Its modular, configurable pipeline facilitates maintenance, future extensions, and downstream NLP integration through JSON output.The software is released under the MIT License and publicly available on GitHub.

Appendix A Updating Optimal thresholds for VietAIDetector

VietAIDetector’s decision thresholds were updated as of July 2026 using new AI-generated datasets from OpenAI and Google LLMs alongside human-written data. The updated thresholds are presented in Figure 6, including Youden, closest, and TPR@5%FPR thresholds.

  • July 2026 thresholds were derived from AI-generated datasets produced by OpenAI and Google LLMs together with human-written datasets from.
  • The updated thresholds are shown in Figure 6.
  • Figure 6 presents Youden and closest thresholds, plus a TPR@5%FPR threshold.

Appendix B Comprehensive benchmark results for gridsearch-optimized VietAIDetector

The appendix reports a grid search over chunking parameters for Vietnamese AI-generated datasets under a Youden threshold. The search varies chunk width and overlap and uses 20 samples per dataset.

  • The grid search covers chunk widths W ∈[200, 650] and overlaps O ∈ [50, 150].
  • The search uses a Youden threshold of t∗= 0.927966.
  • Each dataset contributes N = 20 samples to the grid-search evaluation.
Loading 2608.25478v1…