Source-linked AI summary
CRITICS - Critical Science Without Borders: Language Models to Promote Critical Thinking in Science Education
Rodrigo Agerri, Itziar Aldabe, Elena Cabrio, Mark Cieliebak, Jan Deriu, Mariana Flores, Jurgita Kapociute-Dzikiene, Dovile Kuiziniene, Arantza Rico, Aritz Ruiz-Gonzalez, Aitor Soroa, Mantas Vaskevicius, Serena Villata
TL;DR
Scientific knowledge and advanced educational resources are often concentrated in high-resource languages, limiting access and opportunities to develop evidence-based argumentation in students’ own languages. CRITICS proposes combining scientific machine translation with teaching-sequence generation, argument analysis, and automated assessment. The project aims to deliver multilingual translation, critical feedback, and evaluation methods through applications including a Parallel Reader and Teacher Evaluator.
Problem
Scientific resources are concentrated in high-resource languages, while evidence for LLM-based scientific translation and related educational tools remains limited for lower-resource settings.
Source: §1 · p. 2, §2.1 · p. 3Method
CRITICS combines LLM-based scientific translation with teaching-sequence generation, multilingual argument mining, critical-question generation, and assessment aligned with learning goals.
Source: §1 · pp. 2–3, §2.2 · p. 4, §2.3 · p. 4Results
CRITICS aims to deliver optimized scientific-document translation, argumentation mining, automated critical feedback, and evaluation methods, validated through applications such as the Parallel Reader and Teacher Evaluator.
Source: §4 · p. 7Takeaways & Limitations
The project will investigate how open-weight models, synthetic data, and LLM-as-a-judge metrics can support reproducible, transparent work across diverse linguistic contexts.
Source: §4 · p. 7
Abstract
from arXiv · showhide
The CRITICS project addresses science accessibility and literacy by converging advanced Machine Translation (MT) based on Large Language Models (LLMs) with educational technology. By leveraging MT systems specifically optimized for scientific content, educational institutions can provide accurate, culturally relevant translations of scientific materials in students' native languages, ensuring that complex scientific concepts are comprehensible while maintaining technical accuracy. Building on these translations, the project explores the design and evaluation of innovative science teaching-learning proposals grounded in curriculum-aligned teaching-learning. Thus, CRITICS will investigate key components of scientific argumentation and critical thinking practices together with textual feedback aligned with learning objectives and assessment criteria inspired by competence-based evaluation frameworks. CRITICS aims to break down language barriers to accessing cutting-edge research and educational materials currently available only in high-resourced languages, thereby facilitating the democratization of scientific knowledge and fostering critical thinking in science education.
1. Introduction
CRITICS aims to address language-based barriers to scientific literacy by combining scientific machine translation with educational technology. It proposes a three-component system to translate scientific documents, turn translations into teaching sequences, and assess student responses.
1. Introduction
- CRITICS combines LLM-based scientific translation with educational technology to address unequal access to scientific knowledge and limits on students’ evidence-based argumentation in their own languages.The project is a three-year initiative under the “Science in your own language” topic.Source: §1 · pp. 1–2
- The project will adapt LLMs to generate customized, level-appropriate science education materials from scientific content made accessible through machine translation.The materials are intended to be pedagogically sound.Source: §1 · p. 2
- CRITICS proposes three components: scientific document translation, a Teaching and Learning Sequence generator, and an automatic assessor for student responses.The assessor analyzes evidence-based argumentation and generates critical questions; assessment results feed back into translation, resource development, and assessment criteria.Source: §1 · pp. 2–3
2. Related Work and Novelty
CRITICS targets gaps in scientific-document translation, argument mining, critical-question generation, and evaluation. Its proposed advances focus on low-resource languages, factual scientific argumentation, learning-goal-aligned feedback, and task-specific LLM evaluation.
2. Related Work and Novelty
- CRITICS will involve domain experts, including science teachers and researchers in natural science and AI, in supervision and qualitative evaluation under its TRL 5 plan.The project describes TRL 5 as technology validated in relevant environments.Source: §2 · p. 3
2.1. LLM-based MT of Scientific Documents
- For low-resource scholarly translation, CRITICS will prioritize terminology precision, structural accuracy, and document-level coherence alongside fluency, using glossaries and tools to preserve citations, equations, code, and formatting.The project targets English–Lithuanian, English–Basque, English–French, and English–German, with French included for comparative purposes.Source: §2.1 · p. 3
2.2. Argument(ation) Mining (AM)
- CRITICS will tailor argument mining to scientific texts, study argument similarity across languages, and extract argument graphs for critical-question generation; it will address methods’ insensitivity to factual content, building on medical-education work.The project notes that interpreting ostensibly objective scientific evidence can be ambiguous and subjective.Source: §2.2 · p. 4
2.3. Critical Questions Generation and Automatic Assessment
- CRITICS proposes multilingual critical questions, argument feedback, and grades aligned with Teaching and Learning Sequence goals and assessment criteria, for competency-based assessments in natural sciences and AI.The stated target languages are Basque, French, German, and Lithuanian.Source: §2.3 · p. 4
2.4. Evaluation and Benchmarking
- Because expert assessment remains essential despite emerging automatic text-generation metrics, CRITICS will evaluate question generation, Teaching and Learning Sequences, and feedback with experts assessing grammar, relevance, semantic soundness, argumentation, and sometimes cognitive-framework alignment.The project identifies LLM-based evaluation for scientific MT and critical-question and feedback generation as an open research problem, and proposes methods tailored to these tasks.Source: §2.4 · p. 5
3. Methodology and Work Plan
CRITICS targets natural sciences and AI across Basque, Lithuanian, German, and French. Its work plan combines data collection, model development, evaluation, and prototypes for scientific translation and teacher assessment.
3. Methodology and Work Plan
- CRITICS will address biology, chemistry, physics, and AI in Basque, Lithuanian, German, and French, with AI posing terminology challenges in less-resourced languages.The project characterizes Basque as an agglutinative language isolate, Lithuanian as East Baltic and inflectional, German as West Germanic and inflectional, and French as Romance and synthetic-fusional.Source: §3 · p. 5
- The project will adapt open-weight LLMs for document-level scientific translation, train models to recognize evidence-based arguments and detect fallacies and misconceptions, and generate critical questions and detailed feedback for competency-based assessment.It also plans qualitative evaluation of question and feedback generation using an LLM-as-a-Judge paradigm comparable with human judgments.Source: §3 · pp. 5–6
3.1. Work Plan
- Three development-evaluation cycles will be replicated across WP2–WP5, beginning with use-case requirements and data collection for a Parallel Translator of Scientific Documents and a Teacher Evaluator.The project identifies scientific sources in biology, chemistry, physics, and AI, including preprints and educational resources; final source selection and analysis are assigned to WP2.Source: §3.1 · p. 6
- WP2 will compile and annotate 500 anonymized higher-education competency-based assessments, then annotate argumentation graphs, critical questions, and feedback for benchmarking.WP3 focuses on scientific-data MT, WP4 on argumentation quality, and WP5 on evaluating MT, critical questions, and feedback; WP6 applies WP2–WP4 results to the two prototypes.Source: §3.1 · pp. 6–7
- The work plan combines human and automated evaluation of translated scientific texts, with protocols to reduce rating variance and calibrated metrics to address disagreement with human feedback and system preferences.The project also plans to develop instruction and alignment data from already instructed models using the magpie approach [38].Source: §3.1 · p. 7
4. Concluding Remarks
CRITICS integrates LLM-based machine translation with critical thinking in science education to address language barriers. It plans optimized scientific MT, argumentation mining, critical feedback, and evaluation, with possible future language and modality extensions.
4. Concluding Remarks
- Six work packages will pursue optimized scientific MT, scientific-text argumentation mining, automated critical feedback, and robust evaluation validated through the Parallel Reader and Teacher Evaluator.The project targets low- and medium-resource languages including Basque, Lithuanian, French, and German.Source: §4 · p. 7
- CRITICS targets terminology consistency, document-structure preservation, and multilingual argumentation assessment, alongside open-weight models, synthetic data, and LLM-as-a-judge metrics for reproducible, transparent results.The project frames these efforts as advancing domain-specific NLP and educational technology.Source: §4 · p. 7
- Future extensions could add low-resource languages and multimodal LLMs for visual scientific content.The passage presents these as possible extensions, not current project deliverables.Source: §4 · p. 7