LoRA Fine-Tuned Models for Control Systems Course Q\&A: A Multidimensional Evaluation of Model Scale and Rank Effects

1 Introduction

• Control-systems answers must coordinate terminology, derivations, and explanations in a verifiable, stepwise structure to be suitable as instructional material.Fluent answers without stable organization and teaching points may appear complete while remaining unsuitable for instruction.Source: §1 · p. 2
• Existing work has not systematically tested how model size and LoRA rank jointly affect course question answering or whether higher rank compensates for a smaller model.This gap complicates configuration choices under hardware constraints and instructional goals.Source: §1 · pp. 2–3
• The study investigates that joint effect in Linear Control Systems, focusing on the trade-off between parameter efficiency and performance.Its questions cover similarity to references, stability of the Solution-Method-Teaching Points format, marginal returns, and the limits of interpreting those metrics as reasoning ability.Source: §1 · p. 3
• The authors construct a 360-sample course dataset, compare Qwen2.5-3B and 7B models at LoRA ranks 4, 8, and 16, and evaluate similarity, structure, and bootstrap confidence intervals.They explicitly distinguish textual similarity and structural stability from domain-specific correctness, which remains for future expert evaluation.Source: §1 · p. 3

2.1 Adaptation Approaches for Specialized Course Question Answering

• Prompting guides response form and RAG supplies external materials, but neither directly establishes that a model learned course-specific terminology, solution order, and style.Domain-specific fine-tuning adjusts the generation distribution using task examples.Source: §2.1 · p. 3

2.2 Parameter-Efficient Fine-Tuning and Low-Rank Adaptation

• LoRA freezes pretrained weights and represents task updates with trainable low-rank matrices, reducing the number of parameters that training must update.The rank r controls matrix dimensions and adaptation capacity; increasing it trades parameter efficiency against task fit and generalization risk.Source: §2.2 · p. 4
• LoRA’s modular design allows course-specific adapters to share a base model, while the study compares ranks 4, 8, and 16 across two model sizes.The comparison is intended to examine performance saturation and the effects of model size.Source: §2.2 · p. 4

2.3 Evaluation of Educational Question-Answering Models

• ROUGE and BERTScore measure lexical and semantic similarity to references, but similarity does not validate mathematical correctness.Responses may resemble a reference while containing an incorrect formula, skipped derivation step, or unstated assumption.Source: §2.3 · pp. 4–5
• Structured-output features assess whether answers consistently include Solution, Method, and Teaching Points, but expert assessment is still needed for correctness and instructional usefulness.Human evaluation can assess correctness, completeness, derivation clarity, variable explanations, and the conditions behind conclusions.Source: §2.3 · p. 5
• The study addresses the underexplored relationship among model size, LoRA rank, answer similarity, and structured instructional output for small engineering-course datasets.It treats changes in text and format as distinct from evidence of domain-specific reasoning.Source: §2.3 · p. 5

3.1 Course Context of the Source Data

• Linear Control Systems combines mathematical derivation with engineering analysis across topics including modeling, stability, frequency-domain analysis, and controller design.Course answers are expected to include stepwise derivations, consistent notation, and interpretations of engineering implications.Source: §3.1 · p. 5

3.2 Data Collection and Preprocessing

• Exercises were selected for coverage of core syllabus knowledge, standardized, and split into a 360-sample dataset after excluding trivial or excessively open-ended questions.The original notation and system representations were harmonized; visual diagrams were converted into textual descriptions and problem conditions retained.Source: §3.2 · pp. 5–6

3.3 Standardization and Structured Construction of Reference Answers

• Reference solutions were rewritten under Solution, Method, and Teaching Points, with derivations and calculations separated from key concepts and common errors.Approximately 20% of samples were spot-checked for formula correctness, calculation completeness, notation, formatting, and teaching-point accuracy.Source: §3.3 · p. 6

3.4 Conversion to a Dialogue Format for Model Fine-Tuning

• Each training example uses system, user, and assistant messages, with a shared teaching-assistant system prompt and a four-part structured user prompt.The assistant supplies the structured reference answer; stratified sampling sought similar knowledge-module and question-type distributions across train, validation, and test sets, with no exact duplicates found.Source: §3.4 · pp. 6–7

3.5.1 Basic Statistical Characteristics

• The dataset covers core course topics and four task categories, with computational derivation questions forming the largest category.Answers average approximately 15–25 LaTeX environments, reflecting the course’s reliance on mathematical expression and multistep derivations.Source: §3.5.1 · p. 7

3.5.2 Coverage of Answer Structures

• All 360 reference answers include Solution, Method, and Teaching Points, giving every structural marker 100% coverage.The uniform dialogue, question, and answer structures provide a consistent basis for LoRA fine-tuning across model sizes.Source: §3.5.2 · p. 7

4.1 Research Framework and Experimental Variables

• The framework converts course exercises and reference solutions into dialogue data, applies LoRA fine-tuning, and evaluates test-set outputs across multiple dimensions.Fig. 1 presents the workflow.Source: §4.1 · p. 8

4.2 Base Models and Experimental Environment

• Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct were compared at ranks r = 4, r = 8, and r = 16, varying only model size and rank.The models were selected to compare adaptation at different sizes under resource constraints.Source: §4.1 · p. 8, §4.2 · p. 9

4.3 LoRA-Based Supervised Fine-Tuning

• LoRA freezes base weights and trains low-rank matrices in selected projections, with rank controlling their dimensions.The experiments used q_proj, k_proj, v_proj, and o_proj, with α = 2r.Source: §4.3 · p. 9

4.4 Inference, Generation, and Evaluation Metrics

• Evaluation combined ROUGE and BERTScore similarity with coverage features for the Solution-Method-Teaching Points format.The authors caution that structural features measure formatting, not response quality, and that automatic metrics do not certify reasoning or mathematical correctness.Source: §4.4 · pp. 10–11

4.5 Bootstrap Robustness Analysis

• Paired bootstrap resampling estimated confidence intervals for fine-tuned-versus-base gains using identical test-sample indices.The analysis used 10,000 resamples; BERTScore-F1 intervals were unavailable because per-sample scores were not retained.Source: §4.5 · p. 11

4.6 Experimental Controls and Reproducibility

• Shared splits, prompts, inference settings, and deterministic decoding support comparisons, but results remain observations from one seed and a 54-item test set.Conclusions are restricted to reference-answer similarity and structured output, not general domain-specific reasoning.Source: §4.6 · pp. 11–12

5.2 Training Dynamics and Validation-Set Convergence

• 0.8830 to 0.8080 for 3B and 0.7992 to 0.7330 for 7B were the best validation-loss decreases from r = 4 to r = 16.Higher ranks also corresponded to increased validation-token accuracy and decreased validation entropy.Source: §5.2 · p. 12
• At r = 16, validation loss was 0.7330 for 7B and 0.8080 for 3B, indicating closer validation-set fit for the larger model under these settings.The authors caution that validation-set fit is not evidence of improved domain-specific reasoning.Source: §5.2 · p. 12
• Validation loss decreased across all six experiments, with most reduction in the first three epochs and smaller gains afterward.The comparison used the same splits, prompt, target modules, epochs, and deterministic inference parameters.Source: §5.2 · p. 12, §5.1 · p. 12

5.3 Automatic Evaluation on the Test Set

• At both model sizes, r = 16 achieved the highest ROUGE-L, while 7B-r8 and 7B-r16 differed little in BERTScore-F1; gains were not monotonic on every metric.The 3B model continued improving at r = 16, suggesting it may require greater adaptation capacity to fit the target representation more fully.Source: §5.3 · p. 13

5.4 Bootstrap Confidence Intervals and Metric Robustness

• ROUGE and BERTScore measure textual and semantic proximity, not the correctness of domain-specific reasoning.Bootstrap intervals assess stability under changes in test-sample composition without changing these evaluation boundaries.Source: §5.3 · p. 13, §5.4 · p. 13
• Most bootstrap confidence intervals for ROUGE improvements lay above zero; ∆ROUGE-1 at 7B-r4 crossed zero and was less stable.For 3B, all three ROUGE metrics had intervals above zero at r = 4, 8, and 16; for 7B, this held for r = 8 and 16.Source: §5.4 · p. 13
• Solution and Method coverage improved consistently across LoRA settings, while Teaching Points coverage gains varied with model size and were more pronounced for 7B.Some 3B settings had wider confidence intervals and lower stability for Teaching Points coverage.Source: §5.4 · p. 13

5.3 Automatic Evaluation on the Test Set

• LoRA-fine-tuned models outperformed their corresponding base models on most metrics, and 7B models generally outperformed 3B models, especially on ROUGE-2, ROUGE-L, and BERTScore-F1.Figure 3 reports ROUGE-1, ROUGE-2, ROUGE-L, and BERTScore-F1.Source: §5.3 · p. 12, Figure 3 · p. 14

5.5 Effects of Model Size on Adaptation

• Before adaptation, 7B exceeded 3B on all four metrics, but the ROUGE-L difference was +0.0207 and the BERTScore-F1 difference was +0.0019.The passage characterizes the pre-adaptation margins as small.Source: §5.5 · p. 14
• After LoRA fine-tuning at r = 16, the ROUGE-1 difference narrowed from +0.0263 to +0.0148, while ROUGE-2 and ROUGE-L differences widened to +0.0318 and +0.0317.The larger model gained more on phrase-level and sequence-level matching; LoRA did not eliminate the model-size effect.Source: §5.5 · p. 14
• 3B-r16 exceeded 7B-r4 on ROUGE-L, 0.3776 versus 0.3615, and BERTScore-F1, 0.8606 versus 0.8579.The passage identifies this cross-size comparison as relevant to local deployment or limited-resource settings.Source: §5.5 · p. 14

5.6 LoRA Rank and Parameter Efficiency

• Parameter efficiency is the ROUGE-L improvement per million adapter parameters, quantifying the return between parameter cost and performance gain.The analysis compares this measure across ranks alongside absolute improvements.Source: §5.6 · pp. 15–16
• For 7B, r = 16 raised ∆ROUGE-L to +0.0874 from +0.0726 at r = 8, but reduced gain per million adapter parameters to 0.0087 from 0.0144.The same pattern of larger absolute gain but lower parameter efficiency occurred for 3B; Table 7 compares rank and parameter efficiency.Source: Table 7 · p. 16, §5.6 · p. 16
• r = 8 offered a stable compromise between parameter count and performance; r = 16 suited maximizing reference-answer similarity and structural completeness, with diminishing returns to weigh against training cost and overfitting risk.Figure 6 visualizes the trade-off between absolute performance and parameter efficiency.Source: §5.6 · p. 16, Figure 6 · p. 16

5.7 Analysis of Structured Instructional Output

• Solution and Method coverage reached 1.00 for 3B at r = 8 and r = 16, and for 7B under all three LoRA settings.Base coverage was 0 and 0.13 for 3B, and 0.09 and 0.17 for 7B, respectively; Figure 7 shows marker coverage.Source: Figure 7 · p. 17, §5.7 · p. 17
• At r = 8 and r = 16, 3B Teaching Points coverage reached 0.94 and 0.96, while both Solution and Method coverage reached 1.00.The 3B base model already had relatively high Teaching Points coverage but low Solution and Method coverage.Source: §5.7 · p. 17
• For 7B, Teaching Points coverage rose from 0.56 in the base model to 0.87, 0.91, and 0.96 across the LoRA settings; its positive trend was more consistent.Together with bootstrap results, Solution and Method improvements were stable under most settings, while Teaching Points gains were more sensitive to size and rank.Source: §5.7 · p. 17

• LoRA fine-tuning brought both model sizes closer to course reference answers, with ROUGE-L and structured-output gains stable under bootstrap resampling.Source: §5.8 · p. 17
• The 7B model generally outperformed the 3B model, r = 16 achieved the highest absolute scores, and r = 8 offered a more conservative performance-efficiency compromise.Source: §5.8 · p. 17
• Solution and Method coverage gains were stable across groups, but Teaching Points coverage gains were concentrated in the 7B group; these metrics do not assess reasoning correctness.Source: §5.8 · p. 17

6.1 What Did LoRA Fine-Tuning Improve?

• LoRA aligned outputs with course vocabulary, formula presentation, and step organization; Solution and Method coverage rose from near zero to 100%, while 7B Teaching Points coverage rose from 0.56 to 0.96.Source: §6.1 · p. 18

6.2 Trade-Off Between Model Size and LoRA Rank

• The 7B model outperformed the 3B model on almost every metric, while its ROUGE-L advantage grew from +0.0207 before fine-tuning to +0.0317 at r = 16.Source: §6.2 · p. 18
• Although r = 16 achieved the highest absolute scores, parameter efficiency declined with rank; r = 8 substantially outperformed r = 4 using about half the adapter parameters of r = 16.The discussion recommends 7B-r16 when maximizing similarity without strict resource limits, and 3B-r8 or 7B-r8 for local deployment; 7B-r4 yielded ∆ROUGE-L = +0.0407 for prototyping.Source: §6.2 · p. 18

6.3 Interpretive Boundaries of the Evaluation Metrics

• ROUGE, BERTScore, and structural markers capture textual similarity and pedagogical structure, but cannot verify formulas or derivations; instructor or expert assessment remains necessary.Source: §6.3 · pp. 18–19

6.4 Limitations

• The dataset contains 360 samples, including 54 test samples, and does not cover all course topics or difficulties; complex and open-ended problems need more samples for reliable evaluation.Source: §6.4 · p. 19
• Because the study compared only Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct, its findings should not be generalized directly to other model families or larger models.Source: §6.4 · p. 19
• Evaluation relied on automatic metrics and structural markers, which do not adequately assess formula accuracy, derivational rigor, or instructional usefulness; the authors call for expert rubrics.Source: §6.4 · p. 19
• Robustness is limited by one training seed, no reported multi-seed variation, BERTScore-F1 means without confidence intervals, and no strong-prompt or RAG baseline.Source: §6.4 · p. 19

7.1 Conclusions

• LoRA improved reference-answer similarity and structured-output stability at both model sizes; 7B-r16 achieved ROUGE-L 0.4093 and BERTScore-F1 0.8643.ROUGE-L gains were 0.0764 [0.0613, 0.0915] for 3B-r16 and 0.0874 [0.0687, 0.1042] for 7B-r16; both bootstrap intervals excluded zero.Source: §7.1 · p. 19
• Model size and rank had joint effects: 7B's ROUGE-L advantage grew from +0.0207 to +0.0317 after fine-tuning, while 3B-r16 beat 7B-r4 on some metrics.Source: §7.1 · p. 19
• Rank's marginal return decreased as it increased; r = 8 approached r = 16 performance with approximately half as many adapter parameters, making it a robust resource-constrained compromise.Source: §7.1 · p. 19
• The findings support course alignment in form and style, not certification of reasoning correctness; formula accuracy, derivational rigor, and instructional reliability still require expert evaluation.Source: §7.1 · p. 19
• LoRA offers a feasible, parameter-efficient way to align open-source instruction-tuned models with a specific engineering course, with rank and model size selected for resources, goals, and data.Source: §7.1 · pp. 19–20

7.2 Future Work

• Future work should expand the control-systems dataset across topics, question types, and difficulty levels, while improving consistency in reference formulas, variables, assumptions, and explanations.Source: §7.2 · p. 20
• Evaluation should add multiple random seeds, expert rubrics, and stronger baselines, assessing formula accuracy, derivational rigor, completeness, clarity, and instructional usability.Source: §7.2 · p. 20
• Further comparisons should include other model families, strong-prompt and RAG baselines, and combined LoRA-RAG systems to distinguish the contributions of model size, prompting, retrieval, and fine-tuning.Source: §7.2 · p. 20