Multi-Task Learning for Sparsely-Labeled Time Series: A Case Study on Cold-Hardiness Modeling

1 Introduction

• Limited, temporally sparse labels and cultivar-specific responses make deep models difficult to train, while low-data cultivar models can perform poorly against scientific models.Cultivars share some structure as members of one species, motivating the question of whether combining their data improves individual predictions.Source: §1 · pp. 2–3
• The study investigates wrapper-based MTL by treating cultivars as tasks and learning one model to predict them simultaneously.It focuses on approaches that can be added to an existing single-task base model with little additional work.Source: §1 · p. 3
• Multi-head models attach task-specific output heads to a shared base model, whereas task-embedding models provide learned task vectors at the input.Both approaches can be adapted to transfer learning when data for the original tasks are unavailable.Source: §1 · p. 3
• MTL consistently outperforms single-task and state-of-the-art scientific models for cold-hardiness and budbreak prediction, with especially pronounced gains for cultivars with the sparsest data.Combining the qualitatively different budbreak and cold-hardiness tasks yields additional accuracy benefits for both; the same models are effective for transfer learning with less data for new cultivars.Source: §1 · p. 3
• The paper expands evaluation to MTL across budbreak and cold-hardiness tasks and studies MTL architectures as transfer learners when prior-task data are unavailable.It reports that the resulting transfer-learning approach is a viable alternative to MTL in that setting.Source: §1 · p. 4

2 Cold Hardiness Modeling

• Cold hardiness is quantified by lethal temperatures at which 10%, 50%, and 90% of buds freeze, measured using differential thermal analysis; the case study uses bi-weekly measurements.LTE50 measurements across cultivars show seasonal acclimation and de-acclimation, with differing responses despite exposure to the same weather.Source: §2 · p. 4, Figure 1 · p. 5
• Existing cold-hardiness models estimate lethal temperatures from historical daily temperatures, but the Ferguson model has limited expressiveness and omits other influential weather inputs.It computes daily cold-hardiness changes from accumulated thermal time, weighted by dormancy-stage coefficients.Source: §2 · pp. 4–5
• Budbreak prediction matters because emerging green tissues become vulnerable to frost, and current forcing or chilling-forcing models use daily air temperature and cultivar-specific parameters.These models account for forcing alone or both chilling and forcing, respectively.Source: §2 · p. 6
• Deep models could represent nonlinear relationships and use additional weather variables, but relatively small datasets may limit their benefit, especially for cultivar-specific learning.The paper therefore investigates MTL to combine data across cultivars.Source: §2 · p. 6

3 Multi-Task Learning

• The section formulates multi-task learning in relation to single-task learning and reviews primary MTL methods.It motivates the general methods selected for the case study.Source: §3 · p. 6
• The section's scope moves from general MTL methods to their later application-specific details.Source: §3 · pp. 6–7
• Application-specific instantiations of the general MTL methods are deferred to Section 4.Source: §3 · p. 7

3.1 Problem Formulation

• Single-task learning fits a predictive mapping using one dataset, while multi-task learning models related tasks jointly using their separate datasets.For grape LTE prediction, each cultivar is a task and its dataset contains weather inputs paired with LTE targets.Source: §3.1 · p. 7
• MTL seeks task-specific predictions informed by all tasks, with the hope of improving each task over an independent single-task model.This requires shared task structure that the model architecture can capture; training can optimize joint loss, optionally with task weights and regularization.Source: §3.1 · p. 7
• In transfer learning, a model is first trained on source tasks, then adapted to a target task without access to source-task data.The protocol reflects settings where prior tasks are no longer available, such as because of privacy or storage considerations.Source: §3.1 · p. 8

3.2.1 Soft Parameter Sharing

• Soft parameter sharing trains task-specific model copies while encouraging their parameters to remain close, balancing task losses against pairwise-distance losses.The approach is flexible but memory- and compute-intensive as the number of tasks grows, so the study excludes it.Source: §3.2.1 · p. 8

3.2.2 Hard Parameter Sharing

• Hard parameter sharing trains one model across all tasks and is less memory-intensive than soft sharing, but can be more vulnerable to negative interactions among tasks.The study focuses on hard sharing for its practical computing and memory advantages.Source: §3.2.2 · pp. 8–9

3.2 MTL Methods

• Wrapper-style hard-sharing methods adapt an existing single-task architecture, supporting easier MTL adoption where domain-specific single-task models already exist.The study considers multi-head and task-embedding approaches as wrappers.Source: §3.2.2 · p. 9

Multi-Head Models

• Multi-head models share a base network’s features and attach a separate output head to each task, with prior work reporting gains over independent single-task models.They can bottleneck performance when tasks are weakly related, and model size grows with task count.Source: Multi-Head Models · p. 9

Task Embedding Models.

• Task-embedding models append task information to the input; dense embeddings can encode task similarity, unlike maximally sparse one-hot context vectors.This paper combines learned embeddings with inputs by concatenation, element-wise addition, or element-wise multiplication.Source: Task Embedding Models · p. 10

Non-Wrapper Models.

• Non-wrapper architectures such as cross-stitch, sluice, and NDDR-CNN add complexity and hyperparameters, while the case study’s specialized single-task model is difficult to adapt to them.Wrapper-based approaches were straightforward to apply to that model.Source: Non-Wrapper Models · pp. 10–11

3.3 Related Work on MTL for Time-Series

• Time-series classification for human activity uses fixed-length inputs and is not readily adaptable to numeric output sequences from growing inputs.The paper’s prediction setup is therefore structurally different.Source: §3.3 · p. 11
• Forecasting approaches that treat each future step as a task or aggregate artificial data-view tasks forecast one process, unlike this study’s MTL across distinct related processes.These approaches are described as orthogonal to the paper’s focus.Source: §3.3 · p. 11
• Agricultural MTL for wheat yield and fertilization shares the goal of leveraging related data, but assumes labels at every time-series point rather than sparse targets.The paper’s time series have sparsely labeled targets.Source: §3.3 · p. 11
• Earlier task-embedding MTL for solar and power estimation is extended here with broader embedding variants, direct multi-head comparisons, and transfer learning.The paper describes three prior time-series works using dense task embeddings.Source: §3.3 · p. 11

4.1 Prediction Problems

• Each cultivar defines a task whose model predicts LTE and/or budbreak sequences from daily weather; LTE labels are missing on days without measurements, while budbreak targets switch from zero to one.The prediction formulation uses the weather and ground-truth sequence for each season.Source: §4.1 · p. 12

4.2 Single-Task Model

• The single-task model processes weather sequentially with an RNN backbone and a cultivar-specific fully connected output layer to produce LTE or budbreak estimates.The backbone uses two fully connected layers, a GRU, and another fully connected layer.Source: §4.2 · pp. 12–13
• The GRU’s recurrent latent state represents processed weather history, which the authors suggest can implicitly approximate the plant’s evolving dormant state.Single-task parameters are trained on that cultivar’s dataset.Source: §4.2 · p. 13

4.3 Multi-Task Models for Cold Hardiness and Budbreak

• Multi-head models share RNN backbone features across cultivars and use a separate prediction head for each cultivar.The shared features are learned from combined cultivar data, while each head produces its designated cultivar’s output.Source: §4.3 · pp. 13–14, Figure 2 · p. 13
• Task-embedding models combine learned cultivar embeddings with weather features before the backbone, using element-wise multiplication, addition, or concatenation.The learned structure and cultivar parameters may lack clear scientific interpretation, trading interpretability for potential expressive power.Source: §4.3 · p. 14

4.4 Training Details

• Models use Adam with a 0.001 initial learning rate, batches of 12 seasons, and 400 training epochs.Weather inputs have 12 features; the GRU hidden state and internal memory each have dimensionality 2048.Source: §4.4 · p. 14

5.1 Grapes

• Field-grown grapevine buds from up to 30 diverse cultivars or genotypes have been measured since 1988 at research sites in Washington.Samples were collected from autumn leaf fall to spring bud swell at daily, weekly, or two-week intervals.Source: §5.1 · pp. 14–15
• Differential Thermal Analysis records LTE10, LTE50, and LTE90 cold-hardiness measurements, while budbreak is recorded when green tissue appears in 50% of previously dormant buds.Daily environmental data come from the nearest of three on-site weather stations.Source: §5.1 · p. 15
• Each cultivar dataset contains daily rows across seasons, but LTE is absent on days without a sample; the study’s datasets range from 4 to 34 seasons per cultivar.Seasons are defined from September 7 through May 15.Source: §5.1 · p. 15
• Daily model inputs include air temperature, relative humidity, dew point, precipitation, and wind measurements.Table 1 summarizes LTE and budbreak data collection across cultivars.Source: §5.1 · p. 15, Table 1 · p. 16

6 Experiments

• The study evaluates wrapper-based MTL architectures across 21 grape cultivars and considers transfer learning as an alternative when source-task data are unavailable.The experiments use the ConcatE task-embedding variant after prior experimentation found it tends to outperform MultE and AddE.Source: §6 · pp. 16–17, §6.3 · pp. 21–22

6.2.1 MTL vs. STL

• MTL consistently outperforms STL across cultivars and prediction problems, and MTL models improve mean and median cold-hardiness performance over the Ferguson model.For budbreak, MTL also yields better median day-prediction errors and fewer large errors; at least one MTL variant improves BCE loss over STL for all but two cultivars.Source: §6 · p. 16, §6.2.1 · pp. 18–19

6.1 Experimental Setup

• Experiments use dormant-season cultivar data, mask losses for missing LTE labels, and evaluate LTE50 prediction with RMSE and budbreak with BCE and day error.The setup uses five weather features, two test seasons per cultivar, and averages performance over three train/test trials.Source: §6.1 · pp. 17–18

6.2.2 Comparison to State-of-the-Art

• MTL models outperform the Ferguson model on 14 cultivars for the embedding model and 15 for MultiH, while Ferguson outperforms STL across all cultivars.The MTL models also have improved mean and median performance across cultivars compared with Ferguson.Source: Table 2 · p. 17, §6.2.2 · pp. 19–20

6.2.3 Impact of Task Data Set Size for Grape LTE Prediction.

• With just 2 or 5 seasons of a cultivar’s data, MTL outperforms STL trained on all of that cultivar’s data.MTL performance does not improve consistently as more cultivar data is added, and can degrade in some cases.Source: §6.2.3 · p. 20

6.2.4 MTL vs. Data Aggregation

• MTL models perform better on average than STL trained on aggregated cultivar data, showing gains beyond simply increasing the training-set size.Aggregating data also improves STL performance for grape LTE prediction; similar observations hold for budbreak.Source: §6.2.4 · pp. 20–21

6.3 Transfer Learning

• MultiH transfer learning performs on par with standard MTL, while transfer with the embedding model significantly hurts performance relative to MTL embedding models and STL.Model selection generally outperforms STL but is outperformed by MTL and MultiH transfer; the authors do not have a strong explanation for the embedding-transfer result.Source: §6.3 · p. 23, Table 5 · p. 23

• Finalized MTL models were deployed on AgWeatherNet for beta testing during the 2022–2023 season.Source: §7 · p. 24
• A subset of users received access to the models and were asked for feedback at season’s end.Source: §7 · p. 24
• Farmers could use weather data from the nearest AgWeatherNet station, while monitoring cold hardiness through the Ferguson model and published LTE values remained available.Source: §7 · p. 24

• Wrapper-style MTL consistently outperformed STL and the state-of-the-art scientific model for grape cold-hardiness and budbreak prediction without expert domain knowledge.Source: §8 · p. 24
• MTL models could outperform STL models using only a tiny fraction of the training data for a test cultivar.Source: §8 · p. 24
• The best-performing MTL model varied between grape LTE and budbreak prediction, leading the authors to recommend validation effort when selecting a model for a new application.Source: §8 · p. 24
• Multihead fine-tuning was effective for transfer, whereas the paper’s initial embedding-based transfer efforts were not.Source: §8 · p. 24
• Selected farmers tested the models on AgWeatherNet in 2022–2023, and their feedback was being analyzed to guide interface and model improvements.Source: §8 · p. 24

• On average, ConcatE performed better than the other evaluated task-embedding variants for grape LTE prediction.Source: Appendix A · p. 25, Table A1 · p. 25
• The multiplicative embedding came close to ConcatE and is proposed as a possible alternative for future work.Source: Table A1 · p. 25

• Table B2 reports the full dataset-size ablation results for STL and MTL across all 21 grape cultivars in LTE prediction.Source: Table B2 · p. 27, Appendix B · p. 27
• An MTL model trained with just 2/5 of a cultivar’s seasons could surpass an STL model trained with all seasons for that cultivar.Source: Appendix B · p. 27, Table B2 · p. 27
• Performance degraded when training seasons increased from 20 to all available seasons; the authors suggest unusual-season weather noise might explain the pattern.Source: Appendix B · p. 27