PaperlayerView exact revision on arXiv

Source-linked AI summary

When Occupation-derived Skills Don't Help: On the Limits of Skill-Augmented Career Path Prediction

Ege Süalp, Elena Senger, Toine Bogers, Barbara Plank

arXiv:2610.20186v1cs.CL

TL;DR

This paper asks whether explicit skills add predictive signal beyond text-based career representations, given skills’ presumed role in occupational mobility. It compares predicted and ground-truth ESCO skills across integration strategies, finding only marginal and inconsistent gains over text-only baselines. The authors attribute the limits to redundancy and shortcomings in skill representation and use.

  • Problem

    Because skills are viewed as central to occupational mobility, the paper asks whether explicit skill representations add predictive signal beyond text-based career representations.

    Source: §1 · p. 1

  • Method

    The study compares predicted and ground-truth ESCO skills across score-fusion and neural approaches built on a text-only CPP baseline.

    Source: §1 · p. 1

  • Results

    Occupation-derived skill integration yields only marginal and inconsistent gains over strong text-only baselines, even with ground-truth skills.

    Source: §1 · p. 1, §8 · p. 8

  • Takeaways & Limitations

    Practitioners should not assume skill features improve CPP by default; future work should explore selective integration and skills extracted from individual career narratives.

    Source: §8 · p. 8

    Predicted skills are inherited from occupation labels, so people with the same job title receive identical skill sets despite potentially different career histories.

    Source: §6.6 · pp. 6–7

Abstract

from arXiv · show

Skills are widely assumed to play a central role in career transitions, motivating their use in career path prediction (CPP) systems. In this work, we critically examine whether explicit skill representations provide complementary signal beyond text-based job representations. We evaluate multiple strategies for incorporating skills into CPP models, including score fusion and neural architectures, under two conditions: predicted skills and ground-truth ESCO annotations. Surprisingly, across two benchmark datasets, none of these settings yield substantial improvements over strong text-only baselines. Even when using ground-truth skills, gains remain marginal and inconsistent. Through detailed analysis, we identify key factors underlying this result: high redundancy between textual and skill representations, limited discriminative power of ESCO's taxonomy, and representational collapse in skill embeddings. Furthermore, we find that skill prediction difficulty correlates with overall sample difficulty, limiting the potential for complementary signals. Our findings challenge the common assumption that explicit skill modeling inherently benefits career prediction and suggest that dense text representations already capture most skill-relevant information.

1. Introduction

The introduction asks whether explicit skills add predictive signal beyond text-based career representations, given skills’ presumed role in occupational mobility. The study finds only marginal, inconsistent gains and attributes the limits to redundancy and representation problems.

1. Introduction

  • A retrieval-based pipeline grounds job titles and descriptions in ESCO occupations, then derives associated skills and uses occupational hierarchy to improve coverage and consistency.The introduction notes that the same role can be expressed in varied ways across people and contexts.Source: §1 · p. 1
  • Across predicted and ground-truth skill settings, explicit skills yield no substantial improvement over strong text-only baselines, with oracle gains marginal and inconsistent.The analysis identifies redundant text and skill representations, weak skill discrimination, and degraded performance in the most critical top-k region.Source: §1 · p. 1
  • The study questions whether explicit skill representations improve career path prediction and why they fail to provide complementary signal.It compares predicted skills with ground-truth ESCO annotations to separate skill quality from integration effectiveness.Source: §1 · p. 1
  • The authors argue that limited gains reflect more than imperfect extraction: current models’ skill representations and use create a deeper problem.They call for re-evaluating how structured knowledge is integrated with dense representations.Source: §1 · p. 1

2. Related Work

Career path prediction research has moved from structured transition models toward text-based representations, while skill-aware systems have used taxonomies to represent occupational similarity and fit. This study examines skill integration under more realistic free-text grounding conditions.

2.1. Career Path Prediction

  • Earlier career path prediction used occupation and skill graphs, factorization, and recurrent models; later approaches shifted toward textual occupation representations and retrieval.The paper positions its work alongside text-only CPP and retrieval-based formulations.Source: §2.1 · p. 2
  • Prior work reports benefits from skill-aware similarity, person-job fit, and ESCO hierarchy models, while Decorte et al. [9] report gains from ESCO skill overlap over text-only models.Decorte et al. [9] assume pre-mapped ESCO occupations; this paper tests grounding from free-text histories across two benchmarks.Source: §2.1 · p. 2

2.2. Skill Extraction and Grounding

  • Grounding-based approaches retrieve semantically similar ESCO occupations or skills from job titles and descriptions rather than extracting explicit skill mentions.The paper’s pipeline combines occupation and skill retrieval with hierarchical fusion over ESCO.Source: §2.2 · p. 2
  • Imperfect skill grounding constrains downstream CPP: state-of-the-art skill matching remains below 0.5 MAP, and the paper adopts this retrieval-based paradigm.The pipeline combines job-to-occupation and job-to-skill retrieval with hierarchical fusion.Source: §2.2 · p. 2

3. Methodology

The methodology treats career path prediction as ranking the next ESCO occupation and tests whether retrieved or oracle skills improve text-based career representations. It builds skill features through hierarchical retrieval and integrates them through score fusion or neural models.

3. Methodology

  • For each time-ordered career history, the model ranks candidate ESCO occupations as the next occupation and retrieves relevant ESCO skills for each job experience.Both essential and optional ESCO skills count as positive evidence.Source: §3 · p. 2
  • The skill pipeline retrieves candidate skills, filters them using similar ESCO occupations, then reweights them with predicted ISCO group distributions.Direct retrieval alone reaches MAP 0.157; occupation-mediated filtering reaches MAP 0.408–0.428; ISCO weighting raises MAP to 0.452.Source: §3 · p. 2
  • The CPP baseline is Senger et al.’s text-only MLP, augmented with predicted skills through score fusion or skill-aware neural architectures.Score fusion combines text and skill-overlap scores using linear interpolation or multiplicative reweighting.Source: §3 · p. 2
  • Neural variants encode predicted skills by weighted pooling or top-k concatenation, then combine skill and career-text embeddings through early or late fusion.Pooling weights reflect retrieval confidence, skill specificity, and job recency; models train against target occupation embeddings using cosine similarity.Source: Skill-Enhanced Neural Models · pp. 2–3

4. Experimental Setup

The experiments evaluate CPP on two ESCO-linked career datasets, using ranking metrics and both predicted and oracle skills. DECORTE is the primary testbed, with a second, larger dataset used to assess generalisability.

4. Experimental Setup

  • DECORTE has 2,164 career histories and human-written job descriptions; KARRIEREWEGE+ has 99,997 histories with synthetically generated free-text input.Both datasets have standardized ESCO occupation labels; KARRIEREWEGE+ is used to assess generalisability.Source: §4 · p. 3
  • CPP is evaluated with MRR and Recall@k, while skill prediction uses MAP.MRR and Recall@k assess occupation ranking; MAP measures skill-ranking quality across relevant skills.Source: §4 · p. 3
  • The text baseline encodes job titles and descriptions with MPNet-Decorte, maps career histories to occupation embeddings, and ranks candidates by cosine similarity.Experiments use augmented and clean test splits of N=1,802 and N=227, respectively.Source: §4 · p. 3
  • Downstream CPP uses JobBERT-based ISCO-weighted skills with H2 (MAP 0.435), rather than the highest-MAP JobGTE + H2 configuration (MAP 0.452).The chosen configuration provides better discriminative signal for occupation ranking than the highest-MAP setting.Source: §4 · p. 3
  • The experiments compare score fusion with skill-enhanced neural models and include an oracle score-fusion condition using skills from gold ESCO occupations.Fusion hyperparameters are selected on validation data; the best neural configuration is further tuned with Optuna.Source: §4 · p. 3

5. Results

Across the evaluated skill-scoring and neural approaches, skill augmentation yields little or no improvement over text-only prediction. This pattern also holds on the larger KARRIEREWEGE+ dataset, though its larger size and synthetic text do not isolate scale alone.

5.1. Text-Only Baseline

  • On DECORTE, the text-only baseline reaches MRR 0.266; adding job descriptions improves MRR over titles alone (0.266 vs. 0.224).The clean split has higher MRR and R@5 but lower R@10 than the augmented split.Source: §5.1 · p. 3

5.2. Skill Overlap Scoring

  • Ground-truth ESCO skills score below text-only (MRR 0.198 vs. 0.266), and skill overlap adds no measurable improvement when fused with text.Filtering and taxonomic fusion improve skill scoring toward oracle performance, but all variants remain below text-only and fusion assigns negligible weight to skills.Source: §5.2 · p. 3

5.3. Skill-Enhanced Neural Models

  • Skill-enhanced neural models yield negligible differences on the augmented split (MRR 0.270 vs. 0.266), with up to MRR 0.287 on the clean split (N=227).Pooling consistently beats concatenation on MRR, while concatenation has better clean R@10; architecture and encoder changes have comparatively little effect.Source: §5.3 · pp. 3–4, Table 1 · p. 4

5.4. Scalability on Larger Data

  • On KARRIEREWEGE+, concatenation improves MRR by 0.5% (0.441 vs. 0.439) on augmented and 1.2% (0.508 vs. 0.502) on clean; pooling shows no improvement.Both effects are smaller than on DECORTE, and hybrid fusion converges to ignoring skills (α=1.0). Dataset size and text type differ simultaneously.Source: §5.4 · p. 4

6. Discussion

The discussion attributes limited gains from skill integration to redundant signals and weaker skill-only discrimination. Skill-enhanced predictions can improve low-ranked targets yet disrupt high-ranked predictions, limiting aggregate metric gains.

6. Discussion

  • The analyses indicate that limited skill-integration gains reflect redundancy between textual and skill-based signals, rather than model architecture or skill-prediction quality alone.Source: §6 · p. 4

6.1. Signal Redundancy

  • Text-only and skill-only ranking scores correlate at 0.79 on average (median 0.84); their top-1 predictions agree in 90.9% of cases.Text-only corrects skill-only errors in 6.2% of samples, while the reverse occurs in 2.9%; text-only is also more accurate on disagreements (10.5% vs. 5.0%).Source: §6.1 · p. 4
  • Skill enhancement helps most on managerial-to-non-managerial regressions (+0.020 MRR), but this is limited to 13.8% of samples and offset by degradation on dominant lateral transitions.The benefit is plausibly related to shifting skill profiles signaling downward mobility that text alone underrepresents.Source: §6.1 · p. 4

6.2. Discriminative Power and Confident-Prediction Disruption

  • Skill-only trails text-only on accuracy (0.154 vs. 0.186), mean margin (0.057 vs. 0.065), and calibration correlation (0.213 vs. 0.276).Skill-only margins cluster narrowly around 0.05 regardless of correctness, while text-only shows clearer separation.Source: §6.2 · pp. 4–5, Figure 1 · p. 5

6.3. The Rank Movement Paradox

  • More samples improve (43.3%, average +55.4 positions) than degrade (32.1%, average −39.1 positions), but these movements do not translate into metric gains.Improvements typically lift targets from very low ranks, whereas degradations cluster near the top and directly harm MRR and Recall@k.Source: §6.3 · p. 5

6.4. Representational Collapse and Target Misalignment

Skill embeddings are less diverse than text and overlap with it, while aligning less closely with target occupations. Skill genericness and prediction confidence track sample difficulty rather than added skill utility.

6.4. Representational Collapse and Target Misalignment

  • Rank changes favor low-ranked targets: improvements occur mainly there with larger shifts, while degradations cluster among ranks 1–10 and are individually small.The improvements are described as metric-insensitive.Source: Figure 2 · p. 5
  • Pooled skill embeddings average 0.88–0.91 pairwise cosine similarity, versus 0.072 for text; concatenation lowers similarity to 0.655 but remains less discriminative.Pooled skills concentrate 76.2–80.0% of variance in the top 10 components, compared with 38.6% for text.Source: §6.4 · p. 5, Figure 3 · p. 6
  • Career text and skill embeddings have mean cosine similarity 0.397, while text is closer to target occupations than skills: 0.355 versus 0.256.These measurements indicate that skill content overlaps with text and is less aligned with the target-occupation space used for ranking.Source: §6.4 · pp. 5–6

6.5. Skill Genericness and Sample Difficulty

  • 24.8% versus 11.8% text-only Acc@1 for the most generic versus most specific skill quartile, respectively, reversing the expected genericness pattern.The comparison is reported for DECORTE.Source: §6.5 · p. 6
  • Genericness reflects sample difficulty, not skill utility: generic skills cluster in well-represented occupations, and skill-prediction confidence improves alongside text-only accuracy.Correctly predicted samples have lower genericness across models, including text-only (p <0.001).Source: §6.5 · p. 6

6.6. The Grounding Assumption Limitation

Occupation-level skill grounding assigns identical predicted skill sets to people with the same job title, limiting individual-specific signal. The authors recommend checking category-level utility and considering selective skill integration.

6.6. The Grounding Assumption Limitation

  • Occupation-level grounding gives people with the same job title identical predicted skill sets, and even a perfect occupation classifier cannot recover individual-specific skills.The constraint caps complementary signal independently of model quality.Source: §6.6 · pp. 6–7
  • Occupation-derived skills may identify the right general neighborhood but lack granularity to distinguish similar occupations in the top-k ranks rewarded by MRR and Recall@k.Related ESCO occupations share substantial skill overlap by construction.Source: §6.6 · p. 7

6.7. Implications for Skill-Aware Recommender Design

  • Skill-feature utility varies by occupational category: Professionals gain +0.012 MRR while Service & Sales lose −0.028, motivating category-level redundancy checks.The authors suggest selective integration when textual confidence is low rather than blanket fusion.Source: §6.7 · p. 7

6.8. Comparison with Prior Literature

  • Our reproduced text-only baseline reaches MRR 0.266 versus Decorte et al. [9]’s 0.247, but differing ESCO taxonomy versions limit direct comparison.The paper suggests skill contribution is sensitive to text-encoder strength and taxonomy version.Source: §6.8 · p. 7

7. Limitations

The study’s limitations concern statistical uncertainty, the scope of oracle-skill evaluation, and the text-only baseline architecture.

7. Limitations

  • Single-run results lack significance tests and random-seed variance estimates, so differences of 0.005–0.01 MRR warrant caution.The same marginal, inconsistent-gain pattern across two datasets and several skill-integration settings offers informal robustness, not formal statistical testing.Source: §7 · p. 8
  • Oracle skills were evaluated primarily with score fusion; exploratory neural tests using ESCO-grounded job representations showed no improvement.Those skills are derived from occupation labels, which add little when the job representation already encodes the occupation.Source: §7 · p. 8
  • Experiments use only the MLP text-only baseline of Senger et al., leaving generalization to other CPP architectures open.Source: §7 · p. 8

8. Conclusion

The study finds that occupation-derived skills add only marginal, inconsistent gains over text-only baselines, even with ground-truth skills. It attributes this to redundant signals and collapsed skill embeddings, and recommends selective skill integration as future work.

8. Conclusion

  • Occupation-derived skill integration yields only marginal, inconsistent gains over strong text-only baselines, including with ground-truth skills.Source: §8 · p. 8
  • The error analysis attributes limited gains to redundancy between textual and skill-based signals and severe representational collapse in skill embeddings.Source: §8 · p. 8
  • The authors advise against assuming skill features improve CPP by default and propose selective integration for textual uncertainty or skills drawn from career narratives.Source: §8 · p. 8

Declaration on Generative AI

The authors report using Claude to assist with drafting, restructuring, condensation, and language editing, and state that they reviewed and edited the content and take responsibility for it.

Declaration on Generative AI

Loading 2610.20186v1…