Source-linked AI summary
PointGrade: Geometric Priors for Grading MoonBoard Problems
Beatrice Stotz, Ningna Wang, Daria Nogina, Caroline Zhang, Jiyang Yin, Amy Huang, Ben Yang, Jace Li, Joel Salzman, Steven Feiner, Silvia Sellán
TL;DR
MoonBoard grades are assigned subjectively by uploaders, and hold-identity methods overlook other information in a problem. PointGrade combines hold geometry with sequence-based models and reports the highest test accuracy and ±1 test accuracy across its evaluated metrics. The authors note that subjective labels constrain achievable accuracy.
Problem
Existing hold-identity methods struggle to achieve high accuracy, while uploader-assigned grades can differ greatly in difficulty for problems with the same score.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The authors identify geometric information as important for predicting climb difficulty, particularly in the standardized MoonBoard setting.
Source: §5 · p. 4Because training grades are subjectively assigned by uploaders and no standard grading metric exists, differences between climbers’ difficulty judgments create an accuracy ceiling.
Source: §5 · p. 4
Abstract
from arXiv · showhide
A MoonBoard is a standardized bouldering wall used in gyms around the world. Climbs up the wall limited to only a subset of holds are known as problems. We introduce PointGrade, a novel machine learning approach to predicting the difficulty of a MoonBoard problem. By sampling a point cloud from pre-scanned meshes of every hold, our model combines 3D object classification architecture with existing sequence-based approaches to difficulty grade prediction. Our method captures latent geometric information contained the climb, outperforming other work on the problem that neglect this data.
1. Introduction
MoonBoard problems receive subjective uploader-assigned grades, and existing hold-identity methods overlook other information in a climb. PointGrade combines geometric information with existing approaches and achieves higher accuracy than hold-identity-only methods.
1. Introduction
- Uploader-assigned grades can vary greatly in difficulty even for problems given the same score, and ML-generated problems lack a human creator to assign a grade.The paper identifies both issues as motivation for more reliable grading.Source: §1 · p. 1
- PointGrade combines geometric information from scanned holds with existing models to account for hold sequence and the shapes that are harder or easier to grasp in that sequence.The authors also report experimentally validating algorithmic decisions, including point-cloud size and 3D-model reconstruction.Source: §1 · p. 1
- On community-uploaded problems, PointGrade outperforms all baselines, including GradeNet, across the reported test metrics.The authors report that it sets a new state of the art in MoonBoard problem grading.Source: Table 1 · p. 2, §1 · p. 2
2. Related Work
Prior climbing-grade prediction includes climber-centric methods using biometric or self-reported data and route-centric methods analyzing moves or climb segments. MoonBoard classifiers commonly use hold encodings and sequence models; PointGrade adds geometric information to these approaches.
2.1. General Climbing Difficulty Prediction
- Climber-centric methods use biometric or self-reported climb data, whereas route-centric methods model climb segments or move sequences, including Ansel’s probabilistic method and StrangeBeta [PBB12].Self-reported databases may introduce selective-reporting bias, which automatic climb logging can reduce [DP21, KDK15].Source: §2.1 · p. 2
2.2. MoonBoard Problem Grade Classification
- MoonBoard classifiers commonly encode hold positions as one-hot vectors, and preprocessing the database has consistently improved accuracy amid inaccurately labeled grades.Dobles et al. introduced the one-hot approach; later work explored convolutional, graph, and LSTM models.Source: §2.2 · p. 2
- BetaMove filters some low-quality submissions and searches for an expert-aligned hold sequence; its GradeNet model achieved 46.7% test accuracy as the reported state of the art [DC21].GradeNet embeds each move in a 22-dimensional vector and uses an LSTM recurrent neural network.Source: §2.2 · p. 3
- PointGrade uses geometric information implied by the standardized MoonBoard layout alongside latent encodings from a modified GradeNet to achieve higher accuracy than previously reported.The paper identifies existing MoonBoard classifiers as largely relying on hold positions and sequences.Source: §2.2 · p. 3
3. Method
PointGrade predicts a MoonBoard problem’s grade from its hold coordinates, combining a sequence-based branch with a geometric branch built from scanned hold meshes. The branch representations are concatenated and passed to a classifier.
3. Method
- The model maps a problem to a grade integer from 0 to 9, corresponding to Hueco grades V4 to V13.Problems are represented as lists of grid coordinates for their holds.Source: §3 · p. 3
- The dataset combines meshes scanned from every hold in the 2016 MoonBoard set with community problems split into 20,157 training, 2,442 evaluation, and 2,497 test problems.Scanned point clouds were converted to meshes and decimated to 10,000 triangles each.Source: §3 · p. 3
- One branch processes a BetaMove-sorted sequence of climbing moves with GradeNet, while the geometric branch samples the problem’s remaining hold meshes into a 1,024-point cloud for PointNet++.The hold scans used a Creality Raptor Pro 3D scanner and a rotating disc.Source: §3 · p. 3, Figure 2 · p. 3
- The model concatenates the 20-length sequence representation and 1,024-length geometric representation, then uses a multilayer perceptron to output one of 10 grades.The authors train the two branches separately before optimizing the final MLP; GradeNet uses weighted cross-entropy, while PointNet++ and the MLP use cross-entropy.Source: §3 · p. 3
4. Results
PointGrade achieves the highest exact and ±1 test accuracy across the compared methods. The results also indicate that hold shape matters and that predictions reflect the dataset’s concentration in lower grades, with errors often near the correct grade.
4. Results
- Accuracy is substantially lower when every hold is assigned the same geometry, supporting the authors’ conclusion that hold shape influences climb difficulty.The ablation uses the geometry of hold #1 from the Original School Holds set.Source: §4 · pp. 3–4
- The dataset concentrates around lower grades, and predictions mirror that distribution; errors often suggest a neighboring, usually lower, grade.The authors describe the dataset as a representative sample of community-uploaded problems.Source: §4 · p. 4
- PointGrade obtains the highest test accuracy and ±1 test accuracy across the compared MoonBoard grading methods.The qualitative outcomes are 54.18% exact successes, 33.83% one-grade failures, and 11.99% larger failures.Source: §4 · p. 4
5. Conclusions and Limitations
PointGrade introduces a new approach to MoonBoard difficulty prediction and reports a new state of the art. Its training labels are subjective uploader-assigned grades, and the authors frame broader applications as prospective beyond the standardized MoonBoard setting.
5. Conclusions and Limitations
- The authors report that PointGrade sets a new state of the art in MoonBoard problem difficulty prediction.They hope it can serve as a grading standard for community-uploaded and automatically generated problems.Source: §5 · p. 4
- Training relies on uploader-assigned subjective grades, and differences in perceived difficulty between climbers impose an accuracy ceiling.The authors propose adding climber-specific information in future work to predict the grade a particular climber would assign.Source: §5 · p. 4
- The authors highlight geometric information as relevant to climb difficulty prediction, while noting their findings are better demonstrated on the standardized MoonBoard.They hope the findings may inform applications ranging from recreational bouldering to robotic grasping and disaster rescue planning.Source: §5 · p. 4