PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

Lorenz Kuhn, Yarin Gal, Sebastian Farquhar

arXiv:2302.09664v3cs.CLcs.AIcs.LG

TL;DR

Different sentences can express the same meaning, complicating uncertainty estimation for reliable language generation. The paper introduces semantic entropy, which measures uncertainty over meanings rather than token sequences. It predicts model accuracy better than comparable baselines across open- and closed-book free-form question answering on TriviaQA and CoQA.

  • Problem

    Estimating uncertainty for reliable language generation is challenging because distinct token sequences can express the same meaning.

    Source: §1 · p. 1, §7 · p. 9

  • Method

    Semantic entropy clusters mutually entailing generations and computes entropy over meanings rather than token sequences.

    Source: §1 · p. 1

  • Results

    Semantic entropy predicts model accuracy better than comparable baselines across open- and closed-book free-form question answering on TriviaQA and CoQA.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Using unmodified models without additional training or data collection makes semantic entropy easier to deploy and improves reproducibility.

    Source: §1 · p. 2, §7 · p. 9

    Semantic entropy only partially addresses unequal token importance and still assigns too much importance to non-keyword likelihoods.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measuring uncertainty in natural language is challenging because of "semantic equivalence" -- different sentences can mean the same thing. To overcome these challenges we introduce semantic entropy -- an entropy which incorporates linguistic invariances created by shared meanings. Our method is unsupervised, uses only a single model, and requires no modifications to off-the-shelf language models. In comprehensive ablation studies we show that the semantic entropy is more predictive of model accuracy on question answering data sets than comparable baselines.

1 INTRODUCTION

Semantic entropy measures uncertainty over meanings rather than wording by grouping semantically equivalent generations. It outperforms comparable uncertainty baselines in free-form question answering without additional training or model modifications.

1 INTRODUCTION

  • Reliable uncertainty estimates are needed to trust generated information, but free-form generation poses challenges distinct from image classification and low-dimensional regression.Source: §1 · p. 1
  • Semantic entropy clusters mutually inferable sequences, assigns probabilities to meanings, and computes entropy over these meanings rather than over surface forms.Token likelihoods express lexical confidence: uncertainty between differently worded but equivalent answers need not represent uncertainty about meaning.Source: §1 · p. 1
  • Entropy estimation requires balancing sampled answers’ accuracy and diversity, strengthening the baselines used for comparison.Hyperparameter ablations indicate that far fewer samples suffice than prior work assumes; the analysis also examines length-normalising predictive entropies.Source: §1 · pp. 1–2
  • Semantic entropy outperforms comparable uncertainty baselines on TriviaQA and CoQA in open- and closed-book free-form question answering.It improves over standard entropy and model self-evaluation (Kadavath et al., 2022), with advantages increasing with model size and better use of additional samples.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Free-form question answering offers a difficult, high-stakes evaluation setting where ground truth is easier to establish without expensive human evaluation than for summarisation.Source: §1 · p. 2
  • The unsupervised method requires no additional training, data gathering, or model modifications and works with existing foundation and large language models.Experiments use OPT, but the method is designed to work with similar models.Source: §1 · p. 2

2 BACKGROUND ON UNCERTAINTY ESTIMATION

Predictive entropy measures total output uncertainty, while aleatoric and epistemic uncertainty distinguish uncertainty in the data from missing information. The paper uses samples from a single off-the-shelf model rather than ensembles or mutual-information estimation.

2 BACKGROUND ON UNCERTAINTY ESTIMATION

  • Predictive entropy measures total prediction uncertainty and is highest when all possible outputs have equal probability.It is the conditional entropy of the output given the input.Source: §2 · p. 2
  • Aleatoric uncertainty concerns the underlying data distribution, whereas epistemic uncertainty reflects missing information.Epistemic uncertainty can be measured using mutual information, but estimating it requires special methods and computational expense, especially for very large models.Source: §2 · pp. 2–3
  • The method samples one off-the-shelf model rather than ensembles (Malinin & Gales, 2020), without special training or architectural modifications.Alternatives predict epistemic uncertainty using a second model (Jain et al. (2021)) or emulate ensembles through prior networks (Malinin & Gales, 2018; Malinin et al., 2020). The latter could matter for NLG because models are large.Source: §2 · pp. 2–3
  • Sequence probabilities multiply conditional token probabilities; geometric-mean token probabilities are sometimes used instead of entropy (Malinin & Gales, 2020).Taking logs converts the sequence product into a sum and the geometric mean into an arithmetic mean. Despite empirical success reported by Murray & Chiang (2018), this normalization has little theoretical justification.Source: §2 · p. 3

Direct application of language models to uncertainty.

  • Direct model-based uncertainty approaches require task-specific labels and additional training and appear unreliable out of distribution (Lin et al. (2022a); Kadavath et al. (2022)).Lin et al. (2022a) finetune models to verbalize confidence; Kadavath et al. (2022) sample generations and prompt the model to assess whether an answer is true. Both also propose finetuning predictors on generating-model embeddings.Source: §2 · p. 3

3 CHALLENGES IN UNCERTAINTY ESTIMATION FOR NLG

Uncertainty estimation for free-form language must account for shared meanings, difficult sampling over a vast output space, and variable generation lengths. The paper motivates an unsupervised probabilistic approach that recognizes these language-specific challenges.

3 CHALLENGES IN UNCERTAINTY ESTIMATION FOR NLG

  • The proposed unsupervised approach combines probabilistic modelling with language-specific considerations, unlike black-box checks or probabilistic measures that ignore language's special characteristics.The section critically examines the probabilistic interpretation of language models to ground the method and future exploration.Source: §3 · p. 3

3.1 SEMANTIC EQUIVALENCE IN LANGUAGE OUTPUTS

  • Different language outputs can share a meaning, so reliability depends on consistency of semantic content rather than identical wording.Semantic content differs from syntactic structure and lexical form; meaning is usually an output-space invariance absent from the model specification.Source: §3.1 · p. 3
  • Semantic equivalence classes group sentences sharing a meaning, shifting attention from individual token sequences to the probability of generating any sequence with that meaning.The equivalence relation is reflexive, symmetric, and transitive. Theories of meaning remain contested, requiring care when comparing very different models and deployment contexts.Source: §3.1 · pp. 3–4

3.2 SAMPLING THE EXTREMELY HIGH-DIMENSIONAL LANGUAGE-SPACE

  • An O(|T|^N) output space complicates predictive entropy estimation, whose Monte Carlo approximation is dominated by low-probability sentences.Although autoregressive models support sampling, the paper notes the lack of a normalized probability density function over sentences.Source: §3.2 · p. 4

3.3 VARIABLE LENGTH GENERATIONS

  • Length normalization assumes expected uncertainty is independent of sentence length, but longer answers can genuinely be more uncertain when matching short references.Joint likelihood shrinks exponentially with length, making negative log-probability grow linearly and longer sentences contribute more to entropy. For short-reference tasks such as TriviaQA, normalization's advantages are therefore less clear-cut.Source: §3.3 · p. 4

4 SEMANTIC UNCERTAINTY

Semantic entropy measures uncertainty over meanings rather than exact token sequences by sampling model outputs, clustering semantically equivalent answers, and pooling their probabilities before estimating entropy.

4 SEMANTIC UNCERTAINTY

  • Generation samples M sequences from a single model's predictive distribution using multinomial sampling or multinomial beam sampling.Sampling temperature and sampling method can significantly affect the performance of both semantic entropy and the baselines.Source: §4 · p. 5
  • Semantic equivalence requires bidirectional entailment between context–answer pairs: an NLI classifier must predict entailment in both directions.Context allows short and full answers to count as equivalent. The implementation uses Deberta-large (He et al., 2020a), fine-tuned on MNLI (Williams et al., 2017).Source: §4 · p. 5
  • 92.7% accuracy on TriviaQA and 95.5% on CoQA validated the bidirectional entailment classifier against manual semantic-equivalence labels for 300 generations.Source: §4 · p. 5
  • Despite combinatorial worst-case comparison costs, clustering is practically inexpensive relative to generation, aided by small sample counts, faster comparisons, and transitivity.M < 20 is often sufficient for good uncertainty estimates, and observed cluster counts are low. Transitivity permits comparing only one representative per equivalence class against remaining sequences.Source: §4 · p. 5
  • Semantic entropy pools likelihoods of semantically equivalent sequences and estimates entropy over meanings using Monte Carlo integration over sampled equivalence classes.The estimator is unbiased; some cases additionally use length-normalisation.Source: §4 · pp. 4–6

4.1 HOW THE SEMANTIC ENTROPY ADDRESSES THE CHALLENGES OF NLG

  • Pooling equivalent generations partially addresses unequal token importance, but reduced sensitivity to unimportant tokens remains empirically unverified and non-keyword likelihoods still receive excessive weight.Supervised language-model-based uncertainty tools (Lin et al., 2022a; Kadavath et al., 2022) are suggested as possible improvements.Source: §4.1 · p. 6

5 RELATED WORK

Prior uncertainty research emphasizes classifier and regressor calibration or confidence methods requiring additional training, whereas this paper uses an unsupervised, off-the-shelf approach. Semantic equivalence connects uncertainty estimation to natural-language generation evaluation, paraphrase identification, and textual entailment.

5 RELATED WORK

  • Generative language introduces semantic-equivalence challenges beyond classifier and regressor calibration, making calibration problematic for evaluating generative models.Conventional uncertainty measures such as MC dropout and Deep Ensembles apply unchanged to classification and regression. Jiang et al. (2021) find only a weak correlation between answer log-likelihood and correctness in generative question answering.Source: §5 · p. 6
  • The paper’s unsupervised, off-the-shelf approach avoids the additional training and supervision required by earlier confidence-estimation methods.Methods involving prompted self-evaluation or fine-tuned confidence prediction require additional training and supervision (Mielke et al., 2020; Lin et al., 2022a; Kadavath et al., 2022). The passage identifies expense, reproduction difficulties, and sensitivity to distribution shift as limitations.Source: §5 · p. 6
  • Multiple valid translations complicate machine-translation evaluation (Ott et al., 2018), while paraphrase detection supports natural-language generation evaluation.The passage suggests that related evaluation methods might transfer to uncertainty estimation because the two problems share difficulties.Source: §5 · p. 6
  • Paraphrase identification uses lexical features, embedding similarity, and BERT-based classification; textual entailment offers a longstanding formalization of semantic equivalence.Lexical methods and embedding methods (Yu et al., 2014) contrast with recent state-of-the-art BERT-based classifiers (He et al., 2020b; Tay et al., 2021). Entailment-based semantic equivalence has roots in linguistics and NLP (Pad´o et al., 2009; Androutsopoulos & Malakasiotis, 2010).Source: §5 · p. 6

6 EMPIRICAL EVALUATION

Semantic entropy predicts answer correctness better than the evaluated baselines on TriviaQA and CoQA, with stronger advantages at larger model sizes. Its advantage grows with additional samples, while an intermediate sampling temperature balances answer accuracy and diversity.

6 EMPIRICAL EVALUATION

  • AUROC evaluates correctness discrimination without requiring the intractable probability mass over every equivalent correct phrasing needed for Brier scoring.Entropy can instead be estimated through Monte Carlo integration because it is structured as an expected information.Source: §6 · p. 7
  • 30B-parameter headline experiments use a single unmodified, pretrained OPT model for open-book CoQA and closed-book TriviaQA, without fine-tuning or ensembling.Evaluation uses approximately 8,000 CoQA development questions and 8,000 TriviaQA training questions. Answers count as correct when reference Rouge-L exceeds 0.3, with robustness reported under exact matching and Rouge-1.Source: §6 · p. 7

6.1 SEMANTIC ENTROPY UNCERTAINTY

  • Incorrectly answered questions produce more semantically distinct answer clusters, but semantic entropy predicts correctness better than cluster counts alone, especially on CoQA.Each question receives 10 sampled answers, with substantial overlap in meaning; CoQA exhibits greater spread in predicted probabilities between candidate answers.Source: Table 2 · p. 8, §6.1 · p. 8
  • Semantic entropy outperforms predictive entropy, length-normalised entropy, lexical similarity, and p(True) in predicting answer correctness on TriviaQA and CoQA, especially with larger models.The p(True) comparison uses OPT models up to 30B parameters and 10-shot prompting, whereas Kadavath et al. (2022) use a proprietary 52B model and 20-shot prompting.Source: §6.1 · p. 8, §6 · p. 8
  • 50.6% accuracy on TriviaQA contrasts with 82.3% on CoQA; length-normalisation has a large effect on CoQA but none on TriviaQA.TriviaQA references are generally single words or short phrases, whereas CoQA references include both longer and shorter answers.Source: §6.1 · p. 8
  • Additional samples widen semantic entropy’s advantage over length-normalised entropy on both datasets because it handles duplicated meanings better.The widening performance gap indicates that better use of additional samples accounts for much of the improvement.Source: §6.1 · p. 8, Figure 3 · p. 9

6.2 HYPERPARAMETERS FOR EFFECTIVE SAMPLING

  • 0.5 sampling temperature maximises AUROC for semantic entropy and length-normalised entropy on TriviaQA by balancing sample diversity against answer accuracy.Lower temperatures improve accuracy, while higher temperatures improve diversity. The authors find that using 1.0, as in Kadavath et al. (2022), would significantly weaken the predictive-entropy baseline.Source: §6.2 · p. 9, Figure 3 · p. 9

7 DISCUSSION

Semantic entropy measures uncertainty over meanings rather than token sequences and is more predictive of question-answering accuracy than strong baselines. The discussion highlights its unsupervised deployment advantages, lack of protection against deception, and potential extensions beyond question answering.

7 DISCUSSION

  • Semantic entropy accounts for equivalent meanings across distinct token sequences and predicts question-answering accuracy better than strong baselines.The unsupervised, out-of-the-box approach improves reproducibility and ease of deployment. It may also help address distribution-shift difficulties observed for supervised uncertainty measures.Source: §7 · p. 9
  • Bidirectional entailment clustering uses a smaller natural language inference model, positioning semantic entropy between fully probabilistic methods and language-model-based approaches.The authors favor combining model internals with predictions over relying on either perspective alone, particularly as model capabilities improve. However, their method does not protect against deception.Source: §7 · p. 9
  • Question answering offers relatively clear ground truths; extending semantic equivalence to summarisation likely requires better paraphrase identification and more human evaluation of correctness.Summarisation is a proposed future application rather than a demonstrated result.Source: §7 · p. 9
  • Semantic likelihoods could extend other probabilistic uncertainty tools, including mutual information, potentially enabling new strategies for uncertainty estimation in natural language generation.Source: §7 · p. 9

ETHICS STATEMENT … A.2 FURTHER ALGORITHMIC DETAILS

The work provides an open-source uncertainty pipeline for off-the-shelf language models, while acknowledging risks from misleading confidence and imperfect semantic-equivalence judgments. It recommends extensive deployment-specific evaluation of uncertainty communication.

ETHICS STATEMENT

  • Uncertainty estimates could help mitigate harmful misinformation, but inaccurate assessment or communication could create misplaced confidence.Before deployment, practitioners should conduct extensive context-specific evaluations to ensure uncertainty communication empowers users rather than misleading or confusing them.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The uncertainty pipeline uses publicly available OPT models and entirely open-source tools, requiring no foundation-model training or finetuning.The authors aim to make uncertainty research more accessible to academic groups and easier to replicate.Source: REPRODUCIBILITY STATEMENT · p. 10
  • All code and the hand-labelled semantic-equivalence dataset drawn from TriviaQA and CoQA are released under an MIT license.These artifacts support replication of the work.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 FURTHER DISCUSSION OF SEMANTIC EQUIVALENCE

  • Soft, sometimes erroneous equivalence judgments can break transitivity, making semantic clusters depend on comparison order.Consequently, a unique set of equivalence classes is not guaranteed. The authors report no noticeable practical problem but identify improved clustering as future work.Source: §A.1 · p. 14
  • Semantic entropy partly accounts for unequal token importance, recognizing that some words matter more to meaning than others.Semantic equivalence concerns shared meaning despite potentially different symbols or grammatical forms. Lexical equivalence requires identical symbols, whereas syntactic equivalence requires the same grammatical form.Source: §A.1 · p. 14

A.2 FURTHER ALGORITHMIC DETAILS

  • Algorithm 1 supplements the main method description with pseudocode for the bidirectional entailment algorithm.The implementation's soft equivalence judgments can introduce classification errors and nontransitivity.Source: §A.2 · p. 14, §A.1 · p. 14

A.3 IMPACT OF SAMPLING METHOD ON QUALITY OF UNCERTAINTY ESTIMATE

Sampling choices affect entropy-based uncertainty estimation: multinomial beam search produces less predictive estimates than multinomial sampling, and CoQA’s optimal temperature is 0.5. At high temperatures, semantic clustering becomes unreliable, while length-normalised entropy outperforms semantic entropy.

A.3 IMPACT OF SAMPLING METHOD ON QUALITY OF UNCERTAINTY ESTIMATE

  • The study focuses on temperature and compares sampling methods, leaving detailed investigation of other sampling hyperparameters, including top-p and top-k, to future work.The authors identify sampling choices as an overlooked consideration in uncertainty estimation.Source: §A.3 · p. 14
  • 0.5 is the optimal temperature for both uncertainty methods on CoQA, matching TriviaQA despite CoQA’s lower accuracy.Higher temperatures increase diversity and reduce answer accuracy. Unlike TriviaQA, CoQA shows normalised entropy outperforming semantic entropy at high temperatures.Source: §A.3 · p. 14, Figure 4 · p. 15
  • 61% accuracy for semantic equivalence at T=1.5 contrasts with over 92% at lower temperatures, helping explain semantic entropy’s high-temperature underperformance.Manual inspection and labelling of 100 classifications at T=1.5 revealed many nonsensical combinations of context words. Their likelihood remained somewhat predictive of model accuracy, but semantic clustering became unreliable.Source: §A.3 · p. 15
  • Semantic entropy clearly outperforms the baselines at low temperatures, where generated answers are plausible and well formed.This finding reinforces the importance of appropriate sampling hyperparameters for entropy-based uncertainty estimation.Source: §A.3 · p. 15
  • Multinomial beam search produces less diverse answers and uncertainty estimates less predictive of model accuracy than default multinomial sampling.The authors conjecture that beam search concentrates too heavily on the most likely sequences; its diversity resembles the lowest-temperature setting.Source: §A.3 · pp. 14–15, Table 4 · p. 16

B EXPERIMENTAL DETAILS AND ABLATIONS

Experiments use open-source models, dataset-specific prompting, and separate procedures for answer generation and uncertainty sampling. CoQA ablations support using 10 sampled answers per question.

B EXPERIMENTAL DETAILS AND ABLATIONS

  • The implementation uses OPT and Deberta-large through HuggingFace Transformers, with open-source code and no proprietary models.The authors present this implementation choice as facilitating reproducibility.Source: B EXPERIMENTAL DETAILS AND ABLATIONS · p. 16
  • Evaluated answers use five-beam search with sampling enabled, while uncertainty answer sets default to multinomial sampling with one beam.Explicitly indicated beam-multinomial experiments instead use five beams. Automatic answer-accuracy labels were highly accurate compared with human evaluation.Source: B EXPERIMENTAL DETAILS AND ABLATIONS · p. 16, Table 5 · p. 17
  • 10 sampled answers per question were used for TriviaQA after larger answer sets showed no significant uncertainty-performance improvement on CoQA.CoQA tests used up to 20 samples with 2.7B, 6.7B, and 13B models; the 30B comparison used 10 samples on both datasets. TriviaQA accuracy-metric choice also had little effect on uncertainty-quality assessment.Source: B EXPERIMENTAL DETAILS AND ABLATIONS · p. 16, Table 6 · p. 17
  • CoQA uses zero-shot prompting, whereas accurate closed-book TriviaQA answers require few-shot prompting, implemented with 10 examples.CoQA prompts include the context paragraph and preceding conversational questions with reference answers. Generations are trimmed at observed question-prefix stopwords to remove continued question–answer patterns.Source: B EXPERIMENTAL DETAILS AND ABLATIONS · p. 16

B.1 RELIABILITY OF ACCURACY METRIC AS COMPARED TO HUMAN EVALUATION · B.2 TESTING THE BI-DIRECTIONAL ENTAILMENT CLASSIFIER

Manual evaluations support both the Rouge-L accuracy criterion and the bi-directional entailment classifier on CoQA and TriviaQA.

B.1 RELIABILITY OF ACCURACY METRIC AS COMPARED TO HUMAN EVALUATION

  • Automatic answer-accuracy labels showed high agreement with human ground-truth labels on both CoQA and TriviaQA.The supplied passage reports high agreement without numerical agreement scores.Source: §B.1 · p. 17
  • The accuracy criterion labels a generated answer correct when Rouge-L(y, y′) > 0.3 relative to reference answer y.This criterion underpins evaluation of how well uncertainty measures predict answer accuracy.Source: §B.1 · p. 17
  • 200 answers from the 30B-parameter model were manually evaluated on CoQA and TriviaQA to check the automatic accuracy criterion against human judgments.The validation addresses the difficulty of developing automatic free-form generation metrics that correlate with human evaluations.Source: §B.1 · p. 17

B.2 TESTING THE BI-DIRECTIONAL ENTAILMENT CLASSIFIER

  • 92.7% accuracy on TriviaQA and 95.3% on CoQA were achieved by the bi-directional entailment classifier when identifying answers with the same meaning.Ground truth came from manual semantic-equivalence labels.Source: §B.2 · p. 17
  • 300 samples per dataset from the 13B-parameter model were manually labeled for semantic equivalence to test the entailment classifier on TriviaQA and CoQA.These labels established whether the sampled answers meant the same thing.Source: §B.2 · p. 17
  • To their knowledge, the authors provide the first application of bi-directional entailment to identifying same-meaning answers in question answering.Identifying such answers is a core component of their approach.Source: §B.2 · p. 17

B.3 SENSITIVITY OF RESULTS TO ACCURACY METRIC … B.6 MARGIN-PROBABILITY BASELINE

Uncertainty assessments are largely insensitive to accuracy-metric choice, except for exact matching on CoQA, while larger models improve generation accuracy and semantic-entropy performance. Multinomial sampling improves margin probability, but it remains weaker than the other studied methods.

B.3 SENSITIVITY OF RESULTS TO ACCURACY METRIC

  • TriviaQA uncertainty-estimation AUROCs change little across accuracy metrics, even though exact matching substantially reduces measured generation accuracy.This supports the broader finding that uncertainty-method assessments are relatively insensitive to the accuracy metric.Source: §B.3 · p. 17
  • CoQA assessments are similarly robust across accuracy metrics, except that exact matching is too demanding for its longer, more variable answers.CoQA is an open-book question-answering task.Source: §B.3 · p. 17, Table 7 · p. 18

B.4 ACCURACY ABLATIONS WITH MODEL SIZE

  • Larger models improve generation accuracy on both QA datasets and improve semantic entropy’s uncertainty performance.Both accuracy and uncertainty performance diminish at the smallest model size.Source: Figure 5 · p. 18, §B.4 · p. 18

B.5 EXAMPLE P(TRUE) FORMAT

  • P(True) uses the next-token likelihood of True after sampled brainstormed answers, with 10 few-shot examples following Kadavath et al. (2022).The prompt asks whether a candidate answer is true or false; the cited authors report that zero-shot prompting works poorly.Source: §B.5 · p. 18

B.6 MARGIN-PROBABILITY BASELINE

  • Margin probability improves with multinomial sampling instead of beam search but remains clearly outperformed by the other studied uncertainty methods.Its accuracy-prediction performance is poor on both CoQA and TriviaQA.Source: Figure 6 · p. 19, §B.6 · p. 19
  • Initially applying the Hmargin method proposed in Lin et al. (2022b) with a 13B-parameter model on CoQA yields an AUROC of 0.54, showing limited predictiveness of question-answering accuracy.Source: §B.6 · p. 19
  • Margin probability discards absolute likelihood and conflates paraphrase variation with semantic uncertainty; semantic entropy distinguishes these cases.The margin measures the likelihood difference between the two highest-ranked answers. The authors hypothesize that losing absolute likelihood and ignoring semantic equivalence contribute to poor performance.Source: §B.6 · p. 19, Table 8 · p. 19, Figure 6 · p. 19

Related papers

Loading 2302.09664v3…