PaperlayerView exact revision on arXiv

Source-linked AI summary

Test Martingales for bounded random variables

Harrie Hendriks

arXiv:2109.08923v1stat.ME

TL;DR

Auditors may need to bound average taint in a large population without determining every item’s value. The paper constructs sequential test supermartingales for mean hypotheses and examines their use in audit sampling; its discussion also notes that the resulting confidence interval can be empty.

  • Problem

    In audit sampling, determining all taint values in a large population is impractical, so an iid sample may be used to bound average taint.

    Source: §1 · p. 2

  • Method

    The paper uses positive test-supermartingale processes and the maximal lemma to test mean hypotheses through repeated observations.

    Source: §1 · p. 3, §2 · p. 3

  • Results

    The maximizing test parameter satisfies copt ≥ c0, while the audit discussion says Type I error may be considerably below α when μ is close to 1.

    Source: Dependence on the mean and variance of X · p. 13, §6 · p. 15

  • Takeaways & Limitations

    The framework provides a mathematical account of repeated random audit sampling while controlling the test’s significance level.

    Source: §1 · pp. 2–3

    The constructed confidence interval may be empty, with probability at most α (or α− + α+).

    Source: Confidence intervals · p. 20

Abstract

from arXiv · show

Given a positive random variable $X$, $X\ge0$ a.s., a null hypothesis $H_0:E(X)\le\mu$ and a random sample of infinite size of $X$, we construct test supermartingales for $H_0$, i.e. positive processes that are supermartingale if the null hypothesis is satisfied. We test hypothesis $H_0$ by testing the supermartingale hypothesis on a test supermartingale. We construct test supermartingales that lead to tests with power 1. We derive confidence lower bounds. For bounded random variables we extend the techniques to two-sided tests of $H_0:E(X)=\mu$ and to the construction of confidence intervals. In financial auditing random sampling is proposed as one of the possible techniques to gather enough evidence to justify rejection of the null hypothesis that there is a 'material' misstatement in a financial report. The goal of our work is to provide a mathematical context that could represent such process of gathering evidence by means of repeated random sampling, while ensuring an intended significance level.

1 Introduction

The paper develops sequential tests for a mean under nonnegativity and finite-expectation assumptions, studying their performance and their application to financial audit sampling. In that setting, repeated random sampling can estimate average taint without determining every item’s taint value.

1 Introduction

  • The paper builds on work connecting sequential probability-ratio tests with martingale theory, extending the discussion to hypotheses about the expectation of a nonnegative random variable.It defines an iid sample as independent observations sharing X’s distribution.Source: §1 · pp. 1–2
  • The paper’s novelty is analyzing test-supermartingale performance as a function of the mean and variance under alternatives and applying the method to financial audit sampling.It considers nonnegative random variables with finite expectation, rather than restricting the discussion to parameters in a specified distribution family.Source: §1 · pp. 1–2
  • In the audit example, taint is defined from book and audited values, and an iid sample can support an upper bound on average taint when the population is too large to inspect completely.The paper describes probability-proportional-to-size sampling and notes that missing or zero-book-value items are not addressed.Source: §1 · p. 2
  • The proposed framework constructs positive processes that are supermartingales under H0: E(X) ≤ μ, then uses them to test that hypothesis sequentially.The introduction also points to extensions for samples without replacement and stratified samples, and to confidence bounds and intervals.Source: §1 · p. 3

2 Supermartingales

The section uses the maximal lemma to turn a positive supermartingale into a sequential test with controlled rejection probability. A Bernoulli example relates a fixed-sample test to a process of conditional rejection probabilities.

2 Supermartingales

  • A positive supermartingale starting with expected value E(Y0) reaches level λ at some time with probability at most E(Y0)/λ.The proof stops the process at its first crossing of λ and uses the stopped process’s supermartingale property.Source: §2 · pp. 3–4
  • A test supermartingale yields a test that rejects H0 when Yk reaches E(Y0)/α, with rejection probability at most α when H0 holds.The passage defines test size as the probability of rejection under H0.Source: §2 · p. 4
  • For a Bernoulli null, conditional probabilities of a fixed-sample critical region form a test martingale, and the binomial-tail construction tracks the fixed-sample rejection event.In the construction, the event of at most k successes corresponds to Yn = 1; under the boundary null its probability is bounded by α.Source: §2 · p. 4

3 Test supermartingales for positive random variables

For nonnegative X with E(X) ≤ μ, the section constructs test supermartingales from factors whose expectation is at most one under the null. It also presents integrated test supermartingales as a way to combine choices of test parameter.

3 Test supermartingales for positive random variables

  • Under X ≥ 0 and E(X) ≤ μ, factors f with f(X) ≥ 0 and E(f(X)) ≤ 1 provide the basis for test supermartingales.The two-point-distribution argument motivates affine factors with parameter 0 ≤ c ≤ 1.Source: §3 · pp. 4–5
  • For two-point null and alternative distributions on {0, y}, the affine factor matches their likelihood ratio at both support points.The stated relationship applies for 0 < c < 1 and y > μ.Source: §3 · p. 5
  • The constructed process is a test supermartingale under H0, and rejecting when its running maximum reaches 1/α gives significance level at most α.The chosen test parameter must be determined from information available before the next observation.Source: §3 · p. 5
  • The capital process stakes a predictable fraction ck of current capital before observing Xk, so its update uses the resulting gain or loss.Changing the stake after observing Xk is ruled out by the gambling construction.Source: §3 · pp. 5–6

Choosing the test parameters, integrated test supermartingales

  • Integrating fixed-parameter test supermartingales against a probability density π produces an integrated test supermartingale over parameter choices.The updated parameter is the mean under a density proportional to the current process value times π; a finite weighted grid can be used numerically.Source: Choosing the test parameters,… · p. 6

Test supermartingales for the null hypothesis H0 : E(X) ≥µ or H0 : E(X) = µ

The section extends the testing construction to bounded variables and richer sampling schemes. It combines tests for opposite mean directions, while identifying parameter and stratum selection as unresolved in the described setting.

Test supermartingales for the null hypothesis H0 : E(X) ≥µ or H0 : E(X) = µ

  • For bounded X, transforming X to τ − X converts a lower-tail mean hypothesis into the nonnegative-variable construction, and combining opposite-direction processes gives a test for E(X) = μ.The equality test combines the two processes with nonnegative weights summing to one.Source: Test supermartingales for the null… · p. 6

Filtrations

  • For sampling without replacement, the null can be expressed through conditional expectations of successive observations given prior information.In the audit example, book-value-proportional sampling yields a random ordering and conditional mean constraints for taint.Source: Filtrations · p. 7
  • The stratified-sampling proposal uses known stratum probabilities and conditional means, with the stratum and test parameter chosen before each draw.The passage describes selecting a stratum and parameter at time k − 1 before sampling ωk.Source: Filtrations · p. 7
  • Under the stated filtration, the test process exceeds 1/α with probability at most α when the null holds.The passage also states this bound when E(X) = ν ≤ μ.Source: Filtrations · pp. 7–8
  • Choosing the next stratum and test parameters using conditional means, variances, or prior sampled items remains an open question in the described framework.The passage identifies optimization of these choices as unresolved.Source: Filtrations · p. 8

4 Alternative test supermartingales

The section develops alternative test supermartingales from moment-generating-function bounds and sub-Gaussian assumptions. It also distinguishes a supermartingale used to bound an upper-tail probability from a test supermartingale.

4 Alternative test supermartingales

  • For sub-Gaussian X with Gauss deviation at most τ and E(X) ≤ µ, the proposed factor f(X) is nonnegative and has expectation at most 1.The section notes that bounded X in [a, b] is sub-Gaussian with Gauss deviation at most (b − a)/2.Source: §4 · p. 8
  • For bounded X in [0, 1], the paper also describes factors related to Bernoulli likelihood ratios and a Gamma-density formulation for a Poisson-inspired testing method.The latter is connected to an auditing method described as based on the Poisson distribution in Audit Guide Audit Sampling.Source: §4 · p. 9
  • Hoeffding’s upper bound for the probability that the sample mean exceeds θ is reformulated and proved using a supermartingale.The construction uses iid observations in [0, 1] with mean µ and a threshold θ greater than µ.Source: §4 · p. 9
  • The supremum over h of the supermartingales used for the Hoeffding probability bound is not itself a test supermartingale.The text explicitly warns against treating this supremum as a test supermartingale.Source: §4 · p. 9

5 Behavior of the test supermartingales {Mk(c)}k

The section characterizes the growth of test supermartingales and uses it to establish consistency and assess expected sample size. It also bounds performance in terms of the mean and variance, including a worst-case distribution result.

5 Behavior of the test supermartingales {Mk(c)}k

  • λ(c) is twice continuously differentiable and concave on 0 < c < 1, with λ(0) = 0 and right derivative (E(X) − µ)/µ at 0.It is strictly concave when P{X = µ} < 1.Source: §5 · p. 10
  • If P{X = µ} < 1, λ(c) is strictly concave; under the null it is negative for 0 < c < 1, while under E(X) > µ it is positive for some such c.The corresponding Mn(c) converges to 0 under the null and diverges to infinity almost surely when λ(c) > 0.Source: §5 · pp. 10–11
  • An integrated test supermartingale is consistent, with power 1, when its mixing density assigns positive mass within every interval (0, b) for b < 1.Under E(X) > µ, the section first establishes divergence of Mn(c) for a range of c values.Source: §5 · p. 11

Performance depending on X

  • For the specified positive-integrable X and 0 < c < 1 with ω > 0, the stopping time Nα has finite expectation for every 0 < α < 1.The section derives this from a first-passage-time result.Source: Performance depending on X · pp. 11–12
  • log(1/α)/ω ≤ E(Nα) ≤ log(1/α)/ω + 1 + ρ²/ω².Here ω is the mean and ρ² the variance of the log factor; the bounds apply when ω > 0.Source: Performance depending on X · pp. 11–12

Dependence on the mean and variance of X

  • Among nonnegative X with mean θ > µ and variance at most σ², the two-point distribution on 0 and τ = (θ² + σ²)/θ is worst for the test-supermartingale performance considered.For this distribution, Theorem 8 gives c0 = (θ − µ)/(τ − µ) as the maximizing parameter; the general optimum copt is at least c0.Source: Dependence on the mean and variance of X · pp. 12–13
  • If X is bounded by γ and has mean ν ≥ θ > µ, the lower bound copt ≥ (θ − µ)/(γ − µ) is independent of Var(X).The section obtains this bound using Var(X) ≤ ν(γ − ν).Source: Dependence on the mean and variance of X · p. 13

6 Application in audit sampling

For bounded audit taints, the paper uses sequential test supermartingales to test whether mean misstatement reaches a tolerance level. It analyzes error control and sample requirements, including how test-parameter choices affect performance.

6 Application in audit sampling

  • The audit application assumes bounded T, with 0 ≤ T ≤ 1, and tests H0: E(T) ≥ µ after transforming a bounded variable to that scale.The paper identifies this as the setting closest to audit sampling.Source: §6 · p. 14
  • Rejection at the first time M_n(c) ≥ 1/α controls significance at α; under the stated positive log-growth condition, the test has power 1 and an approximate mean stopping-time bound log(1/α)/ω.For c = c0 under the Bernoulli model, ω is the KL divergence DKL(B(ν)∥B(µ)).Source: §6 · pp. 14–15
  • At the boundary E(T) = µ, Type I error is at most α but exceeds α(1−µ); it is near α for small µ and may be much smaller when µ is near 1.The theorem assumes 0 ≤ T ≤ 1 and P(T = µ) < 1.Source: §6 · p. 15

Average sample number

  • The simulations compare average sample numbers and standard deviations across T distributions and test choices, with fixed-c results reported as close to the approximations from the first-passage analysis.Each reported mean and standard deviation is based on 1,000 tests.Source: Average sample number · p. 15, Table 1 · p. 15
  • Except for Bernoulli T, integrating the test supermartingale over [(µ−ν)/µ, 1] outperforms the fixed choice c = (µ−ν)/µ; uniform integration over [0,1] is suggested when E(T) is otherwise unknown below µ.The paper compares this audit procedure with the Audit Guide’s sample-size table, which gives n = 162 for its chosen parameters; at E(T)=ν, the critical total nν is its expected sample sum, so success is close to 0.50 for large n.Source: Average sample number · p. 16
  • For θ < E(T) < µ, λ(c) can equal zero, in which case the expected decision time is infinite.The reciprocal test supermartingale is used for the alternative H1: E(T) ≤ θ, with θ = (1−c)µ.Source: Average sample number · pp. 16–17
  • Ignoring overshoots, A = (1−β)/α and B = β/(1−α) meet the stated error specifications for sequentially accepting or rejecting a lot.The procedure accepts when H0 is rejected at level α and rejects when H1 is rejected at level β.Source: Average sample number · p. 17

7 Confidence regions

The paper constructs confidence lower bounds from decreasing families of test supermartingales and develops test-supermartingale methods for confidence intervals. These procedures retain their stated confidence levels over repeated sampling, though an interval can be empty.

Confidence lower bounds

  • For bounded T, the paper also describes decreasing families that may produce a lower bound greater than E(T) − ∆.This is stated for 0 ≤ T ≤ 1 and 0 < ∆ < 1.Source: Confidence lower bounds · p. 18
  • A decreasing family of test supermartingales yields a (1 − α)-confidence lower bound by taking the largest bound observed so far.The sample may continue after a bound is obtained in pursuit of a better one without losing confidence.Source: Confidence lower bounds · p. 18
  • A sample initially used to test a fixed mean can be reused to construct a confidence lower bound when the maintained family agrees with the original test through the observed time.The paper relates rejection of the original null at level α to finding a (1 − α)-confidence lower bound no greater than the tested mean.Source: Confidence lower bounds · p. 19

Confidence intervals

  • For confidence intervals, the paper assumes 0 ≤ T ≤ 1 and tests hypotheses of the form H0: E(T) = µ.The target is an interval-valued confidence set for the mean.Source: Confidence intervals · p. 20
  • For convex families, the set of means whose test supermartingales stay below 1/α is a confidence interval.The paper also gives intervals formed from tests up to a given time or from the test at that time alone.Source: Confidence intervals · p. 20
  • Combining decreasing and increasing test-supermartingale families gives (1 − α− − α+)-confidence intervals.The lower and upper bounds use confidence levels 1 − α− and 1 − α+, respectively.Source: Confidence intervals · p. 20
  • The resulting confidence interval may be empty, with probability at most α or α− + α+.The paper suggests using a nonempty interval from an earlier time or another specified interval in that event.Source: Confidence intervals · p. 20
Loading 2109.08923v1…