Source-linked AI summary
Power-law Distributions in Information Science - Making the Case for Logarithmic Binning
Staša Milojević
TL;DR
Power-law analysis in information science is complicated by noisy tails, discrete data, deviations from pure power laws, and misleading cumulative representations. The paper proposes partial logarithmic binning, supports least-squares fitting on binned data in some cases, and cautions against cumulative distributions when they obscure noise and distort exponent estimation.
Problem
Information-science distributions often have noisy tails and non-ideal or discrete forms, making power-law trends and exponents difficult to identify accurately.
Method
The paper recommends partial logarithmic binning and evaluates least-squares exponent fitting on binned data alongside maximum-likelihood methods.
Results
The method retrieves tail trends and can recover an exponent of a = 2.01 from empirical data whose true value is 2.
Takeaways & Limitations
Logarithmic binning is recommended for exploring IS power-law distributions and estimating exponents, especially where tail noise is substantial.
Takeaways & Limitations
The paper does not determine whether data are actually drawn from a pure power law, and cumulative distributions can obscure tail noise and bias exponent assessment.
Abstract
from arXiv · showhide
We suggest partial logarithmic binning as the method of choice for uncovering the nature of many distributions encountered in information science (IS). Logarithmic binning retrieves information and trends "not visible" in noisy power-law tails. We also argue that obtaining the exponent from logarithmically binned data using a simple least square method is in some cases warranted in addition to methods such as the maximum likelihood. We also show why often used cumulative distributions can make it difficult to distinguish noise from genuine features, and make it difficult to obtain an accurate power-law exponent of the underlying distribution. The treatment is non-technical, aimed at IS researchers with little or no background in mathematics.
Introduction
Power-law distributions are common across information science, but their forms and interpretation vary. The paper introduces logarithmic binning as an accessible way to reveal power-law trends in noisy data.
- Power-law distributions describe diverse IS phenomena, including author productivity, citations, literature scattering, and collaborative tagging.
- The paper focuses on logarithmic binning to help IS researchers recognize whether power laws are present in their data.
- Logarithmic binning averages values within equal-sized logarithmic ranges to reduce statistical noise and reveal trends.
- Power-law functions are monotonic and are commonly represented using an exponent and an overall scale or normalization.
- In power-law distributions, positive exponents imply that larger values of x correspond to smaller y values, making high-value events relatively rare.
Power-law distributions in IS
IS power-law data are discrete, noisy, and often depart from ideal mathematical forms. The paper explains these deviations and recommends partial logarithmic binning to recover underlying trends, especially in distribution tails.
- IS distributions may follow Lotka, Zipf, or Bradford forms, which are equivalent power-law distributions expressed in size-frequency, ranked, or cumulative form.
- Deviations from a power law: Empirical power laws can systematically deviate from pure power laws through multiple exponents, smooth transitions, exponential cutoffs, or log-normal components.
- Discrete vs. continuous values: IS power laws are discrete because x typically takes natural-number values, while empirical y values are also non-negative integers subject to probabilistic variation.
- Retrieving information from noise: Partial logarithmic binning averages observations only above a chosen threshold, preserving small-k detail while reducing noise in the tail.
- Retrieving information from noise: Fitting logarithmically binned data reduces tail-noise bias and improves recovery of the underlying power-law exponent without discarding tail data.
Binning procedure
Partial logarithmic binning averages noisy large-k data while leaving small-k values unbinned, revealing trends that plain distributions obscure. The procedure uses logarithmic bin widths chosen to reduce noise without oversmoothing, and simulations show it can recover hidden power-law structure.
- Use unbinned data for small k and begin logarithmic binning where statistical noise becomes significant, with a typical bin size of 0.1 decades.A recommended bin-size range is 0.05–0.2 decades.
- For smaller datasets, binning can begin when successive logarithmic k values are closer than the chosen bin width, such as k ≥ 5 for 0.1-decade bins.This criterion applies when datasets contain fewer than about 10,000 points and statistical error remains above the noise threshold.
- Each logarithmic bin’s value is the average of all y_k values whose integer k falls inside it, including zero values and all integer positions in the final bin.The bin count n includes integers beyond the highest observed k in the last bin.
- In the NanoBank example, binning keeps the author-productivity tail smooth beyond k > 50 and shows the expected power-law decline continuing at large k.The binned trend is slightly curved; a quadratic log-log fit describes it better than a linear fit, with a Pareto type 2 fit also reasonably good.
- Binned points reveal that the simulated double power law, hidden by tail noise, changes exponent from 2 to 1.1 at k = 50.Logarithmic binning starts at k = 6 with 0.1-decade bins and removes most of the noise.
Extracting the exponent
Exponent estimates are sensitive to noise, distribution discreteness, and the fitting method. Logarithmic binning can recover a useful overall exponent, whereas cumulative representations may obscure deviations and amplify apparent tail trends.
- Extracting the exponent: a = 2.10 from logarithmically binned data closely matches the expected a = 2, while unweighted fitting of all unbinned points gives 1.31.The unbinned estimate is biased by noisy tail points and has a reported statistical error of 0.15; the binned estimate has error 0.23.
- Extracting the exponent: For discrete distributions, the continuous maximum-likelihood formula gives the incorrect exponent 2.8, whereas discrete maximum likelihood implemented in LOTKA gives a = 2.01.Discrete maximum likelihood requires numerical integration or lookup tables rather than the simple continuous formula.
- Extracting the exponent: When the underlying distribution combines exponents 2 and 1.1, logarithmically binned least squares gives 1.49 across the range, while maximum likelihood gives 1.84 and diverges in the far tail.The preferred method depends on the question, so binned least squares can complement maximum likelihood when a true power law is uncertain.
- Extracting the exponent: Least-square fitting should not be applied to unbinned data because noisy tails can produce a seriously distorted exponent of 0.55.The warning applies specifically to unbinned data from the double power-law example.
- Logarithmic binning versus cumulative distribution: Cumulative representations can hide tail noise and make it difficult to distinguish genuine features from artifacts, despite their smooth appearance.In the cumulative power-law example, the apparent departure begins around k = 30, is difficult to see before k = 100, and yields a least-squares exponent of 1.23.
- Logarithmic binning versus cumulative distribution: For a cumulative double power law, later points affect earlier ones, obscuring the underlying two-regime structure and producing a non-real downward tail trend.The distortion occurs even where the non-cumulative data show clear power-law behavior for k < 30.
Conclusions
The paper recommends logarithmic binning to reveal functional forms and trends hidden by noisy power-law tails. It also cautions that cumulative distributions may be unsuitable for discrete or non-perfect power-law data.
- Logarithmic binning helps researchers explore power-law distributions, especially their noisy tails.
- Binned data can support unbiased exponent estimation with least squares without discarding tail observations.
- Cumulative distributions may be unsuitable for discrete data or distributions that are not perfect power laws.