Source-linked AI summary
Zipf's law and criticality in multivariate data without fine-tuning
David J. Schwab, Ilya Nemenman, Pankaj Mehta
TL;DR
The paper addresses why observed biological Zipf distributions can arise despite existing mechanisms often requiring fine-tuning or other restrictive conditions. It analyzes and simulates models in which unobserved variables affect the system, finding Zipf-like behavior without fine-tuning, though finite-size observation depends on how the system responds to those variables.
Problem
Many proposed mechanisms for biological Zipf distributions have restrictive conditions, are domain-specific, or require fine-tuning, leaving open how Zipf’s law arises in data-driven models.
Source: p. 1Method
The paper analyzes and simulates models where unobserved variables influence the system and the measured distribution averages over those variables.
Source: p. 1Results
Marginalizing over relevant hidden variables can produce Zipf’s law without fine-tuning or special hidden-variable statistics, with power-law behavior emerging in the thermodynamic limit.
Source: pp. 2–3Takeaways & Limitations
Hidden-variable effects offer a possible explanation for Zipf’s law in biological systems, alongside explanations based on tuning or self-organization.
Source: p. 4At modest system sizes, Zipf’s law requires sufficient adaptation to the hidden-variable statistics; poorly matched or weakly varying hidden variables may require very large systems.
Source: p. 5
Abstract
from arXiv · showhide
The joint probability distribution of many degrees of freedom in biological systems, such as firing patterns in neural networks or antibody sequence composition in zebrafish, often follow Zipf's law, where a power law is observed on a rank-frequency plot. This behavior has recently been shown to imply that these systems reside near to a unique critical point where the extensive parts of the entropy and energy are exactly equal. Here we show analytically, and via numerical simulations, that Zipf-like probability distributions arise naturally if there is an unobserved variable (or variables) that affects the system, e. g. for neural networks an input stimulus that causes individual neurons in the network to fire at time-varying rates. In statistics and machine learning, these models are called latent-variable or mixture models. Our model shows that no fine-tuning is required, i.e. Zipf's law arises generically without tuning parameters to a point, and gives insight into the ubiquity of Zipf's law in a wide range of systems.