Source-linked AI summary
Culture in the Code: Anthropological Concepts Decoding AI's Hidden Assumptions
Sonja Schmer-Galunder
TL;DR
AI alignment often treats values as measurable preferences inferred from behavior, despite the cultural and social contexts that give them meaning. The paper applies seven anthropological concepts to examine those assumptions. It argues that behavioral proxies, reductive metrics, and optimization can miss or undermine what matters in human domains.
Problem
Many alignment proposals seek more inclusive optimization without examining the assumptions behind AI systems or how social context makes observable behavior meaningful.
Source: §1 · p. 2Method
Results
The analysis argues that behavior requires contextual interpretation, measurable proxies can omit significant dimensions, and optimization may undermine values in human domains.
Source: §10 · p. 13Takeaways & Limitations
For relationships, meaning, identity, care, and flourishing, the paper cautions against assuming that valued ends are appropriate optimization targets.
Source: §10 · p. 13The argument compresses a chain in which objectives are proxies for goals that imperfectly represent values, and optimization pressure can decouple objectives from those values.
Source: §9.3 · p. 12
Abstract
from arXiv · showhide
Contemporary AI development operates on foundational assumptions about intelligence, rationality, and human values that reflect narrow cultural perspectives, limiting alternative conceptions of how AI can be globally beneficial. This paper introduces seven anthropological concepts -- origin story, monoculture, thick description, personhood and community, cargo cult, structuralism, and liminality -- to expose hidden assumptions and offer alternative frameworks for understanding AI alignment. The paper examines how AI systems embody Western-centric notions of rationality as utility optimization, personhood as bounded individuals, and values as quantifiable preferences extracted from behavior. The concepts surface three assumptions that current alignment debates rarely examine: that values can be inferred from behavior, quantified, and formalized; that such quantified values capture what actually matters; and that values, once defined with sufficient precision, are appropriate optimization targets. Against "thin alignment" -- optimization for behavioral proxies stripped of context -- the paper proposes directions toward "thick alignment": recognizing that values emerge through social practice rather than existing as extractable properties; defining more appropriate metrics that capture not only end goals but the conditions that lead to long-term flourishing; and moving from models of fixed identity toward relational ontologies where personhood emerges through social participation. The paper concludes that if AI systems are to create conditions conducive to human flourishing rather than optimize metrics that flatten social experiences, three shifts are needed: redistributing interpretive sovereignty to affected communities, protecting what resists optimization, and extending accountability across generations.
1 Introduction
The introduction argues that value alignment often treats Western values as universal and leaves the cultural assumptions behind AI systems unexamined. It turns to anthropology to question how values emerge, whose interpretations count, and what alternative frameworks can reveal.
1 Introduction
- Alignment framed around “human” values often means Western values, sidelining non-Western epistemologies, cosmologies, and value systems.The cited empirical work reports Western-centric biases and inaccurate representation of non-WEIRD demographics [6].Source: §1 · p. 1
- Many proposed alignment responses seek more inclusive optimization without examining the cultural assumptions that make observable behavior meaningful.The paper characterizes this tendency as computational solutionism.Source: §1 · p. 2
- AI systems assume values can be inferred from behavior and encoded into objectives, while treating people as rational agents with quantifiable preferences.The introduction also asks whether technical researchers should hold authority to define values for societies they affect.Source: §1 · p. 2
- Anthropology offers concepts shaped by its own reckoning with colonial practices of extracting meaning, imposing categories, and universalizing one cultural logic.The paper contrasts this disciplinary reckoning with technical alignment frameworks that have not begun a comparable one.Source: §1 · p. 2
- The paper frames values as emerging through collective social practice and asks what is lost when AI treats them as optimization targets.It warns that culturally biased systems entering social domains may damage the practices that make values meaningful.Source: §1 · p. 2
- Seven anthropological concepts provide a vocabulary for exposing assumptions in AI and imagining futures beyond dominant technical frameworks.The analysis applies them to contemporary AI practices and discourse, arguing that the issues are cultural and social as well as technical.Source: §1 · pp. 2–3
2 Origin story
The section treats AI origin stories as culturally particular narratives that shape authority and the futures people can imagine. It contrasts dominant technology myths with alternative traditions that could frame different relationships among people, machines, and society.
2 Origin story
- Silicon Valley narratives of obedient or hostile artificial beings persist in contemporary AI imaginaries and may help explain the doomer–accelerationist divide.Both camps accept the premise of inevitable artificial general intelligence, differing over whether it leads to extinction or flourishing.Source: §2 · p. 3
- Origin stories legitimize social order and authority by making arrangements appear natural; tech founders’ visionary narratives also charter authority.The section invokes Foucault’s “regimes of truth” to describe how such stories shape identities and constrain imaginable futures.Source: §2 · p. 3
- Anthropological examples show origin narratives authorizing present-day claims and duties rather than merely explaining the past.The section contrasts Genesis, Daoist cosmology, and Aboriginal songlines as narratives encoding different relations between creation and authority.Source: §2 · p. 3
- Silicon Valley’s origin stories are culturally particular, while feminist technoscience, Indigenous futurism, and Afrofuturism offer alternatives [20, 24–26].The section argues these traditions could open technological futures grounded in different relationships among humans, machines, and society.Source: §2 · p. 4
3 Monoculture
The monoculture section uses scientific forestry to illustrate how optimizing complex systems around narrow measures can erase sustaining diversity. It connects that pattern to AI’s dominant cultural defaults and argues for preserving plural technological paths.
3 Monoculture
- German scientific forestry increased timber production but removed undergrowth and habitats, with forest decline and costly interventions becoming visible only after generations.The redesigned forest became more vulnerable to disease and storms, while lost foliage and floral diversity degraded soil.Source: §3 · p. 4
- AI risks reproducing this pattern when one epistemological monoculture crowds out alternative knowledge traditions and makes dominant cultural standpoints seem universal.The section points to English dominance in training data as one way linguistic diversity is flattened and a cultural default reinforced.Source: §3 · pp. 4–5
- The paper characterizes AI monoculture as a systemic risk, drawing parallels with forestry’s narrow metrics, short-term optimization, and reduced diversity.It also argues that diverse systems show resilience across domains, including forests, microbiomes, portfolios, teams, and democracies.Source: §3 · p. 5
- Technodiversity names the variety of possible technological paths grounded in distinct cosmologies, rather than a rejection of technology.The section argues that technological ecosystems’ long-term health depends on maintaining conditions for genuine plurality.Source: §3 · p. 5
4 Thick description
The section contrasts thin descriptions, which impose decontextualized categories, with thick description, which interprets actions in their social and cultural contexts. It presents contextual understanding as an alternative to inferring values directly from behavioral signals.
4 Thick description
- Technical correction of AI stereotypes can still impose abstract, decontextualized categories without interpreting what they mean to the people involved.The section describes these patterns as structural features of systems absorbing and reproducing inequalities in training data [36].Source: §4 · p. 5
- Thick description interprets an act through intention and social meaning, while thin description records only its observable surface.The eye-wink example shows how the same muscle movement may be a twitch, a wink, or a parody.Source: §4 · p. 5
- Extended immersion, participation in daily life, and sustained observation produce layered accounts of what it is like to be part of a culture.The paper describes anthropological fieldwork as typically lasting 1–3 years and notes that this alternative may be impractical.Source: §4 · p. 5
- The Balinese cockfight’s meanings include gender roles, male identity, resistance to modernization, and social allegiance beyond the apparent wager.The example shows why a bet that looks economically irrational may be coherent in its cultural context.Source: §4 · p. 6
5 Personhood and community
AI datasets can encode culturally loaded judgments while treating personhood as a bounded, autonomous individual. Anthropological accounts instead understand personhood as relational, formed through exchange, recognition, and social context.
5 Personhood and community
- ImageNet’s person categories encode cultural and moral judgments, despite being treated as neutral taxonomies.Its 2,833 subcategories include labels such as “slattern” for a smiling woman in a bikini and “alcoholic” for a young man drinking beer.Source: §5 · p. 6
- Anthropological accounts describe personhood as relational: people become persons through exchange, recognition, and relation.Gift exchange can create kinship, establish status, and enact moral relationships that make community possible.Source: §5 · p. 6
- Relational accounts do not deny individual traits, biology, or autonomy, but hold that identity depends on social and cultural context.Source: §5 · p. 7
- AI data structures replace relational persons with abstract individuals, risking a loss of shared practices and genuine community.The passage contrasts these representations with “cloakroom communities,” temporary associations characterized by emotional engagement rather than ethical commitment.Source: §5 · p. 7
6 Cargo cult
The cargo-cult analogy frames AI interaction as a mismatch between visible forms and the substance that gives them meaning. AI can reproduce communication without care or intention, while people may interpret its outputs as signs of a social mind.
6 Cargo cult
- Chatbots can perform the surface features of intimacy without the capacity to care, as the paper illustrates with a teenager’s conversations with a Character.AI chatbot.The bot asked about his day and responded to his darkest thoughts with “I love you.”Source: §6 · p. 7
- The cargo-cult analogy draws on islanders who replicated observed military rituals, mistaking effects of an unseen system for causes of returning planes.They built bamboo control towers, carved wooden headphones, and cleared airstrips after the planes stopped coming.Source: §6 · p. 7
- The analogy operates on both sides: AI pattern-matches communication without consciousness, intention, or care, while users attribute psychological states and superhuman qualities to models.People may engage in “ritual prompting” or seek settings they think influence a model’s “mood.”Source: §6 · pp. 7–8
- People describe AI failures in psychological terms, including as “confused” or “sycophantic,” even while understanding models as statistical pattern-matching systems.Source: §6 · p. 8
- The paper argues that people project presence into AI outputs, experiencing care where there is only computation.It connects this tendency to “ethopoeia,” the automatic attribution of human characteristics to computers, and reports that people showed gender stereotypes and in-group favoritism toward their desktop computers in 1990 research.Source: §6 · p. 8
7 Structuralism
The paper traces AI’s use of universal categories to structuralist assumptions, arguing that classification can reproduce social hierarchies rather than neutrally describe people. It calls for systems that preserve human potentiality rather than treating past patterns as destiny.
7 Structuralism
- Gemini’s hidden instruction to generate diverse images across prompts produced historically incongruous images, illustrating how a technical correction can impose another bias.The episode also exposed foundational assumptions that categories such as race, gender, and emotion can be manipulated independent of context.Source: §7 · p. 8
- AI operationalizes assumptions about universal categories, from grammatical forms to discrete emotions read from facial configurations across cultural contexts.The paper connects these assumptions to structuralism and notes that such systems operate in hiring, border security, and classroom monitoring.Source: §7 · p. 8
- Classification systems can encode social hierarchies as neutral technical features, and policing algorithms may reproduce inequalities through self-fulfilling predictions.The paper describes algorithms that treat past behavioral patterns as predictive of future actions.Source: §7 · pp. 8–9
- The paper identifies epistemological determinism in systems that treat people as products of underlying structures, denying their agency to break from established patterns.Source: §7 · p. 9
- Rather than building more accurate predictive models, the paper argues for AI that preserves “potentiality”—the capacity to be otherwise than statistical patterns suggest.Source: §7 · p. 9
8 Liminality
Griefbots and generative AI blur boundaries between human and machine, authorship and instrument, and the living and the dead. The paper describes this as a liminal transition whose unsettled conditions may allow new social arrangements, but require conscious navigation.
8 Liminality
- Griefbots trained on deceased people’s digital footprints may cause distress through unwanted “digital hauntings,” and may leave grief suspended or unresolved.The paper describes them as commercial products that digitally recreate a deceased person’s voice and personality.Source: §8 · p. 9
- The griefbot blurs boundaries between organism and machine, self and other, and deceased person and digital recreation, complicating authorship and agency.The paper describes a recreated voice as neither of human origin nor simply machine.Source: §8 · p. 9
- Liminality is the middle phase of a rite of passage, when normal hierarchies dissolve, norms are suspended, and new social arrangements may emerge.The paper traces the concept from van Gennep’s three-stage account, expanded by Turner.Source: §8 · p. 9
- The paper characterizes the shift from a pre-AI to a post-AI world as a collective transition, with institutions changing before new forms have emerged.It describes old assumptions about authorship, identity, creativity, and knowledge as dissolving without a settled replacement.Source: §8 · p. 10
- The ambiguity of liminal spaces may allow new social arrangements, ways of thinking, and forms of expression, so the passage should be navigated consciously.The paper calls for attention to what should be preserved and what may be transformed.Source: §8 · p. 10
9 Toward alternative futures: beyond the values AI can see?
The paper uses anthropological concepts to question formalist-rationalist assumptions in AI and examines three assumptions about values: that they can be inferred and formalized, that metrics capture what matters, and that formalized values should be optimized.
9 Toward alternative futures: beyond the values AI can see?
- The section frames anthropological concepts as alternatives for questioning formalist-rationalist assumptions and identifies three assumptions about values embedded in AI.These are that values can be inferred from behavior and formalized, that quantified values capture what matters, and that sufficiently precise values are appropriate optimization targets.Source: §9 · p. 10
9.1 On the first assumption
- Even reliably inferred values cannot be made commensurable without categorical, relational, and temporal distortion.Places and people may be categorically different rather than equivalent utilities; acts gain significance through relationships that behavioral data alone cannot reveal.Source: §9.1 · pp. 10–11
- Values change over time, and metrics can arrest their development or encode past expressions made under conditions that no longer exist.The passage describes delayed reciprocity as constitutive of gift exchange and presents values as open and in formation.Source: §9.1 · p. 11
9.2 On the second assumption
- Metrics based on additive quantities may omit what matters in beauty, justice, love, and a tree’s ecological, cultural, and spiritual significance.The tree example contrasts timber’s economic value with ecological effects and cultural meaning; the passage also points to accountability across seven generations.Source: §9.2 · p. 11
9.3 On the third assumption
- AI’s optimization paradigm, inherited from control theory, treats formalized proxies as targets even though human flourishing resists formalization and the paradigm is not culturally neutral.The paper connects this assumption to value frameworks that Confucian, Ubuntu, and Indigenous relational traditions may not recognize as their own.Source: §9.3 · p. 12
- Optimization can detach a metric from the value it represents, overshoot contextual values, and foreclose the openness needed for values to remain alive.The passage describes metrics becoming targets, values having a proper measure rather than a maximum, and optimization closing off deliberation about what it means to be human.Source: §9.3 · p. 12
- AI systems optimize objectives built as proxies for goals, which are themselves imperfect representations of values.The footnote says the argument compresses this chain and notes that optimization pressure can decouple objectives from the values they are meant to serve.Source: §9.3 · p. 12
10 Conclusions
The paper argues that alignment is a cultural as well as a technical problem: anthropological concepts expose assumptions embedded in current approaches. It calls for a reorientation toward AI systems that leave room for values and human flourishing.
10 Conclusions
- The seven anthropological concepts expose assumptions that limit current AI alignment approaches, framing alignment as a cultural as well as technical problem.The paper presents anthropology as offering conceptual tools that technical frameworks cannot supply.Source: §10 · p. 13
- AI systems that infer universal values from behavior perform thin description at civilizational scale.Acts such as gifts, votes, and purchases do not carry their meanings on the surface; thick description attends to the motivations behind visible behavior.Source: §10 · p. 13
- Optimizing reductive metrics can degrade unmeasured dimensions, while some important things resist concatenation.The paper uses a forest reduced to timber yield to illustrate the loss of soil health, animal species, and cultural meaning.Source: §10 · p. 13
- Moving from valuing something to optimizing for it is a cultural choice, and human domains such as care and flourishing differ from operational domains where optimization is powerful.The paper says optimization in human domains can undermine what it claims to serve; it presents this as an inherited origin story, not a tested hypothesis.Source: §10 · p. 13
- Anthropology does not solve alignment; the paper calls for reorienting the question toward systems that leave values living, contested, and human.Whether AI creates conditions conducive to flourishing depends on seeing past hidden assumptions and envisioning more communal, sustainable, and humane systems.Source: §10 · p. 13