PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolutionary Recurrent Decision Model in Developing Adaptive and Maladaptive Behaviors

Andrew Hu

arXiv:2608.23932v1cs.AI

TL;DR

Understanding underlying cognitive processes remains difficult, motivating computational approaches that model cognition as adaptive and environment-dependent. This paper introduces ERDM, which produces diverse stable strategies and suggests that behaviors such as learned helplessness, aggression, and avoidance can emerge as predictable outcomes of internal competing-policy learning.

  • Problem

    Underlying cognitive processes remain difficult to observe, motivating computational approaches that model cognition as adaptive and environment-dependent.

  • Method

    ERDM models cognition through internal competing-policy learning that produces satisficing in adaptive, environment-dependent settings.

  • Results

    ERDM produced diverse, stable strategies despite learning convergence, including learned helplessness, aggression, and avoidance or withdrawal.

  • Takeaways & Limitations

    The findings suggest that learned helplessness, aggression, and avoidance may be predictable outcomes of internal competing-policy learning rather than pure intrinsic dysfunction.

  • Takeaways & Limitations

    The study does not include accompanying experimental validation of the model’s unobservable underlying cognitive mechanisms.

Abstract

from arXiv · show

This study introduces the evolutionarily recurrent decision model (ERDM), a computational reinforcement learning framework designed to examine how evolutionary mismatch, bounded rationality, and satisficing contribute to adaptive and maladaptive behavior. ERDM simulates agents across evolutionary recurrent environments, including threat, prey/goal-pursuits, and alliances. Agents learn through competing rewards abstracted from survival metrics. A validity study under varying adverse childhood experiences demonstrates that distinct adaptive and maladaptive strategies, such as learned helplessness, avoidance, healthy relationships, and aggression, emerge naturally without being hardwired. These results align with empirical literature, showcasing ecological validity. The results suggest that many psychopathology-relevant aspects may be interpreted as bounded cognitive systems operating under modern-ancestral environmental mismatch, positioning ERDM as a key computational cognitive tool that can be extended to other studies.

I. INTRODUCTION

The study introduces ERDM to examine how bounded rationality and satisficing under evolutionary mismatch may produce adaptive and maladaptive behaviors. It uses simulated ancestral and modern-mismatched environments, including varying adverse childhood experiences, to evaluate ecological and theoretical validity against empirical behavioral patterns.

  • Motivation: The framework represents cognition as an adaptive, environment-dependent process shaped by interaction with environmental constraints, addressing cognitive processes that are difficult to observe directly,,,.ERDM is positioned as a computational cognitive approach for studying how ancestral adaptations may generate maladaptive outcomes in modern conditions.
  • Model contribution: ERDM models bounded rationality under competing recurrent survival pressures, allowing context-dependent adaptive and maladaptive strategies to emerge without a universally optimal policy.The model treats survival objectives as inherently conflicting and examines how behavioral strategies change across environmental conditions.
  • Validation design: The validation study compares stable, moderate-adversity, and high-adversity ACE environments with empirical findings on reward processing, perceived control, social behavior, and mental health,,.These comparisons test whether evolutionary mismatch, bounded rationality, and satisficing can generate behavioral patterns relevant to psychopathology and support ecological validity.

II. METHODS · A. Agent and Ecological Architecture

ERDM models agents that repeatedly encounter evolutionarily recurrent ecological problems, select actions, receive fitness-relevant rewards, and update reinforcement-learning policies. Its architecture uses independent action-specific value networks with epsilon-greedy exploration to generate and analyze emergent behavioral strategies.

  • II. METHODS: ERDM agents iteratively interact with evolutionarily recurrent environments, choose actions, receive fitness-relevant rewards, and update internal policies through reinforcement learning.Behavioral patterns are analyzed together with contextual information to categorize complex emergent strategies.
  • II. METHODS: The framework organizes each cycle as environmental recurrence, decision selection, evaluative reward receipt, policy updating, and progression to the next environment.Figure 1 also represents behavioral patterns and reward expectations as organized strategies.
  • A. Agent and Ecological Architecture: The agent is a reinforcement-learning system composed of multiple independent value neural networks, each corresponding to a behavioral action pathway.ERDM was implemented in Python using PyTorch and NumPy.
  • A. Agent and Ecological Architecture: The stochastic environment represents discrete ecological events corresponding to evolutionarily recurrent adaptive problems involving threat, prey, and alliance opportunities.These ecological events are illustrated in Figure 1.
  • A. Agent and Ecological Architecture: Each action-specific network receives internal and environmental states and outputs a scalar predicted reward for associating that action with the current situation.For action ai in state s, Vi(s) represents the expected reward, with networks parameterized by weights θi.
  • A. Agent and Ecological Architecture: The agent selects the action whose action network predicts the highest reward.This greedy choice is based on the networks’ predicted action values.
  • A. Agent and Ecological Architecture: An epsilon-greedy policy balances exploration and exploitation: probability 𝜖 selects a random action, whereas probability 1 – 𝜖 selects the greedy action a*.The policy therefore introduces stochastic action selection alongside value-maximizing behavior.

B. Neural Network Structure · C. Action Selection and Reward Functions

ERDM uses uniform two-hidden-layer feedforward action networks to predict rewards, while context-sensitive competing rewards generate emergent adaptive and maladaptive behaviors. Its three survival dimensions impose tradeoffs that promote satisficing rather than universally optimal or fixed policies.

  • B. Neural Network Structure: Each emotional and behavioral action network uses a feedforward architecture with two hidden layers and a one-unit predicted-reward output.The hidden layers contain 128 and 64 ReLU units, respectively, while the input size matches the state representation.
  • B. Neural Network Structure: The shared network architecture models how simple cognitive mechanisms calibrated to ancestral pressures can produce complex behaviors without explicitly encoding them.Complex behaviors emerge from repeated simple actions within context rather than from hard-coded behavioral patterns.
  • C. Action Selection and Reward Functions: ERDM evaluates each situation-action pair through livelihood/physical fitness (L), defensive belonging/social value (DB), and nurturing belonging/social value (NB) reward functions.The two social values are abstracted from evolved dominance and prestige strategies.
  • C. Action Selection and Reward Functions: Rewards depend on environmental context, agent state, and action, with L capturing physical well-being, DB dominance-related threat neutralization, and NB prestige-related benefits [11],.These rewards are derived from the causal consequences of situation-action results.
  • C. Action Selection and Reward Functions: A single action can produce competing survival rewards; fighting a threat may increase defensive belonging while reducing livelihood and nurturing belonging.The example illustrates how one action creates cross-dimensional tradeoffs rather than a single uniformly positive outcome.
  • C. Action Selection and Reward Functions: Stochastic environments, state-dependent rewards, and competing objectives prevent consistent global optimization or memorized fixed policies, requiring locally sufficient satisficing strategies.No single action is universally optimal across all states or within a singular environment.
  • C. Action Selection and Reward Functions: Bounded rationality and multi-objective rewards generate both adaptive and maladaptive behaviors, including learned helplessness and apathy selected through state-dependent internal reward expectations.Learned helplessness reflects predicting all actions as too negative, whereas apathy reflects predicting none as sufficiently rewarding.

D. Episode Termination, Primary Belonging Value, and Reward Scaling · E. Simulating Adverse Childhood Events

ERDM terminates episodes when livelihood/physical fitness or the agent’s primary belonging dimension reaches zero, while selective reward optimization and state-dependent scaling model bounded rationality and vulnerability. Validation compares agents across adverse-childhood-experience conditions to assess emergent adaptive and maladaptive behavior, including learned helplessness, passive coping, and aggression.

  • D. Episode Termination, Primary Belonging Value, and Reward Scaling: Episodes terminate when livelihood/physical fitness or the configured primary belonging dimension reaches zero, representing death or social exile and applying a fixed -20 penalty.The agent then resets to its initial state and the simulation continues.
  • D. Episode Termination, Primary Belonging Value, and Reward Scaling: Only the agent’s designated primary belonging dimension contributes to reward optimization, creating DB-dominant and NB-dominant types under bounded rationality.The non-primary belonging dimension remains part of the environment but does not influence learning.
  • D. Episode Termination, Primary Belonging Value, and Reward Scaling: Piecewise reward scaling increases the importance of diminished dimensions, capturing greater vulnerability and sensitivity when agents occupy lower-resource states.Scaling is based on discrete stages of each state variable.
  • E. Simulating Adverse Childhood Events: Validation simulates adverse childhood experiences to test whether ERDM produces plausible adaptive and maladaptive behavior from naïve developmental conditions.Threat intensity ranges from 80–100 in ACE environments, 20–40 without ACE, and 40–60 under moderate adversity.
  • E. Simulating Adverse Childhood Events: The experiment holds adaptive-problem frequency, internal reward, community strength, and risk parameters constant across conditions.This isolates differences among no-ACE, moderate-adversity, and ACE environments.
  • E. Simulating Adverse Childhood Events: DB-dominant, NB-dominant, and random agents undergo 200 independent simulations with 1500 training steps, evaluated through behavioral, state, reward, and environmental metrics.The random agent chooses uniformly without state-context, while learned helplessness and apathy require value estimates and are excluded from its action space.
  • E. Simulating Adverse Childhood Events: The hypotheses predict more learned helplessness and apathy under ACE, with NB-dominant agents showing passive coping and DB-dominant agents showing active or misdirected aggression.Predicted passive actions include fleeing and crying, whereas active responses reflect the DB-dominant protective role.

III. RESULTS · A. Learning Stabilization and Behavioral Diversity

Across agent types and adversity conditions, training loss converged and policy performance stabilized while behavioral diversity remained substantially preserved. Loss–entropy coupling was positive across conditions and strongest under ACE, indicating tighter links between convergence and behavioral consistency.

  • A. Learning Stabilization and Behavioral Diversity: Loss decreased significantly across all conditions and agent types, with DB reductions of 92–97% and NB reductions exceeding 99%.DB reductions ranged from 92% under No ACE to 97% under ACE, whereas NB loss reduced by over 99% across conditions.
  • A. Learning Stabilization and Behavioral Diversity: Late-window reward slopes did not differ from zero across agents and conditions, indicating stable policy performance.This result was reported alongside training-step reward stabilization.
  • A. Learning Stabilization and Behavioral Diversity: Behavioral entropy stabilized rather than collapsing, with DB agents retaining 34–44% and NB agents 34–49% of initial entropy despite 92–99% loss reduction.Late-window entropy slopes were not significantly different from zero across most conditions, supporting persistent behavioral diversity after learning stabilization.
  • A. Learning Stabilization and Behavioral Diversity: The dissociation between strong loss convergence and preserved behavioral diversity occurred across all agent types and conditions.ACE nevertheless showed tighter coupling between loss convergence and behavioral consistency than the other adversity conditions.
  • A. Learning Stabilization and Behavioral Diversity: Loss–entropy correlations were positive and significant across all conditions, with Spearman’s ρ= 0.16 to 0.88 and all p < 0.05.The association indicates that convergence and behavioral entropy were related, although the strength varied by adversity condition.
  • A. Learning Stabilization and Behavioral Diversity: Under ACE, loss–entropy coupling was stronger for DB agents (ρ = 0.82, p < 0.001) and NB agents (ρ = 0.88, p < 0.001) than under No Ace or Moderate Adversity.Under No Ace and Moderate Adversity, entropy showed greater residual variability across runs and weaker loss–entropy correlations.

B. Complex Behavioral Strategy Emergence

Behavioral strategies varied sharply by agent type and adversity condition. DB agents shifted toward internalizing and away from externalizing behaviors under ACE, whereas NB agents remained predominantly affiliative with minimal maladaptive responses.

  • NB behavioral policies: NB agents exhibited significant adaptive and maladaptive multivariate effects, but healthy friendliness remained primary across conditions and increased to ~47% under ACE.Misdirected and relational aggression remained near 0%, and learned helplessness was absent entirely.
  • NB agent phenotypes: NB agents showed no variation in internalizing or affiliative clusters, while ACE reduced externalizing behaviors to M = 3.8%, SD = 0.90% from ~6–7%.The externalizing effect was significant for ACE versus both No ACE and Moderate Adversity (both p < .001).
  • Complex behavioral phenotypes: NB agents showed higher affiliative (~52–58% vs. ~21–25%) and internalizing (~39–42% vs. ~0.7–17.6%) scores, while DB agents showed higher externalizing (~57–79% vs. ~4–7%) across conditions.These between-agent differences were significant across all three conditions (all Mann-Whitney p < .001).

C. DB-Dominant Agents Resilience with ACE · D. Adaptive Behavior Emergence by Resilient Vulnerable Groups

ACE-exposed DB-dominant agents separated into resilient and vulnerable latent subgroups, distinguished by learned helplessness and complex adaptive and maladaptive behaviors. Healthy friendliness, but not help-seeking vulnerability, tracked resilience, while vulnerable agents showed stronger maladaptive behavior patterns.

  • C. DB-Dominant Agents Resilience with ACE: ACE-exposed agents formed two latent learned-helplessness subpopulations, favored over a single component (ΔAIC ≈ 1173; ΔBIC ≈ 1158) and separated at 2.04%.The resilient subgroup had mean learned helplessness of 0.43% (n = 296), versus 20% (n = 704) for the vulnerable subgroup, with separation metric S = 2.50.
  • D. Adaptive Behavior Emergence by Resilient Vulnerable Groups: Healthy friendliness was strongly negatively associated with learned helplessness (Pearson r = -0.91, R² = 0.83, p < 0.001; Spearman ρ = -0.84, p < 0.001).Resilient agents also had significantly higher healthy friendliness than vulnerable agents (Mann-Whitney U, p < 0.001).
  • D. Adaptive Behavior Emergence by Resilient Vulnerable Groups: Help-seeking vulnerability showed no significant association with learned helplessness (Pearson r ≈ 0, p = 0.91; Spearman ρ ≈ 0, p = 0.87) and did not differ between groups.This analysis tested help-seeking vulnerability as a potential protective factor against ACE-related learned helplessness.
  • D. Adaptive Behavior Emergence by Resilient Vulnerable Groups: Vulnerable agents also had higher negative expectancy bias (37.65% vs. 17.35%; Cohen’s d = 3.94), while resilient agents had slightly higher dangerous trust (3.61% vs. 2.85%; Cohen’s d = -1.04).Negative expectancy bias had U = 2,309, p < 10⁻¹³¹; dangerous trust had U = 160,898, p = 3.96 × 10⁻⁴².

IV. DISCUSSION · A. Model Validity and Emergent Behaviors · B. Maladaptive Inactivity as Rational Survival Inhibition

ERDM separates reliable learning convergence from diverse, stable behavioral policies, supporting its validity as a model of emergent cognitive adaptation. Under ACE conditions, learned helplessness emerged as risk-focused survival inhibition, distinct from reward-depleted apathy.

  • A. Model Validity and Emergent Behaviors: ERDM’s learning stabilized across conditions, yet agents adopted diverse, stable strategies rather than converging on one behavioral optimum.This dissociation was observed alongside reliable learning from environmental feedback and across repeated runs.
  • A. Model Validity and Emergent Behaviors: The emergence of multiple strategies without explicit encoding supports ERDM’s internal validity as a model of cognitive adaptation rather than scripted behavior.The strategies are generated by survival reward functions, stochastic experience, and recent reward history.
  • A. Model Validity and Emergent Behaviors: Under ACE conditions, stronger associations between behavioral entropy and loss convergence suggest that high adversity narrows available behavioral options.The pattern indicates that threatening environments constrain the viability of diverse adaptive strategies.
  • B. Maladaptive Inactivity as Rational Survival Inhibition: Learned helplessness emerged exclusively under ACE conditions in DB-dominant agents, whereas apathy was absent across all conditions.This partially supported the first hypothesis and was not predetermined by the reward architecture.
  • B. Maladaptive Inactivity as Rational Survival Inhibition: Learned helplessness reflects risk-focused inactivity when all available actions exceed the agent’s risk threshold, whereas apathy reflects reward-focused motivational depletion.The two forms of inactivity therefore arise from distinct computational mechanisms.
  • B. Maladaptive Inactivity as Rational Survival Inhibition: ACE conditions producing learned helplessness rather than apathy suggests chronic adversity prioritizes risk evaluation over reward motivation.This reframes learned helplessness as a computationally logical survival response to overwhelming risks lacking suitable adaptive strategies [19].
  • B. Maladaptive Inactivity as Rational Survival Inhibition: The interpretation aligns with findings linking childhood threats or abuse to threat hypervigilance or risk appraisal, while neglect is more associated with anhedonia.It suggests interventions should target threat or risk appraisal versus motivational systems according to the underlying inactivity state.

C. Ecological Validity of NB and DB-Dominant Agents

NB- and DB-dominant agents spontaneously reproduced internalizing and externalizing profiles through different social-value priorities, supporting ERDM’s ecological validity. Their adversity-dependent behaviors indicate that maladaptive responses can be locally rational strategies under survival pressure and environmental mismatch.

  • Ecological validity: NB- and DB-dominant agents developed internalizing and externalizing profiles from a single difference in prioritizing nurturing versus defensive social value.This suggests that internalizing/externalizing distinctions may partly reflect how agents weigh prestige-oriented NB versus dominance-oriented DB survival strategies in reward processing.
  • NB-dominant agents: As adversity increased, NB-dominant agents showed avoidance-based coping, while their internalizing behaviors remained stable and avoidant behavior shifted toward tend-and-befriend.When allies could not handle threats, befriending incurred fitness costs, making avoidance a locally optimal satisficing response rather than prosocial behavior.
  • DB-dominant agents: Under ACE conditions, DB-dominant agents showed a maladaptive compositional shift: learned helplessness emerged, negative expectancy rose from ~15% to 33%, and aggression declined.The decline was compositional rather than uniform, with misdirected and relational aggression decreasing as learned helplessness displaced aggression under elevated threat.
  • DB-dominant agents: As threat exceeded DB agents’ risk threshold, learned helplessness displaced aggression, although vulnerable agents retained higher aggression, indicating an aggregate compositional decline.Baseline aggression toward non-threatening targets may serve dominance-display and hierarchy-establishment functions rather than indicate aberrant behavior.
  • Mechanistic interpretation: When allies were strong, DB-dominant agents shifted from aggression toward affiliation with high-status allies, lowering baseline dominance display while still establishing hierarchy.Together, these findings frame maladaptive behavior as a rational strategy under survival pressure and environmental mismatch rather than cognitive failure.

D. Resilience and Vulnerability as Emergent Survival Phenotypes · V. LIMITATIONS

ERDM produces resilient and vulnerable behavioral phenotypes under identical adverse conditions, linking social affiliation to reduced learned helplessness and generating intervention hypotheses. Its interpretation is limited by simplified perception, interaction structure, single-agent design, and absent experimental validation.

  • D. Resilience and Vulnerability as Emergent Survival Phenotypes: Under identical environmental conditions and intensity, DB-dominant agents separated into resilient and vulnerable latent subgroups through stochastic early-event ordering.Environmental parameters were held constant, but reward trajectories diverged into distinct behavioral phenotypes.
  • D. Resilience and Vulnerability as Emergent Survival Phenotypes: Social affiliation was strongly associated with resilience and healthy friendliness, showing a near-perfect negative relationship with learned helplessness (r = -0.91).The finding aligns with positive social relationships as a protective factor in trauma-exposed populations,.
  • D. Resilience and Vulnerability as Emergent Survival Phenotypes: DB-dominant agents deprioritized help-seeking because perceived vulnerability signaled weakness and was predicted to reduce dominance capacity and social survival.The model therefore produced a null finding for help-seeking vulnerability as a factor.
  • D. Resilience and Vulnerability as Emergent Survival Phenotypes: The model generated a testable hypothesis that positive social-relationship interventions may outperform help-seeking skills training for children with DB-dominant profiles.This model-derived hypothesis demonstrates ERDM’s usability for producing novel questions and intervention targets.
  • V. LIMITATIONS: ERDM assumes agents perceive environmental states accurately and without bias, omitting real-world distortions, cognitive biases, and self-referential miscalibration.Future versions could add perceptual noise and self-image distortion so agents can overestimate or underestimate capabilities and environmental properties.
  • V. LIMITATIONS: The framework simplifies behavior to three evolutionarily recurrent situations—threat, prey/goal-pursuit, and alliance opportunities—and a handful of conserved actions.This tractable abstraction omits dynamics including empathy, failed efforts, and unique community values.
  • V. LIMITATIONS: As a single-agent reinforcement-learning framework, ERDM cannot fully capture emergent group dynamics or interactions among dynamic agents in fundamentally multi-agent human contexts.A multi-agent extension could study population-level phenomena such as cultural transmission and social evolution.
  • V. LIMITATIONS: The study provides no accompanying experimental validation, so its findings remain computational and its behavioral predictions and suggested interventions require future empirical evaluation.Although ERDM is designed to translate into empirical settings and reveal unobservable cognitive mechanisms, those claims remain unvalidated here.

VI. CONCLUSION · Supplementary

ERDM shows that complex adaptive and maladaptive behaviors can emerge from simple reinforcement-learning agents under evolutionary constraints, environmental pressures, bounded rationality, and competing policy learning. Supplementary simulations further show that ally conditions alter behavioral-cluster profiles, while the framework supports ethical exploration of mechanisms across broader contexts.

  • VI. CONCLUSION: ERDM demonstrates that complex adaptive and maladaptive behaviors emerge from simple reinforcement-learning agents operating under evolutionarily grounded constraints.The conclusion frames this emergence as a result of the model’s evolutionarily recurrent decision structure rather than hard-coded behavioral outcomes.
  • VI. CONCLUSION: Behavioral strategies arise through interactions among environmental pressures, bounded rationality, competing policy learning, and satisficing.These mechanisms are presented as producing the model’s adaptive and maladaptive strategies.
  • VI. CONCLUSION: Learned helplessness, aggression, and avoidance or withdrawal may be predictable outcomes of rational agents in highly adverse or evolutionarily mismatched environments.The conclusion therefore treats maladaptive behavior as a possible rational output of evolutionary survival metrics in mismatched contexts, rather than necessarily intrinsic dysfunction.
  • VI. CONCLUSION: ERDM is a flexible, extensible, and ethical computational framework for studying behavior across real-world conditions mapped to evolutionarily recurrent adaptive problems.Although the present validity study focuses on adverse childhood experiences, environmental parameters and agent properties can be modified for other modern contexts.
  • VI. CONCLUSION: The adverse-childhood-experience study explored only a subset of possible behavioral interpretations and latent mechanisms, leaving substantial space for further discovery.Future extensions could add adaptive problems, conserved actions, detailed outcomes, perceptual and self-referential distortions, and multi-agent interactions.
  • VI. CONCLUSION: ERDM provides a foundation for mechanistic understanding by reproducing empirically observed patterns while uncovering cognitive mechanisms and enabling systematic, ethical hypothesis generation.The framework is positioned as a way to explore new hypotheses about cognition and behavior under evolutionary constraints.
  • Supplementary: Strong ally conditions significantly increased affiliative behavior and decreased externalizing behavior in defensive-belonging agents, while nurturing-belonging agents differed across all three clusters.For defensive-belonging agents, internalizing behavior did not differ significantly; all nurturing-belonging cluster comparisons were significant at p < 10^-4 or smaller.
Loading 2608.23932v1…