Source-linked AI summary
Composable Building Blocks for Resilient Asynchronous Code
Frank Tip
TL;DR
Asynchronous programs face transient failures, slow responses, throttling, atomicity violations, and cancellation, while available solutions often use mismatched interfaces. The paper expresses these concerns as same-type higher-order combinators that compose across promise and async-iterable functions, and case studies show them hardening real packages without changing core application logic. Its scope remains bounded by integration issues such as hidden retry observability and several combinators still being future extensions.
Problem
Asynchronous calls must handle transient errors, delays, throttling, atomicity violations, and cancellation, but JavaScript and TypeScript lack a converged, uniformly composable resilience-library ecosystem.
Method
The paper models each concern as a higher-order combinator that preserves the async function’s type and nests across promise-returning and async-iterable-returning functions.
Results
Three case studies show the combinators adding resilience or concurrency control to real, unmodified packages while leaving their application code untouched.
Takeaways & Limitations
A shared function-to-function shape lets programs express resilience, concurrency, and cancellation policies as composable layers around existing business logic.
Takeaways & Limitations
Integration with Strands’ event-driven retry loop prevents retry counts and per-attempt timing from reaching its existing observability, and circuit breaking, hedging, and debouncing remain future extensions.
Abstract
from arXiv · showhide
Asynchronous calls to a network service, database, or language model must cope with transient errors, slow or missing responses, throttling, and atomicity violations. We show how higher-order combinators solve such problems uniformly, including timeouts, retries, rate limiting, caching, reentrant locking, and cancellation. Every combinator maps an async function to another of the same type, so they share a uniform \emph{shape} and compose by nesting into one expression that implements a program's whole resilience and concurrency policy, leaving its business logic untouched. The same design spans both of JavaScript's native async shapes, promise-returning and async-iterable-returning functions, with one vocabulary of concerns. Solutions exist across the ecosystem but are scattered over differently shaped libraries that are hard to combine. We present case studies where the combinators are used to harden real packages by adding missing resilience or concurrency control and replacing bespoke policy.
1 Introduction
The paper proposes uniform higher-order combinators for hardening asynchronous calls against failures, delays, throttling, duplication, concurrency hazards, and cancellation. Because each combinator preserves the async function’s type, policies compose by nesting while business logic remains largely unchanged.
- Motivation: Production agents can exceed rate limits, stall on slow responses, lose successful sibling results after transient failures, and repeat identical work.These failure modes arise in the example agent because Promise.all launches requests together, lacks deadlines, rejects the whole batch on one failure, and does not deduplicate calls.
- Approach: Higher-order combinators wrap an async function with one policy and return another function of the same type, enabling natural composition.A hardened client can nest caching, retries, rate limiting, and timeouts around the original call.
- Approach: The uniform shape separates an agent’s business logic from its resilience policy, requiring only one line to replace the original call with the hardened version.The agent continues to operate through runAgent and callLLM while combinators handle unreliable-network concerns.
- Scope: The design covers both promise-returning and async-iterable-returning functions with matching combinator names and composition patterns.It also treats cancellation by distinguishing caller cancellation of a pending wait from a computation signalling cancellation, allowing propagation through combinator stacks.
- Scope: The paper presents an open-source package, async-combinators, and applies the combinators to existing packages as case studies.The package is intended to provide a shared vocabulary for resilience and concurrency concerns.
2 A Catalogue of Combinators
The catalogue organizes asynchronous combinators for transient failures, deadlines, request frequency, caching, and shared-state atomicity. Their common function-to-function shape supports composition, while streaming and concurrent operations introduce additional coordination concerns.
- Catalogue structure: Each combinator has promise-returning and async-iterable-returning variants, with streaming implementations raising additional subtleties.Table 1 groups the catalogue by theme, while Section 3.4 discusses streaming-specific issues.
- Single-call concerns: withRetry repeats failed calls up to a fixed attempt limit, optionally spacing attempts with exponential backoff and jitter.The spacing reduces synchronized retries and avoids overwhelming a struggling server.
- Single-call concerns: withTimeout rejects a call that does not settle before its deadline, while withRateLimit spaces requests to respect service caps.Exceeding an API’s cap can produce a 429 Too Many Requests response.
- Single-call concerns: withCache memoizes results within one run and stores pending promises to collapse concurrent duplicate calls into one in-flight request.withRecordReplay extends persistence across runs, supporting offline integration tests.
- Concurrency: Interleaved asynchronous updates can violate atomicity when both operations read shared state before either writes it.The example’s deposit and withdrawal both read 100; the deposit writes 110, the withdrawal writes 95, and the deposit is lost.
- Concurrency: withLock protects a standalone operation’s read-write sequence, restoring atomicity by preventing interleaving with another guarded call.The catalogue distinguishes combinators for individual calls from those coordinating concurrently running calls.
3 Implementation Challenges
The implementation models resilience and concurrency policies as composable higher-order wrappers, while addressing ordering, cancellation, reentrant locking, and streaming-specific retry and caching semantics.
- Composition and retry: withRetry returns a same-shaped async function that forwards arguments and retries until success or attempts are exhausted.Its generic parameters preserve the wrapped function’s argument tuple and promise result type.
- Composition and ordering: Combinator nesting order determines policy interaction: placing withTimeout inside withRetry gives each attempt its own deadline, while the reverse caps the entire retry loop.The same ordering principle applies to rate limiting and caching.
- Cancellation: Cancellation propagates through queued locks, concurrency slots, and retry backoffs, dropping pending work instead of running it after the caller aborts.Outbound AbortError is also excluded from retry, fallback, and caching policies.
- Reentrant locking: Reentrant locking uses AsyncLocalStorage to associate an ownership token with an asynchronous call chain, allowing matching nested acquisitions to bypass the queue.Unrelated chains wait for the lock and execute within a holder context.
- Streaming semantics: Streaming retries skip already delivered items when resumable, but give up after partial output when the stream is not resumable.Streaming caching instead buffers items and lets consumers replay them from independent cursors while sharing one upstream pull.
4 Case Studies
Three case studies apply the combinators to HTTP fetching, a local JSON database, and an LLM-agent SDK. They add resilience and atomicity through composition, while the Strands integration requires a targeted seam to preserve per-attempt observability.
- 4.1 Fetch: fetch gains retry, rate limiting, and per-attempt timeouts without changing application code or the client using the wrapped fetch.Caching and maximum-concurrency control can be composed similarly.
- 4.2 Lowdb: Concurrent lowdb deposits are serialized so updates are not lost, and a shared reentrant lock supports nested deposit and transfer operations without deadlock.The case study keeps the simplicity of a local JSON file while supplying the missing atomicity.
- 4.3 Strands: Strands streaming calls become paced and deadline-bounded by wrapping OpenAIModel.stream with withTimeout and withRateLimit.The wrapped stream fits the async-iterable combinator shape.
- 4.3 Strands: A modelCallDriver seam lets Strands supply withRetry as a pluggable driver while retaining its existing per-attempt event observability.The driver can extend retry decisions to include TimeoutError.
- Case-study evaluation: Runnable versions support the three case studies, with assertions for fetch and lowdb and an interactive logged demo for Strands.These forms make the case-study behavior directly verifiable or observable.
5 Related Work
Existing resilience libraries cover many asynchronous concerns, but ecosystems expose them through incompatible abstractions. This work instead emphasizes a uniform, signature-preserving combinator shape, including support for native async iterables and reentrant locking.
- Resilience ecosystems: Polly and resilience4j provide mature resilience mechanisms, but their pipeline, decorator, configuration, and resource-isolation abstractions differ from this paper’s combinators.Polly uses a separate resilience pipeline, while resilience4j decorators depend on strategy-specific configuration types; neither provides the paper’s mutual-exclusion primitive.
- Paper’s distinction: The paper’s contribution is composing resilience and concurrency policies through one uniform, signature-preserving function-to-function shape without policy objects or builders.Unlike Polly and resilience4j, the catalogue also includes a reentrant lock for mutual exclusion.
- JavaScript ecosystem: JavaScript resilience tooling is fragmented across wrappers, promises, limiter functions, policy objects, and stateful classes that are difficult to assemble.The cited ecosystem includes cockatiel, p-* modules, async-mutex, async-lock, and opossum, each exposing different interfaces.
- Async streams: For streams, RxJS uses its Observable type and IxJS uses wrapper classes and chained operators rather than plain functions over native AsyncIterable.These libraries therefore do not share the paper’s plain function-to-function vocabulary for JavaScript’s native async shapes.
- Async streams: The stream retry combinator avoids silently replaying nondeterministic output by propagating post-output failures unless resumable mode is explicitly enabled for deterministic sources.This differs from RxJS resubscription and IxJS re-iteration, which restart sources from the beginning on error.
- Testing and replay: withRecordReplay generalizes recording and replay beyond transport-specific tools by operating at a lower level than HTTP- or LLM-layer interception.The related systems include VCR, nock, and aimock, each focused on a specific transport layer.
6 Discussion and Conclusion
The paper presents signature-preserving combinators as a unified vocabulary for resilience, concurrency, and cancellation across promises and async iterables. Discussion identifies extensibility as a benefit, while open-source releases and case studies demonstrate the approach on real packages.
- Conclusion: Higher-order combinators let nested policies handle resilience, concurrency, and cancellation while leaving business logic untouched across both native JavaScript async shapes.The paper states that three case studies apply this approach to real, unmodified packages.
- Future extensions: The uniform function-to-function shape is open to adding circuit breaking, request hedging, and debouncing without introducing a new composition model.The discussion identifies circuit breaking and request hedging as candidates already implemented in related libraries, while debouncing exists for reactive types.
- Availability: The combinators and case studies are released as open-source, with the package available on npm and case-study repositories linked from its README.The implementation is hosted in the async-combinators repository.