Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services

José A. Perdiguero López; Miguel A. Durán-Olivencia

Source-linked AI summary

Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services

José A. Perdiguero López, Miguel A. Durán-Olivencia

arXiv:2608.18733v1Aug 19, 2026cs.SEcs.AIcs.LG

TL;DR

Deploying machine-learning models remains more difficult than training them, requiring substantial serving and integration infrastructure. Flama addresses this gap by unifying web APIs, predictive model serving, and LLM inference, while its route-table cost rises roughly 4% from ten to two hundred entries.

Problem
Deploying trained models still consumes more project effort than modeling, creating a persistent gap between working models and user-facing services.
Method
Flama unifies web APIs, predictive model serving, and LLM inference through one async-first, type-driven programming model with protocol-agnostic serving.
Results
Growing the route table from ten to two hundred entries increases request cost by roughly 4%, while ten middleware layers add about 0.6% over none.
Takeaways & Limitations
Flama provides one system for predictive and generative serving where alternatives require composing two or three systems.
Takeaways & Limitations
The multi-backend LLM feature comparison indicates coverage rather than a claim about inference throughput.

Abstract

from arXiv · show

We present Flama, an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-language-model (LLM) applications. Built on the Asynchronous Server Gateway Interface (ASGI), Flama offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture. It is organised around seven subsystems: a component-based dependency injection system resolving handler parameters from type annotations at startup; a pluggable schema layer supporting Pydantic, Marshmallow and Typesystem behind a single adapter; an automatic CRUD generator turning a SQLAlchemy table and a schema class into REST endpoints backed by the Repository and Unit of Work patterns; a portable binary format (.flm) packaging models from scikit-learn, TensorFlow, PyTorch and Hugging Face Transformers with their metadata for zero-code deployment; a multi-backend LLM server running vLLM (Linux/CUDA) or MLX (Apple Silicon) and exposing four wire protocols (OpenAI, Anthropic, Ollama, and a native streaming dialect) through a shared codec; a Rust-accelerated core compiled via Maturin for routing, JSON encoding, compression and parsing; and a Model Context Protocol module turning any application into an MCP server over JSON-RPC 2.0. Built-in capabilities include JWT authentication, two pagination strategies, background tasks in threads or processes, WebSocket endpoints, Server-Sent Event and NDJSON streaming, OpenAPI 3.2.0 generation from handler signatures, and a command-line interface for running applications and for serving, packaging and inspecting models. We describe the architecture, present the programming model through worked examples, and compare Flama with existing frameworks, model serving platforms and LLM inference engines.

Foundations · 1 Introduction · 2 Design principles

Flama addresses the deployment gap by unifying web API development, predictive model serving, and LLM serving in one open-source Python framework. Its design uses asynchronous execution, type annotations, pluggable schemas, dependency injection, generated resources, native model serving, and protocol-independent generative inference.

1 Introduction: Production ML deployment remains difficult because fragmented web, serving, and packaging tools introduce incompatible abstractions, configuration, adapters, and duplicated operational concerns.The deployment and integration burden can exceed the modelling work, with technical debt accumulating at system seams.
1 Introduction: Flama unifies web API and model-serving concerns, providing routing, validation, authentication, pagination, database access, background tasks, documentation, inference, predictions, and metadata inspection in one programming model.The framework argues that these capabilities need not be separated into distinct processes, middleware stacks, or deployment pipelines.
2.3 Pluggable schema libraries: A pluggable schema adapter supports Pydantic, Marshmallow, and Typesystem through one internal representation, insulating validation and OpenAPI generation from library-specific APIs and changes.The choice is made at application initialization, allowing existing schemas to be adopted without rewriting them.
2.4 Dependency injection as a first-class mechanism: Component-based dependency injection resolves handler requirements from type annotations, supporting test substitution, composable dependency graphs, request-scoped caching, and uniform handling of application resources.The same mechanism covers database connections, validated data, authentication tokens, model instances, pagination parameters, and user-defined dependencies.
2.6 ML and LLM serving as native concerns: Predictive and generative models are native framework modules: .flm packages support model deployment, while multi-backend LLM serving exposes shared model instances through OpenAI, Anthropic, Ollama, and native dialects.Model endpoints, metadata handling, deferred loading, CLI serving, canonical message/event transport, and dialect-specific rendering keep inference integrated with the application.
2.1 Asynchronous by default: Flama targets ASGI and applies async/await throughout, multiplexing I/O while offloading CPU-bound inference and transformations to threads or processes.Synchronous handlers remain supported by wrapping them with asyncio.to_thread(), enabling migration without rewriting every handler.
2.2 Type annotations as the source of truth: Python type annotations define request parsing, validation, dependency resolution, and OpenAPI 3.2.0 generation, making the handler signature the endpoint contract without additional registration or configuration.Parameter types determine extraction and schemas, while registered component types determine injected dependencies.
2.5 Convention over configuration for CRUD: From a SQLAlchemy table and schema class, Flama generates CRUD endpoints with validation, pagination, status codes, and error handling, reducing hundreds of handler-code lines to a class declaration under ten.Individual operations can still be overridden or extended without abandoning the generated scaffold.

3 Architecture

Flama uses a layered architecture in which independently reasoned layers expose abstractions upward, while the Flama application composes routing, dependency injection, components, middleware, events, and modules. Requests follow a fixed path through middleware, route resolution, dependency injection, handler execution, response processing, and transmission, and modules extend the application with specialized capabilities.

Layered architecture: Flama’s layers comprise ASGI, a Rust-accelerated Core, the Application layer, and extensible Modules, with each layer handing abstractions to the one above.The Core supplies HTTP and WebSocket primitives, routing, encoding, compression, and parsing; the Application orchestrates lifecycle services; Modules add related functionality.
Application layer: The Application layer is the Flama ASGI callable, assembling a Router, Injector, Components registry, MiddlewareStack, Events manager, and Modules.Its lifecycle orchestration passes connections through middleware and routing, resolves dependencies, runs handlers, and builds responses.
Dependency injection: The Injector builds dependency-resolution trees at startup and resolves handler parameters from context values, registered components, or path and query parameters.Resolved values use a per-request cache, so cacheable components resolve at most once per request by default.
Module system: Modules are self-contained functionality units that can register components, add routes, and use lifecycle hooks; shipped modules include Schema, Models, and MCP capabilities.SchemaModule generates OpenAPI 3.2.0, ModelsModule manages ML-model lifecycles, and MCPModule implements the Model Context Protocol.

The Web Framework · 4 Routing and endpoints

Flama’s routing layer unifies HTTP, WebSocket, mount, and resource routes with typed path resolution, class-based endpoints, streaming responses, and automatic OpenAPI generation. A Rust-compiled route table accelerates matching while preserving modular composition and multiple real-time communication patterns.

The Web Framework: Routing serves as the application’s external-interface boundary, determining handler selection, typed parameter conversion, and behavior when no route matches.This ordering motivates presenting route types, declaration syntax, the compiled route table, and class-based endpoints together.
4.1 Route types: Flama supports four route types: HTTP routes, WebSocket routes, mounts for sub-applications or routers, and resource routes generating CRUD endpoints.HTTP routes accept synchronous or asynchronous functions and class-based endpoints; mounts can include any ASGI-compliant application, while resource routes expand into standard HTTP routes.
4.2 Declaring routes: Routes can be declared with decorators or programmatically, while Annotated schema metadata identifies validated request and response payloads for documentation.Handlers returning Response objects directly or lacking documented payloads omit the return annotation and generate no response schema.
4.3 Path parameters and type converters: Braced path parameters support string, integer, float, UUID, and Decimal converters that validate and convert segments before handler invocation.A conversion failure prevents the route from matching, allowing alternative routes to be searched or producing a 404 response.
4.4 The Rust-accelerated route table: 3.54 M estimated cycles for ten routes and 3.69 M for two hundred routes show near-constant route-resolution cost, with under 3% variation between first and last entries.The Rust-compiled RouteTable performs segment matching and path-parameter extraction in one pass using pre-compiled route patterns.
4.5 Class-based endpoints: Class-based endpoints group HTTP handlers by verb and model WebSocket connections through connect, receive, and disconnect lifecycle hooks.HTTPEndpoint introspection determines allowed methods, unsupported methods return 405, GET endpoints automatically serve HEAD, and WebSocket encoding may be JSON, text, or bytes.
4.5.3 Streaming responses: Flama streams unbuffered content through StreamingResponse, ServerSentEventResponse, and NDJSONResponse, complemented by WebSockets for real-time communication.SSE supports Last-Event-ID resumption and sequence-based replay in the native dialect, whereas NDJSON provides line-oriented records without event types or reconnection semantics.
4.6 OpenAPI schema generation: Every registered route contributes automatically to an OpenAPI 3.2.0 specification generated at startup from route converters, handler parameters, schemas, and return annotations.The specification is served at /schema/ and rendered through Swagger UI at /docs/, with optional operation metadata supplied in YAML docstrings.

5 Schema and data validation

Flama decouples validation from specific schema libraries through a global adapter and library-independent internal representation. Its dependency-injection pipeline automatically parses and validates request data before handlers run, returning structured 400 errors, while response schemas govern serialization and OpenAPI documentation.

Error handling: 400 Bad Request responses contain structured JSON errors keyed by field name when any validation component fails, preventing malformed input from reaching application logic.Each field’s error shape comes from the configured schema library’s own error representation.
Adapter layer: The adapter normalizes field introspection, validation, and JSON Schema emission across Pydantic, Marshmallow, and Typesystem.The adapter is selected globally when the application is constructed, while users continue defining schemas with each library’s native API.
Internal representation: Flama’s internal Field and Schema objects provide a library-independent representation consumed by validation, OpenAPI generation, and response serialization.Fields capture types, nullability, requiredness, defaults, multiplicity, and JSON Schema equivalents; schemas can be built from annotations or explicit field lists.
Request pipeline: Validation runs automatically during dependency resolution, parsing request bodies and validating path, query, and schema-bound parameters before handler execution.RequestDataComponent selects JSON, URL-encoded, or multipart codecs, while additional components validate typed route, query, and body inputs.
Response validation: Declared response types control runtime serialization and generate the JSON Schema included in OpenAPI documentation.When a response schema is declared, returned dictionaries are revalidated through the adapter before JSON encoding.

6 Dependency injection · 7 Resources and domain-driven design

Flama uses type-driven dependency injection to resolve request data, context values, components, and nested dependencies through startup-compiled resolution plans with per-request caching. Its resource system combines generated CRUD APIs with Repository and Worker patterns, while supporting virtual resources, overrides, validation, pagination, and transactional lifecycle management.

6 Dependency injection: Dependency injection is central to Flama: handlers receive request data, validated inputs, database connections, authentication tokens, and model instances through the injector.Every request passes through dependency injection, whether or not the developer explicitly engages with it.
6.1 The component model: Components produce typed values through can_handle_parameter() and resolve(), with annotated upstream dependencies resolved recursively into dependency graphs.A component can depend on another component, allowing chains such as a repository depending on a database connection.
6.2 Resolution trees: At startup, Flama builds a directed acyclic resolution tree from context, component, and primitive parameter nodes, then flattens it into request-time resolution steps.Context values include Request, the application, the matched Route, and WebSocket connections; circular dependencies are detected during compilation.
6.3 Caching: Cacheable components use a per-request LRU cache so expensive dependencies are resolved at most once, and the cache is discarded after the response.Built-in context annotations also expose ASGI scope, receive and send channels, Request, and Response without component registration.
7.2 CRUD resource declaration: A SQLAlchemy table and schema can generate nine REST endpoints with status codes, pagination, error handling, and overridable operations.Generated routes include creation, retrieval, replacement, updates, deletion, listing, and bulk operations; integrity errors return 400 and missing resources return 404.
7.1 The resource abstraction: A resource declares a domain model, validation schema, and API operations; Resource supports virtual entities, while CRUDResource derives database-backed operations from a SQLAlchemy table.The CRUD metaclass inspects columns, generates schemas and a repository, and creates standard CRUD handlers.
7.3 Domain-driven design patterns: Generated CRUD handlers delegate persistence and transactions to repositories and Workers, separating data access from business logic and supporting database or HTTP-backed resources.Repository families use SQLAlchemy clauses and filters or remote resource identifiers, while the Worker provides the Unit of Work boundary.
7.3.2 The Unit of Work pattern (Worker): Workers manage connection acquisition, transaction begin, commit, and rollback through an asynchronous context-manager lifecycle, with repositories created lazily from type annotations.WorkerComponent exposes a configured Worker to dependency injection, alongside the required database module and explicit registration.

8 Authentication and authorization

Flama provides an integrated JWT authentication and authorization system spanning token validation, typed dependency injection, and route-level permission enforcement. Authentication and authorization remain separate and independently usable: token components resolve identities, while middleware checks permissions.

JWT implementation: Flama’s JWT implementation encodes and decodes signed tokens while validating standard claims and rejecting expired or not-yet-valid tokens.HS256 is the default algorithm, with HS384 and HS512 also supported; RSA and ECDSA are not implemented.
Dependency injection: AccessTokenComponent and RefreshTokenComponent extract and decode tokens from request headers or cookies, making typed token objects available through dependency injection.Access tokens use the access_token source and refresh tokens use refresh_token for token rotation flows.
Route authorization: AuthenticationMiddleware authorizes routes by resolving an AccessToken, combining direct and role-derived permissions, and checking whether they cover the route’s requirements.Missing or invalid tokens produce 401 Unauthorized, whereas insufficient permissions produce 403 Forbidden.
Separation of concerns: Authentication and authorization are separated and independently usable: handlers can request an AccessToken without middleware, while middleware can enforce permissions without handler token inspection.Token components verify identity; AuthenticationMiddleware verifies permissions.

9 Pagination … 12 Machine-learning model serving

Flama provides declarative pagination, post-response background execution, ordered ASGI middleware, and a model-serving subsystem designed to minimize the distance between trained models and production APIs. These facilities combine multiple execution strategies, standardized response handling, framework-agnostic artifacts, and zero-configuration deployment.

9 Pagination: Flama offers page-number and limit-offset pagination strategies that wrap list handlers and return standardized response envelopes.Page-number pagination suits numbered tabular interfaces, while limit-offset pagination suits infinite-scroll and cursor-based interfaces.
9.1 Pagination strategies: Pagination supports an optional count boolean, defaulting to False because counting large tables is expensive; omitted counts appear as null.The envelope still reports the pagination parameters when count is omitted.
9.2 Usage: The paginator rewrites handler signatures, intercepts returned collections, applies the selected slice, and delegates counting to the appropriate strategy.Pagination parameters can therefore be resolved and documented without being explicitly named by the handler.
10 Background tasks: Flama executes background work after transmitting the response, using threads for I/O-bound callables and separate processes for CPU-bound computations.BackgroundThreadTask uses asyncio.to_thread(), whereas BackgroundProcessTask uses multiprocessing.Process.
10.2 Usage: BackgroundTasks aggregates multiple tasks into one container and executes them sequentially in the order added.This supports handlers that need to schedule several post-response operations.
11 Middleware: Middleware forms an ordered ASGI onion stack that can inspect or modify requests and responses, short-circuit processing, and apply cross-cutting concerns.The last registered middleware becomes outermost and sees each request first.
11.1 Middleware stack: Five middleware layers cost 3.53 M estimated cycles and ten cost 3.54 M, compared with 3.52 M without middleware.Each additional layer adds on the order of 0.1%, making architectural considerations more important than a per-request budget.
12 Machine-learning model serving: The model-serving subsystem targets framework-agnostic, self-describing, efficiently transported artifacts and zero-configuration REST deployment.It supports scikit-learn, TensorFlow/Keras, PyTorch, and HuggingFace Transformers; metadata includes framework versions, classes, hyperparameters, metrics, and auxiliary artifacts, while generated endpoints include schemas, OpenAPI documentation, and error handling.

13 The FLM binary format

The FLM format is a versioned, self-describing container that packages serialized models, metadata, and auxiliary artifacts, with metadata positioned for inspection before reading weights. Protocol version 2 extends the design for directory-based LLM checkpoints, per-section compression, and persisted serving capabilities.

13 The FLM binary format: FLM packages a serialized model, metadata, and required auxiliary artifacts into one versioned, self-describing file.Metadata is stored at a known offset ahead of the weights, so inspection requires only a header parse.
13.1 File structure; 13.1.1 Outer header: The 16-byte outer header selects protocol version and compression, while version 1 stores opaque traditional-model blobs and version 2 supports directory bundles.The header also records the body size; version 2 is the default written by dump().
13.1 File structure; 13.1.2 Body header (protocol version 1): Version 1 organizes the body as compressed metadata, model weights, and named artifact entries, each independently decompressible.Its body header records metadata size, model size, artifact count, and total artifact size.
13.1.3 Metadata section; 13.1.4 Model weights section: Metadata is compressed JSON at a fixed post-header offset, enabling inspection without reading weights, which use framework-specific serialization.The metadata describes the packaged artifact, while the model section contains the serialized model.
13.1.5 Artifacts section: Named artifact entries store auxiliary inference files such as tokenizers, label maps, preprocessing configurations, and post-processing scripts.Artifacts are extracted to a temporary directory and automatically cleaned up when the ModelArtifact is garbage-collected.
13.1.6 Protocol version 2: Protocol version 2 adds per-section compression discriminators and binary-or-bundle model kinds to accommodate LLM checkpoint directory tarballs.Sections can inherit file-level compression or use bz2, lzma, zlib, zstd, or no compression; bundle models contain multiple checkpoint files.
13.1.7 Model capabilities; 13.1.8 Artifact families: Version 2 persists model capabilities detected during serialization, allowing dispatch, validation, and serving advertisement to use one manifest source.The metadata also records an explicit ml or llm artifact family, while LLM artifacts identify transformers as their on-disk library.
13.2 Compression; 13.3 Serialization and deserialization API: The API serializes to or deserializes from paths or binary streams, requires dump() to receive an explicit family, and warns when stored and installed framework versions differ.The format supports bz2, lzma, zlib, and zstd, with zstd as the default and typical neural-network compression ratios of 1.5–3×.

14 Framework-specific serializers · 15 Model wrappers · 16 Three levels of model integration

Flama separates framework-specific serialization and inference from shared model-serving infrastructure, then exposes three progressively automated integration levels. Serializers normalize model packaging and metadata, wrappers delegate framework-native inference, and developers can choose between full endpoint control and generated resources.

14 Framework-specific serializers: Each framework serializer implements dump(), load(), and info(), while ModelSerializer dispatches live models by module hierarchy without requiring framework declarations.Directory paths are the exception: dump() requires an explicit lib argument because paths have no class to inspect.
14.1 Scikit-learn · 14.2 TensorFlow and Keras · 14.3 PyTorch · 14.4 HuggingFace Transformers: Framework serializers use native strategies: pickle for scikit-learn, .keras files for TensorFlow/Keras, torch.export graphs for PyTorch, and directory tar bundles for Transformers.The PyTorch export preserves dynamic shapes and produces an artifact independent of the Python class definition; Transformers loads an extracted snapshot directory rather than raw bytes.
15 Model wrappers · 15.1 Framework-specific backends: A shared wrapper owns lazy deserialization, metadata, artifacts, and the HTTP-facing surface, while framework-specific backends translate inputs, invoke inference, and return JSON-serializable outputs.MLModel is the single predictive wrapper; framework semantics remain in concrete backends, which also raise FrameworkNotInstalled before request handling when dependencies are unavailable.
16 Three levels of model integration: Flama offers three progressively automated integration levels, ranging from custom endpoints using framework-managed loading to model files and names that generate endpoint infrastructure automatically.The levels build on one another, letting developers select the amount of endpoint customization required.
16.1 Level 1: Model components: Level 1 injects a dynamically typed model component into handlers, distinguishes ML and LLM artifacts from the FLM header, and defers backend initialization and weight loading until startup.Fresh per-instance model types prevent multiple registered models from colliding in dependency injection, while the server port can bind before heavy loading begins.
16.2 Level 2: The add_model() API: Level 2’s add_model() API creates a complete model resource with metadata and prediction endpoints for inspection and inference.The generated routes are GET /classifier/ and POST /classifier/predict/, using PredictInput and PredictOutput schemas.
16.3 Level 3: Model resources: Level 3 uses a metaclass to turn a class declaration into a reusable model resource by loading its component, storing model state, and generating inspect, predict, and stream handlers.This level targets applications deploying multiple models as self-contained units with individual configuration.

17 Large language model serving

Flama’s LLM serving subsystem uses a layered architecture to translate multiple wire protocols into canonical typed requests and streamed events, then execute inference through hardware-aware backends. It supports incremental streaming, protocol-compatible rendering, automatic backend selection, structured tool-call decoding, durable stream replay, and a built-in chatbot interface.

Streaming output: The serving layer delivers incrementally generated tokens through Server-Sent Events or Newline-Delimited JSON rather than waiting for complete sequences.Its flow yields EngineDelta objects incrementally, decodes them into events, and renders SSE or NDJSON frames.
Multi-protocol compatibility: Four wire dialects—OpenAI, Anthropic, Ollama, and Native—share one canonical transport and inference pipeline.Dialect parsers produce canonical messages and tools, while renderers emit protocol-specific streaming frames or buffered envelopes.
Structured event decoding: The LLMCodec finite-state machine converts varied reasoning and tool-call text formats into structured TextEvent, ToolEvent, and TraceEvent objects.Downstream renderers therefore receive identical typed tool events whether calls appear as raw JSON in a <tool_call> block or Python-style function(args) notation.
Hardware-aware backends: Backend selection probes available libraries at model load time, using vLLM on Linux/CUDA and MLX on Apple Silicon without backend-specific application code.vLLM provides PagedAttention, continuous batching, and tensor parallelism; MLX provides Metal acceleration and unified memory.
Native dialect extensions: The native dialect adds durable stream persistence for replay and resumption, plus a self-contained chatbot interface served at /chat/.StreamsBackend stores events under (model, stream id) keys and supports append, range reads, deletion, and length reporting.

18 The Model Context Protocol

Flama provides a first-class Model Context Protocol module that turns applications into MCP servers exposing tools, resources, prompts, and inline UI templates. It implements stateless JSON-RPC communication with type-derived schemas and extensions for tracing, long-running tasks, and user elicitation.

Core capabilities: Flama applications can act as MCP servers, registering callable tools, URI-addressed resources, and reusable parameterized prompts.Tool input schemas are generated from handler signatures, while resources expose text, JSON, or binary data.
Transport and protocol: MCP communication uses POST-only HTTP endpoints with stateless JSON-RPC 2.0 requests, client metadata, protocol validation, and routing-header consistency checks.Each request carries client identity, capabilities, and protocol version in _meta, avoiding an initialize/initialized handshake.
Schemas: Flama emits self-contained JSON Schema 2020-12 documents for tool inputs and annotated outputs, using local $defs references and independently validatable schemas.Input schemas reflect parameter names, types, and defaults; output schemas derive from return annotations when available.
Extensions: The module exposes W3C Trace Context as an injectable component and supports background task execution with polling, persistent state, completion or failure status, and cancellation.Long-running tools can return task identifiers immediately, while clients retrieve results through tasks/get and cancel work through tasks/cancel.
Extensions: Stateless elicitation lets tools request user input mid-execution, while the MCP Apps extension exposes prefetchable HTML or Markdown templates that clients render inline.Continuation state is serialized into a requestState token, and templates are retrieved through resources/templates/list and resources/read.

Operations and Tooling · 19 Command-line interface

Flama’s CLI organizes the operational lifecycle into six top-level commands, spanning application execution, code-free model serving, multi-model deployment, model packaging, offline interaction, and version migration. Its type of deployment workflow also supports environment-variable configuration and integrates model serving with OpenAPI and Swagger documentation.

19.1 Overview: The `flama` entry point provides six commands covering application execution, model serving, multi-model deployment, packaging, offline model use, and major-version migration.Options can bind to environment variables through the `FLAMA_*` prefix, supporting containerized configuration.
19.2 flama run: The `run` command launches a Uvicorn server for a user-defined Flama application specified by a Python import path.It follows the ASGI `module:application` convention and exposes Uvicorn options with the `--server-` prefix.
19.3 flama serve: The `serve` command deploys one or more models behind REST endpoints without application code by generating and running a temporary Flama application.Each model receives its specified URL prefix, alongside OpenAPI at `/schema/` and Swagger UI at `/docs/`; model specifications may be paths, key-value lists, or JSON, YAML, and TOML files.
19.4 flama start: The `start` command manages multi-model deployments from a configuration file specifying application metadata, models, and server options.Its template-generation option supports version-controlled infrastructure-as-code and CI/CD workflows.
19.5 flama get: The `get` command downloads remote models and packages them into `.flm` artifacts ready for `flama serve` or offline `flama model` use.It downloads files concurrently, uses protocol version 2, records the artifact family, and dispatches loading according to the required `--family` value.
19.6 flama model; 19.6.1 flama model inspect; 19.6.2 flama model run; 19.6.3 flama model stream: The `flama model` group enables offline inspection, one-shot inference, and streaming for both ML and LLM artifacts without starting a server.LLM options control transport formatting, system instructions, generation parameters, output channels, and decoder overrides.
19.6.1 flama model inspect; 19.6.2 flama model run; 19.6.3 flama model stream: Offline model commands support metadata verification, batch or file-based scoring, CI/CD validation, debugging, and interactive LLM experimentation with incremental or buffered output.The `stream` subcommand prints tokens as produced, while `--buffer` accumulates output before writing it.
19.7 flama upgrade: The `upgrade` command automates major-version migration by rewriting imports and renamed symbols, previewing changes with `--diff` or applying them with `--write`.Unmatched symbols receive `# flama-upgrade` markers and manual follow-up listings so breaking changes are not silently missed.

20 Configuration and deployment

Flama separates application identity and feature configuration from server settings, while supporting multiple construction, serialization, and deployment paths. Its environment-aware application configuration and deployment safeguards accommodate differing environments without conflating runtime settings with application behavior.

Configuration structure: App separates application identity, feature configuration, and served models from server concerns such as host, port, workers, and SSL.App covers metadata, schema and route settings, debug mode, and models, while Uvicorn contains server options.
Configuration structure: Applications can be constructed from Python import strings, dictionaries, or live Flama instances, with each path producing an App configuration.Examples include StrApp("mymodule:app"), DictApp.from_dict(data), and FlamaApp(app).
Server configuration: Uvicorn options support defaults and overrides from the CLI, environment variables, or JSON, while Config starts the server and supports JSON serialization and deserialization.Config exposes run(), dumps(), loads(data), and load(fs).
Application settings: Application settings resolve environment variables first, then an ini, json, yaml, or toml file, and finally an explicit default.Dotted keys access nested structures, and dataclass casting accepts nested file settings or JSON-valued environment variables.
Safety and deployment: Secret masks values in diagnostics but is not encryption, and debug mode should not be deployed because it exposes source, paths, and request headers without a startup warning.The deployment configuration is responsible for avoiding debug=True.
Deployment: Production deployments typically place Flama behind a reverse proxy for TLS termination, rate limiting, and static files, with TrustedHostMiddleware and CORSMiddleware managing host and cross-origin policies.The simplest direct Uvicorn deployment is suitable for internal services, development servers, and containers behind a load balancer.

21 Lifespan management · 22 Testing

Flama structures application startup and shutdown through the ASGI lifespan protocol, combining event handlers, context managers, and module hooks with typed, application-reachable resources. Its test client drives this lifecycle and supports URL resolution, model endpoint testing, and database-backed Worker and Repository tests.

21 Lifespan management: Flama implements ASGI lifespan management to run initialization at startup and cleanup at shutdown, replacing ad-hoc WSGI signal handlers and module-level initialization.The lifespan protocol provides structured lifecycle boundaries for applications.
21 Lifespan management: Three complementary lifecycle mechanisms are supported: event handlers, lifespan context managers, and automatically registered Module hooks.Handlers may be supplied during construction or through decorators; module hooks are registered when a Module is added.
21 Lifespan management: Startup handlers run concurrently before entering the lifespan context, while shutdown exits that context first and then runs shutdown handlers concurrently.Lifespan events also propagate to child mounted applications.
21 Lifespan management: Flama avoids a general-purpose application state bag, assigning state to named modules or typed components reachable through the injectable application instance.This makes lifecycle-created resources available to request handlers as typed dependencies.
22 Testing: The flama.client.Client subclasses httpx.AsyncClient and wraps application lifespan execution, preventing requests before startup and ensuring shutdown on context exit.A bare httpx.ASGITransport fails because it does not implement the lifespan protocol.
22 Testing: Named resource routes can be tested without hard-coded URLs because app.resolve_url() derives paths from the resource and operation.Changing a mount point therefore does not require updating URLs throughout the test suite.
22.1 Testing ML model endpoints: Model endpoints support tests that mount model files and send predictions, while model_request() addresses models by name after resolving their mounted URL.The models shortcut registers (name, url, path) triples through add_model().
22.2 Testing with Workers and repositories: Worker and Repository applications can use real in-memory databases or mocked repositories, with shared connections and rollback transactions enabling isolated, repeatable tests.Session-scoped fixtures can create schemas once, while per-test transactions roll back changes during teardown.

Ecosystem and Outlook … 25 Conclusion and future work

Flama unifies web APIs, predictive model serving, and LLM inference in one architecture, addressing a gap left by frameworks, model servers, and inference engines. Its feature breadth supports applications combining conventional endpoints with predictive and generative serving, while future work targets versioning, batching, GraphQL, observability, and agentic workflows.

23 Related work: Existing systems divide into web frameworks, predictive model servers, and LLM engines, so missing capabilities arrive through glue code, external tools, or operational workarounds.Web frameworks generally lack native model serving; model servers lack general web composition; LLM engines expose inference protocols rather than full application frameworks.
23.4 Positioning Flama: Flama’s web-framework position combines native model serving with the same routing, validation, dependency injection, middleware, and authentication objects used by conventional endpoints.The framework treats prediction routes as ordinary routes and supports predictive and generative models alike.
23.3 LLM inference engines: One deployment renders OpenAI, Anthropic, and Ollama protocols from a canonical event stream without protocol-specific application code or an intervening proxy.The multi-dialect layer uses shared renderers rather than separate servers.
24 Feature comparison: Flama is the only compared system combining full web APIs, multi-framework predictive serving, multi-backend multi-dialect LLM inference, and MCP in one architecture.The comparison counts built-in support only; third-party extensions and custom code are excluded.
24 Feature comparison: For applications needing conventional endpoints, predictive inference, and generative serving together, alternatives require composing two or three systems, whereas Flama requires one.The comparison measures feature presence rather than quality, and its breadth is not intended to supersede specialized systems for narrower workloads.
25 Conclusion and future work: The conclusion presents Flama as an async-first, type-driven framework with pluggable schemas, component-based dependency injection, convention-over-configuration resources, and unified serving abstractions.Its FLM format packages predictive models and LLM checkpoints in portable, compressed, metadata-rich containers with two protocol versions.
25 Conclusion and future work: Growing the route table from ten to two hundred entries increases request cost by roughly 4%, illustrating the Rust core’s implementation impact in production use.The Rust core covers route resolution, matching, JSON encoding, compression, multipart parsing, cookies, and HTTP utilities.
25 Conclusion and future work: Planned extensions include concurrent model versioning and A/B testing, adaptive GPU batching, GraphQL, built-in observability, and agentic workflows.These plans cover traffic policies, latency-budgeted micro-batches, shared type-driven infrastructure, OpenTelemetry integration, and structured tool-using loops.