Seamless Transitions: A Comprehensive Review of Live Migration Technologies

1 Introduction

• Live migration transfers a running computational entity between hosts while maintaining continuous operation and client connectivity.The review notes applications in Edge, cloud, and HPC, with benefits including flexibility, scalability, fault tolerance, and resource efficiency.Source: §1 · p. 2
• The review addresses prior reviews’ limited treatment of technical fundamentals and practical challenges by examining technologies in real-world operational scenarios.It organizes the analysis around migration technique, migration unit, infrastructure characteristics, and migration objective.Source: §1 · p. 2
• Technique, migration unit, infrastructure conditions, and objectives each shape live-migration efficiency and effectiveness across contexts.Examples include pre-copy, post-copy, and hybrid techniques; containers and VMs; network conditions; and objectives such as downtime or resource use.Source: §1 · p. 2
• The review surveys the state of the art, highlights where technologies excel and where innovation is needed, and provides practical implementation guidelines.Sections address techniques, units, infrastructure characteristics, and migration objectives.Source: §1 · pp. 2–3

2 Scope of the work and relevant studies

• Live migration moves a running entity between hosts with little to no service disruption, unlike cold migration, which permits disruption.Process migration is grouped into no migration, cold migration, and live migration.Source: §2 · p. 3
• Migration time and downtime are key performance measures, but acceptable interruption varies by use case, making a universal fixed definition impractical.The passage attributes this context dependence to differing use cases, technical intricacies, and performance metrics.Source: §2 · p. 3
• Prior reviews cover taxonomies, computing environments, management perspectives, energy optimization, VM methods, and migration metrics.Examples include container techniques across cloud, Fog, and Edge; VM strategies; and Fog comparisons of migration time, downtime, and transferred data.Source: §2 · pp. 3, 5
• This review extends generic and management-centric coverage with technical comparisons of technologies, including their advantages, disadvantages, and trade-offs.Table 1 summarizes recent studies, with • marking broad discussion and ◦ marking partial discussion.Source: Table 1 · p. 4, §2 · p. 5

3.1 Pre-copy

• Pre-copy transfers memory while the process runs, repeatedly sending dirty pages before pausing the process for final state transfer and resumption.Challenges include non-convergence and limited parallelism during dumping and recovery; proposed responses include enhanced page servers, activity detection, Auto-converge, and behavior prediction.Source: §3.1 · pp. 5–6

3.2 Post-copy

• Post-copy resumes the process at the target before memory transfer, fetching missing pages on demand and reducing copy redundancy relative to pre-copy.It has lower overhead but risks data loss if failure occurs before transfer completes; the passage reports almost no real-world adoption due to lower reliability.Source: §3.2 · p. 6
• Hybrid methods combine pre-copy memory transfer with post-copy process-state transfer to pursue reliability and efficiency; speculative variants can help with predictable workloads.The passage also notes that stop-and-copy may count as live migration when downtime remains below an interactive application's timeout.Source: §3.2 · p. 6

3 Migration Techniques

• VM- and container-level migration are the most practical current approaches, balancing granularity trade-offs with portability, resource efficiency, and ecosystem support.Larger units can distribute overhead but waste fast memory when data is unused; smaller units offer control but may increase transfer overhead.Source: §3.2 · pp. 6–7

Checkpoint/Restore Tools

• Checkpoint/restore tools capture a running process’s state and recreate it, operating at system, user, or application level with distinct constraints and goals.Table 2 provides an overview of the tools discussed.Source: Checkpoint/Restore Tools · p. 7, Table 2 · p. 7

3.2.1 Berkeley Lab Checkpoint/Restart

• BLCR was a hybrid kernel/user checkpoint/restart tool for Linux clusters, but it is no longer actively maintained and lacked SysV shared-memory support.It targeted HPC applications and integrated with MPI implementations including MVAPICH and Open MPI.Source: §3.2.1 · p. 7

3.2.2 Distributed MultiThreaded CheckPointing

• DMTCP checkpoints multithreaded distributed applications in user space without kernel modifications, but requires a linked library from process start and can miss calls such as inotify().Separate plugins for network interconnects can be costly to maintain; MANA addresses this through a split-process mechanism for MPI applications.Source: §3.2.2 · p. 8

3.2.3 Checkpoint/Restore In Userspace

• CRIU uses Linux kernel features and supports container migration, incremental memory checkpointing, and direct page transfer through its Page-Server mechanism.The cited discussion concludes that CRIU is viable for individual containers when time constraints are not a primary concern.Source: §3.2.3 · pp. 8–9

3.2.4 Process Hauler

• P.Haul orchestrates CRIU’s migration phases, especially for pre-copy, but synchronizes root filesystems and cannot avoid sending redundant OCI image layers.The passage notes limited research focused solely on P.Haul and reports OpenVZ and LXC attempts to use it.Source: §3.2.4 · p. 9

3.2.5 Fault Tolerance Interface

• FTI is a user-level checkpoint/restore library for HPC that allocates host buffers; DMTCP is described as more versatile due to Slurm integration and RDMA compatibility.Source: §3.2.5 · p. 9

Platform Specific Tools

• Platform-specific options include OpenShift namespace-level workload migration, Fedora process or container migration without service interruption, and Hyper-V VM-state capture and transfer.The supplied discussion identifies MTC for OpenShift and describes Fedora Checkpoint Restore and Hyper-V Checkpoint Restore.Source: Platform Specific Tools · pp. 9–10

Emerging Tools

• Emerging tools target heterogeneous migration: H-Container supports cross-ISA applications, HETCOM uses trusted hardware, and Wharf uses vessels and docks across ISAs and operating systems.The authors of Wharf report checkpoint and restore time averaging 14× faster than CRIU.Source: Emerging Tools · p. 10

• The migration unit is the computational entity transferred between hosts during migration.Source: §4 · p. 10
• Containers, groups of containers (pods), and virtual machines are common migration units, each requiring distinct approaches and tools.Source: §4 · p. 10
• Unit selection affects complexity, resource requirements, and performance, and depends on data volume, transfer time, and system compatibility.Source: §4 · p. 10

4.1 Container

• Containers package applications with their components in portable environments and generally incur less virtualization overhead than virtual machines.Source: §4.1 · p. 10
• Application containers package one application and its dependencies, while system containers provide an isolated user space and can run multiple services using the host kernel.Source: §4.1 · p. 11
• Container engines provide image management and build processes, while runtimes create and run containers; this review uses “container technology” for both.Source: §4.1 · p. 11

4.1.1 Linux Container

• LXC uses namespaces and cgroups and integrates CRIU, but does not yet support checkpointing network namespaces; LXD adds features including live migration.Source: §4.1.1 · p. 12

4.1.2 Docker

• Docker migration work reduces overhead through image-layer reuse and tools including Sledge and Voyager, though Voyager depends heavily on network-attached storage.Source: §4.1.2 · p. 12

4.1.3 Podman

• Podman integrates CRIU with pre-copy, but stateful checkpointing may encounter systemd-state issues, active TCP connections require a flag, and reliable migration requires root permission.Source: §4.1.3 · p. 13

4.1.4 Apptainer

• Apptainer lacks built-in CRIU support, and its image format and library-stack bypass complicate checkpointing; checkpoint/restart remains on its roadmap despite workarounds.Source: §4.1.4 · p. 13

4.2 Virtual Machine

• VM live migration transfers running virtual machines between physical hosts, despite challenges involving large memory states and network connections.Source: §4.2 · p. 13
• Most production VM live-migration implementations rely on hypervisors using pre-copy or post-copy techniques.Source: §4.2 · p. 13

4.2.1 VMware vSphere

• vSphere vMotion migrates workloads between hosts for balancing and maintenance without restarting VMs, unlike vSphere High Availability.Source: §4.2.1 · p. 14

4.2.2 Microsoft Hyper-V

• Hyper-V transfers running VMs between physical hosts, with best practice recommending a dedicated VLAN for migration traffic.Source: §4.2.2 · pp. 14–15

4.2.3 Xen

• Xen uses pre-copy by default, supports storage and block migration without shared storage, and allows migration to KVM for various Linux distributions.Source: §4.2.3 · p. 15

4.2.4 Kernel-based Virtual Machine

• KVM supports migration while the VM remains powered on, connected to the network, and running applications, as well as checkpointing for later resumption.Source: §4.2.4 · p. 15

4.2.5 Quick Emulator

• QEMU supports iterative pre-copy and fault-triggered post-copy; multifd parallelizes memory transfer, while mappedram enables concurrent writes to fixed RAM-page offsets.Source: §4.2.5 · p. 15

4.2.6 OpenStack

• OpenStack supports VM migration on KVM-libvirt and Xen under different node and storage requirements; high dirty-page rates may cause migration failure.Source: §4.2.6 · pp. 15–16

Hybrid and Emerging Solutions

• OpenVZ reduces management overhead by treating containers as regular processes; compared with LXC, it uses fewer resources but takes longer to migrate.LXC has shorter total migration times and downtimes, while OpenVZ is less intrusive.Source: Hybrid and Emerging Solutions · p. 16
• Harvester runs VMs and containers on bare metal using Kubernetes, KubeVirt, and Longhorn, but migration between nodes with different CPU models remains problematic in some scenarios.The review notes this issue in particular for virtual machines.Source: Hybrid and Emerging Solutions · p. 16
• OpenShift supports live migration of containers and VMs with shared RWX storage and can import VMs from other hypervisors; these platforms reflect convergence across virtualization methods.The review also describes proprietary cloud and specialized Edge migration offerings.Source: Hybrid and Emerging Solutions · p. 16
• Hypervisor-based VMware vMotion and KVM/QEMU offer robust, transparent VM migration with minimal overhead, broad compatibility, and strong cloud and enterprise integration.The review positions VM-level solutions as most effective for general-purpose and cloud workloads.Source: Hybrid and Emerging Solutions · p. 17
• Process-level CRIU and DMTCP provide fine-grained migration, but have limits in transparency, platform integration, and support for distributed or HPC workloads.DMTCP offers stronger distributed-application support at the cost of increased overhead.Source: Hybrid and Emerging Solutions · p. 17

5 Infrastructure Characteristics

• Network conditions and resource availability directly affect migration performance, efficiency, and reliability, shaping the overall strategy alongside environmental constraints.The review identifies these as infrastructure characteristics relevant across migration settings.Source: §5 · p. 17

5.1 Network Conditions

• WAN migration can take longer and increase downtime because bandwidth is typically lower than LAN bandwidth; IP reassignment, network failures, and concurrent migrations add planning demands.Replication and erasure coding can mitigate data-loss risk, while scheduling and bandwidth utilization help reduce service disruptions.Source: §5.1 · p. 17

5.2 Resource Availability

• Migration requires sufficient CPU/GPU, memory, and storage on source and target hosts, plus compatible software and hardware stacks and timely resource allocation.These requirements influence live-migration strategies.Source: §5.2 · p. 19

5.3 Cluster Environment

• GPU migration is harder than CPU migration because GPUs maintain extensive internal state; support depends on the technology and setup, and some approaches cannot handle long-running GPU workloads practically.gMig requires GPU tasks to finish before migration and supports only Intel GPUs; periodic GPU-memory checkpointing can also add significant overhead.Source: §5.3 · p. 19

5.4 Cloud Environment

• Cloud multi-tenancy requires resource management to maintain isolation and performance, while elasticity can provide resources for migration; balancing migration time with quality of service remains challenging.Most cloud live-migration studies focus on VMs; container migration research remains limited, and Kubernetes checkpoint/restore support is at an early stage.Source: §5.4 · p. 20

5.5 Edge and Fog Environment

• Edge migration prioritizes downtime, migration time, and latency, and is primarily container-based on resource-constrained servers; resource-rich Fog environments often use VM migration.The review also describes studies addressing VM handover decisions and container migration in Edge settings.Source: §5.5 · pp. 20–21

6 Discussion and Conclusion

• Migration objectives such as service availability and energy efficiency can conflict, and may even discourage live migration despite its benefits.Environmental characteristics guide the choice of migration unit and technique, but objectives ultimately determine the decision.Source: §6 · p. 21
• Bare-metal migration remains impractical, especially in heterogeneous environments; VM migration is mature and widely adopted, while container migration is faster and lower-overhead but still faces complex-scenario challenges.Container migration is approaching production readiness but is still gaining broader acceptance, especially for latency-sensitive applications.Source: §6 · p. 21
• The review expects container migration to advance through hardware support, better data-intensive performance, and cloud-native integration, alongside security, Edge-to-cloud, automation, and stateful-application improvements.It also identifies automated DevOps workflows and data-management integration as ways to document migration processes and support reproducibility.Source: §6 · p. 22

Funding

• Funding came from Germany’s Federal Ministry of Education and Research, the Saxon State Ministry for Science, Culture and Tourism, and the German Research Foundation; funders had no role in the study or writing.The listed projects include ScaDS.AI and NFDI4DataScience.Source: Funding · p. 22

• Checkpointing entries include BLCR (Berkeley Lab Checkpoint/Restart), C/R (Checkpoint/Restore), CRIU (Checkpoint/Restore In Userspace), and DMTCP (Distributed MultiThreaded CheckPointing).Source: Glossary · p. 23
• Migration-related entries include MTC (Migration Toolkit for Containers), MTV (Migration Toolkit for Virtualization), LIMOCE (Live Migration of Containers in the Edge), and MANA (MPI-Agnostic Network-Agnostic Transparent Checkpointing).Source: Glossary · pp. 23–24
• Virtualization and container terms include NPT (Nested Page Table), QEMU (Quick Emulator), KVM (Kernel-based Virtual Machine), LXC (Linux Container), and LXD (Linux Container Daemon).Source: Glossary · pp. 23–24
• Other compute and migration entries include P.Haul (Process Hauler), PID (Process Identifier), POS (PARALLELGPUOS), PV (paravirtualization), and GPU-P (GPU partitioning).Source: Glossary · pp. 23–24
• Infrastructure and networking abbreviations include RDMA (Remote Direct Memory Access), RHV (RedHat Virtualization), RPC (Remote Procedure Call), RWX (ReadWriteMany), and VLAN (Virtual Local Area Network).Source: Glossary · p. 24
• The glossary also expands terms for service levels, security, cloud computing, and systems, including SLA, TEE, TPM, IaaS, IoT, and HPC.Source: Glossary · pp. 23–24