Source-linked AI summary
A Placement Vulnerability Study in Multi-tenant Public Clouds
Venkatanathan Varadarajan, Yinqian Zhang, Thomas Ristenpart, Michael Swift
TL;DR
Multi-tenant public clouds create co-residency risks, but placement vulnerabilities in modern clouds had not been systematically studied. The paper evaluates EC2, GCE, and Azure with new detection tests and launch strategies, finding co-location 10x more successful and up to $114 cheaper than under a secure reference policy.
Problem
The study asks whether adversaries can still arrange and detect co-residency in modern public clouds after stronger isolation technologies were adopted.
Method
The authors use multiple customer accounts, co-residency tests, varied launch strategies, and a reference placement policy to evaluate three cloud providers.
Results
10x higher success rate and up to $114 lower cost were obtained across EC2, GCE, and Azure compared with the secure reference placement policy.
Takeaways & Limitations
Modern isolation blocks many prior techniques, but insufficient hardware performance isolation still permits co-location detection and co-location remains simple and cheap.
Takeaways & Limitations
The experiments used one instance type, one availability zone per region, and one victim account, with at most 9 runs per configuration.
Abstract
from arXiv · showhide
Public infrastructure-as-a-service clouds, such as Amazon EC2, Google Compute Engine (GCE) and Microsoft Azure allow clients to run virtual machines (VMs) on shared physical infrastructure. This practice of multi-tenancy brings economies of scale, but also introduces the risk of sharing a physical server with an arbitrary and potentially malicious VM. Past works have demonstrated how to place a VM alongside a target victim (co-location) in early-generation clouds and how to extract secret information via side- channels. Although there have been numerous works on side-channel attacks, there have been no studies on placement vulnerabilities in public clouds since the adoption of stronger isolation technologies such as Virtual Private Clouds (VPCs). We investigate this problem of placement vulnerabilities and quantitatively evaluate three popular public clouds for their susceptibility to co-location attacks. We find that adoption of new technologies (e.g., VPC) makes many prior attacks, such as cloud cartography, ineffective. We find new ways to reliably test for co-location across Amazon EC2, Google GCE, and Microsoft Azure. We also found ways to detect co-location with victim web servers in a multi-tiered cloud application located behind a load balancer. We use our new co-residence tests and multiple customer accounts to launch VM instances under different strategies that seek to maximize the likelihood of co-residency. We find that it is much easier (10x higher success rate) and cheaper (up to $114 less) to achieve co-location in these three clouds when compared to a secure reference placement policy.
1 Introduction
Multi-tenancy lowers cloud costs but permits co-residency attacks, while the ability to arrange co-residency in modern clouds remains insufficiently understood. This study systematically evaluates whether modern providers remain vulnerable and develops more reliable detection and launch strategies.
- Multi-tenancy shares physical infrastructure among arbitrary customers, reducing costs but enabling attacks by co-resident VMs.Co-residency can breach confidentiality through side channels or degrade victim performance.
- The study addresses whether co-residency can be detected, whether known and new launch strategies work, and what success costs and time.
- The paper develops more reliable co-residency tests and a novel public-interface test for victim VMs behind load balancers.
- 10x higher success rate and up to $114 lower cost were achieved by new launch strategies across EC2, GCE, and Azure versus a secure reference policy.The experiments used multiple customer accounts and at least 190 runs per provider.
2 Background
Public-cloud placement maps user-controlled launch choices to physical hosts, but providers’ undocumented policies can expose observable locality patterns. Attackers can exploit these patterns using launch strategies and co-residency tests, including in some PaaS settings.
- VM placement algorithms select physical hosts, and placement may depend on load, datacenter size, concurrent launches, and other factors.
- Users can influence placement variables such as time of day, requested datacenter, and instance count, combining their values into launch strategies.An example is launching 20 instances 10 minutes after triggering a victim autoscale event.
- Observable locality patterns include parallel placement locality for closely timed launches and sequential placement locality after terminating one instance before launching another.
- Attackers exploit placement behavior by triggering victim scale-up and launching multiple VMs simultaneously or after a delay.
- 6 out of 10 Heroku attempts achieved co-location when two applications were scaled to 30 instances with five-minute spacing, at no cost.The investigation used simple detection and was not extended to further PaaS study.
3 Threat Model
The threat model treats placement vulnerability as cost-effective co-location beyond a conservative reference policy. It formalizes attacker resources, strategy cost, success-rate normalization, and cost-benefit comparison.
- The attacker is resource-limited, uses ordinary VM interfaces, lacks internal placement-policy knowledge, and may use unlimited accounts.
- A launch strategy’s cost is the expense of attacker VMs and co-residency detection across attacker–victim VM pairs.The paper gives CS = a·P(atype)·Td(a,v).
- The reference policy randomly assigns VMs to machines, providing a security yardstick for evaluating real-cloud placement policies.Its simplifying assumptions are conservative because they benefit the attacker.
- A placement policy is vulnerable exactly when some launch strategy has normalized success rate greater than 1.Normalized success rate measures ease of co-location, while cost-benefit measures the additional reference-policy cost for equal success.
4 Detecting Co-Residence
Modern cloud isolation changes undermine prior co-residency tests, so the study develops performance-based tests for cooperative and uncooperative victims. The tests use memory contention and public-service latency to identify co-residency across heterogeneous cloud environments.
- Motivation: Prior co-residency detection techniques became unreliable in modern clouds after VPCs, persistent-storage changes, and multi-package CPUs reduced observable shared state.VPCs prevent cloud cartography through private internal IP addresses, while local storage and LLC-based channels are less reliable.
- Design goals: The study evaluates detection methods using goals of heterogeneous-stack applicability, high confidence, and sufficiently fast testing.The authors choose a performance covert channel based on shared hardware resources because such channels are difficult to remove and likely remain exploitable.
- Co-residency tests: Memory-probing and memory-locking tests compare receiver performance with an idle sender against performance during concurrent sender activity, declaring co-residency above a slowdown threshold.The sender creates memory contention, while the receiver measures degradation in memory-related performance.
- Threshold tuning: AMD machines in Azure were highly sensitive to the detection threshold, while a 1.5 threshold produced no false positives and cross-architecture false detections were not observed.The authors attribute the AMD sensitivity possibly to cache associativity or different handling of locked instructions.
- Test scalability: 2.4 seconds per pair versus 46.3 seconds per pair reduced testing time by 20× when co-residency detection was parallelized.The authors selected this parallelization approach for simplicity despite faster alternatives.
- Uncooperative victims: The adapted memory-probing test detects co-residency with uncooperative web servers by measuring public-interface request latency under baseline and memory-contention conditions.The method was evaluated on a two-tier Olio application with web and database servers running in separate EC2 VMs.
5 Placement Vulnerability Study
The study evaluates placement vulnerabilities across EC2, GCE, and Azure using repeated multi-account experiments that vary launch and placement conditions. It measures co-residency with three metrics and compares cloud behavior against a reference placement policy.
- Experiment design: The evaluation varies launch strategies across EC2, GCE, and Azure to assess placement vulnerabilities and co-residency success.Each run launches VMs from two distinct accounts and checks co-residency between all launched VM pairs.
- Implementation: The methodology automates cloud interaction and co-residency testing, using libcloud for EC2 and GCE and a CLI wrapper for Azure.Azure lacked an explicit interface for simultaneous VM launches.
- Experiment design: The study reuses victim instances across delay configurations to reduce experimental cost.Attacker instances are terminated and relaunched after the specified delays under the leap-frog methodology.
- Measurement: Co-residency is measured by success rate, average co-resident instances, and average co-resident pairs across repeated runs.The experiments vary one placement variable while keeping the others fixed.
5.2 Effect of Number of Instances
The study examines how attacker and victim population sizes affect co-residency across the three clouds. Increasing attacker counts helps on EC2 and GCE, whereas Azure exhibits inverse behavior; increasing victim counts raises co-residency chances across all providers.
- Varying attacker instances: Higher attacker-to-victim instance ratios increase co-residency chances on GCE and EC2.Larger attacker fractions also increase the number of co-resident VM pairs and potential victim coverage.
- Varying attacker instances: Azure shows the opposite pattern: 10 attacker instances yield almost twice the co-residency chance of 30 attacker instances.
- Varying attacker instances: Equal victim and attacker counts produce similar co-residency chances in the 10x10, 20x20, and 30x30 configurations with zero delay on Azure.
- Varying victim instances: Increasing the number of victim VMs raises co-residency chances across EC2, GCE, and Azure.Azure’s absolute chance remains lower than in the other clouds, possibly because its launch delays were longer under the methodology.
5.3 Effect of Instance Launch Time
Launch timing affects placement differently across clouds. EC2 avoids co-residency for near-simultaneous launches, while non-zero delays can help on GCE and zero delay can help on Azure; longer delays do not eliminate co-residency on EC2 and GCE.
- Launch delay: EC2 instances launched within a short time window are never co-resident on the same machine.Even sequential launches denoted 0+ can result in co-residency once the short exclusion window is avoided.
- Launch delay: Non-zero delays increase co-residency chances on GCE, while zero delay increases them on Azure.Azure’s actual launch delays averaged 3 hours and reached 10 hours in some runs because of the leap-frog methodology.
- Longer delays: Co-residency does not fall to zero on EC2 or GCE even after delays of 16 or 32 hours.The authors speculate that constant churn may create fresh co-residency as neighboring VMs terminate.
- Time of day: Time of day has only a small placement effect, with slightly higher chances in early-morning EC2 runs and nighttime GCE runs.
5.4 Effect of Data Center Location
Smaller, less popular cloud regions show higher co-residency rates than larger, more popular regions. They also produce more co-resident pairs, suggesting broader coverage across victim VMs.
- Regional placement: Smaller regions provide higher co-residency chances than larger, more popular regions.At least one co-resident pair appeared in every run conducted in the smaller regions.
- Regional placement: Higher numbers of co-resident pairs in smaller regions suggest greater coverage over victim VMs.
- Observed anomaly: Two EC2 20x20 runs showed an unusually large number of co-resident instances, including three VMs from one account.The authors believe this anomaly may have resulted from an internal management incident in us-west-1.
5.5 Other Observations
The study reports additional placement behaviors across Azure, GCE, and EC2, alongside measurements of launch time, cost, and reference-policy probability.
- Azure and GCE allowed more than two VMs to share a host; one run placed 16 VMs on a single host.EC2 did not exhibit this behavior.
- Launch-strategy execution time was measured across victim-versus-attacker configurations, with error bars showing standard deviation across at least seven runs.
- Many self-co-resident VM pairs appeared on GCE and Azure, whereas none were observed on EC2 except for an anomaly in us-west-1.The authors observed no effect on the actual chance of co-residence but believe the behavior may affect VM placement.
- The experiments also successfully placed a few co-located medium instances on both EC2 and GCE using strategies learned with small instances.
- Strategy costs included compute-instance charges and accounted for EC2 hourly billing versus per-minute billing on GCE and Azure.Network and storage costs were considered insignificant.
5.7 Summary of Placement Vulnerabilities
The authors identify launch strategies whose normalized co-location success exceeds a uniform-random reference policy, establishing placement vulnerabilities across all three clouds.
- A placement vulnerability exists when a launch strategy’s normalized success rate exceeds 1 relative to the reference policy.The normalized success rate compares co-location under the strategy with the reference-policy probability.
- 1.0/0.10 was achieved in Azure by launching ten attacker VMs closely after the victim VMs.
- 1.0/0.10 was achieved in EC2 and GCE smaller datacenters by launching at least ten attacker VMs with a non-zero delay after known victims.
- 1.00/0.60 was achieved in all three clouds with 30 attacker instances against 30 victims, using no delay in Azure or one hour in EC2 and GCE.
- In EC2, an optimal strategy used a one-hour attacker delay during 02:00–10:00 PST, achieving 0.89/0.60.
- $114 cheaper was the cost advantage of the best strategies, S1 and S2, across all three providers versus the reference policy.
5.8 Limitations
The study’s conclusions are bounded by the placement variables, account configurations, and repetition counts used in its experiments.
- Only one instance type, one availability zone per region, and one victim account were used, limiting how broadly the placement results can be generalized.Different instance types may exhibit different placement behavior.
- Using one victim account could have captured an unlucky account whose placement decisions favored co-residence with the attacker account.This caveat applies if account ID influences VM placement.
- At most nine runs per configuration, including three runs per time of day, constrained the experimental replication despite at least 190 runs per provider.The experiments ran over a period of three months.
6 Related Work
Related work established placement attacks and co-residency detection methods, while this study revisits placement vulnerabilities under newer cloud technologies and across three providers.
- VM placement vulnerability studies: Earlier studies showed high-probability co-location attacks using public or network-observable placement behaviors, but their results were outdated by newer cloud technologies.
- VM placement vulnerability studies: A concurrent EC2 study focused on co-residency detection and older classic networking, whereas this work studies placement vulnerabilities with VPC, Azure, and GCE.The authors characterize the studies as mostly complementary.
- VM placement vulnerability studies: Proposed defensive placement policies have been studied, but their compatibility with public-cloud performance and reliability goals remains unclear.
- Co-residency detection techniques: Co-residency detection techniques are categorized into side-channel approaches for uncooperative VMs and covert-channel approaches for cooperative VMs.
- Co-residency detection techniques: Side-channel methods can detect co-residency with attacker-uncontrolled VMs, including network timing and last-level-cache timing approaches.
- Co-residency detection techniques: Covert-channel methods confirm co-residency when both VMs cooperate by communicating through shared hardware components such as caches, disks, or memory buses.
7 Conclusion and Future Work
The paper finds that co-location remains surprisingly simple and inexpensive in modern public clouds despite the failure of past verification techniques. These results motivate smarter placement policies that better balance efficiency and security.
- Co-location in EC2, GCE, and Azure is surprisingly simple and cheap despite modern cloud isolation technologies.
- Past techniques for verifying co-location no longer work, but insufficient hardware performance isolation still permits detection.
- The chances of co-location are far higher than expected even in datacenters containing numerous physical servers.
- Smarter co-location-aware placement policies are needed to improve the balance between cloud efficiency and security.