PaperlayerView exact revision on arXiv

Source-linked AI summary

Challenges in the Decentralised Web: The Mastodon Case

Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, Gareth Tyson

arXiv:1909.05801v1cs.NIcs.CRcs.CY

TL;DR

Decentralised web platforms promise openness and distributed ownership, but their uncoordinated operation raises questions about centralisation and resilience. This paper measures Mastodon’s instances and federation, finding concentration and outage risks while testing replication as a mitigation.

  • Problem

    Decentralised systems may face challenges in scaling, detecting wide-area malicious activity, and protecting users from data loss during instance outages.

    Source: §1 · p. 1

  • Method

    The authors conduct a large-scale Mastodon measurement study of instance- and user-level activity and federation using a 15-month dataset.

    Source: §1 · p. 1

  • Results

    10% of instances host almost half of users, while removing five ASes reduces the federation graph’s LCC from 92% to 46%.

    Source: §7 · p. 12

  • Takeaways & Limitations

    Copying toots onto secondary instances can dramatically improve availability in the authors’ replication experiments.

    Source: §7 · p. 12

    The authors plan longitudinal analysis and further study of hate speech associated with alt-right communities using Mastodon forks.

    Source: §7 · p. 12

Abstract

from arXiv · show

The Decentralised Web (DW) has recently seen a renewed momentum, with a number of DW platforms like Mastodon, Peer-Tube, and Hubzilla gaining increasing traction. These offer alternatives to traditional social networks like Twitter, YouTube, and Facebook, by enabling the operation of web infrastructure and services without centralised ownership or control. Although their services differ greatly, modern DW platforms mostly rely on two key innovations: first, their open source software allows anybody to setup independent servers ("instances") that people can sign-up to and use within a local community; and second, they build on top of federation protocols so that instances can mesh together, in a peer-to-peer fashion, to offer a globally integrated platform. In this paper, we present a measurement-driven exploration of these two innovations, using a popular DW microblogging platform (Mastodon) as a case study. We focus on identifying key challenges that might disrupt continuing efforts to decentralise the web, and empirically highlight a number of properties that are creating natural pressures towards recentralisation. Finally, our measurements shed light on the behaviour of both administrators (i.e., people setting up instances) and regular users who sign-up to the platforms, also discussing a few techniques that may address some of the issues observed.

1 Introduction

The paper uses Mastodon to examine how independent instances and federation shape decentralised social systems. Its measurements identify concentration and outage risks, alongside replication as a possible mitigation.

1 Introduction

  • Independent instances and federation offer locally operated services with cross-instance interaction, while decentralised ownership may improve transparency and robustness.The paper frames scaling, malicious activity detection, and protection from data loss during outages as challenges.Source: §1 · p. 1
  • A 15-month study of 67 million toots measures instance- and user-level activity, federation, and techniques that could reduce centralisation pressures.The analysis examines instance deployment and administrator behavior, then how federation affects system properties and availability.Source: §1 · p. 1
  • Mastodon has varied instance types, but 10% of instances host almost half of users; outages in five ASes reduce the LCC from 92% to 46%.The paper also reports 11% of instances unavailable for half the measurement period.Source: §1 · pp. 1–2
  • With the top 3 ASes offline, federation-based replication loses 11.4% of toots rather than 70% without replication.The proposed scheme persistently stores and indexes posts in followers’ instances.Source: §1 · p. 2

2 Mastodon

Mastodon lets administrators operate independent servers where users post and follow accounts. Federation connects users across servers, and the federated timeline also exposes users to remote posts beyond those they follow.

2 Mastodon

  • Mastodon instances are independent servers where users register, post to followers, and boost toots as they would retweet on Twitter.Mastodon is an open-source decentralised web server platform.Source: §2 · p. 2
  • Federation lets users follow accounts on other instances through their local instance, which subscribes to remote posts for its users.Users need only one account, on their local instance; the local instance performs remote subscriptions on their behalf.Source: §2 · p. 2
  • The federated timeline combines remote toots retrieved by all users on an instance, helping users discover posts beyond accounts they follow.The home timeline shows followed accounts’ posts, while the local timeline shows all posts generated on the same instance.Source: §2 · p. 2

3 Datasets

The study combines instance metadata and availability, public toots, and follower and federation graphs to examine Mastodon’s instances and federation. The datasets span a 15-month collection period, but toot coverage is incomplete.

3 Datasets

  • The study uses instance metadata and availability, historical toots, and follower and federation graphs to investigate instances and federation.The paper also compares Mastodon outage and social-network data with historical Twitter datasets.Source: §3 · pp. 2–3
  • The instance dataset includes 4,328 unique instances, monitored every five minutes from April 2017 to July 2018, yielding approximately half a billion data points.The instance list includes 3.1% Pleroma servers, which federate with Mastodon using ActivityPub.Source: §3 · pp. 2–3
  • The toot crawl collected 67M public toots from 239K users, covering 62% of the toot population; private posts and crawl-blocking account for the remainder.The crawl targeted 1.75K accessible instances in May 2018; 21.3% of instances went offline and never returned during the measurement period.Source: §3 · p. 3
  • The follower data produced a graph of 853K accounts and 9.25M links, with user accounts treated as separate nodes even when names recur across instances.A separate federation graph represents instance-to-instance subscriptions triggered by users following accounts on remote instances.Source: §3 · p. 3
  • Because 38% of toots were unavailable to the researchers, they avoid deeper semantic analysis of toot content and focus on instance-level activity.The missing posts were private or hosted on instances that blocked crawling.Source: §3 · p. 3
  • The collection includes public toot data, anonymized before analysis and stored securely, while the public dataset covers infrastructure information and anonymized toot metadata.The authors report removing text content from their analysis and obtaining an ethics committee waiver.Source: §3 · p. 3

4 Exploring Instances

Mastodon’s bottom-up instance deployment produces diverse communities and hosting arrangements, but usage, infrastructure, and availability show several forms of concentration. Instance policies and activity vary, while outages and third-party dependencies can affect users and content.

4.1 Instance Popularity

  • The top 5% of instances host 90.6% of users and 94.8% of toots, across both open and closed instances.Open instances average 613 users versus 87 on closed instances, but closed instances have a higher median active-user share: 75% versus 50%.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 Instance Categories

  • Among categorized instances, tech is common but adult instances attract disproportionate users: 12.3% of instances host 61.03% of users.Only 697 of 4,328 instances report a category, covering 13.6% of users and 14.4% of toots.Source: §4.2 · p. 4, Figure 3 · p. 5
  • Instance activity policies vary: 82% of categorized instances prohibit at least one activity, while advertising-allowing instances host 61% of users and 75% of toots.Spam is prohibited by 76% of instances; among instances allowing #NSFW, most also allow nudity (84.3%) and pornography (75.6%).Source: §4.2 · pp. 4–5, Figure 4 · p. 5

4.3 Instance Hosting

  • Japan hosts 25.5% of instances but 41% of users; the top five countries attract 93.66% of federated subscription links.Japan, the US, and France hold 89.1% of toots, and 32% of federated links are between instances in the same country.Source: Figure 5 · p. 5, §4.3 · pp. 5–6, Figure 6 · p. 6
  • Instances span 351 ASes, yet the top three account for 62% of users; Amazon hosts more than 30% of users on 6% of instances.The authors note that failure or policy changes at a small number of ASes could affect many users.Source: Figure 5 · p. 5, §4.3 · p. 6

4.4 Instance Availability

  • About half of instances have less than 5% downtime, but 11% are inaccessible more than half the time.Only outages that ended within the 15-month measurement cycle were counted; 4.5% of instances were up 99.5% of the time and covered 44% of toots.Source: Figure 7 · p. 6, §4.4 · p. 6
  • 10.95% average downtime for Mastodon instances versus 1.25% for Twitter; instance popularity is a poor availability predictor (correlation -0.04).Instances with over 1M toots had 2.1% median downtime, compared with 0.34% for instances with 100K–1M toots.Source: Figure 8 · p. 7, §4.4 · p. 7
  • Nearly all instances go down at least once; outages leave 14% of users unable to access their instances for a day at least once.Certificate expirations caused 6.3% of observed outages, while AS outages temporarily removed 4.98M toots and 41.5K user accounts.Source: Figure 9 · p. 7, §4.4 · pp. 7–8, Table 1 · p. 7

5 Exploring Federation

This section examines how Mastodon’s federated user graph and content replication respond to user, instance, and hosting-network failures. It finds substantial vulnerabilities from concentrated users and hosting, while replication—especially random replication—can improve toot availability.

5.1 Breaking the User Federation

  • The ten highest-toot instances are operated by companies, individuals, and crowd-funded groups, and the paper notes that stable, predictable funding would be desirable.Source: §5.1 · p. 8
  • Removing the top 1% of user accounts drops the follower graph’s LCC from 99.95% of users to 26.38%; removing the top 10% still leaves 80% in the LCC.Source: §5.1 · pp. 8–9
  • The federation graph’s LCC decays roughly linearly as large instances are removed, but individual instance failures can still have enormous effects on the social graph.Source: §5.1 · p. 9
  • Removing five ASes ranked by users fragments the federation graph into 272 components, versus 139 when ranked by instance count; the top five instance-ranked ASes reduce LCC size by 49%.The remaining LCC covers 45% of instances and 66% of users.Source: §5.1 · p. 10

5.2 Breaking the Content Federation

  • Remote toots dominate many federated timelines: 78% of instances produce under 10% of their own toots, and 5% produce none.Toot output and replication to other instances correlate at 0.97, suggesting influential instances act as feeders for smaller ones.Source: §5.2 · p. 10
  • The availability simulations compare no replication, replication to follower instances, and random replication, assuming a global index and treating all toots equally.Random replication copies each toot to a selected number of random instances; practical strategies would need to account for instance resources such as storage.Source: §5.2 · pp. 10–11
  • Removing the top 10 instances makes 62.69% of toots unavailable without replication, versus 2.1% with subscription replication; the corresponding AS figures are 90.1% and 18.66%.Source: §5.2 · p. 10
  • After 25 instance removals, random replication achieves 99.2% toot availability, compared with 95% for subscription-based replication.Subscription replication leaves 9.7% of toots without replicas and gives 23% of toots more than 10 replicas; the authors attribute the unevenness to popularity bias.Source: §5.2 · p. 11

6 Related Work

Prior work spans peer-to-peer and federated social networks, federation protocols, privacy research, and social-network measurement. This study adds a focus on Mastodon availability, deployment, and topical interests.

Decentralised Social Networks

  • Early peer-to-peer social networks did not achieve widespread adoption, partly because of performance and reliability challenges; Diaspora later grew but its population has stagnated.Source: §6 · p. 11
  • Federation protocols and data formats have been standardized, enabling Mastodon and PeerTube to exchange data through ActivityPub and ActivityStreams.Source: §6 · p. 11

Related Work

  • Separate DW security and privacy work has examined secure data management and decentralized personal datastores, including DataBox, SOLID, and SocialGate.Source: §6 · p. 11

Social Network Measurements

  • Prior Mastodon studies examined datasets, social graphs, instance placement, content warnings, friend recommendations, and sentiment; this paper adds availability, instance deployment, and topical interests.The authors describe it as the largest Mastodon study to date.Source: §6 · p. 12

7 Conclusion

The conclusion finds an active Mastodon instance ecosystem alongside substantial concentration in users, hosting, and content availability. Simulations identify failure risks and show that replication can improve availability, with random placement outperforming subscription-based replication.

7 Conclusion

  • Ten percent of instances host almost half of Mastodon’s users, while three ASes host almost two thirds; instances nevertheless span a wide range of topics.Source: §7 · p. 12
  • Removing five ASes can reduce the federation graph’s LCC from 92% to 46% of users, while outages in 10 instances can remove almost half of all toots.Source: §7 · p. 12
  • Copying toots to secondary instances can improve availability, and random replication outperforms subscription-based replication because follower-based copies cluster on the same instances.Source: §7 · p. 12
  • The authors warn that ignored centralization pressures may leave the DW semi-centralized and identify longer-term analysis and further mitigations as future work.They also argue that mitigations should not depend on exposing user information to remote parties.Source: §7 · p. 12
Loading 1909.05801v1…