Source-linked AI summary
Memento: Time Travel for the Web
Herbert Van de Sompel, Michael L. Nelson, Robert Sanderson, Lyudmila L. Balakireva, Scott Ainsworth, Harihar Shankar
TL;DR
The Web’s changing resources make prior representations difficult to reach because HTTP lacks temporal capabilities and archival URIs are disconnected from originals. Memento applies HTTP content negotiation in a datetime dimension to connect original URIs with archival resources. The paper reports a proof-of-concept framework that integrates CMS-hosted archives and web archives into regular navigation, while identifying cache handling and legacy-server support as remaining integration concerns.
Problem
HTTP exposes current resource states without a temporal dimension, making archived representations difficult for humans and software agents to discover.
Method
Memento applies datetime content negotiation and archive-resource discovery mechanisms to reach Mementos through an original resource’s URI.
Results
The proof-of-concept demonstrates temporal navigation across CMS-style and web archives using HTTP extended with DT-conneg.
Takeaways & Limitations
Memento integrates regular navigation and time travel while leveraging existing archives and adding an orthogonal time dimension to HTTP.
Takeaways & Limitations
The experiment requires further research on cache handling and client-side workarounds because existing servers do not detect X-Accept-Datetime.
Abstract
from arXiv · showhide
The Web is ephemeral. Many resources have representations that change over time, and many of those representations are lost forever. A lucky few manage to reappear as archived resources that carry their own URIs. For example, some content management systems maintain version pages that reflect a frozen prior state of their changing resources. Archives recurrently crawl the web to obtain the actual representation of resources, and subsequently make those available via special-purpose archived resources. In both cases, the archival copies have URIs that are protocol-wise disconnected from the URI of the resource of which they represent a prior state. Indeed, the lack of temporal capabilities in the most common Web protocol, HTTP, prevents getting to an archived resource on the basis of the URI of its original. This turns accessing archived resources into a significant discovery challenge for both human and software agents, which typically involves following a multitude of links from the original to the archival resource, or of searching archives for the original URI. This paper proposes the protocol-based Memento solution to address this problem, and describes a proof-of-concept experiment that includes major servers of archival content, including Wikipedia and the Internet Archive. The Memento solution is based on existing HTTP capabilities applied in a novel way to add the temporal dimension. The result is a framework in which archived resources can seamlessly be reached via the URI of their original: protocol-based time travel for the Web.
1. INTRODUCTION
The Web changes continuously, but HTTP exposes only current states, making prior representations difficult to discover despite growing archival resources. Memento addresses this gap by integrating archival content into ordinary navigation through a protocol-based temporal dimension.
- The Web’s memory problem: HTTP provides no temporal access to prior resource states, so retrieving an earlier representation becomes a significant discovery challenge.Web architecture favors keeping identifiers independent from changing resource states, rather than publishing a new URI for every change.
- Existing archival resources: Existing CMS version pages and web archives preserve historical representations, but their archival URIs remain disconnected from the original resource URI.Examples include Wikimedia history pages, WebCite, the Internet Archive, and recurrent web crawling services.
- The discovery challenge: Finding an earlier version still requires navigating version histories, searching site-specific archives, or querying multiple web archives.The paper illustrates the problem with Wikipedia’s potentially thousands of history entries and comparable workflows for news sites.
- Impact on applications: Poor integration of archival content hinders applications that analyze, compare, visualize, or search historical Web information.Such applications often build special-purpose archives ad hoc to support temporal functionality.
- Memento’s aim: Memento enables seamless access to archival content through regular Web navigation for both human and software agents.It leverages existing archives rather than addressing their creation, population, or maintenance.
2. CONTENT NEGOTIATION
Transparent HTTP content negotiation lets clients express representation preferences and lets servers choose a matching variant or expose available alternatives. The same framework supports negotiated media type, language, compression, and character set dimensions.
- Negotiation model: Clients request preferred representations through HTTP Accept headers, while servers select among variants associated with a transparently negotiable resource.Preferences can span multiple dimensions, including media type and language.
- Successful selection: A server can return HTTP 200 OK with TCN: Choice and Content-Location, or HTTP 302 Found with TCN: Choice and Location, when selecting a variant.Both responses identify the variant resource that supplies the representation.
- Unsuccessful selection: If no variant satisfies the client’s preferences, the server returns HTTP 406 Not Acceptable with TCN: List and available variant resources.The client can then make an informed selection from the returned list.
- Protocol details: RFC 2295 expresses variant lists through the Alternates response header and lets servers indicate which negotiation dimensions they support through Vary.Variant resources themselves do not support content negotiation.
3. THE MEMENTO SOLUTION
Memento extends Web navigation into the temporal dimension by combining datetime content negotiation with an archive API for discovering available historical resources. It defines stable archival records and the URI roles used to connect originals with their past representations.
- Core architecture: Memento’s two core building blocks are HTTP content negotiation in the datetime dimension and an API for inventories of archived resources.Together they support temporal navigation of the Web.
- Memento: A Memento is an archival resource whose representation remains identical to the representation previously available from an original resource at a specified time.Once created, a Memento keeps the same representation.
- URI terminology: The paper distinguishes an original resource, URI-R, from a Memento, whose URI is denoted URI-M.An original resource is not itself a Memento of another resource.
3.2 HTTP Datetime Content Negotiation
Datetime content negotiation lets a client request a past representation by sending a datetime preference with the original resource’s URI. Servers can return a matching Memento, list alternatives, or redirect the request when another server handles the archive.
- Datetime negotiation: DT-conneg uses the X-Accept-Datetime header to let clients request past rather than current representations.The header carries a datetime instead of a media type or similar representation preference.
- Request handling: A server receiving a DT-conneg request attempts to deliver a Memento matching the client’s datetime or provides available Memento variants.The client issues an HTTP GET at the original resource URI while expressing its archival datetime preference.
- Cross-server access: When the original-resource server cannot honor the request, it redirects the client to a server that can provide the requested archival resource.This distinguishes DT-conneg from ordinary content negotiation.
- Large inventories: Extensive variant inventories use a centered Alternates listing together with an HTTP Link header pointing to a larger list.This avoids requiring every available variant to fit in one Alternates header.
3.3 A TimeGate: A Resource Capable of DT-conneg
A TimeGate is a transparently negotiable resource whose variants are the archived versions, or Mementos, of an original resource.
- A TimeGate URI-G[URI-R] supports datetime negotiation for an original resource URI-R, with each variant resource being a Memento URI-Mi[URI-R@ti].
3.4 Time Travel: Combining DT-conneg and TimeGates
Memento combines datetime content negotiation with TimeGates so clients can request archived versions through original-resource navigation, whether archival knowledge is local or external.
- The framework can support multiple datetime values and multidimensional negotiation, although the scenarios simplify requests to one datetime dimension.
- Alternates offers a limited datetime-centered variant list, while the Link header points to a TimeBundle containing all Mementos and metadata.
- Web servers with archival capabilities: For servers with archival capabilities, URI-R acts as its own TimeGate and its Mementos become variant resources for datetime negotiation.
- Web servers without archival capabilities: For servers without local archives, detecting X-Accept-Datetime triggers a 302 redirect to an archival server’s TimeGate for URI-R.
3.5 Discovering Mementos: TimeBundles and TimeMaps
TimeBundles and TimeMaps organize an archive’s Mementos and related metadata, making archived versions discoverable through standard harvesting and navigation mechanisms.
- A TimeBundle aggregates an original URI-R, its available Mementos, and the archive’s TimeGate URI-G.
- A TimeMap models the TimeBundle as an ORE Aggregation and lists its aggregated resource URIs plus metadata such as archival datetime, media type, and language.
- Archives can expose TimeBundle URIs through HTTP Link headers and discoverability mechanisms including SiteMaps, Atom Feeds, and OAI-PMH.
3.6 A TimeGate Aggregator
A TimeGate aggregator harvests Memento information across archives, expanding coverage and improving navigation to versions near a requested datetime.
- An aggregator collects Memento URIs and metadata from multiple archives, then introduces TimeGates for original resources represented in the harvested collection.
- For the noaa.gov home page around Hurricane Katrina, cross-archive navigation could bridge an unavailable September 9, 2005 Memento to the next available version on September 10.
4. EXPERIMENT
The experiment demonstrates that Memento can support Web time travel across diverse servers and archives, despite limited native support in existing systems. Its components enable navigation to archival pages through HTTP-based datetime negotiation, with performance varying by flow and aggregation overhead.
- 4. EXPERIMENT: The deployed environment demonstrates the feasibility of Web time travel across diverse servers and archives using the Memento framework.The experiment jointly deployed components that realize time travel across various servers.
- 4. EXPERIMENT: By-proxy TimeGates and TimeBundles enabled archival integration for systems that did not implement the protocol natively.These components operated through servers under the authors’ control and obtained archival information using ad-hoc techniques.
- 4. EXPERIMENT: The experiment combined clients, configurable Web servers, archival services, and an aggregator to route datetime-based requests toward suitable Mementos.Clients issued X-Accept-Datetime requests, while the aggregator collected and merged TimeMap information before redirecting clients.
- 4. EXPERIMENT: Navigation generally resembled ordinary Web browsing across CMS-style archives and Web archives, while aggregator and by-proxy flows could be slow.Native-support, cache-bypassing, and limited-cache flows performed noticeably faster; batching, caching, and recurrent refreshing were identified as remedies.
- 4. EXPERIMENT: The experiment included real-time and time-travel navigations, with the latter replaying a browsing session at a specified October 12, 2009 datetime.Re-navigation requires archival records for the resources involved.
5. DISCUSSION
The discussion identifies integration challenges for Memento, including cache handling, archive self-containedness, and selecting suitable Mementos. It also illustrates how DT-conneg can support navigation across vanished resources, domains, and archival servers.
- 5. DISCUSSION: DT-conneg can recover Mementos when a resource URI vanishes, a domain disappears, or a domain changes custodians by consulting other archives or a TimeBundle aggregator.The client is redirected or redirected through archive infrastructure and selects an appropriate archived representation.
- 5. DISCUSSION: DT-conneg must bypass caches for datetime requests, while the experiment used no-cache and obsolete If-Modified-Since headers as an interim solution.The paper calls for a better approach that bypasses caches only when necessary while still using them to deliver Mementos.
- 5. DISCUSSION: Archived pages remain self-contained in current web archives, but that design prevents navigating to archived linked resources unavailable within the same archive.The paper leaves open a strategy combining self-containedness with beneficial external DT-conneg navigation.
- 5. DISCUSSION: Selecting the best Memento is difficult when no archive has an exact datetime match, especially for pages containing many embedded resources.Existing archives often choose the temporally closest Memento, but the paper notes that this choice becomes challenging for composite pages.
6. RELATED WORK
Prior work explored temporal browsing through interfaces, search techniques, and ad hoc archive access. Memento differs by proposing a protocol-level mechanism for integrating prior representations into ordinary Web navigation.
- 6. RELATED WORK: Zoetrope, the Past Web Browser, and DiffIE enhance temporal browsing through interfaces or client-specific mechanisms rather than protocol extensions.The paper positions DT-conneg as a standard access mechanism that could benefit these projects.
- 6. RELATED WORK: Lexical signatures, ErrorZilla, and the UK National Archive plug-in address disappeared pages by helping users find archived or replacement copies.The paper suggests that DT-conneg clients could react intelligently to 404 responses while leveraging these re-finding approaches.
- 6. RELATED WORK: TTApache provided archive access through ad hoc RPC-style mechanisms using query parameters tied to the original URI.Memento instead targets protocol-based temporal access integrated with ordinary Web navigation.
7. CONCLUSIONS
The paper connects Memento to the vision of interconnected, openly accessible archives while extending that vision to live Web servers hosting archival content. It presents time travel as an added HTTP dimension integrated with regular navigation.
- 7. CONCLUSIONS: Memento extends the global archival-grid vision by integrating web archives and live servers hosting archival content into the Web itself.The paper contrasts this with a separate navigation space merely resembling the live Web.
- 7. CONCLUSIONS: The solution adds an orthogonal time dimension to established HTTP practice without disrupting existing web archives or their operating principles.It leverages archival infrastructure while integrating temporal navigation with regular Web navigation.