When a Personal Website Disappears

A great deal of what was written on the open web between the late 1990s and the late 2000s is simply gone. Not moved, not paywalled, not superseded - gone, in the specific sense that the address still exists, resolves to something, and no longer holds what it held. Anyone who follows old citations for a living knows the pattern well. It is worth understanding how it happens, because the mechanism is mundane and largely avoidable.

A domain name is a lease

The first thing to be clear about is what a domain registration actually is. It is not property and it is not permanent. It is a fixed-term right to have a name resolve, bought from a registrar for a period of one to ten years, renewable indefinitely as long as somebody keeps paying and keeps the account reachable.

Nothing about a name is inherently yours. The registration is a contract with an expiry date, and the entire fragility of the personal web follows from that single fact. Hosting has the same shape: a monthly or annual arrangement that continues until a card is declined, a company is acquired, or a service is retired.

What actually happens at expiry

The sequence after a registration lapses is standardized across most common top-level domains, and it is slower than people expect.

1. Expiry. The name stops working, or begins pointing at a holding page served by the registrar. It has not yet been lost.
2. A grace period. For a number of weeks, the original registrant can usually renew at the ordinary price. Most recoveries happen here, when somebody notices the site is down.
3. Redemption. A further period, typically around thirty days, during which only the original registrant may restore the name, and only by paying a substantial redemption fee. The name is off the market but not yet released.
4. Pending delete. A short final window, usually a few days, in which nothing can be done at all.
5. The drop. The name returns to the available pool. Desirable names rarely sit there. Specialized services monitor the delete schedule and attempt registration in the same second the name becomes free, and a name with existing inbound links or search history may be resold immediately.

The whole cycle takes roughly two to three months. It is quite possible for a site to go dark, for its owner to assume a hosting fault, and for the name to be irrecoverable by the time anyone investigates.

Why the personal layer thinned

The disappearance is not evenly distributed. Corporate and institutional sites of the same vintage survive at far higher rates, and the reasons are structural rather than mysterious.

An organization has a budget line, a successor when a person leaves, and a legal interest in keeping its name resolving. A personal site has one maintainer, one payment method and no succession plan. When circumstances change - a move, an illness, a career, a lost interest, a death - there is nobody whose job it is to renew.

Two more forces did the rest. Much early personal publishing lived on free hosting attached to internet service providers and portal companies, and when those services were shut down they took hundreds of thousands of sites with them at a stroke. And the audience for personal publishing migrated into platforms, where a profile requires no payment, no software maintenance and no name of one's own - but where the writing lives at an address the writer does not control, subject to a company's decisions about export, retention and continued existence.

Link rot and reference rot

The consequences reach past the sites themselves. Anything that cited them degrades too, and the phenomenon has been studied carefully in scholarly and legal literature, where citations are supposed to be durable. Repeated studies of academic papers and judicial opinions have found that a large share of cited web addresses no longer resolve to the material that was cited.

Two distinct failures are usually bundled together:

• Link rot is the obvious case. The address returns an error, or nothing at all.
• Reference rot is subtler and worse. The address resolves perfectly well, returns a page, and that page no longer contains what the citation described. A reader who does not already know the original content has no way to detect the substitution.

Reference rot is worse precisely because it is silent. A broken link announces itself; a changed page does not.

What archives preserve, and what they do not

Web archiving is the partial remedy. Crawler-based archives fetch pages on a schedule and store the responses, and the largest of them, the Internet Archive's Wayback Machine, holds an enormous quantity of the ordinary web. National libraries take a more curated approach: the Library of Congress web archiving program builds thematic and event-based collections rather than attempting the whole web, on the reasonable theory that selection is itself a preservation decision.

What archives capture is more limited than most people assume. A crawler is a program requesting URLs, so:

• It can only archive what it can find. Pages nobody linked to, and pages excluded by a site's crawler directives, are often missing entirely.
• It captures moments, not continuity. A site crawled twice in five years is preserved as two snapshots, with everything between them lost.
• Content assembled in the browser by scripts frequently does not survive, because the archive stores what was sent, not what was rendered.
• Large media - video, audio, high-resolution images - is captured inconsistently, and embedded players that call out to a third-party service usually break.
• Anything behind a login, a search box or any other form is essentially invisible, because a crawler does not fill in forms.
• Comments and discussion loaded from an external service disappear with that service, even when the article around them is preserved.

An archived page is therefore a photograph of a room, not the room.

Writing to outlast the arrangement

None of this argues against publishing. It argues for a few unglamorous habits, all of which cost very little at the time and are impossible to apply retroactively.

• Keep the originals. Whatever is published, keep an authored copy on hardware you control, backed up in at least two places, one of them not in your house.
• Prefer open, boring formats. Plain HTML, plain text and Markdown will be readable in thirty years. A proprietary layout file or a database dump for software that no longer exists will not.
• Export on a schedule. If the writing lives inside a platform or a content management system, export it periodically in a documented format rather than trusting that the export button will still be there when it is needed.
• Keep URLs simple and stable. Short, lowercase, descriptive paths without session identifiers or query strings survive migrations. Every change of address breaks somebody's citation; when a move is unavoidable, redirect the old address rather than deleting it.
• Do not hide the writing from crawlers. If a page can only be reached by search box, no archive will hold it.
• Treat the renewal as the critical dependency. Register for multiple years, keep the registrar account details current, and make sure at least one other person knows the arrangement exists.

The last point is the one people find uncomfortable, and it is the important one. A domain name is rented. The writing is yours only in the sense that you keep a copy of it. Everything else is an arrangement with a company, and arrangements end.

This guide's own subject is a city rather than the web, and the rest of it can be found in the index of guides, on the front page, or explained in about this guide.