Efforts underway to ensure no shortlink rots

Internet Archive and Archive Team running separate projects aimed at archiving shortened Web URLs, to ensure longevity of such links.
Written by Liau Yun Qing, Contributor on

Web sites that use Universal Resource Locators (URL) shortening services risk losing page ranks and links to their content and could be forced to replace all shortened URLs, if companies offerings such services close down. To safeguard against this predicament, some organizations are stepping in to save shortened URL links.

In early August, URL shortening service Trim announced it will end its services in December but later said it will continue to "run "indefinitely".

Regardless, the potential threat of service termination has prompted efforts to save shortened URLs, including Archive Team's TinyURL Project, which aims to protect TinyURL links, and Internet Archive, which operates a project called 301Works.

"One day, a large service will go away and people will start to '="" title="URL shortening is hot--but look before you leap -- Tuesday, Jul. 07, 2009" target="_blank" rel="noopener noreferrer nofollow">think twice about using these services if there's no backup somewhere," Steve Webb from Archive Team, told ZDNet Asia in an e-mail.

301Works' project director Stowe Boyd said in an e-mail interview the project was originally formed by a group of URL shortening services, but is now part of the Internet Archive. He added that 301Works has a set of operating principles for companies that wish to participate and has also attracted 40 participating URL shortening services.

Explaining how the initiative works, Boyd noted that participating service providers will archive both long and corresponding shortened URLs in its database. Should a service be terminated, 301Works will continue to redirect existing shortened URLs at the service provider's domain, if it gives 301Works technical control.

"If they do not give technical control of their domain, at the very least, we will be able to resolve short URLs into long URLs one at a time at a new Web page we will be launching," he said.

Boyd said the project will benefit Internet users as they would not need to be concerned about broken links, and large media companies with "millions of shortened URLs will not have to rip them out of millions of pages and replace them".

He noted that 301Works acts as a "final guarantor of shortened URLs", and will only redirect short URLs if the service is shut down.

Manual URL archiving
Retrieving archived links is slightly different at Archive Team. Webb said his team has registered a domain to host the archived shortened URLs, and also offers a FireFox plug-in to facilitate redirects.

"If you have the FireFox plug-in, you can just go to the site as usual and even if [the shortened link] is gone, the plug-in will reference our site instead and give you the long version of the URL," he said.

Unlike 301Works, which works with the service providers, Archive Team "scrapes" URL shortening services by ripping URLs. In a nutshell, it crawls to retrieve additional details of the long URLs.

Webb said: "We have managed to archive a handful of smaller URL shortening services and some larger ones, like bit.ly and tinyurl.com. However, it has been slow-going."

He is hoping that URL shortening services will collaborate with Archive Team by allowing the company to "aggressively" scrape their Web sites.

"Or they can just provide a monthly dump of their database, which would make for less effort and less bandwidth for both parties," he said. "We're happy to archive their content for them--they just need to allow us to it."

According to Boyd, it would not be practical for Web sites to run their own URL shortening service. He likened such moves to creating their own programming tools or browser. "Many URL shortening services offer additional capabilities, like traffic statistics, which are overly expensive to develop," he said.

Editorial standards