How the Dark Web works

Beneath our everyday internet lurks a murky network of encrypted sites known as the Dark Web. Is it all bad? No. But it does fuel a lucrative criminal subculture that could threaten businesses and consumers.
Written by Dan Patterson, Contributor

Image: iStock / Sitikka

The Dark Web is an ominous network of shadowy hackers hellbent on stealing company data, overthrowing the country, and selling drugs to your kids with Bitcoin.

Or is it? The hidden and encrypted internet enables hackers and activists and criminals. It's also a wonderful source for shocking headlines and salacious YouTube stories, and a communication and privacy-enhancing platform. Powered by a network of encrypted websites and accessible only by using a complex set of security tools, the Dark Web is as intriguing as it is beguiling. To understand the realities of the hidden internet, better grab a flashlight.

The Dark Web and the deep web are terms often confused and used interchangeably. The deep web is a term that refers to sites and pages unavailable to the general public and not indexed by traditional search engines, like corporate intranet sites, private social media posts, and pages with nofollow search tags.

SEE: Cybersecurity spotlight: The ransomware battle (Tech Pro Research)

Above the deep web hovers the clearnet, the traditional internet and mobile web used by billions of people around the world. The clearnet is secure, and encryption is used to move secure data from place to place all the time. SSL guards passwords and protects credit card information during e-commerce transactions. But the very nature of the clear internet is that anonymity is rare. Computer and mobile IP addresses are constantly logged and easily traced. Cookies help web marketers track online activity and analyze behavior.

What differentiates the so-called Dark Web is the method by which sites are accessed. The Dark Web, or darknet, is a network of sites with encrypted content, accessible only with a secure suite of secure-browsing tools, like Tor. Tor -- an acronym for the onion router -- is a package of open-source security tools written for a customized version of the Mozilla Firefox browser, compatible with Windows, OS X, and Linux. The software encrypts user traffic and passes the IP address through the complex of Tor nodes.

These 'onion layers' help protect the user's anonymity and provide access to similarly protected websites. These sites range from forums to wiki pages to blogs and function much like clearnet sites. Dark Web domains frequently employ non-memorable, hashed URLs with the .onion top level domain. These sites block inbound traffic from all non-secure internet connections.

Personal and work computers often house mission-critical data, like sensitive files, passwords, and health records. Because Tor can be used and the Dark Web can be accessed on a traditional home PC, security professionals rely on additional security tools like the Tails operating system. Tails is a Linux distribution that can be installed on and run from a portable flash drive. By accessing the Dark Web via Tails, user behavior is never logged locally, and it is significantly more challenging for malicious software to harm the host PC.

The Dark Web is used frequently by good actors for legitimate reasons. Encryption, security, and privacy are championed by news organizations, tech companies, universities, and activists in repressive regimes. The U.S. State Department helps fund the Tor project, and according to the United Nations, encryption is a fundamental human right. Facebook operates a widely used secure Dark Web portal to the social network.

SEE: Down the Deep Dark Web is a movie every technologist should watch (TechRepublic)

Yet it is also true that the Dark Web is an opaque, sometimes twisted, reflection of the clearnet. Crime is profligate. Black markets enable the morally libertine to profit handsomely in Bitcoin. The most famous Dark Web market, the Silk Road, allowed vendors and buyers to conduct business anonymously and enabled the sale of drugs, guns, humans, identities, credit card numbers, zero-day exploits, and malicious software. The site was raided and shut down by the FBI in 2013, but the idea of an anonymous, encrypted black market spread rapidly. Today, the site Deep Dot Web lists dozens of Dark Web markets.

"The Dark Web operates a lot like the clear web," said Emily Wilson, Director of Analysis at security firm Terbium Labs. "The same crime that happens off line, all the time, also happens on the Dark Web." In many ways, she said, because it's relatively easy to visit Dark Web markets, it's sometimes easier to see criminal activity as it happens.

Although it's not necessary for the layperson to visit the Dark Web often, if ever, every consumer is at risk of identity theft and should have a basic understanding of how the encrypted internet functions. Businesses should be aware that data from hacked companies and the government is easy to find and purchase on the encrypted internet. A number of companies, including Tripwire, ID Agent, and Massive, monitor the Dark Web and help businesses respond to Dark Web data leaks.

The Dark Web is not entirely malicious, but it's also not a safe place to visit. Novices and experts alike should exercise care and caution when visiting the Dark Web. ZDNet does not condone illegal or unethical activity. Offensive material can sometimes be just a click away. Browse at your own risk. Never break the law. Use the Dark Web safely, and for legal purposes only.

SEE: Network Security Policy (Tech Pro Research report)

The Dark Web -- like encryption -- is a double-edged sword. The hidden internet enables both good and bad actors to work uninhibited anonymously. And like encryption, the Dark Web is a reality for both consumers and business. Companies need to know about the Dark Web, Wilson said, and they need to be prepared for incidents to occur.

But consumers and companies shouldn't overreact to perceived threats. The Dark Web is not enormous. "Compared to the clearnet, the Dark Web is maybe a few thousand, or few hundred thousand [sites.]," Wilson explained. "Only a few thousand return useful content, and compared to the clearnet there's tiny amount of regular Tor users."

Read more

Editorial standards