Surface, Deep, and Dark Web: OSINT Applications

There are many metaphors for the internet: an iceberg, an ocean, a superhighway, or even a coffee shop with a thousand rooms. In reality, the sheer size of the internet can be hard to conceptualize. Think billions of users, over a billion web pages, and more and more data in the form of text, images, and videos being added every second. 

We generally think that everything on the internet is accessible using a search engine; however, common search engines only scratch the surface of information. The World Wide Web is divided into three parts: the Surface Web, the Deep Web, and the Dark Web. This article provides an overview of each segment and how you can use it to your investigative advantage. 

Surface Web

The Surface Web is what the average person interacts with most on a day-to-day basis. It accounts for approximately 4% of the internet and includes all content that can be indexed by search engines like Google, Bing, or Yahoo. For example, YouTube videos, Wikipedia articles, and blog posts are all located on the Surface Web. On the Surface Web, OSINT tools can gather intel from public sources, as well as monitor forums, social media platforms, and phishing pages for potential threats and information. At this level of the internet, cybercriminals are less likely to reveal personal information, leave tracks, or make mistakes because they know it is all publicly available. Information is also more likely to be true because people know that they are posting content to a public audience. 

The biggest investigative challenge when using the Surface Web is the volume of available data. Searching through all relevant web pages and zeroing in on the critical information can take hours, which is where Skopenow comes in to automate and optimize the process. Skopenow is an open source intelligence (OSINT) tool that uses artificial intelligence, machine learning, and natural language processing to find and analyze publicly available information, which includes all Surface Web data.

Deep Web

Though the Deep Web and the Dark Web together make up the 96% of the internet that is unavailable to search engines, they are very distinct. The Deep Web refers to any web page that can’t be directly indexed by a search engine. This could be because it requires login information to view the content, or it contains very niche information and therefore has not yet been indexed. Emails, bank statements, or a company’s internal database are all examples of the Deep Web. 

Search engines can lead you to websites that have Deep Web content, but they are not able to index those specific pages. For example, a search engine can index some information from a Facebook page, including name, cover/profile photo, and other information depending on the user’s privacy settings, but further information and posts are hidden behind a login and, therefore, are Deep Web.

OSINT tools can collect and analyze parts of the Deep Web, including social media content and federal court records that usually require an account to view, as well as web pages that are not yet indexed. In addition to viewing Deep Web content, Skopenow views these pages anonymously.

Dark Web

The remaining level of the internet is the infamous Dark Web, which is defined by anonymous content only accessible by using specific software. The most popular search engine software to access Dark Web information is called TOR, which uses worldwide proxy servers to give the user completely anonymous browsing.

Although scary tales of the Dark Web are plentiful, in reality, it isn’t all bad. The Dark Web can be a place where users are able to remain anonymous, work in isolation, and find information that is not available through normal channels. Users can search freely without cookies tracking them, the Dark Web allows for a free exchange of ideas without fear of persecution. 

However, because the Dark Web is known for its anonymity and can’t be accessed by traditional means, it does foster criminal activity. Criminals use the Dark Web to buy and sell illegal goods and substances, including data from security breaches, as well as organize more blood-curdling transactions. A Norwich University graphic states that the Dark Web has $100,000,000 in yearly revenue from illegal transactions. Though it is legal to access the Dark Web, participating in any illicit activity can be prosecuted. 

Governments and investigators use the Dark Web to search for criminals, protect political dissenters, and gather potential threat information. Security analysts also monitor the Dark Web for potential data leaks in their organization so that they can quickly fix the problem before news of the leak gets to a more accessible level.

Skopenow will speed up and optimize your Surface and Deep Web investigations, but other, specific OSINT tools are needed to access the Dark Web. 

If you are interested in conducting Dark Web investigations, check out this article where Skopenow’s Product Manager and OSINT expert, Jake Creps, details which OSINT tools to use for Dark Web investigations. For low tech discovery, he recommends Hunchly Dark Web, and for advanced research, he discusses TorBot. Once you’ve found what you’re looking for, he recommends Onioff to inspect potentially malicious links and TorCrawl to extract code from the services’ webpage. Before accessing the Dark Web, make sure to do your research and remain anonymous.

Skopenow is an analytical search engine that uses social media and open web data to provide actionable intelligence. Skopenow's platform identifies, collects, and analyzes public information on people and businesses by scouring millions of sources and data points. While Skopenow is built for use in insurance, government, and law, the product is also highly applicable in HR, real estate, and education.