Web search has become a part of life. Users, living daily with this activity, often expect that they can locate anything on the web using common search tools, such as Google or Bing.
But consider: the web is massive. Anything ever uploaded, anytime resides there. The size is hard for us to comprehend. On it's surface, we find the indexed web. This includes sites that, through demand, have been indexed by search engines.
Deeper, if we have excellent research skills, we will encounter the other 90%--the invisible, the cloaked, or the deep web. BrightPlanet, a company that specializes in searching the deep web, estimates that with nearly 550 billion individual documents on the web currently, with only one billion of those reside on the surface where average search engines can find them.
Search engines like Google and Bing use "spiders" or "robots" to search the open web. These programs are limited in what they can see, and therefore, index. Results returned to us are generally dictated by tags and other "meta data" submitted by the creator that allows search engines to find, rank, and push information to the top of the search results.
But, how do we access items archived in databases or other less accessible places--the content often referred to as the invisible web or deep web. By some estimates[see info graphic at the right], Google and similar search engines have indexed just a little over 8% of the web. They can easily locate another 25% of what's published because it is presented in a static format (i.e. the content of the page doesn't change).
Google and other search engines CAN locate some information tucked deep into the web. Try searching for your topic along with the keyword "database." Use the tools below to search deeper and reach the valuable resources not visible to Google and other standard search engines.
Learn more with The Ultimate Guide to the Invisible Web from the Open Education Database.
Graphic based on:
Gil, Paul. "Cloaked Internet: What Is the 'Invisible Web'?." About Internet for Beginners: Your Guide to Participating in Online Culture and Navigating the Web and Internet.. N.p., n.d. Web. 21 Oct. 2013.
Google has it's own tool for helping you to mine the invisible web. Google Scholar is a great tool and always worth your time. Still, the searches are limited. Not all returns will be available in full text, and not everything on the invisible web will surface. Use Google Scholar in tandem with other deep search tools to harvest more!
Badgerlink, a project of the Wisconsin Department of Public Instruction (DPI), seeks to promote information literacy and access throughout the for all Wisconsin residents in cooperation with the state's public, school, academic, and special libraries. You can search the databases individually, or you can search them all at one time by going to theBadgerlink SuperSearch.
Provided as a public service by BrightPlanet, this is a listing of databases (70,000+), grouped categorically. " By tracing through CompletePlanet's subject structure or searching Deep Web sites, you can go to various topic areas, such as energy or agriculture or food or medicine, and find rich content sites not accessible using conventional search engines."
Library Information and Media Center - Monona Grove High School - Monona, Wisconsin
Answers| Catalog | Guides | Resources | Teachers