Learning Diary: Deep Web: An introduction

Today in my learning Diary I am going to write about Deep web or the invisible web. Since I have made a paper dealing with Deep web I can explain it a little better than the other concepts that I am trying to learn and explain.

We all google for whatever we want to know, don’t we? But do we know that only 20% of data on the internet is queried and displayed when we search? Any search engine of today queries just this 20% data visible to them while the rest of the data goes unnoticed by the search engines. The data that we see with web searches constitute the surface web while the rest is called the deep or invisible web.

Let us take an example to understand the difference better. When results are released by your university, say anna university, what do we do? We go to their site that hosts the result type our roll number and we see our scores against each subject. Now, try typing your roll number in google or yahoo. Do you get a result page with your marks? You won’t. Don’t get surprised as the marks with your rollnumber is in the deep web which the search engines cant index.

The anna university page that gives you the result queries a web database that’s private to anna university. This database is called the web database. Google or yahoo or any search engine can’t query this database. But one can see any result by querying from the annauniversity page, so it’s not literally protected. It is public but invisible to search engines. The result page which shows your marks is called the deep web page.

Another example of Deep web is the ecommerce site data. Take any site for example, let’s take amazon. When we search for a book, say harry potter, we get a result page (the deep web page), the results are queried from amazon’s web database. Say the book costs Rs.100. Now do the same in a different site, say Flipkart.com. We get a result page (deep web page) which shows results querying from Flipkarts web database.

Why are we bothered about the Deep web pages or databases? Imagine if a search engine is able to query all of these web databases wont it be useful to buy the product at the lowest price possible?

This is just one example and use of deep web. 80% additional data will make us do wonders.

There are some deep web search engines.  Read more about Deep web and keep yourself informed.

The paper I worked on can be found Here.

Have a nice day! Please comment or mail me any doubts so that I will learn and try to answer them.