Content deleted Content added
No edit summary Tag: Reverted |
not clear this is notable |
||
| (18 intermediate revisions by 13 users not shown) | |||
Line 2:
{{About|searching the World Wide Web}}
{{pp|small=yes}}
{{Full citations needed|date=July 2021}}
[[File:A screenshot of suggestions by Google Search when "wikip" is typed (new).png|thumb
A '''
▲A '''web search engine''' is a [[software system]] that provides [[hyperlink]]s to [[web page]]s and other relevant information on [[World Wide Web|the Web]] in response to a user's [[web query|query]]. The user [[search box|inputs]] a query within a [[web browser]] or a [[mobile app]], and the [[search engine results page|search results]] are often a list of hyperlinks, accompanied by textual summaries and images. Users also have the option of limiting the search to a specific type of results, such as images, videos, or news.
For a search provider, its [[software engine|engine]] is part of a [[distributed computing]] system that can encompass many [[data center]]s throughout the world. The speed and accuracy of an engine's response to a query is based on a complex system of [[Search engine indexing|indexing]] that is continuously updated by automated [[web crawler]]s. This can include [[data mining]] the [[Computer file|files]] and [[database]]s stored on [[web server]]s, but some content is [[deep web|not accessible]] to crawlers.
There have been many search engines since the dawn of the Web in the 1990s, but [[Google Search]] became the dominant one in the 2000s and has remained so. It currently has a 91% global market share.<ref>{{Cite web |title=Search Engine Market Share Worldwide {{!}} StatCounter Global Stats |url=http://gs.statcounter.com/search-engine-market-share |access-date=19 February 2024 |website=StatCounter}}</ref><ref name="NMS">{{cite web | url=https://www.similarweb.com/engines/ | title=Search Engine Market Share Worldwide | access-date=19 February 2024 | website=Similarweb Top search engines}}</ref> The business of [[website]]s improving their visibility in [[search results]], known as [[search engine marketing|marketing]] and [[search engine optimization|optimization]], has thus largely focused on Google.
== History ==
Line 306 ⟶ 304:
===Pre-1990s===
In 1945, [[Vannevar Bush]] described an information retrieval system that would allow a user to access a great expanse of information, all at a single desk.<ref>{{Cite web |last=Bush |first=Vannevar |date=1945-07-01 |title=As We May Think |url=https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ |access-date=2024-02-22 |website=The Atlantic |language=en|archive-url=https://web.archive.org/web/
[[Link analysis]] eventually became a crucial component of search engines through algorithms such as [[Hyper Search]] and [[PageRank]].<ref>{{Cite web|title=The Quest for Correct Information on the Web: Hyper Search Engines|first1=Massimo|last1=Marchiori|work=Proceedings of the Sixth International World Wide Web Conference (WWW6)|date=1997|url=http://www.w3.org/People/Massimo/papers/WWW6/Overview.html|access-date=2021-01-10}}</ref><ref name="AnatomyOfSearch">{{cite web|title=The Anatomy of a Large-Scale Hypertextual Web Search Engine|last1=Brin|first1=Sergey|last2=Page|first2=Larry|work=Proceedings of the Seventh International World Wide Web Conference (WWW7)|date=1998|url=http://ilpubs.stanford.edu:8090/361/1/1998-8.pdf|access-date=2021-01-10|archive-date=2017-07-13|archive-url=https://web.archive.org/web/20170713070157/http://ilpubs.stanford.edu:8090/361/1/1998-8.pdf|url-status=dead}}</ref>
Line 313 ⟶ 311:
The first internet search engines predate the debut of the Web in December 1990: [[WHOIS]] user search dates back to 1982,<ref>{{cite journal|url=https://tools.ietf.org/html/rfc812|title=RFC 812 - NICNAME/WHOIS|newspaper=Ietf Datatracker|year=1982 |doi=10.17487/RFC0812 |last1=Harrenstien |first1=K. |last2=White |first2=V. |doi-access=free }}</ref> and the [[Knowbot Information Service]] multi-network user search was first implemented in 1989.<ref>{{cite web|url=http://www.cnri.reston.va.us/home/koe/iwooos-full.html|title=Knowbot programming: System support for mobile agents|work=cnri.reston.va.us}}</ref> The first well documented search engine that searched content files, namely [[FTP]] files, was [[Archie search engine|Archie]], which debuted on 10 September 1990.<ref>{{cite web|url=https://groups.google.com/forum/#!msg/comp.archives/LWVA50W8BKk/wyRbF_lDc6cJ|title=[next] An Internet archive server server (was about Lisp)|last=Deutsch|first=Peter|date=September 11, 1990|website=groups.google.com|access-date=2017-12-29}}</ref>
Prior to September 1993, the [[World Wide Web]] was entirely indexed by hand. There was a list of [[webserver]]s edited by [[Tim Berners-Lee]] and hosted on the [[CERN]] [[Web server|webserver]]. One snapshot of the list in 1992 remains,<ref>{{cite web|url=http://www.w3.org/History/19921103-hypertext/hypertext/DataSources/WWW/Servers.html |title=World-Wide Web Servers |publisher=W3C |access-date=2012-05-14}}</ref> but as more and more web servers went online the central list could no longer keep up. On the [[National Center for Supercomputing Applications|NCSA]] site, new servers were announced under the title "What's New!".<ref>{{cite web|url=http://home.mcom.com/home/whatsnew/whats_new_0294.html |title=What's New! February 1994 |publisher=Mosaic Communications Corporation! |access-date=2012-05-14}}</ref>
The first tool used for searching content (as opposed to users) on the [[Internet]] was [[Archie search engine|Archie]].<ref name=LeidenUnivSE>{{cite web |url-status=dead |work=Internet History |title=Search Engines |author1=Search Engine Watch |author-link1=Search Engine Watch |publisher=Universiteit Leiden |location=Netherlands |date=September 2001 |url=http://www.internethistory.leidenuniv.nl/index.php3?c=7 |archive-url=https://web.archive.org/web/20090413030108/http://www.internethistory.leidenuniv.nl/index.php3?c=7 |archive-date=2009-04-13 }}</ref> The name stands for "archive" without the "v".<ref name="2020/09/21pcmag"/> It was created by [[Alan Emtage]],<ref name="2020/09/21pcmag">{{cite web | title = Archie | url = https://www.pcmag.com/encyclopedia/term/archie | publisher=[[PCMag]]| access-date = 2020-09-20 }}</ref><ref>{{cite web | author = Alexandra Samuel| title = Meet Alan Emtage, the Black Technologist Who Invented ARCHIE, the First Internet Search Engine| date = 21 February 2017| url = https://daily.jstor.org/alan-emtage-first-internet-search-engine/ |publisher= [[ITHAKA]]| access-date = 2020-09-20 }}</ref><ref>{{cite web | author = loop news barbados | title = Alan Emtage- a Barbadian you should know | url = http://www.loopnewsbarbados.com/content/alan-emtage-barbadian-you-should-know | publisher = loopnewsbarbados.com | access-date = 2020-09-21 | archive-date = 2020-09-23 | archive-url = https://web.archive.org/web/20200923065914/http://www.loopnewsbarbados.com/content/alan-emtage-barbadian-you-should-know | url-status = dead }}</ref><ref>{{cite web | author = Dino Grandoni, Alan Emtage | title = Alan Emtage: The Man Who Invented The World's First Search Engine (But Didn't Patent It)| date = April 2013| url = https://www.huffingtonpost.co.uk/entry/alan-emtage-search-engine_n_2994090?ri18n=true&guccounter=1&guce_referrer=aHR0cHM6Ly9jb25zZW50LnlhaG9vLmNvbS8&guce_referrer_sig=AQAAABveQefuoczW_8_bxwbOgluVTUPvIfv5s_OP1jMgUJd8MCwKc148lvXb7HAHXY48P_Be6wXMW0LKlLRfQzJNalLpuwnp7F6NpbyDC2BG10OveS2qtubkO0PhJ8-juP3M2a9K2ygbWuoUhOCvO-1NA6-YQKA8BtdZEcsfUUI_M-8S | publisher= [[huffingtonpost]].co.uk|access-date = 2020-09-21 }}</ref> [[computer science]] student at [[McGill University]] in [[Montreal, Quebec]], Canada. The program downloaded the directory listings of all the files located on public anonymous FTP ([[File Transfer Protocol]]) sites, creating a searchable [[database]] of file names; however, [[Archie search engine|Archie Search Engine]] did not index the contents of these sites since the amount of data was so limited it could be readily searched manually.
Line 323 ⟶ 321:
In June 1993, Matthew Gray, then at [[Massachusetts Institute of Technology|MIT]], produced what was probably the first [[web robot]], the [[Perl]]-based [[World Wide Web Wanderer]], and used it to generate an index called "Wandex". The purpose of the Wanderer was to measure the size of the World Wide Web, which it did until late 1995. The web's second search engine [[Aliweb]] appeared in November 1993. Aliweb did not use a [[web robot]], but instead depended on being notified by [[Webmaster|website administrators]] of the existence at each site of an index file in a particular format.
[[JumpStation]] (created in December 1993<ref>{{cite web |url=http://archive.ncsa.uiuc.edu/SDG/Software/Mosaic/Docs/old-whats-new/whats-new-1293.html |archive-url=https://web.archive.org/web/20010620073530/http://archive.ncsa.uiuc.edu/SDG/Software/Mosaic/Docs/old-whats-new/whats-new-1293.html |archive-date=2001-06-20 |title=Archive of NCSA what's new in December 1993 page |date=2001-06-20 |access-date=2012-05-14 |url-status=dead }}</ref> by [[Jonathon Fletcher]]) used a [[web crawler|web robot]] to find web pages and to build its index, and used a [[web form]] as the interface to its query program. It was thus the first [[World Wide Web|WWW]] resource-discovery tool to combine the three essential features of a web search engine (crawling, indexing, and searching) as described below. Because of the limited resources available on the platform it ran on, its indexing and hence searching were limited to the titles and headings found in the [[Web page|web pages]] the crawler encountered.
One of the first "all text" crawler-based search engines was [[WebCrawler]], which came out in 1994. Unlike its predecessors, it allowed users to search for any word in any
The first popular search engine on the Web was [[Yahoo! Search]].<ref>{{cite web |title=What is first mover? |url=https://searchcio.techtarget.com/definition/first-mover |website=SearchCIO |publisher=[[TechTarget]] |access-date=5 September 2019 |date=September 2005}}</ref> The first product from [[Yahoo!]], founded by [[Jerry Yang]] and [[David Filo]] in January 1994, was a [[Web directory]] called [[Yahoo! Directory]]. In 1995, a search function was added, allowing users to search Yahoo! Directory.<ref>{{cite book |last1=Oppitz |first1=Marcus |last2=Tomsu |first2=Peter |title=Inventing the Cloud Century: How Cloudiness Keeps Changing Our Life, Economy and Technology |date=2017 |publisher=Springer |isbn=9783319611617 |page=238 |url=https://books.google.com/books?id=vrEvDwAAQBAJ&pg=PA238}}</ref><ref>{{cite web |title=Yahoo! Search |url=https://www.yahoo.com/search.html |archive-url=https://web.archive.org/web/19961128070718/http://www.yahoo.com/search.html |url-status=dead |archive-date=28 November 1996 |website=Yahoo! |access-date=5 September 2019 |date=28 November 1996}}</ref> It became one of the most popular ways for people to find web pages of interest, but its search function operated on its web directory, rather than its full-text copies of web pages.
Line 331 ⟶ 329:
Soon after, a number of search engines appeared and vied for popularity. These included [[Magellan (search engine)|Magellan]], [[Excite (web portal)|Excite]], [[Infoseek]], [[Inktomi (company)|Inktomi]], [[Northern Light Group|Northern Light]], and [[AltaVista]]. Information seekers could also browse the directory instead of doing a keyword-based search.
In 1996, [[Robin Li]] developed the [[RankDex]] site-scoring [[algorithm]] for search engines results page ranking<ref>Greenberg, Andy, [https://www.forbes.com/forbes/2009/1005/technology-baidu-robin-li-man-whos-beating-google.html "The Man Who's Beating Google"], ''Forbes'' magazine, October 5, 2009</ref><ref>Yanhong Li, "Toward a Qualitative Search Engine", ''IEEE Internet Computing'', vol. 2, no. 4, pp. 24–29, July/Aug. 1998, {{doi|10.1109/4236.707687}}</ref><ref name="rankdex">[http://www.rankdex.com/about.html "About: RankDex"], ''rankdex.com''</ref> and received a US patent for the technology.<ref>USPTO, [https://
In 1996, [[Netscape]] was looking to give a single search engine an exclusive deal as the featured search engine on Netscape's web browser. There was so much interest that instead, Netscape struck deals with five of the major search engines: for $5 million a year, each search engine would be in rotation on the Netscape search engine page. The five engines were Yahoo!, Magellan, Lycos, Infoseek, and Excite.<ref>{{Cite web|url=http://files.shareholder.com/downloads/YHOO/701084386x0x27155/9a3b5ed8-9e84-4cba-a1e5-77a3dc606566/YHOO_News_1997_7_8_General.pdf|title=Yahoo! And Netscape Ink International Distribution Deal|access-date=2009-08-12|archive-url=https://web.archive.org/web/20131116112021/http://files.shareholder.com/downloads/YHOO/701084386x0x27155/9a3b5ed8-9e84-4cba-a1e5-77a3dc606566/YHOO_News_1997_7_8_General.pdf|archive-date=2013-11-16|url-status=dead}}</ref><ref>{{cite news |date=1 April 1996|title=Browser Deals Push Netscape Stock Up 7.8% |newspaper=Los Angeles Times |url=https://
[[Google]] adopted the idea of selling search terms in 1998 from a small search engine company named [[Yahoo! Search Marketing|goto.com]]. This move had a significant effect on the search engine business, which went from struggling to one of the most profitable businesses in the Internet.{{Citation needed|date=January 2024}}
Search engines were also known as some of the brightest stars in the Internet investing frenzy that occurred in the late 1990s.<ref>{{cite journal |last=Gandal |first=Neil |year=2001 |title=The dynamics of competition in the internet search engine market |journal=International Journal of Industrial Organization |volume=19 |issue=7 |pages=1103–1117 |doi=10.1016/S0167-7187(01)00065-0 |url= http://www.escholarship.org/uc/item/0h17g08v |issn=0167-7187}}</ref> Several companies entered the market spectacularly, receiving record gains during their [[initial public offering]]s. Some have taken down their public search engine and are marketing enterprise-only editions, such as Northern Light. Many search engine companies were caught up in the [[dot-com bubble]], a speculation-driven market boom that peaked in March 2000.
===2000s–present: Post dot-com bubble===
Around 2000, [[Google Search|Google's search engine]] rose to prominence.<ref>{{cite web|url=https://www.google.com/about/company/history/ |title=Our history in depth |access-date=2012-10-31 |url-status=deviated |archive-url= https://web.archive.org/web/20121101210037/https://www.google.com/about/company/history/ |archive-date=
By 2000, [[Yahoo!]] was providing search services based on Inktomi's search engine. Yahoo! acquired Inktomi in 2002, and [[Yahoo! Native|Overture]] (which owned [[AlltheWeb]] and AltaVista) in 2003. Yahoo! switched to Google's search engine until 2004, when it launched its own search engine based on the combined technologies of its acquisitions.
Line 375 ⟶ 373:
The usefulness of a search engine depends on the [[relevance (information retrieval)|relevance]] of the ''result set'' it gives back. While there may be millions of web pages that include a particular word or phrase, some pages may be more relevant, popular, or authoritative than others. Most search engines employ methods to [[rank order|rank]] the results to provide the "best" results first. How a search engine decides which pages are the best matches, and what order the results should be shown in, varies widely from one engine to another.<ref name=Jawadekar2011/> The methods also change over time as Internet usage changes and new techniques evolve. There are two main types of search engine that have evolved: one is a system of predefined and hierarchically ordered keywords that humans have programmed extensively. The other is a system that generates an "[[inverted index]]" by analyzing texts it locates. This first form relies much more heavily on the computer itself to do the bulk of the work.
Most Web search engines are commercial ventures supported by [[advertising]] revenue and thus some of them allow advertisers to [[paid inclusion|have their listings ranked higher]] in search results for a fee. Search engines that do not accept money for their search results make money by running [[contextual advertising|search related ads]] alongside the regular search engine results. The search engines make money every time someone clicks on one of these ads.<ref>{{cite web|title=how search engine works?|url=http://globalforumonline.com/detail/how-does-search-engine-works/|publisher= GFO | access-date = 26 June 2018}}</ref>
=== Local search ===
Line 381 ⟶ 379:
==Market share==
{{As of|2022|01|post=,}} [[Google Search|Google]] is by far the world's most used search engine, with a market share of 90.6%, and the world's other most used search engines were [[Microsoft Bing|Bing]], [[Yahoo! Search|Yahoo!]], [[Baidu]], [[Yandex Search|Yandex]], and [[DuckDuckGo]].<ref name="NMS" /> In 2024, Google's dominance was ruled an illegal monopoly in a case brought by the US Department of Justice.<ref>{{cite web |url=https://www.npr.org/2024/05/02/1248152695/google-doj-monopoly-trial-antitrust-closing-arguments |website=[[NPR]] }}</ref>
<graph>{
Line 556 ⟶ 554:
Their project was fully funded by mid-1993. Once funding was secured. they released a version of their search software for webmasters to use on their own web sites. At the time, the software was called Architext, but it now goes by the name of Excite for Web Servers.<ref name="wileyhistory"/>
Excite was the first serious commercial search engine which launched in 1995.<ref>{{cite web|title=The Major Search Engines|url=http://www.pccua.edu/kholland/major_search_engines.htm|accessdate=1 June 2014|date=21 January 2014|archive-date=5 June 2014|archive-url=https://web.archive.org/web/20140605052335/http://www.pccua.edu/kholland/major_search_engines.htm|url-status=dead}}</ref> It was developed in Stanford and was purchased for $6.5 billion by @Home. In 2001 Excite and @Home went bankrupt and [[InfoSpace]] bought Excite for $10 million.
Some of the first analysis of web searching was conducted on search logs from Excite<ref>Jansen, B. J., Spink, A., Bateman, J., and Saracevic, T. 1998. [https://faculty.ist.psu.edu/jjansen/academic/jansen_sigir_forum.pdf Real life information retrieval: A study of user queries on the web]. SIGIR Forum, 32(1), 5 -17.</ref><ref>Jansen, B. J., Spink, A., and Saracevic, T. 2000. [https://faculty.ist.psu.edu/jjansen/academic/pubs/jansen_real_life_real_users_and_real_needs.pdf Real life, real users, and real needs: A study and analysis of user queries on the web]. Information Processing & Management. 36(2), 207–227.</ref>
Line 640 ⟶ 638:
* [[Information retrieval]]
* [[Internet search engines and libraries|Use of web search engines in libraries]]
* [[Itpints]]
* [[List of search engines]]
* [[Question answering]]
| |||