This section explains various aspects of Search Engines web sites. The  portal concept, the differences between directories and crawlers, and  some terrific tool boxes will all be reviewed.

"Portal" is one of the current big buzzwords about the Internet now. That's because they promise to have the most traffic, which means being capable of making a lot of money. Portal stocks like Yahoo and Excite have been doing extremely well. There are a variety of definitions, but most industry people  pretty much agree that a portal combines a variety of Internet services, like E-Mail and Chat, designed to draw traffic into themselves and on through to others.

When I asked industry professionals about the characteristics of a portal, I received the following:

A portal service is a site that gives users an entrance to the Internet.
It aggregates information and services and adds value by presenting the
best of the Web in a highly organized format. It contains the basics of
commerce, content, community and communication and should have as it's
highest goal making people's lives easier.

Amanda Higgins,
Infoseek

High traffic and aggregated services, including search, news, email and commerce.

Andrew de Vries
Director of Marketing Communications
Wired Digital  / Hot Bot

one-stop shopping for news,stocks,weather, and oh yeah, searching.  I'm
currently not planning on being a portal, but rather focussing on
effective searching.

Daniel Dreilinger, creator and webmaster for Savvy Search,
 http://www.media.mit.edu/~daniel

A portal site is a natural entry point for Web/Internet visitors, offering
a "one-stop shop" in general Internet/Web navigation tools including
subject category and keyword searching. Also, other ancillary services
such as free e-mail services, etc. are becoming more prevalent - services
that help to retain visitors as long as possible at the portal site.

Mark Geyman,
SitesOnline

The main difference between the two basic types of search engines boils down to the human element. Directories like Yahoo have humans who review the hyperlinks that are sent into them. They will try to avoid having an adult link show under a category for Beanie Babies. Crawlers, on the other hand, use automated software robots called spiders that prowl the web, looking for fresh spider food... new URL's. When they find these they quickly digest as much information as they can, scanning sometimes one page, sometimes dozens. They take that info back to their main nest, and allow it to be depositeded into the batch of URL's already there. When a user makes an inquiry of the crawler, it reviews all these URL's and sends you the listing of pages that had  the same search string words on it (sometimes several times, increasing it's relevancy and rank in the link listings).

Search Engines

 Yahoo!

• Directory; hierarchical branch structure/created by grads
• Uses keywords and categorical search methods
• Handy for specific country and city searches

Yahoo is in the Directory category of search engine. Designed by two grad students, mega-millionaires now, they borrowed a bit from Gregor Mendal's classification technique and devised a simple to use, hierarchical branch structure to classify web sites under. You can search by keywords, or simply click onto categories that interest you to see where the branch leads.

What makes it handy is that you could key in, say, an author's name like "Stephen King". You'll not only see dozens of sites that have Stephen King in their titles, but see what branches they are located under. You might see several branches mentioned. One branch may be for Literature/Genre/Horror/Authors/ and another might be Business/Collectibles/Books/First_Editions/Horror .

Depending on the reason for your search, you may want to "click up on the branch". After gathering your data on King as an author, you might click on the word Authors to see a listing of other Horror authors who have web sites. You may want to look at Bram Stoker or Clive Barker to gather more data on your main subject's peers.

Almost two years ago Yahoo began spinning off mini-yahoo's for countries and  major American markets. Lately, they've helped define the "Portal" concept by offering free e-mail through their site, many games and chat areas, a personal portfolio builder to track stocks, something VERY popular as Wall Street Internet stocks like Yahoo continue to go through volatile times. Unfortunately, on days when the market is racing, so are Yahoo's engines, and the entire service can slow down. They become a victim of their own popularity. Usually these only last in the mid-afternoon, or at times when there is a significant News story. Yahoo has partnered with Reuters and AP to provide a top notch, state of the art news service through their site.

TIP:
The lesson is,  Yahoo as well as all the other Portal services provide a ton of services, but 100% dependability? Not yet. Don't depend on them for your only e-mail account. The day you have to get back to a critical interviewer will be the day they crawl. The afternoon  you might need to see how the market has treated your  portfolio, well, good luck! Have alternative sites for news and stock prices bookmarked.
 

Go back  to the  Intro page

Continue onto  Crawlers