The BumbleBee Garage Web 2.0
Automotive Search Engine
How It Works
For any
webmasters wondering why they may see our bot or
page referals in their stats logs, we hope this
will explain how it all works.
If you
are seeing traffic from our administrative pages,
then your website is being reviewed for inclusion
in our automotive search engine.
We never
expected to have a fully automated search engine.
The primary approval process to include websites
will be done manually by authorized volunteers.
We don't let just anyone review websites. They
must have a background in the automotive industry
as well as some web experience. We look for
interns from colleges and people that want to
learn more about the internet while contributing
to one of the biggest 2.0 projects online.
We demand that
they understand what they are reviewing and have
the ability to recognize a web feed from unique
content. To notice what is advertising and what
is actual articles within a page.
To assist in
the process our main bot will prequalify websites
based on an initial crawl of about 50 pages. The
sites are presented to reviewers who with decide
to include or exclude the site from the index.
In the event
they include the website, they will rate the site
on a scale of 1 to 10 in different categories.
The average of the score will serve as our base
page rank to weight the site in our keywords
searches. Since our page rank is not based on any
particular page but on the overall website and a
human review it should be a positive benchmark
for search results.
To assist the
site reviewers and speed up the process a series
of automated tools can be accessed that will
crawl your website creating a sitemap to find
hidden pages, counting text on pages and the
number of ads being accessed from programs like
google adsense or overture.
You can block
the automated tools by disallowing our bot from
your site. The automated tools will not obey a
crawl delay because they are manually activated.
The user agent will be "SmartyBot 2.0
Analytics" which will be different that the
standard SmartyBot Crawler and the refering page
will likely be one of the administrative pages
that is not accessible by the general public.
It can be
frustrating getting referals from password
protected pages, but it is necessary in this
process of evaluating websites.
If you
are seeing traffic from our administrative pages,
then your website is being reviewed for inclusion
in our automotive search engine.
Once the site
is included by a reviewer our primary automated
bot will be free to crawl your pages, hopefully
once a month to update all the pages in our index.
|