Web Log Archive · Index · Part 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · 23 · Expand · Web Feed

How to use Google's Personalized Home Page as RSS submitter leading Googlebot to fresh content.

Seeing Ms. Googlebot's sister running wild on RSS feeds, I'm going to assume that RSS feeds may become a valuable tool to support Google's fresh and deep crawls. Since I've not yet gathered enough log data, test it for yourself:

Create a RSS feed with a few unlinked or seldom spidered pages which are not (yet) included in your XML sitemap. Add the feed to your personalized Google Home Page ('Add Content' -> 'Create Section' -> Enter Feed URL -> Go). Track spider accesses to the feed and the included pages as well. Most probably Googlebot will request your feed more often than Yahoo's FeedSeeker and similar bots. Chances are that Googlebot-Mozilla is nosy enough to crawl at least some of the pages linked in the feed.

That does not help a lot with regard to indexing and ranking, but it seems to be a neat procedure helping the Googlebot sisters spotting fresh content. In real life add the pages to your Google XML Sitemap, link to them and acquire inbound links...

To test the waters, I've added RSS generation to my Simple Google Sitemaps Generator. This tool reads a plain page list from a text file to generate a dynamic XML sitemap, a RSS 2.0 site feed, and a hierarchical HTML site map. It is suitable for smaller Web sites with no more than 100 pages.


Update: Google's bot fetching feeds for the personalized home page now identifies itself as Feedfetcher. It doesn't obey robots.txt, because it is a a component of user driven RSS reader. To make Googlebot crawl your feeds, you need to ping Pingomatic and Weblogs.


Tuesday, August 02, 2005

Automated Link Swaps Decrease SE TrafficNext Page

Previous PageMozilla-Googlebot Helps with Debugging


Web Log Archive · Index · Part 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · 23 · Expand · Web Feed



Author: Sebastian
  Web Feed

· Home

· Internet

· Blog

· Web Links

· Link to us

· Contact

· What's new

· Site map

· Get Help


Most popular:

· Site Feeds

· Database Design Guide

· Google Sitemaps

· smartDataPump

· Spider Support

· How To Link Properly


Free Tools:

· Sitemap Validator

· Simple Sitemaps

· Spider Spoofer

· Ad & Click Tracking



Search Google
Web Site

Add to My Yahoo!
Syndicate our Content via RSS FeedSyndicate our Content via RSS Feed



To eliminate unwanted email from ALL sources use SpamArrest!





neatCMS

neat CMS:
Smart Web Publishing



Text Link Ads

Banners don't work anymore. Buy and sell targeted traffic via text links:
Monetize Your Website
Buy Relevant Traffic
text-link-ads.com


[Editor's notes on
buying and selling links
]






Digg this · Add to del.icio.us · Add to Furl · We Can Help You!




Home · Categories · Articles & Tutorials · Syndicated News, Blogs & Knowledge Bases · Web Log Archives


Top of page

No Ads


Copyright © 2004, 2005 by Smart IT Consulting · Reprinting except quotes along with a link to this site is prohibited · Contact · Privacy