Ask Googlebot to Crawl New and Modified Pages on Your Web Site

	Ask Googlebot to Crawl New and Modified Pages on Your Web Site
·Smart IT Consulting Services· ·Internet Business· ·How to Make Use of Google SiteMaps·

How to Make Use of Google SiteMaps · Index · Part 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · Expand · Web Feed

Create a Google Account, then go to the Google SiteMap Submit Page. Enter your sitemap URL and wait for the first download displayed on the stats page. If the status is not 'Ok', correct the errors and resubmit your sitemap until it's approved. Bookmark the stats page and check back every once in a while (and after script changes!) to track Googlebot's usage of your sitemap.

You don't need to resubmit your sitemap manually. Being a smart webmaster, you'll automate the resubmits. The easiest way to automate sitemap resubmits to Google is to trigger a HTTP request on change of released content pages. After updating your database, call a function to ping Google. Since your dynamic sitemap file is always up to date, you don't need to do more. A PHP example:

function pingGoogleSitemap ( $rootUrl ) {

    $fileName = "http://www.google.com/webmasters/sitemaps/ping?sitemap=" .urlencode("$rootUrl/sitemap.xml");

    $url = parse_url($fileName);
    if (!isset($url["port"])) $url["port"] = 80;
    if (!isset($url["path"])) $url["path"] = "/";

    $fp = @fsockopen($url["host"],
                     $url["port"],
                     &$errno, &$errstr, 30);

    if ($fp) {
        $head = "";
        $httpRequest = "HEAD ". $url["path"] ."?"
                     .$url["query"] ." HTTP/1.1\r\n"
                     ."Host: ". $url["host"] ."\r\n"
                     ."Connection: close\r\n\r\n";

        fputs($fp, $httpRequest);
        while(!feof($fp)) $head .= fgets($fp, 1024);
        fclose($fp);

        return $head;

    }

return "ERROR";

}

This function returns something like "HTTP/1.1 200 OK Content-Type: text/html; charset=UTF-8 Content-Language: en Cache-control: private Content-Length: 0 Date: Sat, 04 Jun 2005 21:41:00 GMT Server: GFE/1.3" or "ERROR" on failure. If the string doesn't contain the return code "200 OK" something is fishy. Resubmits via ping don't appear in your account's sitemap stats.

If your content changes frequently, you should set up a cron job pinging Google twice a working day or so, instead of bothering Google with a ping on each record change.

To enable auto-discovering of your sitemap (Google, Yahoo, MSN, Ask) add this line to your robots.txt file:
Sitemap: http://www.example.com/sitemap.xml
(replace domain and file name if necessary)

Google Sitemaps Crawler Stats