View a Web Page as 'Googlebot'

Check out your Web pages as seen by Googlebot and any other search engine crawler. This free SEO tool will show you what a Web robot gets when it requests a page from your Web server. Use it to check what your CMS's browser optimization delivers to crawlers.

This tool spoofs the HTTP_USER_AGENT imitating these search engine crawlers:

Crawler	User Agent
Alexa-1	ia_archiver
Alexa-2	ia_archiver-web.archive.org
AskJeeves-Teoma	Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)
Googlebot-2.1	Googlebot/2.1 (+http://www.google.com/bot.html)
Googlebot-Mozilla-2.1	Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google-AdSense-2.1	Mediapartners-Google/2.1
MSN-1.0	msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Yahoo-Slurp	Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
ZyBorg-1.0	Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)

Enter the URL of the page you want to view 'as search engine crawler'.

URL ¹:	* Enter the full URL incl. http://
Crawler:
Open result page on submit:	Yes No Opens the page automatically on submit. (Enable JavaScript and disable your popup-blocker)
* = Mandatory

Viewing the result in your Web browser, client-sided rendering (for example execution of JavaScript) isn't omitted, but links/images won't show/work if their URL is relative to your root directory (relative links are evil from a SEO's POV, your content management system (CMS) should provide absolute URIs). If you see red 404 images on the result page, don't worry. They indicate relative image URIs which may work on the original page. Also, relative CSS URIs and alike screw the layout. Make your pages spider-safe and get your scripts to prefix relative URIs with scheme and server, then try it again.

SE crawlers do not execute JS and they won't download nor make use of plug-ins, thus check the result page's source code too (click 'view source', or scroll down to the very bottom to find the captured headers, then compare the HTTP response codes sent to your browser and the bot). Tip: install PrefBar (Mozilla/Firefox browsers). PrefBar lets you choose the user agent name, you can switch JavaScript/Flash on and off and more.

This tool is not suitable to detect professional cloaking, because there is no IP spoofing involved. It detects redirects, but doesn't follow them. Its goal is assisting you in configuring the browser optimization functionality of your CMS. Some CMSs need particular settings with regard to web robots, they deliver empty pages to crawlers if you leave the default settings.

If you find this tool helpful, please consider bookmarking it. Also, we love links from your web site! Please feel free to click on 'Link to us' on the left to grab our linking code:)
Thank you!

Related links:
Search Engine Friendly Content Management Systems (CMS)
Steering and Supporting Search Engine Crawling

¹ URLs containing spaces won't work. You must not use spaces in URLs, never, because some user agents --Web robots included-- cannot handle them properly. Query string arguments are part of the URL. More info on well formed URLs here.

· Home

· Steering SE Crawlers

· Googlebot-Spoofer

· Web Links

· Link to us

· Contact

· What's new

· Site map

· Get Help

Most popular:

· Site Feeds

· Database Design Guide