#9. Mediapartners-Google has a role as the Googlebot?
[March 6, 2004]
Once we add the AdSense of the Google in our site, the site is often accessed by the crawler, Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html). Let's verify the difference between the Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html) and normal Googlebot/2.1 (+http://www.googlebot.com/bot.html). Are they identical?
I filled in a script that outputs date and time when crawler accesses in my pages. Then I can see date and time when I check the cache of the Google. (We can see date and time at the bottom of cash source.) Based on the access log, I can recognize if the crawler accesses my page at that time is the Mediapartners-Google or the Googlebot.
If they have the identical roles, sites can be the fresh crawl typed sites (it shows date as a result of search) when we simply add the AdSense.
However, I understood the each role is different at the result of my verification. Only the Googlebot appears to get the cash. That is the Mediapartners-Google visiting site doesn't mean the site targeted by the fresh crawl. The reason the Mediapartners-Google isn't indexed by the Google even if it accesses to pages everyday is due to the difference of the roles.
| Google Cache Date | Crawler Access Time (JST) | Host | Type of Crawler |
| March 1 | 2004/03/02 14:44:38 | crawler13.googlebot.com | Googlebot/2.1 |
| March 2 | 2004/03/03 14:46:03 | crawler14.googlebot.com | Googlebot/2.1 |
| March 3 | 2004/03/04 14:51:20 | crawler14.googlebot.com | Googlebot/2.1 |
| March 4 | 2004/03/05 14:57:39 | crawler14.googlebot.com | Googlebot/2.1 |
This Mediapartners-Google seems to visit pages once the AdSense is put on pages. That only judges theme of pages since the AdSense automatically delivers ads targeted to content on a page-by page basis.
How should we get our site the fresh crawl site? To be verified next time.
|