quarta-feira, 14 de julho de 2010

Robots.txt url without trailing slashes. - ALERT -

To leave a url in the robots.txt without a trailing slash can steal your sleep sometime. This makes the seach engine think you are blocking any match that starts with that url and it can disallow or allow unwanted urls, read these to understand better:

http://www.google.com/support/webmasters/bin/answer.py?hl=br&answer=156449

http://www.webmasterworld.com/forum93/892.htm

Nenhum comentário:

Postar um comentário