Outside the Cube (7594)
Web publishing (65)
about DelphiFAQ (14)
perl CGI (3)
Web Hosting (8)
New related comments
Number of comments in the last 48 hours
Regular expressions in Homesite
Referer spam in my web server's log files
Googlebot requested a document that does not exist
This article has not been rated yet. After reading, feel free to leave comments and rate it.
Question:I looked through my web server's log files and found a 404 request made by googlebot - for a document that I never had nor ever linked to. How did googlebot come up with that URL?
Answer:Did you submit a Google sitemap? In this case Google's googlebot will ask your server for a filename that it's fairly sure to not exist, in order to verify that calls for invalid filenames do - as they should - result in a "404" response.