By splash112 on

Hi everybody,

Below is the response I get from google when using the gsitemap and try to verify. I got the page name from google, pasted it in gsitemap configuration and tried. File was there, but still an error...

Anybody knows how to activate google sitemaps?

drupal 4.7.2
gsitemap 4.7.0
Clean url's are on

Thanks.
Mark

We've detected that your server returns a status of 200 (found successfully) for pages that don't exist.

We've detected that your server returns a status of 200 (found successfully) for pages that don't exist. We can't verify sites with this configuration because it would enable anyone to verify ownership of your site. You can try our meta tag verification method or change your server's configuration.

This configuration presents a security risk for site verification and therefore, we can't verify your site. If your web server is configured to return a status of 200 (found) in the header of 404 (not found) pages, and we enabled you to verify your site with this configuration, others would be able to take advantage of this and verify your site as well. To ensure that no one can take advantage of this configuration to view statistics to sites they don't own, we only verify sites that return a status of 404 in the header of 404 pages.

If your server is configured to return something other than a 404 for non-existent pages, try our meta tag verification method or change your server's configuration.

You can change your web server configuration to return a status of 404 in the header of 404 pages and still provide a custom error page (see Apache information and IIS information). Note that we do a HEAD request (and not a GET request) when we check for this. Once your web server is configured correctly, try to verify the site again and we'll recheck your configuration.

⋅ Categories: Drupal 4.7.x

Comments

brianV’s picture

brianV commented

I've has this same issue. Drupal prefers to handle it's 404s by presenting the interior 404 which will still give the menu et. al.

Of course, Google doesn't like this at all, since it breaks their verification method... If Drupal returns a 200 OK for any random page, then they cannot count on that 200 OK for verification of the file they want you to upload.

I hope there is a fix for this!

splash112’s picture

splash112 commented

Hi BrianV,

I had this issue when I installed Drupal in a subdirectory (www.mysite.com/drupal). When I moved my site to the root, this problem vanished and my sitemap is accepted by Google now.

I also "discovered" Gsitemap will display a special page with "Hi Google" or something on the designated url on a complete white page.

Did you install Drupal in a subdirectory? Please let me know, maybe it's an easy problem to solve.

Thanks
Splash112

Dinilu.eu & Dinilu.nl & Dinilu.de & www.tit.nl

brianV’s picture

brianV commented

No, the page is on the root - it is the outfrontps.com page below.

The Google explanation seems to indicate that it has a problem with the custom 404s that are presented.

It's not a big deal - Google will still accept and use the sitemaps submitted by GSitemap. They just won't let you me view the stats, since they cannot verify that I am the owner of the page that I submitted the sitemap to...

--
Drupal Pages I've Designed:
http://www.outfrontps.com
http://www.infohatter.com

My Blog: http://www.infohatter.com/blog

splash112’s picture

splash112 commented

Great site! I love your theme (hope to get there myself someday www.tit.nl)!

The custom 404's shouldn't even be displayed, the page should be some blank page with a message for Google on it. I can only think of two other things that could have helped me. That is a cleared cache (Truncate * from cache) and a cron job...

You might want to try that. It was shamefull to see my misspeld words being on top in the ranking from Google....

Good luck!

Dinilu.eu & Dinilu.nl & Dinilu.de & www.tit.nl

brianV’s picture

brianV commented

Thank you for your kind comments, althogh I can't take credit for the template - it is the SpreadFirefox theme. And I am having some trouble with it: http://drupal.org/node/74110

I will try clearing the cache, to see it that fixes it. Thanks for the suggestions!

--
Drupal Pages I've Designed:
http://www.outfrontps.com
http://www.infohatter.com

My Blog: http://www.infohatter.com/blog

Kornika’s picture

Kornika commented

Should I be clearing?

I too, am having a problem with this. I attempted the meta tag verification method by creating a blank index.html file within the root of my dupal install and that also returns an error saying this:

We couldn't find your verification file. Make sure it is named correctly and is uploaded to the correct location.

We looked for your verification file, but couldn't locate it. Check the following and then try again:

My drupal install is in http://about.exampledomain.com meaning the meta tag I made can be found at http://about.examplemydomain.com/index.html which, when typed in to the browser, can be found. It just can't be found by Google.

I have no preference on how I verify with Google, I'd just like to verify.

Thank you in advance.

-Kornika

Kornika’s picture

Kornika commented

If you do the math on the time that from my last post to this post then you'll know exactly how long I've been working to fix my Google issues and I'm proud to say that they are resolved.

I fixed my problem through a lot of trial and error so I'm not sure exactly what it was that I fixed. Though I do know that I used the HTML verification method with the .html page Google told me to make placed in the root of my install.

Don't know if this will help, but the HTML for the page Google asked for looked like this:

<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<title>Google Page</title>
</head>
<body>
</body>
</html>
tetramentis’s picture

tetramentis commented

file contents do not matter at all, as far as I understand. I mean the file like google98498394893894934.html
Only its presence/absence, detected by the status code, matter.

ozwebsites’s picture

ozwebsites commented

What is it you use on this site to do the image gallery?

http://www.allcoversystems.com/DisplayUnits.htm

Love it !

yxm0513’s picture

yxm0513 commented

I have this issue, my drupal is installed in a subdirectory. How to fix? I mean not just move all files to webroot :)
Thanks

-------------------
moving files to root fix failed, still get a empty "map"

dvoelker’s picture

dvoelker commented

If you have performance caching turned on turn it off and then try to reverify, this solved my problem.

pavlos’s picture

pavlos commented

From my experience, Drupal returns an Error 404 response in its page not found header. It does not matter at all that this page is themed. What I had installed which was giving a 200 response was the Search 404 module.

Do you have this installed?

tetramentis’s picture

tetramentis commented

I have the same file-verification problem. Does anybody know if that could be because of Google using HEAD and not GET request?
Will look now what Drupal-5b1 really returns as status for non-existing page....

...
checked.
The trouble seems to be really with the status code returned by Drupal.
For non-existing page, response is:

Date: Mon, 06 Nov 2006 21:48:17 GMT
Server: Apache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Pragma: no-cache
Last-Modified: Mon, 06 Nov 2006 21:48:18 GMT
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

404 OK

Notice the final status - "404 OK". For comparison, here is the response of WordPress:

Date: Mon, 06 Nov 2006 21:49:16 GMT
Server: Apache
Cache-Control: no-cache, must-revalidate, max-age=0
Content-Encoding: gzip
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Pragma: no-cache
Vary: Accept-Encoding
X-Pingback: http://www.bogdan.org.ua/xmlrpc.php
Last-Modified: Mon, 06 Nov 2006 21:49:17 GMT
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

404 Not Found

Note the correct "404 Not Found" status here.... (both Drupal and WP hosted on the same server)

Should this be reported as a bug? Or could someone else check this issue - just to be sure :)

tetramentis’s picture

tetramentis commented

drupal_not_found() function in common.inc looks perfectly fine.... cannot understand where is the problem, then...

Probably PHP script is unable to set the status header? Has anyone come across such a case before?

tetramentis’s picture

tetramentis commented

if yes - then here's relevant post: http://drupal.org/node/64645
See also:
http://us3.php.net/header
http://bugs.php.net/bug.php?id=27345

now need to check if this is the case....

tetramentis’s picture

tetramentis commented

the recommended cgi.rfc2616_headers setting in php.ini does not in any way influence the problem... I did not check the headers this time, just trying to verify site ownership via google's sitemaps. so far, no luck :(

tetramentis’s picture

tetramentis commented

common.inc changed, now correct status returned (but still no google's verification.... wonder why)
after changing function drupal_not_found() to use "Status: 404 Not Found" (and corresponding 403 header), returned headers are:

Date: Mon, 06 Nov 2006 22:45:38 GMT
Server: Apache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Pragma: no-cache
Last-Modified: Mon, 06 Nov 2006 22:45:40 GMT
Keep-Alive: timeout=15, max=81
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

404 Not Found

which looks perfectly. probably just need to wait a bit before retrying sitemaps verification....

isaacbowman.com’s picture

isaacbowman.com commented

I had this error too and fixed it by turning Search404 off. Google immediately verified my site.

Isaac Bowman
www.isaacbowman.com

splash112’s picture

splash112 commented

Thanks for the Search404 hint, that worked this time for me!
So far clearing cache and disabling search404 worked.

Dinilu.eu & Dinilu.nl & Dinilu.de & www.tit.nl

summit’s picture

summit commented

Hi,
I took the google sitemap META TAG and put it hardcoded in my page.tpl.php.
Did somebody allready make a variable for this?
Perhaps in Gsitemap.module , now knows as XMLSitemap.module?

Greetings,
Martijn

friendster’s picture

friendster commented

Amended my page.tpl.php file (with the tag given by google) as has been suggested here and after scratching my head for several frustrating hours beforehand (and trying other suggested fixes) this worked for me and google verified my ownership of the site.

Mike_Waters’s picture

Mike_Waters commented

None of the above helped me (5.1); I had to go into common.inc and rewrite drupal_not_found() to be simply header('HTTP/1.0 404 Not Found'); that verified my site with google. Then I changed it back.

Sylvain Lasnier’s picture

Sylvain Lasnier commented

Humm,
Google reject my check file too.

In Apache access log, I can read this :

72.14.193.132 - - [04/Sep/2007:07:02:18 +0200] "GET /google952a0d070ce28ee5.html
HTTP/1.1" 200 28 "-" "Google-Sitemaps/1.0" (125)
72.14.193.132 - - [04/Sep/2007:07:02:19 +0200] "GET /noexist_952a0d070ce28ee5.ht
ml HTTP/1.1" 403 2028 "-" "Google-Sitemaps/1.0" (34)

So, Google don't like 403 error code for a no exist url... :(

wdrupal100’s picture

wdrupal100 commented

This is a caching issue. Clearing your cache under admin > performance solved it for me