By rhg108 on

Parts of my site that are in subdirectories will not send out pings nor will they get indexed in Google Blog Search (they are indexed in Google proper). The content on my homepage http://www.sternrate.com/ pings and gets indexed correctly. But other content, for instance, a blog at http://www.sternrate.com/girl-on-stern does not seem to send out a ping on submission nor will Google Blog Search index it although I've manually given Google Blog Search the RSS feed as well as the URL.

I use the Multiping module, but obviously there is a problem apart from this since Google Blog Search will not index even with manual submission. I thought this might be due to the way my file structure is set-up. Although the blog is at sternrate.com/girl-on-stern the url of the articles is sternrate.com/ I did a test, making the URL of the article in the subdirectory, but that still didn't get indexed or send out a ping. Any ideas?

Thanks in advance!
Rachael

⋅ Categories: Drupal 5.x

Comments

ereq’s picture

ereq commented

I am having the same problem. Does anyone having problem with pinging and being indexed with Google Blog Search? Even manual submission does not do any good.

If you have fixed this problem, would you please post here so we can get some help too please

thank you
Ereq

rhg108’s picture

rhg108 commented

Ereq, unfortunately, I haven't found a solution for this.

Rachael

rhg108’s picture

rhg108 commented

Ereq,

The problem with my site was definitely the inconsistencies I had between where my articles were stored and what was in my RSS feed. Because my articles were stored at root level, and my RSS feed was in a directory, I wasn't getting indexed. I began creating my new articles in that folder, and it seemed to fix it. Just make sure everything is consistent. I'm getting indexed now. Make sure you also keep doing the manual submission because I still don't think my ping is working.

Rachael

Jeff Burnz’s picture

Jeff Burnz commented

I think part of the problem is that the Multiping module appears to push out URL's like:

http://www.sternrate.com/taxonomy/term/1/0/feed

Whereas your sites aliased RSS feed is:

http://www.sternrate.com/girl-on-stern/feed

Also, I believe, that instead of pushing out http://www.sternrate.com/rss.xml as the main feed url, it uses http://www.sternrate.com/node/feed (just like drupal.org), but put that in your site and you'll be asked by Howard "Where the f#&k are you trying to go...", indeed...

So, it would appear that 1) multiping is sending out duplicate URL's for your RSS and 2) sending out the main RSS feed which results in a 404.

For anyone who was blocking bots from the taxonomy/term path (standard practice for avoiding duplicate content issues) this could be a real problem - you would be blocking the very URL's that Multiping is sending out.

If I have this wrong can someone set me strait, I am just going by what I see in the code for Multiping module:

// Determine rss url
  if (module_exists('taxonomy') 
    && ($pingservice->whentoping & _multiping_when_taxonomy)
    && count($pingservice->voc)>0 && $pingservice->submitmainrss==0) {
    // Taxonomy module exists, "ping only for nodes with taxonomy",
    // Some taxonomy elements are actually selected, "main rss" override is off
    $voc=unserialize($pingservice->voc);
    $rss_url=$base_url."/taxonomy/term/".implode("+",$voc)."/0/feed";
  } else {
    $rss_url=$base_url."/node/feed";
  }
Pimp your Drupal 8 Toolbar - make it badass.
Adaptivetheme - theming system for people who don't code.
geme4472’s picture

geme4472 commented

I have the same problem, but I can't figure it out. I can even see google blog bot hitting the server a couple of minutes after my ping, but no indexing whatsoever.

Also, if you're manually pinging, you're only pinging google with the URL, not the feed URL as well? If you were manually pinging with the URL, are you certain the RSS feed comes into play as far as figuring out the root cause of the original issue? I have pulled out my hair on this one for several days now... any theories in addition to your last comment?

Jeff Burnz’s picture

Jeff Burnz commented

To be frank we've had the most consistent results using the standard Drupal Ping module - i.e. simply relying on pingomatic to do our dirty work.

Pimp your Drupal 8 Toolbar - make it badass.
Adaptivetheme - theming system for people who don't code.
geme4472’s picture

geme4472 commented

Out of curiousity, do you get in the google blog search index quickly, or is the lag significant? I'm going to try this approach regardless, I'm just wondering for testing purposes.

I've been testing this again, all night, getting nowhere. Meanwhile the CEO's blog on wordpress hits the blog search index every time he sneezes. Humph! More testing to come, I'm sure, and I'll post anything I find.

Jeff Burnz’s picture

Jeff Burnz commented

Its inconsistent to say the least. Your CEO is probably pinging Blog search directly, which at the very least is what Drupal should be able to do out of the box, alas it can not. That goes in the box of "woefully inadequate blogging tools"...

What I've found is that often pingomatic will drop your ping, especially if you try to often, like more than every half hour.

Pimp your Drupal 8 Toolbar - make it badass.
Adaptivetheme - theming system for people who don't code.
geme4472’s picture

geme4472 commented

The strangest part of all of this is that I can see the google blog search bot in my accesslog records, usually appearing only a minute or two after a ping (via manual means, the ping module, or a custom ping module... all appear to work fine).

I've also tested a completely vanilla drupal 5.8 on a different domain, and I get the same results. I can't help but wonder if there isn't something going on such that google thinks I'm spam.

My next step is to install a wordpress instance on our servers and give that a shot. The afore mentioned wordpress blog is on wordpress.com. Maybe there's something glitchy in my servers? I'm starting to get into the "long shot" testing now.

BTW, thanks for your thoughts. I can't seem to google-up any relevant threads on this subject anywhere. More news soon.