On multilingual sites with language negotiation set to domain, url() always returns absolute URLs. So when your site runs on both http://example.com/ and http://example.nl/, url('flag') returns either http://example.com/flag or http://example.nl/flag. This causes the robots.txt recommendation to be wrong. It should suggest adding Disallow: /flag/, but now it recommends Disallow: http://example.com/flag/.

The following code replacement in flag.install fixes this:

Replace:

$flag_path = url('flag') . '/';

With:

$flag_path = url('flag') . '/';
// url() may return an absolute URL when language negotiation is set to domain
$flag_path = parse_url($flag_path, PHP_URL_PATH);

Comments

Jorrit’s picture

Status: Active » Needs review
StatusFileSize
new606 bytes

The patch.

joachim’s picture

Status: Needs review » Postponed (maintainer needs more info)

Can you not explicitly tell url() to not return an absolute URL?

Jorrit’s picture

Status: Postponed (maintainer needs more info) » Active

Thanks for your quick response. Unfortunately, the rewriting always happens in locale.inc, function locale_language_url_rewrite_url(). $options['absolute'] is set to TRUE when language negotiation is set to domain.

See http://drupalcode.org/project/drupal.git/blob/refs/heads/7.x:/includes/l... .

joachim’s picture

Version: 7.x-2.x-dev » 7.x-3.x-dev
joachim’s picture

Status: Active » Needs review
joachim’s picture

Status: Needs review » Fixed

Thanks!

Committed, with a tweak to the comment wording and spacing.

git commit -m "Issue #1804228 by Jorrit: Fixed robots.txt recommendation wrong when language negotiation is domain based." --author="jorrit "

Jorrit’s picture

Thanks for committing, I appreciate the commit attribution.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.