Redirect search engines

Well, you are the first person to bring it up. Remember, though, that if you have SEO turned on, then search engines should never hit a page from the "wrong" domain.

I would probably contain this inside custom_url_rewrite_outbound -- if that is possible, and it should be. That function is designed to be editable from site-to-site, so making the changes there is preferable to moving the code.

- require_once is a statement and should not use parentheses.
- When including files of modules, don't rely on PHP's include_dir setup, always use the real path via drupal_get_path('module', 'domain').
- Please do not re-factor custom_url_rewrite_outbound() into several functions. It makes customization, copying, and comparison with the original function much harder.
- Also, functions provided by modules must always be prefixed with the module name. Hence, _spider() is wrong.
- In general: Plenty of coding-style errors; please adhere to http://drupal.org/coding-standards

Besides these points, I do not believe this patch achieves what it's intended for. Search engines need to access an old URL and be permanently redirected (HTTP 301 status code) to the new URL. So from my understanding, this patch defers these redirects and proper re-indexing of your site to a later point in time.

It may also be cleaner to implement this through hook_nodeapi() (as in the original post) but through a separate module.

I do, by the way, appreciate the use case and understand now why this matters. I think a stand-alone module is the best approach.

Marking as "won't fix" to indicate that I will not be working on this. Anyone else rolled a module for this yet?

There are one of two code style issues, but I really like the use of the API to make this work.

Run it through Coder module, the developer's best friend.

First issue was the {} on the first function.

I haven't tested it, but the use of hook_domainform() looks correct.

Coder note: there are also lots of places where spaces are missing from the code, which makes reading harder and violates Drupal coding standards.

For example:

  $first_part=array_splice($form['domain_advanced'], 0, $offset);
  $form['domain_advanced']=array_merge($first_part, $insert_me, $form['domain_advanced']);

Is written as so, for added clarity:

  $first_part = array_splice($form['domain_advanced'], 0, $offset);
  $form['domain_advanced'] = array_merge($first_part, $insert_me, $form['domain_advanced']);

If it works, you can port it.

Are there any potential issues if page caching is enabled? As in, if a non-spider anonymous visitor hits a page for the first time, wouldn't it get cached to ignore the redirect, so that when along comes a spider it also ignores the redirect?

I thought everything that happens inside a view op of hook_nodeapi got cached. This may be a misunderstanding on my part.

Marking as postponed, since I do not think this will go into the core module.

Hello,

I tried to open the file in Windows, but after unzipping I get a file with extension tar__2 which I cannot read. Who can help me out?
When I look into the file I can see it contains several other files.

Thanks,

PROMES

I installed 7zip and this one does all the unzipping.
I installed your module and it is up and running on 2 sites both with DA module installed, right out of "the box", with 'Redirect all user agents' set.
Thanks for your good work. I hope you will issue it as a regular Drupal module and have the time to maintain it.

I allready had created my own small module basis on the node_api like your module. But I it had some drawbacks. I compared it with your code and found the missing pieces.
My module is more simple and only based on a general redirect. But it has an advantage: it doesn't do database lookups, My be you can use my code.

if ($a4 === TRUE) {
global $_domain;

$err = TRUE;
$found = 0;
// lookup the domains in the node
foreach ($node->domains as $nr => $dom) {
$found += 1;
if ($dom == -1) {
$dom = 0;
}
// is node in current domain ?
if ($dom == $_domain['domain_id']) {
$err = FALSE;
break;
}
}
// if $node->domains empty --> node is in all domains, so ready
if ($found == 0) {
break;
}
// node is not from current domain
if ($err) {
// lookup the first domain of this node
$domains = domain_domains();
$domain = $domains[$dom];
// Can we and do we need to redirect this page?
if ($domain != -1 && $domain['domain_id'] != $_domain['domain_id']) {
// In this case, the $base_url cannot have a trailing slash
$base_url = rtrim($domain['path'], '/');
if (!$node->path) {
// If the path is not set, use the current uri
$node->path=ltrim(request_uri(), '/');
}
// 301 is a permanent redirect
drupal_goto("$base_url/" . $node->path, NULL, NULL, 301);
}
}
}
break;

I hope you can use it to make your module somewhat less dependent on database calls.

I realize myself there are some pittfals, I didn't account for like you didn't:
- the selected domain can be invalid or deleted at the moment a node is viewed
- no domain is defined in the node (maybe this is not possible, depending on settings)
- in my code: the first domain found is dependent on the sort sequence of the domains. May be there should be a try for domain[0] first.

The 'valid' flag should be embedded in the global $_domain array, so you can check that on the fly without hitting the db. Nodes will always have at least one domain attached (unless your system is misconfigured), but an empty() check is a good idea.

Aside from relying on the sort sequence, it is also necessary to check for the presence of a domain source, which is present (if set) at $node->domain_source. If that value is set, it is the preferred domain for all links, and should be used in all cases.

Hello agentrickard,

Thanks for your update. I don't use domain source, so I didn't know about the presence in the node array. The check for domain source is present in the code of Ibn al-Hazardous. But it is terrific you can do all checking without accessing the db.
I already was aware of the valid flag, I use it a lot in other code in my sites.

Last week I made the time to investigate why the solutions of Ibn al-Hazardous and myself don't work in Drupal 5.
Attached is a working module for D5 whith the combined logic of Ibn al-Hazardous and myself. It runs now in three sites for a couple of days solving a lot of access denied errors.

My changes are mainly:
- acces denied error: added node_grants
- code optimized to use the least possible cpu-cycles when a node is already in the correct domain
- no database access: all needed info is taken from the node
- added settings: define in what order domains should be redirected to
- changed the place of the search engine setting to the redirect settings
- updated the readme.txt file
- used reply #26 of agentrickard to get the domain_source domain
- the current/redirect domain must be valid

Not tested:
- the logic to redirect to the domain_source domain. I don't use this module.

Questions:
- please correct the text in the hook_domainform. English is not my first language.
- please test the module on your own site(s)

I will make a .pot file when the module is accepted.

Changed the status.

This module is not going into the main branch. It should be released separately.

Ibn al-Hazardous and Promes - this is an excellent module. I've ported it to D6 using Deadwood, and it appears to work well in my test system (tarball attached) - although it may need more work (no version number on the info file, etc)

For my needs I wanted pages redirected to the Source domain regardless of them being available in the current domain. To acheive this I commented out these lines (these are not commented out in the tarball)

        // node valid in current domain? -> no redirection
        // 1. check $node->domains: is domain set in editing form ?
        if (array_key_exists(($_domain['domain_id'] == 0 ? -1 : $_domain['domain_id']), $node->domains)) {
          if ($_domain['valid']) {
            break;
          }
        }

however, this it may be useful for others to have the option of skipping the above step. I want the node to be in the current domain so that appears in menus, etc, Any relative links are now redirected to the source domain.

I support this being released as a separate module and can test it if it's released.

regards

Matt

This module is great and is apparently providing a feature I was just about to ask about! Nice... and thanks! I'm using this last contrib by Matt B on my Drupal 6.13 test site -- soon to be tested in 6.14 and on production site, btw.

My question now has to do with how the redirection actually is seen by a search engine -- does this throw back a 301 redirect code, or is the browser just redirected "silently" somehow? I'm just on the tail end of getting my URL rewriting working again (URL rewrites to source domain not working (anymore)) and now I'm wanting to figure out how to get all the URLs that have been indexed for the past several months to point where they need to point!

Along these lines, I was wondering whether this Domain Redirect module might be able to do any of these things or if these are features best implemented elsewhere: (I've only just started this research, so forgive me if I've missed a thread somewhere.)

• Return a 301 Redirect message when a URL is redirected to its Source domain.
• Add a listing with the Path Redirect (path_redirect) module (if enabled) so future incoming links get redirected "properly"
• Utilize the new "canonical" link property (http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonica...) possibly via the Canonical URL module or similar. My thoughts here are to populate the canonical property with the Domain Source-aware URL, which really seems magnificent. (Maybe there's a way to do this already using the built-in DA rewriting code functions?)

Well, hopefully there are some folks working with this module -- maybe there's even an update? Thanks for considering these concepts... or for clarifying if they already work the way I'm thinking, at least on the initial "301" issue.

Cheers.

Regarding the "canonical" link meta tag property, I've come up with a solution that appears to work great so far... it's definitely a hack though. Sorry, but I did attempt to do this more "properly", but I couldn't find a quick solution and I need to get back to work on other stuff:

You can install the Nodewords module, which has support for canonical meta tags now. The built-in support for canonical URLs builds up the $path using the drupal_get_path_alias() function, which won't work right with Domain Access's URL rewriting (as explained by agentrickard on many occasions), so we need to override Nodewords' functionality when handling "node" types so it does not look up the path alias before Domain Access gets hold of it.

[Hack alert!] Do this by editing the Nodewords function definition include file:

./sites/all/modules/nodewords/basic_metatags/basic_metatags.metatags.inc

Inside the function basic_metatags_canonical_prepare(), find the case statement for 'node' (line 84 at this writing):
case 'node':
if (count($options['ids']) == 1) {
$path = 'node/' . $options['ids'][0];
}
break;

Edit this case statement to look like this:

case 'node':
if (count($options['ids']) == 1) {
//$path = 'node/' . $options['ids'][0]; /* Comment or remove this line */
$content['value'] = 'node/' . $options['ids'][0]; /* Add this line */
}
break;

Since Domain Access already handles all the URL rewriting, and Domain Redirect is making sure you're viewing the page from the desired source domain, all bases are covered so the link meta tag's "rel=canonical" property is set correctly. In fact, this solution ought to help out with the creation of canonicals whether you're using Domain Redirect or not, but that's another discussion.

Sweet!

I started to write a custom tag to put in the "./extra_metatags/tags/" directory, but I couldn't get it to work quickly. I think that's a discussion better suited for the Nodewords issue queue though. I'll spare the readers here.

Hopefully this is not considered off-topic for this thread. I think it's the type of thing that anyone using this redirect module would want though. I know I'm excited to get it implemented... now I just need to find a more elegant solution that doesn't involve hacking the Nodewords module for every update!

Ugh... it looks like I had the wrong idea with regards to doing canonical links for Domain Access setups using different domains:

Can this link tag be used to suggest a canonical URL on a completely different domain?
No. To migrate to a completely different domain, permanent (301) redirects are more appropriate. Google currently will take canonicalization suggestions into account across subdomains (or within a domain), but not across domains. So site owners can suggest www.example.com vs. example.com vs. help.example.com, but not example.com vs. example-widgets.com.

Well, there goes that idea for my use case! Bugger. I often forget that my setup appears as totally different domains, considering how completely integrated it feels during development! It all seems like one site to me by now... oh well.

However, if anyone is using Domain Access for a subdomain-only site network, #33 would still work great.

So... back to my other questions about handling 301's, then! That's what I need to do for my case, maybe using the Nodewords' canonical features just as-is, since it will still work for many useful instances.

(Sorry for the diversion.)

Anyone care to take ownership of this code and make a proper stand-alone release?

Hello Hoza,

To your question on the redirect. The kind of redirect is independent from the kind of visitor, whether it is a search engine or regular visitor. The domain redirect module uses the Drupal default redirect code, being 301 (at last in D5 where we are working in). I presume D6 will use the same HTTP-statuscode. Why 301? Because we don't want a node to be connected with the wrong (sub-)domain.

We use on all sites Pathauto, Path Redirect and Global Redirect to be able to have only one URL for a node. With these modules in mind we created domain redirect. So I think we have the canonical URL allready available:
An internal link in a node is a link to node/999. So the search engine sees a reference to node/999. But when requesting this node Global Redirect will return (with a 301) a page with the URL given by Pathauto. And this URL is always the latest version. Older versions will be translated by Path Redirect with a 301.

I hope this can be a good solution for you as well.

Best regards,
PROMES

Hello agentrickard,

We are going to release this code as a separate module soon. Thanks for using your pages for the time being.

PROMES

Excellent. Let me know and I will add it to the project page.

I created a testsite with a download option for those interested in the module. I added also canonical links to the module as an option.
Have a look at: http://domain1.drupalhandboek.nl/en/domain-redirect

Im testing the module and it is working ok so far.
Not sure if this is how you wanted the module to function, but when using the site search page it lists relevant pages from ALL affiliate sites. Is it possible to search just the active domain?

See sections 4.3.3 and 4.4.1 of README.txt for the Domain module.

Im testing the site without being logged in. I set Search settings: to "Search content for the current domain only".
I think the domain redirect module is not using this setting.

Ah. OK.

The domain redirect module doesn't affect the search settings nor is it involved with searching or the content of a node. It only redirects nodes to the defined domains.
For instance:
node/1 is assigned to domain1
node/2 is assigned to domain2.
You have a link in node/1 to node/2 without the correct domain2 (<a href="node/2">some text</a>) and someone clicks on this link DRM will redirect to domain2.example.com/node/2.

I had two domains:
www.domainA.com with node10 assigned to this first domain only
www.domainB.com with node20 assigned to this second domain only

Before installing the redirect module, if I searched from domainA i would not see node20 in the search results. Which is what i expected.

After installing the redirect module, if I searched from domainA i would see node20 in the search results. It seems the search is not taking into account what domains the node is assigned to. It searches all nodes.

Is anyone else seeing this behavior?

I would point you back to sections 4.3.3 and 4.4.1 of README.txt for the Domain module. Both of which might affect how search is run.

If you are using the SOLR module, you also need SOLR node access.

Sorry, those references are to the D6 documentation. Are you on D5?

Has this been made in an official module yet?

Not to my knowledge.

Is this module programed to have views pull in all content and then redirect to appropriate affiliate when the node in the view is clicked?
I would rather have that module still filter views and redirect any direct links or links in content.

Currently the module only can redirect nodes when they are asked for (clicked in a view or a menu), since you store domaininformation only whith nodes.
There is no views plugin to do any advanced filtering in views, since that is not the scope of the module. Domainfiltering in views should be done on project level and / or in a separate module.

Let me explain what I am experiencing with this module. I have a new content type named articles. Domain A has 5 articles and Domain B has 3 articles. I have a view on the frontpage that displays a list of articles.

Before enabling the domain redirect module:
- Domain A shows 5 articles on the frontpage.
- Domain B shows 3 articles on the frontpage

After enabling the domain redirect module:
-Domain A shows 8 articles on frontpage.
-Domain B shows the same 8 articles on frontpage.
- When clicking on the article links it will redirect to appropriate domain

I would like to see the redirect module still filter the articles found on the frontpage based on what domain it belongs to just like it did before the module is enabled. Is this outside its scope?

Yes, filtering is out of the scope of the domain redirect module.
The domain_views module can do the filtering for you. It is written for that purpose.

OK.

There looks like there is a conflict with the Domain Advanced module. When the Domain Advanced module is enable, Domain Redirect no longer works.

Moving to the appropriate issue queue

Sorry to resurrect this since it's in DAA's issue queue... but.. is Domain Redirect an actual module yet or is the only code here in this thread? I've searched for it but don't see it in the modules listing anywhere.

Thanks!

I tried to apply for a cvs account, but it was rejected because my request was not complete. Due to a huge workload I didn't have the time to re-apply for an account.
The D5 module is working on serveral sites with intended results. But on one site I saw some unexpected results (not a real burden for the visitor). My intention is to solve this problem first and then to reapply for an account. This will be in the coming months.

I did reset this issue back to Domain access v5. There is no tested release of D6 up till now. So it cannot conflict with an D6 module.

This is all too confused. Closing.

The concept of this module is really great and useful. Its a shame that its so difficult/confusing to implement.