Hi there,

Have you found the way for generating proper sitemaps
with XML Sitemap and Domain Access for 6th Drupal?

I can see, that:

* Node urls in a sitemap do not contain a proper domain,
only the base one, or the url cron.php was running from.

* A single node shows up on every domain's sitemap,
regardless its DA settings.

Important: there is no required db_rewrite_sql in code
of XML-S. module.

Or maybe there is any other working solution for Domain
Access and XML Sitemap in Drupal 6?

Tia,
Szy.

PS. This is issue for both 6.x-2.x and 6.x-1.x versions.

Comments

Anonymous’s picture

Category: support » feature

What about a hook that would allow you to alter the link. Based on your comment I assume that Domain Access allows for running cron.php on only one domain and thus the problem becomes how to replace the base_url generated in the report for each different domain. A hook_xmlsitemap_link_alter() could allow you to replace the $base_url value with an '@base_url' token string on write and then replace the '@base_url' token string with the new value of $base_url on display of the sitemap.

This sounds a bit cumbersome and may be slower than you desire it to be but is about the only option I can think of where you only execute cron.php on one domain. This is more a feature request that a support request in that you are asking for coordinated work.

agentrickard’s picture

We're already using custom_url_rewrite_outbound() in Domain Access, so I wonder if that logic is getting applied. (I also wonder if node access rules are being applied).

DA's cron hook also fires an internal hook_domaincron() that should allow for cron-per-domain functionality.

I think we may also have to account for a site map per domain, and my understanding is that XML SiteMap only generates one file, which doesn't work in the DA use-case.

This is one of those features/issue I just haven't had time for.

Anonymous’s picture

We're already using custom_url_rewrite_outbound() in Domain Access, so I wonder if that logic is getting applied. (I also wonder if node access rules are being applied).

Module weight issue perhaps?

DA's cron hook also fires an internal hook_domaincron() that should allow for cron-per-domain functionality.

Maybe rewrite the global xmlsitemap for each domain to its own file manipulating the base_url value. Does DA control the files variable so that each site has its own directory appended? Something like files/[domain_id]/xmlsitemap/ may help.

This is one of those features/issue I just haven't had time for.

We all have to set priorities and more recent issues take precedence over previous ones. Don't feel badly.

dave reid’s picture

Re #1: We can't use xmlsitemap_link_alter() because that's run before data is inserted/updated into the database. And we don't store the full path to the link, only the relative part (e.g. node/1).

We don't use db_rewrite_sql() because we don't generate the data when we also generate the sitemap files. We populate the {xmlsitemap} data during cron and instead of using db_rewrite_sql() we use node_access() on each individual node.

I don't easily see a solution right away.

agentrickard’s picture

Hm.

We can control the files variable for individual actions while generating the files (I use this in Domain Theme). Early tests on globally changing the files directory for all file handling failed.

In any event, I see a bridge module handling this.

avpaderno’s picture

Title: Domain Access and XML Sitemap for 6th Drupal » Domain Access and XML Sitemap for Drupal 6

I think the code for the integration between XML sitemap, and Domain access should rather be in a third-party module, not a module included with XML sitemap. The number of users using both the modules should be very limited.

dave reid’s picture

Status: Active » Postponed

FYI This would not be very high on my priority list since I don't use the Domain Access module. This would need someone to step up and contribute it. Marking as postponed for now.

croryx’s picture

Subscribing.

dave reid’s picture

Title: Domain Access and XML Sitemap for Drupal 6 » Integrate with Domain Access
dave reid’s picture

FYI there's also a new hook now called hook_xmlsitemap_query_alter() that can be used to alter what data is selected for the sitemap. It might be of use somehow for this purpose. It came in very handy writing a sub-module to integrate the content selection of i18n w/ xmlsitemap.

agentrickard’s picture

But that would still only output one file, right?

Do we have a setting or variable that controls the name of the output file?

dave reid’s picture

Hmm, yeah we'd need something more for Domain access. I'm going to stop trying to figure this out right now since my head is exploding. :)

agentrickard’s picture

It's one of those things I could figure out, given 4 hours. Maybe in Paris....

dave reid’s picture

Let me know what would need to be altered to make domain access integration work.

agentrickard’s picture

I suspect you need an argument for domain handling, similar to the argument for for $language in the _chunk() functions.

BarisW’s picture

Subscribing. Would be great to have this working!

chawl’s picture

Subs. +1

hkellecioglu’s picture

Subscribing

bartezz’s picture

subscribing

chawl’s picture

This is quite essential, evoking this again.

agentrickard’s picture

@chawl And what are you doing to help the process?

chawl’s picture

Yes, I am trying to figure out the code, but this seems quite beyond my skills. Then I tried to check what vox populi might say.

dave reid’s picture

This is kind of the one big issue that would be holding me back from a 2.0 release because I know that getting the module to support Domain access is going to require a change in the APIs. I'd like to avoid doing that post-2.0 but if I'm nearing an official 2.0 release I don't want to postpone it.

nonsie’s picture

This bridge just became a requirement for a project of mine so sub as well.

agentrickard’s picture

Status: Postponed » Active

Well, I'll try toi look again this weekend, but my initial research suggests that the _chunk functions need an additional argument. Maybe a $context array that allows for the creation and loading of different sitemaps based on conditions.

Perhaps a refactor of the existing $language code to be abstracted as a $context might be the first step?

szy’s picture

Please don't get me hurrying you up, but... is there any chance
for XML Sitemap and Domain Access working together? :]

Big thanks for your work, guys,
Szy.

matteoraggi’s picture

subscribing

ansari.wajid’s picture

Hi

I have two domains pointing to one site chiprv.org and chiprv.dev.triocircle.com.
The problem is that I have installed xmlsitemap throught chiprv.org and I am getting error when navigating to chiprv.org/sitemap.xml but then it is working for chiprv.dev.triocircle.com/sitemap.xml. I am unable to figure out the problem.

Thanks

dave reid’s picture

@ansari.wajid: You fail to understand what we've been trying to tell you. I'm sick of correcting your "noise" in these issues and I hate using all caps. But maybe you'll read it this time:

STOP POSTING THE SAME COMMENT IN ISSUES THAT ARE NOT RELATED TO YOU PROBLEM.

STOP POSTING TO CLOSED ISSUES UNLESS IT WAS CLOSED RECENTLY AND YOU HAVE READ THE ENTIRE THREAD AND ARE HAVING THE EXACT SAME PROBLEM

OPEN A NEW ISSUE IF THERE ISN'T AN EXISTING, RELATED, OPEN ISSUE THAT IS ABOUT YOUR SAME PROBLEM OR IF YOU ARE JUST UNSURE.

You have not specified you are using Domain Access, which is what this report is about.

bartezz’s picture

I could hear you shouting half way around the globe... although Wajid is located a bit further away I would say he should get the message now ;)

On topic: I gave integration a shot but mainly due to lack of time and knowledge I failed horribly :(

Cheers

dave reid’s picture

Marked #663494: Need multi-domain support. as a duplicate of this issue.

Romy-779’s picture

Hi!

I am working on a new drupal project at the moment. For this project, I would need the modules XML Sitemap (V2) and Domain Access to work together.

Is there someone actively working on this problem at the moment, will there be a solution in the near future? Or would it be better to search for another solution?

I think for a first step it would be enough to use one sitemap for the whole drupal page, and just change the domains/subdomains before the path of each entry. This could be possible by making a sql query for the subdomain.

Something like this:

$subdomains_query = db_query("SELECT m.subdomain FROM {domain} m INNER JOIN {domain_access} n ON m.domain_id = n.gid WHERE n.nid = |>NID OF THE NODE<|");
$subdomains_get = db_fetch_object($subdomains_query);
$url='http://' . $subdomains_get->subdomain . '/' . |>PATH OF NODE<|;
agentrickard’s picture

That is not the proper way to build the URL. Simply using l($text, 'node/NID') is fine. The problem is in restricting the list of nodes.

Anonymous’s picture

I would sure like to see this working, too!

ayalon’s picture

Unfortunately I need this 7 month issue/requirement too :-) Maybe we should bundle all the efforts and first define a way how this should work in the near future and then start with the implementation.

_Eli_’s picture

I am also hoping for an integration of xml sitemap with domain access.

What is the status of this request? Can a working module be expected soon, or is this request just on the waiting line?

agentrickard’s picture

It's waiting for someone with the time and incentive to tackle it, at the code level.

sgriffin’s picture

subscribing.
I believe the solution here would ALSO help with domain migration issues, if you could have a per-node base url for the sitemap that would be great.
Then you would be able to slip into a domain migration slowly using redirects.
Research into domain migration shows it to be fraught with danger even in the best of practices, having cross-domain sitemap support would help a great deal to reduce an old content link to a deprecated domain.

damienmckenna’s picture

subscribing.. I believe we'll have a need for this fairly soon too..

Anonymous’s picture

I took a look at the xmlsitemap i18n code, and my uninformed impression (as a competent PHP hacker who knows zero about Drupal internals) is that if someone were desperate (which I am not, yet), you could make a VERY small change in it and have it filter by domain rather than language. (Or even both, if you wanted that.)

This is obviously hackish and not a proper solution, but would get the job done IF you were desperate.

If no one comes up with a clean solution, and I do eventually get desperate, I'll work out the details and post them here. In the meantime, if someone is more desperate than me, maybe he or she can do that and post the kludge and we can all use it until there's a real implementation.

jlogsdon’s picture

We are currently in need of this at my company.

If I find the time in the next few days I might be able to work it out, but it probably wouldn't make a 2.0 release due to the API changes required to make it work properly. I'll post a patch if I get anywhere.

dave reid’s picture

@jlogsdon I've made it more than clear that I'm more than willing to review and change the APIs to help get this in so it isn't a big mess to try and work around. But at some point I've got to move forward and get an official 2.0 release made. I don't use domain access and don't have a good setup to test it, so I can't really drive this.

kevinh456’s picture

I've got a working patch for this. How do I submit it?

Kevin

dave reid’s picture

Use the 'File attachments' right below the comment box on this page. ;)

kevinh456’s picture

StatusFileSize
new14.93 KB

This is my first patch, so don't go too hard on me ;-).

It goes from the current release version to the latest development version. i can reapply my changes to the dev version if needed.

Summary of my changes:

Added a domain variable to certain function calls to allow changing the domain name.

Known Issues:
Didn't include code to filter the xmlsitemap content based on the domain (it doesn't work)
Domains with aliases don't select via subdomain. IE: if your main domain is www.mysite.com and your staging domain alias is staging.mysite.com, then the xml site map is keyed to www.mysite.com and won't load correctly on staging.mysite.com

Kevin

kevinh456’s picture

StatusFileSize
new14.79 KB

Sorry, patch 2. Now without 100% less debugging code. ;-)

Kevin

dave reid’s picture

You'll probably want to do a CVS update and re-do the patch. The attached patch seems to revert a lot of the recent changes in CVS.

kevinh456’s picture

Dave,

I'll do a new patch based on the cvs code when I finish up this bug fix. Sorry about that.

Kevin

kevinh456’s picture

StatusFileSize
new6.37 KB

New patch using latest CVS

Resolved:

Will now load for aliases, but the url's in the site map will be the primary domain for all aliases

Pending:

Filter content
Update admin tool to rebuild ALL site maps

jlogsdon’s picture

Much appreciated kevin! I'm going to look at Filtering content tomorrow, as we definitely need that.

agentrickard’s picture

Sorry, but this is the wrong approach.

if (module_exists('domain')) is just creating an exception handler. You want an API function, like an alter hook to allow modules to set the context for the sitemap.

I think $language should be handled in the same way. So your chunk function end up something like:

function xmlsitemap_generate_chunk($handle, &$status, $chunk, $context)

Where $context may contain declarations by multiple modules.

kevinh456’s picture

StatusFileSize
new2.09 KB
new16.06 KB

Attached is my tentatively final patch. This is NOT merged with the current CVS code.

The following issues are resolved:

  • (FIXED) Content filtering based on domain
  • (FIXED) Domain Alias issues

Known Issues:

  • (WILL FIX) Not merged with latest CVS
  • (WILL FIX) Still need to update admin tool. Currently, use drush to generate sitemaps
  • (WILL NOT FIX) Does not support wildcard domain aliases (each possible subdomain must be listed individually)
  • (WILL NOT FIX) Possible performance issue, must generate sitemaps for each domain and alias

Kevin

agentrickard’s picture

Wildcard domain aliases should not be a problem, as we should be keying on domain_id, and that is loaded from the canonical record.

This is still an exception handler.

Also beware that DA can confuse drush. Often you must pass a --url argument for it to work correctly, just like it does with multisite.

damienmckenna’s picture

kevinh456: Good work on building something that can at least be a starting point. I see agentrickard's point that it should really be a hook, but at least others can use something until a final, clean solution is finished.

davereid: What do you think? Would it be feasible to doing part of it as a hook for the v6.x-2.0 launch?

nonsie’s picture

I'm with @agentrickard on this as this patch does not account for cases where multiple languages and domains are involved. Imagine a scenario when your domains use more than one language or your domains each have different languages enabled. This needs to be a hook rather than exception after exception.

Anonymous’s picture

Better to have something that works for multiple domains and one language, than nothing at all that works for multiple domains. There are probably far more instances of the former use case than the latter.

Obviously, still better to have a clean solution that works for multiple domains and multiple languages -- but we don't want to allow the best to be the enemy of the good. IMHO.

kevinh456’s picture

With @agentrickard's comments in mind, I refactored the patch from 1/20 to be more compatible with a future hook system. I changed the domain code to be part of a generic $context array being passed around in the cvs version. I also use the pre-existing xmlsitemap_query_alter function used by the i18 module to alter the select query.

We should be able to adapt the patch for a hook system when designed/implemented. I tried to build it as close to the language code as possible. Any solution that works to create hooks for the language code should also work for the domain code. It will probably be a good test for any hook system that gets put in place; two different test cases. As hooks are added to the processing code, we can just move the small domain exceptions to the xmlsitemap_domain module.

@nonsie

I'm not sure what you mean. The exceptions that I added in the site map building functions aren't any different from the language exceptions already in place. In fact, I only extended the code that was already there. If you look in the primary driver function for the sitemap construction, you'll see that there are now two for loops, one over domains, then another loop for each language. I have not tested it, but it should be looping over all the languages that are enabled for each domain. Could you give me the use case that's giving you trouble?

@agentrickard

I emailed you about the wildcard problem directly, but you told me to post it here. The problem is getting the correct URL for all active sites working across 4 different environments. Specifically, my configuration was this:

Main domain (id:0):
dev.mysite1.com
Alias:
%.mysite1.com

Secondary Domain:
www.mysite2.com
Alias:
%.mysite2.com

possible choices:

dev.mysiteX.com
test.mysiteX.com
qa.mysiteX.com
www.mysiteX.com

Generating the sitemap files on dev was producing two sitemaps, one for dev.mysite1.com and one for www.mysite2.com. This was happening on all environments. There is no way to tell which of 4 environments should be used to produce a proper sitemap; you have to manually define all the possibilities. Furthermore, you need to be able to generate the sitemap using only domain functions. In the domain data structures provided, there is no way to extrapolate this information for non-active domains. My solution was to produce a sitemap for each of the possible aliases, then select based upon the alias when it goes to display the sitemap. I hope you can tell me a better way.

Kevin

kevinh456’s picture

I'd like to propose the following additional hooks for xmlsitemap:

xmlsitemap_alter_file_name

Allows you to hook in to the file naming system and alter the new filename as need.

xmlsitemap_get_url_options (or similar name)

This should be moved to a hook so that we can alter url options from multiple modules

xmlsitemap_alter_process_cases (or similar name)

This would be the main driver hook. Each module would be expected to return a list of values to execute over.

So a language module would return something like:

[0] => en
[1] => es

And a domain module would return something like:

[0] => domain1.com
[1] => domain2.com

Then, the main driver code would merge these two arrays together in to an array of arrays. Each entry would be a combination of context values to execute over. The main driver code will do this merge.

Here is an example of a final merged array:

[0] => array('domain1.com', 'en');
[1] => array('domain1.com', 'es');
[2] => array('domain2.com', 'en');
[3] => array('domain2.com', 'es');

This would then execute for every domain/language pair, in one 'for' loop. This should allow us to alter the module for most use cases. Certainly all the use cases for domains.

Kevin

nonsie’s picture

This approach assumes that all languages are always the same on all domains. For sites with Domain Locale enabled this is not the case. Here's an example from one of my sites:
domain 0 (EN, FR, SW, RW)
domain 1 (EN, FR)
domain 2 (EN, SW).

Why create a site map in SW + RW for domain 1 or FR + RW for domain 2 if those languages do not exist on those domains?

kevinh456’s picture

@nonsie:

This seems like a very small use case. I only have the domain access requirement; I don't even have a languages test case. Maybe you can create a patch after the main issue is resolved? This is a bit beyond the scope of what I'm doing, and beyond the scope of anything xml site map module does right now.

Kevin

damienmckenna’s picture

@kevinh456:
I think the key issue would be to leave it open enough to not require a one-to-one relationship.

@nonsie:
One question, though, is how you would even handle separate sitemap structures for different languages? I don't see anything in the official protocol specs for defining a language. Would you intend your domain0 to have four files?

kevinh456’s picture

@DamienMcKenna

Maybe if there was a second hook to massage the array for your needs? That way any site could develop custom processing code to remove the combinations they don't need.

Also, right now the module does create multiple files, one for each language. The new domain extensions create a file for each domain/language pair. Unfortunately, I don't see any way around creating a bunch of different files.

Kevin

dave reid’s picture

I've been brainstorming re-thinking how files are generated. Instead of saying 'Generate me the sitemaps for languages or domains' and then just blindly processing the data, instead go through the data in {xmlsitemap} and determine which file it should be written to. For example:


Type | ID | Lang
-----+----+-----
node | 1 | en
node | 2 | fr
node | 3 | und

$context = module_invoke_all('xmlsitemap_link_context');

foreach ($links as $link) {
  $file = xmlsitemap_get_filename_from_context($context, $link);
  if (!$file->opened_already) {
    $file->add_opening_xml;
  }
  $file->add_link($link);
}

The thing about this approach would be domain is free to say 'ok this link needs to be written to a domaina sitemap. ok this next link should be in domainb sitemap'.

The difficulties with this approach are determining when we need to change chunks for each context/file.

kevinh456’s picture

@Dave Reid

How would that work if I decide to add a third condition? Say I want to generate site maps for different search engines too (just a hypothetical case)?

Kevin

dave reid’s picture

@kevinh456 In that case you'd add a column onto the {xmlsitemap} table for what search engine 'belongs' to each link. Then your module would provide context for generation based on that field.

The context process would have to be reversed when executing a page request.

agentrickard’s picture

FWIW, I have the same use-case as nonsie, and if I work on this patch (pending time approval), I need multiple languages across multiple domains.

I don't think we should add columns to the table, but use a context loading system like node_load() does.

dave reid’s picture

Arg, one of the major points of the rewrite was to do *as little as possible* when generating the files. That's the most intensive task and was helped greatly when we shifted to putting all the data in {xmlsitemap} ready to go. So I don't have a problem if other sub-modules want to add-on columns if they can do it. I'd really, really like to avoid any type of loading on file generation. Even this context idea has me thinking, but nervous.

agentrickard’s picture

Yes, but context really messes that up. They should be small hooks, and not used much. I really need time (and an excuse) to dive into this code.

/me must make that client pay for it.

ayalon’s picture

sounds very promising! I feel, that we make some progress although I cannot help with this issue.

antiorario’s picture

subscribing

dave reid’s picture

StatusFileSize
new40.81 KB

Brainstorming and how to best approach this before we hit 2.0...

I'm thinking the way we think about sitemaps and their contexts needs to be revamped. First, we'd have sitemaps, and sitemap contexts. Modules can provide sitemap contexts, and the site administrator chooses to add new sitemaps with different contexts.

For instance, the xmlsitemap_i18n module would expose the 'language' context. An xmlsitemap_domain module would expose a 'domain' context.

When we generate the files, we'd get an array of sitemaps and their associated contexts, and run it through a generate function. We'd figure out how to store each file based on context as well (like xmlsitemap-en-differentdomain.xml). Then when we the module runs the sitemap.xml page callback, it calls all the sub modules to figure out what the current page's 'context' is, and figures out which file to serve.

The attached screenshot shows kinda what I'm thinking.

Anonymous’s picture

That sure looks good from a user perspective!

The technical design sounds entirely sensible, too, but since I don't know the code I can't intelligently comment on that.

I'm really glad that it seems that getting this into 2.0 is a release goal.

agentrickard’s picture

Just to throw monkeys into the mix.

The site we have now has this setup:

Domain Access -- 8 domains
i18n -- 7 separate languages

Theoretically, we might need 2 or 3 language specific sitemaps per domain. Though if we could only generate one, that would probably be OK. But that one would need to allow filtering by language, I suppose.

Now if the URLs in that screenshot are configurable, and each configuration can set contexts for each module that declares one, then it might work.

kevinh456’s picture

@Dave Reid

I like that idea a lot. Let me restate it with @nonsie's example.

domain 0 (EN, FR, SW, RW)
domain 1 (EN, FR)
domain 2 (EN, SW).

So the language module would expose (EN, FR, SW, RW) and the domain module would expose (domain0, domain1, domain2).

So then, I would setup sitemaps for (EN, domain0), (FR, domain0), (SW, domain0), (RW, domain0), (EN, domain1), etc...?

On generate, it would use this list to generate context specific sitemaps. Each module would get a chance to modify the filename as the script runs.

Each module would also alter queries before they run? I guess my questions just get in to the nitty-gritty of this new system.

In theory, I think it's a very elegant solution to the problem. I guess it would just come down to implementation.

KevinH

dave reid’s picture

@agentrickard The URLs wouldn't be configurable. The contexts would be. So you could add a new domain-specific sitemap with Spanish language (http://differentdomain.com/es/sitemap.xml), etc. Basically you could use any combination possible except for certain links like a language-neutral (all) vs default language (en, etc.) since those would techincally have the same URL.

With so much refactoring involved with abstracting the sitemap and generation process, I doubt I'll have time to do this for 2.0, but for sure 2.1. If I get enough donations to push this in, possibly for 2.0. :)

dave reid’s picture

An example for kinda how this would work with xmlsitemap_i18n:

// This would be used on the sitemap table listing.
function xmlsitemap_i18n_xmlsitemap_context_info() {
  $context['language'] = array(
    'label' => t('Language'),
    'summary callback' => 'locale_language_name',
  );
  return $context;
}

// Used to determine the current context of the request.
function xmlsitemap_i18n_xmlsitemap_context_current() {
  global $language;
  $context['language'] = $language->language;
  return $context;
}

and in xmlsitemap.module:

function xmlsitemap_get_current_context() {
  $context = module_invoke_all('xmlsitemap_context_current');
  asort($context);
  return $context;
}

function xmlsitemap_get_context_file(array $context) {
  return 'xmlsitemap-' . implode('-', $context);
}

function xmlsitemap_show_sitemap_page() {
  $context = xmlsitemap_get_current_context();

  // Add chunk number as the last context.
  $chunk = isset($_GET['page']) && is_numeric($_GET['page']) ? $_GET['page'] : 0;
  $context[] = 'chunk' . $chunk;

  // Find the appropriate cached file and serve it.
  $file = xmlsitemap_get_context_file($context);
  file_transfer($file);
}
agentrickard’s picture

I think that sounds like it supports what I'm saying.

So on germany.example.com, I might have:

germany.example.com/de/sitemap.xml
germany.example.com/en/sitemap.xml

Which is different from the default

example.com/sitemap.xml

I wonder if we're accounting for other contexts?

dave reid’s picture

It's the only two primary contexts I could think of right now, but the beauty is that we would have hooks to provide whatever odd contexts people want. And this way context should be able to work with each other peacefully.

pumpkinkid’s picture

Wow, its amazing to see the collaboration you guys have gone through to figure this out... While most of it is over my head and I can only appreciate the process, I too would love to see this work!

finex’s picture

+1 for multiple sitemaps for each language on multiple domains (with different languages). It could be a killer SEO feature :-)

nonsie’s picture

I've got some time I can spend on this feature but I got stuck with #707538 before I even started.

Also is there an issue that describes features that have changed/will change in 2.x branch?

dave reid’s picture

@nonsie: I have to write up a blog post that I can use as a summary of all the changes that have gone in.

agentrickard’s picture

I fear this may end up as a hack-a-thon at DCSF....

dave reid’s picture

@agentrickard: Hey it might be fun. :)

agentrickard’s picture

The fear is only over the delay.

avpaderno’s picture

If the delay is necessary to find the definitive solution, then it should be welcome. I don't feel like getting a half baked solution that needs to be changed in a later version is the correct solution.

The solution reported by Dave would solve also another issue, and it's an elegant solution that would solve possible problems for different contexts.

damienmckenna’s picture

+1 for making it right the first time. I would like to request an alpha/beta build be released before this effort is started, just to give us something semi-stable to work from until the work is finished.

nonsie’s picture

I have an internal deadline of 03/26 which means I should have something working by then. However it is a low priority item and might as well get pushed a month.

dave reid’s picture

I'm working on transitioning the underlying generation code to use contexts and it seems to be working fairly well. One thing I've thought might be a problem is the following situation:

- We're saving sitemaps and their contexts (like the interface in #71) and using that data to perform file generation
- After sitemaps are saved and generated, user enables a module that exposes a new context (like xmlsitemap_domain or xmlsitemap_i18n)
- Now when user visits sitemap.xml, the new module says, hey I have a context, but since the sitemaps were generated without that context, we don't find the right hash (md5 of the $context array) and user gets a 404 since it could match a file to be displayed.

Right now I'm thinking that whenever you enable or disable a module that provides contexts, we'd have to search through sitemaps that have those contexts and disable them.

Maybe we need to have a 'default' sitemap that does not have any context at all. If the sitemap loader can't find the appropriate files, load the default sitemap files.

agentrickard’s picture

Yes to the 'default' idea.

nonsie’s picture

Seems fair enough. It will need to be well documented though as it can lead to a bunch of "my sitemap is out of date" bug reports.

dave reid’s picture

Here's the basic start of a xmlsitemap_domain module that would work once we revamp the context handling.

/**
 * Implements hook_xmlsitemap_context_info().
 */
function xmlsitemap_domain_xmlsitemap_context_info() {
  $context['domain'] = array(
    'label' => t('Domain'),
    'settings callback' => 'xmlsitemap_domain_xmlsitemap_domain_context_settings',
  );
  return $context;
}

/**
 * Implements hook_xmlsitemap_context().
 */
function xmlsitemap_domain_xmlsitemap_context() {
  $domain = domain_get_domain();
  $context['domain'] = $domain['domain_id'];
  return $context;
}

function xmlsitemap_domain_xmlsitemap_context_url_options(array $context) {
  $options = array();
  if (isset($context['domain']) && $domain = domain_load($context['domain'])) {
    $options['base_url'] = rtrim($domain['path'], '/');
  }
  return $options;
}

function xmlsitemap_domain_xmlsitemap_domain_context_settings(stdClass $sitemap) {
  $domains = domain_domains();
  $options = array();
  foreach ($domains as $domain) {
  	$options[$domain['domain_id']] = $domain['path'];
  }
  $form['domain'] = array(
    '#type' => 'select',
    '#title' => t('Domain'),
    '#options' => $options,
    '#default_value' => issset($sitemap->context['domain']) ? $sitemap->context['domain'] : '',
  );
  return $form;
}

Todos:
1. Revamp said context and sitemap handling.
2. Pre-associate domain_id with records in {xmlsitemap} so its ready during generation?
3. Does xmlsitemap_node's use of node_access() during cron/rebuilding get messed up since domain implements node grants?
4. Are there 'entities' other than nodes that are controlled/visible per domain (menus, taxonomy terms, or user accounts)?

dave reid’s picture

Need help solving #710978: Domain causes major testing errors because I will want to have a test to go along with xmlsitemap_domain.

nonsie’s picture

I'm not sure if this domain association data needs to be stored in {xmlsitemap} table. We could do a join to domain_access instead? This is also tied to #3 - perhaps the solution here is to allow each of the modules to alter the query that produces the links instead of allowing each module to define their own?
Menus and taxonomy terms can be controlled per domain - menus are built into Domain Access while taxonomy requires Domain Taxonomy module.

Trent Media’s picture

First of all, I want to thank the developers of this great module. You are doing a fantastic job!

And now to my question ;)
Can one of the developers perhaps give me a rough estimate of when the integration of domain access and xml sitemap will be completed (or when there will at least be a working development release).
Is it more likely weeks – months – years - ... ?

Because I have to build a new drupal based website (including domain access and a working sitemap), this would be a really helpful information for me.

dave reid’s picture

@95: Its being done on my own, unpaid free-time, so whenever I can devote time to getting it done. :) I'd like to have it done before DrupalCon SF and that might be an accurate estimate considering this is going to take a lot more work (properly integrating contexts, adding the new interface screen, writing tests, etc.).

dave reid’s picture

For anyone wishing to be a saint and donate for this feature, I've set up a chip-in at http://davereid.chipin.com/xml-sitemap-and-domain-integration. Note I'm not going to *not* work on this since I want this to happen as well. Donating is completely optional.

__Vlad__’s picture

We are also waiting for this function. In comment #96 you wrote that this function might be available at the time of DrupalCon SF (April).
Would donating some money make this process faster?

Balthasar’s picture

Are there any news of this add-on?
Dave wrote "I've set a personal deadline to accomplish this integration by Friday, March 12th." (http://davereid.chipin.com/xml-sitemap-and-domain-integration). But it seems that there is no new xml-sitemap with this function to download.

Dave, could you perhaps give us a new update what the status of this add-on is. Will it be available in the next days or not before DrupalCon SF.

Sorry to ask so many questions. But I am waiting for this add-on for a new project.

Anonymous’s picture

Priority: Normal » Critical

David, I'm looking for an update on this, I've sponsored the development but haven't seen any updates. You also mentioned that March 12th would be realistic to meet for delivery. I have not heard from you or seen any updates. Can you please let us know where things are at. Thank you.

linksync’s picture

sub

_Romy_’s picture

@ Dave Reid

Please give us some new information about what is happening with this integration.
We need to make some plans about the release of our new web portal and need XML Sitemap + Domain Access for it.

P.S. I hope nothing serious happened to Dave. No answer or activity for over a week, still the old and exceeded deadline on chipin ...

Anonymous’s picture

This is a little frustrating, because when I offered to pay the entire $500 to sponsor this development David responded immediately. Once payment was sent, no response has been received. I was willing to pay the module developer, and not outsource to another developer, because I know many contributors put in a lot of time to develop and maintain modules and it goes unpaid for. However, the fact that David is no where in sight, not responding to my emails or updating anyone on this forum is quite concerning. I too hope nothing has happened to the developer. However, I would really appreciate an update on the status of this development.

Anonymous’s picture

I suspect something unusual has happened with Dave as he is usually most responsive. And I hope that he and his loved ones are fine.

_Romy_’s picture

It seems that we can only wait until Dave returns. I hope it does not take too long because I really need this integration of domain access as soon as possible to release our new web portal.

But reading the feedback of dave's last work (http://www.odesk.com/users/Drupal-module-and-website-developer_~~875b6b3...) doesn't make me very hopeful :(

dave reid’s picture

Sorry everyone. I am still actively working on XML sitemap domain integration as fast as I can. It's been a lot more work than anticipated getting these changes in both the Drupal 7 and Drupal 6 branches since they need to stay in sync. Even more difficult considering there is not a Domain Access available for D7. Things have also gotten very busy and I'm still trying to figure out how to prioritize and balance everything. I got a fair amount of the API cleanup done this weekend and I will try to get a patch for Drupal 6 posted here or in the sub-project later tonight.

@kristyb2008: I apologize to you specifically as well. I missed the deadline for getting this done and I should have kept you better updated. I'd be more than willing to refund part or all of your donation since the March 12th deadline was not met.

@_Romy_: I won't make excuses. I'm not happy with how I did starting out freelancing. I've made several mistakes and I'm far from perfect. I'm taking steps and making an effort to improve myself as a person and a developer in 2010.

Anonymous’s picture

Any Progress??????? I haven't seen anything posted to the sub-module page either.

MJH’s picture

Is there a REALISTIC deadline that we can use for our plans?

agentrickard’s picture

@Dave

There is a Domain Access 7.x-alpha that should have enough for you to test. See the project page for the tarball or go to http://drupal.org/node/749854

panama-fan’s picture

Dave can’t you give us some updates on that project?

- Have you started to work on it?
- Is there some code that you could show us? Perhaps the community could help with some problems you still have?
- Is it nearly ready and only needs some “last improvements”

Some of us are using drupal in their companies and there might be a lot of money and customers involved…
Giving us two deadlines (March 5 -> “personal deadline March 12”, March 22 - > “patch later tonight (March 22)”) and disappearing for days / weeks after you have not kept them really does not help us.

If you have problems with / not enough time for that project, simply tell us the truth so that we can decide if we wait or if we look for another way to get this done.

dave reid’s picture

@agentrickard: Awesome. That will help immensely.

dave reid’s picture

StatusFileSize
new95.46 KB
new107.15 KB

Having a minisprint tonight working on this. Here's some of the current screenshots of the interface.

dave reid’s picture

CVS for Drupal 7 branch and the D7 version of domain_xmlsitemap have been updated. I also enabled the dev releases for the domain_xmlsitemap module. Most of the APIs and UIs are finished, but generating sitemap files does not currently work. Also currently working on the D6 backport.

dave reid’s picture

Make sure to file any domain sub-module issues at http://drupal.org/project/issues/domain_xmlsitemap

Anonymous’s picture

Subscribing.

Anonymous’s picture

Still no functional solution for D6?? Why can you not provide us with a time-line? I see another company sponsored this development, do they not have a deadline they need this for too?

Anonymous’s picture

@kristyb2008: I see a 6.x-dev version here http://drupal.org/project/domain_xmlsitemap. Have you tried it? Or you just failed to read where the development is happening? No that isn't it, you just wanted a speedier answer to #755672: Current State of 6.x version I guess.

agentrickard’s picture

@Dave

If it helps, Domain Source is now available for Drupal 7 testing, which may affect how canonical links are written. See the alpha 3 release.

http://drupal.org/node/758688

dave reid’s picture

@agentrickard Thanks for the update. Thanks for your work getting the domain suite for D7.

For everyone else I'm providing updates on this progress at http://drupal.org/node/755672. Please keep posted there. Anything non-constructive isn't going to make it go faster. I do have to work around my full-time job but I'm doing my best to schedule some regular time each day for XML sitemap.

Minsk-Drupal’s picture

>>> I do have to work around my full-time job but I'm doing my best to schedule some regular time each day for XML sitemap. <<<

As far as I know, the “xml sitemap / domain access” – project is being sponsored (and as it seems, it is sponsored from two sources: a company and through chip – in).

If that is correct, this project should be much more important to you. This means the project is not a "hobby" any more that can be done in the free time of your weekends. With money involved, it’s a regular job that you have accepted and that should be done professional and fast ((nearly) same priority as your "full-time job"). And that’s what many people are missing here.

Just my personal opinion.

I also thought about supporting this project. But I had the feeling (and still have it) that you would take the money but still handle this project as a hobby and not as a professional development job.

Anonymous’s picture

@Minsk-Drupal: How do you know how much has been given to Dave? Has anyone given Dave enough money for Dave to declare this project "full-time" work? I seriously doubt it. Hobbies still need to be funded and my hats off to Dave for finding a method to do so. I suppose though, that those that have donated funds to Dave have more right to complain than others but since you've not considered Dave to be worthy of funding then *you* do not have the right to give such a tongue lashing to Dave.

Thanks Dave for the work you've given to xmlsitemap and other projects of Drupal. Without your work we wouldn't be anywhere near where we are with xmlsitemap regardless of the complainers who seem to do nothing but complain.

damienmckenna’s picture

What @earnie said.

BarisW’s picture

*sigh*. I fully agree with Damien and Earnie. But please stick to the topic :)

@Minsk-Drupal: if this fix is so important to you: step in and help. Or just wait for it to be fixed. If you don't like this way of working, then don't use open source.

dave reid’s picture

Priority: Critical » Normal

Ok, holy crap. Over this weekend and today I've finally got all the rebuilding and regenerating code working with contexts now. Basically had to revamp a *lot* of code, but it's worth it now. It needs some polish, but its working and needs to be backported to Drupal 6 now.

Exploratus’s picture

Nice! Good to hear the progress is coming along well. I am sure a lot of people are anxiously awaiting like me... Cheers.

dave reid’s picture

Initial integration should now be working. Still working out some of the rough edges, but the API and UI are working. Feel free to test. Once I get enough feedback from testers I'll release the next alpha and officials of the domain_xmlsitemap module.

ayalon’s picture

Hi Dave!

Thanks for your effort on this very important feature.

Unfortunatley I'm not able to test the new feature because I get a White Screen of Death as soon as I copy the xmlsitemap module to my module folder on the drupal module page. Because of this, I'm not able to activate the module. Sadly I don't see any error message and don0t find anything in the logs, but as soon as I delete the module folder, the module overview of drupal starts working again.

MJH’s picture

@ayalon #127

XML sitemap and Domain XML sitemap work perfectly on my server.

But I had the same problem (White Screen of Death) yesterday after installing the latest views package (6.x-2.10). The problem was my Alternative PHP Cache (APC). There was some old code in the cache that interfered with the new views 6x-2.10 package. The solution was to restart Apache (of course that only works if you have your own server). Perhaps that helps somehow...

finex’s picture

Hi! I'd like to test the new domain sitemap for D6. Is it available in the -dev version? I could test and give a feedback.

dave reid’s picture

@FiNeX: It's available using both 6.x-2.x-dev of XML sitemap and 6.x-1.x-dev of XML sitemap domain

dave reid’s picture

@ayalon Check your server's error logs, and/or restart like MJH suggests in #128.

dave reid’s picture

Status: Active » Fixed

Marking this as fixed as the majority of people reporting back that it is working as expected. Please file new issues for any bugs encountered.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.