Generalize to work with any Entity Type

So yeah, the plan is to eventually have a version that is entity-agnostic since it makes sense. However my experience implementing those types of modules so far in Drupal 7 has proven horribly frustrating (especially with respect to determining which entity types are 'publicly visible' e.g. Pathauto, Meta tags, etc) - #1332058: Need a way to distinguish "public/final" URIs for entities from admin and redirected ones. For example, comment module implements the entity URI callback when it doesn't actually have it's own pages - they just redirect to node pages. Commerce products implement entity URI pointing to the admin URL of the products, not the public-facing URL. So it's a combined matter of frustration with D7 and not being spread too thin between modules.

We're also missing a unified way to alter the 'bundle' forms for entities (e.g. node_type_form, taxonomy_form_vocabulary, etc).

Ideally this rewrite would happen on a 7.x-3.x branch which makes me queasy since 7.x-2.x isn't technically "finished" yet, but it would help clean things up a bit.

So there's no way to get an entity like Product or Profile2 into an XML Sitemap???

I am trying to figure out how to get some custom entities into XML Sitemap, but I am not sure if I will end up making it general enough for all Entity Type.

After writing some code to integrate my custom entity into XML Sitemap, as well as writing a basic module to integrate Profile2 with XML sitemap. I am not sure if it's possible at all at this moment to write something entity-agnostic. I don't even see a simple way to figure out if an entity has an unique ID. I am thinking maybe the better approach, at least for the time being, is to provide a simple hook for modules with custom entities to easily integrate with xmlsitemap. Currently we need to connect to about 8 hooks to get a custom entity into xmlsitemap.

@kevinquillen, here is my first attempt to integrate Profile2 with XML sitemap. http://drupal.org/sandbox/makangus/1783144

@makangus, what is the status of your profile2_xmlsitemap project?

That's exactly what I need for a project I'm developing and where users profiles (daycares) need to be added to the xmlsitemap so that they can be indexed by search engines.

@bisonbleu, I am not doing much with profile2_xmlsitemap but it does work. You might be able to achieve the same results with the xmlsitemap_user module that comes with xmlsitemap, so I would recommend trying that first. If you do end up using my sandbox module though, feel free to file any bugs.

It has to be a case by case basis due to how entities are implemented through EntityAPI - that's what I have found. You could make some workarounds but it is a bit of work.

The xmlsitemap_user module is not very useful for my uses case because I use the separate profile page option. User links in xmlsitemap point to user accounts where all you can see is the following.

History

Member for
x month y weeks

That's why there's no unified solution, yet.

If you work with the xmlsitemap_profile2 you can adapt it to do what you need.

@bisonbleu, you can try my sandbox module and see how that works out for you, but let's move this conversation about profile2 to the issue queue there http://drupal.org/project/issues/1783144.

Let's keep this thread about XMLSitemap for ALL Entity Type.

The apachesolr module does this really nicely and instantly provides any entity type that is set as "indexable" in it's config form. This is sourced from hook_apachesolr_entity_info_alter which is invoked in apachesolr_entity_info_alter and by simply setting $entity_info['my_entity']['indexable'] = TRUE; it is exposed to the index config form.

Could a similar methodology not be implemented for xmlsitemap?

Thanks @makangus for the sandbox, I used it to implement xmlsitemap functionality for my entity type. Copy-paste-find-replace FTW :D

I've written a patch - using makangus code - to add this feature for all Entity API powered entities.

Please note that an uri and access callback are required. And one of the uri methods must be overridden when entity_class_uri() is used as URI callback.

Matthijs

Fixed "apply patch" issue and some other remaining bugs...

One of the select queries was missing the field to select...

Another quick fix, the bundle should be empty...

Hi...

I tried your patch to get my custom entities into xml sitemap but when I generated the sitemap the following the batch was interrupted and the following messages appeared in drupal logs:

Warning: Missing argument 2 for xmlsitemap_entity_xmlsitemap_process_entity_links(), called in sites/all/modules/xmlsitemap/xmlsitemap.generate.inc on line 456 and defined in xmlsitemap_entity_xmlsitemap_process_entity_links() (Line 212 of /sites/all/modules/xmlsitemap/xmlsitemap_entity/xmlsitemap_entity.module).

Notice: Undefined variable: entity_type in xmlsitemap_entity_xmlsitemap_process_entity_links() (Line 213 of /sites/all/modules/xmlsitemap/xmlsitemap_entity/xmlsitemap_entity.module).
	
Notice: Undefined index: controller class in entity_get_controller() (Line 7836 of /includes/common.inc).

Additionally I find no place where I can define the standard xml settings for the entity type. Here you can see the relevant parts of my entity hook info:

      'admin ui' => array(
        'path' => 'admin/path/entity',
        'menu wildcard' => '%entity',
        'file' => 'customentitiy.admin.inc',
      ),
     'xmlsitemap' => TRUE,

Maybe I miss something... on my admin page I just see a list of my entities and the link to add a new entity which is quite the standard I think?!

Thanks to @matthijs for the patches!

I found a similar issue mentioned in #24

I didn't get the Missing Argument error, but was unsure how to set the status (inclusion) and priority of each bundle.

I've done some digging.

In my case I'm using ECK, so there is nothing in the patch in #23 which provides a per bundle setting for custom (or ECK) entities- or at least from what I could see.

I got the entities to be included by adding a hook_entity_info_alter() in my custom module like this:

/**
 * Implements hook_entity_info_alter().
 */
function MY_MODULE_entity_info_alter(&$entity_info) {
  $entity_info['product']['xmlsitemap'] = TRUE;
}

(if you're creating own custom entities (not ECK) you should be able to add this to your hook_entity_info())

This allowed for the xmlsitemap inclusion in the xmlsitemap_entity module (as provided in #23), but there was still an additional step required. The main xmlsitemap module looks for the bundle settings (normally set in the admin UI). In xmlsitemap_link_bundle_load() it looks for your custom bundle settings in the variable table, much like it does for node. However, given that there is no UI to set this variable (in the case of ECK entities) I had to add these lines to my settings.php:

$conf['xmlsitemap_settings_product_widget'] = array(
  'status' => '1',
  'priority' => '1.0',
);

...where 'product' is the ECK entity type and 'widget' is the bundle.

Flush caches and everything worked as expected.

Note: I'm not concerned with individual entity priorities, so these steps aren't a complete fix for ECK. I might be able to do this by looking at the way xmlsitemap_entity_form_alter() is hooking into the entity form. The conditions in this function won't match those of ECK's entity forms, so some work would be required there is that's what you need to achieve.

Hope this helps somebody.

I use this for field-collection-items. It seems to work, but as field-collections don't have an edit bundle form (as for example nodes have), there is no place to set the status and priority settings. It might be done in the field settings, but this is maybe to complicated for a generic approach. I set the variable xmlsitemap_settings_field_collection_item_{field-collection-name} manually (via hook_update_N() implementation in a custom module). Maybe the xmlsitemap_entity module should get it's own admin page?

I haven't got the errors from #23, it's not clear why argument 2 (variable $entity that contains the entity-type) would be missing there. Probably the patch was not applied completely as the $entity variable is used in different places of the same function and it absence would have caused more errors. The patch in #23 adds $entity as second parameter to the 'process callback', the only place where xmlsitemap_entity_xmlsitemap_process_entity_links() gets called in that function.

I was wondering about the function xmlsitemap_link_queue_process() in xmlsitemap.module. There is not second parameter (entity-type) for the 'process callback'. It couldn't be added that easy, as there is another function, xmlsitemap_menu_xmlsitemap_process_menu_links() which receives a second parameter, although it is not set in xmlsitemap_link_queue_process(). Anyway we could not just add $data['type'], which seems to be the entity-type as a second parameter to the callback, as it would get misinterpret in xmlsitemap_menu_xmlsitemap_process_menu_links().

Therefore I changed the second parameter for all xmlsitemap_*_xmlsitemap_process_*_links() callbacks (only the one above mentioned and the new one xmlsitemap_entity_xmlsitemap_process_entity_links() into a data array which may contain different information.

I haven't yet tested the sitemap for xmlsitemap_menu, but I hope the changes won't affect it.

I get following warning when importing redirects with the Path redirect import module (having the File Entity module installed):

Notice: Undefined property: stdClass::$pid in xmlsitemap_entity_entity_delete() (line 146 of [...]/xmlsitemap/xmlsitemap_entity/xmlsitemap_entity.module).

There entity to be deleted is the (temporary) redirect csv file temporary://redirects.csv.

Would it be okay to test for pid like in the following code snipped?

  if (!empty($info['xmlsitemap']['use entity module']) && isset($entity->pid)) {
    xmlsitemap_link_delete($entity_type, $entity->pid);
  }

This would be fine for the temporary file, as it isn't be part of the sitemap anyway. I am not sure if that is fine in general for all other entities (like other file entities which might be part of the sitemap).

I am getting this error too, when deleting a file.

I don't know what the pid property is meant to be, but files don't have it. It's not something that xmlsitemap sets (there is no other mention of it anywhere in the module's code).

Lots of other entities don't have it either, but 'use entity module' is set on entity types that satisfy fairly general criteria:

    // Check if this type of entity can be indexed.
    $indexable = (isset($info['entity keys']['id']) && isset($info['uri callback']) && isset($info['access callback']));

That's going to be a lot of entity types!

Is pid perhaps a typo?

Yup, looking at the signature of xmlsitemap_link_delete(), this is a typo.

The fix is to use entity_extract_ids() to get the ID before calling the function.

Patch changed as suggested in #32:

   if (!empty($info['xmlsitemap']['use entity module'])) {
-    xmlsitemap_link_delete($entity_type, $entity->pid);
+    list($entity_id) = entity_extract_ids($entity_type, $entity);
+    xmlsitemap_link_delete($entity_type, $entity_id);
   }

Can anyone confirm the latest patch works for custom entities built with ECK?

+++ b/xmlsitemap_entity/xmlsitemap_entity.api.php
@@ -0,0 +1,27 @@
+ * Describes the hooks provided by the Workflow module.

Workflow should be replaced with XML sitemap entity.

I enabled the module and got the following error:
Warning: Invalid argument supplied for foreach() in element_children() (line 6540 of includes/common.inc).

Would it be possible to add some instruction to the README? The patch applied fine, but I wasn't able to make it work. My custom entities were apparently indexed fine, but not added to the sitemap.

@milos.kroulik
I've added some code to my module in form_alter hook (you can use specific, I've added to general):

if ($form_id == 'eck__bundle__edit_form') {
    module_load_include('inc', 'xmlsitemap', 'xmlsitemap.admin');
    xmlsitemap_add_link_bundle_settings($form, $form_state, $form['entity_type']['#value']->name, $form['bundle']['#value']->name);
  }

This code adds a fields to a form and submit function, that's why you don't need do nothing else to get it works.

Need some help on this since i've already patched the codes that we have here but the custom entity that i have is still not yet visible. I already have it indexed and available but its not going to VISIBLE thus not showing on the XML sitemap.

Patch in #41 worked well but did not support adding files to the sitemap. In this patch I added add a form element to the file-type edit form. By editing a file-type and setting the "Inclusion" item to "Include" files are added to sitemap.xml when cron runs.

Re rolled the patch in #43 - which was based on the patch in #41. This updated code adds a new module, xmlsitemap_file which supports adding files to the sitemap. The module also appends "/download" to the file path before adding the link to sitemap.xml - this way a crawler can actually access the contents of a .pdf file.

This patch adds a module to the patch in #41. It adds support for adding files to sitemap.xml and allows for "/download" to be added to url so a text file can be displayed in the browser and indexed.

This patch adds a module (xmlsitemap_file) to the patch in #41.

XML sitemap file supports adding files to sitemap.xml and allows for "/download" to be added to link url (optional) so a text file can be displayed in the browser and indexed.

XML sitemap file passes "timestamp" as "changed property" so $entity->timestamp is used for "Last modification date"

patch fixes issue in comment #35 - replaces reference to "Workflow module" with "XML sitemap entity module"

Please review

This update adds the per-entity override form to file entities, so files can be individually included/excluded. It also adds some code that seems to be needed in order for the entity edit form to remember the overrides for an entity. Interdiff attached.

General notes: The xmlsitemap_file module added in recent patches seems like a good example of leveraging the xmlsitemap_entity module for integration with another contrib module. I notice that ECK (entity construction kit) needs a bit of help as well, in getting the various form elements added. Maybe a similar module (xmlsitemap_eck?) could be added in this patch? Field Collection is mentioned above also, so might be another candidate for a module.

It could be argued that these contrib integrations should be in separate patches, and that only the xmlsitemap_entity module should be added here. I would support that, except that it's not clear what the xmlsitemap_entity module should do by itself. If it's only an "API module", then that should be made clear, and maybe an example implementation provided (like the xmlsitemap_file from recent patches).

Looking at the interdiff from #41 to #47 (in comment #47) the second argument to the process callback (array('entity_type' => $entity)) was removed again, which causes it to fail indexing for me. Subsequent patch in #48 doesn't seem to fix that.

Also, xmlsitemap_entity_query_alter() contains a fix for EntityFieldQueries for users (because of #1054168: EntityFieldQuery fails for entities that have no bundle). As long as #1054168: EntityFieldQuery fails for entities that have no bundle is not fixed, maybe we should add a fix for entities without bundles in general?

Thanks for pointing that out @Jelle_S about the argument issue - I'm getting the same thing. Here's an attempt to add that code back in.

Your idea about entities without bundles seems reasonable to me. I don't have time at the moment to try implementing that, but it does seem like something that should be done in this issue.

My stab at it.

Rerolled against latest dev.

I installed the latest dev version and recognized some problems:

• The sitemap urls result in a 404 page not found
• I have two languages installed on my page but I can't choose a language for the sitemaps.
• I have no UI to choose the entities that I want to include in the sitemap
• Field collections are not included in the sitemap

Running cron, deleting cache and rebuilding sitemaps do not resolve this.

Regarding the file entity integration: After further testing in production I believe that a change I added to this patch in #48 was probably unnecessary. Instead a simpler form alteration is need for a specific use-case involving some common file-related modules (file_entity, media, media_bulk_upload, and multiform). So in the interests of keeping unnecessary stuff out of the patch, here is an update to make those changes.

In regards to the review by @ThuleNB: This is great feedback. It sounds like this patch does NOT provide complete integration with field_collection entities. Getting feedback like this on a variety of entity types will probably be needed to truly "generalize" this module.

@ThuleNB: I don't have much experience using field collections as stand-alone entities -- they do get their own URLs? I always thought field collections were purely for attaching to other entities, but sounds like I'm mistaken there?

General note: Obviously also, this patch is really in need of some tests.

@brockfanning: That is the way I understood as wel, but from the Field Collection project page:

A field collection is internally represented as an entity, which is embedded in the host entity. Thus, if desired field collections may be viewed and edited separately too.

So yes, they do get their own urls.

See also field_collection_field_get_path() and FieldCollectionItemEntity::defaultUri().

@Jelle_S: A minor tweak to the entity query alter, to avoid a PHP warning that happens in my Drupal install.

Arg, fixing a typo.

Hm, weird, seems like entityConditions should never be a string... See https://api.drupal.org/api/drupal/includes%21entity.inc/function/EntityF...

Wich means we should not only check the value, but also the operator...

Which got me thinking. Maybe we shouldn't be fixing this here... It seems this is in place to fix the uses of EntityFieldQuery in xmlsitemap itself. Maybe we should fix it there (used in xmlsitemap.drush.inc, xmlsitemap.generate.inc, xmlsitemap.module). We could just do the same check there before adding the bundle condition. Maybe that's something to fix in a different patch though? Since a similar hook is already in place for the user entity.

@Jelle_S: Yes I think I'm leaning that way too. Probably the best this patch can do is to lay the (potentially incomplete) groundwork for support of other entity types, and then maybe to include complete support for some common/popular entity types (like the currently included xmlsitemap_file). If that entity query alter is mainly for support of user entities, then it may belong in an "xmlsitemap_user" submodule?

Well it's for bundle-less entity types, of which user is a core example. But we can't special case it for user since https://www.drupal.org/project/user_bundle exists. That is why I created the patch over at #3011651: Fix EntityFieldQuery uses. It checks the entity info to see if the entity type we're querying for has bundles. It's a generic approach that should work for all entities (even when they're altered like User Bundle does). That patch also removes the hardcoded special case for user entities.

Did some code cleanup (trailing spaces, comment punctuation, code extraction to a separate function, and some early returns to limit if-nesting). If or when #3011651: Fix EntityFieldQuery uses get committed we can remove xmlsitemap_entity_entity_query_alter().

Fixed some copy paste errors.

Patch contained changes from the other issue...