Hello,

I would really appreciate some help debugging a serious problem I am having with Views. I have set up a node view based on 2 taxonomies.

Essentially, I want to be able to control the view according to any term (at any level in the heirarchy) in the first taxonomy ("countries").
And also control it according to parent and child terms, separately, in the second vocabularly (called "types". there are only 2 levels in the heirarchy of this second taxonomy and each node is categorized to a child term).

I have set up 3 arguments for the view:
1) Taxonomy: Term ID (with depth) - relating to the first taxonomy. I have selected taxonomy term argument validator, specified the "countries" vocabulary & "term name/ synonym converted to Term ID" as argument type. I have not specified a relationship against this argument.
2) (Parent) Taxonomy: Term ID - relating to the parent terms of second taxonomy. I have linked this to a parent relationship that itself references the "types" relationship below. Selected taxonomy term argument validator, specified "types" vocabulary, & "term name/ synonym converted to Term ID" as argument type.
3) (types) Taxonomy: Term - relating to the child terms of second taxonomy linked to a relationship ("types") that specifies that vocabulary. Selected taxonomy term argument validator, specified "types" vocabulary, & "term name or synonym" as argument type (I cannot use Term ID on this argument as some of the terms in this taxonomy are duplicated under different parents. Using Term ID seemed to resolve the term name to only one of these child terms, rather than all matching a given term).

The View seemed to be working fine, exactly as I wanted it to, with about 2000 nodes. However, when I increased the site size to c. 65000 nodes I experienced Ajax error messages when saving the view "An error occurred at /admin/build/views/ajax/preview/", although the view nevertheless saved when I dismissed the messages.

More seriously, the view itself then seemed to experience major performance issues. It often worked fine when some of the arguments were set (eg United States/Other/all ) but when there are no arguments set (eg all/all/all ) the query it triggers never resolves and the MySQL server crashes.

There is nothing obvious I can see that I am doing wrong (although, I have to admit, I am relatively new to Views), so possibly the problem results from a bug?

I believe the SQL processes that causes the problem is:

SELECT COUNT(*) FROM (SELECT node.nid AS nid\n FROM node node \n LEFT JOIN (SELECT td.*, tn.vid AS revision FROM term_data td INNER JOIN term_node tn ON tn.tid = td.tid WHERE td.vid IN (3)) term_data_node ON node.vid = term_data_node.revision\n LEFT JOIN term_hierarchy term_data_node__term_hierarchy ON term_data_node.tid = term_data_node__term_hierarchy.tid\n LEFT JOIN term_data term_data_term_hierarchy ON term_data_node__term_hierarchy.parent = term_data_term_hierarchy.tid\n WHERE node.type in ('names')\n   ) count_alias

The export of the view is below.

Thanks for the great module and very much hope you can help.

$view = new view;
$view->name = 'Names';
$view->description = '';
$view->tag = '';
$view->view_php = '';
$view->base_table = 'node';
$view->is_cacheable = FALSE;
$view->api_version = 2;
$view->disabled = FALSE; /* Edit this to true to make a default view disabled initially */
$handler = $view->new_display('default', 'Defaults', 'default');
$handler->override_option('relationships', array(
  'term_node_tid' => array(
    'label' => 'types',
    'required' => 0,
    'vids' => array(
      '3' => 3,
      '1' => 0,
    ),
    'id' => 'term_node_tid',
    'table' => 'node',
    'field' => 'term_node_tid',
    'relationship' => 'none',
  ),
  'parent' => array(
    'label' => 'Parent',
    'required' => 0,
    'id' => 'parent',
    'table' => 'term_hierarchy',
    'field' => 'parent',
    'relationship' => 'term_node_tid',
  ),
));
$handler->override_option('fields', array(
  'title' => array(
    'label' => 'Title',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'link_class' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'target' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'empty' => '',
    'hide_empty' => 0,
    'empty_zero' => 0,
    'link_to_node' => 0,
    'exclude' => 0,
    'id' => 'title',
    'table' => 'node',
    'field' => 'title',
    'relationship' => 'none',
  ),
  'name' => array(
    'label' => 'Sub category',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 1,
      'path' => 'names/[name]',
      'link_class' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'target' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'empty' => '',
    'hide_empty' => 0,
    'empty_zero' => 0,
    'link_to_taxonomy' => 0,
    'exclude' => 0,
    'id' => 'name',
    'table' => 'term_data',
    'field' => 'name',
    'relationship' => 'term_node_tid',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'name_1' => array(
    'label' => 'Category',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'link_class' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'target' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'empty' => '',
    'hide_empty' => 0,
    'empty_zero' => 0,
    'link_to_taxonomy' => 0,
    'exclude' => 0,
    'id' => 'name_1',
    'table' => 'term_data',
    'field' => 'name',
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'parent',
  ),
  'tid' => array(
    'label' => 'Region',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'link_class' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'target' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'empty' => '',
    'hide_empty' => 0,
    'empty_zero' => 0,
    'type' => 'separator',
    'separator' => ', ',
    'link_to_taxonomy' => 1,
    'limit' => 1,
    'vids' => array(
      '1' => 1,
      '3' => 0,
    ),
    'exclude' => 0,
    'id' => 'tid',
    'table' => 'term_node',
    'field' => 'tid',
    'relationship' => 'none',
  ),
  'created' => array(
    'label' => 'Post date',
    'alter' => array(
      'alter_text' => 0,
      'text' => '',
      'make_link' => 0,
      'path' => '',
      'link_class' => '',
      'alt' => '',
      'prefix' => '',
      'suffix' => '',
      'target' => '',
      'help' => '',
      'trim' => 0,
      'max_length' => '',
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),
    'empty' => '',
    'hide_empty' => 0,
    'empty_zero' => 0,
    'date_format' => 'small',
    'custom_date_format' => '',
    'exclude' => 0,
    'id' => 'created',
    'table' => 'node',
    'field' => 'created',
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'none',
  ),
));
$handler->override_option('sorts', array(
  'nid' => array(
    'order' => 'DESC',
    'id' => 'nid',
    'table' => 'node',
    'field' => 'nid',
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'none',
  ),
));
$handler->override_option('arguments', array(
  'term_node_tid_depth' => array(
    'default_action' => 'ignore',
    'style_plugin' => 'default_summary',
    'style_options' => array(),
    'wildcard' => 'all',
    'wildcard_substitution' => 'All',
    'title' => '',
    'breadcrumb' => '',
    'default_argument_type' => 'fixed',
    'default_argument' => '',
    'validate_type' => 'taxonomy_term',
    'validate_fail' => 'ignore',
    'depth' => '3',
    'break_phrase' => 0,
    'set_breadcrumb' => 0,
    'id' => 'term_node_tid_depth',
    'table' => 'node',
    'field' => 'term_node_tid_depth',
    'validate_user_argument_type' => 'uid',
    'validate_user_roles' => array(
      '2' => 0,
    ),
    'relationship' => 'none',
    'default_options_div_prefix' => '',
    'default_argument_user' => 0,
    'default_argument_fixed' => '',
    'default_argument_php' => '',
    'validate_argument_node_type' => array(
      'history' => 0,
      'names' => 0,
      'page' => 0,
      'story' => 0,
      'twitter_history' => 0,
      'twitter_names' => 0,
      'youtube_names' => 0,
    ),
    'validate_argument_node_access' => 0,
    'validate_argument_nid_type' => 'nid',
    'validate_argument_vocabulary' => array(
      '1' => 1,
      '3' => 0,
    ),
    'validate_argument_type' => 'convert',
    'validate_argument_transform' => 0,
    'validate_user_restrict_roles' => 0,
    'validate_argument_php' => '',
    'override' => array(
      'button' => 'Override',
    ),
  ),
  'tid_1' => array(
    'default_action' => 'ignore',
    'style_plugin' => 'default_summary',
    'style_options' => array(),
    'wildcard' => 'all',
    'wildcard_substitution' => 'All',
    'title' => '',
    'breadcrumb' => '',
    'default_argument_type' => 'fixed',
    'default_argument' => '',
    'validate_type' => 'taxonomy_term',
    'validate_fail' => 'not found',
    'break_phrase' => 0,
    'not' => 0,
    'id' => 'tid_1',
    'table' => 'term_data',
    'field' => 'tid',
    'relationship' => 'parent',
    'validate_user_argument_type' => 'uid',
    'validate_user_roles' => array(
      '2' => 0,
    ),
    'override' => array(
      'button' => 'Override',
    ),
    'default_options_div_prefix' => '',
    'default_argument_user' => 0,
    'default_argument_fixed' => '',
    'default_argument_php' => '',
    'validate_argument_node_type' => array(
      'history' => 0,
      'names' => 0,
      'page' => 0,
      'story' => 0,
      'twitter_history' => 0,
      'twitter_names' => 0,
      'youtube_names' => 0,
    ),
    'validate_argument_node_access' => 0,
    'validate_argument_nid_type' => 'nid',
    'validate_argument_vocabulary' => array(
      '3' => 3,
      '1' => 0,
    ),
    'validate_argument_type' => 'convert',
    'validate_argument_transform' => 0,
    'validate_user_restrict_roles' => 0,
    'validate_argument_php' => '',
  ),
  'name' => array(
    'default_action' => 'ignore',
    'style_plugin' => 'default_summary',
    'style_options' => array(),
    'wildcard' => 'all',
    'wildcard_substitution' => 'All',
    'title' => '',
    'breadcrumb' => '',
    'default_argument_type' => 'fixed',
    'default_argument' => '',
    'validate_type' => 'taxonomy_term',
    'validate_fail' => 'not found',
    'glossary' => 0,
    'limit' => '0',
    'case' => 'none',
    'path_case' => 'none',
    'transform_dash' => 0,
    'add_table' => 0,
    'require_value' => 0,
    'id' => 'name',
    'table' => 'term_data',
    'field' => 'name',
    'validate_user_argument_type' => 'uid',
    'validate_user_roles' => array(
      '2' => 0,
    ),
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'term_node_tid',
    'default_options_div_prefix' => '',
    'default_argument_user' => 0,
    'default_argument_fixed' => '',
    'default_argument_php' => '',
    'validate_argument_node_type' => array(
      'history' => 0,
      'names' => 0,
      'page' => 0,
      'story' => 0,
      'twitter_history' => 0,
      'twitter_names' => 0,
      'youtube_names' => 0,
    ),
    'validate_argument_node_access' => 0,
    'validate_argument_nid_type' => 'nid',
    'validate_argument_vocabulary' => array(
      '3' => 3,
      '1' => 0,
    ),
    'validate_argument_type' => 'name',
    'validate_argument_transform' => 0,
    'validate_user_restrict_roles' => 0,
    'validate_argument_php' => '',
  ),
));
$handler->override_option('filters', array(
  'type' => array(
    'operator' => 'in',
    'value' => array(
      'names' => 'names',
    ),
    'group' => '0',
    'exposed' => FALSE,
    'expose' => array(
      'operator' => FALSE,
      'label' => '',
    ),
    'id' => 'type',
    'table' => 'node',
    'field' => 'type',
    'override' => array(
      'button' => 'Override',
    ),
    'relationship' => 'none',
  ),
));
$handler->override_option('access', array(
  'type' => 'none',
));
$handler->override_option('cache', array(
  'type' => 'none',
));
$handler->override_option('use_pager', '1');
$handler->override_option('style_plugin', 'table');
$handler->override_option('style_options', array(
  'grouping' => '',
  'override' => 1,
  'sticky' => 0,
  'order' => 'asc',
  'columns' => array(
    'title' => 'title',
    'name' => 'name',
  ),
  'info' => array(
    'title' => array(
      'sortable' => 0,
      'separator' => '',
    ),
    'name' => array(
      'sortable' => 0,
      'separator' => '',
    ),
  ),
  'default' => '-1',
));
$handler = $view->new_display('page', 'Page', 'page_1');
$handler->override_option('path', 'charts');
$handler->override_option('menu', array(
  'type' => 'none',
  'title' => '',
  'description' => '',
  'weight' => 0,
  'name' => 'navigation',
));
$handler->override_option('tab_options', array(
  'type' => 'none',
  'title' => '',
  'description' => '',
  'weight' => 0,
));

Comments

merlinofchaos’s picture

I don't think this is a bug, I just think you're creating a view that ends up with a hideously slow query. Using that much taxonomy is rarely a good thing with Views.

danieldd’s picture

Thanks for the extremely quick response. Is there anything you can see that I am doing wrong in setting up the view?

Or is it then, just that Views is not set up to handle queries against multiple taxonomies, or different levels in a taxonomy the way I am attempting? Would using CCK fields instead as arguments/filters (or content taxonomy module) improve the performance?

If the problem was trying to load too much into the query are the Ajax errors I experienced when saving the view also a symptom of this?

thanks again. Really appreciate your help.

merlinofchaos’s picture

I would say that the taxonomy data structures are set up in such a way that really complex queries against them (multiple taxonomies across multiple vocabularies) produce very poor queries. I tend to suggest to people that they not use taxonomy for things that aren't actively categorizing and tagging. Using several vocabularies to control a view will probably always result in a poorly performing query, unfortunately.

You might well have better results with CCK for at least one of those. (Though the hierarchical one may be difficult). But there is always the content taxonomy module, too.

danieldd’s picture

Thanks for the advice.

It's actually not ideal to use CCK as both my vocabularies are heirarchical. But, having dug around a bit, I've found that actually its' possible to configure the 3 arguments, against the two taxonomies in a way that does not (fingers crossed) seem to have performance issues or display the errors I experienced before.
1) Taxonomy: Term ID (with depth) - validated against taxonomy term, converted to term id, to get any term from vocabulary 1.
2) Taxonomy: Term ID (with depth) - validated against taxonomy term, converted to term id, can be used to validate parents (or children) for vocab 2
3) Taxonomy: Term - can be used to validate against child terms for vocab 2.

The performance issues seem to arise when creating taxonomy based relationships (particularly two linked relationships) - which is something that is not required for the above configuration.

However, while I can configure arguments that validate against parent or child terms, it does not currently seem possible to display the parent terms of a node (and particularly to display the terms of just one taxonomy, and/or avoid duplicate nodes) without creating multiple relationships. There do not seem to be the same range of taxonomy options against "fields" as there are against "arguments" in the Views UI. Presumably this is because the queries required are different/ more complex? In any case this is not a major issue for me, but would be a nice feature request if it was simple to implement. Thanks again.

esmerel’s picture

Status: Active » Closed (fixed)

No activity for 6 months

d.clarke’s picture

Version: 6.x-2.8 » 6.x-3.x-dev
StatusFileSize
new2.87 KB

I think I ran into the same issue and after a little digging into it I think I have a solution. It appears that the issue is caused by inefficiencies by joining on the sub select. If you do an explain on the view, or if you 'show full processlist' while the query is executing, you'll see that the delay is from MySQL having to create a tmp table and write it to disk. If the query is rewritten to use two joins instead of one on a sub-select, the problem is resolved. In my case the original query was taking 51-52 seconds on average. After the patch it decreased to just over .1 seconds (the result set is identical). The patch also improves performance by replacing the IN clause with an equals comparison when only comparing against a single value. In my case that saved one second off the total query.

BTW... This sub select only appears if you have an optional relationship. So if you are experiencing this issue and you can make the relationship required, that should resolve the issue for you in the meantime.

Here is the original query (I replaced the select fields with just a count(*) to shorten it for the post)... note the difference at the 7th join...

mysql> SELECT count(*) FROM node node  LEFT JOIN term_node term_node ON node.vid = term_node.vid  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid  LEFT JOIN term_hierarchy term_hierarchy ON term_data.tid = term_hierarchy.tid  INNER JOIN term_data term_data_term_hierarchy ON term_hierarchy.parent = term_data_term_hierarchy.tid  LEFT JOIN content_type_event node_data_field_event_venue_ref ON node.vid = node_data_field_event_venue_ref.vid  INNER JOIN node node_node_data_field_event_venue_ref ON node_data_field_event_venue_ref.field_event_venue_ref_nid = node_node_data_field_event_venue_ref.nid  LEFT JOIN (SELECT td.*, tn.vid AS revision FROM term_data td INNER JOIN term_node tn ON tn.tid = td.tid WHERE td.vid IN (3)) term_data_node ON node.vid = term_data_node.revision  LEFT JOIN content_type_event node_data_field_event_performer_ref ON node.vid = node_data_field_event_performer_ref.vid  LEFT JOIN node node_node_data_field_event_performer_ref ON node_data_field_event_performer_ref.field_event_performer_ref_nid = node_node_data_field_event_performer_ref.nid  LEFT JOIN content_type_event node_data_field_event_date_utc ON node.vid = node_data_field_event_date_utc.vid  LEFT JOIN content_type_event node_data_field_event_announce_utc ON node.vid = node_data_field_event_announce_utc.vid  LEFT JOIN content_type_performer node_node_data_field_event_performer_ref_node_data_field_performer_thumb_url ON node_node_data_field_event_performer_ref.vid = node_node_data_field_event_performer_ref_node_data_field_performer_thumb_url.vid  LEFT JOIN content_type_venue node_node_data_field_event_venue_ref_node_data_field_venue_thumb_url ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref_node_data_field_venue_thumb_url.vid  LEFT JOIN location_instance node_node_data_field_event_venue_ref__location_instance ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref__location_instance.vid  LEFT JOIN location_instance node_node_data_field_event_venue_ref_node_node_data_field_event_venue_ref__location_instance ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref_node_node_data_field_event_venue_ref__location_instance.vid  LEFT JOIN location node_node_data_field_event_venue_ref__location ON node_node_data_field_event_venue_ref__location_instance.lid = node_node_data_field_event_venue_ref__location.lid   INNER JOIN node_access na ON na.nid = node.nid    WHERE (na.grant_view >= 1 AND ((na.gid = 0 AND na.realm = 'all') OR (na.gid = 0 AND na.realm = 'domain_site') OR (na.gid = 0 AND na.realm = 'domain_id') OR (na.gid = 0 AND na.realm = 'og_public'))) AND (  ((node.type in ('event')) AND (node.status = 1) AND (term_data_term_hierarchy.tid = 5 ))     AND ((DATE_FORMAT(STR_TO_DATE(node_data_field_event_date_utc.field_event_date_utc_value, '%Y-%m-%dT%T'), '%Y-%m-%d\T%H:%i') >= '2012-05-12T17:06') AND (DATE_FORMAT(node_data_field_event_announce_utc.field_event_announce_utc_value, '%Y-%m-%d\T%H:%i') <= '2012-05-12T19:06'))    );+----------+
| count(*) |
+----------+
|     2288 |
+----------+
1 row in set (51.39 sec)

Here is the updated query...

mysql> SELECT count(*)  FROM node node  LEFT JOIN term_node term_node ON node.vid = term_node.vid  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid  LEFT JOIN term_hierarchy term_hierarchy ON term_data.tid = term_hierarchy.tid  INNER JOIN term_data term_data_term_hierarchy ON term_hierarchy.parent = term_data_term_hierarchy.tid  LEFT JOIN content_type_event node_data_field_event_venue_ref ON node.vid = node_data_field_event_venue_ref.vid  INNER JOIN node node_node_data_field_event_venue_ref ON node_data_field_event_venue_ref.field_event_venue_ref_nid = node_node_data_field_event_venue_ref.nid  LEFT JOIN term_node term_node2 ON term_node2.vid = node.vid LEFT JOIN term_data term_data2 ON term_node2.tid = term_data2.tid LEFT JOIN content_type_event node_data_field_event_performer_ref ON node.vid = node_data_field_event_performer_ref.vid  LEFT JOIN node node_node_data_field_event_performer_ref ON node_data_field_event_performer_ref.field_event_performer_ref_nid = node_node_data_field_event_performer_ref.nid  LEFT JOIN content_type_event node_data_field_event_date_utc ON node.vid = node_data_field_event_date_utc.vid  LEFT JOIN content_type_event node_data_field_event_announce_utc ON node.vid = node_data_field_event_announce_utc.vid  LEFT JOIN content_type_performer node_node_data_field_event_performer_ref_node_data_field_performer_thumb_url ON node_node_data_field_event_performer_ref.vid = node_node_data_field_event_performer_ref_node_data_field_performer_thumb_url.vid  LEFT JOIN content_type_venue node_node_data_field_event_venue_ref_node_data_field_venue_thumb_url ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref_node_data_field_venue_thumb_url.vid  LEFT JOIN location_instance node_node_data_field_event_venue_ref__location_instance ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref__location_instance.vid  LEFT JOIN location_instance node_node_data_field_event_venue_ref_node_node_data_field_event_venue_ref__location_instance ON node_node_data_field_event_venue_ref.vid = node_node_data_field_event_venue_ref_node_node_data_field_event_venue_ref__location_instance.vid  LEFT JOIN location node_node_data_field_event_venue_ref__location ON node_node_data_field_event_venue_ref__location_instance.lid = node_node_data_field_event_venue_ref__location.lid   INNER JOIN node_access na ON na.nid = node.nid    WHERE (na.grant_view >= 1 AND ((na.gid = 0 AND na.realm = 'all') OR (na.gid = 0 AND na.realm = 'domain_site') OR (na.gid = 0 AND na.realm = 'domain_id') OR (na.gid = 0 AND na.realm = 'og_public'))) AND (  ((node.type in ('event')) AND (node.status = 1) AND (term_data_term_hierarchy.tid = 5 ))     AND ((DATE_FORMAT(STR_TO_DATE(node_data_field_event_date_utc.field_event_date_utc_value, '%Y-%m-%dT%T'), '%Y-%m-%d\T%H:%i') >= '2012-05-12T17:06') AND (DATE_FORMAT(node_data_field_event_announce_utc.field_event_announce_utc_value, '%Y-%m-%d\T%H:%i') <= '2012-05-12T19:06'))    ) AND term_data2.vid = 3;
+----------+
| count(*) |
+----------+
|     2288 |
+----------+
1 row in set (0.12 sec)

Please review the patch and let me know if any changes are needed. If this breaks for some cases, let me know and I'll work to resolve them. This is a major performance issue and is worth resolving. Thanks

d.clarke’s picture

Status: Closed (fixed) » Needs review

Forgot to update the status in previous comment. Please review.

mustanggb’s picture

Status: Needs review » Closed (outdated)