New pager code broken taxonomy pagers

Why can't this code use pager_query() and query the database directly? If free tagging vocabularies can get so big, it seems a very inefficient thing to fetch the entire list.

Note: if you need to fetch a list of tags ordered by hierarchy in one query, you should look at how comment.module does it.

Maybe it's crazy, but it has a single, pagable query and an unambigous sorting both ways. It works well for what it needs to do.

In fact, the only problem is the assignment of numbers because we cannot do a "natural order" sort in SQL. Right now it uses a rather inefficient coding that scales O(n) with size and adds another digit every 10 values.

This prompted me to experiment with better codings. First I generalized the current scheme a bit, then tweaked it to get near O(log n) scaling and on average only 20-30% more digits than an equivalent decimal notation, while still retaining the string sorting property. Quite fun actually.

I don't see the problem with the approach though... all it does is store a compact ordering key per row. Compare this to the convoluted tree logic in taxonomy_get_tree() and I know which one I prefer.

Of course, I'm not at all sure if the comment.module approach can be implemented here, but I can't believe you'd do something like this after talking about mega-taxonomies for so long. Loading in 8000 tags just to show 50 is nowhere near optimal.

Morbus: the funky comment thread stuff rocks. We should do the same for book pages and taxonomy terms if it can be done effectively. This doens't mean we should remove the traditional pid-columns.

needs work according to people who matter.

Moving to 6.x-dev queue.

#292073: Long hierarchies not displayed on Taxonomy admin page is a duplicate, but it's having some patch:

Index: taxonomy.admin.inc
===================================================================
--- taxonomy.admin.inc  (drupal 6.3)
+++ taxonomy.admin.inc  (patched)
@@ -299,11 +299,13 @@
       // Count entries before the current page.
       if ($page && ($page * $page_increment) > $before_entries && !isset($back_peddle)) {
         $before_entries++;
+        $term = next($tree);
         continue;
       }
       // Count entries after the current page.
       elseif ($page_entries > $page_increment && isset($complete_tree)) {
         $after_entries++;
+        $term = next($tree);
         continue;
       }

@@ -314,18 +316,19 @@
           $before_entries--;
           $back_peddle++;
           if ($pterm->depth == 0) {
-            prev($tree);
+            $term = $pterm;
             continue 2; // Jump back to the start of the root level parent.
           }
         }
       }
       $back_peddle = isset($back_peddle) ? $back_peddle : 0;
-
+
       // Continue rendering the tree until we reach the a new root item.
       if ($page_entries >= $page_increment + $back_peddle + 1 && $term->depth == 0 && $root_entries > 1) {
         $complete_tree = TRUE;
         // This new item at the root level is the first item on the next page.
         $after_entries++;
+        $term = next($tree);
         continue;
       }
       if ($page_entries >= $page_increment + $back_peddle) {
@@ -346,7 +349,9 @@
         $root_entries++;
       }
       $current_page[$key] = $term;
-    } while ($term = next($tree));
+
+      $term = next($tree);
+    } while ($term);

     // Because we didn't use a pager query, set the necessary pager variables.
     $total_entries = $before_entries + $page_entries + $after_entries;

Hi bug fixers :)

I'm not entirely sure that #292073 should have been made a duplicate of this issue, as it's for a very specific problem ; it details what the symptoms are and how to re-create it. Instead this issue here seems to be an open-ended "things are broken" issue.

Anyway, just in case people are interested, here is the symptoms/how to recreate comment that go with #292073 (ie. the patch posted in #10)

Symptoms :

I have a very long vocabulary with hierarchies (for regions/countries). When displaying in the admin page (admin/content/taxonomy/xxx) only the first page shows any items - subsequent pages (eg. admin/content/taxonomy/xxx?page=2) show no items (even though there should be some).

Supposition :

I have looked at the code, and from what I can tell this will happen if :

1. The very first item is a root of a hierarchy
2. That hierarchy has more than one page's worth of elements

That's it :) You can view the full report (+some comment about how the patch works) on http://drupal.org/node/292073

Does this issue exist in current D6?

Still an issue?

this is still happening .. yes.
there is a helper module, (waaay more performant)
which actually could be integrated into core .. ??

see here too #292073: Long hierarchies not displayed on Taxonomy admin page

The taxonomy page is a view now, which works.