TableSorts and PagerDefault queries broken

R.Muilwijk - August 31, 2008 - 13:42
Project:Drupal
Version:7.x-dev
Component:database system
Category:task
Priority:critical
Assigned:Unassigned
Status:closed
Issue tags:DBTNG Conversion, dc2009 code sprint
Description

I've started working on this and are at like 80% now. I will finish this somewhere this week.

#1

Crell - November 28, 2008 - 06:57

It's somewhat more than a week... :-) Are you still working on this, or should someone else jump in?

#2

R.Muilwijk - November 28, 2008 - 08:27

Sorry, I'll put in on top of my list for next thursday. We have open source thursday at our company so should have plenty of time then.

#3

Crell - March 7, 2009 - 16:41
Assigned to:R.Muilwijk» Anonymous

This is open for anyone who wants it.

#4

Dave Reid - March 7, 2009 - 16:42
Title:DBTNG: dblog» DBTNG dblog.module
Component:database system» dblog.module

#5

tizzo - March 7, 2009 - 17:40
Assigned to:Anonymous» tizzo

Working on this now!

#6

Ryan Palmer - March 7, 2009 - 17:50

Also working on this atm. Can somebody give me a quick sanity check to see if I'm doing this right? tx!

AttachmentSizeStatusTest resultOperations
dblog.302268.6.patch2.12 KBIdleFailed: 10444 passes, 1 fail, 4 exceptionsView details

#7

Dave Reid - March 7, 2009 - 17:53
Status:active» needs work

$max = db_query('SELECT MAX(wid) FROM {watchdog}')->fetchObject();
should use ->fetchResult();

#8

Ryan Palmer - March 7, 2009 - 18:03

Noticed that just after I submitted it... otherwise ok? (rerolled)

AttachmentSizeStatusTest resultOperations
dblog.302268.8.patch2.14 KBIdleFailed: Failed to install HEAD.View details

#9

Ryan Palmer - March 7, 2009 - 18:04
Status:needs work» needs review

#10

Dave Reid - March 7, 2009 - 18:07

foreach ($results AS $object) {
should just be lowercase 'as' :)

Other than that, it looks ok. I don't have anything to test that it actually works, but I guess the testbot can help us there.

#11

System Message - March 7, 2009 - 18:20
Status:needs review» needs work

The last submitted patch failed testing.

#12

Ryan Palmer - March 7, 2009 - 18:26
Status:needs work» needs review

Oh really! I've been doing it wrong all this time!

Thanks for the help.

AttachmentSizeStatusTest resultOperations
dblog.302268.11.patch2.14 KBIdleFailed: Failed to install HEAD.View details

#13

Ryan Palmer - March 7, 2009 - 18:35
Assigned to:tizzo» Ryan Palmer

Looks like it failed.. not sure why. Try this once more.

AttachmentSizeStatusTest resultOperations
dblog.302268.13.patch2.09 KBIdleFailed: Failed to install HEAD.View details

#14

tizzo - March 7, 2009 - 18:46
Assigned to:Ryan Palmer» Anonymous

[Edit] Sorry, hadn't refreshed the page in a while. I think my patch applies and all tests pass. Big frown on duplicate work but I guess we'll all get a little more experience with DBTNG!

[Edit Edit] Durp! Also I missed the admin.inc, oh well! Hopefully your patch works!

AttachmentSizeStatusTest resultOperations
dblog_dbtng.patch1.69 KBIdleFailed: 10425 passes, 1 fail, 0 exceptionsView details

#15

System Message - March 7, 2009 - 19:00
Status:needs review» needs work

The last submitted patch failed testing.

#16

tizzo - March 7, 2009 - 19:15
Status:needs work» needs review

Fixed up a few other things in admin.inc, I do think everything is done and working now. All pretty straightforward stuff, switched all of the while loops to foreach.

[Edit - I found a problem with my setup that caused that fail, I also realized that none of us have updated the test. Working on that now]

AttachmentSizeStatusTest resultOperations
dblog_dbtng-2.patch3.37 KBIdleFailed: 10425 passes, 1 fail, 0 exceptionsView details

#17

System Message - March 7, 2009 - 19:25
Status:needs review» needs work

The last submitted patch failed testing.

#18

tizzo - March 7, 2009 - 19:54
Status:needs work» needs review

Ok, I think I have this working. Really this time.

Went through the test file and updated that as well as fixing my previous issue (and typos that accompanied it).

AttachmentSizeStatusTest resultOperations
dblog_dbtng-3.patch7.64 KBIdleFailed: Failed to apply patch.View details

#19

Crell - March 7, 2009 - 21:45
Status:needs review» needs work

<?php
+    $result = db_query('SELECT wid FROM {watchdog} WHERE uid = :uid', array(':uid' => $user->uid));
+    foreach(
$result as $row) {
+     
$ids[] = $row->wid;
?>

The foreach block can be replaced with $ides = db_query()->fetchCol(); I believe that appears twice.

<?php
   $result
= db_query('SELECT DISTINCT(type) FROM {watchdog} ORDER BY type');
-  while (
$object = db_fetch_object($result)) {
+  foreach (
$result as $object) {
    
$types[] = $object->type;
   }
?>

This can also be condensed into just fetchCol().

And of course there's still a pager query.

#20

bubbasan - March 7, 2009 - 22:34
Title:DBTNG dblog.module» DBTNG dblog.admin.inc
Status:needs work» needs review

Here is the dblog.admin.inc Still having issues with setHeader and setCountQuery, but simpletest seems to pass.

AttachmentSizeStatusTest resultOperations
dblog.admin_.inc_.dbtng_.patch3.09 KBIdleFailed: Failed to apply patch.View details

#21

tizzo - March 7, 2009 - 22:44
Assigned to:Anonymous» tizzo
Status:needs review» needs work

I'll roll this patch into the one I'm repairing as per Crell's comments above.

#22

tizzo - March 12, 2009 - 21:25
Assigned to:tizzo» Anonymous
Status:needs work» needs review

Sorry for the delay! Patch re-rolled with bubbasan's patch and Crell's comments above implemented. Hopefully this works!

AttachmentSizeStatusTest resultOperations
dblog_dbtng-4.patch10.78 KBIdleFailed: Failed to apply patch.View details

#23

tizzo - March 12, 2009 - 21:37

Rolled a cleaner patch, not sure if the last one would have worked or not but I think this one will.

AttachmentSizeStatusTest resultOperations
dblog_dbtng-5.patch10.9 KBIdleFailed: Failed to apply patch.View details

#24

tizzo - March 13, 2009 - 13:25
Title:DBTNG dblog.admin.inc» DBTNG dblog

Changing title to reflect that this patch is for the whole dblog module and not just dblog.admin.inc

#25

Dries - March 13, 2009 - 14:28
Status:needs review» fixed

Committed to CVS HEAD. Thanks!

#26

Dave Reid - March 17, 2009 - 18:00
Priority:normal» critical
Status:fixed» active

The recent log events tablesort is now completely broken and sorting by filter does not work as well. I'm looking into fixing it.

#27

Dave Reid - March 17, 2009 - 18:15
Status:active» needs work

This at least gets the table sort working. Filtering is still broken completely.

AttachmentSizeStatusTest resultOperations
302268-dblog-dbtng-critical-D7.patch2.59 KBIdleFailed: Failed to apply patch.View details

#28

Dave Reid - March 17, 2009 - 18:23

Note to anyone doing DBTNG conversion. To convert table sorts, you must do:

<?php
...
$query = $query->extend('PagerDefault')->limit(your_limit);
$query = $query->extend('TableSort')->setHeader($header);
...
?>

These have to be separate lines and cannot be chained. This happened in dblog, poll, and comment.

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7.patch5.19 KBIdleFailed: 10551 passes, 7 fails, 6 exceptionsView details

#29

Dave Reid - March 17, 2009 - 18:31
Status:needs work» active

Left in an unwanted debugging statement.

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7.patch5.17 KBIdleFailed: Failed to apply patch.View details

#30

Dave Reid - March 17, 2009 - 19:18
Status:active» needs review

#31

Crell - March 17, 2009 - 20:24

That seems wrong then. I know I ran into a case where ordered mattered, but you should be able to extend in a chain as long as you do assign back to $query at the end.

#32

Dave Reid - March 17, 2009 - 20:39

That seems like it would work. Comparing:

<?php
   $query
= db_select('watchdog', 'w');
  
$query = $query
    
->addExpression('COUNT(wid)', 'count');
     ->
fields('w', array('message', 'variables'))
     ->
condition('w.type', $type)
     ->
groupBy('message')
     ->
groupBy('variables')
     ->
extend('PagerDefault')
     ->
extend('TableSort')
     ->
limit(30)
     ->
setHeader($header)
     ->
setCountQuery($count_query);
  
$result = $query->execute();
?>

to:

<?php
   $query
= db_select('watchdog', 'w');
  
$query->addExpression('COUNT(wid)', 'count');
  
$query->fields('w', array('message', 'variables'));
  
$query->condition('w.type', $type);
  
$query->groupBy('message');
  
$query->groupBy('variables');
  
$query = $query->extend('PagerDefault')->limit(30);
  
$query = $query->extend('TableSort')->setHeader($header);
  
$query->setCountQuery($count_query);
  
$result = $query->execute();
?>

The latter is easier for me to see what is going on. I also remember something about some parts of db_select not able to be chained (fields and conditions?), but I guess it depends on what we're deciding is our offical DBTNG syntax.

#33

Crell - March 17, 2009 - 22:42

I believe the standard is "chain whenever possible", because it's easier to read overall. addField() and addExpression() are not chainable, neither are the join*() methods. extend() ischainable, however, the returned object is the extender, not the original query, so if you don't call ->execute() in that chain then you have to assign back to $query, otherwise the extending object gets lost.

#34

Dave Reid - April 6, 2009 - 18:47
Title:DBTNG dblog» TableSorts and PagerDefault queries broken
Component:dblog.module» database system

#35

Berdir - April 25, 2009 - 13:10
Status:needs review» needs work

Same here,

Regarding coding style..

After talking with Crell/webchick, I try to follow the following rules when working with DBTNG:

- The docs at http://drupal.org/node/310069 should be used as examples for coding style
- Chain methods if possible
- If chaining is used, all methods should be on their own line
- Extend early (see #425872: DX problem with DB extenders)

I know that some disagree with point 2 for example, but we should really agree on something, whatever it is..

#36

Berdir - May 11, 2009 - 20:49
Status:needs work» needs review

Re-roll.

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7_1.patch5.4 KBIdleFailed: Failed to apply patch.View details

#37

System Message - May 21, 2009 - 14:05
Status:needs review» needs work

The last submitted patch failed testing.

#38

Berdir - May 28, 2009 - 21:15
Status:needs work» needs review

Re-rolled.

Also cleaned up a few queries that didn't early extend, even if they were working correctly.

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7_2.patch6.77 KBIdleFailed: Failed to apply patch.View details

#39

System Message - June 2, 2009 - 23:50
Status:needs review» needs work

The last submitted patch failed testing.

#40

Berdir - June 3, 2009 - 09:29
Status:needs work» needs review

Re-roll.

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7_3.patch4.91 KBIdleFailed: 11785 passes, 23 fails, 6 exceptionsView details

#41

System Message - June 3, 2009 - 19:46
Status:needs review» needs work

The last submitted patch failed testing.

#42

Berdir - June 3, 2009 - 20:33
Status:needs work» needs review

Re-roll, forgot to change a setHeader() call...

AttachmentSizeStatusTest resultOperations
302268-dbtng-tablesorts-critical-D7_4.patch4.91 KBIdlePassed: 11881 passes, 0 fails, 0 exceptionsView details

#43

Dries - June 5, 2009 - 16:20

Should we write tests for this?

#44

Berdir - June 6, 2009 - 07:36

We have generic paging tests and we have tests for the functionality of those pages. Moshe was against writing paging tests for every page where we used it, see #394582-8: DBTNG: Tracker module.

I'd say that with the new early extend CS, it is unlikely that it is used wrong and usually, we don't protect against wrong usage of our API.

#45

Dries - June 6, 2009 - 10:27

Fair enough. Committed to CVS HEAD.

#46

Dries - June 6, 2009 - 10:33
Status:needs review» fixed

#47

System Message - June 20, 2009 - 10:40
Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.

 
 

Drupal is a registered trademark of Dries Buytaert.