Comments

Anonymous’s picture

Status: Needs review » Needs work

Indentation issues following arrays.

hswong3i’s picture

Title: [TNGDB]: simplify menu.inc with db_insert and db_update » [DBTNG]: simplify menu.inc with db_insert and db_update
Status: Needs work » Needs review

@earnie: I think this is our new coding standard? E.g. includes/actions.inc, line 329:

  db_merge('actions')
    ->key(array('aid' => $aid))
    ->fields(array(
      'callback' => $function,
      'type' => $type,
      'parameters' => serialize($params),
      'description' => $desc,
    ))
    ->execute();
Anonymous’s picture

Right, but look at your patch in the browser. Some of the lines after closing )) for the fields(array( are not indented correctly.

hswong3i’s picture

StatusFileSize
new7.83 KB

@earnie: Yes, get it and fixed :D

hswong3i’s picture

StatusFileSize
new7.79 KB

Oooh... Some don't need to be too fancy :S

hswong3i’s picture

StatusFileSize
new7.92 KB

Patch reroll via CVS HEAD.

Crell’s picture

Subscribing...

Crell’s picture

Status: Needs review » Needs work

And the test server doesn't like this.

hswong3i’s picture

Status: Needs work » Needs review
StatusFileSize
new7.92 KB

Well... Its strange. I fetch code from CVS HEAD, apply patch, run installer with expert mode, activate simpletest, run simpletest for "menu", and all pass and looks good :-S

Anyway, re-submit the patch and hopefully the test server would like it :-)

Crell’s picture

Status: Needs review » Needs work

From visual inspection:

Do we really have to duplicate the array in the query? It looks like the array that is already built is what should be passed into db_update() and db_insert() anyway.

The menu rebuild db_insert() is exactly the sort of place we want to be using a multi-insert statement. That should give us a nice performance bump.

Let's go ahead and do a full menu.inc DBTNG conversion as long as we're here.

hswong3i’s picture

Status: Needs work » Needs review
StatusFileSize
new8.6 KB

Hmm... I try to revamp with using $item directly, but failed. I guess it is because $item contain some other required keys. db_insert() and db_update() will not reference to database schema so it will build the RAW SQL with unexpected result. In order to have blind-insert or blind-update, we should use drupal_write_record() instead.

BTW, as drupal_write_record() is now implement within common.inc so sometime it may still not yet include during early stage of bootscript. Moreover, drupal_write_record() coming with additional abstraction which must result with performance degrade. If we hope to boost up our libraries as much as possible, using db_insert() and db_update() seems to be a better choice.

P.S. Patch revamp with multi-insert, and I can FEEL the different during simpletest! Amazing! Maybe we can first pass this issue and follow up the others later?

dries’s picture

The fact that you can 'feel' the difference is interesting and most promising.

We should make Simpletest report its total running time in the test summary. We'd be able to quantify what you feel and we'd be able to figure out who is using the fastest machine ... I created an issue for this: #329407: Make simpletest report running time

dries’s picture

I did some tests and I couldn't measure a significant performance impact when running the tests. Without this patch, running all tests took 21 minutes 18 seconds. With your patch it took 21 minutes 30 seconds.

Crell’s picture

I agree with sticking to db_insert()/db_update() here. In fact I'd like to go as far as eliminating drupal_write_record() entirely if possible. However, can we just unset the unneeded keys from the $items array and then pass it to insert?

Also, the fields() definition on the insert should just put one field per line. It's more lines that way, but is visually cleaner and more consistent with the rest of core, I think. (Dries & Angie can overrule me if they want there. I can see the argument to keep it as is.)

hswong3i’s picture

StatusFileSize
new9.14 KB

Oh #329407: Make simpletest report running time is a good idea. Feeling is not accurate enough...

...can we just unset the unneeded keys from the $items array and then pass it to insert?

I give a try for this approach, but found that lines for explicitly unset unneeded keys is almost equal as that for explicitly use required keys. Moreover, those "unneeded keys" for db_insert() or db_update() maybe required for on going procedure, unset them may trigger other hidden issues. Finally, explicitly define keys seems much accurate and clear for debug and maintenance. (Please correct me if possible.)

Patch reroll based on above indent suggestion. Tested with MySQL simpletest and it looks fine.

hswong3i’s picture

StatusFileSize
new11.43 KB

Keep on update with DBTNG syntax. Just half-done. Keep as record.

dries’s picture

Status: Needs review » Needs work

If this patch is not complete yet, please mark it 'CNW'. Thanks, and keep up the good work, hswong3i.

hswong3i’s picture

Title: [DBTNG]: simplify menu.inc with db_insert and db_update » [DBTNG] revamp menu.inc with DBTNG syntax
Status: Needs work » Needs review
StatusFileSize
new31.29 KB

Latest progress. I try my best (simpletest during development for 100+ times...), and most queries are now in DBTNG syntax. Some missing:

  1. Some while ($item= db_fetch_array($result)) { is not able to revamp as foreach ($result as $item) { style. Once change it error is triggered. I don't know why this happened.
  2. I don't know how to revamp query with db_rewrite_sql().

Tested with MySQL + simpletest and pass all menu test. I need some help for the rest of revamp :S

dries’s picture

At first glance this looks good. Would be great to get Crell's approval. :-)

hswong3i’s picture

StatusFileSize
new33.44 KB

I add the following comment for those incomplete, so simplify tracing:

// @todo: Revamp with DBTNG syntax.

Also minor fix some other syntax and coding style.

hswong3i’s picture

StatusFileSize
new34.16 KB

Patch reroll based on code cleanup. Also integrate with progress of #328110: Recoverable fatal error: Argument 3 passed to l() must be an array.

Patch via CVS HEAD, tested with MySQL + menu simpletest. All pass without error message.

hswong3i’s picture

StatusFileSize
new22.75 KB

According to commit of #328110: Recoverable fatal error: Argument 3 passed to l() must be an array, patch reroll, update, and focus on menu.inc only.

Patch via CVS HEAD, tested with MySQL + menu simpletest. All pass without error message.

Anonymous’s picture

Status: Needs review » Needs work

The last submitted patch failed testing.

hswong3i’s picture

StatusFileSize
new33.07 KB

Ooops... "Module list functionality" is failed. It is because passing $item as object into _menu_delete_item(), but we use it as array. Quick fix with following code snippet:

@@ -1852,17 +1968,19 @@ function menu_link_delete($mlid, $path =
  *   Forces deletion. Internal use only, setting to TRUE is discouraged.
  */
 function _menu_delete_item($item, $force = FALSE) {
+  $item = (array) $item;
   if ($item && ($item['module'] != 'system' || $item['updated'] || $force)) {
     // Children get re-attached to the item's parent.
     if ($item['has_children']) {

Also handle some more revamp from while (...db_fetch_array()) to foreach (...).

hswong3i’s picture

Status: Needs work » Needs review
Anonymous’s picture

Status: Needs review » Needs work

Object to array conversion should happen with http://php.net/get_object_vars instead.

function _menu_delete_item($item, $force = FALSE) {
  if (is_object($item)) {
    $item = get_object_vars($item);
  }
hswong3i’s picture

Status: Needs work » Needs review
StatusFileSize
new5.26 KB
new33.46 KB

Reference to #302207: DBTNG: book module, also handle db_query_rewrite() and some other minor stuff. Update "object to array" with get_object_vars(). Please refer to diff file for more information.

Status: Needs review » Needs work

The last submitted patch failed testing.

Crell’s picture

Status: Needs work » Needs review

Probably mis-marked as CNW by the recent bot bug.

hswong3i’s picture

StatusFileSize
new3.28 KB
new34.07 KB

Crell: Not only about bot bug. forum.test coming with illegal (!?) statement, so the test fail. Patch updated, please refer to diff file for more information.

hswong3i’s picture

StatusFileSize
new34.1 KB

Reroll via CVS HEAD.

lilou’s picture

You can replace (twice) :

+        for ($i = 1; $i <= 9; $i++) {
+          $query->orderBy('p' . $i, 'ASC');
+        }

with :

+        for ($i = 1; $i <= MENU_MAX_DEPTH; $i++) {
+          $query->orderBy('p' . $i, 'ASC');
+        }
hswong3i’s picture

StatusFileSize
new303.33 KB

Yes, this looks more elegant :D

lilou’s picture

Status: Needs review » Needs work

Wrong patch :-(

hswong3i’s picture

Status: Needs work » Needs review
StatusFileSize
new34.12 KB

Ooops... How a silly mistake... Patch reroll :(

Status: Needs review » Needs work

The last submitted patch failed testing.

hswong3i’s picture

Status: Needs work » Needs review
StatusFileSize
new34.12 KB

I double check with CVS HEAD and my testbed, simpletest should pass. Let's reroll once again.

dries’s picture

When I apply this patch, I get a WSOD (fatal error).

hswong3i’s picture

It shouldn't be... I test it with PHP5.2 CVS HEAD, and all simpletest pass. Or any Apache error log message as reference?

dries’s picture

Status: Needs review » Fixed

After another review, I decided to commit this patch. If necessary, we can follow-up with new patches. Thanks for your hard work hswong3i, and thanks to the many reviewers along the way. Committed to CVS HEAD.

dave reid’s picture

I've got the following error now trying to install sqlite HEAD:

PDOException: UPDATE {menu_links} SET menu_name=:db_update_placeholder_0, plid=:db_update_placeholder_1, link_path=:db_update_placeholder_2, router_path=:db_update_placeholder_3, hidden=:db_update_placeholder_4, external=:db_update_placeholder_5, has_children=:db_update_placeholder_6, expanded=:db_update_placeholder_7, weight=:db_update_placeholder_8, depth=:db_update_placeholder_9, p1=:db_update_placeholder_10, p2=:db_update_placeholder_11, p3=:db_update_placeholder_12, p4=:db_update_placeholder_13, p5=:db_update_placeholder_14, p6=:db_update_placeholder_15, p7=:db_update_placeholder_16, p8=:db_update_placeholder_17, p9=:db_update_placeholder_18, module=:db_update_placeholder_19, link_title=:db_update_placeholder_20, options=:db_update_placeholder_21, customized=:db_update_placeholder_22 WHERE (mlid = :db_condition_placeholder_142) AND ( (menu_name <> :db_condition_placeholder_119) OR (menu_name IS NULL) OR (plid <> :db_condition_placeholder_120) OR (plid IS NULL) OR (link_path <> :db_condition_placeholder_121) OR (link_path IS NULL) OR (router_path <> :db_condition_placeholder_122) OR (router_path IS NULL) OR (hidden <> :db_condition_placeholder_123) OR (hidden IS NULL) OR (external <> :db_condition_placeholder_124) OR (external IS NULL) OR (has_children <> :db_condition_placeholder_125) OR (has_children IS NULL) OR (expanded <> :db_condition_placeholder_126) OR (expanded IS NULL) OR (weight <> :db_condition_placeholder_127) OR (weight IS NULL) OR (depth <> :db_condition_placeholder_128) OR (depth IS NULL) OR (p1 <> :db_condition_placeholder_129) OR (p1 IS NULL) OR (p2 <> :db_condition_placeholder_130) OR (p2 IS NULL) OR (p3 <> :db_condition_placeholder_131) OR (p3 IS NULL) OR (p4 <> :db_condition_placeholder_132) OR (p4 IS NULL) OR (p5 <> :db_condition_placeholder_133) OR (p5 IS NULL) OR (p6 <> :db_condition_placeholder_134) OR (p6 IS NULL) OR (p7 <> :db_condition_placeholder_135) OR (p7 IS NULL) OR (p8 <> :db_condition_placeholder_136) OR (p8 IS NULL) OR (p9 <> :db_condition_placeholder_137) OR (p9 IS NULL) OR (module <> :db_condition_placeholder_138) OR (module IS NULL) OR (link_title <> :db_condition_placeholder_139) OR (link_title IS NULL) OR (options <> :db_condition_placeholder_140) OR (options IS NULL) OR (customized <> :db_condition_placeholder_141) OR (customized IS NULL) ) - Array ( [:db_update_placeholder_0] => navigation [:db_update_placeholder_1] => 0 [:db_update_placeholder_2] => batch [:db_update_placeholder_3] => batch [:db_update_placeholder_4] => -1 [:db_update_placeholder_5] => 0 [:db_update_placeholder_6] => 0 [:db_update_placeholder_7] => 0 [:db_update_placeholder_8] => 0 [:db_update_placeholder_9] => 1 [:db_update_placeholder_10] => 1 [:db_update_placeholder_11] => 0 [:db_update_placeholder_12] => 0 [:db_update_placeholder_13] => 0 [:db_update_placeholder_14] => 0 [:db_update_placeholder_15] => 0 [:db_update_placeholder_16] => 0 [:db_update_placeholder_17] => 0 [:db_update_placeholder_18] => 0 [:db_update_placeholder_19] => system [:db_update_placeholder_20] => [:db_update_placeholder_21] => a:0:{} [:db_update_placeholder_22] => 0 [:db_condition_placeholder_142] => 1 [:db_condition_placeholder_119] => navigation [:db_condition_placeholder_120] => 0 [:db_condition_placeholder_121] => batch [:db_condition_placeholder_122] => batch [:db_condition_placeholder_123] => -1 [:db_condition_placeholder_124] => 0 [:db_condition_placeholder_125] => 0 [:db_condition_placeholder_126] => 0 [:db_condition_placeholder_127] => 0 [:db_condition_placeholder_128] => 1 [:db_condition_placeholder_129] => 1 [:db_condition_placeholder_130] => 0 [:db_condition_placeholder_131] => 0 [:db_condition_placeholder_132] => 0 [:db_condition_placeholder_133] => 0 [:db_condition_placeholder_134] => 0 [:db_condition_placeholder_135] => 0 [:db_condition_placeholder_136] => 0 [:db_condition_placeholder_137] => 0 [:db_condition_placeholder_138] => system [:db_condition_placeholder_139] => [:db_condition_placeholder_140] => a:0:{} [:db_condition_placeholder_141] => 0 ) SQLSTATE[HY000]: General error: 25 bind or column index out of range in menu_link_save() (line 2168 of /home/davereid/Projects/drupal-head/includes/menu.inc).

dave reid’s picture

Priority: Normal » Critical
Status: Fixed » Active

Rolling back this patch allows HEAD to be installed on sqlite. Marking as critical since this patch broke HEAD on sqlite.

dave reid’s picture

Status: Active » Fixed

Followup issue for broken sqlite HEAD in #342725: Broken HEAD - Super long update queries.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

Crell’s picture

Issue tags: +DBTNG Conversion

Tagging.