Slow loading module list page

Deprecated - Performance and scalability · Drupal 6.x

kilemol - December 3, 2008 - 09:36

Number one:

On the module list page count of SQL queries more than 1000.

In sources I find this:

  drupal_rebuild_theme_registry();
  node_types_rebuild();
  menu_rebuild();
  cache_clear_all('schema', 'cache');

I suggest to add check:

if ($form_state['post']) {
  drupal_rebuild_theme_registry();
  node_types_rebuild();
  menu_rebuild();
  cache_clear_all('schema', 'cache');
}

After this, queries count bring down to 100+

Number two:

file menu.inc

    foreach ($menu_links as $item) {
      $existing_item = db_fetch_array(db_query("SELECT mlid, menu_name, plid, customized, has_children, updated FROM {menu_links} WHERE link_path = '%s' AND module = '%s'", $item['link_path'], 'system'));
      if ($existing_item) {
        $item['mlid'] = $existing_item['mlid'];
        // A change in hook_menu may move the link to a different menu
        if (empty($item['menu_name']) || ($item['menu_name'] == $existing_item['menu_name'])) {
          $item['menu_name'] = $existing_item['menu_name'];
          $item['plid'] = $existing_item['plid'];
        }
        $item['has_children'] = $existing_item['has_children'];
        $item['updated'] = $existing_item['updated'];
      }

There we have db_fetch_array(db_query.. executed on each iteration of `foreach`. Why not make like this:

    $paths = db_query("SELECT mlid, menu_name, plid, customized, has_children, updated, link_path FROM {menu_links} WHERE link_path IN ('" . implode('\',\'',array_keys($menu_links)) . "') AND module = 'system'");
    while ( $item = db_fetch_array($paths) )
        $m_links[$item['link_path']] = $item;
 
    foreach ($menu_links as $item) {
      if ($m_links[$item['link_path']]) {
        $existing_item = $m_links[$item['link_path']];
        $item['mlid'] = $existing_item['mlid'];
...

I guess, my variant little faster, eh? (-:

» Login or register to post comments

Don't like hacks, but I like

kees@qrios - March 6, 2009 - 19:49

Don't like hacks, but I like this for development sites, this really speeds up loading the module page and caches are cleared/registries rebuilt anyway upon saving.
Good enough for me.

Related: http://drupal.org/node/311626

content management systemen, Drupal ontwikkeling

I have no idea what I just did to my system, but by making change #1 (in system.admin.inc), The modules page load time dropped from 112 seconds to 14 seconds. The number of queries dropped from about 4,000 to 179.

--
Tony Freixas
http://www.tigerheron.com/

First solution works! Cut it down 90%

lrobeson - April 13, 2009 - 15:53

I tried your first solution and my Module List page went from 4964 queries in 62830.76 milliseconds to 288 queries in 6566.13 milliseconds. Thank you! I'm new to Drupal so I'm not sure what that just did, but it worked perfectly!

_{Laura

[[http://twitter.com/robeson|Twitter]] - [[http://www.linkedin.com/in/robeson|LinkedIn]]}

I added it too and it seems to have had a postive effect

SocialNicheGuru - May 4, 2009 - 22:23

working on my site has been hard because of the slow speed. so i have been scavenging drupal.org for a way to make things quicker.

I deleted all my content and yet the site is still slow. I have lots of modules but on my dev system which is still faster.

I would start from my dev system except I don't know how to import my taxonomy terms with the exact same IDs so when I import from my production site everything will be in symc.

--
Bringing value to the social web by connecting people with events, products, and services that match their interests and values

Whats the difference between

kees@qrios - May 5, 2009 - 17:47

Whats the difference between "my site" and "my dev system"? Different OS? HW? SW? Drupal version? Different modules? config?

content management systemen, Drupal ontwikkeling

Slow loading module list page - hack #2

RedStaff - June 1, 2009 - 17:33

Can you please tell me what go where?

I tried to substitue the "foreach ($menu_links as $item)" block with

$paths = db_query("SELECT mlid, menu_name, plid, customized, has_children, updated, link_path FROM {menu_links} WHERE link_path IN ('" . implode('\',\'',array_keys($menu_links)) . "') AND module = 'system'");
while ( $item = db_fetch_array($paths) )
$m_links[$item['link_path']] = $item;

foreach ($menu_links as $item) {
if ($m_links[$item['link_path']]) {
$existing_item = $m_links[$item['link_path']];
$item['mlid'] = $existing_item['mlid'];

but I keep on getting parsing errors.

Thank you very much.

The instructions aren't very

espirates - July 7, 2009 - 16:32

The instructions aren't very clear, what goes where.

look in the system.admin.inc

irarab - July 19, 2009 - 01:33

for solution #1
look in the system.admin.inc file.

For Number One, are we

espirates - July 19, 2009 - 16:34

For Number One, are we suppose to replace the original code or add it underneath ?

For Number Two I get a blank page,

Does this

$existing_item = db_fetch_array(db_query("SELECT mlid, menu_name, plid, customized, has_children, updated FROM {menu_links} WHERE link_path = '%s' AND module = '%s'", $item['link_path'], 'system'));

get replaced with this

$paths = db_query("SELECT mlid, menu_name, plid, customized, has_children, updated, link_path FROM {menu_links} WHERE link_path IN ('" . implode('\',\'',array_keys($menu_links)) . "') AND module = 'system'");
    while ( $item = db_fetch_array($paths) )
        $m_links[$item['link_path']] = $item;

and then where does this code go ?

 foreach ($menu_links as $item) {
      if ($m_links[$item['link_path']]) {
        $existing_item = $m_links[$item['link_path']];
        $item['mlid'] = $existing_item['mlid'];

Hurt instead of helped

lestu - August 6, 2009 - 10:56

See here: http://drupal.org/node/534630#comment-1895096

You, Sir, are a GOD among men!!!

austintnacious - August 17, 2009 - 22:04

I can't contain myself! If you were here I'd kiss you man!

I'd started to HATE Drupal, both @ localhost and live.
Admin was a nightmare, especially admin/build/modules.

I feel like I got into Drupal at exactly the WRONG moment in time. . .
i.e the whole lifespan of D6.x (I got in @ 6.2, now at 6.13 within a year).

I only know enough to be dangerous but I'll say this "Something is seriously f'dup with D6.x. . ."

That is until your patches get implemented!
I went from admin/build/modules, update.php etc taking several minutes to now just seconds.

I combined the patch found here [http://drupal.org/node/193366#comment-1820128] with your patch.

I also implemented some other fixes I found including changing all tables from InnoDB to MyISMAM
(see http://drupal.org/node/311626#comment-1098842)

and making some other PHP and MySQL config changes that I found suggested here.

If I can replicate the performance improvement on the live server I'm going to nominate you for a Nobel Prize!!!

=-=

VM - August 17, 2009 - 22:15

The shame of this thread is that it wasn't created as a patch file and put in an issue for developers to look it over for it's inclusion into core.

When WILL this become a patch OR get rolled into D6.x

austintnacious - August 18, 2009 - 00:39

It's not as if there aren't enough releases of D6.x anyway.

This is such a BIG issue, I used to think it was just me or a module that I'd installed that was causing the problem.

If I asked anyone why it was happening they'd say, "wow, that's a lot of modules! Try dis-abling some!"

(Begs the question "isn't that the point? To get a standard set of functions you need dozens of modules and their dependencies! It doesn't take long to gather a LOT of modules! That's the baseline scenario!")

Unless these fixes raise serious security issues they should be put foward for inclusion in a D6.x release IMHO.

Thanks again!

I can actually take some pleasure in working with my site again. . .

=-=

VM - August 18, 2009 - 01:55

the codes here likely won't ever see the light of day in drupal because a patch wasn't created and an issue created for it for that core developers can review/audit.

When will it become a patch? I'd guess when someone in this thread takes the time to create one, and write up the issue explaining what the OP has said. That said, it would have to be applied to D7 first then backported to Drupal 6.x. It wouldn't be just thrown into D6 now that D7 is on the horizon. Any changes would have be included in both releases.

Whether this code is safe or not, or will cause other issues? I've no idea. Like any other code snippet found on d.o it should be used with caution.

yes, administer -> modules can be slow when many module are installed. This is because right now drupal scans every .info file to make sure it doesn't need to update. The same would happen @ administer -> themes if you upload 40 - 50 themes. Though this scenario is far less likely to happen as virtually noone uses that many themes on a single site.

the first solution work great

abou rokaya - September 11, 2009 - 17:21

the first solution work great for me...