First thing I wan't to note is that this is my first Drupal patch, so hopefully I won't get flamed directly rather then constructive improvements to my patch.

One of the mayor performance trouble I've seen happening (mostly DR5 sites) are performance drop-downs in circumstances where the same users are loaded twice on the same request, a particular example for this problem are forum topics where 1 user posts several comments - which leads in that user being loaded over and over again.

I took a quite radical approach with my patch by trying to cache as much as possible within the user_load() function while still respecting the $array (which I renamed to $conditions) arguments array, which may not always contain a uid key.

First, the new API header:

/**
 * Fetch a user object.
 *
 * @param $conditions
 *   An associative array of attributes to search for in selecting the
 *   user, such as user name or e-mail address.
 * @param $reset
 *   Resets the internal cache for the first user object that is saved
 *   in the cache that matches the $conditions, or when $conditions is
 *   an empty array it resets the whole internal cache
 * @return
 *   A fully-loaded $user object upon successful user load or FALSE if user
 *   cannot be loaded.
 */

As you can see there are no big changes API wise, except the new reset parameter and the caching behavior. Doing a user_load(2) 5 times will still result in 1 query, except when using the $reset parameter (which will delete only uid2 from the cache). The function changed drastically:

  • The quite confusing $array got renamed to $conditions
  • A new parameter $reset got added, it works twofold. When $conditions is a empty array it deletes the complete node_load() cache, else it only deletes the first matching userobject
  • When a user object is queried using only a uid it tries to look up the user object in the cache directly, without checking the $cache['conditions'] cache.
  • When other array keys are present in $conditions it will look up and eliminate uid's from a second cache: $cache['conditions'], from which it tries to select the first matching user object to return it.
  • If nothing matches the caching mechanism it returns to it's old user_load() behavior. It queries the data and saves both the user object and the conditions to the cache on result

Besides the direct work into the user_load() function I've tried to fix up the other related user functions, like user_save() to reset a user within the cache of user_load() when this is necessary. Besides that and two changes in the unittest everything runs fine, and all Simpletests are on green.

Personally I think the biggest risk of this patch is other modules that assume certain behavior like the following example:

function foobar($account) {
  $refresh_user = user_load(array('uid' => $account->uid));
  
  //instead of adding the reset parameter to get it truly reloaded:
  $refresh_user = user_load(array('uid' => $account->uid), true);
}

Comments

catch’s picture

Status: Needs review » Needs work

Quick visual review.

We don't normally use count - empty and !empty are faster. Also there's some code style issues - comments should be: // Sentence with a full stop at the end. - false should be FALSE (or maybe a constant following recent discussions in other issues) - general guidelines at http://drupal.org/coding-standards

nielsvm’s picture

Catch,

You're right, empty is faster, though at at least 1 statement I do need count because cases with more arguments in $conditions can make it a whole different situation. I will improve it and adhere to the codestyle guidelines and replay the patch this week.

Thanks

catch’s picture

A couple of things I missed,

This issue is actually a duplicate of http://drupal.org/node/91786 - which has a fairly recent patch on it. This should be merged back into that one ideally.

Also - running all tests, this appears to introduce no regressions. At first sight, pwolanin's patch at 91786 looks more concise, but I've not done any real comparison of the approaches at all - and the static cache facility needs to be considered in relation to both.

damien tournoud’s picture

@catch: the difference between the two is that pwolanin's patch only cache simple uid user_load, while this one attempts to solve the general problem.

If it was me, I would have done:

ksort($array);
$key = md5(serialize($array));
if (isset($cache[$key])) {
 return $cache[$key];
}

But this is probably awful performance-wise.

damien tournoud’s picture

Well, you can even drop the md5().

nielsvm’s picture

I've considered the serialize approach at first, but there are two mayor drawbacks:

1. Performance, I'm afraid that hashing serialized array's is way too much CPU bloat.

2. Preciseness. When having very precise conditions in $conditions it will return the user from the cache the next time when asking the exact same conditions, fine. But when you ask for a user with fewer conditions, but all that are in the cache, the serialize approach will still result in a query as the hash doesn't match.

Niels

nielsvm’s picture

Status: Needs work » Needs review
StatusFileSize
new7.16 KB

Updated the patch and tested lots of different cases by clicking in the user admin pages, also improved the code style. Have tested the patch in a production environment (Drupal 5) for two months now, without any trouble so far.

Please review the code, as I hope to get this patch in before code freeze. It would boost Drupal's performance a lot (especially in the forums).

Perhaps a test-request for testing user_load() can be added to the testing party?

TIA folks!

keith.smith’s picture

Status: Needs review » Needs work

On a quick glance through, there's a good number of code style issues with the commenting (comments should end with periods; in-line comments should be set off the code with spaces, etc.)

nielsvm’s picture

Status: Needs work » Needs review
StatusFileSize
new7.15 KB

Indeed, comments weren't proper according to the code style. Also removed trailing whitespace at some places.

nielsvm’s picture

StatusFileSize
new11.47 KB

Updated the patch merely by adding a test suite to user.test for the loading part. For now it simply tests the static cache in user_load(), simple loading and refreshing the static cache.

nielsvm’s picture

StatusFileSize
new11.47 KB

For some strange reason I named the patch wrong, confused with node_load(). Renamed the patch to user_load_static_cache_xx.patch.

Anonymous’s picture

Status: Needs review » Needs work

not a thorough review, just nitpicking. i'll follow up with more when i get a chance to benchmark this.

* the space after $reset needs to go

+ if (!$cache || (empty($conditions) && $reset) ) {

* there's probably a better way to say this than 'jack out' ;-)

+ // Make sure $conditions is an array at all times or jack out.

* spaces before and after '=='

+ if (isset($conditions['uid']) && (count($conditions)==1) && !$reset) {

* IMO this

+    if (isset($cache['conditions'][$key])) {
+      if (isset($cache['conditions'][$key][$value])) {

could be this

+    if (isset($cache['conditions'][$key], $cache['conditions'][$key][$value])) {

* get rid of space at start of if condition

+ if ( (array_sum($uid_matches) / $match_count) == $uid_matches[0]) { // Are all uid's in the array equal?

nielsvm’s picture

StatusFileSize
new12.93 KB

Hi,

Thanks for feedback. I've updated the patch to version 12.

Command I used for diff: hg diff -U4 -p

Niels

nielsvm’s picture

Status: Needs work » Needs review

Changed status.

Anonymous’s picture

Status: Needs review » Needs work

The last submitted patch failed testing.

nielsvm’s picture

StatusFileSize
new11.41 KB

Updated to apply on the newest version of core. All unit tests pass again.

nielsvm’s picture

Status: Needs work » Reviewed & tested by the community

Changed status.

nielsvm’s picture

Status: Reviewed & tested by the community » Needs review

Changed status.

Status: Needs review » Needs work

The last submitted patch failed testing.

dries’s picture

michtoen’s picture

Why #16 failed? The test said all was ok... I don't know much about the testing, what i had
overseen?

michtoen’s picture

I converted for testing the last patch to D6.

I tried it on a demo site with a medium web 2.0 module setup.

It gives an instant 10% speed increase for a profile with full enabled caching include
block caching for logged users, i assume the speed increase for a production site will be around
10-15%. Thats pretty nice for such a small patch, even more because it takes effect on a page
with high execution time (for my demo site the page render time was 340-306ms without and down to
266ms with).

Combined with the path_cache module, you will see a pretty clean mysql fingerprint
without the redundant multi-calls.

Really sad this is not in D6. At last there should be a extensional patch for people building
a social site.

catch’s picture

Status: Needs work » Closed (duplicate)

#347250: Multiple load users added this to Drupal 7 but won't be backported because it's an API change.

If someone wants to try to get a basic static cache into D6, re-opening #91786: user_load() static caching is probably the best bet. But for now closing these both out as duplicates.

michtoen’s picture

Version: 7.x-dev » 6.x-dev
Status: Closed (duplicate) » Needs review
StatusFileSize
new5.16 KB

Here is a D6 port of this patch.
Its running fine so far for us at www.daimonin.org.

Not sure it will have side effects for other sites and other non core module installations,
so count it for now as experimental.

crea’s picture

Subscribing

n4964918’s picture

Hi guys. Just a quick question. This is just a basic 3 line code to be added at the top of the function for user_load.

it is assuming that the $array variable is a string of uid, which occurs most if not all the time. I was wondering if there is anything wrong with the code below assuming the array parameter is a string variable of uid.

function user_load($array = array()) {
// Dynamically compose a SQL query:
$query = array();
$params = array();

//My Code
static $user = null;

if($array==$user->uid)
{
return $user;
}
//End of my code

if (is_numeric($array)) {
$array = array('uid' => $array);
}
elseif (!is_array($array)) {
return FALSE;
}

Thanks

GrzegorzNowak’s picture

Version: 6.x-dev » 6.16
StatusFileSize
new3.8 KB

Hi.
My first post.

Problem is that this assumption is definitely to strong. There's high probability to have this function get called with array as a parameter (i'm having that case on few of my installations).

I've made an alternative version of user_load patch that uses serialize() to compose function arguments into unique string, that can be use as a static cache key. Maybe more time consuming but bit clearer and "theoretically" more reliable -)

Status: Needs review » Needs work

The last submitted patch, user.module.user_load.rc1_.patch, failed testing.

GrzegorzNowak’s picture

StatusFileSize
new3.73 KB

Previous patch was created from Subversive SVN, this one is from "diff" command.

crea’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, user.optimize.user_load.rc2_.patch, failed testing.

GrzegorzNowak’s picture

StatusFileSize
new3.69 KB
crea’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, user.user_load.optimize.rc4_.patch, failed testing.

alippai’s picture

Status: Needs work » Closed (fixed)
damien tournoud’s picture

Status: Closed (fixed) » Needs work

Reopening, as this is not closed for D6.

Patch in #24 seems close.

damien tournoud’s picture

Version: 6.16 » 6.x-dev

Moving to the correct version.

damien tournoud’s picture

This is actually a significant API change, as some code might expect to get a fully refreshed object from a call to user_load(). At the minimum, we need to clone the returned object, and we might need to go as far as defaulting to non-cached.

Status: Needs work » Closed (outdated)

Automatically closed because Drupal 6 is no longer supported. If the issue verifiably applies to later versions, please reopen with details and update the version.