Our userpoints and userpoints_txn tables are inconsistent. Some of the transactions are not resulting in matching updates in the "cache" -- the userpoints table.

This is a sign of a larger problem.

There are two tables being updated on every "userpoint transaction" and this is fundamentally wrong. If any of the queries or checks fails, there is going to be inconsistency. Example, from _userpoints_transaction function:

    db_query('UPDATE {userpoints_txn} SET '. implode(', ', $arr) .' 
              WHERE txn_id = %d',
              $params['txn_id']
            );
    _userpoints_update_cache($params);

If db_query fails for any of a million reasons, _userpoints_update_cache will be called anyway, resulting in inconsistency.

Another example, from #407356: Error in _userpoints_update_cache() prevents awarding or removing uncategorized points from users:

  if (!isset($params['tid'])) {
    $params['tid'] = NULL;
  }

Results in failed query in _userpoints_update_cache. So the original "transaction" wont be reflected in the "cache".

Proposed solution:

  • Rewrite _userpoints_update_cache so it recalculates the cache from data in userpoints_txn table instead of the running sum based on the new transaction and existing data in userpoints table that it is doing now
  • In addition, the _userpoints_update_cache should be called after each transaction, failed or not, the cost is minimal. This ensures that any future inconsistencies are automatically fixed after the next transaction.

I think these two changes will ensure that the two tables will stay consistent eradicating a whole family of existing bugs that we will chase forever. I am willing to provide a patch if there is interest from a maintainer.

CommentFileSizeAuthor
#2 userpoints_cache_fix.patch3.25 KBnick.dap

Comments

kbahey’s picture

Version: 6.x-1.0 » 6.x-1.x-dev
Component: Code: userpoints_basic » Code: userpoints API
Priority: Critical » Normal
Status: Active » Needs work

If there is a patch that others can review, please include it.

nick.dap’s picture

StatusFileSize
new3.25 KB

This is fresh off the presses. In particular the userpoints_update_6012 function in userpoints.install is probably not up to standards.

Please excuse the mercurial diff against my own repository. Patch is against 6.x-1.1 (I've updated since posting the original report).

kbahey’s picture

First, rename update_6012 to update_6013, because 6012 is already used now (another patch introduced it).

Second, I don't like this part. It is too scary. Specifically, the inner join on a sub select. We need to simplify this and make sure that it does not bog down performance for sites with lots of transactions.

+  $tnx_data = db_fetch_array(db_query("
+      SELECT SUM(p.points) AS current_points, mp.max_points
+      FROM {userpoints_txn} AS p
+      INNER JOIN ( SELECT ut.uid, ut.tid, SUM(ut.points) AS max_points
+                   FROM {userpoints_txn} AS ut
+                   WHERE ut.status = 0
+                         AND ut.points > 0
+                         AND ut.uid = %d
+                         AND ut.tid = %d
+                 ) AS mp
+            ON (mp.uid = p.uid AND mp.tid = p.tid)",
+     $params['uid'],
+     $params['tid']
+  ));
mile23’s picture

Status: Needs work » Fixed

This looks fixed in 6.x-1.x-dev. See lines 766 and 483, respectively.

What say ye, folks?

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

IWasBornToWin’s picture

Component: Code: userpoints API » Code: userpoints
Status: Closed (fixed) » Active

I'm not sure this is the correct place to ask this but; I'm using version 7 and it still does two table updates. My totals in my userpoint table does not equal the amounts added together in my individual transaction records. Did this not get carried over to version 7?

thanks

berdir’s picture

Status: Active » Closed (fixed)

It is not the correct place.

Please create a new issue and provide steps to reproduce your issue *on a clean install*. Without that, I most likely can't fix your problem as I have not experienced anything like that.

attheshow’s picture

I ended up using the second half of the #2 patch above (starting at the line:)
$params['tid'] = 0;