New node access sytem

chx - October 6, 2005 - 01:59
Project:Drupal
Version:7.x-dev
Component:node system
Category:task
Priority:normal
Assigned:Unassigned
Status:won't fix
Description

Goal: create an API which given a (nid, uid) pair can hand back a SQL fragment, which is capable of determining without requiring DISTINCT, whether there is a connection (grant) between nid and uid. That Drupal handles three grants and not one is not a problem -- we will use three sets of tables.

Problem: lots of (nid, uid) pairs. We just can't store (number of nodes) * (number of users) lines in access, that'd mean more than 900 million lines for drupal.org.

Solution: we group nodes and groups.

Our tables are

access (ngid, ugid)
node_groups (nid, ngid)
user_groups (uid, ugid)

Will build this in such a way that there if we join the three tables and there is a given (nid, uid) then there will be only one row in a result set. That's the trick.

Let's assume we already have a DB in this state and we are have just added a node to a node group. After that, we check whether our (nid, uid) unicity failed and fix it. Fixing happens by creating a new user group, which consists of all users which has access to node. Also we create a new node group which consists of only this node. We add an access rule between the two new groups. Finally, we remove the node from all old groups that caused unicity violation. It can be seen easily that this node will have one row for each users it belongs to. We have not touched any other relationship.

CREATE TEMPORARY temp1
SELECT uid
FROM node_groups ng
INNER JOIN access a ON ng.ngid = a.ngid
INNER JOIN user_groups ug ON a.ugid = ug.ugid
WHERE nid = $nid
GROUP BY nid, uid
HAVING count(*) > 1;

INSERT INTO access (ngid, ugid) VALUES ($new_ngid, $new_ugid)
INSERT INTO user_groups (ugid, uid) SElECT ($ugid, uid) FROM temp1

CREATE TEMPORARY temp2
SELECT ng.ngid
FROM t1
INNER JOIN node_groups ng ON t1.nid = ng.nid
INNER JOIN user_groups ug ON t1.uid = ug.uid
INNER JOIN access a ON a.ngid = ng.ngid AND a.ugid = ug.ugid
WHERE ng.nid = %d

DELETE node_groups FROM node_groups INNER JOIN temp2 ON node_groups.nid = $nid AND node_groups.ngid = temp2.ngid
INSERT INTO node_groups (nid, ngid) VALUES ($nid, $new_ngid)

The multi table delete can be removed and replace by a PHP loop because it's likely that there will be few lines. Same with temporary tables, but this is certainly faster.

The DELETE should cascade, ie. if a node group becomes empty, it should be deleted and the access rules which has that ngid, too. Now, if a user group has no more access rules, then that can be trashed, too.

The exact same goes for adding a single user to a user group.

Phew! Now, it's nice, but usually we do not need to deal with a single nid but an array of them, like belonging to a certain term. Let's call these 'administrative node groups'. When such a beast is created, we just create a ngid for it and create a new user group, too and add relevant users to it. Later, when node groups are split, nodes from one administrative group may have more than ngid but one ngid will always belong to only one administrative node group. So if you add a user to an 'administrative node group', you know which ngids belong to it, and which user groups has access to those groups and you add the user to said groups.

#1

kbahey - October 6, 2005 - 02:41

How frequent is this cascading delete and creation of two temporary tables?

This can be a real performance drain ...

#2

chx - October 6, 2005 - 03:20

This happens only on node insert / update. And why do you think temporary tables which reside in RAM are slow?

That DELETE can be slow, true.

#3

chx - October 6, 2005 - 06:13

Correction: Fixing happens by creating a new user group, which consists of all _offending_ users.

Optmization note: to avoid too many one-node groups which have arisen from splitting, it worths trying to select an user group which contains exactly those URLs that the node has access to and a matching node group from that.

Hint: if you want to have a clear picture, draw. Points in four columns, first nid, second ngid, thrid ugid, fourth uid. It really helps. I may try later drawing by Inkscape or Dia.

#4

Chris Johnson - October 6, 2005 - 18:24
Title:New node access sytem» 4 step illustration of duplicate access grant resolution algorithm

Algorithm Illustration Step 1 -- Original situation

AttachmentSizeStatusTest resultOperations
original.gif6.09 KBIgnoredNoneNone

#5

Chris Johnson - October 6, 2005 - 18:27

Algorithm Illustration Step 2 -- add a nid to an ngid

AttachmentSizeStatusTest resultOperations
addingnid.gif6.91 KBIgnoredNoneNone

#6

Chris Johnson - October 6, 2005 - 18:27

Algorithm Illustration Step 3 -- resolving the conflict

AttachmentSizeStatusTest resultOperations
resolve.gif9.68 KBIgnoredNoneNone

#7

Chris Johnson - October 6, 2005 - 18:28

Algorithm Illustration Step 4 -- cleanup

AttachmentSizeStatusTest resultOperations
resolve_0.gif9.68 KBIgnoredNoneNone

#8

Chris Johnson - October 6, 2005 - 18:29

Algorithm Illustration Step 4 -- cleanup

(previous post had wrong attachment, sorry)

AttachmentSizeStatusTest resultOperations
cleanup.gif7.87 KBIgnoredNoneNone

#9

chx - October 6, 2005 - 18:33
Title:4 step illustration of duplicate access grant resolution algorithm» New node access sytem

Original situation:

Add a nid to a group:

Resolve the conflict:

Cleanup:

Thanks Chris!

#10

chx - October 6, 2005 - 18:34

I tried to inline the images and they were working during preview (yes I have full HTML access) :(

disregard my previous post.

#11

Steve Dondley - October 7, 2005 - 19:15

Would a new taxonomy_access module be able to tie into this?

#12

chx - October 7, 2005 - 20:38

Sure it would create administrative node groups based on tids.

#13

tostinni - October 7, 2005 - 23:26

And as chx told on the devel list, the trick is to provide a taxonomy_access without needing patching core ;)

#14

syllance - October 20, 2005 - 13:07

please check http://drupal.org/node/34072

keve rewrote taxonomy_access using the _db_rewrite_sql hooks added to the taxonomy module, and it seems to work well, at least for the tests i've done until now. no patch needed :-)

#15

chx - November 26, 2005 - 04:41

Another illustration (by Balázs Nagykékesi, who has helped me lots with the algorithm itself, thanks!)

AttachmentSizeStatusTest resultOperations
1_0.jpg14.17 KBIgnoredNoneNone

#16

LAsan - April 7, 2008 - 08:23
Version:x.y.z» 7.x-dev

Feature request moving to cvs.

#17

Steven Jones - April 12, 2008 - 06:33
Assigned to:chx» Anonymous
Status:active» won't fix

This is way old, and now very much outdated.

@LAs4n: please check issues, don't just blindly set to 7.x

 
 

Drupal is a registered trademark of Dries Buytaert.