Hey guys, I'm digging in on setting up OG and LDAP to work together & i've dug up at least this problem:


  public function usersAuthorizations(&$user) {
     $groups = og_load_multiple(og_get_all_group()); 
		$authorizations = array();
		if (is_object($user) && is_array($groups)) {
			foreach ($groups as $gid => $discard) {
				$roles = og_get_user_roles($gid, $user->uid);
				foreach ($roles as $rid => $discard) {
					$authorizations[] = ldap_authorization_og_authorization_id($gid, $rid);
				}
			}
		}
    return $authorizations;
  }

this function loads every group and iterates through them every time it runs. On systems with a small number of groups this is fine, but this wont be typical - especially in an enterprise env. the organization I'm currently working with only has 280 groups and 3 group roles, but my previous client had over 30,000 groups and 4+ group roles. this presents a serious scaling problem because in traditional groups architectures all group roles are available to all groups. This means you have an N times X problem where the loops are going to process x times for each node. If this ran on my previous client's system, the admin page would show 120,000 example records (30k groups x 4 possible group roles). Adding an additional group role would cause the loops to run another 30k times.

The two places i see the following code used $groups = og_load_multiple(og_get_all_group()); are in the formatter examples page in admin (less of a problem, but still a big problem) and the userAuthorizations function (more of a problem because normal users can essentially run it).

The admin examples can be simplified by limiting the data to only a few nodes, but the userAuthorizations function is going to need to be refactored. There should be a better way to determine these authorizations somehow. I'm working on at least allowing a field to be attached to the group so that the group DN can be placed there. This will at least let the authorization be searchable in the DB to reduce the PHP data loading & looping that is happening here.

Anyone have any added thoughts on this?
-Chris

Comments

johnbarclay’s picture

Aside from the admin interface which is just to produce some example for usability,

1. Since the authorization may not be LDAP group based (such as strategy II.A.), the DN as such may not be appropriate to store in the OG entity. And there may be a many to many relationship between ldap groups and og groups so it would have to be a field that allowed multiple values.

2. If you come up with a way to use caching, it just needs to convert the cached values when role mappings change.

More importantly, I don't see why all the og groups needed to be loaded at all. Why can't the LDAP groups be mapped to OG Group names and roles first, then mapped to og ids, then check if the og group exists when granting the membership? Perhaps this needs to be reworked in the base class or the og consumer class if it must load all possible groups beforehand.

johnbarclay’s picture

Title: dangerous iterations using og_load_multiple & og_get_all_group() » LDAP Authorization: dangerous iterations using og_load_multiple & og_get_all_group()
Assigned: Unassigned » netw3rker
johnbarclay’s picture

I've made good progress on this. I'm converting the og test coverage to the new test scheme, then it will be ready for testing. See http://drupal.org/node/1115704#comment-6750004

johnbarclay’s picture

I committed this to 7.x-2.0-dev. The basic improvements were:
- query with entity field query and load multiple with entity load.
- only load og groups that are being granted to user. If scaling is still an issue, we can avoid loading the groups at all I believe. But its nice to have them for the hooks available to other modules.

It has test coverage, but not for scaling/load testing.

johnbarclay’s picture

Status: Active » Needs review
johnbarclay’s picture

Category: bug » feature
Status: Needs review » Closed (won't fix)

This is resolved for og 2, but not og 1. I don't anticipate going back to finish this for og1. If anyone wants to follow through on this for og1, just follow the pattern in LdapAuthorizationConsumerOg::usersAuthorizations() for og2. Perhaps by adding conditions to og_membership_load_multiple or a using a query like in og_get_group_membership() that doesn't have a condition for entity type.

johnbarclay’s picture

Issue summary: View changes

meant to say 'adding a single group role'. adding a single group would only result in 3-4 more iterations