I am writing a module to keep LDAP and Drupal users in sync. The module will create a new user for each LDAP user not present in Drupal, update user information for each LDAP user present in Drupal, disable the account of each user disabled in LDAP, and disable/delete the account of each LDAP-authentified user missing from LDAP.

The main reason for this module is that ldapauth currently creates a Drupal user only when a new LDAP user logs in. LDAPsync will create the users before they log in, so that they may have nodes attributed to them and be referenced by other nodes before first login. We are migrating content from an old site into a new one, so can't wait for users to log in before linking to them in various ways.

This module is dependent on ldapauth, allowing us to use existing settings for the ldap server, bind DN, and OU to search and existing functions to create new users and edit existing ones when needed. Users will continue to log in using LDAP.

I'm not sure how this module will play with ldapgroups. As it currently exists, ldapgroups creates Drupal user information for ldap users only when the user logs in, and if I understand correctly, it attempts to create a Drupal group for each LDAP group detected. In our case, we need a simpler roles structure in Drupal than we have in LDAP, so I am building group-role mapping instead. I will defer to the crowd to suggest how this should play with ldapgroups.

This project could potentially just be a patch to ldapauth ("keep users in sync"), instead of standing alone as its own module.

I am developing and testing on Drupal 6/MS Active Directory/Windows Server 2003.

I plan to post code within about a week. As a matter of interest, I am pulling some of the synchronization information from Blackbaud Education Edge but will remove that code in this space!

This need was also described by taliesin in 2007 (http://drupal.org/node/170197).

This is my first time publishing a module to Drupal community, so I ask for your patience and tips as I navigate this online community! My next question: when I have code to post, should I post it here or somewhere else?

Thank you in advance for your comments and support.

Richard

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

kassissieh’s picture

FileSize
4.62 KB

Here's a first run at code. Thanks for taking a look. Help is welcome.

roball’s picture

Thank you so much Richard for your work. Haven't tried your first code yet - want to clarify one thing first: do you attempt to only make changes in the Drupal database, but leave the data in the LDAP server untouched? If so, that's exactly what I'm looking for. My LDAP server is OpenLDAP and I am afraid of a possible data loss on its side if I try out your module.

kassissieh’s picture

That's right. At our institution, the LDAP information is authoritative.

Richard

roball’s picture

Anyone tested Richard's code so far? It would be great if Miglius could have a look into it and comment here what he thinks.

wvd_vegt’s picture

Hi

Some comments from my posting at ldapauth (VERY promising module btw):


If the LDAPSync where to provide some configuration options to skip new users (not all of our ldap users are allowed to login) or present a checkable list of new users found before adding them, I could surely use it!

Another useful option would be a button to update a single user (I now force that by filtering on a UserId in the patch I provided).

roball’s picture

wvd_vegt, the sync should ideally happen silently in the background, via cron. Thus adding a manual confirmation step before actually adding new users to Drupal won't do it.

kassissieh’s picture

wvd_vegt, I would recommend that the LDAP administrator place accounts that should not have Drupal privileges into a different OU, and then list only the OUs that should have Drupal accounts in the ldapauth settings.

roball’s picture

+1 for #7. That's also the way we do it and the way supposed to be by other CMSes like Moodle. You should maintain an exclusive OU in LDAP that contains *all* users to be synched to Drupal and the other systems you want to use the same set of users (Moodle in my case). For example, the contexts I am using for both Drupal and Moodle is "ou=moodleusers,dc=mydomain,dc=tld" only.

kassissieh’s picture

It may not be practical to maintain an exclusive OU to Drupal users, b/c the sysadmin may also be using OUs to apply group policies, etc. However, if your Drupal user organization mirrors your LDAP organization, then at least all of the members of a particular OU should either get or not get Drupal user accounts. For example, you might have an OU for real people, an OU for system accounts, and an OU for computers. Only the OU for real users should get Drupal accounts.

wvd_vegt’s picture

Hi,

Due to some pollution we need to mop-up some things (ie pull it through ldapdata). A cron solution might do the trick though.

At the moment though we update manually as the view on top & user records only updates after explicitly pressing F5 (refresh) in the browser.

Furthermore do I have no influence over the AD so any changes there are out-of-scope

wvd_vegt’s picture

Hi,

As i said I have no influence over the AD content or procedures. I only pull users belonging to our department (if data was correct they should all have access) and even there is a bit pollution so I have to filter them and convert/adjust some attributes that are not updated on regular basis.

So our users are already in a nice group, but there is some pollution of the data so back to square one.

I agree that the AD data should be correct at the first place but to force it that's out of my reach (and i'm forced to live with it).

kassissieh’s picture

Here's another approach. You could allow ldapsync to pull all users from specified OUs into Drupal and then manually disable those users you don't want logging into Drupal. They don't have to know that you've done this, as ldapsync won't notify users of account creation. Ldapsync will post a watchdog message each time it makes changes to Drupal users, so you could keep an eye out for that and take action when ldapsync creates new accounts that you don't want enabled.

roball’s picture

Richard, I'm not sure If you have read Miglius's comment about your new module in another ticket (which I can't find anymore now). He mentioned that he prefers your module to be released as its own sub module of the "LDAP" module group. Are you going to pull it at http://drupal.org/project/ldap_sync?

kassissieh’s picture

Yes, I imagined that it would be a sub-module of the LDAP module group. It already depends on ldapauth. It could even be a patch to ldapauth if people prefer it that way.

roball’s picture

I also think a separated module is the better choice than providing a patch, as long as Miglius prefers this code to be independently maintained.

roball’s picture

I have now uploaded your module to my site and reviewed its code through the excellent Coder module (http://drupal.org/project/coder), having the Show warnings at or above the severity warning level option set to minor (most). It gave a lot of (more than 100) warnings. Thus, I would suggest you to also use the Coder module in assisting for cleaning up your code to be drupalish.

Also, adding a line like
version = 6.x-0.1
to ldapsync.info would be helpful.

kassissieh’s picture

FileSize
4.8 KB

Easy enough. Done.

Basic functionality seems solid in my environment -- I created 935 users in a few seconds tonight. It would be great to find out how it does in yours.

I have noted in the code what line you can comment out if you want to test the module but don't actually want to create new Drupal users yet.

Next, I need to work on the settings page and think about how to reconcile the profile field needs of this module with the work of the existing ldapdata module.

Anyone else who would like to write code is most welcome to do so.

miglius’s picture

There are two options. This module can be release as a separate project and be developed there, or when it's ready and well tested I could include it in the ldap_integration bundle.

roball’s picture

Excellent - thanks a lot, Richard! Now, the code of LDAP Synchronization 6.x-0.1-dev is reviewed by Coder with the result "No Problems Found". Perfect!

Once it is ready and tested enough, I would love to see it bundled in the ldap_integration package. It is so important for me, that it's a must have. And reducing Drupal's "yet-another-never-ending-must-have-module" pollution would be more user friendly, IMO.

roball’s picture

I have now tried to enable the module for the first time, but immediately got a white screen of death afterwards. Inspecting the error log gave that the ldap_integration sub directory is missing in your package. You are referring to 2 files expected in that directory in the ldapsync_init() function.

Now, I can even no longer access any Drupal page at all!!!

roball’s picture

Priority: Normal » Critical
kassissieh’s picture

Did you install ldapsync in /modules/ldap_integration? If not, please try that.

Yes, ldapsync does require files in /modules/ldap_integration/ldap_integration. However, it requires them in the same manner as ldapauth, ldapgroups, and ldapdata require them. Additionally, I specify ldapauth as a required module for ldapsync to function. So, I believe I have implemented these requirements in an appropriate manner.

Please let me know if this is fine or if you have a suggestion for improvement.

Richard

kassissieh’s picture

My ldap_integration directory looks like this. Note the ldap_integration subdirectory.

drwxr-x--- 4 4096 2009-03-19 12:02 .
drwxr-x--- 36 4096 2009-03-17 21:29 ..
-rwxr-x--- 1 1206 2007-08-29 19:05 ChangeLog
-rwxr-x--- 1 9 2007-01-29 21:06 .cvsignore
-rwxr-x--- 1 17434 2008-08-11 04:10 ldapauth.admin.inc
-rwxr-x--- 1 333 2008-08-12 16:15 ldapauth.info
-rwxr-x--- 1 3519 2008-08-11 04:10 ldapauth.install
-rwxr-x--- 1 15747 2008-08-11 04:10 ldapauth.module
-rwxr-x--- 1 10208 2008-08-11 04:10 ldapdata.admin.inc
-rwxr-x--- 1 339 2008-08-12 16:15 ldapdata.info
-rwxr-x--- 1 1822 2008-07-22 06:06 ldapdata.install
-rwxr-x--- 1 21693 2008-08-12 15:21 ldapdata.module
-rwxr-x--- 1 1675 2008-07-22 06:06 ldapdata.theme.inc
-rwxr-x--- 1 7110 2008-08-11 04:10 ldapgroups.admin.inc
-rwxr-x--- 1 363 2008-08-12 16:15 ldapgroups.info
-rwxr-x--- 1 2374 2008-08-11 04:10 ldapgroups.install
-rwxr-x--- 1 10029 2008-08-11 04:10 ldapgroups.module
drwxr-x--- 2 4096 2008-08-12 16:15 ldap_integration
-rwxr-x--- 1 173 2009-03-17 22:03 ldapsync.info
-rwxr-x--- 1 10088 2009-03-19 11:58 ldapsync.module
-rwxr-x--- 1 14940 2006-04-23 02:54 LICENSE.txt
-rwxr-x--- 1 1546 2008-08-08 05:42 README.txt
drwxr-x--- 2 4096 2008-08-12 16:15 translations

roball’s picture

Priority: Critical » Normal

Ah, did not know that your module directory must be a sub directory of ldap_integration. Anyway, the LDAP integration module package has changed its file structure as of version 6.x-1.x-dev (2009-Mar-19). There is no longer a ldap_integration sub directory.

Its structure now looks this way:

-rw-r--r-- 1 apache apache  1206 Aug 30  2007 ChangeLog
drwxr-xr-x 2 apache apache  4096 Mar 19 00:12 includes
-rw-r--r-- 1 apache apache 20861 Mar 18 12:15 ldapauth.admin.inc
-rw-r--r-- 1 apache apache   341 Mar 19 00:12 ldapauth.info
-rw-r--r-- 1 apache apache  3660 Mar 18 12:15 ldapauth.install
-rw-r--r-- 1 apache apache 17788 Mar 18 12:15 ldapauth.module
-rw-r--r-- 1 apache apache 14061 Mar 18 12:15 ldapdata.admin.inc
-rw-r--r-- 1 apache apache   371 Mar 19 00:12 ldapdata.info
-rw-r--r-- 1 apache apache  2233 Mar 18 12:15 ldapdata.install
-rw-r--r-- 1 apache apache 21975 Mar 18 12:15 ldapdata.module
-rw-r--r-- 1 apache apache  1812 Mar 18 12:15 ldapdata.theme.inc
-rw-r--r-- 1 apache apache 13900 Mar 18 12:15 ldapgroups.admin.inc
-rw-r--r-- 1 apache apache   366 Mar 19 00:12 ldapgroups.info
-rw-r--r-- 1 apache apache  3419 Mar 18 12:15 ldapgroups.install
-rw-r--r-- 1 apache apache 12407 Mar 18 12:15 ldapgroups.module
-rw-r--r-- 1 apache apache 14940 Apr 23  2006 LICENSE.txt
-rw-r--r-- 1 apache apache  1546 Aug  8  2008 README.txt
drwxr-xr-x 2 apache apache  4096 Mar 19 00:12 translations
ssudahar’s picture

Hi,

LDAPsync component team, very happy to see the solution for the LDAP sync. I'm also looking for the same component and I would like to start the test with LDAP. I think that I need to use my own function as $faculty in line 66 right. please confirm

Thank you
Sudahar

kassissieh’s picture

FileSize
5.45 KB

Here is a new version of ldapsync that is compatible with ldapauth-1.x-dev.

To try ldapsync, install the attached files within modules/ldap_integration. To sync users, go to /admin/settings/ldap/ldapsync. The sync is one-way (from LDAP to Drupal). I strongly suggest that you first try this module on a test site, as it will create a Drupal user for each LDAP user!

ssudahar, I have removed $faculty from the module code. I was building a profile data synchronization feature into the module but have instead decided to use existing functions in ldapgroups and ldapdata to add features to ldapsync. I will add those features next.

roball’s picture

I have tried your current dev snapshot, commented out the line
$user = user_save('', $userinfo);
and uncommented the line
$page_content .= "<p>Create user ". $name ." ". $pass ." ". $mail ." ". $init ."<br />". $ldap_users[$name]['dn'] ."</p>";
within ldapsync.module to enable "debug only" mode.

Then pressed the [ Sync now ] button. This gave the message
Completed LDAP sync. New users: 0. Disabled users: . Orphaned users: 1.
but nothing else. Shouldn't that at least state to which users the mentioned actions apply? Also note the result of Disabled users (empty instead of zero). Why would your module not have added any new user? I have more than 100 on the LDAP server that should by synced.

roball’s picture

I am trying to debug your code now. First error found is that you have defined $count_updated_users to be 0, but never used that variable anywhere. It seems that you intended that variable to be named $count_disabled_users instead.

roball’s picture

Next error found: when starting the sync, a database error occurs. admin/reports/dblog reports "mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given".

spjsche’s picture

subscribing.

roball’s picture

The current code is far away from working with OpenLDAP. I would suggest to completely remove the code related to $count_disabled_users, since it won't work on LDAP servers other than MS AD.

The $count_orphaned_users related code has to implement a measure of proof before actually blocking users. For me, that code blocked most of my fully valid users. Had to comment out db_query() there.

kassissieh’s picture

Roball, I welcome your help. From your report, I wonder whether the script is not finding any LDAP users when it executes the search. May I assume that LDAP users may already log into your site using ldapauth? Do you know whether the filter $name_attr=* works in OpenLDAP as it does with Active Directory? Can you insert watchdog statements periodically to monitor the buildup of the $users and $ldap_users arrays? For example:

watchdog('ldapsync',implode(',',$ldap_users));

I did previously see the mysqli error but solved it for my setup. This is a difficult error to troubleshoot (try Googling it), since the source of the error is not noted in the error statement.

$count_updated_users is fixed in the next module update (below).

$count_orphaned_users will misbehave if querying your LDAP directory isn't working. I'm not sure how to "prove" that a user doesn't exist if the script is not properly reading from LDAP.

kassissieh’s picture

FileSize
6.52 KB

Here is a new version that implements cron and groups support. I could use advice on groups support. Right now, it's simple and well integrated with ldapgroups but woefully inefficient, making one query to ldapgroups for each user in your directory. Can you think of a better way to implement groups support without modifying the ldapgroups module?

Please use this module on a test site only while it is in early development.

roball’s picture

Some comments to your new version:

  1. Replace require_once(drupal_get_path('module', 'ldapsync') .'/includes/LDAPInterface.inc'); by require_once(drupal_get_path('module', 'ldapauth') .'/includes/LDAPInterface.inc'); to prevent the first problem I had (see reply #20). This allows your new dev code to be tested inside an own directory ldapsync and removes the requirement of being inside the ldap_integration dir.
  2. $page_content is not used anywhere - thus remove it, including the misleading comment
  3. $count_updated_users problem reported in #28 is still there
  4. You REALLY have to remove the code related to $count_disabled_users as mentioned at #31 - or at least make it optional by a checkbox setting
  5. As mentioned at #31 as well, the code related to $count_orphaned_users is dangerous, at least with OpenLDAP and thus should be removed or made optional for now!
  6. $num_matches is not used anywhere, thus remove it for now to avoid code pollution
  7. The indentation at foreach ($ldapresult as $entry) is wrong and thus difficult to debug
  8. ldapsync_admin_settings_submit is no hook - thus the comment is not correct
  9. ldapsync_admin_settings_submit is no hook - thus the comment is not correct
  10. variable_set() within hook_install() does not work - thus remove it and set the vars in ldapsync.admin.inc instead.
  11. Please do not add new features unless known critical bugs are fixed first - thank you!
roball’s picture

FileSize
4.01 KB

Here is a DEBUG ONLY version, with most of the fixes mentioned in #34 above. Also with code cleanup so coder does not complain. It does not change anything, so it is save to experiment with.

kassissieh’s picture

Many thanks for these, Roball. Any luck finding out why the module's LDAP search is not returning results from your OpenLDAP? It seems that finding this out should be a top priority at this point in time.

roball’s picture

Here is a new DEBUG_ONLY version, that fixes one important bug: the ldapsync_search() function has been renamed to _ldapsync_search() - otherwise it would be the implementation of hook_search() - so it would be called every time when somebody uses the site wide search box!

gettysburger’s picture

subscribing

kassissieh’s picture

We currently have two active developers/testers for this project. We are currently waiting for someone running OpenLDAP to test this module and troubleshoot the LDAP query that selects all users (currently "$name_attr=*"). Thank you.

Richard

roball’s picture

Richard, I will continue to debug against OpenLDAP in the next view days.

erikwebb’s picture

Great module. Subscribing...

jeff.k’s picture

subscribing

roball’s picture

I have updated the DEBUG_ONLY further (to the below attached 0.4 version), which requires ldapdata.module as well because all the sync mapping is defined in that sub module. Currently, when pressing the [ Sync now ] button at Administer » User management » LDAP » Synchronization, it should only display the LDAP attributes Drupal user fields should be synchronized to, like this:

Drupal user fields to be synchronized from the LDAP attributes listed below:
Array
(
    [ldap_amap-4] => c
    [ldap_amap-5] => l
    [ldap_amap-6] => postalAddress
    [ldap_amap-7] => o
    [ldap_amap-8] => departmentNumber
    [ldap_amap-9] => labeledURI
    [ldap_amap-10] => telephoneNumber
    [ldap_amap-11] => facsimileTelephoneNumber
    [ldap_amap-12] => description
    [ldap_amap-13] => givenName
    [ldap_amap-14] => sn
)
roball’s picture

FileSize
3.82 KB

Here is the first working version (ldapsync-6.x-0.5). It works for me with OpenLDAP 2.3.43 coming with CentOS 5.3. I have removed the dependency of the ldapdata.module for now, since data mapping can be done with that module itself.

Tested successfully with the LDAP integration module 6.x-1.x-dev (2009-May-06). The sync will only be performed on pressing the [ Sync now ] button.

kassissieh’s picture

FileSize
4.37 KB

Many thanks, roball. Here is one small fix to the module. It now confirms that $ldap_users contains users before declaring Drupal users orphaned.

kassissieh’s picture

Roball, something interesting happened to ldapsync when I upgraded to the latest dev versions. My search of Active Directory failed until I changed the user attribute to lower case. In other words, the setting "sAMAccountName" worked fine when I was on the previous dev version, but only "samaccountname" worked after I upgraded. I was unable to figure out why. I was still able to authenticate as an LDAP-authentified user, so I don't think anything is wrong in ldapauth. However, the help instructions for ldapauth settings suggest sAMAccountName, so that instruction is now incorrect, at least for my setup.

Richard

jeff.k’s picture

Same here.

gettysburger’s picture

I just tried this module, which I am very excited about, and wanted to report my findings.

I clicked the Sync Now button and it created 917 new users and 15 orphaned users.

1. In the new accounts there is only the username. If I open the account and save it then the rest of the fields come into the account. Should they not all come in initially?

2. What is an "orphaned user"?

Thanks again for making this module.

gettysburger’s picture

BTW, I was using a March 30 version of the -dev and upgraded to the latest, thinking that might fix things, but I got this error:

• warning: ldap_get_option(): supplied argument is not a valid ldap link resource in /srv/www/htdocs/sites/all/modules/ldap_integration/includes/LDAPInterface.inc on line 82.
• warning: ldap_errno(): supplied argument is not a valid ldap link resource in /srv/www/htdocs/sites/all/modules/ldap_integration/includes/LDAPInterface.inc on line 167.
• warning: ldap_error(): supplied argument is not a valid ldap link resource in /srv/www/htdocs/sites/all/modules/ldap_integration/includes/LDAPInterface.inc on line 167.

Any thoughts? Thanks.

tomhung’s picture

FileSize
8.96 KB

So there were a couple problems with the latest version 6.x-0.5.

A. The attributes needed to be drupal_strtolower so the return from LDAPInterface.inc matched the attributes
B. There wasn't a filter to allow ldapsync to only get real users instead of users & computers, etc. (similar to ldapdirectory)

We made the changes to the above problems.

Please test this version and see if there are any problems.

Greg

tomhung’s picture

FileSize
4.24 KB

better tar ball....

punkassjim’s picture

I don't see it in any comments in module files, and I don't see clear notation in this thread: are you guys working specifically against the latest development snapshots of LDAP Integration? Or the latest recommended releases? Just want to make sure to grab the right versions.

mtndan’s picture

Category: task » feature

Worked great, thanks! Is there a way to bulk update the imported accounts with group/role mappings taken from our existing ldap_integration groups module settings?

kassissieh’s picture

Yes, are you comfortable looking at the module code? I thought I put that in the current version, but I may have it only in my Drupal installation. Let me know, and I can publish what I did locally.

Richard

mtndan’s picture

Pretty comfortable - I don't see it in the current version. Can you post your most recent please?

gettysburger’s picture

I had this working now I'm out of the water again. When I try to sync, it takes forever (a few minutes) then gives me no status update. I remember before it would tell me how many accounts it saw and updates. Now it just goes to a white screen. I tried upgrading the LDAP Authentication module to the beta 1 and got the same thing.

mtndan’s picture

Richard, would love to see this feature. Any idea when you'll have a chance to upload the code?

kassissieh’s picture

FileSize
3.88 KB

Here is the latest code from my production site. I suggest taking a look at this code for ideas, not just installing it on your site, as it does not contain the latest improvements from other contributors to this thread. It would be much appreciated if someone were interested in merging the changes into a new version.

kassissieh’s picture

gettysburger, if you turn "display errors" temporarily, you may find out why your site is going to white screen.

In sites/default/settings.php, add:

ini_set('display_errors', 1);

cels’s picture

subscribing.

This is a *must* component! I hope that this component is integrated in the module. Thanks for it.

geste’s picture

Richard,

EDIT: This post was way too long and had a lot of extraneous stuff in it. So I am shortening it..

Our Drupal/intranet users are authenticated by an enterprise HTTPD ID (via the PubCookie system) and I am able to auto-create Drupal accounts compliments of the Webserver Auth module. Previously, getting LDAP->Drupal group mapping has depended on LDAP authentication which we don't have. I have installed the latest ldap_integration with John Barclay's group mapping enhancements. But still no luck getting group info for users

But then I found this thread. If we can simply use this for cron'd LDAP->Drupal synch of users and groups, it could hugely reduce the number of hits on our LDAP lookups on our server.

Anybody try it against "389" (nee Fedora Directory Services) or will I be the first?

Jim

gettysburger’s picture

Thanks. I'll give this a try.

geste’s picture

I set up another test Drupal 6.14 test instance and added latest WebServer Auth and LDAP Integration modules. Then I added LDAP Sync from post #58 and made no modifications whatsoever to those 4 files..

- I got the LDAP Sync configuration menu immediately
- I saw the "Sync Now" button (Duh!)
- I ran sync and it successfully synched all users in the defined ou=People container

I found no evidence of any sync of group/role information. EDIT: Wrong. I was looking at some users that had "webserver_auth" in their authmap and, no, their group membership did not update.

FYI, this was run against Fedora Directory Services v1.x (now "389 Server").

Thanks,

Jim

geste’s picture

Regarding group sync, I checked and saw that your group section:

// update user's groups if ldapgroups is enabled
    if (function_exists('ldapgroups_user_login')) {
      ldapgroups_user_login($account);
    }

was being called as expected. So I followed the trail to the ldap_groups_login function in the updated ldapgroups.module. There I see it is doing a test for ldapauth right at the top of the function:

if (!isset($authmap['ldapauth'])) {
    // This user is not authenticated via ldapauth.
 return;
}

I bypassed this by simply commenting out the test and the next time I sync'd, the user and user_role tables populated all of the LDAP groups and user group/role membership. Edit: I think this was a red herring. I had a couple of users in the user table already and their authmap shows "webserver" auth type. Group sync for remaining newly-synced users with authmap auth value of "ldapauth" worked. I guess I can just leave all of these authmap values as "ldapauth" (even though they are not).

Great. So now it looks like I should turn my attention to the ldapgroups module and understand it better. Biggest issue is:

- having orphaned users is OK (we actually want to keep them) but group membership must be kept up to date with LDAP being authoritative.

I need to see what the ldapgroups module does to group memberships when a user is removed form a group in LDAP.

Edit: I checked -- I removed a user from an LDAP group and the change was propagated from LDAP to Drupal when I sync'd.

Anyhow, I am using SSO/HTTPD authentication and, am synching LDAP group membership for (Drupal) roles/authorization. Yay!

Thanks!!

Jim

crabcakes’s picture

subscribed (to the right one this time :] )

markDrupal’s picture

This is incredibly useful, when will we see this rolled into LDAP integration? or moved into it's own module : http://drupal.org/project/ldap_sync ?

EgonO’s picture

subscribe

bstoppel’s picture

Subscribe. I am planning on installing and testing the latest patch as indicated in comment #58.

kassissieh’s picture

Thank you for installing and testing! This components needs more of that to move to the next step. Please do suggest code changes or attach a new version if you can.

Richard

bomarmonk’s picture

Subscribing

bendiy’s picture

I've tested the code in #51 and it appears to be working fine. I can sync and it will bring in my users.

I did find one error with PostgreSQL in #51 as well as #58.

Line 205:

    $result = db_query("SELECT 'uid', 'name', 'data' FROM {users} WHERE 'status'=1");

Should be changed to this for PostgreSQL to work:

    $result = db_query("SELECT uid, name, data FROM {users} WHERE status=1");
peter panes’s picture

Hi this will be an excellent module, when will this be available as a dev download, i'm just wondering if i should start fiddling with it now or wait for the module to be released?

kassissieh’s picture

FileSize
5.49 KB

Here is a new version with the following improvements:
- added new sync time interval of "never (manual sync only)" to admin settings
- added new admin setting to control whether Drupal users missing from LDAP are automatically blocked
- limited LDAP search to enabled accounts (only works with Active Directory)
- confirmed that ldapsync works with LDAP integration 6.x-1.0-beta2
- fixed SQL compatibility issue noted in #71. Thanks.

To do:
- need someone with a non-AD LDAP directory to add code that limits LDAP search to enabled accounts (line 230)

Given previous comments, I'm happy to move this into its own module, once I look into how to do that! Please let me know what additional issues you find. Thanks.

vincetingey’s picture

Subscribing

djpavlo’s picture

Category: feature » bug

Has established ldapsync-6.x-0.7, two times it was synchronised, then in general has ceased to work. I add new users in AD, then I press SYNC, gives out "Completed LDAP sync. New users: 0. LDAP-authentified users that do not have an enabled LDAP account: 0." In what there can be a problem?

Added. Has found a problem. FCKEditor has added tags in filter LDAP Integration parametres. Therefore synchronisation also did not work.

There was a second question. The module now can block users that do not exist or are disabled in LDAP, but whether can it unblock again enabled users?

kassissieh’s picture

Thanks for trying this out. I'm glad you found the issue with FCKeditor and the LDAP server settings.

Thank you for your suggestion to enable blocked Drupal users that are enabled in LDAP. The only problem is that a webmaster might have blocked such users in Drupal manually, so I don't want to take that control out of the webmaster's hands. I'm interested in what others think about this feature suggestion.

vincetingey’s picture

LDAPsync seems to be working fine for us running the latest 389 Directory Server!

kassissieh’s picture

Category: bug » feature
Status: Active » Needs review

Great! Thanks for reporting back.

(changing category back to "feature request" and status to "needs review")

We still encourage more testing to confirm the reliability of this module. I have submitted a CVS account request in order to move this into its own project.

BTMash’s picture

Status: Needs review » Needs work

The module is working very well! Its running on our servers and the synchronization with our AD server is coming up correctly. However, there *is* one issue which has to do with the cron tasks. On our servers we noticed a reasonably large spike whenever we ran cron (in our case of running every 10 minutes, it wasn't looking good). What I found is that currently, placing a setting for the ldapsync to run cron a max with its minimum time does not currently work. The reason is that while you store the time interval between syncs in hours, the difference is checked in seconds. So for the values you currently have:

    0 => t('As often as possible'),
    1 => t('1 hour'),
    6 => t('6 hours'),
    12 => t('12 hours'),
    24 => t('24 hours'),
    168 => t('1 week'),
    999999 => t('never (only sync manually)')

1 hour is 1 second, 6 hours is 6 seconds and so on (with the exception of 999999 which you have a special case for). So my recommendation would be (if you are sticking with hours) to multiply the time interval by 3600. Your cron function then becomes:

/**
 * Implements hook_cron().
 * checks ldapsync_time_interval and ldapsync_last_sync_time variables to determine whether to run ldapsync
 */
function ldapsync_cron() {
  // Time is stored in hours so we need to multiply by seconds for the real difference.
  // 999999 hours translates to over 100 years which makes it friendly to manual sync.
  $time_interval = variable_get('ldapsync_time_interval', 999999) * 3600; 
  $last_sync_time = variable_get('ldapsync_last_sync_time', 0);
  if ((time() - $last_sync_time) > $time_interval) {
    _ldapsync_sync();
  }
}

999999 is a very long time (over 100 years) in either case and you don't need the extra check.

And that should fix the cron issue.

kassissieh’s picture

Status: Needs work » Needs review

Thank you BTMash for finding that issue. I've fixed it and will include in the next version I post. I actually made the change in the admin settings field value instead of here, so that the stored value in the database is in seconds.

Richard

kassissieh’s picture

FileSize
5.31 KB

Here is a new version that includes a fix for #79 as well as changes requested as part of my CVS application review.

verta’s picture

Category: feature » task
Priority: Normal » Minor
Status: Needs review » Fixed

Subscribing

roball’s picture

Priority: Minor » Normal
Status: Fixed » Active

I think the status of this issue should be kept on "active" as long as this module gets official.

jeff.k’s picture

The latest sync works great. They one thing I do notice is that I need to sync twice. 1) gets the the user and adds them 2) gets all their data and group membership). Is this behaving as it should or is this a known bug.

bomarmonk’s picture

Subscribing

bomarmonk’s picture

I get the following error when trying the "sync now" button: Fatal error: Call to undefined function ldap_connect() in /homepages/mysite/sites/all/modules/ldap_integration/includes/LDAPInterface.inc on line 132
Any ideas of what I did wrong? Or what is going wrong? Thanks for the help.

ssudahar’s picture

I think that you did not enable the PHP extension called "php_ldap" if so please enable it.

kassissieh’s picture

I have not seen the behavior described in comment 84 before. Has anyone else?

worldlinemine’s picture

In regards to issue #84, I too have been attempting to implement the sync, and it definitely seems to work fine for grabbing and creating the accounts, but I have not been able to get it to grab any data module attributes beyond the initial mail that the system acquires on creating an account. This is regardless of how often I attempt to sync. What sort of trouble shooting information would be helpful for identifying a possible cause? Or is this the nature of the patch and I shouldn't expect the fields to be obtained from LDAP unless a user logs in? By this I mean that when a user logs in all the fields are properly populated from LDAP, only when I use sync is it skipping any additional fields.

sa1nt’s picture

Regarding post #89.

I'm having same problem with LDAPsync.

In my case, the module cannot find the ldapgroups_user_login function, which is very strange because NetBeans finds it (it appears in the autocomplete list).

To check this I modified the ldapsync.module file like this:

Before:

    // update user's groups if ldapgroups is enabled
    if (function_exists('ldapgroups_user_login')) {
      ldapgroups_user_login($account);
    }

After:

    // update user's groups if ldapgroups is enabled
    if (function_exists('ldapgroups_user_login')) {
      ldapgroups_user_login($account);
    }
    else{
        watchdog('ldapsync', t('function ldapgroups_user_login not found, unable to synchronize groups'));
    }

Then the watchdog message appeared in the Drupal log.

I would appreciate any help regarding this, as I have no idea why this is happening.

Thanks in advance.

bomarmonk’s picture

Darn, the PHP extension called "php_ldap" is not available to 1&1 shared hosting customers....

urbanmac’s picture

I have this installed on 6.16.

On clicking sync now after about 10 seconds my browser goes to a blank white page. No users are synced.

urbanmac’s picture

I got this working by upping the Resource Limits in the php.ini, something like:

max_execution_time = 360 ; Maximum execution time of each script, in seconds
max_input_time = 360 ; Maximum amount of time each script may spend parsing request data
memory_limit = 256M ; Maximum amount of memory a script may consume (8MB)

Only pulls in 950 users though, should be around 4000.

lavamind’s picture

Tested LDAPSync 0.8 with LDAP integration 1.0beta2 and seems to work well. I found one problem : when sync'ing, the user's role isn't added. There seems to be an include missing somewhere, because it seems that if (function_exists('ldapgroups_user_login')) { always returns false, regardless if ldapgroups is enabled of not.

Adding include_once(drupal_get_path('module', 'ldapauth') .'/ldapgroups.inc'); just before that line seems to fix the problem : sync'ing now correctly attributes the user's role. However this include could be placed somewhere better...

BTMash’s picture

In response to post #93,

I had run into the same issue with hitting a limit of 1000 users from AD. I found out that AD has a limit of returning on 1000 users per query and it may seem like either you set AD to return more than 1000 users or the ldap module has to modify the query to page through the results (the dev branch of PHP actually resolves this issue but I'm not sure if anyone would be up for testing that out or think its wise to have it up no a production site).

verta’s picture

I haven't tried it with the latest beta, but when I did try this module, it did not return all of our users. (We have a LOT.) It cut off somewhere in the Cs or Ds as I recall.

I also got a lot of system accounts (with a $ sign in the userid) which I would like to exclude, and might need to be excluded just on general design principles. I was unable to find a tool for mass deleting of users, and I'd prefer to not go at the database with SQL deletes - therefore I need to find a mass delete tool before if I can find myself testing this again.

I'll give it a try when there is a new dev release.

lavamind’s picture

@BTMash: I've tried the PHP ldap module patches to enable LDAP results paging, but on another project (Moodle). It works well but the PHP code that connects to LDAP needs to create and manage the connection in a different way. So as long as PHP hasn't officially include that functionality, it should be supported by this module. Your only option right now is to change you LDAP structure to include less accounts per OU, or configure AD to return more results (but be careful of memory usage on your server).

@verta: You can exclude accounts based on different criteria using the LDAP filter option provided by LDAPsync.

lavamind’s picture

Status: Active » Reviewed & tested by the community

Marking as reviewed. This module does have some minor flaws but in my case, and other's, it seems to be working well enough.

I think that including this release in the development branch would provide encouragement for developers in general to look deeper into LDAPsync and volunteer improvement patches.

kassissieh’s picture

Just FYI, I recently received my CVS credentials and plan to release this as its own module soon. I just need to learn the CVS system (first-timer) and find some time to do this. Thanks all for your support and comments.

Richard

lavamind’s picture

Is there any specific reason why it can't be integrated into this project (LDAP integration) ?

One of the maintainers of LDAP integration, miglius, said a year ago that it was a possibility. Does the offer still stand ?

In my opinion, it would be a better idea than starting a separate project with different issue tracker / releases / documentation / etc.

kassissieh’s picture

I recall miglius saying in a more recent exchange that he would prefer it reside in a separate module, like the ldap_provisioning module, though I can't find the actual text of that exchange, either.

verta’s picture

As a non-developer webmaster, I'd rather see this in one module. Drupal's modular architecture opens it up to great innovation, but also makes it a fragmented platform, with the bits all at different revision levels, support levels, and integration levels.

To those using LDAP, the authentication, role sync and user sync are just facets of our need to interface to a corporate authentication system. But I do realize it's complicated for the programmers.

kassissieh’s picture

I am happy to go either way. It's really up to miglius, so I'd recommend sending him the request. I'll hold off creating a project page until I hear the outcome, or a couple of weeks pass, whichever comes first!

roball’s picture

miglius has already offered to include this sub module in his main package, on May 8, 2009, here: http://drupal.org/node/376974#comment-1568612 (@ #376974: Next release). Also would prefer to not need so much separated modules.

miglius’s picture

I have looked at the code of the ldapsync-6.x-0.8.gz version in the #81 comment and have several minor comments:

1. I don't like the idea of using hardcoded big numbers as the time interval which will never be reached, 999999 as "only sync manually". This reminds me a Y2K issues... Could you please change this number to for instance -1 and then change the code logic accordingly?

2. Please don't use $GLOBALS, use, for instance a $_ldapsync_ldap->getOption('mail_attr') instead.

3. if (($status & 2) != 0) - do you really need a bit operation here? Would ($status != 1) not be sufficient?

4. Nice to have. ldappauth supports a "PHP to transform login name: ", would be nice to have a configuration option for a reverse transformation in the ldapsync module.

kassissieh’s picture

FileSize
4.57 KB

#105 (miglius)
1. Done
2. Done
3. MS Active Directory stores a number of different statuses in this variable. Bit 2 determines whether the account is enabled. We only want to check for that.
4. Not done, but I'm happy to take a look when I have more time.

** Please see new version attached **

#90 (sa1nt)
I've added the include code for ldapgroups. Not sure when and why this broke. Thanks for the suggestion.

#84 (jeff.k)
Fixed the problem where one had to sync twice to get roles

FYI, ldapdata integration is not yet enabled. That's why you won't get ldapdata sync. Some trial code exists in the module at line 198 but is disabled.

I also improved the blocking and unblocking of users based on their LDAP account status.

vetagrosup’s picture

Very great job, more than 1000 users added in few seconds!

miglius’s picture

Status: Reviewed & tested by the community » Fixed

I have cleaned the code a little according to the drupal coding standard and committed it to the cvs. Please check that the committed version works as expected.

Thanks all for contributing to this issue.

vetagrosup’s picture

"ldapdata integration is not yet enabled"
OK, do you know when it can be enabled?

2nd question : I work with multiples LDAP Server, but LdapSync update only data with the last one in the list in the configuration panel of LdapAuth, I try to make a cycle like that :

  $result = db_query("SELECT sid FROM {ldapauth} WHERE status = '1' ORDER BY sid");
  while ($row = db_fetch_object($result)) {

  $ldap_users = _ldapsync_search($row->sid);
}
function _ldapsync_search($sid) {

  global $_ldapsync_ldap;

  // Cycle through LDAP configurations.
  $result = db_query("SELECT sid FROM {ldapauth} WHERE status = '1' where sid='".$sid."'");
  while ($row = db_fetch_object($result)) {

But unsuccessfully

hawleyal’s picture

vetagrosup’s picture

It's ok, I just moved this line
$users=array();
outside the Cycle through LDAP configurations.
And now I trace all my Ldap Servers, but with 2800 users I take about 4 minutes

I'm going to work on ldapdata now

roball’s picture

miglius, great that this module has now become a part of the official main LDAP module - thanks! Do you also plan to committ it to the DRUPAL-6--1 branch?

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

roball’s picture

Status: Closed (fixed) » Active

Can we see it also in 6.x-1.x-dev?

syaman’s picture

Hi I have extracted the contents of ldapsync_0.8.1.gz into my "sites/all/modules/ldap_integration/" directory of my Drupal 6.16 install

However, I get this error message at admin/build/modules/list

"warning: file_get_contents(sites/all/modules/ldap_integration/ldapsync.info) [function.file-get-contents]: failed to open stream: Permission denied in /var/www/html/htdocs/includes/common.inc on line 3568."

vincetingey’s picture

Double check the file permissions for the module directory. Do a recursive permissions change "chmod 755 -R /sites/all/modules/ldap_integration/". If 755 doesnt work you could try 777.

jeff.k’s picture

Great Thanks for the fix

I have tried the latest version and it gives me the WSOD. Reverting to Synchronization 6.x-0.7 or 6.x-0.8 works totally fine.

Any idea what might cause that in this new version?

Update: I went to LDAP Integration HEAD and it seems to be resolved

http://ftp.drupal.org/files/projects/ldap_integration-HEAD.tar.gz

johan.mach’s picture

Hi all,

Latest LDAP Integration suite is working great for me - including the LDAP sync component over about 850 users.

To #109, I got all my users and their profile fields mapped with ldapdata all synced by temporarily switching ldapdata to "sync every time a user object is loaded in Drupal". As LDAP sync goes through and checks all the users, it has to load the user object, so this forces ldapdata to do its thing.

I'm not sure if you need to sync twice for unregistered users or not (I did a sync without and then with "sync every time..blah" and it worked).

Obviously, you don't want to leave ldatdata's settings like that forever as it will kill your LDAP performance eventually.

Thanks to the devs for this great set of modules, so far they have done everything I wanted of them.

Cheers,
Johan

dgeo’s picture

I do not understand well... This module seems to do quite the same thing as http://drupal.org/project/ldap_provisioning ...

We are looking for a solution tha could fetch user's ldap informations at login time (I prefer not create/synchronize ~1500 users * 10 drupal instances every night, while only 30 or 40 of them will log in into drupal)
We do NOT user ldapauth for authentication (SSO here), but we'd like to use ldapdata and ldapgroup...

Would it be possible to sync one account at login time with ldapsync (the way ldapdata does, but working without ldapauth auth.) ?

mtbradle’s picture

FileSize
923 bytes

LDAP sync worked great for me, but I was running into the same question as #109 concerning ldapdata synchronization. #118 's solution helped but was impractical as it is not automated, and the settings of ldapdata would have to be changed every time I needed to sync. Which wouldn't work if my ldapsync was running from cron.

Made this patch to get ldapsync to sync the data from ldapdata as well. Basically does what #118 said, which is force a user load when syncing the users. But this way, you don't have to change your ldapdata settings to "sync every time a user object is loaded in Drupal" everytime you want to sync.

ashtonium’s picture

Version: 6.x-1.x-dev » master
Status: Active » Fixed
Issue tags: +ldapsync

Further issues with the ldapsync component should probably be filed on their own issues. I believe that this issue is fixed because the code has been committed to the HEAD branch (and hopefully 6.x-dev soon).

I've started a couple ldapsync issues over here: #867284: ldapsync - error in querry to re-enable blocked Drupal accounts and here: #867356: ldapsync - synchronize accounts ldapdata as well so please test those out and comment on them.

The second one is obviously a slightly modified version of mtbradle's patch in #120 (@mtbradle: the .module files are always loaded if the module is enabled, so there's no need for that part.)

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

gettysburger’s picture

Post #121 contains a patch to update an LDAPSYNC module. Which module will it update? Thanks. I am using ldap Integration 6.x-1.0-beta2. Thanks.

bomarmonk’s picture

"Further issues with the ldapsync component should probably be filed on their own issues. I believe that this issue is fixed because the code has been committed to the HEAD branch (and hopefully 6.x-dev soon)." This means, I believe, that the code for this should be committed to the head or development version of this module, so I think you can download the appropriate version, rather than worrying about any patching.

pembertona’s picture

Sorry; what is the status of the LDAPSync module? Is it available as an independent module? Or only through this issue thread in the LDAP Integration project?

lavamind’s picture

The ldapsync component has been committed to the module's CVS repository. If you want a copy of ldap_integration with the latest ldaysync, you need to checkout a copy of the code through CVS : http://drupal.org/node/15109/cvs-instructions/HEAD

zeezhao’s picture

Thanks. Would be great if ldap_integration-6.x-1.x-dev can be refreshed so that the patch makes it in...

Renee S’s picture

(edit: removed. wrong issue. :)

ssudahar’s picture

Assigned: kassissieh » Unassigned
Dimas_’s picture

I can't apply the patch (#120, #121), but LDAP Sync doesn't synchronize data for me. Is there any renewed patch? Thx

toralpatelin’s picture

Is there anything similar for drupal 7? I need to import all the users at a time from LDAP. This will not be for authentication or login purpose. I am gonn use NTLM authentication kind of thing. There wont be any login screen in my site.

puddyglum’s picture

This patch resolves the Data updating issue when doing a sync.

#1865636: profile fields not syncing

evocage’s picture

Hi ,

I want to import drupal users to Openldap that ldap is completely new and there is no data. Can any one help me to do the same. Running LDAPSync but getting : Completed LDAP sync. LDAP users found: 0. Existing users updated: 0. New users created: 0. LDAP-authentified users that do not have an enabled LDAP account: 0.

I think this module is available to sync from ldap to drupal users.