Due to a longstanding bug in the PHP language it is not possible to access more than 1,000 AD objects in a single search. This has a dramatic impact when LDAP is enabled with the block users option. Any users not returned in the first 1,000 results from a server will be blocked.

I recommend that the block option have some help text warning of this particular issue.

Comments

johnbarclay’s picture

johnbarclay commented
Category: task » bug

Yes. That would help. But that bug in ldap can be worked around by paging and batch processing, so I would say this is a bug. Thanks for pointing this out. Its an issue I need to revolve in project/ldap for drupal 7.

antgiant’s picture

antgiant commented

The bug cannot be worked around in PHP without resorting to direct COM calls or recompiling the php/ldap extension with a minor patch applied. The actual required paging functions will be available in PHP 5.4. However, there is not yet a release version of PHP 5.4, and I suspect that Drupal will not require or even work reliably with PHP 5.4 for a while after that.

Having said that I would love for the LDAP module to natively support paging and batch processing if I'm running PHP 5.4 or a patched extension.

johnbarclay’s picture

johnbarclay commented
Title: LDAP Sync limited to 1,000 AD users per server » LDAP Sync limited to 1,000 AD users per server. Add user warning for admins.
Category: bug » task

makes sense. I'm tagging this as a task again.

antgiant’s picture

antgiant commented

Additional note. If your AD usercount is any where near 1,000 LDAP sync will never complete because PHP will timeout after 2 minutes on Cron. I'm at 2,000+ users spread over 4 servers. Typical sync run takes nearly 18 minutes. The default of 1 minute when clicking the manual run actually syncs about 2 dozen accounts before timing out. (The bulk of that first minute is spent querying the LDAP server.)

antgiant’s picture

antgiant commented

PHP 5.4 is officially out if you want to implement the paging and batch processing. :-)

johnbarclay’s picture

johnbarclay commented

the way I'm dealing with this in the Drupal 7 ldap is:

- store last ldap synch in user attached field
- combine ldap synch, ldap provision, ldap profile, and ldap authentication provisioning functionality into a single ldap user module.
- do synch/update/provision batch query based on last updated order and query # specified in admin so it can be tailored to server limitations.
- add pagination to the ldap server module with hacks for pre 5.4 and ldap pagination for post 5.4

A simple fix for the 6 modules might be:
-- add a last ldap modified date in a queriable location, or always save user object on ldap synch and use the user record last modified date.
-- specify the number of users to update each cron and do them in order of farthest modification date.
-- this doesn't account for manually changed user accounts and doesn't address provisioning queries.

cgmonroe’s picture

cgmonroe commented
Status: Active » Fixed

Marking this as fixed since the changes just committed fix the main issue here by allowing multiple search filters to be defined. The description for this talks briefly about the AD limits.

In addition, there have been a fair number of memory and performance changes made... but timeout for large amounts of users may still occur and there is no restart tracking yet.

I'll probably open a new issue / task specifically for that.

For details of the ldapsync changes see: #1475272: 6.x-1.0 Release Candidate 1 Status

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.