Giving permission to add users, but only view user's with certain roles.

I'm trying to setup a site that allows certain users with the applied role of say account manager to add user's, but only edit user's with no assigned role or user's with assigned role of X.

Currently I have the Role Delegation module installed, which works very well. I have a role called Account Manager which has been given permission to add users, but only assign four different types of roles, which all have no abilities to edit or manager content. However the user is still able to login and view all accounts under the List tab when going to add user regardless of the role they have assigned. Ideally I would like it so that they are only able to see user's with roles which they have the permission to assign.

Does anyone have any ideas? Or could possibly just get me started in the right direction?

Thanks in advance.