Add User Delete module permission that explicitly grants access to User Delete settings

I want to give a user access to manage users (add, edit, delete). To do so, I grant "administer permissions" under the user module. However, doing so also grants the user access to the User Delete module settings (/admin/user/user_delete). I would like to have an explicit permission under the User Delete module that grants access to the User Delete settings page. "administer permissions" should not be granting access to the User Delete settings page.