It has been narrowed down now to specifically when the cron runs. Everytime cron runs about a dozen new anonymous users get created. I have played around with very little response.
I am sure now it is some how releated to this notice I get. (Note I get a few with different offset numbers from about 3 to 15)
Notice: Undefined offset: 15 in _ldap_user_orphans() (line 156 of /var/www/html/sites/all/modules/ldap/ldap_user/ldap_user.cron.inc).
Each one of these appears to be an anonymous user that gets created.
I am also getting the following message which I am not sure if it is connected.
Notice: unserialize(): Error at offset 108 of 140 bytes in UserController->attachLoad() (line 307 of /var/www/html/modules/user/user.module).
I have had a look at it and will keep looking but if anyone can see something I am missing about this then that would be great. It gets really annoying constantly having to delete all the anonymous users. They do come blocked but there is no point in having them.