When using the input format "Full HTML", the admin links appears also for anonymous users but they have no access. I could get rid of the admin links by changing Full html to filtered html.
I don't know why this is.
Anonymous users have only access to content, none for editing. Please correct me if this is not a bug.
Drupal core 6.19
Admin links 6.x-1.8