Deleting orphan terms especially in a free tagging vocabulary is reasonably common task for site administrators. Orphan terms in this case means taxonomy terms that are not used by any nodes (content pieces). I had to look up various issue queues for this and found this in the issue queue of the term node count module. If interested here is the link to the page #603868: Rework Term Node Count for D7
Steps to get this working:

1. Create a new view. Name it anything you want.
2. Display: choose taxonomy term. If you want to delete orphan tags from a particular vocabulary, choose the vocabulary. Add the page title etc and click create and edit to move to the next page
3. In the section Relationships choose "Taxonomy Term: Content with Term" from the list of options. Leave the default identifier "node" or choose a descriptive label if you wish.
4. In the Fields section add Content: nid and Bulk Operations: Taxonomy Term. Label the Content: nid field with a descriptive label such as "node count". For the Bulk Operation field type in label as "select" . Ensure enable select all items and display processing result are checked . In the selected operation choose the delete operation. Check the delete without confirmation if you want to delete the terms without an intermediate confirmatory dialog page.
5. On the left side in the others section where it is written use aggregation click on the no and toggle it to yes.
6. Click on the aggregation settings beside the Node:nid filter and from the aggregation type drop down choose COUNT. What this will do is it will give a count of all nodes associated with the specific term.
7. Now the only thing left to do is add a filter criteria as Content: nid and choose operator is equal to 0

There you have it a view bulk operation page that allows you to delete the orphan taxonomy terms easily. You can delete all terms in one fell swoop. For better security choose the access setting and change to a higher role or permission so that unwanted users don't delete the terms. You can easily add this link to a menu and do the delete operations at your leisure. Mind you if you are using a hierarchial vocabulary it is probably best to keep a way to remove the top level terms from the list so that the tree remains intact. However I havent found a way of getting that done.