Just a word of caution. In combination with the entitycache module, there are situations where terms that you are not allowed to list will still be returned when your code does a taxonomy_term_load or entity_load.
It took me a slight while to find that out.