Is there for drupal a sitemap module that crawl the web site doing the same as the Google crawler. It crawls your drupal site and stores all the urls in a XML file.
for joola there is an opensource plugin like this and this is very useful and simple to use:
http://extensions.joomla.org/component/option,com_mtree/task,viewlink/li...
in case of joomla, it is useful for example to crawl profiles coming from civicrm.org that for sure are not in sync with urls of normal drupal profiles