Very long Access Denied and Page Not Found page generation times

My logs are reporting these average page generation times:

Page Not Found page_not_found 7131 ms
and
Access Denied access_denied 18660 ms

But when I try this the times are normal, under 1 sec.

What might be happening? How can I check to see when these long page loads are occurring? If those are averages, there must be some really long load times in there as well.

I have a couple thousand path_redirect entries. I'm guessing that may be the problem on page not found.

Thanks.