Handling Patched Modules

Hi, I was just wondering if there was an accepted procedure for handling patched modules? In other words, should you place them in a "patched" directory under "sites/all/modules" for example to keep them from being inadvertently overwritten in the future if downloading another version of the module?

I have patched a few modules to add features that were missing and I would hate to accidentally overwrite them and lose the changes. I may forget which modules were patched to begin with.

Thanks