If only the admin (me) is allowed to upload any files do they still need to be outside web root for security. Does the danger come in the upload or download or both?

thanks