There are additional configuration steps needed to get Drush to work with git bash. This is due to the Drush Windows Installer package not having a couple necessary configuration options enabled by default to provide proper integration with git bash.
Fresh Drush Installation
When installing drush using the Drush 5.7/5.8/6.0 Windows Installer, this screen will appear:
There are three boxes that are marked "X" (not installed).
Click the box titled "Register Environmental Variables", then choose "Entire feature will be installed on local hard drive".
Then, click the box titled "cwRsync Optional Component box", then choose "Entire feature will be installed on local hard drive". Enabling this option will (with a few other tweaks) enable you to use the drush rsync command.
Proceed with the installation.
The "Register Environment Variables" option will update the PATH so that git bash knows where to find drush.
Fixing An Existing Drush Installation