Are you running a pre-release version of Bundler? This commit might be related to your problem; it looks like earlier versions of Bundler scan your %PATH% for "git", but not "git.exe".

The easiest solution would be to backport bundler to 1.3.5:

gem uninstall bundler
gem install bundler