OK, I figured it out myself:

The spec file I was running had a "require 'cucumber_rails'". I don't know if and why I put that there or if it was put there by some script. I removed this and now everything works.

I am not sure why this caused a problem, but now it works. If someone knows why this causes trouble, I would be happy about an explanation.