且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Scrapy:Google 抓取不起作用

更新时间:2023-08-22 22:36:52

是的,看起来那个地址正在重定向到主页:

Yes, looks like that address is redirecting to the home page:

带有 scrapy shell 的示例 http://www.google.com/#q=finance.google.com:+3m+co:

...
[s]   request    <GET http://www.google.com/#q=finance.google.com:+3m+co>
[s]   response   <200 http://www.google.com/>
...

检查您的 url 是有道理的,它不包含参数,但是 #q(不是 url 参数)和浏览器是识别这一点并使其成为谷歌搜索的浏览器,所以它不完全是一个 url 路径.

Checking your url it makes sense, it isn't containing parameters, but #q (which isn't a url parameter) and the browser is the one recognizing that and making it a google search, so it is not exactly a url path.

正确的谷歌搜索网址是:http://www.google.com/search?q=YOURQUERY

the correct google search url is: http://www.google.com/search?q=YOURQUERY