且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Beautifulsoup:.find()和.select()之间的区别

更新时间:2023-12-04 11:39:40

总结评论:

  • select 查找多个实例并返回一个列表, find 查找第一个实例,因此它们不会执行相同的操作. select_one 将等同于 find .
  • 如果要查找不带类的单个元素,我通常会使用 find 来链接标签或使用 tag.classname 时使用css选择器.从本质上讲,这取决于用例和个人喜好.
  • 就灵活性而言,我认为您已经知道答案了,使用多个链接的 find/find_all 调用,soup.select("div[id=foo] > div > div > div[class=fee] > span > span > a")看起来很难看.
  • bs4中css选择器的唯一问题是非常有限的支持, nth-of-type 是唯一实现的伪类,并且也没有像a [href] [src]这样的链接属性与CSS选择器的许多其他部分一样受支持.但是诸如 a [href = ..] *, a [href ^ =] a [href $ =] 之类的东西都是我认为比find("a", href=re.compile(....))好很多,但这又是个人喜好.
  • select finds multiple instances and returns a list, find finds the first, so they don't do the same thing. select_one would be the equivalent to find.
  • I almost always use css selectors when chaining tags or using tag.classname, if looking for a single element without a class I use find. Essentially it comes down to the use case and personal preference.
  • As far as flexibility goes I think you know the answer, soup.select("div[id=foo] > div > div > div[class=fee] > span > span > a") would look pretty ugly using multiple chained find/find_all calls.
  • The only issue with the css selectors in bs4 is the very limited support, nth-of-type is the only pseudo class implemented and chaining attributes like a[href][src] is also not supported as are many other parts of css selectors. But things like a[href=..]* , a[href^=], a[href$=] etc.. are I think much nicer than find("a", href=re.compile(....)) but again that is personal preference.

为了提高性能,我们可以运行一些测试,我修改了在此处回答的代码,该代码在800多个html文件上运行摘自此处,虽然不详尽,但应提供可读性线索一些选项和性能:

For performance we can run some tests, I modified the code from an answer here running on 800+ html files taken from here, is is not exhaustive but should give a clue to the readability of some of the options and the performance:

修改后的功能为:

from bs4 import BeautifulSoup
from glob import iglob


def parse_find(soup):
    author = soup.find("h4", class_="h12 talk-link__speaker").text
    title = soup.find("h4", class_="h9 m5").text
    date = soup.find("span", class_="meta__val").text.strip()
    soup.find("footer",class_="footer").find_previous("data", {
        "class": "talk-transcript__para__time"}).text.split(":")
    soup.find_all("span",class_="talk-transcript__fragment")



def parse_select(soup):
    author = soup.select_one("h4.h12.talk-link__speaker").text
    title = soup.select_one("h4.h9.m5").text
    date = soup.select_one("span.meta__val").text.strip()
    soup.select_one("footer.footer").find_previous("data", {
        "class": "talk-transcript__para__time"}).text
    soup.select("span.talk-transcript__fragment")


def  test(patt, func):
    for html in iglob(patt):
        with open(html) as f:
            func(BeautifulSoup(f, "lxml")

现在开始计时:

In [7]: from testing import test, parse_find, parse_select

In [8]: timeit test("./talks/*.html",parse_find)
1 loops, best of 3: 51.9 s per loop

In [9]: timeit test("./talks/*.html",parse_select)
1 loops, best of 3: 32.7 s per loop

就像我说的不是详尽无遗,但我认为我们可以肯定地说CSS选择器肯定更有效.

Like I said not exhaustive but I think we can safely say the css selectors are definitely more efficient.