更新时间:2023-12-05 22:11:04
由于此解析作业需要提取文本和属性
值,它不能完全通过开箱即用"的功能来完成,例如
pd.read_html
.其中一些必须手动完成.
Since this parsing job requires the extraction of both text and attribute
values, it can not be done entirely "out-of-the-box" by a function such as
pd.read_html
. Some of it has to be done by hand.
使用 lxml ,您可以使用XPath提取属性值:
Using lxml, you could extract the attribute values with XPath:
import lxml.html as LH
import pandas as pd
content = '''
<table>
<tbody>
<tr>
<td>265</td>
<td> <a href="/j/jones03.shtml">Jones</a>Blue</td>
<td >29</td>
</tr>
<tr >
<td>266</td>
<td> <a href="/s/smith01.shtml">Smith</a></td>
<td>34</td>
</tr>
</tbody>
</table>'''
table = LH.fromstring(content)
for df in pd.read_html(content):
df['refname'] = table.xpath('//tr/td/a/@href')
df['refname'] = df['refname'].str.extract(r'([^./]+)[.]')
print(df)
收益
0 1 2 refname
0 265 JonesBlue 29 jones03
1 266 Smith 34 smith01
上面的内容可能有用,因为它只需要几个
额外的代码行来添加refname
列.
The above may be useful since it requires only a few
extra lines of code to add the refname
column.
但是LH.fromstring
和pd.read_html
都解析HTML.
因此,通过删除pd.read_html
和
用LH.fromstring
解析表一次:
But both LH.fromstring
and pd.read_html
parse the HTML.
So it's efficiency could be improved by removing pd.read_html
and
parsing the table once with LH.fromstring
:
table = LH.fromstring(content)
# extract the text from `<td>` tags
data = [[elt.text_content() for elt in tr.xpath('td')]
for tr in table.xpath('//tr')]
df = pd.DataFrame(data, columns=['id', 'name', 'val'])
for col in ('id', 'val'):
df[col] = df[col].astype(int)
# extract the href attribute values
df['refname'] = table.xpath('//tr/td/a/@href')
df['refname'] = df['refname'].str.extract(r'([^./]+)[.]')
print(df)
收益
id name val refname
0 265 JonesBlue 29 jones03
1 266 Smith 34 smith01