网站提交按钮rendering js or ajax request.您应该尝试使用自动化selenium库.它可以让您剪贴动态的rendering request(js or ajax)页面数据.

website submit button rendering js or ajax request. You should try automation selenium library. it allows you to scrap dynamic rendering request(js or ajax) page data.

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Chrome('/usr/bin/chromedriver')
browser.get('https://www.icsi.in/student/Members/MemberSearch.aspx')

reg = browser.find_element_by_name('dnn$ctr410$MemberSearch$txtCpNumber')
reg.send_keys('16803')

sub = browser.find_element_by_class_name('dnnPrimaryAction')
sub.click()

WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "rgMasterTable")))

soup = BeautifulSoup(browser.page_source, 'lxml')
table = soup.find("table",{'class':"rgMasterTable"}).find("tr",{'class':"rgRow"})

data = {}
for div in table.find_all("div",{'class':"chart_att"}):
    for div2 in div.find_all("div"):
        _class = div2.get("class")

        if "chart_row" in _class[0]:
            key = None
            value = None
            for td in div2.find_all("td"):
                _class1 = td.get("class")

                if "chart_head" in _class1[0]:
                    key = td.text.strip()
                else:
                    value = td.text.strip()
            if key is not None and value is not None:
                data[key] = value

print(data)

O/P:

{'Organization': 'RAHUL SHINDE AND COMPANY', 'Designation': 'COMPANY SECRETARIES (*)', 'Membership Number': 'A32412', 'CP Number': '16803', 'Benevolent Member': 'No', 'Address': '25/26, 3RD FLOOR, PAREERA BUILDING NAVJEEVAN WADI, KALBADEVI POST DHOBI TALAV, MARINE LINES', 'City': 'MUMBAI', 'Phone': '', 'Email': 'jurisrahul@gmail.com', 'Mobile': '8369683685'}

其中'/usr/bin/chromedriver'硒Web驱动程序路径.

where '/usr/bin/chromedriver' selenium web driver path.

下载适用于Chrome浏览器的Selenium Web驱动程序:

http://chromedriver.chromium.org/downloads

为Chrome浏览器安装Web驱动程序:

https://christopher.su/2015/selenium-chromedriver-ubuntu/

硒教程:

https://selenium-python.readthedocs.io/