且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何使脚本在迭代中等待,直到重新建立Internet连接?

更新时间:2023-10-26 23:45:34

在其中编写另一个while循环,该循环将继续尝试连接到Internet.

只有当它收到状态码200时它才会中断,然后您才能继续执行程序.

喜欢的种类

retry = True
while retry:
    try:
        r = br.open(//your site)
        if r.getcode()/10==20:
            retry = False
    except:
          // code to handle any exception

// rest of your code

I have a scraping code within a for loop, but it would take several hours to complete, and the program stops when my Internet connection breaks. What I (think I) need is a condition at the beginning of the scraper that tells Python to keep trying at that point. I tried to use the answer from here:

for w in wordlist:

#some text processing, works fine, returns 'textresult'

    if textresult == '___':  #if there's nothing in the offline resources
        bufferlist = list()
        str1=str()
        mlist=list()  # I use these in scraping

        br = mechanize.Browser()

        tried=0
        while True:
            try:
                br.open("http://the_site_to_scrape/")

                # scraping, with several ifs. Each 'for w' iteration results with scrape_result string.


            except (mechanize.HTTPError, mechanize.URLError) as e:
                tried += 1
                if isinstance(e,mechanize.HTTPError):
                    print e.code
                else:
                    print e.reason.args
            if tried > 4:
                    exit()
                    time.sleep(120)
                    continue
            break

Works while I'm online. When the connection breaks, Python writes the 403 code and skips that word from wordlist, moves on to the next and does the same. How can I tell Python to wait for connection within the iteration?

EDIT: I would appreciate it if you could write at least some of the necessary commands and tell me where they should be placed in my code, because I've never dealt with exception loops.

EDIT - SOLUTION I applied Abhishek Jebaraj's modified solution. I just added a very simple exception handling command:

except:
    print "connection interrupted"
    time.sleep(30)

Also, Jebaraj's getcode command will raise an error. Before r.getcode, I used this:

import urllib

r = urllib.urlopen("http: the site ")

The top answer to this question helped me as well.

Write another while loop inside which will keep trying to connect to the internet.

It will break only when it receives status code of 200 and then you can continue with your program.

Kind of like

retry = True
while retry:
    try:
        r = br.open(//your site)
        if r.getcode()/10==20:
            retry = False
    except:
          // code to handle any exception

// rest of your code