有没有一种 Python 方法可以尝试最多次数?

is there a pythonic way to try something up to a maximum number of times?

提问人:Ben 提问时间:2/20/2009 最后编辑:Ben 更新时间:7/30/2022 访问量:80561

问:

我有一个 python 脚本,它正在查询共享 linux 主机上的 MySQL 服务器。出于某种原因,对MySQL的查询经常返回“服务器已消失”错误:

_mysql_exceptions.OperationalError: (2006, 'MySQL server has gone away')

如果之后立即再次尝试查询,通常会成功。所以,我想知道 python 中是否有一种明智的方法来尝试执行查询,如果失败,可以再试一次,最多尝试固定次数。可能我希望它尝试 5 次,然后完全放弃。

这是我拥有的代码类型:

conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()

try:
    cursor.execute(query)
    rows = cursor.fetchall()
    for row in rows:
        # do something with the data
except MySQLdb.Error, e:
    print "MySQL Error %d: %s" % (e.args[0], e.args[1])

显然,我可以通过在 except 子句中再次尝试来做到这一点,但这非常丑陋,而且我有一种感觉,必须有一种体面的方法来实现这一目标。

Python 异常

评论

2赞 Ben 2/20/2009
这是一个很好的观点。我可能会睡几秒钟。我不知道服务器上的MySQL安装有什么问题,但似乎它确实在一秒钟失败了,下一秒就起作用了。
3赞 jfs 2/20/2009
@Yuval 答:这是一项常见的任务。我怀疑它甚至内置在 Erlang 中。
1赞 andy 5/12/2016
只是说也许没有什么问题,Mysql有一个wait_timeout变量来配置mysql以删除非活动连接。
0赞 Karl Knechtel 7/30/2022
@MadPhysicist我以另一种方式指出了重复的闭包,因为从我能想到的每个指标来看,这个版本的问题对我来说显然更好。

答:

133赞 Dana 2/20/2009 #1

怎么样:

conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
attempts = 0

while attempts < 3:
    try:
        cursor.execute(query)
        rows = cursor.fetchall()
        for row in rows:
            # do something with the data
        break
    except MySQLdb.Error, e:
        attempts += 1
        print "MySQL Error %d: %s" % (e.args[0], e.args[1])

评论

27赞 cdleary 2/20/2009
for attempt_number in range(3)
10赞 Dana 2/20/2009
好吧,我有点喜欢我的,因为它明确表示只有在发生异常时才会增加尝试次数。
2赞 cdleary 2/20/2009
是的,我想我比大多数人更偏执无限循环。while
7赞 S.Lott 2/20/2009
-1:不喜欢休息。比如“虽然没有完成并尝试< 3:”更好。
5赞 hasen 2/25/2009
我喜欢休息,但不喜欢休息。这更像是 C-ish 而不是 pythonic。因为我在范围内更好恕我直言。
6赞 cdleary 2/20/2009 #2

我会像这样重构它:

def callee(cursor):
    cursor.execute(query)
    rows = cursor.fetchall()
    for row in rows:
        # do something with the data

def caller(attempt_count=3, wait_interval=20):
    """:param wait_interval: In seconds."""
    conn = MySQLdb.connect(host, user, password, database)
    cursor = conn.cursor()
    for attempt_number in range(attempt_count):
        try:
            callee(cursor)
        except MySQLdb.Error, e:
            logging.warn("MySQL Error %d: %s", e.args[0], e.args[1])
            time.sleep(wait_interval)
        else:
            break

分解函数似乎会分解功能,以便很容易看到业务逻辑,而不会陷入重试代码中。callee

评论

0赞 S.Lott 2/20/2009
-1:否则,打破...讨厌。更喜欢更清晰的“while not done and count != attempt_count”而不是 break。
1赞 cdleary 2/20/2009
真?我认为这样更有意义——如果没有发生异常,就跳出循环。我可能过于害怕无限 while 循环。
4赞 xorsyst 10/5/2011
+1:我讨厌当语言包含代码结构来为你做这件事时的标志变量。对于奖励积分,请添加其他选项以处理所有尝试失败的情况。
83赞 dwc 2/20/2009 #3

基于 Dana 的回答,您可能希望以装饰者的身份执行此操作:

def retry(howmany):
    def tryIt(func):
        def f():
            attempts = 0
            while attempts < howmany:
                try:
                    return func()
                except:
                    attempts += 1
        return f
    return tryIt

然后。。。

@retry(5)
def the_db_func():
    # [...]

使用该模块的增强版本decorator

import decorator, time

def retry(howmany, *exception_types, **kwargs):
    timeout = kwargs.get('timeout', 0.0) # seconds
    @decorator.decorator
    def tryIt(func, *fargs, **fkwargs):
        for _ in xrange(howmany):
            try: return func(*fargs, **fkwargs)
            except exception_types or Exception:
                if timeout is not None: time.sleep(timeout)
    return tryIt

然后。。。

@retry(5, MySQLdb.Error, timeout=0.5)
def the_db_func():
    # [...]

要安装装饰器模块:

$ easy_install decorator

评论

2赞 cdleary 2/20/2009
装饰器可能也应该采用异常类,因此您不必使用裸例外;即 @retry(5, MySQLdb.Error)
0赞 Dana 2/20/2009
漂亮!我从没想过要用装饰器:P
0赞 Robert Rossney 2/20/2009
这应该是“return func() in the try block”,而不仅仅是“func()”。
0赞 dwc 2/20/2009
呸!谢谢你的提醒。
0赞 Steve Losh 2/20/2009
你真的尝试过运行这个吗?这是行不通的。问题在于,tryIt 函数中的 func() 调用会在您修饰函数时立即执行,而不是在实际调用修饰函数时执行。您需要另一个嵌套函数。
8赞 webjunkie 2/20/2009 #4
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()

for i in range(3):
    try:
        cursor.execute(query)
        rows = cursor.fetchall()
        for row in rows:
            # do something with the data
        break
    except MySQLdb.Error, e:
        print "MySQL Error %d: %s" % (e.args[0], e.args[1])

评论

1赞 Bob Stein 8/29/2015
您可以在底部添加其他内容:else: raise TooManyRetriesCustomException
7赞 Kiv 2/22/2009 #5

像 S.Lott 一样,我喜欢一个标志来检查我们是否完成了:

conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()

success = False
attempts = 0

while attempts < 3 and not success:
    try:
        cursor.execute(query)
        rows = cursor.fetchall()
        for row in rows:
            # do something with the data
        success = True 
    except MySQLdb.Error, e:
        print "MySQL Error %d: %s" % (e.args[0], e.args[1])
        attempts += 1
0赞 user1970198 1/11/2013 #6

这是我的通用解决方案:

class TryTimes(object):
    ''' A context-managed coroutine that returns True until a number of tries have been reached. '''

    def __init__(self, times):
        ''' times: Number of retries before failing. '''
        self.times = times
        self.count = 0

    def __next__(self):
        ''' A generator expression that counts up to times. '''
        while self.count < self.times:
            self.count += 1
        yield False

    def __call__(self, *args, **kwargs):
        ''' This allows "o() calls for "o = TryTimes(3)". '''
        return self.__next__().next()

    def __enter__(self):
        ''' Context manager entry, bound to t in "with TryTimes(3) as t" '''
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        ''' Context manager exit. '''
        return False # don't suppress exception

这允许如下所示的代码:

with TryTimes(3) as t:
    while t():
        print "Your code to try several times"

也可能:

t = TryTimes(3)
while t():
    print "Your code to try several times"

我希望可以通过以更直观的方式处理异常来改善这一点。乐于接受建议。

1赞 Peter Wood 3/27/2015 #7
def successful_transaction(transaction):
    try:
        transaction()
        return True
    except SQL...:
        return False

succeeded = any(successful_transaction(transaction)
                for transaction in repeat(transaction, 3))
16赞 Elias Dorneles 6/23/2015 #8

更新:重试库有一个维护得更好的分支,称为 tenacity,它支持更多功能并且通常更灵活。

API 略有变化:

@retry(stop=stop_after_attempt(7))
def stop_after_7_attempts():
    print("Stopping after 7 attempts")

@retry(wait=wait_fixed(2))
def wait_2_s():
    print("Wait 2 second between retries")

@retry(wait=wait_exponential(multiplier=1, min=4, max=10))
def wait_exponential_1000():
    print("Wait 2^x * 1000 milliseconds between each retry,")
    print("up to 10 seconds, then 10 seconds afterwards")

是的,有重试,它有一个装饰器,可以实现几种可以组合的重试逻辑:

一些例子:

@retry(stop_max_attempt_number=7)
def stop_after_7_attempts():
    print("Stopping after 7 attempts")

@retry(wait_fixed=2000)
def wait_2_s():
    print("Wait 2 second between retries")

@retry(wait_exponential_multiplier=1000, wait_exponential_max=10000)
def wait_exponential_1000():
    print("Wait 2^x * 1000 milliseconds between each retry,")
    print("up to 10 seconds, then 10 seconds afterwards")

评论

2赞 Seth 9/10/2017
重试库已被 tenacity 库取代。
1赞 user5637641 12/4/2015 #9

1.定义:

def try_three_times(express):
    att = 0
    while att < 3:
        try: return express()
        except: att += 1
    else: return u"FAILED"

2.用途:

try_three_times(lambda: do_some_function_or_express())

我用它来解析html上下文。

2赞 Mad Physicist 7/24/2020 #10

您可以使用带有子句的循环以获得最大效果:forelse

conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()

for n in range(3):
    try:
        cursor.execute(query)
    except MySQLdb.Error, e:
        print "MySQL Error %d: %s" % (e.args[0], e.args[1])
    else:
        rows = cursor.fetchall()
        for row in rows:
            # do something with the data
        break
else:
    # All attempts failed, raise a real error or whatever

关键是一旦查询成功,就要跳出循环。仅当循环在没有 .elsebreak