[python-committers] miss-islington backport pipeline is stalled?

[Mariatta]

I'm seeing more of this today, just heads up in case you see miss-islington
not working.

Seems like this is a known issue with celery + kombu 🙁
https://github.com/celery/kombu/issues/1019
ᐧ

On Wed, May 8, 2019 at 10:02 AM Mariatta <mariatta at python.org> wrote:

> There was an error from Redis. I think this is the first time I've seen
> it, so I don't have any resolution on how to fix it right now.  😟
> I will look into handling the error and have miss-islington leave a
> comment in the PR when there is such error.
>
> log:
>
> at=info method=POST path="/" host=miss-islington.herokuapp.com request_id=8de611a2-6112-4039-a425-97850ac989b0 fwd="140.82.115.15" dyno=web.1 connect=1ms service=1005ms status=200 bytes=168 protocol=https
> May 08 09:35:15 miss-islington app/web.1: Traceback (most recent call last):
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/connection.py", line 600, in send_packed_command
> May 08 09:35:15 miss-islington app/web.1:     self._sock.sendall(item)
> May 08 09:35:15 miss-islington app/web.1: TimeoutError: [Errno 110] Connection timed out
> May 08 09:35:15 miss-islington app/web.1: During handling of the above exception, another exception occurred:
> May 08 09:35:15 miss-islington app/web.1: Traceback (most recent call last):
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 431, in _reraise_as_library_errors
> May 08 09:35:15 miss-islington app/web.1:     yield
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/celery/app/base.py", line 744, in send_task
> May 08 09:35:15 miss-islington app/web.1:     self.backend.on_task_call(P, task_id)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/celery/backends/redis.py", line 265, in on_task_call
> May 08 09:35:15 miss-islington app/web.1:     self.result_consumer.consume_from(task_id)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/celery/backends/redis.py", line 126, in consume_from
> May 08 09:35:15 miss-islington app/web.1:     self._consume_from(task_id)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/celery/backends/redis.py", line 132, in _consume_from
> May 08 09:35:15 miss-islington app/web.1:     self._pubsub.subscribe(key)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/client.py", line 3096, in subscribe
> May 08 09:35:15 miss-islington app/web.1:     ret_val = self.execute_command('SUBSCRIBE', *iterkeys(new_channels))
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/client.py", line 3009, in execute_command
> May 08 09:35:15 miss-islington app/web.1:     self._execute(connection, connection.send_command, *args)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/client.py", line 3013, in _execute
> May 08 09:35:15 miss-islington app/web.1:     return command(*args)
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/connection.py", line 620, in send_command
> May 08 09:35:15 miss-islington app/web.1:     self.send_packed_command(self.pack_command(*args))
> May 08 09:35:15 miss-islington app/web.1:   File "/app/.heroku/python/lib/python3.6/site-packages/redis/connection.py", line 613, in send_packed_command
> May 08 09:35:15 miss-islington app/web.1:     (errno, errmsg))
> May 08 09:35:15 miss-islington app/web.1: redis.exceptions.ConnectionError: Error 110 while writing to socket. Connection timed out.
>
>
>
> ᐧ
>
> On Wed, May 8, 2019 at 9:44 AM Gregory P. Smith <greg at krypto.org> wrote:
>
>> Yesterday it failed to produce a backport or tell me that it couldn't
>> (after the "i'm now working on ..." message was left on the master PR).  I
>> waited a couple hours just in case and ran cherry_picker myself instead.
>> Same thing today apparently on
>> https://github.com/python/cpython/pull/13192 assuming the usual backport
>> latency is no more than a minute or two.
>>
>> -gps
>> _______________________________________________
>> python-committers mailing list
>> python-committers at python.org
>> https://mail.python.org/mailman/listinfo/python-committers
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-committers/attachments/20190509/c7fa7e85/attachment.html>