Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 08 Nov 2019 12:46:54 +0000
From:      bugzilla-noreply@freebsd.org
To:        python@FreeBSD.org
Subject:   maintainer-feedback requested: [Bug 241801] lang/python38: regression: hang in locking in multiprocessing.Pool
Message-ID:  <bug-241801-21822-T6NNyZLQvv@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-241801-21822@https.bugs.freebsd.org/bugzilla/>
References:  <bug-241801-21822@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
Bugzilla Automation <bugzilla@FreeBSD.org> has asked freebsd-python mailing
list <python@FreeBSD.org> for maintainer-feedback:
Bug 241801: lang/python38: regression: hang in locking in multiprocessing.P=
ool
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D241801



--- Description ---
This simple program

    from multiprocessing import Pool
    from time import sleep

    Pool().map(sleep, [0.01] * 10)

works fine with python 3.7, but is likely (about 20-50% probability on my 4
core box) to hang with python 3.8. Example backtraces after interruption:

% python3.8 1.py
^CException ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in
__call__
Process ForkPoolWorker-2:
Process ForkPoolWorker-4:
    res =3D self._callback(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in
_terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in
_help_stuff_finish
    inqueue._rlock.acquire()
KeyboardInterrupt:=20
Process ForkPoolWorker-3:
Process ForkPoolWorker-1:

% python3.8 1.py
^CException ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in
__call__
Process ForkPoolWorker-3:
Process ForkPoolWorker-4:
Process ForkPoolWorker-1:
    res =3D self._callback(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in
_terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in
_help_stuff_finish
    inqueue._rlock.acquire()
KeyboardInterrupt:=20
Process ForkPoolWorker-2:

% python3.8 1.py
^CException ignored in: <Finalize object, dead>
Process ForkPoolWorker-2:
Process ForkPoolWorker-3:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in
__call__
Process ForkPoolWorker-1:
    res =3D self._callback(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in
_terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in
_help_stuff_finish
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in
_bootstrap
    self.run()
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in
_bootstrap
    self.run()
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in =
run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in =
run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in wor=
ker
    task =3D get()
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in wor=
ker
    task =3D get()
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 355, in g=
et
    with self._rlock:
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 355, in g=
et
    with self._rlock:
  File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 95, =
in
__enter__
    return self._semlock.__enter__()
  File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 95, =
in
__enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
    inqueue._rlock.acquire()
KeyboardInterrupt:=20
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in
_bootstrap
    self.run()
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in =
run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in wor=
ker
    task =3D get()
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 356, in g=
et
    res =3D self._reader.recv_bytes()
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 216, =
in
recv_bytes
    buf =3D self._recv_bytes(maxlength)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 414, =
in
_recv_bytes
    buf =3D self._recv(4)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 379, =
in
_recv
    chunk =3D read(handle, remaining)
KeyboardInterrupt
Process ForkPoolWorker-4:


Should I submit this upstream?



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-241801-21822-T6NNyZLQvv>