From owner-freebsd-current@FreeBSD.ORG Wed Apr 6 08:13:29 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBAFA106564A; Wed, 6 Apr 2011 08:13:29 +0000 (UTC) (envelope-from nwf@cs.jhu.edu) Received: from blaze.cs.jhu.edu (blaze.cs.jhu.edu [128.220.13.50]) by mx1.freebsd.org (Postfix) with ESMTP id 825EF8FC12; Wed, 6 Apr 2011 08:13:29 +0000 (UTC) Received: from gradx.cs.jhu.edu (gradx.cs.jhu.edu [128.220.13.52]) by blaze.cs.jhu.edu (8.14.3/8.14.3) with ESMTP id p3680hhV021057 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 6 Apr 2011 04:00:43 -0400 (EDT) Received: from gradx.cs.jhu.edu (localhost [127.0.0.1]) by gradx.cs.jhu.edu (8.14.3/8.13.1) with ESMTP id p3680h1d009359; Wed, 6 Apr 2011 04:00:43 -0400 Received: (from nwf@localhost) by gradx.cs.jhu.edu (8.14.3/8.13.8/Submit) id p3680hCM009358; Wed, 6 Apr 2011 04:00:43 -0400 Date: Wed, 6 Apr 2011 04:00:43 -0400 From: Nathaniel W Filardo To: freebsd-current@freebsd.org, freebsd-fs@freebsd.org Message-ID: <20110406080043.GQ609@gradx.cs.jhu.edu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NEvmWj2iiHf6S+l1" Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-08-17) X-Mailman-Approved-At: Wed, 06 Apr 2011 11:12:10 +0000 Cc: Subject: ZFS panic with concurrent recv and read-heavy workload X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 08:13:30 -0000 --NEvmWj2iiHf6S+l1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable When racing two workloads, one doing > zfs recv -v -d testpool and the other > find /testpool -type f -print0 | xargs -0 sha1 I can (seemingly reliably) trigger this panic: panic: Lock buf_hash_table.ht_locks[i].ht_lock not exclusively locked @ /us= r/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.= c:1869 =20 = =20 cpuid =3D 1 = =20 KDB: stack backtrace: = =20 panic() at panic+0x1c8 = =20 _sx_assert() at _sx_assert+0xc4 = =20 _sx_xunlock() at _sx_xunlock+0x98 = =20 arc_evict() at arc_evict+0x614 = =20 arc_get_data_buf() at arc_get_data_buf+0x360 = =20 arc_buf_alloc() at arc_buf_alloc+0x94 = =20 dmu_buf_will_fill() at dmu_buf_will_fill+0xfc dmu_write() at dmu_write+0xec dmu_recv_stream() at dmu_recv_stream+0x8a8 = =20 zfs_ioc_recv() at zfs_ioc_recv+0x354 = =20 zfsdev_ioctl() at zfsdev_ioctl+0xe0 = =20 devfs_ioctl_f() at devfs_ioctl_f+0xe8 = =20 kern_ioctl() at kern_ioctl+0x294 = =20 ioctl() at ioctl+0x198 syscallenter() at syscallenter+0x270 syscall() at syscall+0x74 = =20 -- syscall (54, FreeBSD ELF64, ioctl) %o7=3D0x40c13e24 -- = =20 userland() at 0x40e72cc8 = =20 user trace: trap %o7=3D0x40c13e24 = =20 pc 0x40e72cc8, sp 0x7fdffff4641 pc 0x40c158f4, sp 0x7fdffff4721 = =20 pc 0x40c1e878, sp 0x7fdffff47f1 = =20 pc 0x40c1ce54, sp 0x7fdffff8b01 = =20 pc 0x40c1dbe0, sp 0x7fdffff9431 = =20 pc 0x40c1f718, sp 0x7fdffffd741 = =20 pc 0x10731c, sp 0x7fdffffd831 = =20 pc 0x10c90c, sp 0x7fdffffd8f1 = =20 pc 0x103ef0, sp 0x7fdffffe1d1 = =20 pc 0x4021aff4, sp 0x7fdffffe291 = =20 done The machine is a freshly installed and built sparc64 2-way SMP, running today's -CURRENT with http://people.freebsd.org/~mm/patches/zfs/zfs_ioctl_compat_bugfix.patch applied. Of note, it has only 1G of RAM in it, so kmem_max <=3D 512M. Thoughts? More information? Thanks in advance. --nwf; --NEvmWj2iiHf6S+l1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAk2cHasACgkQTeQabvr9Tc+K4QCeOE6VM+JEoPVsvgDXPpIhSuG/ Nq4Ani1uL00nzKfhxpAQRxWCc51hWxw2 =ep5z -----END PGP SIGNATURE----- --NEvmWj2iiHf6S+l1--