From owner-freebsd-fs@FreeBSD.ORG Thu Jul 31 05:46:48 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26C07660; Thu, 31 Jul 2014 05:46:48 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 05BEB28D3; Thu, 31 Jul 2014 05:46:47 +0000 (UTC) Received: from delphij-macbook.local (c-24-5-244-32.hsd1.ca.comcast.net [24.5.244.32]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id CA97AA22C; Wed, 30 Jul 2014 22:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1406785606; x=1406800006; bh=LPBrsJpjgtF+5G4xFcqjROYXHy4PKW1B32BCG8fbSlY=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=2AKIQhmtlyST05OfefNn0ll/VdSVZzjyaMM4fZwQiYZrLSdBnLYMUMTkbr+is5/r1 ITOWtvBnYCyZEwuQ1YMJN6T5Y9HHHfSJCFtwZHnkZXIbezcqy7T6KLkhJusiiXExP8 pCm9m1wzEZPRbeTA6MqoEmczvla0jAvPfodOmuQ8= Message-ID: <53D9D845.3070905@delphij.net> Date: Wed, 30 Jul 2014 22:46:45 -0700 From: Xin Li Reply-To: d@delphij.net Organization: The FreeBSD Project MIME-Version: 1.0 To: Allan Jude , delphij@freebsd.org Subject: Re: panic: avl_find() succeeded inside avl_add() References: <53D9C306.6060105@freebsd.org> In-Reply-To: <53D9C306.6060105@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Andriy Gapon X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 05:46:48 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 7/30/14 9:16 PM, Allan Jude wrote: > I updated one of our storage servers to the latest -CURRENT > yesterday and now it is panic'ing every few minutes > > I suspect this is related to r269229 (MFV r269223) > > Jul 31 03:25:23 Tango syslogd: kernel boot file is > /boot/kernel/kernel Jul 31 03:25:23 Tango kernel: panic: avl_find() > succeeded inside avl_add() Jul 31 03:25:23 Tango kernel: cpuid = 0 > Jul 31 03:25:23 Tango kernel: KDB: stack backtrace: Jul 31 03:25:23 > Tango kernel: #0 0xffffffff809ab070 at kdb_backtrace+0x60 Jul 31 > 03:25:23 Tango kernel: #1 0xffffffff8096c3a5 at panic+0x155 Jul 31 > 03:25:23 Tango kernel: #2 0xffffffff819dbc04 at avl_add+0x84 Jul 31 > 03:25:23 Tango kernel: #3 0xffffffff81a04560 at dbuf_create+0x390 > Jul 31 03:25:23 Tango kernel: #4 0xffffffff81a03f5a at > dbuf_hold_impl+0x11a Jul 31 03:25:23 Tango kernel: #5 > 0xffffffff81a0467b at dbuf_hold+0x1b Jul 31 03:25:23 Tango kernel: > #6 0xffffffff81a09593 at dmu_buf_hold_array_by_dnode+0x153 Jul 31 > 03:25:23 Tango kernel: #7 0xffffffff81a0a526 at dmu_read_uio+0x66 > Jul 31 03:25:23 Tango kernel: #8 0xffffffff81a9bb0a at > zfs_freebsd_read+0x58a Jul 31 03:25:23 Tango kernel: #9 > 0xffffffff80eddf11 at VOP_READ_APV+0xa1 Jul 31 03:25:23 Tango > kernel: #10 0xffffffff80a32427 at vn_read+0x167 Jul 31 03:25:23 > Tango kernel: #11 0xffffffff80a2e5db at vn_io_fault+0x10b Jul 31 > 03:25:23 Tango kernel: #12 0xffffffff809c4a95 at dofileread+0x95 > Jul 31 03:25:23 Tango kernel: #13 0xffffffff809c47b8 at > kern_readv+0x68 Jul 31 03:25:23 Tango kernel: #14 > 0xffffffff809c4743 at sys_read+0x63 Jul 31 03:25:23 Tango kernel: > #15 0xffffffff80db84fb at amd64_syscall+0x3fb Jul 31 03:25:23 Tango > kernel: #16 0xffffffff80d9a47b at Xfast_syscall+0xfb Jul 31 > 03:25:23 Tango kernel: Uptime: 12m11s > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192284 Can you try reverting r221991? (Andriy: in r221991 -- why reverting that portion?) Cheers, -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJT2dhFAAoJEJW2GBstM+nsLQcP/3/lgH5C7n4v8zlLFrZFuPES Ml7XwK+NGk98/oVNGL984H88ec5p2L5vNBj54dSg1ntCDF9cororBrSaThBZNbvM xjQeQOAX300jL72vkIcZuEgNTRiPfufF+SsKSLVHduySIAJDJg+bakq8JP/VMH6X s4djO6u7RZYWmmIo7knKAWzml/mQ/A4SEh03o17nmcxp9ERkxDxWXyb/dWzVbYG7 J2oJa3Ar1SptGlA3SieLos4t08Ki/UeERZAcQD77LhhgO0jfSueBediDadNVyY9n tY1d/iGRMtck27B8zft0is50DJhRmgCTO50p++HHWQrUxH0I4kbEDEXmq3NBo+Sh YPBFSOPIuas/1pCLZB3eKvXi4ocY8XT+VUeqD01rDnbbu8lQCg1XCd3sOgMNn7Is 6GbEEwXUx9erNeTRU0thMzONbA20Qxn70CkmS+45llDjjfWlqqDxAYJHSldt3QMU MKqUp1K7R97pH3WOowZ+kNrfIlqXR7FAJVYqzQlYs0vgTfrhkmhVpM0nnICtTbCg GPqazvanI9gr+Mtncz366YlJ/zXhj0JGoK345uLPQzECf718QcaOBdfOGODXDrYb zXgcFihO09UFXLloStwmF5fRmOAvfE0UOnKAaMrzQe4mUPAm9dI7VGsOFcxMSV7T HasfIf0w5TZupbP1G9+6 =8O9r -----END PGP SIGNATURE-----