From owner-freebsd-questions@FreeBSD.ORG Sun May 19 20:28:16 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B3F41525 for ; Sun, 19 May 2013 20:28:16 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-ye0-f180.google.com (mail-ye0-f180.google.com [209.85.213.180]) by mx1.freebsd.org (Postfix) with ESMTP id 79E9F8E0 for ; Sun, 19 May 2013 20:28:16 +0000 (UTC) Received: by mail-ye0-f180.google.com with SMTP id r11so1343113yen.11 for ; Sun, 19 May 2013 13:28:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:mime-version:content-type:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=3satUd4FtrICgETB7ojToVPRrf8GVIBOcbMvYG+c8/g=; b=evWYK4Pr+Z1mHIKki11J1GHRS9Ty/7C0Sj8ZxoKELOdPTyW/VTaUaHiUIKkCx3ae/l W5bP6E/vOmQTkU8Q2prc2DG34fUIFNLxU47CYbCWhtFJw891E7dd1hQS4jlXp4xEeG7g AWhW1SsmtYo6Ke7QRUVinJc/SnRiKZCtLDOE0vJosibqVNWXc3/hS7R7A6j6RRfj/fb+ M86tFNuo6yc4rLRmJAyOTnizDEAzT3wvaICp2Rk7Dup6ooM34a1JTLggKYB8SxzmM5H7 yzRrp0A0Mo6N4pOBjDEPnQdzej7eb+51L3S0Z+hRJv/Dxc+3JAmm8Zc+VFaQYmL/hpup dujA== X-Received: by 10.236.73.165 with SMTP id v25mr9346058yhd.141.1368995289614; Sun, 19 May 2013 13:28:09 -0700 (PDT) Received: from [192.168.2.66] ([96.236.21.119]) by mx.google.com with ESMTPSA id w62sm35117195yhd.27.2013.05.19.13.28.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 19 May 2013 13:28:08 -0700 (PDT) Subject: Re: More than 32 CPUs under 8.4-P Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Content-Type: text/plain; charset=us-ascii From: Paul Kraus In-Reply-To: <1368978686.16472.25.camel@btw.pki2.com> Date: Sun, 19 May 2013 16:28:06 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1368897188.16472.19.camel@btw.pki2.com> <51989FDA.5070302@coosemans.org> <1368978686.16472.25.camel@btw.pki2.com> To: Dennis Glatting X-Mailer: Apple Mail (2.1503) X-Gm-Message-State: ALoCoQnQaNpMdw4izi/5+KyGQENdfB1qCCi0b0COB8/P+aj1NS/Of/YR7l7I1ApBSUM12z+Oa4v8 Cc: Tijl Coosemans , freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 20:28:16 -0000 On May 19, 2013, at 11:51 AM, Dennis Glatting wrote: > ZFS hangs on multi-socket systems (Tyan, Supermicro) under 9.1. ZFS = does > not hang under 8.4. This (and one other 4 socket) is a production > system. Can you be more specific, I have been running 9.0 and 9.1 = systems with multi-CPU and all ZFS with no (CPU related*) issues. * I say no CPU related issues because I have run into SATA timeout = issues with an external SATA enclosure with 4 drives (I know, SATA port = expanders are evil, but it is my best option here). Sometimes the zpool = hangs hard, sometimes just becomes unresponsive for a while. My "fix", = such as it is, is to tune the zfs per vdev queue depth as follows: vfs.zfs.vdev.min_pending=3D"3" vfs.zfs.vdev.max_pending=3D"5" The defaults are 5 and 10 respectively, and when I run with those I have = the timeout issues, but only under very heavy I/O load. I only generate = such load when migrating large amounts of data, which thankfully does = not happen all that often. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company