From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 10:21:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C99621065672 for ; Sun, 3 Apr 2011 10:21:52 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) by mx1.freebsd.org (Postfix) with ESMTP id 5C2918FC0A for ; Sun, 3 Apr 2011 10:21:51 +0000 (UTC) Received: from hugo10.ka.punkt.de ([217.29.45.10]) by gate1.intern.punkt.de with ESMTP id p33ALoQa008752; Sun, 3 Apr 2011 12:21:50 +0200 (CEST) Received: from [217.29.46.3] ([217.29.46.3]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id p33ALnUI022416; Sun, 3 Apr 2011 12:21:50 +0200 (CEST) (envelope-from hausen@punkt.de) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: "Patrick M. Hausen" In-Reply-To: <20110402094038.GA3521@icarus.home.lan> Date: Sun, 3 Apr 2011 12:21:55 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> References: <20110402094038.GA3521@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1084) Cc: FreeBSD Stable Subject: Re: ahci.ko in RELENG_8_2, what about atacontrol cap? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 10:21:52 -0000 Hello, Am 02.04.2011 um 11:40 schrieb Jeremy Chadwick: > You want "camcontrol identify adaX". DO NOT confuse this with > "camcontrol inquiry adaX" (this won't work). >=20 > identify =3D for ATA > inquiry =3D for SCSI Works perfectly, but I just noticed one really odd thing: 1. boot without ahci.ko: nas-pmh# atacontrol cap ad4 ... write cache yes yes 2. boot with ahci.ko: nas-pmh# camcontrol identify ada0 ... write cache yes no Well? ;-) The system is a HP NL36 - I just found a couple of articles mentioning that the default setting in the BIOS setup was write cache disabled. I can check that in the next couple of days when I take the machine back to the lab (no monitor/keyboard at my home office). I'd prefer a way to make sure write cache is enabled via some tuning from FreeBSD. The disks are dedicated to a raidz2, so from what I found around the net, write cache should not pose a major problem. Of course, one will lose _some_ data at a power outage - what I want to avoid for home office use is a completely lost file system. Losing the last time machine backup of my Mac is tolerable. Thanks, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=FCrgen Egeling AG Mannheim 108285 From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 10:53:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7790A106564A for ; Sun, 3 Apr 2011 10:53:01 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id DA6088FC0C for ; Sun, 3 Apr 2011 10:53:00 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id E6313E8136; Sun, 3 Apr 2011 11:52:59 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=mail; bh=tHWhPV19mf4aKYK7di6ZviUV7sE=; b=IrQZqR wWxLUL8Sl97tY1ESGYQmkZ+2P3Wf+S+XuC8mfx2ATngLs/VUN2kdDwDVWvH75/qp S7qzzm2YVAxRpN2v/C5MJVm/fQQAbPPLO9lbryl0AfTbMX40i+VNKGaiiRrAqfVP P1zG5V+2MKCFuM9IiXG5TdT2iusZ5N1eCPk5k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; q=dns; s=mail; b=Cs29Bb9JGxji7sOYI87Ak8ExPQDwOzOu nKwZxzh30paShRW60U8uRPyoRfoMF9t0F4iiu8beyLpIdXraNvef+qARe4BoaG41 3XWCbc37EgaJmGjK9fB2TRXQPJeAVXLYjQD6P+DRGyKWffj88lFrN8npF8g8FeYb Ne97ADIqSos= Received: from [192.168.1.69] (188-222-18-231.zone13.bethere.co.uk [188.222.18.231]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id A8A10E7B60; Sun, 3 Apr 2011 11:52:59 +0100 (BST) Message-ID: <4D985184.5050409@cran.org.uk> Date: Sun, 03 Apr 2011 11:52:52 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: "Marat N.Afanasyev" References: <87d3l6p5xv.fsf@cosmos.claresco.hr> <874o6ip0ak.fsf@cosmos.claresco.hr> <4D95F143.8080001@ksu.ru> In-Reply-To: <4D95F143.8080001@ksu.ru> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms070907080703040202040804" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Marko Lerota , freebsd-stable@freebsd.org Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 10:53:01 -0000 This is a cryptographically signed message in MIME format. --------------ms070907080703040202040804 Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: quoted-printable On 01/04/2011 16:37, Marat N.Afanasyev wrote: > to ensure consistency you should turn off physical drive caches, and > degrade performance significantly, sometimes up to 1000x. if this is > what you want, you may use either zfs or sync ufs. in such case you may= > be almost sure that your filesystems are consistent. but if you use > drive's cache, then without UPS you will face data loss and vanished > filesystem earlier or later I'm not sure the performance loss is so severe with modern drives=20 (attached to a decent controller). If they do tagged queuing (NCQ/TCQ) I = think a lot of the performance lost through disabling the write cache=20 can be regained through the queueing system. --=20 Bruce Cran --------------ms070907080703040202040804-- From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 11:03:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B716F1065673 for ; Sun, 3 Apr 2011 11:03:52 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta07.emeryville.ca.mail.comcast.net (qmta07.emeryville.ca.mail.comcast.net [76.96.30.64]) by mx1.freebsd.org (Postfix) with ESMTP id 9E7D88FC12 for ; Sun, 3 Apr 2011 11:03:52 +0000 (UTC) Received: from omta19.emeryville.ca.mail.comcast.net ([76.96.30.76]) by qmta07.emeryville.ca.mail.comcast.net with comcast id Sz3m1g0011eYJf8A7z3s9R; Sun, 03 Apr 2011 11:03:52 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta19.emeryville.ca.mail.comcast.net with comcast id Sz3r1g0081t3BNj01z3rYk; Sun, 03 Apr 2011 11:03:52 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 196119B429; Sun, 3 Apr 2011 04:03:51 -0700 (PDT) Date: Sun, 3 Apr 2011 04:03:51 -0700 From: Jeremy Chadwick To: "Patrick M. Hausen" Message-ID: <20110403110351.GA30312@icarus.home.lan> References: <20110402094038.GA3521@icarus.home.lan> <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: mav@freebsd.org, FreeBSD Stable Subject: Re: ahci.ko in RELENG_8_2, what about atacontrol cap? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 11:03:52 -0000 On Sun, Apr 03, 2011 at 12:21:55PM +0200, Patrick M. Hausen wrote: > Hello, > > Am 02.04.2011 um 11:40 schrieb Jeremy Chadwick: > > You want "camcontrol identify adaX". DO NOT confuse this with > > "camcontrol inquiry adaX" (this won't work). > > > > identify = for ATA > > inquiry = for SCSI > > Works perfectly, but I just noticed one really odd thing: > > 1. boot without ahci.ko: > > nas-pmh# atacontrol cap ad4 > ... > write cache yes yes > > > 2. boot with ahci.ko: > > nas-pmh# camcontrol identify ada0 > ... > write cache yes no > > > Well? ;-) The system is a HP NL36 - I just found a couple > of articles mentioning that the default setting in the BIOS > setup was write cache disabled. I can check that in the > next couple of days when I take the machine back to the > lab (no monitor/keyboard at my home office). > > I'd prefer a way to make sure write cache is enabled via > some tuning from FreeBSD. The disks are dedicated to > a raidz2, so from what I found around the net, write cache > should not pose a major problem. Of course, one will lose > _some_ data at a power outage - what I want to avoid > for home office use is a completely lost file system. > Losing the last time machine backup of my Mac is tolerable. I don't have an explanation for what you're seeing. I can't reproduce it on any of our Supermicro systems (ICH9R-based, backed by a multitude of disk types; Intel X25-M and X25-V SSDs, WD Caviar Black 750GB, 1TB, and 2TB, etc.). CC'ing mav@ who might have some ideas. I don't see anything in RELENG_8 that looks relevant (just went through src/sys/dev/ahci's commit log). Alexander, disk type is here (Seagate ST32000542AS): http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062142.html -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 12:05:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06F201065673 for ; Sun, 3 Apr 2011 12:05:03 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 5C1348FC1B for ; Sun, 3 Apr 2011 12:05:03 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 52F5DE8136; Sun, 3 Apr 2011 13:05:02 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=mail; bh=0pen5W7KYmrkLGcj7MCgDvMd6Ic=; b=o93nUm 0SlWU5SNGMEERTMabzxuuVObr5z6YN/u3eXePmGgeJszAu1bbu/Z4dcBwwK9vWzf f1LJ8LqvQtftI7FnWyl5LfsNbKc/OM6sT6woA9pz/lMYKcWRufZ4t0W65Cj7hdRW w96TqDAdVmu5QJ86CA5om6clBUW0xBchbOmrU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; q=dns; s=mail; b=dPmnhCIBPeZR7qZxA8EEs0OA3vWdbIZ7 EO8JBagk+UcwQ6SW9S1u0XEbm6RZI7ezvuC+lkWRq1G+v590kwjiHw0M/4GfJxjR NGz7oILxOz9+L9M7b+RJlRexAd5HdzpxLEmPBc/KkIJbTUZYJMeJq8XZLdSoYL+A xGNNeXbnfEE= Received: from [192.168.1.69] (188-222-18-231.zone13.bethere.co.uk [188.222.18.231]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 0FAD2E7B60; Sun, 3 Apr 2011 13:05:02 +0100 (BST) Message-ID: <4D986266.9060205@cran.org.uk> Date: Sun, 03 Apr 2011 13:04:54 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: Stefan `Sec` Zehl References: <87d3l6p5xv.fsf@cosmos.claresco.hr> <20110401154732.GC37730@ice.42.org> In-Reply-To: <20110401154732.GC37730@ice.42.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms030306070808020902020101" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Marko Lerota , freebsd-stable@freebsd.org Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 12:05:04 -0000 This is a cryptographically signed message in MIME format. --------------ms030306070808020902020101 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 01/04/2011 16:47, Stefan `Sec` Zehl wrote: > If you want to get rid of the reboot loop, set: > > background_fsck=3D"NO" > > Then it will either come up, or ask for help if anything fails. I realise that people like having systems that come up quickly after a=20 crash, but is it worth reconsidering disabling background fsck by=20 default since it can cause issues like this? --=20 Bruce Cran --------------ms030306070808020902020101-- From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 13:10:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76E8106564A for ; Sun, 3 Apr 2011 13:10:07 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 791B18FC15 for ; Sun, 3 Apr 2011 13:10:07 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p33DA5R6007871 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 3 Apr 2011 09:10:05 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4D9871AC.5020506@sentex.net> Date: Sun, 03 Apr 2011 09:10:04 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: "Patrick M. Hausen" References: <20110402094038.GA3521@icarus.home.lan> <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> In-Reply-To: <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on IPv6:2607:f3e0:0:1::12 Cc: FreeBSD Stable , Jeremy Chadwick Subject: Re: ahci.ko in RELENG_8_2, what about atacontrol cap? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 13:10:07 -0000 On 4/3/2011 6:21 AM, Patrick M. Hausen wrote: > Works perfectly, but I just noticed one really odd thing: > > 1. boot without ahci.ko: > > nas-pmh# atacontrol cap ad4 > ... > write cache yes yes > 2. boot with ahci.ko: > nas-pmh# camcontrol identify ada0 > ... > write cache yes no > Well? ;-) The system is a HP NL36 - I just found a couple Is not the default from the ata driver hw.ata.wc=1 ? Are you setting this to zero in /boot/loader.conf or /etc/sysctl.conf perhaps ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 13:44:52 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4792A1065675; Sun, 3 Apr 2011 13:44:52 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) by mx1.freebsd.org (Postfix) with ESMTP id D28AE8FC0A; Sun, 3 Apr 2011 13:44:51 +0000 (UTC) Received: from hugo10.ka.punkt.de ([217.29.45.10]) by gate1.intern.punkt.de with ESMTP id p33DioLZ010034; Sun, 3 Apr 2011 15:44:50 +0200 (CEST) Received: from [217.29.46.3] ([217.29.46.3]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id p33Dioet026259; Sun, 3 Apr 2011 15:44:50 +0200 (CEST) (envelope-from hausen@punkt.de) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: "Patrick M. Hausen" In-Reply-To: <4D9877E7.6050807@FreeBSD.org> Date: Sun, 3 Apr 2011 15:44:56 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1846D8C1-99F9-4258-A428-14E7D5CF58E1@punkt.de> References: <20110402094038.GA3521@icarus.home.lan> <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> <20110403110351.GA30312@icarus.home.lan> <4D9877E7.6050807@FreeBSD.org> To: Alexander Motin X-Mailer: Apple Mail (2.1084) Cc: FreeBSD Stable , Jeremy Chadwick Subject: Re: ahci.ko in RELENG_8_2, what about atacontrol cap? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 13:44:52 -0000 Hi, Am 03.04.2011 um 15:36 schrieb Alexander Motin: > I've noticed that some RAID BIOS'es disable write cache on their = disks. ata(4) enabled cache in such cases, but CAM doesn't now. I'll = take care of it for ATA. For now you can manage it via `camcontrol cmd`. I understand. So probably my disks' WC is set to off in the system BIOS setup - wich is changed upon boot by ata, but not by ahci/cam. Thanks, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=FCrgen Egeling AG Mannheim 108285= From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 14:07:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CED981065672 for ; Sun, 3 Apr 2011 14:07:40 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5CEAB8FC17 for ; Sun, 3 Apr 2011 14:07:40 +0000 (UTC) Received: by fxm11 with SMTP id 11so4538229fxm.13 for ; Sun, 03 Apr 2011 07:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=15KfRgOBvnRYMG6y2cSbi7pgtDzbaZMAkxKpI6Yi/Ys=; b=HO9x8FOfUz4nsf/95wVHeidLxnDGIMsDyryseib1vmRx+07mJCZy4P8r2+SNKzLC6G o9sQpz90fFq1AlVlhdrhbRuyYajjKCgPnqC0UaMQ8qur4Hd5B557thKu8tPVKb5irkCi gI/nKZbWoFuKZz254Je4jsOcL0GxIqPmJlGbs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=VgUtBG17FDWLgXT0opakld51rJAiEwAm5M+YXjGFgi+f0iYrXWIxivP/YqlDj8q5SY FYWSBU9b7j66HxYwsZRw7jHZt99FaGl2YLv+J0WvvaZ3XY9fvuDX3zbKCSkfanLMoD85 /UiaMdqLm/CYPo6Tqor7zO+sd46Na7156y1gk= Received: by 10.223.1.198 with SMTP id 6mr122675fag.120.1301837803649; Sun, 03 Apr 2011 06:36:43 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([91.198.175.1]) by mx.google.com with ESMTPS id e17sm1365293fak.24.2011.04.03.06.36.41 (version=SSLv3 cipher=OTHER); Sun, 03 Apr 2011 06:36:42 -0700 (PDT) Sender: Alexander Motin Message-ID: <4D9877E7.6050807@FreeBSD.org> Date: Sun, 03 Apr 2011 16:36:39 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: Jeremy Chadwick References: <20110402094038.GA3521@icarus.home.lan> <3B6BFB01-3AC4-432B-8713-6CED09FFF9A2@punkt.de> <20110403110351.GA30312@icarus.home.lan> In-Reply-To: <20110403110351.GA30312@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: ahci.ko in RELENG_8_2, what about atacontrol cap? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 14:07:41 -0000 On 03.04.2011 14:03, Jeremy Chadwick wrote: > On Sun, Apr 03, 2011 at 12:21:55PM +0200, Patrick M. Hausen wrote: >> Hello, >> >> Am 02.04.2011 um 11:40 schrieb Jeremy Chadwick: >>> You want "camcontrol identify adaX". DO NOT confuse this with >>> "camcontrol inquiry adaX" (this won't work). >>> >>> identify = for ATA >>> inquiry = for SCSI >> >> Works perfectly, but I just noticed one really odd thing: >> >> 1. boot without ahci.ko: >> >> nas-pmh# atacontrol cap ad4 >> ... >> write cache yes yes >> >> >> 2. boot with ahci.ko: >> >> nas-pmh# camcontrol identify ada0 >> ... >> write cache yes no >> >> >> Well? ;-) The system is a HP NL36 - I just found a couple >> of articles mentioning that the default setting in the BIOS >> setup was write cache disabled. I can check that in the >> next couple of days when I take the machine back to the >> lab (no monitor/keyboard at my home office). >> >> I'd prefer a way to make sure write cache is enabled via >> some tuning from FreeBSD. The disks are dedicated to >> a raidz2, so from what I found around the net, write cache >> should not pose a major problem. Of course, one will lose >> _some_ data at a power outage - what I want to avoid >> for home office use is a completely lost file system. >> Losing the last time machine backup of my Mac is tolerable. > > I don't have an explanation for what you're seeing. I can't reproduce > it on any of our Supermicro systems (ICH9R-based, backed by a multitude > of disk types; Intel X25-M and X25-V SSDs, WD Caviar Black 750GB, 1TB, > and 2TB, etc.). > > CC'ing mav@ who might have some ideas. I don't see anything in RELENG_8 > that looks relevant (just went through src/sys/dev/ahci's commit log). > > Alexander, disk type is here (Seagate ST32000542AS): > http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062142.html I've noticed that some RAID BIOS'es disable write cache on their disks. ata(4) enabled cache in such cases, but CAM doesn't now. I'll take care of it for ATA. For now you can manage it via `camcontrol cmd`. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 14:29:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A2D11065674 for ; Sun, 3 Apr 2011 14:29:23 +0000 (UTC) (envelope-from gkontos.mail@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 30C778FC08 for ; Sun, 3 Apr 2011 14:29:22 +0000 (UTC) Received: by iwn33 with SMTP id 33so6229820iwn.13 for ; Sun, 03 Apr 2011 07:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=jkeWS86+WRSuIHg0crDlEdAR0cJUaAKl7iP12FLBOww=; b=j0xHkETkS4fXVtO/iEflLOBCfY3iE0lx3aKZZIFbNML6PMQ+7OcrDJOiNJ/gZnPX4y Akcr5jPgO8d7d9k713uXz8xrymxdXaRy8gP/jINzNQWzQug1Q5ulNFlW1aXpjVRv7CbX mQ5mCC7cpnPMv/LjoqNM2iwSO6TwhY91qsO8o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=P06i8L+8UVNSZ2nyPSIAwh9NqH+g0E3QY1QpO6gI4o1Lvc6ojuMsIpaGtoQ5BWOM/I 0nsBZijL0tO86ej524GF2qj3JgX2eZilAC13bUHK5tYXJZOwTn36GDdYqBEleBmwh8fi DjMgMxgc7vAY2DPupb1GGCmw+8fYIi/InawzU= MIME-Version: 1.0 Received: by 10.231.4.139 with SMTP id 11mr6357096ibr.65.1301840962358; Sun, 03 Apr 2011 07:29:22 -0700 (PDT) Received: by 10.231.33.1 with HTTP; Sun, 3 Apr 2011 07:29:22 -0700 (PDT) In-Reply-To: <4D986266.9060205@cran.org.uk> References: <87d3l6p5xv.fsf@cosmos.claresco.hr> <20110401154732.GC37730@ice.42.org> <4D986266.9060205@cran.org.uk> Date: Sun, 3 Apr 2011 17:29:22 +0300 Message-ID: From: George Kontostanos To: Bruce Cran Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Stefan `Sec` Zehl , freebsd-stable@freebsd.org, Marko Lerota Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 14:29:23 -0000 Those are workarounds. A system should not reach to a point where it doesn't come up because of a pending fsck. And even if it does console access must be ensured. My point is that by disabling these you might bring it up but you seriously risk data integrity. Regards On Sun, Apr 3, 2011 at 3:04 PM, Bruce Cran wrote: > On 01/04/2011 16:47, Stefan `Sec` Zehl wrote: > > If you want to get rid of the reboot loop, set: >> >> background_fsck="NO" >> >> Then it will either come up, or ask for help if anything fails. >> > > I realise that people like having systems that come up quickly after a > crash, but is it worth reconsidering disabling background fsck by default > since it can cause issues like this? > > -- > Bruce Cran > > -- George Kontostanos aisecure.net From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 14:29:26 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5D9E106564A for ; Sun, 3 Apr 2011 14:29:26 +0000 (UTC) (envelope-from torfinn.ingolfsen@broadpark.no) Received: from thalia-smout.broadpark.no (thalia-smout.broadpark.no [80.202.8.21]) by mx1.freebsd.org (Postfix) with ESMTP id 801868FC0C for ; Sun, 3 Apr 2011 14:29:26 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from terra-smin.broadpark.no ([80.202.8.13]) by thalia-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-15.01 64bit (built Feb 12 2010)) with ESMTP id <0LJ2009I7YX0G8B0@thalia-smout.broadpark.no> for freebsd-stable@freebsd.org; Sun, 03 Apr 2011 16:29:24 +0200 (CEST) Received: from kg-v2.kg4.no ([84.48.120.215]) by terra-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-15.01 64bit (built Feb 12 2010)) with SMTP id <0LJ20023BYWZBPC0@terra-smin.broadpark.no> for freebsd-stable@freebsd.org; Sun, 03 Apr 2011 16:29:24 +0200 (CEST) Date: Sun, 03 Apr 2011 16:29:23 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20110403162923.b4ae0ce9.torfinn.ingolfsen@broadpark.no> In-reply-to: <4D986266.9060205@cran.org.uk> References: <87d3l6p5xv.fsf@cosmos.claresco.hr> <20110401154732.GC37730@ice.42.org> <4D986266.9060205@cran.org.uk> X-Mailer: Sylpheed 3.1.0 (GTK+ 2.22.1; amd64-portbld-freebsd8.1) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 14:29:26 -0000 On Sun, 03 Apr 2011 13:04:54 +0100 Bruce Cran wrote: > On 01/04/2011 16:47, Stefan `Sec` Zehl wrote: > > > If you want to get rid of the reboot loop, set: > > > > background_fsck="NO" > > > > Then it will either come up, or ask for help if anything fails. > > I realise that people like having systems that come up quickly after a > crash, but is it worth reconsidering disabling background fsck by > default since it can cause issues like this? Based _only_ on my own experience, and personal preference, I would say "NO". The reason? It works nicely the way it is now, and any problems will be noticed, and can be fixed with a bit of effort, even for a person with little experience in running FreeBSD machines. One of the reasons that I like FreeBSD is that it gives me tools I need, but it doesn't try to automate or abstract away my responsiblites as a system administrator. I think the FreeBSD project has managed something impressive; being a platform for bringing in new ideas / tools, and at the same time staying conservative (POLA, and well thought out changes). I like that. -- Regards, Torfinn Ingolfsen FreeBSD user & FreeBSD sysadmin on the personal / hobby level, for many, many years From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 17:20:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ECBCB1065674 for ; Sun, 3 Apr 2011 17:20:36 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 4C2C18FC08 for ; Sun, 3 Apr 2011 17:20:36 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 6EE08E8136; Sun, 3 Apr 2011 18:20:35 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=mail; bh=4eir9yxKq8jP0enD2c7gjsDmCBo=; b=cvygrI O71Nnrxq92d3np+eE+Xp7cUSCt1XOmJYmOykaBmk56SN8/dv93QHbIBrlRZ6c1yc y00nC9+BiG8XQQX39a69ZHsU2EMN2GA6VlMGMCMst+an96SZBNepRGt3NT9n3KVr B2CNFUAxq3vGILMbitKFFqxOBUile/Rh1NqUM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; q=dns; s=mail; b=c0gQDs66BzboEZGBDnfu/l1fkeqb2nM8 wftVUfM+IDQ3z7Mm4inWdvyssTThYijlVi4+9TWG80Bxvl2thw2sFEp9Rrx4gDuM esgnol2JN6cFoxbLPJudYYc1aw8rycM4aDb86UBU/T8RgO/PaGfarqJoO66Q9CxZ nPzeNxbcBzU= Received: from [192.168.1.69] (188-222-18-231.zone13.bethere.co.uk [188.222.18.231]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 378F0E7B60; Sun, 3 Apr 2011 18:20:35 +0100 (BST) Message-ID: <4D98AC5B.1050606@cran.org.uk> Date: Sun, 03 Apr 2011 18:20:27 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: Matthew Dillon References: <87d3l6p5xv.fsf@cosmos.claresco.hr> <874o6ip0ak.fsf@cosmos.claresco.hr> <7b15d37d28f8ddac9eb81e4390231c96.HRCIM@webmail.1command.com> <14c23d4bf5b47a7790cff65e70c66151.HRCIM@webmail.1command.com> <201104020335.p323Zp8Q018666@apollo.backplane.com> In-Reply-To: <201104020335.p323Zp8Q018666@apollo.backplane.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms080506010204030007090301" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 17:20:37 -0000 This is a cryptographically signed message in MIME format. --------------ms080506010204030007090301 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 02/04/2011 04:35, Matthew Dillon wrote: > First, a power loss to the drive will cause the drive's dirty writ= e cache > to be lost, that data will not make it to disk. Nor do you really= want > to turn of write caching on the physical drive. Well, you CAN tur= n it > off, but if you do performance will become so bad that there's no = point. > So turning off the write caching is really a non-starter. Do you know if that's changed at all with NCQ on modern SATA drives?=20 I've seen people commenting that using tags recovers most, if not all,=20 of the performance lost by disabling the write cache. --=20 Bruce Cran --------------ms080506010204030007090301-- From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 17:43:51 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E026106566C; Sun, 3 Apr 2011 17:43:51 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4D6108FC12; Sun, 3 Apr 2011 17:43:50 +0000 (UTC) Received: by fxm11 with SMTP id 11so4616642fxm.13 for ; Sun, 03 Apr 2011 10:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:references:x-comment-to :sender:date:in-reply-to:message-id:user-agent:mime-version :content-type; bh=fIvncfIi8AuR7jO0555BYaTyfa/IvO0ntbJYAkGP8+Y=; b=Ie0FS6i8fFn/RebIU7AtMCTFo4EtZbW5sltKOmbt8ktrRdWB50RsesTu2+ArJ8jFbT dZI6hSnaQRpHGwXpavs6U2urEDE/tGcivZw5jkWDSYau5NYkTCiexANPVpniNZDhJDrA /5sji1jcQ9M9dPVMT3kR0+GyMCcuh8lTRrO5s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=XJLhlzbPYOU+Hx5PXlJZcGPjo2lKvA+1Aop7RlVB5IiEsC/PGcUIuQXtAEcbeKTLzS /R/7xBlvgBPtgq/3hldUonUS7VbqO+0FQoyfaU2dl1oinjLAg/jdhbkdzY4PYuDFHbN4 5nV3+q7I1yWWhnatYDbuimJitUkK7DUNjGNns= Received: by 10.223.29.4 with SMTP id o4mr34283fac.27.1301852629452; Sun, 03 Apr 2011 10:43:49 -0700 (PDT) Received: from localhost ([95.69.172.154]) by mx.google.com with ESMTPS id p16sm1422916fax.21.2011.04.03.10.43.47 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 03 Apr 2011 10:43:48 -0700 (PDT) From: Mikolaj Golub To: Pawel Jakub Dawidek References: <20110326003348.GQ36706@equilibrium.bsdes.net> <20110401174354.GE1289@equilibrium.bsdes.net> <86pqp53cqe.fsf@kopusha.home.net> <20110402101750.GD1849@garage.freebsd.pl> X-Comment-To: Pawel Jakub Dawidek Sender: Mikolaj Golub Date: Sun, 03 Apr 2011 20:43:45 +0300 In-Reply-To: <20110402101750.GD1849@garage.freebsd.pl> (Pawel Jakub Dawidek's message of "Sat, 2 Apr 2011 12:17:50 +0200") Message-ID: <86zko7nsby.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Mikolaj Golub , Victor Balada Diaz , stable@freebsd.org, Kostik Belousov Subject: Re: geli(4) memory leak X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 17:43:51 -0000 On Sat, 2 Apr 2011 12:17:50 +0200 Pawel Jakub Dawidek wrote: PJD> On Sat, Apr 02, 2011 at 12:04:09AM +0300, Mikolaj Golub wrote: >> For me your patch look correct. But the same issue is for read :-). Also, to >> avoid the leak I think we can just do g_destroy_bio() before "all sectors" >> check. See the attached patch (had some testing). PJD> The patch looks good. Please commit. Commited, thanks. -- Mikolaj Golub From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 23:29:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54564106566C for ; Sun, 3 Apr 2011 23:29:42 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 41F5C8FC1D for ; Sun, 3 Apr 2011 23:29:42 +0000 (UTC) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id p33NTbvL023277 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 3 Apr 2011 16:29:38 -0700 Received: from ptavv.es.net (localhost [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id E14CA1CC0C; Sun, 3 Apr 2011 16:29:37 -0700 (PDT) To: Bruce Cran In-reply-to: Your message of "Sun, 03 Apr 2011 18:20:27 BST." <4D98AC5B.1050606@cran.org.uk> Date: Sun, 03 Apr 2011 16:29:37 -0700 From: "Kevin Oberman" Message-Id: <20110403232937.E14CA1CC0C@ptavv.es.net> Cc: freebsd-stable@freebsd.org Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 23:29:42 -0000 > Date: Sun, 03 Apr 2011 18:20:27 +0100 > From: Bruce Cran > Sender: owner-freebsd-stable@freebsd.org > > On 02/04/2011 04:35, Matthew Dillon wrote: > > > First, a power loss to the drive will cause the drive's dirty write cache > > to be lost, that data will not make it to disk. Nor do you really want > > to turn of write caching on the physical drive. Well, you CAN turn it > > off, but if you do performance will become so bad that there's no point. > > So turning off the write caching is really a non-starter. > > Do you know if that's changed at all with NCQ on modern SATA drives? > I've seen people commenting that using tags recovers most, if not all, > of the performance lost by disabling the write cache. I may be confused, but I don't think you are interpreting the response correctly as I don't see nay way that the use of NCQ would begin to provide the performance of having a write cache. In fact, if I understand NCQ correctly, it requires that the drive write-cache data so NCQ can do it's thing. I believe that point was that properly functioning NCQ (or TCQ) can assure that the metadata is safely updated so that power loss will never engender data corruption while enhancing performance. It still will not save you from losing the data that is in cache and not written, but that is the extent of the damage. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 From owner-freebsd-stable@FreeBSD.ORG Sun Apr 3 23:52:13 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA639106564A; Sun, 3 Apr 2011 23:52:13 +0000 (UTC) (envelope-from victor@bsdes.net) Received: from equilibrium.bsdes.net (244.Red-217-126-240.staticIP.rima-tde.net [217.126.240.244]) by mx1.freebsd.org (Postfix) with ESMTP id 65C938FC13; Sun, 3 Apr 2011 23:52:12 +0000 (UTC) Received: by equilibrium.bsdes.net (Postfix, from userid 1001) id 6BD4539845; Mon, 4 Apr 2011 01:51:24 +0200 (CEST) Date: Mon, 4 Apr 2011 01:51:24 +0200 From: Victor Balada Diaz To: Mikolaj Golub Message-ID: <20110403235124.GF1289@equilibrium.bsdes.net> References: <20110326003348.GQ36706@equilibrium.bsdes.net> <20110401174354.GE1289@equilibrium.bsdes.net> <86pqp53cqe.fsf@kopusha.home.net> <20110402101750.GD1849@garage.freebsd.pl> <86zko7nsby.fsf@kopusha.home.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86zko7nsby.fsf@kopusha.home.net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Kostik Belousov , stable@freebsd.org, Pawel Jakub Dawidek Subject: Re: geli(4) memory leak X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 23:52:13 -0000 On Sun, Apr 03, 2011 at 08:43:45PM +0300, Mikolaj Golub wrote: > > On Sat, 2 Apr 2011 12:17:50 +0200 Pawel Jakub Dawidek wrote: > > PJD> On Sat, Apr 02, 2011 at 12:04:09AM +0300, Mikolaj Golub wrote: > >> For me your patch look correct. But the same issue is for read :-). Also, to > >> avoid the leak I think we can just do g_destroy_bio() before "all sectors" > >> check. See the attached patch (had some testing). > > PJD> The patch looks good. Please commit. > > Commited, thanks. I've been out all the weekend, so i've been unable to answer before. I'm glad it got commited and it's great you discovered and fixed the same problem on the read path. Are there any plans to MFC this? Thanks a lot. Regards. -- La prueba más fehaciente de que existe vida inteligente en otros planetas, es que no han intentado contactar con nosotros. From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 01:46:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C16F106566B for ; Mon, 4 Apr 2011 01:46:09 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [65.120.238.197]) by mx1.freebsd.org (Postfix) with ESMTP id 2E07E8FC14 for ; Mon, 4 Apr 2011 01:46:08 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id p341k8Rr030401 for ; Sun, 3 Apr 2011 18:46:08 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id p341k8Lp030400; Sun, 3 Apr 2011 18:46:08 -0700 (PDT) Date: Sun, 3 Apr 2011 18:46:08 -0700 (PDT) From: Matthew Dillon Message-Id: <201104040146.p341k8Lp030400@apollo.backplane.com> To: freebsd-stable@freebsd.org References: <20110403232937.E14CA1CC0C@ptavv.es.net> Subject: Re: Constant rebooting after power loss X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 01:46:09 -0000 :> Do you know if that's changed at all with NCQ on modern SATA drives? :> I've seen people commenting that using tags recovers most, if not all, :> of the performance lost by disabling the write cache. :... I've never tried that combination. Theoretically the 32 tags SATA supports would just barely be enough for sequential write service loads but I really doubt it would be enough for mixed service loads and you would be blowing up your read performance to achieve even that due to the length of time the tags stay busy with writes. With some driver massaging, such as partitioning the tag space and dedicating a specific number of tags for writing, read performance could probably be maintained but write performance (with caches off) would definitely still suffer. It might not horrible, though. One advantage of turning off the drive's write cache is that it would be possible for the OS to control write interference vs read loads, which is impossible to do with caches turned on. That is, with caches turned on your writes are instantly acknowledged until the drive's own caches exceed their dirty limits and by that time the drive is juggling so much dirty data that we (the OS/driver) have no control over read vs write performance. This is why it is so blasted difficult to write I/O schedulers in OS's that actually work. With caches disabled the OS/driver would have a great deal more control over read vs write performance. I/O scheduling would become viable. But to really make it work well I think we would need 64-128 tags (or more) to be able to cover multiple writing zones. With only 32 tags the drive's zone cache will be defeated. It would be a very interesting test. I can't immediately dismiss tagged I/O with write caches disabled. -Matt From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 06:20:13 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA6F5106564A; Mon, 4 Apr 2011 06:20:13 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id EE6048FC08; Mon, 4 Apr 2011 06:20:12 +0000 (UTC) Received: by wwc33 with SMTP id 33so5831312wwc.31 for ; Sun, 03 Apr 2011 23:20:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:organization:references :sender:date:in-reply-to:message-id:user-agent:mime-version :content-type; bh=13htVigszmlm9jdpNu+vTh4GZTdPVwJlMEzHKxfSnKs=; b=TdmbtRIUTM6AFL+PvueBOv8W5xpXnCT8ap3s7HYUpRF/fMnM8CSbMeZiO8Opsd2m/a J0aCISvOtLrt9/nnjQ9pBTmeacYv1dan0ULu3TK+ksoniRgDpjmo8xJgj/a+/PWa/5tV LBnUpmcTF9kIcq6pSys9hCpItdQvynj5AW4aM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:organization:references:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=pXUX0WW9oURDbpc/3fz6j5sctYCjyjmiAx8TqBwXzQ8yByaxon5988q+JUgbJl14qL ioquQ+WPMeqruSom4+pxwLJssiklm6YhkLdPA9VgLQPp00iUSQDmp7mcA5K+XPXnHueV dXz/dLbdbplRrhXFx8wP5j06AFyKairUK6aMA= Received: by 10.216.121.208 with SMTP id r58mr3389718weh.61.1301898012007; Sun, 03 Apr 2011 23:20:12 -0700 (PDT) Received: from localhost ([94.27.39.186]) by mx.google.com with ESMTPS id x1sm2715815wbh.19.2011.04.03.23.20.10 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 03 Apr 2011 23:20:10 -0700 (PDT) From: Mikolaj Golub To: Victor Balada Diaz Organization: TOA Ukraine References: <20110326003348.GQ36706@equilibrium.bsdes.net> <20110401174354.GE1289@equilibrium.bsdes.net> <86pqp53cqe.fsf@kopusha.home.net> <20110402101750.GD1849@garage.freebsd.pl> <86zko7nsby.fsf@kopusha.home.net> <20110403235124.GF1289@equilibrium.bsdes.net> Sender: Mikolaj Golub Date: Mon, 04 Apr 2011 09:20:08 +0300 In-Reply-To: <20110403235124.GF1289@equilibrium.bsdes.net> (Victor Balada Diaz's message of "Mon, 4 Apr 2011 01:51:24 +0200") Message-ID: <864o6ek06f.fsf@in138.ua3> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Kostik Belousov , stable@freebsd.org, Pawel Jakub Dawidek Subject: Re: geli(4) memory leak X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 06:20:14 -0000 On Mon, 4 Apr 2011 01:51:24 +0200 Victor Balada Diaz wrote: VBD> On Sun, Apr 03, 2011 at 08:43:45PM +0300, Mikolaj Golub wrote: >> >> On Sat, 2 Apr 2011 12:17:50 +0200 Pawel Jakub Dawidek wrote: >> >> PJD> On Sat, Apr 02, 2011 at 12:04:09AM +0300, Mikolaj Golub wrote: >> >> For me your patch look correct. But the same issue is for read :-). Also, to >> >> avoid the leak I think we can just do g_destroy_bio() before "all sectors" >> >> check. See the attached patch (had some testing). >> >> PJD> The patch looks good. Please commit. >> >> Commited, thanks. VBD> I've been out all the weekend, so i've been unable to answer before. I'm glad VBD> it got commited and it's great you discovered and fixed the same problem on the VBD> read path. VBD> Are there any plans to MFC this? Approximately after one week. -- Mikolaj Golub From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 13:20:11 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3B74106566B for ; Mon, 4 Apr 2011 13:20:11 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 76EDE8FC1C for ; Mon, 4 Apr 2011 13:20:11 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.4/8.14.4) with ESMTP id p34D72tG015721 for ; Mon, 4 Apr 2011 15:07:03 +0200 Received: from pmp.uni-hannover.de (unknown [130.75.117.3]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 05A412B3 for ; Mon, 4 Apr 2011 15:07:02 +0200 (CEST) Date: Mon, 4 Apr 2011 15:07:01 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-stable@freebsd.org Message-Id: <20110404150701.c5c050ff.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 3.0.3 (GTK+ 2.20.1; amd64-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.395186, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.4.4.125733 Subject: drives >2TB on mpt device X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gerrit.kuehn@aei.mpg.de List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 13:20:12 -0000 Hi all, I have a freshly installed 8.2-REL with a SuperMicro AOC-USASLP-L8i controller (LSI/MPT 1068E chipset). I have several of these controllers working nicely in other systems. However, this time I tried drives >2TB for the first time (Hitachi Deskstar 3TB). It appears that the mpt device reports only 2TB in this case. I have already flashed the controller's firmware to the latest available version (from 2009), but that did not change anything. The drive is working fine on the standard SATA connectors on the mainboard (Supermicro H8DME-2) and reports 2.8TB there. Are there any hints how to access the full drive? Am I seeing a limitation of the controller/firmware or rather of the driver (mpt)? cu Gerrit From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 13:36:39 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8785A1065675 for ; Mon, 4 Apr 2011 13:36:39 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 1A6298FC0C for ; Mon, 4 Apr 2011 13:36:39 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 2D1FDE8136; Mon, 4 Apr 2011 14:36:38 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=HqYU1/MwaP+Q 03usAH1nvuyC9o4=; b=QW2csb1/qvHIQm0LeQKETtgYBfowsISgH6oQ5WTb30UK 6xHKu7W72e79yUYB44gnNEjgbg+TnrogVjufCS4lQFcJc6ElOCGAitkSmoQPtCm+ XQU3Boc9Zy8KwYdj7vIdMcgLvh5RHj2w93ZSGm5YIT2Q87SOHdZBcP0dCs/W2BE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=E8JtIx BPWmcHQRsILmhrJR/96b2VQkqspC/89dKufIRcKqej9+gvWMg5RQ8KK8AkcPVxHz L1o/HFuNyuHP4m6lmGnGlWyXRukOtMYXd/cYryYBHpJ5xNg3GdlntKqHpwG8NOtN 8Wt6YAdwaU/aqxHpKurhoRcNadNFrCbpYQWtM= Received: from unknown (188-222-18-231.zone13.bethere.co.uk [188.222.18.231]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id D742BE64F2; Mon, 4 Apr 2011 14:36:37 +0100 (BST) Date: Mon, 4 Apr 2011 14:36:25 +0100 From: Bruce Cran To: gerrit.kuehn@aei.mpg.de Message-ID: <20110404143625.00002eb1@unknown> In-Reply-To: <20110404150701.c5c050ff.gerrit@pmp.uni-hannover.de> References: <20110404150701.c5c050ff.gerrit@pmp.uni-hannover.de> X-Mailer: Claws Mail 3.7.8cvs9 (GTK+ 2.16.6; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: gerrit@pmp.uni-hannover.de, freebsd-stable@freebsd.org Subject: Re: drives >2TB on mpt device X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 13:36:39 -0000 On Mon, 4 Apr 2011 15:07:01 +0200 Gerrit K=FChn wrote: > Are there any hints how to access the full drive? Am I seeing a > limitation of the controller/firmware or rather of the driver (mpt)? It looks like a known issue: http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/147572 --=20 Bruce Cran From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 14:03:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 870A9106564A for ; Mon, 4 Apr 2011 14:03:34 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 185ED8FC08 for ; Mon, 4 Apr 2011 14:03:33 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.4/8.14.4) with ESMTP id p34E3EW2019852; Mon, 4 Apr 2011 16:03:17 +0200 Received: from pmp.uni-hannover.de (unknown [130.75.117.3]) by www.pmp.uni-hannover.de (Postfix) with SMTP id DD20010A; Mon, 4 Apr 2011 16:03:14 +0200 (CEST) Date: Mon, 4 Apr 2011 16:03:14 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Bruce Cran Message-Id: <20110404160314.aef3ca92.gerrit@pmp.uni-hannover.de> In-Reply-To: <20110404143625.00002eb1@unknown> References: <20110404150701.c5c050ff.gerrit@pmp.uni-hannover.de> <20110404143625.00002eb1@unknown> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 3.0.3 (GTK+ 2.20.1; amd64-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.395186, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.4.4.135425 Cc: freebsd-stable@freebsd.org Subject: Re: drives >2TB on mpt device X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gerrit.kuehn@aei.mpg.de List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 14:03:34 -0000 On Mon, 4 Apr 2011 14:36:25 +0100 Bruce Cran wrote about Re: drives >2TB on mpt device: Hi Bruce, BC> It looks like a known issue: BC> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/147572 Hm, I don't know if this is exactly what I'm seeing here (although the cause may be the same): I do not use mptutil. The controller is "dumb" (without actual raid processor), and I intend to use it with zfs. However, I cannot even get gpart to create a partition larger than 2TB, because mpt comes back with only 2TB after probing the drive. As this is a problem that already exists with 1 drive, I cannot use gstripe or zfs to get around this. But the PR above states that this limitation is already built into mpt, so my only chance is probably to try a different controller/driver (any suggestions for a cheap 8port controller to use with zfs?), or to wait until mpt is updated to support larger drives. Does anyone know if there is already ongoing effort to do this? cu Gerrit From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 15:03:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FA3C106564A for ; Mon, 4 Apr 2011 15:03:03 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id E470F8FC17 for ; Mon, 4 Apr 2011 15:03:02 +0000 (UTC) Received: by qwc9 with SMTP id 9so3934289qwc.13 for ; Mon, 04 Apr 2011 08:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=/eA+Z2M0MzY9qEG9nqzv1kFUEFwDU2hZVjU6Bq28Dwk=; b=UpbT6Go62TajqJ2hW4Zh10qQkzJBganVvdx0x/ZBHDP/fihx8HKd/xW/8SkO96s24r t0f7I5KyFYvFttEO2nu4mue7wJJ5lCSWF0f6xm38nRRKGwzCvjwVZJnomKEHsCypNcMx CJ2wghbVVRrsdL+weufniAtc8SADCIPMWXG48= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=JCpnhzl/diL81mkp6cGr+ZY/RKxZmHfc12oSfuKiNrw3Dg+1Lf7hQPvG0i7H6L1N85 Uty4PDQPEhRaDvoFH/uHxm/U6C6w8HcLabIPczEMWdWemwwzYVG0MLj1ZK8Zp7IM27v0 DHxw1PtMDfUrpDzOKkMknp1RJf7S9i9+uv5J8= MIME-Version: 1.0 Received: by 10.224.212.65 with SMTP id gr1mr5508026qab.382.1301927835958; Mon, 04 Apr 2011 07:37:15 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.229.233.195 with HTTP; Mon, 4 Apr 2011 07:37:15 -0700 (PDT) In-Reply-To: <20110404160314.aef3ca92.gerrit@pmp.uni-hannover.de> References: <20110404150701.c5c050ff.gerrit@pmp.uni-hannover.de> <20110404143625.00002eb1@unknown> <20110404160314.aef3ca92.gerrit@pmp.uni-hannover.de> Date: Mon, 4 Apr 2011 07:37:15 -0700 X-Google-Sender-Auth: c4xjQUpeCinSeQ0r-MgPgBElOto Message-ID: From: Artem Belevich To: gerrit.kuehn@aei.mpg.de Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Bruce Cran , =?ISO-8859-1?Q?Gerrit_K=FChn?= , freebsd-stable@freebsd.org Subject: Re: drives >2TB on mpt device X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 15:03:03 -0000 2011/4/4 Gerrit K=FChn : > On Mon, 4 Apr 2011 14:36:25 +0100 Bruce Cran wrote > about Re: drives >2TB on mpt device: > > Hi Bruce, > > BC> It looks like a known issue: > BC> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/147572 > > Hm, I don't know if this is exactly what I'm seeing here (although the > cause may be the same): > I do not use mptutil. The controller is "dumb" (without actual raid > processor), and I intend to use it with zfs. However, I cannot even get > gpart to create a partition larger than 2TB, because mpt comes back with > only 2TB after probing the drive. As this is a problem that already exist= s > with 1 drive, I cannot use gstripe or zfs to get around this. > But the PR above states that this limitation is already built into mpt, s= o > my only chance is probably to try a different controller/driver (any > suggestions for a cheap 8port controller to use with zfs?), or to wait > until mpt is updated to support larger drives. Does anyone know if there > is already ongoing effort to do this? You're probably out of luck as far as 2Tb+ support for 1068-based HBAs: http://kb.lsi.com/KnowledgebaseArticle16399.aspx Newer controllers based on LSI2008 (mps driver?) should not have that limit= . --Artem From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 15:10:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D470106566C for ; Mon, 4 Apr 2011 15:10:54 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [46.4.210.97]) by mx1.freebsd.org (Postfix) with ESMTP id E1C0B8FC0C for ; Mon, 4 Apr 2011 15:10:53 +0000 (UTC) Received: from localhost (angelica.4lin.net [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id A94612C6A8 for ; Mon, 4 Apr 2011 17:14:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lm6iyKajDogk for ; Mon, 4 Apr 2011 17:14:39 +0200 (CEST) Received: from [130.83.160.152] (pcdenny.rbg.informatik.tu-darmstadt.de [130.83.160.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id 4BAC820081 for ; Mon, 4 Apr 2011 17:14:39 +0200 (CEST) From: Denny Schierz To: freebsd-stable@freebsd.org Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-sTvcvfVti2aMO0NdY5kv" Date: Mon, 04 Apr 2011 17:10:38 +0200 Message-ID: <1301929838.26698.213.camel@pcdenny> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Subject: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 15:10:54 -0000 --=-sTvcvfVti2aMO0NdY5kv Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable hi, I testing the maximum throughput from ISCSI, but I've reached only ~50MB/s (dd if=3D/dev/zero of=3D/dev/da13 bs=3D1M count=3D2048) with crosso= ver 1Gb/s cabel and raw disk. Both machines are FreeBSD 8.2-stable with istgt and the Onboard ISCSI initiator=20 With ZFS as target we loose round about 8-10MB/s. istgt.conf =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [global] Timeout 30 NopInInterval 20 DiscoveryAuthMethod Auto MaxSessions 32 MaxConnections 8 #FirstBurstLength 65536 MaxBurstLength 1048576 MaxRecvDataSegmentLength 262144 # maximum number of sending R2T in each connection # actual number is limited to QueueDepth and MaxCmdSN and ExpCmdSN # 0=3Ddisabled, 1-256=3Dimproves large writing MaxR2T 32 # iSCSI initial parameters negotiate with initiators # NOTE: incorrect values might crash MaxOutstandingR2T 16 DefaultTime2Wait 2 DefaultTime2Retain 60 MaxBurstLength 1048576 [....] [LogicalUnit4] Comment "40GB Disk (iqn.san.foo:40gb)" TargetName 40gb TargetAlias "Data 40GB" Mapping PortalGroup1 InitiatorGroup1 #AuthMethod Auto #AuthGroup AuthGroup2 UnitType Disk UnitInquiry "FreeBSD" "iSCSI Disk" "01234" "10000004" QueueDepth 32 LUN0 Storage /failover/bigPool/disk40gb 40960MB [LogicalUnit5] Comment "2TB Disk (iqn.san.foo:2tb)" TargetName 2tb=20 TargetAlias "Data 2TB" Mapping PortalGroup1 InitiatorGroup1 #AuthMethod Auto #AuthGroup AuthGroup2 UnitType Disk UnitInquiry "FreeBSD" "iSCSI Disk" "01235" "10000005" QueueDepth 32 LUN0 Storage /dev/da12 200480MB =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The raw disks, itself reaches over 150-200MB/s with or without ZFS (raidz2) We have 4GB Ram and 4 x 3Ghz Xeon CPUs on board. I thought, we should reach over 80-100MB/s, so, ISTGT or the Initiator is a bis slow, I think. I've tested just in the moment with Ubuntu 10.10 Initiator and I've got round about 70>MB/s - or without ZFS - constant 80>MB/s, over a regular switched network. Is this the end what we could reach? 'Cause of TCP and ISCSI overhead? What we can't: enable Jumbo frames. Our switches (Cisco catalyst WS-X4515) doesn't support jumbo frames. I've tested Jumbo Frames (9k) over the crossover, but the performance was worse. Round about 20MB/s .... So, does anyone has some hints for me? :-) cu denny --=-sTvcvfVti2aMO0NdY5kv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAk2Z324ACgkQKlzhkqt9P+CH8QCglvQbgHu81wlVSeggbFN/R1cf rsUAnRAV/CNy8LUKp7aKFI+KEiXBM50W =PymT -----END PGP SIGNATURE----- --=-sTvcvfVti2aMO0NdY5kv-- From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 16:04:59 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F188F1065670 for ; Mon, 4 Apr 2011 16:04:59 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id A5BBA8FC1C for ; Mon, 4 Apr 2011 16:04:59 +0000 (UTC) Received: by qyk27 with SMTP id 27so4239520qyk.13 for ; Mon, 04 Apr 2011 09:04:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=4tkkhMXk5jAN8I7nO3njn78hQpXSh8VstVy1auTnMMM=; b=brKVJeCeWWiazv1b6cWOTj65sAe51j1aaoicYnYtRG0j+vOq1kOKAY0PlooGfWo6Ur 9TYbDiG4GlgjKkWUsMXmnQPQmknhoOtt+i6Ui9iCJ51TifudjPVaZdn5htZ495/8L3UW +VOIPNbALCAWSLy1rZtYuRz8x3aljWczxpiqg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=omFM+9UM41q/XHW7CLt5SHiqxkr7DO50HL8J1Fk2D2y+2XIn8SXClnnTFNJQ5hY9u3 AqcXtXQS+tJZStpUo+4AxCVE7NgCGjfQ7dWqPPxY+dSmrguQEKe+JtF9yTH+naDA0ANa gXUOdDPsXf2Xtdu1EomPRldKcldltvDo1Ec8Y= MIME-Version: 1.0 Received: by 10.229.62.6 with SMTP id v6mr5808124qch.223.1301933098464; Mon, 04 Apr 2011 09:04:58 -0700 (PDT) Received: by 10.229.213.143 with HTTP; Mon, 4 Apr 2011 09:04:58 -0700 (PDT) In-Reply-To: <1301929838.26698.213.camel@pcdenny> References: <1301929838.26698.213.camel@pcdenny> Date: Mon, 4 Apr 2011 18:04:58 +0200 Message-ID: From: Claus Guttesen To: Denny Schierz Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org Subject: Re: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 16:05:00 -0000 > I testing the maximum throughput from ISCSI, but I've reached only > ~50MB/s (dd if=/dev/zero of=/dev/da13 bs=1M count=2048) with crossover > 1Gb/s cabel and raw disk. Both machines are FreeBSD 8.2-stable with > istgt and the Onboard ISCSI initiator I've reached almost 118 MB/s but I don't have access to the configuration atm. This was from a windows 7 client. From vmware I've gotten 107 MB/s during a debian 6 server installation. I'll post the settings when I get back to work. You could verify that there are no mismatch between the nics. Have you tried a plain scp of a large file and some rsync of ie. the ports-tree (with distfiles)? -- regards Claus When lenity and cruelty play for a kingdom, the gentler gamester is the soonest winner. Shakespeare twitter.com/kometen From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 18:08:18 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3233A1065670; Mon, 4 Apr 2011 18:08:18 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id AAAF78FC15; Mon, 4 Apr 2011 18:08:17 +0000 (UTC) Received: by gwb15 with SMTP id 15so2676328gwb.13 for ; Mon, 04 Apr 2011 11:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=EfLMaC/AC8XXDP+HKtYpN1Z/ELniFgXm5K1acXZv2rs=; b=bAXSY0zIBAuXiKxHLQ0k6PY27CZ4rTEBxtHVRY/5W1vasifLwpMFFlbY1l6jqeofj9 tzRBbnFvWjeaxB/ScfbigxL0ILFlT9lkh8CcKxtueJ0R3187y9njWRbrfXmTv7cZKC9S iLXk9hUbxfufV8yP4SuDr0c4Vt8RBosannMao= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=GkkPKX+LCkVq8lM1AM3B4EY08TKLTSch/Cej/mMXtTqaNLTJRd1GxaUZmtKq376T1S N1olonMvaJIyLkufy4YL9B23zmSup6hAtkrMSowQv9vj/RtZhV2FJCwLo4tc+IojV7Vw 4FeGcrhGE3qJ3cVcrWxO2LN6rtvl8k7nddvBY= MIME-Version: 1.0 Received: by 10.90.14.39 with SMTP id 39mr4164095agn.127.1301940496804; Mon, 04 Apr 2011 11:08:16 -0700 (PDT) Received: by 10.90.51.14 with HTTP; Mon, 4 Apr 2011 11:08:16 -0700 (PDT) In-Reply-To: <20110402084431.GB1849@garage.freebsd.pl> References: <20110402084431.GB1849@garage.freebsd.pl> Date: Mon, 4 Apr 2011 11:08:16 -0700 Message-ID: From: Freddie Cash To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems , FreeBSD-Current , FreeBSD Stable Subject: Re: Any success stories for HAST + ZFS? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 18:08:18 -0000 On Sat, Apr 2, 2011 at 1:44 AM, Pawel Jakub Dawidek wrote= : > On Thu, Mar 24, 2011 at 01:36:32PM -0700, Freddie Cash wrote: >> [Not sure which list is most appropriate since it's using HAST + ZFS >> on -RELEASE, -STABLE, and -CURRENT. =C2=A0Feel free to trim the CC: on >> replies.] >> >> I'm having a hell of a time making this work on real hardware, and am >> not ruling out hardware issues as yet, but wanted to get some >> reassurance that someone out there is using this combination (FreeBSD >> + HAST + ZFS) successfully, without kernel panics, without core dumps, >> without deadlocks, without issues, etc. =C2=A0I need to know I'm not >> chasing a dead rabbit. > > I just committed a fix for a problem that might look like a deadlock. > With trociny@ patch and my last fix (to GEOM GATE and hastd) do you > still have any issues? Just to confirm, this is commit r220264, 220265, 220266 to -CURRENT? Looking through the commit logs, I don't see any of these MFC'd to -STABLE yet, so I can't test them directly. The storage box that was having the issues is running 8-STABLE r219754 at the moment (with ZFSv28 and Mikolag's ggate patches). I see there have been a lot of hast/ggate-related MFCs in the past week, but they don't include the deadlock patches. Once the deadlock patches above are MFC'd to -STABLE, I can do an upgrade cycle and test them. I do have the previous 9-CURRENT install saved, just nothing to run it on a= tm. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 19:40:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9166310656D6 for ; Mon, 4 Apr 2011 19:40:19 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [46.4.210.97]) by mx1.freebsd.org (Postfix) with ESMTP id 4C6968FC12 for ; Mon, 4 Apr 2011 19:40:18 +0000 (UTC) Received: from localhost (angelica.4lin.net [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id 6C97C2C6E7 for ; Mon, 4 Apr 2011 21:44:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MiHRsXQ_c8bV for ; Mon, 4 Apr 2011 21:44:03 +0200 (CEST) Received: from mac.fritz.box (ip-109-90-188-179.unitymediagroup.de [109.90.188.179]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id 0B47020081 for ; Mon, 4 Apr 2011 21:44:03 +0200 (CEST) From: Denny Schierz Mime-Version: 1.0 (Apple Message framework v1084) Date: Mon, 4 Apr 2011 21:40:09 +0200 In-Reply-To: To: freebsd-stable@freebsd.org References: <1301929838.26698.213.camel@pcdenny> Message-Id: X-Mailer: Apple Mail (2.1084) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 19:40:19 -0000 hi, Am 04.04.2011 um 18:04 schrieb Claus Guttesen: >=20 > I've reached almost 118 MB/s but I don't have access to the > configuration atm. This was from a windows 7 client. =46rom vmware = I've > gotten 107 MB/s during a debian 6 server installation. I'll post the > settings when I get back to work. that would be nice. I will test also a Windows7 client, maybe the = initiator aren't the best, from Ubuntu and FreeBSD. > You could verify that there are no > mismatch between the nics. both are the same hardware, so two Intel e1000 ... > Have you tried a plain scp of a large file and some rsync of ie. the > ports-tree (with distfiles)? nope, nothing should be faster than dd :-) So there is no more protocol = overhead. cu denny From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 20:56:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F1A21065672 for ; Mon, 4 Apr 2011 20:56:33 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id A05D28FC1D for ; Mon, 4 Apr 2011 20:56:30 +0000 (UTC) Received: (qmail 9167 invoked from network); 4 Apr 2011 20:56:29 -0000 Received: from unknown (HELO ?10.50.50.234?) (spawk@64.147.100.2) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 4 Apr 2011 20:56:29 -0000 Message-ID: <4D9A307F.9070408@acm.poly.edu> Date: Mon, 04 Apr 2011 16:56:31 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101106 Thunderbird/3.1.6 MIME-Version: 1.0 To: Kostik Belousov References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> In-Reply-To: <4D974393.80606@acm.poly.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 20:56:33 -0000 On 04/02/11 11:41, Boris Kochergin wrote: > On 04/02/11 11:33, Kostik Belousov wrote: >> On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: >>> Ahoy. This morning, I awoke to the following on one of my servers: >>> >>> pid 59630 (httpd), uid 80, was killed: out of swap space >>> pid 59341 (find), uid 0, was killed: out of swap space >>> pid 23134 (irssi), uid 1001, was killed: out of swap space >>> pid 49332 (sshd), uid 1001, was killed: out of swap space >>> pid 69074 (httpd), uid 0, was killed: out of swap space >>> pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space >>> ... >>> >>> And so on. >>> >>> The machine is: >>> >>> FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu >>> Dec 2 11:39:21 EST 2010 >>> spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 >>> >>> 10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 >>> >>> The memory line from top intrigued me: >>> >>> Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, 605M >>> Free >>> >>> The machine has 8 gigs of memory, and I don't know what all that wired >>> memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 >>> on it which has had a disk in the UNAVAIL state for a few months: >>> >>> # zpool status >>> pool: home >>> state: DEGRADED >>> status: One or more devices could not be used because the label is >>> missing or >>> invalid. Sufficient replicas exist for the pool to continue >>> functioning in a degraded state. >>> action: Replace the device using 'zpool replace'. >>> see: http://www.sun.com/msg/ZFS-8000-4J >>> scrub: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> home DEGRADED 0 0 0 >>> raidz2 DEGRADED 0 0 0 >>> ada0 ONLINE 0 0 0 >>> ada1 ONLINE 0 0 0 >>> ada2 ONLINE 0 0 0 >>> ada3 ONLINE 0 0 0 >>> ada4 ONLINE 0 0 0 >>> ada5 UNAVAIL 0 85 11 experienced I/O >>> failures >>> >>> errors: No known data errors >>> >>> "vmstat -m" and "vmstat -z" output: >>> >>> http://acm.poly.edu/~spawk/vmstat-m.txt >>> http://acm.poly.edu/~spawk/vmstat-z.txt >>> >>> Anyone have a clue? I know it's just going to happen again if I reboot >>> the machine. It is still up in case there are diagnostics for me to >>> run. >> Try r218795. Most likely, your issue is not leak. > > Thanks. Will update to today's 8-STABLE and report back. > > -Boris The problem persists, I'm afraid, and seems to have crept up a lot more quickly than before: # uname -a FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 11:48:43 EDT 2011 spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free Any ideas for a diagnostic recourse? -Boris From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 21:24:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCB2E106566B for ; Mon, 4 Apr 2011 21:24:09 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8BC948FC13 for ; Mon, 4 Apr 2011 21:24:09 +0000 (UTC) Received: by qyk27 with SMTP id 27so4469422qyk.13 for ; Mon, 04 Apr 2011 14:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Kpe38f7Cr6WDYdhmHLnJvSVKcZNFP6Rr3bwcDl7Tfn8=; b=x0c5polilErEcIB53oE7BJgNMgnNLWScelrYyNLJx51yIZiopR6VzeSwBaA4vU6L17 B9g6wU9qtc0Fn4d+2A6whQJcNfF5jNAw7NOLd2WAVgHnEFHa6uzidMnPMgd2EhjgYdAX p0BnNDzmRu7h6/0sQGxXo7bkiERNadU31ftks= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=jW6dkm8wH3W3xQDxMTTcAfN99S5MBX9GxWNJFsqjcDgX5VJraHNeabdlc60KQUGn6D OMkMcITZnmIeg8TrnRhl72ER9CyRnEKKaPqyTesRjBNXLg9wqh7HKkqbA0Dil9cpgVP4 TQ3GGV81o8TxoFx32EA1LZ2dpGNXQ1WxZPHtE= MIME-Version: 1.0 Received: by 10.224.212.65 with SMTP id gr1mr5926255qab.382.1301952247723; Mon, 04 Apr 2011 14:24:07 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.229.233.195 with HTTP; Mon, 4 Apr 2011 14:24:07 -0700 (PDT) In-Reply-To: <4D9A307F.9070408@acm.poly.edu> References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> Date: Mon, 4 Apr 2011 14:24:07 -0700 X-Google-Sender-Auth: cNMb8B_vkvGXkFGmKv3RMNyNNZk Message-ID: From: Artem Belevich To: Boris Kochergin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 21:24:09 -0000 On Mon, Apr 4, 2011 at 1:56 PM, Boris Kochergin wrote: > The problem persists, I'm afraid, and seems to have crept up a lot more > quickly than before: > > # uname -a > FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr =A02 > 11:48:43 EDT 2011 =A0 =A0 spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXOD= US > =A0amd64 > > Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free > > Any ideas for a diagnostic recourse? My bet would be that the wired memory is used by ZFS ARC. In your vmstat-m output you can see that ~2.2G were allocated for 'solaris' subsystem. Due to the fact that ARC allocations tend to be random, we do waste a lot of memory on that. There were few patches floating on stable@ adn fs@ that were supposed to mitigate the issue, but so far there's no good out of the box solution. General advice is to tune ARC so that it works in your case. Key loader tunables are vfs.zfs.arc_min and vfs.zfs.arc_max. Don't set min too high and experimentally set max to the maximum value that does not cause problems. One of the factors that makes things noticeably worse is presence of i/o actifity on non-ZFS filesystems. Regular filesystems cache competes with ZFS for RAM. In the past ZFS used to give up memory way too easily. Currently it's a bit more balanced, but is still far from perfect. By the way, if you don't have on-disk swap configured, I'd recommend adding some. It may help avoiding processes being killed during intermittent memory shortages. If would also help if you could post your /boot/loader.conf and output of "zfs-stats -a" (available in ports). --Artem From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 22:43:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 465A4106564A for ; Mon, 4 Apr 2011 22:43:37 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 042338FC12 for ; Mon, 4 Apr 2011 22:43:36 +0000 (UTC) Received: from omta23.westchester.pa.mail.comcast.net ([76.96.62.74]) by qmta01.westchester.pa.mail.comcast.net with comcast id TaWX1g0041c6gX851ajdfQ; Mon, 04 Apr 2011 22:43:37 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta23.westchester.pa.mail.comcast.net with comcast id Tajb1g00r1t3BNj3jajcxa; Mon, 04 Apr 2011 22:43:36 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 197EF9B429; Mon, 4 Apr 2011 15:43:34 -0700 (PDT) Date: Mon, 4 Apr 2011 15:43:34 -0700 From: Jeremy Chadwick To: Boris Kochergin Message-ID: <20110404224334.GA64297@icarus.home.lan> References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D9A307F.9070408@acm.poly.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 22:43:37 -0000 On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote: > On 04/02/11 11:41, Boris Kochergin wrote: > >On 04/02/11 11:33, Kostik Belousov wrote: > >>On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: > >>>Ahoy. This morning, I awoke to the following on one of my servers: > >>> > >>>pid 59630 (httpd), uid 80, was killed: out of swap space > >>>pid 59341 (find), uid 0, was killed: out of swap space > >>>pid 23134 (irssi), uid 1001, was killed: out of swap space > >>>pid 49332 (sshd), uid 1001, was killed: out of swap space > >>>pid 69074 (httpd), uid 0, was killed: out of swap space > >>>pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space > >>>... > >>> > >>>And so on. > >>> > >>>The machine is: > >>> > >>>FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu > >>>Dec 2 11:39:21 EST 2010 > >>>spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > >>> > >>>10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 > >>> > >>>The memory line from top intrigued me: > >>> > >>>Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, > >>>605M Free > >>> > >>>The machine has 8 gigs of memory, and I don't know what all that wired > >>>memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 > >>>on it which has had a disk in the UNAVAIL state for a few months: > >>> > >>># zpool status > >>> pool: home > >>> state: DEGRADED > >>>status: One or more devices could not be used because the label is > >>>missing or > >>> invalid. Sufficient replicas exist for the pool to continue > >>> functioning in a degraded state. > >>>action: Replace the device using 'zpool replace'. > >>> see: http://www.sun.com/msg/ZFS-8000-4J > >>> scrub: none requested > >>>config: > >>> > >>> NAME STATE READ WRITE CKSUM > >>> home DEGRADED 0 0 0 > >>> raidz2 DEGRADED 0 0 0 > >>> ada0 ONLINE 0 0 0 > >>> ada1 ONLINE 0 0 0 > >>> ada2 ONLINE 0 0 0 > >>> ada3 ONLINE 0 0 0 > >>> ada4 ONLINE 0 0 0 > >>> ada5 UNAVAIL 0 85 11 experienced > >>>I/O failures > >>> > >>>errors: No known data errors > >>> > >>>"vmstat -m" and "vmstat -z" output: > >>> > >>>http://acm.poly.edu/~spawk/vmstat-m.txt > >>>http://acm.poly.edu/~spawk/vmstat-z.txt > >>> > >>>Anyone have a clue? I know it's just going to happen again if I reboot > >>>the machine. It is still up in case there are diagnostics for > >>>me to run. > >>Try r218795. Most likely, your issue is not leak. > > > >Thanks. Will update to today's 8-STABLE and report back. > > > >-Boris > > The problem persists, I'm afraid, and seems to have crept up a lot > more quickly than before: > > # uname -a > FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 > 11:48:43 EDT 2011 > spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > > Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free > > Any ideas for a diagnostic recourse? Can you please provide the details I requested here? Thanks. http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 00:56:17 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD25C106564A for ; Tue, 5 Apr 2011 00:56:17 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6C8CC8FC17 for ; Tue, 5 Apr 2011 00:56:15 +0000 (UTC) Received: (qmail 13395 invoked from network); 5 Apr 2011 00:56:14 -0000 Received: from unknown (HELO ?192.168.0.2?) (spawk@96.224.221.101) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 5 Apr 2011 00:56:14 -0000 Message-ID: <4D9A68AA.6040803@acm.poly.edu> Date: Mon, 04 Apr 2011 20:56:10 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101031 Thunderbird/3.1.6 MIME-Version: 1.0 To: Jeremy Chadwick References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> In-Reply-To: <20110404224334.GA64297@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 00:56:17 -0000 On 04/04/11 18:43, Jeremy Chadwick wrote: > On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote: >> On 04/02/11 11:41, Boris Kochergin wrote: >>> On 04/02/11 11:33, Kostik Belousov wrote: >>>> On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: >>>>> Ahoy. This morning, I awoke to the following on one of my servers: >>>>> >>>>> pid 59630 (httpd), uid 80, was killed: out of swap space >>>>> pid 59341 (find), uid 0, was killed: out of swap space >>>>> pid 23134 (irssi), uid 1001, was killed: out of swap space >>>>> pid 49332 (sshd), uid 1001, was killed: out of swap space >>>>> pid 69074 (httpd), uid 0, was killed: out of swap space >>>>> pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space >>>>> ... >>>>> >>>>> And so on. >>>>> >>>>> The machine is: >>>>> >>>>> FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu >>>>> Dec 2 11:39:21 EST 2010 >>>>> spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 >>>>> >>>>> 10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 >>>>> >>>>> The memory line from top intrigued me: >>>>> >>>>> Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, >>>>> 605M Free >>>>> >>>>> The machine has 8 gigs of memory, and I don't know what all that wired >>>>> memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 >>>>> on it which has had a disk in the UNAVAIL state for a few months: >>>>> >>>>> # zpool status >>>>> pool: home >>>>> state: DEGRADED >>>>> status: One or more devices could not be used because the label is >>>>> missing or >>>>> invalid. Sufficient replicas exist for the pool to continue >>>>> functioning in a degraded state. >>>>> action: Replace the device using 'zpool replace'. >>>>> see: http://www.sun.com/msg/ZFS-8000-4J >>>>> scrub: none requested >>>>> config: >>>>> >>>>> NAME STATE READ WRITE CKSUM >>>>> home DEGRADED 0 0 0 >>>>> raidz2 DEGRADED 0 0 0 >>>>> ada0 ONLINE 0 0 0 >>>>> ada1 ONLINE 0 0 0 >>>>> ada2 ONLINE 0 0 0 >>>>> ada3 ONLINE 0 0 0 >>>>> ada4 ONLINE 0 0 0 >>>>> ada5 UNAVAIL 0 85 11 experienced >>>>> I/O failures >>>>> >>>>> errors: No known data errors >>>>> >>>>> "vmstat -m" and "vmstat -z" output: >>>>> >>>>> http://acm.poly.edu/~spawk/vmstat-m.txt >>>>> http://acm.poly.edu/~spawk/vmstat-z.txt >>>>> >>>>> Anyone have a clue? I know it's just going to happen again if I reboot >>>>> the machine. It is still up in case there are diagnostics for >>>>> me to run. >>>> Try r218795. Most likely, your issue is not leak. >>> Thanks. Will update to today's 8-STABLE and report back. >>> >>> -Boris >> The problem persists, I'm afraid, and seems to have crept up a lot >> more quickly than before: >> >> # uname -a >> FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 >> 11:48:43 EDT 2011 >> spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 >> >> Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free >> >> Any ideas for a diagnostic recourse? > Can you please provide the details I requested here? Thanks. > > http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html > No swap, blank /boot/loader.conf, default /etc/sysctl.conf. I'm going to try this ARC tuning thing. I vaguely recall several claims that tuning wasn't necessary anymore on amd64 systems with the amount of memory mine has, but that's obviously not the case. -Boris From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 01:01:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97BD5106566C for ; Tue, 5 Apr 2011 01:01:50 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id 7C0E68FC08 for ; Tue, 5 Apr 2011 01:01:50 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta14.emeryville.ca.mail.comcast.net with comcast id Td1b1g0021vN32cAEd1qWi; Tue, 05 Apr 2011 01:01:50 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.emeryville.ca.mail.comcast.net with comcast id Td1o1g00j1t3BNj8id1ohW; Tue, 05 Apr 2011 01:01:49 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 2EE849B429; Mon, 4 Apr 2011 18:01:48 -0700 (PDT) Date: Mon, 4 Apr 2011 18:01:48 -0700 From: Jeremy Chadwick To: Boris Kochergin Message-ID: <20110405010148.GA67821@icarus.home.lan> References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D9A68AA.6040803@acm.poly.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 01:01:50 -0000 On Mon, Apr 04, 2011 at 08:56:10PM -0400, Boris Kochergin wrote: > On 04/04/11 18:43, Jeremy Chadwick wrote: > >On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote: > >>On 04/02/11 11:41, Boris Kochergin wrote: > >>>On 04/02/11 11:33, Kostik Belousov wrote: > >>>>On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: > >>>>>Ahoy. This morning, I awoke to the following on one of my servers: > >>>>> > >>>>>pid 59630 (httpd), uid 80, was killed: out of swap space > >>>>>pid 59341 (find), uid 0, was killed: out of swap space > >>>>>pid 23134 (irssi), uid 1001, was killed: out of swap space > >>>>>pid 49332 (sshd), uid 1001, was killed: out of swap space > >>>>>pid 69074 (httpd), uid 0, was killed: out of swap space > >>>>>pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space > >>>>>... > >>>>> > >>>>>And so on. > >>>>> > >>>>>The machine is: > >>>>> > >>>>>FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu > >>>>>Dec 2 11:39:21 EST 2010 > >>>>>spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > >>>>> > >>>>>10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 > >>>>> > >>>>>The memory line from top intrigued me: > >>>>> > >>>>>Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, > >>>>>605M Free > >>>>> > >>>>>The machine has 8 gigs of memory, and I don't know what all that wired > >>>>>memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 > >>>>>on it which has had a disk in the UNAVAIL state for a few months: > >>>>> > >>>>># zpool status > >>>>> pool: home > >>>>> state: DEGRADED > >>>>>status: One or more devices could not be used because the label is > >>>>>missing or > >>>>> invalid. Sufficient replicas exist for the pool to continue > >>>>> functioning in a degraded state. > >>>>>action: Replace the device using 'zpool replace'. > >>>>> see: http://www.sun.com/msg/ZFS-8000-4J > >>>>> scrub: none requested > >>>>>config: > >>>>> > >>>>> NAME STATE READ WRITE CKSUM > >>>>> home DEGRADED 0 0 0 > >>>>> raidz2 DEGRADED 0 0 0 > >>>>> ada0 ONLINE 0 0 0 > >>>>> ada1 ONLINE 0 0 0 > >>>>> ada2 ONLINE 0 0 0 > >>>>> ada3 ONLINE 0 0 0 > >>>>> ada4 ONLINE 0 0 0 > >>>>> ada5 UNAVAIL 0 85 11 experienced > >>>>>I/O failures > >>>>> > >>>>>errors: No known data errors > >>>>> > >>>>>"vmstat -m" and "vmstat -z" output: > >>>>> > >>>>>http://acm.poly.edu/~spawk/vmstat-m.txt > >>>>>http://acm.poly.edu/~spawk/vmstat-z.txt > >>>>> > >>>>>Anyone have a clue? I know it's just going to happen again if I reboot > >>>>>the machine. It is still up in case there are diagnostics for > >>>>>me to run. > >>>>Try r218795. Most likely, your issue is not leak. > >>>Thanks. Will update to today's 8-STABLE and report back. > >>> > >>>-Boris > >>The problem persists, I'm afraid, and seems to have crept up a lot > >>more quickly than before: > >> > >># uname -a > >>FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 > >>11:48:43 EDT 2011 > >>spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > >> > >>Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free > >> > >>Any ideas for a diagnostic recourse? > >Can you please provide the details I requested here? Thanks. > > > >http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html > > > > No swap, blank /boot/loader.conf, default /etc/sysctl.conf. I'm > going to try this ARC tuning thing. I vaguely recall several claims > that tuning wasn't necessary anymore on amd64 systems with the > amount of memory mine has, but that's obviously not the case. Given that you don't have swap (again: very, very bad idea), your applications crashing due to there not being any swap space is expected: no place to swap them out to. All you should need to set, in /boot/loader.conf, is: vfs.zfs.arc_max For example, if you want to limit the ARC to only use up to 2GB of RAM: vfs.zfs.arc_max="2048M" This would reserve (on an 8GB machine) approximately ~6GB of RAM for userland applications, the kernel, network buffers/mbufs, etc.. Finally, please note that most of the stuff you'll read online for ZFS tuning on FreeBSD is outdated with 8.2. E.g. you should not need to set vm.kmem_size and you should never need to adjust vm.kmem_size_max. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 01:10:27 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BF971065673 for ; Tue, 5 Apr 2011 01:10:27 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 029378FC19 for ; Tue, 5 Apr 2011 01:10:26 +0000 (UTC) Received: (qmail 13668 invoked from network); 5 Apr 2011 01:10:20 -0000 Received: from unknown (HELO ?192.168.0.2?) (spawk@96.224.221.101) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 5 Apr 2011 01:10:20 -0000 Message-ID: <4D9A6BF7.5000106@acm.poly.edu> Date: Mon, 04 Apr 2011 21:10:15 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101031 Thunderbird/3.1.6 MIME-Version: 1.0 To: Jeremy Chadwick References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> In-Reply-To: <20110405010148.GA67821@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 01:10:27 -0000 On 04/04/11 21:01, Jeremy Chadwick wrote: > On Mon, Apr 04, 2011 at 08:56:10PM -0400, Boris Kochergin wrote: >> On 04/04/11 18:43, Jeremy Chadwick wrote: >>> On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote: >>>> On 04/02/11 11:41, Boris Kochergin wrote: >>>>> On 04/02/11 11:33, Kostik Belousov wrote: >>>>>> On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: >>>>>>> Ahoy. This morning, I awoke to the following on one of my servers: >>>>>>> >>>>>>> pid 59630 (httpd), uid 80, was killed: out of swap space >>>>>>> pid 59341 (find), uid 0, was killed: out of swap space >>>>>>> pid 23134 (irssi), uid 1001, was killed: out of swap space >>>>>>> pid 49332 (sshd), uid 1001, was killed: out of swap space >>>>>>> pid 69074 (httpd), uid 0, was killed: out of swap space >>>>>>> pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space >>>>>>> ... >>>>>>> >>>>>>> And so on. >>>>>>> >>>>>>> The machine is: >>>>>>> >>>>>>> FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu >>>>>>> Dec 2 11:39:21 EST 2010 >>>>>>> spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 >>>>>>> >>>>>>> 10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 >>>>>>> >>>>>>> The memory line from top intrigued me: >>>>>>> >>>>>>> Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, >>>>>>> 605M Free >>>>>>> >>>>>>> The machine has 8 gigs of memory, and I don't know what all that wired >>>>>>> memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 >>>>>>> on it which has had a disk in the UNAVAIL state for a few months: >>>>>>> >>>>>>> # zpool status >>>>>>> pool: home >>>>>>> state: DEGRADED >>>>>>> status: One or more devices could not be used because the label is >>>>>>> missing or >>>>>>> invalid. Sufficient replicas exist for the pool to continue >>>>>>> functioning in a degraded state. >>>>>>> action: Replace the device using 'zpool replace'. >>>>>>> see: http://www.sun.com/msg/ZFS-8000-4J >>>>>>> scrub: none requested >>>>>>> config: >>>>>>> >>>>>>> NAME STATE READ WRITE CKSUM >>>>>>> home DEGRADED 0 0 0 >>>>>>> raidz2 DEGRADED 0 0 0 >>>>>>> ada0 ONLINE 0 0 0 >>>>>>> ada1 ONLINE 0 0 0 >>>>>>> ada2 ONLINE 0 0 0 >>>>>>> ada3 ONLINE 0 0 0 >>>>>>> ada4 ONLINE 0 0 0 >>>>>>> ada5 UNAVAIL 0 85 11 experienced >>>>>>> I/O failures >>>>>>> >>>>>>> errors: No known data errors >>>>>>> >>>>>>> "vmstat -m" and "vmstat -z" output: >>>>>>> >>>>>>> http://acm.poly.edu/~spawk/vmstat-m.txt >>>>>>> http://acm.poly.edu/~spawk/vmstat-z.txt >>>>>>> >>>>>>> Anyone have a clue? I know it's just going to happen again if I reboot >>>>>>> the machine. It is still up in case there are diagnostics for >>>>>>> me to run. >>>>>> Try r218795. Most likely, your issue is not leak. >>>>> Thanks. Will update to today's 8-STABLE and report back. >>>>> >>>>> -Boris >>>> The problem persists, I'm afraid, and seems to have crept up a lot >>>> more quickly than before: >>>> >>>> # uname -a >>>> FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 >>>> 11:48:43 EDT 2011 >>>> spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 >>>> >>>> Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free >>>> >>>> Any ideas for a diagnostic recourse? >>> Can you please provide the details I requested here? Thanks. >>> >>> http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html >>> >> No swap, blank /boot/loader.conf, default /etc/sysctl.conf. I'm >> going to try this ARC tuning thing. I vaguely recall several claims >> that tuning wasn't necessary anymore on amd64 systems with the >> amount of memory mine has, but that's obviously not the case. > Given that you don't have swap (again: very, very bad idea), your > applications crashing due to there not being any swap space is expected: > no place to swap them out to. > > All you should need to set, in /boot/loader.conf, is: > > vfs.zfs.arc_max > > For example, if you want to limit the ARC to only use up to 2GB of RAM: > > vfs.zfs.arc_max="2048M" Thanks. I will attempt just this and report back. -Boris > This would reserve (on an 8GB machine) approximately ~6GB of RAM for > userland applications, the kernel, network buffers/mbufs, etc.. > > Finally, please note that most of the stuff you'll read online for ZFS > tuning on FreeBSD is outdated with 8.2. E.g. you should not need to set > vm.kmem_size and you should never need to adjust vm.kmem_size_max. > From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 02:15:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71DAB1065674 for ; Tue, 5 Apr 2011 02:15:01 +0000 (UTC) (envelope-from spork@bway.net) Received: from xena.bway.net (xena.bway.net [216.220.96.26]) by mx1.freebsd.org (Postfix) with ESMTP id 047F48FC08 for ; Tue, 5 Apr 2011 02:15:00 +0000 (UTC) Received: (qmail 49757 invoked by uid 0); 5 Apr 2011 01:48:19 -0000 Received: from smtp.bway.net (216.220.96.25) by xena.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 5 Apr 2011 01:48:19 -0000 Received: (qmail 49743 invoked by uid 90); 5 Apr 2011 01:48:19 -0000 Received: from unknown (HELO hotlap.nat.fasttrackmonkey.com) (spork@96.57.144.66) by smtp.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 5 Apr 2011 01:48:19 -0000 Date: Mon, 4 Apr 2011 21:48:17 -0400 (EDT) From: Charles Sprickman X-X-Sender: spork@hotlap.local To: Jeremy Chadwick In-Reply-To: <20110405010148.GA67821@icarus.home.lan> Message-ID: References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kostik Belousov , Boris Kochergin , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 02:15:01 -0000 On Mon, 4 Apr 2011, Jeremy Chadwick wrote: > Finally, please note that most of the stuff you'll read online for ZFS > tuning on FreeBSD is outdated with 8.2. E.g. you should not need to set > vm.kmem_size and you should never need to adjust vm.kmem_size_max. Slight tangent, does this apply to i386 as well or just amd64? Someone should open the zfs wiki page to a broader range of editors I think - it seems like this mailing list is the only decent reference for tuning these days. Charles > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP 4BD6C0CB | > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 02:20:18 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E17E9106566B for ; Tue, 5 Apr 2011 02:20:18 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 88F0B8FC16 for ; Tue, 5 Apr 2011 02:20:18 +0000 (UTC) Received: from omta15.westchester.pa.mail.comcast.net ([76.96.62.87]) by qmta01.westchester.pa.mail.comcast.net with comcast id Tdc31g0071swQuc51eLJCC; Tue, 05 Apr 2011 02:20:18 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta15.westchester.pa.mail.comcast.net with comcast id TeLG1g00t1t3BNj3beLHTu; Tue, 05 Apr 2011 02:20:18 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 7C4C49B42B; Mon, 4 Apr 2011 19:20:15 -0700 (PDT) Date: Mon, 4 Apr 2011 19:20:15 -0700 From: Jeremy Chadwick To: Charles Sprickman Message-ID: <20110405022015.GA69120@icarus.home.lan> References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , Boris Kochergin , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 02:20:19 -0000 On Mon, Apr 04, 2011 at 09:48:17PM -0400, Charles Sprickman wrote: > On Mon, 4 Apr 2011, Jeremy Chadwick wrote: > > >Finally, please note that most of the stuff you'll read online for ZFS > >tuning on FreeBSD is outdated with 8.2. E.g. you should not need to set > >vm.kmem_size and you should never need to adjust vm.kmem_size_max. > > Slight tangent, does this apply to i386 as well or just amd64? > > Someone should open the zfs wiki page to a broader range of editors > I think - it seems like this mailing list is the only decent > reference for tuning these days. Off the top of my head: wouldn't know, I tend not to run i386 anywhere anymore. I'm not sure what all i386 requires, other than adjusting KVA_PAGES in one's kernel config. Sadly I'd probably refer to the Wiki.... oops. ;-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 02:52:55 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FC28106564A for ; Tue, 5 Apr 2011 02:52:55 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2D6BB8FC15 for ; Tue, 5 Apr 2011 02:52:53 +0000 (UTC) Received: (qmail 15451 invoked from network); 5 Apr 2011 02:52:52 -0000 Received: from unknown (HELO ?192.168.0.2?) (spawk@96.224.221.101) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 5 Apr 2011 02:52:52 -0000 Message-ID: <4D9A8400.1030006@acm.poly.edu> Date: Mon, 04 Apr 2011 22:52:48 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101031 Thunderbird/3.1.6 MIME-Version: 1.0 To: FreeBSD-STABLE Mailing List References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> <4D9A6BF7.5000106@acm.poly.edu> In-Reply-To: <4D9A6BF7.5000106@acm.poly.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 02:52:55 -0000 So, vfs.zfs.arc_max="2048M" in /boot/loader.conf was indeed apparently all that was necessary to bring the situation under control. I remember it being a lot more nightmarish, so it's nice to see that it's improved. Thanks for everyone's advice. Per an earlier request, here is the output of "zfs-stats -a" right now (not when the system is running out of memory, but perhaps still interesting): # zfs-stats -a ------------------------------------------------------------------------ ZFS Subsystem Report Mon Apr 4 22:43:43 2011 ------------------------------------------------------------------------ System Information: Kernel Version: 802502 (osreldate) Hardware Platform: amd64 Processor Architecture: amd64 FreeBSD 8.2-STABLE #3: Sat Apr 2 11:48:43 EDT 2011 spawk 10:43PM up 1:32, 3 users, load averages: 0.13, 0.09, 0.07 ------------------------------------------------------------------------ System Memory Statistics: Physical Memory: 8181.32M Kernel Memory: 2134.72M DATA: 99.65% 2127.20M TEXT: 0.35% 7.52M ------------------------------------------------------------------------ ZFS pool information: Storage pool Version (spa): 15 Filesystem Version (zpl): 4 ------------------------------------------------------------------------ ARC Misc: Deleted: 583540 Recycle Misses: 355 Mutex Misses: 11 Evict Skips: 11 ARC Size: Current Size (arcsize): 100.00% 2048.07M Target Size (Adaptive, c): 100.00% 2048.00M Min Size (Hard Limit, c_min): 12.50% 256.00M Max Size (High Water, c_max): ~8:1 2048.00M ARC Size Breakdown: Recently Used Cache Size (p): 93.32% 1911.27M Frequent;y Used Cache Size (arcsize-p): 6.68% 136.80M ARC Hash Breakdown: Elements Max: 38168 Elements Current: 99.18% 37856 Collisions: 127822 Chain Max: 5 Chains: 4567 ARC Eviction Statistics: Evicts Total: 80383607808 Evicts Eligible for L2: 9.95% 8001851392 Evicts Ineligible for L2: 90.05% 72381756416 Evicts Cached to L2: 0 ARC Efficiency: Cache Access Total: 1439376 Cache Hit Ratio: 55.64% 800797 Cache Miss Ratio: 44.36% 638579 Actual Hit Ratio: 51.12% 735809 Data Demand Efficiency: 97.21% Data Prefetch Efficiency: 8.78% CACHE HITS BY CACHE LIST: Anonymously Used: 5.96% 47738 Most Recently Used (mru): 28.97% 232003 Most Frequently Used (mfu): 62.91% 503806 MRU Ghost (mru_ghost): 0.74% 5933 MFU Ghost (mfu_ghost): 1.41% 11317 CACHE HITS BY DATA TYPE: Demand Data: 31.17% 249578 Prefetch Data: 7.47% 59804 Demand Metadata: 60.33% 483145 Prefetch Metadata: 1.03% 8270 CACHE MISSES BY DATA TYPE: Demand Data: 1.12% 7162 Prefetch Data: 97.31% 621373 Demand Metadata: 0.91% 5805 Prefetch Metadata: 0.66% 4239 ------------------------------------------------------------------------ VDEV Cache Summary: Access Total: 32382 Hits Ratio: 21.62% 7001 Miss Ratio: 78.38% 25381 Delegations: 11361 ------------------------------------------------------------------------ File-Level Prefetch Stats (DMU): DMU Efficiency: Access Total: 640507 Hit Ratio: 92.40% 591801 Miss Ratio: 7.60% 48706 Colinear Access Total: 48706 Colinear Hit Ratio: 0.20% 99 Colinear Miss Ratio: 99.80% 48607 Stride Access Total: 533456 Stride Hit Ratio: 99.97% 533296 Stride Miss Ratio: 0.03% 160 DMU misc: Reclaim successes: 8857 Reclaim failures: 39750 Stream resets: 126 Stream noresets: 58504 Bogus streams: 0 ------------------------------------------------------------------------ ZFS Tunable (sysctl): kern.maxusers=384 vfs.zfs.l2c_only_size=0 vfs.zfs.mfu_ghost_data_lsize=1956972544 vfs.zfs.mfu_ghost_metadata_lsize=12218880 vfs.zfs.mfu_ghost_size=1969191424 vfs.zfs.mfu_data_lsize=103428096 vfs.zfs.mfu_metadata_lsize=17053184 vfs.zfs.mfu_size=124921344 vfs.zfs.mru_ghost_data_lsize=133824512 vfs.zfs.mru_ghost_metadata_lsize=40669184 vfs.zfs.mru_ghost_size=174493696 vfs.zfs.mru_data_lsize=1984430080 vfs.zfs.mru_metadata_lsize=14490112 vfs.zfs.mru_size=2005246464 vfs.zfs.anon_data_lsize=0 vfs.zfs.anon_metadata_lsize=0 vfs.zfs.anon_size=0 vfs.zfs.l2arc_norw=1 vfs.zfs.l2arc_feed_again=1 vfs.zfs.l2arc_noprefetch=0 vfs.zfs.l2arc_feed_min_ms=200 vfs.zfs.l2arc_feed_secs=1 vfs.zfs.l2arc_headroom=2 vfs.zfs.l2arc_write_boost=8388608 vfs.zfs.l2arc_write_max=8388608 vfs.zfs.arc_meta_limit=536870912 vfs.zfs.arc_meta_used=59699288 vfs.zfs.mdcomp_disable=0 vfs.zfs.arc_min=268435456 vfs.zfs.arc_max=2147483648 vfs.zfs.zfetch.array_rd_sz=1048576 vfs.zfs.zfetch.block_cap=256 vfs.zfs.zfetch.min_sec_reap=2 vfs.zfs.zfetch.max_streams=8 vfs.zfs.prefetch_disable=0 vfs.zfs.check_hostid=1 vfs.zfs.recover=0 vfs.zfs.txg.write_limit_override=0 vfs.zfs.txg.synctime=5 vfs.zfs.txg.timeout=30 vfs.zfs.scrub_limit=10 vfs.zfs.vdev.cache.bshift=16 vfs.zfs.vdev.cache.size=10485760 vfs.zfs.vdev.cache.max=16384 vfs.zfs.vdev.aggregation_limit=131072 vfs.zfs.vdev.ramp_rate=2 vfs.zfs.vdev.time_shift=6 vfs.zfs.vdev.min_pending=4 vfs.zfs.vdev.max_pending=10 vfs.zfs.cache_flush_disable=0 vfs.zfs.zil_disable=0 vfs.zfs.zio.use_uma=0 vfs.zfs.version.zpl=4 vfs.zfs.version.spa=15 vfs.zfs.version.dmu_backup_stream=1 vfs.zfs.version.dmu_backup_header=2 vfs.zfs.version.acl=1 vfs.zfs.debug=0 vfs.zfs.super_owner=0 vm.kmem_size=8294764544 vm.kmem_size_scale=1 vm.kmem_size_min=0 vm.kmem_size_max=329853485875 ------------------------------------------------------------------------ -Boris From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 07:22:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56DD0106566C for ; Tue, 5 Apr 2011 07:22:29 +0000 (UTC) (envelope-from lists@c0mplx.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) by mx1.freebsd.org (Postfix) with ESMTP id 18B1E8FC08 for ; Tue, 5 Apr 2011 07:22:29 +0000 (UTC) Received: from pi by home.opsec.eu with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Q70ap-000Nvo-SO for freebsd-stable@freebsd.org; Tue, 05 Apr 2011 09:22:27 +0200 Date: Tue, 5 Apr 2011 09:22:27 +0200 From: Kurt Jaeger To: freebsd-stable@freebsd.org Message-ID: <20110405072227.GE34314@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: Filesystem strangeness X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 07:22:29 -0000 Hi! On a FreeBSD 8.0-RELEASE i386 (GENERIC kernel): # cd /usr/src/sys/boot/efi/libefi/ # ls -l ls: efinet.c: Bad file descriptor total 38 -rw-r--r-- 1 root wheel 461 Oct 25 2009 Makefile -rw-r--r-- 1 root wheel 1777 Oct 25 2009 delay.c -rw-r--r-- 1 root wheel 2682 Oct 25 2009 efi_console.c -rw-r--r-- 1 root wheel 9957 Oct 25 2009 efifs.c -rw-r--r-- 1 root wheel 2269 Oct 25 2009 errno.c -rw-r--r-- 1 root wheel 2536 Oct 25 2009 handles.c -rw-r--r-- 1 root wheel 5712 Oct 25 2009 libefi.c -rw-r--r-- 1 root wheel 6087 Oct 25 2009 time.c # df /usr Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad4s1e 101554150 3143488157741142862 -3143488157647713044 3364544879817% /usr gk-hwkrt-new# The last reboot did not do any fsck, smartctl does not complain. So, what can I do to fix this ? -- pi@opsec.eu +49 171 3101372 9 years to go ! From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 07:29:59 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9647C106564A for ; Tue, 5 Apr 2011 07:29:59 +0000 (UTC) (envelope-from lists@c0mplx.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) by mx1.freebsd.org (Postfix) with ESMTP id 579898FC0A for ; Tue, 5 Apr 2011 07:29:59 +0000 (UTC) Received: from pi by home.opsec.eu with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Q70i7-000O3E-CI for freebsd-stable@freebsd.org; Tue, 05 Apr 2011 09:29:59 +0200 Date: Tue, 5 Apr 2011 09:29:59 +0200 From: Kurt Jaeger To: freebsd-stable@freebsd.org Message-ID: <20110405072959.GF34314@home.opsec.eu> References: <20110405072227.GE34314@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110405072227.GE34314@home.opsec.eu> Subject: Re: Filesystem strangeness X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 07:29:59 -0000 Hi! > On a FreeBSD 8.0-RELEASE i386 (GENERIC kernel): [...] > /dev/ad4s1e 101554150 3143488157741142862 -3143488157647713044 3364544879817% /usr > The last reboot did not do any fsck, smartctl does not complain. > So, what can I do to fix this ? The last reboot *did* indeed fsck, sorry for the confusion. -- pi@opsec.eu +49 171 3101372 9 years to go ! From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 07:49:30 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D3C7106566B for ; Tue, 5 Apr 2011 07:49:30 +0000 (UTC) (envelope-from eugen@eg.sd.rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221]) by mx1.freebsd.org (Postfix) with ESMTP id AF9218FC15 for ; Tue, 5 Apr 2011 07:49:29 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p357UuJn065883; Tue, 5 Apr 2011 14:30:56 +0700 (NOVST) (envelope-from eugen@eg.sd.rdtc.ru) Received: (from eugen@localhost) by eg.sd.rdtc.ru (8.14.4/8.14.4/Submit) id p357UpFO065882; Tue, 5 Apr 2011 14:30:51 +0700 (NOVST) (envelope-from eugen) Date: Tue, 5 Apr 2011 14:30:51 +0700 From: Eugene Grosbein To: Kurt Jaeger Message-ID: <20110405073051.GA65834@rdtc.ru> References: <20110405072227.GE34314@home.opsec.eu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110405072227.GE34314@home.opsec.eu> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: Filesystem strangeness X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 07:49:30 -0000 On Tue, Apr 05, 2011 at 09:22:27AM +0200, Kurt Jaeger wrote: > On a FreeBSD 8.0-RELEASE i386 (GENERIC kernel): > > # cd /usr/src/sys/boot/efi/libefi/ > # ls -l > ls: efinet.c: Bad file descriptor Your file system it broken and needs fsck. > total 38 > -rw-r--r-- 1 root wheel 461 Oct 25 2009 Makefile > -rw-r--r-- 1 root wheel 1777 Oct 25 2009 delay.c > -rw-r--r-- 1 root wheel 2682 Oct 25 2009 efi_console.c > -rw-r--r-- 1 root wheel 9957 Oct 25 2009 efifs.c > -rw-r--r-- 1 root wheel 2269 Oct 25 2009 errno.c > -rw-r--r-- 1 root wheel 2536 Oct 25 2009 handles.c > -rw-r--r-- 1 root wheel 5712 Oct 25 2009 libefi.c > -rw-r--r-- 1 root wheel 6087 Oct 25 2009 time.c > # df /usr > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/ad4s1e 101554150 3143488157741142862 -3143488157647713044 3364544879817% /usr > gk-hwkrt-new# > > The last reboot did not do any fsck, smartctl does not complain. > So, what can I do to fix this ? Go to single user mode, unmount /usr and run "fsck -y /usr" Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 07:54:30 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 394FC1065674 for ; Tue, 5 Apr 2011 07:54:30 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id 1FD188FC0C for ; Tue, 5 Apr 2011 07:54:29 +0000 (UTC) Received: from omta20.emeryville.ca.mail.comcast.net ([76.96.30.87]) by qmta01.emeryville.ca.mail.comcast.net with comcast id Tjts1g0021smiN4A1juVDu; Tue, 05 Apr 2011 07:54:29 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta20.emeryville.ca.mail.comcast.net with comcast id TjuT1g0021t3BNj8gjuTDJ; Tue, 05 Apr 2011 07:54:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DF2519B429; Tue, 5 Apr 2011 00:54:26 -0700 (PDT) Date: Tue, 5 Apr 2011 00:54:26 -0700 From: Jeremy Chadwick To: Kurt Jaeger Message-ID: <20110405075426.GA74151@icarus.home.lan> References: <20110405072227.GE34314@home.opsec.eu> <20110405072959.GF34314@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110405072959.GF34314@home.opsec.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Filesystem strangeness X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 07:54:30 -0000 On Tue, Apr 05, 2011 at 09:29:59AM +0200, Kurt Jaeger wrote: > Hi! > > > On a FreeBSD 8.0-RELEASE i386 (GENERIC kernel): > [...] > > /dev/ad4s1e 101554150 3143488157741142862 -3143488157647713044 3364544879817% /usr > > > The last reboot did not do any fsck, smartctl does not complain. > > So, what can I do to fix this ? > > The last reboot *did* indeed fsck, sorry for the confusion. Stuff that might help: 1) Are you using background_fsck="no" in rc.conf? (I think you are, which is good/wise, but please verify regardless) 2) Tried booting into single-user to run "fsck -f /dev/ad4" anyway? 3) Full output from the fsck which did run 4) Output from "bsdlabel -A ad4s1" 5) Output from "gpart ad4" 6) Output from "gpart ad4s1" 7) Output from "ffsinfo ad4s1" 8) Output from "ffsinfo ad4s1e" 9) Output from "tunefs -p ad4s1e" 10) Output from "smartctl -a /dev/ad4" anyway (for my review)? I think that's about all I can think of. :-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 08:01:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B7A1106566B for ; Tue, 5 Apr 2011 08:01:07 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 04EDD8FC08 for ; Tue, 5 Apr 2011 08:01:06 +0000 (UTC) Received: by qwc9 with SMTP id 9so61998qwc.13 for ; Tue, 05 Apr 2011 01:01:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=BucHkn67DAeqHn8vzpjXOYInDosyp6CfXTZM6qz7Qxs=; b=pGZKJul4lYeId33sQgpboHAjwBB1Yf8MZT68w1cMDbpdOgXoQ+YMnmItP7oqfLUga1 oorqK+uLDPRYvwnWiZ+ZHi5Xh8HSVErCL2lM9I0xE1xeQe6KSuxicgc3L2GPA7nrnm37 d0M0EY+bnQsfBWS3ugExpnGce7hQZwiD/GoTI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=R42IUFp2ExSbHGrHRGuvtn6qMdLFvlXZcUw8cBPUDfWDSa1V4/V1Wd09X3K4vp3ufU D2532Ckl1lhWyeuzzV4uAnQ2mhj8AUVDQTSYfLURZhXdelMOI/6lCI+p8fkNujIIpNSz lSq26SnOkaQuveb96w9npSlpCAlptkebjXaio= MIME-Version: 1.0 Received: by 10.229.214.204 with SMTP id hb12mr6182778qcb.261.1301990466086; Tue, 05 Apr 2011 01:01:06 -0700 (PDT) Received: by 10.229.213.143 with HTTP; Tue, 5 Apr 2011 01:01:06 -0700 (PDT) In-Reply-To: References: <1301929838.26698.213.camel@pcdenny> Date: Tue, 5 Apr 2011 10:01:06 +0200 Message-ID: From: Claus Guttesen To: Denny Schierz Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org Subject: Re: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 08:01:07 -0000 >> I've reached almost 118 MB/s but I don't have access to the >> configuration atm. This was from a windows 7 client. From vmware I've >> gotten 107 MB/s during a debian 6 server installation. I'll post the >> settings when I get back to work. > > that would be nice. I will test also a Windows7 client, maybe the initiator aren't the best, from Ubuntu and FreeBSD. The only setting I changed was: QueueDepth 64 This was commented out (as I recall it). Other (global) settings related to iscsi has been left at their default values. One LUN example: [LogicalUnit1] Comment "LogicalUnit1 Sample" TargetName disk1 TargetAlias "Data Disk1" Mapping PortalGroup1 InitiatorGroup1 AuthMethod CHAP AuthGroup AuthGroup1 UseDigest Auto UnitType Disk QueueDepth 64 LUN0 Storage /dev/zvol/data/iscsi/disk1 Auto hth -- regards Claus When lenity and cruelty play for a kingdom, the gentler gamester is the soonest winner. Shakespeare twitter.com/kometen From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 08:09:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 081201065670 for ; Tue, 5 Apr 2011 08:09:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id E38A18FC0C for ; Tue, 5 Apr 2011 08:09:40 +0000 (UTC) Received: from omta17.emeryville.ca.mail.comcast.net ([76.96.30.73]) by qmta14.emeryville.ca.mail.comcast.net with comcast id Tk7P1g0021afHeLAEk9gKk; Tue, 05 Apr 2011 08:09:40 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta17.emeryville.ca.mail.comcast.net with comcast id Tk9f1g0071t3BNj8dk9fuU; Tue, 05 Apr 2011 08:09:40 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 7654B9B429; Tue, 5 Apr 2011 01:09:39 -0700 (PDT) Date: Tue, 5 Apr 2011 01:09:39 -0700 From: Jeremy Chadwick To: Kurt Jaeger Message-ID: <20110405080939.GA74795@icarus.home.lan> References: <20110405072227.GE34314@home.opsec.eu> <20110405072959.GF34314@home.opsec.eu> <20110405075426.GA74151@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110405075426.GA74151@icarus.home.lan> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Filesystem strangeness X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 08:09:41 -0000 On Tue, Apr 05, 2011 at 12:54:26AM -0700, Jeremy Chadwick wrote: > 2) Tried booting into single-user to run "fsck -f /dev/ad4" anyway? Sorry, this should have been "fsck -f /dev/ad4s1e". Derp. :-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 12:05:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED327106566B; Tue, 5 Apr 2011 12:05:09 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id AF8C48FC18; Tue, 5 Apr 2011 12:05:08 +0000 (UTC) Received: by wwc33 with SMTP id 33so328233wwc.31 for ; Tue, 05 Apr 2011 05:05:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:organization:references :sender:date:in-reply-to:message-id:user-agent:mime-version :content-type; bh=Ug//cj8gOU7r+ZBzLxHLlM/jpS1tdTg1rmEJuXBLH0M=; b=XPhIxNPoY/e/Iq3fgHivi0QtEEVRUFHNKDsy9D+DEwCQ4iQT1HCP5WqPm5LWxGuZAS 03dhLm8tPz8YaLglOZB1MKZVcpRkYA2jD4adHRDy+OzW19QLeIGz/87uNvUZZjXMhIlc I0yjKPHr5H2GHpAcT7iLC7xTu0ygeyfVRmduU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:organization:references:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=KY4YPv8Gxr+RfycpX+td8nlSA+uaqmZAZVslWW2/PfuCh+3CTuGUxXQoQlKMBhJ0Fr ekQd/zRk4be8E1Z6zaZQsbrx/1d0BWHWV9ZzehM7DS7xkYuKWcpGZgTOTIeeJKJ7r54c fKSO9hT/WYAidtfh9sFN3e1t4Rq/6Is1bRdFo= Received: by 10.227.151.15 with SMTP id a15mr8258351wbw.184.1302005107660; Tue, 05 Apr 2011 05:05:07 -0700 (PDT) Received: from localhost ([94.27.39.186]) by mx.google.com with ESMTPS id y29sm3514096wbd.55.2011.04.05.05.05.04 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 05 Apr 2011 05:05:05 -0700 (PDT) From: Mikolaj Golub To: Freddie Cash Organization: TOA Ukraine References: <20110402084431.GB1849@garage.freebsd.pl> Sender: Mikolaj Golub Date: Tue, 05 Apr 2011 15:05:02 +0300 In-Reply-To: (Freddie Cash's message of "Mon, 4 Apr 2011 11:08:16 -0700") Message-ID: <86tyecnbtd.fsf@in138.ua3> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: FreeBSD Filesystems , FreeBSD Stable , FreeBSD-Current , Pawel Jakub Dawidek Subject: Re: Any success stories for HAST + ZFS? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 12:05:10 -0000 On Mon, 4 Apr 2011 11:08:16 -0700 Freddie Cash wrote: FC> On Sat, Apr 2, 2011 at 1:44 AM, Pawel Jakub Dawidek wrote: >> >> I just committed a fix for a problem that might look like a deadlock. >> With trociny@ patch and my last fix (to GEOM GATE and hastd) do you >> still have any issues? FC> Just to confirm, this is commit r220264, 220265, 220266 to -CURRENT? Yes, r220264 and 220266. As it is stated in the commit log MFC is planned after 1 week. -- Mikolaj Golub From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 13:05:26 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9F42106566B for ; Tue, 5 Apr 2011 13:05:26 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [46.4.210.97]) by mx1.freebsd.org (Postfix) with ESMTP id 82BCF8FC15 for ; Tue, 5 Apr 2011 13:05:26 +0000 (UTC) Received: from localhost (angelica.4lin.net [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id C77722C7B7 for ; Tue, 5 Apr 2011 15:09:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WPVld56K1cHE for ; Tue, 5 Apr 2011 15:09:11 +0200 (CEST) Received: from [130.83.160.152] (pcdenny.rbg.informatik.tu-darmstadt.de [130.83.160.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id 677352C70F for ; Tue, 5 Apr 2011 15:09:11 +0200 (CEST) From: Denny Schierz To: freebsd-stable In-Reply-To: References: <1301929838.26698.213.camel@pcdenny> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-ipljWZWA6oZBt3oN+0+P" Date: Tue, 05 Apr 2011 15:05:05 +0200 Message-ID: <1302008705.26698.283.camel@pcdenny> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Subject: Re: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 13:05:26 -0000 --=-ipljWZWA6oZBt3oN+0+P Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable hi, Am Dienstag, den 05.04.2011, 10:01 +0200 schrieb Claus Guttesen: > The only setting I changed was: >=20 > QueueDepth 64 >=20 [...] I've tested now the default params with your Queue, but nothing changed: On same time startet with clusterssh: root@dhcp1 ~ # dd if=3D/dev/zero of=3D/dev/sda bs=3D1M count=3D2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB) copied, 33.3878 s, 64.3 MB/s root@dhcp2:~# dd if=3D/dev/zero of=3D/dev/sdb bs=3D1M count=3D2048 2048+0 Datens=C3=A4tze ein 2048+0 Datens=C3=A4tze aus 2147483648 Bytes (2,1 GB) kopiert, 48,319 s, 44,4 MB/s but, I did a plain network test again (iperf) and i got 111MB/s so, the network is the problem. :-/ cu denny --=-ipljWZWA6oZBt3oN+0+P Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAk2bE30ACgkQKlzhkqt9P+D7UgCeKtXIpRPmvBFIeaBCJdkmnbqg E58AoJvVQdEqJErSprLqwQ2xlSMe+x1o =bQbU -----END PGP SIGNATURE----- --=-ipljWZWA6oZBt3oN+0+P-- From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 13:51:20 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7F3D106567C for ; Tue, 5 Apr 2011 13:51:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 25AD48FC18 for ; Tue, 5 Apr 2011 13:51:19 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA05003; Tue, 05 Apr 2011 16:51:12 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4D9B1E50.9020403@FreeBSD.org> Date: Tue, 05 Apr 2011 16:51:12 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Jeremy Chadwick , Boris Kochergin References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> In-Reply-To: <20110405010148.GA67821@icarus.home.lan> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 13:51:20 -0000 on 05/04/2011 04:01 Jeremy Chadwick said the following: > On Mon, Apr 04, 2011 at 08:56:10PM -0400, Boris Kochergin wrote: >> No swap, blank /boot/loader.conf, default /etc/sysctl.conf. I'm >> going to try this ARC tuning thing. I vaguely recall several claims >> that tuning wasn't necessary anymore on amd64 systems with the >> amount of memory mine has, but that's obviously not the case. > > Given that you don't have swap (again: very, very bad idea), your > applications crashing due to there not being any swap space is expected: > no place to swap them out to. Jeremy, very true indeed. Boris, ARC is an adaptive cache (as its name says), but the adaption doesn't happen instantly. So, when your applications do not use a lot of memory, but there is steady filesystem usage, then ZFS ARC is going to gradually grow to consume an optimum amount of RAM. Then, your applications suddenly need a lot more memory, they put pressure on VM system, ARC starts to shrink. But if memory demand grows faster than ARC shrinks, you are going to get a memory shortage. And since you don't have any swap to act as a safety net, you are getting out-of-memory situation. So no surprises here, no system problems, just a normal foot-shooting :) Clamping maximum ARC size, as Jeremy has suggested, should help some. Adding some swap would help a lot more. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:04:24 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DA5F106566B; Tue, 5 Apr 2011 14:04:24 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [IPv6:2001:470:1f09:176e::3]) by mx1.freebsd.org (Postfix) with ESMTP id 27EE58FC0A; Tue, 5 Apr 2011 14:04:24 +0000 (UTC) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ingresso.co.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1Q76rm-000P4Q-By; Tue, 05 Apr 2011 15:04:22 +0100 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.74 (FreeBSD)) (envelope-from ) id 1Q76rm-0001Cr-B2; Tue, 05 Apr 2011 15:04:22 +0100 To: avg@FreeBSD.org, freebsd@jdc.parodius.com, spawk@acm.poly.edu In-Reply-To: <4D9B1E50.9020403@FreeBSD.org> Message-Id: From: Pete French Date: Tue, 05 Apr 2011 15:04:22 +0100 Cc: freebsd-stable@FreeBSD.org Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:04:24 -0000 > Adding some swap would help a lot more. So, I run a lot of systems without swap - basically my thinking at the time I set them up went like this. "I have 4 gig of memory, and 4 gig of swap. Surely running 8 gig of memory and no swap will be just as good ?" but, is that actually true ? Is real RAM as good as an equivalent amount of swap, or is there smething special about swap which means you shoud have some no matter how much RAM you have ? -pete. From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:09:31 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B05E41065673 for ; Tue, 5 Apr 2011 14:09:31 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 504868FC0A for ; Tue, 5 Apr 2011 14:09:31 +0000 (UTC) Received: (qmail 30657 invoked from network); 5 Apr 2011 14:09:30 -0000 Received: from unknown (HELO ?10.50.50.234?) (spawk@64.147.100.2) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 5 Apr 2011 14:09:30 -0000 Message-ID: <4D9B229C.1050103@acm.poly.edu> Date: Tue, 05 Apr 2011 10:09:32 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101106 Thunderbird/3.1.6 MIME-Version: 1.0 To: Pete French References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org, freebsd@jdc.parodius.com, avg@FreeBSD.org Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:09:31 -0000 On 04/05/11 10:04, Pete French wrote: >> Adding some swap would help a lot more. > So, I run a lot of systems without swap - basically my > thinking at the time I set them up went like this. > > "I have 4 gig of memory, and 4 gig of swap. Surely running 8 gig of > memory and no swap will be just as good ?" > > but, is that actually true ? Is real RAM as good as an equivalent amount > of swap, or is there smething special about swap which means you shoud > have some no matter how much RAM you have ? > > -pete. I guess swap is special since I assume memory used by the kernel will never be offloaded to it (could be wrong), but userspace memory will, so it is guaranteed to be available to userspace processes only. -Boris From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:28:49 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F62B106564A for ; Tue, 5 Apr 2011 14:28:49 +0000 (UTC) (envelope-from gkontos.mail@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id ED2258FC12 for ; Tue, 5 Apr 2011 14:28:48 +0000 (UTC) Received: by iwn33 with SMTP id 33so496690iwn.13 for ; Tue, 05 Apr 2011 07:28:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=eDiAexLXpY7tYuJD1IjApT1WICd4/GdBCVNCxs/59LU=; b=bPibjgLeDoftsfT2tE/mWhrMKrAaHjNKWLFfaUWnEE4fAyUcM5r0jYH5wiID3Jj4hr gYEFN5pI2xGVGPFHsde49Kwy5mje9x4QKD15IwIRf0sIl9mtVF1frRs5S8jlL8H5BoNo 496RHNnlhqaiKetdmBiEhUzffkVJDiMe65v6k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=vp4oROSgueXSuIZwddYp3SdBVXI+NMbwJuvYW0LRk7bfYFw+PNE6mjISIAXRWLgx1E vsL/GEu16WotLy1671hShZxUTNpoa6yuciVtbbXzvMF+EQK7NedyACyRRn9K5GL4d97K 4K3iKOsUX2aQ5AzAv0C201C3Zv2hwAhQojkks= MIME-Version: 1.0 Received: by 10.231.34.4 with SMTP id j4mr5239705ibd.83.1302013728213; Tue, 05 Apr 2011 07:28:48 -0700 (PDT) Received: by 10.231.11.196 with HTTP; Tue, 5 Apr 2011 07:28:48 -0700 (PDT) In-Reply-To: <4D9B229C.1050103@acm.poly.edu> References: <4D9B229C.1050103@acm.poly.edu> Date: Tue, 5 Apr 2011 17:28:48 +0300 Message-ID: From: George Kontostanos To: Pete French , freebsd-stable@freebsd.org, freebsd@jdc.parodius.com, avg@freebsd.org, Boris Kochergin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:28:49 -0000 Years ago when RAM was expensive swap a necessary workaround. That doesn't mean that swap is useless. It all depends on what a server is doing. If you are using a database server then it is absolutely normal and expected to cache. Also, unlike other OS FreeBSD tends to make use of the ram. Swaping recently used data is not bad and it is the way it works. Regards On Tue, Apr 5, 2011 at 5:09 PM, Boris Kochergin wrote: > On 04/05/11 10:04, Pete French wrote: > >> Adding some swap would help a lot more. >>> >> So, I run a lot of systems without swap - basically my >> thinking at the time I set them up went like this. >> >> "I have 4 gig of memory, and 4 gig of swap. Surely running 8 gig of >> memory and no swap will be just as good ?" >> >> but, is that actually true ? Is real RAM as good as an equivalent amount >> of swap, or is there smething special about swap which means you shoud >> have some no matter how much RAM you have ? >> >> -pete. >> > > I guess swap is special since I assume memory used by the kernel will never > be offloaded to it (could be wrong), but userspace memory will, so it is > guaranteed to be available to userspace processes only. > > -Boris > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > -- George Kontostanos aisecure.net From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:29:56 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49BD81065675 for ; Tue, 5 Apr 2011 14:29:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 92EE08FC24 for ; Tue, 5 Apr 2011 14:29:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA05564; Tue, 05 Apr 2011 17:29:48 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4D9B275B.3060000@FreeBSD.org> Date: Tue, 05 Apr 2011 17:29:47 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Pete French References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: spawk@acm.poly.edu, freebsd-stable@FreeBSD.org, freebsd@jdc.parodius.com Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:29:56 -0000 on 05/04/2011 17:04 Pete French said the following: >> Adding some swap would help a lot more. > > So, I run a lot of systems without swap - basically my > thinking at the time I set them up went like this. > > "I have 4 gig of memory, and 4 gig of swap. Surely running 8 gig of > memory and no swap will be just as good ?" > > but, is that actually true ? Is real RAM as good as an equivalent amount > of swap, or is there smething special about swap which means you shoud > have some no matter how much RAM you have ? I think that it depends. I usually do use swap for the following reasons: 1. some anonymous memory ("malloced") may reasonably go to swap to free some RAM for caching data; that can have overall performance benefits depending in system usage patterns; 2. VM is happy dealing out RAM for any uses until some low watermarks are reached, then the system tries to free up some RAM. Depending on the amount of memory (and those thresholds) and "burstiness" of memory demand a system may potentially run completely out of memory and would have to kill some processes. Having swap provides some cushion. Swap kind of smooths any bursts. (And it can also slow things down as a side effect) Of course, the system can run out of swap as well, but that would mean that you really need more RAM. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:37:31 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 768CF1065676 for ; Tue, 5 Apr 2011 14:37:31 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp05.sth.basefarm.net (ch-smtp05.sth.basefarm.net [80.76.153.6]) by mx1.freebsd.org (Postfix) with ESMTP id 07D298FC21 for ; Tue, 5 Apr 2011 14:37:30 +0000 (UTC) Received: from c83-255-51-20.bredband.comhem.se ([83.255.51.20]:41350 helo=falcon.midgard.homeip.net) by ch-smtp05.sth.basefarm.net with esmtp (Exim 4.73) (envelope-from ) id 1Q77AV-0004c4-IL for freebsd-stable@FreeBSD.org; Tue, 05 Apr 2011 16:23:48 +0200 Received: (qmail 67302 invoked from network); 5 Apr 2011 16:23:41 +0200 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with ESMTP; 5 Apr 2011 16:23:41 +0200 Received: (qmail 91723 invoked by uid 1001); 5 Apr 2011 16:23:41 +0200 Date: Tue, 5 Apr 2011 16:23:41 +0200 From: Erik Trulsson To: Pete French Message-ID: <20110405142341.GA91693@owl.midgard.homeip.net> References: <4D9B1E50.9020403@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: 83.255.51.20 X-Scan-Result: No virus found in message 1Q77AV-0004c4-IL. X-Scan-Signature: ch-smtp05.sth.basefarm.net 1Q77AV-0004c4-IL 8a8bd891cb07046cbed37a5e3b3ace74 Cc: spawk@acm.poly.edu, freebsd-stable@FreeBSD.org, freebsd@jdc.parodius.com, avg@FreeBSD.org Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:37:31 -0000 On Tue, Apr 05, 2011 at 03:04:22PM +0100, Pete French wrote: > > Adding some swap would help a lot more. > > So, I run a lot of systems without swap - basically my > thinking at the time I set them up went like this. > > "I have 4 gig of memory, and 4 gig of swap. Surely running 8 gig of > memory and no swap will be just as good ?" > > but, is that actually true ? Is real RAM as good as an equivalent amount > of swap, or is there smething special about swap which means you shoud > have some no matter how much RAM you have ? I believe some things (caches/buffers and the like) are sized according to how much real RAM you have, i.e. if you have 8G RAM the system will actuallu use more memory than if you have only 4G RAM. I also think that parts of the system are designed with the assumption that there is some swap available that can act as some sort of "overflow buffer" from time to time. -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 14:54:50 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 151B71065673; Tue, 5 Apr 2011 14:54:50 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [IPv6:2001:470:1f09:176e::3]) by mx1.freebsd.org (Postfix) with ESMTP id D2B078FC1B; Tue, 5 Apr 2011 14:54:49 +0000 (UTC) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ingresso.co.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1Q77eY-0000yp-Lf; Tue, 05 Apr 2011 15:54:46 +0100 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.74 (FreeBSD)) (envelope-from ) id 1Q77eY-0001Mc-Ke; Tue, 05 Apr 2011 15:54:46 +0100 To: avg@FreeBSD.org, petefrench@ingresso.co.uk In-Reply-To: <4D9B275B.3060000@FreeBSD.org> Message-Id: From: Pete French Date: Tue, 05 Apr 2011 15:54:46 +0100 Cc: spawk@acm.poly.edu, freebsd-stable@FreeBSD.org, freebsd@jdc.parodius.com Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:54:50 -0000 > Having swap provides some cushion. Swap kind of smooths any bursts. (And it can > also slow things down as a side effect) This is why I got rid of it - my application is a lot of CGI scripts. The overload condition is that we run out of memory - and we run *way* out of memory .... its never just a little overflow, it;s either handleable or completely crushed. But swap makes that mre llikely to happen, because as the processes are swapped out they run slower, take longer to finish and thus use memory for longer. What I saw was that as soon as any web server would start tos wap it would swftly fall down. Without swap they stay up, but reject requests. Its a better failure mode... these days I run a compormise - swap on internal machines, and no swap on customer facing ones, but lots of RAM (16 gig). -pete. From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 15:23:02 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDABE106564A for ; Tue, 5 Apr 2011 15:23:01 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 3E4408FC12 for ; Tue, 5 Apr 2011 15:23:00 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.4/8.14.4) with ESMTP id p35F0ooK001511 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 5 Apr 2011 18:00:56 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4D9B2EA2.9020700@digsys.bg> Date: Tue, 05 Apr 2011 18:00:50 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110307 Thunderbird/3.1.9 MIME-Version: 1.0 To: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS HAST config preference X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 15:23:03 -0000 This is more of an proof of concept question: I am building an redundant cluster of blade servers, and toy with the idea to use HAST and ZFS for the storage. Blades will work in pairs and each pair will provide various services, from SQL databases, to hosting virtual machines (jails and otherwise). Each pair will use CARP for redundancy. My original idea was to set up blades so that they run HAST on pairs of disks, and run ZFS in number of mirror vdevs on top of HAST. The ZFS pool will exist only on the master HAST node. Let's call this setup1. Or, I could use ZFS volumes and run HAST on top of these. This means, that on each blade, I will have an local ZFS pool. Let's call this setup2. Third idea would be to have the blades completely diskless, boot from separate boot/storage server and mount filesystems or iSCSI volumes as needed from the storage server. HAST might not be necessary here. ZFS pool will exist on the storage server only. Let's call this setup3. While setup1 is most straightforward, it has some drawbacks: - disks handled by HAST need to be either identical or have matching partitions created; - the 'spare' blade would do nothing, as it's disk subsystem will be gone as long as it is HAST slave. As the blades are quite powerful (4x8 core AMD) that would be wasteful, at least in the beginning. With setup2, I can get away with different size disks in each blade. All blades can also be used for whatever additional processing, shared data will be only presented by HAST to whichever node needs it, for "important" services. One drawback here: - can't just pull off one of the blades, without stopping/transferring first all of it's services. It seems that in larger scale, setup3 would be best. I am not yet here, although close (the storage server is missing). HAST replication speed should not be an issue, there is 10Gbit network between the blade servers. Has anyone already setup something similar? What was the experience? There were recently some bugs that sort of plagued setup1, but these seem to be resolved now. Daniel From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 17:45:08 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42A4E1065672 for ; Tue, 5 Apr 2011 17:45:08 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout022.mac.com (asmtpout022.mac.com [17.148.16.97]) by mx1.freebsd.org (Postfix) with ESMTP id 23C3C8FC18 for ; Tue, 5 Apr 2011 17:45:07 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from cswiger1.apple.com ([17.209.4.71]) by asmtp022.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LJ6004BCUIAPL90@asmtp022.mac.com> for freebsd-stable@FreeBSD.org; Tue, 05 Apr 2011 09:44:35 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15,1.0.148,0.0.0000 definitions=2011-04-05_07:2011-04-05, 2011-04-05, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1104050108 From: Chuck Swiger In-reply-to: Date: Tue, 05 Apr 2011 09:44:34 -0700 Message-id: <62BEFA1C-9A7D-4981-9F71-6DE15973B7C1@mac.com> References: To: Pete French X-Mailer: Apple Mail (2.1084) Cc: FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 17:45:08 -0000 On Apr 5, 2011, at 7:54 AM, Pete French wrote: >> Having swap provides some cushion. Swap kind of smooths any bursts. (And it can >> also slow things down as a side effect) > > This is why I got rid of it - my application is a lot of CGI scripts. The > overload condition is that we run out of memory - and we run *way* out > of memory .... its never just a little overflow, it;s either handleable or > completely crushed. But swap makes that mre llikely to happen, because > as the processes are swapped out they run slower, take longer to > finish and thus use memory for longer. > > What I saw was that as soon as any web server would start tos wap it would > swftly fall down. Without swap they stay up, but reject requests. Its a better > failure mode... You'll be better off providing swap space with every machine, and tuning Apache's MaxClients to a suitable value such that you don't swap excessively under load spikes. Without any swap, the system will be unable to page out unused portions of processes and will handle less load than it would with swap available. Regards, -- -Chuck From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 18:37:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9B611065672 for ; Tue, 5 Apr 2011 18:37:50 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by mx1.freebsd.org (Postfix) with SMTP id EFAB48FC0C for ; Tue, 5 Apr 2011 18:37:49 +0000 (UTC) Received: (qmail invoked by alias); 05 Apr 2011 18:37:47 -0000 Received: from f055202252.adsl.alicedsl.de (EHLO apollo.emma.line.org) [78.55.202.252] by mail.gmx.net (mp062) with SMTP; 05 Apr 2011 20:37:47 +0200 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX1+FRuwv1meFuhuS+ANrZu+ouF/KxB153IJwz+leUZ oDkY7LptHTlSeP Received: from [IPv6:::1] (unknown [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id A6DF725AD7B for ; Tue, 5 Apr 2011 20:37:44 +0200 (CEST) Message-ID: <4D9B6178.7090809@gmx.de> Date: Tue, 05 Apr 2011 20:37:44 +0200 From: Matthias Andree User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.14) Gecko/20110223 Mnenhy/0.8.3 Thunderbird/3.1.8 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> <4D9B1E50.9020403@FreeBSD.org> In-Reply-To: <4D9B1E50.9020403@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 18:37:50 -0000 Am 05.04.2011 15:51, schrieb Andriy Gapon: > Boris, > ARC is an adaptive cache (as its name says), but the adaption doesn't happen > instantly. So, when your applications do not use a lot of memory, but there is > steady filesystem usage, then ZFS ARC is going to gradually grow to consume an > optimum amount of RAM. Then, your applications suddenly need a lot more memory, > they put pressure on VM system, ARC starts to shrink. But if memory demand grows > faster than ARC shrinks, you are going to get a memory shortage. And since you > don't have any swap to act as a safety net, you are getting out-of-memory situation. > So no surprises here, no system problems, just a normal foot-shooting :) > > Clamping maximum ARC size, as Jeremy has suggested, should help some. > Adding some swap would help a lot more. The problem to me seems that ARC, the way you describe it, isn't really integrated with the system. It's not buffer or cache memory, but some separate application memory that can't adapt as quickly to system memory demands as all other kernel-managed caches and buffers can. FWIW, I've been using it on a 4 GB amd64 computer. From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 19:13:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A152106566C; Tue, 5 Apr 2011 19:13:58 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id E9A408FC13; Tue, 5 Apr 2011 19:13:57 +0000 (UTC) Received: by gxk28 with SMTP id 28so328952gxk.13 for ; Tue, 05 Apr 2011 12:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=KIJYwFEg5+6yEFkgBTlCy2o3RQ7tedqCIu25RT91Ts4=; b=FhKGmJE3BStG8udaH8AjCrMz6L6SS90Z01rxhBcxPhAqtx/PIi4sg/2xgBg8pIOtNE aGz2VYDaArs3nAJuQsNBXl+Dkhb0cndcrz4k09xNGdhkYniWGt3chamSL/XcMyDpziV8 jBL+0pQuQ+RGgBcU1ZWugO294RKRY2ECxM7nI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=FNHXK39QXdonQEYEjpav511iqHCMvflOtsGWe1GujXFiykJBmqsXwCtF6JmFHEg5vR qmW4AaEsfy/lo1KTvlIgRP8SeB5uwG04Y3SaNrMcb68FMmfPZAwVteDsEnJ0qdvOcH1/ NQYa3dl01HUFypjPM2IQ2+pkVmE/Ep/L3TXDo= MIME-Version: 1.0 Received: by 10.90.14.39 with SMTP id 39mr928443agn.127.1302030837143; Tue, 05 Apr 2011 12:13:57 -0700 (PDT) Received: by 10.90.51.14 with HTTP; Tue, 5 Apr 2011 12:13:57 -0700 (PDT) In-Reply-To: <86tyecnbtd.fsf@in138.ua3> References: <20110402084431.GB1849@garage.freebsd.pl> <86tyecnbtd.fsf@in138.ua3> Date: Tue, 5 Apr 2011 12:13:57 -0700 Message-ID: From: Freddie Cash To: Mikolaj Golub Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems , FreeBSD Stable , FreeBSD-Current , Pawel Jakub Dawidek Subject: Re: Any success stories for HAST + ZFS? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 19:13:58 -0000 On Tue, Apr 5, 2011 at 5:05 AM, Mikolaj Golub wrote: > On Mon, 4 Apr 2011 11:08:16 -0700 Freddie Cash wrote: > > =C2=A0FC> On Sat, Apr 2, 2011 at 1:44 AM, Pawel Jakub Dawidek wrote: > =C2=A0>> > =C2=A0>> I just committed a fix for a problem that might look like a dead= lock. > =C2=A0>> With trociny@ patch and my last fix (to GEOM GATE and hastd) do = you > =C2=A0>> still have any issues? > > =C2=A0FC> Just to confirm, this is commit r220264, 220265, 220266 to -CUR= RENT? > > Yes, r220264 and 220266. As it is stated in the commit log MFC is planned > after 1 week. Okay. I'll keep an eye out next week for the MFC of those patches to hit -STABLE, and do an upgrade/test cycle after that point. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 19:15:11 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B20681065676 for ; Tue, 5 Apr 2011 19:15:11 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id F3F858FC13 for ; Tue, 5 Apr 2011 19:15:07 +0000 (UTC) Received: by bwz12 with SMTP id 12so706585bwz.13 for ; Tue, 05 Apr 2011 12:15:06 -0700 (PDT) Received: by 10.204.20.142 with SMTP id f14mr10792bkb.155.1302029271982; Tue, 05 Apr 2011 11:47:51 -0700 (PDT) Received: from [10.254.254.77] (ppp95-165-141-212.pppoe.spdop.ru [95.165.141.212]) by mx.google.com with ESMTPS id t1sm3924243bkx.7.2011.04.05.11.47.49 (version=SSLv3 cipher=OTHER); Tue, 05 Apr 2011 11:47:50 -0700 (PDT) Message-ID: <4D9B63D4.8010604@zonov.org> Date: Tue, 05 Apr 2011 22:47:48 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.24) Gecko/20100228 Thunderbird/2.0.0.24 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS with compression causes deadlock X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 19:15:11 -0000 Hi, Today I had deadlock on several machines. Almost all processes stucked in [tx->tx_cpu[c].tc_lock]. Machines were helped only `reboot -n'. I've created new gzip-ed filesystem a few days ago. I didn't have any problems with ZFS before. System was built from svn://svn.freebsd.org/base/stable/8@215508. Machines have 24Gb RAM, 4 SATA 500Gb disks in raidz. zfs list: NAME USED AVAIL REFER MOUNTPOINT place 373G 982G 83.0G /place place/gzip 290G 982G 290G /place/gzip root 723M 1.25G 722M legacy var 9.90M 974M 9.77M /var loader.conf: ahci_load="YES" zfs_load="YES" vfs.zfs.arc_max="2G" vfs.zfs.cache_flush_disable="1" # default: 0 vfs.zfs.prefetch_disable="1" # default: 0 vfs.zfs.txg.timeout="5" # default: 30 vfs.zfs.zio.use_uma="0" # default: 0 (earlier was 1) Can anyone help me to find a problem? -- Andrey Zonov From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 19:26:07 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7B59106566B; Tue, 5 Apr 2011 19:26:07 +0000 (UTC) (envelope-from prvs=1076fd9788=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 268E38FC1E; Tue, 5 Apr 2011 19:26:06 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 05 Apr 2011 20:15:08 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 05 Apr 2011 20:15:08 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50012794731.msg; Tue, 05 Apr 2011 20:15:08 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1076fd9788=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <38ABC8EC81164D648092640447554493@multiplay.co.uk> From: "Steven Hartland" To: , References: Date: Tue, 5 Apr 2011 20:15:41 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="Windows-1252"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 Cc: spawk@acm.poly.edu, freebsd-stable@FreeBSD.org, freebsd@jdc.parodius.com Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 19:26:07 -0000 ----- Original Message ----- From: "Pete French" > This is why I got rid of it - my application is a lot of CGI scripts. The > overload condition is that we run out of memory - and we run *way* out > of memory .... its never just a little overflow, it;s either handleable or > completely crushed. But swap makes that mre llikely to happen, because > as the processes are swapped out they run slower, take longer to > finish and thus use memory for longer. > > What I saw was that as soon as any web server would start tos wap it would > swftly fall down. Without swap they stay up, but reject requests. Its a better > failure mode... > > these days I run a compormise - swap on internal machines, and no swap > on customer facing ones, but lots of RAM (16 gig). If that's php under apache either cap maxclients so this doesn't happen or even better ditch apache and switch to something like nginx + php-fpm, much quicker much more stable solution which doesn't suddenly eat all your ram and blow-up the machine. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-stable@FreeBSD.ORG Tue Apr 5 19:44:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8368E1065670 for ; Tue, 5 Apr 2011 19:44:42 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5B3748FC20 for ; Tue, 5 Apr 2011 19:44:42 +0000 (UTC) Received: by pwj8 with SMTP id 8so373221pwj.13 for ; Tue, 05 Apr 2011 12:44:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=bnMdXXMIetx1rU+ta/wbD8LLAY7ibKxUTs+NOErA4YY=; b=JMAVmeaQ/1KCSoTN5YbbVmpnr6QAMAcfSWuH578M/JzHnZpQXmlExcUs01j958h8Va XUadThXm8mAhlE83TUyqxdsViW0J1B3HyyQE4+4gRwUpFn69qmE2bZ9U4b+4k4+ikEX8 GnT78LHasXJlVe0tzGtcytU0n77I8GmSKgTmk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=LxITjOAb+/a18qUTfdc5pc5IB4gneli/SrS4MvMK0oCCFtxU9tpEgt2n2tZU+4Z19U or1IFQkyidy/PDAPLUlSwPEYMSQtKS7CwpMvIFbAENaPikRzR1stDtxOwMWo/VNEzO5s pcmLOnmBlYx7Ufmj/vZSiGCvzpoc0MWrTaFug= MIME-Version: 1.0 Received: by 10.142.131.1 with SMTP id e1mr58135wfd.292.1302032681708; Tue, 05 Apr 2011 12:44:41 -0700 (PDT) Received: by 10.142.89.16 with HTTP; Tue, 5 Apr 2011 12:44:41 -0700 (PDT) In-Reply-To: <4D9B63D4.8010604@zonov.org> References: <4D9B63D4.8010604@zonov.org> Date: Tue, 5 Apr 2011 21:44:41 +0200 Message-ID: From: Claus Guttesen To: Andrey Zonov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS with compression causes deadlock X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 19:44:42 -0000 > Today I had deadlock on several machines. Almost all processes stucked in > [tx->tx_cpu[c].tc_lock]. Machines were helped only `reboot -n'. > I've created new gzip-ed filesystem a few days ago. I didn't have any > problems with ZFS before. > > System was built from svn://svn.freebsd.org/base/stable/8@215508. > Machines have 24Gb RAM, 4 SATA 500Gb disks in raidz. > > loader.conf: > ahci_load=3D"YES" > zfs_load=3D"YES" > vfs.zfs.arc_max=3D"2G" > vfs.zfs.cache_flush_disable=3D"1" # default: 0 > vfs.zfs.prefetch_disable=3D"1" =A0 =A0# default: 0 > vfs.zfs.txg.timeout=3D"5" =A0 =A0 =A0 =A0 # default: 30 > vfs.zfs.zio.use_uma=3D"0" =A0 =A0 =A0 =A0 # default: 0 (earlier was 1) Have you tried to remove vfs.zfs.* in rc.conf? Newer versions of zfs may not need tuning except in specific cases now. I've had problems using compression in (much) earlier zfs-versions some years ago and avoided this feature. A few days ago I've setup FreeBSD current with zfs. ver. 28 and enabled compression and have performed a zfs receive with approx. 660 GB of data that boils down to 350 GB without problems. --=20 regards Claus When lenity and cruelty play for a kingdom, the gentler gamester is the soonest winner. Shakespeare twitter.com/kometen From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 10:14:06 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFA05106566B for ; Wed, 6 Apr 2011 10:14:06 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2202D8FC08 for ; Wed, 6 Apr 2011 10:14:05 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA21945; Wed, 06 Apr 2011 13:14:01 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4D9C3CE8.5090201@FreeBSD.org> Date: Wed, 06 Apr 2011 13:14:00 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Matthias Andree References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> <20110404224334.GA64297@icarus.home.lan> <4D9A68AA.6040803@acm.poly.edu> <20110405010148.GA67821@icarus.home.lan> <4D9B1E50.9020403@FreeBSD.org> <4D9B6178.7090809@gmx.de> In-Reply-To: <4D9B6178.7090809@gmx.de> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 10:14:06 -0000 on 05/04/2011 21:37 Matthias Andree said the following: > Am 05.04.2011 15:51, schrieb Andriy Gapon: > >> Boris, >> ARC is an adaptive cache (as its name says), but the adaption doesn't happen >> instantly. So, when your applications do not use a lot of memory, but there is >> steady filesystem usage, then ZFS ARC is going to gradually grow to consume an >> optimum amount of RAM. Then, your applications suddenly need a lot more memory, >> they put pressure on VM system, ARC starts to shrink. But if memory demand grows >> faster than ARC shrinks, you are going to get a memory shortage. And since you >> don't have any swap to act as a safety net, you are getting out-of-memory situation. >> So no surprises here, no system problems, just a normal foot-shooting :) >> >> Clamping maximum ARC size, as Jeremy has suggested, should help some. >> Adding some swap would help a lot more. > > The problem to me seems that ARC, the way you describe it, isn't really > integrated with the system. Define "really integrated". > It's not buffer or cache memory, but some True. > separate application memory that can't adapt as quickly to system memory > demands as all other kernel-managed caches and buffers can. Other kernel-managed caches and buffers are not instant either. But I have never compared "speed of adaptions". -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 10:15:36 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF4301065675 for ; Wed, 6 Apr 2011 10:15:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1942F8FC1B for ; Wed, 6 Apr 2011 10:15:35 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA21978; Wed, 06 Apr 2011 13:15:32 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4D9C3D43.3070302@FreeBSD.org> Date: Wed, 06 Apr 2011 13:15:31 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Andrey Zonov References: <4D9B63D4.8010604@zonov.org> In-Reply-To: <4D9B63D4.8010604@zonov.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: ZFS with compression causes deadlock X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 10:15:37 -0000 on 05/04/2011 21:47 Andrey Zonov said the following: > Hi, > > Today I had deadlock on several machines. Almost all processes stucked in > [tx->tx_cpu[c].tc_lock]. Machines were helped only `reboot -n'. > I've created new gzip-ed filesystem a few days ago. I didn't have any problems > with ZFS before. Sometime procstat -kk provides useful additional information. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 12:27:27 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1ADE3106566C for ; Wed, 6 Apr 2011 12:27:27 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 012BA8FC14 for ; Wed, 6 Apr 2011 12:27:26 +0000 (UTC) Received: by pvg11 with SMTP id 11so644362pvg.13 for ; Wed, 06 Apr 2011 05:27:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.142.131.7 with SMTP id e7mr867931wfd.348.1302092846031; Wed, 06 Apr 2011 05:27:26 -0700 (PDT) Received: by 10.68.63.71 with HTTP; Wed, 6 Apr 2011 05:27:25 -0700 (PDT) In-Reply-To: <4D9B63D4.8010604@zonov.org> References: <4D9B63D4.8010604@zonov.org> Date: Wed, 6 Apr 2011 14:27:25 +0200 Message-ID: From: Olivier Smedts To: Andrey Zonov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS with compression causes deadlock X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 12:27:27 -0000 2011/4/5 Andrey Zonov : > loader.conf: > ahci_load=3D"YES" > zfs_load=3D"YES" > vfs.zfs.arc_max=3D"2G" > vfs.zfs.cache_flush_disable=3D"1" # default: 0 Are you absolutely sure of what you're doing with the vfs.zfs.cache_flush_disable tunable ? Since you have SATA disks, I assume you don't have a controller with memory-backed write cache or NVRAM. > vfs.zfs.prefetch_disable=3D"1" =A0 =A0# default: 0 > vfs.zfs.txg.timeout=3D"5" =A0 =A0 =A0 =A0 # default: 30 > vfs.zfs.zio.use_uma=3D"0" =A0 =A0 =A0 =A0 # default: 0 (earlier was 1) --=20 Olivier Smedts=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0 _ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 ASCII ribbon campaign ( ) e-mail: olivier@gid0.org=A0 =A0 =A0 =A0 - against HTML email & vCards=A0 X www: http://www.gid0.org=A0 =A0 - against proprietary attachments / \ =A0 "Il y a seulement 10 sortes de gens dans le monde : =A0 ceux qui comprennent le binaire, =A0 et ceux qui ne le comprennent pas." From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 12:37:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AD46106564A for ; Wed, 6 Apr 2011 12:37:33 +0000 (UTC) (envelope-from olivier@gid0.org) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2FDAB8FC1D for ; Wed, 6 Apr 2011 12:37:32 +0000 (UTC) Received: by pvg11 with SMTP id 11so648062pvg.13 for ; Wed, 06 Apr 2011 05:37:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.143.154.35 with SMTP id g35mr843795wfo.120.1302093452283; Wed, 06 Apr 2011 05:37:32 -0700 (PDT) Received: by 10.68.63.71 with HTTP; Wed, 6 Apr 2011 05:37:32 -0700 (PDT) In-Reply-To: <4D972FF7.6010901@acm.poly.edu> References: <4D972FF7.6010901@acm.poly.edu> Date: Wed, 6 Apr 2011 14:37:32 +0200 Message-ID: From: Olivier Smedts To: Boris Kochergin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 12:37:33 -0000 2011/4/2 Boris Kochergin : > pid 59630 (httpd), uid 80, was killed: out of swap space > pid 59341 (find), uid 0, was killed: out of swap space > pid 23134 (irssi), uid 1001, was killed: out of swap space > pid 49332 (sshd), uid 1001, was killed: out of swap space > pid 69074 (httpd), uid 0, was killed: out of swap space > pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space Like others, I'll also suggest adding at least a little swap. If you don't have disk space outside of the ZFS pool (recommended way to create a swap), you can create one inside, with a zvol : zfs create -V 2G -o org.freebsd:swap=3Don -o primarycache=3Dnone -o secondarycache=3Dnone -o tank/swap I sometimes use "-b 8K" and "-o checksum=3Doff" for the swap, but haven't stress tested this under 9-CURRENT and ZFS v28. > # zpool status > =A0pool: home > =A0state: DEGRADED > status: One or more devices could not be used because the label is missin= g > or > =A0 =A0 =A0 =A0invalid. =A0Sufficient replicas exist for the pool to cont= inue > =A0 =A0 =A0 =A0functioning in a degraded state. > action: Replace the device using 'zpool replace'. > =A0 see: http://www.sun.com/msg/ZFS-8000-4J > =A0scrub: none requested > config: > > =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0STATE =A0 =A0 READ WRITE CKSUM > =A0 =A0 =A0 =A0home =A0 =A0 =A0 =A0DEGRADED =A0 =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0raidz2 =A0 =A0DEGRADED =A0 =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0ada0 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0ada1 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0ada2 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0ada3 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0ada4 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0ada5 =A0 =A0UNAVAIL =A0 =A0 =A00 =A0 =A085 =A0 =A0= 11 =A0experienced I/O failures > > errors: No known data errors Like others, I'll also *strongly* suggest fixing that ada5 problem. Try to run smartctl on the disk to see the problem. If the disk is bad, replace it ! Don't wait "for a few months" if you don't want to definitely loose your data. Cheers --=20 Olivier Smedts=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0 _ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 ASCII ribbon campaign ( ) e-mail: olivier@gid0.org=A0 =A0 =A0 =A0 - against HTML email & vCards=A0 X www: http://www.gid0.org=A0 =A0 - against proprietary attachments / \ =A0 "Il y a seulement 10 sortes de gens dans le monde : =A0 ceux qui comprennent le binaire, =A0 et ceux qui ne le comprennent pas." From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 12:55:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39746106566B for ; Wed, 6 Apr 2011 12:55:52 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [IPv6:2001:470:1f09:176e::3]) by mx1.freebsd.org (Postfix) with ESMTP id 031668FC0C for ; Wed, 6 Apr 2011 12:55:52 +0000 (UTC) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ingresso.co.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1Q7SH0-000I9z-K0; Wed, 06 Apr 2011 13:55:50 +0100 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.74 (FreeBSD)) (envelope-from ) id 1Q7SH0-0004lT-JB; Wed, 06 Apr 2011 13:55:50 +0100 To: daniel@digsys.bg, freebsd-stable@freebsd.org In-Reply-To: <4D9B2EA2.9020700@digsys.bg> Message-Id: From: Pete French Date: Wed, 06 Apr 2011 13:55:50 +0100 Cc: Subject: Re: ZFS HAST config preference X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 12:55:52 -0000 > My original idea was to set up blades so that they run HAST on pairs of > disks, and run ZFS in number of mirror vdevs on top of HAST. The ZFS > pool will exist only on the master HAST node. Let's call this setup1. This is exactly how I run things. Personally I think it is the best solution, as ZFS then knows about the mirroring of the drives, always a good thing, and you geta ZFS filesystem on the top. I also enable compression to reduce the bandwidth to the drives, as tha reduces the data flowing across the network. For what I am doing (running mysql on top) this is actually faster for both selects and inserts. test with your own application first though. > Or, I could use ZFS volumes and run HAST on top of these. This means, > that on each blade, I will have an local ZFS pool. Let's call this setup2. ...you would need to put a filesystem on to of the HAST filesystem though, what would that be ? > While setup1 is most straightforward, it has some drawbacks: > - disks handled by HAST need to be either identical or have matching > partitions created; This is true. I run identical machines as primary and secondary. > - the 'spare' blade would do nothing, as it's disk subsystem will be > gone as long as it is HAST slave. As the blades are quite powerful (4x8 > core AMD) that would be wasteful, at least in the beginning. If you are keeping a machine as a hot spare then thats something you just have to live with in my opinion. I've run this way for several years - before HAST and ZFS we used gmirror and UFS to do the same thing. It does work very nicely, but you do end up with a machine idle. > HAST replication speed should not be an issue, there is 10Gbit network > between the blade servers. I actually ut separate ether interfaces for each of the hast drives. So for 2 drives there are 2 space ether ports on the machine, with a cible between them, deidcated to just that drive. Those are gigabit cards though, not 10 gig. > Has anyone already setup something similar? What was the experience? very good actually. One thing I would say is to write and test a set of scvripts to do the failover - avoids shooting yourself in the foot when trying to do the commands by hand (which is rather easy to do). I have a script to make the orimary into a secondary, and one to d the reverse. The first scipt waits unitl the HAST data is all flushed before changing role, and makes sure the services are stoped, pool exported before ripping out the disc from underneath. The script also handles removing a shared IP address from the interface. -pete. From owner-freebsd-stable@FreeBSD.ORG Wed Apr 6 15:29:47 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E0AD106564A for ; Wed, 6 Apr 2011 15:29:47 +0000 (UTC) (envelope-from mamalos@eng.auth.gr) Received: from vergina.eng.auth.gr (vergina.eng.auth.gr [155.207.18.1]) by mx1.freebsd.org (Postfix) with ESMTP id 9B3088FC19 for ; Wed, 6 Apr 2011 15:29:46 +0000 (UTC) Received: from mamalacation.ee.auth.gr (mamalacation.ee.auth.gr [155.207.33.29]) by vergina.eng.auth.gr (8.14.3/8.14.3) with ESMTP id p36FTitR057082 for ; Wed, 6 Apr 2011 18:29:44 +0300 (EEST) (envelope-from mamalos@eng.auth.gr) Message-ID: <4D9C86E8.3090402@eng.auth.gr> Date: Wed, 06 Apr 2011 18:29:44 +0300 From: George Mamalakis User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110109 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: stable@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: mod_auth_kerb2 broken in 8-STABLE? Or is it heimdal to blame? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 15:29:47 -0000 Dear all, I installed mod_auth_kerb2 on my FreeBSD 8-STABLE machine and tried to use it. After the installation (which was successful(?!?)), the server refused to start giving the error: # /usr/local/etc/rc.d/apache22 start Performing sanity check on apache22 configuration: httpd: Syntax error on line 103 of /usr/local/etc/apache22/httpd.conf: Cannot load /usr/local/libexec/apache22/mod_auth_kerb.so into server: /usr/local/libexec/apache22/mod_auth_kerb.so: Undefined symbol "gsskrb5_register_acceptor_identity" Starting apache22. httpd: Syntax error on line 103 of /usr/local/etc/apache22/httpd.conf: Cannot load /usr/local/libexec/apache22/mod_auth_kerb.so into server: /usr/local/libexec/apache22/mod_auth_kerb.so: Undefined symbol "gsskrb5_register_acceptor_identity" /usr/local/etc/rc.d/apache22: WARNING: failed to start apache22 but ldd showed: # ldd /usr/local/libexec/apache22/mod_auth_kerb.so /usr/local/libexec/apache22/mod_auth_kerb.so: libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x800c00000) libheimntlm.so.10 => /usr/lib/libheimntlm.so.10 (0x800d0a000) libkrb5.so.10 => /usr/lib/libkrb5.so.10 (0x800e0f000) libhx509.so.10 => /usr/lib/libhx509.so.10 (0x800f7e000) libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x8010be000) libcrypto.so.6 => /lib/libcrypto.so.6 (0x8011c0000) libasn1.so.10 => /usr/lib/libasn1.so.10 (0x801461000) libroken.so.10 => /usr/lib/libroken.so.10 (0x8015e3000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016f5000) libc.so.7 => /lib/libc.so.7 (0x800647000) which showed that everything should have been fine. I googled it a bit and found this thread regarding my error message: http://forum.nginx.org/read.php?23,88476 , which started on May 2010, and pointed to this PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=147454 , which started on June 2010. What is stated, is that heimdal-1.1 was broken in FreeBSD, and that it should be fixed at some moment in the future. (I tested mod_auth_kerb2 on another machine running heimdal from ports (1.4_1) and I had exactly the same problem). I searched to find where this notorious function (gsskrb5_register_acceptor_identity) was located, and I found its declaration in: /usr/include/gssapi/gssapi_krb5.h, and its definition in: /usr/lib/libgssapi_krb5.so. So, I added -lgssapi_krb5 in KRB5_LDFLAGS variable of /usr/ports/www/mod_auth_kerb2/work/mod_auth_kerb-5.4/Makefile , since this where the location of gsskrb5_register_acceptor_identity originally seemed to be, and reinstalled the port using gmake this time (inside the port's work directory). After that, the module works just fine. The initial content of this line was: KRB5_LDFLAGS = -L/usr/lib -lgssapi -lheimntlm -lkrb5 -lhx509 -lcom_err -lcrypto -lasn1 -lroken -lcrypt I've sent an analogous email to the port maintainer, but I am not sure if it is their "fault". Hence, I decided to send this email to the stable list for two reasons: First, someone else may be having a similar problem and wants to find a rough solution. Secondly, there are people reading this list that know heimdal's code, so somebody may know another (much more elegant) way to fix this bug. Thank you all for your time in advance, Regards, mamalos. -- George Mamalakis IT Officer Electrical and Computer Engineer (Aristotle Un. of Thessaloniki), MSc (Imperial College of London) Department of Electrical and Computer Engineering Faculty of Engineering Aristotle University of Thessaloniki phone number : +30 (2310) 994379 From owner-freebsd-stable@FreeBSD.ORG Thu Apr 7 11:38:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 524B8106564A for ; Thu, 7 Apr 2011 11:38:40 +0000 (UTC) (envelope-from mamalos@eng.auth.gr) Received: from vergina.eng.auth.gr (vergina.eng.auth.gr [155.207.18.1]) by mx1.freebsd.org (Postfix) with ESMTP id BB0C98FC0C for ; Thu, 7 Apr 2011 11:38:39 +0000 (UTC) Received: from mamalacation.ee.auth.gr (titan.eng.auth.gr [155.207.18.10]) by vergina.eng.auth.gr (8.14.3/8.14.3) with ESMTP id p37B8No5007135 for ; Thu, 7 Apr 2011 14:08:23 +0300 (EEST) (envelope-from mamalos@eng.auth.gr) Message-ID: <4D9D9B22.2020701@eng.auth.gr> Date: Thu, 07 Apr 2011 14:08:18 +0300 From: George Mamalakis User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110109 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D9C86E8.3090402@eng.auth.gr> In-Reply-To: <4D9C86E8.3090402@eng.auth.gr> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: mod_auth_kerb2 broken in 8-STABLE? Or is it heimdal to blame? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Apr 2011 11:38:40 -0000 On 06/04/2011 18:29, George Mamalakis wrote: > Dear all, > > I installed mod_auth_kerb2 on my FreeBSD 8-STABLE machine and tried to > use it. After the installation (which was successful(?!?)), the server > refused to start giving the error: > > # /usr/local/etc/rc.d/apache22 start > Performing sanity check on apache22 configuration: > httpd: Syntax error on line 103 of /usr/local/etc/apache22/httpd.conf: > Cannot load /usr/local/libexec/apache22/mod_auth_kerb.so into server: > /usr/local/libexec/apache22/mod_auth_kerb.so: Undefined symbol > "gsskrb5_register_acceptor_identity" > Starting apache22. > httpd: Syntax error on line 103 of /usr/local/etc/apache22/httpd.conf: > Cannot load /usr/local/libexec/apache22/mod_auth_kerb.so into server: > /usr/local/libexec/apache22/mod_auth_kerb.so: Undefined symbol > "gsskrb5_register_acceptor_identity" > /usr/local/etc/rc.d/apache22: WARNING: failed to start apache22 > > but ldd showed: > > # ldd /usr/local/libexec/apache22/mod_auth_kerb.so > /usr/local/libexec/apache22/mod_auth_kerb.so: > libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x800c00000) > libheimntlm.so.10 => /usr/lib/libheimntlm.so.10 (0x800d0a000) > libkrb5.so.10 => /usr/lib/libkrb5.so.10 (0x800e0f000) > libhx509.so.10 => /usr/lib/libhx509.so.10 (0x800f7e000) > libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x8010be000) > libcrypto.so.6 => /lib/libcrypto.so.6 (0x8011c0000) > libasn1.so.10 => /usr/lib/libasn1.so.10 (0x801461000) > libroken.so.10 => /usr/lib/libroken.so.10 (0x8015e3000) > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016f5000) > libc.so.7 => /lib/libc.so.7 (0x800647000) > > which showed that everything should have been fine. I googled it a bit > and found this thread regarding my error message: > http://forum.nginx.org/read.php?23,88476 , which started on May 2010, > and pointed to this PR: > http://www.freebsd.org/cgi/query-pr.cgi?pr=147454 , which started on > June 2010. What is stated, is that heimdal-1.1 was broken in FreeBSD, > and that it should be fixed at some moment in the future. (I tested > mod_auth_kerb2 on another machine running heimdal from ports (1.4_1) > and I had exactly the same problem). > > I searched to find where this notorious function > (gsskrb5_register_acceptor_identity) was located, and I found its > declaration in: /usr/include/gssapi/gssapi_krb5.h, and its definition > in: /usr/lib/libgssapi_krb5.so. > > So, I added -lgssapi_krb5 in KRB5_LDFLAGS variable of > /usr/ports/www/mod_auth_kerb2/work/mod_auth_kerb-5.4/Makefile , since > this where the location of gsskrb5_register_acceptor_identity > originally seemed to be, and reinstalled the port using gmake this > time (inside the port's work directory). After that, the module works > just fine. The initial content of this line was: > > KRB5_LDFLAGS = -L/usr/lib -lgssapi -lheimntlm -lkrb5 -lhx509 -lcom_err > -lcrypto -lasn1 -lroken -lcrypt > > I've sent an analogous email to the port maintainer, but I am not sure > if it is their "fault". Hence, I decided to send this email to the > stable list for two reasons: First, someone else may be having a > similar problem and wants to find a rough solution. Secondly, there > are people reading this list that know heimdal's code, so somebody may > know another (much more elegant) way to fix this bug. > > Thank you all for your time in advance, > > Regards, > > mamalos. > OK, I spoke with the maintainer who confirmed the problem. He also suggested to change line 96 of /usb/bin/krb5-config to include gssapi_krb5 among its libraries. He also gave me the relevant patch, and asked me to send a PR to FreeBSD. The patch is as follows: --- /usr/bin/krb5-config.orig 2011-02-17 03:18:57.000000000 +0100 +++ /usr/bin/krb5-config 2011-04-06 23:41:31.000000000 +0200 @@ -93,7 +93,7 @@ lib_flags="-L${libdir}" case $library in gssapi) - lib_flags="$lib_flags -lgssapi -lheimntlm" + lib_flags="$lib_flags -lgssapi -lgssapi_krb5 -lheimntlm" ;; kadm-client) lib_flags="$lib_flags -lkadm5clnt" And the relevant PR is: http://www.freebsd.org/cgi/query-pr.cgi?pr=156245 Thank you all for your time, mamalos -- George Mamalakis IT Officer Electrical and Computer Engineer (Aristotle Un. of Thessaloniki), MSc (Imperial College of London) Department of Electrical and Computer Engineering Faculty of Engineering Aristotle University of Thessaloniki phone number : +30 (2310) 994379 From owner-freebsd-stable@FreeBSD.ORG Thu Apr 7 12:16:47 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6DBE1065672 for ; Thu, 7 Apr 2011 12:16:47 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 373F28FC1E for ; Thu, 7 Apr 2011 12:16:46 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.4/8.14.4) with ESMTP id p37CGanp017038 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 7 Apr 2011 15:16:42 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4D9DAB24.1050707@digsys.bg> Date: Thu, 07 Apr 2011 15:16:36 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110307 Thunderbird/3.1.9 MIME-Version: 1.0 To: Pete French References: In-Reply-To: Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: ZFS HAST config preference X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Apr 2011 12:16:47 -0000 On 06.04.11 15:55, Pete French wrote: > Or, I could use ZFS volumes and run HAST on top of these. This means, >> that on each blade, I will have an local ZFS pool. Let's call this setup2. > ...you would need to put a filesystem on to of the > HAST filesystem though, what would that be ? Thanks for your comments Pete. The idea here is to have local ZFS pool(s) on each of the nodes. Then, in each local ZFS pool create ZFS volumes and use these for HAST. On the HAST device, I would again run ZFS, just without any redundancy. Just to avoid having to run fsck after switching roles. This should, I guess, permit safe use of cache and log devices for pools, thus increasing performance. However, no idea how reliability will be affected if sudden loss of one node happens, especially the active node. It seems I am going with setup1 for now, going live in about a week. Daniel From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 10:30:08 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F4084106566B for ; Fri, 8 Apr 2011 10:30:07 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [46.4.210.97]) by mx1.freebsd.org (Postfix) with ESMTP id 95CCB8FC0A for ; Fri, 8 Apr 2011 10:30:07 +0000 (UTC) Received: from localhost (angelica.4lin.net [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id 9CD222CB01 for ; Fri, 8 Apr 2011 12:34:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZRrczRJcBnJH for ; Fri, 8 Apr 2011 12:33:58 +0200 (CEST) Received: from [130.83.160.152] (pcdenny.rbg.informatik.tu-darmstadt.de [130.83.160.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id 4F86C2C058 for ; Fri, 8 Apr 2011 12:33:58 +0200 (CEST) From: Denny Schierz To: freebsd-stable Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-yCQSpFxY2GqCgxDiX1/h" Date: Fri, 08 Apr 2011 12:29:50 +0200 Message-ID: <1302258590.3109.91.camel@pcdenny> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Subject: FreeBSD-8.2: Channel Bounding: LACP or Roundrobin? With Cisco Catalyst X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 10:30:08 -0000 --=-yCQSpFxY2GqCgxDiX1/h Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable hi, I want to bound two e1000 (1Gb/s) channels and use at the moment LCAP, but the max throughput is slower, than without channel bounding. I've got round about 70MB/s instead of > 150MB/s - 200MB/s. I used iperf with standard options: :~$ iperf -f M -c 1.2.3.4 ------------------------------------------------------------ Client connecting to 1.2.3.4, TCP port 5001 TCP window size: 0.02 MByte (default) ------------------------------------------------------------ [ 3] local 1.2.3.5 port 58637 connected with 1.2.3.4 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 705 MBytes 70.5 MBytes/sec If a second PC do the same, than my 70MB/s splittet into ~30MB/s and ~40MB/s config: root@iscsihead-m:~# ifconfig lagg0 lagg0: flags=3D8843 metric 0 mtu 1500 options=3D219b ether 00:15:17:f1:5d:5f inet6 fe80::215:17ff:fef1:5d5f%lagg0 prefixlen 64 scopeid 0x5=20 inet 1.2.3.4 netmask 0xffffffc0 broadcast 1.2.3.255 nd6 options=3D3 media: Ethernet autoselect status: active laggproto lacp laggport: em1 flags=3D1c laggport: em0 flags=3D1c Config from the Cisco: cisco#sh run int po3 Building configuration... Current configuration : 119 bytes ! interface Port-channel3 description iscsi-test switchport switchport access vlan 111 switchport mode access end #sh etherchannel summary=20 Flags: D - down P - bundled in port-channel I - stand-alone s - suspended R - Layer3 S - Layer2 U - in use f - failed to allocate aggregator M - not in use, minimum links not met u - unsuitable for bundling w - waiting to be aggregated d - default port Number of channel-groups in use: 3 Number of aggregators: 3 Group Port-channel Protocol Ports ------+-------------+-----------+------------------------------------------= ----- 1 Po1(SU) - Gi1/1(P) Gi1/2(P) =20 2 Po2(SU) - Gi6/17(P) Gi6/18(P) Gi6/19(P) =20 3 Po3(SU) LACP Gi5/41(P) Gi5/44(P) =20 cisco#sh run int gi5/41 Building configuration... Current configuration : 182 bytes ! interface GigabitEthernet5/41 description iscsi-head1 switchport access vlan 111 switchport mode access no cdp enable channel-group 3 mode active spanning-tree portfast end #sh run int gi5/44 Building configuration... Current configuration : 183 bytes ! interface GigabitEthernet5/44 description GRAU1_iscsi2 switchport access vlan 111 switchport mode access no cdp enable channel-group 3 mode active spanning-tree portfast end #sh etherchannel load-balance=20 EtherChannel Load-Balancing Configuration: src-dst-mac EtherChannel Load-Balancing Addresses Used Per-Protocol: Non-IP: Source XOR Destination MAC address IPv4: Source XOR Destination MAC address IPv6: Source XOR Destination MAC address What I saw with tcpdump: it seems, that only one device is used. Maybe, Cisco uses mac and FreeBSD IP ? Any suggestions? --=-yCQSpFxY2GqCgxDiX1/h Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAk2e45wACgkQKlzhkqt9P+ArWgCbBYIk9TAHCRNhbtdIIWYL0llx sPwAn2evRxvJuZjbs/0vwfuO1NdRLuIV =hJEQ -----END PGP SIGNATURE----- --=-yCQSpFxY2GqCgxDiX1/h-- From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 11:28:39 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CCE2106564A; Fri, 8 Apr 2011 11:28:39 +0000 (UTC) (envelope-from danger@rulez.sk) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id AA5BE8FC12; Fri, 8 Apr 2011 11:28:38 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id 5B3DF94D70; Fri, 8 Apr 2011 13:12:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id 6s+sVI58EEvg; Fri, 8 Apr 2011 13:12:48 +0200 (CEST) Received: from danger-mbp.local (59576.ba.3pp.slovanet.sk [84.16.39.226]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: danger@rulez.sk) by services.syscare.sk (Postfix) with ESMTPSA id 5B63294D61; Fri, 8 Apr 2011 13:12:48 +0200 (CEST) Message-ID: <4D9EEDAF.3020803@rulez.sk> Date: Fri, 08 Apr 2011 13:12:47 +0200 From: =?windows-1252?Q?Daniel_Ger=9Eo?= User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.17pre) Gecko/20110331 Lanikai/3.1.10pre MIME-Version: 1.0 To: stable@freebsd.org Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Motin , brucec@freebsd.org Subject: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 11:28:39 -0000 Hello guys, I have a new machine with Xeon(R) CPU X5650 2666.77-MHz and I would like to utilize powerd(8) on it however, when I run `powerd -v -r90' I see something like this: load 64%, current freq 2668 MHz ( 0), wanted freq 5336 MHz load 120%, current freq 2668 MHz ( 0), wanted freq 5336 MHz load 173%, current freq 2668 MHz ( 0), wanted freq 5336 MHz load 62%, current freq 2668 MHz ( 0), wanted freq 5336 MHz load 82%, current freq 2668 MHz ( 0), wanted freq 5336 MHz load 110%, current freq 2668 MHz ( 0), wanted freq 5336 MHz even though the machine is according to top(1) ~90% idle; So I realized, that powerd might take the load as the sum of loads of all the cores (12), so I tried to tweak powerd arguments like this: `powerd -v -r 1000 -i 600' but that errors for me with: root@[s1-a ~]# powerd -v -r 1000 -i 600 powerd: 1000 is not a valid percent Well, that makes sense, but why powerd itself knows about load > 100% but doesn't allow me to specify it? Is this bug? I suppose not if it works for other people... Other question would be why powerd wants to set freq 5336, when it is not available at all (would be nice to have it heh.): dev.cpu.0.freq_levels: 2668/109000 2533/81000 2400/69000 2267/58000 2133/48000 2000/40000 1867/32000 1733/26000 1600/20000 1400/17500 1200/15000 1000/12500 The symptoms seem to show that there's a bug in the code calculating the cpu load. Any ideas what may be wrong? Examle of two consecutive cp_times sysctl output: kern.cp_times: 4182996 0 306925 85623 13563403 3164971 0 201479 93110 14679313 3450792 0 258166 80198 14349717 2795270 0 180252 76701 15086650 2952777 0 217156 119627 14849313 2418067 0 158594 73497 15488715 2408492 0 175131 104377 15450873 2003803 0 131790 75753 15927527 2456736 0 178894 36963 15466280 1607095 0 117396 4197 16410185 2127878 0 147639 30804 15832552 1406621 0 92686 1058 16638508 kern.cp_times: 4183013 0 306927 85626 13563469 3164980 0 201482 93110 14679390 3450796 0 258167 80199 14349800 2795274 0 180252 76701 15086735 2952780 0 217157 119629 14849396 2418070 0 158597 73497 15488798 2408499 0 175132 104377 15450954 2003804 0 131791 75753 15927614 2456744 0 178897 36963 15466358 1607098 0 117398 4197 16410269 2127880 0 147640 30804 15832638 1406621 0 92686 1058 16638597 Thanks! -- S pozdravom / Best regards Daniel Gerzo From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 12:14:16 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BAF2106566B; Fri, 8 Apr 2011 12:14:16 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 600EF8FC0A; Fri, 8 Apr 2011 12:14:14 +0000 (UTC) Received: by bwz12 with SMTP id 12so3519863bwz.13 for ; Fri, 08 Apr 2011 05:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=s+3rCIwTChJBg4otuw5qeQzEZ25xtH9lhER1usKloCI=; b=knxk7KqpYAiCembC0UFFP7sDX/f4TMi3k/d9gq1FL9yLM/2brlogGwKqwSirqcTZxi 0yuHThmsSgR+Gz3ptjs++PHshgURyG7dZPIyRI7aEv+W9j6rB3eS0qWNmHKc9eVrEnOr 8tQRDBpQwBmPjuH492UYh7u2FTFqV5HqgGzyk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=OMX0PQ/nv4GXWf0jm0ZJh3oihEgtU4ZQwk7xwxni/N0gDxF8khrJqva9BDRtAaJlEE KtZdkZxcoBJyYsvIV+06CcEy5XBP9JWZriI+aOs3rMHoU/2A7TBDjeDmDkfEqt6yZtob o0upJa1o2OjjWLVNKT3CmPEDdNlsPH+9hRJuA= Received: by 10.204.19.16 with SMTP id y16mr1865589bka.186.1302262961363; Fri, 08 Apr 2011 04:42:41 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (95-109-204-34.dialup.umc.net.ua [95.109.204.34]) by mx.google.com with ESMTPS id q24sm1597766bks.21.2011.04.08.04.42.33 (version=SSLv3 cipher=OTHER); Fri, 08 Apr 2011 04:42:40 -0700 (PDT) Sender: Alexander Motin Message-ID: <4D9EF48C.9070907@FreeBSD.org> Date: Fri, 08 Apr 2011 14:42:04 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: =?windows-1252?Q?Daniel_Ger=9Eo?= References: <4D9EEDAF.3020803@rulez.sk> In-Reply-To: <4D9EEDAF.3020803@rulez.sk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: stable@freebsd.org, brucec@freebsd.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 12:14:16 -0000 Hi. On 08.04.2011 14:12, Daniel Geržo wrote: > I have a new machine with Xeon(R) CPU X5650 2666.77-MHz and I would like > to utilize powerd(8) on it however, when I run `powerd -v -r90' I see > something like this: > > load 64%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > load 120%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > load 173%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > load 62%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > load 82%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > load 110%, current freq 2668 MHz ( 0), wanted freq 5336 MHz > > even though the machine is according to top(1) ~90% idle; So I realized, > that powerd might take the load as the sum of loads of all the cores > (12), so I tried to tweak powerd arguments like this: > > `powerd -v -r 1000 -i 600' > > but that errors for me with: > > root@[s1-a ~]# powerd -v -r 1000 -i 600 > powerd: 1000 is not a valid percent > > Well, that makes sense, but why powerd itself knows about load > 100% > but doesn't allow me to specify it? Is this bug? I suppose not if it > works for other people... It is reasonable limitation. powerd can't know how load distributed among multiple cores in time. If all cores are equally busy at lets say 10% (that gives 120% total) and cores are never waiting for each other then obviously frequency could be reduced. But if the same 120% mean 100%+20%, or if load is equally spread, but processes on different cores are waiting for each other, then reducing frequency will reduce performance. powerd can't know that and so stays on a safe side. > Other question would be why powerd wants to set freq 5336, when it is > not available at all (would be nice to have it heh.): You may see there it is a "wanted" frequency, not real one. :) It is internal implementation details. In such way powerd implements keeping a full frequency for some time after the load dropped. It's not a bug. On multi-core systems like this power management can better be done on per-core bases. Powerd can't control frequencies on per-core basis (also because it require non-trivial interoperation with scheduler). But if your ACPI BIOS allows, you can try to put unused cores into deeper C-states, that may give better power saving and TurboBoost on busy cores as a bonus. It works better on 9-CURRENT, but on 8-STABLE some bonuses still could be achieved. You may want to look here: http://wiki.freebsd.org/TuningPowerConsumption -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 13:52:49 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFEF4106564A for ; Fri, 8 Apr 2011 13:52:49 +0000 (UTC) (envelope-from telbizov@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id A7F358FC15 for ; Fri, 8 Apr 2011 13:52:49 +0000 (UTC) Received: by qwc9 with SMTP id 9so2327076qwc.13 for ; Fri, 08 Apr 2011 06:52:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=H0Ml+G0CPPPjtg6LW43y48+03PgCbXytxfpdZGUzMwk=; b=OGpWAzUfUlXBNB10oXdSVSV0pcklv0QxVzpdHeJxrIN4nRUOVRlvcg2JJMqo0aIwU7 mK6vd+xQawNiCL6B7cPJoqhdr6i8sflW4egwsu35LvVaGQSfLGU5ijJcUaDdlzQLzwE8 S0TW2lU1JfcemLmTLH4wV2iDdOEZPITg6ITfs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CIBePegDPelBzWDJmJZx1QcCKPdfScteFwwNphqNg1maEyB528EGi+5+JtVtGAlE5P EeYGindP2fi/ebgaC2iomuXSCZYtEd10ZecTQzlZN7w54jtIdyb+Xv8f8tyZxwi9DN/y 2kNgUlVaZNJRsxHBhhv/Itt7DVvKGpu/5j3WQ= MIME-Version: 1.0 Received: by 10.229.141.71 with SMTP id l7mr1845107qcu.44.1302270768506; Fri, 08 Apr 2011 06:52:48 -0700 (PDT) Received: by 10.229.36.147 with HTTP; Fri, 8 Apr 2011 06:52:48 -0700 (PDT) In-Reply-To: <1302258590.3109.91.camel@pcdenny> References: <1302258590.3109.91.camel@pcdenny> Date: Fri, 8 Apr 2011 06:52:48 -0700 Message-ID: From: Rumen Telbizov To: Denny Schierz Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable Subject: Re: FreeBSD-8.2: Channel Bounding: LACP or Roundrobin? With Cisco Catalyst X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 13:52:50 -0000 Denny, Since LACP uses hashing to determine which channel to send the packet to the traffic between two nodes (ip:mac ip:mac) will always get bound to only one of the two channels. I am using HP procurve's here and they do seem to hash by ip too although I can't see/tweak that as in catalyst. If you can add the IP address in the hashing function of the switch then you have a better chance of achieving what you're trying to by bringing an IP alias (might have to try a few until you find one that maps to the opposite channel) and then having two iperfs between different IP addresses. In the real live scenario it will be equivalent to having multiple iscsi targets between the two hosts on top of different IPs if this works for you. As for the load balancing option - my HP 2910al's seemed to get CPU overloaded when I push a gigabit of traffic with this option. My guess is the crazy mac address change might be exhausting the cpu much faster. I'd be interested to see how this affects catalyst - let me know. Good luck, Rumen Telbizov On Fri, Apr 8, 2011 at 3:29 AM, Denny Schierz wrote: > hi, > > I want to bound two e1000 (1Gb/s) channels and use at the moment LCAP, > but the max throughput is slower, than without channel bounding. I've > got round about 70MB/s instead of > 150MB/s - 200MB/s. > > I used iperf with standard options: > > :~$ iperf -f M -c 1.2.3.4 > ------------------------------------------------------------ > Client connecting to 1.2.3.4, TCP port 5001 > TCP window size: 0.02 MByte (default) > ------------------------------------------------------------ > [ 3] local 1.2.3.5 port 58637 connected with 1.2.3.4 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 705 MBytes 70.5 MBytes/sec > > If a second PC do the same, than my 70MB/s splittet into ~30MB/s and > ~40MB/s > > config: > > root@iscsihead-m:~# ifconfig lagg0 > lagg0: flags=8843 metric 0 mtu > 1500 > > > options=219b > ether 00:15:17:f1:5d:5f > inet6 fe80::215:17ff:fef1:5d5f%lagg0 prefixlen 64 scopeid 0x5 > inet 1.2.3.4 netmask 0xffffffc0 broadcast 1.2.3.255 > nd6 options=3 > media: Ethernet autoselect > status: active > laggproto lacp > laggport: em1 flags=1c > laggport: em0 flags=1c > > > Config from the Cisco: > > > cisco#sh run int po3 > Building configuration... > > Current configuration : 119 bytes > ! > interface Port-channel3 > description iscsi-test > switchport > switchport access vlan 111 > switchport mode access > end > > > > #sh etherchannel summary > Flags: D - down P - bundled in port-channel > I - stand-alone s - suspended > R - Layer3 S - Layer2 > U - in use f - failed to allocate aggregator > > M - not in use, minimum links not met > u - unsuitable for bundling > w - waiting to be aggregated > d - default port > > > Number of channel-groups in use: 3 > Number of aggregators: 3 > > Group Port-channel Protocol Ports > > ------+-------------+-----------+----------------------------------------------- > 1 Po1(SU) - Gi1/1(P) Gi1/2(P) > 2 Po2(SU) - Gi6/17(P) Gi6/18(P) Gi6/19(P) > 3 Po3(SU) LACP Gi5/41(P) Gi5/44(P) > > cisco#sh run int gi5/41 > Building configuration... > > Current configuration : 182 bytes > ! > interface GigabitEthernet5/41 > description iscsi-head1 > switchport access vlan 111 > switchport mode access > no cdp enable > channel-group 3 mode active > spanning-tree portfast > end > > > > #sh run int gi5/44 > Building configuration... > > Current configuration : 183 bytes > ! > interface GigabitEthernet5/44 > description GRAU1_iscsi2 > switchport access vlan 111 > switchport mode access > no cdp enable > channel-group 3 mode active > spanning-tree portfast > end > > > #sh etherchannel load-balance > EtherChannel Load-Balancing Configuration: > src-dst-mac > > EtherChannel Load-Balancing Addresses Used Per-Protocol: > Non-IP: Source XOR Destination MAC address > IPv4: Source XOR Destination MAC address > IPv6: Source XOR Destination MAC address > > > What I saw with tcpdump: it seems, that only one device is used. > > Maybe, Cisco uses mac and FreeBSD IP ? > > Any suggestions? > > -- Rumen Telbizov http://telbizov.com From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 14:58:39 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C5F31065673 for ; Fri, 8 Apr 2011 14:58:39 +0000 (UTC) (envelope-from hosting@syscare.sk) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id EC01F8FC18 for ; Fri, 8 Apr 2011 14:58:38 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id E8A9494140; Fri, 8 Apr 2011 16:42:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id HJ1sj8t7mTPA; Fri, 8 Apr 2011 16:42:42 +0200 (CEST) Received: from hosting.syscare.sk (hosting [188.40.39.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by services.syscare.sk (Postfix) with ESMTPS id 92F4A9410F; Fri, 8 Apr 2011 16:42:42 +0200 (CEST) Received: (from www@localhost) by hosting.syscare.sk (8.14.4/8.14.4/Submit) id p38Egg0W035596; Fri, 8 Apr 2011 16:42:42 +0200 (CEST) (envelope-from hosting@syscare.sk) X-Authentication-Warning: hosting.syscare.sk: www set sender to hosting@syscare.sk using -f To: Alexander Motin X-PHP-Originating-Script: 0:func.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 08 Apr 2011 15:42:42 +0100 From: Daniel Gerzo Organization: The FreeBSD Project In-Reply-To: <4D9EF48C.9070907@FreeBSD.org> References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> Message-ID: X-Sender: danger@FreeBSD.org User-Agent: Roundcube Webmail/0.5.1 Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 14:58:39 -0000 On Fri, 08 Apr 2011 14:42:04 +0300, Alexander Motin wrote: Hello Alexander, thanks for quick reply; >> root@[s1-a ~]# powerd -v -r 1000 -i 600 >> powerd: 1000 is not a valid percent >> >> Well, that makes sense, but why powerd itself knows about load > >> 100% >> but doesn't allow me to specify it? Is this bug? I suppose not if it >> works for other people... > > It is reasonable limitation. powerd can't know how load distributed > among multiple cores in time. If all cores are equally busy at lets > say 10% (that gives 120% total) and cores are never waiting for each > other then obviously frequency could be reduced. But if the same 120% > mean 100%+20%, or if load is equally spread, but processes on > different cores are waiting for each other, then reducing frequency > will reduce performance. powerd can't know that and so stays on a > safe > side. OK, I understand what you are saying here. On the other side, I know pretty well how the load is distributed - in this particular case, the box is a web server, running ~30 php-cgi processes. This kind of operation doesn't require very high frequency and I suspect the cores are never waiting for each other. There could be an option which would allow an administrator to decide whether this is the case and allow him to set a higher -r and -i values, what do you think? >> Other question would be why powerd wants to set freq 5336, when it >> is >> not available at all (would be nice to have it heh.): > > You may see there it is a "wanted" frequency, not real one. :) It is > internal implementation details. In such way powerd implements > keeping > a full frequency for some time after the load dropped. It's not a > bug. OK :-) I actually though powerd always honors the values from dev.cpu.0.freq_levels (and 5336 is not there), so it looked a little weird to me. > On multi-core systems like this power management can better be done > on per-core bases. Powerd can't control frequencies on per-core basis > (also because it require non-trivial interoperation with scheduler). > But if your ACPI BIOS allows, you can try to put unused cores into > deeper C-states, that may give better power saving and TurboBoost on > busy cores as a bonus. It works better on 9-CURRENT, but on 8-STABLE > some bonuses still could be achieved. Any idea what I should look for in the BIOS? This is 8-STABLE, any idea whether there's a MFC plan for the extra 9-CURRENT bonuses? > You may want to look here: > http://wiki.freebsd.org/TuningPowerConsumption From reading this, are you reffering above to the C2 states? (seems like C3 is not optimal for this kind of operation...) Thanks. -- Kind regards Daniel From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 15:02:42 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5FD56106564A for ; Fri, 8 Apr 2011 15:02:42 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id DDF678FC18 for ; Fri, 8 Apr 2011 15:02:41 +0000 (UTC) Received: by bwz12 with SMTP id 12so3685111bwz.13 for ; Fri, 08 Apr 2011 08:02:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=xytLL3J54DO8RZCXdTVZviZ0JdeBFg9QkwwWKdXIKDY=; b=QOrzzumbAXe9e7pYLF5xY/O89gQHIhj4rFzOTOidZ3/CA5lbhZHYVEWt4sH7eAP3ab eEZKEXonJu51lt5JljX01imKA1aJ4zB8Z1ncPaejOJ+Apq+kCfj1tls7R/iZ2p41hQvx X5q0NfBvtF38uhNjt3eosZO5yGWkeK2Y0Z4lI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=MRKTdgSenjweC3QyIXqqWHRDgyCykhqRImM6M/IvGiI9Lm+Mcjikzi5/F6ulgCgORF nkqYlI9t3mOYwc6akR6x7cq0SOXK0JZKk5gZuZ6i+GHTtCpDOLnWpYcNtvPxj31P13/M RtL6OaSi6Jfb6LIgk6T5XY1m7lJ5ZxjJ95iaY= Received: by 10.204.16.216 with SMTP id p24mr2077433bka.5.1302274960498; Fri, 08 Apr 2011 08:02:40 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (95-109-175-10.dialup.umc.net.ua [95.109.175.10]) by mx.google.com with ESMTPS id c11sm1693936bkc.14.2011.04.08.08.02.34 (version=SSLv3 cipher=OTHER); Fri, 08 Apr 2011 08:02:39 -0700 (PDT) Sender: Alexander Motin Message-ID: <4D9F2384.5000104@FreeBSD.org> Date: Fri, 08 Apr 2011 18:02:28 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: Daniel Gerzo References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 15:02:42 -0000 On 08.04.2011 17:42, Daniel Gerzo wrote: > On Fri, 08 Apr 2011 14:42:04 +0300, Alexander Motin wrote: >>> root@[s1-a ~]# powerd -v -r 1000 -i 600 >>> powerd: 1000 is not a valid percent >>> >>> Well, that makes sense, but why powerd itself knows about load > 100% >>> but doesn't allow me to specify it? Is this bug? I suppose not if it >>> works for other people... >> >> It is reasonable limitation. powerd can't know how load distributed >> among multiple cores in time. If all cores are equally busy at lets >> say 10% (that gives 120% total) and cores are never waiting for each >> other then obviously frequency could be reduced. But if the same 120% >> mean 100%+20%, or if load is equally spread, but processes on >> different cores are waiting for each other, then reducing frequency >> will reduce performance. powerd can't know that and so stays on a safe >> side. > > OK, I understand what you are saying here. On the other side, I know > pretty well how the load is distributed - in this particular case, the > box is a web server, running ~30 php-cgi processes. > This kind of operation doesn't require very high frequency and I suspect > the cores are never waiting for each other. There could be an option > which would allow an administrator to decide whether this is the case > and allow him to set a higher -r and -i values, what do you think? I think it should be possible with minimal changes. >>> Other question would be why powerd wants to set freq 5336, when it is >>> not available at all (would be nice to have it heh.): >> >> You may see there it is a "wanted" frequency, not real one. :) It is >> internal implementation details. In such way powerd implements keeping >> a full frequency for some time after the load dropped. It's not a bug. > > OK :-) I actually though powerd always honors the values from > dev.cpu.0.freq_levels (and 5336 is not there), so it looked a little > weird to me. It does it on left side, but no longer on the right side. Abstracting from real frequencies made behavior more universal and predictable. >> On multi-core systems like this power management can better be done >> on per-core bases. Powerd can't control frequencies on per-core basis >> (also because it require non-trivial interoperation with scheduler). >> But if your ACPI BIOS allows, you can try to put unused cores into >> deeper C-states, that may give better power saving and TurboBoost on >> busy cores as a bonus. It works better on 9-CURRENT, but on 8-STABLE >> some bonuses still could be achieved. > > Any idea what I should look for in the BIOS? Something about C-states, or Cx-states on the CPU page. But first look at dev.cpu.X.cx_supported to make sure it is not already present and just unused. > This is 8-STABLE, any idea whether there's a MFC plan for the extra > 9-CURRENT bonuses? I suppose around May. >> You may want to look here: >> http://wiki.freebsd.org/TuningPowerConsumption > > From reading this, are you reffering above to the C2 states? (seems > like C3 is not optimal for this kind of operation...) The deeper state, the more power saved. To get most of it and to get TurboBoost working you need at least C3 CPU state (ACPI may report it with different number). Some latest Intel CPUs have no described problems with C3 and LAPIC, for others described system tuning requited. PS: Using powerd in best case wont hurt performance, while using C-states may even increase it in some cases because of TurboBoost. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 16:53:40 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F5FF1065674; Fri, 8 Apr 2011 16:53:40 +0000 (UTC) (envelope-from hosting@syscare.sk) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id 4AC6B8FC14; Fri, 8 Apr 2011 16:53:39 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id A6E8494B58; Fri, 8 Apr 2011 18:53:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id gyUhI9B9mjDL; Fri, 8 Apr 2011 18:53:36 +0200 (CEST) Received: from hosting.syscare.sk (hosting [188.40.39.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by services.syscare.sk (Postfix) with ESMTPS id 5CF5494B36; Fri, 8 Apr 2011 18:53:36 +0200 (CEST) Received: (from www@localhost) by hosting.syscare.sk (8.14.4/8.14.4/Submit) id p38GraLA051995; Fri, 8 Apr 2011 18:53:36 +0200 (CEST) (envelope-from hosting@syscare.sk) X-Authentication-Warning: hosting.syscare.sk: www set sender to hosting@syscare.sk using -f To: Alexander Motin X-PHP-Originating-Script: 0:func.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 08 Apr 2011 17:53:36 +0100 From: Daniel Gerzo Organization: The FreeBSD Project In-Reply-To: <4D9F2384.5000104@FreeBSD.org> References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> Message-ID: <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> X-Sender: danger@FreeBSD.org User-Agent: Roundcube Webmail/0.5.1 Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 16:53:40 -0000 On Fri, 08 Apr 2011 18:02:28 +0300, Alexander Motin wrote: >> OK, I understand what you are saying here. On the other side, I know >> pretty well how the load is distributed - in this particular case, >> the >> box is a web server, running ~30 php-cgi processes. >> This kind of operation doesn't require very high frequency and I >> suspect >> the cores are never waiting for each other. There could be an option >> which would allow an administrator to decide whether this is the >> case >> and allow him to set a higher -r and -i values, what do you think? > > I think it should be possible with minimal changes. So, here is my attempt to implement it: http://danger.rulez.sk/powerd.diff Can you please review & comment? I should be able to commit it mysqlf if you consider it acceptable. It seems to work for me :) >> >> Any idea what I should look for in the BIOS? > > Something about C-states, or Cx-states on the CPU page. But first > look at dev.cpu.X.cx_supported to make sure it is not already present > and just unused. Seems like it was enabled by default. I have like these: dev.cpu.0.cx_supported: C1/3 C2/96 C3/128 Does that mean I only need to set these in rc.conf?: performance_cx_lowest="C3" economy_cx_lowest="C3" Then run /etc/rc.d/power_profile 0x00? May it cause any instability? >> This is 8-STABLE, any idea whether there's a MFC plan for the extra >> 9-CURRENT bonuses? > > I suppose around May. Do you have some patches? If not you don't really need to make them just for me, I can wait a little. >>> You may want to look here: >>> http://wiki.freebsd.org/TuningPowerConsumption >> >> From reading this, are you reffering above to the C2 states? (seems >> like C3 is not optimal for this kind of operation...) > > The deeper state, the more power saved. To get most of it and to get > TurboBoost working you need at least C3 CPU state (ACPI may report it > with different number). Some latest Intel CPUs have no described > problems with C3 and LAPIC, for others described system tuning > requited. I believe this is pretty recent CPU (6 core Xeon X5650). Do you know about any problems? > PS: Using powerd in best case wont hurt performance, while using > C-states may even increase it in some cases because of TurboBoost. If I want to use C-states, should I stop to use powerd, or is it possible to use them both together? Thanks! -- Kind regards Daniel From owner-freebsd-stable@FreeBSD.ORG Fri Apr 8 17:52:43 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 514AC106564A; Fri, 8 Apr 2011 17:52:43 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id AF54B8FC0A; Fri, 8 Apr 2011 17:52:42 +0000 (UTC) Received: by wyf23 with SMTP id 23so3792550wyf.13 for ; Fri, 08 Apr 2011 10:52:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=Tl+BxZ8FMbdQr6O2aWIuPB0DqsJQCL7/N8KCcGYpDNI=; b=VRTE3EiSAh4ScvYD3hKsxSzII6xI2tKff+5pha+DpUGrzUPmkFCtgUQ75XKpeoBa3H jPAJ+imSSgdNe8wgqOGpJcEqelv2+2NQ7bGVB67LMAMz1iasNAZsX3WGR7LLXxQ9Hug/ TAlBdrj/8iIaKgPiL0z4+t6yFtNr1Oo1SSaJ4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=TAPOOYuLneuICe2bMbYl5ffwCelGYozVkFVkFpqu8UmwrwWfFJAyitKbP645wfxmrc U0GAiRmMubgiq5DyD0zto2rTyheiI95ArZNuFPlc/XNM2pu8ZEhMM5iL4DYEU2c1JDQ+ tf7BegfA9Kqr2PItjOTMvVATEfbjnDJGrfRFg= Received: by 10.216.62.74 with SMTP id x52mr2427058wec.45.1302285161619; Fri, 08 Apr 2011 10:52:41 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (95-109-196-139.dialup.umc.net.ua [95.109.196.139]) by mx.google.com with ESMTPS id d54sm1472451wej.10.2011.04.08.10.52.36 (version=SSLv3 cipher=OTHER); Fri, 08 Apr 2011 10:52:40 -0700 (PDT) Sender: Alexander Motin Message-ID: <4D9F4B58.3050104@FreeBSD.org> Date: Fri, 08 Apr 2011 20:52:24 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: Daniel Gerzo References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> In-Reply-To: <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 17:52:43 -0000 On 08.04.2011 19:53, Daniel Gerzo wrote: > On Fri, 08 Apr 2011 18:02:28 +0300, Alexander Motin wrote: >>> OK, I understand what you are saying here. On the other side, I know >>> pretty well how the load is distributed - in this particular case, the >>> box is a web server, running ~30 php-cgi processes. >>> This kind of operation doesn't require very high frequency and I suspect >>> the cores are never waiting for each other. There could be an option >>> which would allow an administrator to decide whether this is the case >>> and allow him to set a higher -r and -i values, what do you think? >> >> I think it should be possible with minimal changes. > > So, here is my attempt to implement it: > http://danger.rulez.sk/powerd.diff > Can you please review & comment? I should be able to commit it mysqlf if > you consider it acceptable. It seems to work for me :) Looks fine, except that -f option have to be the first, that is not obvious. Another moment -- I've noticed some load constants hardcoded there. They should also be handled to make higher values to work properly. >>> Any idea what I should look for in the BIOS? >> >> Something about C-states, or Cx-states on the CPU page. But first >> look at dev.cpu.X.cx_supported to make sure it is not already present >> and just unused. > > Seems like it was enabled by default. I have like these: > dev.cpu.0.cx_supported: C1/3 C2/96 C3/128 > > Does that mean I only need to set these in rc.conf?: > performance_cx_lowest="C3" > economy_cx_lowest="C3" > > Then run /etc/rc.d/power_profile 0x00? It short - yes. In long - read the link I've given. > May it cause any instability? It you won't switch from LAPIC to other timer and it stop - your system will freeze, or at least not work well. You should notice problems immediately, if there are. >>> This is 8-STABLE, any idea whether there's a MFC plan for the extra >>> 9-CURRENT bonuses? >> >> I suppose around May. > > Do you have some patches? If not you don't really need to make them just > for me, I can wait a little. Last ones I've generated are five months old: http://people.freebsd.org/~mav/timers_merge/ They are large and I am not sure how good they apply now. >>>> You may want to look here: >>>> http://wiki.freebsd.org/TuningPowerConsumption >>> >>> From reading this, are you reffering above to the C2 states? (seems >>> like C3 is not optimal for this kind of operation...) >> >> The deeper state, the more power saved. To get most of it and to get >> TurboBoost working you need at least C3 CPU state (ACPI may report it >> with different number). Some latest Intel CPUs have no described >> problems with C3 and LAPIC, for others described system tuning >> requited. > > I believe this is pretty recent CPU (6 core Xeon X5650). Do you know > about any problems? I have no idea about these Xeons. I know just that LAPIC of the my Core i5 works fine in C3, while one of the my Core i7 doesn't. >> PS: Using powerd in best case wont hurt performance, while using >> C-states may even increase it in some cases because of TurboBoost. > > If I want to use C-states, should I stop to use powerd, or is it > possible to use them both together? I am using both together on my laptop. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 07:42:52 2011 Return-Path: Delivered-To: Stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B30B106566B for ; Sat, 9 Apr 2011 07:42:52 +0000 (UTC) (envelope-from david.marec@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [91.121.44.19]) by mx1.freebsd.org (Postfix) with ESMTP id 153BB8FC18 for ; Sat, 9 Apr 2011 07:42:51 +0000 (UTC) Received: from www.lamaiziere.net (unknown [192.168.1.1]) by smtp.lamaiziere.net (Postfix) with ESMTP id 323DF633205 for ; Sat, 9 Apr 2011 09:25:50 +0200 (CEST) Received: from 93.28.96.112 (SquirrelMail authenticated user david.nul) by lamaiziere.net with HTTP; Sat, 9 Apr 2011 09:25:50 +0200 (CEST) Message-ID: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> Date: Sat, 9 Apr 2011 09:25:50 +0200 (CEST) From: "David Marec" To: Stable@FreeBSD.org User-Agent: SquirrelMail/1.4.17 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-15 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: Subject: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: david.marec@davenulle.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 07:42:52 -0000 Hi guys. Since the release of FreeBSD 8.2, building world fails on the following error: <------> david:/home/david#cd /usr/src david:/usr/src#make -j4 buildworld && make kernel -------------------------------------------------------------- >>> World build started on Sat Apr 9 09:16:50 CEST 2011 -------------------------------------------------------------- -------------------------------------------------------------- >>> Rebuilding the temporary build tree -------------------------------------------------------------- rm -rf /usr/obj/usr/src/tmp rm -rf /usr/obj/usr/src/lib32 rm: /usr/obj/usr/src/lib32/usr/lib32/libc.so.7: Operation not permitted rm: /usr/obj/usr/src/lib32/usr/lib32/libcrypt.so.5: Operation not permitted rm: /usr/obj/usr/src/lib32/usr/lib32/libthr.so.3: Operation not permitted rm: /usr/obj/usr/src/lib32/usr/lib32/librt.so.1: Operation not permitted rm: /usr/obj/usr/src/lib32/usr/lib32: Directory not empty rm: /usr/obj/usr/src/lib32/usr: Directory not empty rm: /usr/obj/usr/src/lib32: Directory not empty *** Error code 1 1 error *** Error code 2 1 error david:/usr/src#ls -lo /usr/obj/usr/src/lib32/usr/lib32/ total 1262 -r--r--r-- 1 root wheel schg 1143468 22 mar 21:19 libc.so.7 -r--r--r-- 1 root wheel schg 32060 22 mar 21:19 libcrypt.so.5 -r--r--r-- 1 root wheel schg 16412 22 mar 21:22 librt.so.1 -r--r--r-- 1 root wheel schg 76412 22 mar 21:20 libthr.so.3 <------> Im a running FreeBSD 8.2-Stable for amd64. So, before building world, i have to change the flags for the files above. There was no need to do this before. Any idea to get rid of this issue ? -- Cordialement, -- David Marec: http://user.lamaiziere.net/david/Site/ http://www.freebsd.org/fr/ http://www.diablotins.org/ From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 08:13:54 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 535A7106564A for ; Sat, 9 Apr 2011 08:13:54 +0000 (UTC) (envelope-from danger@FreeBSD.org) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id DC9FC8FC0A for ; Sat, 9 Apr 2011 08:13:53 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id 14CE795C08; Sat, 9 Apr 2011 09:57:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id J9pVVAU8wqKn; Sat, 9 Apr 2011 09:57:29 +0200 (CEST) Received: from danger-mbp.local (188-167-63-91.dynamic.chello.sk [188.167.63.91]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: danger@rulez.sk) by services.syscare.sk (Postfix) with ESMTPSA id CA94095BF6; Sat, 9 Apr 2011 09:57:29 +0200 (CEST) Message-ID: <4DA01168.8050704@FreeBSD.org> Date: Sat, 09 Apr 2011 09:57:28 +0200 From: Daniel Gerzo Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.17pre) Gecko/20110331 Lanikai/3.1.10pre MIME-Version: 1.0 To: Alexander Motin References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> <4D9F4B58.3050104@FreeBSD.org> In-Reply-To: <4D9F4B58.3050104@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 08:13:54 -0000 On 8.4.2011 19:52, Alexander Motin wrote: >> So, here is my attempt to implement it: >> http://danger.rulez.sk/powerd.diff >> Can you please review & comment? I should be able to commit it mysqlf if >> you consider it acceptable. It seems to work for me :) > > Looks fine, except that -f option have to be the first, that is not > obvious. Another moment -- I've noticed some load constants hardcoded > there. They should also be handled to make higher values to work properly. I tried to be more explicit in the error message which tries to emphasis the need to put it first. I don't know myself how it would be possible to code it so that the -f doesn't need to be first. Ideas? Do you mean the values around lines of 730 - 762? From what I have observed, if I have a machine that is a little more loaded (say 300%) and the load goes up, it tries to increases the performance to quite high freq (5336) and when the load decreases again, it takes quite a while to go down from 5366 to a frequency that is actually available to decrease the performance (something less than 2934). So the lower frequency is used for too short time because it takes too much time to get it... >> Seems like it was enabled by default. I have like these: >> dev.cpu.0.cx_supported: C1/3 C2/96 C3/128 >> >> Does that mean I only need to set these in rc.conf?: >> performance_cx_lowest="C3" >> economy_cx_lowest="C3" >> >> Then run /etc/rc.d/power_profile 0x00? > > It short - yes. In long - read the link I've given. > >> May it cause any instability? > > It you won't switch from LAPIC to other timer and it stop - your system > will freeze, or at least not work well. You should notice problems > immediately, if there are. So I will also need to change the kern.timecounter.hardware to i8254? I suppose it will cause a little less precise time, but should I expect lower performance? I don't care that much about the time accuracy. How do I know the C3 is active? And how does it switch back to C1 for example? >>>> This is 8-STABLE, any idea whether there's a MFC plan for the extra >>>> 9-CURRENT bonuses? >>> >>> I suppose around May. >> >> Do you have some patches? If not you don't really need to make them just >> for me, I can wait a little. > > Last ones I've generated are five months old: > http://people.freebsd.org/~mav/timers_merge/ > They are large and I am not sure how good they apply now. I guess I will just stick with vanilla 8-stable and then update. >>>>> You may want to look here: >>>>> http://wiki.freebsd.org/TuningPowerConsumption -- S pozdravom / Best regards Daniel Gerzo, FreeBSD committer From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 08:24:33 2011 Return-Path: Delivered-To: Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6638A106564A for ; Sat, 9 Apr 2011 08:24:33 +0000 (UTC) (envelope-from lichray@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 1FCD98FC08 for ; Sat, 9 Apr 2011 08:24:32 +0000 (UTC) Received: by qyk35 with SMTP id 35so300127qyk.13 for ; Sat, 09 Apr 2011 01:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=qSBX9+RirXQb0j+yAtouPL/gpI6cJ/Wqxj5dFgtu6Lo=; b=vkscWW+DaDnDg+32q3tj/Lg37ZTspRok+cBWzi5ArudWaYDl0V1B9Exm6vkh4WcomH v3lq01J1+knOjdVhC5NuSOx2pQr6uLSzSDmoOAW5Ha/582ehgU6QjbSkCobQuGrQGjnG 12WmT3C0O5ugc67+cTXAw1FVyoYf2v2s96yBQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=KxJiEzfvif0aSMkZVWRMjdYYOAocw+7fP03fgKaYh0WQDbgUXcSAagW3xQb174G29f M6ZASNQk3PWYe7W6yyRbVUz9b1AMtBCvM0L+RsuNzYb75Ez5j1/0Kl6Mna5Rmixg9CPk ULCwZUwRi312dV8EXqa2c5riI10Uc5IOaWNTY= MIME-Version: 1.0 Received: by 10.224.190.68 with SMTP id dh4mr2665586qab.256.1302337472216; Sat, 09 Apr 2011 01:24:32 -0700 (PDT) Received: by 10.224.6.78 with HTTP; Sat, 9 Apr 2011 01:24:32 -0700 (PDT) In-Reply-To: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> References: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> Date: Sat, 9 Apr 2011 03:24:32 -0500 Message-ID: From: Zhihao Yuan To: david.marec@davenulle.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Stable@freebsd.org Subject: Re: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 08:24:33 -0000 /usr/src is controlled by root. Use `sudo` to perform the system management related works. On Sat, Apr 9, 2011 at 2:25 AM, David Marec wro= te: > Hi guys. > > > Since the release of FreeBSD 8.2, building world fails on the following > error: > > > <------> > david:/home/david#cd /usr/src > david:/usr/src#make -j4 buildworld && make kernel > -------------------------------------------------------------- >>>> World build started on Sat Apr =C2=A09 09:16:50 CEST 2011 > -------------------------------------------------------------- > -------------------------------------------------------------- >>>> Rebuilding the temporary build tree > -------------------------------------------------------------- > rm -rf /usr/obj/usr/src/tmp > rm -rf /usr/obj/usr/src/lib32 > rm: /usr/obj/usr/src/lib32/usr/lib32/libc.so.7: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/libcrypt.so.5: Operation not permitt= ed > rm: /usr/obj/usr/src/lib32/usr/lib32/libthr.so.3: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/librt.so.1: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32: Directory not empty > rm: /usr/obj/usr/src/lib32/usr: Directory not empty > rm: /usr/obj/usr/src/lib32: Directory not empty > *** Error code 1 > 1 error > *** Error code 2 > 1 error > david:/usr/src#ls -lo /usr/obj/usr/src/lib32/usr/lib32/ > total 1262 > -r--r--r-- =C2=A01 root =C2=A0wheel =C2=A0schg 1143468 22 mar 21:19 libc.= so.7 > -r--r--r-- =C2=A01 root =C2=A0wheel =C2=A0schg =C2=A0 32060 22 mar 21:19 = libcrypt.so.5 > -r--r--r-- =C2=A01 root =C2=A0wheel =C2=A0schg =C2=A0 16412 22 mar 21:22 = librt.so.1 > -r--r--r-- =C2=A01 root =C2=A0wheel =C2=A0schg =C2=A0 76412 22 mar 21:20 = libthr.so.3 > <------> > > Im a running FreeBSD 8.2-Stable for amd64. > > So, before building world, i have to change the flags for the files above= . > There was no need to do this before. > > > Any idea to get rid of this issue ? > > > -- > Cordialement, > -- > David Marec: http://user.lamaiziere.net/david/Site/ > http://www.freebsd.org/fr/ =C2=A0 =C2=A0 http://www.diablotins.org/ > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > --=20 Zhihao Yuan The best way to predict the future is to invent it. From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 09:53:32 2011 Return-Path: Delivered-To: Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F9FA106564A for ; Sat, 9 Apr 2011 09:53:32 +0000 (UTC) (envelope-from amarat@ksu.ru) Received: from mx3.ksu.ru (mx3.ksu.ru [83.69.96.40]) by mx1.freebsd.org (Postfix) with ESMTP id 689B68FC0A for ; Sat, 9 Apr 2011 09:53:30 +0000 (UTC) Received: from unknown (HELO ruby.ksu.ru) ([178.213.240.4]) by ironport1.ksu.ru with ESMTP; 09 Apr 2011 13:24:12 +0400 X-Pass-Through: Kazan State University Network Received: from zealot.ksu.ru ([194.85.245.161]) by ksu.ru (8.13.4/8.13.4) with ESMTP id p399HXsL007336; Sat, 9 Apr 2011 09:17:33 GMT Received: from zealot.ksu.ru (localhost.lnet [127.0.0.1]) by zealot.ksu.ru (8.14.4/8.14.4) with ESMTP id p399LnJW038544; Sat, 9 Apr 2011 13:21:49 +0400 (MSD) (envelope-from amarat@ksu.ru) Message-ID: <4DA0252D.7040804@ksu.ru> Date: Sat, 09 Apr 2011 13:21:49 +0400 From: "Marat N.Afanasyev" User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20101215 Firefox/3.6.15 MIME-Version: 1.0 To: david.marec@davenulle.org References: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> In-Reply-To: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms090903050302060007060802" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Stable@freebsd.org Subject: Re: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 09:53:32 -0000 This is a cryptographically signed message in MIME format. --------------ms090903050302060007060802 Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: quoted-printable David Marec wrote: > Hi guys. > > > Since the release of FreeBSD 8.2, building world fails on the following= > error: > > > <------> > david:/home/david#cd /usr/src > david:/usr/src#make -j4 buildworld&& make kernel > -------------------------------------------------------------- >>>> World build started on Sat Apr 9 09:16:50 CEST 2011 > -------------------------------------------------------------- > -------------------------------------------------------------- >>>> Rebuilding the temporary build tree > -------------------------------------------------------------- > rm -rf /usr/obj/usr/src/tmp > rm -rf /usr/obj/usr/src/lib32 > rm: /usr/obj/usr/src/lib32/usr/lib32/libc.so.7: Operation not permitted= > rm: /usr/obj/usr/src/lib32/usr/lib32/libcrypt.so.5: Operation not permi= tted > rm: /usr/obj/usr/src/lib32/usr/lib32/libthr.so.3: Operation not permitt= ed > rm: /usr/obj/usr/src/lib32/usr/lib32/librt.so.1: Operation not permitte= d > rm: /usr/obj/usr/src/lib32/usr/lib32: Directory not empty > rm: /usr/obj/usr/src/lib32/usr: Directory not empty > rm: /usr/obj/usr/src/lib32: Directory not empty > *** Error code 1 > 1 error > *** Error code 2 > 1 error > david:/usr/src#ls -lo /usr/obj/usr/src/lib32/usr/lib32/ > total 1262 > -r--r--r-- 1 root wheel schg 1143468 22 mar 21:19 libc.so.7 > -r--r--r-- 1 root wheel schg 32060 22 mar 21:19 libcrypt.so.5 > -r--r--r-- 1 root wheel schg 16412 22 mar 21:22 librt.so.1 > -r--r--r-- 1 root wheel schg 76412 22 mar 21:20 libthr.so.3 > <------> > > Im a running FreeBSD 8.2-Stable for amd64. > > So, before building world, i have to change the flags for the files abo= ve. > There was no need to do this before. > > > Any idea to get rid of this issue ? > > man chflags --=20 =F3 =D5=D7=C1=D6=C5=CE=C9=C5=CD, =ED=C1=D2=C1=D4 =E1=C6=C1=CE=C1=D3=D8=C5= =D7 --------------ms090903050302060007060802-- From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 13:40:14 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6116106566C for ; Sat, 9 Apr 2011 13:40:14 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id C42008FC13 for ; Sat, 9 Apr 2011 13:40:12 +0000 (UTC) Received: by pzk27 with SMTP id 27so2029345pzk.13 for ; Sat, 09 Apr 2011 06:40:12 -0700 (PDT) Received: by 10.142.132.3 with SMTP id f3mr2893837wfd.36.1302356371755; Sat, 09 Apr 2011 06:39:31 -0700 (PDT) Received: from dfleuriot.local ([83.167.62.196]) by mx.google.com with ESMTPS id s39sm5161438wfc.4.2011.04.09.06.39.29 (version=SSLv3 cipher=OTHER); Sat, 09 Apr 2011 06:39:31 -0700 (PDT) Message-ID: <4DA06190.90507@my.gd> Date: Sat, 09 Apr 2011 15:39:28 +0200 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> In-Reply-To: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Subject: Re: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 13:40:15 -0000 On 4/9/11 9:25 AM, David Marec wrote: > Hi guys. > > > Since the release of FreeBSD 8.2, building world fails on the following > error: > > > <------> > david:/home/david#cd /usr/src > david:/usr/src#make -j4 buildworld && make kernel > -------------------------------------------------------------- >>>> World build started on Sat Apr 9 09:16:50 CEST 2011 > -------------------------------------------------------------- > -------------------------------------------------------------- >>>> Rebuilding the temporary build tree > -------------------------------------------------------------- > rm -rf /usr/obj/usr/src/tmp > rm -rf /usr/obj/usr/src/lib32 > rm: /usr/obj/usr/src/lib32/usr/lib32/libc.so.7: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/libcrypt.so.5: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/libthr.so.3: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/librt.so.1: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32: Directory not empty > rm: /usr/obj/usr/src/lib32/usr: Directory not empty > rm: /usr/obj/usr/src/lib32: Directory not empty > *** Error code 1 > 1 error > *** Error code 2 > 1 error > david:/usr/src#ls -lo /usr/obj/usr/src/lib32/usr/lib32/ > total 1262 > -r--r--r-- 1 root wheel schg 1143468 22 mar 21:19 libc.so.7 > -r--r--r-- 1 root wheel schg 32060 22 mar 21:19 libcrypt.so.5 > -r--r--r-- 1 root wheel schg 16412 22 mar 21:22 librt.so.1 > -r--r--r-- 1 root wheel schg 76412 22 mar 21:20 libthr.so.3 > <------> > > Im a running FreeBSD 8.2-Stable for amd64. > > So, before building world, i have to change the flags for the files above. > There was no need to do this before. > > > Any idea to get rid of this issue ? > > I experience no such problems on *many* boxes running 8.2 at work here. You will want to: chflags -R noschg /usr/obj/ && cd /usr/src && make -j4 buildworld && make buildkernel From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 13:49:28 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96E791065670 for ; Sat, 9 Apr 2011 13:49:28 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id 7EA618FC13 for ; Sat, 9 Apr 2011 13:49:28 +0000 (UTC) Received: from omta09.emeryville.ca.mail.comcast.net ([76.96.30.20]) by qmta09.emeryville.ca.mail.comcast.net with comcast id VRYg1g0010S2fkCA9RcHGH; Sat, 09 Apr 2011 13:36:17 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta09.emeryville.ca.mail.comcast.net with comcast id VRcG1g00N1t3BNj8VRcGBi; Sat, 09 Apr 2011 13:36:17 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 1EE1F9B422; Sat, 9 Apr 2011 06:36:16 -0700 (PDT) Date: Sat, 9 Apr 2011 06:36:16 -0700 From: Jeremy Chadwick To: David Marec Message-ID: <20110409133616.GA53981@icarus.home.lan> References: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Stable@FreeBSD.org Subject: Re: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 13:49:28 -0000 On Sat, Apr 09, 2011 at 09:25:50AM +0200, David Marec wrote: > rm: /usr/obj/usr/src/lib32/usr/lib32/libc.so.7: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/libcrypt.so.5: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/libthr.so.3: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32/librt.so.1: Operation not permitted > rm: /usr/obj/usr/src/lib32/usr/lib32: Directory not empty > rm: /usr/obj/usr/src/lib32/usr: Directory not empty > rm: /usr/obj/usr/src/lib32: Directory not empty > *** Error code 1 > 1 error > *** Error code 2 > 1 error > david:/usr/src#ls -lo /usr/obj/usr/src/lib32/usr/lib32/ > total 1262 > -r--r--r-- 1 root wheel schg 1143468 22 mar 21:19 libc.so.7 > -r--r--r-- 1 root wheel schg 32060 22 mar 21:19 libcrypt.so.5 > -r--r--r-- 1 root wheel schg 16412 22 mar 21:22 librt.so.1 > -r--r--r-- 1 root wheel schg 76412 22 mar 21:20 libthr.so.3 > > Any idea to get rid of this issue ? "chflags noschg /usr/obj/usr/src/lib32/usr/lib32/*" I would also strongly recommend "rm -fr /usr/obj" after doing the above, just to make sure absolutely nothing is left. I've seen "make clean" not catch things before. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 14:14:59 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3ABC9106566B; Sat, 9 Apr 2011 14:14:59 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8C61A8FC08; Sat, 9 Apr 2011 14:14:58 +0000 (UTC) Received: by bwz12 with SMTP id 12so4471244bwz.13 for ; Sat, 09 Apr 2011 07:14:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=JFNzSi9mdtGcU5AI9BavdJgvxUk8deZ7rHE6YdzmX8o=; b=Vst9iNGlNu1OJGgIdNwMYpG097rJsEU78v01HTxJxaIW43W2fxh0fkL3UqeMuu1A2i g8UVjLRKfZAvoA/CDvBuD1FY8SYTY9jCZzAnPYzcS4mFSOBgJ9J11iF2fgLgr0lAAdSy XsRAyxwNNBAprDZzv4rRXioMmMXNrKWIlT5KY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=YWm7gvRescb0DVuQ5VP3rrE4cXY5C9s/C+l840YXtAS546FBtmbPGQhPXuDjzO9eYC OwKlElMGXrcapUPp4/3hSVfyQJXlvfQ+sMZAmIDDtJ5+XsAa60Mff48KTOSwW74eElH6 t7qU+JRvZwblwqqr+mTBHqdwnlPZViq7nlmMs= Received: by 10.204.151.207 with SMTP id d15mr3192289bkw.123.1302358495539; Sat, 09 Apr 2011 07:14:55 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id q18sm2192247bka.3.2011.04.09.07.14.54 (version=SSLv3 cipher=OTHER); Sat, 09 Apr 2011 07:14:54 -0700 (PDT) Sender: Alexander Motin Message-ID: <4DA069DC.3030306@FreeBSD.org> Date: Sat, 09 Apr 2011 17:14:52 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: Daniel Gerzo References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> <4D9F4B58.3050104@FreeBSD.org> <4DA01168.8050704@FreeBSD.org> In-Reply-To: <4DA01168.8050704@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 14:14:59 -0000 On 09.04.2011 10:57, Daniel Gerzo wrote: > On 8.4.2011 19:52, Alexander Motin wrote: >>> So, here is my attempt to implement it: >>> http://danger.rulez.sk/powerd.diff >>> Can you please review & comment? I should be able to commit it mysqlf if >>> you consider it acceptable. It seems to work for me :) >> >> Looks fine, except that -f option have to be the first, that is not >> obvious. Another moment -- I've noticed some load constants hardcoded >> there. They should also be handled to make higher values to work >> properly. > > I tried to be more explicit in the error message which tries to emphasis > the need to put it first. I don't know myself how it would be possible > to code it so that the -f doesn't need to be first. Ideas? Move checks after the loop? Just an idea. > Do you mean the values around lines of 730 - 762? Yes. When load is more the twice higher then limit - frequency rises faster. To make it work with limit > 50%, there is hardcoded additional check for 95% level. > From what I have observed, if I have a machine that is a little more > loaded (say 300%) and the load goes up, it tries to increases the > performance to quite high freq (5336) and when the load decreases again, > it takes quite a while to go down from 5366 to a frequency that is > actually available to decrease the performance (something less than > 2934). So the lower frequency is used for too short time because it > takes too much time to get it... It is intended behavior in hiadaptive mode, where performance is preferable to power-saving. >>> Seems like it was enabled by default. I have like these: >>> dev.cpu.0.cx_supported: C1/3 C2/96 C3/128 >>> >>> Does that mean I only need to set these in rc.conf?: >>> performance_cx_lowest="C3" >>> economy_cx_lowest="C3" >>> >>> Then run /etc/rc.d/power_profile 0x00? >> >> It short - yes. In long - read the link I've given. >> >>> May it cause any instability? >> >> It you won't switch from LAPIC to other timer and it stop - your system >> will freeze, or at least not work well. You should notice problems >> immediately, if there are. > > So I will also need to change the kern.timecounter.hardware to i8254? I > suppose it will cause a little less precise time, but should I expect > lower performance? I don't care that much about the time accuracy. I wasn't mentioning timecounter there. In terms of 9-CURRENT I was talking about eventtimer. In 8-STABLE it is not formalized yet and so the guide mentions number of tunables. > How do I know the C3 is active? sysctl dev.cpu.X.cx_usage > And how does it switch back to C1 for example? When CPU is idle, depending on previous idle statistics, system puts it into one of reported and allowed C-states. CPU goes back to C0 state on any hardware interrupt. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 14:55:33 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15ECC106566C; Sat, 9 Apr 2011 14:55:33 +0000 (UTC) (envelope-from freebsd@chillt.de) Received: from dd16434.kasserver.com (dd16434.kasserver.com [85.13.137.111]) by mx1.freebsd.org (Postfix) with ESMTP id C935A8FC16; Sat, 9 Apr 2011 14:55:32 +0000 (UTC) Received: from taiko.lan (ppp-17-44.21-151.libero.it [151.21.44.17]) by dd16434.kasserver.com (Postfix) with ESMTPSA id 0FBED188602D; Sat, 9 Apr 2011 16:40:02 +0200 (CEST) Message-ID: <4DA06F92.4070702@chillt.de> Date: Sat, 09 Apr 2011 16:39:14 +0200 From: Bartosz Fabianowski User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Thunderbird/3.1.9 MIME-Version: 1.0 To: Daniel Gerzo References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> In-Reply-To: <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> Content-Type: multipart/mixed; boundary="------------090008050507000001080607" Cc: Alexander Motin , stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 14:55:33 -0000 This is a multi-part message in MIME format. --------------090008050507000001080607 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit I just noticed this thread a day after my own fight with powerd and load percentages that did not seem to make any sense. The patch I came up with is attached. It modifies powerd to use the load percentage of the busiest core. This reduces the range of values back to 0%...100% also for multi-core systems. On my Core i7 setup here, the change seems to work well. - Bartosz --------------090008050507000001080607 Content-Type: text/plain; name="powerd.c.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="powerd.c.diff" LS0tIHBvd2VyZC5jLm9sZAkyMDExLTA0LTA3IDE3OjMwOjU4LjAwMDAwMDAwMCArMDIwMAor KysgcG93ZXJkLmMJMjAxMS0wNC0wNyAxNzozODoyOC4wMDAwMDAwMDAgKzAyMDAKQEAgLTEy OCw3ICsxMjgsNyBAQAogCXN0YXRpYyBsb25nICpjcF90aW1lcyA9IE5VTEwsICpjcF90aW1l c19vbGQgPSBOVUxMOwogCXN0YXRpYyBpbnQgbmNwdXMgPSAwOwogCXNpemVfdCBjcF90aW1l c19sZW47Ci0JaW50IGVycm9yLCBjcHUsIGksIHRvdGFsOworCWludCBlcnJvciwgY3B1LCBp LCB0b3RhbCwgbWF4OwogCiAJaWYgKGNwX3RpbWVzID09IE5VTEwpIHsKIAkJY3BfdGltZXNf bGVuID0gMDsKQEAgLTE1MSw3ICsxNTEsNyBAQAogCQlyZXR1cm4gKGVycm9yKTsKIAkJCiAJ aWYgKGxvYWQpIHsKLQkJKmxvYWQgPSAwOworCQltYXggPSAwOwogCQlmb3IgKGNwdSA9IDA7 IGNwdSA8IG5jcHVzOyBjcHUrKykgewogCQkJdG90YWwgPSAwOwogCQkJZm9yIChpID0gMDsg aSA8IENQVVNUQVRFUzsgaSsrKSB7CkBAIC0xNjAsOSArMTYwLDEyIEBACiAJCQl9CiAJCQlp ZiAodG90YWwgPT0gMCkKIAkJCQljb250aW51ZTsKLQkJCSpsb2FkICs9IDEwMCAtIChjcF90 aW1lc1tjcHUgKiBDUFVTVEFURVMgKyBDUF9JRExFXSAtIAorCQkJdG90YWwgPSAxMDAgLSAo Y3BfdGltZXNbY3B1ICogQ1BVU1RBVEVTICsgQ1BfSURMRV0gLSAKIAkJCSAgICBjcF90aW1l c19vbGRbY3B1ICogQ1BVU1RBVEVTICsgQ1BfSURMRV0pICogMTAwIC8gdG90YWw7CisJCQlp ZiAodG90YWwgPiBtYXgpCisJCQkJbWF4ID0gdG90YWw7CiAJCX0KKwkJKmxvYWQgPSBtYXg7 CiAJfQogCiAJbWVtY3B5KGNwX3RpbWVzX29sZCwgY3BfdGltZXMsIGNwX3RpbWVzX2xlbik7 Cg== --------------090008050507000001080607-- From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 15:06:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44C08106566B for ; Sat, 9 Apr 2011 15:06:33 +0000 (UTC) (envelope-from taro@mouri.dnsalias.com) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id 218AA8FC1B for ; Sat, 9 Apr 2011 15:06:32 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.69) (envelope-from ) id 1Q8ZV1-0000GQ-Pk for freebsd-stable@freebsd.org; Sat, 09 Apr 2011 07:50:55 -0700 Date: Sat, 9 Apr 2011 07:50:55 -0700 (PDT) From: Tatsudin To: freebsd-stable@freebsd.org Message-ID: <1302360655778-4293015.post@n5.nabble.com> In-Reply-To: <20110214003704.GA2049@comcast.net> References: <20110214003704.GA2049@comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: 8.2-PRERELEASE generating warnings re my hard drive X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 15:06:33 -0000 Hello. I got the same error message on 8.2-PRERELEASE and 9.0-CURRENT, after kernel source version-up and compilation of it, by Intel D510MO motherboard. After changing BIOS setting of HDD operation mode from 'IDE' to 'AHCI', the error seemed to have gone. Please try it. Sincerely, Taro Mohri -- View this message in context: http://freebsd.1045724.n5.nabble.com/8-2-PRERELEASE-generating-warnings-re-my-hard-drive-tp3949272p4293015.html Sent from the freebsd-stable mailing list archive at Nabble.com. From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 15:29:28 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D292106566C; Sat, 9 Apr 2011 15:29:28 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9DB378FC0A; Sat, 9 Apr 2011 15:29:27 +0000 (UTC) Received: by bwz12 with SMTP id 12so4510866bwz.13 for ; Sat, 09 Apr 2011 08:29:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=Z+hK0rlVEGjbibzGp/fwaHpLFpw9rm9ffIgvlRU3an4=; b=UzxmFN8pu/ebeJU4p5rCdLO6XixopzMgMMO3DUdh6+bKWodIsQknVKWJQRSmM1gKXk h4v252FCMQDM/6/A6U/v98Fgaa9Ws1CTre2iOmatIJTrKfbid4YsXYGURxYWxKp0r7K1 4MoqtAcPkbYDRRzoTUllcajrnTXAgGvzU/PuE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=WdDGfiLKe6J8rOTzQBpuzq8gzNVIhDzU7XkUv3+9LSB65pjZLyZvoFiMR0ktXm7zBr Jvy6/vn5Xdg7Pnelkwni5eI8g/HEp5yByJuhqPVSGywpYTTBvbEpEFCZ7dS8nzEGo3uK o2I0z3eXwgRCFS2FtxQMLfzDZoluCNZlORHo0= Received: by 10.204.22.197 with SMTP id o5mr3089277bkb.68.1302362966425; Sat, 09 Apr 2011 08:29:26 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id x6sm2218673bkv.12.2011.04.09.08.29.24 (version=SSLv3 cipher=OTHER); Sat, 09 Apr 2011 08:29:25 -0700 (PDT) Sender: Alexander Motin Message-ID: <4DA07B53.2090803@FreeBSD.org> Date: Sat, 09 Apr 2011 18:29:23 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110310 Thunderbird/3.1.9 MIME-Version: 1.0 To: Bartosz Fabianowski References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> <4DA06F92.4070702@chillt.de> In-Reply-To: <4DA06F92.4070702@chillt.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Daniel Gerzo , stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 15:29:28 -0000 On 09.04.2011 17:39, Bartosz Fabianowski wrote: > I just noticed this thread a day after my own fight with powerd and load > percentages that did not seem to make any sense. > > The patch I came up with is attached. It modifies powerd to use the load > percentage of the busiest core. This reduces the range of values back to > 0%...100% also for multi-core systems. While using maximum of loads can be better then using levels above 100%, it won't properly handle cases of dependent or frequently migrating threads, that are handled now with summary load and levels less then 100%. While existing powerd algorithm is indeed not perfect, it is the only relatively performance-safe, unlike others propositions. I won't argue about adding more algorithms/options to powerd, optimized for handling different situations, but I believe that default should remain safe. > On my Core i7 setup here, the change seems to work well. ... in your specific workload. And you haven't described how you measured system performance to prove that it haven't decreased. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 16:20:07 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9E31106566B; Sat, 9 Apr 2011 16:20:07 +0000 (UTC) (envelope-from freebsd@chillt.de) Received: from dd16434.kasserver.com (dd16434.kasserver.com [85.13.137.111]) by mx1.freebsd.org (Postfix) with ESMTP id 8161F8FC16; Sat, 9 Apr 2011 16:20:06 +0000 (UTC) Received: from taiko.lan (ppp-17-44.21-151.libero.it [151.21.44.17]) by dd16434.kasserver.com (Postfix) with ESMTPSA id 09ABA188602D; Sat, 9 Apr 2011 18:20:05 +0200 (CEST) Message-ID: <4DA08705.9050507@chillt.de> Date: Sat, 09 Apr 2011 18:19:17 +0200 From: Bartosz Fabianowski User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110309 Thunderbird/3.1.9 MIME-Version: 1.0 To: Alexander Motin References: <4D9EEDAF.3020803@rulez.sk> <4D9EF48C.9070907@FreeBSD.org> <4D9F2384.5000104@FreeBSD.org> <85cda6f83d328e67a552b2cd5758dbd3@rulez.sk> <4DA06F92.4070702@chillt.de> <4DA07B53.2090803@FreeBSD.org> In-Reply-To: <4DA07B53.2090803@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Daniel Gerzo , stable@FreeBSD.org Subject: Re: powerd / cpufreq question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 16:20:08 -0000 >> On my Core i7 setup here, the change seems to work well. > > ... in your specific workload. And you haven't described how you > measured system performance to prove that it haven't decreased. My measure of "performance" is entirely unscientific: This is a desktop box. Performance is good if KDE reacts to inputs quickly. My patch preserves this for me while making the box run a bit cooler. I am by no means advocating that my patch be made the default behavior. But as you said, it may be nice to include it as one of several algorithms the user can choose from. - Bartosz From owner-freebsd-stable@FreeBSD.ORG Sat Apr 9 17:27:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C51A41065672 for ; Sat, 9 Apr 2011 17:27:52 +0000 (UTC) (envelope-from david.marec@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [91.121.44.19]) by mx1.freebsd.org (Postfix) with ESMTP id 8EBDC8FC08 for ; Sat, 9 Apr 2011 17:27:52 +0000 (UTC) Received: from lamaiziere.net (unknown [192.168.1.1]) by smtp.lamaiziere.net (Postfix) with ESMTP id CFCC9633205 for ; Sat, 9 Apr 2011 19:12:02 +0200 (CEST) Received: from 93.28.96.112 (SquirrelMail authenticated user david.nul) by lamaiziere.net with HTTP; Sat, 9 Apr 2011 19:12:02 +0200 (CEST) Message-ID: <54103ac1c1ae9a9459a5256cea2ac975.squirrel@lamaiziere.net> In-Reply-To: <4DA06190.90507@my.gd> References: <54fbb6b0cf9c063b3e3d58d7e2d1d28e.squirrel@lamaiziere.net> <4DA06190.90507@my.gd> Date: Sat, 9 Apr 2011 19:12:02 +0200 (CEST) From: "David Marec" To: freebsd-stable@freebsd.org User-Agent: SquirrelMail/1.4.17 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-15 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: Re: Cleaning temporary build tree failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: david.marec@davenulle.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 17:27:52 -0000 Damien Fleuriot a écrit : > I experience no such problems on *many* boxes running 8.2 at work here. > > You will want to: > > chflags -R noschg /usr/obj/ && cd /usr/src && make -j4 buildworld && > make buildkernel You are right, that 's what i have to do *each time* i want to proceed to a "rebuild world". Therefore, the issue is why; moreover, why since the 8.2 release ? Is there any clue to avoid to change these flags, previously to a 'rebuild world' -- Cordialement, -- David Marec: http://user.lamaiziere.net/david/Site/ http://www.freebsd.org/fr/ http://www.diablotins.org/