From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 07:56:35 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E37B816A417; Sun, 6 Jan 2008 07:56:35 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id 68C3413C468; Sun, 6 Jan 2008 07:56:35 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1JBQMz-000CNu-6o; Sun, 06 Jan 2008 09:56:33 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: freebsd-current@freebsd.org In-reply-to: References: <477C82F0.5060809@freebsd.org> <863ateemw2.fsf@ds4.des.no> <200801032200.25650.peter.schuller@infidyne.com> <8663yac62d.fsf@ds4.des.no> <477E72FC.5070304@freebsd.org> <477EA466.6060204@FreeBSD.org> <477EFEAB.8090807@freebsd.org> Comments: In-reply-to "Peter Wemm" message dated "Sat, 05 Jan 2008 14:24:30 -0800." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 06 Jan 2008 09:56:32 +0200 From: Danny Braniss Message-ID: Cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8?=@freebsd.org, Tim Kientzle , Peter Schuller , Jason Evans , rgrav Subject: Re: ELF dynamic loader name [was: sbrk(2) broken] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 07:56:36 -0000 > > > Peter Wemm wrote: > > > > > On the other hand, if ld-elf.so.1 is fairly unique in this > > > > > concern, it might be simpler to rename it to: > > > > > ld-elf-{i386,amd64,ppc,...}.so.1 > > > > > > > > While this doesn't count as an explicit vote against the rename, we > > can > > > > solve the chroot problem easily. > > > > > > Details? Does your approach also solve the problem of > > > sharing /usr across different architectures (either in > > > a diskless NFS environment or a dual-boot scenario with > > > a shared /usr partition)? > > > > > > > However, renaming ld-elf.so.1 is a bad idea in general. ... things > > like gdb > > > > "know" how to handle ld-elf.so.1. Getting those upstream folks to add > > > > additional strcmp()'s for ld-elf-i386.so.1, ld-elf-amd64.so.1 etc will > > > > be hard enough, and it will add another hurdle ... > > > > > > I'm not sure that I see the problem. What am I missing? > > > 1) gdb is built to debug binaries for a particular architecture. > > > (gdb/ARM can't debug gdb/i386 binaries) > > > 2) gdb therefore only needs to check for "ld-elf-"`uname -m`".so.1", > > > which is easy to handle when gdb itself is built. > > > > > > I can see some subtleties for cross-builds, but nothing > > > outrageous. > > > > > > It also seems that your argument applies just as well to > > > ld-elf.so.1 and ld-elf32.so.1. Either way, there's more > > > than one ld-elf.so.1, and therefore more than one name > > > to keep track of. > > > > > > I'm not championing the rename by any means, just trying > > > to better understand the issues. The fact that amd64 can > > > run i386 binaries but not vice-versa has a lot of subtle > > > implications. Also, this is the first time that FreeBSD > > > has really had large user bases on two fundamentally > > > different architectures, so it's the first time we've > > > really had to confront some of these support issues > > > (such as the shared /usr scenario). > > > > > > Tim Kientzle > > > > The main issue is NOT sharing / or /usr or /usr/local, that is peenuts. > > root and usr is less that 500 MGB, /usr/local though big, is handled > > neatly by amd (the automounter). > > cross building is one issue, but the real problem is sharing user's > > binaries. > > in Apple one can compile a binary for both i386 & ppc, and the binary is > > twice as big. side note, I compiled such a program, but by mistake chose > > two different binaries to be joined, and imagine my surprice when it acted > > differently from expected. > > We have come a long way since the days that a wrong architecture a.outwould > > just coredump. > > In the old days, we had ~/bin/$arch in our path to keep different > > binaries, it was the days of VAX/Sun, but since i386 arrived, this has > > been > > forgotten. Now we are concidering to deploy amd64, and it would be nice > > if it can be a 2way street - amd64 can run i386, but i386 should run the > > i386 > > version ... > > > > just blaberring before coffee. > > danny > > > It isn't very hard to do this at all. I did it as a proof-of-concept a few > months ago: > > peter@overcee[2:18pm]/tmp/demo-218> cat foo.c > #include > > main() > { > #ifdef __i386__ > printf("Platform = i386\n"); > #endif > #ifdef __amd64__ > printf("Platform = amd64\n"); > #endif > } > peter@overcee[2:18pm]/tmp/demo-219> ./foo_i386 > Platform = i386 > peter@overcee[2:19pm]/tmp/demo-220> ./foo_amd64 > Platform = amd64 > peter@overcee[2:19pm]/tmp/demo-221> cat foo.c > #include > > main() > { > #ifdef __i386__ > printf("Platform = i386\n"); > #endif > #ifdef __amd64__ > printf("Platform = amd64\n"); > #endif > } > peter@overcee[2:19pm]/tmp/demo-222> which cc > /usr/bin/cc > peter@overcee[2:19pm]/tmp/demo-223> cc -o foo_amd64 foo.c > peter@overcee[2:19pm]/tmp/demo-224> cc -m32 -o foo_i386 foo.c > peter@overcee[2:19pm]/tmp/demo-225> file foo_* > foo_amd64: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), for > FreeBSD 8.0 (800006), dynamically linked (uses shared libs), FreeBSD-style, > not stripped > foo_i386: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), for > FreeBSD 8.0 (800006), dynamically linked (uses shared libs), FreeBSD-style, > not stripped > peter@overcee[2:19pm]/tmp/demo-226> ./foo_i386 > Platform = i386 > peter@overcee[2:19pm]/tmp/demo-227> ./foo_amd64 > Platform = amd64 > peter@overcee[2:19pm]/tmp/demo-228> uname -m > amd64 > > What I did was a half-dozen lines of a hack to our bmake glue for gcc. It > is a hack though because I did it as specs overrides rather than have it > figure the correct #include paths. This means my version doesn't interact > with -nostdinc mode correctly. Doing it to correctly handle the paths isn't > much harder. > > -Peter what Apple has is one file, that will run the appropiate binary if run on an i386 or a ppc, not 2 different files - universal binary - not rosetta. [macbook:system/danny/tmp] danny% uname -p i386 [macbook:system/danny/tmp] danny% gcc -arch i386 foo.c -o foo_i386 [macbook:system/danny/tmp] danny% gcc -arch ppc foo.c -o foo_ppc [macbook:system/danny/tmp] danny% lipo -create -arch ppc foo_ppc -arch i386 foo_i386 -output foo [macbook:system/danny/tmp] danny% file foo foo: Mach-O universal binary with 2 architectures foo (for architecture ppc7400): Mach-O executable ppc foo (for architecture i386): Mach-O executable i386 [macbook:system/danny/tmp] danny% ./foo Platform = i386 [macbook:system/danny/tmp] danny% ls -lsi foo 17768042 57 -rwxr-xr-x 1 danny wheel 28972 Jan 6 09:32 foo =================================== twister> uname -p powerpc twister> ./foo Platform = ppc twister> ls -lsi foo 17768042 57 -rwxr-xr-x 1 danny wheel 28972 Jan 6 09:32 foo danny From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 09:51:23 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8432816A41B; Sun, 6 Jan 2008 09:51:23 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from smtp.infidyne.com (ds9.infidyne.com [88.80.6.206]) by mx1.freebsd.org (Postfix) with ESMTP id 1EB2713C458; Sun, 6 Jan 2008 09:51:23 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from c-8216e555.03-51-73746f3.cust.bredbandsbolaget.se (c-8216e555.03-51-73746f3.cust.bredbandsbolaget.se [85.229.22.130]) by smtp.infidyne.com (Postfix) with ESMTP id 5583077485; Sun, 6 Jan 2008 10:51:21 +0100 (CET) From: Peter Schuller To: freebsd-current@freebsd.org Date: Sun, 6 Jan 2008 10:51:18 +0100 User-Agent: KMail/1.9.7 References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> In-Reply-To: <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart13184023.iyG8TNZ5Up"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200801061051.26817.peter.schuller@infidyne.com> Cc: Brooks Davis , Ivan Voras Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 09:51:23 -0000 --nextPart13184023.iyG8TNZ5Up Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline > This number is not so large. It seems to be easily crashed by rsync, > for example (speaking from my own experience, and also some of my > colleagues). I can definitely say this is not *generally* true, as I do a lot of=20 rsyncing/rdiff-backup:ing and similar stuff (with many files / large files)= =20 on ZFS without any stability issues. Problems for me have been limited to=20 32bit and the memory exhaustion issue rather than "hard" issues. But perhaps that's all you are referring to. =2D-=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --nextPart13184023.iyG8TNZ5Up Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHgKSeDNor2+l1i30RAjynAKCl+ehI7a8/xoTBjc9Z5DTcR58obgCcDZF+ qvGLR/LxRjX47PfCCPo29r8= =Pjj5 -----END PGP SIGNATURE----- --nextPart13184023.iyG8TNZ5Up-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 10:04:38 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AC5616A417 for ; Sun, 6 Jan 2008 10:04:38 +0000 (UTC) (envelope-from dnelen@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.190]) by mx1.freebsd.org (Postfix) with ESMTP id 2A87D13C458 for ; Sun, 6 Jan 2008 10:04:37 +0000 (UTC) (envelope-from dnelen@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so7258144rvb.43 for ; Sun, 06 Jan 2008 02:04:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=Vd0oI6I2A/dvy6OXprQ8JE9LBlh761bza/yKyl1Eol0=; b=Y76UsWZP6OObEfL8QOH9OrxHCVQuwGxBwckG16qlTzIl4qKeY63wX+pPD0WK0c2gAHoawcJZKAb700ZqtJXstN2LPFUmKsEzoIY1WkP74fXjX0qMn9eacs3AqqiB6rRbW7gZ5UxC5gc0ARPQ1n5mSdNsinP13QvjyYdKlOaCE1I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=evGqxbCwCTXdQgPYPBu1QxsScT2gc7RGgaY2fCWDCBiHoBYipPsY/xjb7cYvmkNRKwqSY9hbsrqpwyyU15kenhML9KKnhrq8k59U0ImnVDJKFinr1WzW/129vNnEJOqbSmconu59YgZhXXJhiwjFDgeWrbCedN2QoTO4QmnRZY8= Received: by 10.141.49.6 with SMTP id b6mr9882227rvk.18.1199612391376; Sun, 06 Jan 2008 01:39:51 -0800 (PST) Received: by 10.141.75.2 with HTTP; Sun, 6 Jan 2008 01:39:51 -0800 (PST) Message-ID: <319b588b0801060139m5e99e857of3a8f44ab3d2e078@mail.gmail.com> Date: Sun, 6 Jan 2008 11:39:51 +0200 From: "Dima Nelen" To: freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: bad packet length error X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 10:04:38 -0000 hi, i found strange problem with working ssh on server: gr login failures: Jan 4 08:21:57 gr sshd[16218]: Received disconnect from 193.125.78.113: 2: Bad packet length 1626711409. Jan 4 08:35:37 gr sshd[16254]: Received disconnect from 77.123.194.155: 2: Bad packet length 236618258. what is it and what should i fix? FreeBSD gr.holydns.com 7.0-BETA4 FreeBSD 7.0-BETA4 #0: Wed Dec 5 01:12:18 UTC 2007 dexx@gr:/usr/obj/usr/src/sys/GR amd64 -- *#) From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 12:12:39 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFBDA16A417 for ; Sun, 6 Jan 2008 12:12:39 +0000 (UTC) (envelope-from darranc@deejc.net) Received: from smtp1.freeserve.com (smtp1.wanadoo.co.uk [193.252.22.158]) by mx1.freebsd.org (Postfix) with ESMTP id 6AA8013C447 for ; Sun, 6 Jan 2008 12:12:39 +0000 (UTC) (envelope-from darranc@deejc.net) Received: from smtp1.freeserve.com (mwinf3016 [172.22.159.44]) by mwinf3005.me.freeserve.com (SMTP Server) with ESMTP id CDA1C1002709 for ; Sun, 6 Jan 2008 12:43:26 +0100 (CET) Received: from me-wanadoo.net (localhost [127.0.0.1]) by mwinf3016.me.freeserve.com (SMTP Server) with ESMTP id 4A948C400087 for ; Sun, 6 Jan 2008 12:43:25 +0100 (CET) Received: from Vostro (unknown [91.104.113.116]) by mwinf3016.me.freeserve.com (SMTP Server) with ESMTP id 09501C400086 for ; Sun, 6 Jan 2008 12:43:25 +0100 (CET) X-ME-UUID: 20080106114325382.09501C400086@mwinf3016.me.freeserve.com From: "Darran" To: References: <319b588b0801060139m5e99e857of3a8f44ab3d2e078@mail.gmail.com> Date: Sun, 6 Jan 2008 11:43:13 -0000 Message-ID: <003401c85059$581790f0$6501a8c0@Vostro> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <319b588b0801060139m5e99e857of3a8f44ab3d2e078@mail.gmail.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Thread-Index: AchQS5FE0OH2L8U9TgSHXsBOX82HJgACrfpQ Subject: Hung laptop now will not boot 7.0-RC1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 12:12:39 -0000 Hello all, I downloaded, burnt and installed 7.0-RC1 (all torrent) on amd64 I downloaded the dell windows drivers for my laptop wireless card (which worked on i3686 6.2-RELEASE) and used ndisgen to create the .ko file (bcmwl5_sys.ko) and loaded it and then realised I had created the 32 bit driver so I re-did it and used the 64 bit file and loaded the 64bit .ko and at that point my laptop (Dell Vostro 1000) hung. I had put bcmwl5_sys_load="YES" in the loader.conf file I rebooted and now I don't get anything. As soon as it starts to boot it stops. There is a screen showing kernel dump info and I can copy it if needed but I have to burn the rescue iso otherwise I cant boot. Is this a known problem ? Does anyone have any advice ? Darran http://www.deejc.net From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 12:58:24 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC36716A41B for ; Sun, 6 Jan 2008 12:58:24 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.185]) by mx1.freebsd.org (Postfix) with ESMTP id 79BC213C469 for ; Sun, 6 Jan 2008 12:58:24 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so7287273rvb.43 for ; Sun, 06 Jan 2008 04:58:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=aHqARD3BQ6S3hAnjXMcnbml2SkD2H4MHRER01Bl5uhQ=; b=FzOl6OBfqT+zUC0hVHkIF+9FJG4E253vWqLDHID8TIvnSnig2z0ZjpoAoT01wxRyB6y1plDqfplHIGdEvcMEarHUaqrOMQYsCMIDgPdX7/Q4WKK9MIi5mZ8Mo+mh8fyemRH4NXKcY1rLBYrlsAC4x3qdbTJwVOwMDU75TgW+VO8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=YIeZadMJX4+4YBfSWd9+rWfKEakyGFNR8/mVG7V/vP/yuql5gtXMPApGc26pOoSoc1COX/mGCYHiUJ8v2RltdlbHHOcWIaoLly19rx2fdK3b7w843iX5ezRY3fs+OapndOqePImJsJLTGHFCd5oFUsIQSv56DCXlr4vUnRcOgC8= Received: by 10.141.15.19 with SMTP id s19mr9939755rvi.39.1199624303577; Sun, 06 Jan 2008 04:58:23 -0800 (PST) Received: by 10.141.212.1 with HTTP; Sun, 6 Jan 2008 04:58:23 -0800 (PST) Message-ID: <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> Date: Sun, 6 Jan 2008 13:58:23 +0100 From: "Ivan Voras" Sender: ivoras@gmail.com To: "Peter Schuller" In-Reply-To: <200801061051.26817.peter.schuller@infidyne.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> X-Google-Sender-Auth: 271adb9d4d98f96c Cc: freebsd-current@freebsd.org, Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 12:58:24 -0000 On 06/01/2008, Peter Schuller wrote: > > This number is not so large. It seems to be easily crashed by rsync, > > for example (speaking from my own experience, and also some of my > > colleagues). > > I can definitely say this is not *generally* true, as I do a lot of > rsyncing/rdiff-backup:ing and similar stuff (with many files / large files) > on ZFS without any stability issues. Problems for me have been limited to > 32bit and the memory exhaustion issue rather than "hard" issues. It's not generally true since kmem problems with rsync are often hard to repeat - I have them on one machine, but not on another, similar machine. This nonrepeatability is also a part of the problem. > But perhaps that's all you are referring to. Mostly. I did have a ZFS crash with rsync that wasn't kmem related, but only once. From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 13:07:24 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCA2616A419; Sun, 6 Jan 2008 13:07:24 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F03DC13C44B; Sun, 6 Jan 2008 13:07:22 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4780D289.7020509@FreeBSD.org> Date: Sun, 06 Jan 2008 14:07:21 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> In-Reply-To: <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Peter Schuller , Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 13:07:24 -0000 Ivan Voras wrote: > On 06/01/2008, Peter Schuller wrote: >>> This number is not so large. It seems to be easily crashed by rsync, >>> for example (speaking from my own experience, and also some of my >>> colleagues). >> I can definitely say this is not *generally* true, as I do a lot of >> rsyncing/rdiff-backup:ing and similar stuff (with many files / large files) >> on ZFS without any stability issues. Problems for me have been limited to >> 32bit and the memory exhaustion issue rather than "hard" issues. > > It's not generally true since kmem problems with rsync are often hard > to repeat - I have them on one machine, but not on another, similar > machine. This nonrepeatability is also a part of the problem. > >> But perhaps that's all you are referring to. > > Mostly. I did have a ZFS crash with rsync that wasn't kmem related, > but only once. kmem problems are just tuning. They are not indicative of stability problems in ZFS. Please report any further non-kmem panics you experience. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 13:51:03 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0431816A419 for ; Sun, 6 Jan 2008 13:51:03 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 7D03A13C447 for ; Sun, 6 Jan 2008 13:51:02 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBVtu-0006ih-LM for freebsd-current@freebsd.org; Sun, 06 Jan 2008 13:50:54 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 13:50:54 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 13:50:54 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 14:50:47 +0100 Lines: 47 Message-ID: References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCE1F2BBCDE2C97088BADBDD9" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <4780D289.7020509@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 13:51:03 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCE1F2BBCDE2C97088BADBDD9 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > kmem problems are just tuning. They are not indicative of stability=20 > problems in ZFS. =20 I disagree - anything that causes a panic is a stability problem. Panics = persist AFTER the tunings (for i386 certainly, and there are unsolved=20 reports about it on amd64 also) and are present even when driving kmem=20 size to the maximum. The tunings *can not solve the problems* currently, = they can only delay the time until they appear, which, by Murphy, often=20 means "sometime around midnight at Saturday". See also the possibility=20 of deadlocks in the ZIL, reported by some users. > Please report any further non-kmem panics you experience. I did, once to Pawel and once to the lists. Pawel couldn't help me and=20 nobody responded on the lists. Can you perform a MySQL read-write=20 benchmark on one of the 8-core machines with database on ZFS for about=20 an hour without pause? On a machine with 2 GB (or less) of RAM,=20 preferrably? I've seen problems on i386 but maybe they are also present=20 on amd64. --------------enigCE1F2BBCDE2C97088BADBDD9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgNy3ldnAQVacBcgRAqzHAKDfZWQg5+0b7chMA8z3yclmReYs6gCeJ3ir GBVQIzcpbjgFk9JfyTrb1m0= =1aOH -----END PGP SIGNATURE----- --------------enigCE1F2BBCDE2C97088BADBDD9-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 14:13:17 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE5E716A417 for ; Sun, 6 Jan 2008 14:13:17 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: from 42.mail-out.ovh.net (42.mail-out.ovh.net [213.251.189.42]) by mx1.freebsd.org (Postfix) with SMTP id 6531913C46E for ; Sun, 6 Jan 2008 14:13:17 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: (qmail 10693 invoked by uid 503); 6 Jan 2008 13:46:42 -0000 Received: from gw2.ovh.net (HELO mail246.ha.ovh.net) (213.251.189.202) by 42.mail-out.ovh.net with SMTP; 6 Jan 2008 13:46:42 -0000 Received: from b0.ovh.net (HELO queue-out) (213.186.33.50) by b0.ovh.net with SMTP; 6 Jan 2008 13:46:10 -0000 Received: from 217-153-241-141.zab.nat.hnet.pl (HELO arsenic) (maciej@suszko.eu@217.153.241.141) by ns0.ovh.net with SMTP; 6 Jan 2008 13:46:08 -0000 Date: Sun, 6 Jan 2008 14:46:27 +0100 From: Maciej Suszko To: freebsd-current@freebsd.org Message-Id: <20080106144627.a91a62c1.maciej@suszko.eu> In-Reply-To: <4780D289.7020509@FreeBSD.org> References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> X-Mailer: Sylpheed 2.4.8 (GTK+ 2.12.3; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Ovh-Remote: 217.153.241.141 (217-153-241-141.zab.nat.hnet.pl) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-Spam-Check: DONE|H 0.505683/N Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 14:13:18 -0000 Kris Kennaway wrote: > Ivan Voras wrote: > > On 06/01/2008, Peter Schuller wrote: > >>> This number is not so large. It seems to be easily crashed by > >>> rsync, for example (speaking from my own experience, and also > >>> some of my colleagues). > >> I can definitely say this is not *generally* true, as I do a lot of > >> rsyncing/rdiff-backup:ing and similar stuff (with many files / > >> large files) on ZFS without any stability issues. Problems for me > >> have been limited to 32bit and the memory exhaustion issue rather > >> than "hard" issues. > > > > It's not generally true since kmem problems with rsync are often > > hard to repeat - I have them on one machine, but not on another, > > similar machine. This nonrepeatability is also a part of the > > problem. > > > >> But perhaps that's all you are referring to. > > > > Mostly. I did have a ZFS crash with rsync that wasn't kmem related, > > but only once. > > kmem problems are just tuning. They are not indicative of stability > problems in ZFS. Please report any further non-kmem panics you > experience. I agree that ZFS is pretty stable itself. I use 32bit machine with 2gigs od RAM and all hang cases are kmem related, but the fact is that I haven't found any way of tuning to stop it crashing. When I do some rsyncing, especially beetwen different pools - it hangs or reboots - mostly on bigger files (i.e. rsyncing ports tree with distfiles). At the moment I patched the kernel with vm_kern.c.2.patch and it just stopped crashing, but from time to time the machine looks like beeing freezed for a second or two, after that it works normally. Have you got any similar experience? -- regards, Maciej Suszko. From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 14:27:20 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7695B16A418; Sun, 6 Jan 2008 14:27:20 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C8BE013C44B; Sun, 6 Jan 2008 14:27:19 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4780E546.9050303@FreeBSD.org> Date: Sun, 06 Jan 2008 15:27:18 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 14:27:20 -0000 Ivan Voras wrote: > Kris Kennaway wrote: > >> kmem problems are just tuning. They are not indicative of stability >> problems in ZFS. > > I disagree - anything that causes a panic is a stability problem. Panics > persist AFTER the tunings (for i386 certainly, and there are unsolved > reports about it on amd64 also) and are present even when driving kmem > size to the maximum. The tunings *can not solve the problems* currently, > they can only delay the time until they appear, which, by Murphy, often > means "sometime around midnight at Saturday". That's an assertion directly contradicted by my experience running a heavily loaded 8-core i386 package builder. Please explain in detail the steps you have taken to tune your kernel. Do you have the vm_kern.c patch applied? > See also the possibility > of deadlocks in the ZIL, reported by some users. Yes, this is an outstanding issue. There are a couple of others I run into in the above configuration, but kmem panics aren't among them. >> Please report any further non-kmem panics you experience. > > I did, once to Pawel and once to the lists. Pawel couldn't help me and > nobody responded on the lists. Can you perform a MySQL read-write > benchmark on one of the 8-core machines with database on ZFS for about > an hour without pause? On a machine with 2 GB (or less) of RAM, > preferrably? I've seen problems on i386 but maybe they are also present > on amd64. I am not set up to test this right now. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 14:51:06 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 953ED16A469 for ; Sun, 6 Jan 2008 14:51:06 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.188]) by mx1.freebsd.org (Postfix) with ESMTP id 682EA13C45A for ; Sun, 6 Jan 2008 14:51:06 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so7312222rvb.43 for ; Sun, 06 Jan 2008 06:51:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=mYFTaSd9TitXM0YfAOjSauk3dDQx7TBh1aPM+PGTKFs=; b=F5HjgJTECqDKlZk49YLlLQkiEz791+cL+VLmkUBMffwL6edNwQVaVSgDU1aq2mytw3RXMu7GOfniea7aNvdc99z3jP8q5UGmxERpMomA0QopKAjTLNExNC6osuEKLVsFiXe71tzRDXEkmk6YA2awhU9/Vf4IB+2w7DGRiOXbP8E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=n1saDmBCS3dNJ0FvI6KMPKImoZx1tJUbBBke91vc7VTBksblXyVxhDd2CoVPLpS32PY6tCzIB2NbiZ025IYAxCKmtWM4FeqRKodbch+WUOXnvPmXsiK/6dRStHMp4TO+l8gjJrN3ZnTJHtO44yDb3RdvvavHUtuhujh1QD9Imyw= Received: by 10.141.50.17 with SMTP id c17mr9974330rvk.295.1199631066007; Sun, 06 Jan 2008 06:51:06 -0800 (PST) Received: by 10.141.212.1 with HTTP; Sun, 6 Jan 2008 06:51:05 -0800 (PST) Message-ID: <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> Date: Sun, 6 Jan 2008 15:51:05 +0100 From: "Ivan Voras" Sender: ivoras@gmail.com To: "Kris Kennaway" In-Reply-To: <4780E546.9050303@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> X-Google-Sender-Auth: 4c88e74dcc71c0d1 Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 14:51:06 -0000 On 06/01/2008, Kris Kennaway wrote: > That's an assertion directly contradicted by my experience running a > heavily loaded 8-core i386 package builder. What is the IO profile of this usage? I'd guess that it's "short bursts of high activity (archive extraction, installing) followed by long periods of low activity (compiling)". From what I see on the lists and somewhat from my own experience, the problem appears more often when the load is more like "constant high r+w activity", probably with several users (applications) doing the activity in parallel. > Please explain in detail > the steps you have taken to tune your kernel. vm.kmem_size="512M" vm.kmem_size_max="512M" This should be enough for a 2 GB machine that does other things. > Do you have the vm_kern.c > patch applied? I can confirm that while it delays the panics, it doesn't eliminate them (this also seems to be the conclusion of several users that have tested it shortly after it's been posted). The fact that it's not committed is good enough indication that it's not The Answer. (And besides, asking users to apply non-committed patches just to run their systems normally is bad practice :) I can just imagine the Release Notes: "if you're using ZFS, you'll have to manually patch the kernel with this patch:..." :) This close to the -RELEASE, I judge the chances of it being committed are low). From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 15:08:59 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE82B16A420; Sun, 6 Jan 2008 15:08:59 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0CB8C13C46B; Sun, 6 Jan 2008 15:08:58 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4780EF09.4090908@FreeBSD.org> Date: Sun, 06 Jan 2008 16:08:57 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> In-Reply-To: <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 15:08:59 -0000 Ivan Voras wrote: > On 06/01/2008, Kris Kennaway wrote: > >> That's an assertion directly contradicted by my experience running a >> heavily loaded 8-core i386 package builder. > > What is the IO profile of this usage? I'd guess that it's "short > bursts of high activity (archive extraction, installing) followed by > long periods of low activity (compiling)". From what I see on the > lists and somewhat from my own experience, the problem appears more > often when the load is more like "constant high r+w activity", > probably with several users (applications) doing the activity in > parallel. This is a high I/O environment including lots of parallel activity. >> Please explain in detail >> the steps you have taken to tune your kernel. > > vm.kmem_size="512M" > vm.kmem_size_max="512M" > > This should be enough for a 2 GB machine that does other things. No, clearly it is not enough (and you claimed previously to have done more tuning than this). I have it set to 600MB on the i386 system with a 1.5GB KVA. Both were necessary. >> Do you have the vm_kern.c >> patch applied? > > I can confirm that while it delays the panics, it doesn't eliminate > them (this also seems to be the conclusion of several users that have > tested it shortly after it's been posted). The fact that it's not > committed is good enough indication that it's not The Answer. It is planned to be committed. Pawel has been away for a while. > (And besides, asking users to apply non-committed patches just to run > their systems normally is bad practice :) I can just imagine the > Release Notes: "if you're using ZFS, you'll have to manually patch the > kernel with this patch:..." :) ZFS already tells you up front that it's experimental code and likely to have problems. Users of 7.0-RELEASE should not have unrealistic expectations. > This close to the -RELEASE, I judge the chances of it being committed are low). Perhaps, but that only applies to 7.0-RELEASE. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 15:36:06 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2346016A41B; Sun, 6 Jan 2008 15:36:06 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 062D413C468; Sun, 6 Jan 2008 15:36:05 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id DDD404A712; Sun, 6 Jan 2008 10:36:04 -0500 (EST) Date: Sun, 6 Jan 2008 15:36:04 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ivan Voras In-Reply-To: Message-ID: <20080106141157.I105@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 15:36:06 -0000 On Fri, 4 Jan 2008, Ivan Voras wrote: > As far as I know about the details of implementation and what would it take > to fix the problems, is it safe to assume ZFS will never become stable > during 7.x lifetime? I'm not sure if anyone has mentioned this yet in the thread, but another thing worth taking into account in considering the stability of ZFS is whether or not Sun considers it a production feature in Solaris. Last I heard, it was still considered an experimental feature there as well. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 15:46:12 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 027ED16A421 for ; Sun, 6 Jan 2008 15:46:12 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0BD7E13C447; Sun, 6 Jan 2008 15:46:09 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4780F7C0.5010101@FreeBSD.org> Date: Sun, 06 Jan 2008 16:46:08 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Maciej Suszko References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <20080106144627.a91a62c1.maciej@suszko.eu> In-Reply-To: <20080106144627.a91a62c1.maciej@suszko.eu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 15:46:12 -0000 Maciej Suszko wrote: > Kris Kennaway wrote: >> Ivan Voras wrote: >>> On 06/01/2008, Peter Schuller wrote: >>>>> This number is not so large. It seems to be easily crashed by >>>>> rsync, for example (speaking from my own experience, and also >>>>> some of my colleagues). >>>> I can definitely say this is not *generally* true, as I do a lot of >>>> rsyncing/rdiff-backup:ing and similar stuff (with many files / >>>> large files) on ZFS without any stability issues. Problems for me >>>> have been limited to 32bit and the memory exhaustion issue rather >>>> than "hard" issues. >>> It's not generally true since kmem problems with rsync are often >>> hard to repeat - I have them on one machine, but not on another, >>> similar machine. This nonrepeatability is also a part of the >>> problem. >>> >>>> But perhaps that's all you are referring to. >>> Mostly. I did have a ZFS crash with rsync that wasn't kmem related, >>> but only once. >> kmem problems are just tuning. They are not indicative of stability >> problems in ZFS. Please report any further non-kmem panics you >> experience. > > I agree that ZFS is pretty stable itself. I use 32bit machine with > 2gigs od RAM and all hang cases are kmem related, but the fact is that > I haven't found any way of tuning to stop it crashing. When I do some > rsyncing, especially beetwen different pools - it hangs or reboots - > mostly on bigger files (i.e. rsyncing ports tree with distfiles). > At the moment I patched the kernel with vm_kern.c.2.patch and it just > stopped crashing, but from time to time the machine looks like beeing > freezed for a second or two, after that it works normally. > Have you got any similar experience? That is expected. That patch makes the system do more work to try and reclaim memory when it would previously have panicked from lack of memory. However, the same advice applies as to Ivan: you should try and tune the memory parameters better to avoid this last-ditch sitation. Kris P.S. It sounds like you do not have sufficient debugging configured either: crashes should produce either a DDB prompt or a coredump so they can be studied and understood. From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 15:48:15 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75FED16A419; Sun, 6 Jan 2008 15:48:15 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (unknown [IPv6:2001:41d0:1:2ad2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 09DC113C458; Sun, 6 Jan 2008 15:48:15 +0000 (UTC) (envelope-from hlh@restart.be) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:1:2ad2::fffe:0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "avoriaz.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTP id CC0D21BAC24; Sun, 6 Jan 2008 16:48:13 +0100 (CET) Received: from morzine.restart.bel (morzine6.restart.bel [IPv6:2001:41d0:1:2ad2::1:2]) (authenticated bits=0) by restart.be (8.14.2/8.14.2) with ESMTP id m06Fm9G8009039; Sun, 6 Jan 2008 16:48:09 +0100 (CET) (envelope-from hlh@restart.be) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1199634493; bh=3aosA0Xyqgl7ukOp1MMKt0aIKlDWkDOznPQtJYN NEik=; h=DomainKey-Signature:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:X-Scanned-By; b=ckYPZ6H5+ZI XILhPgVISBsVE37ca+T7+fKBhYY1LjxCLXkh9+VMDoXW4R7R5/jmi7okKovZSgtZDVY 1z/yTAAg== DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to:cc: subject:references:in-reply-to:content-type: content-transfer-encoding:x-scanned-by; b=DDHXYTcSoHlsFNLturM84/hECu/yIrW1KHBy8UQ3gMUAnq15UwuM15Q0uvANJSAai 3BmAt+IhgZcUbU/WRxpnQ== Message-ID: <4780F839.5020200@restart.be> Date: Sun, 06 Jan 2008 16:48:09 +0100 From: Henri Hennebert Organization: RestartSoft User-Agent: Thunderbird 2.0.0.9 (X11/20071118) MIME-Version: 1.0 To: Kris Kennaway References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> In-Reply-To: <4780D289.7020509@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.63 on IPv6:2001:41d0:1:2ad2::1:1 Cc: freebsd-current@freebsd.org, Peter Schuller , Ivan Voras , Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 15:48:15 -0000 Kris Kennaway wrote: > Ivan Voras wrote: >> On 06/01/2008, Peter Schuller wrote: >>>> This number is not so large. It seems to be easily crashed by rsync, >>>> for example (speaking from my own experience, and also some of my >>>> colleagues). >>> I can definitely say this is not *generally* true, as I do a lot of >>> rsyncing/rdiff-backup:ing and similar stuff (with many files / large >>> files) >>> on ZFS without any stability issues. Problems for me have been >>> limited to >>> 32bit and the memory exhaustion issue rather than "hard" issues. >> >> It's not generally true since kmem problems with rsync are often hard >> to repeat - I have them on one machine, but not on another, similar >> machine. This nonrepeatability is also a part of the problem. >> >>> But perhaps that's all you are referring to. >> >> Mostly. I did have a ZFS crash with rsync that wasn't kmem related, >> but only once. > > kmem problems are just tuning. They are not indicative of stability > problems in ZFS. Please report any further non-kmem panics you experience. I encounter 2 times a deadlock during high I/O activity (the last one during rsync + rm -r on a 5GB hierarchy (openoffice-2/work). I was running with this patch: http://people.freebsd.org/~pjd/patches/zgd_done.patch db> show allpcpu Current CPU: 1 cpuid = 0 curthread = 0xa5ebe440: pid 3422 "txg_thread_enter" curpcb = 0xeb175d90 fpcurthread = none idlethread = 0xa5529aa0: pid 12 "idle: cpu0" APIC ID = 0 currentldt = 0x50 cpuid = 1 curthread = 0xa56ab220: pid 47 "arc_reclaim_thread" curpcb = 0xe6837d90 fpcurthread = none idlethread = 0xa5529880: pid 11 "idle: cpu1" APIC ID = 1 currentldt = 0x50 With the 2 times arc_reclaim_thread `running` > > Kris > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:03:49 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A202A16A41B; Sun, 6 Jan 2008 16:03:49 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D827913C467; Sun, 6 Jan 2008 16:03:47 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4780FBE2.8040208@FreeBSD.org> Date: Sun, 06 Jan 2008 17:03:46 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Henri Hennebert References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780F839.5020200@restart.be> In-Reply-To: <4780F839.5020200@restart.be> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Peter Schuller , Ivan Voras , Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:03:49 -0000 Henri Hennebert wrote: > Kris Kennaway wrote: >> Ivan Voras wrote: >>> On 06/01/2008, Peter Schuller wrote: >>>>> This number is not so large. It seems to be easily crashed by rsync, >>>>> for example (speaking from my own experience, and also some of my >>>>> colleagues). >>>> I can definitely say this is not *generally* true, as I do a lot of >>>> rsyncing/rdiff-backup:ing and similar stuff (with many files / large >>>> files) >>>> on ZFS without any stability issues. Problems for me have been >>>> limited to >>>> 32bit and the memory exhaustion issue rather than "hard" issues. >>> >>> It's not generally true since kmem problems with rsync are often hard >>> to repeat - I have them on one machine, but not on another, similar >>> machine. This nonrepeatability is also a part of the problem. >>> >>>> But perhaps that's all you are referring to. >>> >>> Mostly. I did have a ZFS crash with rsync that wasn't kmem related, >>> but only once. >> >> kmem problems are just tuning. They are not indicative of stability >> problems in ZFS. Please report any further non-kmem panics you >> experience. > > I encounter 2 times a deadlock during high I/O activity (the last one > during rsync + rm -r on a 5GB hierarchy (openoffice-2/work). > > I was running with this patch: > http://people.freebsd.org/~pjd/patches/zgd_done.patch > db> show allpcpu > Current CPU: 1 > > cpuid = 0 > curthread = 0xa5ebe440: pid 3422 "txg_thread_enter" > curpcb = 0xeb175d90 > fpcurthread = none > idlethread = 0xa5529aa0: pid 12 "idle: cpu0" > APIC ID = 0 > currentldt = 0x50 > > cpuid = 1 > curthread = 0xa56ab220: pid 47 "arc_reclaim_thread" > curpcb = 0xe6837d90 > fpcurthread = none > idlethread = 0xa5529880: pid 11 "idle: cpu1" > APIC ID = 1 > currentldt = 0x50 > > With the 2 times arc_reclaim_thread `running` Backtraces of the affected processes (or just alltrace) are usually required to proceed with debugging, and lock status is also often vital (show alllocks, requires witness). Also, in the case when threads are actually running (not deadlocked), then it is often useful to repeatedly break/continue and sample many backtraces to try and determine where the threads are looping. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:06:01 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4308A16A468 for ; Sun, 6 Jan 2008 16:05:59 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: from 26.mail-out.ovh.net (26.mail-out.ovh.net [91.121.27.225]) by mx1.freebsd.org (Postfix) with SMTP id 5EB8813C45D for ; Sun, 6 Jan 2008 16:05:59 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: (qmail 30736 invoked by uid 503); 6 Jan 2008 16:06:21 -0000 Received: (QMFILT: 1.0); 06 Jan 2008 16:06:21 -0000 Received: from unknown (HELO mail147.ha.ovh.net) (213.186.33.59) by 26.mail-out.ovh.net with SMTP; 6 Jan 2008 16:06:21 -0000 Received: from b0.ovh.net (HELO queue-out) (213.186.33.50) by b0.ovh.net with SMTP; 6 Jan 2008 16:05:55 -0000 Received: from 217-153-241-141.zab.nat.hnet.pl (HELO arsenic) (maciej@suszko.eu@217.153.241.141) by ns0.ovh.net with SMTP; 6 Jan 2008 16:05:51 -0000 Date: Sun, 6 Jan 2008 17:05:44 +0100 From: Maciej Suszko To: freebsd-current@freebsd.org Message-Id: <20080106170544.93f7ab1b.maciej@suszko.eu> In-Reply-To: <4780F7C0.5010101@FreeBSD.org> References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <20080106144627.a91a62c1.maciej@suszko.eu> <4780F7C0.5010101@FreeBSD.org> X-Mailer: Sylpheed 2.4.8 (GTK+ 2.12.3; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Ovh-Remote: 217.153.241.141 (217-153-241-141.zab.nat.hnet.pl) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-Spam-Check: DONE|H 0.500111/N Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:06:01 -0000 Kris Kennaway wrote: > Maciej Suszko wrote: > > Kris Kennaway wrote: > >> Ivan Voras wrote: > >>> On 06/01/2008, Peter Schuller wrote: > >>>>> This number is not so large. It seems to be easily crashed by > >>>>> rsync, for example (speaking from my own experience, and also > >>>>> some of my colleagues). > >>>> I can definitely say this is not *generally* true, as I do a lot > >>>> of rsyncing/rdiff-backup:ing and similar stuff (with many files / > >>>> large files) on ZFS without any stability issues. Problems for me > >>>> have been limited to 32bit and the memory exhaustion issue rather > >>>> than "hard" issues. > >>> It's not generally true since kmem problems with rsync are often > >>> hard to repeat - I have them on one machine, but not on another, > >>> similar machine. This nonrepeatability is also a part of the > >>> problem. > >>> > >>>> But perhaps that's all you are referring to. > >>> Mostly. I did have a ZFS crash with rsync that wasn't kmem > >>> related, but only once. > >> kmem problems are just tuning. They are not indicative of > >> stability problems in ZFS. Please report any further non-kmem > >> panics you experience. > > > > I agree that ZFS is pretty stable itself. I use 32bit machine with > > 2gigs od RAM and all hang cases are kmem related, but the fact is > > that I haven't found any way of tuning to stop it crashing. When I > > do some rsyncing, especially beetwen different pools - it hangs or > > reboots - mostly on bigger files (i.e. rsyncing ports tree with > > distfiles). At the moment I patched the kernel with > > vm_kern.c.2.patch and it just stopped crashing, but from time to > > time the machine looks like beeing freezed for a second or two, > > after that it works normally. Have you got any similar experience? > > That is expected. That patch makes the system do more work to try > and reclaim memory when it would previously have panicked from lack > of memory. However, the same advice applies as to Ivan: you should > try and tune the memory parameters better to avoid this last-ditch > sitation. As Ivan said - tuning kmem_size only delay the moment system crash, earlier or after it happens - that's my point of view. > P.S. It sounds like you do not have sufficient debugging configured > either: crashes should produce either a DDB prompt or a coredump so > they can be studied and understood. You're right - I turned debugging off, because it's not a production machine and I can afford such behaviour. Right now, using kernel with kmem patch applied it's ,,usable''. -- regards, Maciej Suszko. From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:10:46 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54DC616A419; Sun, 6 Jan 2008 16:10:46 +0000 (UTC) (envelope-from srw@udor.net) Received: from mail.udor.net (mail.udor.net [64.34.95.190]) by mx1.freebsd.org (Postfix) with ESMTP id 35E2713C43E; Sun, 6 Jan 2008 16:10:46 +0000 (UTC) (envelope-from srw@udor.net) Received: from localhost (KD121110020099.ppp-bb.dion.ne.jp [121.110.20.99]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.udor.net (Postfix) with ESMTP id 31FF64294; Sun, 6 Jan 2008 11:06:54 -0500 (EST) Date: Mon, 7 Jan 2008 01:10:39 +0900 From: srwadleigh To: Pawel Jakub Dawidek Message-ID: <20080107011039.6f886630@udor.net> In-Reply-To: <20080105150355.GB6472@garage.freebsd.pl> References: <20080102114327.62f661f5@udor.net> <20080105150355.GB6472@garage.freebsd.pl> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.3; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/BRHvKKQAf7wU7xIU8wwNOMg"; protocol="application/pgp-signature"; micalg=PGP-SHA1 Cc: freebsd-current@freebsd.org Subject: Re: Gjournal troubles X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:10:46 -0000 --Sig_/BRHvKKQAf7wU7xIU8wwNOMg Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sat, 5 Jan 2008 16:03:55 +0100 Pawel Jakub Dawidek wrote: > On Wed, Jan 02, 2008 at 11:43:27AM +0900, srwadleigh wrote: > > I am having a strange problem with gjournal on a thinkpad T41, > >=20 > > I am running: 7.0-PRERELEASE - Wed Jan 2 06:05:20 JST 2008 > > And have the problem with both a custom and generic kernel. > >=20 > > If I load gjournal through loader.conf or compile GEOM_JOURNAL into > > the kernel, upon reboot all my slices change, and break booting. > >=20 > > ad0s1a becomes ad0a > > ad0s1d becomes ad0d, and so on.. > >=20 > > If I manually run gjournal load after boot the slices are fine, > > everything works as expected. > >=20 > > Here is my journal setup: > >=20 > > /dev/ad0s1f.journal 64G /usr/home > >=20 > > Geom name: gjournal 2080874044 > > ID: 2080874044 > > Providers: > > 1. Name: ad0s1f.journal > > Mediasize: 70812433920 (66G) > > Sectorsize: 512 > > Mode: r1w1e1 > > Consumers: > > 1. Name: ad0s1f > > Mediasize: 71886176256 (67G) > > Sectorsize: 512 > > Mode: r1w1e1 > > Jend: 71886175744 > > Jstart: 70812433920 > > Role: Data,Journal > >=20 > >=20 > > Doing some research on the list I found a similar problem in the > > past with gmirror, where the slice and device ending at the same > > place was being confused. The solution suggested there seemed to be > > to hardcode the provider names into the metadata. > >=20 > > http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014448.html > >=20 > > My question is, is this possibly the same issue now with gjournal? > > and is it possible to hardcode provider names after a journal has > > been created? >=20 > Just unmount the file system, stop the journal and call 'gjournal > label' with exactly the sam parameters as originally plus '-h' option. >=20 I will try this, currently my journal is on the same slice as data so I get the used by file system message. I may have to re-configure with a dedicated journal slice so I can use the -f option? Thanks for the help! I will see how it goes. srw --Sig_/BRHvKKQAf7wU7xIU8wwNOMg Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFHgP1/pXXBX0jZoJYRAlx/AKCsOaty/nE/GG4V1bY6pqwEHu04FgCgmg/Z C3dGhUxRNu7eno87h9YVn+g= =Tm54 -----END PGP SIGNATURE----- --Sig_/BRHvKKQAf7wU7xIU8wwNOMg-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:22:09 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D62C716A421 for ; Sun, 6 Jan 2008 16:22:09 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3E09113C448; Sun, 6 Jan 2008 16:22:09 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4781002F.5020708@FreeBSD.org> Date: Sun, 06 Jan 2008 17:22:07 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Maciej Suszko References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <20080106144627.a91a62c1.maciej@suszko.eu> <4780F7C0.5010101@FreeBSD.org> <20080106170544.93f7ab1b.maciej@suszko.eu> In-Reply-To: <20080106170544.93f7ab1b.maciej@suszko.eu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:22:09 -0000 Maciej Suszko wrote: > Kris Kennaway wrote: >> Maciej Suszko wrote: >>> Kris Kennaway wrote: >>>> Ivan Voras wrote: >>>>> On 06/01/2008, Peter Schuller wrote: >>>>>>> This number is not so large. It seems to be easily crashed by >>>>>>> rsync, for example (speaking from my own experience, and also >>>>>>> some of my colleagues). >>>>>> I can definitely say this is not *generally* true, as I do a lot >>>>>> of rsyncing/rdiff-backup:ing and similar stuff (with many files / >>>>>> large files) on ZFS without any stability issues. Problems for me >>>>>> have been limited to 32bit and the memory exhaustion issue rather >>>>>> than "hard" issues. >>>>> It's not generally true since kmem problems with rsync are often >>>>> hard to repeat - I have them on one machine, but not on another, >>>>> similar machine. This nonrepeatability is also a part of the >>>>> problem. >>>>> >>>>>> But perhaps that's all you are referring to. >>>>> Mostly. I did have a ZFS crash with rsync that wasn't kmem >>>>> related, but only once. >>>> kmem problems are just tuning. They are not indicative of >>>> stability problems in ZFS. Please report any further non-kmem >>>> panics you experience. >>> I agree that ZFS is pretty stable itself. I use 32bit machine with >>> 2gigs od RAM and all hang cases are kmem related, but the fact is >>> that I haven't found any way of tuning to stop it crashing. When I >>> do some rsyncing, especially beetwen different pools - it hangs or >>> reboots - mostly on bigger files (i.e. rsyncing ports tree with >>> distfiles). At the moment I patched the kernel with >>> vm_kern.c.2.patch and it just stopped crashing, but from time to >>> time the machine looks like beeing freezed for a second or two, >>> after that it works normally. Have you got any similar experience? >> That is expected. That patch makes the system do more work to try >> and reclaim memory when it would previously have panicked from lack >> of memory. However, the same advice applies as to Ivan: you should >> try and tune the memory parameters better to avoid this last-ditch >> sitation. > > As Ivan said - tuning kmem_size only delay the moment system crash, > earlier or after it happens - that's my point of view. So the same question applies: exactly what steps did you take to tune the memory parameters? Extracting this information from you guys shouldn't be as hard as this :) Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:45:13 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60C9816A46E for ; Sun, 6 Jan 2008 16:45:13 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id B6C1E13C510 for ; Sun, 6 Jan 2008 16:45:12 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBYcW-0007oK-3U for freebsd-current@freebsd.org; Sun, 06 Jan 2008 16:45:08 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 16:45:08 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 16:45:08 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 17:45:01 +0100 Lines: 70 Message-ID: References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigEDC04B50439687A9573D2846" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <4780EF09.4090908@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:45:13 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigEDC04B50439687A9573D2846 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > No, clearly it is not enough=20 This looks like we're constantly chasing the "right amount". Does it=20 depend so much on CPU and IO speed that there's never a generally=20 sufficient "right amount"? So when CPU and drive speed increase, the new = amount will always be some bigger value? >(and you claimed previously to have done=20 > more tuning than this).=20 Where? What else is there except kmem tuning (including KVA_PAGES)? IIRC = Pawel said all other suggested tunings don't do much. > I have it set to 600MB on the i386 system with=20 > a 1.5GB KVA. Both were necessary. My point is that the fact that such things are necessary (1.5 GB KVA os=20 a lot on i386) mean that there are serious problems which aren't getting = fixed since ZFS was imported (that's over 6 months ago). I see you've added to http://wiki.freebsd.org/ZFSTuningGuide; can you=20 please add the values that work for you to it (especially for KVA_PAGES=20 since the exact kernel configuration line is never spelled out in the=20 document; and say for which hardware are the values known to work)? > ZFS already tells you up front that it's experimental code and likely t= o=20 > have problems. =20 I know it's experimental, but requiring users to perform so much tuning=20 just to get it work without crashing will mean it will get a bad=20 reputation early on. Do you (or anyone) know what are the reasons for=20 not having vm.kmem_size to 512 MB by default? Better yet, why not=20 increase both vm.kmem_size and KVA_PAGES to (the equivalent of) 640 MB=20 or 768 MB by default for 7.0? >Users of 7.0-RELEASE should not have unrealistic > expectations. As I've said at the first post of this thread: I'm interested in if it's = ever going to be stable for 7.x. --------------enigEDC04B50439687A9573D2846 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgQWNldnAQVacBcgRAhigAKDK9bFXJy5Y6nLyyk7Xb98iA57cwQCgrTRz as4xA3tKTpL2jXpYtGIKDuI= =dzLD -----END PGP SIGNATURE----- --------------enigEDC04B50439687A9573D2846-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:47:33 2008 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A02E116A419; Sun, 6 Jan 2008 16:47:33 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (unknown [IPv6:2001:41d0:1:2ad2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 555E013C46E; Sun, 6 Jan 2008 16:47:33 +0000 (UTC) (envelope-from hlh@restart.be) Received: from restart.be (avoriaz.tunnel.bel [IPv6:2001:41d0:1:2ad2::fffe:0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "avoriaz.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTP id 818161BAC24; Sun, 6 Jan 2008 17:47:32 +0100 (CET) Received: from morzine.restart.bel (morzine6.restart.bel [IPv6:2001:41d0:1:2ad2::1:2]) (authenticated bits=0) by restart.be (8.14.2/8.14.2) with ESMTP id m06GlUL0009254; Sun, 6 Jan 2008 17:47:30 +0100 (CET) (envelope-from hlh@restart.be) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=restart.be; s=avoriaz; t=1199638051; bh=Nxm2sVPv8+nDgveUYUpIlkxy7q/JQBlU5EAiTX5 Uwjs=; h=DomainKey-Signature:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:X-Scanned-By; b=tYgfi3LyqaP RgUdlBmzZy0G4AUOdX6tos56BA9gX7jch5kr7PGf9aOSHv6JEIiy+O66wnPlmbJ9XnW UcVvakMg== DomainKey-Signature: a=rsa-sha1; s=avoriaz; d=restart.be; c=nofws; q=dns; h=message-id:date:from:organization:user-agent:mime-version:to:cc: subject:references:in-reply-to:content-type: content-transfer-encoding:x-scanned-by; b=PjvlaThtwmrFLWE+TCkeHGBrZ1px7h/9q2oFH94wfUq46eP7qG1LGHf7tipNmeyxv hNAnykhsi1J6fDhOUJi+A== Message-ID: <47810621.8080406@restart.be> Date: Sun, 06 Jan 2008 17:47:29 +0100 From: Henri Hennebert Organization: RestartSoft User-Agent: Thunderbird 2.0.0.9 (X11/20071118) MIME-Version: 1.0 To: Kris Kennaway References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780F839.5020200@restart.be> <4780FBE2.8040208@FreeBSD.org> In-Reply-To: <4780FBE2.8040208@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.63 on IPv6:2001:41d0:1:2ad2::1:1 Cc: freebsd-current@FreeBSD.org, Peter Schuller , Ivan Voras , Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:47:33 -0000 Kris Kennaway wrote: > Henri Hennebert wrote: >> Kris Kennaway wrote: >>> Ivan Voras wrote: >>>> On 06/01/2008, Peter Schuller wrote: >>>>>> This number is not so large. It seems to be easily crashed by rsync, >>>>>> for example (speaking from my own experience, and also some of my >>>>>> colleagues). >>>>> I can definitely say this is not *generally* true, as I do a lot of >>>>> rsyncing/rdiff-backup:ing and similar stuff (with many files / >>>>> large files) >>>>> on ZFS without any stability issues. Problems for me have been >>>>> limited to >>>>> 32bit and the memory exhaustion issue rather than "hard" issues. >>>> >>>> It's not generally true since kmem problems with rsync are often hard >>>> to repeat - I have them on one machine, but not on another, similar >>>> machine. This nonrepeatability is also a part of the problem. >>>> >>>>> But perhaps that's all you are referring to. >>>> >>>> Mostly. I did have a ZFS crash with rsync that wasn't kmem related, >>>> but only once. >>> >>> kmem problems are just tuning. They are not indicative of stability >>> problems in ZFS. Please report any further non-kmem panics you >>> experience. >> >> I encounter 2 times a deadlock during high I/O activity (the last one >> during rsync + rm -r on a 5GB hierarchy (openoffice-2/work). >> >> I was running with this patch: >> http://people.freebsd.org/~pjd/patches/zgd_done.patch >> db> show allpcpu >> Current CPU: 1 >> >> cpuid = 0 >> curthread = 0xa5ebe440: pid 3422 "txg_thread_enter" >> curpcb = 0xeb175d90 >> fpcurthread = none >> idlethread = 0xa5529aa0: pid 12 "idle: cpu0" >> APIC ID = 0 >> currentldt = 0x50 >> >> cpuid = 1 >> curthread = 0xa56ab220: pid 47 "arc_reclaim_thread" >> curpcb = 0xe6837d90 >> fpcurthread = none >> idlethread = 0xa5529880: pid 11 "idle: cpu1" >> APIC ID = 1 >> currentldt = 0x50 >> >> With the 2 times arc_reclaim_thread `running` > > Backtraces of the affected processes (or just alltrace) are usually noted for next time > required to proceed with debugging, and lock status is also often vital > (show alllocks, requires witness). I add it to my kernel config Also, in the case when threads are > actually running (not deadlocked), then it is often useful to repeatedly > break/continue and sample many backtraces to try and determine where the > threads are looping. I do this after the second deadlock and arc_reclaim_thread was always there and second cpu was idle. Henri > > Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 16:50:05 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2C1A16A418 for ; Sun, 6 Jan 2008 16:50:05 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 9437713C45A for ; Sun, 6 Jan 2008 16:50:05 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1JBYhG-00080w-Ty for freebsd-current@freebsd.org; Sun, 06 Jan 2008 16:50:02 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 16:50:02 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 16:50:02 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 17:47:52 +0100 Lines: 33 Message-ID: References: <20080106141157.I105@fledge.watson.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig4083928B2011A1784CF8EBD9" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <20080106141157.I105@fledge.watson.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 16:50:05 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig4083928B2011A1784CF8EBD9 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Robert Watson wrote: > I'm not sure if anyone has mentioned this yet in the thread, but anothe= r=20 > thing worth taking into account in considering the stability of ZFS is = > whether or not Sun considers it a production feature in Solaris. Last = I=20 > heard, it was still considered an experimental feature there as well. Last I heard, rsync didn't crash Solaris on ZFS :) --------------enig4083928B2011A1784CF8EBD9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgQY4ldnAQVacBcgRAgPaAJ9VdX3dBVBaTUe6sHPZp8BC7fGUrwCfQTy7 aQdjHiLPe1K5WOwdh67nJ4I= =Z7cc -----END PGP SIGNATURE----- --------------enig4083928B2011A1784CF8EBD9-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:08:55 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5D5816A418; Sun, 6 Jan 2008 17:08:55 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9EB6913C447; Sun, 6 Jan 2008 17:08:55 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 2F4344EB8C; Sun, 6 Jan 2008 12:08:55 -0500 (EST) Date: Sun, 6 Jan 2008 17:08:55 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ivan Voras In-Reply-To: Message-ID: <20080106170452.L105@fledge.watson.org> References: <20080106141157.I105@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:08:55 -0000 On Sun, 6 Jan 2008, Ivan Voras wrote: > Robert Watson wrote: > >> I'm not sure if anyone has mentioned this yet in the thread, but another >> thing worth taking into account in considering the stability of ZFS is >> whether or not Sun considers it a production feature in Solaris. Last I >> heard, it was still considered an experimental feature there as well. > > Last I heard, rsync didn't crash Solaris on ZFS :) My admittedly second-hand understanding is that ZFS shows similarly gratuitous memory use on both Mac OS X and Solaris. One advantage Solaris has is that it runs primarily on expensive 64-bit servers with lots of memory. Part of the problem on FreeBSD is that people run ZFS on sytems with 32-bit CPUs and a lot less memory. It could be that ZFS should be enforcing higher minimum hardware requirements to mount (i.e., refusing to run on systems with 32-bit address spaces or <4gb of memory and inadequate tuning). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:12:05 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7A2816A41A; Sun, 6 Jan 2008 17:12:05 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3076B13C467; Sun, 6 Jan 2008 17:12:05 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47810BE3.4080601@FreeBSD.org> Date: Sun, 06 Jan 2008 18:12:03 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:12:05 -0000 Ivan Voras wrote: > Kris Kennaway wrote: > >> No, clearly it is not enough > > This looks like we're constantly chasing the "right amount". Does it > depend so much on CPU and IO speed that there's never a generally > sufficient "right amount"? So when CPU and drive speed increase, the new > amount will always be some bigger value? It depends on your workload, which in turn depends on your hardware. The harder you can drive ZFS the more memory it will require. >> (and you claimed previously to have done more tuning than this). > > Where? What else is there except kmem tuning (including KVA_PAGES)? IIRC > Pawel said all other suggested tunings don't do much. Tuning is an interactive process. If 512MB is not enough kmem_map, then increase it. Repeat as necessary. >> I have it set to 600MB on the i386 system with a 1.5GB KVA. Both were >> necessary. > > My point is that the fact that such things are necessary (1.5 GB KVA os > a lot on i386) mean that there are serious problems which aren't getting > fixed since ZFS was imported (that's over 6 months ago). ZFS is a memory hog. There is nothing that can really be done about this, and it is just not a good fit on i386 because of limitations of the hardware architecture. Note that Sun does not recommend using ZFS on a 32-bit system either, for the same reasons. It is unlikely this can really be fixed, although mitigation strategies like the vm_kern.c patch are possible. > I see you've added to http://wiki.freebsd.org/ZFSTuningGuide; can you > please add the values that work for you to it (especially for KVA_PAGES > since the exact kernel configuration line is never spelled out in the > document; and say for which hardware are the values known to work)? OK. >> ZFS already tells you up front that it's experimental code and likely >> to have problems. > > I know it's experimental, but requiring users to perform so much tuning > just to get it work without crashing will mean it will get a bad > reputation early on. Do you (or anyone) know what are the reasons for > not having vm.kmem_size to 512 MB by default? Increasing vm.kmem_size.max to 512MB by default has other implications, but it is something that should be considered. > Better yet, why not > increase both vm.kmem_size and KVA_PAGES to (the equivalent of) 640 MB > or 768 MB by default for 7.0? That is answered in the tuning guide. Tuning KVA_PAGES by default is not appropriate. > >Users of 7.0-RELEASE should not have unrealistic > > expectations. > > As I've said at the first post of this thread: I'm interested in if it's > ever going to be stable for 7.x. This was in reply to a comment you made about the vm_kern.c patch affecting users of 7.0-RELEASE. Anyway, to sum up, ZFS has known bugs, some of which are unresolved by the authors, and it is difficult to make it work on i386. It is likely that the bugs will be fixed over time (obviously), but amd64 will always be a better choice than i386 for using ZFS because you will not be continually bumping up against the hardware limitations. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:13:38 2008 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17ECD16A46C; Sun, 6 Jan 2008 17:13:37 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3EA2013C4F2; Sun, 6 Jan 2008 17:13:31 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47810C39.8010302@FreeBSD.org> Date: Sun, 06 Jan 2008 18:13:29 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Henri Hennebert References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780F839.5020200@restart.be> <4780FBE2.8040208@FreeBSD.org> <47810621.8080406@restart.be> In-Reply-To: <47810621.8080406@restart.be> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, Peter Schuller , Ivan Voras , Brooks Davis Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:13:38 -0000 Henri Hennebert wrote: > Kris Kennaway wrote: >> Henri Hennebert wrote: >>> Kris Kennaway wrote: >>>> Ivan Voras wrote: >>>>> On 06/01/2008, Peter Schuller wrote: >>>>>>> This number is not so large. It seems to be easily crashed by rsync, >>>>>>> for example (speaking from my own experience, and also some of my >>>>>>> colleagues). >>>>>> I can definitely say this is not *generally* true, as I do a lot of >>>>>> rsyncing/rdiff-backup:ing and similar stuff (with many files / >>>>>> large files) >>>>>> on ZFS without any stability issues. Problems for me have been >>>>>> limited to >>>>>> 32bit and the memory exhaustion issue rather than "hard" issues. >>>>> >>>>> It's not generally true since kmem problems with rsync are often hard >>>>> to repeat - I have them on one machine, but not on another, similar >>>>> machine. This nonrepeatability is also a part of the problem. >>>>> >>>>>> But perhaps that's all you are referring to. >>>>> >>>>> Mostly. I did have a ZFS crash with rsync that wasn't kmem related, >>>>> but only once. >>>> >>>> kmem problems are just tuning. They are not indicative of stability >>>> problems in ZFS. Please report any further non-kmem panics you >>>> experience. >>> >>> I encounter 2 times a deadlock during high I/O activity (the last one >>> during rsync + rm -r on a 5GB hierarchy (openoffice-2/work). >>> >>> I was running with this patch: >>> http://people.freebsd.org/~pjd/patches/zgd_done.patch >>> db> show allpcpu >>> Current CPU: 1 >>> >>> cpuid = 0 >>> curthread = 0xa5ebe440: pid 3422 "txg_thread_enter" >>> curpcb = 0xeb175d90 >>> fpcurthread = none >>> idlethread = 0xa5529aa0: pid 12 "idle: cpu0" >>> APIC ID = 0 >>> currentldt = 0x50 >>> >>> cpuid = 1 >>> curthread = 0xa56ab220: pid 47 "arc_reclaim_thread" >>> curpcb = 0xe6837d90 >>> fpcurthread = none >>> idlethread = 0xa5529880: pid 11 "idle: cpu1" >>> APIC ID = 1 >>> currentldt = 0x50 >>> >>> With the 2 times arc_reclaim_thread `running` >> >> Backtraces of the affected processes (or just alltrace) are usually > > noted for next time > >> required to proceed with debugging, and lock status is also often >> vital (show alllocks, requires witness). > > I add it to my kernel config > > Also, in the case when threads are >> actually running (not deadlocked), then it is often useful to >> repeatedly break/continue and sample many backtraces to try and >> determine where the threads are looping. > > I do this after the second deadlock and arc_reclaim_thread was always > there and second cpu was idle. To repeat, it is important not just to note which thread is running, but *what the thread is doing*. This means repeatedly comparing the backtraces, which will allow you to build up a picture of which part of the code it is looping in. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:20:47 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71E0316A41A; Sun, 6 Jan 2008 17:20:47 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 710B513C455; Sun, 6 Jan 2008 17:20:43 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47810DE3.3050106@FreeBSD.org> Date: Sun, 06 Jan 2008 18:20:35 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080106141157.I105@fledge.watson.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:20:47 -0000 Ivan Voras wrote: > Robert Watson wrote: > >> I'm not sure if anyone has mentioned this yet in the thread, but >> another thing worth taking into account in considering the stability >> of ZFS is whether or not Sun considers it a production feature in >> Solaris. Last I heard, it was still considered an experimental >> feature there as well. > > Last I heard, rsync didn't crash Solaris on ZFS :) > [Citation needed] From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:21:07 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADB3816A41A for ; Sun, 6 Jan 2008 17:21:07 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 3B9F413C468 for ; Sun, 6 Jan 2008 17:21:07 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBZBJ-00013z-TC for freebsd-current@freebsd.org; Sun, 06 Jan 2008 17:21:05 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:21:05 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:21:05 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 18:20:52 +0100 Lines: 50 Message-ID: References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigBF0E8DBB5E5B643C0C0784B4" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <47810BE3.4080601@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:21:07 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigBF0E8DBB5E5B643C0C0784B4 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > > Better yet, why not >> increase both vm.kmem_size and KVA_PAGES to (the equivalent of) 640 MB= =20 >> or 768 MB by default for 7.0? >=20 > That is answered in the tuning guide. Tuning KVA_PAGES by default is=20 > not appropriate. Ok. I'd like to understand what is the relationship between KVA_PAGES=20 and vm.kmem_size. The tuning guide says: """By default the kernel receives 1GB of the 4GB of address space=20 available on the i386 architecture, and this is used for all of the=20 kernel address space needs, not just the kmem map. By increasing=20 KVA_PAGES you can allocate a larger proportion of the 4GB address=20 space...""" and: """recompile your kernel with increased KVA_PAGES option, to increase=20 the size of the kernel address space, before vm.kmem_size can be=20 increased beyond 512M""" What is the other 512 MB of the 1 GB used for? --------------enigBF0E8DBB5E5B643C0C0784B4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgQ36ldnAQVacBcgRArjoAJ4+6LvxL9bOjNOIHLiCA5rs2o99hACfZlBl mQUKsJ1MR9ZyUXbn5EoYyKE= =d0JH -----END PGP SIGNATURE----- --------------enigBF0E8DBB5E5B643C0C0784B4-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:21:33 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BB3B16A46C for ; Sun, 6 Jan 2008 17:21:33 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from mail1-backupmx.mn2.visi.com (lugh.stpaul.visi.com [208.42.42.102]) by mx1.freebsd.org (Postfix) with ESMTP id DBD9D13C46E for ; Sun, 6 Jan 2008 17:21:32 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from mail1-backupmx.mn2.visi.com (localhost.visi.com [127.0.0.1]) by mail1-backupmx.mn2.visi.com (Postfix) with ESMTP id 95CEC1120C1 for ; Sun, 6 Jan 2008 11:02:59 -0600 (CST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on mail1.mn2.visi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.2.3 Received: from cenn-smtp.mc.mpls.visi.com (cenn.mc.mpls.visi.com [208.42.156.9]) by mail1-backupmx.mn2.visi.com (Postfix) with ESMTP for ; Sun, 6 Jan 2008 11:02:59 -0600 (CST) Received: from mail.tcbug.org (mail.tcbug.org [208.42.70.163]) by cenn-smtp.mc.mpls.visi.com (Postfix) with ESMTP id BB4698106 for ; Sun, 6 Jan 2008 11:02:53 -0600 (CST) Received: from [172.24.141.6] (unknown [207.229.236.211]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.tcbug.org (Postfix) with ESMTP id D264E214A2DD for ; Sun, 6 Jan 2008 11:03:00 -0600 (CST) From: Josh Paetzel To: current@freebsd.org Date: Sun, 6 Jan 2008 11:02:34 -0600 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3726984.iSCzRLiZYO"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200801061102.46233.josh@tcbug.org> Cc: Subject: state of 7.0-RC1 on an IBM t60p X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:21:33 -0000 --nextPart3726984.iSCzRLiZYO Content-Type: multipart/mixed; boundary="Boundary-01=_rmQgHSVm+OsIfEZ" Content-Transfer-Encoding: 7bit Content-Disposition: inline --Boundary-01=_rmQgHSVm+OsIfEZ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I recently installed FBSD 7.0-BETA4 on my laptop in an effort to get workin= g=20 intel 3945abg wireless. I bumped it up to 7.0-RC1 the other day. I've bee= n=20 in a bit of a state health wise and have gotten to the point where i need t= o=20 depend on my laptop, so it has to go back to releng_6, but I thought I'd=20 write out the problems I've been having. 1) wpi is nearly non-functional. It seems to associate best to WPA2 networ= ks,=20 struggles with WEP and won't associate at all to open networks. Once it's= =20 associated it spams the console with "discarding packet without header"=20 and "wpi_cmd: couldn't set tx power" quite a bit, as well as other random=20 output. The real problem I've had with it is that it panics under moderate= =20 data rates, in the 1000-1500K/sec range. 2) pantech px-500 evdo pccard. This thing uses the ucom/umodem driver and= =20 panics after a few seconds of transfer at anything over 30K/sec or so. It= =20 also panics instantly on insertion or removal. I've been unable to get a=20 crash dump or to the debugger for this, my swap is encrypted and seeming ge= ts=20 cleared before savecore runs, and i've been unable to dump to usb swap=20 devices. (The card is a pccard that has a usb controller with a usb-> seri= al=20 adapter on it) 3) sutdown output is garbled. This doesn't seem to hurt anything, but at= =20 some point in shutdown the console messages start interleaving. 4) acpi suspend and resume doesn't work, but that's not a huge shock.l 5) ath generates interrupt storms that are throttled. console output snippet.... wpi0: link state changed to UP wpi0: could not set power mode wpi0: wpi_cmd: cmd 119 not sent, busy wpi0: could not set power mode wpi0: wpi_cmd: cmd 151 not sent, busy wpi0: could not adjust Tx power wpi0: wpi_cmd: cmd 72 not sent, busy wpi0: link state changed to DOWN wpi0: wpi_cmd: cmd 119 not sent, busy wpi0: could not set power mode wpi0: wpi_cmd: cmd 119 not sent, busy wpi0: could not set power mode wpi0: wpi_cmd: cmd 72 not sent, busy wpi0: wpi_cmd: cmd 16 not sent, busy wpi0: could not configure wpi0: could not send authentication request ath0: mem 0x88000000-0x8800ffff irq 16 at device 0.0 on=20 cardbus0 ath0: [ITHREAD] ath0: using obsoleted if_watchdog interface ath0: Ethernet address: 00:1b:2f:5e:fa:d7 ath0: mac 7.9 phy 4.5 radio 5.6 interrupt storm detected on "irq16:"; throttling interrupt source Hopefully soon I'll be in a better position to help debug and troubleshoot= ,=20 for now all I can do is enumerate my issues. dmesg and stuff attached... =2D-=20 Thanks, Josh Paetzel PGP: 8A48 EF36 5E9F 4EDA 5A8C 11B4 26F9 01F1 27AF AECB --Boundary-01=_rmQgHSVm+OsIfEZ Content-Type: text/plain; charset="us-ascii"; name="dmesg.boot" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dmesg.boot" Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-RC1 #0: Sat Dec 29 12:06:15 CST 2007 jpaetzel@roadrash.tcbug.org:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 CPU T5500 @ 1.66GHz (1661.96-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f6 Stepping = 6 Features=0xbfebfbff Features2=0xe39d AMD Features=0x20100000 AMD Features2=0x1 Cores per package: 2 real memory = 1072496640 (1022 MB) avail memory = 1035886592 (987 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ACPI Warning (tbfadt-0505): Optional field "Gpe1Block" has zero address or length: 0 102C/0 [20070320] ioapic0: Changing APIC ID to 1 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) hptrr: HPT RocketRAID controller driver v1.1 (Dec 29 2007 12:04:17) acpi0: on motherboard acpi0: [ITHREAD] acpi_ec0: port 0x62,0x66 on acpi0 acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 3ff00000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 cpu0: on acpi0 est0: on cpu0 p4tcc0: on cpu0 cpu1: on acpi0 est1: on cpu1 p4tcc1: on cpu1 acpi_lid0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0x2000-0x20ff mem 0xd8000000-0xdfffffff,0xee100000-0xee10ffff irq 16 at device 0.0 on pci1 pcm0: mem 0xee400000-0xee403fff irq 17 at device 27.0 on pci0 pcm0: [ITHREAD] pcib2: irq 20 at device 28.0 on pci0 pci2: on pcib2 em0: port 0x3000-0x301f mem 0xee000000-0xee01ffff irq 16 at device 0.0 on pci2 em0: Using MSI interrupt em0: Ethernet address: 00:16:41:e5:e1:f8 em0: [FILTER] pcib3: irq 21 at device 28.1 on pci0 pci3: on pcib3 wpi0: mem 0xedf00000-0xedf00fff irq 17 at device 0.0 on pci3 bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. bus_dmamem_alloc failed to align memory properly. wpi0: Ethernet address: 00:19:d2:9f:9f:30 wpi0: [ITHREAD] wpi0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wpi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wpi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps pcib4: irq 22 at device 28.2 on pci0 pci4: on pcib4 pcib5: irq 23 at device 28.3 on pci0 pci12: on pcib5 uhci0: port 0x1800-0x181f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x1820-0x183f irq 17 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x1860-0x187f irq 19 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xee404000-0xee4043ff irq 19 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered pcib6: at device 30.0 on pci0 pci21: on pcib6 cbb0: mem 0xe4300000-0xe4300fff irq 16 at device 0.0 on pci21 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 cbb0: [ITHREAD] isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1880-0x188f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atapci1: port 0x18c8-0x18cf,0x18ac-0x18af,0x18c0-0x18c7,0x18a8-0x18ab,0x18b0-0x18bf mem 0xee404400-0xee4047ff irq 16 at device 31.2 on pci0 atapci1: [ITHREAD] atapci1: AHCI Version 01.10 controller with 4 ports detected ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: port not implemented ata3: [ITHREAD] ata4: on atapci1 ata4: port not implemented ata4: [ITHREAD] ata5: on atapci1 ata5: port not implemented ata5: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 acpi_tz1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled battery0: on acpi0 acpi_acad0: on acpi0 acpi_ibm0: on acpi0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd0fff,0xd1000-0xd1fff,0xdc000-0xdffff,0xe0000-0xeffff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 8250 or not responding sio0: [FILTER] sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ugen0: on uhub3 ugen1: on uhub3 Timecounters tick every 1.000 msec hptrr: no controller detected. acd0: DVDR at ata0-master UDMA33 ad4: 114473MB at ata2-master SATA150 GEOM_LABEL: Label for provider acd0 is iso9660/300. pcm0: pcm0: SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/ad4s3a cryptosoft0: on motherboard GEOM_ELI: Device ad4s4.eli created. GEOM_ELI: Encryption: AES-CBC 128 GEOM_ELI: Crypto: software --Boundary-01=_rmQgHSVm+OsIfEZ Content-Type: text/plain; charset="us-ascii"; name="loader.conf" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="loader.conf" umodem_load="YES" snd_hda_load="YES" acpi_ibm_load="YES" legal.intel_wpi.license_ack=1 if_wpi_load="YES" wlan_load="YES" wlan_amrr_load="YES" firmware_load="YES" wpifw_load="YES" --Boundary-01=_rmQgHSVm+OsIfEZ Content-Type: text/plain; charset="us-ascii"; name="ROADRASH" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ROADRASH" cpu I686_CPU ident ROADRASH # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journaling options MD_ROOT # MD is a potential root device options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options NFS_ROOT # NFS usable as /, requires NFSCLIENT options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization options COMPAT_43TTY # BSD 4.3 TTY compat [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options ADAPTIVE_GIANT # Giant mutex is adaptive. options STOP_NMI # Stop CPUS using NMI instead of IPI options AUDIT # Security event auditing # To make an SMP kernel, the next two lines are needed options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC # CPU frequency control device cpufreq # Bus support. device pci # ATA and ATAPI devices device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives options ATA_STATIC_ID # Static device numbering # SCSI peripherals device scbus # SCSI bus (required for SCSI) device da # Direct Access (disks) device cd # CD device pass # Passthrough device (direct SCSI access) # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc device agp # support several AGP chipsets # Power management support (see NOTES for more options) #device apm # Add suspend/resume support for the i8254. device pmtimer # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support device cbb # cardbus (yenta) bridge device pccard # PC Card (16-bit) bus device cardbus # CardBus (32-bit) bus # Serial (COM) ports device sio # 8250, 16[45]50 based serial ports device uart # Generic UART driver # PCI Ethernet NICs. device em # Intel PRO/1000 adapter Gigabit Ethernet Card # Wireless NIC cards device wlan # 802.11 support device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support device wlan_tkip # 802.11 TKIP support device wlan_amrr # AMRR transmit rate control algorithm device wlan_scan_ap # 802.11 AP mode scanning device wlan_scan_sta # 802.11 STA mode scanning device ath # Atheros pci/cardbus NIC's device ath_hal # Atheros HAL (Hardware Access Layer) device ath_rate_sample # SampleRate tx rate control for ath # Pseudo devices. device loop # Network loopback device random # Entropy device device ether # Ethernet support device sl # Kernel SLIP device ppp # Kernel PPP device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device faith # IPv6-to-IPv4 relaying (translation) device firmware # firmware assist module # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter # USB support device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device usb # USB Bus (required) #device udbp # USB Double Bulk Pipe devices device ugen # Generic device uhid # "Human Interface Devices" device ukbd # Keyboard device ulpt # Printer device umass # Disks/Mass storage - Requires scbus and da device ums # Mouse options KDB options KDB_TRACE --Boundary-01=_rmQgHSVm+OsIfEZ-- --nextPart3726984.iSCzRLiZYO Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHgQm2JvkB8SevrssRApsdAJ4mXr8em02p4Zu3AyuzxgtPekePyQCeMEvZ /vIjT94eOfwx1UBJGHuBGDg= =fY4/ -----END PGP SIGNATURE----- --nextPart3726984.iSCzRLiZYO-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:28:50 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C90A516A419 for ; Sun, 6 Jan 2008 17:28:50 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5B4A913C44B for ; Sun, 6 Jan 2008 17:28:50 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBZIm-0001S4-P9 for freebsd-current@freebsd.org; Sun, 06 Jan 2008 17:28:48 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:28:48 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:28:48 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 18:28:30 +0100 Lines: 53 Message-ID: References: <20080106141157.I105@fledge.watson.org> <20080106170452.L105@fledge.watson.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig601868FFD7E993E6262A8244" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <20080106170452.L105@fledge.watson.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:28:50 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig601868FFD7E993E6262A8244 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Robert Watson wrote: > On Sun, 6 Jan 2008, Ivan Voras wrote: >> Last I heard, rsync didn't crash Solaris on ZFS :) >=20 > My admittedly second-hand understanding is that ZFS shows similarly=20 > gratuitous memory use on both Mac OS X and Solaris. One advantage=20 > Solaris has is that it runs primarily on expensive 64-bit servers with = > lots of memory. Part of the problem on FreeBSD is that people run ZFS = > on sytems with 32-bit CPUs and a lot less memory. It could be that ZFS= =20 > should be enforcing higher minimum hardware requirements to mount (i.e.= ,=20 > refusing to run on systems with 32-bit address spaces or <4gb of memory= =20 > and inadequate tuning). Solaris nowadays refuses to install on anything without at least 1 GB of = memory. I'm all for ZFS refusing to run on inadequatly tuned hardware,=20 but apparently there's no algorithmic way to say what *is* adequately=20 tuned, except for "try X and if it crashes, try Y, repeat as necessary". The reason why I'm arguing this topic is that it isn't a matter of=20 tuning like "it will run slowly if you don't tune it" - it's more like=20 "it won't run at all if you don't go through the laborious=20 trial-and-error process of tuning it, including patching your kernel and = running a non-GENERIC configuration". --------------enig601868FFD7E993E6262A8244 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgQ+/ldnAQVacBcgRAtnAAKCtMumii9wevIzasHr8NZ6x5aGQ2ACcDmBf Nqzn2r1T/d1ngFr8i4tyZHU= =PAma -----END PGP SIGNATURE----- --------------enig601868FFD7E993E6262A8244-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:34:58 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4833216A417; Sun, 6 Jan 2008 17:34:58 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C9BB813C442; Sun, 6 Jan 2008 17:34:54 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4781113C.3090904@FreeBSD.org> Date: Sun, 06 Jan 2008 18:34:52 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:34:58 -0000 Ivan Voras wrote: > What is the other 512 MB of the 1 GB used for? Everything else that the kernel needs address space for. Buffer cache, mbuf allocation, etc. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:36:42 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00FE916A421 for ; Sun, 6 Jan 2008 17:36:42 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5F01213C46E for ; Sun, 6 Jan 2008 17:36:41 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBZQJ-0001oS-1q for freebsd-current@freebsd.org; Sun, 06 Jan 2008 17:36:35 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:36:35 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 17:36:35 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 18:36:24 +0100 Lines: 54 Message-ID: References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCF28DF68F34EF7BB67EC6EA2" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <47810DE3.3050106@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:36:42 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCF28DF68F34EF7BB67EC6EA2 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > Ivan Voras wrote: >> Robert Watson wrote: >> >>> I'm not sure if anyone has mentioned this yet in the thread, but=20 >>> another thing worth taking into account in considering the stability = >>> of ZFS is whether or not Sun considers it a production feature in=20 >>> Solaris. Last I heard, it was still considered an experimental=20 >>> feature there as well. >> >> Last I heard, rsync didn't crash Solaris on ZFS :) >=20 > [Citation needed] I can't provide citation about a thing that doesn't happen - you don't=20 hear things like "oh and yesterday I ran rsync on my Solaris with ZFS=20 and *it didn't crash*!" often. But, with some grains of salt taken, consider this Google results: * searching for "rsync crash solaris zfs": 790 results, most of them=20 obviously irrelevant * searching for "rsync crash freebsd zfs": 10,800 results; a small=20 number of the results is from this thread, some are duplicates, but it's = a large number in any case. I feel that the number of Solaris+ZFS installations worldwide is larger=20 than that of FreeBSD+ZFS and they've had ZFS longer. --------------enigCF28DF68F34EF7BB67EC6EA2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgRGYldnAQVacBcgRAl22AKCNn8JJNdP7fYk3PNsnRhSGdwEn7gCfYUHc clThYOP6zkJD3wFZrFqFHXc= =Et/1 -----END PGP SIGNATURE----- --------------enigCF28DF68F34EF7BB67EC6EA2-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 17:43:27 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA76A16A418; Sun, 6 Jan 2008 17:43:27 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C427713C465; Sun, 6 Jan 2008 17:43:21 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47811336.4060605@FreeBSD.org> Date: Sun, 06 Jan 2008 18:43:18 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080106141157.I105@fledge.watson.org> <20080106170452.L105@fledge.watson.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 17:43:27 -0000 Ivan Voras wrote: > Robert Watson wrote: >> On Sun, 6 Jan 2008, Ivan Voras wrote: > >>> Last I heard, rsync didn't crash Solaris on ZFS :) >> >> My admittedly second-hand understanding is that ZFS shows similarly >> gratuitous memory use on both Mac OS X and Solaris. One advantage >> Solaris has is that it runs primarily on expensive 64-bit servers with >> lots of memory. Part of the problem on FreeBSD is that people run ZFS >> on sytems with 32-bit CPUs and a lot less memory. It could be that >> ZFS should be enforcing higher minimum hardware requirements to mount >> (i.e., refusing to run on systems with 32-bit address spaces or <4gb >> of memory and inadequate tuning). > > Solaris nowadays refuses to install on anything without at least 1 GB of > memory. I'm all for ZFS refusing to run on inadequatly tuned hardware, > but apparently there's no algorithmic way to say what *is* adequately > tuned, except for "try X and if it crashes, try Y, repeat as necessary". What you appear to be still missing is that ZFS also causes memory exhaustion panics when run on 32-bit Solaris. In fact (unless they have since fixed it), the opensolaris ZFS code makes *absolutely no attempt* to accomodate i386 memory limitations at all. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:00:32 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D419716A41A for ; Sun, 6 Jan 2008 18:00:32 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 969D213C467 for ; Sun, 6 Jan 2008 18:00:32 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBZnR-0003C2-2a for freebsd-current@freebsd.org; Sun, 06 Jan 2008 18:00:29 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:00:29 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:00:29 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 19:00:10 +0100 Lines: 28 Message-ID: References: <20080106141157.I105@fledge.watson.org> <20080106170452.L105@fledge.watson.org> <47811336.4060605@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig0A3831A65593ADD3330D3F5D" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <47811336.4060605@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:00:32 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig0A3831A65593ADD3330D3F5D Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > What you appear to be still missing is that ZFS also causes memory=20 > exhaustion panics when run on 32-bit Solaris.=20 Citation needed. I'm interested. --------------enig0A3831A65593ADD3330D3F5D Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgRcqldnAQVacBcgRAqiCAKDYA86boV1pvgRfvuZZtppnVuxWyACgq30c 0ZqL8GfJSoOGiutpQHA1S+U= =ZNrd -----END PGP SIGNATURE----- --------------enig0A3831A65593ADD3330D3F5D-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:09:50 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2908A16A417; Sun, 6 Jan 2008 18:09:50 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3647F13C43E; Sun, 6 Jan 2008 18:09:46 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47811967.3050906@FreeBSD.org> Date: Sun, 06 Jan 2008 19:09:43 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080106141157.I105@fledge.watson.org> <20080106170452.L105@fledge.watson.org> <47811336.4060605@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:09:50 -0000 Ivan Voras wrote: > Kris Kennaway wrote: > >> What you appear to be still missing is that ZFS also causes memory >> exhaustion panics when run on 32-bit Solaris. > > Citation needed. I'm interested. > Reports on the zfs-discuss mailing list. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:10:58 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4636716A418; Sun, 6 Jan 2008 18:10:58 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8859513C46A; Sun, 6 Jan 2008 18:10:54 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <478119AB.8050906@FreeBSD.org> Date: Sun, 06 Jan 2008 19:10:51 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:10:58 -0000 Ivan Voras wrote: > Kris Kennaway wrote: >> Ivan Voras wrote: >>> Robert Watson wrote: >>> >>>> I'm not sure if anyone has mentioned this yet in the thread, but >>>> another thing worth taking into account in considering the stability >>>> of ZFS is whether or not Sun considers it a production feature in >>>> Solaris. Last I heard, it was still considered an experimental >>>> feature there as well. >>> >>> Last I heard, rsync didn't crash Solaris on ZFS :) >> >> [Citation needed] > > I can't provide citation about a thing that doesn't happen - you don't > hear things like "oh and yesterday I ran rsync on my Solaris with ZFS > and *it didn't crash*!" often. > > But, with some grains of salt taken, consider this Google results: > > * searching for "rsync crash solaris zfs": 790 results, most of them > obviously irrelevant > * searching for "rsync crash freebsd zfs": 10,800 results; a small > number of the results is from this thread, some are duplicates, but it's > a large number in any case. > > I feel that the number of Solaris+ZFS installations worldwide is larger > than that of FreeBSD+ZFS and they've had ZFS longer. Almost all Solaris systems are 64 bit. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:26:19 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C08E16A468 for ; Sun, 6 Jan 2008 18:26:19 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id DEBF313C45D for ; Sun, 6 Jan 2008 18:26:18 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBaCJ-0004UM-8D for freebsd-current@freebsd.org; Sun, 06 Jan 2008 18:26:11 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:26:11 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:26:11 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 19:25:50 +0100 Lines: 37 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF2FA9CF3EA05E3277142F2EC" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) X-Enigmail-Version: 0.95.6 Sender: news Subject: ZFS on AMD64 - any recent crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:26:19 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF2FA9CF3EA05E3277142F2EC Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Hi, In light of the recent discussion about ZFS stability, does anyone still = have kmem_map_too_small panics on AMD64 after tuning it (as presented in = http://wiki.freebsd.org/ZFSTuningGuide)? I'm interested in reports like this:=20 http://kerneltrap.org/mailarchive/freebsd-current/2007/9/21/271557 (note = that this report is for an untuned system). --------------enigF2FA9CF3EA05E3277142F2EC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgR00ldnAQVacBcgRAsHuAJ9BcMeFX5lyB7gNUf3ztkKlPndhgwCgnf77 msaT78vTslNOzCv3sCVD/Kk= =GBVC -----END PGP SIGNATURE----- --------------enigF2FA9CF3EA05E3277142F2EC-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:30:54 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D4C416A417 for ; Sun, 6 Jan 2008 18:30:54 +0000 (UTC) (envelope-from pphajek@lbl.gov) Received: from india.jgi-psf.org (india.jgi-psf.org [198.129.90.139]) by mx1.freebsd.org (Postfix) with ESMTP id 2D92613C4E5 for ; Sun, 6 Jan 2008 18:30:54 +0000 (UTC) (envelope-from pphajek@lbl.gov) Received: from india.jgi-psf.org (localhost [127.0.0.1]) by india.jgi-psf.org (8.14.2/8.14.1) with ESMTP id m06HwsJH005889 for ; Sun, 6 Jan 2008 09:58:54 -0800 (PST) (envelope-from pphajek@lbl.gov) Received: (from phajek@localhost) by india.jgi-psf.org (8.14.2/8.14.1/Submit) id m06HwsAd005888 for freebsd-current@freebsd.org; Sun, 6 Jan 2008 09:58:54 -0800 (PST) (envelope-from pphajek@lbl.gov) X-Authentication-Warning: india.jgi-psf.org: phajek set sender to pphajek@lbl.gov using -f Date: Sun, 6 Jan 2008 09:58:54 -0800 From: Patrick Hajek To: freebsd-current@freebsd.org Message-ID: <20080106175854.GB5659@lbl.gov> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:30:54 -0000 > I'm not sure if anyone has mentioned this yet in the thread, but > another thing worth taking into account in considering the stability of ZFS is > whether or not Sun considers it a production feature in Solaris. Last I heard, > it was still considered an experimental feature there as well. It was not production ready for solaris until they included it the quarterly release of Solaris 10-- appox a year ago. -Patrick -- Patrick Hajek /"\ DOE Joint Genome Institute \ / ASCII RIBBON CAMPAIGN Desk: 925.296.5762 X HELP CURE HTML MAIL Cell: 925.997.4826 / \ PGP Fingerprint 688E B579 3449 28B1 DB14 E15A 76BB C1CA A745 9DAB From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:34:03 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7F0716A469 for ; Sun, 6 Jan 2008 18:34:03 +0000 (UTC) (envelope-from swhetzel@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.155]) by mx1.freebsd.org (Postfix) with ESMTP id 6789013C43E for ; Sun, 6 Jan 2008 18:34:03 +0000 (UTC) (envelope-from swhetzel@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so4855862fgg.35 for ; Sun, 06 Jan 2008 10:34:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=euuFTvI3oxrbGPC95tT98YmTxdbAnDF7u47AhXIrjOM=; b=F52QjNPyyHj+pYIu/IyZNgm3tlxo2GdjCsv+ezafOwLHubMUKwzrrvefrhCXo2DeNxTZ0rbwOX4qKg6mqiuMq+2MQeJugRnmFLdzd4Z2a3nK8LXuoBkHCKiOgq9w9ZYVjiJ42qBkSDTVFkf7GeprZ7LyX0A/wRX91ekE2YBgyTs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=SYGgWBHpgXnWCHb+Gn0cWnlyIdkQ/rxgv93umHnWmc3oeIaYhaxjr2nNUi+Aot+w6bLbXAh0V6LZYOlHH9mm2j86i4jtuWFOCCZq36k629CW6TGzBAR0o8MU14JHw1jdAtrmfXLdOqYeaYlp3yTtwMAzmg/rtVcg+lI7YDXzT3k= Received: by 10.86.66.1 with SMTP id o1mr19246461fga.36.1199644441034; Sun, 06 Jan 2008 10:34:01 -0800 (PST) Received: by 10.86.3.20 with HTTP; Sun, 6 Jan 2008 10:34:00 -0800 (PST) Message-ID: <790a9fff0801061034x79c6eec0sc6220247b3cb76fd@mail.gmail.com> Date: Sun, 6 Jan 2008 12:34:00 -0600 From: "Scot Hetzel" To: Darran In-Reply-To: <003401c85059$581790f0$6501a8c0@Vostro> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <319b588b0801060139m5e99e857of3a8f44ab3d2e078@mail.gmail.com> <003401c85059$581790f0$6501a8c0@Vostro> Cc: freebsd-current@freebsd.org Subject: Re: Hung laptop now will not boot 7.0-RC1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:34:03 -0000 On 1/6/08, Darran wrote: > Hello all, > > I downloaded, burnt and installed 7.0-RC1 (all torrent) on amd64 > I downloaded the dell windows drivers for my laptop wireless card (which > worked on i3686 6.2-RELEASE) and used ndisgen to create the .ko file > (bcmwl5_sys.ko) and loaded it and then realised I had created the 32 bit > driver so I re-did it and used the 64 bit file and loaded the 64bit .ko and > at that point my laptop (Dell Vostro 1000) hung. > I had put bcmwl5_sys_load="YES" in the loader.conf file > I rebooted and now I don't get anything. As soon as it starts to boot it > stops. > There is a screen showing kernel dump info and I can copy it if needed but I > have to burn the rescue iso otherwise I cant boot. > > Is this a known problem ? > Does anyone have any advice ? > If you have bcmwl5_sys_load in your loader.conf, this is causing the 32bit driver to load at boot, bcmwl563_sys_load needs to be placed in loader.conf for the 64 bit driver. To boot your system without the rescue CD, at the boot menu choose "Escape to loader prompt". Then use "unload" to unload all modules, and "unset bcmwl5_sys" to disable loading of the bcmwl5_sys module. Finally, use "boot" to continue booting. When the system has finished booting, check /boot/loader.conf, does it have bcmwl5_sys_load or bcmwl564_sys_load? Also check that you placed the 64bit driver in your module path, and removed the 32bit driver from the module path. Scot From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 18:49:35 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59B5E16A418 for ; Sun, 6 Jan 2008 18:49:35 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id E1AAD13C457 for ; Sun, 6 Jan 2008 18:49:34 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBaYv-0005gn-8q for freebsd-current@freebsd.org; Sun, 06 Jan 2008 18:49:33 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:49:33 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 18:49:33 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 19:49:17 +0100 Lines: 46 Message-ID: References: <20080106141157.I105@fledge.watson.org> <20080106170452.L105@fledge.watson.org> <47811336.4060605@FreeBSD.org> <47811967.3050906@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig3EB635B71F4C09B8E6DDE3BC" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <47811967.3050906@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 18:49:35 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig3EB635B71F4C09B8E6DDE3BC Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > Ivan Voras wrote: >> Kris Kennaway wrote: >> >>> What you appear to be still missing is that ZFS also causes memory=20 >>> exhaustion panics when run on 32-bit Solaris.=20 >> >> Citation needed. I'm interested. >=20 > Reports on the zfs-discuss mailing list. Thanks for the pointer. I'm looking at the archives. So far I've found this:=20 http://www.archivum.info/zfs-discuss@opensolaris.org/2007-07/msg00016.htm= l=20 which doesn't mention panics; and this:=20 http://www.archivum.info/zfs-discuss@opensolaris.org/2007-07/msg00054.htm= l=20 which didn't get any replies but the backtrace doesn't include anything=20 resembling a malloc-like call. --------------enig3EB635B71F4C09B8E6DDE3BC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgSKzldnAQVacBcgRAkGmAKCDYOMbWXcSSaJtuWo7YS8MjSNlcwCePKAq 9q/fY9HjXfbs5Q7jXAgBjuM= =W2hT -----END PGP SIGNATURE----- --------------enig3EB635B71F4C09B8E6DDE3BC-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 19:07:33 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCD6116A419; Sun, 6 Jan 2008 19:07:32 +0000 (UTC) (envelope-from gcorcoran@rcn.com) Received: from smtp02.lnh.mail.rcn.net (smtp02.lnh.mail.rcn.net [207.172.157.102]) by mx1.freebsd.org (Postfix) with ESMTP id 6250A13C442; Sun, 6 Jan 2008 19:07:32 +0000 (UTC) (envelope-from gcorcoran@rcn.com) Received: from mr08.lnh.mail.rcn.net ([207.172.157.28]) by smtp02.lnh.mail.rcn.net with ESMTP; 06 Jan 2008 13:38:44 -0500 Received: from smtp01.lnh.mail.rcn.net (smtp01.lnh.mail.rcn.net [207.172.4.11]) by mr08.lnh.mail.rcn.net (MOS 3.8.6-GA) with ESMTP id JNE00144; Sun, 6 Jan 2008 13:38:44 -0500 (EST) Received: from 207-172-55-230.c3-0.tlg-ubr5.atw-tlg.pa.cable.rcn.com (HELO [10.56.78.161]) ([207.172.55.230]) by smtp01.lnh.mail.rcn.net with ESMTP; 06 Jan 2008 13:37:42 -0500 Message-ID: <4781227B.5020800@rcn.com> Date: Sun, 06 Jan 2008 13:48:27 -0500 From: Gary Corcoran User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Kris Kennaway References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> In-Reply-To: <47810BE3.4080601@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Junkmail-Whitelist: YES (by domain whitelist at mr08.lnh.mail.rcn.net) Cc: freebsd-current@freebsd.org, Ivan Voras Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 19:07:33 -0000 Kris Kennaway wrote: > Ivan Voras wrote: >> Kris Kennaway wrote: >> >>> No, clearly it is not enough >> >> This looks like we're constantly chasing the "right amount". Does it >> depend so much on CPU and IO speed that there's never a generally >> sufficient "right amount"? So when CPU and drive speed increase, the >> new amount will always be some bigger value? > > It depends on your workload, which in turn depends on your hardware. The > harder you can drive ZFS the more memory it will require. As a user, I would expect the above to mean "to continue running quickly". If it has to slow to a crawl for a moment, due to inadequate memory in your system, then that's just tough cookies. But crashing (panicing) is not really acceptable for most people (maybe except a developer). Again from a user perspective, if ZFS needs "tuning" to run at full speed, or even at all, I would expect *it* to be able to do a few simple calculations and do the tuning itself! :-) (even if, in worst case, it requires a clean shutdown and reboot for the new values to take effect) The above is not meant as a criticism of the current explicitly-labeled "experimental" code. Rather, it is what I would hope we might be able to see sometime over the next year... >>> (and you claimed previously to have done more tuning than this). >> >> Where? What else is there except kmem tuning (including KVA_PAGES)? >> IIRC Pawel said all other suggested tunings don't do much. > > Tuning is an interactive process. If 512MB is not enough kmem_map, then > increase it. Repeat as necessary. > >>> I have it set to 600MB on the i386 system with a 1.5GB KVA. Both >>> were necessary. >> >> My point is that the fact that such things are necessary (1.5 GB KVA >> os a lot on i386) mean that there are serious problems which aren't >> getting fixed since ZFS was imported (that's over 6 months ago). > > ZFS is a memory hog. There is nothing that can really be done about > this, and it is just not a good fit on i386 because of limitations of > the hardware architecture. Note that Sun does not recommend using ZFS > on a 32-bit system either, for the same reasons. It is unlikely this > can really be fixed, although mitigation strategies like the vm_kern.c > patch are possible. Perhaps the 7.0 release notes should include a note to the effect that ZFS is *strongly* NOT RECOMMENDED on 32-bit systems at this time, due to the likelihood of panics. I say this because it sure sounds like "out of the box" that is what you're most likely to end up with, and even with manual "corrections" you may still have panics. So why not just be upfront about it and tell people that, at least at this time, ZFS is only recommended for 64-bit systems, with a minimum of "N" (2?) GB of memory? If you were already planning something like this for the release notes, my apologies. >> I see you've added to http://wiki.freebsd.org/ZFSTuningGuide; can you >> please add the values that work for you to it (especially for >> KVA_PAGES since the exact kernel configuration line is never spelled >> out in the document; and say for which hardware are the values known >> to work)? > > OK. > >>> ZFS already tells you up front that it's experimental code and likely >>> to have problems. >> >> I know it's experimental, but requiring users to perform so much >> tuning just to get it work without crashing will mean it will get a >> bad reputation early on. Do you (or anyone) know what are the reasons >> for not having vm.kmem_size to 512 MB by default? > > Increasing vm.kmem_size.max to 512MB by default has other implications, > but it is something that should be considered. > > > Better yet, why not >> increase both vm.kmem_size and KVA_PAGES to (the equivalent of) 640 MB >> or 768 MB by default for 7.0? > > That is answered in the tuning guide. Tuning KVA_PAGES by default is > not appropriate. > >> >Users of 7.0-RELEASE should not have unrealistic >> > expectations. >> >> As I've said at the first post of this thread: I'm interested in if >> it's ever going to be stable for 7.x. > > This was in reply to a comment you made about the vm_kern.c patch > affecting users of 7.0-RELEASE. > > Anyway, to sum up, ZFS has known bugs, some of which are unresolved by > the authors, and it is difficult to make it work on i386. It is likely > that the bugs will be fixed over time (obviously), but amd64 will always > be a better choice than i386 for using ZFS because you will not be > continually bumping up against the hardware limitations. BTW, I am a happy user of ZFS on a 2GB Core2Duo 64-bit system. I never did any "tuning", it "just worked" for my light-duty file serving needs. This was from the (I believe) May 2007 snapshot. Gary From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 19:56:32 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 991F016A46B for ; Sun, 6 Jan 2008 19:56:32 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: from 30.mail-out.ovh.net (30.mail-out.ovh.net [213.186.62.213]) by mx1.freebsd.org (Postfix) with SMTP id 1B41013C442 for ; Sun, 6 Jan 2008 19:56:31 +0000 (UTC) (envelope-from maciej@suszko.eu) Received: (qmail 31368 invoked by uid 503); 6 Jan 2008 19:56:50 -0000 Received: (QMFILT: 1.0); 06 Jan 2008 19:56:50 -0000 Received: from unknown (HELO mail139.ha.ovh.net) (213.186.33.59) by 30.mail-out.ovh.net with SMTP; 6 Jan 2008 19:56:50 -0000 Received: from b0.ovh.net (HELO queue-out) (213.186.33.50) by b0.ovh.net with SMTP; 6 Jan 2008 19:56:32 -0000 Received: from 217-153-241-141.zab.nat.hnet.pl (HELO arsenic) (maciej@suszko.eu@217.153.241.141) by ns0.ovh.net with SMTP; 6 Jan 2008 19:56:30 -0000 Date: Sun, 6 Jan 2008 20:56:23 +0100 From: Maciej Suszko To: freebsd-current@freebsd.org Message-Id: <20080106205623.d1024ae6.maciej@suszko.eu> In-Reply-To: <4781002F.5020708@FreeBSD.org> References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <20080106144627.a91a62c1.maciej@suszko.eu> <4780F7C0.5010101@FreeBSD.org> <20080106170544.93f7ab1b.maciej@suszko.eu> <4781002F.5020708@FreeBSD.org> X-Mailer: Sylpheed 2.4.8 (GTK+ 2.12.3; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Ovh-Remote: 217.153.241.141 (217-153-241-141.zab.nat.hnet.pl) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-Spam-Check: DONE|H 0.500019/N Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 19:56:32 -0000 Kris Kennaway wrote: > Maciej Suszko wrote: > > Kris Kennaway wrote: > >> Maciej Suszko wrote: > >>> Kris Kennaway wrote: > >>>> Ivan Voras wrote: > >>>>> On 06/01/2008, Peter Schuller > >>>>> wrote: > >>>>>>> This number is not so large. It seems to be easily crashed by > >>>>>>> rsync, for example (speaking from my own experience, and also > >>>>>>> some of my colleagues). > >>>>>> I can definitely say this is not *generally* true, as I do a > >>>>>> lot of rsyncing/rdiff-backup:ing and similar stuff (with many > >>>>>> files / large files) on ZFS without any stability issues. > >>>>>> Problems for me have been limited to 32bit and the memory > >>>>>> exhaustion issue rather than "hard" issues. > >>>>> It's not generally true since kmem problems with rsync are often > >>>>> hard to repeat - I have them on one machine, but not on another, > >>>>> similar machine. This nonrepeatability is also a part of the > >>>>> problem. > >>>>> > >>>>>> But perhaps that's all you are referring to. > >>>>> Mostly. I did have a ZFS crash with rsync that wasn't kmem > >>>>> related, but only once. > >>>> kmem problems are just tuning. They are not indicative of > >>>> stability problems in ZFS. Please report any further non-kmem > >>>> panics you experience. > >>> I agree that ZFS is pretty stable itself. I use 32bit machine with > >>> 2gigs od RAM and all hang cases are kmem related, but the fact is > >>> that I haven't found any way of tuning to stop it crashing. When I > >>> do some rsyncing, especially beetwen different pools - it hangs or > >>> reboots - mostly on bigger files (i.e. rsyncing ports tree with > >>> distfiles). At the moment I patched the kernel with > >>> vm_kern.c.2.patch and it just stopped crashing, but from time to > >>> time the machine looks like beeing freezed for a second or two, > >>> after that it works normally. Have you got any similar experience? > >> That is expected. That patch makes the system do more work to try > >> and reclaim memory when it would previously have panicked from lack > >> of memory. However, the same advice applies as to Ivan: you should > >> try and tune the memory parameters better to avoid this last-ditch > >> sitation. > > > > As Ivan said - tuning kmem_size only delay the moment system crash, > > earlier or after it happens - that's my point of view. > > So the same question applies: exactly what steps did you take to tune > the memory parameters? Extracting this information from you guys > shouldn't be as hard as this :) I was playing around with kmem_max_size mainly. I suppose messing up with KVA_PAGES is not a good idea unless you exactly know how much memory you software consume... -- regards, Maciej Suszko. From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 20:25:32 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B15B916A418 for ; Sun, 6 Jan 2008 20:25:32 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.244]) by mx1.freebsd.org (Postfix) with ESMTP id 7D4CD13C467 for ; Sun, 6 Jan 2008 20:25:32 +0000 (UTC) (envelope-from kometen@gmail.com) Received: by an-out-0708.google.com with SMTP id c14so1545512anc.13 for ; Sun, 06 Jan 2008 12:25:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=8Wt3RV0dR3Gd/DRYltjriEImDVmjU6DYIoQpGYiMemU=; b=uMDXwH9Z6LT39DrCO0x2aXIAOCCKsXOUxZqk4QKhK07y9tTlmm4cpXDzIHS/Xpk36pzqY993ywAg88ehX0OxBCAyzK51a+NrUWvw5SkU1cPwBFIKHj+tuCd+TR25vLMgvL13faA7CErLP1uYzPepN0nAM5+QprP2p9Mffs8hgjo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=B2PTTqTdakQH4bAN35pRuFZPOnDdnC00QiXKlc89/rsvJDedGJ7W6JPQrqnUq8Sh1Ai5KNPcvolyXPkR/sb4z4dGH9j7mJ4SgKoHT3kziR+l+8xS0tdy/rSt9Zq/h6wteCfmximkqgahSx67Lq/PkKbvcjTkxNDK50LphXDijIA= Received: by 10.100.128.20 with SMTP id a20mr17442485and.109.1199649635643; Sun, 06 Jan 2008 12:00:35 -0800 (PST) Received: by 10.100.242.10 with HTTP; Sun, 6 Jan 2008 12:00:35 -0800 (PST) Message-ID: Date: Sun, 6 Jan 2008 21:00:35 +0100 From: "Claus Guttesen" To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 20:25:32 -0000 > >> Last I heard, rsync didn't crash Solaris on ZFS :) > > > > [Citation needed] > > I can't provide citation about a thing that doesn't happen - you don't > hear things like "oh and yesterday I ran rsync on my Solaris with ZFS > and *it didn't crash*!" often. > > But, with some grains of salt taken, consider this Google results: > > * searching for "rsync crash solaris zfs": 790 results, most of them > obviously irrelevant > * searching for "rsync crash freebsd zfs": 10,800 results; a small > number of the results is from this thread, some are duplicates, but it's > a large number in any case. > > I feel that the number of Solaris+ZFS installations worldwide is larger > than that of FreeBSD+ZFS and they've had ZFS longer. I used zfs on FreeBSD current amd64 around summer 2006 as a samba-server for internal use on a dual xeon (first generation 64-bit, somewhat slow and hot) with 4 GB ram and two qlogic hba's attached to approx. 8 TB of storage. I did not once experience any kernel panic or other unplanned stop. But I whenever I manually mounted a smbfs-share the terminal would not return to the command line. I upgraded in october 2007 and the smbfs-mount returned to the command line and I thought I was happy. Until I started to get the kmem_map too small kernel-panics when doing much I/O (syncing 40 GB of small files). I tuned the values as indicated in the zfs tuning guide and rebooted and increased the values as the kernel panics persisted. When I increased the values even more I ended up with a kernel which refused to boot, boy I was almost getting a panic myself :-) Applying Pawel's patch did make the server survive two or three 40 GB rsyncing so the patch did help. But we were approching xmas season which is a very critical time for us so I migrated to solaris 10. The solaris server has had no downtime but to conclude that solaris is more stable in my situation is premature. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 20:30:36 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 871FE16A417; Sun, 6 Jan 2008 20:30:36 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from smtp.infidyne.com (ds9.infidyne.com [88.80.6.206]) by mx1.freebsd.org (Postfix) with ESMTP id 541B613C447; Sun, 6 Jan 2008 20:30:35 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from c-8216e555.03-51-73746f3.cust.bredbandsbolaget.se (c-8216e555.03-51-73746f3.cust.bredbandsbolaget.se [85.229.22.130]) by smtp.infidyne.com (Postfix) with ESMTP id F07FD770D4; Sun, 6 Jan 2008 21:30:33 +0100 (CET) From: Peter Schuller To: freebsd-current@freebsd.org Date: Sun, 6 Jan 2008 21:30:32 +0100 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart5240114.PTpqoRGpdc"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200801062130.40755.peter.schuller@infidyne.com> Cc: Ivan Voras Subject: Re: ZFS on AMD64 - any recent crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 20:30:36 -0000 --nextPart5240114.PTpqoRGpdc Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline > In light of the recent discussion about ZFS stability, does anyone still > have kmem_map_too_small panics on AMD64 after tuning it (as presented in > http://wiki.freebsd.org/ZFSTuningGuide)? Not sure if you wanted negatives, but I have not seen such crashes on any o= f=20 the three amd64 systems I'm running ZFS on (3 gb, 4 gb and 4 gb of RAM). Th= e=20 only tuning done is to increase the kmem size and arc_max and disabling=20 prefetch. The increase in kmem was not to avoid crashes, but was to=20 accomodate the larger arc_max chosen. =2D-=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --nextPart5240114.PTpqoRGpdc Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHgTpwDNor2+l1i30RAkhCAJ90OJp+jDb5RLy9xbn7gahBx0DBjgCeMTGb sZSjzxZgvaEme7z8yFQfJhQ= =4Xpl -----END PGP SIGNATURE----- --nextPart5240114.PTpqoRGpdc-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:01:07 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA1CC16A41B for ; Sun, 6 Jan 2008 21:01:07 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 62ACD13C455 for ; Sun, 6 Jan 2008 21:01:07 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JBccB-0003yF-R3 for freebsd-current@freebsd.org; Sun, 06 Jan 2008 21:01:03 +0000 Received: from 89-172-37-126.adsl.net.t-com.hr ([89.172.37.126]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 21:01:03 +0000 Received: from ivoras by 89-172-37-126.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Jan 2008 21:01:03 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Sun, 06 Jan 2008 22:00:47 +0100 Lines: 46 Message-ID: References: <200801062130.40755.peter.schuller@infidyne.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5A55C2134572116F91A85FF5" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-37-126.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <200801062130.40755.peter.schuller@infidyne.com> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: ZFS on AMD64 - any recent crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:01:07 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5A55C2134572116F91A85FF5 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Peter Schuller wrote: >> In light of the recent discussion about ZFS stability, does anyone sti= ll >> have kmem_map_too_small panics on AMD64 after tuning it (as presented = in >> http://wiki.freebsd.org/ZFSTuningGuide)? >=20 > Not sure if you wanted negatives, but I have not seen such crashes on a= ny of=20 All reports are welcome :) > the three amd64 systems I'm running ZFS on (3 gb, 4 gb and 4 gb of RAM)= =2E The=20 > only tuning done is to increase the kmem size and arc_max and disabling= =20 > prefetch. The increase in kmem was not to avoid crashes, but was to=20 > accomodate the larger arc_max chosen. This is somewhat a special case - you also did tuning besides kmem size. = I'm especially interested in this, hoping to gather the "combination=20 that works" from the reports. --------------enig5A55C2134572116F91A85FF5 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgUGFldnAQVacBcgRAo9eAKDcTZ82c+08ijQwHhqI6tjp3Oc2oACg+0Yr Rqf+7GZZVMSm7o/llxHdJsA= =EO+h -----END PGP SIGNATURE----- --------------enig5A55C2134572116F91A85FF5-- From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:20:52 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C286516A421; Sun, 6 Jan 2008 21:20:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 7E80513C448; Sun, 6 Jan 2008 21:20:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m06L0GB7096862; Sun, 6 Jan 2008 14:00:17 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <47814160.4050401@samsco.org> Date: Sun, 06 Jan 2008 14:00:16 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.11) Gecko/20071128 SeaMonkey/1.1.7 MIME-Version: 1.0 To: Kris Kennaway References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> <478119AB.8050906@FreeBSD.org> In-Reply-To: <478119AB.8050906@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Sun, 06 Jan 2008 14:00:17 -0700 (MST) X-Spam-Status: No, score=-1.4 required=5.4 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-current@freebsd.org, Ivan Voras Subject: ZFS honesty X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:20:52 -0000 Kris Kennaway wrote: > Ivan Voras wrote: >> Kris Kennaway wrote: >>> Ivan Voras wrote: >>>> Robert Watson wrote: >>>> >>>>> I'm not sure if anyone has mentioned this yet in the thread, but >>>>> another thing worth taking into account in considering the >>>>> stability of ZFS is whether or not Sun considers it a production >>>>> feature in Solaris. Last I heard, it was still considered an >>>>> experimental feature there as well. >>>> >>>> Last I heard, rsync didn't crash Solaris on ZFS :) >>> >>> [Citation needed] >> >> I can't provide citation about a thing that doesn't happen - you don't >> hear things like "oh and yesterday I ran rsync on my Solaris with ZFS >> and *it didn't crash*!" often. >> >> But, with some grains of salt taken, consider this Google results: >> >> * searching for "rsync crash solaris zfs": 790 results, most of them >> obviously irrelevant >> * searching for "rsync crash freebsd zfs": 10,800 results; a small >> number of the results is from this thread, some are duplicates, but >> it's a large number in any case. >> >> I feel that the number of Solaris+ZFS installations worldwide is >> larger than that of FreeBSD+ZFS and they've had ZFS longer. > > Almost all Solaris systems are 64 bit. > > Kris So, let's be honest here. ZFS is simply unreliable on FreeBSD/i386. There are things that you can do mitigate the problems, and in certain well controlled environments you might be able to make it work well enough for your needs. But as a general rule, don't expect it to work reliably, period. This is backed up by Sun's own recommendation to not run it on 32-bit Solaris. But let's also be honest about ZFS in the 64-bit world. There is ample evidence that ZFS basically wants to grow unbounded in proportion to the workload that you give it. Indeed, even Sun recommends basically throwing more RAM at most problems. Again, tuning is often needed, and I think it's fair to say that it can't be expected to work on arbitrary workloads out of the box. Now, what about the other problems that have been reported in this thread by Ivan and others? I don't think that it can be said that the only problem that ZFS has is with memory. Unfortunately, it looks like these "other" problems aren't well quantified, so I think that they are being unfairly dismissed. But at the same time, maybe these other problems are rare and unique enough that they represent very special cases that won't be encountered by most people. But it also tells me that ZFS is still immature, at least in FreeBSD. The universal need for tuning combined with the poorly understood problem reports tells me that administrators considering ZFS should expect to spend a fair amount of timing testing and tuning. Don't expect it to work out of the box for your situation. That's not to say that it's useless; there are certainly many people who can attest to it working well for them. Just be prepared to spend time and possibly money making it work, and be willing to provide good problem reports for any non-memory related problems that you encounter. Scott From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 20:57:05 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D270B16A417 for ; Sun, 6 Jan 2008 20:57:05 +0000 (UTC) (envelope-from vadimnuclight@tpu.ru) Received: from relay1.tpu.ru (relay1.tpu.ru [213.183.112.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3C8EA13C448 for ; Sun, 6 Jan 2008 20:57:05 +0000 (UTC) (envelope-from vadimnuclight@tpu.ru) Received: from localhost (localhost.localdomain [127.0.0.1]) by relay1.tpu.ru (Postfix) with ESMTP id 7F595104816; Mon, 7 Jan 2008 02:57:02 +0600 (NOVT) X-Virus-Scanned: amavisd-new at tpu.ru Received: from relay1.tpu.ru ([127.0.0.1]) by localhost (relay1.tpu.ru [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3VYCAciAu0sG; Mon, 7 Jan 2008 02:56:59 +0600 (NOVT) Received: from mail.main.tpu.ru (mail.main.tpu.ru [10.0.0.3]) by relay1.tpu.ru (Postfix) with ESMTP id 5834010478C; Mon, 7 Jan 2008 02:56:59 +0600 (NOVT) Received: from mail.tpu.ru ([213.183.112.105]) by mail.main.tpu.ru with Microsoft SMTPSVC(6.0.3790.3959); Mon, 7 Jan 2008 02:56:58 +0600 Received: from nuclight.avtf.net ([78.140.2.250]) by mail.tpu.ru over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Mon, 7 Jan 2008 02:56:57 +0600 Date: Mon, 07 Jan 2008 02:56:53 +0600 To: "Kris Kennaway" References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> <4781113C.3090904@FreeBSD.org> From: "Vadim Goncharov" Organization: AVTF TPU Hostel Content-Type: text/plain; format=flowed; delsp=yes; charset=koi8-r MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID: In-Reply-To: <4781113C.3090904@FreeBSD.org> User-Agent: Opera M2/7.54 (Win32, build 3865) X-OriginalArrivalTime: 06 Jan 2008 20:56:57.0661 (UTC) FILETIME=[ACF54ED0:01C850A6] X-Mailman-Approved-At: Sun, 06 Jan 2008 21:26:26 +0000 Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 20:57:05 -0000 06.01.08 @ 23:34 Kris Kennaway wrote: >> What is the other 512 MB of the 1 GB used for? > > Everything else that the kernel needs address space for. Buffer cache, > mbuf allocation, etc. Aren't they allocated from the same memory zones? I have a router with 256 Mb RAM, it had a panic with ng_nat once due to exhausted kmem. So, what these number from it's sysctl do really mean? vm.kmem_size: 83415040 vm.kmem_size_max: 335544320 vm.kmem_size_scale: 3 vm.kvm_size: 1073737728 vm.kvm_free: 704638976 -- WBR, Vadim Goncharov From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:32:47 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D4F5516A419; Sun, 6 Jan 2008 21:32:47 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B703913C4D9; Sun, 6 Jan 2008 21:32:46 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <478148FD.20605@FreeBSD.org> Date: Sun, 06 Jan 2008 22:32:45 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Scott Long References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> <478119AB.8050906@FreeBSD.org> <47814160.4050401@samsco.org> In-Reply-To: <47814160.4050401@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Ivan Voras Subject: Re: ZFS honesty X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:32:48 -0000 Scott Long wrote: > Kris Kennaway wrote: >> Ivan Voras wrote: >>> Kris Kennaway wrote: >>>> Ivan Voras wrote: >>>>> Robert Watson wrote: >>>>> >>>>>> I'm not sure if anyone has mentioned this yet in the thread, but >>>>>> another thing worth taking into account in considering the >>>>>> stability of ZFS is whether or not Sun considers it a production >>>>>> feature in Solaris. Last I heard, it was still considered an >>>>>> experimental feature there as well. >>>>> >>>>> Last I heard, rsync didn't crash Solaris on ZFS :) >>>> >>>> [Citation needed] >>> >>> I can't provide citation about a thing that doesn't happen - you >>> don't hear things like "oh and yesterday I ran rsync on my Solaris >>> with ZFS and *it didn't crash*!" often. >>> >>> But, with some grains of salt taken, consider this Google results: >>> >>> * searching for "rsync crash solaris zfs": 790 results, most of them >>> obviously irrelevant >>> * searching for "rsync crash freebsd zfs": 10,800 results; a small >>> number of the results is from this thread, some are duplicates, but >>> it's a large number in any case. >>> >>> I feel that the number of Solaris+ZFS installations worldwide is >>> larger than that of FreeBSD+ZFS and they've had ZFS longer. >> >> Almost all Solaris systems are 64 bit. >> >> Kris > > So, let's be honest here. ZFS is simply unreliable on FreeBSD/i386. > There are things that you can do mitigate the problems, and in certain > well controlled environments you might be able to make it work well > enough for your needs. But as a general rule, don't expect it to work > reliably, period. This is backed up by Sun's own recommendation to not > run it on 32-bit Solaris. > > But let's also be honest about ZFS in the 64-bit world. There is ample > evidence that ZFS basically wants to grow unbounded in proportion to the > workload that you give it. Indeed, even Sun recommends basically > throwing more RAM at most problems. Again, tuning is often needed, and > I think it's fair to say that it can't be expected to work on arbitrary > workloads out of the box. > > Now, what about the other problems that have been reported in this > thread by Ivan and others? I don't think that it can be said that the > only problem that ZFS has is with memory. Unfortunately, it looks like > these "other" problems aren't well quantified, so I think that they are > being unfairly dismissed. But at the same time, maybe these other > problems are rare and unique enough that they represent very special > cases that won't be encountered by most people. But it also tells me > that ZFS is still immature, at least in FreeBSD. > > The universal need for tuning combined with the poorly understood > problem reports tells me that administrators considering ZFS should > expect to spend a fair amount of timing testing and tuning. Don't > expect it to work out of the box for your situation. That's not to > say that it's useless; there are certainly many people who can attest to > it working well for them. Just be prepared to spend time and possibly > money making it work, and be willing to provide good problem reports for > any non-memory related problems that you encounter. To be clear, in this thread I have been mostly restricting myself to discussion of kmem problems only, although I have also noted that there are known ZFS bugs including bugs that are unfixed even in solaris (the ZIL low memory deadlock is one of them). Indeed, pjd has a long list of bug reports from me :) I agree with the rest of this summary. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:33:18 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A20516A421 for ; Sun, 6 Jan 2008 21:33:18 +0000 (UTC) (envelope-from morten@lightworkings.dk) Received: from mail.tobocom.net (mail.tobocom.net [89.221.166.157]) by mx1.freebsd.org (Postfix) with SMTP id 9984A13C4EE for ; Sun, 6 Jan 2008 21:33:15 +0000 (UTC) (envelope-from morten@lightworkings.dk) Received: from atlantis.local ([85.80.153.38]) by mail.tobocom.net with hMailServer ; Sun, 6 Jan 2008 22:33:09 +0100 Message-ID: <47814912.8090704@lightworkings.dk> Date: Sun, 06 Jan 2008 22:33:06 +0100 From: =?UTF-8?B?TW9ydGVuIFN0csOlcnVw?= User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: freebsd-current@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: ZFS on AMD64 - any recent crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:33:18 -0000 Ivan Voras wrote: > Hi, > > In light of the recent discussion about ZFS stability, does anyone > still have kmem_map_too_small panics on AMD64 after tuning it (as > presented in http://wiki.freebsd.org/ZFSTuningGuide)? > > I'm interested in reports like this: > http://kerneltrap.org/mailarchive/freebsd-current/2007/9/21/271557 > (note that this report is for an untuned system). > > > > Yes, I've had one such crash. The machine has 4GB of RAM and initially I set it up with vm.kmem_size and vm.kmem_size_max set to 1024M. It's my home fileserver and I'm the only regular user. After booting the box I had tested the transfer speed back and forth over the net and after that not really used it any more that day. The next day I discovered that it had crashed sometime during the night. I've since changed the two tuneables above to 1536M and I've not had any crashes since. I also tried with 2048M but that only resulted in a crash upon boot. Kind regards Morten Strårup From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:42:45 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CFFD16A420 for ; Sun, 6 Jan 2008 21:42:45 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (pointyhat.freebsd.org [IPv6:2001:4f8:fff6::2b]) by mx1.freebsd.org (Postfix) with ESMTP id C224613C474; Sun, 6 Jan 2008 21:42:44 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47814B53.50405@FreeBSD.org> Date: Sun, 06 Jan 2008 22:42:43 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Vadim Goncharov References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> <4781113C.3090904@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: When will ZFS become stable? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:42:45 -0000 Vadim Goncharov wrote: > 06.01.08 @ 23:34 Kris Kennaway wrote: > >>> What is the other 512 MB of the 1 GB used for? >> >> Everything else that the kernel needs address space for. Buffer >> cache, mbuf allocation, etc. > > Aren't they allocated from the same memory zones? I have a router with > 256 Mb RAM, it had a panic with ng_nat once due to exhausted kmem. So, > what these number from it's sysctl do really mean? > > vm.kmem_size: 83415040 > vm.kmem_size_max: 335544320 > vm.kmem_size_scale: 3 > vm.kvm_size: 1073737728 > vm.kvm_free: 704638976 > I believe that mbufs are allocated from a separate map. In your case you only have ~80MB available in your kmem_map, which is used for malloc() in the kernel. It is possible that ng_nat in combination with the other kernel malloc usage exhausted this relatively small amount of space without mbuf use being a factor. Kris From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:51:16 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B7C116A46C for ; Sun, 6 Jan 2008 21:51:16 +0000 (UTC) (envelope-from andrew-freebsd@areilly.bpc-users.org) Received: from qsrv03sl.mx.bigpond.com (qsrv03sl.mx.bigpond.com [144.140.92.183]) by mx1.freebsd.org (Postfix) with ESMTP id 36E8613C4EC for ; Sun, 6 Jan 2008 21:51:16 +0000 (UTC) (envelope-from andrew-freebsd@areilly.bpc-users.org) Received: from oaamta02sl.mx.bigpond.com ([124.188.162.219]) by omta04sl.mx.bigpond.com with ESMTP id <20080106204241.ZYTK6742.omta04sl.mx.bigpond.com@oaamta02sl.mx.bigpond.com> for ; Sun, 6 Jan 2008 20:42:41 +0000 Received: from areilly.bpa.nu ([124.188.162.219]) by oaamta02sl.mx.bigpond.com with ESMTP id <20080106204241.VHGW25607.oaamta02sl.mx.bigpond.com@areilly.bpa.nu> for ; Sun, 6 Jan 2008 20:42:41 +0000 Received: (qmail 32906 invoked from network); 6 Jan 2008 20:42:09 -0000 Received: from localhost (HELO duncan.reilly.home) (127.0.0.1) by localhost with SMTP; 6 Jan 2008 20:42:09 -0000 Date: Mon, 7 Jan 2008 07:42:09 +1100 From: Andrew Reilly To: Danny Braniss Message-ID: <20080107074209.5c20f083@duncan.reilly.home> In-Reply-To: References: <477C82F0.5060809@freebsd.org> <863ateemw2.fsf@ds4.des.no> <200801032200.25650.peter.schuller@infidyne.com> <8663yac62d.fsf@ds4.des.no> <477E72FC.5070304@freebsd.org> <477EA466.6060204@FreeBSD.org> <477EFEAB.8090807@freebsd.org> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.3; amd64-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Dag-Erling=?ISO-8859-1?Q?Sm=F8@freebsd.o?=, =?ISO-8859-1?Q?rg, Jason Evans , ?= Tim Kientzle , rgrav , Peter Schuller Subject: Re: ELF dynamic loader name [was: sbrk(2) broken] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:51:16 -0000 On Sun, 06 Jan 2008 09:56:32 +0200 Danny Braniss wrote: > what Apple has is one file, that will run the appropiate binary if run > on an i386 or a ppc, not 2 different files - universal binary - not rosetta. Sure, but that's got a bunch of different driving factors. I don't know, for example, whether you can build a four-way executable (ia32, x86_64, ppc, ppc64). Well, you probably can, but I'd be a bit surprised if anyone has. FreeBSD supports even more architectures: it just doesn't scale. The best bet for something that has to run everywhere is probably LLVM or TNEF. The advantage that Unix has over MacOS is that we aren't trying to squeeze everything into single "application" directories. So it's reasonable to have "share", and select executables on the basis of PATH. That's how it has worked before. Most sites don't have more than two or three different architectures to support, anyway. If we do get much further with multi-architecture bin and lib, and people actively use these on diskless setups or multi-architecture hosts (amd64/ia32, or other 64/32 bit combinations being the most common) then perhaps it would be nice to have a share/bin where platform-independent scripts (shell, perl, python) as well as dynamic-translated binaries (JVM, LLVM, etc) can live? Cheers, -- Andrew From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:57:49 2008 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40A0916A418; Sun, 6 Jan 2008 21:57:49 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk.360sip.com [72.236.70.226]) by mx1.freebsd.org (Postfix) with ESMTP id 0DF1413C4CC; Sun, 6 Jan 2008 21:57:48 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from [192.168.0.3] ([204.244.149.125]) (authenticated bits=0) by sippysoft.com (8.13.8/8.13.8) with ESMTP id m06Lvkde048475 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 6 Jan 2008 13:57:47 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <47814EAB.70405@FreeBSD.org> Date: Sun, 06 Jan 2008 13:56:59 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Gary Corcoran References: <20080104163352.GA42835@lor.one-eyed-alien.net> <9bbcef730801040958t36e48c9fjd0fbfabd49b08b97@mail.gmail.com> <200801061051.26817.peter.schuller@infidyne.com> <9bbcef730801060458k4bc9f2d6uc3f097d70e087b68@mail.gmail.com> <4780D289.7020509@FreeBSD.org> <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> <4781227B.5020800@rcn.com> In-Reply-To: <4781227B.5020800@rcn.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Kris Kennaway , freebsd-current@FreeBSD.org, Ivan Voras Subject: Should we simply disallow ZFS on FreeBSD/i386? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:57:49 -0000 Gary Corcoran wrote: > Perhaps the 7.0 release notes should include a note to the effect that > ZFS is *strongly* NOT RECOMMENDED on 32-bit systems at this time, due By watching this and other threads on ZFS and reading Sun's own design papers I am getting strong impression that this should be even more strong than strong NOT RECOMMENDED. Perhaps ZFS should BE DISALLOWED to run on i386 at all (unless one does some manual source code tweaking or something like this, and hence can ask no official support from the project). I believe that 95% of hardware today that realistically is capable of running ZFS is also capable of running 64bit code, so that potential ZFS users are far better off switching to FreeBSD/amd64 and help testing/improving that architecture than fighting architectural limitations of already dying i386. And we are as a project are better off too, by spending out limited resources on something that has future. From my own experience FreeBSD/amd64 is quite mature for running most if not all of the server tasks today and ZFS is first and foremost a server FS. The only place where FreeBSD/i386 beats FreeBSD/amd64 is desktop, due to binary drivers and such, but ZFS is almost useless there.