From owner-freebsd-stable@FreeBSD.ORG Mon Sep 5 09:35:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B58D1065670 for ; Mon, 5 Sep 2011 09:35:07 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from mail.incore.de (dss.incore.de [195.145.1.138]) by mx1.freebsd.org (Postfix) with ESMTP id E23968FC18 for ; Mon, 5 Sep 2011 09:35:01 +0000 (UTC) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by mail.incore.de (Postfix) with ESMTP id 1BC275EC6C for ; Mon, 5 Sep 2011 11:15:44 +0200 (CEST) X-Virus-Scanned: amavisd-new at incore.de Received: from mail.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id e0WXVpovcz0v for ; Mon, 5 Sep 2011 11:15:42 +0200 (CEST) Received: from mail.incore (fwintern.dmz [10.0.0.253]) by mail.incore.de (Postfix) with ESMTP id E16885EC42 for ; Mon, 5 Sep 2011 11:15:42 +0200 (CEST) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.incore (Postfix) with ESMTP id DAB2445088 for ; Mon, 5 Sep 2011 11:15:42 +0200 (CEST) Message-ID: <4E64933E.8030908@incore.de> Date: Mon, 05 Sep 2011 11:15:42 +0200 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Subject: UFS_DIRHASH panics on a dozen server within 30 hours X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2011 09:35:07 -0000 Hi, a week ago a dozen of my FreeBSD server crashed within a time span of 30 hours. On the server run very different applications, some of them were only standby. All server has the same kernel with FreeBSD 6 STABLE and there were no problems for yours until the "black monday". Yes I know that FreeBSD 6 is out of date now, but I don't like to change a very good running system. Another reason is that my hardware needs the amr driver and because of the outstanding solution of the amr_ioctl problem described in kern/155658 it is not possible for me to upgrade my production sytems without changing hardware. Now I have a dozen core dumps and try to understand what happened. All dumps looks very similar and the panic is always "page fault" in _mtx_lock_sleep called from ufsdirhash_recycle or ufsdirhash_free because the used mtx_object is overwritten with zeros by someone before _mtx_lock_sleep is called. A typical stack trace and some kgdb output follows: (kgdb) where #0 doadump () at pcpu.h:165 #1 0xc03c5b25 in boot (howto=260) at ../../../kern/kern_shutdown.c:410 #2 0xc03c5e7d in panic (fmt=0xc05931cb "%s") at ../../../kern/kern_shutdown.c:566 #3 0xc0564606 in trap_fatal (frame=0xec6ed77c, eva=256) at ../../../i386/i386/trap.c:838 #4 0xc0563d1e in trap (frame= {tf_fs = 8, tf_es = -328335320, tf_ds = -328335320, tf_edi = -901761536, tf_esi = 0, tf_ebp = -328280120, tf_isp = -328280152, tf_ebx = -827089920, tf_edx = 0, tf_ecx = 2, tf_eax = 1, tf_trapno = 12, tf_err = 0, tf_eip = -1069829895, tf_cs = 32, tf_eflags = 65538, tf_esp = -827089920, tf_ss = 2}) at ../../../i386/i386/trap.c:270 #5 0xc054ddda in calltrap () at ../../../i386/i386/exception.s:139 #6 0xc03bb0f9 in _mtx_lock_sleep (m=0xceb39c00, tid=3393205760, opts=0, file=0x0, line=0) at ../../../kern/kern_mutex.c:550 #7 0xc04eb3c5 in ufsdirhash_recycle (wanted=57230) at ../../../ufs/ufs/ufs_dirhash.c:1035 #8 0xc04e981b in ufsdirhash_build (ip=0xca6b6084) at ../../../ufs/ufs/ufs_dirhash.c:173 #9 0xc04ebbdd in ufs_lookup (ap=0xec6ed920) at ../../../ufs/ufs/ufs_lookup.c:202 #10 0xc057116c in VOP_CACHEDLOOKUP_APV (vop=0x1, a=0x0) at vnode_if.c:150 #11 0xc04164fa in vfs_cache_lookup (ap=0x1) at vnode_if.h:82 #12 0xc05710fb in VOP_LOOKUP_APV (vop=0xc05f90a0, a=0xec6ed9c0) at vnode_if.c:99 #13 0xc041add4 in lookup (ndp=0xec6edbcc) at vnode_if.h:56 #14 0xc041a66a in namei (ndp=0xec6edbcc) at ../../../kern/vfs_lookup.c:216 #15 0xc042ec31 in vn_open_cred (ndp=0xec6edbcc, flagp=0xec6edccc, cmode=384, cred=0xc9bceb80, fdidx=97) at ../../../kern/vfs_vnops.c:183 #16 0xc042e982 in vn_open (ndp=0x0, flagp=0xec6edccc, cmode=384, fdidx=97) at ../../../kern/vfs_vnops.c:91 #17 0xc042749a in kern_open (td=0xca403600, path=0x1
, pathseg=UIO_SYSSPACE, flags=1, mode=438) at ../../../kern/vfs_syscalls.c:1016 #18 0xc04271d2 in open (td=0xca403600, uap=0xec6edd04) at ../../../kern/vfs_syscalls.c:971 #19 0xc056494b in syscall (frame= {tf_fs = -1082195909, tf_es = -1082195909, tf_ds = -1082195909, tf_edi = -1082141792, tf_esi = -1082155856, tf_ebp = -1082151736, tf_isp = -328278684, tf_ebx = -1982551028, tf_edx = 41, tf_ecx = 0, tf_eax = 5, tf_trapno = 0, tf_err = 2, tf_eip = -2008413713, tf_cs = 51, tf_eflags = 642, tf_esp = -1082155972, tf_ss = 59}) at ../../../i386/i386/trap.c:984 #20 0xc054de2f in Xint0x80_syscall () at ../../../i386/i386/exception.s:200 (kgdb) f 8 #8 0xc04e981b in ufsdirhash_build (ip=0xca6b6084) at ../../../ufs/ufs/ufs_dirhash.c:173 173 if (ufsdirhash_recycle(memreqd) != 0) (kgdb) p *ip $1 = {i_nextsnap = {tqe_next = 0x0, tqe_prev = 0x0}, i_vnode = 0xca6c0bb0, i_ump = 0xc9bd3300, i_flag = 0, i_dev = 0xc9b4f400, i_number = 4686848, i_effnlink = 2, i_fs = 0xc9ba5800, i_dquot = {0x0, 0x0}, i_modrev = 14753454826293, i_lockf = 0x0, i_count = 24, i_endoff = 112640, i_diroff = 72704, i_offset = 73056, i_ino =3357131, i_reclen = 16, i_un = {dirhash = 0x0, snapblklist = 0x0}, i_ea_area = 0x0, i_ea_len = 0, i_ea_error = 0, i_mode = 16832, i_nlink = 2, i_size = 112640, i_flags = 0, i_gen = -1337636365, i_uid = 60, i_gid = 60, dinode_u = {din1 = 0xca6c7d00, din2 = 0xca6c7d00}} kgdb) f 7 #7 0xc04eb3c5 in ufsdirhash_recycle (wanted=57230) at ../../../ufs/ufs/ufs_dirhash.c:1035 1035 DIRHASH_LOCK(dh); (kgdb) p dh $2 = (struct dirhash *) 0xceb39c00 (kgdb) p *dh $3 = {dh_mtx = {mtx_object = {lo_class = 0x0, lo_name = 0x0, lo_type = 0x0, lo_flags = 0, lo_list = { tqe_next = 0x0, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 2, mtx_recurse = 0}, dh_hash = 0x0, dh_narrays = 0, dh_hlen = 0, dh_hused = 0, dh_blkfree = 0x0, dh_nblk = 0, dh_dirblks = 0, dh_firstfree = { 0 , -16777216, -1 }, dh_seqopt = 1, dh_seqoff = 3440, dh_score =64, dh_onlist = 1, dh_list = {tqe_next = 0xcf919a00, tqe_prev = 0xc063cfb0 (kgdb) f 6 #6 0xc03bb0f9 in _mtx_lock_sleep (m=0xceb39c00, tid=3393205760, opts=0, file=0x0, line=0) at ../../../kern/kern_mutex.c:550 550 if (m != &Giant && TD_IS_RUNNING(owner)) { (kgdb) p m $4 = (struct mtx *) 0xceb39c00 (kgdb) p *m $5 = {mtx_object = {lo_class = 0x0, lo_name = 0x0, lo_type = 0x0, lo_flags = 0, lo_list = {tqe_next = 0x0, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 2, mtx_recurse = 0} (kgdb) p &Giant $6 = (struct mtx *) 0xc062a0e0 (kgdb) p owner $7 = (volatile struct thread *) 0x0 info local owner = (volatile struct thread *) 0x0 v = 0 (kgdb) list 545 */ 546 owner = (struct thread *)(v & MTX_FLAGMASK); 547 #ifdef ADAPTIVE_GIANT 548 if (TD_IS_RUNNING(owner)) { 549 #else 550 if (m != &Giant && TD_IS_RUNNING(owner)) { 551 #endif The crash occurs in line 550 because owner is zero and should be a thread id that holds the dirhash mutex. When _mtx_lock_sleep is called the mtx_object already is filled with zeros and especially mtx_lock should be 4 (UNOWNED) or the thread id of someone. What may be the reason, that the panics never occured before and then on a dozen server in a short time ? No further crashs since a week now. Any hints are welcome. -- Dr. Andreas Longwitz Data Service GmbH Beethovenstr. 2A 23617 Stockelsdorf Amtsgericht Lübeck, HRB 318 BS Geschäftsführer: Wilfried Paepcke, Dr. Andreas Longwitz, Josef Flatau From owner-freebsd-stable@FreeBSD.ORG Tue Sep 6 06:12:08 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4E17106566B; Tue, 6 Sep 2011 06:12:08 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 1E6D48FC0A; Tue, 6 Sep 2011 06:12:07 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p866C4Wt009208; Tue, 6 Sep 2011 13:12:04 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E65B9AF.6070203@rdtc.ru> Date: Tue, 06 Sep 2011 13:11:59 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Andriy Gapon References: <4E6102DA.8090605@FreeBSD.org> In-Reply-To: <4E6102DA.8090605@FreeBSD.org> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org Subject: Re: stop scheduler on panic patches updated X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2011 06:12:08 -0000 02.09.2011 23:22, Andriy Gapon ÐÉÛÅÔ: > > The patches can be found at the same locations: > head - http://people.freebsd.org/~avg/stop_scheduler_on_panic.diff > stable/8 - http://people.freebsd.org/~avg/stop_scheduler_on_panic.8.x.diff > > Additionally, if you use a USB keyboard, then the following patch is required for > proper operation in post-panic environment: > http://people.freebsd.org/~avg/stop_scheduler_on_panic.usb.diff > The patch is the same for both head and stable/8. > It shouldn't hurt if you don't use USB devices or use other USB devices. > Thanks, I'm going to try it just now because my dummynet panics are back with start of new school year :-) Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Tue Sep 6 15:04:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76B51065674 for ; Tue, 6 Sep 2011 15:04:44 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9F5F98FC1C for ; Tue, 6 Sep 2011 15:04:44 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 5661146B37; Tue, 6 Sep 2011 11:04:44 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DB17C8A02E; Tue, 6 Sep 2011 11:04:43 -0400 (EDT) From: John Baldwin To: freebsd-stable@freebsd.org Date: Tue, 6 Sep 2011 11:04:43 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <4E64933E.8030908@incore.de> In-Reply-To: <4E64933E.8030908@incore.de> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201109061104.43409.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 06 Sep 2011 11:04:44 -0400 (EDT) Cc: Andreas Longwitz Subject: Re: UFS_DIRHASH panics on a dozen server within 30 hours X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2011 15:04:44 -0000 On Monday, September 05, 2011 5:15:42 am Andreas Longwitz wrote: > Hi, > > a week ago a dozen of my FreeBSD server crashed within a time span of > 30 hours. On the server run very different applications, some of them > were only standby. All server has the same kernel with FreeBSD 6 STABLE > and there were no problems for yours until the "black monday". > > Yes I know that FreeBSD 6 is out of date now, but I don't like to > change a very good running system. Another reason is that my hardware > needs the amr driver and because of the outstanding solution of the > amr_ioctl problem described in kern/155658 it is not possible for me > to upgrade my production sytems without changing hardware. Hmm, the patch in that PR should still apply to newer versions. Also, you could just change the malloc() call to always allocate the maximum size (instead of using a static buffer) for a smaller diff. It seems though that a specific command is overrunning its buffer. > Now I have a dozen core dumps and try to understand what happened. > All dumps looks very similar and the panic is always "page fault" > in _mtx_lock_sleep called from ufsdirhash_recycle or ufsdirhash_free > because the used mtx_object is overwritten with zeros by someone > before _mtx_lock_sleep is called. I don't know of anything in particular that would explain this, esp. as to why you would see them all occur at the same time. Maybe look to see if the machines were doing something unusual at that time (a cron job, etc.)? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Tue Sep 6 22:21:22 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3011106564A for ; Tue, 6 Sep 2011 22:21:22 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-pz0-f45.google.com (mail-pz0-f45.google.com [209.85.210.45]) by mx1.freebsd.org (Postfix) with ESMTP id C1D618FC08 for ; Tue, 6 Sep 2011 22:21:22 +0000 (UTC) Received: by pzk33 with SMTP id 33so17171398pzk.18 for ; Tue, 06 Sep 2011 15:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=78kRDVZejyxXfs2INENCc5lzmZr8vEjV8C0Xv7KkCMk=; b=ugCdc1SUd1LkRIDPLtKXfqYo9UVpIdfaxnxprpKildikMKcKuJzprEfnjEfOMHBv0w WGh+FLXtwHXFaAGgI3Z+iw3abNFMgAuOf8jhpWgFcqhx+U/dU9i8JZtCChaxGiv498PX 7FFFOCnvnv30M6W2YdHqy7I+BBS/TgCWmzTcc= MIME-Version: 1.0 Received: by 10.68.64.164 with SMTP id p4mr11379014pbs.128.1315346109748; Tue, 06 Sep 2011 14:55:09 -0700 (PDT) Received: by 10.142.131.15 with HTTP; Tue, 6 Sep 2011 14:55:09 -0700 (PDT) Date: Tue, 6 Sep 2011 17:55:09 -0400 Message-ID: From: Arnaud Lacombe To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: FS corruption between 8-STABLE and 7-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2011 22:21:23 -0000 Hi, I got a strange boot failure when booting a FreeBSD 7-stable after a 8-stable kernel, with a FreeBSD 7.4 kernel: pid 100 (fsck_ufs), uid 0: exited on signal 8 pid 101 (fsck_ufs), uid 0: exited on signal 8 WARNING: R/W mount of / denied. Filesystem is not clean - run fsck WARNING: R/W mount of / denied. Filesystem is not clean - run fsck it would seem that going 8-STABLE change the filesystem in such a way 7-STABLE cannot be booted after. Thanks guys ... - Arnaud ps: please CC me, I'm not subscribed to that list From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 00:48:31 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E33C1106566C; Wed, 7 Sep 2011 00:48:31 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5124F8FC17; Wed, 7 Sep 2011 00:48:21 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p870lpDU023337; Wed, 7 Sep 2011 09:48:01 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p870lkCs065537; Wed, 7 Sep 2011 09:47:46 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Wed, 07 Sep 2011 09:47:17 +0900 (JST) Message-Id: <20110907.094717.2272609566853905102.hrs@allbsd.org> To: attilio@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110903104410.GX17489@deviant.kiev.zoral.com.ua> References: <20110820.105229.834911491934932780.hrs@allbsd.org> <20110903.071908.971549835606878048.hrs@allbsd.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3.51 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Wed_Sep__7_09_47_17_2011_726)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Wed, 07 Sep 2011 09:48:03 +0900 (JST) X-Spam-Status: No, score=-104.6 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: pjd@FreeBSD.org, sterling@camdensoftware.com, avg@FreeBSD.org, nick@desert.net, freebsd-stable@FreeBSD.org, kib@FreeBSD.org Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 00:48:32 -0000 ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Attilio Rao wrote in : at> This should be enough for someone NFS-aware to look into it. at> at> Were you also able to get a core? Yes. But as kib@ pointed out it seems a deadlock in ZFS. Some experiments I did showed that this deadlock can be triggered at least by doing "rm -rf" against a local directory that has a large number of files/sub-directories. Then, I updated the kernel with the latest 8-STABLE + WITNESS option because a fix for LOR of spa_config lock was committed and tracking locks without WITNESS was hard. The deadlock can still be triggered after that. During this investigation an disk has to be replaced and resilvering it is now in progress. A deadlock and a forced reboot after that make recovering of the zfs datasets take a long time (for committing logs, I think), so I will try to reproduce the deadlock and get a core dump after it finished. If the old kernel and core of the deadlock I reported on Saturday are still useful for debugging, I can put them to somewhere you can access. -- Hiroki ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk5mvxUACgkQTyzT2CeTzy14sACfRMSxYlKQktsuL8w/10/89VUU Q6sAnAu5mnWcMUzllH3eTzEXr04g3s2j =iR8M -----END PGP SIGNATURE----- ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)---- From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 08:45:44 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA199106566B for ; Wed, 7 Sep 2011 08:45:44 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 213F78FC14 for ; Wed, 7 Sep 2011 08:45:43 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p878jdXF018132; Wed, 7 Sep 2011 15:45:40 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E672F2E.7090400@rdtc.ru> Date: Wed, 07 Sep 2011 15:45:34 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jeremy Chadwick References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> In-Reply-To: <20110731173129.GA53635@icarus.home.lan> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 08:45:44 -0000 01.08.2011 00:31, Jeremy Chadwick ÐÉÛÅÔ: > On Sun, Jul 31, 2011 at 11:51:40PM +0700, Eugene Grosbein wrote: >> Hi! >> >> Suppose, there is a machine which writes two kinds of log files through syslogd: >> quickly-growing that need to be rotated based on their size (hourly is too seldom) >> and other that should be rotated once a day, at midnight only. >> >> For first kind of logs we have to run newsyslog once every 5 minutes using cron: >> >> */5 * * * * root newsyslog >> >> For second kind of logs we have lines in newsyslog.conf such as following: >> >> /var/log/mpd.log 640 16 * @T0000 JC >> >> This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only. >> Note, that compressing the file takes 8 minutes. >> >> However, every night at 00:05 I get an error: >> >> bzip2: I/O or other error, bailing out. Possible reason follows. >> bzip2: No such file or directory >> Input file = /var/log/mpd.log.0, output file = /var/log/mpd.log.0.bz2 >> newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status (1) >> >> It seems, newsyslog still wants to process my file at 00:05 despite @T0000 >> time specification. Is it broken? > > I have three things to say on the matter, all of which are somewhat > independent of one another so please keep that in mind. I imagine #1 > below is your problem. > > 1) The newsyslog.conf(5) man page has this clause in it, for the "when" > field (in your case, @T0000): > > when ... If the when field contains an asterisk (`*'), log rotation > will solely depend on the contents of the size field. Otherwise, > the when field consists of an optional interval in hours, usually > followed by an `@'-sign and a time in restricted ISO 8601 format. > > If a time is specified, the log file will only be trimmed if > newsyslog(8) is run within one hour of the specified time. If an > interval is specified, the log file will be trimmed if that many > hours have passed since the last rotation. ... > > You might think that "one hour of the specified time" value/clause > correlates with the interval that newsyslog is run at via cron, but that > would be wrong. newsyslog REALLY DOES have hard-coded values for 3600 > seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I > have not looked at the code, but the fact of the matter is, 1 hour > appears to be a "special" value. I would heed that as a warning. After reading newsyslog code, now it's obvious it just ignores minutes and seconds while making decision if a file should be rotated. It looks at hours only. That's sad. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 10:21:39 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BE40106564A for ; Wed, 7 Sep 2011 10:21:39 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id 641328FC15 for ; Wed, 7 Sep 2011 10:21:39 +0000 (UTC) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta11.emeryville.ca.mail.comcast.net with comcast id Vm8P1h0050x6nqcABm8P5o; Wed, 07 Sep 2011 10:08:23 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta12.emeryville.ca.mail.comcast.net with comcast id Vm8P1h00F1t3BNj8Ym8PM4; Wed, 07 Sep 2011 10:08:24 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id E40D5102C1B; Wed, 7 Sep 2011 03:08:27 -0700 (PDT) Date: Wed, 7 Sep 2011 03:08:27 -0700 From: Jeremy Chadwick To: Eugene Grosbein Message-ID: <20110907100827.GA96216@icarus.home.lan> References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E672F2E.7090400@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E672F2E.7090400@rdtc.ru> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 10:21:39 -0000 On Wed, Sep 07, 2011 at 03:45:34PM +0700, Eugene Grosbein wrote: > 01.08.2011 00:31, Jeremy Chadwick ?????: > > On Sun, Jul 31, 2011 at 11:51:40PM +0700, Eugene Grosbein wrote: > >> Hi! > >> > >> Suppose, there is a machine which writes two kinds of log files through syslogd: > >> quickly-growing that need to be rotated based on their size (hourly is too seldom) > >> and other that should be rotated once a day, at midnight only. > >> > >> For first kind of logs we have to run newsyslog once every 5 minutes using cron: > >> > >> */5 * * * * root newsyslog > >> > >> For second kind of logs we have lines in newsyslog.conf such as following: > >> > >> /var/log/mpd.log 640 16 * @T0000 JC > >> > >> This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only. > >> Note, that compressing the file takes 8 minutes. > >> > >> However, every night at 00:05 I get an error: > >> > >> bzip2: I/O or other error, bailing out. Possible reason follows. > >> bzip2: No such file or directory > >> Input file = /var/log/mpd.log.0, output file = /var/log/mpd.log.0.bz2 > >> newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status (1) > >> > >> It seems, newsyslog still wants to process my file at 00:05 despite @T0000 > >> time specification. Is it broken? > > > > I have three things to say on the matter, all of which are somewhat > > independent of one another so please keep that in mind. I imagine #1 > > below is your problem. > > > > 1) The newsyslog.conf(5) man page has this clause in it, for the "when" > > field (in your case, @T0000): > > > > when ... If the when field contains an asterisk (`*'), log rotation > > will solely depend on the contents of the size field. Otherwise, > > the when field consists of an optional interval in hours, usually > > followed by an `@'-sign and a time in restricted ISO 8601 format. > > > > If a time is specified, the log file will only be trimmed if > > newsyslog(8) is run within one hour of the specified time. If an > > interval is specified, the log file will be trimmed if that many > > hours have passed since the last rotation. ... > > > > You might think that "one hour of the specified time" value/clause > > correlates with the interval that newsyslog is run at via cron, but that > > would be wrong. newsyslog REALLY DOES have hard-coded values for 3600 > > seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I > > have not looked at the code, but the fact of the matter is, 1 hour > > appears to be a "special" value. I would heed that as a warning. > > After reading newsyslog code, now it's obvious it just ignores minutes and seconds > while making decision if a file should be rotated. It looks at hours only. > That's sad. I imagine this "design limitation" is due to the fact that newsyslog is called from cron, which only supports minute-level granularity. The newsyslog.conf man page even hints at this while describing the "when" column: There is no provision for the specification of a timezone. There is little point in specifying an explicit minutes or seconds com- ponent in the current implementation, since the only comparison is ``within the hour''. Given this, I would say the "special" 3600-second value within the source code makes sense. I'm not sure what you could use for an alternate method of log rotation for syslog-logged data. Let's circle back to the reason you're rotating logs every 5 minutes: >>> Most of my boxes are diskless NanoBSD installations having /var in >>> memory and I need very detailed debug logs that grow quickly. These >>> logs can easily overflow /var partition in case of network problems >>> (storms etc.) so newsyslog have to check them often. >>> >>> And I have another router that has an HDD to keep daily log and I'd >>> like to have their crontabs unified. I think what the rest of the world might tell you is something to the effect of "you can't have your cake and eat it too". You've got diskless systems that aren't syslogging via network (e.g. to a pool of syslog servers, or a single syslog server) but instead to a memory-backed filesystem, in addition to enabling debug-level logging in mpd by default. A memory-backed filesystem means you don't have much disk space, and you know this based on the need to rotate logs every 5 minutes, right? So I'm confused why one would need debug logging. I imagine that the newsyslog.conf on these machines has a very small number for the "count" column for /var/log/mpd.log. So chances are, by the time you noticed a problem, the logs would have been rotated and removed, no? So why the debug logging? If debug logging really is something you absolutely need, no argument about it, then honestly it sounds like you need some sort of "centralised" logging infrastructure for all of these diskless machines. Most diskless machines I've used utilise some form of centralised "something" -- whether it be a centralised DHCP/PXE server (which you obviously have in some form), or an NFS-mounted root or /home, etc... You get the idea. Could you deploy similar infrastructure for syslog and simply use a remote syslog server in syslog.conf? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 11:27:28 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E20CF106564A for ; Wed, 7 Sep 2011 11:27:27 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 47E758FC08 for ; Wed, 7 Sep 2011 11:27:27 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p87BRN6M019315; Wed, 7 Sep 2011 18:27:23 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E675516.50304@rdtc.ru> Date: Wed, 07 Sep 2011 18:27:18 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jeremy Chadwick References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E672F2E.7090400@rdtc.ru> <20110907100827.GA96216@icarus.home.lan> In-Reply-To: <20110907100827.GA96216@icarus.home.lan> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 11:27:28 -0000 07.09.2011 17:08, Jeremy Chadwick writes: >> After reading newsyslog code, now it's obvious it just ignores minutes and seconds >> while making decision if a file should be rotated. It looks at hours only. >> That's sad. > > I imagine this "design limitation" is due to the fact that newsyslog is > called from cron, which only supports minute-level granularity. > > The newsyslog.conf man page even hints at this while describing the > "when" column: > > There is no provision for the specification of a timezone. There > is little point in specifying an explicit minutes or seconds com- > ponent in the current implementation, since the only comparison > is ``within the hour''. > > Given this, I would say the "special" 3600-second value within the > source code makes sense. > > I'm not sure what you could use for an alternate method of log rotation > for syslog-logged data. I have just followed some of past advices and split my newsyslog.conf in two, moving mpd-like logs with size-based rotation only to /etc/newsyslog-quick.conf. And made another cron job for fiveminly running newsyslog -f /etc/newsyslog-quick.conf > I think what the rest of the world might tell you is something to the > effect of "you can't have your cake and eat it too". You've got > diskless systems that aren't syslogging via network (e.g. to a pool of > syslog servers, or a single syslog server) but instead to a > memory-backed filesystem, in addition to enabling debug-level logging in > mpd by default. > > A memory-backed filesystem means you don't have much disk space, and you > know this based on the need to rotate logs every 5 minutes, right? So > I'm confused why one would need debug logging. I imagine that the > newsyslog.conf on these machines has a very small number for the "count" > column for /var/log/mpd.log. So chances are, by the time you noticed a > problem, the logs would have been rotated and removed, no? So why the > debug logging? > > If debug logging really is something you absolutely need, no argument > about it, then honestly it sounds like you need some sort of > "centralised" logging infrastructure for all of these diskless machines. > Most diskless machines I've used utilise some form of centralised > "something" -- whether it be a centralised DHCP/PXE server (which you > obviously have in some form), or an NFS-mounted root or /home, etc... > You get the idea. Could you deploy similar infrastructure for syslog > and simply use a remote syslog server in syslog.conf? In fact, I do have centralized syslogd server that collects logs from diskless servers. But, I need also local copies of individual server's logs in the MFS. I was in hope to make it with one cron job and one newsyslog.conf but as it seems impossible, I will use two cron jobs :-) Local (compressed) logs residing on the MFS give me convinience to manage and debug a server within one ssh session without need to consult with remote syslog archives. In general, I do have enough MFS space to keep needed backlog but in case of network PPPoE PADI broadcast storms I need quick rotation to prevent MFS overflows. I think I'll get all of this now. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 13:54:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E87E21065677 for ; Wed, 7 Sep 2011 13:54:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C01F68FC1E for ; Wed, 7 Sep 2011 13:54:04 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 5BF9746B06; Wed, 7 Sep 2011 09:54:04 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D40218A037; Wed, 7 Sep 2011 09:54:03 -0400 (EDT) From: John Baldwin To: freebsd-stable@freebsd.org Date: Wed, 7 Sep 2011 09:54:03 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201109070954.03245.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 07 Sep 2011 09:54:03 -0400 (EDT) Cc: Arnaud Lacombe Subject: Re: FS corruption between 8-STABLE and 7-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 13:54:05 -0000 On Tuesday, September 06, 2011 5:55:09 pm Arnaud Lacombe wrote: > Hi, > > I got a strange boot failure when booting a FreeBSD 7-stable after a > 8-stable kernel, with a FreeBSD 7.4 kernel: > > pid 100 (fsck_ufs), uid 0: exited on signal 8 > pid 101 (fsck_ufs), uid 0: exited on signal 8 > WARNING: R/W mount of / denied. Filesystem is not clean - run fsck > WARNING: R/W mount of / denied. Filesystem is not clean - run fsck > > it would seem that going 8-STABLE change the filesystem in such a way > 7-STABLE cannot be booted after. > > Thanks guys ... I boot 8 kernels on machines with a 7 world and back to a 7 kernel all the time without any issues. Did you upgrade your world to 8 and then try to boot it with a 7.4 kernel? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 14:13:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7547F106566C for ; Wed, 7 Sep 2011 14:13:13 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 337348FC0C for ; Wed, 7 Sep 2011 14:13:12 +0000 (UTC) Received: by qyk9 with SMTP id 9so4612407qyk.13 for ; Wed, 07 Sep 2011 07:13:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=cUvC1tQJy7DWp/dbEB8wUs7Jj8wDEbjh+LCxP/rwdSY=; b=N3eaAbcscEImuWVnaPXtA6wYz16SJNqPwmq5DODUYh4QmGzymjT2NFQ5szZVrToB9Y 6v117OcZ7cq/hNEUdF034uD0AqXPJPQXa/+Xcp4YykodaIFwaq9e/9NHrIhBX5xlBcte mRax3FPwJ/Uxz16Yr/Sdiny/J0KJTibit/a9M= MIME-Version: 1.0 Received: by 10.68.7.170 with SMTP id k10mr2108329pba.176.1315404791765; Wed, 07 Sep 2011 07:13:11 -0700 (PDT) Received: by 10.142.232.15 with HTTP; Wed, 7 Sep 2011 07:13:11 -0700 (PDT) In-Reply-To: <201109070954.03245.jhb@freebsd.org> References: <201109070954.03245.jhb@freebsd.org> Date: Wed, 7 Sep 2011 10:13:11 -0400 Message-ID: From: Arnaud Lacombe To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: FS corruption between 8-STABLE and 7-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 14:13:13 -0000 Hi, On Wed, Sep 7, 2011 at 9:54 AM, John Baldwin wrote: > On Tuesday, September 06, 2011 5:55:09 pm Arnaud Lacombe wrote: >> Hi, >> >> I got a strange boot failure when booting a FreeBSD 7-stable after a >> 8-stable kernel, with a FreeBSD 7.4 kernel: >> >> pid 100 (fsck_ufs), uid 0: exited on signal 8 >> pid 101 (fsck_ufs), uid 0: exited on signal 8 >> WARNING: R/W mount of / denied. =A0Filesystem is not clean - run fsck >> WARNING: R/W mount of / denied. =A0Filesystem is not clean - run fsck >> >> it would seem that going 8-STABLE change the filesystem in such a way >> 7-STABLE cannot be booted after. >> >> Thanks guys ... > > I boot 8 kernels on machines with a 7 world and back to a 7 kernel all th= e > time without any issues. =A0Did you upgrade your world to 8 and then try = to boot > it with a 7.4 kernel? > No. - Arnaud From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 14:57:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F4140106564A; Wed, 7 Sep 2011 14:57:02 +0000 (UTC) (envelope-from kensmith@buffalo.edu) Received: from localmailA.acsu.buffalo.edu (localmaila.acsu.buffalo.edu [128.205.5.196]) by mx1.freebsd.org (Postfix) with ESMTP id C559E8FC0A; Wed, 7 Sep 2011 14:57:02 +0000 (UTC) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id EBF8FF857; Wed, 7 Sep 2011 10:51:52 -0400 (EDT) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailA.acsu.buffalo.edu (Postfix) with ESMTP id BE736F8C3; Wed, 7 Sep 2011 10:51:50 -0400 (EDT) Received: from smtp2.acsu.buffalo.edu (smtp2.acsu.buffalo.edu [128.205.5.254]) by localmailA.acsu.buffalo.edu (Prefixe) with ESMTP id B86D4F857; Wed, 7 Sep 2011 10:51:50 -0400 (EDT) Received: from [128.205.32.76] (bauer.cse.buffalo.edu [128.205.32.76]) (Authenticated sender: kensmith@buffalo.edu) by smtp2.acsu.buffalo.edu (Postfix) with ESMTPSA id AF95F45E80; Wed, 7 Sep 2011 10:51:50 -0400 (EDT) From: Ken Smith To: freebsd-current , freebsd-stable Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-sYmUTzQJ5x1meQ9QnIpH" Date: Wed, 07 Sep 2011 10:51:50 -0400 Message-ID: <1315407110.19861.90.camel@bauer.cse.buffalo.edu> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-PM-EL-Spam-Prob: : 8% Cc: Subject: FreeBSD 9.0-BETA2 Available... X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 14:57:03 -0000 --=-sYmUTzQJ5x1meQ9QnIpH Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The second BETA build of the 9.0-RELEASE release cycle is now available. Since this is the first release of a brand new branch I cross-post the announcements on both -current and -stable. But just so you know most of the developers active in head pay more attention to the -current mailing list. If you notice problems you can report them through the normal Gnats PR system or on the -current mailing list. The 9.0-RELEASE cycle will be tracked here: http://wiki.freebsd.org/Releng/9.0TODO though the schedule listed there is now way off. BETA1 and some other factors caused a lot of activity. We'll re-work the schedule some time soon. NOTE: The location of the FTP install tree and ISOs have changed slightly. What we used for BETA2 reflects a directory structure that would let us fully utilize building and distributing a wider variety of architectures, based on this: farrell 1 % cd /usr/src farrell 2 % make targets Supported TARGET/TARGET_ARCH pairs for world and kernel targets amd64/amd64 arm/arm arm/armeb i386/i386 ia64/ia64 mips/mipsel mips/mipseb mips/mips64el mips/mips64eb mips/mipsn32eb pc98/i386 powerpc/powerpc powerpc/powerpc64 sparc64/sparc64 farrell 3 %=20 However we currently, and likely will never, do builds of everything. The new layout does add a some extra complexity. So we're actively discussing whether or not to change the layout from previous releases, and if we do change it whether or not to change it this much... What's there now can be viewed as an almost "worst case" scenario (one of the possibilities being discussed includes having both $TARGET and $TARGET_ARCH in the filenames). It's entirely possible we'll back off and revert to the old layout despite that layout potentially limiting what we can do with the new build infrastructure. ISO images for the following architectures are available, with pathnames given relative to the top-level of the FTP site: amd64: .../releases/amd64/amd64/ISO-IMAGES/9.0/ i386: .../releases/i386/i386/ISO-IMAGES/9.0/ powerpc: .../releases/powerpc/powerpc/ISO-IMAGES/9.0/ powerpc64: .../releases/powerpc/powerpc64/ISO-IMAGES/9.0/ sparc64: .../releases/sparc64/sparc64/ISO-IMAGES/9.0/ Due to a bug that crept in just before BETA2 the ia64 architecture builds failed. That bug has since been fixed. MD5/SHA256 checksums are tacked on below. In addition to lots of bugfixes and a few new things that crept in since BETA1 we have done what we hope will be the final bump of versions for the non-symbol-versioned system libraries. This time we did not bump the version of every non-symbol-versioned library, we just did the ones that had an API and/or ABI change. As before one of the many new features of 9.0 we would like tested is the new installer, so fresh installs on test systems are encouraged. The shift in the directory layout described above is in part due to the new installer. FTP-based installs of BETA1 would have failed, as-is the new installer expects this for where to find the FTP install trees: .../releases/`uname -m`/`uname -p`/`uname -r` We're discussing whether or not the ISO images need to tag along with the FTP install bits to these sub-directories as part of the re-structuring, etc. If you would like to use csup/cvsup mechanisms to access the source tree the branch tag to use is "." (head). If you would like to access the source tree via SVN it is "svn://svn.freebsd.org/base/head/". At this time FreeBSD-Update is not available, in part to help encourage testing the installer. Checksums: MD5 (FreeBSD-9.0-BETA2-amd64-bootonly.iso) =3D 888a6eacd82a716846ce1f5151bc= e1a6 MD5 (FreeBSD-9.0-BETA2-amd64-dvd1.iso) =3D 38178ecee5924950fbcace6ed9881210 MD5 (FreeBSD-9.0-BETA2-amd64-memstick.img) =3D 741a1fb240fde587d4208540e1c1= 3779 MD5 (FreeBSD-9.0-BETA2-i386-bootonly.iso) =3D b74f87a5a5d3da0e9241e3485d3eb= 2cf MD5 (FreeBSD-9.0-BETA2-i386-dvd1.iso) =3D d81b89d182bdbe11155dfd1e9e165fee MD5 (FreeBSD-9.0-BETA2-i386-memstick.img) =3D acfc9797273704008c08c416f7aa8= d8b MD5 (FreeBSD-9.0-BETA2-powerpc-bootonly.iso) =3D f2f0274b655c4a79d6b8847216= 2905f9 MD5 (FreeBSD-9.0-BETA2-powerpc-memstick) =3D 200ca101d6bb761e15d70aa743a632= f6 MD5 (FreeBSD-9.0-BETA2-powerpc-release.iso) =3D 23df0f1e07b375be50f139e7dc9= 5dcf5 MD5 (FreeBSD-9.0-BETA2-powerpc64-bootonly.iso) =3D ed0ac1edfcfe71596f740933= 9cfabc9f MD5 (FreeBSD-9.0-BETA2-powerpc64-memstick) =3D eb552d31fe999396c0bd027b0586= d2ce MD5 (FreeBSD-9.0-BETA2-powerpc64-release.iso) =3D c4469fe01301ac835c40701d5= 2422b55 MD5 (FreeBSD-9.0-BETA2-sparc64-CHECKSUM.MD5) =3D d41d8cd98f00b204e9800998ec= f8427e MD5 (FreeBSD-9.0-BETA2-sparc64-bootonly.iso) =3D e72add6b8f2ad4d9564eddbf7a= e533f0 MD5 (FreeBSD-9.0-BETA2-sparc64-dvd1.iso) =3D 2227e2095d57c4353400bbc4409cac= ff SHA256 (FreeBSD-9.0-BETA2-amd64-bootonly.iso) =3D 01319df05dbccaac2a8c56c42= 84798a4c8c3399a0c0fc8ba04407cd8eea89f76 SHA256 (FreeBSD-9.0-BETA2-amd64-dvd1.iso) =3D e4fa69243aeefcfbce98ac8c30a81= 20a33ff122e7eb8c5fd73c23d49e3b2fad5 SHA256 (FreeBSD-9.0-BETA2-amd64-memstick.img) =3D ac5e08a1a7ed4e6e8f19c18fc= d53306e9ab5b4636c1b390e7fccf271e2ffbbcc SHA256 (FreeBSD-9.0-BETA2-i386-bootonly.iso) =3D e31ca81caa14866c2f3eb837b3= 46f1b853ddb9181a9aa7b8d45915a0ebebbfd1 SHA256 (FreeBSD-9.0-BETA2-i386-dvd1.iso) =3D 2ed439c308c874deb17f20682fa3d2= 4ab64452912a1408d1b8c4220c29ef2728 SHA256 (FreeBSD-9.0-BETA2-i386-memstick.img) =3D bb8d8151863784c4147ba4e62f= 073fcdd4c9b194b55470d3133f7504c2bad5c6 SHA256 (FreeBSD-9.0-BETA2-powerpc-bootonly.iso) =3D 64d7fc1b97a00118b577103= d9882f14b98523716c24d01c6498cce5b8b555154 SHA256 (FreeBSD-9.0-BETA2-powerpc-memstick) =3D 0fd99234ad81687d90f9cbffb37= 5bf938444ca41e5f090d0c1cf86ce5b373422 SHA256 (FreeBSD-9.0-BETA2-powerpc-release.iso) =3D fbe44a2665f68f45c86e8f99= a5e569ed5f9905e38c6f516472bac58e8e742be6 SHA256 (FreeBSD-9.0-BETA2-powerpc64-bootonly.iso) =3D e8ab1ece6460080951dfb= 7bac9de9e80703b0ccf7f684e8be82fd5a32405b093 SHA256 (FreeBSD-9.0-BETA2-powerpc64-memstick) =3D f9778f2c8d85709e027ceaf9d= 8b1c99cae8719bee0a363a29b0e97594a6b6b85 SHA256 (FreeBSD-9.0-BETA2-powerpc64-release.iso) =3D 92a754150b17eb8567fa3b= 9dd306dbb96f2476075ab646ba90840aa733364b7c SHA256 (FreeBSD-9.0-BETA2-sparc64-bootonly.iso) =3D 0ef5b1b41683bfa9047bbd5= 56a81466ed5aa4609335380d611e63ad2d8517f51 SHA256 (FreeBSD-9.0-BETA2-sparc64-dvd1.iso) =3D 00477eaf46f93678a28878d87c4= 551f529915266aee96cbb7141445503efce8d --=20 Ken Smith - From there to here, from here to | kensmith@buffalo.edu there, funny things are everywhere. | - Theodor Geisel | --=-sYmUTzQJ5x1meQ9QnIpH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEUEABECAAYFAk5nhQYACgkQ/G14VSmup/Y/YwCggrtbkd5oTCfh9aGOzglz624c N4QAl1v8HDezbsVhRT7tcdAo/RZmjIY= =vlIG -----END PGP SIGNATURE----- --=-sYmUTzQJ5x1meQ9QnIpH-- From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 19:00:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53F6E106566C for ; Wed, 7 Sep 2011 19:00:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 29CAD8FC1A for ; Wed, 7 Sep 2011 19:00:33 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id D758E46B0D; Wed, 7 Sep 2011 15:00:32 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6AC7B8A02E; Wed, 7 Sep 2011 15:00:32 -0400 (EDT) From: John Baldwin To: freebsd-stable@freebsd.org Date: Wed, 7 Sep 2011 15:00:31 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <201109070954.03245.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201109071500.31950.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 07 Sep 2011 15:00:32 -0400 (EDT) Cc: Arnaud Lacombe Subject: Re: FS corruption between 8-STABLE and 7-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 19:00:33 -0000 On Wednesday, September 07, 2011 10:13:11 am Arnaud Lacombe wrote: > Hi, > > On Wed, Sep 7, 2011 at 9:54 AM, John Baldwin wrote: > > On Tuesday, September 06, 2011 5:55:09 pm Arnaud Lacombe wrote: > >> Hi, > >> > >> I got a strange boot failure when booting a FreeBSD 7-stable after a > >> 8-stable kernel, with a FreeBSD 7.4 kernel: > >> > >> pid 100 (fsck_ufs), uid 0: exited on signal 8 > >> pid 101 (fsck_ufs), uid 0: exited on signal 8 > >> WARNING: R/W mount of / denied. Filesystem is not clean - run fsck > >> WARNING: R/W mount of / denied. Filesystem is not clean - run fsck > >> > >> it would seem that going 8-STABLE change the filesystem in such a way > >> 7-STABLE cannot be booted after. > >> > >> Thanks guys ... > > > > I boot 8 kernels on machines with a 7 world and back to a 7 kernel all the > > time without any issues. Did you upgrade your world to 8 and then try to boot > > it with a 7.4 kernel? > > > No. Is your 7-stable world newer than the 7.4 kernel? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 19:03:26 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAECB106566C for ; Wed, 7 Sep 2011 19:03:25 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id ADB808FC0A for ; Wed, 7 Sep 2011 19:03:25 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p87J3NOp054467 for ; Wed, 7 Sep 2011 15:03:23 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4E67BFFE.9000705@sentex.net> Date: Wed, 07 Sep 2011 15:03:26 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: FreeBSD-STABLE Mailing List X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Subject: Aug 29 RELENG_8 crash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 19:03:26 -0000 LNS server that was running with ipv6 enabled. Anyone know what this might be related to ? Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x12 fault code = supervisor read, page not present instruction pointer = 0x20:0xc07383e7 stack pointer = 0x28:0xc5658be8 frame pointer = 0x28:0xc5658bfc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 11 (swi4: clock) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xc068dd47 at kdb_backtrace+0x47 #1 0xc065e607 at panic+0x117 #2 0xc0883cc3 at trap_fatal+0x323 #3 0xc0883f40 at trap_pfault+0x270 #4 0xc088445a at trap+0x43a #5 0xc086b24c at calltrap+0x6 #6 0xc06b4ce9 at pfslowtimo+0x29 #7 0xc0671f87 at softclock+0x237 #8 0xc063525b at intr_event_execute_handlers+0x13b #9 0xc063691b at ithread_loop+0x6b #10 0xc0632547 at fork_exit+0x97 #11 0xc086b2c4 at fork_trampoline+0x8 Uptime: 9d2h15m49s Physical memory: 2515 MB Dumping 256 MB: 241 225 209 193 177 161 145 129 113 97 81 65 49 33 17 1 #0 doadump () at pcpu.h:231 231 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:231 #1 0xc065e3a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:429 #2 0xc065e640 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:602 #3 0xc0883cc3 in trap_fatal (frame=0xc5658ba8, eva=18) at /usr/src/sys/i386/i386/trap.c:973 #4 0xc0883f40 in trap_pfault (frame=0xc5658ba8, usermode=0, eva=18) at /usr/src/sys/i386/i386/trap.c:886 #5 0xc088445a in trap (frame=0xc5658ba8) at /usr/src/sys/i386/i386/trap.c:559 #6 0xc086b24c in calltrap () at /usr/src/sys/i386/i386/exception.s:168 #7 0xc07383e7 in igmp_slowtimo () at /usr/src/sys/netinet/igmp.c:2184 #8 0xc06b4ce9 in pfslowtimo (arg=0x0) at /usr/src/sys/kern/uipc_domain.c:518 #9 0xc0671f87 in softclock (arg=0xc09814a0) at /usr/src/sys/kern/kern_timeout.c:532 #10 0xc063525b in intr_event_execute_handlers (p=0xc591c810, ie=0xc595d780) at /usr/src/sys/kern/kern_intr.c:1216 #11 0xc063691b in ithread_loop (arg=0xc58fc890) at /usr/src/sys/kern/kern_intr.c:1229 #12 0xc0632547 in fork_exit (callout=0xc06368b0 , arg=0xc58fc890, frame=0xc5658d28) at /usr/src/sys/kern/kern_fork.c:876 #13 0xc086b2c4 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:275 (kgdb) up 7 #7 0xc07383e7 in igmp_slowtimo () at /usr/src/sys/netinet/igmp.c:2184 2184 LIST_FOREACH(igi, &V_igi_head, igi_link) { (kgdb) list 2179 { 2180 struct igmp_ifinfo *igi; 2181 2182 IGMP_LOCK(); 2183 2184 LIST_FOREACH(igi, &V_igi_head, igi_link) { 2185 igmp_v1v2_process_querier_timers(igi); 2186 } 2187 2188 IGMP_UNLOCK(); -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 05:17:15 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84726106564A for ; Fri, 9 Sep 2011 05:17:15 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id B384D8FC12 for ; Fri, 9 Sep 2011 05:17:14 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p895HBDE032876 for ; Fri, 9 Sep 2011 12:17:11 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E69A152.6090408@rdtc.ru> Date: Fri, 09 Sep 2011 12:17:06 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: FreeBSD Stable Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 05:17:15 -0000 Hi! For long time I experience same UFS2 filesystem problems with several 8.2 systems running on gmirror+gjournal+async. In case of unclean shutdown, kernel panic or power failure gjournal makes fsck skip its checks and that's why I use it. But quite often my /var partition (and sometimes others) still has severe damage in it and running with such /var mounted read-write leads to another panics or hangs and so on. For example, I have such 8.2-STABLE system with ad4 and ad6 drives combined to /dev/mirror/gm0. I have just removed ad6 from the mirror, ran fsck -y manually for all its filesystems, shut down this machine again cleanly and booted it next time from ad6 while keeping mirror with ad4 not mounted nor checked. Then, I ran fsck -y /dev/mirror/gm0.journals1e (/var on the mirrored drive) and got LOTS of bad errors on presumably clean file system. Of course, I've seen the same errors while checking ad6 after it was removed from running mirror. I have auto-sync gmirror feature turned ON. I've tried to turn it OFF but that just increase frequency of such damages not fixed after reboot. It seems that gjournal cannot handle system crashes reliably, can it? I basically run in without any manual tuning. I've also tried to tune it - without luck, it works nice when there are no unclean shutdowns but it's here to deal with them in the first place. # fsck -t ffs -y /dev/mirror/gm0.journals1e ** /dev/mirror/gm0.journals1e ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes 3955872 DUP I=989242 3955873 DUP I=989242 3955874 DUP I=989242 3955875 DUP I=989242 3955876 DUP I=989242 3955877 DUP I=989242 3955878 DUP I=989242 3955879 DUP I=989242 3955880 DUP I=989242 3955881 DUP I=989242 3955882 DUP I=989242 EXCESSIVE DUP BLKS I=989242 CONTINUE? yes INCORRECT BLOCK COUNT I=989242 (448 should be 424) CORRECT? yes 3955888 DUP I=989289 3955889 DUP I=989289 3955890 DUP I=989289 3955891 DUP I=989289 3955892 DUP I=989289 3955893 DUP I=989289 3955894 DUP I=989289 3955895 DUP I=989289 ** Phase 1b - Rescan For More DUPS 3955872 DUP I=989242 3955873 DUP I=989242 3955874 DUP I=989242 3955875 DUP I=989242 3955876 DUP I=989242 3955877 DUP I=989242 3955878 DUP I=989242 3955879 DUP I=989242 3955880 DUP I=989242 3955881 DUP I=989242 3955888 DUP I=989242 3955889 DUP I=989242 3955890 DUP I=989242 3955891 DUP I=989242 3955892 DUP I=989242 3955893 DUP I=989242 3955894 DUP I=989242 3955895 DUP I=989242 ** Phase 2 - Check Pathnames DUP/BAD I=989289 OWNER=root MODE=100640 SIZE=14367 MTIME=Sep 9 11:30 2011 FILE=/log/kernel.log REMOVE? yes DUP/BAD I=989242 OWNER=root MODE=100640 SIZE=202631 MTIME=Sep 8 19:52 2011 FILE=/log/mpd.log.0 REMOVE? yes ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts UNREF FILE I=376866 OWNER=root MODE=140666 SIZE=0 MTIME=Sep 5 12:27 2011 CLEAR? yes UNREF FILE I=376868 OWNER=root MODE=140666 UNREF FILE I=376868 OWNER=root MODE=140666 SIZE=0 MTIME=Sep 7 20:30 2011 CLEAR? yes UNREF FILE I=376869 OWNER=root MODE=140666 SIZE=0 MTIME=Sep 8 11:17 2011 CLEAR? yes UNREF FILE I=376870 OWNER=root MODE=140666 SIZE=0 MTIME=Sep 8 12:11 2011 CLEAR? yes BAD/DUP FILE I=989242 OWNER=root MODE=100640 SIZE=202631 MTIME=Sep 8 19:52 2011 CLEAR? yes UNREF FILE I=989259 OWNER=root MODE=100640 SIZE=648 MTIME=Aug 27 00:00 2011 RECONNECT? yes BAD/DUP FILE I=989289 OWNER=root MODE=100640 SIZE=14367 MTIME=Sep 9 11:30 2011 CLEAR? yes LINK COUNT FILE I=989293 OWNER=root MODE=100640 SIZE=961 MTIME=Sep 9 11:26 2011 COUNT 1 SHOULD BE 2 ADJUST? yes UNREF FILE I=989327 OWNER=root MODE=100640 SIZE=114 MTIME=Aug 27 00:00 2011 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 1188 files, 90007 used, 4987072 free (360 frags, 623339 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 08:21:21 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FEC6106566C for ; Fri, 9 Sep 2011 08:21:21 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 3699D8FC0A for ; Fri, 9 Sep 2011 08:21:21 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:14f9:cfdf:caf2:2c18]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 679084AC31; Fri, 9 Sep 2011 12:21:20 +0400 (MSD) Date: Fri, 9 Sep 2011 12:21:15 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <389310276.20110909122115@serebryakov.spb.ru> To: Eugene Grosbein In-Reply-To: <4E69A152.6090408@rdtc.ru> References: <4E69A152.6090408@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Stable Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 08:21:21 -0000 Hello, Eugene. You wrote 9 =F1=E5=ED=F2=FF=E1=F0=FF 2011 =E3., 9:17:06: > # fsck -t ffs -y /dev/mirror/gm0.journals1e I may be wrong, but I've encountered strong advice not to gjournal whole disk, but make gjournal on per-FS basis, many times. And it seems, that your first create big journal, and splice/partition/new= fs it for several FSes. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 09:41:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1525106566B; Fri, 9 Sep 2011 09:41:00 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 5CD628FC1D; Fri, 9 Sep 2011 09:41:00 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p899eukd035726; Fri, 9 Sep 2011 16:40:57 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E69DF23.4040003@rdtc.ru> Date: Fri, 09 Sep 2011 16:40:51 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: lev@freebsd.org References: <4E69A152.6090408@rdtc.ru> <389310276.20110909122115@serebryakov.spb.ru> In-Reply-To: <389310276.20110909122115@serebryakov.spb.ru> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: FreeBSD Stable Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 09:41:01 -0000 09.09.2011 15:21, Lev Serebryakov ÐÉÛÅÔ: > Hello, Eugene. > You wrote 9 ÓÅÎÔÑÂÒÑ 2011 Ç., 9:17:06: > >> # fsck -t ffs -y /dev/mirror/gm0.journals1e > I may be wrong, but I've encountered strong advice not > to gjournal whole disk, but make gjournal on per-FS basis, many times. > And it seems, that your first create big journal, and splice/partition/newfs > it for several FSes. Yes, I did. Shoud not this kind of partitioning work too? Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 10:02:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57751106564A for ; Fri, 9 Sep 2011 10:02:51 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id EA7DA8FC12 for ; Fri, 9 Sep 2011 10:02:50 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:14f9:cfdf:caf2:2c18]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 286734AC31; Fri, 9 Sep 2011 14:02:48 +0400 (MSD) Date: Fri, 9 Sep 2011 14:02:43 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1762406626.20110909140243@serebryakov.spb.ru> To: Eugene Grosbein In-Reply-To: <4E69DF23.4040003@rdtc.ru> References: <4E69A152.6090408@rdtc.ru> <389310276.20110909122115@serebryakov.spb.ru> <4E69DF23.4040003@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Stable Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 10:02:51 -0000 Hello, Eugene. You wrote 9 =D3=C5=CE=D4=D1=C2=D2=D1 2011 =C7., 13:40:51: >>> # fsck -t ffs -y /dev/mirror/gm0.journals1e >> I may be wrong, but I've encountered strong advice not >> to gjournal whole disk, but make gjournal on per-FS basis, many times. >> And it seems, that your first create big journal, and splice/partition/= newfs >> it for several FSes. > Yes, I did. Shoud not this kind of partitioning work too? I'm not sure, should or should not it work. But it is common answer/advice in mailing lists not to do so and to use one gjournal per FS. I think, freebsd-fs@ could give more qualified answer. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 10:31:56 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71694106566B; Fri, 9 Sep 2011 10:31:56 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 81DB28FC15; Fri, 9 Sep 2011 10:31:55 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p89AVs3f036137; Fri, 9 Sep 2011 17:31:54 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E69EB15.50808@rdtc.ru> Date: Fri, 09 Sep 2011 17:31:49 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: FreeBSD Stable References: <4E69A152.6090408@rdtc.ru> In-Reply-To: <4E69A152.6090408@rdtc.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: pjd@freebsd.org Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 10:31:56 -0000 Dear Pawel Jakub, 09.09.2011 12:17, Eugene Grosbein writes: > Hi! > > For long time I experience same UFS2 filesystem problems with several 8.2 systems > running on gmirror+gjournal+async. In case of unclean shutdown, kernel panic or power failure > gjournal makes fsck skip its checks and that's why I use it. > > But quite often my /var partition (and sometimes others) still has severe damage in it > and running with such /var mounted read-write leads to another panics or hangs and so on. > > For example, I have such 8.2-STABLE system with ad4 and ad6 drives combined to /dev/mirror/gm0. > I have just removed ad6 from the mirror, ran fsck -y manually for all its filesystems, > shut down this machine again cleanly and booted it next time from ad6 > while keeping mirror with ad4 not mounted nor checked. > > Then, I ran fsck -y /dev/mirror/gm0.journals1e (/var on the mirrored drive) > and got LOTS of bad errors on presumably clean file system. > Of course, I've seen the same errors while checking ad6 after it was removed from running mirror. > I have auto-sync gmirror feature turned ON. I've tried to turn it OFF but that just > increase frequency of such damages not fixed after reboot. > > It seems that gjournal cannot handle system crashes reliably, can it? > I basically run in without any manual tuning. I've also tried to tune it - without luck, > it works nice when there are no unclean shutdowns but it's here to deal with them in the first place. > > # fsck -t ffs -y /dev/mirror/gm0.journals1e > ** /dev/mirror/gm0.journals1e > ** Last Mounted on /var > ** Phase 1 - Check Blocks and Sizes > 3955872 DUP I=989242 > 3955873 DUP I=989242 > 3955874 DUP I=989242 > 3955875 DUP I=989242 > 3955876 DUP I=989242 > 3955877 DUP I=989242 > 3955878 DUP I=989242 > 3955879 DUP I=989242 > 3955880 DUP I=989242 > 3955881 DUP I=989242 > 3955882 DUP I=989242 > EXCESSIVE DUP BLKS I=989242 > CONTINUE? yes > > INCORRECT BLOCK COUNT I=989242 (448 should be 424) > CORRECT? yes > > 3955888 DUP I=989289 > 3955889 DUP I=989289 > 3955890 DUP I=989289 > 3955891 DUP I=989289 > 3955892 DUP I=989289 > 3955893 DUP I=989289 > 3955894 DUP I=989289 > 3955895 DUP I=989289 > ** Phase 1b - Rescan For More DUPS > 3955872 DUP I=989242 > 3955873 DUP I=989242 > 3955874 DUP I=989242 > 3955875 DUP I=989242 > 3955876 DUP I=989242 > 3955877 DUP I=989242 > 3955878 DUP I=989242 > 3955879 DUP I=989242 > 3955880 DUP I=989242 > 3955881 DUP I=989242 > 3955888 DUP I=989242 > 3955889 DUP I=989242 > 3955890 DUP I=989242 > 3955891 DUP I=989242 > 3955892 DUP I=989242 > 3955893 DUP I=989242 > 3955894 DUP I=989242 > 3955895 DUP I=989242 > ** Phase 2 - Check Pathnames > DUP/BAD I=989289 OWNER=root MODE=100640 > SIZE=14367 MTIME=Sep 9 11:30 2011 > FILE=/log/kernel.log > > REMOVE? yes > > DUP/BAD I=989242 OWNER=root MODE=100640 > SIZE=202631 MTIME=Sep 8 19:52 2011 > FILE=/log/mpd.log.0 > > REMOVE? yes > > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > UNREF FILE I=376866 OWNER=root MODE=140666 > SIZE=0 MTIME=Sep 5 12:27 2011 > CLEAR? yes > > UNREF FILE I=376868 OWNER=root MODE=140666 > > UNREF FILE I=376868 OWNER=root MODE=140666 > SIZE=0 MTIME=Sep 7 20:30 2011 > CLEAR? yes > > UNREF FILE I=376869 OWNER=root MODE=140666 > SIZE=0 MTIME=Sep 8 11:17 2011 > CLEAR? yes > > UNREF FILE I=376870 OWNER=root MODE=140666 > SIZE=0 MTIME=Sep 8 12:11 2011 > CLEAR? yes > > BAD/DUP FILE I=989242 OWNER=root MODE=100640 > SIZE=202631 MTIME=Sep 8 19:52 2011 > CLEAR? yes > > UNREF FILE I=989259 OWNER=root MODE=100640 > SIZE=648 MTIME=Aug 27 00:00 2011 > RECONNECT? yes > > BAD/DUP FILE I=989289 OWNER=root MODE=100640 > SIZE=14367 MTIME=Sep 9 11:30 2011 > CLEAR? yes > LINK COUNT FILE I=989293 OWNER=root MODE=100640 > SIZE=961 MTIME=Sep 9 11:26 2011 COUNT 1 SHOULD BE 2 > ADJUST? yes > > UNREF FILE I=989327 OWNER=root MODE=100640 > SIZE=114 MTIME=Aug 27 00:00 2011 > RECONNECT? yes > > ** Phase 5 - Check Cyl groups > FREE BLK COUNT(S) WRONG IN SUPERBLK > SALVAGE? yes > > SUMMARY INFORMATION BAD > SALVAGE? yes > > BLK(S) MISSING IN BIT MAPS > SALVAGE? yes > > 1188 files, 90007 used, 4987072 free (360 frags, 623339 blocks, 0.0% > fragmentation) > > ***** FILE SYSTEM IS CLEAN ***** > > ***** FILE SYSTEM WAS MODIFIED ***** Please explain if such partitioning is supported? physical drive - geom_mirror - geom_journal - geom_part_mbr - geom_part_bsd - journalled UFS2 If not, mounting such UFS2 should warn us, shouldn't it? No warnings now. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 13:53:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 461AE106564A for ; Fri, 9 Sep 2011 13:53:12 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 005D78FC1A for ; Fri, 9 Sep 2011 13:53:11 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 867BEED9; Fri, 9 Sep 2011 15:35:34 +0200 (CEST) Date: Fri, 9 Sep 2011 15:35:10 +0200 From: Pawel Jakub Dawidek To: Eugene Grosbein Message-ID: <20110909133509.GH1639@garage.freebsd.pl> References: <4E69A152.6090408@rdtc.ru> <4E69EB15.50808@rdtc.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cfJ13FhsvNR/yOpm" Content-Disposition: inline In-Reply-To: <4E69EB15.50808@rdtc.ru> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Stable Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 13:53:12 -0000 --cfJ13FhsvNR/yOpm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 09, 2011 at 05:31:49PM +0700, Eugene Grosbein wrote: > Please explain if such partitioning is supported? > physical drive - geom_mirror - geom_journal - geom_part_mbr - geom_part_b= sd - journalled UFS2 No. It will only work properly for journaling UFS if UFS is placed directory on gjournal provider. You configured slices and several pratitions on one gjournal provider, which simply cannot work, as one UFS file system has to talk to one gjournal provider. > If not, mounting such UFS2 should warn us, shouldn't it? > No warnings now. It might be a bit hard to tell, but it could be at least better documented. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --cfJ13FhsvNR/yOpm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk5qFg0ACgkQForvXbEpPzRu4wCgpUOTSMEQukLvnrKRy+fAJjbF nOsAoM76Jns0/D8A+4MBOCyprbSYbYfZ =eDrO -----END PGP SIGNATURE----- --cfJ13FhsvNR/yOpm-- From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 17:53:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89AF7106566C; Fri, 9 Sep 2011 17:53:01 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2CB218FC12; Fri, 9 Sep 2011 17:53:01 +0000 (UTC) Received: by gyf2 with SMTP id 2so2172072gyf.13 for ; Fri, 09 Sep 2011 10:53:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yoA1oyYTkF70Y5K6yza2ITuihX68VuDHgBaWjSj9q4k=; b=Fu8E59z5WfVx1xFrK3Omq+CWnv7zvYO+bxUUGomWavTbTnRKxcovaT1rGiLbXywrXY WZW5I3bjub4ALyAKwCK6hR7T2lRvj0qKZpY0etdRWasMQu1ViYId0GyUavVIRdYhSh8o xjIiH/wtpStMLY8uK57WvMkUJF8AJDBuu7RuQ= MIME-Version: 1.0 Received: by 10.42.73.5 with SMTP id q5mr759520icj.100.1315590780380; Fri, 09 Sep 2011 10:53:00 -0700 (PDT) Received: by 10.231.36.69 with HTTP; Fri, 9 Sep 2011 10:53:00 -0700 (PDT) In-Reply-To: <20110909133509.GH1639@garage.freebsd.pl> References: <4E69A152.6090408@rdtc.ru> <4E69EB15.50808@rdtc.ru> <20110909133509.GH1639@garage.freebsd.pl> Date: Fri, 9 Sep 2011 10:53:00 -0700 Message-ID: From: Kevin Oberman To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Stable , Eugene Grosbein Subject: Re: gmirror+gjournal often makes inconsistens file systems X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 17:53:01 -0000 On Fri, Sep 9, 2011 at 6:35 AM, Pawel Jakub Dawidek wrote: > On Fri, Sep 09, 2011 at 05:31:49PM +0700, Eugene Grosbein wrote: >> Please explain if such partitioning is supported? >> physical drive - geom_mirror - geom_journal - geom_part_mbr - geom_part_bsd - journalled UFS2 > > No. It will only work properly for journaling UFS if UFS is placed > directory on gjournal provider. You configured slices and several > pratitions on one gjournal provider, which simply cannot work, as > one UFS file system has to talk to one gjournal provider. > >> If not, mounting such UFS2 should warn us, shouldn't it? >> No warnings now. > > It might be a bit hard to tell, but it could be at least better > documented. Yes, the documentation could be better, especially since the gjournal(8) man page example where the entire disk (da0) is tied to a single journal. At least change the example to read: gjournal load gjournal label da0p5 newfs -J /dev/da0p5.journal mount -o async /dev/da05.journal /mnt It MIGHT be better to use da0s1g, but I think we want to move toward GPT, so I suggest that type of example. Either make it clear that a file system, not a drive, is the appropriate application. I am quite aware of this as I just created my first gjournal file system last night and was briefly confused by this. -- R. Kevin Oberman, Network Engineer - Retired E-mail: kob6558@gmail.com From owner-freebsd-stable@FreeBSD.ORG Fri Sep 9 20:10:46 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE5351065670; Fri, 9 Sep 2011 20:10:45 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2EEB68FC13; Fri, 9 Sep 2011 20:10:44 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p89KASb9005483; Sat, 10 Sep 2011 05:10:38 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p89KARbm026576; Sat, 10 Sep 2011 05:10:28 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Sat, 10 Sep 2011 04:48:41 +0900 (JST) Message-Id: <20110910.044841.232160047547388224.hrs@allbsd.org> To: pjd@FreeBSD.org, mm@FreeBSD.org, freebsd-stable@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110907.094717.2272609566853905102.hrs@allbsd.org> References: <20110903.071908.971549835606878048.hrs@allbsd.org> <20110907.094717.2272609566853905102.hrs@allbsd.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3.51 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Sat, 10 Sep 2011 05:10:42 +0900 (JST) X-Spam-Status: No, score=-104.6 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: attilio@FreeBSD.org, kib@FreeBSD.org Subject: ZFS panic on a RELENG_8 NFS server (Was: panic: spin lock held too long (RELENG_8 from today)) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 20:10:46 -0000 Hiroki Sato wrote in <20110907.094717.2272609566853905102.hrs@allbsd.org>: hr> During this investigation an disk has to be replaced and resilvering hr> it is now in progress. A deadlock and a forced reboot after that hr> make recovering of the zfs datasets take a long time (for committing hr> logs, I think), so I will try to reproduce the deadlock and get a hr> core dump after it finished. I think I could reproduce the symptoms. I have no idea about if these are exactly the same as occurred on my box before because the kernel was replaced with one with some debugging options, but these are reproducible at least. There are two symptoms. One is a panic. A DDB output when the panic occurred is the following: ---- Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x100000040 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8065b926 stack pointer = 0x28:0xffffff8257b94d70 frame pointer = 0x28:0xffffff8257b94e10 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 992 (nfsd: service) [thread pid 992 tid 100586 ] Stopped at witness_checkorder+0x246: movl 0x40(%r13),%ebx db> bt Tracing pid 992 tid 100586 td 0xffffff00595d9000 witness_checkorder() at witness_checkorder+0x246 _sx_slock() at _sx_slock+0x35 dmu_bonus_hold() at dmu_bonus_hold+0x57 zfs_zget() at zfs_zget+0x237 zfs_dirent_lock() at zfs_dirent_lock+0x488 zfs_dirlook() at zfs_dirlook+0x69 zfs_lookup() at zfs_lookup+0x26b zfs_freebsd_lookup() at zfs_freebsd_lookup+0x81 vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40 lookup() at lookup+0x384 nfsvno_namei() at nfsvno_namei+0x268 nfsrvd_lookup() at nfsrvd_lookup+0xd6 nfsrvd_dorpc() at nfsrvd_dorpc+0x745 nfssvc_program() at nfssvc_program+0x447 svc_run_internal() at svc_run_internal+0x51b svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x11d fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x8006a031c, rsp = 0x7fffffffe6c8, rbp = 0x6 --- ---- The complete output can be found at: http://people.allbsd.org/~hrs/zfs_panic_20110909_1/pool-zfs-20110909-1.txt Another is getting stuck at ZFS access. The kernel is running with no panic but any access to ZFS datasets causes a program non-responsive. The DDB output can be found at: http://people.allbsd.org/~hrs/zfs_panic_20110909_2/pool-zfs-20110909-2.txt The trigger for the both was some access to a ZFS dataset from the NFS clients. Because the access pattern was complex I could not narrow down what was the culprit, but it seems timing-dependent and simply doing "rm -rf" locally on the server can sometimes trigger them. The crash dump and the kernel can be found at the following URLs: panic: http://people.allbsd.org/~hrs/zfs_panic_20110909_1/ no panic but unresponsive: http://people.allbsd.org/~hrs/zfs_panic_20110909_2/ kernel: http://people.allbsd.org/~hrs/zfs_panic_20110909_kernel/ -- Hiroki From owner-freebsd-stable@FreeBSD.ORG Sat Sep 10 19:20:45 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBE651065676 for ; Sat, 10 Sep 2011 19:20:45 +0000 (UTC) (envelope-from yshd.snd@gmail.com) Received: from mail-pz0-f45.google.com (mail-pz0-f45.google.com [209.85.210.45]) by mx1.freebsd.org (Postfix) with ESMTP id AFA0A8FC0A for ; Sat, 10 Sep 2011 19:20:45 +0000 (UTC) Received: by pzk33 with SMTP id 33so15255251pzk.18 for ; Sat, 10 Sep 2011 12:20:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type; bh=gNeMcTalVERV8nhjVKSKPqgw9Fe6z/6qxxjdwPBfiEg=; b=S4T2ewqyFCDWrWpDrTZU4L+R4QDrqBDa/8FabSKcuUBpEuNE2RnlzxlIXdEgCkQKxg 1auCrAiu3/Tv0Ze4pt2lLHi6fQgjsXSpOWkcoHDNP1ykyRiy4AGh22cVdoqMRAyf8n0S pfBImew7sHzN+2NsuoJtWAcO+azBt1h9kaPRU= Received: by 10.68.7.194 with SMTP id l2mr2520510pba.397.1315680775318; Sat, 10 Sep 2011 11:52:55 -0700 (PDT) Received: from pc036.ys.sokohiki.org (p12124-ipngn701hodogaya.kanagawa.ocn.ne.jp. [114.175.63.124]) by mx.google.com with ESMTPS id i3sm21525695pbg.10.2011.09.10.11.52.52 (version=SSLv3 cipher=OTHER); Sat, 10 Sep 2011 11:52:53 -0700 (PDT) Sender: Yoshihide Sonoda Message-ID: <4E6BB202.1080700@na.rim.or.jp> Date: Sun, 11 Sep 2011 03:52:50 +0900 From: Yoshihide Sonoda User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: multipart/mixed; boundary="------------020105040408020609090904" Subject: ahci(4) + zfs chksum error with Marvell SATA controller X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Sep 2011 19:20:45 -0000 This is a multi-part message in MIME format. --------------020105040408020609090904 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Hi all, I have a problem about ahci(4) and zfs. I'm using a mother board, ASUS P8Z68-V PRO which has two AHCI controllers, and these are Intel Cougar Point AHCI SATA controller and Marvell 88SE9172 AHCI SATA controller. In addition, I connected a HighPoint RocketRAID 620 to PCI-Express x 1 slot of the M/B. I attach dmesg and pciconf -lvb. The Intel Cougar Point controller works without any problem. However, Marvell 88SE9172 and HighPoint RocketRAID 620 are very unstable. Marvell 88SE9172 and HighPoint Rocket 620 behave almost same. Then ZFS checksum errors often occur on disks connected to HighPoint and Marvell. The error does not occur when I connect the same disk to the Intel controller. In the following case, the pool zp2 is configured with the disks connected to HighPoint controller. Other pools are connected to Intel controller. As shown below, any READ and WRITE errors have never occurred. However, checksum errors occur very often. I suspect ahci(4) driver. But I have no idea how to fix it. Please help me. ---- # zpool status pool: zp0 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zp0 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/wd2tb-0 ONLINE 0 0 0 gpt/wd2tb-1 ONLINE 0 0 0 errors: No known data errors pool: zp1 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zp1 ONLINE 0 0 0 gpt/wd500gb-0 ONLINE 0 0 0 gpt/wd500gb-1 ONLINE 0 0 0 errors: No known data errors pool: zp2 state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scan: scrub repaired 0 in 1h42m with 13 errors on Sat Sep 10 02:59:03 2011 config: NAME STATE READ WRITE CKSUM zp2 ONLINE 0 0 26 mirror-0 ONLINE 0 0 52 gpt/wd3tb-2 ONLINE 0 0 69 gpt/wd3tb-3 ONLINE 0 0 55 errors: 13 data errors, use '-v' for a list pool: zpeagle state: ONLINE scan: scrub repaired 0 in 8h23m with 0 errors on Fri Aug 19 16:13:49 2011 config: NAME STATE READ WRITE CKSUM zpeagle ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/a45b41cd-9512-11e0-a493-f46d0449b073 ONLINE 0 0 0 gptid/af6665e2-9512-11e0-a493-f46d0449b073 ONLINE 0 0 0 errors: No known data errors --- Yoshihide Sonoda --------------020105040408020609090904 Content-Type: text/plain; name="dmesg.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmesg.txt" Q29weXJpZ2h0IChjKSAxOTkyLTIwMTEgVGhlIEZyZWVCU0QgUHJvamVjdC4KQ29weXJpZ2h0 IChjKSAxOTc5LCAxOTgwLCAxOTgzLCAxOTg2LCAxOTg4LCAxOTg5LCAxOTkxLCAxOTkyLCAx OTkzLCAxOTk0CglUaGUgUmVnZW50cyBvZiB0aGUgVW5pdmVyc2l0eSBvZiBDYWxpZm9ybmlh LiBBbGwgcmlnaHRzIHJlc2VydmVkLgpGcmVlQlNEIGlzIGEgcmVnaXN0ZXJlZCB0cmFkZW1h cmsgb2YgVGhlIEZyZWVCU0QgRm91bmRhdGlvbi4KRnJlZUJTRCA4LjItU1RBQkxFICMzNzog VGh1IFNlcCAgOCAwMjo0NjozMSBKU1QgMjAxMQogICAgcm9vdEBlYWdsZS55cy5zb2tvaGlr aS5vcmc6L3Vzci9vYmovdXNyL3NyYy9zeXMvZWFnbGUgYW1kNjQKVGltZWNvdW50ZXIgImk4 MjU0IiBmcmVxdWVuY3kgMTE5MzE4MiBIeiBxdWFsaXR5IDAKQ1BVOiBJbnRlbChSKSBDb3Jl KFRNKSBpNy0yNjAwIENQVSBAIDMuNDBHSHogKDM0MTEuMTUtTUh6IEs4LWNsYXNzIENQVSkK ICBPcmlnaW4gPSAiR2VudWluZUludGVsIiAgSWQgPSAweDIwNmE3ICBGYW1pbHkgPSA2ICBN b2RlbCA9IDJhICBTdGVwcGluZyA9IDcKICBGZWF0dXJlcz0weGJmZWJmYmZmPEZQVSxWTUUs REUsUFNFLFRTQyxNU1IsUEFFLE1DRSxDWDgsQVBJQyxTRVAsTVRSUixQR0UsTUNBLENNT1Ys UEFULFBTRTM2LENMRkxVU0gsRFRTLEFDUEksTU1YLEZYU1IsU1NFLFNTRTIsU1MsSFRULFRN LFBCRT4KICBGZWF0dXJlczI9MHgxN2JhZTNmZjxTU0UzLFBDTE1VTFFEUSxEVEVTNjQsTU9O LERTX0NQTCxWTVgsU01YLEVTVCxUTTIsU1NTRTMsQ1gxNix4VFBSLFBEQ00sUENJRCxTU0U0 LjEsU1NFNC4yLHgyQVBJQyxQT1BDTlQsVFNDRExULEFFU05JLFhTQVZFLEFWWD4KICBBTUQg RmVhdHVyZXM9MHgyODEwMDgwMDxTWVNDQUxMLE5YLFJEVFNDUCxMTT4KICBBTUQgRmVhdHVy ZXMyPTB4MTxMQUhGPgogIFRTQzogUC1zdGF0ZSBpbnZhcmlhbnQKcmVhbCBtZW1vcnkgID0g MTcxNzk4NjkxODQgKDE2Mzg0IE1CKQphdmFpbCBtZW1vcnkgPSAxNjQzMDcwMjU5MiAoMTU2 NjkgTUIpCkFDUEkgQVBJQyBUYWJsZTogPEFMQVNLQSBBIE0gST4KRnJlZUJTRC9TTVA6IE11 bHRpcHJvY2Vzc29yIFN5c3RlbSBEZXRlY3RlZDogOCBDUFVzCkZyZWVCU0QvU01QOiAxIHBh Y2thZ2UocykgeCA0IGNvcmUocykgeCAyIFNNVCB0aHJlYWRzCiBjcHUwIChCU1ApOiBBUElD IElEOiAgMAogY3B1MSAoQVApOiBBUElDIElEOiAgMQogY3B1MiAoQVApOiBBUElDIElEOiAg MgogY3B1MyAoQVApOiBBUElDIElEOiAgMwogY3B1NCAoQVApOiBBUElDIElEOiAgNAogY3B1 NSAoQVApOiBBUElDIElEOiAgNQogY3B1NiAoQVApOiBBUElDIElEOiAgNgogY3B1NyAoQVAp OiBBUElDIElEOiAgNwppb2FwaWMwIDxWZXJzaW9uIDIuMD4gaXJxcyAwLTIzIG9uIG1vdGhl cmJvYXJkCmtiZDEgYXQga2JkbXV4MAphY3BpMDogPEFMQVNLQSBBIE0gST4gb24gbW90aGVy Ym9hcmQKYWNwaTA6IFtJVEhSRUFEXQpBQ1BJIEVycm9yOiBbUkFNQl0gTmFtZXNwYWNlIGxv b2t1cCBmYWlsdXJlLCBBRV9OT1RfRk9VTkQgKDIwMTAxMDEzL3BzYXJncy00NjQpCkFDUEkg RXhjZXB0aW9uOiBBRV9OT1RfRk9VTkQsIENvdWxkIG5vdCBleGVjdXRlIGFyZ3VtZW50cyBm b3IgW1JBTVddIChSZWdpb24pICgyMDEwMTAxMy9uc2luaXQtNDUyKQphY3BpMDogUG93ZXIg QnV0dG9uIChmaXhlZCkKVGltZWNvdW50ZXIgIkFDUEktZmFzdCIgZnJlcXVlbmN5IDM1Nzk1 NDUgSHogcXVhbGl0eSAxMDAwCmFjcGlfdGltZXIwOiA8MjQtYml0IHRpbWVyIGF0IDMuNTc5 NTQ1TUh6PiBwb3J0IDB4NDA4LTB4NDBiIG9uIGFjcGkwCmNwdTA6IDxBQ1BJIENQVT4gb24g YWNwaTAKY3B1MTogPEFDUEkgQ1BVPiBvbiBhY3BpMApjcHUyOiA8QUNQSSBDUFU+IG9uIGFj cGkwCmNwdTM6IDxBQ1BJIENQVT4gb24gYWNwaTAKY3B1NDogPEFDUEkgQ1BVPiBvbiBhY3Bp MApjcHU1OiA8QUNQSSBDUFU+IG9uIGFjcGkwCmNwdTY6IDxBQ1BJIENQVT4gb24gYWNwaTAK Y3B1NzogPEFDUEkgQ1BVPiBvbiBhY3BpMAphY3BpX2VjMDogPEVtYmVkZGVkIENvbnRyb2xs ZXI6IEdQRSAweDE4PiBwb3J0IDB4NjIsMHg2NiBvbiBhY3BpMApwY2liMDogPEFDUEkgSG9z dC1QQ0kgYnJpZGdlPiBwb3J0IDB4Y2Y4LTB4Y2ZmIG9uIGFjcGkwCnBjaTA6IDxBQ1BJIFBD SSBidXM+IG9uIHBjaWIwCnBjaWIxOiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gaXJxIDE2IGF0 IGRldmljZSAxLjAgb24gcGNpMApwY2kxOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMQp2Z2Fw Y2kwOiA8VkdBLWNvbXBhdGlibGUgZGlzcGxheT4gcG9ydCAweGYwMDAtMHhmMDNmIG1lbSAw eGZiNDAwMDAwLTB4ZmI3ZmZmZmYsMHhkMDAwMDAwMC0weGRmZmZmZmZmIGlycSAxNiBhdCBk ZXZpY2UgMi4wIG9uIHBjaTAKcGNpMDogPHNpbXBsZSBjb21tcz4gYXQgZGV2aWNlIDIyLjAg KG5vIGRyaXZlciBhdHRhY2hlZCkKZW0wOiA8SW50ZWwoUikgUFJPLzEwMDAgTmV0d29yayBD b25uZWN0aW9uIDcuMi4zPiBwb3J0IDB4ZjA4MC0weGYwOWYgbWVtIDB4ZmJmMDAwMDAtMHhm YmYxZmZmZiwweGZiZjI4MDAwLTB4ZmJmMjhmZmYgaXJxIDE4IGF0IGRldmljZSAyNS4wIG9u IHBjaTAKZW0wOiBVc2luZyBhbiBNU0kgaW50ZXJydXB0CmVtMDogW0ZJTFRFUl0KZW0wOiBF dGhlcm5ldCBhZGRyZXNzOiBmNDo2ZDowNDo0OTpiMDo3MwplaGNpMDogPEVIQ0kgKGdlbmVy aWMpIFVTQiAyLjAgY29udHJvbGxlcj4gbWVtIDB4ZmJmMjcwMDAtMHhmYmYyNzNmZiBpcnEg MjMgYXQgZGV2aWNlIDI2LjAgb24gcGNpMAplaGNpMDogW0lUSFJFQURdCnVzYnVzMDogRUhD SSB2ZXJzaW9uIDEuMAp1c2J1czA6IDxFSENJIChnZW5lcmljKSBVU0IgMi4wIGNvbnRyb2xs ZXI+IG9uIGVoY2kwCmhkYWMwOiA8SW50ZWwgQ291Z2FyIFBvaW50IEhpZ2ggRGVmaW5pdGlv biBBdWRpbyBDb250cm9sbGVyPiBtZW0gMHhmYmYyMDAwMC0weGZiZjIzZmZmIGlycSAyMiBh dCBkZXZpY2UgMjcuMCBvbiBwY2kwCmhkYWMwOiBIREEgRHJpdmVyIFJldmlzaW9uOiAyMDEw MDIyNl8wMTQyCmhkYWMwOiBbSVRIUkVBRF0KcGNpYjI6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdl PiBpcnEgMTcgYXQgZGV2aWNlIDI4LjAgb24gcGNpMApwY2kyOiA8QUNQSSBQQ0kgYnVzPiBv biBwY2liMgphaGNpMDogPEhpZ2hQb2ludCBSb2NrZXRSQUlEIDYyMCBBSENJIFNBVEEgY29u dHJvbGxlcj4gcG9ydCAweGUwOTAtMHhlMDk3LDB4ZTA4MC0weGUwODMsMHhlMDcwLTB4ZTA3 NywweGUwNjAtMHhlMDYzLDB4ZTA1MC0weGUwNWYgbWVtIDB4ZmJlMjEwMDAtMHhmYmUyMTdm ZiBpcnEgMTYgYXQgZGV2aWNlIDAuMCBvbiBwY2kyCmFoY2kwOiBbSVRIUkVBRF0KYWhjaTA6 IEFIQ0kgdjEuMDAgd2l0aCAyIDZHYnBzIHBvcnRzLCBQb3J0IE11bHRpcGxpZXIgc3VwcG9y dGVkIHdpdGggRkJTCmFoY2ljaDA6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgMCBvbiBh aGNpMAphaGNpY2gwOiBbSVRIUkVBRF0KYWhjaWNoMTogPEFIQ0kgY2hhbm5lbD4gYXQgY2hh bm5lbCAxIG9uIGFoY2kwCmFoY2ljaDE6IFtJVEhSRUFEXQphdGFwY2kwOiA8TWFydmVsbCA4 OFNFOTEyeCBVRE1BMTMzIGNvbnRyb2xsZXI+IHBvcnQgMHhlMDQwLTB4ZTA0NywweGUwMzAt MHhlMDMzLDB4ZTAyMC0weGUwMjcsMHhlMDEwLTB4ZTAxMywweGUwMDAtMHhlMDBmIG1lbSAw eGZiZTIwMDAwLTB4ZmJlMjAwMGYgaXJxIDE3IGF0IGRldmljZSAwLjEgb24gcGNpMgpwY2li MzogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSAxNiBhdCBkZXZpY2UgMjguMSBvbiBwY2kw CnBjaTM6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIzCnhoY2kwOiA8WEhDSSAoZ2VuZXJpYykg VVNCIDMuMCBjb250cm9sbGVyPiBtZW0gMHhmYmQwMDAwMC0weGZiZDA3ZmZmIGlycSAxNyBh dCBkZXZpY2UgMC4wIG9uIHBjaTMKeGhjaTA6IFtJVEhSRUFEXQp4aGNpMDogMzIgYnl0ZSBj b250ZXh0IHNpemUuCnVzYnVzMSBvbiB4aGNpMApwY2liNDogPEFDUEkgUENJLVBDSSBicmlk Z2U+IGlycSAxOCBhdCBkZXZpY2UgMjguMiBvbiBwY2kwCnBjaTQ6IDxBQ1BJIFBDSSBidXM+ IG9uIHBjaWI0CmVtMTogPEludGVsKFIpIFBSTy8xMDAwIE5ldHdvcmsgQ29ubmVjdGlvbiA3 LjIuMz4gcG9ydCAweGQwMDAtMHhkMDFmIG1lbSAweGZiYzQwMDAwLTB4ZmJjNWZmZmYsMHhm YmMyMDAwMC0weGZiYzNmZmZmIGlycSAxOCBhdCBkZXZpY2UgMC4wIG9uIHBjaTQKZW0xOiBV c2luZyBhbiBNU0kgaW50ZXJydXB0CmVtMTogW0ZJTFRFUl0KZW0xOiBFdGhlcm5ldCBhZGRy ZXNzOiAwMDoxYjoyMToxNDpkNjo0MgpwY2liNTogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGly cSAxOSBhdCBkZXZpY2UgMjguMyBvbiBwY2kwCnBjaTU6IDxBQ1BJIFBDSSBidXM+IG9uIHBj aWI1CmF0YXBjaTE6IDxKTWljcm9uIEFUQSBjb250cm9sbGVyPiBwb3J0IDB4YzA0MC0weGMw NDcsMHhjMDMwLTB4YzAzMywweGMwMjAtMHhjMDI3LDB4YzAxMC0weGMwMTMsMHhjMDAwLTB4 YzAwZiBtZW0gMHhmYmIxMDAwMC0weGZiYjEwMWZmIGlycSAxOSBhdCBkZXZpY2UgMC4wIG9u IHBjaTUKYXRhcGNpMTogW0lUSFJFQURdCmF0YTI6IDxBVEEgY2hhbm5lbCAwPiBvbiBhdGFw Y2kxCmF0YTI6IFtJVEhSRUFEXQphdGEzOiA8QVRBIGNoYW5uZWwgMT4gb24gYXRhcGNpMQph dGEzOiBbSVRIUkVBRF0KcGNpYjY6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMTcgYXQg ZGV2aWNlIDI4LjQgb24gcGNpMApwY2k2OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liNgp4aGNp MTogPFhIQ0kgKGdlbmVyaWMpIFVTQiAzLjAgY29udHJvbGxlcj4gbWVtIDB4ZmJhMDAwMDAt MHhmYmEwN2ZmZiBpcnEgMTYgYXQgZGV2aWNlIDAuMCBvbiBwY2k2CnhoY2kxOiBbSVRIUkVB RF0KeGhjaTE6IDMyIGJ5dGUgY29udGV4dCBzaXplLgp1c2J1czIgb24geGhjaTEKcGNpYjc6 IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMTggYXQgZGV2aWNlIDI4LjYgb24gcGNpMApw Y2k3OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liNwpwY2liODogPEFDUEkgUENJLVBDSSBicmlk Z2U+IGlycSAxOCBhdCBkZXZpY2UgMC4wIG9uIHBjaTcKcGNpODogPEFDUEkgUENJIGJ1cz4g b24gcGNpYjgKZndvaGNpMDogPFZJQSBGaXJlIElJIChWVDYzMDYpPiBwb3J0IDB4YjAwMC0w eGIwN2YgbWVtIDB4ZmI5MDAwMDAtMHhmYjkwMDdmZiBpcnEgMTcgYXQgZGV2aWNlIDIuMCBv biBwY2k4CmZ3b2hjaTA6IFtJVEhSRUFEXQpmd29oY2kwOiBPSENJIHZlcnNpb24gMS4xMCAo Uk9NPTEpCmZ3b2hjaTA6IE5vLiBvZiBJc29jaHJvbm91cyBjaGFubmVscyBpcyA0Lgpmd29o Y2kwOiBFVUk2NCAwMDoxZjpjNjowMDowMDoxMjphZjo3Zgpmd29oY2kwOiBQaHkgMTM5NGEg YXZhaWxhYmxlIFM0MDAsIDIgcG9ydHMuCmZ3b2hjaTA6IExpbmsgUzQwMCwgbWF4X3JlYyAy MDQ4IGJ5dGVzLgpmaXJld2lyZTA6IDxJRUVFMTM5NChGaXJlV2lyZSkgYnVzPiBvbiBmd29o Y2kwCmZ3ZTA6IDxFdGhlcm5ldCBvdmVyIEZpcmVXaXJlPiBvbiBmaXJld2lyZTAKaWZfZndl MDogRmFrZSBFdGhlcm5ldCBhZGRyZXNzOiAwMjoxZjpjNjoxMjphZjo3Zgpmd2UwOiBFdGhl cm5ldCBhZGRyZXNzOiAwMjoxZjpjNjoxMjphZjo3Zgpmd2lwMDogPElQIG92ZXIgRmlyZVdp cmU+IG9uIGZpcmV3aXJlMApmd2lwMDogRmlyZXdpcmUgYWRkcmVzczogMDA6MWY6YzY6MDA6 MDA6MTI6YWY6N2YgQCAweGZmZmUwMDAwMDAwMCwgUzQwMCwgbWF4cmVjIDIwNDgKZGNvbnNf Y3JvbTA6IDxkY29ucyBjb25maWd1cmF0aW9uIFJPTT4gb24gZmlyZXdpcmUwCmRjb25zX2Ny b20wOiBidXNfYWRkciAweGNhNzQ4MDAwCmZ3b2hjaTA6IEluaXRpYXRlIGJ1cyByZXNldApm d29oY2kwOiBmd29oY2lfaW50cl9jb3JlOiBCVVMgcmVzZXQKZndvaGNpMDogZndvaGNpX2lu dHJfY29yZTogbm9kZV9pZD0weDAwMDAwMDAwLCBTZWxmSUQgQ291bnQ9MSwgQ1lDTEVNQVNU RVIgbW9kZQpwY2liOTogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSAxOSBhdCBkZXZpY2Ug MjguNyBvbiBwY2kwCnBjaTk6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWI5CmFoY2kxOiA8TWFy dmVsbCA4OFNFOTE3MiBBSENJIFNBVEEgY29udHJvbGxlcj4gcG9ydCAweGEwNDAtMHhhMDQ3 LDB4YTAzMC0weGEwMzMsMHhhMDIwLTB4YTAyNywweGEwMTAtMHhhMDEzLDB4YTAwMC0weGEw MGYgbWVtIDB4ZmI4MTAwMDAtMHhmYjgxMDFmZiBpcnEgMTkgYXQgZGV2aWNlIDAuMCBvbiBw Y2k5CmFoY2kxOiBbSVRIUkVBRF0KYWhjaTE6IEFIQ0kgdjEuMDAgd2l0aCAyIDZHYnBzIHBv cnRzLCBQb3J0IE11bHRpcGxpZXIgc3VwcG9ydGVkIHdpdGggRkJTCmFoY2ljaDI6IDxBSENJ IGNoYW5uZWw+IGF0IGNoYW5uZWwgMCBvbiBhaGNpMQphaGNpY2gyOiBbSVRIUkVBRF0KYWhj aWNoMzogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCAxIG9uIGFoY2kxCmFoY2ljaDM6IFtJ VEhSRUFEXQplaGNpMTogPEVIQ0kgKGdlbmVyaWMpIFVTQiAyLjAgY29udHJvbGxlcj4gbWVt IDB4ZmJmMjYwMDAtMHhmYmYyNjNmZiBpcnEgMjMgYXQgZGV2aWNlIDI5LjAgb24gcGNpMApl aGNpMTogW0lUSFJFQURdCnVzYnVzMzogRUhDSSB2ZXJzaW9uIDEuMAp1c2J1czM6IDxFSENJ IChnZW5lcmljKSBVU0IgMi4wIGNvbnRyb2xsZXI+IG9uIGVoY2kxCmlzYWIwOiA8UENJLUlT QSBicmlkZ2U+IGF0IGRldmljZSAzMS4wIG9uIHBjaTAKaXNhMDogPElTQSBidXM+IG9uIGlz YWIwCmFoY2kyOiA8SW50ZWwgQ291Z2FyIFBvaW50IEFIQ0kgU0FUQSBjb250cm9sbGVyPiBw b3J0IDB4ZjBkMC0weGYwZDcsMHhmMGMwLTB4ZjBjMywweGYwYjAtMHhmMGI3LDB4ZjBhMC0w eGYwYTMsMHhmMDYwLTB4ZjA3ZiBtZW0gMHhmYmYyNTAwMC0weGZiZjI1N2ZmIGlycSAyMCBh dCBkZXZpY2UgMzEuMiBvbiBwY2kwCmFoY2kyOiBbSVRIUkVBRF0KYWhjaTI6IEFIQ0kgdjEu MzAgd2l0aCA2IDZHYnBzIHBvcnRzLCBQb3J0IE11bHRpcGxpZXIgbm90IHN1cHBvcnRlZAph aGNpY2g0OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDAgb24gYWhjaTIKYWhjaWNoNDog W0lUSFJFQURdCmFoY2ljaDU6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgMSBvbiBhaGNp MgphaGNpY2g1OiBbSVRIUkVBRF0KYWhjaWNoNjogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5l bCAyIG9uIGFoY2kyCmFoY2ljaDY6IFtJVEhSRUFEXQphaGNpY2g3OiA8QUhDSSBjaGFubmVs PiBhdCBjaGFubmVsIDMgb24gYWhjaTIKYWhjaWNoNzogW0lUSFJFQURdCmFoY2ljaDg6IDxB SENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgNCBvbiBhaGNpMgphaGNpY2g4OiBbSVRIUkVBRF0K YWhjaWNoOTogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCA1IG9uIGFoY2kyCmFoY2ljaDk6 IFtJVEhSRUFEXQpwY2kwOiA8c2VyaWFsIGJ1cywgU01CdXM+IGF0IGRldmljZSAzMS4zIChu byBkcml2ZXIgYXR0YWNoZWQpCmFjcGlfYnV0dG9uMDogPFBvd2VyIEJ1dHRvbj4gb24gYWNw aTAKYWNwaV9ocGV0MDogPEhpZ2ggUHJlY2lzaW9uIEV2ZW50IFRpbWVyPiBpb21lbSAweGZl ZDAwMDAwLTB4ZmVkMDAzZmYgb24gYWNwaTAKVGltZWNvdW50ZXIgIkhQRVQiIGZyZXF1ZW5j eSAxNDMxODE4MCBIeiBxdWFsaXR5IDkwMAphdHJ0YzA6IDxBVCByZWFsdGltZSBjbG9jaz4g cG9ydCAweDcwLTB4NzEgaXJxIDggb24gYWNwaTAKb3JtMDogPElTQSBPcHRpb24gUk9NPiBh dCBpb21lbSAweGQ1MDAwLTB4ZDVmZmYgb24gaXNhMApzYzA6IDxTeXN0ZW0gY29uc29sZT4g YXQgZmxhZ3MgMHgxMDAgb24gaXNhMApzYzA6IFZHQSA8MTYgdmlydHVhbCBjb25zb2xlcywg ZmxhZ3M9MHgzMDA+CnZnYTA6IDxHZW5lcmljIElTQSBWR0E+IGF0IHBvcnQgMHgzYzAtMHgz ZGYgaW9tZW0gMHhhMDAwMC0weGJmZmZmIG9uIGlzYTAKYXRrYmRjMDogPEtleWJvYXJkIGNv bnRyb2xsZXIgKGk4MDQyKT4gYXQgcG9ydCAweDYwLDB4NjQgb24gaXNhMAphdGtiZDA6IDxB VCBLZXlib2FyZD4gaXJxIDEgb24gYXRrYmRjMAprYmQwIGF0IGF0a2JkMAphdGtiZDA6IFtH SUFOVC1MT0NLRURdCmF0a2JkMDogW0lUSFJFQURdCnBwYzA6IGNhbm5vdCByZXNlcnZlIEkv TyBwb3J0IHJhbmdlCmNvcmV0ZW1wMDogPENQVSBPbi1EaWUgVGhlcm1hbCBTZW5zb3JzPiBv biBjcHUwCmVzdDA6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJlcXVlbmN5IENvbnRyb2w+IG9u IGNwdTAKcDR0Y2MwOiA8Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENvbnRyb2w+IG9uIGNwdTAK Y29yZXRlbXAxOiA8Q1BVIE9uLURpZSBUaGVybWFsIFNlbnNvcnM+IG9uIGNwdTEKZXN0MTog PEVuaGFuY2VkIFNwZWVkU3RlcCBGcmVxdWVuY3kgQ29udHJvbD4gb24gY3B1MQpwNHRjYzE6 IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwgQ29udHJvbD4gb24gY3B1MQpjb3JldGVtcDI6IDxD UFUgT24tRGllIFRoZXJtYWwgU2Vuc29ycz4gb24gY3B1Mgplc3QyOiA8RW5oYW5jZWQgU3Bl ZWRTdGVwIEZyZXF1ZW5jeSBDb250cm9sPiBvbiBjcHUyCnA0dGNjMjogPENQVSBGcmVxdWVu Y3kgVGhlcm1hbCBDb250cm9sPiBvbiBjcHUyCmNvcmV0ZW1wMzogPENQVSBPbi1EaWUgVGhl cm1hbCBTZW5zb3JzPiBvbiBjcHUzCmVzdDM6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJlcXVl bmN5IENvbnRyb2w+IG9uIGNwdTMKcDR0Y2MzOiA8Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENv bnRyb2w+IG9uIGNwdTMKY29yZXRlbXA0OiA8Q1BVIE9uLURpZSBUaGVybWFsIFNlbnNvcnM+ IG9uIGNwdTQKZXN0NDogPEVuaGFuY2VkIFNwZWVkU3RlcCBGcmVxdWVuY3kgQ29udHJvbD4g b24gY3B1NApwNHRjYzQ6IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwgQ29udHJvbD4gb24gY3B1 NApjb3JldGVtcDU6IDxDUFUgT24tRGllIFRoZXJtYWwgU2Vuc29ycz4gb24gY3B1NQplc3Q1 OiA8RW5oYW5jZWQgU3BlZWRTdGVwIEZyZXF1ZW5jeSBDb250cm9sPiBvbiBjcHU1CnA0dGNj NTogPENQVSBGcmVxdWVuY3kgVGhlcm1hbCBDb250cm9sPiBvbiBjcHU1CmNvcmV0ZW1wNjog PENQVSBPbi1EaWUgVGhlcm1hbCBTZW5zb3JzPiBvbiBjcHU2CmVzdDY6IDxFbmhhbmNlZCBT cGVlZFN0ZXAgRnJlcXVlbmN5IENvbnRyb2w+IG9uIGNwdTYKcDR0Y2M2OiA8Q1BVIEZyZXF1 ZW5jeSBUaGVybWFsIENvbnRyb2w+IG9uIGNwdTYKY29yZXRlbXA3OiA8Q1BVIE9uLURpZSBU aGVybWFsIFNlbnNvcnM+IG9uIGNwdTcKZXN0NzogPEVuaGFuY2VkIFNwZWVkU3RlcCBGcmVx dWVuY3kgQ29udHJvbD4gb24gY3B1NwpwNHRjYzc6IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwg Q29udHJvbD4gb24gY3B1NwpaRlMgZmlsZXN5c3RlbSB2ZXJzaW9uIDUKWkZTIHN0b3JhZ2Ug cG9vbCB2ZXJzaW9uIDI4ClRpbWVjb3VudGVycyB0aWNrIGV2ZXJ5IDEuMDAwIG1zZWMKdmJv eGRydjogZkFzeW5jPTAgb2ZmTWluPTB4MzgwIG9mZk1heD0weDVhYwpmaXJld2lyZTA6IDEg bm9kZXMsIG1heGhvcCA8PSAwIGNhYmxlIElSTSBpcm0oMCkgIChtZSkgCmZpcmV3aXJlMDog YnVzIG1hbmFnZXIgMCAKaGRhYzA6IEhEQSBDb2RlYyAjMDogUmVhbHRlayBBTEM4OTIKaGRh YzA6IEhEQSBDb2RlYyAjMzogSW50ZWwgQ291Z2FyIFBvaW50IEhETUkKcGNtMDogPEhEQSBS ZWFsdGVrIEFMQzg5MiBQQ00gIzAgQW5hbG9nPiBhdCBjYWQgMCBuaWQgMSBvbiBoZGFjMApw Y20xOiA8SERBIFJlYWx0ZWsgQUxDODkyIFBDTSAjMSBBbmFsb2c+IGF0IGNhZCAwIG5pZCAx IG9uIGhkYWMwCnBjbTI6IDxIREEgUmVhbHRlayBBTEM4OTIgUENNICMyIERpZ2l0YWw+IGF0 IGNhZCAwIG5pZCAxIG9uIGhkYWMwCnBjbTM6IDxIREEgUmVhbHRlayBBTEM4OTIgUENNICMz IERpZ2l0YWw+IGF0IGNhZCAwIG5pZCAxIG9uIGhkYWMwCnBjbTQ6IDxIREEgSW50ZWwgQ291 Z2FyIFBvaW50IEhETUkgUENNICMwIERpc3BsYXlQb3J0PiBhdCBjYWQgMyBuaWQgMSBvbiBo ZGFjMAp1c2J1czA6IDQ4ME1icHMgSGlnaCBTcGVlZCBVU0IgdjIuMAp1c2J1czE6IDUuMEdi cHMgU3VwZXIgU3BlZWQgVVNCIHYzLjAKdXNidXMyOiA1LjBHYnBzIFN1cGVyIFNwZWVkIFVT QiB2My4wCnVzYnVzMzogNDgwTWJwcyBIaWdoIFNwZWVkIFVTQiB2Mi4wCnVnZW4wLjE6IDxJ bnRlbD4gYXQgdXNidXMwCnVodWIwOiA8SW50ZWwgRUhDSSByb290IEhVQiwgY2xhc3MgOS8w LCByZXYgMi4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMAp1Z2VuMS4xOiA8MHgxYjIxPiBh dCB1c2J1czEKdWh1YjE6IDwweDFiMjEgWEhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYg My4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMQp1Z2VuMi4xOiA8MHgxYjIxPiBhdCB1c2J1 czIKdWh1YjI6IDwweDFiMjEgWEhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMy4wMC8x LjAwLCBhZGRyIDE+IG9uIHVzYnVzMgp1Z2VuMy4xOiA8SW50ZWw+IGF0IHVzYnVzMwp1aHVi MzogPEludGVsIEVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAvMS4wMCwgYWRk ciAxPiBvbiB1c2J1czMKdWh1YjE6IDQgcG9ydHMgd2l0aCA0IHJlbW92YWJsZSwgc2VsZiBw b3dlcmVkCnVodWIyOiA0IHBvcnRzIHdpdGggNCByZW1vdmFibGUsIHNlbGYgcG93ZXJlZAph ZGEwIGF0IGFoY2ljaDAgYnVzIDAgc2NidXMwIHRhcmdldCAwIGx1biAwCmFkYTA6IDxXREMg V0QzMEVaUlgtMDBNTU1CMCA4MC4wMEE4MD4gQVRBLTggU0FUQSAzLnggZGV2aWNlCmFkYTA6 IDYwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAzLngsIFVETUE2LCBQSU8gODE5MmJ5dGVz KQphZGEwOiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQKYWRhMDogMjg2MTU4OE1CICg1ODYw NTMzMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpCmFkYTEgYXQgYWhj aWNoMSBidXMgMCBzY2J1czEgdGFyZ2V0IDAgbHVuIDAKYWRhMTogPFdEQyBXRDMwRVpSWC0w ME1NTUIwIDgwLjAwQTgwPiBBVEEtOCBTQVRBIDMueCBkZXZpY2UKYWRhMTogNjAwLjAwME1C L3MgdHJhbnNmZXJzIChTQVRBIDMueCwgVURNQTYsIFBJTyA4MTkyYnl0ZXMpCmFkYTE6IENv bW1hbmQgUXVldWVpbmcgZW5hYmxlZAphZGExOiAyODYxNTg4TUIgKDU4NjA1MzMxNjggNTEy IGJ5dGUgc2VjdG9yczogMTZIIDYzUy9UIDE2MzgzQykKYWRhMiBhdCBhaGNpY2g0IGJ1cyAw IHNjYnVzNCB0YXJnZXQgMCBsdW4gMAphZGEyOiA8V0RDIFdEMzBFWlJYLTAwTU1NQjAgODAu MDBBODA+IEFUQS04IFNBVEEgMy54IGRldmljZQphZGEyOiA2MDAuMDAwTUIvcyB0cmFuc2Zl cnMgKFNBVEEgMy54LCBVRE1BNiwgUElPIDgxOTJieXRlcykKYWRhMjogQ29tbWFuZCBRdWV1 ZWluZyBlbmFibGVkCmFkYTI6IDI4NjE1ODhNQiAoNTg2MDUzMzE2OCA1MTIgYnl0ZSBzZWN0 b3JzOiAxNkggNjNTL1QgMTYzODNDKQphZGEzIGF0IGFoY2ljaDUgYnVzIDAgc2NidXM1IHRh cmdldCAwIGx1biAwCmFkYTM6IDxXREMgV0QzMEVaUlgtMDBNTU1CMCA4MC4wMEE4MD4gQVRB LTggU0FUQSAzLnggZGV2aWNlCmFkYTM6IDYwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAz LngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQphZGEzOiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJs ZWQKYWRhMzogMjg2MTU4OE1CICg1ODYwNTMzMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2 M1MvVCAxNjM4M0MpCmFkYTQgYXQgYWhjaWNoNiBidXMgMCBzY2J1czYgdGFyZ2V0IDAgbHVu IDAKYWRhNDogPFdEQyBXRDIwRUFSUy0wME1WV0IwIDUwLjBBQjUwPiBBVEEtOCBTQVRBIDIu eCBkZXZpY2UKYWRhNDogMzAwLjAwME1CL3MgdHJhbnNmZXJzIChTQVRBIDIueCwgVURNQTYs IFBJTyA4MTkyYnl0ZXMpCmFkYTQ6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZAphZGE0OiAx OTA3NzI5TUIgKDM5MDcwMjkxNjggNTEyIGJ5dGUgc2VjdG9yczogMTZIIDYzUy9UIDE2Mzgz QykKYWRhNSBhdCBhaGNpY2g3IGJ1cyAwIHNjYnVzNyB0YXJnZXQgMCBsdW4gMAphZGE1OiA8 V0RDIFdEMjBFQVJTLTAwTVZXQjAgNTAuMEFCNTA+IEFUQS04IFNBVEEgMi54IGRldmljZQph ZGE1OiAzMDAuMDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEgMi54LCBVRE1BNiwgUElPIDgxOTJi eXRlcykKYWRhNTogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmFkYTU6IDE5MDc3MjlNQiAo MzkwNzAyOTE2OCA1MTIgYnl0ZSBzZWN0b3JzOiAxNkggNjNTL1QgMTYzODNDKQphZGE2IGF0 IGFoY2ljaDggYnVzIDAgc2NidXM4IHRhcmdldCAwIGx1biAwCmFkYTY6IDxXREMgV0Q1MDAw QUFLUy0wMEE3QjAgMDEuMDNCMDE+IEFUQS04IFNBVEEgMi54IGRldmljZQphZGE2OiAzMDAu MDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEgMi54LCBVRE1BNiwgUElPIDgxOTJieXRlcykKYWRh NjogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmFkYTY6IDQ3Njk0ME1CICg5NzY3NzMxNjgg NTEyIGJ5dGUgc2VjdG9yczogMTZIIDYzUy9UIDE2MzgzQykKYWRhNyBhdCBhaGNpY2g5IGJ1 cyAwIHNjYnVzOSB0YXJnZXQgMCBsdW4gMAphZGE3OiA8V0RDIFdENTAwMEFBS1MtMDBBN0Iw IDAxLjAzQjAxPiBBVEEtOCBTQVRBIDIueCBkZXZpY2UKYWRhNzogMzAwLjAwME1CL3MgdHJh bnNmZXJzIChTQVRBIDIueCwgVURNQTYsIFBJTyA4MTkyYnl0ZXMpCmFkYTc6IENvbW1hbmQg UXVldWVpbmcgZW5hYmxlZAphZGE3OiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNl Y3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpClNNUDogQVAgQ1BVICMxIExhdW5jaGVkIQpTTVA6 IEFQIENQVSAjNyBMYXVuY2hlZCEKU01QOiBBUCBDUFUgIzUgTGF1bmNoZWQhClNNUDogQVAg Q1BVICMyIExhdW5jaGVkIQpTTVA6IEFQIENQVSAjNiBMYXVuY2hlZCEKU01QOiBBUCBDUFUg IzMgTGF1bmNoZWQhClNNUDogQVAgQ1BVICM0IExhdW5jaGVkIQp1aHViMDogMiBwb3J0cyB3 aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQKdWh1YjM6IDIgcG9ydHMgd2l0aCAyIHJl bW92YWJsZSwgc2VsZiBwb3dlcmVkCnVnZW4wLjI6IDx2ZW5kb3IgMHg4MDg3PiBhdCB1c2J1 czAKdWh1YjQ6IDx2ZW5kb3IgMHg4MDg3IHByb2R1Y3QgMHgwMDI0LCBjbGFzcyA5LzAsIHJl diAyLjAwLzAuMDAsIGFkZHIgMj4gb24gdXNidXMwCnVnZW4zLjI6IDx2ZW5kb3IgMHg4MDg3 PiBhdCB1c2J1czMKdWh1YjU6IDx2ZW5kb3IgMHg4MDg3IHByb2R1Y3QgMHgwMDI0LCBjbGFz cyA5LzAsIHJldiAyLjAwLzAuMDAsIGFkZHIgMj4gb24gdXNidXMzCnVodWI0OiA2IHBvcnRz IHdpdGggNiByZW1vdmFibGUsIHNlbGYgcG93ZXJlZAp1aHViNTogOCBwb3J0cyB3aXRoIDgg cmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQKdWdlbjAuMzogPHZlbmRvciAweDA1NjY+IGF0IHVz YnVzMAp1a2JkMDogPHZlbmRvciAweDA1NjYgcHJvZHVjdCAweDMwMTAsIGNsYXNzIDAvMCwg cmV2IDEuMTAvMS4wMSwgYWRkciAzPiBvbiB1c2J1czAKa2JkMiBhdCB1a2JkMAp1aGlkMDog PHZlbmRvciAweDA1NjYgcHJvZHVjdCAweDMwMTAsIGNsYXNzIDAvMCwgcmV2IDEuMTAvMS4w MSwgYWRkciAzPiBvbiB1c2J1czAKdWdlbjMuMzogPHZlbmRvciAweDBjZjM+IGF0IHVzYnVz MwpUcnlpbmcgdG8gbW91bnQgcm9vdCBmcm9tIHpmczp6cGVhZ2xlL3pyb290CkdFT006IHp2 b2wvenAwL3Zib3gtZmVkb3JhMDogcGFydGl0aW9uIDIgZG9lcyBub3Qgc3RhcnQgb24gYSB0 cmFjayBib3VuZGFyeS4KR0VPTTogenZvbC96cDAvdmJveC1mZWRvcmEwOiBwYXJ0aXRpb24g MiBkb2VzIG5vdCBlbmQgb24gYSB0cmFjayBib3VuZGFyeS4KR0VPTTogenZvbC96cDAvdmJv eC1mZWRvcmEwOiBwYXJ0aXRpb24gMSBkb2VzIG5vdCBzdGFydCBvbiBhIHRyYWNrIGJvdW5k YXJ5LgpHRU9NOiB6dm9sL3pwMC92Ym94LWZlZG9yYTA6IHBhcnRpdGlvbiAxIGRvZXMgbm90 IGVuZCBvbiBhIHRyYWNrIGJvdW5kYXJ5LgpHRU9NOiB6dm9sL3pwMC92Ym94LWZyZWVic2Qw czE6IGdlb21ldHJ5IGRvZXMgbm90IG1hdGNoIGxhYmVsICgxNmgsNjNzICE9IDI1NWgsNjNz KS4KR0VPTTogenZvbC96cDAvdm9sMDogcGFydGl0aW9uIDMgZG9lcyBub3Qgc3RhcnQgb24g YSB0cmFjayBib3VuZGFyeS4KR0VPTTogenZvbC96cDAvdm9sMDogcGFydGl0aW9uIDMgZG9l cyBub3QgZW5kIG9uIGEgdHJhY2sgYm91bmRhcnkuCkdFT006IHp2b2wvenAwL3ZvbDA6IHBh cnRpdGlvbiAyIGRvZXMgbm90IHN0YXJ0IG9uIGEgdHJhY2sgYm91bmRhcnkuCkdFT006IHp2 b2wvenAwL3ZvbDA6IHBhcnRpdGlvbiAyIGRvZXMgbm90IGVuZCBvbiBhIHRyYWNrIGJvdW5k YXJ5LgpHRU9NOiB6dm9sL3pwMC92b2wwOiBwYXJ0aXRpb24gMSBkb2VzIG5vdCBzdGFydCBv biBhIHRyYWNrIGJvdW5kYXJ5LgpHRU9NOiB6dm9sL3pwMC92b2wwOiBwYXJ0aXRpb24gMSBk b2VzIG5vdCBlbmQgb24gYSB0cmFjayBib3VuZGFyeS4KR0VPTTogenZvbC96cDEvaXNjc2kt dm9sMTogcGFydGl0aW9uIDEgZG9lcyBub3Qgc3RhcnQgb24gYSB0cmFjayBib3VuZGFyeS4K R0VPTTogenZvbC96cDEvaXNjc2ktdm9sMTogcGFydGl0aW9uIDEgZG9lcyBub3QgZW5kIG9u IGEgdHJhY2sgYm91bmRhcnkuCnZib3huZXQwOiBFdGhlcm5ldCBhZGRyZXNzOiAwYTowMDoy NzowMDowMDowMAplbTA6IHByb21pc2N1b3VzIG1vZGUgZW5hYmxlZApHRU9NOiB6dm9sL3pw MS9pc2NzaS12b2wxOiBwYXJ0aXRpb24gMSBkb2VzIG5vdCBzdGFydCBvbiBhIHRyYWNrIGJv dW5kYXJ5LgpHRU9NOiB6dm9sL3pwMS9pc2NzaS12b2wxOiBwYXJ0aXRpb24gMSBkb2VzIG5v dCBlbmQgb24gYSB0cmFjayBib3VuZGFyeS4KR0VPTTogenZvbC96cDEvaXNjc2ktdm9sMTog cGFydGl0aW9uIDEgZG9lcyBub3Qgc3RhcnQgb24gYSB0cmFjayBib3VuZGFyeS4KR0VPTTog enZvbC96cDEvaXNjc2ktdm9sMTogcGFydGl0aW9uIDEgZG9lcyBub3QgZW5kIG9uIGEgdHJh Y2sgYm91bmRhcnkuCg== --------------020105040408020609090904 Content-Type: text/plain; name="pciconf.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pciconf.txt" aG9zdGIwQHBjaTA6MDowOjA6CWNsYXNzPTB4MDYwMDAwIGNhcmQ9MHg4NDRkMTA0MyBjaGlw PTB4MDEwMDgwODYgcmV2PTB4MDkgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwg Q29ycG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAgID0g SE9TVC1QQ0kKcGNpYjFAcGNpMDowOjE6MDoJY2xhc3M9MHgwNjA0MDAgY2FyZD0weDg0NGQx MDQzIGNoaXA9MHgwMTAxODA4NiByZXY9MHgwOSBoZHI9MHgwMQogICAgdmVuZG9yICAgICA9 ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGNsYXNzICAgICAgPSBicmlkZ2UKICAgIHN1YmNs YXNzICAgPSBQQ0ktUENJCnZnYXBjaTBAcGNpMDowOjI6MDoJY2xhc3M9MHgwMzAwMDAgY2Fy ZD0weDg0NGQxMDQzIGNoaXA9MHgwMTAyODA4NiByZXY9MHgwOSBoZHI9MHgwMAogICAgdmVu ZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGNsYXNzICAgICAgPSBkaXNwbGF5 CiAgICBzdWJjbGFzcyAgID0gVkdBCiAgICBiYXIgICBbMTBdID0gdHlwZSBNZW1vcnksIHJh bmdlIDY0LCBiYXNlIDB4ZmI0MDAwMDAsIHNpemUgNDE5NDMwNCwgZW5hYmxlZAogICAgYmFy ICAgWzE4XSA9IHR5cGUgUHJlZmV0Y2hhYmxlIE1lbW9yeSwgcmFuZ2UgNjQsIGJhc2UgMHhk MDAwMDAwMCwgc2l6ZSAyNjg0MzU0NTYsIGVuYWJsZWQKICAgIGJhciAgIFsyMF0gPSB0eXBl IEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAweGYwMDAsIHNpemUgNjQsIGVuYWJsZWQKbm9u ZTBAcGNpMDowOjIyOjA6CWNsYXNzPTB4MDc4MDAwIGNhcmQ9MHg4NDRkMTA0MyBjaGlwPTB4 MWMzYTgwODYgcmV2PTB4MDQgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29y cG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0gc2ltcGxlIGNvbW1zCiAgICBiYXIgICBbMTBd ID0gdHlwZSBNZW1vcnksIHJhbmdlIDY0LCBiYXNlIDB4ZmJmMjkwMDAsIHNpemUgMTYsIGVu YWJsZWQKZW0wQHBjaTA6MDoyNTowOgljbGFzcz0weDAyMDAwMCBjYXJkPTB4ODQ5YzEwNDMg Y2hpcD0weDE1MDM4MDg2IHJldj0weDA1IGhkcj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0lu dGVsIENvcnBvcmF0aW9uJwogICAgY2xhc3MgICAgICA9IG5ldHdvcmsKICAgIHN1YmNsYXNz ICAgPSBldGhlcm5ldAogICAgYmFyICAgWzEwXSA9IHR5cGUgTWVtb3J5LCByYW5nZSAzMiwg YmFzZSAweGZiZjAwMDAwLCBzaXplIDEzMTA3MiwgZW5hYmxlZAogICAgYmFyICAgWzE0XSA9 IHR5cGUgTWVtb3J5LCByYW5nZSAzMiwgYmFzZSAweGZiZjI4MDAwLCBzaXplIDQwOTYsIGVu YWJsZWQKICAgIGJhciAgIFsxOF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAw eGYwODAsIHNpemUgMzIsIGVuYWJsZWQKZWhjaTBAcGNpMDowOjI2OjA6CWNsYXNzPTB4MGMw MzIwIGNhcmQ9MHg4NDRkMTA0MyBjaGlwPTB4MWMyZDgwODYgcmV2PTB4MDUgaGRyPTB4MDAK ICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0g c2VyaWFsIGJ1cwogICAgc3ViY2xhc3MgICA9IFVTQgogICAgYmFyICAgWzEwXSA9IHR5cGUg TWVtb3J5LCByYW5nZSAzMiwgYmFzZSAweGZiZjI3MDAwLCBzaXplIDEwMjQsIGVuYWJsZWQK aGRhYzBAcGNpMDowOjI3OjA6CWNsYXNzPTB4MDQwMzAwIGNhcmQ9MHg4NDEwMTA0MyBjaGlw PTB4MWMyMDgwODYgcmV2PTB4MDUgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwg Q29ycG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0gbXVsdGltZWRpYQogICAgc3ViY2xhc3Mg ICA9IEhEQQogICAgYmFyICAgWzEwXSA9IHR5cGUgTWVtb3J5LCByYW5nZSA2NCwgYmFzZSAw eGZiZjIwMDAwLCBzaXplIDE2Mzg0LCBlbmFibGVkCnBjaWIyQHBjaTA6MDoyODowOgljbGFz cz0weDA2MDQwMCBjYXJkPTB4ODQ0ZDEwNDMgY2hpcD0weDFjMTA4MDg2IHJldj0weGI1IGhk cj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgY2xhc3Mg ICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IFBDSS1QQ0kKcGNpYjNAcGNpMDowOjI4 OjE6CWNsYXNzPTB4MDYwNDAwIGNhcmQ9MHg4NDRkMTA0MyBjaGlwPTB4MWMxMjgwODYgcmV2 PTB4YjUgaGRyPTB4MDEKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAg ICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAgID0gUENJLVBDSQpwY2liNEBw Y2kwOjA6Mjg6MjoJY2xhc3M9MHgwNjA0MDAgY2FyZD0weDg0NGQxMDQzIGNoaXA9MHgxYzE0 ODA4NiByZXY9MHhiNSBoZHI9MHgwMQogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3Jh dGlvbicKICAgIGNsYXNzICAgICAgPSBicmlkZ2UKICAgIHN1YmNsYXNzICAgPSBQQ0ktUENJ CnBjaWI1QHBjaTA6MDoyODozOgljbGFzcz0weDA2MDQwMCBjYXJkPTB4ODQ0ZDEwNDMgY2hp cD0weDFjMTY4MDg2IHJldj0weGI1IGhkcj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVs IENvcnBvcmF0aW9uJwogICAgY2xhc3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9 IFBDSS1QQ0kKcGNpYjZAcGNpMDowOjI4OjQ6CWNsYXNzPTB4MDYwNDAwIGNhcmQ9MHg4NDRk MTA0MyBjaGlwPTB4MWMxODgwODYgcmV2PTB4YjUgaGRyPTB4MDEKICAgIHZlbmRvciAgICAg PSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJj bGFzcyAgID0gUENJLVBDSQpwY2liN0BwY2kwOjA6Mjg6NjoJY2xhc3M9MHgwNjA0MDEgY2Fy ZD0weDg0NGQxMDQzIGNoaXA9MHgyNDRlODA4NiByZXY9MHhiNSBoZHI9MHgwMQogICAgdmVu ZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRldmljZSAgICAgPSAnODI4MDEg RmFtaWx5IChJQ0gyLzMvNC81LzYvNy84LzksNjN4eEVTQikgSHViIEludGVyZmFjZSB0byBQ Q0kgQnJpZGdlJwogICAgY2xhc3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IFBD SS1QQ0kKcGNpYjlAcGNpMDowOjI4Ojc6CWNsYXNzPTB4MDYwNDAwIGNhcmQ9MHg4NDRkMTA0 MyBjaGlwPTB4MWMxZTgwODYgcmV2PTB4YjUgaGRyPTB4MDEKICAgIHZlbmRvciAgICAgPSAn SW50ZWwgQ29ycG9yYXRpb24nCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFz cyAgID0gUENJLVBDSQplaGNpMUBwY2kwOjA6Mjk6MDoJY2xhc3M9MHgwYzAzMjAgY2FyZD0w eDg0NGQxMDQzIGNoaXA9MHgxYzI2ODA4NiByZXY9MHgwNSBoZHI9MHgwMAogICAgdmVuZG9y ICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGNsYXNzICAgICAgPSBzZXJpYWwgYnVz CiAgICBzdWJjbGFzcyAgID0gVVNCCiAgICBiYXIgICBbMTBdID0gdHlwZSBNZW1vcnksIHJh bmdlIDMyLCBiYXNlIDB4ZmJmMjYwMDAsIHNpemUgMTAyNCwgZW5hYmxlZAppc2FiMEBwY2kw OjA6MzE6MDoJY2xhc3M9MHgwNjAxMDAgY2FyZD0weDg0NGQxMDQzIGNoaXA9MHgxYzQ0ODA4 NiByZXY9MHgwNSBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlv bicKICAgIGNsYXNzICAgICAgPSBicmlkZ2UKICAgIHN1YmNsYXNzICAgPSBQQ0ktSVNBCmFo Y2kyQHBjaTA6MDozMToyOgljbGFzcz0weDAxMDYwMSBjYXJkPTB4ODQ0ZDEwNDMgY2hpcD0w eDFjMDI4MDg2IHJldj0weDA1IGhkcj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENv cnBvcmF0aW9uJwogICAgY2xhc3MgICAgICA9IG1hc3Mgc3RvcmFnZQogICAgc3ViY2xhc3Mg ICA9IFNBVEEKICAgIGJhciAgIFsxMF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFz ZSAweGYwZDAsIHNpemUgIDgsIGVuYWJsZWQKICAgIGJhciAgIFsxNF0gPSB0eXBlIEkvTyBQ b3J0LCByYW5nZSAzMiwgYmFzZSAweGYwYzAsIHNpemUgIDQsIGVuYWJsZWQKICAgIGJhciAg IFsxOF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAweGYwYjAsIHNpemUgIDgs IGVuYWJsZWQKICAgIGJhciAgIFsxY10gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFz ZSAweGYwYTAsIHNpemUgIDQsIGVuYWJsZWQKICAgIGJhciAgIFsyMF0gPSB0eXBlIEkvTyBQ b3J0LCByYW5nZSAzMiwgYmFzZSAweGYwNjAsIHNpemUgMzIsIGVuYWJsZWQKICAgIGJhciAg IFsyNF0gPSB0eXBlIE1lbW9yeSwgcmFuZ2UgMzIsIGJhc2UgMHhmYmYyNTAwMCwgc2l6ZSAy MDQ4LCBlbmFibGVkCm5vbmUxQHBjaTA6MDozMTozOgljbGFzcz0weDBjMDUwMCBjYXJkPTB4 ODQ0ZDEwNDMgY2hpcD0weDFjMjI4MDg2IHJldj0weDA1IGhkcj0weDAwCiAgICB2ZW5kb3Ig ICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgY2xhc3MgICAgICA9IHNlcmlhbCBidXMK ICAgIHN1YmNsYXNzICAgPSBTTUJ1cwogICAgYmFyICAgWzEwXSA9IHR5cGUgTWVtb3J5LCBy YW5nZSA2NCwgYmFzZSAweGZiZjI0MDAwLCBzaXplIDI1NiwgZW5hYmxlZAogICAgYmFyICAg WzIwXSA9IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4ZjA0MCwgc2l6ZSAzMiwg ZW5hYmxlZAphaGNpMEBwY2kwOjI6MDowOgljbGFzcz0weDAxMDQwMCBjYXJkPTB4MDAwMTEx MDMgY2hpcD0weDA2MjAxMTAzIHJldj0weDAxIGhkcj0weDAwCiAgICB2ZW5kb3IgICAgID0g J1RyaW9uZXMgVGVjaG5vbG9naWVzIEluYy4gKEhpZ2hQb2ludCknCiAgICBjbGFzcyAgICAg ID0gbWFzcyBzdG9yYWdlCiAgICBzdWJjbGFzcyAgID0gUkFJRAogICAgYmFyICAgWzEwXSA9 IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4ZTA5MCwgc2l6ZSAgOCwgZW5hYmxl ZAogICAgYmFyICAgWzE0XSA9IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4ZTA4 MCwgc2l6ZSAgNCwgZW5hYmxlZAogICAgYmFyICAgWzE4XSA9IHR5cGUgSS9PIFBvcnQsIHJh bmdlIDMyLCBiYXNlIDB4ZTA3MCwgc2l6ZSAgOCwgZW5hYmxlZAogICAgYmFyICAgWzFjXSA9 IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4ZTA2MCwgc2l6ZSAgNCwgZW5hYmxl ZAogICAgYmFyICAgWzIwXSA9IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4ZTA1 MCwgc2l6ZSAxNiwgZW5hYmxlZAogICAgYmFyICAgWzI0XSA9IHR5cGUgTWVtb3J5LCByYW5n ZSAzMiwgYmFzZSAweGZiZTIxMDAwLCBzaXplIDIwNDgsIGVuYWJsZWQKYXRhcGNpMEBwY2kw OjI6MDoxOgljbGFzcz0weDAxMDE4ZiBjYXJkPTB4OTFhNDFiNGIgY2hpcD0weDkxYTQxYjRi IHJldj0weDExIGhkcj0weDAwCiAgICBjbGFzcyAgICAgID0gbWFzcyBzdG9yYWdlCiAgICBz dWJjbGFzcyAgID0gQVRBCiAgICBiYXIgICBbMTBdID0gdHlwZSBJL08gUG9ydCwgcmFuZ2Ug MzIsIGJhc2UgMHhlMDQwLCBzaXplICA4LCBlbmFibGVkCiAgICBiYXIgICBbMTRdID0gdHlw ZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhlMDMwLCBzaXplICA0LCBlbmFibGVkCiAg ICBiYXIgICBbMThdID0gdHlwZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhlMDIwLCBz aXplICA4LCBlbmFibGVkCiAgICBiYXIgICBbMWNdID0gdHlwZSBJL08gUG9ydCwgcmFuZ2Ug MzIsIGJhc2UgMHhlMDEwLCBzaXplICA0LCBlbmFibGVkCiAgICBiYXIgICBbMjBdID0gdHlw ZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhlMDAwLCBzaXplIDE2LCBlbmFibGVkCiAg ICBiYXIgICBbMjRdID0gdHlwZSBNZW1vcnksIHJhbmdlIDMyLCBiYXNlIDB4ZmJlMjAwMDAs IHNpemUgMTYsIGVuYWJsZWQKeGhjaTBAcGNpMDozOjA6MDoJY2xhc3M9MHgwYzAzMzAgY2Fy ZD0weDg0ODgxMDQzIGNoaXA9MHgxMDQyMWIyMSByZXY9MHgwMCBoZHI9MHgwMAogICAgY2xh c3MgICAgICA9IHNlcmlhbCBidXMKICAgIHN1YmNsYXNzICAgPSBVU0IKICAgIGJhciAgIFsx MF0gPSB0eXBlIE1lbW9yeSwgcmFuZ2UgNjQsIGJhc2UgMHhmYmQwMDAwMCwgc2l6ZSAzMjc2 OCwgZW5hYmxlZAplbTFAcGNpMDo0OjA6MDoJY2xhc3M9MHgwMjAwMDAgY2FyZD0weDEwODM4 MDg2IGNoaXA9MHgxMGI5ODA4NiByZXY9MHgwNiBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9 ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRldmljZSAgICAgPSAnODI1NzJFSSBQUk8vMTAw MCBQVCBEZXNrdG9wIEFkYXB0ZXIgKENvcHBlciknCiAgICBjbGFzcyAgICAgID0gbmV0d29y awogICAgc3ViY2xhc3MgICA9IGV0aGVybmV0CiAgICBiYXIgICBbMTBdID0gdHlwZSBNZW1v cnksIHJhbmdlIDMyLCBiYXNlIDB4ZmJjNDAwMDAsIHNpemUgMTMxMDcyLCBlbmFibGVkCiAg ICBiYXIgICBbMTRdID0gdHlwZSBNZW1vcnksIHJhbmdlIDMyLCBiYXNlIDB4ZmJjMjAwMDAs IHNpemUgMTMxMDcyLCBlbmFibGVkCiAgICBiYXIgICBbMThdID0gdHlwZSBJL08gUG9ydCwg cmFuZ2UgMzIsIGJhc2UgMHhkMDAwLCBzaXplIDMyLCBlbmFibGVkCmF0YXBjaTFAcGNpMDo1 OjA6MDoJY2xhc3M9MHgwMTAxODUgY2FyZD0weDg0NjAxMDQzIGNoaXA9MHgyMzYyMTk3YiBy ZXY9MHgxMCBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9ICdKTWljcm9uIFRlY2hub2xvZ3kg Q29ycC4nCiAgICBjbGFzcyAgICAgID0gbWFzcyBzdG9yYWdlCiAgICBzdWJjbGFzcyAgID0g QVRBCiAgICBiYXIgICBbMTBdID0gdHlwZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhj MDQwLCBzaXplICA4LCBlbmFibGVkCiAgICBiYXIgICBbMTRdID0gdHlwZSBJL08gUG9ydCwg cmFuZ2UgMzIsIGJhc2UgMHhjMDMwLCBzaXplICA0LCBlbmFibGVkCiAgICBiYXIgICBbMThd ID0gdHlwZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhjMDIwLCBzaXplICA4LCBlbmFi bGVkCiAgICBiYXIgICBbMWNdID0gdHlwZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHhj MDEwLCBzaXplICA0LCBlbmFibGVkCiAgICBiYXIgICBbMjBdID0gdHlwZSBJL08gUG9ydCwg cmFuZ2UgMzIsIGJhc2UgMHhjMDAwLCBzaXplIDE2LCBlbmFibGVkCiAgICBiYXIgICBbMjRd ID0gdHlwZSBNZW1vcnksIHJhbmdlIDMyLCBiYXNlIDB4ZmJiMTAwMDAsIHNpemUgNTEyLCBl bmFibGVkCnhoY2kxQHBjaTA6NjowOjA6CWNsYXNzPTB4MGMwMzMwIGNhcmQ9MHg4NDg4MTA0 MyBjaGlwPTB4MTA0MjFiMjEgcmV2PTB4MDAgaGRyPTB4MDAKICAgIGNsYXNzICAgICAgPSBz ZXJpYWwgYnVzCiAgICBzdWJjbGFzcyAgID0gVVNCCiAgICBiYXIgICBbMTBdID0gdHlwZSBN ZW1vcnksIHJhbmdlIDY0LCBiYXNlIDB4ZmJhMDAwMDAsIHNpemUgMzI3NjgsIGVuYWJsZWQK cGNpYjhAcGNpMDo3OjA6MDoJY2xhc3M9MHgwNjA0MDEgY2FyZD0weDEwODAxYjIxIGNoaXA9 MHgxMDgwMWIyMSByZXY9MHgwMSBoZHI9MHgwMQogICAgY2xhc3MgICAgICA9IGJyaWRnZQog ICAgc3ViY2xhc3MgICA9IFBDSS1QQ0kKZndvaGNpMEBwY2kwOjg6MjowOgljbGFzcz0weDBj MDAxMCBjYXJkPTB4ODFmZTEwNDMgY2hpcD0weDMwNDQxMTA2IHJldj0weGMwIGhkcj0weDAw CiAgICB2ZW5kb3IgICAgID0gJ1ZJQSBUZWNobm9sb2dpZXMsIEluYy4nCiAgICBkZXZpY2Ug ICAgID0gJ1ZUNjMwNiBWSUEgRmlyZSBJSSBJRUVFLTEzOTQgT0hDSSBMaW5rIExheWVyIENv bnRyb2xsZXInCiAgICBjbGFzcyAgICAgID0gc2VyaWFsIGJ1cwogICAgc3ViY2xhc3MgICA9 IEZpcmVXaXJlCiAgICBiYXIgICBbMTBdID0gdHlwZSBNZW1vcnksIHJhbmdlIDMyLCBiYXNl IDB4ZmI5MDAwMDAsIHNpemUgMjA0OCwgZW5hYmxlZAogICAgYmFyICAgWzE0XSA9IHR5cGUg SS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4YjAwMCwgc2l6ZSAxMjgsIGVuYWJsZWQKYWhj aTFAcGNpMDo5OjA6MDoJY2xhc3M9MHgwMTA2MDEgY2FyZD0weDg0NzcxMDQzIGNoaXA9MHg5 MTcyMWI0YiByZXY9MHgxMSBoZHI9MHgwMAogICAgY2xhc3MgICAgICA9IG1hc3Mgc3RvcmFn ZQogICAgc3ViY2xhc3MgICA9IFNBVEEKICAgIGJhciAgIFsxMF0gPSB0eXBlIEkvTyBQb3J0 LCByYW5nZSAzMiwgYmFzZSAweGEwNDAsIHNpemUgIDgsIGVuYWJsZWQKICAgIGJhciAgIFsx NF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAweGEwMzAsIHNpemUgIDQsIGVu YWJsZWQKICAgIGJhciAgIFsxOF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAw eGEwMjAsIHNpemUgIDgsIGVuYWJsZWQKICAgIGJhciAgIFsxY10gPSB0eXBlIEkvTyBQb3J0 LCByYW5nZSAzMiwgYmFzZSAweGEwMTAsIHNpemUgIDQsIGVuYWJsZWQKICAgIGJhciAgIFsy MF0gPSB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAweGEwMDAsIHNpemUgMTYsIGVu YWJsZWQKICAgIGJhciAgIFsyNF0gPSB0eXBlIE1lbW9yeSwgcmFuZ2UgMzIsIGJhc2UgMHhm YjgxMDAwMCwgc2l6ZSA1MTIsIGVuYWJsZWQK --------------020105040408020609090904-- From owner-freebsd-stable@FreeBSD.ORG Sat Sep 10 20:46:54 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 050801065672; Sat, 10 Sep 2011 20:46:54 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 590C68FC13; Sat, 10 Sep 2011 20:46:53 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p8AKkeJr061076; Sun, 11 Sep 2011 05:46:50 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p8AKkc1N038766; Sun, 11 Sep 2011 05:46:40 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Sun, 11 Sep 2011 05:46:01 +0900 (JST) Message-Id: <20110911.054601.1424617155148336027.hrs@allbsd.org> To: pjd@FreeBSD.org, mm@FreeBSD.org, freebsd-stable@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110910.044841.232160047547388224.hrs@allbsd.org> References: <20110907.094717.2272609566853905102.hrs@allbsd.org> <20110910.044841.232160047547388224.hrs@allbsd.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3.51 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Sun_Sep_11_05_46_01_2011_447)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Sun, 11 Sep 2011 05:46:51 +0900 (JST) X-Spam-Status: No, score=-104.4 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, QENCPTR1, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: attilio@FreeBSD.org, kib@FreeBSD.org Subject: Re: ZFS panic on a RELENG_8 NFS server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Sep 2011 20:46:54 -0000 ----Security_Multipart(Sun_Sep_11_05_46_01_2011_447)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hiroki Sato wrote in <20110910.044841.232160047547388224.hrs@allbsd.org>: hr> Hiroki Sato wrote hr> in <20110907.094717.2272609566853905102.hrs@allbsd.org>: hr> hr> hr> During this investigation an disk has to be replaced and resilvering hr> hr> it is now in progress. A deadlock and a forced reboot after that hr> hr> make recovering of the zfs datasets take a long time (for committing hr> hr> logs, I think), so I will try to reproduce the deadlock and get a hr> hr> core dump after it finished. hr> hr> I think I could reproduce the symptoms. I have no idea about if hr> these are exactly the same as occurred on my box before because the hr> kernel was replaced with one with some debugging options, but these hr> are reproducible at least. hr> hr> There are two symptoms. One is a panic. A DDB output when the panic hr> occurred is the following: I am trying vfs.lookup_shared=0 and seeing how it goes. It seems the box can endure a high load which quickly caused these symptoms. -- Hiroki ----Security_Multipart(Sun_Sep_11_05_46_01_2011_447)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk5rzIkACgkQTyzT2CeTzy0qawCgqgbT/OfkbyC89k82JlakipOz RHoAnA17gfukuS58aMXY+G6pAcQAwC9s =3f+I -----END PGP SIGNATURE----- ----Security_Multipart(Sun_Sep_11_05_46_01_2011_447)----