From owner-freebsd-stable@FreeBSD.ORG Sun Jul 24 11:14:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18832106566B for ; Sun, 24 Jul 2011 11:14:58 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep21.mx.upcmail.net (fep21.mx.upcmail.net [62.179.121.41]) by mx1.freebsd.org (Postfix) with ESMTP id E0AE78FC08 for ; Sun, 24 Jul 2011 11:14:56 +0000 (UTC) Received: from edge01.upcmail.net ([192.168.13.236]) by viefep20-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20110724105756.EIPY1387.viefep20-int.chello.at@edge01.upcmail.net> for ; Sun, 24 Jul 2011 12:57:56 +0200 Received: from pinky ([95.96.138.26]) by edge01.upcmail.net with edge id Bmxu1h02F0aMTqv01mxvpb; Sun, 24 Jul 2011 12:57:56 +0200 X-SourceIP: 95.96.138.26 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-stable@freebsd.org References: <20110721215617.7C7F0B827@mail.bitblocks.com> Date: Sun, 24 Jul 2011 12:57:57 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <20110721215617.7C7F0B827@mail.bitblocks.com> User-Agent: Opera Mail/11.50 (Win32) X-Cloudmark-Analysis: v=1.1 cv=zlRBWuFCZaNL9+WHNm1pWLowY5Lx061w2zJBJiDkNAU= c=1 sm=0 a=jSLzLkXI7GEA:10 a=LpIY22MEHDsA:10 a=bgpUlknNv7MA:10 a=kj9zAlcOel0A:10 a=mpmaN9d1AAAA:8 a=d8xbraedAAAA:8 a=ud3xay5K_2NmokssbDcA:9 a=CjuIK1q_8ugA:10 a=9klllKCzMBAA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Subject: Re: OS X Lion time machine => (afpd|iSCSI) => ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jul 2011 11:14:58 -0000 On Thu, 21 Jul 2011 23:56:17 +0200, Bakul Shah wrote: > I am in no hurry to upgrade my MBP to OS X Lion but given Lion > time machine and netatalk issues, I got wondering if iSCSI on > FreeBSD is stable enough for time machine use. How much duct > tape and baling wire are needed to make it work?! A friend of mine posted this on twitter this morning. http://www.alexanderwilde.com/2011/04/os-x-lion-connection-error-with-afp-and-workaround/ Apparently some hash algorithm or something like that is disabled in Lion, but not in a lot of NAS's. Ronald/ From owner-freebsd-stable@FreeBSD.ORG Sun Jul 24 11:46:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9BB21065670 for ; Sun, 24 Jul 2011 11:46:13 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id A40408FC13 for ; Sun, 24 Jul 2011 11:46:13 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 7221A8955E; Sun, 24 Jul 2011 13:46:12 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20110721215617.7C7F0B827@mail.bitblocks.com> Date: Sun, 24 Jul 2011 13:46:12 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1FEED526-2CD9-4220-B5DA-6C3CF143E4A6@lassitu.de> References: <20110721215617.7C7F0B827@mail.bitblocks.com> To: Bakul Shah X-Mailer: Apple Mail (2.1084) Cc: freebsd-stable@freebsd.org Subject: Re: OS X Lion time machine => (afpd|iSCSI) => ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jul 2011 11:46:13 -0000 Am 21.07.2011 um 23:56 schrieb Bakul Shah: > I am in no hurry to upgrade my MBP to OS X Lion but given Lion > time machine and netatalk issues, I got wondering if iSCSI on > FreeBSD is stable enough for time machine use. How much duct > tape and baling wire are needed to make it work?! After having had odd behavior from TM on a netatalk volume, I've = switched over to istgt and the globalSAN iSCSI initiator, using a ZVOL. = I found the istgt configuration non-obvious, but I also have little = background in iSCSI. Took me about an hour to get it up and running = without authentication; haven't bothered since trying to get = authentication to work. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 02:31:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBE42106566B; Mon, 25 Jul 2011 02:31:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6679F8FC16; Mon, 25 Jul 2011 02:31:29 +0000 (UTC) Received: by ywf7 with SMTP id 7so2395571ywf.13 for ; Sun, 24 Jul 2011 19:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cukEPyZGEkMAAb/RcKcHzHC7C/193Ar01SATqpUhmWE=; b=ZYsdeKR8m4YFs64NKpGOeTLT44nq1B+byqrhOLtaPYr5JnYNhe3gGyR+k8ZoDPHffN GV26quorj0mbW0K9tcyt5aWaiLxxjXF5LmgEL/mjc4w41x+CmJnel8vbGYK8SKIKlUAH pI8vLTPJ4ymepC7l9Yru0PmJ8OxyLjvPAoRXA= MIME-Version: 1.0 Received: by 10.150.74.3 with SMTP id w3mr3897719yba.329.1311559656626; Sun, 24 Jul 2011 19:07:36 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.197.5 with HTTP; Sun, 24 Jul 2011 19:07:36 -0700 (PDT) In-Reply-To: <20110711115947.51686v4930s7ze37@webmail.in-berlin.de> References: <20110706122339.61453nlqra1vqsrv@webmail.in-berlin.de> <20110706023234.GA72048@icarus.home.lan> <20110706130753.182053f3ellasn0p@webmail.in-berlin.de> <20110706032425.GA72757@icarus.home.lan> <20110706135412.15276i0fxavg09k4@webmail.in-berlin.de> <20110706041504.GA73698@icarus.home.lan> <20110706143129.10696235ldx9bjmp@webmail.in-berlin.de> <20110706173242.23404ffbhkxz6mqi@webmail.in-berlin.de> <20110706182141.13056plxp148y61h@webmail.in-berlin.de> <20110711115947.51686v4930s7ze37@webmail.in-berlin.de> Date: Mon, 25 Jul 2011 10:07:36 +0800 X-Google-Sender-Auth: 5mqU44_Bs8O5ufa9k2Oeni2Aci8 Message-ID: From: Adrian Chadd To: Peter Ross Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Yong-Hyeon Pyun , freebsd-stable List , "Vogel, Jack" , Scott Sipe , davidch@freebsd.org, Jeremy Chadwick Subject: Re: scp: Write Failed: Cannot allocate memory X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 02:31:29 -0000 Has someone asked for the output of netstat -mb? That error message is mbuf related, so I bet it's something to do with mbuf allocation. Is it possible that the system is incorrectly tuned when virtualbox is enab= led? Adrian On 11 July 2011 09:59, Peter Ross wrote: > Quoting "Scott Sipe" : > >> On Wed, Jul 6, 2011 at 4:21 AM, Peter Ross >> wrote: >> >>> Quoting "Peter Ross" : >>> >>> =A0Quoting "Peter Ross" : >>>> >>>> =A0Quoting "Jeremy Chadwick" : >>>>> >>>>> =A0On Wed, Jul 06, 2011 at 01:54:12PM +1000, Peter Ross wrote: >>>>>> >>>>>>> Quoting "Jeremy Chadwick" : >>>>>>> >>>>>>> =A0On Wed, Jul 06, 2011 at 01:07:53PM +1000, Peter Ross wrote: >>>>>>>> >>>>>>>>> Quoting "Jeremy Chadwick" : >>>>>>>>> >>>>>>>>> =A0On Wed, Jul 06, 2011 at 12:23:39PM +1000, Peter Ross wrote: >>>>>>>>>> >>>>>>>>>>> Quoting "Jeremy Chadwick" : >>>>>>>>>>> >>>>>>>>>>> =A0On Tue, Jul 05, 2011 at 01:03:20PM -0400, Scott Sipe wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'm running virtualbox 3.2.12_1 if that has anything to do wi= th >>>>>>>>>>>>> it. >>>>>>>>>>>>> >>>>>>>>>>>>> sysctl vfs.zfs.arc_max: 6200000000 >>>>>>>>>>>>> >>>>>>>>>>>>> While I'm trying to scp, kstat.zfs.misc.arcstats.size is >>>>>>>>>>>>> hovering right around that value, sometimes above, sometimes >>>>>>>>>>>>> below (that's as it should be, right?). I don't think that it >>>>>>>>>>>>> dies when crossing over arc_max. I can run the same scp 10 >>>>>>>>>>>>> times >>>>>>>>>>>>> and it might fail 1-3 times, with no correlation to the >>>>>>>>>>>>> arcstats.size being above/below arc_max that I can see. >>>>>>>>>>>>> >>>>>>>>>>>>> Scott >>>>>>>>>>>>> >>>>>>>>>>>>> On Jul 5, 2011, at 3:00 AM, Peter Ross wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> =A0Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> just as an addition: an upgrade to last Friday's >>>>>>>>>>>>>> FreeBSD-Stable and to VirtualBox 4.0.8 does not fix the >>>>>>>>>>>>>> problem. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will experiment a bit more tomorrow after hours and grab >>>>>>>>>>>>>> >>>>>>>>>>>>> some statistics. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> Regards >>>>>>>>>>>>>> Peter >>>>>>>>>>>>>> >>>>>>>>>>>>>> Quoting "Peter Ross" : >>>>>>>>>>>>>> >>>>>>>>>>>>>> =A0Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I noticed a similar problem last week. It is also very >>>>>>>>>>>>>>> similar to one reported last year: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2010-** >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> September/058708.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> My server is a Dell T410 server with the same bge card (the >>>>>>>>>>>>>>> same pciconf -lvc output as described by Mahlon: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2010-** >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> September/058711.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yours, Scott, is a em(4).. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Another similarity: In all cases we are using VirtualBox. I >>>>>>>>>>>>>>> just want to mention it, in case it matters. I am still >>>>>>>>>>>>>>> running VirtualBox 3.2. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Most of the time kstat.zfs.misc.arcstats.size was reaching >>>>>>>>>>>>>>> vfs.zfs.arc_max then, but I could catch one or two cases >>>>>>>>>>>>>>> then the value was still below. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I added vfs.zfs.prefetch_disable=3D1 to sysctl.conf but it >>>>>>>>>>>>>>> >>>>>>>>>>>>>> does not help. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>> BTW: It looks as ARC only gives back the memory when I >>>>>>>>>>>>>>> destroy the ZFS (a cloned snapshot containing virtual >>>>>>>>>>>>>>> machines). Even if nothing happens for hours the buffer >>>>>>>>>>>>>>> isn't released.. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> My machine was still running 8.2-PRERELEASE so I am >>>>>>>>>>>>>>> upgrading. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am happy to give information gathered on old/new kernel i= f >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>> helps. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards >>>>>>>>>>>>>>> Peter >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Quoting "Scott Sipe" : >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Jul 2, 2011, at 12:54 AM, jhell wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> =A0On Fri, Jul 01, 2011 at 03:22:32PM -0700, Jeremy Chadwi= ck >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jul 01, 2011 at 03:13:17PM -0400, Scott Sipe >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm running 8.2-RELEASE and am having new problems >>>>>>>>>>>>>>>>>>> with scp. When scping >>>>>>>>>>>>>>>>>>> files to a ZFS directory on the FreeBSD server -- >>>>>>>>>>>>>>>>>>> most notably large files >>>>>>>>>>>>>>>>>>> -- the transfer frequently dies after just a few >>>>>>>>>>>>>>>>>>> seconds. In my last test, I >>>>>>>>>>>>>>>>>>> tried to scp an 800mb file to the FreeBSD system and >>>>>>>>>>>>>>>>>>> the transfer died after >>>>>>>>>>>>>>>>>>> 200mb. It completely copied the next 4 times I >>>>>>>>>>>>>>>>>>> tried, and then died again on >>>>>>>>>>>>>>>>>>> the next attempt. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On the client side: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> "Connection to home closed by remote host. >>>>>>>>>>>>>>>>>>> lost connection" >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In /var/log/auth.log: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Jul =A01 14:54:42 freebsd sshd[18955]: fatal: Write >>>>>>>>>>>>>>>>>>> failed: Cannot allocate >>>>>>>>>>>>>>>>>>> memory >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I've never seen this before and have used scp before >>>>>>>>>>>>>>>>>>> to transfer large files >>>>>>>>>>>>>>>>>>> without problems. This computer has been used in >>>>>>>>>>>>>>>>>>> production for months and >>>>>>>>>>>>>>>>>>> has a current uptime of 36 days. I have not been >>>>>>>>>>>>>>>>>>> able to notice any problems >>>>>>>>>>>>>>>>>>> copying files to the server via samba or netatalk, or >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> any problems in >>>>>>>>>>> >>>>>>>>>>>> apache. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Uname: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> FreeBSD xeon 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Sat >>>>>>>>>>>>>>>>>>> Feb 19 01:02:54 EST >>>>>>>>>>>>>>>>>>> 2011 =A0 =A0 root@xeon:/usr/obj/usr/src/**sys/GENERIC = =A0amd64 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I've attached my dmesg and output of vmstat -z. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I have not restarted the sshd daemon or rebooted the >>>>>>>>>>>>>>>>>>> computer. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Am glad to provide any other information or test anythi= ng >>>>>>>>>>>>>>>>>>> else. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> {snip vmstat -z and dmesg} >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You didn't provide details about your networking setup >>>>>>>>>>>>>>>>>> (rc.conf, >>>>>>>>>>>>>>>>>> ifconfig -a, etc.). =A0netstat -m would be useful too. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Next, please see this thread circa September 2010, title= d >>>>>>>>>>>>>>>>>> "Network >>>>>>>>>>>>>>>>>> memory allocation failures": >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2010= -** >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> September/thread.html#58708 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The user in that thread is using rsync, which relies on >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> scp by default. >>>>>>>>>>> >>>>>>>>>>>> I believe this problem is similar, if not identical, to yours. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please also provide your output of ( /usr/bin/limits -a ) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> for the server >>>>>>>>>>> >>>>>>>>>>>> end and the client. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I am not quite sure I agree with the need for ifconfig -a >>>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>> information about the networking driver your using for th= e >>>>>>>>>>>>>>>>> interface >>>>>>>>>>>>>>>>> would be helpful, uptime of the boxes. And configuration >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> of the pool. >>>>>>>>> >>>>>>>>>> e.g. ( zpool status -a ;zfs get all ) You should >>>>>>>>>> probably >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> prop this information up somewhere so you can reference b= y >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> URL whenever >>>>>>>>>>> >>>>>>>>>>>> needed. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> rsync(1) does not rely on scp(1) whatsoever but rsync(1) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> can be made to >>>>>>>>>>> >>>>>>>>>>>> use ssh(1) instead of rsh(1) and I believe that is what Jeremy >>>>>>>>>>>> is >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> stating here but correct me if I am wrong. It does use >>>>>>>>>>>>>>>>> ssh(1) >>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>> default. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Its a possiblity as well that if using tmpfs(5) or mdmfs(= 8) >>>>>>>>>>>>>>>>> for /tmp >>>>>>>>>>>>>>>>> type filesystems that rsync(1) may be just filling up you= r >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> temp ram area >>>>>>>>>>> >>>>>>>>>>>> and causing the connection abort which would be >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> expected. ( df -h ) would >>>>>>>>>>>>>>>>> help here. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not using tmpfs/mdmfs at all. The clients yesterday >>>>>>>>>>>>>>>> were 3 different OSX computers (over gigabit). The FreeBSD >>>>>>>>>>>>>>>> server has 12gb of ram and no bce adapter. For what it's >>>>>>>>>>>>>>>> worth, the server is backed up remotely every night with >>>>>>>>>>>>>>>> rsync (remote FreeBSD uses rsync to pull) to an offsite >>>>>>>>>>>>>>>> (slow cable connection) FreeBSD computer, and I have not >>>>>>>>>>>>>>>> seen any errors in the nightly rsync. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry for the omission of networking info, here's the >>>>>>>>>>>>>>>> output of the requested commands and some that popped up >>>>>>>>>>>>>>>> in the other thread: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://www.cap-press.com/misc/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In rc.conf: =A0ifconfig_em1=3D"inet 10.1.1.1 netmask >>>>>>>>>>>>>>>> 255.255.0.0" >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Scott >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> Just to make it crystal clear to everyone: >>>>>>>>>>>> >>>>>>>>>>>> There is no correlation between this problem and use of ZFS. >>>>>>>>>>>> =A0People are >>>>>>>>>>>> attempting to correlate "cannot allocate memory" messages with >>>>>>>>>>>> "anything >>>>>>>>>>>> on the system that uses memory". =A0The VM is much more comple= x >>>>>>>>>>>> than >>>>>>>>>>>> that. >>>>>>>>>>>> >>>>>>>>>>>> Given the nature of this problem, it's much more likely the >>>>>>>>>>>> issue >>>>>>>>>>>> is >>>>>>>>>>>> "somewhere" within a networking layer within FreeBSD, whether = it >>>>>>>>>>>> be >>>>>>>>>>>> driver-level or some sort of intermediary layer. >>>>>>>>>>>> >>>>>>>>>>>> Two people who have this issue in this thread are both using >>>>>>>>>>>> VirtualBox. >>>>>>>>>>>> Can one, or both, of you remove VirtualBox from the >>>>>>>>>>>> configuration >>>>>>>>>>>> entirely (kernel, etc. -- not sure what is required) and then >>>>>>>>>>>> see >>>>>>>>>>>> if the >>>>>>>>>>>> issue goes away? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On the machine in question I only can do it after hours so I wi= ll >>>>>>>>>>> do >>>>>>>>>>> it tonight. >>>>>>>>>>> >>>>>>>>>>> I was _successfully_ sending the file over the loopback interfa= ce >>>>>>>>>>> using >>>>>>>>>>> >>>>>>>>>>> cat /zpool/temp/zimbra_oldroot.vdi | ssh localhost "cat > >>>>>>>>>>> /dev/null" >>>>>>>>>>> >>>>>>>>>>> I did it, btw, with the IPv6 localhost address first >>>>>>>>>>> (accidently), >>>>>>>>>>> and then using IPv4. Both worked. >>>>>>>>>>> >>>>>>>>>>> It always fails if I am sending it through the bce(4) interface= , >>>>>>>>>>> even if my target is the VirtualBox bridged to the bce card (so >>>>>>>>>>> it >>>>>>>>>>> does not "leave" the computer physically). >>>>>>>>>>> >>>>>>>>>>> Below the uname -a, ifconfig -a, netstat -rn, pciconf -lv and >>>>>>>>>>> kldstat output. >>>>>>>>>>> >>>>>>>>>>> I have another box where I do not see that problem. It copies >>>>>>>>>>> files >>>>>>>>>>> happily over the net using ssh. >>>>>>>>>>> >>>>>>>>>>> It is an an older HP ML 150 with 3GB RAM only but with a bge(4) >>>>>>>>>>> driver instead. It runs the same last week's RELENG_8. I >>>>>>>>>>> installed >>>>>>>>>>> VirtualBox and enabled vboxnet (so it loads the kernel modules)= . >>>>>>>>>>> But >>>>>>>>>>> I do not run VirtualBox on it (because it hasn't enough RAM). >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> Peter >>>>>>>>>>> >>>>>>>>>>> DellT410one# uname -a >>>>>>>>>>> FreeBSD DellT410one.vv.fda 8.2-STABLE FreeBSD 8.2-STABLE #1: Th= u >>>>>>>>>>> Jun >>>>>>>>>>> 30 17:07:18 EST 2011 >>>>>>>>>>> root@DellT410one.vv.fda:/usr/**obj/usr/src/sys/GENERIC =A0amd64 >>>>>>>>>>> DellT410one# ifconfig -a >>>>>>>>>>> bce0: flags=3D8943>>>>>>>>>> MULTICAST> >>>>>>>>>>> metric 0 mtu 1500 >>>>>>>>>>> =A0 =A0 =A0 options=3Dc01bb>>>>>>>>>> VLAN_MTU,VLAN_HWTAGGING,JUMBO_**MTU,VLAN_HWCSUM,TSO4,VLAN_** >>>>>>>>>>> HWTSO,LINKSTATE> >>>>>>>>>>> =A0 =A0 =A0 ether 84:2b:2b:68:64:e4 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.220 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.221 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.223 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.224 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.225 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.226 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.227 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 inet 192.168.50.219 netmask 0xffffff00 broadcast >>>>>>>>>>> 192.168.50.255 >>>>>>>>>>> =A0 =A0 =A0 media: Ethernet autoselect (1000baseT = ) >>>>>>>>>>> =A0 =A0 =A0 status: active >>>>>>>>>>> bce1: flags=3D8802 metric 0 mtu = 1500 >>>>>>>>>>> =A0 =A0 =A0 options=3Dc01bb>>>>>>>>>> VLAN_MTU,VLAN_HWTAGGING,JUMBO_**MTU,VLAN_HWCSUM,TSO4,VLAN_** >>>>>>>>>>> HWTSO,LINKSTATE> >>>>>>>>>>> =A0 =A0 =A0 ether 84:2b:2b:68:64:e5 >>>>>>>>>>> =A0 =A0 =A0 media: Ethernet autoselect >>>>>>>>>>> lo0: flags=3D8049 metric 0 mtu >>>>>>>>>>> 16384 >>>>>>>>>>> =A0 =A0 =A0 options=3D3 >>>>>>>>>>> =A0 =A0 =A0 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xb >>>>>>>>>>> =A0 =A0 =A0 inet6 ::1 prefixlen 128 >>>>>>>>>>> =A0 =A0 =A0 inet 127.0.0.1 netmask 0xff000000 >>>>>>>>>>> =A0 =A0 =A0 nd6 options=3D3 >>>>>>>>>>> vboxnet0: flags=3D8802 metric 0 = mtu >>>>>>>>>>> 1500 >>>>>>>>>>> =A0 =A0 =A0 ether 0a:00:27:00:00:00 >>>>>>>>>>> DellT410one# netstat -rn >>>>>>>>>>> Routing tables >>>>>>>>>>> >>>>>>>>>>> Internet: >>>>>>>>>>> Destination =A0 =A0 =A0 =A0Gateway =A0 =A0 =A0 =A0 =A0 =A0Flags= =A0 =A0Refs =A0 =A0 =A0Use >>>>>>>>>>> =A0Netif >>>>>>>>>>> Expire >>>>>>>>>>> default =A0 =A0 =A0 =A0 =A0 =A0192.168.50.201 =A0 =A0 UGS =A0 = =A0 =A0 =A0 0 =A0 =A052195 >>>>>>>>>>> bce0 >>>>>>>>>>> 127.0.0.1 =A0 =A0 =A0 =A0 =A0link#11 =A0 =A0 =A0 =A0 =A0 =A0UH = =A0 =A0 =A0 =A0 =A00 =A0 =A0 =A0 =A06 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.0/24 =A0 =A0link#1 =A0 =A0 =A0 =A0 =A0 =A0 U =A0 =A0= =A0 =A0 =A0 0 =A01118212 >>>>>>>>>>> bce0 >>>>>>>>>>> 192.168.50.219 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 9670 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.220 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 8347 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.221 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 103024 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.223 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A043614 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.224 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 8358 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.225 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 8438 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.226 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 8338 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.50.227 =A0 =A0 link#1 =A0 =A0 =A0 =A0 =A0 =A0 UHS =A0 = =A0 =A0 =A0 0 =A0 =A0 8333 >>>>>>>>>>> =A0lo0 >>>>>>>>>>> 192.168.165.0/24 =A0 192.168.50.200 =A0 =A0 UGS =A0 =A0 =A0 =A0= 0 =A0 =A0 3311 >>>>>>>>>>> bce0 >>>>>>>>>>> 192.168.166.0/24 =A0 192.168.50.200 =A0 =A0 UGS =A0 =A0 =A0 =A0= 0 =A0 =A0 =A0699 >>>>>>>>>>> bce0 >>>>>>>>>>> 192.168.167.0/24 =A0 192.168.50.200 =A0 =A0 UGS =A0 =A0 =A0 =A0= 0 =A0 =A0 3012 >>>>>>>>>>> bce0 >>>>>>>>>>> 192.168.168.0/24 =A0 192.168.50.200 =A0 =A0 UGS =A0 =A0 =A0 =A0= 0 =A0 =A0 =A0552 >>>>>>>>>>> bce0 >>>>>>>>>>> >>>>>>>>>>> Internet6: >>>>>>>>>>> Destination =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Gateway >>>>>>>>>>> Flags =A0 =A0 =A0Netif Expire >>>>>>>>>>> ::1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= ::1 >>>>>>>>>>> UH >>>>>>>>>>> lo0 >>>>>>>>>>> fe80::%lo0/64 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 link#11 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 U >>>>>>>>>>> lo0 >>>>>>>>>>> fe80::1%lo0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 link#11 >>>>>>>>>>> UHS >>>>>>>>>>> lo0 >>>>>>>>>>> ff01::%lo0/32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 fe80::1%l= o0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 U >>>>>>>>>>> lo0 >>>>>>>>>>> ff02::%lo0/32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 fe80::1%l= o0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 U >>>>>>>>>>> lo0 >>>>>>>>>>> DellT410one# kldstat >>>>>>>>>>> Id Refs Address =A0 =A0 =A0 =A0 =A0 =A0Size =A0 =A0 Name >>>>>>>>>>> 1 =A0 19 0xffffffff80100000 dbf5d0 =A0 kernel >>>>>>>>>>> 2 =A0 =A03 0xffffffff80ec0000 4c358 =A0 =A0vboxdrv.ko >>>>>>>>>>> 3 =A0 =A01 0xffffffff81012000 131998 =A0 zfs.ko >>>>>>>>>>> 4 =A0 =A01 0xffffffff81144000 1ff1 =A0 =A0 opensolaris.ko >>>>>>>>>>> 5 =A0 =A02 0xffffffff81146000 2940 =A0 =A0 vboxnetflt.ko >>>>>>>>>>> 6 =A0 =A02 0xffffffff81149000 8e38 =A0 =A0 netgraph.ko >>>>>>>>>>> 7 =A0 =A01 0xffffffff81152000 153c =A0 =A0 ng_ether.ko >>>>>>>>>>> 8 =A0 =A01 0xffffffff81154000 e70 =A0 =A0 =A0vboxnetadp.ko >>>>>>>>>>> DellT410one# pciconf -lv >>>>>>>>>>> .. >>>>>>>>>>> bce0@pci0:1:0:0: =A0 =A0 =A0 =A0class=3D0x020000 card=3D0x028d1= 028 >>>>>>>>>>> chip=3D0x163b14e4 rev=3D0x20 hdr=3D0x00 >>>>>>>>>>> =A0vendor =A0 =A0 =3D 'Broadcom Corporation' >>>>>>>>>>> =A0class =A0 =A0 =A0=3D network >>>>>>>>>>> =A0subclass =A0 =3D ethernet >>>>>>>>>>> bce1@pci0:1:0:1: =A0 =A0 =A0 =A0class=3D0x020000 card=3D0x028d1= 028 >>>>>>>>>>> chip=3D0x163b14e4 rev=3D0x20 hdr=3D0x00 >>>>>>>>>>> =A0vendor =A0 =A0 =3D 'Broadcom Corporation' >>>>>>>>>>> =A0class =A0 =A0 =A0=3D network >>>>>>>>>>> =A0subclass =A0 =3D ethernet >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Could you please provide "pciconf -lvcb" output instead, specifi= c >>>>>>>>>> to >>>>>>>>>> the >>>>>>>>>> bce chips? =A0Thanks. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Her it is: >>>>>>>>> >>>>>>>>> bce0@pci0:1:0:0: =A0 =A0 =A0 =A0class=3D0x020000 card=3D0x028d102= 8 >>>>>>>>> chip=3D0x163b14e4 rev=3D0x20 hdr=3D0x00 >>>>>>>>> =A0vendor =A0 =A0 =3D 'Broadcom Corporation' >>>>>>>>> =A0class =A0 =A0 =A0=3D network >>>>>>>>> =A0subclass =A0 =3D ethernet >>>>>>>>> =A0bar =A0 [10] =3D type Memory, range 64, base 0xda000000, size >>>>>>>>> 33554432, enabled >>>>>>>>> =A0cap 01[48] =3D powerspec 3 =A0supports D0 D3 =A0current D0 >>>>>>>>> =A0cap 03[50] =3D VPD >>>>>>>>> =A0cap 05[58] =3D MSI supports 16 messages, 64 bit enabled with 1 >>>>>>>>> message >>>>>>>>> =A0cap 11[a0] =3D MSI-X supports 9 messages in map 0x10 >>>>>>>>> =A0cap 10[ac] =3D PCI-Express 2 endpoint max data 256(512) link x= 4(x4) >>>>>>>>> ecap 0003[100] =3D Serial 1 842b2bfffe6864e4 >>>>>>>>> ecap 0001[110] =3D AER 1 0 fatal 0 non-fatal 1 corrected >>>>>>>>> ecap 0004[150] =3D unknown 1 >>>>>>>>> ecap 0002[160] =3D VC 1 max VC0 >>>>>>>>> >>>>>>>> >>>>>>>> Thanks Peter. >>>>>>>> >>>>>>>> Adding Yong-Hyeon and David to the discussion, since they've both >>>>>>>> worked >>>>>>>> on the bce(4) driver in recent months (most of the changes made >>>>>>>> recently >>>>>>>> are only in HEAD), and also adding Jack Vogel of Intel who maintai= ns >>>>>>>> em(4). =A0Brief history for the devs: >>>>>>>> >>>>>>>> The issue is described "Network memory allocation failures" and wa= s >>>>>>>> reported last year, but two users recently (Scott and Peter) have >>>>>>>> reported the issue again: >>>>>>>> >>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2010-** >>>>>>>> >>>>>>>> September/thread.html#58708 >>>>>>>> >>>>>>>> And was mentioned again by Scott here, which also contains some >>>>>>>> technical details: >>>>>>>> >>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2011-** >>>>>>>> >>>>>>>> July/063172.html >>>>>>>> >>>>>>>> What's interesting is that Scott's issue is identical in form but >>>>>>>> he's >>>>>>>> using em(4), which isn't known to behave like this. =A0Both >>>>>>>> individuals >>>>>>>> are using VirtualBox, though we're not sure at this point if that = is >>>>>>>> the >>>>>>>> piece which is causing the anomaly. >>>>>>>> >>>>>>>> Relevant details of Scott's system (em-based): >>>>>>>> >>>>>>>> http://www.cap-press.com/misc/ >>>>>>>> >>>>>>>> Relevant details of Peter's system (bce-based): >>>>>>>> >>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2011-** >>>>>>>> >>>>>>>> July/063221.html >>>>>>>> http://lists.freebsd.org/**pipermail/freebsd-stable/2011-** >>>>>>>> >>>>>>>> July/063223.html >>>>>>>> >>>>>>>> I think the biggest complexity right now is figuring out how/why s= cp >>>>>>>> fails intermittently in this nature. =A0The errno probably "trickl= es >>>>>>>> down" >>>>>>>> to userland from the kernel, but the condition regarding why it >>>>>>>> happens >>>>>>>> is unknown. >>>>>>>> >>>>>>> >>>>>>> BTW: I also saw 2 of the errors coming from a BIND9 running in a >>>>>>> jail on that box. >>>>>>> >>>>>>> DellT410one# fgrep -i allocate >>>>>>> /jails/bind/20110315/var/log/**messages >>>>>>> Apr 13 05:17:41 bind named[23534]: internal_send: >>>>>>> 192.168.50.145#65176: Cannot allocate memory >>>>>>> Jun 21 23:30:44 bind named[39864]: internal_send: >>>>>>> 192.168.50.251#36155: Cannot allocate memory >>>>>>> Jun 24 15:28:00 bind named[39864]: internal_send: >>>>>>> 192.168.50.251#28651: Cannot allocate memory >>>>>>> Jun 28 12:57:52 bind named[2462]: internal_send: >>>>>>> 192.168.165.154#1201: Cannot allocate memory >>>>>>> >>>>>>> My initial guess: it happens sooner or later somehow - whether it i= s >>>>>>> a lot of traffic in one go (ssh/scp copies of virtual disks) or a >>>>>>> lot of traffic over a longer period (a nameserver gets asked again >>>>>>> and again). >>>>>>> >>>>>> >>>>>> Scott, are you also using jails? =A0If both of you are: is there any >>>>>> possibility you can remove use of those? =A0I'm not sure how Virtual= Box >>>>>> fits into the picture (jails + VirtualBox that is), but I can imagin= e >>>>>> jails having different environmental constraints that might cause >>>>>> this. >>>>>> >>>>>> Basically the troubleshooting process here is to remove pieces of th= e >>>>>> puzzle until you figure out which piece is causing the issue. =A0I d= on't >>>>>> want to get the NIC driver devs all spun up for something that, for >>>>>> example, might be an issue with the jail implementation. >>>>>> >>>>> >>>>> I understand this. As said, I do some afterhours debugging tonight. >>>>> >>>>> The scp/ssh problems are happening _outside_ the jails. The bind runs >>>>> _inside_ the jail. >>>>> >>>>> I wanted to use the _host_ system to send VirtualBox virtual disks an= d >>>>> =A0filesystems used by jails to archive them and/or having them avail= able >>>>> on >>>>> other FreeBSD systems (as a cold standby solution). >>>>> >>>> >>>> I just switched off the VirtualBox (without removing the kernel >>>> modules). >>>> >>>> The copy succeeds now. >>>> >>>> Well, it could be a VirtualBox related problem, or is the server just >>>> relieved to have 2GB more memory at hands now? >>>> >>>> Do you have a quick idea to "emulate" the 2GB memory load usually >>>> delivered by VirtualBox? >>>> >>> >>> Well, managed that (using lookbusy) >>> >>> Interestingly I could copy a large file (30GB) without problems, as soo= n >>> as >>> I switched off the VirtualBox. As said, the kernel modules weren't >>> unloaded, >>> they are still there. >>> >>> The copy crashes seconds after I started the VirtualBox. According to >>> vmstat and top I had more free memory (ca. 1.5GB) as I had without >>> VirtualBox and lookbusy (ca. 350MB). >>> >>> So, it looks (to me, at least) as I have a VirtualBox related problem, >>> somehow. >>> >>> Any ideas? I am happy to play a bit more to get it sorted although it h= as >>> some limits (it is running the company mailserver, after all) >>> >>> Regards >>> Peter >>> >> >> This is it -- I'm seeing the exact same thing. >> >> Scp dies reliably with VirtualBox running. Quit VirtualBox and I was abl= e >> to >> scp about 30 large files with no errors. Once I started VirtualBox an >> in-progress scp died within seconds. >> >> Ditto that the Kernel modules merely being loaded don't seem to make a >> difference, it's VirtualBox actually running. >> >> virtualbox-ose-3.2.12_1 > > Hi, > > I wonder whether anyone has new ideas. > > I am puzzled that it happens when VirtualBoxes are running, while the loa= d > or unload of the VirtualBox kernel modules doesn't seem to have an effect= . > > Should I describe the case at the -emulation mailing list to get some ide= as > from the engineers working on VirtualBox? > > I do not want to create too much noise so I would like to know your thoug= hts > on it first. > > I experimented a little bit with the ssh code and know which write(2) in > /usr/src/crypto/openssh/roaming_common.c (in function roaming_write) retu= rns > the ENOMEM (an error it should never return, according to the mainpage;-) > > but unfortunately I am lost to track it further down in the kernel. I do = not > know enough about it, to be frankly. > > Are there any memory stats inside the kernel that could help? > > Thank you for all ideas > Peter > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 08:25:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D511106564A for ; Mon, 25 Jul 2011 08:25:04 +0000 (UTC) (envelope-from amon@aelita.org) Received: from aelita.org (unknown [IPv6:2001:7a8:7003::3]) by mx1.freebsd.org (Postfix) with ESMTP id BA9FC8FC13 for ; Mon, 25 Jul 2011 08:25:03 +0000 (UTC) Received: from ra.aabs (localhost [127.0.0.1]) by aelita.org (8.14.4/8.14.4) with ESMTP id p6PAL7ud082363 for ; Mon, 25 Jul 2011 12:21:07 +0200 (CEST) (envelope-from amon@ra.aabs) Received: (from amon@localhost) by ra.aabs (8.14.4/8.14.4/Submit) id p6PAL7Sd082362 for freebsd-stable@freebsd.org; Mon, 25 Jul 2011 12:21:07 +0200 (CEST) (envelope-from amon) Date: Mon, 25 Jul 2011 12:21:07 +0200 From: Herve Boulouis To: freebsd-stable@freebsd.org Message-ID: <20110725102107.GB17204@ra.aabs> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 08:25:04 -0000 Hi list, We have 2 freebsd 8.2-STABLE (cvsuped june 22) that keeps crashing in a bad way : The are doing heavy apache / php4 web serving from a nfs mount and panic at least once a day with the following message (no crash dump produced, hand copied from the console) : Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) r = 0 (0xffffff0201798000) locked @ nfsclient/nfs_subs.c:538 lock order reversal: 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_turnstile.c:190 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 lock order reversal: 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_turnstile.c:190 2nd 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnstile.c:203 lock order reversal: 1st 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnstile.c:203 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 Sleeping thread (tid 100998, pid 20700) owns a non-sleepable lock panic: sleeping thread cpuid = 1 panic: bufwrite: buffer is not busy??? cpuid = 1 The 2 servers share the same load and panic consistently. I enabled WITNESS on the 2 in the hope it would allow the boxes to auto reboot after panic and get extra debug info. I got debug info but the servers still hangs after the double panic :( I also noticed that immediately after rebooting following this panic, I got the following LORs (approximatively at the time rc.d is launching ports like apache & co) lock order reversal: 1st 0xffffff81ee00e388 bufwait (bufwait) @ kern/vfs_bio.c:2636 2nd 0xffffff0006e56c00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:285 lock order reversal: 1st 0xffffff0009c709e0 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xffffff0124282620 ufs (ufs) @ kern/uipc_syscalls.c:2086 lock order reversal: 1st 0xffffff0009c709e0 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xffffff01243569d0 nfs (nfs) @ kern/uipc_syscalls.c:2086 The server continued to work despite the lors so I don't know if this is related to the panics or not. What can I do from there to debug this further ? Regards, -- Herve Boulouis From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 08:59:10 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A237106564A; Mon, 25 Jul 2011 08:59:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 841348FC0A; Mon, 25 Jul 2011 08:59:08 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6P8x3YB080833 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 25 Jul 2011 11:59:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6P8x3b2033855; Mon, 25 Jul 2011 11:59:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6P8x3Zw033854; Mon, 25 Jul 2011 11:59:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 25 Jul 2011 11:59:03 +0300 From: Kostik Belousov To: Herve Boulouis Message-ID: <20110725085902.GM17489@deviant.kiev.zoral.com.ua> References: <20110725102107.GB17204@ra.aabs> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OxDl9SlxSp5FbYFo" Content-Disposition: inline In-Reply-To: <20110725102107.GB17204@ra.aabs> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: rmacklem@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 08:59:10 -0000 --OxDl9SlxSp5FbYFo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 25, 2011 at 12:21:07PM +0200, Herve Boulouis wrote: > Hi list, >=20 > We have 2 freebsd 8.2-STABLE (cvsuped june 22) that keeps crashing in a b= ad way : >=20 > The are doing heavy apache / php4 web serving from a nfs mount and panic = at least once a day > with the following message (no crash dump produced, hand copied from the = console) : >=20 > Sleeping on "vmopar" with the following non-sleepable locks held: > exclusive sleep mutex NFSnode lock (NFSnode lock) r =3D 0 (0xffffff02017= 98000) locked @ nfsclient/nfs_subs.c:538 > lock order reversal: > 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_tur= nstile.c:190 > 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 > lock order reversal: > 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_tur= nstile.c:190 > 2nd 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnsti= le.c:203 > lock order reversal: > 1st 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnsti= le.c:203 > 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 > Sleeping thread (tid 100998, pid 20700) owns a non-sleepable lock > panic: sleeping thread > cpuid =3D 1 > panic: bufwrite: buffer is not busy??? > cpuid =3D 1 >=20 > The 2 servers share the same load and panic consistently. I enabled WITNE= SS on the 2 in the hope > it would allow the boxes to auto reboot after panic and get extra debug i= nfo. I got debug info > but the servers still hangs after the double panic :( Try this. Calling vnode_pager_setsize() while holding a mutex is prohibited. On the other hand, I remember that my attempt to add a strict assert that a vnode is exclusively locked in vnode_pager_setsize() had to be reversed because nfs_loadattrcache() sometimes called without vnode lock held. commit 2aa7d15c38b0c01e3f724f04d7ed02ce11c82cc0 Author: Konstantin Belousov Date: Mon Jul 25 11:56:04 2011 +0300 Postpone the vnode_pager_setsize() call until the nfs node mutex is dro= pped. diff --git a/sys/nfsclient/nfs_subs.c b/sys/nfsclient/nfs_subs.c index 19fde06..351885a 100644 --- a/sys/nfsclient/nfs_subs.c +++ b/sys/nfsclient/nfs_subs.c @@ -478,7 +478,9 @@ nfs_loadattrcache(struct vnode **vpp, struct mbuf **mdp= , caddr_t *dposp, struct timespec mtime, mtime_save; int v3 =3D NFS_ISV3(vp); int error =3D 0; + int do_setsize; =20 + do_setsize =3D 0; md =3D *mdp; t1 =3D (mtod(md, caddr_t) + md->m_len) - *dposp; cp2 =3D nfsm_disct(mdp, dposp, NFSX_FATTR(v3), t1, M_WAIT); @@ -606,7 +608,7 @@ nfs_loadattrcache(struct vnode **vpp, struct mbuf **mdp= , caddr_t *dposp, np->n_size =3D vap->va_size; np->n_flag |=3D NSIZECHANGED; } - vnode_pager_setsize(vp, np->n_size); + do_setsize =3D 1; } else { np->n_size =3D vap->va_size; } @@ -643,6 +645,8 @@ nfs_loadattrcache(struct vnode **vpp, struct mbuf **mdp= , caddr_t *dposp, KDTRACE_NFS_ATTRCACHE_LOAD_DONE(vp, &np->n_vattr, 0); #endif mtx_unlock(&np->n_mtx); + if (do_setsize) + vnode_pager_setsize(vp, np->n_size); out: #ifdef KDTRACE_HOOKS if (error) --OxDl9SlxSp5FbYFo Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4tMFYACgkQC3+MBN1Mb4hbdQCdFW1D6Ic5r1zMXlMEMV0GoieS pbQAoL7U3cJ2KV17OwDi6JkqnQQc+cQe =8/06 -----END PGP SIGNATURE----- --OxDl9SlxSp5FbYFo-- From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 09:34:32 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3770106566C for ; Mon, 25 Jul 2011 09:34:32 +0000 (UTC) (envelope-from lists@c0mplx.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7BB278FC13 for ; Mon, 25 Jul 2011 09:34:32 +0000 (UTC) Received: from pi by home.opsec.eu with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1QlHYW-000P8r-SY for freebsd-stable@freebsd.org; Mon, 25 Jul 2011 11:34:32 +0200 Date: Mon, 25 Jul 2011 11:34:32 +0200 From: Kurt Jaeger To: freebsd-stable@freebsd.org Message-ID: <20110725093432.GI1202@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 09:34:32 -0000 Hi! What kind of SATA 6g 4-port non-RAID controller is currently suggested for use in 8/9 setups with large RAM (64G) setups with ZFS ? Thanks! -- pi@opsec.eu +49 171 3101372 9 years to go ! From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 12:23:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA2AA106564A; Mon, 25 Jul 2011 12:23:41 +0000 (UTC) (envelope-from amon@aelita.org) Received: from aelita.org (unknown [IPv6:2001:7a8:7003::3]) by mx1.freebsd.org (Postfix) with ESMTP id 11D468FC14; Mon, 25 Jul 2011 12:23:40 +0000 (UTC) Received: from ra.aabs (localhost [127.0.0.1]) by aelita.org (8.14.4/8.14.4) with ESMTP id p6PEJcri083751; Mon, 25 Jul 2011 16:19:38 +0200 (CEST) (envelope-from amon@ra.aabs) Received: (from amon@localhost) by ra.aabs (8.14.4/8.14.4/Submit) id p6PEJbAl083750; Mon, 25 Jul 2011 16:19:37 +0200 (CEST) (envelope-from amon) Date: Mon, 25 Jul 2011 16:19:37 +0200 From: Herve Boulouis To: Kostik Belousov Message-ID: <20110725141937.GE17204@ra.aabs> References: <20110725102107.GB17204@ra.aabs> <20110725085902.GM17489@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=unknown-8bit Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110725085902.GM17489@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i Cc: rmacklem@freebsd.org, Herve Boulouis , freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 12:23:41 -0000 Le 25/07/2011 11:59, Kostik Belousov a écrit: > On Mon, Jul 25, 2011 at 12:21:07PM +0200, Herve Boulouis wrote: > > Hi list, > > > > We have 2 freebsd 8.2-STABLE (cvsuped june 22) that keeps crashing in a bad way : > > > > The are doing heavy apache / php4 web serving from a nfs mount and panic at least once a day > > with the following message (no crash dump produced, hand copied from the console) : > > > > Sleeping on "vmopar" with the following non-sleepable locks held: > > exclusive sleep mutex NFSnode lock (NFSnode lock) r = 0 (0xffffff0201798000) locked @ nfsclient/nfs_subs.c:538 > > lock order reversal: > > 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_turnstile.c:190 > > 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 > > lock order reversal: > > 1st 0xffffffff018ff6da80 turnstile lock (turnstile lock) @ kern/subr_turnstile.c:190 > > 2nd 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnstile.c:203 > > lock order reversal: > > 1st 0xffffffffff80b78ef8 sleepq chain (sleepq chain) @ kern/subr_turnstile.c:203 > > 2nd 0xffffffffff80b52b10 scrlock (scrlock) @ dev/syscons.c:2570 > > Sleeping thread (tid 100998, pid 20700) owns a non-sleepable lock > > panic: sleeping thread > > cpuid = 1 > > panic: bufwrite: buffer is not busy??? > > cpuid = 1 > > > > The 2 servers share the same load and panic consistently. I enabled WITNESS on the 2 in the hope > > it would allow the boxes to auto reboot after panic and get extra debug info. I got debug info > > but the servers still hangs after the double panic :( > > Try this. Calling vnode_pager_setsize() while holding a mutex is prohibited. > On the other hand, I remember that my attempt to add a strict assert > that a vnode is exclusively locked in vnode_pager_setsize() had to be > reversed because nfs_loadattrcache() sometimes called without vnode > lock held. > > commit 2aa7d15c38b0c01e3f724f04d7ed02ce11c82cc0 > Author: Konstantin Belousov > Date: Mon Jul 25 11:56:04 2011 +0300 > > Postpone the vnode_pager_setsize() call until the nfs node mutex is dropped. 1 of the boxes crashed so its kernel is now running with the patch. I still get the 3 LORs when services are starting thought : lock order reversal: 1st 0xffffff81ee061268 bufwait (bufwait) @ kern/vfs_bio.c:2636 2nd 0xffffff0006901000 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:285 lock order reversal: 1st 0xffffff0125236c88 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xffffff01256e9448 nfs (nfs) @ kern/uipc_syscalls.c:2086 lock order reversal: 1st 0xffffff01253e1c88 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xffffff01252b5620 ufs (ufs) @ kern/uipc_syscalls.c:2086 I'll keep you posted if the patch improves the stability or not. Regards, -- Herve Boulouis From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 13:24:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB2CF1065674 for ; Mon, 25 Jul 2011 13:24:12 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 56C918FC08 for ; Mon, 25 Jul 2011 13:24:12 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QlL8j-000AgX-Hc; Mon, 25 Jul 2011 09:24:09 -0400 Date: Mon, 25 Jul 2011 09:24:09 -0400 From: Gary Palmer To: Chuck Swiger Message-ID: <20110725132409.GA1339@in-addr.com> References: <20110718234124.GA5626@icarus.home.lan> <20110719211039.GA16085@server.vk2pj.dyndns.org> <02D367A5-CA74-4E8A-BE3E-F81485B287A7@mac.com> <4e26a250.iKKzhkOLoTB3sdOr%perryh@pluto.rain.com> <20110720031431.GA33758@icarus.home.lan> <84CC369B-4E70-414C-8C57-5FE772C7134F@mac.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <84CC369B-4E70-414C-8C57-5FE772C7134F@mac.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: freebsd-stable@freebsd.org Subject: Re: Status of support for 4KB disk sectors X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 13:24:12 -0000 On Tue, Jul 19, 2011 at 08:48:54PM -0700, Chuck Swiger wrote: > On Jul 19, 2011, at 8:14 PM, Jeremy Chadwick wrote: > > On Wed, Jul 20, 2011 at 02:39:28AM -0700, perryh@pluto.rain.com wrote: > >> IIRC, Plextor (and maybe some others) had a switch to select 512 or > >> 2048 as the default transfer size, precisely so that they could be > >> used as boot devices with systems that supported only 512. > > Come to think of it, I do remember that switch, yes. > > Do you happen to know whether this limitation was part of the Sun hardware, > or of SunOS? CMU had a lot of Sun3 machines and NeXT clusters, so I ended > up mixing NeXT CD-ROM and the Canon? magneto-optical drives with Sun H/W, > and vice versa. My memory is a bit fuzzy but I beleive it is at least partly because they used ufs on the CDs rather than ISO 9660 based filesystems so they could boot from the CD and have the device nodes there as well as all the other features a ufs filesystem supports but ISO 9660 doesn't. Gary From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 13:59:16 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A96D106564A; Mon, 25 Jul 2011 13:59:16 +0000 (UTC) (envelope-from cscotts@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id B5F028FC13; Mon, 25 Jul 2011 13:59:15 +0000 (UTC) Received: by vxg33 with SMTP id 33so3955819vxg.13 for ; Mon, 25 Jul 2011 06:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=tziKXXK9vcI6NeVIBiitVBKUGdFkoekurVbwZSsFzNE=; b=VMO/QPj/Lm9WI38uQsfPOAo/eC96Zms67Cd3Hfizkc3Akf4nOcMgd5H2/0Acp8beNK wvg05p3PKgAFlscza62L5mF5/KsAW6RkH/fHJFakuE+9l0YXy4q8XKmuiPn2v+mrzrIx AFCaRGxqHe8gSLNERbfDTY9drkfj1i0+lvDSE= Received: by 10.52.67.84 with SMTP id l20mr4398654vdt.285.1311602355047; Mon, 25 Jul 2011 06:59:15 -0700 (PDT) Received: from [192.168.1.44] (90.sub-75-246-109.myvzw.com [75.246.109.90]) by mx.google.com with ESMTPS id ca9sm2649706vdc.39.2011.07.25.06.59.11 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Jul 2011 06:59:13 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Scott Sipe In-Reply-To: Date: Mon, 25 Jul 2011 09:59:08 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <1AFA4756-A1C0-4129-8134-2B43724947DC@gmail.com> References: <20110706122339.61453nlqra1vqsrv@webmail.in-berlin.de> <20110706023234.GA72048@icarus.home.lan> <20110706130753.182053f3ellasn0p@webmail.in-berlin.de> <20110706032425.GA72757@icarus.home.lan> <20110706135412.15276i0fxavg09k4@webmail.in-berlin.de> <20110706041504.GA73698@icarus.home.lan> <20110706143129.10696235ldx9bjmp@webmail.in-berlin.de> <20110706173242.23404ffbhkxz6mqi@webmail.in-berlin.de> <20110706182141.13056plxp148y61h@webmail.in-berlin.de> <20110711115947.51686v4930s7ze37@webmail.in-berlin.de> To: Adrian Chadd X-Mailer: Apple Mail (2.1084) Cc: Yong-Hyeon Pyun , freebsd-stable List , Peter Ross , davidch@freebsd.org, "Vogel, Jack" , Jeremy Chadwick Subject: Re: scp: Write Failed: Cannot allocate memory X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 13:59:16 -0000 On Jul 24, 2011, at 10:07 PM, Adrian Chadd wrote: > Has someone asked for the output of netstat -mb? That error message is > mbuf related, so I bet it's something to do with mbuf allocation. >=20 > Is it possible that the system is incorrectly tuned when virtualbox is = enabled? >=20 >=20 > Adrian I never ran netstat -mb when I was encountering the network memory = allocation error. At the moment I have adjusted net.graph.maxdata from = the default value to 8192, rebooted, and have not encountered any more = network memory allocation errors (and vmstat -z shows 0 NetGraph data = item failures). If, at this maxdata setting, they show up again I'll = post the output of netstat -mb. If it's worth anything, current output = of netstat -mb is below. # netstat -mb 1027/6668/7695 mbufs in use (current/cache/total) 1024/4034/5058/25600 mbuf clusters in use (current/cache/total/max) 1024/3456 mbuf+clusters out of packet secondary zone in use = (current/cache) 0/1303/1303/12800 4k (page size) jumbo clusters in use = (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 2304K/14947K/17251K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 19 requests for I/O initiated by sendfile 0 calls to protocol drain routines Scott= From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 16:50:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09568106566B for ; Mon, 25 Jul 2011 16:50:01 +0000 (UTC) (envelope-from markmc@dataabstractsolutions.com) Received: from new-smtp02.pacifier.net (new-smtp02.pacifier.net [64.255.237.176]) by mx1.freebsd.org (Postfix) with ESMTP id E3C478FC0A for ; Mon, 25 Jul 2011 16:50:00 +0000 (UTC) Received: from [192.168.1.35] (75-150-37-150-Oregon.hfc.comcastbusiness.net [75.150.37.150]) by new-smtp02.pacifier.net (Postfix) with ESMTP id 79C022CA21; Mon, 25 Jul 2011 09:50:00 -0700 (PDT) From: "Mark McConnell" To: freebsd-stable@freebsd.org Date: Mon, 25 Jul 2011 09:49:59 -0700 MIME-Version: 1.0 Message-ID: <4E2D9EB7.13030.199D25ED@markmc.dataabstractsolutions.com> Priority: normal In-reply-to: <4E24C902.4460.164669ED@markmc.dataabstractsolutions.com> References: <4E20BA23.13717.66C6F57@markmcconnell.iinet.com>, <797CACDE-729E-4F3A-AEFF-531C00C2B83A@samsco.org>, <4E24C902.4460.164669ED@markmc.dataabstractsolutions.com> X-mailer: Pegasus Mail for Windows (4.52) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: disable 64-bit dma for one PCI slot only? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: markmc@dataabstractsolutions.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 16:50:01 -0000 On 18 Jul 2011 at 17:00, Mark McConnell wrote: {Re: disable 64-bit dma for one PCI ...}: > On 18 Jul 2011 at 15:06, Scott Long wrote: > {Re: disable 64-bit dma for one PCI ...}: > > > >> I would like to disable 64-bit addressing for the SATA card, but > > >> permit it for the SCSI card. Is this possible? > > > > > > You'd have to hack the driver perhaps to only disable 64-bit DMA for certain > > > PCI IDs. It probably already does this? > > > > > > > The driver already had a table for determining 64bit DMA > > based on the PCI ID. I guess there's a mistake in the > > table for this particular card. I think that changing > > the following line to remove the AMR_ID_DO_SG64 flag > > will fix the problem: > > > > {0x1000, 0x1960, AMR_ID_QUARTZ | AMR_ID_DO_SG64 | AMR_ID_PROBE_SIG}, > > > > Actually, what's probably going on is that the driver is > > only looking at the vendor and device id's, and is > > ignoring the subvendor and subdevice id's that would > > give it a better clue on the exact hardware in use. > > Fixing the driver to look at all 64bits of id info (and > > take into account wildcards where needed) would be a > > good project, if anyone is interested. > > > > Btw, I *HATE* the "chip" and "card" identifiers used in > > pciconf. Can we change it to emit the standard > > (sub)vendor/(sub)device terminology? > > > > Scott > > Thank you for the discussion, John and Scott. I see > where this change would be made, Scott, and I want to try > it when I have an opportunity. > > Mark I made this change to the amr driver, and it works as advertised. Both cards are prevented from using 64-bit DMA however, because the id's are the same as far as they are examined. Mark -- From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 18:15:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CD66106566B for ; Mon, 25 Jul 2011 18:15:07 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 518108FC0C for ; Mon, 25 Jul 2011 18:15:07 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QlPgI-000B3O-4N; Mon, 25 Jul 2011 14:15:06 -0400 Date: Mon, 25 Jul 2011 14:15:06 -0400 From: Gary Palmer To: Bakul Shah Message-ID: <20110725181506.GC1339@in-addr.com> References: <20110721215617.7C7F0B827@mail.bitblocks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110721215617.7C7F0B827@mail.bitblocks.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: freebsd-stable@freebsd.org Subject: Re: OS X Lion time machine => (afpd|iSCSI) => ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 18:15:07 -0000 On Thu, Jul 21, 2011 at 02:56:17PM -0700, Bakul Shah wrote: > I am in no hurry to upgrade my MBP to OS X Lion but given Lion > time machine and netatalk issues, I got wondering if iSCSI on > FreeBSD is stable enough for time machine use. How much duct > tape and baling wire are needed to make it work?! Just a word of caution - I read somewhere that using iSCSI for Time Machine means that you cannot use TM during an OS reinstall to restore your files from the Time Machine archive as iSCSI isn't available at that point. It seems you have to use a locally attached disk or AFP for the support to be there during reinstall. Not sure if thats still the case in Lion or not, I would tend to suspect they still don't consider iSCSI a top tier transport. Regards, Gary From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 18:20:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2D751065676 for ; Mon, 25 Jul 2011 18:20:54 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f44.google.com (mail-fx0-f44.google.com [209.85.161.44]) by mx1.freebsd.org (Postfix) with ESMTP id 144B48FC1C for ; Mon, 25 Jul 2011 18:20:53 +0000 (UTC) Received: by fxe6 with SMTP id 6so7067262fxe.17 for ; Mon, 25 Jul 2011 11:20:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=9k+EfXs+2TA5YrmEc/ETaX7kSgg+3WXtC00R6cIYzb4=; b=EckejKZljVI8L8VQAOY0rbzbzZdqt+nl728d1ZY/dV9+viP+Qhkc9D3MoH4g0HsHj+ vKIaSwQtjtxWb7+dZPCQjK7+sBsbfoymhxdXHIYqH2yxUhZ3xufWruB4QaPyaqiBEKlc 1ThdEGPsS2+zX5NoirAz+qoTVMfZM09KE22sM= Received: by 10.223.56.6 with SMTP id w6mr4783505fag.92.1311618052720; Mon, 25 Jul 2011 11:20:52 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id h10sm3934433fai.19.2011.07.25.11.20.51 (version=SSLv3 cipher=OTHER); Mon, 25 Jul 2011 11:20:51 -0700 (PDT) Sender: Alexander Motin Message-ID: <4E2DB3F5.3070604@FreeBSD.org> Date: Mon, 25 Jul 2011 21:20:37 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110709 Thunderbird/5.0 MIME-Version: 1.0 To: Kurt Jaeger References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 18:20:54 -0000 On 25.07.2011 12:34, Kurt Jaeger wrote: > What kind of SATA 6g 4-port non-RAID controller is currently suggested > for use in 8/9 setups with large RAM (64G) setups with ZFS ? If you need exactly SATA, not SAS 6g controller, then choice is not so big: either something integrated into latest Intel (only two ports) or AMD (6 ports) chipset, or something based on Marvell 88SE91xx chips. Last case also has only 2 ports, but you may install two cards, or use Highpoint RocketRAID 640, which is just two above chips connected with PCIe bridge. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 18:51:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF0581065670 for ; Mon, 25 Jul 2011 18:51:13 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id BDB268FC1E for ; Mon, 25 Jul 2011 18:51:13 +0000 (UTC) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 2C956B827; Mon, 25 Jul 2011 11:51:13 -0700 (PDT) To: Gary Palmer In-reply-to: Your message of "Mon, 25 Jul 2011 14:15:06 EDT." <20110725181506.GC1339@in-addr.com> References: <20110721215617.7C7F0B827@mail.bitblocks.com> <20110725181506.GC1339@in-addr.com> Comments: In-reply-to Gary Palmer message dated "Mon, 25 Jul 2011 14:15:06 -0400." Date: Mon, 25 Jul 2011 11:51:13 -0700 From: Bakul Shah Message-Id: <20110725185113.2C956B827@mail.bitblocks.com> Cc: freebsd-stable@freebsd.org Subject: Re: OS X Lion time machine => (afpd|iSCSI) => ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 18:51:13 -0000 On Mon, 25 Jul 2011 14:15:06 EDT Gary Palmer wrote: > On Thu, Jul 21, 2011 at 02:56:17PM -0700, Bakul Shah wrote: > > I am in no hurry to upgrade my MBP to OS X Lion but given Lion > > time machine and netatalk issues, I got wondering if iSCSI on > > FreeBSD is stable enough for time machine use. How much duct > > tape and baling wire are needed to make it work?! > > Just a word of caution - I read somewhere that using iSCSI for Time > Machine means that you cannot use TM during an OS reinstall to > restore your files from the Time Machine archive as iSCSI isn't > available at that point. It seems you have to use a locally > attached disk or AFP for the support to be there during > reinstall. Not sure if thats still the case in Lion or not, I > would tend to suspect they still don't consider iSCSI a top tier > transport. I read you can create a boot dvd from one of the files in the upgrade. I will play with iscsi and report anything significant. Thanks to everyone who responded. From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 18:55:30 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 5D141106567C; Mon, 25 Jul 2011 18:55:29 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: Callum Gibson Date: Mon, 25 Jul 2011 14:55:08 -0400 User-Agent: KMail/1.6.2 References: <20110719112033.GA51765@omma.gibson.athome> <201107221758.01272.jkim@FreeBSD.org> <20110723082108.GB14172@omma.gibson.athome> In-Reply-To: <20110723082108.GB14172@omma.gibson.athome> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201107251455.11482.jkim@FreeBSD.org> Cc: freebsd-stable@FreeBSD.org, freebsd-amd64@FreeBSD.org Subject: Re: powernow regression in 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 18:55:30 -0000 On Saturday 23 July 2011 04:21 am, Callum Gibson wrote: > On 22Jul11 17:57, Jung-uk Kim wrote: > }Please try the attached patch. If it doesn't work, I need to see > }"acpidump -dt" output. > > Sorry, no difference. Given jhb's response is there any point > pursuing this further? Seems to be a "known issue" with legacy usb > that perhaps can't be fixed. > > In any case, acpidump output here: > http://members.optusnet.com.au/callumgibson/acpidump.out.gz I agree that the legacy USB issue must be fixed but your BIOS is more broken than I originally thought. powernow expects a static table (via acpi_perf) but _PSS is dynamically constructed at runtime. I don't think we can support this case easily, sorry. Jung-uk Kim From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 20:30:33 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F61B1065679 for ; Mon, 25 Jul 2011 20:30:33 +0000 (UTC) (envelope-from callumgibson@optusnet.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 829708FC15 for ; Mon, 25 Jul 2011 20:30:31 +0000 (UTC) Received: from omma.gibson.athome (c122-106-15-156.rivrw1.nsw.optusnet.com.au [122.106.15.156]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with SMTP id p6PKUTbB005180 for ; Tue, 26 Jul 2011 06:30:30 +1000 Received: (qmail 24116 invoked by uid 107); 26 Jul 2011 06:30:29 +1000 Date: 26 Jul 2011 06:30:29 +1000 Date: Tue, 26 Jul 2011 06:30:29 +1000 From: Callum Gibson To: Jung-uk Kim Message-ID: <20110725203029.GA23755@omma.gibson.athome> References: <20110719112033.GA51765@omma.gibson.athome> <201107221758.01272.jkim@FreeBSD.org> <20110723082108.GB14172@omma.gibson.athome> <201107251455.11482.jkim@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201107251455.11482.jkim@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@FreeBSD.org, freebsd-amd64@FreeBSD.org Subject: Re: powernow regression in 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 20:30:33 -0000 On 25Jul11 14:55, Jung-uk Kim wrote: }I agree that the legacy USB issue must be fixed but your BIOS is more }broken than I originally thought. powernow expects a static table }(via acpi_perf) but _PSS is dynamically constructed at runtime. I }don't think we can support this case easily, sorry. Thanks for looking at it. John's hack is working for now. It would be interesting to see a response from either of you to Jeremy's question/comments. C -- Callum Gibson @ home http://members.optusnet.com.au/callumgibson/ From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 21:04:33 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id BB13A106566C; Mon, 25 Jul 2011 21:04:33 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-stable@FreeBSD.org Date: Mon, 25 Jul 2011 17:04:18 -0400 User-Agent: KMail/1.6.2 References: <20110719112033.GA51765@omma.gibson.athome> <20110723081304.GA14172@omma.gibson.athome> <20110723084721.GA7931@icarus.home.lan> In-Reply-To: <20110723084721.GA7931@icarus.home.lan> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201107251704.20389.jkim@FreeBSD.org> Cc: Attilio Rao , Callum Gibson , freebsd-amd64@freebsd.org, Jeremy Chadwick , John Baldwin Subject: Re: powernow regression in 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 21:04:33 -0000 On Saturday 23 July 2011 04:47 am, Jeremy Chadwick wrote: > On Sat, Jul 23, 2011 at 06:13:04PM +1000, Callum Gibson wrote: > > On 22Jul11 08:16, John Baldwin wrote: > > }The problem is that we calibrate the TSC using this algorithm: > > } > > } - grab the TSC > > } - spin on the ISA timer waiting for it to run for a second > > } - grab the TSC > > } > > }The issue is that the SMI# fires at the same time we want to be > > execuiting }step 3, and step 3 is deferred while the SMI# handler > > runs. As a result, the }TSC delta ends up being "1 second + time > > of an SMI# to poll USB". We have a }hack fix for this at work > > that originally came from Attilio Rao. This is a }patch for it > > relative to 8. It disables interrupt generation for the ISA > > }timer while we calibrate the TSC (which disables the SMI# > > temporarily): > > > > Thanks, John. The hack works as intended, but I guess it's not a > > "real" solution which is why you haven't committed it? > > This sort of thing is going to have to get dealt with officially > sooner or later, hack-fix or elegant solution either way[1]. > > More and more systems are using native USB keyboards, and most > system BIOSes I've seen (from multiple vendors, specifically > Supermicro, Lenovo, Asus, Dell, HP, and Gigabyte[2]) are defaulting > to having these options enabled (and rightfully so). > > If the hack-fix has repercussions, it would be helpful to know what > those are or how those might manifest themselves. Otherwise, has > anyone taken a look at how Linux addresses this problem? To my > knowledge GRUB and similar bootstraps do not have a native USB > stack, so I'm left wondering how they deal with this. > > [1]: I'm in no way saying "Hey! Fix this! {contributes nothing}", > I'm simply pointing out that we're at a point where we really don't > have much of a choice. The more I read technical explanations from > John the more hate x86 architecture. ;-) > > [2]: On a couple Gigabyte boards I have the default values for said > option is enabled (for both keyboard and mouse). It is really annoying problem and it has to be fixed *properly*. Previously, we dealt with similar SMI# problem per platform, e.g., http://svnweb.freebsd.org/base?view=revision&revision=174557 This quirk table matches 6 platforms now: if (strncmp(sysenv, "MacBook1,1", 10) == 0 || strncmp(sysenv, "MacBook3,1", 10) == 0 || strncmp(sysenv, "MacBookPro1,1", 13) == 0 || strncmp(sysenv, "MacBookPro1,2", 13) == 0 || strncmp(sysenv, "MacBookPro3,1", 13) == 0 || strncmp(sysenv, "Macmini1,1", 10) == 0) { if (bootverbose) printf("Disabling LEGACY_USB_EN bit on " "Intel ICH.\n"); outl(ICH_SMI_EN, inl(ICH_SMI_EN) & ~0x8); } ... We definitely need to generalize it as soon as possible. Jung-uk Kim From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 21:08:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB4CC106564A for ; Mon, 25 Jul 2011 21:08:54 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id AE1CF8FC0C for ; Mon, 25 Jul 2011 21:08:54 +0000 (UTC) Received: by ywf7 with SMTP id 7so3042828ywf.13 for ; Mon, 25 Jul 2011 14:08:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Cw0Darb8r75NaXK2xOKp0xrARK9xuKhoGGFSEcWVE/A=; b=i/GzjTacNxUW9TW/y3BkA6OXrEJVAG4YovTTxlGEQpL9+b6KqUVaW3Bmdn5SqBeZBn yOO4kTop1Bj1933d8yUpdCG9jVPZf4XlyXK8gDF+Y8+NOgRBmsamWlhU5rGTNZU8q8Lk IsBF1jIxXiTCoqjm+SjexnrVtEqdeHEb0ewJc= MIME-Version: 1.0 Received: by 10.91.111.15 with SMTP id o15mr4926200agm.148.1311626527917; Mon, 25 Jul 2011 13:42:07 -0700 (PDT) Received: by 10.90.209.12 with HTTP; Mon, 25 Jul 2011 13:42:07 -0700 (PDT) In-Reply-To: <20110725093432.GI1202@home.opsec.eu> References: <20110725093432.GI1202@home.opsec.eu> Date: Mon, 25 Jul 2011 13:42:07 -0700 Message-ID: From: Freddie Cash To: Kurt Jaeger Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 21:08:55 -0000 On Mon, Jul 25, 2011 at 2:34 AM, Kurt Jaeger wrote: > What kind of SATA 6g 4-port non-RAID controller is currently suggested > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > SuperMicro AOC-USAS2-L8i PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 SAS/SATA ports. Supports two firmwares: IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no RAID5+) IT mode enables straight HBA mode Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been MFC'd into 8.2-STABLE. Sure, it's 8-ports, but for under $300 CDN, you really can't do much better. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 21:23:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51DEF106564A; Mon, 25 Jul 2011 21:23:31 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from lennier.cc.vt.edu (lennier.cc.vt.edu [198.82.162.213]) by mx1.freebsd.org (Postfix) with ESMTP id 0C3C38FC08; Mon, 25 Jul 2011 21:23:30 +0000 (UTC) Received: from zidane.cc.vt.edu (zidane.cc.vt.edu [198.82.163.227]) by lennier.cc.vt.edu (8.13.8/8.13.8) with ESMTP id p6PJPOhN009207; Mon, 25 Jul 2011 15:25:24 -0400 Received: from auth3.smtp.vt.edu (EHLO auth3.smtp.vt.edu) ([198.82.161.152]) by zidane.cc.vt.edu (MOS 4.2.2-FCS FastPath queued) with ESMTP id QAX95414; Mon, 25 Jul 2011 15:25:23 -0400 (EDT) Received: from pmather.tower.lib.vt.edu (pmather.tower.lib.vt.edu [128.173.51.28]) (authenticated bits=0) by auth3.smtp.vt.edu (8.13.8/8.13.8) with ESMTP id p6PJPNbc025425 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 25 Jul 2011 15:25:23 -0400 Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Paul Mather In-Reply-To: <20110725185113.2C956B827@mail.bitblocks.com> Date: Mon, 25 Jul 2011 15:25:23 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <977DC729-3DEA-44DB-9A30-9C18972A7F00@gromit.dlib.vt.edu> References: <20110721215617.7C7F0B827@mail.bitblocks.com> <20110725181506.GC1339@in-addr.com> <20110725185113.2C956B827@mail.bitblocks.com> To: Bakul Shah X-Mailer: Apple Mail (2.1084) X-Mirapoint-Received-SPF: 198.82.161.152 auth3.smtp.vt.edu paul@gromit.dlib.vt.edu 5 none X-Junkmail-Status: score=10/50, host=zidane.cc.vt.edu X-Junkmail-Signature-Raw: score=unknown, refid=str=0001.0A02020A.4E2DC324.0019,ss=1,fgs=0, ip=0.0.0.0, so=2010-07-22 22:03:31, dmn=2009-09-10 00:05:08, mode=single engine X-Junkmail-IWF: false Cc: freebsd-stable@freebsd.org Subject: Re: OS X Lion time machine => (afpd|iSCSI) => ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 21:23:31 -0000 On Jul 25, 2011, at 2:51 PM, Bakul Shah wrote: > On Mon, 25 Jul 2011 14:15:06 EDT Gary Palmer = wrote: >> On Thu, Jul 21, 2011 at 02:56:17PM -0700, Bakul Shah wrote: >>> I am in no hurry to upgrade my MBP to OS X Lion but given Lion >>> time machine and netatalk issues, I got wondering if iSCSI on >>> FreeBSD is stable enough for time machine use. How much duct >>> tape and baling wire are needed to make it work?! >>=20 >> Just a word of caution - I read somewhere that using iSCSI for Time >> Machine means that you cannot use TM during an OS reinstall to >> restore your files from the Time Machine archive as iSCSI isn't >> available at that point. It seems you have to use a locally=20 >> attached disk or AFP for the support to be there during=20 >> reinstall. Not sure if thats still the case in Lion or not, I >> would tend to suspect they still don't consider iSCSI a top tier >> transport. >=20 > I read you can create a boot dvd from one of the files in the > upgrade. The problem is that the Boot DVD or Recovery Partition does not have an = iSCSI initiator. In fact, the Mac does not ship with an iSCSI = initiator, and the common solution to obtaining one is to install the = free globalSAN iSCSI Initiator for OS X. Without an installed iSCSI initiator, you won't be able to mount the = LUNs upon which is located the Time Machine volume from which you are = trying to restore files. Cheers, Paul. From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 21:54:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04FAD106564A for ; Mon, 25 Jul 2011 21:54:03 +0000 (UTC) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: from emmi.physik-pool.tu-berlin.de (emmi.physik-pool.tu-berlin.de [130.149.58.146]) by mx1.freebsd.org (Postfix) with ESMTP id 7AE068FC12 for ; Mon, 25 Jul 2011 21:54:01 +0000 (UTC) Received: from emmi.physik-pool.tu-berlin.de (localhost.physik-pool.tu-berlin.de [127.0.0.1]) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4) with ESMTP id p6PLSATG049577; Mon, 25 Jul 2011 23:28:10 +0200 (CEST) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: (from elon@localhost) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4/Submit) id p6PLS8cX049572; Mon, 25 Jul 2011 23:28:08 +0200 (CEST) (envelope-from elon) Date: Mon, 25 Jul 2011 23:28:08 +0200 From: Leon =?iso-8859-15?Q?Me=DFner?= To: Freddie Cash Message-ID: <20110725212808.GF48651@emmi.physik-pool.tu-berlin.de> Mail-Followup-To: Freddie Cash , Kurt Jaeger , freebsd-stable@freebsd.org References: <20110725093432.GI1202@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Kurt Jaeger , freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 21:54:03 -0000 On Mon, Jul 25, 2011 at 01:42:07PM -0700, Freddie Cash wrote: > On Mon, Jul 25, 2011 at 2:34 AM, Kurt Jaeger wrote: > > > What kind of SATA 6g 4-port non-RAID controller is currently suggested > > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > > > > SuperMicro AOC-USAS2-L8i > > PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 > SAS/SATA ports. Supports two firmwares: > IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no > RAID5+) > IT mode enables straight HBA mode > > Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been > MFC'd into 8.2-STABLE. Yes, it's in -STABLE. Does anyone know if mps is going to support IR-Firmware in the near future ? From owner-freebsd-stable@FreeBSD.ORG Mon Jul 25 22:03:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B2FA106564A; Mon, 25 Jul 2011 22:03:06 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 04B4C8FC08; Mon, 25 Jul 2011 22:03:06 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 5020A1899EC; Tue, 26 Jul 2011 00:03:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id iWm9O7hAKI0M; Tue, 26 Jul 2011 00:03:01 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 8D56D1899DB; Tue, 26 Jul 2011 00:03:00 +0200 (CEST) Message-ID: <4E2DE815.5010803@FreeBSD.org> Date: Tue, 26 Jul 2011 00:03:01 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Eugene Mitrofanov References: <201107221148.13100.eugene@imedia.ru> In-Reply-To: <201107221148.13100.eugene@imedia.ru> X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, freebsd-questions@freebsd.org Subject: Re: ZFS v28 and aclmode X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 22:03:06 -0000 Dňa 22. 7. 2011 9:48, Eugene Mitrofanov wrote / napísal(a): > Hi all, > > I just updated to 8.2S and zfs v28 and notice strange thing: in the manual i > can read about aclmode but i can not use it in the real life: > > # zfs set aclmode=passthrough data/public > cannot set property for 'data/public': invalid property 'aclmode' > > Obsolete manual? > > Good luck I imported reintroduction of aclmode from Illumos to 9-CURRENT in revsion 224174 MFC to 8-STABLE will be around Aug 1, 2011. More information: https://www.illumos.org/issues/742 http://svnweb.freebsd.org/base?view=revision&revision=224174 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 01:50:47 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70DA5106564A; Tue, 26 Jul 2011 01:50:47 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id CD1968FC13; Tue, 26 Jul 2011 01:50:46 +0000 (UTC) Received: by qwc9 with SMTP id 9so3208968qwc.13 for ; Mon, 25 Jul 2011 18:50:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=G52OPD2/P3nEav1djuNfo2RG4pk3oc1vSJ6s9L/JZcQ=; b=b69mIhU0qhRumhCFvOHSnNLSxKkG7euhq6SZWucrgHxFAQarYEwsXoRthOyE6CcCAb toZNHOwsOSwZOGP/rH2AIQuhtwaUMGozGFyoTxJSGLWC6htqJgpa6PnixJqgRxDKYdh2 o1z7OOZpqC/1N59HgWGADr/PxtYAEietVDJx0= MIME-Version: 1.0 Received: by 10.229.215.202 with SMTP id hf10mr3958419qcb.212.1311643243166; Mon, 25 Jul 2011 18:20:43 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Mon, 25 Jul 2011 18:20:43 -0700 (PDT) Date: Mon, 25 Jul 2011 18:20:43 -0700 Message-ID: From: maestro something To: freebsd-stable@freebsd.org Content-Type: multipart/mixed; boundary=00163630f0b194bd1104a8eebdcf X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: rpaulo@freebsd.org Subject: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 01:50:47 -0000 --00163630f0b194bd1104a8eebdcf Content-Type: text/plain; charset=ISO-8859-1 Hi, when using the ustack action on the latest FreeBSD8.2 i386 the kernel panics. Here is the information I could gather: let me know if I can provide more information. (i.e., i have the full crash information 80+mbs handy) Here is the backtrace: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x108 fault code = supervisor write, page not present instruction pointer = 0x20:0xc11012d7 stack pointer = 0x28:0xcd3ed9f4 frame pointer = 0x28:0xcd3eda0c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 1132 (nc) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xc09036a7 at kdb_backtrace+0x47 #1 0xc08d1a07 at panic+0x117 #2 0xc0c158c3 at trap_fatal+0x323 #3 0xc0c15bc0 at trap_pfault+0x2f0 #4 0xc0c1612a at trap+0x48a #5 0xc0bfc97c at calltrap+0x6 #6 0xc10e992b at dtrace_panic+0x1b #7 0xc10e995d at dtrace_assfail+0x2d #8 0xc10fb28c at dtrace_probe+0x135c #9 0xc1237f72 at systrace_probe+0x62 #10 0xc090f63f at syscallenter+0x47f #11 0xc0c15c14 at syscall+0x34 #12 0xc0bfca11 at Xint0x80_syscall+0x21 Uptime: 3m0s Physical memory: 239 MB Dumping 79 MB: 64 48 32 16 Steps To reproduce: the dtrace program (i.e., test.d) was: #!/usr/sbin/dtrace -s syscall::accept:return / execname == "nc" / { printf("%s accept:return\n", execname); ustack(); } % dtrace -s test.d then running % nc -vl 8080 on one shell and % nc localhost 8080 on another would make the kernel panic thanks --m --00163630f0b194bd1104a8eebdcf Content-Type: application/octet-stream; name="core.txt.0" Content-Disposition: attachment; filename="core.txt.0" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gqk6bxqk0 ZmI4MmkzODYgZHVtcGVkIGNvcmUgLSBzZWUgL3Zhci9jcmFzaC92bWNvcmUuMAoKVHVlIEp1bCAy NiAwMTowMzo0OSBQRFQgMjAxMQoKRnJlZUJTRCBmYjgyaTM4NiA4LjItUkVMRUFTRSBGcmVlQlNE IDguMi1SRUxFQVNFICMwOiBGcmkgSnVuIDI0IDA2OjI4OjI5IFBEVCAyMDExICAgICByb290QGZi ODJpMzg2Oi91c3Ivb2JqL3Vzci9zcmMvc3lzL01ZS0VSTkVMICBpMzg2CgpwYW5pYzogcGFnZSBm YXVsdAoKR05VIGdkYiA2LjEuMSBbRnJlZUJTRF0KQ29weXJpZ2h0IDIwMDQgRnJlZSBTb2Z0d2Fy ZSBGb3VuZGF0aW9uLCBJbmMuCkdEQiBpcyBmcmVlIHNvZnR3YXJlLCBjb3ZlcmVkIGJ5IHRoZSBH TlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSwgYW5kIHlvdSBhcmUKd2VsY29tZSB0byBjaGFuZ2Ug aXQgYW5kL29yIGRpc3RyaWJ1dGUgY29waWVzIG9mIGl0IHVuZGVyIGNlcnRhaW4gY29uZGl0aW9u cy4KVHlwZSAic2hvdyBjb3B5aW5nIiB0byBzZWUgdGhlIGNvbmRpdGlvbnMuClRoZXJlIGlzIGFi c29sdXRlbHkgbm8gd2FycmFudHkgZm9yIEdEQi4gIFR5cGUgInNob3cgd2FycmFudHkiIGZvciBk ZXRhaWxzLgpUaGlzIEdEQiB3YXMgY29uZmlndXJlZCBhcyAiaTM4Ni1tYXJjZWwtZnJlZWJzZCIu Li4KClVucmVhZCBwb3J0aW9uIG9mIHRoZSBrZXJuZWwgbWVzc2FnZSBidWZmZXI6Cmtlcm5lbCB0 cmFwIDEyIHdpdGggaW50ZXJydXB0cyBkaXNhYmxlZAoKCkZhdGFsIHRyYXAgMTI6IHBhZ2UgZmF1 bHQgd2hpbGUgaW4ga2VybmVsIG1vZGUKY3B1aWQgPSAwOyBhcGljIGlkID0gMDAKZmF1bHQgdmly dHVhbCBhZGRyZXNzCT0gMHgxMDgKZmF1bHQgY29kZQkJPSBzdXBlcnZpc29yIHdyaXRlLCBwYWdl IG5vdCBwcmVzZW50Cmluc3RydWN0aW9uIHBvaW50ZXIJPSAweDIwOjB4YzExMDEyZDcKc3RhY2sg cG9pbnRlcgkgICAgICAgID0gMHgyODoweGNkM2VkOWY0CmZyYW1lIHBvaW50ZXIJICAgICAgICA9 IDB4Mjg6MHhjZDNlZGEwYwpjb2RlIHNlZ21lbnQJCT0gYmFzZSAweDAsIGxpbWl0IDB4ZmZmZmYs IHR5cGUgMHgxYgoJCQk9IERQTCAwLCBwcmVzIDEsIGRlZjMyIDEsIGdyYW4gMQpwcm9jZXNzb3Ig ZWZsYWdzCT0gcmVzdW1lLCBJT1BMID0gMApjdXJyZW50IHByb2Nlc3MJCT0gMTEzMiAobmMpCnRy YXAgbnVtYmVyCQk9IDEyCnBhbmljOiBwYWdlIGZhdWx0CmNwdWlkID0gMApLREI6IHN0YWNrIGJh Y2t0cmFjZToKIzAgMHhjMDkwMzZhNyBhdCBrZGJfYmFja3RyYWNlKzB4NDcKIzEgMHhjMDhkMWEw NyBhdCBwYW5pYysweDExNwojMiAweGMwYzE1OGMzIGF0IHRyYXBfZmF0YWwrMHgzMjMKIzMgMHhj MGMxNWJjMCBhdCB0cmFwX3BmYXVsdCsweDJmMAojNCAweGMwYzE2MTJhIGF0IHRyYXArMHg0OGEK IzUgMHhjMGJmYzk3YyBhdCBjYWxsdHJhcCsweDYKIzYgMHhjMTBlOTkyYiBhdCBkdHJhY2VfcGFu aWMrMHgxYgojNyAweGMxMGU5OTVkIGF0IGR0cmFjZV9hc3NmYWlsKzB4MmQKIzggMHhjMTBmYjI4 YyBhdCBkdHJhY2VfcHJvYmUrMHgxMzVjCiM5IDB4YzEyMzdmNzIgYXQgc3lzdHJhY2VfcHJvYmUr MHg2MgojMTAgMHhjMDkwZjYzZiBhdCBzeXNjYWxsZW50ZXIrMHg0N2YKIzExIDB4YzBjMTVjMTQg YXQgc3lzY2FsbCsweDM0CiMxMiAweGMwYmZjYTExIGF0IFhpbnQweDgwX3N5c2NhbGwrMHgyMQpV cHRpbWU6IDNtMHMKUGh5c2ljYWwgbWVtb3J5OiAyMzkgTUIKRHVtcGluZyA3OSBNQjogNjQgNDgg MzIgMTYKClJlYWRpbmcgc3ltYm9scyBmcm9tIC9ib290L2tlcm5lbC9kdHJhY2VhbGwua28uLi5S ZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvZHRyYWNlYWxsLmtvLnN5bWJvbHMuLi5k b25lLgpkb25lLgpMb2FkZWQgc3ltYm9scyBmb3IgL2Jvb3Qva2VybmVsL2R0cmFjZWFsbC5rbwpS ZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvY3ljbGljLmtvLi4uUmVhZGluZyBzeW1i b2xzIGZyb20gL2Jvb3Qva2VybmVsL2N5Y2xpYy5rby5zeW1ib2xzLi4uZG9uZS4KZG9uZS4KTG9h ZGVkIHN5bWJvbHMgZm9yIC9ib290L2tlcm5lbC9jeWNsaWMua28KUmVhZGluZyBzeW1ib2xzIGZy b20gL2Jvb3Qva2VybmVsL29wZW5zb2xhcmlzLmtvLi4uUmVhZGluZyBzeW1ib2xzIGZyb20gL2Jv b3Qva2VybmVsL29wZW5zb2xhcmlzLmtvLnN5bWJvbHMuLi5kb25lLgpkb25lLgpMb2FkZWQgc3lt Ym9scyBmb3IgL2Jvb3Qva2VybmVsL29wZW5zb2xhcmlzLmtvClJlYWRpbmcgc3ltYm9scyBmcm9t IC9ib290L2tlcm5lbC9kdHJhY2Uua28uLi5SZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJu ZWwvZHRyYWNlLmtvLnN5bWJvbHMuLi5kb25lLgpkb25lLgpMb2FkZWQgc3ltYm9scyBmb3IgL2Jv b3Qva2VybmVsL2R0cmFjZS5rbwpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvZHRt YWxsb2Mua28uLi5SZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvZHRtYWxsb2Mua28u c3ltYm9scy4uLmRvbmUuCmRvbmUuCkxvYWRlZCBzeW1ib2xzIGZvciAvYm9vdC9rZXJuZWwvZHRt YWxsb2Mua28KUmVhZGluZyBzeW1ib2xzIGZyb20gL2Jvb3Qva2VybmVsL2R0bmZzY2xpZW50Lmtv Li4uUmVhZGluZyBzeW1ib2xzIGZyb20gL2Jvb3Qva2VybmVsL2R0bmZzY2xpZW50LmtvLnN5bWJv bHMuLi5kb25lLgpkb25lLgpMb2FkZWQgc3ltYm9scyBmb3IgL2Jvb3Qva2VybmVsL2R0bmZzY2xp ZW50LmtvClJlYWRpbmcgc3ltYm9scyBmcm9tIC9ib290L2tlcm5lbC9mYnQua28uLi5SZWFkaW5n IHN5bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvZmJ0LmtvLnN5bWJvbHMuLi5kb25lLgpkb25lLgpM b2FkZWQgc3ltYm9scyBmb3IgL2Jvb3Qva2VybmVsL2ZidC5rbwpSZWFkaW5nIHN5bWJvbHMgZnJv bSAvYm9vdC9rZXJuZWwvbG9ja3N0YXQua28uLi5SZWFkaW5nIHN5bWJvbHMgZnJvbSAvYm9vdC9r ZXJuZWwvbG9ja3N0YXQua28uc3ltYm9scy4uLmRvbmUuCmRvbmUuCkxvYWRlZCBzeW1ib2xzIGZv ciAvYm9vdC9rZXJuZWwvbG9ja3N0YXQua28KUmVhZGluZyBzeW1ib2xzIGZyb20gL2Jvb3Qva2Vy bmVsL3NkdC5rby4uLlJlYWRpbmcgc3ltYm9scyBmcm9tIC9ib290L2tlcm5lbC9zZHQua28uc3lt Ym9scy4uLmRvbmUuCmRvbmUuCkxvYWRlZCBzeW1ib2xzIGZvciAvYm9vdC9rZXJuZWwvc2R0Lmtv ClJlYWRpbmcgc3ltYm9scyBmcm9tIC9ib290L2tlcm5lbC9zeXN0cmFjZS5rby4uLlJlYWRpbmcg c3ltYm9scyBmcm9tIC9ib290L2tlcm5lbC9zeXN0cmFjZS5rby5zeW1ib2xzLi4uZG9uZS4KZG9u ZS4KTG9hZGVkIHN5bWJvbHMgZm9yIC9ib290L2tlcm5lbC9zeXN0cmFjZS5rbwpSZWFkaW5nIHN5 bWJvbHMgZnJvbSAvYm9vdC9rZXJuZWwvcHJvZmlsZS5rby4uLlJlYWRpbmcgc3ltYm9scyBmcm9t IC9ib290L2tlcm5lbC9wcm9maWxlLmtvLnN5bWJvbHMuLi5kb25lLgpkb25lLgpMb2FkZWQgc3lt Ym9scyBmb3IgL2Jvb3Qva2VybmVsL3Byb2ZpbGUua28KIzAgIGRvYWR1bXAgKCkgYXQgcGNwdS5o OjIzMQoyMzEJcGNwdS5oOiBObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5LgoJaW4gcGNwdS5oCihr Z2RiKSAjMCAgZG9hZHVtcCAoKSBhdCBwY3B1Lmg6MjMxCiMxICAweGMwOGQxN2EzIGluIGJvb3Qg KGhvd3RvPTI2MCkgYXQgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9zaHV0ZG93bi5jOjQxOQojMiAg MHhjMDhkMWE0MCBpbiBwYW5pYyAoZm10PVZhcmlhYmxlICJmbXQiIGlzIG5vdCBhdmFpbGFibGUu CikgYXQgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9zaHV0ZG93bi5jOjU5MgojMyAgMHhjMGMxNThj MyBpbiB0cmFwX2ZhdGFsIChmcmFtZT0weGNkM2VkOWI0LCBldmE9MjY0KQogICAgYXQgL3Vzci9z cmMvc3lzL2kzODYvaTM4Ni90cmFwLmM6OTQ2CiM0ICAweGMwYzE1YmMwIGluIHRyYXBfcGZhdWx0 IChmcmFtZT0weGNkM2VkOWI0LCB1c2VybW9kZT0wLCBldmE9MjY0KQogICAgYXQgL3Vzci9zcmMv c3lzL2kzODYvaTM4Ni90cmFwLmM6ODU5CiM1ICAweGMwYzE2MTJhIGluIHRyYXAgKGZyYW1lPTB4 Y2QzZWQ5YjQpIGF0IC91c3Ivc3JjL3N5cy9pMzg2L2kzODYvdHJhcC5jOjUzMgojNiAgMHhjMGJm Yzk3YyBpbiBjYWxsdHJhcCAoKSBhdCAvdXNyL3NyYy9zeXMvaTM4Ni9pMzg2L2V4Y2VwdGlvbi5z OjE2NgojNyAgMHhjMTEwMTJkNyBpbiBkdHJhY2VfcGFuaWNfdHJpZ2dlciAoKSBmcm9tIC9ib290 L2tlcm5lbC9kdHJhY2Uua28KUHJldmlvdXMgZnJhbWUgaW5uZXIgdG8gdGhpcyBmcmFtZSAoY29y cnVwdCBzdGFjaz8pCihrZ2RiKSAKCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpwcyAtYXhsCgogIFVJRCAgIFBJ RCAgUFBJRCBDUFUgUFJJIE5JICAgVlNaICAgUlNTIE1XQ0hBTiBTVEFUICBUVCAgICAgICBUSU1F IENPTU1BTkQKICAgIDAgICAgIDAgICAgIDAgICAwICAgOCAgMCAgICAgMCAgICAgMCAtICAgICAg RExzICAgPz8gIDE1MDM3OjQ4LjAwIFtrZXJuZWxdCiAgICAwICAgICAxICAgICAwICAgMCAgNDQg IDAgIDI5MTIgICAgIDAgd2FpdCAgIERMcyAgID8/ICAxMzA3MDM2OjEyLjAwIFtpbml0XQogICAg MCAgICAgMiAgICAgMCAgIDAgIC04ICAwICAgICAwICAgICAwIC0gICAgICBETCAgICA/PyAgMTg5 NTc1OjM0LjAwIFtnX2V2ZW50XQogICAgMCAgICAgMyAgICAgMCAgIDAgIC04ICAwICAgICAwICAg ICAwIC0gICAgICBETCAgICA/PyAgMTQwNDI5MjowNy4wMCBbZ191cF0KICAgIDAgICAgIDQgICAg IDAgICAwICAtOCAgMCAgICAgMCAgICAgMCAtICAgICAgREwgICAgPz8gIDMxNDAxNjc6MzMuMDAg W2dfZG93bl0KICAgIDAgICAgIDUgICAgIDAgICAwIC0xNiAgMCAgICAgMCAgICAgMCB3YWl0aW4g REwgICAgPz8gIDM2OToyNi4wMCBbc2N0cF9pdGVyYQogICAgMCAgICAgNiAgICAgMCAgIDAgIDc2 ICAwICAgICAwICAgICAwIGNjYl9zYyBETCAgICA/PyAgICAwOjAwLjAwIFt4cHRfdGhyZF0KICAg IDAgICAgIDcgICAgIDAgICAwICA3NiAgMCAgICAgMCAgICAgMCBwc2xlZXAgREwgICAgPz8gIDI5 MzU6NDcuMDAgW3BhZ2VkYWVtb24KICAgIDAgICAgIDggICAgIDAgICAwIC0xNiAgMCAgICAgMCAg ICAgMCBwc2xlZXAgREwgICAgPz8gIDE0MDoxNS4wMCBbdm1kYWVtb25dCiAgICAwICAgICA5ICAg ICAwICAgMCAgNzYgIDAgICAgIDAgICAgIDAgcGd6ZXJvIERMICAgID8/ICAgNjA6MTguMDAgW3Bh Z2V6ZXJvXQogICAgMCAgICAxMCAgICAgMCAgIDAgLTE2ICAwICAgICAwICAgICAwIGF1ZGl0XyBE TCAgICA/PyAgICAwOjAwLjAwIFthdWRpdF0KICAgIDAgICAgMTEgICAgIDAgICAwIDE3MSAgMCAg ICAgMCAgICAgMCAtICAgICAgUkwgICAgPz8gIC0zMzA5MDkzMTotNTEuNTUgW2lkbGVdCiAgICAw ICAgIDEyICAgICAwICAgMCAtNDggIDAgICAgIDAgICAgIDAgLSAgICAgIFdMICAgID8/ICAzOTIy MjI2OjI2LjAwIFtpbnRyXQogICAgMCAgICAxMyAgICAgMCAgIDAgLTE2ICAwICAgICAwICAgICAw IC0gICAgICBETCAgICA/PyAgMTExNTAyOjU1LjAwIFt5YXJyb3ddCiAgICAwICAgIDE0ICAgICAw ICAgMCAtNjQgIDAgICAgIDAgICAgIDAgLSAgICAgIERMICAgID8/ICAxNDkyMjUzOjMwLjAwIFt1 c2JdCiAgICAwICAgIDE1ICAgICAwICAgMCAtMTYgIDAgICAgIDAgICAgIDAgcHNsZWVwIERMICAg ID8/ICAxODU3OjQyLjAwIFtidWZkYWVtb25dCiAgICAwICAgIDE2ICAgICAwICAgMCAtMTYgIDAg ICAgIDAgICAgIDAgdmxydXd0IERMICAgID8/ICAxMTI2OjAzLjAwIFt2bmxydV0KICAgIDAgICAg MTcgICAgIDAgICAwIC0xNiAgMCAgICAgMCAgICAgMCBzeW5jZXIgREwgICAgPz8gIDI4MTA1OjA3 LjAwIFtzeW5jZXJdCiAgICAwICAgIDE4ICAgICAwICAgMCAtMTYgIDAgICAgIDAgICAgIDAgc2Rm bHVzIERMICAgID8/ICA4ODA0OjU0LjAwIFtzb2Z0ZGVwZmx1CiAgICAwICAgIDE5ICAgICAwICAg MCAtMTYgIDAgICAgIDAgICAgIDAgZmxvd2NsIERMICAgID8/ICAzNDU6MzkuMDAgW2Zsb3djbGVh bmUKICAgIDAgICAxNDIgICAgIDEgICAwICA3NiAgMCAgMTU0NCAgICAgMCBwYXVzZSAgRHMgICAg Pz8gIDE0OTkzOjA4LjAwIFthZGprZXJudHpdCiAgICAwICAgNTA2ICAgICAxICAgMCAgNDQgIDAg IDM0NTYgICAgIDAgc2VsZWN0IERzICAgID8/ICA1NTYxOjEwLjAwIFttb3VzZWRdCiAgICAwICAg NTQ4ICAgICAxICAgMCAgNDQgIDAgIDE4ODggICAgIDAgc2VsZWN0IERzICAgID8/ICAxMDg2OToy Ni4wMCBbZGV2ZF0KICAgIDAgICA2NzIgICAgIDEgICAwICA0NCAgMCAgMzM1MiAgICAgMCBzZWxl Y3QgRHMgICAgPz8gIDIyMzU3OToxMS4wMCBbc3lzbG9nZF0KICAgIDAgICA5NjUgICAgIDEgICAw ICA0NCAgMCAgNjA5MiAgICAgMCBzZWxlY3QgRHMgICAgPz8gIDQ1NDAwOjQxLjAwIFtzZW5kbWFp bF0KICAgMjUgICA5NjkgICAgIDEgICAwICA3NiAgMCAgNjA5MiAgICAgMCBwYXVzZSAgRHMgICAg Pz8gIDI4MTczOjQyLjAwIFtzZW5kbWFpbF0KICAgIDAgICA5NzYgICAgIDEgICAwICA0NCAgMCAg MzM4MCAgICAgMCBuYW5zbHAgRHMgICAgPz8gIDUxNTIxOjQyLjAwIFtjcm9uXQogICAgMCAgMTA0 NCAgICAgMSAgIDAgIDc2ICAwICAzNjMyICAgICAwIHdhaXQgICBEICAgICA/PyAgICAwOjAwLjAw IFtzaF0KICAgIDAgIDEwNDUgICAgIDEgICAwICA3NiAgMCAgMzI5MiAgICAgMCBwaXBlcmQgRCAg ICAgPz8gICAgMDowMC4wMCBbbG9nZ2VyXQogICAgMCAgMTA0NyAgMTA0NCAgIDAgIDc2ICAwICAx NTQwICAgICAwIG5hbnNscCBEICAgICA/PyAgICAwOjAwLjAwIFtzbGVlcF0KICAgIDAgIDEwNDgg ICAgIDEgICAwICA1MiAgMCAgMzgxNiAgICAgMCB3YWl0ICAgRHMgICAgPz8gICAgMDowMC4wMCBb bG9naW5dCiAgICAwICAxMDQ5ICAgICAxICAgMCAgNzYgIDAgIDMzNTIgICAgIDAgdHR5aW4gIERz KyAgID8/ICA0ODAyNTo1MS4wMCBbZ2V0dHldCiAgICAwICAxMDUwICAgICAxICAgMCAgNzYgIDAg IDMzNTIgICAgIDAgdHR5aW4gIERzKyAgID8/ICA0ODYyMToxNC4wMCBbZ2V0dHldCiAgICAwICAx MDUxICAgICAxICAgMCAgNzYgIDAgIDMzNTIgICAgIDAgdHR5aW4gIERzKyAgID8/ICA0NzE4OToz OC4wMCBbZ2V0dHldCiAgICAwICAxMDUyICAgICAxICAgMCAgNzYgIDAgIDMzNTIgICAgIDAgdHR5 aW4gIERzKyAgID8/ICA1MjIxNzozMi4wMCBbZ2V0dHldCiAgICAwICAxMDUzICAgICAxICAgMCAg NzYgIDAgIDMzNTIgICAgIDAgdHR5aW4gIERzKyAgID8/ICA1MjM3MDoxOS4wMCBbZ2V0dHldCiAg ICAwICAxMDU0ICAgICAxICAgMCAgNzYgIDAgIDMzNTIgICAgIDAgdHR5aW4gIERzKyAgID8/ICA1 NDEwNzo0OC4wMCBbZ2V0dHldCiAgICAwICAxMDU1ICAgICAxICAgMCAgNzYgIDAgIDMzNTIgICAg IDAgdHR5aW4gIERzKyAgID8/ICA1MzY3NDoxOC4wMCBbZ2V0dHldCiAgICAwICAxMDU2ICAxMDQ4 ICAgMCAgNTcgIDAgIDU2NTYgICAgIDAgcGF1c2UgIEQgICAgID8/ICAxNzIyMTM6NTQuMDAgW2Nz aF0KICAgIDAgIDEwODEgICAgIDEgICAwICA3NiAgMCAgMzI5MiAgICAgMCBzZWxlY3QgRHMgICAg Pz8gICAgMDowMC4wMCBbZGhjbGllbnRdCiAgIDY1ICAxMDk3ICAgICAxICAgMCAgNDcgIDAgIDMy OTIgICAgIDAgc2VsZWN0IERzICAgID8/ICAgIDA6MDAuMDAgW2RoY2xpZW50XQogICAgMCAgMTEy NCAgICAgMSAgIDAgIDc2ICAwICA2NzEyICAgICAwIHNlbGVjdCBEcyAgICA/PyAgICAwOjAwLjAw IFtzc2hkXQogICAgMCAgMTEyNSAgMTEyNCAgIDAgIDUwICAwICA5NDM2ICAgICAwIHNlbGVjdCBE cyAgICA/PyAgICAwOjAwLjAwIFtzc2hkXQogICAgMCAgMTEyOCAgMTEyNSAgIDAgIDYwICAwICA1 NjU2ICAgICAwIHBhdXNlICBEcyAgICA/PyAgMTg2MjYxOjUyLjAwIFtjc2hdCiAgICAwICAxMTMy ICAxMDU2ICAgMCAgNTcgIDAgIDMzMjAgICAgIDAgLSAgICAgIFIrICAgID8/ICAgIDA6MDAuMDAg W25jXQogICAgMCAgMTEzNCAgMTEyOCAgIDAgIDUyICAwIDQwODQwICAgICAwIHVjb25kICBEKyAg ICA/PyAgICAwOjAwLjAwIFtkdHJhY2VdCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kdm1zdGF0IC1zCgogICAg MzYwNzggY3B1IGNvbnRleHQgc3dpdGNoZXMKICAgICAyMzUwIGRldmljZSBpbnRlcnJ1cHRzCiAg ICAgNzg0NiBzb2Z0d2FyZSBpbnRlcnJ1cHRzCiAgICA3NjU5OCB0cmFwcwogICAxNDIzNDUgc3lz dGVtIGNhbGxzCiAgICAgICAxOSBrZXJuZWwgdGhyZWFkcyBjcmVhdGVkCiAgICAgMTA5NCAgZm9y aygpIGNhbGxzCiAgICAgICAyMSB2Zm9yaygpIGNhbGxzCiAgICAgICAgMCByZm9yaygpIGNhbGxz CiAgICAgICAgMCBzd2FwIHBhZ2VyIHBhZ2VpbnMKICAgICAgICAwIHN3YXAgcGFnZXIgcGFnZXMg cGFnZWQgaW4KICAgICAgICAwIHN3YXAgcGFnZXIgcGFnZW91dHMKICAgICAgICAwIHN3YXAgcGFn ZXIgcGFnZXMgcGFnZWQgb3V0CiAgICAgIDM2MSB2bm9kZSBwYWdlciBwYWdlaW5zCiAgICAgMjYz NCB2bm9kZSBwYWdlciBwYWdlcyBwYWdlZCBpbgogICAgICAgIDAgdm5vZGUgcGFnZXIgcGFnZW91 dHMKICAgICAgICAwIHZub2RlIHBhZ2VyIHBhZ2VzIHBhZ2VkIG91dAogICAgICAgIDAgcGFnZSBk YWVtb24gd2FrZXVwcwogICAgICAgIDAgcGFnZXMgZXhhbWluZWQgYnkgdGhlIHBhZ2UgZGFlbW9u CiAgICAgICA4NiBwYWdlcyByZWFjdGl2YXRlZAogICAgMjk5OTggY29weS1vbi13cml0ZSBmYXVs dHMKICAgICAgIDExIGNvcHktb24td3JpdGUgb3B0aW1pemVkIGZhdWx0cwogICAgMjEzODEgemVy byBmaWxsIHBhZ2VzIHplcm9lZAogICAgICA4NTMgemVybyBmaWxsIHBhZ2VzIHByZXplcm9lZAog ICAgICAgIDQgaW50cmFuc2l0IGJsb2NraW5nIHBhZ2UgZmF1bHRzCiAgICA3MDAyMiB0b3RhbCBW TSBmYXVsdHMgdGFrZW4KICAgICAgICAwIHBhZ2VzIGFmZmVjdGVkIGJ5IGtlcm5lbCB0aHJlYWQg Y3JlYXRpb24KICAgMjA2NTU1IHBhZ2VzIGFmZmVjdGVkIGJ5ICBmb3JrKCkKICAgICAzNjQ0IHBh Z2VzIGFmZmVjdGVkIGJ5IHZmb3JrKCkKICAgICAgICAwIHBhZ2VzIGFmZmVjdGVkIGJ5IHJmb3Jr KCkKICAgICAgNjkwIHBhZ2VzIGNhY2hlZAogICAgNTY4MjQgcGFnZXMgZnJlZWQKICAgICAgICAw IHBhZ2VzIGZyZWVkIGJ5IGRhZW1vbgogICAgNDAwODcgcGFnZXMgZnJlZWQgYnkgZXhpdGluZyBw cm9jZXNzZXMKICAgICA0NTAwIHBhZ2VzIGFjdGl2ZQogICAgIDE3NjkgcGFnZXMgaW5hY3RpdmUK ICAgICAgNjA0IHBhZ2VzIGluIFZNIGNhY2hlCiAgICAxNDA0MSBwYWdlcyB3aXJlZCBkb3duCiAg ICAzOTA4NCBwYWdlcyBmcmVlCiAgICAgNDA5NiBieXRlcyBwZXIgcGFnZQogICAgMjM3MzIgdG90 YWwgbmFtZSBsb29rdXBzCiAgICAgICAgICBjYWNoZSBoaXRzICg4MyUgcG9zICsgMTAlIG5lZykg c3lzdGVtIDAlIHBlci1kaXJlY3RvcnkKICAgICAgICAgIGRlbGV0aW9ucyAwJSwgZmFsc2VoaXRz IDAlLCB0b29sb25nIDAlCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kdm1zdGF0IC1tCgogICAgICAgICBUeXBl IEluVXNlIE1lbVVzZSBIaWdoVXNlIFJlcXVlc3RzICBTaXplKHMpCiAgICAgICAgIGNkZXYgICAg MTAgICAgIDJLICAgICAgIC0gICAgICAgMTAgIDEyOAogICAgICBhY3BpZGV2ICAgIDUwICAgICAy SyAgICAgICAtICAgICAgIDUwICAzMgogICAgICAgIHNpZ2lvICAgICAxICAgICAxSyAgICAgICAt ICAgICAgICAxICAzMgogICAgIGZpbGVkZXNjICAgIDQ2ICAgIDEySyAgICAgICAtICAgICAxMTM1 ICAyNTYKICAgICAga2R0cmFjZSAgIDEyNyAgICAyNEsgICAgICAgLSAgICAgMTIxNiAgNjQsMjU2 CiAgICAgICAgIGtlbnYgICAgNjcgICAgIDdLICAgICAgIC0gICAgICAgNzEgIDE2LDMyLDY0LDEy OCw0MDk2CiAgICAgICBrcXVldWUgICAgIDAgICAgIDBLICAgICAgIC0gICAgICAgMTQgIDEyOCwx MDI0CiAgICBwcm9jLWFyZ3MgICAgMjcgICAgIDJLICAgICAgIC0gICAgICAzOTMgIDE2LDMyLDY0 LDEyOAogICAgICBpdGhyZWFkICAgIDYzICAgICA2SyAgICAgICAtICAgICAgIDYzICAxNiw2NCwx MjgKICAgICAgIEtUUkFDRSAgIDEwMCAgICAxM0sgICAgICAgLSAgICAgIDEwMCAgMTI4CiAgICAg cGNpX2xpbmsgICAgMTAgICAgIDFLICAgICAgIC0gICAgICAgMTAgIDE2LDEyOAogICAgICAgbGlu a2VyICAgMTkyICAgICA5SyAgICAgICAtICAgICAgMzExICAxNiwzMiw2NCwyNTYKICAgICAgICBs b2NrZiAgICAyMCAgICAgMksgICAgICAgLSAgICAgICA0NCAgMzIsNjQKICAgICBwcGJ1c2RldiAg ICAgMyAgICAgMUsgICAgICAgLSAgICAgICAgMyAgMTI4CiAgICAgICBpcDZuZHAgICAgIDQgICAg IDFLICAgICAgIC0gICAgICAgIDQgIDY0LDEyOAogICAgICAgICB0ZW1wICAgIDE3ICAgMTAxSyAg ICAgICAtICAgICA1NTY1ICAxNiwzMiw2NCwxMjgsMjU2LDUxMiwxMDI0LDIwNDgsNDA5NgogICAg ICAgZGV2YnVmICAgOTQ3ICAyNDAzSyAgICAgICAtICAgICAgOTg4ICAxNiwzMiw2NCwxMjgsMjU2 LDUxMiwxMDI0LDIwNDgsNDA5NgogICAgICAgbW9kdWxlICAgNDczICAgIDMwSyAgICAgICAtICAg ICAgNDczICA2NCwxMjgKICAgICBtdHhfcG9vbCAgICAgMiAgICAgOEsgICAgICAgLSAgICAgICAg MiAgNDA5NgogICAgICBlbnRyb3B5ICAxMDI0ICAgIDY0SyAgICAgICAtICAgICAxMDI0ICA2NAog ICAgICBzdWJwcm9jICAgMTA3ICAgMjAwSyAgICAgICAtICAgICAxMTk2ICAyNTYsNDA5NgogICAg ICAgICBwcm9jICAgICAyICAgICAySyAgICAgICAtICAgICAgICAyICAxMDI0CiAgICAgIHNlc3Np b24gICAgMjIgICAgIDJLICAgICAgIC0gICAgICAgMzEgIDY0CiAgICAgICAgIHBncnAgICAgMjUg ICAgIDJLICAgICAgIC0gICAgICAgNDEgIDY0CiAgICAgICAgIGNyZWQgICAgMzYgICAgIDRLICAg ICAgIC0gICAgIDM5MTQgIDY0LDEyOAogICAgICB1aWRpbmZvICAgICA0ICAgICAxSyAgICAgICAt ICAgICAgICA5ICA2NCwyNTYKICAgICAgIHBsaW1pdCAgICAxNSAgICAgNEsgICAgICAgLSAgICAg IDIxOCAgMjU2CiAgICBzeXNjdGx0bXAgICAgIDAgICAgIDBLICAgICAgIC0gICAgICAyOTEgIDE2 LDMyLDY0LDEyOCw0MDk2CiAgICBzeXNjdGxvaWQgIDE4MjkgICAgNTZLICAgICAgIC0gICAgIDE5 MjUgIDE2LDMyLDY0LDEyOAogICAgICAgc3lzY3RsICAgICAwICAgICAwSyAgICAgICAtICAgICAg NjMzICAxNiwzMiw2NAogICAgICAgICB1bXR4ICAgIDkwICAgICA2SyAgICAgICAtICAgICAgIDkw ICA2NAogICAgIHAxMDAzLjFiICAgICAxICAgICAxSyAgICAgICAtICAgICAgICAxICAxNgogICAg ICAgICBTV0FQICAgICAyICAgIDY5SyAgICAgICAtICAgICAgICAyICA2NAogICAgICAgICBVQVJU ICAgICAzICAgICAySyAgICAgICAtICAgICAgICAzICAxNiwyNTYsMTAyNAogICAgICAgYnVzLXNj ICAgIDM3ICAgIDQ1SyAgICAgICAtICAgICAxNTk3ICAxNiwzMiw2NCwxMjgsMjU2LDUxMiwxMDI0 LDIwNDgsNDA5NgogICAgICAgICAgYnVzICAxMDE4ICAgIDQ0SyAgICAgICAtICAgICAzNTY5ICAx NiwzMiw2NCwxMjgsMjU2LDUxMiwxMDI0CiAgICAgIGRldnN0YXQgICAgIDYgICAgMTNLICAgICAg IC0gICAgICAgIDYgIDE2LDQwOTYKIGV2ZW50aGFuZGxlciAgICA4MSAgICAgNEsgICAgICAgLSAg ICAgICA4MSAgMzIsNjQsMTI4CiAgICAgICAgIGtvYmogICAzMzIgICA2NjRLICAgICAgIC0gICAg ICAzOTAgIDIwNDgKICAgICAgUGVyLWNwdSAgICAgMSAgICAgMUsgICAgICAgLSAgICAgICAgMSAg MTYKICAgICAgIGFjcGljYSAgMTQ1NSAgICA4N0sgICAgICAgLSAgICA1NzY5MiAgMTYsMzIsNjQs MTI4LDI1NiwxMDI0CiAgICAgYWNwaXRhc2sgICAgIDEgICAgIDFLICAgICAgIC0gICAgICAgIDEg IDEwMjQKICAgICAgICAgcm1hbiAgICA3MCAgICAgNUsgICAgICAgLSAgICAgIDU1NiAgMTYsMzIs NjQKICAgICAgICAgc2J1ZiAgICAgMCAgICAgMEsgICAgICAgLSAgICAgIDYyNCAgMTYsMzIsNjQs MTI4LDI1Niw1MTIsMTAyNCwyMDQ4LDQwOTYKICAgICAgICBzdGFjayAgICAgMCAgICAgMEsgICAg ICAgLSAgICAgICAgMiAgMTI4CiAgICB0YXNrcXVldWUgICAgMTEgICAgIDFLICAgICAgIC0gICAg ICAgMTEgIDE2LDY0CiAgICAgICBVbml0bm8gICAgMTMgICAgIDFLICAgICAgIC0gICAgICAgMjYg IDE2LDY0CiAgICAgICAgICBpb3YgICAgIDAgICAgIDBLICAgICAgIC0gICAgICAxOTcgIDY0LDEy OCwyNTYKICAgICAgIHNlbGVjdCAgICAxOSAgICAgMksgICAgICAgLSAgICAgICAxOSAgNjQKICAg ICBpb2N0bG9wcyAgICAgMCAgICAgMEsgICAgICAgLSAgICAgMTEzOCAgMTYsMzIsNjQsMTI4LDI1 Niw1MTIsMTAyNAogICAgICAgICAgbXNnICAgICA0ICAgIDI1SyAgICAgICAtICAgICAgICA0ICAx MDI0LDQwOTYKICAgICAgICAgIHNlbSAgICAgNCAgICAgNksgICAgICAgLSAgICAgICAgNCAgMjU2 LDUxMiwxMDI0LDQwOTYKICAgICAgICAgIHNobSAgICAgMSAgICAxMksgICAgICAgLSAgICAgICAg MSAgCiAgICAgICAgICB0dHkgICAgMjEgICAgMTFLICAgICAgIC0gICAgICAgMjMgIDUxMiwyMDQ4 CiAgICAgICAgICBwdHMgICAgIDEgICAgIDFLICAgICAgIC0gICAgICAgIDEgIDEyOAogICAgIG1i dWZfdGFnICAgICAwICAgICAwSyAgICAgICAtICAgICAgIDEwICAzMgogICAgICAgICBrc2VtICAg ICAxICAgICA0SyAgICAgICAtICAgICAgICAxICA0MDk2CiAgICAgICAgc2htZmQgICAgIDEgICAg IDRLICAgICAgIC0gICAgICAgIDEgIDQwOTYKICAgICAgICAgIHBjYiAgICAxNyAgICA3OUsgICAg ICAgLSAgICAgICAyOCAgMTYsNjQsNTEyLDEwMjQsMjA0OCw0MDk2CiAgICAgICBzb25hbWUgICAg IDMgICAgIDFLICAgICAgIC0gICAgICAyODcgIDE2LDMyLDEyOAogICAgICAgYmlvYnVmICAgICA0 ICAgICA4SyAgICAgICAtICAgICAgICA2ICAyMDQ4CiAgICAgdmZzY2FjaGUgICAgIDEgICAxMjhL ICAgICAgIC0gICAgICAgIDEgIAogICAgIHZmc19oYXNoICAgICAxICAgIDY0SyAgICAgICAtICAg ICAgICAxICAKICAgICAgIHZub2RlcyAgICAgMiAgICAgMUsgICAgICAgLSAgICAgICAgMiAgMTI4 CiAgdm5vZGVtYXJrZXIgICAgIDAgICAgIDBLICAgICAgIC0gICAgICAgMjUgIDUxMgogICAgICAg IG1vdW50ICAgIDY5ICAgICAzSyAgICAgICAtICAgICAgMTEzICAxNiwzMiw2NCwxMjgsMjU2CiAg ICAgICAgICBCUEYgICAgIDggICAgIDlLICAgICAgIC0gICAgICAgMTMgIDY0LDEyOCwyNTYsNDA5 NgogIGV0aGVyX211bHRpICAgIDE1ICAgICAxSyAgICAgICAtICAgICAgIDIyICAxNiwzMiw2NAog ICAgICAgaWZhZGRyICAgIDM3ICAgICA4SyAgICAgICAtICAgICAgIDM4ICAxNiwzMiw2NCwxMjgs MjU2LDUxMiwyMDQ4CiAgICAgICAgaWZuZXQgICAgIDQgICAgIDRLICAgICAgIC0gICAgICAgIDQg IDY0LDEwMjQKICAgICAgICBjbG9uZSAgICAgNiAgICAyNEsgICAgICAgLSAgICAgICAgNiAgNDA5 NgogICAgICAgYXJwY29tICAgICAxICAgICAxSyAgICAgICAtICAgICAgICAxICAxNgogICAgICBs bHRhYmxlICAgIDExICAgICAzSyAgICAgICAtICAgICAgIDExICAxMjgsMjU2CiAgICAgICBVU0Jk ZXYgICAgMTMgICAgIDVLICAgICAgIC0gICAgICAgMTMgIDMyLDEyOCwyNTYsMTAyNCwyMDQ4CiAg ICAgICAgICBVU0IgICAgMTYgICAgMjFLICAgICAgIC0gICAgICAgMTcgIDE2LDMyLDY0LDEwMjQs MjA0OAogICAgICBhY3Bpc2VtICAgIDE1ICAgICAySyAgICAgICAtICAgICAgIDE1ICA2NCwxMjgK ICAgICByb3V0ZXRibCAgICA1NCAgMjA1OUsgICAgICAgLSAgICAgIDEyOCAgMTYsMzIsNjQsMTI4 LDI1NgogICAgICAgICBpZ21wICAgICAzICAgICAxSyAgICAgICAtICAgICAgICAzICAxMjgKICAg ICAgIGtiZG11eCAgICAgNiAgICAxMEsgICAgICAgLSAgICAgICAgNiAgMTYsMjU2LDEwMjQsMjA0 OCw0MDk2CiAgICAgaW5fbXVsdGkgICAgIDIgICAgIDFLICAgICAgIC0gICAgICAgIDMgIDEyOAog ICAgc2N0cF9pdGVyICAgICAwICAgICAwSyAgICAgICAtICAgICAgICAyICAyNTYKICAgICBzY3Rw X2lmbiAgICAgMiAgICAgMUsgICAgICAgLSAgICAgICAgMiAgMTI4CiAgICAgc2N0cF9pZmEgICAg IDQgICAgIDFLICAgICAgIC0gICAgICAgIDQgIDEyOAogICAgIHNjdHBfdnJmICAgICAxICAgICAx SyAgICAgICAtICAgICAgICAxICA2NAogICAgc2N0cF9hX2l0ICAgICAwICAgICAwSyAgICAgICAt ICAgICAgICAyICAxNgogICAgaG9zdGNhY2hlICAgICAxICAgIDE2SyAgICAgICAtICAgICAgICAx ICAKICAgICBzeW5jYWNoZSAgICAgMSAgICA3MksgICAgICAgLSAgICAgICAgMSAgCiAgICBpbjZf bXVsdGkgICAgMTIgICAgIDJLICAgICAgIC0gICAgICAgMTIgIDE2LDI1NgpDQU0gZGV2IHF1ZXVl ICAgICAxICAgICAxSyAgICAgICAtICAgICAgICAxICAxMjgKICAgICAgIERFVkZTMSAgICA4OSAg ICAyM0sgICAgICAgLSAgICAgICA5NSAgMjU2CiAgICAgICAgICBtbGQgICAgIDMgICAgIDFLICAg ICAgIC0gICAgICAgIDMgIDEyOAogICAgICBORlMgRkhBICAgICAxICAgICAxSyAgICAgICAtICAg ICAgICAxICAxMDI0CiAgICAgICAgICBycGMgICAgIDIgICAgIDVLICAgICAgIC0gICAgICAgIDIg IDEyOCw0MDk2CmF1ZGl0X2V2Y2xhc3MgICAxNzIgICAgIDNLICAgICAgIC0gICAgICAyMTEgIDE2 CiAgICAgc2F2ZWRpbm8gICAgIDAgICAgIDBLICAgICAgIC0gICAgICAgMjQgIDI1NgogICAgICAg ZGlycmVtICAgICA4ICAgICAxSyAgICAgICAtICAgICAgIDQ0ICAzMgogICAgICAgIG1rZGlyICAg ICAwICAgICAwSyAgICAgICAtICAgICAgIDEyICAzMgogICAgICAgZGlyYWRkICAgICA5ICAgICAx SyAgICAgICAtICAgICAgIDUyICA2NAogICAgIGZyZWVmaWxlICAgICA0ICAgICAxSyAgICAgICAt ICAgICAgIDI3ICAzMgogICAgIGZyZWVibGtzICAgICA0ICAgICAxSyAgICAgICAtICAgICAgIDIx ICAyNTYKICAgICBmcmVlZnJhZyAgICAgMSAgICAgMUsgICAgICAgLSAgICAgICAgMSAgMzIKICBh bGxvY2RpcmVjdCAgICAgMyAgICAgMUsgICAgICAgLSAgICAgICAzNyAgMTI4CiAgICBibXNhZmVt YXAgICAgIDEgICAgIDFLICAgICAgIC0gICAgICAgMTYgIDY0CiAgICAgICBuZXdibGsgICAgIDEg ICAgIDFLICAgICAgIC0gICAgICAgMzggIDY0LDI1NgogICAgIGlub2RlZGVwICAgIDIxICAgIDY3 SyAgICAgICAtICAgICAgIDcyICAxMjgKICAgICAgcGFnZWRlcCAgICAgNiAgICAgOUsgICAgICAg LSAgICAgICAxOCAgNjQKICB1ZnNfZGlyaGFzaCAgICAyNCAgICAgNUsgICAgICAgLSAgICAgICAy NCAgMTYsMzIsNjQsMTI4LDUxMgogICAgdWZzX21vdW50ICAgIDEyICAgIDI1SyAgICAgICAtICAg ICAgIDEyICAyNTYsMjA0OCw0MDk2CiAgICAgICBERVZGUzMgICAxMDkgICAgMTRLICAgICAgIC0g ICAgICAxMTYgIDEyOAogICAgICAgIERFVkZTICAgIDEyICAgICAxSyAgICAgICAtICAgICAgIDEz ICAxNiw2NAogICAgdm1fcGdkYXRhICAgICAyICAgIDE3SyAgICAgICAtICAgICAgICAyICA2NAog ICAgICAgREVWRlNQICAgICAzICAgICAxSyAgICAgICAtICAgICAgICA1ICAzMgogICAgIGF0a2Jk ZGV2ICAgICAyICAgICAxSyAgICAgICAtICAgICAgICAyICAzMgogICAgcGZzX25vZGVzICAgIDIx ICAgICAzSyAgICAgICAtICAgICAgIDIxICAxMjgKICAgICAgIGFwbWRldiAgICAgMSAgICAgMUsg ICAgICAgLSAgICAgICAgMSAgNjQKICAgIENBTSBxdWV1ZSAgICAgMyAgICAgMUsgICAgICAgLSAg ICAgICAgNyAgMTYKICAgICAgICAgR0VPTSAgICA2OCAgICAxNUsgICAgICAgLSAgICAgIDQwNiAg MTYsMzIsNjQsMTI4LDUxMiwxMDI0LDIwNDgKICAgICAgQ0FNIFNJTSAgICAgMSAgICAgMUsgICAg ICAgLSAgICAgICAgMSAgMTI4CiAgICAgIGlvX2FwaWMgICAgIDEgICAgIDFLICAgICAgIC0gICAg ICAgIDEgIDEwMjQKICAgICAgbWVtZGVzYyAgICAgMSAgICAgNEsgICAgICAgLSAgICAgICAgMSAg NDA5NgogICBDQU0gcGVyaXBoICAgICAyICAgICAxSyAgICAgICAtICAgICAgIDEyICAxNiwzMiw2 NCwxMjgKICBhdGFfZ2VuZXJpYyAgICAgMiAgICAgMksgICAgICAgLSAgICAgICAgMiAgMTAyNAog ICAgYWRfZHJpdmVyICAgICAxICAgICAxSyAgICAgICAtICAgICAgICAxICAzMgogICAgYXJfZHJp dmVyICAgICAwICAgICAwSyAgICAgICAtICAgICAgICA2ICA1MTIsMjA0OAogICAgICAgaXNhZGV2 ICAgICA1ICAgICAxSyAgICAgICAtICAgICAgICA1ICA2NAogICBhY2RfZHJpdmVyICAgICAxICAg ICAySyAgICAgICAtICAgICAgICAxICAyMDQ4CiAgICAgIENBTSBYUFQgICAgMTIgICAgIDJLICAg ICAgIC0gICAgICAgMjYgIDE2LDMyLDY0LDEwMjQKICAgICBuZXh1c2RldiAgICAgNCAgICAgMUsg ICAgICAgLSAgICAgICAgNCAgMTYKICAgICAgIGN5Y2xpYyAgICAgNCAgICAgMUsgICAgICAgLSAg ICAgICAgNCAgMTYsMzIsNjQKICAgICAgc29sYXJpcyAxODI2NTMgMzQwMzdLICAgICAgIC0gICAx ODI3MDUgIDE2LDMyLDY0LDEyOCwyNTYsNTEyLDEwMjQsMjA0OCw0MDk2CiAgICAgICAgICBmYnQg Mzg1NDAgIDI1MzdLICAgICAgIC0gICAgMzg1NDAgIDY0CgotLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kdm1zdGF0 IC16CgpJVEVNICAgICAgICAgICAgICAgICAgICAgU0laRSAgICAgTElNSVQgICAgICBVU0VEICAg ICAgRlJFRSAgUkVRVUVTVFMgIEZBSUxVUkVTCgpVTUEgS2VnczogICAgICAgICAgICAgICAgIDEy OCwgICAgICAgIDAsICAgICAgIDkwLCAgICAgICAgMCwgICAgICAgOTAsICAgICAgICAwClVNQSBa b25lczogICAgICAgICAgICAgICAgODg4LCAgICAgICAgMCwgICAgICAgOTAsICAgICAgICAyLCAg ICAgICA5MCwgICAgICAgIDAKVU1BIFNsYWJzOiAgICAgICAgICAgICAgICAyODQsICAgICAgICAw LCAgICAgIDQzNSwgICAgICAgMTMsICAgICAgNzc0LCAgICAgICAgMApVTUEgUkNudFNsYWJzOiAg ICAgICAgICAgIDU0NCwgICAgICAgIDAsICAgICAgIDY5LCAgICAgICAgMSwgICAgICAgNjksICAg ICAgICAwClVNQSBIYXNoOiAgICAgICAgICAgICAgICAgMTI4LCAgICAgICAgMCwgICAgICAgIDQs ICAgICAgIDI2LCAgICAgICAgNCwgICAgICAgIDAKMTYgQnVja2V0OiAgICAgICAgICAgICAgICAg NzYsICAgICAgICAwLCAgICAgICAzNCwgICAgICAgMTYsICAgICAgIDU1LCAgICAgICAgMAozMiBC dWNrZXQ6ICAgICAgICAgICAgICAgIDE0MCwgICAgICAgIDAsICAgICAgIDIyLCAgICAgICAgNiwg ICAgICAgNTQsICAgICAgICAwCjY0IEJ1Y2tldDogICAgICAgICAgICAgICAgMjY4LCAgICAgICAg MCwgICAgICAgMjQsICAgICAgICA0LCAgICAgICA3MCwgICAgICAgMTMKMTI4IEJ1Y2tldDogICAg ICAgICAgICAgICA1MjQsICAgICAgICAwLCAgICAgICAyOCwgICAgICAgIDAsICAgICAxMjI3LCAg ICAgIDExMgpWTSBPQkpFQ1Q6ICAgICAgICAgICAgICAgIDEzNiwgICAgICAgIDAsICAgICAgODYz LCAgICAgICAzNiwgICAgMTM3OTUsICAgICAgICAwCk1BUDogICAgICAgICAgICAgICAgICAgICAg MTQwLCAgICAgICAgMCwgICAgICAgIDcsICAgICAgIDIxLCAgICAgICAgNywgICAgICAgIDAKS01B UCBFTlRSWTogICAgICAgICAgICAgICAgNzIsICAgIDE1MjExLCAgICAgICAyNywgICAgICAxMzIs ICAgICA0OTI3LCAgICAgICAgMApNQVAgRU5UUlk6ICAgICAgICAgICAgICAgICA3MiwgICAgICAg IDAsICAgICAgNTA3LCAgICAgIDEyOSwgICAgMjUzMDgsICAgICAgICAwCkRQIGZha2VwZzogICAg ICAgICAgICAgICAgIDcyLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwg ICAgICAgIDAKU0cgZmFrZXBnOiAgICAgICAgICAgICAgICAgNzIsICAgICAgICAwLCAgICAgICAg MCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMAptdF96b25lOiAgICAgICAgICAgICAgICAg MjA1NiwgICAgICAgIDAsICAgICAgMjcxLCAgICAgICAgMCwgICAgICAyNzEsICAgICAgICAwCjE2 OiAgICAgICAgICAgICAgICAgICAgICAgIDE2LCAgICAgICAgMCwgICAxMjU0NzYsICAgICAgMTgx LCAgIDE1MzgzMiwgICAgICAgIDAKMzI6ICAgICAgICAgICAgICAgICAgICAgICAgMzIsICAgICAg ICAwLCAgICAxOTU2NywgICAgICAgOTUsICAgIDUwNzY1LCAgICAgICAgMAo2NDogICAgICAgICAg ICAgICAgICAgICAgICA2NCwgICAgICAgIDAsICAgIDQyNzQyLCAgICAgIDE1MSwgICAgNDc2OTEs ICAgICAgICAwCjEyODogICAgICAgICAgICAgICAgICAgICAgMTI4LCAgICAgICAgMCwgICAgNDE1 ODAsICAgICAgIDYwLCAgICA0NjAzMywgICAgICAgIDAKMjU2OiAgICAgICAgICAgICAgICAgICAg ICAyNTYsICAgICAgICAwLCAgICAgIDM2MSwgICAgICAgMjksICAgICAyMjc0LCAgICAgICAgMAo1 MTI6ICAgICAgICAgICAgICAgICAgICAgIDUxMiwgICAgICAgIDAsICAgICAgIDYzLCAgICAgICAg OSwgICAgICA3MDYsICAgICAgICAwCjEwMjQ6ICAgICAgICAgICAgICAgICAgICAxMDI0LCAgICAg ICAgMCwgICAgICAgMjgsICAgICAgIDg0LCAgICAgIDk2MSwgICAgICAgIDAKMjA0ODogICAgICAg ICAgICAgICAgICAgIDIwNDgsICAgICAgICAwLCAgICAgIDM4OSwgICAgICAgMTEsICAgICAgNTYz LCAgICAgICAgMAo0MDk2OiAgICAgICAgICAgICAgICAgICAgNDA5NiwgICAgICAgIDAsICAgICAg IDkxLCAgICAgICAgNywgICAgIDU3NjMsICAgICAgICAwCkZpbGVzOiAgICAgICAgICAgICAgICAg ICAgIDU2LCAgICAgICAgMCwgICAgICAgNzAsICAgICAgMTMxLCAgICAgNTAyOCwgICAgICAgIDAK VFVSTlNUSUxFOiAgICAgICAgICAgICAgICAgNzIsICAgICAgICAwLCAgICAgICA5MSwgICAgICAg MjksICAgICAgIDkxLCAgICAgICAgMAp1bXR4IHBpOiAgICAgICAgICAgICAgICAgICA1MiwgICAg ICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwCk1BQyBsYWJlbHM6 ICAgICAgICAgICAgICAgIDIwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAg MCwgICAgICAgIDAKUFJPQzogICAgICAgICAgICAgICAgICAgICA2ODAsICAgICAgICAwLCAgICAg ICA0NSwgICAgICAgMTUsICAgICAxMTM0LCAgICAgICAgMApUSFJFQUQ6ICAgICAgICAgICAgICAg ICAgIDcyMCwgICAgICAgIDAsICAgICAgIDgwLCAgICAgICAxMCwgICAgICAgODAsICAgICAgICAw ClNMRUVQUVVFVUU6ICAgICAgICAgICAgICAgIDQ0LCAgICAgICAgMCwgICAgICAgOTEsICAgICAg IDg2LCAgICAgICA5MSwgICAgICAgIDAKVk1TUEFDRTogICAgICAgICAgICAgICAgICAyMzIsICAg ICAgICAwLCAgICAgICAyNywgICAgICAgMjQsICAgICAxMTE3LCAgICAgICAgMApjcHVzZXQ6ICAg ICAgICAgICAgICAgICAgICA0MCwgICAgICAgIDAsICAgICAgICAyLCAgICAgIDE4MiwgICAgICAg IDIsICAgICAgICAwCmN5Y2xpY19pZF9jYWNoZTogICAgICAgICAgIDMyLCAgICAgICAgMCwgICAg ICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAKYXVkaXRfcmVjb3JkOiAgICAgICAg ICAgICA4MTYsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAg MAptYnVmX3BhY2tldDogICAgICAgICAgICAgIDI1NiwgICAgICAgIDAsICAgICAgIDY0LCAgICAg ICA3OCwgICAgICAyNjEsICAgICAgICAwCm1idWY6ICAgICAgICAgICAgICAgICAgICAgMjU2LCAg ICAgICAgMCwgICAgICAgIDEsICAgICAgMTQxLCAgICAgIDM5MSwgICAgICAgIDAKbWJ1Zl9jbHVz dGVyOiAgICAgICAgICAgIDIwNDgsICAgICA4NjQwLCAgICAgIDEyOCwgICAgICAgIDYsICAgICAg MTI4LCAgICAgICAgMAptYnVmX2p1bWJvX3BhZ2U6ICAgICAgICAgNDA5NiwgICAgIDQzMjAsICAg ICAgICAwLCAgICAgICAgMiwgICAgICAgIDYsICAgICAgICAwCm1idWZfanVtYm9fOWs6ICAgICAg ICAgICA5MjE2LCAgICAgNjQ4MCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAg IDAKbWJ1Zl9qdW1ib18xNms6ICAgICAgICAgMTYzODQsICAgICA0MzIwLCAgICAgICAgMCwgICAg ICAgIDAsICAgICAgICAwLCAgICAgICAgMAptYnVmX2V4dF9yZWZjbnQ6ICAgICAgICAgICAgNCwg ICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwCmR0cmFjZV9z dGF0ZV9jYWNoZTogICAgICAxNzkyLCAgICAgICAgMCwgICAgICAgIDEsICAgICAgICAzLCAgICAg ICAgMSwgICAgICAgIDAKZ19iaW86ICAgICAgICAgICAgICAgICAgICAxNDAsICAgICAgICAwLCAg ICAgICAgMCwgICAgICAyMjQsICAgICA0NjYzLCAgICAgICAgMAp0dHlpbnE6ICAgICAgICAgICAg ICAgICAgIDE1MiwgICAgICAgIDAsICAgICAgMTM1LCAgICAgICAyMSwgICAgICA0OTUsICAgICAg ICAwCnR0eW91dHE6ICAgICAgICAgICAgICAgICAgMjU2LCAgICAgICAgMCwgICAgICAgNzIsICAg ICAgIDE4LCAgICAgIDI2NCwgICAgICAgIDAKYXRhX3JlcXVlc3Q6ICAgICAgICAgICAgICAyMDQs ICAgICAgICAwLCAgICAgICAgMSwgICAgICAgNzUsICAgICAyNDcyLCAgICAgICAgMAphdGFfY29t cG9zaXRlOiAgICAgICAgICAgIDE4MCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAg ICAgIDAsICAgICAgICAwClZOT0RFOiAgICAgICAgICAgICAgICAgICAgMjY4LCAgICAgICAgMCwg ICAgICA1MjksICAgICAgIDE3LCAgICAgIDU1NywgICAgICAgIDAKVk5PREVQT0xMOiAgICAgICAg ICAgICAgICAgNjAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAg ICAgMApOQU1FSTogICAgICAgICAgICAgICAgICAgMTAyNCwgICAgICAgIDAsICAgICAgICAwLCAg ICAgICAgOCwgICAgIDk0NzAsICAgICAgICAwClMgVkZTIENhY2hlOiAgICAgICAgICAgICAgIDcy LCAgICAgICAgMCwgICAgICA0OTAsICAgICAgIDQwLCAgICAgMTM1NywgICAgICAgIDAKTCBWRlMg Q2FjaGU6ICAgICAgICAgICAgICAyOTIsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAg ICAgICAwLCAgICAgICAgMApORlNNT1VOVDogICAgICAgICAgICAgICAgIDUyOCwgICAgICAgIDAs ICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwCk5GU05PREU6ICAgICAgICAg ICAgICAgICAgNDg0LCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAg ICAgIDAKRElSSEFTSDogICAgICAgICAgICAgICAgIDEwMjQsICAgICAgICAwLCAgICAgICAzNywg ICAgICAgMTEsICAgICAgIDM3LCAgICAgICAgMApwaXBlOiAgICAgICAgICAgICAgICAgICAgIDM5 MiwgICAgICAgIDAsICAgICAgICAzLCAgICAgICAxNywgICAgICA2OTIsICAgICAgICAwCmtzaWdp bmZvOiAgICAgICAgICAgICAgICAgIDgwLCAgICAgICAgMCwgICAgICAgNDgsICAgICAxMDA4LCAg ICAgICA2NSwgICAgICAgIDAKaXRpbWVyOiAgICAgICAgICAgICAgICAgICAyMjAsICAgICAgICAw LCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMApLTk9URTogICAgICAgICAg ICAgICAgICAgICA3MiwgICAgICAgIDAsICAgICAgICAwLCAgICAgIDEwNiwgICAgICAgIDgsICAg ICAgICAwCnNvY2tldDogICAgICAgICAgICAgICAgICAgNDEyLCAgICAgODY0MCwgICAgICAgMTcs ICAgICAgIDE5LCAgICAgIDIzMywgICAgICAgIDAKdW5wY2I6ICAgICAgICAgICAgICAgICAgICAx NzIsICAgICA4NjQ4LCAgICAgICAgNywgICAgICAgMzksICAgICAgIDc0LCAgICAgICAgMAppcHE6 ICAgICAgICAgICAgICAgICAgICAgICAzMiwgICAgICAzMzksICAgICAgICAwLCAgICAgICAgMCwg ICAgICAgIDAsICAgICAgICAwCnVkcF9pbnBjYjogICAgICAgICAgICAgICAgMjIwLCAgICAgODY0 MCwgICAgICAgIDIsICAgICAgIDM0LCAgICAgIDEzNywgICAgICAgIDAKdWRwY2I6ICAgICAgICAg ICAgICAgICAgICAgIDgsICAgICA4NzI5LCAgICAgICAgMiwgICAgICAyMDEsICAgICAgMTM3LCAg ICAgICAgMAp0Y3BfaW5wY2I6ICAgICAgICAgICAgICAgIDIyMCwgICAgIDg2NDAsICAgICAgICA2 LCAgICAgICAzMCwgICAgICAgMTQsICAgICAgICAwCnRjcGNiOiAgICAgICAgICAgICAgICAgICAg NjMyLCAgICAgODY0MCwgICAgICAgIDYsICAgICAgICA2LCAgICAgICAxNCwgICAgICAgIDAKdGNw dHc6ICAgICAgICAgICAgICAgICAgICAgNTIsICAgICAxNzI4LCAgICAgICAgMCwgICAgICAgIDAs ICAgICAgICAwLCAgICAgICAgMApzeW5jYWNoZTogICAgICAgICAgICAgICAgIDExMiwgICAgMTUz NjUsICAgICAgICAwLCAgICAgICA3MCwgICAgICAgIDUsICAgICAgICAwCmhvc3RjYWNoZTogICAg ICAgICAgICAgICAgIDc2LCAgICAxNTQwMCwgICAgICAgIDEsICAgICAgIDk5LCAgICAgICAgMSwg ICAgICAgIDAKdGNwcmVhc3M6ICAgICAgICAgICAgICAgICAgMjAsICAgICAgNjc2LCAgICAgICAg MCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMApzYWNraG9sZTogICAgICAgICAgICAgICAg ICAyMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwCnNj dHBfZXA6ICAgICAgICAgICAgICAgICAgODY0LCAgICAgODY0MCwgICAgICAgIDAsICAgICAgICAw LCAgICAgICAgMCwgICAgICAgIDAKc2N0cF9hc29jOiAgICAgICAgICAgICAgIDE0ODgsICAgIDQw MDAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMApzY3RwX2xhZGRyOiAg ICAgICAgICAgICAgICAyNCwgICAgODAwNDAsICAgICAgICAwLCAgICAgIDE0NSwgICAgICAgIDMs ICAgICAgICAwCnNjdHBfcmFkZHI6ICAgICAgICAgICAgICAgNDMyLCAgICA4MDAwMSwgICAgICAg IDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAKc2N0cF9jaHVuazogICAgICAgICAgICAg ICAgOTIsICAgNDAwMDA4LCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMApz Y3RwX3JlYWRxOiAgICAgICAgICAgICAgICA3NiwgICA0MDAwMDAsICAgICAgICAwLCAgICAgICAg MCwgICAgICAgIDAsICAgICAgICAwCnNjdHBfc3RyZWFtX21zZ19vdXQ6ICAgICAgIDY0LCAgIDQw MDAyMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAKc2N0cF9hc2NvbmY6 ICAgICAgICAgICAgICAgMjQsICAgNDAwMDU1LCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAw LCAgICAgICAgMApzY3RwX2FzY29uZl9hY2s6ICAgICAgICAgICAyNCwgICA0MDAwNTUsICAgICAg ICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAwCnJpcGNiOiAgICAgICAgICAgICAgICAg ICAgMjIwLCAgICAgODY0MCwgICAgICAgIDEsICAgICAgIDM1LCAgICAgICAgMiwgICAgICAgIDAK cnRlbnRyeTogICAgICAgICAgICAgICAgICAxMDgsICAgICAgICAwLCAgICAgICAgOSwgICAgICAg NjMsICAgICAgIDExLCAgICAgICAgMApzZWxmZDogICAgICAgICAgICAgICAgICAgICAyOCwgICAg ICAgIDAsICAgICAgIDM4LCAgICAgIDIxNiwgICAgIDExMTcsICAgICAgICAwCmlwNGZsb3c6ICAg ICAgICAgICAgICAgICAgIDQwLCAgICAgODY0OCwgICAgICAgIDIsICAgICAgMTgyLCAgICAgICAg NCwgICAgICAgIDAKaXA2ZmxvdzogICAgICAgICAgICAgICAgICAgNjQsICAgICA4NjQyLCAgICAg ICAgMCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMApTV0FQTUVUQTogICAgICAgICAgICAg ICAgIDI3NiwgICAgMzAwNDQsICAgICAgICAwLCAgICAgICAgMCwgICAgICAgIDAsICAgICAgICAw Ck1vdW50cG9pbnRzOiAgICAgICAgICAgICAgNjQ0LCAgICAgICAgMCwgICAgICAgIDUsICAgICAg ICA3LCAgICAgICAgNSwgICAgICAgIDAKRkZTIGlub2RlOiAgICAgICAgICAgICAgICAxMTYsICAg ICAgICAwLCAgICAgIDQ4MCwgICAgICAgNDgsICAgICAgNTA3LCAgICAgICAgMApGRlMxIGRpbm9k ZTogICAgICAgICAgICAgIDEyOCwgICAgICAgIDAsICAgICAgICAwLCAgICAgICAgMCwgICAgICAg IDAsICAgICAgICAwCkZGUzIgZGlub2RlOiAgICAgICAgICAgICAgMjU2LCAgICAgICAgMCwgICAg ICA0ODAsICAgICAgIDQ1LCAgICAgIDUwNywgICAgICAgIDAKCgotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0Kdm1z dGF0IC1pCgppbnRlcnJ1cHQgICAgICAgICAgICAgICAgICAgICAgICAgIHRvdGFsICAgICAgIHJh dGUKaXJxMTogYXRrYmQwICAgICAgICAgICAgICAgICAgICAgICAgIDU5MSAgICAgICAgIDE1Cmly cTExOiByZTAgdWhjaTAgICAgICAgICAgICAgICAgICAgICAzMDQgICAgICAgICAgOAppcnExNDog YXRhMCAgICAgICAgICAgICAgICAgICAgICAgICAxNDAyICAgICAgICAgMzcKaXJxMTU6IGF0YTEg ICAgICAgICAgICAgICAgICAgICAgICAgICA0MiAgICAgICAgICAxCmNwdTA6IHRpbWVyICAgICAg ICAgICAgICAgICAgICAgICAgMjc1MTMgICAgICAgIDc0MwpUb3RhbCAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIDI5ODUyICAgICAgICA4MDYKCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpwc3RhdCAtVAoK IDcwLzM4NDggZmlsZXMKME0vNDcwTSBzd2FwIHNwYWNlCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KcHN0YXQg LXMKCkRldmljZSAgICAgICAgICA1MTItYmxvY2tzICAgICBVc2VkICAgIEF2YWlsIENhcGFjaXR5 Ci9kZXYvYWQwczFiICAgICAgICAgOTY0MTc2ICAgICAgICAwICAgOTY0MTc2ICAgICAwJQoKLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tCmlvc3RhdAoKaW9zdGF0OiBrdm1fcmVhZChfdGtfbmluKTogaW52YWxpZCBh ZGRyZXNzICgweDApCmlvc3RhdDogZGlzYWJsaW5nIFRUWSBzdGF0aXN0aWNzCmlvc3RhdDoga3Zt X2dldGNwdGltZTogaW52YWxpZCBhZGRyZXNzICgweDApCmlvc3RhdDogZGlzYWJsaW5nIENQVSB0 aW1lIHN0YXRpc3RpY3MKICAgICAgICAgICAgIGFkMCAKICBLQi90IHRwcyAgTUIvcyAKIDE1Ljc3 ICAzMSAgMC40OCAKCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQppcGNzIC1hCgpNZXNzYWdlIFF1ZXVlczoKVCAg ICAgICAgICAgSUQgICAgICAgICAgS0VZIE1PREUgICAgICAgIE9XTkVSICAgIEdST1VQICAgIENS RUFUT1IgIENHUk9VUCAgICAgICAgICAgICAgICAgQ0JZVEVTICAgICAgICAgICAgICAgICBRTlVN ICAgICAgICAgICAgICAgUUJZVEVTICAgICAgICBMU1BJRCAgICAgICAgTFJQSUQgU1RJTUUgICAg UlRJTUUgICAgQ1RJTUUgICAKClNoYXJlZCBNZW1vcnk6ClQgICAgICAgICAgIElEICAgICAgICAg IEtFWSBNT0RFICAgICAgICBPV05FUiAgICBHUk9VUCAgICBDUkVBVE9SICBDR1JPVVAgICAgICAg ICBOQVRUQ0ggICAgICAgIFNFR1NaICAgICAgICAgQ1BJRCAgICAgICAgIExQSUQgQVRJTUUgICAg RFRJTUUgICAgQ1RJTUUgICAKClNlbWFwaG9yZXM6ClQgICAgICAgICAgIElEICAgICAgICAgIEtF WSBNT0RFICAgICAgICBPV05FUiAgICBHUk9VUCAgICBDUkVBVE9SICBDR1JPVVAgICAgICAgICAg TlNFTVMgT1RJTUUgICAgQ1RJTUUgICAKCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KaXBjcyAtVAoKbXNnaW5m bzoKCW1zZ21heDogICAgICAgIDE2Mzg0CShtYXggY2hhcmFjdGVycyBpbiBhIG1lc3NhZ2UpCglt c2dtbmk6ICAgICAgICAgICA0MAkoIyBvZiBtZXNzYWdlIHF1ZXVlcykKCW1zZ21uYjogICAgICAg ICAyMDQ4CShtYXggY2hhcmFjdGVycyBpbiBhIG1lc3NhZ2UgcXVldWUpCgltc2d0cWw6ICAgICAg ICAgICA0MAkobWF4ICMgb2YgbWVzc2FnZXMgaW4gc3lzdGVtKQoJbXNnc3N6OiAgICAgICAgICAg IDgJKHNpemUgb2YgYSBtZXNzYWdlIHNlZ21lbnQpCgltc2dzZWc6ICAgICAgICAgMjA0OAkoIyBv ZiBtZXNzYWdlIHNlZ21lbnRzIGluIHN5c3RlbSkKCnNobWluZm86CglzaG1tYXg6ICAgICAzMzU1 NDQzMgkobWF4IHNoYXJlZCBtZW1vcnkgc2VnbWVudCBzaXplKQoJc2htbWluOiAgICAgICAgICAg IDEJKG1pbiBzaGFyZWQgbWVtb3J5IHNlZ21lbnQgc2l6ZSkKCXNobW1uaTogICAgICAgICAgMTky CShtYXggbnVtYmVyIG9mIHNoYXJlZCBtZW1vcnkgaWRlbnRpZmllcnMpCglzaG1zZWc6ICAgICAg ICAgIDEyOAkobWF4IHNoYXJlZCBtZW1vcnkgc2VnbWVudHMgcGVyIHByb2Nlc3MpCglzaG1hbGw6 ICAgICAgICAgODE5MgkobWF4IGFtb3VudCBvZiBzaGFyZWQgbWVtb3J5IGluIHBhZ2VzKQoKc2Vt aW5mbzoKCXNlbW1hcDogICAgICAgICAgIDMwCSgjIG9mIGVudHJpZXMgaW4gc2VtYXBob3JlIG1h cCkKCXNlbW1uaTogICAgICAgICAgIDEwCSgjIG9mIHNlbWFwaG9yZSBpZGVudGlmaWVycykKCXNl bW1uczogICAgICAgICAgIDYwCSgjIG9mIHNlbWFwaG9yZXMgaW4gc3lzdGVtKQoJc2VtbW51OiAg ICAgICAgICAgMzAJKCMgb2YgdW5kbyBzdHJ1Y3R1cmVzIGluIHN5c3RlbSkKCXNlbW1zbDogICAg ICAgICAgIDYwCShtYXggIyBvZiBzZW1hcGhvcmVzIHBlciBpZCkKCXNlbW9wbTogICAgICAgICAg MTAwCShtYXggIyBvZiBvcGVyYXRpb25zIHBlciBzZW1vcCBjYWxsKQoJc2VtdW1lOiAgICAgICAg ICAgMTAJKG1heCAjIG9mIHVuZG8gZW50cmllcyBwZXIgcHJvY2VzcykKCXNlbXVzejogICAgICAg ICAgMTM2CShzaXplIGluIGJ5dGVzIG9mIHVuZG8gc3RydWN0dXJlKQoJc2Vtdm14OiAgICAgICAg MzI3NjcJKHNlbWFwaG9yZSBtYXhpbXVtIHZhbHVlKQoJc2VtYWVtOiAgICAgICAgMTYzODQJKGFk anVzdCBvbiBleGl0IG1heCB2YWx1ZSkKCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KbmZzc3RhdAoKQ2xpZW50 IEluZm86ClJwYyBDb3VudHM6CiAgR2V0YXR0ciAgIFNldGF0dHIgICAgTG9va3VwICBSZWFkbGlu ayAgICAgIFJlYWQgICAgIFdyaXRlICAgIENyZWF0ZSAgICBSZW1vdmUKICAgICAgICAwICAgICAg ICAgMCAgICAgICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwICAg ICAgICAgMAogICBSZW5hbWUgICAgICBMaW5rICAgU3ltbGluayAgICAgTWtkaXIgICAgIFJtZGly ICAgUmVhZGRpciAgUmRpclBsdXMgICAgQWNjZXNzCiAgICAgICAgMCAgICAgICAgIDAgICAgICAg ICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAKICAg IE1rbm9kICAgIEZzc3RhdCAgICBGc2luZm8gIFBhdGhDb25mICAgIENvbW1pdAogICAgICAgIDAg ICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwClJwYyBJbmZvOgogVGltZWRP dXQgICBJbnZhbGlkIFggUmVwbGllcyAgIFJldHJpZXMgIFJlcXVlc3RzCiAgICAgICAgMCAgICAg ICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAKQ2FjaGUgSW5mbzoKQXR0ciBIaXRz ICAgIE1pc3NlcyBMa3VwIEhpdHMgICAgTWlzc2VzIEJpb1IgSGl0cyAgICBNaXNzZXMgQmlvVyBI aXRzICAgIE1pc3NlcwogICAgICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAg ICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwCkJpb1JMSGl0cyAgICBNaXNzZXMg QmlvRCBIaXRzICAgIE1pc3NlcyBEaXJFIEhpdHMgICAgTWlzc2VzCiAgICAgICAgMCAgICAgICAg IDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwCgpTZXJ2ZXIgSW5mbzoK ICBHZXRhdHRyICAgU2V0YXR0ciAgICBMb29rdXAgIFJlYWRsaW5rICAgICAgUmVhZCAgICAgV3Jp dGUgICAgQ3JlYXRlICAgIFJlbW92ZQogICAgICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAg ICAgIDAgICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwCiAgIFJlbmFtZSAg ICAgIExpbmsgICBTeW1saW5rICAgICBNa2RpciAgICAgUm1kaXIgICBSZWFkZGlyICBSZGlyUGx1 cyAgICBBY2Nlc3MKICAgICAgICAwICAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAwICAgICAg ICAgMCAgICAgICAgIDAgICAgICAgICAwICAgICAgICAgMAogICAgTWtub2QgICAgRnNzdGF0ICAg IEZzaW5mbyAgUGF0aENvbmYgICAgQ29tbWl0CiAgICAgICAgMCAgICAgICAgIDAgICAgICAgICAw ICAgICAgICAgMCAgICAgICAgIDAKU2VydmVyIFJldC1GYWlsZWQKICAgICAgICAgICAgICAgIDAK U2VydmVyIEZhdWx0cwogICAgICAgICAgICAwClNlcnZlciBDYWNoZSBTdGF0czoKICAgSW5wcm9n ICAgICAgSWRlbSAgTm9uLWlkZW0gICAgTWlzc2VzCiAgICAgICAgMCAgICAgICAgIDAgICAgICAg ICAwICAgICAgICAgMApTZXJ2ZXIgV3JpdGUgR2F0aGVyaW5nOgogV3JpdGVPcHMgIFdyaXRlUlBD ICAgT3BzYXZlZAogICAgICAgIDAgICAgICAgICAwICAgICAgICAgMAoKLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t Cm5ldHN0YXQgLXMKCnRjcDoKCTEwNyBwYWNrZXRzIHNlbnQKCQk5NCBkYXRhIHBhY2tldHMgKDg4 NjQgYnl0ZXMpCgkJMCBkYXRhIHBhY2tldHMgKDAgYnl0ZXMpIHJldHJhbnNtaXR0ZWQKCQkwIGRh dGEgcGFja2V0cyB1bm5lY2Vzc2FyaWx5IHJldHJhbnNtaXR0ZWQKCQkwIHJlc2VuZHMgaW5pdGlh dGVkIGJ5IE1UVSBkaXNjb3ZlcnkKCQkxMCBhY2stb25seSBwYWNrZXRzICgwIGRlbGF5ZWQpCgkJ MCBVUkcgb25seSBwYWNrZXRzCgkJMCB3aW5kb3cgcHJvYmUgcGFja2V0cwoJCTAgd2luZG93IHVw ZGF0ZSBwYWNrZXRzCgkJMyBjb250cm9sIHBhY2tldHMKCTE1NyBwYWNrZXRzIHJlY2VpdmVkCgkJ OTkgYWNrcyAoZm9yIDg4NzIgYnl0ZXMpCgkJMyBkdXBsaWNhdGUgYWNrcwoJCTAgYWNrcyBmb3Ig dW5zZW50IGRhdGEKCQk2MSBwYWNrZXRzICg1OTIwIGJ5dGVzKSByZWNlaXZlZCBpbi1zZXF1ZW5j ZQoJCTAgY29tcGxldGVseSBkdXBsaWNhdGUgcGFja2V0cyAoMCBieXRlcykKCQkwIG9sZCBkdXBs aWNhdGUgcGFja2V0cwoJCTAgcGFja2V0cyB3aXRoIHNvbWUgZHVwLiBkYXRhICgwIGJ5dGVzIGR1 cGVkKQoJCTAgb3V0LW9mLW9yZGVyIHBhY2tldHMgKDAgYnl0ZXMpCgkJMCBwYWNrZXRzICgwIGJ5 dGVzKSBvZiBkYXRhIGFmdGVyIHdpbmRvdwoJCTAgd2luZG93IHByb2JlcwoJCTAgd2luZG93IHVw ZGF0ZSBwYWNrZXRzCgkJMCBwYWNrZXRzIHJlY2VpdmVkIGFmdGVyIGNsb3NlCgkJMCBkaXNjYXJk ZWQgZm9yIGJhZCBjaGVja3N1bXMKCQkwIGRpc2NhcmRlZCBmb3IgYmFkIGhlYWRlciBvZmZzZXQg ZmllbGRzCgkJMCBkaXNjYXJkZWQgYmVjYXVzZSBwYWNrZXQgdG9vIHNob3J0CgkJMCBkaXNjYXJk ZWQgZHVlIHRvIG1lbW9yeSBwcm9ibGVtcwoJMCBjb25uZWN0aW9uIHJlcXVlc3RzCgk1IGNvbm5l Y3Rpb24gYWNjZXB0cwoJMCBiYWQgY29ubmVjdGlvbiBhdHRlbXB0cwoJMCBsaXN0ZW4gcXVldWUg b3ZlcmZsb3dzCgkwIGlnbm9yZWQgUlNUcyBpbiB0aGUgd2luZG93cwoJNSBjb25uZWN0aW9ucyBl c3RhYmxpc2hlZCAoaW5jbHVkaW5nIGFjY2VwdHMpCgk4IGNvbm5lY3Rpb25zIGNsb3NlZCAoaW5j bHVkaW5nIDAgZHJvcHMpCgkJMCBjb25uZWN0aW9ucyB1cGRhdGVkIGNhY2hlZCBSVFQgb24gY2xv c2UKCQkwIGNvbm5lY3Rpb25zIHVwZGF0ZWQgY2FjaGVkIFJUVCB2YXJpYW5jZSBvbiBjbG9zZQoJ CTAgY29ubmVjdGlvbnMgdXBkYXRlZCBjYWNoZWQgc3N0aHJlc2ggb24gY2xvc2UKCTAgZW1icnlv bmljIGNvbm5lY3Rpb25zIGRyb3BwZWQKCTk0IHNlZ21lbnRzIHVwZGF0ZWQgcnR0IChvZiA5NCBh dHRlbXB0cykKCTAgcmV0cmFuc21pdCB0aW1lb3V0cwoJCTAgY29ubmVjdGlvbnMgZHJvcHBlZCBi eSByZXhtaXQgdGltZW91dAoJMCBwZXJzaXN0IHRpbWVvdXRzCgkJMCBjb25uZWN0aW9ucyBkcm9w cGVkIGJ5IHBlcnNpc3QgdGltZW91dAoJMCBDb25uZWN0aW9ucyAoZmluX3dhaXRfMikgZHJvcHBl ZCBiZWNhdXNlIG9mIHRpbWVvdXQKCTAga2VlcGFsaXZlIHRpbWVvdXRzCgkJMCBrZWVwYWxpdmUg cHJvYmVzIHNlbnQKCQkwIGNvbm5lY3Rpb25zIGRyb3BwZWQgYnkga2VlcGFsaXZlCgk0MyBjb3Jy ZWN0IEFDSyBoZWFkZXIgcHJlZGljdGlvbnMKCTQ4IGNvcnJlY3QgZGF0YSBwYWNrZXQgaGVhZGVy IHByZWRpY3Rpb25zCgk1IHN5bmNhY2hlIGVudHJpZXMgYWRkZWQKCQkwIHJldHJhbnNtaXR0ZWQK CQkwIGR1cHN5bgoJCTAgZHJvcHBlZAoJCTUgY29tcGxldGVkCgkJMCBidWNrZXQgb3ZlcmZsb3cK CQkwIGNhY2hlIG92ZXJmbG93CgkJMCByZXNldAoJCTAgc3RhbGUKCQkwIGFib3J0ZWQKCQkwIGJh ZGFjawoJCTAgdW5yZWFjaAoJCTAgem9uZSBmYWlsdXJlcwoJNSBjb29raWVzIHNlbnQKCTAgY29v a2llcyByZWNlaXZlZAoJMCBTQUNLIHJlY292ZXJ5IGVwaXNvZGVzCgkwIHNlZ21lbnQgcmV4bWl0 cyBpbiBTQUNLIHJlY292ZXJ5IGVwaXNvZGVzCgkwIGJ5dGUgcmV4bWl0cyBpbiBTQUNLIHJlY292 ZXJ5IGVwaXNvZGVzCgkwIFNBQ0sgb3B0aW9ucyAoU0FDSyBibG9ja3MpIHJlY2VpdmVkCgkwIFNB Q0sgb3B0aW9ucyAoU0FDSyBibG9ja3MpIHNlbnQKCTAgU0FDSyBzY29yZWJvYXJkIG92ZXJmbG93 CgkwIHBhY2tldHMgd2l0aCBFQ04gQ0UgYml0IHNldAoJMCBwYWNrZXRzIHdpdGggRUNOIEVDVCgw KSBiaXQgc2V0CgkwIHBhY2tldHMgd2l0aCBFQ04gRUNUKDEpIGJpdCBzZXQKCTAgc3VjY2Vzc2Z1 bCBFQ04gaGFuZHNoYWtlcwoJMCB0aW1lcyBFQ04gcmVkdWNlZCB0aGUgY29uZ2VzdGlvbiB3aW5k b3cKdWRwOgoJNiBkYXRhZ3JhbXMgcmVjZWl2ZWQKCTAgd2l0aCBpbmNvbXBsZXRlIGhlYWRlcgoJ MCB3aXRoIGJhZCBkYXRhIGxlbmd0aCBmaWVsZAoJMCB3aXRoIGJhZCBjaGVja3N1bQoJMCB3aXRo IG5vIGNoZWNrc3VtCgkwIGRyb3BwZWQgZHVlIHRvIG5vIHNvY2tldAoJMiBicm9hZGNhc3QvbXVs dGljYXN0IGRhdGFncmFtcyB1bmRlbGl2ZXJlZAoJMCBkcm9wcGVkIGR1ZSB0byBmdWxsIHNvY2tl dCBidWZmZXJzCgkwIG5vdCBmb3IgaGFzaGVkIHBjYgoJNCBkZWxpdmVyZWQKCTQgZGF0YWdyYW1z IG91dHB1dAoJMCB0aW1lcyBtdWx0aWNhc3Qgc291cmNlIGZpbHRlciBtYXRjaGVkCmlwOgoJMTgw IHRvdGFsIHBhY2tldHMgcmVjZWl2ZWQKCTAgYmFkIGhlYWRlciBjaGVja3N1bXMKCTAgd2l0aCBz aXplIHNtYWxsZXIgdGhhbiBtaW5pbXVtCgkwIHdpdGggZGF0YSBzaXplIDwgZGF0YSBsZW5ndGgK CTAgd2l0aCBpcCBsZW5ndGggPiBtYXggaXAgcGFja2V0IHNpemUKCTAgd2l0aCBoZWFkZXIgbGVu Z3RoIDwgZGF0YSBzaXplCgkwIHdpdGggZGF0YSBsZW5ndGggPCBoZWFkZXIgbGVuZ3RoCgkwIHdp dGggYmFkIG9wdGlvbnMKCTAgd2l0aCBpbmNvcnJlY3QgdmVyc2lvbiBudW1iZXIKCTAgZnJhZ21l bnRzIHJlY2VpdmVkCgkwIGZyYWdtZW50cyBkcm9wcGVkIChkdXAgb3Igb3V0IG9mIHNwYWNlKQoJ MCBmcmFnbWVudHMgZHJvcHBlZCBhZnRlciB0aW1lb3V0CgkwIHBhY2tldHMgcmVhc3NlbWJsZWQg b2sKCTE2MyBwYWNrZXRzIGZvciB0aGlzIGhvc3QKCTAgcGFja2V0cyBmb3IgdW5rbm93bi91bnN1 cHBvcnRlZCBwcm90b2NvbAoJMCBwYWNrZXRzIGZvcndhcmRlZCAoMCBwYWNrZXRzIGZhc3QgZm9y d2FyZGVkKQoJMTcgcGFja2V0cyBub3QgZm9yd2FyZGFibGUKCTAgcGFja2V0cyByZWNlaXZlZCBm b3IgdW5rbm93biBtdWx0aWNhc3QgZ3JvdXAKCTAgcmVkaXJlY3RzIHNlbnQKCTExMyBwYWNrZXRz IHNlbnQgZnJvbSB0aGlzIGhvc3QKCTAgcGFja2V0cyBzZW50IHdpdGggZmFicmljYXRlZCBpcCBo ZWFkZXIKCTAgb3V0cHV0IHBhY2tldHMgZHJvcHBlZCBkdWUgdG8gbm8gYnVmcywgZXRjLgoJMCBv dXRwdXQgcGFja2V0cyBkaXNjYXJkZWQgZHVlIHRvIG5vIHJvdXRlCgkwIG91dHB1dCBkYXRhZ3Jh bXMgZnJhZ21lbnRlZAoJMCBmcmFnbWVudHMgY3JlYXRlZAoJMCBkYXRhZ3JhbXMgdGhhdCBjYW4n dCBiZSBmcmFnbWVudGVkCgkwIHR1bm5lbGluZyBwYWNrZXRzIHRoYXQgY2FuJ3QgZmluZCBnaWYK CTAgZGF0YWdyYW1zIHdpdGggYmFkIGFkZHJlc3MgaW4gaGVhZGVyCmljbXA6CgkwIGNhbGxzIHRv IGljbXBfZXJyb3IKCTAgZXJyb3JzIG5vdCBnZW5lcmF0ZWQgaW4gcmVzcG9uc2UgdG8gYW4gaWNt cCBtZXNzYWdlCgkwIG1lc3NhZ2VzIHdpdGggYmFkIGNvZGUgZmllbGRzCgkwIG1lc3NhZ2VzIGxl c3MgdGhhbiB0aGUgbWluaW11bSBsZW5ndGgKCTAgbWVzc2FnZXMgd2l0aCBiYWQgY2hlY2tzdW0K CTAgbWVzc2FnZXMgd2l0aCBiYWQgbGVuZ3RoCgkwIG11bHRpY2FzdCBlY2hvIHJlcXVlc3RzIGln bm9yZWQKCTAgbXVsdGljYXN0IHRpbWVzdGFtcCByZXF1ZXN0cyBpZ25vcmVkCgkwIG1lc3NhZ2Ug cmVzcG9uc2VzIGdlbmVyYXRlZAoJMCBpbnZhbGlkIHJldHVybiBhZGRyZXNzZXMKCTAgbm8gcmV0 dXJuIHJvdXRlcwppZ21wOgoJMCBtZXNzYWdlcyByZWNlaXZlZAoJMCBtZXNzYWdlcyByZWNlaXZl ZCB3aXRoIHRvbyBmZXcgYnl0ZXMKCTAgbWVzc2FnZXMgcmVjZWl2ZWQgd2l0aCB3cm9uZyBUVEwK CTAgbWVzc2FnZXMgcmVjZWl2ZWQgd2l0aCBiYWQgY2hlY2tzdW0KCTAgVjEvVjIgbWVtYmVyc2hp cCBxdWVyaWVzIHJlY2VpdmVkCgkwIFYzIG1lbWJlcnNoaXAgcXVlcmllcyByZWNlaXZlZAoJMCBt ZW1iZXJzaGlwIHF1ZXJpZXMgcmVjZWl2ZWQgd2l0aCBpbnZhbGlkIGZpZWxkKHMpCgkwIGdlbmVy YWwgcXVlcmllcyByZWNlaXZlZAoJMCBncm91cCBxdWVyaWVzIHJlY2VpdmVkCgkwIGdyb3VwLXNv dXJjZSBxdWVyaWVzIHJlY2VpdmVkCgkwIGdyb3VwLXNvdXJjZSBxdWVyaWVzIGRyb3BwZWQKCTAg bWVtYmVyc2hpcCByZXBvcnRzIHJlY2VpdmVkCgkwIG1lbWJlcnNoaXAgcmVwb3J0cyByZWNlaXZl ZCB3aXRoIGludmFsaWQgZmllbGQocykKCTAgbWVtYmVyc2hpcCByZXBvcnRzIHJlY2VpdmVkIGZv ciBncm91cHMgdG8gd2hpY2ggd2UgYmVsb25nCgkwIFYzIHJlcG9ydHMgcmVjZWl2ZWQgd2l0aG91 dCBSb3V0ZXIgQWxlcnQKCTAgbWVtYmVyc2hpcCByZXBvcnRzIHNlbnQKYXJwOgoJMyBBUlAgcmVx dWVzdHMgc2VudAoJMCBBUlAgcmVwbGllcyBzZW50CgkwIEFSUCByZXF1ZXN0cyByZWNlaXZlZAoJ MiBBUlAgcmVwbGllcyByZWNlaXZlZAoJMiBBUlAgcGFja2V0cyByZWNlaXZlZAoJMCB0b3RhbCBw YWNrZXRzIGRyb3BwZWQgZHVlIHRvIG5vIEFSUCBlbnRyeQoJMCBBUlAgZW50cnlzIHRpbWVkIG91 dAoJMCBEdXBsaWNhdGUgSVBzIHNlZW4KaXA2OgoJMCB0b3RhbCBwYWNrZXRzIHJlY2VpdmVkCgkw IHdpdGggc2l6ZSBzbWFsbGVyIHRoYW4gbWluaW11bQoJMCB3aXRoIGRhdGEgc2l6ZSA8IGRhdGEg bGVuZ3RoCgkwIHdpdGggYmFkIG9wdGlvbnMKCTAgd2l0aCBpbmNvcnJlY3QgdmVyc2lvbiBudW1i ZXIKCTAgZnJhZ21lbnRzIHJlY2VpdmVkCgkwIGZyYWdtZW50cyBkcm9wcGVkIChkdXAgb3Igb3V0 IG9mIHNwYWNlKQoJMCBmcmFnbWVudHMgZHJvcHBlZCBhZnRlciB0aW1lb3V0CgkwIGZyYWdtZW50 cyB0aGF0IGV4Y2VlZGVkIGxpbWl0CgkwIHBhY2tldHMgcmVhc3NlbWJsZWQgb2sKCTAgcGFja2V0 cyBmb3IgdGhpcyBob3N0CgkwIHBhY2tldHMgZm9yd2FyZGVkCgkwIHBhY2tldHMgbm90IGZvcndh cmRhYmxlCgkwIHJlZGlyZWN0cyBzZW50CgkwIHBhY2tldHMgc2VudCBmcm9tIHRoaXMgaG9zdAoJ MCBwYWNrZXRzIHNlbnQgd2l0aCBmYWJyaWNhdGVkIGlwIGhlYWRlcgoJMCBvdXRwdXQgcGFja2V0 cyBkcm9wcGVkIGR1ZSB0byBubyBidWZzLCBldGMuCgkwIG91dHB1dCBwYWNrZXRzIGRpc2NhcmRl ZCBkdWUgdG8gbm8gcm91dGUKCTAgb3V0cHV0IGRhdGFncmFtcyBmcmFnbWVudGVkCgkwIGZyYWdt ZW50cyBjcmVhdGVkCgkwIGRhdGFncmFtcyB0aGF0IGNhbid0IGJlIGZyYWdtZW50ZWQKCTAgcGFj a2V0cyB0aGF0IHZpb2xhdGVkIHNjb3BlIHJ1bGVzCgkwIG11bHRpY2FzdCBwYWNrZXRzIHdoaWNo IHdlIGRvbid0IGpvaW4KCU1idWYgc3RhdGlzdGljczoKCQkwIG9uZSBtYnVmCgkJMCBvbmUgZXh0 IG1idWYKCQkwIHR3byBvciBtb3JlIGV4dCBtYnVmCgkwIHBhY2tldHMgd2hvc2UgaGVhZGVycyBh cmUgbm90IGNvbnRpbnVvdXMKCTAgdHVubmVsaW5nIHBhY2tldHMgdGhhdCBjYW4ndCBmaW5kIGdp ZgoJMCBwYWNrZXRzIGRpc2NhcmRlZCBiZWNhdXNlIG9mIHRvbyBtYW55IGhlYWRlcnMKCTAgZmFp bHVyZXMgb2Ygc291cmNlIGFkZHJlc3Mgc2VsZWN0aW9uCglTb3VyY2UgYWRkcmVzc2VzIHNlbGVj dGlvbiBydWxlIGFwcGxpZWQ6CmljbXA2OgoJMCBjYWxscyB0byBpY21wNl9lcnJvcgoJMCBlcnJv cnMgbm90IGdlbmVyYXRlZCBpbiByZXNwb25zZSB0byBhbiBpY21wNiBtZXNzYWdlCgkwIGVycm9y cyBub3QgZ2VuZXJhdGVkIGJlY2F1c2Ugb2YgcmF0ZSBsaW1pdGF0aW9uCgkwIG1lc3NhZ2VzIHdp dGggYmFkIGNvZGUgZmllbGRzCgkwIG1lc3NhZ2VzIDwgbWluaW11bSBsZW5ndGgKCTAgYmFkIGNo ZWNrc3VtcwoJMCBtZXNzYWdlcyB3aXRoIGJhZCBsZW5ndGgKCUhpc3RvZ3JhbSBvZiBlcnJvciBt ZXNzYWdlcyB0byBiZSBnZW5lcmF0ZWQ6CgkJMCBubyByb3V0ZQoJCTAgYWRtaW5pc3RyYXRpdmVs eSBwcm9oaWJpdGVkCgkJMCBiZXlvbmQgc2NvcGUKCQkwIGFkZHJlc3MgdW5yZWFjaGFibGUKCQkw IHBvcnQgdW5yZWFjaGFibGUKCQkwIHBhY2tldCB0b28gYmlnCgkJMCB0aW1lIGV4Y2VlZCB0cmFu c2l0CgkJMCB0aW1lIGV4Y2VlZCByZWFzc2VtYmx5CgkJMCBlcnJvbmVvdXMgaGVhZGVyIGZpZWxk CgkJMCB1bnJlY29nbml6ZWQgbmV4dCBoZWFkZXIKCQkwIHVucmVjb2duaXplZCBvcHRpb24KCQkw IHJlZGlyZWN0CgkJMCB1bmtub3duCgkwIG1lc3NhZ2UgcmVzcG9uc2VzIGdlbmVyYXRlZAoJMCBt ZXNzYWdlcyB3aXRoIHRvbyBtYW55IE5EIG9wdGlvbnMKCTAgbWVzc2FnZXMgd2l0aCBiYWQgTkQg b3B0aW9ucwoJMCBiYWQgbmVpZ2hib3Igc29saWNpdGF0aW9uIG1lc3NhZ2VzCgkwIGJhZCBuZWln aGJvciBhZHZlcnRpc2VtZW50IG1lc3NhZ2VzCgkwIGJhZCByb3V0ZXIgc29saWNpdGF0aW9uIG1l c3NhZ2VzCgkwIGJhZCByb3V0ZXIgYWR2ZXJ0aXNlbWVudCBtZXNzYWdlcwoJMCBiYWQgcmVkaXJl Y3QgbWVzc2FnZXMKCTAgcGF0aCBNVFUgY2hhbmdlcwpyaXA2OgoJMCBtZXNzYWdlcyByZWNlaXZl ZAoJMCBjaGVja3N1bSBjYWxjdWxhdGlvbnMgb24gaW5ib3VuZAoJMCBtZXNzYWdlcyB3aXRoIGJh ZCBjaGVja3N1bQoJMCBtZXNzYWdlcyBkcm9wcGVkIGR1ZSB0byBubyBzb2NrZXQKCTAgbXVsdGlj YXN0IG1lc3NhZ2VzIGRyb3BwZWQgZHVlIHRvIG5vIHNvY2tldAoJMCBtZXNzYWdlcyBkcm9wcGVk IGR1ZSB0byBmdWxsIHNvY2tldCBidWZmZXJzCgkwIGRlbGl2ZXJlZAoJMCBkYXRhZ3JhbXMgb3V0 cHV0CgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0KbmV0c3RhdCAtbQoKNjUvMjE5LzI4NCBtYnVmcyBpbiB1c2Ug KGN1cnJlbnQvY2FjaGUvdG90YWwpCjUwLzg0LzEzNC84NjQwIG1idWYgY2x1c3RlcnMgaW4gdXNl IChjdXJyZW50L2NhY2hlL3RvdGFsL21heCkKNjQvNzggbWJ1ZitjbHVzdGVycyBvdXQgb2YgcGFj a2V0IHNlY29uZGFyeSB6b25lIGluIHVzZSAoY3VycmVudC9jYWNoZSkKMC8yLzIvNDMyMCA0ayAo cGFnZSBzaXplKSBqdW1ibyBjbHVzdGVycyBpbiB1c2UgKGN1cnJlbnQvY2FjaGUvdG90YWwvbWF4 KQowLzAvMC82NDgwIDlrIGp1bWJvIGNsdXN0ZXJzIGluIHVzZSAoY3VycmVudC9jYWNoZS90b3Rh bC9tYXgpCjAvMC8wLzQzMjAgMTZrIGp1bWJvIGNsdXN0ZXJzIGluIHVzZSAoY3VycmVudC9jYWNo ZS90b3RhbC9tYXgpCjExNksvMjMwSy8zNDdLIGJ5dGVzIGFsbG9jYXRlZCB0byBuZXR3b3JrIChj dXJyZW50L2NhY2hlL3RvdGFsKQowLzAvMCByZXF1ZXN0cyBmb3IgbWJ1ZnMgZGVuaWVkIChtYnVm cy9jbHVzdGVycy9tYnVmK2NsdXN0ZXJzKQowLzAvMCByZXF1ZXN0cyBmb3IganVtYm8gY2x1c3Rl cnMgZGVuaWVkICg0ay85ay8xNmspCjAgcmVxdWVzdHMgZm9yIHNmYnVmcyBkZW5pZWQKMCByZXF1 ZXN0cyBmb3Igc2ZidWZzIGRlbGF5ZWQKMCByZXF1ZXN0cyBmb3IgSS9PIGluaXRpYXRlZCBieSBz ZW5kZmlsZQowIGNhbGxzIHRvIHByb3RvY29sIGRyYWluIHJvdXRpbmVzCgotLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0KbmV0c3RhdCAtaWQKCk5hbWUgICAgTXR1IE5ldHdvcmsgICAgICAgQWRkcmVzcyAgICAgICAg ICAgICAgSXBrdHMgSWVycnMgSWRyb3AgICAgT3BrdHMgT2VycnMgIENvbGwgRHJvcApyZTAgICAg MTUwMCA8TGluayMxPiAgICAgIDUyOjU0OjAwOjEyOjM0OjU2ICAgICAgMTgyICAgICAwICAgICAw ICAgICAgMTE4ICAgICAwICAgMTEzICAgIDAgCnJlMCAgICAxNTAwIDEwLjAuMi4wICAgICAgMTAu MC4yLjE1ICAgICAgICAgICAgICAxNjEgICAgIC0gICAgIC0gICAgICAxMTMgICAgIC0gICAgIC0g ICAgLSAKcGxpcDAgIDE1MDAgPExpbmsjMj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg MCAgICAgMCAgICAgMCAgICAgICAgMCAgICAgMCAgICAgMCAgICAwIApsbzAgICAxNjM4NCA8TGlu ayMzPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwICAgICAwICAgICAwICAgICAgICAw ICAgICAwICAgICAwICAgIDAgCmxvMCAgIDE2Mzg0IGZlODA6Mzo6MSAgICAgZmU4MDozOjoxICAg ICAgICAgICAgICAgIDAgICAgIC0gICAgIC0gICAgICAgIDAgICAgIC0gICAgIC0gICAgLSAKbG8w ICAgMTYzODQgbG9jYWxob3N0ICAgICA6OjEgICAgICAgICAgICAgICAgICAgICAgMCAgICAgLSAg ICAgLSAgICAgICAgMCAgICAgLSAgICAgLSAgICAtIApsbzAgICAxNjM4NCB5b3VyLW5ldCAgICAg IGxvY2FsaG9zdCAgICAgICAgICAgICAgICAwICAgICAtICAgICAtICAgICAgICAwICAgICAtICAg ICAtICAgIC0gCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KbmV0c3RhdCAtYW5yCgpSb3V0aW5nIHRhYmxlcwoK SW50ZXJuZXQ6CkRlc3RpbmF0aW9uICAgICAgICBHYXRld2F5ICAgICAgICAgICAgRmxhZ3MgICAg UmVmcyAgICAgIFVzZSAgTmV0aWYgRXhwaXJlCmRlZmF1bHQgICAgICAgICAgICAxMC4wLjIuMiAg ICAgICAgICAgVUdTICAgICAgICAgMCAgICAgICAgMCAgICByZTAKMTAuMC4yLjAvMjQgICAgICAg IGxpbmsjMSAgICAgICAgICAgICBVICAgICAgICAgICAyICAgICAgMTEzICAgIHJlMAoxMC4wLjIu MTUgICAgICAgICAgbGluayMxICAgICAgICAgICAgIFVIUyAgICAgICAgIDAgICAgICAgIDAgICAg bG8wCjEyNy4wLjAuMSAgICAgICAgICBsaW5rIzMgICAgICAgICAgICAgVUggICAgICAgICAgMCAg ICAgICAgMCAgICBsbzAKCkludGVybmV0NjoKRGVzdGluYXRpb24gICAgICAgICAgICAgICAgICAg ICAgIEdhdGV3YXkgICAgICAgICAgICAgICAgICAgICAgIEZsYWdzICAgICAgTmV0aWYgRXhwaXJl Cjo6MSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6OjEgICAgICAgICAgICAgICAgICAg ICAgICAgICBVSCAgICAgICAgICBsbzAKZmU4MDo6JWxvMC82NCAgICAgICAgICAgICAgICAgICAg IGxpbmsjMyAgICAgICAgICAgICAgICAgICAgICAgIFUgICAgICAgICAgIGxvMApmZTgwOjoxJWxv MCAgICAgICAgICAgICAgICAgICAgICAgbGluayMzICAgICAgICAgICAgICAgICAgICAgICAgVUhT ICAgICAgICAgbG8wCmZmMDE6Mzo6LzMyICAgICAgICAgICAgICAgICAgICAgICBmZTgwOjoxJWxv MCAgICAgICAgICAgICAgICAgICBVICAgICAgICAgICBsbzAKZmYwMjo6JWxvMC8zMiAgICAgICAg ICAgICAgICAgICAgIGZlODA6OjElbG8wICAgICAgICAgICAgICAgICAgIFUgICAgICAgICAgIGxv MAoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tCm5ldHN0YXQgLWFuQQoKQWN0aXZlIEludGVybmV0IGNvbm5lY3Rp b25zIChpbmNsdWRpbmcgc2VydmVycykKVGNwY2IgICAgUHJvdG8gUmVjdi1RIFNlbmQtUSAgTG9j YWwgQWRkcmVzcyAgICAgIEZvcmVpZ24gQWRkcmVzcyAgIChzdGF0ZSkKYzMwZGYwMDAgdGNwNCAg ICAgICAwICAgICAgMCAxMC4wLjIuMTUuMjA0OCAgICAgMTAuMC4yLjIuNDY0NDAgICAgIEVTVEFC TElTSEVECmMzMGUwMDAwIHRjcDQgICAgICAgMCAgICAgIDAgKi4yMDQ4ICAgICAgICAgICAgICou KiAgICAgICAgICAgICAgICBMSVNURU4KYzMwZGZjNTggdGNwNCAgICAgICAwICAgICAgMCAxMC4w LjIuMTUuMjIgICAgICAgMTAuMC4yLjIuNTM2MzcgICAgIEVTVEFCTElTSEVECmMzMGRmOWUwIHRj cDQgICAgICAgMCAgICAgIDAgKi4yMiAgICAgICAgICAgICAgICouKiAgICAgICAgICAgICAgICBM SVNURU4KYzMwZGY0ZjAgdGNwNiAgICAgICAwICAgICAgMCAqLjIyICAgICAgICAgICAgICAgKi4q ICAgICAgICAgICAgICAgIExJU1RFTgpjMzBkZjc2OCB0Y3A0ICAgICAgIDAgICAgICAwIDEyNy4w LjAuMS4yNSAgICAgICAqLiogICAgICAgICAgICAgICAgTElTVEVOCmMyZWQwMWI4IHVkcDQgICAg ICAgMCAgICAgIDAgKi41MTQgICAgICAgICAgICAgICouKiAgICAgICAgICAgICAgICAKYzJlZDAw ZGMgdWRwNiAgICAgICAwICAgICAgMCAqLjUxNCAgICAgICAgICAgICAgKi4qICAgICAgICAgICAg ICAgIApBY3RpdmUgVU5JWCBkb21haW4gc29ja2V0cwpBZGRyZXNzICBUeXBlICAgUmVjdi1RIFNl bmQtUSAgICBJbm9kZSAgICAgQ29ubiAgICAgUmVmcyAgTmV4dHJlZiBBZGRyCmMyZWQzZDcwIHN0 cmVhbSAgICAgIDAgICAgICAwIGMyZWMwYTc4ICAgICAgICAwICAgICAgICAwICAgICAgICAwIC92 YXIvcnVuL2RldmQucGlwZQpjMmVkMzU2MCBkZ3JhbSAgICAgICAwICAgICAgMCAgICAgICAgMCBj MmVkMzM1YyAgICAgICAgMCBjMmVkMzAwMApjMmVkMzAwMCBkZ3JhbSAgICAgICAwICAgICAgMCAg ICAgICAgMCBjMmVkMzM1YyAgICAgICAgMCBjMmVkMzE1OApjMmVkMzBhYyBkZ3JhbSAgICAgICAw ICAgICAgMCAgICAgICAgMCBjMmVkMzQwOCAgICAgICAgMCAgICAgICAgMApjMmVkMzE1OCBkZ3Jh bSAgICAgICAwICAgICAgMCAgICAgICAgMCBjMmVkMzM1YyAgICAgICAgMCAgICAgICAgMApjMmVk MzM1YyBkZ3JhbSAgICAgICAwICAgICAgMCBjMzBjNTk2YyAgICAgICAgMCBjMmVkMzU2MCAgICAg ICAgMCAvdmFyL3J1bi9sb2dwcml2CmMyZWQzNDA4IGRncmFtICAgICAgIDAgICAgICAwIGMzMGM1 YTc4ICAgICAgICAwIGMyZWQzMGFjICAgICAgICAwIC92YXIvcnVuL2xvZwoKLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tCm5ldHN0YXQgLWFMCgpDdXJyZW50IGxpc3RlbiBxdWV1ZSBzaXplcyAocWxlbi9pbmNxbGVu L21heHFsZW4pClByb3RvIExpc3RlbiAgICAgICAgIExvY2FsIEFkZHJlc3MgICAgICAgICAKdGNw NCAgMC8wLzEgICAgICAgICAgKi5kbHMtbW9uaXRvciAgICAgICAgICAKdGNwNCAgMC8wLzEyOCAg ICAgICAgKi5zc2ggICAgICAgICAgICAgICAgICAKdGNwNiAgMC8wLzEyOCAgICAgICAgKi5zc2gg ICAgICAgICAgICAgICAgICAKdGNwNCAgMC8wLzEwICAgICAgICAgbG9jYWxob3N0LnNtdHAgICAg ICAgICAKdW5peCAgMC8wLzQgICAgICAgICAgL3Zhci9ydW4vZGV2ZC5waXBlCgotLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0KZnN0YXQKClVTRVIgICAgIENNRCAgICAgICAgICBQSUQgICBGRCBNT1VOVCAgICAgIElO VU0gTU9ERSAgICAgICAgIFNafERWIFIvVwpyb290ICAgICBkdHJhY2UgICAgICAxMTM0IHJvb3Qg LyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBkdHJhY2UgICAg ICAxMTM0ICAgd2QgLyAgICAgICAgIDk0MjYwIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAg ICBkdHJhY2UgICAgICAxMTM0IHRleHQgL3VzciAgICAgNDAwMzg3IC1yLXhyLXhyLXggICAzMTU2 OCAgcgpyb290ICAgICBkdHJhY2UgICAgICAxMTM0ICAgIDAgL2RldiAgICAgICAgIDg2IGNydy0t dy0tLS0gICBwdHMvMCBydwpyb290ICAgICBkdHJhY2UgICAgICAxMTM0ICAgIDEgL2RldiAgICAg ICAgIDg2IGNydy0tdy0tLS0gICBwdHMvMCBydwpyb290ICAgICBkdHJhY2UgICAgICAxMTM0ICAg IDIgL2RldiAgICAgICAgIDg2IGNydy0tdy0tLS0gICBwdHMvMCBydwpyb290ICAgICBkdHJhY2Ug ICAgICAxMTM0ICAgIDggL2RldiAgICAgICAgICAzIGNydy0tLS0tLS0gIGR0cmFjZS9kdHJhY2Ug cncKcm9vdCAgICAgbmMgICAgICAgICAgMTEzMiByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14 ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbmMgICAgICAgICAgMTEzMiAgIHdkIC8gICAgICAgICA5 NDIxMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbmMgICAgICAgICAgMTEzMiB0ZXh0 IC91c3IgICAgICAyMzc4OSAtci14ci14ci14ICAgMjE2NDggIHIKcm9vdCAgICAgbmMgICAgICAg ICAgMTEzMiAgICAwIC9kZXYgICAgICAgICA1NSBjcnctLS0tLS0tICAgdHR5djAgcncKcm9vdCAg ICAgbmMgICAgICAgICAgMTEzMiAgICAxIC9kZXYgICAgICAgICA1NSBjcnctLS0tLS0tICAgdHR5 djAgcncKcm9vdCAgICAgbmMgICAgICAgICAgMTEzMiAgICAyIC9kZXYgICAgICAgICA1NSBjcnct LS0tLS0tICAgdHR5djAgcncKcm9vdCAgICAgbmMgICAgICAgICAgMTEzMiAgICAzKiBpbnRlcm5l dCBzdHJlYW0gdGNwIGMzMGUwMDAwCnJvb3QgICAgIG5jICAgICAgICAgIDExMzIgICAgNCogaW50 ZXJuZXQgc3RyZWFtIHRjcCBjMzBkZjAwMApyb290ICAgICBjc2ggICAgICAgICAxMTI4IHJvb3Qg LyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBjc2ggICAgICAg ICAxMTI4ICAgd2QgLyAgICAgICAgIDk0MjYwIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAg ICBjc2ggICAgICAgICAxMTI4IHRleHQgLyAgICAgICAgIDk0MjIwIC1yLXhyLXhyLXggIDMyNzc4 OCAgcgpyb290ICAgICBjc2ggICAgICAgICAxMTI4ICAgMTUgL2RldiAgICAgICAgIDg2IGNydy0t dy0tLS0gICBwdHMvMCBydwpyb290ICAgICBjc2ggICAgICAgICAxMTI4ICAgMTYgL2RldiAgICAg ICAgIDg2IGNydy0tdy0tLS0gICBwdHMvMCBydwpyb290ICAgICBjc2ggICAgICAgICAxMTI4ICAg MTcgL2RldiAgICAgICAgIDg2IGNydy0tdy0tLS0gICBwdHMvMCBydwpyb290ICAgICBjc2ggICAg ICAgICAxMTI4ICAgMTggL2RldiAgICAgICAgIDg2IGNydy0tdy0tLS0gICBwdHMvMCBydwpyb290 ICAgICBjc2ggICAgICAgICAxMTI4ICAgMTkgL2RldiAgICAgICAgIDg2IGNydy0tdy0tLS0gICBw dHMvMCBydwpyb290ICAgICBzc2hkICAgICAgICAxMTI1IHJvb3QgLyAgICAgICAgICAgICAyIGRy d3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBzc2hkICAgICAgICAxMTI1ICAgd2QgLyAgICAg ICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBzc2hkICAgICAgICAxMTI1 IHRleHQgL3VzciAgICAgNDAwMzkzIC1yLXhyLXhyLXggIDIzMzM4OCAgcgpyb290ICAgICBzc2hk ICAgICAgICAxMTI1ICAgIDAgL2RldiAgICAgICAgIDI4IGNydy1ydy1ydy0gICAgbnVsbCBydwpy b290ICAgICBzc2hkICAgICAgICAxMTI1ICAgIDEgL2RldiAgICAgICAgIDI4IGNydy1ydy1ydy0g ICAgbnVsbCBydwpyb290ICAgICBzc2hkICAgICAgICAxMTI1ICAgIDIgL2RldiAgICAgICAgIDI4 IGNydy1ydy1ydy0gICAgbnVsbCBydwpyb290ICAgICBzc2hkICAgICAgICAxMTI1ICAgIDMqIGlu dGVybmV0IHN0cmVhbSB0Y3AgYzMwZGZjNTgKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNSAgICA0 KiBwaXBlIGMyZTY2MTg4IDwtPiBjMmU2NjI0MCAgICAgIDAgcncKcm9vdCAgICAgc3NoZCAgICAg ICAgMTEyNSAgICA1KiBwaXBlIGMyZTY2MjQwIDwtPiBjMmU2NjE4OCAgICAgIDAgcncKcm9vdCAg ICAgc3NoZCAgICAgICAgMTEyNSAgICA2KiBwc2V1ZG8tdGVybWluYWwgbWFzdGVyICAgICAgcHRz LzAgcncKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNSAgICA4KiBwc2V1ZG8tdGVybWluYWwgbWFz dGVyICAgICAgcHRzLzAgcncKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNSAgICA5KiBwc2V1ZG8t dGVybWluYWwgbWFzdGVyICAgICAgcHRzLzAgcncKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNCBy b290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgc3NoZCAg ICAgICAgMTEyNCAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9v dCAgICAgc3NoZCAgICAgICAgMTEyNCB0ZXh0IC91c3IgICAgIDQwMDM5MyAtci14ci14ci14ICAy MzMzODggIHIKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNCAgICAwIC9kZXYgICAgICAgICAyOCBj cnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgc3NoZCAgICAgICAgMTEyNCAgICAxIC9kZXYg ICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgc3NoZCAgICAgICAgMTEy NCAgICAyIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgc3No ZCAgICAgICAgMTEyNCAgICAzKiBpbnRlcm5ldDYgc3RyZWFtIHRjcCBjMzBkZjRmMApyb290ICAg ICBzc2hkICAgICAgICAxMTI0ICAgIDQqIGludGVybmV0IHN0cmVhbSB0Y3AgYzMwZGY5ZTAKX2Ro Y3AgICAgZGhjbGllbnQgICAgMTA5NyByb290IC92YXIgICAgIDE2NDg2NCBkci14ci14ci14ICAg ICA1MTIgIHIKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5NyAgIHdkIC92YXIgICAgIDE2NDg2NCBk ci14ci14ci14ICAgICA1MTIgIHIKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5NyBqYWlsIC92YXIg ICAgIDE2NDg2NCBkci14ci14ci14ICAgICA1MTIgIHIKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5 NyB0ZXh0IC8gICAgICAgICAgICAyNiAtci14ci14ci14ICAgNzg2MDAgIHIKX2RoY3AgICAgZGhj bGllbnQgICAgMTA5NyAgICAwIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncK X2RoY3AgICAgZGhjbGllbnQgICAgMTA5NyAgICAxIC9kZXYgICAgICAgICAyOCBjcnctcnctcnct ICAgIG51bGwgcncKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5NyAgICAyIC9kZXYgICAgICAgICAy OCBjcnctcnctcnctICAgIG51bGwgcncKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5NyAgICAzKiBs b2NhbCBkZ3JhbSBjMmVkMzU2MCA8LT4gYzJlZDMzNWMKX2RoY3AgICAgZGhjbGllbnQgICAgMTA5 NyAgICA1KiByb3V0ZSByYXcgMCBjMmVjZjE5YwpfZGhjcCAgICBkaGNsaWVudCAgICAxMDk3ICAg IDYqIHBpcGUgYzJlNjcwYjggPC0+IGMyZTY3MDAwICAgICAgMCBydwpfZGhjcCAgICBkaGNsaWVu dCAgICAxMDk3ICAgIDcgL3ZhciAgICAgIDk0MjE4IC0tLS0tLS0tLS0gICAgIDcwOCAgdwpfZGhj cCAgICBkaGNsaWVudCAgICAxMDk3ICAgIDggL2RldiAgICAgICAgIDE0IGNydy0tLS0tLS0gICAg IGJwZiBydwpfZGhjcCAgICBkaGNsaWVudCAgICAxMDk3ICAgIDkqIGludGVybmV0IHJhdyBpcCBj MzBjMmU5Ywpyb290ICAgICBkaGNsaWVudCAgICAxMDgxIHJvb3QgLyAgICAgICAgICAgICAyIGRy d3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBkaGNsaWVudCAgICAxMDgxICAgd2QgLyAgICAg ICAgIDk0MjEyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBkaGNsaWVudCAgICAxMDgx IHRleHQgLyAgICAgICAgICAgIDI2IC1yLXhyLXhyLXggICA3ODYwMCAgcgpyb290ICAgICBkaGNs aWVudCAgICAxMDgxICAgIDAgL2RldiAgICAgICAgIDI4IGNydy1ydy1ydy0gICAgbnVsbCBydwpy b290ICAgICBkaGNsaWVudCAgICAxMDgxICAgIDEgL2RldiAgICAgICAgIDI4IGNydy1ydy1ydy0g ICAgbnVsbCBydwpyb290ICAgICBkaGNsaWVudCAgICAxMDgxICAgIDIgL2RldiAgICAgICAgIDI4 IGNydy1ydy1ydy0gICAgbnVsbCBydwpyb290ICAgICBkaGNsaWVudCAgICAxMDgxICAgIDMqIGxv Y2FsIGRncmFtIGMyZWQzNTYwIDwtPiBjMmVkMzM1Ywpyb290ICAgICBkaGNsaWVudCAgICAxMDgx ICAgIDUqIHBpcGUgYzJlNjcwMDAgPC0+IGMyZTY3MGI4ICAgICAgMCBydwpyb290ICAgICBjc2gg ICAgICAgICAxMDU2IHJvb3QgLyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpy b290ICAgICBjc2ggICAgICAgICAxMDU2ICAgd2QgLyAgICAgICAgIDk0MjEyIGRyd3hyLXhyLXgg ICAgIDUxMiAgcgpyb290ICAgICBjc2ggICAgICAgICAxMDU2IHRleHQgLyAgICAgICAgIDk0MjIw IC1yLXhyLXhyLXggIDMyNzc4OCAgcgpyb290ICAgICBjc2ggICAgICAgICAxMDU2ICAgMTUgL2Rl diAgICAgICAgIDU1IGNydy0tLS0tLS0gICB0dHl2MCBydwpyb290ICAgICBjc2ggICAgICAgICAx MDU2ICAgMTYgL2RldiAgICAgICAgIDU1IGNydy0tLS0tLS0gICB0dHl2MCBydwpyb290ICAgICBj c2ggICAgICAgICAxMDU2ICAgMTcgL2RldiAgICAgICAgIDU1IGNydy0tLS0tLS0gICB0dHl2MCBy dwpyb290ICAgICBjc2ggICAgICAgICAxMDU2ICAgMTggL2RldiAgICAgICAgIDU1IGNydy0tLS0t LS0gICB0dHl2MCBydwpyb290ICAgICBjc2ggICAgICAgICAxMDU2ICAgMTkgL2RldiAgICAgICAg IDU1IGNydy0tLS0tLS0gICB0dHl2MCBydwpyb290ICAgICBnZXR0eSAgICAgICAxMDU1IHJvb3Qg LyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBnZXR0eSAgICAg ICAxMDU1ICAgd2QgLyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAg ICBnZXR0eSAgICAgICAxMDU1IHRleHQgL3VzciAgICAgMTEzMDUwNiAtci14ci14ci14ICAgMjIw NjggIHIKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1NSAgICAwIC9kZXYgICAgICAgICA2MiBjcnct LS0tLS0tICAgdHR5djcgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1NSAgICAxIC9kZXYgICAg ICAgICA2MiBjcnctLS0tLS0tICAgdHR5djcgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1NSAg ICAyIC9kZXYgICAgICAgICA2MiBjcnctLS0tLS0tICAgdHR5djcgcncKcm9vdCAgICAgZ2V0dHkg ICAgICAgMTA1NCByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9v dCAgICAgZ2V0dHkgICAgICAgMTA1NCAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAg ICA1MTIgIHIKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1NCB0ZXh0IC91c3IgICAgIDExMzA1MDYg LXIteHIteHIteCAgIDIyMDY4ICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTQgICAgMCAvZGV2 ICAgICAgICAgNjEgY3J3LS0tLS0tLSAgIHR0eXY2IHJ3CnJvb3QgICAgIGdldHR5ICAgICAgIDEw NTQgICAgMSAvZGV2ICAgICAgICAgNjEgY3J3LS0tLS0tLSAgIHR0eXY2IHJ3CnJvb3QgICAgIGdl dHR5ICAgICAgIDEwNTQgICAgMiAvZGV2ICAgICAgICAgNjEgY3J3LS0tLS0tLSAgIHR0eXY2IHJ3 CnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTMgcm9vdCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIt eCAgICAgNTEyICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTMgICB3ZCAvICAgICAgICAgICAg IDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTMgdGV4dCAv dXNyICAgICAxMTMwNTA2IC1yLXhyLXhyLXggICAyMjA2OCAgcgpyb290ICAgICBnZXR0eSAgICAg ICAxMDUzICAgIDAgL2RldiAgICAgICAgIDYwIGNydy0tLS0tLS0gICB0dHl2NSBydwpyb290ICAg ICBnZXR0eSAgICAgICAxMDUzICAgIDEgL2RldiAgICAgICAgIDYwIGNydy0tLS0tLS0gICB0dHl2 NSBydwpyb290ICAgICBnZXR0eSAgICAgICAxMDUzICAgIDIgL2RldiAgICAgICAgIDYwIGNydy0t LS0tLS0gICB0dHl2NSBydwpyb290ICAgICBnZXR0eSAgICAgICAxMDUyIHJvb3QgLyAgICAgICAg ICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBnZXR0eSAgICAgICAxMDUyICAg d2QgLyAgICAgICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBnZXR0eSAg ICAgICAxMDUyIHRleHQgL3VzciAgICAgMTEzMDUwNiAtci14ci14ci14ICAgMjIwNjggIHIKcm9v dCAgICAgZ2V0dHkgICAgICAgMTA1MiAgICAwIC9kZXYgICAgICAgICA1OSBjcnctLS0tLS0tICAg dHR5djQgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1MiAgICAxIC9kZXYgICAgICAgICA1OSBj cnctLS0tLS0tICAgdHR5djQgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1MiAgICAyIC9kZXYg ICAgICAgICA1OSBjcnctLS0tLS0tICAgdHR5djQgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA1 MSByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgZ2V0 dHkgICAgICAgMTA1MSAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIK cm9vdCAgICAgZ2V0dHkgICAgICAgMTA1MSB0ZXh0IC91c3IgICAgIDExMzA1MDYgLXIteHIteHIt eCAgIDIyMDY4ICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTEgICAgMCAvZGV2ICAgICAgICAg NTggY3J3LS0tLS0tLSAgIHR0eXYzIHJ3CnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTEgICAgMSAv ZGV2ICAgICAgICAgNTggY3J3LS0tLS0tLSAgIHR0eXYzIHJ3CnJvb3QgICAgIGdldHR5ICAgICAg IDEwNTEgICAgMiAvZGV2ICAgICAgICAgNTggY3J3LS0tLS0tLSAgIHR0eXYzIHJ3CnJvb3QgICAg IGdldHR5ICAgICAgIDEwNTAgcm9vdCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEy ICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTAgICB3ZCAvICAgICAgICAgICAgIDIgZHJ3eHIt eHIteCAgICAgNTEyICByCnJvb3QgICAgIGdldHR5ICAgICAgIDEwNTAgdGV4dCAvdXNyICAgICAx MTMwNTA2IC1yLXhyLXhyLXggICAyMjA2OCAgcgpyb290ICAgICBnZXR0eSAgICAgICAxMDUwICAg IDAgL2RldiAgICAgICAgIDU3IGNydy0tLS0tLS0gICB0dHl2MiBydwpyb290ICAgICBnZXR0eSAg ICAgICAxMDUwICAgIDEgL2RldiAgICAgICAgIDU3IGNydy0tLS0tLS0gICB0dHl2MiBydwpyb290 ICAgICBnZXR0eSAgICAgICAxMDUwICAgIDIgL2RldiAgICAgICAgIDU3IGNydy0tLS0tLS0gICB0 dHl2MiBydwpyb290ICAgICBnZXR0eSAgICAgICAxMDQ5IHJvb3QgLyAgICAgICAgICAgICAyIGRy d3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBnZXR0eSAgICAgICAxMDQ5ICAgd2QgLyAgICAg ICAgICAgICAyIGRyd3hyLXhyLXggICAgIDUxMiAgcgpyb290ICAgICBnZXR0eSAgICAgICAxMDQ5 IHRleHQgL3VzciAgICAgMTEzMDUwNiAtci14ci14ci14ICAgMjIwNjggIHIKcm9vdCAgICAgZ2V0 dHkgICAgICAgMTA0OSAgICAwIC9kZXYgICAgICAgICA1NiBjcnctLS0tLS0tICAgdHR5djEgcncK cm9vdCAgICAgZ2V0dHkgICAgICAgMTA0OSAgICAxIC9kZXYgICAgICAgICA1NiBjcnctLS0tLS0t ICAgdHR5djEgcncKcm9vdCAgICAgZ2V0dHkgICAgICAgMTA0OSAgICAyIC9kZXYgICAgICAgICA1 NiBjcnctLS0tLS0tICAgdHR5djEgcncKcm9vdCAgICAgbG9naW4gICAgICAgMTA0OCByb290IC8g ICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbG9naW4gICAgICAg MTA0OCAgIHdkIC8gICAgICAgICA5NDIxMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAg bG9naW4gICAgICAgMTA0OCB0ZXh0IC91c3IgICAgICAyMzc2NyAtci1zci14ci14ICAgMjE5ODQg IHIKcm9vdCAgICAgbG9naW4gICAgICAgMTA0OCAgICAwIC9kZXYgICAgICAgICA1NSBjcnctLS0t LS0tICAgdHR5djAgcncKcm9vdCAgICAgbG9naW4gICAgICAgMTA0OCAgICAxIC9kZXYgICAgICAg ICA1NSBjcnctLS0tLS0tICAgdHR5djAgcncKcm9vdCAgICAgbG9naW4gICAgICAgMTA0OCAgICAy IC9kZXYgICAgICAgICA1NSBjcnctLS0tLS0tICAgdHR5djAgcncKcm9vdCAgICAgbG9naW4gICAg ICAgMTA0OCAgICAzKiBsb2NhbCBkZ3JhbSBjMmVkMzAwMCA8LT4gYzJlZDMzNWMKcm9vdCAgICAg c2xlZXAgICAgICAgMTA0NyByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIg IHIKcm9vdCAgICAgc2xlZXAgICAgICAgMTA0NyAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4ci14 ci14ICAgICA1MTIgIHIKcm9vdCAgICAgc2xlZXAgICAgICAgMTA0NyB0ZXh0IC8gICAgICAgICA5 NDI0OCAtci14ci14ci14ICAgIDQwNDQgIHIKcm9vdCAgICAgc2xlZXAgICAgICAgMTA0NyAgICAw IC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgIHIKcm9vdCAgICAgc2xlZXAgICAg ICAgMTA0NyAgICAxKiBwaXBlIGMyZTY2ODYwIDwtPiBjMmU2NjdhOCAgICAgIDAgcncKcm9vdCAg ICAgc2xlZXAgICAgICAgMTA0NyAgICAyKiBwaXBlIGMyZTY2ODYwIDwtPiBjMmU2NjdhOCAgICAg IDAgcncKcm9vdCAgICAgbG9nZ2VyICAgICAgMTA0NSByb290IC8gICAgICAgICAgICAgMiBkcnd4 ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbG9nZ2VyICAgICAgMTA0NSAgIHdkIC8gICAgICAg ICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbG9nZ2VyICAgICAgMTA0NSB0 ZXh0IC91c3IgICAgICAyMzc2NiAtci14ci14ci14ICAgMTA2MjAgIHIKcm9vdCAgICAgbG9nZ2Vy ICAgICAgMTA0NSAgICAwKiBwaXBlIGMyZTY2N2E4IDwtPiBjMmU2Njg2MCAgICAgIDAgcncKcm9v dCAgICAgbG9nZ2VyICAgICAgMTA0NSAgICAyIC0gICAgICAgICAtICAgICAgICAgYmFkICAgIC0K cm9vdCAgICAgc2ggICAgICAgICAgMTA0NCByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14 ICAgICA1MTIgIHIKcm9vdCAgICAgc2ggICAgICAgICAgMTA0NCAgIHdkIC8gICAgICAgICAgICAg MiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgc2ggICAgICAgICAgMTA0NCB0ZXh0IC8g ICAgICAgICA5NDI0NyAtci14ci14ci14ICAxMTgwMjggIHIKcm9vdCAgICAgc2ggICAgICAgICAg MTA0NCAgICAwIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgIHIKcm9vdCAgICAg c2ggICAgICAgICAgMTA0NCAgICAxKiBwaXBlIGMyZTY2ODYwIDwtPiBjMmU2NjdhOCAgICAgIDAg cncKcm9vdCAgICAgc2ggICAgICAgICAgMTA0NCAgICAyKiBwaXBlIGMyZTY2ODYwIDwtPiBjMmU2 NjdhOCAgICAgIDAgcncKcm9vdCAgICAgY3JvbiAgICAgICAgIDk3NiByb290IC8gICAgICAgICAg ICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgY3JvbiAgICAgICAgIDk3NiAgIHdk IC92YXIgICAgICA3MDY1NiBkcnd4ci14LS0tICAgICA1MTIgIHIKcm9vdCAgICAgY3JvbiAgICAg ICAgIDk3NiB0ZXh0IC91c3IgICAgIDQwMDQ0OSAtci14ci14ci14ICAgMzQ0MjAgIHIKcm9vdCAg ICAgY3JvbiAgICAgICAgIDk3NiAgICAwIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51 bGwgcncKcm9vdCAgICAgY3JvbiAgICAgICAgIDk3NiAgICAxIC9kZXYgICAgICAgICAyOCBjcnct cnctcnctICAgIG51bGwgcncKcm9vdCAgICAgY3JvbiAgICAgICAgIDk3NiAgICAyIC9kZXYgICAg ICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgY3JvbiAgICAgICAgIDk3NiAg ICAzIC92YXIgICAgIDQyMzk2MiAtcnctLS0tLS0tICAgICAgIDMgIHcKc21tc3AgICAgc2VuZG1h aWwgICAgIDk2OSByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKc21t c3AgICAgc2VuZG1haWwgICAgIDk2OSAgIHdkIC92YXIgICAgIDE0MTMxOCBkcnd4cnd4LS0tICAg ICA1MTIgIHIKc21tc3AgICAgc2VuZG1haWwgICAgIDk2OSB0ZXh0IC91c3IgICAgIDExMzA1NTQg LXIteHItc3IteCAgNjg2MjE2ICByCnNtbXNwICAgIHNlbmRtYWlsICAgICA5NjkgICAgMCAvZGV2 ICAgICAgICAgMjggY3J3LXJ3LXJ3LSAgICBudWxsICByCnNtbXNwICAgIHNlbmRtYWlsICAgICA5 NjkgICAgMSAvZGV2ICAgICAgICAgMjggY3J3LXJ3LXJ3LSAgICBudWxsICB3CnNtbXNwICAgIHNl bmRtYWlsICAgICA5NjkgICAgMiAvZGV2ICAgICAgICAgMjggY3J3LXJ3LXJ3LSAgICBudWxsICB3 CnNtbXNwICAgIHNlbmRtYWlsICAgICA5NjkgICAgMyogbG9jYWwgZGdyYW0gYzJlZDMwYWMgPC0+ IGMyZWQzNDA4CnNtbXNwICAgIHNlbmRtYWlsICAgICA5NjkgICAgNCAvdmFyICAgICAxNDEzMjAg LXJ3LS0tLS0tLSAgICAgIDQ5ICB3CnJvb3QgICAgIHNlbmRtYWlsICAgICA5NjUgcm9vdCAvICAg ICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAgIHNlbmRtYWlsICAgICA5 NjUgICB3ZCAvdmFyICAgICAxNDEzMTUgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAgIHNl bmRtYWlsICAgICA5NjUgdGV4dCAvdXNyICAgICAxMTMwNTU0IC1yLXhyLXNyLXggIDY4NjIxNiAg cgpyb290ICAgICBzZW5kbWFpbCAgICAgOTY1ICAgIDAgL2RldiAgICAgICAgIDI4IGNydy1ydy1y dy0gICAgbnVsbCAgcgpyb290ICAgICBzZW5kbWFpbCAgICAgOTY1ICAgIDEgL2RldiAgICAgICAg IDI4IGNydy1ydy1ydy0gICAgbnVsbCAgdwpyb290ICAgICBzZW5kbWFpbCAgICAgOTY1ICAgIDIg L2RldiAgICAgICAgIDI4IGNydy1ydy1ydy0gICAgbnVsbCAgdwpyb290ICAgICBzZW5kbWFpbCAg ICAgOTY1ICAgIDMqIGxvY2FsIGRncmFtIGMyZWQzMTU4IDwtPiBjMmVkMzM1Ywpyb290ICAgICBz ZW5kbWFpbCAgICAgOTY1ICAgIDQqIGludGVybmV0IHN0cmVhbSB0Y3AgYzMwZGY3NjgKcm9vdCAg ICAgc2VuZG1haWwgICAgIDk2NSAgICA1IC92YXIgICAgIDQyMzk2MSAtcnctLS0tLS0tICAgICAg NzggIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiByb290IC8gICAgICAgICAgICAgMiBkcnd4 ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIHdkIC8gICAgICAg ICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiB0 ZXh0IC91c3IgICAgIDQwMDYxMCAtci14ci14ci14ICAgMzYwMjggIHIKcm9vdCAgICAgc3lzbG9n ZCAgICAgIDY3MiAgICAwIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9v dCAgICAgc3lzbG9nZCAgICAgIDY3MiAgICAxIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAg IG51bGwgcncKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgICAyIC9kZXYgICAgICAgICAyOCBj cnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgICAzIC92YXIg ICAgIDQyMzk1MCAtcnctLS0tLS0tICAgICAgIDMgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3 MiAgICA0KiBsb2NhbCBkZ3JhbSBjMmVkMzQwOApyb290ICAgICBzeXNsb2dkICAgICAgNjcyICAg IDUqIGxvY2FsIGRncmFtIGMyZWQzMzVjCnJvb3QgICAgIHN5c2xvZ2QgICAgICA2NzIgICAgNiog aW50ZXJuZXQ2IGRncmFtIHVkcCBjMmVkMDBkYwpyb290ICAgICBzeXNsb2dkICAgICAgNjcyICAg IDcqIGludGVybmV0IGRncmFtIHVkcCBjMmVkMDFiOApyb290ICAgICBzeXNsb2dkICAgICAgNjcy ICAgIDggL2RldiAgICAgICAgIDM0IGNydy0tLS0tLS0gICAga2xvZyAgcgpyb290ICAgICBzeXNs b2dkICAgICAgNjcyICAgMTAgL2RldiAgICAgICAgIDEyIGNydy0tLS0tLS0gIGNvbnNvbGUgIHcK cm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDExIC92YXIgICAgIDQwMDM5OSAtcnctci0tci0t ICAgICA0NDkgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDEyIC92YXIgICAgIDQwMDM5 NiAtcnctLS0tLS0tICAgICAgNjQgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDEzIC92 YXIgICAgIDQwMDM4OSAtcnctLS0tLS0tICAgIDc4MzAgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAg IDY3MiAgIDE0IC92YXIgICAgIDQwMDM5MyAtcnctci0tLS0tICAgIDQ2MjQgIHcKcm9vdCAgICAg c3lzbG9nZCAgICAgIDY3MiAgIDE1IC92YXIgICAgIDQwMDM5MiAtcnctci0tci0tICAgICAgNjQg IHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDE2IC92YXIgICAgIDQwMDM5NyAtcnctLS0t LS0tICAgICAgNjQgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDE3IC92YXIgICAgIDQw MDM5MCAtcnctLS0tLS0tICAgIDUzODcgIHcKcm9vdCAgICAgc3lzbG9nZCAgICAgIDY3MiAgIDE4 IC92YXIgICAgIDQwMDM5MSAtcnctLS0tLS0tICAgICAgNjQgIHcKcm9vdCAgICAgc3lzbG9nZCAg ICAgIDY3MiAgIDE5IC92YXIgICAgIDQwMDM5NSAtcnctci0tLS0tICAgICAgNjQgIHcKcm9vdCAg ICAgZGV2ZCAgICAgICAgIDU0OCByb290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1 MTIgIHIKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4 ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCB0ZXh0IC8gICAgICAg ICAgICAyNCAtci14ci14ci14ICAzODI3ODggIHIKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCAg ICAwIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgZGV2ZCAg ICAgICAgIDU0OCAgICAxIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9v dCAgICAgZGV2ZCAgICAgICAgIDU0OCAgICAyIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAg IG51bGwgcncKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCAgICAzIC9kZXYgICAgICAgICAxMSBj cnctLS0tLS0tICBkZXZjdGwgIHIKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCAgICA0KiBsb2Nh bCBzdHJlYW0gYzJlZDNkNzAKcm9vdCAgICAgZGV2ZCAgICAgICAgIDU0OCAgICA1IC92YXIgICAg IDQyMzk0OSAtcnctLS0tLS0tICAgICAgIDMgIHcKcm9vdCAgICAgbW91c2VkICAgICAgIDUwNiBy b290IC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9vdCAgICAgbW91c2Vk ICAgICAgIDUwNiAgIHdkIC8gICAgICAgICAgICAgMiBkcnd4ci14ci14ICAgICA1MTIgIHIKcm9v dCAgICAgbW91c2VkICAgICAgIDUwNiB0ZXh0IC91c3IgICAgIDQwMDUyNSAtci14ci14ci14ICAg MzYyNTIgIHIKcm9vdCAgICAgbW91c2VkICAgICAgIDUwNiAgICAwIC9kZXYgICAgICAgICAyOCBj cnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgbW91c2VkICAgICAgIDUwNiAgICAxIC9kZXYg ICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgbW91c2VkICAgICAgIDUw NiAgICAyIC9kZXYgICAgICAgICAyOCBjcnctcnctcnctICAgIG51bGwgcncKcm9vdCAgICAgbW91 c2VkICAgICAgIDUwNiAgICAzIC9kZXYgICAgICAgICA5NCBjcnctci0tci0tICAgIHVtczAgcncK cm9vdCAgICAgbW91c2VkICAgICAgIDUwNiAgICA0KiBsb2NhbCBzdHJlYW0gYzJlZDNkNzAKcm9v dCAgICAgbW91c2VkICAgICAgIDUwNiAgICA1IC9kZXYgICAgICAgICA3MSBjcnctLS0tLS0tICBj b25zb2xlY3RsIHJ3CnJvb3QgICAgIG1vdXNlZCAgICAgICA1MDYgICAgNiAvdmFyICAgICA0MjM5 NDggLXJ3LS0tLS0tLSAgICAgICAzICB3CnJvb3QgICAgIGFkamtlcm50eiAgICAxNDIgcm9vdCAv ICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAgIGFkamtlcm50eiAg ICAxNDIgICB3ZCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAg IGFkamtlcm50eiAgICAxNDIgdGV4dCAvICAgICAgICAgICAgMTMgLXIteHIteHIteCAgICA3NjEy ICByCnJvb3QgICAgIGFkamtlcm50eiAgICAxNDIgICAgMCAvZGV2ICAgICAgICAgMjggY3J3LXJ3 LXJ3LSAgICBudWxsIHJ3CnJvb3QgICAgIGFkamtlcm50eiAgICAxNDIgICAgMSAvZGV2ICAgICAg ICAgMjggY3J3LXJ3LXJ3LSAgICBudWxsIHJ3CnJvb3QgICAgIGFkamtlcm50eiAgICAxNDIgICAg MiAvZGV2ICAgICAgICAgMjggY3J3LXJ3LXJ3LSAgICBudWxsIHJ3CnJvb3QgICAgIGluaXQgICAg ICAgICAgIDEgcm9vdCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3Qg ICAgIGluaXQgICAgICAgICAgIDEgICB3ZCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAg NTEyICByCnJvb3QgICAgIGluaXQgICAgICAgICAgIDEgdGV4dCAvICAgICAgICAgICAgNTAgLXIt eHIteHIteCAgNjY2NTUyICByCnJvb3QgICAgIGtlcm5lbCAgICAgICAgIDAgcm9vdCAvICAgICAg ICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCnJvb3QgICAgIGtlcm5lbCAgICAgICAgIDAg ICB3ZCAvICAgICAgICAgICAgIDIgZHJ3eHIteHIteCAgICAgNTEyICByCgotLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0KZG1lc2cKCkNvcHlyaWdodCAoYykgMTk5Mi0yMDExIFRoZSBGcmVlQlNEIFByb2plY3QuCkNv cHlyaWdodCAoYykgMTk3OSwgMTk4MCwgMTk4MywgMTk4NiwgMTk4OCwgMTk4OSwgMTk5MSwgMTk5 MiwgMTk5MywgMTk5NAoJVGhlIFJlZ2VudHMgb2YgdGhlIFVuaXZlcnNpdHkgb2YgQ2FsaWZvcm5p YS4gQWxsIHJpZ2h0cyByZXNlcnZlZC4KRnJlZUJTRCBpcyBhIHJlZ2lzdGVyZWQgdHJhZGVtYXJr IG9mIFRoZSBGcmVlQlNEIEZvdW5kYXRpb24uCkZyZWVCU0QgOC4yLVJFTEVBU0UgIzA6IEZyaSBK dW4gMjQgMDY6Mjg6MjkgUERUIDIwMTEKICAgIHJvb3RAZmI4MmkzODY6L3Vzci9vYmovdXNyL3Ny Yy9zeXMvTVlLRVJORUwgaTM4NgpUaW1lY291bnRlciAiaTgyNTQiIGZyZXF1ZW5jeSAxMTkzMTgy IEh6IHF1YWxpdHkgMApDUFU6IFFFTVUgVmlydHVhbCBDUFUgdmVyc2lvbiAwLjEyLjUgKDE4MzUu NTEtTUh6IDY4Ni1jbGFzcyBDUFUpCiAgT3JpZ2luID0gIkdlbnVpbmVJbnRlbCIgIElkID0gMHg2 MjMgIEZhbWlseSA9IDYgIE1vZGVsID0gMiAgU3RlcHBpbmcgPSAzCiAgRmVhdHVyZXM9MHg3ODNm YmZkPEZQVSxERSxQU0UsVFNDLE1TUixQQUUsTUNFLENYOCxBUElDLFNFUCxNVFJSLFBHRSxNQ0Es Q01PVixQQVQsUFNFMzYsTU1YLEZYU1IsU1NFLFNTRTI+CiAgRmVhdHVyZXMyPTB4ODAwMDIwMDE8 U1NFMyxDWDE2LDxiMzE+PgogIEFNRCBGZWF0dXJlcz0weDIwMTAwODAwPFNZU0NBTEwsTlgsTE0+ CiAgQU1EIEZlYXR1cmVzMj0weDE8TEFIRj4KcmVhbCBtZW1vcnkgID0gMjY4NDM1NDU2ICgyNTYg TUIpCmF2YWlsIG1lbW9yeSA9IDI0MzYxNzc5MiAoMjMyIE1CKQpBQ1BJIEFQSUMgVGFibGU6IDxC T0NIUyAgQlhQQ0FQSUM+CmlvYXBpYzA6IENoYW5naW5nIEFQSUMgSUQgdG8gMQppb2FwaWMwIDxW ZXJzaW9uIDEuMT4gaXJxcyAwLTIzIG9uIG1vdGhlcmJvYXJkCmtiZDEgYXQga2JkbXV4MAphY3Bp MDogPEJPQ0hTIEJYUENSU0RUPiBvbiBtb3RoZXJib2FyZAphY3BpMDogW0lUSFJFQURdCmFjcGkw OiBQb3dlciBCdXR0b24gKGZpeGVkKQpUaW1lY291bnRlciAiQUNQSS1zYWZlIiBmcmVxdWVuY3kg MzU3OTU0NSBIeiBxdWFsaXR5IDg1MAphY3BpX3RpbWVyMDogPDI0LWJpdCB0aW1lciBhdCAzLjU3 OTU0NU1Iej4gcG9ydCAweGIwMDgtMHhiMDBiIG9uIGFjcGkwCmNwdTA6IDxBQ1BJIENQVT4gb24g YWNwaTAKcGNpYjA6IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gcG9ydCAweGNmOC0weGNmZiBvbiBh Y3BpMApwY2kwOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMApwY2lfbGluazQ6IFVuYWJsZSB0byBy b3V0ZSBJUlFzOiBBRV9OT1RfRk9VTkQKaXNhYjA6IDxQQ0ktSVNBIGJyaWRnZT4gYXQgZGV2aWNl IDEuMCBvbiBwY2kwCmlzYTA6IDxJU0EgYnVzPiBvbiBpc2FiMAphdGFwY2kwOiA8SW50ZWwgUElJ WDMgV0RNQTIgY29udHJvbGxlcj4gcG9ydCAweDFmMC0weDFmNywweDNmNiwweDE3MC0weDE3Nyww eDM3NiwweGMwMDAtMHhjMDBmIGF0IGRldmljZSAxLjEgb24gcGNpMAphdGEwOiA8QVRBIGNoYW5u ZWwgMD4gb24gYXRhcGNpMAphdGEwOiBbSVRIUkVBRF0KYXRhMTogPEFUQSBjaGFubmVsIDE+IG9u IGF0YXBjaTAKYXRhMTogW0lUSFJFQURdCnVoY2kwOiA8SW50ZWwgODIzNzFTQiAoUElJWDMpIFVT QiBjb250cm9sbGVyPiBwb3J0IDB4YzAyMC0weGMwM2YgaXJxIDExIGF0IGRldmljZSAxLjIgb24g cGNpMAp1aGNpMDogW0lUSFJFQURdCnVzYnVzMDogY29udHJvbGxlciBkaWQgbm90IHN0b3AKdXNi dXMwOiA8SW50ZWwgODIzNzFTQiAoUElJWDMpIFVTQiBjb250cm9sbGVyPiBvbiB1aGNpMApwY2kw OiA8YnJpZGdlPiBhdCBkZXZpY2UgMS4zIChubyBkcml2ZXIgYXR0YWNoZWQpCnZnYXBjaTA6IDxW R0EtY29tcGF0aWJsZSBkaXNwbGF5PiBtZW0gMHhmMDAwMDAwMC0weGYxZmZmZmZmLDB4ZjIwMDAw MDAtMHhmMjAwMGZmZiBhdCBkZXZpY2UgMi4wIG9uIHBjaTAKcmUwOiA8UmVhbFRlayA4MTM5Qysg MTAvMTAwQmFzZVRYPiBwb3J0IDB4YzEwMC0weGMxZmYgbWVtIDB4ZjIwMjAwMDAtMHhmMjAyMDBm ZiBpcnEgMTEgYXQgZGV2aWNlIDMuMCBvbiBwY2kwCnJlMDogQ2hpcCByZXYuIDB4NzQ4MDAwMDAK cmUwOiBNQUMgcmV2LiAweDAwMDAwMDAwCm1paWJ1czA6IDxNSUkgYnVzPiBvbiByZTAKcmxwaHkw OiA8UmVhbFRlayBpbnRlcm5hbCBtZWRpYSBpbnRlcmZhY2U+IFBIWSAwIG9uIG1paWJ1czAKcmxw aHkwOiAgMTBiYXNlVCwgMTBiYXNlVC1GRFgsIDEwYmFzZVQtRkRYLWZsb3csIDEwMGJhc2VUWCwg MTAwYmFzZVRYLUZEWCwgMTAwYmFzZVRYLUZEWC1mbG93LCBhdXRvLCBhdXRvLWZsb3cKcmUwOiBF dGhlcm5ldCBhZGRyZXNzOiA1Mjo1NDowMDoxMjozNDo1NgpyZTA6IFtGSUxURVJdCmFjcGlfaHBl dDA6IDxIaWdoIFByZWNpc2lvbiBFdmVudCBUaW1lcj4gaW9tZW0gMHhmZWQwMDAwMC0weGZlZDAw M2ZmIG9uIGFjcGkwCmFjcGlfaHBldDA6IGludmFsaWQgcGVyaW9kCmRldmljZV9hdHRhY2g6IGFj cGlfaHBldDAgYXR0YWNoIHJldHVybmVkIDYKYXRydGMwOiA8QVQgcmVhbHRpbWUgY2xvY2s+IHBv cnQgMHg3MC0weDcxLDB4NzItMHg3NyBpcnEgOCBvbiBhY3BpMAphdGtiZGMwOiA8S2V5Ym9hcmQg Y29udHJvbGxlciAoaTgwNDIpPiBwb3J0IDB4NjAsMHg2NCBpcnEgMSBvbiBhY3BpMAphdGtiZDA6 IDxBVCBLZXlib2FyZD4gaXJxIDEgb24gYXRrYmRjMAprYmQwIGF0IGF0a2JkMAphdGtiZDA6IFtH SUFOVC1MT0NLRURdCmF0a2JkMDogW0lUSFJFQURdCnBzbTA6IDxQUy8yIE1vdXNlPiBpcnEgMTIg b24gYXRrYmRjMApwc20wOiBbR0lBTlQtTE9DS0VEXQpwc20wOiBbSVRIUkVBRF0KcHNtMDogbW9k ZWwgSW50ZWxsaU1vdXNlIEV4cGxvcmVyLCBkZXZpY2UgSUQgNApmZGMwOiA8ZmxvcHB5IGRyaXZl IGNvbnRyb2xsZXI+IHBvcnQgMHgzZjItMHgzZjUsMHgzZjcgaXJxIDYgZHJxIDIgb24gYWNwaTAK ZmRjMDogZG9lcyBub3QgcmVzcG9uZApkZXZpY2VfYXR0YWNoOiBmZGMwIGF0dGFjaCByZXR1cm5l ZCA2CnBwYzA6IDxQYXJhbGxlbCBwb3J0PiBwb3J0IDB4Mzc4LTB4MzdmIGlycSA3IG9uIGFjcGkw CnBwYzA6IEdlbmVyaWMgY2hpcHNldCAoTklCQkxFLW9ubHkpIGluIENPTVBBVElCTEUgbW9kZQpw cGMwOiBbSVRIUkVBRF0KcHBidXMwOiA8UGFyYWxsZWwgcG9ydCBidXM+IG9uIHBwYzAKcGxpcDA6 IDxQTElQIG5ldHdvcmsgaW50ZXJmYWNlPiBvbiBwcGJ1czAKcGxpcDA6IFtJVEhSRUFEXQpscHQw OiA8UHJpbnRlcj4gb24gcHBidXMwCmxwdDA6IFtJVEhSRUFEXQpscHQwOiBJbnRlcnJ1cHQtZHJp dmVuIHBvcnQKcHBpMDogPFBhcmFsbGVsIEkvTz4gb24gcHBidXMwCnVhcnQwOiA8Tm9uLXN0YW5k YXJkIG5zODI1MCBjbGFzcyBVQVJUIHdpdGggRklGT3M+IHBvcnQgMHgzZjgtMHgzZmYgaXJxIDQg ZmxhZ3MgMHgxMCBvbiBhY3BpMAp1YXJ0MDogW0ZJTFRFUl0KcG10aW1lcjAgb24gaXNhMApvcm0w OiA8SVNBIE9wdGlvbiBST00+IGF0IGlvbWVtIDB4YzkwMDAtMHhkMGZmZiBwbnBpZCBPUk0wMDAw IG9uIGlzYTAKc2MwOiA8U3lzdGVtIGNvbnNvbGU+IGF0IGZsYWdzIDB4MTAwIG9uIGlzYTAKc2Mw OiBWR0EgPDE2IHZpcnR1YWwgY29uc29sZXMsIGZsYWdzPTB4MzAwPgp2Z2EwOiA8R2VuZXJpYyBJ U0EgVkdBPiBhdCBwb3J0IDB4M2MwLTB4M2RmIGlvbWVtIDB4YTAwMDAtMHhiZmZmZiBvbiBpc2Ew ClRpbWVjb3VudGVyICJUU0MiIGZyZXF1ZW5jeSAxODM1NTA1ODE1IEh6IHF1YWxpdHkgODAwClRp bWVjb3VudGVycyB0aWNrIGV2ZXJ5IDEwLjAwMCBtc2VjCnVzYnVzMDogMTJNYnBzIEZ1bGwgU3Bl ZWQgVVNCIHYxLjAKYWQwOiAxNjM4NE1CIDxRRU1VIEhBUkRESVNLIDAuMTIuNT4gYXQgYXRhMC1t YXN0ZXIgV0RNQTIgCmFjZDA6IENEUk9NIDxRRU1VIERWRC1ST00vMC4xMi41PiBhdCBhdGExLW1h c3RlciBXRE1BMiAKUm9vdCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXMwCnVnZW4wLjE6IDxJbnRl bD4gYXQgdXNidXMwCnVodWIwOiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYg MS4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMAp1aHViMDogMiBwb3J0cyB3aXRoIDIgcmVtb3Zh YmxlLCBzZWxmIHBvd2VyZWQKUm9vdCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXMwCnVnZW4wLjI6 IDxRRU1VIDAuMTIuNT4gYXQgdXNidXMwCnVtczA6IDxFbmRwb2ludDEgSW50ZXJydXB0IFBpcGU+ IG9uIHVzYnVzMApSb290IG1vdW50IHdhaXRpbmcgZm9yOiB1c2J1czAKUm9vdCBtb3VudCB3YWl0 aW5nIGZvcjogdXNidXMwCnVtczA6IDMgYnV0dG9ucyBhbmQgW1pdIGNvb3JkaW5hdGVzIElEPTAK VHJ5aW5nIHRvIG1vdW50IHJvb3QgZnJvbSB1ZnM6L2Rldi9hZDBzMWEKU2V0dGluZyBob3N0dXVp ZDogMDg0MDZlZjUtOWUyMy0xMWUwLWI5ZTItNTI1NDAwMTIzNDU2LgpTZXR0aW5nIGhvc3RpZDog MHg0YmU4NThkNi4KRW50cm9weSBoYXJ2ZXN0aW5nOgogaW50ZXJydXB0cwogZXRoZXJuZXQKIHBv aW50X3RvX3BvaW50CiBraWNrc3RhcnQKLgpTdGFydGluZyBmaWxlIHN5c3RlbSBjaGVja3M6Ci9k ZXYvYWQwczFhOiBGSUxFIFNZU1RFTSBDTEVBTjsgU0tJUFBJTkcgQ0hFQ0tTCi9kZXYvYWQwczFh OiBjbGVhbiwgMzMzMjI4IGZyZWUgKDM2OTIgZnJhZ3MsIDQxMTkyIGJsb2NrcywgMC43JSBmcmFn bWVudGF0aW9uKQovZGV2L2FkMHMxZjogREVGRVIgRk9SIEJBQ0tHUk9VTkQgQ0hFQ0tJTkcKL2Rl di9hZDBzMWQ6IERFRkVSIEZPUiBCQUNLR1JPVU5EIENIRUNLSU5HCi9kZXYvYWQwczFlOiBGSUxF IFNZU1RFTSBDTEVBTjsgU0tJUFBJTkcgQ0hFQ0tTCi9kZXYvYWQwczFlOiBjbGVhbiwgNDc1MTUz IGZyZWUgKDQxIGZyYWdzLCA1OTM4OSBibG9ja3MsIDAuMCUgZnJhZ21lbnRhdGlvbikKTW91bnRp bmcgbG9jYWwgZmlsZSBzeXN0ZW1zOgpXQVJOSU5HOiAvdXNyIHdhcyBub3QgcHJvcGVybHkgZGlz bW91bnRlZApXQVJOSU5HOiAvdmFyIHdhcyBub3QgcHJvcGVybHkgZGlzbW91bnRlZAouClNldHRp bmcgaG9zdG5hbWU6IGZiODJpMzg2Ci4KU3RhcnRpbmcgTmV0d29yazogbG8wIHJlMC4KbG8wOiBm bGFncz04MDQ5PFVQLExPT1BCQUNLLFJVTk5JTkcsTVVMVElDQVNUPiBtZXRyaWMgMCBtdHUgMTYz ODQKCW9wdGlvbnM9MzxSWENTVU0sVFhDU1VNPgoJaW5ldDYgZmU4MDo6MSVsbzAgcHJlZml4bGVu IDY0IHNjb3BlaWQgMHgzIAoJaW5ldDYgOjoxIHByZWZpeGxlbiAxMjggCglpbmV0IDEyNy4wLjAu MSBuZXRtYXNrIDB4ZmYwMDAwMDAgCgluZDYgb3B0aW9ucz0zPFBFUkZPUk1OVUQsQUNDRVBUX1JU QURWPgpyZTA6IGZsYWdzPTg4NDM8VVAsQlJPQURDQVNULFJVTk5JTkcsU0lNUExFWCxNVUxUSUNB U1Q+IG1ldHJpYyAwIG10dSAxNTAwCglvcHRpb25zPTliPFJYQ1NVTSxUWENTVU0sVkxBTl9NVFUs VkxBTl9IV1RBR0dJTkcsVkxBTl9IV0NTVU0+CglldGhlciA1Mjo1NDowMDoxMjozNDo1NgoJbWVk aWE6IEV0aGVybmV0IGF1dG9zZWxlY3QgKDEwMGJhc2VUWCA8ZnVsbC1kdXBsZXg+KQoJc3RhdHVz OiBhY3RpdmUKU3RhcnRpbmcgZGV2ZC4KU3RhcnRpbmcgdW1zMCBtb3VzZWQKLgpESENQUkVRVUVT VCBvbiByZTAgdG8gMjU1LjI1NS4yNTUuMjU1IHBvcnQgNjcKClNjcmlwdCAvZXRjL3JjLmQvZGV2 ZCBpbnRlcnJ1cHRlZApXYWl0aW5nIDMwcyBmb3IgdGhlIGRlZmF1bHQgcm91dGUgaW50ZXJmYWNl OiAKU2NyaXB0IC9ldGMvcmMuZC9kZWZhdWx0cm91dGUgaW50ZXJydXB0ZWQKQ3JlYXRpbmcgYW5k L29yIHRyaW1taW5nIGxvZyBmaWxlcwouClN0YXJ0aW5nIHN5c2xvZ2QuCk5vIGNvcmUgZHVtcHMg Zm91bmQuCkVMRiBsZGNvbmZpZyBwYXRoOiAvbGliIC91c3IvbGliIC91c3IvbGliL2NvbXBhdCAv dXNyL2xvY2FsL2xpYgphLm91dCBsZGNvbmZpZyBwYXRoOiAvdXNyL2xpYi9hb3V0IC91c3IvbGli L2NvbXBhdC9hb3V0CkNsZWFyaW5nIC90bXAgKFggcmVsYXRlZCkuClJlY292ZXJpbmcgdmkgZWRp dG9yIHNlc3Npb25zOgouClVwZGF0aW5nIG1vdGQ6Ci4KQ29uZmlndXJpbmcgc3lzY29uczoKIGJs YW5rdGltZQouClN0YXJ0aW5nIHNzaGQuClN0YXJ0aW5nIGNyb24uClN0YXJ0aW5nIGJhY2tncm91 bmQgZmlsZSBzeXN0ZW0gY2hlY2tzIGluIDYwIHNlY29uZHMuCgpUdWUgSnVsIDI2IDAwOjU5OjUy IFBEVCAyMDExCkp1bCAyNiAwMDo1OTo1NyBmYjgyaTM4NiBsb2dpbjogUk9PVCBMT0dJTiAocm9v dCkgT04gdHR5djAKSnVsIDI2IDAxOjAxOjIxIGZiODJpMzg2IHNzaGRbMTA5OV06IGVycm9yOiBQ QU06IGF1dGhlbnRpY2F0aW9uIGVycm9yIGZvciByb290IGZyb20gMTAuMC4yLjIKa2VybmVsIHRy YXAgMTIgd2l0aCBpbnRlcnJ1cHRzIGRpc2FibGVkCgoKRmF0YWwgdHJhcCAxMjogcGFnZSBmYXVs dCB3aGlsZSBpbiBrZXJuZWwgbW9kZQpjcHVpZCA9IDA7IGFwaWMgaWQgPSAwMApmYXVsdCB2aXJ0 dWFsIGFkZHJlc3MJPSAweDEwOApmYXVsdCBjb2RlCQk9IHN1cGVydmlzb3Igd3JpdGUsIHBhZ2Ug bm90IHByZXNlbnQKaW5zdHJ1Y3Rpb24gcG9pbnRlcgk9IDB4MjA6MHhjMTEwMTJkNwpzdGFjayBw b2ludGVyCSAgICAgICAgPSAweDI4OjB4Y2QzZWQ5ZjQKZnJhbWUgcG9pbnRlcgkgICAgICAgID0g MHgyODoweGNkM2VkYTBjCmNvZGUgc2VnbWVudAkJPSBiYXNlIDB4MCwgbGltaXQgMHhmZmZmZiwg dHlwZSAweDFiCgkJCT0gRFBMIDAsIHByZXMgMSwgZGVmMzIgMSwgZ3JhbiAxCnByb2Nlc3NvciBl ZmxhZ3MJPSByZXN1bWUsIElPUEwgPSAwCmN1cnJlbnQgcHJvY2VzcwkJPSAxMTMyIChuYykKdHJh cCBudW1iZXIJCT0gMTIKcGFuaWM6IHBhZ2UgZmF1bHQKY3B1aWQgPSAwCktEQjogc3RhY2sgYmFj a3RyYWNlOgojMCAweGMwOTAzNmE3IGF0IGtkYl9iYWNrdHJhY2UrMHg0NwojMSAweGMwOGQxYTA3 IGF0IHBhbmljKzB4MTE3CiMyIDB4YzBjMTU4YzMgYXQgdHJhcF9mYXRhbCsweDMyMwojMyAweGMw YzE1YmMwIGF0IHRyYXBfcGZhdWx0KzB4MmYwCiM0IDB4YzBjMTYxMmEgYXQgdHJhcCsweDQ4YQoj NSAweGMwYmZjOTdjIGF0IGNhbGx0cmFwKzB4NgojNiAweGMxMGU5OTJiIGF0IGR0cmFjZV9wYW5p YysweDFiCiM3IDB4YzEwZTk5NWQgYXQgZHRyYWNlX2Fzc2ZhaWwrMHgyZAojOCAweGMxMGZiMjhj IGF0IGR0cmFjZV9wcm9iZSsweDEzNWMKIzkgMHhjMTIzN2Y3MiBhdCBzeXN0cmFjZV9wcm9iZSsw eDYyCiMxMCAweGMwOTBmNjNmIGF0IHN5c2NhbGxlbnRlcisweDQ3ZgojMTEgMHhjMGMxNWMxNCBh dCBzeXNjYWxsKzB4MzQKIzEyIDB4YzBiZmNhMTEgYXQgWGludDB4ODBfc3lzY2FsbCsweDIxClVw dGltZTogM20wcwpQaHlzaWNhbCBtZW1vcnk6IDIzOSBNQgpEdW1waW5nIDc5IE1COiA2NCA0OCAz MiAxNgoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCmtlcm5lbCBjb25maWcKCm9wdGlvbnMJQ09ORklHX0FVVE9H RU5FUkFURUQKaWRlbnQJR0VORVJJQwptYWNoaW5lCWkzODYKY3B1CUk2ODZfQ1BVCmNwdQlJNTg2 X0NQVQpjcHUJSTQ4Nl9DUFUKbWFrZW9wdGlvbnMJREVCVUc9LWcKb3B0aW9ucwlVU0JfREVCVUcK b3B0aW9ucwlBSF9TVVBQT1JUX0FSNTQxNgpvcHRpb25zCUlFRUU4MDIxMV9TVVBQT1JUX01FU0gK b3B0aW9ucwlJRUVFODAyMTFfQU1QRFVfQUdFCm9wdGlvbnMJSUVFRTgwMjExX0RFQlVHCm9wdGlv bnMJQUhEX1JFR19QUkVUVFlfUFJJTlQKb3B0aW9ucwlBSENfUkVHX1BSRVRUWV9QUklOVApvcHRp b25zCUFUQV9TVEFUSUNfSUQKb3B0aW9ucwlTTVAKb3B0aW9ucwlLREJfVFJBQ0UKb3B0aW9ucwlL REIKb3B0aW9ucwlJTkNMVURFX0NPTkZJR19GSUxFCm9wdGlvbnMJRERCX0NURgpvcHRpb25zCUtE VFJBQ0VfSE9PS1MKb3B0aW9ucwlGTE9XVEFCTEUKb3B0aW9ucwlNQUMKb3B0aW9ucwlBVURJVApv cHRpb25zCUhXUE1DX0hPT0tTCm9wdGlvbnMJS0JEX0lOU1RBTExfQ0RFVgpvcHRpb25zCVBSSU5U Rl9CVUZSX1NJWkU9MTI4Cm9wdGlvbnMJX0tQT1NJWF9QUklPUklUWV9TQ0hFRFVMSU5HCm9wdGlv bnMJUDEwMDNfMUJfU0VNQVBIT1JFUwpvcHRpb25zCVNZU1ZTRU0Kb3B0aW9ucwlTWVNWTVNHCm9w dGlvbnMJU1lTVlNITQpvcHRpb25zCVNUQUNLCm9wdGlvbnMJS1RSQUNFCm9wdGlvbnMJU0NTSV9E RUxBWT01MDAwCm9wdGlvbnMJQ09NUEFUX0ZSRUVCU0Q3Cm9wdGlvbnMJQ09NUEFUX0ZSRUVCU0Q2 Cm9wdGlvbnMJQ09NUEFUX0ZSRUVCU0Q1Cm9wdGlvbnMJQ09NUEFUX0ZSRUVCU0Q0Cm9wdGlvbnMJ Q09NUEFUXzQzVFRZCm9wdGlvbnMJR0VPTV9MQUJFTApvcHRpb25zCUdFT01fUEFSVF9HUFQKb3B0 aW9ucwlQU0VVRE9GUwpvcHRpb25zCVBST0NGUwpvcHRpb25zCUNEOTY2MApvcHRpb25zCU1TRE9T RlMKb3B0aW9ucwlORlNfUk9PVApvcHRpb25zCU5GU0xPQ0tECm9wdGlvbnMJTkZTU0VSVkVSCm9w dGlvbnMJTkZTQ0xJRU5UCm9wdGlvbnMJTURfUk9PVApvcHRpb25zCVVGU19HSk9VUk5BTApvcHRp b25zCVVGU19ESVJIQVNICm9wdGlvbnMJVUZTX0FDTApvcHRpb25zCVNPRlRVUERBVEVTCm9wdGlv bnMJRkZTCm9wdGlvbnMJU0NUUApvcHRpb25zCUlORVQ2Cm9wdGlvbnMJSU5FVApvcHRpb25zCVBS RUVNUFRJT04Kb3B0aW9ucwlTQ0hFRF9VTEUKb3B0aW9ucwlOQVRJVkUKb3B0aW9ucwlHRU9NX1BB UlRfTUJSCm9wdGlvbnMJR0VPTV9QQVJUX0VCUl9DT01QQVQKb3B0aW9ucwlHRU9NX1BBUlRfRUJS Cm9wdGlvbnMJR0VPTV9QQVJUX0JTRApvcHRpb25zCUlTQVBOUApkZXZpY2UJaXNhCmRldmljZQlu cHgKZGV2aWNlCW1lbQpkZXZpY2UJaW8KZGV2aWNlCXVhcnRfbnM4MjUwCmRldmljZQlhdHBpYwpk ZXZpY2UJYXBpYwpkZXZpY2UJY3B1ZnJlcQpkZXZpY2UJYWNwaQpkZXZpY2UJZWlzYQpkZXZpY2UJ cGNpCmRldmljZQlmZGMKZGV2aWNlCWF0YQpkZXZpY2UJYXRhZGlzawpkZXZpY2UJYXRhcmFpZApk ZXZpY2UJYXRhcGljZApkZXZpY2UJYXRhcGlmZApkZXZpY2UJYXRhcGlzdApkZXZpY2UJYWhiCmRl dmljZQlhaGMKZGV2aWNlCWFoZApkZXZpY2UJYW1kCmRldmljZQlocHRpb3AKZGV2aWNlCWlzcApk ZXZpY2UJbXB0CmRldmljZQlzeW0KZGV2aWNlCXRybQpkZXZpY2UJYWR2CmRldmljZQlhZHcKZGV2 aWNlCWFoYQpkZXZpY2UJYWljCmRldmljZQlidApkZXZpY2UJbmN2CmRldmljZQluc3AKZGV2aWNl CXN0ZwpkZXZpY2UJc2NidXMKZGV2aWNlCWNoCmRldmljZQlkYQpkZXZpY2UJc2EKZGV2aWNlCWNk CmRldmljZQlwYXNzCmRldmljZQlzZXMKZGV2aWNlCWFtcgpkZXZpY2UJYXJjbXNyCmRldmljZQlh c3IKZGV2aWNlCWNpc3MKZGV2aWNlCWRwdApkZXZpY2UJaHB0bXYKZGV2aWNlCWhwdHJyCmRldmlj ZQlpaXIKZGV2aWNlCWlwcwpkZXZpY2UJbWx5CmRldmljZQl0d2EKZGV2aWNlCWFhYwpkZXZpY2UJ YWFjcApkZXZpY2UJaWRhCmRldmljZQltZmkKZGV2aWNlCW1seApkZXZpY2UJcHN0CmRldmljZQl0 d2UKZGV2aWNlCWF0a2JkYwpkZXZpY2UJYXRrYmQKZGV2aWNlCXBzbQpkZXZpY2UJa2JkbXV4CmRl dmljZQl2Z2EKZGV2aWNlCXNwbGFzaApkZXZpY2UJc2MKZGV2aWNlCWFncApkZXZpY2UJcG10aW1l cgpkZXZpY2UJY2JiCmRldmljZQlwY2NhcmQKZGV2aWNlCWNhcmRidXMKZGV2aWNlCXVhcnQKZGV2 aWNlCXBwYwpkZXZpY2UJcHBidXMKZGV2aWNlCWxwdApkZXZpY2UJcGxpcApkZXZpY2UJcHBpCmRl dmljZQlkZQpkZXZpY2UJZW0KZGV2aWNlCWlnYgpkZXZpY2UJaXhnYgpkZXZpY2UJbGUKZGV2aWNl CXRpCmRldmljZQl0eHAKZGV2aWNlCXZ4CmRldmljZQltaWlidXMKZGV2aWNlCWFlCmRldmljZQlh Z2UKZGV2aWNlCWFsYwpkZXZpY2UJYWxlCmRldmljZQliY2UKZGV2aWNlCWJmZQpkZXZpY2UJYmdl CmRldmljZQlkYwpkZXZpY2UJZXQKZGV2aWNlCWZ4cApkZXZpY2UJam1lCmRldmljZQlsZ2UKZGV2 aWNlCW1zawpkZXZpY2UJbmZlCmRldmljZQluZ2UKZGV2aWNlCXBjbgpkZXZpY2UJcmUKZGV2aWNl CXJsCmRldmljZQlzZgpkZXZpY2UJc2dlCmRldmljZQlzaXMKZGV2aWNlCXNrCmRldmljZQlzdGUK ZGV2aWNlCXN0Z2UKZGV2aWNlCXRsCmRldmljZQl0eApkZXZpY2UJdmdlCmRldmljZQl2cgpkZXZp Y2UJd2IKZGV2aWNlCXhsCmRldmljZQljcwpkZXZpY2UJZWQKZGV2aWNlCWV4CmRldmljZQllcApk ZXZpY2UJZmUKZGV2aWNlCWllCmRldmljZQlzbgpkZXZpY2UJeGUKZGV2aWNlCXdsYW4KZGV2aWNl CXdsYW5fd2VwCmRldmljZQl3bGFuX2NjbXAKZGV2aWNlCXdsYW5fdGtpcApkZXZpY2UJd2xhbl9h bXJyCmRldmljZQlhbgpkZXZpY2UJYXRoCmRldmljZQlhdGhfaGFsCmRldmljZQlhdGhfcmF0ZV9z YW1wbGUKZGV2aWNlCXJhbApkZXZpY2UJd2kKZGV2aWNlCWxvb3AKZGV2aWNlCXJhbmRvbQpkZXZp Y2UJZXRoZXIKZGV2aWNlCXZsYW4KZGV2aWNlCXR1bgpkZXZpY2UJcHR5CmRldmljZQltZApkZXZp Y2UJZ2lmCmRldmljZQlmYWl0aApkZXZpY2UJZmlybXdhcmUKZGV2aWNlCWJwZgpkZXZpY2UJdWhj aQpkZXZpY2UJb2hjaQpkZXZpY2UJZWhjaQpkZXZpY2UJdXNiCmRldmljZQl1aGlkCmRldmljZQl1 a2JkCmRldmljZQl1bHB0CmRldmljZQl1bWFzcwpkZXZpY2UJdW1zCmRldmljZQl1cmlvCmRldmlj ZQl1M2cKZGV2aWNlCXVhcmsKZGV2aWNlCXVic2EKZGV2aWNlCXVmdGRpCmRldmljZQl1aXBhcQpk ZXZpY2UJdXBsY29tCmRldmljZQl1c2xjb20KZGV2aWNlCXV2aXNvcgpkZXZpY2UJdXZzY29tCmRl dmljZQlhdWUKZGV2aWNlCWF4ZQpkZXZpY2UJY2RjZQpkZXZpY2UJY3VlCmRldmljZQlrdWUKZGV2 aWNlCXJ1ZQpkZXZpY2UJdWRhdgpkZXZpY2UJcnVtCmRldmljZQl1YXRoCmRldmljZQl1cmFsCmRl dmljZQl6eWQKZGV2aWNlCWZpcmV3aXJlCmRldmljZQlmd2UKZGV2aWNlCWZ3aXAKZGV2aWNlCWRj b25zCmRldmljZQlkY29uc19jcm9tCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KZGRiIGNhcHR1cmUgYnVmZmVy CgpkZGI6IGRkYl9jYXB0dXJlOiBrdm1fbmxpc3QK --00163630f0b194bd1104a8eebdcf Content-Type: application/octet-stream; name="info.0" Content-Disposition: attachment; filename="info.0" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gqk6cn231 RHVtcCBoZWFkZXIgZnJvbSBkZXZpY2UgL2Rldi9hZDBzMWIKICBBcmNoaXRlY3R1cmU6IGkzODYK ICBBcmNoaXRlY3R1cmUgVmVyc2lvbjogMgogIER1bXAgTGVuZ3RoOiA4MzYwMzQ1NkIgKDc5IE1C KQogIEJsb2Nrc2l6ZTogNTEyCiAgRHVtcHRpbWU6IFR1ZSBKdWwgMjYgMDE6MDI6MjggMjAxMQog IEhvc3RuYW1lOiBmYjgyaTM4NgogIE1hZ2ljOiBGcmVlQlNEIEtlcm5lbCBEdW1wCiAgVmVyc2lv biBTdHJpbmc6IEZyZWVCU0QgOC4yLVJFTEVBU0UgIzA6IEZyaSBKdW4gMjQgMDY6Mjg6MjkgUERU IDIwMTEKICAgIHJvb3RAZmI4MmkzODY6L3Vzci9vYmovdXNyL3NyYy9zeXMvTVlLRVJORUwKICBQ YW5pYyBTdHJpbmc6IHBhZ2UgZmF1bHQKICBEdW1wIFBhcml0eTogMzY1MDkxNjE1MQogIEJvdW5k czogMAogIER1bXAgU3RhdHVzOiBnb29kCg== --00163630f0b194bd1104a8eebdcf-- From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 03:08:02 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98255106566B for ; Tue, 26 Jul 2011 03:08:02 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id 569788FC15 for ; Tue, 26 Jul 2011 03:08:02 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id p6Q2X5ss062842; Mon, 25 Jul 2011 20:33:05 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id p6Q2X4xf062841; Mon, 25 Jul 2011 20:33:04 -0600 (MDT) (envelope-from ken) Date: Mon, 25 Jul 2011 20:33:04 -0600 From: "Kenneth D. Merry" To: Freddie Cash , Kurt Jaeger , freebsd-stable@freebsd.org Message-ID: <20110726023304.GA62385@nargothrond.kdm.org> References: <20110725093432.GI1202@home.opsec.eu> <20110725212808.GF48651@emmi.physik-pool.tu-berlin.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110725212808.GF48651@emmi.physik-pool.tu-berlin.de> User-Agent: Mutt/1.4.2i Cc: Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 03:08:02 -0000 On Mon, Jul 25, 2011 at 23:28:08 +0200, Leon Me?ner wrote: > On Mon, Jul 25, 2011 at 01:42:07PM -0700, Freddie Cash wrote: > > On Mon, Jul 25, 2011 at 2:34 AM, Kurt Jaeger wrote: > > > > > What kind of SATA 6g 4-port non-RAID controller is currently suggested > > > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > > > > > > > SuperMicro AOC-USAS2-L8i > > > > PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 > > SAS/SATA ports. Supports two firmwares: > > IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no > > RAID5+) > > IT mode enables straight HBA mode > > > > Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been > > MFC'd into 8.2-STABLE. > > Yes, it's in -STABLE. Does anyone know if mps is going to support > IR-Firmware in the near future ? LSI has a driver that supports it, they're just trying to get it approved by their legal folks so they can release it. Unfortunately there is no way to know how long that will take. Ken -- Kenneth Merry ken@FreeBSD.org From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 05:54:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9E5E106566C; Tue, 26 Jul 2011 05:54:37 +0000 (UTC) (envelope-from eugene@imedia.ru) Received: from lynx.imedia.ru (lynx.imedia.ru [83.242.167.254]) by mx1.freebsd.org (Postfix) with ESMTP id 2F8878FC0C; Tue, 26 Jul 2011 05:54:36 +0000 (UTC) X-All-Recipients: DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=imedia.ru; s=common; t=1311659674; bh=k8WSZ+kNnFBjVFiglkydPQqR0Uz8ssdP1uHa2E3zQ/M=; h=From:Reply-To:To:Subject:Date:Cc:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id; b=m4o6TV+yYasNLZQpNUp7hWZ9snZOeRH4SeidgAqzzp3ub4yjGl6FDmzID/Om5jB8T OPNLFu9tqV7TZDVwdeIr6TZ4Z9GXigEVBWUN869y2Y2OOiCWO9Vc4XQejqfTdskNQv 1J3uxYyTOr7It1Eeuqs3zFASgF3dPuA+Y0D4R4H8= Received: from badger.imedia.ru (root@badger.imedia.ru [10.167.1.243]) by lynx.imedia.ru (8.14.3/8.14.3/TWINS7_LDAP) with ESMTP id p6Q5sYCg086167; Tue, 26 Jul 2011 09:54:34 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from badger.imedia.ru (eugene@localhost [127.0.0.1]) by badger.imedia.ru (8.14.4/8.14.4) with ESMTP id p6Q5sYvQ058620; Tue, 26 Jul 2011 09:54:34 +0400 (MSD) (envelope-from eugene@imedia.ru) Received: from localhost (localhost [[UNIX: localhost]]) by badger.imedia.ru (8.14.4/8.14.4/Submit) id p6Q5sYI9058619; Tue, 26 Jul 2011 09:54:34 +0400 (MSD) (envelope-from eugene@imedia.ru) X-Authentication-Warning: badger.imedia.ru: eugene set sender to eugene@imedia.ru using -f From: Eugene Mitrofanov Organization: Sanoma Independent Media To: Martin Matuska Date: Tue, 26 Jul 2011 09:54:33 +0400 User-Agent: KMail/1.9.10 References: <201107221148.13100.eugene@imedia.ru> <4E2DE815.5010803@FreeBSD.org> In-Reply-To: <4E2DE815.5010803@FreeBSD.org> X-Origin: badger.imedia.ru MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <201107260954.33930.eugene@imedia.ru> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (lynx.imedia.ru [10.167.0.252]); Tue, 26 Jul 2011 09:54:34 +0400 (MSD) X-Virus-Scanned: clamav-milter 0.97-exp at lynx.imedia.ru X-Virus-Status: Clean Cc: freebsd-stable@freebsd.org, freebsd-questions@freebsd.org Subject: Re: ZFS v28 and aclmode X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Mitrofanov List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 05:54:38 -0000 Thanks a lot On Tuesday 26 July 2011, Martin Matuska wrote: > Dňa 22. 7. 2011 9:48, Eugene Mitrofanov wrote / napísal(a): > > Hi all, > > > > I just updated to 8.2S and zfs v28 and notice strange thing: in the manual i > > can read about aclmode but i can not use it in the real life: > > > > # zfs set aclmode=passthrough data/public > > cannot set property for 'data/public': invalid property 'aclmode' > > > > Obsolete manual? > > > > Good luck > I imported reintroduction of aclmode from Illumos to 9-CURRENT in > revsion 224174 > MFC to 8-STABLE will be around Aug 1, 2011. > > More information: > https://www.illumos.org/issues/742 > http://svnweb.freebsd.org/base?view=revision&revision=224174 > > -- > Martin Matuska > FreeBSD committer > http://blog.vx.sk -- EMIT-RIPN, EVM7-RIPE From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 07:25:58 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F977106566B; Tue, 26 Jul 2011 07:25:58 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-legacy.sentex.ca (freebsd-legacy.sentex.ca [IPv6:2607:f3e0:0:3::6502:9a]) by mx1.freebsd.org (Postfix) with ESMTP id 31D0D8FC15; Tue, 26 Jul 2011 07:25:58 +0000 (UTC) Received: from freebsd-legacy.sentex.ca (localhost [127.0.0.1]) by freebsd-legacy.sentex.ca (8.14.5/8.14.5) with ESMTP id p6Q7Pv5u082137; Tue, 26 Jul 2011 07:25:57 GMT (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-legacy.sentex.ca (8.14.5/8.14.5/Submit) id p6Q7PvL2082107; Tue, 26 Jul 2011 07:25:57 GMT (envelope-from tinderbox@freebsd.org) Date: Tue, 26 Jul 2011 07:25:57 GMT Message-Id: <201107260725.p6Q7PvL2082107@freebsd-legacy.sentex.ca> X-Authentication-Warning: freebsd-legacy.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_7 tinderbox] failure on i386/pc98 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 07:25:58 -0000 TB --- 2011-07-26 06:15:19 - tinderbox 2.6 running on freebsd-legacy.sentex.ca TB --- 2011-07-26 06:15:19 - starting RELENG_7 tinderbox run for i386/pc98 TB --- 2011-07-26 06:15:19 - cleaning the object tree TB --- 2011-07-26 06:15:33 - cvsupping the source tree TB --- 2011-07-26 06:15:33 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca -s /usr/home/tinderbox/RELENG_7/i386/pc98/supfile TB --- 2011-07-26 06:15:43 - building world TB --- 2011-07-26 06:15:43 - MAKEOBJDIRPREFIX=/obj TB --- 2011-07-26 06:15:43 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-07-26 06:15:43 - TARGET=pc98 TB --- 2011-07-26 06:15:43 - TARGET_ARCH=i386 TB --- 2011-07-26 06:15:43 - TZ=UTC TB --- 2011-07-26 06:15:43 - __MAKE_CONF=/dev/null TB --- 2011-07-26 06:15:43 - cd /src TB --- 2011-07-26 06:15:43 - /usr/bin/make -B buildworld >>> World build started on Tue Jul 26 06:15:44 UTC 2011 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] cc -Os -fno-guess-branch-probability -fomit-frame-pointer -fno-unit-at-a-time -mno-align-long-strings -mrtd -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -DUFS1_AND_UFS2 -DFLAGS= -DSIOPRT=0x238 -DSIOFMT=0x3 -DSIOSPD=115200 -I/src/sys/boot/pc98/boot2/../../.. -I/src/sys/boot/pc98/boot2/../../i386/boot2 -I/src/sys/boot/pc98/boot2/../../common -I/src/sys/boot/pc98/boot2/../btx/lib -I. -Wall -Waggregate-return -Wbad-function-cast -Wcast-align -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wshadow -Wstrict-prototypes -Wwrite-strings -Winline --param max-inline-insns-single=100 -ffreestanding -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -Os -DPC98 -c /src/sys/boot/pc98/boot2/../../i386/boot2/sio.S ld -static -N --gc-sections -nostdlib -Ttext 0x2000 -o boot2.out /obj/pc98/src/sys/boot/pc98/boot2/../btx/lib/crt0.o boot2.o sio.o objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b /obj/pc98/src/sys/boot/pc98/boot2/../btx/btx/btx -l boot2.ldr -o boot2.ld -P 1 boot2.bin kernel: ver=1.02 size=6d0 load=9000 entry=9010 map=16M pgctl=1:1 client: fmt=bin size=16a1 text=0 data=0 bss=0 entry=0 output: fmt=bin size=1e85 text=114 data=1d71 org=0 entry=0 -133 bytes available *** Error code 1 Stop in /src/sys/boot/pc98/boot2. *** Error code 1 Stop in /src/sys/boot/pc98. *** Error code 1 Stop in /src/sys/boot. *** Error code 1 Stop in /src/sys. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2011-07-26 07:25:57 - WARNING: /usr/bin/make returned exit code 1 TB --- 2011-07-26 07:25:57 - ERROR: failed to build world TB --- 2011-07-26 07:25:57 - 3201.93 user 574.00 system 4237.13 real http://tinderbox.freebsd.org/tinderbox-releng_7-RELENG_7-i386-pc98.full From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 07:53:14 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B36A3106566B; Tue, 26 Jul 2011 07:53:14 +0000 (UTC) (envelope-from amon@aelita.org) Received: from aelita.org (unknown [IPv6:2001:7a8:7003::3]) by mx1.freebsd.org (Postfix) with ESMTP id B5E418FC12; Tue, 26 Jul 2011 07:53:12 +0000 (UTC) Received: from ra.aabs (localhost [127.0.0.1]) by aelita.org (8.14.4/8.14.4) with ESMTP id p6Q9nEZw090394; Tue, 26 Jul 2011 11:49:14 +0200 (CEST) (envelope-from amon@ra.aabs) Received: (from amon@localhost) by ra.aabs (8.14.4/8.14.4/Submit) id p6Q9nDx5090393; Tue, 26 Jul 2011 11:49:13 +0200 (CEST) (envelope-from amon) Date: Tue, 26 Jul 2011 11:49:13 +0200 From: Herve Boulouis To: Kostik Belousov Message-ID: <20110726094913.GF17204@ra.aabs> References: <20110725102107.GB17204@ra.aabs> <20110725085902.GM17489@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=unknown-8bit Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110725085902.GM17489@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i Cc: rmacklem@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 07:53:14 -0000 Le 25/07/2011 11:59, Kostik Belousov a écrit: Ok the patched server crashed this morning strangely : all httpd processes were stuck in nfs or vmopar and were unkillable. Below is the full ps. When tried to reboot it got stuck after "All buffers synced" with LOR : lock order reversal: 1st 0xffffff00094a4270 ufs (ufs) @ kern/vfs_mount.c:1204 2nd 0xffffff000960f7f8 syncer (syncer) @ kern/vfs_subr.c:2232 lock order reversal: 1st 0xffffff000960f9d0 nfs (nfs) @ kern/vfs_subr.c:2120 2nd 0xffffffff80b567e0 allproc (allproc) @ kern/kern_descrip.c:2518 In my original post I forgot to add that this box (Dell M610) has 24 cores inside (2 physicals CPUs with 6 cores each and HT enabled). Full ps before reboot : UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 8 0 0 96 - DLs ?? 0:45.66 [kernel] 0 1 0 0 60 0 3204 120 wait ILs ?? 0:00.78 /sbin/init -- 0 2 0 0 -8 0 0 16 - DL ?? 0:03.70 [g_event] 0 3 0 0 -8 0 0 16 - DL ?? 3:18.53 [g_up] 0 4 0 0 -8 0 0 16 - DL ?? 1:01.57 [g_down] 0 5 0 0 -16 0 0 16 idle DL ?? 0:00.00 [mpt_recovery0] 0 6 0 0 -16 0 0 16 idle DL ?? 0:00.00 [mpt_raid0] 0 7 0 0 -16 0 0 16 waitin DL ?? 0:00.00 [sctp_iterator] 0 8 0 0 -16 0 0 16 pftm DL ?? 0:00.38 [pfpurge] 0 9 0 0 -16 0 0 16 ccb_sc DL ?? 0:00.00 [xpt_thrd] 0 10 0 0 -16 0 0 16 audit_ DL ?? 0:00.00 [audit] 0 11 0 0 171 0 0 384 - RL ?? 21204:47.83 [idle] 0 12 0 0 -60 0 0 640 - WL ?? 21:01.69 [intr] 0 13 0 0 44 0 0 16 - DL ?? 0:42.12 [yarrow] 0 14 0 0 -64 0 0 448 - DL ?? 0:01.85 [usb] 0 15 0 0 -8 0 0 16 mdwait DL ?? 0:00.00 [md0] 0 16 0 0 52 0 0 16 psleep DL ?? 25:15.74 [pagedaemon] 0 17 0 0 53 0 0 16 psleep DL ?? 1:54.70 [vmdaemon] 0 18 0 0 76 0 0 16 pgzero DL ?? 0:00.00 [pagezero] 0 19 0 0 -16 0 0 16 psleep DL ?? 0:00.35 [bufdaemon] 0 20 0 0 46 0 0 16 syncer DL ?? 9:22.33 [syncer] 0 21 0 0 46 0 0 16 vlruwt DL ?? 1:38.87 [vnlru] 0 22 0 0 44 0 0 16 sdflus DL ?? 1:31.12 [softdepflush] 0 23 0 0 -16 0 0 16 flowcl DL ?? 0:01.61 [flowcleaner] 80 169 1 0 46 0 132088 10752 nfs D ?? 0:00.05 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 465 1 0 44 -20 175352 10652 nfs D< ?? 0:13.53 /usr/local/sbin/httpd -DNOHTTPACCEPT 0 715 1 0 44 0 3200 32 select Is ?? 0:00.00 /sbin/devd 0 888 1 0 44 0 5864 900 select Ss ?? 3:47.20 /usr/sbin/syslogd -ss 0 914 1 0 44 0 6924 936 select Ss ?? 0:11.19 /usr/sbin/rpcbind 0 1020 1 0 44 0 267976 800 select Ss ?? 0:08.92 /usr/sbin/rpc.statd 0 1028 1 0 76 0 6916 760 rpcsvc Ss ?? 0:00.09 /usr/sbin/rpc.lockd 0 1043 1 0 47 0 15200 2660 nanslp I ?? 6:01.08 /usr/local/bin/monit -c /usr/local/etc/monitrc 122 1052 1 0 76 5 11988 0 wait IWN ?? 0:00.00 zabbix_agentd: main process (zabbix_agentd) 122 1061 1052 0 49 5 11988 1244 nanslp SN ?? 2:49.93 zabbix_agentd: collector [sleeping for 1 seconds] (zabbix_agentd 122 1062 1052 0 49 5 11988 800 select SN ?? 1:44.39 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) 122 1063 1052 0 49 5 11988 800 accept SN ?? 1:43.88 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) 122 1064 1052 0 49 5 11988 808 accept SN ?? 1:35.01 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) 122 1065 1052 0 49 5 11988 1192 nanslp SN ?? 3:00.97 zabbix_agentd: poller [sleeping for 1 second(s)] (zabbix_agentd) 0 1077 1 0 45 0 28064 2316 select S ?? 15:20.13 /usr/local/sbin/snmpd -p /var/run/snmpd.pid 0 1269 1 0 44 0 5992 536 kqread Ss ?? 0:06.57 /usr/local/libexec/postfix/master 125 1282 1269 0 44 0 5996 792 kqread S ?? 0:02.04 qmgr -l -t fifo -u 0 1288 1 0 44 0 43724 3212 select S ?? 0:40.03 /usr/local/bin/python2.6 /usr/local/bin/fail2ban-server -b -s /v 0 1339 1 0 44 0 36936 1824 select Ss ?? 0:00.08 sshd: root@pts/0 (sshd) 0 1340 1 0 76 0 6920 480 nanslp Is ?? 0:05.94 /usr/sbin/cron -s 80 1721 1 0 44 -20 212216 10664 nfs D< ?? 0:06.65 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 1746 1 0 60 -20 208120 10640 nfs D< ?? 0:29.05 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 1747 1 0 44 -20 156920 10692 nfs D< ?? 0:04.32 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 2324 1 0 76 -20 212216 10704 nfs D< ?? 0:01.95 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 3400 1 0 44 -20 163064 10660 nfs D< ?? 0:05.01 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 3401 1 0 44 -20 181496 10688 nfs D< ?? 0:00.80 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 3942 1 0 47 -20 181496 10620 nfs D< ?? 0:29.44 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 3978 1 0 44 -20 179448 10672 nfs D< ?? 0:03.49 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 3979 1 0 44 -20 181496 10648 nfs D< ?? 0:09.13 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 4494 1 0 44 -20 175352 10640 nfs D< ?? 0:25.07 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 4495 1 0 68 -20 183544 10624 nfs D< ?? 0:33.51 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 5110 1 0 76 -20 208120 10688 nfs D< ?? 0:01.41 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 5675 1 0 44 -20 197880 10592 nfs D< ?? 0:42.38 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 6943 1 0 68 -20 208120 10656 nfs D< ?? 0:15.50 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 6944 1 0 44 -20 183544 10644 nfs D< ?? 0:18.46 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 7350 1 0 44 0 154744 15180 nfs D ?? 0:01.61 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 7436 1 0 44 0 177400 15952 nfs D ?? 0:19.34 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 7496 1 0 44 -20 175352 10648 nfs D< ?? 0:12.21 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 7647 1 0 44 -20 152824 10660 nfs D< ?? 0:02.36 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 8127 1 0 47 -20 173304 10616 nfs D< ?? 0:46.18 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 8713 1 0 76 -20 181496 10656 nfs D< ?? 0:10.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 8961 1 0 44 -20 154872 10692 nfs D< ?? 0:02.43 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9219 1 0 54 -20 173304 10612 nfs D< ?? 0:18.19 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9236 1 0 44 0 171256 15400 nfs D ?? 0:07.30 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9238 1 0 44 -20 199928 79656 nfs D< ?? 0:01.20 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9267 1 0 44 -20 199928 79988 nfs D< ?? 0:34.80 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9273 1 0 76 -20 199928 81060 nfs D< ?? 0:10.44 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9693 1 0 76 -20 181496 10656 nfs D< ?? 0:17.64 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9831 1 0 44 -20 173304 10672 nfs D< ?? 0:07.07 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 9871 1 0 44 -20 169208 10684 nfs D< ?? 0:05.17 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10223 1 0 44 -20 181496 10636 nfs D< ?? 0:25.70 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10279 1 0 63 0 175352 15724 nfs D ?? 0:16.30 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10361 1 0 44 -20 191736 10672 nfs D< ?? 0:08.45 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10655 1 0 59 -20 181496 61448 nfs D< ?? 0:01.71 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10656 1 0 71 -20 199928 79648 nfs D< ?? 0:05.74 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10697 1 0 59 -20 199928 79980 nfs D< ?? 0:12.17 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 10994 1 0 62 0 171256 15868 nfs D ?? 0:11.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11044 1 0 72 -20 199928 79952 nfs D< ?? 0:27.56 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11078 1 0 59 -20 181496 61992 nfs D< ?? 0:57.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11093 1 0 59 -20 199928 79968 nfs D< ?? 0:12.68 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11245 1 0 44 -20 161016 10696 nfs D< ?? 0:01.55 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11274 1 0 64 -20 181496 10684 nfs D< ?? 0:09.52 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11300 1 0 76 -20 208120 10708 nfs D< ?? 0:01.58 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11301 1 0 76 -20 201976 10688 nfs D< ?? 0:08.91 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11303 1 0 56 -20 181496 11256 nfs D< ?? 0:48.15 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11307 1 0 70 -20 201976 11072 nfs D< ?? 0:23.23 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11661 1 0 76 -20 201976 82356 nfs D< ?? 0:38.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 11978 1 0 73 -20 201976 11028 nfs D< ?? 0:23.06 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 12423 1 0 65 -20 210168 12328 nfs D< ?? 0:20.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 12441 1 0 56 -20 201976 81556 nfs D< ?? 1:23.33 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 12757 1 0 65 0 158968 14772 nfs D ?? 0:00.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 13281 1 0 67 -20 201848 11304 nfs D< ?? 0:02.11 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 13855 1 0 53 -20 173304 10680 nfs D< ?? 0:09.40 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 13874 1 0 44 -20 175352 10640 nfs D< ?? 0:15.75 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14008 1 0 58 -20 185720 14796 nfs D< ?? 0:48.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14041 1 0 54 -20 179448 10664 nfs D< ?? 0:16.97 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14043 1 0 44 -20 173304 10648 nfs D< ?? 0:15.35 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14082 1 0 44 -20 171256 10672 nfs D< ?? 0:07.50 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14264 1 0 60 0 173304 15328 nfs D ?? 0:04.78 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14327 1 0 44 -20 169208 10656 nfs D< ?? 0:14.71 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14510 1 0 58 0 173304 15800 nfs D ?? 0:17.82 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14701 1 0 64 -20 199928 79836 nfs D< ?? 0:04.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14891 1 0 44 -20 200056 80060 nfs D< ?? 0:26.68 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 14930 1 0 46 0 132088 10760 nfs D ?? 0:00.02 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15102 1 0 44 -20 183544 10648 nfs D< ?? 0:36.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15118 1 0 46 -20 201976 80812 nfs D< ?? 0:42.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15119 1 0 44 -20 165112 10664 nfs D< ?? 0:05.73 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15275 1 0 76 -20 201976 81900 nfs D< ?? 0:06.18 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15583 1 0 44 -20 136184 10688 nfs D< ?? 0:00.12 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15617 1 0 52 0 165112 15532 nfs D ?? 0:03.31 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15956 1 0 56 -20 181496 10636 nfs D< ?? 0:28.79 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 15957 1 0 44 -20 136184 10700 nfs D< ?? 0:00.06 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 16395 1 0 59 0 175352 15604 nfs D ?? 0:06.21 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 16635 1 0 44 -20 156920 10680 nfs D< ?? 0:00.76 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 16680 1 0 58 -20 212216 10680 nfs D< ?? 0:07.00 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 16937 1 0 52 -20 199928 80224 nfs D< ?? 0:18.39 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17044 1 0 44 -20 181496 10748 nfs D< ?? 0:39.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17173 1 0 44 -20 167160 11100 nfs D< ?? 0:11.24 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17250 1 0 45 0 163064 15148 nfs D ?? 0:03.63 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17251 1 0 44 0 169208 15564 nfs D ?? 0:05.59 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17466 1 0 57 -20 199928 79908 nfs D< ?? 0:04.42 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17483 1 0 62 -20 181496 61652 nfs D< ?? 0:12.78 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17489 1 0 58 0 187640 15816 nfs D ?? 0:20.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17506 1 0 44 -20 204024 78260 nfs D< ?? 0:03.26 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17656 1 0 46 -20 201976 10804 nfs D< ?? 0:16.42 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 17657 1 0 72 -20 202104 10704 nfs D< ?? 0:19.79 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 18163 1 0 45 0 132088 10756 nfs D ?? 0:00.01 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 18900 1 0 65 -20 201976 81880 nfs D< ?? 0:12.15 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 18956 1 0 44 -20 181496 61316 nfs D< ?? 0:15.83 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 18986 1 0 65 -20 212216 31548 nfs D< ?? 0:07.71 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19127 1 0 76 -20 201976 10696 nfs D< ?? 0:01.33 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19307 1 0 44 0 185592 15512 nfs D ?? 0:28.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19489 1 0 61 -20 181496 33452 nfs D< ?? 0:05.68 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19517 1 0 44 0 163064 13824 nfs D ?? 0:03.39 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19566 1 0 60 -20 199928 52336 nfs D< ?? 0:53.69 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19580 1 0 44 -20 171256 10932 nfs D< ?? 0:08.18 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19593 1 0 76 -20 201976 10888 nfs D< ?? 0:09.98 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19595 1 0 44 -20 167160 10760 nfs D< ?? 0:07.23 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 19596 1 0 51 -20 173304 10700 nfs D< ?? 0:08.23 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20109 1 0 76 -20 201848 11664 nfs D< ?? 0:01.64 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20293 1 0 65 -20 199928 79584 nfs D< ?? 0:01.05 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20570 1 0 76 -20 201976 10680 nfs D< ?? 0:02.63 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20773 1 0 60 -20 199928 10684 nfs D< ?? 0:02.42 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20774 1 0 44 -20 167160 10860 nfs D< ?? 0:20.15 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 20913 1 0 44 -20 204024 84968 nfs D< ?? 0:22.02 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 21013 1 0 72 -20 210168 42436 nfs D< ?? 0:16.85 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 21091 1 0 54 -20 201976 81764 nfs D< ?? 0:21.27 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 21551 1 0 45 -20 201976 81896 nfs D< ?? 0:07.26 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 21581 1 0 61 -20 201976 81884 nfs D< ?? 0:32.97 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 21692 1 0 66 -20 199928 80132 nfs D< ?? 0:44.93 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22467 1 0 48 -20 200056 80068 vmopar D< ?? 0:23.91 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22473 1 0 44 -20 181496 58076 nfs D< ?? 0:06.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22474 1 0 49 -20 181496 60980 nfs D< ?? 0:06.15 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22722 1 0 46 -20 161016 10676 nfs D< ?? 0:01.12 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22751 1 0 54 -20 212216 10896 nfs D< ?? 0:24.68 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22808 1 0 44 -20 179448 10892 nfs D< ?? 0:26.90 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22874 1 0 49 -20 181496 10648 nfs D< ?? 0:13.46 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 22949 1 0 48 -20 181368 60744 nfs D< ?? 0:00.73 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23342 1 0 45 0 132088 10900 nfs D ?? 0:00.03 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23353 1 0 51 -20 193784 10680 nfs D< ?? 0:05.29 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23758 1 0 72 -20 201976 81968 nfs D< ?? 0:36.37 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23779 1 0 53 0 175352 15680 nfs D ?? 0:15.83 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23906 1 0 44 -20 158968 10676 nfs D< ?? 0:03.77 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23948 1 0 76 -20 199928 79984 nfs D< ?? 0:25.26 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23956 1 0 62 0 154872 15204 nfs D ?? 0:05.36 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 23976 1 0 67 -20 181368 10692 nfs D< ?? 0:00.79 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 24713 1 0 44 -20 175352 10680 nfs D< ?? 0:04.47 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 25264 1 0 59 0 163064 15208 nfs D ?? 0:05.28 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 25290 1 0 51 0 163064 15716 nfs D ?? 0:07.25 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 25731 1 0 59 0 163064 15600 nfs D ?? 0:02.72 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 25739 1 0 76 -20 201976 81956 nfs D< ?? 0:05.77 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 25986 1 0 45 -20 181496 10692 nfs D< ?? 0:03.28 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 26873 1 0 44 -20 199928 78284 nfs D< ?? 0:36.38 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 26877 1 0 76 -20 181496 61292 nfs D< ?? 0:05.44 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27215 1 0 49 -20 181496 10676 nfs D< ?? 0:03.17 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27435 1 0 63 -20 201976 81992 nfs D< ?? 0:20.09 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27543 1 0 44 0 154872 14984 vmopar D ?? 0:03.65 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27562 1 0 60 -20 201848 81788 nfs D< ?? 0:03.04 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27648 1 0 51 0 163064 13392 nfs D ?? 0:04.92 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27667 1 0 44 -20 183544 14120 nfs D< ?? 0:18.14 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 27840 1 0 44 0 156920 14824 nfs D ?? 0:06.36 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28078 1 0 63 -20 181496 15416 nfs D< ?? 0:11.50 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28090 1 0 46 -20 181496 10672 vmopar D< ?? 0:09.44 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28155 1 0 50 0 169208 15200 nfs D ?? 0:04.77 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28162 1 0 54 0 167160 5548 nfs D ?? 0:06.27 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28625 1 0 46 -20 201976 81752 nfs D< ?? 0:08.19 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28895 1 0 76 0 163064 14828 nfs D ?? 0:02.60 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 28953 1 0 45 0 132088 12660 nfs D ?? 0:00.02 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29015 1 0 76 -20 199928 79512 nfs D< ?? 0:11.14 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29210 1 0 45 -20 181496 10796 nfs D< ?? 0:08.80 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29429 1 0 57 0 136184 13440 nfs D ?? 0:01.68 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29473 1 0 47 0 136184 13636 nfs D ?? 0:01.19 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29688 1 0 59 0 173304 8336 nfs D ?? 0:13.10 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 29879 1 0 58 0 132088 12752 nfs D ?? 0:00.39 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 30280 1 0 76 0 132088 12752 vmopar D ?? 0:00.03 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 30439 1 0 76 0 132088 12820 nfs D ?? 0:00.10 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 30595 1 0 44 -20 201976 81104 nfs D< ?? 0:33.13 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31237 1 0 44 -20 181368 61032 nfs D< ?? 0:01.83 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31238 1 0 76 -20 201976 82036 nfs D< ?? 1:15.62 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31312 1 0 76 -20 181496 21012 nfs D< ?? 0:06.26 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31438 1 0 64 -20 201976 81716 nfs D< ?? 0:14.72 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31572 1 0 57 0 169208 15444 nfs D ?? 0:21.23 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31617 1 0 76 0 175352 11000 nfs D ?? 0:22.85 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 31938 1 0 46 -20 181496 61108 nfs D< ?? 0:03.33 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 32070 1 0 57 0 173304 15600 nfs D ?? 0:15.80 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 32071 1 0 65 -20 201976 81628 nfs D< ?? 0:27.76 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 32451 1 0 59 0 158968 14864 nfs D ?? 0:07.64 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 32572 1 0 44 0 161016 14104 nfs D ?? 0:06.11 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 32980 1 0 46 -20 181496 54796 nfs D< ?? 0:08.39 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 33042 1 0 61 0 169208 15384 nfs D ?? 0:03.58 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 33333 1 0 76 -20 199928 75412 nfs D< ?? 0:13.56 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 33353 1 0 53 -20 199928 62204 nfs D< ?? 0:40.47 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 33682 1 0 76 0 136184 13664 nfs D ?? 0:03.65 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 33794 1 0 55 0 165112 14860 nfs D ?? 0:01.62 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 34043 1 0 76 0 136184 9184 nfs D ?? 0:06.89 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 34373 1 0 51 0 132088 12772 nfs D ?? 0:00.03 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 34833 1 0 55 0 132088 12676 nfs D ?? 0:00.88 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 35460 1 0 49 0 132088 12900 nfs D ?? 0:00.20 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 35508 1 0 57 0 136184 12836 nfs D ?? 0:00.70 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 35573 1 0 49 -20 189688 59980 nfs D< ?? 0:19.03 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 35646 1 0 64 0 136184 14216 nfs D ?? 0:03.65 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 35921 1 0 51 0 132088 12828 nfs D ?? 0:00.17 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 36368 1 0 63 0 181496 16032 nfs D ?? 0:14.35 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 36391 1 0 57 -20 181496 60416 nfs D< ?? 0:12.79 /usr/local/sbin/httpd -DNOHTTPACCEPT 0 36410 1 0 76 0 25004 3212 select Ss ?? 0:05.12 /usr/sbin/sshd 80 36431 1 0 44 0 163064 15404 nfs D ?? 0:22.64 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 36497 1 0 70 0 136184 13352 nfs D ?? 0:02.78 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 37113 1 0 69 -20 199928 79680 nfs D< ?? 0:18.34 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 37358 1 0 44 0 132088 12680 nfs D ?? 0:00.02 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 38182 1 0 76 0 185592 14604 nfs D ?? 0:51.43 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 38907 1 0 57 -20 199928 79748 nfs D< ?? 0:12.44 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 40607 1 0 57 -20 181496 50724 nfs D< ?? 0:09.08 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 41682 1 0 64 0 175352 14088 nfs D ?? 0:07.51 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 41705 1 0 76 0 181496 14436 nfs D ?? 0:27.59 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 43380 1 0 44 -20 181496 33816 nfs D< ?? 0:02.26 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 43381 1 0 45 0 154872 13536 nfs D ?? 0:01.87 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 43671 1 0 46 0 132088 12600 nfs D ?? 0:00.02 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 43794 1 0 65 0 177400 13604 nfs D ?? 0:11.35 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 44522 1 0 57 0 169208 14128 nfs D ?? 0:06.29 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 45082 1 0 45 0 181496 15528 nfs D ?? 0:24.88 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 45421 1 0 58 -20 199928 55868 nfs D< ?? 0:02.67 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 46241 1 0 70 -20 201976 59196 nfs D< ?? 0:10.51 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 46243 1 0 56 -20 199928 70824 nfs D< ?? 0:11.06 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 46604 1 0 71 -20 181496 49116 nfs D< ?? 0:02.93 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 46649 1 0 44 -20 181496 20004 nfs D< ?? 0:07.93 /usr/local/sbin/httpd -DNOHTTPACCEPT 0 46791 1 0 44 0 36936 1652 select Ss ?? 0:29.76 sshd: root@pts/1 (sshd) 80 47119 1 0 71 -20 201976 57668 nfs D< ?? 0:48.74 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 47124 1 0 44 -20 185720 54048 nfs D< ?? 0:42.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 47542 1 0 55 -20 201976 71656 nfs D< ?? 0:44.89 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 48148 1 0 71 -20 201976 42480 nfs D< ?? 1:24.99 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 48657 1 0 54 0 173304 15340 nfs D ?? 0:03.34 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 49373 1 0 44 -20 204024 68580 nfs D< ?? 0:07.56 /usr/local/sbin/httpd -DNOHTTPACCEPT 0 49523 1 0 44 0 36936 1636 select Is ?? 0:22.79 sshd: root@pts/2 (sshd) 80 49624 1 0 44 -20 201848 81272 nfs D< ?? 0:02.94 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 49641 1 0 45 0 156920 14616 nfs D ?? 0:08.58 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 49726 1 0 53 0 165112 15344 nfs D ?? 0:10.71 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 49866 1 0 44 0 161016 8728 nfs D ?? 0:05.48 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 49880 1 0 63 0 169208 13820 nfs D ?? 0:51.78 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 51064 1 0 49 -20 199928 73000 nfs D< ?? 0:08.01 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 52192 1 0 76 -20 191736 25312 nfs D< ?? 0:22.50 /usr/local/sbin/httpd -DNOHTTPACCEPT 80 52481 1 0 44 -20 207208 10492 vmopar D Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2084F106564A; Tue, 26 Jul 2011 09:06:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 912AE8FC15; Tue, 26 Jul 2011 09:06:57 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6Q96qut008291 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 26 Jul 2011 12:06:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6Q96qju017537; Tue, 26 Jul 2011 12:06:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6Q96qQV017536; Tue, 26 Jul 2011 12:06:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 26 Jul 2011 12:06:52 +0300 From: Kostik Belousov To: Herve Boulouis Message-ID: <20110726090652.GE17489@deviant.kiev.zoral.com.ua> References: <20110725102107.GB17204@ra.aabs> <20110725085902.GM17489@deviant.kiev.zoral.com.ua> <20110726094913.GF17204@ra.aabs> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fsTlqAwC4TOVu2P/" Content-Disposition: inline In-Reply-To: <20110726094913.GF17204@ra.aabs> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: rmacklem@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 09:06:58 -0000 --fsTlqAwC4TOVu2P/ Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > Le 25/07/2011 11:59, Kostik Belousov a =E9crit: >=20 > Ok the patched server crashed this morning strangely : all httpd processe= s were stuck in nfs or vmopar > and were unkillable. Below is the full ps. Please see the http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kernel= debug-deadlocks.html for information required to debug the deadlocks. --fsTlqAwC4TOVu2P/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4ug6sACgkQC3+MBN1Mb4j42gCffeo7TtTdzBJVDTuePnylPp5q iSMAoOv/WyhQ930WDTeAWJu5MfCf7ko6 =4K7s -----END PGP SIGNATURE----- --fsTlqAwC4TOVu2P/-- From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 09:21:55 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CD74106564A; Tue, 26 Jul 2011 09:21:55 +0000 (UTC) (envelope-from amon@aelita.org) Received: from aelita.org (unknown [IPv6:2001:7a8:7003::3]) by mx1.freebsd.org (Postfix) with ESMTP id 1579B8FC13; Tue, 26 Jul 2011 09:21:54 +0000 (UTC) Received: from ra.aabs (localhost [127.0.0.1]) by aelita.org (8.14.4/8.14.4) with ESMTP id p6QBHqpO090950; Tue, 26 Jul 2011 13:17:52 +0200 (CEST) (envelope-from amon@ra.aabs) Received: (from amon@localhost) by ra.aabs (8.14.4/8.14.4/Submit) id p6QBHqYT090949; Tue, 26 Jul 2011 13:17:52 +0200 (CEST) (envelope-from amon) Date: Tue, 26 Jul 2011 13:17:52 +0200 From: Herve Boulouis To: Kostik Belousov Message-ID: <20110726111752.GG17204@ra.aabs> References: <20110725102107.GB17204@ra.aabs> <20110725085902.GM17489@deviant.kiev.zoral.com.ua> <20110726094913.GF17204@ra.aabs> <20110726090652.GE17489@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=unknown-8bit Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110726090652.GE17489@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i Cc: rmacklem@freebsd.org, Herve Boulouis , freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 09:21:55 -0000 Le 26/07/2011 12:06, Kostik Belousov a écrit: > On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > > Le 25/07/2011 11:59, Kostik Belousov a ?crit: > > > > Ok the patched server crashed this morning strangely : all httpd processes were stuck in nfs or vmopar > > and were unkillable. Below is the full ps. > > Please see the > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug-deadlocks.html > for information required to debug the deadlocks. the box was not stricly deadlocked since I was able to interact with it but I suppose you want me to break into debugger when the symptoms appears again and report all the commands listed in the handbook deadlock section ? Regards, -- Herve Boulouis From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 09:33:14 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8984C106566C; Tue, 26 Jul 2011 09:33:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id E075F8FC17; Tue, 26 Jul 2011 09:33:13 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6Q9Wx2P010702 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 26 Jul 2011 12:32:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6Q9Wxw7018446; Tue, 26 Jul 2011 12:32:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6Q9WxhI018445; Tue, 26 Jul 2011 12:32:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 26 Jul 2011 12:32:59 +0300 From: Kostik Belousov To: Herve Boulouis Message-ID: <20110726093258.GF17489@deviant.kiev.zoral.com.ua> References: <20110725102107.GB17204@ra.aabs> <20110725085902.GM17489@deviant.kiev.zoral.com.ua> <20110726094913.GF17204@ra.aabs> <20110726090652.GE17489@deviant.kiev.zoral.com.ua> <20110726111752.GG17204@ra.aabs> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Gr7JUiY80ghPT8n7" Content-Disposition: inline In-Reply-To: <20110726111752.GG17204@ra.aabs> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: rmacklem@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 09:33:14 -0000 --Gr7JUiY80ghPT8n7 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 01:17:52PM +0200, Herve Boulouis wrote: > Le 26/07/2011 12:06, Kostik Belousov a =E9crit: > > On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > > > Le 25/07/2011 11:59, Kostik Belousov a ?crit: > > >=20 > > > Ok the patched server crashed this morning strangely : all httpd proc= esses were stuck in nfs or vmopar > > > and were unkillable. Below is the full ps. > >=20 > > Please see the > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ke= rneldebug-deadlocks.html > > for information required to debug the deadlocks. >=20 > the box was not stricly deadlocked since I was able to interact with it b= ut I suppose you want me to > break into debugger when the symptoms appears again and report all the co= mmands listed in the handbook > deadlock section ? Exactly. I think everything was hung that accessed an nfs mount point. =46rom the usermode, procstat -kk could catch some interesting information, but it is redundant if ddb output is captured. --Gr7JUiY80ghPT8n7 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4uicoACgkQC3+MBN1Mb4i7RACgocU5dOEiHnWgt3fgv4Hym8Y2 q/oAoKZHSahU1cWu/smQ/ZyiargMeH3A =e+BT -----END PGP SIGNATURE----- --Gr7JUiY80ghPT8n7-- From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 10:09:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5E8D106566C for ; Tue, 26 Jul 2011 10:09:25 +0000 (UTC) (envelope-from lists@c0mplx.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) by mx1.freebsd.org (Postfix) with ESMTP id 87C018FC16 for ; Tue, 26 Jul 2011 10:09:25 +0000 (UTC) Received: from pi by home.opsec.eu with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1QleZp-000OG0-PT for freebsd-stable@freebsd.org; Tue, 26 Jul 2011 12:09:25 +0200 Date: Tue, 26 Jul 2011 12:09:25 +0200 From: Kurt Jaeger To: freebsd-stable@freebsd.org Message-ID: <20110726100925.GK1202@home.opsec.eu> References: <20110725093432.GI1202@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:09:25 -0000 Hi! > > What kind of SATA 6g 4-port non-RAID controller is currently suggested > > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > SuperMicro AOC-USAS2-L8i > > PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 > SAS/SATA ports. Supports two firmwares: > IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no > RAID5+) > IT mode enables straight HBA mode > > Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been > MFC'd into 8.2-STABLE. Then the LSISAS9211-4i, using the same chip, should work with mps as well ? http://www.lsi.com/channel/products/storagecomponents/Pages/LSISAS9211-4i.aspx -- pi@opsec.eu +49 171 3101372 9 years to go ! From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 10:34:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 248C3106566C for ; Tue, 26 Jul 2011 10:34:57 +0000 (UTC) (envelope-from mitya@cabletv.dp.ua) Received: from mail.cabletv.dp.ua (mail.cabletv.dp.ua [193.34.20.8]) by mx1.freebsd.org (Postfix) with ESMTP id 7F3A28FC13 for ; Tue, 26 Jul 2011 10:34:56 +0000 (UTC) Received: from [193.34.20.2] (helo=m18.cabletv.dp.ua) by mail.cabletv.dp.ua with esmtp (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Qlefx-0008Cw-Ob for freebsd-stable@freebsd.org; Tue, 26 Jul 2011 13:15:45 +0300 Message-ID: <4E2E9266.7030106@cabletv.dp.ua> Date: Tue, 26 Jul 2011 13:09:42 +0300 From: Mitya User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:5.0) Gecko/20110721 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Installworld failure X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:34:57 -0000 I installed FreeBSD from ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/201105/FreeBSD-8.2-STABLE-201105-amd64-memstick.img cvsup with latest sources In make.conf set CPUTYPE?=nocona make buildworld pass without any errors make installworld print me this messages: ===> sys/boot/i386/boot2 (install) cc -Os -fno-guess-branch-probability -fomit-frame-pointer -fno-unit-at-a-time -mno-align-long-strings -mrtd -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -DUSE_XREAD -DUFS1_AND_UFS2 -DFLAGS=0x80 -DSIOPRT=0x3f8 -DSIOFMT=0x3 -DSIOSPD=9600 -I/usr/src/sys/boot/i386/boot2/../../common -I/usr/src/sys/boot/i386/boot2/../btx/lib -I. -Wall -Waggregate-return -Wbad-function-cast -Wcast-align -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wshadow -Wstrict-prototypes -Wwrite-strings -Winline --param max-inline-insns-single=100 -ffreestanding -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -m32 -march=i386 -std=gnu99 -S -o boot2.s.tmp /usr/src/sys/boot/i386/boot2/boot2.c sed -e '/align/d' -e '/nop/d' < boot2.s.tmp > boot2.s rm -f boot2.s.tmp as --32 -o boot2.o boot2.s ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x2000 -o boot2.out /usr/obj/usr/src/sys/boot/i386/boot2/../btx/lib/crt0.o boot2.o sio.o objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b /usr/obj/usr/src/sys/boot/i386/boot2/../btx/btx/btx -l boot2.ldr -o boot2.ld -P 1 boot2.bin btxld:No such file or directory *** Error code 1 Stop in /usr/src/sys/boot/i386/boot2. *** Error code 1 Stop in /usr/src/sys/boot/i386. *** Error code 1 Stop in /usr/src/sys/boot. *** Error code 1 Stop in /usr/src/sys. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 10:48:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AA13106564A for ; Tue, 26 Jul 2011 10:48:57 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5FBDF8FC1E for ; Tue, 26 Jul 2011 10:48:57 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:8c40:e6eb:d519:2a58] (unknown [IPv6:2001:7b8:3a7:0:8c40:e6eb:d519:2a58]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 94D045C37; Tue, 26 Jul 2011 12:48:56 +0200 (CEST) Message-ID: <4E2E9B97.2010005@FreeBSD.org> Date: Tue, 26 Jul 2011 12:48:55 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Mitya References: <4E2E9266.7030106@cabletv.dp.ua> In-Reply-To: <4E2E9266.7030106@cabletv.dp.ua> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Installworld failure X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:48:57 -0000 On 2011-07-26 12:09, Mitya wrote: ... > make buildworld pass without any errors > > make installworld print me this messages: ... > btxld:No such file or directory This is usually due to the system clock being off, or the object tree being modified after building. Most of the time, you can fix it by doing: make -C /usr/src/sys/boot and then running installworld again. From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 10:57:46 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A8D3106564A; Tue, 26 Jul 2011 10:57:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 892828FC0A; Tue, 26 Jul 2011 10:57:44 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA08195; Tue, 26 Jul 2011 13:57:42 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E2E9DA5.8020106@FreeBSD.org> Date: Tue, 26 Jul 2011 13:57:41 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: Jung-uk Kim References: <20110719112033.GA51765@omma.gibson.athome> <201107221758.01272.jkim@FreeBSD.org> <20110723082108.GB14172@omma.gibson.athome> <201107251455.11482.jkim@FreeBSD.org> In-Reply-To: <201107251455.11482.jkim@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Callum Gibson , freebsd-stable@FreeBSD.org, freebsd-amd64@FreeBSD.org Subject: Re: powernow regression in 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:57:46 -0000 on 25/07/2011 21:55 Jung-uk Kim said the following: > powernow expects a static table > (via acpi_perf) but _PSS is dynamically constructed at runtime. I > don't think we can support this case easily, sorry. Just a side note. Actually I do not see anything wrong with that _PSS except that ASL seems to have been passed through a mangler and it makes reading the code hard. _PSS is a method and it is evaluated as a method in acpi_perf, the method returns a package. I am not sure what exactly you meant when you said that powernow expects a static table. I do not see why the return value can not be dynamically constructed while _PSS is evaluated and why that would affect any of our code. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 11:05:08 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B599106566B for ; Tue, 26 Jul 2011 11:05:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 793418FC08 for ; Tue, 26 Jul 2011 11:05:07 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA08411; Tue, 26 Jul 2011 14:05:04 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E2E9F60.1060808@FreeBSD.org> Date: Tue, 26 Jul 2011 14:05:04 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 11:05:08 -0000 on 26/07/2011 04:20 maestro something said the following: > Hi, > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > panics. > > Here is the information I could gather: > > let me know if I can provide more information. (i.e., i have the full crash > information 80+mbs handy) Use kgdb on the crash dump, go to the dtrace_probe frame and see which exactly assert fails and why. > Here is the backtrace: > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x108 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc11012d7 > stack pointer = 0x28:0xcd3ed9f4 > frame pointer = 0x28:0xcd3eda0c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = resume, IOPL = 0 > current process = 1132 (nc) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e992b at dtrace_panic+0x1b > #7 0xc10e995d at dtrace_assfail+0x2d > #8 0xc10fb28c at dtrace_probe+0x135c > #9 0xc1237f72 at systrace_probe+0x62 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > Uptime: 3m0s > Physical memory: 239 MB > Dumping 79 MB: 64 48 32 16 > > > Steps To reproduce: > > the dtrace program (i.e., test.d) was: > > #!/usr/sbin/dtrace -s > > syscall::accept:return > / execname == "nc" / > { > printf("%s accept:return\n", execname); > ustack(); > } > > % dtrace -s test.d > > then running > % nc -vl 8080 > on one shell and > % nc localhost 8080 > on another would make the kernel panic -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 11:22:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C57841065673 for ; Tue, 26 Jul 2011 11:22:42 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 88F588FC0C for ; Tue, 26 Jul 2011 11:22:42 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 8C6393DD06F for ; Tue, 26 Jul 2011 13:00:52 +0200 (CEST) Message-ID: <4E2E9F24.1040108@dichotomia.fr> Date: Tue, 26 Jul 2011 13:04:04 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 13:00:52 +0200 (CEST) Subject: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 11:22:42 -0000 Hello, I would like to know if it is possible to rebuild world, but without upgrading or even compiling the kernel. The problem is such : I am presently working on a FreeBSD station that seems to have quite a lot of problem, notably with fsck. I am starting to wonder whether this BSD station was properly installed, or if some of the system tools were pasted from older FreeBSD setup. Since the machine is in a remote location, I would prefer to avoid full reinstall if possible. Among other things, single user mode is not available. So I was wondering, if I get the full sources with sysinstall, can I make buildworld and then installworld without going through the kernel phase or would this be a bad idea ? Thanks for your help Jerome Herman From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 11:44:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5849E106566B for ; Tue, 26 Jul 2011 11:44:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta13.emeryville.ca.mail.comcast.net (qmta13.emeryville.ca.mail.comcast.net [76.96.27.243]) by mx1.freebsd.org (Postfix) with ESMTP id 40D938FC13 for ; Tue, 26 Jul 2011 11:44:40 +0000 (UTC) Received: from omta24.emeryville.ca.mail.comcast.net ([76.96.30.92]) by qmta13.emeryville.ca.mail.comcast.net with comcast id Cbbg1h0051zF43QADbkdgi; Tue, 26 Jul 2011 11:44:37 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta24.emeryville.ca.mail.comcast.net with comcast id Cbjw1h0051t3BNj8kbjwuj; Tue, 26 Jul 2011 11:43:57 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DBDB4102C36; Tue, 26 Jul 2011 04:44:38 -0700 (PDT) Date: Tue, 26 Jul 2011 04:44:38 -0700 From: Jeremy Chadwick To: Jerome Herman Message-ID: <20110726114438.GA86683@icarus.home.lan> References: <4E2E9F24.1040108@dichotomia.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E2E9F24.1040108@dichotomia.fr> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 11:44:41 -0000 On Tue, Jul 26, 2011 at 01:04:04PM +0200, Jerome Herman wrote: > I would like to know if it is possible to rebuild world, but without > upgrading or even compiling the kernel. > > The problem is such : I am presently working on a FreeBSD station > that seems to have quite a lot of problem, notably with fsck. I am > starting to wonder whether this BSD station was properly installed, > or if some of the system tools were pasted from older FreeBSD setup. > Since the machine is in a remote location, I would prefer to avoid > full reinstall if possible. Among other things, single user mode is > not available. > > So I was wondering, if I get the full sources with sysinstall, can I > make buildworld and then installworld without going through the > kernel phase or would this be a bad idea ? Is it possible? Yes. Is it a bad idea? Generally yes. World and kernel effectively need to be "in sync"; some kernel binary structures (particularly for things like libkvm) need to be what userland binaries expect them to be. Nobody will be able to provide any support for this configuration. If you're trying to do things ""in phases"" because of this "fsck problem" (see below for more on that), then please be sure that after you rebuild world and reinstall world, that you DO NOT empty out /usr/obj before rebuilding kernel/reinstalling kernel. The kernel build does refer to things in /usr/obj which were built as a result of buildworld. All that said: can we please get some deeper insight as to this "problems with fsck" you're referring to? I'm of the strong opinion that it's better to try and solve the root cause of an issue than do "hackish stuff" like the above (though it's not that hackish, you get what I mean I hope). I don't understand how fsck would cause you a problem unless the machine is constantly losing power or has serious issues with its storage. Are you sure the problem, for example, isn't with the underlying storage device (disk)? If you aren't sure, would you like to verify that's not the problem piece? If so, please post some details like: * dmesg * Contents of /etc/fstab * sysctl kern.disks If the disks are backed by ata(4): * atacontrol list * atacontrol cap XXX (where XXX = each disk shown in kern.disks) If the disks are backed by ada(4) or are SCSI (da(4)): * camcontrol devlist * For ada(4) disks only: camcontrol identify XXX * For da(4) disks only: camcontrol inquiry XXX And regardless of if ata(4), ada(4), or da(4): * smartctl -a /dev/XXX (where XXX = each disk shown in kern.disks; this will require you install ports/sysutils/smartmontools first) I can assist with the disk analysis portion in particular. And with regards to smartctl, please try to ensure the output doesn't get munged (forced line wrapping, newlines injected, etc.). It makes it more difficult to read. Put the output up on the web if you're worried about this. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 11:45:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0056D106566C for ; Tue, 26 Jul 2011 11:45:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id B8F618FC17 for ; Tue, 26 Jul 2011 11:45:36 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:8c40:e6eb:d519:2a58] (unknown [IPv6:2001:7b8:3a7:0:8c40:e6eb:d519:2a58]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id CA9115C37; Tue, 26 Jul 2011 13:45:35 +0200 (CEST) Message-ID: <4E2EA8DF.5090900@FreeBSD.org> Date: Tue, 26 Jul 2011 13:45:35 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Jerome Herman References: <4E2E9F24.1040108@dichotomia.fr> In-Reply-To: <4E2E9F24.1040108@dichotomia.fr> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 11:45:37 -0000 On 2011-07-26 13:04, Jerome Herman wrote: > I would like to know if it is possible to rebuild world, but without > upgrading or even compiling the kernel. > > The problem is such : I am presently working on a FreeBSD station that > seems to have quite a lot of problem, notably with fsck. I am starting > to wonder whether this BSD station was properly installed, or if some of > the system tools were pasted from older FreeBSD setup. > Since the machine is in a remote location, I would prefer to avoid full > reinstall if possible. Among other things, single user mode is not > available. > > So I was wondering, if I get the full sources with sysinstall, can I > make buildworld and then installworld without going through the kernel > phase or would this be a bad idea ? If you build world from *exactly* the same sources your currently running kernel was built from, then yes, it should work. That is, if you are not running with kern.securelevel >= 1. If you build world from newer sources, however, all bets are off. It might work, but also fail in various interesting ways. :) From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 12:26:39 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 022F8106564A for ; Tue, 26 Jul 2011 12:26:39 +0000 (UTC) (envelope-from varga.michal@gmail.com) Received: from mail-ey0-f176.google.com (mail-ey0-f176.google.com [209.85.215.176]) by mx1.freebsd.org (Postfix) with ESMTP id 878808FC15 for ; Tue, 26 Jul 2011 12:26:38 +0000 (UTC) Received: by eya28 with SMTP id 28so654352eya.21 for ; Tue, 26 Jul 2011 05:26:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:organization :date:message-id:mime-version:x-mailer:content-transfer-encoding; bh=c1N4j5OqcVX90hXVmAUgsewtJaIvamuLwk62Zy67ztg=; b=SL+qhCHyvp+Y/VHgwtU3oMpNBMPBxpIzqeUNI84Yrav9VtUmOyQTTtJoc6M54IS/oJ G7yRJXcRQOeN+4gc8Arb3oxJ14RLmSeXPoZZX9We0kQnc2TPbq2FIuM86XQXpUrFyK1Y JWF8pagaYHxtfoygugVjxUuHPFkM0ydR87/gA= Received: by 10.213.13.72 with SMTP id b8mr771123eba.16.1311681543141; Tue, 26 Jul 2011 04:59:03 -0700 (PDT) Received: from [10.0.101.2] (254.166.broadband10.iol.cz [90.177.166.254]) by mx.google.com with ESMTPS id n42sm312483eef.2.2011.07.26.04.59.01 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 26 Jul 2011 04:59:02 -0700 (PDT) From: Michal Varga To: Jerome Herman In-Reply-To: <4E2E9F24.1040108@dichotomia.fr> References: <4E2E9F24.1040108@dichotomia.fr> Content-Type: text/plain; charset="UTF-8" Organization: Stonehenge Date: Tue, 26 Jul 2011 13:58:59 +0200 Message-ID: <1311681539.1799.54.camel@xenon> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:26:39 -0000 On Tue, 2011-07-26 at 13:04 +0200, Jerome Herman wrote: > Hello, > > I would like to know if it is possible to rebuild world, but without > upgrading or even compiling the kernel. > > The problem is such : I am presently working on a FreeBSD station that > seems to have quite a lot of problem, notably with fsck. I am starting > to wonder whether this BSD station was properly installed, or if some of > the system tools were pasted from older FreeBSD setup. > Since the machine is in a remote location, I would prefer to avoid full > reinstall if possible. Among other things, single user mode is not > available. > > So I was wondering, if I get the full sources with sysinstall, can I > make buildworld and then installworld without going through the kernel > phase or would this be a bad idea ? > > Thanks for your help > > Jerome Herman `make buildworld installworld` won't build and install new kernel at all, so that basically answers your first question. You'd need to use `make buildworld installworld kernel` for that effect. To answer your other concern - reinstalling FreeBSD "on the fly" should be without any issues as long as you use the right src revisions corresponding to your current system (and kernel). Mixing worlds and kernels of different revisions should *mostly* work if there were no ABI changes during that time period, but you probably don't want trying this blindly without any means of recovery. Basically - it's doable, but I wouldn't do it with just a single shot on my disposal. Note that you don't necessarily need to install a new kernel in single user mode. While this is generally a good practice and a "safer way to do things", I haven't even done this for half a decade, and I'm re/installing FreeBSD builds practically on a daily basis. My advice: Personally, I'd consider it much safer to roll a new build of kernel along with the world, but again, that's just me. As you're already fully rebuilding a possibly broken installation (which you didn't do and thus don't know everything that might be rotting inside), chances of some magical failure are already pretty decent. Rolling an up-to-date kernel with the rest of the world shouldn't make them any lower, on the opposite, might even raise your chances of a successfull reboot. m. PS: Whatever that means, please don't get your sources through "sysinstall", that monster shouldn't even be present in a seriously maintained FreeBSD installation. Get your sources "the proper way" with csup: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html (note - "csup", not "cvsup", it's explained on the page in detail) -- Michal Varga, Stonehenge (Gmail account) From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 12:50:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51D77106566C for ; Tue, 26 Jul 2011 12:50:06 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 006A18FC14 for ; Tue, 26 Jul 2011 12:50:05 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 73E843DD069; Tue, 26 Jul 2011 14:46:55 +0200 (CEST) Message-ID: <4E2EB814.9040704@dichotomia.fr> Date: Tue, 26 Jul 2011 14:50:28 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Jeremy Chadwick References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> In-Reply-To: <20110726114438.GA86683@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 14:46:55 +0200 (CEST) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:50:06 -0000 On 26/07/2011 13:44, Jeremy Chadwick wrote: > On Tue, Jul 26, 2011 at 01:04:04PM +0200, Jerome Herman wrote: >> I would like to know if it is possible to rebuild world, but without >> upgrading or even compiling the kernel. >> >> The problem is such : I am presently working on a FreeBSD station >> that seems to have quite a lot of problem, notably with fsck. I am >> starting to wonder whether this BSD station was properly installed, >> or if some of the system tools were pasted from older FreeBSD setup. >> Since the machine is in a remote location, I would prefer to avoid >> full reinstall if possible. Among other things, single user mode is >> not available. >> >> So I was wondering, if I get the full sources with sysinstall, can I >> make buildworld and then installworld without going through the >> kernel phase or would this be a bad idea ? > Is it possible? Yes. Is it a bad idea? Generally yes. World and > kernel effectively need to be "in sync"; some kernel binary structures > (particularly for things like libkvm) need to be what userland binaries > expect them to be. Nobody will be able to provide any support for this > configuration. I think kernel and world are already out of sync. This machine is a pre-installed BSD from an ISP, and I have no clues as to how it was done. But I suspect that world was not built or rebuilt properly. I of course got the sources that matches my kernel, and plan to reinstall world just to make sure it is in sync with kernel. > > If you're trying to do things ""in phases"" because of this "fsck > problem" (see below for more on that), then please be sure that after > you rebuild world and reinstall world, that you DO NOT empty out > /usr/obj before rebuilding kernel/reinstalling kernel. The kernel build > does refer to things in /usr/obj which were built as a result of > buildworld. Yes I know the entire compilation chain is in /usr/obj for make kernel. So I won't touch it until I can see clearer on this box. > > All that said: can we please get some deeper insight as to this > "problems with fsck" you're referring to? I'm of the strong opinion > that it's better to try and solve the root cause of an issue than do > "hackish stuff" like the above (though it's not that hackish, you get > what I mean I hope). I don't understand how fsck would cause you a > problem unless the machine is constantly losing power or has serious > issues with its storage. Neither one, nor the other. I have a gvinum setup for data disks. After a forced reboot due to power failure, the box would not come up. Booting into rescue drive I realized that it refused to boot because it could not mount the data partition (/dev/gvinum/data), and this in turn because fsck would not work on the said partition. So I turned off daemons, removed /dev/gvinum/data from fstab and booted again. No problems. Tried to fsck /dev/gvinum/data and got fsck: Could not determine filesystem type fsck_ufs /dev/gvinum/data got stuck on phase 1 for 8 hours before I hard-canceled it. trying to mount the drive resulted in mount: /dev/gvinum/data : Operation not permitted gvinum list giving the following informations : 3 drives: D c State: up /dev/ad7 A: 1/1430799 MB (0%) D a State: up /dev/ad5 A: 1/1430799 MB (0%) D b State: up /dev/ad6 A: 1/1430799 MB (0%) 1 volume: V data State: up Plexes: 2 Size: 2095 GB 2 plexes: P data.p0 S State: up Subdisks: 3 Size: 2095 GB P data.p1 S State: up Subdisks: 3 Size: 2095 GB 6 subdisks: S data.p0.s0 State: up D: a Size: 698 GB S data.p0.s1 State: up D: b Size: 698 GB S data.p0.s2 State: up D: c Size: 698 GB S data.p1.s0 State: up D: c Size: 698 GB S data.p1.s1 State: up D: a Size: 698 GB S data.p1.s2 State: up D: b Size: 698 GB The I did a newfs on the drive, which went well, and I was able to mount it again without any problem. Still testing I decided to umount the drive and to use fsck on it. Same problems came back. Unable to fsck simply, fsck_ufs getting stuck on phase 1 and mount returning "operation not permitted". Newfs again - no problems. Mount again - no problems. Destroyed the gvinum drive, made every disk into a standard UFS drive and fsck on each of them : no problems. Tried to create FreeBSD partition with gvinum slices instead of using disk directly : same old, same old. So here I am starting to think that my disklabel and fsck are not in sync with my kernel. > > Are you sure the problem, for example, isn't with the underlying storage > device (disk)? If you aren't sure, would you like to verify that's > not the problem piece? If so, please post some details like: > > * dmesg > * Contents of /etc/fstab > * sysctl kern.disks > > If the disks are backed by ata(4): > > * atacontrol list > * atacontrol cap XXX (where XXX = each disk shown in kern.disks) > > If the disks are backed by ada(4) or are SCSI (da(4)): > > * camcontrol devlist > * For ada(4) disks only: camcontrol identify XXX > * For da(4) disks only: camcontrol inquiry XXX > > And regardless of if ata(4), ada(4), or da(4): > > * smartctl -a /dev/XXX (where XXX = each disk shown in kern.disks; this > will require you install ports/sysutils/smartmontools first) > > I can assist with the disk analysis portion in particular. > > And with regards to smartctl, please try to ensure the output doesn't > get munged (forced line wrapping, newlines injected, etc.). It makes it > more difficult to read. Put the output up on the web if you're worried > about this. > From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 13:17:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AD481065674 for ; Tue, 26 Jul 2011 13:17:01 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.emeryville.ca.mail.comcast.net (qmta08.emeryville.ca.mail.comcast.net [76.96.30.80]) by mx1.freebsd.org (Postfix) with ESMTP id BD3488FC1A for ; Tue, 26 Jul 2011 13:17:00 +0000 (UTC) Received: from omta19.emeryville.ca.mail.comcast.net ([76.96.30.76]) by qmta08.emeryville.ca.mail.comcast.net with comcast id CdGp1h0011eYJf8A8dGxj6; Tue, 26 Jul 2011 13:16:57 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta19.emeryville.ca.mail.comcast.net with comcast id CdHS1h00H1t3BNj01dHUdG; Tue, 26 Jul 2011 13:17:28 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 543C9102C36; Tue, 26 Jul 2011 06:16:55 -0700 (PDT) Date: Tue, 26 Jul 2011 06:16:55 -0700 From: Jeremy Chadwick To: Jerome Herman Message-ID: <20110726131655.GA88280@icarus.home.lan> References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E2EB814.9040704@dichotomia.fr> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 13:17:03 -0000 On Tue, Jul 26, 2011 at 02:50:28PM +0200, Jerome Herman wrote: > [very large snip] > > So here I am starting to think that my disklabel and fsck are not in > sync with my kernel. I've never heard of either of these utilities (bsdlabel/disklabel, nor fsck) having to be "in sync with the kernel". My opinion at this moment in time is that you're barking up the wrong tree. As I'm not familiar with the vinum infrastructure, GEOM-based or without, others will have to assist with that. However, I'm still not able to discern what your "type" of gvinum volume is -- is it a mirror, a stripe, or a raid5? Others who are more familiar with vinum are probably going to ask you to provide the full configuration details of your vinum setup, including all the commands you issued to create it. "gvinum printconfig" would be a great start. Furthermore, could you please provide the data I asked for with regards to your storage devices? In this case, /dev/ad5, /dev/ad6, and /dev/ad7 (assuming those are all which are on the system)? Let's try to rule out ANY underlying disk issues first, otherwise the rest of the above may be wasted effort. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 14:24:47 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4994106566B for ; Tue, 26 Jul 2011 14:24:47 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 536FF8FC16 for ; Tue, 26 Jul 2011 14:24:46 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id EB2A33DD070 for ; Tue, 26 Jul 2011 16:21:34 +0200 (CEST) Message-ID: <4E2ECE62.4050605@dichotomia.fr> Date: Tue, 26 Jul 2011 16:25:38 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> <20110726131655.GA88280@icarus.home.lan> In-Reply-To: <20110726131655.GA88280@icarus.home.lan> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 16:21:36 +0200 (CEST) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 14:24:47 -0000 On 26/07/2011 15:16, Jeremy Chadwick wrote: > On Tue, Jul 26, 2011 at 02:50:28PM +0200, Jerome Herman wrote: >> [very large snip] >> >> So here I am starting to think that my disklabel and fsck are not in >> sync with my kernel. > I've never heard of either of these utilities (bsdlabel/disklabel, nor > fsck) having to be "in sync with the kernel". My opinion at this moment > in time is that you're barking up the wrong tree. Actually fsck and disklabel needed to be heavily modified in order to support gvinum fully, mainly because the underlying device is very different from a standard drive. > > As I'm not familiar with the vinum infrastructure, GEOM-based or > without, others will have to assist with that. However, I'm still not > able to discern what your "type" of gvinum volume is -- is it a mirror, > a stripe, or a raid5? Actually it is Raid 10 of a sort. Three first halves of the three disk concatenated and mirrored on the three second half of the same drives. > > Others who are more familiar with vinum are probably going to ask you to > provide the full configuration details of your vinum setup, including > all the commands you issued to create it. "gvinum printconfig" would be > a great start. Here is gvinum printconfig drive c device /dev/ad7 drive b device /dev/ad6 drive a device /dev/ad5 volume backup plex name backup.p1 org striped 1024s vol backup plex name backup.p0 org striped 1024s vol backup sd name backup.p1.s2 drive b len 1465137152s driveoffset 1465137417s plex backup.p1 plexoffset 2048s sd name backup.p1.s1 drive a len 1465137152s driveoffset 1465137417s plex backup.p1 plexoffset 1024s sd name backup.p1.s0 drive c len 1465137152s driveoffset 1465137417s plex backup.p1 plexoffset 0s sd name backup.p0.s2 drive c len 1465137152s driveoffset 265s plex backup.p0 plexoffset 2048s sd name backup.p0.s1 drive b len 1465137152s driveoffset 265s plex backup.p0 plexoffset 1024s sd name backup.p0.s0 drive a len 1465137152s driveoffset 265s plex backup.p0 plexoffset 0s By the way, I did the make buildworld, make installworld. results : a) it did reboot and started fine b) it did reboot in 43 seconds (according to monitoring) instead of 8+minutes. c) fsck is now working fine, in under 10 minutes. Boy I love when I do something completely stupid, and it works. (This is a test machine by the way, I would not do this in production) > > Furthermore, could you please provide the data I asked for with regards > to your storage devices? In this case, /dev/ad5, /dev/ad6, and /dev/ad7 > (assuming those are all which are on the system)? Let's try to rule out > ANY underlying disk issues first, otherwise the rest of the above may > be wasted effort. > I completely agree with "removing underlying issues first", that is why when I realized that my base install was borked I went for the make installworld first. The dmesg is very long (it holds about 12 reboots) but for the rest : *> /etc/fstab* # Device Mountpoint FStype Options Dump Pass# /dev/ad4s1a / ufs rw 1 1 /dev/ad4s1b none swap sw 0 0 /dev/ad4s1d /var ufs rw 2 2 /dev/ad4s1e /usr ufs rw 2 2 /dev/ad4s1f /data ufs rw 2 2 /dev/gvinum/backup /backup ufs rw 2 2 proc /proc procfs rw 0 0 *> sysctl kern.disks* kern.disks: ad7 ad6 ad5 ad4 *> atacontrol list* ATA channel 0: Master: no device present Slave: no device present ATA channel 2: Master: ad4 SATA revision 2.x Slave: ad5 SATA revision 2.x ATA channel 3: Master: ad6 SATA revision 2.x Slave: ad7 SATA revision 2.x *> atacontrol cap ad5* Protocol SATA revision 2.x device model ST31500341AS serial number 9VS4QNSC firmware revision CC1H cylinders 16383 heads 16 sectors/track 63 lba supported 268435455 sectors lba48 supported 2930277168 sectors dma supported overlap not supported Feature Support Enable Value Vendor write cache yes yes read ahead yes yes Native Command Queuing (NCQ) yes - 31/0x1F Tagged Command Queuing (TCQ) no no 31/0x1F SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management no no 0/0x00 automatic acoustic management yes yes 254/0xFE 254/0xFE ad6 and 7 are indentical except for serial number. *> smartctl -a /dev/ad5* smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.11 Device Model: ST31500341AS Serial Number: 9VS4QNSC LU WWN Device Id: 5 000c50 02d019b97 Firmware Version: CC1H User Capacity: 1,500,301,910,016 bytes [1.50 TB] Sector Size: 512 bytes logical/physical Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 4 Local Time is: Tue Jul 26 14:21:07 2011 UTC SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 617) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x103f) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 115 099 006 Pre-fail Always - 87140948 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 24 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 074 060 030 Pre-fail Always - 28683116 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 5418 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 24 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 1 189 High_Fly_Writes 0x003a 099 099 000 Old_age Always - 1 190 Airflow_Temperature_Cel 0x0022 060 047 045 Old_age Always - 40 (Min/Max 38/53) 194 Temperature_Celsius 0x0022 040 053 000 Old_age Always - 40 (0 18 0 0) 195 Hardware_ECC_Recovered 0x001a 039 023 000 Old_age Always - 87140948 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 261580688201002 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 3949374025 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 4071332224 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 5417 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. *>smartctl -a /dev/ad6* smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.11 Device Model: ST31500341AS Serial Number: 9VS4MXD5 LU WWN Device Id: 5 000c50 02cd319bf Firmware Version: CC1H User Capacity: 1,500,301,910,016 bytes [1.50 TB] Sector Size: 512 bytes logical/physical Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 4 Local Time is: Tue Jul 26 14:22:34 2011 UTC SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 609) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x103f) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 116 099 006 Pre-fail Always - 107861899 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 23 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 074 060 030 Pre-fail Always - 26454013 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 5419 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 23 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 096 096 000 Old_age Always - 4 190 Airflow_Temperature_Cel 0x0022 056 044 045 Old_age Always In_the_past 44 (0 14 56 39) 194 Temperature_Celsius 0x0022 044 056 000 Old_age Always - 44 (0 18 0 0) 195 Hardware_ECC_Recovered 0x001a 057 029 000 Old_age Always - 107861899 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 161821482816811 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2546745907 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 3981257233 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 5417 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. *> smartctl -a /dev/ad7* smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.11 Device Model: ST31500341AS Serial Number: 9VS4FSDY LU WWN Device Id: 5 000c50 0274ee0d7 Firmware Version: CC1H User Capacity: 1,500,301,910,016 bytes [1.50 TB] Sector Size: 512 bytes logical/physical Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 4 Local Time is: Tue Jul 26 14:23:08 2011 UTC SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 617) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x103f) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 110 099 006 Pre-fail Always - 26689916 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 24 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 8 7 Seek_Error_Rate 0x000f 073 060 030 Pre-fail Always - 22747051 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 5401 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 037 020 Old_age Always - 24 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 1 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 058 045 045 Old_age Always In_the_past 42 (Min/Max 35/55) 194 Temperature_Celsius 0x0022 042 055 000 Old_age Always - 42 (0 18 0 0) 195 Hardware_ECC_Recovered 0x001a 041 028 000 Old_age Always - 26689916 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 18356690227381 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 125910856 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 1003871140 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 5399 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 14:27:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB26F106566B for ; Tue, 26 Jul 2011 14:27:19 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id A13158FC1E for ; Tue, 26 Jul 2011 14:27:19 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 7017B3DD070 for ; Tue, 26 Jul 2011 16:24:09 +0200 (CEST) Message-ID: <4E2ECEFE.5030302@dichotomia.fr> Date: Tue, 26 Jul 2011 16:28:14 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> In-Reply-To: <1311681539.1799.54.camel@xenon> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 16:24:09 +0200 (CEST) Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 14:27:20 -0000 On 26/07/2011 13:58, Michal Varga wrote: > On Tue, 2011-07-26 at 13:04 +0200, Jerome Herman wrote: >> Hello, >> >> I would like to know if it is possible to rebuild world, but without >> upgrading or even compiling the kernel. >> >> The problem is such : I am presently working on a FreeBSD station that >> seems to have quite a lot of problem, notably with fsck. I am starting >> to wonder whether this BSD station was properly installed, or if some of >> the system tools were pasted from older FreeBSD setup. >> Since the machine is in a remote location, I would prefer to avoid full >> reinstall if possible. Among other things, single user mode is not >> available. >> >> So I was wondering, if I get the full sources with sysinstall, can I >> make buildworld and then installworld without going through the kernel >> phase or would this be a bad idea ? >> >> Thanks for your help >> >> Jerome Herman > `make buildworld installworld` won't build and install new kernel at > all, so that basically answers your first question. You'd need to use > `make buildworld installworld kernel` for that effect. > > To answer your other concern - reinstalling FreeBSD "on the fly" should > be without any issues as long as you use the right src revisions > corresponding to your current system (and kernel). Mixing worlds and > kernels of different revisions should *mostly* work if there were no ABI > changes during that time period, but you probably don't want trying this > blindly without any means of recovery. Basically - it's doable, but I > wouldn't do it with just a single shot on my disposal. > > Note that you don't necessarily need to install a new kernel in single > user mode. While this is generally a good practice and a "safer way to > do things", I haven't even done this for half a decade, and I'm > re/installing FreeBSD builds practically on a daily basis. > > My advice: > Personally, I'd consider it much safer to roll a new build of kernel > along with the world, but again, that's just me. As you're already fully > rebuilding a possibly broken installation (which you didn't do and thus > don't know everything that might be rotting inside), chances of some > magical failure are already pretty decent. Rolling an up-to-date kernel > with the rest of the world shouldn't make them any lower, on the > opposite, might even raise your chances of a successfull reboot. > > m. > > PS: Whatever that means, please don't get your sources through > "sysinstall", that monster shouldn't even be present in a seriously > maintained FreeBSD installation. Get your sources "the proper way" with > csup: > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html > > (note - "csup", not "cvsup", it's explained on the page in detail) Well since I am not upgrading, but just making sure things are where they should be I figured that sysinstall was OK for the job. Any problems with sysinstall ? I have been quite happy with it in the 10+ years I have been using FreeBSD. > > From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 14:34:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D318106566B for ; Tue, 26 Jul 2011 14:34:34 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id A1F4C8FC12 for ; Tue, 26 Jul 2011 14:34:33 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 5986F3DD070; Tue, 26 Jul 2011 16:31:22 +0200 (CEST) Message-ID: <4E2ED0B2.7040604@dichotomia.fr> Date: Tue, 26 Jul 2011 16:35:30 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Lars Eighner References: <4E2E9F24.1040108@dichotomia.fr> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 16:31:23 +0200 (CEST) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 14:34:34 -0000 On 26/07/2011 15:01, Lars Eighner wrote: > On Tue, 26 Jul 2011, Jerome Herman wrote: > >> Hello, >> >> I would like to know if it is possible to rebuild world, but without >> upgrading or even compiling the kernel. >> >> The problem is such : I am presently working on a FreeBSD station >> that seems to have quite a lot of problem, notably with fsck. I am >> starting to wonder whether this BSD station was properly installed, >> or if some of the system tools were pasted from older FreeBSD setup. > > You should look into the problems you have without borrowing trouble. > >> Since the machine is in a remote location, I would prefer to avoid >> full reinstall if possible. Among other things, single user mode is >> not available. > >> >> So I was wondering, if I get the full sources with sysinstall, can I >> make buildworld and then installworld without going through the >> kernel phase or would this be a bad idea ? > > You do not have to enter single user mode to make kernel. You "should" > enter single user mode to install world. And in any event, you have > to be > able to reboot the machine. > > Actually, it is not absolutely necessary to enter single-user mode to > install world, but you have to kick all the users off, shut down the > daemons, and sync the discs. This can be difficult and error-prone. And > you have to be able to reboot the machine. If you can reboot the > machine, I > don't quite understand why you cannot enter single-user mode. > > I strongly suggest you document the problems you are having with fsck and > any other system tools to obtain an accurate diagnosis of those problems. I sure wil if the problem arises in a "sane" environment. But here, I realized that my setup was twisted, probably due to an unholy mix of FreeBSD versions. I do not want to get people to try to figure out a problem that only arise when people installing the OS where drunk. Right now I am at step one : "make sure your system looks like something we might be able to support". This box is supposed to be a 8.2, so I am trying hard to make it a real 8.2. Step two will be "make a long and clever bug report stating everything you did and what you expected in result" Step three "Find the bug, submit a patch, be the hero of the day" will be someone else... sadly. > > From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 14:54:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F568106566C for ; Tue, 26 Jul 2011 14:54:57 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta03.emeryville.ca.mail.comcast.net (qmta03.emeryville.ca.mail.comcast.net [76.96.30.32]) by mx1.freebsd.org (Postfix) with ESMTP id 661948FC21 for ; Tue, 26 Jul 2011 14:54:56 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta03.emeryville.ca.mail.comcast.net with comcast id Ceep1h0061bwxycA3eutoj; Tue, 26 Jul 2011 14:54:53 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta18.emeryville.ca.mail.comcast.net with comcast id CeuM1h0191t3BNj8eeuPMZ; Tue, 26 Jul 2011 14:54:24 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 42E95102C36; Tue, 26 Jul 2011 07:54:47 -0700 (PDT) Date: Tue, 26 Jul 2011 07:54:47 -0700 From: Jeremy Chadwick To: Jerome Herman Message-ID: <20110726145447.GA89817@icarus.home.lan> References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> <20110726131655.GA88280@icarus.home.lan> <4E2ECE62.4050605@dichotomia.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E2ECE62.4050605@dichotomia.fr> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 14:54:57 -0000 On Tue, Jul 26, 2011 at 04:25:38PM +0200, Jerome Herman wrote: > On 26/07/2011 15:16, Jeremy Chadwick wrote: > >On Tue, Jul 26, 2011 at 02:50:28PM +0200, Jerome Herman wrote: > >>[very large snip] > >> > >>So here I am starting to think that my disklabel and fsck are not in > >>sync with my kernel. > >I've never heard of either of these utilities (bsdlabel/disklabel, nor > >fsck) having to be "in sync with the kernel". My opinion at this moment > >in time is that you're barking up the wrong tree. > Actually fsck and disklabel needed to be heavily modified in order > to support gvinum fully, mainly because the underlying device is > very different from a standard drive. > > > > >As I'm not familiar with the vinum infrastructure, GEOM-based or > >without, others will have to assist with that. However, I'm still not > >able to discern what your "type" of gvinum volume is -- is it a mirror, > >a stripe, or a raid5? > Actually it is Raid 10 of a sort. Three first halves of the three > disk concatenated and mirrored on the three second half of the same > drives. > > > >Others who are more familiar with vinum are probably going to ask you to > >provide the full configuration details of your vinum setup, including > >all the commands you issued to create it. "gvinum printconfig" would be > >a great start. > Here is gvinum printconfig > > drive c device /dev/ad7 > drive b device /dev/ad6 > drive a device /dev/ad5 > volume backup > plex name backup.p1 org striped 1024s vol backup > plex name backup.p0 org striped 1024s vol backup > sd name backup.p1.s2 drive b len 1465137152s driveoffset 1465137417s > plex backup.p1 plexoffset 2048s > sd name backup.p1.s1 drive a len 1465137152s driveoffset 1465137417s > plex backup.p1 plexoffset 1024s > sd name backup.p1.s0 drive c len 1465137152s driveoffset 1465137417s > plex backup.p1 plexoffset 0s > sd name backup.p0.s2 drive c len 1465137152s driveoffset 265s plex > backup.p0 plexoffset 2048s > sd name backup.p0.s1 drive b len 1465137152s driveoffset 265s plex > backup.p0 plexoffset 1024s > sd name backup.p0.s0 drive a len 1465137152s driveoffset 265s plex > backup.p0 plexoffset 0s > > By the way, I did the make buildworld, make installworld. > > results : > a) it did reboot and started fine > b) it did reboot in 43 seconds (according to monitoring) instead of > 8+minutes. > c) fsck is now working fine, in under 10 minutes. > > Boy I love when I do something completely stupid, and it works. > (This is a test machine by the way, I would not do this in > production) > > > > >Furthermore, could you please provide the data I asked for with regards > >to your storage devices? In this case, /dev/ad5, /dev/ad6, and /dev/ad7 > >(assuming those are all which are on the system)? Let's try to rule out > >ANY underlying disk issues first, otherwise the rest of the above may > >be wasted effort. > > > I completely agree with "removing underlying issues first", that is > why when I realized that my base install was borked I went for the > make installworld first. > The dmesg is very long (it holds about 12 reboots) > > but for the rest : > *> /etc/fstab* > # Device Mountpoint FStype Options Dump > Pass# > /dev/ad4s1a / ufs rw 1 1 > /dev/ad4s1b none swap sw 0 0 > /dev/ad4s1d /var ufs rw 2 2 > /dev/ad4s1e /usr ufs rw 2 2 > /dev/ad4s1f /data ufs rw 2 2 > /dev/gvinum/backup /backup ufs rw 2 2 > proc /proc procfs rw 0 0 > > > *> sysctl kern.disks* > kern.disks: ad7 ad6 ad5 ad4 > > *> atacontrol list* > ATA channel 0: > Master: no device present > Slave: no device present > ATA channel 2: > Master: ad4 SATA revision 2.x > Slave: ad5 SATA revision 2.x > ATA channel 3: > Master: ad6 SATA revision 2.x > Slave: ad7 SATA revision 2.x > > > *> atacontrol cap ad5* > Protocol SATA revision 2.x > device model ST31500341AS > serial number 9VS4QNSC > firmware revision CC1H > cylinders 16383 > heads 16 > sectors/track 63 > lba supported 268435455 sectors > lba48 supported 2930277168 sectors > dma supported > overlap not supported > > Feature Support Enable Value Vendor > write cache yes yes > read ahead yes yes > Native Command Queuing (NCQ) yes - 31/0x1F > Tagged Command Queuing (TCQ) no no 31/0x1F > SMART yes yes > microcode download yes yes > security yes no > power management yes yes > advanced power management no no 0/0x00 > automatic acoustic management yes yes 254/0xFE 254/0xFE > > ad6 and 7 are indentical except for serial number. > > > > *> smartctl -a /dev/ad5* > smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) > Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net > > === START OF INFORMATION SECTION === > Model Family: Seagate Barracuda 7200.11 > Device Model: ST31500341AS > Serial Number: 9VS4QNSC > LU WWN Device Id: 5 000c50 02d019b97 > Firmware Version: CC1H > User Capacity: 1,500,301,910,016 bytes [1.50 TB] > Sector Size: 512 bytes logical/physical > Device is: In smartctl database [for details use: -P show] > ATA Version is: 8 > ATA Standard is: ATA-8-ACS revision 4 > Local Time is: Tue Jul 26 14:21:07 2011 UTC > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > > General SMART Values: > Offline data collection status: (0x82) Offline data collection activity > was completed without error. > Auto Offline Data > Collection: Enabled. > Self-test execution status: ( 0) The previous self-test > routine completed > without error or no > self-test has ever > been run. > Total time to complete Offline > data collection: ( 617) seconds. > Offline data collection > capabilities: (0x7b) SMART execute Offline immediate. > Auto Offline data collection > on/off support. > Suspend Offline collection upon new > command. > Offline surface scan supported. > Self-test supported. > Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > General Purpose Logging supported. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 255) minutes. > Conveyance self-test routine > recommended polling time: ( 2) minutes. > SCT capabilities: (0x103f) SCT Status supported. > SCT Error Recovery Control > supported. > SCT Feature Control supported. > SCT Data Table supported. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE > UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 115 099 006 Pre-fail > Always - 87140948 > 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail > Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age > Always - 24 > 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail > Always - 0 > 7 Seek_Error_Rate 0x000f 074 060 030 Pre-fail > Always - 28683116 > 9 Power_On_Hours 0x0032 094 094 000 Old_age > Always - 5418 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail > Always - 0 > 12 Power_Cycle_Count 0x0032 100 100 020 Old_age > Always - 24 > 184 End-to-End_Error 0x0032 100 100 099 Old_age > Always - 0 > 187 Reported_Uncorrect 0x0032 100 100 000 Old_age > Always - 0 > 188 Command_Timeout 0x0032 100 100 000 Old_age > Always - 1 > 189 High_Fly_Writes 0x003a 099 099 000 Old_age > Always - 1 > 190 Airflow_Temperature_Cel 0x0022 060 047 045 Old_age > Always - 40 (Min/Max 38/53) > 194 Temperature_Celsius 0x0022 040 053 000 Old_age > Always - 40 (0 18 0 0) > 195 Hardware_ECC_Recovered 0x001a 039 023 000 Old_age > Always - 87140948 > 197 Current_Pending_Sector 0x0012 100 100 000 Old_age > Always - 0 > 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age > Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age > Always - 0 > 240 Head_Flying_Hours 0x0000 100 253 000 Old_age > Offline - 261580688201002 > 241 Total_LBAs_Written 0x0000 100 253 000 Old_age > Offline - 3949374025 > 242 Total_LBAs_Read 0x0000 100 253 000 Old_age > Offline - 4071332224 > > SMART Error Log Version: 1 > No Errors Logged > > SMART Self-test log structure revision number 1 > Num Test_Description Status Remaining > LifeTime(hours) LBA_of_first_error > # 1 Short offline Completed without error 00% 5417 > - > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. > > > *>smartctl -a /dev/ad6* > smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) > Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net > > === START OF INFORMATION SECTION === > Model Family: Seagate Barracuda 7200.11 > Device Model: ST31500341AS > Serial Number: 9VS4MXD5 > LU WWN Device Id: 5 000c50 02cd319bf > Firmware Version: CC1H > User Capacity: 1,500,301,910,016 bytes [1.50 TB] > Sector Size: 512 bytes logical/physical > Device is: In smartctl database [for details use: -P show] > ATA Version is: 8 > ATA Standard is: ATA-8-ACS revision 4 > Local Time is: Tue Jul 26 14:22:34 2011 UTC > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > See vendor-specific Attribute list for marginal Attributes. > > General SMART Values: > Offline data collection status: (0x82) Offline data collection activity > was completed without error. > Auto Offline Data > Collection: Enabled. > Self-test execution status: ( 0) The previous self-test > routine completed > without error or no > self-test has ever > been run. > Total time to complete Offline > data collection: ( 609) seconds. > Offline data collection > capabilities: (0x7b) SMART execute Offline immediate. > Auto Offline data collection > on/off support. > Suspend Offline collection upon new > command. > Offline surface scan supported. > Self-test supported. > Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > General Purpose Logging supported. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 255) minutes. > Conveyance self-test routine > recommended polling time: ( 2) minutes. > SCT capabilities: (0x103f) SCT Status supported. > SCT Error Recovery Control > supported. > SCT Feature Control supported. > SCT Data Table supported. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE > UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 116 099 006 Pre-fail > Always - 107861899 > 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail > Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age > Always - 23 > 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail > Always - 0 > 7 Seek_Error_Rate 0x000f 074 060 030 Pre-fail > Always - 26454013 > 9 Power_On_Hours 0x0032 094 094 000 Old_age > Always - 5419 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail > Always - 0 > 12 Power_Cycle_Count 0x0032 100 100 020 Old_age > Always - 23 > 184 End-to-End_Error 0x0032 100 100 099 Old_age > Always - 0 > 187 Reported_Uncorrect 0x0032 100 100 000 Old_age > Always - 0 > 188 Command_Timeout 0x0032 100 100 000 Old_age > Always - 0 > 189 High_Fly_Writes 0x003a 096 096 000 Old_age > Always - 4 > 190 Airflow_Temperature_Cel 0x0022 056 044 045 Old_age > Always In_the_past 44 (0 14 56 39) > 194 Temperature_Celsius 0x0022 044 056 000 Old_age > Always - 44 (0 18 0 0) > 195 Hardware_ECC_Recovered 0x001a 057 029 000 Old_age > Always - 107861899 > 197 Current_Pending_Sector 0x0012 100 100 000 Old_age > Always - 0 > 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age > Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age > Always - 0 > 240 Head_Flying_Hours 0x0000 100 253 000 Old_age > Offline - 161821482816811 > 241 Total_LBAs_Written 0x0000 100 253 000 Old_age > Offline - 2546745907 > 242 Total_LBAs_Read 0x0000 100 253 000 Old_age > Offline - 3981257233 > > SMART Error Log Version: 1 > No Errors Logged > > SMART Self-test log structure revision number 1 > Num Test_Description Status Remaining > LifeTime(hours) LBA_of_first_error > # 1 Short offline Completed without error 00% 5417 > - > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. > > > > *> smartctl -a /dev/ad7* > smartctl 5.41 2011-06-09 r3365 [FreeBSD 8.2-RELEASE amd64] (local build) > Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net > > === START OF INFORMATION SECTION === > Model Family: Seagate Barracuda 7200.11 > Device Model: ST31500341AS > Serial Number: 9VS4FSDY > LU WWN Device Id: 5 000c50 0274ee0d7 > Firmware Version: CC1H > User Capacity: 1,500,301,910,016 bytes [1.50 TB] > Sector Size: 512 bytes logical/physical > Device is: In smartctl database [for details use: -P show] > ATA Version is: 8 > ATA Standard is: ATA-8-ACS revision 4 > Local Time is: Tue Jul 26 14:23:08 2011 UTC > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > See vendor-specific Attribute list for marginal Attributes. > > General SMART Values: > Offline data collection status: (0x82) Offline data collection activity > was completed without error. > Auto Offline Data > Collection: Enabled. > Self-test execution status: ( 0) The previous self-test > routine completed > without error or no > self-test has ever > been run. > Total time to complete Offline > data collection: ( 617) seconds. > Offline data collection > capabilities: (0x7b) SMART execute Offline immediate. > Auto Offline data collection > on/off support. > Suspend Offline collection upon new > command. > Offline surface scan supported. > Self-test supported. > Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > General Purpose Logging supported. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 255) minutes. > Conveyance self-test routine > recommended polling time: ( 2) minutes. > SCT capabilities: (0x103f) SCT Status supported. > SCT Error Recovery Control > supported. > SCT Feature Control supported. > SCT Data Table supported. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE > UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 110 099 006 Pre-fail > Always - 26689916 > 3 Spin_Up_Time 0x0003 100 100 000 Pre-fail > Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age > Always - 24 > 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail > Always - 8 > 7 Seek_Error_Rate 0x000f 073 060 030 Pre-fail > Always - 22747051 > 9 Power_On_Hours 0x0032 094 094 000 Old_age > Always - 5401 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail > Always - 0 > 12 Power_Cycle_Count 0x0032 100 037 020 Old_age > Always - 24 > 184 End-to-End_Error 0x0032 100 100 099 Old_age > Always - 0 > 187 Reported_Uncorrect 0x0032 100 100 000 Old_age > Always - 0 > 188 Command_Timeout 0x0032 100 100 000 Old_age > Always - 1 > 189 High_Fly_Writes 0x003a 100 100 000 Old_age > Always - 0 > 190 Airflow_Temperature_Cel 0x0022 058 045 045 Old_age > Always In_the_past 42 (Min/Max 35/55) > 194 Temperature_Celsius 0x0022 042 055 000 Old_age > Always - 42 (0 18 0 0) > 195 Hardware_ECC_Recovered 0x001a 041 028 000 Old_age > Always - 26689916 > 197 Current_Pending_Sector 0x0012 100 100 000 Old_age > Always - 0 > 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age > Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age > Always - 0 > 240 Head_Flying_Hours 0x0000 100 253 000 Old_age > Offline - 18356690227381 > 241 Total_LBAs_Written 0x0000 100 253 000 Old_age > Offline - 125910856 > 242 Total_LBAs_Read 0x0000 100 253 000 Old_age > Offline - 1003871140 > > SMART Error Log Version: 1 > No Errors Logged > > SMART Self-test log structure revision number 1 > Num Test_Description Status Remaining > LifeTime(hours) LBA_of_first_error > # 1 Short offline Completed without error 00% 5399 > - > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. Thank you. Here's a quick analysis -- I need to go to sleep now as I have work in about 6 hours: 1) The disks in your system are running generally hot. Now before someone flames me saying "44C isn't hot!", it's important to look at the maximum temperatures seen for the disks: /dev/ad5 = current 40C, maximum seen 53C /dev/ad6 = current 44C, maximum seen 56C /dev/ad7 = current 42C, maximum seen 55C The current temperatures are almost certainly while the drives are idling or undergoing extremely little I/O. Heavy I/O across all 3 drives (it matters due to the above gvinum configuration) could almost certainly get these drives up to 60C, which is extremely hot by my standards. What's contributing to this? Two things: 1) surrounding cooling (or lack thereof) or environmental issues (high ambient room temperature, bad placement inside of a case, etc.), 2) platter count. Now I haven't looked up your drive models to see how many platters they have, but platter count in my experience plays a huge role with regards to drive temperatures and stability. Less platters, "generally", means less chance of failure (I say "generally" because we keep cramming more and more data onto a physical medium that isn't increasing in size; e.g. are 3 platters of 250GB each "more stable" than 2 platters of 500GB each?), but also means better airflow and less heat generated. Real life example: I just recently upgraded from a WD -FALS drive to a -FAEX drive solely to go from 3 platters to 2, and my drives' temperature is literally 6C lower because of such. Same case, same environment, same everything. (The -FALS idled at around 42C, while my -FAEX idles at 36C). Given that I run my cases very quiet (~1200rpm fans, etc.), cooling matters to me. 2) /dev/ad7 has 8 reallocated LBAs during its lifetime (total of 5401 hours, so roughly 225 days of use). None of the remapping failed, which is good. Eight LBAs remapped over the course of ~7 months isn't that bad, but that's a little more than one a month on average. If it bothers you, you should RMA the drive. If it were one of my drives, I would RMA it, given that it seems to imply the disk, over time, is getting worse (this is extremely common). Remapping operations happen during writes to the suspect LBAs. Those writes can be delayed (take a long time) depending on what the drive does internally to determine if they're permanently bad or can be remapped. I've heard of cases where remapping of really bad conditioned platters can take up to 30 seconds, which usually induces timeout errors on the console of FreeBSD. (BTW, this specific situation is what TLER was invented to solve, but I don't like TLER; that's for another discussion elsewhere and isn't relevant). Could these remaps be responsible for the weirdness you saw with fsck and gvinum? Simple answer: very, VERY unlikely. 3) /dev/ad5 and /dev/ad7 have seen 1 ATA command timeout during their lifetimes. This isn't much to worry about in my opinion. One is totally acceptable, and may have come this way from the factory (not all vendors clear SMART attributes when shipping drives out; my Intel SSDs, for example, come with things like 7 power-cycles already on them). I wouldn't worry about this, but keep an eye on it over time. Otherwise your disks seem fine. Off to bed... -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 14:59:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF47B1065672 for ; Tue, 26 Jul 2011 14:59:06 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout026.mac.com (asmtpout026.mac.com [17.148.16.101]) by mx1.freebsd.org (Postfix) with ESMTP id C64138FC12 for ; Tue, 26 Jul 2011 14:59:06 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from [10.1.2.163] ([173.200.178.70]) by asmtp026.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LOY00HCQ49KHF00@asmtp026.mac.com> for freebsd-stable@freebsd.org; Tue, 26 Jul 2011 07:58:34 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.4.6813,1.0.211,0.0.0000 definitions=2011-07-26_05:2011-07-26, 2011-07-26, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1107260101 From: Chuck Swiger In-reply-to: <4E2ECE62.4050605@dichotomia.fr> Date: Tue, 26 Jul 2011 07:58:32 -0700 Message-id: <23BD778B-B9A4-43ED-97C6-4DF2D13F80F2@mac.com> References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> <20110726131655.GA88280@icarus.home.lan> <4E2ECE62.4050605@dichotomia.fr> To: Jerome Herman X-Mailer: Apple Mail (2.1084) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 14:59:07 -0000 On Jul 26, 2011, at 7:25 AM, Jerome Herman wrote: > Actually it is Raid 10 of a sort. Three first halves of the three disk concatenated and mirrored on the three second half of the same drives. There's a significant problem right there. Not only will that configuration badly degrade the performance of the RAID volume, it also compromises the goal of redundancy which RAID-1 is supposed to provide. Regards, -- -Chuck From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:03:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D24DB1065672 for ; Tue, 26 Jul 2011 15:03:35 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 9424E8FC0C for ; Tue, 26 Jul 2011 15:03:35 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 2FED53DD070; Tue, 26 Jul 2011 17:00:24 +0200 (CEST) Message-ID: <4E2ED74B.9000807@dichotomia.fr> Date: Tue, 26 Jul 2011 17:03:39 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Chuck Swiger References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> <20110726131655.GA88280@icarus.home.lan> <4E2ECE62.4050605@dichotomia.fr> <23BD778B-B9A4-43ED-97C6-4DF2D13F80F2@mac.com> In-Reply-To: <23BD778B-B9A4-43ED-97C6-4DF2D13F80F2@mac.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 17:00:25 +0200 (CEST) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:03:35 -0000 On 26/07/2011 16:58, Chuck Swiger wrote: > On Jul 26, 2011, at 7:25 AM, Jerome Herman wrote: >> Actually it is Raid 10 of a sort. Three first halves of the three disk concatenated and mirrored on the three second half of the same drives. > There's a significant problem right there. Not only will that configuration badly degrade the performance of the RAID volume, it also compromises the goal of redundancy which RAID-1 is supposed to provide. > > Regards, Disk are interweaved, so the performances are quite good (about 160% of a single drive) and the redundancy is here. Any single drive can fail, and the other two will be there to provide data. Basically the first plesk is a-b-c, and the second is b-c-a, so everything should be fine. From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:19:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98917106566B for ; Tue, 26 Jul 2011 15:19:31 +0000 (UTC) (envelope-from varga.michal@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 234278FC08 for ; Tue, 26 Jul 2011 15:19:30 +0000 (UTC) Received: by ewy1 with SMTP id 1so720459ewy.13 for ; Tue, 26 Jul 2011 08:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:organization :date:message-id:mime-version:x-mailer:content-transfer-encoding; bh=zyz9sFhUWHHs8/hbfKx0P/5pre5GQLgSz5B6F1YiKH8=; b=VrfiAAbGUHNlsoimN49uOQMCsUEZqIxf4nk0c8xqfN992tIowBmevH4volve9H55/z qxkd5A1QGJOAs59b5rhHB/Csb9/Hp6ffabfBVjlFgVXFvCnObs8es3qFhCBD+cJ77ojZ 7285Nq1gf78HF7QUJmJgRhpe3OSBrhpANYVdE= Received: by 10.14.14.165 with SMTP id d37mr2248064eed.100.1311693569983; Tue, 26 Jul 2011 08:19:29 -0700 (PDT) Received: from [10.0.101.2] (254.166.broadband10.iol.cz [90.177.166.254]) by mx.google.com with ESMTPS id t48sm436538eea.14.2011.07.26.08.19.27 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 26 Jul 2011 08:19:29 -0700 (PDT) From: Michal Varga To: Jerome Herman In-Reply-To: <4E2ECEFE.5030302@dichotomia.fr> References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> <4E2ECEFE.5030302@dichotomia.fr> Content-Type: text/plain; charset="UTF-8" Organization: Stonehenge Date: Tue, 26 Jul 2011 17:19:25 +0200 Message-ID: <1311693565.1799.81.camel@xenon> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:19:31 -0000 On Tue, 2011-07-26 at 16:28 +0200, Jerome Herman wrote: > > PS: Whatever that means, please don't get your sources through > > "sysinstall", that monster shouldn't even be present in a seriously > > maintained FreeBSD installation. Get your sources "the proper way" with > > csup: > > > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html > > > > (note - "csup", not "cvsup", it's explained on the page in detail) > > Well since I am not upgrading, but just making sure things are where > they should be I figured that sysinstall was OK for the job. > Any problems with sysinstall ? I have been quite happy with it in the > 10+ years I have been using FreeBSD. > Well, that depends. Probably every single time I've seen someone touching sysinstal in a post-install environment, that OS was instantly rendered as much as good for a complete reinstall. It's just "one of those things" that shouldn't be present in any rescue scenario. Anyway - I can't comment on the specific procedure you're going to employ through sysinstall as I haven't even physically seen that thing for years (and as far as I know, it has been finally nuked from orbit for the upcoming FreeBSD 9 release), but it wasn't even being actively developed for those past 10 years you mention (I might be exaggerating a bit, but it has been a well known fact for years that sysinstall is an unmaintained rotting mess, which in turn recently led to it being finally replaced). Using csup was just an advice to make yourself perfectly sure about the outcome. It's very easy to point your supfile to - *default release=cvs tag=RELENG_8_2 (that's the vanilla release I understand you need to rebuild) - and be 100% sure about what's going to happen and what sources you're going to get. Upgrading or not, that plays no role in this scenario, as csup is just a way of getting a specific source tree, no matter how you're going to use it later. Basically, I wouldn't trust sysinstall with installing FreeBSD for a start, and surely not with a recovery procedure where you have exactly 0 tries left in case you fail. So I'm just pointing that out. m. -- Michal Varga, Stonehenge (Gmail account) From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:20:22 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B10BF1065670 for ; Tue, 26 Jul 2011 15:20:22 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout027.mac.com (asmtpout027.mac.com [17.148.16.102]) by mx1.freebsd.org (Postfix) with ESMTP id 964878FC14 for ; Tue, 26 Jul 2011 15:20:22 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from [10.1.2.163] ([173.200.187.194]) by asmtp027.mac.com (Oracle Communications Messaging Exchange Server 7u4-18.01 64bit (built Jul 15 2010)) with ESMTPSA id <0LOY003FN59L8T30@asmtp027.mac.com> for freebsd-stable@freebsd.org; Tue, 26 Jul 2011 08:20:10 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.4.6813,1.0.211,0.0.0000 definitions=2011-07-26_05:2011-07-26, 2011-07-26, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1107260106 From: Chuck Swiger In-reply-to: <4E2ED74B.9000807@dichotomia.fr> Date: Tue, 26 Jul 2011 08:20:09 -0700 Message-id: <02D59047-E03A-4259-B52F-5CD6B35E3304@mac.com> References: <4E2E9F24.1040108@dichotomia.fr> <20110726114438.GA86683@icarus.home.lan> <4E2EB814.9040704@dichotomia.fr> <20110726131655.GA88280@icarus.home.lan> <4E2ECE62.4050605@dichotomia.fr> <23BD778B-B9A4-43ED-97C6-4DF2D13F80F2@mac.com> <4E2ED74B.9000807@dichotomia.fr> To: Jerome Herman X-Mailer: Apple Mail (2.1084) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:20:22 -0000 On Jul 26, 2011, at 8:03 AM, Jerome Herman wrote: > On 26/07/2011 16:58, Chuck Swiger wrote: >> On Jul 26, 2011, at 7:25 AM, Jerome Herman wrote: >>> Actually it is Raid 10 of a sort. Three first halves of the three disk concatenated and mirrored on the three second half of the same drives. >> There's a significant problem right there. Not only will that configuration badly degrade the performance of the RAID volume, it also compromises the goal of redundancy which RAID-1 is supposed to provide. >> >> Regards, > > Disk are interweaved, so the performances are quite good (about 160% of a single drive) A six-disk RAID-10 setup ought to provide nearly 600% read performance improvement and 300% of the write performance of a single drive-- real numbers tend to be perhaps 550%/250% or so. > and the redundancy is here. Any single drive can fail, and the other two will be there to provide data. Basically the first plesk is a-b-c, and the second is b-c-a, so everything should be fine. Yes, if you do that you can survive a single disk failure, but the degraded performance is going to suck, and you have no chance of surviving a second disk outage. A six-disk RAID-10 volume can survive up to 3 disks failing (although it has a 20% chance of losing the RAID with a two-disk failure and a 50% chance of losing data if a third disk goes without anyone fixing things). Regards, -- -Chuck From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:34:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB95E106566C for ; Tue, 26 Jul 2011 15:34:04 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout026.mac.com (asmtpout026.mac.com [17.148.16.101]) by mx1.freebsd.org (Postfix) with ESMTP id D31628FC12 for ; Tue, 26 Jul 2011 15:34:04 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from [10.1.2.163] ([173.200.178.70]) by asmtp026.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LOY00G0D5WRQ570@asmtp026.mac.com> for freebsd-stable@freebsd.org; Tue, 26 Jul 2011 08:34:04 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.4.6813,1.0.211,0.0.0000 definitions=2011-07-26_05:2011-07-26, 2011-07-26, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1107260110 From: Chuck Swiger In-reply-to: <1311693565.1799.81.camel@xenon> Date: Tue, 26 Jul 2011 08:34:03 -0700 Message-id: References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> <4E2ECEFE.5030302@dichotomia.fr> <1311693565.1799.81.camel@xenon> To: Michal Varga X-Mailer: Apple Mail (2.1084) Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:34:05 -0000 On Jul 26, 2011, at 8:19 AM, Michal Varga wrote: > Well, that depends. Probably every single time I've seen someone > touching sysinstal in a post-install environment, that OS was instantly > rendered as much as good for a complete reinstall. It's just "one of > those things" that shouldn't be present in any rescue scenario. I've been using sysinstall at dozens of customer sites since the 1990's without ever once running into a problem resembling what you've described. Considering the absence of specific details, I think you're exaggerating well past the point of credibility. Anyway, if someone does something bad using sysinstall and needs to fix it, restoring from backups should be all that is needed. When people talk about doing a complete reinstall, it implies to me that they don't have backups in the first place. Regards, -- -Chuck From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:44:59 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC489106566C for ; Tue, 26 Jul 2011 15:44:59 +0000 (UTC) (envelope-from jherman@dichotomia.fr) Received: from mail.dichotomia.fr (hydrogen.dichotomia.net [91.121.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 7E9A88FC0A for ; Tue, 26 Jul 2011 15:44:59 +0000 (UTC) Received: from [192.168.1.18] (unknown [178.33.164.134]) (Authenticated sender: kha@dichotomia.fr) by sslmail.dichotomia.fr (Postfix) with ESMTPSA id 03B593DD070 for ; Tue, 26 Jul 2011 17:41:48 +0200 (CEST) Message-ID: <4E2EE108.3020005@dichotomia.fr> Date: Tue, 26 Jul 2011 17:45:12 +0200 From: Jerome Herman User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> <4E2ECEFE.5030302@dichotomia.fr> <1311693565.1799.81.camel@xenon> In-Reply-To: <1311693565.1799.81.camel@xenon> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (sslmail.dichotomia.fr); Tue, 26 Jul 2011 17:41:49 +0200 (CEST) Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:44:59 -0000 On 26/07/2011 17:19, Michal Varga wrote: > On Tue, 2011-07-26 at 16:28 +0200, Jerome Herman wrote: > >>> PS: Whatever that means, please don't get your sources through >>> "sysinstall", that monster shouldn't even be present in a seriously >>> maintained FreeBSD installation. Get your sources "the proper way" with >>> csup: >>> >>> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html >>> >>> (note - "csup", not "cvsup", it's explained on the page in detail) >> Well since I am not upgrading, but just making sure things are where >> they should be I figured that sysinstall was OK for the job. >> Any problems with sysinstall ? I have been quite happy with it in the >> 10+ years I have been using FreeBSD. >> > Well, that depends. Probably every single time I've seen someone > touching sysinstal in a post-install environment, that OS was instantly > rendered as much as good for a complete reinstall. It's just "one of > those things" that shouldn't be present in any rescue scenario. Oh, I think I know what you are talking about. Sometimes, on some packages would replace /etc files by their default counterpart. This was fixed quite a long time ago, but it was fun indeed. > > Anyway - I can't comment on the specific procedure you're going to > employ through sysinstall as I haven't even physically seen that thing > for years (and as far as I know, it has been finally nuked from orbit > for the upcoming FreeBSD 9 release), but it wasn't even being actively > developed for those past 10 years you mention (I might be exaggerating a > bit, but it has been a well known fact for years that sysinstall is an > unmaintained rotting mess, which in turn recently led to it being > finally replaced). > > Using csup was just an advice to make yourself perfectly sure about the > outcome. It's very easy to point your supfile to - > > *default release=cvs tag=RELENG_8_2 > > (that's the vanilla release I understand you need to rebuild) > > - and be 100% sure about what's going to happen and what sources you're > going to get. Upgrading or not, that plays no role in this scenario, as > csup is just a way of getting a specific source tree, no matter how > you're going to use it later. > > Basically, I wouldn't trust sysinstall with installing FreeBSD for a > start, and surely not with a recovery procedure where you have exactly 0 > tries left in case you fail. So I'm just pointing that out. > > m. > > From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 15:51:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62C211065670 for ; Tue, 26 Jul 2011 15:51:50 +0000 (UTC) (envelope-from varga.michal@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id DFAF38FC13 for ; Tue, 26 Jul 2011 15:51:49 +0000 (UTC) Received: by ewy1 with SMTP id 1so756846ewy.13 for ; Tue, 26 Jul 2011 08:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:organization :date:message-id:mime-version:x-mailer:content-transfer-encoding; bh=osIJSKxkJIDRcUkbuF9jeppWP8bZWcK//JIzlVaEsUk=; b=pCrWrSmsHgqpuwkQbioPyvGku5vS982a7iDerj5nyxiMffThGQRXkxi5o3syjCDVYh VWB7e8tcI6Irlji13DxhKd+HtnjF3y9fAI2wpr0hHXbXzqA/q+Ae/NmURaKhk+LH2s67 efiA5K6ptS9cOStS9KwrBpkm5eU+72uSPqs70= Received: by 10.213.3.213 with SMTP id 21mr2325119ebo.26.1311695508246; Tue, 26 Jul 2011 08:51:48 -0700 (PDT) Received: from [10.0.101.2] (254.166.broadband10.iol.cz [90.177.166.254]) by mx.google.com with ESMTPS id q16sm455553eef.7.2011.07.26.08.51.46 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 26 Jul 2011 08:51:47 -0700 (PDT) From: Michal Varga To: Chuck Swiger In-Reply-To: References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> <4E2ECEFE.5030302@dichotomia.fr> <1311693565.1799.81.camel@xenon> Content-Type: text/plain; charset="UTF-8" Organization: Stonehenge Date: Tue, 26 Jul 2011 17:51:44 +0200 Message-ID: <1311695504.1799.93.camel@xenon> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 15:51:50 -0000 On Tue, 2011-07-26 at 08:34 -0700, Chuck Swiger wrote: > On Jul 26, 2011, at 8:19 AM, Michal Varga wrote: > > Well, that depends. Probably every single time I've seen someone > > touching sysinstal in a post-install environment, that OS was instantly > > rendered as much as good for a complete reinstall. It's just "one of > > those things" that shouldn't be present in any rescue scenario. > > I've been using sysinstall at dozens of customer sites since the > 1990's without ever once running into a problem resembling what you've > described. Considering the absence of specific details, I think > you're exaggerating well past the point of credibility. This is an opinion you're entitled to and I have no particular need to start proving you otherwise. As far as my concerns go, you're free to update your source trees by whatever means that suit you best, I was just pointing the OP a possible point of failure that is easy to avoid by a simple and easily controlled procedure, as there is perfectly nothing that can go wrong with csup. A vital part of any one-shot rescue operation is minimizing things that may go wrong, beforehand. Sysinstall is everything but 'minimal'. > Anyway, if someone does something bad using sysinstall and needs to > fix it, restoring from backups should be all that is needed. When > people talk about doing a complete reinstall, it implies to me that > they don't have backups in the first place. Yes, and interestingly this is, in my own experience, very common among people who use sysinstall to manage their systems (usually only about once or twice). Regards, m. -- Michal Varga, Stonehenge (Gmail account) From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 18:06:55 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F284E1065670 for ; Tue, 26 Jul 2011 18:06:55 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 6E3B28FC0A for ; Tue, 26 Jul 2011 18:06:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id p6QI6psM022759; Wed, 27 Jul 2011 04:06:52 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Wed, 27 Jul 2011 04:06:51 +1000 (EST) From: Ian Smith To: Chuck Swiger In-Reply-To: Message-ID: <20110727033138.Q94719@sola.nimnet.asn.au> References: <4E2E9F24.1040108@dichotomia.fr> <1311681539.1799.54.camel@xenon> <4E2ECEFE.5030302@dichotomia.fr> <1311693565.1799.81.camel@xenon> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-stable@freebsd.org, Michal Varga Subject: Re: Making world but no kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 18:06:56 -0000 On Tue, 26 Jul 2011, Chuck Swiger wrote: > On Jul 26, 2011, at 8:19 AM, Michal Varga wrote: > > Well, that depends. Probably every single time I've seen someone > > touching sysinstal in a post-install environment, that OS was instantly > > rendered as much as good for a complete reinstall. It's just "one of > > those things" that shouldn't be present in any rescue scenario. > > I've been using sysinstall at dozens of customer sites since the > 1990's without ever once running into a problem resembling what > you've described. Considering the absence of specific details, I > think you're exaggerating well past the point of credibility. I think so too. Once I'd learned to use sysinstall post-installation to do more or less just one thing at a time - about 10 years ago - I've not had a problem; I regularly use it for base installs, often on multiple slices on laptops. Ah that's right, 3.3-RELEASE sysinstall was dodgy :) > Anyway, if someone does something bad using sysinstall and needs to > fix it, restoring from backups should be all that is needed. When > people talk about doing a complete reinstall, it implies to me that > they don't have backups in the first place. Indeed, but in this case we're talking about reinstalling sources for a release version. I've never seen sysinstall barf at installing sources (or packages, for that matter) from CD, net or memstick, and don't see how it could be inferior to using csup for the sources, except maybe it'd be a good idea to blow away /usr/src first, against any cruft. Hopefully the new installer will be as useful for multi-booting setups, and for installing selected packages etc. Guess it's time to try 9 .. cheers, Ian From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 22:30:27 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D65F1065672 for ; Tue, 26 Jul 2011 22:30:27 +0000 (UTC) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: from emmi.physik-pool.tu-berlin.de (emmi.physik-pool.tu-berlin.de [130.149.58.146]) by mx1.freebsd.org (Postfix) with ESMTP id C216F8FC12 for ; Tue, 26 Jul 2011 22:30:26 +0000 (UTC) Received: from emmi.physik-pool.tu-berlin.de (localhost.physik-pool.tu-berlin.de [127.0.0.1]) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4) with ESMTP id p6QMULwi071482; Wed, 27 Jul 2011 00:30:21 +0200 (CEST) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: (from elon@localhost) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4/Submit) id p6QMUKwZ071481; Wed, 27 Jul 2011 00:30:20 +0200 (CEST) (envelope-from elon) Date: Wed, 27 Jul 2011 00:30:19 +0200 From: Leon =?iso-8859-15?Q?Me=DFner?= To: Kurt Jaeger Message-ID: <20110726223019.GG48651@emmi.physik-pool.tu-berlin.de> Mail-Followup-To: Kurt Jaeger , freebsd-stable@freebsd.org References: <20110725093432.GI1202@home.opsec.eu> <20110726100925.GK1202@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110726100925.GK1202@home.opsec.eu> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 22:30:27 -0000 On Tue, Jul 26, 2011 at 12:09:25PM +0200, Kurt Jaeger wrote: > Hi! > > > > What kind of SATA 6g 4-port non-RAID controller is currently suggested > > > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > > > SuperMicro AOC-USAS2-L8i > > > > PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 > > SAS/SATA ports. Supports two firmwares: > > IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no > > RAID5+) > > IT mode enables straight HBA mode > > > > Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been > > MFC'd into 8.2-STABLE. > > Then the LSISAS9211-4i, using the same chip, should work with mps as well ? > > http://www.lsi.com/channel/products/storagecomponents/Pages/LSISAS9211-4i.aspx According to the manpage of mps(4) it is supported. From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 22:56:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 766EC106566B for ; Tue, 26 Jul 2011 22:56:03 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 2246B8FC19 for ; Tue, 26 Jul 2011 22:56:02 +0000 (UTC) Received: from omta06.westchester.pa.mail.comcast.net ([76.96.62.51]) by qmta09.westchester.pa.mail.comcast.net with comcast id Cmvd1h00416LCl059mw3ba; Tue, 26 Jul 2011 22:56:03 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta06.westchester.pa.mail.comcast.net with comcast id Cmvy1h01K1t3BNj3Smw08K; Tue, 26 Jul 2011 22:56:03 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 26331102C36; Tue, 26 Jul 2011 15:55:57 -0700 (PDT) Date: Tue, 26 Jul 2011 15:55:57 -0700 From: Jeremy Chadwick To: Freddie Cash Message-ID: <20110726225557.GA97184@icarus.home.lan> References: <20110725093432.GI1202@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kurt Jaeger , freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 22:56:03 -0000 On Mon, Jul 25, 2011 at 01:42:07PM -0700, Freddie Cash wrote: > On Mon, Jul 25, 2011 at 2:34 AM, Kurt Jaeger wrote: > > > What kind of SATA 6g 4-port non-RAID controller is currently suggested > > for use in 8/9 setups with large RAM (64G) setups with ZFS ? > > > > SuperMicro AOC-USAS2-L8i > > PCIe controller with 2 multi-lane connectors (SFF-1087) supporting 8 > SAS/SATA ports. Supports two firmwares: > IR mode enables RAID0, RAID1, RAID10 and a couple of other modes (no > RAID5+) > IT mode enables straight HBA mode > > Supported by the mps(4) driver in FreeBSD 9-CURRENT, and I believe has been > MFC'd into 8.2-STABLE. > > Sure, it's 8-ports, but for under $300 CDN, you really can't do much better. It's important to note that this device -- despite not being mentioned anywhere on their site or in the user manual (!) -- appears to use a PCI Express x8 connector, and will work on PCIe 1.0 or PCIe 2.0. http://www.supermicro.com/products/accessories/addon/AOC-USAS2-L8i.cfm?TYP=I What's confusing about this card is that the web page states "Compatible UIO Motherboards". UIO is Supermicro's proprietary slot type (Universal I/O) which can work with UIO-only cards, OR, using a riser/adapter, can be made into a PCI, PCI-X, or PCIe x{1,2,4,8} slot. So the web page wording and description of this device is really sub-par. And before someone asks: in most cases you *cannot* use this card in a PCIe x16 connector on a motherboard. Most generic motherboard manufacturers at this point have special one-offs that assume their PCIe x16 slots are for video cards only. If you aren't sure, you'll need to ask your motherboard manufacturer/vendor if you can use a non-VGA-adapter in their PCIe x16 slot. Some Supermicro boards do have PCIe x16 slots that can be used by non-VGA adapters, but I haven't seen this on, say, Asus/Gigabyte/Dell/Intel motherboards. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 23:12:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB6341065672; Tue, 26 Jul 2011 23:12:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6CCB98FC19; Tue, 26 Jul 2011 23:12:24 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAHZJL06DaFvO/2dsb2JhbAA1AQEFKVwdGAICDSUCFlEHJIRJo3KJALA/kS6BK4QGgQ8EknKQfA X-IronPort-AV: E=Sophos;i="4.67,272,1309752000"; d="scan'208";a="132358196" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 26 Jul 2011 19:12:23 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 612F3B3F3A; Tue, 26 Jul 2011 19:12:23 -0400 (EDT) Date: Tue, 26 Jul 2011 19:12:23 -0400 (EDT) From: Rick Macklem To: Kostik Belousov Message-ID: <132828699.1045046.1311721943354.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20110726093258.GF17489@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: rmacklem@freebsd.org, Herve Boulouis , freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 23:12:24 -0000 Kostik Belousov wrote: > On Tue, Jul 26, 2011 at 01:17:52PM +0200, Herve Boulouis wrote: > > Le 26/07/2011 12:06, Kostik Belousov a =D0=98crit: > > > On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > > > > Le 25/07/2011 11:59, Kostik Belousov a ?crit: > > > > > > > > Ok the patched server crashed this morning strangely : all httpd > > > > processes were stuck in nfs or vmopar > > > > and were unkillable. Below is the full ps. > > > > > > Please see the > > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/= kerneldebug-deadlocks.html > > > for information required to debug the deadlocks. > > > > the box was not stricly deadlocked since I was able to interact with > > it but I suppose you want me to > > break into debugger when the symptoms appears again and report all > > the commands listed in the handbook > > deadlock section ? >=20 > Exactly. >=20 > I think everything was hung that accessed an nfs mount point. > From the usermode, procstat -kk could catch some interesting > information, > but it is redundant if ddb output is captured. Would it be worth considering reverting r223054? (Note that I don't understand the VM side, so this may be completely wrong:-) The sleeps on vmopar could be happening because a dirty page is busy and r223054 changes the VM_PAGER_xx value set a couple of ways. 1 - When it returns VM_PAGER_ERROR instead of VM_PAGER_AGAIN, the return value of "runlen" from vm_pageout_flush() changes. 2 - I'm not sure, but I think the pre-r223054 code marked a partially written page as VM_PAGER_OK instead of VM_PAGER_AGAIN? (I'm wondering about this one, since the problem seems to happen when the file's size has been truncated.) Herve Boulouis, if you want to see what r223054 changes, just go to http://svn.freebsd.org/viewvc/stable/8/sys/nfsclient and then click on nfs_bio.c. (The changes are small and could easily be reverted with a manual edit.) Since r223054 went into stable/8 on Jun 13, it seems a possible explanation? rick From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 09:38:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70173106564A for ; Wed, 27 Jul 2011 09:38:37 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2A2BE8FC08 for ; Wed, 27 Jul 2011 09:38:36 +0000 (UTC) Received: by vxg33 with SMTP id 33so1354334vxg.13 for ; Wed, 27 Jul 2011 02:38:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=cAS2rYiH/M5U4JmH9dOzQa8qB/zCUI5DWcueaaSNnDo=; b=rjUEbfuKIL9fE5tLdiT+kDr50i+b0jMN2IslHADpWV+HS7A34DWo02Xt7NSjKlfOJH WpFOPgu0ABfl0Kty/zXr3XoYk/lSXkq9SuHPAJtCYo0OQFBdzEvPCqTimkGwOa3gB7Cm YFdeTuxWmaBBtnHSjxRdzmbRabRRam8LkZIf0= MIME-Version: 1.0 Received: by 10.52.175.132 with SMTP id ca4mr6515083vdc.98.1311759516359; Wed, 27 Jul 2011 02:38:36 -0700 (PDT) Received: by 10.52.169.225 with HTTP; Wed, 27 Jul 2011 02:38:36 -0700 (PDT) In-Reply-To: <20110726225557.GA97184@icarus.home.lan> References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> Date: Wed, 27 Jul 2011 10:38:36 +0100 Message-ID: From: Tom Evans To: Jeremy Chadwick Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 09:38:37 -0000 On Tue, Jul 26, 2011 at 11:55 PM, Jeremy Chadwick wrote: > > And before someone asks: in most cases you *cannot* use this card in a > PCIe x16 connector on a motherboard. =C2=A0Most generic motherboard > manufacturers at this point have special one-offs that assume their PCIe > x16 slots are for video cards only. =C2=A0If you aren't sure, you'll need= to > ask your motherboard manufacturer/vendor if you can use a > non-VGA-adapter in their PCIe x16 slot. =C2=A0Some Supermicro boards do h= ave > PCIe x16 slots that can be used by non-VGA adapters, but I haven't seen > this on, say, Asus/Gigabyte/Dell/Intel motherboards. > Jeremy may not have seen PCI express x16 HBAs working on consumer boards, but I have, plenty of times. I don't know why the FUD, but I have had no problems with an Intel SASUC8I (LSI 1068 based) running on an Asus P5Q Pro, a simple consumer P45 chipset. I'm not unique, many people have success doing this. Cheers Tom From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 09:51:53 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC019106564A for ; Wed, 27 Jul 2011 09:51:53 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:176e::2]) by mx1.freebsd.org (Postfix) with ESMTP id A664E8FC12 for ; Wed, 27 Jul 2011 09:51:53 +0000 (UTC) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1Qm0mN-000CJV-D4; Wed, 27 Jul 2011 10:51:51 +0100 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Qm0mN-0009qn-B1; Wed, 27 Jul 2011 10:51:51 +0100 To: freebsd@jdc.parodius.com, tevans.uk@googlemail.com In-Reply-To: Message-Id: From: Pete French Date: Wed, 27 Jul 2011 10:51:51 +0100 Cc: freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 09:51:53 -0000 > Jeremy may not have seen PCI express x16 HBAs working on consumer > boards, but I have, plenty of times. Ditto. Indeed, I wasn't aware you could do anything to a PCIe slot to prevent it working. Have seen motherboards which wont *boot* from an HBA in a PCIe slot, but never one which couldnt use the card when plugged in. I generaly use MSI boards, and all of tose have taken PCIe controllers quite hapily - inclduing booting off HP/Compaq SMART RAID controllers which do have a reputation for being fussy outside of HP machines. Am currently using a variets of Silicon Image based eSATA controllers on the same motherboards. -pete. From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 10:12:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBBD1106566C for ; Wed, 27 Jul 2011 10:12:35 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id CC5D98FC08 for ; Wed, 27 Jul 2011 10:12:35 +0000 (UTC) Received: from omta05.emeryville.ca.mail.comcast.net ([76.96.30.43]) by qmta14.emeryville.ca.mail.comcast.net with comcast id CyCV1h0030vp7WLAEyCY65; Wed, 27 Jul 2011 10:12:32 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta05.emeryville.ca.mail.comcast.net with comcast id CyCk1h0051t3BNj8RyCk3H; Wed, 27 Jul 2011 10:12:44 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 338C6102C36; Wed, 27 Jul 2011 03:12:34 -0700 (PDT) Date: Wed, 27 Jul 2011 03:12:34 -0700 From: Jeremy Chadwick To: Tom Evans Message-ID: <20110727101234.GA7659@icarus.home.lan> References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 10:12:36 -0000 On Wed, Jul 27, 2011 at 10:38:36AM +0100, Tom Evans wrote: > On Tue, Jul 26, 2011 at 11:55 PM, Jeremy Chadwick > wrote: > > > > And before someone asks: in most cases you *cannot* use this card in a > > PCIe x16 connector on a motherboard. ??Most generic motherboard > > manufacturers at this point have special one-offs that assume their PCIe > > x16 slots are for video cards only. ??If you aren't sure, you'll need to > > ask your motherboard manufacturer/vendor if you can use a > > non-VGA-adapter in their PCIe x16 slot. ??Some Supermicro boards do have > > PCIe x16 slots that can be used by non-VGA adapters, but I haven't seen > > this on, say, Asus/Gigabyte/Dell/Intel motherboards. > > Jeremy may not have seen PCI express x16 HBAs working on consumer > boards, but I have, plenty of times. I'm glad -- people should probably start making a list, because the number of boards I've seen it not work in easily exceeds that of the times I HAVE seen it work. :-) > I don't know why the FUD, but I have had no problems with an Intel > SASUC8I (LSI 1068 based) running on an Asus P5Q Pro, a simple consumer > P45 chipset. I'm not unique, many people have success doing this. Once in a blue moon, an intelligent board manufacturer puts comments in their user manual to the effect of "the PCIe x16 slot is only to be used for VGA adapters and will not work with non-VGA cards", or vice-versa. Here's some hard evidence of my claim: Supermicro X7SBA server-class motherboard user manual has the following line in it: "Note: The Intel 3210 chipset does not support add-in graphics cards in the PCI-E interface provided by the Memory Controller Hub (MCH)." So in that boards' case, the PCIe x16 slot (which only has PCIe x8 worth of lanes wired) will work with controller cards but not VGA adapters. And here's another, for the X7DVL series boards: http://www.supermicro.com/support/faqs/faq.cfm?faq=10582 "This is a server board, and you cannot place a x8 or x16 VGA card on this board. Only onboard VGA is possible. If you need to place a VGA card please use X7DA(x) board". And up until last week I owned and used an Asus P5Q SE (P45-based with ICH10 SB) board, so I can refer you to the fact that the P5Q Pro user manual "hints" that the PCIe x16 slot is for graphics only but doesn't downright say that. TL;DR -- it varies from board to board, ask your motherboard manufacturer whether or not it'll work *before* just jamming a card in there. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 10:55:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37BE0106566C for ; Wed, 27 Jul 2011 10:55:19 +0000 (UTC) (envelope-from ml@os2.kiev.ua) Received: from s1.sdv.com.ua (s1.sdv.com.ua [77.120.97.61]) by mx1.freebsd.org (Postfix) with ESMTP id ED1EF8FC0A for ; Wed, 27 Jul 2011 10:55:18 +0000 (UTC) Received: from 90-105-243-80.cust.centrio.cz ([80.243.105.90] helo=[192.168.100.107]) by s1.sdv.com.ua with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Qm1lj-000JyF-3t for freebsd-stable@freebsd.org; Wed, 27 Jul 2011 13:55:17 +0300 Message-ID: <4E2FEE8C.4020607@os2.kiev.ua> Date: Wed, 27 Jul 2011 12:55:08 +0200 From: Alex Samorukov User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Lightning/1.0b2 Thunderbird/3.1.11 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SA-Score: -1.0 Subject: dtrace/mysqld X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 10:55:19 -0000 If anyone interested - i was able to compile dtrace support in mysql-server55 port. During this i found a bug in dtrace/bsd - if it is running more then 1 time on the same object (this is the case for mysqld) then object is broken. I was able to do workaround (with preserving original object and then copying it) and then everything compiled and started correctly. This is an example: bsd# ./work/mysql-5.5.14/support-files/dtrace/query-time.d dtrace: buffer size lowered to 2m Who Database Query Time(ms) root@localhost mysql select sleep(1) from user limit 1 1000 root@localhost mysql select * from user limit 1 0 I will try to cleanup my patches (its currently very dirty hacks) and submit them to the mysql bugtracker (and port). Feel free to contact me if you want to test this (or to fix dtrace/obects bug). From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 12:08:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7AA6106564A; Wed, 27 Jul 2011 12:08:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 39D668FC15; Wed, 27 Jul 2011 12:08:57 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6RC8pj1048173 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 27 Jul 2011 15:08:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6RC8pBu023999; Wed, 27 Jul 2011 15:08:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6RC8oqN023998; Wed, 27 Jul 2011 15:08:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 27 Jul 2011 15:08:50 +0300 From: Kostik Belousov To: Rick Macklem Message-ID: <20110727120850.GT17489@deviant.kiev.zoral.com.ua> References: <20110726093258.GF17489@deviant.kiev.zoral.com.ua> <132828699.1045046.1311721943354.JavaMail.root@erie.cs.uoguelph.ca> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Ro3o5bxpeh65nwUH" Content-Disposition: inline In-Reply-To: <132828699.1045046.1311721943354.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: rmacklem@freebsd.org, Herve Boulouis , freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 12:08:58 -0000 --Ro3o5bxpeh65nwUH Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 07:12:23PM -0400, Rick Macklem wrote: > Kostik Belousov wrote: > > On Tue, Jul 26, 2011 at 01:17:52PM +0200, Herve Boulouis wrote: > > > Le 26/07/2011 12:06, Kostik Belousov a =E9crit: > > > > On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > > > > > Le 25/07/2011 11:59, Kostik Belousov a ?crit: > > > > > > > > > > Ok the patched server crashed this morning strangely : all httpd > > > > > processes were stuck in nfs or vmopar > > > > > and were unkillable. Below is the full ps. > > > > > > > > Please see the > > > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handboo= k/kerneldebug-deadlocks.html > > > > for information required to debug the deadlocks. > > > > > > the box was not stricly deadlocked since I was able to interact with > > > it but I suppose you want me to > > > break into debugger when the symptoms appears again and report all > > > the commands listed in the handbook > > > deadlock section ? > >=20 > > Exactly. > >=20 > > I think everything was hung that accessed an nfs mount point. > > From the usermode, procstat -kk could catch some interesting > > information, > > but it is redundant if ddb output is captured. >=20 > Would it be worth considering reverting r223054? > (Note that I don't understand the VM side, so this may be completely > wrong:-) >=20 > The sleeps on vmopar could be happening because a dirty page is busy > and r223054 changes the VM_PAGER_xx value set a couple of ways. > 1 - When it returns VM_PAGER_ERROR instead of VM_PAGER_AGAIN, the > return value of "runlen" from vm_pageout_flush() changes. > 2 - I'm not sure, but I think the pre-r223054 code marked a partially > written page as VM_PAGER_OK instead of VM_PAGER_AGAIN? > (I'm wondering about this one, since the problem seems to happen > when the file's size has been truncated.) >=20 > Herve Boulouis, if you want to see what r223054 changes, just go to > http://svn.freebsd.org/viewvc/stable/8/sys/nfsclient > and then click on nfs_bio.c. > (The changes are small and could easily be reverted with a manual > edit.) >=20 > Since r223054 went into stable/8 on Jun 13, it seems a possible > explanation? rick I doubt it. The ps output makes it not very inplausible that the reporter got the LOR between vnode lock and page busy flag. The correct order is vnode lock -> busy bit. vmopar is a wait for the busy page state. Mentioned revision does not change the lock order. Anyway, this is only a speculation, until the requested data is provided. --Ro3o5bxpeh65nwUH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4v/9IACgkQC3+MBN1Mb4g1wwCdHVz5RdsVMC8sia2S5qw36Czi TH0Anj9y7UxYNzmvj80NZAdIxfvTNB20 =pguu -----END PGP SIGNATURE----- --Ro3o5bxpeh65nwUH-- From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 16:55:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C715F106566B for ; Wed, 27 Jul 2011 16:55:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.emeryville.ca.mail.comcast.net (qmta10.emeryville.ca.mail.comcast.net [76.96.30.17]) by mx1.freebsd.org (Postfix) with ESMTP id AA9CA8FC14 for ; Wed, 27 Jul 2011 16:55:23 +0000 (UTC) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta10.emeryville.ca.mail.comcast.net with comcast id D4vC1h0020x6nqcAA4vKew; Wed, 27 Jul 2011 16:55:19 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta12.emeryville.ca.mail.comcast.net with comcast id D4vF1h0061t3BNj8Y4vG37; Wed, 27 Jul 2011 16:55:17 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B96C8102C36; Wed, 27 Jul 2011 09:55:20 -0700 (PDT) Date: Wed, 27 Jul 2011 09:55:20 -0700 From: Jeremy Chadwick To: Jan Mikkelsen Message-ID: <20110727165520.GA15633@icarus.home.lan> References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> <20110727101234.GA7659@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Tom Evans , freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 16:55:23 -0000 On Thu, Jul 28, 2011 at 02:31:20AM +1000, Jan Mikkelsen wrote: > On 27/07/2011, at 8:12 PM, Jeremy Chadwick wrote: > > > On Wed, Jul 27, 2011 at 10:38:36AM +0100, Tom Evans wrote: > >> On Tue, Jul 26, 2011 at 11:55 PM, Jeremy Chadwick > >> wrote: > >>> > >>> And before someone asks: in most cases you *cannot* use this card in a > >>> PCIe x16 connector on a motherboard. ??Most generic motherboard > >>> manufacturers at this point have special one-offs that assume their PCIe > >>> x16 slots are for video cards only. ??If you aren't sure, you'll need to > >>> ask your motherboard manufacturer/vendor if you can use a > >>> non-VGA-adapter in their PCIe x16 slot. ??Some Supermicro boards do have > >>> PCIe x16 slots that can be used by non-VGA adapters, but I haven't seen > >>> this on, say, Asus/Gigabyte/Dell/Intel motherboards. > >> > >> Jeremy may not have seen PCI express x16 HBAs working on consumer > >> boards, but I have, plenty of times. > > > > I'm glad -- people should probably start making a list, because the > > number of boards I've seen it not work in easily exceeds that of the > > times I HAVE seen it work. :-) > > Where did it fail? Nothing visible from the BIOS; not talking about POST, I'm talking about after power-on the system would spin up its fans and sit there. > >> I don't know why the FUD, but I have had no problems with an Intel > >> SASUC8I (LSI 1068 based) running on an Asus P5Q Pro, a simple consumer > >> P45 chipset. I'm not unique, many people have success doing this. > > > > Once in a blue moon, an intelligent board manufacturer puts comments in > > their user manual to the effect of "the PCIe x16 slot is only to be used > > for VGA adapters and will not work with non-VGA cards", or vice-versa. > > Here's some hard evidence of my claim: > > > > Supermicro X7SBA server-class motherboard user manual has the following > > line in it: > > > > "Note: The Intel 3210 chipset does not support add-in graphics cards in > > the PCI-E interface provided by the Memory Controller Hub (MCH)." > > > > So in that boards' case, the PCIe x16 slot (which only has PCIe x8 worth > > of lanes wired) will work with controller cards but not VGA adapters. > > > > And here's another, for the X7DVL series boards: > > http://www.supermicro.com/support/faqs/faq.cfm?faq=10582 > > > > "This is a server board, and you cannot place a x8 or x16 VGA card on > > this board. Only onboard VGA is possible. If you need to place a VGA > > card please use X7DA(x) board". > > Perhaps I'm being thick, but this says "a 16x slot on this board does not support a display card" not "a 16x slot on this board only supports a display card." > > The examples you cite only support the "vice-versa" part of your claim, not the primary claim that 16x slots on some motherboards are only for display cards. Do you have a reference for that? You're not being thick. I'm simply using what's in the manual as a hard, validated example that the *type* of card that goes into a PCIe x16 slot can matter, and that some types are compatible while others are not. I cannot find you a hard documented reference for my claims right now, so you can consider them lies/FUD/whatever for the time being, that's fine by me at this point. I can find you examples on Google of people who invested in Areca ARC-1220 cards (PCIe x8) only to find out that when inserted into one of their two PCIe x16 slots the mainboard wouldn't start (see above). I can also find you examples on Google of people with Intel 915GM chipsets whose user manuals explicitly state the PCIe x16 slot on their board is "intended for use with graphics cards only". > > And up until last week I owned and used an Asus P5Q SE (P45-based with > > ICH10 SB) board, so I can refer you to the fact that the P5Q Pro user > > manual "hints" that the PCIe x16 slot is for graphics only but doesn't > > downright say that. > > Did you try a non-graphics card in the slot to see what happened? Saying "The 16x slot is where you put the graphics card" is probably not unreasonable for 99% of the users of that board. No I did not. Let's recap my point because this will be my last reply on the matter: If your motherboard only has PCIe x16 slot(s) available in it, and the user manual doesn't say anything about the slot(s) being usable for non-graphics purposes, BEFORE shoving a non-VGA card in the slot, be sure to consult your user manual and the motherboard vendor to ensure the card will work. The assumption within the industry appears to be that PCIe x16 is "intended for VGA adapters", even though on an engineering level we all know there's no reason (other than design limitations of the board or chipsets) that non-VGA adapters can't be used. I can't explain to you how or why this limitation would come into play, but it obviously can, as the Supermicro board examples I provided are hard evidence of some boards and configurations discriminating based on what type of card goes into the slot. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Wed Jul 27 16:58:08 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47048106564A for ; Wed, 27 Jul 2011 16:58:08 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id C07D28FC12 for ; Wed, 27 Jul 2011 16:58:07 +0000 (UTC) Received: (qmail 4831 invoked by uid 907); 27 Jul 2011 16:31:23 -0000 Received: from b13FC.static.pacific.net.au (HELO [192.168.1.153]) (202.7.88.252) (smtp-auth username janm, mechanism plain) by midgard.transactionware.com (qpsmtpd/0.84) with (AES128-SHA encrypted) ESMTPSA; Thu, 28 Jul 2011 02:31:23 +1000 Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: Jan Mikkelsen In-Reply-To: <20110727101234.GA7659@icarus.home.lan> Date: Thu, 28 Jul 2011 02:31:20 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> <20110727101234.GA7659@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1244.3) Cc: Tom Evans , freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 16:58:08 -0000 On 27/07/2011, at 8:12 PM, Jeremy Chadwick wrote: > On Wed, Jul 27, 2011 at 10:38:36AM +0100, Tom Evans wrote: >> On Tue, Jul 26, 2011 at 11:55 PM, Jeremy Chadwick >> wrote: >>>=20 >>> And before someone asks: in most cases you *cannot* use this card in = a >>> PCIe x16 connector on a motherboard. ??Most generic motherboard >>> manufacturers at this point have special one-offs that assume their = PCIe >>> x16 slots are for video cards only. ??If you aren't sure, you'll = need to >>> ask your motherboard manufacturer/vendor if you can use a >>> non-VGA-adapter in their PCIe x16 slot. ??Some Supermicro boards do = have >>> PCIe x16 slots that can be used by non-VGA adapters, but I haven't = seen >>> this on, say, Asus/Gigabyte/Dell/Intel motherboards. >>=20 >> Jeremy may not have seen PCI express x16 HBAs working on consumer >> boards, but I have, plenty of times. >=20 > I'm glad -- people should probably start making a list, because the > number of boards I've seen it not work in easily exceeds that of the > times I HAVE seen it work. :-) Where did it fail? >> I don't know why the FUD, but I have had no problems with an Intel >> SASUC8I (LSI 1068 based) running on an Asus P5Q Pro, a simple = consumer >> P45 chipset. I'm not unique, many people have success doing this. >=20 > Once in a blue moon, an intelligent board manufacturer puts comments = in > their user manual to the effect of "the PCIe x16 slot is only to be = used > for VGA adapters and will not work with non-VGA cards", or vice-versa. > Here's some hard evidence of my claim: >=20 > Supermicro X7SBA server-class motherboard user manual has the = following > line in it: >=20 > "Note: The Intel 3210 chipset does not support add-in graphics cards = in > the PCI-E interface provided by the Memory Controller Hub (MCH)." >=20 > So in that boards' case, the PCIe x16 slot (which only has PCIe x8 = worth > of lanes wired) will work with controller cards but not VGA adapters. >=20 > And here's another, for the X7DVL series boards: > http://www.supermicro.com/support/faqs/faq.cfm?faq=3D10582 >=20 > "This is a server board, and you cannot place a x8 or x16 VGA card on > this board. Only onboard VGA is possible. If you need to place a VGA > card please use X7DA(x) board". Perhaps I'm being thick, but this says "a 16x slot on this board does = not support a display card" not "a 16x slot on this board only supports = a display card." The examples you cite only support the "vice-versa" part of your claim, = not the primary claim that 16x slots on some motherboards are only for = display cards. Do you have a reference for that? >=20 > And up until last week I owned and used an Asus P5Q SE (P45-based with > ICH10 SB) board, so I can refer you to the fact that the P5Q Pro user > manual "hints" that the PCIe x16 slot is for graphics only but doesn't > downright say that. Did you try a non-graphics card in the slot to see what happened? Saying = "The 16x slot is where you put the graphics card" is probably not = unreasonable for 99% of the users of that board. Regards, Jan Mikkelsen= From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 07:57:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 209B5106564A for ; Thu, 28 Jul 2011 07:57:58 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id A05FA8FC08 for ; Thu, 28 Jul 2011 07:57:56 +0000 (UTC) Received: (qmail 28991 invoked by uid 907); 28 Jul 2011 07:57:54 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Thu, 28 Jul 2011 17:57:54 +1000 Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=windows-1252 From: Jan Mikkelsen In-Reply-To: <20110727165520.GA15633@icarus.home.lan> Date: Thu, 28 Jul 2011 17:57:52 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: <7504B95C-F76E-4817-AA83-683907B247F8@transactionware.com> References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> <20110727101234.GA7659@icarus.home.lan> <20110727165520.GA15633@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1244.3) Cc: Tom Evans , freebsd-stable@freebsd.org Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 07:57:58 -0000 Hi, On 28/07/2011, at 2:55 AM, Jeremy Chadwick wrote: >> [ =85 ] >> Perhaps I'm being thick, but this says "a 16x slot on this board does = not support a display card" not "a 16x slot on this board only supports = a display card." >>=20 >> The examples you cite only support the "vice-versa" part of your = claim, not the primary claim that 16x slots on some motherboards are = only for display cards. Do you have a reference for that? >=20 > You're not being thick. I'm simply using what's in the manual as a > hard, validated example that the *type* of card that goes into a PCIe > x16 slot can matter, and that some types are compatible while others = are > not. >=20 > I cannot find you a hard documented reference for my claims right now, > so you can consider them lies/FUD/whatever for the time being, that's > fine by me at this point. >=20 > I can find you examples on Google of people who invested in Areca > ARC-1220 cards (PCIe x8) only to find out that when inserted into one = of > their two PCIe x16 slots the mainboard wouldn't start (see above). I = can > also find you examples on Google of people with Intel 915GM chipsets > whose user manuals explicitly state the PCIe x16 slot on their board = is > "intended for use with graphics cards only". Just trying to understand; I think I can recall reading about issues = with the 915 chipset. I agree a "check, don't assume" warning is = reasonable. Regards, Jan. From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 19:24:17 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 237E91065675 for ; Thu, 28 Jul 2011 19:24:17 +0000 (UTC) (envelope-from sclark46@earthlink.net) Received: from elasmtp-spurfowl.atl.sa.earthlink.net (elasmtp-spurfowl.atl.sa.earthlink.net [209.86.89.66]) by mx1.freebsd.org (Postfix) with ESMTP id EF6E08FC12 for ; Thu, 28 Jul 2011 19:24:16 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=aS+GILRytKPh7evutzJdCYocn+rY7n52u19Tm4bz72BRgd/0RocLoJKx4zAXdrEN; h=Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [69.22.83.66] (helo=joker.seclark.com) by elasmtp-spurfowl.atl.sa.earthlink.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1QmVpO-0004qs-LR for freebsd-stable@freebsd.org; Thu, 28 Jul 2011 15:01:02 -0400 Message-ID: <4E31B1ED.1030701@earthlink.net> Date: Thu, 28 Jul 2011 15:01:01 -0400 From: Stephen Clark User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Fedora/3.0.10-1.fc12 Thunderbird/3.0.10 MIME-Version: 1.0 To: FreeBSD Stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ELNK-Trace: a437fbc6971e80f61aa676d7e74259b7b3291a7d08dfec79cb9a76bc293186900de0decbb3662939350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 69.22.83.66 Subject: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sclark46@earthlink.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 19:24:17 -0000 Hello List, Could someone enlighten me as to when FreeBSD 6.3 does UDP packet reassembly? I am having a problem where I am getting a fragmented udp packet (2 pieces) everthing is fine if I get the first frag first. but if the second frag comes first then both fragments get dropped. I am using ipfilter and a bimap to redirect these packets to a host inside of the FreeBSD box, so I suspicion it is ipfilter causing the drops. I know, I know 6.3 is ancient history, but any insight would be appreciated. Thank, Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 19:34:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 379A5106564A for ; Thu, 28 Jul 2011 19:34:51 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout028.mac.com (asmtpout028.mac.com [17.148.16.103]) by mx1.freebsd.org (Postfix) with ESMTP id 20E3F8FC17 for ; Thu, 28 Jul 2011 19:34:50 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from cswiger1.apple.com ([17.209.4.71]) by asmtp028.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LP20080M6DATP80@asmtp028.mac.com> for freebsd-stable@freebsd.org; Thu, 28 Jul 2011 12:34:23 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.4.6813,1.0.211,0.0.0000 definitions=2011-07-28_05:2011-07-28, 2011-07-28, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=1 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1107280169 From: Chuck Swiger In-reply-to: <4E31B1ED.1030701@earthlink.net> Date: Thu, 28 Jul 2011 12:34:22 -0700 Message-id: <4654E38A-B5EB-4020-BA17-B016F77FDB25@mac.com> References: <4E31B1ED.1030701@earthlink.net> To: sclark46@earthlink.net X-Mailer: Apple Mail (2.1084) Cc: FreeBSD Stable Subject: Re: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 19:34:51 -0000 On Jul 28, 2011, at 12:01 PM, Stephen Clark wrote: > Could someone enlighten me as to when FreeBSD 6.3 does UDP packet reassembly? Packet reassembly is done at the IP layer, not the UDP layer. Normally, reassembly is performed on the destination host, but routers or firewalls along the path conceivably might also reassemble packets. > I am having a problem where I am getting a fragmented udp packet (2 pieces) everthing is > fine if I get the first frag first. but if the second frag comes first then both fragments get dropped. > > I am using ipfilter and a bimap to redirect these packets to a host inside of the FreeBSD box, > so I suspicion it is ipfilter causing the drops. > > I know, I know 6.3 is ancient history, but any insight would be appreciated. It's probably the firewall dropping the traffic, yes-- running tcpdump on the firewall versus the destination host would confirm this. Something like "keep frags" on your pass rules would help if it is ipfilter... Regards, -- -Chuck From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 19:36:22 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D9561065672 for ; Thu, 28 Jul 2011 19:36:22 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id D7EEE8FC20 for ; Thu, 28 Jul 2011 19:36:21 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p6SJaJxW003919; Thu, 28 Jul 2011 15:36:19 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4E31BA23.2060505@sentex.net> Date: Thu, 28 Jul 2011 15:36:03 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: sclark46@earthlink.net References: <4E31B1ED.1030701@earthlink.net> In-Reply-To: <4E31B1ED.1030701@earthlink.net> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Cc: FreeBSD Stable Subject: Re: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 19:36:22 -0000 On 7/28/2011 3:01 PM, Stephen Clark wrote: > Hello List, > > Could someone enlighten me as to when FreeBSD 6.3 does UDP packet > reassembly? > > I am having a problem where I am getting a fragmented udp packet (2 > pieces) everthing is > fine if I get the first frag first. but if the second frag comes first > then both fragments get dropped. > > I am using ipfilter and a bimap to redirect these packets to a host > inside of the FreeBSD box, > so I suspicion it is ipfilter causing the drops. Not sure, but you try pf instead ? And use scrub log fragment reassemble ---Mike > > I know, I know 6.3 is ancient history, but any insight would be > appreciated. > > Thank, > Steve > -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 20:24:48 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 935D71065673 for ; Thu, 28 Jul 2011 20:24:48 +0000 (UTC) (envelope-from kc5vdj.freebsd@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5E05D8FC0C for ; Thu, 28 Jul 2011 20:24:48 +0000 (UTC) Received: by iyb11 with SMTP id 11so4368582iyb.13 for ; Thu, 28 Jul 2011 13:24:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=yE5iZmuWtvqRlYgAZ1Q85bfGRNlXC/IefrczwRqLbN8=; b=WeWnR0OijmQKXFX44SkbmDjGrcKPKWPQCAHWiJwHzfOQY498+oQZ/NjCTAHB8sCB5T OuomClClAVHJJmbmmZxJc8EAwex9I91YHM1VWcp4qvQym3kAo5k6CjzvXM3g98nZtAh9 oAC8B0+fuC+fDSVkDqeQyZRcJGnyrfdMmzn7c= Received: by 10.231.167.83 with SMTP id p19mr335002iby.67.1311884687752; Thu, 28 Jul 2011 13:24:47 -0700 (PDT) Received: from argus.electron-tube.net (desm-46-244.dsl.netins.net [167.142.46.244]) by mx.google.com with ESMTPS id er13sm850241ibb.2.2011.07.28.13.24.40 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 28 Jul 2011 13:24:42 -0700 (PDT) Message-ID: <4E30E35F.9020308@gmail.com> Date: Wed, 27 Jul 2011 23:19:43 -0500 From: Jim Bryant User-Agent: Thunderbird 2.0.0.24 (X11/20100911) MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <20110712151858.GA1091@faust> <4E1F554A.90405@gmail.com> <20110714205659.GB16595@libertas.local.camdensoftware.com> In-Reply-To: <20110714205659.GB16595@libertas.local.camdensoftware.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: recommendations for laptop and desktop X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 20:24:48 -0000 heh, the dv-series.... really nice exterior, but, inside, poor choices in cable routing, increased use of polymer ribbons prone to breakage and open circuit from even a single disassembly (lint/dust is the #1 laptop killer, they have to be cleaned out regularly, no less than twice a year in most environments), undocumented screws and chinese finger puzzles, undocumented cables, incorrect instructions, parts that don't exist, parts that exist but aren't documented, incorrect part numbers.... Interestingly, my primary laptops (two of them for me, and one for the lady) are Compaq C-300 series (upgraded to the 945GM motherboard and T7600 65nm Core2 Duo) are reliable to the extreme, well-documented (except for that one screw with the silk-screened arrow pointing at it in the upper left of the mobo), a complete airpath cleaning with practice can be done in about an hour with three screwdrivers, a muffin tin, pad and paper (which screws in which muffin hole), a toothbrush, unscented toilet paper (wipe the silicon to a mirror finish again), and arctic silver 5. The series is also labeled as the HP G-3000 series. The Compaq C-500 series is in the same chassis with only minor changes. The problem lies in that most of these were sold with the 940GML chipset (Celery only), but 945GM was an option and those mobos are about $100 USD +- $30 USD on eBay, mostly HannStar refurbs from China. These were introduced in 2007, and must be flashed to the most recent BIOS before putting in a Core2 Duo T7200 to T7600. My only complaint under BSD has been the Mini-PCI has bad ACPI, as the broadcom and intel driver both will panic the system at the worst, or at minimum refuse to load the driver with a "could not allocate resource". No biggie, it works under winblowz xp and 7. Under BSD, just use a USB wifi. Interestingly, under SuSE, the NDIS driver for the broadcom wifi does work. Conclusion: Get a Compaq C-300 or C-500 series system, preferably non-working (most likely a 940GML/Celery anyway), put in a good 945GM replacement board, and you will have a reliable, reasonably recent, low-cost laptop, just remember to clean it out when idle is in the mid to high 70's Celsius. The newer dv-series laptops are slick and all that on the outside, and very tempting when you play with it at the store, but they were made with the intent that people throw them away instead of having regular maintenance performed (cleaning), based on the number I have repaired, all needing new motherboards. Don't spend $800-1500 on a laptop that will fail in a year or two and be that hard to clean without having to buy replacement parts (ribbons, cable bundles, etc). STAY AWAY FROM HP dv-ANYTHING ESPECIALLY!!! THE AMD VERSIONS ARE EVEN MORE PRONE TO FAILURE!!! Chip Camden wrote: > Quoth Jim Bryant on Thursday, 14 July 2011: > >> stay away from newer hp laptops. >> >> been repairing laptops for money in recent years. HP laptops made after >> 2008 generally have serious issues with BGA lifting, and usually require >> motherboard replacement after 1-2 years. also their manuals have even >> more undocumented disassembly instructions, as well as incorrect >> disassembly instructions (use common sense and your eyes, following >> instructions will destroy hp laptops). >> >> > > Another data point: I bought an hp dv9000t in 2007, three years later the > motherboard died. I agree with avoiding HP. I like my ASUS K72F, except > that the Ironlake graphics aren't yet supported (soon, right Kostik?) > > From owner-freebsd-stable@FreeBSD.ORG Thu Jul 28 21:52:15 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7F1E106566C for ; Thu, 28 Jul 2011 21:52:15 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 2FD388FC0A for ; Thu, 28 Jul 2011 21:52:14 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p6SLqBtx012934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 29 Jul 2011 07:52:12 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id p6SLqBIG024045; Fri, 29 Jul 2011 07:52:11 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id p6SLqAMO024044; Fri, 29 Jul 2011 07:52:10 +1000 (EST) (envelope-from peter) Date: Fri, 29 Jul 2011 07:52:10 +1000 From: Peter Jeremy To: Jan Mikkelsen Message-ID: <20110728215210.GC23699@server.vk2pj.dyndns.org> References: <20110725093432.GI1202@home.opsec.eu> <20110726225557.GA97184@icarus.home.lan> <20110727101234.GA7659@icarus.home.lan> <20110727165520.GA15633@icarus.home.lan> <7504B95C-F76E-4817-AA83-683907B247F8@transactionware.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KN5l+BnMqAQyZLvT" Content-Disposition: inline In-Reply-To: <7504B95C-F76E-4817-AA83-683907B247F8@transactionware.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Tom Evans , freebsd-stable@freebsd.org, Jeremy Chadwick Subject: Re: SATA 6g 4-port non-RAID controller ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 21:52:15 -0000 --KN5l+BnMqAQyZLvT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2011-Jul-28 17:57:52 +1000, Jan Mikkelsen wro= te: >On 28/07/2011, at 2:55 AM, Jeremy Chadwick wrote: >> I can find you examples on Google of people who invested in Areca >> ARC-1220 cards (PCIe x8) only to find out that when inserted into one of >> their two PCIe x16 slots the mainboard wouldn't start (see above). I can >> also find you examples on Google of people with Intel 915GM chipsets >> whose user manuals explicitly state the PCIe x16 slot on their board is >> "intended for use with graphics cards only". > >Just trying to understand; I think I can recall reading about issues >with the 915 chipset. I agree a "check, don't assume" warning is >reasonable. I have also run into problems (wouldn't POST from memory) trying to use a NIC in the x16 slot of Dell GX620 boxes, which use an i945 chipset. --=20 Peter Jeremy --KN5l+BnMqAQyZLvT Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iEYEARECAAYFAk4x2goACgkQ/opHv/APuIdo7QCfRV8ubTbCpQgH6H97dtrHQydI amgAoKjDQXGqtl41wGj1vDO4y+Of1v8J =aZ2I -----END PGP SIGNATURE----- --KN5l+BnMqAQyZLvT-- From owner-freebsd-stable@FreeBSD.ORG Fri Jul 29 21:28:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFD7A106564A for ; Fri, 29 Jul 2011 21:28:31 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 783D88FC13 for ; Fri, 29 Jul 2011 21:28:31 +0000 (UTC) Received: by qyk30 with SMTP id 30so95946qyk.13 for ; Fri, 29 Jul 2011 14:28:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=T+zixGPc+1rrLvASX59TkqnBEc3xSOYB8sTlqpN1m6g=; b=Q+QKCEBgUacIDnVyQXNnCYlnewfrAzACfAbzWm2/FhASf6zAAT14PKs8UcSCSBdPv4 qSAD8btU3fIcExg2oHn7YFKNWAO8I/6jOD4BbOvKBIVtIJx7w4+TWomgwsY0+MDfGBuy CkOTQItb3gMW6V+2PlYz3D7geLa+ymYGOLSeQ= MIME-Version: 1.0 Received: by 10.229.215.202 with SMTP id hf10mr1424262qcb.212.1311974910894; Fri, 29 Jul 2011 14:28:30 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Fri, 29 Jul 2011 14:28:30 -0700 (PDT) In-Reply-To: <4E2E9F60.1060808@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> Date: Fri, 29 Jul 2011 14:28:30 -0700 Message-ID: From: maestro something To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2011 21:28:31 -0000 Hi, trying to do so I don't really find my way around. This is what I get when I run kgdb On startup the assert frame is #7 and the probe frame is #8. kgdb kernel.debug /var/crash/vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled - Show quoted text - Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x108 fault code = supervisor write, page not present instruction pointer = 0x20:0xc11012d7 stack pointer = 0x28:0xcd3ed9f4 frame pointer = 0x28:0xcd3eda0c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 1132 (nc) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xc09036a7 at kdb_backtrace+0x47 #1 0xc08d1a07 at panic+0x117 #2 0xc0c158c3 at trap_fatal+0x323 #3 0xc0c15bc0 at trap_pfault+0x2f0 #4 0xc0c1612a at trap+0x48a #5 0xc0bfc97c at calltrap+0x6 #6 0xc10e992b at dtrace_panic+0x1b #7 0xc10e995d at dtrace_assfail+0x2d #8 0xc10fb28c at dtrace_probe+0x135c #9 0xc1237f72 at systrace_probe+0x62 #10 0xc090f63f at syscallenter+0x47f #11 0xc0c15c14 at syscall+0x34 #12 0xc0bfca11 at Xint0x80_syscall+0x21 Uptime: 3m0s Physical memory: 239 MB Dumping 79 MB: 64 48 32 16 Reading symbols from /boot/kernel/dtraceall.ko... Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtraceall.ko Reading symbols from /boot/kernel/cyclic.ko...Reading symbols from /boot/kernel/cyclic.ko.symbols...done. done. Loaded symbols for /boot/kernel/cyclic.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /boot/kernel/dtrace.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtrace.ko Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtmalloc.ko Reading symbols from /boot/kernel/dtnfsclient.ko...Reading symbols from /boot/kernel/dtnfsclient.ko.symbols...done. done. Loaded symbols for /boot/kernel/dtnfsclient.ko Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /boot/kernel/fbt.ko.symbols...done. done. Loaded symbols for /boot/kernel/fbt.ko Reading symbols from /boot/kernel/lockstat.ko...Reading symbols from /boot/kernel/lockstat.ko.symbols...done. done. Loaded symbols for /boot/kernel/lockstat.ko Reading symbols from /boot/kernel/sdt.ko...Reading symbols from /boot/kernel/sdt.ko.symbols...done. done. Loaded symbols for /boot/kernel/sdt.ko Reading symbols from /boot/kernel/systrace.ko...Reading symbols from /boot/kernel/systrace.ko.symbols...done. done. Loaded symbols for /boot/kernel/systrace.ko Reading symbols from /boot/kernel/profile.ko...Reading symbols from /boot/kernel/profile.ko.symbols...done. done. Loaded symbols for /boot/kernel/profile.ko #0 doadump () at pcpu.h:231 231 __asm("movl %%fs:0,%0" : "=r" (td)); once I'm in the kgdb console i type where and all of a sudden the stack trace has only 7 frames... that does not seem correct. Furthermore, the "Previous frame inner to this frame (corrupt stack?)" error message is not too encuraging either. (kgdb) where #0 doadump () at pcpu.h:231 #1 0xc08d17a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 #2 0xc08d1a40 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:592 #3 0xc0c158c3 in trap_fatal (frame=0xcd3ed9b4, eva=264) at /usr/src/sys/i386/i386/trap.c:946 #4 0xc0c15bc0 in trap_pfault (frame=0xcd3ed9b4, usermode=0, eva=264) at /usr/src/sys/i386/i386/trap.c:859 #5 0xc0c1612a in trap (frame=0xcd3ed9b4) at /usr/src/sys/i386/i386/trap.c:532 #6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166 #7 0xc11012d7 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko Previous frame inner to this frame (corrupt stack?) (kgdb) what am I doing wrong and what do I have to do to get the specific assert that fails? cheers --m (sorry adriy hI only hit reply the first time not reply-all) On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon wrote: > on 26/07/2011 04:20 maestro something said the following: > > Hi, > > > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > > panics. > > > > Here is the information I could gather: > > > > let me know if I can provide more information. (i.e., i have the full > crash > > information 80+mbs handy) > > Use kgdb on the crash dump, go to the dtrace_probe frame and see which > exactly > assert fails and why. > > > Here is the backtrace: > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x108 > > fault code = supervisor write, page not present > > instruction pointer = 0x20:0xc11012d7 > > stack pointer = 0x28:0xcd3ed9f4 > > frame pointer = 0x28:0xcd3eda0c > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 1132 (nc) > > trap number = 12 > > panic: page fault > > cpuid = 0 > > KDB: stack backtrace: > > #0 0xc09036a7 at kdb_backtrace+0x47 > > #1 0xc08d1a07 at panic+0x117 > > #2 0xc0c158c3 at trap_fatal+0x323 > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > #4 0xc0c1612a at trap+0x48a > > #5 0xc0bfc97c at calltrap+0x6 > > #6 0xc10e992b at dtrace_panic+0x1b > > #7 0xc10e995d at dtrace_assfail+0x2d > > #8 0xc10fb28c at dtrace_probe+0x135c > > #9 0xc1237f72 at systrace_probe+0x62 > > #10 0xc090f63f at syscallenter+0x47f > > #11 0xc0c15c14 at syscall+0x34 > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > Uptime: 3m0s > > Physical memory: 239 MB > > Dumping 79 MB: 64 48 32 16 > > > > > > Steps To reproduce: > > > > the dtrace program (i.e., test.d) was: > > > > #!/usr/sbin/dtrace -s > > > > syscall::accept:return > > / execname == "nc" / > > { > > printf("%s accept:return\n", execname); > > ustack(); > > } > > > > % dtrace -s test.d > > > > then running > > % nc -vl 8080 > > on one shell and > > % nc localhost 8080 > > on another would make the kernel panic > > > -- > Andriy Gapon > From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 07:50:43 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2C57106564A for ; Sat, 30 Jul 2011 07:50:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1D0928FC0C for ; Sat, 30 Jul 2011 07:50:42 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA24282; Sat, 30 Jul 2011 10:50:40 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Qn4Jk-00080K-9Q; Sat, 30 Jul 2011 10:50:40 +0300 Message-ID: <4E33B7CF.90200@FreeBSD.org> Date: Sat, 30 Jul 2011 10:50:39 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 07:50:43 -0000 on 30/07/2011 00:27 maestro something said the following: > Hi, > > trying to do so I don't really find my way around. This is what I get when I run > kgdb > > On startup the assert frame is #7 and the probe frame is #8. > [snip] > kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x108 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc11012d7 > stack pointer = 0x28:0xcd3ed9f4 > frame pointer = 0x28:0xcd3eda0c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = resume, IOPL = 0 > current process = 1132 (nc) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e992b at dtrace_panic+0x1b > #7 0xc10e995d at dtrace_assfail+0x2d > #8 0xc10fb28c at dtrace_probe+0x135c > #9 0xc1237f72 at systrace_probe+0x62 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > Uptime: 3m0s > Physical memory: 239 MB > Dumping 79 MB: 64 48 32 16 [snip] > #0 doadump () at pcpu.h:231 > 231 __asm("movl %%fs:0,%0" : "=r" (td)); > > once I'm in the kgdb console i type where and all of a sudden the stack trace > has only 7 frames... that does not seem correct. Furthermore, the "Previous > frame inner to this frame (corrupt stack?)" error message is not too encuraging > either. > > (kgdb) where > #0 doadump () at pcpu.h:231 > #1 0xc08d17a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 > #2 0xc08d1a40 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:592 > #3 0xc0c158c3 in trap_fatal (frame=0xcd3ed9b4, eva=264) at > /usr/src/sys/i386/i386/trap.c:946 > #4 0xc0c15bc0 in trap_pfault (frame=0xcd3ed9b4, usermode=0, eva=264) at > /usr/src/sys/i386/i386/trap.c:859 > #5 0xc0c1612a in trap (frame=0xcd3ed9b4) at /usr/src/sys/i386/i386/trap.c:532 > #6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166 > #7 0xc11012d7 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko > Previous frame inner to this frame (corrupt stack?) > (kgdb) > > what am I doing wrong and what do I have to do to get the specific assert that > fails? I am not quite sure. Apparently there is some issue with either what information we store in a crash dump and how, or with how kgdb interprets that information, or with a combination of both... Perhaps this is a result of -O2 optimization and -O1 would improve the situation - not sure again... Meanwhile, there is something simple that you can do without much hassle: (kgdb) list *dtrace_probe+0x135c where the address is taken from the backtrace printed by panic(9). > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon > wrote: > > on 26/07/2011 04:20 maestro something said the following: > > Hi, > > > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > > panics. > > > > Here is the information I could gather: > > > > let me know if I can provide more information. (i.e., i have the full crash > > information 80+mbs handy) > > Use kgdb on the crash dump, go to the dtrace_probe frame and see which exactly > assert fails and why. > > > Here is the backtrace: > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x108 > > fault code = supervisor write, page not present > > instruction pointer = 0x20:0xc11012d7 > > stack pointer = 0x28:0xcd3ed9f4 > > frame pointer = 0x28:0xcd3eda0c > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 1132 (nc) > > trap number = 12 > > panic: page fault > > cpuid = 0 > > KDB: stack backtrace: > > #0 0xc09036a7 at kdb_backtrace+0x47 > > #1 0xc08d1a07 at panic+0x117 > > #2 0xc0c158c3 at trap_fatal+0x323 > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > #4 0xc0c1612a at trap+0x48a > > #5 0xc0bfc97c at calltrap+0x6 > > #6 0xc10e992b at dtrace_panic+0x1b > > #7 0xc10e995d at dtrace_assfail+0x2d > > #8 0xc10fb28c at dtrace_probe+0x135c > > #9 0xc1237f72 at systrace_probe+0x62 > > #10 0xc090f63f at syscallenter+0x47f > > #11 0xc0c15c14 at syscall+0x34 > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > Uptime: 3m0s > > Physical memory: 239 MB > > Dumping 79 MB: 64 48 32 16 > > > > > > Steps To reproduce: > > > > the dtrace program (i.e., test.d) was: > > > > #!/usr/sbin/dtrace -s > > > > syscall::accept:return > > / execname == "nc" / > > { > > printf("%s accept:return\n", execname); > > ustack(); > > } > > > > % dtrace -s test.d > > > > then running > > % nc -vl 8080 > > on one shell and > > % nc localhost 8080 > > on another would make the kernel panic -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 08:14:38 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E9B3106566B for ; Sat, 30 Jul 2011 08:14:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B50B98FC0A for ; Sat, 30 Jul 2011 08:14:37 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA24462; Sat, 30 Jul 2011 11:14:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Qn4gt-000812-9E; Sat, 30 Jul 2011 11:14:35 +0300 Message-ID: <4E33BD6A.4030303@FreeBSD.org> Date: Sat, 30 Jul 2011 11:14:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> In-Reply-To: <4E33B7CF.90200@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 08:14:38 -0000 on 30/07/2011 00:27 maestro something said the following: > on 30/07/2011 10:50 Andriy Gapon said the following: >> #7 0xc11012d7 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko >> > Previous frame inner to this frame (corrupt stack?) >> > (kgdb) >> > >> > what am I doing wrong and what do I have to do to get the specific assert that >> > fails? > I am not quite sure. Apparently there is some issue with either what > information we store in a crash dump and how, or with how kgdb interprets that > information, or with a combination of both... Or maybe it's because dtrace_panic_trigger is a function implemented in assembly and we do not provide enough debug information to (k)gdb in that case. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 18:19:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B10B4106566B; Sat, 30 Jul 2011 18:19:41 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC0F8FC0A; Sat, 30 Jul 2011 18:19:41 +0000 (UTC) Received: by qwc9 with SMTP id 9so3187361qwc.13 for ; Sat, 30 Jul 2011 11:19:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OlZOErIHtZI0iEpwIWUm3m4KLb8YDT4iSgMnoSWEUQs=; b=GYNzk65x8mHn7UNTYTBKPB/lAHjaJ8EJLBUJDwATPA7zQlYOcInWN3oEnGL+QdqifX GS3yFaCfU2miGy71ELY11iIYZvDRU/va+OEtC8xHjBw/CXAqpClNS4bAwo8CxM4d7RkZ G+tF8ofcbrmgi2oMUcWXXxluO1sSp4QBiYW3E= MIME-Version: 1.0 Received: by 10.229.16.200 with SMTP id p8mr2021624qca.22.1312049980584; Sat, 30 Jul 2011 11:19:40 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 11:19:40 -0700 (PDT) In-Reply-To: <4E33B7CF.90200@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> Date: Sat, 30 Jul 2011 11:19:40 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 18:19:41 -0000 On Sat, Jul 30, 2011 at 12:50 AM, Andriy Gapon wrote: > on 30/07/2011 00:27 maestro something said the following: > > Hi, > > > > trying to do so I don't really find my way around. This is what I get > when I run > > kgdb > > > > On startup the assert frame is #7 and the probe frame is #8. > > > [snip] > > kernel trap 12 with interrupts disabled > > > > I am not quite sure. Apparently there is some issue with either what > information we store in a crash dump and how, or with how kgdb interprets > that > information, or with a combination of both... > Perhaps this is a result of -O2 optimization and -O1 would improve the > situation > > I guess I have to recompile the kernel without -O then ... does that work for freebsd (I remember linux has issues i.e., does not compile without any -O) How would I do that the in the most straight forward way? > Meanwhile, there is something simple that you can do without much hassle: > (kgdb) list *dtrace_probe+0x135c > True there was not a lot hassle involved. However the result is not really satisfactory either :-) (kgdb) list *dtrace_probe+0x135c No source file for address 0xc10fb28c. The address is in accordance with the stack-trace (i.e., frame #8) > where the address is taken from the backtrace printed by panic(9). > cheers --m > > > > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon > > wrote: > > > > on 26/07/2011 04:20 maestro something said the following: > > > Hi, > > > > > > when using the ustack action on the latest FreeBSD8.2 i386 the > kernel > > > panics. > > > > > > Here is the information I could gather: > > > > > > let me know if I can provide more information. (i.e., i have the > full crash > > > information 80+mbs handy) > > > > Use kgdb on the crash dump, go to the dtrace_probe frame and see > which exactly > > assert fails and why. > > > > > Here is the backtrace: > > > > > > Fatal trap 12: page fault while in kernel mode > > > cpuid = 0; apic id = 00 > > > fault virtual address = 0x108 > > > fault code = supervisor write, page not present > > > instruction pointer = 0x20:0xc11012d7 > > > stack pointer = 0x28:0xcd3ed9f4 > > > frame pointer = 0x28:0xcd3eda0c > > > code segment = base 0x0, limit 0xfffff, type 0x1b > > > = DPL 0, pres 1, def32 1, gran 1 > > > processor eflags = resume, IOPL = 0 > > > current process = 1132 (nc) > > > trap number = 12 > > > panic: page fault > > > cpuid = 0 > > > KDB: stack backtrace: > > > #0 0xc09036a7 at kdb_backtrace+0x47 > > > #1 0xc08d1a07 at panic+0x117 > > > #2 0xc0c158c3 at trap_fatal+0x323 > > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > > #4 0xc0c1612a at trap+0x48a > > > #5 0xc0bfc97c at calltrap+0x6 > > > #6 0xc10e992b at dtrace_panic+0x1b > > > #7 0xc10e995d at dtrace_assfail+0x2d > > > #8 0xc10fb28c at dtrace_probe+0x135c > > > #9 0xc1237f72 at systrace_probe+0x62 > > > #10 0xc090f63f at syscallenter+0x47f > > > #11 0xc0c15c14 at syscall+0x34 > > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > > Uptime: 3m0s > > > Physical memory: 239 MB > > > Dumping 79 MB: 64 48 32 16 > > > > > > > > > Steps To reproduce: > > > > > > the dtrace program (i.e., test.d) was: > > > > > > #!/usr/sbin/dtrace -s > > > > > > syscall::accept:return > > > / execname == "nc" / > > > { > > > printf("%s accept:return\n", execname); > > > ustack(); > > > } > > > > > > % dtrace -s test.d > > > > > > then running > > > % nc -vl 8080 > > > on one shell and > > > % nc localhost 8080 > > > on another would make the kernel panic > > > -- > Andriy Gapon > From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 18:27:38 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 880F0106564A for ; Sat, 30 Jul 2011 18:27:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E709C8FC0C for ; Sat, 30 Jul 2011 18:27:37 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA29154; Sat, 30 Jul 2011 21:27:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QnEG7-0008Jc-Cs; Sat, 30 Jul 2011 21:27:35 +0300 Message-ID: <4E344D15.1040508@FreeBSD.org> Date: Sat, 30 Jul 2011 21:27:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 18:27:38 -0000 on 30/07/2011 21:19 maestro something said the following: > > > On Sat, Jul 30, 2011 at 12:50 AM, Andriy Gapon > wrote: > > on 30/07/2011 00:27 maestro something said the following: > > Hi, > > > > trying to do so I don't really find my way around. This is what I get when > I run > > kgdb > > > > On startup the assert frame is #7 and the probe frame is #8. > > > [snip] > > kernel trap 12 with interrupts disabled > > > > I am not quite sure. Apparently there is some issue with either what > information we store in a crash dump and how, or with how kgdb interprets that > information, or with a combination of both... > Perhaps this is a result of -O2 optimization and -O1 would improve the situation > > > I guess I have to recompile the kernel without -O then ... does that work for > freebsd (I remember linux has issues i.e., does not compile without any -O) Not sure about -O0; -O1/-O should work fine. > How would I do that the in the most straight forward way? See man make.conf: CFLAGS, etc. > Meanwhile, there is something simple that you can do without much hassle: > (kgdb) list *dtrace_probe+0x135c > > > True there was not a lot hassle involved. However the result is not really > satisfactory either :-) > > (kgdb) list *dtrace_probe+0x135c > No source file for address 0xc10fb28c. > > The address is in accordance with the stack-trace (i.e., frame #8) > > > where the address is taken from the backtrace printed by panic(9). Have you started kgdb with the correct kernel and core file? If yes, then I am out of ideas. > > > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon > > >> wrote: > > > > on 26/07/2011 04:20 maestro something said the following: > > > Hi, > > > > > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > > > panics. > > > > > > Here is the information I could gather: > > > > > > let me know if I can provide more information. (i.e., i have the > full crash > > > information 80+mbs handy) > > > > Use kgdb on the crash dump, go to the dtrace_probe frame and see which > exactly > > assert fails and why. > > > > > Here is the backtrace: > > > > > > Fatal trap 12: page fault while in kernel mode > > > cpuid = 0; apic id = 00 > > > fault virtual address = 0x108 > > > fault code = supervisor write, page not present > > > instruction pointer = 0x20:0xc11012d7 > > > stack pointer = 0x28:0xcd3ed9f4 > > > frame pointer = 0x28:0xcd3eda0c > > > code segment = base 0x0, limit 0xfffff, type 0x1b > > > = DPL 0, pres 1, def32 1, gran 1 > > > processor eflags = resume, IOPL = 0 > > > current process = 1132 (nc) > > > trap number = 12 > > > panic: page fault > > > cpuid = 0 > > > KDB: stack backtrace: > > > #0 0xc09036a7 at kdb_backtrace+0x47 > > > #1 0xc08d1a07 at panic+0x117 > > > #2 0xc0c158c3 at trap_fatal+0x323 > > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > > #4 0xc0c1612a at trap+0x48a > > > #5 0xc0bfc97c at calltrap+0x6 > > > #6 0xc10e992b at dtrace_panic+0x1b > > > #7 0xc10e995d at dtrace_assfail+0x2d > > > #8 0xc10fb28c at dtrace_probe+0x135c > > > #9 0xc1237f72 at systrace_probe+0x62 > > > #10 0xc090f63f at syscallenter+0x47f > > > #11 0xc0c15c14 at syscall+0x34 > > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > > Uptime: 3m0s > > > Physical memory: 239 MB > > > Dumping 79 MB: 64 48 32 16 > > > > > > > > > Steps To reproduce: > > > > > > the dtrace program (i.e., test.d) was: > > > > > > #!/usr/sbin/dtrace -s > > > > > > syscall::accept:return > > > / execname == "nc" / > > > { > > > printf("%s accept:return\n", execname); > > > ustack(); > > > } > > > > > > % dtrace -s test.d > > > > > > then running > > > % nc -vl 8080 > > > on one shell and > > > % nc localhost 8080 > > > on another would make the kernel panic > > > -- > Andriy Gapon > > -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 18:30:28 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A82E106564A; Sat, 30 Jul 2011 18:30:28 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1E3208FC12; Sat, 30 Jul 2011 18:30:27 +0000 (UTC) Received: by qyk38 with SMTP id 38so3178314qyk.13 for ; Sat, 30 Jul 2011 11:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DegtesGDB7hLCcbBYcsELBgoRnH9TQfhntwuo2mpGtk=; b=jEBoVLokC9bq3+xq6wZf0cVSobBWHcowYGmz8iPf7kcEKNvvel5n6I+8Kb4hpVHgjg BkO/wgAJwnzrYmP2YZueM6LHgRWix79NB+/4DRQSfVgIrCTABl0U8r6+eXldiamlKIuF mvGSr5UMFYKvKjrUSM4yeZbmm3fbUxweyFOxk= MIME-Version: 1.0 Received: by 10.229.215.202 with SMTP id hf10mr2033587qcb.212.1312050627193; Sat, 30 Jul 2011 11:30:27 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 11:30:27 -0700 (PDT) In-Reply-To: <4E344D15.1040508@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> Date: Sat, 30 Jul 2011 11:30:27 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 18:30:28 -0000 On Sat, Jul 30, 2011 at 11:27 AM, Andriy Gapon wrote: > on 30/07/2011 21:19 maestro something said the following: > > > > > > On Sat, Jul 30, 2011 at 12:50 AM, Andriy Gapon > > wrote: > > > > on 30/07/2011 00:27 maestro something said the following: > > > Hi, > > > > > > trying to do so I don't really find my way around. This is what I > get when > > I run > > > kgdb > > > > > > On startup the assert frame is #7 and the probe frame is #8. > > > > > [snip] > > > kernel trap 12 with interrupts disabled > > > > > > > I am not quite sure. Apparently there is some issue with either what > > information we store in a crash dump and how, or with how kgdb > interprets that > > information, or with a combination of both... > > Perhaps this is a result of -O2 optimization and -O1 would improve > the situation > > > > > > I guess I have to recompile the kernel without -O then ... does that work > for > > freebsd (I remember linux has issues i.e., does not compile without any > -O) > > Not sure about -O0; -O1/-O should work fine. > > > How would I do that the in the most straight forward way? > > See man make.conf: CFLAGS, etc. > Looking into that as I type. > > > Meanwhile, there is something simple that you can do without much > hassle: > > (kgdb) list *dtrace_probe+0x135c > > > > > > True there was not a lot hassle involved. However the result is not > really > > satisfactory either :-) > > > > (kgdb) list *dtrace_probe+0x135c > > No source file for address 0xc10fb28c. > > > > The address is in accordance with the stack-trace (i.e., frame #8) > > > > > > where the address is taken from the backtrace printed by panic(9). > > Have you started kgdb with the correct kernel and core file? > If yes, then I am out of ideas. > I hope so, I only recompiled the kernel once according to the DTRACE wiki instructions and I certainly only have one /var/crash/vmcore.* file. I'll try recompiling the kernel with -O1 and try again. In the meantime, I'm wondering whether I'm really the only/first one that ran into this problem or if there are people that actually successfully used the ustack() target on freebsd-8.2? cheers --m > > > > > > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon > > > > >> wrote: > > > > > > on 26/07/2011 04:20 maestro something said the following: > > > > Hi, > > > > > > > > when using the ustack action on the latest FreeBSD8.2 i386 > the kernel > > > > panics. > > > > > > > > Here is the information I could gather: > > > > > > > > let me know if I can provide more information. (i.e., i have > the > > full crash > > > > information 80+mbs handy) > > > > > > Use kgdb on the crash dump, go to the dtrace_probe frame and > see which > > exactly > > > assert fails and why. > > > > > > > Here is the backtrace: > > > > > > > > Fatal trap 12: page fault while in kernel mode > > > > cpuid = 0; apic id = 00 > > > > fault virtual address = 0x108 > > > > fault code = supervisor write, page not present > > > > instruction pointer = 0x20:0xc11012d7 > > > > stack pointer = 0x28:0xcd3ed9f4 > > > > frame pointer = 0x28:0xcd3eda0c > > > > code segment = base 0x0, limit 0xfffff, type 0x1b > > > > = DPL 0, pres 1, def32 1, gran 1 > > > > processor eflags = resume, IOPL = 0 > > > > current process = 1132 (nc) > > > > trap number = 12 > > > > panic: page fault > > > > cpuid = 0 > > > > KDB: stack backtrace: > > > > #0 0xc09036a7 at kdb_backtrace+0x47 > > > > #1 0xc08d1a07 at panic+0x117 > > > > #2 0xc0c158c3 at trap_fatal+0x323 > > > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > > > #4 0xc0c1612a at trap+0x48a > > > > #5 0xc0bfc97c at calltrap+0x6 > > > > #6 0xc10e992b at dtrace_panic+0x1b > > > > #7 0xc10e995d at dtrace_assfail+0x2d > > > > #8 0xc10fb28c at dtrace_probe+0x135c > > > > #9 0xc1237f72 at systrace_probe+0x62 > > > > #10 0xc090f63f at syscallenter+0x47f > > > > #11 0xc0c15c14 at syscall+0x34 > > > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > > > Uptime: 3m0s > > > > Physical memory: 239 MB > > > > Dumping 79 MB: 64 48 32 16 > > > > > > > > > > > > Steps To reproduce: > > > > > > > > the dtrace program (i.e., test.d) was: > > > > > > > > #!/usr/sbin/dtrace -s > > > > > > > > syscall::accept:return > > > > / execname == "nc" / > > > > { > > > > printf("%s accept:return\n", execname); > > > > ustack(); > > > > } > > > > > > > > % dtrace -s test.d > > > > > > > > then running > > > > % nc -vl 8080 > > > > on one shell and > > > > % nc localhost 8080 > > > > on another would make the kernel panic > > > > > > -- > > Andriy Gapon > > > > > > > -- > Andriy Gapon > From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 19:05:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55BD11065676; Sat, 30 Jul 2011 19:05:34 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id F032C8FC15; Sat, 30 Jul 2011 19:05:33 +0000 (UTC) Received: by qyk30 with SMTP id 30so432038qyk.13 for ; Sat, 30 Jul 2011 12:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dLN3zrGHbMKyCERm+P9Ww1JmWUp4GwMldzBt8Ly2MMc=; b=uiYfvBLTtlyoxDpg3Cketx8HPVqjBArGC5v1K8TnNYpFMwz/JV7sWW9D8Q2XJjuUIm 1AvypezPDqGvHRrdejHcrCvnDV1E2+aHN6NsX9s6XjxQlyUsgBVW24jpEKDixH7PbhDw a2oqJrvd1MLgcsClAUjE3VzlnArY5Tt8zIhdQ= MIME-Version: 1.0 Received: by 10.229.16.200 with SMTP id p8mr2043926qca.22.1312052733162; Sat, 30 Jul 2011 12:05:33 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 12:05:33 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> Date: Sat, 30 Jul 2011 12:05:33 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 19:05:34 -0000 Hi, >> Have you started kgdb with the correct kernel and core file? >> If yes, then I am out of ideas. >> > > I hope so, I only recompiled the kernel once according to the DTRACE wiki > instructions and I certainly only have one /var/crash/vmcore.* file. > > I'll try recompiling the kernel with -O1 and try again. In the meantime, > I'm wondering whether I'm really the only/first one that ran into this > problem or if there are people that actually successfully used the ustack() > target on freebsd-8.2? > I could not get the information even after recompiling the kernel here is the relevant (I think information). fb82i386# cat /etc/make.conf CFLAGS= -O (accodring to man make.conf only -O and -O2 is supported for CFLAGS anyways) kernel.debug is the newly compiled kernel (according to the timestamp) fb82i386# kgdb kernel.debug /var/crash/vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x108 fault code = supervisor write, page not present instruction pointer = 0x20:0xc1100847 stack pointer = 0x28:0xcd39a9e4 frame pointer = 0x28:0xcd39a9fc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 1060 (nc) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xc09036a7 at kdb_backtrace+0x47 #1 0xc08d1a07 at panic+0x117 #2 0xc0c158c3 at trap_fatal+0x323 #3 0xc0c15bc0 at trap_pfault+0x2f0 #4 0xc0c1612a at trap+0x48a #5 0xc0bfc97c at calltrap+0x6 #6 0xc10e99db at dtrace_panic+0x1b #7 0xc10e9a0d at dtrace_assfail+0x2d #8 0xc10fa6a6 at dtrace_probe+0xfd6 #9 0xc1237ce4 at systrace_probe+0x84 #10 0xc090f63f at syscallenter+0x47f #11 0xc0c15c14 at syscall+0x34 #12 0xc0bfca11 at Xint0x80_syscall+0x21 Uptime: 2m39s Physical memory: 239 MB Dumping 78 MB: 63 47 31 15 (kgdb) where #0 doadump () at pcpu.h:231 #1 0xc08d17a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 #2 0xc08d1a40 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:592 #3 0xc0c158c3 in trap_fatal (frame=0xcd39a9a4, eva=264) at /usr/src/sys/i386/i386/trap.c:946 #4 0xc0c15bc0 in trap_pfault (frame=0xcd39a9a4, usermode=0, eva=264) at /usr/src/sys/i386/i386/trap.c:859 #5 0xc0c1612a in trap (frame=0xcd39a9a4) at /usr/src/sys/i386/i386/trap.c:532 #6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166 #7 0xc1100847 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko Previous frame inner to this frame (corrupt stack?) (kgdb) list *dtrace_probe+0xfd6 No source file for address 0xc10fa6a6. So I'm stuck at the same point. any other ideas? cheers --m > > cheers > --m > > >> >> > >> > > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon > > >> > > >> wrote: >> > > >> > > on 26/07/2011 04:20 maestro something said the following: >> > > > Hi, >> > > > >> > > > when using the ustack action on the latest FreeBSD8.2 i386 >> the kernel >> > > > panics. >> > > > >> > > > Here is the information I could gather: >> > > > >> > > > let me know if I can provide more information. (i.e., i have >> the >> > full crash >> > > > information 80+mbs handy) >> > > >> > > Use kgdb on the crash dump, go to the dtrace_probe frame and >> see which >> > exactly >> > > assert fails and why. >> > > >> > > > Here is the backtrace: >> > > > >> > > > Fatal trap 12: page fault while in kernel mode >> > > > cpuid = 0; apic id = 00 >> > > > fault virtual address = 0x108 >> > > > fault code = supervisor write, page not present >> > > > instruction pointer = 0x20:0xc11012d7 >> > > > stack pointer = 0x28:0xcd3ed9f4 >> > > > frame pointer = 0x28:0xcd3eda0c >> > > > code segment = base 0x0, limit 0xfffff, type 0x1b >> > > > = DPL 0, pres 1, def32 1, gran 1 >> > > > processor eflags = resume, IOPL = 0 >> > > > current process = 1132 (nc) >> > > > trap number = 12 >> > > > panic: page fault >> > > > cpuid = 0 >> > > > KDB: stack backtrace: >> > > > #0 0xc09036a7 at kdb_backtrace+0x47 >> > > > #1 0xc08d1a07 at panic+0x117 >> > > > #2 0xc0c158c3 at trap_fatal+0x323 >> > > > #3 0xc0c15bc0 at trap_pfault+0x2f0 >> > > > #4 0xc0c1612a at trap+0x48a >> > > > #5 0xc0bfc97c at calltrap+0x6 >> > > > #6 0xc10e992b at dtrace_panic+0x1b >> > > > #7 0xc10e995d at dtrace_assfail+0x2d >> > > > #8 0xc10fb28c at dtrace_probe+0x135c >> > > > #9 0xc1237f72 at systrace_probe+0x62 >> > > > #10 0xc090f63f at syscallenter+0x47f >> > > > #11 0xc0c15c14 at syscall+0x34 >> > > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 >> > > > Uptime: 3m0s >> > > > Physical memory: 239 MB >> > > > Dumping 79 MB: 64 48 32 16 >> > > > >> > > > >> > > > Steps To reproduce: >> > > > >> > > > the dtrace program (i.e., test.d) was: >> > > > >> > > > #!/usr/sbin/dtrace -s >> > > > >> > > > syscall::accept:return >> > > > / execname == "nc" / >> > > > { >> > > > printf("%s accept:return\n", execname); >> > > > ustack(); >> > > > } >> > > > >> > > > % dtrace -s test.d >> > > > >> > > > then running >> > > > % nc -vl 8080 >> > > > on one shell and >> > > > % nc localhost 8080 >> > > > on another would make the kernel panic >> > >> > >> > -- >> > Andriy Gapon >> > >> > >> >> >> -- >> Andriy Gapon >> > > From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 19:11:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77F7B106567A for ; Sat, 30 Jul 2011 19:11:37 +0000 (UTC) (envelope-from sclark46@earthlink.net) Received: from elasmtp-spurfowl.atl.sa.earthlink.net (elasmtp-spurfowl.atl.sa.earthlink.net [209.86.89.66]) by mx1.freebsd.org (Postfix) with ESMTP id 4FAE98FC16 for ; Sat, 30 Jul 2011 19:11:37 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=S5yJmHpLW4d3K0CEVhGi/c4kVZcgrl9WWaNdYsRngctxbTVSc3aBCDLpX19khPkm; h=Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [69.22.83.66] (helo=joker.seclark.com) by elasmtp-spurfowl.atl.sa.earthlink.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1QnEwi-00022J-Pi for freebsd-stable@freebsd.org; Sat, 30 Jul 2011 15:11:36 -0400 Message-ID: <4E345767.5070408@earthlink.net> Date: Sat, 30 Jul 2011 15:11:35 -0400 From: Stephen Clark User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Fedora/3.0.10-1.fc12 Thunderbird/3.0.10 MIME-Version: 1.0 To: FreeBSD Stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ELNK-Trace: a437fbc6971e80f61aa676d7e74259b7b3291a7d08dfec792a8ae9b39fbf169049976921617c4783350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 69.22.83.66 X-Mailman-Approved-At: Sat, 30 Jul 2011 19:20:04 +0000 Subject: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sclark46@earthlink.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 19:11:37 -0000 Hello List, Didn't see this show up in the mailing list so I am resending. Could someone enlighten me as to when FreeBSD 6.3 does UDP packet reassembly? I am having a problem where I am getting a fragmented udp packet (2 pieces) everthing is fine if I get the first frag first. but if the second frag comes first then both fragments get dropped. I am using ipfilter and a bimap to redirect these packets to a host inside of the FreeBSD box, so I suspicion it is ipfilter causing the drops. I know, I know 6.3 is ancient history, but any insight would be appreciated. Thank, Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 19:26:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02CE31065670; Sat, 30 Jul 2011 19:26:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8A5348FC15; Sat, 30 Jul 2011 19:26:50 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6UJQklw073439 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 30 Jul 2011 22:26:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6UJQksJ032832; Sat, 30 Jul 2011 22:26:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6UJQkMU032831; Sat, 30 Jul 2011 22:26:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 30 Jul 2011 22:26:46 +0300 From: Kostik Belousov To: maestro something Message-ID: <20110730192646.GC17489@deviant.kiev.zoral.com.ua> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zKUU1INf9EXO6DqP" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 19:26:51 -0000 --zKUU1INf9EXO6DqP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jul 30, 2011 at 12:05:33PM -0700, maestro something wrote: > Hi, >=20 >=20 > >> Have you started kgdb with the correct kernel and core file? > >> If yes, then I am out of ideas. > >> > > > > I hope so, I only recompiled the kernel once according to the DTRACE wi= ki > > instructions and I certainly only have one /var/crash/vmcore.* file. > > > > I'll try recompiling the kernel with -O1 and try again. In the meantime, > > I'm wondering whether I'm really the only/first one that ran into this > > problem or if there are people that actually successfully used the usta= ck() > > target on freebsd-8.2? > > >=20 > I could not get the information even after recompiling the kernel here is > the relevant (I think information). >=20 > fb82i386# cat /etc/make.conf > CFLAGS=3D -O >=20 > (accodring to man make.conf only -O and -O2 is supported for CFLAGS anywa= ys) >=20 > kernel.debug is the newly compiled kernel (according to the timestamp) >=20 > fb82i386# kgdb kernel.debug /var/crash/vmcore.0 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd"... >=20 > Unread portion of the kernel message buffer: > kernel trap 12 with interrupts disabled >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0x108 > fault code =3D supervisor write, page not present > instruction pointer =3D 0x20:0xc1100847 > stack pointer =3D 0x28:0xcd39a9e4 > frame pointer =3D 0x28:0xcd39a9fc > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, def32 1, gran 1 > processor eflags =3D resume, IOPL =3D 0 > current process =3D 1060 (nc) > trap number =3D 12 > panic: page fault > cpuid =3D 0 > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e99db at dtrace_panic+0x1b > #7 0xc10e9a0d at dtrace_assfail+0x2d > #8 0xc10fa6a6 at dtrace_probe+0xfd6 > #9 0xc1237ce4 at systrace_probe+0x84 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > Uptime: 2m39s > Physical memory: 239 MB > Dumping 78 MB: 63 47 31 15 >=20 >=20 >=20 > (kgdb) where > #0 doadump () at pcpu.h:231 > #1 0xc08d17a3 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :419 > #2 0xc08d1a40 in panic (fmt=3DVariable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:592 > #3 0xc0c158c3 in trap_fatal (frame=3D0xcd39a9a4, eva=3D264) at > /usr/src/sys/i386/i386/trap.c:946 > #4 0xc0c15bc0 in trap_pfault (frame=3D0xcd39a9a4, usermode=3D0, eva=3D26= 4) at > /usr/src/sys/i386/i386/trap.c:859 > #5 0xc0c1612a in trap (frame=3D0xcd39a9a4) at > /usr/src/sys/i386/i386/trap.c:532 > #6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166 > #7 0xc1100847 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko > Previous frame inner to this frame (corrupt stack?) > (kgdb) list *dtrace_probe+0xfd6 > No source file for address 0xc10fa6a6. >=20 > So I'm stuck at the same point. >=20 > any other ideas? This is i386, right ? I think the cause is that assembler routine panic_trigger does not establish the standard i386 frame. Basically, you need either this, or dwarf annotations, for gdb to be able to walk over the frame. You need to add the standard prologue pushl %ebp movl %esp,%ebp and standard epilogue leave to the function. No idea whether it will continue to operate correctly after. --zKUU1INf9EXO6DqP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk40WvYACgkQC3+MBN1Mb4i2kgCfVqaQufsgM0TTwXxtLQFRhXQf Z7YAnispHHCwi5nf3Gn7iPxOcl4oDY+4 =Gmmb -----END PGP SIGNATURE----- --zKUU1INf9EXO6DqP-- From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 19:52:29 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57F4C106566B for ; Sat, 30 Jul 2011 19:52:29 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A1CD28FC12 for ; Sat, 30 Jul 2011 19:52:28 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA29779; Sat, 30 Jul 2011 22:52:26 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QnFaE-0008M9-4n; Sat, 30 Jul 2011 22:52:26 +0300 Message-ID: <4E3460F9.7090909@FreeBSD.org> Date: Sat, 30 Jul 2011 22:52:25 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 19:52:29 -0000 on 30/07/2011 22:05 maestro something said the following: > fb82i386# cat /etc/make.conf > CFLAGS= -O > > (accodring to man make.conf only -O and -O2 is supported for CFLAGS anyways) > > kernel.debug is the newly compiled kernel (according to the timestamp) > > fb82i386# kgdb kernel.debug /var/crash/vmcore.0 Is vmcore.0 also new (i.e. produced with the new kernel)? Does 'list ...' command still produces no useful output? P.S. Also, hmm, I think that you shouldn't run kgdb on kernel.debug, just to avoid any confusion... I think that you should run something like kgdb /boot/${your installed kernel}/kernel /var/crash/vmcore.0; the debug symbols are automatically picked up from *.symbols files in the kernel directory. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 20:03:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18A921065675; Sat, 30 Jul 2011 20:03:41 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id ADEAB8FC15; Sat, 30 Jul 2011 20:03:40 +0000 (UTC) Received: by qyk38 with SMTP id 38so3200965qyk.13 for ; Sat, 30 Jul 2011 13:03:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=88z+gpWU/XzDp7ARrl77mOsrVFq72Sou5qRW3P7PCqg=; b=Y2HzyKN5mlTphh2WhMIiegqi0es4VeEdEqXTSzEe0A8N+VUxb4DJlWMNLPv8+S3LlV FUxwypNY8W+6GxlatEMG2H466lKYN5mt4cO2TOJODiH/QaswvcLlwTsjkYFXskqGkpOB U+Q8yhk747U6Lbgr3d1IQmh7FZdkxpa5stZOQ= MIME-Version: 1.0 Received: by 10.229.62.150 with SMTP id x22mr2037233qch.136.1312056219747; Sat, 30 Jul 2011 13:03:39 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 13:03:39 -0700 (PDT) In-Reply-To: <4E3460F9.7090909@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <4E3460F9.7090909@FreeBSD.org> Date: Sat, 30 Jul 2011 13:03:39 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 20:03:41 -0000 Hi, On Sat, Jul 30, 2011 at 12:52 PM, Andriy Gapon wrote: > on 30/07/2011 22:05 maestro something said the following: > > fb82i386# cat /etc/make.conf > > CFLAGS= -O > > > > (accodring to man make.conf only -O and -O2 is supported for CFLAGS > anyways) > > > > kernel.debug is the newly compiled kernel (according to the timestamp) > > > > fb82i386# kgdb kernel.debug /var/crash/vmcore.0 > > Is vmcore.0 also new (i.e. produced with the new kernel)? > Does 'list ...' command still produces no useful output? > As mentioned before, the list command still gives the same error message: > > P.S. Also, hmm, I think that you shouldn't run kgdb on kernel.debug, just > to > avoid any confusion... I think that you should run something like kgdb > /boot/${your installed kernel}/kernel /var/crash/vmcore.0; the debug > symbols are > automatically picked up from *.symbols files in the kernel directory. > Same result when using /boot/kernel/kernel (kgdb) list *dtrace_probe+0xfd6 No source file for address 0xc10fa6a6. cheers --m From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 20:10:39 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A8AC1065672 for ; Sat, 30 Jul 2011 20:10:39 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 538938FC15 for ; Sat, 30 Jul 2011 20:10:38 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA29930; Sat, 30 Jul 2011 23:10:36 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QnFrn-0008Mq-TV; Sat, 30 Jul 2011 23:10:35 +0300 Message-ID: <4E34653B.6050305@FreeBSD.org> Date: Sat, 30 Jul 2011 23:10:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <4E3460F9.7090909@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 20:10:39 -0000 on 30/07/2011 23:03 maestro something said the following: > (kgdb) list *dtrace_probe+0xfd6 > No source file for address 0xc10fa6a6. Are the sources on the same machine? This is probably the last idea from me. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 20:13:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EF871065674; Sat, 30 Jul 2011 20:13:00 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id E65448FC1E; Sat, 30 Jul 2011 20:12:59 +0000 (UTC) Received: by qyk38 with SMTP id 38so3203217qyk.13 for ; Sat, 30 Jul 2011 13:12:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Muust/gWEPXMsp9x4vwjyEcUq84N/O1q5S7Iaxi7GgA=; b=M9NrTThpWAnaZYDxP/ooZtfvHfGpCdVkk/8VvODbINeJb+WzhHeybJVCN2NGGbXv0T R6qx/5zUvr91swTnq8/0yG3dTWKbE/DQlYNjZOKubzCOUj+nlQofp6MpGqArhmMQQKiH RdtU7smqUHQs9ZTxsGN5yaglyLe5j+8pmnybc= MIME-Version: 1.0 Received: by 10.229.105.72 with SMTP id s8mr2054612qco.98.1312056779235; Sat, 30 Jul 2011 13:12:59 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 13:12:59 -0700 (PDT) In-Reply-To: <4E34653B.6050305@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <4E3460F9.7090909@FreeBSD.org> <4E34653B.6050305@FreeBSD.org> Date: Sat, 30 Jul 2011 13:12:59 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 20:13:00 -0000 Hi, on 30/07/2011 23:03 maestro something said the following: > > (kgdb) list *dtrace_probe+0xfd6 > > No source file for address 0xc10fa6a6. > > Are the sources on the same machine? > This is probably the last idea from me. > Yes all is done on the same (virtual 32bit) machine. Sources are there too. If i do make installkernel KERNCONF=MYKERNEL the kernel should be installed right? (I assume so b/c /boot/kernel/kernel has the correct new timestamp) cheers -m From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 20:15:14 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 196321065677; Sat, 30 Jul 2011 20:15:14 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id B4E7E8FC12; Sat, 30 Jul 2011 20:15:13 +0000 (UTC) Received: by qyk30 with SMTP id 30so448109qyk.13 for ; Sat, 30 Jul 2011 13:15:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JFvKZEjqlfRzYFtUeGqmfb0T3op/2ff7267QFVHgCJk=; b=Awjv0y96AeMUdtRzt43Z2V5c2s1Tq0ibfA8i5sOFKNanEmzRAH7whctCs6Q+eY1aFI dA2TWIZmEu0veX9drCWGG492VgGmeJ833CfxcGY4zmAzE/lb1LZ1Gjhh9q1vmgiKDVQv 1l4O+uMqWsI+am/WEmhmFy7w9eMhQTWHYPKa8= MIME-Version: 1.0 Received: by 10.229.16.200 with SMTP id p8mr2076558qca.22.1312056913001; Sat, 30 Jul 2011 13:15:13 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 13:15:12 -0700 (PDT) In-Reply-To: <20110730192646.GC17489@deviant.kiev.zoral.com.ua> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> Date: Sat, 30 Jul 2011 13:15:12 -0700 Message-ID: From: maestro something To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 20:15:14 -0000 Hi, This is i386, right ? > I think the cause is that assembler routine panic_trigger does not > establish the standard i386 frame. Basically, you need either this, > or dwarf annotations, for gdb to be able to walk over the frame. > > You need to add the standard prologue > pushl %ebp > movl %esp,%ebp > and standard epilogue > leave > to the function. No idea whether it will continue to operate correctly > after. > my panic_trigger looks like this now: /* int panic_trigger(int *tp) */ ENTRY(panic_trigger) pushl %ebp movl %esp,%ebp xorl %eax, %eax movl $0xdefacedd, %edx lock xchgl %edx, (%edi) cmpl $0, %edx je 0f movl $0, %eax leave ret 0: movl $1, %eax leave ret END(panic_trigger) same result, (actually too same as the address in the stack trace is still the same, is that possible?) cheers --m From owner-freebsd-stable@FreeBSD.ORG Sat Jul 30 21:05:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45BDE106566B; Sat, 30 Jul 2011 21:05:34 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id DC8908FC13; Sat, 30 Jul 2011 21:05:33 +0000 (UTC) Received: by qyk38 with SMTP id 38so3215163qyk.13 for ; Sat, 30 Jul 2011 14:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sgHsQ8EOkhwgBTLgJ+Z+2sIMceQO/lMQOaY2FbPOTXg=; b=PCv9l98ii2jgMoiLHpbSW/WeagVXwta+kFKUxZZIaB6AeI6lPbXC25JoMnqY9qlQaa Bq9X29TweAwMOYJOG0Q1lq5xCKUcRMMHltfQuJEx8UKmB9VoLBeAa7sbnjLuLRLsOC2N gtuuETltHHE4etNvXROBnDR5DOPjdQhmUib0E= MIME-Version: 1.0 Received: by 10.229.22.20 with SMTP id l20mr2110875qcb.60.1312059933184; Sat, 30 Jul 2011 14:05:33 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Sat, 30 Jul 2011 14:05:33 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> Date: Sat, 30 Jul 2011 14:05:33 -0700 Message-ID: From: maestro something To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2011 21:05:34 -0000 Hi, > This is i386, right ? >> I think the cause is that assembler routine panic_trigger does not >> establish the standard i386 frame. Basically, you need either this, >> or dwarf annotations, for gdb to be able to walk over the frame. >> >> You need to add the standard prologue >> pushl %ebp >> movl %esp,%ebp >> and standard epilogue >> leave >> to the function. No idea whether it will continue to operate correctly >> after. >> > > my panic_trigger looks like this now: > > /* > int > panic_trigger(int *tp) > */ > ENTRY(panic_trigger) > > pushl %ebp > movl %esp,%ebp > xorl %eax, %eax > movl $0xdefacedd, %edx > lock > xchgl %edx, (%edi) > cmpl $0, %edx > je 0f > movl $0, %eax > leave > ret > 0: movl $1, %eax > leave > ret > END(panic_trigger) > > same result, (actually too same as the address in the stack trace is still > the same, is that possible?) > > KDB: stack backtrace: #0 0xc09036a7 at kdb_backtrace+0x47 #1 0xc08d1a07 at panic+0x117 #2 0xc0c158c3 at trap_fatal+0x323 #3 0xc0c15bc0 at trap_pfault+0x2f0 #4 0xc0c1612a at trap+0x48a #5 0xc0bfc97c at calltrap+0x6 #6 0xc10e99db at dtrace_panic+0x1b #7 0xc10e9a0d at dtrace_assfail+0x2d #8 0xc10fa6a6 at dtrace_probe+0xfd6 #9 0xc1237ce4 at systrace_probe+0x84 #10 0xc090f63f at syscallenter+0x47f #11 0xc0c15c14 at syscall+0x34 #12 0xc0bfca11 at Xint0x80_syscall+0x21 I tried playing around with kgdb a bit. It finds source files until frame #10 (i.e., syscallenter + 0x47f) (kgdb) list *syscallenter+0x47f 0xc090f63f is in syscallenter (/usr/src/sys/kern/subr_trap.c:328). 323 * If the systrace module has registered it's probe 324 * callback and if there is a probe active for the 325 * syscall 'return', process the probe. 326 */ 327 if (systrace_probe_func != NULL && sa->callp->sy_return != 0) 328 (*systrace_probe_func)(sa->callp->sy_return, sa->code, 329 sa->callp, sa->args); 330 #endif 331 CTR4(KTR_SYSC, "syscall: p=%p error=%d return %#lx %#lx", 332 p, error, td->td_retval[0], td->td_retval[1]); (kgdb) list *systrace_probe+0x84 No source file for address 0xc1237ce4. Thus, it seems that the first frame where kgdb cannot find a source file is #9 (i.e., systrace_probe + 0x84) As far as I understand that's when the (imho) function pointer systrace_probe_func gets invoked. Therefore, it seems that kgdb has a hard time finding the source file for the function invoked through that pointer. Is this complete nonsense, or does that actually sound familiar to anyone? And still I'm wondering whether anybody ever successfully used the ustack action on Freebsd-8.2 i386 cheers --m