From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 02:35:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80EBB106564A for ; Sun, 31 Jul 2011 02:35:31 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.westchester.pa.mail.comcast.net (qmta08.westchester.pa.mail.comcast.net [76.96.62.80]) by mx1.freebsd.org (Postfix) with ESMTP id 39A8A8FC0A for ; Sun, 31 Jul 2011 02:35:30 +0000 (UTC) Received: from omta23.westchester.pa.mail.comcast.net ([76.96.62.74]) by qmta08.westchester.pa.mail.comcast.net with comcast id ESaZ1h0021c6gX858SbX8R; Sun, 31 Jul 2011 02:35:31 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta23.westchester.pa.mail.comcast.net with comcast id ESbU1h0031t3BNj3jSbV8S; Sun, 31 Jul 2011 02:35:31 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B7D21102C36; Sat, 30 Jul 2011 19:35:26 -0700 (PDT) Date: Sat, 30 Jul 2011 19:35:26 -0700 From: Jeremy Chadwick To: Stephen Clark Message-ID: <20110731023526.GA39724@icarus.home.lan> References: <4E345767.5070408@earthlink.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E345767.5070408@earthlink.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Stable Subject: Re: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 02:35:31 -0000 On Sat, Jul 30, 2011 at 03:11:35PM -0400, Stephen Clark wrote: > Didn't see this show up in the mailing list so I am resending. It showed up, and people have responded. Search for "UDP Packet reassembly" and you'll see. http://lists.freebsd.org/pipermail/freebsd-stable/2011-July/thread.html -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 03:37:49 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 724C91065672 for ; Sun, 31 Jul 2011 03:37:49 +0000 (UTC) (envelope-from marka@isc.org) Received: from mx.ams1.isc.org (mx.ams1.isc.org [IPv6:2001:500:60::65]) by mx1.freebsd.org (Postfix) with ESMTP id 0F8048FC08 for ; Sun, 31 Jul 2011 03:37:49 +0000 (UTC) Received: from bikeshed.isc.org (bikeshed.isc.org [IPv6:2001:4f8:3:d::19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "bikeshed.isc.org", Issuer "ISC CA" (verified OK)) by mx.ams1.isc.org (Postfix) with ESMTPS id 397EE5F984C; Sun, 31 Jul 2011 03:37:36 +0000 (UTC) (envelope-from marka@isc.org) Received: from drugs.dv.isc.org (unknown [IPv6:2001:470:1f00:820:6233:4bff:fe01:7585]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by bikeshed.isc.org (Postfix) with ESMTPSA id 54B31216C83; Sun, 31 Jul 2011 03:37:34 +0000 (UTC) (envelope-from marka@isc.org) Received: from drugs.dv.isc.org (localhost [127.0.0.1]) by drugs.dv.isc.org (Postfix) with ESMTP id DF5241257B8D; Sun, 31 Jul 2011 13:37:31 +1000 (EST) To: sclark46@earthlink.net From: Mark Andrews References: <4E345767.5070408@earthlink.net> In-reply-to: Your message of "Sat, 30 Jul 2011 15:11:35 -0400." <4E345767.5070408@earthlink.net> Date: Sun, 31 Jul 2011 13:37:31 +1000 Message-Id: <20110731033731.DF5241257B8D@drugs.dv.isc.org> X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mx.ams1.isc.org Cc: FreeBSD Stable Subject: Re: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 03:37:49 -0000 In message <4E345767.5070408@earthlink.net>, Stephen Clark writes: > Hello List, > > Didn't see this show up in the mailing list so I am resending. > > > > Could someone enlighten me as to when FreeBSD 6.3 does UDP packet reassembly? > > I am having a problem where I am getting a fragmented udp packet (2 pieces) > everthing is > fine if I get the first frag first. but if the second frag comes first then b > oth > fragments get dropped. > > I am using ipfilter and a bimap to redirect these packets to a host inside of > > the FreeBSD box, > so I suspicion it is ipfilter causing the drops. > > I know, I know 6.3 is ancient history, but any insight would be appreciated. Know issue, http://www.FreeBSD.org/cgi/query-pr.cgi?pr=kern/82806 > Thank, > Steve -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 08:32:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A707D106566C for ; Sun, 31 Jul 2011 08:32:25 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6555B8FC0A for ; Sun, 31 Jul 2011 08:32:25 +0000 (UTC) Received: by yxl31 with SMTP id 31so3565475yxl.13 for ; Sun, 31 Jul 2011 01:32:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=h+iGiVXv5+LZh2SH3WehPqCGWjWkkt5KXPHAUWvPsWQ=; b=cykJlka2XOAsi7Egpj3VlaJV4RYQnsPfcuj9BUvLHcHST5T7uwDQ+uPuEjaDHzlmo+ hmOcM+6V9CxAAcq+BwtbbvUSw6I4XtHnURvIqgPNzTqEZ4oUTD2DGKRTi6k76bK/qSDT 4ZrC/ypXybM+wGe8TJoYmKlUeqGZJiEK7CHbk= MIME-Version: 1.0 Received: by 10.150.59.2 with SMTP id h2mr47777yba.44.1312101144058; Sun, 31 Jul 2011 01:32:24 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.151.26.14 with HTTP; Sun, 31 Jul 2011 01:32:23 -0700 (PDT) In-Reply-To: <20110714162120.U5838@sola.nimnet.asn.au> References: <1829828854.535346.1310583542523.JavaMail.root@erie.cs.uoguelph.ca> <20110714162120.U5838@sola.nimnet.asn.au> Date: Sun, 31 Jul 2011 16:32:23 +0800 X-Google-Sender-Auth: gCxPbDeip66Xnyeg-NnYDBaTO1M Message-ID: From: Adrian Chadd To: Ian Smith Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Rick Macklem , freebsd-stable@freebsd.org, Zoran Kolic Subject: Re: recommendations for laptop and desktop X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 08:32:25 -0000 Hi, If you'd like space on the freebsd wiki site, please create an account and then email me (or grab me on IRC) to sort out access. Non-developers can have access to the Wiki. I don't know why people think it's developers only. :) Adrian On 14 July 2011 17:22, Ian Smith wrote: > On Wed, 13 Jul 2011, Rick Macklem wrote: > =A0> Kevin Oberman wrote: > =A0> > On Jul 13, 2011 7:31 AM, "Zoran Kolic" wrote: > =A0> > > > =A0> > > > There is this list for laptops: > =A0> > > > http://laptop.bsdgroup.de/freebsd/ > =A0> > > > =A0> > > Been there. Seen that. Obsolete. > =A0> > > My very idea would be to have recent models in some kind > =A0> > > of wiki. I believe that at least hundred guys on the list > =A0> > > could post quality articles on the subject regarding lap- > =A0> > > tops they regurarly use. > =A0> > > > =A0> > > > See the recent thread on the freebsd-mobile list with subject > =A0> > > > "Laptop recommendations?" > =A0> > > > =A0> > > Mostly older stuff recommended. Hard to find or I dislike > =A0> > > what I see on the review for particular model. > =A0> > > Thank you for answering my question. > =A0> > > > =A0> > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Zoran > =A0> > > =A0> > I agree that a wiki would be ideal, but it would require active > =A0> > management. > =A0> > That's the real issue. > =A0> > > =A0> > It's also the reason wiki.FreeBSD.org would not be practical. I mi= ght > =A0> > be > =A0> > able to admin such a wiki, but I have no place to put it. But I'm > =A0> > retired, > =A0> > so I should have time. > =A0> > > =A0> I recently installed Fedora15 and I thought it had a fairly clever > =A0> idea in it. At the end of the install (so presumably it had worked f= or > =A0> the hardware), it asked you if you wanted to email your hardware con= fig > =A0> to them. > > I recall one Debian install doing something similar. =A0People seem to be > forgetting about bsdstats (port sysutils/bsdstats, http://bsdstats.org/) > which attempts to do something similar though it focuses more on FreeBSD > versions, hardware lists via pciconf and optionally ports lists, posting > stats monthly and anonymously .. a plus for security but perhaps a minus > for being able to seek information from people having certain hardware. > > As seen at bsdstats.org, the very much greater number of PC-BSD systems > reporting reflects the fact that PC-BSD installs bsdstats as a matter of > course; clearly the voluntary approach is far less effective, and people > don't know that bsdstats reveals no personally identifying information > until they've looked it over; many likely don't even know it exists. > > Zoran's comments that http://laptop.bsdgroup.de/freebsd/ is 'obsolete' > are a bit misplaced regarding older kit that some of us use by choice or > necessity (eg my 2.5 Thinkpad T23s :) but that site did get swamped with > spam for ages, not all of which has been removed, and there's no way to > group posts describing the same models in slightly different terms. > > Hard to get past the notion that someone needs to actively work on it, > yet it needs to be largely self-maintained by its users independent of > anyone's ongoing enthusiasm; difficult, perhaps contradictory criteria? > > =A0> Something that just captured such emails and put them in a list (esp= ecially > =A0> if could catch duplicates) for people to look at might be nice. The = list > =A0> would get long (and not really indicate how well the hardware worked= ), but > =A0> at least it would be up-to-date and not require manual maintenance. > > It could only be up to date initially, and just grow from there .. one > advantage of the bsdstats approach is that only systems that continue > reporting in monthly continue being reported. =A0One could argue about th= e > various ways that database information is presented, but any skillful > database programmer could do quite a lot with such anonymised data, > particularly if there were some identification of hardware make/model, > though that's not available in the environment of reporting programs. > > =A0> I'm not volunteering to do this;-) although I'm retired too, but it = might > =A0> be a useful thing to have? rick > > Me neither, me too, and yes indeed. =A0Just a few pent-up thoughts .. > > cheers, Ian > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 13:46:46 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D4220106566B for ; Sun, 31 Jul 2011 13:46:46 +0000 (UTC) (envelope-from ml@os2.kiev.ua) Received: from s1.sdv.com.ua (s1.sdv.com.ua [77.120.97.61]) by mx1.freebsd.org (Postfix) with ESMTP id 901638FC15 for ; Sun, 31 Jul 2011 13:46:46 +0000 (UTC) Received: from 90-105-243-80.cust.centrio.cz ([80.243.105.90] helo=[192.168.100.107]) by s1.sdv.com.ua with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QnWLp-000EV7-Bi; Sun, 31 Jul 2011 16:46:44 +0300 Message-ID: <4E355CB9.5090209@os2.kiev.ua> Date: Sun, 31 Jul 2011 15:46:33 +0200 From: Alex Samorukov User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Lightning/1.0b2 Thunderbird/3.1.11 MIME-Version: 1.0 To: maestro something References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-SA-Score: -1.0 Cc: rpaulo@freebsd.org, freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 13:46:46 -0000 Hi, I just tried your test on -STABLE + 1 fix from current [1] and got no trap. dtrace: buffer size lowered to 2m CPU ID FUNCTION:NAME 4 42224 accept:return nc accept:return Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. Abort (core dumped) Also 1st dtrace was killed as well. I tried 2 times with the same result. Box is amd64. This is bt from gdb: (gdb) bt #0 0x0000000800fa2f9c in thr_kill () from /lib/libc.so.7 #1 0x000000080103f6cb in abort () from /lib/libc.so.7 #2 0x0000000800786758 in dt_proc_lookup () from /lib/libdtrace.so.2 #3 0x00000008007867db in dt_proc_lock () from /lib/libdtrace.so.2 #4 0x00000008007a5bcb in dt_print_ustack () from /lib/libdtrace.so.2 #5 0x00000008007a8504 in dt_print_agg () from /lib/libdtrace.so.2 #6 0x00000008007a88ca in dtrace_consume () from /lib/libdtrace.so.2 #7 0x000000080077e1a2 in dtrace_work () from /lib/libdtrace.so.2 #8 0x00000000004044ba in ?? () #9 0x000000000040222e in ?? () [skip] [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/159064 On 07/26/2011 03:20 AM, maestro something wrote: > Hi, > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > panics. From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 14:32:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11CF2106564A for ; Sun, 31 Jul 2011 14:32:20 +0000 (UTC) (envelope-from niabai.torin@yandex.ru) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id D8C4A8FC08 for ; Sun, 31 Jul 2011 14:32:19 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1QnWmx-0000ed-LU for freebsd-stable@freebsd.org; Sun, 31 Jul 2011 07:14:43 -0700 Date: Sun, 31 Jul 2011 07:14:43 -0700 (PDT) From: torin To: freebsd-stable@freebsd.org Message-ID: <1312121683650-4652032.post@n5.nabble.com> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Sun, 31 Jul 2011 14:41:09 +0000 Subject: Re: root mount error X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 14:32:20 -0000 Hi! I had th=D1=83 same problem after upgrading 7.3 to 8.1. And not me only... Solution is here: http://www.bzzzz.biz/blog/freebsd/root-mount-error-after-upgrade-to-freebsd= -8.bzzzz works on 100%!!! details on this bug here: http://lists.freebsd.org/pipermail/freebsd-current/2009-January/001892.html Good luck! -- View this message in context: http://freebsd.1045724.n5.nabble.com/root-mou= nt-error-tp3939891p4652032.html Sent from the freebsd-stable mailing list archive at Nabble.com. From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 16:51:53 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40DD3106564A for ; Sun, 31 Jul 2011 16:51:53 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 92F118FC0C for ; Sun, 31 Jul 2011 16:51:52 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p6VGpj72047974 for ; Sun, 31 Jul 2011 23:51:47 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E35881C.2010505@rdtc.ru> Date: Sun, 31 Jul 2011 23:51:40 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: stable@freebsd.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 16:51:53 -0000 Hi! Suppose, there is a machine which writes two kinds of log files through syslogd: quickly-growing that need to be rotated based on their size (hourly is too seldom) and other that should be rotated once a day, at midnight only. For first kind of logs we have to run newsyslog once every 5 minutes using cron: */5 * * * * root newsyslog For second kind of logs we have lines in newsyslog.conf such as following: /var/log/mpd.log 640 16 * @T0000 JC This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only. Note, that compressing the file takes 8 minutes. However, every night at 00:05 I get an error: bzip2: I/O or other error, bailing out. Possible reason follows. bzip2: No such file or directory Input file = /var/log/mpd.log.0, output file = /var/log/mpd.log.0.bz2 newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status (1) It seems, newsyslog still wants to process my file at 00:05 despite @T0000 time specification. Is it broken? Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 17:03:34 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 719151065672 for ; Sun, 31 Jul 2011 17:03:34 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id A50E68FC0C for ; Sun, 31 Jul 2011 17:03:33 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p6VH3VxU048040; Mon, 1 Aug 2011 00:03:32 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E358ADE.20006@rdtc.ru> Date: Mon, 01 Aug 2011 00:03:26 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Freddie Cash References: <4E35881C.2010505@rdtc.ru> In-Reply-To: Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 17:03:34 -0000 31.07.2011 23:57, Freddie Cash пишет: > Simplest solution is to have two separate config files for newsyslog My question is about possibly broken newsyslog, not how about "real task", it is described for clarification only. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 17:26:56 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C630106566C for ; Sun, 31 Jul 2011 17:26:56 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3CC6C8FC16 for ; Sun, 31 Jul 2011 17:26:56 +0000 (UTC) Received: by yxl31 with SMTP id 31so3641468yxl.13 for ; Sun, 31 Jul 2011 10:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dWWvrSdNxuTWtMx4AoW+rNU048PMx+NQBlZLJ7DnR3w=; b=RmyNhRGkfIA+iQbPrVW+iOpSO3TGTNGYzsMXnrmQBHhqo/3hmc7QCkS3FFuWzjAQbW /1lFNYxDKyKAJVkdWWTLBGdeNfnIGRkgNMmBDqAfls99XXeLyF6FD+H8gey4gdE+JN3U TPLW6onZXOYUktAxSrqYEzMzGQP3SgAC0rErQ= MIME-Version: 1.0 Received: by 10.91.65.1 with SMTP id s1mr2546455agk.121.1312131425970; Sun, 31 Jul 2011 09:57:05 -0700 (PDT) Received: by 10.90.209.12 with HTTP; Sun, 31 Jul 2011 09:57:05 -0700 (PDT) In-Reply-To: <4E35881C.2010505@rdtc.ru> References: <4E35881C.2010505@rdtc.ru> Date: Sun, 31 Jul 2011 09:57:05 -0700 Message-ID: From: Freddie Cash To: Eugene Grosbein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 17:26:56 -0000 Simplest solution is to have two separate config files for newsyslog, and to edit /etc/crontab to point it at the two separate configs. One job runs every 5 mins, the other runs normally. I do this with three separate config files for different sets of logs that need to be rotated into different directories at different times. On Sun, Jul 31, 2011 at 9:51 AM, Eugene Grosbein wrote: > Hi! > > Suppose, there is a machine which writes two kinds of log files through > syslogd: > quickly-growing that need to be rotated based on their size (hourly is too > seldom) > and other that should be rotated once a day, at midnight only. > > For first kind of logs we have to run newsyslog once every 5 minutes using > cron: > > */5 * * * * root newsyslog > > For second kind of logs we have lines in newsyslog.conf such as following: > > /var/log/mpd.log 640 16 * @T0000 JC > > This must ensure that /var/log/mpd.log is rotated and compressed at midnigt > only. > Note, that compressing the file takes 8 minutes. > > However, every night at 00:05 I get an error: > > bzip2: I/O or other error, bailing out. Possible reason follows. > bzip2: No such file or directory > Input file = /var/log/mpd.log.0, output file = > /var/log/mpd.log.0.bz2 > newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status > (1) > > It seems, newsyslog still wants to process my file at 00:05 despite @T0000 > time specification. Is it broken? > > Eugene Grosbein > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 17:31:34 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC4C2106566C for ; Sun, 31 Jul 2011 17:31:34 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id 94FBC8FC0A for ; Sun, 31 Jul 2011 17:31:34 +0000 (UTC) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta01.emeryville.ca.mail.comcast.net with comcast id EhXR1h0010FhH24A1hXWqj; Sun, 31 Jul 2011 17:31:30 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta08.emeryville.ca.mail.comcast.net with comcast id EhXZ1h0041t3BNj8UhXbap; Sun, 31 Jul 2011 17:31:35 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 8506B102C36; Sun, 31 Jul 2011 10:31:29 -0700 (PDT) Date: Sun, 31 Jul 2011 10:31:29 -0700 From: Jeremy Chadwick To: Eugene Grosbein Message-ID: <20110731173129.GA53635@icarus.home.lan> References: <4E35881C.2010505@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E35881C.2010505@rdtc.ru> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 17:31:34 -0000 On Sun, Jul 31, 2011 at 11:51:40PM +0700, Eugene Grosbein wrote: > Hi! > > Suppose, there is a machine which writes two kinds of log files through syslogd: > quickly-growing that need to be rotated based on their size (hourly is too seldom) > and other that should be rotated once a day, at midnight only. > > For first kind of logs we have to run newsyslog once every 5 minutes using cron: > > */5 * * * * root newsyslog > > For second kind of logs we have lines in newsyslog.conf such as following: > > /var/log/mpd.log 640 16 * @T0000 JC > > This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only. > Note, that compressing the file takes 8 minutes. > > However, every night at 00:05 I get an error: > > bzip2: I/O or other error, bailing out. Possible reason follows. > bzip2: No such file or directory > Input file = /var/log/mpd.log.0, output file = /var/log/mpd.log.0.bz2 > newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status (1) > > It seems, newsyslog still wants to process my file at 00:05 despite @T0000 > time specification. Is it broken? I have three things to say on the matter, all of which are somewhat independent of one another so please keep that in mind. I imagine #1 below is your problem. 1) The newsyslog.conf(5) man page has this clause in it, for the "when" field (in your case, @T0000): when ... If the when field contains an asterisk (`*'), log rotation will solely depend on the contents of the size field. Otherwise, the when field consists of an optional interval in hours, usually followed by an `@'-sign and a time in restricted ISO 8601 format. If a time is specified, the log file will only be trimmed if newsyslog(8) is run within one hour of the specified time. If an interval is specified, the log file will be trimmed if that many hours have passed since the last rotation. ... You might think that "one hour of the specified time" value/clause correlates with the interval that newsyslog is run at via cron, but that would be wrong. newsyslog REALLY DOES have hard-coded values for 3600 seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I have not looked at the code, but the fact of the matter is, 1 hour appears to be a "special" value. I would heed that as a warning. 2) Are you absolutely sure mpd.log is being rotated AND compressed within the 5 minute window? If mpd.log is extremely large and your disks are slow, this could take a long time. If possible, try (temporarily) removing bzip2 from the picture (remove J flag). 3) mpd(8) logs via syslog(3). When newsyslog(8), are you aware that it sends a SIGHUP to syslogd(8)? As such, are you absolutely certain when this happen (every 5 minutes!) that the new log files are getting created correctly and promptly? 4) To debug this, you're probably going to need to run some cronjobs or daemons that keep a very close eye on /var/log/mpd.log* when the log rotation runs, in combination with running syslogd(8) in debug mode and/or verbose mode. 5) Why do you need to rotate logs every 5 minutes? Why do you need such extreme levels of granularity in your rotated logs? Just how much data are you logging via syslog? If a lot, why so much? It might be more effective to consider expanding your logging infrastructure to multiple machines if this the case. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 17:56:46 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1066B106566C for ; Sun, 31 Jul 2011 17:56:46 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 6C9678FC13 for ; Sun, 31 Jul 2011 17:56:45 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p6VHueWv048254; Mon, 1 Aug 2011 00:56:40 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E359753.3050004@rdtc.ru> Date: Mon, 01 Aug 2011 00:56:35 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jeremy Chadwick References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> In-Reply-To: <20110731173129.GA53635@icarus.home.lan> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 17:56:46 -0000 01.08.2011 00:31, Jeremy Chadwick writes: >> For second kind of logs we have lines in newsyslog.conf such as following: >> >> /var/log/mpd.log 640 16 * @T0000 JC >> >> This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only. >> Note, that compressing the file takes 8 minutes. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > I have three things to say on the matter, all of which are somewhat Five things really :-) > independent of one another so please keep that in mind. I imagine #1 > below is your problem. > > 1) The newsyslog.conf(5) man page has this clause in it, for the "when" > field (in your case, @T0000): > > when ... If the when field contains an asterisk (`*'), log rotation > will solely depend on the contents of the size field. Otherwise, > the when field consists of an optional interval in hours, usually > followed by an `@'-sign and a time in restricted ISO 8601 format. > > If a time is specified, the log file will only be trimmed if > newsyslog(8) is run within one hour of the specified time. If an > interval is specified, the log file will be trimmed if that many > hours have passed since the last rotation. ... > > You might think that "one hour of the specified time" value/clause > correlates with the interval that newsyslog is run at via cron, but that > would be wrong. newsyslog REALLY DOES have hard-coded values for 3600 > seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I > have not looked at the code, but the fact of the matter is, 1 hour > appears to be a "special" value. I would heed that as a warning. > > 2) Are you absolutely sure mpd.log is being rotated AND compressed within > the 5 minute window? If mpd.log is extremely large and your disks are > slow, this could take a long time. If possible, try (temporarily) > removing bzip2 from the picture (remove J flag). I've noted (see above) that compression takes 8 minutes. I just think newsyslog should not deal with the file at 00:05. > 3) mpd(8) logs via syslog(3). When newsyslog(8), are you aware that it > sends a SIGHUP to syslogd(8)? As such, are you absolutely certain when > this happen (every 5 minutes!) that the new log files are getting > created correctly and promptly? I see no other problems. > 4) To debug this, you're probably going to need to run some cronjobs or > daemons that keep a very close eye on /var/log/mpd.log* when the log > rotation runs, in combination with running syslogd(8) in debug mode > and/or verbose mode. syslogd or newsyslo needs debug mode? > 5) Why do you need to rotate logs every 5 minutes? Why do you need such > extreme levels of granularity in your rotated logs? Just how much data > are you logging via syslog? If a lot, why so much? It might be more > effective to consider expanding your logging infrastructure to multiple > machines if this the case. Most of my boxes are diskless NanoBSD installations having /var in memory and I need very detailed debug logs that grow quickly. These logs can easily overflow /var partition in case of network problems (storms etc.) so newsyslog have to check them often. And I have another router that has an HDD to keep daily log and I'd like to have their crontabs unified. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 19:31:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 419E110656FC for ; Sun, 31 Jul 2011 19:31:58 +0000 (UTC) (envelope-from sclark46@earthlink.net) Received: from elasmtp-curtail.atl.sa.earthlink.net (elasmtp-curtail.atl.sa.earthlink.net [209.86.89.64]) by mx1.freebsd.org (Postfix) with ESMTP id 151778FC15 for ; Sun, 31 Jul 2011 19:31:57 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=XWoZnFXfkyvuVgMGeJk2B2OcStjr1Ldcj2le4pgn8OINPeMksY/gxoyNJG7wgFIo; h=Received:Message-ID:Date:From:Reply-To:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [69.22.83.66] (helo=joker.seclark.com) by elasmtp-curtail.atl.sa.earthlink.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1Qnbjx-0005HV-9b; Sun, 31 Jul 2011 15:31:57 -0400 Message-ID: <4E35ADAB.1020105@earthlink.net> Date: Sun, 31 Jul 2011 15:31:55 -0400 From: Stephen Clark User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Fedora/3.0.10-1.fc12 Thunderbird/3.0.10 MIME-Version: 1.0 To: Mark Andrews References: <4E345767.5070408@earthlink.net> <20110731033731.DF5241257B8D@drugs.dv.isc.org> In-Reply-To: <20110731033731.DF5241257B8D@drugs.dv.isc.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ELNK-Trace: a437fbc6971e80f61aa676d7e74259b7b3291a7d08dfec79b58016d279eee997abc98d31cc1d0e24350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 69.22.83.66 Cc: FreeBSD Stable Subject: Re: UDP Packet reassembly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sclark46@earthlink.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 19:31:58 -0000 On 07/30/2011 11:37 PM, Mark Andrews wrote: > In message<4E345767.5070408@earthlink.net>, Stephen Clark writes: > >> Hello List, >> >> Didn't see this show up in the mailing list so I am resending. >> >> >> >> Could someone enlighten me as to when FreeBSD 6.3 does UDP packet reassembly? >> >> I am having a problem where I am getting a fragmented udp packet (2 pieces) >> everthing is >> fine if I get the first frag first. but if the second frag comes first then b >> oth >> fragments get dropped. >> >> I am using ipfilter and a bimap to redirect these packets to a host inside of >> >> the FreeBSD box, >> so I suspicion it is ipfilter causing the drops. >> >> I know, I know 6.3 is ancient history, but any insight would be appreciated. >> > Know issue, http://www.FreeBSD.org/cgi/query-pr.cgi?pr=kern/82806 > > >> Thank, >> Steve >> Thanks to everbody that responded. -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) From owner-freebsd-stable@FreeBSD.ORG Sun Jul 31 22:43:10 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7A33106564A for ; Sun, 31 Jul 2011 22:43:10 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4BE5F8FC0C for ; Sun, 31 Jul 2011 22:43:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.4/8.14.4) with ESMTP id p6VMWmtH095284 for ; Mon, 1 Aug 2011 02:32:48 +0400 (MSD) (envelope-from marck@rinet.ru) Date: Mon, 1 Aug 2011 02:32:48 +0400 (MSD) From: Dmitry Morozovsky To: freebsd-stable@FreeBSD.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (woozle.rinet.ru [0.0.0.0]); Mon, 01 Aug 2011 02:32:48 +0400 (MSD) Cc: Subject: stable/8 nfsd: is it normal to have one worker regardless of -n setting? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 22:43:10 -0000 Dear colleagues. just noticed that contemporary nfsd does not fork children in accordance to -n setting: stable/8: root@beaver:/usr/local/tb/scripts# pid nfs 1745 ?? Is 0:00.02 nfsd: master (nfsd) 1746 ?? S 0:03.29 nfsd: server (nfsd) root@beaver:/usr/local/tb/scripts# grep nfs_server_flags /etc/rc.conf nfs_server_flags="-u -t -n 4" stable/7: marck@kucha:~> pid nfs 654 ?? Is 0:00.01 nfsd: master (nfsd) 655 ?? I 0:05.64 nfsd: server (nfsd) 656 ?? I 0:00.00 nfsd: server (nfsd) 657 ?? I 0:00.00 nfsd: server (nfsd) 658 ?? I 0:00.00 nfsd: server (nfsd) marck@kucha:~> grep nfs_server_flags /etc/rc.conf marck@kucha:~> grep nfs_server_flags /etc/defaults/rc.conf nfs_server_flags="-u -t -n 4" # Flags to nfsd (if enabled). is it normal/expected? Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 03:02:39 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38603106564A for ; Mon, 1 Aug 2011 03:02:39 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id DF72D8FC0A for ; Mon, 1 Aug 2011 03:02:38 +0000 (UTC) Received: by yic13 with SMTP id 13so4133787yic.13 for ; Sun, 31 Jul 2011 20:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=zKu4XnttCOr7JRtJrXVL/7WEOVKsdvC79OdVD5EZtt0=; b=oTJ4W01d7X5rEqY8ROwTU1MjnowBFI3ddv3YK+HWIza/b10H4JEFqacV68E+4PZhb6 bQwX58VGFrLmsJepv60Y5eSHiZzXSWM813nVYvuJx6wIRkIJY8vxmmDRY/XD17PkGiH6 FeSIrehOYOhm/w+yGymScpjDHG7Mt4cgwKFiw= Received: by 10.231.117.79 with SMTP id p15mr2686296ibq.29.1312166394519; Sun, 31 Jul 2011 19:39:54 -0700 (PDT) Received: from DataIX.net ([99.56.120.66]) by mx.google.com with ESMTPS id uz2sm3029536icb.12.2011.07.31.19.39.52 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 31 Jul 2011 19:39:52 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id p712dnN3003940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 31 Jul 2011 22:39:49 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id p712dmkd003939; Sun, 31 Jul 2011 22:39:48 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Sun, 31 Jul 2011 22:39:48 -0400 From: Jason Hellenthal To: Eugene Grosbein Message-ID: <20110801023948.GA77144@DataIX.net> References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E359753.3050004@rdtc.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="k1lZvvs/B4yU6o8G" Content-Disposition: inline In-Reply-To: <4E359753.3050004@rdtc.ru> Cc: stable@freebsd.org, Jeremy Chadwick Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 03:02:39 -0000 --k1lZvvs/B4yU6o8G Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable What line of the newsyslog.conf file is your line inserted and can you move that to a higher line number. FIFO On Mon, Aug 01, 2011 at 12:56:35AM +0700, Eugene Grosbein wrote: > 01.08.2011 00:31, Jeremy Chadwick writes: >=20 > >> For second kind of logs we have lines in newsyslog.conf such as follow= ing: > >> > >> /var/log/mpd.log 640 16 * @T0000 JC > >> > >> This must ensure that /var/log/mpd.log is rotated and compressed at mi= dnigt only. > >> Note, that compressing the file takes 8 minutes. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >=20 > > I have three things to say on the matter, all of which are somewhat >=20 > Five things really :-) >=20 > > independent of one another so please keep that in mind. I imagine #1 > > below is your problem. > >=20 > > 1) The newsyslog.conf(5) man page has this clause in it, for the "when" > > field (in your case, @T0000): > >=20 > > when ... If the when field contains an asterisk (`*'), log rot= ation > > will solely depend on the contents of the size field. Oth= erwise, > > the when field consists of an optional interval in hours, = usually > > followed by an `@'-sign and a time in restricted ISO 8601 = format. > >=20 > > If a time is specified, the log file will only be trimmed = if > > newsyslog(8) is run within one hour of the specified time.= If an > > interval is specified, the log file will be trimmed if tha= t many > > hours have passed since the last rotation. ... > >=20 > > You might think that "one hour of the specified time" value/clause > > correlates with the interval that newsyslog is run at via cron, but that > > would be wrong. newsyslog REALLY DOES have hard-coded values for 3600 > > seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I > > have not looked at the code, but the fact of the matter is, 1 hour > > appears to be a "special" value. I would heed that as a warning. > >=20 > > 2) Are you absolutely sure mpd.log is being rotated AND compressed with= in > > the 5 minute window? If mpd.log is extremely large and your disks are > > slow, this could take a long time. If possible, try (temporarily) > > removing bzip2 from the picture (remove J flag). >=20 > I've noted (see above) that compression takes 8 minutes. > I just think newsyslog should not deal with the file at 00:05. >=20 > > 3) mpd(8) logs via syslog(3). When newsyslog(8), are you aware that it > > sends a SIGHUP to syslogd(8)? As such, are you absolutely certain when > > this happen (every 5 minutes!) that the new log files are getting > > created correctly and promptly? >=20 > I see no other problems. >=20 > > 4) To debug this, you're probably going to need to run some cronjobs or > > daemons that keep a very close eye on /var/log/mpd.log* when the log > > rotation runs, in combination with running syslogd(8) in debug mode > > and/or verbose mode. >=20 > syslogd or newsyslo needs debug mode? >=20 > > 5) Why do you need to rotate logs every 5 minutes? Why do you need such > > extreme levels of granularity in your rotated logs? Just how much data > > are you logging via syslog? If a lot, why so much? It might be more > > effective to consider expanding your logging infrastructure to multiple > > machines if this the case. >=20 > Most of my boxes are diskless NanoBSD installations having /var in memory > and I need very detailed debug logs that grow quickly. These logs > can easily overflow /var partition in case of network problems (storms et= c.) > so newsyslog have to check them often. >=20 > And I have another router that has an HDD to keep daily log and I'd like > to have their crontabs unified. >=20 > Eugene Grosbein >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" --k1lZvvs/B4yU6o8G Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJONhH0AAoJEJBXh4mJ2FR+mCEIAJnZNnh9S5R8c6mR1PMEuxMY GHgaNU5no0mSm5X2w3k0+4jxJ2L4PM53QILNb/A/RCvCd5/fiMbNNmtZgQ1Yatm+ cdH0pQINYnKH5BWeKqWkKcwK7PjGnWRb71kgoxiLIik/eDFpZjNo//Y8X5BcnSAQ GkeIRC50CNaf9vd+KnDDV1WkBVJsbWLCAI4ewSezsW8r/4ZxsNM3zVdO5780GbUf FEnoZfzoEoex/fuEQ80iEXn1xwMCnSXYGMVt9UqMAX4qgliAjpkvP0Ok5gXdJ9p5 A9g+37WMX5qDlZxuO7zfi5ZZ5dHl1YU/koZFUdakbvELgojOLM3FUoj64HQTK60= =0yyt -----END PGP SIGNATURE----- --k1lZvvs/B4yU6o8G-- From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 03:16:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A292106566B for ; Mon, 1 Aug 2011 03:16:24 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 254668FC12 for ; Mon, 1 Aug 2011 03:16:23 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id p712mWbr010645 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 31 Jul 2011 21:48:33 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.5/8.14.5) with ESMTP id p712mVZe097249 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 31 Jul 2011 21:48:31 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.5/8.14.5/Submit) id p712mVWW097248; Sun, 31 Jul 2011 21:48:31 -0500 (CDT) (envelope-from dan) Date: Sun, 31 Jul 2011 21:48:31 -0500 From: Dan Nelson To: Dmitry Morozovsky Message-ID: <20110801024830.GC59252@dan.emsphone.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 8.2-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Sun, 31 Jul 2011 21:48:33 -0500 (CDT) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: freebsd-stable@freebsd.org Subject: Re: stable/8 nfsd: is it normal to have one worker regardless of -n setting? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 03:16:24 -0000 In the last episode (Aug 01), Dmitry Morozovsky said: > just noticed that contemporary nfsd does not fork children in accordance > to -n setting: > > stable/8: > > root@beaver:/usr/local/tb/scripts# pid nfs > 1745 ?? Is 0:00.02 nfsd: master (nfsd) > 1746 ?? S 0:03.29 nfsd: server (nfsd) > root@beaver:/usr/local/tb/scripts# grep nfs_server_flags /etc/rc.conf > nfs_server_flags="-u -t -n 4" They are threads now: # ps axw | grep nfsd 1373 ?? Is 0:00.02 nfsd: master (nfsd) 1374 ?? S 5:47.14 nfsd: server (nfsd) # ps axwH | grep nfsd 1373 ?? Is 0:00.02 nfsd: master (nfsd) 1374 ?? S 1:25.79 nfsd: server (nfsd) 1374 ?? S 1:26.65 nfsd: server (nfsd) 1374 ?? S 1:27.67 nfsd: server (nfsd) 1374 ?? S 1:27.04 nfsd: server (nfsd) -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 04:20:57 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43E9B106564A for ; Mon, 1 Aug 2011 04:20:57 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 8929F8FC13 for ; Mon, 1 Aug 2011 04:20:56 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p714KeSM052029; Mon, 1 Aug 2011 11:20:41 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E362993.4050907@rdtc.ru> Date: Mon, 01 Aug 2011 11:20:35 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jason Hellenthal References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E359753.3050004@rdtc.ru> <20110801023948.GA77144@DataIX.net> In-Reply-To: <20110801023948.GA77144@DataIX.net> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 04:20:57 -0000 01.08.2011 09:39, Jason Hellenthal пишет: > > What line of the newsyslog.conf file is your line inserted and can you > move that to a higher line number. FIFO I don't get why position of line in newsyslog.conf can have any meaning. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 07:21:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A175106564A for ; Mon, 1 Aug 2011 07:21:57 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id A8BA88FC08 for ; Mon, 1 Aug 2011 07:21:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.4/8.14.4) with ESMTP id p717LmV3012990; Mon, 1 Aug 2011 11:21:48 +0400 (MSD) (envelope-from marck@rinet.ru) Date: Mon, 1 Aug 2011 11:21:48 +0400 (MSD) From: Dmitry Morozovsky To: Dan Nelson In-Reply-To: <20110801024830.GC59252@dan.emsphone.com> Message-ID: References: <20110801024830.GC59252@dan.emsphone.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (woozle.rinet.ru [0.0.0.0]); Mon, 01 Aug 2011 11:21:48 +0400 (MSD) Cc: freebsd-stable@freebsd.org Subject: Re: stable/8 nfsd: is it normal to have one worker regardless of -n setting? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 07:21:57 -0000 On Sun, 31 Jul 2011, Dan Nelson wrote: > In the last episode (Aug 01), Dmitry Morozovsky said: > > just noticed that contemporary nfsd does not fork children in accordance > > to -n setting: > > > > stable/8: > > > > root@beaver:/usr/local/tb/scripts# pid nfs > > 1745 ?? Is 0:00.02 nfsd: master (nfsd) > > 1746 ?? S 0:03.29 nfsd: server (nfsd) > > root@beaver:/usr/local/tb/scripts# grep nfs_server_flags /etc/rc.conf > > nfs_server_flags="-u -t -n 4" > > They are threads now: > > # ps axw | grep nfsd > 1373 ?? Is 0:00.02 nfsd: master (nfsd) > 1374 ?? S 5:47.14 nfsd: server (nfsd) > # ps axwH | grep nfsd > 1373 ?? Is 0:00.02 nfsd: master (nfsd) > 1374 ?? S 1:25.79 nfsd: server (nfsd) > 1374 ?? S 1:26.65 nfsd: server (nfsd) > 1374 ?? S 1:27.67 nfsd: server (nfsd) > 1374 ?? S 1:27.04 nfsd: server (nfsd) Ah, yeah, how dumb is me, heh. Thank you for the pointer, and sorry for the noise. -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 18:10:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAAF61065680; Mon, 1 Aug 2011 18:10:23 +0000 (UTC) (envelope-from kensmith@buffalo.edu) Received: from localmailB.acsu.buffalo.edu (localmail.buffalo.edu [128.205.5.200]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA7D8FC0C; Mon, 1 Aug 2011 18:10:23 +0000 (UTC) Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 3E8C7597A; Mon, 1 Aug 2011 14:02:10 -0400 (EDT) Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailB.acsu.buffalo.edu (Postfix) with ESMTP id D5D4E5D48; Mon, 1 Aug 2011 14:02:08 -0400 (EDT) Received: from smtp2.acsu.buffalo.edu (smtp2.acsu.buffalo.edu [128.205.5.254]) by localmailB.acsu.buffalo.edu (Prefixe) with ESMTP id CFC59597A; Mon, 1 Aug 2011 14:02:08 -0400 (EDT) Received: from [128.205.32.76] (bauer.cse.buffalo.edu [128.205.32.76]) (Authenticated sender: kensmith@buffalo.edu) by smtp2.acsu.buffalo.edu (Postfix) with ESMTPSA id C417F42ACC; Mon, 1 Aug 2011 14:02:08 -0400 (EDT) From: Ken Smith To: freebsd-current , freebsd-stable Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-NOMhOCw3Pe/vS7u0ueYW" Date: Mon, 01 Aug 2011 14:02:07 -0400 Message-ID: <1312221727.85876.27.camel@bauer.cse.buffalo.edu> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-PM-EL-Spam-Prob: : 8% Cc: Subject: FreeBSD 9.0-BETA1 Available... X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 18:10:23 -0000 --=-NOMhOCw3Pe/vS7u0ueYW Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The first BETA build of the 9.0-RELEASE release cycle is now available. Since this will be the first release on a brand new branch I'll cross-post the announcements on both -current and -stable. But just so you know most of the developers active in head pay more attention to the -current mailing list. If you notice problems you can report them through the normal Gnats PR system or on the -current mailing list. The 9.0-RELEASE cycle will be tracked here: http://wiki.freebsd.org/Releng/9.0TODO Those of you familiar with FreeBSD release schedules will not be shocked to notice we're already a bit behind schedule... :-) ISO images for the following architectures are available: amd64, i386, ia64, powerpc, powerpc64, and sparc64 Due to lack of support for boot floppies and issues with the new ATA_CAM infrastructure the pc98 architecture is being dropped to Tier-2 status and at this time there are no plans for 9.0 builds for pc98. I just got the sparc64 architecture ISOs loaded on ftp-master shortly before starting to type this so it may take a day or so to propagate out to the mirror sites. MD5/SHA256 checksums for the images are at the bottom of this message. If you would like to use csup/cvsup mechanisms to access the source tree the branch tag to use is "." (head). If you would like to access the source tree via SVN it is "svn://svn.freebsd.org/base/head/". Note that some time between now and BETA2 we will finalize the list of shared libraries that need to have their versions bumped (only a subset of libraries have been converted to symbol-versioning) and take care of doing the bumps. So at the moment packages built for current should work. But when the bump happens you will need to recompile any applications you have installed and there will likely be a delay for suitable pre-built packages to become available on the FTP sites. Sorry in advance for that hassle but it's somewhat normal for the testing phase of a .0 release. One of the many new features in 9.0 we would like tested is the new installer, so fresh installs on test systems are encouraged. At this time FreeBSD-Update is not available, in part because of the above issues (libs bump still coming, and encouraging testing of the installer... :-). Checksums: MD5 (FreeBSD-9.0-BETA1-amd64-bootonly.iso) =3D 8735758ac97a54289eb9697e4e91= b4f6 MD5 (FreeBSD-9.0-BETA1-amd64-disc1.iso) =3D 0f66b7f1cf9f0841bcbb2a448f0d147= 6 MD5 (FreeBSD-9.0-BETA1-amd64-memstick.img) =3D be053143500e39703543d8975fab= a7c6 MD5 (FreeBSD-9.0-BETA1-i386-bootonly.iso) =3D e1e2cb4ab7213fc458a2eabd8c725= 63c MD5 (FreeBSD-9.0-BETA1-i386-disc1.iso) =3D 61467b53bfa327b4af1deb9426d76f0f MD5 (FreeBSD-9.0-BETA1-i386-memstick.img) =3D 5637bcb95e2f7d555963ad219fe44= 12f MD5 (FreeBSD-9.0-BETA1-ia64-bootonly.iso) =3D a14c604135d600018aa0531ae4d3f= a31 MD5 (FreeBSD-9.0-BETA1-ia64-memstick) =3D a270eb61a73407f040f1f205c62b3881 MD5 (FreeBSD-9.0-BETA1-ia64-release.iso) =3D a697c4bc92600bd9e8978be35f61e7= ea MD5 (FreeBSD-9.0-BETA1-powerpc-bootonly.iso) =3D 089f0ac6aec8706d99bd07a51f= d3eb51 MD5 (FreeBSD-9.0-BETA1-powerpc-memstick) =3D 4b772ad3cf933d1bb5f6e70faee3d6= ca MD5 (FreeBSD-9.0-BETA1-powerpc-release.iso) =3D 239be45712e827bb3b3bcff91ad= 5f201 MD5 (FreeBSD-9.0-BETA1-powerpc64-bootonly.iso) =3D 5acab3ab0388b0556209e41f= b31dfbba MD5 (FreeBSD-9.0-BETA1-powerpc64-memstick) =3D de1a4e5b96c13d3b68131257ebb4= 99dc MD5 (FreeBSD-9.0-BETA1-powerpc64-release.iso) =3D d4dfefcbfc7580318a2aaca1a= 545d585 MD5 (FreeBSD-9.0-BETA1-sparc64-bootonly.iso) =3D c1020e3f52f5f760ef2a5ed20c= edd574 MD5 (FreeBSD-9.0-BETA1-sparc64-disc1.iso) =3D 8b8def377862103d8bca3fdf921ac= 38a SHA256 (FreeBSD-9.0-BETA1-amd64-bootonly.iso) =3D f5e4cf07a86d27381f961d63d= 1b6b2f224e52eb3bacc7ed3435605bf384722bb SHA256 (FreeBSD-9.0-BETA1-amd64-disc1.iso) =3D 27c4d6c85c8bb53ee7284869b9ab= c694d43d0d5110f4faf3d6057d766da5e984 SHA256 (FreeBSD-9.0-BETA1-amd64-memstick.img) =3D bdab308a347805935409f592a= 5bdca1e8288953fe0ae07d7cdce1965c4af7fb2 SHA256 (FreeBSD-9.0-BETA1-i386-bootonly.iso) =3D 0b0f13c2eebd779241ad491201= 97bc346e7ff2950e35e26832cfe7498f219bbe SHA256 (FreeBSD-9.0-BETA1-i386-disc1.iso) =3D 21099d74030500b3ca89d648c611c= f74c79a841558a7827740645e82be8901e5 SHA256 (FreeBSD-9.0-BETA1-i386-memstick.img) =3D e333fa7721f153f523d11e8af4= 87bc6c442197b3df31b67111d3b1b5ac57f18a SHA256 (FreeBSD-9.0-BETA1-ia64-bootonly.iso) =3D 8d7257cededc33bf2c62fe29d8= 36efa56375bfe0a5c3d219fe895894afcea9c1 SHA256 (FreeBSD-9.0-BETA1-ia64-memstick) =3D 9c05bf4d69f4aa2929db3e252c6ae1= 5f9e957542840e5b090ae6cdb26ff1ec98 SHA256 (FreeBSD-9.0-BETA1-ia64-release.iso) =3D 656c604d902d3a9ad1697e4dea7= 8a76740f899b7e2414c29905ffd1640260cb5 SHA256 (FreeBSD-9.0-BETA1-powerpc-bootonly.iso) =3D 8b2dacacd0fbeba98b63d41= f022335803c4cebb360e3c9a968cc9aea6bed059f SHA256 (FreeBSD-9.0-BETA1-powerpc-memstick) =3D ef7741c0d8431db886ef79c7acd= 03c7efdf281d1d5fbba634bdba371d22ef72c SHA256 (FreeBSD-9.0-BETA1-powerpc-release.iso) =3D 662eff3463ec9a05bc1617f1= af620ad5bbeac239668af3dd0a8405d79ef6d05f SHA256 (FreeBSD-9.0-BETA1-powerpc64-bootonly.iso) =3D 7e4a8a24cd942222d4fce= 60294d444525d9aea6c8d3e0ee55e6a9dda84a5ee9f SHA256 (FreeBSD-9.0-BETA1-powerpc64-memstick) =3D ec7eb9d6f5a69a0f3fbafd433= ac309dcafd411921c4c51f2886c39649d59ca07 SHA256 (FreeBSD-9.0-BETA1-powerpc64-release.iso) =3D 2721be511db390a95ddf30= 81afd8152be14f147d5d8a8213c4045875631950e7 SHA256 (FreeBSD-9.0-BETA1-sparc64-bootonly.iso) =3D 3db8cf663cd1722488277c5= c63f8daa2cd04a1e1aeb8bd54932cbc30d507bc3e SHA256 (FreeBSD-9.0-BETA1-sparc64-disc1.iso) =3D c051b5b9fad2e3183c595b3e7f= 786afb181bce91c7ce1c869a2c593ab9d29205 --=20 Ken Smith - From there to here, from here to | kensmith@buffalo.edu there, funny things are everywhere. | - Theodor Geisel | --=-NOMhOCw3Pe/vS7u0ueYW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk426hcACgkQ/G14VSmup/aHEwCfVR6YLDRgTjFndtxtOK+yFmVo WecAn0MWFm5etNEh7YX7vMFAWwzT92a5 =6vnZ -----END PGP SIGNATURE----- --=-NOMhOCw3Pe/vS7u0ueYW-- From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 22:27:57 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5732E106566B for ; Mon, 1 Aug 2011 22:27:57 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (mail.ip6.digiware.nl [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id B27E28FC19 for ; Mon, 1 Aug 2011 22:27:56 +0000 (UTC) Received: from rack1.digiware.nl (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id 27C4C153434 for ; Tue, 2 Aug 2011 00:27:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CJMjxxv03W7q for ; Tue, 2 Aug 2011 00:27:52 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:c02b:ce62:71ff:9cbc] (unknown [IPv6:2001:4cb8:3:1:c02b:ce62:71ff:9cbc]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.digiware.nl (Postfix) with ESMTPSA id EAB28153433 for ; Tue, 2 Aug 2011 00:27:51 +0200 (CEST) Message-ID: <4E37286D.5070203@digiware.nl> Date: Tue, 02 Aug 2011 00:27:57 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: stable@freebsd.org X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: em0 timeout disconnects server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 22:27:57 -0000 A server just all of a sudden dropped from the network. uptime was 26days. This got my ZFS server hanging: Aug 1 23:39:58 zfs kernel: em0: Watchdog timeout -- resetting Aug 1 23:39:58 zfs kernel: em0: Queue(0) tdh = 942, hw tdt = 977 Aug 1 23:39:58 zfs kernel: em0: TX(0) desc avail = 985,Next TX to Clean = 938 Aug 1 23:43:24 zfs kernel: em0: Watchdog timeout -- resetting Aug 1 23:43:24 zfs kernel: em0: Queue(0) tdh = 147, hw tdt = 163 Aug 1 23:43:24 zfs kernel: em0: TX(0) desc avail = 1006,Next TX to Clean = 145 ifconfig down/up did not fix anything, un/plugging the ethernet did not do anything either. rebooting did fix it. Serious maintenance jobs only starts after 0:00. --WjW uname -a: FreeBSD zfs.digiware.nl 8.2-STABLE FreeBSD 8.2-STABLE #10: Wed Jul 6 21:57:36 CEST 2011 root@zfs.digiware.nl:/home/obj/usr/src/src8/src/sys/ZFS amd64 pciconf -lv: hostb0@pci0:0:0:0: class=0x060000 card=0xd98015d9 chip=0x29e08086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X38/X48 (Bearlake) Processor to I/O Controller' class = bridge subclass = HOST-PCI em0@pci0:0:25:0: class=0x020000 card=0x10bd15d9 chip=0x10bd8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82566DM Gigabit Ethernet Adapter (82566DM)' class = network subclass = ethernet uhci0@pci0:0:26:0: class=0x0c0300 card=0xd98015d9 chip=0x29378086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci1@pci0:0:26:1: class=0x0c0300 card=0xd98015d9 chip=0x29388086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci2@pci0:0:26:2: class=0x0c0300 card=0xd98015d9 chip=0x29398086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci0@pci0:0:26:7: class=0x0c0320 card=0xd98015d9 chip=0x293c8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB none0@pci0:0:27:0: class=0x040300 card=0xd98015d9 chip=0x293e8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) HD Audio Controller' class = multimedia subclass = HDA pcib1@pci0:0:28:0: class=0x060400 card=0xd98015d9 chip=0x29408086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1' class = bridge subclass = PCI-PCI uhci3@pci0:0:29:0: class=0x0c0300 card=0xd98015d9 chip=0x29348086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci4@pci0:0:29:1: class=0x0c0300 card=0xd98015d9 chip=0x29358086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci5@pci0:0:29:2: class=0x0c0300 card=0xd98015d9 chip=0x29368086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci1@pci0:0:29:7: class=0x0c0320 card=0xd98015d9 chip=0x293a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB pcib4@pci0:0:30:0: class=0x060401 card=0xd98015d9 chip=0x244e8086 rev=0x92 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0xd98015d9 chip=0x29168086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IR (ICH9R) LPC Interface Controller' class = bridge subclass = PCI-ISA atapci1@pci0:0:31:2: class=0x010601 card=0xd98015d9 chip=0x29228086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controller' class = mass storage subclass = SATA none1@pci0:0:31:3: class=0x0c0500 card=0xd98015d9 chip=0x29308086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel(R) ICH9 Family SMBus Controller working fine with http://download.cnet.com/Chipset-Driver-Inte (8086)' class = serial bus subclass = SMBus none2@pci0:0:31:6: class=0x118000 card=0x000015d9 chip=0x29328086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) Thermal Subsystem' class = dasp pcib2@pci0:5:0:0: class=0x060400 card=0x00000000 chip=0x032c8086 rev=0x09 hdr=0x01 vendor = 'Intel Corporation' device = 'PCI Express-to-PCI Express Bridge (6702PXH)' class = bridge subclass = PCI-PCI ioapic0@pci0:5:0:1: class=0x080020 card=0xd98015d9 chip=0x03268086 rev=0x09 hdr=0x00 vendor = 'Intel Corporation' device = '6700/6702PXH I/OxAPIC Interrupt Controller A' class = base peripheral subclass = interrupt controller pcib3@pci0:6:7:0: class=0x060400 card=0x00000000 chip=0x03358086 rev=0x0a hdr=0x01 vendor = 'Intel Corporation' device = '80331 [Lindsay] I/O processor PCI-X bridge' class = bridge subclass = PCI-PCI arcmsr0@pci0:7:14:0: class=0x010400 card=0x112017d3 chip=0x112017d3 rev=0x00 hdr=0x00 vendor = 'Areca Technology Corporation' device = 'ARC-1120 8-Port PCI-X to SATA RAID Controller' class = mass storage subclass = RAID vgapci0@pci0:17:1:0: class=0x030000 card=0x47501002 chip=0x47501002 rev=0x5c hdr=0x00 vendor = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.' device = 'ATI 3D Rage Pro 215GP (ATI 3D Rage Pro 215GP)' class = display subclass = VGA atapci0@pci0:17:4:0: class=0x010185 card=0x82131283 chip=0x82131283 rev=0x00 hdr=0x00 vendor = 'Integrated Technology Express (ITE) Inc' device = 'IDE Controller (IT8213F)' class = mass storage subclass = ATA From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 22:49:02 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36875106564A for ; Mon, 1 Aug 2011 22:49:02 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id ED5E98FC13 for ; Mon, 1 Aug 2011 22:49:01 +0000 (UTC) Received: by iyb11 with SMTP id 11so9751209iyb.13 for ; Mon, 01 Aug 2011 15:49:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=7swC0g1VBPC2iKDESYdQHlpxKCoc6oIRYUeOxn2SrwM=; b=LCmO12luIFCOuICg7laNjck+fgedy8naxzGyeG2+KFj5OIpU2JSJ1yX5+ff0LxN3aq n41YkniHLmzZp0Q4EFlz+BR1dhKOmFXb98hLhV5KGz5LMwDIzGK4IcV61Lf1s/LErvFM k0f1DMRdVIVO95ywE7H/bfxrR3a5aYu0FPGIE= Received: by 10.231.180.156 with SMTP id bu28mr3229659ibb.134.1312238941422; Mon, 01 Aug 2011 15:49:01 -0700 (PDT) Received: from DataIX.net (adsl-99-181-155-167.dsl.klmzmi.sbcglobal.net [99.181.155.167]) by mx.google.com with ESMTPS id b6sm3630441ibg.31.2011.08.01.15.48.59 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Aug 2011 15:49:00 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id p71MmvMA051653 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 1 Aug 2011 18:48:57 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id p71Mmtsw051652; Mon, 1 Aug 2011 18:48:55 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Mon, 1 Aug 2011 18:48:55 -0400 From: Jason Hellenthal To: Eugene Grosbein Message-ID: <20110801224855.GD77144@DataIX.net> References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E359753.3050004@rdtc.ru> <20110801023948.GA77144@DataIX.net> <4E362993.4050907@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E362993.4050907@rdtc.ru> Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 22:49:02 -0000 On Mon, Aug 01, 2011 at 11:20:35AM +0700, Eugene Grosbein wrote: > 01.08.2011 09:39, Jason Hellenthal ?????: > > > > What line of the newsyslog.conf file is your line inserted and can you > > move that to a higher line number. FIFO > > I don't get why position of line in newsyslog.conf can have any meaning. > Normally things don't process from the inside out. Its first in first out. newsyslog processes from the first line it see's and then moves down. If your first line that processes is a large file that takes a while to compress then the delay you are seeing is coming from processing those files first. From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 22:50:52 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81CC0106564A for ; Mon, 1 Aug 2011 22:50:52 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 463DB8FC08 for ; Mon, 1 Aug 2011 22:50:52 +0000 (UTC) Received: by iyb11 with SMTP id 11so9753370iyb.13 for ; Mon, 01 Aug 2011 15:50:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=IexWekB35hmfUl5oy9/cg3qRTyKmuXv2KVoFtZM0V10=; b=mpI5Tbc+8OFdrWxGYCjG6HwgLgB6Dzk+LUxstUqu5BXTKvmi3pGRoNtxny4HqaL/x9 l6FJd27TSg4e2s7HHAzxy97vcDOJrwTn+9r7FjlpzlN3+EEHyd2vaX64n/WOPLxNsa1a lFWXnOYFrVTo2b9j5E17oZwSMiq6H3X6Zt7nk= Received: by 10.42.144.69 with SMTP id a5mr748259icv.463.1312239051673; Mon, 01 Aug 2011 15:50:51 -0700 (PDT) Received: from DataIX.net (adsl-99-181-155-167.dsl.klmzmi.sbcglobal.net [99.181.155.167]) by mx.google.com with ESMTPS id z18sm3635540ibf.1.2011.08.01.15.50.50 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Aug 2011 15:50:50 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id p71Mokg6051816 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 1 Aug 2011 18:50:47 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id p71Mok6c051815; Mon, 1 Aug 2011 18:50:46 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Mon, 1 Aug 2011 18:50:46 -0400 From: Jason Hellenthal To: Eugene Grosbein Message-ID: <20110801225046.GE77144@DataIX.net> References: <4E35881C.2010505@rdtc.ru> <20110731173129.GA53635@icarus.home.lan> <4E359753.3050004@rdtc.ru> <20110801023948.GA77144@DataIX.net> <4E362993.4050907@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E362993.4050907@rdtc.ru> Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 22:50:52 -0000 On Mon, Aug 01, 2011 at 11:20:35AM +0700, Eugene Grosbein wrote: > 01.08.2011 09:39, Jason Hellenthal ?????: > > > > What line of the newsyslog.conf file is your line inserted and can you > > move that to a higher line number. FIFO > > I don't get why position of line in newsyslog.conf can have any meaning. > Note on the previous comment... check the timestamps on the files and you should see the timestamps for the files listed at the top of newsyslog.conf have an earlier timestamp than the ones at the bottom. From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 23:00:37 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F535106566B for ; Mon, 1 Aug 2011 23:00:37 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.emeryville.ca.mail.comcast.net (qmta08.emeryville.ca.mail.comcast.net [76.96.30.80]) by mx1.freebsd.org (Postfix) with ESMTP id 680BD8FC15 for ; Mon, 1 Aug 2011 23:00:37 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta08.emeryville.ca.mail.comcast.net with comcast id FB041h00C1vN32cA8B0ZeU; Mon, 01 Aug 2011 23:00:33 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.emeryville.ca.mail.comcast.net with comcast id FB2Z1h00i1t3BNj8iB2a5Y; Mon, 01 Aug 2011 23:02:37 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 4AC74102C36; Mon, 1 Aug 2011 16:00:28 -0700 (PDT) Date: Mon, 1 Aug 2011 16:00:28 -0700 From: Jeremy Chadwick To: Willem Jan Withagen Message-ID: <20110801230028.GA83293@icarus.home.lan> References: <4E37286D.5070203@digiware.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E37286D.5070203@digiware.nl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: stable@freebsd.org, "Vogel, Jack" Subject: Re: em0 timeout disconnects server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 23:00:37 -0000 On Tue, Aug 02, 2011 at 12:27:57AM +0200, Willem Jan Withagen wrote: > A server just all of a sudden dropped from the network. > uptime was 26days. > > This got my ZFS server hanging: > > Aug 1 23:39:58 zfs kernel: em0: Watchdog timeout -- resetting > Aug 1 23:39:58 zfs kernel: em0: Queue(0) tdh = 942, hw tdt = 977 > Aug 1 23:39:58 zfs kernel: em0: TX(0) desc avail = 985,Next TX to Clean > = 938 > Aug 1 23:43:24 zfs kernel: em0: Watchdog timeout -- resetting > Aug 1 23:43:24 zfs kernel: em0: Queue(0) tdh = 147, hw tdt = 163 > Aug 1 23:43:24 zfs kernel: em0: TX(0) desc avail = 1006,Next TX to > Clean = 145 > > ifconfig down/up did not fix anything, un/plugging the ethernet did not > do anything either. rebooting did fix it. > > Serious maintenance jobs only starts after 0:00. > > --WjW > > uname -a: > FreeBSD zfs.digiware.nl 8.2-STABLE FreeBSD 8.2-STABLE #10: Wed Jul 6 > 21:57:36 CEST 2011 > root@zfs.digiware.nl:/home/obj/usr/src/src8/src/sys/ZFS amd64 Please provide "dmesg" output pertaining to the NIC (dmesg | grep em0 would be sufficient). > pciconf -lv: Please re-run this with -lvcb and include only the Intel NIC (em0). Also please provide output from command "sysctl dev.em.0". Thanks. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 23:20:56 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 847FB1065678 for ; Mon, 1 Aug 2011 23:20:56 +0000 (UTC) (envelope-from kometen@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0DFDE8FC0C for ; Mon, 1 Aug 2011 23:20:55 +0000 (UTC) Received: by fxe4 with SMTP id 4so6953063fxe.13 for ; Mon, 01 Aug 2011 16:20:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=references:in-reply-to:mime-version:content-transfer-encoding :content-type:message-id:cc:x-mailer:from:subject:date:to; bh=CXv+TY5Y8MMqKkp2khr9ikOoMxuOVdtyvjyK483/rdM=; b=j/CHSUfboYP6LhyKhvdXDK96YKgS+ni4BDzfkMsqIiaSqDlGnZjHetU5KzvMol0TEJ qgr7ckv4QU4qfbBswr3r9w2xyI30rx1lSEcch6WgyXjw7XsHouj5nDzHn7JDhz/a1u0A fsyUtBPpsUvsIOwuP1+ikXdUQaHhktIDMXIiM= Received: by 10.204.24.9 with SMTP id t9mr1539108bkb.305.1312238993538; Mon, 01 Aug 2011 15:49:53 -0700 (PDT) Received: from [10.0.0.5] (ti0128a380-dhcp0306.bb.online.no [88.88.53.53]) by mx.google.com with ESMTPS id e4sm1429020bkt.62.2011.08.01.15.49.50 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Aug 2011 15:49:52 -0700 (PDT) References: <4E37286D.5070203@digiware.nl> In-Reply-To: <4E37286D.5070203@digiware.nl> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <01EC81F0-1462-4F63-AC6F-822615326832@gmail.com> X-Mailer: iPhone Mail (9A5274d) From: Claus Guttesen Date: Tue, 2 Aug 2011 00:49:47 +0200 To: Willem Jan Withagen Cc: "stable@freebsd.org" Subject: Re: em0 timeout disconnects server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 23:20:56 -0000 Do you happen to run nfs on the server? I had weird problems with igb-timeouts when many nfs-reads occured and a dow= n and up on the interface would restore the network connection for a while. I= had vmware-servers on a nfs-share and either when booting or installing pro= grams from windows Regards Claus Sendt fra min iPhone Den 02/08/2011 kl. 00.27 skrev Willem Jan Withagen : > A server just all of a sudden dropped from the network. > uptime was 26days. >=20 > This got my ZFS server hanging: >=20 > Aug 1 23:39:58 zfs kernel: em0: Watchdog timeout -- resetting > Aug 1 23:39:58 zfs kernel: em0: Queue(0) tdh =3D 942, hw tdt =3D 977 > Aug 1 23:39:58 zfs kernel: em0: TX(0) desc avail =3D 985,Next TX to Clean= > =3D 938 > Aug 1 23:43:24 zfs kernel: em0: Watchdog timeout -- resetting > Aug 1 23:43:24 zfs kernel: em0: Queue(0) tdh =3D 147, hw tdt =3D 163 > Aug 1 23:43:24 zfs kernel: em0: TX(0) desc avail =3D 1006,Next TX to > Clean =3D 145 >=20 > ifconfig down/up did not fix anything, un/plugging the ethernet did not > do anything either. rebooting did fix it. >=20 > Serious maintenance jobs only starts after 0:00. >=20 > --WjW >=20 > uname -a: > FreeBSD zfs.digiware.nl 8.2-STABLE FreeBSD 8.2-STABLE #10: Wed Jul 6 > 21:57:36 CEST 2011 > root@zfs.digiware.nl:/home/obj/usr/src/src8/src/sys/ZFS amd64 >=20 > pciconf -lv: > hostb0@pci0:0:0:0: class=3D0x060000 card=3D0xd98015d9 chip=3D0x29e080= 86 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D 'X38/X48 (Bearlake) Processor to I/O Controller' > class =3D bridge > subclass =3D HOST-PCI > em0@pci0:0:25:0: class=3D0x020000 card=3D0x10bd15d9 chip=3D0x10bd80= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D 'Intel 82566DM Gigabit Ethernet Adapter (82566DM)' > class =3D network > subclass =3D ethernet > uhci0@pci0:0:26:0: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293780= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > uhci1@pci0:0:26:1: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293880= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > uhci2@pci0:0:26:2: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293980= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > ehci0@pci0:0:26:7: class=3D0x0c0320 card=3D0xd98015d9 chip=3D0x293c80= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controll= er' > class =3D serial bus > subclass =3D USB > none0@pci0:0:27:0: class=3D0x040300 card=3D0xd98015d9 chip=3D0x293e80= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) HD Audio Controller' > class =3D multimedia > subclass =3D HDA > pcib1@pci0:0:28:0: class=3D0x060400 card=3D0xd98015d9 chip=3D0x294080= 86 > rev=3D0x02 hdr=3D0x01 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1' > class =3D bridge > subclass =3D PCI-PCI > uhci3@pci0:0:29:0: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293480= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > uhci4@pci0:0:29:1: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293580= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > uhci5@pci0:0:29:2: class=3D0x0c0300 card=3D0xd98015d9 chip=3D0x293680= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Controll= er' > class =3D serial bus > subclass =3D USB > ehci1@pci0:0:29:7: class=3D0x0c0320 card=3D0xd98015d9 chip=3D0x293a80= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controll= er' > class =3D serial bus > subclass =3D USB > pcib4@pci0:0:30:0: class=3D0x060401 card=3D0xd98015d9 chip=3D0x244e80= 86 > rev=3D0x92 hdr=3D0x01 > vendor =3D 'Intel Corporation' > device =3D '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub > Interface to PCI Bridge' > class =3D bridge > subclass =3D PCI-PCI > isab0@pci0:0:31:0: class=3D0x060100 card=3D0xd98015d9 chip=3D0x291680= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IR (ICH9R) LPC Interface Controller' > class =3D bridge > subclass =3D PCI-ISA > atapci1@pci0:0:31:2: class=3D0x010601 card=3D0xd98015d9 chip=3D0x292280= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controller= ' > class =3D mass storage > subclass =3D SATA > none1@pci0:0:31:3: class=3D0x0c0500 card=3D0xd98015d9 chip=3D0x293080= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D 'Intel(R) ICH9 Family SMBus Controller working fine > with http://download.cnet.com/Chipset-Driver-Inte (8086)' > class =3D serial bus > subclass =3D SMBus > none2@pci0:0:31:6: class=3D0x118000 card=3D0x000015d9 chip=3D0x293280= 86 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82801IB/IR/IH (ICH9 Family) Thermal Subsystem' > class =3D dasp > pcib2@pci0:5:0:0: class=3D0x060400 card=3D0x00000000 chip=3D0x032c80= 86 > rev=3D0x09 hdr=3D0x01 > vendor =3D 'Intel Corporation' > device =3D 'PCI Express-to-PCI Express Bridge (6702PXH)' > class =3D bridge > subclass =3D PCI-PCI > ioapic0@pci0:5:0:1: class=3D0x080020 card=3D0xd98015d9 chip=3D0x032680= 86 > rev=3D0x09 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '6700/6702PXH I/OxAPIC Interrupt Controller A' > class =3D base peripheral > subclass =3D interrupt controller > pcib3@pci0:6:7:0: class=3D0x060400 card=3D0x00000000 chip=3D0x033580= 86 > rev=3D0x0a hdr=3D0x01 > vendor =3D 'Intel Corporation' > device =3D '80331 [Lindsay] I/O processor PCI-X bridge' > class =3D bridge > subclass =3D PCI-PCI > arcmsr0@pci0:7:14:0: class=3D0x010400 card=3D0x112017d3 chip=3D0x112017= d3 > rev=3D0x00 hdr=3D0x00 > vendor =3D 'Areca Technology Corporation' > device =3D 'ARC-1120 8-Port PCI-X to SATA RAID Controller' > class =3D mass storage > subclass =3D RAID > vgapci0@pci0:17:1:0: class=3D0x030000 card=3D0x47501002 chip=3D0x475010= 02 > rev=3D0x5c hdr=3D0x00 > vendor =3D 'ATI Technologies Inc. / Advanced Micro Devices, Inc.' > device =3D 'ATI 3D Rage Pro 215GP (ATI 3D Rage Pro 215GP)' > class =3D display > subclass =3D VGA > atapci0@pci0:17:4:0: class=3D0x010185 card=3D0x82131283 chip=3D0x821312= 83 > rev=3D0x00 hdr=3D0x00 > vendor =3D 'Integrated Technology Express (ITE) Inc' > device =3D 'IDE Controller (IT8213F)' > class =3D mass storage > subclass =3D ATA >=20 >=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Mon Aug 1 23:31:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF582106566C; Mon, 1 Aug 2011 23:31:20 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 93AFA8FC19; Mon, 1 Aug 2011 23:31:20 +0000 (UTC) Received: by qyk38 with SMTP id 38so4240797qyk.13 for ; Mon, 01 Aug 2011 16:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZUgmq/kNXANQn4ggBiVtnfR00zuNNVFuuJf71kn27Eg=; b=oICbQgp6/2w49/+mOOWwRJ2TJUeZa4KmFQibJxiBNHB7USPiXRaLJSEmBUonu4PD2b KXjq8demQdjM9m5P0xAnrBKgof2UV99WHVX02Vh3Y/3aPrl23+kPdspTfTFilvcG/Nb3 EbBh9u7wyrKWpctSkGxFHVhU3e939xnt4gdks= MIME-Version: 1.0 Received: by 10.229.66.200 with SMTP id o8mr3671651qci.250.1312241479787; Mon, 01 Aug 2011 16:31:19 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Mon, 1 Aug 2011 16:31:19 -0700 (PDT) In-Reply-To: <4E355CB9.5090209@os2.kiev.ua> References: <4E355CB9.5090209@os2.kiev.ua> Date: Mon, 1 Aug 2011 16:31:19 -0700 Message-ID: From: maestro something To: Alex Samorukov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: rpaulo@freebsd.org, freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 23:31:21 -0000 Running the same example on amd64 has the same behavior as on i386 (granted the backtrace is slightly different). That is, the kernel panics immediately when you use the ustack() action. Furthermore, the crashdump does not provide the nececcary information to get at the root of the cause (i.e., no souce found for the systrace_probe and dtrace_probe frames). still I'm wondering ... Did anybody successfully use the ustack() action on FreeBSD-8.2? cheers --m backtrace on amd64: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80ff1fa6 stack pointer = 0x28:0xffffff80001d7500 frame pointer = 0x28:0xffffff80001d7850 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 1068 (nc) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff80618d2e at kdb_backtrace+0x5e #1 0xffffffff805e4d37 at panic+0x187 #2 0xffffffff808dcac0 at trap_fatal+0x290 #3 0xffffffff808dcf0d at trap_pfault+0x2fd #4 0xffffffff808dd343 at trap+0x343 #5 0xffffffff808c5298 at calltrap+0x8 #6 0xffffffff80ff638a at dtrace_probe+0x47a #7 0xffffffff81192285 at systrace_probe+0x45 #8 0xffffffff80624f50 at syscallenter+0x3c0 #9 0xffffffff808dcf7b at syscall+0x4b #10 0xffffffff808c5572 at Xfast_syscall+0xe2 Uptime: 3m0s On Sun, Jul 31, 2011 at 6:46 AM, Alex Samorukov wrote: > Hi, > > I just tried your test on -STABLE + 1 fix from current [1] and got no trap. > > dtrace: buffer size lowered to 2m > CPU ID FUNCTION:NAME > 4 42224 accept:return nc accept:return > > Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/..** > /../../cddl/contrib/**opensolaris/lib/libdtrace/**common/dt_proc.c, line > 751. > Abort (core dumped) > > > Also 1st dtrace was killed as well. > > I tried 2 times with the same result. Box is amd64. > > This is bt from gdb: > > (gdb) bt > #0 0x0000000800fa2f9c in thr_kill () from /lib/libc.so.7 > #1 0x000000080103f6cb in abort () from /lib/libc.so.7 > #2 0x0000000800786758 in dt_proc_lookup () from /lib/libdtrace.so.2 > #3 0x00000008007867db in dt_proc_lock () from /lib/libdtrace.so.2 > #4 0x00000008007a5bcb in dt_print_ustack () from /lib/libdtrace.so.2 > #5 0x00000008007a8504 in dt_print_agg () from /lib/libdtrace.so.2 > #6 0x00000008007a88ca in dtrace_consume () from /lib/libdtrace.so.2 > #7 0x000000080077e1a2 in dtrace_work () from /lib/libdtrace.so.2 > #8 0x00000000004044ba in ?? () > #9 0x000000000040222e in ?? () > [skip] > > [1] http://www.freebsd.org/cgi/**query-pr.cgi?pr=kern/159064 > > > On 07/26/2011 03:20 AM, maestro something wrote: > >> Hi, >> >> when using the ustack action on the latest FreeBSD8.2 i386 the kernel >> panics. >> > > From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 05:14:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21BFF106564A; Tue, 2 Aug 2011 05:14:34 +0000 (UTC) (envelope-from alex@zagrebin.ru) Received: from mail.zagrebin.ru (gw.zagrebin.ru [91.215.205.128]) by mx1.freebsd.org (Postfix) with ESMTP id B2F9F8FC08; Tue, 2 Aug 2011 05:14:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zagrebin.ru; s=mail; h=Content-Type:MIME-Version:Message-ID:Subject:Cc:To:From:Date; bh=l7yZn36CRGax4l5waVH//ZhZ79VpBvJZKjOjtHhtzmg=; b=T6yHHvjevRSkSCf8Q5ickJh6X4gKgj3lC/Kc9QWpOrZEokA8WLF3ojTzi1H4tQrVgRybkxpvP1rer3EWolJ1zohzNTHchMZdn+LrLq/gB3txTRycr/8tomW0SPHAuGsy1lyMImPqrzoou4MTusYd2Gp4Ol/B75gmy3eIxgYpfgY=; Received: from alex by mail.zagrebin.ru with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Qo6sx-0002SC-Je; Tue, 02 Aug 2011 08:47:19 +0400 Date: Tue, 2 Aug 2011 08:47:19 +0400 From: Alexander Zagrebin To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Message-ID: <20110802044718.GA8838@gw.zagrebin.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="NzB8fVQJ5HfG6fxh" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@freebsd.org Subject: ZFS v28: kernel panics while reading an extended attribute X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 05:14:34 -0000 --NzB8fVQJ5HfG6fxh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi! It seems, I've found a bug in the ZFS v28 on the latest stable: if we have a snapshot with some files having an extended attributes, then attempt to read an extended attributes's value leads to a well reproducible kernel panic. The part of backtrace follows: #6 0xffffffff804bbe44 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff80950ea7 in zil_commit (zilog=0x0, foid=5795917) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1497 #8 0xffffffff80979e6b in zfs_freebsd_read (ap=Variable "ap" is not available.) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:622 #9 0xffffffff80979750 in zfs_getextattr (ap=0xffffff80dd5d8820) at vnode_if.h:384 #10 0xffffffff8038921b in extattr_get_vp (vp=0xffffff0056a01588, attrnamespace=1, attrname=0xffffff80dd5d89a0 "DOSATTRIB", data=Variable "data" is not available.) at vnode_if.h:1332 It seems that ZIL isn't available for snapshots, but zfs_freebsd_read doesn't check this when calling zil_commit. The attached patch fixes this issue. Can anybody confirm this? -- Alexander Zagrebin --NzB8fVQJ5HfG6fxh Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="patch-zfs_vnops.c" --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c.orig 2011-08-01 23:04:07.358173627 +0400 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c 2011-08-02 00:10:02.674585604 +0400 @@ -618,7 +618,8 @@ zfs_read(vnode_t *vp, uio_t *uio, int io /* * If we're in FRSYNC mode, sync out this znode before reading it. */ - if (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + if (zfsvfs->z_log && + (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS)) zil_commit(zfsvfs->z_log, zp->z_id); /* --NzB8fVQJ5HfG6fxh-- From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 06:31:23 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E2DF106566B for ; Tue, 2 Aug 2011 06:31:23 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (mail.ip6.digiware.nl [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id AE6EF8FC0A for ; Tue, 2 Aug 2011 06:31:22 +0000 (UTC) Received: from rack1.digiware.nl (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id 16852153434; Tue, 2 Aug 2011 08:31:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8cyT3wpZfW0P; Tue, 2 Aug 2011 08:31:19 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:8964:6f2b:1e23:8042] (unknown [IPv6:2001:4cb8:3:1:8964:6f2b:1e23:8042]) by mail.digiware.nl (Postfix) with ESMTP id 53A75153433; Tue, 2 Aug 2011 08:31:19 +0200 (CEST) Message-ID: <4E3799B6.1020707@digiware.nl> Date: Tue, 02 Aug 2011 08:31:18 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Claus Guttesen References: <4E37286D.5070203@digiware.nl> <01EC81F0-1462-4F63-AC6F-822615326832@gmail.com> In-Reply-To: <01EC81F0-1462-4F63-AC6F-822615326832@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "stable@freebsd.org" Subject: Re: em0 timeout disconnects server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:31:23 -0000 On 2011-08-02 0:49, Claus Guttesen wrote: > Do you happen to run nfs on the server? > > I had weird problems with igb-timeouts when many nfs-reads occured > and a down and up on the interface would restore the network > connection for a while. I had vmware-servers on a nfs-share and > either when booting or installing programs from windows Yup, this server runs: nfsd samba rsyncd --WjW From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 06:42:05 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E36D1065670 for ; Tue, 2 Aug 2011 06:42:05 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (mail.ip6.digiware.nl [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id AD6A78FC16 for ; Tue, 2 Aug 2011 06:42:04 +0000 (UTC) Received: from rack1.digiware.nl (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id 15FDC153434; Tue, 2 Aug 2011 08:42:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e+UNAnFf3QEH; Tue, 2 Aug 2011 08:42:01 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:8964:6f2b:1e23:8042] (unknown [IPv6:2001:4cb8:3:1:8964:6f2b:1e23:8042]) by mail.digiware.nl (Postfix) with ESMTP id 8CEFC153433; Tue, 2 Aug 2011 08:42:01 +0200 (CEST) Message-ID: <4E379C38.4000109@digiware.nl> Date: Tue, 02 Aug 2011 08:42:00 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Jeremy Chadwick , "stable@freebsd.org" References: <4E37286D.5070203@digiware.nl> <20110801230028.GA83293@icarus.home.lan> In-Reply-To: <20110801230028.GA83293@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: em0 timeout disconnects server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:42:05 -0000 On 2011-08-02 1:00, Jeremy Chadwick wrote: > On Tue, Aug 02, 2011 at 12:27:57AM +0200, Willem Jan Withagen wrote: >> A server just all of a sudden dropped from the network. >> uptime was 26days. >> >> This got my ZFS server hanging: >> >> Aug 1 23:39:58 zfs kernel: em0: Watchdog timeout -- resetting >> Aug 1 23:39:58 zfs kernel: em0: Queue(0) tdh = 942, hw tdt = 977 >> Aug 1 23:39:58 zfs kernel: em0: TX(0) desc avail = 985,Next TX to Clean >> = 938 >> Aug 1 23:43:24 zfs kernel: em0: Watchdog timeout -- resetting >> Aug 1 23:43:24 zfs kernel: em0: Queue(0) tdh = 147, hw tdt = 163 >> Aug 1 23:43:24 zfs kernel: em0: TX(0) desc avail = 1006,Next TX to >> Clean = 145 >> >> ifconfig down/up did not fix anything, un/plugging the ethernet did not >> do anything either. rebooting did fix it. >> >> Serious maintenance jobs only starts after 0:00. >> >> --WjW >> >> uname -a: >> FreeBSD zfs.digiware.nl 8.2-STABLE FreeBSD 8.2-STABLE #10: Wed Jul 6 >> 21:57:36 CEST 2011 >> root@zfs.digiware.nl:/home/obj/usr/src/src8/src/sys/ZFS amd64 > > Please provide "dmesg" output pertaining to the NIC (dmesg | grep em0 > would be sufficient). em0: port 0x1820-0x183f mem 0xdf900000-0xdf91ffff,0xdf924000-0xdf924fff irq 16 at device 25.0 on pci0 em0: Using an MSI interrupt em0: [FILTER] em0: Ethernet address: 00:30:48:de:97:cd > >> pciconf -lv: > > Please re-run this with -lvcb and include only the Intel NIC (em0). em0@pci0:0:25:0: class=0x020000 card=0x10bd15d9 chip=0x10bd8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82566DM Gigabit Ethernet Adapter (82566DM)' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xdf900000, size 131072, enabled bar [14] = type Memory, range 32, base 0xdf924000, size 4096, enabled bar [18] = type I/O Port, range 32, base 0x1820, size 32, enabled cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 13[e0] = PCI Advanced Features: FLR TP > Also please provide output from command "sysctl dev.em.0". dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.2.3 dev.em.0.%driver: em dev.em.0.%location: slot=25 function=0 handle=\_SB_.PCI0.LAN_ dev.em.0.%pnpinfo: vendor=0x8086 device=0x10bd subvendor=0x15d9 subdevice=0x10bd class=0x020000 dev.em.0.%parent: pci0 dev.em.0.nvm: -1 dev.em.0.debug: -1 dev.em.0.rx_int_delay: 0 dev.em.0.tx_int_delay: 66 dev.em.0.rx_abs_int_delay: 66 dev.em.0.tx_abs_int_delay: 66 dev.em.0.rx_processing_limit: 100 dev.em.0.flow_control: 3 dev.em.0.eee_control: 0 dev.em.0.link_irq: 0 dev.em.0.mbuf_alloc_fail: 0 dev.em.0.cluster_alloc_fail: 0 dev.em.0.dropped: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.rx_overruns: 0 dev.em.0.watchdog_timeouts: 0 dev.em.0.device_control: 1074790976 dev.em.0.rx_control: 67141634 dev.em.0.fc_high_water: 8192 dev.em.0.fc_low_water: 6692 dev.em.0.queue0.txd_head: 710 dev.em.0.queue0.txd_tail: 712 dev.em.0.queue0.tx_irq: 0 dev.em.0.queue0.no_desc_avail: 0 dev.em.0.queue0.rxd_head: 558 dev.em.0.queue0.rxd_tail: 556 dev.em.0.queue0.rx_irq: 0 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.missed_packets: 0 dev.em.0.mac_stats.recv_no_buff: 0 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.xon_recvd: 0 dev.em.0.mac_stats.xon_txd: 0 dev.em.0.mac_stats.xoff_recvd: 0 dev.em.0.mac_stats.xoff_txd: 0 dev.em.0.mac_stats.total_pkts_recvd: 10942830 dev.em.0.mac_stats.good_pkts_recvd: 10942830 dev.em.0.mac_stats.bcast_pkts_recvd: 83730 dev.em.0.mac_stats.mcast_pkts_recvd: 861 dev.em.0.mac_stats.rx_frames_64: 0 dev.em.0.mac_stats.rx_frames_65_127: 0 dev.em.0.mac_stats.rx_frames_128_255: 0 dev.em.0.mac_stats.rx_frames_256_511: 0 dev.em.0.mac_stats.rx_frames_512_1023: 0 dev.em.0.mac_stats.rx_frames_1024_1522: 0 dev.em.0.mac_stats.good_octets_recvd: 8091153275 dev.em.0.mac_stats.good_octets_txd: 5995000222 dev.em.0.mac_stats.total_pkts_txd: 10302157 dev.em.0.mac_stats.good_pkts_txd: 10302157 dev.em.0.mac_stats.bcast_pkts_txd: 1240 dev.em.0.mac_stats.mcast_pkts_txd: 41 dev.em.0.mac_stats.tx_frames_64: 0 dev.em.0.mac_stats.tx_frames_65_127: 0 dev.em.0.mac_stats.tx_frames_128_255: 0 dev.em.0.mac_stats.tx_frames_256_511: 0 dev.em.0.mac_stats.tx_frames_512_1023: 0 dev.em.0.mac_stats.tx_frames_1024_1522: 0 dev.em.0.mac_stats.tso_txd: 179532 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.interrupts.asserts: 10151890 dev.em.0.interrupts.rx_pkt_timer: 0 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.tx_pkt_timer: 0 dev.em.0.interrupts.tx_abs_timer: 0 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_queue_min_thresh: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.rx_overrun: 0 dev.em.0.wake: 0 Note that this is all from the rebooted server. This is my main filestore, so can not leave it offline (too long). --WjW From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 08:07:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B35C1065675 for ; Tue, 2 Aug 2011 08:07:06 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 60DB88FC13 for ; Tue, 2 Aug 2011 08:07:06 +0000 (UTC) Received: by vxg33 with SMTP id 33so6621549vxg.13 for ; Tue, 02 Aug 2011 01:07:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=EwyOVgKo/BipgixWlCaMQX1cUJls5kcrhz2NVayQtFA=; b=TbPneLO6yUmCvsbrTz4ff1lpG6Pg+5h28vgkcu2I3+17/XnzA1iib0jDscoynys3Ht j7Sc4uahDslmXYTnfhDJInIXRxqKrTfF+kcro4sKECf7hosWvhM0Pti0bCPNganNkwRI CXsuePuXFMi9eb6KiSz6AaeFGRizJpDy8Pmps= MIME-Version: 1.0 Received: by 10.52.64.143 with SMTP id o15mr5479030vds.45.1312270743931; Tue, 02 Aug 2011 00:39:03 -0700 (PDT) Received: by 10.52.168.68 with HTTP; Tue, 2 Aug 2011 00:39:03 -0700 (PDT) Date: Tue, 2 Aug 2011 08:39:03 +0100 Message-ID: From: "seanrees@gmail.com" To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 08:07:06 -0000 Hi there, I Googled around and checked the PRs and wasn't successful in finding any reports of what I'm seeing. I'm hoping someone here can help me debug what's going on. On my FreeBSD 8.2-S machine (built circa 12th June), I created a directory and populated it over the course of 3 weeks with about 2 million individual files. As you might imagine, a 'ls' of this directory took quite some time. The files were conveniently named with a timestamp in the filename (still images from a security camera, once per second) so I've since moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I found though was the original directory the images were in is still very slow to ls -- and it only has 1 file in it, another directory. To clarify: % ls second [lots of time and many many files enumerated] % # rename files using rename script % ls second [wait ages] 2011 dead % mkdir second2 && mv second/2011 second2 % ls second2 [fast!] 2011 % ls second [still very slow] dead % time ls second dead/ gls -F --color 0.00s user 1.56s system 0% cpu 3:09.61 total (timings are similar for /bin/ls) This data is stored on a striped ZFS pool (version 15, though the kernel reports version 28 is available but zpool upgrade seems to disagree), 2T in size. I've run zpool scrub with no effect. ZFS is busily driving the disks away; my iostat monitoring has all three drives in the zpool running at 40-60% busy for the duration of the ls (it was quiet before). I've attached truss to the ls process. It spends a lot of time here: fstatfs(0x5,0x7fffffffe0d0,0x800ad5548,0x7fffffffdfd8,0x0,0x0) = 0 (0x0) I'm thinking there's some old ZFS metadata that it's looking into, but I'm not sure how to best dig into this to understand what's going on under the hood. Can anyone perhaps point me the right direction on this? Thanks, Sean From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 08:51:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEB08106564A for ; Tue, 2 Aug 2011 08:51:36 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out1.tiscali.nl (smtp-out1.tiscali.nl [195.241.79.176]) by mx1.freebsd.org (Postfix) with ESMTP id 6BAD78FC08 for ; Tue, 2 Aug 2011 08:51:36 +0000 (UTC) Received: from [212.182.167.131] (helo=sjakie.klop.ws) by smtp-out1.tiscali.nl with esmtp (Exim) (envelope-from ) id 1QoALa-0002q0-T0; Tue, 02 Aug 2011 10:29:06 +0200 Received: from 212-182-167-131.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id B07E34182; Tue, 2 Aug 2011 10:29:02 +0200 (CEST) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-stable@freebsd.org, "seanrees@gmail.com" References: Date: Tue, 02 Aug 2011 10:29:02 +0200 MIME-Version: 1.0 From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/11.50 (FreeBSD) Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 08:51:36 -0000 Not an in depth solution for ZFS, but maybe a solution for you. mkdir images2 mv images/* images2 rmdir images Ronald. On Tue, 02 Aug 2011 09:39:03 +0200, seanrees@gmail.com =20 wrote: > Hi there, > > I Googled around and checked the PRs and wasn't successful in finding > any reports of what I'm seeing. I'm hoping someone here can help me > debug what's going on. > > On my FreeBSD 8.2-S machine (built circa 12th June), I created a > directory and populated it over the course of 3 weeks with about 2 > million individual files. As you might imagine, a 'ls' of this > directory took quite some time. > > The files were conveniently named with a timestamp in the filename > (still images from a security camera, once per second) so I've since > moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I > found though was the original directory the images were in is still > very slow to ls -- and it only has 1 file in it, another directory. > > To clarify: > % ls second > [lots of time and many many files enumerated] > % # rename files using rename script > % ls second > [wait ages] > 2011 dead > % mkdir second2 && mv second/2011 second2 > % ls second2 > [fast!] > 2011 > % ls second > [still very slow] > dead > % time ls second > dead/ > gls -F --color 0.00s user 1.56s system 0% cpu 3:09.61 total > > (timings are similar for /bin/ls) > > This data is stored on a striped ZFS pool (version 15, though the > kernel reports version 28 is available but zpool upgrade seems to > disagree), 2T in size. I've run zpool scrub with no effect. ZFS is > busily driving the disks away; my iostat monitoring has all three > drives in the zpool running at 40-60% busy for the duration of the ls > (it was quiet before). > > I've attached truss to the ls process. It spends a lot of time here: > fstatfs(0x5,0x7fffffffe0d0,0x800ad5548,0x7fffffffdfd8,0x0,0x0) =3D 0 (0= x0) > > I'm thinking there's some old ZFS metadata that it's looking into, but > I'm not sure how to best dig into this to understand what's going on > under the hood. > > Can anyone perhaps point me the right direction on this? > > Thanks, > > Sean > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:08:32 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FDCA106566C for ; Tue, 2 Aug 2011 09:08:32 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id 880808FC0C for ; Tue, 2 Aug 2011 09:08:32 +0000 (UTC) Received: from omta04.emeryville.ca.mail.comcast.net ([76.96.30.35]) by qmta11.emeryville.ca.mail.comcast.net with comcast id FM8U1h0070lTkoCABM8U1y; Tue, 02 Aug 2011 09:08:28 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta04.emeryville.ca.mail.comcast.net with comcast id FM8W1h00W1t3BNj8QM8Xv2; Tue, 02 Aug 2011 09:08:31 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 938AF102C36; Tue, 2 Aug 2011 02:08:30 -0700 (PDT) Date: Tue, 2 Aug 2011 02:08:30 -0700 From: Jeremy Chadwick To: "seanrees@gmail.com" Message-ID: <20110802090830.GA92646@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:08:32 -0000 On Tue, Aug 02, 2011 at 08:39:03AM +0100, seanrees@gmail.com wrote: > On my FreeBSD 8.2-S machine (built circa 12th June), I created a > directory and populated it over the course of 3 weeks with about 2 > million individual files. I'll keep this real simple: Why did you do this? I hope this was a stress test of some kind. If not: This is the 2nd or 3rd mail in recent months from people saying "I decided to do something utterly stupid with my filesystem[1] and now I'm asking why performance sucks". Why can people not create proper directory tree layouts to avoid this problem regardless of what filesystem is used? I just don't get it. [1]: Applies to any filesystem, not just ZFS. There was a UFS one a month or two ago too... -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:16:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59827106566C for ; Tue, 2 Aug 2011 09:16:37 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 192138FC13 for ; Tue, 2 Aug 2011 09:16:36 +0000 (UTC) Received: by vxg33 with SMTP id 33so6664123vxg.13 for ; Tue, 02 Aug 2011 02:16:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4hQGSPwwJ/xf3w/7HGjqY/h2LlLnoWj2UwJAluvtAV8=; b=W5cu0iA95zA6kZTawL84M2CiwLOwXgvughX6X1xBt79r1mopBKtYnkUXW07SPAi9oF mqD4klMaZCNFCKJp8hkd6P40VBVkkNcIc/RVVeILoPap/VhXsipmgBeST61fYU/s4Cfy B9x6VHylP0PWDMtXZyJqnRA9LzT8NwyCg7SNI= MIME-Version: 1.0 Received: by 10.52.182.67 with SMTP id ec3mr5516747vdc.246.1312276596008; Tue, 02 Aug 2011 02:16:36 -0700 (PDT) Received: by 10.52.168.68 with HTTP; Tue, 2 Aug 2011 02:16:35 -0700 (PDT) In-Reply-To: <20110802090830.GA92646@icarus.home.lan> References: <20110802090830.GA92646@icarus.home.lan> Date: Tue, 2 Aug 2011 10:16:35 +0100 Message-ID: From: "seanrees@gmail.com" To: Jeremy Chadwick Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:16:37 -0000 inline On Tue, Aug 2, 2011 at 10:08 AM, Jeremy Chadwick wrote: > On Tue, Aug 02, 2011 at 08:39:03AM +0100, seanrees@gmail.com wrote: >> On my FreeBSD 8.2-S machine (built circa 12th June), I created a >> directory and populated it over the course of 3 weeks with about 2 >> million individual files. > > I'll keep this real simple: > > Why did you do this? > > I hope this was a stress test of some kind. =A0If not: Not really, but it turned into one. The camera I was using had the ability (rather handily) to upload a still image once per second via FTP to a server of my choosing. It didn't have the ability to organize them for me in a neat directory hierarchy. So on holidays I went for 3 weeks and came back to ~2M images in the same directory. > This is the 2nd or 3rd mail in recent months from people saying "I > decided to do something utterly stupid with my filesystem[1] and now I'm > asking why performance sucks". > > Why can people not create proper directory tree layouts to avoid this > problem regardless of what filesystem is used? =A0I just don't get it. I'm not sure it's utterly stupid; I didn't expect legendarily fast performance from 'ls' or anything else that enumerated the contents of the directory when all the files were there. Now that the files are neatly organized, I expected fstatfs() on the directory to become fast again. It isn't. I'd like to understand why (or maybe learn a new trick or two about inspecting ZFS...) Sean From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:30:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6ED03106566C for ; Tue, 2 Aug 2011 09:30:23 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2F2F18FC0A for ; Tue, 2 Aug 2011 09:30:22 +0000 (UTC) Received: by vxg33 with SMTP id 33so6675808vxg.13 for ; Tue, 02 Aug 2011 02:30:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.23.225 with SMTP id p1mr4980087vdf.134.1312277422272; Tue, 02 Aug 2011 02:30:22 -0700 (PDT) Received: by 10.220.25.7 with HTTP; Tue, 2 Aug 2011 02:30:22 -0700 (PDT) X-Originating-IP: [93.221.180.236] In-Reply-To: References: Date: Tue, 2 Aug 2011 11:30:22 +0200 Message-ID: From: "C. P. Ghost" To: "seanrees@gmail.com" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:30:23 -0000 On Tue, Aug 2, 2011 at 9:39 AM, seanrees@gmail.com wro= te: > On my FreeBSD 8.2-S machine (built circa 12th June), I created a > directory and populated it over the course of 3 weeks with about 2 > million individual files. As you might imagine, a 'ls' of this > directory took quite some time. What actually takes some time here isn't zfs, but the sorting of ls(1). Usually, running ls(1) with -f (Output is not sorted) speeds up things enormously. > The files were conveniently named with a timestamp in the filename > (still images from a security camera, once per second) so I've since > moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I > found though was the original directory the images were in is still > very slow to ls -- and it only has 1 file in it, another directory. That is strange... and shouldn't happen. According to the ZFS Performance Wiki [1], operations on ZFS file systems are supposed to be pretty efficient: Concurrent, constant time directory operations Large directories need constant time operations (lookup, create, delete, etc). Hot directories need concurrent operations. ZFS uses extensible hashing to solve this. Block based, amortized growth cost, short chains for constant time ops, per-block locking for high concurrency. A caveat is that readir returns entries in hash order. Directories are implemented via the ZFS Attribute Processor (ZAP) in ZFS. ZAP can be used to arbitrary name value pairs. ZAP uses two algorithms are optimized for large lists (large directories) and small lists (attribute lists). The ZAP implementation is in zap.c and zap_leaf.c. Each directory is maintained as a table of pointers to constant sized buckets holding a variable number of entries. Each directory record is 16k in size. When this block gets full, a new block of size next power of two is allocated. A directory starts off as a microzap, and then upgraded to a fat zap (via mzap_upgrade) if the size of the name exceeds MZAP_NAME_LEN ( MZAP_ENT_LEN - 8 - 4 - 2) or 50 or if the size of the microzap exceeds MZAP_MAX_BLKSZ (128k) [1]: http://www.solarisinternals.com/wiki/index.php/ZFS_Performance I don't know what's going on there, but someone with ZFS internals expertise may want to have a closer look. > To clarify: > % ls second > [lots of time and many many files enumerated] > % # rename files using rename script > % ls second > [wait ages] > 2011 dead > % mkdir second2 && mv second/2011 second2 > % ls second2 > [fast!] > 2011 > % ls second > [still very slow] > dead > % time ls second > dead/ > gls -F --color =A00.00s user 1.56s system 0% cpu 3:09.61 total > > (timings are similar for /bin/ls) > > This data is stored on a striped ZFS pool (version 15, though the > kernel reports version 28 is available but zpool upgrade seems to > disagree), 2T in size. I've run zpool scrub with no effect. ZFS is > busily driving the disks away; my iostat monitoring has all three > drives in the zpool running at 40-60% busy for the duration of the ls > (it was quiet before). > > I've attached truss to the ls process. It spends a lot of time here: > fstatfs(0x5,0x7fffffffe0d0,0x800ad5548,0x7fffffffdfd8,0x0,0x0) =3D 0 (0x0= ) That's a very good hint indeed! > I'm thinking there's some old ZFS metadata that it's looking into, but > I'm not sure how to best dig into this to understand what's going on > under the hood. > > Can anyone perhaps point me the right direction on this? > > Thanks, > > Sean Regards, -cpghost. --=20 Cordula's Web. http://www.cordula.ws/ From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:42:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B98F1065674 for ; Tue, 2 Aug 2011 09:42:29 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta04.westchester.pa.mail.comcast.net (qmta04.westchester.pa.mail.comcast.net [76.96.62.40]) by mx1.freebsd.org (Postfix) with ESMTP id 471058FC18 for ; Tue, 2 Aug 2011 09:42:28 +0000 (UTC) Received: from omta17.westchester.pa.mail.comcast.net ([76.96.62.89]) by qmta04.westchester.pa.mail.comcast.net with comcast id FM6i1h0051vXlb854MiVtU; Tue, 02 Aug 2011 09:42:29 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta17.westchester.pa.mail.comcast.net with comcast id FMiU1h00C1t3BNj3dMiUxj; Tue, 02 Aug 2011 09:42:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B3C7F102C36; Tue, 2 Aug 2011 02:42:26 -0700 (PDT) Date: Tue, 2 Aug 2011 02:42:26 -0700 From: Jeremy Chadwick To: "seanrees@gmail.com" Message-ID: <20110802094226.GA93114@icarus.home.lan> References: <20110802090830.GA92646@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:42:29 -0000 On Tue, Aug 02, 2011 at 10:16:35AM +0100, seanrees@gmail.com wrote: > On Tue, Aug 2, 2011 at 10:08 AM, Jeremy Chadwick > wrote: > > On Tue, Aug 02, 2011 at 08:39:03AM +0100, seanrees@gmail.com wrote: > >> On my FreeBSD 8.2-S machine (built circa 12th June), I created a > >> directory and populated it over the course of 3 weeks with about 2 > >> million individual files. > > > > I'll keep this real simple: > > > > Why did you do this? > > > > I hope this was a stress test of some kind. ?If not: > > Not really, but it turned into one. > > The camera I was using had the ability (rather handily) to upload a > still image once per second via FTP to a server of my choosing. It > didn't have the ability to organize them for me in a neat directory > hierarchy. So on holidays I went for 3 weeks and came back to ~2M > images in the same directory. I equate this to the following conversation with a doctor: "I shoved 2 million cotton balls into my ear, and now my hearing is sub-par". "Why did you do this?" In this situation, the correct reply to the physician is: "because I wasn't thinking, doc". I suppose the reason I'm being so brash is that people doing this always brings into question the thought process that went into the decision to do said thing (both on your part, as well as the engineers of your camera who thought such a feature would be a wise choice, especially with an interval of 1 picture per second[1]). When I was being taught the ropes of system administration at Oregon State, the team of crotchety UNIX admins there made it quite clear that there were things you just Did Not Do(tm) to computer systems. Shoving thousands of files into a single directory with no hierarchy was one of them. Again I will point this out: it doesn't matter what filesystem is used, they all will suffer (just to different degrees) in this situation. This is just one of those things where I have to say: Please Don't Do This(tm). At least this is an educational experience for you. > > This is the 2nd or 3rd mail in recent months from people saying "I > > decided to do something utterly stupid with my filesystem[1] and now I'm > > asking why performance sucks". > > > > Why can people not create proper directory tree layouts to avoid this > > problem regardless of what filesystem is used? ?I just don't get it. > > I'm not sure it's utterly stupid; I didn't expect legendarily fast > performance from 'ls' or anything else that enumerated the contents of > the directory when all the files were there. You shouldn't expect any kind of performance, at all, on any filesystem, when/if you do this. The effects can linger for quite some time. I can't come up with a good analogy at the moment. > Now that the files are neatly organized, I expected fstatfs() on the > directory to become fast again. It isn't. I'd like to understand why > (or maybe learn a new trick or two about inspecting ZFS...) Ronald's recommendation should address the problem, or at least diminish it in the least. [1]: I would also strongly advocate contacting your camera manufacturer and asking them to add some extremely simple/basic code for adding a directory hierarchy when the images are put on an FTP server. The amount of code is extremely nominal. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:58:39 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 281A71065773 for ; Tue, 2 Aug 2011 09:58:39 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 6D4E38FC0A for ; Tue, 2 Aug 2011 09:58:37 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p729k2Se052713 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 2 Aug 2011 19:16:08 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: <20110802094226.GA93114@icarus.home.lan> Date: Tue, 2 Aug 2011 19:16:02 +0930 Content-Transfer-Encoding: quoted-printable Message-Id: <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: -4.164 () ALL_TRUSTED,BAYES_00,RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-stable@freebsd.org, "seanrees@gmail.com" Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:58:39 -0000 On 02/08/2011, at 19:12, Jeremy Chadwick wrote: > When I was being taught the ropes of system administration at Oregon > State, the team of crotchety UNIX admins there made it quite clear = that > there were things you just Did Not Do(tm) to computer systems. = Shoving > thousands of files into a single directory with no hierarchy was one = of > them. Sounds like a terminal case of Stockholm syndrome ;) It might be avoidable by the user being nice to the computer, but come = on.. The computer is supposed to do tedious crap that humans don't like. I am pretty sure UFS does not have this problem. i.e. once you = delete/move the files out of the directory its performance would be good = again. If it is a limitation in ZFS it would be nice to know that, perhaps it = truly, really is a bug that can be avoided (or it's inherent in the way = ZFS handles such things) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 09:58:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD10D1065674 for ; Tue, 2 Aug 2011 09:58:40 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id E920F8FC18 for ; Tue, 2 Aug 2011 09:58:39 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p729gORK052555 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 2 Aug 2011 19:12:29 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=windows-1252 From: "Daniel O'Connor" In-Reply-To: <20110802090830.GA92646@icarus.home.lan> Date: Tue, 2 Aug 2011 19:12:24 +0930 Content-Transfer-Encoding: quoted-printable Message-Id: <960112E7-2331-449A-8736-C8790B05FE20@gsoft.com.au> References: <20110802090830.GA92646@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: -4.164 () ALL_TRUSTED,BAYES_00,RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-stable@freebsd.org, "seanrees@gmail.com" Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:58:40 -0000 On 02/08/2011, at 18:38, Jeremy Chadwick wrote: > On Tue, Aug 02, 2011 at 08:39:03AM +0100, seanrees@gmail.com wrote: >> On my FreeBSD 8.2-S machine (built circa 12th June), I created a >> directory and populated it over the course of 3 weeks with about 2 >> million individual files. >=20 > I'll keep this real simple: >=20 > Why did you do this? >=20 > I hope this was a stress test of some kind. If not: >=20 > This is the 2nd or 3rd mail in recent months from people saying "I > decided to do something utterly stupid with my filesystem[1] and now = I'm > asking why performance sucks". >=20 > Why can people not create proper directory tree layouts to avoid this > problem regardless of what filesystem is used? I just don't get it. >=20 > [1]: Applies to any filesystem, not just ZFS. There was a UFS one a > month or two ago too=85 The problem is that he is being punished with shitty FS performance even = though the directory structure is now non-silly. It sounds like the FS hasn't GC'd some (now unneeded) metadata.. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 10:37:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A8B3106564A for ; Tue, 2 Aug 2011 10:37:52 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 1EA328FC15 for ; Tue, 2 Aug 2011 10:37:51 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.4/8.14.4) with ESMTP id p72AARU0006708 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 2 Aug 2011 13:10:33 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4E37CD13.1070402@digsys.bg> Date: Tue, 02 Aug 2011 13:10:27 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110720 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> In-Reply-To: <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 10:37:52 -0000 On 02.08.11 12:46, Daniel O'Connor wrote: > I am pretty sure UFS does not have this problem. i.e. once you > delete/move the files out of the directory its performance would be > good again. UFS would be the classic example of poor performance if you do this. > If it is a limitation in ZFS it would be nice to know that, perhaps it > truly, really is a bug that can be avoided (or it's inherent in the > way ZFS handles such things) It is possible that there is not enough memory in ARC to cache that large directory. Other than that, perhaps in ZFS it would be easier to prune the unused directory entries, than it is in UFS. It looks like this is not implemented. Another reason might be some FreeBSD specific implementation issue for fstatfs. In any case, the data available is not sufficient. More information would help, like how much RAM this system has, how much ARC uses, some ARC stats. What made me wonder is .. how exactly the kernel and zpool disagree on zpool version? What is the pool version in fact? Daniel From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 10:55:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BE201065678 for ; Tue, 2 Aug 2011 10:55:44 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 37B488FC22 for ; Tue, 2 Aug 2011 10:55:43 +0000 (UTC) Received: by vxg33 with SMTP id 33so6772420vxg.13 for ; Tue, 02 Aug 2011 03:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=khoX/ChfFBRZI3Qt95h/yzl5ykJN/PW6uH/IAWwSmZU=; b=e7LjdBEbO8A2dKbi7oTCiVAg/EBcsqjDoGZxJDBSffwtAkpA/HaIZwp4Abu5Ew6zkZ uwNUW/me4DI0hIGXsVLK0lIP5Yz6avW8u+UnQqpdVcF3JFPFdUWghBgEeOGgHr0GJitO 5HVHN4fs0uPnVkuax1j30RM14N/XNJ9kDucVU= MIME-Version: 1.0 Received: by 10.52.71.1 with SMTP id q1mr5168895vdu.380.1312282543498; Tue, 02 Aug 2011 03:55:43 -0700 (PDT) Received: by 10.52.168.68 with HTTP; Tue, 2 Aug 2011 03:55:43 -0700 (PDT) In-Reply-To: <4E37CD13.1070402@digsys.bg> References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> Date: Tue, 2 Aug 2011 11:55:43 +0100 Message-ID: From: "seanrees@gmail.com" To: Daniel Kalchev Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 10:55:44 -0000 On Tue, Aug 2, 2011 at 11:10 AM, Daniel Kalchev wrote: >> If it is a limitation in ZFS it would be nice to know that, perhaps it >> truly, really is a bug that can be avoided (or it's inherent in the way = ZFS >> handles such things) > > It is possible =A0that there is not enough memory in ARC to cache that la= rge > directory. > > Other than that, perhaps in ZFS it would be easier to prune the unused > directory entries, than it is in UFS. It looks like this is not implement= ed. > > Another reason might be some FreeBSD specific implementation issue for > fstatfs. > > In any case, the data available is not sufficient. More information would > help, like how much RAM this system has, how much ARC uses, some ARC stat= s. Which sysctl's would you like? I grabbed these to start: kstat.zfs.misc.arcstats.size: 118859656 kstat.zfs.misc.arcstats.hdr_size: 3764416 kstat.zfs.misc.arcstats.data_size: 53514240 kstat.zfs.misc.arcstats.other_size: 61581000 kstat.zfs.misc.arcstats.hits: 46762467 kstat.zfs.misc.arcstats.misses: 16999907 The machine has 2GB of memory. > What made me wonder is .. how exactly the kernel and zpool disagree on zp= ool > version? What is the pool version in fact? % dmesg | grep ZFS ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is pres= ent; to enable, add "vfs.zfs.prefetch_disable=3D0" to /boot/loader.c= onf. ZFS filesystem version 5 ZFS storage pool version 28 % zpool get version tank NAME PROPERTY VALUE SOURCE tank version 15 local % zpool upgrade tank This system is currently running ZFS pool version 15. Pool 'tank' is already formatted using the current version. Sean From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 05:10:23 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4166106564A; Tue, 2 Aug 2011 05:10:23 +0000 (UTC) (envelope-from miwi.freebsd@googlemail.com) Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by mx1.freebsd.org (Postfix) with ESMTP id 5D0818FC08; Tue, 2 Aug 2011 05:10:23 +0000 (UTC) Received: by pzk5 with SMTP id 5so36759193pzk.17 for ; Mon, 01 Aug 2011 22:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=sender:from:content-type:subject:date:message-id:cc:to:mime-version :x-mailer; bh=IDijeas8D5NwjNiMNJCSgZj0HL+ZYwVZaFNHYDtNwf8=; b=Ct6iHw9jRzcrOF9bzif8EQlSBvUFwUHQ45WQ63F0u4oAh5C0lShWb8HaVNaUH5T/gD bYlvGD0x6bzbBJXcLv2lPUOx0oN/YNIsDfFK+cx4Py4ekzp5Ltq3B7f7z5gcZWOzSucD 2+i/mzxOsAZ7DIVxiFz13+f3ZKGhRTspGwx3A= Received: by 10.68.39.137 with SMTP id p9mr1785024pbk.436.1312260098213; Mon, 01 Aug 2011 21:41:38 -0700 (PDT) Received: from [192.168.0.103] ([175.142.228.27]) by mx.google.com with ESMTPS id e6sm6315115pbm.39.2011.08.01.21.41.32 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Aug 2011 21:41:35 -0700 (PDT) Sender: Martin Wilke From: Martin Wilke Date: Tue, 2 Aug 2011 12:41:29 +0800 Message-Id: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> To: current@FreeBSD.org Mime-Version: 1.0 (Apple Message framework v1247) X-Mailer: Apple Mail (2.1247) X-Mailman-Approved-At: Tue, 02 Aug 2011 10:56:20 +0000 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: stable@FreeBSD.org Subject: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 05:10:23 -0000 9.0 Beta1 Panic Hi guys, I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please = view attached screenshot for the error. If you need more information, do = let us know. Thanks. From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 11:07:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58F11106564A for ; Tue, 2 Aug 2011 11:07:42 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id 3BF048FC14 for ; Tue, 2 Aug 2011 11:07:40 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta01.emeryville.ca.mail.comcast.net with comcast id FNDA1h0071vN32cA1P7co9; Tue, 02 Aug 2011 11:07:36 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.emeryville.ca.mail.comcast.net with comcast id FP9m1h0051t3BNj8iP9mSp; Tue, 02 Aug 2011 11:09:47 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 463D6102C36; Tue, 2 Aug 2011 04:07:38 -0700 (PDT) Date: Tue, 2 Aug 2011 04:07:38 -0700 From: Jeremy Chadwick To: "seanrees@gmail.com" Message-ID: <20110802110738.GA95225@icarus.home.lan> References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org, Daniel Kalchev Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 11:07:42 -0000 On Tue, Aug 02, 2011 at 11:55:43AM +0100, seanrees@gmail.com wrote: > On Tue, Aug 2, 2011 at 11:10 AM, Daniel Kalchev wrote: > >> If it is a limitation in ZFS it would be nice to know that, perhaps it > >> truly, really is a bug that can be avoided (or it's inherent in the way ZFS > >> handles such things) > > > > It is possible ?that there is not enough memory in ARC to cache that large > > directory. > > > > Other than that, perhaps in ZFS it would be easier to prune the unused > > directory entries, than it is in UFS. It looks like this is not implemented. > > > > Another reason might be some FreeBSD specific implementation issue for > > fstatfs. > > > > In any case, the data available is not sufficient. More information would > > help, like how much RAM this system has, how much ARC uses, some ARC stats. > > Which sysctl's would you like? Output from "sysctl vfs.zfs kstat.zfs" would be sufficient. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 11:10:20 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A732106566C for ; Tue, 2 Aug 2011 11:10:20 +0000 (UTC) (envelope-from laa@cemu.ru) Received: from m.cemu.ru (m.cemu.ru [194.186.216.68]) by mx1.freebsd.org (Postfix) with ESMTP id EB0E08FC13 for ; Tue, 2 Aug 2011 11:10:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=laa.zp.ua; s=dkim; h=Sender:In-Reply-To:Content-Type:Mime-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=Tf09e/+J6Tzkw5FiZA/DUk6q+NJqZniF8IY7YIEymg4=; b=YtvaozjjOEH3Ba4G859C6ZKaFn61nOej5ePjZIMe0MCxhHXkuKX2ad2emVhW2QPsBqmPlkxhO4U+8lf1eQHWIQS110J4SNNrDMnVREjByl1dTbvRyBZIp1z6M9uJ58Gb1fZq/oYiOB/+xVOlnA++KYLOq7jJEdrrrQlijXHf8RQ=; Received: by m.cemu.ru with local (Exim) (envelope-from ) id 1QoCfp-000NSk-Gt; Tue, 02 Aug 2011 14:58:09 +0400 Date: Tue, 2 Aug 2011 14:58:09 +0400 From: Lystopad Olexandr To: Martin Wilke Message-ID: <20110802105809.GT2369@laa.zp.ua> References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Sender: laa Cc: stable@FreeBSD.org Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 11:10:20 -0000 Hello, Martin Wilke! On Tue, Aug 02, 2011 at 12:41:29PM +0800 miwi@FreeBSD.org wrote about "9.0 B1 Panic": > 9.0 Beta1 Panic > > Hi guys, > > I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please view attached screenshot for the error. If you need more information, do let us know. There no attachments in your mail. -- Lystopad Olexandr From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 11:37:34 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02E4D106566C; Tue, 2 Aug 2011 11:37:34 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id C78698FC1D; Tue, 2 Aug 2011 11:37:33 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QoDHw-000Pog-Qi; Tue, 02 Aug 2011 07:37:32 -0400 Date: Tue, 2 Aug 2011 07:37:32 -0400 From: Gary Palmer To: Lystopad Olexandr , Martin Wilke Message-ID: <20110802113732.GA88904@in-addr.com> References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> <20110802105809.GT2369@laa.zp.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110802105809.GT2369@laa.zp.ua> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: stable@FreeBSD.org Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 11:37:34 -0000 On Tue, Aug 02, 2011 at 02:58:09PM +0400, Lystopad Olexandr wrote: > Hello, Martin Wilke! > > On Tue, Aug 02, 2011 at 12:41:29PM +0800 > miwi@FreeBSD.org wrote about "9.0 B1 Panic": > > 9.0 Beta1 Panic > > > > Hi guys, > > > > I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please view attached screenshot for the error. If you need more information, do let us know. > > There no attachments in your mail. The mailing list strips all non-text attachments. Please post the attachments (like screen shots) on a web site and post the URL Gary From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 11:38:33 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4CA0C1065672; Tue, 2 Aug 2011 11:38:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 271C78FC16; Tue, 2 Aug 2011 11:38:33 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id BEFC746B09; Tue, 2 Aug 2011 07:38:32 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5062B8A02C; Tue, 2 Aug 2011 07:38:32 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org, stable@freebsd.org Date: Tue, 2 Aug 2011 07:38:30 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> In-Reply-To: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201108020738.31282.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 02 Aug 2011 07:38:32 -0400 (EDT) Cc: Martin Wilke Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 11:38:33 -0000 On Tuesday, August 02, 2011 12:41:29 am Martin Wilke wrote: > 9.0 Beta1 Panic > > Hi guys, > > I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please view attached screenshot for the error. If you need more information, do let us know. > > Thanks. Unfortunately the attachment was lost, can you post it to a URL? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 12:02:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6535D106567D for ; Tue, 2 Aug 2011 12:02:51 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1A4F68FC16 for ; Tue, 2 Aug 2011 12:02:50 +0000 (UTC) Received: by vxg33 with SMTP id 33so6849238vxg.13 for ; Tue, 02 Aug 2011 05:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XGUnc7Sn90hjSgGSiNdNIyCuDgAgRw/CeaZx2v6Rjz8=; b=UWqNKk224qkfog6QCzUOX6SB73+RudJg4aFsQI74LB6kShozCn4/O8Qmiu66fmn5CO cnNkZL7Pe1ZJN9FG1QkVwJP0nx6kQ0sjTqsS0MedLZ+Uriawm9pufMDBJHd6u4sHYfx/ 58UjvKbaoMFUxOQqBlUf+pAKcqjaHWVT8YjvA= MIME-Version: 1.0 Received: by 10.52.91.130 with SMTP id ce2mr5413981vdb.447.1312286570349; Tue, 02 Aug 2011 05:02:50 -0700 (PDT) Received: by 10.52.168.68 with HTTP; Tue, 2 Aug 2011 05:02:50 -0700 (PDT) In-Reply-To: <20110802110738.GA95225@icarus.home.lan> References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> <20110802110738.GA95225@icarus.home.lan> Date: Tue, 2 Aug 2011 13:02:50 +0100 Message-ID: From: "seanrees@gmail.com" To: Jeremy Chadwick Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org, Daniel Kalchev Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:02:51 -0000 On Tue, Aug 2, 2011 at 12:07 PM, Jeremy Chadwick wrote: > On Tue, Aug 02, 2011 at 11:55:43AM +0100, seanrees@gmail.com wrote: >> On Tue, Aug 2, 2011 at 11:10 AM, Daniel Kalchev wrote: >> >> If it is a limitation in ZFS it would be nice to know that, perhaps it >> >> truly, really is a bug that can be avoided (or it's inherent in the way ZFS >> >> handles such things) >> > >> > It is possible ?that there is not enough memory in ARC to cache that large >> > directory. >> > >> > Other than that, perhaps in ZFS it would be easier to prune the unused >> > directory entries, than it is in UFS. It looks like this is not implemented. >> > >> > Another reason might be some FreeBSD specific implementation issue for >> > fstatfs. >> > >> > In any case, the data available is not sufficient. More information would >> > help, like how much RAM this system has, how much ARC uses, some ARC stats. >> >> Which sysctl's would you like? > > Output from "sysctl vfs.zfs kstat.zfs" would be sufficient. Here we are: vfs.zfs.l2c_only_size: 0 vfs.zfs.mfu_ghost_data_lsize: 0 vfs.zfs.mfu_ghost_metadata_lsize: 26383360 vfs.zfs.mfu_ghost_size: 26383360 vfs.zfs.mfu_data_lsize: 0 vfs.zfs.mfu_metadata_lsize: 154112 vfs.zfs.mfu_size: 3944960 vfs.zfs.mru_ghost_data_lsize: 0 vfs.zfs.mru_ghost_metadata_lsize: 76250624 vfs.zfs.mru_ghost_size: 76250624 vfs.zfs.mru_data_lsize: 30208 vfs.zfs.mru_metadata_lsize: 16896 vfs.zfs.mru_size: 29353984 vfs.zfs.anon_data_lsize: 0 vfs.zfs.anon_metadata_lsize: 0 vfs.zfs.anon_size: 150016 vfs.zfs.l2arc_norw: 1 vfs.zfs.l2arc_feed_again: 1 vfs.zfs.l2arc_noprefetch: 1 vfs.zfs.l2arc_feed_min_ms: 200 vfs.zfs.l2arc_feed_secs: 1 vfs.zfs.l2arc_headroom: 2 vfs.zfs.l2arc_write_boost: 8388608 vfs.zfs.l2arc_write_max: 8388608 vfs.zfs.arc_meta_limit: 26214400 vfs.zfs.arc_meta_used: 108539456 vfs.zfs.arc_min: 33554432 vfs.zfs.arc_max: 104857600 vfs.zfs.dedup.prefetch: 1 vfs.zfs.mdcomp_disable: 0 vfs.zfs.write_limit_override: 0 vfs.zfs.write_limit_inflated: 6360993792 vfs.zfs.write_limit_max: 265041408 vfs.zfs.write_limit_min: 33554432 vfs.zfs.write_limit_shift: 3 vfs.zfs.no_write_throttle: 0 vfs.zfs.zfetch.array_rd_sz: 1048576 vfs.zfs.zfetch.block_cap: 256 vfs.zfs.zfetch.min_sec_reap: 2 vfs.zfs.zfetch.max_streams: 8 vfs.zfs.prefetch_disable: 1 vfs.zfs.check_hostid: 1 vfs.zfs.recover: 0 vfs.zfs.txg.synctime_ms: 1000 vfs.zfs.txg.timeout: 5 vfs.zfs.scrub_limit: 10 vfs.zfs.vdev.cache.bshift: 16 vfs.zfs.vdev.cache.size: 10485760 vfs.zfs.vdev.cache.max: 16384 vfs.zfs.vdev.write_gap_limit: 4096 vfs.zfs.vdev.read_gap_limit: 32768 vfs.zfs.vdev.aggregation_limit: 131072 vfs.zfs.vdev.ramp_rate: 2 vfs.zfs.vdev.time_shift: 6 vfs.zfs.vdev.min_pending: 4 vfs.zfs.vdev.max_pending: 10 vfs.zfs.vdev.bio_flush_disable: 0 vfs.zfs.cache_flush_disable: 0 vfs.zfs.zil_replay_disable: 0 vfs.zfs.zio.use_uma: 0 vfs.zfs.version.zpl: 5 vfs.zfs.version.spa: 28 vfs.zfs.version.acl: 1 vfs.zfs.debug: 0 vfs.zfs.super_owner: 0 kstat.zfs.misc.xuio_stats.onloan_read_buf: 0 kstat.zfs.misc.xuio_stats.onloan_write_buf: 0 kstat.zfs.misc.xuio_stats.read_buf_copied: 0 kstat.zfs.misc.xuio_stats.read_buf_nocopy: 0 kstat.zfs.misc.xuio_stats.write_buf_copied: 0 kstat.zfs.misc.xuio_stats.write_buf_nocopy: 107064 kstat.zfs.misc.zfetchstats.hits: 0 kstat.zfs.misc.zfetchstats.misses: 0 kstat.zfs.misc.zfetchstats.colinear_hits: 0 kstat.zfs.misc.zfetchstats.colinear_misses: 0 kstat.zfs.misc.zfetchstats.stride_hits: 0 kstat.zfs.misc.zfetchstats.stride_misses: 0 kstat.zfs.misc.zfetchstats.reclaim_successes: 0 kstat.zfs.misc.zfetchstats.reclaim_failures: 0 kstat.zfs.misc.zfetchstats.streams_resets: 0 kstat.zfs.misc.zfetchstats.streams_noresets: 0 kstat.zfs.misc.zfetchstats.bogus_streams: 0 kstat.zfs.misc.arcstats.hits: 47091548 kstat.zfs.misc.arcstats.misses: 17064059 kstat.zfs.misc.arcstats.demand_data_hits: 15357194 kstat.zfs.misc.arcstats.demand_data_misses: 3077290 kstat.zfs.misc.arcstats.demand_metadata_hits: 31102404 kstat.zfs.misc.arcstats.demand_metadata_misses: 8692242 kstat.zfs.misc.arcstats.prefetch_data_hits: 0 kstat.zfs.misc.arcstats.prefetch_data_misses: 0 kstat.zfs.misc.arcstats.prefetch_metadata_hits: 631950 kstat.zfs.misc.arcstats.prefetch_metadata_misses: 5294527 kstat.zfs.misc.arcstats.mru_hits: 27566971 kstat.zfs.misc.arcstats.mru_ghost_hits: 2179308 kstat.zfs.misc.arcstats.mfu_hits: 18950663 kstat.zfs.misc.arcstats.mfu_ghost_hits: 2714218 kstat.zfs.misc.arcstats.allocated: 19825272 kstat.zfs.misc.arcstats.deleted: 12619489 kstat.zfs.misc.arcstats.stolen: 9003539 kstat.zfs.misc.arcstats.recycle_miss: 10224598 kstat.zfs.misc.arcstats.mutex_miss: 1984 kstat.zfs.misc.arcstats.evict_skip: 216358592 kstat.zfs.misc.arcstats.evict_l2_cached: 0 kstat.zfs.misc.arcstats.evict_l2_eligible: 433025541120 kstat.zfs.misc.arcstats.evict_l2_ineligible: 87633796096 kstat.zfs.misc.arcstats.hash_elements: 15988 kstat.zfs.misc.arcstats.hash_elements_max: 43365 kstat.zfs.misc.arcstats.hash_collisions: 5599202 kstat.zfs.misc.arcstats.hash_chains: 3944 kstat.zfs.misc.arcstats.hash_chain_max: 21 kstat.zfs.misc.arcstats.p: 28381184 kstat.zfs.misc.arcstats.c: 104857600 kstat.zfs.misc.arcstats.c_min: 33554432 kstat.zfs.misc.arcstats.c_max: 104857600 kstat.zfs.misc.arcstats.size: 108700736 kstat.zfs.misc.arcstats.hdr_size: 3677448 kstat.zfs.misc.arcstats.data_size: 33448960 kstat.zfs.misc.arcstats.other_size: 71574328 kstat.zfs.misc.arcstats.l2_hits: 0 kstat.zfs.misc.arcstats.l2_misses: 0 kstat.zfs.misc.arcstats.l2_feeds: 0 kstat.zfs.misc.arcstats.l2_rw_clash: 0 kstat.zfs.misc.arcstats.l2_read_bytes: 0 kstat.zfs.misc.arcstats.l2_write_bytes: 0 kstat.zfs.misc.arcstats.l2_writes_sent: 0 kstat.zfs.misc.arcstats.l2_writes_done: 0 kstat.zfs.misc.arcstats.l2_writes_error: 0 kstat.zfs.misc.arcstats.l2_writes_hdr_miss: 0 kstat.zfs.misc.arcstats.l2_evict_lock_retry: 0 kstat.zfs.misc.arcstats.l2_evict_reading: 0 kstat.zfs.misc.arcstats.l2_free_on_write: 0 kstat.zfs.misc.arcstats.l2_abort_lowmem: 0 kstat.zfs.misc.arcstats.l2_cksum_bad: 0 kstat.zfs.misc.arcstats.l2_io_error: 0 kstat.zfs.misc.arcstats.l2_size: 0 kstat.zfs.misc.arcstats.l2_hdr_size: 0 kstat.zfs.misc.arcstats.memory_throttle_count: 521 kstat.zfs.misc.arcstats.l2_write_trylock_fail: 0 kstat.zfs.misc.arcstats.l2_write_passed_headroom: 0 kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 kstat.zfs.misc.arcstats.l2_write_in_l2: 0 kstat.zfs.misc.arcstats.l2_write_io_in_progress: 0 kstat.zfs.misc.arcstats.l2_write_not_cacheable: 5352891 kstat.zfs.misc.arcstats.l2_write_full: 0 kstat.zfs.misc.arcstats.l2_write_buffer_iter: 0 kstat.zfs.misc.arcstats.l2_write_pios: 0 kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 0 kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 0 kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 0 kstat.zfs.misc.vdev_cache_stats.delegations: 5349195 kstat.zfs.misc.vdev_cache_stats.hits: 16581136 kstat.zfs.misc.vdev_cache_stats.misses: 5391734 Sean From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 12:55:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D62BE1065670 for ; Tue, 2 Aug 2011 12:55:40 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3fd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 414FE8FC12 for ; Tue, 2 Aug 2011 12:55:40 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id p72CtZsc066415 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 2 Aug 2011 13:55:36 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp.infracaninophile.co.uk p72CtZsc066415 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1312289736; bh=eBw6j4b2hdx+W457qKqWNSNlJgL9opOh0qa099FRY0A=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Cc:Content-Type:Date:From:In-Reply-To: Message-ID:Mime-Version:References:To; z=Message-ID:=20<4E37F3BE.5040207@infracaninophile.co.uk>|Date:=20T ue,=2002=20Aug=202011=2013:55:26=20+0100|From:=20Matthew=20Seaman= 20|User-Agent:=20Mozilla/5.0=20(M acintosh=3B=20Intel=20Mac=20OS=20X=2010.6=3B=20rv:5.0)=20Gecko/201 10624=20Thunderbird/5.0|MIME-Version:=201.0|To:=20freebsd-stable@f reebsd.org|Subject:=20Re:=20ZFS=20directory=20with=20a=20large=20n umber=20of=20files|References:=20=20<20110802090830.GA92646@icar us.home.lan>=20=20<20110802094226.GA93114@icarus.home.lan>=20<42 039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au>=20<4E37CD13.10704 02@digsys.bg>|In-Reply-To:=20<4E37CD13.1070402@digsys.bg>|X-Enigma il-Version:=201.2|OpenPGP:=20id=3D60AE908C|Content-Type:=20multipa rt/signed=3B=20micalg=3Dpgp-sha1=3B=0D=0A=20protocol=3D"applicatio n/pgp-signature"=3B=0D=0A=20boundary=3D"------------enigB5D6A62DEB 22158D1A0D9F01"; b=sNCB/tK+M1647ft7h6WROA3MWdSOUM7XTPBxmCdRSbTtow96ro1gmwecNEBde8Tpn 9SPFksJmn279yXxsXAzIK7OYuLaF+j4G5IFtwXMuPtJGviBnKtazq+PxpP5KFxla3v opyDGpEMI+wCPwVxaUFu1Y5A9i4XRUGa/a8+uR1s= Message-ID: <4E37F3BE.5040207@infracaninophile.co.uk> Date: Tue, 02 Aug 2011 13:55:26 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> In-Reply-To: <4E37CD13.1070402@digsys.bg> X-Enigmail-Version: 1.2 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB5D6A62DEB22158D1A0D9F01" X-Virus-Scanned: clamav-milter 0.97.2 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=BAYES_40,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,SPF_FAIL autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:55:40 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB5D6A62DEB22158D1A0D9F01 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 02/08/2011 11:10, Daniel Kalchev wrote: > Other than that, perhaps in ZFS it would be easier to prune the unused > directory entries, than it is in UFS. It looks like this is not > implemented. Remember that ZFS uses copy-on-write for all filesystem updates. Any change to a directory contents means the whole directory data is rewritten. In which case, there's no benefit to retaining a large data structure with lots of empty slots (as happened on Unix FSes in the past.) I'd expect, and I see in my (admittedly fairly cursory) testing that ZFS directory data sizes update immediately whenever files are added or removed from the directory. Where this gets interesting is when the directory gets sufficiently large that the directory data is larger than the 128kB block size used by ZFS. As that takes many more files than any sensible person would put into one directory it's possible there's a bug in handling such large structures which is only rarely tickled. But this is all speculation on my behalf, and I have no evidence to back it up. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enigB5D6A62DEB22158D1A0D9F01 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4388YACgkQ8Mjk52CukIxSbwCfRFz96Y2hycFGs2NPQPJepM87 iWgAnilMjLGPTepRGr37CBMd2M0DoY3f =V6y7 -----END PGP SIGNATURE----- --------------enigB5D6A62DEB22158D1A0D9F01-- From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 12:58:17 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AA011065679; Tue, 2 Aug 2011 12:58:17 +0000 (UTC) (envelope-from miwi.freebsd@googlemail.com) Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3BB608FC14; Tue, 2 Aug 2011 12:58:17 +0000 (UTC) Received: by pzk5 with SMTP id 5so38540021pzk.17 for ; Tue, 02 Aug 2011 05:58:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=DD3qIFq0WO1e5y1sg06c5RX1BikXQJD/e7XKk4pl82k=; b=ZtM0Q8hfiD+Hce4NPMXFYvhsj7TcAr+JX3grXXOp0XM8C7ibBYAa4PdfExOUmCCotq rb9hRhEgcxtEf1SYSAOgDlfLf7Qvu/olBsMM/f6Kl7uLHMjJJNHJVQDbi+c7uWvhT2xa g2+8NEtRE/syYoFmGRQYTfMONBnUgmYjtXtCc= Received: by 10.68.0.170 with SMTP id 10mr9190992pbf.244.1312289896454; Tue, 02 Aug 2011 05:58:16 -0700 (PDT) Received: from [192.168.0.103] ([175.142.228.27]) by mx.google.com with ESMTPS id i9sm6738328pbk.84.2011.08.02.05.58.13 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 02 Aug 2011 05:58:15 -0700 (PDT) Sender: Martin Wilke Mime-Version: 1.0 (Apple Message framework v1247) Content-Type: text/plain; charset=koi8-r From: Martin Wilke In-Reply-To: <20110802105809.GT2369@laa.zp.ua> Date: Tue, 2 Aug 2011 20:58:10 +0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> <20110802105809.GT2369@laa.zp.ua> To: Lystopad Olexandr X-Mailer: Apple Mail (2.1247) Cc: stable@FreeBSD.org, current@FreeBSD.org Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:58:17 -0000 On Aug 2, 2011, at 6:58 PM, Lystopad Olexandr wrote: > Hello, Martin Wilke! >=20 > On Tue, Aug 02, 2011 at 12:41:29PM +0800 > miwi@FreeBSD.org wrote about "9.0 B1 Panic": >> 9.0 Beta1 Panic >>=20 >> Hi guys, >>=20 >> I just downloaded and install 9.0 BETA1 but it panics on ACPI. = Please view attached screenshot for the error. If you need more = information, do let us know. >=20 > There no attachments in your mail. Erms Sorry forgot about that. http://people.freebsd.org/~miwi/90b1.png >=20 > --=20 > Lystopad Olexandr=20 From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 16:56:55 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31091106567E; Tue, 2 Aug 2011 16:56:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 01CCE8FC26; Tue, 2 Aug 2011 16:56:55 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AE1DF46B2D; Tue, 2 Aug 2011 12:56:54 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4EF968A02E; Tue, 2 Aug 2011 12:56:54 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Tue, 2 Aug 2011 12:56:53 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> <20110802105809.GT2369@laa.zp.ua> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Message-Id: <201108021256.53224.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 02 Aug 2011 12:56:54 -0400 (EDT) Cc: current@freebsd.org, stable@freebsd.org, Lystopad Olexandr , Martin Wilke Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 16:56:55 -0000 On Tuesday, August 02, 2011 8:58:10 am Martin Wilke wrote: > > On Aug 2, 2011, at 6:58 PM, Lystopad Olexandr wrote: > > > Hello, Martin Wilke! > > > > On Tue, Aug 02, 2011 at 12:41:29PM +0800 > > miwi@FreeBSD.org wrote about "9.0 B1 Panic": > >> 9.0 Beta1 Panic > >> > >> Hi guys, > >> > >> I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please view attached screenshot for the error. If you need more information, do let us know. > > > > There no attachments in your mail. > > > Erms Sorry forgot about that. > > http://people.freebsd.org/~miwi/90b1.png Can you get a stack trace? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 17:46:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11A80106564A; Tue, 2 Aug 2011 17:46:19 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 900458FC0A; Tue, 2 Aug 2011 17:46:18 +0000 (UTC) Received: by qyk38 with SMTP id 38so4159qyk.13 for ; Tue, 02 Aug 2011 10:46:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=c5ovBfIE9mFuAaMFeQL+B4pSz/glrZ6LlQ40+9UghjI=; b=cfLP7nIzdncWtS2Cuca37F+exK/JME9DeCYHjt08LG3DCZf/2RfrkYI1UOZBvk31ps +x0pJZ4Kb7FymLlsTyL1KG82KSreBB0LCYBlJIuDsL53IQQ5b1vipcJcGvL2tHJDYe60 HzGuOywoa7L6em+2eywxrHGhxvbsCw3nGCOqU= MIME-Version: 1.0 Received: by 10.229.2.68 with SMTP id 4mr1973521qci.174.1312307177684; Tue, 02 Aug 2011 10:46:17 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Tue, 2 Aug 2011 10:46:17 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> Date: Tue, 2 Aug 2011 10:46:17 -0700 Message-ID: From: maestro something To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 17:46:19 -0000 Hi, just finished installing FreeBSD-9BETA1 and recompiling the kernel with dtrace. This is even worse, I have the same behavior as mentioned here: http://freebsd.1045724.n5.nabble.com/bin-158431-dtrace-crash-in-dt-proc-lookup-when-attaching-to-PID-assert-dpr-NULL-tt4535367.html#none i.e., dtrace regardless of whether with or without any probes just quits with the following error message Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. that's inside dt_proc_lookup() I think I have to give up on ustack(), too bad cheers --m On Sat, Jul 30, 2011 at 2:05 PM, maestro something wrote: > Hi, > >> This is i386, right ? >>> I think the cause is that assembler routine panic_trigger does not >>> establish the standard i386 frame. Basically, you need either this, >>> or dwarf annotations, for gdb to be able to walk over the frame. >>> >>> You need to add the standard prologue >>> pushl %ebp >>> movl %esp,%ebp >>> and standard epilogue >>> leave >>> to the function. No idea whether it will continue to operate correctly >>> after. >>> >> >> my panic_trigger looks like this now: >> >> /* >> int >> panic_trigger(int *tp) >> */ >> ENTRY(panic_trigger) >> >> pushl %ebp >> movl %esp,%ebp >> xorl %eax, %eax >> movl $0xdefacedd, %edx >> lock >> xchgl %edx, (%edi) >> cmpl $0, %edx >> je 0f >> movl $0, %eax >> leave >> ret >> 0: movl $1, %eax >> leave >> ret >> END(panic_trigger) >> >> same result, (actually too same as the address in the stack trace is still >> the same, is that possible?) >> >> > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e99db at dtrace_panic+0x1b > #7 0xc10e9a0d at dtrace_assfail+0x2d > #8 0xc10fa6a6 at dtrace_probe+0xfd6 > #9 0xc1237ce4 at systrace_probe+0x84 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > I tried playing around with kgdb a bit. It finds source files until frame > #10 (i.e., syscallenter + 0x47f) > > (kgdb) list *syscallenter+0x47f > 0xc090f63f is in syscallenter (/usr/src/sys/kern/subr_trap.c:328). > 323 * If the systrace module has registered it's probe > 324 * callback and if there is a probe active for the > 325 * syscall 'return', process the probe. > 326 */ > 327 if (systrace_probe_func != NULL && sa->callp->sy_return != > 0) > 328 (*systrace_probe_func)(sa->callp->sy_return, sa->code, > 329 sa->callp, sa->args); > 330 #endif > 331 CTR4(KTR_SYSC, "syscall: p=%p error=%d return %#lx %#lx", > 332 p, error, td->td_retval[0], td->td_retval[1]); > > (kgdb) list *systrace_probe+0x84 > No source file for address 0xc1237ce4. > > Thus, it seems that the first frame where kgdb cannot find a source file is > #9 (i.e., systrace_probe + 0x84) > As far as I understand that's when the (imho) function pointer > systrace_probe_func gets invoked. Therefore, it seems that kgdb has a hard > time finding the source file for the function invoked through that pointer. > Is this complete nonsense, or does that actually sound familiar to anyone? > > And still I'm wondering whether anybody ever successfully used the ustack > action on Freebsd-8.2 i386 > > cheers > --m > From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 18:34:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B43D106564A; Tue, 2 Aug 2011 18:34:31 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2D0818FC0A; Tue, 2 Aug 2011 18:34:30 +0000 (UTC) Received: by qwc9 with SMTP id 9so48564qwc.13 for ; Tue, 02 Aug 2011 11:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=60A+cFrSsV4TxFwqFrazNj59C/6U8lL19w19t29iD0Y=; b=pjMW0+UlaV0YMv5TcCVXBBtSv8rVIwzp4OjzkRR+ZKYM+NHZOQ7zQIYZs3iGRB6dn0 y7xk2TAUkaAVlPp+qICwiYzmH4AgNTOZWj/TeukvnMkcvnVnbIHqTBETkfJACa9+GdPH L32iXJ3FcAQ8Aw55pDeMcASKgwG3zntOTwfmQ= MIME-Version: 1.0 Received: by 10.229.2.68 with SMTP id 4mr2013050qci.174.1312310070250; Tue, 02 Aug 2011 11:34:30 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Tue, 2 Aug 2011 11:34:30 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> Date: Tue, 2 Aug 2011 11:34:30 -0700 Message-ID: From: maestro something To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 18:34:31 -0000 Hi, This is even worse, I have the same behavior as mentioned here: > > http://freebsd.1045724.n5.nabble.com/bin-158431-dtrace-crash-in-dt-proc-lookup-when-attaching-to-PID-assert-dpr-NULL-tt4535367.html#none > > i.e., dtrace regardless of whether with or without any probes just quits > with the following error message this is not accurate. It seems that this message only occurs if i use the ustack() action. i.e, dtrace -n 'syscall::accept:return /execname == "nc" / { printf("%s accept:return\n", execname);}' works as expected However, dtrace -n 'syscall::accept:return /execname == "nc" / { printf("%s accept:return\n", execname);ustack();}' does not: it results in the mentioned error message. Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. cheers --m > > > Assertion failed: (dpr != NULL), file > /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, > line 751. > > that's inside dt_proc_lookup() > > I think I have to give up on ustack(), too bad > > cheers > --m > > > On Sat, Jul 30, 2011 at 2:05 PM, maestro something wrote: > >> Hi, >> >>> This is i386, right ? >>>> I think the cause is that assembler routine panic_trigger does not >>>> establish the standard i386 frame. Basically, you need either this, >>>> or dwarf annotations, for gdb to be able to walk over the frame. >>>> >>>> You need to add the standard prologue >>>> pushl %ebp >>>> movl %esp,%ebp >>>> and standard epilogue >>>> leave >>>> to the function. No idea whether it will continue to operate correctly >>>> after. >>>> >>> >>> my panic_trigger looks like this now: >>> >>> /* >>> int >>> panic_trigger(int *tp) >>> */ >>> ENTRY(panic_trigger) >>> >>> pushl %ebp >>> movl %esp,%ebp >>> xorl %eax, %eax >>> movl $0xdefacedd, %edx >>> lock >>> xchgl %edx, (%edi) >>> cmpl $0, %edx >>> je 0f >>> movl $0, %eax >>> leave >>> ret >>> 0: movl $1, %eax >>> leave >>> ret >>> END(panic_trigger) >>> >>> same result, (actually too same as the address in the stack trace is >>> still the same, is that possible?) >>> >>> >> KDB: stack backtrace: >> #0 0xc09036a7 at kdb_backtrace+0x47 >> #1 0xc08d1a07 at panic+0x117 >> #2 0xc0c158c3 at trap_fatal+0x323 >> #3 0xc0c15bc0 at trap_pfault+0x2f0 >> #4 0xc0c1612a at trap+0x48a >> #5 0xc0bfc97c at calltrap+0x6 >> #6 0xc10e99db at dtrace_panic+0x1b >> #7 0xc10e9a0d at dtrace_assfail+0x2d >> #8 0xc10fa6a6 at dtrace_probe+0xfd6 >> #9 0xc1237ce4 at systrace_probe+0x84 >> #10 0xc090f63f at syscallenter+0x47f >> #11 0xc0c15c14 at syscall+0x34 >> #12 0xc0bfca11 at Xint0x80_syscall+0x21 >> >> I tried playing around with kgdb a bit. It finds source files until frame >> #10 (i.e., syscallenter + 0x47f) >> >> (kgdb) list *syscallenter+0x47f >> 0xc090f63f is in syscallenter (/usr/src/sys/kern/subr_trap.c:328). >> 323 * If the systrace module has registered it's probe >> 324 * callback and if there is a probe active for the >> 325 * syscall 'return', process the probe. >> 326 */ >> 327 if (systrace_probe_func != NULL && sa->callp->sy_return != >> 0) >> 328 (*systrace_probe_func)(sa->callp->sy_return, sa->code, >> 329 sa->callp, sa->args); >> 330 #endif >> 331 CTR4(KTR_SYSC, "syscall: p=%p error=%d return %#lx %#lx", >> 332 p, error, td->td_retval[0], td->td_retval[1]); >> >> (kgdb) list *systrace_probe+0x84 >> No source file for address 0xc1237ce4. >> >> Thus, it seems that the first frame where kgdb cannot find a source file >> is #9 (i.e., systrace_probe + 0x84) >> As far as I understand that's when the (imho) function pointer >> systrace_probe_func gets invoked. Therefore, it seems that kgdb has a hard >> time finding the source file for the function invoked through that pointer. >> Is this complete nonsense, or does that actually sound familiar to anyone? >> >> And still I'm wondering whether anybody ever successfully used the ustack >> action on Freebsd-8.2 i386 >> >> cheers >> --m >> > > From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 19:59:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DD3B1065674 for ; Tue, 2 Aug 2011 19:59:50 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep21.mx.upcmail.net (fep21.mx.upcmail.net [62.179.121.41]) by mx1.freebsd.org (Postfix) with ESMTP id 0DDBF8FC0C for ; Tue, 2 Aug 2011 19:59:48 +0000 (UTC) Received: from edge02.upcmail.net ([192.168.13.237]) by viefep17-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20110802195939.YVOG8134.viefep17-int.chello.at@edge02.upcmail.net> for ; Tue, 2 Aug 2011 21:59:39 +0200 Received: from pinky ([95.96.138.26]) by edge02.upcmail.net with edge id FXzd1h06T0aMTqv02Xzeva; Tue, 02 Aug 2011 21:59:39 +0200 X-SourceIP: 95.96.138.26 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-stable@freebsd.org References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> Date: Tue, 02 Aug 2011 21:59:37 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/11.50 (Win32) X-Cloudmark-Analysis: v=1.1 cv=HQ3F56nxkum+cgCiDL7AXQpbvw7DWrWCBJRnYYnM0Zc= c=1 sm=0 a=jSLzLkXI7GEA:10 a=vcqGe1LRAV8A:10 a=bgpUlknNv7MA:10 a=IkcTkHD0fZMA:10 a=pGLkceISAAAA:8 a=D9flXBp1TUvwEZZsISwA:9 a=QEXdDO2ut3YA:10 a=MSl-tDqOz04A:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Subject: zpool doesn't upgrade - Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 19:59:50 -0000 On Tue, 02 Aug 2011 12:55:43 +0200, seanrees@gmail.com wrote: > On Tue, Aug 2, 2011 at 11:10 AM, Daniel Kalchev wrote: >>> If it is a limitation in ZFS it would be nice to know that, perhaps it >>> truly, really is a bug that can be avoided (or it's inherent in the >>> way ZFS >>> handles such things) >> >> It is possible that there is not enough memory in ARC to cache that >> large >> directory. >> >> Other than that, perhaps in ZFS it would be easier to prune the unused >> directory entries, than it is in UFS. It looks like this is not >> implemented. >> >> Another reason might be some FreeBSD specific implementation issue for >> fstatfs. >> >> In any case, the data available is not sufficient. More information >> would >> help, like how much RAM this system has, how much ARC uses, some ARC >> stats. > > Which sysctl's would you like? > > I grabbed these to start: > kstat.zfs.misc.arcstats.size: 118859656 > kstat.zfs.misc.arcstats.hdr_size: 3764416 > kstat.zfs.misc.arcstats.data_size: 53514240 > kstat.zfs.misc.arcstats.other_size: 61581000 > > kstat.zfs.misc.arcstats.hits: 46762467 > kstat.zfs.misc.arcstats.misses: 16999907 > > The machine has 2GB of memory. > >> What made me wonder is .. how exactly the kernel and zpool disagree on >> zpool >> version? What is the pool version in fact? > > % dmesg | grep ZFS > ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is > present; > to enable, add "vfs.zfs.prefetch_disable=0" to > /boot/loader.conf. > ZFS filesystem version 5 > ZFS storage pool version 28 > > % zpool get version tank > NAME PROPERTY VALUE SOURCE > tank version 15 local > > % zpool upgrade tank > This system is currently running ZFS pool version 15. > > Pool 'tank' is already formatted using the current version. > > > Sean I think this zpool upgrade thing is weird. Can you try 'zpool upgrade -a'? Mine says: zpool get version zroot NAME PROPERTY VALUE SOURCE zroot version 28 default Mind the SOURCE=default vs. SOURCE=local. Is it possible you did 'zpool set version=15 tank' in the past? You can check that with 'zpool history'. NB: if you upgrade the boot pool, don't forget to upgrade to boot loader. (See UPDATING) Ronald. From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 20:54:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE5C7106566B for ; Tue, 2 Aug 2011 20:54:36 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 223FD8FC12 for ; Tue, 2 Aug 2011 20:54:36 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p72KsX5J095548 for ; Tue, 2 Aug 2011 16:54:33 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4E38640E.9020005@sentex.net> Date: Tue, 02 Aug 2011 16:54:38 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: FreeBSD-STABLE Mailing List X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Subject: ATA_IDENTIFY requeued due to channel reset LBA=0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 20:54:36 -0000 I upgraded a RELENG_8 box from a kernel from ~ June 15th to one today to get some of the zfs and bind updates and on reboot, the box panic'd twice, and booted fine the third time. I have not seen this error before and not sure if its a hardware issue, or some odd timing issue I ran into ? smartctl shows no errors on any of the disks recorded in their logs >From the serial console, this is what I saw atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x3410-0x341f,0x3400-0x340f irq 21 at device 31.2 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 31.3 (no driver attached) atapci1: port 0x3428-0x342f,0x3444-0x3447,0x3420-0x3427,0x3440-0x3443,0x30f0-0x30ff,0x30e0-0x30ef irq 21 at device 31.5 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 . . . ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered ad0: 1430799MB at ata0-master UDMA100 SATA 3Gb/s uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 unknown: WARNING - ATA_IDENTIFY taskqueue timeout - completing request directly unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 ad4: 1430799MB at ata2-master UDMA100 SATA 3Gb/s subdisk4: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x308 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80419eae stack pointer = 0x28:0xffffff800009dab0 frame pointer = 0x28:0xffffff800009dad0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (thread taskq) trap number = 12 ad6: 1430799MB at ata3-master UDMA100 SATA 3Gb/s panic: page fault cpuid = 0 Uptime: 14s Cannot dump. Device not defined or unavailable. The normal boot looks like uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered ad0: 1430799MB at ata0-master UDMA100 SATA 3Gb/s ad2: 76319MB at ata1-master UDMA100 SATA 1.5Gb/s ad3: 1907729MB at ata1-slave UDMA100 SATA 3Gb/s GEOM: ad2s1: geometry does not match label (255h,63s != 16h,63s). ad4: 1430799MB at ata2-master UDMA100 SATA 3Gb/s ad6: 1430799MB at ata3-master UDMA100 SATA 3Gb/s uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered pmp0 at siisch0 bus 0 scbus0 target 15 lun 0 pmp0: ATA-0 device pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes) pmp0: 5 fan-out ports pmp1 at siisch1 bus 0 scbus1 target 15 lun 0 pmp1: ATA-0 device pmp1: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes) pmp1: 5 fan-out ports ada0 at siisch0 bus 0 scbus0 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada1 at siisch0 bus 0 scbus0 target 1 lun 0 ada1: ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada2 at siisch0 bus 0 scbus0 target 2 lun 0 ada2: ATA-8 SATA 2.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada3 at siisch0 bus 0 scbus0 target 3 lun 0 ada3: ATA-8 SATA 2.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada4 at siisch1 bus 0 scbus1 target 0 lun 0 ada4: ATA-8 SATA 2.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5 at siisch1 bus 0 scbus1 target 1 lun 0 ada5: ATA-8 SATA 2.x device ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: Command Queueing enabled ada5: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) ada6 at siisch1 bus 0 scbus1 target 2 lun 0 ada6: ATA-8 SATA 2.x device ada6: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada6: Command Queueing enabled ada6: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) ada7 at siisch1 bus 0 scbus1 target 3 lun 0 ada7: ATA-8 SATA 2.x device ada7: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada7: Command Queueing enabled ada7: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) ada8 at siisch1 bus 0 scbus1 target 4 lun 0 ada8: ATA-8 SATA 2.x device ada8: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada8: Command Queueing enabled ada8: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) lapic1: Forcing LINT1 to edge trigger SMP: AP CPU #1 Launched! e.g. smartctl -a /dev/ad3 smartctl 5.39.1 2010-01-28 r3054 [FreeBSD 8.2-STABLE amd64] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD2001FASS-00U0B0 Serial Number: WD-WMAUR0328005 Firmware Version: 01.00101 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Aug 2 16:52:52 2011 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (30900) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3037) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 047 047 021 Pre-fail Always - 14650 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 21 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 089 089 000 Old_age Always - 8630 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 19 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 18 193 Load_Cycle_Count 0x0032 134 134 000 Old_age Always - 200377 194 Temperature_Celsius 0x0022 121 113 000 Old_age Always - 31 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 4992 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. smartctl -a /dev/ad2 smartctl 5.39.1 2010-01-28 r3054 [FreeBSD 8.2-STABLE amd64] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.9 family Device Model: ST380811AS Serial Number: 6PS01Q17 Firmware Version: 3.AAE User Capacity: 80,026,361,856 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 7 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Aug 2 16:52:59 2011 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 430) seconds. Offline data collection capabilities: (0x5b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 27) minutes. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 119 095 006 Pre-fail Always - 0 3 Spin_Up_Time 0x0003 095 095 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 63 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 084 060 030 Pre-fail Always - 248537285 9 Power_On_Hours 0x0032 095 095 000 Old_age Always - 5104 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 89 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 069 045 045 Old_age Always In_the_past 31 (Lifetime Min/Max 27/32) 194 Temperature_Celsius 0x0022 031 055 000 Old_age Always - 31 (0 20 0 0) 195 Hardware_ECC_Recovered 0x001a 057 048 000 Old_age Always - 24265782 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0000 100 253 000 Old_age Offline - 0 202 Data_Address_Mark_Errs 0x0032 100 253 000 Old_age Always - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 21:03:56 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D4DA106566C for ; Tue, 2 Aug 2011 21:03:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AB86E8FC16 for ; Tue, 2 Aug 2011 21:03:55 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA27488; Wed, 03 Aug 2011 00:03:52 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoM7z-000I1B-Q6; Wed, 03 Aug 2011 00:03:51 +0300 Message-ID: <4E386636.2000507@FreeBSD.org> Date: Wed, 03 Aug 2011 00:03:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 21:03:56 -0000 on 02/08/2011 20:46 maestro something said the following: > Hi, > > just finished installing FreeBSD-9BETA1 and recompiling the kernel with dtrace. > > This is even worse, I have the same behavior as mentioned here: > http://freebsd.1045724.n5.nabble.com/bin-158431-dtrace-crash-in-dt-proc-lookup-when-attaching-to-PID-assert-dpr-NULL-tt4535367.html#none Kind of a mentoring note: it would be much shorter and much more useful to paste "PR 158431" or, even better, an http URL to the said PR in FreeBSD PR DB web interface. I know, I know, it's really Google to blame, right? :-) > i.e., dtrace regardless of whether with or without any probes just quits with > the following error message > > Assertion failed: (dpr != NULL), file > /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, > line 751. > > that's inside dt_proc_lookup() > > I think I have to give up on ustack(), too bad Or maybe you still have a chance to help us here, maybe it will even help you in the long term. It would be good if you pasted a little bit more of the output that you got. Please note that in that PR the (meaningful) output starts with: dtrace: no probes specified I.e. it starts with what dtrace considers to be a fatal condition. I tried to run dtruss (as you did) and I got this pre-amble before the assertion: [some dtrace script body] : probe description proc:::exit does not match any probes I guess that in my case I got it because my userland was not compiled with CTF support. Not sure about yours... Of course, it's still rather bad that dtrace crashes when it prematurely exits. But maybe it doesn't crash in the correct environment... I don't know. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 21:08:23 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B8CE106566B for ; Tue, 2 Aug 2011 21:08:23 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 49FDC8FC18 for ; Tue, 2 Aug 2011 21:08:21 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA27572; Wed, 03 Aug 2011 00:08:18 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoMCI-000I1O-BT; Wed, 03 Aug 2011 00:08:18 +0300 Message-ID: <4E386741.5030801@FreeBSD.org> Date: Wed, 03 Aug 2011 00:08:17 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> In-Reply-To: <4E386636.2000507@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 21:08:23 -0000 on 03/08/2011 00:03 Andriy Gapon said the following: > I tried to run dtruss (as you did) and I got this pre-amble before the assertion: > [some dtrace script body] > : probe description proc:::exit does not match any probes > > I guess that in my case I got it because my userland was not compiled with CTF > support. Not sure about yours... > > Of course, it's still rather bad that dtrace crashes when it prematurely exits. > But maybe it doesn't crash in the correct environment... I don't know. OK, here's a patch that should fix the abort via assertion - with this you shouldn't get the crash that you've reported. I hope that you will be able to tackle other conditions that dtrace considers to be errors. diff --git a/lib/libproc/proc_create.c b/lib/libproc/proc_create.c index c372a47..9bd24a2 100644 --- a/lib/libproc/proc_create.c +++ b/lib/libproc/proc_create.c @@ -79,12 +79,11 @@ proc_attach(pid_t pid, int flags, struct proc_handle **pphdl) else phdl->status = PS_STOP; +out: if (error) proc_free(phdl); else *pphdl = phdl; -out: - proc_free(phdl); return (error); } -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 21:12:04 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A97A2106566C; Tue, 2 Aug 2011 21:12:04 +0000 (UTC) (envelope-from miwi.freebsd@googlemail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 51C608FC08; Tue, 2 Aug 2011 21:12:03 +0000 (UTC) Received: by gyf3 with SMTP id 3so159176gyf.13 for ; Tue, 02 Aug 2011 14:12:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=fFbtUaN5/B6lqaJlkTPJzk0gmHcGiYzmlZE4dsV0VaU=; b=Ngu55uEWZPSNjqaobgBlwliD9a65Is6Q5So3V8A5EkAdeZNYl/fhGvcu2Sj+f1g7Q/ QGbGGNkbp2RBGpkvXZXp+DVAXspASWmeKVf7KZY9EM7ycC/6ejoLTFVOx61LNkwbsD0k mN4/fwBtQ5UJfciVzFAZ+NBB3qGiidIwspE3w= Received: by 10.142.47.2 with SMTP id u2mr3859896wfu.80.1312319523012; Tue, 02 Aug 2011 14:12:03 -0700 (PDT) Received: from [192.168.0.103] ([175.142.228.27]) by mx.google.com with ESMTPS id i9sm208700pbk.84.2011.08.02.14.11.59 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 02 Aug 2011 14:12:01 -0700 (PDT) Sender: Martin Wilke Mime-Version: 1.0 (Apple Message framework v1247) Content-Type: text/plain; charset=koi8-r From: Martin Wilke In-Reply-To: <201108021256.53224.jhb@freebsd.org> Date: Wed, 3 Aug 2011 05:11:58 +0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> <20110802105809.GT2369@laa.zp.ua> <201108021256.53224.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1247) Cc: stable@freebsd.org, freebsd-current@freebsd.org, Lystopad Olexandr , current@freebsd.org Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 21:12:04 -0000 On Aug 3, 2011, at 12:56 AM, John Baldwin wrote: > On Tuesday, August 02, 2011 8:58:10 am Martin Wilke wrote: >>=20 >> On Aug 2, 2011, at 6:58 PM, Lystopad Olexandr wrote: >>=20 >>> Hello, Martin Wilke! >>>=20 >>> On Tue, Aug 02, 2011 at 12:41:29PM +0800 >>> miwi@FreeBSD.org wrote about "9.0 B1 Panic": >>>> 9.0 Beta1 Panic >>>>=20 >>>> Hi guys, >>>>=20 >>>> I just downloaded and install 9.0 BETA1 but it panics on ACPI. = Please=20 > view attached screenshot for the error. If you need more information, = do let=20 > us know. >>>=20 >>> There no attachments in your mail. >>=20 >>=20 >> Erms Sorry forgot about that. >>=20 >> http://people.freebsd.org/~miwi/90b1.png >=20 > Can you get a stack trace? Hi, unfortunately no, because the system freeze few sec after the panic. - Martin >=20 > --=20 > John Baldwin From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 22:39:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 535C3106566B; Tue, 2 Aug 2011 22:39:07 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id E9D228FC0C; Tue, 2 Aug 2011 22:39:06 +0000 (UTC) Received: by qyk38 with SMTP id 38so266907qyk.13 for ; Tue, 02 Aug 2011 15:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LE+MEiwvNQCDwQnVdLbNLxENhJFFNSY6FmmYCeFFHc0=; b=OwDseuLBCJFJfkyF8Az89bxCYaH2PHoQzahzNwjOLK1yU5u2m0exeZpzmeY76puwzr 5IjhbsZ3eo/pGGOF4ymOgpWLMsAX66nR8zATXzbGm2NwCKqZx8Rc4gvaPtti7jZEEflq hZG/Tu2pucdvXarPz+rvdYsyJK3kj906A706M= MIME-Version: 1.0 Received: by 10.229.66.200 with SMTP id o8mr4618166qci.250.1312324746188; Tue, 02 Aug 2011 15:39:06 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Tue, 2 Aug 2011 15:39:06 -0700 (PDT) In-Reply-To: <4E386636.2000507@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> Date: Tue, 2 Aug 2011 15:39:06 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Kostik Belousov , freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 22:39:07 -0000 Hi, > > http://freebsd.1045724.n5.nabble.com/bin-158431-dtrace-crash-in-dt-proc-lookup-when-attaching-to-PID-assert-dpr-NULL-tt4535367.html#none > > Kind of a mentoring note: it would be much shorter and much more useful to > paste > "PR 158431" or, even better, an http URL to the said PR in FreeBSD PR DB > web > interface. I know, I know, it's really Google to blame, right? :-) > True, google was showing this thread as a result to my search query. > > > i.e., dtrace regardless of whether with or without any probes just quits > with > > the following error message > > > > Assertion failed: (dpr != NULL), file > > > /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, > > line 751. > > > > that's inside dt_proc_lookup() > > > > I think I have to give up on ustack(), too bad > > Or maybe you still have a chance to help us here, maybe it will even help > you in > the long term. > It would be good if you pasted a little bit more of the output that you > got. > Please note that in that PR the (meaningful) output starts with: > dtrace: no probes specified > I.e. it starts with what dtrace considers to be a fatal condition. > > I tried to run dtruss (as you did) and I got this pre-amble before the > assertion: > [some dtrace script body] > : probe description proc:::exit does not match any probes > ok this is the command line and all the output i get from dtrace: fb90i386# dtrace -n 'syscall::accept:return / execname == "nc" / { @sc[execname,probefunc] = count(); printf("%s accept:return\n", execname); ustack();}' dtrace: description 'syscall::accept:return ' matched 1 probe CPU ID FUNCTION:NAME 0 46457 accept:return nc accept:return Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. Abort (core dumped) fb90i386# I get the same thing when using dtruss: fb90i386# dtruss -a -p 1139 Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. Abort trap (core dumped) fb90i386# > > I guess that in my case I got it because my userland was not compiled with > CTF > support. Not sure about yours... > hm userland and CTF not that I know. I only recompiled the kernel with CTF. Do I have to recompile userland (and would that be make world?) If so can i just pass the same WITH_CTF=1 option to make? > > Of course, it's still rather bad that dtrace crashes when it prematurely > exits. > But maybe it doesn't crash in the correct environment... I don't know. > as soon as I understand what you mean by userland I can recompile the necessary bits. cheers and thanks a lot for your patience --m From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 22:46:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1B4D106564A; Tue, 2 Aug 2011 22:46:34 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 503948FC0A; Tue, 2 Aug 2011 22:46:34 +0000 (UTC) Received: by qyk38 with SMTP id 38so269929qyk.13 for ; Tue, 02 Aug 2011 15:46:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zC0yPkl/iZh/fl+MT5QK5uLWgIa8Y+wnJgFEoape3GM=; b=tBa5ZITCWuiWVu9dEWz70e0GR75z+NLXdSuIF9EzdJ4zBxdrE+i520MFfJgl9Mo5Lg Y1o+AWxPozwo8KH2LiYuQ/Y4STIPBI+uDqqntndeW5tV/382MjCIKUfiblVkbKr+PbU9 B0YM9BVF7dGpzSDLJLoC1gXlF8UYzGWneYOHo= MIME-Version: 1.0 Received: by 10.229.67.213 with SMTP id s21mr866110qci.212.1312325193594; Tue, 02 Aug 2011 15:46:33 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Tue, 2 Aug 2011 15:46:33 -0700 (PDT) In-Reply-To: <4E386741.5030801@FreeBSD.org> References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> Date: Tue, 2 Aug 2011 15:46:33 -0700 Message-ID: From: maestro something To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Kostik Belousov , freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 22:46:34 -0000 Hi, diff --git a/lib/libproc/proc_create.c b/lib/libproc/proc_create.c > index c372a47..9bd24a2 100644 > --- a/lib/libproc/proc_create.c > +++ b/lib/libproc/proc_create.c > @@ -79,12 +79,11 @@ proc_attach(pid_t pid, int flags, struct proc_handle > **pphdl) > else > phdl->status = PS_STOP; > > +out: > if (error) > proc_free(phdl); > else > *pphdl = phdl; > -out: > - proc_free(phdl); > return (error); > } > What do I have to recompile for the patch to be picked up? make libproc in /usr/src/lib works but make install libproc fails the following way: fb90i386# make libproc Warning: Object directory not changed from original /usr/src/lib/libproc cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_bkpt.c cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_create.c cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_regs.c cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_sym.c cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_rtld.c cc -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_util.c building static proc library ranlib libproc.a cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_bkpt.c -o proc_bkpt.po cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_create.c -o proc_create.po cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_regs.c -o proc_regs.po cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_sym.c -o proc_sym.po cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_rtld.c -o proc_rtld.po cc -pg -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_util.c -o proc_util.po building profiled proc library ranlib libproc_p.a cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_bkpt.c -o proc_bkpt.So cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_create.c -o proc_create.So cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_regs.c -o proc_regs.So cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_sym.c -o proc_sym.So cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_rtld.c -o proc_rtld.So cc -fpic -DPIC -O2 -pipe -I/usr/src/lib/libproc -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c proc_util.c -o proc_util.So building shared library libproc.so.2 fb90i386# make install libproc ===> csu/i386-elf (install) install -o root -g wheel -m 444 crti.o crtn.o gcrt1.o crt1.o Scrt1.o /usr/lib ===> libc (install) install -C -o root -g wheel -m 444 libc.a /usr/lib install: libc.a: No such file or directory *** Error code 71 Stop in /usr/src/lib/libc. *** Error code 1 Stop in /usr/src/lib. From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 23:11:55 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B178C106564A for ; Tue, 2 Aug 2011 23:11:55 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.westchester.pa.mail.comcast.net (qmta14.westchester.pa.mail.comcast.net [76.96.59.212]) by mx1.freebsd.org (Postfix) with ESMTP id 5DF348FC08 for ; Tue, 2 Aug 2011 23:11:55 +0000 (UTC) Received: from omta05.westchester.pa.mail.comcast.net ([76.96.62.43]) by qmta14.westchester.pa.mail.comcast.net with comcast id FbBl1h0080vyq2s5EbBvUK; Tue, 02 Aug 2011 23:11:55 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta05.westchester.pa.mail.comcast.net with comcast id FbBl1h00i1t3BNj3RbBqxv; Tue, 02 Aug 2011 23:11:52 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id E2AFD102C36; Tue, 2 Aug 2011 16:11:43 -0700 (PDT) Date: Tue, 2 Aug 2011 16:11:43 -0700 From: Jeremy Chadwick To: Mike Tancsa Message-ID: <20110802231143.GA5450@icarus.home.lan> References: <4E38640E.9020005@sentex.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E38640E.9020005@sentex.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD-STABLE Mailing List Subject: Re: ATA_IDENTIFY requeued due to channel reset LBA=0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 23:11:55 -0000 On Tue, Aug 02, 2011 at 04:54:38PM -0400, Mike Tancsa wrote: > I upgraded a RELENG_8 box from a kernel from ~ June 15th to one today to > get some of the zfs and bind updates and on reboot, the box panic'd > twice, and booted fine the third time. I have not seen this error > before and not sure if its a hardware issue, or some odd timing issue I > ran into ? smartctl shows no errors on any of the disks recorded in > their logs > > >From the serial console, this is what I saw > > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x3410-0x341f,0x3400-0x340f irq 21 > at device 31.2 on pci0 > ata0: on atapci0 > ata0: [ITHREAD] > ata1: on atapci0 > ata1: [ITHREAD] > pci0: at device 31.3 (no driver attached) > atapci1: port > 0x3428-0x342f,0x3444-0x3447,0x3420-0x3427,0x3440-0x3443,0x30f0-0x30ff,0x30e0-0x30ef > irq 21 at device 31.5 on pci0 > atapci1: [ITHREAD] > ata2: on atapci1 > ata2: [ITHREAD] > ata3: on atapci1 > ata3: [ITHREAD] > acpi_hpet0: iomem 0xfed00000-0xfed003ff on > acpi0 > . > . > . > ugen1.1: at usbus1 > uhub1: on usbus1 > ugen2.1: at usbus2 > uhub2: on usbus2 > ugen3.1: at usbus3 > uhub3: on usbus3 > ugen4.1: at usbus4 > uhub4: on usbus4 > ugen5.1: at usbus5 > uhub5: on usbus5 > ugen6.1: at usbus6 > uhub6: on usbus6 > ugen7.1: at usbus7 > uhub7: on usbus7 > uhub0: 2 ports with 2 removable, self powered > uhub1: 2 ports with 2 removable, self powered > uhub2: 2 ports with 2 removable, self powered > uhub4: 2 ports with 2 removable, self powered > uhub5: 2 ports with 2 removable, self powered > uhub6: 2 ports with 2 removable, self powered > ad0: 1430799MB at ata0-master UDMA100 SATA 3Gb/s > uhub3: 6 ports with 6 removable, self powered > uhub7: 6 ports with 6 removable, self powered > unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 > ... > unknown: WARNING - ATA_IDENTIFY taskqueue timeout - completing request directly > unknown: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 > ad4: 1430799MB at ata2-master UDMA100 SATA 3Gb/s > subdisk4: WARNING - ATA_IDENTIFY requeued due to channel reset LBA=0 This looks like a failure on ata1, both -master and -slave. I can tell based on later output: > ad0: 1430799MB at ata0-master UDMA100 SATA 3Gb/s > ad2: 76319MB at ata1-master UDMA100 SATA 1.5Gb/s > ad3: 1907729MB at ata1-slave UDMA100 SATA 3Gb/s > GEOM: ad2s1: geometry does not match label (255h,63s != 16h,63s). > ad4: 1430799MB at ata2-master UDMA100 SATA 3Gb/s > smartctl -a /dev/ad3 > ... > smartctl -a /dev/ad2 > .... These disks look in fine shape. The only thing I could think might have happened is that either the ata1 channel went wonky, or one of the two disks (ad2 or ad3) has an intermittent problem within its HPA area (which TMK wouldn't show up in SMART but would manifest itself as an issue when ATA_IDENTIFY/0xec would be called since some of the data that CDB returns comes from the HPA), or there's a strange quirk/bug in the older ata(4) code. Does your motherboard support AHCI? I see it's ICH9 and you're running it in Compatible mode (which makes SATA devices appear as classic PATA). Alexander (mav@) should act as a more authoritative source for this. However, I can tell you up front that booting verbose is your best choice of option here, as it gives significant controller status/state debug information which is useful when troubleshooting things like this. So unless the problem is reproducible, I'm not sure if you're willing to constantly boot verbose or not. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 00:12:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFEA7106566C for ; Wed, 3 Aug 2011 00:12:42 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 534748FC1A for ; Wed, 3 Aug 2011 00:12:41 +0000 (UTC) Received: by gxk28 with SMTP id 28so244413gxk.13 for ; Tue, 02 Aug 2011 17:12:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Nq5CTp4Ff6TeGf5wgVKgIeS1n13b81kkKqV7eZw7BQ8=; b=rUB6APEi4F84cH3/BHaWJeY6Nkm/1XmbJw+z72vLb0na9nmp0H6aiZ5pVHI8/rbMk3 4rkf+IWdKHA5CkPiBwJYH2BXKACevkfBGZGJw7b6vS0MJfVxLqU5aU5Ui1tzRGjBLv8q u7HTwFGjVEkEKx0i+QCPabonPJv1+lz4FC0Kk= MIME-Version: 1.0 Received: by 10.151.39.11 with SMTP id r11mr2277347ybj.411.1312330361471; Tue, 02 Aug 2011 17:12:41 -0700 (PDT) Received: by 10.150.97.3 with HTTP; Tue, 2 Aug 2011 17:12:41 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> Date: Tue, 2 Aug 2011 17:12:41 -0700 Message-ID: From: Kevin Oberman To: maestro something Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 00:12:42 -0000 On Tue, Aug 2, 2011 at 3:46 PM, maestro something wro= te: > Hi, > > diff --git a/lib/libproc/proc_create.c b/lib/libproc/proc_create.c >> index c372a47..9bd24a2 100644 >> --- a/lib/libproc/proc_create.c >> +++ b/lib/libproc/proc_create.c >> @@ -79,12 +79,11 @@ proc_attach(pid_t pid, int flags, struct proc_handle >> **pphdl) >> =A0 =A0 =A0 =A0else >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0phdl->status =3D PS_STOP; >> >> +out: >> =A0 =A0 =A0 =A0if (error) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0proc_free(phdl); >> =A0 =A0 =A0 =A0else >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*pphdl =3D phdl; >> -out: >> - =A0 =A0 =A0 proc_free(phdl); >> =A0 =A0 =A0 =A0return (error); >> =A0} >> > > What do I have to recompile for the patch to be picked up? > make libproc in /usr/src/lib > works but > make install libproc > fails the following way: > > fb90i386# make install libproc > =3D=3D=3D> csu/i386-elf (install) > install -o root -g wheel =A0-m 444 crti.o crtn.o gcrt1.o crt1.o Scrt1.o > /usr/lib > =3D=3D=3D> libc (install) > install -C -o root -g wheel -m 444 =A0 libc.a /usr/lib > install: libc.a: No such file or directory > *** Error code 71 > > Stop in /usr/src/lib/libc. > *** Error code 1 > > Stop in /usr/src/lib. Did you remember to 'make obj'? If you didn't, the library would be left in the wrong place where 'make install' would not find it. make obj make depend (possibly a no-op) make make install --=20 R. Kevin Oberman, Network Engineer - Retired E-mail: kob6558@gmail.com From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 00:47:10 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D126D106564A; Wed, 3 Aug 2011 00:47:10 +0000 (UTC) (envelope-from maestro82@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 64AAD8FC12; Wed, 3 Aug 2011 00:47:10 +0000 (UTC) Received: by qwc9 with SMTP id 9so307105qwc.13 for ; Tue, 02 Aug 2011 17:47:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+xImsJtY6dIOOe67Z0kC0MfBMjhar9H0xvIkZXluSes=; b=YDjRSEMF9Opb2nr6qClGoI+sdpGsAcSAk44B3jAsMHNVuNWZlXt159qSSmiNxsyosu KmIlFH4Fm0qPe5i785b6kId8FGnKegXvLd+QWRTXbjiNj64bxWDy1eWWJyevTyMATGWK WBgpqEXOVsKG1KDUejdh2jOb+4VwJqUirFuS8= MIME-Version: 1.0 Received: by 10.229.2.68 with SMTP id 4mr2277776qci.174.1312332429327; Tue, 02 Aug 2011 17:47:09 -0700 (PDT) Received: by 10.229.69.218 with HTTP; Tue, 2 Aug 2011 17:47:09 -0700 (PDT) In-Reply-To: References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> Date: Tue, 2 Aug 2011 17:47:09 -0700 Message-ID: From: maestro something To: Kevin Oberman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Kostik Belousov , freebsd-stable@freebsd.org, Andriy Gapon Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 00:47:10 -0000 Hi, > Hi, > > > > diff --git a/lib/libproc/proc_create.c b/lib/libproc/proc_create.c > >> index c372a47..9bd24a2 100644 > >> --- a/lib/libproc/proc_create.c > >> +++ b/lib/libproc/proc_create.c > >> @@ -79,12 +79,11 @@ proc_attach(pid_t pid, int flags, struct proc_handle > >> **pphdl) > >> else > >> phdl->status = PS_STOP; > >> > >> +out: > >> if (error) > >> proc_free(phdl); > >> else > >> *pphdl = phdl; > >> -out: > >> - proc_free(phdl); > >> return (error); > >> } > >> > > > > What do I have to recompile for the patch to be picked up? > > make libproc in /usr/src/lib > > works but > > make install libproc > > fails the following way: > > > > > fb90i386# make install libproc > > ===> csu/i386-elf (install) > > install -o root -g wheel -m 444 crti.o crtn.o gcrt1.o crt1.o Scrt1.o > > /usr/lib > > ===> libc (install) > > install -C -o root -g wheel -m 444 libc.a /usr/lib > > install: libc.a: No such file or directory > > *** Error code 71 > > > > Stop in /usr/src/lib/libc. > > *** Error code 1 > > > > Stop in /usr/src/lib. > > Did you remember to 'make obj'? If you didn't, the library would be > left in the wrong place where 'make install' would not find it. > > make obj > make depend (possibly a no-op) > make > make install > Thank you very much that helped. now it seems that the ustack() action on i386 and FreeBSD-9.0BETA1 almost works. Almost because dtrace still exits (this time without an error message) The process for which the ustack should be genereated is terminated Here are the specifics: fb90i386# dtrace -n 'syscall::accept:return / execname == "nc" / { ustack();}' dtrace: description 'syscall::accept:return ' matched 1 probe CPU ID FUNCTION:NAME 0 46457 accept:return libc.so.7`__sys_accept+0x7 0x3 0xed96e824 This alsmost looks like a stack trace I'd say However, dtrace quits once I connect to nc nc quits with the following error message fb90i386# nc -vl 4444 nc: Polling Error: Interrupted system call cheers -m From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 06:52:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C482B106566C for ; Wed, 3 Aug 2011 06:52:03 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail18.syd.optusnet.com.au (mail18.syd.optusnet.com.au [211.29.132.199]) by mx1.freebsd.org (Postfix) with ESMTP id 55A288FC1B for ; Wed, 3 Aug 2011 06:52:02 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail18.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p736q0b4004762 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Aug 2011 16:52:01 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id p736pxb8081932; Wed, 3 Aug 2011 16:51:59 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id p736pxP2081931; Wed, 3 Aug 2011 16:51:59 +1000 (EST) (envelope-from peter) Date: Wed, 3 Aug 2011 16:51:59 +1000 From: Peter Jeremy To: "seanrees@gmail.com" Message-ID: <20110803065159.GC78870@server.vk2pj.dyndns.org> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GPJrCs/72TxItFYR" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 06:52:03 -0000 --GPJrCs/72TxItFYR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2011-Aug-02 08:39:03 +0100, "seanrees@gmail.com" wr= ote: >On my FreeBSD 8.2-S machine (built circa 12th June), I created a >directory and populated it over the course of 3 weeks with about 2 >million individual files. As you might imagine, a 'ls' of this >directory took quite some time. > >The files were conveniently named with a timestamp in the filename >(still images from a security camera, once per second) so I've since >moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I >found though was the original directory the images were in is still >very slow to ls -- and it only has 1 file in it, another directory. I've also seen this behaviour on Solaris 10 after cleaning out a directory with a large number of files (though not as pathological as your case). I tried creating and deleting entries in an unsuccessful effort to trigger directory compaction. I wound up moving the remaining contents into a new directory, deleting the original one and renaming the new directory. It would appear te be a garbage collection bug in ZFS. On 2011-Aug-02 13:10:27 +0300, Daniel Kalchev wrote: >On 02.08.11 12:46, Daniel O'Connor wrote: >> I am pretty sure UFS does not have this problem. i.e. once you=20 >> delete/move the files out of the directory its performance would be=20 >> good again.=20 > >UFS would be the classic example of poor performance if you do this. Traditional UFS (including Solaris) behave badly in this scenario but 4.4BSD derivatives will release unused space at the end of a directory and have smarts to more efficiently skip unused entries at the start of a directory. --=20 Peter Jeremy --GPJrCs/72TxItFYR Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iEYEARECAAYFAk448A8ACgkQ/opHv/APuIfmjQCgg4ijcCrG0q7oX4cLwKxDd9io TWcAoLDGoEoEkURljItE768LbQddMILj =ZKso -----END PGP SIGNATURE----- --GPJrCs/72TxItFYR-- From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 08:20:33 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44DB81065672 for ; Wed, 3 Aug 2011 08:20:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 871BE8FC08 for ; Wed, 3 Aug 2011 08:20:32 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA06689; Wed, 03 Aug 2011 11:20:29 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoWgm-000Kkg-P1; Wed, 03 Aug 2011 11:20:28 +0300 Message-ID: <4E3904CA.8010206@FreeBSD.org> Date: Wed, 03 Aug 2011 11:20:26 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 08:20:33 -0000 on 03/08/2011 01:39 maestro something said the following: > hm userland and CTF not that I know. I only recompiled the kernel with CTF. Do I > have to recompile userland (and would that be make world?) If so can i just pass > the same WITH_CTF=1 option to make? I _think_ (as in: not sure) that for ustack() to work correctly it needs CTF information for userland bits. I've never done this myself, so be warned, but I guess that, yes, you need to recompile/reinstall world with WITH_CTF=1. Also, I seem to recall that there were some reports that some things were getting broken because of WITH_CTF=1, so be cautious, plan a way to recover your system from non-working world. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 08:22:00 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15EF4106566B for ; Wed, 3 Aug 2011 08:22:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 55D878FC08 for ; Wed, 3 Aug 2011 08:21:59 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA06706; Wed, 03 Aug 2011 11:21:56 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoWiB-000Kkj-V3; Wed, 03 Aug 2011 11:21:55 +0300 Message-ID: <4E390523.8070809@FreeBSD.org> Date: Wed, 03 Aug 2011 11:21:55 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 08:22:00 -0000 on 03/08/2011 01:46 maestro something said the following: > What do I have to recompile for the patch to be picked up? > make libproc in /usr/src/lib Correct way is: $ cd /usr/src/lib/libproc $ make obj $ make depend $ make $ make install -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 08:42:25 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 769C11065670; Wed, 3 Aug 2011 08:42:25 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A57F08FC12; Wed, 3 Aug 2011 08:42:24 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA06995; Wed, 03 Aug 2011 11:42:22 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoX1y-000Klx-1N; Wed, 03 Aug 2011 11:42:22 +0300 Message-ID: <4E3909ED.1000807@FreeBSD.org> Date: Wed, 03 Aug 2011 11:42:21 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E2E9F60.1060808@FreeBSD.org> <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 08:42:25 -0000 on 03/08/2011 03:47 maestro something said the following: > now it seems that the ustack() action on i386 and FreeBSD-9.0BETA1 almost works. > > Almost because dtrace still exits (this time without an error message) The > process for which the ustack should be genereated is terminated > Here are the specifics: > > fb90i386# dtrace -n 'syscall::accept:return / execname == "nc" / { ustack();}' > dtrace: description 'syscall::accept:return ' matched 1 probe > CPU ID FUNCTION:NAME > 0 46457 accept:return > libc.so.7`__sys_accept+0x7 > 0x3 > 0xed96e824 > > > This alsmost looks like a stack trace I'd say > However, dtrace quits once I connect to nc > > nc quits with the following error message > > fb90i386# nc -vl 4444 > nc: Polling Error: Interrupted system call This could be a combination of several factors. First, for some reason nc gets EINTR in one of its system calls (poll(2)). Second, nc treats EINTR as a serious error and exits. Third, this is what happens on the dtrace side: 30495 100974 dtrace CALL write(0x1,0x8035c2000,0x1a) 30495 100974 dtrace GIO fd 1 wrote 26 bytes "libc.so.7`__sys_accept+0xc" 30495 100974 dtrace RET write 26/0x1a 30495 100974 dtrace CALL write(0x1,0x8035c2000,0x1) 30495 100974 dtrace GIO fd 1 wrote 1 byte " " 30495 100974 dtrace RET write 1 30495 100974 dtrace CALL thr_kill(0x1c129,SIGUSR1) 30495 100974 dtrace RET thr_kill 0 30495 100974 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffffbb10,0) 30495 100974 dtrace RET ptrace 0 30495 100974 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffffbb10,0) 30495 114985 dtrace RET _umtx_op -1 errno 4 Interrupted system call 30495 100974 dtrace RET ptrace 0 30495 114985 dtrace PSIG SIGUSR1 caught handler=0x80085d610 mask=0x5ffefedf code=0x10007 30495 100974 dtrace CALL _umtx_op(0x802007668,0x15,0x1,0,0) 30495 114985 dtrace CALL sigprocmask(SIG_SETMASK,0x7fffffbfd9fc,0) 30495 100974 dtrace RET _umtx_op 0 30495 114985 dtrace RET sigprocmask 0 30495 100974 dtrace CALL _umtx_op(0x800a71778,0xf,0,0,0) 30495 114985 dtrace CALL sigreturn(0x7fffffbfd630) 30495 114985 dtrace RET sigreturn JUSTRETURN 30495 114985 dtrace CALL ptrace(PT_CONTINUE,0x771e,0x1,0) 30495 114985 dtrace RET ptrace 0 30495 114985 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffbfdee0,0) 30495 114985 dtrace RET ptrace -1 errno 16 Device busy 30495 114985 dtrace CALL _umtx_op(0x80200d668,0x15,0x1,0,0) 30495 114985 dtrace RET _umtx_op 0 30495 114985 dtrace CALL madvise(0x803c52000,0x11000,MADV_FREE) 30495 114985 dtrace RET madvise 0 30495 114985 dtrace CALL madvise(0x803c1d000,0x14000,MADV_FREE) 30495 114985 dtrace RET madvise 0 30495 114985 dtrace CALL madvise(0x803c07000,0x1000,MADV_FREE) 30495 114985 dtrace RET madvise 0 30495 114985 dtrace CALL madvise(0x8037fc000,0x2000,MADV_FREE) 30495 114985 dtrace RET madvise 0 30495 114985 dtrace CALL madvise(0x8037f2000,0x8000,MADV_FREE) 30495 114985 dtrace RET madvise 0 30495 114985 dtrace CALL madvise(0x8037f0000,0x1000,MADV_FREE) 30495 100974 dtrace RET _umtx_op 0 30495 114985 dtrace RET madvise 0 30495 100974 dtrace CALL ptrace(PT_DETACH,0x771e,0,0) 30495 114985 dtrace CALL madvise(0x8037e0000,0x9000,MADV_FREE) 30495 100974 dtrace RET ptrace -1 errno 3 No such process Looks like something fishy happens when that SIGUSR1 is generated. I wish a dtrace-for-userland expert and/or a ptrace(2) expert could help us here. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 09:32:48 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FACA106566B; Wed, 3 Aug 2011 09:32:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1F5B08FC08; Wed, 3 Aug 2011 09:32:46 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA08118; Wed, 03 Aug 2011 12:32:44 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QoXoi-000KoD-5P; Wed, 03 Aug 2011 12:32:44 +0300 Message-ID: <4E3915BA.2010801@FreeBSD.org> Date: Wed, 03 Aug 2011 12:32:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: maestro something References: <4E33B7CF.90200@FreeBSD.org> <4E344D15.1040508@FreeBSD.org> <20110730192646.GC17489@deviant.kiev.zoral.com.ua> <4E386636.2000507@FreeBSD.org> <4E386741.5030801@FreeBSD.org> <4E3909ED.1000807@FreeBSD.org> In-Reply-To: <4E3909ED.1000807@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org Subject: Re: dtrace ustack kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 09:32:48 -0000 on 03/08/2011 11:42 Andriy Gapon said the following: > on 03/08/2011 03:47 maestro something said the following: >> now it seems that the ustack() action on i386 and FreeBSD-9.0BETA1 almost works. >> >> Almost because dtrace still exits (this time without an error message) The >> process for which the ustack should be genereated is terminated >> Here are the specifics: >> >> fb90i386# dtrace -n 'syscall::accept:return / execname == "nc" / { ustack();}' >> dtrace: description 'syscall::accept:return ' matched 1 probe >> CPU ID FUNCTION:NAME >> 0 46457 accept:return >> libc.so.7`__sys_accept+0x7 >> 0x3 >> 0xed96e824 >> >> >> This alsmost looks like a stack trace I'd say >> However, dtrace quits once I connect to nc >> >> nc quits with the following error message >> >> fb90i386# nc -vl 4444 >> nc: Polling Error: Interrupted system call > > This could be a combination of several factors. > First, for some reason nc gets EINTR in one of its system calls (poll(2)). > Second, nc treats EINTR as a serious error and exits. Looks like nc gets EINTR when dtrace does ptrace(PT_CONTINUE). So this is the culprit for your test case. > Third, this is what happens on the dtrace side: Recorded dtrace behavior seems to be correct. > 30495 100974 dtrace CALL write(0x1,0x8035c2000,0x1a) > 30495 100974 dtrace GIO fd 1 wrote 26 bytes > "libc.so.7`__sys_accept+0xc" > 30495 100974 dtrace RET write 26/0x1a > 30495 100974 dtrace CALL write(0x1,0x8035c2000,0x1) > 30495 100974 dtrace GIO fd 1 wrote 1 byte > " > " > 30495 100974 dtrace RET write 1 > 30495 100974 dtrace CALL thr_kill(0x1c129,SIGUSR1) > 30495 100974 dtrace RET thr_kill 0 > 30495 100974 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffffbb10,0) > 30495 100974 dtrace RET ptrace 0 > 30495 100974 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffffbb10,0) > 30495 114985 dtrace RET _umtx_op -1 errno 4 Interrupted system call > 30495 100974 dtrace RET ptrace 0 > 30495 114985 dtrace PSIG SIGUSR1 caught handler=0x80085d610 mask=0x5ffefedf > code=0x10007 > 30495 100974 dtrace CALL _umtx_op(0x802007668,0x15,0x1,0,0) > 30495 114985 dtrace CALL sigprocmask(SIG_SETMASK,0x7fffffbfd9fc,0) > 30495 100974 dtrace RET _umtx_op 0 > 30495 114985 dtrace RET sigprocmask 0 > 30495 100974 dtrace CALL _umtx_op(0x800a71778,0xf,0,0,0) > 30495 114985 dtrace CALL sigreturn(0x7fffffbfd630) > 30495 114985 dtrace RET sigreturn JUSTRETURN > 30495 114985 dtrace CALL ptrace(PT_CONTINUE,0x771e,0x1,0) > 30495 114985 dtrace RET ptrace 0 > 30495 114985 dtrace CALL ptrace(PT_IO,0x771e,0x7fffffbfdee0,0) > 30495 114985 dtrace RET ptrace -1 errno 16 Device busy > 30495 114985 dtrace CALL _umtx_op(0x80200d668,0x15,0x1,0,0) > 30495 114985 dtrace RET _umtx_op 0 > 30495 114985 dtrace CALL madvise(0x803c52000,0x11000,MADV_FREE) > 30495 114985 dtrace RET madvise 0 > 30495 114985 dtrace CALL madvise(0x803c1d000,0x14000,MADV_FREE) > 30495 114985 dtrace RET madvise 0 > 30495 114985 dtrace CALL madvise(0x803c07000,0x1000,MADV_FREE) > 30495 114985 dtrace RET madvise 0 > 30495 114985 dtrace CALL madvise(0x8037fc000,0x2000,MADV_FREE) > 30495 114985 dtrace RET madvise 0 > 30495 114985 dtrace CALL madvise(0x8037f2000,0x8000,MADV_FREE) > 30495 114985 dtrace RET madvise 0 > 30495 114985 dtrace CALL madvise(0x8037f0000,0x1000,MADV_FREE) > 30495 100974 dtrace RET _umtx_op 0 > 30495 114985 dtrace RET madvise 0 > 30495 100974 dtrace CALL ptrace(PT_DETACH,0x771e,0,0) > 30495 114985 dtrace CALL madvise(0x8037e0000,0x9000,MADV_FREE) > 30495 100974 dtrace RET ptrace -1 errno 3 No such process > > Looks like something fishy happens when that SIGUSR1 is generated. > I wish a dtrace-for-userland expert and/or a ptrace(2) expert could help us here. > -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Aug 3 14:05:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4295F1065679; Wed, 3 Aug 2011 14:05:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 15B218FC14; Wed, 3 Aug 2011 14:05:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AFDD846B43; Wed, 3 Aug 2011 10:05:34 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4E8D38A02E; Wed, 3 Aug 2011 10:05:34 -0400 (EDT) From: John Baldwin To: freebsd-stable@freebsd.org Date: Wed, 3 Aug 2011 10:05:33 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <09BCDEAE-1EF8-455A-B468-25617230C73A@FreeBSD.org> <201108021256.53224.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Message-Id: <201108031005.33413.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 03 Aug 2011 10:05:34 -0400 (EDT) Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, Lystopad Olexandr , Martin Wilke Subject: Re: 9.0 B1 Panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 14:05:35 -0000 On Tuesday, August 02, 2011 5:11:58 pm Martin Wilke wrote: > > On Aug 3, 2011, at 12:56 AM, John Baldwin wrote: > > > On Tuesday, August 02, 2011 8:58:10 am Martin Wilke wrote: > >> > >> On Aug 2, 2011, at 6:58 PM, Lystopad Olexandr wrote: > >> > >>> Hello, Martin Wilke! > >>> > >>> On Tue, Aug 02, 2011 at 12:41:29PM +0800 > >>> miwi@FreeBSD.org wrote about "9.0 B1 Panic": > >>>> 9.0 Beta1 Panic > >>>> > >>>> Hi guys, > >>>> > >>>> I just downloaded and install 9.0 BETA1 but it panics on ACPI. Please > > view attached screenshot for the error. If you need more information, do let > > us know. > >>> > >>> There no attachments in your mail. > >> > >> > >> Erms Sorry forgot about that. > >> > >> http://people.freebsd.org/~miwi/90b1.png > > > > Can you get a stack trace? > > Hi, > unfortunately no, because the system freeze few sec after the panic. Can you set the loader tunable to force an automatic trace on panic? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Fri Aug 5 15:46:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 001F1106566B for ; Fri, 5 Aug 2011 15:46:00 +0000 (UTC) (envelope-from mailnull@mips.inka.de) Received: from mail-in-06.arcor-online.net (mail-in-06.arcor-online.net [151.189.21.46]) by mx1.freebsd.org (Postfix) with ESMTP id AB21C8FC15 for ; Fri, 5 Aug 2011 15:46:00 +0000 (UTC) Received: from mail-in-18-z2.arcor-online.net (mail-in-18-z2.arcor-online.net [151.189.8.35]) by mx.arcor.de (Postfix) with ESMTP id 6983110BEAB for ; Fri, 5 Aug 2011 17:12:30 +0200 (CEST) Received: from mail-in-05.arcor-online.net (mail-in-05.arcor-online.net [151.189.21.45]) by mail-in-18-z2.arcor-online.net (Postfix) with ESMTP id 6627733A333 for ; Fri, 5 Aug 2011 17:12:30 +0200 (CEST) Received: from lorvorc.mips.inka.de (dslb-092-075-216-193.pools.arcor-ip.net [92.75.216.193]) by mail-in-05.arcor-online.net (Postfix) with ESMTPS id 20EDAE3BA1 for ; Fri, 5 Aug 2011 17:12:30 +0200 (CEST) X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-05.arcor-online.net 20EDAE3BA1 Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.14.5/8.14.3) with ESMTP id p75FCT3U020077 for ; Fri, 5 Aug 2011 17:12:29 +0200 (CEST) (envelope-from mailnull@lorvorc.mips.inka.de) Received: (from mailnull@localhost) by lorvorc.mips.inka.de (8.14.5/8.14.5/Submit) id p75FCTKS020076 for freebsd-stable@freebsd.org; Fri, 5 Aug 2011 17:12:29 +0200 (CEST) (envelope-from mailnull) From: naddy@mips.inka.de (Christian Weisgerber) Date: Fri, 5 Aug 2011 15:12:29 +0000 (UTC) Message-ID: References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> Originator: naddy@mips.inka.de (Christian Weisgerber) To: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 15:46:01 -0000 Daniel Kalchev wrote: > On 02.08.11 12:46, Daniel O'Connor wrote: > > I am pretty sure UFS does not have this problem. i.e. once you > > delete/move the files out of the directory its performance would be > > good again. > > UFS would be the classic example of poor performance if you do this. "Classic" indeed. UFS dirhash has pretty much taken care of this a decade ago. -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-stable@FreeBSD.ORG Fri Aug 5 20:01:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2241106564A for ; Fri, 5 Aug 2011 20:01:44 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) by mx1.freebsd.org (Postfix) with ESMTP id 3E7468FC08 for ; Fri, 5 Aug 2011 20:01:43 +0000 (UTC) Received: from hugo10.ka.punkt.de ([217.29.45.10]) by gate1.intern.punkt.de with ESMTP id p75JlC0Y008461; Fri, 5 Aug 2011 21:47:12 +0200 (CEST) Received: from [217.29.46.6] ([217.29.46.6]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id p75JlBOr079882; Fri, 5 Aug 2011 21:47:12 +0200 (CEST) (envelope-from hausen@punkt.de) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Patrick M. Hausen" In-Reply-To: Date: Fri, 5 Aug 2011 21:47:40 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> To: naddy@mips.inka.de (Christian Weisgerber) X-Mailer: Apple Mail (2.1084) Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 20:01:44 -0000 Hi, all, Am 05.08.2011 um 17:12 schrieb Christian Weisgerber: > Daniel Kalchev wrote: > >> On 02.08.11 12:46, Daniel O'Connor wrote: >>> I am pretty sure UFS does not have this problem. i.e. once you >>> delete/move the files out of the directory its performance would be >>> good again. >> >> UFS would be the classic example of poor performance if you do this. > > "Classic" indeed. UFS dirhash has pretty much taken care of this > a decade ago. While dirhash is quite an improvement, it is definitely no silver bullet. When I asked Kirk McKusick at last year's EuroBSDCon if having a six-figure number of files in a single directory was a clever idea (I just had a customer who ran into that situation), he just smiled and shook his head. The directory in question was the typo3temp/pics directory that TYPO3 uses to scale images that are part of the website, so they are handed to the browser in exactly the size they are supposed to be rendered. The performance impact was quite heavy, because at some point requests started to pile up, PHP scripts did not finish in time, fcgi slots stayed used ... most of you will know that scenario. At some threshold a machine goes from "loaded, maybe a bit slow, but generally responsive" to "no f*ing way". Best regards, Patrick From owner-freebsd-stable@FreeBSD.ORG Sat Aug 6 03:38:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC68A1065670 for ; Sat, 6 Aug 2011 03:38:20 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 2288B8FC0A for ; Sat, 6 Aug 2011 03:38:18 +0000 (UTC) Received: from ur.dons.net.au (ppp121-45-110-64.lns20.adl6.internode.on.net [121.45.110.64]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p763c7Pv052557 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 6 Aug 2011 13:08:14 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: Date: Sat, 6 Aug 2011 13:08:07 +0930 Content-Transfer-Encoding: quoted-printable Message-Id: <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> To: "Patrick M. Hausen" X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: 0.163 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-stable@freebsd.org, Christian Weisgerber Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Aug 2011 03:38:20 -0000 On 06/08/2011, at 5:17, Patrick M. Hausen wrote: > Am 05.08.2011 um 17:12 schrieb Christian Weisgerber: >> Daniel Kalchev wrote: >>=20 >>> On 02.08.11 12:46, Daniel O'Connor wrote: >>>> I am pretty sure UFS does not have this problem. i.e. once you=20 >>>> delete/move the files out of the directory its performance would be=20= >>>> good again.=20 >>>=20 >>> UFS would be the classic example of poor performance if you do this. >>=20 >> "Classic" indeed. UFS dirhash has pretty much taken care of this >> a decade ago. >=20 > While dirhash is quite an improvement, it is definitely no silver = bullet. >=20 > When I asked Kirk McKusick at last year's EuroBSDCon if having > a six-figure number of files in a single directory was a clever idea > (I just had a customer who ran into that situation), he just smiled > and shook his head. Ahh, but OP had moved these files away and performance was still poor.. = _that_ is the bug. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-stable@FreeBSD.ORG Sat Aug 6 03:56:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id D8C4B1065675 for ; Sat, 6 Aug 2011 03:56:36 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 12-207-105-211.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 955102001E2; Sat, 6 Aug 2011 03:56:36 +0000 (UTC) Message-ID: <4E3CBB74.9020208@FreeBSD.org> Date: Fri, 05 Aug 2011 20:56:36 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: Daniel O'Connor References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> In-Reply-To: <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> X-Enigmail-Version: 1.2pre OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Aug 2011 03:56:36 -0000 On 08/05/2011 20:38, Daniel O'Connor wrote: > Ahh, but OP had moved these files away and performance was still poor.. _that_ is the bug. I'm no file system expert, but it seems to me the key questions are; how long does it take the system to recover from this condition, and if it's more than N $periods is that a problem? We can't stop users from doing wacky stuff, but the system should be robust in the face of this. -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-stable@FreeBSD.ORG Sat Aug 6 05:52:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C552106564A for ; Sat, 6 Aug 2011 05:52:01 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id DA66F8FC0C for ; Sat, 6 Aug 2011 05:52:00 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1QpZni-00089r-Fo>; Sat, 06 Aug 2011 07:51:58 +0200 Received: from e178016095.adsl.alicedsl.de ([85.178.16.95] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1QpZni-0002mp-Cd>; Sat, 06 Aug 2011 07:51:58 +0200 Message-ID: <4E3CD67E.7090201@zedat.fu-berlin.de> Date: Sat, 06 Aug 2011 07:51:58 +0200 From: "Hartmann, O." User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110712 Thunderbird/5.0 MIME-Version: 1.0 To: "Patrick M. Hausen" References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: 85.178.16.95 Cc: freebsd-stable@freebsd.org, Christian Weisgerber Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Aug 2011 05:52:01 -0000 On 08/05/11 21:47, Patrick M. Hausen wrote: > Hi, all, > > Am 05.08.2011 um 17:12 schrieb Christian Weisgerber: >> Daniel Kalchev wrote: >> >>> On 02.08.11 12:46, Daniel O'Connor wrote: >>>> I am pretty sure UFS does not have this problem. i.e. once you >>>> delete/move the files out of the directory its performance would be >>>> good again. >>> UFS would be the classic example of poor performance if you do this. >> "Classic" indeed. UFS dirhash has pretty much taken care of this >> a decade ago. > While dirhash is quite an improvement, it is definitely no silver bullet. > > When I asked Kirk McKusick at last year's EuroBSDCon if having > a six-figure number of files in a single directory was a clever idea > (I just had a customer who ran into that situation), he just smiled > and shook his head. > > The directory in question was the typo3temp/pics directory that > TYPO3 uses to scale images that are part of the website, so they > are handed to the browser in exactly the size they are supposed > to be rendered. The performance impact was quite heavy, because > at some point requests started to pile up, PHP scripts did not finish > in time, fcgi slots stayed used ... most of you will know that scenario. > At some threshold a machine goes from "loaded, maybe a bit slow, > but generally responsive" to "no f*ing way". > > Best regards, > Patrick > I have similar situation here, but with a numerical simulation software, which drops for each timestep of integration a file of all integrated objects. Since the code is adopted and not very clever written in terms of doinf its I/O, I have to deal with this. While performing dynamical high resolution integrations of several hundreds of thousand objects over a time scale of 1 Ga produces even with a larger dump-delta creates a lot of files. making those files bigger results in a situation where they are hard to analyse, so its a tradeoff situation. ZFS and UFS2 perform bad on this situation, UFS2 even more than ZFS, but also ZFS is still a pain in the ass. Oliver From owner-freebsd-stable@FreeBSD.ORG Sat Aug 6 06:24:21 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07030106564A; Sat, 6 Aug 2011 06:24:21 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id CB79B8FC08; Sat, 6 Aug 2011 06:24:20 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QpaIx-000AOp-6x; Sat, 06 Aug 2011 02:24:15 -0400 Date: Sat, 6 Aug 2011 02:24:15 -0400 From: Gary Palmer To: Doug Barton Message-ID: <20110806062415.GB88904@in-addr.com> References: <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> <4E3CBB74.9020208@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E3CBB74.9020208@FreeBSD.org> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Aug 2011 06:24:21 -0000 On Fri, Aug 05, 2011 at 08:56:36PM -0700, Doug Barton wrote: > On 08/05/2011 20:38, Daniel O'Connor wrote: > > > Ahh, but OP had moved these files away and performance was still poor.. _that_ is the bug. > > I'm no file system expert, but it seems to me the key questions are; how > long does it take the system to recover from this condition, and if it's > more than N $periods is that a problem? We can't stop users from doing > wacky stuff, but the system should be robust in the face of this. Its been quite a while since I worked on the filesystem stuff in any detail but I believe, at least for UFS, it doesn't GC the directory, just truncate it if enough of the entries at the end are deleted to free up at least one fragment or block. If you create N files and then a directory and move the N files into the directory, the directory entry will still be N+1 records into the directory and the only way to "recover" is to recreate the directory that formerly contained the N files. It is theoretically possible to compat the directory but since the code to do that wasn't written when I last worked with UFS I suspect its non trivial. I don't know what ZFS does in this situation Gary