From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 01:06:00 2011 Return-Path: Delivered-To: current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 48F93106566B; Sat, 1 Jan 2011 01:06:00 +0000 (UTC) Date: Sat, 1 Jan 2011 01:06:00 +0000 From: Alexander Best To: Alexander Kabaev Message-ID: <20110101010600.GA26829@freebsd.org> References: <20101231174351.127b7764@kan.dnsalias.net> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20101231174351.127b7764@kan.dnsalias.net> Cc: =?iso-8859-15?Q?Ren=E9?= Ladan , current@freebsd.org Subject: Re: userland weirdness between r216351 and r216738 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 01:06:00 -0000 On Fri Dec 31 10, Alexander Kabaev wrote: > On Fri, 31 Dec 2010 22:35:05 +0100 > René Ladan wrote: > > > Hi, > > > > somewhere between 9.0-amd64 r216351 and r216738, I've noticed some > > userland weirdness. > > Symptoms are: > > - pseudo-random number generator not starting, preventing ssh(d) from > > working > > - fonts in X.org (xfce4) missing or replaced > > - mouse only working when hald is running > > > > I don't know if the above symptoms are somehow related, or what > > causes them. The kernel is GENERIC without (u)lpt and umass and with > > these modules loaded: fdescfs.ko > > if_iwn.ko > > snd_hda.ko > > sound.ko > > umass.ko > > iwn5000fw.ko > > nvidia.ko (256.53) > > linux.ko > > cuse4bsd.ko > > atapicam.ko > > linprocfs.ko > > > > Kernel and world are compiled with > > FreeBSD clang version 2.8 (tags/RELEASE_28 115870) 20101007 > > > > Reverting to r216351 (kernel, world, mergemaster) brought things back > > to normal. I can do a binary search if desired. Did someone else also > > see this? > > > > Happy 2011, > > Rene > > Try backing out rtld down to version prior to this commit > http://svn.freebsd.org/changeset/base/216695 . There is an issue with > rtld's use of SSE on amd64 which will be fixed soon. i tried adding the following to CFLAGS to prevent clang from using any SSE* instructions, but it seems that doesn't work: CFLAGS=-mno-sse -mno-sse2 -mno-sse3 -mno-3dnow -mno-ssse3 returned: fatal error: error in backend: SSE register return with SSE disabled *** Error code 1 in lib/libcompiler_rt. cheers. alex > > -- > Alexander Kabaev -- a13x From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 01:19:42 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 382D8106566C for ; Sat, 1 Jan 2011 01:19:42 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id D9E8D8FC13 for ; Sat, 1 Jan 2011 01:19:41 +0000 (UTC) Received: (qmail 26196 invoked by uid 399); 1 Jan 2011 01:19:40 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 1 Jan 2011 01:19:40 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D1E812A.5060500@FreeBSD.org> Date: Fri, 31 Dec 2010 17:19:38 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101210 Thunderbird/3.1.7 MIME-Version: 1.0 To: churanov.port.maintainer@gmail.com X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, freebsd-ports@FreeBSD.org Subject: boost libs error X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 01:19:42 -0000 I'm getting the following with qbittorrent-23 which depends on libtorrent-rasterbar-15 after the latest boost lib update: qbittorrent terminate called after throwing an instance of 'std::runtime_error' what(): locale::facet::_S_create_c_locale name not valid Abort trap: 6 (core dumped) This is on amd64-current, r216834M #0 0x0000000805a6c4ac in thr_kill () at thr_kill.S:3 3 RSYSCALL(thr_kill) [New Thread 807807400 (LWP 101197/initial thread)] (gdb) where #0 0x0000000805a6c4ac in thr_kill () at thr_kill.S:3 #1 0x0000000805b04e8b in abort () at /home/svn/head/lib/libc/stdlib/abort.c:65 #2 0x000000080552f0a4 in __gnu_cxx::__verbose_terminate_handler () at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/libsupc++/vterminate.cc:98 #3 0x00000008055335a3 in __cxxabiv1::__terminate (handler=Variable "handler" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/libsupc++/eh_terminate.cc:43 #4 0x00000008055335e3 in std::terminate () at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/libsupc++/eh_terminate.cc:53 #5 0x000000080553354a in __cxa_throw (obj=Variable "obj" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/libsupc++/eh_throw.cc:76 #6 0x0000000805584a02 in std::__throw_runtime_error (__s=Variable "__s" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/src/functexcept.cc:84 #7 0x0000000805583a8d in std::locale::facet::_S_create_c_locale (__cloc=Variable "__cloc" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/config/locale/generic/c_locale.cc:141 #8 0x000000080550c24c in _Impl (this=0x80781a1c0, __s=0x7fffffffec72 "en_US.UTF-8", __refs=Variable "__refs" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/src/localename.cc:185 #9 0x000000080550df59 in locale (this=0x800fefcd0, __s=Variable "__s" is not available. ) at /home/svn/head/gnu/lib/libstdc++/../../../contrib/libstdc++/src/localename.cc:57 #10 0x0000000800ee3b86 in path_locale () at libs/filesystem/v3/src/path.cpp:763 #11 0x0000000800ee3d88 in boost::filesystem3::path::wchar_t_codecvt_facet () at libs/filesystem/v3/src/path.cpp:791 #12 0x0000000800ee47a6 in __static_initialization_and_destruction_0 ( __initialize_p=Variable "__initialize_p" is not available. ) at path.hpp:377 #13 0x0000000800ee9122 in __do_global_ctors_aux () from /usr/local/lib/libboost_filesystem.so #14 0x0000000800ed3fc6 in _init () from /usr/local/lib/libboost_filesystem.so #15 0x0000000800b0c180 in list_fini () from /libexec/ld-elf.so.1 #16 0x00000008009e2168 in objlist_call_init (list=Variable "list" is not available. ) at /home/svn/head/libexec/rtld-elf/rtld.c:1801 #17 0x00000008009e49b9 in _rtld (sp=0x7fffffffe8c8, exit_proc=0x7fffffffe8a0, objp=0x7fffffffe8a8) at /home/svn/head/libexec/rtld-elf/rtld.c:523 #18 0x00000008009de849 in .rtld_start () at /home/svn/head/libexec/rtld-elf/amd64/rtld_start.S:39 #19 0x0000000000000000 in ?? () ... and then similar for 766 frames. -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 07:59:35 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D248106566B; Sat, 1 Jan 2011 07:59:35 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 090728FC08; Sat, 1 Jan 2011 07:59:34 +0000 (UTC) Received: by qyk8 with SMTP id 8so12476233qyk.13 for ; Fri, 31 Dec 2010 23:59:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type; bh=o06RMegQTLer/u+rOurJM+m9H85eft0u7rdS6nuOq4Y=; b=SDenvWdWnBsZy2ewvolv7AKJD/IbFEmbjPQ58Q9PbuiQZ3F7FTfE+u5WsAWIi7AID2 en3vT3ae2dC4isjz9J03GZFtSrHQrNP+maJTkIrBVmgJpBV8ZtYDAxNXHiix6/7RTGXw SHgP9/ZpHPMNu0m0+DnKChO1P91Fk/5zEd9Hw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=bESBFiNn9KC4M6CZiAP0NhN1L9VjhoBx2yeWUogWjmn4EaQaLyummzCVyYgl5qWSqa 4B5BvyV7D3MuvvvlIglh7YHLEIYuf2MVfIurI/SRA+EkqgSMXONjDcBGuSLFGz1se/g2 7sO5QBkJyZnKDy4UKJMg9qh3CVkL1hsj2EO9g= Received: by 10.229.189.14 with SMTP id dc14mr16062046qcb.58.1293868774112; Fri, 31 Dec 2010 23:59:34 -0800 (PST) Received: from kan.dnsalias.net (c-24-63-226-98.hsd1.ma.comcast.net [24.63.226.98]) by mx.google.com with ESMTPS id l12sm10570654qcu.43.2010.12.31.23.59.31 (version=SSLv3 cipher=RC4-MD5); Fri, 31 Dec 2010 23:59:32 -0800 (PST) Date: Sat, 1 Jan 2011 02:59:26 -0500 From: Alexander Kabaev To: Alexander Best Message-ID: <20110101025926.3903ae15@kan.dnsalias.net> In-Reply-To: <20110101010600.GA26829@freebsd.org> References: <20101231174351.127b7764@kan.dnsalias.net> <20110101010600.GA26829@freebsd.org> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.22.1; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/RCyqTxOjQZFsfUzFfY=V0pf"; protocol="application/pgp-signature" Cc: =?iso-8859-15?Q?Ren=E9?= Ladan , current@freebsd.org Subject: Re: userland weirdness between r216351 and r216738 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 07:59:35 -0000 --Sig_/RCyqTxOjQZFsfUzFfY=V0pf Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sat, 1 Jan 2011 01:06:00 +0000 Alexander Best wrote: >=20 > i tried adding the following to CFLAGS to prevent clang from using > any SSE* instructions, but it seems that doesn't work: >=20 > CFLAGS=3D-mno-sse -mno-sse2 -mno-sse3 -mno-3dnow -mno-ssse3 returned: >=20 > fatal error: error in backend: SSE register return with SSE disabled > *** Error code 1 >=20 > in lib/libcompiler_rt. >=20 > cheers. > alex >=20 I do not remember instructing anyone to do what you did. --=20 Alexander Kabaev --Sig_/RCyqTxOjQZFsfUzFfY=V0pf Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iD8DBQFNHt7iQ6z1jMm+XZYRAuhGAJ98+xNgnNTDXxQMhpEIAwRPHJyehwCdGKCP vTPmRpXPgHnCZzEdkg6rT5k= =BWV1 -----END PGP SIGNATURE----- --Sig_/RCyqTxOjQZFsfUzFfY=V0pf-- From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 08:02:26 2011 Return-Path: Delivered-To: current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id CDF271065673; Sat, 1 Jan 2011 08:02:26 +0000 (UTC) Date: Sat, 1 Jan 2011 08:02:26 +0000 From: Alexander Best To: Alexander Kabaev Message-ID: <20110101080226.GA72459@freebsd.org> References: <20101231174351.127b7764@kan.dnsalias.net> <20110101010600.GA26829@freebsd.org> <20110101025926.3903ae15@kan.dnsalias.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110101025926.3903ae15@kan.dnsalias.net> Cc: =?iso-8859-15?Q?Ren=E9?= Ladan , current@freebsd.org Subject: Re: userland weirdness between r216351 and r216738 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 08:02:26 -0000 On Sat Jan 1 11, Alexander Kabaev wrote: > On Sat, 1 Jan 2011 01:06:00 +0000 > Alexander Best wrote: > > > > > i tried adding the following to CFLAGS to prevent clang from using > > any SSE* instructions, but it seems that doesn't work: > > > > CFLAGS=-mno-sse -mno-sse2 -mno-sse3 -mno-3dnow -mno-ssse3 returned: > > > > fatal error: error in backend: SSE register return with SSE disabled > > *** Error code 1 > > > > in lib/libcompiler_rt. > > > > cheers. > > alex > > > > I do not remember instructing anyone to do what you did. *lol* i've heard people sometimes try thinks without being instructed to do so. > -- > Alexander Kabaev -- a13x From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 11:57:44 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A2791065670 for ; Sat, 1 Jan 2011 11:57:44 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id C1DC68FC13 for ; Sat, 1 Jan 2011 11:57:43 +0000 (UTC) Received: by qyk36 with SMTP id 36so11847479qyk.13 for ; Sat, 01 Jan 2011 03:57:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=ZN3O2MP+UOXjIAa+Odef1qFaV1gExccGSbWasak2h0E=; b=wRC2xrVefNEaBDoDQ/KQ49DbhvF9NfBIoCGCVBZxQ4l9QYgP618MBS4Z1xD/LtQ0HQ gKhqmzx4A9SuHbGA0p8SuSHQ0B3R1gj9Ib9GcLtt7XMGw9KErKJzEGcupMTnquanGXDZ A0p+4sCeHMfOKTucWDSzBo7TRoIiDv1SRPvEU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=pWvlgOIbMxSxCad4lpnXH/lCZZ1tTpoA1fv4Bq8vw3QBVGTIcEcJKuN4SCm2ks13cU XFMpmMf71ipFkaQE93YhyeswlyyveiscIDpDKFIi8QXSR4EA+udCZ5bV2RGt9uDSk6iA dg5IYaW/w4SDSsap8mbEWk7qI/cyg4LJQE1HU= MIME-Version: 1.0 Received: by 10.229.236.193 with SMTP id kl1mr17018408qcb.114.1293883062894; Sat, 01 Jan 2011 03:57:42 -0800 (PST) Sender: r.c.ladan@gmail.com Received: by 10.229.222.14 with HTTP; Sat, 1 Jan 2011 03:57:42 -0800 (PST) In-Reply-To: <868vz58sik.wl%poyopoyo@puripuri.plala.or.jp> References: <868vz58sik.wl%poyopoyo@puripuri.plala.or.jp> Date: Sat, 1 Jan 2011 12:57:42 +0100 X-Google-Sender-Auth: 97sIli5rhhyErvX_qm5B77TRx9w Message-ID: From: =?ISO-8859-1?Q?Ren=E9_Ladan?= To: current@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: userland weirdness between r216351 and r216738 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 11:57:44 -0000 2010/12/31 : > A happy new yaer Ren=EF=BF=BD+1, > > At Fri, 31 Dec 2010 22:35:05 +0100, > Ren=EF=BF=BD+1 Ladan wrote: >> somewhere between 9.0-amd64 r216351 and r216738, I've noticed some >> userland weirdness. > > I suppose you've been hit by rtld bug between r216695[*1] and r216728[*2]= . > Maybe, but then something is still wrong, as the version I reported is 10 revisions ahead of [*2]. > > They all have happened on me and already gone. > Maybe recompiling with gcc helps, as suggested elsewhere in this thread. > [*1] r216695: http://docs.FreeBSD.org/cgi/mid.cgi?201012250851.oBP8pLLm01= 7014 > [*2] r216728: http://docs.FreeBSD.org/cgi/mid.cgi?201012270030.oBR0UTq900= 4790 > [*3] "problem with port gobject-introspection on 9.0-CURRENT" > =C2=A0 =C2=A0 http://docs.FreeBSD.org/cgi/mid.cgi?alpine.LRH.2.02.1012261= 329550.8533 > =C2=A0 =C2=A0 and > =C2=A0 =C2=A0 http://docs.FreeBSD.org/cgi/mid.cgi?alpine.LRH.2.02.1012282= 053100.28301 > =C2=A0 =C2=A0 (He is my hero.) > Rene From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 12:15:34 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85596106566B for ; Sat, 1 Jan 2011 12:15:34 +0000 (UTC) (envelope-from beat@chruetertee.ch) Received: from marvin.chruetertee.ch (marvin.chruetertee.ch [217.150.245.55]) by mx1.freebsd.org (Postfix) with ESMTP id 069138FC14 for ; Sat, 1 Jan 2011 12:15:33 +0000 (UTC) Received: from daedalus.network.local (89-131.77-83.cust.bluewin.ch [83.77.131.89]) (authenticated bits=0) by marvin.chruetertee.ch (8.14.3/8.14.3) with ESMTP id p01CFWqg091975 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO) for ; Sat, 1 Jan 2011 12:15:32 GMT (envelope-from beat@chruetertee.ch) Message-ID: <4D1F1AE8.5040704@chruetertee.ch> Date: Sat, 01 Jan 2011 13:15:36 +0100 From: =?ISO-8859-1?Q?Beat_G=E4tzi?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.16) Gecko/20101210 Lightning/1.0b1 Thunderbird/3.0.11 MIME-Version: 1.0 To: current@freebsd.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 12:15:34 -0000 Hi, Since a couple of days commands like ls(1) are very slow on one of my tinderboxes. Checking with ktrace/kdump I see that the lstat syscall takes about one second: 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) 70559 ls 0.004651 NAMI "Mk" 70559 ls 0.004664 STRU struct stat {... } 70559 ls 0.004672 RET lstat 0 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) 70559 ls 0.004696 NAMI "README,v" 70559 ls 1.004245 STRU struct stat {...} 70559 ls 1.004263 RET lstat 0 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) 70559 ls 1.004286 NAMI ".cvsignore,v" 70559 ls 2.004282 STRU struct stat {...} 70559 ls 2.004302 RET lstat 0 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) 70559 ls 2.004325 NAMI "CHANGES,v" 70559 ls 3.004275 STRU struct stat {...} 70559 ls 3.004296 RET lstat 0 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) 70559 ls 3.004318 NAMI "COPYRIGHT,v" 70559 ls 4.004300 STRU struct stat {...} 70559 ls 4.004318 RET lstat 0 When i immediately re-execute the command in the same directory it looks normal: 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) 78007 ls 0.004673 NAMI "Mk" 78007 ls 0.004686 STRU struct stat {...} 78007 ls 0.004693 RET lstat 0 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) 78007 ls 0.004715 NAMI "README,v" 78007 ls 0.004728 STRU struct stat {...} 78007 ls 0.004735 RET lstat 0 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) 78007 ls 0.004749 NAMI ".cvsignore,v" 78007 ls 0.004761 STRU struct stat {...} 78007 ls 0.004768 RET lstat 0 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) 78007 ls 0.004782 NAMI "CHANGES,v" 78007 ls 0.004795 STRU struct stat {...} 78007 ls 0.004802 RET lstat 0 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) 78007 ls 0.004817 NAMI "COPYRIGHT,v" 78007 ls 0.004828 STRU struct stat {...} 78007 ls 0.004835 RET lstat 0 System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There is no evidence of a RAID controller or hard disk problem in the system logs. Also writing to a file and reading from a file looks normal. The running kernel is a GENERIC kernel from Juli without debugging options: # uname -a FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 r209980: Tue Jul 13 11:25:50 CEST 2010 root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/BEASTIE i386 # uptime 12:22pm up 170 days, 20:13, 2 users, load averages: 0.00, 0.00, 0.00 Does anybody knows what could cause such a behavior or how to track this down? Thanks, Beat From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 15:10:08 2011 Return-Path: Delivered-To: current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id B388E1065670; Sat, 1 Jan 2011 15:10:08 +0000 (UTC) Date: Sat, 1 Jan 2011 15:10:08 +0000 From: Alexander Best To: Beat =?iso-8859-15?Q?G=E4tzi?= Message-ID: <20110101151008.GA7762@freebsd.org> References: <4D1F1AE8.5040704@chruetertee.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4D1F1AE8.5040704@chruetertee.ch> Cc: current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 15:10:08 -0000 On Sat Jan 1 11, Beat Gätzi wrote: > Hi, > > Since a couple of days commands like ls(1) are very slow on one of my > tinderboxes. Checking with ktrace/kdump I see that the lstat syscall > takes about one second: > > 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) > 70559 ls 0.004651 NAMI "Mk" > 70559 ls 0.004664 STRU struct stat {... } > 70559 ls 0.004672 RET lstat 0 > 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) > 70559 ls 0.004696 NAMI "README,v" > 70559 ls 1.004245 STRU struct stat {...} > 70559 ls 1.004263 RET lstat 0 > 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) > 70559 ls 1.004286 NAMI ".cvsignore,v" > 70559 ls 2.004282 STRU struct stat {...} > 70559 ls 2.004302 RET lstat 0 > 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) > 70559 ls 2.004325 NAMI "CHANGES,v" > 70559 ls 3.004275 STRU struct stat {...} > 70559 ls 3.004296 RET lstat 0 > 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) > 70559 ls 3.004318 NAMI "COPYRIGHT,v" > 70559 ls 4.004300 STRU struct stat {...} > 70559 ls 4.004318 RET lstat 0 > > When i immediately re-execute the command in the same directory it looks > normal: > > 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) > 78007 ls 0.004673 NAMI "Mk" > 78007 ls 0.004686 STRU struct stat {...} > 78007 ls 0.004693 RET lstat 0 > 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) > 78007 ls 0.004715 NAMI "README,v" > 78007 ls 0.004728 STRU struct stat {...} > 78007 ls 0.004735 RET lstat 0 > 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) > 78007 ls 0.004749 NAMI ".cvsignore,v" > 78007 ls 0.004761 STRU struct stat {...} > 78007 ls 0.004768 RET lstat 0 > 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) > 78007 ls 0.004782 NAMI "CHANGES,v" > 78007 ls 0.004795 STRU struct stat {...} > 78007 ls 0.004802 RET lstat 0 > 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) > 78007 ls 0.004817 NAMI "COPYRIGHT,v" > 78007 ls 0.004828 STRU struct stat {...} > 78007 ls 0.004835 RET lstat 0 > > System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There > is no evidence of a RAID controller or hard disk problem in the system > logs. Also writing to a file and reading from a file looks normal. > The running kernel is a GENERIC kernel from Juli without debugging options: > > # uname -a > FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 > r209980: Tue Jul 13 11:25:50 CEST 2010 > root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/BEASTIE > i386 no problems here. first 'ls': ... 58144 ls 0.166961 CALL lstat(0x800c4cf08,0x800c4ce90) 58144 ls 0.166998 RET lstat 0 58144 ls 0.167019 CALL lstat(0x800c4d048,0x800c4cfd0) 58144 ls 0.167061 RET lstat 0 58144 ls 0.167066 CALL lstat(0x800c4d188,0x800c4d110) 58144 ls 0.167089 RET lstat 0 58144 ls 0.167093 CALL lstat(0x800c4d2c8,0x800c4d250) 58144 ls 0.216333 RET lstat 0 58144 ls 0.216345 CALL lstat(0x800c4d408,0x800c4d390) 58144 ls 0.216380 RET lstat 0 58144 ls 0.216385 CALL lstat(0x800c4d548,0x800c4d4d0) 58144 ls 0.216687 RET lstat 0 58144 ls 0.216695 CALL lstat(0x800c4d688,0x800c4d610) 58144 ls 0.216720 RET lstat 0 58144 ls 0.216725 CALL lstat(0x800c4d7c8,0x800c4d750) 58144 ls 0.216748 RET lstat 0 58144 ls 0.216752 CALL lstat(0x800c4d908,0x800c4d890) 58144 ls 0.216788 RET lstat 0 58144 ls 0.216793 CALL lstat(0x800c4da48,0x800c4d9d0) 58144 ls 0.216814 RET lstat 0 58144 ls 0.216818 CALL lstat(0x800c4db88,0x800c4db10) 58144 ls 0.216835 RET lstat 0 second 'ls': ... 58158 ls 0.005801 CALL lstat(0x800c4cf08,0x800c4ce90) 58158 ls 0.005817 RET lstat 0 58158 ls 0.005827 CALL lstat(0x800c4d048,0x800c4cfd0) 58158 ls 0.005844 RET lstat 0 58158 ls 0.005849 CALL lstat(0x800c4d188,0x800c4d110) 58158 ls 0.005870 RET lstat 0 58158 ls 0.005875 CALL lstat(0x800c4d2c8,0x800c4d250) 58158 ls 0.005891 RET lstat 0 58158 ls 0.005896 CALL lstat(0x800c4d408,0x800c4d390) 58158 ls 0.005912 RET lstat 0 58158 ls 0.005917 CALL lstat(0x800c4d548,0x800c4d4d0) 58158 ls 0.005933 RET lstat 0 58158 ls 0.005938 CALL lstat(0x800c4d688,0x800c4d610) 58158 ls 0.005955 RET lstat 0 58158 ls 0.005959 CALL lstat(0x800c4d7c8,0x800c4d750) 58158 ls 0.005981 RET lstat 0 58158 ls 0.005986 CALL lstat(0x800c4d908,0x800c4d890) 58158 ls 0.006002 RET lstat 0 58158 ls 0.006006 CALL lstat(0x800c4da48,0x800c4d9d0) 58158 ls 0.006023 RET lstat 0 58158 ls 0.006028 CALL lstat(0x800c4db88,0x800c4db10) 58158 ls 0.006043 RET lstat 0 ...seems like the usual time improvement due to caching. this is under: 'uname -a': FreeBSD otaku 9.0-CURRENT FreeBSD 9.0-CURRENT #0 r216763: Thu Dec 30 00:56:03 CET 2010 root@:/usr/obj/usr/subversion-src/sys/ARUNDEL amd64 'uptime': 4:04pm up 7:17, 1 user, load averages: 1,01 1,07 1,00 'diskinfo -ct ada1': ada1 512 # sectorsize 1000204886016 # mediasize in bytes (932G) 1953525168 # mediasize in sectors 0 # stripesize 0 # stripeoffset 1938021 # Cylinders according to firmware. 16 # Heads according to firmware. 63 # Sectors according to firmware. S246J9EZ805506 # Disk ident. I/O command overhead: time to read 10MB block 0.141371 sec = 0.007 msec/sector time to read 20480 sectors 1.585743 sec = 0.077 msec/sector calculated command overhead = 0.071 msec/sector Seek times: Full stroke: 250 iter in 5.347934 sec = 21.392 msec Half stroke: 250 iter in 4.037380 sec = 16.150 msec Quarter stroke: 500 iter in 6.783579 sec = 13.567 msec Short forward: 400 iter in 3.569570 sec = 8.924 msec Short backward: 400 iter in 3.197921 sec = 7.995 msec Seq outer: 2048 iter in 0.102532 sec = 0.050 msec Seq inner: 2048 iter in 0.105670 sec = 0.052 msec Transfer rates: outside: 102400 kbytes in 0.971045 sec = 105453 kbytes/sec middle: 102400 kbytes in 0.876063 sec = 116887 kbytes/sec inside: 102400 kbytes in 1.426225 sec = 71798 kbytes/sec cheers. alex > > # uptime > 12:22pm up 170 days, 20:13, 2 users, load averages: 0.00, 0.00, 0.00 > > Does anybody knows what could cause such a behavior or how to track this > down? > > Thanks, > Beat -- a13x From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 15:37:42 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81E36106566C for ; Sat, 1 Jan 2011 15:37:42 +0000 (UTC) (envelope-from beat@chruetertee.ch) Received: from marvin.chruetertee.ch (marvin.chruetertee.ch [217.150.245.55]) by mx1.freebsd.org (Postfix) with ESMTP id 2AE9C8FC0A for ; Sat, 1 Jan 2011 15:37:41 +0000 (UTC) Received: from daedalus.network.local (89-131.77-83.cust.bluewin.ch [83.77.131.89]) (authenticated bits=0) by marvin.chruetertee.ch (8.14.3/8.14.3) with ESMTP id p01FbeZe062566 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 15:37:41 GMT (envelope-from beat@chruetertee.ch) Message-ID: <4D1F4A48.6080604@chruetertee.ch> Date: Sat, 01 Jan 2011 16:37:44 +0100 From: =?ISO-8859-15?Q?Beat_G=E4tzi?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.16) Gecko/20101210 Lightning/1.0b1 Thunderbird/3.0.11 MIME-Version: 1.0 To: Alexander Best References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> In-Reply-To: <20110101151008.GA7762@freebsd.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 15:37:42 -0000 On 01.01.2011 16:10, Alexander Best wrote: > On Sat Jan 1 11, Beat Gätzi wrote: >> Hi, >> >> Since a couple of days commands like ls(1) are very slow on one of my >> tinderboxes. Checking with ktrace/kdump I see that the lstat syscall >> takes about one second: >> >> 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) >> 70559 ls 0.004651 NAMI "Mk" >> 70559 ls 0.004664 STRU struct stat {... } >> 70559 ls 0.004672 RET lstat 0 >> 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) >> 70559 ls 0.004696 NAMI "README,v" >> 70559 ls 1.004245 STRU struct stat {...} >> 70559 ls 1.004263 RET lstat 0 >> 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) >> 70559 ls 1.004286 NAMI ".cvsignore,v" >> 70559 ls 2.004282 STRU struct stat {...} >> 70559 ls 2.004302 RET lstat 0 >> 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) >> 70559 ls 2.004325 NAMI "CHANGES,v" >> 70559 ls 3.004275 STRU struct stat {...} >> 70559 ls 3.004296 RET lstat 0 >> 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) >> 70559 ls 3.004318 NAMI "COPYRIGHT,v" >> 70559 ls 4.004300 STRU struct stat {...} >> 70559 ls 4.004318 RET lstat 0 >> >> When i immediately re-execute the command in the same directory it looks >> normal: >> >> 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) >> 78007 ls 0.004673 NAMI "Mk" >> 78007 ls 0.004686 STRU struct stat {...} >> 78007 ls 0.004693 RET lstat 0 >> 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) >> 78007 ls 0.004715 NAMI "README,v" >> 78007 ls 0.004728 STRU struct stat {...} >> 78007 ls 0.004735 RET lstat 0 >> 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) >> 78007 ls 0.004749 NAMI ".cvsignore,v" >> 78007 ls 0.004761 STRU struct stat {...} >> 78007 ls 0.004768 RET lstat 0 >> 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) >> 78007 ls 0.004782 NAMI "CHANGES,v" >> 78007 ls 0.004795 STRU struct stat {...} >> 78007 ls 0.004802 RET lstat 0 >> 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) >> 78007 ls 0.004817 NAMI "COPYRIGHT,v" >> 78007 ls 0.004828 STRU struct stat {...} >> 78007 ls 0.004835 RET lstat 0 >> >> System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There >> is no evidence of a RAID controller or hard disk problem in the system >> logs. Also writing to a file and reading from a file looks normal. >> The running kernel is a GENERIC kernel from Juli without debugging options: >> >> # uname -a >> FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 >> r209980: Tue Jul 13 11:25:50 CEST 2010 >> root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/BEASTIE >> i386 > > no problems here. I haven't had this problem the first ~166 days since I rebooted the box the last time. Then the problem suddenly occur in all directories on all partitions. The box was pretty much under load all the time as it builds packages almost 7x24. I forgot to say the used filesystem is UFS. # diskinfo -ct da0 da0 512 # sectorsize 299992412160 # mediasize in bytes (279G) 585922680 # mediasize in sectors 0 # stripesize 0 # stripeoffset 36472 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. # Disk ident. I/O command overhead: time to read 10MB block 0.333161 sec = 0.016 msec/sector time to read 20480 sectors 3.392359 sec = 0.166 msec/sector calculated command overhead = 0.149 msec/sector Seek times: Full stroke: 250 iter in 1.202862 sec = 4.811 msec Half stroke: 250 iter in 1.120656 sec = 4.483 msec Quarter stroke: 500 iter in 2.109077 sec = 4.218 msec Short forward: 400 iter in 1.892342 sec = 4.731 msec Short backward: 400 iter in 1.399378 sec = 3.498 msec Seq outer: 2048 iter in 0.399352 sec = 0.195 msec Seq inner: 2048 iter in 0.385460 sec = 0.188 msec Transfer rates: outside: 102400 kbytes in 2.325967 sec = 44025 kbytes/sec middle: 102400 kbytes in 2.157681 sec = 47458 kbytes/sec inside: 102400 kbytes in 2.717089 sec = 37687 kbytes/sec I also observed that the cpu output of vmstat is a little bit confusing on this system: # vmstat procs memory page disk faults cpu r b w avm fre flt re pi po fr sr da0 in sy cs us sy id 0 0 0 1663M 1120M 117 0 1 0 120 21 0 172 27 245 -29 -10 139 Thanks, Beat From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 15:45:46 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 148101065698 for ; Sat, 1 Jan 2011 15:45:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 51CCD8FC15 for ; Sat, 1 Jan 2011 15:45:44 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p01FjbNT025572 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 17:45:37 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p01FjbNh075219; Sat, 1 Jan 2011 17:45:37 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p01Fjb9b075218; Sat, 1 Jan 2011 17:45:37 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 1 Jan 2011 17:45:37 +0200 From: Kostik Belousov To: Beat G?tzi Message-ID: <20110101154537.GW90883@deviant.kiev.zoral.com.ua> References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3mne3ZyEaQEM38Z6" Content-Disposition: inline In-Reply-To: <4D1F4A48.6080604@chruetertee.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 15:45:46 -0000 --3mne3ZyEaQEM38Z6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 01, 2011 at 04:37:44PM +0100, Beat G?tzi wrote: > On 01.01.2011 16:10, Alexander Best wrote: > > On Sat Jan 1 11, Beat G?tzi wrote: > >> Hi, > >> > >> Since a couple of days commands like ls(1) are very slow on one of my > >> tinderboxes. Checking with ktrace/kdump I see that the lstat syscall > >> takes about one second: > >> > >> 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) > >> 70559 ls 0.004651 NAMI "Mk" > >> 70559 ls 0.004664 STRU struct stat {... } > >> 70559 ls 0.004672 RET lstat 0 > >> 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) > >> 70559 ls 0.004696 NAMI "README,v" > >> 70559 ls 1.004245 STRU struct stat {...} > >> 70559 ls 1.004263 RET lstat 0 > >> 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) > >> 70559 ls 1.004286 NAMI ".cvsignore,v" > >> 70559 ls 2.004282 STRU struct stat {...} > >> 70559 ls 2.004302 RET lstat 0 > >> 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) > >> 70559 ls 2.004325 NAMI "CHANGES,v" > >> 70559 ls 3.004275 STRU struct stat {...} > >> 70559 ls 3.004296 RET lstat 0 > >> 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) > >> 70559 ls 3.004318 NAMI "COPYRIGHT,v" > >> 70559 ls 4.004300 STRU struct stat {...} > >> 70559 ls 4.004318 RET lstat 0 > >> > >> When i immediately re-execute the command in the same directory it loo= ks > >> normal: > >> > >> 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) > >> 78007 ls 0.004673 NAMI "Mk" > >> 78007 ls 0.004686 STRU struct stat {...} > >> 78007 ls 0.004693 RET lstat 0 > >> 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) > >> 78007 ls 0.004715 NAMI "README,v" > >> 78007 ls 0.004728 STRU struct stat {...} > >> 78007 ls 0.004735 RET lstat 0 > >> 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) > >> 78007 ls 0.004749 NAMI ".cvsignore,v" > >> 78007 ls 0.004761 STRU struct stat {...} > >> 78007 ls 0.004768 RET lstat 0 > >> 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) > >> 78007 ls 0.004782 NAMI "CHANGES,v" > >> 78007 ls 0.004795 STRU struct stat {...} > >> 78007 ls 0.004802 RET lstat 0 > >> 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) > >> 78007 ls 0.004817 NAMI "COPYRIGHT,v" > >> 78007 ls 0.004828 STRU struct stat {...} > >> 78007 ls 0.004835 RET lstat 0 > >> > >> System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There > >> is no evidence of a RAID controller or hard disk problem in the system > >> logs. Also writing to a file and reading from a file looks normal. > >> The running kernel is a GENERIC kernel from Juli without debugging opt= ions: > >> > >> # uname -a > >> FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 > >> r209980: Tue Jul 13 11:25:50 CEST 2010 > >> root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/= BEASTIE > >> i386 > >=20 > > no problems here. >=20 > I haven't had this problem the first ~166 days since I rebooted the box > the last time. Then the problem suddenly occur in all directories on all > partitions. The box was pretty much under load all the time as it builds > packages almost 7x24. I forgot to say the used filesystem is UFS. >=20 > # diskinfo -ct da0 > da0 > 512 # sectorsize > 299992412160 # mediasize in bytes (279G) > 585922680 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 36472 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > # Disk ident. >=20 > I/O command overhead: > time to read 10MB block 0.333161 sec =3D 0.016 msec/sector > time to read 20480 sectors 3.392359 sec =3D 0.166 msec/sector > calculated command overhead =3D 0.149 msec/sector >=20 > Seek times: > Full stroke: 250 iter in 1.202862 sec =3D 4.811 msec > Half stroke: 250 iter in 1.120656 sec =3D 4.483 msec > Quarter stroke: 500 iter in 2.109077 sec =3D 4.218 msec > Short forward: 400 iter in 1.892342 sec =3D 4.731 msec > Short backward: 400 iter in 1.399378 sec =3D 3.498 msec > Seq outer: 2048 iter in 0.399352 sec =3D 0.195 msec > Seq inner: 2048 iter in 0.385460 sec =3D 0.188 msec > Transfer rates: > outside: 102400 kbytes in 2.325967 sec =3D 44025 kbytes/sec > middle: 102400 kbytes in 2.157681 sec =3D 47458 kbytes/sec > inside: 102400 kbytes in 2.717089 sec =3D 37687 kbytes/sec >=20 > I also observed that the cpu output of vmstat is a little bit confusing > on this system: >=20 > # vmstat > procs memory page disk faults cpu > r b w avm fre flt re pi po fr sr da0 in sy cs us sy id > 0 0 0 1663M 1120M 117 0 1 0 120 21 0 172 27 245 -29 -10 139 >=20 Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect they are quite close or equial. If yes, consider increasing maxvnodes. Another workaround, if you have huge nested directories hierarhy, is to set vfs.vlru_allow_cache_src to 1. You did not specified how much memory your machine have, but I assume it is > 1GB. Anyway, increase of maxvnodes on i386 should be done very cautiously, since it is easy to exhaust KVA. --3mne3ZyEaQEM38Z6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0fTCEACgkQC3+MBN1Mb4iMXwCfezYXDS/NRNn1POAIV4uKfsCw Lb8An34m1u0jmmJc6dWGEF2OtuoTf4rX =JmR7 -----END PGP SIGNATURE----- --3mne3ZyEaQEM38Z6-- From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:00:54 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C205A106564A; Sat, 1 Jan 2011 16:00:54 +0000 (UTC) (envelope-from beat@chruetertee.ch) Received: from marvin.chruetertee.ch (marvin.chruetertee.ch [217.150.245.55]) by mx1.freebsd.org (Postfix) with ESMTP id 6AE248FC1D; Sat, 1 Jan 2011 16:00:54 +0000 (UTC) Received: from daedalus.network.local (89-131.77-83.cust.bluewin.ch [83.77.131.89]) (authenticated bits=0) by marvin.chruetertee.ch (8.14.3/8.14.3) with ESMTP id p01G0r8M029535 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 16:00:53 GMT (envelope-from beat@chruetertee.ch) Message-ID: <4D1F4FB8.3030303@chruetertee.ch> Date: Sat, 01 Jan 2011 17:00:56 +0100 From: =?ISO-8859-1?Q?Beat_G=E4tzi?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.16) Gecko/20101210 Lightning/1.0b1 Thunderbird/3.0.11 MIME-Version: 1.0 To: Kostik Belousov References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> In-Reply-To: <20110101154537.GW90883@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:00:54 -0000 On 01.01.2011 16:45, Kostik Belousov wrote: > On Sat, Jan 01, 2011 at 04:37:44PM +0100, Beat G?tzi wrote: >> On 01.01.2011 16:10, Alexander Best wrote: >>> On Sat Jan 1 11, Beat G?tzi wrote: >>>> Hi, >>>> >>>> Since a couple of days commands like ls(1) are very slow on one of my >>>> tinderboxes. Checking with ktrace/kdump I see that the lstat syscall >>>> takes about one second: >>>> >>>> 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) >>>> 70559 ls 0.004651 NAMI "Mk" >>>> 70559 ls 0.004664 STRU struct stat {... } >>>> 70559 ls 0.004672 RET lstat 0 >>>> 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) >>>> 70559 ls 0.004696 NAMI "README,v" >>>> 70559 ls 1.004245 STRU struct stat {...} >>>> 70559 ls 1.004263 RET lstat 0 >>>> 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) >>>> 70559 ls 1.004286 NAMI ".cvsignore,v" >>>> 70559 ls 2.004282 STRU struct stat {...} >>>> 70559 ls 2.004302 RET lstat 0 >>>> 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) >>>> 70559 ls 2.004325 NAMI "CHANGES,v" >>>> 70559 ls 3.004275 STRU struct stat {...} >>>> 70559 ls 3.004296 RET lstat 0 >>>> 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) >>>> 70559 ls 3.004318 NAMI "COPYRIGHT,v" >>>> 70559 ls 4.004300 STRU struct stat {...} >>>> 70559 ls 4.004318 RET lstat 0 >>>> >>>> When i immediately re-execute the command in the same directory it looks >>>> normal: >>>> >>>> 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) >>>> 78007 ls 0.004673 NAMI "Mk" >>>> 78007 ls 0.004686 STRU struct stat {...} >>>> 78007 ls 0.004693 RET lstat 0 >>>> 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) >>>> 78007 ls 0.004715 NAMI "README,v" >>>> 78007 ls 0.004728 STRU struct stat {...} >>>> 78007 ls 0.004735 RET lstat 0 >>>> 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) >>>> 78007 ls 0.004749 NAMI ".cvsignore,v" >>>> 78007 ls 0.004761 STRU struct stat {...} >>>> 78007 ls 0.004768 RET lstat 0 >>>> 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) >>>> 78007 ls 0.004782 NAMI "CHANGES,v" >>>> 78007 ls 0.004795 STRU struct stat {...} >>>> 78007 ls 0.004802 RET lstat 0 >>>> 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) >>>> 78007 ls 0.004817 NAMI "COPYRIGHT,v" >>>> 78007 ls 0.004828 STRU struct stat {...} >>>> 78007 ls 0.004835 RET lstat 0 >>>> >>>> System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There >>>> is no evidence of a RAID controller or hard disk problem in the system >>>> logs. Also writing to a file and reading from a file looks normal. >>>> The running kernel is a GENERIC kernel from Juli without debugging options: >>>> >>>> # uname -a >>>> FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 >>>> r209980: Tue Jul 13 11:25:50 CEST 2010 >>>> root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/BEASTIE >>>> i386 >>> >>> no problems here. >> >> I haven't had this problem the first ~166 days since I rebooted the box >> the last time. Then the problem suddenly occur in all directories on all >> partitions. The box was pretty much under load all the time as it builds >> packages almost 7x24. I forgot to say the used filesystem is UFS. >> >> # diskinfo -ct da0 >> da0 >> 512 # sectorsize >> 299992412160 # mediasize in bytes (279G) >> 585922680 # mediasize in sectors >> 0 # stripesize >> 0 # stripeoffset >> 36472 # Cylinders according to firmware. >> 255 # Heads according to firmware. >> 63 # Sectors according to firmware. >> # Disk ident. >> >> I/O command overhead: >> time to read 10MB block 0.333161 sec = 0.016 msec/sector >> time to read 20480 sectors 3.392359 sec = 0.166 msec/sector >> calculated command overhead = 0.149 msec/sector >> >> Seek times: >> Full stroke: 250 iter in 1.202862 sec = 4.811 msec >> Half stroke: 250 iter in 1.120656 sec = 4.483 msec >> Quarter stroke: 500 iter in 2.109077 sec = 4.218 msec >> Short forward: 400 iter in 1.892342 sec = 4.731 msec >> Short backward: 400 iter in 1.399378 sec = 3.498 msec >> Seq outer: 2048 iter in 0.399352 sec = 0.195 msec >> Seq inner: 2048 iter in 0.385460 sec = 0.188 msec >> Transfer rates: >> outside: 102400 kbytes in 2.325967 sec = 44025 kbytes/sec >> middle: 102400 kbytes in 2.157681 sec = 47458 kbytes/sec >> inside: 102400 kbytes in 2.717089 sec = 37687 kbytes/sec >> >> I also observed that the cpu output of vmstat is a little bit confusing >> on this system: >> >> # vmstat >> procs memory page disk faults cpu >> r b w avm fre flt re pi po fr sr da0 in sy cs us sy id >> 0 0 0 1663M 1120M 117 0 1 0 120 21 0 172 27 245 -29 -10 139 >> > Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect > they are quite close or equial. If yes, consider increasing maxvnodes. > Another workaround, if you have huge nested directories hierarhy, is > to set vfs.vlru_allow_cache_src to 1. Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: # sysctl kern.maxvnodes vfs.numvnodes kern.maxvnodes: 100000 vfs.numvnodes: 100765 I've increased kern.maxvnodes and the problem was gone until vfs.numvnodes reached the value of kern.maxvnodes again: # sysctl kern.maxvnodes vfs.numvnodes kern.maxvnodes: 150000 vfs.numvnodes: 150109 As the directory structure is quite huge on this server I've set vfs.vlru_allow_cache_src to one now. > You did not specified how much memory your machine have, but I assume it > is > 1GB. Anyway, increase of maxvnodes on i386 should be done very > cautiously, since it is easy to exhaust KVA. The server has 4GB of RAM. Is it possible to check how much i could increase kern.maxvnodes without exhausting KVA? Thanks a lot! Beat From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:13:02 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13F481065670; Sat, 1 Jan 2011 16:13:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 83A678FC0C; Sat, 1 Jan 2011 16:13:00 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p01GCsXw027371 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 18:12:54 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p01GCsXH075656; Sat, 1 Jan 2011 18:12:54 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p01GCsUQ075655; Sat, 1 Jan 2011 18:12:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 1 Jan 2011 18:12:54 +0200 From: Kostik Belousov To: Beat G?tzi Message-ID: <20110101161254.GX90883@deviant.kiev.zoral.com.ua> References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> <4D1F4FB8.3030303@chruetertee.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HLiOwqdVwqQciAhA" Content-Disposition: inline In-Reply-To: <4D1F4FB8.3030303@chruetertee.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:13:02 -0000 --HLiOwqdVwqQciAhA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 01, 2011 at 05:00:56PM +0100, Beat G?tzi wrote: > On 01.01.2011 16:45, Kostik Belousov wrote: > > Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect > > they are quite close or equial. If yes, consider increasing maxvnodes. > > Another workaround, if you have huge nested directories hierarhy, is > > to set vfs.vlru_allow_cache_src to 1. >=20 > Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: > # sysctl kern.maxvnodes vfs.numvnodes > kern.maxvnodes: 100000 > vfs.numvnodes: 100765 >=20 > I've increased kern.maxvnodes and the problem was gone until > vfs.numvnodes reached the value of kern.maxvnodes again: > # sysctl kern.maxvnodes vfs.numvnodes > kern.maxvnodes: 150000 > vfs.numvnodes: 150109 The processes should be stuck in "vlruwk" state, that can be checked with ps or '^T' on the terminal. >=20 > As the directory structure is quite huge on this server I've set > vfs.vlru_allow_cache_src to one now. Did it helped ? >=20 > > You did not specified how much memory your machine have, but I assume it > > is > 1GB. Anyway, increase of maxvnodes on i386 should be done very > > cautiously, since it is easy to exhaust KVA. >=20 > The server has 4GB of RAM. Is it possible to check how much i could > increase kern.maxvnodes without exhausting KVA? >=20 The RAM check is mostly to make sure that you have enough physical RAM to cover whole possible KVA (1GB). Too aggressive settings for maxvnodes would cause panic. Default settings of 100000 is more or less optimal for the i386 arch. If you need more, you should switch to amd64. --HLiOwqdVwqQciAhA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0fUoUACgkQC3+MBN1Mb4iilQCfZuk7CZmVRY5RLdfjhYX1efM8 slgAnRChjXqJT8ewK30vL2DEpjxpPrXo =ddIR -----END PGP SIGNATURE----- --HLiOwqdVwqQciAhA-- From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:16:57 2011 Return-Path: Delivered-To: current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 0D6BB1065672; Sat, 1 Jan 2011 16:16:57 +0000 (UTC) Date: Sat, 1 Jan 2011 16:16:57 +0000 From: Alexander Best To: Beat =?iso-8859-15?Q?G=E4tzi?= Message-ID: <20110101161657.GA16319@freebsd.org> References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4D1F4A48.6080604@chruetertee.ch> Cc: current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:16:57 -0000 On Sat Jan 1 11, Beat Gätzi wrote: > On 01.01.2011 16:10, Alexander Best wrote: > > On Sat Jan 1 11, Beat Gätzi wrote: > >> Hi, > >> > >> Since a couple of days commands like ls(1) are very slow on one of my > >> tinderboxes. Checking with ktrace/kdump I see that the lstat syscall > >> takes about one second: > >> > >> 70559 ls 0.004644 CALL lstat(0x284472f8,0x28447298) > >> 70559 ls 0.004651 NAMI "Mk" > >> 70559 ls 0.004664 STRU struct stat {... } > >> 70559 ls 0.004672 RET lstat 0 > >> 70559 ls 0.004688 CALL lstat(0x2844a1b8,0x2844a158) > >> 70559 ls 0.004696 NAMI "README,v" > >> 70559 ls 1.004245 STRU struct stat {...} > >> 70559 ls 1.004263 RET lstat 0 > >> 70559 ls 1.004277 CALL lstat(0x2844a2b8,0x2844a258) > >> 70559 ls 1.004286 NAMI ".cvsignore,v" > >> 70559 ls 2.004282 STRU struct stat {...} > >> 70559 ls 2.004302 RET lstat 0 > >> 70559 ls 2.004316 CALL lstat(0x2844a3b8,0x2844a358) > >> 70559 ls 2.004325 NAMI "CHANGES,v" > >> 70559 ls 3.004275 STRU struct stat {...} > >> 70559 ls 3.004296 RET lstat 0 > >> 70559 ls 3.004310 CALL lstat(0x2844a4b8,0x2844a458) > >> 70559 ls 3.004318 NAMI "COPYRIGHT,v" > >> 70559 ls 4.004300 STRU struct stat {...} > >> 70559 ls 4.004318 RET lstat 0 > >> > >> When i immediately re-execute the command in the same directory it looks > >> normal: > >> > >> 78007 ls 0.004665 CALL lstat(0x284472f8,0x28447298) > >> 78007 ls 0.004673 NAMI "Mk" > >> 78007 ls 0.004686 STRU struct stat {...} > >> 78007 ls 0.004693 RET lstat 0 > >> 78007 ls 0.004708 CALL lstat(0x2844a1b8,0x2844a158) > >> 78007 ls 0.004715 NAMI "README,v" > >> 78007 ls 0.004728 STRU struct stat {...} > >> 78007 ls 0.004735 RET lstat 0 > >> 78007 ls 0.004742 CALL lstat(0x2844a2b8,0x2844a258) > >> 78007 ls 0.004749 NAMI ".cvsignore,v" > >> 78007 ls 0.004761 STRU struct stat {...} > >> 78007 ls 0.004768 RET lstat 0 > >> 78007 ls 0.004775 CALL lstat(0x2844a3b8,0x2844a358) > >> 78007 ls 0.004782 NAMI "CHANGES,v" > >> 78007 ls 0.004795 STRU struct stat {...} > >> 78007 ls 0.004802 RET lstat 0 > >> 78007 ls 0.004809 CALL lstat(0x2844a4b8,0x2844a458) > >> 78007 ls 0.004817 NAMI "COPYRIGHT,v" > >> 78007 ls 0.004828 STRU struct stat {...} > >> 78007 ls 0.004835 RET lstat 0 > >> > >> System is a HP DL360 G3 with a Compaq Smart Array 5i controller. There > >> is no evidence of a RAID controller or hard disk problem in the system > >> logs. Also writing to a file and reading from a file looks normal. > >> The running kernel is a GENERIC kernel from Juli without debugging options: > >> > >> # uname -a > >> FreeBSD tinderbox.chruetertee.ch 9.0-CURRENT FreeBSD 9.0-CURRENT #5 > >> r209980: Tue Jul 13 11:25:50 CEST 2010 > >> root@tinderbox.chruetertee.ch:/usr/obj/usr/home/beat/dev/src/head/sys/BEASTIE > >> i386 > > > > no problems here. > > I haven't had this problem the first ~166 days since I rebooted the box > the last time. Then the problem suddenly occur in all directories on all > partitions. The box was pretty much under load all the time as it builds > packages almost 7x24. I forgot to say the used filesystem is UFS. > > # diskinfo -ct da0 > da0 > 512 # sectorsize > 299992412160 # mediasize in bytes (279G) > 585922680 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 36472 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > # Disk ident. > > I/O command overhead: > time to read 10MB block 0.333161 sec = 0.016 msec/sector > time to read 20480 sectors 3.392359 sec = 0.166 msec/sector > calculated command overhead = 0.149 msec/sector > > Seek times: > Full stroke: 250 iter in 1.202862 sec = 4.811 msec > Half stroke: 250 iter in 1.120656 sec = 4.483 msec > Quarter stroke: 500 iter in 2.109077 sec = 4.218 msec > Short forward: 400 iter in 1.892342 sec = 4.731 msec > Short backward: 400 iter in 1.399378 sec = 3.498 msec > Seq outer: 2048 iter in 0.399352 sec = 0.195 msec > Seq inner: 2048 iter in 0.385460 sec = 0.188 msec > Transfer rates: > outside: 102400 kbytes in 2.325967 sec = 44025 kbytes/sec > middle: 102400 kbytes in 2.157681 sec = 47458 kbytes/sec > inside: 102400 kbytes in 2.717089 sec = 37687 kbytes/sec > > I also observed that the cpu output of vmstat is a little bit confusing > on this system: > > # vmstat > procs memory page disk faults cpu > r b w avm fre flt re pi po fr sr da0 in sy cs us sy id > 0 0 0 1663M 1120M 117 0 1 0 120 21 0 172 27 245 -29 -10 139 yeah that bug really needs to be fixed. it occurs on almost all systems with high load and high uptime (> 100 days). see http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/30360 cheers. alex > > Thanks, > Beat -- a13x From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:42:56 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32B7F1065673; Sat, 1 Jan 2011 16:42:56 +0000 (UTC) (envelope-from beat@chruetertee.ch) Received: from marvin.chruetertee.ch (marvin.chruetertee.ch [217.150.245.55]) by mx1.freebsd.org (Postfix) with ESMTP id 9FFCD8FC08; Sat, 1 Jan 2011 16:42:55 +0000 (UTC) Received: from daedalus.network.local (89-131.77-83.cust.bluewin.ch [83.77.131.89]) (authenticated bits=0) by marvin.chruetertee.ch (8.14.3/8.14.3) with ESMTP id p01GgsHp008026 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 16:42:54 GMT (envelope-from beat@chruetertee.ch) Message-ID: <4D1F5992.8030309@chruetertee.ch> Date: Sat, 01 Jan 2011 17:42:58 +0100 From: =?ISO-8859-1?Q?Beat_G=E4tzi?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.16) Gecko/20101210 Lightning/1.0b1 Thunderbird/3.0.11 MIME-Version: 1.0 To: Kostik Belousov References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> <4D1F4FB8.3030303@chruetertee.ch> <20110101161254.GX90883@deviant.kiev.zoral.com.ua> In-Reply-To: <20110101161254.GX90883@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:42:56 -0000 On 01.01.2011 17:12, Kostik Belousov wrote: > On Sat, Jan 01, 2011 at 05:00:56PM +0100, Beat G?tzi wrote: >> On 01.01.2011 16:45, Kostik Belousov wrote: >>> Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect >>> they are quite close or equial. If yes, consider increasing maxvnodes. >>> Another workaround, if you have huge nested directories hierarhy, is >>> to set vfs.vlru_allow_cache_src to 1. >> >> Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: >> # sysctl kern.maxvnodes vfs.numvnodes >> kern.maxvnodes: 100000 >> vfs.numvnodes: 100765 >> >> I've increased kern.maxvnodes and the problem was gone until >> vfs.numvnodes reached the value of kern.maxvnodes again: >> # sysctl kern.maxvnodes vfs.numvnodes >> kern.maxvnodes: 150000 >> vfs.numvnodes: 150109 > The processes should be stuck in "vlruwk" state, that can be > checked with ps or '^T' on the terminal. Yes, there are various processes in "vlruwk" state, >> As the directory structure is quite huge on this server I've set >> vfs.vlru_allow_cache_src to one now. > Did it helped ? No, it doesn't looks like setting vfs.vlru_allow_cache_src helped. The problem was gone when I increased kern.maxvnodes until vfs.numvnodes reached that level. I've stopped all running deamons but numvnodes doesn't decrease. >>> You did not specified how much memory your machine have, but I assume it >>> is > 1GB. Anyway, increase of maxvnodes on i386 should be done very >>> cautiously, since it is easy to exhaust KVA. >> >> The server has 4GB of RAM. Is it possible to check how much i could >> increase kern.maxvnodes without exhausting KVA? >> > The RAM check is mostly to make sure that you have enough physical RAM > to cover whole possible KVA (1GB). Too aggressive settings for maxvnodes > would cause panic. > > Default settings of 100000 is more or less optimal for the i386 arch. > If you need more, you should switch to amd64. I see. Unfortunately the used CPUs are still 32bit CPUs so switching to amd64 is not that easy. Thanks. Beat From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:46:56 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F85A1065672; Sat, 1 Jan 2011 16:46:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id AE56C8FC08; Sat, 1 Jan 2011 16:46:55 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p01GkohM029360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 18:46:50 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p01GknCt076185; Sat, 1 Jan 2011 18:46:49 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p01GknmR076184; Sat, 1 Jan 2011 18:46:49 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 1 Jan 2011 18:46:49 +0200 From: Kostik Belousov To: Beat G?tzi Message-ID: <20110101164649.GY90883@deviant.kiev.zoral.com.ua> References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> <4D1F4FB8.3030303@chruetertee.ch> <20110101161254.GX90883@deviant.kiev.zoral.com.ua> <4D1F5992.8030309@chruetertee.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wmVhWgj1OZ04es6B" Content-Disposition: inline In-Reply-To: <4D1F5992.8030309@chruetertee.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:46:56 -0000 --wmVhWgj1OZ04es6B Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 01, 2011 at 05:42:58PM +0100, Beat G?tzi wrote: > On 01.01.2011 17:12, Kostik Belousov wrote: > > On Sat, Jan 01, 2011 at 05:00:56PM +0100, Beat G?tzi wrote: > >> On 01.01.2011 16:45, Kostik Belousov wrote: > >>> Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect > >>> they are quite close or equial. If yes, consider increasing maxvnodes. > >>> Another workaround, if you have huge nested directories hierarhy, is > >>> to set vfs.vlru_allow_cache_src to 1. > >> > >> Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: > >> # sysctl kern.maxvnodes vfs.numvnodes > >> kern.maxvnodes: 100000 > >> vfs.numvnodes: 100765 > >> > >> I've increased kern.maxvnodes and the problem was gone until > >> vfs.numvnodes reached the value of kern.maxvnodes again: > >> # sysctl kern.maxvnodes vfs.numvnodes > >> kern.maxvnodes: 150000 > >> vfs.numvnodes: 150109 > > The processes should be stuck in "vlruwk" state, that can be > > checked with ps or '^T' on the terminal. >=20 > Yes, there are various processes in "vlruwk" state, >=20 > >> As the directory structure is quite huge on this server I've set > >> vfs.vlru_allow_cache_src to one now. > > Did it helped ? >=20 > No, it doesn't looks like setting vfs.vlru_allow_cache_src helped. The > problem was gone when I increased kern.maxvnodes until vfs.numvnodes > reached that level. I've stopped all running deamons but numvnodes > doesn't decrease. Stopping the daemons would not decrease the count of cached vnodes. What you can do is to call unmount on the filesystems. Supposedly, the filesystems are busy and unmount shall fail, but it will force freed the vnodes that are unused by any process. --wmVhWgj1OZ04es6B Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0fWnkACgkQC3+MBN1Mb4hnfQCeKuWmedxT+a+27Ez1rmgU8/zh wRgAnA6F8LmqaCJdSDHjoqp+eha/Cfth =ikth -----END PGP SIGNATURE----- --wmVhWgj1OZ04es6B-- From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 16:59:12 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25C2F1065747; Sat, 1 Jan 2011 16:59:12 +0000 (UTC) (envelope-from beat@chruetertee.ch) Received: from marvin.chruetertee.ch (marvin.chruetertee.ch [217.150.245.55]) by mx1.freebsd.org (Postfix) with ESMTP id 5BF318FC17; Sat, 1 Jan 2011 16:59:08 +0000 (UTC) Received: from daedalus.network.local (89-131.77-83.cust.bluewin.ch [83.77.131.89]) (authenticated bits=0) by marvin.chruetertee.ch (8.14.3/8.14.3) with ESMTP id p01Gx7vK050369 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 16:59:07 GMT (envelope-from beat@chruetertee.ch) Message-ID: <4D1F5D5E.8030602@chruetertee.ch> Date: Sat, 01 Jan 2011 17:59:10 +0100 From: =?ISO-8859-1?Q?Beat_G=E4tzi?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.16) Gecko/20101210 Lightning/1.0b1 Thunderbird/3.0.11 MIME-Version: 1.0 To: Kostik Belousov References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> <4D1F4FB8.3030303@chruetertee.ch> <20110101161254.GX90883@deviant.kiev.zoral.com.ua> <4D1F5992.8030309@chruetertee.ch> <20110101164649.GY90883@deviant.kiev.zoral.com.ua> In-Reply-To: <20110101164649.GY90883@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 16:59:12 -0000 On 01.01.2011 17:46, Kostik Belousov wrote: > On Sat, Jan 01, 2011 at 05:42:58PM +0100, Beat G?tzi wrote: >> On 01.01.2011 17:12, Kostik Belousov wrote: >>> On Sat, Jan 01, 2011 at 05:00:56PM +0100, Beat G?tzi wrote: >>>> On 01.01.2011 16:45, Kostik Belousov wrote: >>>>> Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I suspect >>>>> they are quite close or equial. If yes, consider increasing maxvnodes. >>>>> Another workaround, if you have huge nested directories hierarhy, is >>>>> to set vfs.vlru_allow_cache_src to 1. >>>> >>>> Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: >>>> # sysctl kern.maxvnodes vfs.numvnodes >>>> kern.maxvnodes: 100000 >>>> vfs.numvnodes: 100765 >>>> >>>> I've increased kern.maxvnodes and the problem was gone until >>>> vfs.numvnodes reached the value of kern.maxvnodes again: >>>> # sysctl kern.maxvnodes vfs.numvnodes >>>> kern.maxvnodes: 150000 >>>> vfs.numvnodes: 150109 >>> The processes should be stuck in "vlruwk" state, that can be >>> checked with ps or '^T' on the terminal. >> >> Yes, there are various processes in "vlruwk" state, >> >>>> As the directory structure is quite huge on this server I've set >>>> vfs.vlru_allow_cache_src to one now. >>> Did it helped ? >> >> No, it doesn't looks like setting vfs.vlru_allow_cache_src helped. The >> problem was gone when I increased kern.maxvnodes until vfs.numvnodes >> reached that level. I've stopped all running deamons but numvnodes >> doesn't decrease. > Stopping the daemons would not decrease the count of cached vnodes. > What you can do is to call unmount on the filesystems. Supposedly, the > filesystems are busy and unmount shall fail, but it will force freed > the vnodes that are unused by any process. That freed around 1500 vnodes. At the moment the vfs.numvnodes doesn't increase rapidly and the server is usable. I will keep an eye it to see if I run into the same problem again. Thanks for your help! Beat From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 17:26:41 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87828106566C; Sat, 1 Jan 2011 17:26:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1D6E28FC12; Sat, 1 Jan 2011 17:26:40 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p01HQZ4b031663 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 1 Jan 2011 19:26:35 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p01HQZqH076826; Sat, 1 Jan 2011 19:26:35 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p01HQZLO076825; Sat, 1 Jan 2011 19:26:35 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 1 Jan 2011 19:26:35 +0200 From: Kostik Belousov To: Beat G?tzi Message-ID: <20110101172635.GZ90883@deviant.kiev.zoral.com.ua> References: <4D1F1AE8.5040704@chruetertee.ch> <20110101151008.GA7762@freebsd.org> <4D1F4A48.6080604@chruetertee.ch> <20110101154537.GW90883@deviant.kiev.zoral.com.ua> <4D1F4FB8.3030303@chruetertee.ch> <20110101161254.GX90883@deviant.kiev.zoral.com.ua> <4D1F5992.8030309@chruetertee.ch> <20110101164649.GY90883@deviant.kiev.zoral.com.ua> <4D1F5D5E.8030602@chruetertee.ch> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6+qwXIXENytJnbGu" Content-Disposition: inline In-Reply-To: <4D1F5D5E.8030602@chruetertee.ch> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Alexander Best , current@freebsd.org Subject: Re: Suddenly slow lstat syscalls on CURRENT from Juli X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 17:26:41 -0000 --6+qwXIXENytJnbGu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 01, 2011 at 05:59:10PM +0100, Beat G?tzi wrote: > On 01.01.2011 17:46, Kostik Belousov wrote: > > On Sat, Jan 01, 2011 at 05:42:58PM +0100, Beat G?tzi wrote: > >> On 01.01.2011 17:12, Kostik Belousov wrote: > >>> On Sat, Jan 01, 2011 at 05:00:56PM +0100, Beat G?tzi wrote: > >>>> On 01.01.2011 16:45, Kostik Belousov wrote: > >>>>> Check the output of sysctl kern.maxvnodes and vfs.numvnodes. I susp= ect > >>>>> they are quite close or equial. If yes, consider increasing maxvnod= es. > >>>>> Another workaround, if you have huge nested directories hierarhy, is > >>>>> to set vfs.vlru_allow_cache_src to 1. > >>>> > >>>> Thanks for the hint. kern.maxvnodes and vfs.numvnodes were equal: > >>>> # sysctl kern.maxvnodes vfs.numvnodes > >>>> kern.maxvnodes: 100000 > >>>> vfs.numvnodes: 100765 > >>>> > >>>> I've increased kern.maxvnodes and the problem was gone until > >>>> vfs.numvnodes reached the value of kern.maxvnodes again: > >>>> # sysctl kern.maxvnodes vfs.numvnodes > >>>> kern.maxvnodes: 150000 > >>>> vfs.numvnodes: 150109 > >>> The processes should be stuck in "vlruwk" state, that can be > >>> checked with ps or '^T' on the terminal. > >> > >> Yes, there are various processes in "vlruwk" state, > >> > >>>> As the directory structure is quite huge on this server I've set > >>>> vfs.vlru_allow_cache_src to one now. > >>> Did it helped ? > >> > >> No, it doesn't looks like setting vfs.vlru_allow_cache_src helped. The > >> problem was gone when I increased kern.maxvnodes until vfs.numvnodes > >> reached that level. I've stopped all running deamons but numvnodes > >> doesn't decrease. > > Stopping the daemons would not decrease the count of cached vnodes. > > What you can do is to call unmount on the filesystems. Supposedly, the > > filesystems are busy and unmount shall fail, but it will force freed > > the vnodes that are unused by any process. >=20 > That freed around 1500 vnodes. At the moment the vfs.numvnodes doesn't > increase rapidly and the server is usable. I will keep an eye it to see > if I run into the same problem again. This is too small amount of vnodes to be freed for the typical system, and it feels like a real vnode leak. It would be helpful if you tried to identify the load that causes the situation to occur. You are on the UFS, right ? --6+qwXIXENytJnbGu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0fY8sACgkQC3+MBN1Mb4jLOQCfar3tWLQkujs0UZX5YW+OzFJT IzIAn1Oqeuneymr9stKhCWJxgLXd/UwQ =/cZf -----END PGP SIGNATURE----- --6+qwXIXENytJnbGu-- From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 18:47:25 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 516C1106564A; Sat, 1 Jan 2011 18:47:25 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by mx1.freebsd.org (Postfix) with ESMTP id 11BD28FC0C; Sat, 1 Jan 2011 18:47:25 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.4/8.14.4) with ESMTP id p01IlORS009391; Sat, 1 Jan 2011 10:47:24 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.4/8.14.4/Submit) id p01IlLPj009390; Sat, 1 Jan 2011 10:47:21 -0800 (PST) (envelope-from sgk) Date: Sat, 1 Jan 2011 10:47:21 -0800 From: Steve Kargl To: Bernhard Schmidt Message-ID: <20110101184721.GA9383@troutmask.apl.washington.edu> References: <20101226195556.GA45505@troutmask.apl.washington.edu> <201012262325.05784.bschmidt@freebsd.org> <20101227003256.GA46611@troutmask.apl.washington.edu> <201012270205.25867.bschmidt@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201012270205.25867.bschmidt@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org, Etienne Robillard Subject: Re: wlan/wpi are more broken than 3 weeks. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 18:47:25 -0000 On Mon, Dec 27, 2010 at 02:05:25AM +0100, Bernhard Schmidt wrote: > On Monday 27 December 2010 01:32:56 Steve Kargl wrote: > > On Sun, Dec 26, 2010 at 11:25:05PM +0100, Bernhard Schmidt wrote: > > > How about providing the info I asked for last time? Now that you have > > > build the necessary options into the kernel you should be able to run > > > with wlandebug 0xffffffff enabled. > > > > Perhaps, I got busy with real life work, and perhaps, > > I forgot you asked for this info. In the end, /var/log/messages > > get stuff full of > > > > Dec 26 16:30:04 laptop kernel: wlan0: received beacon from > > 00:18:e7:d4:f2:1b rssi 37 Dec 26 16:30:04 laptop kernel: wlan0: received > > beacon from 00:18:e7:d4:f2:1b rssi 38 Dec 26 16:30:04 laptop kernel: > > wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 40 Dec 26 16:30:04 > > laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 34 Dec > > 26 16:30:04 laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b > > rssi 38 Dec 26 16:30:04 laptop kernel: wlan0: received beacon from > > 00:18:e7:d4:f2:1b rssi 38 Dec 26 16:30:04 laptop kernel: wlan0: received > > beacon from 00:18:e7:d4:f2:1b rssi 41 Dec 26 16:30:04 laptop kernel: > > wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 40 Dec 26 16:30:04 > > laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 32 Dec > > 26 16:30:05 laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b > > rssi 34 Dec 26 16:30:05 laptop kernel: wlan0: received beacon from > > 00:18:e7:d4:f2:1b rssi 41 Dec 26 16:30:05 laptop kernel: wlan0: received > > beacon from 00:18:e7:d4:f2:1b rssi 41 Dec 26 16:30:05 laptop kernel: > > wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 40 Dec 26 16:30:05 > > laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 37 Dec > > 26 16:30:05 laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b > > rssi 40 Dec 26 16:30:05 laptop kernel: wlan0: received beacon from > > 00:18:e7:d4:f2:1b rssi 40 Dec 26 16:30:05 laptop kernel: wlan0: received > > beacon from 00:18:e7:d4:f2:1b rssi 32 Dec 26 16:30:05 laptop kernel: > > wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 35 Dec 26 16:30:05 > > laptop kernel: wlan0: received beacon from 00:18:e7:d4:f2:1b rssi 40 > > > > because apparently this message isn't rate limited. So, perhaps, this > > is the problem? > > I don't think so, those messages are usually an indicator for a successful > connection. AP sends those as a 'hello, i'm still here' kinda thing. You > should be able to supress those with 'wlandebug 0xffffffff -dumppkts'. > > If you can get debug output while the UPs/DOWNs happen, that would help a lot. > Sorry about the delay. The laptop has been in Windows land for the last week. I finally have a log where the interface is going UP/DOWN and have wlandebug in effect. It's 220KB. You can find it at http://troutmask.apl.washington.edu/~kargl/wlan0.msg -- Steve From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 22:11:02 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 601841065694; Sat, 1 Jan 2011 22:11:02 +0000 (UTC) (envelope-from alexanderchuranov@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9ACAB8FC0C; Sat, 1 Jan 2011 22:11:01 +0000 (UTC) Received: by eyf6 with SMTP id 6so5681514eyf.13 for ; Sat, 01 Jan 2011 14:11:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=IOckVQ02kIeftOhymFnwOzV0wjdNVvVqPCJBkqdRF+U=; b=w+X0bm6rXn3KIWM5OJg+4awQe3jq6Y+zPvF//i9AvDJSw/g+jLFds1kxnirFudmDzq uoy+5R7GBgim7jt/NCdXqMakhDxpuGSi7HTkMlfGvC5EVQarVqOJXhHPsije+ZU4o2G+ 46mO52UmYy+sVMi+I0h9B6TwOafTJ4dV4LtZA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=RuHCeGSvGHOo/n5HLdRVSfD9Ro4PMN4c3mM9Qv8P9JWxk73KO+u2moS7dy0bOX7vbU QcyKlx8GRqREY/hyxjDWEv6f6ww1gikTHecxGvnz8csnkjT0fT+aZ9aZHiNpQfTOnh7B Q9evE6yJTvsCzIbgZooubPz95yqVHdpZLKCbk= MIME-Version: 1.0 Received: by 10.213.15.144 with SMTP id k16mr7286174eba.45.1293918033675; Sat, 01 Jan 2011 13:40:33 -0800 (PST) Received: by 10.213.7.8 with HTTP; Sat, 1 Jan 2011 13:40:33 -0800 (PST) In-Reply-To: <4D1E812A.5060500@FreeBSD.org> References: <4D1E812A.5060500@FreeBSD.org> Date: Sun, 2 Jan 2011 00:40:33 +0300 Message-ID: From: Alexander Churanov To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, freebsd-ports@freebsd.org Subject: Re: boost libs error X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 22:11:02 -0000 2011/1/1 Doug Barton : > I'm getting the following with qbittorrent-23 which depends on > libtorrent-rasterbar-15 after the latest boost lib update: > > qbittorrent > terminate called after throwing an instance of 'std::runtime_error' > =A0what(): =A0locale::facet::_S_create_c_locale name not valid > Abort trap: 6 (core dumped) Doug, please, check whether you have are observing the issue ports/153561. Alexander Churanov, maintainer of devel/boost-* From owner-freebsd-current@FreeBSD.ORG Sat Jan 1 23:15:31 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E1FA106564A for ; Sat, 1 Jan 2011 23:15:31 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 885658FC08 for ; Sat, 1 Jan 2011 23:15:30 +0000 (UTC) Received: (qmail 13446 invoked by uid 399); 1 Jan 2011 23:15:29 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 1 Jan 2011 23:15:29 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D1FB590.1020805@FreeBSD.org> Date: Sat, 01 Jan 2011 15:15:28 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20101212 Thunderbird/3.1.7 MIME-Version: 1.0 To: Alexander Churanov References: <4D1E812A.5060500@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-ports@freebsd.org Subject: Re: boost libs error X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Jan 2011 23:15:31 -0000 On 01/01/2011 13:40, Alexander Churanov wrote: > 2011/1/1 Doug Barton: >> I'm getting the following with qbittorrent-23 which depends on >> libtorrent-rasterbar-15 after the latest boost lib update: >> >> qbittorrent >> terminate called after throwing an instance of 'std::runtime_error' >> what(): locale::facet::_S_create_c_locale name not valid >> Abort trap: 6 (core dumped) > > Doug, please, check whether you have are observing the issue ports/153561. Yes, that's it precisely! I have my locale set to en_US.UTF-8, and adding the patch in that PR solved the problem for me. Let me know if you'd like me to commit it for you. Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/