From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 11:53:57 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2613CF63 for ; Mon, 4 Aug 2014 11:53:57 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B9EB62001 for ; Mon, 4 Aug 2014 11:53:56 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s74BpWEc028150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 4 Aug 2014 11:51:32 GMT Date: Mon, 4 Aug 2014 14:53:42 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: sysctl hw add nphyscpu variable Message-Id: <20140804145342.2306d7681f0b852908dd4865@systemdatarecorder.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 11:53:57 -0000 Hi, Would be possible to have under sysctl interface a new variable which will track the number of physical CPU sockets a system has ? Would be useful for hardware and data inventory. Something like: hw.machine: amd64 hw.model: Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz hw.ncpu: 32 hw.nphyscpu: 2 ^^^^^^^^^^^^ This is an example from my system with two 2 physical CPUs installed. No system virtualization in place, like Xen, etc ... Or is it possible currently to get easily this information ? I havent found one except dmesg information. FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 8 cpu9 (AP): APIC ID: 9 cpu10 (AP): APIC ID: 10 cpu11 (AP): APIC ID: 11 cpu12 (AP): APIC ID: 12 cpu13 (AP): APIC ID: 13 cpu14 (AP): APIC ID: 14 cpu15 (AP): APIC ID: 15 cpu16 (AP): APIC ID: 32 cpu17 (AP): APIC ID: 33 cpu18 (AP): APIC ID: 34 cpu19 (AP): APIC ID: 35 cpu20 (AP): APIC ID: 36 cpu21 (AP): APIC ID: 37 cpu22 (AP): APIC ID: 38 cpu23 (AP): APIC ID: 39 cpu24 (AP): APIC ID: 40 cpu25 (AP): APIC ID: 41 cpu26 (AP): APIC ID: 42 cpu27 (AP): APIC ID: 43 cpu28 (AP): APIC ID: 44 cpu29 (AP): APIC ID: 45 cpu30 (AP): APIC ID: 46 cpu31 (AP): APIC ID: 47 Probable things will get complicated a bit if the system is a guest under Xen or other hypervisor. But probable there the nphyscpu should be NA ? -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 12:23:18 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F118D2F for ; Mon, 4 Aug 2014 12:23:18 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:214:c2ff:fe64:b2d3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8100C24EE for ; Mon, 4 Aug 2014 12:23:18 +0000 (UTC) Received: from [IPv6:2001:470:1f07:93:225:90ff:fef4:eb0a] (unknown [IPv6:2001:470:1f07:93:225:90ff:fef4:eb0a]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 27E7233FD1B; Mon, 4 Aug 2014 12:23:17 +0000 (UTC) Message-ID: <53E023EC.7010001@gentoo.org> Date: Mon, 04 Aug 2014 20:23:08 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Stefan Parvu Subject: Re: sysctl hw add nphyscpu variable References: <20140804145342.2306d7681f0b852908dd4865@systemdatarecorder.org> In-Reply-To: <20140804145342.2306d7681f0b852908dd4865@systemdatarecorder.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0ex8UDXPAht3R2wKhPqQFTg0F2kMcaOVu" Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 12:23:18 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --0ex8UDXPAht3R2wKhPqQFTg0F2kMcaOVu Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon 04 Aug 2014 07:53:42 AM EDT, Stefan Parvu wrote: > Hi, > > Would be possible to have under sysctl interface a new variable which w= ill track the > number of physical CPU sockets a system has ? Would be useful for hardw= are > and data inventory. It should be. > Something like: > > hw.machine: amd64 > hw.model: Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz > hw.ncpu: 32 > hw.nphyscpu: 2 > ^^^^^^^^^^^^ I think hw.nsocket would make more sense. Whoever does this should=20 check to see if any other UNIX platforms have picked a suitable name. > This is an example from my system with two 2 physical CPUs installed. N= o system > virtualization in place, like Xen, etc ... > > Or is it possible currently to get easily this information ? I havent f= ound one > except dmesg information. > > FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs > FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > cpu2 (AP): APIC ID: 2 > cpu3 (AP): APIC ID: 3 > cpu4 (AP): APIC ID: 4 > cpu5 (AP): APIC ID: 5 > cpu6 (AP): APIC ID: 6 > cpu7 (AP): APIC ID: 7 > cpu8 (AP): APIC ID: 8 > cpu9 (AP): APIC ID: 9 > cpu10 (AP): APIC ID: 10 > cpu11 (AP): APIC ID: 11 > cpu12 (AP): APIC ID: 12 > cpu13 (AP): APIC ID: 13 > cpu14 (AP): APIC ID: 14 > cpu15 (AP): APIC ID: 15 > cpu16 (AP): APIC ID: 32 > cpu17 (AP): APIC ID: 33 > cpu18 (AP): APIC ID: 34 > cpu19 (AP): APIC ID: 35 > cpu20 (AP): APIC ID: 36 > cpu21 (AP): APIC ID: 37 > cpu22 (AP): APIC ID: 38 > cpu23 (AP): APIC ID: 39 > cpu24 (AP): APIC ID: 40 > cpu25 (AP): APIC ID: 41 > cpu26 (AP): APIC ID: 42 > cpu27 (AP): APIC ID: 43 > cpu28 (AP): APIC ID: 44 > cpu29 (AP): APIC ID: 45 > cpu30 (AP): APIC ID: 46 > cpu31 (AP): APIC ID: 47 Whoever implements this would want to find the code that prints this=20 and use the information to initialize the sysctl variable. > Probable things will get complicated a bit if the system is a guest und= er Xen or other > hypervisor. But probable there the nphyscpu should be NA ? The hypervisor is able to pass through a geometry, so this would still=20 make sense there. --0ex8UDXPAht3R2wKhPqQFTg0F2kMcaOVu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJT4CPyAAoJECDuEZm+6ExkQKUP/0jK/02jOFBk0Ax3qF5ObuNF 3gd2LcWXB0TAXc2pthPthTrN8L3NunWXbJ1CzZhrFvsb0OMRAthh4t818T+7E4Ik n/jhvMf+GMT7YOueRTriCOojRiP88MWizT17ugdMxibQuqdl0uicsCCyBE8pcYPe sekdQjJFc33xjIvkMEGe8oVTP+vK461Xv64HUq8Gy2kQEEwlTZSVBNx+xVx84Cus JkpGlP56HCltWZcA83IttHQL09ylsK9z6e9AHBl/aGHttmkwyBv4kSwQ8MJc+ock 8xzQN8jQoDPgaHEo+QwvbmH2XvJyC6TAYDmohue1qca00yu8/nfXypcHBdFMsZk0 ad3FrN3URV8uLBTD9aC+0SNUs04dHBPALU+qCXv2ZXzTJMTea/T/yx+RK6s9zUXW reI1nwrJVLWpiz7R3DD9BBQ/Vunz/BNRRHmSgpxVu5En6Qx97bjsRh1CEnBh+TcP 0VDTqN/owFOAIktFnKveJur3srjTXjkrCokNwGVFX1PWYJ3v85mXmj0PtCpCY6Sr EKsqJ8TMyG65wA0X6dXkT3kdUXp0zn0YwMmP5J8gS1XtPdLCHL+brrXjeX/GSpjZ Ta3ta0gr1MZ6byfAvW/XwtbLGNr3AwOsNB96XRbZ09DxGV+O0ayk9r4/3TZeSL/Y 6JrK1AEns92g3BvbvQwf =J9jx -----END PGP SIGNATURE----- --0ex8UDXPAht3R2wKhPqQFTg0F2kMcaOVu-- From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 13:11:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33209D20 for ; Mon, 4 Aug 2014 13:11:47 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C2C812B03 for ; Mon, 4 Aug 2014 13:11:45 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s74D9Qv1028465 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 4 Aug 2014 13:09:27 GMT Date: Mon, 4 Aug 2014 16:11:37 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: Re: sysctl hw add nphyscpu variable Message-Id: <20140804161137.bb0453ee11197f3db232c645@systemdatarecorder.org> In-Reply-To: <53E023EC.7010001@gentoo.org> References: <20140804145342.2306d7681f0b852908dd4865@systemdatarecorder.org> <53E023EC.7010001@gentoo.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 13:11:47 -0000 > I think hw.nsocket would make more sense. Whoever does this should > check to see if any other UNIX platforms have picked a suitable name. > even better naming: hw.nsocket so would be nice to have it around. Im reading now kern.sched.topology_spec but will require some work and not sure if I understand it correctly. BTW, what is the process to log a new enhancement for sysctl interface: bugzilla ? -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 15:22:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F2EDFEC for ; Mon, 4 Aug 2014 15:22:15 +0000 (UTC) Received: from mail-ig0-x243.google.com (mail-ig0-x243.google.com [IPv6:2607:f8b0:4001:c05::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 642D12EE0 for ; Mon, 4 Aug 2014 15:22:15 +0000 (UTC) Received: by mail-ig0-f195.google.com with SMTP id uq10so1440021igb.2 for ; Mon, 04 Aug 2014 08:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=TJsFEaxN5spuve3bjLXzpdVIsUUOWDwBLWC0GAVyq4I=; b=ukctVv2BV5okX+uWxwU77R0p086B1dKxn1cmuUJaQIoiKzHYA7Oe1Qantcv7DsLsKK IVzFRl5ZRdp50w9DlDtdPdsSZsjA7vTvabuQrQBDj32f+/IvOQPuX+OvljgvUtocoeXS zZrbLOubxdP6cZwiTK6mNXmiS1iFUA+9vphLhEef5FHENyx3olbr+YOKlKd+yo41Gab6 sZRTUPq6sKkDhKVGZi9awGlhC6AcCeJs83slj5EM0YIa+TBclmg3cJkWpF/2u3IW7cwa CKr50sqtM/OUEkpFheNWjWdL0JuXHYE7a8xYrtqcuc4x1SpR4VAwigzKYSIhXgnooxdo DZnA== MIME-Version: 1.0 X-Received: by 10.50.164.202 with SMTP id ys10mr37782734igb.6.1407165734684; Mon, 04 Aug 2014 08:22:14 -0700 (PDT) Received: by 10.64.7.203 with HTTP; Mon, 4 Aug 2014 08:22:14 -0700 (PDT) Date: Mon, 4 Aug 2014 08:22:14 -0700 Message-ID: Subject: Re: sysctl hw add nphyscpu variable From: Dieter BSD To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 15:22:15 -0000 > I think hw.nsocket would make more sense. Whoever does this should > check to see if any other UNIX platforms have picked a suitable name. There are other sockets besides cpu sockets, so it might be a good idea to include something that says what kind of socket. I was thinking n_cpu_socket, which isn't perfect either, since there are other things that can be plugged into cpu sockets. FPGAs for one. Is there a more generic term for cpu socket? >> Probable things will get complicated a bit if the system is a guest >> under Xen or other hypervisor. But probable there the nphyscpu should >> be NA ? > > The hypervisor is able to pass through a geometry, so this would still > make sense there. If you are running bochs on a non-x86 machine (Alpha, Sparc, ARM, ...), you have a virtual cpu plugged into a virtual socket. From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 16:09:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F20CA848 for ; Mon, 4 Aug 2014 16:09:35 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7849F2486 for ; Mon, 4 Aug 2014 16:09:34 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s74G7Fse029271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 4 Aug 2014 16:07:16 GMT Date: Mon, 4 Aug 2014 19:09:23 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: cpu utilization break down via sysctl Message-Id: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 16:09:36 -0000 Hi, Im trying to understand if there is a simple way, without writing C code, to consume sysctl interface via BSD::Sysctl or using sysctl(8) and output user, system and idle across all CPUs from a running system ? Im trying to port sysrec [1] to FreeBSD and it seems a bit harder to find the correct approach. Pointers, comments ? [1] - http://www.systemdatarecorder.org/recording/sysrec.html Many thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 16:21:05 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28E8CF8F for ; Mon, 4 Aug 2014 16:21:05 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D76CB25BA for ; Mon, 4 Aug 2014 16:21:04 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id s74GKuRm009532; Mon, 4 Aug 2014 09:20:56 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id s74GKuAA009531; Mon, 4 Aug 2014 09:20:56 -0700 (PDT) (envelope-from david) Date: Mon, 4 Aug 2014 09:20:56 -0700 From: David Wolfskill To: Stefan Parvu Subject: Re: cpu utilization break down via sysctl Message-ID: <20140804162056.GP1228@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Stefan Parvu , freebsd-hackers@freebsd.org References: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="eAnxKwVhzStH6fSc" Content-Disposition: inline In-Reply-To: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 16:21:05 -0000 --eAnxKwVhzStH6fSc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 04, 2014 at 07:09:23PM +0300, Stefan Parvu wrote: > Hi, >=20 > Im trying to understand if there is a simple way, without writing C code,= to consume > sysctl interface via BSD::Sysctl or using sysctl(8) and output user, syst= em and idle=20 > across all CPUs from a running system ? Sure; I've done the data acquisition with sysctl & awk, then post-processed that with a bit of Perl. I wrote some code in R to generate graphs.... I did some similar stuff using RRDTool a whlie back. > Im trying to port sysrec [1] to FreeBSD and it seems a bit harder to find= the correct > approach. >=20 > Pointers, comments ? >=20 > [1] - http://www.systemdatarecorder.org/recording/sysrec.html > ... kern.cp_time output is an ordered quintuple showing (aggregate) CPU counters for: * user * nice * system * interrupt * idle CPU states. kern.cp_times output is the same information, but repeated for each core on the system. For each core (or the aggregate), you need to look at an interval. Get the differences for each of the 5 possible CPU states; the sum of those differences is the total of CPU resources available during that interval. It's then a fairly straightforward application of arithmetic to determine what proportion of that total is ascribable to each of the states. The only part I think of where things could get "interesting" is if you have one or more counters wrap around. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --eAnxKwVhzStH6fSc Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJT37LnXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7T9wP/jsS72wvAIR/mGJ5C+i5cn1O 2O/zv8gwS0nItPU/yHlUmfhriBqJKF7SPNCRTRulm4BDanVEUKkRyOi5FPdi8jbm SmT5w3EL2tA0elnPsjpAHn7X1OrE/DJPOt5ycnVc75QZ/xdmGqUATlr0LEcCk5F8 QZYUmhn56SiOcv7dWleVOXkKE7g8nHINqwjOYj3bbmjaNvDAdOwpruqxV8aZ+p4B S0nzEt2Qu8mKU2VWpbDCemT7J3rn4MYau1VdCBZZgKepu9ZVTqYYKPo+/dGVWXQU STbm7NL8KiHhrF+dbtSfnjQE45HoFXi0Q09wGoRYMZzFC0xahw1lb6gYVagHiAAD VE7EGjL1UXa4COYEks/t6C55JcemSGAFYbZMEZDdVRcSOSxcUur6oiI3mEXXKn/n HTgzbmb0glQjQzkClq/we4ZenlbPO2Hv3dmnO35aazGuyhJcd2gwszhIIWQfCqH/ 6n10FacacVllSWud0QxrmXdYiXXMDu47IYUKK+jMucJl8dI4XUv9hRQQOT+PTCCQ pVhVezPczIhE1LlHEdIjAZdgWtl1ofciEBjGwGSdxmltqpvTDgmU7sq1sZBwh8ys Uy1qhAngnfNXairu8tV0sefN6BEubkynpvzMoYJzoaxXZHY3zI3t6llcTdF9hA2a SF5/2CkPmR+kebIe/DMq =lVye -----END PGP SIGNATURE----- --eAnxKwVhzStH6fSc-- From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 16:41:11 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 159408B0 for ; Mon, 4 Aug 2014 16:41:11 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A403B2876 for ; Mon, 4 Aug 2014 16:41:09 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s74GcPbg029395 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 4 Aug 2014 16:38:26 GMT Date: Mon, 4 Aug 2014 19:40:36 +0300 From: Stefan Parvu To: David Wolfskill Subject: Re: cpu utilization break down via sysctl Message-Id: <20140804194036.be325ee5822bbcb2623db358@systemdatarecorder.org> In-Reply-To: <20140804162056.GP1228@albert.catwhisker.org> References: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> <20140804162056.GP1228@albert.catwhisker.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 16:41:11 -0000 > Sure; I've done the data acquisition with sysctl & awk, then > post-processed that with a bit of Perl. I wrote some code in R to > generate graphs.... I did some similar stuff using RRDTool a whlie > back. Im planning in using BSD::Sysctl to fetch all needed values. > kern.cp_time output is an ordered quintuple showing (aggregate) CPU > counters for: > > * user > * nice > * system > * interrupt > * idle > I see. Cool. If I understood right these will be the break down for 4 vcpu system: $ sysctl kern.cp_times kern.cp_times: 28845 2060 22253 1805 1197920 27677 1477 22399 2109 1199180 32342 1649 19499 894 1198458 29670 1678 20324 1471 1199699 which maps to: time1: cpu0: user_28845: nice_2060: system_22253: intr_1805: idle_1197920 time1: cpu1: user_27677: nice_1477 system_22399 intr_2109: idle_1199180 time1: cpu2: user_32342: nice_1649: system_19499: intr_894: idle_1198458 time1: cpu3 user_29670: nice_1678: system_20324: intr_1471: idle_1199699 > For each core (or the aggregate), you need to look at an interval. Get > the differences for each of the 5 possible CPU states; the sum of those > differences is the total of CPU resources available during that > interval. Right I need to do delta between 2 readings to get the correct values. Same as in KSTAT in Solaris perl interface. One more check here: CPU_Utilization = User + System + Nice + Intr ? Many thanks indeed. I need to experiment with these ... -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 16:53:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FAB3E15 for ; Mon, 4 Aug 2014 16:53:51 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3AAF52AB1 for ; Mon, 4 Aug 2014 16:53:50 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id s74GrosT009705; Mon, 4 Aug 2014 09:53:50 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id s74GroT7009704; Mon, 4 Aug 2014 09:53:50 -0700 (PDT) (envelope-from david) Date: Mon, 4 Aug 2014 09:53:50 -0700 From: David Wolfskill To: Stefan Parvu Subject: Re: cpu utilization break down via sysctl Message-ID: <20140804165350.GQ1228@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Stefan Parvu , freebsd-hackers@freebsd.org References: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> <20140804162056.GP1228@albert.catwhisker.org> <20140804194036.be325ee5822bbcb2623db358@systemdatarecorder.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="5KxTQ9fdN6Op3ksq" Content-Disposition: inline In-Reply-To: <20140804194036.be325ee5822bbcb2623db358@systemdatarecorder.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 16:53:51 -0000 --5KxTQ9fdN6Op3ksq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 04, 2014 at 07:40:36PM +0300, Stefan Parvu wrote: > > Sure; I've done the data acquisition with sysctl & awk, then > > post-processed that with a bit of Perl. I wrote some code in R to > > generate graphs.... I did some similar stuff using RRDTool a whlie > > back. >=20 > Im planning in using BSD::Sysctl to fetch all needed values. For my data acquisition, I wasn't assuming that Perl would be available -- I therefore intentionally restricted myself to tools in base FreeBSD. Your situation is, of course, almost certainly different. :-) > ... > I see. Cool. If I understood right these will be the break down for 4 vcp= u system: >=20 > $ sysctl kern.cp_times > kern.cp_times: 28845 2060 22253 1805 1197920 27677 1477 22399 2109 119918= 0 32342 1649 19499 894 1198458 29670 1678 20324 1471 1199699 >=20 > which maps to: >=20 > time1: cpu0: user_28845: nice_2060: system_22253: intr_1805: idle_1197920 > time1: cpu1: user_27677: nice_1477 system_22399 intr_2109: idle_1199180= =20 > time1: cpu2: user_32342: nice_1649: system_19499: intr_894: idle_1198458 > time1: cpu3 user_29670: nice_1678: system_20324: intr_1471: idle_1199699 Err... Well... Maybe? The numeric values are the values of the counters at the time the information was acquired. I'd probably depict them more as: user nice system intr idle cpu0: 28845 2060 22253 1805 1197920 cpu1: 27677 1477 22399 2109 1199180 cpu2: 32342 1649 19499 894 1198458 cpu3: 29670 1678 20324 1471 1199699 > .. > Right I need to do delta between 2 readings to get the correct values. Yes. > Same as in KSTAT in Solaris perl interface.=20 It's been a long time since I did anything with Solaris. :-} > One more check here:=20 >=20 > CPU_Utilization =3D User + System + Nice + Intr ?=20 If you mean total non-idle CPU, yes. For determining available CPU, you need to include idle. > Many thanks indeed. I need to experiment with these ... Glad to help; the source for top(1) can also be instructive. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --5KxTQ9fdN6Op3ksq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJT37qcXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7UEEP/AsRYLCI3uOwWJalt/M+joZa 3xO+rNOlcyAIO7655iP8BDo0nZPZeCvtRAWVdT3QEmz3iHDTINPXHNtHu55R5CzC J/FmlMA7F8C/mEay9tZwm18KxZ5aXP1A0g1a/xGLHk2OgIphOzZlZM32RgCujqQZ EY9qF3ifkV44LSfKBA+tE8TsbtOljcSBTsLkh8t8F2I4dn6DGjm7ezBISEN7UsJS vh++zgEAUeV701Q88zlxwfZX9Yxg5KLrEtM1DQfaMa730U4DDut5oUryHi11DbEx HC7r3Ydlf5ClQbn03vgpAAu/IBHqQqVTwEbbPv6WI80v1QHMWKRu7BI01MjIYk1e FBplecrFn5z5OlkCwIYfue3/kFUHyAJkGtmlIu/HBMMv5AfsJnzDgJ48h/xgnSxR Wmty4s4CWRYlj/RJqgLcSfwPRVAxysh2DUF+UWdCRMgNasgMakx4BYOipFuedc/Z NxpjFfGZ4Jm25y/AfcfKP9GS1dDI1HcwQ/69iBIfC3xNNhB5oxy0RGobQwDKGB7O 8jxAXScUwpBURnaoQm6IYp+vDCtgecmiI1L3jyjml1u5cycd2A4d56kdz1pYlh3g h3KWGmSFy4gt7lKfzv+vM+MK4Ek+r5jz+BZwp1KNOFbe3xgE7zN9aO4zGZaRc8e3 2oFpNjUwMqK9bajvF9XH =2Ifl -----END PGP SIGNATURE----- --5KxTQ9fdN6Op3ksq-- From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 17:03:39 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39A76127 for ; Mon, 4 Aug 2014 17:03:39 +0000 (UTC) Received: from nm22-vm1.access.bullet.mail.bf1.yahoo.com (nm22-vm1.access.bullet.mail.bf1.yahoo.com [216.109.115.144]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D92C02C72 for ; Mon, 4 Aug 2014 17:03:38 +0000 (UTC) Received: from [66.196.81.155] by nm22.access.bullet.mail.bf1.yahoo.com with NNFMP; 04 Aug 2014 17:00:16 -0000 Received: from [66.196.81.128] by tm1.access.bullet.mail.bf1.yahoo.com with NNFMP; 04 Aug 2014 17:00:16 -0000 Received: from [127.0.0.1] by omp1004.access.mail.bf1.yahoo.com with NNFMP; 04 Aug 2014 17:00:16 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 936239.25161.bm@omp1004.access.mail.bf1.yahoo.com Received: (qmail 19881 invoked by uid 60001); 4 Aug 2014 17:00:16 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1407171616; bh=jCPnWh2gj5e5EbBV1LEV1emIyNhv8uKiQxdRfE/GgDY=; h=Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=o5+l4mnRuMfJ6A7iTMpPLTv1OLtOhFy4sIUtYYJd7pMPViIWrMOIWKX9a7JG6Wg0M5/odOEu+duUoZpJJqzx4aY2iqhJiGLWOwHIjVTluw5cxueo7RX9QC3/RKWv19K+ikFwGPg0M/SIpcns1aBVYXLvC8ynuj4ljhKVYm2VfVM= X-YMail-OSG: r1E..j0VM1nqfQfPirRF5wAcG.YnypHP9LwoGnXSRleBjIe lSlN1tC4Le3DCKiJkmGFomE7hdD_lTLiWfOTOHxVbi9O7Xz5o5NrYc0bwMvh YFJqiIML0VaNOI3fscx6WvKrZJd007Gt9yXmHanON3S09TxaJmnWTt9DDj7X usxLkDhPf5BlB0Rh3R5pihPuxP1XASUlm_J5ngbmmxZZXmmckD6t8s_1GuDo VBL2M2meMeTz3qrfcSw2Ysv5xNum3kl81ubdxQprr8JuJFTbfIgUi2OdKqD4 ZYhmV4NAySivlwmjFflYhPqzLdlxxcA0NA4wJtgEl0qniVUuRj8Eyss2SvTt YO_I6xXTrGh1Yq_dJU2.IWh6nyGl2NThKX1.K4VjaTEtw3QVuCs1YPY3g0cZ LuEfKBcmyfrvs9DCQ3mwcESm6OCIvFeRhiw8ZESSvvMXnSYNkR4m.wMvc6Rz CNHs2IomrWSXD_CeP5C1kyaZLTPc3fm2TG9bNTVl5E2I3XEhLtEuygg8sHD_ .Pzv3A0wFHNxm8TTcNKYZJjaWXqdMgdQ5OMkgYUF0tpRl_VoIPiBjtA-- Received: from [76.211.235.171] by web181702.mail.ne1.yahoo.com via HTTP; Mon, 04 Aug 2014 10:00:16 PDT X-Rocket-MIMEInfo: 002.001, SGVsbG8sDQoNCkZyZWVCU0QgMTAgcmVsZWFzZSBzb3VyY2VzIGRvZXNuJ3Qgc2VlbSB0byBoYXZlIHplcm8gY29weSBzb2NrZXQgY29kZSBhbnltb3JlLiBXaGF0J3MgaXMgYWx0ZXJuYXRpdmUgdG8gZG8gemVyb19jb3B5ID8NCg0KVGhhbmtzLA0KU3VzaGFudGgNCgEwAQEBAQ-- X-Mailer: YahooMailClassic/698 YahooMailWebService/0.8.198.689 Message-ID: <1407171616.44440.YahooMailBasic@web181702.mail.ne1.yahoo.com> Date: Mon, 4 Aug 2014 10:00:16 -0700 From: Sushanth Rai Subject: Support for zero copy sockets To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 17:03:39 -0000 Hello, FreeBSD 10 release sources doesn't seem to have zero copy socket code anymore. What's is alternative to do zero_copy ? Thanks, Sushanth From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 18:40:17 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EE62C5A for ; Mon, 4 Aug 2014 18:40:17 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DC7A527FB for ; Mon, 4 Aug 2014 18:40:16 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s74IbcHj029820 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 4 Aug 2014 18:37:38 GMT Date: Mon, 4 Aug 2014 21:39:48 +0300 From: Stefan Parvu To: David Wolfskill Subject: Re: cpu utilization break down via sysctl Message-Id: <20140804213948.4d1d2eb86b59e4a7e28c838b@systemdatarecorder.org> In-Reply-To: <20140804165350.GQ1228@albert.catwhisker.org> References: <20140804190923.95fec6a00d175ffee89c4af5@systemdatarecorder.org> <20140804162056.GP1228@albert.catwhisker.org> <20140804194036.be325ee5822bbcb2623db358@systemdatarecorder.org> <20140804165350.GQ1228@albert.catwhisker.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 18:40:17 -0000 > Err... Well... Maybe? The numeric values are the values of the counters > at the time the information was acquired. > > I'd probably depict them more as: > > user nice system intr idle > cpu0: 28845 2060 22253 1805 1197920 > cpu1: 27677 1477 22399 2109 1199180 > cpu2: 32342 1649 19499 894 1198458 > cpu3: 29670 1678 20324 1471 1199699 right. > If you mean total non-idle CPU, yes. For determining available CPU, you > need to include idle. > super. Many thanks again. I will put together these and post my progress. I will do first the CPU utilization, then go to mem, disk and network io utilizations. I need as well the get the run-queue length, load average seems very simple. Not sure how hairy things will get on those parts when I will need per disk or per NIC throughput iops or read KB/sec. One thing at the time. Cheers, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 4 21:56:44 2014 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F004F9F9; Mon, 4 Aug 2014 21:56:44 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B39BE211E; Mon, 4 Aug 2014 21:56:44 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 07773B8057; Mon, 4 Aug 2014 23:56:42 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id E8EC028497; Mon, 4 Aug 2014 23:56:41 +0200 (CEST) Date: Mon, 4 Aug 2014 23:56:41 +0200 From: Jilles Tjoelker To: Baptiste Daroussin Subject: Re: Add ohash (OpenBSD hash) into libutil Message-ID: <20140804215641.GA80850@stack.nl> References: <20140727230110.GI50802@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140727230110.GI50802@ivaldir.etoilebsd.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org, hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 21:56:45 -0000 On Mon, Jul 28, 2014 at 01:01:10AM +0200, Baptiste Daroussin wrote: > If noone objects I would like to bring ohash from OpenBSD into our > libutil. > Ohash is already in base (usr.bin/m4/ohash) having it into libutil > will increase compatibility with OpenBSD as well as allowing the rest > of the base system to be able to benefit from ohash implementation. Following OpenBSD's general low priority for binary compatibility issues, the functions have the application allocate struct ohash. For use in a symbol-versioned library that promises backwards compatibility, it would be better to leave the type incomplete in the public header and have the library allocate it. Adding these functions encourages using open addressing (no chaining) because that's what's available. This may be OK but it might be bad in specific situations. The automatic resizing of the hash table, absent in many home-grown hash table implementations, may help avoid such problems somewhat. The code in ohash_resize() that may set the hash table size to UINT_MAX violates the requirement that the probe interval (incr) must be relative prime to the hash table size (otherwise ohash_lookup_interval() might loop indefinitely examining only a subset of the table, while the rest of the table still has free space). Normally, the code enforces this requirement by making the hash table size a power of two and the probe interval an odd number. On another note, if the hash table size is always a power of two, the slow hv % h->size can be replaced by hv & (h->size - 1). The incr should probably only be calculated when needed, or using a method that does not involve dividing by a variable. (Note that hv doesn't contain sufficient bits for an independent i and hv if the hash table is larger than 65536 entries, so such large hash tables will be used less uniformly.) Utilities that value startup and fork performance (such as sh(1)) do not want to depend on additional DSOs (why is libutil separate from libc?). -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 11:53:29 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6E42EB0 for ; Wed, 6 Aug 2014 11:53:29 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A8282D43 for ; Wed, 6 Aug 2014 11:53:28 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s76Bp1wk007890 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Aug 2014 11:51:02 GMT Date: Wed, 6 Aug 2014 14:53:14 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: memory utilization questions Message-Id: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 11:53:30 -0000 Im planning to calculate real used, free memory. Can anyone confirm if these are correct: 1. real used memory usedmem = totalmem - freemem, totalmem = vm.stats.vm.v_page_count x page_size freemem = vm.stats.vm.v_free_count x page_size 2. real free memory realfree = inactive + cache + free ? inactive = vm.stats.vm.v_inactive_count x page_size cache = vm.stats.vm.v_cache_count x page_size free = vm.stats.vm.v_free_count x page_size Thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 12:38:13 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8005F26 for ; Wed, 6 Aug 2014 12:38:13 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9560E22D9 for ; Wed, 6 Aug 2014 12:38:13 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id s76CcCpv023215; Wed, 6 Aug 2014 05:38:12 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id s76CcB5u023214; Wed, 6 Aug 2014 05:38:11 -0700 (PDT) (envelope-from david) Date: Wed, 6 Aug 2014 05:38:11 -0700 From: David Wolfskill To: Stefan Parvu Subject: Re: memory utilization questions Message-ID: <20140806123811.GB1228@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Stefan Parvu , freebsd-hackers@freebsd.org References: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="w8QetmdSDwT395M8" Content-Disposition: inline In-Reply-To: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 12:38:14 -0000 --w8QetmdSDwT395M8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 06, 2014 at 02:53:14PM +0300, Stefan Parvu wrote: >=20 > Im planning to calculate real used, free memory. Can anyone confirm if th= ese > are correct: >=20 > 1. real used memory >=20 > usedmem =3D totalmem - freemem, >=20 > totalmem =3D vm.stats.vm.v_page_count x page_size > freemem =3D vm.stats.vm.v_free_count x page_size >=20 >=20 > 2. real free memory >=20 > realfree =3D inactive + cache + free ? >=20 > inactive =3D vm.stats.vm.v_inactive_count x page_size > cache =3D vm.stats.vm.v_cache_count x page_size > free =3D vm.stats.vm.v_free_count x page_size > ... In what I have: * I assume(!) that page size is invariant during any interval I care about. * I do the calulations in KB (1024B), so I get the hw.pagesize OID & integer-divide that by 1024 to get a page_to_kb factor (e.g., for 4K pagesize, the factor is 4). * Other OIDs: vm.stats.vm.v_active_count, vm.stats.vm.v_inactive_count, vm.stats.vm.v_wire_count, vm.stats.vm.v_cache_count, vm.stats.vm.v_free_count; I multiply each of these by the "page_to_kb factor" determined above (to get results in KB). * Another OID: vfs.bufspace; I integer-divide this by 1024 (to get results in KB), as it seems to be in bytes vs. pages. * Correspondence between names as used by top(1) vs. OID: Active: vm.stats.vm.v_active_count Inact: vm.stats.vm.v_inactive_count Wired: vm.stats.vm.v_wire_count Cache: vm.stats.vm.v_cache_count Buf: vfs.bufspace Free: vm.stats.vm.v_free_count Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --w8QetmdSDwT395M8 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJT4iGyXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7gTcQAISqsRdBbeUlTRSSTQxkOIjc ZriNv0WQcGez8bh2S1BL492zCPs+bJn19hCaPbFTwR3GLJJE0PKLl1nC9cXgc3pg ofkeKfcvsxg+UKMSmwi5gaza7wPtpQkSwHrSERBttLueu73fEVRSiecJNKFs/ka+ cefmHNXDfgepaLWgvM62XkBSy9rJfKakyxBaIabkJZ+s63g8RbbKtvpOv9T3x8GB RHIPPFsZjelINe1lNQQ6GT6LSW73Kp+0igoHTyifBN7G+LeAwycF3nAU6LewnH+K GnitAE10SFzKZtRG6M8N7nIT0UKI60jIHKw4DkWWTwXIDPdNIZLxTrih/CYyGePH WWrji/twlNLolKlJSyDyKlJdRhXRAKtZY4/WEwGQAdvOxTkEoyqFDTD4Kv8UH7wL +FiY+tBBjUIk7M23IQwKjXv06uXDmZtDh67c67v1b/FbM4PD/nXi3QzanjaPFjiq d6iqQKK3Tm4abTE9tPdNgY1QnN2li883m2Ok7g7PYZsFKMk2IwCj+YaC1oK27HL5 4ofkHb5CbZKaDmhZ+buGmDhh8yIRug8FmEsZJ3l24S4NkT3lcGdE1xJu7HKrDpQZ UfIhj0WLyrLURVLVISnahQesTzIhsE1xC88gJwbNiP2tUpL4GtmpRCN7FyuHL87w eXK3enzdUtNI93WBJNnx =zp6l -----END PGP SIGNATURE----- --w8QetmdSDwT395M8-- From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 13:00:34 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFA6A68C for ; Wed, 6 Aug 2014 13:00:34 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AEBA25F2 for ; Wed, 6 Aug 2014 13:00:33 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s76Cvt4V008144 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Aug 2014 12:57:56 GMT Date: Wed, 6 Aug 2014 16:00:08 +0300 From: Stefan Parvu To: David Wolfskill Subject: Re: memory utilization questions Message-Id: <20140806160008.40ba39ce85ff59e059a1005e@systemdatarecorder.org> In-Reply-To: <20140806123811.GB1228@albert.catwhisker.org> References: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> <20140806123811.GB1228@albert.catwhisker.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 13:00:34 -0000 > * Other OIDs: vm.stats.vm.v_active_count, vm.stats.vm.v_inactive_count, > vm.stats.vm.v_wire_count, vm.stats.vm.v_cache_count, > vm.stats.vm.v_free_count; I multiply each of these by the "page_to_kb > factor" determined above (to get results in KB). > > * Another OID: vfs.bufspace; I integer-divide this by 1024 (to get > results in KB), as it seems to be in bytes vs. pages. > > * Correspondence between names as used by top(1) vs. OID: > Active: vm.stats.vm.v_active_count > Inact: vm.stats.vm.v_inactive_count > Wired: vm.stats.vm.v_wire_count > Cache: vm.stats.vm.v_cache_count > Buf: vfs.bufspace > Free: vm.stats.vm.v_free_count super. I need now to do as well the memory stats. Many thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 14:12:08 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADFE7A9B for ; Wed, 6 Aug 2014 14:12:08 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id 79799206B for ; Wed, 6 Aug 2014 14:12:08 +0000 (UTC) Received: from marvin.beer.town (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 6999456444 for ; Wed, 6 Aug 2014 09:12:07 -0500 (CDT) Message-ID: <53E237B6.4040703@vangyzen.net> Date: Wed, 06 Aug 2014 10:12:06 -0400 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Sanity Check: Bogus(?) General Protection Fault Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 14:12:08 -0000 Can someone give me a quick sanity check? I'm debugging a General Protection Fault on an amd64 system. The faulting instruction appears to be an immediate mov into %r11...right? I ask because I can't imagine how that instruction could cause a GPF. Any ideas? Thanks, Eric ==== Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0xffffffff805d6e23 stack pointer = 0x28:0xfffffe0463d45660 frame pointer = 0x28:0xfffffe0463d456d0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 db> bt Tracing pid 1017 tid 100169 td 0xfffff801d4500000 vm_reserv_alloc_contig() at vm_reserv_alloc_contig+0x3b3/frame 0xfffffe0463d456d0 vm_page_alloc_contig() at vm_page_alloc_contig+0x13b/frame 0xfffffe0463d45760 kmem_alloc_contig() at kmem_alloc_contig+0x12f/frame 0xfffffe0463d45810 contigmalloc() at contigmalloc+0x33/frame 0xfffffe0463d45850 [snip] 0xffffffff805d6e0f : mov 0x30(%rax),%r9 0xffffffff805d6e13 : shr $0x15,%r9 0xffffffff805d6e17 : shl $0x6,%r9 0xffffffff805d6e1b : mov 0xffffffff809bf7a8,%r11 0xffffffff805d6e23 : mov %rcx,0x10(%r11,%r9,1) From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 14:48:43 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A01F2B3 for ; Wed, 6 Aug 2014 14:48:43 +0000 (UTC) Received: from mail-we0-f178.google.com (mail-we0-f178.google.com [74.125.82.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 12D8F25D7 for ; Wed, 6 Aug 2014 14:48:42 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id w61so2834339wes.9 for ; Wed, 06 Aug 2014 07:48:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=lpRkL1Bl+QKgZ9uxLQ71YEEtiruBv5OrwfHU+oL4ETI=; b=ZLFZRdNETAlUU++8OVGFlOivl9MG/zMUXCCPNLDrPaAJruyaPRLJUheNlwluY+2lYF jWOJXl4TruZBtE2aHQwmQW0yr5phosSKST5s3+Ysc2FZxTBHz5Lc6PXY25KhbrAfA+ES QB5nOqCl0BYNcy27gfR74m9JWN8newd0QL/XHU9sFIh0rjzUEhH87D3IfTTNfrcQrYMf 5E0VNvmoyFpY33kI215/dMm6uEYePFuGzKBD3cKxLcomdmUi8Q13ZI7tzi6GIxAjXl7/ z+YlvhgVaLoHCSRqtwFHnwad5B8ZgY94nanyh4/ywnmmM4XXHID4EJmo2qt+luT6RU4Q nt6g== X-Gm-Message-State: ALoCoQkDX7Jk+miJCrrJaTS06hwivncWKE9h+Bq21QXxqDE9VKikM6ZoftndiZDLi3zmGKSEaiYN X-Received: by 10.194.122.73 with SMTP id lq9mr15960893wjb.133.1407336514900; Wed, 06 Aug 2014 07:48:34 -0700 (PDT) Received: from localhost (054142ff.skybroadband.com. [5.65.66.255]) by mx.google.com with ESMTPSA id dc3sm2975402wjc.27.2014.08.06.07.48.33 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Aug 2014 07:48:34 -0700 (PDT) Date: Wed, 6 Aug 2014 15:48:33 +0100 From: Matt Fleming To: Eric van Gyzen Subject: Re: Sanity Check: Bogus(?) General Protection Fault Message-ID: <20140806144833.GE15082@console-pimps.org> References: <53E237B6.4040703@vangyzen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53E237B6.4040703@vangyzen.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 14:48:43 -0000 On Wed, 06 Aug, at 10:12:06AM, Eric van Gyzen wrote: > Can someone give me a quick sanity check? I'm debugging a General > Protection Fault on an amd64 system. The faulting instruction appears > to be an immediate mov into %r11...right? I ask because I can't imagine > how that instruction could cause a GPF. Any ideas? > > Thanks, > > Eric > > ==== > > Fatal trap 9: general protection fault while in kernel mode > cpuid = 0; apic id = 00 > instruction pointer = 0x20:0xffffffff805d6e23 [...] > 0xffffffff805d6e23 : mov %rcx,0x10(%r11,%r9,1) This is your faulting instruction. -- Matt Fleming, Intel Open Source Technology Center From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 15:55:30 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5FB59E1 for ; Wed, 6 Aug 2014 15:55:30 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id 73ECC2F5E for ; Wed, 6 Aug 2014 15:55:30 +0000 (UTC) Received: from marvin.beer.town (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 5EEA756444; Wed, 6 Aug 2014 10:55:29 -0500 (CDT) Message-ID: <53E24FF0.7030305@vangyzen.net> Date: Wed, 06 Aug 2014 11:55:28 -0400 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Sanity Check: Bogus(?) General Protection Fault References: <53E237B6.4040703@vangyzen.net> <20140806144833.GE15082@console-pimps.org> In-Reply-To: <20140806144833.GE15082@console-pimps.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Matt Fleming X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 15:55:30 -0000 On 08/06/2014 10:48, Matt Fleming wrote: > On Wed, 06 Aug, at 10:12:06AM, Eric van Gyzen wrote: >> Can someone give me a quick sanity check? I'm debugging a General >> Protection Fault on an amd64 system. The faulting instruction appears >> to be an immediate mov into %r11...right? I ask because I can't imagine >> how that instruction could cause a GPF. Any ideas? >> >> Thanks, >> >> Eric >> >> ==== >> >> Fatal trap 9: general protection fault while in kernel mode >> cpuid = 0; apic id = 00 >> instruction pointer = 0x20:0xffffffff805d6e23 > [...] > >> 0xffffffff805d6e23 : mov %rcx,0x10(%r11,%r9,1) > This is your faulting instruction. Thanks, Matt. That has always been my understanding (and I just found the docs to confirm). I doubted myself because the problem is now even more bizarre. The mov before the faulting instruction apparently didn't complete. %r11 is still an old value, not 0x....f7a8. Any ideas, anyone? Eric ==== 0xffffffff805d6e0f : mov 0x30(%rax),%r9 0xffffffff805d6e13 : shr $0x15,%r9 0xffffffff805d6e17 : shl $0x6,%r9 0xffffffff805d6e1b : mov 0xffffffff809bf7a8,%r11 0xffffffff805d6e23 : mov %rcx,0x10(%r11,%r9,1) db> show reg cs 0x20 ds 0x3b es 0x3b003b fs 0x1b0013 gs 0x1b ss 0x28 rax 0xfffff8043efd6980 rcx 0xfffff80423501000 rdx 0xfffff80423501000 rbx 0xffffffffffdce222 rsp 0xfffffe0463d45660 rbp 0xfffffe0463d456d0 rsi 0xffffffff80a0f898 kernel_object_store+0xb0 rdi 0xffffffff80a0f7e8 kernel_object_store r8 0 r9 0x1fffff0087dc0 r10 0 r11 0xfffff80423501000 r12 0xfffff80430b86980 r13 0x707e00 r14 0 r15 0 rip 0xffffffff805d6e23 vm_reserv_alloc_contig+0x3b3 rflags 0x10206 vm_reserv_alloc_contig+0x3b3: movq %rcx,0x10(%r11,%r9,1) From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 16:18:00 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01A216DE for ; Wed, 6 Aug 2014 16:18:00 +0000 (UTC) Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com [IPv6:2607:f8b0:4003:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C159D21D8 for ; Wed, 6 Aug 2014 16:17:59 +0000 (UTC) Received: by mail-ob0-f173.google.com with SMTP id vb8so1975533obc.32 for ; Wed, 06 Aug 2014 09:17:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZmS6Q6Uh7+5ghz4O5OxIkhxSOaH5ig0zE40w55gBoOo=; b=w7AiaV9fxhSSaBlhiAXQVMgrEioVlQtgKZzkfbqXOeVwBv6mS4Uuzd1LuBOSBncDxV 8wwDumNDziAEOD9qim4Cyw8JnfR8MMKuWj6e9ZfHCPDTICOZOfZnpXMY8x0ni3VPmphD ppNjVESmtQGSkH8XLeIOQgTeDVESbf9Mb5agI/lo+2NSb3ugvfK1fN8pyw6wWyxXDFsO egaK3hwyAfFaQ42XWhJMP8+OE7UCuCCoWyzVPjNsVBwHrzJcD2K0ir+N8kJKMXBWVrDF GyQru8MGtEvMgX31dAIQSO+is47yLLRQoZsonSr5nAfRJzndIa5xtT5yxSv3Xk/IUpzJ FTGA== MIME-Version: 1.0 X-Received: by 10.182.125.103 with SMTP id mp7mr16797130obb.52.1407341879099; Wed, 06 Aug 2014 09:17:59 -0700 (PDT) Received: by 10.182.216.197 with HTTP; Wed, 6 Aug 2014 09:17:58 -0700 (PDT) In-Reply-To: <53E24FF0.7030305@vangyzen.net> References: <53E237B6.4040703@vangyzen.net> <20140806144833.GE15082@console-pimps.org> <53E24FF0.7030305@vangyzen.net> Date: Wed, 6 Aug 2014 18:17:58 +0200 Message-ID: Subject: Re: Sanity Check: Bogus(?) General Protection Fault From: Oliver Pinter To: Eric van Gyzen Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Matt Fleming X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 16:18:00 -0000 Try to remove CPUTYPE?=foo from make.conf. On 8/6/14, Eric van Gyzen wrote: > On 08/06/2014 10:48, Matt Fleming wrote: >> On Wed, 06 Aug, at 10:12:06AM, Eric van Gyzen wrote: >>> Can someone give me a quick sanity check? I'm debugging a General >>> Protection Fault on an amd64 system. The faulting instruction appears >>> to be an immediate mov into %r11...right? I ask because I can't imagine >>> how that instruction could cause a GPF. Any ideas? >>> >>> Thanks, >>> >>> Eric >>> >>> ==== >>> >>> Fatal trap 9: general protection fault while in kernel mode >>> cpuid = 0; apic id = 00 >>> instruction pointer = 0x20:0xffffffff805d6e23 >> [...] >> >>> 0xffffffff805d6e23 : mov >>> %rcx,0x10(%r11,%r9,1) >> This is your faulting instruction. > > Thanks, Matt. That has always been my understanding (and I just found > the docs to confirm). I doubted myself because the problem is now even > more bizarre. The mov before the faulting instruction apparently didn't > complete. %r11 is still an old value, not 0x....f7a8. > > Any ideas, anyone? > > Eric > > ==== > > 0xffffffff805d6e0f : mov 0x30(%rax),%r9 > 0xffffffff805d6e13 : shr $0x15,%r9 > 0xffffffff805d6e17 : shl $0x6,%r9 > 0xffffffff805d6e1b : mov > 0xffffffff809bf7a8,%r11 > 0xffffffff805d6e23 : mov > %rcx,0x10(%r11,%r9,1) > > db> show reg > cs 0x20 > ds 0x3b > es 0x3b003b > fs 0x1b0013 > gs 0x1b > ss 0x28 > rax 0xfffff8043efd6980 > rcx 0xfffff80423501000 > rdx 0xfffff80423501000 > rbx 0xffffffffffdce222 > rsp 0xfffffe0463d45660 > rbp 0xfffffe0463d456d0 > rsi 0xffffffff80a0f898 kernel_object_store+0xb0 > rdi 0xffffffff80a0f7e8 kernel_object_store > r8 0 > r9 0x1fffff0087dc0 > r10 0 > r11 0xfffff80423501000 > r12 0xfffff80430b86980 > r13 0x707e00 > r14 0 > r15 0 > rip 0xffffffff805d6e23 vm_reserv_alloc_contig+0x3b3 > rflags 0x10206 > vm_reserv_alloc_contig+0x3b3: movq %rcx,0x10(%r11,%r9,1) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 16:26:56 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF2CEF40 for ; Wed, 6 Aug 2014 16:26:56 +0000 (UTC) Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com [74.125.82.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44FA623F0 for ; Wed, 6 Aug 2014 16:26:55 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id x12so2873111wgg.4 for ; Wed, 06 Aug 2014 09:26:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=+78RaMKm3sI1Tu3gE0eoCGtdD+2BFNmewrvpZ49kQAw=; b=C2d/x8mrm076Ya2miqySqW2/d53npn95C+ePh7XrXSyAdX7287c269leCzNnuXyggw pEzjQCthXUW+EOSmV3roj4Z1gfoOGmyYl7YggABoKFkc02Y1/yTSSCukUV1r/X/WhBlK hB0I+LAbXp8wR3Ixao3mijFvFYzuP9fKfJ1zm4OMCoxW3B+7WtSfsArRTszUSG7czvFD ORlZ4tjSILfOpq6tOKA+3z1nEH4iiRbYrj4aJEP/qIPWZ1Mv19ZZOxYdT79VrxQ7yv+0 ETNvpfWHuHjyRcJxHIhwLVxpv3whFmzS7QUg2tXoQ4V8VjwyUfCi6sd0sN/A08E580GD cLrA== X-Gm-Message-State: ALoCoQmsWfDXAnYvvzRLEQWUESs/4DP5WEG2sCMxj79vcA3PrnM3SPv9qaceS2dG+dBrj2xRlOUr X-Received: by 10.194.222.230 with SMTP id qp6mr17383204wjc.23.1407341974121; Wed, 06 Aug 2014 09:19:34 -0700 (PDT) Received: from localhost (054142ff.skybroadband.com. [5.65.66.255]) by mx.google.com with ESMTPSA id lh15sm7549333wic.10.2014.08.06.09.19.32 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Aug 2014 09:19:33 -0700 (PDT) Date: Wed, 6 Aug 2014 17:19:32 +0100 From: Matt Fleming To: Eric van Gyzen Subject: Re: Sanity Check: Bogus(?) General Protection Fault Message-ID: <20140806161932.GG15082@console-pimps.org> References: <53E237B6.4040703@vangyzen.net> <20140806144833.GE15082@console-pimps.org> <53E24FF0.7030305@vangyzen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53E24FF0.7030305@vangyzen.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 16:26:56 -0000 On Wed, 06 Aug, at 11:55:28AM, Eric van Gyzen wrote: > > Thanks, Matt. That has always been my understanding (and I just found > the docs to confirm). I doubted myself because the problem is now even > more bizarre. The mov before the faulting instruction apparently didn't > complete. %r11 is still an old value, not 0x....f7a8. The mov to %r11 isn't an immediate move, it's a memory read. So %r11 will contain the *contents* of 0xffffffff809bf7a, not the address itself (note the missing '$' as in the shr instruction above it). -- Matt Fleming, Intel Open Source Technology Center From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 6 17:11:14 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1973F59 for ; Wed, 6 Aug 2014 17:11:14 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id C35E628F5 for ; Wed, 6 Aug 2014 17:11:14 +0000 (UTC) Received: from marvin.beer.town (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id DCBDD56444; Wed, 6 Aug 2014 12:11:13 -0500 (CDT) Message-ID: <53E261B1.5010108@vangyzen.net> Date: Wed, 06 Aug 2014 13:11:13 -0400 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Matt Fleming Subject: Re: Sanity Check: Bogus(?) General Protection Fault References: <53E237B6.4040703@vangyzen.net> <20140806144833.GE15082@console-pimps.org> <53E24FF0.7030305@vangyzen.net> <20140806161932.GG15082@console-pimps.org> In-Reply-To: <20140806161932.GG15082@console-pimps.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 17:11:15 -0000 On 08/06/2014 12:19, Matt Fleming wrote: > On Wed, 06 Aug, at 11:55:28AM, Eric van Gyzen wrote: >> Thanks, Matt. That has always been my understanding (and I just found >> the docs to confirm). I doubted myself because the problem is now even >> more bizarre. The mov before the faulting instruction apparently didn't >> complete. %r11 is still an old value, not 0x....f7a8. > > The mov to %r11 isn't an immediate move, it's a memory read. So > %r11 will contain the *contents* of 0xffffffff809bf7a, not the address > itself (note the missing '$' as in the shr instruction above it). Argh! THAT was the sanity-check I needed. Thanks again, Matt. "It's always some mundane detail." --Michael Bolton Eric From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 13:30:39 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83489A05 for ; Thu, 7 Aug 2014 13:30:39 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 22415282A for ; Thu, 7 Aug 2014 13:30:37 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s77DS9JQ015797 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 7 Aug 2014 13:28:10 GMT Date: Thu, 7 Aug 2014 16:30:24 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: run-queue length via sysctl Message-Id: <20140807163024.30fa15298e9dfee4661f68ec@systemdatarecorder.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 13:30:39 -0000 Hi, I made some progress in porting 2 data recorders to FreeBSD: cpurec, hdwrec: http://www.systemdatarecorder.org/recording/agents.html Im working now on sysrec and found very powerful the sysctl interface since last time I checked in 2007 or so. But I could not find any way to get the value of run-queue length via sysctl. procs memory page disks faults cpu r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy id 0 0 0 757M 61G 6 0 0 0 12 1 0 0 9 50 135 0 0 100 0 0 0 757M 61G 0 0 0 0 0 1 0 0 21 140 208 0 0 100 0 0 0 757M 61G 0 0 0 0 0 1 0 0 3 125 94 0 0 100 0 0 0 757M 61G 0 0 0 0 0 1 0 0 4 125 99 0 0 100 I dont want to call vmstat in my script ... Any ideas how to do this ? thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 13:47:38 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 198E71C1 for ; Thu, 7 Aug 2014 13:47:38 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id D83942A99 for ; Thu, 7 Aug 2014 13:47:37 +0000 (UTC) Received: from marvin.beer.town (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 2009156444; Thu, 7 Aug 2014 08:47:37 -0500 (CDT) Message-ID: <53E38378.60401@vangyzen.net> Date: Thu, 07 Aug 2014 09:47:36 -0400 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Stefan Parvu Subject: Re: run-queue length via sysctl References: <20140807163024.30fa15298e9dfee4661f68ec@systemdatarecorder.org> In-Reply-To: <20140807163024.30fa15298e9dfee4661f68ec@systemdatarecorder.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 13:47:38 -0000 On 08/07/2014 09:30, Stefan Parvu wrote: > Hi, > > I made some progress in porting 2 data recorders to FreeBSD: > cpurec, hdwrec: http://www.systemdatarecorder.org/recording/agents.html > > Im working now on sysrec and found very powerful the sysctl interface since last > time I checked in 2007 or so. > > But I could not find any way to get the value of run-queue length via sysctl. > > procs memory page disks faults cpu > r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy id > 0 0 0 757M 61G 6 0 0 0 12 1 0 0 9 50 135 0 0 100 > 0 0 0 757M 61G 0 0 0 0 0 1 0 0 21 140 208 0 0 100 > 0 0 0 757M 61G 0 0 0 0 0 1 0 0 3 125 94 0 0 100 > 0 0 0 757M 61G 0 0 0 0 0 1 0 0 4 125 99 0 0 100 > > I dont want to call vmstat in my script ... Any ideas how to do this ? $ sysctl vm.vmtotal vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) =============================================== Processes: (RUNQ: 2 Disk Wait: 0 Page Wait: 0 Sleep: 239) Virtual Memory: (Total: 1083133168K Active: 9251728K) Real Memory: (Total: 7492844K Active: 1953232K) Shared Virtual Memory: (Total: 122304K Active: 113348K) Shared Real Memory: (Total: 48648K Active: 48268K) Free Memory: 764348K vmstat uses the vm.vmtotal sysctl to get a struct vmtotal{} for the "r b w" columns. See fill_vmtotal() in src/usr.bin/vmstat/vmstat.c and vmtotal{} in /usr/include/sys/vmmeter.h. Eric From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 15:22:01 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33D3A3AD for ; Thu, 7 Aug 2014 15:22:01 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AA79928B6 for ; Thu, 7 Aug 2014 15:21:59 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s77FJZHE016260 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 7 Aug 2014 15:19:36 GMT Date: Thu, 7 Aug 2014 18:21:46 +0300 From: Stefan Parvu To: Eric van Gyzen Subject: Re: run-queue length via sysctl Message-Id: <20140807182146.43b3f1c2239a5aa4456ff19f@systemdatarecorder.org> In-Reply-To: <53E38378.60401@vangyzen.net> References: <20140807163024.30fa15298e9dfee4661f68ec@systemdatarecorder.org> <53E38378.60401@vangyzen.net> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 15:22:01 -0000 > $ sysctl vm.vmtotal > vm.vmtotal: > System wide totals computed every five seconds: (values in kilobytes) super. many thanks. Reading it now. > vmstat uses the vm.vmtotal sysctl to get a struct vmtotal{} for the "r b > w" columns. See fill_vmtotal() in src/usr.bin/vmstat/vmstat.c and > vmtotal{} in /usr/include/sys/vmmeter.h. I will check as well vmstat.c -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 18:11:13 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 997EC434 for ; Thu, 7 Aug 2014 18:11:13 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2853B219D for ; Thu, 7 Aug 2014 18:11:12 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id mc6so3871109lab.34 for ; Thu, 07 Aug 2014 11:11:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=OvZGLOSUje49jGbUn2YJW4JeeWLL9pYp6RNjFtsDGsQ=; b=M5Kl4JB5DEIIXfuHn0p/SBbFsdTuFBAKycsMNT83bBldhjTKajRzf2+CWzpqgNW+Nv n+zbM63v/9EyK/N0oOUz+HdSinb/HFP0r5Fj2Etm52v7aQ9Val5CgLlE/2D+U3PP1K/g ziyQaoY43CEkoooAWiArN/FWck5RYnhNBiBYDwDnbG2qcNRbjA6TqB9O4MK3Lfp9eRdK XM3t+G8ZYF0URE3f0KdBOyykS542u+igwL3hgclz0lY08XlkFF3ReU4f2BZfwkUL6RTa qfS9Ncf3/UQgRmfCQ4wBuWLqsNBJJpWx1XcPiwf8muYFJQ0QHXG6ZmPTetYIXWNiguBn LXgQ== MIME-Version: 1.0 X-Received: by 10.112.166.200 with SMTP id zi8mr2980928lbb.102.1407435070798; Thu, 07 Aug 2014 11:11:10 -0700 (PDT) Sender: hiren.panchasara@gmail.com Received: by 10.114.81.73 with HTTP; Thu, 7 Aug 2014 11:11:10 -0700 (PDT) Date: Thu, 7 Aug 2014 11:11:10 -0700 X-Google-Sender-Auth: zIeuCWLSUzH9I4ibhC5SD7n24rc Message-ID: Subject: Where is loader.conf.gz? From: hiren panchasara To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 18:11:13 -0000 I am trying to netboot minnowboard max with loader.efi and then nfsmount the root fs from my FreeBSD-Current zfs laptop. It gets stuck right at "Loading /boot/defaults/loader.conf" Looking at tcpdump (on interface serving nfsd on laptop), it tries to find loader.conf.gz and fails. I've failed to find loader.conf.gz anywhere on box or in the source code. Where does it come from and who is looking for it? cheers, Hiren From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 18:24:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BF57708; Thu, 7 Aug 2014 18:24:15 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A32E22AD; Thu, 7 Aug 2014 18:24:15 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 02C951218A; Thu, 7 Aug 2014 11:24:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1407435853; x=1407450253; bh=339lKTIwR4xt9DPZgiqrEDEqQEXLWvFGUU1iWjPLYm8=; h=Date:From:Reply-To:To:Subject:References:In-Reply-To; b=hlM+l1f4jIZhfyliXEECrcYizMoRqXvy6JnIRznb6jaFXCOrMiiX4AC8lPWpmBccO srbri6VG0a5qwmpCsE84xazwo5cKt+ypP5kfXRHZaArlaXhuu8TT4k7zfEXTQusCak In0eyO00ywwTSFyQAHspd8LIsH/KP1mhk5SQJp4Q= Message-ID: <53E3C44C.9000500@delphij.net> Date: Thu, 07 Aug 2014 11:24:12 -0700 From: Xin Li Reply-To: d@delphij.net Organization: The FreeBSD Project MIME-Version: 1.0 To: hiren panchasara , "freebsd-hackers@freebsd.org" Subject: Re: Where is loader.conf.gz? References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 18:24:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 08/07/14 11:11, hiren panchasara wrote: > I am trying to netboot minnowboard max with loader.efi and then > nfsmount the root fs from my FreeBSD-Current zfs laptop. > > It gets stuck right at "Loading /boot/defaults/loader.conf" > > Looking at tcpdump (on interface serving nfsd on laptop), it tries > to find loader.conf.gz and fails. > > I've failed to find loader.conf.gz anywhere on box or in the source > code. Where does it come from and who is looking for it? lib/libstand/gzipfs.c I think? (zf_open) It's weird that it didn't handle the ENOENT error, though... Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0 iQIcBAEBCgAGBQJT48RMAAoJEJW2GBstM+nsn00QAKIs2nxYx1Q6gFFdS3EC4oKA 7dGu0wM9H+srEfyiG8oVVbhqxr95WoPVUihcA+KVH6uB/68R2BfBCuY4DhsAr3kL c3U+pNiAJFOMFINXAyHHjoYxTcUU5U1ef/xhGEuWEsud4MLzxvL4hPhUpmDVs5ON QWE+G3tDpVjUKyFCeY9MfBmkkOu7bQC7f6AV/HnmtWkMZVyZSjnhOvOptUzGcUop TKIvmAz0fymAqnOGKD/sLUuvHFQj4JHCrBgSzX9kYOXuqPRiZm2WKtFswmtbtl+t Bd5Enx6k4yUUjArcAs4GYmxzily6q/qcMpUPvCjMxPjXyOkmdEtK+jrE5VeIyZmv IKaK5qvVFD4qBrv2EMz10UinxQ4zBks1Iz2VE005V/Kx2GkADnpcGyVr/vyxATNv l6qcRvmD3o/WcOSsqSZpW1WS5VVvPGVo0WF1Yq91IDZfCxl0vwihnHvtj1jHDTLe XLWgWOmfn/hOkKO4+7BzXo1jAd//mSU/YT9Pzo+XVA+FyQL8pWDlVmRfv9Bz3kM4 0MNw6YESGUgRPWnN/eiGwIHlgJgcv69YATGnFXBANFtHzhXRoWZ08pGf+2hL8wN/ jxKsQLHATwa2mZe+JQVq6AhVDuFxZl1eZ0KeFw1Lf6w84g2yKv+qq7KW7CvcBrmP 66ihi4u/FQbyj+z3BKAk =Gt2B -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 7 21:46:32 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 824C34C2 for ; Thu, 7 Aug 2014 21:46:32 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0CD6E2B0C for ; Thu, 7 Aug 2014 21:46:31 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id l4so3230809lbv.30 for ; Thu, 07 Aug 2014 14:46:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=HuYtbBX1DdGWPXmTNn1YYKIN8cHVtOuelRhXkChHxts=; b=VmHBs3zW2+FFBABk6pcbqtuCzGanvgBvyHRKj9Ka64PKW2FZoPG8ovNEWGJ69r/NDN IwiKpN/2VSBK7lvFHigOqOhZ32+B4j8RSSNy0QBwAM5qWwKNrDtGkhZh+nTVY5jSfqvq OEZSNv3OtzWnjq0NPQGNNEWghztUHzyT3+uHGN7lguG4J0phE4qhqtNaqODSWHuznayb Rd750oepsVcHJV3e1ub6digEEsBWR9gCtWILlgnzVlDMLem3iqH+Uv3Mt9gTERaf35Ja Auti4cex3xKKC3BF1OtHm2L3DigogOi2EKlWMUubAhWe3MxnqOCuQAkIlYRNHwIlB/xI y/cQ== MIME-Version: 1.0 X-Received: by 10.152.234.71 with SMTP id uc7mr18995189lac.22.1407447989888; Thu, 07 Aug 2014 14:46:29 -0700 (PDT) Sender: hiren.panchasara@gmail.com Received: by 10.114.81.73 with HTTP; Thu, 7 Aug 2014 14:46:29 -0700 (PDT) In-Reply-To: <53E3C44C.9000500@delphij.net> References: <53E3C44C.9000500@delphij.net> Date: Thu, 7 Aug 2014 14:46:29 -0700 X-Google-Sender-Auth: hkLO2z9w3mxvTUDrXCR8X2YOMtk Message-ID: Subject: Re: Where is loader.conf.gz? From: hiren panchasara To: Xin LI Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 21:46:32 -0000 On Thu, Aug 7, 2014 at 11:24 AM, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 08/07/14 11:11, hiren panchasara wrote: >> I am trying to netboot minnowboard max with loader.efi and then >> nfsmount the root fs from my FreeBSD-Current zfs laptop. >> >> It gets stuck right at "Loading /boot/defaults/loader.conf" >> >> Looking at tcpdump (on interface serving nfsd on laptop), it tries >> to find loader.conf.gz and fails. >> >> I've failed to find loader.conf.gz anywhere on box or in the source >> code. Where does it come from and who is looking for it? > > lib/libstand/gzipfs.c I think? (zf_open) > > It's weird that it didn't handle the ENOENT error, though... > Xin, You are right. lib/libstand/open.c: open() looks like this: /* pass file name to the different filesystem open routines */ besterror = ENOENT; for (i = 0; file_system[i] != NULL; i++) { fs = file_system[i]; error = (fs->fo_open)(file, f); <-----zf_open() is called which returns ENOENT if (error == 0) goto ok; if (error != EINVAL) besterror = error; } error = besterror; So, the error is ENOENT at this point. But it fails in what happens after this: fail: if ((f->f_flags & F_NODEV) == 0 && f->f_dev != NULL) f->f_dev->dv_close(f); if (error) devclose(f); Here, it gets into both if() conditions and stalls/hangs in the call to devclose(): int devclose(struct open_file *f) { if (f->f_devdata != NULL) { free(f->f_devdata); <-- hangs right here and never returns. } return (0); } It probably panics and doesn't tell me. I have to hard reset the board. Is it because we are trying to close something we couldn't open? cheers, Hiren From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 02:58:38 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96CACBF4 for ; Fri, 8 Aug 2014 02:58:38 +0000 (UTC) Received: from nm14-vm3.bullet.mail.ne1.yahoo.com (nm14-vm3.bullet.mail.ne1.yahoo.com [98.138.91.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E1792B49 for ; Fri, 8 Aug 2014 02:58:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1407466570; bh=aaxgsCJOFIbEVdl+6ZDXoWYoRXhtc/17p8Qvrm1XkwM=; h=Received:Received:Received:DKIM-Signature:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Subject:From:Content-Type:X-Mailer:Message-Id:Date:To:Content-Transfer-Encoding:Mime-Version; b=CCx8acxREslgGMTx7OdYByp+bhSc870DNElWmmhoOCxeJEGA07W8ASsRUpdKYg5GVIe3CDWdsCSGFPB1fDA8ZEbsPDoWAmab1d1FrmJNIfN/UkTbDzcqNbcAxIqvEg0URUlkRaI+XQPQxOsUeFAnCODjg74IWR+uty2X3pAqk8ISSlqUYqFAUCcqMdsUbNqeeQrmG//nIsFaN4+sQImiJhU+KZRM4bVwWOj//HRv1PvJEIEgNvExdD3NzePXS14CuUNUJBmdUoG3+Q6KWgs+Uh7rqlPPhHkP6cmU5rPsu+egTLMzs8aReMAx4jFG/agbd+P4ESLfy2AbIWgKWM5xlw== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=KOgf+O+VE8PRMB0IepLCOOeXe31S8QJTVdsgbpyybZqkV2CZBdb815Kgo4/AfR31pfw+O78omRCMEkBDn6g8oGk6ujJGcj9Yf5vXL5Rwr85WD1m9HY34MB+sMchQ4hHlk4ZixONsQo0b74LafKJOzuFOoTKfRXinq/6daqZQs/WQPTX4n91gapSGZDM6Eh754JYxVx0iy/N4ydP8E2hbWmOOX5WISjpZHOWsqbyI3/DEFCN6VlAEWAKDHm7u2CYH+pGFaz9txBXlquN3f5DKE3EqRMRsYPzD3qoenZ2Qh/nIzDCFnkcaM8axw1j2/G3Ha2wIw87G17W/znGD8lhEZA==; Received: from [98.138.100.113] by nm14.bullet.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 02:56:10 -0000 Received: from [98.138.84.37] by tm104.bullet.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 02:56:10 -0000 Received: from [127.0.0.1] by smtp105.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 02:56:10 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1407466570; bh=aaxgsCJOFIbEVdl+6ZDXoWYoRXhtc/17p8Qvrm1XkwM=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Subject:From:Content-Type:X-Mailer:Message-Id:Date:To:Content-Transfer-Encoding:Mime-Version; b=ixDCJgtj5k+bngMx1gzmIcaarSEuujgN3yKfpQxbq318KfC8l+BkHwSrJ/y3AYfsNEnmPuXAmCYeX5OUbl5H40GjSFT50JkFPoGwJ75lPoHMbFA2Se/sTDjnun816j74ZHjUn4a6+6A8al3FJhLUcnxA099yuEfoADcZtKFw/m0= X-Yahoo-Newman-Id: 80461.5394.bm@smtp105.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: nM.aH7oVM1kuUaziPydR3DP07krDkZJVdTSZDaFmc.EQG3v tl2eaMH9sffNTUp12aIbB9APLSX3g7d7Nck3P.kZpOQpf7rg8OW6KCU9AzGd NabNELgEI85RtFbkNYqHsbx0eofHZOH6fSMfi6NZFYOKvEe4saR9tLIoMjD5 T0pqays7MnXxNIHJeHRFZHKl6wmWhsAYUtv1Ig9cYR1sNJiRP1GcB5gBGc3v f.S2J8xdhXYFQeTZKR05GIuQutif4v3pH1RNGfW.mKMpCnLcdDd9CNeKJ_Qr klbBBW4HWvI1EAysnZPy2SJj_xfna3NlfQKNOzpeFmYSWOrcr_ynAaX4u7rV K2Td0ObXCOOpP4QnSiF2ntmxXOBmBZWQdHalea6ojFP6FGno4CMOnsnE04KF WWYKofFJRUokGRmbTFCl5jmV7.Wlnw.tBTMvp6ijprQy_cvMat7k7Dy79ZEg Z9rEM6J30Ua5gYuywYVgDPgQjLeVe2f9e126BTKhNHs_aI6AgAgD4hgUTKrL EQVWPAYuM8MenWG0nLhY- X-Yahoo-SMTP: LAFNfTaswBDguI7meB90l2l3wOU- Subject: PKG ISSUE From: by Content-Type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (11D257) Message-Id: Date: Fri, 8 Aug 2014 10:55:55 +0800 To: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 02:58:38 -0000 Hi, I would say, I find a problem happens in the new pkg(1.3.5.1), which is: #pkg update Upgrade pkg to 1.3.5.1 #pkg install en-freebsd-doc Install the documents of freebsd Then I try to clean the cache #pkg clean And I get: Nothing to do. But question is, in /var/cache/pkg, downloaded packages still there. I would say, does this means some bug get into pkg 1.3.5.1, or how can I do t= o clean the cache using pkg tool. - by= From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 06:59:24 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29656E63 for ; Fri, 8 Aug 2014 06:59:24 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B2E932586 for ; Fri, 8 Aug 2014 06:59:23 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hi2so494803wib.16 for ; Thu, 07 Aug 2014 23:59:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=26qa8ziqkhLEYS8yvb96bhnS2xWJtfkiruFgZwjlk6w=; b=h7zs1qTGLhJfvBVlQy7rSlveqM+f5OfjoYjrYF0ooWTDDD49Ng/F7xqfTljuAliN2I VNaLhfU3H2SnkCMjdFJ8K6VKm/ZlLKGpQYmsbBsbU2U7OoTAxEUPv0AYo48ccl2LaUz2 E9RqsYMSIUaX5cjetwSXZE4VSpF2F9HIkMt/chU2R+/KvHm/oWv2e61gyUOyl/skWQR1 5sGUCHAwYj+WVlPPbLLJBMC+0ii57E54Kdmn+bs6bSH7RvYbYP+/pzNTLwFvF5NoJBms 0uxuWJa/rKZ6xXsiXU2wyhxcQ7L8SdWQyzv8zpPNtUH5ZtflZ2oC//iDFx6NINNZU/T0 HcoA== X-Received: by 10.180.183.36 with SMTP id ej4mr1896986wic.77.1407481161824; Thu, 07 Aug 2014 23:59:21 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id fv10sm4153908wic.23.2014.08.07.23.59.20 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Aug 2014 23:59:20 -0700 (PDT) Sender: Baptiste Daroussin Date: Fri, 8 Aug 2014 08:59:18 +0200 From: Baptiste Daroussin To: by Subject: Re: PKG ISSUE Message-ID: <20140808065918.GA17537@ivaldir.etoilebsd.net> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="T4NxgPrvSrIDR5b4" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 06:59:24 -0000 --T4NxgPrvSrIDR5b4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 08, 2014 at 10:55:55AM +0800, by via freebsd-hackers wrote: > Hi, >=20 > I would say, I find a problem happens in the new pkg(1.3.5.1), which is: > #pkg update > Upgrade pkg to 1.3.5.1 > #pkg install en-freebsd-doc > Install the documents of freebsd > Then I try to clean the cache > #pkg clean > And I get: Nothing to do. > But question is, in /var/cache/pkg, downloaded packages still there. >=20 > I would say, does this means some bug get into pkg 1.3.5.1, or how can I = do to clean the cache using pkg tool. >=20 Reading the man page of pkg clean via man pkg-clean of pkg help clean would= have given you, to quote some part of it: [...] pkg clean is used to cleanup the local cache of packages downloaded from remote repositories. It removes packages that have been superseded by newer versions, and any packages that are no longer provided. [...] -a, --all Delete all cached packages. [...] regards, Bapt --T4NxgPrvSrIDR5b4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlPkdUYACgkQ8kTtMUmk6ExEAACfWE579WytAA6OQnblDC+EUJXC HvoAoINscDbnnEMWOJjPwth0kSNV1l1X =yKgG -----END PGP SIGNATURE----- --T4NxgPrvSrIDR5b4-- From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 08:56:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44D28903 for ; Fri, 8 Aug 2014 08:56:47 +0000 (UTC) Received: from nm1-vm4.bullet.mail.ne1.yahoo.com (nm1-vm4.bullet.mail.ne1.yahoo.com [98.138.91.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED7C52297 for ; Fri, 8 Aug 2014 08:56:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1407488200; bh=mrsaiGNYaYezVbKN6r/BBLqpDTUcaZFsi8aCRwYipZk=; h=Received:Received:Received:DKIM-Signature:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Subject:References:From:Content-Type:X-Mailer:In-Reply-To:Message-Id:Date:To:Content-Transfer-Encoding:Mime-Version; b=HRkh/omRB6TzJe7WzwVBLTSO8MPCRERdLsT7G74lpZKek62z7VAwuZwKq+VHzSJq/FpYfejrafI+rEOQWxd1ubfHnrdMKdko2yWxWGuXKGt6Bj9gftkMf8zV9JziALayndcs2c5fjYHiSJiEgJcYcUziJQv4N9GWAWxYQyPEf1N16xRNrRMOwmUDM7em9qmuM0yN0kb86tO1iHHNommHq60VwJgCVnF8zSB4byY6mBu4CkeOTsVkrZLgaoLI2dJZrjicTB3jGoh6cODSVYzzi+sNJZJPp9YU4Wru167nxvtIlj6AwcdFQ04jOoAMW8OhZVNNtc76KxdwcbAv8I1vMw== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=pt0pK863DqY8gtHYP2yiQQ9T3Ox1JoapUG/sy+GsRLb7FRAtB7fbXcoVP2ETBnHpdzPQyVdW0vC3gE1m8jc49JhajZjipphMxBJYMc4gNE2fxBzbeaHNhRqtXT0a1GAuF9ngQcxfKINnkaLdtceyqETeQKwCbndDwV0SSYvWTKnfxnr02i5oWi2mDmxI+/tUv11iDfISeY4itZyALBVthB2notvamZ9SDnTuB+EMVaZXE6kSkBzBMLa8El7Vzg5S/tKethMeoS/N6JAWnMYXhGUOUd3ZH2jYrdhSmWUP8+UOrt7RPNOn+Ms5QHGiQ+sdaW0uU5H1/6dLe+nXWANqfQ==; Received: from [98.138.101.131] by nm1.bullet.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 08:56:40 -0000 Received: from [98.138.104.113] by tm19.bullet.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 08:56:40 -0000 Received: from [127.0.0.1] by smtp222.mail.ne1.yahoo.com with NNFMP; 08 Aug 2014 08:56:40 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1407488200; bh=mrsaiGNYaYezVbKN6r/BBLqpDTUcaZFsi8aCRwYipZk=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Subject:References:From:Content-Type:X-Mailer:In-Reply-To:Message-Id:Date:To:Content-Transfer-Encoding:Mime-Version; b=IK/raIRVtWjdXN3BIt8UFyYKReLPZUYIGMtnhhMTtZBB3z4ccOoue9PRGotTeWNCVZWU2KV28oWlooT2HEXIkJIQuIw57O0zaZooRn/7eTFAFi0uCsH1BSl0BNHfySta4nZ3JjFsmWskkko31+Xo+gtQDz4E/GD6a5d3WI3KpiQ= X-Yahoo-Newman-Id: 417475.65787.bm@smtp222.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: WYBrLmQVM1kwlln5jmNSgYw.jsDC1FsKr2g0h5TtmA3qYXs 5Q8scrNcWddRQvgWEWpvxsYpzYJHCQZpUUW7cOd.7AJqCkobwwJVR5u3aOAp BgMrFG6WPp1dTKgwnpBHd24dLQzxf3zqKfh3VhLyePspd8ggQSu0eJh_uAYU 9S0rru4u4RRNky8GibKv5rge1JFv.4epLkURUfif4rGUesLquSCoXgufyYh1 Xsl9zQkw2sLynoGyvPkeBEe5yeB53c7QVopYd3I0UbbvyiEWlPaHPjtgkmLt y._AaoKfy.w29mtKdz11x6y.m_wmfus49dnUJfaqw0evKj1lDmIjEw8BWx5R 3PQsyXCqTxS5oZc34WO2Hv04GKydsz9g.fl4Mygc.tCJ1DVTz9sibA.qynV0 ttg1_ivmoukE2uiePYWDn6SNHbTGUOzBLtX0Dn2Yyr7ToEC.y92B.xivVvhP FAjs35.0yd6f635qxyfDjuwhXd6FPVdmhywaoRRA79ut5CgPhplz3Qk1JK4b 1dVZelY_lcAZ5dnl74KE- X-Yahoo-SMTP: LAFNfTaswBDguI7meB90l2l3wOU- Subject: Re: PKG ISSUE References: <20140808065918.GA17537@ivaldir.etoilebsd.net> From: by Content-Type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (11D257) In-Reply-To: <20140808065918.GA17537@ivaldir.etoilebsd.net> Message-Id: Date: Fri, 8 Aug 2014 16:56:28 +0800 To: "freebsd-hackers@freebsd.org" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 08:56:47 -0000 Hi, Thank you for your kindness reply, I have not refer the pkg-clean before, be= cause what I referred is just the pkg manual page. So, with the -a option issued, it works! - by > On Aug 8, 2014, at 14:59, Baptiste Daroussin wrote: >=20 >> On Fri, Aug 08, 2014 at 10:55:55AM +0800, by via freebsd-hackers wrote: >> Hi, >>=20 >> I would say, I find a problem happens in the new pkg(1.3.5.1), which is: >> #pkg update >> Upgrade pkg to 1.3.5.1 >> #pkg install en-freebsd-doc >> Install the documents of freebsd >> Then I try to clean the cache >> #pkg clean >> And I get: Nothing to do. >> But question is, in /var/cache/pkg, downloaded packages still there. >>=20 >> I would say, does this means some bug get into pkg 1.3.5.1, or how can I d= o to clean the cache using pkg tool. > Reading the man page of pkg clean via man pkg-clean of pkg help clean woul= d have > given you, to quote some part of it: >=20 > [...] > pkg clean is used to cleanup the local cache of packages downloaded fro= m > remote repositories. It removes packages that have been superseded by > newer versions, and any packages that are no longer provided. >=20 > [...] >=20 > -a, --all > Delete all cached packages. >=20 > [...] >=20 >=20 > regards, > Bapt From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 10:32:08 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 295B27BC; Fri, 8 Aug 2014 10:32:08 +0000 (UTC) Received: from forward8l.mail.yandex.net (forward8l.mail.yandex.net [IPv6:2a02:6b8:0:1819::8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D59102DEE; Fri, 8 Aug 2014 10:32:07 +0000 (UTC) Received: from smtp17.mail.yandex.net (smtp17.mail.yandex.net [95.108.252.17]) by forward8l.mail.yandex.net (Yandex) with ESMTP id 96FCB1A40D10; Fri, 8 Aug 2014 14:31:56 +0400 (MSK) Received: from smtp17.mail.yandex.net (localhost [127.0.0.1]) by smtp17.mail.yandex.net (Yandex) with ESMTP id 0F0DF190060C; Fri, 8 Aug 2014 14:31:53 +0400 (MSK) Received: from 84.201.167.16-vpn.dhcp.yndx.net (84.201.167.16-vpn.dhcp.yndx.net [84.201.167.16]) by smtp17.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id O1a3Tq2fTQ-VrLi99IW; Fri, 8 Aug 2014 14:31:53 +0400 (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) X-Yandex-Uniq: b4ff4c40-05f1-42f8-8fa3-baf462f8a90e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1407493913; bh=4YbSNKs92bB+IIWsU0Z6I5mwNHGZQRcERT9Mhvh/rZs=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=ULKrjqwGofIzyoZHprMDtEGWKfETyY1i5ByQysWR/BM6usp1vmBU6yOn4yWxC8H7X /09NinXSnomDbxxs7iOEOgXmcwsuLXyeTXkma+D96L90GxYFOXFMSw8wqpT9jxJMB0 tHjNX2BdyE0tIuHR2nEnRZPyA1JJwA96r4OOBvzY= Authentication-Results: smtp17.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <53E4A6E7.3080309@yandex.ru> Date: Fri, 08 Aug 2014 14:31:03 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: hiren panchasara , Xin LI Subject: Re: Where is loader.conf.gz? References: <53E3C44C.9000500@delphij.net> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 10:32:08 -0000 On 08.08.2014 01:46, hiren panchasara wrote: > /* pass file name to the different filesystem open routines */ > besterror = ENOENT; > for (i = 0; file_system[i] != NULL; i++) { > fs = file_system[i]; > error = (fs->fo_open)(file, f); <-----zf_open() is called > which returns ENOENT > if (error == 0) > goto ok; > if (error != EINVAL) > besterror = error; > } > error = besterror; > > So, the error is ENOENT at this point. But it fails in what happens after this: It looks like loader.efi should first open /boot/loader.conf via devopen() a bit earlier in this code. Did you see this attempt in the tcpdump? Can you stop loader and see what value has currdev environment variable? -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 10:57:46 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 772BB187; Fri, 8 Aug 2014 10:57:46 +0000 (UTC) Received: from forward7l.mail.yandex.net (forward7l.mail.yandex.net [IPv6:2a02:6b8:0:1819::7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D0D820CE; Fri, 8 Aug 2014 10:57:46 +0000 (UTC) Received: from smtp16.mail.yandex.net (smtp16.mail.yandex.net [95.108.252.16]) by forward7l.mail.yandex.net (Yandex) with ESMTP id 26B36BC0511; Fri, 8 Aug 2014 14:57:43 +0400 (MSK) Received: from smtp16.mail.yandex.net (localhost [127.0.0.1]) by smtp16.mail.yandex.net (Yandex) with ESMTP id 964406A0886; Fri, 8 Aug 2014 14:57:42 +0400 (MSK) Received: from 84.201.167.16-vpn.dhcp.yndx.net (84.201.167.16-vpn.dhcp.yndx.net [84.201.167.16]) by smtp16.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id frBuwGIR8N-vgROK9bi; Fri, 8 Aug 2014 14:57:42 +0400 (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) X-Yandex-Uniq: f3bbacab-fa85-4685-863d-4a9500cee5b9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1407495462; bh=rC5aC4KZ47/qVqEX4r2eML3+v9ZQq49oRgbDMCk/zKk=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=NPntrSNFopUQZ1n+xxj960/2ykZa6JHfSrfbvzZPmsqSgF0YGu3DA4BIiZvj3pHkT 35qblxH2Xb3TroN3kzTgy1YPYGNzrvJ5A/tHoqSMEuiJdnIDDXWJGvbLipf1OHVZOG WF+m4c8kTKV0LEnTeuNFt40P53/wicSf/CAa2uBQ= Authentication-Results: smtp16.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <53E4ACF4.4030001@yandex.ru> Date: Fri, 08 Aug 2014 14:56:52 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: hiren panchasara , Xin LI Subject: Re: Where is loader.conf.gz? References: <53E3C44C.9000500@delphij.net> <53E4A6E7.3080309@yandex.ru> In-Reply-To: <53E4A6E7.3080309@yandex.ru> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 10:57:46 -0000 On 08.08.2014 14:31, Andrey V. Elsukov wrote: > On 08.08.2014 01:46, hiren panchasara wrote: >> /* pass file name to the different filesystem open routines */ >> besterror = ENOENT; >> for (i = 0; file_system[i] != NULL; i++) { >> fs = file_system[i]; >> error = (fs->fo_open)(file, f); <-----zf_open() is called >> which returns ENOENT >> if (error == 0) >> goto ok; >> if (error != EINVAL) >> besterror = error; >> } >> error = besterror; >> >> So, the error is ENOENT at this point. But it fails in what happens after this: > > It looks like loader.efi should first open /boot/loader.conf via > devopen() a bit earlier in this code. Did you see this attempt in the > tcpdump? Can you stop loader and see what value has currdev environment > variable? Sorry, devopen() doesn't open a file, but nfs_fsops should be tried first. Maybe this code has been failed in nfs_open? #ifndef __i386__ if (strcmp(f->f_dev->dv_name, "net") != 0) return(EINVAL); #else -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 11:35:17 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88542AFE for ; Fri, 8 Aug 2014 11:35:17 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A09B2576 for ; Fri, 8 Aug 2014 11:35:15 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s78BWdUA021484 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 8 Aug 2014 11:32:39 GMT Date: Fri, 8 Aug 2014 14:34:55 +0300 From: Stefan Parvu To: David Wolfskill Subject: Re: memory utilization questions Message-Id: <20140808143455.459e4ed0740920f99aaeb4d7@systemdatarecorder.org> In-Reply-To: <20140806123811.GB1228@albert.catwhisker.org> References: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> <20140806123811.GB1228@albert.catwhisker.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 11:35:17 -0000 > * Another OID: vfs.bufspace; I integer-divide this by 1024 (to get > results in KB), as it seems to be in bytes vs. pages. Any ideas what is this parameter monitoring ? I found references about KVA memory used for bufs structures. Why should we even keep a track of it ? -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 12:01:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3658CCB6 for ; Fri, 8 Aug 2014 12:01:47 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D62D027BE for ; Fri, 8 Aug 2014 12:01:46 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id s78C1ckK039313; Fri, 8 Aug 2014 05:01:38 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id s78C1b1p039312; Fri, 8 Aug 2014 05:01:37 -0700 (PDT) (envelope-from david) Date: Fri, 8 Aug 2014 05:01:37 -0700 From: David Wolfskill To: Stefan Parvu Subject: Re: memory utilization questions Message-ID: <20140808120137.GZ30083@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Stefan Parvu , freebsd-hackers@freebsd.org References: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> <20140806123811.GB1228@albert.catwhisker.org> <20140808143455.459e4ed0740920f99aaeb4d7@systemdatarecorder.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="csehs8AeUiGwWnbr" Content-Disposition: inline In-Reply-To: <20140808143455.459e4ed0740920f99aaeb4d7@systemdatarecorder.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 12:01:47 -0000 --csehs8AeUiGwWnbr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 08, 2014 at 02:34:55PM +0300, Stefan Parvu wrote: >=20 > > * Another OID: vfs.bufspace; I integer-divide this by 1024 (to get > > results in KB), as it seems to be in bytes vs. pages. >=20 > Any ideas what is this parameter monitoring ? I found references about=20 > KVA memory used for bufs structures. Why should we even keep a track of i= t ? > ... Please see src/usr.bin/top/machine.c. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --csehs8AeUiGwWnbr Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJT5LwgXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7jcoP/AoWvAaaQwi/lXww/qpYCLnI T6oZ+o9pfUdExs0+0//sPHFdY0l3v7R9MvmIwbXZMEUYj5dS6Jb/LNzpzrGsqvWp Nfiv8qlXVR2r9xAKZnuD7cOQ+aVgKRGAOinGLJWTvMc75x8xq6yATxRiQRKwCoQD rnI4/7pmzpF3/Bm+eFflG1WbFu35tXfPz/JAl3AEd1Dh9+BrJ0q8sAQ8B9qTHJCD yiy8Vd1/WO6vXLA1znl8qEKEmX9nxhpEUYwYxhL6f1vstjCEZL2p3eY9egKGoQDf YMzTSRP0AWVqKQfR0WctuqFUq9YdMkmvEyc30XeJRz2GTF9/VM0xcwjLyzOxAGqj rhHgnpoPS8MgmdGNOIfFn7djRaaCYK5dA7F5Ww8pA9R7l+uFeQoaF1KJyDZDvMWa gbRnrWb0jsmC7kEFeZewmxryF7IL8+Gl1xzdRChfl1qRx7+AJeeg+Sq66KJkONoC KDC8Oo2YIgCcDKLJh0rAMVtImvx3mhTjjFiWCDt9y/EM9JSwUq/HKvVITiFDlWER uTpq9dTmMnuV3x4YBBvmKvRng1wX50hjTH03bdpo21LAs+EYllA8mDymZN5TJhs6 W6coKOlXZIbpfTJ87sUJRHVQp6Tme6qsuRKfjwnIEim/ODwt9BpYsDOIyIdJ9ExO 9ThsrRjZ/vaqWx/wlajs =KsEu -----END PGP SIGNATURE----- --csehs8AeUiGwWnbr-- From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 12:23:50 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B818958B for ; Fri, 8 Aug 2014 12:23:50 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 37CE92A99 for ; Fri, 8 Aug 2014 12:23:49 +0000 (UTC) Received: from nereid (188-127-209-196.cust.suomicom.net [188.127.209.196]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s78CLEqa021702 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 8 Aug 2014 12:21:15 GMT Date: Fri, 8 Aug 2014 15:23:30 +0300 From: Stefan Parvu To: David Wolfskill Subject: Re: memory utilization questions Message-Id: <20140808152330.6eb4e7c708042eaa5248f82f@systemdatarecorder.org> In-Reply-To: <20140808120137.GZ30083@albert.catwhisker.org> References: <20140806145314.643dcc34a360f8284083930e@systemdatarecorder.org> <20140806123811.GB1228@albert.catwhisker.org> <20140808143455.459e4ed0740920f99aaeb4d7@systemdatarecorder.org> <20140808120137.GZ30083@albert.catwhisker.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 12:23:50 -0000 > > Please see src/usr.bin/top/machine.c. The vfs.bufspace seems related to the amount of virtual memory that is allocated to the buffer cache. Time for me to read back the internals of FreeBSD. Thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 15:40:36 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A016DEF3 for ; Fri, 8 Aug 2014 15:40:36 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2521D23E9 for ; Fri, 8 Aug 2014 15:40:35 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s78Fc5Tx022420 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 8 Aug 2014 15:38:06 GMT Date: Fri, 8 Aug 2014 18:40:21 +0300 From: Stefan Parvu To: freebsd-hackers@freebsd.org Subject: disk and NIC io statistics via sysctl Message-Id: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 15:40:36 -0000 Hi, Fetching CPU and Memory statistics, using sysctl interface and perl seems nice and easy. I made great progress with sysrec a generic recorder reporting: cpu mem disk and nic stats on FreeBSD. Now Im getting into disk and nic sections: 1. Disk Im able to get a list of all disks on a system using kern.disks sysctl variable. I want to be able to report per disk something like: readreq rate read requests per second, rate writereq rate write requests per second, rate totreq rate total read + write requests per second, rate readbyt rate read bytes per second, in KB, rate writebyt rate write bytes per second, in KB, rate totbyt rate total read + write bytes per second, in KB, rate iostat seems the simplest way to fetch these, but how about sysctl interface ? I could not figure out any sysctl parameters for disk throughput. How one will fetch throughput per disk via sysctl ? 2. NIC Same will go for network card interfaces. I want to fetch per NIC throughput and errors via sysctl, something like these: 1 timestamp time seconds since Epoch 2 interface number NIC name, string 3 rxKB rate the number of KBytes received per sec 4 rxpcks rate the no. of packets received per sec 5 rxerrs rate the number of errors while received packets per sec 6 rxdrop rate the number of packets that were dropped per sec 7 rxfifo rate the number of FIFO overruns on received packets per sec 8 rxframe rate the number of carrier errors on received packet per sec 9 rxcompr rate the number of compressed packets received per sec 10 rxmulti rate the number of multicast packets received per sec 11 txKB rate the number of KBytes transmitted per sec 12 txpcks rate the number of packets transmitted per sec 13 txerrs rate the number of errors transmitting packets per sec 14 txdrop rate the number of packets that were dropped per sec 15 txfifo rate the number of FIFO overruns on transmitted packets per sec 16 txcolls rate the number of collisions that were detecte per sec 17 txcarr rate the number of carrier errors on transmitted packets per sec 18 txcompr rate the number of compressed packets transmitted per sec 19 ttpcks rate the total number of packets (received + transmitted) per sec 20 ttKB rate the total number of KBytes (received + transmitted) per sec Any ideas ? Many thanks, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 16:29:23 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5A4EE49 for ; Fri, 8 Aug 2014 16:29:23 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99D5E28FC for ; Fri, 8 Aug 2014 16:29:22 +0000 (UTC) Received: from [73.34.117.227] (helo=ilsoft.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1XFn2f-000Ct5-13; Fri, 08 Aug 2014 16:29:21 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id s78GTJwG021730; Fri, 8 Aug 2014 10:29:19 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19gyFo3HSqvOX7pIeXLrPLT X-Authentication-Warning: paranoia.hippie.lan: Host revolution.hippie.lan [172.22.42.240] claimed to be [172.22.42.240] Subject: Re: disk and NIC io statistics via sysctl From: Ian Lepore To: Stefan Parvu In-Reply-To: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> Content-Type: text/plain; charset="us-ascii" Date: Fri, 08 Aug 2014 10:29:18 -0600 Message-ID: <1407515358.56408.374.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 16:29:23 -0000 On Fri, 2014-08-08 at 18:40 +0300, Stefan Parvu wrote: > Hi, > > Fetching CPU and Memory statistics, using sysctl interface and perl seems nice and easy. > I made great progress with sysrec a generic recorder reporting: cpu mem disk and nic > stats on FreeBSD. Now Im getting into disk and nic sections: > > 1. Disk > Im able to get a list of all disks on a system using kern.disks sysctl variable. > I want to be able to report per disk something like: > > readreq rate read requests per second, rate > writereq rate write requests per second, rate > totreq rate total read + write requests per second, rate > readbyt rate read bytes per second, in KB, rate > writebyt rate write bytes per second, in KB, rate > totbyt rate total read + write bytes per second, in KB, rate > > iostat seems the simplest way to fetch these, but how about sysctl interface ? > I could not figure out any sysctl parameters for disk throughput. How one will > fetch throughput per disk via sysctl ? > > > 2. NIC > Same will go for network card interfaces. I want to fetch per NIC throughput and > errors via sysctl, something like these: > > 1 timestamp time seconds since Epoch > 2 interface number NIC name, string > 3 rxKB rate the number of KBytes received per sec > 4 rxpcks rate the no. of packets received per sec > 5 rxerrs rate the number of errors while received packets per sec > 6 rxdrop rate the number of packets that were dropped per sec > 7 rxfifo rate the number of FIFO overruns on received packets per sec > 8 rxframe rate the number of carrier errors on received packet per sec > 9 rxcompr rate the number of compressed packets received per sec > 10 rxmulti rate the number of multicast packets received per sec > 11 txKB rate the number of KBytes transmitted per sec > 12 txpcks rate the number of packets transmitted per sec > 13 txerrs rate the number of errors transmitting packets per sec > 14 txdrop rate the number of packets that were dropped per sec > 15 txfifo rate the number of FIFO overruns on transmitted packets per sec > 16 txcolls rate the number of collisions that were detecte per sec > 17 txcarr rate the number of carrier errors on transmitted packets per sec > 18 txcompr rate the number of compressed packets transmitted per sec > 19 ttpcks rate the total number of packets (received + transmitted) per sec > 20 ttKB rate the total number of KBytes (received + transmitted) per sec > > > Any ideas ? > > Many thanks, > All of this series of questions you've been asking really have the same answer: look at the source code for the very apps you cite as examples of outputting the info you want, and do what they do. There are no magic secret kernel backdoor interfaces, all these userland tools are using documented interfaces such as sysctl to get their info. (There may be a few miscreants that open /dev/kmem and rudely poke around in kernel memory, but I'm not sure we have any of them in base. The lsof tool in ports is one that comes to mind for that.) In addition to the tools you've already mentioned that have the info you want, have a look at gstat for IO stats, netstat for net throughput, and systat for lots of stuff. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 17:04:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0288FD91 for ; Fri, 8 Aug 2014 17:04:51 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id CFDFA2DC9 for ; Fri, 8 Aug 2014 17:04:50 +0000 (UTC) Received: from [192.168.1.2] (senat1-01.HML3.ScaleEngine.net [209.51.186.5]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 63D0937802 for ; Fri, 8 Aug 2014 17:04:49 +0000 (UTC) Message-ID: <53E50347.9050206@freebsd.org> Date: Fri, 08 Aug 2014 13:05:11 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: disk and NIC io statistics via sysctl References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> <1407515358.56408.374.camel@revolution.hippie.lan> In-Reply-To: <1407515358.56408.374.camel@revolution.hippie.lan> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="an0ffcWd34E6W8B0AJeTWj207CHANw89X" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 17:04:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --an0ffcWd34E6W8B0AJeTWj207CHANw89X Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-08-08 12:29, Ian Lepore wrote: > On Fri, 2014-08-08 at 18:40 +0300, Stefan Parvu wrote: >> Hi,=20 >> >> Fetching CPU and Memory statistics, using sysctl interface and perl se= ems nice and easy. >> I made great progress with sysrec a generic recorder reporting: cpu me= m disk and nic=20 >> stats on FreeBSD. Now Im getting into disk and nic sections: >> >> 1. Disk=20 >> Im able to get a list of all disks on a system using kern.disks sysctl= variable. >> I want to be able to report per disk something like: >> >> readreq rate read requests per second, rate >> writereq rate write requests per second, rate >> totreq rate total read + write requests per second, rate >> readbyt rate read bytes per second, in KB, rate >> writebyt rate write bytes per second, in KB, rate >> totbyt rate total read + write bytes per second, in KB, rate=20 >> >> iostat seems the simplest way to fetch these, but how about sysctl int= erface ? >> I could not figure out any sysctl parameters for disk throughput. How = one will=20 >> fetch throughput per disk via sysctl ?=20 >> >> >> 2. NIC >> Same will go for network card interfaces. I want to fetch per NIC thro= ughput and >> errors via sysctl, something like these: >> >> 1 timestamp time seconds since Epoch >> 2 interface number NIC name, string >> 3 rxKB rate the number of KBytes received per sec >> 4 rxpcks rate the no. of packets received per sec >> 5 rxerrs rate the number of errors while received packets per sec >> 6 rxdrop rate the number of packets that were dropped per sec >> 7 rxfifo rate the number of FIFO overruns on received packets per s= ec >> 8 rxframe rate the number of carrier errors on received packet per = sec >> 9 rxcompr rate the number of compressed packets received per sec >> 10 rxmulti rate the number of multicast packets received per sec >> 11 txKB rate the number of KBytes transmitted per sec >> 12 txpcks rate the number of packets transmitted per sec >> 13 txerrs rate the number of errors transmitting packets per sec >> 14 txdrop rate the number of packets that were dropped per sec >> 15 txfifo rate the number of FIFO overruns on transmitted packets p= er sec >> 16 txcolls rate the number of collisions that were detecte per sec >> 17 txcarr rate the number of carrier errors on transmitted packets = per sec >> 18 txcompr rate the number of compressed packets transmitted per se= c >> 19 ttpcks rate the total number of packets (received + transmitted)= per sec >> 20 ttKB rate the total number of KBytes (received + transmitted) pe= r sec=20 >> >> >> Any ideas ? >> >> Many thanks, >> >=20 > All of this series of questions you've been asking really have the same= > answer: look at the source code for the very apps you cite as examples= > of outputting the info you want, and do what they do. There are no > magic secret kernel backdoor interfaces, all these userland tools are > using documented interfaces such as sysctl to get their info. (There > may be a few miscreants that open /dev/kmem and rudely poke around in > kernel memory, but I'm not sure we have any of them in base. The lsof > tool in ports is one that comes to mind for that.) >=20 > In addition to the tools you've already mentioned that have the info yo= u > want, have a look at gstat for IO stats, netstat for net throughput, an= d > systat for lots of stuff. >=20 > -- Ian >=20 >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 I often use net/nload from ports to monitor interfaces, it may be helpful as well. --=20 Allan Jude --an0ffcWd34E6W8B0AJeTWj207CHANw89X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJT5QNJAAoJEJrBFpNRJZKfoo4P/09cqFl8U5tU6Ua1ccjvA8Gp ADD7NiLbRR8gq5xhHWTPNYY5LGsaT34PWXqlArxq4Hx/DCDsor15kY1UnVGyFh1+ IR4sCHHoZOYmdx9SS7zqJKBTp/6nYYS5a8DpPXIm2JS3IFnIcPrmZWviRWUCglNj XrrKlApqrBNOq8oZTBt1Oyum8WTdblfsLVS84ucJy31jk/gPIPX0xJS7VecuNWkz c484Kk/cLWyzz9Cb1Mq2ikWAs1nFCC8DSWdEu0OiowRlKLreDVZJC9j8qlpUMcsl zeUvBw7TRsHqi9gLOLhv0YHnLeeFe1wvVwbTFhKgdssvu/YpZDOlBDvJBz75MVJR 2wdXRhF3Uqf/cd3nXCXrIKedwj4nIDBOxNo60UK/M3fW6QooYo14vDwzQgjryZIP NyQFHIjixuHSZOY/1eOhqkVMBhnaLZFDCfxDp6ES9MVSHg34Mkh7j9CB3yzF148K Bjuzon3N/XfMH3It4kaqKiJL6k7P+XksDrcDcfHT6yIVEq1u3+HnG5tNB+ktmK7g SUtDcXTIb4aAevDYuVVXsRgt4tWclrvmTWuDM7buBb7oAhOmmEgo8g52b8R3Dt9w UCbqDjQRe4r3K4iY7dMz68LtLjBkykq0Ad7SMa2BuPgjGUWJlcV3C5IaEr4DmEKr lzfi23p1WsIZduBHs04P =xuDu -----END PGP SIGNATURE----- --an0ffcWd34E6W8B0AJeTWj207CHANw89X-- From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 18:18:27 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F08BDC76; Fri, 8 Aug 2014 18:18:27 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A7AC274D; Fri, 8 Aug 2014 18:18:26 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s78IG1Nr023133 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 8 Aug 2014 18:16:02 GMT Date: Fri, 8 Aug 2014 21:18:14 +0300 From: Stefan Parvu To: Ian Lepore Subject: Re: disk and NIC io statistics via sysctl Message-Id: <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> In-Reply-To: <1407515358.56408.374.camel@revolution.hippie.lan> References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> <1407515358.56408.374.camel@revolution.hippie.lan> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 18:18:28 -0000 > magic secret kernel backdoor interfaces, all these userland tools are > using documented interfaces such as sysctl to get their info. (There > may be a few miscreants that open /dev/kmem and rudely poke around in > kernel memory, but I'm not sure we have any of them in base. The lsof > tool in ports is one that comes to mind for that.) Ian, understood - no magic here. I was looking to see if there are ready sysctl structures, arrays or hashes which can package already the mentioned stats. Like kern.cp_times, a very nice thing which is hidden and undocumented. I see very big improvement in sysctl and things are much organized since FreeBSD 5. But we will need better documentation. Im on iostat now - to understand how throughput per disk gets calculated. > In addition to the tools you've already mentioned that have the info you > want, have a look at gstat for IO stats, netstat for net throughput, and > systat for lots of stuff. gstat, thanks. havent used that. I will look over iostat, netstat. Probable would be nice to have a section on sysctl man page or probable something totally new which describes cpu | mem | disk | net and kernel statistics. Cheers, -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 20:03:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05B117B; Fri, 8 Aug 2014 20:03:35 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B7D7A24F3; Fri, 8 Aug 2014 20:03:34 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s78K3XFe082105 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 8 Aug 2014 13:03:34 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s78K3XZF082104; Fri, 8 Aug 2014 13:03:33 -0700 (PDT) (envelope-from jmg) Date: Fri, 8 Aug 2014 13:03:32 -0700 From: John-Mark Gurney To: Stefan Parvu Subject: Re: disk and NIC io statistics via sysctl Message-ID: <20140808200332.GF88623@funkthat.com> Mail-Followup-To: Stefan Parvu , Ian Lepore , freebsd-hackers@freebsd.org References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> <1407515358.56408.374.camel@revolution.hippie.lan> <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 08 Aug 2014 13:03:34 -0700 (PDT) Cc: freebsd-hackers@freebsd.org, Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 20:03:35 -0000 Stefan Parvu wrote this message on Fri, Aug 08, 2014 at 21:18 +0300: > > > magic secret kernel backdoor interfaces, all these userland tools are > > using documented interfaces such as sysctl to get their info. (There > > may be a few miscreants that open /dev/kmem and rudely poke around in > > kernel memory, but I'm not sure we have any of them in base. The lsof > > tool in ports is one that comes to mind for that.) > > Ian, understood - no magic here. I was looking to see if there are ready sysctl > structures, arrays or hashes which can package already the mentioned stats. > Like kern.cp_times, a very nice thing which is hidden and undocumented. > > I see very big improvement in sysctl and things are much organized since FreeBSD 5. > But we will need better documentation. I agree... If you write some, I will clean it up and commit it... :) > Im on iostat now - to understand how throughput per disk gets calculated. > > > In addition to the tools you've already mentioned that have the info you > > want, have a look at gstat for IO stats, netstat for net throughput, and > > systat for lots of stuff. > > gstat, thanks. havent used that. I will look over iostat, netstat. Probable would be nice > to have a section on sysctl man page or probable something totally new which describes > cpu | mem | disk | net and kernel statistics. These should be described in their own page.. Putting detailed information like this is sysctl(3) is wrong... Creating a new page and cross-ref'ing them is best... Most of the sysctl(3) entries are ones that were assigned numbers, now most sysctl's are OID_AUTO, and we should be using names instead... I don't see any names used in sysctl(3)... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 20:24:53 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F1074BE; Fri, 8 Aug 2014 20:24:53 +0000 (UTC) Received: from systemdatarecorder.org (mail.systemdatarecorder.org [54.246.96.61]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "localhost" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DDFAD26F4; Fri, 8 Aug 2014 20:24:51 +0000 (UTC) Received: from nereid (84-253-211-213.bb.dnainternet.fi [84.253.211.213]) (authenticated bits=0) by systemdatarecorder.org (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id s78KMQLD023702 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 8 Aug 2014 20:22:27 GMT Date: Fri, 8 Aug 2014 23:24:43 +0300 From: Stefan Parvu To: John-Mark Gurney Subject: Re: disk and NIC io statistics via sysctl Message-Id: <20140808232443.62f36b6a36816d9e9f417a9a@systemdatarecorder.org> In-Reply-To: <20140808200332.GF88623@funkthat.com> References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> <1407515358.56408.374.camel@revolution.hippie.lan> <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> <20140808200332.GF88623@funkthat.com> Organization: systemdatarecorder.org X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 20:24:53 -0000 > If you write some, I will clean it up and commit it... :) I will do that after im done with all 5 data recorders and my findings. Would be very nice to have even better documentation. Linux is a mess. FreeBSD is more organized and structured. > These should be described in their own page.. Putting detailed > information like this is sysctl(3) is wrong... Creating a new page > and cross-ref'ing them is best... Most of the sysctl(3) entries are > ones that were assigned numbers, now most sysctl's are OID_AUTO, and > we should be using names instead... I don't see any names used in > sysctl(3)... ok, a new page describing all these things would be super. I will take care of that. -- Stefan Parvu From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 8 22:24:48 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 108A69AA for ; Fri, 8 Aug 2014 22:24:48 +0000 (UTC) Received: from nm2-vm7.bullet.mail.ir2.yahoo.com (nm2-vm7.bullet.mail.ir2.yahoo.com [212.82.96.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E4B72683 for ; Fri, 8 Aug 2014 22:24:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s2048; t=1407536678; bh=96iobyb79G2S9rpxE6XuC23foh2WQyGJJiDEkp8H/mo=; h=Received:Received:Received:DKIM-Signature:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Date:From:To:Subject:Message-Id:In-Reply-To:References:X-Mailer:Disposition-Notification-To:Mime-Version:Content-Type:Content-Transfer-Encoding; b=n1Z49oFJg+Hw7kU5kKhfFRzsw+bxxyTEqqIKzY+nzVgB7Jpk7WviJXRT8jTkw24OLunImcvWCRnyg6LUv8RKwYDMETXPKg0A8QFzaKA3hA89kPyY1V3ndweMHY3CdyveToaRcNuz6WjwggXje3dnq7qcOZbPBWpxElaBajhC6nytvJgLDErgtzJDDIvnRLtFTmFveluuRxGKSk4H+IbVOU4M5x4NQ83+7kbbgjoekq27sknkv6MnosrHy6usQMVyjsSZxZac5ZD+ZRLXmMwSFk594xLmeZ0YASeom6+2j55MVcOPnKn1046AQwoSq8Y1eT/P5GDt8m+js4CU6Ie8bg== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.es; b=sYTYiZNUMgsXz81XnWrK/IlktcsSGsZdglFOza8q3A1yHiFwYhcB6vqjA5eHJQXSHjkP3fIfiIulBi/zS5DE3wJgqHJJjmPrXUZVISdAyjRIHkJ6n7L/ZbzSOnjpblLC7Hb/0HqKOScGsSEgpN8sxcxQ6rg9TyDjJpy+XKHAVZnia1YbCwbdY8BFznSl/WIOUYIuzZpHEG+BK6qK0xCdH3eR1NtMH0v6LnqnFQWG6LBo0FtpKplLPfFq0TJdgwoy8cu+Gfl8CSd0A/0PKkoULN6Tyj7WWUHGr7T6QWySVuwrQS62/GIRXcriSzlYqYDhMfTn9LTB4ZXpDsUYwpngDg==; Received: from [212.82.98.56] by nm2.bullet.mail.ir2.yahoo.com with NNFMP; 08 Aug 2014 22:24:38 -0000 Received: from [46.228.39.75] by tm9.bullet.mail.ir2.yahoo.com with NNFMP; 08 Aug 2014 22:24:38 -0000 Received: from [127.0.0.1] by smtp112.mail.ir2.yahoo.com with NNFMP; 08 Aug 2014 22:24:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s1024; t=1407536678; bh=96iobyb79G2S9rpxE6XuC23foh2WQyGJJiDEkp8H/mo=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Date:From:To:Subject:Message-Id:In-Reply-To:References:X-Mailer:Disposition-Notification-To:Mime-Version:Content-Type:Content-Transfer-Encoding; b=ENVovDzdHtUdmY/tw+Qmjn1A7yfErhu9HqwLdryUMIM98ZKnYNqWvt5nfuHIUFRvFHrpmvkVUL+4+XTA/JdKqzGe3DgC0IubR5tK6w0f0LI/BecbFG+gT0opgi/vJMKOh1Ck04fEoS/Ql7PlacGCM2mn4ZeDvgjoN7VLd61RWu4= X-Yahoo-Newman-Id: 148971.76298.bm@smtp112.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: k.Zy_LcVM1mXB53R6I9cJzI2y3AffoJJLSiM2RB0OxAIzap 3TNfK0EuInE1Djclqv.dZScUOx3TgszYVMeeI3kbLxZQJUQJcNcEFYxPADLZ 5k3PCr1QTQuOzubBEbSUcOfCZBS1Ty7yt5yo6Oa3tBI_Y7PlYE52RmzUrnTN ZIN62Zp_DrEJJloluyAcHuCrhSn8RcvvL2Vn3eOYhg7aFWTst0N6khsVNpJ2 2VwazKMy1AwaYcxdMzvAupY91pxkDEJrg14W8Why9s6Iy0UFqh1ycC3qJISF eysB8JgDVHjvsDo9B7RqY9e_KQTEEhvZqLz5XVyNXx8q6v5EwoQQwftC0DMf zascwVxIWuJVHpnpvBtvUA9CFrGdLo1FUT5FHn9kO2LJFQzmdt_yl0KmLAzx bdRS95lSh3D0jPgk8EexoWuXl67sFiDWK3UZTdidcmQiCB75lWwgSqthKqtk I2GYwANp9mS2mX9Mu6f4cK_8mIQqy_OeYup3Zk353eWY44gScwu1XB04xhYY - X-Yahoo-SMTP: mX392iiswBAeJNdO_s.EW62LZDJR Date: Sat, 9 Aug 2014 00:24:36 +0200 From: Eduardo Morras To: freebsd-hackers@freebsd.org Subject: Re: disk and NIC io statistics via sysctl Message-Id: <20140809002436.33396bfc2fa1a3c9015da052@yahoo.es> In-Reply-To: <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> References: <20140808184021.537feca9b15e3a261ea27fa7@systemdatarecorder.org> <1407515358.56408.374.camel@revolution.hippie.lan> <20140808211814.e14706bd0949b7a1a7827785@systemdatarecorder.org> X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd9.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 22:24:48 -0000 On Fri, 8 Aug 2014 21:18:14 +0300 Stefan Parvu wrote: > gstat, thanks. havent used that. I will look over iostat, netstat. > Probable would be nice to have a section on sysctl man page or > probable something totally new which describes cpu | mem | disk | net > and kernel statistics. systat is another good candidate, specially vmstat page (press ':' and type 'vmstat' + enter) > > Cheers, --- --- Eduardo Morras From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 9 00:56:11 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53C6CB8 for ; Sat, 9 Aug 2014 00:56:11 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D070B24FC for ; Sat, 9 Aug 2014 00:56:10 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id z11so4445069lbi.3 for ; Fri, 08 Aug 2014 17:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ry6bnQl5BPkYZ8Wk/j8+rHfSUhcArIO7KJpoWCvEm2c=; b=GVSA82BxTYmIOmcQareUZc+KDSWavY15ff305H0+tvarcSyemawrbWb6AgA7aicWO1 DejIu5OyNlVSPzgxtwiq+GogX41bpSz9ojEe5w1V5zUcwXOGjbKjQ7JAmUwEElR1EULX pnOEO+RKJqMFk1x+/zDfpJEKGKy+0gJuCOwaEyg/dOZv4zJgamzKR4vLIDTEnmopuE5N O6cqBVh//7HDD+yTiNmohe54dIIbz5g2Yc8nwhwc6M9AYBhGuBNIWOikXV1K9/iGGGxP YQs87bXlyNqMeRIQVSuyuxI+fbpcY2nNj83oCs1s7GsfOQKN42NlzSXPy8F+CXnacu8/ mIrA== MIME-Version: 1.0 X-Received: by 10.152.25.170 with SMTP id d10mr24258679lag.37.1407545768017; Fri, 08 Aug 2014 17:56:08 -0700 (PDT) Sender: hiren.panchasara@gmail.com Received: by 10.114.81.73 with HTTP; Fri, 8 Aug 2014 17:56:07 -0700 (PDT) In-Reply-To: <53E4ACF4.4030001@yandex.ru> References: <53E3C44C.9000500@delphij.net> <53E4A6E7.3080309@yandex.ru> <53E4ACF4.4030001@yandex.ru> Date: Fri, 8 Aug 2014 17:56:07 -0700 X-Google-Sender-Auth: jd3xv01FNn2ZzAHLg_TLIW2WUtY Message-ID: Subject: Re: Where is loader.conf.gz? From: hiren panchasara To: "Andrey V. Elsukov" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Xin LI X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 00:56:11 -0000 On Fri, Aug 8, 2014 at 3:56 AM, Andrey V. Elsukov wrote: > On 08.08.2014 14:31, Andrey V. Elsukov wrote: >> On 08.08.2014 01:46, hiren panchasara wrote: >>> /* pass file name to the different filesystem open routines */ >>> besterror = ENOENT; >>> for (i = 0; file_system[i] != NULL; i++) { >>> fs = file_system[i]; >>> error = (fs->fo_open)(file, f); <-----zf_open() is called >>> which returns ENOENT >>> if (error == 0) >>> goto ok; >>> if (error != EINVAL) >>> besterror = error; >>> } >>> error = besterror; >>> >>> So, the error is ENOENT at this point. But it fails in what happens after this: >> >> It looks like loader.efi should first open /boot/loader.conf via >> devopen() a bit earlier in this code. Did you see this attempt in the >> tcpdump? Can you stop loader and see what value has currdev environment >> variable? > > Sorry, devopen() doesn't open a file, but nfs_fsops should be tried > first. Maybe this code has been failed in nfs_open? > > #ifndef __i386__ > if (strcmp(f->f_dev->dv_name, "net") != 0) > return(EINVAL); > #else I never reach this path actually. I am not sure if I need to provide any special LOADER_*_SUPPORT while building the loader.efi. I need to look closer. Btw, on loader prompt: currdev and loaddev is net0. (and not "net") Complete set: Type '?' for a list of commands, 'help' for more detailed help. OK show LINES=24 beastie_disable=YES boot.netif.gateway=192.168.2.1 boot.netif.hwaddr=00:13:20:fe:30:af boot.netif.ip=192.168.2.30 boot.netif.netmask=255.255.255.0 boot.nfsroot.nfshandle=X09e70638de2da5770a000400000000003cb674000000000000000000X boot.nfsroot.nfshandlelen=28 boot.nfsroot.path=/nfsroot boot.nfsroot.server=192.168.2.216 bootfile=kernel console=efi currdev=net0: dhcp.dhcp-lease-time=300 dhcp.dhcp-message-type=5 dhcp.dhcp-server-identifier=192.168.2.1 dhcp.root-path=/nfsroot dhcp.routers=192.168.2.1 dhcp.subnet-mask=255.255.255.0 dhcp.tags=53,54,51,1,3,17 hint.acpi_throttle.0.disabled=1 hint.atkbd.0.at=atkbdc hint.atkbd.0.irq=1 hint.atkbdc.0.at=isa hint.atkbdc.0.port=0x060 hint.atrtc.0.at=isa hint.atrtc.0.irq=8 hint.atrtc.0.port=0x70 hint.attimer.0.at=isa hint.attimer.0.irq=0 hint.attimer.0.port=0x40 hint.fd.0.at=fdc0 hint.fd.0.drive=0 hint.fd.1.at=fdc0 hint.fd.1.drive=1 hint.fdc.0.at=isa hint.fdc.0.drq=2 hint.fdc.0.irq=6 hint.fdc.0.port=0x3F0 hint.p4tcc.0.disabled=1 hint.ppc.0.at=isa hint.ppc.0.irq=7 hint.psm.0.at=atkbdc hint.psm.0.irq=12 hint.sc.0.at=isa hint.sc.0.flags=0x100 hint.uart.0.at=isa hint.uart.0.flags=0x10 hint.uart.0.irq=4 hint.uart.0.port=0x3F8 hint.uart.1.at=isa hint.uart.1.irq=3 hint.uart.1.port=0x2F8 hint.wbwd.0.at=isa interpret=OK kernel=kernel kernel_options= kernelname=/boot/kernel/kernel loaddev=net0: loader_conf_files=/boot/device.hints /boot/loader.conf /boot/loader.conf.local module_path=/boot/kernel prompt=${interpret} cheers, Hiren From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 9 01:59:38 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E2FB8DB for ; Sat, 9 Aug 2014 01:59:38 +0000 (UTC) Received: from elf.torek.net (50-73-42-1-utah.hfc.comcastbusiness.net [50.73.42.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2FCAE2972 for ; Sat, 9 Aug 2014 01:59:37 +0000 (UTC) Received: from elf.torek.net (localhost [127.0.0.1]) by elf.torek.net (8.14.5/8.14.5) with ESMTP id s791VeDx049988 for ; Fri, 8 Aug 2014 19:31:40 -0600 (MDT) (envelope-from torek@torek.net) Message-Id: <201408090131.s791VeDx049988@elf.torek.net> From: Chris Torek To: freebsd-hackers@freebsd.org Subject: crash in bpf catchpacket() code MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <49986.1407547900.1@elf.torek.net> Date: Fri, 08 Aug 2014 19:31:40 -0600 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (elf.torek.net [127.0.0.1]); Fri, 08 Aug 2014 19:31:40 -0600 (MDT) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 01:59:38 -0000 We saw a crash with this stack trace (I snipped out the "frame"s here as not very useful to others, plus there's no full dump, just a text dump backtrace and some other bits): bcopy() at bcopy+0x16 bpf_mtap() at bpf_mtap+0x1d0 ether_nh_input() at ether_nh_input+0x167 netisr_dispatch_src() at netisr_dispatch_src+0x5e igb_rxeof() at igb_rxeof+0x56d igb_msix_que() at igb_msix_que+0x101 The bpf_mtap() pc is actually in (inlined) catchpacket(), where we do: bpf_append_bytes(d, d->bd_sbuf, curlen, ...) where the bd_bufmode is BPF_BUFMODE_BUFFER so this becomes bpf_buffer_append_bytes() which becomes bcopy(), and it appears the whole mess of calls has been inlined. The crash clearly has a NULL d->bd_sbuf: bcopy+0x16: repe movsq (%rsi),%es:(%rdi) rsi 0xfffff82044dbd768 rdi 0 (%es is not very interesting). This means d->bd_sbuf was NULL, which was a bit of a mystery, as the code path starts with a lock assertion: BPFD_LOCK_ASSERT(d); and then has this bit: if (d->bd_fbuf == NULL) { /* * There's no room in the store buffer, and no * prospect of room, so drop the packet. Notify * the * buffer model. */ bpf_buffull(d); ++d->bd_dcount; return; } before doing this: ROTATE_BUFFERS(d); (the ROTATE causes bd_fbuf to move into bd_sbuf). But then I realized that it did this extra step in between these "test fbuf for NULL" and "ROTATE" steps: while (d->bd_hbuf_in_use) mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, PRINET, "bd_hbuf", 0); So, if we assume that the hbuf (hold buffer) *is* in use, we'll sleep waiting for the user to finish with it and then wake us up. While we're asleep, we'll give up d->bd_lock. It sure looks like someone else (some other bpf consumer using the same d->bd_* fields) snuck in while we were asleep and used up d->bd_fbuf, setting it to NULL. Then we got the lock and did our own ROTATE_BUFFERS() and moved the NULL to d->bd_sbuf. If this analysis is correct, the fix is simply to wait for the hbuf to be available *before* checking d->bd_fbuf, i.e., move the while loop up. Because the bug is (apparently) really hard to hit (it's not like we can reproduce this at will), I'm not 100% convinced this is the fix (or the entire fix). But here it is in patch form... Chris --- bpf: check d->bd_fbuf after sleep, not before The code in catchpacket() checks that there's a valid d->bd_fbuf before doing a ROTATE_BUFFERS, which will move the sbuf (store buffer) to the hbuf (hold buffer) and make the fbuf (free buffer) become the sbuf. OK so far, but *after* it verifies this fbuf, it then mtx_sleep-waits for the hold buffer to be available. If the fbuf goes NULL during the wait when we drop the lock, there will be no sbuf once we do the ROTATE_BUFFERS. To fix this, simply check fbuf after possibly waiting for hbuf, rather than before. diff --git a/sys/net/bpf.c b/sys/net/bpf.c --- a/sys/net/bpf.c +++ b/sys/net/bpf.c @@ -2352,6 +2352,9 @@ catchpacket(struct bpf_d *d, u_char *pkt #endif curlen = BPF_WORDALIGN(d->bd_slen); if (curlen + totlen > d->bd_bufsize || !bpf_canwritebuf(d)) { + while (d->bd_hbuf_in_use) + mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, + PRINET, "bd_hbuf", 0); if (d->bd_fbuf == NULL) { /* * There's no room in the store buffer, and no @@ -2362,9 +2365,6 @@ catchpacket(struct bpf_d *d, u_char *pkt ++d->bd_dcount; return; } - while (d->bd_hbuf_in_use) - mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, - PRINET, "bd_hbuf", 0); ROTATE_BUFFERS(d); do_wakeup = 1; curlen = 0; From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 9 07:49:57 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD7BC6A7 for ; Sat, 9 Aug 2014 07:49:57 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E10A2A80 for ; Sat, 9 Aug 2014 07:49:57 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id la4so9504214vcb.19 for ; Sat, 09 Aug 2014 00:49:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5M8wuZzo1xR3UL9qUUdwsCMTxddU9m/X2vGBZ0v9ikc=; b=XwqZZoGxL5+39vFME/Fo6gDfWmI5OzmA6+b/hspNwV75FBZ4WuDspvFeLtavVoavxI jK7tY5KlOCCDj75rZkvAtwNsC15fR6PUXENo1eChnvkGKPSdiDG3Z24f9bC+yOuNegt5 wha73wSFqPsEwUYkjlvbZX+t162ukTixUVdety9lucv8X6EphvL8nh4t3/dO56sv0lb3 Pt8/xcobVNo6UHeG/uRGLWyuzbPXOOdBfMUSsjdm1u7e1XR7Nno9AEGfl2SyUxU0SlTx y/jlSuLv9jMUCuil2wcEarKqBb9sWBragLVAuPLv8d53jCrRZbHrtctofB+Gumw4OZtv XZsA== MIME-Version: 1.0 X-Received: by 10.220.49.10 with SMTP id t10mr25870404vcf.34.1407570596594; Sat, 09 Aug 2014 00:49:56 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.220.186.193 with HTTP; Sat, 9 Aug 2014 00:49:56 -0700 (PDT) In-Reply-To: <201408090131.s791VeDx049988@elf.torek.net> References: <201408090131.s791VeDx049988@elf.torek.net> Date: Sat, 9 Aug 2014 00:49:56 -0700 X-Google-Sender-Auth: zYEZVbWm-2AoGfTg18swvRvS2TY Message-ID: Subject: Re: crash in bpf catchpacket() code From: Adrian Chadd To: Chris Torek Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 07:49:57 -0000 Hi! Would you mind submitting a PR for this? You've done all the great work needed to chase this down; I'd hate for it to be forgotten! -a On 8 August 2014 18:31, Chris Torek wrote: > We saw a crash with this stack trace (I snipped out the "frame"s > here as not very useful to others, plus there's no full dump, just > a text dump backtrace and some other bits): > > bcopy() at bcopy+0x16 > bpf_mtap() at bpf_mtap+0x1d0 > ether_nh_input() at ether_nh_input+0x167 > netisr_dispatch_src() at netisr_dispatch_src+0x5e > igb_rxeof() at igb_rxeof+0x56d > igb_msix_que() at igb_msix_que+0x101 > > The bpf_mtap() pc is actually in (inlined) catchpacket(), where > we do: > > bpf_append_bytes(d, d->bd_sbuf, curlen, ...) > > where the bd_bufmode is BPF_BUFMODE_BUFFER so this becomes > bpf_buffer_append_bytes() which becomes bcopy(), and it appears > the whole mess of calls has been inlined. > > The crash clearly has a NULL d->bd_sbuf: > > bcopy+0x16: repe movsq (%rsi),%es:(%rdi) > > rsi 0xfffff82044dbd768 > rdi 0 > > (%es is not very interesting). This means d->bd_sbuf was NULL, > which was a bit of a mystery, as the code path starts with a lock > assertion: > > BPFD_LOCK_ASSERT(d); > > and then has this bit: > > if (d->bd_fbuf == NULL) { > /* > * There's no room in the store buffer, and no > * prospect of room, so drop the packet. Notify > * the > * buffer model. > */ > bpf_buffull(d); > ++d->bd_dcount; > return; > } > > before doing this: > > ROTATE_BUFFERS(d); > > (the ROTATE causes bd_fbuf to move into bd_sbuf). > > But then I realized that it did this extra step in between these > "test fbuf for NULL" and "ROTATE" steps: > > while (d->bd_hbuf_in_use) > mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, > PRINET, "bd_hbuf", 0); > > So, if we assume that the hbuf (hold buffer) *is* in use, we'll > sleep waiting for the user to finish with it and then wake us up. > While we're asleep, we'll give up d->bd_lock. > > It sure looks like someone else (some other bpf consumer using the > same d->bd_* fields) snuck in while we were asleep and used up > d->bd_fbuf, setting it to NULL. Then we got the lock and did our > own ROTATE_BUFFERS() and moved the NULL to d->bd_sbuf. > > If this analysis is correct, the fix is simply to wait for the > hbuf to be available *before* checking d->bd_fbuf, i.e., move the > while loop up. > > Because the bug is (apparently) really hard to hit (it's not like > we can reproduce this at will), I'm not 100% convinced this is the > fix (or the entire fix). But here it is in patch form... > > Chris > > --- > > bpf: check d->bd_fbuf after sleep, not before > > The code in catchpacket() checks that there's a valid d->bd_fbuf > before doing a ROTATE_BUFFERS, which will move the sbuf (store > buffer) to the hbuf (hold buffer) and make the fbuf (free buffer) > become the sbuf. OK so far, but *after* it verifies this fbuf, > it then mtx_sleep-waits for the hold buffer to be available. If > the fbuf goes NULL during the wait when we drop the lock, there > will be no sbuf once we do the ROTATE_BUFFERS. > > To fix this, simply check fbuf after possibly waiting for hbuf, > rather than before. > > > diff --git a/sys/net/bpf.c b/sys/net/bpf.c > --- a/sys/net/bpf.c > +++ b/sys/net/bpf.c > @@ -2352,6 +2352,9 @@ catchpacket(struct bpf_d *d, u_char *pkt > #endif > curlen = BPF_WORDALIGN(d->bd_slen); > if (curlen + totlen > d->bd_bufsize || !bpf_canwritebuf(d)) { > + while (d->bd_hbuf_in_use) > + mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, > + PRINET, "bd_hbuf", 0); > if (d->bd_fbuf == NULL) { > /* > * There's no room in the store buffer, and no > @@ -2362,9 +2365,6 @@ catchpacket(struct bpf_d *d, u_char *pkt > ++d->bd_dcount; > return; > } > - while (d->bd_hbuf_in_use) > - mtx_sleep(&d->bd_hbuf_in_use, &d->bd_lock, > - PRINET, "bd_hbuf", 0); > ROTATE_BUFFERS(d); > do_wakeup = 1; > curlen = 0; > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 9 20:49:13 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFA37346 for ; Sat, 9 Aug 2014 20:49:13 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2001:470:1:2d5:26:3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id DA52B26B8 for ; Sat, 9 Aug 2014 20:49:13 +0000 (UTC) Received: from [IPv6:2601:9:8280:5fd:d5a4:20d0:8128:6903] (unknown [IPv6:2601:9:8280:5fd:d5a4:20d0:8128:6903]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 82AAA34A9E4 for ; Sat, 9 Aug 2014 13:49:13 -0700 (PDT) From: Rui Paulo Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ELF relocations on i386 Message-Id: Date: Sat, 9 Aug 2014 13:49:23 -0700 To: FreeBSD Hackers Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 20:49:14 -0000 Hi, I'm having difficulty understanding why I can't create a relocatable = i386 object file with ld: % ld -o test.o -b elf32-i386-freebsd --oformat=3Delf32-i386-freebsd -r = /usr/lib32/dtrace/drti.o=20 ld: Relocatable linking with relocations from format elf32-i386-freebsd = (/usr/lib32/dtrace/drti.o) to format elf32-i386-freebsd (test.o) is not = supported % objdump -x /usr/lib32/dtrace/drti.o /usr/lib32/dtrace/drti.o: file format elf32-i386-freebsd /usr/lib32/dtrace/drti.o architecture: i386, flags 0x00000011: HAS_RELOC, HAS_SYMS start address 0x00000000 [...] Any ideas why this isn't supported? The error message is odd. -- Rui Paulo