From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 02:19:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39CC040B; Sun, 23 Nov 2014 02:19:02 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02A1E2E1; Sun, 23 Nov 2014 02:19:02 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id fp1so7644240pdb.15 for ; Sat, 22 Nov 2014 18:19:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=WaThMR+dukbH29fNT1r4EJZvXYUHoFv/hjprvhgCeyM=; b=Q+itBFCOPFhGGTnQiqg/S/sNY1LWyRJBXPLqz4kWk+8emewA4XdejjdbYwBpQMbuny HRjMbJrTNdgIdZjV4k/ZUs2X830pj8ZvePi1FE0DZeELP7cBZzHQ7eVe6G7GHHfXhA3L g42b4X+q7EKNYob/mUyVuon4Iqc4JGv6fBBhokuEv0ltmRlDggVkUpnEx6vRrUb6dUdW GvAH3UfHpiRI7QfEHDGR81Gj06rBqR4OJ/45+cR2SC3gJL4tLcmGiVb10w5H+7Q0Yl66 Lg7OALzlUx6NJdx9PFxu0fK2+2RqoyenuKs913QupGOPDfwCc7QFhM3BAx7+EpfrAlhk hWbA== X-Received: by 10.69.16.99 with SMTP id fv3mr20258545pbd.43.1416709141581; Sat, 22 Nov 2014 18:19:01 -0800 (PST) Received: from raichu ([198.244.104.6]) by mx.google.com with ESMTPSA id oq6sm8557860pdb.45.2014.11.22.18.19.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 22 Nov 2014 18:19:00 -0800 (PST) Sender: Mark Johnston Date: Sat, 22 Nov 2014 18:18:56 -0800 From: Mark Johnston To: Rui Paulo Subject: Re: DTrace: stack() does not print kernel module functions for i386 Message-ID: <20141123021856.GA54708@raichu> References: <20141109093632.GV53947@kib.kiev.ua> <9011F920-3092-4E61-9CDC-68FD9092BB7D@me.com> <201411131336.12334.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Shrikanth Kamath , freebsd-hackers@freebsd.org, avg@freebsd.org, Konstantin Belousov , freebsd-dtrace@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 02:19:02 -0000 On Thu, Nov 13, 2014 at 07:49:27PM -0800, Rui Paulo wrote: > On Nov 13, 2014, at 10:36, John Baldwin wrote: > > Why have the #ifdef? In theory other platforms besides amd64 could use > > sys/kern/link_elf_obj.c. It doesn't hurt to just let the code always accept > > both ET_DYN and ET_REL does it? > > No, it doesn't hurt. The suggested patch doesn't seem quite right; there are other functions in dt_module.c with the same assignment (i.e. "is_elf_obj = ehdr.e_type == ET_REL"), but the same modification is not correct in all cases - fixing it everywhere breaks stack() again - and "is_elf_obj" seems like the wrong name if DSOs are counted as well. The root of the problem is that dmp->dm_*_va offsets don't have the kld load address taken into account on i386, since they're currently set based only on the ELF section addresses. This is handled by dmp->dm_reloc_offset for symbols, but that's a separate case. When is_elf_obj is true, we include the load address when setting the dmp->dm_*_va fields. I suggest we do that unconditionally, and only set elements of dmp->dm_sec_offsets if is_elf_obj is true. This fixes the bug for me on i386. Any opinions? Thanks, -Mark diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c index e3905c1..9dd52b5 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c @@ -1211,13 +1211,13 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat *k_stat) #if defined(__FreeBSD__) if (sh.sh_size == 0) continue; - if (is_elf_obj && (sh.sh_type == SHT_PROGBITS || - sh.sh_type == SHT_NOBITS)) { + if (sh.sh_type == SHT_PROGBITS || sh.sh_type == SHT_NOBITS) { alignmask = sh.sh_addralign - 1; mapbase += alignmask; mapbase &= ~alignmask; sh.sh_addr = mapbase; - dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; + if (is_elf_obj) + dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; mapbase += sh.sh_size; } #endif From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 17:11:05 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0014CD42 for ; Sun, 23 Nov 2014 17:11:04 +0000 (UTC) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89555C32 for ; Sun, 23 Nov 2014 17:11:04 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id y19so10330482wgg.21 for ; Sun, 23 Nov 2014 09:11:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=7zR3dz+TaSYYjW2oQBCTLSh0uSwKJEDImSSHmMp+iqI=; b=RWGDU6YQaaupkCI45b6IHifQ/+OD54JUslEkqwFZozbsXxW8GoG8Vb2t3dpC9Y5u/W 9SjmsgFzjG4c0cU4MWnhd5fO92ltT8o4XLyj50J8c2kTMIpspiUlknHaiu7LL30//Gs+ 13CpL8NvM1etzXFoApg9bIwWMpFzh8YZWcctDvmzsitAeV/OiaGCTm3HK8GpI9QoqhFC 88b4mlqev8rmkpokK2O/3a7VNi2ja2E33ICMdBKMIXUbK38ULGHAEtpNuRSUDnEWNrzW Xsgqe33CfXS4M2Mg2Ax/+SzVg9ZAmpoM7avW5Vg4IeMYWmeS5wiPFatfXDGissx6rjFB tjDg== MIME-Version: 1.0 X-Received: by 10.180.109.194 with SMTP id hu2mr14533167wib.24.1416762662899; Sun, 23 Nov 2014 09:11:02 -0800 (PST) Received: by 10.180.8.7 with HTTP; Sun, 23 Nov 2014 09:11:02 -0800 (PST) Date: Sun, 23 Nov 2014 18:11:02 +0100 Message-ID: Subject: top -d1 behavior From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 17:11:05 -0000 Hi hackers, While writing a small script for a friend, I run into a peculiar top behavior: top -d1 shows only one 'iteration' of the information screen, but in it, the CPU percentages line is not well formed: there are 5 columns with the '%' symbol, but no values are shown. This only happens for -d1 and this behavior is deliberately done in the sources (see 'dostates' variable). Also we are using pc_cp_time from struct pcpu as the output for the kern.cp_time sysctl (used by top to know the time spent by the cpu in every state), and we are using the same fields in linprocfs for the 'stat' file in linux emulation (so a linux top version would read and print out those values). My point is: why are we doing this? If we remove that constraint, top would show the values for -d1. I don't know if they would be really accurate, but not printing anything doesn't seem a nice behavior either (especially when this is not documented in the man page) Thanks in advance. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 17:45:57 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADD6C1C3 for ; Sun, 23 Nov 2014 17:45:57 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 76652F6A for ; Sun, 23 Nov 2014 17:45:56 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id sANHjtaf001726; Sun, 23 Nov 2014 09:45:55 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id sANHjtkl001725; Sun, 23 Nov 2014 09:45:55 -0800 (PST) (envelope-from david) Date: Sun, 23 Nov 2014 09:45:55 -0800 From: David Wolfskill To: Fernando =?iso-8859-1?Q?Apestegu=EDa?= Subject: Re: top -d1 behavior Message-ID: <20141123174555.GB1228@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Fernando =?iso-8859-1?Q?Apestegu=EDa?= , FreeBSD Hackers References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="JYK4vJDZwFMowpUq" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 17:45:57 -0000 --JYK4vJDZwFMowpUq Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Nov 23, 2014 at 06:11:02PM +0100, Fernando Apestegu=EDa wrote: > Hi hackers, >=20 > While writing a small script for a friend, I run into a peculiar top beha= vior: >=20 > top -d1 shows only one 'iteration' of the information screen, but in > it, the CPU percentages line is not well formed: there are 5 columns > with the '%' symbol, but no values are shown. This only happens for > -d1 and this behavior is deliberately done in the sources (see > 'dostates' variable). > ... > My point is: why are we doing this? If we remove that constraint, top > would show the values for -d1. I don't know if they would be really > accurate, but not printing anything doesn't seem a nice behavior > either (especially when this is not documented in the man page) > .... At the time of the inital display, the program only has access to the counters (from kern.cp_time) from one instant. In order to calculate the relative proportion of CPU time spent in each of the 5 states, it is necessary to determine the extent to which those counters changed over an interval. If we call the time when the first sample is taken "T0" and the time when the second sample is takne "T1", we get something like: Time user nice sys intr idle T0 user_0 nice_0 sys_0 intr_0 idle_0 T1 user_1 nice_1 sys_1 intr_1 idle_1 To determine the relative proportions for the interval from T0:T1, we first determnine the differences (or deltas): user_delta <=3D=3D user_1 - user_0 nice_delta <=3D=3D nice_1 - nice_0 sys_delta <=3D=3D sys_1 - sys_0 intr_delta <=3D=3D intr_1 - intr_0 idle_delta <=3D=3D idle_1 - idle_0 Then sum the deltas: Interval_total =3D user_delta + nice_delta + sys_delta + intr_delta + idle_= delta Then, for each of user, nice, sys, intr, and idle, the percentage for the interval is 100 * _delta / Interval_total. Here's a sample (produced by a little shell script): albert(10.1-S)[6] get_sys_info -N 0 -c 2 -w 10 kern.cp_time time:1416763949 kern.cp_time:20934 0 28872 5664 1027225=20 time:1416763959 kern.cp_time:20980 0 28933 5674 1029686=20 Given those numbers, we have: Time user nice sys intr idle 1416763949 20934 0 28872 5664 1027225 1416763959 20980 0 28933 5674 1029686 The differences: user nice sys intr idle 46 0 61 10 2461 The total of the deltas is 46+0+61+10+2461 =3D=3D> 2578. The percentages: user nice sys intr idle 1.78 0.00 2.37 0.39 95.46 (System's uptime, in this case, was a little over an hour -- I had just updated in to stable/10 @r274909.) Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --JYK4vJDZwFMowpUq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJUch1SXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7MPsP/ivKBYwNjc5qPhW/55eJz/w9 LHa03wTPo7AXIS2eS+RxLSyUdQADa8Vv6Z/DfOjfUQnSG3V+92YyjLSQNS9KPPbu DCWikSWirMbe2V+Vj9dUusxX2HFaQushViK8lD52ZMwNXehsIqkTg8WgyyFH6MUh 8Ss2sSn1choU25ZSlhs/vcLz8rBKLc8UQ+uiXTkC0ARD80xhRizF9NbzKZcBAd2W 4Eq5rjobDrPLGrlLqcvyAjzfRu94Pv0MVpSFnwiobLfcig5pBEVx9G+EsLLL/FQu 63T4M4t9NRwsyuRDtVuDj60PtXkGumGi+YpznldLjL1nqL9m3wNrKcyoDp+91+FC hjZ4Y2BFYi+1oEGAvUyUJExj6WESCbESfPMRdHWnIPgl/NoRHosPomPdQDAZueZm NwrjHr4M/1iW7iiO4wtwikwbG3KVZaRTmBHgu8C78hwrvhogQshJua9bGdaMFKZS E464Fpyn6iZQVH1ucWo6kQP5qp9G/Dc10IJxpmRNsiMJvTcMGplSe8+GMncW2Rl5 K80Kjs0W2/61kATsgiOzgE4Wrixpu90ewIx8bdLwTNxpthV7PhKVDj5OmFAWy1jo T8tGA9fWaly72AMoEJhSoOn6wSq6eYpIBKG15PBH2eXR+im4cs67syfmFMO502/I ehcZZUOG8FO8osPUceKa =tRby -----END PGP SIGNATURE----- --JYK4vJDZwFMowpUq-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 19:34:33 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F0FEC17 for ; Sun, 23 Nov 2014 19:34:33 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EE4CC3B for ; Sun, 23 Nov 2014 19:34:33 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id b13so10650215wgh.17 for ; Sun, 23 Nov 2014 11:34:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=nEBZx1rMUl64DjyhrkgneVDbsXF3hwp4iNeSdDSv0pk=; b=XZSJYVIpEchd/FIkHfz+ZC6PnAui8gnBxEzgw/QHN3nqp/TRQXTf78uxalKuVRFpZm QAqLOk7x5TDYHtSxRsZu+yyh79joMXdzdRomuXm43fszVGFU1EBFMU5NfGiiwZCkIA9R SRmVkSNg11t26BGSwKMzABEHnOWljSllDDlv9JbyA95fFtJXabRmuw9dilgHgOhNl+C7 HOnT5hzfZEsxDAdO9B4qLxSXob0wcAykppzvIO7LlrBpeBUHCbbFegnkvlbZte9lgdKO 6ykNMmVKz9q/AO4yQf0cRhyV63CW79tsdj5ThP4ZYEovhibXwNMAhYM+8uBNjpm+KdxZ Tn7A== MIME-Version: 1.0 X-Received: by 10.180.211.79 with SMTP id na15mr9324410wic.36.1416771271069; Sun, 23 Nov 2014 11:34:31 -0800 (PST) Received: by 10.180.8.7 with HTTP; Sun, 23 Nov 2014 11:34:31 -0800 (PST) In-Reply-To: <20141123174555.GB1228@albert.catwhisker.org> References: <20141123174555.GB1228@albert.catwhisker.org> Date: Sun, 23 Nov 2014 20:34:31 +0100 Message-ID: Subject: Re: top -d1 behavior From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= To: David Wolfskill , =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= , FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 19:34:33 -0000 On Sun, Nov 23, 2014 at 6:45 PM, David Wolfskill wro= te: > On Sun, Nov 23, 2014 at 06:11:02PM +0100, Fernando Apestegu=C3=ADa wrote: >> Hi hackers, >> >> While writing a small script for a friend, I run into a peculiar top beh= avior: >> >> top -d1 shows only one 'iteration' of the information screen, but in >> it, the CPU percentages line is not well formed: there are 5 columns >> with the '%' symbol, but no values are shown. This only happens for >> -d1 and this behavior is deliberately done in the sources (see >> 'dostates' variable). >> ... >> My point is: why are we doing this? If we remove that constraint, top >> would show the values for -d1. I don't know if they would be really >> accurate, but not printing anything doesn't seem a nice behavior >> either (especially when this is not documented in the man page) >> .... > > At the time of the inital display, the program only has access to the > counters (from kern.cp_time) from one instant. > > In order to calculate the relative proportion of CPU time spent in each > of the 5 states, it is necessary to determine the extent to which those > counters changed over an interval. > > If we call the time when the first sample is taken "T0" and the time > when the second sample is takne "T1", we get something like: > > Time user nice sys intr idle > T0 user_0 nice_0 sys_0 intr_0 idle_0 > T1 user_1 nice_1 sys_1 intr_1 idle_1 > > To determine the relative proportions for the interval from T0:T1, we > first determnine the differences (or deltas): > > user_delta <=3D=3D user_1 - user_0 > nice_delta <=3D=3D nice_1 - nice_0 > sys_delta <=3D=3D sys_1 - sys_0 > intr_delta <=3D=3D intr_1 - intr_0 > idle_delta <=3D=3D idle_1 - idle_0 > > Then sum the deltas: > > Interval_total =3D user_delta + nice_delta + sys_delta + intr_delta + idl= e_delta > > Then, for each of user, nice, sys, intr, and idle, the percentage for > the interval is > > 100 * _delta / Interval_total. > > Here's a sample (produced by a little shell script): > > albert(10.1-S)[6] get_sys_info -N 0 -c 2 -w 10 kern.cp_time > time:1416763949 kern.cp_time:20934 0 28872 5664 1027225 > time:1416763959 kern.cp_time:20980 0 28933 5674 1029686 > > Given those numbers, we have: > > Time user nice sys intr idle > 1416763949 20934 0 28872 5664 1027225 > 1416763959 20980 0 28933 5674 1029686 > > The differences: > user nice sys intr idle > 46 0 61 10 2461 > > The total of the deltas is 46+0+61+10+2461 =3D=3D> 2578. > > The percentages: > user nice sys intr idle > 1.78 0.00 2.37 0.39 95.46 > > > (System's uptime, in this case, was a little over an hour -- I had just > updated in to stable/10 @r274909.) OK, but if we use -d1, T0 is 0 since cp_time, cp_old and cp_diff are all declared as static, so we should not have any problems with just having one lecture of the kern.cp_time sysctl and doing all the calculation above. Am I right? That's in fact what top in linux (from the procps package) does. They have the CPU_t struct declared as static. top in Linux is able to show CPU statistics for number of iterations =3D=3D 1 but we are not. Cheers. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 20:07:01 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEFAF331 for ; Sun, 23 Nov 2014 20:07:01 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id A78A9EF2 for ; Sun, 23 Nov 2014 20:07:01 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 6CDDE741BD for ; Sun, 23 Nov 2014 20:07:00 +0000 (UTC) Message-ID: <54723E7E.8080509@freebsd.org> Date: Sun, 23 Nov 2014 15:07:26 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: top -d1 behavior References: <20141123174555.GB1228@albert.catwhisker.org> In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="84eH9cpPAC0AlhsLFSx4XV5lirn4Cc6mq" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 20:07:01 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --84eH9cpPAC0AlhsLFSx4XV5lirn4Cc6mq Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2014-11-23 14:34, Fernando Apestegu=C3=ADa wrote: > On Sun, Nov 23, 2014 at 6:45 PM, David Wolfskill = wrote: >> On Sun, Nov 23, 2014 at 06:11:02PM +0100, Fernando Apestegu=C3=ADa wro= te: >>> Hi hackers, >>> >>> While writing a small script for a friend, I run into a peculiar top = behavior: >>> >>> top -d1 shows only one 'iteration' of the information screen, but in >>> it, the CPU percentages line is not well formed: there are 5 columns >>> with the '%' symbol, but no values are shown. This only happens for >>> -d1 and this behavior is deliberately done in the sources (see >>> 'dostates' variable). >>> ... >>> My point is: why are we doing this? If we remove that constraint, top= >>> would show the values for -d1. I don't know if they would be really >>> accurate, but not printing anything doesn't seem a nice behavior >>> either (especially when this is not documented in the man page) >>> .... >> >> At the time of the inital display, the program only has access to the >> counters (from kern.cp_time) from one instant. >> >> In order to calculate the relative proportion of CPU time spent in eac= h >> of the 5 states, it is necessary to determine the extent to which thos= e >> counters changed over an interval. >> >> If we call the time when the first sample is taken "T0" and the time >> when the second sample is takne "T1", we get something like: >> >> Time user nice sys intr idle >> T0 user_0 nice_0 sys_0 intr_0 idle_0 >> T1 user_1 nice_1 sys_1 intr_1 idle_1 >> >> To determine the relative proportions for the interval from T0:T1, we >> first determnine the differences (or deltas): >> >> user_delta <=3D=3D user_1 - user_0 >> nice_delta <=3D=3D nice_1 - nice_0 >> sys_delta <=3D=3D sys_1 - sys_0 >> intr_delta <=3D=3D intr_1 - intr_0 >> idle_delta <=3D=3D idle_1 - idle_0 >> >> Then sum the deltas: >> >> Interval_total =3D user_delta + nice_delta + sys_delta + intr_delta + = idle_delta >> >> Then, for each of user, nice, sys, intr, and idle, the percentage for >> the interval is >> >> 100 * _delta / Interval_total. >> >> Here's a sample (produced by a little shell script): >> >> albert(10.1-S)[6] get_sys_info -N 0 -c 2 -w 10 kern.cp_time >> time:1416763949 kern.cp_time:20934 0 28872 5664 1027225 >> time:1416763959 kern.cp_time:20980 0 28933 5674 1029686 >> >> Given those numbers, we have: >> >> Time user nice sys intr idle >> 1416763949 20934 0 28872 5664 1027225 >> 1416763959 20980 0 28933 5674 1029686 >> >> The differences: >> user nice sys intr idle >> 46 0 61 10 2461 >> >> The total of the deltas is 46+0+61+10+2461 =3D=3D> 2578. >> >> The percentages: >> user nice sys intr idle >> 1.78 0.00 2.37 0.39 95.46 >> >> >> (System's uptime, in this case, was a little over an hour -- I had jus= t >> updated in to stable/10 @r274909.) >=20 > OK, but if we use -d1, T0 is 0 since cp_time, cp_old and cp_diff are > all declared as static, so we should not have any problems with just > having one lecture of the kern.cp_time sysctl and doing all the > calculation above. Am I right? >=20 > That's in fact what top in linux (from the procps package) does. They > have the CPU_t struct declared as static. top in Linux is able to show > CPU statistics for number of iterations =3D=3D 1 but we are not. >=20 > Cheers. >=20 >> >> Peace, >> david >> -- >> David H. Wolfskill david@catwhisker.org >> Taliban: Evil cowards with guns afraid of truth from a 14-year old gir= l. >> >> See http://www.catwhisker.org/~david/publickey.gpg for my public key. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 You would either need to make top sleep for 1 second and then calculate the delta, or make it calculate the average of each time by taking the counter and dividing it by the uptime. Neither seem like what the user would expect. --=20 Allan Jude --84eH9cpPAC0AlhsLFSx4XV5lirn4Cc6mq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUcj6CAAoJEJrBFpNRJZKfGX4P/j9VBa3IfmuUyiv74vqFAyJr 3IYR7ZmNmGMWvBPGYdhABejrKu3zasc+YsLrFnRZ9K4Qp1c8XBAO+l4uEc9lwX2b 0qZE2ZnBjDhpdQGkq8K73Kz18piOExcPcDY9MAG+WmdyIvB+VH0LU1ILFddqMyg0 Maq2AQJprQ6omXPjJElZygkWi+57xmiE/X0+QJn2R5gsJbAV+UASZLVJ2mE2RGiI oOJOahqsysT7XeKk6tII51NsOVVixjNMoMrFPtFBiP8uxwiqlhzhhJldKE/G6IkQ E26PztFhDcdpc2gYJoITFVs2Ry4h6wMhtshbTocKtbBd2kt+lDKv15nfkaQnppjC 986iGxmEnjo6hBn/wQ9L8hBAze+wwL+gS9UobgOP9CSUiowSUeM7Qb6czTnfNq10 Sp06mxNHmp0LJY5+2OUIAbGhjeU015L5dzCskpUFOcw2pNrhEorVyC1GYHDgkEKa +OBx+Mc/tRDOh+Qv//QS69QOcXXhRUTRmSfr33DzrJtCluTt68YVDjgTkOhZwGF3 0zYLdJF4tJiOWyqGP8RBJBuCzoJ/ed1qfK8V7FTsIdSoqsZNFZjhEEtibMF0qunm 5PHqXwOW6zOAobtKy4wTKxGwCHZfHmoLndI4o5OTVB7aRefcFP8qLwuP6LADoTw4 q1rfvDcB9zAAll6tllr/ =n4Rv -----END PGP SIGNATURE----- --84eH9cpPAC0AlhsLFSx4XV5lirn4Cc6mq-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 23 21:57:36 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E1B98BD; Sun, 23 Nov 2014 21:57:36 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92F6EC73; Sun, 23 Nov 2014 21:57:35 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id r20so4020355wiv.8 for ; Sun, 23 Nov 2014 13:57:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=mWCZhQnMA3mCVuCtcYYp1W0SnYq0W0afeR+a7E+hI+8=; b=ILv7HEKZI8ywse+KkXQmF8xt2r32xZdr0U6lDaVsfzlaaXlp0L+JCO1Nedt4XPg/zL CaEjLjNlgVW8UrPgUft+n1izKPcOhZswRK5qS8xfMxzZACj3lJlQa2VCoBdioMkFuflJ fF6lYKOmaXVmRYD/TNfQ020mKecZhwsuxv9qfRqx8LTVOzNAhZVr3M3Drd8Sn3smUuaZ 8SIy0nTCML/p8ae7Y5FeT/ATPmIeeZW5S5lWO8jCYQ2YljOFsY1SLeDNtmiKUq1dlDhI kuDrtb6UXxVxtSXfydt5iYQvDXbQ2kJpuGc82+T/blKPqUz4ZforT6dR53+l/9l8YJW4 UJpg== MIME-Version: 1.0 X-Received: by 10.180.186.40 with SMTP id fh8mr15751777wic.40.1416779853965; Sun, 23 Nov 2014 13:57:33 -0800 (PST) Received: by 10.180.8.7 with HTTP; Sun, 23 Nov 2014 13:57:33 -0800 (PST) In-Reply-To: <54723E7E.8080509@freebsd.org> References: <20141123174555.GB1228@albert.catwhisker.org> <54723E7E.8080509@freebsd.org> Date: Sun, 23 Nov 2014 22:57:33 +0100 Message-ID: Subject: Re: top -d1 behavior From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= To: Allan Jude Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2014 21:57:36 -0000 On Sun, Nov 23, 2014 at 9:07 PM, Allan Jude wrote: > On 2014-11-23 14:34, Fernando Apestegu=C3=ADa wrote: >> On Sun, Nov 23, 2014 at 6:45 PM, David Wolfskill = wrote: >>> On Sun, Nov 23, 2014 at 06:11:02PM +0100, Fernando Apestegu=C3=ADa wrot= e: >>>> Hi hackers, >>>> >>>> While writing a small script for a friend, I run into a peculiar top b= ehavior: >>>> >>>> top -d1 shows only one 'iteration' of the information screen, but in >>>> it, the CPU percentages line is not well formed: there are 5 columns >>>> with the '%' symbol, but no values are shown. This only happens for >>>> -d1 and this behavior is deliberately done in the sources (see >>>> 'dostates' variable). >>>> ... >>>> My point is: why are we doing this? If we remove that constraint, top >>>> would show the values for -d1. I don't know if they would be really >>>> accurate, but not printing anything doesn't seem a nice behavior >>>> either (especially when this is not documented in the man page) >>>> .... >>> >>> At the time of the inital display, the program only has access to the >>> counters (from kern.cp_time) from one instant. >>> >>> In order to calculate the relative proportion of CPU time spent in each >>> of the 5 states, it is necessary to determine the extent to which those >>> counters changed over an interval. >>> >>> If we call the time when the first sample is taken "T0" and the time >>> when the second sample is takne "T1", we get something like: >>> >>> Time user nice sys intr idle >>> T0 user_0 nice_0 sys_0 intr_0 idle_0 >>> T1 user_1 nice_1 sys_1 intr_1 idle_1 >>> >>> To determine the relative proportions for the interval from T0:T1, we >>> first determnine the differences (or deltas): >>> >>> user_delta <=3D=3D user_1 - user_0 >>> nice_delta <=3D=3D nice_1 - nice_0 >>> sys_delta <=3D=3D sys_1 - sys_0 >>> intr_delta <=3D=3D intr_1 - intr_0 >>> idle_delta <=3D=3D idle_1 - idle_0 >>> >>> Then sum the deltas: >>> >>> Interval_total =3D user_delta + nice_delta + sys_delta + intr_delta + i= dle_delta >>> >>> Then, for each of user, nice, sys, intr, and idle, the percentage for >>> the interval is >>> >>> 100 * _delta / Interval_total. >>> >>> Here's a sample (produced by a little shell script): >>> >>> albert(10.1-S)[6] get_sys_info -N 0 -c 2 -w 10 kern.cp_time >>> time:1416763949 kern.cp_time:20934 0 28872 5664 1027225 >>> time:1416763959 kern.cp_time:20980 0 28933 5674 1029686 >>> >>> Given those numbers, we have: >>> >>> Time user nice sys intr idle >>> 1416763949 20934 0 28872 5664 1027225 >>> 1416763959 20980 0 28933 5674 1029686 >>> >>> The differences: >>> user nice sys intr idle >>> 46 0 61 10 2461 >>> >>> The total of the deltas is 46+0+61+10+2461 =3D=3D> 2578. >>> >>> The percentages: >>> user nice sys intr idle >>> 1.78 0.00 2.37 0.39 95.46 >>> >>> >>> (System's uptime, in this case, was a little over an hour -- I had just >>> updated in to stable/10 @r274909.) >> >> OK, but if we use -d1, T0 is 0 since cp_time, cp_old and cp_diff are >> all declared as static, so we should not have any problems with just >> having one lecture of the kern.cp_time sysctl and doing all the >> calculation above. Am I right? >> >> That's in fact what top in linux (from the procps package) does. They >> have the CPU_t struct declared as static. top in Linux is able to show >> CPU statistics for number of iterations =3D=3D 1 but we are not. >> >> Cheers. >> >>> >>> Peace, >>> david >>> -- >>> David H. Wolfskill david@catwhisker.org >>> Taliban: Evil cowards with guns afraid of truth from a 14-year old girl= . >>> >>> See http://www.catwhisker.org/~david/publickey.gpg for my public key. >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g" >> > > You would either need to make top sleep for 1 second and then calculate Not really. The computation is possible. Do you mean we would need two readings of cp_time to be _accurate_? We state in the man page something like this: "As with ps(1), things can change while top is collecting information for an update. The picture it gives is only a close approximation to reality" And in any case, why 1 second? The default interval between updates is 2 seconds. This can be overridden with -s but the flag is ignored if -d1 is present so we don't have a way to let the user decide for how long he/she wants to wait until the result is calculated. > the delta, or make it calculate the average of each time by taking the > counter and dividing it by the uptime. > > Neither seem like what the user would expect. Agreed. But this is mostly unexpected (and can lead scripts to fail): $ top -d1 last pid: 1013; load averages: 0.54, 0.59, 0.43 up 0+00:11:33 22:39:27 37 processes: 2 running, 35 sleeping CPU: % user, % nice, % system, % interrupt, % idle Mem: 327M Active, 169M Inact, 98M Wired, 1984K Cache, 76M Buf, 375M Free Swap: 3615M Total, 3615M Free What I would suggest is this: - Change top so it can show results for cpu states even for -d1 _and_ specifically state in the man page that in this case, the result can be specially inaccurate due to 'cold start' effect. - If the previous thing doesn't seem reasonable, state in the man page that the CPU states line is kind of "broken" when -d1 is specified. Cheers. > > -- > Allan Jude > From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 16:50:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94236BF0 for ; Mon, 24 Nov 2014 16:50:35 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C926DD2 for ; Mon, 24 Nov 2014 16:50:35 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id x19so9326231ier.13 for ; Mon, 24 Nov 2014 08:50:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=1oo6Rq98AM3joUCiS785GMj6z3eK0X19W4LKTZmVK2g=; b=0Zbu9ZMnxNML+ieS4QETAPBEX5MKwiagJHOOmlkc9HAnvh2ZWgHHiiGfWDZVdWdSJw T2Qglxy2bgJH1IRZap1v+x/bPTd21Hg06H+vaB/SIdurOPeLujfOAhfQkeuTJaod+hUe l1OOR+oCgDLeFTiAiN5IAMTVSHTQQckRXpiPtZgFaMAU5P1QzOD2TOEabqAB+huV+gC7 /zlS5nB0Q/iExPGW5pNSU17kWg+qdEfkTZLtULwa1DkXirc94fs2g7fi02yN9It7d7IY UEH2ZI10IKDUi0fnbqRozGlX+GwBnOQZMzbAqeDYKpT+O50uzQBO/KhgsEr1INr8jxnk aZ8g== X-Received: by 10.107.40.205 with SMTP id o196mr19052510ioo.26.1416847834686; Mon, 24 Nov 2014 08:50:34 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.29.141 with HTTP; Mon, 24 Nov 2014 08:50:14 -0800 (PST) In-Reply-To: <6a4308e3ce1d4835be5caef3f96b0c50@tedunangst.com> References: <6a4308e3ce1d4835be5caef3f96b0c50@tedunangst.com> From: Ed Maste Date: Mon, 24 Nov 2014 11:50:14 -0500 X-Google-Sender-Auth: PhWflPlOYwMPObckMrIbGlD1haM Message-ID: Subject: Re: fix base64.c "overrun" To: Ted Unangst Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 16:50:35 -0000 On 22 November 2014 at 17:09, Ted Unangst wrote: > From inspection, it appears FreeBSD's libc base64.c will sometimes > erroneously fail to decode a base64 string into a precisely sized > buffer. The overflow check is a little too greedy. The same bug was > fixed in OpenBSD; it would be helpful if FreeBSD adopted the fix as > well. :) > > http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/net/base64.c.diff?r1=1.6&r2=1.7 Thanks for the heads-up Ted, I'll take care of getting the fix in. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 16:56:11 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BDE9D8A for ; Mon, 24 Nov 2014 16:56:11 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C53FEBB for ; Mon, 24 Nov 2014 16:56:10 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sAOGu0eO014273 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 24 Nov 2014 08:56:02 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <5473631B.80801@freebsd.org> Date: Tue, 25 Nov 2014 00:55:55 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Dieter BSD , freebsd-hackers@freebsd.org Subject: Re: Running 8 building 10 part2: ctfmerge hangs References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 16:56:11 -0000 On 11/20/14, 3:22 PM, Dieter BSD wrote: > After thinking about this a bit, it looks like ctfmerge is broken. > 8.2 ctfmerge running under 8.2 kernel. Use a bogus argument and > it complains. (So it *is* running before it goes into a coma.) > But with normal arguments it quickly falls asleep. Thus it is broken. yes there was a period in history when ctfmerge was broken. there was afix but you cou;dn;t compile it because ctfmerge had WITH_CTF=1 in its own build and failed :-) I forget the fix unfortunately. but it hot me some time back. there IS a fix but I can't remember it > > But... if I run it under gdb, it says program exited normally: > > (gdb) run -v -L VERSION -o kernel.debug.verbose cam.o > Starting program: /new/freebsd/tmp/ctfmerge -v -L VERSION -o > kernel.debug.verbose cam.o > (no debugging symbols found)...(no debugging symbols found)... > (no debugging symbols found)...(no debugging symbols found)... > (no debugging symbols found)...[New LWP 100342] > (no debugging symbols found)...(no debugging symbols found)... > [New Thread 800e041c0 (LWP 100342)] > [New Thread 800e0ae40 (LWP 100405)] > [Thread 800e0ae40 (LWP 100405) exited] > GFun: 7 SFun: 1 GVar: 4 SVar: 9 T 73 SOU: 76 > > Program exited normally. > (gdb) > > And the kernel file shrank from 117MB to 20MB, so it did something. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 17:20:06 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82B3881D for ; Mon, 24 Nov 2014 17:20:06 +0000 (UTC) Received: from mail-yk0-f172.google.com (mail-yk0-f172.google.com [209.85.160.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46439263 for ; Mon, 24 Nov 2014 17:20:05 +0000 (UTC) Received: by mail-yk0-f172.google.com with SMTP id 131so4350119ykp.31 for ; Mon, 24 Nov 2014 09:19:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=OReJ4DG471ZPZ0U3t9X4ULVwIwFtoMeNaarUITbMjiQ=; b=dnrFua2gul3aLzaJlZ/+OSZ67loPFs+QM3gvS3jJoxp6M1WLeiF0p8DnU3jVCH9lZH 93zEMZRl+N5cJ37iR7IEGVcKsvqIOQManN/+53rXWLX7zgzIXMIbYLX7Kb0Lwhc9KCfv KBR1SKEi09ByPI6nNH+24GxR4l9DIcr41tNfxA2xosKQTQke8Y3rS1CILniA0Kjeu2W3 IP02ZfLG5EHXmLLEG5LUpGaYc5PX8CkIc6i1gIAEKXr9uMC2E7RL8l75eE+t95H7VzrF e+r5EoSxo3/smQgjpPlwy73+h1bLIX4ANtSz+/60pMIjvpHdWwSi1oyChbchvB4hogam XEsg== X-Gm-Message-State: ALoCoQlD+RmDjmcMzMCkr7BFWo9P2rYsXyo1ocbrQbLhsv7UWOBdxGerhXGFjyPLIeR0dHJhiTPv MIME-Version: 1.0 X-Received: by 10.170.98.135 with SMTP id p129mr12145318yka.101.1416849187284; Mon, 24 Nov 2014 09:13:07 -0800 (PST) Received: by 10.170.46.213 with HTTP; Mon, 24 Nov 2014 09:13:07 -0800 (PST) X-Originating-IP: [80.95.88.175] In-Reply-To: References: <6a4308e3ce1d4835be5caef3f96b0c50@tedunangst.com> Date: Mon, 24 Nov 2014 18:13:07 +0100 Message-ID: Subject: Re: fix base64.c "overrun" From: Oliver Pinter To: Ed Maste Content-Type: text/plain; charset=UTF-8 Cc: Ted Unangst , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 17:20:06 -0000 On Mon, Nov 24, 2014 at 5:50 PM, Ed Maste wrote: > On 22 November 2014 at 17:09, Ted Unangst wrote: >> From inspection, it appears FreeBSD's libc base64.c will sometimes >> erroneously fail to decode a base64 string into a precisely sized >> buffer. The overflow check is a little too greedy. The same bug was >> fixed in OpenBSD; it would be helpful if FreeBSD adopted the fix as >> well. :) >> >> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/net/base64.c.diff?r1=1.6&r2=1.7 > > Thanks for the heads-up Ted, I'll take care of getting the fix in. Hi Ed! Check this: https://github.com/HardenedBSD/hardenedBSD/issues/68 https://github.com/HardenedBSD/hardenedBSD/commit/d6e5388b831b190a0ff39898d354ab68b9b6a965 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 19:22:29 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 763B386B; Mon, 24 Nov 2014 19:22:29 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 499533CD; Mon, 24 Nov 2014 19:22:29 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rp18so9183483iec.25 for ; Mon, 24 Nov 2014 11:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Q4G+6GWGlUFjjMU3+n+HsbKsKCd7BaorCX9H55CfNUE=; b=0rhRaV7LP2ulUNuwg/Kte8s5ZkvC57ZrHMjWFDogEYbKQTyR7lfu0+KiQfK0C37D+l K41iXqDfd5noF8QNdwchT5j7yiCUJIPVGtt0+MEtiJaBDY4acaK3IAf2f1xozpdzae9K M/kpoR5IystI6nTqIENSlpuRIc/j24FNIUfrMsczxTdk9qD51elx3c4QR/cw4UVcXKtV M4odJBm3wXzq2xrX/0h8xBEa5iIX/T1S6KZSxvIiWydQWAq6ZsEaVCKPCm1tzkM+gEjU vALfk8ogvlM22bWHmHjla/WuQRYUbbTqkoa3hEKOsk6FSGdVdXCmg7O3VIvexrVokzIo gGIw== MIME-Version: 1.0 X-Received: by 10.107.12.234 with SMTP id 103mr18700928iom.71.1416856948581; Mon, 24 Nov 2014 11:22:28 -0800 (PST) Received: by 10.50.42.162 with HTTP; Mon, 24 Nov 2014 11:22:28 -0800 (PST) In-Reply-To: <5473631B.80801@freebsd.org> References: <5473631B.80801@freebsd.org> Date: Mon, 24 Nov 2014 11:22:28 -0800 Message-ID: Subject: Re: Running 8 building 10 part2: ctfmerge hangs From: NGie Cooper To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Dieter BSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 19:22:29 -0000 On Mon, Nov 24, 2014 at 8:55 AM, Julian Elischer wrote: > On 11/20/14, 3:22 PM, Dieter BSD wrote: >> >> After thinking about this a bit, it looks like ctfmerge is broken. >> 8.2 ctfmerge running under 8.2 kernel. Use a bogus argument and >> it complains. (So it *is* running before it goes into a coma.) >> But with normal arguments it quickly falls asleep. Thus it is broken. > > yes there was a period in history when ctfmerge was broken. > there was afix but you cou;dn;t compile it because ctfmerge had > WITH_CTF=1 in its own build and failed :-) > > I forget the fix unfortunately. but it hot me some time back. > there IS a fix but I can't remember it Dieter, The easiest (and only sane) way to resolve this is to: 1. Boot a 10.1 livecd. 2. Mount all of your mountpoints properly, in /mnt (but do not chroot your install). 3. Run the standard build process with DESTDIR=/mnt in the install steps : https://www.freebsd.org/doc/handbook/makeworld.html . Don't reboot in-between. 4. Reboot when done and verify that everything's ok. Whatever happened put your system into a funky state (my guess is that the build process isn't scripted/doesn't catch errors if scripted), and you're chasing down ghosts by trying to build with a 10.1 userland and 8.2 kernel. Cheers, From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 20:25:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1228A12 for ; Mon, 24 Nov 2014 20:25:02 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7427D00 for ; Mon, 24 Nov 2014 20:25:02 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id tp5so9568903ieb.40 for ; Mon, 24 Nov 2014 12:25:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=g/DBbIINbM2SOceAdKU8PBKtssVbw9sbCtL7eKkWYKo=; b=puEOgxOYf9lcy2fvLazAeXm6Nymj2T6lZKuhp3dPHG+iYNLeZoy5R18WVc5EAfetzZ ue/H6g6LP5akeKTauzjK/pugqx1UPCDtPDbBjt1CySUa3rvrrk6JpsuZOZMSgQHv05Mn KM27dY/Ix0v0SctRY6wXNzkDRT6Q3IzJjzd33AgW+SAkreATzcW7Gky7hqlUsujFzR7w +PmTMaU2wJxWzAb/frymbzaOyOc/WiSrtrvDTuaxyxaJRSG93Sc9ybdMXCokmq1vcUjC dqLr8EUJ6+j78KR5k4+5nh1BFjuhWRs6iJhHvi5X5OPtz5UoNrInVZrfhcowA6KiZCx4 jhUA== X-Received: by 10.50.134.225 with SMTP id pn1mr13555974igb.33.1416860702123; Mon, 24 Nov 2014 12:25:02 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.29.141 with HTTP; Mon, 24 Nov 2014 12:24:41 -0800 (PST) In-Reply-To: References: <6a4308e3ce1d4835be5caef3f96b0c50@tedunangst.com> From: Ed Maste Date: Mon, 24 Nov 2014 15:24:41 -0500 X-Google-Sender-Auth: FQIuJU_nATd_AJQxP5jOtUN5Gp0 Message-ID: Subject: Re: fix base64.c "overrun" To: Oliver Pinter Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 20:25:03 -0000 On 24 November 2014 at 12:13, Oliver Pinter wrote: > > Hi Ed! > > Check this: https://github.com/HardenedBSD/hardenedBSD/issues/68 > https://github.com/HardenedBSD/hardenedBSD/commit/d6e5388b831b190a0ff39898d354ab68b9b6a965 Thanks. I had it up already in Phabricator here though: https://reviews.freebsd.org/D1218 -Ed From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 24 21:22:06 2014 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C5492DE; Mon, 24 Nov 2014 21:22:06 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E97263D7; Mon, 24 Nov 2014 21:22:04 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA08789; Mon, 24 Nov 2014 23:23:55 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Xt157-000ClE-Jl; Mon, 24 Nov 2014 23:22:01 +0200 Message-ID: <5473A156.6070502@FreeBSD.org> Date: Mon, 24 Nov 2014 23:21:26 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Mark Johnston , Rui Paulo Subject: Re: DTrace: stack() does not print kernel module functions for i386 References: <20141109093632.GV53947@kib.kiev.ua> <9011F920-3092-4E61-9CDC-68FD9092BB7D@me.com> <201411131336.12334.jhb@freebsd.org> <20141123021856.GA54708@raichu> In-Reply-To: <20141123021856.GA54708@raichu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , freebsd-hackers@FreeBSD.org, freebsd-dtrace@FreeBSD.org, Shrikanth Kamath X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2014 21:22:06 -0000 On 23/11/2014 04:18, Mark Johnston wrote: > On Thu, Nov 13, 2014 at 07:49:27PM -0800, Rui Paulo wrote: >> On Nov 13, 2014, at 10:36, John Baldwin wrote: >>> Why have the #ifdef? In theory other platforms besides amd64 could use >>> sys/kern/link_elf_obj.c. It doesn't hurt to just let the code always accept >>> both ET_DYN and ET_REL does it? >> >> No, it doesn't hurt. > > The suggested patch doesn't seem quite right; there are other functions > in dt_module.c with the same assignment (i.e. > "is_elf_obj = ehdr.e_type == ET_REL"), but the same modification is not > correct in all cases - fixing it everywhere breaks stack() again - and > "is_elf_obj" seems like the wrong name if DSOs are counted as well. > > The root of the problem is that dmp->dm_*_va offsets don't have the kld > load address taken into account on i386, since they're currently set based > only on the ELF section addresses. This is handled by > dmp->dm_reloc_offset for symbols, but that's a separate case. > > When is_elf_obj is true, we include the load address when setting the > dmp->dm_*_va fields. I suggest we do that unconditionally, and only set > elements of dmp->dm_sec_offsets if is_elf_obj is true. This fixes the > bug for me on i386. Any opinions? This totally makes sense to me. Thank you! > diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > index e3905c1..9dd52b5 100644 > --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > @@ -1211,13 +1211,13 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat *k_stat) > #if defined(__FreeBSD__) > if (sh.sh_size == 0) > continue; > - if (is_elf_obj && (sh.sh_type == SHT_PROGBITS || > - sh.sh_type == SHT_NOBITS)) { > + if (sh.sh_type == SHT_PROGBITS || sh.sh_type == SHT_NOBITS) { > alignmask = sh.sh_addralign - 1; > mapbase += alignmask; > mapbase &= ~alignmask; > sh.sh_addr = mapbase; > - dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > + if (is_elf_obj) > + dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > mapbase += sh.sh_size; > } > #endif > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 25 02:36:28 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA9C3923; Tue, 25 Nov 2014 02:36:27 +0000 (UTC) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FC59A01; Tue, 25 Nov 2014 02:36:27 +0000 (UTC) Received: by mail-ig0-f173.google.com with SMTP id r2so4281568igi.6 for ; Mon, 24 Nov 2014 18:36:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LCZxffryjKxSWWqZi7tOdjpt0mQZ8pX6UG2MprIt+qU=; b=QDjD78Ham9Hrj8C+XnAOzOSFSETjlfO9BDPPrHaQHtF2u9F11SEl9yKg2AGZsIXBGG afjgPyZTjBONSx/3hIQtEvm4ogTg8vg0BlF0iq0dOA05ObzrhyKSwKLaT8STvCr1/unJ UWwPEHhShw5mlNHB8rYqlYPKHE3EuF+8UGmC1cxnRBX4ZghOfu4+dHDyNETI6eC/TDGw xFOAnSRxtVIaILv99IzsOc62RXXb1ZyMiDndWkXLBPcgpd89msJnzCeDcy3PtQVCGtaL qZ/uPD+8IkxqgkxwPJCB6uMM6AHOoWHhDcITOi0lWdsmT+oh93uABlvcJAIpQG1awBO5 EMMw== MIME-Version: 1.0 X-Received: by 10.107.26.193 with SMTP id a184mr7247435ioa.80.1416882986858; Mon, 24 Nov 2014 18:36:26 -0800 (PST) Received: by 10.107.9.13 with HTTP; Mon, 24 Nov 2014 18:36:26 -0800 (PST) In-Reply-To: <20141123021856.GA54708@raichu> References: <20141109093632.GV53947@kib.kiev.ua> <9011F920-3092-4E61-9CDC-68FD9092BB7D@me.com> <201411131336.12334.jhb@freebsd.org> <20141123021856.GA54708@raichu> Date: Mon, 24 Nov 2014 18:36:26 -0800 Message-ID: Subject: Re: DTrace: stack() does not print kernel module functions for i386 From: Shrikanth Kamath To: Mark Johnston Content-Type: text/plain; charset=UTF-8 Cc: Rui Paulo , freebsd-hackers@freebsd.org, avg@freebsd.org, Konstantin Belousov , freebsd-dtrace@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2014 02:36:28 -0000 > The suggested patch doesn't seem quite right; there are other functions > in dt_module.c with the same assignment (i.e. > "is_elf_obj = ehdr.e_type == ET_REL"), but the same modification is not > correct in all cases - fixing it everywhere breaks stack() again - and > "is_elf_obj" seems like the wrong name if DSOs are counted as well. > > The root of the problem is that dmp->dm_*_va offsets don't have the kld > load address taken into account on i386, since they're currently set based > only on the ELF section addresses. This is handled by > dmp->dm_reloc_offset for symbols, but that's a separate case. > > When is_elf_obj is true, we include the load address when setting the > dmp->dm_*_va fields. I suggest we do that unconditionally, and only set > elements of dmp->dm_sec_offsets if is_elf_obj is true. This fixes the > bug for me on i386. Any opinions? > > Thanks, > -Mark > > diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > index e3905c1..9dd52b5 100644 > --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > @@ -1211,13 +1211,13 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat *k_stat) > #if defined(__FreeBSD__) > if (sh.sh_size == 0) > continue; > - if (is_elf_obj && (sh.sh_type == SHT_PROGBITS || > - sh.sh_type == SHT_NOBITS)) { > + if (sh.sh_type == SHT_PROGBITS || sh.sh_type == SHT_NOBITS) { > alignmask = sh.sh_addralign - 1; > mapbase += alignmask; > mapbase &= ~alignmask; > sh.sh_addr = mapbase; > - dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > + if (is_elf_obj) > + dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > mapbase += sh.sh_size; > } > #endif Hello Mark, I tested the patch and works good, did some basic testing. Marcel too pointed to the other occurrence of is_elf_obj in the file and that it should not include ET_DYN which is when I realized it was not the right thing to do, i was trying to read up more on this, guess this patch takes care of it. Thanks for fixing this. Thanks, -- Shrikanth R K From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 25 07:12:19 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 942F3290; Tue, 25 Nov 2014 07:12:19 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BF28A0D; Tue, 25 Nov 2014 07:12:19 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id eu11so11253886pac.39 for ; Mon, 24 Nov 2014 23:12:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=qiQxsLSS+mkWQlY2YMRQFChaZExiZ0GFQBXZkncBoEk=; b=AuuB382m3MPd1rntQPTJ8915sOcpoI14EjK7I7GR9pcpKYh9TxpIsD2Ew9oRokzLS+ i8Vrr4MnFS1j/mjukW9fW08p3++iKrl5yjXbZZEY81t9dJ4gte8wvDREU7UJ14gVR+dU 3ByysQCQhmPc4mBf4fH4f3DwxYbO4B+NXYxj2x78BBNO560rcfMOLrVn3NM9Y6ShZA6K Fx0hrqOg/8Ozd9JUHF6gneLuepa8zzBMeORLN2VIKBM7Znx+Nx5dbqBSnWeXRH+EvJat 5uML2fQXOlbGtrzVRtETsAou5MS2EOivEGqQ2l70A08JeCvZmpnD9XQNQzDtv3qDHbUS rzJw== X-Received: by 10.66.140.76 with SMTP id re12mr39289193pab.147.1416899538917; Mon, 24 Nov 2014 23:12:18 -0800 (PST) Received: from raichu ([198.244.104.6]) by mx.google.com with ESMTPSA id je4sm486555pbd.94.2014.11.24.23.12.17 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Nov 2014 23:12:17 -0800 (PST) Sender: Mark Johnston Date: Mon, 24 Nov 2014 23:12:12 -0800 From: Mark Johnston To: Shrikanth Kamath Subject: Re: DTrace: stack() does not print kernel module functions for i386 Message-ID: <20141125071212.GA2005@raichu> References: <20141109093632.GV53947@kib.kiev.ua> <9011F920-3092-4E61-9CDC-68FD9092BB7D@me.com> <201411131336.12334.jhb@freebsd.org> <20141123021856.GA54708@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Rui Paulo , freebsd-hackers@freebsd.org, avg@freebsd.org, Konstantin Belousov , freebsd-dtrace@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2014 07:12:19 -0000 On Mon, Nov 24, 2014 at 06:36:26PM -0800, Shrikanth Kamath wrote: > > The suggested patch doesn't seem quite right; there are other functions > > in dt_module.c with the same assignment (i.e. > > "is_elf_obj = ehdr.e_type == ET_REL"), but the same modification is not > > correct in all cases - fixing it everywhere breaks stack() again - and > > "is_elf_obj" seems like the wrong name if DSOs are counted as well. > > > > The root of the problem is that dmp->dm_*_va offsets don't have the kld > > load address taken into account on i386, since they're currently set based > > only on the ELF section addresses. This is handled by > > dmp->dm_reloc_offset for symbols, but that's a separate case. > > > > When is_elf_obj is true, we include the load address when setting the > > dmp->dm_*_va fields. I suggest we do that unconditionally, and only set > > elements of dmp->dm_sec_offsets if is_elf_obj is true. This fixes the > > bug for me on i386. Any opinions? > > > > Thanks, > > -Mark > > > > diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > > index e3905c1..9dd52b5 100644 > > --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > > +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c > > @@ -1211,13 +1211,13 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat *k_stat) > > #if defined(__FreeBSD__) > > if (sh.sh_size == 0) > > continue; > > - if (is_elf_obj && (sh.sh_type == SHT_PROGBITS || > > - sh.sh_type == SHT_NOBITS)) { > > + if (sh.sh_type == SHT_PROGBITS || sh.sh_type == SHT_NOBITS) { > > alignmask = sh.sh_addralign - 1; > > mapbase += alignmask; > > mapbase &= ~alignmask; > > sh.sh_addr = mapbase; > > - dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > > + if (is_elf_obj) > > + dmp->dm_sec_offsets[elf_ndxscn(sp)] = sh.sh_addr; > > mapbase += sh.sh_size; > > } > > #endif > > Hello Mark, > I tested the patch and works good, did some basic testing. Marcel > too pointed to the other occurrence of is_elf_obj in the file and that > it should not include ET_DYN which is when I realized it was not the > right thing to do, i was trying to read up more on this, guess this > patch takes care of it. Thanks for fixing this. I've committed it as r275011. Thanks for the report. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 25 18:48:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2812215A for ; Tue, 25 Nov 2014 18:48:02 +0000 (UTC) Received: from nm12-vm6.bullet.mail.ne1.yahoo.com (nm12-vm6.bullet.mail.ne1.yahoo.com [98.138.91.105]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D71FB618 for ; Tue, 25 Nov 2014 18:48:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1416941275; bh=6wMxnYhUOW6Cb6sVOSqI+VjBD4wad9QzWtN3hdOqK7g=; h=Date:From:Reply-To:To:Subject:From:Subject; b=NU+G1nsIdSPY6qt9X/NcCZK7VxJ3rFhc6NbZET436MtLOiXZRvSUaELI4LhWPN1nwioECcnTx4X+rBhwf6HJ/G6wg+oXnVCgN+mCJKso0BsgowSHGTms8ZFqoHAXHUhUGDpqen7eE9uK631raGldagNuwg7+/w2fU36uERXV4w2l/NnwEYFR3GbCI69pecOOfjpIIVyyYg7suoYnEFCUAdoB4+wTznwuM6Hv/ubeYzGkn5ELC+E3vhiHL/9nc7vwgOLIj+Vx6Jttc6YKgiD883ovLDU81qGpGMxGTHtytuEU0AS/ZPW+5emB22HoUGn+YT1Fxs7UFOaIDuACSI176A== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=gF/euK7K/Bo1NL/B06ijXtq7VJDPKCfd1vLbMHfo8/Q5kLVkcqA8nvemDYjWz9ixfwQNc06IZljYxzJyiJZ/pMN9d4gU4rWAGD15rcKyzOTj8KnXKqVWx9/gvcby5aMwver96Ma3qOEImvHuFhXkqY0fiHhn4ZHuFOBWV1e7MFrom0BxZGpjd8VwW2vQuxCO2A+Jkd8fQI6ENBKVwX7kDHVhOu78pNqcOjeIKKZXDmlAVZujm2T+aDqKm0WV7+Km+LYDTo76fDfH5fjUktU2404GEC58H4zBR8D9JrZUzcWwL/0Fg/U82o3m6s00IC6sRRwcwFQdMzrMIY6QJfTxXw==; Received: from [98.138.226.180] by nm12.bullet.mail.ne1.yahoo.com with NNFMP; 25 Nov 2014 18:47:55 -0000 Received: from [98.138.89.246] by tm15.bullet.mail.ne1.yahoo.com with NNFMP; 25 Nov 2014 18:47:55 -0000 Received: from [127.0.0.1] by omp1060.mail.ne1.yahoo.com with NNFMP; 25 Nov 2014 18:47:55 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 18310.76518.bm@omp1060.mail.ne1.yahoo.com X-YMail-OSG: r1B45UwVM1lPqowz_m7SzG9p_GmpKflbTqj9TM_JeWGR1vOpLuG2w.2DLIwg2pu nxmWr.Kk40PVYpMF.BKit9Kq7YAj.UejO0LVLFuZ.K610DNe8p0pul..l9qEy5tWRSY1T6_lo6x6 G9cCKGma6F0PXY4SsAB0tvL_Sy_xkrMFnrUdsLtB2kYBjE0oTQSJohZNOzWVfyLFvbPspDRAnNlT s6pFXkTEZJWrcqaDbhd_UYMRNPiyuqCHCDjsUgfMsnnXfchBZKFqH3TsYpW8uBVTOaef6WkBhwUj xfEG14hKoxQtk6TlX0OxxeXdrmlN3_VSx7DY9XeMZlkIXXjJbPnxuGMtuXXx.wLbj24zI5atqSAW KvGERVzK8HKBAReW.3NLEPqxLOZlbb2iU9VcCnxVdO.xVW7GyhQKjHWXjbT0IzLjrLQnxqIlB9XC Mb.YYh84dCTVvt2_okjf8TDOryVAF7RFg1J95w4y.P2s1Bus3B6DacIPB0SBFZt78bUE- Received: by 98.138.105.195; Tue, 25 Nov 2014 18:47:54 +0000 Date: Tue, 25 Nov 2014 18:47:52 +0000 (UTC) From: muthira mammootty Reply-To: muthira mammootty To: "freebsd-hackers@freebsd.org" Message-ID: <1298295958.570735.1416941272940.JavaMail.yahoo@jws100173.mail.ne1.yahoo.com> Subject: GPT partition scheme on a stripe MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 25 Nov 2014 19:03:21 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2014 18:48:02 -0000 I am trying to create a GPT partition scheme on a striped volume made out o= f two drives and getting error. Is this not valid? # gstripe label -v journal2 /dev/ad12 /dev/ad13# gpart create -s GPT stripe= /journal2GEOM: ad12: corrupt or invalid GPT detected.GEOM: ad12: GPT reject= ed =E2=80=94 may not be recoverable. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 25 19:07:33 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E6F9B2F for ; Tue, 25 Nov 2014 19:07:33 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 0948E8D9 for ; Tue, 25 Nov 2014 19:07:32 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 10A1D7A9B8 for ; Tue, 25 Nov 2014 19:07:26 +0000 (UTC) Message-ID: <5474D370.4080809@freebsd.org> Date: Tue, 25 Nov 2014 14:07:28 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: GPT partition scheme on a stripe References: <1298295958.570735.1416941272940.JavaMail.yahoo@jws100173.mail.ne1.yahoo.com> In-Reply-To: <1298295958.570735.1416941272940.JavaMail.yahoo@jws100173.mail.ne1.yahoo.com> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="N7DkEdcbUASbs6jgmjFHlI6dxBsEehMKG" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2014 19:07:33 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --N7DkEdcbUASbs6jgmjFHlI6dxBsEehMKG Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2014-11-25 13:47, muthira mammootty via freebsd-hackers wrote: > I am trying to create a GPT partition scheme on a striped volume made o= ut of two drives and getting error. Is this not valid? > # gstripe label -v journal2 /dev/ad12 /dev/ad13# gpart create -s GPT st= ripe/journal2GEOM: ad12: corrupt or invalid GPT detected.GEOM: ad12: GPT = rejected =E2=80=94 may not be recoverable. >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 GPT and GEOM store their metadata in the same place, the last sector of the drive. The recommended way to deal with this is to create GPT partitions on both of the raw disks, then stripe the partitions together, rather than the other way around. --=20 Allan Jude --N7DkEdcbUASbs6jgmjFHlI6dxBsEehMKG Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUdNNzAAoJEJrBFpNRJZKfRbsP/32NQYNCjfk4R89nGyQpT+1E nlmTpwTOK+TIevxBDkui3HiV8VbjebnJsZUful4N645nSpNGikcVsg1+nmaOq/ZE +eSR+GhVPwHWwUhTnL5fnumOHH1vUF73QP5T94ZAN+jCt62X0u1CwypL0QpMiyWH EWJA5XI4LTzROY0vICnQMmTdI8/MVdBD3A4MEwq92pec/STPjW6g1k7FjwoYqZpW ajkNhO1GiNNr0VDV89H3xCRkXx0Hg1wbc4c6qBwv3FnQSO4859M1dbt5Jy/NGBHp 7J1J1TxUbAORKjXbmdYZtQZsjjPPqZ5TlSiwQSveK35hYDxhzz7PBetU/WkNZXeF q9Fa0X895uvf+qp9iLaHrzYmlWP738vyGh1ItEu2oUkAV60whIyXZIV8k/DYVmIg LXIZN10+qvX7P6FUHQi/mnvR1XQzgUyNxtSVX77c0RQxqtBVYl/gUnAjewHv4mmP mvtwuE4Cp4DUK28czjUke7BKKnm7RU+FznL2YIzPppDl3quCJyReeH2skU/Z8h9g SKeJ93lQO1V/bqSVa61efwXNlsXs3IPykGyyztoiSm3ahJdC3p+soegWJ0Mmd434 JAPKp0E+Sk00bygRs4zky8VdkbjGM/6860JwVpbktVunnwaVCWGUtkqTMixb7U9c vIB1OhsoiLZwncNfz+pg =MOsY -----END PGP SIGNATURE----- --N7DkEdcbUASbs6jgmjFHlI6dxBsEehMKG-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 26 23:26:00 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35A5AB45 for ; Wed, 26 Nov 2014 23:26:00 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 112D1281 for ; Wed, 26 Nov 2014 23:25:59 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sAQNPrVG092691 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 26 Nov 2014 15:25:53 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sAQNPrH3092690; Wed, 26 Nov 2014 15:25:53 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Nov 2014 15:25:52 -0800 From: John-Mark Gurney To: Dieter BSD Subject: Re: Running 8 building 10 success (I think?) Message-ID: <20141126232552.GM99957@funkthat.com> Mail-Followup-To: Dieter BSD , freebsd-hackers@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Nov 2014 15:25:53 -0800 (PST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2014 23:26:00 -0000 Dieter BSD wrote this message on Fri, Nov 21, 2014 at 23:30 -0800: > > Well, since you've supplied plenty of sarcasm but once again no actual > > information about what problem you ran into trying to build from sources > > the right way, without crazy hackery like runing a 10.1 chroot on an 8.2 > > kernel, I guess I"m done with this. > > Sorry, my sarcasm detector must be broken today. > > What do you consider the right way? > > I tried building a kernel without doing chroot, using 8.2's toolchain. > 8.2's config failed. Built 10.1 config. Just use the stadard, documented way to upgrade your machine... If you're too lazy to read src/UPDATING or follow the documentation in the handbook, then you get these problems... Hint: you don't run config directly, you use make to build the correct toolchain and compile the kernel... Running an 8.2 toolchain to build a 10.1 kernel is unsupported... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."