From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 04:48:45 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6EA701A1 for ; Mon, 1 Apr 2013 04:48:45 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by mx1.freebsd.org (Postfix) with ESMTP id 4CA51780 for ; Mon, 1 Apr 2013 04:48:45 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id bh4so1130340pad.12 for ; Sun, 31 Mar 2013 21:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:from:date:message-id:subject:to :content-type; bh=4GZ0Fv5h3a8uIyRxgdbuv3D78vr8q1okbTsBsO8Q9y0=; b=Ixio29gE+rx6kRSuLIs4dwBWIiRDEPwuvp2hzG85XKG2PO3d8SII+jrvBswCJSy66c xdTzFTL7XTHAZyrGhS0wa63Apx5CUQKKhAf+AfA1hN9Mnj3QOgkjPAwseyjPTgod7RrO QKmX+TS1TnjtMyJbnJHCBEBPkE62UsIkRZA9o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:from:date:message-id:subject:to :content-type:x-gm-message-state; bh=4GZ0Fv5h3a8uIyRxgdbuv3D78vr8q1okbTsBsO8Q9y0=; b=VQwO/grEr4JowFKIf29LIKA/SsH8PwSE3ouNxIL703rhmd7oX5tWzkN5dMyCxZSXyK kGmvW+XfXQ7uuZ8X3PH+bubVOu31GcZL4UJZZfP5ZMAmj43BhNghxHETJ8y0Bc5y9ccI +D4Ttg+46HPZTaes63WWANLRWGGsMLC6SIwf/jJ+kTU+5SBaWqySgBZCobeT45xPwAMV 63NXOcIdf4J/Dx2HiA5A7JmZVa9ErUYSlEpXy5bWA25Iax2eFzul3krQAmQ6EbFvoPIn 7bfj1OipI/4jo1WzEBYpJBPwG2ed68DfpmnJl2g/Jauxj/9HfZ7wFnA70AqyQDqWv2E5 cP3w== X-Received: by 10.66.122.196 with SMTP id lu4mr16505157pab.129.1364791719424; Sun, 31 Mar 2013 21:48:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.86.34 with HTTP; Sun, 31 Mar 2013 21:48:08 -0700 (PDT) From: Eitan Adler Date: Mon, 1 Apr 2013 00:48:08 -0400 Message-ID: Subject: considering i386 as a tier 1 architecture To: freebsd-arch@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlfJotGa7mS6R8HKTGCnFGKpMl99Avoay+yu4uVg9JMXg+Kj8ZhZqqYNSNLqXs/14gcD3Kw X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 04:48:45 -0000 Hi, I am writing this email to discuss the i386 architecture in FreeBSD. Computers are getting faster, but also more memory intensive. I can not find a laptop with less than 4 or 8 GB of RAM. Modern browsers, such as Firefox, require a 64bit architecture and 8GB of RAM. A 32 bit platform is not enough now a days on systems with more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in the 1990s. Even in the embedded world ARM is going 64 bit with ARMv8. Secondly, the i386 port is unmaintained. Very few developers run it, so it doesn't get the testing it deserves. Almost every user post or bug report I see from a x86 compatible processor is running amd64. When was the last time you booted i386 outside a virtual machine? Often times the build works for amd64 but fails for i386. Finally, others are dropping support for i386. Windows Server 2008 is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users and downstream vendors no longer care about preserving ancient hardware. I hope this email is enough to convince you that on this date we should drop support for the i386 architecture for 10.0 to tier 2 and replace it with the ARM architecture as Tier 1. -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 05:40:02 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C6A96E4C; Mon, 1 Apr 2013 05:40:02 +0000 (UTC) (envelope-from joel@vnode.se) Received: from mail.vnode.se (mail.vnode.se [212.247.52.13]) by mx1.freebsd.org (Postfix) with ESMTP id 8CFE5B43; Mon, 1 Apr 2013 05:40:02 +0000 (UTC) Received: from mail.vnode.se (localhost [127.0.0.1]) by mail.vnode.se (Postfix) with ESMTP id 466C8E3F07A; Mon, 1 Apr 2013 07:30:44 +0200 (CEST) X-Virus-Scanned: amavisd-new at vnode.se Received: from mail.vnode.se ([127.0.0.1]) by mail.vnode.se (mail.vnode.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kcF8kYO4Na37; Mon, 1 Apr 2013 07:30:42 +0200 (CEST) Received: from devbox.vnode.local (unknown [83.223.1.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.vnode.se (Postfix) with ESMTPSA id 72BEDE3F079; Mon, 1 Apr 2013 07:30:41 +0200 (CEST) Date: Mon, 1 Apr 2013 07:30:39 +0200 From: Joel Dahl To: Eitan Adler Subject: Re: considering i386 as a tier 1 architecture Message-ID: <20130401053039.GA82735@devbox.vnode.local> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 05:40:02 -0000 On Mon, Apr 01, 2013 at 12:48:08AM -0400, Eitan Adler wrote: > Finally, others are dropping support for i386. Windows Server 2008 > is 64 bit only. No, it isn't. Windows Server 2008 comes in both 32 bit and 64 bit versions. Windows Server 2008 R2 is 64 bit only however. The same goes for Windows Server 2012. -- Joel From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 05:41:08 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D70CFFFA; Mon, 1 Apr 2013 05:41:08 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx1.freebsd.org (Postfix) with ESMTP id 894A1B64; Mon, 1 Apr 2013 05:41:08 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id hz11so2043273vcb.37 for ; Sun, 31 Mar 2013 22:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=azqr0HtDAA9x6f0IZpGYSI+n/GmPmiywAMzHvbV0vhs=; b=ysCSodKZD4bKYiLgpwYEDs18I+SYIyXEsRvp3gpoYnGGA0ndPGMRlEDwNQ9Z77EZ9p ZeFt+lv+SnkUSvD/jyk2zxbDnsZTeQWSc++N3dkUL+a/05T59X4yTgKlXRFUQs4izdbv CF/jHZbG4kaXscbmWlu5hUSCV5YaaG6BBmiXmYOiOgaaFQsSnVPa7pNi+FX64VtHNFsk I/6+0XZqIHIZfEchqP6WfhJnAD0c0biR65PN+/UUxS5namZA4tlVDS4xnEwTkzoKKWxV hmlY8aY+5DDEiEfGR5kIbhcVpM3bi3VHh4QMKrbuRQH40VTc06YBE25l0uv2+xgVfyBQ aX1w== MIME-Version: 1.0 X-Received: by 10.220.140.18 with SMTP id g18mr8263838vcu.54.1364794862694; Sun, 31 Mar 2013 22:41:02 -0700 (PDT) Received: by 10.58.132.203 with HTTP; Sun, 31 Mar 2013 22:41:02 -0700 (PDT) In-Reply-To: References: Date: Sun, 31 Mar 2013 22:41:02 -0700 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Mehmet Erol Sanliturk To: Eitan Adler Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 05:41:08 -0000 On Sun, Mar 31, 2013 at 9:48 PM, Eitan Adler wrote: > Hi, > > I am writing this email to discuss the i386 architecture in FreeBSD. > > Computers are getting faster, but also more memory intensive. I > can not find a laptop with less than 4 or 8 GB of RAM. Modern > browsers, such as Firefox, require a 64bit architecture and 8GB of > RAM. A 32 bit platform is not enough now a days on systems with > more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in > the 1990s. Even in the embedded world ARM is going 64 bit with > ARMv8. > > Secondly, the i386 port is unmaintained. Very few developers run > it, so it doesn't get the testing it deserves. Almost every user > post or bug report I see from a x86 compatible processor is running > amd64. When was the last time you booted i386 outside a virtual > machine? Often times the build works for amd64 but fails for i386. > > Finally, others are dropping support for i386. Windows Server 2008 > is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users > and downstream vendors no longer care about preserving ancient > hardware. > > I hope this email is enough to convince you that on this date we > should drop support for the i386 architecture for 10.0 to tier 2 > and replace it with the ARM architecture as Tier 1. > > -- > Eitan Adler > This idea is really very good . The FreeBSD Project man power , for me , is wasted to maintain a branch that it is NOT necessary to make it a first class branch . 1 Giga Bytes , and even 2 Giga Bytes memory chips are disappearing from the computer shops slowly . At present , there is NO any processor which is ONLY 32-bits . Not only the Windows Server , if I am not remembering incorrectly , new regular Windows ( desk top , etc. ) versions will drop 32 bits branches : They only supply 64 bits versions . By concentrating on 64 bits ( amd64 ) branch and work toward distributed processing and high performance computing for super or clustered computers or graphics chips ( cards ) is much more useful than working on 32 bits version . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 05:48:28 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6F8AB2CD; Mon, 1 Apr 2013 05:48:28 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) by mx1.freebsd.org (Postfix) with ESMTP id D6702C18; Mon, 1 Apr 2013 05:48:27 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn17so1276457wib.4 for ; Sun, 31 Mar 2013 22:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=k2ypdD9bWkASjyRd/I1ciEZY+JlceNNpc0sayriixCA=; b=G1PI4Efs5fQ+DHi8VhEVVtcK742gnnB4cfh2N/ppi6DWXHItwtax7n2W/zddFt6TNa jJkBAY0yEK5WDstQmNRpMPS/NyoaEcZV1KT/Q3tohTlpj/VLuq+/vfYvyOM93Qnufpy7 I1rL356L0KDWwaiJH99OmyzQ8hEEC9UIVmiyJM7mAva0LtvpbGNBoSXvXWuVBf9tdHJq 3V5T8gGO/CSkTCTBGnKFyfdy8P2ZFyA0lyo5zCET0Cb0/jIrKSM8iVooEwZofDVt8vmm LzsZyWRNJaYPOFgR+7iAO7KlJSr+Z4SiqtTmHWuv1YEWKIeFEsVG1cUD4qjK0RSSJB+R IU6g== MIME-Version: 1.0 X-Received: by 10.180.73.6 with SMTP id h6mr8172558wiv.27.1364795306989; Sun, 31 Mar 2013 22:48:26 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sun, 31 Mar 2013 22:48:26 -0700 (PDT) In-Reply-To: References: Date: Mon, 1 Apr 2013 08:48:26 +0300 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Kimmo Paasiala To: Eitan Adler Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 05:48:28 -0000 I think the only ones who are going to object are the users of embedded hardware. Some of them are still using CPUs that are only i586 equivalent. Personally I support the notion. -Kimmo On Mon, Apr 1, 2013 at 7:48 AM, Eitan Adler wrote: > Hi, > > I am writing this email to discuss the i386 architecture in FreeBSD. > > Computers are getting faster, but also more memory intensive. I > can not find a laptop with less than 4 or 8 GB of RAM. Modern > browsers, such as Firefox, require a 64bit architecture and 8GB of > RAM. A 32 bit platform is not enough now a days on systems with > more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in > the 1990s. Even in the embedded world ARM is going 64 bit with > ARMv8. > > Secondly, the i386 port is unmaintained. Very few developers run > it, so it doesn't get the testing it deserves. Almost every user > post or bug report I see from a x86 compatible processor is running > amd64. When was the last time you booted i386 outside a virtual > machine? Often times the build works for amd64 but fails for i386. > > Finally, others are dropping support for i386. Windows Server 2008 > is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users > and downstream vendors no longer care about preserving ancient > hardware. > > I hope this email is enough to convince you that on this date we > should drop support for the i386 architecture for 10.0 to tier 2 > and replace it with the ARM architecture as Tier 1. > > -- > Eitan Adler > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 05:50:41 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D1BA1476; Mon, 1 Apr 2013 05:50:41 +0000 (UTC) (envelope-from grog@lemis.com) Received: from w3.lemis.com (w3.lemis.com [208.86.224.149]) by mx1.freebsd.org (Postfix) with ESMTP id AC596C37; Mon, 1 Apr 2013 05:50:41 +0000 (UTC) Received: from eureka.lemis.com (1032.x.rootbsd.net [208.86.224.149]) by w3.lemis.com (Postfix) with ESMTP id DECE73B764; Mon, 1 Apr 2013 05:50:34 +0000 (UTC) Received: by eureka.lemis.com (Postfix, from userid 1004) id 7DB8AF74FA; Mon, 1 Apr 2013 16:50:31 +1100 (EST) Date: Mon, 1 Apr 2013 16:50:31 +1100 From: Greg 'groggy' Lehey To: Eitan Adler Subject: Re: considering i386 as a tier 1 architecture Message-ID: <20130401055031.GB47589@eureka.lemis.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3lcZGd9BuhuYXNfi" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Organization: The FreeBSD Project Phone: +61-3-5346-1370 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 05:50:41 -0000 --3lcZGd9BuhuYXNfi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 1 April 2013 at 0:48:08 -0400, Eitan Adler wrote: > Hi, > > I am writing this email to discuss the i386 architecture in FreeBSD. > > Computers are getting faster, but also more memory intensive. I > can not find a laptop with less than 4 or 8 GB of RAM. Modern > browsers, such as Firefox, require a 64bit architecture and 8GB of > RAM. A 32 bit platform is not enough now a days on systems with > more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in > the 1990s. Even in the embedded world ARM is going 64 bit with > ARMv8. > > Secondly, the i386 port is unmaintained. Very few developers run > it, so it doesn't get the testing it deserves. Almost every user > post or bug report I see from a x86 compatible processor is running > amd64. When was the last time you booted i386 outside a virtual > machine? Often times the build works for amd64 but fails for i386. > > Finally, others are dropping support for i386. Windows Server 2008 > is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users > and downstream vendors no longer care about preserving ancient > hardware. > > I hope this email is enough to convince you that on this date we > should drop support for the i386 architecture for 10.0 to tier 2 > and replace it with the ARM architecture as Tier 1. Nice one! And only 48 minutes into the day. I've seen a number of people take it seriously. Greg -- Sent from my desktop computer. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua --3lcZGd9BuhuYXNfi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFZICYACgkQIubykFB6QiPo5QCfbZklgxo/h4moVfwrzlmCDj3N yVwAnA2RsSCOqjI9Ot2NP8C9tdDfAKLd =eMP3 -----END PGP SIGNATURE----- --3lcZGd9BuhuYXNfi-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 05:53:37 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A7239651; Mon, 1 Apr 2013 05:53:37 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) by mx1.freebsd.org (Postfix) with ESMTP id E9B30CD8; Mon, 1 Apr 2013 05:53:36 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn17so1278676wib.10 for ; Sun, 31 Mar 2013 22:53:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=tGUA89l9lgpIC8jtd17hs1hpbja2ZhmeXPeStSgiAvA=; b=HJTE/yNreWmzsaLLfDdDCqddf5c8U+eoDhVlKpRVZa3gPrYg43omM7YNdlhw43538j RSPCNlVwOB2pp9PSPT0DZhhQwJx9AaPVPPYIfDhrG61ZD4a5uhA6kzIYbI1qrHxW0aM2 qANoGcpbJQdXV1dxJ97fn3tUF7Svwo8WJSNSaIi4LBr4W3azKbWbETvCyLD9GMulM1Ts WPjcZuJa+1t8A/W5EEipQnEkVIGcrupQxMsmFwWvIgYr66pB5KLqj1neH4ktVdv241w6 +t98YH/nLmX4AcYzYWwdXG3oER0R0GH8rzVANnjUb0JxCb4gSWsAzUTksshF7swfSnsC t2Og== MIME-Version: 1.0 X-Received: by 10.194.60.195 with SMTP id j3mr13853005wjr.33.1364795616171; Sun, 31 Mar 2013 22:53:36 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sun, 31 Mar 2013 22:53:36 -0700 (PDT) In-Reply-To: <20130401055031.GB47589@eureka.lemis.com> References: <20130401055031.GB47589@eureka.lemis.com> Date: Mon, 1 Apr 2013 08:53:36 +0300 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Kimmo Paasiala To: "Greg 'groggy' Lehey" Content-Type: text/plain; charset=UTF-8 Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 05:53:37 -0000 On Mon, Apr 1, 2013 at 8:50 AM, Greg 'groggy' Lehey wrote: > On Monday, 1 April 2013 at 0:48:08 -0400, Eitan Adler wrote: >> Hi, >> >> I am writing this email to discuss the i386 architecture in FreeBSD. >> >> Computers are getting faster, but also more memory intensive. I >> can not find a laptop with less than 4 or 8 GB of RAM. Modern >> browsers, such as Firefox, require a 64bit architecture and 8GB of >> RAM. A 32 bit platform is not enough now a days on systems with >> more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in >> the 1990s. Even in the embedded world ARM is going 64 bit with >> ARMv8. >> >> Secondly, the i386 port is unmaintained. Very few developers run >> it, so it doesn't get the testing it deserves. Almost every user >> post or bug report I see from a x86 compatible processor is running >> amd64. When was the last time you booted i386 outside a virtual >> machine? Often times the build works for amd64 but fails for i386. >> >> Finally, others are dropping support for i386. Windows Server 2008 >> is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users >> and downstream vendors no longer care about preserving ancient >> hardware. >> >> I hope this email is enough to convince you that on this date we >> should drop support for the i386 architecture for 10.0 to tier 2 >> and replace it with the ARM architecture as Tier 1. > > Nice one! And only 48 minutes into the day. I've seen a number of > people take it seriously. > > Greg > -- > Oh crap :P However, this discussion will not be out of place some day, may be 2 or 3 years and practicly everything will be 64-bits. -Kimmo From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 06:26:20 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DAADEE35; Mon, 1 Apr 2013 06:26:20 +0000 (UTC) (envelope-from erichsfreebsdlist@alogt.com) Received: from alogt.com (alogt.com [69.36.191.58]) by mx1.freebsd.org (Postfix) with ESMTP id A906AE9D; Mon, 1 Apr 2013 06:26:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=einjhPmIJUhHUwkm1y4kBVpOsQKZX1TOJnmKcUTNg1g=; b=jqIQdb39QDi0bLQg53yTc91D6w9z0c07dGJLTKTRcDEt16H+zTYWdtJ9BLbYdQto7rFG6ua5mhUI1VIuQANbYyaaM0lHHe5Paqqv3C+YPXgglOQOXLhDUsWgszEuFy8Tsx3+obxnzFmM3/9HcIbYtjcDhaSxOs3yVNkqtUahjY8=; Received: from [122.129.203.50] (port=54667 helo=X220.ovitrap.com) by sl-508-2.slc.westdc.net with esmtpsa (SSLv3:DHE-RSA-AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1UMYCB-003gNA-9e; Mon, 01 Apr 2013 00:26:19 -0600 Date: Mon, 1 Apr 2013 13:26:14 +0700 From: Erich Dollansky To: Eitan Adler Subject: Re: considering i386 as a tier 1 architecture Message-ID: <20130401132614.1252f827@X220.ovitrap.com> In-Reply-To: References: X-Mailer: Claws Mail 3.9.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 06:26:20 -0000 Hi, On Mon, 1 Apr 2013 00:48:08 -0400 Eitan Adler wrote: > Hi, > > I am writing this email to discuss the i386 architecture in FreeBSD. if not for the date, I just wonder, what significance it real has. Erich From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 08:26:16 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 430627C7; Mon, 1 Apr 2013 08:26:16 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0817B86D; Mon, 1 Apr 2013 08:26:15 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:434:c78e:2de8:4e1f]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id ED6FB4AC57; Mon, 1 Apr 2013 12:26:07 +0400 (MSK) Date: Mon, 1 Apr 2013 12:26:00 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <917043347.20130401122600@serebryakov.spb.ru> To: Eitan Adler Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 08:26:16 -0000 Hello, Eitan. You wrote 1 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2013 =D0=B3., 8:48:08: EA> I hope this email is enough to convince you that on this date we EA> should drop support for the i386 architecture for 10.0 to tier 2 EA> and replace it with the ARM architecture as Tier 1. A lot of people (myself included) uses FreeBSD on small "integrated" boards from Soekris, ALIX and others, equipped with low-power non-interl i386-compatible processors. Now such boards start (only start!) to migrate to Intel Atom, but not every Intel Atom platform is 64bit capable (it depends on CPU, chipset, BIOS and some other unknown conditions, but the same CPU on different boards could be and could be not 64-bit capable in practice). And these die-hard integrated mult-NIC soldered-memory soldered-CPU motherboards will work for long time. I'm using -CURRENT on my board, because I need very last WiFi, for example, and as such boards are often used as small custom routers, it is not only my case. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 10:39:17 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BCAE7FC6; Mon, 1 Apr 2013 10:39:17 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 809451B1; Mon, 1 Apr 2013 10:39:17 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id D66CFC4B1; Mon, 1 Apr 2013 10:39:15 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 83A1A946E; Mon, 1 Apr 2013 12:39:15 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mehmet Erol Sanliturk Subject: Re: considering i386 as a tier 1 architecture References: Date: Mon, 01 Apr 2013 12:39:15 +0200 In-Reply-To: (Mehmet Erol Sanliturk's message of "Sun, 31 Mar 2013 22:41:02 -0700") Message-ID: <86wqsmpmb0.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 10:39:17 -0000 Mehmet Erol Sanliturk writes: > At present, there is NO any processor which is ONLY 32-bits. All the world is not a PC. There are still 32-bit x86-based embedded or small-form-factor systems, such as the soekris net5501 and net6501, which are widely used in the BSD community. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 11:08:41 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C2889E29; Mon, 1 Apr 2013 11:08:41 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vc0-f179.google.com (mail-vc0-f179.google.com [209.85.220.179]) by mx1.freebsd.org (Postfix) with ESMTP id 741066A3; Mon, 1 Apr 2013 11:08:41 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id gf12so2178519vcb.24 for ; Mon, 01 Apr 2013 04:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=J0Rg/rEkVXgp5Zgkb0YgagSbrQP2gnt3buSkhK6R6Kw=; b=BCVxWLNPlmjhhunl4wjjLvbBVYQVVLBOR3JyABwox9k22KYgYbBFPeji79p3OhKAev F6+WculKS9fnGg1LR2LdBdBogHFpNLvvQZYeyc0Igk/tR2KDc+4DKjlCbFtOhRX1qlNu 6FnBcAgtgm0QBtjDa69+CohBXeM4SkQJEXflVnyC2AbzVmnfDPQ+0I71qd0wphwPPGfS iz7+URfwh8iQ+Fl3Mn2M3upiTIUa2emgtEMbIFPKPB9GVtklcnM7uGFPyXCWwdSEVpHS DyoBEeGDeY/7KvuatlioK24nGSCvRvJAabcJYaCYdTEU6/uCmMizLX5jEguHYPSUFNZ9 DBKw== MIME-Version: 1.0 X-Received: by 10.52.68.116 with SMTP id v20mr7307593vdt.126.1364814515218; Mon, 01 Apr 2013 04:08:35 -0700 (PDT) Received: by 10.58.132.203 with HTTP; Mon, 1 Apr 2013 04:08:35 -0700 (PDT) In-Reply-To: <86wqsmpmb0.fsf@ds4.des.no> References: <86wqsmpmb0.fsf@ds4.des.no> Date: Mon, 1 Apr 2013 04:08:35 -0700 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Mehmet Erol Sanliturk To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 11:08:41 -0000 On Mon, Apr 1, 2013 at 3:39 AM, Dag-Erling Sm=C3=B8rgrav wrote= : > Mehmet Erol Sanliturk writes: > > At present, there is NO any processor which is ONLY 32-bits. > > All the world is not a PC. There are still 32-bit x86-based embedded or > small-form-factor systems, such as the soekris net5501 and net6501, > which are widely used in the BSD community. > > DES > -- > Dag-Erling Sm=C3=B8rgrav - des@des.no > These are special purpose systems and they are not developed by , let's say , "ordinary" users . Therefore , their developers may maintain the existing i386 branch as a specialized branch with a freedom to tailor it to more specific needs . Since I am not a developer or user of such a system , I can not say whether 25000 packages are necessary for them or not . Reducing any amount of work load which its outcome is not directly used is a contribution to the FreeBSD project by diverting such efforts to other man power or resources required areas . When costs are considered , some times 64 bit capable systems are not so expensive : As an example : From a computer shop in Turkey with many branch shops : Memory : 1 Giga Bytes chips : From $27 + VAT to $40 + VAT 4 Giga Bytes chips : From $31 + VAT to $43 + VAT There is NO reason to buy 4 x ( 1 Giga Bytes ) , when the difference is a few dollars with 4 x ( 4 Giga Bytes ) . AMD Sempron 145 AM3 2.8GHz processor is $ 34 + VAT which is 64 bits capable . As I said above , I am not able to make knowledgeable comparisons about such systems but my opinion is that these specialized systems are not so much advantageous . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 11:51:30 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 97480F6B for ; Mon, 1 Apr 2013 11:51:30 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 0190B900 for ; Mon, 1 Apr 2013 11:51:29 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r31BpSYf073257 for ; Mon, 1 Apr 2013 15:51:28 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r31BpSrL073256 for arch@FreeBSD.org; Mon, 1 Apr 2013 15:51:28 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 1 Apr 2013 15:51:28 +0400 From: Gleb Smirnoff To: arch@FreeBSD.org Subject: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130401115128.GZ76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 11:51:30 -0000 Hi! Together with Konstantin Belousov (kib@) we developed a new API that is initially purposed for (but not limited to) collecting statistical data in kernel. Right now in kernel a statistical counter not protected by a lock is updated either using basic '+=' operator, or via an atomic(9) operation. The first case is prone to data loss due to simultaneous update by more than one CPU. The second case doesn't lose data, but locks memory bus to establish atomic operation, effectively blocking other CPUs. Both cases invalidate cache line associated with the address being updated, thus provoking cache miss when other CPU will update a counter. The situation with cache misses shows itself in situations when multiple threads running on multiple CPUs access mostly their private dataset, and a counter is the only write-shared data between them. Example of such situation is packet forwarding, where struct ipstat and interface byte/packet counters are the data that experiences excessive cache misses. This was found by Alexander Chernikov (melifaro@) and Andrey Elsukov (ae@) at Yandex. They report that removal of stat updates raises benchmarking results in packet forwarding significantly. The decision is to make statistical counters private to CPUs. To achieve that, the following steps were taken: o uma(9) got support for UMA_ZONE_PCPU zones. A slab in UMA_ZONE_PCPU has mp_ncpu slabs following it in virtual memory, so that each CPU has its own slab. Allocation from such zone gives caller the base allocation address, and private address for a CPU can be calculated using base address + curcpu * slab size. The slab size in UMA_ZONE_PCPU is sizeof(struct pcpu). This sizing is required for optimisation described below. Access to private member is coded in tiny inline zpcpu_get(). Note that UMA_ZONE_PCPU zones aren't dedicated for counter(9), they can be utilized for other purposes. You may use them for any kind of dynamic per-CPU data you need. o Tiny API for counter(9): counter_u64_t counter_u64_alloc(int wait); void counter_u64_free(counter_u64_t cnt); void counter_u64_add(counter_u64_t cnt, uint64_t inc); uint64_t counter_u64_fetch(counter_u64_t cnt); o MI implementation of counter_u64_add() is: critical_enter(); *(uint64_t *)zpcpu_get(c) += inc; critical_exit(); o Since distance between members has special sizing, on 64-bit architectures that allow base+offset addressing, namely amd64, it is possible to perform update in single lockless intruction: __asm __volatile("addq\t%1,%%gs:(%0)" : : "r" ((char *)c - (char *)&__pcpu[0]), "r" (inc) : "memory", "cc"); Here we exploit the fact that %gs always points to the current CPU static per-CPU data. And alignment of static per-CPU structures matches alignment of slabs in an UMA_ZONE_PCPU zone. Author of this idea is Konstantin (kib@). I've got a simple benchmark. A syscall that solely updates a counter is implemented. Number of processes is spawned, equal to number of cores, each process binds itself to a dedicated CPU and calls the syscall 10^6 times and exits. Parent wait(2)s for them all and I measure real time of the parent. Here are results for my 4-core Xeon E5620: x new counter(9), resulting counter == mp_ncpus * 10^6 + racy '+=', resulting counter ~25% less than expected * atomic(9), resulting counter == mp_ncpus * 10^6 +------------------------------------------------------------------------------+ |x + *| |x + *| |x + *| |x + *| |x + *| |x + *| |x + *| |x + *| |x + *| |x + *| |A A A| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 17.65 17.68 17.66 17.665 0.0097182532 + 10 18.53 18.56 18.55 18.543 0.011595018 Difference at 95.0% confidence 0.878 +/- 0.0100517 4.97028% +/- 0.0569016% (Student's t, pooled s = 0.0106979) * 10 34.15 34.2 34.16 34.171 0.020789955 Difference at 95.0% confidence 16.506 +/- 0.0152473 93.439% +/- 0.0863138% (Student's t, pooled s = 0.0162275) So, we got 5% speedup comparing to racy counter and 93% speedup comparing to atomic(9). If binding of processes to CPUs is omitted, results are: x new counter(9), resulting counter == mp_ncpus * 10^6 + racy '+=', resulting counter ~20% less than expected * atomic(9), resulting counter == mp_ncpus * 10^6 +------------------------------------------------------------------------------+ | x + * | | x x x + + + + + + ** * | |xxxx x +x xxxxxx *+ x + +++++ + + *** * * *| | |______A__M___| |_______A_M____| |__A__| | +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 20 19.04 23.94 21.83 21.3555 1.372836 + 20 20.69 27.33 25.44 24.902 1.4900957 Difference at 95.0% confidence 3.5465 +/- 0.916971 16.607% +/- 4.29384% (Student's t, pooled s = 1.43267) * 10 31.97 33.77 32.85 32.819 0.63079227 Difference at 95.0% confidence 11.4635 +/- 0.940783 53.6794% +/- 4.40534% (Student's t, pooled s = 1.18608) So, we got 16.6% speedup comparing to racy counter and 53.7% speedup comparing to atomic(9). (Note that dispersion of results is much higher, when binding is omitted. Not perfect scheduling in ULE?) Results may vary depending on number of CPUs and cache architecture, not speaking about actual usage of counter(9). Andrey (ae@) had benchmarked RX side of networking stack, where the IP statistics and TCP statistics were converted to counter(9). He reports that although maximum pps didn't increase measurably, the CPU utilisation had dropped from 94% to 45%. Probably something else in the stack prevented to receive more pps, despite of more CPU cycles available. The code is in subversion branch: http://svn.freebsd.org/base/projects/counters/ If there are no objections during this week, I'd like to merge the code to head. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 13:11:58 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E6D8BC0B; Mon, 1 Apr 2013 13:11:58 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A942324A; Mon, 1 Apr 2013 13:11:58 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 7CBE6C6EB; Mon, 1 Apr 2013 13:11:56 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 431929488; Mon, 1 Apr 2013 15:11:56 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mehmet Erol Sanliturk Subject: Re: considering i386 as a tier 1 architecture References: <86wqsmpmb0.fsf@ds4.des.no> Date: Mon, 01 Apr 2013 15:11:55 +0200 In-Reply-To: (Mehmet Erol Sanliturk's message of "Mon, 1 Apr 2013 04:08:35 -0700") Message-ID: <86li92pf8k.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 13:11:59 -0000 Mehmet Erol Sanliturk writes: > Since I am not a developer or user of such a system , I can not say > whether 25000 packages are necessary for them or not. Reducing any > amount of work load which its outcome is not directly used is a > contribution to the FreeBSD project by diverting such efforts to other > man power or resources required areas. You're assuming that maintaining i386 as a tier 1 platform really *does* add significantly to our workload. You should also check your calendar :) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 13:31:52 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9149EB0D for ; Mon, 1 Apr 2013 13:31:52 +0000 (UTC) (envelope-from mexas@bristol.ac.uk) Received: from eu1sys200aog102.obsmtp.com (eu1sys200aog102.obsmtp.com [207.126.144.113]) by mx1.freebsd.org (Postfix) with ESMTP id E9B185EA for ; Mon, 1 Apr 2013 13:31:51 +0000 (UTC) Received: from mail-ea0-f199.google.com ([209.85.215.199]) (using TLSv1) by eu1sys200aob102.postini.com ([207.126.147.11]) with SMTP ID DSNKUVmMKS3ejPMiqPriftPOENCgo0aLn4Qg@postini.com; Mon, 01 Apr 2013 13:31:52 UTC Received: by mail-ea0-f199.google.com with SMTP id z7so3603213eaf.10 for ; Mon, 01 Apr 2013 06:31:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-received:date:from:message-id:to:subject:reply-to :in-reply-to:x-gm-message-state; bh=46xsJaPAwaiZQ13WUVeDdt1aPeJ25vyEfMzO6zqRmXw=; b=ljeTaolHtNyWTDbgfURaFOS6l/P84dkX1kAGNbqoRVI2ZgrW0f72tGkkhw6PU3uUDO GsBymuFi2O9lQBNdgCOD7egzU9J3eHNIWNXO5Gu8Afz+edO8LlmSjmqlsl/UdA4ncTYi 27YDiCmFPW4Js8QvP7WJIEwsropQBOK7g9sKIMztpwHi48oEExFoRinBcZfJiJLwW1qR ZweSeuzX4mWheUqwUH3HY44pABey3gu3YKM/TjZbq2E9Z94cG89X1iwbrmW9ZwbN12TL PY40nfrZ0DtF7Tvp9qw2C6GggMFl/um9fpNBHSsNizUZnJrLHvL1t8aLWIZL02OWt+8A /iCg== X-Received: by 10.180.185.197 with SMTP id fe5mr10094175wic.3.1364823081899; Mon, 01 Apr 2013 06:31:21 -0700 (PDT) X-Received: by 10.180.185.197 with SMTP id fe5mr10094166wic.3.1364823081833; Mon, 01 Apr 2013 06:31:21 -0700 (PDT) Received: from mech-cluster241.men.bris.ac.uk (mech-cluster241.men.bris.ac.uk. [137.222.187.241]) by mx.google.com with ESMTPS id g9sm15030078wix.1.2013.04.01.06.31.14 (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 01 Apr 2013 06:31:20 -0700 (PDT) Received: from mech-cluster241.men.bris.ac.uk (localhost [127.0.0.1]) by mech-cluster241.men.bris.ac.uk (8.14.6/8.14.6) with ESMTP id r31DVDPd052857 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Apr 2013 14:31:13 +0100 (BST) (envelope-from mexas@mech-cluster241.men.bris.ac.uk) Received: (from mexas@localhost) by mech-cluster241.men.bris.ac.uk (8.14.6/8.14.6/Submit) id r31DVCJw052856; Mon, 1 Apr 2013 14:31:12 +0100 (BST) (envelope-from mexas) Date: Mon, 1 Apr 2013 14:31:12 +0100 (BST) From: Anton Shterenlikht Message-Id: <201304011331.r31DVCJw052856@mech-cluster241.men.bris.ac.uk> To: freebsd-arch@freebsd.org, freebsd-hackers@freebsd.org, lists@eitanadler.com Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: X-Gm-Message-State: ALoCoQmDLWSgrX5nR5Z4afoByeW+g92TcA5BL7lkkeQNGQrhgaMqjueEcHEQvhy2g8jKPM7WW4A1ue1tqTvju9zy7OR2dSznHkKcCohxnrkrD+gLQaJXpOAB2aE3UGMO744EhRv1hBjtFQnanC4B+ksq76I+OA3CuhbvgHVkep9SEX0uJw3DX9Y= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: mexas@bristol.ac.uk List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 13:31:52 -0000 From: Eitan Adler Date: Mon, 1 Apr 2013 00:48:08 -0400 Subject: considering i386 as a tier 1 architecture To: freebsd-arch@freebsd.org, FreeBSD Hackers should drop support for the i386 architecture for 10.0 to tier 2 and replace it with the ARM architecture as Tier 1. don't care for i386, don't use it anymore. Is ARM really ready for tier 1? Including the ports infrastructure? Installation images? Anton From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 14:00:56 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C9E296E9 for ; Mon, 1 Apr 2013 14:00:56 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ia0-x236.google.com (mail-ia0-x236.google.com [IPv6:2607:f8b0:4001:c02::236]) by mx1.freebsd.org (Postfix) with ESMTP id 9A4567D2 for ; Mon, 1 Apr 2013 14:00:56 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id u8so1893324iag.41 for ; Mon, 01 Apr 2013 07:00:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=oYn5c/OLzc5K7yzMoym3Rqdpu/A9vFPOjUAYu9Zt+rI=; b=RX+yHFIKxVKh/9dYxLGmPz922KDRKbnZgl2Ghp+MR3PvhCyWAHODCUEV5E16IVUAt8 MgWWCnCofuify5SMHa3ePmNhNKEKLuo2zj5Lwd07ZEVMm1upcm0f5y4THOCFSLKwxAI2 t7pNdkC1Wo7qWW43HUnfUYCoL9YFBKBsLHaiRsn4VF1pry4xNuf/OUNcXTdUPA1L4AFI TrNa533CwV/EsI5Gp5otA7qrH6AO51oqX8UVGFDATAhSwC0zQxRzfTDvSfdq3WuEOF5F 7W+M3s1b3/348mZkF5ApDPGCE2GgNDQJ2Co7xGzQZbmH+oYKw+0vIXPRLCUw7U3ws2BT u8EQ== X-Received: by 10.50.178.105 with SMTP id cx9mr3370692igc.111.1364824855658; Mon, 01 Apr 2013 07:00:55 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id vb15sm4719797igb.9.2013.04.01.07.00.53 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 01 Apr 2013 07:00:54 -0700 (PDT) Sender: Warner Losh Subject: Re: considering i386 as a tier 1 architecture Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Mon, 1 Apr 2013 08:00:52 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Kimmo Paasiala X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQkyrsaLXnSx8aluX6LKqpUNu1OYVkoSuLImNEExFpgfJvulSvpE76Duod2c0/Jtb3WRsNv9 Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 14:00:56 -0000 On Mar 31, 2013, at 11:48 PM, Kimmo Paasiala wrote: > I think the only ones who are going to object are the users of = embedded > hardware. Some of them are still using CPUs that are only i586 = equivalent. >=20 > Personally I support the notion. >=20 > -Kimmo >=20 >=20 > On Mon, Apr 1, 2013 at 7:48 AM, Eitan Adler = wrote: >=20 >> Hi, >>=20 >> I am writing this email to discuss the i386 architecture in FreeBSD. >>=20 >> Computers are getting faster, but also more memory intensive. I >> can not find a laptop with less than 4 or 8 GB of RAM. Modern >> browsers, such as Firefox, require a 64bit architecture and 8GB of >> RAM. A 32 bit platform is not enough now a days on systems with >> more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in >> the 1990s. Even in the embedded world ARM is going 64 bit with >> ARMv8. Actually, that's not true. ARM is producing a 64-bit thing, but (a) it = hasn't been released yet and (b) the vast majority of all embedded arm = boards are 32-bits. >> Secondly, the i386 port is unmaintained. Very few developers run >> it, so it doesn't get the testing it deserves. Almost every user >> post or bug report I see from a x86 compatible processor is running >> amd64. When was the last time you booted i386 outside a virtual >> machine? Often times the build works for amd64 but fails for i386. I've not seen this to be the case, and I still run i386 in several = virtual machines as well as on my firewall. Running in a virtual = environment isn't good support for dropping i386, frankly. I've had the = build be broken for me about equal times for both. >> Finally, others are dropping support for i386. Windows Server 2008 >> is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users >> and downstream vendors no longer care about preserving ancient >> hardware. >>=20 >> I hope this email is enough to convince you that on this date we >> should drop support for the i386 architecture for 10.0 to tier 2 >> and replace it with the ARM architecture as Tier 1. arm can be Tier 1 without dropping i386 as Tier 1. Are there specific = bugs in i386 that haven't gone fixed for a long time? Basically, I see no benefit to this move. At least none has been = articulated. Warner= From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 14:17:23 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A84DBBF3 for ; Mon, 1 Apr 2013 14:17:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-da0-x229.google.com (mail-da0-x229.google.com [IPv6:2607:f8b0:400e:c00::229]) by mx1.freebsd.org (Postfix) with ESMTP id 83E8E8A8 for ; Mon, 1 Apr 2013 14:17:23 +0000 (UTC) Received: by mail-da0-f41.google.com with SMTP id w4so1090468dam.14 for ; Mon, 01 Apr 2013 07:17:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=cKWww2eRmx5bvNSBWro4sMgJpFjI0tzMRFVV3uBVCc0=; b=au/MVXq3m47cHKGQJH8UiRcu7EoV+4zU57GzLzBV7Rb+/i6mL0fg95qelTWHA6u+Pg L3uQtiKj9CVYxnTrna0ppzi42ITvPhYEaiC2ns64KHaBqIP5A9ht+JWFt/Xfz3ZJ4yay qlDfzximupHdwpQ4cvuO2Nm1WPFg0+2gxnBnfqNeXC6mmHJPaC3dn9pvL6DgV3ZFOF01 AfuwLwaWIn9CwHEKdX2VUyKC8KNxJ2e79jWps5nrxn3G+eUsJCVOD/HRL9ZEghmRjERY yUY0MHC2yvWnlyxjUd7+pUWaisNQjaUth+GP2AC0CLUFJoNQ3i0J/9WKTOYJeN9aSaui tifA== X-Received: by 10.66.13.35 with SMTP id e3mr19248852pac.186.1364825843233; Mon, 01 Apr 2013 07:17:23 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id lb8sm15496374pab.13.2013.04.01.07.17.20 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 01 Apr 2013 07:17:21 -0700 (PDT) Sender: Warner Losh Subject: Re: considering i386 as a tier 1 architecture Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <201304011331.r31DVCJw052856@mech-cluster241.men.bris.ac.uk> Date: Mon, 1 Apr 2013 08:17:18 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <432EFDBB-9CE8-480A-8B6A-8171BA9F4A83@bsdimp.com> References: <201304011331.r31DVCJw052856@mech-cluster241.men.bris.ac.uk> To: mexas@bristol.ac.uk X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQnUWdHQrtJ7nAT2H4WAceICy9mTZm0k4KH+nrdNBM2niQroC3HLxq02YlbSzwGvHtoKMF/w Cc: freebsd-hackers@freebsd.org, lists@eitanadler.com, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 14:17:23 -0000 On Apr 1, 2013, at 7:31 AM, Anton Shterenlikht wrote: > From: Eitan Adler > Date: Mon, 1 Apr 2013 00:48:08 -0400 > Subject: considering i386 as a tier 1 architecture > To: freebsd-arch@freebsd.org, FreeBSD Hackers = >=20 > should drop support for the i386 architecture for 10.0 to tier 2 > and replace it with the ARM architecture as Tier 1. >=20 > don't care for i386, don't use it anymore. >=20 > Is ARM really ready for tier 1? > Including the ports infrastructure? > Installation images? It is certainly getting close, at least for the newer armv6+ boards. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 16:02:24 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C35AB2DF; Mon, 1 Apr 2013 16:02:24 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 46B13F4C; Mon, 1 Apr 2013 16:02:23 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r31G25mf097481; Mon, 1 Apr 2013 18:02:05 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r31G25Ji097478; Mon, 1 Apr 2013 18:02:05 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 1 Apr 2013 18:02:05 +0200 (CEST) From: Wojciech Puchar To: Eitan Adler Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 01 Apr 2013 18:02:06 +0200 (CEST) Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 16:02:24 -0000 > Computers are getting faster, but also more memory intensive. I > can not find a laptop with less than 4 or 8 GB of RAM. Modern > browsers, such as Firefox, require a 64bit architecture and 8GB of > RAM. what? i rarely see firefox exceed 1GB and it is already way too much IMHO. ? A 32 bit platform is not enough now a days on systems with > more than 4 GB of RAM. and never was. > A 32 bit core now is like 640K of RAM in > the 1990s. Even in the embedded world ARM is going 64 bit with > ARMv8. but ALL NEW x86 computers have 64-bit instruction set. > Secondly, the i386 port is unmaintained. Very few developers run > it, so it doesn't get the testing it deserves. Almost every user > post or bug report I see from a x86 compatible processor is running > amd64. When was the last time you booted i386 outside a virtual > machine? now 1 server running. because it's older and not 64-bit capable. > and replace it with the ARM architecture as Tier 1. true. no need to support it as tier 1. users of older hardware usually don't want to upgrade to latest freebsd kernel and userland. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 16:03:38 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DED284D8; Mon, 1 Apr 2013 16:03:38 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 6030DF70; Mon, 1 Apr 2013 16:03:38 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r31G3RtE097492; Mon, 1 Apr 2013 18:03:27 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r31G3RXs097489; Mon, 1 Apr 2013 18:03:27 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 1 Apr 2013 18:03:27 +0200 (CEST) From: Wojciech Puchar To: Mehmet Erol Sanliturk Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 01 Apr 2013 18:03:27 +0200 (CEST) Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 16:03:38 -0000 > that it is NOT necessary to make it a first class branch . 1 Giga Bytes , > and even 2 Giga Bytes memory chips are disappearing from the computer shops > slowly . at now 2GB RAM is smallest you can get, and intel atom is lowest end - but still 64-bit - CPU. > At present , there is NO any processor which is ONLY 32-bits . Not only the > Windows Server , if I am not remembering incorrectly , new regular Windows it should not matter what microsoft do. It is Unix. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 16:39:59 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 78CA8BA1 for ; Mon, 1 Apr 2013 16:39:59 +0000 (UTC) (envelope-from mjacob@freebsd.org) Received: from virtual.feral.com (virtual.feral.com [216.224.170.83]) by mx1.freebsd.org (Postfix) with ESMTP id 2DC6D1E6 for ; Mon, 1 Apr 2013 16:39:58 +0000 (UTC) Received: from [192.168.135.7] (76-14-49-207.sf-cable.astound.net [76.14.49.207]) by virtual.feral.com (8.14.4/8.14.4) with ESMTP id r31Gcosi020895 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 1 Apr 2013 09:38:51 -0700 Message-ID: <5159B81A.4000101@freebsd.org> Date: Mon, 01 Apr 2013 09:38:50 -0700 From: Matthew Jacob Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: considering i386 as a tier 1 architecture References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (virtual.feral.com [216.224.170.83]); Mon, 01 Apr 2013 09:38:52 -0700 (PDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: mjacob@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 16:39:59 -0000 On 4/1/2013 9:02 AM, Wojciech Puchar wrote: > > > users of older hardware usually don't want to upgrade to latest > freebsd kernel and userland. I have several i386 only (Pentium 4 is the newest) that I run FreeBSD-current on, including (up until last week) feral.com itself. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 17:01:28 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 37CEC7CA; Mon, 1 Apr 2013 17:01:28 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id F09AC2FD; Mon, 1 Apr 2013 17:01:27 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:434:c78e:2de8:4e1f]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 631374AC58; Mon, 1 Apr 2013 21:01:25 +0400 (MSK) Date: Mon, 1 Apr 2013 21:01:18 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <944760435.20130401210118@serebryakov.spb.ru> To: Wojciech Puchar Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 17:01:28 -0000 Hello, Wojciech. You wrote 1 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2013 =D0=B3., 20:03:27: >> that it is NOT necessary to make it a first class branch . 1 Giga Bytes , >> and even 2 Giga Bytes memory chips are disappearing from the computer sh= ops >> slowly . WP> at now 2GB RAM is smallest you can get, and intel atom is lowest end - = but WP> still 64-bit - CPU. It is not exact so. Some Atoms on some motherboards with some firmwares are 64-bit CPU. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 18:31:33 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 095F6D34; Mon, 1 Apr 2013 18:31:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) by mx1.freebsd.org (Postfix) with ESMTP id 72EE184D; Mon, 1 Apr 2013 18:31:32 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id k14so1989963wer.27 for ; Mon, 01 Apr 2013 11:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=jC1i8KF/ZfFhlPipJvbbPMYDSmdx3l1ZVl78mydk0r8=; b=YQiCb1Q5KFd4L6qtXFQ0ExYTkBHYQrFPPSrc4iiUalhmR/rbnjj2leX0aLyFH3F55m 0itNyTVJmr24hK/ND7RUypHBHF7D8OvJ8pn28dWB4pSQksoZ9PDF+vTv+lfclmwwdqRu dXMJ5/OqX7A8GbZzAk8Wb9POa46O53dBC3pirAyH/Qz4nKpPJxU0JETl7yEqP9TVdM/I e+TPJPCfrOpLTZ60+AY1cXluXgM4zlIOzR8lJXq1/QGLYtEli31P89clATnYw0U89a41 LdxtAZTS+bMc75eq9NNGcpZhrDtg3CYnNDc9i5Xdi0ZPzjoRpYrnBI9MJkXYm74ZjygY Vn6A== MIME-Version: 1.0 X-Received: by 10.194.171.74 with SMTP id as10mr17094376wjc.0.1364841091599; Mon, 01 Apr 2013 11:31:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.108.130 with HTTP; Mon, 1 Apr 2013 11:31:31 -0700 (PDT) In-Reply-To: References: Date: Mon, 1 Apr 2013 11:31:31 -0700 X-Google-Sender-Auth: tGjYVM4yVN8hdWQo0bGKUDPKgDQ Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Adrian Chadd To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 18:31:33 -0000 Why stop there? Noone runs FreeBSD on real hardware anymore. Except, say netflix. Let's just drop actual native hardware support and instead support only the bare minimum needed to boot inside vmware, virtualbox and xen. Anyone needing real hardware support can install NetBSD and xen. Adrian On 31 March 2013 21:48, Eitan Adler wrote: > Hi, > > I am writing this email to discuss the i386 architecture in FreeBSD. > > Computers are getting faster, but also more memory intensive. I > can not find a laptop with less than 4 or 8 GB of RAM. Modern > browsers, such as Firefox, require a 64bit architecture and 8GB of > RAM. A 32 bit platform is not enough now a days on systems with > more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in > the 1990s. Even in the embedded world ARM is going 64 bit with > ARMv8. > > Secondly, the i386 port is unmaintained. Very few developers run > it, so it doesn't get the testing it deserves. Almost every user > post or bug report I see from a x86 compatible processor is running > amd64. When was the last time you booted i386 outside a virtual > machine? Often times the build works for amd64 but fails for i386. > > Finally, others are dropping support for i386. Windows Server 2008 > is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users > and downstream vendors no longer care about preserving ancient > hardware. > > I hope this email is enough to convince you that on this date we > should drop support for the i386 architecture for 10.0 to tier 2 > and replace it with the ARM architecture as Tier 1. > > -- > Eitan Adler > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 18:32:04 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B158AEDF; Mon, 1 Apr 2013 18:32:04 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 33DF6883; Mon, 1 Apr 2013 18:32:03 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r31IVptQ098329; Mon, 1 Apr 2013 20:31:51 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r31IVpjM098326; Mon, 1 Apr 2013 20:31:51 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 1 Apr 2013 20:31:51 +0200 (CEST) From: Wojciech Puchar To: Lev Serebryakov Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: <944760435.20130401210118@serebryakov.spb.ru> Message-ID: References: <944760435.20130401210118@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 01 Apr 2013 20:31:51 +0200 (CEST) Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 18:32:04 -0000 > WP> still 64-bit - CPU. > It is not exact so. Some Atoms on some motherboards with some > firmwares are 64-bit CPU. don't know of any now in shops that are not > > -- > // Black Lion AKA Lev Serebryakov > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 20:21:41 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 356D0431; Mon, 1 Apr 2013 20:21:41 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id E5E55E68; Mon, 1 Apr 2013 20:21:40 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:434:c78e:2de8:4e1f]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 4F3574AC57; Tue, 2 Apr 2013 00:21:38 +0400 (MSK) Date: Tue, 2 Apr 2013 00:21:30 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1437965352.20130402002130@serebryakov.spb.ru> To: Wojciech Puchar Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: References: <944760435.20130401210118@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 20:21:41 -0000 Hello, Wojciech. You wrote 1 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2013 =D0=B3., 22:31:51: >> It is not exact so. Some Atoms on some motherboards with some >> firmwares are 64-bit CPU. WP> don't know of any now in shops that are not Are you sure about Chinese-made MoBos with 6x1G on-board NICs and soldered memory and other such "embedded" devices? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Mon Apr 1 22:52:29 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3B1F0940; Mon, 1 Apr 2013 22:52:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E61AD721; Mon, 1 Apr 2013 22:52:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAOoOWlGDaFvO/2dsb2JhbABDgzuDILwugRJ0gh8BAQEEAQEBICsgCxsSBgICDRkCKQEJGA4OBwQBHASHcwyuYJIrgSOMXH40BxaCF4ETA5Qtgj6BH49sgycgMoEFNQ X-IronPort-AV: E=Sophos;i="4.87,387,1363147200"; d="scan'208";a="21876002" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 01 Apr 2013 18:52:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 6A1C3B3EA2; Mon, 1 Apr 2013 18:52:22 -0400 (EDT) Date: Mon, 1 Apr 2013 18:52:22 -0400 (EDT) From: Rick Macklem To: mjacob@freebsd.org Message-ID: <1034806374.377542.1364856742419.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <5159B81A.4000101@freebsd.org> Subject: Re: considering i386 as a tier 1 architecture MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 22:52:29 -0000 mjacob@freebsd.org wrote: > On 4/1/2013 9:02 AM, Wojciech Puchar wrote: > > > > > > users of older hardware usually don't want to upgrade to latest > > freebsd kernel and userland. > > I have several i386 only (Pentium 4 is the newest) that I run > FreeBSD-current on, including (up until last week) feral.com itself. > It's the only thing I have, so it's what the NFS code gets tested on. I haven't yet decided if this was meant to be an April fools post, but I decided to play along... rick ps: And, yea, I don't upgrade userland often and I don't do "make universe" often, because it takes many days to run;-) > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to > "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 01:36:47 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D8CBFFD8; Tue, 2 Apr 2013 01:36:47 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 9EA87C05; Tue, 2 Apr 2013 01:36:45 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id A53A273027; Tue, 2 Apr 2013 03:37:58 +0200 (CEST) Date: Tue, 2 Apr 2013 03:37:58 +0200 From: Luigi Rizzo To: Gleb Smirnoff Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130402013758.GA96904@onelab2.iet.unipi.it> References: <20130401115128.GZ76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130401115128.GZ76816@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 01:36:47 -0000 On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > Hi! > > Together with Konstantin Belousov (kib@) we developed a new API that is > initially purposed for (but not limited to) collecting statistical > data in kernel. ... really great work, thanks for tacking this. I have some comments inline: API: > o MI implementation of counter_u64_add() is: > > critical_enter(); > *(uint64_t *)zpcpu_get(c) += inc; > critical_exit(); - there are several places which use multiple counters (e.g. packet and byte counters, global and per flow/socket), so i wonder if it may help to provide a "protected" version of counter_u64_add() that requires the critical_enter/exit only once. Something like PROTECT_COUNTERS( safe_counter_u64_add(c, x); safe_counter_u64_add(c, x); safe_counter_u64_add(c, x); ); where PROTECT_COUNTERS() would translate into the critical_enter/exit where required, and nothing on other architectures. ... BENCHMARK: > I've got a simple benchmark. A syscall that solely updates a counter is > implemented. Number of processes is spawned, equal to number of cores, > each process binds itself to a dedicated CPU and calls the syscall 10^6 > times and exits. Parent wait(2)s for them all and I measure real time of - I am under the impression that these benchmarks are dominated by the syscall time, and the new counters would exhibit a lot better relative performance (compared to racy or atomic) by doing 10..100 counter ops per syscall. Any chance to retry the test in this configuration ? cheers luigi From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 02:12:12 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 706C3472; Tue, 2 Apr 2013 02:12:12 +0000 (UTC) (envelope-from lists@rewt.org.uk) Received: from abby.lhr1.as41113.net (hosted.mx.as41113.net [91.208.177.22]) by mx1.freebsd.org (Postfix) with ESMTP id 3B790DAD; Tue, 2 Apr 2013 02:12:11 +0000 (UTC) Received: from [172.16.9.23] (bella.stf.rewt.org.uk [91.208.177.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: lists@rewt.org.uk) by abby.lhr1.as41113.net (Postfix) with ESMTPSA id 3Zfv7R4PGLzss; Tue, 2 Apr 2013 03:12:03 +0100 (BST) Message-ID: <515A3E6C.7030404@rewt.org.uk> Date: Tue, 02 Apr 2013 03:11:56 +0100 From: Joe Holden User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Adrian Chadd Subject: Re: considering i386 as a tier 1 architecture References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Eitan Adler , freebsd-arch@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 02:12:12 -0000 Adrian Chadd wrote: > Why stop there? > > Noone runs FreeBSD on real hardware anymore. Except, say netflix. > > Let's just drop actual native hardware support and instead support > only the bare minimum needed to boot inside vmware, virtualbox and > xen. > > Anyone needing real hardware support can install NetBSD and xen. > The irony being that NetBSD runs on really obscure hardware but nothing that anybody anywhere uses? ;) > > Adrian > > On 31 March 2013 21:48, Eitan Adler wrote: >> Hi, >> >> I am writing this email to discuss the i386 architecture in FreeBSD. >> >> Computers are getting faster, but also more memory intensive. I >> can not find a laptop with less than 4 or 8 GB of RAM. Modern >> browsers, such as Firefox, require a 64bit architecture and 8GB of >> RAM. A 32 bit platform is not enough now a days on systems with >> more than 4 GB of RAM. A 32 bit core now is like 640K of RAM in >> the 1990s. Even in the embedded world ARM is going 64 bit with >> ARMv8. >> >> Secondly, the i386 port is unmaintained. Very few developers run >> it, so it doesn't get the testing it deserves. Almost every user >> post or bug report I see from a x86 compatible processor is running >> amd64. When was the last time you booted i386 outside a virtual >> machine? Often times the build works for amd64 but fails for i386. >> >> Finally, others are dropping support for i386. Windows Server 2008 >> is 64 bit only, OSX Mountain Lion (10.8) is 64-bit only. Users >> and downstream vendors no longer care about preserving ancient >> hardware. >> >> I hope this email is enough to convince you that on this date we >> should drop support for the i386 architecture for 10.0 to tier 2 >> and replace it with the ARM architecture as Tier 1. >> >> -- >> Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 08:29:21 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A8CD6B98; Tue, 2 Apr 2013 08:29:21 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [46.33.159.39]) by mx1.freebsd.org (Postfix) with ESMTP id 05DA5F27; Tue, 2 Apr 2013 08:29:20 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.87,392,1363132800"; d="scan'208";a="3106983" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 02 Apr 2013 08:29:19 +0000 Received: from [192.168.1.30] (10.30.249.104) by LONPMAILMX01.citrite.net (10.30.203.162) with Microsoft SMTP Server id 8.3.298.1; Tue, 2 Apr 2013 09:29:18 +0100 Message-ID: <515A96DD.5030606@citrix.com> Date: Tue, 2 Apr 2013 10:29:17 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: considering i386 as a tier 1 architecture References: In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Cc: Eitan Adler , "freebsd-arch@freebsd.org" , FreeBSD Hackers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 08:29:21 -0000 On 01/04/13 20:31, Adrian Chadd wrote: > Why stop there? > > Noone runs FreeBSD on real hardware anymore. Except, say netflix. > > Let's just drop actual native hardware support and instead support > only the bare minimum needed to boot inside vmware, virtualbox and > xen. > > Anyone needing real hardware support can install NetBSD and xen. No need for NetBSD anymore, Xen is going to integrate the Linux tree and glibc, so you can build a full distro form the Xen tree: http://blog.xen.org/index.php/2013/04/01/bringing-open-source-communities-closer-together/ From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 09:04:12 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9E5E4ACB; Tue, 2 Apr 2013 09:04:12 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 5EC8E14E; Tue, 2 Apr 2013 09:04:12 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id AEC44C441; Tue, 2 Apr 2013 09:04:04 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 714EB954E; Tue, 2 Apr 2013 11:04:04 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Wojciech Puchar Subject: Re: considering i386 as a tier 1 architecture References: <944760435.20130401210118@serebryakov.spb.ru> Date: Tue, 02 Apr 2013 11:04:04 +0200 In-Reply-To: (Wojciech Puchar's message of "Mon, 1 Apr 2013 20:31:51 +0200 (CEST)") Message-ID: <8638v9e22j.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , Lev Serebryakov , FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 09:04:12 -0000 Wojciech Puchar writes: > Lev Serebryakov writes: > > It is not exact so. Some Atoms on some motherboards with some > > firmwares are 64-bit CPU. > don't know of any now in shops that are not http://soekris.com/products/net5501.html http://soekris.com/products/net6501.html DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 09:26:13 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D43AC2E6; Tue, 2 Apr 2013 09:26:13 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 96956329; Tue, 2 Apr 2013 09:26:13 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:434:c78e:2de8:4e1f]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 28EEA4AC57; Tue, 2 Apr 2013 13:26:04 +0400 (MSK) Date: Tue, 2 Apr 2013 13:26:03 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1241302374.20130402132603@serebryakov.spb.ru> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= Subject: Re: considering i386 as a tier 1 architecture In-Reply-To: <8638v9e22j.fsf@ds4.des.no> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Wojciech Puchar , freebsd-arch@freebsd.org, FreeBSD Hackers , Eitan Adler X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 09:26:13 -0000 Hello, Dag-Erling. You wrote 2 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2013 =D0=B3., 13:04:04: DES> http://soekris.com/products/net6501.html This one is 64-bit capable according to their mailing list --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 10:11:02 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D4C11B67; Tue, 2 Apr 2013 10:11:02 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by mx1.freebsd.org (Postfix) with ESMTP id 71BA6781; Tue, 2 Apr 2013 10:11:02 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id kw10so228196vcb.28 for ; Tue, 02 Apr 2013 03:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=46V4rXKKd+rp6JEB5kpy7Bnz7V2QkXVxJAym3aWypu8=; b=naAARCTQ4pewK1YES+1ocxbqDYzvnDkrVe4yVWwmKIMo1zIQsDly5OAwToZJMsd78V mo0BUZ+ktul7MOJl0RuwEJS25yATdA5dnqDq2giXHAO72KELyy3d9w+yDnsQHZg9kaJi Mo3D+p4K10iczZNTEpG5CsBBpuekI3qyOw0KM5wr71lFCpi6An52QBuck0qdQ9mFhO5S 02GZL7kEpbsxX/UH3bQKf0IvkSTdNNrbpoipwRKFhBC8/hItRSl2eSf7/1uU0XI+pfSG RD9v/+jMb020iQgiDGjeV5YqHIBsJ0grGBWiU6oGv/698SFN8G8z4G3MY4Gb5SQzBb1N TvWA== MIME-Version: 1.0 X-Received: by 10.52.68.116 with SMTP id v20mr9967475vdt.126.1364897456694; Tue, 02 Apr 2013 03:10:56 -0700 (PDT) Received: by 10.58.132.203 with HTTP; Tue, 2 Apr 2013 03:10:56 -0700 (PDT) In-Reply-To: <8638v9e22j.fsf@ds4.des.no> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> Date: Tue, 2 Apr 2013 03:10:56 -0700 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Mehmet Erol Sanliturk To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Wojciech Puchar , freebsd-arch@freebsd.org, Lev Serebryakov , FreeBSD Hackers , Eitan Adler X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 10:11:03 -0000 On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm=C3=B8rgrav wrote= : > Wojciech Puchar writes: > > Lev Serebryakov writes: > > > It is not exact so. Some Atoms on some motherboards with some > > > firmwares are 64-bit CPU. > > don't know of any now in shops that are not > > http://soekris.com/products/net5501.html > http://soekris.com/products/net6501.html > > DES > -- > Dag-Erling Sm=C3=B8rgrav - des@des.no > I am NOT able to understand the merit of these products with respect to their features and PRICES . It is possible to assemble much more cheaper full featured PC like systems ( DDR3 memory , 64-bit capable processors , with a disadvantage about power requirements ) . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 10:13:45 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EF46DD31; Tue, 2 Apr 2013 10:13:45 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) by mx1.freebsd.org (Postfix) with ESMTP id 384A47B1; Tue, 2 Apr 2013 10:13:45 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id y10so2944527wgg.0 for ; Tue, 02 Apr 2013 03:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=PuK42h39W2ZoDLGvlDN5G5Nx6mwRhBVughQxtzvF0V8=; b=c2ebSkVmeSVr2X5T5PaXvPoV5ZtYUQ5xwK/gcg7lFFVZ8Lu3YQVxd+496xbnbruicz Te9niw+q42UjSMslzTKQrRidj1iEDAiPfj7tCBW1gAhtvEjIRIrr73tzAIdNORTXbhqF C0oyiwusvKcBBkZeQszR4kJt1udBlEM8EqywB6ADa4WmMp1NXS/6q1pFOt/u0/wLpBB8 ZVQaH/MrAb5NLjDqUdKDNh561DOPpzOe+iC1bI6nGAjaFdkQatzDNib0Eh9cYe+XJytX CG74dLRhdEUrDuYgpHQZ+r1OAeRTjgeY6rKUOhHHAzomTNDHCirQpxk6jLMuM5xRXdsf 8ijQ== MIME-Version: 1.0 X-Received: by 10.180.90.116 with SMTP id bv20mr3381183wib.32.1364897624348; Tue, 02 Apr 2013 03:13:44 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Tue, 2 Apr 2013 03:13:44 -0700 (PDT) In-Reply-To: References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> Date: Tue, 2 Apr 2013 13:13:44 +0300 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Kimmo Paasiala To: Mehmet Erol Sanliturk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Lev Serebryakov , FreeBSD Hackers , Eitan Adler , freebsd-arch@freebsd.org, =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , Wojciech Puchar X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 10:13:46 -0000 On Tue, Apr 2, 2013 at 1:10 PM, Mehmet Erol Sanliturk wrote: > On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm=C3=B8rgrav wro= te: > >> Wojciech Puchar writes: >> > Lev Serebryakov writes: >> > > It is not exact so. Some Atoms on some motherboards with some >> > > firmwares are 64-bit CPU. >> > don't know of any now in shops that are not >> >> http://soekris.com/products/net5501.html >> http://soekris.com/products/net6501.html >> >> DES >> -- >> Dag-Erling Sm=C3=B8rgrav - des@des.no >> > > > I am NOT able to understand the merit of these products with respect to > their features and PRICES . > > It is possible to assemble much more cheaper full featured PC like system= s > ( DDR3 memory , 64-bit capable processors , with a disadvantage about pow= er > requirements ) . > > Power consumption. From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 10:22:35 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 24D0CD0 for ; Tue, 2 Apr 2013 10:22:35 +0000 (UTC) (envelope-from mail25@bzerk.org) Received: from eris.bzerk.org (unknown [IPv6:2001:980:18dd:1:192:168:179:45]) by mx1.freebsd.org (Postfix) with ESMTP id 8DCE4803 for ; Tue, 2 Apr 2013 10:22:34 +0000 (UTC) Received: from eris.bzerk.org (BOFH@localhost [127.0.0.1]) by eris.bzerk.org (8.14.6/8.14.5) with ESMTP id r32AMKOj029581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 2 Apr 2013 10:22:21 GMT (envelope-from mail25@bzerk.org) Received: (from bulk@localhost) by eris.bzerk.org (8.14.6/8.14.6/Submit) id r32AMKIe029580; Tue, 2 Apr 2013 10:22:20 GMT (envelope-from mail25@bzerk.org) X-Authentication-Warning: eris.bzerk.org: bulk set sender to mail25@bzerk.org using -f Date: Tue, 2 Apr 2013 10:22:20 +0000 From: Ruben de Groot To: Mehmet Erol Sanliturk Subject: Re: considering i386 as a tier 1 architecture Message-ID: <20130402102220.GA28545@eris.bzerk.org> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-11.0 required=5.0 tests=ALL_TRUSTED,AUTHD_RELAY autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eris.bzerk.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (eris.bzerk.org [127.0.0.1]); Tue, 02 Apr 2013 10:22:26 +0000 (UTC) Cc: Lev Serebryakov , FreeBSD Hackers , Eitan Adler , freebsd-arch@freebsd.org, Dag-Erling Sm??rgrav , Wojciech Puchar X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 10:22:35 -0000 On Tue, Apr 02, 2013 at 03:10:56AM -0700, Mehmet Erol Sanliturk typed: > On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm??rgrav wrote: > > > Wojciech Puchar writes: > > > Lev Serebryakov writes: > > > > It is not exact so. Some Atoms on some motherboards with some > > > > firmwares are 64-bit CPU. > > > don't know of any now in shops that are not > > > > http://soekris.com/products/net5501.html > > http://soekris.com/products/net6501.html > > > > DES > > -- > > Dag-Erling Sm??rgrav - des@des.no > > > > > I am NOT able to understand the merit of these products with respect to > their features and PRICES . They are extremely stable and robust. > It is possible to assemble much more cheaper full featured PC like systems > ( DDR3 memory , 64-bit capable processors , with a disadvantage about power > requirements ) . You can also get much bigger portions at MacDonald than what you get in a five star restaurant. Ruben From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 11:41:49 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 997D0A5F; Tue, 2 Apr 2013 11:41:49 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 59192CE3; Tue, 2 Apr 2013 11:41:49 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 5A935C638; Tue, 2 Apr 2013 11:41:48 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 13D4D9562; Tue, 2 Apr 2013 13:41:47 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mehmet Erol Sanliturk Subject: Re: considering i386 as a tier 1 architecture References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> Date: Tue, 02 Apr 2013 13:41:47 +0200 In-Reply-To: (Mehmet Erol Sanliturk's message of "Tue, 2 Apr 2013 03:10:56 -0700") Message-ID: <86y5d1cg78.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Wojciech Puchar , freebsd-arch@freebsd.org, Lev Serebryakov , FreeBSD Hackers , Eitan Adler X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 11:41:49 -0000 Mehmet Erol Sanliturk writes: > I am NOT able to understand the merit of these products with respect > to their features and PRICES. Please stop SHOUTING, and learn to accept and respect the fact that other people have other opinions and priorities than you do, and to stop trying to force your worldview on them. Maybe they know something you haven't learned yet. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 11:51:16 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 31D054AC; Tue, 2 Apr 2013 11:51:16 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-ve0-f172.google.com (mail-ve0-f172.google.com [209.85.128.172]) by mx1.freebsd.org (Postfix) with ESMTP id C09FAD77; Tue, 2 Apr 2013 11:51:15 +0000 (UTC) Received: by mail-ve0-f172.google.com with SMTP id oz10so333361veb.17 for ; Tue, 02 Apr 2013 04:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=7rMoQ6qtVBl2gtomkRwhW9nLhajCSgYxL+qSTq73Hd4=; b=DoeuKKE2m3bOqTO8UcssF5wRjd2SaksA/fsVgwXCDvCoEZTDldGos2sXfzS6dqUsIj 1++SfE8uF7FuQQHMotsxJomJY3L1xLAmQpnk55CarFOoLKLXeOeLJSmRYqFmsCITW9ON tGit4B1ybrh/VoWyQV4CPFfVMT60SUZpWALZBAImgnQKostpRv+hjPcsBzd8aMEaEto6 WCGDA10WLnfM0umgocu+I8jjPqmUktCWGofAabosthg69zHL+4ua4xIOGbwW7533x4Vd nU5g6wjEjxhpNa07y5041mq5+f7ZtGlf1U625zZud9hAvuM9AP0f2CLoCw6iGx/24u5I BL5Q== MIME-Version: 1.0 X-Received: by 10.52.27.17 with SMTP id p17mr10593237vdg.0.1364903475057; Tue, 02 Apr 2013 04:51:15 -0700 (PDT) Received: by 10.58.132.203 with HTTP; Tue, 2 Apr 2013 04:51:14 -0700 (PDT) In-Reply-To: <86y5d1cg78.fsf@ds4.des.no> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> <86y5d1cg78.fsf@ds4.des.no> Date: Tue, 2 Apr 2013 04:51:14 -0700 Message-ID: Subject: Re: considering i386 as a tier 1 architecture From: Mehmet Erol Sanliturk To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Wojciech Puchar , freebsd-arch@freebsd.org, Lev Serebryakov , FreeBSD Hackers , Eitan Adler X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 11:51:16 -0000 On Tue, Apr 2, 2013 at 4:41 AM, Dag-Erling Sm=C3=B8rgrav wrote= : > Mehmet Erol Sanliturk writes: > > I am NOT able to understand the merit of these products with respect > > to their features and PRICES. > > Please stop SHOUTING, and learn to accept and respect the fact that > other people have other opinions and priorities than you do, and to stop > trying to force your worldview on them. Maybe they know something you > haven't learned yet. > > DES > -- > Dag-Erling Sm=C3=B8rgrav - des@des.no > You are right , but my idea was in affirmative sense to understand the reasons . I know that persons are using such systems with respect to some advantages other than the cost and their producers have reasons to assign such prices in a free economic structure . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 13:54:10 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CA7AA465 for ; Tue, 2 Apr 2013 13:54:10 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pa0-f45.google.com (mail-pa0-f45.google.com [209.85.220.45]) by mx1.freebsd.org (Postfix) with ESMTP id A0C423F3 for ; Tue, 2 Apr 2013 13:54:10 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id kl13so311146pab.4 for ; Tue, 02 Apr 2013 06:54:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=nLwA19wamwMbCphoxWjLl76hrd6D1DfM6a53dOqqhSw=; b=Skne5P0BmuqteR8KaEq/DMwPRKGEPw9tktSmjJyQOnpMxW4muk4RFuv8YEe4QUdyGZ kKB0vsBtdJxpQxXYYxgekdN4yITOx7V+SYW1ZV6kr6oz1ejKO2R86iFBHh53pbwwim6Y A2wYniu2dlRlbBB00kApiLSRHJAHWMztw1sWv+TRHWzideJ2MKZd8/DFCe5BbdRHTNhn 50ZHk4AZlgdvXl1rh4lC1ASn52rsMaedwuGtFzouMHJknAmID5w7kL8bu1zbJ5VH5I64 lVaoiZJ32CGnCrlrgr5b+x4WJbjj35AVCkpkmLlJdV/LvVCgUNxfm96sbAyy08G7mxtv IFoQ== X-Received: by 10.66.150.165 with SMTP id uj5mr25124473pab.37.1364910849935; Tue, 02 Apr 2013 06:54:09 -0700 (PDT) Received: from [10.0.0.53] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id kl4sm1826476pbc.31.2013.04.02.06.54.07 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 02 Apr 2013 06:54:08 -0700 (PDT) Sender: Warner Losh Subject: Re: considering i386 as a tier 1 architecture Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=iso-8859-1 From: Warner Losh In-Reply-To: Date: Tue, 2 Apr 2013 07:54:03 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <4FCBC174-DF7D-4935-A09B-F9BA34185462@bsdimp.com> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> To: Mehmet Erol Sanliturk X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQlYt6Dqd984ufH60dnJCazXYJ14e4jCkWxAPKymoJEApewael+arMcD5wsiZXyHJRP4n161 Cc: Lev Serebryakov , FreeBSD Hackers , Eitan Adler , freebsd-arch@freebsd.org, =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Wojciech Puchar X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 13:54:10 -0000 On Apr 2, 2013, at 4:10 AM, Mehmet Erol Sanliturk wrote: > On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm=F8rgrav = wrote: >=20 >> Wojciech Puchar writes: >>> Lev Serebryakov writes: >>>> It is not exact so. Some Atoms on some motherboards with some >>>> firmwares are 64-bit CPU. >>> don't know of any now in shops that are not >>=20 >> http://soekris.com/products/net5501.html >> http://soekris.com/products/net6501.html >>=20 >> DES >> -- >> Dag-Erling Sm=F8rgrav - des@des.no >>=20 >=20 >=20 > I am NOT able to understand the merit of these products with respect = to > their features and PRICES . >=20 > It is possible to assemble much more cheaper full featured PC like = systems > ( DDR3 memory , 64-bit capable processors , with a disadvantage about = power > requirements ) . Often times the power consumption is the most important bit, so much so = you sacrifice speed and memory to get the power down to fit into a small = power budget. Just because you have the ability to purchase a faster = machine for less doesn't make that faster machine suitable for the job. Warner >=20 > Thank you very much . >=20 > Mehmet Erol Sanliturk > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 14:17:20 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4DEC4D92; Tue, 2 Apr 2013 14:17:20 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 7A7866EF; Tue, 2 Apr 2013 14:17:19 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r32EHI91082326; Tue, 2 Apr 2013 18:17:18 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r32EHHK9082325; Tue, 2 Apr 2013 18:17:17 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 2 Apr 2013 18:17:17 +0400 From: Gleb Smirnoff To: Luigi Rizzo , kib@FreeBSD.org Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130402141717.GK76816@glebius.int.ru> References: <20130401115128.GZ76816@FreeBSD.org> <20130402013758.GA96904@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="YyxzkC/DtE3JUx8+" Content-Disposition: inline In-Reply-To: <20130402013758.GA96904@onelab2.iet.unipi.it> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 14:17:20 -0000 --YyxzkC/DtE3JUx8+ Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Luigi, On Tue, Apr 02, 2013 at 03:37:58AM +0200, Luigi Rizzo wrote: L> API: L> L> > o MI implementation of counter_u64_add() is: L> > L> > critical_enter(); L> > *(uint64_t *)zpcpu_get(c) += inc; L> > critical_exit(); L> L> - there are several places which use multiple counters L> (e.g. packet and byte counters, global and per flow/socket), L> so i wonder if it may help to provide a "protected" version of L> counter_u64_add() that requires the critical_enter/exit L> only once. Something like L> L> PROTECT_COUNTERS( L> safe_counter_u64_add(c, x); L> safe_counter_u64_add(c, x); L> safe_counter_u64_add(c, x); L> ); L> L> where PROTECT_COUNTERS() would translate into the critical_enter/exit L> where required, and nothing on other architectures. Here is patch for review. It adds 4 more primitives: counter_enter(); counter_u64_add_prot(c, x); counter_u64_subtract_prot(c, x); counter_exit(); L> BENCHMARK: L> L> > I've got a simple benchmark. A syscall that solely updates a counter is L> > implemented. Number of processes is spawned, equal to number of cores, L> > each process binds itself to a dedicated CPU and calls the syscall 10^6 L> > times and exits. Parent wait(2)s for them all and I measure real time of L> L> - I am under the impression that these benchmarks are dominated L> by the syscall time, and the new counters would exhibit a lot L> better relative performance (compared to racy or atomic) L> by doing 10..100 counter ops per syscall. Any chance to retry L> the test in this configuration ? Ok, as you wish. Apparently compiler optimises things like: for (int i = 0; i < rounds; i++) the_counter += v; To avoid optimisations here I declared the_counter as volatile. Is the benchmark fair after that? Anyway, here are results for (rounds == 2000000000): x counter_u64_add(), result == 2000000000 * ncpus + racy increment, result == 2022232241 (!!!) +------------------------------------------------------------------------------+ | x + | | x + | | x ++ | | x ++ | | x x +++ +| ||_MA__| |_MA_| | +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 5 5.44 5.01 5.053 0.13605963 + 10 8.16 8.53 8.2 8.235 0.10721215 Difference at 95.0% confidence 3.182 +/- 0.115089 62.9725% +/- 2.27764% (Student's t, pooled s = 0.122488) So 63% speedup, not speaking on the fact that in such a tight loop 98% of parallel updates are lost on racy counter :) A tight loop with atomic_add() is 22 times (twenty two times) slower than new counter. I didn't bother to run ministat :) -- Totus tuus, Glebius. --YyxzkC/DtE3JUx8+ Content-Type: text/x-diff; charset=koi8-r Content-Disposition: attachment; filename="counter_API_extend.diff" Index: sys/sparc64/include/counter.h =================================================================== --- sys/sparc64/include/counter.h (revision 249002) +++ sys/sparc64/include/counter.h (working copy) @@ -31,22 +31,33 @@ #include +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() + +#define counter_u64_add_prot(c, inc) do { \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/ia64/include/counter.h =================================================================== --- sys/ia64/include/counter.h (revision 249002) +++ sys/ia64/include/counter.h (working copy) @@ -31,22 +31,33 @@ #include +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() + +#define counter_u64_add_prot(c, inc) do { \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/i386/include/counter.h =================================================================== --- sys/i386/include/counter.h (revision 249002) +++ sys/i386/include/counter.h (working copy) @@ -33,6 +33,16 @@ #include #include +#define counter_enter() do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + critical_enter(); \ +} while (0) + +#define counter_exit() do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + critical_exit(); \ +} while (0) + static inline void counter_64_inc_8b(uint64_t *p, uint64_t inc) { @@ -52,24 +62,36 @@ counter_64_inc_8b(uint64_t *p, uint64_t inc) : "memory", "cc", "eax", "edx", "ebx", "ecx"); } +#define counter_u64_add_prot(c, inc) do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + *(uint64_t *)zpcpu_get(c) += (inc); \ + else \ + counter_64_inc_8b((c), (inc)); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ + else \ + counter_64_inc_8b((c), -(int64_t)(dec));\ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - if ((cpu_feature & CPUID_CX8) == 0) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); - } else { - counter_64_inc_8b(c, inc); - } + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - counter_u64_add(c, -(int64_t)dec); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/amd64/include/counter.h =================================================================== --- sys/amd64/include/counter.h (revision 249002) +++ sys/amd64/include/counter.h (working copy) @@ -33,6 +33,12 @@ extern struct pcpu __pcpu[1]; +#define counter_enter() do {} while (0) +#define counter_exit() do {} while (0) + +#define counter_u64_add_prot(c, i) counter_u64_add(c, i) +#define counter_u64_subtract_prot(c, i) counter_u64_subtract(c, i) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { Index: sys/arm/include/counter.h =================================================================== --- sys/arm/include/counter.h (revision 249002) +++ sys/arm/include/counter.h (working copy) @@ -31,22 +31,33 @@ #include +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() + +#define counter_u64_add_prot(c, inc) do { \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/powerpc/include/counter.h =================================================================== --- sys/powerpc/include/counter.h (revision 249002) +++ sys/powerpc/include/counter.h (working copy) @@ -33,6 +33,12 @@ #if defined(AIM) && defined(__powerpc64__) +#define counter_enter() do {} while (0) +#define counter_exit() do {} while (0) + +#define counter_u64_add_prot(c, i) counter_u64_add(c, i) +#define counter_u64_subtract_prot(c, i) counter_u64_subtract(c, i) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { @@ -59,24 +65,33 @@ counter_u64_subtract(counter_u64_t c, uint64_t dec #else /* !AIM || !64bit */ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() + +#define counter_u64_add_prot(c, inc) do { \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } -#endif /* AIM 64bit */ - #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/mips/include/counter.h =================================================================== --- sys/mips/include/counter.h (revision 249002) +++ sys/mips/include/counter.h (working copy) @@ -31,22 +31,33 @@ #include +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() + +#define counter_u64_add_prot(c, inc) do { \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) + +#define counter_u64_subtract_prot(c, dec) do { \ + *(uint64_t *)zpcpu_get(c) -= (dec); \ +} while (0) + static inline void counter_u64_add(counter_u64_t c, uint64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } static inline void counter_u64_subtract(counter_u64_t c, uint64_t dec) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_subtract_prot(c, dec); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ --YyxzkC/DtE3JUx8+-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 14:36:05 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 864DA270; Tue, 2 Apr 2013 14:36:05 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 1F0427DE; Tue, 2 Apr 2013 14:36:04 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id r32EZvX5031751 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 2 Apr 2013 16:36:02 +0200 (CEST) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id r32DMR8s095574; Tue, 2 Apr 2013 15:22:27 +0200 (CEST) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Tue, 2 Apr 2013 15:22:27 +0200 From: Paul Schenkeveld To: freebsd-arch@freebsd.org, FreeBSD Hackers Subject: Re: considering i386 as a tier 1 architecture Message-ID: <20130402132227.GA73670@psconsult.nl> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> <20130402102220.GA28545@eris.bzerk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130402102220.GA28545@eris.bzerk.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 14:36:05 -0000 On Tue, Apr 02, 2013 at 10:22:20AM +0000, Ruben de Groot wrote: > On Tue, Apr 02, 2013 at 03:10:56AM -0700, Mehmet Erol Sanliturk typed: > > On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm??rgrav wrote: > > > > > Wojciech Puchar writes: > > > > Lev Serebryakov writes: > > > > > It is not exact so. Some Atoms on some motherboards with some > > > > > firmwares are 64-bit CPU. > > > > don't know of any now in shops that are not > > > > > > http://soekris.com/products/net5501.html > > > http://soekris.com/products/net6501.html > > > > > > DES > > > -- > > > Dag-Erling Sm??rgrav - des@des.no > > > > > > > > > I am NOT able to understand the merit of these products with respect to > > their features and PRICES . > > They are extremely stable and robust. > > > It is possible to assemble much more cheaper full featured PC like systems > > ( DDR3 memory , 64-bit capable processors , with a disadvantage about power > > requirements ) . > > You can also get much bigger portions at MacDonald than what you get in a > five star restaurant. Soekris boards are perhaps not five star boards but at least they have four :) Although the thread started as an april fools day prank, it's getting serious now about the value of having i386 next to amd64. I'm using quite a number of net4501/net4801/net5501/net6501 in many places just because I haven't found anything that can to the same job with the same reliability at the same low power diet for a reasonable price. For people on a tight budget with lower reliability expectations there are the PC-engines Alix boards. Except for the net6501, none of these can run amd64. Even though the net6501 can run amd64, I prefer running i386 on them (and other boards where I do not need >= 4GB of RAM or the large address space) instead of amd64 just because the system image is so much smaller, requiring less storage on your filesystem (often a small flash device), less time to upload changes over the Internet when doing remote upgrades and they are more efficient with virtual memory. Running amd64 when not really needed is just a waste of resources. There have been discussions in the past whether is would make sense to run a 32-bit userland on top of a amd64 kernel sou you can have >4GB of RAM but keep your userland relatively small. There are only few applications that really benefit from 64 bit address space, others could well be 32 bit apps. Just some random numbers to illustrate my point: i386$ size /bin/sh /bin/ls /usr/bin/find /usr/bin/cc text data bss dec hex filename 111533 1048 7460 120041 1d4e9 /bin/sh 22808 572 396 23776 5ce0 /bin/ls 33098 760 3392 37250 9182 /usr/bin/find 314841 9376 18204 342421 53995 /usr/bin/cc amd64$ size /bin/sh /bin/ls /usr/bin/find /usr/bin/cc text data bss dec hex filename 129371 1992 10272 141635 22943 /bin/sh 26255 1144 536 27935 6d1f /bin/ls 43464 1352 4680 49496 c158 /usr/bin/find 383330 15296 58664 457290 6fa4a /usr/bin/cc Kind regards, Paul Schenkeveld From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 14:56:04 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1A2C4A37; Tue, 2 Apr 2013 14:56:04 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 95619895; Tue, 2 Apr 2013 14:56:03 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 4DEC273027; Tue, 2 Apr 2013 16:57:22 +0200 (CEST) Date: Tue, 2 Apr 2013 16:57:22 +0200 From: Luigi Rizzo To: Gleb Smirnoff Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130402145722.GA9161@onelab2.iet.unipi.it> References: <20130401115128.GZ76816@FreeBSD.org> <20130402013758.GA96904@onelab2.iet.unipi.it> <20130402141717.GK76816@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130402141717.GK76816@glebius.int.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: arch@FreeBSD.org, kib@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 14:56:04 -0000 On Tue, Apr 02, 2013 at 06:17:17PM +0400, Gleb Smirnoff wrote: > Luigi, > > On Tue, Apr 02, 2013 at 03:37:58AM +0200, Luigi Rizzo wrote: > L> API: ... > L> - there are several places which use multiple counters > L> (e.g. packet and byte counters, global and per flow/socket), > L> so i wonder if it may help to provide a "protected" version of > L> counter_u64_add() that requires the critical_enter/exit > L> only once. Something like ... > Here is patch for review. It adds 4 more primitives: > > counter_enter(); > counter_u64_add_prot(c, x); > counter_u64_subtract_prot(c, x); > counter_exit(); thanks for the patch. I have three more comments: - is it really necessary to have the "subtract" version ? Couldn't one just make "x" an int64_t ? or it gives too many warnings at runtime maybe ? - (this can be fixed later) in the i386 version, counter_enter() and counter_exit() have an if statement which may become quite expensive if mispredicted. Also, the test is repeated 3 times in counter_u64_add() (enter/add/exit). Hopefully the compiler optimizes out the extra calls, but the previous version seemed more readable. Anyways, at some point we should figure out whether putting likely/unlikely annotations on the result of (cpu_feature & CPUID_CX8) may improve performance where it matters. - do you plan to provide an API to initialize a counter to 0 or a specific value ? I suppose this is done implicitly on allocation, but there are cases (e.g. ipfw) where the application explicitly zeroes counters. > L> BENCHMARK: > L> > L> > I've got a simple benchmark. A syscall that solely updates a counter is > L> > implemented. Number of processes is spawned, equal to number of cores, > L> > each process binds itself to a dedicated CPU and calls the syscall 10^6 > L> > times and exits. Parent wait(2)s for them all and I measure real time of > L> > L> - I am under the impression that these benchmarks are dominated > L> by the syscall time, and the new counters would exhibit a lot > L> better relative performance (compared to racy or atomic) > L> by doing 10..100 counter ops per syscall. Any chance to retry > L> the test in this configuration ? > > Ok, as you wish. > > Apparently compiler optimises things like: > > for (int i = 0; i < rounds; i++) > the_counter += v; > > To avoid optimisations here I declared the_counter as volatile. Is the > benchmark fair after that? Anyway, here are results for (rounds == 2000000000): > > x counter_u64_add(), result == 2000000000 * ncpus > + racy increment, result == 2022232241 (!!!) > +------------------------------------------------------------------------------+ > | x + | > | x + | > | x ++ | > | x ++ | > | x x +++ +| > ||_MA__| |_MA_| | > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 10 5 5.44 5.01 5.053 0.13605963 > + 10 8.16 8.53 8.2 8.235 0.10721215 > Difference at 95.0% confidence > 3.182 +/- 0.115089 > 62.9725% +/- 2.27764% > (Student's t, pooled s = 0.122488) > > So 63% speedup, not speaking on the fact that in such a tight loop 98% of > parallel updates are lost on racy counter :) > > A tight loop with atomic_add() is 22 times (twenty two times) slower than > new counter. I didn't bother to run ministat :) yes i think this really makes justice of the improvements of the new code (i am a bit unclear on what actual test you ran / how many counter_u64_add() per syscall you have, but i do expect the racy counters to be much slower and much less reliable, and the 20x slowdown with atomics is completely expected.) cheers luigi From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 15:40:19 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9E99C30E; Tue, 2 Apr 2013 15:40:19 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id CC4C4AFF; Tue, 2 Apr 2013 15:40:18 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r32FeHbl082937; Tue, 2 Apr 2013 19:40:17 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r32FeHUT082936; Tue, 2 Apr 2013 19:40:17 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 2 Apr 2013 19:40:17 +0400 From: Gleb Smirnoff To: Luigi Rizzo Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130402154017.GN76816@FreeBSD.org> References: <20130401115128.GZ76816@FreeBSD.org> <20130402013758.GA96904@onelab2.iet.unipi.it> <20130402141717.GK76816@glebius.int.ru> <20130402145722.GA9161@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="z87VqPJ/HsYrR2WM" Content-Disposition: inline In-Reply-To: <20130402145722.GA9161@onelab2.iet.unipi.it> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org, kib@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 15:40:19 -0000 --z87VqPJ/HsYrR2WM Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Luigi, On Tue, Apr 02, 2013 at 04:57:22PM +0200, Luigi Rizzo wrote: L> > Here is patch for review. It adds 4 more primitives: L> > L> > counter_enter(); L> > counter_u64_add_prot(c, x); L> > counter_u64_subtract_prot(c, x); L> > counter_exit(); L> L> thanks for the patch. I have three more comments: L> L> - is it really necessary to have the "subtract" version ? L> Couldn't one just make "x" an int64_t ? or it gives L> too many warnings at runtime maybe ? Agreed. See patch. L> - (this can be fixed later) in the i386 version, counter_enter() L> and counter_exit() have an if statement which may become quite L> expensive if mispredicted. Also, the test is repeated 3 times in L> counter_u64_add() (enter/add/exit). Hopefully the compiler L> optimizes out the extra calls, but the previous version seemed L> more readable. Anyways, at some point we should figure out L> whether putting likely/unlikely annotations on the result of L> (cpu_feature & CPUID_CX8) may improve performance where it matters. Agreed. See patch. L> - do you plan to provide an API to initialize a counter to 0 or a L> specific value ? I suppose this is done implicitly on allocation, L> but there are cases (e.g. ipfw) where the application explicitly L> zeroes counters. There already is counter_u64_zero(). L> > So 63% speedup, not speaking on the fact that in such a tight loop 98% of L> > parallel updates are lost on racy counter :) L> > L> > A tight loop with atomic_add() is 22 times (twenty two times) slower than L> > new counter. I didn't bother to run ministat :) L> L> yes i think this really makes justice of the improvements of the new code L> (i am a bit unclear on what actual test you ran / how many counter_u64_add() L> per syscall you have, but i do expect the racy counters to be much slower L> and much less reliable, and the 20x slowdown with atomics is completely L> expected.) The test made 2 * 10^6 iterations of updating a counter in a for loop. -- Totus tuus, Glebius. --z87VqPJ/HsYrR2WM Content-Type: text/x-diff; charset=koi8-r Content-Disposition: attachment; filename="counter_API_extend.diff" Index: sys/sparc64/include/counter.h =================================================================== --- sys/sparc64/include/counter.h (revision 249002) +++ sys/sparc64/include/counter.h (working copy) @@ -31,22 +31,21 @@ #include -static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) -{ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); -} +#define counter_u64_add_prot(c, inc) do { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) +counter_u64_add(counter_u64_t c, int64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/ia64/include/counter.h =================================================================== --- sys/ia64/include/counter.h (revision 249002) +++ sys/ia64/include/counter.h (working copy) @@ -31,22 +31,21 @@ #include -static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) -{ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); -} +#define counter_u64_add_prot(c, inc) do { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) +counter_u64_add(counter_u64_t c, int64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/netinet/ip_input.c =================================================================== --- sys/netinet/ip_input.c (revision 249002) +++ sys/netinet/ip_input.c (working copy) @@ -293,7 +293,7 @@ void kmod_ipstat_dec(int statnum) { - counter_u64_subtract((counter_u64_t )&V_ipstatp + statnum, 1); + counter_u64_add((counter_u64_t )&V_ipstatp + statnum, -1); } static int Index: sys/netinet/ip_var.h =================================================================== --- sys/netinet/ip_var.h (revision 249002) +++ sys/netinet/ip_var.h (working copy) @@ -175,7 +175,7 @@ VNET_DECLARE(struct ipstat_p, ipstatp); #define IPSTAT_ADD(name, val) counter_u64_add(V_ipstatp.name, (val)) #define IPSTAT_SUB(name, val) counter_u64_subtract(V_ipstatp.name, (val)) #define IPSTAT_INC(name) IPSTAT_ADD(name, 1) -#define IPSTAT_DEC(name) IPSTAT_SUB(name, 1) +#define IPSTAT_DEC(name) IPSTAT_ADD(name, -1) /* * Kernel module consumers must use this accessor macro. Index: sys/i386/include/counter.h =================================================================== --- sys/i386/include/counter.h (revision 249002) +++ sys/i386/include/counter.h (working copy) @@ -33,8 +33,18 @@ #include #include +#define counter_enter() do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + critical_enter(); \ +} while (0) + +#define counter_exit() do { \ + if ((cpu_feature & CPUID_CX8) == 0) \ + critical_exit(); \ +} while (0) + static inline void -counter_64_inc_8b(uint64_t *p, uint64_t inc) +counter_64_inc_8b(uint64_t *p, int64_t inc) { __asm __volatile( @@ -52,8 +62,16 @@ static inline void : "memory", "cc", "eax", "edx", "ebx", "ecx"); } +#define counter_u64_add_prot(c, inc) do { \ + if ((cpu_feature & CPUID_CX8) == 0) { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ + } else \ + counter_64_inc_8b((c), (inc)); \ +} while (0) + static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) +counter_u64_add(counter_u64_t c, int64_t inc) { if ((cpu_feature & CPUID_CX8) == 0) { @@ -65,11 +83,4 @@ static inline void } } -static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) -{ - - counter_u64_add(c, -(int64_t)dec); -} - #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/amd64/include/counter.h =================================================================== --- sys/amd64/include/counter.h (revision 249002) +++ sys/amd64/include/counter.h (working copy) @@ -33,8 +33,13 @@ extern struct pcpu __pcpu[1]; +#define counter_enter() do {} while (0) +#define counter_exit() do {} while (0) + +#define counter_u64_add_prot(c, i) counter_u64_add(c, i) + static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) +counter_u64_add(counter_u64_t c, int64_t inc) { __asm __volatile("addq\t%1,%%gs:(%0)" @@ -43,14 +48,4 @@ static inline void : "memory", "cc"); } -static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) -{ - - __asm __volatile("subq\t%1,%%gs:(%0)" - : - : "r" ((char *)c - (char *)&__pcpu[0]), "r" (dec) - : "memory", "cc"); -} - #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/arm/include/counter.h =================================================================== --- sys/arm/include/counter.h (revision 249002) +++ sys/arm/include/counter.h (working copy) @@ -31,22 +31,21 @@ #include -static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) -{ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); -} +#define counter_u64_add_prot(c, inc) do { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) +counter_u64_add(counter_u64_t c, int64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/powerpc/include/counter.h =================================================================== --- sys/powerpc/include/counter.h (revision 249002) +++ sys/powerpc/include/counter.h (working copy) @@ -33,8 +33,13 @@ #if defined(AIM) && defined(__powerpc64__) +#define counter_enter() do {} while (0) +#define counter_exit() do {} while (0) + +#define counter_u64_add_prot(c, i) counter_u64_add(c, i) + static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) +counter_u64_add(counter_u64_t c, int64_t inc) { uint64_t ccpu, old; @@ -50,33 +55,23 @@ static inline void : "cc", "memory"); } -static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) -{ - - counter_u64_add(c, -dec); -} - #else /* !AIM || !64bit */ -static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) -{ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); -} +#define counter_u64_add_prot(c, inc) do { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) +counter_u64_add(counter_u64_t c, int64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } -#endif /* AIM 64bit */ - #endif /* ! __MACHINE_COUNTER_H__ */ Index: sys/mips/include/counter.h =================================================================== --- sys/mips/include/counter.h (revision 249002) +++ sys/mips/include/counter.h (working copy) @@ -31,22 +31,21 @@ #include -static inline void -counter_u64_add(counter_u64_t c, uint64_t inc) -{ +#define counter_enter() critical_enter() +#define counter_exit() critical_exit() - critical_enter(); - *(uint64_t *)zpcpu_get(c) += inc; - critical_exit(); -} +#define counter_u64_add_prot(c, inc) do { \ + CRITICAL_ASSERT(td); \ + *(uint64_t *)zpcpu_get(c) += (inc); \ +} while (0) static inline void -counter_u64_subtract(counter_u64_t c, uint64_t dec) +counter_u64_add(counter_u64_t c, int64_t inc) { - critical_enter(); - *(uint64_t *)zpcpu_get(c) -= dec; - critical_exit(); + counter_enter(); + counter_u64_add_prot(c, inc); + counter_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ --z87VqPJ/HsYrR2WM-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 16:35:35 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A92C424B; Tue, 2 Apr 2013 16:35:35 +0000 (UTC) (envelope-from chris@behanna.org) Received: from alayta.pair.com (alayta.pair.com [209.68.4.24]) by mx1.freebsd.org (Postfix) with ESMTP id 87677EE1; Tue, 2 Apr 2013 16:35:35 +0000 (UTC) Received: from tourmalet.ticom-geo.com (unknown [64.132.190.26]) by alayta.pair.com (Postfix) with ESMTPSA id EC406D9843; Tue, 2 Apr 2013 12:27:30 -0400 (EDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: considering i386 as a tier 1 architecture From: Chris BeHanna In-Reply-To: <20130402132227.GA73670@psconsult.nl> Date: Tue, 2 Apr 2013 11:27:30 -0500 Content-Transfer-Encoding: 7bit Message-Id: <610E1139-3CCC-4BDF-B15A-4B24F26D7A95@behanna.org> References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> <20130402102220.GA28545@eris.bzerk.org> <20130402132227.GA73670@psconsult.nl> To: freebsd-arch@freebsd.org X-Mailer: Apple Mail (2.1503) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 16:35:35 -0000 Goodness gracious, did no one see the date on the original post? What's the limit on this fishing hole? -- Chris BeHanna chris@behanna.org From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 16:43:45 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B7148455 for ; Tue, 2 Apr 2013 16:43:45 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-qe0-f45.google.com (mail-qe0-f45.google.com [209.85.128.45]) by mx1.freebsd.org (Postfix) with ESMTP id 76BBCF3B for ; Tue, 2 Apr 2013 16:43:45 +0000 (UTC) Received: by mail-qe0-f45.google.com with SMTP id b4so337364qen.32 for ; Tue, 02 Apr 2013 09:43:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=lNMoZWAG+kGqJT+OfLpmBVSzV2ZtcuRpmABaEcTg3oQ=; b=cWZP1tXXlOeIUYVv1uPtHkV+mazeX3CLDyEpyNSwX2eoK7UbH6jBJ38YA4m5M3Y/gm uLCbUOPkHu4RaklKAo07SMQ7LFH0ZVWNrJCoqRrcqiv8tvhn2fKV+zpbI9rwGPdTp3vE Xpf1xOAcUjFfSGUpPXAybqG5u5QxrRpdyB+ATre2Vvq/5BDf0Gtsf2YTjWqLeyW24J1B 4DSaz5/VfIVvFxZeBwzfGttxXI/WdNDNEnUh1zFUUI4pLH+xD4GeDn+R4OJsE9ju9TSp OqYLoNn/iK03TdPG23izyLlviTCrz9ipEu4x3eR0uaBGQ+E5IXg3CO7FElHKXNZWDHpe sfVA== X-Received: by 10.49.117.7 with SMTP id ka7mr18267771qeb.38.1364921019154; Tue, 02 Apr 2013 09:43:39 -0700 (PDT) Received: from monkey-bot.int.fusionio.com ([209.117.142.2]) by mx.google.com with ESMTPS id fr4sm2906360qab.3.2013.04.02.09.43.36 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 02 Apr 2013 09:43:37 -0700 (PDT) Sender: Warner Losh Subject: Re: considering i386 as a tier 1 architecture Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <610E1139-3CCC-4BDF-B15A-4B24F26D7A95@behanna.org> Date: Tue, 2 Apr 2013 10:43:34 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> <20130402102220.GA28545@eris.bzerk.org> <20130402132227.GA73670@psconsult.nl> <610E1139-3CCC-4BDF-B15A-4B24F26D7A95@behanna.org> To: Chris BeHanna X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQkbXWz1IwTYVv0llt+SKyXge3thTcCKPwVNym8PQ1jTYp8ZqOX3b2nAilchIVxRXKUuQbM2 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 16:43:45 -0000 On Apr 2, 2013, at 10:27 AM, Chris BeHanna wrote: > Goodness gracious, did no one see the date on the original post? >=20 > What's the limit on this fishing hole? Three internet Trolls, two wise old owls and a april fool in a pear tree = from the looks of it. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 18:11:42 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4D3BE9CB; Tue, 2 Apr 2013 18:11:42 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2F496BD6; Tue, 2 Apr 2013 18:11:42 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 116851A3CD6; Tue, 2 Apr 2013 11:11:34 -0700 (PDT) Message-ID: <515B1F4C.9030001@mu.org> Date: Tue, 02 Apr 2013 11:11:24 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Paul Schenkeveld Subject: Re: considering i386 as a tier 1 architecture References: <944760435.20130401210118@serebryakov.spb.ru> <8638v9e22j.fsf@ds4.des.no> <20130402102220.GA28545@eris.bzerk.org> <20130402132227.GA73670@psconsult.nl> In-Reply-To: <20130402132227.GA73670@psconsult.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 18:11:42 -0000 As far I can tell it's now April 2nd in all time zones. Can we now end this thread? thank you, -Alfred On 4/2/13 6:22 AM, Paul Schenkeveld wrote: > On Tue, Apr 02, 2013 at 10:22:20AM +0000, Ruben de Groot wrote: >> On Tue, Apr 02, 2013 at 03:10:56AM -0700, Mehmet Erol Sanliturk typed: >>> On Tue, Apr 2, 2013 at 2:04 AM, Dag-Erling Sm??rgrav wrote: >>> >>>> Wojciech Puchar writes: >>>>> Lev Serebryakov writes: >>>>>> It is not exact so. Some Atoms on some motherboards with some >>>>>> firmwares are 64-bit CPU. >>>>> don't know of any now in shops that are not >>>> http://soekris.com/products/net5501.html >>>> http://soekris.com/products/net6501.html >>>> >>>> DES >>>> -- >>>> Dag-Erling Sm??rgrav - des@des.no >>>> >>> >>> I am NOT able to understand the merit of these products with respect to >>> their features and PRICES . >> They are extremely stable and robust. >> >>> It is possible to assemble much more cheaper full featured PC like systems >>> ( DDR3 memory , 64-bit capable processors , with a disadvantage about power >>> requirements ) . >> You can also get much bigger portions at MacDonald than what you get in a >> five star restaurant. > Soekris boards are perhaps not five star boards but at least they have > four :) > > Although the thread started as an april fools day prank, it's getting > serious now about the value of having i386 next to amd64. > > I'm using quite a number of net4501/net4801/net5501/net6501 in many > places just because I haven't found anything that can to the same job > with the same reliability at the same low power diet for a reasonable > price. > > For people on a tight budget with lower reliability expectations there > are the PC-engines Alix boards. Except for the net6501, none of these > can run amd64. > > Even though the net6501 can run amd64, I prefer running i386 on them > (and other boards where I do not need >= 4GB of RAM or the large address > space) instead of amd64 just because the system image is so much smaller, > requiring less storage on your filesystem (often a small flash device), > less time to upload changes over the Internet when doing remote upgrades > and they are more efficient with virtual memory. Running amd64 when not > really needed is just a waste of resources. > > There have been discussions in the past whether is would make sense to > run a 32-bit userland on top of a amd64 kernel sou you can have >4GB of > RAM but keep your userland relatively small. There are only few > applications that really benefit from 64 bit address space, others could > well be 32 bit apps. > > Just some random numbers to illustrate my point: > > i386$ size /bin/sh /bin/ls /usr/bin/find /usr/bin/cc > > text data bss dec hex filename > 111533 1048 7460 120041 1d4e9 /bin/sh > 22808 572 396 23776 5ce0 /bin/ls > 33098 760 3392 37250 9182 /usr/bin/find > 314841 9376 18204 342421 53995 /usr/bin/cc > > amd64$ size /bin/sh /bin/ls /usr/bin/find /usr/bin/cc > > text data bss dec hex filename > 129371 1992 10272 141635 22943 /bin/sh > 26255 1144 536 27935 6d1f /bin/ls > 43464 1352 4680 49496 c158 /usr/bin/find > 383330 15296 58664 457290 6fa4a /usr/bin/cc > > Kind regards, > > Paul Schenkeveld > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Apr 2 23:24:11 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D5EE5522; Tue, 2 Apr 2013 23:24:11 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id A30A9339; Tue, 2 Apr 2013 23:24:10 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 82E0DAAC; Wed, 3 Apr 2013 01:20:41 +0200 (CEST) Date: Wed, 3 Apr 2013 01:26:07 +0200 From: Pawel Jakub Dawidek To: Gleb Smirnoff Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130402232606.GC1810@garage.freebsd.pl> References: <20130401115128.GZ76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HG+GLK89HZ1zG0kk" Content-Disposition: inline In-Reply-To: <20130401115128.GZ76816@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 23:24:11 -0000 --HG+GLK89HZ1zG0kk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > Hi! >=20 > Together with Konstantin Belousov (kib@) we developed a new API that is > initially purposed for (but not limited to) collecting statistical > data in kernel. Is there any plan to implement universal way of exporting those statistics out of the kernel? Solaris has a framework for in-kernel statistics, which are exported via kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you can try 'sysctl kstat'. It would be nice for counter_u64_alloc() to take additional argument 'name' and to create sysctl for the counter automatically. We could then slowly start migrating userland tools to use sysctls (or some wrapper userland API), but we immediately make those statistics available for use in scripts. > o Tiny API for counter(9): >=20 > counter_u64_t > counter_u64_alloc(int wait); >=20 > void > counter_u64_free(counter_u64_t cnt); >=20 > void > counter_u64_add(counter_u64_t cnt, uint64_t inc); >=20 > uint64_t > counter_u64_fetch(counter_u64_t cnt); Do you really expect other types in the future? If so, could we at least create generic counter_t that internally keeps the type? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --HG+GLK89HZ1zG0kk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFbaQ4ACgkQForvXbEpPzSimwCeN6N1ztUz4EQrQrX9yDAouWAy bEsAoOZkQe7H7tjPAXFK3beyZjv4LD8z =qcsC -----END PGP SIGNATURE----- --HG+GLK89HZ1zG0kk-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 00:25:25 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C3955EF5; Wed, 3 Apr 2013 00:25:25 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 8D71A729; Wed, 3 Apr 2013 00:25:25 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1UNBVz-0002AW-UT; Tue, 02 Apr 2013 20:25:23 -0400 Date: Tue, 2 Apr 2013 20:25:23 -0400 From: Gary Palmer To: Pawel Jakub Dawidek Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403002523.GA96431@in-addr.com> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130402232606.GC1810@garage.freebsd.pl> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 00:25:25 -0000 On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: > On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > > Hi! > > > > Together with Konstantin Belousov (kib@) we developed a new API that is > > initially purposed for (but not limited to) collecting statistical > > data in kernel. > > Is there any plan to implement universal way of exporting those > statistics out of the kernel? > > Solaris has a framework for in-kernel statistics, which are exported via > kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you > can try 'sysctl kstat'. > > It would be nice for counter_u64_alloc() to take additional argument > 'name' and to create sysctl for the counter automatically. We could then > slowly start migrating userland tools to use sysctls (or some wrapper > userland API), but we immediately make those statistics available for > use in scripts. Sorry for potentially turning this into a bikeshed, but is sysctl the best interface for this? It is great for scripts as the CLI is already there, however it is not a bulk interface so grabbing all the ZFS statistics takes quite a few trips through our system call handler - 438 on my 9.1 box for "sysctl kstat" (found via ktrace and then kdump | grep -c SCTL), which is not ideal when there are only 87 stats there. (5 calls per OID returned and 3 initial calls to get set up) I'm not sure I have a better alternative other than a geom style bulk export via XML or some other format, however I wanted to raise it for consideration. I wouldn't hold up the checkin of the code for this, however if another way of gathering the data is needed/desired then it would be good to get it in before people get too used to the sysctl way. Thanks, Gary From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 00:27:27 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 857C5FAA; Wed, 3 Apr 2013 00:27:27 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 49995737; Wed, 3 Apr 2013 00:27:27 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id B400473027; Wed, 3 Apr 2013 02:28:46 +0200 (CEST) Date: Wed, 3 Apr 2013 02:28:46 +0200 From: Luigi Rizzo To: Pawel Jakub Dawidek Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403002846.GB15334@onelab2.iet.unipi.it> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130402232606.GC1810@garage.freebsd.pl> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 00:27:27 -0000 On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: > On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > > Hi! > > > > Together with Konstantin Belousov (kib@) we developed a new API that is > > initially purposed for (but not limited to) collecting statistical > > data in kernel. > > Is there any plan to implement universal way of exporting those > statistics out of the kernel? > > Solaris has a framework for in-kernel statistics, which are exported via > kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you > can try 'sysctl kstat'. > > It would be nice for counter_u64_alloc() to take additional argument > 'name' and to create sysctl for the counter automatically. We could then > slowly start migrating userland tools to use sysctls (or some wrapper > userland API), but we immediately make those statistics available for > use in scripts. that is an interesting idea but i believe it can be effectively built as a wrapper on top of the counter_u64_alloc() routine: name_counter(counter_t c, const char *fmt, ...); free_named_counter(counter_t c); After all the name->counter mapping is unidirectional, and possibly not even necessary on every single counter (think of ipfw dynamic rules, created on packet arrivals, so the counter alloc/dealloc needs to be fast). It might be useful for the name_counter() routine to support a printf-style argument to make it easy to build names. > > o Tiny API for counter(9): > > > > counter_u64_t > > counter_u64_alloc(int wait); > > > > void > > counter_u64_free(counter_u64_t cnt); > > > > void > > counter_u64_add(counter_u64_t cnt, uint64_t inc); > > > > uint64_t > > counter_u64_fetch(counter_u64_t cnt); > > Do you really expect other types in the future? If so, could we at least > create generic counter_t that internally keeps the type? I read the u64 in the name mostly as a reminder to users of the counter size. It might actually make sense is to change the type to s64. This way we could have counters that go negative, and also use them to accumulate sbintime_t values. But otherwise i am not sure that we want other types. u32/s32 might save atomic/critical_enter ops on some archs, but they saturate so quickly that probably are a bad idea. And 63/64 bits are quite large already. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 03:49:50 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 08BEA2AB; Wed, 3 Apr 2013 03:49:50 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 5E851174; Wed, 3 Apr 2013 03:49:49 +0000 (UTC) Received: from ur.dons.net.au (ppp14-2-47-50.lns21.adl2.internode.on.net [14.2.47.50]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id r333nSQZ004904 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 3 Apr 2013 14:19:34 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Content-Type: multipart/signed; boundary="Apple-Mail=_04E353E7-9BEC-46B0-8E8A-E9ADA3A56B16"; protocol="application/pkcs7-signature"; micalg=sha1 From: "Daniel O'Connor" In-Reply-To: <20130403002523.GA96431@in-addr.com> Date: Wed, 3 Apr 2013 14:19:28 +1030 Message-Id: References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002523.GA96431@in-addr.com> To: Gary Palmer X-Mailer: Apple Mail (2.1503) X-Spam-Score: 0.163 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 03:49:50 -0000 --Apple-Mail=_04E353E7-9BEC-46B0-8E8A-E9ADA3A56B16 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 03/04/2013, at 10:55, Gary Palmer wrote: > I'm not sure I have a better alternative other than a geom style bulk > export via XML or some other format, however I wanted to raise it for > consideration. I wouldn't hold up the checkin of the code for this, > however if another way of gathering the data is needed/desired then it > would be good to get it in before people get too used to the sysctl=20 > way. =20 Perhaps the sysctl interface could change to accommodate bulk requests = (bikeshed ^ 2 or bikeception :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --Apple-Mail=_04E353E7-9BEC-46B0-8E8A-E9ADA3A56B16 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIINWTCCBjQw ggQcoAMCAQICAR4wDQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0 Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAn BgNVBAMTIFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA3MTAyNDIxMDE1NVoX DTE3MTAyNDIxMDE1NVowgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSsw KQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFy dENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQTCCASIwDQYJKoZIhvcN AQEBBQADggEPADCCAQoCggEBAMcJg8zOLdgasSmkLhOrlr6KMoOMpohBllVHrdRvEg/q6r8jR+EK 75xCGhR8ToREoqe7zM9/UnC6TS2y9UKTpT1v7RSMzR0t6ndl0TWBuUr/UXBhPk+Kmy7bI4yW4urC +y7P3/1/X7U8ocb8VpH/Clt+4iq7nirMcNh6qJR+xjOhV+VHzQMALuGYn5KZmc1NbJQYclsGkDxD z2UbFqE2+6vIZoL+jb9x4Pa5gNf1TwSDkOkikZB1xtB4ZqtXThaABSONdfmv/Z1pua3FYxnCFmdr /+N2JLKutIxMYqQOJebr/f/h5t95m4JgrM3Y/w7YX9d7YAL9jvN4SydHsU6n65cCAwEAAaOCAa0w ggGpMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBRTcu2SnODaywFc fH6WNU7y1LhRgjAfBgNVHSMEGDAWgBROC+8apEBbpRdphzDKNGhD0EGu8jBmBggrBgEFBQcBAQRa MFgwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLnN0YXJ0c3NsLmNvbS9jYTAtBggrBgEFBQcwAoYh aHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3J0MFsGA1UdHwRUMFIwJ6AloCOGIWh0dHA6 Ly93d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNybDAnoCWgI4YhaHR0cDovL2NybC5zdGFydHNzbC5j b20vc2ZzY2EuY3JsMIGABgNVHSAEeTB3MHUGCysGAQQBgbU3AQIBMGYwLgYIKwYBBQUHAgEWImh0 dHA6Ly93d3cuc3RhcnRzc2wuY29tL3BvbGljeS5wZGYwNAYIKwYBBQUHAgEWKGh0dHA6Ly93d3cu c3RhcnRzc2wuY29tL2ludGVybWVkaWF0ZS5wZGYwDQYJKoZIhvcNAQEFBQADggIBAAqDCH14qywG XLhjjF6uHLkjd02hcdh9hrw+VUsv+q1eeQWB21jWj3kJ96AUlPCoEGZ/ynJNScWy6QMVQjbbMXlt UfO4n4bGGdKo3awPWp61tjAFgraLJgDk+DsSvUD6EowjMTNx25GQgyYJ5RPIzKKR9tQW8gGK+2+R HxkUCTbYFnL6kl8Ch507rUdPPipJ9CgJFws3kDS3gOS5WFMxcjO5DwKfKSETEPrHh7p5shuuNktv sv6hxHTLhiMKX893gxdT3XLS9OKmCv87vkINQcNEcIIoFWbP9HORz9v3vQwR4e3ksLc2JZOAFK+s sS5XMEoznzpihEP0PLc4dCBYjbvSD7kxgDwZ+Aj8Q9PkbvE9sIPP7ON0fz095HdThKjiVJe6vofq +n6b1NBc8XdrQvBmunwxD5nvtTW4vtN6VY7mUCmxsCieuoBJ9OlqmsVWQvifIYf40dJPZkk9YgGT zWLpXDSfLSplbY2LL9C9U0ptvjcDjefLTvqSFc7tw1sEhF0n/qpA2r0GpvkLRDmcSwVyPvmjFBGq Up/pNy8ZuPGQmHwFi2/14+xeSUDG2bwnsYJQG2EdJCB6luQ57GEnTA/yKZSTKI8dDQa8Sd3zfXb1 9mOgSF0bBdXbuKhEpuP9wirslFe6fQ1t5j5R0xi72MZ8ikMu1RQZKCyDbMwazlHiMIIHHTCCBgWg AwIBAgIDBHnoMA0GCSqGSIb3DQEBBQUAMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRD b20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYG A1UEAxMvU3RhcnRDb20gQ2xhc3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwHhcN MTIwNzA1MDIyMTI5WhcNMTMwNzA2MTU1NDQ5WjBhMRkwFwYDVQQNExBtdW11Rmk0OXh3cGRrUWh3 MR4wHAYDVQQDDBVkb2Nvbm5vckBnc29mdC5jb20uYXUxJDAiBgkqhkiG9w0BCQEWFWRvY29ubm9y QGdzb2Z0LmNvbS5hdTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANLhGSNBVp7lteWC Kgmh5gw0Z7wnwJqESu3IYZ885kbCP4lMpZwQv9GOVDCqPuuVUzY+K3kwaf91g/kz2oRcU+bw1bpY iOvwMzwPD1V3hZ7r8NNToUdT/VKQRTVcAP9/Cr4NYekyVTehQZ7GGF7cyqXKvMpOHriW7KzNq7R0 p6eJ2uadOL5VPY+uTsVjmizQlFMl+My34vgkQ1Wo765MRjzHzwvpmWiqHrgjpvJL33bmgHB2ZtbG KcQ55Ze2T2ysAsa+SoDBwYC+8uBhuFtdSl0/Nm2q1eZBFNamCmGzpEjVDQPlAxhOwjde15WMsJPB mTibsOer4LN8cN0XfeBqKrMCAwEAAaOCA7AwggOsMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0G A1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUGrouPvtucJ95hJDEw/MyVdgH 2JowHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIAYDVR0RBBkwF4EVZG9jb25ub3JA Z3NvZnQuY29tLmF1MIICIQYDVR0gBIICGDCCAhQwggIQBgsrBgEEAYG1NwECAjCCAf8wLgYIKwYB BQUHAgEWImh0dHA6Ly93d3cuc3RhcnRzc2wuY29tL3BvbGljeS5wZGYwNAYIKwYBBQUHAgEWKGh0 dHA6Ly93d3cuc3RhcnRzc2wuY29tL2ludGVybWVkaWF0ZS5wZGYwgfcGCCsGAQUFBwICMIHqMCcW IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MAMCAQEagb5UaGlzIGNlcnRpZmljYXRl IHdhcyBpc3N1ZWQgYWNjb3JkaW5nIHRvIHRoZSBDbGFzcyAxIFZhbGlkYXRpb24gcmVxdWlyZW1l bnRzIG9mIHRoZSBTdGFydENvbSBDQSBwb2xpY3ksIHJlbGlhbmNlIG9ubHkgZm9yIHRoZSBpbnRl bmRlZCBwdXJwb3NlIGluIGNvbXBsaWFuY2Ugb2YgdGhlIHJlbHlpbmcgcGFydHkgb2JsaWdhdGlv bnMuMIGcBggrBgEFBQcCAjCBjzAnFiBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAD AgECGmRMaWFiaWxpdHkgYW5kIHdhcnJhbnRpZXMgYXJlIGxpbWl0ZWQhIFNlZSBzZWN0aW9uICJM ZWdhbCBhbmQgTGltaXRhdGlvbnMiIG9mIHRoZSBTdGFydENvbSBDQSBwb2xpY3kuMDYGA1UdHwQv MC0wK6ApoCeGJWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL2NydHUxLWNybC5jcmwwgY4GCCsGAQUF BwEBBIGBMH8wOQYIKwYBBQUHMAGGLWh0dHA6Ly9vY3NwLnN0YXJ0c3NsLmNvbS9zdWIvY2xhc3Mx L2NsaWVudC9jYTBCBggrBgEFBQcwAoY2aHR0cDovL2FpYS5zdGFydHNzbC5jb20vY2VydHMvc3Vi LmNsYXNzMS5jbGllbnQuY2EuY3J0MCMGA1UdEgQcMBqGGGh0dHA6Ly93d3cuc3RhcnRzc2wuY29t LzANBgkqhkiG9w0BAQUFAAOCAQEAgT50PcSRWSCNYUsBvFh33Iku7Us501FqkCqN2YcnwULjz1z1 9f9BXQd6EpyUpAGjpoclBh8NCwaezXIXbp6qMktLBvoRNiH+dd1HtefqPQ7b4rt5LrS3vrptaVWa Qiqmxg/sFdQ20uNGpXYlUDoIHRoD4DT7oa9n45eLjE3S8Jw2RZO2y0Ve3HZvwpQOkTbYsESBHZmh szYsOHd5kZH1S573LlMffbH1IztIBtx8ozlyT4io3T5e4GJBro1jSepC3oIaqpwnDOzQqdnoPNHd QUxlI0YlhAaYUC9bNNfHJtQo7jyYhQcTK/MGYeJ6NJa5A+6BQWrZhHl7v/OOlaxwyjGCA28wggNr AgEBMIGUMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0ECAwR56DAJBgUrDgMCGgUAoIIBrzAY BgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xMzA0MDMwMzQ5MjhaMCMG CSqGSIb3DQEJBDEWBBTRHS2ZdExZXywrN5BicHT5lQM1njCBpQYJKwYBBAGCNxAEMYGXMIGUMIGM MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERp Z2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMSBQcmlt YXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0ECAwR56DCBpwYLKoZIhvcNAQkQAgsxgZeggZQwgYwx CzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGln aXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1h cnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBHnoMA0GCSqGSIb3DQEBAQUABIIBAMFF6DKnrged EgX2nVYXADkhck5ikOSLHd0oe//9B7zvDP0TdjrAkrNHdiMboNbrChH36t7Ucv0yYOm8VGDovv2E 4/A7frKhqvYjihQiK1zUdJ/kdd1gyzj4TxDKt6tldJJw0f78CK/wEzBIWZfGVdfhIwVznDBc3yFZ 5Y9nEYrquprUDX67UcA5dnh6GBcoTbY7LMAzMR8z/78BwF6Ap1EherZRmPXpn4VKE4sOoWiT1mTF JsQMAX5a0oFK857Ipa0BvkkZxnmabgqzfyvk3gA/WfBQg3tjoE+W/srEFRpkH+AT8kPqRkf1LqsA DW2o25hm9bR5zqP2OeUWSyr6tUIAAAAAAAA= --Apple-Mail=_04E353E7-9BEC-46B0-8E8A-E9ADA3A56B16-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 09:42:22 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D7E0F294; Wed, 3 Apr 2013 09:42:22 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4E55DDA; Wed, 3 Apr 2013 09:42:21 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r339QKYd089859; Wed, 3 Apr 2013 13:26:20 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r339QKtq089858; Wed, 3 Apr 2013 13:26:20 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 3 Apr 2013 13:26:20 +0400 From: Gleb Smirnoff To: Pawel Jakub Dawidek Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403092620.GT76816@glebius.int.ru> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20130402232606.GC1810@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 09:42:22 -0000 Pawel, On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: P> > Together with Konstantin Belousov (kib@) we developed a new API that is P> > initially purposed for (but not limited to) collecting statistical P> > data in kernel. P> P> Is there any plan to implement universal way of exporting those P> statistics out of the kernel? P> P> Solaris has a framework for in-kernel statistics, which are exported via P> kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you P> can try 'sysctl kstat'. There are already primitives for sysctl in the branch: #include SYSCTL_COUNTER_U64(parent, nbr, name, access, ptr, val, descr); SYSCTL_ADD_COUNTER_U64(ctx, parent, nbr, name, access, ptr, descr); I do not think that every counter(9) in system should create an oid automatically. We are going to have counters per ipfw and per pf rule. P> It would be nice for counter_u64_alloc() to take additional argument P> 'name' and to create sysctl for the counter automatically. We could then P> slowly start migrating userland tools to use sysctls (or some wrapper P> userland API), but we immediately make those statistics available for P> use in scripts. Oh, in this bikeshed I am somewhere between your plan and the XML junta :) I'd better stay away from this discussion :) Let's keep topic on the new primitive. Just keep in mind that having a name would require another memory allocation, from normal not per-cpu malloc(9). P> > o Tiny API for counter(9): P> > P> > counter_u64_t P> > counter_u64_alloc(int wait); P> > P> > void P> > counter_u64_free(counter_u64_t cnt); P> > P> > void P> > counter_u64_add(counter_u64_t cnt, uint64_t inc); P> > P> > uint64_t P> > counter_u64_fetch(counter_u64_t cnt); P> P> Do you really expect other types in the future? If so, could we at least P> create generic counter_t that internally keeps the type? We will probably have signed 64-bit, actually allocated from the same zone. A type keeping type would require either additional allocation from malloc(9) or UMA directly, that would store the type, or we will store type in every per-cpu copy, wasting memory. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 10:02:11 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6C1047DE; Wed, 3 Apr 2013 10:02:11 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 07E131B4; Wed, 3 Apr 2013 10:02:10 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 93986C07; Wed, 3 Apr 2013 11:58:35 +0200 (CEST) Date: Wed, 3 Apr 2013 12:04:01 +0200 From: Pawel Jakub Dawidek To: Luigi Rizzo Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403100401.GA1349@garage.freebsd.pl> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZPt4rx8FFjLCG7dd" Content-Disposition: inline In-Reply-To: <20130403002846.GB15334@onelab2.iet.unipi.it> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 10:02:11 -0000 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Apr 03, 2013 at 02:28:46AM +0200, Luigi Rizzo wrote: > On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: > > On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > > > Hi! > > >=20 > > > Together with Konstantin Belousov (kib@) we developed a new API tha= t is > > > initially purposed for (but not limited to) collecting statistical > > > data in kernel. > >=20 > > Is there any plan to implement universal way of exporting those > > statistics out of the kernel? > >=20 > > Solaris has a framework for in-kernel statistics, which are exported via > > kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you > > can try 'sysctl kstat'. > >=20 > > It would be nice for counter_u64_alloc() to take additional argument > > 'name' and to create sysctl for the counter automatically. We could then > > slowly start migrating userland tools to use sysctls (or some wrapper > > userland API), but we immediately make those statistics available for > > use in scripts. >=20 > that is an interesting idea but i believe it can be effectively > built as a wrapper on top of the counter_u64_alloc() routine: >=20 > name_counter(counter_t c, const char *fmt, ...); > free_named_counter(counter_t c); >=20 > After all the name->counter mapping is unidirectional, > and possibly not even necessary on every single counter > (think of ipfw dynamic rules, created on packet arrivals, so > the counter alloc/dealloc needs to be fast). Right, although I'd optimize API naming and usage for the common case. Eventhough we do want to able to alloc/free counters quickly sometimes, most of the time we don't care about alloc/free speed and we would like to have a name. Having a name argument that could be NULL for short-living counter would allow to call only one allocation function in the common case (actually in every case). > It might be useful for the name_counter() routine to support > a printf-style argument to make it easy to build names. Indeed. > > > o Tiny API for counter(9): > > >=20 > > > counter_u64_t > > > counter_u64_alloc(int wait); > > >=20 > > > void > > > counter_u64_free(counter_u64_t cnt); > > >=20 > > > void > > > counter_u64_add(counter_u64_t cnt, uint64_t inc); > > >=20 > > > uint64_t > > > counter_u64_fetch(counter_u64_t cnt); > >=20 > > Do you really expect other types in the future? If so, could we at least > > create generic counter_t that internally keeps the type? >=20 > I read the u64 in the name mostly as a reminder to users > of the counter size.=20 Should the users care? As a user of this KPI I'd prefer to have simpler name and just assume the counter is big enough. > It might actually make sense is to change the type to s64. > This way we could have counters that go negative, > and also use them to accumulate sbintime_t values. Agreed, int64_t seems better. > But otherwise i am not sure that we want other types. >=20 > u32/s32 might save atomic/critical_enter ops on some archs, > but they saturate so quickly that probably are a bad idea. > And 63/64 bits are quite large already. Right, I don't think 32bit counters are needed at all and I can't find any use for 128bit counters either. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --ZPt4rx8FFjLCG7dd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFb/pEACgkQForvXbEpPzRuNQCeJF3FFX/ScKUvnlfQLICFTdkt zjUAoMXpy82nx9Ukq8RNB1g1JC4EYU0H =Qm/z -----END PGP SIGNATURE----- --ZPt4rx8FFjLCG7dd-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 17:37:05 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C12ABF8D; Wed, 3 Apr 2013 17:37:05 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 9A60823A; Wed, 3 Apr 2013 17:37:05 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 03C4F617AD; Wed, 3 Apr 2013 10:37:05 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 59001-04; Wed, 3 Apr 2013 10:37:04 -0700 (PDT) Received: from Alfreds-MacBook-Pro-9.local (unknown [10.8.0.26]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 3F503617AA; Wed, 3 Apr 2013 10:37:04 -0700 (PDT) Message-ID: <515C68B5.2010006@ixsystems.com> Date: Wed, 03 Apr 2013 10:36:53 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Pawel Jakub Dawidek Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> In-Reply-To: <20130403100401.GA1349@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 17:37:05 -0000 Hey folks, sorry for the top post here, but I just came into this thread. Here at iXsystems we've just developed a set of scripts to scrape the various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, etc) and put them into graphs based on time. The goal is to be able to line up all these metrics with whatever benchmark we are currently running and be able to see what may be causing issues. Potentially you should be able to scroll through the graphs and see things like "ran out of mbufs @time", "vm system began paging at @time", "buffer deaemon went nuts @time" Then we can take the information back and leverage it to make tuning decisions, or potentially change kernel algorithms. The only problem we have is that every user land tool has its own format, so along with my team we have written some shell to coerce the output from the various programs into pseudo-CSV (key/value pair) which can then be post processed by tools to convert to CSV which can then be put into something like open office, or put through an R program to graph it. I'm hoping to have something shortly. What I was hoping to do over the next few days was discuss with people how we can (or should we even) fix the user land statistics tools to output machine readable output that can be easily parsed. Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to parse. The idea of outputting xml is good, CSV is OK, however CSV is problematic as in the case of sysctl, if new nodes appear, then we can't begin to emit them, we must either ignore them, or abort, or log them to auxiliary files. Anything that makes life easier is good. I should be able to share our scripts within the next couple of days. -Alfred On 4/3/13 3:04 AM, Pawel Jakub Dawidek wrote: > On Wed, Apr 03, 2013 at 02:28:46AM +0200, Luigi Rizzo wrote: >> On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: >>> On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: >>>> Hi! >>>> >>>> Together with Konstantin Belousov (kib@) we developed a new API that is >>>> initially purposed for (but not limited to) collecting statistical >>>> data in kernel. >>> Is there any plan to implement universal way of exporting those >>> statistics out of the kernel? >>> >>> Solaris has a framework for in-kernel statistics, which are exported via >>> kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you >>> can try 'sysctl kstat'. >>> >>> It would be nice for counter_u64_alloc() to take additional argument >>> 'name' and to create sysctl for the counter automatically. We could then >>> slowly start migrating userland tools to use sysctls (or some wrapper >>> userland API), but we immediately make those statistics available for >>> use in scripts. >> that is an interesting idea but i believe it can be effectively >> built as a wrapper on top of the counter_u64_alloc() routine: >> >> name_counter(counter_t c, const char *fmt, ...); >> free_named_counter(counter_t c); >> >> After all the name->counter mapping is unidirectional, >> and possibly not even necessary on every single counter >> (think of ipfw dynamic rules, created on packet arrivals, so >> the counter alloc/dealloc needs to be fast). > Right, although I'd optimize API naming and usage for the common case. > Eventhough we do want to able to alloc/free counters quickly sometimes, > most of the time we don't care about alloc/free speed and we would like > to have a name. Having a name argument that could be NULL for > short-living counter would allow to call only one allocation function in > the common case (actually in every case). > >> It might be useful for the name_counter() routine to support >> a printf-style argument to make it easy to build names. > Indeed. > >>>> o Tiny API for counter(9): >>>> >>>> counter_u64_t >>>> counter_u64_alloc(int wait); >>>> >>>> void >>>> counter_u64_free(counter_u64_t cnt); >>>> >>>> void >>>> counter_u64_add(counter_u64_t cnt, uint64_t inc); >>>> >>>> uint64_t >>>> counter_u64_fetch(counter_u64_t cnt); >>> Do you really expect other types in the future? If so, could we at least >>> create generic counter_t that internally keeps the type? >> I read the u64 in the name mostly as a reminder to users >> of the counter size. > Should the users care? As a user of this KPI I'd prefer to have simpler > name and just assume the counter is big enough. > >> It might actually make sense is to change the type to s64. >> This way we could have counters that go negative, >> and also use them to accumulate sbintime_t values. > Agreed, int64_t seems better. > >> But otherwise i am not sure that we want other types. >> >> u32/s32 might save atomic/critical_enter ops on some archs, >> but they saturate so quickly that probably are a bad idea. >> And 63/64 bits are quite large already. > Right, I don't think 32bit counters are needed at all and I can't find > any use for 128bit counters either. > From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 17:45:14 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 04F9F1AA; Wed, 3 Apr 2013 17:45:14 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) by mx1.freebsd.org (Postfix) with ESMTP id 459CB27B; Wed, 3 Apr 2013 17:45:13 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id y10so4505789wgg.2 for ; Wed, 03 Apr 2013 10:45:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=VZiuGLxlTwXYfGaZSv20CU3FgUUzEqhUxF8JeK7df+0=; b=kTgVg99MAkoXXrK9J/kZz0CqAnO/KaF0pu0Eiqv4ORzBQYjPnq/qUGtWcbSeoj8YqC QutDKEOYY7CFRuDPFLpSQwEquDkQzWe2AT3nZ5Mnvr75JxzXEEBPNSofJOWGTAGptF8h Ibd0xM1NxYt6hKbWnPbui7F35tyxi5KmK/xFMOjR0BeZX+D3c+6ZOFK6aT7G2zFW4GGV t2IoZp0FeFXIkhWb9kAWYVaPlJX047qrdG2fR9VABvAkDltM/19ZxiQtw73H98RnYESt WaTw786M78EEUrjfqSMTdL8MD3fYBVwGtZqf300vb6Xay9cKPpHwdu9+T1NcfXZKrYBN HY8A== MIME-Version: 1.0 X-Received: by 10.194.119.33 with SMTP id kr1mr4314848wjb.36.1365011112389; Wed, 03 Apr 2013 10:45:12 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.243.7 with HTTP; Wed, 3 Apr 2013 10:45:12 -0700 (PDT) In-Reply-To: <515C68B5.2010006@ixsystems.com> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> Date: Wed, 3 Apr 2013 10:45:12 -0700 X-Google-Sender-Auth: 2XL9mwIQrjTTagskhnTXwEtqJFA Message-ID: Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters From: Adrian Chadd To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 17:45:14 -0000 On 3 April 2013 10:36, Alfred Perlstein wrote: > Hey folks, sorry for the top post here, but I just came into this thread. > > Here at iXsystems we've just developed a set of scripts to scrape the > various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, > etc) and put them into graphs based on time. Cool! > The goal is to be able to line up all these metrics with whatever benchmark > we are currently running and be able to see what may be causing issues. > > Potentially you should be able to scroll through the graphs and see things > like "ran out of mbufs @time", "vm system began paging at @time", "buffer > deaemon went nuts @time" > > Then we can take the information back and leverage it to make tuning > decisions, or potentially change kernel algorithms. > > The only problem we have is that every user land tool has its own format, so > along with my team we have written some shell to coerce the output from the > various programs into pseudo-CSV (key/value pair) which can then be post > processed by tools to convert to CSV which can then be put into something > like open office, or put through an R program to graph it. > > I'm hoping to have something shortly. > > What I was hoping to do over the next few days was discuss with people how > we can (or should we even) fix the user land statistics tools to output > machine readable output that can be easily parsed. > > Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to > parse. > > The idea of outputting xml is good, CSV is OK, however CSV is problematic as > in the case of sysctl, if new nodes appear, then we can't begin to emit > them, we must either ignore them, or abort, or log them to auxiliary files. > Anything that makes life easier is good. > > I should be able to share our scripts within the next couple of days. that's quite shiny. It'd be really nice if we could come up with a consistent statistics display, summary and export library. That way we could write tools that used a given data fetch/display API and then we could have optional "things" that implement various export methods. For example, I'm packing up Sam Leffler's "libstatfoo" for inclusion into -HEAD, primarily so the tools that use it (wlanstats, mwlstats, athstats and all the other ath stats programs I'm using) can use it. But once I've converted the stats tool over to that, I can do a few cute things: * the library has a generic way to list all of the supported statistics fields - you register the statistic names with the library, then you can create arbitrary format strings with the information; * the library handles "now" versus "time series" data display itself - you just need to populate the statistic structure with the relevant stats and then call the right function to display things; * I plan on extending it to spit out CSV output as a generic feature, so I can start doing things like importing the output of those tools direct into rrdtool/etc without any intermediary parsing scripts; * .. and if I then add new stats fields, the (requested) output of the script won't change, so tools won't break. adrian From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 18:00:56 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 79FEA9DF; Wed, 3 Apr 2013 18:00:56 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 476F6353; Wed, 3 Apr 2013 18:00:56 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id C252E61EB5; Wed, 3 Apr 2013 11:00:54 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 59753-07-4; Wed, 3 Apr 2013 11:00:54 -0700 (PDT) Received: from Alfreds-MacBook-Pro-9.local (unknown [10.8.0.26]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 5031E61E8F; Wed, 3 Apr 2013 11:00:38 -0700 (PDT) Message-ID: <515C6E3A.9020300@ixsystems.com> Date: Wed, 03 Apr 2013 11:00:26 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 18:00:56 -0000 On 4/3/13 10:45 AM, Adrian Chadd wrote: > On 3 April 2013 10:36, Alfred Perlstein wrote: >> Hey folks, sorry for the top post here, but I just came into this thread. >> >> Here at iXsystems we've just developed a set of scripts to scrape the >> various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, >> etc) and put them into graphs based on time. > Cool! > >> The goal is to be able to line up all these metrics with whatever benchmark >> we are currently running and be able to see what may be causing issues. >> >> Potentially you should be able to scroll through the graphs and see things >> like "ran out of mbufs @time", "vm system began paging at @time", "buffer >> deaemon went nuts @time" >> >> Then we can take the information back and leverage it to make tuning >> decisions, or potentially change kernel algorithms. >> >> The only problem we have is that every user land tool has its own format, so >> along with my team we have written some shell to coerce the output from the >> various programs into pseudo-CSV (key/value pair) which can then be post >> processed by tools to convert to CSV which can then be put into something >> like open office, or put through an R program to graph it. >> >> I'm hoping to have something shortly. >> >> What I was hoping to do over the next few days was discuss with people how >> we can (or should we even) fix the user land statistics tools to output >> machine readable output that can be easily parsed. >> >> Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to >> parse. >> >> The idea of outputting xml is good, CSV is OK, however CSV is problematic as >> in the case of sysctl, if new nodes appear, then we can't begin to emit >> them, we must either ignore them, or abort, or log them to auxiliary files. >> Anything that makes life easier is good. >> >> I should be able to share our scripts within the next couple of days. > that's quite shiny. > > It'd be really nice if we could come up with a consistent statistics > display, summary and export library. > > That way we could write tools that used a given data fetch/display API > and then we could have optional "things" that implement various export > methods. > > For example, I'm packing up Sam Leffler's "libstatfoo" for inclusion > into -HEAD, primarily so the tools that use it (wlanstats, mwlstats, > athstats and all the other ath stats programs I'm using) can use it. > > But once I've converted the stats tool over to that, I can do a few cute things: > > * the library has a generic way to list all of the supported > statistics fields - you register the statistic names with the library, > then you can create arbitrary format strings with the information; > * the library handles "now" versus "time series" data display itself - > you just need to populate the statistic structure with the relevant > stats and then call the right function to display things; > * I plan on extending it to spit out CSV output as a generic feature, > so I can start doing things like importing the output of those tools > direct into rrdtool/etc without any intermediary parsing scripts; > * .. and if I then add new stats fields, the (requested) output of the > script won't change, so tools won't break. > > Very cool. One thing to note, the CSV format: COLNAME1,COLNAME2,COLNAME3 DATA1,DATA2,DATA3 is vulnerable to problems where a new column will spring into being due to loading of a kernel module/driver/something. Imo it's better to look at XML or some other pseudo-CSV like: COLNAME1:DATA1,COLNAME2:DATA2,COLNAME3:DATA3 so that we are OK with columns springing into existence or leaving. -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 18:07:08 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 07713EFF; Wed, 3 Apr 2013 18:07:08 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 694823C8; Wed, 3 Apr 2013 18:07:07 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hm11so4626275wib.1 for ; Wed, 03 Apr 2013 11:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=2r5hOHKvnp41558IKcK6OxcPsgRGiLCp9174cpGBxRE=; b=cKMZ/nmB0QLHYU712SRU5Yt8xWvqAXN11PfzOuxZkJBfG8QJc/0d1wJcSNzFWw4rGo tMxTlUq1uxFbVbeiSOnfowND7pD3KI46Q0jszRN2JvlU7I7Wh5EH6CTis6h87t/Bb/h4 H3MHM5hxAe+EvZfFWySUl9+NTULE812xNWDhijI9MrqX+goMhN91ThBuL7SjvcFIf3Z5 iaymYrphLyGv4AHJ4DUBiPdBPd5YSQZjDs5+bOI4QdwdVpx3/sEPvCNaPySJsaWa/ukY bMbAsi+R6rORtxzM11NHj0Vl4GAeYUcP3wK14Rna2gThMSRUa/WqwSSM9gQBA99N40um kR/Q== MIME-Version: 1.0 X-Received: by 10.180.94.135 with SMTP id dc7mr24301449wib.11.1365012426095; Wed, 03 Apr 2013 11:07:06 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.243.7 with HTTP; Wed, 3 Apr 2013 11:07:05 -0700 (PDT) In-Reply-To: <515C6E3A.9020300@ixsystems.com> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <515C6E3A.9020300@ixsystems.com> Date: Wed, 3 Apr 2013 11:07:05 -0700 X-Google-Sender-Auth: Ix1KDTvYI8j1dAugERMWwD9iQiA Message-ID: Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters From: Adrian Chadd To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 18:07:08 -0000 On 3 April 2013 11:00, Alfred Perlstein wrote: > One thing to note, the CSV format: > COLNAME1,COLNAME2,COLNAME3 > DATA1,DATA2,DATA3 > > is vulnerable to problems where a new column will spring into being due to > loading of a kernel module/driver/something. > > Imo it's better to look at XML or some other pseudo-CSV like: > COLNAME1:DATA1,COLNAME2:DATA2,COLNAME3:DATA3 > so that we are OK with columns springing into existence or leaving. Only if its parsed badly. :-) CSV in the above format should be parsed fine, because your parser should _first_ establish the column->name mapping and use that moving forward. It just so happens that most people don't bother with that. But yes. My (vague) plan with libstatfoo or something else was to convert Sam's tools to use a base system library and then implement optional data output filters. (Optional because I may not want them all on my ridiculously slim embedded stuff.) But the general output (CSV, human-readable) should be there by default. Hell, if we're going down this path, it's almost worth suggesting that the tools should only output machine-parsable output and some _other_ tool should translate that into human-readable. But that may be a bit too radical.. Thanks, Adrian From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 18:15:54 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8C92C3B0; Wed, 3 Apr 2013 18:15:54 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 62112604; Wed, 3 Apr 2013 18:15:53 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 7CC05611B4; Wed, 3 Apr 2013 11:15:53 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 63211-02; Wed, 3 Apr 2013 11:15:53 -0700 (PDT) Received: from Alfreds-MacBook-Pro-9.local (unknown [10.8.0.26]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id DA058611AE; Wed, 3 Apr 2013 11:15:52 -0700 (PDT) Message-ID: <515C71CC.3090908@ixsystems.com> Date: Wed, 03 Apr 2013 11:15:40 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <515C6E3A.9020300@ixsystems.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 18:15:54 -0000 On 4/3/13 11:07 AM, Adrian Chadd wrote: > On 3 April 2013 11:00, Alfred Perlstein wrote: > >> One thing to note, the CSV format: >> COLNAME1,COLNAME2,COLNAME3 >> DATA1,DATA2,DATA3 >> >> is vulnerable to problems where a new column will spring into being due to >> loading of a kernel module/driver/something. >> >> Imo it's better to look at XML or some other pseudo-CSV like: >> COLNAME1:DATA1,COLNAME2:DATA2,COLNAME3:DATA3 >> so that we are OK with columns springing into existence or leaving. > Only if its parsed badly. :-) > > CSV in the above format should be parsed fine, because your parser > should _first_ establish the column->name mapping and use that moving > forward. It just so happens that most people don't bother with that. > > But yes. My (vague) plan with libstatfoo or something else was to > convert Sam's tools to use a base system library and then implement > optional data output filters. (Optional because I may not want them > all on my ridiculously slim embedded stuff.) But the general output > (CSV, human-readable) should be there by default. > > Hell, if we're going down this path, it's almost worth suggesting that > the tools should only output machine-parsable output and some _other_ > tool should translate that into human-readable. But that may be a bit > too radical.. > > > Thanks, > > > Adrian You're not hearing me. How do you xlate sysctl-a when a module being loaded/unloaded can cause mibs to appear/disappear. Your choice with the former format is: 1) case where new mib appears -> ignore it 2) case where mib disappears -> give it some absurd value (nul?) I'm more concerned with #1. -Alfred -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 19:05:35 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D01891AF; Wed, 3 Apr 2013 19:05:35 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id 3D43185D; Wed, 3 Apr 2013 19:05:35 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.5/8.14.5) with ESMTP id r33J5Xai098600; Wed, 3 Apr 2013 14:05:33 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.5/8.14.5/Submit) id r33J5XMV098599; Wed, 3 Apr 2013 14:05:33 -0500 (CDT) (envelope-from brooks) Date: Wed, 3 Apr 2013 14:05:33 -0500 From: Brooks Davis To: Gleb Smirnoff Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403190533.GA97453@lor.one-eyed-alien.net> References: <20130401115128.GZ76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cWoXeonUoKmBZSoM" Content-Disposition: inline In-Reply-To: <20130401115128.GZ76816@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 19:05:35 -0000 --cWoXeonUoKmBZSoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > o Tiny API for counter(9): >=20 > counter_u64_t > counter_u64_alloc(int wait); >=20 > void > counter_u64_free(counter_u64_t cnt); >=20 > void > counter_u64_add(counter_u64_t cnt, uint64_t inc); >=20 > uint64_t > counter_u64_fetch(counter_u64_t cnt); I wonder if there might be value in an interface to retrieve the per-cpu values individually. The use case I have in mind is interrupt counters on our BERI CPU. Similar to the Sibyte MIPS SoCs, our PIC always routes each interrupt to a specific hardware thread. I'd ideally like to be able to look at at hardware interrupts on a per-thread basis and avoid the cache trashing behavior we'd get if we allocated current interrupt counters to each one, but we probably want to be able to preserve something like the current behavior. -- Brooks --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFRXH19XY6L6fI4GtQRAgw4AKCFFv5Avxk+GygNi9wfjRUjjXHWfQCaAj89 jGFJUk7SrxddS/AwpUF2UvY= =gVnZ -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 21:45:39 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1FCBAC9F; Wed, 3 Apr 2013 21:45:39 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id A300616C; Wed, 3 Apr 2013 21:45:38 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r33Ljapv094400; Thu, 4 Apr 2013 01:45:36 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r33Lja5l094399; Thu, 4 Apr 2013 01:45:36 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 4 Apr 2013 01:45:36 +0400 From: Gleb Smirnoff To: Brooks Davis Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130403214536.GJ76816@glebius.int.ru> References: <20130401115128.GZ76816@FreeBSD.org> <20130403190533.GA97453@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20130403190533.GA97453@lor.one-eyed-alien.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 21:45:39 -0000 On Wed, Apr 03, 2013 at 02:05:33PM -0500, Brooks Davis wrote: B> > o Tiny API for counter(9): B> > B> > counter_u64_t B> > counter_u64_alloc(int wait); B> > B> > void B> > counter_u64_free(counter_u64_t cnt); B> > B> > void B> > counter_u64_add(counter_u64_t cnt, uint64_t inc); B> > B> > uint64_t B> > counter_u64_fetch(counter_u64_t cnt); B> B> I wonder if there might be value in an interface to retrieve the per-cpu B> values individually. The use case I have in mind is interrupt counters B> on our BERI CPU. Similar to the Sibyte MIPS SoCs, our PIC always routes B> each interrupt to a specific hardware thread. I'd ideally like to be able B> to look at at hardware interrupts on a per-thread basis and avoid the B> cache trashing behavior we'd get if we allocated current interrupt B> counters to each one, but we probably want to be able to preserve B> something like the current behavior. You can just utilize directly the per-CPU UMA zones, that will come together with counter(9). So, the code will look like: critical_enter(); foo = (foo *)zpcpu_get(base); /* get private to CPU data */ do smth with foo critical_exit(); -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 08:59:12 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 381BDDCC; Thu, 4 Apr 2013 08:59:12 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id EFB6BFA0; Thu, 4 Apr 2013 08:59:11 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 2795CC68E; Thu, 4 Apr 2013 08:59:10 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id D39ED92DF; Thu, 4 Apr 2013 10:59:09 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alfred Perlstein Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> Date: Thu, 04 Apr 2013 10:59:08 +0200 In-Reply-To: <515C68B5.2010006@ixsystems.com> (Alfred Perlstein's message of "Wed, 03 Apr 2013 10:36:53 -0700") Message-ID: <86r4iqoen7.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 08:59:12 -0000 Alfred Perlstein writes: > Here at iXsystems we've just developed a set of scripts to scrape the > various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, > etc, etc) and put them into graphs based on time. in other words, you've reinvented Munin and Graphite? > The only problem we have is that every user land tool has its own > format, so along with my team we have written some shell to coerce the > output from the various programs into pseudo-CSV (key/value pair) > which can then be post processed by tools to convert to CSV which can > then be put into something like open office, or put through an R > program to graph it. in other words, you've reinvented rrdtool? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 09:06:55 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4232DEEC; Thu, 4 Apr 2013 09:06:55 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 0C9C7FDE; Thu, 4 Apr 2013 09:06:54 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 73DACF3B; Thu, 4 Apr 2013 11:03:23 +0200 (CEST) Date: Thu, 4 Apr 2013 11:08:52 +0200 From: Pawel Jakub Dawidek To: Gary Palmer Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters Message-ID: <20130404090851.GA1335@garage.freebsd.pl> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002523.GA96431@in-addr.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OXfL5xGRrasGEqWY" Content-Disposition: inline In-Reply-To: <20130403002523.GA96431@in-addr.com> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 09:06:55 -0000 --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 02, 2013 at 08:25:23PM -0400, Gary Palmer wrote: > On Wed, Apr 03, 2013 at 01:26:07AM +0200, Pawel Jakub Dawidek wrote: > > On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote: > > > Hi! > > >=20 > > > Together with Konstantin Belousov (kib@) we developed a new API tha= t is > > > initially purposed for (but not limited to) collecting statistical > > > data in kernel. > >=20 > > Is there any plan to implement universal way of exporting those > > statistics out of the kernel? > >=20 > > Solaris has a framework for in-kernel statistics, which are exported via > > kstat tool. For ZFS I export them via sysctl. If you have ZFS loaded you > > can try 'sysctl kstat'. > >=20 > > It would be nice for counter_u64_alloc() to take additional argument > > 'name' and to create sysctl for the counter automatically. We could then > > slowly start migrating userland tools to use sysctls (or some wrapper > > userland API), but we immediately make those statistics available for > > use in scripts. >=20 > Sorry for potentially turning this into a bikeshed, but is sysctl the > best interface for this? It is great for scripts as the CLI is already > there, however it is not a bulk interface so grabbing all the ZFS > statistics takes quite a few trips through our system call handler - 438 > on my 9.1 box for "sysctl kstat" (found via ktrace and then > kdump | grep -c SCTL), which is not ideal when there are only 87 > stats there. (5 calls per OID returned and 3 initial calls to get set up) >=20 > I'm not sure I have a better alternative other than a geom style bulk > export via XML or some other format, however I wanted to raise it for > consideration. I wouldn't hold up the checkin of the code for this, > however if another way of gathering the data is needed/desired then it > would be good to get it in before people get too used to the sysctl=20 > way. =20 XML is no go for me, as it is not really easy to use in scripts. We would need to create a tool to parse it and then I'd much prefer to import my API for dealing with name/value pairs that could be used in so many more places. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --OXfL5xGRrasGEqWY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFdQyMACgkQForvXbEpPzQ7nQCffc5WCAP3N/0UcICJSbaCzbJr 9RsAnj7q9rmMP0G+useB/tzTvBQdExp8 =IExW -----END PGP SIGNATURE----- --OXfL5xGRrasGEqWY-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 16:04:10 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 523B316C; Thu, 4 Apr 2013 16:04:10 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 2D848D5D; Thu, 4 Apr 2013 16:04:09 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 63DC6614A6; Thu, 4 Apr 2013 09:04:09 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 64364-02; Thu, 4 Apr 2013 09:04:09 -0700 (PDT) Received: from Alfreds-MacBook-Pro-9.local (unknown [10.8.0.14]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 9392E614A0; Thu, 4 Apr 2013 09:04:08 -0700 (PDT) Message-ID: <515DA46D.2060808@ixsystems.com> Date: Thu, 04 Apr 2013 09:03:57 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <86r4iqoen7.fsf@ds4.des.no> In-Reply-To: <86r4iqoen7.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 16:04:10 -0000 Since we're taking the "blunt route" I'll be blunt as well: You have completely missed the point of the project I am working on. Here is the point which you have missed: I was attempting to graph as many stats as possible, in addition I was looking at some heuristics to find out when machines go bad. In doing so I found that the user land utilities are require quite a bit of parsing to figure this stuff out due to varied output formats. Both Munin and Graphite suffer from this as well. Go read through the plugins (which lack many of the metrics I've pulled in a generic fashion) and you'll see what I'm talking about. What we are inventing and proposing is adding output options for various tools in the base system to better plug into these monitoring tools. This means a CSV/XML-like output for tools like sysctl/netstat/etc/etc and I was probing the community for feedback on this matter. What I was not looking for was a pat on the back for the graphing system we worked on for a total of 5 days. I hope this helps. -Alfred On 4/4/13 1:59 AM, Dag-Erling Smørgrav wrote: > Alfred Perlstein writes: >> Here at iXsystems we've just developed a set of scripts to scrape the >> various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, >> etc, etc) and put them into graphs based on time. > in other words, you've reinvented Munin and Graphite? > >> The only problem we have is that every user land tool has its own >> format, so along with my team we have written some shell to coerce the >> output from the various programs into pseudo-CSV (key/value pair) >> which can then be post processed by tools to convert to CSV which can >> then be put into something like open office, or put through an R >> program to graph it. > in other words, you've reinvented rrdtool? > > DES From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 16:33:47 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 62079A91 for ; Thu, 4 Apr 2013 16:33:47 +0000 (UTC) (envelope-from jos@catnook.com) Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) by mx1.freebsd.org (Postfix) with ESMTP id 3E781F55 for ; Thu, 4 Apr 2013 16:33:47 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id kx1so1584236pab.28 for ; Thu, 04 Apr 2013 09:33:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=eaNo/3stEkshzsy3SYo8DPCWTHN6G1igv+CwV+jYwj0=; b=QDHJAT+WxJ8viHswIDP5ZHqwxJMAqLciUA/PaLxu7sWIbeYCpcmq5AsFMLJZM5uuW0 LECuzKNxpcvGMQH1PYPmJYsxTGcBa65cmho2A3dbhQhTP/Txtv0eTkEOF+I69nqjpvAu 8mZj6G5QR1ysZTUCK1sTYgZ9e6t9dfFTRHr5QhBOyV4/O8y+FEJoPBFiCz093Mv6Go3G ZdoNEHcUoVgDPEJ3LcTZO46hGrQcn7bDCtIRrMam5rW8m51ez3CaeNJIikJFpRpIpVLb GR5eYEONOhb59Kimb0pihy7xLDc5gLGaZ6qGYznR7SyYAhLWxG3IGnAnfHJnstFiJ4rR oFyA== MIME-Version: 1.0 X-Received: by 10.68.104.1 with SMTP id ga1mr9464413pbb.182.1365093221583; Thu, 04 Apr 2013 09:33:41 -0700 (PDT) Received: by 10.70.21.66 with HTTP; Thu, 4 Apr 2013 09:33:40 -0700 (PDT) In-Reply-To: <515DA46D.2060808@ixsystems.com> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <86r4iqoen7.fsf@ds4.des.no> <515DA46D.2060808@ixsystems.com> Date: Thu, 4 Apr 2013 09:33:40 -0700 Message-ID: Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters From: Jos Backus To: Alfred Perlstein X-Gm-Message-State: ALoCoQndubU6IEDRCvBFmPooq92aJssVsjFlJv5f68Qm6Kl1BgLqMYH0NJuydhwiNJPfH4aP19FW Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , Gleb Smirnoff , arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 16:33:47 -0000 Why not use YAML or JSON? Those formats sit nicely between CSV and XML, and we already have libyaml in base in -current. Jos -- Jos Backus jos at catnook.com From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 21:57:50 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6DA30816; Thu, 4 Apr 2013 21:57:50 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC2CF80; Thu, 4 Apr 2013 21:57:50 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 15AC661E95; Thu, 4 Apr 2013 14:57:50 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 88965-06; Thu, 4 Apr 2013 14:57:49 -0700 (PDT) Received: from kruse-124.4.ixsystems.com (kruse-124.4.ixsystems.com [10.2.4.124]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id DFD8A61E92; Thu, 4 Apr 2013 14:57:49 -0700 (PDT) Message-ID: <515DF75E.5050401@ixsystems.com> Date: Thu, 04 Apr 2013 14:57:50 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jos Backus Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <86r4iqoen7.fsf@ds4.des.no> <515DA46D.2060808@ixsystems.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Gleb Smirnoff , arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 21:57:50 -0000 On 4/4/13 9:33 AM, Jos Backus wrote: > Why not use YAML or JSON? Those formats sit nicely between CSV and > XML, and we already have libyaml in base in -current. > I'm very open to that. We plan on open sourcing what we have now shortly. I may have someone from the team look into adding YAML output to sysctl, mind you I haven't looked at the format yet, so we'll see how well it fits. Thank you Jos. -Alfred From owner-freebsd-arch@FreeBSD.ORG Thu Apr 4 22:30:39 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0F9E02AF; Thu, 4 Apr 2013 22:30:39 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id F0346139; Thu, 4 Apr 2013 22:30:38 +0000 (UTC) Received: from kruse-124.4.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) by elvis.mu.org (Postfix) with ESMTPSA id 7A1741A3D4C; Thu, 4 Apr 2013 15:30:30 -0700 (PDT) Message-ID: <515DFF05.10206@mu.org> Date: Thu, 04 Apr 2013 15:30:29 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Pawel Jakub Dawidek Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002523.GA96431@in-addr.com> <20130404090851.GA1335@garage.freebsd.pl> In-Reply-To: <20130404090851.GA1335@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 22:30:39 -0000 On 4/4/13 2:08 AM, Pawel Jakub Dawidek wrote: > XML is no go for me, as it is not really easy to use in scripts. > We would need to create a tool to parse it and then I'd much prefer to > import my API for dealing with name/value pairs that could be used in so > many more places. > Do you have a link to this format? I'm looking at YAML and it's interesting, although I'm not sure if a more abbreviated format wouldn't be better. What I'm currently using is like: date|vfs.freebuffers:5000|vfs.highbuffers: 343433|...| -Alfred From owner-freebsd-arch@FreeBSD.ORG Fri Apr 5 08:10:33 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 495CECDC; Fri, 5 Apr 2013 08:10:33 +0000 (UTC) (envelope-from erik@cederstrand.it) Received: from csmtp3.one.com (csmtp3.one.com [91.198.169.23]) by mx1.freebsd.org (Postfix) with ESMTP id 0E2C17C0; Fri, 5 Apr 2013 08:10:32 +0000 (UTC) Received: from dev.local (unknown [217.157.7.221]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by csmtp3.one.com (Postfix) with ESMTPSA id 1AC6E697F5; Fri, 5 Apr 2013 08:01:50 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters From: Erik Cederstrand In-Reply-To: <515DFF05.10206@mu.org> Date: Fri, 5 Apr 2013 09:44:08 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002523.GA96431@in-addr.com> <20130404090851.GA1335@garage.freebsd.pl> <515DFF05.10206@mu.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.1503) Cc: arch@FreeBSD.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2013 08:10:33 -0000 Den 05/04/2013 kl. 00.30 skrev Alfred Perlstein : > On 4/4/13 2:08 AM, Pawel Jakub Dawidek wrote: >> XML is no go for me, as it is not really easy to use in scripts. >> We would need to create a tool to parse it and then I'd much prefer = to >> import my API for dealing with name/value pairs that could be used in = so >> many more places. >>=20 > Do you have a link to this format? >=20 > I'm looking at YAML and it's interesting, although I'm not sure if a = more abbreviated format wouldn't be better. >=20 > What I'm currently using is like: >=20 > date|vfs.freebuffers:5000|vfs.highbuffers: 343433|...| If you want to implement of machine-parseable output of sysctl that = supports key-value output, then please use a generally-accepted format. = This would make it so much easier for third-party software to use the = output. YAML or JSON are obvious choices. Powershell from Microsoft solves this generally by having all tools pipe = output as objects with a well-defined contract to the next command. = Whatever is printed to stdout is just human-friendly output. It's nice = because you can skip a lot of grep/sed/awk/cut trickery and just filter = and pick fields in a more object-oriented way. But it's also a PITA = because what's printed to the terminal is not what's piped to the next = command, so you need to look up the object spec every time you add a = pipe. I think it would be going overboard to add a --yaml or --json flag to = every single userland tool in FreeBSD, but designing key-value output = for sysctl so it can be applied to other tools would be really nice. Thanks, Erik From owner-freebsd-arch@FreeBSD.ORG Fri Apr 5 08:42:28 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BAC4B4B4 for ; Fri, 5 Apr 2013 08:42:28 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EA8698E0 for ; Fri, 5 Apr 2013 08:42:27 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA19258; Fri, 05 Apr 2013 11:42:23 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UO2E3-000PdN-Jh; Fri, 05 Apr 2013 11:42:23 +0300 Message-ID: <515E8E6E.4030706@FreeBSD.org> Date: Fri, 05 Apr 2013 11:42:22 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130321 Thunderbird/17.0.4 MIME-Version: 1.0 To: Alfred Perlstein Subject: collecting statistics / metrics References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> In-Reply-To: <515C68B5.2010006@ixsystems.com> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2013 08:42:28 -0000 on 03/04/2013 20:36 Alfred Perlstein said the following: > Hey folks, sorry for the top post here, but I just came into this thread. > > Here at iXsystems we've just developed a set of scripts to scrape the various > FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, etc) and put > them into graphs based on time. > > The goal is to be able to line up all these metrics with whatever benchmark we > are currently running and be able to see what may be causing issues. > > Potentially you should be able to scroll through the graphs and see things like > "ran out of mbufs @time", "vm system began paging at @time", "buffer deaemon > went nuts @time" > > Then we can take the information back and leverage it to make tuning decisions, > or potentially change kernel algorithms. This is very very useful! > The only problem we have is that every user land tool has its own format, so > along with my team we have written some shell to coerce the output from the > various programs into pseudo-CSV (key/value pair) which can then be post > processed by tools to convert to CSV which can then be put into something like > open office, or put through an R program to graph it. > > I'm hoping to have something shortly. > > What I was hoping to do over the next few days was discuss with people how we > can (or should we even) fix the user land statistics tools to output machine > readable output that can be easily parsed. > > Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to parse. > > The idea of outputting xml is good, CSV is OK, however CSV is problematic as in > the case of sysctl, if new nodes appear, then we can't begin to emit them, we > must either ignore them, or abort, or log them to auxiliary files. Anything > that makes life easier is good. > > I should be able to share our scripts within the next couple of days. Just an alternative idea... I think gathering all this information via plugins to e.g. collectd could be more flexible and less processing / parsing intensive. That would allow to avoid unnecessary formatting and re-parsing and to store the data in a convenient format. Ideally it would be great to have an umbrella library on top of sysctl, devstat, etc that would expose various stats in a convenient form. Another thing of convenience would be an ability to know which sysctls are actually stats. I think that you have already done work towards this goal. There are certain heuristics that may help to distinguish stats from knobs, constants, etc, but the explicit "this is a metric" should be used. Of course, it would take a lot of work to properly mark all the sysctls. Just thinking out loud. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 5 15:00:03 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 57860EA7; Fri, 5 Apr 2013 15:00:03 +0000 (UTC) (envelope-from alfred@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 2F3D2B5D; Fri, 5 Apr 2013 15:00:02 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 8CC1C625F9; Fri, 5 Apr 2013 08:00:02 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 64721-06-5; Fri, 5 Apr 2013 08:00:02 -0700 (PDT) Received: from [10.226.170.14] (45.sub-174-234-67.myvzw.com [174.234.67.45]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 2185162597; Fri, 5 Apr 2013 07:59:54 -0700 (PDT) References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> <515E8E6E.4030706@FreeBSD.org> Mime-Version: 1.0 (1.0) In-Reply-To: <515E8E6E.4030706@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <471D2765-393A-473F-A17C-FE1B77D15A6B@ixsystems.com> X-Mailer: iPhone Mail (10B329) From: Alfred Perlstein Subject: Re: collecting statistics / metrics Date: Fri, 5 Apr 2013 07:59:50 -0700 To: Andriy Gapon Cc: "arch@FreeBSD.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2013 15:00:03 -0000 On Apr 5, 2013, at 1:42 AM, Andriy Gapon wrote: > on 03/04/2013 20:36 Alfred Perlstein said the following: >> Hey folks, sorry for the top post here, but I just came into this thread.= >>=20 >> Here at iXsystems we've just developed a set of scripts to scrape the var= ious >> FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, etc) a= nd put >> them into graphs based on time. >>=20 >> The goal is to be able to line up all these metrics with whatever benchma= rk we >> are currently running and be able to see what may be causing issues. >>=20 >> Potentially you should be able to scroll through the graphs and see thing= s like >> "ran out of mbufs @time", "vm system began paging at @time", "buffer deae= mon >> went nuts @time" >>=20 >> Then we can take the information back and leverage it to make tuning deci= sions, >> or potentially change kernel algorithms. >=20 > This is very very useful! >=20 >> The only problem we have is that every user land tool has its own format,= so >> along with my team we have written some shell to coerce the output from t= he >> various programs into pseudo-CSV (key/value pair) which can then be post >> processed by tools to convert to CSV which can then be put into something= like >> open office, or put through an R program to graph it. >>=20 >> I'm hoping to have something shortly. >>=20 >> What I was hoping to do over the next few days was discuss with people ho= w we >> can (or should we even) fix the user land statistics tools to output mach= ine >> readable output that can be easily parsed. >>=20 >> Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy t= o parse. >>=20 >> The idea of outputting xml is good, CSV is OK, however CSV is problematic= as in >> the case of sysctl, if new nodes appear, then we can't begin to emit them= , we >> must either ignore them, or abort, or log them to auxiliary files. Anyth= ing >> that makes life easier is good. >>=20 >> I should be able to share our scripts within the next couple of days. >=20 > Just an alternative idea... > I think gathering all this information via plugins to e.g. collectd could b= e > more flexible and less processing / parsing intensive. That would allow t= o > avoid unnecessary formatting and re-parsing and to store the data in a > convenient format. Ideally it would be great to have an umbrella library o= n top > of sysctl, devstat, etc that would expose various stats in a convenient fo= rm. > Another thing of convenience would be an ability to know which sysctls are= > actually stats. I think that you have already done work towards this goal= . > There are certain heuristics that may help to distinguish stats from knobs= , > constants, etc, but the explicit "this is a metric" should be used. Of co= urse, > it would take a lot of work to properly mark all the sysctls. >=20 > Just thinking out loud. I'm going to bring these suggestions to my team and I think we can incorpora= te some of these ideas for sure.=20 -Alfred=20=