From owner-freebsd-arch@freebsd.org Mon Nov 16 01:18:11 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCBF0A2F339; Mon, 16 Nov 2015 01:18:11 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from c.mail.sonic.net (c.mail.sonic.net [64.142.111.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A0B371351; Mon, 16 Nov 2015 01:18:11 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from zeppelin.tachypleus.net (75-101-50-44.static.sonic.net [75.101.50.44]) (authenticated bits=0) by c.mail.sonic.net (8.15.1/8.15.1) with ESMTPSA id tAG1HeFC025825 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sun, 15 Nov 2015 17:17:41 -0800 Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 To: Justin Hibbits , Marius Strobl References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> Cc: sbruno@freebsd.org, freebsd-arch , sparc64@freebsd.org From: Nathan Whitehorn Message-ID: <56492EB4.1080307@freebsd.org> Date: Sun, 15 Nov 2015 17:17:40 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Sonic-CAuth: UmFuZG9tSVaXlyTcvSKUeK34kIATMPzVvr98eH+yLwlejlQgMjhOLvxzkTIy0JViFEX1hqRNy496m8IzL441MCSdjXF/ybgeYPyRrGanO50= X-Sonic-ID: C;2Imc1P+L5RGCOL0U9jFv0A== M;fEz51P+L5RGCOL0U9jFv0A== X-Spam-Flag: No X-Sonic-Spam-Details: 0.0/5.0 by cerberusd X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 01:18:11 -0000 On 11/08/15 12:46, Justin Hibbits wrote: > On 11/8/15, Marius Strobl wrote: > >> As for getting forward, the FreeBSD Software License Policy >> (https://www.freebsd.org/internal/software-license.html) >> specifically allows for existing GPLv2 licensed software in >> the FreeBSD source tree to be updated to GPLv3 licensed one. >> The initial, longer draft of this policy posted by brooks@ to >> developers@ even explicitly mentioned key technologies such >> as toolchains of other licenses being allowed when no mature >> BSD-licensed alternative exists. So I propose just that: >> Let's upgrade binutils and GCC in base to recent versions. >> Seriously. That way we 1) don't need to get external toolchain >> support into shape, 2) don't need to solve the chicken-and-egg >> problem of getting a toolchain onto a machine installed from >> a distribution built with an external toolchain and 3) once >> clang becomes mature on additional architectures, we have an >> upgrade path. Don't get me wrong, I'm only proposing that >> for !arm and !x86. >> As a side note: A while back I talked to grehan@ and marcel@ >> regarding the immaturities of clang and - as expected -, a >> GPL'ed toolchain just is no problem for either NetApp or >> Juniper as the binaries they ship don't include the toolchain >> itself. With the possible exception of the current incarnation >> of SCO which apparently sells a FreeBSD-based OS likely having >> a system compiler, for the same reason I can't think of why a >> GPLv3 licensed toolchain would matter for any of the commercial >> downstream consumers of FreeBSD. Thus, I really can't understand >> all that aggression regarding making FreeBSD 11 clang-only. >> > > > I 100% agree with you on this. If we can update binutils to the > latest and greatest, I believe powerpc64 would be able to work with > clang. I've backported several patches, with IBM's permission, to > binutils for handling new relocations, etc. However, not all patches > are straight forward, and currently we're missing something, which is > causing odd segfaults in ld(1), when linking as(1). No other binary, > only as(1). I've tried looking through it, but the binutils code is a > mess. I'm sure the bug that's getting hit was fixed with newer > binutils, but have had a very hard time trying to test with it. > > TL;DR, let's update binutils at the very least, and gcc if it makes > sense. We shouldn't be relying on external toolchain for some archs, > and internal for others. It completely snubs already second class > citizens. Just look at the various build failures we've had because > to some people All The World is clang/x86. > This would be super valuable. It would restore the upgrade path to clang broken at 3.5, allow us to use modern ABIs, everything. Is this something that is actually politically/legal doable? -Nathan From owner-freebsd-arch@freebsd.org Mon Nov 16 17:35:04 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 948D9A303C5 for ; Mon, 16 Nov 2015 17:35:04 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 70E93175E for ; Mon, 16 Nov 2015 17:35:04 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 6CE63A303C4; Mon, 16 Nov 2015 17:35:04 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C7D3A303C3 for ; Mon, 16 Nov 2015 17:35:04 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yk0-x231.google.com (mail-yk0-x231.google.com [IPv6:2607:f8b0:4002:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28660175A for ; Mon, 16 Nov 2015 17:35:04 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by ykdr82 with SMTP id r82so251044346ykd.3 for ; Mon, 16 Nov 2015 09:35:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi_nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=w+SQfdug50iXZwXDn5BlfBJyw4MaUv9ukVYfol3CLKA=; b=BETBEBMvh6u5MukF6nn54Xmk4TWVlfYd5AUTLAvzKSVa4ZjGa+0Vn0kBRlKIn3shQd 7mffu83ul33O8bQaOwT0BoUmkOtVKzHPOUhxmK6pG9XyZxjRVXWay2dKlpC5Mz3xV83Q c0Q8D1aC+Q41uoSrQOtNHH2RIUr+RT2qCC4pfruNgmLGi3sRS3LbM1N4dqVV3GzpCq/j g122ROVIUKqQq4SdSJpNPf/mgKrS3Zy7KH0SbzA9/YNTDZVkQiLz+kl3qET4h3QRi7Er UI986CppEw8lJGjABW5QVMLnbVyrklJ322aEWIrnV+vndqWu8dUoMDo7uZXtMfJWGtUi cgyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=w+SQfdug50iXZwXDn5BlfBJyw4MaUv9ukVYfol3CLKA=; b=bIzocc8Aw7TQDiDXpWJ9UqNfJiS/i77dp7FWmaxMcvxWJ9vaxwbRvkRlsfgQBS2MnK vP7K+NZYS/hEyrG87Zpu6EpAMgnaoy4QcrgSKk0hhX2KUYa2xEXS6cbaZtbLfiaJa82A 4TNhwI8D4afK4A1iKMeNujwqfaY+IhCioFxZ983jO2EcoUIgIoRzOuQ/8rhaXIxnIt7R xt+f60FSOgzVaJVqSJRiCf7d3q7V2IIIWbTfFTQd6LnJTVS0Cm15iM8R/njmmYHDS3xL nhz1aGHa0kDMPoVDC+soaNtOHy+QG9gsAhu1bWVRtNkE3Y6aTlQJ3/iyFkvFaVjKjlj1 pm2g== X-Gm-Message-State: ALoCoQkgAvhK1WwjpvuL4csC90auJkaAEwPkJpdWo/4H/chdgxWM8R04HeCuhe4RT467xFTmbmoR MIME-Version: 1.0 X-Received: by 10.129.96.84 with SMTP id u81mr9581163ywb.80.1447695303136; Mon, 16 Nov 2015 09:35:03 -0800 (PST) Received: by 10.129.105.86 with HTTP; Mon, 16 Nov 2015 09:35:03 -0800 (PST) In-Reply-To: <20151110222636.GN10134@ivaldir.etoilebsd.net> References: <20151110222636.GN10134@ivaldir.etoilebsd.net> Date: Mon, 16 Nov 2015 18:35:03 +0100 Message-ID: Subject: Re: Question about ASCII and nl_langinfo (locale work) From: Ed Schouten To: Baptiste Daroussin Cc: arch@freebsd.org, marino@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 17:35:04 -0000 Hi Baptiste, I personally think it's a shame if we were to deviate from returning "US-ASCII", for the reason that "US-ASCII" also happens to be the preferred MIME name for the character set: http://www.iana.org/assignments/character-sets/character-sets.xhtml "ASCII" doesn't even seem to be an alias for this character set. Though "ANSI_X3.4-1968" is an alias for ASCII, I wouldn't even know that this is ASCII without doing a Google search. In my opinion a decent implementation of newlocale() should support any of the character set names and aliases provided on the IANA page, but let nl_langinfo(CODESET) return the preferred MIME name. > That means we need to teach all upstream about US-ASCII all the time. Could you come up with a concrete list of pieces of software that need to be changed? Is it just those three pieces of software that you mentioned above? If so, then I think it would be a shame to make the concession. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-arch@freebsd.org Mon Nov 16 19:00:39 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A022EA30983 for ; Mon, 16 Nov 2015 19:00:39 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7C0391F9C for ; Mon, 16 Nov 2015 19:00:39 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: by mailman.ysv.freebsd.org (Postfix) id 7B838A3097E; Mon, 16 Nov 2015 19:00:39 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62CEAA3097D for ; Mon, 16 Nov 2015 19:00:39 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B5831F97 for ; Mon, 16 Nov 2015 19:00:38 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: by lfs39 with SMTP id 39so92786608lfs.3 for ; Mon, 16 Nov 2015 11:00:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=upYORXjRpJCDnlaagLPb2e99FHm54EOg0QM4e+0p8tA=; b=eZMuPVcXwXSQMyHIeq9GglfCMxKFB1kjMapIOaIGvQLr9oiYhLLSPAICqBC1aSSNcd BJnvOp39BtuNz7Pu1+9VSByyWVqkIJy4mTaQszGLjH/Nl+GwRZNJKck7Ki8Q4/1lTNuh LbPVQCi0DpOjkunoYYdo5fTx0yZY2PuPp+ESxUCpTgrSE5szHmiAsnVRNRgwQWaLLTVW fTQRP0X21eYwTZZTQyRtjQA1S6hDhDv22HkczIlyiVSaLvgW9jikb1gdHliO2b1uQZvp Y7RyuIWcwSfxmsDSOPgK4DGnPpYUFnMzxSCU+V9pT+Qx4ZnHfNhRzVarPFV2pRPnwaHF 4l+Q== X-Gm-Message-State: ALoCoQntjGLh3yHWG54se6WNcNzHvXuP+0J5Ba0lMS3JpWlO03jRjtFY/eMrTt0caTsPBuuNVPO7 X-Received: by 10.25.33.4 with SMTP id h4mr4371718lfh.3.1447700431248; Mon, 16 Nov 2015 11:00:31 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by smtp.gmail.com with ESMTPSA id j189sm821893lfg.46.2015.11.16.11.00.30 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Nov 2015 11:00:30 -0800 (PST) Subject: Re: Question about ASCII and nl_langinfo (locale work) To: Ed Schouten , Baptiste Daroussin References: <20151110222636.GN10134@ivaldir.etoilebsd.net> Cc: arch@freebsd.org, marino@freebsd.org From: Andrey Chernov X-Enigmail-Draft-Status: N1110 Message-ID: <564A27CD.7090908@freebsd.org> Date: Mon, 16 Nov 2015 22:00:29 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 19:00:39 -0000 On 16.11.2015 20:35, Ed Schouten wrote: > I personally think it's a shame if we were to deviate from returning > "US-ASCII", for the reason that "US-ASCII" also happens to be the > preferred MIME name for the character set: > > http://www.iana.org/assignments/character-sets/character-sets.xhtml > > "ASCII" doesn't even seem to be an alias for this character set. Yes, I overlook it somehow. ASCII is not in the IANA, while both ANSI_X3.4-1968 and US-ASCII are. So, I reconsider the proposal. We can return ANSI_X3.4-1968 for POSIX/C (for Linux compatibility reasons) and left pure US-ASCII as it was (since it is used rarely). > In my opinion a decent implementation of newlocale() should support > any of the character set names and aliases provided on the IANA page, > but let nl_langinfo(CODESET) return the preferred MIME name. BTW, we already have and return non-IANA codesets historically (inspired by X11). I.e. we have ISO8859-* instead of preferred names ISO-8859-*, moreover, ISO8859-* even not the aliases (!) and IANA knows nothing about them. Linux have IANA preferred names here, i.e. ISO-8859-*. So the question is: should we rename ISO8859-* to ISO-8859-* to be IANA and Linux compatible? We can strip first (or all) "_" and "-" from the environment names (as Linux does), to not violate POLA. >> That means we need to teach all upstream about US-ASCII all the time. > > Could you come up with a concrete list of pieces of software that need > to be changed? Is it just those three pieces of software that you > mentioned above? If so, then I think it would be a shame to make the > concession. No, I see such checks many times in other programs too, tcl is just one which can be found quickly. The proper procedure to examine situation will be to unpack _all_ ports and search through the code, but my machine can't handle it. -- http://ache.vniz.net/ From owner-freebsd-arch@freebsd.org Mon Nov 16 19:14:45 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE06CA30DA5; Mon, 16 Nov 2015 19:14:45 +0000 (UTC) (envelope-from superbisquit@gmail.com) Received: from mail-yk0-x242.google.com (mail-yk0-x242.google.com [IPv6:2607:f8b0:4002:c07::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 692A11973; Mon, 16 Nov 2015 19:14:45 +0000 (UTC) (envelope-from superbisquit@gmail.com) Received: by ykba77 with SMTP id a77so25406440ykb.2; Mon, 16 Nov 2015 11:14:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1UwCbrVJfGphtXYocT1kovcrc89WcjfgXHWyTK3+64s=; b=Il5HZXXLgm35vQ67uzLOqnxTaBO2WQxxh6kVKfC9M99RyUkvT8HhBocnCJb8KZzp9b LTJNtKldE7YXbR2pCx5/QV3xrqLVZTUDw47CoBsZzNN1j2guUFMp2tdz2fe306BA9slZ GKTUQVbZHZj4Tn2gwxw9w9d7uNnDe5pc8S9Q3lBp3pDWjwnYAS42UQV8L4DB5SVGOHSG sj8+nMzUqYSpLjgKNQo0vXT/SrQlA8LRckSc77eNTJfikYxSqMJL8jdtyMu9tHoVz2bE d2sFNZMTLNV1nMcqVoqD7pLXSj7inTEzsYzxGBHB5IFOAQhf+gQvU2XH5uhX+fWw6nuX V7CQ== MIME-Version: 1.0 X-Received: by 10.129.135.7 with SMTP id x7mr36501575ywf.95.1447701284682; Mon, 16 Nov 2015 11:14:44 -0800 (PST) Received: by 10.103.9.195 with HTTP; Mon, 16 Nov 2015 11:14:44 -0800 (PST) In-Reply-To: <56492EB4.1080307@freebsd.org> References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <56492EB4.1080307@freebsd.org> Date: Mon, 16 Nov 2015 14:14:44 -0500 Message-ID: Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 From: Joe Nosay To: Nathan Whitehorn Cc: Justin Hibbits , Marius Strobl , Sean Bruno , sparc64@freebsd.org, freebsd-arch Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 19:14:45 -0000 Will there be a project using Z-RAM and other technologies as such? SPARC64 processors should be capable of holding a database for fourth, fifth, and sixth dimensional graphs based on time. (Fourth is the single point in time, fifth is the path of least resistance, sixth would be variations known as experiential. NASA already released a virtual program "4S" for this. Performance of RISC over CISC has always been for the former. SPARC64 is practical in audio applications which call for access to such a database. Each architecture has a specific purpose. Think about it. ARM32/64 non-mmu temporary memory, sampling, data input. POWER32/64- cache for instructions and not trash, AI, repetitive instructions, graphics, audio. SPARC32/64 - scalability, complex databases AMD64/i386 complex compression, algorithms. Performance on each differs with kernel hertz, OS choice, etc. You use them together such as parts of a biological brain. Seeing that memory is now becoming three dimensional in form, we probably should be looking towards working these together. NFS and 9p - if still around - would create two types of accessible bootstrapping methods - applications/binaries and data. On Sun, Nov 15, 2015 at 8:17 PM, Nathan Whitehorn wrote: > > > On 11/08/15 12:46, Justin Hibbits wrote: > >> On 11/8/15, Marius Strobl wrote: >> >> >>> As for getting forward, the FreeBSD Software License Policy >>> (https://www.freebsd.org/internal/software-license.html) >>> specifically allows for existing GPLv2 licensed software in >>> the FreeBSD source tree to be updated to GPLv3 licensed one. >>> The initial, longer draft of this policy posted by brooks@ to >>> developers@ even explicitly mentioned key technologies such >>> as toolchains of other licenses being allowed when no mature >>> BSD-licensed alternative exists. So I propose just that: >>> Let's upgrade binutils and GCC in base to recent versions. >>> Seriously. That way we 1) don't need to get external toolchain >>> support into shape, 2) don't need to solve the chicken-and-egg >>> problem of getting a toolchain onto a machine installed from >>> a distribution built with an external toolchain and 3) once >>> clang becomes mature on additional architectures, we have an >>> upgrade path. Don't get me wrong, I'm only proposing that >>> for !arm and !x86. >>> As a side note: A while back I talked to grehan@ and marcel@ >>> regarding the immaturities of clang and - as expected -, a >>> GPL'ed toolchain just is no problem for either NetApp or >>> Juniper as the binaries they ship don't include the toolchain >>> itself. With the possible exception of the current incarnation >>> of SCO which apparently sells a FreeBSD-based OS likely having >>> a system compiler, for the same reason I can't think of why a >>> GPLv3 licensed toolchain would matter for any of the commercial >>> downstream consumers of FreeBSD. Thus, I really can't understand >>> all that aggression regarding making FreeBSD 11 clang-only. >>> >>> >> >> I 100% agree with you on this. If we can update binutils to the >> latest and greatest, I believe powerpc64 would be able to work with >> clang. I've backported several patches, with IBM's permission, to >> binutils for handling new relocations, etc. However, not all patches >> are straight forward, and currently we're missing something, which is >> causing odd segfaults in ld(1), when linking as(1). No other binary, >> only as(1). I've tried looking through it, but the binutils code is a >> mess. I'm sure the bug that's getting hit was fixed with newer >> binutils, but have had a very hard time trying to test with it. >> >> TL;DR, let's update binutils at the very least, and gcc if it makes >> sense. We shouldn't be relying on external toolchain for some archs, >> and internal for others. It completely snubs already second class >> citizens. Just look at the various build failures we've had because >> to some people All The World is clang/x86. >> >> > This would be super valuable. It would restore the upgrade path to clang > broken at 3.5, allow us to use modern ABIs, everything. Is this something > that is actually politically/legal doable? > -Nathan > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org" > From owner-freebsd-arch@freebsd.org Mon Nov 16 21:07:05 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A273CA30CA6 for ; Mon, 16 Nov 2015 21:07:05 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7C9AF11A3 for ; Mon, 16 Nov 2015 21:07:05 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7CAA9A30CA4; Mon, 16 Nov 2015 21:07:05 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62820A30CA3 for ; Mon, 16 Nov 2015 21:07:05 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC35511A2; Mon, 16 Nov 2015 21:07:04 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by wmdw130 with SMTP id w130so129003665wmd.0; Mon, 16 Nov 2015 13:07:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=x+UTk5KdMQHNCoRSe/qh09X8JWdcXsDbE07QMxrdbZk=; b=sHR/Lie4EmUBnXn6eCw6Pn5Hm79Sr7waK6LkCSfnUIdzhFaDVLMNSlE4WQuck4jn3v aGo3Pz/FghayjIyyszMyCVsl0EqPqJFfDhNn3fdSJSB/kTGotNljjwADqdKVV+V/5AzC 13jZGQyaNH4fEJ3gnv7Qn/mRCnfNwYfUqUEMMDkrckWP6fIOCSXq62ukHxEPHv05gcr7 h0R5MS5IyMBvJoABzk1IwSx6moAsRUewkQPg1daocvHJXaIIRnEloalbNAOt/Bdq0Er6 sT5Ul1fenuUn+3tD0U4Jf8M9R4OYo+IRjxJIBzn3ugwvC01PdS5AmE6fv8f7D/Az6eqi Sxsw== X-Received: by 10.28.183.198 with SMTP id h189mr7548802wmf.44.1447708023220; Mon, 16 Nov 2015 13:07:03 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id c13sm20380191wmd.14.2015.11.16.13.07.01 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Nov 2015 13:07:01 -0800 (PST) Sender: Baptiste Daroussin Date: Mon, 16 Nov 2015 22:06:59 +0100 From: Baptiste Daroussin To: Andrey Chernov Cc: Ed Schouten , arch@freebsd.org, marino@freebsd.org Subject: Re: Question about ASCII and nl_langinfo (locale work) Message-ID: <20151116210659.GB59189@ivaldir.etoilebsd.net> References: <20151110222636.GN10134@ivaldir.etoilebsd.net> <564A27CD.7090908@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1LKvkjL3sHcu1TtY" Content-Disposition: inline In-Reply-To: <564A27CD.7090908@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 21:07:05 -0000 --1LKvkjL3sHcu1TtY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 16, 2015 at 10:00:29PM +0300, Andrey Chernov wrote: > On 16.11.2015 20:35, Ed Schouten wrote: > > I personally think it's a shame if we were to deviate from returning > > "US-ASCII", for the reason that "US-ASCII" also happens to be the > > preferred MIME name for the character set: > >=20 > > http://www.iana.org/assignments/character-sets/character-sets.xhtml > >=20 > > "ASCII" doesn't even seem to be an alias for this character set. >=20 > Yes, I overlook it somehow. ASCII is not in the IANA, while both > ANSI_X3.4-1968 and US-ASCII are. >=20 > So, I reconsider the proposal. We can return ANSI_X3.4-1968 for POSIX/C > (for Linux compatibility reasons) and left pure US-ASCII as it was > (since it is used rarely). To tell the truth, the locale change I made were painful enough (mostly my fault)and I (for now) won't do anywork further beside fixing the fallouts i= f any are left. But I do support this proposal! >=20 > > In my opinion a decent implementation of newlocale() should support > > any of the character set names and aliases provided on the IANA page, > > but let nl_langinfo(CODESET) return the preferred MIME name. >=20 > BTW, we already have and return non-IANA codesets historically (inspired > by X11). I.e. we have ISO8859-* instead of preferred names ISO-8859-*, > moreover, ISO8859-* even not the aliases (!) and IANA knows nothing > about them. Linux have IANA preferred names here, i.e. ISO-8859-*. >=20 > So the question is: should we rename ISO8859-* to ISO-8859-* to be IANA > and Linux compatible? >=20 > We can strip first (or all) "_" and "-" from the environment names (as > Linux does), to not violate POLA. I would like to see that as well, lots of new comers I have seen setup the locales the IANA way and are unhappy because that does not work. The first = plan in the collation branch was to introduce the IANA syntax via an alias but i= n the end I removed it, because there was already to many changes. If one want to go further on the locale changes like the above proposal ple= ase proceed. Best regards, Bapt --1LKvkjL3sHcu1TtY Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlZKRXMACgkQ8kTtMUmk6EyWtwCeIVKFFZPT1lXelA5fxjt1AWx2 x0MAoIgm9AqE/5zXGaAWma2msbykkL0j =ar0A -----END PGP SIGNATURE----- --1LKvkjL3sHcu1TtY-- From owner-freebsd-arch@freebsd.org Mon Nov 16 21:51:47 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6028BA31585 for ; Mon, 16 Nov 2015 21:51:47 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 377B61C2D for ; Mon, 16 Nov 2015 21:51:47 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: by mailman.ysv.freebsd.org (Postfix) id 3721BA31584; Mon, 16 Nov 2015 21:51:47 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36B85A31583 for ; Mon, 16 Nov 2015 21:51:47 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: from mail-lf0-f52.google.com (mail-lf0-f52.google.com [209.85.215.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3E0C1C2A for ; Mon, 16 Nov 2015 21:51:46 +0000 (UTC) (envelope-from mailing-machine@vniz.net) Received: by lffu14 with SMTP id u14so95386692lff.1 for ; Mon, 16 Nov 2015 13:51:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type; bh=HlGzjNKEXfI/ikJnFVbi+M03PtR1k2byvkfFuFkGRLc=; b=isI9O5SY4zxmWoimyeJ+QkgP/MKBf40TN3/9V+zElB0aUoe+E7ZP0HbDz6WpRjcWnH lzcmFHC8FgYO+SDZge9f6RzNLgStkKIObZMQQehFYULGg4nwGeNUaejK8A6yq9QxdW4I GNunO5K08cg04KVl28i6y0hGmfkVcRIdW0hsqIZcKqft9cTVhdQQwDb4sQOcGT+8QrbX z833ISR4kGvHAx6ypHl6epQUPnr0q3o2mazHz5NrmonO7qYy9PFAGgirnNaf+GRE6cGB 12ft/lTMY8Z3RPkaGWlSquHTs/wDjbhRcP+5o78vjStJ828eTaTgtQf3gMgUykpdDkD2 KN+A== X-Gm-Message-State: ALoCoQnJiqmF2G6DyNokJxelMzfZbdjq1d6nrUBoza8lFDIvDFST0EHqVXWoD2rtdHdMo62yGpIL X-Received: by 10.25.37.207 with SMTP id l198mr18201598lfl.111.1447710699375; Mon, 16 Nov 2015 13:51:39 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by smtp.gmail.com with ESMTPSA id jj4sm5793674lbc.14.2015.11.16.13.51.38 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Nov 2015 13:51:38 -0800 (PST) Subject: Re: Question about ASCII and nl_langinfo (locale work) To: Baptiste Daroussin References: <20151110222636.GN10134@ivaldir.etoilebsd.net> <564A27CD.7090908@freebsd.org> <20151116210659.GB59189@ivaldir.etoilebsd.net> Cc: Ed Schouten , arch@freebsd.org, marino@freebsd.org From: Andrey Chernov X-Enigmail-Draft-Status: N1110 Message-ID: <564A4FE9.6000403@freebsd.org> Date: Tue, 17 Nov 2015 00:51:37 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151116210659.GB59189@ivaldir.etoilebsd.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="4JVdCN2phdgUEH1IVhFbLtc6Oj1V3db3w" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 21:51:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --4JVdCN2phdgUEH1IVhFbLtc6Oj1V3db3w Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 17.11.2015 0:06, Baptiste Daroussin wrote: > locales the IANA way and are unhappy because that does not work. The fi= rst plan > in the collation branch was to introduce the IANA syntax via an alias b= ut in the > end I removed it, because there was already to many changes. For ISO case we don't need aliases and can keep our internal names hierarchy honoring POLA. All we need is: 1) Convert "ISO-" and "ISO_" to "ISO" for setlocale(3) input. 2) Convert from "ISO" to "ISO-" for setlocale(3), nl_langinfo(3) and locale(1) output. --=20 http://ache.vniz.net/ --4JVdCN2phdgUEH1IVhFbLtc6Oj1V3db3w Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJWSk/qAAoJEKUckv0MjfbKMQwH/2wnw5PePa5Kx4bnIOdaihiA WTN6lM6IZqx1rKlQO2NlqmC5zcbVMPEON+IRENpm6yClrjJbJcVdDOVAJGhxmufp EREZ73lwnG0NvExOjajMhUYRDRZ7Oo7PpyTZKvM0dxHHtF/CpdRAsChJWZa7qZVQ /CWU/t92ElOMpPFbwQXL6ZOJ6hB+PwoR5bc7WCgOU0tBUE1fHZbVNX7Rypy3kYOs w14rq4G9dqy6/GLGwdP7POYVqdG3SqAQ4AtZ71nlu7xjUG8Kl6wBCKGhD6e8XXXY 7hSbKfSjcUJnqHeK51u1yAroTb0g+cYwQFe85qIN0bBEfUo1GZ4IuSmRB/Bhfdo= =xGWO -----END PGP SIGNATURE----- --4JVdCN2phdgUEH1IVhFbLtc6Oj1V3db3w-- From owner-freebsd-arch@freebsd.org Mon Nov 16 23:04:50 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E13F8A3014F for ; Mon, 16 Nov 2015 23:04:49 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7123418C2; Mon, 16 Nov 2015 23:04:49 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.15.2/8.15.2/ALCHEMY.FRANKEN.DE) with ESMTPS id tAGN4dEn078211 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 17 Nov 2015 00:04:39 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.15.2/8.15.2/Submit) id tAGN4dw0078210; Tue, 17 Nov 2015 00:04:39 +0100 (CET) (envelope-from marius) Date: Tue, 17 Nov 2015 00:04:39 +0100 From: Marius Strobl To: John Baldwin Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Message-ID: <20151116230439.GA77914@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <111428878.dys4vNKReT@ralph.baldwin.cx> <20151112234146.GA76156@alchemy.franken.de> <5385051.zAN7Yc63R0@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5385051.zAN7Yc63R0@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (alchemy.franken.de [0.0.0.0]); Tue, 17 Nov 2015 00:04:39 +0100 (CET) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2015 23:04:50 -0000 On Fri, Nov 13, 2015 at 11:50:37AM -0800, John Baldwin wrote: > On Friday, November 13, 2015 12:41:46 AM Marius Strobl wrote: > > On Thu, Nov 12, 2015 at 02:36:42PM -0800, John Baldwin wrote: > > > On Monday, August 31, 2015 02:21:19 PM John Baldwin wrote: > > > > On Wednesday, August 12, 2015 10:50:20 AM John Baldwin wrote: > > > > > On Tuesday, August 04, 2015 10:56:09 AM John Baldwin wrote: > > > > > > Many debuggers (recent gdb and lldb) support cross-architecture debugging > > > > > > just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for > > > > > > remote targets already, but I wanted it to also support cross-debugging for > > > > > > vmcores. > > > > > > > > > > > > The existing libkvm/kgdb code in the tree has some limited support for > > > > > > cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a) > > > > > > and custom kgdb for each target platform. However, gdb (and lldb) both > > > > > > support multiple targets in a single binary, so I'd like to have a single > > > > > > kgdb binary that can cross-debug anything. > > > > > > > > > > > > I started hacking on libkvm last weekend and have a prototype that I've used > > > > > > (along with some patches to my kgdb port) to debug an amd64 vmcore on an > > > > > > i386 machine and vice versa. > > > > > > > > > > > > ... > > > > > > > > > > > > What I'm mostly after is comments on the API, etc. Once that is settled I > > > > > > will move forward on converting and/or stubbing the other backends (the > > > > > > stub route would be to only support other backends on native systems for > > > > > > now). > > > > > > > > > > I guess this is closer to a nuclear power plant than a bikeshed judging by the > > > > > feedback. I have ported the rest of the MD backends and verified that the > > > > > updated libkvm passes a universe build (including various static assertions > > > > > for the duplicated constants in other backends). What I have not done is any > > > > > runtime testing and I would like to ask for help with that now. In particular > > > > > I need someone to test that kgdb and/or ps works against a native core dump > > > > > on all platforms other than amd64 and i386. Note that some of the trickiness > > > > > is that the backends now have to make runtime decisions for things that were > > > > > previously compile-time decisions. The biggest one affected by this is the > > > > > MIPS backend as that backend handles three ABIs (mipso32, mipsn32, and mipsn64). > > > > > I believe I have the handling for that correct (mips[on]32 use 32-bit KSEGs > > > > > where as mipsn64 uses the extended segments and compat32 KSEGS, and mipso32 > > > > > uses 32-bit PTEs and mipsn32/n64 both use 64-bit PTEs) (plus both endians > > > > > for both in theory). The ARM backend also handles both endians (in theory). > > > > > > > > > > Another wrinkle is that sparc64 uses its own dump format instead of writing > > > > > out an ELF file. I had to convert the header structures to use fixed-width > > > > > types to be cross-friendly. It would be good to ensure that a new libkvm > > > > > can read a vmcore from an old kernel and vice versa to make sure my conversion > > > > > is correct (I added an explicit padding field that I believe was implicit > > > > > before). > > > > > > > > > > The code is currently available for review in phabric at > > > > > https://reviews.freebsd.org/D3341 > > > > > > > > > > To test, you can run 'arc patch D3341' in a clean tree to apply the patch. > > > > > > > > I've just rebased this to port aarch64's minidump support. I just need people > > > > willing and able to test on non-x86. Testing with the in-tree kgdb using an > > > > updated libkvm would be sufficient. > > > > > > After a lot of crickets, I have updated the manpages for the new API. I will > > > commit this "soon". If you want kgdb to keep working on your non-x86 > > > platform, this is your chance to test this before it hits the tree. > > > > > > > What exact test procedure do you suggest for full coverage of an > > architecture? > > Just ensuring that kgdb and things like ps -M -N still work. With the patch from D3341 applied, kgdb(1) still seems to work fine on sparc64. However, `ps -M -N ` doesn't; it just prints the header and then exists after a short pause. Using the same core and kernel with ps(1) on a machine with userland built without your patch, ps(1) just segfaults after a short period of time. I can't tell whether that's a regression or not as I've never used ps(1) on a core before and you also have added padding to struct sparc64_dump_hdr, which might be responsible for triggering the segfault. On the other hand, an old kgb(1) seemingly works fine with the new core. FYI, I needed the follow patch on top of D3341 (based on the amd64 counterpart): --- lib/libkvm/kvm_minidump_aarch64.c 2015-11-16 23:41:58.075242000 +0100 +++ lib/libkvm/kvm_minidump_aarch64.c 2015-11-16 13:25:26.411577000 +0100 @@ -122,7 +122,7 @@ return (-1); } if (pread(kd->pmfd, bitmap, vmst->hdr.bitmapsize, off) != - vmst->hdr.bitmapsize) { + (ssize_t)vmst->hdr.bitmapsize) { _kvm_err(kd, kd->program, "cannot read %d bytes for page bitmap", vmst->hdr.bitmapsize); @@ -215,7 +215,7 @@ } invalid: - _kvm_err(kd, 0, "invalid address (0x%lx)", va); + _kvm_err(kd, 0, "invalid address (0x%jx)", (uintmax_t)va); return (0); } Also, parallel builds failed with something not finding libelf but building with a single jobs succeeded. I don't know whether D3341 introduces that or if it's a bug in head (the latter probably is unlikely but I didn't investigate). > Btw, Mark Linimon tried to generate a crashdump for me on his sparc64 running > HEAD recently so I could test the updated kgdb but it failed to generate a > dump. Ah, that reminds me of something; fixed in r290957. > I was hoping that the thread on sparc64 with the guy from qemu would > result in working qemu as that would let me do the testing I need for this > and kgdb locally. > Yeah, I also thought that after all that time, OpenBIOS and QEMU would have progressed some more. Well, some bugs are fixed now but they're also still not quite there, yet. Marius From owner-freebsd-arch@freebsd.org Tue Nov 17 00:41:19 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 52FABA31571 for ; Tue, 17 Nov 2015 00:41:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12D071A8F for ; Tue, 17 Nov 2015 00:41:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 48350B946; Mon, 16 Nov 2015 19:41:17 -0500 (EST) From: John Baldwin To: Marius Strobl Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Date: Mon, 16 Nov 2015 16:37:32 -0800 Message-ID: <5992121.1Qh8fceFnn@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20151116230439.GA77914@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <5385051.zAN7Yc63R0@ralph.baldwin.cx> <20151116230439.GA77914@alchemy.franken.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 16 Nov 2015 19:41:17 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 00:41:19 -0000 On Tuesday, November 17, 2015 12:04:39 AM Marius Strobl wrote: > On Fri, Nov 13, 2015 at 11:50:37AM -0800, John Baldwin wrote: > > On Friday, November 13, 2015 12:41:46 AM Marius Strobl wrote: > > > On Thu, Nov 12, 2015 at 02:36:42PM -0800, John Baldwin wrote: > > > > On Monday, August 31, 2015 02:21:19 PM John Baldwin wrote: > > > > > On Wednesday, August 12, 2015 10:50:20 AM John Baldwin wrote: > > > > > > On Tuesday, August 04, 2015 10:56:09 AM John Baldwin wrote: > > > > > > > Many debuggers (recent gdb and lldb) support cross-architecture debugging > > > > > > > just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for > > > > > > > remote targets already, but I wanted it to also support cross-debugging for > > > > > > > vmcores. > > > > > > > > > > > > > > The existing libkvm/kgdb code in the tree has some limited support for > > > > > > > cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a) > > > > > > > and custom kgdb for each target platform. However, gdb (and lldb) both > > > > > > > support multiple targets in a single binary, so I'd like to have a single > > > > > > > kgdb binary that can cross-debug anything. > > > > > > > > > > > > > > I started hacking on libkvm last weekend and have a prototype that I've used > > > > > > > (along with some patches to my kgdb port) to debug an amd64 vmcore on an > > > > > > > i386 machine and vice versa. > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > What I'm mostly after is comments on the API, etc. Once that is settled I > > > > > > > will move forward on converting and/or stubbing the other backends (the > > > > > > > stub route would be to only support other backends on native systems for > > > > > > > now). > > > > > > > > > > > > I guess this is closer to a nuclear power plant than a bikeshed judging by the > > > > > > feedback. I have ported the rest of the MD backends and verified that the > > > > > > updated libkvm passes a universe build (including various static assertions > > > > > > for the duplicated constants in other backends). What I have not done is any > > > > > > runtime testing and I would like to ask for help with that now. In particular > > > > > > I need someone to test that kgdb and/or ps works against a native core dump > > > > > > on all platforms other than amd64 and i386. Note that some of the trickiness > > > > > > is that the backends now have to make runtime decisions for things that were > > > > > > previously compile-time decisions. The biggest one affected by this is the > > > > > > MIPS backend as that backend handles three ABIs (mipso32, mipsn32, and mipsn64). > > > > > > I believe I have the handling for that correct (mips[on]32 use 32-bit KSEGs > > > > > > where as mipsn64 uses the extended segments and compat32 KSEGS, and mipso32 > > > > > > uses 32-bit PTEs and mipsn32/n64 both use 64-bit PTEs) (plus both endians > > > > > > for both in theory). The ARM backend also handles both endians (in theory). > > > > > > > > > > > > Another wrinkle is that sparc64 uses its own dump format instead of writing > > > > > > out an ELF file. I had to convert the header structures to use fixed-width > > > > > > types to be cross-friendly. It would be good to ensure that a new libkvm > > > > > > can read a vmcore from an old kernel and vice versa to make sure my conversion > > > > > > is correct (I added an explicit padding field that I believe was implicit > > > > > > before). > > > > > > > > > > > > The code is currently available for review in phabric at > > > > > > https://reviews.freebsd.org/D3341 > > > > > > > > > > > > To test, you can run 'arc patch D3341' in a clean tree to apply the patch. > > > > > > > > > > I've just rebased this to port aarch64's minidump support. I just need people > > > > > willing and able to test on non-x86. Testing with the in-tree kgdb using an > > > > > updated libkvm would be sufficient. > > > > > > > > After a lot of crickets, I have updated the manpages for the new API. I will > > > > commit this "soon". If you want kgdb to keep working on your non-x86 > > > > platform, this is your chance to test this before it hits the tree. > > > > > > > > > > What exact test procedure do you suggest for full coverage of an > > > architecture? > > > > Just ensuring that kgdb and things like ps -M -N still work. > > With the patch from D3341 applied, kgdb(1) still seems to work fine on > sparc64. However, `ps -M -N ` doesn't; it just prints > the header and then exists after a short pause. Using the same core and > kernel with ps(1) on a machine with userland built without your patch, > ps(1) just segfaults after a short period of time. I can't tell whether > that's a regression or not as I've never used ps(1) on a core before > and you also have added padding to struct sparc64_dump_hdr, which might > be responsible for triggering the segfault. On the other hand, an old > kgb(1) seemingly works fine with the new core. Hmm, I had thought that the old and new sparc64_dump_hdr would be the same? I was just using fixed width types so that any platform could #include the header and get the same layout. In particular, I don't want the dump format to change on disk after this change so that once kgdb (or lldb) has cross-debugging support we can read both old and new sparc64 vmcores. > FYI, I needed the follow patch on top of D3341 (based on the amd64 > counterpart): > --- lib/libkvm/kvm_minidump_aarch64.c 2015-11-16 23:41:58.075242000 +0100 > +++ lib/libkvm/kvm_minidump_aarch64.c 2015-11-16 13:25:26.411577000 +0100 > @@ -122,7 +122,7 @@ > return (-1); > } > if (pread(kd->pmfd, bitmap, vmst->hdr.bitmapsize, off) != > - vmst->hdr.bitmapsize) { > + (ssize_t)vmst->hdr.bitmapsize) { > _kvm_err(kd, kd->program, > "cannot read %d bytes for page bitmap", > vmst->hdr.bitmapsize); > @@ -215,7 +215,7 @@ > } > > invalid: > - _kvm_err(kd, 0, "invalid address (0x%lx)", va); > + _kvm_err(kd, 0, "invalid address (0x%jx)", (uintmax_t)va); > return (0); > } Oops, yes. I fixed this in my git branch when I built universe with it recently but I might not have pushed that update to phabricator yet. > Also, parallel builds failed with something not finding libelf but > building with a single jobs succeeded. I don't know whether D3341 > introduces that or if it's a bug in head (the latter probably is > unlikely but I didn't investigate). Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox builds did not trip over that, and lib/Makefile has libelf in its "early" list of libraries (SUBDIR_ORDERED), so it seems like it should be built before libkvm is tried? > > Btw, Mark Linimon tried to generate a crashdump for me on his sparc64 running > > HEAD recently so I could test the updated kgdb but it failed to generate a > > dump. > > Ah, that reminds me of something; fixed in r290957. Thanks! -- John Baldwin From owner-freebsd-arch@freebsd.org Tue Nov 17 01:53:33 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0C2AA3071F; Tue, 17 Nov 2015 01:53:33 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id AAB0517B8; Tue, 17 Nov 2015 01:53:33 +0000 (UTC) (envelope-from bright@mu.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8004:7515:6d56:aa8e:b437:27b3]) by elvis.mu.org (Postfix) with ESMTPSA id 947C4345A916; Mon, 16 Nov 2015 17:53:32 -0800 (PST) Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 To: Warner Losh , Elizabeth Myers References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <563F8385.3090603@freebsd.org> <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> Cc: Anna Wilcox , "Brian McGovern (bmcgover)" , freebsd-arch , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard From: Alfred Perlstein Message-ID: <564A889C.9070209@mu.org> Date: Mon, 16 Nov 2015 17:53:32 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 01:53:33 -0000 On 11/14/15 9:16 AM, Warner Losh wrote: > On Fri, Nov 13, 2015 at 11:15 PM, Elizabeth Myers > wrote: > >> You are seriously going to use "we're not NetBSD" as an argument? > > You noticed I didn't reply to it. The argument is completely lame. FreeBSD > runs > today in a variety of markets. Some new, some not so new. The thing that > makes > each of these areas unique is that there's a thriving community around them, > FreeBSD still runs well enough on these machines to get something done, and > when things break, they get fixed in a timely manner. > > Alpha was removed because it got broken by some changes, and stayed broken > for a long time despite repeated requests to fix it. Sparc64 is on the cusp > of that: > some minor things are broken, but have been fixed. The current crisis is > due to > the end of life of gcc in the tree and its fallout coupled with some > neglect of the > port due to time constraints. > > At first I was all for removal. With more data, I'm less sure. If the > promises are kept > made in this thread, it looks to remain viable for a while, though the lack > of a > qemu-user solution means that packages for a slow platform (where they are > really quite useful) will remain limited. Maybe there's enough hardware > around > that third-party pkg repos can fill the gap, maybe not. I think we should > experiment > with this model and see what it produces. Give the branching of 11 as the > deadline > to show something viable... One of the things I never understood about FreeBSD's method of maintaining a port was the way the platform porting was done. We really do things in a different manner than what my perception of other OSes is. My impression (please do correct me if I'm wrong) was that other OSes such as NetBSD and Linux had "platform maintainers". These maintainers were around to: 1) keep the ship sailing on those platforms 2) guide the general code base from becoming non-portable to other architectures (within reason). 3) drive the release of the architecture in question, helping the release engineer with image building and release testing. For point 1 above, what that meant to me was that let's say Linus or NetBSD in general wanted to do a major or minor change on a tier 1 platform, then it was the responsibility of the *platform maintainers* to do the work on the non tier-1 platforms to keep them up to date. Those "platform maintainers" kept those ships sailing and in return they got to be called "the $arch maintainer" which looks plenty good on a resume and also feels good for those that get excited for status. For point 2, let's say someone had a change that pushed some form of *completely* non-portable code into the base which would break a reasonable to support platform, then the "platform maintainer" would speak up and tell the general community "uh no, you can't do X on this platform, we need to rethink this". For point 3, there may be a lag between release of the OS for tier 1 (x86/x64) and the secondary architectures, but that is OK because the maintainer will eventually provide images themselves or in collaboration with the release engineer. FreeBSD seems to take a different approach. This approach is that someone (or some people) form a team to port to a platform. These "platform porter" groups sole responsibility is to get a new architecture running. After it's mostly running they are mostly without responsibility, however we tend to give them the right of change-set veto in perpetuity of the marginal relevance of the ported to platform. What this means is that instead of a assigning a title and ownership of the platform to someone, who maintains the status as "maintainer" by keeping that platform working. By keeping the platform working I am saying that they would do items 1, 2 and 3 from the NetBSD/Linux list. However, instead nearly immediately hoist the "platform maintenance" onto the general community of people that may not have access to the hardware in question. Maybe this is just my perception, but it would seem to make a ton more sense to follow the NetBSD/Linux model which implies a somewhat decoupled release model (not all arches must come out on the same day) and assigning ownership and responsibility in exchange for status based on being the "platform maintainer". Finally it would be pretty obvious when everyone steps down or just doesn't participate in the release process that it may be time to sunset a platform. -Alfred From owner-freebsd-arch@freebsd.org Tue Nov 17 01:57:56 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1202EA307CA; Tue, 17 Nov 2015 01:57:56 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC91819D2; Tue, 17 Nov 2015 01:57:55 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by ioek191 with SMTP id k191so455577ioe.3; Mon, 16 Nov 2015 17:57:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=NI7EEh4VpBKHkd5+JpRrOXqI/61eSad3bf0PIxd3ixc=; b=DJu01wTD+vXJLGDMph4XRamNd/su46mY2FfR29M2OmUKh9Dom8TLDzIJs2z8drL/ua 55pYYbW3nKvUzF8f4U2hj6ho2X5z1SUQfTlz3ZUa1R/kp8Hn90MW4k+bU0scvr0e+ePK 0LqXZBRUheuPw5rm/jGEMRzhZyGx63znYXZ/UZPud04ETQ0Nie+RInoB7w3gpKvV5A7q l4KnP3X+eZ75KvOXKUi3uDWVth1qWacIsvuwkk4OmvJDWctcFEa6A+GuUMUZTKnXwpDo VyKcAd4NLa4syAW0tmwINSu1qxa+E3tOG4pCWHhM0zoLTN+ksi5VlLbo0V7NmrqVGx6y y3tA== MIME-Version: 1.0 X-Received: by 10.107.162.21 with SMTP id l21mr7575815ioe.123.1447725475279; Mon, 16 Nov 2015 17:57:55 -0800 (PST) Received: by 10.36.217.196 with HTTP; Mon, 16 Nov 2015 17:57:55 -0800 (PST) In-Reply-To: <564A889C.9070209@mu.org> References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <563F8385.3090603@freebsd.org> <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> Date: Mon, 16 Nov 2015 17:57:55 -0800 Message-ID: Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 From: Adrian Chadd To: Alfred Perlstein Cc: Warner Losh , Elizabeth Myers , Anna Wilcox , freebsd-arch , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 01:57:56 -0000 On 16 November 2015 at 17:53, Alfred Perlstein wrote: > > > > On 11/14/15 9:16 AM, Warner Losh wrote: >> >> On Fri, Nov 13, 2015 at 11:15 PM, Elizabeth Myers >> >> wrote: >> >>> You are seriously going to use "we're not NetBSD" as an argument? >> >> >> You noticed I didn't reply to it. The argument is completely lame. FreeBSD >> runs >> today in a variety of markets. Some new, some not so new. The thing that >> makes >> each of these areas unique is that there's a thriving community around >> them, >> FreeBSD still runs well enough on these machines to get something done, >> and >> when things break, they get fixed in a timely manner. >> >> Alpha was removed because it got broken by some changes, and stayed broken >> for a long time despite repeated requests to fix it. Sparc64 is on the >> cusp >> of that: >> some minor things are broken, but have been fixed. The current crisis is >> due to >> the end of life of gcc in the tree and its fallout coupled with some >> neglect of the >> port due to time constraints. >> >> At first I was all for removal. With more data, I'm less sure. If the >> promises are kept >> made in this thread, it looks to remain viable for a while, though the >> lack >> of a >> qemu-user solution means that packages for a slow platform (where they are >> really quite useful) will remain limited. Maybe there's enough hardware >> around >> that third-party pkg repos can fill the gap, maybe not. I think we should >> experiment >> with this model and see what it produces. Give the branching of 11 as the >> deadline >> to show something viable... > > > One of the things I never understood about FreeBSD's method of maintaining a > port was the way the platform porting was done. We really do things in a > different manner than what my perception of other OSes is. > > My impression (please do correct me if I'm wrong) was that other OSes such > as NetBSD and Linux had "platform maintainers". > > These maintainers were around to: > 1) keep the ship sailing on those platforms > 2) guide the general code base from becoming non-portable to other > architectures (within reason). > 3) drive the release of the architecture in question, helping the release > engineer with image building and release testing. > > For point 1 above, what that meant to me was that let's say Linus or NetBSD > in general wanted to do a major or minor change on a tier 1 platform, then > it was the responsibility of the *platform maintainers* to do the work on > the non tier-1 platforms to keep them up to date. Those "platform > maintainers" kept those ships sailing and in return they got to be called > "the $arch maintainer" which looks plenty good on a resume and also feels > good for those that get excited for status. > > For point 2, let's say someone had a change that pushed some form of > *completely* non-portable code into the base which would break a reasonable > to support platform, then the "platform maintainer" would speak up and tell > the general community "uh no, you can't do X on this platform, we need to > rethink this". > > For point 3, there may be a lag between release of the OS for tier 1 > (x86/x64) and the secondary architectures, but that is OK because the > maintainer will eventually provide images themselves or in collaboration > with the release engineer. > > FreeBSD seems to take a different approach. This approach is that someone > (or some people) form a team to port to a platform. These "platform porter" > groups sole responsibility is to get a new architecture running. After it's > mostly running they are mostly without responsibility, however we tend to > give them the right of change-set veto in perpetuity of the marginal > relevance of the ported to platform. > > What this means is that instead of a assigning a title and ownership of the > platform to someone, who maintains the status as "maintainer" by keeping > that platform working. By keeping the platform working I am saying that > they would do items 1, 2 and 3 from the NetBSD/Linux list. However, instead > nearly immediately hoist the "platform maintenance" onto the general > community of people that may not have access to the hardware in question. > > Maybe this is just my perception, but it would seem to make a ton more sense > to follow the NetBSD/Linux model which implies a somewhat decoupled release > model (not all arches must come out on the same day) and assigning ownership > and responsibility in exchange for status based on being the "platform > maintainer". > > Finally it would be pretty obvious when everyone steps down or just doesn't > participate in the release process that it may be time to sunset a platform. +1 -a From owner-freebsd-arch@freebsd.org Tue Nov 17 06:22:34 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEE77A30512 for ; Tue, 17 Nov 2015 06:22:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk0-x22c.google.com (mail-qk0-x22c.google.com [IPv6:2607:f8b0:400d:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7679B1ACA for ; Tue, 17 Nov 2015 06:22:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qkfo3 with SMTP id o3so126832895qkf.1 for ; Mon, 16 Nov 2015 22:22:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp_com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dKk2eNnyXsy5DJ397r4P8+ziAZJ+fKOjQqEuyizWLVw=; b=kr7BUTzNQ+ONsJo8BBEoSF8YTst5T0JIF4iXHFsavgMgzlQ1sr8STnFZxDv4O1dUXr BdxrLHsY3KPLires2UFD1wnv40SYKXRIBD/ZOFDnL2ljeyWADP3WZ63YLHLDqoljMDDb Q+uUwG4xQBc3w8+H28cfxyLMqJIfZTcDfAp7IXZSc3j2pjKLZEfAJB2BoLhQ/rtwZEog BR8GvWRune+oJfaddiMMS3JpTDsSS07qYfTc1nisQtp+ic818sLRMQ0ZHdzujzZtaJal HT5ZD2tz7OJ0ApMJyNi5Ke0+tSxhFcAGSoOJA6LcsNPA4wUd2K2FWRBwKQYtrVb48oH1 7hsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=dKk2eNnyXsy5DJ397r4P8+ziAZJ+fKOjQqEuyizWLVw=; b=lUwL9qkLAOZAwOfrni3jRZyE2pq3kd0SQFoWv1IORUWVextljbxqt+LtEGJDgdSp3G 4NK6Y/+NJ8cKhiLbPlTambOnMzHGwE9NhI0fBOuGqhYQlgdXl7C3Lk5J/3pGJEp9auXJ SA4OU4ecSqvy5/jxbZ3p8RHQGegIo4CUR5bC0YmiLDdM/6HZmSGOM7smN9MagqVcD1gx rvACY67efb/qSUvOcje0TSbqCW2fcUoaBd2pI/pI4pQZHw5+Fxqf1d06UMwcuEVzpmiY Qc+atxD8TUPuLgwp6SV3yg/aQkHAkeiK5g4QUry/otYpAc4dKjmg8H65tUYMebv6kn2W T7dA== X-Gm-Message-State: ALoCoQl8nzJ4CMD1XINJbWqwSR/V2fDcXW6ReYC/CBspdA7oGM3EllvJtR5s+ND8s5U6IMQRqQva MIME-Version: 1.0 X-Received: by 10.55.33.40 with SMTP id h40mr39744971qkh.77.1447741353261; Mon, 16 Nov 2015 22:22:33 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Mon, 16 Nov 2015 22:22:32 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: <564A889C.9070209@mu.org> References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <563F8385.3090603@freebsd.org> <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> Date: Mon, 16 Nov 2015 23:22:32 -0700 X-Google-Sender-Auth: g4TgqM2p3Vub1dt8tyQjJIkzAS8 Message-ID: Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 From: Warner Losh To: Alfred Perlstein Cc: Elizabeth Myers , Anna Wilcox , "Brian McGovern (bmcgover)" , freebsd-arch , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 06:22:35 -0000 On Mon, Nov 16, 2015 at 6:53 PM, Alfred Perlstein wrote: > > > > On 11/14/15 9:16 AM, Warner Losh wrote: > >> On Fri, Nov 13, 2015 at 11:15 PM, Elizabeth Myers < >> elizabeth@interlinked.me> >> wrote: >> >> You are seriously going to use "we're not NetBSD" as an argument? >>> >> >> You noticed I didn't reply to it. The argument is completely lame. FreeBSD >> runs >> today in a variety of markets. Some new, some not so new. The thing that >> makes >> each of these areas unique is that there's a thriving community around >> them, >> FreeBSD still runs well enough on these machines to get something done, >> and >> when things break, they get fixed in a timely manner. >> >> Alpha was removed because it got broken by some changes, and stayed broken >> for a long time despite repeated requests to fix it. Sparc64 is on the >> cusp >> of that: >> some minor things are broken, but have been fixed. The current crisis is >> due to >> the end of life of gcc in the tree and its fallout coupled with some >> neglect of the >> port due to time constraints. >> >> At first I was all for removal. With more data, I'm less sure. If the >> promises are kept >> made in this thread, it looks to remain viable for a while, though the >> lack >> of a >> qemu-user solution means that packages for a slow platform (where they are >> really quite useful) will remain limited. Maybe there's enough hardware >> around >> that third-party pkg repos can fill the gap, maybe not. I think we should >> experiment >> with this model and see what it produces. Give the branching of 11 as the >> deadline >> to show something viable... >> > > One of the things I never understood about FreeBSD's method of maintaining > a port was the way the platform porting was done. We really do things in a > different manner than what my perception of other OSes is. > > My impression (please do correct me if I'm wrong) was that other OSes such > as NetBSD and Linux had "platform maintainers". > > These maintainers were around to: > 1) keep the ship sailing on those platforms > 2) guide the general code base from becoming non-portable to other > architectures (within reason). > 3) drive the release of the architecture in question, helping the release > engineer with image building and release testing. > > For point 1 above, what that meant to me was that let's say Linus or > NetBSD in general wanted to do a major or minor change on a tier 1 > platform, then it was the responsibility of the *platform maintainers* to > do the work on the non tier-1 platforms to keep them up to date. Those > "platform maintainers" kept those ships sailing and in return they got to > be called "the $arch maintainer" which looks plenty good on a resume and > also feels good for those that get excited for status. > I'm not sure how the people that actually take care of these things on FreeBSD differ. There are people recognized as go-to people for the different ports that are fairly active in the on-going issues that come up with kernel code. Userland code doesn't seem to matter that much given the platforms we support. For PowerPC, you have Nathan W and Justin Hibbits. For mips, there's Adrian, myself, Julie Mallet and a few others. For arm there's a long cast of characters. For PC98 there's Takahashi-san. For sparc64 there's Marius. These people keep the ship sailing (or in some cases they remove the ship form the tree). They advise discussions about issues that are relevant to the platform, like cache lines and cache coherence, they call people out when they break these platforms or when people used to big systems adjust the tuning and break small systems. > For point 2, let's say someone had a change that pushed some form of > *completely* non-portable code into the base which would break a reasonable > to support platform, then the "platform maintainer" would speak up and tell > the general community "uh no, you can't do X on this platform, we need to > rethink this". > People generally don't push this kind of code into the tree these days. When they do, they get called out on it. Some of them even listen to the calling out and fix things, others don't and one of the platform maintainers has to fix the stupid pushed into the tree. Sometimes this happens right away and sometimes there's a lag. sometimes it's code for newer versions of the platform that break older versions (or vice versa). Other times there's code from another platform that breaks things. USB is a textbook example of this happening. It went in, and didn't worth a damn on arm or mips. The ports maintainers of the arm and mips platforms tried to explain what the issues were. It took some time, but it got mostly worked out as the embedded folks got to know USB issues, and hps got to understand the issues with embedded hardware. > For point 3, there may be a lag between release of the OS for tier 1 > (x86/x64) and the secondary architectures, but that is OK because the > maintainer will eventually provide images themselves or in collaboration > with the release engineer. > For this point, we've pushed the knowledge of how to build the images into the release engineer. The folks that are around that are using the port test the images. Sometimes it's the port maintainers, but recently it has been a large cast of characters for popular platforms. > FreeBSD seems to take a different approach. This approach is that someone > (or some people) form a team to port to a platform. These "platform > porter" groups sole responsibility is to get a new architecture running. > After it's mostly running they are mostly without responsibility, however > we tend to give them the right of change-set veto in perpetuity of the > marginal relevance of the ported to platform. > so like when did this actually happen? > What this means is that instead of a assigning a title and ownership of > the platform to someone, who maintains the status as "maintainer" by > keeping that platform working. By keeping the platform working I am saying > that they would do items 1, 2 and 3 from the NetBSD/Linux list. However, > instead nearly immediately hoist the "platform maintenance" onto the > general community of people that may not have access to the hardware in > question. > Do you have a specific example of when we've done this? As far as I know, based on powerpc, arm and mips anyway, the people claiming to be maintainers are actively doing 1, 2 and helping RE do number 3 to varying degrees. As far as I know, they all have access to some or all of the hardware they are maintaining, and many of our power users participate in the process as well. > Maybe this is just my perception, but it would seem to make a ton more > sense to follow the NetBSD/Linux model which implies a somewhat decoupled > release model (not all arches must come out on the same ) and assigning > ownership and responsibility in exchange for status based on being the > "platform maintainer". > So, rather than generalizations, be specific. Who do you think is claiming to be a port maintainer, blocking progress and needs to be replaced? And what, beyond what the re@ does today, would you do differently? What do we gain over what we do for tier 1 platforms? Is there a platform wanting a release that isn't getting one? mips has two different groups that have put out releases for it, with one of them fading into the background. Adiran is making wifi builds available, already following this model you say we should adopt. The japanese user groups are putting out PC98 releases now that the re@ has dropped them (they never really stopped in the mean time). sparc64, powerpc, arm, i386 and amd64 are all released by re@. ia64 has been removed from the tree. What other platforms are there? What else needs to be done. > Finally it would be pretty obvious when everyone steps down or just > doesn't participate in the release process that it may be time to sunset a > platform. That's why we are having the conversation about sparc64. It looked like it might no longer be participating in the normal process. Now, while there are some issues that were identified with sparc64, some of them are real (see qemu and difficulties building in the cluster). Some of them were just perception (the reduced numbers of commits to sparc64 didn't seem to represent a problem with the platform and the perceived issues had been cleared up) So what, specific, actionable items do we as a project actually need to do here? I'm sure there are some and that we can improve our process. I'm having trouble teasing out what I, as someone who dabbles in arm and mips to varying degrees of 'maintainership' for different parts, can do better or different. Warner > > -Alfred > From owner-freebsd-arch@freebsd.org Tue Nov 17 06:44:04 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB81FA30A22; Tue, 17 Nov 2015 06:44:04 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 8F2C2139B; Tue, 17 Nov 2015 06:44:04 +0000 (UTC) (envelope-from bright@mu.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8004:7515:6d56:aa8e:b437:27b3]) by elvis.mu.org (Postfix) with ESMTPSA id B6705345A920; Mon, 16 Nov 2015 22:44:03 -0800 (PST) Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 To: Warner Losh References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <563F8385.3090603@freebsd.org> <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> Cc: Elizabeth Myers , Anna Wilcox , "Brian McGovern (bmcgover)" , freebsd-arch , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard From: Alfred Perlstein Message-ID: <564ACCB3.4070603@mu.org> Date: Mon, 16 Nov 2015 22:44:03 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 06:44:04 -0000 Warner, thanks for addressing this email. I think I wasn't clear which lead to some misunderstanding. I'll keep this reply succinct and the rest of it inline. Please don't take the succinctness as anything other than getting to the point. On 11/16/15 10:22 PM, Warner Losh wrote: > > > On Mon, Nov 16, 2015 at 6:53 PM, Alfred Perlstein > wrote: > > > > > On 11/14/15 9:16 AM, Warner Losh wrote: > > On Fri, Nov 13, 2015 at 11:15 PM, Elizabeth Myers > > > wrote: > > You are seriously going to use "we're not NetBSD" as an > argument? > > > You noticed I didn't reply to it. The argument is completely > lame. FreeBSD > runs > today in a variety of markets. Some new, some not so new. The > thing that > makes > each of these areas unique is that there's a thriving > community around them, > FreeBSD still runs well enough on these machines to get > something done, and > when things break, they get fixed in a timely manner. > > Alpha was removed because it got broken by some changes, and > stayed broken > for a long time despite repeated requests to fix it. Sparc64 > is on the cusp > of that: > some minor things are broken, but have been fixed. The current > crisis is > due to > the end of life of gcc in the tree and its fallout coupled > with some > neglect of the > port due to time constraints. > > At first I was all for removal. With more data, I'm less sure. > If the > promises are kept > made in this thread, it looks to remain viable for a while, > though the lack > of a > qemu-user solution means that packages for a slow platform > (where they are > really quite useful) will remain limited. Maybe there's enough > hardware > around > that third-party pkg repos can fill the gap, maybe not. I > think we should > experiment > with this model and see what it produces. Give the branching > of 11 as the > deadline > to show something viable... > > > One of the things I never understood about FreeBSD's method of > maintaining a port was the way the platform porting was done. We > really do things in a different manner than what my perception of > other OSes is. > > My impression (please do correct me if I'm wrong) was that other > OSes such as NetBSD and Linux had "platform maintainers". > > These maintainers were around to: > 1) keep the ship sailing on those platforms > 2) guide the general code base from becoming non-portable to other > architectures (within reason). > 3) drive the release of the architecture in question, helping the > release engineer with image building and release testing. > > For point 1 above, what that meant to me was that let's say Linus > or NetBSD in general wanted to do a major or minor change on a > tier 1 platform, then it was the responsibility of the *platform > maintainers* to do the work on the non tier-1 platforms to keep > them up to date. Those "platform maintainers" kept those ships > sailing and in return they got to be called "the $arch maintainer" > which looks plenty good on a resume and also feels good for those > that get excited for status. > > > I'm not sure how the people that actually take care of these things on > FreeBSD differ. There are people > recognized as go-to people for the different ports that are fairly > active in the on-going issues that come > up with kernel code. Userland code doesn't seem to matter that much > given the platforms we support. > > For PowerPC, you have Nathan W and Justin Hibbits. For mips, there's > Adrian, myself, Julie Mallet and a few > others. For arm there's a long cast of characters. For PC98 there's > Takahashi-san. For sparc64 there's Marius. > These people keep the ship sailing (or in some cases they remove the > ship form the tree). They advise > discussions about issues that are relevant to the platform, like cache > lines and cache coherence, This is how we diverge: > they call people out when they break these platforms or when people > used to big systems adjust the > tuning and break small systems. That is not done as far as I can tell in NetBSD/Linux. In Linux/NetBSD it is the job of the maintainer to keep the platform up to date, not to call out when someone breaks it. Meaning ideal: "Oh someone broke alignment in this struct on my platform, let me ask them how to fix it." As opposed to (not ideal): "Something broke my platform, let me track that guy down and make them fix it." > > For point 2, let's say someone had a change that pushed some form > of *completely* non-portable code into the base which would break > a reasonable to support platform, then the "platform maintainer" > would speak up and tell the general community "uh no, you can't do > X on this platform, we need to rethink this". > > > People generally don't push this kind of code into the tree these > days. When they do, they get called > out on it. Some of them even listen to the calling out and fix things, > others don't and one of the > platform maintainers has to fix the stupid pushed into the tree. > Sometimes this happens right away > and sometimes there's a lag. sometimes it's code for newer versions of > the platform that break older > versions (or vice versa). Other times there's code from another > platform that breaks things. > > USB is a textbook example of this happening. It went in, and didn't > worth a damn on arm or mips. > The ports maintainers of the arm and mips platforms tried to explain > what the issues were. It took > some time, but it got mostly worked out as the embedded folks got to > know USB issues, and hps > got to understand the issues with embedded hardware. > > For point 3, there may be a lag between release of the OS for tier > 1 (x86/x64) and the secondary architectures, but that is OK > because the maintainer will eventually provide images themselves > or in collaboration with the release engineer. > > > For this point, we've pushed the knowledge of how to build the images > into the release engineer. The > folks that are around that are using the port test the images. > Sometimes it's the port maintainers, > but recently it has been a large cast of characters for popular platforms. > > FreeBSD seems to take a different approach. This approach is that > someone (or some people) form a team to port to a platform. These > "platform porter" groups sole responsibility is to get a new > architecture running. After it's mostly running they are mostly > without responsibility, however we tend to give them the right of > change-set veto in perpetuity of the marginal relevance of the > ported to platform. > > > so like when did this actually happen? Well, earlier in this email you said this exact thing: > they call people out when they break these platforms or when people > used to big systems adjust the > tuning and break small systems. That's how I see the difference. > What this means is that instead of a assigning a title and > ownership of the platform to someone, who maintains the status as > "maintainer" by keeping that platform working. By keeping the > platform working I am saying that they would do items 1, 2 and 3 > from the NetBSD/Linux list. However, instead nearly immediately > hoist the "platform maintenance" onto the general community of > people that may not have access to the hardware in question. > > > Do you have a specific example of when we've done this? As far as I > know, based on powerpc, arm and > mips anyway, the people claiming to be maintainers are actively doing > 1, 2 and helping RE do number > 3 to varying degrees. As far as I know, they all have access to some > or all of the hardware they are > maintaining, and many of our power users participate in the process as > well. > Maybe this is just my perception, but it would seem to make a ton > more sense to follow the NetBSD/Linux model which implies a > somewhat decoupled release model (not all arches must come out on > the same ) and assigning ownership and responsibility in exchange > for status based on being the "platform maintainer". > > > So, rather than generalizations, be specific. Who do you think is > claiming to be a port maintainer, blocking > progress and needs to be replaced? > > And what, beyond what the re@ does today, would you do differently? > What do we gain over what we do > for tier 1 platforms? Is there a platform wanting a release that isn't > getting one? mips has two different > groups that have put out releases for it, with one of them fading into > the background. Adiran is making > wifi builds available, already following this model you say we should > adopt. The japanese user groups are > putting out PC98 releases now that the re@ has dropped them (they > never really stopped in the mean > time). sparc64, powerpc, arm, i386 and amd64 are all released by re@. > ia64 has been removed from the > tree. What other platforms are there? What else needs to be done. > > Finally it would be pretty obvious when everyone steps down or > just doesn't participate in the release process that it may be > time to sunset a platform. > > > That's why we are having the conversation about sparc64. It looked > like it might no longer be > participating in the normal process. Now, while there are some issues > that were identified with > sparc64, some of them are real (see qemu and difficulties building in > the cluster). Some of them > were just perception (the reduced numbers of commits to sparc64 didn't > seem to represent > a problem with the platform and the perceived issues had been cleared up) > > So what, specific, actionable items do we as a project actually need > to do here? I'm sure there are some > and that we can improve our process. I'm having trouble teasing out > what I, as someone who dabbles > in arm and mips to varying degrees of 'maintainership' for different > parts, can do better or different. > Three things: 1) I am wondering if core (or the community in general) should have some way of nominating a particular person as a platform maintainer. This would give accolades to that person and at the same time give us a point person. I believe part of the problem is that we don't give enough status to the port maintainers, are they on the website? How would I know who is the "king of mips" right now? Does someone get to put in on their resume with backing from the project? If not, then the maintainer will be grumbly as opposed to facilitating. 2) The role of platform maintainer needs not to be a blocking role, but rather a continual porting role. Specifically to avoid the "calling out" (sorry for the quotes here, but just want to get the point across) and instead function as a do-er/facilitator. Meaning if something breaks a secondary platform it's a shared responsibility of the port maintainer and developer to fix it, but solely the developer. 3) Tight coupling of the system. While it's good to cross train release engineering with various platforms and even more so it's REALLY great that this is being well received, the end result is a very tight coupling of the system that can lead to frustrations and logjams when things go wrong. Something that FreeBSD very often gets wrong is the strong unification of its various parts, you can see this in so many aspects of how we do things (releases, VFS, platforms) as opposed to other projects. It is an admirable aspiration, however it results in much lock-step of the entire project and doesn't scale. -Alfred From owner-freebsd-arch@freebsd.org Tue Nov 17 07:06:39 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 659C4A30F37 for ; Tue, 17 Nov 2015 07:06:39 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C7451BED for ; Tue, 17 Nov 2015 07:06:39 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by igcph11 with SMTP id ph11so73399312igc.1 for ; Mon, 16 Nov 2015 23:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=ByHKHu4ejnDY1O0kwAh5PZO3+2EgV/haiaUFcVzB3p0=; b=JHMb8M3XjTG+4cix5rtMA553eFRh6nkKchok7qb3j68cUcsNbjRJTVWv2YMZtWJbzi rc8NxdUHq3tIdQEO1UZMvT1g+F63auRpN2DVgzPIwoV5+5oa2DmrHTB8+W8HzCxweLY3 m9VQvRVYeQHdWf1deFJlE2cEvNfXDAJGfLHRGWf0Y7qqhmlzkUcKr3QqyE9jmGgJWpXj QERTdovhKyhKurVpE9luOEi+n+Z2FCZhbESV6FRWasShMfJ7fMqg/4E1ehP9gYZS9RYO VkKSwx/uIJGdbgMZZ/jLSycrZV4U/TeuNJHqscR4V/+Q1jlwFF3NPFahNjW9wOofsx26 GPSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=ByHKHu4ejnDY1O0kwAh5PZO3+2EgV/haiaUFcVzB3p0=; b=kgtxJl8cd8O/b1clHZKjCzd9xWqDeKXNLiQC1k5P8k8fT3EXsPiqUeuAegnh+2UeqI s89JRI1CTcoEk99GLCWWjq+gxlWhYSrsQwFIcjAJ64rqmHYRqlVwQQiOnwWr7Gwrj6Pu Lr0P/bbKnncWYtYLt91vcQLbSGbtuHLYfNbefv9tQNtTatI4lGj4Yo6dWfCZD/SL2t+K 8PePwNeSoizHijKkF6fFcmUsnWrIAPlbn+mp/7MSK74YwgtitGPBI0XVEf0GEPa60vM5 lBLLLM70ftnvb9QG+8tXY8Zc7WyZ17yNQlYkmCTAFOXgRq5FLS4ysgFMQl7xQrElCDfR kbjg== X-Gm-Message-State: ALoCoQnPTLaLQK6qG5ojSB5NCRAT9u+MsMFPzohIOifWpuNqCcd8jbhwvLs5NWMaFm2u0Ww7lrTe X-Received: by 10.50.59.211 with SMTP id b19mr399558igr.36.1447743998038; Mon, 16 Nov 2015 23:06:38 -0800 (PST) Received: from ?IPv6:2601:280:4900:3700:a49c:fa0f:af5a:6678? ([2601:280:4900:3700:a49c:fa0f:af5a:6678]) by smtp.gmail.com with ESMTPSA id ht2sm7658521igb.22.2015.11.16.23.06.36 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 16 Nov 2015 23:06:37 -0800 (PST) Sender: Warner Losh Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_FA5B2277-28B9-4431-A79E-40E7FA0627D3"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <564ACCB3.4070603@mu.org> Date: Tue, 17 Nov 2015 00:06:35 -0700 Cc: Elizabeth Myers , Anna Wilcox , "Brian McGovern (bmcgover)" , freebsd-arch , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard Message-Id: References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> <20151108155501.GA1901@alchemy.franken.de> <563F8385.3090603@freebsd.org> <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> <564ACCB3.4070603@mu.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 07:06:39 -0000 --Apple-Mail=_FA5B2277-28B9-4431-A79E-40E7FA0627D3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 16, 2015, at 11:44 PM, Alfred Perlstein wrote: >=20 > Warner, thanks for addressing this email. I think I wasn't clear = which lead to some misunderstanding. I'll keep this reply succinct and = the rest of it inline. Please don't take the succinctness as anything = other than getting to the point. >=20 > On 11/16/15 10:22 PM, Warner Losh wrote: >>=20 >>=20 >> On Mon, Nov 16, 2015 at 6:53 PM, Alfred Perlstein = wrote: >>=20 >>=20 >>=20 >> On 11/14/15 9:16 AM, Warner Losh wrote: >> On Fri, Nov 13, 2015 at 11:15 PM, Elizabeth Myers = >> wrote: >>=20 >> You are seriously going to use "we're not NetBSD" as an argument? >>=20 >> You noticed I didn't reply to it. The argument is completely lame. = FreeBSD >> runs >> today in a variety of markets. Some new, some not so new. The thing = that >> makes >> each of these areas unique is that there's a thriving community = around them, >> FreeBSD still runs well enough on these machines to get something = done, and >> when things break, they get fixed in a timely manner. >>=20 >> Alpha was removed because it got broken by some changes, and stayed = broken >> for a long time despite repeated requests to fix it. Sparc64 is on = the cusp >> of that: >> some minor things are broken, but have been fixed. The current crisis = is >> due to >> the end of life of gcc in the tree and its fallout coupled with some >> neglect of the >> port due to time constraints. >>=20 >> At first I was all for removal. With more data, I'm less sure. If the >> promises are kept >> made in this thread, it looks to remain viable for a while, though = the lack >> of a >> qemu-user solution means that packages for a slow platform (where = they are >> really quite useful) will remain limited. Maybe there's enough = hardware >> around >> that third-party pkg repos can fill the gap, maybe not. I think we = should >> experiment >> with this model and see what it produces. Give the branching of 11 as = the >> deadline >> to show something viable... >>=20 >> One of the things I never understood about FreeBSD's method of = maintaining a port was the way the platform porting was done. We really = do things in a different manner than what my perception of other OSes = is. >>=20 >> My impression (please do correct me if I'm wrong) was that other OSes = such as NetBSD and Linux had "platform maintainers". >>=20 >> These maintainers were around to: >> 1) keep the ship sailing on those platforms >> 2) guide the general code base from becoming non-portable to other = architectures (within reason). >> 3) drive the release of the architecture in question, helping the = release engineer with image building and release testing. >>=20 >> For point 1 above, what that meant to me was that let's say Linus or = NetBSD in general wanted to do a major or minor change on a tier 1 = platform, then it was the responsibility of the *platform maintainers* = to do the work on the non tier-1 platforms to keep them up to date. = Those "platform maintainers" kept those ships sailing and in return they = got to be called "the $arch maintainer" which looks plenty good on a = resume and also feels good for those that get excited for status. >>=20 >> I'm not sure how the people that actually take care of these things = on FreeBSD differ. There are people >> recognized as go-to people for the different ports that are fairly = active in the on-going issues that come >> up with kernel code. Userland code doesn't seem to matter that much = given the platforms we support. >>=20 >> For PowerPC, you have Nathan W and Justin Hibbits. For mips, there's = Adrian, myself, Julie Mallet and a few >> others. For arm there's a long cast of characters. For PC98 there's = Takahashi-san. For sparc64 there's Marius. >> These people keep the ship sailing (or in some cases they remove the = ship form the tree). They advise >> discussions about issues that are relevant to the platform, like = cache lines and cache coherence, >=20 > This is how we diverge: >> they call people out when they break these platforms or when people = used to big systems adjust the >> tuning and break small systems. > That is not done as far as I can tell in NetBSD/Linux. In = Linux/NetBSD it is the job of the maintainer to keep the platform up to = date, not to call out when someone breaks it. >=20 > Meaning ideal: > "Oh someone broke alignment in this struct on my platform, let me ask = them how to fix it." >=20 > As opposed to (not ideal): > "Something broke my platform, let me track that guy down and make them = fix it.=E2=80=9D The person that broke it is often the best person to fix it. If they = don=E2=80=99t fix it themselves the maintainers fix it for them. It=E2=80=99s common courtesy. I think = you=E2=80=99re making too fine a distinction here to actually be useful. And if you know anything about NetBSD, you=E2=80=99ll know they do = exactly the same thing when someone does something that breaks a particular platform. The port = maintainers, who build and run the stuff the most (though they aren=E2=80=99t listed = in any file) notice and complain, often times within hours of the commit. They generally ask the = original committer to fix it, just like we do. And just like we do, those = suggestions sometimes come in the form of a patch or a general description of what to do and = why. Tell me, who are the NetBSD/hpcmips maintainers? Who are the = NetBSD/hpcarm folks that are still active? I just did a search, and couldn=E2=80=99t = find this information. You could look at the hpcmips or hpcarm trees, but they are rather quiet = these last few years, and many of the folks that contributed code there have = wandered off. >>=20 >> For point 2, let's say someone had a change that pushed some form of = *completely* non-portable code into the base which would break a = reasonable to support platform, then the "platform maintainer" would = speak up and tell the general community "uh no, you can't do X on this = platform, we need to rethink this". >>=20 >> People generally don't push this kind of code into the tree these = days. When they do, they get called >> out on it. Some of them even listen to the calling out and fix = things, others don't and one of the >> platform maintainers has to fix the stupid pushed into the tree. = Sometimes this happens right away >> and sometimes there's a lag. sometimes it's code for newer versions = of the platform that break older >> versions (or vice versa). Other times there's code from another = platform that breaks things. >>=20 >> USB is a textbook example of this happening. It went in, and didn't = worth a damn on arm or mips. >> The ports maintainers of the arm and mips platforms tried to explain = what the issues were. It took >> some time, but it got mostly worked out as the embedded folks got to = know USB issues, and hps >> got to understand the issues with embedded hardware. >>=20 >> For point 3, there may be a lag between release of the OS for tier 1 = (x86/x64) and the secondary architectures, but that is OK because the = maintainer will eventually provide images themselves or in collaboration = with the release engineer. >>=20 >> For this point, we've pushed the knowledge of how to build the images = into the release engineer. The >> folks that are around that are using the port test the images. = Sometimes it's the port maintainers, >> but recently it has been a large cast of characters for popular = platforms. >>=20 >> FreeBSD seems to take a different approach. This approach is that = someone (or some people) form a team to port to a platform. These = "platform porter" groups sole responsibility is to get a new = architecture running. After it's mostly running they are mostly without = responsibility, however we tend to give them the right of change-set = veto in perpetuity of the marginal relevance of the ported to platform. >>=20 >> so like when did this actually happen? > Well, earlier in this email you said this exact thing: >=20 >> they call people out when they break these platforms or when people = used to big systems adjust the >> tuning and break small systems. > That's how I see the difference. First step is always education. That=E2=80=99s a strength, not a = weakness. You educate people that break things because more often than not, those people didn=E2=80=99t bother to ask = for a review. Part of the education is needing to make sure people social changes appropriately. If that=E2=80=99s *ALL* maintainers did, I=E2=80=99d agree with you. = However, there=E2=80=99s much proactive education that also happens, which is exactly your number 2: everybody working together to = make sure that new changes fit will with the platform set. That is real. It happens every day. = Focusing on only one, narrow situation that happens maybe once a month and saying we=E2=80=99re doing that = wrong seems petty and wrong-headed and would actually break more than it fixes if we changed it. So because you have a world view that doesn=E2=80=99t match what=E2=80=99s= going on, you want to change what=E2=80=99s going on to match some ideal from another project when this project is = actually doing that idea? I still don=E2=80=99t get it. >=20 >>=20 >> What this means is that instead of a assigning a title and ownership = of the platform to someone, who maintains the status as "maintainer" by = keeping that platform working. By keeping the platform working I am = saying that they would do items 1, 2 and 3 from the NetBSD/Linux list. = However, instead nearly immediately hoist the "platform maintenance" = onto the general community of people that may not have access to the = hardware in question. >>=20 >> Do you have a specific example of when we've done this? As far as I = know, based on powerpc, arm and >> mips anyway, the people claiming to be maintainers are actively doing = 1, 2 and helping RE do number >> 3 to varying degrees. As far as I know, they all have access to some = or all of the hardware they are >> maintaining, and many of our power users participate in the process = as well. >=20 >>=20 >> Maybe this is just my perception, but it would seem to make a ton = more sense to follow the NetBSD/Linux model which implies a somewhat = decoupled release model (not all arches must come out on the same ) and = assigning ownership and responsibility in exchange for status based on = being the "platform maintainer". >>=20 >> So, rather than generalizations, be specific. Who do you think is = claiming to be a port maintainer, blocking >> progress and needs to be replaced? >>=20 >> And what, beyond what the re@ does today, would you do differently? = What do we gain over what we do >> for tier 1 platforms? Is there a platform wanting a release that = isn't getting one? mips has two different >> groups that have put out releases for it, with one of them fading = into the background. Adiran is making >> wifi builds available, already following this model you say we should = adopt. The japanese user groups are >> putting out PC98 releases now that the re@ has dropped them (they = never really stopped in the mean >> time). sparc64, powerpc, arm, i386 and amd64 are all released by re@. = ia64 has been removed from the >> tree. What other platforms are there? What else needs to be done. >>=20 >> Finally it would be pretty obvious when everyone steps down or just = doesn't participate in the release process that it may be time to sunset = a platform. >>=20 >> That's why we are having the conversation about sparc64. It looked = like it might no longer be >> participating in the normal process. Now, while there are some issues = that were identified with >> sparc64, some of them are real (see qemu and difficulties building in = the cluster). Some of them >> were just perception (the reduced numbers of commits to sparc64 = didn't seem to represent >> a problem with the platform and the perceived issues had been cleared = up) >>=20 >> So what, specific, actionable items do we as a project actually need = to do here? I'm sure there are some >> and that we can improve our process. I'm having trouble teasing out = what I, as someone who dabbles >> in arm and mips to varying degrees of 'maintainership' for different = parts, can do better or different. >>=20 > Three things: > 1) I am wondering if core (or the community in general) should have = some way of nominating a particular person as a platform maintainer. = This would give accolades to that person and at the same time give us a = point person. I believe part of the problem is that we don't give = enough status to the port maintainers, are they on the website? How = would I know who is the "king of mips" right now? Does someone get to = put in on their resume with backing from the project? If not, then the = maintainer will be grumbly as opposed to facilitating. We don=E2=80=99t have a =E2=80=9Cking of the kernel=E2=80=9D or =E2=80=9Ck= ing of mips=E2=80=9D or anything like that. We document that you send = mail to mips@ when you don=E2=80=99t know the right person to send. As for status reports: I agree with you on that. As for putting things on one=E2=80=99s resume: I never needed the = project=E2=80=99s blessing to claim to have done a lot with = FreeBSD/mips, FreeBSD/arm, CardBus, PC Card, SD, MMC, PCI, etc. I just = did it. The lack of a stamp from the project hasn=E2=80=99t really been an issue. > 2) The role of platform maintainer needs not to be a blocking role, = but rather a continual porting role. Specifically to avoid the "calling = out" (sorry for the quotes here, but just want to get the point across) = and instead function as a do-er/facilitator. Meaning if something = breaks a secondary platform it's a shared responsibility of the port = maintainer and developer to fix it, but solely the developer. To the best of my knowledge, it isn=E2=80=99t a blocking role today. All = the people working on arm, mips and powerpc that I interact with already do that. Rather than fix the stupid mistake = people make, educating them not to make them again in the future is the best way to keep them from repeating. That = sure as hell sounds like a shared responsibility to me. I fail to see any evidence to support your assertion that port = maintainers just whine when things break. I don=E2=80=99t see that in the commit logs (although they are full of hundreds of = commits from people that do complain). I don=E2=80=99t see that in the interactions I=E2=80=99ve had. > 3) Tight coupling of the system. While it's good to cross train = release engineering with various platforms and even more so it's = REALLY great that this is being well received, the end result is a very = tight coupling of the system that can lead to frustrations and logjams = when things go wrong. Something that FreeBSD very often gets wrong is = the strong unification of its various parts, you can see this in so many = aspects of how we do things (releases, VFS, platforms) as opposed to = other projects. It is an admirable aspiration, however it results in = much lock-step of the entire project and doesn't scale. I can=E2=80=99t recall any release ever being held up for longer than it = takes to build on the slowest hardware the RE has used to cut the release. I can recall many times a release wasn=E2=80=99t = held, or I wasn=E2=80=99t allowed to put changes into a secondary platform because they might affect the primary one too close = to a release. Absent any evidence of when, where and how these things happen, I=E2=80=99m having a hard time = seeing that the actual situation on the ground matches the supposed one you are complaining about, at least as it comes = to other platforms. What logjams are they causing? I=E2=80=99ve not seen any. Do you have specific examples? Warner --Apple-Mail=_FA5B2277-28B9-4431-A79E-40E7FA0627D3 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWStH7AAoJEGwc0Sh9sBEAtKUQAMc5aDF/jmGZfV3/wwvQWdrD Kv34ABnKF2i2kpvW7ADWrLuetZ/JFmrKOX8Szru9JGDHF/Rd63vXxJuLjv5bX/5n sawlGo4MhNczAeqg2xmYUT2KXkNi9qTnX7DgNb960iLg7VM65PiTX2yj/CXoYGss Kc3b4AyCyv4TG3PYyHZj1WtIOdlNV/TVuCT1fcRitufItI4RbkkxKul4HuCZqN9u PNeh1eQ50gbJIEissRgLTLJKwhaeLs0Iqma6nGFbFUnTfeelCu+cgyhHjg6t4jQn wJaqzuEVIyIlhFhnO1z9NzmB0tezqiGfrPW7fgCAV2oiNPSWp5vhLiztpRK9DBRf oyhr2EHbZBuPnvhX7e5G869QkrAgLUAr0wARC+/KIjN+3cg2Vq2bS+NupiRTc9wc Qnq4OFt7Znkll+uMjuYtuNbLhY894WJ81dh8mUUbOItjkyOcExgzYnhIoRbacsU3 VQTsCfVW+UU/KVhwua/9OrxkF/n0GEq3lyjsbEt0fjDICltiWsuWCCISrZJS/RR/ gUPPbDrUjIsUSl/F7glr+iIVgJhMwnQGGia+SZ6tiA4uxKZWAMCx7tZ5a254UQfc 4O2fMyLsvexKDURCOvZcxu/TURVJ0THccq8/j4DpUdFoFqQCsstdtlbn4/KAtGix 14dWVaouwbiWgBjhJ9+E =GPrS -----END PGP SIGNATURE----- --Apple-Mail=_FA5B2277-28B9-4431-A79E-40E7FA0627D3-- From owner-freebsd-arch@freebsd.org Tue Nov 17 08:23:06 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 251F2A31DAC for ; Tue, 17 Nov 2015 08:23:06 +0000 (UTC) (envelope-from freebsd.contact@marino.st) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0DC071A28 for ; Tue, 17 Nov 2015 08:23:06 +0000 (UTC) (envelope-from freebsd.contact@marino.st) Received: by mailman.ysv.freebsd.org (Postfix) id 0C7A0A31DAB; Tue, 17 Nov 2015 08:23:06 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C1F1A31DAA for ; Tue, 17 Nov 2015 08:23:06 +0000 (UTC) (envelope-from freebsd.contact@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA9271A27; Tue, 17 Nov 2015 08:23:05 +0000 (UTC) (envelope-from freebsd.contact@marino.st) Received: by shepard.synsport.net (Postfix, from userid 80) id 23EAD43C0E; Tue, 17 Nov 2015 02:22:57 -0600 (CST) To: Andrey Chernov , Baptiste Daroussin Subject: Re: Question about ASCII and =?UTF-8?Q?nl=5Flanginfo=20=28locale?= =?UTF-8?Q?=20work=29?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 17 Nov 2015 09:22:57 +0100 From: "John Marino (FreeBSD)" Cc: Ed Schouten , arch@freebsd.org, marino@freebsd.org Reply-To: marino@freebsd.org Mail-Reply-To: marino@freebsd.org In-Reply-To: <564A4FE9.6000403@freebsd.org> References: <20151110222636.GN10134@ivaldir.etoilebsd.net> <564A27CD.7090908@freebsd.org> <20151116210659.GB59189@ivaldir.etoilebsd.net> <564A4FE9.6000403@freebsd.org> Message-ID: X-Sender: freebsd.contact@marino.st User-Agent: Roundcube Webmail/0.9.1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 08:23:06 -0000 On 11/16/2015 10:51 PM, Andrey Chernov wrote: > On 17.11.2015 0:06, Baptiste Daroussin wrote: >> locales the IANA way and are unhappy because that does not work. The >> first plan >> in the collation branch was to introduce the IANA syntax via an alias >> but in the >> end I removed it, because there was already to many changes. > > For ISO case we don't need aliases and can keep our internal names > hierarchy honoring POLA. All we need is: > 1) Convert "ISO-" and "ISO_" to "ISO" for setlocale(3) input. > 2) Convert from "ISO" to "ISO-" for setlocale(3), nl_langinfo(3) and > locale(1) output. A huge patch just went into GCC libstdc++ testsuite to change all the locale names to "ISO8859-" because it works for both Linux and *BSD. This is a change for changes sake. Locale -m lists the encodings. Locale -a lists the available locales This is true on Linux as well. Nobody is getting POLA'D here. Moveover, there is significant work to implement this. We brought up the possibility of hyphen- and case- sensitivity on DragonFly and the idea was shot down. The reasons were solid enough. There is no standard for encoding, period. Using one source is as valid another another. I say leave it alone. John From owner-freebsd-arch@freebsd.org Tue Nov 17 09:35:36 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6023A30BEF; Tue, 17 Nov 2015 09:35:36 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [66.135.54.68]) by mx1.freebsd.org (Postfix) with ESMTP id C33AB1C92; Tue, 17 Nov 2015 09:35:36 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: by mail.soaustin.net (Postfix, from userid 502) id F1BF25607A; Tue, 17 Nov 2015 03:27:03 -0600 (CST) Date: Tue, 17 Nov 2015 03:27:03 -0600 From: Mark Linimon To: Warner Losh Cc: Alfred Perlstein , Anna Wilcox , "Brian McGovern (bmcgover)" , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , freebsd-arch , Jordan Hubbard , Elizabeth Myers Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 Message-ID: <20151117092703.GA30582@lonesome.com> References: <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> <564ACCB3.4070603@mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Mailman-Approved-At: Tue, 17 Nov 2015 12:16:06 +0000 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 09:35:37 -0000 On Tue, Nov 17, 2015 at 12:06:35AM -0700, Warner Losh wrote: > We don’t have a “king of the kernel” or “king of mips” or anything > like that. We document that you send mail to mips@ when you don’t > know the right person to send. And I don't see why we should. What would we gain from this administrative overhead? What we lose is: dealing with people who have the title not doing work, or people not having the title, doing the work, being irritated by not having the title. svn can, with a little effort, demonstrate who is looking after what. All I see is cost and no benefit. This isn't an "org chart" kind of issue IMHO. In FreeBSD, why delegate responsibility to anyone other than those who take it by their own initiative? mcl From owner-freebsd-arch@freebsd.org Tue Nov 17 16:46:30 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F3C3A31CD0 for ; Tue, 17 Nov 2015 16:46:30 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DF0E21EF7 for ; Tue, 17 Nov 2015 16:46:29 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id DE1B8A31CCE; Tue, 17 Nov 2015 16:46:29 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDB57A31CCD for ; Tue, 17 Nov 2015 16:46:29 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE0D31EF6; Tue, 17 Nov 2015 16:46:29 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by pacdm15 with SMTP id dm15so13702421pac.3; Tue, 17 Nov 2015 08:46:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=uFRp4vO/FbrgDvO8aUqergL18EuaRd8eIHRoAUoBMDk=; b=hufDPkBhAvQJzA7QN9jhfp3op2w1dCQ0GbQTiJAo54g3tNoq6GCNPziQBiFtS/j8c7 ijG9v5So2U5jjl4Lh0nbvAV0dAPWMuPXcusX0/yeDRh3pDCv+517KGow6zDRPku0pxn3 WeyuEx/pIdeOgcq0p7fnj880MHg8mK/mZulI+vgWJH6lzZUYBlUzBopNYu6G1HMPqQBU 6TSvppq2mNBbI02EFhRFw7BqheXvHKrOo0ZEMhjdXZr4/TVoE1Ec9ncSEDHCqUWzENwu 7xU/W3B3RsN2aYe0CPAXT2PDbBaEPjDWpDi3g94L6tSfZQVhbqYlU5+04G67xLXUCLE+ YFpw== X-Received: by 10.68.65.72 with SMTP id v8mr63774967pbs.142.1447778789260; Tue, 17 Nov 2015 08:46:29 -0800 (PST) Received: from [192.168.20.11] (c-24-16-212-205.hsd1.wa.comcast.net. [24.16.212.205]) by smtp.gmail.com with ESMTPSA id fe6sm33433581pab.40.2015.11.17.08.46.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 17 Nov 2015 08:46:27 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Question about ASCII and nl_langinfo (locale work) From: Garrett Cooper X-Mailer: iPhone Mail (13B143) In-Reply-To: Date: Tue, 17 Nov 2015 08:46:26 -0800 Cc: Andrey Chernov , Baptiste Daroussin , arch@freebsd.org, Ed Schouten Content-Transfer-Encoding: quoted-printable Message-Id: <05BE03FC-365D-46BF-9CB9-A83917784F20@gmail.com> References: <20151110222636.GN10134@ivaldir.etoilebsd.net> <564A27CD.7090908@freebsd.org> <20151116210659.GB59189@ivaldir.etoilebsd.net> <564A4FE9.6000403@freebsd.org> To: marino@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 16:46:30 -0000 > On Nov 17, 2015, at 00:22, John Marino (FreeBSD) wrote: >=20 >> On 11/16/2015 10:51 PM, Andrey Chernov wrote: >>> On 17.11.2015 0:06, Baptiste Daroussin wrote: >>> locales the IANA way and are unhappy because that does not work. The fir= st plan >>> in the collation branch was to introduce the IANA syntax via an alias bu= t in the >>> end I removed it, because there was already to many changes. >> For ISO case we don't need aliases and can keep our internal names >> hierarchy honoring POLA. All we need is: >> 1) Convert "ISO-" and "ISO_" to "ISO" for setlocale(3) input. >> 2) Convert from "ISO" to "ISO-" for setlocale(3), nl_langinfo(3) and >> locale(1) output. >=20 > A huge patch just went into GCC libstdc++ testsuite to change all the > locale names to "ISO8859-" because it works for both Linux and *BSD. >=20 > This is a change for changes sake. >=20 > Locale -m lists the encodings. > Locale -a lists the available locales >=20 > This is true on Linux as well. > Nobody is getting POLA'D here. >=20 > Moveover, there is significant work to implement this. We brought up > the possibility of hyphen- and case- sensitivity on DragonFly and the > idea was shot down. The reasons were solid enough. >=20 > There is no standard for encoding, period. Using one source is as valid > another another. I say leave it alone. Windows is probably the closest thing to a standard here. What does it use -= - dashes or underscores? Thanks, -NGie= From owner-freebsd-arch@freebsd.org Tue Nov 17 22:45:09 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB3DBA3112F for ; Tue, 17 Nov 2015 22:45:09 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E6361978; Tue, 17 Nov 2015 22:45:08 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.15.2/8.15.2/ALCHEMY.FRANKEN.DE) with ESMTPS id tAHMj5Jl085194 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 17 Nov 2015 23:45:05 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.15.2/8.15.2/Submit) id tAHMj5sL085193; Tue, 17 Nov 2015 23:45:05 +0100 (CET) (envelope-from marius) Date: Tue, 17 Nov 2015 23:45:05 +0100 From: Marius Strobl To: John Baldwin Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Message-ID: <20151117224505.GA85136@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <5385051.zAN7Yc63R0@ralph.baldwin.cx> <20151116230439.GA77914@alchemy.franken.de> <5992121.1Qh8fceFnn@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5992121.1Qh8fceFnn@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (alchemy.franken.de [0.0.0.0]); Tue, 17 Nov 2015 23:45:05 +0100 (CET) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 22:45:09 -0000 On Mon, Nov 16, 2015 at 04:37:32PM -0800, John Baldwin wrote: > On Tuesday, November 17, 2015 12:04:39 AM Marius Strobl wrote: > > On Fri, Nov 13, 2015 at 11:50:37AM -0800, John Baldwin wrote: > > > On Friday, November 13, 2015 12:41:46 AM Marius Strobl wrote: > > > > On Thu, Nov 12, 2015 at 02:36:42PM -0800, John Baldwin wrote: > > > > > On Monday, August 31, 2015 02:21:19 PM John Baldwin wrote: > > > > > > On Wednesday, August 12, 2015 10:50:20 AM John Baldwin wrote: > > > > > > > On Tuesday, August 04, 2015 10:56:09 AM John Baldwin wrote: > > > > > > > > Many debuggers (recent gdb and lldb) support cross-architecture debugging > > > > > > > > just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for > > > > > > > > remote targets already, but I wanted it to also support cross-debugging for > > > > > > > > vmcores. > > > > > > > > > > > > > > > > The existing libkvm/kgdb code in the tree has some limited support for > > > > > > > > cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a) > > > > > > > > and custom kgdb for each target platform. However, gdb (and lldb) both > > > > > > > > support multiple targets in a single binary, so I'd like to have a single > > > > > > > > kgdb binary that can cross-debug anything. > > > > > > > > > > > > > > > > I started hacking on libkvm last weekend and have a prototype that I've used > > > > > > > > (along with some patches to my kgdb port) to debug an amd64 vmcore on an > > > > > > > > i386 machine and vice versa. > > > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > What I'm mostly after is comments on the API, etc. Once that is settled I > > > > > > > > will move forward on converting and/or stubbing the other backends (the > > > > > > > > stub route would be to only support other backends on native systems for > > > > > > > > now). > > > > > > > > > > > > > > I guess this is closer to a nuclear power plant than a bikeshed judging by the > > > > > > > feedback. I have ported the rest of the MD backends and verified that the > > > > > > > updated libkvm passes a universe build (including various static assertions > > > > > > > for the duplicated constants in other backends). What I have not done is any > > > > > > > runtime testing and I would like to ask for help with that now. In particular > > > > > > > I need someone to test that kgdb and/or ps works against a native core dump > > > > > > > on all platforms other than amd64 and i386. Note that some of the trickiness > > > > > > > is that the backends now have to make runtime decisions for things that were > > > > > > > previously compile-time decisions. The biggest one affected by this is the > > > > > > > MIPS backend as that backend handles three ABIs (mipso32, mipsn32, and mipsn64). > > > > > > > I believe I have the handling for that correct (mips[on]32 use 32-bit KSEGs > > > > > > > where as mipsn64 uses the extended segments and compat32 KSEGS, and mipso32 > > > > > > > uses 32-bit PTEs and mipsn32/n64 both use 64-bit PTEs) (plus both endians > > > > > > > for both in theory). The ARM backend also handles both endians (in theory). > > > > > > > > > > > > > > Another wrinkle is that sparc64 uses its own dump format instead of writing > > > > > > > out an ELF file. I had to convert the header structures to use fixed-width > > > > > > > types to be cross-friendly. It would be good to ensure that a new libkvm > > > > > > > can read a vmcore from an old kernel and vice versa to make sure my conversion > > > > > > > is correct (I added an explicit padding field that I believe was implicit > > > > > > > before). > > > > > > > > > > > > > > The code is currently available for review in phabric at > > > > > > > https://reviews.freebsd.org/D3341 > > > > > > > > > > > > > > To test, you can run 'arc patch D3341' in a clean tree to apply the patch. > > > > > > > > > > > > I've just rebased this to port aarch64's minidump support. I just need people > > > > > > willing and able to test on non-x86. Testing with the in-tree kgdb using an > > > > > > updated libkvm would be sufficient. > > > > > > > > > > After a lot of crickets, I have updated the manpages for the new API. I will > > > > > commit this "soon". If you want kgdb to keep working on your non-x86 > > > > > platform, this is your chance to test this before it hits the tree. > > > > > > > > > > > > > What exact test procedure do you suggest for full coverage of an > > > > architecture? > > > > > > Just ensuring that kgdb and things like ps -M -N still work. > > > > With the patch from D3341 applied, kgdb(1) still seems to work fine on > > sparc64. However, `ps -M -N ` doesn't; it just prints > > the header and then exists after a short pause. Using the same core and > > kernel with ps(1) on a machine with userland built without your patch, > > ps(1) just segfaults after a short period of time. I can't tell whether > > that's a regression or not as I've never used ps(1) on a core before > > and you also have added padding to struct sparc64_dump_hdr, which might > > be responsible for triggering the segfault. On the other hand, an old > > kgb(1) seemingly works fine with the new core. > > Hmm, I had thought that the old and new sparc64_dump_hdr would be the > same? I was just using fixed width types so that any platform could > #include the header and get the same layout. In particular, I don't > want the dump format to change on disk after this change so that once > kgdb (or lldb) has cross-debugging support we can read both old and > new sparc64 vmcores. Yes, you are right; you added dh_pad to struct sparc64_dump_hdr but that doesn't change its size or the native offsets of the members. > > Also, parallel builds failed with something not finding libelf but > > building with a single jobs succeeded. I don't know whether D3341 > > introduces that or if it's a bug in head (the latter probably is > > unlikely but I didn't investigate). > > Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox > builds did not trip over that, and lib/Makefile has libelf in its > "early" list of libraries (SUBDIR_ORDERED), so it seems like it should > be built before libkvm is tried? Well, I'd agree in principle but also just can say that -j16 builds reliably fail here: --- lib/libkvm__L --- /home/marius/co/build/head3/i386.i386/usr/home/marius/co/head3/src/tmp/usr/bin/ ld: cannot find -lelf cc: error: linker command failed with exit code 1 (use -v to see invocation) *** [libkvm.so.6] Error code 1 Marius From owner-freebsd-arch@freebsd.org Wed Nov 18 19:33:06 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09BD8A32C2E for ; Wed, 18 Nov 2015 19:33:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DBF411D84 for ; Wed, 18 Nov 2015 19:33:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EFBC4B915; Wed, 18 Nov 2015 14:33:04 -0500 (EST) From: John Baldwin To: Marius Strobl Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Date: Wed, 18 Nov 2015 11:32:57 -0800 Message-ID: <2820062.CW4ER9qaIh@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20151117224505.GA85136@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <5992121.1Qh8fceFnn@ralph.baldwin.cx> <20151117224505.GA85136@alchemy.franken.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 18 Nov 2015 14:33:05 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 19:33:06 -0000 On Tuesday, November 17, 2015 11:45:05 PM Marius Strobl wrote: > On Mon, Nov 16, 2015 at 04:37:32PM -0800, John Baldwin wrote: > > On Tuesday, November 17, 2015 12:04:39 AM Marius Strobl wrote: > > > On Fri, Nov 13, 2015 at 11:50:37AM -0800, John Baldwin wrote: > > > Also, parallel builds failed with something not finding libelf but > > > building with a single jobs succeeded. I don't know whether D3341 > > > introduces that or if it's a bug in head (the latter probably is > > > unlikely but I didn't investigate). > > > > Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox > > builds did not trip over that, and lib/Makefile has libelf in its > > "early" list of libraries (SUBDIR_ORDERED), so it seems like it should > > be built before libkvm is tried? > > Well, I'd agree in principle but also just can say that -j16 builds > reliably fail here: > --- lib/libkvm__L --- > /home/marius/co/build/head3/i386.i386/usr/home/marius/co/head3/src/tmp/usr/bin/ > ld: cannot find -lelf > cc: error: linker command failed with exit code 1 (use -v to see invocation) > *** [libkvm.so.6] Error code 1 Ok. I'll see if I can't reproduce that on a test machine myself. -- John Baldwin From owner-freebsd-arch@freebsd.org Wed Nov 18 21:38:12 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F01E5A32218 for ; Wed, 18 Nov 2015 21:38:11 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-io0-f175.google.com (mail-io0-f175.google.com [209.85.223.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE59B189A for ; Wed, 18 Nov 2015 21:38:11 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: by iofh3 with SMTP id h3so68830074iof.3 for ; Wed, 18 Nov 2015 13:38:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:date:message-id:subject:from:to:content-type; bh=BKW84/cUNzJi0CRehHT3eMIgvtCQ1r2HrjCg6amX7DA=; b=f9uEEQFoaDHTe+uMsJb7U5CcZaxh4rOL25bHmOXdCqR9dG8CQFoLT/gg5xASz1EMeC YYRdfMd5e9VkyUmOGN+fR3HPJIy5HCRf0cBKRQKKr0jJK5H7aON/pWaTeRp0rPRN1aEW F/FI/uKIu777sqbW6x2tZoGFZjgpeLElp1SbJe+9ysgS+b1RnmMH3v4KAh2gwiqab10O S5Nhimea0fer6tkMctp7oyH+5BlwB5ELlkYQCR/wu+MxWuyXoak8+cXGDKaaedjqBIBQ meDANvpdbca9M4JYk7wT+4eZNt0D37tD992Ok89PO0woN9ad7Hkv/56WYSdUmM8p+6wB a2eQ== X-Received: by 10.107.4.213 with SMTP id 204mr4103368ioe.195.1447862450668; Wed, 18 Nov 2015 08:00:50 -0800 (PST) Received: from mail-ig0-f174.google.com (mail-ig0-f174.google.com. [209.85.213.174]) by smtp.gmail.com with ESMTPSA id wc5sm11840774igb.1.2015.11.18.08.00.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Nov 2015 08:00:50 -0800 (PST) Received: by igvg19 with SMTP id g19so120479115igv.1 for ; Wed, 18 Nov 2015 08:00:49 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.128.194 with SMTP id nq2mr3534476igb.19.1447862449706; Wed, 18 Nov 2015 08:00:49 -0800 (PST) Received: by 10.64.130.38 with HTTP; Wed, 18 Nov 2015 08:00:49 -0800 (PST) Date: Wed, 18 Nov 2015 17:00:49 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: a question about BUS_DMA_MIN_ALLOC_COMP flag meaning From: Svatopluk Kraus To: FreeBSD Arch , Konstantin Belousov Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 21:38:12 -0000 Hi, I have fallen to some problem with inconsistent use of BUS_DMA_MIN_ALLOC_COMP flag. This flag was introduced in x86 MD code very very long ago and so, the problem covers all archs which came out from it. However, it's only about bus_dma_tag_t with BUS_DMA_COULD_BOUNCE flag set. (1) When bus_dma_tag_t is being created with BUS_DMA_ALLOCNOW flag specified, some bounce pages could be allocated in advance and BUS_DMA_MIN_ALLOC_COMP flag is set to the tag. The bounce pages are allocated only if the tag's maxsize property is higher than size of all bounce pages already allocated in a bounce zone. (2) When bus_dmamap_t is being created, then if BUS_DMA_MIN_ALLOC_COMP is not set on associated tag, some bounce pages are ALWAYS allocated and BUS_DMA_MIN_ALLOC_COMP is set afterwards, (3) else some bounce pages could be allocated if there is not enough pages in a bounce zone and BUS_DMA_MIN_ALLOC_COMP is set afterwards. The problem is the following. Due to case (2), the number of pages in bounce zone can grow infinitely, as bounce pages once allocated are never freed. It can happen when a big number of bus_dma_tag_t together with bus_dmamap_t are created, or they are created dynamically either because of a loadable module or by design. The inconsistency is that when bus_dma_tag_t is being created, there is no limit for how much pages could be allocated. On the other hand, when bus_dmamap_t is being created, there is MAX_BPAGES limitation. I think that fix for case (2) presented as x86 fix is the following: diff --git a/sys/x86/x86/busdma_bounce.c b/sys/x86/x86/busdma_bounce.c index 4826a2b..a15139f 100644 --- a/sys/x86/x86/busdma_bounce.c +++ b/sys/x86/x86/busdma_bounce.c @@ -308,7 +308,7 @@ bounce_bus_dmamap_create(bus_dma_tag_t dmat, int flags, bus_dmamap_t *mapp) else maxpages = MIN(MAX_BPAGES, Maxmem - atop(dmat->common.lowaddr)); - if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 || + if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 && (bz->map_count > 0 && bz->total_bpages < maxpages)) { pages = MAX(atop(dmat->common.maxsize), 1); pages = MIN(maxpages - bz->total_bpages, pages); IMO, it also fixes logic by making it same as in bus_dma_tag_t case. The next question is, if case (1) should be limited by MAX_BPAGES as in case (3) or maybe better if there should be some internal limitation for bounce zone itself. As this is quite old code which covers many archs, I really would appreciate your comments. Thanks, Svatopluk Kraus From owner-freebsd-arch@freebsd.org Thu Nov 19 19:16:33 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89B6DA33FD3 for ; Thu, 19 Nov 2015 19:16:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 667CD19F4 for ; Thu, 19 Nov 2015 19:16:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A3AEBB946; Thu, 19 Nov 2015 14:16:30 -0500 (EST) From: John Baldwin To: Marius Strobl Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Date: Thu, 19 Nov 2015 11:16:24 -0800 Message-ID: <2429833.yYfvNJzKe9@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20151117224505.GA85136@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <5992121.1Qh8fceFnn@ralph.baldwin.cx> <20151117224505.GA85136@alchemy.franken.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 19 Nov 2015 14:16:30 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 19:16:33 -0000 On Tuesday, November 17, 2015 11:45:05 PM Marius Strobl wrote: > On Mon, Nov 16, 2015 at 04:37:32PM -0800, John Baldwin wrote: > > Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox > > builds did not trip over that, and lib/Makefile has libelf in its > > "early" list of libraries (SUBDIR_ORDERED), so it seems like it should > > be built before libkvm is tried? > > Well, I'd agree in principle but also just can say that -j16 builds > reliably fail here: > --- lib/libkvm__L --- > /home/marius/co/build/head3/i386.i386/usr/home/marius/co/head3/src/tmp/usr/bin/ > ld: cannot find -lelf > cc: error: linker command failed with exit code 1 (use -v to see invocation) > *** [libkvm.so.6] Error code 1 I found this. There are three(!) places I've had to annotate the libkvm now depends on libelf though it seems only one of them is actually used by buildworld (and that was the one I had missed). -- John Baldwin From owner-freebsd-arch@freebsd.org Thu Nov 19 21:46:52 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A2B1A332DF; Thu, 19 Nov 2015 21:46:52 +0000 (UTC) (envelope-from superbisquit@gmail.com) Received: from mail-yk0-x244.google.com (mail-yk0-x244.google.com [IPv6:2607:f8b0:4002:c07::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE4151584; Thu, 19 Nov 2015 21:46:51 +0000 (UTC) (envelope-from superbisquit@gmail.com) Received: by ykay124 with SMTP id y124so10460745yka.1; Thu, 19 Nov 2015 13:46:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SSJr9N6hq7Yewpo2UKh8aO11czqq6LSHUQ7o/OcIcJQ=; b=My6fObmneB46Jc5JgvmvwwVEMrDqFZyewjj0Cws6sUUua3uOW1n8LXLQE6dUfHsRWb Vsrf+zwkXU97oCHnvhcganoc5HQwGaQQmiMrK6iz8GTVLlbDAZzDYjbsjHEjy+twLxSS qeBuxON0fPGDwwPW0INfKR3m90nwxKeYsc3UilzzHtTpkNWgUvMZtsgVgGIOssFxtjVo EsJHDACXy/S0t1dxuApMEWTCy1RIKFGeteBj5R01vNMW2kFNoSHJL79DWqnROvZnM8Ix xqHyXBndnqNznTit0volF/cLXr+NetVcqrkeRHDljUwvIE2QaoU77rLtiTevpZ2FQ8zK CjqA== MIME-Version: 1.0 X-Received: by 10.129.53.132 with SMTP id c126mr9276369ywa.108.1447969610779; Thu, 19 Nov 2015 13:46:50 -0800 (PST) Received: by 10.103.9.195 with HTTP; Thu, 19 Nov 2015 13:46:50 -0800 (PST) In-Reply-To: <20151117092703.GA30582@lonesome.com> References: <56417100.5050600@Wilcox-Tech.com> <39947478-4710-47D8-BAB1-FC93979570B6@mail.turbofuzz.com> <5646D19C.9010304@interlinked.me> <564A889C.9070209@mu.org> <564ACCB3.4070603@mu.org> <20151117092703.GA30582@lonesome.com> Date: Thu, 19 Nov 2015 16:46:50 -0500 Message-ID: Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 From: Joe Nosay To: Mark Linimon Cc: Warner Losh , Anna Wilcox , freebsd-arch , Alfred Perlstein , Marius Strobl , Sean Bruno , "sparc64@freebsd.org" , Jordan Hubbard , Elizabeth Myers X-Mailman-Approved-At: Thu, 19 Nov 2015 22:45:33 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 21:46:52 -0000 Is it possible to have the instruction sets moved from one architecture to another? Would this require more registers? Since there are Open Architectures, it should be possible to create a CPU with shared instruction sets. On Tue, Nov 17, 2015 at 4:27 AM, Mark Linimon wrote: > On Tue, Nov 17, 2015 at 12:06:35AM -0700, Warner Losh wrote: > > We don=E2=80=99t have a =E2=80=9Cking of the kernel=E2=80=9D or =E2=80= =9Cking of mips=E2=80=9D or anything > > like that. We document that you send mail to mips@ when you don=E2=80= =99t > > know the right person to send. > > And I don't see why we should. What would we gain from this administrati= ve > overhead? > > What we lose is: dealing with people who have the title not doing work, > or people not having the title, doing the work, being irritated by not > having the title. > > svn can, with a little effort, demonstrate who is looking after what. > > All I see is cost and no benefit. This isn't an "org chart" kind of > issue IMHO. > > In FreeBSD, why delegate responsibility to anyone other than those who > take it by their own initiative? > > mcl > _______________________________________________ > freebsd-sparc64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-sparc64 > To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@freebsd.org= " > From owner-freebsd-arch@freebsd.org Fri Nov 20 01:56:26 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47C1BA3267C for ; Fri, 20 Nov 2015 01:56:26 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D38231432; Fri, 20 Nov 2015 01:56:25 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.15.2/8.15.2/ALCHEMY.FRANKEN.DE) with ESMTPS id tAK1uLCn027051 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Nov 2015 02:56:21 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.15.2/8.15.2/Submit) id tAK1uLMA027050; Fri, 20 Nov 2015 02:56:21 +0100 (CET) (envelope-from marius) Date: Fri, 20 Nov 2015 02:56:21 +0100 From: Marius Strobl To: John Baldwin Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Message-ID: <20151120015621.GQ31931@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <5992121.1Qh8fceFnn@ralph.baldwin.cx> <20151117224505.GA85136@alchemy.franken.de> <2429833.yYfvNJzKe9@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2429833.yYfvNJzKe9@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (alchemy.franken.de [0.0.0.0]); Fri, 20 Nov 2015 02:56:21 +0100 (CET) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 01:56:26 -0000 On Thu, Nov 19, 2015 at 11:16:24AM -0800, John Baldwin wrote: > On Tuesday, November 17, 2015 11:45:05 PM Marius Strobl wrote: > > On Mon, Nov 16, 2015 at 04:37:32PM -0800, John Baldwin wrote: > > > Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox > > > builds did not trip over that, and lib/Makefile has libelf in its > > > "early" list of libraries (SUBDIR_ORDERED), so it seems like it should > > > be built before libkvm is tried? > > > > Well, I'd agree in principle but also just can say that -j16 builds > > reliably fail here: > > --- lib/libkvm__L --- > > /home/marius/co/build/head3/i386.i386/usr/home/marius/co/head3/src/tmp/usr/bin/ > > ld: cannot find -lelf > > cc: error: linker command failed with exit code 1 (use -v to see invocation) > > *** [libkvm.so.6] Error code 1 > > I found this. There are three(!) places I've had to annotate the libkvm now > depends on libelf though it seems only one of them is actually used by > buildworld (and that was the one I had missed). > Does this mean that the .WAITs in SUBDIR_ORDERED of lib/Makefile don't have the desired effect? I see other build failures which suggest that other .WAITs in tree just don't work as expected. Usually I only hit these with -j128 or higher, though. Marius From owner-freebsd-arch@freebsd.org Fri Nov 20 14:45:53 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DC9DA3440B for ; Fri, 20 Nov 2015 14:45:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD9221AD0; Fri, 20 Nov 2015 14:45:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tAKEjju6085936 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 20 Nov 2015 16:45:45 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tAKEjju6085936 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tAKEjidJ085934; Fri, 20 Nov 2015 16:45:44 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 20 Nov 2015 16:45:44 +0200 From: Konstantin Belousov To: Svatopluk Kraus Cc: FreeBSD Arch Subject: Re: a question about BUS_DMA_MIN_ALLOC_COMP flag meaning Message-ID: <20151120144544.GB58629@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 14:45:53 -0000 On Wed, Nov 18, 2015 at 05:00:49PM +0100, Svatopluk Kraus wrote: > Hi, > > I have fallen to some problem with inconsistent use of > BUS_DMA_MIN_ALLOC_COMP flag. This flag was introduced in x86 MD code > very very long ago and so, the problem covers all archs which came out > from it. > > However, it's only about bus_dma_tag_t with BUS_DMA_COULD_BOUNCE flag set. > > (1) When bus_dma_tag_t is being created with BUS_DMA_ALLOCNOW flag > specified, some bounce pages could be allocated in advance and > BUS_DMA_MIN_ALLOC_COMP flag is set to the tag. The bounce pages are > allocated only if the tag's maxsize property is higher than size of > all bounce pages already allocated in a bounce zone. > > (2) When bus_dmamap_t is being created, then if BUS_DMA_MIN_ALLOC_COMP > is not set on associated tag, some bounce pages are ALWAYS allocated > and BUS_DMA_MIN_ALLOC_COMP is set afterwards, > > (3) else some bounce pages could be allocated if there is not enough > pages in a bounce zone and BUS_DMA_MIN_ALLOC_COMP is set afterwards. > > The problem is the following. Due to case (2), the number of pages in > bounce zone can grow infinitely, as bounce pages once allocated are > never freed. It can happen when a big number of bus_dma_tag_t together > with bus_dmamap_t are created, or they are created dynamically either > because of a loadable module or by design. > > The inconsistency is that when bus_dma_tag_t is being created, there > is no limit for how much pages could be allocated. On the other hand, > when bus_dmamap_t is being created, there is MAX_BPAGES limitation. > > I think that fix for case (2) presented as x86 fix is the following: > > diff --git a/sys/x86/x86/busdma_bounce.c b/sys/x86/x86/busdma_bounce.c > index 4826a2b..a15139f 100644 > --- a/sys/x86/x86/busdma_bounce.c > +++ b/sys/x86/x86/busdma_bounce.c > @@ -308,7 +308,7 @@ bounce_bus_dmamap_create(bus_dma_tag_t dmat, int > flags, bus_dmamap_t *mapp) > else > maxpages = MIN(MAX_BPAGES, Maxmem - > atop(dmat->common.lowaddr)); > - if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 || > + if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 && > (bz->map_count > 0 && bz->total_bpages < maxpages)) { > pages = MAX(atop(dmat->common.maxsize), 1); > pages = MIN(maxpages - bz->total_bpages, pages); > > > IMO, it also fixes logic by making it same as in bus_dma_tag_t case. I think that this patch is correct. > > The next question is, if case (1) should be limited by MAX_BPAGES as > in case (3) or maybe better if there should be some internal > limitation for bounce zone itself. Could we apply e.g. MAX_BPAGES limit to bounce zones, or should we allow the limit to change based on the tag ? But I am not sure if there is any reasonable way to formulate the limit. MAX_BPAGES looks like some arbitrary sanity limit, e.g. we could have unlimited maxsize, but also have an alignment constraints, and then tag requires bouncing. I am not sure that hard-coded values, esp. the amd64 32MB limit, makes much sense, or that basing the limit on the tag constraints makes more sense. Might be, we should allow some total percentage of the physical memory on machine to be consumed by all bounce zones altogether ? From owner-freebsd-arch@freebsd.org Fri Nov 20 17:28:47 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D654A337C8 for ; Fri, 20 Nov 2015 17:28:47 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 58B5B1382 for ; Fri, 20 Nov 2015 17:28:47 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6BF1AB9A9; Fri, 20 Nov 2015 12:28:46 -0500 (EST) From: John Baldwin To: Marius Strobl Cc: freebsd-arch@freebsd.org Subject: Re: Supporting cross-debugging vmcores in libkvm (Testing needed) Date: Fri, 20 Nov 2015 08:27:01 -0800 Message-ID: <10500589.8oRBRieQhc@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20151120015621.GQ31931@alchemy.franken.de> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> <2429833.yYfvNJzKe9@ralph.baldwin.cx> <20151120015621.GQ31931@alchemy.franken.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 20 Nov 2015 12:28:46 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 17:28:47 -0000 On Friday, November 20, 2015 02:56:21 AM Marius Strobl wrote: > On Thu, Nov 19, 2015 at 11:16:24AM -0800, John Baldwin wrote: > > On Tuesday, November 17, 2015 11:45:05 PM Marius Strobl wrote: > > > On Mon, Nov 16, 2015 at 04:37:32PM -0800, John Baldwin wrote: > > > > Hmm, it is true that libkvm now depends on libelf. My -j 16 tinderbox > > > > builds did not trip over that, and lib/Makefile has libelf in its > > > > "early" list of libraries (SUBDIR_ORDERED), so it seems like it should > > > > be built before libkvm is tried? > > > > > > Well, I'd agree in principle but also just can say that -j16 builds > > > reliably fail here: > > > --- lib/libkvm__L --- > > > /home/marius/co/build/head3/i386.i386/usr/home/marius/co/head3/src/tmp/usr/bin/ > > > ld: cannot find -lelf > > > cc: error: linker command failed with exit code 1 (use -v to see invocation) > > > *** [libkvm.so.6] Error code 1 > > > > I found this. There are three(!) places I've had to annotate the libkvm now > > depends on libelf though it seems only one of them is actually used by > > buildworld (and that was the one I had missed). > > > > Does this mean that the .WAITs in SUBDIR_ORDERED of lib/Makefile > don't have the desired effect? I see other build failures which > suggest that other .WAITs in tree just don't work as expected. > Usually I only hit these with -j128 or higher, though. They work if you do 'make' in lib. But buildworld does 'make libraries' and then later does 'make' in lib, and the 'make libraries' step uses a separate set of variables (_prereq_libs, _startup_libs, etc.) defined in Makefile.inc1 to set the order of 'make libraries'. :-( -- John Baldwin