From owner-freebsd-arch@freebsd.org Sun Aug 27 06:32:46 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E13ACDE89F8 for ; Sun, 27 Aug 2017 06:32:46 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail107.syd.optusnet.com.au (mail107.syd.optusnet.com.au [211.29.132.53]) by mx1.freebsd.org (Postfix) with ESMTP id A5139676D1; Sun, 27 Aug 2017 06:32:46 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail107.syd.optusnet.com.au (Postfix) with ESMTPS id 7AE24D42B12; Sun, 27 Aug 2017 16:32:44 +1000 (AEST) Date: Sun, 27 Aug 2017 16:32:43 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ian Lepore cc: "Rodney W. Grimes" , Bruce Evans , Don Lewis , avg@freebsd.org, freebsd-arch@freebsd.org Subject: Re: ULE steal_idle questions In-Reply-To: <1503771494.56799.49.camel@freebsd.org> Message-ID: <20170827160115.B2471@besplex.bde.org> References: <201708261812.v7QIC2eJ074443@pdx.rh.CN85.dnsmgr.net> <1503771494.56799.49.camel@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=KeqiiUQD c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=v5YSMayklnJFhNnzUYgA:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Aug 2017 06:32:47 -0000 On Sat, 26 Aug 2017, Ian Lepore wrote: > On Sat, 2017-08-26 at 11:12 -0700, Rodney W. Grimes wrote: >>> >>> On Fri, 25 Aug 2017, Don Lewis wrote: >>> [... context mostly lost to mangling of spaces to \xa0's] >>> on the newer system.\xa0\xa0SCHED_ULE still has bugfeatures which tend to Oops, I meant SCHED_4BSD. >>> help large builds by reducing context switching, e.g., by bogusly >>> clamping all CPU-bound threads to nearly maximal priority. >> That last bugfeature is probably what makes current systems >> interactive performance tank rather badly when under heavy >> loads.\xa0\xa0Would it be hard to fix? I fix it in some of my versions of SCHED_4BSD. It rarely matters. I even turn off PREEMPTION and IPI_PREEMPTION on SMP systems to favour large builds with fewer context switches and don't notice interactivity problems. This depends on the shell not running the build and not starting to many CPU hogs so that it stays at numerically low priority. > I would second that sentiment... as time goes on, heavily loaded > systems seem to become less and less interactive-friendly. \xa0Also, > running the heavy-load jobs such as builds with nice, even -n 20, > doesn't seem to make any noticible difference in terms of making un- > nice'd processes more responsive (not sure there's any relationship in > the underlying causes of that, though). niceness is quite broken in both SCHED_4BSD and SCHED_ULE. It was partly fixed in SCHED_4BSD in ~1999, but re-broken soon after (it is difficult to map a large dynamic range of CPU usage counts (estcpu) into the user priority range. The niceness sub-range wants to more than it was (81), but even 81 didn't fit and caused bugs. Fixes reduced it to 41 where it barely does anything. ULE intentionally copied some bugs from this (like ensuring that nice -20 processes never run in competetion with nice --20 processes). This is fixed in SCHED_4BSD in some of my versions, but I only use nice to test the fix. The implementation uses virtual ticks, with nice'd processes being charged more. This is almost perfectly fair, with the relative CPU allocation following a table, but Linux in 2004 somehow does better (except for no way to change the policy) using an apparently much simpler algorithm. Bruce From owner-freebsd-arch@freebsd.org Wed Aug 30 12:24:46 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90099E007E8 for ; Wed, 30 Aug 2017 12:24:46 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [IPv6:2a01:4f8:171:f902::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36A066A91D for ; Wed, 30 Aug 2017 12:24:46 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (localhost [127.0.0.1]) by mail.rlwinm.de (Postfix) with ESMTP id 8639114A66 for ; Wed, 30 Aug 2017 12:24:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at rlwinm.de Received: from mail.rlwinm.de by mail.rlwinm.de (amavisd-new, unix socket) with LMTP id 2_nWGU30u0Qe for ; Wed, 30 Aug 2017 12:24:30 +0000 (UTC) Received: from crest.local (unknown [IPv6:2a00:c380:c0d5:1:5c53:2df6:670f:72f1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id 8CBEC14A4D for ; Wed, 30 Aug 2017 12:24:30 +0000 (UTC) Subject: Re: ULE steal_idle questions To: freebsd-arch@freebsd.org References: <201708261958.v7QJwbGK054320@gw.catspoiler.org> From: Jan Bramkamp Message-ID: Date: Wed, 30 Aug 2017 14:24:30 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <201708261958.v7QJwbGK054320@gw.catspoiler.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2017 12:24:46 -0000 On 26.08.17 21:58, Don Lewis wrote: > On 26 Aug, To: kostikbel@gmail.com wrote: >> On 26 Aug, Konstantin Belousov wrote: >>> On Sat, Aug 26, 2017 at 11:29:29AM -0700, Don Lewis wrote: >>>> I actually haven't noticed that problem on my package build boxes. I've >>>> experienced decent interactive performance even when the load average is >>>> in the 60 to 80 range. I also have poudriere configured to use tmpfs >>>> and the only issue I run into is when it starts getting heavily into >>>> swap (like 20G) and I leave my session idle for a while, which lets my >>>> shell and sshd get swapped out. Then it takes them a while to wake up >>>> again. Once they are paged in, then things feel snappy again. This is >>>> remote access, so I can't comment on what X11 feels like. >>> I believe what people complain about is the following scenario: >>> they have some interactive long living process, say firefox or mplayer. >>> The process' threads consume CPU cycles, so the ULE interactivity >>> detection logic actually classifies the threads as non-interactive. >>> >>> This is not much problematic until a parallel build starts where >>> toolchain processes are typically short-lived. This makes them >>> classified as interactive, and their dynamic priority are lower than the >>> priority of long-lived threads which are interactive by user perception. >>> >>> I did not analyzed the KTR dumps but this explanation more or less >>> coincides with the system slugginess when attempt to use mplayer while >>> heavily oversubscribed build (e.g. make -j 10 on 4 cores x 2 SMT >>> machine) is started. >> I can believe that. I keep an excessive number of tabs open in firefox >> and it would frequenty get into a state where it would consume 100% of a >> CPU core. Very recent versions of firefox are a lot better. >> >> Xorg is another possible victim. I've just noticed that when certain >> windows have mouse focus (firefox being one, wish-based apps are >> another) that the Xorg %CPU goes to 80%-90%. I think this crept in with >> the lastest MATE upgrade. If Xorg is treated as non-interactive, then >> the desktop experience is going to be less than optimal if there is >> competing load. > I've got poudriere running right now on my primary package build box. > The priorties of the compiler processes are currently in the range of > 74-96. > > On my desktop, firefox is running at priority 24. Xorg when it is not > being a CPU hog gets all the way down to priority 20. When the mouse is > pointing to one of the windows that makes it go nuts, then it gets all > the way up to priority 98. On my old desktop (AMD Phenom II X6 1060T) which doubled as poudriere compile server I wrapped Xorg and moused with cpuset and rtprio. It solved my problem, but felt wrong. From owner-freebsd-arch@freebsd.org Wed Aug 30 21:56:33 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25CEEE0B48F for ; Wed, 30 Aug 2017 21:56:33 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0100.outbound.protection.outlook.com [104.47.33.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACA6581DE9; Wed, 30 Aug 2017 21:56:31 +0000 (UTC) (envelope-from sjg@juniper.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=PARJT11tdHYT51WOOyVQXKuWp4d+8eV7gG6GoyBuFO8=; b=FOEAVNLZ26h4SRewJIbnZ9prFBdv1zeNE4Jndo5yTASPEIQSZjDbSdHbN6UsSEDtvc+Y9YpCuCpU5fAV6O1rSk5uLkwM/DATyz/GFCM9/7k99w5jd9CNpHB1lw1AmUqdIG6yArjhxtoVcPHrNINY9T4LvElkmpNg6wPcIYrCpkM= Received: from BN3PR05CA0024.namprd05.prod.outlook.com (10.174.64.34) by BN3PR0501MB1250.namprd05.prod.outlook.com (10.160.183.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.13.2; Wed, 30 Aug 2017 21:56:30 +0000 Received: from BY2NAM05FT024.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e52::208) by BN3PR05CA0024.outlook.office365.com (2603:10b6:400::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.35.3 via Frontend Transport; Wed, 30 Aug 2017 21:56:29 +0000 Authentication-Results: spf=softfail (sender IP is 66.129.239.12) smtp.mailfrom=juniper.net; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=fail action=none header.from=juniper.net; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.12 as permitted sender) Received: from p-emfe01a-sac.jnpr.net (66.129.239.12) by BY2NAM05FT024.mail.protection.outlook.com (10.152.100.161) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256) id 15.1.1385.11 via Frontend Transport; Wed, 30 Aug 2017 21:56:29 +0000 Received: from p-mailhub01.juniper.net (10.160.2.17) by p-emfe01a-sac.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.123.3; Wed, 30 Aug 2017 14:55:48 -0700 Received: from kaos.jnpr.net (kaos.jnpr.net [172.21.30.60]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id v7ULtlck027259; Wed, 30 Aug 2017 14:55:47 -0700 (envelope-from sjg@juniper.net) Received: from kaos.jnpr.net (localhost [127.0.0.1]) by kaos.jnpr.net (Postfix) with ESMTP id 2DEF1385520; Wed, 30 Aug 2017 14:55:48 -0700 (PDT) To: CC: Steve Kiernan , , "Baptiste Daroussin" , Ed Maste , Allan Jude , "Toomas Soome" , =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= , Subject: Import BearSSL ? (Adding verification to loader) In-Reply-To: References: <44449.1497382261@kaos.jnpr.net> Comments: In-reply-to: Ed Maste message dated "Tue, 13 Jun 2017 15:51:02 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 8.6; nmh 1.6; GNU Emacs 25.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <24255.1504130148.1@kaos.jnpr.net> Date: Wed, 30 Aug 2017 14:55:48 -0700 Message-ID: <24256.1504130148@kaos.jnpr.net> X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:66.129.239.12; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(39860400002)(2980300002)(189002)(199003)(2950100002)(81166006)(81156014)(97736004)(50986999)(8676002)(76176999)(6916009)(50466002)(8936002)(77096006)(69596002)(46406003)(4326008)(23726003)(626005)(54906002)(450100002)(97876018)(9686003)(55016002)(50226002)(47776003)(2906002)(53936002)(7126002)(53416004)(6266002)(105596002)(76506005)(356003)(110136004)(107886003)(305945005)(5660300001)(2351001)(2810700001)(106466001)(86362001)(7696004)(68736007)(478600001)(97756001)(189998001)(15650500001)(117636001)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN3PR0501MB1250; H:p-emfe01a-sac.jnpr.net; FPR:; SPF:SoftFail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2NAM05FT024; 1:SQop3r2tULtLhln8TLuIPlLajxultANZvksPl2B3zlgS4cF+BuNpeC12JcROkR03/QjSun+1DbbS2Pc/dOrB4vNG3LPzmQFMR+MpbrDEjHqpYi1Lr1ko4sTwGB1ZbpW2 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d982fb09-3968-4eab-7171-08d4eff1f845 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:BN3PR0501MB1250; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0501MB1250; 3:Y+yFAVAMByx/x01SyJ4D51VGBz+dYnmTH0HejWNu1IX6A2pPZF7PFqju0XcLSObmMvIME5WR9ULdBikUtiEA0wmDWGQziowQpRt7OldGGJBMhE1U7K2NW4Ubv6yKs236gg4IaTx9xWRTiLZgYw5VkMTUplG6sM178D6rVXxtKW6DiQvg0Dt0FF1ukR4IOe2kcCMidUJmeMNBRyoLdFga9XFcbwizcj61ntYpGOdfTkJHUIkpCFcD/X09EDVCTJBSF5sT898AwBl5z4fc6zVdTFkC5rRXBUbprNixmAjArdls16N05zbmX9AYYg9XwOqff2/imlOlgyMSAC6bLuPxL/5JcXpIoFCyr+xqo9x4aZk=; 25:ZEdQ2zO5cv8gj7IPdIlvb9zDAqRXCeP0gAJdMM5zqTexF/ZgJctreolw9JfDfppX5O+GL7buICM6Tbe2s8mn6eUsgmOXe/p8VGoIY+h//BcZzRhHPvEr46BzAgNwaZsdzWeV+DGEyeacn8LaoLekLd0FlEMhxElzfh7aCteAdkTDJUhsDYk/5ybolny+Z+kDWDQ1qLtQo1xChuQw2q587+L80W9ilUskyU/JYhp4M/ZExJ1cvryRmyG9g218MjR5J46HmhTdSFUNWwiwIZ8DYzdUgkrWPKsSPNFeZSu8CiKak/gXkECeoIWmbCzOhVNO7Owtsawg+Q660GCd8eiOvg== X-MS-TrafficTypeDiagnostic: BN3PR0501MB1250: X-Microsoft-Exchange-Diagnostics: 1; BN3PR0501MB1250; 31:gcHS/61MUZFYCPG1MFxMUmgijVVRueNapILeu5a15gx/bX13i4pROSsB56IHe6+YloweCSToFshxp67RLL/vQ1JoOw4IWTkEPrekbL+3YprTCzMGXdE0kb4XvKfdtUhnjMLwcrQeifp3foAJRwpIXN31Gt8h5QTOI4BiYQXZaZuLlOBiQCZQfUvh3neenPXPrizkLnDzJwXQGI1Y63ik5TposRQzulEV8r7pH6Pg/8s=; 20:Kn+U5ELAXHvCre+RyRuU3RHUwKQXmFaqIwDHJpYk6i1PSq+OniIzFJ3RXsSbGoln/AYauhiNt2dcLYAILN2OrNbfRudwMOHrgSsnYhQgE69e5aiDPJmVK01fMnwGd9c6BkESdQp84IKJfrJEtbUjfTDqevBDuZHslh5taNHRCU8fD9gFXNvYn4NuDAMssUbRCsLD65k0a8qqGc/SzffBdx6Bji/c1ofFJY7etpi2LRd9R0CrzHOFZrYOS0NnWTTh8AR2VwNnIoKpQMxLHxTTvFSMq5+d0sDcMNu5j11c/c647SPTMn1/xB9ujc+taTDZ29iMQx5wRNtkWwYGZgedWnw0cWvItS+K5OXSqSY1h0/qoMpVG7LamkkB7skMUU04M9oefKoOwJ/PyFPsaZpHk7Gxgqu8tR9KARh7JuzhBxw7/HZqeqKV//r1m/dT/I9ZlOMrlqeCUThTfXA0P7DLy2DX7l5O2YLLvPzR9RWI3VKy53XecwWGa6TKfKeAYZ5M X-Exchange-Antispam-Report-Test: UriScan:(192374486261705); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(13018025)(13016025)(5005006)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93003095)(6055026)(6041248)(20161123562025)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN3PR0501MB1250; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN3PR0501MB1250; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0501MB1250; 4:Sha7iPP23g++L94j0k+kaSrvyZYoMShORR1MSRtRl2xkkNtFGIONcblIV/44hB/91C/Jy3uSarLa5kmjctxWUoek1wdFCll6q9CVqeBSHzILSbLkpWjcKAeobJwYj+pvfeTt5fzdvFD84vnHIvsmg4DDsvP2hxEcofKqUX/ON7EtHb+17Q4QEUCZVWUJx3tf7tCxqjTP5+PdjmVwVp56E6bDQNC046ilSFStgQSGp4bkKiZ+QpDOzL7gEXYnqKolUwPp2+Nf08b7OBW+aPrGFIj9fUBfoKAkVM8T/zkDHxY= X-Forefront-PRVS: 041517DFAB X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0501MB1250; 23:juoOjPh65LsGKByWVgRqzDoXUMg0r2WdBcdDbcx?= =?us-ascii?Q?QGOuQAjO58stIv4pcduEebMrjVPKa+96n3AGqDzTOduQ0cYTQYCXuxuiADTx?= =?us-ascii?Q?BfdSjkPmpLYgGlXZj2pJ3V+INjuo0S0sWXByphVNxrD94RPq6UnqzGZMGMzM?= =?us-ascii?Q?N4lS1HEWnKYBWZl+7FVfuOMrjX8gfWHuWaPaANCXVRHIhg6Y4f20HmczNavD?= =?us-ascii?Q?lIWrgyIBtn0+JOxT5oFe2yWTeY6jyxACWStK4mJdqEwnE+R7CQ+Pm0vPCbQq?= =?us-ascii?Q?7LuObxoOx86kxxCOpyC7FAp0Ny4mtYzUfbw4WTFmCsyn8g0u0Twz2iSV/g8l?= =?us-ascii?Q?C5lofTla0ONGwOo82XWRCnO7HACNoLCkPEPZaqkWMXQspP8Jh65+BaPWdZSc?= =?us-ascii?Q?Mwpi7i/e0gjusCjGptJW/SQ45+NuFHRfpvpzKrtUj9oorNewaIiP8uFLsmC5?= =?us-ascii?Q?4cvVECQnIK6aEQrgxT1ybAQubQoAxkjqDa7jhSyoWSX4nKJRTljiAARU/65M?= =?us-ascii?Q?kpXAc7xwLMrBYRj+A5MZwmmz/ASmbW8/85OBoWcKpgxAbohP0kSmg3F2QgdP?= =?us-ascii?Q?hhUboYLf0NZ7YVqFut2rw0yBRnpF4bHYZcnrDZkcEWnJ1v/wuEUTmR2M2ijp?= =?us-ascii?Q?01TnFXwMz5eg0Xko+H8SLmp7MZslvnid6EEawL46TBK33CrKE6QtT+i07PgF?= =?us-ascii?Q?gwwUXu057+0dAfTgm1hQKrRbHKqiCNduOULMS5CvHCW6K5bhZkoCXwLEYbSs?= =?us-ascii?Q?W4Pn2Vwt3JtzOO8VOH6eHYVr+Uo/mBFnsdHFFH6DpE1QqjqnDpNDEgqJmeYm?= =?us-ascii?Q?D3K+TiwKonVolW4PQyXx2AXPmC3+q80Iq8PB8BkVoFB9uH3C2IbMQY5r4o/4?= =?us-ascii?Q?GXaHqEPMisqE4slGlU7DwZ1JUOYRTHmsBKcVCnVZDgD+JVIAAZf/xeaGOxK+?= =?us-ascii?Q?dc878gAqcaW5g6e6Gmf8jtmxmBE4A7z4eLzPZwqhRGaepFE7+Pwb6V7wP2gC?= =?us-ascii?Q?/QNHyPKg0ZscmT77VZSoJjbUQfcIt++qQKVBOSPRdE1s+qBt05Iv32RX0G9m?= =?us-ascii?Q?UfH5ZrQYjIzq5YuwkhF/6LV2kJZjL0R6UsyoQ/zVID1B9RDymIRE686e/AeN?= =?us-ascii?Q?cHRA+mEk0BCZ+jbc9JgNzDWjRVf1L7yUKdNkaMJvv+MilhCE7mnEGza1PkGL?= =?us-ascii?Q?XKEv/4paWiQycBb5YAocUy7wJxygnn7wOu1QMeDlwghqXVvByBshue8FYTA?= =?us-ascii?Q?=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0501MB1250; 6:HCyl1EgWuhTJILP98nLdU58EYQbD3cB1zATRF+1edYEKFd0XReRI2sci+cJDrTRL+VoM9AEJpU195uvEUr6bH86jPP7M6qtiP8GOYHrHZ0Dx3Wk2U/XPcS2GLToV4prTUoUBn6ByyB7M6WnPxWcnDYa0ODJLFi+9bEFmLPexKf1AqkZiRtVUVHVlIJC2ZHImkqTP5SL2F6QGpEjua20FbZyipiKdQB89teArZocepWl8dF15uthkgXFuRRHoT2bhYL4ZaWh7tadZwhPK10Svt5BsjSW2ysa9A4CALsWe5Np1V1M4qThWbToSyvKBaex8n9UO6Urgt0R8kh+rxXUf1w==; 5:9jvepWBR/CYhK1ESQ+SNA7wQ2I5ed7CdO05iZeswvvNbPjrYVWIj3xlTRNMZt/dflNW5uCxNO9z8f8yWCxvRS9ZbVw5fXhh1o40NjoUIGgrwx13jvZIop0GjmobeQvxxFsVR6v0J0PABkovs7idv8w==; 24:fvVySNYGA/HVYLxuDIbfoLAbBy2uLvdfTtcbUktTF0Wx/p6e0H/ZxObi7/HbAzj6p5qMXBiR5ztO/r/QLZESE0ZJNDKJX2v0GoFJSopjYpo=; 7:YyS8hKd1kGOzYxmRYJ/RWEm479radfIYSyPFB3+OxTOeIr/ceLwQ+KhLSUF5NMrwHbEVe1hdqf/MpO/acTRTVGnC12BSeRLR/Afs8Rit0UKkKap5EZfzKhp10uS+UfXQkDnas6RFJE7dsEdjN4fKHd6Hcq6bPLaCiI87LXU0lbLbu41SBhYDvayh83MLas93w373dtltmJa3iYkUyca+FJOH8qPKpyANkzcs30dOIoQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2017 21:56:29.5196 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.12]; Helo=[p-emfe01a-sac.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0501MB1250 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2017 21:56:33 -0000 Hi, Background: I've been adding what amounts to a mini "verified exec" to the freebsd loader for use in Junos. What this means is that the loader verifies the kernel and all the modules before loading them, and can reject anything for which a registered fingerprint (eg. sha1 hash) does not match. This work is probably mainly of interest to folk doing emeded devices or security appliances etc and can be seen as closing the gap between the secure BIOS (which verifies initial loader - grub? whatever) and mac_veriexec which we use in the kernel to control what can run in userland. The boot process on Junos is much more complicated - but also more flexible than stock FreeBSD. We potentially load lots of "loader.conf" snippets from different packages which contribute modules that need to be pre-loaded. Of particular interest, we always provide the kernel with an md_image for initial rootfs, which means the loader can verify the kernel and everything it uses before mac_veriexec is initialized. This obviates the need to touch the kernel at all. For efficiency and flexibility of signing, we use signed 'manifest' files to carry the trusted fingerprints. These manifest files are signed using RSA or ECDSA and an accompanying X.509 certificate chain, allows one to verify the public key was issued by a trusted entity. This approach has proven useful for more than a decade, and allowing the loader to do the same, was an obvious choice for us. Which brings me to BearSSL (www.BearSSL.org) This is a very small library designed to work in embedded environments. The author gave a talk about it at BSDCan earlier this year and it is just what I've been looking for for this project. All the code to do signature verification, fingerprint matching etc, in fact the entire mini-veriexec for the loader adds only about 80K. Last I looked at trying to achieve the same using OpenSSL - I gave up at 6M ;-) The question is what to do - for upstreaming any of this. Assuming of course anyone is interested in this functionality. The changes to the loader itself are trivial. Most of the code is in libve (naming stuff is hard) which handles fingerprint loading, lookup and of course verifying signatures using code from; libbearssl - which is just a reachover build of BearSSL. I have it setup such that BearSSL need not be part of the tree at all so there is no burning need to import it; lib/libbearssl will simply not build if ${BEARSSL} isn't defined and pointing to a BearSSL tree. >From an internal paper-work point-of-view, contrib/bearssl is attractive to me ;-), but it could just as easily be in ports no where at all. If it were in contrib, then it would be feasible to leverage it for other uses in the loader that currently use libmd etc for hashing. Discuss ? Thanks --sjg From owner-freebsd-arch@freebsd.org Wed Aug 30 22:43:19 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0C13E0BEFE for ; Wed, 30 Aug 2017 22:43:19 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from bat.oak.relay.mailchannels.net (bat.oak.relay.mailchannels.net [23.83.215.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F757833DE for ; Wed, 30 Aug 2017 22:43:19 +0000 (UTC) (envelope-from ian@freebsd.org) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 13AD120165A for ; Wed, 30 Aug 2017 22:43:18 +0000 (UTC) Received: from outbound1a.eu.mailhop.org (unknown [100.96.126.149]) (Authenticated sender: duocircle) by relay.mailchannels.net (Postfix) with ESMTPA id 665A22018E5 for ; Wed, 30 Aug 2017 22:43:17 +0000 (UTC) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [172.20.107.195]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Wed, 30 Aug 2017 22:43:18 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: _forwarded-from|73.78.92.27 X-MailChannels-Auth-Id: duocircle X-White-Thread: 12f0e50e5f41445b_1504132997884_2093115617 X-MC-Loop-Signature: 1504132997884:2348899577 X-MC-Ingress-Time: 1504132997883 X-MHO-User: 966682c0-8dd4-11e7-83af-a91f44540cb3 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id 966682c0-8dd4-11e7-83af-a91f44540cb3; Wed, 30 Aug 2017 22:43:07 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v7UMh3nd004637; Wed, 30 Aug 2017 16:43:03 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1504132983.56799.90.camel@freebsd.org> Subject: Re: Import BearSSL ? (Adding verification to loader) From: Ian Lepore To: "Simon J. Gerraty" , freebsd-arch@freebsd.org Cc: gtetlow@freebsd.org, Ed Maste , Steve Kiernan , Baptiste Daroussin , Toomas Soome , Allan Jude , Edward Tomasz =?iso-8859-2?Q?Napiera=B3a?= Date: Wed, 30 Aug 2017 16:43:03 -0600 In-Reply-To: <24256.1504130148@kaos.jnpr.net> References: <44449.1497382261@kaos.jnpr.net> <24256.1504130148@kaos.jnpr.net> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2017 22:43:19 -0000 On Wed, 2017-08-30 at 14:55 -0700, Simon J. Gerraty wrote: > Hi, >=20 > Background: >=20 > I've been adding what amounts to a mini "verified exec" to the freebsd > loader for use in Junos. >=20 > What this means is that the loader verifies the kernel and all the > modules before loading them, and can reject anything for which a > registered fingerprint (eg. sha1 hash) does not match. >=20 >=20 [...] > The question is what to do - for upstreaming any of this. > Assuming of course anyone is interested in this functionality. >=20 > The changes to the loader itself are trivial. > Most of the code is in libve (naming stuff is hard) which handles > fingerprint loading, lookup and of course verifying signatures using > code from; libbearssl - which is just a reachover build of BearSSL. >=20 > I have it setup such that BearSSL need not be part of the tree at all s= o > there is no burning need to import it; lib/libbearssl will simply not > build if ${BEARSSL} isn't defined and pointing to a BearSSL tree. >=20 > From an internal paper-work point-of-view, contrib/bearssl is attractiv= e > to me ;-), but it could just as easily be in ports no where at all. >=20 > If it were in contrib, then it would be feasible to leverage it for > other uses in the loader that currently use libmd etc for hashing. >=20 > Discuss ? >=20 > Thanks > --sjg We need this exact feature (verification of kernel and modules) for an upcoming product at work. =A0Including the library code in contrib certainly sounds attractive to me, too. I wouldn't be surprised if interest in this goes beyond those of us building embedded appliances. -- Ian From owner-freebsd-arch@freebsd.org Thu Aug 31 01:34:45 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5DE0E0FB64 for ; Thu, 31 Aug 2017 01:34:45 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL SHA256 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62DB83AED; Thu, 31 Aug 2017 01:34:44 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.15.1/8.15.1/NETPLEX) with ESMTP id v7V1LKVe039427; Wed, 30 Aug 2017 21:21:20 -0400 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Wed, 30 Aug 2017 21:21:20 -0400 (EDT) Date: Wed, 30 Aug 2017 21:21:20 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Ian Lepore cc: "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Import BearSSL ? (Adding verification to loader) In-Reply-To: <1504132983.56799.90.camel@freebsd.org> Message-ID: References: <44449.1497382261@kaos.jnpr.net> <24256.1504130148@kaos.jnpr.net> <1504132983.56799.90.camel@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Aug 2017 01:34:45 -0000 On Wed, 30 Aug 2017, Ian Lepore wrote: > On Wed, 2017-08-30 at 14:55 -0700, Simon J. Gerraty wrote: >> Hi, >> >> Background: >> >> I've been adding what amounts to a mini "verified exec" to the freebsd >> loader for use in Junos. >> >> What this means is that the loader verifies the kernel and all the >> modules before loading them, and can reject anything for which a >> registered fingerprint (eg. sha1 hash) does not match. [ ... ] > > We need this exact feature (verification of kernel and modules) for an > upcoming product at work. =A0Including the library code in contrib > certainly sounds attractive to me, too. > > I wouldn't be surprised if interest in this goes beyond those of us > building embedded appliances. Indeed, why couldn't it be enabled by default for FreeBSD.org packaged distribs? Or am I jumping the gun by a few years? --=20 DE From owner-freebsd-arch@freebsd.org Thu Aug 31 01:55:39 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6598DE10CBD for ; Thu, 31 Aug 2017 01:55:39 +0000 (UTC) (envelope-from amutu@amutu.com) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F8DC63B0A for ; Thu, 31 Aug 2017 01:55:39 +0000 (UTC) (envelope-from amutu@amutu.com) Received: by mail-oi0-x22c.google.com with SMTP id r203so65038734oih.0 for ; Wed, 30 Aug 2017 18:55:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amutu-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=LvgYARGA/Dr/L4RHVKdA2DtdzqlDu5CZSad95KovYkM=; b=bg3faZ/Qj6T8cubfCRuz3fVRaptGFgMxSvidad1XS6lv6zhI3MxE5fxBQRZ7k7P46c sKQhff0WCj/52yO6S2WQ/yeRxpjmBtLIBJVVce4tJ2RHDfkUGs5tToxmJc2WBC1wWTqo 2L37/ANkW9TLBJkxdrZeMvgVOBZjFjGyM8O50uyIb9cPvMySoRfj48rYwOJPWEh65+bB RTHr0R/y9/n5gYH5MX+WZXi95Zv7UWRd4zP9YJQKwqN02+mVMLyffRgSMPDt8PG8bqZK mavAe7WsHy2yeYGL7HaYN7RGQnQci4wn8OoKLmvOciP4ksotQKxlONpUnagAp6bEukpE r3ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=LvgYARGA/Dr/L4RHVKdA2DtdzqlDu5CZSad95KovYkM=; b=NzJ85Ko0iuhAmIFeeWUEpfj8whIYeSBgr4t0JK8sXIW+SIGOXZGJkBM3SGKUe9h7yG i0PJuBVgyDJkLoYrV0UwfBxcE7zyRio1+8ZT1jDoEsfz9Ic7auEblTi6k5TZXjG/8hwa pjUSIGZygEsDOkvxqTmJuah4ezUEY3FXgMW1va/u644dZrwl6elN8S5bMcZCEjKURtd/ rhTrI24sSab9klZUmO1c856dcQIbWOUjsuApR0lj0DIGO4rhNi7ZSB4qc9IZ12hVVMC4 cRw0sr6G+BFUF7muLq9wtaWvHx9EnIqmQTRjn0no2hvAvLctIh8FJqz2Jz0wV73teBdT rtgQ== X-Gm-Message-State: AHYfb5iKZTQ2SGdJmmoXg3/CtfdklhL3Dii4FO/NXZ88WBrykx7LvQNh Y79r5KvPgsHmn9G8 X-Google-Smtp-Source: ADKCNb6wnuK6//7pny23FJ1dCYJhhnz6TlFcRhZsJb+D2V9ZbhoJnlaS0e4Y4IFesNykebzjTIC//w== X-Received: by 10.202.245.216 with SMTP id t207mr3112347oih.228.1504144538222; Wed, 30 Aug 2017 18:55:38 -0700 (PDT) Received: from mail-oi0-f52.google.com (mail-oi0-f52.google.com. [209.85.218.52]) by smtp.gmail.com with ESMTPSA id r129sm5863305oif.8.2017.08.30.18.55.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Aug 2017 18:55:37 -0700 (PDT) Received: by mail-oi0-f52.google.com with SMTP id t75so65180173oie.3; Wed, 30 Aug 2017 18:55:37 -0700 (PDT) X-Received: by 10.202.172.205 with SMTP id v196mr3130881oie.49.1504144537125; Wed, 30 Aug 2017 18:55:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.128.197 with HTTP; Wed, 30 Aug 2017 18:55:16 -0700 (PDT) In-Reply-To: References: <44449.1497382261@kaos.jnpr.net> <24256.1504130148@kaos.jnpr.net> <1504132983.56799.90.camel@freebsd.org> From: Jov Date: Thu, 31 Aug 2017 09:55:16 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Import BearSSL ? (Adding verification to loader) To: Daniel Eischen Cc: Ian Lepore , freebsd-arch@freebsd.org, "Simon J. Gerraty" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Aug 2017 01:55:39 -0000 +1 I use zfs+geli encrypted root for several VPS=EF=BC=8Cas the bootpool is no= t encrypted, I am always worried that someone may replace my kernel or kernel module.(I know 11.x support full disk encrypt without bootpool but an upgrade is not a choice now). Jov 2017-08-31 9:21 GMT+08:00 Daniel Eischen : > On Wed, 30 Aug 2017, Ian Lepore wrote: > > On Wed, 2017-08-30 at 14:55 -0700, Simon J. Gerraty wrote: >> >>> Hi, >>> >>> Background: >>> >>> I've been adding what amounts to a mini "verified exec" to the freebsd >>> loader for use in Junos. >>> >>> What this means is that the loader verifies the kernel and all the >>> modules before loading them, and can reject anything for which a >>> registered fingerprint (eg. sha1 hash) does not match. >>> >> [ ... ] > >> >> We need this exact feature (verification of kernel and modules) for an >> upcoming product at work. Including the library code in contrib >> certainly sounds attractive to me, too. >> >> I wouldn't be surprised if interest in this goes beyond those of us >> building embedded appliances. >> > > Indeed, why couldn't it be enabled by default for FreeBSD.org > packaged distribs? Or am I jumping the gun by a few years? > > -- > DE > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@freebsd.org Thu Aug 31 03:40:55 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 877E4E14604 for ; Thu, 31 Aug 2017 03:40:55 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0119.outbound.protection.outlook.com [104.47.38.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC0BF67148; Thu, 31 Aug 2017 03:40:54 +0000 (UTC) (envelope-from sjg@juniper.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=iDp48yrwQ2dV3+f2gP/eP2nsynnFdKWN/kIKlvUm/f0=; b=Rd9t9/msqFI6GVHOAl3cdkSPE9h9m4LvuCEWXC7vmRQpXZk0je3XuIyskl7CF6wJO+bldXlGDGeHX3Bd9znN3BDX+tq+pVefMYpXOPg1euSsw8ekKfMrPveaIbzYmOGTKaMNvhiIS8/mqqPg9OTRt5JoCnrNCzGJhkbQPXXxBQQ= Received: from BY2PR05CA024.namprd05.prod.outlook.com (2a01:111:e400:2c5f::14) by DM5PR0501MB3845.namprd05.prod.outlook.com (2603:10b6:4:7b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.13.2; Thu, 31 Aug 2017 03:40:52 +0000 Received: from DM3NAM05FT036.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e51::209) by BY2PR05CA024.outlook.office365.com (2a01:111:e400:2c5f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.35.3 via Frontend Transport; Thu, 31 Aug 2017 03:40:52 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.12 as permitted sender) Received: from p-emfe01a-sac.jnpr.net (66.129.239.12) by DM3NAM05FT036.mail.protection.outlook.com (10.152.98.149) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256) id 15.1.1385.11 via Frontend Transport; Thu, 31 Aug 2017 03:40:51 +0000 Received: from p-mailhub01.juniper.net (10.160.2.17) by p-emfe01a-sac.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.123.3; Wed, 30 Aug 2017 20:40:50 -0700 Received: from kaos.jnpr.net (kaos.jnpr.net [172.21.30.60]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id v7V3eovS026937; Wed, 30 Aug 2017 20:40:50 -0700 (envelope-from sjg@juniper.net) Received: from kaos.jnpr.net (localhost [127.0.0.1]) by kaos.jnpr.net (Postfix) with ESMTP id 841F1385520; Wed, 30 Aug 2017 20:40:50 -0700 (PDT) To: Daniel Eischen CC: Ian Lepore , , Subject: Re: Import BearSSL ? (Adding verification to loader) In-Reply-To: References: <44449.1497382261@kaos.jnpr.net> <24256.1504130148@kaos.jnpr.net> <1504132983.56799.90.camel@freebsd.org> Comments: In-reply-to: Daniel Eischen message dated "Wed, 30 Aug 2017 21:21:20 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 8.6; nmh 1.6; GNU Emacs 25.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Wed, 30 Aug 2017 20:40:50 -0700 Message-ID: <47823.1504150850@kaos.jnpr.net> X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:66.129.239.12; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(39860400002)(2980300002)(189002)(24454002)(199003)(97876018)(356003)(8676002)(81166006)(81156014)(305945005)(8746002)(8936002)(50226002)(6266002)(77096006)(15650500001)(4326008)(53936002)(626005)(450100002)(50466002)(97736004)(47776003)(6246003)(106466001)(76506005)(105596002)(229853002)(6916009)(2950100002)(93886005)(53416004)(110136004)(107886003)(23676002)(86362001)(189998001)(54906002)(50986999)(55016002)(76176999)(68736007)(117636001)(5660300001)(69596002)(7696004)(2810700001)(2906002)(7126002)(9686003)(478600001)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR0501MB3845; H:p-emfe01a-sac.jnpr.net; FPR:; SPF:SoftFail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM05FT036; 1:AnTO9xS+AtrXk8u2xvYUhm0UcG3zEDQS9g81kECqunNVub/LBFiZq3b764k3fP3AUUUagR316JVyBoGBBRApfgroibEO6aXJfwQZHjRoZjz/LrL1POQjxvRUPlJ4Swvm X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2de3b431-4faf-4e9c-79e4-08d4f02213f2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DM5PR0501MB3845; X-Microsoft-Exchange-Diagnostics: 1; DM5PR0501MB3845; 3:JB5K1aiOZruffQwE1XV6udgCakFmDGwK044nFSytxL82+4l/ZcGb9r4K1A1OklSQMu4WsPQuoTlFScmoUesLNIz04U9Olh4ljVhI6tPY0Ph5zqOqE5KvgNgfQNPX5GX+JGhalWaun4V0KopCtdq9ao/GDfP2gGbQy4nQWY9fQbFyudqaXjhYo5WFGNpXB4CmP7eIA+cfJGdLo6Pjhw9yQICIlz/lbO8Yo0IcO3sg+PKXE9FNHMYjLLPjodrTspUbgAT4EOdiHVBcXeIZ/BGrh8gjVbgzrwq3TtbxIzVSYmmAf8QgrM7YA8lE+GoNtGqX21v1iq7eNQ+tt83AApynVh6c6vRkZN0IxCR/1bcLqpA=; 25:a7yVGgkMceI9QqtsMA/Uyh/B4xXZgnp8tRdZ/+tzUaJpMCUD8RvFSfyH9X2jhOzeEN//C8fK7eRB5//9RDfxg0GN9IjrN7DDv3CDgNxXtpTsE1ZnU/wuEelxQk25jRSztcNz1288ewhFX4tn4Xu7QioSMHeHOXWTtPzaHVEXtkNpr6kyIPUbLQL27fQK2WDe18lRd4x3ISfQ45Oib5aM4IFiVXuPoTN1R5hQ99Ut6e4CtZwFH7sMGaoXtSyxuMcV+N7rmzpJ8eg3bTQXRT2gTLtfSPNw52t6fmbyXabWo2lF5A9RADBpflACUowqddl/vyUChHyUvSNDVXFsTmLbwA== X-MS-TrafficTypeDiagnostic: DM5PR0501MB3845: X-Microsoft-Exchange-Diagnostics: 1; DM5PR0501MB3845; 31:q8/rgMZdqHlaqdp5SLe5cc96QLF0mdMPEBUpNLRE+6oTEgcegJGuiEZmou/dbULSxD+pSm6dDbkOFneMrTFavvDCnY0VJY9bMG9KxAfEHl4GCzdnL/ygf+VEZQqlb5d96ciLqtDpZ4mHx9/MlziSkNMu8bsBzDmLMnnAlbNTbj88UM3G0BiPQ01pS9PmefFhxL/WD+rBYQfKnMzTu0PBwmlUz1KDdKRYrPjpEmKwJNs=; 20:RGWxyKbAu+jxfNQmU1iszp9fJ0E8fdqymGNHMVSXsl1weg3bFIYVyIIfMEFficQ9BZ9WJcMdktN2XGPR5CVB3JR+gYW/NcqNFluIudWiceyp70OSu4RjoFcDYWZzxmL01IxE40TsbItnTTK5aNCydGBTlEeb4QQeWcNzvzN96DLdZyfEX0TZeH7Ahonyr7n/uazTiQyjxw4i3MMvcnWh3WikLAGttW4JDXRB2BA61Q6mnMw2DRUIGuEYuR1Gsd13DSRVQGKsMorbTaKgDV+mnbXZ10IUc/JHL9rpPS+7mANUR636K8pulAkRaIit7N+k9ySZqySFJvxmFFht68+Tk3qkJWAY7kONYChWc267M8PBWlEzvAxlgQsl+vkb5xpG+Qx23RwaKhmh4WxCKKi6a8j4ExahlBJd9ZDzeN65JxFH4xfkAPLBqii7CL2AJz4cq4DfvR8ls/m25UoBh8Cm01IQYAd05V6ESuACbIa3K0yEkwX8fLKAMjU17A3tU1Go X-Exchange-Antispam-Report-Test: UriScan:(158342451672863); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(13018025)(13016025)(8121501046)(93006095)(93003095)(3002001)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123564025)(20161123558100)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR0501MB3845; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR0501MB3845; X-Microsoft-Exchange-Diagnostics: 1; DM5PR0501MB3845; 4:/YGFf5d6hlxdK91gHN3p7fV+rOOaeMsWIZ0ZIXkLwDhNvXHj9mIBj8nWLKKwFgCP+zUbWnrU0QEpBN7i/GBVjPucuADj8R1RZKl1xhq5fM8f6juyYFxBB+t/cU3GkUh3wFL7wp/FLQe3fpJy7WyJrbhnjyXdxf1ak0B/n9pazeUqrifIcWOiwoBHHuebcNvH7Wt0mwY1f6PfSSMjX1FGnerxsCIc5iUdfR92SVsVb88a5juUkjKLgJw6gPxxtQ/5MJjti6hjBeCxAo8Frq5YQhwH/YWJ9HqJ/Bi3FoFqm/U= X-Forefront-PRVS: 04163EF38A X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTVQUjA1MDFNQjM4NDU7MjM6ZlRLUDU2YWZjaXdtUmIxcWtyM21MQ3A5?= =?utf-8?B?NFM3VDQxdDNXMDUrTHVXcEVkY0F5NzI4YWluMXRZNGVXdEpTdnR5MTl1eUFF?= =?utf-8?B?OHlJUmZLN0RqU2dXWHNvL1kzZnJqTUhVdE5KWVBwaVVCTzZ5eElDSDRZb0JJ?= =?utf-8?B?Rk90ZVBKb2h6azFZK1NnTXBRd0EvNEhCL0N4YitOQURROWoraDlzWHVBUnVN?= =?utf-8?B?ZitpYng5NGJuQ1ZwVnkzdlNrMlpBZEFTRU8zcnl5N0xrcHJvVUF1QUs3Y2Jq?= =?utf-8?B?NkNMWGNRUitrZHpoT2dIN09RUEx1RjhybVdTK1ZKT2JTSGJsVTlDc25qVndi?= =?utf-8?B?am1VQnM0OXBRMUEzUVhQajZIMVJxM1EvNDNYOVY2TzVYS2Qxb0ZHMG5kWFlC?= =?utf-8?B?STNhQmR3VXhJMWtYY29EU0JmTUJGU1ZwQTE0bTBWQ2tMeWhLVGsvSjcrN1Mw?= =?utf-8?B?R1FiMmF1cUVMc2NYYjlzVWRzeHlhbEc3WU1Heis5WUhsUlNWbGdHOXRpMHRT?= =?utf-8?B?SHE0cjBiZmM2NUFWRGVvYWRSYnlXL3lBM1pEeHNBMWFjbS9hQlptbXU4U1pu?= =?utf-8?B?YjJ2ZFZxVnVQazhzZTd1Q3FXZENJdHc3d0FpODdHdDIyd3dkd0QzQ1JpUEVB?= =?utf-8?B?Y2l1T0lZUm9MTDBNMHVCWERKMHE2cEZpWUxlalBOVC8xdVJBRzlRTFA4UlQ2?= =?utf-8?B?OTFlTDJTSXRwSTAyWGJPdWhiNHhVQnA0WGFBb1NvaFhua1hETnJ0UXRqZUVJ?= =?utf-8?B?NS82WFd5VllZZlA2OHU1bHM4bEsrY3dhd2kzTjJEeStESE1BU0QxQStPY3NO?= =?utf-8?B?WThrT3p0RVRZVU9zU3duZGxPN3F5ekVpcExSaUJBTkV6M0tVV3RKeTRPella?= =?utf-8?B?MHdHRjcxNzg5S2ZuUEVYREVPSW1iRTUvL2tCNEFmRWg5aWJHMTV2RXVKdDNV?= =?utf-8?B?Zk1qQURNd0hQTjJZdkVZbzJjSzdiYyttM2hGOUZkM0dWUStoTUNWWFJzRVdU?= =?utf-8?B?ZFJxZDFMZHlhZ01YL1BtL3lPZXBCcEZDRG9VVll1Q01sRFNSTm5VVGtVOGp6?= =?utf-8?B?SlV1aCtCOUhObU5OYkhRcG9KTGJiOWdzN04xN0NlRkUyZitKSWwweG02czdH?= =?utf-8?B?SldXaW85TVdFd2l0dU1XbU1aMVVSbmNuamgvNXhPTHI0VkxXdzNORm5lcHJi?= =?utf-8?B?V1ZaVDdJMVJ3TWZjL0pXWTg2ajBMYWRhZWluWFg3Z1lWMFpPMlFxZ3Q5bUla?= =?utf-8?B?SkZ4aUd5b3ZnV0JIVHdhQjE1QkRVVmtCdklUNkRlODdySTFsUjloZHdnSjVu?= =?utf-8?B?dzVKeWU2bEdxdjk2NFM5U0JVTjl3RHZya2dib1h3Uk5sUmdBZ054RG1TKzhZ?= =?utf-8?B?MGxYbGwvbUR6MHcyc3g5ZlFJZXlVeURQRVZ5bEZMUDB3Ly9uMXZYWWpoNFVo?= =?utf-8?B?TWJYNVp3WWl1YUlyQlYvZUtRdWt3bjR1TVlJRTF6cFY3b01vbGtub0VIQTNB?= =?utf-8?B?VlJOMWhjcHJKVkZ6MXV5b1FOZ21UdTB2UHk5Z3lTdkdxK28xVk1JQndsL0t5?= =?utf-8?B?QmJMUE5UNURIaVhNNzIyaUx4OFJhUjdFUDBwa2EwRlhiWkkwZlVJTy9nYis1?= =?utf-8?B?dUNZQ1NvQkFUeEZzeWdEdlNWaE1Rb0YyKzlHTTBuRTB6OG1kb3JMU0dFT3Iy?= =?utf-8?Q?kZEaPFP0uzZSvCT/cpT3vv9wAJC2y/KoDB3fr0ggw?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR0501MB3845; 6:D2rQRCB+o1ananIr2YZnRCXdmvPpcy3r9ak9RFUxC2eXu/MTct0aDRpDT5BvpeSQ/L5pqqgTZsTwtF8nEzka2sj8bfecl9EaC+fTXyQJ/kTwglT4A6M3uP6Ug/rGk4NPSBb8yqQUXGtANU4KeJ0SF++yDT+Gk+i9Q9+P7rgnrDtPAVVhQcq0z0OiRa4s2vBqi38n4C8zyyOBYQBLO/Y+lEmV046tv1xJShaEvENzFvsfE3f8evSvyuRh5S0vj21HI5y+AK5ADjHKJjq0GE+MeZ9vMxD0VZnNyTg8QiToddx3zbpw90j1rZggtoqPWPuWcWrOY7gU1sQ8761WRTq86Q==; 5:M0MeluGAqNH+7DCjqu4On4M9ktoRZNYUbAtHzYGCEma0NLOtNHKIPn6/Rl7NqZz2NQXmt6GkT251RN+ryD2WV2dvqa75dPzGnwcOs01nTbO/aeUVnPWiEUtvSNPJNjAO+CjnqCdY8BuffRJwkD4QCw==; 24:dfrc3vLJO6LgcjqBHyMbemFpBBHSkB/KsjySFUbLMhzcfDij5izh0/hXUpGdzQqIN84LJO3efNKrkrI4AVs97A4vmcZK93W1SWX3o4d6f2U=; 7:XZ5lCX3pTeFxcJeu88pvP4sAUrxiLbo62ZmBCPEoonpOzcI9u0tRGYcb9fwc5dG6c/YbyugaTVwLg/gjYbRHvfmXB11Hq83lP/uAIMGWsllWac21HGLdK9W9UuDENVjIdbMpwsSHVDJWe5y9RuPCl0fhcXwilKLQT8v5OOhCCU6talR2uJj9UTv48rvtN/Eo1muJBQpX+waxEorcGlBa3p2n9xm1K8CSCZxUTQEblI8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2017 03:40:51.7013 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.12]; Helo=[p-emfe01a-sac.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR0501MB3845 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Aug 2017 03:40:55 -0000 Daniel Eischen wrote: > >> Background: > >> > >> I've been adding what amounts to a mini "verified exec" to the freebsd > >> loader for use in Junos. > >> > >> What this means is that the loader verifies the kernel and all the > >> modules before loading them, and can reject anything for which a > >> registered fingerprint (eg. sha1 hash) does not match. > [ ... ] > > > > We need this exact feature (verification of kernel and modules) for an > > upcoming product at work. =C2=A0Including the library code in contrib > > certainly sounds attractive to me, too. > > > > I wouldn't be surprised if interest in this goes beyond those of us > > building embedded appliances. >=20 > Indeed, why couldn't it be enabled by default for FreeBSD.org > packaged distribs? Or am I jumping the gun by a few years? The problem with that, boils down to key management. As an embeded device vendor we totally controll the "trust anchors" and the keys used to sign things. And absent the signing, this is all pretty pointless. My loader for example has 3 Juniper root CA certs in its trust store, which it can use to verify any signature we have generated over the last 10+ years. But it will not accept anything signed by anyone else. That's perfect for an embedded device, not not exactly useful for a stock system. That's why I said this is all mostly likely of interest to embedded vendors - since the generic case is much harder ;-) Now, there is absolutely nothing to stop anyone/everyone doing as we did, and setting up their own X.509 hierarchy, and the signing server we use (freely available from crufty.net) helps a lot with keeping private keys private even in a company with 1000's of people signing stuff. And perhaps FreeBSD.org could sign releases with their own keys. But if you want to build your own modules you need a way to sign them such that your loader will accept them. For something like the loader, an embedded trust-store is a must IMO. But there's no way you could classify this as a zero effort thing. Still, with all that said, I currently have the loader defaulting to a "best effort" mode - where it will attempt to verify everything, but won't get upset if there is no fingerprint for some file - it will tell you (unless it was loader.conf - which is more or less expected to be mutable), though it will not accept a fingerprint miss-match. This lets me experiment with various platforms etc without bricking lots of boxes (makes the test folk unhappy). So you can boot using a verifying loader without everything signed just fine. The behavior is of course tunable from "off", to "strict". --sjg