From nobody Mon Apr 15 00:06:16 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VHnX95x6sz5HLb9 for ; Mon, 15 Apr 2024 00:06:33 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VHnX85sbtz465D for ; Mon, 15 Apr 2024 00:06:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a51b008b3aeso310058066b.3 for ; Sun, 14 Apr 2024 17:06:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713139590; x=1713744390; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=b0IMDOz2KVBZcWavPie2Cas7+k0Jzb8uUyxrnzDgQcw=; b=Dkt3AKfmWoxP2Y5xZV1GmUJviscvxgodQTmyFgI4JPCoogSETV59MFFHZNOfACF5P5 ZFB0TZccxsrG+kFT+1dbQfL61YAW1V4isaJ5hkKHBiRo9rI4hB8gd6fTMc/HqzaWVR+a 0O2M+h5XIPYjRXf0XUCRPzEdSdAQFgti0j67V3RII9i0tvelMgbkdUIvsdU6YeDmH+p6 eDI7RN8foag18Lld4Dc3nE49HyZ8M79i458KU4mlmr+b009GoUIPXzDLDJGb1ZA43Rx0 PA+fluIVz9FROUbMyn0E7C3CxqQQD6Bmkq2pKzEVjOk6KTyzvzqIW5ulLGHCrrfkAlJz eIzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713139590; x=1713744390; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b0IMDOz2KVBZcWavPie2Cas7+k0Jzb8uUyxrnzDgQcw=; b=XskQNFDVV2FgQeeblUeitvde5t+YLdHdtFPUVLNn45+beMwuwNpepymgJxeLHhcTiP 5flZJ1ZzCUjs0f7sHNNRJneClNwEjrLgGcHsp0GD53+dj/JLksa53eNtzS4PDXkI666B ps/6/i7JE7XdL/Vge+/H6QYxCYMx1YHMpDWV7uQN7cEQ7eSLYYA/IpLCGJv4U7ltAI9r lmjwq2/gmVPf5TGd0RLsokD5p2BL2Xgb1TPSu5L4sZWo9pi8L0ImZ1QT9vZQIz6g52MC 8JydqpBN4w8WQC75LzP9TvDhtnKl7bPzSVwWaiXOp3VEWuBib5ejdkxlM2mNiZilCrGd YPZA== X-Gm-Message-State: AOJu0Yzt3JnR78FMc6/VeODze1oT0PMkHFkS+mN3PbOz4W8on65/MQQH gjPDz8E+BwKtraCAvJSJaWUmavDvzUt7GACrGkzIuQgQKnWm3E5FcUXsk0WYdj0rMIE99ubtoRp UEsZgfdGrZ8CY34ywpkpMp77qsjAstRf6NcfBJaOfYc+nWHzs X-Google-Smtp-Source: AGHT+IFDyaVBnNbYIwv5DGQMUm/hN31b0QsMTgSAt2BnH0WWohXulI6qARPSvqihepCgVigyhRWEtbz8c5HMRFjoyo0= X-Received: by 2002:a17:907:770b:b0:a52:5460:a1c6 with SMTP id kw11-20020a170907770b00b00a525460a1c6mr3101349ejc.48.1713139589356; Sun, 14 Apr 2024 17:06:29 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sun, 14 Apr 2024 18:06:16 -0600 Message-ID: Subject: Re: Question regarding crunchgen(1) binaries To: Shawn Webb Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000acb33e061617646b" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VHnX85sbtz465D --000000000000acb33e061617646b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Apr 14, 2024, 5:43=E2=80=AFPM Shawn Webb wrote: > Hey FreeBSD Hackers, > > Note: I originally posted this to the HardenedBSD users mailing list. > I'm posting to freebsd-hackers@ to hopefully learn from a wider > audience. > > I wanted to ping the HardenedBSD community, asking about the > usefulness of crunchgen(1)-built applications in 2024. > For FreeBSD they are quite useful still. The HardenedBSD community can make up its own mind. >From the crunchgen(1) manual page: > > > The main reason to crunch programs together is for fitting as many > > programs as possible onto an installation or system recovery floppy. > Floppy is antiquated. /resur hasn't fit on a floppy since FreeBSD 3 or so. Yet, it's still useful to have a tiny ramdisk that's either recovery or simple servers only. The binaries in /rescue are built with crunchgen. It seems that > crunchgen-built applications are not (currently) compatible with a > libc built with LTO due to the recent CSU and libc changes. > PR? Seems is aweful vague. The size of the binaries in /rescue on HardenedBSD 15-CURRENT/amd64 > are 17MB in size. That application size alone makes it impossible to > build a "system recovery floppy". Additionally, floppy drives aren't > all too common on the amd64, arm64, and riscv64 systems HardenedBSD > targets. > Don't take floppy litterally here. Control Flow Integrity (CFI) is a compiler-based exploit mitigation > that we apply to applications in HardenedBSD 15-CURRENT and 14-STABLE. > In order to apply CFI to applications, application code must be built > with Link Time Optimization (LTO). > > Over the past few years, I've slowly been working on applying CFI to > shared objects (aka, Cross-DSO CFI). This requires building library > code with LTO as well. > > It seems that with the recent changes to the CSU and libc, the > crunchgen(1) built tool does not produce workable applications when > libc is built with LTO. With libc having such a huge surface area, it > would be prudent to apply Cross-DSO CFI to it. > PR? This presents two possible solutions: > > 1. Enhance crunchgen(1) to support libc built with LTO. > 2. Kick crunchgen(1) to the curb. > 3. Other ideas from the community are possible. > > Does anyone find crunchgen(1) to be truly useful in 2024? If we kick > crunchgen(1) to the curb, we need to modify the build system for > /rescue binaries. > > My own preference would indeed to rid ourselves of crunchgen(1) so > that we can progress towards applying Cross-DSO CFI and LTO to libc. > /rescue is still the last line of defence against botched libc updates. We use it for rare cases where /usr isn't mounted yet and moving binaries is too hard in rc. Crunchgen'd binaries make the cost trivial. Plus, I have several appliances that are a RAM disk with one crunchgen binary. It makes deployment easy, but I know I'm stongly opposed to kicking crunchgen to the curb. It's still quite useful and the reasons proffered are vague and seem little more than LTO bugs... So I'm for (3) fixing LTO to not suck. The attack mitigation is a good feature, but it's not worth killing resue over... But my comments are explicitly in a FreeBSD context. Warner Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --000000000000acb33e061617646b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Apr 14, 2024, 5:43=E2=80=AFPM Shawn Webb <<= a href=3D"mailto:shawn.webb@hardenedbsd.org">shawn.webb@hardenedbsd.org= > wrote:
Hey FreeBSD Hackers,
Note: I originally posted this to the HardenedBSD users mailing list.
I'm posting to freebsd-hackers@ to hopefully learn from a wider
audience.

I wanted to ping the HardenedBSD community, asking about the
usefulness of crunchgen(1)-built applications in 2024.

For FreeBSD they are = quite useful still. The HardenedBSD community can make up its own mind.

=
>From the crunchgen(1) manual page:

> The main reason to crunch programs together is for fitting as many
> programs as possible onto an installation or system recovery floppy.

Fl= oppy is antiquated. /resur hasn't fit on a floppy since FreeBSD 3 or so= . Yet, it's still useful to have a tiny ramdisk that's either recov= ery or simple servers only.

The binaries in /rescue are built with crunchgen. It seems that
crunchgen-built applications are not (currently) compatible with a
libc built with LTO due to the recent CSU and libc changes.

PR? Seems is awe= ful vague.

The size of the binaries in /rescue on HardenedBSD 15-CURRENT/amd64
are 17MB in size. That application size alone makes it impossible to
build a "system recovery floppy". Additionally, floppy drives are= n't
all too common on the amd64, arm64, and riscv64 systems HardenedBSD
targets.

Don't take floppy litterally here.
Control Flow Integrity (CFI) is a compiler-based exploit mitigation
that we apply to applications in HardenedBSD 15-CURRENT and 14-STABLE.
In order to apply CFI to applications, application code must be built
with Link Time Optimization (LTO).

Over the past few years, I've slowly been working on applying CFI to shared objects (aka, Cross-DSO CFI). This requires building library
code with LTO as well.

It seems that with the recent changes to the CSU and libc, the
crunchgen(1) built tool does not produce workable applications when
libc is built with LTO. With libc having such a huge surface area, it
would be prudent to apply Cross-DSO CFI to it.
=

PR?
This presents two possible solutions:

1. Enhance crunchgen(1) to support libc built with LTO.
2. Kick crunchgen(1) to the curb.
3. Other ideas from the community are possible.

Does anyone find crunchgen(1) to be truly useful in 2024? If we kick
crunchgen(1) to the curb, we need to modify the build system for
/rescue binaries.

My own preference would indeed to rid ourselves of crunchgen(1) so
that we can progress towards applying Cross-DSO CFI and LTO to libc.

/rescue= is still the last line of defence against botched libc updates. We use it = for rare cases where /usr isn't mounted yet and moving binaries is too = hard in rc.

Crunchgen= 9;d binaries make the cost trivial. Plus, I have several appliances that ar= e a RAM disk with one crunchgen binary. It makes deployment easy, but I kno= w=C2=A0

I'm stongly = opposed to kicking crunchgen to the curb. It's still quite useful and t= he reasons proffered are vague and seem little more than LTO bugs...
<= div dir=3D"auto">
So I'm for (3) fixing LTO = to not suck. The attack mitigation is a good feature, but it's not wort= h killing resue over...

= But my comments are explicitly in a FreeBSD context.

Warner



Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubk= eys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.as= c
--000000000000acb33e061617646b-- From nobody Mon Apr 15 01:05:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VHprN2dnXz5HQwn for ; Mon, 15 Apr 2024 01:05:40 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4VHprM72Gyz4G51 for ; Mon, 15 Apr 2024 01:05:39 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-hackers@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 43F15VLX068211; Mon, 15 Apr 2024 02:05:31 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 43F15VoL068210; Mon, 15 Apr 2024 02:05:31 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202404150105.43F15VoL068210@donotpassgo.dyslexicfish.net> Date: Mon, 15 Apr 2024 02:05:31 +0100 Organization: Dyslexic Fish To: shawn.webb@hardenedbsd.org, freebsd-hackers@FreeBSD.org Subject: Re: Question regarding crunchgen(1) binaries References: In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Mon, 15 Apr 2024 02:05:31 +0100 (BST) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Queue-Id: 4VHprM72Gyz4G51 Shawn Webb wrote: > 1. Enhance crunchgen(1) to support libc built with LTO. > 2. Kick crunchgen(1) to the curb. > 3. Other ideas from the community are possible. > > Does anyone find crunchgen(1) to be truly useful in 2024? If we kick > crunchgen(1) to the curb, we need to modify the build system for > /rescue binaries. Please note, my response is not considering the security aspects you raise, and is only based on the usefulness of /rescue itself. Do you mean get rid of /rescue, or just getting rid of crunchgen producing it? I've been "rescued" by rescue on more than one location - usually systems that won't mount /usr and also have a screwed up lib. I wouldn't want to see a static /rescue disappear, and the size would probably be too large for individual binaries. Cheers, Jamie From nobody Mon Apr 15 14:06:17 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VJ8985whYz5GfSF for ; Mon, 15 Apr 2024 14:06:20 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VJ8983qf6z4YN2 for ; Mon, 15 Apr 2024 14:06:20 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-36a118d3b30so20193005ab.2 for ; Mon, 15 Apr 2024 07:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1713189979; x=1713794779; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=7LD6uTjGt9XkuWOpfEzItCjnkZNNlJhd00ne+FhfOI8=; b=NiE/eesceuRboKqLtGcfkUEbAb/NbXFGuOcecbOfmr65p77m5gBD6rOP1o5lE4v2PL YBAslXALyaT0Gk6Mn2VInzgYysRYvKiIteKw3QJflSHlt9Jv+9CVwQC++RI1xrKMGMve xMU+hd2DgAIjTT866Ty6QMKdgh2GzR2ivTyLJCNEiU4NBOoe7lMnVLu/1CIkTGODzogL HAi22tZvCR/VPa3z7W+keSCthaNT93GzLDI1hSur97Flg3bPzXQz410t3Pv9lpb1xts+ M+Grto8F6x4xwSkUQqMLtRX2PravzmM2AyAJpt8Nqok2apdL4hD75bd6ZK4U3FD6rGYF WtHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713189979; x=1713794779; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=7LD6uTjGt9XkuWOpfEzItCjnkZNNlJhd00ne+FhfOI8=; b=NoqFEjAF0KpaLLFYUBJA+MQq8lbbjz21BqYoLXoYxlhV5tT7GnFkm0bdiacFAcc2Na Fh6i0skKz9L3LcjRTtU1oZNu2svccYjIgdA0eLncBNCFHg1ouKl1QQSHMFek6Z71qzBU bzX9o9zV9S4Nbd2uKGewazaPbWNR6bENK0f5z5t6sHaBxcLdB4xSAJDHAlTr0daijDAS bFqxBBH2MbCZw/NrMRzbVkpHYvldvQNvzZ6E3+KMIrLVdvnVXIzJ1OWRc6mAvTaoIkN3 Uof5K9/yQaYX8AkMDNCYJnBA3pRDz1hwK9mZ/8ir1K+VNbsgYE1Zh/kKB5X/LvBHVhcs iI7w== X-Gm-Message-State: AOJu0YyNJFn7W1ERa3N8JrXlNyfMG5cn6t2tU40VqG9j+xqVO1CRLngT RHa6lvPEQo2Yh+0ZXvhhew3s/kfs9Y8D/IF+F/HQ4IqKMa8SRHL1Y9T90essl4JyxX2qbXlIc21 1 X-Google-Smtp-Source: AGHT+IHAjx+iACXFwGtsJrLGtelnpB7UC3lmnbcL19MiaNvRJ3hvtNzGEvqolMGm34LUYQR8j2nN+w== X-Received: by 2002:a05:6e02:1609:b0:36a:2934:a8e5 with SMTP id t9-20020a056e02160900b0036a2934a8e5mr11505733ilu.9.1713189978797; Mon, 15 Apr 2024 07:06:18 -0700 (PDT) Received: from mutt-hbsd ([184.99.37.29]) by smtp.gmail.com with ESMTPSA id s15-20020a056e02216f00b0036afaeb2c95sm2658587ilv.50.2024.04.15.07.06.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Apr 2024 07:06:17 -0700 (PDT) Date: Mon, 15 Apr 2024 14:06:17 +0000 From: Shawn Webb To: Jamie Landeg-Jones Cc: freebsd-hackers@freebsd.org Subject: Re: Question regarding crunchgen(1) binaries Message-ID: <7lmqszm7n35b5jitwvzagmlc2lecl6p3dhu2bnhri4unnjtlow@f5txrntbo7yw> X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <202404150105.43F15VoL068210@donotpassgo.dyslexicfish.net> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="4fnylkubg4nkpsqf" Content-Disposition: inline In-Reply-To: <202404150105.43F15VoL068210@donotpassgo.dyslexicfish.net> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VJ8983qf6z4YN2 --4fnylkubg4nkpsqf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 15, 2024 at 02:05:31AM +0100, Jamie Landeg-Jones wrote: > Shawn Webb wrote: >=20 > > 1. Enhance crunchgen(1) to support libc built with LTO. > > 2. Kick crunchgen(1) to the curb. > > 3. Other ideas from the community are possible. > > > > Does anyone find crunchgen(1) to be truly useful in 2024? If we kick > > crunchgen(1) to the curb, we need to modify the build system for > > /rescue binaries. >=20 > Please note, my response is not considering the security aspects you rais= e, > and is only based on the usefulness of /rescue itself. >=20 > Do you mean get rid of /rescue, or just getting rid of crunchgen producing > it? I recognize now that the way I phrased things left room for ambiguity. I apologize for the ambiguity. We do indeed want to keep /rescue around. I still have the occasional use for it, as do many others. The only thing that would change would be that the applications in /rescue would be regular statically-linked executables. We would stop using crunchgen(1) to produce those executables. >=20 > I've been "rescued" by rescue on more than one location - usually systems > that won't mount /usr and also have a screwed up lib. >=20 > I wouldn't want to see a static /rescue disappear, and the size would pro= bably > be too large for individual binaries. There are around 148 files in my 15-CURRENT/amd64 /rescue. The size would likely baloon quite drastically. I think I will likely determine the level of effort to fix crunchgen(1) to work with LTO-ified libc. I might base my decision off that. Meanwhile, if anyone else has any info to pass along that could help in this journey, I would very much appreciate it. This touches bits that have a lot of history, and this is definitely a blind spot of mine. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --4fnylkubg4nkpsqf Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmYdNE8ACgkQ/y5nonf4 4fqqKQ//Y8RShCks4+t6k4HHpLCDn1zYMGa+QCQihbdnabWysDanLudtXLg8l91O QhLAlqwo7HKRwB7TwwORSlLR0XxgGxvK0xjIdQ3nRvV/IopXFmfVEMU/adjsiqZI qjHrkHCn5ws225DtaKOCdkDobNG6WZJsmpbmFQT0kSccZUn0WVqGpJ9i9g4fWS0t +OwAlVanCzokFeN0TJKLA752eSji5chTX+OcErt4OhsPoRteF6dBXHI8a7fkcATC cQXpvgUVpl/NPnY6t9ZNVfLuuFDGJjUmNdmwrzwWjmANynfs+iDi1gfaTNQ0/fyE 3q1pIRCvk+bgcYDNOlePzQt/6T6o6CCEehwlNOjvdn7bHOQwRuC0JKlWzWL4lHIZ 54U6Db9PmOQ9c8Vd/nVLUv/ffRA9zyi/Fbt3Bxa82xh5nnBH5sh6DSy7PgJ/d+S1 Pk2XdWPDAc6EzSx879QMhM9kDmY1z8kbvZX2Auwa+adWkb3szKkIPPiv3poqqYfI iBoDg/HqMqNHSU48YEtJ4O74Rop7MvHRBEyNLPeZGZlTFvgElIQCAv4KUH8ggdM8 UiGYTHRXHfuEOg4VDgWj6FKTJEXJ8bY79HTOBhsWI4wuh9gwbrSdz/8q24mx64hk 0qjL7N3YUXDtZh4RwQd2SMJ/QpA0i+JnCI9Rd2bDnPrew33o04Y= =aG7e -----END PGP SIGNATURE----- --4fnylkubg4nkpsqf-- From nobody Mon Apr 15 15:45:59 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VJBNQ3JLZz5GqRC for ; Mon, 15 Apr 2024 15:46:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VJBNQ02Jfz4pwm for ; Mon, 15 Apr 2024 15:46:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x532.google.com with SMTP id 4fb4d7f45d1cf-565c6cf4819so7278057a12.1 for ; Mon, 15 Apr 2024 08:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713195972; x=1713800772; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=f5PqBtXMXyHNFqhdzrIYM8sfOQw5bXkNFLK23tNh464=; b=j0s5sGs0OSyZNM+hLpPoYKPbrZ6D/TzY7+qeogsoMuEYGzE/mKVkXHVKNTroKVK76P zYsQojf7Tr8UqiY3cPkYYUQ9lyY7r13FdIGwSE2jFVO0TCUOgf/SYukYFWNpTaGs4+K7 OLl7sX7R6895x3U6KP79ILnDMUgg+dXPWLs8xIN1qLFb8HKELWibcHFKwh/JS4F79FBJ 64BlhRwty0T1IDomxbP16geyZq+ii0F1mk6Y+Z9buipwv6Nt35T0c36C/Z7qqcprR1+O 35/H301JunyyaJCIfuHe0dLNY/gTwO84oo53KsTlyDOLbeQqvg5nGqvmXbwZqJiYfz9x Au6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713195972; x=1713800772; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f5PqBtXMXyHNFqhdzrIYM8sfOQw5bXkNFLK23tNh464=; b=jWqMuJ3sDuOiGbgn8XCz7pejm265abd2Ywsp/wr93xk7FfbpVNFEWmGqB38vUXWVrL 8YdDXTilzFNt5cqWq4qBUCEcee1asPQOgVdosHoUyGSP/t5xvwwSMUZhQzLU8JyT/lpn Cp+haf2D/mRYuWYnEFftZ8LZP9xEuDe+F7uj+NLTbNQt+PjJAVpfEcorGDS8OvBiJO4R 8OAFMYd6MuVGmvD3NHHHJUZiw913cukPXNUWdViO83XEk0tS41keAZU9oyTI2sunK5WX nHiQZBbXeiSN9ZpBIC5+dPYfwr/L0EZqtiW0+rfR3uC5fio0H8xVB4wyVmf9yJ8cMGxY v9pA== X-Forwarded-Encrypted: i=1; AJvYcCV1lZ6bvN/VA4RPhESU1cNvUnU3x/GtYzTcLdT58jzVLIp74voJFEcdtz5f8quVFWVogKovfUBEnlPYLk9DxbMYGGhfU1d7qgX+lZU= X-Gm-Message-State: AOJu0YxVtJUq4IPAlikPgHOOcgCQe81/WaxQdJuPQdmghesk4MZGZM1c LhHTKP64c9xeUCST9MC5sElpr3p3Bc3rcgAaBICiaaL24QZeEOXR4m3ZlGCSFYS/L+zRuWHOPNS Z8FtFJjsAsa0CdMWKlK2dY7pBpa0lU3au4VSBF1OR3kGBVeu1ARQ= X-Google-Smtp-Source: AGHT+IEo6UYr16Vi/E69qC5bNGwqPYAdVm3RjxyONQQ4Gclm79/9D1caK79joNeae+8dBNOx5EUThZkEydizekjJDPg= X-Received: by 2002:a17:907:7d8d:b0:a51:f46a:b000 with SMTP id oz13-20020a1709077d8d00b00a51f46ab000mr57066ejc.20.1713195971803; Mon, 15 Apr 2024 08:46:11 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202404150105.43F15VoL068210@donotpassgo.dyslexicfish.net> <7lmqszm7n35b5jitwvzagmlc2lecl6p3dhu2bnhri4unnjtlow@f5txrntbo7yw> In-Reply-To: <7lmqszm7n35b5jitwvzagmlc2lecl6p3dhu2bnhri4unnjtlow@f5txrntbo7yw> From: Warner Losh Date: Mon, 15 Apr 2024 09:45:59 -0600 Message-ID: Subject: Re: Question regarding crunchgen(1) binaries To: Shawn Webb Cc: Jamie Landeg-Jones , FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000005494690616248505" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VJBNQ02Jfz4pwm --0000000000005494690616248505 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Apr 15, 2024, 8:07=E2=80=AFAM Shawn Webb wrote: > On Mon, Apr 15, 2024 at 02:05:31AM +0100, Jamie Landeg-Jones wrote: > > Shawn Webb wrote: > > > > > 1. Enhance crunchgen(1) to support libc built with LTO. > > > 2. Kick crunchgen(1) to the curb. > > > 3. Other ideas from the community are possible. > > > > > > Does anyone find crunchgen(1) to be truly useful in 2024? If we kick > > > crunchgen(1) to the curb, we need to modify the build system for > > > /rescue binaries. > > > > Please note, my response is not considering the security aspects you > raise, > > and is only based on the usefulness of /rescue itself. > > > > Do you mean get rid of /rescue, or just getting rid of crunchgen > producing > > it? > > I recognize now that the way I phrased things left room for ambiguity. > I apologize for the ambiguity. > > We do indeed want to keep /rescue around. I still have the occasional > use for it, as do many others. > > The only thing that would change would be that the applications in > /rescue would be regular statically-linked executables. We would > stop using crunchgen(1) to produce those executables. > I'm going to say what others have said privately: this is a non-starter and has no support at all. We are not going to stop using crunchgen unless there is a viable alternative to do the same thing. > > > I've been "rescued" by rescue on more than one location - usually syste= ms > > that won't mount /usr and also have a screwed up lib. > > > > I wouldn't want to see a static /rescue disappear, and the size would > probably > > be too large for individual binaries. > > There are around 148 files in my 15-CURRENT/amd64 /rescue. The size > would likely baloon quite drastically. > > I think I will likely determine the level of effort to fix > crunchgen(1) to work with LTO-ified libc. I might base my decision off > that. > > Meanwhile, if anyone else has any info to pass along that could help > in this journey, I would very much appreciate it. This touches bits > that have a lot of history, and this is definitely a blind spot of > mine. > So far all you've said is they appear not to work. Sounts like an LTO bug in llvm. My advice: start with a crunchgen'd cat or hello world and see if you can at least produce a test case that's small and manageable. You can submit that upstream to see if they can fix it. Or you can chase down in gdb where this goes off the rails. At a wild guess, though, you are talking about adding a security package that makes things safe somehow. That's typically with symbol redirection. Maybe start there to understand what "LTO" the security thing is doing and why it's either wrong or violates an assumption in crunchgen that can be fixed. Warner Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --0000000000005494690616248505 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Mon, Apr 15, 2024, 8:07=E2=80=AFAM Shawn Webb <<= a href=3D"mailto:shawn.webb@hardenedbsd.org">shawn.webb@hardenedbsd.org= > wrote:
On Mon, Apr 15, 2024 at= 02:05:31AM +0100, Jamie Landeg-Jones wrote:
> Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
>
> > 1. Enhance crunchgen(1) to support libc built with LTO.
> > 2. Kick crunchgen(1) to the curb.
> > 3. Other ideas from the community are possible.
> >
> > Does anyone find crunchgen(1) to be truly useful in 2024? If we k= ick
> > crunchgen(1) to the curb, we need to modify the build system for<= br> > > /rescue binaries.
>
> Please note, my response is not considering the security aspects you r= aise,
> and is only based on the usefulness of /rescue itself.
>
> Do you mean get rid of /rescue, or just getting rid of crunchgen produ= cing
> it?

I recognize now that the way I phrased things left room for ambiguity.
I apologize for the ambiguity.

We do indeed want to keep /rescue around. I still have the occasional
use for it, as do many others.

The only thing that would change would be that the applications in
/rescue would be regular statically-linked executables. We would
stop using crunchgen(1) to produce those executables.

I'm going to say what others have said privately: this is a non-sta= rter and has no support at all. We are not going to stop using crunchgen un= less there is a viable alternative to do the same thing.


>
> I've been "rescued" by rescue on more than one location = - usually systems
> that won't mount /usr and also have a screwed up lib.
>
> I wouldn't want to see a static /rescue disappear, and the size wo= uld probably
> be too large for individual binaries.

There are around 148 files in my 15-CURRENT/amd64 /rescue. The size
would likely baloon quite drastically.

I think I will likely determine the level of effort to fix
crunchgen(1) to work with LTO-ified libc. I might base my decision off
that.

Meanwhile, if anyone else has any info to pass along that could help
in this journey, I would very much appreciate it. This touches bits
that have a lot of history, and this is definitely a blind spot of
mine.

So far all you've said is they appear not to work. Sounts like an = LTO bug in llvm.

My advi= ce: start with a crunchgen'd cat or hello world and see if you can at l= east produce a test case that's small and manageable. You can submit th= at upstream to see if they can fix it. Or you can chase down in gdb where t= his goes off the rails.

= At a wild guess, though, you are talking about adding a security package th= at makes things safe somehow. That's typically with symbol redirection.= Maybe start there to understand what "LTO" the security thing is= doing and why it's either wrong or violates an assumption in crunchgen= that can be fixed.

Warn= er


Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubk= eys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.as= c
--0000000000005494690616248505-- From nobody Mon Apr 15 19:55:22 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VJHw63psJz5HD12 for ; Mon, 15 Apr 2024 19:55:34 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VJHw61Th0z4SKh for ; Mon, 15 Apr 2024 19:55:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id B7F0D89293; Mon, 15 Apr 2024 19:55:24 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.18.1/8.16.1) with ESMTPS id 43FJtOKT083781 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 15 Apr 2024 19:55:24 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 43FJtMnU083779; Mon, 15 Apr 2024 19:55:22 GMT (envelope-from phk) Message-Id: <202404151955.43FJtMnU083779@critter.freebsd.dk> To: Warner Losh cc: Shawn Webb , Jamie Landeg-Jones , FreeBSD Hackers Subject: Re: Question regarding crunchgen(1) binaries In-reply-to: From: "Poul-Henning Kamp" References: <202404150105.43F15VoL068210@donotpassgo.dyslexicfish.net> <7lmqszm7n35b5jitwvzagmlc2lecl6p3dhu2bnhri4unnjtlow@f5txrntbo7yw> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <83777.1713210922.1@critter.freebsd.dk> Date: Mon, 15 Apr 2024 19:55:22 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4VJHw61Th0z4SKh -------- Warner Losh writes: > Maybe start there to understand what "LTO" the security thing is doing and > why it's either wrong or violates an assumption in crunchgen that can be > fixed. Crunch binaries were invented 30 years ago, to make FreeBSD installation program fit on a single floppy disk. Note that the goal was saving disk-space rather than RAM. The "architecture" of crunchgen is to take a lot of programs, rename their main() and link them all together with a new main() which dispatches to the right program's main() based on argv[0] Statistically you save half a disk-allocation unit for each program which was nothing to sneeze at, but the real disk-space dividend comes from linking the resulting combi-program static. Because it is linked static, only those .o files which are referenced gets pulled in from the libraries, libm::j0.o only gets pulled in if you Bessel functions, which, countrary to rumours, sysinstall did not. (The goal of shared libraries is saving RAM: Everybody gets the complete library, but only one copy of it's code ever gets loaded.) But the real trick is actually not crunchgen, which was originally just a shell script, but rather crunchide(1). Crunchide(1) does unnatural acts to an objectfile's symboltabel, to get around the fact that all the programs have a function called "main" and that they litter the global symbol namespace with their private inter-file references. To make a crunched binary, the .o files for the individual programs are first "pre-linked" without libraries so that internal interfile references are resolved. Then crunchide changes all global symbols, except "main" to be local symbols, so that they become unavailable for symbol resolution in the final run of the linker. The "main" symbol is also renamed to a per-program name, something like "cp_main" for cp(1) etc. And then all the prelinked .o files, one per program, gets linked together with the "dispatch main" and this time with libraries. I see no reason why crunchgen cannot be done with Link Time Optimization, but somebody has to write the new crunchide(1), and I suspect it will have a tougher row to hoe, because pre-linking cannot be used to take care of the inter-program symbols. As I understand it LTO can also link with "normal libraries" so one option might be to only LTO the final linking step of the crunch process, treating all the programs as "normal libraries", but still getting LTO advantage internally in the libraries. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Tue Apr 16 20:15:27 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VJwJj28bDz5H0gp for ; Tue, 16 Apr 2024 20:15:33 +0000 (UTC) (envelope-from paulf2718@gmail.com) Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VJwJh20Tnz444f for ; Tue, 16 Apr 2024 20:15:32 +0000 (UTC) (envelope-from paulf2718@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=bu2fyyq1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of paulf2718@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) smtp.mailfrom=paulf2718@gmail.com Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-343e46ec237so3875950f8f.2 for ; Tue, 16 Apr 2024 13:15:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713298529; x=1713903329; darn=freebsd.org; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=RfvAYVUb55Spl33GFpFCuohWsIsLOFTVrwHyS3P0KJU=; b=bu2fyyq1csbWKkCd8C/t1RJqOPJyjNjDkHPYaBt6EK9+lLws5UkmBuQEjxAK2DMmTd PqEM325UFvGuB+Vl39FkcuT2kukuu8cKRljwgWQ7NLDO/kgPuXtB87VLwLL0eZepCkIT 2d/LAS0JLCA5Mjlpv3kvARZTMEouNKPfxgNolG6ixhj5L3wB3wdXYztpu8MdDPnlZUDh HR4UpzUephVlAc/+gk9haPAEwrNkG3sjFj1ab38iJXqUu/k8g7Ybw2sy+zs4iTQZD1We 0/txfxZs1+7koO3erZZ0HR4/whrgBCgC3QMP6rxg75vguXuIrCYkogyPRYGTKgGVMLr4 bbYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713298529; x=1713903329; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=RfvAYVUb55Spl33GFpFCuohWsIsLOFTVrwHyS3P0KJU=; b=qUaAgNUlAN+q3u1rIel6qCrCnGISweDGA//mdlUhFl+N0RuQkxicwcMKuDYqH65n+P ieX0O6MGTm6e043KFJok5XEItbiV3bZe53CPDrjpBN6Msk+pK0u0nJnYsPpRfC5D3x+R tZdgunxXJ85hthzdKLWlqpXQwK/adZgrBTkdy1c9PjZlOotnt0Vm2Bc0m0VHdMnI2la6 uOX21DESZ4u5VxhbmYT8bsB0RBbd0Pv8i1//8nDbsGFdM83QynWrOmaqyCzHpmL3GN0V fELw1+zJ4hwBseknWgceltfG+tjqS5G+YeJ42F1c4pgfisRg/CVK94ksNwlhnBmypj4T XqTw== X-Gm-Message-State: AOJu0YwE+jJFZg8D02I1xcV2cUapiTwJGH7lJhQG01APorpbUF+n0i/r UK3J+t4VIfW9iuI8urz2yOP3XvjUusaNsDRR4eXrrrNMZ9GxIVlQ4+20TA== X-Google-Smtp-Source: AGHT+IEqnt5HdlL1IaCnynnuGIAR4qdfMKzTUCx7Tiuh+3LGEGrg3C9BVAtECoATFt7ebVcGTwaIrg== X-Received: by 2002:adf:e38a:0:b0:343:2e09:58d1 with SMTP id e10-20020adfe38a000000b003432e0958d1mr10957349wrm.44.1713298529109; Tue, 16 Apr 2024 13:15:29 -0700 (PDT) Received: from ?IPV6:2a01:cb15:8010:2f00:1aa9:5ff:fe16:2efb? ([2a01:cb15:8010:2f00:1aa9:5ff:fe16:2efb]) by smtp.gmail.com with ESMTPSA id k9-20020adfe3c9000000b00344a8f9cf18sm15811094wrm.7.2024.04.16.13.15.28 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 16 Apr 2024 13:15:28 -0700 (PDT) Message-ID: <6f400ce0-b792-4ecf-ae6e-7f806a7c3d52@gmail.com> Date: Tue, 16 Apr 2024 20:15:27 +0000 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: freebsd-hackers@FreeBSD.org Content-Language: en-US From: Paul Floyd Subject: Valgrind on aarch64 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::42a:from] X-Rspamd-Queue-Id: 4VJwJh20Tnz444f Hi Valgrind now has support for FreeBSD aarch64. The code has been pushed upstream and I've opened a bugzilla item to update the port. Regards Paul From nobody Wed Apr 17 08:02:02 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VKCzw117xz5J6Kk for ; Wed, 17 Apr 2024 08:02:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VKCzw0Yz4z47DR; Wed, 17 Apr 2024 08:02:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713340924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9HEb7+JGSAFWugw9n2i8fj9oAcT5TxjB6QRAno+IGco=; b=IjIaQqHXhBf8AvXhE81ut7TRpmOzgKQyzSFSw5FnL52hzkb/ubH4T24q2PGFSsemZft1Xr SY151iB8XXXMD9az8NXqZe7gMUfC9+IS/Zbr6gYLWLL+tuEgEHKLTuIvOfBfISzKpaZBRr 35LjOKnk6hvUlWSm7GCvSLgtTGVHzMM1tdqKx41K8JAFfrQ7SoecR1DucUERf+IqvYegwO AN+lsVxFNb6l/OycbFQQ7EXXYDtK/66gTUSuOVVbbIdJ8uribeYMAOhQO4pu4saL+c/cRL h1uCgjmHfWWstzzvuCSPIY22KBx0uoLsNCrCFKXZjTVKnfNLBOZyZ4515x5KRA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713340924; a=rsa-sha256; cv=none; b=KRsXxeiQ+154aCBENEKCHcgc2uM5sbJsxcFeufc3JnU+bjobp5zRoLcwdkuP0MWnU08GQD uca8o6qToNvjPYtgpbcwz9QmDQlmZWYYVAaf3GUiE0/IMuzdMfJ9+HlEvwvjzLrM6yxgHX tosCtE2nn3uaENhAIbL7suSCzDtRwcUfDAkXcnxx/qONogzKF/F5XordDNqAGRIEtHopgP P9oq5qRYSPJCx6Svjoa/hyLTh1xb/1aZ+bXOg0OrA4clfmJCAETNn61yCQh6BanPv41cL6 dEiJSJh9xZRU26L3QTKP3RL9e++rlSp6LYAdC9vfZ1NT8R/IHLkSGS2CcEo6Zw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713340924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9HEb7+JGSAFWugw9n2i8fj9oAcT5TxjB6QRAno+IGco=; b=WqOvukIIqWWI7kp6PP9DudrVZzBbksQFgcUtzZonlqNEd4aKGiQDL2ArKHuwA72RGw4WFv 3imGPCt50Zp3PmOBPnUs+5TECwekjzZYO29Hd7W8zTFmsRUqZB1YyQjB5qn8TIvDFlM8Lm MDIAMDONL6i5mMlw6iNE78UVWy523eHYg1AuTNuuQNGCrGHkql0/yjt8wQkwAKIDsORp8p UxjPqRFuQbPoG2qCY2tqORsx/IXGDHd0vff2YExiT6+e90mYm/1/TjpyEoCV6BBQltniv4 ZkettYahXmS85aceVIhKVHv70d1+83ippSoMNBUe4tPsAgIeUJBpF1MeOdmF0g== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VKCzv6ZPNzVpr; Wed, 17 Apr 2024 08:02:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EFAC25E0AC; Wed, 17 Apr 2024 10:02:02 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Valgrind on aarch64 From: Dimitry Andric In-Reply-To: <6f400ce0-b792-4ecf-ae6e-7f806a7c3d52@gmail.com> Date: Wed, 17 Apr 2024 10:02:02 +0200 Cc: "freebsd-hackers@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <6A8C44D1-E595-4743-A91A-411866FCC209@FreeBSD.org> References: <6f400ce0-b792-4ecf-ae6e-7f806a7c3d52@gmail.com> To: Paul Floyd X-Mailer: Apple Mail (2.3731.700.6.1.1) On 16 Apr 2024, at 22:15, Paul Floyd wrote: >=20 > Valgrind now has support for FreeBSD aarch64. The code has been pushed = upstream and I've opened a bugzilla item to update the port Great news, thanks! That is = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D278393, right? -Dimitry From nobody Wed Apr 17 10:58:19 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VKHvZ42zQz5Gf5N for ; Wed, 17 Apr 2024 10:58:34 +0000 (UTC) (envelope-from paulf2718@gmail.com) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VKHvY56gZz4Tmj for ; Wed, 17 Apr 2024 10:58:33 +0000 (UTC) (envelope-from paulf2718@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=hDFI0dgS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of paulf2718@gmail.com designates 2a00:1450:4864:20::429 as permitted sender) smtp.mailfrom=paulf2718@gmail.com Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-346406a5fb9so4223485f8f.1 for ; Wed, 17 Apr 2024 03:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713351510; x=1713956310; darn=freebsd.org; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=NQvp/tz0I10X6OEow217CtJioziRj878+is4le8BQUA=; b=hDFI0dgSqyyhGl946tK16panZDfb0MHo/xSyhZJvY9Khrm/zzhx3Tz7UD+vadEpToO tsyR3s2evIWdrqIQ5XJfY5KnoIpwMWYCF8QQeiOAW0A0rGaNE2frmWUaDSbzUQV7vjZV 7MpevVb4OP5rkvuCL5j+tS+mvOBm4eWQTaqf1p0HvuQzx3dgK/PxXoeqGL+RRXqlSx5O lz50SKw+hWuWUGmPL7GtMc8opZtpvZ9clbSmUyAu3K95r9J4mepUhGHNdwGIhIhHiV7h /ljFjGstJHDCw2CGfNcYyVirGzkrwe3lXJtvNb0FVKTlTcl/o1AKZD5N5elhFpuD11hN g++w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713351510; x=1713956310; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NQvp/tz0I10X6OEow217CtJioziRj878+is4le8BQUA=; b=E1vhY70kg1HnfVpgaXujVkwsKtUBaw1q15Gi86nyA9AVUiALqiiIbzyH/OTgNbijBK 3ATVdvpKnWi4n9jg2/PyT2B3kkPb7ndlOkkNGQUsWpFrrJS/qzfsEWywg/h11uaOBVOq TyIcJmnZkBtJJuPb15E1xMwt+vzZ2LjvKb4areG6cSlLEcFrEPMJ22tlqU80ex8hcbqO +lRGx7uFrSesEfFds6iySkX7EQ9x6YdnwIU6gulJC8x+45FkL+3u6lCikiqlgQsn7kvJ PIQNK1+OS6dAzm4z1SobJ0Nt0mnO88g4kjopjeKLQ8xujJmcfk7q57TIBJT5gMFctb7J WeqQ== X-Gm-Message-State: AOJu0YwTNDSxFe76ayTxj4BcPvk6GsRqnlkMkLLsggKDJCOSe/AuRSL/ 5Z79f21x318c3kK71ERuLTnmhor8BL5bxdeVjxfjaW+eLCv59F6qtj4xMA== X-Google-Smtp-Source: AGHT+IFSwjfFm153+2+KHkQO1ny6Q3JC5XPzFL1i5wsAIg1XAnhWnSTE3zo9T+uW0Zt1Bx3TH+ybmg== X-Received: by 2002:a05:6000:2c1:b0:343:6f88:5de with SMTP id o1-20020a05600002c100b003436f8805demr9983962wry.44.1713351510297; Wed, 17 Apr 2024 03:58:30 -0700 (PDT) Received: from smtpclient.apple ([2a01:cb15:8010:2f00:1d71:3f4c:c06:fe5a]) by smtp.gmail.com with ESMTPSA id q4-20020adff504000000b0034635bd6ba5sm17137200wro.92.2024.04.17.03.58.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Apr 2024 03:58:29 -0700 (PDT) From: Paul Floyd Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Valgrind on aarch64 Date: Wed, 17 Apr 2024 12:58:19 +0200 References: <6f400ce0-b792-4ecf-ae6e-7f806a7c3d52@gmail.com> <6A8C44D1-E595-4743-A91A-411866FCC209@FreeBSD.org> To: "freebsd-hackers@freebsd.org" In-Reply-To: <6A8C44D1-E595-4743-A91A-411866FCC209@FreeBSD.org> Message-Id: <4816861D-9CF8-4367-9435-43EEBC6BF952@gmail.com> X-Mailer: Apple Mail (2.3731.700.6.1.1) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::429:from] X-Rspamd-Queue-Id: 4VKHvY56gZz4Tmj > On 17 Apr 2024, at 10:02, Dimitry Andric wrote: >=20 > On 16 Apr 2024, at 22:15, Paul Floyd wrote: >>=20 >> Valgrind now has support for FreeBSD aarch64. The code has been = pushed upstream and I've opened a bugzilla item to update the port >=20 > Great news, thanks! That is = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D278393, right? >=20 Yes, that=E2=80=99s the one. If anyone is really impatient then you can always build directly from = source. See the wiki https://wiki.freebsd.org/Valgrind for details. A+ Paul From nobody Wed Apr 17 11:53:25 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VKK731Y4Vz5GlT3 for ; Wed, 17 Apr 2024 11:53:35 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from slim.berklix.org (slim.berklix.org [94.185.90.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "slim", Issuer "slim" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VKK722F1tz4b4r for ; Wed, 17 Apr 2024 11:53:34 +0000 (UTC) (envelope-from jhs@berklix.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jhs@berklix.com has no SPF policy when checking 94.185.90.68) smtp.mailfrom=jhs@berklix.com Received: from dell.no.berklix.net (p4fc4ce15.dip0.t-ipconnect.de [79.196.206.21]) (authenticated bits=128) by slim.berklix.org (8.17.1/8.17.1) with ESMTPSA id 43HBrQNI096021 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Wed, 17 Apr 2024 13:53:26 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from dell.no.berklix.net (localhost [127.0.0.1]) by dell.no.berklix.net (8.16.1/8.16.1) with ESMTPS id 43HBrPX2033676 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 17 Apr 2024 13:53:26 +0200 (CEST) (envelope-from jhs@localhost.no.berklix.net) Received: (from jhs@localhost) by dell.no.berklix.net (8.16.1/8.16.1/Submit) id 43HBrPhj033675; Wed, 17 Apr 2024 13:53:25 +0200 (CEST) (envelope-from jhs) Message-Id: <202404171153.43HBrPhj033675@dell.no.berklix.net> To: "Poul-Henning Kamp" cc: FreeBSD Hackers Subject: Re: Question regarding crunchgen(1) binaries From: "Julian H. Stacey" Organization: http://berklix.com/jhs/ User-agent: EXMH on FreeBSD http://berklix.com/free/ X-From: http://www.berklix.org/~jhs/ In-reply-to: Your message "Mon, 15 Apr 2024 19:55:22 -0000." <202404151955.43FJtMnU083779@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <33673.1713354805.1@localhost> Date: Wed, 17 Apr 2024 13:53:25 +0200 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.09 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.988]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:33824, ipnet:94.185.88.0/22, country:DE]; FREEFALL_USER(0.00)[jhs]; HAS_ORG_HEADER(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DMARC_NA(0.00)[berklix.com]; R_SPF_NA(0.00)[no SPF record] X-Rspamd-Queue-Id: 4VKK722F1tz4b4r Hi, Reference: > From: "Poul-Henning Kamp" > Date: Mon, 15 Apr 2024 19:55:22 +0000 "Poul-Henning Kamp" wrote: > -------- > Warner Losh writes: > > > Maybe start there to understand what "LTO" the security thing is doing and > > why it's either wrong or violates an assumption in crunchgen that can be > > fixed. > > Crunch binaries were invented 30 years ago, to make FreeBSD > installation program fit on a single floppy disk. > > Note that the goal was saving disk-space rather than RAM. > > The "architecture" of crunchgen is to take a lot of programs, rename > their main() and link them all together with a new main() which > dispatches to the right program's main() based on argv[0] > > Statistically you save half a disk-allocation unit for each program > which was nothing to sneeze at, but the real disk-space dividend > comes from linking the resulting combi-program static. > > Because it is linked static, only those .o files which are referenced > gets pulled in from the libraries, libm::j0.o only gets pulled in > if you Bessel functions, which, countrary to rumours, sysinstall > did not. > > (The goal of shared libraries is saving RAM: Everybody gets the > complete library, but only one copy of it's code ever gets loaded.) > > But the real trick is actually not crunchgen, which was originally just > a shell script, but rather crunchide(1). > > Crunchide(1) does unnatural acts to an objectfile's symboltabel, > to get around the fact that all the programs have a function called > "main" and that they litter the global symbol namespace with their > private inter-file references. > > To make a crunched binary, the .o files for the individual programs > are first "pre-linked" without libraries so that internal interfile > references are resolved. > > Then crunchide changes all global symbols, except "main" to be local > symbols, so that they become unavailable for symbol resolution in > the final run of the linker. The "main" symbol is also renamed > to a per-program name, something like "cp_main" for cp(1) etc. > > And then all the prelinked .o files, one per program, gets linked > together with the "dispatch main" and this time with libraries. > > I see no reason why crunchgen cannot be done with Link Time > Optimization, but somebody has to write the new crunchide(1), and > I suspect it will have a tougher row to hoe, because pre-linking > cannot be used to take care of the inter-program symbols. > > As I understand it LTO can also link with "normal libraries" > so one option might be to only LTO the final linking step of > the crunch process, treating all the programs as "normal libraries", > but still getting LTO advantage internally in the libraries. > > Poul-Henning Interesting, Nice if some of that were added to man crunchide. Cheers, -- Julian Stacey. Gmail & Googlemail Fail http://berklix.org/jhs/mail/#bad Brits abroad reclaim http://StolenVotes.UK http://www.gov.uk/register-to-vote Arm Ukraine defence. Contraception reduces global warming & resource wars. From nobody Thu Apr 18 13:04:23 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VKyfX06Y0z5GqPp for ; Thu, 18 Apr 2024 13:04:36 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from HK3PR03CU002.outbound.protection.outlook.com (mail-eastasiaazon11021006.outbound.protection.outlook.com [52.101.128.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VKyfT1YhLz4FkB for ; Thu, 18 Apr 2024 13:04:33 +0000 (UTC) (envelope-from weh@microsoft.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=microsoft.com header.s=selector2 header.b="ddjDR/Gb"; dmarc=pass (policy=reject) header.from=microsoft.com; spf=pass (mx1.freebsd.org: domain of weh@microsoft.com designates 52.101.128.6 as permitted sender) smtp.mailfrom=weh@microsoft.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CfvXLINaZE8CX0TngYgQbQR6k1I6dv0mbjSTmlkQY/KNNaFWAXRenRsk5hiQxOEi2j5xKuB6ilgxGdWH9LUbyuIvZq90qYl4SK/OkVrVm3v4BYb7HeiZOIPoKfxM+Hu7Y6YY2eVd0SvR7GBpIUZHuz7ETLA9wI9iBktnn2+5Yirt+RAaGMh2H7C64jO/6Wk0zqeaNTpIHIP6hjTrwlhrhQAHgm4F3X6dpPsXZlIDgyF/CtHH/MhBg+flxnmqEzGk8/A4PGFLZ2gOldkwoI41erNUK2E+/7Xro7+kGcpZyVgWe6eYx1GOuGjCCiR9EewgCMdzPqdrXpei4T6qcLPi2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=99/SMcXwFQIHuIIRpZe5PInVi/XCncLn+EYa6AZ3pJs=; b=W+h8wfa1glNd+aLC5n9FWHRB76IK8FRRtYG88NUDaDgfkFw8BkbaZ0zI/n9VTj6NBoszLWieulpUG5OK7+qQkDrDGM8VrV9Hmr/jTXnfDvJSVoCxZDWoA8TztpD7vM7X4QRDoiVvfiRz0sgw6uP2De6+HWwqkNO94aRlP6izWGgwW1kAnfXLHjts4ldjbX7t11ixjLUAm0AH3OFIsadlbWU7YF99od2idd6WFq4opPb+JQShvSJZ+9SXzm/J1PWhZckzN76dlrv24WqQb4ULc7n4WxPzqoKI5bOmI8hw0zybHyCadp2bzOxej8c8NZ+fylZ1CNpyR1EVlG+WgHxs9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=99/SMcXwFQIHuIIRpZe5PInVi/XCncLn+EYa6AZ3pJs=; b=ddjDR/Gb0B81Vl4dZ9ipIxnkXhDd8j8V8xMixltsDvIXzWWGIHA6SXyYmuY0OedjmwyGc+loF0vCVXwhV4S1gO9IAH6jjLM06UhVPRg3zkPaTZi91qqlaeuilw4sHig978H9XCt59ClXc2R/VhTt+xed7/gUTFwLFsOfKNzhvlc= Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM (2603:1096:400:21::12) by TYZP153MB0705.APCP153.PROD.OUTLOOK.COM (2603:1096:400:259::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.10; Thu, 18 Apr 2024 13:04:23 +0000 Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e]) by TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e%7]) with mapi id 15.20.7519.010; Thu, 18 Apr 2024 13:04:23 +0000 From: Wei Hu To: "freebsd-hackers@FreeBSD.org" CC: Wei Hu Subject: How to add a -W flag in local Makefile Thread-Topic: How to add a -W flag in local Makefile Thread-Index: AdqRkOe7luhT44chS0qlVqfwwelwwg== Date: Thu, 18 Apr 2024 13:04:23 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=3e65ffce-ae26-44af-8246-f712289059b1;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2024-04-18T12:56:18Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYZP153MB0399:EE_|TYZP153MB0705:EE_ x-ms-office365-filtering-correlation-id: b07dd117-a817-4b5d-4882-08dc5fa811a9 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: =?us-ascii?Q?zD6Gdivkwg103ATWe5sf9DzpdIspXG3qNsRXhHqbLQTHequgpd8LJ+TYpJkA?= =?us-ascii?Q?4wYEdK6qOcyK5V2bCf3TeuAp8a9wiCEY7pZ8fV7XrlIetDkfJvC2S48iNX9a?= =?us-ascii?Q?yGegUSJojtUiKhtitLFdEO4LUJTTsQ83mmcKM31aEfAH0Kav28EXk8X4EOvS?= =?us-ascii?Q?xFjqpFhXd0AJWHIX/kJfW/rY9yCkO7KsmdmkhdVyHCMizAHsrnfHfR5/kBPD?= =?us-ascii?Q?pv0QngMqHS5GZmxnyL9za93QpdnwwT/xOMOBCnwlfSKwhCriEiWW2R17Kb+U?= =?us-ascii?Q?MomOrJh/UmvlmJ6ybJUqnxhepA+3ZL4pXRGm2D5iH0PdYvvnqisg0jjYWNX5?= =?us-ascii?Q?7pnwtW3IdcstygX5f1S1OllpISOD+IzCbbts1FxS513F4fTsq1AyAKh41IBU?= =?us-ascii?Q?bVq2NrTKTPoAcQ05tE701VG+QjCcmeIdiGsQsM/mlFIW6pgTVFatK/B4AXTp?= =?us-ascii?Q?u4HmxEdYTw86pIgTynV0j+Dun1JAvbHGeIIElxZawmeecVHKiR/iIWXdI4tB?= =?us-ascii?Q?M+ANSGaS9qDjbBGAq4Uz1Afu0ygBsYWOzqKeTEA7ZICkoUY+kEXjHs87XMOV?= =?us-ascii?Q?ox2j/WGj4lA7jC+Ac9EYTUV6Eq4U0RoZEZzWr+5i8x8PJua7jYmeNBoiXjm/?= =?us-ascii?Q?8jx+RFBrNPIN0uSgmgAvrzBqwJwV/DLOYKoRF13bx5v6uBmvg37bktrO+oVD?= =?us-ascii?Q?E1tHOYIggEBRoFdqT9peXsCs5FopyhxBU5MmYE3Wc2EwuqghApJmvXjaBwyZ?= =?us-ascii?Q?gVfbop4/jliqH6u/oemvbXRutScYJqpI/dAD3KqXXQALe0NbuW3O4mKyzwjB?= =?us-ascii?Q?MMShZoDM7n31YCXhXUFPIcVvpvHqq8SLoKCZ+bEhjtGH+YPk7fTKaZn7QmZv?= =?us-ascii?Q?UlLMY8WmJgDcHxWQAi6ruMCk3pWmlj3UNVgV8UoMYLxrfsTHsbfa+01mDoRJ?= =?us-ascii?Q?chxnQ/v+P60HkqxgR+4dX101LsxpxDhvCxPwJmM0euCkr4+2w5T9pk9x2Xi9?= =?us-ascii?Q?QYE5F1+39Bp3sxP6+78osXv7ltiJHoHqLqpPn4yei6j35q0qHg219ZXJkeal?= =?us-ascii?Q?a/beHKqFzjLNe7ehx4taeY+2+8gGyVCDqaG0vGG8G+MHqt6ZxOlceJDebHK9?= =?us-ascii?Q?Mq4nCjXXtEz90tOz3/YnTQD5NHw0PnKHjsYIpa0tQSrpmOvfnEN9GB5b95A9?= =?us-ascii?Q?4mMEoXQ8XqmiVp0CdMp9sbD31cFjM4E3E/CSRu6PqBIKE5Jbyc0sXx+/J9XB?= =?us-ascii?Q?UiVM+BRHD1I8ajV48P/9rVnPd3ltAINbGT7LsKkQf9bEpx0+WrM3UQ1G9/K5?= =?us-ascii?Q?TmxCoBuO75CcwiTWL7lzgAPojPrfaQcUwZ+O6bH0Rm5+Dg=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYZP153MB0399.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?0z3KX2WDccVzmNaQr1aiaeMY5/GGUmq45KLzobxTNiLgmOUvOF0ho7482iCw?= =?us-ascii?Q?J71B0jVsO2Uf6ft4xG1bSfhj8SETRrGN5CZlVPV7NEk890i1xk5UduV98x55?= =?us-ascii?Q?3Y4HP+Coq0GTWpcXJ7pmzFrRebsVFNy3ZVfR1DsL1qgKWpXwjqThAYDBPSFe?= =?us-ascii?Q?8F2yhuXhPnKEtS3zd4BI2Xp3ZxUCPNlY1McQQjZZG8rCPssVOaMizrUez7Vc?= =?us-ascii?Q?2c5g57KwaaYZ6xuwxA2OaLKTWnMhXiM1qbvKfOnVDGPHmJwlPOzcsXeaS366?= =?us-ascii?Q?66LKPFRel4upBX0AeEWWSXdaNwJgG/bx4cxz7C/TGXp2+w98cjNm0LT5fM79?= =?us-ascii?Q?hZ0/Mxh2XKET/g6b+Cq34YukpVS2EHd02GE77Ld2f8jE83hU2SLFJueZzDlQ?= =?us-ascii?Q?eNWB8kUkhTmvBhak2lSEL1nya+CnFKD0LoevRwIidTlk67q/jS04kdffc7qR?= =?us-ascii?Q?dZfY3dg9T6vEA75nuyIGRWQWgHzcpepBbXZdPuwjg3UewUbwOOPtnJeDC5o+?= =?us-ascii?Q?PqgRrpa9u9Wq/EKZ90+xUogiz9jltO9lFfq754A1NlL514erVOL6+5PnCMFk?= =?us-ascii?Q?ELPlAqLazT9bpGjd3gMgx134m9lJVv6FI7XAd5/o4nZc5Z6fH8XZ5Fy32G+d?= =?us-ascii?Q?yrgNdH5gM8tVNuM8IU55q36HPRC0156GXli9Wd5SsflE5dImapXzFVxrAjct?= =?us-ascii?Q?vRk6rbsyQ/kivAtOqerMEWDxYJg2T8E1MvKuENbeGgEQGZAoYifBYrgy37R4?= =?us-ascii?Q?qYLeEBgqSOITFiAprmPjBSZze1ZJd8PW/5GoMYeXbbb0neHCCMLDYXWrwoFD?= =?us-ascii?Q?IKjrFVGjI8yRV2FDlLCli0mfsffqDXh33V5T6UsRY2nrIaJEHQsjF60fsYHd?= =?us-ascii?Q?xARbx26C2lNZTAqo8h9PQa2KODKtSBwC2xojTZBkhCnKGtm8QmMJmX6p/Dwd?= =?us-ascii?Q?O/o23AOxf/YqZskiGFmO7zJSprE0r30c8GGN+RAZIjIpuIUVlAmGaPrko0wm?= =?us-ascii?Q?jJv/hneLpl4kRfYn0uqBYKrKUDuRpbnVdy2x/OmuF3vWdMb5s8PvNbS5orvZ?= =?us-ascii?Q?PBr+Y3pJonvc/fkDIdAEcGagXbLjrTrneHRNEaOMq85hy5LjRvxw/Cf/8qtc?= =?us-ascii?Q?JK5lOy7XNJyxyWoNOwHXuP7ca0LV2BWWYC1k8aZlHHCDdaNGbdZ3yjqqfTMs?= =?us-ascii?Q?5LlBj0pGM9zIo9lBtzZb9Up3pf8duESup0vbhwbWo3MEO4W9Dhxn4sukHZHK?= =?us-ascii?Q?3QOH26ROiQ67LZthz3ETRj2+xQQCwFaKxJI1CLT9b1mGvMOlQQClWSlqFPPd?= =?us-ascii?Q?ZnNMhFIDctmsw6WXfnQvjAjmcigiJhCfuRPJFnpm6tBtrmPyK/21zgkyH40U?= =?us-ascii?Q?rCWVQwhpnI8jJoeKLHvfsMDde5BfRQSxUtZ/H5OtHx2aw+P10beo07ynQqUM?= =?us-ascii?Q?XRp1ddqLi/5EWRJD+RIUrto1L6xAVFJqN4D8rsbDY0vpwpKwx1RPbeDR3+ab?= =?us-ascii?Q?Wmqw/co1vHJG+usffgKzA6jQu28eyP3ARno6JAt6j+SWotobcqInSezzlfeu?= =?us-ascii?Q?hVHGnEDhk5EjjmB2ViKiG30rMha0QDOePaEIFjMp4s6y9CQztD95ETqeIYtF?= =?us-ascii?Q?xA=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYZP153MB0399.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: b07dd117-a817-4b5d-4882-08dc5fa811a9 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Apr 2024 13:04:23.4922 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: +tV5Qz6ygghNBwxB728srNDsGH4vRg+RFcBgCqZZ0CNmCaCEFI3pVEBxle38Rm8dYzmEG5/o6rRkqFzfdjNovw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZP153MB0705 X-Spamd-Bar: -------- X-Spamd-Result: default: False [-9.00 / 15.00]; WHITELIST_SPF_DKIM(-3.00)[microsoft.com:d:+,microsoft.com:s:+]; DWL_DNSWL_LOW(-1.00)[microsoft.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[microsoft.com,reject]; R_DKIM_ALLOW(-0.20)[microsoft.com:s=selector2]; R_SPF_ALLOW(-0.20)[+ip4:52.100.0.0/14]; MIME_GOOD(-0.10)[text/plain]; TO_DN_EQ_ADDR_SOME(0.00)[]; ASN(0.00)[asn:8075, ipnet:52.96.0.0/12, country:US]; TO_DN_SOME(0.00)[]; MISSING_XM_UA(0.00)[]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[52.101.128.6:from]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[52.101.128.6:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[microsoft.com:+] X-Rspamd-Queue-Id: 4VKyfT1YhLz4FkB Hi, I am trying to add a -W flag to local Makefile so it would only be effectiv= e for the local source files. But it seems not working when I build the ent= ire kernel. For example, I added a structure in sys/dev/hyperv/vmbus/vmbus_var.h. The s= tructure requires adding a -W flag ( -Wno-gnu-variable-sized-type-not-at-en= d ) to build successfully for all .c files included this header file.=20 What I did was I add this line in sys/modules/hyperv/vmbus/Makefile: CWARNFLAGS +=3D -Wno-gnu-variable-sized-type-not-at-end This seems working fine if I build the module by typing 'make' under sys/mo= dules/hyperv/vmbus subdir. But it seems having no effect when building the = kernel by using 'make buildkernel' under global directory. Those .c files s= till fail to build due to lacking this flag.=20 If I add this flag in the global sys/conf/kern.mk, it seems to be working. = However, I don't like to add it globally as only a few source files under h= yperv/vmbus need it. What did I do wrong? Do you know what the proper way t= o add this flag? Thanks, Wei From nobody Thu Apr 18 14:47:15 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VL0xG6K1Tz5H1HZ for ; Thu, 18 Apr 2024 14:47:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VL0xG4Kydz4c8Y for ; Thu, 18 Apr 2024 14:47:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-5176f217b7bso1642015e87.0 for ; Thu, 18 Apr 2024 07:47:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713451648; x=1714056448; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Oht3M/ZTL8s9dnSUifad+dJJgGOMcAKv67CJrobFbXk=; b=iJDQc6/RT20pd7hs9uw5bDxWSqOhjtfnoQQ0ybwqCLTwdjvhXH1R3vKmpTDlInQ4ha 6/x8IBDwi5kHpAlpAAhqB4cOfYCq5fo1WnrFoD/sr+Ze0ScjrMqjheZs1jq52aQSHWvW BVehehCpfMQtKM7OFrBtShP5uhVjdzfuVspIWjLbTGNDoC6e5OWTTQCOmKrawFfbcGbh bBWuUaySMaWUSZEY9/mA4TtZDjOoUuK0YyFxagZMNqy8SgDJmIyuz3rbJm3T8S+HGFNh hQkrvrw+CRlN20jJ/DT66L2r0vWslU70M3TRIz7LKH4h1kI5O9OVPz/POYHhZz2jYvTo QIeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713451648; x=1714056448; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Oht3M/ZTL8s9dnSUifad+dJJgGOMcAKv67CJrobFbXk=; b=HorhCNyIqpmNkk8Vcc0a3DK7MxHL823WZAhWXEwowC/NYXYj4DK8cjvSy4VAOJlLiv vQ8/5q0T3Us8j4k2fvYDy9qkE7RLCklA8qfQ4+h63wNimYUglayPBQx8ShZEFd0Cc+Fz HLv3Zc1ka8jGfz5Dd8VQgFE6IZevT+O+EwMm+bFBF8qC0rqlBjqNLyUs+1NDJFBMwYgS tu5MmQDHw/epK4QoisuWisVFrDp6/V66e4e8ocZfVPgyUow6uxqP4QEVb2yYeKQK+ocU wgXNUFgWd3nU8jODqLj+np4eQRjMuecfme57LQNXlwCrYjCy7CEu2xTSEaoTuS5Cw75Q e5Ag== X-Gm-Message-State: AOJu0YxW2tHBExTTVj56NTM9R+RRs05BHXxV6yuTReT9i4VMx06QZYWH kNUvWJPWR/nEusAaeKdcqrANsFJNlkMXtZyPdP8RKUhHPq+JI8QtutFp9TB+ay4q+MhpVRrexxp 2eOJX4GqD8W+GKo+olK0NQsHblOHEhwU/gv7uYU39am8yPhEfbng= X-Google-Smtp-Source: AGHT+IFqDrkG/jHNiUpc4tbezlWHB484ft3TSOhyrHO5LJNgN7fReoBMO8HXWqBzkHVCNcmde+F/Wpzin6Q2GMBWCKQ= X-Received: by 2002:a19:645b:0:b0:515:acda:77f0 with SMTP id b27-20020a19645b000000b00515acda77f0mr2323812lfj.29.1713451647930; Thu, 18 Apr 2024 07:47:27 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Thu, 18 Apr 2024 08:47:15 -0600 Message-ID: Subject: Re: How to add a -W flag in local Makefile To: Wei Hu Cc: "freebsd-hackers@FreeBSD.org" Content-Type: multipart/alternative; boundary="000000000000d0bdae0616600c33" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VL0xG4Kydz4c8Y --000000000000d0bdae0616600c33 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Apr 18, 2024, 7:04=E2=80=AFAM Wei Hu wrote: > Hi, > > I am trying to add a -W flag to local Makefile so it would only be > effective for the local source files. But it seems not working when I bui= ld > the entire kernel. > > For example, I added a structure in sys/dev/hyperv/vmbus/vmbus_var.h. The > structure requires adding a -W flag ( > -Wno-gnu-variable-sized-type-not-at-end ) to build successfully for all .= c > files included this header file. > What does this type look like? Maybe the right answer is changing it? What I did was I add this line in sys/modules/hyperv/vmbus/Makefile: > > CWARNFLAGS +=3D -Wno-gnu-variable-sized-type-not-at-end > Where did you add it? I think it needs to be after the .includes Warner This seems working fine if I build the module by typing 'make' under > sys/modules/hyperv/vmbus subdir. But it seems having no effect when > building the kernel by using 'make buildkernel' under global directory. > Those .c files still fail to build due to lacking this flag. > > If I add this flag in the global sys/conf/kern.mk, it seems to be > working. However, I don't like to add it globally as only a few source > files under hyperv/vmbus need it. What did I do wrong? Do you know what t= he > proper way to add this flag? > > Thanks, > Wei > > > --000000000000d0bdae0616600c33 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Apr 18, 2024, 7:04=E2=80=AFAM Wei Hu <weh@microsoft.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">Hi,

I am trying to add a -W flag to local Makefile so it would only be effectiv= e for the local source files. But it seems not working when I build the ent= ire kernel.

For example, I added a structure in sys/dev/hyperv/vmbus/vmbus_var.h. The s= tructure requires adding a -W flag ( -Wno-gnu-variable-sized-type-not-at-en= d ) to build successfully for all .c files included this header file.


What does this type look like?

Maybe the right answer is changing it?

What I did was I add this line in sys/modules/hyperv/vmbus/Makefile:

CWARNFLAGS +=3D -Wno-gnu-variable-sized-type-not-at-end

Where did you add it= ? I think it needs to be after the .includes

Warner=C2=A0


This seems working fine if I build the module by typing 'make' unde= r sys/modules/hyperv/vmbus subdir. But it seems having no effect when build= ing the kernel by using 'make buildkernel' under global directory. = Those .c files still fail to build due to lacking this flag.

If I add this flag in the global sys/conf/kern.mk, it seems to be worki= ng. However, I don't like to add it globally as only a few source files= under hyperv/vmbus need it. What did I do wrong? Do you know what the prop= er way to add this flag?

Thanks,
Wei


--000000000000d0bdae0616600c33-- From nobody Thu Apr 18 15:27:13 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VL1qF1h0wz5H4yy for ; Thu, 18 Apr 2024 15:27:21 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01on2111.outbound.protection.outlook.com [40.107.215.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VL1qD2DQGz4p48 for ; Thu, 18 Apr 2024 15:27:20 +0000 (UTC) (envelope-from weh@microsoft.com) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SoieZGDjwlxvZUb1XhXD3ZKOlgX10P6KWi2HXiTxW10x53GqARaHcy3Pe21BrOYXnwPOVXmVoDkH8AB6wf6+qjxb3mrPlJ4ACfCljA2IKsckZOHBDLAhhdmOHkW4tcorMl0Pq6mKiOnnVTrodc1qmq/aWu6f22lkMQ+LR3s2pf8EJC+Gx90gaQvL60ZcePjKKLprvTuTe9bMKAn0O74f01dzmuk8tZB9LMbyj3cj0/vW0qAQyhLjZSm3A821XbKHAZPmsKHVYcJ4RtwW3tEJ8g2P6SkASDshVbBdbM15Q8XpZQ0ynSoIJpVny07Jbk1+gc/dmOU6ph5cEVz0zAg8yA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kViUZ9tI/nlwrWX2gXGQ0l4aAFFyKRvRYfapsT9zWjI=; b=dOQDG10s1aDxpZQC/yvRlpKdI/72LIFzUFdTZE0wjpV5kd6Lbz0+wc8YyXIWU+mWEUSnNJULVtwGc/Slh6FeE+vLvBiXzaVxPwJBYHh2Lx8erY7s7gMjS9uHqOIK1LvKtqi3b01jzlYmG0288bUP/qSmurAFqYi2n0Q3MKqkaTSoXOarKkMeXxbMJrLZnvtPn3H8cHtnsdLsDaQEu6w73dpIXmMJdtGjD0VrqNzxq684f3m5RiiI44uz9VKyUynpRf19QDUm2p9t8toEIxVgeVIAUUsOizTLt8RVEwJaawiH3i3NkoNY6hX8bVTiwSMy/YZgRQK/KIYqS1MnsZxL9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kViUZ9tI/nlwrWX2gXGQ0l4aAFFyKRvRYfapsT9zWjI=; b=NJUDhreGPUe7CC2X20NFneYw3zGdUr1THINrnugij8NjW44e28yUa9rVa1ofIyV2cB0gJz2qlt/ogwH1V26mt70vABqDS9pnWPT2NyA2ndMKi0xoP9wusGzJG4YxLo5t1tLvMTBbotlu/9ImL9lEqR1iUmkYenAFLulgP6cKppY= Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM (2603:1096:400:21::12) by SI2P153MB0703.APCP153.PROD.OUTLOOK.COM (2603:1096:4:19e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7495.13; Thu, 18 Apr 2024 15:27:14 +0000 Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e]) by TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e%7]) with mapi id 15.20.7519.010; Thu, 18 Apr 2024 15:27:13 +0000 From: Wei Hu To: Warner Losh CC: "freebsd-hackers@FreeBSD.org" Subject: RE: How to add a -W flag in local Makefile Thread-Topic: How to add a -W flag in local Makefile Thread-Index: AdqRkOe7luhT44chS0qlVqfwwelwwgADmX2AAAFW5NA= Date: Thu, 18 Apr 2024 15:27:13 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=f47e90c3-45bf-48af-bcbb-4a0721be1f75;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2024-04-18T15:25:36Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYZP153MB0399:EE_|SI2P153MB0703:EE_ x-ms-office365-filtering-correlation-id: b2ad02a8-903c-4964-85fe-08dc5fbc05a7 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: =?utf-8?B?akxsRnV5NDBjL3JubGVic1RuZG14eXcvam4ra0xZRWw1d1Jidnd5T2VMcEZq?= =?utf-8?B?WWZKRXF1aytrUUltWTVrbGJ2SS91UUZ1S2pHeUNqNWhxcHAvR2JicGNJSTRP?= =?utf-8?B?NGtZRGhKVmtselZYUTdhdXd5WE5yRlJmeklVZUFRSVB1aUN3R0s0dXFkeXlT?= =?utf-8?B?V2ZoMnZlUTZYblp3czM5OFV3WGwrWkRBZEhOSysvVk9OckVBN25ldm92NTA2?= =?utf-8?B?UFcyb21LaUJaTExYdmVzb2V6ZU0vQ3RyaURuRVZ0WUM4YmtteW9rd0grUGlT?= =?utf-8?B?REthQjBJeXI2TXMyQ05uSzJHMlM1WWFsdFREUVUrKzhWaVFNaXJGTVlvaGdX?= =?utf-8?B?aU91QmVXNm1wK3BvMG5jVlVYKzEzanlIWGZZTTlkL0pzZjNzaEwxYnhWTWpX?= =?utf-8?B?ejNSeXUzTEpHNHlKVUdaNUZkYjlOTnAzMzdYcll1Ynp5NGdVY0lnMlV2Vkts?= =?utf-8?B?djRiZmxUbEZZL1pZN3RHNFFtR0dIUmtGUFlRU3RGOTJCbWRIYlhPenZsMzlS?= =?utf-8?B?SzF1UloxZU5ieXpyUzRYb0E5eW1zd082VGp4OElyNkdnYmVBeW1sVTNETzNE?= =?utf-8?B?b3ZqdFFwSGNLbkEzdURNeXdRNlNDTERTRXVWcTgvcklsTEFDZU9CNFJKeGZu?= =?utf-8?B?TXNpZzlWbXNlY0RrRVhSYTJPeFlwa21TcmFMb0wyM1UvRjNuNlRja3RLWTYv?= =?utf-8?B?ODIyMXk0Sjg1d3ZXWU90ZzZoYmFDOFB2UEZzSVNnVjFRQUFrbzZvQWRkbXho?= =?utf-8?B?OHVYSTFZOFd2NWdzZGFmdW5PN1o4NkNhSmxkS1JHN2Qvc0E4V2RITk5iTUp6?= =?utf-8?B?OVhLcEtWeXo3ZHhReDFHZzd0SU9wZytQRDBjMWRNRlpDK1Aydk1vR2VXeTNL?= =?utf-8?B?aWZ4UzRWdko4OG03SXBzYXlkODZNRDJmZzdsYnEwcmFHVGc4VmV3V2greWhr?= =?utf-8?B?Q3lVczRWTU43NlRzOEVxaVVlQkxFOFE3Wm9hOHNrS3NxWThWUGpCbXdLT2tt?= =?utf-8?B?TmF2ZlVpVmhWcTNFUXhsOUhUWGVOZjhsb3dkQW1Pa2NjSXo0TDVEbkQ5K05w?= =?utf-8?B?bG9vWG1QaW9lczBDU05LWXBReHp6YUxIS0UvN1BJUERzdnpUYUhpRnhMR2Y2?= =?utf-8?B?K3V4TTA5L1loVDk0NEFYckJSYksxdExZeUJhV1I2RjJuRmpKWEt0cmpKUEdn?= =?utf-8?B?MGdack5mQnVKNlhMTzl6SHF2ZGdMNEJIK1d2M0lwVkN6Ujd1M256TEl2L2Y4?= =?utf-8?B?TjgvY0dKTjZra25xZE83VUY3aWw5UTY4Ti9Vdm9EWjhFUUg0L2ZYV1JwTlYw?= =?utf-8?B?bWV6eVFkZTIyejFTRHpLaFZlbDBTemVDWFdLZVFyREZYVU0zSFdDZThZT0Nu?= =?utf-8?B?UUgvdGZLT1RaaDBzelphMXdXTzRsYWNhN1RsVDB2eWVBTTdJZUVkLzZLbVJU?= =?utf-8?B?Vnd0dDMxU2lYSFg1RFNWRzR0SHowWXNncHlnUlBrblQ1bTZzQlhvOE9Uc1Ba?= =?utf-8?B?SU14VVlqb2NSWmpsaGx2MjRsYVkxaHRFendkRUJCMUprRVNzNk1TQVJhZFM3?= =?utf-8?B?Z2pVS2Q1ZXJMaDNvWFNGZlQzTllCeVQ4L0ZUMXZ0bis2eWpnd29wYmtmeEFG?= =?utf-8?B?bVRjU2dtTjRQU1VDbldkWWJxUzBVMG01QzhzOWZPWXJudmVVRmJOZSswK3N2?= =?utf-8?B?QXEyZFl3TTA0Z2RXZUtic2E4L1ZFaDUva2hIS0xLRWREUldGNE0yUGJRPT0=?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYZP153MB0399.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(376005)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?K21MTkZvOUQvOURkdnc5SzY2WXZoMGdKS0ZzVUZkR0hVQzdrTzZOK2EyQ0tZ?= =?utf-8?B?OTlLaDR4aGxxZVpPU3lxWjRBSDVsNitSc2lkNzNpNndYVkYxTmhaeWRmaTBO?= =?utf-8?B?T29GcW1QanMwR0F5K2ZtL2pLNlNMczNCcUZYcWdhSFNySXhRL1JYOGpzTk9r?= =?utf-8?B?OXMzaFIrRmRXa2JsVkR5eGJGWTMrYW5HcTJSYUJ3WURQSncvcG9scUFjVlRo?= =?utf-8?B?akpOeDNMRnEzdjBCUjZuYVpuZTB0Y3ZkNGQ0SmwyTVlOK0ZvUyt6TUkwWlBu?= =?utf-8?B?R0ZBdEVDRFFTOFFwbkxzYXdVZmVqeUZ5b0tnQUc2WElsODdYUW1wNjkyU3ow?= =?utf-8?B?Z3JMSHVOVGNibmpIMFVMbmpsMjVtNkUzYXJVY1BmRE5xbFlOZGtRckRjTVNa?= =?utf-8?B?L282L2hPSE5kYXRwcHk1b2srSTB1U2xGUnFyaHE1VVN6VUtYRHhnMm94VUIx?= =?utf-8?B?aHRGSG11RSt4Sk1uTjVGTVVVUDJndHpUR3B2Znh6bjlhcTRXaWFYczlZQ1h5?= =?utf-8?B?YVFPbDZnU2IvYkljVGgrK0srNzIzRlB4U2VGT1oySlMwQzI1bExJcmtrVGs1?= =?utf-8?B?VDYrcXFHRjNVQVF2bi9pSENINXJkaW1HVjZ4R3JuY2tzMWRrbEcrc1I4SS9I?= =?utf-8?B?RTlVaVR1Y2RmQmhBOHlrVzhsTGdBSEpyOTlxWFFBeEdnSHFqejYxU2ZsMDdw?= =?utf-8?B?V01ibFNIU0ovVXFFdzcwaFNISmc1WXBuNGgrSmVmeHBGeThLa3lPcE1DWFFR?= =?utf-8?B?N1VpT2Z4REJBcHRNSlZwUXk0ek9vM0ZnYmZ4YmhmOFVYeWZwNDIyTVVIMVk0?= =?utf-8?B?czcwc0IzbTJqTnh3d2NVcjY4L1B2RU1UdGhld1AramFBWFBWL3NWY2JPd282?= =?utf-8?B?ZHVkVkh1aysvUmE3THN0MU9CQ0FPRy90aU9HRTd0OCtqMG5uQWRTODkrYjdQ?= =?utf-8?B?RWxxWkRrUkwvUDA5Ry9lbys3d3FsV1R5VjZtVjJqdEczTXJGcGxqU1VPTzBR?= =?utf-8?B?cVJzLzJac0VvNTJXNXNuTFo5NnZ1bEhiNkZSa1NpcmFWSjFZWWVxWW5mejg1?= =?utf-8?B?NDg0RzNoSnZydXF0MEFTbXRHb1FwaG14Rmh0TFZObTIwcitnNllJVElIeFh0?= =?utf-8?B?TUhnOTBaUGY1a0J6WlZ0UEpzQ3ZpamJHVDdoZUdwOUtDdldxRHBaNCttWlRq?= =?utf-8?B?NVhlb2E4Rm4wR0hFb0padmVhOWs1TWhyZitJMnRRTmljQk1wWnc0WWJ3NUxa?= =?utf-8?B?RlZXUHdTWm82bThsYzdWWWJGRTdwRWJ4dFJTRllWSHZmejBDYTRPWFlrMS9K?= =?utf-8?B?cG1JYUM2UTllS29aTjRuUDM3enJXRmJMNjhiK3cwenpPK3VmRFZsbEcvOVBy?= =?utf-8?B?RUJhSkRSdC9tYWZQUW14UGlDWUVLaEJnSTB3S212SVk3MytYZVpOT0F4UTd0?= =?utf-8?B?eGlEME93c0xZS3g5QWRGUkozYjBXMUpMNjJkeGFiSkF4dGNicWg3c3o3MHdI?= =?utf-8?B?MHpNSFdzV2FJeTcwSlFBOG5wRndQL1crVE1FSGphcVY0aUpEZ21pMTFnWmtx?= =?utf-8?B?OGNsYXAvUVFIckJYRVpJeXNxOVQ1bTJPbzNYUUc4THhXaW9nR0JtcHBwVi81?= =?utf-8?B?VERPQ29QTDJadktuVUdvSnVRR3JzdjUvTDEvZm03bjhZeXlta0k4cjdYOGtt?= =?utf-8?B?ekxCZFJtMWpwblF1R01zZURJcW9kWXFBTXVmMWZDNGw2TlIyYUM2dmNFQll6?= =?utf-8?B?NW9GMkVLeTZzeGxQMHhiT3Rma2hvazhJZnZqUVMvRzA0bnhkRHBXajUzQkhu?= =?utf-8?B?ZVNZZkZNeTJ3eE5iU3RHbjF3K2d0QzN4STBLa1lnOU54aHVNYlFTQTVLNmVI?= =?utf-8?B?c1RMMjF5U0xjTUIrZWxxekt0SHRBdElQTkx3T2ZjK2wzVUczZW9mQUNjVGRv?= =?utf-8?B?end6S1FTV1ZuZ3RBMG5TK0luMVhHWGZHOWNEaVM5YnkrL0VPZlJBRStiSGR6?= =?utf-8?B?Sklwb00vaWlGVmlINzhLZlhjVDQzMDlNZEtxU2pxNVB6cFlzdEFMT05KdzE0?= =?utf-8?B?SWVzcXZBcjU4Vlo2NVFoWEF2aFNBVlFkS201OWNSdXpTcWxwTUFUUGxvWjF2?= =?utf-8?B?amRwL1VqUVo0V0NhSWJDSkozZjhQNkZwMThqeXMvYWZkK1I0Mmw2NXgzZFNz?= =?utf-8?B?amc9PQ==?= Content-Type: multipart/alternative; boundary="_000_TYZP153MB039998D645FD84C28A9A057EBB0E2TYZP153MB0399APCP_" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYZP153MB0399.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: b2ad02a8-903c-4964-85fe-08dc5fbc05a7 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Apr 2024 15:27:13.3078 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BTivM6efDt8iAyF767tH+IzLh9XgqqjklXuhQi+3zUB3LxKqcSG8QAjO9rncqwSSNS5Zigtd8MNzWCARQq1M4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2P153MB0703 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US] X-Rspamd-Queue-Id: 4VL1qD2DQGz4p48 --_000_TYZP153MB039998D645FD84C28A9A057EBB0E2TYZP153MB0399APCP_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SSBhZGRlZCBzb21ldGhpbmcgbGlrZToNCg0KZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L3Zt YnVzL3ZtYnVzX3Zhci5oIGIvc3lzL2Rldi9oeXBlcnYvdm1idXMvdm1idXNfdmFyLmgNCmluZGV4 IGI1OThmNzgyOTQ3ZS4uNmUzYjdiMDQwODI3IDEwMDY0NA0KLS0tIGEvc3lzL2Rldi9oeXBlcnYv dm1idXMvdm1idXNfdmFyLmgNCisrKyBiL3N5cy9kZXYvaHlwZXJ2L3ZtYnVzL3ZtYnVzX3Zhci5o DQpAQCAtMTkzLDQgKzE5MywxNyBAQCBzdHJ1Y3QgaHlwZXJ2X3RsYl9mbHVzaCB7DQp1aW50NjRf dCAgICAgICAgaHZfdm1fdGxiX2ZsdXNoKHBtYXBfdCBwbWFwLCB2bV9vZmZzZXRfdCBhZGRyMSwN CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdm1fb2Zmc2V0X3QgYWRkcjIsIGNwdXNl dF90IG1hc2spOw0KDQorc3RydWN0IGh2X3Zwc2V0IHsNCisgICAgICAgdWludDY0X3QgZm9ybWF0 Ow0KKyAgICAgICB1aW50NjRfdCB2YWxpZF9iYW5rX21hc2s7DQorICAgICAgIHVpbnQ2NF90IGJh bmtfY29udGVudHNbXTsNCit9IF9fcGFja2VkOw0KKw0KK3N0cnVjdCBodl90bGJfZmx1c2hfZXgg ew0KKyAgICAgICB1aW50NjRfdCBhZGRyZXNzX3NwYWNlOw0KKyAgICAgICB1aW50NjRfdCBmbGFn czsNCisgICAgICAgc3RydWN0IGh2X3Zwc2V0IGh2X3ZwX3NldDsNCisgICAgICAgdWludDY0X3Qg Z3ZhX2xpc3RbXTsNCit9IF9fcGFja2VkOw0KKw0KI2VuZGlmIC8qICFfVk1CVVNfVkFSX0hfICov DQoNClNvLCB0aGUgc3RydWN0IGh2X3Zwc2V0IGlzIHRoZSBzZWNvbmQgbGFzdCBtZW1iZXIgb2Yg c3RydWN0IGh2X3RsYl9mbHVzaF9leC4gVGhlIG1lbWJlciBiYW5rX2NvbnRlbnRzW10gaW4gc3Ry dWN0IGh2X3Zwc2V0IGlzIG9mIHZhcmlhYmxlIGxlbmd0aC4gVGhpcyB3b3VsZCBtYWtlcyB0aGUg bGFzdCB0d28gbWVtYmVycyBvZiBzdHJ1Y3QgaHZfdGxiX2ZsdXNoX2V4IGJvdGggdmFyaWFibGUg bGVuZ3RoLiBUaGVyZWZvcmUsIHRoZSBmbGFnICctV25vLWdudS12YXJpYWJsZS1zaXplZC10eXBl LW5vdC1hdC1lbmQnIGlzIG5lZWRlZCwgb3RoZXJ3aXNlIGl0IHdvdWxkIGNvbXBsYWluIGFib3V0 IHRoaXMgd2l0aCBlcnJvcnMgbGlrZToNCg0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC93b3JrL2Zy ZWVic2Qtc3JjL3N5cy9kZXYvaHlwZXJ2L3ZtYnVzL3ZtYnVzLmM6Njk6DQovd29yay9mcmVlYnNk LXNyYy9zeXMvZGV2L2h5cGVydi92bWJ1cy92bWJ1c192YXIuaDoyMDM6MjU6IGVycm9yOiBmaWVs ZCAnaHZfdnBfc2V0Jw0Kd2l0aCB2YXJpYWJsZSBzaXplZCB0eXBlICdzdHJ1Y3QgaHZfdnBzZXQn IG5vdCBhdCB0aGUgZW5kIG9mIGEgc3RydWN0IG9yIGNsYXNzIGlzIGENCkdOVSBleHRlbnNpb24g Wy1XZXJyb3IsLVdnbnUtdmFyaWFibGUtc2l6ZWQtdHlwZS1ub3QtYXQtZW5kXQ0KICAyMDMgfCAg ICAgICAgIHN0cnVjdCBodl92cHNldCBodl92cF9zZXQ7DQoNCkkgZGlkIGFkZCB0aGUgZmxhZyBh dCB0aGUgZW5kIG9mIHRoZSBsb2NhbCBNYWtlZmlsZSwgYWZ0ZXIgLmluY2x1ZGUsIGxpa2UgZm9s bG93aW5nOg0KDQpkaWZmIC0tZ2l0IGEvc3lzL21vZHVsZXMvaHlwZXJ2L3ZtYnVzL01ha2VmaWxl IGIvc3lzL21vZHVsZXMvaHlwZXJ2L3ZtYnVzL01ha2VmaWxlDQppbmRleCAxNjU5ZDUxODY0OTMu Ljg3MGZmNzEyOTlkMCAxMDA2NDQNCi0tLSBhL3N5cy9tb2R1bGVzL2h5cGVydi92bWJ1cy9NYWtl ZmlsZQ0KKysrIGIvc3lzL21vZHVsZXMvaHlwZXJ2L3ZtYnVzL01ha2VmaWxlDQpAQCAtNDIsMyAr NDIsNyBAQCBDRkxBR1MrPSAtSSR7U1JDVE9QfS9zeXMvZGV2L2h5cGVydi9pbmNsdWRlIFwNCkVY UE9SVF9TWU1TPSAgIFlFUw0KDQouaW5jbHVkZSA8YnNkLmttb2QubWs+DQorDQorQ1dBUk5GTEFH UyArPSAtV25vLWdudS12YXJpYWJsZS1zaXplZC10eXBlLW5vdC1hdC1lbmQNCg0KVGhhbmtzLA0K V2VpDQoNCg0KRnJvbTogV2FybmVyIExvc2ggPGltcEBic2RpbXAuY29tPg0KU2VudDogVGh1cnNk YXksIEFwcmlsIDE4LCAyMDI0IDEwOjQ3IFBNDQpUbzogV2VpIEh1IDx3ZWhAbWljcm9zb2Z0LmNv bT4NCkNjOiBmcmVlYnNkLWhhY2tlcnNARnJlZUJTRC5vcmcNClN1YmplY3Q6IFJlOiBIb3cgdG8g YWRkIGEgLVcgZmxhZyBpbiBsb2NhbCBNYWtlZmlsZQ0KDQoNCk9uIFRodSwgQXByIDE4LCAyMDI0 LCA3OjA04oCvQU0gV2VpIEh1IDx3ZWhAbWljcm9zb2Z0LmNvbTxtYWlsdG86d2VoQG1pY3Jvc29m dC5jb20+PiB3cm90ZToNCkhpLA0KDQpJIGFtIHRyeWluZyB0byBhZGQgYSAtVyBmbGFnIHRvIGxv Y2FsIE1ha2VmaWxlIHNvIGl0IHdvdWxkIG9ubHkgYmUgZWZmZWN0aXZlIGZvciB0aGUgbG9jYWwg c291cmNlIGZpbGVzLiBCdXQgaXQgc2VlbXMgbm90IHdvcmtpbmcgd2hlbiBJIGJ1aWxkIHRoZSBl bnRpcmUga2VybmVsLg0KDQpGb3IgZXhhbXBsZSwgSSBhZGRlZCBhIHN0cnVjdHVyZSBpbiBzeXMv ZGV2L2h5cGVydi92bWJ1cy92bWJ1c192YXIuaC4gVGhlIHN0cnVjdHVyZSByZXF1aXJlcyBhZGRp bmcgYSAtVyBmbGFnICggLVduby1nbnUtdmFyaWFibGUtc2l6ZWQtdHlwZS1ub3QtYXQtZW5kICkg dG8gYnVpbGQgc3VjY2Vzc2Z1bGx5IGZvciBhbGwgLmMgZmlsZXMgaW5jbHVkZWQgdGhpcyBoZWFk ZXIgZmlsZS4NCg0KDQpXaGF0IGRvZXMgdGhpcyB0eXBlIGxvb2sgbGlrZT8NCg0KTWF5YmUgdGhl IHJpZ2h0IGFuc3dlciBpcyBjaGFuZ2luZyBpdD8NCg0KV2hhdCBJIGRpZCB3YXMgSSBhZGQgdGhp cyBsaW5lIGluIHN5cy9tb2R1bGVzL2h5cGVydi92bWJ1cy9NYWtlZmlsZToNCg0KQ1dBUk5GTEFH UyArPSAtV25vLWdudS12YXJpYWJsZS1zaXplZC10eXBlLW5vdC1hdC1lbmQNCg0KV2hlcmUgZGlk IHlvdSBhZGQgaXQ/IEkgdGhpbmsgaXQgbmVlZHMgdG8gYmUgYWZ0ZXIgdGhlIC5pbmNsdWRlcw0K DQpXYXJuZXINCg0KDQpUaGlzIHNlZW1zIHdvcmtpbmcgZmluZSBpZiBJIGJ1aWxkIHRoZSBtb2R1 bGUgYnkgdHlwaW5nICdtYWtlJyB1bmRlciBzeXMvbW9kdWxlcy9oeXBlcnYvdm1idXMgc3ViZGly LiBCdXQgaXQgc2VlbXMgaGF2aW5nIG5vIGVmZmVjdCB3aGVuIGJ1aWxkaW5nIHRoZSBrZXJuZWwg YnkgdXNpbmcgJ21ha2UgYnVpbGRrZXJuZWwnIHVuZGVyIGdsb2JhbCBkaXJlY3RvcnkuIFRob3Nl IC5jIGZpbGVzIHN0aWxsIGZhaWwgdG8gYnVpbGQgZHVlIHRvIGxhY2tpbmcgdGhpcyBmbGFnLg0K DQpJZiBJIGFkZCB0aGlzIGZsYWcgaW4gdGhlIGdsb2JhbCBzeXMvY29uZi9rZXJuLm1rPGh0dHA6 Ly9rZXJuLm1rLz4sIGl0IHNlZW1zIHRvIGJlIHdvcmtpbmcuIEhvd2V2ZXIsIEkgZG9uJ3QgbGlr ZSB0byBhZGQgaXQgZ2xvYmFsbHkgYXMgb25seSBhIGZldyBzb3VyY2UgZmlsZXMgdW5kZXIgaHlw ZXJ2L3ZtYnVzIG5lZWQgaXQuIFdoYXQgZGlkIEkgZG8gd3Jvbmc/IERvIHlvdSBrbm93IHdoYXQg dGhlIHByb3BlciB3YXkgdG8gYWRkIHRoaXMgZmxhZz8NCg0KVGhhbmtzLA0KV2VpDQoNCg== --_000_TYZP153MB039998D645FD84C28A9A057EBB0E2TYZP153MB0399APCP_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5Ouetiee6vzsNCglwYW5vc2UtMToyIDEgNiAwIDMgMSAxIDEgMSAx O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToyIDE1IDUg MiAyIDIgNCAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkFwdG9zO30NCkBmb250 LWZhY2UNCgl7Zm9udC1mYW1pbHk6IlxA562J57q/IjsNCglwYW5vc2UtMToyIDEgNiAwIDMgMSAx IDEgMSAxO30NCi8qIFN0eWxlIERlZmluaXRpb25zICovDQpwLk1zb05vcm1hbCwgbGkuTXNvTm9y bWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjowY207DQoJZm9udC1zaXplOjEyLjBwdDsNCglm b250LWZhbWlseToiQXB0b3MiLHNhbnMtc2VyaWY7fQ0KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGlu aw0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJY29sb3I6Ymx1ZTsNCgl0ZXh0LWRlY29yYXRp b246dW5kZXJsaW5lO30NCnNwYW4uRW1haWxTdHlsZTE4DQoJe21zby1zdHlsZS10eXBlOnBlcnNv bmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJBcHRvcyIsc2Fucy1zZXJpZjsNCgljb2xvcjp3aW5k b3d0ZXh0O30NCnNwYW4uRW1haWxTdHlsZTE5DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsOw0K CWZvbnQtZmFtaWx5OiJBcHRvcyIsc2Fucy1zZXJpZjsNCgljb2xvcjp3aW5kb3d0ZXh0O30NCi5N c29DaHBEZWZhdWx0DQoJe21zby1zdHlsZS10eXBlOmV4cG9ydC1vbmx5O30NCkBwYWdlIFdvcmRT ZWN0aW9uMQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDkwLjBwdCA3 Mi4wcHQgOTAuMHB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0K LS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVmYXVsdHMgdjpl eHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0 ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxvOmlkbWFwIHY6 ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0t Pg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9ImJsdWUiIHZsaW5rPSJwdXJwbGUi IHN0eWxlPSJ3b3JkLXdyYXA6YnJlYWstd29yZCI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlvbjEi Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9u dC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj5JIGFkZGVkIHNvbWV0aGlu ZyBsaWtlOjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFu IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss c2Fucy1zZXJpZiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs aWJyaSZxdW90OyxzYW5zLXNlcmlmIj5kaWZmIC0tZ2l0IGEvc3lzL2Rldi9oeXBlcnYvdm1idXMv dm1idXNfdmFyLmggYi9zeXMvZGV2L2h5cGVydi92bWJ1cy92bWJ1c192YXIuaDxvOnA+PC9vOnA+ PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6 MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+aW5kZXgg YjU5OGY3ODI5NDdlLi42ZTNiN2IwNDA4MjcgMTAwNjQ0PG86cD48L286cD48L3NwYW4+PC9wPg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4tLS0gYS9zeXMvZGV2L2h5cGVy di92bWJ1cy92bWJ1c192YXIuaDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29O b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0Nh bGlicmkmcXVvdDssc2Fucy1zZXJpZiI+KysrIGIvc3lzL2Rldi9oeXBlcnYvdm1idXMvdm1idXNf dmFyLmg8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNh bnMtc2VyaWYiPkBAIC0xOTMsNCArMTkzLDE3IEBAIHN0cnVjdCBoeXBlcnZfdGxiX2ZsdXNoIHs8 bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2Vy aWYiPnVpbnQ2NF90Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IGh2 X3ZtX3RsYl9mbHVzaChwbWFwX3QgcG1hcCwgdm1fb2Zmc2V0X3QgYWRkcjEsPG86cD48L286cD48 L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4mbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsgdm1fb2Zmc2V0X3QgYWRkcjIsIGNwdXNldF90IG1hc2spOzxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+PG86cD4mbmJz cDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlm Ij4rc3RydWN0IGh2X3Zwc2V0IHs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD YWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPismbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsgdWludDY0X3QgZm9ybWF0OzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+KyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyB1aW50NjRfdCB2YWxpZF9iYW5rX21hc2s7PG86cD48L286cD48L3NwYW4+PC9wPg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4rJm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7IHVpbnQ2NF90IGJhbmtfY29udGVudHNbXTs8bzpwPjwvbzpwPjwv c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPit9IF9fcGFj a2VkOzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fu cy1zZXJpZiI+KzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVv dDssc2Fucy1zZXJpZiI+K3N0cnVjdCBodl90bGJfZmx1c2hfZXggezxvOnA+PC9vOnA+PC9zcGFu PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+KyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyB1aW50NjRfdCBhZGRyZXNzX3NwYWNlOzxvOnA+PC9v OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+KyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyB1aW50NjRfdCBmbGFnczs8bzpwPjwv bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPism bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgc3RydWN0IGh2X3Zwc2V0IGh2X3Zw X3NldDs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz dHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNh bnMtc2VyaWYiPismbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgdWludDY0X3Qg Z3ZhX2xpc3RbXTs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1 b3Q7LHNhbnMtc2VyaWYiPit9IF9fcGFja2VkOzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5 OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+KzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+I2VuZGlmIC8qICFfVk1CVVNf VkFSX0hfICovPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90 OyxzYW5zLXNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD YWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPlNvLCB0aGUgc3RydWN0IGh2X3Zwc2V0IGlzIHRoZSBz ZWNvbmQgbGFzdCBtZW1iZXIgb2Ygc3RydWN0IGh2X3RsYl9mbHVzaF9leC4gVGhlIG1lbWJlciBi YW5rX2NvbnRlbnRzW10gaW4gc3RydWN0IGh2X3Zwc2V0IGlzIG9mIHZhcmlhYmxlIGxlbmd0aC4g VGhpcyB3b3VsZCBtYWtlcyB0aGUgbGFzdA0KIHR3byBtZW1iZXJzIG9mIHN0cnVjdCBodl90bGJf Zmx1c2hfZXggYm90aCB2YXJpYWJsZSBsZW5ndGguIFRoZXJlZm9yZSwgdGhlIGZsYWcgJy1Xbm8t Z251LXZhcmlhYmxlLXNpemVkLXR5cGUtbm90LWF0LWVuZCcgaXMgbmVlZGVkLCBvdGhlcndpc2Ug aXQgd291bGQgY29tcGxhaW4gYWJvdXQgdGhpcyB3aXRoIGVycm9ycyBsaWtlOjxvOnA+PC9vOnA+ PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6 MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+PG86cD4m bmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNl cmlmIj5JbiBmaWxlIGluY2x1ZGVkIGZyb20gL3dvcmsvZnJlZWJzZC1zcmMvc3lzL2Rldi9oeXBl cnYvdm1idXMvdm1idXMuYzo2OTo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD YWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPi93b3JrL2ZyZWVic2Qtc3JjL3N5cy9kZXYvaHlwZXJ2 L3ZtYnVzL3ZtYnVzX3Zhci5oOjIwMzoyNTxzcGFuIHN0eWxlPSJjb2xvcjpyZWQiPjogZXJyb3I6 IGZpZWxkICdodl92cF9zZXQnDQo8bzpwPjwvbzpwPjwvc3Bhbj48L3NwYW4+PC9wPg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6 JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOnJlZCI+d2l0aCB2YXJpYWJsZSBz aXplZCB0eXBlICdzdHJ1Y3QgaHZfdnBzZXQnIG5vdCBhdCB0aGUgZW5kIG9mIGEgc3RydWN0IG9y IGNsYXNzIGlzIGENCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkm cXVvdDssc2Fucy1zZXJpZjtjb2xvcjpyZWQiPkdOVSBleHRlbnNpb24gWy1XZXJyb3IsLVdnbnUt dmFyaWFibGUtc2l6ZWQtdHlwZS1ub3QtYXQtZW5kXTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFt aWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+Jm5ic3A7IDIwMyB8Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IHN0cnVjdCBodl92cHNldCBo dl92cF9zZXQ7PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90 OyxzYW5zLXNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtD YWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPkkgZGlkIGFkZCB0aGUgZmxhZyBhdCB0aGUgZW5kIG9m IHRoZSBsb2NhbCBNYWtlZmlsZSwgYWZ0ZXIgLmluY2x1ZGUsIGxpa2UgZm9sbG93aW5nOg0KPG86 cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlm Ij48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh biBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7 LHNhbnMtc2VyaWYiPmRpZmYgLS1naXQgYS9zeXMvbW9kdWxlcy9oeXBlcnYvdm1idXMvTWFrZWZp bGUgYi9zeXMvbW9kdWxlcy9oeXBlcnYvdm1idXMvTWFrZWZpbGU8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPmluZGV4IDE2NTlkNTE4 NjQ5My4uODcwZmY3MTI5OWQwIDEwMDY0NDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZx dW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+LS0tIGEvc3lzL21vZHVsZXMvaHlwZXJ2L3Zt YnVzL01ha2VmaWxlPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZx dW90OyxzYW5zLXNlcmlmIj4rKysgYi9zeXMvbW9kdWxlcy9oeXBlcnYvdm1idXMvTWFrZWZpbGU8 bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i Zm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2Vy aWYiPkBAIC00MiwzICs0Miw3IEBAIENGTEFHUys9IC1JJHtTUkNUT1B9L3N5cy9kZXYvaHlwZXJ2 L2luY2x1ZGUgXDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVv dDssc2Fucy1zZXJpZiI+RVhQT1JUX1NZTVM9Jm5ic3A7Jm5ic3A7IFlFUzxvOnA+PC9vOnA+PC9z cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+PG86cD4mbmJz cDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlm Ij4uaW5jbHVkZSAmbHQ7YnNkLmttb2QubWsmZ3Q7PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p bHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4rPG86cD48L286cD48L3NwYW4+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9u dC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj4rQ1dBUk5GTEFHUyArPSAt V25vLWdudS12YXJpYWJsZS1zaXplZC10eXBlLW5vdC1hdC1lbmQ8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPlRoYW5rcyw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PldlaTxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286 cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXYg c3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkIGJsdWUgMS41cHQ7cGFkZGluZzow Y20gMGNtIDBjbSA0LjBwdCI+DQo8ZGl2Pg0KPGRpdiBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVy LXRvcDpzb2xpZCAjRTFFMUUxIDEuMHB0O3BhZGRpbmc6My4wcHQgMGNtIDBjbSAwY20iPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PGI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj5Gcm9tOjwvc3Bhbj48L2I+PHNw YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90 OyxzYW5zLXNlcmlmIj4gV2FybmVyIExvc2ggJmx0O2ltcEBic2RpbXAuY29tJmd0Ow0KPGJyPg0K PGI+U2VudDo8L2I+IFRodXJzZGF5LCBBcHJpbCAxOCwgMjAyNCAxMDo0NyBQTTxicj4NCjxiPlRv OjwvYj4gV2VpIEh1ICZsdDt3ZWhAbWljcm9zb2Z0LmNvbSZndDs8YnI+DQo8Yj5DYzo8L2I+IGZy ZWVic2QtaGFja2Vyc0BGcmVlQlNELm9yZzxicj4NCjxiPlN1YmplY3Q6PC9iPiBSZTogSG93IHRv IGFkZCBhIC1XIGZsYWcgaW4gbG9jYWwgTWFrZWZpbGU8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8 L2Rpdj4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+ DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9t OjEyLjBwdCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPk9uIFRodSwgQXByIDE4LCAyMDI0LCA3OjA0PHNwYW4gc3R5bGU9ImZvbnQtZmFt aWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+4oCvPC9zcGFuPkFNIFdlaSBI dSAmbHQ7PGEgaHJlZj0ibWFpbHRvOndlaEBtaWNyb3NvZnQuY29tIj53ZWhAbWljcm9zb2Z0LmNv bTwvYT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHls ZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBj bSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+SGksPGJyPg0KPGJyPg0KSSBhbSB0cnlpbmcgdG8gYWRkIGEgLVcg ZmxhZyB0byBsb2NhbCBNYWtlZmlsZSBzbyBpdCB3b3VsZCBvbmx5IGJlIGVmZmVjdGl2ZSBmb3Ig dGhlIGxvY2FsIHNvdXJjZSBmaWxlcy4gQnV0IGl0IHNlZW1zIG5vdCB3b3JraW5nIHdoZW4gSSBi dWlsZCB0aGUgZW50aXJlIGtlcm5lbC48YnI+DQo8YnI+DQpGb3IgZXhhbXBsZSwgSSBhZGRlZCBh IHN0cnVjdHVyZSBpbiBzeXMvZGV2L2h5cGVydi92bWJ1cy92bWJ1c192YXIuaC4gVGhlIHN0cnVj dHVyZSByZXF1aXJlcyBhZGRpbmcgYSAtVyBmbGFnICggLVduby1nbnUtdmFyaWFibGUtc2l6ZWQt dHlwZS1ub3QtYXQtZW5kICkgdG8gYnVpbGQgc3VjY2Vzc2Z1bGx5IGZvciBhbGwgLmMgZmlsZXMg aW5jbHVkZWQgdGhpcyBoZWFkZXIgZmlsZS48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4N CjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8 L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNw OzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPldoYXQgZG9l cyB0aGlzIHR5cGUgbG9vayBsaWtlPzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj5NYXliZSB0aGUgcmlnaHQgYW5zd2VyIGlzIGNoYW5naW5nIGl0 PzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86 cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGJsb2NrcXVvdGUgc3R5 bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzow Y20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPldoYXQgSSBkaWQgd2FzIEkgYWRkIHRoaXMgbGluZSBpbiBzeXMv bW9kdWxlcy9oeXBlcnYvdm1idXMvTWFrZWZpbGU6PGJyPg0KPGJyPg0KQ1dBUk5GTEFHUyArPSAt V25vLWdudS12YXJpYWJsZS1zaXplZC10eXBlLW5vdC1hdC1lbmQ8bzpwPjwvbzpwPjwvcD4NCjwv YmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs Ij5XaGVyZSBkaWQgeW91IGFkZCBpdD8gSSB0aGluayBpdCBuZWVkcyB0byBiZSBhZnRlciB0aGUg LmluY2x1ZGVzPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O b3JtYWwiPldhcm5lciZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxk aXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0ND Q0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21h cmdpbi1yaWdodDowY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0 b206MTIuMHB0Ij5UaGlzIHNlZW1zIHdvcmtpbmcgZmluZSBpZiBJIGJ1aWxkIHRoZSBtb2R1bGUg YnkgdHlwaW5nICdtYWtlJyB1bmRlciBzeXMvbW9kdWxlcy9oeXBlcnYvdm1idXMgc3ViZGlyLiBC dXQgaXQgc2VlbXMgaGF2aW5nIG5vIGVmZmVjdCB3aGVuIGJ1aWxkaW5nIHRoZSBrZXJuZWwgYnkg dXNpbmcgJ21ha2UgYnVpbGRrZXJuZWwnIHVuZGVyIGdsb2JhbCBkaXJlY3RvcnkuDQogVGhvc2Ug LmMgZmlsZXMgc3RpbGwgZmFpbCB0byBidWlsZCBkdWUgdG8gbGFja2luZyB0aGlzIGZsYWcuIDxi cj4NCjxicj4NCklmIEkgYWRkIHRoaXMgZmxhZyBpbiB0aGUgZ2xvYmFsIHN5cy9jb25mLzxhIGhy ZWY9Imh0dHA6Ly9rZXJuLm1rLyIgdGFyZ2V0PSJfYmxhbmsiPmtlcm4ubWs8L2E+LCBpdCBzZWVt cyB0byBiZSB3b3JraW5nLiBIb3dldmVyLCBJIGRvbid0IGxpa2UgdG8gYWRkIGl0IGdsb2JhbGx5 IGFzIG9ubHkgYSBmZXcgc291cmNlIGZpbGVzIHVuZGVyIGh5cGVydi92bWJ1cyBuZWVkIGl0LiBX aGF0IGRpZCBJIGRvIHdyb25nPyBEbyB5b3Uga25vdyB3aGF0IHRoZQ0KIHByb3BlciB3YXkgdG8g YWRkIHRoaXMgZmxhZz88YnI+DQo8YnI+DQpUaGFua3MsPGJyPg0KV2VpPGJyPg0KPGJyPg0KPG86 cD48L286cD48L3A+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rp dj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_TYZP153MB039998D645FD84C28A9A057EBB0E2TYZP153MB0399APCP_-- From nobody Thu Apr 18 17:57:16 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VL58W6P3Pz5HJ8m for ; Thu, 18 Apr 2024 17:57:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VL58W2MjDz56lp for ; Thu, 18 Apr 2024 17:57:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 43IHvG9n086861; Thu, 18 Apr 2024 20:57:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 43IHvG9n086861 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 43IHvGYe086860; Thu, 18 Apr 2024 20:57:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Apr 2024 20:57:16 +0300 From: Konstantin Belousov To: Wei Hu Cc: Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: Re: How to add a -W flag in local Makefile Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4VL58W2MjDz56lp On Thu, Apr 18, 2024 at 03:27:13PM +0000, Wei Hu wrote: > I added something like: > > diff --git a/sys/dev/hyperv/vmbus/vmbus_var.h b/sys/dev/hyperv/vmbus/vmbus_var.h > index b598f782947e..6e3b7b040827 100644 > --- a/sys/dev/hyperv/vmbus/vmbus_var.h > +++ b/sys/dev/hyperv/vmbus/vmbus_var.h > @@ -193,4 +193,17 @@ struct hyperv_tlb_flush { > uint64_t hv_vm_tlb_flush(pmap_t pmap, vm_offset_t addr1, > vm_offset_t addr2, cpuset_t mask); > > +struct hv_vpset { > + uint64_t format; > + uint64_t valid_bank_mask; > + uint64_t bank_contents[]; > +} __packed; > + > +struct hv_tlb_flush_ex { > + uint64_t address_space; > + uint64_t flags; > + struct hv_vpset hv_vp_set; > + uint64_t gva_list[]; > +} __packed; > + > #endif /* !_VMBUS_VAR_H_ */ > > So, the struct hv_vpset is the second last member of struct hv_tlb_flush_ex. The member bank_contents[] in struct hv_vpset is of variable length. This would makes the last two members of struct hv_tlb_flush_ex both variable length. Therefore, the flag '-Wno-gnu-variable-sized-type-not-at-end' is needed, otherwise it would complain about this with errors like: > No, the compiler' complain is correct. The structure definition for hv_tlb_flush_ex does not make sense: you never can access gva_list (except for the case of bank_contents being zero-sized). From nobody Fri Apr 19 04:20:13 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLLzB3Gfcz5HmKx for ; Fri, 19 Apr 2024 04:20:22 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01on2070f.outbound.protection.outlook.com [IPv6:2a01:111:f400:feab::70f]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLLz963DWz4FmQ for ; Fri, 19 Apr 2024 04:20:21 +0000 (UTC) (envelope-from weh@microsoft.com) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ebUHEk8UNoyqtvXoCjZ0Rp6SZo3ChUuC/Ful5Ccc4CpMHMZDxEKOviS1DKKFfiz6Tl5pgJ1hC2EJ2XoINpepFBI9DxLpaZGVicN3kRRkKfOb81KR1azuIjuEQaK23afgzsEVwFjYqn79jkVBmzHWhvnKskjtXRkOjpWfliFAHUG/E9hocLJAWUrOt4UuG7ggDzJLXcZ2YjmMDVPkJ+jRKMLfctTrO0kzn2ADKz0IRu9b657+nLMju5y+n7VNcjM9m/ON2W6ApXsI73xR90M2sJGVTbLNEXFIFwGdMf41D6Q1zRktx0nWXhgmOWfUXv2fkOkAblkoik+Tl9jnNcrGbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NShcA1V2HV3epTS6o8lL4kLKbRNw+R5RpTK7qjXYbhA=; b=KPGRVDR4WzeP8vSft1YMHxcegxqQR3SJ/J6AvXXDhdzsXhS+QrClyrFgX6cK5e4f3H8K2ZVUYXfFEJP7lb4XRJ+Sln/6JfcBUO3ByGLFOxIPmu90hvB01+0I3KzTYpEd0EdOYaaxmBBT578BYvz/qCLPF9rheNa7yQgouzutxAbRRIS0aOvnPR/P9Nsq5sNVBEBInDK/w55FQUrC5oHFDbpHZ+9PneqPD4cd5sm42wbL2tPLHNPa5hwp615m/P4kbUJc1N923hQVExP2hMnLUNiHOvz222mZ8RKQY1BYCnxp/HJDFN2CBdA+DZYoBU9JVT25RCi8PUuMFrTg/uQZcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NShcA1V2HV3epTS6o8lL4kLKbRNw+R5RpTK7qjXYbhA=; b=Acn6SlJ1g/NaRga9gob/PIwRviKiuOe9/FcI/tE0F0j6V1YWnXqNrNN57koA4ouC6KnVo14WILSBkZfWV4k5+gYn4/6YzxJwcDlTokvo1YScwsFLZf4pxh6vNnCLAfj+0Q/lQB8mCWhWKv6OgpuVZeUO15LRCx+oP/CY5Quual8= Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM (2603:1096:400:21::12) by TYZP153MB0399.APCP153.PROD.OUTLOOK.COM (2603:1096:400:21::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.12; Fri, 19 Apr 2024 04:20:14 +0000 Received: from TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e]) by TYZP153MB0399.APCP153.PROD.OUTLOOK.COM ([fe80::788:1883:1f09:480e%7]) with mapi id 15.20.7519.010; Fri, 19 Apr 2024 04:20:14 +0000 From: Wei Hu To: Konstantin Belousov CC: Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: RE: How to add a -W flag in local Makefile Thread-Topic: How to add a -W flag in local Makefile Thread-Index: AdqRkOe7luhT44chS0qlVqfwwelwwgADmX2AAAFW5NAABUv9AAAVW9Dg Date: Fri, 19 Apr 2024 04:20:13 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=20511b01-d971-4d38-8cb9-1a0461ab90ae;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2024-04-19T04:08:49Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYZP153MB0399:EE_ x-ms-office365-filtering-correlation-id: d3e01d49-c83a-4f73-90a2-08dc60280290 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230031|1800799015|376005|366007|38070700009; x-microsoft-antispam-message-info: =?us-ascii?Q?qrJxUMCBTIiki46j/9A9qzt0q5+Rz9ojSWQpZkLS441aOiMNg2ilBJKbBnFl?= =?us-ascii?Q?W10FQae4E2MJQ3D6kklqpAGwke4u2Idc82hGnaDuqAmOQCKpinWlBLUAKcHM?= =?us-ascii?Q?lCTMjMhTytez5shj/jczXuo+yEGc8Y9uwe9RsOlyXVgBg7AzGKBmUV3LQZo9?= =?us-ascii?Q?UyshCDt5xRBv3QR/d7J9K1Bco/nWbrLj1YjW4cCPaioyb7ECuYSNi1hHHPTx?= =?us-ascii?Q?e6qHB4Lgoy9vFGoAoktU3T8/nhanqq4obdFEUXyWWhN//HzSDCACAbuyoAfn?= =?us-ascii?Q?946kvxx1ybi9E9hDN32HlDMt5qqY6aI6uwqhsntsSC3pPmzPPR3MNAsjx0M6?= =?us-ascii?Q?fFkumordIMm+32qH7Cc6nqcjy+P2JcnCbveFUXQDFyNMKCQ3gYs+QO8byEqn?= =?us-ascii?Q?DPHSSulyYCdGilox1ON1VtBgC/zNuXdN/PM3Zeiioxe2dB8h2iZ7UFB/iB13?= =?us-ascii?Q?yK1dT/xY5oZjmnVQ/HmEoPNUm1B2pL3x2mLhzOPxC19YW3MY8+uIDDtBU477?= =?us-ascii?Q?RliMcWnVMTj+wrFusNHY3QvOmfdqBJwowxDVJMwLUoL5Df3JYgG/qv9CqfEy?= =?us-ascii?Q?fwE2he4qR/PPghpqTzCg4NrOcbB8seGFKjmVKAOxh2g9SkYZEjoPu4+pZK1q?= =?us-ascii?Q?MHKHQCpG3MFAnN1rVfL/aaohbSMLfU4amFW3jZZC174AEzaNVIuppRtn2mHc?= =?us-ascii?Q?iCuUuT+Qsz9/3InG8TMtQxkTz9J3rfaY5tFUd55iYQdil4gL06Pi8YCiYcMN?= =?us-ascii?Q?B253EDBC4uIq9FfWn9cpdr3JeQlylE/IbfigWtxYSfkcf3Yt/XV9vsN5f+8V?= =?us-ascii?Q?xZ5LgmMo59YGXU8hXUsMfDeKaQqJQUBWe6qQ/Kv5hnQts5MNHjuXBT3aba+o?= =?us-ascii?Q?stau8b4AU8j6I8rTucQSYRuvON8suGEDZuqpSVS3uUGRMclG6Is0kd+YTNZf?= =?us-ascii?Q?CkWiB9LC8vRKidJ7h3yi+ggIuvpVC3nM4Qch6uEv7bb0f11dhDbyLf7HJdhw?= =?us-ascii?Q?SnUf06t4PxFm1VMvTkVz5T+VXETMRxtxa3Eq5QA4QgNbJRsUM5+Ki9pH9dSw?= =?us-ascii?Q?1Pf8nXVsAheyYdDOe4qpo5GPCY7NMZBl2GsG5WogATchvIcLHUeUdc0gbFw/?= =?us-ascii?Q?d/u8zWzZazB+M8afLncz7qVpoGXmQPM8eAuUWEN6Xh1BnuI5HA4X5h2KFLg/?= =?us-ascii?Q?blwCANqeLR6FOpisAtkUJEOaL0WwmI1FGCGcNl9PgujDm3qLF5uAhto3qnMA?= =?us-ascii?Q?1CcWf6H+uFqmiR+SufFciyAgm8u68FgOyoaENn+5kNYvzFysO5pzJLOUy0Gw?= =?us-ascii?Q?10r0ZH6xS6D7KN0RFGdbLnaStjxX6Sgciatrm8RWWR2m2g=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYZP153MB0399.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?m0+MKk+MCoMqHY/1blwSEtsh460NNi2WTDYo3yfq1xhLsKTXxsdypOKJx06v?= =?us-ascii?Q?T9APJtNfJgfRxF50S+farIiPgGXB176mXbp2C5m34bwubl8jbv0vXK4marlZ?= =?us-ascii?Q?9Y0A9jUDkwkjJYmJmX1z41aXAeQrrU5NKYATt7rOuZkVRwmNKCz6tKvIsrsL?= =?us-ascii?Q?wYsLEbPW7dKUOHOl8Bhqu2dsAyfeYRab9+EYem2aCWtmH5DdNc0L0lqVrAcu?= =?us-ascii?Q?5KS/nHyqY9v1NT1T+JQWwUEB53cUtEMxhfwwPQXhWLxs0EuiEuiWTMZz9m2V?= =?us-ascii?Q?X1ODIEQwNaOW3Lw6WmvORCQn+ZCGFHXJTMW1e8k2ScmDC70u+w69MUL3LIci?= =?us-ascii?Q?h6vyJscn7xzxbPzPfpfuCmj9FY2K2x4B8vYAjyi9Dl/vNUl+z76Qxvf+1OIf?= =?us-ascii?Q?IcTJOc7r1FIX7bTlUTuACihZBu1e3c0KNurUBj9kn4uwgN6UVyANswNPfgOw?= =?us-ascii?Q?+2lP4DmnRduYBcj2kPBoUf9KaL4C0wOGv0XoT2KZXJk0U5U6QYVjDGqRE1V2?= =?us-ascii?Q?U98HCv3v+vNwbHTEynAWGY7WMOEog42ekfbPEMFfW+gdAv2zdINKe3njk2zs?= =?us-ascii?Q?9Mr7AxZx2PJV8GD/fRCeZyr+gnwGG8bG1eil8RHF/iMNGc/M7C56AedKu061?= =?us-ascii?Q?sixbZxwtDqw3Gc7c4wbdcNckuBeMhrNRa9UZ4WlIrrJ3KnmtwXn8sxpI0XvW?= =?us-ascii?Q?1gLvCrrJpNlbYaT59ijtr/hJ945beRIthIrkwWJ5RgBh46CG+JcozEPtjqjO?= =?us-ascii?Q?8X9KJvbOFqgF/UrGTI0R9y5DTtLx0S/00GMaIBaFBc5TNYTlfjHJgq/hqZHn?= =?us-ascii?Q?V5hqTxYgncKBJJbSRTETNRVNpG49Nj0WIWd13tiU8LxWLFJwW7KAu3Gh2YiB?= =?us-ascii?Q?V7KlI/D3oAmEo+m7SL8d1N5fFs4Etwl548+W7m3Cs1kg2YsxtEHmg5sXEzyh?= =?us-ascii?Q?64BVDbMf6dXlDE0FQ62v/Or+rm/0yF8olMmsy0W46H7gYzoPROuMWgTkfVvR?= =?us-ascii?Q?i8/DtvT3IrVemDiZPHLf9CUvJU9Wf30hw4/R5BeicQQBqgjDp5BqIz7Ur/ym?= =?us-ascii?Q?1hXJVexrJ0f4hZQSxjr+0Ai5/HCb5ZnboEB1szhntFEWupttxbgHEtuRcQEm?= =?us-ascii?Q?WALO82vpnues6SLODL3AcibZMwOen6iRWcqFO13cDbIpQ/wkXUBFOLQPR8Z1?= =?us-ascii?Q?ALuOuccITLbXCWLqE9RLOj4lpNhRGhrjx0wJvUfiWXJFpb+6Oj1butl0egwr?= =?us-ascii?Q?8KXVC5VYeHh28HzlawXp8JESb5ihX3MIvoGLusti06dbl6ryNV624SM4yPsx?= =?us-ascii?Q?v/fR1fSBjhnbkPMpMWBQGYNnTQL1yjkHoidNkTAaEElmVTzccvnwoGk/GSkZ?= =?us-ascii?Q?gdt8NN4BcFLYEKMVWmdDaRYS8KPLdFpaqpCwBZJ2AFUaB6fVpYWJkN4i/NNj?= =?us-ascii?Q?IDOtSmbASfqHi+uL6X//tv+75jhc/XSI1llxm9HqeVBpOiEuYURlkuAIe8sg?= =?us-ascii?Q?k6y4vSfbsfnUdFJv4MCy0eWJH7e+5BiLrnTMROcJ1oldQHUIYkqVyBVcOAEd?= =?us-ascii?Q?qlXI4NW08EEzmd17fv2BZNY3Ba4lWoqea5hNjT7tICjyzCwMMP1T2/wczbwI?= =?us-ascii?Q?RbTKMra3+6GzOzZ5UtaIXKw=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYZP153MB0399.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: d3e01d49-c83a-4f73-90a2-08dc60280290 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Apr 2024 04:20:13.7365 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dtrDmcnvk0q86ZErH7YdtbK8tMtxj97R2D131RlM62n/QCwi5LkfHoaZrmDreggpSB6gtNPj0EbWorNLTXyFFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZP153MB0399 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US] X-Rspamd-Queue-Id: 4VLLz963DWz4FmQ > -----Original Message----- > From: Konstantin Belousov > Sent: Friday, April 19, 2024 1:57 AM > To: Wei Hu > Cc: Warner Losh ; freebsd-hackers@FreeBSD.org > Subject: Re: How to add a -W flag in local Makefile >=20 > On Thu, Apr 18, 2024 at 03:27:13PM +0000, Wei Hu wrote: > > I added something like: > > > > diff --git a/sys/dev/hyperv/vmbus/vmbus_var.h > > b/sys/dev/hyperv/vmbus/vmbus_var.h > > index b598f782947e..6e3b7b040827 100644 > > --- a/sys/dev/hyperv/vmbus/vmbus_var.h > > +++ b/sys/dev/hyperv/vmbus/vmbus_var.h > > @@ -193,4 +193,17 @@ struct hyperv_tlb_flush { > > uint64_t hv_vm_tlb_flush(pmap_t pmap, vm_offset_t addr1, > > vm_offset_t addr2, cpuset_t mask); > > > > +struct hv_vpset { > > + uint64_t format; > > + uint64_t valid_bank_mask; > > + uint64_t bank_contents[]; > > +} __packed; > > + > > +struct hv_tlb_flush_ex { > > + uint64_t address_space; > > + uint64_t flags; > > + struct hv_vpset hv_vp_set; > > + uint64_t gva_list[]; > > +} __packed; > > + > > #endif /* !_VMBUS_VAR_H_ */ > > > > So, the struct hv_vpset is the second last member of struct hv_tlb_flus= h_ex. > The member bank_contents[] in struct hv_vpset is of variable length. This > would makes the last two members of struct hv_tlb_flush_ex both variable > length. Therefore, the flag '-Wno-gnu-variable-sized-type-not-at-end' is > needed, otherwise it would complain about this with errors like: > > > No, the compiler' complain is correct. The structure definition for > hv_tlb_flush_ex does not make sense: you never can access gva_list (excep= t for > the case of bank_contents being zero-sized). The same code already exists in Linux and Windows. Linux also added a compi= ling flag to suppress similar warnings.=20 Anyway, the purpose of this email is to understand how to add such flags in= local Makefiles and make it effective for global buildkernel. Adding such flags in local Makefiles already proves to be working when only building under lo= cal=20 directory. Thanks, Wei From nobody Fri Apr 19 10:58:19 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLWpR0dc9z5GqL1 for ; Fri, 19 Apr 2024 10:58:23 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLWpR03Yxz4sJY; Fri, 19 Apr 2024 10:58:23 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713524303; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pJRIhblUze02avDnXXmMMaUugNGbDBDQcqopnQTN0gQ=; b=HWjEw6yPTaYxxzGN3jfm+C9820fmG8d5bBk2uiTgGekXMwGnqIt3wIOLaBafVHU8Ae/g4L lrtp+HiSMm73QST3jjOa25yPNTRm6B2JwG+Is1T3zF/ieG8rUrvUGHVjF52Ogw9EvVEMKn Np5er3qJBOleCdXPcCScSctxdhOKzgR65YhwfmF1C38lmyTzVKBHiA3MnoMtavzCEbn8tr np+i+fZPwRoNe4Oy07HOoLCWJXxSYV9GVoxzUHrCjSk9QdrLrCinnoW2pkQq4MLli86+9j Aix8KZINJ5bm+5fqYVlguwmgRbk+WILGypjDNa/bNKw/Acisol5WI3yEsV+CkQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713524303; a=rsa-sha256; cv=none; b=y4a29BeC6rPIaLe+LT5A/E56w1fyjr4Ud8gegN3WQ2uN2Xb165g8tFaitMy30ObU3j17WK 5mDyh6nH1qQ1Gxjjy2UWoitC0Hcyfu7Q3DTacFetR+xAC5+T1r1ie/m4jCgSSOztTJImtO RRex6Q1XHxw38BhZYcF2Fg7kpIu/KddZxNr8P9AWq2fEZZJhstsMfkcNVK3jMunbOTpOj7 RO0ryTCX5asLwziw2ODeZR0JJOqe8+ln2eX6PqNfx5Iim9toWqi0fRTcFwyG5OR5AZbvA4 WoYCSyl/Us2AWzd/x+R2T9cea5j+PZCSj1+75oWx4uHhRNmOgkSXvDIKZo3FcQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713524303; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pJRIhblUze02avDnXXmMMaUugNGbDBDQcqopnQTN0gQ=; b=C1xOeror/sNayS4H5ObbSkfdiODG8gpIsLc5wbD+D7K5ItItAhv+dCm1ufhY4WajmJByI5 BJTCa+0Z3MLJc/yWEEl9mbTfxoBL5vYFiMs0/4v0Si0canNVDEGxBHEbX0w+Wqsdcx/03L /60WqTQfvMKaYLd4U9gHIVIL6A/mfub3PY5XB0u3WVQZExUA0HUP5cIbrRasoiYW1HWoL/ HzTnsVe+lXBZyFE/JwETpi8PuZqblF20cZELZOAGqU25KwJvwRK0AE09Bjg8z1xciZSKub H/ITBKIuIFUFAO3TNPaLwqxvFTFiaekiO8cK6rJdYZBtZtI3QwLFfJmY7J3jQg== Received: from ltc.des.dev (163.23.65.37.rev.sfr.net [37.65.23.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VLWpQ640JzYFl; Fri, 19 Apr 2024 10:58:22 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 7B0521C2BA; Fri, 19 Apr 2024 12:58:19 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Wei Hu Cc: Konstantin Belousov , Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: Re: How to add a -W flag in local Makefile In-Reply-To: (Wei Hu's message of "Fri, 19 Apr 2024 04:20:13 +0000") References: User-Agent: Gnus/5.13 (Gnus v5.13) Date: Fri, 19 Apr 2024 12:58:19 +0200 Message-ID: <86cyqlbovo.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Wei Hu writes: > The same code already exists in Linux and Windows. Linux also added a com= piling > flag to suppress similar warnings.=20 That's not an excuse. The code makes no sense. > Anyway, the purpose of this email is to understand how to add such flags = in local > Makefiles and make it effective for global buildkernel. Adding such flags > in local Makefiles already proves to be working when only building under = local=20 > directory. You would have to add it to every line in sys/conf/files* that references a C file which includes this header. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Fri Apr 19 13:52:07 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLbgg3pSdz5H5vm; Fri, 19 Apr 2024 13:52:47 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLbgf3zBLz4DyD; Fri, 19 Apr 2024 13:52:46 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=R1jYgTIo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2607:f8b0:4864:20::1031 as permitted sender) smtp.mailfrom=marietto2008@gmail.com Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2ab1ddfded1so1682064a91.1; Fri, 19 Apr 2024 06:52:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713534764; x=1714139564; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=76HQ1IRkLQS/sSO3lxdfUJGz301AuR//klk9TW/6nt8=; b=R1jYgTIoHZ53s0hJn4aSXHBCCMNGKhPjrlPvrj/Xjrhqq3PbbULqkKmD0Z2dyvKQ1S ul+5vFVR4RG10whv2m3FjQN1xguI4+r9ERV9GL/Gg5tJIytRLOHADUMbPIbqPuv7OmOF 0Eu6knls972yXYShWJql4YqmxKQJQDBqX4Q5ZcwwMNeGtnoUiH3o6vPxOKoua1JDjsA0 lb6f7FWW/9YMYow3wQe/ixWZRwdb7DCUPMaYCH9TclJYPWwH35sM3BHkCw2JxdeNU0wc KZ5VdFGzUclGxTPuuUDJgwpI7RGIbRswS6UO2LL3Wl89yZtA0+CZE4fWB276fBll9PiQ YfEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713534764; x=1714139564; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=76HQ1IRkLQS/sSO3lxdfUJGz301AuR//klk9TW/6nt8=; b=kfg8+tiY489i6l5my4XDqw1OLhILzmH9WuhKu/vMiwlQSZpW/Z24CzFbplbHu5gS8j xcwlRm4DQ50sOAiWrNikhjvm/4WmvL+ug83+0R1qgBadmOrN+VO94wPdH+t0YDFn+b16 jtUYv1QeC9PndFajSP847VabgNxOOQwY7z15mFQxIOalOv/MEP15fVYwoTh/FG2fH3gN Y9drf13tbZx3do2yqz1u05vrNt9t1nRZsxhNKi3zt8/DG7B0PlrgOQ3madzBKodPkaGj 1z2SyXLRqoeyDGiT6iv1Ui1lVUDU6lGINXDu8+0rnHxMBO9i8XzVFn+syuAUzm7RVz/f R+ew== X-Forwarded-Encrypted: i=1; AJvYcCUmML8bpzRg8F9zbixPyrkhSReU07VZtz9F2ZkwXWaYQnxXQe0slLfamG6LQlbfMq5XvCZis6kHGYRvJ7Xn7IkdW4W6p2S0W3VRY62bILY7PkN7BO15Vsxu7zX8ITImgshk0rykAdwgwMRe X-Gm-Message-State: AOJu0Yz0s0h7/6EAoof0XR7vvPgfQCs3HUp9GzBQHt7aZsEQ+oQJpSIw PVA+TM8jjSW2fLq4EArGPwqTDHgC6unhuFR5Mdt0EVd0XD0lksKA3nrJd8jviUlbR/yMGP9vKyD KQmhqA/9mM8zrB1A5NF2zvfa/0LQnbAh2zvQ= X-Google-Smtp-Source: AGHT+IH/xpO4IP5/G2Mp8Ppvh9Pg5WfISfkvugD+nXcWP/NzXStlxkmI0+S77Ir46+YdhbWfgJp6szvJ2PFTiONe3Rs= X-Received: by 2002:a17:90b:3c4:b0:2a5:8ff:9d1 with SMTP id go4-20020a17090b03c400b002a508ff09d1mr2263570pjb.14.1713534764349; Fri, 19 Apr 2024 06:52:44 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: Mario Marietto Date: Fri, 19 Apr 2024 15:52:07 +0200 Message-ID: Subject: Trying to virtualize ReactOS with bhyve : I don't see a viable way to do it... To: FreeBSD virtualization , freebsd-hackers , FreeBSD Mailing List Content-Type: multipart/alternative; boundary="000000000000f0854506167366a7" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.990]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1031:from]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org,freebsd-hackers@freebsd.org,freebsd-questions@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+] X-Rspamd-Queue-Id: 4VLbgf3zBLz4DyD --000000000000f0854506167366a7 Content-Type: text/plain; charset="UTF-8" Hello. I'm trying to boot the 32 bit ISO image of ReactOS using bhyve or qemu and seabios as bootloader. As first experiment I tried to boot the x86 version of ReactOS with qemu (8.2.2) and seabios (version 1.16.1_1),like this : qemu-system-x86_64 -machine q35 -m 1G -cdrom /home/marietto/Desktop/Downloads/OS/ReactOS/reactos-livecd-0.4.15-dev-7921-g6d853be-x86-msvc-win-dbg.iso -boot order=d Unfortunately,I get the error below : https://ibb.co/tZmFh2x but my real goal is to boot ReactOS by using seabios. Even here I'm unlucky. Infact on this web page : https://eradman.com/posts/bhyve-ipxe.html we can read : *Bhyve is somewhat similar to QEMU-KVM without the emulation required to boot SeaBIOS.* Is really true that I can't use seabios to boot ReactOS with bhyve ? Anyway,ok. I've continued my searching and I found this thread : https://forums.freebsd.org/threads/bhyve-with-bios-boot.79794/ where we can read : *bhyveload has nothing to do with BIOS. As detailed in the man page you linked to, it is for loading a FreeBSD guest.* So,it seems that I can't even use bhyveload. So,what can I use ? What can I do ? I've gone on the ReactOS forum and I've created a thread where I 've asked how to boot the 64 bit + UEFI version of ReactOS using bhyve. The developers helped me,but in the end we have done nothing. Because after some progress,I found this error : https://ibb.co/N9QHdnp Unfortunately it is frozen when it tries to load the mountmgr.sys driver. Actually unfixed and no one knows how to fix it. Now I'm here,asking for some further ideas and suggestions. -- Mario. --000000000000f0854506167366a7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello.

I'm trying to boo= t the 32 bit ISO image of ReactOS using bhyve or qemu and seabios as bootlo= ader.=C2=A0

As first experiment I tried to boot th= e x86 version of ReactOS with qemu (8.2.2) and seabios (version 1.16.1_1),l= ike this :

qemu-system-x86_64 -machine q35 -m 1G -cdrom=20 /home/marietto/Desktop/Downloads/OS/ReactOS/reactos-livecd-0.4.15-dev-7921-= g6d853be-x86-msvc-win-dbg.iso -boot order=3Dd

Unfortunately,I get the error below= :


but my real goal is to boot React= OS by using seabios. Even here I'm unlucky. Infact on this web page :


w= e can read :

Bhyve is somewhat similar to QEMU-= KVM without the emulation required to boot SeaBIOS.

Is really true that I can't use s= eabios to boot ReactOS with bhyve ?
Anyway,ok. I've cont= inued my searching and I found this thread :

<= a href=3D"https://forums.freebsd.org/threads/bhyve-with-bios-boot.79794/">h= ttps://forums.freebsd.org/threads/bhyve-with-bios-boot.79794/

where we can read :

bhyveload h= as nothing to do with BIOS. As detailed in the man page you linked to, it i= s for loading a FreeBSD guest.

So,it seems tha= t I can't even use bhyveload. So,what can I use ? What can I do ?
<= /div>

I've gone on the ReactOS forum and I've cr= eated a thread where=C2=A0
I 've asked how to boot the 64 bit= + UEFI version of ReactOS using bhyve. The developers helped me,but in the= end we have done nothing. Because after some progress,I found this error :=

https://ibb.co/= N9QHdnp

Unfortunately it is frozen when it tries to load the mountmgr.sys driver. A= ctually unfixed and no one knows how to fix it.

Now= I'm here,asking for some further ideas and suggestions.
=
--
Mario.
--000000000000f0854506167366a7-- From nobody Fri Apr 19 22:23:51 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLq1d23RLz5HsB6 for ; Fri, 19 Apr 2024 22:24:05 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f174.google.com (mail-vk1-f174.google.com [209.85.221.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLq1c36d9z4Hfq for ; Fri, 19 Apr 2024 22:24:04 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-vk1-f174.google.com with SMTP id 71dfb90a1353d-4daa91c0344so770979e0c.3 for ; Fri, 19 Apr 2024 15:24:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713565443; x=1714170243; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=x71Exf+DYrTrHrLNSKVuEDO8b23MEqFpBlcYsekN/JU=; b=HozcKKEHZJAmmL3DvFhoghPytlOSf5UDDVLNOrcPq85aLKjny1aeFJaKTMVI+wby+Z OxT0lOXAdKcpyw+B7YAd05c3V7JA4ZS9F3qvwEEAfJoFI/GcLzTToAfWvyBf3W3r1MjP 7OBgbHvORyCqa0S9cU3XJHBX2TGN+anAqQypHUlnVJh+/70vAyF2bhWLA/DJ2poIQsnK SowBdptrUBVD29LQW/xjRtdF4WIGIVnVWd9oc/z0IuS82AuIgDfD7LO/RTKJtJC0PDeE EkGt0N6CZj/Bl08vLWIvwWEIhGkXNq7FaRNXh6tstG3DlWWErQLcGjnxoDc1t68bLV0M psiQ== X-Gm-Message-State: AOJu0YzpHzcOXmpAHnIIaN1jclJAlW+2tnoZj1CHq1LxyKSDj8nG9NZA 53mfaphD/Qpw0V8lrn/F0MwSgefjkFXMeL8AYfjLFG7vCdZX2fM3RnXDnCgHnkVERRUo9VDr71K +MfULaXjyQWYa88Lxiq/jCVm+gBmvm2QP X-Google-Smtp-Source: AGHT+IET23J+45MdLa1MLoxkwB+liWUHCMH3/fQw+Wwrn+DRTl6+LQ3G5luxvXqAhuyttND1B2uSKifXb3bdLHK5esc= X-Received: by 2002:a05:6122:3d05:b0:4d3:43f8:8541 with SMTP id ga5-20020a0561223d0500b004d343f88541mr3784652vkb.1.1713565442665; Fri, 19 Apr 2024 15:24:02 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: Alan Somers Date: Fri, 19 Apr 2024 16:23:51 -0600 Message-ID: Subject: Stressing malloc(9) To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: - X-Spamd-Result: default: False [-1.92 / 15.00]; NEURAL_HAM_LONG(-0.97)[-0.970]; NEURAL_HAM_MEDIUM(-0.53)[-0.531]; NEURAL_HAM_SHORT(-0.52)[-0.516]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.174:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.174:from] X-Rspamd-Queue-Id: 4VLq1c36d9z4Hfq TLDR; How can I create a workload that causes malloc(9)'s performance to plummet? Background: I recently witnessed a performance problem on a production server. Overall throughput dropped by over 30x. dtrace showed that 60% of the CPU time was dominated by lock_delay as called by three functions: printf (via ctl_worker_thread), g_eli_alloc_data, and g_eli_write_done. One thing those three have in common is that they all use malloc(9). Fixing the problem was as simple as telling CTL to stop printing so many warnings, by tuning kern.cam.ctl.time_io_secs=100000. But even with CTL quieted, dtrace still reports ~6% of the CPU cycles in lock_delay via g_eli_alloc_data. So I believe that malloc is limiting geli's performance. I would like to try replacing it with uma(9). But on a non-production server, none of my benchmark workloads causes g_eli_alloc_data to break a sweat. I can't get its CPU consumption to rise higher than 0.5%. And that's using the smallest sector size and block size that I can. So my question is: does anybody have a program that can really stress malloc(9)? I'd like to run it in parallel with my geli benchmarks to see how much it interferes. -Alan From nobody Fri Apr 19 22:46:04 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLqW84QGrz5Httc for ; Fri, 19 Apr 2024 22:46:12 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VLqW72pm9z4Mtt for ; Fri, 19 Apr 2024 22:46:11 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jeffpc@josefsipek.net designates 71.174.62.3 as permitted sender) smtp.mailfrom=jeffpc@josefsipek.net Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 295BC2BDAB for ; Fri, 19 Apr 2024 22:46:05 +0000 (UTC) Date: Fri, 19 Apr 2024 18:46:04 -0400 From: Josef 'Jeff' Sipek To: freebsd-hackers@freebsd.org Subject: Upgrading -RELEASE to -CURRENT Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_NO_TLS_LAST(0.10)[]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[josefsipek.net]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VLqW72pm9z4Mtt Perhaps this is a newbie question, but whenever I try to move a freshly installed 14.0-RELEASE-p6 to -CURRENT, I seem to have to first install -STABLE and then build -CURRENT. That is: 1. install 14.0 from ISO 2. freebsd-update 3. get stable/14 4. buildworld; buildkernel; installkernel; reboot; etcupdate -p; installworld; etcupdate -B; make delete-old; reboot 5. get main 6. buildworld; buildkernel; installkernel; reboot; etcupdate -p; installworld; etcupdate -B; make delete-old; reboot If I skip steps 3 & 4, I end up with every C++ binary (notably cc) fails to execute because ld.so is unhappy about libc++.so not having the right version. (Sorry, I didn't write down exact error and don't have a busted system on hand.) Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work? I didn't see anything in the docs that says it shouldn't. Thanks, Jeff. From nobody Sat Apr 20 00:08:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLsL15lNmz58rg9 for ; Sat, 20 Apr 2024 00:08:25 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wfout6-smtp.messagingengine.com (wfout6-smtp.messagingengine.com [64.147.123.149]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLsL06jYgz4XxF for ; Sat, 20 Apr 2024 00:08:24 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=azarQ5Yn; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=O+KP0NFe; dmarc=pass (policy=none) header.from=f-m.fm; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.149 as permitted sender) smtp.mailfrom=void@f-m.fm Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.west.internal (Postfix) with ESMTP id 6207A1C00101 for ; Fri, 19 Apr 2024 20:08:23 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 19 Apr 2024 20:08:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1713571702; x=1713658102; bh=W+UgVpu/mE BedcfV/D8toqyVPNF+214BsEFnXorC0+U=; b=azarQ5Ynpz7jxo3Lvr9etO+v3f WVKj1wuqkjaZH5qEO31Sb4Ngoza9Bbi7G8ZtByivEWxIIBuuv/in8IgjP3pv0afu KUOImuh50ooxIhElglOeneWplWNQvVkc3SbSCbb/65QLh2Oum6hGD1OG/koh1Hnk Miqw1EAxprzszrHj8e+Bdgv+aZLfDNXd5xf7mXtvhzNJnOq+PhLGCkj+oQ7C/NoK QvDppwxy8mVm9jYfUY+KUn6ioq0imjLCO5SmEf1uGn8glyk8XdYYBX3Oh5RbfKMZ N3BEq+39AEtKuu4bWcvEUhFO2mhUYdHShFibtqxu37HHUSYdSDvTFgmQCG3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1713571702; x=1713658102; bh=W+UgVpu/mEBedcfV/D8toqyVPNF+ 214BsEFnXorC0+U=; b=O+KP0NFelunoE8Y7R3/oE+r4F5smsNe14xjo/xcHhVo9 RGcnMzz4zNL233pAMdM/VDSc13+d0BDXZJ51CDzJJxINfJTzWXSwEGX1EO0Fpm0v Ark5sUxZqDdL/l4zs54cj5rDWLP2AFAhatjxKqawbSdVxFr/alXw0mN71e4tymzT h6klhBgH5ExbYXpY+LOPl+hapvoCJ7+ZMi8wYKdG2YuxgQblsTK1sDFSJ9nRV4mN xC7RzNYdRgaKFDnEloG75JdyR8/YqAxKljkOABPztDbaOdb/fYT2POVbrIdQyxyK pNBccYh+SxjqZEXlTWyBd7kqUFTenj/LcQaHRCeRPQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudekfedgfedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepkeeluddvlefhieelfefggffhffektdehleelgfdugfdvgeekjeejuddthe ehgfeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep vhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 19 Apr 2024 20:08:22 -0400 (EDT) Date: Sat, 20 Apr 2024 01:08:20 +0100 From: void To: freebsd-hackers@freebsd.org Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.58 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.982]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.128/27]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.149:from]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim] X-Rspamd-Queue-Id: 4VLsL06jYgz4XxF On Fri, Apr 19, 2024 at 06:46:04PM -0400, Josef 'Jeff' Sipek wrote: >Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work? I >didn't see anything in the docs that says it shouldn't. I think the general principle is "upgrade to latest version of thing before upgrading major version" so for 14.0 that would involve building & installing 14-stable and then 15-current. -- From nobody Sat Apr 20 00:15:59 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLsWF4ltcz58sPl for ; Sat, 20 Apr 2024 00:16:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLsWD37pgz4ZLZ for ; Sat, 20 Apr 2024 00:16:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=xjOQyZDn; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::32c) smtp.mailfrom=wlosh@bsdimp.com Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-419d320b899so3695985e9.1 for ; Fri, 19 Apr 2024 17:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713572180; x=1714176980; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=qxvizyzSmfRHLLpv4jYHGBCPnDs6PaP9FAHpT+lB8hk=; b=xjOQyZDnNoFbwoi8NfRntNMlJkyGLKFAIa1bSm/tcxz1oVS1LfLR7NwVKgwl8et8Tw 8+AIoYhvqL/7NhPy9u9JGjRRPSm2pizBsBrL9BUtl0qPNAInq3S/S6aya8yKCSJB50u7 aNlCUQNEgPRv2JVqTKt9VhxW8xuEg0rvS5NjHp+w2Z5NX/uJyBXabwy1OwKa6S/nRW8X AI/80uyvxUJc+6910+l3OC4GC5mLJgZrfSdfw0Oo7CESfvvjYMidByLkl5suphLNeQfK EMV5PMkefDMV1La8CYiaE8XdSmuQaPzwdSbk3jkzS5sx6rJtMk53wBxlf5ouf8TmR356 f/Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713572180; x=1714176980; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qxvizyzSmfRHLLpv4jYHGBCPnDs6PaP9FAHpT+lB8hk=; b=rN+VOkl033gU/ekebAx2FyVTJHcm5y7EHtFprO4gsefS/iypGIiqc6MU+htya5MePf URoS3Ydy8zawDFb4VVb50STGoNAgvBizvRjB8M6jHT5G/IPmMNp0o4GZ/ASR5Xj8UF3B uihqwQURsOD9r2U2frLhhbTg/4r1zvWv44YVzm5mqE+FJ38xGD8a5EIpfmuvpAz5QENt iUEjz1LzEa1nHVjzCMB06nnqxaUGFwCXsjG5/lYYzZXDXNEEwE7FrxnQ8dbN5K9MmO+N Qg9sM2OC+G/9Fc5AL6V/dBUaPubBbbN7rPVY555904aISU1koFYLU+0pqWLdRc89ICLw kFBA== X-Gm-Message-State: AOJu0YyuAz8lpxdbJibvRRJmsSdfW4fHKTxk+4q8aCRoRpszMwKIkpom H9adZJI8Dkeaadhw1ZFBd38x2oEiVQjErAwVjaUiLY5d1OXa4LmvCHnIgmrQUZBu+BdN7U0T2RW jnmRaQjWILBDWiJkvhoC3SGwalf3riY3PviYirPq15gyodoQqzZ8= X-Google-Smtp-Source: AGHT+IHr9lKwK0XHy8FgNrp4t1v29BxSeTXN9DkbhkIp/DPxUH/S6jTnlYpeQwX5+LrzO3C1fqPRwTenLXEExlJB82Y= X-Received: by 2002:adf:f9c7:0:b0:349:d779:3d60 with SMTP id w7-20020adff9c7000000b00349d7793d60mr2863581wrr.57.1713572180184; Fri, 19 Apr 2024 17:16:20 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Fri, 19 Apr 2024 18:15:59 -0600 Message-ID: Subject: Re: Upgrading -RELEASE to -CURRENT To: FreeBSD Hackers Content-Type: multipart/alternative; boundary="00000000000018f89306167c1da3" X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MISSING_XM_UA(0.00)[]; DMARC_NA(0.00)[bsdimp.com]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32c:from]; ARC_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4VLsWD37pgz4ZLZ --00000000000018f89306167c1da3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Apr 19, 2024, 6:08=E2=80=AFPM void wrote: > On Fri, Apr 19, 2024 at 06:46:04PM -0400, Josef 'Jeff' Sipek wrote: > > >Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work? I > >didn't see anything in the docs that says it shouldn't. > > I think the general principle is "upgrade to latest version of thing > before upgrading major version" so for 14.0 that would involve building & > installing 14-stable and then 15-current. > Last release to current almost always works. So 14.0 to current is doable. I did 14.0 beta1 -> current 3 weeks ago. Etcupdate pre mode, Buildworld, buildkernel, installkernel, reboot, installworld, etcupdate, reboot -r But OP said they did that twice, once for stale/14 and once for main so I'm not sure what is going on. Need error message i think. Warner Warner > --00000000000018f89306167c1da3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, Apr 19, 2024, 6:08=E2=80=AFPM void <void@f-m.fm> wrote:
On Fri, Apr 19, 2024 at 06:46:04PM -0400, Josef 'Jeff&#= 39; Sipek wrote:

>Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work?= =C2=A0 I
>didn't see anything in the docs that says it shouldn't.

I think the general principle is "upgrade to latest version of thing before upgrading major version" so for 14.0 that would involve buildin= g &
installing 14-stable and then 15-current.


Last r= elease to current almost always works. So 14.0 to current is doable. I did = 14.0 beta1 -> current 3 weeks ago.

Etcupdate pre mode, Buildworld, buildkernel, installkernel, r= eboot, installworld, etcupdate, reboot -r

=
But OP said they did that twice, once for stale/14 and on= ce for main so I'm not sure what is going=C2=A0on.=C2=A0 Need error mes= sage i think.

Warner

Warner
--00000000000018f89306167c1da3-- From nobody Sat Apr 20 01:35:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLvGx5Jw4z59199 for ; Sat, 20 Apr 2024 01:35:53 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLvGt5QRdz4jqM for ; Sat, 20 Apr 2024 01:35:50 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp designates 153.125.133.21 as permitted sender) smtp.mailfrom=junchoon@dec.sakura.ne.jp Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 43K1Zf3A030433 for ; Sat, 20 Apr 2024 10:35:41 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sat, 20 Apr 2024 10:35:41 +0900 From: Tomoaki AOKI To: freebsd-hackers@freebsd.org Subject: Re: Upgrading -RELEASE to -CURRENT Message-Id: <20240420103541.0762b8f6bf6baa2c4b8b9f47@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.51 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.81)[-0.814]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:153.125.133.16/28]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; HAS_ORG_HEADER(0.00)[]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[sakura.ne.jp]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VLvGt5QRdz4jqM On Sat, 20 Apr 2024 01:08:20 +0100 void wrote: > On Fri, Apr 19, 2024 at 06:46:04PM -0400, Josef 'Jeff' Sipek wrote: > > >Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work? I > >didn't see anything in the docs that says it shouldn't. > > I think the general principle is "upgrade to latest version of thing > before upgrading major version" so for 14.0 that would involve building & > installing 14-stable and then 15-current. Another option would be installing the latest snapshot of stable/14 as the start line instead of 14.0-RELEASE. https://www.freebsd.org/snapshots/ https://download.freebsd.org/snapshots/ -- Tomoaki AOKI From nobody Sat Apr 20 02:05:17 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLvww0Plhz5GkW1 for ; Sat, 20 Apr 2024 02:05:20 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VLvwv5KMdz4mkH for ; Sat, 20 Apr 2024 02:05:19 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; none Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 0B6172BE8A; Sat, 20 Apr 2024 02:05:19 +0000 (UTC) Date: Fri, 19 Apr 2024 22:05:17 -0400 From: Josef 'Jeff' Sipek To: Warner Losh Cc: FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US] X-Rspamd-Queue-Id: 4VLvwv5KMdz4mkH On Fri, Apr 19, 2024 at 18:15:59 -0600, Warner Losh wrote: > On Fri, Apr 19, 2024, 6:08 PM void wrote: > > > On Fri, Apr 19, 2024 at 06:46:04PM -0400, Josef 'Jeff' Sipek wrote: > > > > >Is upgrading from 14.0-RELEASE directly to -CURRENT supposed to work? I > > >didn't see anything in the docs that says it shouldn't. > > > > I think the general principle is "upgrade to latest version of thing > > before upgrading major version" so for 14.0 that would involve building & > > installing 14-stable and then 15-current. > > > > > Last release to current almost always works. So 14.0 to current is doable. > I did 14.0 beta1 -> current 3 weeks ago. > > Etcupdate pre mode, Buildworld, buildkernel, installkernel, reboot, > installworld, etcupdate, reboot -r > > But OP said they did that twice, once for stale/14 and once for main so I'm > not sure what is going on. Need error message i think. Ok, trying again: 0. install from ISO & freebsd-update to 14.0-RELEASE-p6; install git from pkg 1. git clone main (1bd4f769caf) into /usr/src 2. cd /usr/src 3. make buildworld -j48 First build I get a fairly prompt failure: sh: llvm-min-tblgen: not found Some flailing around with 'make clean' and retrying the same buildworld gets past it. At this point, I have llvm-min-tblgen{,.debug} in /usr/obj/... (I'm assuming that there is some missing dependency in tools build.) Weirdly enough, from this point on, if blow everything away (git clean -fdx; rm -rf /usr/obj/*), the subsequent 'make buildworld -j48' is fine. So, a. the first time(?) I try buildworld, it fails because of missing llvm-min-tblgen b. subsequent buildworlds complete fine even if there is no OBJDIR and the tree is clean 4. make buildkernel -j48 5. etcupdate -p 6. make installkernel -j48 7. reboot (into single user mode) 8. zfs set readonly=off zroot/ROOT/default 9. zfs mount -a 10. cd /usr/src 11. make installworld -j48 12. etcupdate -B This prints "Failed to build new tree." and exits with 1 13. cc ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required by /lib/libc++.so.1 not found 14. reboot -r (sort of pointless to mention given that the system is already in a broken state) (At this point, cc, and other C++-based programs fail to run.) So, it looks like 'etcupdate -B' is upset for whatever reason. I'll try to poke at various things, but I've seen these two failures (llvm-min-tblgen and CXXABI) about half a dozen times when I try to go from 14.0-RELEASE to -CURRENT, but *zero* times if I make a detour through -STABLE. Thanks, Jeff. P.S. I've seen the same exact failures on my "antique" 4-core Nehalem. So, I don't think it has to do with the high parallelism (-j48) in the above steps. From nobody Sat Apr 20 02:30:48 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLx9t6sDqz5GqCM for ; Sat, 20 Apr 2024 03:01:38 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4VLx9s6Gm4z4sLc for ; Sat, 20 Apr 2024 03:01:37 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none); spf=softfail (mx1.freebsd.org: 198.144.192.42 is neither permitted nor denied by domain of yuri@FreeBSD.org) smtp.mailfrom=yuri@FreeBSD.org Received: from [192.168.5.3] (c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id 43K2Uo1h090029 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 19 Apr 2024 19:30:50 -0700 (PDT) (envelope-from yuri@FreeBSD.org) X-Authentication-Warning: shell1.rawbw.com: Host c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116] claimed to be [192.168.5.3] Content-Type: multipart/alternative; boundary="------------ia7MQZaA0Ws91RXSo70zGLao" Message-ID: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> Date: Fri, 19 Apr 2024 19:30:48 -0700 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Freebsd hackers list From: Yuri Subject: What does this error mean: No space available for static Thread Local Storage ? X-Spamd-Bar: + X-Spamd-Result: default: False [1.72 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.988]; ONCE_RECEIVED(0.10)[]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_NO_TLS_LAST(0.10)[]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; GREYLIST(0.00)[pass,body]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[yuri]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_SPF_SOFTFAIL(0.00)[~all:c]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:7961, ipnet:198.144.192.0/23, country:US]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org]; TO_DOM_EQ_FROM_DOM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; HAS_XAW(0.00)[] X-Rspamd-Queue-Id: 4VLx9s6Gm4z4sLc This is a multi-part message in MIME format. --------------ia7MQZaA0Ws91RXSo70zGLao Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, The shared library that is built by the Rust's toolchain for the port misc/py-polars fails to be loaded: No space available for static Thread Local Storage What does this mean, and what might be wrong? Thank you, Yuri --------------ia7MQZaA0Ws91RXSo70zGLao Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hi,


The shared library that is built by the Rust's toolchain for the port misc/py-polars fails to be loaded:

No space available for static Thread Local Storage


What does this mean, and what might be wrong?



Thank you,

Yuri


--------------ia7MQZaA0Ws91RXSo70zGLao-- From nobody Sat Apr 20 08:29:38 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM4Sw6gqLz5HK2s for ; Sat, 20 Apr 2024 08:30:08 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-vs1-f50.google.com (mail-vs1-f50.google.com [209.85.217.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM4Sw2RtQz46CK; Sat, 20 Apr 2024 08:30:08 +0000 (UTC) (envelope-from 6yearold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vs1-f50.google.com with SMTP id ada2fe7eead31-479ccc89792so1355202137.0; Sat, 20 Apr 2024 01:30:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713601807; x=1714206607; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=53Sri8KmOduU2fmWIjY+RwNKi/9jP9Qadp1VF5Qita8=; b=HdRqbzV6rsD5Vq20KHvsTkhNkxWwIXsUJOG7/TdHqo3oK2xi1pzWXdbPAO6Sfn/PTD HWNnRqqMD+zNAbOo8TlyudSsMAWPeVMo7SLcAKQ1dIrOCZ8tEyiVEMOPTr8R/qH8rSru nN+ykCbdTB/EpoGweG686Qvj+Dop3ueNzZA3UqX0p0X4X3+JxRmvCBrSRynyN0ewgjtd ty+fCpbIINL/nneLTLhsx3TYOGxQINakOjR5ZvxNriFsjlGO6Vnu9oFHwexeW79M2wUw 0h3Z9B5bLGCw9LWH5LK6dqyPKaZahHmCnLyHa8ELRfy/mg04FJvmj7/qgkrMnzovv68R 4wYA== X-Gm-Message-State: AOJu0YwB0t0NUBakB3zVDZtTGehrCTCIu5m/sjTmVubF+kfNsuumpMaB rwfqDIsr+5H3/fAv1J8AIOvsiQ6SCQkbA+4VdnUrRwtueAkMpYeGXQjlmoSuYiA= X-Google-Smtp-Source: AGHT+IFY1vwzCchB8MbP43j/yAzvE0XtAHD6fp6n3pp8qnLhFPdVYB56O/C5zp4ktcGC0EpWGQEnZQ== X-Received: by 2002:a67:e441:0:b0:47a:62c9:22e3 with SMTP id n1-20020a67e441000000b0047a62c922e3mr11441002vsm.1.1713601807168; Sat, 20 Apr 2024 01:30:07 -0700 (PDT) Received: from mail-ua1-f49.google.com (mail-ua1-f49.google.com. [209.85.222.49]) by smtp.gmail.com with ESMTPSA id w20-20020ab07294000000b007ec332c469asm548785uao.14.2024.04.20.01.30.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 20 Apr 2024 01:30:07 -0700 (PDT) Received: by mail-ua1-f49.google.com with SMTP id a1e0cc1a2514c-7ec48a0306bso931805241.0; Sat, 20 Apr 2024 01:30:07 -0700 (PDT) X-Received: by 2002:a05:6102:4b0f:b0:47b:d493:5561 with SMTP id ia15-20020a0561024b0f00b0047bd4935561mr5853215vsb.11.1713601806852; Sat, 20 Apr 2024 01:30:06 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> In-Reply-To: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> From: Gleb Popov Date: Sat, 20 Apr 2024 11:29:38 +0300 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: What does this error mean: No space available for static Thread Local Storage ? To: Yuri Cc: Freebsd hackers list Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VM4Sw2RtQz46CK On Sat, Apr 20, 2024 at 6:01=E2=80=AFAM Yuri wrote: > > Hi, > > > The shared library that is built by the Rust's toolchain for the port mis= c/py-polars fails to be loaded: > > No space available for static Thread Local Storage > > > What does this mean, and what might be wrong? > This message probably comes from the library code. I'd start looking there. From nobody Sat Apr 20 08:37:07 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM59G476qz5HMbc for ; Sat, 20 Apr 2024 09:01:38 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4VM59F6yZsz4C9T for ; Sat, 20 Apr 2024 09:01:37 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none); spf=softfail (mx1.freebsd.org: 198.144.192.42 is neither permitted nor denied by domain of yuri@FreeBSD.org) smtp.mailfrom=yuri@FreeBSD.org Received: from [192.168.5.3] (c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id 43K8bAwM041394 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sat, 20 Apr 2024 01:37:10 -0700 (PDT) (envelope-from yuri@FreeBSD.org) X-Authentication-Warning: shell1.rawbw.com: Host c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116] claimed to be [192.168.5.3] Content-Type: multipart/alternative; boundary="------------jNgiaOgM8j9BEgB0B4Eeb7mA" Message-ID: <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> Date: Sat, 20 Apr 2024 01:37:07 -0700 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: What does this error mean: No space available for static Thread Local Storage ? Content-Language: en-US To: freebsd-hackers@freebsd.org References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> From: Yuri In-Reply-To: X-Spamd-Bar: + X-Spamd-Result: default: False [1.73 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.983]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; RCVD_NO_TLS_LAST(0.10)[]; ONCE_RECEIVED(0.10)[]; XM_UA_NO_VERSION(0.01)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; GREYLIST(0.00)[pass,body]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all:c]; ASN(0.00)[asn:7961, ipnet:198.144.192.0/23, country:US]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEFALL_USER(0.00)[yuri]; TO_DOM_EQ_FROM_DOM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_NONE(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_XAW(0.00)[] X-Rspamd-Queue-Id: 4VM59F6yZsz4C9T This is a multi-part message in MIME format. --------------jNgiaOgM8j9BEgB0B4Eeb7mA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 4/20/24 01:29, Gleb Popov wrote: > This message probably comes from the library code. I'd start looking there. No, it comes from the FreeBSD src tree. The problem with this message is that it is cryptic, and doesn't lead the user to a solution. Yuri --------------jNgiaOgM8j9BEgB0B4Eeb7mA Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
On 4/20/24 01:29, Gleb Popov wrote:
This message probably comes from the library code. I'd start looking there.


No, it comes from the FreeBSD src tree.


The problem with this message is that it is cryptic, and doesn't lead the user to a solution.



Yuri


--------------jNgiaOgM8j9BEgB0B4Eeb7mA-- From nobody Sat Apr 20 09:03:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM5CD6QgJz5HMTH for ; Sat, 20 Apr 2024 09:03:20 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM5CD2qCPz4DMJ; Sat, 20 Apr 2024 09:03:20 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-516db2214e6so3727834e87.1; Sat, 20 Apr 2024 02:03:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713603794; x=1714208594; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ygLUU0Tu0y/gE0gkdmj2bgp/Qe22D+6FLCZG4juURxY=; b=UGlHIyyqZd7NB20GqAM1xbhJOrb1oKxkl83i5X4KnBKMzC/wHi1Sh20VyM55ZUUhHq Mpp95Op3FXXmT7pAuE4W6S7lUujrjVb6XLsAUg2z+Bsy+u9NN7nwrEjRlIE59wKYjLIz KV/wOh8CSwin5Cbxd9N2FDhZSolRtp0ehIL2pslAr6uk8wwdvsbBer674fQNo+ntMxRJ CgABr6W+NCMYqivOJN+oR68Z3UVj9O52UxlEDvNejd5XFmOO65P2/0eESiL23bAuaU3n MRQhVhRRgiaVYU/i45oClVVzK5gyMV4t+ycFQ0EAjhwx0hwkfIvIoDou9yJy+B0VD4sb P8BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713603794; x=1714208594; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ygLUU0Tu0y/gE0gkdmj2bgp/Qe22D+6FLCZG4juURxY=; b=qc7B9+zPnOlgDtBQTNF2jEx6UY3Fd2aHGb1ibJ7Gao7+SvMVy5imMTtbuWisDYZqpD UhRWgkqKKhmiQ95lnrixZYwNy/QDdFTfQck+Zz/5mxdonEIo36+fR1p/kXaX6Wx2Ryf0 xtSN389orqIT7LV7rCU/mpj/rprxbc237HTFq6ekITYMKQ7TA7/dSOXijL0Hh1mRberN 1ZsJji5WSqVJnyNu8bJToGCgcNvK2FYctrlNDRM9k8GjZGySnoPXUD1lh8/MCRYJX9bv c5fkoCNQEkqbyAxX9Id2HnGFYX7F38hpj+Kr7iTawFSvUdyQ2lTD2OBBJvyvh06muhVu 1u3g== X-Gm-Message-State: AOJu0Yz6bBEu96gd3hMGv5+/LeB461lWQ5P5fjJv7Cz576ft2VgXcGZl RkyB3sMpSGmeh14cwFZAr+MHbnzLx6//JEXIKt/o7OueyQkZkLLETE/4pfzAtlqjpYA1OcqVZSD VJv3bYV73dy2G6xQSaV+TtuCjqSywCw== X-Google-Smtp-Source: AGHT+IEltz4J418qnEL9lH1S7kq4NXP25fq+gTM1WfYBoOw99oVt+Dzu9VpfXwQ3LcCcGrXerpmpt2+MewzVU28nDWM= X-Received: by 2002:ac2:46e9:0:b0:515:d135:68f2 with SMTP id q9-20020ac246e9000000b00515d13568f2mr2579185lfo.53.1713603794053; Sat, 20 Apr 2024 02:03:14 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> In-Reply-To: <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= Date: Sat, 20 Apr 2024 11:03:01 +0200 Message-ID: Subject: Re: What does this error mean: No space available for static Thread Local Storage ? To: Yuri Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000006e346c0616837907" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VM5CD2qCPz4DMJ --0000000000006e346c0616837907 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable El s=C3=A1b, 20 abr 2024, 11:01, Yuri escribi=C3=B3: > On 4/20/24 01:29, Gleb Popov wrote: > > This message probably comes from the library code. I'd start looking ther= e. > > > No, it comes from the FreeBSD src tree. > > > The problem with this message is that it is cryptic, and doesn't lead the > user to a solution. > Might it be incompatible build options? https://forums.freebsd.org/threads/apache24-php7-1-got-no-space-available-f= or-static-thread-local-storage-error-when-start.63531/ > > Yuri > > > --0000000000006e346c0616837907 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


El s=C3=A1b, 20 abr 2024, 11:01, Yuri <yuri@freebsd.org> escribi=C3=B3:
=20 =20 =20
On 4/20/24 01:29, Gleb Popov wrote:
This message probably comes from the library code. I'd start=
 looking there.


No, it comes from the FreeBSD src tree.


The problem with this message is that it is cryptic, and doesn't lead the user to a solution.

Might it be incompatible build options?



<= div>



Yuri


--0000000000006e346c0616837907-- From nobody Sat Apr 20 09:10:10 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM5MV18hRz5HNHZ for ; Sat, 20 Apr 2024 09:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM5MV0cc1z4FQN; Sat, 20 Apr 2024 09:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713604230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JgHpA8EGCsOur4D6Pb9TvPz9cGQlBO7x81zhqLm9O6I=; b=TRs2yeRp5coOrOQobCQG72AqsSIhufUb/dDBCfuiodFllhF22p6Rm2SPMXGVtxCufqA95q EeEM6TVPI+PejrePhyC5O8i7chZ9EUGvXt57cBunSUBlQz/4nd4AY6LisetRPfC0LS7IJg 2njUVy/yGxxciRB0LGUtsm64buksONnJieeRv01qccbzQS/fuuhFxi3E46JeYAaVhLkoY5 sjSj8tmCRGuViG1cfhkk3d9Xkb0CXIzbi5LslE1dzV99voICoglQZMRkeZhkqGqlMs3dHt Bdngv3e8B96bzX0bMvNIahgmY4Gf07mbIvm3BM0G9Dmbp5RR0tQbbA6OOg3CUQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713604230; a=rsa-sha256; cv=none; b=pOoCnv0v7u55QBvzv5pU8ToMIFRCuof4W5uCAiJydF6Jn7s6EUGnOJ2RqGCf0tlFdR2edg jagzwcxqj7rC1kVg05AvNPonZbD+Tbnx1WacEyth8CpoyC80HP4sO4KkRRR/0Ub/GbEyae NZsgZNfG+zEJxHxgYBZhvWikAqJHXB9QZjdMQeCqoEUZM9t66EFWB9lCEnsNGY7BwM2U3n s75P+uamSGop6o3BVi0f1HHQXfuV7l0JWMTAoVZm/35gLCqm0TBKVFrFyAt2imfxt9gRKh XkNjpmPGJsfndJcNyF5HkbXkdJh9VEjkHBjv2EKq7fWvBM0U1PJfvwDeia7ugQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713604230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JgHpA8EGCsOur4D6Pb9TvPz9cGQlBO7x81zhqLm9O6I=; b=XOLGOgCUAWx+BPqRHyaA6fN6zilDI6w5yl+naofDbX/PP+RyXDhZdOT5hdNEJHX/lz1H9o kpxtnkeNNjMAIpAVu4PKEdc7OmYSSxN5InrEKzOY/2ain4wmtVtbTFP8vc/P14oP9w+Zre D1V3oRkYvWDNCBCoyK8ujdN+XLlFyQQnnZ4QUthm92dN0SeDTvt93DN4Ywkyyu0t8WB7SR f4F65TaJxOQycha0AOuHS4+fv7qvCT5Ux4p/+Rvh9yWeaQ9n8rkzO7qz+Eu2CXFvmL0Dvi +RRACq4aUnIYxoiSZxRHR625xXAnRgDp6ISIws5YBwnAUnKLj2xrfnLS15BB+Q== Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VM5MT6QNnz1Fxp; Sat, 20 Apr 2024 09:10:29 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 4C94562191; Sat, 20 Apr 2024 11:10:23 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: What does this error mean: No space available for static Thread Local Storage ? From: Juraj Lutter In-Reply-To: <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> Date: Sat, 20 Apr 2024 11:10:10 +0200 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> To: Yuri X-Mailer: Apple Mail (2.3774.500.171.1.1) > On 20 Apr 2024, at 10:37, Yuri wrote: >=20 > On 4/20/24 01:29, Gleb Popov wrote: >> This message probably comes from the library code. I'd start looking = there. >>=20 >=20 > No, it comes from the FreeBSD src tree. >=20 > The problem with this message is that it is cryptic, and doesn't lead = the user to a solution. When we=E2=80=99re at rust and memory (thread-local storage?) related: = I=E2=80=99ve tried to run databases/qdrant in production. It works until it=E2=80=99s idle. When some collections = are being created/loaded, it crashes. I have been able to track it down to malloc() called from within = strdup() called from within thr_set_name(). That internally calls, something like: frame #6: 0x00003f6e7cbc8955 = libc.so.7`__je_tsd_fetch_slow(tsd=3D0x00003f6faeb53090, = minimal=3D) at jemalloc_tsd.c:0 frame #7: 0x00003f6e7cb77e3a libc.so.7`__je_malloc_default [inlined] = tsd_fetch_impl(init=3Dtrue, minimal=3Dfalse) at tsd.h:355:10 frame #8: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default [inlined] = tsd_fetch at tsd.h:381:9 frame #9: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default [inlined] = imalloc(sopts=3D, dopts=3D) at = jemalloc_jemalloc.c:2256:15 frame #10: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default(size=3D40)= at jemalloc_jemalloc.c:2293:2 I can reliably reproduce the problem on both stable/13, stable/14 and = main. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sat Apr 20 11:57:19 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM94G1jd8z5Hf5c for ; Sat, 20 Apr 2024 11:57:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM94F3GjRz44GR; Sat, 20 Apr 2024 11:57:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 43KBvJPp014110; Sat, 20 Apr 2024 14:57:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 43KBvJPp014110 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 43KBvJLU014109; Sat, 20 Apr 2024 14:57:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Apr 2024 14:57:19 +0300 From: Konstantin Belousov To: Yuri Cc: Freebsd hackers list Subject: Re: What does this error mean: No space available for static Thread Local Storage ? Message-ID: References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4VM94F3GjRz44GR On Fri, Apr 19, 2024 at 07:30:48PM -0700, Yuri wrote: > Hi, > > > The shared library that is built by the Rust's toolchain for the port > misc/py-polars fails to be loaded: > > No space available for static Thread Local Storage > > > What does this mean, and what might be wrong? The error mean that the process tries to load a dso (shared library) that was build with initial-exec TLS model, and there is not enough space in the initial TLS segment reserved by rtld to accomodate the dso needs. As a temporary measure, you might try to play with the LD_STATIC_TLS_EXTRA env variable to specify the desired size. Default value is 128 bytes. For gcc and clang, the TLS model is controlled by -ftls-model switch. No idea how to pass this through rustc. One possible reason why you get the initial-exec model is when dso code was compiled in non-pic mode. From nobody Sat Apr 20 12:01:13 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM98f4X59z5HfQQ for ; Sat, 20 Apr 2024 12:01:22 +0000 (UTC) (envelope-from schakrabarti@microsoft.com) Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on20700.outbound.protection.outlook.com [IPv6:2a01:111:f403:2011::700]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM98d4csbz45Cm; Sat, 20 Apr 2024 12:01:21 +0000 (UTC) (envelope-from schakrabarti@microsoft.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=microsoft.com header.s=selector2 header.b=En3yxR8u; dmarc=pass (policy=reject) header.from=microsoft.com; spf=pass (mx1.freebsd.org: domain of schakrabarti@microsoft.com designates 2a01:111:f403:2011::700 as permitted sender) smtp.mailfrom=schakrabarti@microsoft.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d+IVKTtQHkvFghKrG+u+IL7FzP0jQ7VpNAxrZa8WH09Gj/ym1xC4wG29QvkgPM4CdRTL+wCiX3pM0AHmGJUd+87H03WM+htntBpZf4DrcBOcfBAspS2h6yeokcbTIGqnQhEhijttk7wgftv15bhB6R3Qeh78+72uJI1f+sbl7NbUBbEs2Cu5kW8J0NO0BiUHU16HAuEOehrOq5aC3IpjwJbEEZDCk3Nhy8Q2YbDf4lIFTqU2nCZlUlGMkd95XUazuOFVVrx0bjE1WfFo2GVc+amgUeb+kC57jBTnDZdC5Dm+MFQS7nP4/qP3r7GofPFKTt6X3Kw/4iKGPzBKyJGgSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1I3vQnx/iXqR165USzmD+RuLNm6e5qpc4IzhjFF780E=; b=JVhH8N9V/LwHWL27Bb13Mm5x3pM7cOxdSF6n9tLNKdBbSTHY7Y40Rc/vixanyZTWz2jlB5HL2pmSvzFezESTIYk/Nf6CSJt5akaKo4gO2zbWdqvxtX63B7OVXjGMDPrIvM3qY5a5ziQS7VLf5E6WOiGSilvcvQ5tZWMTAhjIem09ssF5BcROqfg2NE89SknzldS4DdkgezQjMkof4HtuNX4S+HlkcKbipUsT47cFSOlquM7AymDm19HSb8ItmhqSE+TVi7dy2ZLxFwcTz35JloEQsotSF1QVLZrJD+90eUYGpKcyrpgynAAHDaA+RGdc6vhOm9X7jbvH2RlEpPGokw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1I3vQnx/iXqR165USzmD+RuLNm6e5qpc4IzhjFF780E=; b=En3yxR8u+NWWvFHQ9RRg9PAazvG+ldaS2NxRiUHE4nWW0vAPoXPLqQgZOC9wRY4PXQ2jzI5G5OdWlZW/YhWHxRDQ0RKUxFzQ8GnYK4ZzG3N1hSN7oiQxhWuIGCkJyH+wCTmyIoyUi7wdBIPqxXZ5znfchDgX1bWhq4ByyesFveA= Received: from TYZP153MB0397.APCP153.PROD.OUTLOOK.COM (2603:1096:400:20::13) by TYSP153MB1013.APCP153.PROD.OUTLOOK.COM (2603:1096:400:468::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.18; Sat, 20 Apr 2024 12:01:14 +0000 Received: from TYZP153MB0397.APCP153.PROD.OUTLOOK.COM ([fe80::1b9b:e2ca:3a9c:5698]) by TYZP153MB0397.APCP153.PROD.OUTLOOK.COM ([fe80::1b9b:e2ca:3a9c:5698%4]) with mapi id 15.20.7519.014; Sat, 20 Apr 2024 12:01:13 +0000 From: Souradeep Chakrabarti To: =?utf-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= , Wei Hu CC: Konstantin Belousov , Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: RE: [EXTERNAL] Re: How to add a -W flag in local Makefile Thread-Topic: [EXTERNAL] Re: How to add a -W flag in local Makefile Thread-Index: AQHakhD0rYhqaaTE2Ey/8jO+Da81B7FvbPIpgAGhpsA= Date: Sat, 20 Apr 2024 12:01:13 +0000 Message-ID: References: <86cyqlbovo.fsf@ltc.des.dev> In-Reply-To: <86cyqlbovo.fsf@ltc.des.dev> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=2b0279df-db76-44c5-8788-de445bd7a1cd;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2024-04-20T11:53:30Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYZP153MB0397:EE_|TYSP153MB1013:EE_ x-ms-office365-filtering-correlation-id: c4853cb2-80f8-494c-24f2-08dc61319369 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230031|1800799015|376005|366007|38070700009; x-microsoft-antispam-message-info: =?utf-8?B?M0dxSkc3NHY3eFBYb0Z1UFZ0dFJCcEM5ZVAwR21Qd2I3SkNDK0JIbHQ3N3JE?= =?utf-8?B?clNSU1JBckdkVkFVdS9qS2pseDlMV0h4Sno3cHVsaUpwZ0dJdWFucWFrTzBR?= =?utf-8?B?REVmU05vK3pBWXBOOHJyUjlpUDF1OTM4UCtDaUUzZnZCV056ZDFRTEgra3FP?= =?utf-8?B?VXNTRTF0d1E2dWc5NDJpK2grMHJOYXljQkV6emZtdkJoZ1paVHFnK1BYYWRw?= =?utf-8?B?WTVHRHZIWHdTbzJFVVc2aHlEMHI1TWt5enBoUzZUYVBrbk40TFlRejYzSVV3?= =?utf-8?B?dEhOTXFDbXkyNnYrODRuMWdXTlN5RzlwNGNKbTYyY1RVV08ydDh1Q1h5QzZQ?= =?utf-8?B?b3pvNWQxamxTaXN5cVdmMlhRQ2ZiN3o4VVhaVEwxNmovWTZWYmdCL2RaSVZp?= =?utf-8?B?WkQwSCtnbWxFaTdSclQvUkNiN3JEVjZmUHdXNnhJWThxVGJVYUFlaGM1czFY?= =?utf-8?B?UmkyU0pCVDI4b25sMUNGNFZ4VGhZN1ZEZ3lka0F5R013QXVPalArYXBobkNH?= =?utf-8?B?R0NmV0xrTTZ1WGJxMXRLdXg1MVZkNVl3T3Z2TjlUK3BsY283NXN1SkFmYzE3?= =?utf-8?B?emN5TDJWbWI4NWdTdVRPb3dpWXFtVVY4bDQrRmdkaHVTQSt2dWV0Q1FSZDJP?= =?utf-8?B?Vlp2WnRwQkJQTTI1RzRSelFWei83dlFWL2tuRzQzS3VENnppZnUxT3lmY0dv?= =?utf-8?B?aWFJcmI4WkUyS1EwN1FBZlRWWjFHVzJvR29UVkthbXVPSGhEZzVITUk1eW16?= =?utf-8?B?dDVhQVJQckFLV05OSlZXcFhvalcway9DbklIRHlVNzJhZFI3ZzYzK3F2c2V5?= =?utf-8?B?L0IrQW90NGVWeThJZHZicmMrdGhSTWgxdTZNaVR3VnUySURYbndrOUJqR3ZK?= =?utf-8?B?Rk5Vc1lVNUhjc2hjUlo0K2VDOXIwNmZNWXdSemMwSExFUk93Tmo0bGdWcWx5?= =?utf-8?B?STlIanh5YVZGU2tQVVRLdXdrMWtJU1NrS2g1SlZYTXROTnREbllpMnRYUGZH?= =?utf-8?B?UEZYQlJMZnFZMFVXUytnTEVjZ1g2QUZTdkl5UWt2WGRpYnAxTFpsWnlZSGU5?= =?utf-8?B?UDI5NWlNaWRlZmZwV1h3UWpWUU9wQmlET3V0MUhhSGNuV0prZkdteEV5RVFR?= =?utf-8?B?QjlubEJLVE1mU1pGWTBZMFV2MGlXWTRBQUN4elZOaytIUEJ0YVNYQmsvd2Yr?= =?utf-8?B?T0lpbW1VeTc0Y2gveDRsWDJ4dEM2QWRiZ0RsWmI1OFRSTzB5SzJTU1FNWitH?= =?utf-8?B?Nm9VNklvMTNudTBBT3dSWmw2Q05mTm9zdXJYajFlcEQwakVkcmZUcy9tSGhQ?= =?utf-8?B?UDZva2xpemIwQWp3aDNCOXB0ZVF3ZlNOZVphSkY3RDRNWGdYYzlLWlhpUFFL?= =?utf-8?B?R3l3bExRejBGQkU5Ni84UW8zRnhmYklBcmh6SXVWVUZjVUhKNldYcEgzaFlt?= =?utf-8?B?TEpjY2xlQUlxTlVIMFhXYy81bzhDeWFuTGV0WWpGVDFCVEt2S211bmlKUG1R?= =?utf-8?B?T0VNdDg0empLMXNXMEJqQm45ZHBsUGFqb01sZDV3Z3JiT3p5OGJoazdCUUht?= =?utf-8?B?UFJCdFNwVjBMSzBKSW9VdHd6SGtSa2tsS2orc1UyTm9SNDBGQ2g3cHp0OE94?= =?utf-8?B?SDhGbVJtR3h6RFlabTlOaW1JZjFOVWJmTUlielVvMHhLVTVoWmsvVUt4bDEx?= =?utf-8?B?R1M3R0t4SDlyRVZkSXJkU0VHbnRvaUpvMXoxMTh2SC9jdU12Si9OM2k4bEY3?= =?utf-8?B?VlRPbmhaSldqbGVoQmg3dTdCeitoZVB5VmxDd2FLcUQ1L09qUFdGcVczVGtD?= =?utf-8?B?TXYyOUZyZ2s1SlZLRk95UT09?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYZP153MB0397.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(376005)(366007)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?TDdHQlhoR1dnZDcwYTdEUEl3WVZpZzlnVkJGQ3BEUk1TL3l6R3pONjdRd1FK?= =?utf-8?B?c294dW04Sis2WFR0cENxeEV5MTgxTVFaNmlzRDVlVlFtWVpuYVRjMnFzbFJV?= =?utf-8?B?L3FPK0dmNXcyWDN0cXlxeEFLMlNPQmJiT1lESmJzajlCQmpFM3kxMzhqVGVx?= =?utf-8?B?aWUwL29WQkF0bTVKdURXWE5TeEZrZzl0clNPZmlMNG56cERNYkdEVTNyR25Y?= =?utf-8?B?Q0JXbStUSzFNNlNNUGRJa25wTFl1TXhxcGw3Y3VvbVF4dWZTK2plTGZYSjIw?= =?utf-8?B?cHBhdklpeTlRcVRubEw4TDFLT01wQm1zZEl1dE9TYVQwcWF1clNwRUpLRTJX?= =?utf-8?B?TzZUc3FlR2FQZFZPTFFPQ2I3Ymx6Q3NMTzViT21qZGRNSjdrZXA1N0VCTWZ2?= =?utf-8?B?RjFLblVWNEgrTGlYTjN1c1ZsbXpabzhueHpxVW1ZMjJvYS9LZlJrdGowUWQy?= =?utf-8?B?RXJqOW51QkZwRmsvRnVNT0MyelJnREVpdU5LK2sxUmNRdC8zV2l2TDF5UGJu?= =?utf-8?B?VzRzRlZWbzVXZUJEL3g4TDQ2dGwzQ2czd1grQ0ZPQ2FRNnBTRW4rNjc4WTVp?= =?utf-8?B?ajBUMFg0Yk1kTE9ibGsvUFYvVVNyT013L1JHbGZuREpTcGlUOHJYKzhqVWlv?= =?utf-8?B?aFF1ZVBkTS9YeEM3T1IrYkhiR2tSeC83VTB4SzZzWEpYQkgwSUh6RUFjTmNK?= =?utf-8?B?a0VZSkxFSzNQU2VXWm9IdUZpNGVCRjlTTVhwaUN1RkFzSzdyL0ZKNzY0eE1N?= =?utf-8?B?TEFWRGxVV2tIRGxBbEVWR0JyeExXTlVFUUFJSjFMM1ZoMXZIeGdCNGRNYXds?= =?utf-8?B?bkhoMC9nQmlZSlg0RFZBeE8vWEp1M21oMmdEK1gzY2NaN1c3eGRMWGFGR3pr?= =?utf-8?B?dzB2ekNMRi9RMm9ZdmFnM3RGS2wxVVJFeVQ5WDZ4QWo1ZWswTHU4WStWWk90?= =?utf-8?B?eHUvc1ZQRVJhZEVja05jTXMwa3pZd1JRTEdmMXd2YUVvOUVkbVhyanBOZkw0?= =?utf-8?B?SnZBT0hoWCtDOXltTkpDMklmdlJJbEpCSWRDdmpWQ0d1QzZOeXUrYm1haUpK?= =?utf-8?B?aGlrVk5mbkdMamVLWjlJdVJ5ZjFPZVVQQ1RPU3NpZ1VjRDBLczZoNkNqTDRZ?= =?utf-8?B?bCt2Sm04bGRSN0o0UVFLWk11ZmVwYVNDM0oyNzBvYWFZeHlkMHpVSEFTQXY4?= =?utf-8?B?aXZXUnJTRk5OdGt1Z2hTN09wVEtBd2RLUWZDcWFLTFBjbUZJVC9UUWhwZ25a?= =?utf-8?B?bUNpbzVVZW5seXU2Y0pCdllBS3FLVm54aE5sM1pIMklkQjhLY3hOaitiK0RJ?= =?utf-8?B?amdPWm8ybU1WaVJMTG9YTDg0SlorWnYvUGF1eld0VlZrQ1dqc1FyVUNwOWwz?= =?utf-8?B?aEZ5dkh1Nkd4TmNHY3NOdzFhZGc0bWY1UFk2ZHZheDlZbGJLSnVrYi9ZTU54?= =?utf-8?B?MWx1ZGdhckNVTTVzclFuRkdUZzM5eUpjZXN1SXhVVFRsaHRRS3dUTnJvRUhL?= =?utf-8?B?NXN3WlBGMCt4Ujc4ek0wRmNCZzFUWTZ5eWRLa25wUldQdzU0UTZpQVQ2SWw2?= =?utf-8?B?TXoyTzBQTzZKVi9JU3Z4RkxLaXViSHZDMHlFZ0NneXFVVkRNZnF5NlhqSG1m?= =?utf-8?B?azlKNENKWFd6blM5Wlh4S0lySVJ1ckpYYlhWcjVZRVNRYzMvajBpTFZjTUFm?= =?utf-8?B?NHJNZnNkaWZxR21VdWJFVEdUV3BQREtqcnpOeWVWWk1lYjZOWnliTndFY2VS?= =?utf-8?B?aWZlS1NRZ3gzcU1pL2YreU14RzM1TEpxSER2SzJZRno3VlhqbkVhekJoZmpq?= =?utf-8?B?THhHbEFHTkNoYmg4RzB6MTY2SjNOa0RoYzRFbXE0ZXY3QWtBZFZHeEVmbStO?= =?utf-8?B?aFFTT2FxMGVseWN4MGFxRHdPY01OMWdnaklGejgzVzFGTm9VcytnRnJMRWJY?= =?utf-8?B?ay9IWjJ6a01wTENPRmJFZ2VFRjVYQlh0ek5sNkhCbDR3cEJmZStrZnFVQjIr?= =?utf-8?B?RGk2K0NBVTg5THh1UXVGaC9KYTlZYVFnVzVpMytxMlZxbTh5TFBubkgvR0Ni?= =?utf-8?Q?c5N3PI?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYZP153MB0397.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: c4853cb2-80f8-494c-24f2-08dc61319369 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Apr 2024 12:01:13.3943 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zGm25m4OC+ExTUJN0HUlOWe+v14aTmlefVNovFF8gdFzRdJXot3nLE6M8BdRG4DCrA8lcRGCbTJ91lKXpfQaADL2jFpD+x/nnBoYxaJkXpE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYSP153MB1013 X-Spamd-Bar: -------- X-Spamd-Result: default: False [-8.90 / 15.00]; WHITELIST_SPF_DKIM(-3.00)[microsoft.com:d:+,microsoft.com:s:+]; DWL_DNSWL_LOW(-1.00)[microsoft.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[microsoft.com,reject]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f403::/49]; R_DKIM_ALLOW(-0.20)[microsoft.com:s=selector2]; MIME_BASE64_TEXT(0.10)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,bsdimp.com,freebsd.org]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[microsoft.com:+] X-Rspamd-Queue-Id: 4VM98d4csbz45Cm DQoNCj4tLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPkZyb206IG93bmVyLWZyZWVic2QtaGFj a2Vyc0BGcmVlQlNELm9yZyA8b3duZXItZnJlZWJzZC0NCj5oYWNrZXJzQEZyZWVCU0Qub3JnPiBP biBCZWhhbGYgT2YgRGFnLUVybGluZyBTbcO4cmdyYXYNCj5TZW50OiBGcmlkYXksIEFwcmlsIDE5 LCAyMDI0IDQ6MjggUE0NCj5UbzogV2VpIEh1IDx3ZWhAbWljcm9zb2Z0LmNvbT4NCj5DYzogS29u c3RhbnRpbiBCZWxvdXNvdiA8a29zdGlrYmVsQGdtYWlsLmNvbT47IFdhcm5lciBMb3NoDQo+PGlt cEBic2RpbXAuY29tPjsgZnJlZWJzZC1oYWNrZXJzQEZyZWVCU0Qub3JnDQo+U3ViamVjdDogW0VY VEVSTkFMXSBSZTogSG93IHRvIGFkZCBhIC1XIGZsYWcgaW4gbG9jYWwgTWFrZWZpbGUNCj4NCj5X ZWkgSHUgPHdlaEBtaWNyb3NvZnQuY29tPiB3cml0ZXM6DQo+PiBUaGUgc2FtZSBjb2RlIGFscmVh ZHkgZXhpc3RzIGluIExpbnV4IGFuZCBXaW5kb3dzLiBMaW51eCBhbHNvIGFkZGVkIGENCj4+IGNv bXBpbGluZyBmbGFnIHRvIHN1cHByZXNzIHNpbWlsYXIgd2FybmluZ3MuDQo+DQo+VGhhdCdzIG5v dCBhbiBleGN1c2UuICBUaGUgY29kZSBtYWtlcyBubyBzZW5zZS4NClRoZSB3YXkgaXQgd29ya3Mg aXMgYXMgZm9sbG93czoNCg0KPiA+ICtzdHJ1Y3QgaHZfdnBzZXQgew0KPiA+ICsgICAgICAgdWlu dDY0X3QgZm9ybWF0Ow0KPiA+ICsgICAgICAgdWludDY0X3QgdmFsaWRfYmFua19tYXNrOw0KPiA+ ICsgICAgICAgdWludDY0X3QgYmFua19jb250ZW50c1tdOw0KPiA+ICt9IF9fcGFja2VkOw0KPiA+ ICsNCj4gPiArc3RydWN0IGh2X3RsYl9mbHVzaF9leCB7DQo+ID4gKyAgICAgICB1aW50NjRfdCBh ZGRyZXNzX3NwYWNlOw0KPiA+ICsgICAgICAgdWludDY0X3QgZmxhZ3M7DQo+ID4gKyAgICAgICBz dHJ1Y3QgaHZfdnBzZXQgaHZfdnBfc2V0Ow0KPiA+ICsgICAgICAgdWludDY0X3QgZ3ZhX2xpc3Rb XTsNCj4gPiArfSBfX3BhY2tlZDsNCg0KSGVyZSBhbiBwb2ludGVyIGlzIGFsbG9jYXRlZCBmb3Ig c3RydWN0IGh2X3RsYl9mbHVzaF9leCB3aXRoIDRLIHNpemUuDQpOb3cgZ3ZhX2xpc3QgYW5kIGJh bmtfY29udGVudHMgYXJlIHNoYXJpbmcgc2FtZSBzcGFjZS4NCkJ1dCBndmFfbGlzdCBnZXRzIGZp bGxlZCB1cCBmcm9tIHRoZSBwYXJ0aWN1bGFyIGluZGV4IHBvc2l0aW9uLCB3aGVyZSBiYW5rX2Nv bnRlbnRzDQppcyBlbmRpbmcuIA0KVG90YWwgc3BhY2UgdXNlZCBieSBiYW5rX2NvbnRlbnRzWzAt KG4tMSldICsgZ3ZhX2xpc3Rbbi1lbmRfb2ZfdGhlIGhlYXAgYWxsb2NhdGVkXSA9IDRLIC0gKHNp emVvZihmb3JtYXQpICsgc2l6ZW9mKHZhbGlkX2JhbmtfbWFzaykgKyBzaXplb2YoYWRkcmVzc19z cGFjZSkgKyBzaXplb2YoZmxhZ3MpICkNCkR1cmluZyBhIGNhbGwgdG8gaHlwZXJ2aXNvciB3ZSBu ZWVkIHRvIHNlbmQgaW4gdGhpcyBtZXNzYWdlIGZvcm1hdCBmb3IgcHJvcGVyIGRlc2VyaWFsaXph dGlvbiBhbmQgdGhlcmUgd2UgYWxzbyBzcGVjaWZ5IHRoZSBiYW5rX2NvbnRlbnRzIGluZGV4IHRo YXQgaXMgbi0xLg0KVGhhbmtzIA0KU291cmFkZWVwDQo+DQo+PiBBbnl3YXksIHRoZSBwdXJwb3Nl IG9mIHRoaXMgZW1haWwgaXMgdG8gdW5kZXJzdGFuZCBob3cgdG8gYWRkIHN1Y2gNCj4+IGZsYWdz IGluIGxvY2FsIE1ha2VmaWxlcyBhbmQgbWFrZSBpdCBlZmZlY3RpdmUgZm9yIGdsb2JhbCBidWls ZGtlcm5lbC4NCj4+IEFkZGluZyBzdWNoIGZsYWdzIGluIGxvY2FsIE1ha2VmaWxlcyBhbHJlYWR5 IHByb3ZlcyB0byBiZSB3b3JraW5nIHdoZW4NCj4+IG9ubHkgYnVpbGRpbmcgdW5kZXIgbG9jYWwg ZGlyZWN0b3J5Lg0KPg0KPllvdSB3b3VsZCBoYXZlIHRvIGFkZCBpdCB0byBldmVyeSBsaW5lIGlu IHN5cy9jb25mL2ZpbGVzKiB0aGF0IHJlZmVyZW5jZXMgYSBDIGZpbGUNCj53aGljaCBpbmNsdWRl cyB0aGlzIGhlYWRlci4NCj4NCj5ERVMNCj4tLQ0KPkRhZy1FcmxpbmcgU23DuHJncmF2IC0gZGVz QEZyZWVCU0Qub3JnDQoNCg== From nobody Sat Apr 20 12:11:38 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM9PG3Nsbz5HgHG; Sat, 20 Apr 2024 12:12:18 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM9PF1FPcz4CMT; Sat, 20 Apr 2024 12:12:17 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=DGSHM6R4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2607:f8b0:4864:20::1030 as permitted sender) smtp.mailfrom=marietto2008@gmail.com Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2a559928f46so1954429a91.0; Sat, 20 Apr 2024 05:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713615135; x=1714219935; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=G3XSPvuso1yxRpxP0fxpRYkkuDl1di2B3JRSykDUA44=; b=DGSHM6R4S8dVgV8U/g63uSC5S8f9Zbov1CFdyUmJy4YeNXV8jI119yg/PF63OJWep9 7mKHnVc0uF7I1aEGTeB5WZhe5tGEDlGlgwkPg/U5y8Yy21xwoxmE5Xs48SVl+2wTOFry /5TQjPl0umpIwASRhkbm6Rsp0CNCkNifmWH1oMVl+PCjARcD9DYpS2udVU6pqOI0M2XL x/Ds1TyPpwvSgfV5BHjYP1eDjkZ/2OgYr8gqxTpuXnAAHqTzLgbXFAqR7u5+B7asSNEb HJrQojnotfGvJ0Nx9GZCIPNeD7ULaU1SD737uJT8og8i/Kn1JJL3QqlFXNq/gQh0erWz Pl9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713615135; x=1714219935; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G3XSPvuso1yxRpxP0fxpRYkkuDl1di2B3JRSykDUA44=; b=OavmmiQgfOvYN51xujqeEbNveEnEiLG6AVjFtvn9Y+t2K90+txaviGc9cgaC+nGmPM nm3xhOLzy11VBvnCsbDXEU+4R5ZP1vRmpr4f4kyEB+b0N9K8fQ2tJMan0ac+FV84fdIm Tko7RLUoMHMeR9OuAQ56kTZbySsP9sUWLuEqcE+Xlo2cAWK77Cn10c51vJxF+3mY2cdF eGrfza8Ra8LnWwpnovstQLeG2askLLWiWUNAys+zHSf+/viNiMt8uT8fj+w4AW7POf2Y mVkfZTzHjp9KTV2i8rHv0zB9cee9LpWi7N8jgop/PEkNzLvvs18YUFpL7jjK4yS0Y7ia M/hA== X-Forwarded-Encrypted: i=1; AJvYcCUdXIgmCGhNPOMY30tmuron9dBHfsC5quSVbp9nTz+4HNPqgWaVu0MtXqUmvManeP0HLXlTVfzx4OD0yk1vuIdq+DXxd2AEeUGGoNeSgBDtOiRmZL0xpyI+dGjYU18d6auJswqXn9pjinRS X-Gm-Message-State: AOJu0YwXgjKv3IPfzrTfZUvaaU1dqLS+EtK0U+8N+JyuIHwwws3rMLyG ix1L6plL+/qoWvN5nGzKd6SehSrpbRIJg3POgLHMf0CumYcmFDE0HMLEfzgFo9oPeRmD0Ad1PEC ik6fszuL+zqRfO2VT+LBkTgkP5BMEv/HYhYQ= X-Google-Smtp-Source: AGHT+IGzDHRnyPo9Bkgiv6fTvOGPpbhaNHeWYE3nh3tmxr/AeUMuHHvf1rNdu01WWGJmV3jWJNgsvwtvDpPT9KdE7NA= X-Received: by 2002:a17:90a:a085:b0:2a0:4495:1f3d with SMTP id r5-20020a17090aa08500b002a044951f3dmr5641483pjp.0.1713615135010; Sat, 20 Apr 2024 05:12:15 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Sat, 20 Apr 2024 14:11:38 +0200 Message-ID: Subject: Re: Trying to virtualize ReactOS with bhyve : I don't see a viable way to do it... To: FreeBSD virtualization , freebsd-hackers , FreeBSD Mailing List Content-Type: multipart/alternative; boundary="000000000000677a0e0616861d3b" X-Spamd-Bar: - X-Spamd-Result: default: False [-1.62 / 15.00]; URI_COUNT_ODD(1.00)[21]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_SPAM_SHORT(0.38)[0.379]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1030:from]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org,freebsd-hackers@freebsd.org,freebsd-questions@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4VM9PF1FPcz4CMT --000000000000677a0e0616861d3b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable UPDATE : I've been able to unfreeze ReactOS on the boot stage. I've just removed the bhyve "-A" offending parameter that can't be used because ReactOS probably does not support ACPI. Now I'm at this stage : https://ibb.co/Dtw8Hj1 Now the problem is that I can't move the mouse inside the vm and the keyboard does not work...so I need to find some workaround to supply the absence of the arbiter... I've thought about using virtio-input and I've added this parameter to bhyve : -s 12,virtio-input,/dev/input/event7 \ I've chosen event7,because I've got it here : [root@marietto /bhyve]=3D=3D> ls /dev/input/ event0 event1 event2 event3 event4 event5 event6 event7 Anyway I also tried event6,event5...it didn't work. Can someone quickly explain how it works ? Is it supposed to work even if ReactOS does not have the arbiter ? More precisely the ReactOS developer said : "Hardware initialization doesn=E2=80=99t occur on UEFI due to missing arbiter support : It means tha= t the hardware (even essential such as input devices) will not work and you may not be able to interact with the system in any way" My idea is to use Barrier to be able to use the mouse and keyboard inside the vm...I will install the server on freebsd and the client inside the reactos vm... Could it work as long as reactOS has a network connection ? Let me know. On Fri, Apr 19, 2024 at 3:52=E2=80=AFPM Mario Marietto wrote: > Hello. > > I'm trying to boot the 32 bit ISO image of ReactOS using bhyve or qemu an= d > seabios as bootloader. > > As first experiment I tried to boot the x86 version of ReactOS with qemu > (8.2.2) and seabios (version 1.16.1_1),like this : > > qemu-system-x86_64 -machine q35 -m 1G -cdrom > /home/marietto/Desktop/Downloads/OS/ReactOS/reactos-livecd-0.4.15-dev-792= 1-g6d853be-x86-msvc-win-dbg.iso > -boot order=3Dd > > Unfortunately,I get the error below : > > https://ibb.co/tZmFh2x > > but my real goal is to boot ReactOS by using seabios. Even here I'm > unlucky. Infact on this web page : > > https://eradman.com/posts/bhyve-ipxe.html > > we can read : > > *Bhyve is somewhat similar to QEMU-KVM without the emulation required to > boot SeaBIOS.* > > Is really true that I can't use seabios to boot ReactOS with bhyve ? > Anyway,ok. I've continued my searching and I found this thread : > > https://forums.freebsd.org/threads/bhyve-with-bios-boot.79794/ > > where we can read : > > *bhyveload has nothing to do with BIOS. As detailed in the man page you > linked to, it is for loading a FreeBSD guest.* > > So,it seems that I can't even use bhyveload. So,what can I use ? What can > I do ? > > I've gone on the ReactOS forum and I've created a thread where > I 've asked how to boot the 64 bit + UEFI version of ReactOS using bhyve. > The developers helped me,but in the end we have done nothing. Because aft= er > some progress,I found this error : > > https://ibb.co/N9QHdnp > > Unfortunately it is frozen when it tries to load the mountmgr.sys driver. > Actually unfixed and no one knows how to fix it. > > Now I'm here,asking for some further ideas and suggestions. > > -- > Mario. > --=20 Mario. --000000000000677a0e0616861d3b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
UPDATE :

=
I've been able to unfreeze ReactOS on the boot stage. I've just removed the= bhyve "-A" offending parameter that can't be used because ReactOS probably does no= t=20 support ACPI. Now I'm at this stage :

https://ibb.co/Dtw8Hj1=

Now the problem is that I can't move the mouse inside the vm and the=20 keyboard does not work...so I need to find some workaround to supply the absence of the arbiter...

I've thought about using virtio-input and I've added= this parameter to bhyve :

-s 12,virtio-input= ,/dev/input/event7 \

I've chosen event7,becaus= e I've got it here :

[root@marietto /bhyve]=3D= =3D> ls /dev/input/
event0 event1 event2 event3 event4 event5 event6 = event7

Anyway I also tried event6,event5...it didn= 't work.

Can someone quickly explain how it wo= rks ? Is it supposed to work even if ReactOS does not have the arbiter ?
=
More precisely the ReactOS developer = said : "Hardware initialization doesn=E2=80=99t occur on UEFI due to = missing arbiter support :=20 It means that the hardware (even essential such as input devices) will=20 not work and you may not be able to interact with the system in any way&quo= t;

My idea is to use Barrier to be able to use the = mouse and keyboard inside the vm...I will install the server on freebsd and= the client inside the reactos vm...
=

C= ould it work=C2=A0 as long as reactOS has a network connection ?=C2=A0

<= p>Let me know.

3D""

On Fri, Apr 19, 2024 at 3:52=E2=80=AFPM Mario Marietto <marietto2008@gmail.= com> wrote:
Hello.

I'm trying to = boot the 32 bit ISO image of ReactOS using bhyve or qemu and seabios as boo= tloader.=C2=A0

As first experiment I tried to boot= the x86 version of ReactOS with qemu (8.2.2) and seabios (version 1.16.1_1= ),like this :

qemu-system-x86_64 -machine q35 -m 1G -cdrom=20 /home/marietto/Desktop/Downloads/OS/ReactOS/reactos-livecd-0.4.15-dev-7921-= g6d853be-x86-msvc-win-dbg.iso -boot order=3Dd

Unfortunately,I get the error below= :


but my real goal is to boot ReactOS by u= sing seabios. Even here I'm unlucky. Infact on this web page :


<= /div>
we can read :

Bhyve is somewhat simil= ar to QEMU-KVM without the emulation required to boot SeaBIOS.

Is really true that I can't use s= eabios to boot ReactOS with bhyve ?
Anyway,ok. I've cont= inued my searching and I found this thread :

<= a href=3D"https://forums.freebsd.org/threads/bhyve-with-bios-boot.79794/" t= arget=3D"_blank">https://forums.freebsd.org/threads/bhyve-with-bios-boot.79= 794/

where we can read :

<= div>bhyveload has nothing to do with BIOS. As detailed in the man page y= ou linked to, it is for loading a FreeBSD guest.

So,it seems that I can't even use bhyveload. So,what can I use ? Wha= t can I do ?

I've gone on the ReactOS for= um and I've created a thread where=C2=A0
I 've asked how = to boot the 64 bit + UEFI version of ReactOS using bhyve. The developers he= lped me,but in the end we have done nothing. Because after some progress,I = found this error :

https://ibb.co/N9QHdnp=

Unfortunately it is frozen when it tries to load the mountmgr.sys driver. A= ctually unfixed and no one knows how to fix it.

Now= I'm here,asking for some further ideas and suggestions.
=
--
Mario.


--
Mario.
--000000000000677a0e0616861d3b-- From nobody Sat Apr 20 14:29:23 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMDRT57x9z5HsxK for ; Sat, 20 Apr 2024 14:29:25 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VMDRT3kC9z4Zbm; Sat, 20 Apr 2024 14:29:25 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713623365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KtscXpfRl8R8dcwgUX87rtgY8wmpBMY79B7jYifCXmw=; b=rU6KG/tlKHkuTAlnIeL9ABi5p4bhmISCqPH2JdWdK3cuj94od7t/KHVSouGlhHBHNS3C5B uSqILIkrlJ7X91JITiMVpu4XlOsmShwlvS+Ffvq2zPzcz+tHiS9+nbqPkOgAPDtD4ydOVL kUATfntxut2bRg758RlZwb7iDgHlbtYaU53UVa6EcWAte9NuwIJbOZ1WtEfyFOdXcaM0X0 +sHfoVs7cL2ZHjAFoL4oOFVqLW9+EVuCNizFZfsJ05F9ENc/DlmVQofAmTqllOGRZomZ16 P2+w3JDcsAbyyRIJUzfNlBjgzu2RXKecNGboTdC7V1ndfft5UrvckMWJGltNlg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713623365; a=rsa-sha256; cv=none; b=MosiQR4yzBdpYA48YBqqBWjV97Nml3E9Xfz9ASjZSe8Mq6yqsroT26w8YZ4NlSboNyLnBs YtalmnHpxSMA5M+weEhSFic8e+HChr7Eli6gFl49oHxAmpNiFPNThiERRRzgwUWlIZVMn1 vz29h1ehFSJbLXJPpJ4Y+2d9Ts9QMHN/2PKvJbXlnq2y45VMTeKjmhUvTXtaoeaqpuMZcb IH1zjwQl65z4+mWXcnb4NEyuqj+aF7mb9AqRHqC5ljGpeYzMhtvJ3bEf//VmntyGg4u2u7 KualH6zrnvR+U5HaM9RQNy3CP1C2FSGfuCsXODbAuFCLb+91Z79xbRdoK9WjtQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713623365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KtscXpfRl8R8dcwgUX87rtgY8wmpBMY79B7jYifCXmw=; b=mPFm11iAMdTEJTqGEguNvnQ8Kl75Ytt1nqcYC9HsLrTmqKQQpJlLUvI8SJ+7Ko9K84/Wph dXs/pXTkJYSjm/L8uVbOrmLSzdqBJerCwr4B8/M4H4xaC4BTNtoT3vjXZvyMwJULdFQ+PC HigdpkM+e6jub4wL6WdfmHWVI2FsY85AF8ab3TrOWpd+1/QwwhXy/TGmBXu2qZw+IwOwiE hsSls7mRZqh3tbdX8DDYbJoj4BZKobKL7Qk1sA02HonnqUkDRxLls9KH1LjTE/h1VXdhPd fbbD/jKH8jBGmBBrJlXcSxsMUhsAS+/pqujAnqTI9NARnty9OmzuWav/fQhtTg== Received: from ltc.des.dev (163.23.65.37.rev.sfr.net [37.65.23.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VMDRT22n0z1LkR; Sat, 20 Apr 2024 14:29:25 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 1A4FD1F10E; Sat, 20 Apr 2024 16:29:23 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Souradeep Chakrabarti Cc: Wei Hu , Konstantin Belousov , Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: Re: [EXTERNAL] Re: How to add a -W flag in local Makefile In-Reply-To: (Souradeep Chakrabarti's message of "Sat, 20 Apr 2024 12:01:13 +0000") References: <86cyqlbovo.fsf@ltc.des.dev> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Sat, 20 Apr 2024 16:29:23 +0200 Message-ID: <86ttjw9kfw.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Souradeep Chakrabarti writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Wei Hu writes: > > > The same code already exists in Linux and Windows. Linux also added a > > > compiling flag to suppress similar warnings. > >That's not an excuse. The code makes no sense. > The way it works is as follows: I don't care, it's bad code and the compiler is right to complain about it. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Sat Apr 20 15:07:03 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMFH14Gdvz5Hy0q for ; Sat, 20 Apr 2024 15:07:09 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VMFH10svxz4lwd; Sat, 20 Apr 2024 15:07:09 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qk1-x729.google.com with SMTP id af79cd13be357-78efd1a0022so217503485a.3; Sat, 20 Apr 2024 08:07:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713625627; x=1714230427; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=db0YGeBPkiMNHJsxuXjwEeNKFs/oxXobQwmGg2Saq/Y=; b=YORk3nOkF43so0E8xgeiVjODXvfPvHyybCopQOUtheuTDMdGgkIDt6op+HHm4U7zS1 5RUwZq0W0crQtLzrnX9TKOPYl3wWCwetT0LIsQL7mapPOjikaPTA/vj9zQpjt/NoCZZ8 rj5UNxGSDhE8ezRVefBjz+92MMJtWR3OW5gx1YdsUwE2v1s00KjApfuLtvZNYUsQJ45W kmkTvrmYF4SkjfvBtCfkdVH8rma6Ap66kRKhrzhgnLAKJOt1UOAg1tQKGGo3fHaTm0ur 4p6UEjP78JVT06xPpInyoejTNy38yftis1af3PUY8Xzuhhe846pKJwP2ThuE62+94tnJ NcTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713625627; x=1714230427; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=db0YGeBPkiMNHJsxuXjwEeNKFs/oxXobQwmGg2Saq/Y=; b=dF4q5ndObAS3bL9cHLgCKHDcssPSPHeCr7eV5AN9czPVx9JETVkNiE8yAlTEbXB5NH dTGPf+6GvA05dJw168ARpJn/CaVhXNIsyIuvSoKhGEMNF/hPCUl9rEIH3ZSJkwbR9p/L XCHmVBuEPQPAYa0HQDGYJI5aH1uIQoGPohPdFhJVaNOYfoX62tsPm2Cg2e8/L7g4gwVX +BPRpVFIpxasq7N/SWgv8quHpJPnKDKDJHMVx+IwdtxEYIVDsw40W5JGAqRpUS5Sqb8l qhxNIBgX6IRUgj0RiyJSy1uUrgLMMI7cTpcBbD9j4Cg8jCVzOqjFzNimXRlvElwG6oGq 5Saw== X-Gm-Message-State: AOJu0YwlxqVDunHFyQFUlsC1rjxx50Olk5By/n9qLqrqKcL0H3R59USm GvEWpdQNfyYmnNWZJMrHqsvV2tHD60vxxWyKl78ipVjkZL/bzliSJrYwgg== X-Google-Smtp-Source: AGHT+IE/HyZE0F2IOM3FmdOkj5NnLh7gGf8NpymhdyeQ1CDH8kxAXNQFWdY9NJ4zCmw9vX6HQ3X25A== X-Received: by 2002:a05:620a:29c2:b0:790:675c:5f4b with SMTP id s2-20020a05620a29c200b00790675c5f4bmr1245984qkp.49.1713625627034; Sat, 20 Apr 2024 08:07:07 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id u5-20020a05620a022500b0078f13d368f3sm2116256qkm.95.2024.04.20.08.07.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Apr 2024 08:07:06 -0700 (PDT) Date: Sat, 20 Apr 2024 11:07:03 -0400 From: Mark Johnston To: Alan Somers Cc: FreeBSD Hackers Subject: Re: Stressing malloc(9) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VMFH10svxz4lwd On Fri, Apr 19, 2024 at 04:23:51PM -0600, Alan Somers wrote: > TLDR; > How can I create a workload that causes malloc(9)'s performance to plummet? > > Background: > I recently witnessed a performance problem on a production server. > Overall throughput dropped by over 30x. dtrace showed that 60% of the > CPU time was dominated by lock_delay as called by three functions: > printf (via ctl_worker_thread), g_eli_alloc_data, and > g_eli_write_done. One thing those three have in common is that they > all use malloc(9). Fixing the problem was as simple as telling CTL to > stop printing so many warnings, by tuning > kern.cam.ctl.time_io_secs=100000. > > But even with CTL quieted, dtrace still reports ~6% of the CPU cycles > in lock_delay via g_eli_alloc_data. So I believe that malloc is > limiting geli's performance. I would like to try replacing it with > uma(9). What is the size of the allocations that g_eli_alloc_data() is doing? malloc() is a pretty thin layer over UMA for allocations <= 64KB. Larger allocations are handled by a different path (malloc_large()) which goes directly to the kmem_* allocator functions. Those functions are very expensive: they're serialized by global locks and need to update the pmap (and perform TLB shootdowns when memory is freed). They're not meant to be used at a high rate. My first guess would be that your production workload was hitting this path, and your benchmarks are not. If you have stack traces or lock names from DTrace, that would help validate this theory, in which case using UMA to cache buffers would be a reasonable solution. > But on a non-production server, none of my benchmark workloads causes > g_eli_alloc_data to break a sweat. I can't get its CPU consumption to > rise higher than 0.5%. And that's using the smallest sector size and > block size that I can. > > So my question is: does anybody have a program that can really stress > malloc(9)? I'd like to run it in parallel with my geli benchmarks to > see how much it interferes. > > -Alan > From nobody Sat Apr 20 16:01:12 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMGTT4QpNz5J2nn for ; Sat, 20 Apr 2024 16:01:17 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4VMGTT20dSz4sSP; Sat, 20 Apr 2024 16:01:17 +0000 (UTC) (envelope-from yuri@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.5.3] (c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id 43KG1FPY098392 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sat, 20 Apr 2024 09:01:15 -0700 (PDT) (envelope-from yuri@FreeBSD.org) X-Authentication-Warning: shell1.rawbw.com: Host c-98-42-44-116.hsd1.ca.comcast.net [98.42.44.116] claimed to be [192.168.5.3] Content-Type: multipart/alternative; boundary="------------lOhfGVTApTsArjSP8mHuNsKK" Message-ID: <9dd25b52-184d-4e06-af91-1f54f1f882a7@FreeBSD.org> Date: Sat, 20 Apr 2024 09:01:12 -0700 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: What does this error mean: No space available for static Thread Local Storage ? Content-Language: en-US To: Juraj Lutter , Yuri Cc: freebsd-hackers@freebsd.org References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> From: Yuri In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7961, ipnet:198.144.192.0/23, country:US] X-Rspamd-Queue-Id: 4VMGTT20dSz4sSP This is a multi-part message in MIME format. --------------lOhfGVTApTsArjSP8mHuNsKK Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Juraj, On 4/20/24 02:10, Juraj Lutter wrote: > When we’re at rust and memory (thread-local storage?) related: I’ve tried to run databases/qdrant > in production. It works until it’s idle. When some collections are being created/loaded, it crashes. > I have been able to track it down to malloc() called from within strdup() called from within thr_set_name(). This sounds like a serious problem. Changing an allocator to MiMalloc might help. This is an easy change. Did you report this to Rust or to quadrant? Thanks, YUri --------------lOhfGVTApTsArjSP8mHuNsKK Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
Hi Juraj,


On 4/20/24 02:10, Juraj Lutter wrote:
When we’re at rust and memory (thread-local storage?) related: I’ve tried to run databases/qdrant
in production. It works until it’s idle. When some collections are being created/loaded, it crashes.
I have been able to track it down to malloc() called from within strdup() called from within thr_set_name().


This sounds like a serious problem.

Changing an allocator to MiMalloc might help.

This is an easy change.


Did you report this to Rust or to quadrant?



Thanks,

YUri

--------------lOhfGVTApTsArjSP8mHuNsKK-- From nobody Sat Apr 20 17:23:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMJJr0kSlz5J8XC for ; Sat, 20 Apr 2024 17:23:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com [209.85.221.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VMJJq4TFKz43Kb; Sat, 20 Apr 2024 17:23:55 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-4dabbd69c71so833457e0c.1; Sat, 20 Apr 2024 10:23:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713633833; x=1714238633; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/sLfHja7xqhpW1mSfTEg4t+dp4YPmEGvUjKWIfniI4c=; b=RFVdONZeSz8t7YT9Nxa1yqclX6sA2eiW/UmEY8ApuhV2YyZy3Upe3aTksjKLp1wSIl NM7vh9RQWo/7t7FcECFfGhEJKmypURkXE2J9P2cQ4TFHA7SRyt3SfZmS6/JHtOvL7wjY Chgj2/ZWa5m5tAjalzRjQhwqhI7ONX2Mh/is5VqKtMnJ2MFiGk3O+QDFpbS26MUdD7rH M4EvMk2uiz7jBIV1l8IPTBxnvSKywTtpUVRnV6FiRY5pJPrGjERtvEdMjTGk9VF5HBQo LN7aoJQ6Lvh52r6PH2kerZwhytRFvzgh7WU2dvY/wEk8de5YPNvrUxomA/iBFBEpXUoX aHFw== X-Gm-Message-State: AOJu0YyaHm+t0R9RUZ+qOtaQcRnSGGb47q5cwRGCUy6rL7Xs26R/ZnRj fN3DvAOyGFrYwq9goPNhIUg5NB8jA4mQHZ0Ivh1+yeb8YqWzGy+uDQJJ/8E00gfaxJ6lbox6UWW m4kAdL2YsEnEu0BtdSNF7wqagjkklgB9z X-Google-Smtp-Source: AGHT+IFpiAcs/HwUMWeD3u6Jbb+GITOpIfD0A06fgrEt/ZiX9rUx7BZ+aLNsEz/IutFsLwXMv+eezccCRuuYc/TI8xs= X-Received: by 2002:a67:f788:0:b0:47b:b94e:9162 with SMTP id j8-20020a67f788000000b0047bb94e9162mr6089919vso.4.1713633833500; Sat, 20 Apr 2024 10:23:53 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sat, 20 Apr 2024 11:23:41 -0600 Message-ID: Subject: Re: Stressing malloc(9) To: Mark Johnston Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VMJJq4TFKz43Kb On Sat, Apr 20, 2024 at 9:07=E2=80=AFAM Mark Johnston w= rote: > > On Fri, Apr 19, 2024 at 04:23:51PM -0600, Alan Somers wrote: > > TLDR; > > How can I create a workload that causes malloc(9)'s performance to plum= met? > > > > Background: > > I recently witnessed a performance problem on a production server. > > Overall throughput dropped by over 30x. dtrace showed that 60% of the > > CPU time was dominated by lock_delay as called by three functions: > > printf (via ctl_worker_thread), g_eli_alloc_data, and > > g_eli_write_done. One thing those three have in common is that they > > all use malloc(9). Fixing the problem was as simple as telling CTL to > > stop printing so many warnings, by tuning > > kern.cam.ctl.time_io_secs=3D100000. > > > > But even with CTL quieted, dtrace still reports ~6% of the CPU cycles > > in lock_delay via g_eli_alloc_data. So I believe that malloc is > > limiting geli's performance. I would like to try replacing it with > > uma(9). > > What is the size of the allocations that g_eli_alloc_data() is doing? > malloc() is a pretty thin layer over UMA for allocations <=3D 64KB. > Larger allocations are handled by a different path (malloc_large()) > which goes directly to the kmem_* allocator functions. Those functions > are very expensive: they're serialized by global locks and need to > update the pmap (and perform TLB shootdowns when memory is freed). > They're not meant to be used at a high rate. In my benchmarks so far, 512B. In the real application the size is mostly between 4k and 16k, and it's always a multiple of 4k. But it's sometimes great enough to use malloc_large, and it's those malloc_large calls that account for the majority of the time spent in g_eli_alloc_data. lockstat shows that malloc_large, as called by g_elI_alloc_data, sometimes blocks for multiple ms. But oddly, if I change the parameters so that g_eli_alloc_data allocates 128kB, I still don't see malloc_large getting called. And both dtrace and vmstat show that malloc is mostly operating on 512B allocations. But dtrace does confirm that g_eli_alloc_data is being called with 128kB arguments. Maybe something is getting inlined? I don't understand how this is happening. I could probably figure out if I recompile with some extra SDT probes, though. > > My first guess would be that your production workload was hitting this > path, and your benchmarks are not. If you have stack traces or lock > names from DTrace, that would help validate this theory, in which case > using UMA to cache buffers would be a reasonable solution. Would that require creating an extra UMA zone for every possible geli allocation size above 64kB? > > > But on a non-production server, none of my benchmark workloads causes > > g_eli_alloc_data to break a sweat. I can't get its CPU consumption to > > rise higher than 0.5%. And that's using the smallest sector size and > > block size that I can. > > > > So my question is: does anybody have a program that can really stress > > malloc(9)? I'd like to run it in parallel with my geli benchmarks to > > see how much it interferes. > > > > -Alan > > From nobody Sat Apr 20 19:34:30 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMMCg431Dz5JKwg for ; Sat, 20 Apr 2024 19:34:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VMMCg128sz4N9X; Sat, 20 Apr 2024 19:34:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 43KJYUAM028135; Sat, 20 Apr 2024 22:34:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 43KJYUAM028135 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 43KJYUia028134; Sat, 20 Apr 2024 22:34:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Apr 2024 22:34:30 +0300 From: Konstantin Belousov To: Souradeep Chakrabarti Cc: Dag-Erling =?utf-8?B?U23DuHJncmF2?= , Wei Hu , Warner Losh , "freebsd-hackers@FreeBSD.org" Subject: Re: [EXTERNAL] Re: How to add a -W flag in local Makefile Message-ID: References: <86cyqlbovo.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4VMMCg128sz4N9X On Sat, Apr 20, 2024 at 12:01:13PM +0000, Souradeep Chakrabarti wrote: > > > >-----Original Message----- > >From: owner-freebsd-hackers@FreeBSD.org >hackers@FreeBSD.org> On Behalf Of Dag-Erling Smørgrav > >Sent: Friday, April 19, 2024 4:28 PM > >To: Wei Hu > >Cc: Konstantin Belousov ; Warner Losh > >; freebsd-hackers@FreeBSD.org > >Subject: [EXTERNAL] Re: How to add a -W flag in local Makefile > > > >Wei Hu writes: > >> The same code already exists in Linux and Windows. Linux also added a > >> compiling flag to suppress similar warnings. > > > >That's not an excuse. The code makes no sense. > The way it works is as follows: > > > > +struct hv_vpset { > > > + uint64_t format; > > > + uint64_t valid_bank_mask; > > > + uint64_t bank_contents[]; > > > +} __packed; > > > + > > > +struct hv_tlb_flush_ex { > > > + uint64_t address_space; > > > + uint64_t flags; > > > + struct hv_vpset hv_vp_set; > > > + uint64_t gva_list[]; > > > +} __packed; > > Here an pointer is allocated for struct hv_tlb_flush_ex with 4K size. > Now gva_list and bank_contents are sharing same space. > But gva_list gets filled up from the particular index position, where bank_contents > is ending. > Total space used by bank_contents[0-(n-1)] + gva_list[n-end_of_the heap allocated] = 4K - (sizeof(format) + sizeof(valid_bank_mask) + sizeof(address_space) + sizeof(flags) ) > During a call to hypervisor we need to send in this message format for proper deserialization and there we also specify the bank_contents index that is n-1. Indexing bank_contents is not a valid C code. You do not need the second flexible member, instead you should calculate the location manually and access it there. Your hack relies on the internal layouts of the structures, but should instead use proper address arithmetic. > Thanks > Souradeep > > > >> Anyway, the purpose of this email is to understand how to add such > >> flags in local Makefiles and make it effective for global buildkernel. > >> Adding such flags in local Makefiles already proves to be working when > >> only building under local directory. > > > >You would have to add it to every line in sys/conf/files* that references a C file > >which includes this header. > > > >DES > >-- > >Dag-Erling Smørgrav - des@FreeBSD.org > From nobody Sun Apr 21 16:09:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMtc73lRHz5HFGK for ; Sun, 21 Apr 2024 16:09:11 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VMtc65mJ8z4LWt; Sun, 21 Apr 2024 16:09:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-436f1a770bdso33734001cf.0; Sun, 21 Apr 2024 09:09:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713715749; x=1714320549; darn=freebsd.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=FIUdFWwBbqsRD3j7L8HgBvuUXbS8uTtsYyJsOdepD3k=; b=L2/Hfzyb2JUmNqVcF8Kpe/Hl9dhBA+QWahd2cfkBxnWDw5PdCTo9TBY1qaHWKvC8lZ yi8ZpHIKJqosspMjtxny0v9kCmiS5n9KBVYMM3zlPFU9hOBBI0Yia9bHt2DzliUZs25N pS3+rPRElkVWbO+xXKnBspY20mI5vtLgpkvVZaveJbBuJ7BHgi856zu0cQ3Q5qOIFzVZ LVhVXS6e/bumpxG46bL42XS6RzFkmjp7E/SkoMGiZ8zy5mfWwtMLLqmaNNzeO+03KKbc A1CYRtsaEyWL21Fb2Wr/mcbNDlVVrLDyayBLJr4nKzbrStpIvADdEEDXBC5Kk8WZqxG1 kEiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713715749; x=1714320549; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FIUdFWwBbqsRD3j7L8HgBvuUXbS8uTtsYyJsOdepD3k=; b=kIq0C2DQVqgcpP67odcsEsQ+cK4chJX0LZPdzDrLv97AWGu8EMgO+SV9CuPnNz1Tuw YzipZNJ7h1WuQbcS6dQxWAUZLvecIPRbKbW5IoNV4oc8iLUMweXg8/usk5KXbdnqehpV ZZtJO9GnnpytPzolsy5EMl6Bhk7sWT59P7wd0BESqi9BBhSTY28ERNmpxz4zsmYQvP9x 3HgZDIyHla9XxPIasDrhefZLmygJUWySL0zrw9vbc663NQrxKVy2OESqGDYVIBnYuWE6 fg6P35F6oyD9EoMFukozCCi2RPmPlDu1k3MWJJEC78YMV8qX7Yfk5iWJZOhHAhwGKUOv s+9Q== X-Gm-Message-State: AOJu0Yx+d2w3fmiWsyYLqEELfHcg85yuN6M5WXAT65/S9Vs5cGNSP7dq ckgeef0F31mnZ0N0pQfLc4PHHHEaIluX9c8ujWK1mu48IeYSsFcCN4Em0g== X-Google-Smtp-Source: AGHT+IE9D3KQIDl3cglGnpf2In7TE7B8WVAQ7/8AL6fWrcbwAuHpXdyQtkITOeIzPiPv9CddjIxz1w== X-Received: by 2002:a05:622a:5189:b0:439:7526:2bad with SMTP id ex9-20020a05622a518900b0043975262badmr8753784qtb.16.1713715748873; Sun, 21 Apr 2024 09:09:08 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id v10-20020ac873ca000000b004378b8ef629sm3479637qtp.31.2024.04.21.09.09.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 09:09:07 -0700 (PDT) Date: Sun, 21 Apr 2024 12:09:05 -0400 From: Mark Johnston To: Alan Somers Cc: FreeBSD Hackers Subject: Re: Stressing malloc(9) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VMtc65mJ8z4LWt On Sat, Apr 20, 2024 at 11:23:41AM -0600, Alan Somers wrote: > On Sat, Apr 20, 2024 at 9:07 AM Mark Johnston wrote: > > > > On Fri, Apr 19, 2024 at 04:23:51PM -0600, Alan Somers wrote: > > > TLDR; > > > How can I create a workload that causes malloc(9)'s performance to plummet? > > > > > > Background: > > > I recently witnessed a performance problem on a production server. > > > Overall throughput dropped by over 30x. dtrace showed that 60% of the > > > CPU time was dominated by lock_delay as called by three functions: > > > printf (via ctl_worker_thread), g_eli_alloc_data, and > > > g_eli_write_done. One thing those three have in common is that they > > > all use malloc(9). Fixing the problem was as simple as telling CTL to > > > stop printing so many warnings, by tuning > > > kern.cam.ctl.time_io_secs=100000. > > > > > > But even with CTL quieted, dtrace still reports ~6% of the CPU cycles > > > in lock_delay via g_eli_alloc_data. So I believe that malloc is > > > limiting geli's performance. I would like to try replacing it with > > > uma(9). > > > > What is the size of the allocations that g_eli_alloc_data() is doing? > > malloc() is a pretty thin layer over UMA for allocations <= 64KB. > > Larger allocations are handled by a different path (malloc_large()) > > which goes directly to the kmem_* allocator functions. Those functions > > are very expensive: they're serialized by global locks and need to > > update the pmap (and perform TLB shootdowns when memory is freed). > > They're not meant to be used at a high rate. > > In my benchmarks so far, 512B. In the real application the size is > mostly between 4k and 16k, and it's always a multiple of 4k. But it's > sometimes great enough to use malloc_large, and it's those > malloc_large calls that account for the majority of the time spent in > g_eli_alloc_data. lockstat shows that malloc_large, as called by > g_elI_alloc_data, sometimes blocks for multiple ms. > > But oddly, if I change the parameters so that g_eli_alloc_data > allocates 128kB, I still don't see malloc_large getting called. And > both dtrace and vmstat show that malloc is mostly operating on 512B > allocations. But dtrace does confirm that g_eli_alloc_data is being > called with 128kB arguments. Maybe something is getting inlined? malloc_large() is annotated __noinline, for what it's worth. > I > don't understand how this is happening. I could probably figure out > if I recompile with some extra SDT probes, though. What is g_eli_alloc_sz on your system? > > My first guess would be that your production workload was hitting this > > path, and your benchmarks are not. If you have stack traces or lock > > names from DTrace, that would help validate this theory, in which case > > using UMA to cache buffers would be a reasonable solution. > > Would that require creating an extra UMA zone for every possible geli > allocation size above 64kB? Something like that. Or have a zone of maxphys-sized buffers (actually I think it needs to be slightly larger than that?) and accept the corresponding waste, given that these allocations are short-lived. This is basically what g_eli_alloc_data() already does. > > > But on a non-production server, none of my benchmark workloads causes > > > g_eli_alloc_data to break a sweat. I can't get its CPU consumption to > > > rise higher than 0.5%. And that's using the smallest sector size and > > > block size that I can. > > > > > > So my question is: does anybody have a program that can really stress > > > malloc(9)? I'd like to run it in parallel with my geli benchmarks to > > > see how much it interferes. > > > > > > -Alan > > > From nobody Sun Apr 21 18:45:22 2024 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VMy4h1rmqz5HW78 for ; Sun, 21 Apr 2024 18:45:40 +0000 (UTC) (envelope-from lexi@le-fay.org) Received: from fuchsia.eden.le-Fay.ORG (fuchsia.eden.le-fay.org [IPv6:2001:8b0:aab5:107::11]) by mx1.freebsd.org (Postfix) with ESMTP id 4VMy4g46X9z4c8K for ; Sun, 21 Apr 2024 18:45:39 +0000 (UTC) (envelope-from lexi@le-fay.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=le-fay.org header.s=fuchsia header.b=fMdfDT+N; dmarc=none; spf=pass (mx1.freebsd.org: domain of lexi@le-fay.org designates 2001:8b0:aab5:107::11 as permitted sender) smtp.mailfrom=lexi@le-fay.org Received: from iris.eden.le-Fay.ORG (iris.eden.le-fay.org [IPv6:2001:8b0:aab5:106:3::6]) by fuchsia.eden.le-Fay.ORG (Postfix) with ESMTP id 40DD5952A for ; Sun, 21 Apr 2024 18:45:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=le-fay.org; s=fuchsia; t=1713725131; bh=DnNC4MXBYIz0mHVz6t/zqSOCVj3qfRQXNOSSGdguMWM=; h=Date:From:To:Subject; b=fMdfDT+N29R8QnxhU+kV3VUIvjOmBYpb1OgHkziUMbzPWUFc5DIwbKhdyxJc8Q31S 96Ed1Hp1isDvJM2RPGOHdrYqpHQkw8rs2cS3LsxvpRFqKuaizcMVswiVnU0VnbrpSF tumb3p3ZEFJzPBzDxGp00nSYXcwARWbl+esrjImM= Received: from ilythia.eden.le-fay.org (ilythia.eden.le-fay.org [IPv6:2001:8b0:aab5:106:3::10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by iris.eden.le-Fay.ORG (Postfix) with ESMTPSA id 05EE32C0421 for ; Sun, 21 Apr 2024 19:45:31 +0100 (BST) Date: Sun, 21 Apr 2024 19:45:22 +0100 From: Lexi Winter To: hackers@freebsd.org Subject: building memstick without root Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UdEh3BBIxHwM4I4G" Content-Disposition: inline X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.41 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.91)[-0.911]; R_DKIM_ALLOW(-0.20)[le-fay.org:s=fuchsia]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2001:8b0:aab5:107::11]; RCVD_NO_TLS_LAST(0.10)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:20712, ipnet:2001:8b0::/32, country:GB]; MISSING_XM_UA(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[le-fay.org:dkim]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DMARC_NA(0.00)[le-fay.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; MLMMJ_DEST(0.00)[hackers@freebsd.org]; DKIM_TRACE(0.00)[le-fay.org:+] X-Rspamd-Queue-Id: 4VMy4g46X9z4c8K --UdEh3BBIxHwM4I4G Content-Type: text/plain; charset=us-ascii Content-Disposition: inline hello, is it expected that running 'make -C release memstick' requires root? the build fails for me with: install: /src/obj/src/freebsd/src/main/riscv.riscv64/release/dist/kernel/boot/kernel/kernel: chown/chgrp: Operation not permitted as i understand it, makefs should be able to build images as a non-root user using mtree, but i'm not sure if this is hooked up to the build system, or if i'm doing something else wrong - maybe i need a make.conf/src.conf option set? --UdEh3BBIxHwM4I4G Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCAAdFiEEuwt6MaPcv/+Mo+ftDHqbqZ41x5kFAmYlXr8ACgkQDHqbqZ41 x5mrRQv8CgpjugBuJUDP9Q5RJmU4U86uTlARZyL9hyEKtkidljDwIKONKzOqtQeT lMoh9OVuj6eqjGTxWBHVpV8Lv/G09X/DI5soVlanAHHNw9QSJzKx8lxPuPzpPTO0 nI3qhOuMxcHdw3zq0rg6OHZMMwLWFS8b7GXQhZHalPuvcC36Q7zXP8MF/v9rNg4i RcoHXeWVHv2DfmNaoVKsLsidUuMO9GQW07zEot9/kf3cxbjZnIe0zgp+ckiBHvuq oHSHU1/gA7I3GkuRt4GJ9nYAtu7hVkO6jbMS7SMp1qLtftPc3iFGGzstVk6pwVfD 9+B56upb4uP7pDNwzXO7eqTfmA6Y6DXj5EXt02JFOAqfbERq1uv3M85Yzdw8ZQF2 7u/Aqz4bmS4bxDAIu6oDrpMUcyka0xGbESLl5y8eMGfvRbmzTtH2W1mmJurwXcoB bafugpiNuciQAppYgWI+1enmVJ+7Kv1wUAa81caw1qg1ZEwfTMUA9dqhOdfenqxB E7u1D8w8 =ZE3S -----END PGP SIGNATURE----- --UdEh3BBIxHwM4I4G-- From nobody Sun Apr 21 23:47:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VN4nR5jWfz5HpQH for ; Sun, 21 Apr 2024 23:47:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f42.google.com (mail-ua1-f42.google.com [209.85.222.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VN4nR1tp6z4LyD; Sun, 21 Apr 2024 23:47:55 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f42.google.com with SMTP id a1e0cc1a2514c-7e3e0b2f653so1243131241.0; Sun, 21 Apr 2024 16:47:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713743273; x=1714348073; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BgEgqqN9xzpss+9fNA45cMgyI1fMp1hGVLnbpKdZseg=; b=EAZKJap+vXnsz15e0IH0uRwbHISGTBOq6kvCMLQez7F89dWmdEBk3m85XiBvfBsnOX P/9+ekyFGcC3J4YYI1n0cvKN96kkM+tqnJ/gPxR02ryI5wyQq4lfOyEyMf+T3GNkel02 XVupjLBvT6SvONUzTkf9HOGrdddIgLHv+h1wZpIKHe1a9X6xc0qoCVHRVLjQnhxuf4tL IhkirVdlw/XWg25PQsM1q1EGaI3B4kAEOhGm6qzxtProMpX1SPeuRTd+GmtWIVm3fbCS hjuCiGjKDZcUvP+uD7OeudukEknyoxxGEoDaMD5Fn+5F/DCos/LwYRLIdGERfp/5Rqrl 4RGQ== X-Gm-Message-State: AOJu0YzWlbB8iU6C9QQMm3bHNmdv/Hfk1ctcuLAu2ZZNbGFckLctXAcX r5syoBZCvvimjDSIotGla0J9AXcQlr0hdXAXRO/t9cAPpJGkhoVww0ol67jbnBaam3aISbQPY2m pF+mgmynfMc10Slc9rXMp/F2HlfT4vQ== X-Google-Smtp-Source: AGHT+IFxDnbNtOqtZKCywqh/op1O7Fda8X/4bVveoZWZlW9OM4xBqxlVdI+6DZzDa5GDy3UQzUKLwoStLxC2vY/kBhk= X-Received: by 2002:a05:6122:2a0e:b0:4dc:d7b2:7602 with SMTP id fw14-20020a0561222a0e00b004dcd7b27602mr9166467vkb.1.1713743272966; Sun, 21 Apr 2024 16:47:52 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sun, 21 Apr 2024 17:47:41 -0600 Message-ID: Subject: Re: Stressing malloc(9) To: Mark Johnston Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VN4nR1tp6z4LyD On Sun, Apr 21, 2024 at 10:09=E2=80=AFAM Mark Johnston = wrote: > > On Sat, Apr 20, 2024 at 11:23:41AM -0600, Alan Somers wrote: > > On Sat, Apr 20, 2024 at 9:07=E2=80=AFAM Mark Johnston wrote: > > > > > > On Fri, Apr 19, 2024 at 04:23:51PM -0600, Alan Somers wrote: > > > > TLDR; > > > > How can I create a workload that causes malloc(9)'s performance to = plummet? > > > > > > > > Background: > > > > I recently witnessed a performance problem on a production server. > > > > Overall throughput dropped by over 30x. dtrace showed that 60% of = the > > > > CPU time was dominated by lock_delay as called by three functions: > > > > printf (via ctl_worker_thread), g_eli_alloc_data, and > > > > g_eli_write_done. One thing those three have in common is that the= y > > > > all use malloc(9). Fixing the problem was as simple as telling CTL= to > > > > stop printing so many warnings, by tuning > > > > kern.cam.ctl.time_io_secs=3D100000. > > > > > > > > But even with CTL quieted, dtrace still reports ~6% of the CPU cycl= es > > > > in lock_delay via g_eli_alloc_data. So I believe that malloc is > > > > limiting geli's performance. I would like to try replacing it with > > > > uma(9). > > > > > > What is the size of the allocations that g_eli_alloc_data() is doing? > > > malloc() is a pretty thin layer over UMA for allocations <=3D 64KB. > > > Larger allocations are handled by a different path (malloc_large()) > > > which goes directly to the kmem_* allocator functions. Those functio= ns > > > are very expensive: they're serialized by global locks and need to > > > update the pmap (and perform TLB shootdowns when memory is freed). > > > They're not meant to be used at a high rate. > > > > In my benchmarks so far, 512B. In the real application the size is > > mostly between 4k and 16k, and it's always a multiple of 4k. But it's > > sometimes great enough to use malloc_large, and it's those > > malloc_large calls that account for the majority of the time spent in > > g_eli_alloc_data. lockstat shows that malloc_large, as called by > > g_elI_alloc_data, sometimes blocks for multiple ms. > > > > But oddly, if I change the parameters so that g_eli_alloc_data > > allocates 128kB, I still don't see malloc_large getting called. And > > both dtrace and vmstat show that malloc is mostly operating on 512B > > allocations. But dtrace does confirm that g_eli_alloc_data is being > > called with 128kB arguments. Maybe something is getting inlined? > > malloc_large() is annotated __noinline, for what it's worth. > > > I > > don't understand how this is happening. I could probably figure out > > if I recompile with some extra SDT probes, though. > > What is g_eli_alloc_sz on your system? 33kiB. That's larger than I expected. When I use a larger blocksize in my benchmark, then I do indeed see malloc_large activity, and 11% of the CPU is spend in g_eli_alloc_data. I guess I'll add some UMA zones for this purpose. I'll try 256k and 512k zones, rounding up allocations as necessary. Thanks for the tip. > > > > My first guess would be that your production workload was hitting thi= s > > > path, and your benchmarks are not. If you have stack traces or lock > > > names from DTrace, that would help validate this theory, in which cas= e > > > using UMA to cache buffers would be a reasonable solution. > > > > Would that require creating an extra UMA zone for every possible geli > > allocation size above 64kB? > > Something like that. Or have a zone of maxphys-sized buffers (actually > I think it needs to be slightly larger than that?) and accept the > corresponding waste, given that these allocations are short-lived. This > is basically what g_eli_alloc_data() already does. > > > > > But on a non-production server, none of my benchmark workloads caus= es > > > > g_eli_alloc_data to break a sweat. I can't get its CPU consumption= to > > > > rise higher than 0.5%. And that's using the smallest sector size a= nd > > > > block size that I can. > > > > > > > > So my question is: does anybody have a program that can really stre= ss > > > > malloc(9)? I'd like to run it in parallel with my geli benchmarks = to > > > > see how much it interferes. > > > > > > > > -Alan > > > >