From owner-freebsd-stable@freebsd.org Sun Feb 10 17:31:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFA4114D3539 for ; Sun, 10 Feb 2019 17:31:41 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 222D088195; Sun, 10 Feb 2019 17:31:30 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1AHVLlQ051726 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 10 Feb 2019 18:31:21 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-stable@freebsd.org Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1AHVKRt027578 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 11 Feb 2019 00:31:20 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable From: Eugene Grosbein Subject: amd64, run-time linker and 32bit Cc: kib@freebsd.org Message-ID: Date: Mon, 11 Feb 2019 00:31:19 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 222D088195 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.24 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.43)[ip: (-2.48), ipnet: 2a01:4f8::/29(-2.38), asn: 24940(-2.25), country: DE(-0.01)]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.22)[-0.218,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 17:31:42 -0000 Hi! Why our 32-bit run-time linker looks for shared libraries in the /usr/local/lib despite of its absence in /var/run/ld-elf32.so.hints while 32-bit binary is started under FreeBSD 11.2-STABLE/amd64 ? If it finds 64-bit version of library in /usr/local/lib, it fails immediately and does not even re-try to look at other directories noted in /var/run/ld-elf32.so.hints such as /usr/lib32:/usr/local/lib/compat/lib32:/usr/local/lib/compat/lib32/compat/pkg:/usr/local/lib32/compat where right 32-bit version is located. As workaround, I can use /etc/libmap32.conf and then the binary starts just fine but there are so many libraries. It should not even try to look to /usr/local/lib if it is not in the /var/run/ld-elf32.so.hints, should it? From owner-freebsd-stable@freebsd.org Sun Feb 10 17:37:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EF9E14D3979 for ; Sun, 10 Feb 2019 17:37:17 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6CA538864D; Sun, 10 Feb 2019 17:37:16 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 3E0832110BA; Sun, 10 Feb 2019 12:37:15 -0500 (EST) Received: from [192.168.10.12] (D2.Denninger.Net [192.168.10.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id DE6222050BF; Sun, 10 Feb 2019 11:37:14 -0600 (CST) Subject: Re: Fwd: Serious ZFS Bootcode Problem (GPT NON-UEFI) To: freebsd-stable@freebsd.org References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> Cc: Allan Jude , Warner Losh From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Sun, 10 Feb 2019 11:37:14 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms000403020203000908000202" X-Rspamd-Queue-Id: 6CA538864D X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.75 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; DMARC_NA(0.00)[denninger.net]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.29)[-0.292,0]; IP_SCORE(-2.25)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.02), asn: 14061(2.73), country: US(-0.07)]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 17:37:17 -0000 This is a cryptographically signed message in MIME format. --------------ms000403020203000908000202 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/10/2019 09:28, Allan Jude wrote: > Are you sure it is non-UEFI? As the instructions you followed, > overwriting da0p1 with gptzfsboot, will make quite a mess if that > happens to be the EFI system partition, rather than the freebsd-boot > partition. Absolutely certain.=C2=A0 The system board in this machine (and a bunch I= have in the field) are SuperMicro X8DTL-IFs which do not support UEFI at all (they have no available EFI-capable bios.) They have encrypted root pools but due to the inability of gptzfsboot to read them they have a small freebsd-zfs partition that, when upgraded, I copy /boot/* to after the kernel upgrade is done but before they are rebooted.=C2=A0 That partition is not mounted during normal operation; it= 's only purpose is to load the kernel (and pre-boot .kos such as geli.) > Can you show 'gpart show' output? [karl@NewFS ~]$ gpart show da1 =3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0 468862061=C2=A0 da1=C2=A0= GPT=C2=A0 (224G) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 2014=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.0= M) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2048=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 1024=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (512K) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 3072=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 1024=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (512K) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0=C2=A0 20971520=C2=A0=C2=A0= =C2=A0 2=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (10G) =C2=A0=C2=A0 20975616=C2=A0 134217728=C2=A0=C2=A0=C2=A0 3=C2=A0 freebsd-s= wap=C2=A0 (64G) =C2=A0 155193344=C2=A0 313667584=C2=A0=C2=A0=C2=A0 4=C2=A0 freebsd-zfs=C2= =A0 (150G) =C2=A0 468860928=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1167=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 - free -=C2=A0 (584K) Partition "2" is the one that should boot. There is also a da2 that has an identical layout (mirrored; the drives are 240Gb Intel 730 SSDs) > What is the actual boot error? It says it can't load the kernel and gives me a prompt.=C2=A0 "lsdev" sho= ws all the disks and all except the two (zfs mirror) that have the "bootme" partition on them don't show up as zfs pools at all (they're geli-encrypted, so that's not unexpected.)=C2=A0 I don't believe the load= er ever gets actually loaded. An attempt to use "ls" from the bootloader to look inside that "bootme" partition fails; gptzfsboot cannot get it open. My belief was that I screwed up and wrote the old 11.1 gptzfsboot to the freebsd-boot partition originally -- but that is clearly not the case. Late last night I took my "rescue media" (which is a "make memstick" from the build of -STABLE), booted that on my sandbox machine, stuck two disks in there and made a base system -- which booted.=C2=A0 Thus whateve= r is going on here it is not as simple as it first appears as that system had the spacemap_v2 flag on and active once it came up. This may be my own foot-shooting since I was able to make a bootable system on my sandbox using the same media (a clone hardware-wise so also no EFI) -- there may have been some part of the /boot hierarchy that didn't get copied over, and if so that would explain it. Update: Indeed that appears to be what it was -- a couple of the *other* files in the boot partition didn't get copied from the -STABLE build (although the entire kernel directory did)....=C2=A0 I need to look at wh= y that happened as the update process is my own due to the dual-partition requirement for booting with non-EFI but that's not your problem -- it's mine. Sorry about this one; turns out to be something in my update scripts that failed to move over some of the files to the non-encrypted /boot....= BTW am I correct that gptzfsboot did *not* get the ability to read geli-encrypted pools in 12.0?=C2=A0 The UEFI loader does know how (which = I'm using on my laptop) but I was under the impression that for non-UEFI systems you still needed the unencrypted boot partition from which to load the kernel. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms000403020203000908000202 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEwMTczNzE0 WjBPBgkqhkiG9w0BCQQxQgRAFu+ztIEdV6hFoX4qWWypJrkTASAwlNiK7oEe4jcri6zwIVjj Vklp9wI50lzuq7zJvu6HEDZwZ7hqWEhRg6eKpzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgALPKO9xD/hk3kBxEmN/Flo2TG0ofc0i2cX5hvoDhkIWxlSm72A/Zlmz8ZkwUAWI5jJ la3uYtrcViGdgJfsum1dMjDhnwwnYPhdEjlsdgduPwV1vEWUb9eMi6bleR+UMdOCP86U2RtY at+4/Bt2BL/WqvdM6CzBDPgp+oxuqtanNlwv/p+HJZJwBs5YvaqfNwbw7hnVSEfDzBDlDKkp KC92xqv2eNc0FQ578Ub7UvJrLiKAy6d5YbeMLX44N+dMTw682uhYkGPqu01IBsnGGkAcmkSp SOad22ShAd8U1geYl+3m6JXfpt/0RYmeet7yrKyZoIun8eotZwzlJBCXMX1Xgx+w+zV5IBkM xb++kLDmq94zeT5rtHDnZi+RcowaHmICNZuZinK+gPUH7Bh60SCGe/cWN2nN5fP65f9KylUQ r2WB+fdzEYI7Qbi2jX2ZRaMYoFwa5U9kQ0hsNj/knjdD2RSq4sZGsIoQBD495VUxeWCdUGg7 kzCRpq40qfyDklTJeS9eohsz+r9DtQiPAabIfhJAEgi+Hr1VS625DKU5Sr5rcejeaKbhfW36 aInMwZfwY9ngPec9o6FWTVWTidDKHDfSKBkbkbQ7AE9w8bsaZGEYuRh44dLovl2aoWOI6pT5 c+UpvrkmYgJ2DABy1X8i/dOU6sE0xyzYS7Yl5xad/AAAAAAAAA== --------------ms000403020203000908000202-- From owner-freebsd-stable@freebsd.org Sun Feb 10 17:54:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00E8614D413B for ; Sun, 10 Feb 2019 17:54:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5551B88F3B; Sun, 10 Feb 2019 17:54:52 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 0DF252110AF; Sun, 10 Feb 2019 12:54:50 -0500 (EST) Received: from [192.168.10.12] (D2.Denninger.Net [192.168.10.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id E0E4C2053D5; Sun, 10 Feb 2019 11:54:50 -0600 (CST) Subject: Re: Fwd: Serious ZFS Bootcode Problem (GPT NON-UEFI) To: Ian Lepore Cc: freebsd-stable@freebsd.org, Allan Jude References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> Date: Sun, 10 Feb 2019 11:54:50 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050201080404070104040606" X-Rspamd-Queue-Id: 5551B88F3B X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.03 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.995,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; DMARC_NA(0.00)[denninger.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.57)[-0.571,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.26)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.06), asn: 14061(2.73), country: US(-0.07)]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 17:54:53 -0000 This is a cryptographically signed message in MIME format. --------------ms050201080404070104040606 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/10/2019 11:50, Ian Lepore wrote: > On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: >> On 2/10/2019 09:28, Allan Jude wrote: >>> Are you sure it is non-UEFI? As the instructions you followed, >>> overwriting da0p1 with gptzfsboot, will make quite a mess if that >>> happens to be the EFI system partition, rather than the freebsd- >>> boot >>> partition. >> [...] >> >> BTW am I correct that gptzfsboot did *not* get the ability to read >> geli-encrypted pools in 12.0? The UEFI loader does know how (which I'= m >> using on my laptop) but I was under the impression that for non-UEFI >> systems you still needed the unencrypted boot partition from which to >> load the kernel. >> > Nope, that's not correct. GELI support was added to the boot and loader= > programs for both ufs and zfs in freebsd 12. You must set the geli '-g'= =20 > option to be prompted for the passphrase while booting (this is > separate from the '-b' flag that enables mounting the encrypted > partition as the rootfs). You can use "geli configure -g" to turn on > the flag on any existing geli partition. > > -- Ian Excellent - this will eliminate the need for me to run down the foot-shooting that occurred in my update script since the unencrypted kernel partition is no longer needed at all.=C2=A0 That also significantl= y reduces the attack surface on such a machine (although you could still tamper with the contents of freebsd-boot of course.) The "-g" flag I knew about from experience in putting 12 on my X1 Carbon (which works really well incidentally; the only issue I'm aware of is that there's no 5Ghz WiFi support.) --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms050201080404070104040606 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEwMTc1NDUw WjBPBgkqhkiG9w0BCQQxQgRAo8+0Fu9IZS4M+EGdG2NYDZ8iycbhKv4KtiVjfkXII9gX88W7 0kRnsZfbyBW7HsfGm8W0t/xhhahUFOHD4/MJ+jBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCspP9AMQO1I+bDEkyMHj2OEJmr71nUfgqcBShl6t0VBP1MoLYAiUDuax3+8JswAQWG 14z0RpdigJsmHNp+NMBRSySIazY0KwoLkJVpXAzNibdaIi33mLto1ul1HqYaVn+RENEGmqDi XYjQErFvYuDHPdGUYWlS4LE23hh0jDl8OHO59JMqv4PIzfpMsqPaNRmoBOkD2Z78aZpqqTQ0 mvj0gp5SHfF10bPe9iL0r7yKF7cBoL3VL7Bu7wShFQ0BasfOoS/vWLgwplhMo9jM/3CaB1bQ ykseK9fQxDMO1N2+XM/v2AmjBi/A50MKspy02ZXWD4nPbG8BWbQ6RT9BQdxAToSUjV4p3FAb 634BMS18FWi/K+/Lk+T1WAt17g5XikUcCveP9mF5kqqeccSjznVLVQmWr4TKIDan+h9VQFKI f2coORG4B9BYn7eYXusIPg6a6i8ZkuFN65EbE8T/mTVnoFO6W8v60G2MUE+2K4xjfie7SkA8 4faL8cxxrbkTzdzYa10PIue/WmL0Gcc6jqobW3PyX6GZiClrFZ9pavBgGFqtktOFComoCx8K SkZI/nonpKKp7Cp8VKhhMby83OsJPCAwZG0VzMjkn72x5n+uqRu54YMFnhm0cmTdYEcZ8VB5 nusRrGrPV85bRHUziaEmBQnTTZK2JhCZkPHrY4D9nAAAAAAAAA== --------------ms050201080404070104040606-- From owner-freebsd-stable@freebsd.org Sun Feb 10 18:02:06 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B034714D4831 for ; Sun, 10 Feb 2019 18:02:06 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1.eu.mailhop.org (outbound1.eu.mailhop.org [52.28.251.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EEC9D89603 for ; Sun, 10 Feb 2019 18:02:05 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1549821724; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=ibzwW41Yq8OpggHlWV3PI7SA9bz7RLHEDJ+rNWwifUADJbOtklN63KoLE5VxIGyAjjlu0bodsFGxe 4v1qi+4DQyOHA8RymBMCGGMFWo4A01j6oOpWjTCjFPHVXLRAAHoN/s/Iirsydy7vyM8wI6WQyM1RcM DVneQpRb5TEEHj/h/ozV7bKsHFY9aVhQuAg5vGBpusB+fSDdGlL7l1pSUoQGmoekNP1eVvjg/t9crB xz0V2KOS+DFX5cOK1LRhUjusovQdUg9WPiKLNG8hwtYqqSYwIs0fAZV6MveMnN222q9ETyJ0kg92YY RDcMn41DKJt+wdWM1tiuFiesKVoP+OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=XqOIj1Ux+NcMXfC4JzACp6pkO5ENeCLkpv+PhtIUoCQ=; b=LdHHB01O2X1JT+I5bXHEZBbwXT6hYS1bIWm+E7cDRX+m6fPKv/aNuUXB7FBwZwrHycVAD2aeM1z9h WLy5YaLyucEILBuOAiKukWMoFyNw7kDKnTz3wQeG03N4DMblW3DXb9rzhjZZiZP8REKQlLG4hB3P5c 2ZpXebjE+OLxdgf+37ASYcXe9np49wMZBDgypgNr0mHZzUDTe4IA5wij7rebpvPpZjzF1szddAwrp6 MmNAIyASetB87ZYXgEzBwacbqfmPUrrtkhS2FxbeWgJwupl98JAC3aDQz7DWt2WsBTNR/bR8Pr6tBX 1vje6GSpPauOq9Vj0zMfyKQ9kmFHDBg== ARC-Authentication-Results: i=1; outbound3.eu.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=XqOIj1Ux+NcMXfC4JzACp6pkO5ENeCLkpv+PhtIUoCQ=; b=d+N0tBawqfofGF0e4ewSyA5PTS7coQurwNDPxY7F1D6fwOUYcfiQOwWBaV98462ExEx8Dchvb/xAz XMwRb/ZSKpqtPHz4xzceMx2AoXo2Ne6jYQnP/00PNmaOba++d6E54QKa1gKH3GFCIKbFhlO0xEG0Kh qfbuQWzhoyYIwWDSPinUXtHN2qfEVAd71uppdp+iSr4ilWE1x98uf/3qeLzs+UFrt/DwQByxQhXJTl QNsJUjF8Cj9sei1rdb+qJ5xqmTCXVkS25VIWxc5Do25D7nCCCB1BOvDhiEWxNjwft11tWVYMRehSxJ u1BJdjbUyX3gI3EH/H1showX0uWRufQ== X-MHO-RoutePath: aGlwcGll X-MHO-User: f614285f-2d5d-11e9-908b-352056dbf2de X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.eu.mailhop.org (Halon) with ESMTPSA id f614285f-2d5d-11e9-908b-352056dbf2de; Sun, 10 Feb 2019 18:02:00 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x1AI1xvc052693; Sun, 10 Feb 2019 11:01:59 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> Subject: Re: Fwd: Serious ZFS Bootcode Problem (GPT NON-UEFI) From: Ian Lepore To: Karl Denninger Cc: freebsd-stable@freebsd.org, Allan Jude Date: Sun, 10 Feb 2019 11:01:59 -0700 In-Reply-To: <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: EEC9D89603 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.93 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.93)[-0.926,0]; ASN(0.00)[asn:16509, ipnet:52.28.0.0/16, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 18:02:06 -0000 On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote: > On 2/10/2019 11:50, Ian Lepore wrote: > > On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: > > > On 2/10/2019 09:28, Allan Jude wrote: > > > > Are you sure it is non-UEFI? As the instructions you followed, > > > > overwriting da0p1 with gptzfsboot, will make quite a mess if > > > > that > > > > happens to be the EFI system partition, rather than the > > > > freebsd- > > > > boot > > > > partition. > > > > > > [...] > > > > > > BTW am I correct that gptzfsboot did *not* get the ability to > > > read > > > geli-encrypted pools in 12.0? The UEFI loader does know how > > > (which I'm > > > using on my laptop) but I was under the impression that for non- > > > UEFI > > > systems you still needed the unencrypted boot partition from > > > which to > > > load the kernel. > > > > > > > Nope, that's not correct. GELI support was added to the boot and > > loader > > programs for both ufs and zfs in freebsd 12. You must set the geli > > '-g' > > option to be prompted for the passphrase while booting (this is > > separate from the '-b' flag that enables mounting the encrypted > > partition as the rootfs). You can use "geli configure -g" to turn > > on > > the flag on any existing geli partition. > > > > -- Ian > > Excellent - this will eliminate the need for me to run down the > foot-shooting that occurred in my update script since the unencrypted > kernel partition is no longer needed at all. That also significantly > reduces the attack surface on such a machine (although you could > still > tamper with the contents of freebsd-boot of course.) > > The "-g" flag I knew about from experience in putting 12 on my X1 > Carbon > (which works really well incidentally; the only issue I'm aware of is > that there's no 5Ghz WiFi support.) > One thing that is rather unfortunate... if you have multiple geli encrypted partitions that all have the same passphrase, you will be required to enter that passphrase twice while booting -- once in gpt[zfs]boot, then again during kernel startup when the rest of the drives/partitions get tasted by geom. This is because APIs within the boot process got changed to pass keys instead of the passphrase itself from one stage of booting to the next, and the fallout of that is the key for the rootfs is available to the kernel for mountroot, but the passphrase is not available to the system when geom is probing all the devices, so you get prompted for it again. -- Ian From owner-freebsd-stable@freebsd.org Sun Feb 10 18:06:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7248714D4B8D for ; Sun, 10 Feb 2019 18:06:58 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1.eu.mailhop.org (outbound1.eu.mailhop.org [52.28.251.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2FD989A00 for ; Sun, 10 Feb 2019 18:06:57 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1549821047; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=Br67rJ7gWV5oM4/YOlhPpSTyQcnGvNquldyegeg+FdZZMbapisTZYLAy6sJnwdLxQKtsmMnmMhZCP GTuHzdorNSISxQ5+3HdHapqLEfyeie2ZzmlRBr+Da1LeytYPnj4XfCqlNpS5dj8QS8XXQVQgAFliDN +QR1qUcCEKVPKBxV7do+DfeYaly52l1kWDC0LoRIAO4IRRWhEMaAMl89rbQ3Dtuhqh/9o94PQghcAl Xgj1JXf2R6M5CL39nbVcsguscWpuvbUcJFP+48QkWhUN9IGxuHRffg9HsM5Vxa4xhU+VQy//4vskeX Coh5XAteMOORqSTHv9OT1nvtqqPGPsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=fDxaKb5lAM+lX+rESb2Ntilx7EdA6xUXxW0kCb0y0h4=; b=oPGtrTbYW4iDv3TXFSibr1j0Z2wmjsv/OF/VhbPmLLxfEMSEB0GDnYS11mk0tbEby7yFOI06WeF8l IXUjQROnr1i6bOsD8w+lDhBOZTrdpgs3faD+x+Bbx7s1oQ9LvMX8DWy1kQH1GZRX/4VXg3wr4YA2xZ uheC+Rui40Sp2c4VtKGcih4u1F70mrf1s/tVbeqJ6FfbMfup0I28j9kgYDhUIZ8/RazynJLVB6jXML X/8mdX/MntF3fTinp2A4PXgMRA6rNEvXnmbeOK/C5EvUGrMe0DCEhyLPrKlu9SxWa2l3aC+6+DoDKx CGocwqa1q1dzlrxAodYLG3PDQ0LwYHg== ARC-Authentication-Results: i=1; outbound3.eu.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=fDxaKb5lAM+lX+rESb2Ntilx7EdA6xUXxW0kCb0y0h4=; b=qDHe0Z4WckpleVWdgxx7/HPB9WUwOemQVRKkeHCbXmINSqG9HyZFTMca1QkjYZfdiTlvcbvF/4eFk 29JukOonr+s48r9yYjYejHqbrFBjjsiep1UtTRd+bqn9sAjfhGcwXzFt9vpOtHXsp9vjR6JMPgRy66 vhH5dBMC/CcbCtc8blA6ogGMS8NY5OmHrI88P8Lj2BSwHz4vGUyGakQ49ZUlADowIeFFbJDI4E3dd8 4nyuCjnvNaU+7FsDCYg/2hVZvT6L5pRodMj4++/nsfRt1J2iCsUjDH9CfnmqbS1yg3uF+dylDmbvlP dlwz0LJ6PENvemI28sNgihWyhgTunNQ== X-MHO-RoutePath: aGlwcGll X-MHO-User: 62864090-2d5c-11e9-908b-352056dbf2de X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.eu.mailhop.org (Halon) with ESMTPSA id 62864090-2d5c-11e9-908b-352056dbf2de; Sun, 10 Feb 2019 17:50:44 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x1AHog78052648; Sun, 10 Feb 2019 10:50:42 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> Subject: Re: Fwd: Serious ZFS Bootcode Problem (GPT NON-UEFI) From: Ian Lepore To: Karl Denninger , freebsd-stable@freebsd.org Cc: Allan Jude Date: Sun, 10 Feb 2019 10:50:42 -0700 In-Reply-To: References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: E2FD989A00 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.955,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:52.28.0.0/16, country:US] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 18:06:58 -0000 On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: > On 2/10/2019 09:28, Allan Jude wrote: > > Are you sure it is non-UEFI? As the instructions you followed, > > overwriting da0p1 with gptzfsboot, will make quite a mess if that > > happens to be the EFI system partition, rather than the freebsd- > > boot > > partition. > > [...] > > BTW am I correct that gptzfsboot did *not* get the ability to read > geli-encrypted pools in 12.0? The UEFI loader does know how (which I'm > using on my laptop) but I was under the impression that for non-UEFI > systems you still needed the unencrypted boot partition from which to > load the kernel. > Nope, that's not correct. GELI support was added to the boot and loader programs for both ufs and zfs in freebsd 12. You must set the geli '-g' option to be prompted for the passphrase while booting (this is separate from the '-b' flag that enables mounting the encrypted partition as the rootfs). You can use "geli configure -g" to turn on the flag on any existing geli partition. -- Ian From owner-freebsd-stable@freebsd.org Sun Feb 10 18:36:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A12A614D5F0F for ; Sun, 10 Feb 2019 18:36:28 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BF6008AE56; Sun, 10 Feb 2019 18:36:27 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id B5D652110AF; Sun, 10 Feb 2019 13:35:56 -0500 (EST) Received: from [192.168.10.12] (D2.Denninger.Net [192.168.10.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 1D6DB24686A; Sun, 10 Feb 2019 12:35:56 -0600 (CST) Subject: Re: Geli prompts on gptzfsboot (Was:: Serious ZFS Bootcode Problem (GPT NON-UEFI -- RESOLVED) To: Ian Lepore Cc: freebsd-stable@freebsd.org References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Sun, 10 Feb 2019 12:35:55 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010004050002090800050409" X-Rspamd-Queue-Id: BF6008AE56 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.40 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; DMARC_NA(0.00)[denninger.net]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: px.denninger.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.93)[-0.925,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.27)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.11), asn: 14061(2.73), country: US(-0.07)]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 18:36:28 -0000 This is a cryptographically signed message in MIME format. --------------ms010004050002090800050409 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/10/2019 12:01, Ian Lepore wrote: > On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote: >> On 2/10/2019 11:50, Ian Lepore wrote: >>> On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: >>> >>>> [...] >>>> >>>> BTW am I correct that gptzfsboot did *not* get the ability to >>>> read >>>> geli-encrypted pools in 12.0? The UEFI loader does know how >>>> (which I'm >>>> using on my laptop) but I was under the impression that for non- >>>> UEFI >>>> systems you still needed the unencrypted boot partition from >>>> which to >>>> load the kernel. >>>> >>> Nope, that's not correct. GELI support was added to the boot and >>> loader >>> programs for both ufs and zfs in freebsd 12. You must set the geli >>> '-g'=20 >>> option to be prompted for the passphrase while booting (this is >>> separate from the '-b' flag that enables mounting the encrypted >>> partition as the rootfs). You can use "geli configure -g" to turn >>> on >>> the flag on any existing geli partition. >>> >>> -- Ian >> Excellent - this will eliminate the need for me to run down the >> foot-shooting that occurred in my update script since the unencrypted >> kernel partition is no longer needed at all. That also significantly >> reduces the attack surface on such a machine (although you could >> still >> tamper with the contents of freebsd-boot of course.) >> >> The "-g" flag I knew about from experience in putting 12 on my X1 >> Carbon >> (which works really well incidentally; the only issue I'm aware of is >> that there's no 5Ghz WiFi support.) >> > One thing that is rather unfortunate... if you have multiple geli > encrypted partitions that all have the same passphrase, you will be > required to enter that passphrase twice while booting -- once in > gpt[zfs]boot, then again during kernel startup when the rest of the > drives/partitions get tasted by geom. This is because APIs within the > boot process got changed to pass keys instead of the passphrase itself > from one stage of booting to the next, and the fallout of that is the > key for the rootfs is available to the kernel for mountroot, but the > passphrase is not available to the system when geom is probing all the > devices, so you get prompted for it again. > > -- Ian Let me see if I understand this before I do it then... :-) I have the following layout: 1. Two SSDs that contain the OS as a two-provider ZFS pool, which has "-b" set on both members; I get the "GELI Passphrase:" prompt from the loader and those two providers (along with encrypted swap) attach early in the boot process.=C2=A0 The same SSDs contain a mirrored non-encrypted= pool that has /boot (and only /boot) on it because previously you couldn't boot from an EFI-encrypted pool at all. Thus: [\u@NewFS /root]# gpart show da1 =3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0 468862061=C2=A0 da1=C2=A0= GPT=C2=A0 (224G) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 2014=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.0= M) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2048=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 1024=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (512K) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 3072=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 1024=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (512K) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0=C2=A0 20971520=C2=A0=C2=A0= =C2=A0 2=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (10G) =C2=A0=C2=A0 20975616=C2=A0 134217728=C2=A0=C2=A0=C2=A0 3=C2=A0 freebsd-s= wap=C2=A0 (64G) =C2=A0 155193344=C2=A0 313667584=C2=A0=C2=A0=C2=A0 4=C2=A0 freebsd-zfs=C2= =A0 (150G) =C2=A0 468860928=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1167=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 - free -=C2=A0 (584K) There is of course a "da2" that is identical.=C2=A0 The actual encrypted = root pool is on partition 4 with "-b" set at present.=C2=A0 I get prompted fro= m loader as a result after the unencrypted partition (#2) boots. 2. Multiple additional "user space" pools on a bunch of other disks. Right now #2 is using geli groups.=C2=A0 Prior to 12.0 they were handled using a custom /etc/rc.d script I wrote that did basically the same thing that geli groups does because all use the same passphrase and entering the same thing over and over on a boot was a pain in the butt.=C2= =A0 It prompted cleanly with no echo, took a password and then iterated over a list of devices attaching them one at a time.=C2=A0 That requirement is= now gone with geli groups, which is nice since mergemaster always complained about it being a "non-standard" thing; it *had* to go in /etc/rc.d and not in /usr/etc/rc.d else I couldn't get it to run early enough -- unfortunately. So if I remove the non-encrypted freebsd-zfs mirror that the system boots from in favor of setting "-g" on the root pool (both providers) gptzfsboot will find and prompt for the password to boot before loader gets invoked at all, much like the EFI loader does.=C2=A0 That's good.=C2= =A0 (My assumption is that the "-g" is sufficient; I don't need (or want) "bootme" set -- correct?) /However, /once the kernel boots somewhere in the mishmash of boot-time messages, and probably not where it's instantly obvious nor where it will halt the cascade display on the console, I'm going to get asked for that passphrase again?=C2=A0 I assume I want to remove 'geom_eli_passphrase_prompt=3D"YES"' from loader.conf as well -- or would= leaving it in there save me from the prompt that's hard to find in the cascade? Or, even better, would that situation of a double-prompt only apply if I had "-b" set on something /other than /the boot device pool vdevs (I don't -- those are handled by #2 for this exact reason.) --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010004050002090800050409 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEwMTgzNTU1 WjBPBgkqhkiG9w0BCQQxQgRATm5uYoTkaMef4IeaeOLYsNQUBHz1Nb9BywQA9nrmdsambcwI nyJNMjMXL+/FUpInVOmWT4C0qlyysr1ApdKl0zBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgA95qyGdJKQfwsDIHL/bHHKq7yS/UhBp/gh3c2nEWYvNk0Lq2ouAzP6Ro8Ic8CcwaeM Eu0WsRtPUguYKtqp9cYueY2M0DbOoz7gem9UQfzWuNJiAmycZY2rLUTRcMn1jAVlvM/gSd9A W6ILx5uHQr1PKVcr+tXDknVOeEfezjJWINxnF6AIAyZVQ6rh2zgc84KHcRRiXiqjZh69VHE0 b94We1KNMAtq2k0rhvlvWYG7reL6eMlI2NqXmLhayKvO7AsRWeopPGgl6qJzyIoxHmPZkX/a /xvvknAVAIAyc/lmgfHYBFI22mlTT0ME+zVDMWajEFVeOnKGBpPPmnxzOnVY8vWFJbiAaFPh JL75oD08cNesYEERskZB6i/xtiekiWAyQbWP1DTInsOahCASJTy7mQs2oXLg+oFqWBBHodwn r2X94yYItQ/xPiUwB2fhrl3j22IOdbnOWFC4zAkQ6UwREHPB8lAyjBp275cx5UzS+Hgz/DIo zETaG3m/82KaIhdRSAENytDlU4vxtUWMsCPAsGmrUZerFHX3e7+24mcRYZVNCofb4YI9XipI ui+vWO82VPNt7jaUeuRDXMO/ZRKPLy4b5M2eJtco6MO7A6opcxH9HYyyktyPT64BpQiT2Jt1 XCBUDisuzYaFix0DovEoglMLxRoNpzUeLFzJ/yJHDgAAAAAAAA== --------------ms010004050002090800050409-- From owner-freebsd-stable@freebsd.org Sun Feb 10 18:40:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C77414D6096 for ; Sun, 10 Feb 2019 18:40:39 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 61EFF8AFBB for ; Sun, 10 Feb 2019 18:40:37 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1549824000; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=GVid3O2oQ5cGNa9U/tnpd6++vY8NJSXkaRiLrEdPuSDYvcYWTcIXZfXQ963d0xmMv9AYMoeSpD8Y0 1mS3CFDxU/xz7/L7i0mpfd9dQB9M+uU1JFdVBHTX1o//L+J924SPCkgBOEYGKbAvkMCBUt2Yj+W6A/ oDrHJLWkpyVbTM4+lxdYwkK9sFPRdJygp4/PrScgCx51BDj2BiplB6q9FaOSAEkvMKJyB1tTaDObYs NfAi3zdJW6htk0hilXKnH54w4BP0OchtZ7BnGD3BFH6RFJKJhHYrErbRdpKHWoouiZktv+nnZx/I6A G4sJmJL5BbwrGNCK5gxb5HP8gvsapKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=AqQeNayZrcpJjzs1VpwudWYGefdIQW7bz4mgMsGMtoE=; b=f3O16CILlknOWmgwpZH0OWibZ30oP+dgHrHnp/CjXcJupUqSTYdFN2bW5m8U5Ttw9WFSrUDbCUrwr KxjKbjJQVTQMBmjE2lRS1qtnKMbqAfvMVsOUBKYOVjSxlgeRPsFWTrHQNvcRO6skCuXaIZ3jIQFV/e YD/dmc5QeLQrIvpA27BdaKMzg7Pn4AxXM83M7HDgq/vhxcLKJFZyKy6E/0MZ1Z0bRT+/G8i3reWQIS Gw7E092ZfEckgzuyCBQpErAiiqTZkSPudssniXYK/+vaz1wZAOOb3Oy5fZZZRquOHskt4L7RoXDuYx EvnXsFNmP3WKgFzJASXOguWuL4kJtmg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=AqQeNayZrcpJjzs1VpwudWYGefdIQW7bz4mgMsGMtoE=; b=tFdcgpMx6I18TBVZ8AwCUz69K7M0aTXWueZw20faM3F753JXY+VQUp8KHd7CYY36zb4PACFDB06B5 UZkXUJeExyoLGKYEGRly3DeqVxpBIXDK6EFCSuh6uAzUBcn8LgEZ3efgFUyVKaEgjdhgg0UTDpq2Rh zd3StMBL55zvcdeSiAYqFqiF9n6LuRZxUwelSHpvgs69RyznGL++phW/jOacmP1XmuuyoqED2TEu4f mbbJSpBi4VnG2EEqmeC0ncUM1YJDAVzoCckgU3vWvi23QtW2Tqf/qKs3V5vpPkIh/p5WpE0U3AMhSI WTU0bshD+098/jLTZJv3VZuQmUg0NXw== X-MHO-RoutePath: aGlwcGll X-MHO-User: 444b0273-2d63-11e9-9789-75353a1f43cf X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 444b0273-2d63-11e9-9789-75353a1f43cf; Sun, 10 Feb 2019 18:39:59 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x1AIeSQU052770; Sun, 10 Feb 2019 11:40:28 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org> Subject: Re: Geli prompts on gptzfsboot (Was:: Serious ZFS Bootcode Problem (GPT NON-UEFI -- RESOLVED) From: Ian Lepore To: Karl Denninger Cc: freebsd-stable@freebsd.org Date: Sun, 10 Feb 2019 11:40:28 -0700 In-Reply-To: References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 61EFF8AFBB X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 18:40:39 -0000 On Sun, 2019-02-10 at 12:35 -0600, Karl Denninger wrote: > On 2/10/2019 12:01, Ian Lepore wrote: > > On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote: > > > On 2/10/2019 11:50, Ian Lepore wrote: > > > > On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: > > > > > > > > > [...] > > > > > > > > > > BTW am I correct that gptzfsboot did *not* get the ability to > > > > > read > > > > > geli-encrypted pools in 12.0? The UEFI loader does know how > > > > > (which I'm > > > > > using on my laptop) but I was under the impression that for > > > > > non- > > > > > UEFI > > > > > systems you still needed the unencrypted boot partition from > > > > > which to > > > > > load the kernel. > > > > > > > > > > > > > Nope, that's not correct. GELI support was added to the boot > > > > and > > > > loader > > > > programs for both ufs and zfs in freebsd 12. You must set the > > > > geli > > > > '-g' > > > > option to be prompted for the passphrase while booting (this is > > > > separate from the '-b' flag that enables mounting the encrypted > > > > partition as the rootfs). You can use "geli configure -g" to > > > > turn > > > > on > > > > the flag on any existing geli partition. > > > > > > > > -- Ian > > > > > > Excellent - this will eliminate the need for me to run down the > > > foot-shooting that occurred in my update script since the > > > unencrypted > > > kernel partition is no longer needed at all. That also > > > significantly > > > reduces the attack surface on such a machine (although you could > > > still > > > tamper with the contents of freebsd-boot of course.) > > > > > > The "-g" flag I knew about from experience in putting 12 on my X1 > > > Carbon > > > (which works really well incidentally; the only issue I'm aware > > > of is > > > that there's no 5Ghz WiFi support.) > > > > > > > One thing that is rather unfortunate... if you have multiple geli > > encrypted partitions that all have the same passphrase, you will be > > required to enter that passphrase twice while booting -- once in > > gpt[zfs]boot, then again during kernel startup when the rest of the > > drives/partitions get tasted by geom. This is because APIs within > > the > > boot process got changed to pass keys instead of the passphrase > > itself > > from one stage of booting to the next, and the fallout of that is > > the > > key for the rootfs is available to the kernel for mountroot, but > > the > > passphrase is not available to the system when geom is probing all > > the > > devices, so you get prompted for it again. > > > > -- Ian > > Let me see if I understand this before I do it then... :-) > > I have the following layout: > > 1. Two SSDs that contain the OS as a two-provider ZFS pool, which has > "-b" set on both members; I get the "GELI Passphrase:" prompt from > the > loader and those two providers (along with encrypted swap) attach > early > in the boot process. The same SSDs contain a mirrored non-encrypted > pool that has /boot (and only /boot) on it because previously you > couldn't boot from an EFI-encrypted pool at all. > > Thus: > > [\u@NewFS /root]# gpart show da1 > => 34 468862061 da1 GPT (224G) > 34 2014 - free - (1.0M) > 2048 1024 1 freebsd-boot (512K) > 3072 1024 - free - (512K) > 4096 20971520 2 freebsd-zfs [bootme] (10G) > 20975616 134217728 3 freebsd-swap (64G) > 155193344 313667584 4 freebsd-zfs (150G) > 468860928 1167 - free - (584K) > > There is of course a "da2" that is identical. The actual encrypted > root > pool is on partition 4 with "-b" set at present. I get prompted from > loader as a result after the unencrypted partition (#2) boots. > > 2. Multiple additional "user space" pools on a bunch of other disks. > > Right now #2 is using geli groups. Prior to 12.0 they were handled > using a custom /etc/rc.d script I wrote that did basically the same > thing that geli groups does because all use the same passphrase and > entering the same thing over and over on a boot was a pain in the > butt. > It prompted cleanly with no echo, took a password and then iterated > over > a list of devices attaching them one at a time. That requirement is > now > gone with geli groups, which is nice since mergemaster always > complained > about it being a "non-standard" thing; it *had* to go in /etc/rc.d > and > not in /usr/etc/rc.d else I couldn't get it to run early enough -- > unfortunately. > > So if I remove the non-encrypted freebsd-zfs mirror that the system > boots from in favor of setting "-g" on the root pool (both providers) > gptzfsboot will find and prompt for the password to boot before > loader > gets invoked at all, much like the EFI loader does. That's good. > (My > assumption is that the "-g" is sufficient; I don't need (or want) > "bootme" set -- correct?) > > /However, /once the kernel boots somewhere in the mishmash of boot- > time > messages, and probably not where it's instantly obvious nor where it > will halt the cascade display on the console, I'm going to get asked > for > that passphrase again? I assume I want to remove > 'geom_eli_passphrase_prompt="YES"' from loader.conf as well -- or > would > leaving it in there save me from the prompt that's hard to find in > the > cascade? > > Or, even better, would that situation of a double-prompt only apply > if I > had "-b" set on something /other than /the boot device pool vdevs (I > don't -- those are handled by #2 for this exact reason.) > I think at this point I have to ease out of the conversation, because I know almost nothing about zfs, despite having somehow managed to add geli support to the zfs code in loader. I did so without understanding zfs in any way, because I added the support at a more generic "disk drive support" layer in loader, and did all my testing using automated scripts Alan and Warner created to test zfs booting using qemu. -- Ian From owner-freebsd-stable@freebsd.org Sun Feb 10 19:43:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E062114D9355 for ; Sun, 10 Feb 2019 19:43:47 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91A2D8DF9E for ; Sun, 10 Feb 2019 19:43:46 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 5E3322110BA for ; Sun, 10 Feb 2019 14:43:15 -0500 (EST) Received: from [192.168.10.12] (D2.Denninger.Net [192.168.10.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id C90BF660B1 for ; Sun, 10 Feb 2019 13:43:14 -0600 (CST) Subject: Re: Geli prompts on gptzfsboot (Was:: Serious ZFS Bootcode Problem (GPT NON-UEFI -- RESOLVED) To: freebsd-stable@freebsd.org References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Sun, 10 Feb 2019 13:43:13 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020808030001000309060003" X-Rspamd-Queue-Id: 91A2D8DF9E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.93)[-0.929,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.28)[ip: (-9.89), ipnet: 104.236.64.0/18(-4.19), asn: 14061(2.73), country: US(-0.07)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 19:43:48 -0000 This is a cryptographically signed message in MIME format. --------------ms020808030001000309060003 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/10/2019 12:40, Ian Lepore wrote: > On Sun, 2019-02-10 at 12:35 -0600, Karl Denninger wrote: >> On 2/10/2019 12:01, Ian Lepore wrote: >>> On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote: >>>> On 2/10/2019 11:50, Ian Lepore wrote: >>>>> On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: >>>>> >>>>>> [...] >>>>>> >>>>>> BTW am I correct that gptzfsboot did *not* get the ability to >>>>>> read >>>>>> geli-encrypted pools in 12.0? The UEFI loader does know how >>>>>> (which I'm >>>>>> using on my laptop) but I was under the impression that for >>>>>> non- >>>>>> UEFI >>>>>> systems you still needed the unencrypted boot partition from >>>>>> which to >>>>>> load the kernel. >>>>>> >>>>> Nope, that's not correct. GELI support was added to the boot >>>>> and >>>>> loader >>>>> programs for both ufs and zfs in freebsd 12. You must set the >>>>> geli >>>>> '-g'=20 >>>>> option to be prompted for the passphrase while booting (this is >>>>> separate from the '-b' flag that enables mounting the encrypted >>>>> partition as the rootfs). You can use "geli configure -g" to >>>>> turn >>>>> on >>>>> the flag on any existing geli partition. >>>>> >>>>> -- Ian >>>> Excellent - this will eliminate the need for me to run down the >>>> foot-shooting that occurred in my update script since the >>>> unencrypted >>>> kernel partition is no longer needed at all. That also >>>> significantly >>>> reduces the attack surface on such a machine (although you could >>>> still >>>> tamper with the contents of freebsd-boot of course.) >>>> >>>> The "-g" flag I knew about from experience in putting 12 on my X1 >>>> Carbon >>>> (which works really well incidentally; the only issue I'm aware >>>> of is >>>> that there's no 5Ghz WiFi support.) >>>> >>> One thing that is rather unfortunate... if you have multiple geli >>> encrypted partitions that all have the same passphrase, you will be >>> required to enter that passphrase twice while booting -- once in >>> gpt[zfs]boot, then again during kernel startup when the rest of the >>> drives/partitions get tasted by geom. This is because APIs within >>> the >>> boot process got changed to pass keys instead of the passphrase >>> itself >>> from one stage of booting to the next, and the fallout of that is >>> the >>> key for the rootfs is available to the kernel for mountroot, but >>> the >>> passphrase is not available to the system when geom is probing all >>> the >>> devices, so you get prompted for it again. >>> >>> -- Ian >> Let me see if I understand this before I do it then... :-) >> >> I have the following layout: >> >> 1. Two SSDs that contain the OS as a two-provider ZFS pool, which has >> "-b" set on both members; I get the "GELI Passphrase:" prompt from >> the >> loader and those two providers (along with encrypted swap) attach >> early >> in the boot process. The same SSDs contain a mirrored non-encrypted >> pool that has /boot (and only /boot) on it because previously you >> couldn't boot from an EFI-encrypted pool at all. >> >> Thus: >> >> [\u@NewFS /root]# gpart show da1 >> =3D> 34 468862061 da1 GPT (224G) >> 34 2014 - free - (1.0M) >> 2048 1024 1 freebsd-boot (512K) >> 3072 1024 - free - (512K) >> 4096 20971520 2 freebsd-zfs [bootme] (10G) >> 20975616 134217728 3 freebsd-swap (64G) >> 155193344 313667584 4 freebsd-zfs (150G) >> 468860928 1167 - free - (584K) >> >> There is of course a "da2" that is identical. The actual encrypted >> root >> pool is on partition 4 with "-b" set at present. I get prompted from >> loader as a result after the unencrypted partition (#2) boots. >> >> 2. Multiple additional "user space" pools on a bunch of other disks. >> >> Right now #2 is using geli groups. Prior to 12.0 they were handled >> using a custom /etc/rc.d script I wrote that did basically the same >> thing that geli groups does because all use the same passphrase and >> entering the same thing over and over on a boot was a pain in the >> butt.=20 >> It prompted cleanly with no echo, took a password and then iterated >> over >> a list of devices attaching them one at a time. That requirement is >> now >> gone with geli groups, which is nice since mergemaster always >> complained >> about it being a "non-standard" thing; it *had* to go in /etc/rc.d >> and >> not in /usr/etc/rc.d else I couldn't get it to run early enough -- >> unfortunately. >> >> So if I remove the non-encrypted freebsd-zfs mirror that the system >> boots from in favor of setting "-g" on the root pool (both providers) >> gptzfsboot will find and prompt for the password to boot before >> loader >> gets invoked at all, much like the EFI loader does. That's good.=20 >> (My >> assumption is that the "-g" is sufficient; I don't need (or want) >> "bootme" set -- correct?) >> >> /However, /once the kernel boots somewhere in the mishmash of boot- >> time >> messages, and probably not where it's instantly obvious nor where it >> will halt the cascade display on the console, I'm going to get asked >> for >> that passphrase again? I assume I want to remove >> 'geom_eli_passphrase_prompt=3D"YES"' from loader.conf as well -- or >> would >> leaving it in there save me from the prompt that's hard to find in >> the >> cascade? >> >> Or, even better, would that situation of a double-prompt only apply >> if I >> had "-b" set on something /other than /the boot device pool vdevs (I >> don't -- those are handled by #2 for this exact reason.) >> > I think at this point I have to ease out of the conversation, because I= > know almost nothing about zfs, despite having somehow managed to add > geli support to the zfs code in loader. I did so without understanding > zfs in any way, because I added the support at a more generic "disk > drive support" layer in loader, and did all my testing using automated > scripts Alan and Warner created to test zfs booting using qemu. > > -- Ian I can confirm that this boots and comes up cleanly without re-prompting for the boot pool password. The machines I have in the field in this config, during the next upgrade cycle, are going to get set up this way.=C2=A0 When it makes sense to rep= lace these with UEFI boards (likely when Coffee Lake Xeons and Mobos that can handle them get a bit more reasonable and start showing up with IPMI/kvm ports) I'll likely start getting rid of these older devices simply on the performance-for-power equation, but these are likely to be out there for me, anyway, for the next few years. In short very nice work -- and thank you! --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020808030001000309060003 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEwMTk0MzEz WjBPBgkqhkiG9w0BCQQxQgRAQtuMDejq28bF5KWo9owoPCbJy2ROQ5Jtm9h3WzG290BuwPLB Ko2Ns0lMmroC73wkYpSAb5b2k/rw9y6Uemb6KjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgAZqsF1t3F3WZqerUyFjwaK3DSxvVzc6dBcc33fkwWDcWPdV8O4wMsi0tszswJB7smA NhEfJihmxQUg+Qd8cptcWnB6p7iN25VTN2siB+Kc61VW8GXhkwlJg6KadqW6AVGUwGiQC1A6 kSpLmvp3vnLZMwR9j1GWxOFGSBVTirWWuUksRTMsiVJNz9ABYuskBZuk/1VLWioplQeA9yAy eQKKky7s2oJfwwVazVFs1tsqY9MLJPDAUB8fmGMpbCIC3rCdkNGG/m7SAGqu1Vf7gBgQGNQZ Wfv5a6LGFISENYWg7iwtF5F6YVMxfKQ994xhwzFOxfywEWIyH63F0Ln35S2ZW5TuQtzA/Y5B 6+zykTUxr9j0KX+b44tSehbrLcpLXm5d2x5ns1aOErnvQYHq+GTmuMekws1ZGfhdtmALWFMp wwfSePKMVmSpUpbky0vyTvKn0AntlzvFr5y4Bg05B5+hvy9PuZuPu0/A0Z1W7Ce2/cloY5MT n8ha/PVJoWakkGyQhiEbL14cwv9ymK5lZB7OCR3TER9QiO8MLMvFzIEfnMy28PVLhEaYpYyP URlsjGJSKUMNIcCkN+zMZh6qmUIIlY+AGziZDwzEg5Kbc0tSZ86aBlOdqq4ldKudRssTkld6 p7I3uiV/LcZusUdAEeT30DtzsuMMEba/2n7Iic5j6QAAAAAAAA== --------------ms020808030001000309060003-- From owner-freebsd-stable@freebsd.org Sun Feb 10 21:00:12 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6BA714DBAA3 for ; Sun, 10 Feb 2019 21:00:12 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3B2316AA6C for ; Sun, 10 Feb 2019 21:00:12 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id F05F414DBA9A; Sun, 10 Feb 2019 21:00:11 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE39514DBA94 for ; Sun, 10 Feb 2019 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A05AD6AA64 for ; Sun, 10 Feb 2019 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id EBF3715F34 for ; Sun, 10 Feb 2019 21:00:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1AL0AWZ039681 for ; Sun, 10 Feb 2019 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1AL0Asx039674 for stable@FreeBSD.org; Sun, 10 Feb 2019 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201902102100.x1AL0Asx039674@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: stable@FreeBSD.org Subject: Problem reports for stable@FreeBSD.org that need special attention Date: Sun, 10 Feb 2019 21:00:10 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 21:00:12 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 230620 | "install -d" issue 1 problems total for which you should take action. From owner-freebsd-stable@freebsd.org Sun Feb 10 21:57:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB7AF14DE8DE for ; Sun, 10 Feb 2019 21:57:03 +0000 (UTC) (envelope-from jurij.kovacic@ocpea.com) Received: from mail-it1-x12f.google.com (mail-it1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3430B6E30B for ; Sun, 10 Feb 2019 21:57:01 +0000 (UTC) (envelope-from jurij.kovacic@ocpea.com) Received: by mail-it1-x12f.google.com with SMTP id z20so21651758itc.3 for ; Sun, 10 Feb 2019 13:57:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=W+3wDM/jiS2MbcEiQhv1L8e2yObictqd3sphSkh6ygo=; b=YA70ONyN7X4uKcyrqmiFeOwjm6CG4nP4ZIsg43JOiBIgritVjs6SVwMDg+esr1A/fm URG6hcxKpK5yjdFKK3hTPveO+Risvm2lTu62rDpcTmuBEk29+UFM/k8K7W0i/5b44/1p 3zglbnDozL7y5VyVaqOAKsvKXKUztWURt+07ePKYRzl8Ma5b07hLmlzdmi2YiypYzYgW Exv9FGo2Wk0YzuoUOigyZZJxfCLfYyQ9YBuxKL8FtHN7pBuVsyI+sRQ5T/ToikgHVq5T /f+63ttHSglu6xvVryDzrjjO3zPMQJjL9xVjhrtY0wr+6Q5BoBs60LYn55UTKxABb9At 9s8g== X-Gm-Message-State: AHQUAuZNlEZHDH9q7OQrLvCaXoDwe7mGV13DkUVtbprRUNx0Fuqg1iei liIkCsl8tP0JEfEgilD1K+sgvctaypiqtAoucEeCBg0ragE= X-Google-Smtp-Source: AHgI3Ia69SVoUoGN3hJ4cfchiD8G5c1EfKPLDNhU4iELvhxdbE+mbvwgWWIe1KVM8OwI00/O7S2TmCqCnhf8q3taqqw= X-Received: by 2002:a24:2c50:: with SMTP id i77mr3707154iti.131.1549835819475; Sun, 10 Feb 2019 13:56:59 -0800 (PST) MIME-Version: 1.0 From: =?UTF-8?B?SnVyaWogS292YcSNacSN?= Date: Sun, 10 Feb 2019 22:56:48 +0100 Message-ID: Subject: Repeated kernel panic on 11.2-RELEASE-p7 To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 3430B6E30B X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.70 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ocpea.com:s=google]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-2.72)[ip: (-9.09), ipnet: 2607:f8b0::/32(-2.47), asn: 15169(-1.95), country: US(-0.07)]; DKIM_TRACE(0.00)[ocpea.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,alt4.aspmx.l.google.com,alt3.aspmx.l.google.com,alt2.aspmx.l.google.com,aspmx.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[f.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.97)[-0.974,0]; DMARC_POLICY_ALLOW(-0.50)[ocpea.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 21:57:04 -0000 Dear list, After some time, I have again experienced a kernel panic on a (physical) server, running Freebsd 11.2-RELEASE-p7 with custom/debug kernel, ZFS root. Fatal trap 9: general protection fault while in kernel mode cpuid = 2; apic id = 02 instruction pointer = 0x20:0xffffffff82299013 stack pointer = 0x28:0xfffffe0352893ad0 frame pointer = 0x28:0xfffffe0352893b10 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 9 (dbuf_evict_thread) trap number = 9 panic: general protection fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80b3d567 at kdb_backtrace+0x67 #1 0xffffffff80af6b07 at vpanic+0x177 #2 0xffffffff80af6983 at panic+0x43 #3 0xffffffff80f77fdf at trap_fatal+0x35f #4 0xffffffff80f7759e at trap+0x5e #5 0xffffffff80f5807c at calltrap+0x8 #6 0xffffffff8229c049 at dbuf_evict_one+0xe9 #7 0xffffffff82297a15 at dbuf_evict_thread+0x1a5 #8 0xffffffff80aba083 at fork_exit+0x83 #9 0xffffffff80f58f9e at fork_trampoline+0xe Uptime: 20d6h13m55s Dumping 2593 out of 12248 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% I have used "crashinfo" utility to generate the text file which is available at this URL: http://www.ocpea.com/dump/core-3.txt. All advice is deeply appreciated as this is a production server. :) Kind regards, Jurij From owner-freebsd-stable@freebsd.org Sun Feb 10 22:02:33 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4CAA14DECAE for ; Sun, 10 Feb 2019 22:02:33 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F205E6E874 for ; Sun, 10 Feb 2019 22:02:32 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 977652110AF for ; Sun, 10 Feb 2019 17:02:01 -0500 (EST) Received: from [192.168.10.12] (D2.Denninger.Net [192.168.10.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id E6AC1B19AA for ; Sun, 10 Feb 2019 16:02:00 -0600 (CST) To: freebsd-stable@freebsd.org From: Karl Denninger Subject: Oddball error from "batch" Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Sun, 10 Feb 2019 16:01:59 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms060906000209080205030907" X-Rspamd-Queue-Id: F205E6E874 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.09 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.59)[-0.586,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.29)[ip: (-9.89), ipnet: 104.236.64.0/18(-4.22), asn: 14061(2.72), country: US(-0.07)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 22:02:34 -0000 This is a cryptographically signed message in MIME format. --------------ms060906000209080205030907 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Note -- working fine on 11.1 and 11.2, upgraded machine to 12.0-STABLE and everything is ok that I'm aware of *except*..... # batch who df ^D Job 170 will be executed using /bin/sh Then the time comes and... no output is emailed to me. In the cron log file I find: Feb 10 16:00:00 NewFS atrun[65142]: cannot open input file E000aa018a24c3: No such file or directory Note that scheduled cron jobs are running as expected, and the permissions on /var/at are correct (match exactly my 11. 1 and 11.2 boxes), and in addition of looking BEFORE the job runs the named job number IS THERE. [\u@NewFS /var/at/jobs]# ls -al total 13 drwxr-xr-x=C2=A0 2 daemon=C2=A0 wheel=C2=A0=C2=A0=C2=A0 5 Feb 10 15:55 . drwxr-xr-x=C2=A0 4 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 5 Oct=C2= =A0 8=C2=A0 2013 .. -rw-r--r--=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 6 Feb 1= 0 15:55 .SEQ -rw-------=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 0 Jul=C2= =A0 5=C2=A0 2008 .lockfile -rwx------=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0 615 Feb 10 15:55 E0= 00aa018a24c3 After the error.... the file isn't there.=C2=A0 It was removed (as one wo= uld expect when the job is complete.) What the blankety-blank?! --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms060906000209080205030907 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEwMjIwMTU5 WjBPBgkqhkiG9w0BCQQxQgRAKJnpLV1tEufjVE5L7gF/gWBjhevxY9n33RXhgWIYmvpAScB0 chFWIcYWZH+8YY7a9fiHJUNGW91PCVjasLE/LTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCXzc7BGbI4zMUf9wO9YbTk1vs1ovBpTtvGTdwIJR+AusAZw0brpC5Y/KuxQjlOyu33 Le3u/zsNi+2kD4aDIqqgoHqvix1S+ImiwqDkjgeFKxWRxoiV5vr6zqz0gTi5jrAGVPAslVQA n/yfQBvhAigK8Mmnkb/+gauL6RLU3IsQwDJQqvEDdeHbSHm2ebCCpuaPwtj9N2oWj20BqAxE 2Grph1HKEekOKOLz7Q48YNIkyYmxmFEq9JOA/Z2uRmTxwWO+o69oaYQ9RWu3AmNJktdLsT7t lvWC8uxQNJy2NOFYzlBF6IcmEAgceyLFOIyg/Nk+ooxiy2oh46pJeLj+82aQgHkVMZ4a+7C+ Ok4mJE7cS7qsDVG2w7V8TfwrfS4yBlBNIO5zcnEJNBvyJ6SvCYGtND/+GnErQXDe4zJbOoGB gXGP/njUEusfVnmg76GaulmgwlUhIgEX4KDEroVrFRe0p57qPs6bMRLi8MDKbox6HuTJ3x/4 dPFZQ0OkB/KSCTW7oH0B5LYRbrNWOMdfW+WRNd/F9MFi8E31c9VTebTyYMLTlYO9WrgMepY1 JTlDonGJ44SWFHoBLRTxhgq6mVRu+tdzUsnYNOx9npOChov4cyWwsQi36DG6f4xaxe4FhVYP cCETCi6j1PSM+KEXmlEUHLzySy4QhwFbXqd4CUwBHAAAAAAAAA== --------------ms060906000209080205030907-- From owner-freebsd-stable@freebsd.org Mon Feb 11 05:48:31 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB0B714E9B27 for ; Mon, 11 Feb 2019 05:48:30 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E9CB84DF1 for ; Mon, 11 Feb 2019 05:48:29 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 558FC2110AF for ; Mon, 11 Feb 2019 00:47:52 -0500 (EST) Received: from [192.168.10.13] (D3.Denninger.Net [192.168.10.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id AB251B2BDF for ; Sun, 10 Feb 2019 23:47:50 -0600 (CST) Subject: Re: Oddball error from "batch" To: freebsd-stable@freebsd.org References: From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <762222d2-42bd-1a01-30c7-d6e52833bb14@denninger.net> Date: Sun, 10 Feb 2019 23:47:49 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms040609070207010106010309" X-Rspamd-Queue-Id: 4E9CB84DF1 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.31 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[px.denninger.net]; NEURAL_HAM_SHORT(-0.80)[-0.801,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.30)[ip: (-9.89), ipnet: 104.236.64.0/18(-4.26), asn: 14061(2.72), country: US(-0.07)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 05:48:31 -0000 This is a cryptographically signed message in MIME format. --------------ms040609070207010106010309 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/10/2019 16:01, Karl Denninger wrote: > Note -- working fine on 11.1 and 11.2, upgraded machine to 12.0-STABLE > and everything is ok that I'm aware of *except*..... > > # batch > who > df > ^D > > Job 170 will be executed using /bin/sh > > Then the time comes and... no output is emailed to me. > > In the cron log file I find: > > Feb 10 16:00:00 NewFS atrun[65142]: cannot open input file > E000aa018a24c3: No such file or directory > > Note that scheduled cron jobs are running as expected, and the > permissions on /var/at are correct (match exactly my 11. 1 and 11.2 > boxes), and in addition of looking BEFORE the job runs the named job > number IS THERE. > > [\u@NewFS /var/at/jobs]# ls -al > total 13 > drwxr-xr-x=C2=A0 2 daemon=C2=A0 wheel=C2=A0=C2=A0=C2=A0 5 Feb 10 15:55 = =2E > drwxr-xr-x=C2=A0 4 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 5 Oct= =C2=A0 8=C2=A0 2013 .. > -rw-r--r--=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 6 Feb= 10 15:55 .SEQ > -rw-------=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0=C2=A0=C2=A0 0 Jul= =C2=A0 5=C2=A0 2008 .lockfile > -rwx------=C2=A0 1 root=C2=A0=C2=A0=C2=A0 wheel=C2=A0 615 Feb 10 15:55 = E000aa018a24c3 > > After the error.... the file isn't there.=C2=A0 It was removed (as one = would > expect when the job is complete.) > > What the blankety-blank?! Turns out it's a nasty race in the atrun code.... I have no idea why this hasn't bit the living daylights out of lots of people before, but it's sure biting me! https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235657 Includes a proposed fix... :) --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040609070207010106010309 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjExMDU0NzQ5 WjBPBgkqhkiG9w0BCQQxQgRAlYfVeoDtjsYF0PCugjbYvG3rAD9hOst633JI3LSYkdBIxbgt 4jHMB0kD3SEwK1mAQNDmgs1VqJtz6mABiKB+9jBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgClfHNxSeeGDNl/qDekczlSz9+Jwa+ZNAjubppit0tJkABXAuzs38DidG5RfNaaPdgP fgtudX7NmkLUCcFgSnudysfZL4snr1UO6q+7yfQth7ulva7oXf4WLtxdRTSsMT1/R4tRG2gI mkAn6DzxiKIc7xb20SmycTKTWrYwlnPgkzn8cm2CwUFrkV4skerGBCvzhHJTUnR71cmL4W9m tddMI/w/eYitWT4VewyLmeDbrsWNkjzXdDV3lOHSGJTSiESF03RDj1LDb/0udIEe6ubn29UV Ws/1xz/fu3aXHBIpZEXQmqCKvBSVB+2T3wCMD/7ZZdtfZfVY7w4TMLEKDVtBVqpXOaChhU7U gueKO4TRCHAB+FXKaGri96ICEICoAjW8AwJFXHcyMdJveBYbpJvTMa9wSCa/JBBmvHgA5NOJ wjd5LTPF10aCKYYdVcNn7aIX0MoW1Io/hAYa7IJBA3KWqkdOiyDMpAuf/c2Yfpy1cfVP82Vi ikcdVymGsbhDAKI4nAjkD5SHfmOaKRamdI4LKiXEMmIjRZI45ZxEV1ilE99QVjnWaVtEkECf 3+IpXDkJ5tlpFkdGDe9z0xF5YaJmstRHGTSu/9mBV3bFKNetKPlOcsJ7b+j9HtMkhnjAh9VQ ljQs1Mrf3c8+/ai5nY06DntgsbEEaG89b4B6E+/RyQAAAAAAAA== --------------ms040609070207010106010309-- From owner-freebsd-stable@freebsd.org Mon Feb 11 10:19:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA38014D155B for ; Mon, 11 Feb 2019 10:19:25 +0000 (UTC) (envelope-from list_freebsd@bluerosetech.com) Received: from echo.brtsvcs.net (echo.brtsvcs.net [208.111.40.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DC0E8D7A3 for ; Mon, 11 Feb 2019 10:19:24 +0000 (UTC) (envelope-from list_freebsd@bluerosetech.com) Received: from chombo.houseloki.net (chombo [IPv6:2601:1c2:1402:1770:ae1f:6bff:fe6b:9e1c]) by echo.brtsvcs.net (Postfix) with ESMTPS id 9F0A638D14; Mon, 11 Feb 2019 02:19:16 -0800 (PST) Received: from [10.26.25.66] (leon.pilgrimaccounting.com [10.26.25.66]) by chombo.houseloki.net (Postfix) with ESMTPSA id CAB3BC7F; Mon, 11 Feb 2019 02:19:15 -0800 (PST) Subject: Re: Serious ZFS Bootcode Problem (GPT NON-UEFI) To: Karl Denninger , freebsd-stable@freebsd.org References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> From: Mel Pilgrim Message-ID: Date: Mon, 11 Feb 2019 02:19:18 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DC0E8D7A3 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of list_freebsd@bluerosetech.com designates 208.111.40.118 as permitted sender) smtp.mailfrom=list_freebsd@bluerosetech.com X-Spamd-Result: default: False [-4.08 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[bluerosetech.com]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[echo.brtsvcs.net,foxtrot.brtsvcs.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; IP_SCORE(-0.80)[asn: 36236(-3.90), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:36236, ipnet:208.111.40.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 10:19:26 -0000 On 02/09/2019 14:30, Karl Denninger wrote: > FreeBSD 12.0-STABLE r343809 > > After upgrading to this (without material incident) zfs was telling me > that the pools could be upgraded (this machine was running 11.1, then 11.2.) > > I did so, /and put the new bootcode on with gpart bootcode -b /boot/pmbr > -p /boot/gptzfsboot -i .... da... /on both of the candidate (mirrored > ZFS boot disk) devices, in the correct partition. > > Then I rebooted to test and..... /could not find the zsboot pool > containing the kernel./ > > I booted the rescue image off my SD and checked -- the copy of > gptzfsboot that I put on the boot partition is exactly identical to the > one on the rescue image SD. > > Then, to be /absolutely sure /I wasn't going insane I grabbed the > mini-memstick img for 12-RELEASE and tried THAT copy of gptzfsboot. > > /Nope; that won't boot either!/ > > Fortunately I had a spare drive slot so I stuck in a piece of spinning > rust, gpart'ed THAT with an old-style UFS boot filesystem, wrote > bootcode on that, mounted the ZFS "zsboot" filesystem and copied it > over.  That boots fine (of course) and mounts the root pool, and off it > goes. > > I'm going to blow away the entire /usr/obj tree and rebuild the kernel > to see if that gets me anything that's more-sane, but right now this > looks pretty bad. > > BTW just to be absolutely sure I blew away the entire /usr/obj directory > and rebuilt -- same size and checksum on the binary that I have > installed, so..... > > Not sure what's going on here -- did something get moved? I smashed my head against the wall for days with a very similar-sounding problem: pure ZFS with a GELI root and separate /boot pool that would not import the /boot pool at boot, resulting in the kernel not having the keys to attach the GELI+ZFS root. That configuration needs some extra bits in loader.conf so that zpool.cache and the GELI keys get loaded for the kernel by the loader. This loads the zpool.cache into the kernel so it imports everything before /etc/rc.d/zfs can run (the case where you have a ZFS /boot that isn't imported after a reboot: zpool_cache_load="YES" zpool_cache_name="/boot/zfs/zpool.cache" zpool_cache_type="/boot/zfs/zpool.cache" Run geli init with -b so the providers are flagged for attachment at boot (instead of by /etc/rc.d/geli), then add this for every GELI provider you want the kernel to attach before starting the userland: geli_FOO_keyfile0_load="YES" geli_FOO_keyfile0_name="/boot/path/to/key" geli_FOO_keyfile0_type="devicename:geli_keyfile0" FOO can be any alphanumeric string, and needs to be consistent for all three lines and unique per device. The "devicename" is gpt/BAR for a device with a GPT label of BAR. It can also be the unlabeled device (e.g., da0p3), but using GPT labels is recommended because it makes the keys follow a device renumber. For example, my GELI+ZFS root is a mirror of partitions with nvmezfs0 and nvmezfs1 GPT labels, so I have in my loader.conf: geli_nvmezfs0_keyfile0_load="YES" geli_nvmezfs0_keyfile0_name="/boot/gelikeys/nvmezfs0.key" geli_nvmezfs0_keyfile0_type="gpt/nvmezfs0:geli_keyfile0" geli_nvmezfs1_keyfile0_load="YES" geli_nvmezfs1_keyfile0_name="/boot/gelikeys/nvmezfs1.key" geli_nvmezfs1_keyfile0_type="gpt/nvmezfs1:geli_keyfile0" If you use GPT labels, you can safely ignore the "GEOM_ELI: Found no key files in loader.conf for DEVICE" messages where DEVICE is the unlabeled device--the GELI module doesn't currently recognize that the unlabeled and labeled devices are the same provider. This doesn't appear to be documented in the Handbook or any man pages that I could find. The zpool_cache_load trick is mentioned in a FreeBSD wiki page[1], and the geli_* config is pulled from the zfsboot script used by bsdinstall to install a pure-ZFS system with GELI root. I'm not sure if this is exactly your problem, but maybe it helps? 1: https://wiki.freebsd.org/MasonLoringBliss/UEFIandZFSandGELIbyHAND From owner-freebsd-stable@freebsd.org Mon Feb 11 19:18:11 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 318A714E1354 for ; Mon, 11 Feb 2019 19:18:11 +0000 (UTC) (envelope-from leslie@b2bglobedata.com) Received: from NEW-02-4.privateemail.com (new-02-4.privateemail.com [198.54.122.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "privateemail.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 982AF8645B for ; Mon, 11 Feb 2019 19:18:09 +0000 (UTC) (envelope-from leslie@b2bglobedata.com) Received: from NEW-02-1.privateemail.com (unknown [10.20.150.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by NEW-02-4.privateemail.com (Postfix) with ESMTPS id 19F5A806A8 for ; Mon, 11 Feb 2019 19:18:02 +0000 (UTC) Received: from MTA-08-1.privateemail.com (unknown [10.20.147.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by NEW-02.privateemail.com (Postfix) with ESMTPS id 137E4603D2 for ; Mon, 11 Feb 2019 19:18:02 +0000 (UTC) Received: from MTA-08.privateemail.com (localhost [127.0.0.1]) by MTA-08.privateemail.com (Postfix) with ESMTP id E9C3660044 for ; Mon, 11 Feb 2019 14:18:01 -0500 (EST) Received: from DESKTOP9H8BFK9 (unknown [10.20.151.210]) by MTA-08.privateemail.com (Postfix) with ESMTPA id 2C5606003E for ; Mon, 11 Feb 2019 19:18:00 +0000 (UTC) From: "Leslie Long" To: References: In-Reply-To: Subject: RE: Digital Signage Expo - DSE 2019 Attendee Portal Date: Mon, 11 Feb 2019 14:16:14 -0500 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdS/HU4R3wiJzMmWToWTeyvUysn4IwDILOxQ Content-Language: en-us X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 982AF8645B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of leslie@b2bglobedata.com designates 198.54.122.48 as permitted sender) smtp.mailfrom=leslie@b2bglobedata.com X-Spamd-Result: default: False [-2.20 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-0.78)[-0.781,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.54.122.32/27]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; MANY_INVISIBLE_PARTS(0.30)[4]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[mx2.privateemail.com,mx1.privateemail.com]; NEURAL_HAM_SHORT(-0.34)[-0.340,0]; DMARC_NA(0.00)[b2bglobedata.com]; IP_SCORE(-0.07)[ipnet: 198.54.122.0/24(-0.21), asn: 22612(-0.08), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:22612, ipnet:198.54.122.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 19:18:11 -0000 Hi, I haven't heard you from a while. Hence floating this to the top of your inbox to understand, if you did get a chance to review my earlier mail. Kindly let me know your interest so that I can get back to you with counts and pricing available. Regards, Leslie From: Leslie Long [mailto:leslie@b2bglobedata.com] Sent: Thursday, February 7, 2019 2:43 PM To: 'freebsd-stable@freebsd.org' Subject: Digital Signage Expo - DSE 2019 Attendee Portal Dear Exhibitor, May we wish you every success in the forthcoming exhibition/expo? Digital Signage Expo - DSE 2019 Attendees list is available now, all the contacts include only permission-based opt-in information. Hence, you directly reach the right point of contact. We can accord the insights on an excel spread-sheet or whichever the format your CRM supports to send invitations and to run an effective marketing campaign before and after the show. Attendees : . decision-makers Attendee list fields include: Company Name, Website, Contact Name, Title, Contact Number, Mailing Address and Contact Person Verified Business Email Address etc. Information provided is for perpetual and unlimited usage including telemarketing, direct marketing and email marketing initiatives. Please let me know your thoughts and interest, So that I can send you the Exhibitor discount costs of the lists. Swift response would be appreciated. Regards Leslie Digital Signage Expo - DSE 2019 Attendees Specialist From owner-freebsd-stable@freebsd.org Tue Feb 12 08:23:51 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A4DC14DBF29 for ; Tue, 12 Feb 2019 08:23:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0179E77BCD for ; Tue, 12 Feb 2019 08:23:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B8EC114DBF28; Tue, 12 Feb 2019 08:23:50 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A66E314DBF27 for ; Tue, 12 Feb 2019 08:23:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B72677BCC for ; Tue, 12 Feb 2019 08:23:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 86B6215B0 for ; Tue, 12 Feb 2019 08:23:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1C8NnYq090815 for ; Tue, 12 Feb 2019 08:23:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1C8NnVJ090814 for stable@FreeBSD.org; Tue, 12 Feb 2019 08:23:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235683] ZFS kernel panic when access to data or scrub Date: Tue, 12 Feb 2019 08:23:49 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 08:23:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235683 Sergey Anokhin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stable@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 09:15:19 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A80D714DE886 for ; Tue, 12 Feb 2019 09:15:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 61E08825F5 for ; Tue, 12 Feb 2019 09:15:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1BBA014DE87C; Tue, 12 Feb 2019 09:15:19 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A5FF14DE87A for ; Tue, 12 Feb 2019 09:15:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 99CBE825ED for ; Tue, 12 Feb 2019 09:15:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C77CB1CE3 for ; Tue, 12 Feb 2019 09:15:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1C9FH2Y049049 for ; Tue, 12 Feb 2019 09:15:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1C9FHwC049039 for stable@FreeBSD.org; Tue, 12 Feb 2019 09:15:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 09:15:18 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 09:15:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 Sergey Anokhin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bugs@FreeBSD.org, | |stable@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 11:20:12 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2975914E3011 for ; Tue, 12 Feb 2019 11:20:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A8B1087563 for ; Tue, 12 Feb 2019 11:20:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 649B014E300C; Tue, 12 Feb 2019 11:20:11 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5256814E300B for ; Tue, 12 Feb 2019 11:20:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E13088755D for ; Tue, 12 Feb 2019 11:20:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 104A02E9C for ; Tue, 12 Feb 2019 11:20:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CBK9H6035926 for ; Tue, 12 Feb 2019 11:20:09 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CBK91U035923 for stable@FreeBSD.org; Tue, 12 Feb 2019 11:20:09 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 11:20:08 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 11:20:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ae@FreeBSD.org --- Comment #1 from Andrey V. Elsukov --- (In reply to Sergey Anokhin from comment #0) > I see kernel panic during racoon restart. >=20 > # uname -rv > 12.0-STABLE FreeBSD 12.0-STABLE r343904 SERVER Please, show the content of your kernel config and what sysctl variables do= you changed against default configuration. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 13:49:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE27B14E716E for ; Tue, 12 Feb 2019 13:49:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0A9C68C956 for ; Tue, 12 Feb 2019 13:49:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id BAB3E14E7169; Tue, 12 Feb 2019 13:49:03 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DD7C14E7167 for ; Tue, 12 Feb 2019 13:49:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E80A58C950 for ; Tue, 12 Feb 2019 13:49:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id F135646EC for ; Tue, 12 Feb 2019 13:49:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CDn1av089736 for ; Tue, 12 Feb 2019 13:49:01 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CDn1Au089735 for stable@FreeBSD.org; Tue, 12 Feb 2019 13:49:01 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 13:49:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 13:49:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #2 from Sergey Anokhin --- (In reply to Andrey V. Elsukov from comment #1) kernel config: (pts/2)[root@server:~]# cat /usr/src/sys/amd64/conf/SERVER # # GENERIC -- Generic kernel configuration file for FreeBSD/amd64 # # For more information on this file, please read the config(5) manual page, # and/or the handbook section on Kernel Configuration Files: # #=20=20=20 https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-con= fig.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (https://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # If you are in doubt as to the purpose or necessity of a line, check first # in NOTES. # # $FreeBSD: stable/12/sys/amd64/conf/GENERIC 340695 2018-11-20 19:37:09Z zeising $ cpu HAMMER ident SERVER makeoptions DEBUG=3D-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=3D1 # Run ctfconvert(1) for DTrace su= pport options SCHED_ULE # ULE scheduler options NUMA # Non-Uniform Memory Architecture support options PREEMPTION # Enable kernel thread preemption options VIMAGE # Subsystem virtualization, e.g. VN= ET options INET # InterNETworking options INET6 # IPv6 communications protocols options IPSEC # IP (v4/v6) security options IPSEC_SUPPORT # Allow kldload of ipsec and tcpmd5 options TCP_OFFLOAD # TCP offload options TCP_BLACKBOX # Enhanced TCP event logging options TCP_HHOOK # hhook(9) framework for TCP options TCP_RFC7413 # TCP Fast Open options SCTP # Stream Control Transmission Proto= col options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journal= ing options QUOTA # Enable disk quotas for UFS options MD_ROOT # MD is a potential root device options NFSCL # Network Filesystem Client options NFSD # Network Filesystem Server options NFSLOCKD # Network Lock Manager options NFS_ROOT # NFS usable as /, requires NFSCL options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options EFIRT # EFI Runtime Services support options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options COMPAT_FREEBSD7 # Compatible with FreeBSD7 options COMPAT_FREEBSD9 # Compatible with FreeBSD9 options COMPAT_FREEBSD10 # Compatible with FreeBSD10 options COMPAT_FREEBSD11 # Compatible with FreeBSD11 options SCSI_DELAY=3D5000 # Delay (in ms) before probing SC= SI options KTRACE # ktrace(1) support options STACK # stack(9) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options PRINTF_BUFR_SIZE=3D128 # Prevent printf output being interspersed. options KBD_INSTALL_CDEV # install a CDEV entry in /dev options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(= 4) options AUDIT # Security event auditing options CAPABILITY_MODE # Capsicum capability mode options CAPABILITIES # Capsicum capabilities options MAC # TrustedBSD MAC Framework options KDTRACE_FRAME # Ensure frames are compiled in options KDTRACE_HOOKS # Kernel DTrace hooks options DDB_CTF # Kernel ELF linker loads CTF data options INCLUDE_CONFIG_FILE # Include this file in kernel options RACCT # Resource accounting framework options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=3D0 by de= fault options RCTL # Resource limits # Debugging support. Always need this: options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. # Kernel dump features. options EKCD # Support for encrypted kernel dumps options GZIO # gzip-compressed kernel and user d= umps options ZSTDIO # zstd-compressed kernel and user d= umps options NETDUMP # netdump(4) client support # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel options EARLY_AP_STARTUP # CPU frequency control device cpufreq # Bus support. device acpi options ACPI_DMAR device pci options PCI_HP # PCI-Express native HotPlug options PCI_IOV # PCI SR-IOV support # Floppy drives device fdc # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3= 531 SATA # SCSI Controllers device ahc # AHA2940 and onboard AIC7xxx devic= es device ahd # AHA39320/29320 and onboard AIC79xx devices device esp # AMD Am53C974 (Tekram DC-390(T)) device hptiop # Highpoint RocketRaid 3xxx series device isp # Qlogic family #device ispfw # Firmware for QLogic HBAs- normall= y a module device mpt # LSI-Logic MPT-Fusion device mps # LSI-Logic MPT-Fusion 2 device mpr # LSI-Logic MPT-Fusion 3 #device ncr # NCR/Symbios Logic device sym # NCR/Symbios Logic (newer chipsets= + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device isci # Intel C600 SAS controller device ocs_fc # Emulex FC adapters # ATA/SCSI peripherals device scbus # SCSI bus (required for ATA/SCSI) device ch # SCSI media changers device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct ATA/SC= SI access) device ses # Enclosure Services (SES and SAF-T= E) #device ctl # CAM Target Layer # RAID controllers interfaced to the SCSI subsystem device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options device hptmv # Highpoint RocketRAID 182x device hptnr # Highpoint DC7280, R750 device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx device hpt27xx # Highpoint RocketRAID 27xx device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID device twa # 3ware 9000 series PATA/SATA RAID device smartpqi # Microsemi smartpqi driver device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller # RAID controllers device aac # Adaptec FSA RAID device aacp # SCSI passthrough for aac (requires CAM) device aacraid # Adaptec by PMC RAID device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS device mlx # Mylex DAC960 family device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s device pmspcv # PMC-Sierra SAS/SATA Controller dr= iver #XXX pointer/int warnings #device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID # NVM Express (NVMe) support device nvme # base NVMe driver device nvd # expose NVMe namespaces as disks, depends on nvme # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver options VESA # Add support for VESA BIOS Extensi= ons (VBE) device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc options SC_PIXEL_MODE # add support for the raster text m= ode # vt is the new video console driver device vt device vt_vga device vt_efifb device agp # support several AGP chipsets # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support device cbb # cardbus (yenta) bridge device pccard # PC Card (16-bit) bus device cardbus # CardBus (32-bit) bus # Serial (COM) ports device uart # Generic UART driver # Parallel port device ppc device ppbus # Parallel port bus (required) device lpt # Printer device ppi # Parallel port interface device #device vpo # Requires scbus and da device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family device ix # Intel PRO/10GbE PCIE PF Ethernet device ixv # Intel PRO/10GbE PCIE VF Ethernet device ixl # Intel 700 Series Physical Function device iavf # Intel Adaptive Virtual Function device le # AMD Am7900 LANCE and Am79C9xx PCn= et device ti # Alteon Networks Tigon I/II gigabit Ethernet device txp # 3Com 3cR990 (``Typhoon'') device vx # 3Com 3c590, 3c595 (``Vortex'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support device ae # Attansic/Atheros L2 FastEthernet device age # Attansic/Atheros L1 Gigabit Ether= net device alc # Atheros AR8131/AR8132 Ethernet device ale # Atheros AR8121/AR8113/AR8114 Ethe= rnet device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device cas # Sun Cassini/Cassini+ and NS DP830= 65 Saturn device dc # DEC/Intel 21143 and various workalikes device et # Agere ET1310 10/100/Gigabit Ether= net device fxp # Intel EtherExpress PRO/100B (8255= 7, 82558) device gem # Sun GEM/Sun ERI/Apple GMAC device hme # Sun HME (Happy Meal Ethernet) device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet device lge # Level 1 LXT1001 gigabit Ethernet device msk # Marvell/SysKonnect Yukon II Gigab= it Ethernet device nfe # nVidia nForce MCP on-board Ethern= et device nge # NatSemi DP83820 gigabit Ethernet device pcn # AMD Am79C97x PCI 10/100 (preceden= ce over 'le') device re # RealTek 8139C+/8169/8169S/8110S device rl # RealTek 8129/8139 device sf # Adaptec AIC-6915 (``Starfire'') device sge # Silicon Integrated Systems SiS190= /191 device sis # Silicon Integrated Systems SiS 900/SiS 7016 device sk # SysKonnect SK-984x & SK-982x giga= bit Ethernet device ste # Sundance ST201 (D-Link DFE-550TX) device stge # Sundance/Tamarack TC9021 gigabit Ethernet device tl # Texas Instruments ThunderLAN device tx # SMC EtherPower II (83c170 ``EPIC'= ') device vge # VIA VT612x gigabit Ethernet device vr # VIA Rhine, Rhine II device wb # Winbond W89C840F device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # Wireless NIC cards device wlan # 802.11 support options IEEE80211_DEBUG # enable debug msgs options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's options IEEE80211_SUPPORT_MESH # enable 802.11s draft support device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support device wlan_tkip # 802.11 TKIP support device wlan_amrr # AMRR transmit rate control algori= thm device an # Aironet 4500/4800 802.11 wireless NICs. device ath # Atheros NICs device ath_pci # Atheros pci/cardbus glue device ath_hal # pci/cardbus chip support options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation options ATH_ENABLE_11N # Enable 802.11n support for AR5416= and later device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. device ipw # Intel 2100 wireless NICs. device iwi # Intel 2200BG/2225BG/2915ABG wirel= ess NICs. device iwn # Intel 4965/1000/5000/6000 wireless NICs. device malo # Marvell Libertas wireless NICs. device mwl # Marvell 88W8363 802.11n wireless NICs. device ral # Ralink Technology RT2500 wireless NICs. device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device crypto # core crypto support device loop # Network loopback device random # Entropy device device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support device tun # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter # USB support options USB_DEBUG # enable debug msgs device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device xhci # XHCI PCI->USB interface (USB 3.0) device usb # USB Bus (required) device ukbd # Keyboard device umass # Disks/Mass storage - Requires scb= us and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio device snd_ich # Intel, NVidia and other ICH AC'97 Audio device snd_via8233 # VIA VT8233x Audio # MMC/SD device mmc # MMC/SD bus device mmcsd # MMC/SD memory card device sdhci # Generic PCI SD Host Controller # VirtIO support device virtio # Generic VirtIO bus (required) device virtio_pci # VirtIO PCI device device vtnet # VirtIO Ethernet device device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device # HyperV drivers and enhancement support device hyperv # HyperV drivers # Xen HVM Guest Optimizations # NOTE: XENHVM depends on xenpci. They must be added or removed together. options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver # VMware support device vmx # VMware VMXNET3 Ethernet # Netmap provides direct access to TX/RX rings on supported NICs device netmap # netmap(4) support # evdev interface options EVDEV_SUPPORT # evdev support in legacy drivers device evdev # input event device support device uinput # install /dev/uinput cdev #CUSTOM KERNEL FOLLOWING... options NETGRAPH options NETGRAPH_PPP options NETGRAPH_PPTPGRE options NETGRAPH_ETHER options NETGRAPH_SOCKET options NETGRAPH_TEE options NETGRAPH_ASYNC options NETGRAPH_IFACE options NETGRAPH_MPPC_ENCRYPTION options NETGRAPH_MPPC_COMPRESSION options NETGRAPH_BPF options NETGRAPH_KSOCKET options NETGRAPH_TCPMSS options NETGRAPH_VJC options NETGRAPH_ONE2MANY options NETGRAPH_RFC1490 options NETGRAPH_TTY options NETGRAPH_UI options LIBALIAS options MROUTING options NETGRAPH_PPPOE options NETGRAPH_HOLE options NETGRAPH_ECHO options NETGRAPH_L2TP # By Executor (vlad.admin@mail.ru) options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=3D999 options IPFILTER options IPFILTER_LOG options IPDIVERT options DUMMYNET options DEVICE_POLLING #options IPFIREWALL_FORWARD options IPFIREWALL_NAT options IPFIREWALL_DEFAULT_TO_ACCEPT #colortag options SC_NORM_ATTR=3D"(FG_GREEN|BG_BLACK)" options SC_NORM_REV_ATTR=3D"(FG_YELLOW|BG_GREEN)" options SC_KERNEL_CONS_REV_ATTR=3D"(FG_BLACK|BG_RED)" options SC_KERNEL_CONS_ATTR=3D"(FG_RED|BG_BLACK)" # For HTTP Server maxusers 512 # options HZ=3D1000 # PF support device pf device pflog device pfsync options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_PRIQ options ALTQ_NOPCC options SHMMAXPGS=3D65536 options SEMMNI=3D40 options SEMMNS=3D240 options SEMUME=3D40 options SEMMNU=3D120 #options RADIX_MPATH #options COMPAT_FREEBSD8 # Compatible with FreeBSD8 #22-08-2012 for ZFS #options KVA_PAGES=3D160 #03-10-2013 # IPSec #options IPSEC_FILTERTUNNEL #options IPSEC_NAT_T options IPSEC_DEBUG device enc #19-11-2013 device tap #28-02-2014 options MAC_PORTACL sysctl config: # cat /etc/sysctl.conf # $FreeBSD: stable/12/sbin/sysctl/sysctl.conf 337624 2018-08-11 13:28:03Z b= rd $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes t= hat # are being run under another UID. #security.bsd.see_other_uids=3D0 net.inet6.ip6.v6only=3D0 kern.maxfiles=3D65536 kern.maxfilesperproc=3D32768 kern.ipc.somaxconn=3D32768 kern.ipc.shmmax=3D204800000 kern.ipc.shmall=3D409600 #kern.ipc.nmbclusters=3D65535 net.inet.ip.random_id=3D1 net.inet.tcp.blackhole=3D2 net.inet.udp.blackhole=3D1 net.inet.tcp.mssdflt=3D1500 #kern.kstack_pages=3D4 nen.inet.ip.portrange.reservedlow=3D0 net.inet.ip.portrange.reservedhigh=3D0 security.mac.portacl.port_high=3D1023 security.mac.portacl.suser_exempt=3D1 security.mac.portacl.rules=3Duid:53:tcp:53,uid:53:udp:53 #vfs.zfs.arc_max=3D2000000000 debug.debugger_on_panic=3D0 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 14:02:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7C3E14E786F for ; Tue, 12 Feb 2019 14:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 56BF38D484 for ; Tue, 12 Feb 2019 14:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1020A14E786C; Tue, 12 Feb 2019 14:02:22 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F024A14E786B for ; Tue, 12 Feb 2019 14:02:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 851198D46F for ; Tue, 12 Feb 2019 14:02:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A41ED49C7 for ; Tue, 12 Feb 2019 14:02:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CE2Kgl052764 for ; Tue, 12 Feb 2019 14:02:20 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CE2KST052757 for stable@FreeBSD.org; Tue, 12 Feb 2019 14:02:20 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235683] [zfs] Panic during data access or scrub on 12.0-STABLE r343904 (blkptr at DVA 0 has invalid OFFSET) Date: Tue, 12 Feb 2019 14:02:20 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avos@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc short_desc keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 14:02:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235683 Andriy Voskoboinyk changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |avos@freebsd.org Summary|ZFS kernel panic when |[zfs] Panic during data |access to data or scrub |access or scrub on | |12.0-STABLE r343904 (blkptr | |at DVA 0 has invalid | |OFFSET) Keywords| |panic Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 14:21:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C20F814E813F for ; Tue, 12 Feb 2019 14:21:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 50E668E269 for ; Tue, 12 Feb 2019 14:21:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1179F14E813B; Tue, 12 Feb 2019 14:21:16 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F14B814E813A for ; Tue, 12 Feb 2019 14:21:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85A758E261 for ; Tue, 12 Feb 2019 14:21:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B12AF4B63 for ; Tue, 12 Feb 2019 14:21:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CELEPK087572 for ; Tue, 12 Feb 2019 14:21:14 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CELEhC087571 for stable@FreeBSD.org; Tue, 12 Feb 2019 14:21:14 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 14:21:14 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 14:21:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #3 from Andrey V. Elsukov --- KEYDBG() macro executed only when net.key.debug is set to non-zero value. It looks like your sysctl.conf didn't set it. Also, it looks impossible to get page fault with fault address 0x28 in this line of code. I suspect, that you have some sort of memory corruption. Not sure, is it hardware related or it= is overwritten by some code. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 14:53:23 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6CD514E910A for ; Tue, 12 Feb 2019 14:53:23 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A3578F5A9 for ; Tue, 12 Feb 2019 14:53:22 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6]) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1gtZR2-0000eV-EN; Tue, 12 Feb 2019 14:53:20 +0000 Subject: Networking panic on 12 - found the cause To: "Andrey V. Elsukov" , freebsd-stable@freebsd.org References: <4a350f47-eaca-5aca-9268-bf7a6618e71c@yandex.ru> From: Pete French Message-ID: Date: Tue, 12 Feb 2019 14:53:20 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <4a350f47-eaca-5aca-9268-bf7a6618e71c@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 8A3578F5A9 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 2a02:b90:3002:411::3 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [3.01 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:b90:3002:411::3]; NEURAL_SPAM_SHORT(0.93)[0.927,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.02)[country: GB(-0.09)]; RDNS_NONE(1.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.898,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; NEURAL_SPAM_LONG(0.31)[0.314,0]; FREEMAIL_TO(0.00)[yandex.ru]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:2a02:b90::/32, country:GB]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 14:53:23 -0000 I found my panic. If I take everything out of rc.conf and loader.conf and sysctl.conf and boot the system it works fine when I add an IP address. If I add this one line to sysctl.conf net.link.ether.inet.garp_rexmit_count=2 Then I get a panic when I configure the interface: root@serpentine-passive:~ # ifconfig igb0 inet 10.32.10.4/16 up Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80c987f1 stack pointer = 0x28:0xfffffe00004d5730 frame pointer = 0x28:0xfffffe00004d5750 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock (0)) trap number = 12 panic: page fault cpuid = 0 time = 1549981620 KDB: stack backtrace: #0 0xffffffff80bdfdc7 at kdb_backtrace+0x67 #1 0xffffffff80b93fa3 at vpanic+0x1a3 #2 0xffffffff80b93df3 at panic+0x43 #3 0xffffffff8106a7bf at trap_fatal+0x35f #4 0xffffffff8106a819 at trap_pfault+0x49 #5 0xffffffff81069e3e at trap+0x29e #6 0xffffffff810450c5 at calltrap+0x8 #7 0xffffffff80c986f6 at ether_output+0x6b6 #8 0xffffffff80d03354 at arprequest+0x4c4 #9 0xffffffff80d0515c at garp_rexmit+0xbc #10 0xffffffff80bade19 at softclock_call_cc+0x129 #11 0xffffffff80bae2f9 at softclock+0x79 #12 0xffffffff80b57c57 at ithread_loop+0x1a7 #13 0xffffffff80b54da2 at fork_exit+0x82 #14 0xffffffff810460be at fork_trampoline+0xe Uptime: 2m6s From owner-freebsd-stable@freebsd.org Tue Feb 12 15:43:19 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA16314EA4CC for ; Tue, 12 Feb 2019 15:43:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 48B026AD57 for ; Tue, 12 Feb 2019 15:43:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E773A14EA4C7; Tue, 12 Feb 2019 15:43:17 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEC4C14EA4C5 for ; Tue, 12 Feb 2019 15:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9776AD54 for ; Tue, 12 Feb 2019 15:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 48F44585E for ; Tue, 12 Feb 2019 15:43:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CFhGYq085591 for ; Tue, 12 Feb 2019 15:43:16 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CFhG9o085590 for stable@FreeBSD.org; Tue, 12 Feb 2019 15:43:16 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 15:43:15 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 15:43:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #4 from Sergey Anokhin --- (In reply to Andrey V. Elsukov from comment #3) There is a mind that if turn off options IPSEC_DEBUG kernel panic will disappear --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:13:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 153BC14EB1B7 for ; Tue, 12 Feb 2019 16:13:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 91C2C6BDCA for ; Tue, 12 Feb 2019 16:13:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4720414EB1B5; Tue, 12 Feb 2019 16:13:43 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31E2F14EB1B3 for ; Tue, 12 Feb 2019 16:13:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B50076BDC8 for ; Tue, 12 Feb 2019 16:13:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id D2E0D5CB8 for ; Tue, 12 Feb 2019 16:13:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGDfG5080002 for ; Tue, 12 Feb 2019 16:13:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGDfs3080001 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:13:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 11.1-RELEASE amd64: GENERIC kernel compiled without any CPUTYPE?= in /etc/make.conf fail to boot, but with CPUTYPE?=nocona it work fine Date: Tue, 12 Feb 2019 16:13:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vvd@unislabs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:13:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 --- Comment #4 from VVD --- Created attachment 201960 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D201960&action= =3Dedit dmesg from boot -v with GENERIC kernel 11.2-RELEASE-p8 amd64 11.2 GENERIC kernel from freebsd-update boot fine. But any 12.0 kernel freezes just after start. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:14:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B6A714EB229 for ; Tue, 12 Feb 2019 16:14:57 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D3F06BE5D for ; Tue, 12 Feb 2019 16:14:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CGEXZf071845 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 12 Feb 2019 17:14:34 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CGEWF7051369 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Tue, 12 Feb 2019 23:14:32 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable From: Eugene Grosbein Subject: 11.2-STABLE kernel wired memory leak Message-ID: Date: Tue, 12 Feb 2019 23:14:31 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 5D3F06BE5D X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.42)[ip: (-2.45), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.84)[-0.843,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:14:57 -0000 Hi! Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel wired memory over 81 days uptime out of 8GB total RAM. Details follow. I have a workstation running Xorg, Firefox, Thunderbird, LibreOffice and occasionally VirtualBox for single VM. It has two identical 320GB HDDs combined with single graid-based array with "Intel" on-disk format having 3 volumes: - one "RAID1" volume /dev/raid/r0 occupies first 10GB or each HDD; - two "SINGLE" volumes /dev/raid/r1 and /dev/raid/r2 that utilize "tails" of HDDs (310GB each). /dev/raid/r0 (10GB) has MBR partitioning and two slices: - /dev/raid/r0s1 (8GB) is used for swap; - /dev/raid/r0s2 (2GB) is used by non-redundant ZFS pool named "os" that contains only root file system (177M used) and /usr file system (340M used). There is also second pool (ZMIRROR) named "z" built directly on top of /dev/raid/r[12] volumes, this pool contains all other file systems including /var, /home, /usr/ports, /usr/local, /usr/{src|obj} etc. # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT os 1,98G 520M 1,48G - - 55% 25% 1.00x ONLINE - z 288G 79,5G 209G - - 34% 27% 1.00x ONLINE - This way I have swap outside of ZFS, boot blocks and partitioning mirrored by means of GEOM_RAID and can use local console to break to single user mode to unmount all file system other than root and /usr and can even export bigger ZFS pool "z". And I did that to see that ARC usage (limited with vfs.zfs.arc_max="3G" in /boot/loader.conf) dropped from over 2500MB down to 44MB but "Wired" stays high. Now after I imported "z" back and booted to multiuser mode top(1) shows: last pid: 51242; load averages: 0.24, 0.16, 0.13 up 81+02:38:38 22:59:18 104 processes: 1 running, 103 sleeping CPU: 0.0% user, 0.0% nice, 0.4% system, 0.2% interrupt, 99.4% idle Mem: 84M Active, 550M Inact, 4K Laundry, 4689M Wired, 2595M Free ARC: 273M Total, 86M MFU, 172M MRU, 64K Anon, 1817K Header, 12M Other 117M Compressed, 333M Uncompressed, 2.83:1 Ratio Swap: 8192M Total, 940K Used, 8191M Free I have KDB and DDB in my custom kernel also. How do I debug the leak further? I use nvidia-driver-340-340.107 driver for GK208 [GeForce GT 710B] video card. Here are outputs of "vmstat -m": http://www.grosbein.net/freebsd/leak/vmstat-m.txt and "vmstat -z": http://www.grosbein.net/freebsd/leak/vmstat-z.txt as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt From owner-freebsd-stable@freebsd.org Tue Feb 12 16:16:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A72414EB2B8 for ; Tue, 12 Feb 2019 16:16:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D4EDA6BF4B for ; Tue, 12 Feb 2019 16:16:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8F27414EB2B6; Tue, 12 Feb 2019 16:16:02 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CE9614EB2B5 for ; Tue, 12 Feb 2019 16:16:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19A096BF48 for ; Tue, 12 Feb 2019 16:16:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 56EC65CBC for ; Tue, 12 Feb 2019 16:16:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGG1cM082641 for ; Tue, 12 Feb 2019 16:16:01 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGG1qu082640 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:16:01 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 11.1-RELEASE amd64: GENERIC kernel compiled without any CPUTYPE?= in /etc/make.conf fail to boot, but with CPUTYPE?=nocona it work fine Date: Tue, 12 Feb 2019 16:16:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vvd@unislabs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:16:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 --- Comment #5 from VVD --- Created attachment 201961 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D201961&action= =3Dedit Photo for screen from boot -v with kernel 12.0-RELEASE-p3 amd64 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:16:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F21C514EB303 for ; Tue, 12 Feb 2019 16:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AB7546BFB4 for ; Tue, 12 Feb 2019 16:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 64B6514EB2FF; Tue, 12 Feb 2019 16:16:31 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5279E14EB2FE for ; Tue, 12 Feb 2019 16:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E2A766BFB0 for ; Tue, 12 Feb 2019 16:16:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 29A175CBE for ; Tue, 12 Feb 2019 16:16:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGGU4N083168 for ; Tue, 12 Feb 2019 16:16:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGGUTx083167 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:16:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 11.1-RELEASE amd64: GENERIC kernel compiled without any CPUTYPE?= in /etc/make.conf fail to boot, but with CPUTYPE?=nocona it work fine Date: Tue, 12 Feb 2019 16:16:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vvd@unislabs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:16:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 VVD changed: What |Removed |Added ---------------------------------------------------------------------------- Version|11.1-RELEASE |12.0-RELEASE --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:18:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20B6614EB49C for ; Tue, 12 Feb 2019 16:18:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 78EC46C27B for ; Tue, 12 Feb 2019 16:18:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3CDF414EB497; Tue, 12 Feb 2019 16:18:43 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28CD814EB496 for ; Tue, 12 Feb 2019 16:18:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B65216C275 for ; Tue, 12 Feb 2019 16:18:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0277A5CC5 for ; Tue, 12 Feb 2019 16:18:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGIfIi085365 for ; Tue, 12 Feb 2019 16:18:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGIfZF085364 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:18:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 16:18:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:18:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #5 from Andrey V. Elsukov --- (In reply to Sergey Anokhin from comment #4) > (In reply to Andrey V. Elsukov from comment #3) >=20 > There is a mind that if turn off >=20 > options IPSEC_DEBUG >=20 > kernel panic will disappear Disabling IPSEC_DEBUG also reduces the requirement to kernel stack size. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:20:29 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A54814EB654 for ; Tue, 12 Feb 2019 16:20:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 21AEB6C45B for ; Tue, 12 Feb 2019 16:20:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DA36914EB653; Tue, 12 Feb 2019 16:20:28 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C80F114EB651 for ; Tue, 12 Feb 2019 16:20:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 621766C459 for ; Tue, 12 Feb 2019 16:20:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A23F15CCF for ; Tue, 12 Feb 2019 16:20:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGKRsu087236 for ; Tue, 12 Feb 2019 16:20:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGKRW4087233 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:20:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 11.1-RELEASE amd64: GENERIC kernel compiled without any CPUTYPE?= in /etc/make.conf fail to boot, but with CPUTYPE?=nocona it work fine Date: Tue, 12 Feb 2019 16:20:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vvd@unislabs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.description Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:20:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 VVD changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #201961|Photo for screen from boot |Photo of screen from boot description|-v with kernel |-v with kernel |12.0-RELEASE-p3 amd64 |12.0-RELEASE-p3 amd64 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:21:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1F1014EB8FC for ; Tue, 12 Feb 2019 16:21:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 405536C84E for ; Tue, 12 Feb 2019 16:21:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0484F14EB8F6; Tue, 12 Feb 2019 16:21:43 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4CED14EB8F5 for ; Tue, 12 Feb 2019 16:21:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7ADDF6C849 for ; Tue, 12 Feb 2019 16:21:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B680B5E0A for ; Tue, 12 Feb 2019 16:21:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGLfA2091230 for ; Tue, 12 Feb 2019 16:21:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGLf1Z091229 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:21:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 12.0-RELEASE amd64: any kernel fail to boot - freezes just after run kernel Date: Tue, 12 Feb 2019 16:21:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vvd@unislabs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:21:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 VVD changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|11.1-RELEASE amd64: GENERIC |12.0-RELEASE amd64: any |kernel compiled without any |kernel fail to boot - |CPUTYPE?=3D in /etc/make.conf |freezes just after run |fail to boot, but with |kernel |CPUTYPE?=3Dnocona it work | |fine | --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:34:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6E3D14EC1AD for ; Tue, 12 Feb 2019 16:34:53 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-it1-x12e.google.com (mail-it1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDF2F6D45F for ; Tue, 12 Feb 2019 16:34:52 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-it1-x12e.google.com with SMTP id r11so8685811itc.2 for ; Tue, 12 Feb 2019 08:34:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=NSPpygq0DO9X63ZV9JVE4Y5cN8VOj/WHNq/g89YjciE=; b=ZzObx2B2DMRlNgWzO3V79uzQthEBwjx0xsTs9VqxSIrtCp47jry6f15rCa46Y6yCll Ai4DT4nFyDdmpA9qJ4uvxmSb6S/PXqGawzaBv8nWArAz2I/e0x1X6l88hAkxMM4cx/UD M+N1heilZE2mnpZC8oaINak+gpo4MXCZB4+8k2FHp65srzgO+EWBwZNwaq7KqyYFjxbD t5/pz21vFSSgVgrDODBkerHNGKIfR20xCUtRHTGXqH9sWUykKe/9tzE2t4dmH968mbgt ovE3dXkk/UAoiM02MQ3ayB/D/c5Gh6nJ+9vKyGXs8R6FtKonk5+VKksv9zVe+i/WhbV2 1B5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=NSPpygq0DO9X63ZV9JVE4Y5cN8VOj/WHNq/g89YjciE=; b=XiV+jboncfZZffijjdBf6tHWa/ilsr4Sz4279bqi3JI1sLR8elghqojPTfOL665o2F X8wNxIs/emRaz2BaagR1mK8oua8r+ot/XI3TcZ9u/evNM8Ph4YLZ92VLWyxTHsU/SElb 4iFnKBj5rj/LgKKXXLMD9WNzKhsUr6bDQFqkaKeHVQ5a0hJkmOTK+UDDPZBPD0j6SwA8 fJDRwNycWoXN+QOxFvFnA0x5RnoJeOiz+FQTeeFfN71ZyDlq9Luj15F3HXGkj0O2uZnH 514uhikWFLGeg6gnBsTmqkp208BU5kIndGnzIjXKrKtfuRf9VCwFbt3x5JDafKDyRTva oHKg== X-Gm-Message-State: AHQUAuYGWAwBh0B11iC4BtAKnXOhBRFpB5zeIRnv/CVDt6/Rm4Pp/56w aOnqDtoeMCZRkdPF5cwDefp3JeCN X-Google-Smtp-Source: AHgI3IZjHlW5iI/N/Epfw53WamJ8yq4vhPrA81hhqTLCUtp3VJOjGBdHQkdbez0fcpzwYKJ+MJXzcQ== X-Received: by 2002:a5d:9a98:: with SMTP id c24mr2650105iom.227.1549989292054; Tue, 12 Feb 2019 08:34:52 -0800 (PST) Received: from raichu (toroon0560w-lp140-01-69-159-36-102.dsl.bell.ca. [69.159.36.102]) by smtp.gmail.com with ESMTPSA id u3sm1607470ita.18.2019.02.12.08.34.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 08:34:51 -0800 (PST) Sender: Mark Johnston Date: Tue, 12 Feb 2019 11:34:46 -0500 From: Mark Johnston To: Eugene Grosbein Cc: FreeBSD stable Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212163446.GA29847@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: DDF2F6D45F X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ZzObx2B2; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::12e as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-5.34 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[e.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.65)[ip: (-8.71), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-1.97), country: US(-0.07)]; MID_RHS_NOT_FQDN(0.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:34:54 -0000 On Tue, Feb 12, 2019 at 11:14:31PM +0700, Eugene Grosbein wrote: > Hi! > > Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel wired memory over 81 days uptime > out of 8GB total RAM. > > Details follow. > > I have a workstation running Xorg, Firefox, Thunderbird, LibreOffice and occasionally VirtualBox for single VM. > > It has two identical 320GB HDDs combined with single graid-based array with "Intel" > on-disk format having 3 volumes: > - one "RAID1" volume /dev/raid/r0 occupies first 10GB or each HDD; > - two "SINGLE" volumes /dev/raid/r1 and /dev/raid/r2 that utilize "tails" of HDDs (310GB each). > > /dev/raid/r0 (10GB) has MBR partitioning and two slices: > - /dev/raid/r0s1 (8GB) is used for swap; > - /dev/raid/r0s2 (2GB) is used by non-redundant ZFS pool named "os" that contains only > root file system (177M used) and /usr file system (340M used). > > There is also second pool (ZMIRROR) named "z" built directly on top of /dev/raid/r[12] volumes, > this pool contains all other file systems including /var, /home, /usr/ports, /usr/local, /usr/{src|obj} etc. > > # zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > os 1,98G 520M 1,48G - - 55% 25% 1.00x ONLINE - > z 288G 79,5G 209G - - 34% 27% 1.00x ONLINE - > > This way I have swap outside of ZFS, boot blocks and partitioning mirrored by means of GEOM_RAID and > can use local console to break to single user mode to unmount all file system other than root and /usr > and can even export bigger ZFS pool "z". And I did that to see that ARC usage > (limited with vfs.zfs.arc_max="3G" in /boot/loader.conf) dropped from over 2500MB > down to 44MB but "Wired" stays high. Now after I imported "z" back and booted to multiuser mode > top(1) shows: > > last pid: 51242; load averages: 0.24, 0.16, 0.13 up 81+02:38:38 22:59:18 > 104 processes: 1 running, 103 sleeping > CPU: 0.0% user, 0.0% nice, 0.4% system, 0.2% interrupt, 99.4% idle > Mem: 84M Active, 550M Inact, 4K Laundry, 4689M Wired, 2595M Free > ARC: 273M Total, 86M MFU, 172M MRU, 64K Anon, 1817K Header, 12M Other > 117M Compressed, 333M Uncompressed, 2.83:1 Ratio > Swap: 8192M Total, 940K Used, 8191M Free > > I have KDB and DDB in my custom kernel also. How do I debug the leak further? > > I use nvidia-driver-340-340.107 driver for GK208 [GeForce GT 710B] video card. > Here are outputs of "vmstat -m": http://www.grosbein.net/freebsd/leak/vmstat-m.txt > and "vmstat -z": http://www.grosbein.net/freebsd/leak/vmstat-z.txt I suspect that the "leaked" memory is simply being used to cache UMA items. Note that the values in the FREE column of vmstat -z output are quite large. The cached items are reclaimed only when the page daemon wakes up to reclaim memory; if there are no memory shortages, large amounts of memory may accumulate in UMA caches. In this case, the sum of the product of columns 2 and 5 gives a total of roughly 4GB cached. > as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt > and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt From owner-freebsd-stable@freebsd.org Tue Feb 12 16:37:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBF5514EC2FA for ; Tue, 12 Feb 2019 16:37:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7C6FE6D5BF for ; Tue, 12 Feb 2019 16:37:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3636814EC2F2; Tue, 12 Feb 2019 16:37:14 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24B4414EC2F1 for ; Tue, 12 Feb 2019 16:37:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7E756D5BA for ; Tue, 12 Feb 2019 16:37:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id E9AE25FA3 for ; Tue, 12 Feb 2019 16:37:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CGbCiH023601 for ; Tue, 12 Feb 2019 16:37:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CGbC7S023600 for stable@FreeBSD.org; Tue, 12 Feb 2019 16:37:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 16:37:12 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: crest@bultmann.eu X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:37:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 Jan Bramkamp changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |crest@bultmann.eu --- Comment #6 from Jan Bramkamp --- Can you try again with IPSEC_DEBUG and a doubled kernel stack size? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 16:49:19 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AACF14ECA0F for ; Tue, 12 Feb 2019 16:49:19 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47F376DFE3; Tue, 12 Feb 2019 16:49:18 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CGn7bb072117 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 17:49:11 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CGn7Ar051751 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 12 Feb 2019 23:49:07 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <763f9373-747a-fe5b-8376-cc9bb3293219@grosbein.net> Date: Tue, 12 Feb 2019 23:49:00 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212163446.GA29847@raichu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47F376DFE3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.41)[ip: (-2.42), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.86)[-0.859,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:49:19 -0000 12.02.2019 23:34, Mark Johnston wrote: > I suspect that the "leaked" memory is simply being used to cache UMA > items. Note that the values in the FREE column of vmstat -z output are > quite large. The cached items are reclaimed only when the page daemon > wakes up to reclaim memory; if there are no memory shortages, large > amounts of memory may accumulate in UMA caches. In this case, the sum > of the product of columns 2 and 5 gives a total of roughly 4GB cached. Forgot to note, that before I got system to single user mode, there was heavy swap usage (over 3.5GB) and heavy page-in/page-out, 10-20 megabytes per second and system was crawling slow due to pageing. From owner-freebsd-stable@freebsd.org Tue Feb 12 16:56:23 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4514F14ECEE3 for ; Tue, 12 Feb 2019 16:56:23 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 62B826E78F; Tue, 12 Feb 2019 16:56:22 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CGuD5i072183 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 17:56:14 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CGuC80051802 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 12 Feb 2019 23:56:12 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> <763f9373-747a-fe5b-8376-cc9bb3293219@grosbein.net> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: Date: Tue, 12 Feb 2019 23:56:06 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <763f9373-747a-fe5b-8376-cc9bb3293219@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 62B826E78F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.41)[ip: (-2.39), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.89)[-0.886,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 16:56:23 -0000 12.02.2019 23:49, Eugene Grosbein wrote: > 12.02.2019 23:34, Mark Johnston wrote: > >> I suspect that the "leaked" memory is simply being used to cache UMA >> items. Note that the values in the FREE column of vmstat -z output are >> quite large. The cached items are reclaimed only when the page daemon >> wakes up to reclaim memory; if there are no memory shortages, large >> amounts of memory may accumulate in UMA caches. In this case, the sum >> of the product of columns 2 and 5 gives a total of roughly 4GB cached. > > Forgot to note, that before I got system to single user mode, there was heavy swap usage (over 3.5GB) > and heavy page-in/page-out, 10-20 megabytes per second and system was crawling slow due to pageing. There was significant memory shortage due to Firefox having over 5GB RSS plus other processes like Xorg, parts of xfce4 etc. Still, over 4GB of Wired memory. From owner-freebsd-stable@freebsd.org Tue Feb 12 17:12:02 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40CFD14EDC33 for ; Tue, 12 Feb 2019 17:12:02 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B8F86F719 for ; Tue, 12 Feb 2019 17:12:01 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id A5816211080 for ; Tue, 12 Feb 2019 12:11:29 -0500 (EST) Received: from [192.168.10.15] (D5.Denninger.Net [192.168.10.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id ED475DE9A6 for ; Tue, 12 Feb 2019 11:11:28 -0600 (CST) Subject: Re: 11.2-STABLE kernel wired memory leak To: freebsd-stable@freebsd.org References: <20190212163446.GA29847@raichu> <763f9373-747a-fe5b-8376-cc9bb3293219@grosbein.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <154942d0-3a37-434f-cafc-fafe84c606a0@denninger.net> Date: Tue, 12 Feb 2019 11:11:29 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <763f9373-747a-fe5b-8376-cc9bb3293219@grosbein.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms070205060906050105020508" X-Rspamd-Queue-Id: 7B8F86F719 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.51 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.33)[ip: (-9.89), ipnet: 104.236.64.0/18(-4.39), asn: 14061(2.70), country: US(-0.07)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 17:12:02 -0000 This is a cryptographically signed message in MIME format. --------------ms070205060906050105020508 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/12/2019 10:49, Eugene Grosbein wrote: > 12.02.2019 23:34, Mark Johnston wrote: > >> I suspect that the "leaked" memory is simply being used to cache UMA >> items. Note that the values in the FREE column of vmstat -z output ar= e >> quite large. The cached items are reclaimed only when the page daemon= >> wakes up to reclaim memory; if there are no memory shortages, large >> amounts of memory may accumulate in UMA caches. In this case, the sum= >> of the product of columns 2 and 5 gives a total of roughly 4GB cached.= > Forgot to note, that before I got system to single user mode, there was= heavy swap usage (over 3.5GB) > and heavy page-in/page-out, 10-20 megabytes per second and system was c= rawling slow due to pageing. This is a manifestation of the general issue I've had an ongoing "discussion" running in a long-running thread on bugzilla and the interaction between UMA, ARC and the VM system. The short version is that the VM system does pathological things including paging out working set when there is a large amount of allocated-but-unused UMA and the means by which the ARC code is "told" that it needs to release RAM also interacts with the same mechanisms and exacerbates the problem. I've basically given up on getting anything effective to deal with this merged into the code and have my own private set of patches that I published for a while in that thread (and which had some collaborative development and testing) but have given up on seeing anything meaningful put into the base code.=C2=A0 To the extent I need them in a given worklo= ad and environment I simply apply them on my own and go on my way. I don't have enough experience with 12 yet to know if the same approach will be necessary there (that is, what if any changes got into the 12.x code), and never ran 11.2 much, choosing to stay on 11.1 where said patches may not have been the most-elegant means of dealing with it but were successful.=C2=A0 There was also a phabricator thread on this but I don't know what part of it, if any, got merged (it was more-elegant, in my opinion, than what I had coded up.)=C2=A0 Under certain workloads here= without the patches I was experiencing "freezes" due to unnecessary page-outs onto spinning rust that in some cases reached into double-digit *seconds.*=C2=A0 With them the issue was entirely resolved. At the core of the issue is that "something" has to be taught that *before* the pager starts evicting working set to swap if you have large amounts of UMA allocated to ARC but not in use that RAM should be released, and beyond that if you have ARC allocated and in use but are approaching where VM is going to page working set out you need to come up with some meaningful way of deciding whether to release some of the ARC rather than take the page hit -- and in virtually every case the answer to that question is to release the RAM consumed by ARC.=C2=A0 Part= of the issue is that UMA can be allocated for other things besides ARC yet you really only want to release the ARC-related UMA that is allocated-but-unused in this instance. The logic is IMHO pretty simple on this -- a page-out of a process that will run again always requires TWO disk operations -- one to page it out right now and a second at a later time to page it back in.=C2=A0 A releas= ed ARC cache *MAY* (if there would have been a cache hit in the future) require ONE disk operation (to retrieve it from disk.) Two is always greater than one and one is never worse than "maybe one later" therefore prioritizing taking two *definite* disk I/Os or one definite I/O now and one possible one later instead of one *possible* disk I/O later is always a net lose -- and thus IMHO substantial effort should be made to avoid doing that. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms070205060906050105020508 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEyMTcxMTI5 WjBPBgkqhkiG9w0BCQQxQgRAB3DTcFW+YPoHU5J4cQ7Yy/1JAHrCHx3A9d9Tiy70A/PKvaKY GY8IDIQRUtwCSt9HQfsZV3yB7mQ8/h/0jeg79DBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBpkUQON2uEdXKhFcm+fINWW++ypc44I9A+ceIOvV0bDcqSjDu4OQkTw0jl7A5T50aI Vq59a4TFo9yhGGuBp/b7EuUCKwNkkbEiPGdyHzIhXleC58pQdD3CFs664q7U6Ls1ju9f2yBH /jPbiRNzWnkNcB7N6TBiiXeai8jEiYKsuL4akd9mOI6qhBXjz+xlmsaX2vb9yt1dVS73cISN XIQXuzy4hs4hfve5jprWYtaqKfG9Sd5qoVS9YH4yYimm1lUApr0YVSZCHCfwkoWDhZWLkiD0 B1haz+N2oGeh+5hNaEZgeAikQNg7CxGDaSp5BfJ2eK+xyluNYI4ntj2oFTTkHgGZvO0LbSne +D3gsvyW4fn/Wx6iOlIRku9q1S1f+sQUhYKj+dtQ/rkG+zBjpZsMZsnx+IbNokYOQNnUud3o n2fn1ZCUu5nkaWtjkWOSoj8gU7iTCMIclk3yLFCaRPfzJhztty2aRxhQ16b1EfNv+YTlJZc1 JSdMkzMN8cwnDU0ukrW5BB23Ejly951M9Np13M1qUrvSxg7LLwzgAEQQHsyK6skkgg1b6Ekx QDZvngEBukUuBdQThYFrBsXy+XW6JObp/aw7j6VKEml8c+vjzhsoPODByDiIgPRPf7L/7ahC ckgSlw0JLYKolbC0sja4yqcHyqVUdkzA8MbXSpYd/AAAAAAAAA== --------------ms070205060906050105020508-- From owner-freebsd-stable@freebsd.org Tue Feb 12 17:57:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 023BF14EE8C8 for ; Tue, 12 Feb 2019 17:57:43 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (tunnel82308-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6768671167 for ; Tue, 12 Feb 2019 17:57:42 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.15.2/8.15.2) with ESMTP id x1CHvedd056877 for ; Tue, 12 Feb 2019 12:57:40 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.15.2/8.14.4/Submit) id x1CHve0h056876; Tue, 12 Feb 2019 12:57:40 -0500 (EST) (envelope-from wollman) Date: Tue, 12 Feb 2019 12:57:40 -0500 (EST) Message-Id: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> From: Garrett Wollman To: freebsd-stable@freebsd.org Subject: Re: 11.2-STABLE kernel wired memory leak References: Organization: none Cc: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (hergotha.csail.mit.edu [127.0.0.1]); Tue, 12 Feb 2019 12:57:40 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED, HEADER_FROM_DIFFERENT_DOMAINS autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hergotha.csail.mit.edu X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 17:57:43 -0000 In article eugen@grosbein.net writes: >Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel >wired memory over 81 days uptime >out of 8GB total RAM. Not a whole lot of evidence yet, but anecdotally I'm seeing the same thing on some huge-memory NFS servers running releng/11.2. They seem to run fine for a few weeks, then mysteriously start swapping continuously, a few hundred pages a second. The continues for hours at a time, and then stops just as mysteriously. Over time the total memory dedicated to ZFS ARC goes down but there's no decrease in wired memory. I've tried disabling swap, but this seems to make the server unstable. I have yet to find any obvious commonality (aside from the fact that these are all large-memory NFS servers which don't do much of anything else -- the only software running on them is related to managing and monitoring the NFS service). -GAWollman From owner-freebsd-stable@freebsd.org Tue Feb 12 18:03:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C92BC14EEDA3 for ; Tue, 12 Feb 2019 18:03:55 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C72B71899; Tue, 12 Feb 2019 18:03:54 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CI3iHc072646 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:03:45 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CI3hkM052505 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:03:43 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> Cc: FreeBSD stable , kib@freebsd.org From: Eugene Grosbein Message-ID: <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> Date: Wed, 13 Feb 2019 01:03:37 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212163446.GA29847@raichu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 7C72B71899 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.40)[ip: (-2.36), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.23), country: DE(-0.01)]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.93)[-0.929,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:03:56 -0000 12.02.2019 23:34, Mark Johnston wrote: > I suspect that the "leaked" memory is simply being used to cache UMA > items. Note that the values in the FREE column of vmstat -z output are > quite large. The cached items are reclaimed only when the page daemon > wakes up to reclaim memory; if there are no memory shortages, large > amounts of memory may accumulate in UMA caches. In this case, the sum > of the product of columns 2 and 5 gives a total of roughly 4GB cached. > >> as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt >> and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt It seems page daemon is broken somehow as it did not reclaim several gigs of wired memory despite of long period of vm thrashing: $ sed 's/:/,/' vmstat-z.txt | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head 1892 abd_chunk 454.629 dnode_t 351.35 zio_buf_512 228.391 zio_buf_16384 173.968 dmu_buf_impl_t 130.25 zio_data_buf_131072 93.6887 VNODE 81.6978 arc_buf_hdr_t_full 74.9368 256 57.4102 4096 From owner-freebsd-stable@freebsd.org Tue Feb 12 18:15:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2BEE14EF902 for ; Tue, 12 Feb 2019 18:15:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7433072645 for ; Tue, 12 Feb 2019 18:15:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CIF1rL072737 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:15:02 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: wollman@bimajority.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CIF04r052700 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:15:00 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Garrett Wollman , freebsd-stable@freebsd.org References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> From: Eugene Grosbein Message-ID: <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> Date: Wed, 13 Feb 2019 01:14:54 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 7433072645 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:15:28 -0000 13.02.2019 0:57, Garrett Wollman wrote: > In article > eugen@grosbein.net writes: > >> Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel >> wired memory over 81 days uptime >> out of 8GB total RAM. > > Not a whole lot of evidence yet, but anecdotally I'm seeing the same > thing on some huge-memory NFS servers running releng/11.2. They seem > to run fine for a few weeks, then mysteriously start swapping > continuously, a few hundred pages a second. The continues for hours > at a time, and then stops just as mysteriously. Over time the total > memory dedicated to ZFS ARC goes down but there's no decrease in wired > memory. I've tried disabling swap, but this seems to make the server > unstable. I have yet to find any obvious commonality (aside from the > fact that these are all large-memory NFS servers which don't do much > of anything else -- the only software running on them is related to > managing and monitoring the NFS service). I started to understand the issue. FreeBSD 11 has uma(9) zone allocator for kernel subsystems and vmstat -z shows some stats for UMA zones. When some subsystem using UMA frees its memory (including networking mbufs or ZFS ARC), some kernel memory blocks are moved from USED to FREE category inside corresponding UMA zone (see vmstat -z again) but this memory stays unavailable to other consumers including userland applications until pagedaemon reclaims this "FREE" memory back to global free pool. This part seems to be broken in 11.2-STABLE. Use following command to see how much memory is wasted in your case: vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head From owner-freebsd-stable@freebsd.org Tue Feb 12 18:18:11 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C584114EFA14 for ; Tue, 12 Feb 2019 18:18:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-it1-x135.google.com (mail-it1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9718072779; Tue, 12 Feb 2019 18:18:09 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-it1-x135.google.com with SMTP id i2so9934068ite.5; Tue, 12 Feb 2019 10:18:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=wlVgQwIv3PyPELFXwQWEFnRYPugM4Vk9H0YCeCJlvus=; b=jDXE++00hovMQwhhr1Y3DL46TQ++8KA+m1IJf3Gily/IzoycpzL5aD9Dt3GmFYG1Ep Djry7MWdKvU4UXdB8t+4DAJiBkikE7s/QU0L5PGRJsoN53Wf/vq2R27gdYvmydY9ed+r QkmaM9o4XKhcsgw9XngFDM3UpXs2Pk63o0Gu77bsGWjlHNTzoBnbWytnAspBIJACfQCn sJwyTLs+yG33M6CZGN2pPvPm4PiQ+PL+drzAJ6Aei+vr7XU5pueJF4bgX7oOpktOARVT u6A9YuEtwiDv6X2ab5Z8G1OeT9s5tKE1+j4yvE7ZJyV4yXj3jyOit9ECBg0OovxxVhSZ XYGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=wlVgQwIv3PyPELFXwQWEFnRYPugM4Vk9H0YCeCJlvus=; b=j2R4H2IqMSVD7gSDi9/hx9d9sXmYcqI4mE/pQstWG82zblWRwq+veznbkV3QFazRJI +TbPxTBp/fmB/IpSNr1P6goLSzeuE3+45ghHnE7h8DVYJSCYtL0rj6DPzIhQII3nil8I WCzJ0rErzbJdJcYzRYjqKUThJY3QJLG6AB5s6KWsBRstLRzj4Q2vVRVODwju1cnt2DD9 INov+tLzUfOksJfu0fiDJk/YOuaST12psMyiTyAX34dDUyRIejMbvPmdwO2+T6dsw6JZ RY1xTknzlxGOoEHPAAnK7+PTg5kMSa6EwuhNFrHWHpESFBmdNzgIbzMPdjHb41NYnOmx N6Dg== X-Gm-Message-State: AHQUAub+8ET2Nhz1CsE1STXQZSj44RrVoAZgdb1wTpYR4nLmK5bvDs47 h0k9aw0lyrsS54hDHP43M1y5+8pZ X-Google-Smtp-Source: AHgI3IZDYXfYUnpJ9/II1IdZ5ckFwsbw+J1t3xwhR94AIX3NEColhFpFEe8e7ps6z/WbqGUX3cAPQA== X-Received: by 2002:a05:660c:30e:: with SMTP id u14mr100677itj.13.1549995488749; Tue, 12 Feb 2019 10:18:08 -0800 (PST) Received: from raichu (toroon0560w-lp140-01-69-159-36-102.dsl.bell.ca. [69.159.36.102]) by smtp.gmail.com with ESMTPSA id h8sm8469ith.2.2019.02.12.10.18.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 10:18:08 -0800 (PST) Sender: Mark Johnston Date: Tue, 12 Feb 2019 13:18:05 -0500 From: Mark Johnston To: Eugene Grosbein Cc: FreeBSD stable , kib@freebsd.org Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212181805.GB29847@raichu> References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 9718072779 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=jDXE++00; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::135 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-5.28 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[5.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.94)[-0.944,0]; IP_SCORE(-2.62)[ip: (-8.56), ipnet: 2607:f8b0::/32(-2.52), asn: 15169(-1.97), country: US(-0.07)]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:18:11 -0000 On Wed, Feb 13, 2019 at 01:03:37AM +0700, Eugene Grosbein wrote: > 12.02.2019 23:34, Mark Johnston wrote: > > > I suspect that the "leaked" memory is simply being used to cache UMA > > items. Note that the values in the FREE column of vmstat -z output are > > quite large. The cached items are reclaimed only when the page daemon > > wakes up to reclaim memory; if there are no memory shortages, large > > amounts of memory may accumulate in UMA caches. In this case, the sum > > of the product of columns 2 and 5 gives a total of roughly 4GB cached. > > > >> as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt > >> and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt > > It seems page daemon is broken somehow as it did not reclaim several gigs of wired memory > despite of long period of vm thrashing: Depending on the system's workload, it is possible for the caches to grow quite quickly after a reclaim. If you are able to run kgdb on the kernel, you can find the time of the last reclaim by comparing the values of lowmem_uptime and time_uptime. > $ sed 's/:/,/' vmstat-z.txt | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > 1892 abd_chunk > 454.629 dnode_t > 351.35 zio_buf_512 > 228.391 zio_buf_16384 > 173.968 dmu_buf_impl_t > 130.25 zio_data_buf_131072 > 93.6887 VNODE > 81.6978 arc_buf_hdr_t_full > 74.9368 256 > 57.4102 4096 > From owner-freebsd-stable@freebsd.org Tue Feb 12 18:18:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5ABEA14EFA22 for ; Tue, 12 Feb 2019 18:18:13 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C45CE7276E for ; Tue, 12 Feb 2019 18:18:02 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CIHhKN072752 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:17:44 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: wollman@bimajority.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CIHhko052727 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:17:43 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Garrett Wollman , freebsd-stable@freebsd.org References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> From: Eugene Grosbein Message-ID: <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> Date: Wed, 13 Feb 2019 01:17:36 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: C45CE7276E X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.92)[-0.918,0]; IP_SCORE(-1.39)[ip: (-2.33), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:18:13 -0000 13.02.2019 1:14, Eugene Grosbein wrote: > Use following command to see how much memory is wasted in your case: > > vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head Oops, small correction: vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head From owner-freebsd-stable@freebsd.org Tue Feb 12 18:29:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8484714F0144 for ; Tue, 12 Feb 2019 18:29:22 +0000 (UTC) (envelope-from pi@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E281D7333C for ; Tue, 12 Feb 2019 18:29:21 +0000 (UTC) (envelope-from pi@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1gtcnz-0000Aq-6h; Tue, 12 Feb 2019 19:29:15 +0100 Date: Tue, 12 Feb 2019 19:29:15 +0100 From: Kurt Jaeger To: Eugene Grosbein Cc: Garrett Wollman , freebsd-stable@freebsd.org Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212182915.GM2748@home.opsec.eu> References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:29:22 -0000 Hi! > > Use following command to see how much memory is wasted in your case: > > > > vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > > Oops, small correction: > > vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head On a 8 GB 11.2p8 box doing mostly routing: 42.3047 abd_chunk 40 zio_buf_131072 31.75 zio_data_buf_131072 19.8901 swblk 12.9224 RADIX NODE 11.7344 zio_buf_16384 10.0664 zio_data_buf_12288 9.84375 zio_data_buf_40960 9.375 zio_data_buf_81920 7.96875 zio_data_buf_98304 So, how do I understand this ? Please note that I use: vfs.zfs.arc_max=1216348160 -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-stable@freebsd.org Tue Feb 12 18:38:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B6AC14F07C3 for ; Tue, 12 Feb 2019 18:38:03 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B240573C71 for ; Tue, 12 Feb 2019 18:38:02 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CIbcL7072910 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:37:43 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: pi@opsec.eu Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CIbbgJ052908 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:37:37 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Kurt Jaeger References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> Cc: Garrett Wollman , freebsd-stable@freebsd.org From: Eugene Grosbein Message-ID: <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> Date: Wed, 13 Feb 2019 01:37:31 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212182915.GM2748@home.opsec.eu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: B240573C71 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:38:03 -0000 13.02.2019 1:29, Kurt Jaeger wrote: > On a 8 GB 11.2p8 box doing mostly routing: > > 42.3047 abd_chunk > 40 zio_buf_131072 > 31.75 zio_data_buf_131072 > 19.8901 swblk > 12.9224 RADIX NODE > 11.7344 zio_buf_16384 > 10.0664 zio_data_buf_12288 > 9.84375 zio_data_buf_40960 > 9.375 zio_data_buf_81920 > 7.96875 zio_data_buf_98304 > > So, how do I understand this ? Please note that I use: > > vfs.zfs.arc_max=1216348160 Each line shows how many megabytes is allocated but currently unused by corresponding UMA zone (and unavailable for other consumers). Your numbers are pretty low, you have nothing to worry about IMHO. I have hundreds of megabytes and gigabytes there. From owner-freebsd-stable@freebsd.org Tue Feb 12 18:40:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3ED5214F0974 for ; Tue, 12 Feb 2019 18:40:24 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 475F1742DD; Tue, 12 Feb 2019 18:40:23 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CIeDc2072942 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:40:13 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CIeCHj052923 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:40:12 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> Date: Wed, 13 Feb 2019 01:40:06 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212181805.GB29847@raichu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 475F1742DD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.92 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.39)[ip: (-2.31), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.93)[-0.933,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:40:24 -0000 13.02.2019 1:18, Mark Johnston wrote: > On Wed, Feb 13, 2019 at 01:03:37AM +0700, Eugene Grosbein wrote: >> 12.02.2019 23:34, Mark Johnston wrote: >> >>> I suspect that the "leaked" memory is simply being used to cache UMA >>> items. Note that the values in the FREE column of vmstat -z output are >>> quite large. The cached items are reclaimed only when the page daemon >>> wakes up to reclaim memory; if there are no memory shortages, large >>> amounts of memory may accumulate in UMA caches. In this case, the sum >>> of the product of columns 2 and 5 gives a total of roughly 4GB cached. >>> >>>> as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt >>>> and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt >> >> It seems page daemon is broken somehow as it did not reclaim several gigs of wired memory >> despite of long period of vm thrashing: > > Depending on the system's workload, it is possible for the caches to > grow quite quickly after a reclaim. If you are able to run kgdb on the > kernel, you can find the time of the last reclaim by comparing the > values of lowmem_uptime and time_uptime. Yes, I have debugger compiled into running kernel and have console access. What commands should I use? From owner-freebsd-stable@freebsd.org Tue Feb 12 18:43:00 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81AD114F0C66 for ; Tue, 12 Feb 2019 18:43:00 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-it1-x141.google.com (mail-it1-x141.google.com [IPv6:2607:f8b0:4864:20::141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9685F747A4 for ; Tue, 12 Feb 2019 18:42:59 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-it1-x141.google.com with SMTP id a6so10173830itl.4 for ; Tue, 12 Feb 2019 10:42:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=f+AL2DQeyA2CwUFSwRtJocZPe80+6XymJjV2DEPQWJI=; b=UK6yb9s2N+ZNIttH1vc9HtxDWVb4wUlVGnRMDVJX0WvcP8u4KGyEqqOeCwktHxcMHw tX6ObyKNetsbj0+xXq5DolXcjZPZcmhDRWTFhtavQuv80to3t4siqDfLbVwGbp1GfJJt HaeXfD9keOkN5OyAvSlAeiE9w9jsxI3M1kIe33hQQQDlzrGCZxATsjzhQ4e+8Ga1MScr Fj3IsbZAnGEFhiPa+CAq1Qqe6caLcHyeDUfxc1Mu/Ut63ryYuFPlaWnYmnEp13T5nppM ruGcc4NtiA516JO9FSmipZAU/PwgJLBspXc2WcVurGiyKFQMc4bAA7FAiGYOE71FLD+W 17Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=f+AL2DQeyA2CwUFSwRtJocZPe80+6XymJjV2DEPQWJI=; b=Qt9WK6e1hRyKA98MvSXhUPY6ijILggb5FiCSacu558xBBf50dPkc5frfjsNoXtlcUS bT29HT3u7bxSBkmyfsS2AhIod+LcCvHOGXIWmZR9snLoRvfdiAnWGlb7zpkVWeqTUhE1 pC65ocnHme4BsYBeTwrzFI8453Tcl1A3eU+yEGKTYJR3+tADxI6isQXz07SBneurn9uf af99DXQ2c8fORP1D69Dt1wTtQizYg1o3buOR8faQtfTMCoNB+hPMBEiC3O98Ge+fxx7x GF79oAtPjfGYHeAcvSxI4a6CTalxWB3Ks05bOCY+xgMlgu5a2fnjr6znrJLW7m8/mqN+ iSlQ== X-Gm-Message-State: AHQUAuaHCYQibCDi7B5AdBTN9qAQUG5+Ugxsegq73PrgODZmUYrPipAP TuBnhxBr5YWJsggmbiK9khpdBevC X-Google-Smtp-Source: AHgI3IaZeNmZ+jCrPFp70oKpTQ2TWJEfYlQNplzhfz0MGgSX/mMEsn9si9WfnFVls2lx/7Y0ZXNVzw== X-Received: by 2002:a02:5184:: with SMTP id s126mr2608420jaa.12.1549996978959; Tue, 12 Feb 2019 10:42:58 -0800 (PST) Received: from raichu (toroon0560w-lp140-01-69-159-36-102.dsl.bell.ca. [69.159.36.102]) by smtp.gmail.com with ESMTPSA id h123sm1723587itb.36.2019.02.12.10.42.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 10:42:58 -0800 (PST) Sender: Mark Johnston Date: Tue, 12 Feb 2019 13:42:56 -0500 From: Mark Johnston To: Eugene Grosbein Cc: FreeBSD stable Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212184256.GC29847@raichu> References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 9685F747A4 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=UK6yb9s2; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::141 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.38 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.885,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.67)[-0.671,0]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.12)[ip: (3.93), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-1.97), country: US(-0.07)]; MID_RHS_NOT_FQDN(0.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:43:00 -0000 On Wed, Feb 13, 2019 at 01:40:06AM +0700, Eugene Grosbein wrote: > 13.02.2019 1:18, Mark Johnston wrote: > > > On Wed, Feb 13, 2019 at 01:03:37AM +0700, Eugene Grosbein wrote: > >> 12.02.2019 23:34, Mark Johnston wrote: > >> > >>> I suspect that the "leaked" memory is simply being used to cache UMA > >>> items. Note that the values in the FREE column of vmstat -z output are > >>> quite large. The cached items are reclaimed only when the page daemon > >>> wakes up to reclaim memory; if there are no memory shortages, large > >>> amounts of memory may accumulate in UMA caches. In this case, the sum > >>> of the product of columns 2 and 5 gives a total of roughly 4GB cached. > >>> > >>>> as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt > >>>> and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt > >> > >> It seems page daemon is broken somehow as it did not reclaim several gigs of wired memory > >> despite of long period of vm thrashing: > > > > Depending on the system's workload, it is possible for the caches to > > grow quite quickly after a reclaim. If you are able to run kgdb on the > > kernel, you can find the time of the last reclaim by comparing the > > values of lowmem_uptime and time_uptime. > > Yes, I have debugger compiled into running kernel and have console access. > What commands should I use? I meant kgdb(1). If you can run that, try: (kgdb) p time_uptime (kgdb) p lowmem_uptime If you are willing to drop the system into DDB, do so and run db> x time_uptime db> x lowmem_uptime From owner-freebsd-stable@freebsd.org Tue Feb 12 18:48:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2490E14F109C for ; Tue, 12 Feb 2019 18:48:36 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3138474C94; Tue, 12 Feb 2019 18:48:35 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CImSsZ073000 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 19:48:29 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CImRGP053046 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 01:48:27 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> <20190212184256.GC29847@raichu> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: Date: Wed, 13 Feb 2019 01:48:21 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212184256.GC29847@raichu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 3138474C94 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.38)[ip: (-2.28), ipnet: 2a01:4f8::/29(-2.39), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.92)[-0.924,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:48:36 -0000 13.02.2019 1:42, Mark Johnston wrote: >> Yes, I have debugger compiled into running kernel and have console access. >> What commands should I use? > > I meant kgdb(1). If you can run that, try: > > (kgdb) p time_uptime > (kgdb) p lowmem_uptime > > If you are willing to drop the system into DDB, do so and run > > db> x time_uptime > db> x lowmem_uptime I will reach the console next day only. Is it wise to use kgdb over ssh for running remote system? :-) From owner-freebsd-stable@freebsd.org Tue Feb 12 18:50:45 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65AC714F119A for ; Tue, 12 Feb 2019 18:50:45 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE39474DB0; Tue, 12 Feb 2019 18:50:44 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id B601C13D43; Tue, 12 Feb 2019 18:50:44 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.23.186] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id C62B5BB0D; Tue, 12 Feb 2019 21:50:41 +0300 (MSK) Reply-To: lev@FreeBSD.org Subject: Re: 11.2-STABLE kernel wired memory leak To: Eugene Grosbein , Kurt Jaeger Cc: freebsd-stable@freebsd.org, Garrett Wollman References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> From: Lev Serebryakov Openpgp: preference=signencrypt Autocrypt: addr=lev@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFKbGksBEADeguVs+XyJc3mL3iiOBqDd16wSk97YTJYOi4VsHsINzJr09oFvNDiaDBIi fLn2p8XcJvehcsF2GSgrfXfw+uK4O1jyNIKJmiYA0EtE+ZbRtvDrrE0w6Q8+SDeKA21SWh3Y vSQ0DJUontbgW55ER2CbEiIUTIn34uQ0kmESAaw/v5p/9ue8yPTmURvv130FqPFz8VPzltqL NxyGt54TxPfKAzAHEIwxlEZ63JOwzloKh1UDBExcsf9nJO08/TAVgR5UZ5njFBPzaaquhRoP qPJLEQQDqxPIlvMNtHKf7iIebE4BHeqgCdJA0BoiR6gpa0wlsZtdrTPK3n4wYSphLvGbhfOZ YW/hbcu7HYS/FImkVxB3iY17kcC1UTnx4ZaYeASPBGOOPbXky1lLfmDGWIFT//70yx+G17qD OZzF1SvJJhGvh6ilFYaWMX7T+nIp6Mcafc4D7AakXM+XdubNXOMlCJhzPcZ0skgAEnYV587w V7em5fDVwQccwvtfezzqKeJAU5TGiywBHSR5Svzk2FwRNf6M//hWkpq0SRR63iOhkHGOAEBi 69GfEIwH2/w24rLxP0E+Hqq8n+EWNkPatw1Mhcl5PKkdvGCjJUaGNMkpBffjyYo254JXRscR eEnwdIkJt4ErDvjb2/UrOFq31wWMOiLzJeVchAgvTHBMRfP9aQARAQABtCFMZXYgU2VyZWJy eWFrb3YgPGxldkBGcmVlQlNELm9yZz6JAlYEEwEIAEACGwMHCwkIBwMCAQYVCAIJCgsEFgID AQIeAQIXgBYhBPltHKC19DGLZ0szCuqwPFi/3EePBQJb/TeXBQkPBbfGAAoJEOqwPFi/3EeP LtEQANQLS89OCDSnLDZLyCj7sH5GZbTikJp9Or2McqEOpjgtfY/OgqCe7lfo8f71tTN3gL2t VGtNEtsl6CqKeBdm6fqsetCAed2+waJfbXLZWReLRSFQJ/cbof8MB3h/uEw8Zng1ZbyEU0eG rc41Mhk8bTfxjNRjkXDbp0+2xug+eRr2RYhiY2SxH+iy57AbRIu9gkjfL05VztfvsV5dPxt7 +reHf2Vhtu+qCRtCytGQqDvYWGpVZ8U5fHJxFdsJpn66LAe8w+iRGCbppB5TKeHkqWqDb++u sd5ZHzwep/7qcLBeKggi8+a2C3J1J4TpYBwdPgusrGtBPf2EYSTc/0mW5j86g1l8UDN9qXAI FvWbKf22p5SlYUzf5qRLny4ZdDl1mH13lmJV7ZkZGMKnt0IjJ2x0LRgLGhiSm/D4Lh8yc/MX uhj9BB020XbWcMUPpA7S7YaWeIXbnK3NrPU0yk77prFJblfskOwuaxJIwa2mX+cCAQA95As5 2talifo4Kh8OLSvS1gpbMo64LllTzuGbCWuKecH3fpMlZlD7/10z9lBvkSXA7KgadDfN0yHO xGCK8KE9gXXmEVRlbgGDYOJw9FLWHuSZHUvQTkmr4goXrmMIKa9A4BKO3s9vowP4pJ0vp3kz MIBeEwKSvAeZZfgKbIKggKUuQ5gsAqvF360sG3+UuQINBFKbGksBEAC0a9wfjo2P3JyT7Lc+ QlbFVshGbSbazb4ma7QYG5IZZD5vfLBFkePoG6cnrn3WCXp4A43hszAynCwe4eXyAkv4+gPF 3ZSeNE5Wz3zYG+jh2nm2iGCkyaVykfbA+2chor2DKH5tHpuNMBlF+wSJHZKJmlo/sFIktAnV 1NBVg4/cL+9/hIpvl82cl3hYCD7/e7/qRE+w38CpAAzn65FvbODn7xlY3fsJt+cHPBJ4EBM9 KnTwcce+F+72RQMZQEl7vIAwSRmLdgZHN0MFC533l62SVoKjT0eaOOIBrvesmojhWjfwugib Xr+WRF/tGcW77Bxwe2eQLbEVESqWeMORxRxocx7Q7aACoHmf4G4U1Vzx7zUEfNfHjfjZeQVf AURf/MoUelZSW/BmMIfKCg3lRlWAt+Pq2h2UADPVqAZze45beE/c8z8LZsOZiGoRhYL8NSg6 +ziLTdmYLWdtFGAuZhqOtNp5h6tGj21OksBotcaIa5YjbCmmnImIjGlSBkUKvIhq/RXth5b2 gNwaQdu+Yv4AlZVHRsuVywL/skDFL5+We11bDK6MQ5PzvmntRJcgbyoisn1hiV04OV1LpJJM kJn1j8VlBqDQNT/z+BjB0ru/0anv+5uLj7v0ck06rEo4yiXT/ZAcBM76j7V7FaGbkoba6bUU CQ2H5YYBOKpikjCnpwARAQABiQI8BBgBCAAmAhsMFiEE+W0coLX0MYtnSzMK6rA8WL/cR48F Alv9N7IFCQ8Ft+cACgkQ6rA8WL/cR49wpw//W7QrZHKYUWEVHtPLVMlcM1f7MgwnYlIRe/I6 gykwvt87sqxSYu7eHxfX6JZR2M9UYuUYscSR57gxXKu2Uzqaz2KjvEIXJCwCTsuXZjkQ19oL hAPsucw2AcMI6YqPXbZ7hO0Mh8jMeJzD6vDvx3zvunQxGQoZKxr3BvRUyO7NCYbA7N8ccI3E Tmdjj9JTbtue9WaUeGLszFDiITmkASBdsK07y4ylLAJzUGCvYLBU3gx0bBOB2S8SbxHYmQlj ewjovbp+MbT+CoXNxZp+gDRModGyIMdRIotkRiDbgNl0VWIT0SS1GUUlUHKzQ/ZPUvpvk6Gq s/SYBS7rBQNSoXXmJxNW6I6atVNW7JhInC1tkHxBlbpik4FK07J6NHFTx+9ygyCxbxwNA/Le FGkL3kU0VFXR2dNiAVbeueUPF6tU5Bye5ftgpLKOSAbo1g++EKUkc8335/4oYRBMcGEk4xUr NAvzoFPhA2W3KtklruJ9ThrFt9+//NB5ySlVgTwGWMuBJoskEmNOTBv34/UQPUIuE4xSwy6y H+nRQJooiMFo5QCosslJPlwyV63NS9lYXB6n3QWOCP6sFdWACNUCOFv4uk7LQdY9BsCmt2Tk cLuHmoS+fvUesQXnYV2aQi9HciriPIj2gvJ6WjgiaC/PpePil0fzyrfG/JMyHL0qcgmYoj0= Organization: FreeBSD Message-ID: <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> Date: Tue, 12 Feb 2019 21:50:35 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Va2WCPvVyXZYqrxGTT1uOjcca5jjpwdSH" X-Rspamd-Queue-Id: EE39474DB0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:50:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Va2WCPvVyXZYqrxGTT1uOjcca5jjpwdSH Content-Type: multipart/mixed; boundary="PzkU46LhxU8XwDWAxYGhh2lsCEBvnXSo1"; protected-headers="v1" From: Lev Serebryakov Reply-To: lev@FreeBSD.org To: Eugene Grosbein , Kurt Jaeger Cc: freebsd-stable@freebsd.org, Garrett Wollman Message-ID: <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> Subject: Re: 11.2-STABLE kernel wired memory leak References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> In-Reply-To: <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> --PzkU46LhxU8XwDWAxYGhh2lsCEBvnXSo1 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 12.02.2019 21:37, Eugene Grosbein wrote: >> vfs.zfs.arc_max=3D1216348160 >=20 > Each line shows how many megabytes is allocated but currently unused by= corresponding > UMA zone (and unavailable for other consumers). Your numbers are pretty= low, > you have nothing to worry about IMHO. I have hundreds of megabytes and = gigabytes there. I'm have same problem. According to top(1) I have 29G Wired, but only 17G Total ARC (12G difference! System has 32G of RAM), and this statistic shows: 5487.5 zio_data_buf_524288 920.125 zio_data_buf_131072 626 zio_buf_131072 468 zio_data_buf_1048576 398.391 zio_buf_16384 305.464 dnode_t 227.989 zio_buf_512 171.5 zio_data_buf_458752 141.75 zio_data_buf_393216 116.456 dmu_buf_impl_t So, more than 6G (!) is not used in ARC, but hold by ZFS anyway. --=20 // Lev Serebryakov --PzkU46LhxU8XwDWAxYGhh2lsCEBvnXSo1-- --Va2WCPvVyXZYqrxGTT1uOjcca5jjpwdSH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE+W0coLX0MYtnSzMK6rA8WL/cR48FAlxjFXtfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEY5 NkQxQ0EwQjVGNDMxOEI2NzRCMzMwQUVBQjAzQzU4QkZEQzQ3OEYACgkQ6rA8WL/c R49stg//eNLzNEgNe/dxIeSZn7Kk+JPAuSbCQhNNNyzxDRu68nlOpIfQOf7ihrIG NJx4z0BgD0gALZWJC//fKwtPqihO4gQWFjr0KxaYp17W6UHs3zIEIyFaz615tXv4 E3bwbKZz+Q5qwFY1+SEOuVnJ0u7ZjvgizZKYOuvI5+/8W9r8/TOZaGfzQw2yjx7l 6+QZsVUOnrCIBA+nWlI4+XAqM0Hy5tpb+ek6q8fN18Ey3z5m0kEk4XjdexSLUQCH pi8JyloFJm/Ek5xYHTmfMsqhoNUR/pBeM18+pg/mhi3jat7zIhbi8qUmqrAmV/aX U60RkeWxqiWrO8T/Ezt9/xGg/DlwoY9sjs7tBdIzn6un5fUbarKHBOwO0T2OrLNG OK6KWrWBG3orcORcBgaYvs0VKBUsYlxqj+T6OIqikJMxKIXba+zN0LHzNIreEeT0 tOuemjic/kAvMcFPROqMsoo/gm4MTF7+tUBGklNCKnSpR08q1MOIR/hmi/ApRkwT e1qpLEKf8gh+2dNZUEfTbitYRJYx99SHnygM9iPrhCSJkkUBn4eFe1b+/JYodOBa Xxc3Jc9Rd9TOiMoRrD3F5tnm4vSNU/EQiX7wYHN9ISwS57seOgnYv4O5JondByz1 ubNTo7R/DzXO0fboBjXQm8uriEvGXLyzesBeU/vN2bpNspnO/G4= =RBRK -----END PGP SIGNATURE----- --Va2WCPvVyXZYqrxGTT1uOjcca5jjpwdSH-- From owner-freebsd-stable@freebsd.org Tue Feb 12 18:53:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDDD414F1454 for ; Tue, 12 Feb 2019 18:53:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7100675248; Tue, 12 Feb 2019 18:53:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 3501813D44; Tue, 12 Feb 2019 18:53:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.23.186] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id E3D7BBB1C; Tue, 12 Feb 2019 21:53:13 +0300 (MSK) Reply-To: lev@FreeBSD.org Subject: Re: 11.2-STABLE kernel wired memory leak To: Eugene Grosbein , Mark Johnston Cc: FreeBSD stable References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> <20190212184256.GC29847@raichu> From: Lev Serebryakov Openpgp: preference=signencrypt Autocrypt: addr=lev@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFKbGksBEADeguVs+XyJc3mL3iiOBqDd16wSk97YTJYOi4VsHsINzJr09oFvNDiaDBIi fLn2p8XcJvehcsF2GSgrfXfw+uK4O1jyNIKJmiYA0EtE+ZbRtvDrrE0w6Q8+SDeKA21SWh3Y vSQ0DJUontbgW55ER2CbEiIUTIn34uQ0kmESAaw/v5p/9ue8yPTmURvv130FqPFz8VPzltqL NxyGt54TxPfKAzAHEIwxlEZ63JOwzloKh1UDBExcsf9nJO08/TAVgR5UZ5njFBPzaaquhRoP qPJLEQQDqxPIlvMNtHKf7iIebE4BHeqgCdJA0BoiR6gpa0wlsZtdrTPK3n4wYSphLvGbhfOZ YW/hbcu7HYS/FImkVxB3iY17kcC1UTnx4ZaYeASPBGOOPbXky1lLfmDGWIFT//70yx+G17qD OZzF1SvJJhGvh6ilFYaWMX7T+nIp6Mcafc4D7AakXM+XdubNXOMlCJhzPcZ0skgAEnYV587w V7em5fDVwQccwvtfezzqKeJAU5TGiywBHSR5Svzk2FwRNf6M//hWkpq0SRR63iOhkHGOAEBi 69GfEIwH2/w24rLxP0E+Hqq8n+EWNkPatw1Mhcl5PKkdvGCjJUaGNMkpBffjyYo254JXRscR eEnwdIkJt4ErDvjb2/UrOFq31wWMOiLzJeVchAgvTHBMRfP9aQARAQABtCFMZXYgU2VyZWJy eWFrb3YgPGxldkBGcmVlQlNELm9yZz6JAlYEEwEIAEACGwMHCwkIBwMCAQYVCAIJCgsEFgID AQIeAQIXgBYhBPltHKC19DGLZ0szCuqwPFi/3EePBQJb/TeXBQkPBbfGAAoJEOqwPFi/3EeP LtEQANQLS89OCDSnLDZLyCj7sH5GZbTikJp9Or2McqEOpjgtfY/OgqCe7lfo8f71tTN3gL2t VGtNEtsl6CqKeBdm6fqsetCAed2+waJfbXLZWReLRSFQJ/cbof8MB3h/uEw8Zng1ZbyEU0eG rc41Mhk8bTfxjNRjkXDbp0+2xug+eRr2RYhiY2SxH+iy57AbRIu9gkjfL05VztfvsV5dPxt7 +reHf2Vhtu+qCRtCytGQqDvYWGpVZ8U5fHJxFdsJpn66LAe8w+iRGCbppB5TKeHkqWqDb++u sd5ZHzwep/7qcLBeKggi8+a2C3J1J4TpYBwdPgusrGtBPf2EYSTc/0mW5j86g1l8UDN9qXAI FvWbKf22p5SlYUzf5qRLny4ZdDl1mH13lmJV7ZkZGMKnt0IjJ2x0LRgLGhiSm/D4Lh8yc/MX uhj9BB020XbWcMUPpA7S7YaWeIXbnK3NrPU0yk77prFJblfskOwuaxJIwa2mX+cCAQA95As5 2talifo4Kh8OLSvS1gpbMo64LllTzuGbCWuKecH3fpMlZlD7/10z9lBvkSXA7KgadDfN0yHO xGCK8KE9gXXmEVRlbgGDYOJw9FLWHuSZHUvQTkmr4goXrmMIKa9A4BKO3s9vowP4pJ0vp3kz MIBeEwKSvAeZZfgKbIKggKUuQ5gsAqvF360sG3+UuQINBFKbGksBEAC0a9wfjo2P3JyT7Lc+ QlbFVshGbSbazb4ma7QYG5IZZD5vfLBFkePoG6cnrn3WCXp4A43hszAynCwe4eXyAkv4+gPF 3ZSeNE5Wz3zYG+jh2nm2iGCkyaVykfbA+2chor2DKH5tHpuNMBlF+wSJHZKJmlo/sFIktAnV 1NBVg4/cL+9/hIpvl82cl3hYCD7/e7/qRE+w38CpAAzn65FvbODn7xlY3fsJt+cHPBJ4EBM9 KnTwcce+F+72RQMZQEl7vIAwSRmLdgZHN0MFC533l62SVoKjT0eaOOIBrvesmojhWjfwugib Xr+WRF/tGcW77Bxwe2eQLbEVESqWeMORxRxocx7Q7aACoHmf4G4U1Vzx7zUEfNfHjfjZeQVf AURf/MoUelZSW/BmMIfKCg3lRlWAt+Pq2h2UADPVqAZze45beE/c8z8LZsOZiGoRhYL8NSg6 +ziLTdmYLWdtFGAuZhqOtNp5h6tGj21OksBotcaIa5YjbCmmnImIjGlSBkUKvIhq/RXth5b2 gNwaQdu+Yv4AlZVHRsuVywL/skDFL5+We11bDK6MQ5PzvmntRJcgbyoisn1hiV04OV1LpJJM kJn1j8VlBqDQNT/z+BjB0ru/0anv+5uLj7v0ck06rEo4yiXT/ZAcBM76j7V7FaGbkoba6bUU CQ2H5YYBOKpikjCnpwARAQABiQI8BBgBCAAmAhsMFiEE+W0coLX0MYtnSzMK6rA8WL/cR48F Alv9N7IFCQ8Ft+cACgkQ6rA8WL/cR49wpw//W7QrZHKYUWEVHtPLVMlcM1f7MgwnYlIRe/I6 gykwvt87sqxSYu7eHxfX6JZR2M9UYuUYscSR57gxXKu2Uzqaz2KjvEIXJCwCTsuXZjkQ19oL hAPsucw2AcMI6YqPXbZ7hO0Mh8jMeJzD6vDvx3zvunQxGQoZKxr3BvRUyO7NCYbA7N8ccI3E Tmdjj9JTbtue9WaUeGLszFDiITmkASBdsK07y4ylLAJzUGCvYLBU3gx0bBOB2S8SbxHYmQlj ewjovbp+MbT+CoXNxZp+gDRModGyIMdRIotkRiDbgNl0VWIT0SS1GUUlUHKzQ/ZPUvpvk6Gq s/SYBS7rBQNSoXXmJxNW6I6atVNW7JhInC1tkHxBlbpik4FK07J6NHFTx+9ygyCxbxwNA/Le FGkL3kU0VFXR2dNiAVbeueUPF6tU5Bye5ftgpLKOSAbo1g++EKUkc8335/4oYRBMcGEk4xUr NAvzoFPhA2W3KtklruJ9ThrFt9+//NB5ySlVgTwGWMuBJoskEmNOTBv34/UQPUIuE4xSwy6y H+nRQJooiMFo5QCosslJPlwyV63NS9lYXB6n3QWOCP6sFdWACNUCOFv4uk7LQdY9BsCmt2Tk cLuHmoS+fvUesQXnYV2aQi9HciriPIj2gvJ6WjgiaC/PpePil0fzyrfG/JMyHL0qcgmYoj0= Organization: FreeBSD Message-ID: <4411160d-a485-fd29-9be0-8430841841d2@FreeBSD.org> Date: Tue, 12 Feb 2019 21:53:13 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="aXrlqg4eP8u5JukztIQK0hMEDinPH4l91" X-Rspamd-Queue-Id: 7100675248 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:53:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --aXrlqg4eP8u5JukztIQK0hMEDinPH4l91 Content-Type: multipart/mixed; boundary="djRKJqmFRCLoaQZig5VzU89dZPRmgCDP4"; protected-headers="v1" From: Lev Serebryakov Reply-To: lev@FreeBSD.org To: Eugene Grosbein , Mark Johnston Cc: FreeBSD stable Message-ID: <4411160d-a485-fd29-9be0-8430841841d2@FreeBSD.org> Subject: Re: 11.2-STABLE kernel wired memory leak References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> <20190212184256.GC29847@raichu> In-Reply-To: --djRKJqmFRCLoaQZig5VzU89dZPRmgCDP4 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 12.02.2019 21:48, Eugene Grosbein wrote: > I will reach the console next day only. Is it wise to use kgdb over ssh= for running remote system? :-) It works for me :-) BTW, my is: (kgdb) p time_uptime $1 =3D 81369 (kgdb) p lowmem_uptime $2 =3D 0 (yes, this system have been rebooted less than a day ago). --=20 // Lev Serebryakov --djRKJqmFRCLoaQZig5VzU89dZPRmgCDP4-- --aXrlqg4eP8u5JukztIQK0hMEDinPH4l91 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE+W0coLX0MYtnSzMK6rA8WL/cR48FAlxjFhlfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEY5 NkQxQ0EwQjVGNDMxOEI2NzRCMzMwQUVBQjAzQzU4QkZEQzQ3OEYACgkQ6rA8WL/c R4+Uyg//bfwm32w3f5CZ6dyHjUqBijfsvvVv81W+nVZVePtgv63Ijj4InAvPU88Q 4nUMpy9uw/m2X0/pm9At5cZYEHrKuGESr/KMNH7oVhjSB83ZAUkaIhuv1evJWtvW AZnUk10/vWYmN87o133OKjqHte9VOJwIN/CouQJK1saN2PxN+/tBAiRedJLSjzfo 1RCHiPqEzcXGvDaVzY2/BohbTuR88sG3P4LPGVBF+D7+nj7IhFRm9fvkAwNI8363 f12xOn4OQnNV5D6yfysdpg5EvqalYxBNc4IQmor7cAEgXWW0stM4qCVZSu9SPBUd IzKMgVvIL+vhix+2A0SeEna3+HwYvanuhGNEk4KRUZfKgzWgiShIRvZoZrR1g4Mv X8WAt5B7iRsc4V1xn64EXAoVpDr7GIPP/go7cQGNnxn7A6goqyPY7m/K3tuCp6lo IqBUfdhEv2dL3esgLcYxlYaGZVqaMzvYc6KLMtaMc04V/U6QgfalZ+eFx7BWZ9wD 7ehEgLQ9xZ++rheLtzMtsIFC4rD6zbRAygsNmk7NusilITwvLlaPogYJlGItyG0C 9MrJT/6EC7+QrGEbmNRYh4E7tnmLvX/0nrhqVruffaQR+fAFWxfll091HFxo8lKI S9aPAEpTRdj0rNWuWIQU7on9KU8ubNeEutlbpy9w8i5CmkEC8wE= =/VXw -----END PGP SIGNATURE----- --aXrlqg4eP8u5JukztIQK0hMEDinPH4l91-- From owner-freebsd-stable@freebsd.org Tue Feb 12 18:53:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51EF014F1492 for ; Tue, 12 Feb 2019 18:53:42 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-it1-x12d.google.com (mail-it1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58B0675282 for ; Tue, 12 Feb 2019 18:53:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-it1-x12d.google.com with SMTP id i2so10235835ite.5 for ; Tue, 12 Feb 2019 10:53:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=yfWIY6nDswSWOjOoeJ/LVMU1Cv4sus8JelcjFa+04kg=; b=kABo+9W9CQzEF24G9sXxqHKhDx+0+UyF2QS6oAaFjA+XgHLI/08eeWDZpgIjvbq3Uy d5L4HvAtSLFpHMlTImofCEYEQt+hzB+Ts0CJb/t7WsIHWe7tFxI7ENVquqNIBqLevmB5 rXS05i37mJW05ZkFOElaA4QCePsSX0oZHm5cUWGvGIjXzIIYiBDmhAUmvNa0gk9yZ93E pq8oiaEbMyt4gO3MISrIMD3dUDMLbzx2aVHIwyaUTMic5mIFi0xr2oN8CRimGlgh/Tok UwQ6Sirx1Ln68REejyp5DqGYom7GqKmJ17rltQu6kXdC1oAz6+MwkAyOXrRqoYswp5TJ b38g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=yfWIY6nDswSWOjOoeJ/LVMU1Cv4sus8JelcjFa+04kg=; b=OJIP7uO1om0TTlEqwacNNGsLsljLiv/p0+DS4RluLMRjOCW7CF1Z5aSQ+GldQ5Q5E5 OwhBgJ/zQDfPJwIhP6nrVLdt2r4DiMnh05so36Puhvl37JP4Fqiq3fLmoNbg1tsl2odY YvidhmZ/X5Oc+Z1MjzzlD3KTB1TiGx15Hkihu8Sk3KHvdwusJmM3lYF1V3ULLPLAWr3p Ip0UI1rXfxHko0S8jhWVlDplBzJ5Y+UMGzRhIfnLZCTvxRPM+/CLcsU4AERXgxBLe2hO ZxvMI3FT4tnysjR8+G6vL03w6qHdTGCehxDa/ktfLB+dwQQqzX9iGFMei6LTA4+4X24j P/+g== X-Gm-Message-State: AHQUAuYT9Fzjrc2ZsdLkwVxd6UhUCLOhfI9AayoI6CarraP4a8Xy9pi1 brcRp/LfT3f5ACuWFrCrF4Y= X-Google-Smtp-Source: AHgI3IbOFNWg2mO9ZcQ134lQj7AJqNN7iyohvDIGbXzioEu3K6HkTw0xiULEh1tmA8t/sz3CmzVKcw== X-Received: by 2002:a24:301:: with SMTP id e1mr144675ite.109.1549997620642; Tue, 12 Feb 2019 10:53:40 -0800 (PST) Received: from raichu (toroon0560w-lp140-01-69-159-36-102.dsl.bell.ca. [69.159.36.102]) by smtp.gmail.com with ESMTPSA id s26sm6148951ioe.86.2019.02.12.10.53.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 10:53:40 -0800 (PST) Sender: Mark Johnston Date: Tue, 12 Feb 2019 13:53:38 -0500 From: Mark Johnston To: Eugene Grosbein Cc: FreeBSD stable Subject: Re: 11.2-STABLE kernel wired memory leak Message-ID: <20190212185337.GD29847@raichu> References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> <973d9dc6-8dd4-8cdf-b279-f5c9483da884@grosbein.net> <20190212184256.GC29847@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 58B0675282 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=kABo+9W9; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::12d as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-5.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.976,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.61)[ip: (-8.50), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-1.97), country: US(-0.07)]; MID_RHS_NOT_FQDN(0.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 18:53:42 -0000 On Wed, Feb 13, 2019 at 01:48:21AM +0700, Eugene Grosbein wrote: > 13.02.2019 1:42, Mark Johnston wrote: > > >> Yes, I have debugger compiled into running kernel and have console access. > >> What commands should I use? > > > > I meant kgdb(1). If you can run that, try: > > > > (kgdb) p time_uptime > > (kgdb) p lowmem_uptime > > > > If you are willing to drop the system into DDB, do so and run > > > > db> x time_uptime > > db> x lowmem_uptime > > I will reach the console next day only. Is it wise to use kgdb over ssh for running remote system? :-) It should be fine. kgdb opens /dev/(k)mem read-only. From owner-freebsd-stable@freebsd.org Tue Feb 12 19:01:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A95F314F18A8 for ; Tue, 12 Feb 2019 19:01:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0E4907596B for ; Tue, 12 Feb 2019 19:01:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x1CJ1VfS035474 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 12 Feb 2019 21:01:34 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x1CJ1VfS035474 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x1CJ1Vjq035473; Tue, 12 Feb 2019 21:01:31 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 12 Feb 2019 21:01:31 +0200 From: Konstantin Belousov To: Eugene Grosbein Cc: FreeBSD stable Subject: Re: amd64, run-time linker and 32bit Message-ID: <20190212190131.GC24863@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:01:40 -0000 On Mon, Feb 11, 2019 at 12:31:19AM +0700, Eugene Grosbein wrote: > Hi! > > Why our 32-bit run-time linker looks for shared libraries in the /usr/local/lib despite of its absence in /var/run/ld-elf32.so.hints > while 32-bit binary is started under FreeBSD 11.2-STABLE/amd64 ? Most likely because you configured your system this way, or because your binary sets rpath this way. Without the data, we can only use a physhic service. > > If it finds 64-bit version of library in /usr/local/lib, it fails immediately > and does not even re-try to look at other directories noted in /var/run/ld-elf32.so.hints > such as /usr/lib32:/usr/local/lib/compat/lib32:/usr/local/lib/compat/lib32/compat/pkg:/usr/local/lib32/compat > where right 32-bit version is located. > > As workaround, I can use /etc/libmap32.conf and then the binary starts just fine > but there are so many libraries. It should not even try to look to /usr/local/lib > if it is not in the /var/run/ld-elf32.so.hints, should it? Compat32 linker default search path is /lib32:/usr/lib32 unless overritten or reconfigured (not the same as for native linker for 32bit). From owner-freebsd-stable@freebsd.org Tue Feb 12 19:10:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A005D14F1CC0 for ; Tue, 12 Feb 2019 19:10:05 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B23975E2F; Tue, 12 Feb 2019 19:10:04 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CJ9vKv073153 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 20:09:57 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CJ9u2j053181 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 02:09:56 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <20190212181805.GB29847@raichu> Cc: FreeBSD stable , kib@freebsd.org From: Eugene Grosbein Message-ID: Date: Wed, 13 Feb 2019 02:09:50 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212181805.GB29847@raichu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4B23975E2F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.38)[ip: (-2.25), ipnet: 2a01:4f8::/29(-2.39), asn: 24940(-2.23), country: DE(-0.01)]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.91)[-0.912,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:10:05 -0000 13.02.2019 1:18, Mark Johnston wrote: > Depending on the system's workload, it is possible for the caches to > grow quite quickly after a reclaim. If you are able to run kgdb on the > kernel, you can find the time of the last reclaim by comparing the > values of lowmem_uptime and time_uptime. (kgdb) p time_uptime $1 = 7019265 (kgdb) p lowmem_uptime $2 = 7000568 (kgdb) quit # uptime 2:08 up 81 days, 5:48, 5 users, load averages: 0,19 0,13 0,09 From owner-freebsd-stable@freebsd.org Tue Feb 12 19:18:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5975E14BD03B for ; Tue, 12 Feb 2019 19:18:32 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DB02A7637B; Tue, 12 Feb 2019 19:18:30 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CJIGTq073220 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 20:18:17 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: lev@FreeBSD.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CJIGas053236 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 02:18:16 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: lev@FreeBSD.org, Kurt Jaeger References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> Cc: freebsd-stable@freebsd.org, Garrett Wollman From: Eugene Grosbein Message-ID: <3d7f4ac4-67e4-b010-64c6-0d84a4451246@grosbein.net> Date: Wed, 13 Feb 2019 02:18:09 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: DB02A7637B X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.93)[-0.928,0]; IP_SCORE(-1.37)[ip: (-2.23), ipnet: 2a01:4f8::/29(-2.39), asn: 24940(-2.23), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:18:32 -0000 13.02.2019 1:50, Lev Serebryakov wrote: > I'm have same problem. > > According to top(1) I have 29G Wired, but only 17G Total ARC (12G > difference! System has 32G of RAM), and this statistic shows: > > 5487.5 zio_data_buf_524288 > 920.125 zio_data_buf_131072 > 626 zio_buf_131072 > 468 zio_data_buf_1048576 > 398.391 zio_buf_16384 > 305.464 dnode_t > 227.989 zio_buf_512 > 171.5 zio_data_buf_458752 > 141.75 zio_data_buf_393216 > 116.456 dmu_buf_impl_t > > So, more than 6G (!) is not used in ARC, but hold by ZFS anyway. dnode_t and dmu_buf_impl_t are parts of ZFS too, so these numbers represent about 9G, not 6G. Do you have/had some memory pressure here? Growth of swap usage? From owner-freebsd-stable@freebsd.org Tue Feb 12 19:18:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08B9614BD0AF for ; Tue, 12 Feb 2019 19:18:53 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 438D576387; Tue, 12 Feb 2019 19:18:42 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.15.2/8.15.2) with ESMTP id x1CJIcm4026512; Tue, 12 Feb 2019 14:18:39 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Eugene Grosbein , Mark Johnston Cc: kib@freebsd.org, FreeBSD stable References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: <672edbe6-ac42-f21e-ad8a-7a5f5d4c4e3c@sentex.net> Date: Tue, 12 Feb 2019 14:18:37 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.78 X-Rspamd-Queue-Id: 438D576387 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-0.80 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.03)[-0.032,0]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sentex.net]; NEURAL_SPAM_SHORT(0.56)[0.565,0]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sentex.ca]; NEURAL_HAM_LONG(-0.92)[-0.921,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.0.0.0.0.0.e.3.f.7.0.6.2.list.dnswl.org : 127.0.5.1] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:18:53 -0000 On 2/12/2019 1:03 PM, Eugene Grosbein wrote: > It seems page daemon is broken somehow as it did not reclaim several > gigs of wired memory > despite of long period of vm thrashing: > > $ sed 's/:/,/' vmstat-z.txt | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > 1892 abd_chunk > 454.629 dnode_t > 351.35 zio_buf_512 > 228.391 zio_buf_16384 > 173.968 dmu_buf_impl_t > 130.25 zio_data_buf_131072 > 93.6887 VNODE > 81.6978 arc_buf_hdr_t_full > 74.9368 256 > 57.4102 4096 On an nfs server, serving a few large files, my 32G box is showing vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head    11014.3 abd_chunk     2090.5 zio_data_buf_131072    1142.67 mbuf_jumbo_page    1134.25 zio_buf_131072     355.28 mbuf_jumbo_9k     233.42 zio_cache    163.099 arc_buf_hdr_t_full    130.738 128    97.2812 zio_buf_16384    96.5099 UMA Slabs CPU:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle Mem: 1348K Active, 98M Inact, 3316K Laundry, 30G Wired, 1022M Free ARC: 11G Total, 7025M MFU, 3580M MRU, 11M Anon, 78M Header, 681M Other      9328M Compressed, 28G Uncompressed, 3.05:1 Ratio Swap: 64G Total, 13M Used, 64G Free Right now its OK, but prior to limiting ARC, I had an issue with memory and the disk thrashing due to swapping pid 643 (devd), uid 0, was killed: out of swap space     ---Mike > - > ------------------- > Mike Tancsa, tel +1 519 651 3400 x203 > Sentex Communications, mike@sentex.net > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Tue Feb 12 19:19:07 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE99F14BD101 for ; Tue, 12 Feb 2019 19:19:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 40F6D763FE for ; Tue, 12 Feb 2019 19:19:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id F2BBC14BD0FC; Tue, 12 Feb 2019 19:19:06 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E079D14BD0FA for ; Tue, 12 Feb 2019 19:19:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A939763F6 for ; Tue, 12 Feb 2019 19:19:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 7EE5F76E8 for ; Tue, 12 Feb 2019 19:19:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CJJ59F067469 for ; Tue, 12 Feb 2019 19:19:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CJJ5Yh067468 for stable@FreeBSD.org; Tue, 12 Feb 2019 19:19:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 19:19:03 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:19:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #7 from Sergey Anokhin --- btw, perhaps it can be helpful: if port security/ipsec-tools was built with default options (make rmconfig), so the bug doesn't reproduced --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 19:31:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8461714BDE17 for ; Tue, 12 Feb 2019 19:31:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1B32776FE6 for ; Tue, 12 Feb 2019 19:31:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id D020814BDE11; Tue, 12 Feb 2019 19:31:07 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD5B214BDE0E for ; Tue, 12 Feb 2019 19:31:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5731C76FDF for ; Tue, 12 Feb 2019 19:31:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 827AC7978 for ; Tue, 12 Feb 2019 19:31:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CJV6RL093112 for ; Tue, 12 Feb 2019 19:31:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CJV6mg093111 for stable@FreeBSD.org; Tue, 12 Feb 2019 19:31:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 19:31:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:31:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #8 from Sergey Anokhin --- (In reply to Jan Bramkamp from comment #6) Did you mean try to set kern.maxssiz into /boot/loader.conf? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 19:39:12 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 587DA14CE347 for ; Tue, 12 Feb 2019 19:39:12 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E8D4774CD for ; Tue, 12 Feb 2019 19:39:11 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1CJd0Mq073353 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 12 Feb 2019 20:39:05 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: mike@sentex.net Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1CJd0ou053404 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 02:39:00 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mike Tancsa References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <672edbe6-ac42-f21e-ad8a-7a5f5d4c4e3c@sentex.net> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: Date: Wed, 13 Feb 2019 02:38:53 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <672edbe6-ac42-f21e-ad8a-7a5f5d4c4e3c@sentex.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 8E8D4774CD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; IP_SCORE(-1.37)[ip: (-2.20), ipnet: 2a01:4f8::/29(-2.39), asn: 24940(-2.23), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:39:12 -0000 13.02.2019 2:18, Mike Tancsa wrote: > On an nfs server, serving a few large files, my 32G box is showing > > vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", > $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > 11014.3 abd_chunk > 2090.5 zio_data_buf_131072 > 1142.67 mbuf_jumbo_page > 1134.25 zio_buf_131072 > 355.28 mbuf_jumbo_9k > 233.42 zio_cache > 163.099 arc_buf_hdr_t_full > 130.738 128 > 97.2812 zio_buf_16384 > 96.5099 UMA Slabs > > > CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > Mem: 1348K Active, 98M Inact, 3316K Laundry, 30G Wired, 1022M Free > ARC: 11G Total, 7025M MFU, 3580M MRU, 11M Anon, 78M Header, 681M Other > 9328M Compressed, 28G Uncompressed, 3.05:1 Ratio > Swap: 64G Total, 13M Used, 64G Free > > > Right now its OK, but prior to limiting ARC, I had an issue with memory > and the disk thrashing due to swapping > > pid 643 (devd), uid 0, was killed: out of swap space mbuf* numbers represent memory being wasted IMHO. In contrast to ZFS memory that can contain useful cached data, contents of freed mbufs cannot be re-used, right? Some amount of "free" mbufs in the zone's just fine to serve bursts of traffic without need to grow the zone but your numbers are way too big IMHO. Do you have memory pressure here? Growth of swap usage? From owner-freebsd-stable@freebsd.org Tue Feb 12 19:41:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A54E14CE5B9 for ; Tue, 12 Feb 2019 19:41:16 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B59B77609 for ; Tue, 12 Feb 2019 19:41:15 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.15.2/8.15.2) with ESMTP id x1CJfEaU031531; Tue, 12 Feb 2019 14:41:14 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Eugene Grosbein Cc: FreeBSD stable References: <20190212163446.GA29847@raichu> <4ab1331f-80e3-b856-b402-9985e73618bc@grosbein.net> <672edbe6-ac42-f21e-ad8a-7a5f5d4c4e3c@sentex.net> From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: Date: Tue, 12 Feb 2019 14:41:13 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.78 X-Rspamd-Queue-Id: 3B59B77609 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-5.12 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sentex.net]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: smtp.sentex.ca]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; IP_SCORE(-1.73)[ipnet: 2607:f3e0::/32(-4.95), asn: 11647(-3.61), country: CA(-0.09)]; RCVD_IN_DNSWL_LOW(-0.10)[2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.0.0.0.0.0.e.3.f.7.0.6.2.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:41:16 -0000 On 2/12/2019 2:38 PM, Eugene Grosbein wrote: > > mbuf* numbers represent memory being wasted IMHO. > > In contrast to ZFS memory that can contain useful cached data, > contents of freed mbufs cannot be re-used, right? > > Some amount of "free" mbufs in the zone's just fine to serve bursts of traffic > without need to grow the zone but your numbers are way too big IMHO. > > Do you have memory pressure here? Growth of swap usage? > I had to radically cut ARC down, otherwise the box would start swapping all sorts of processes out % sysctl -a vfs.zfs.arc_max vfs.zfs.arc_max: 12307744768     ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Tue Feb 12 19:54:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C067C14CEAC6; Tue, 12 Feb 2019 19:54:43 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE4F177D5E; Tue, 12 Feb 2019 19:54:40 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from disco.vangyzen.net (unknown [70.97.188.230]) by smtp.vangyzen.net (Postfix) with ESMTPSA id AE2EA56468; Tue, 12 Feb 2019 13:54:25 -0600 (CST) Subject: Re: Networking panic on 12 - found the cause To: Pete French , "Andrey V. Elsukov" , freebsd-stable@freebsd.org, freebsd-current References: <4a350f47-eaca-5aca-9268-bf7a6618e71c@yandex.ru> From: Eric van Gyzen Message-ID: <0a0f2668-19f7-ad62-dff4-e2997e57ea5c@vangyzen.net> Date: Tue, 12 Feb 2019 13:54:21 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CE4F177D5E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of eric@vangyzen.net designates 2607:fc50:1000:7400:216:3eff:fe72:314f as permitted sender) smtp.mailfrom=eric@vangyzen.net X-Spamd-Result: default: False [-6.55 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[vangyzen.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[hotblack.vangyzen.net]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; IP_SCORE(-3.28)[ip: (-8.18), ipnet: 2607:fc50:1000::/36(-4.23), asn: 36236(-3.90), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:36236, ipnet:2607:fc50:1000::/36, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 19:54:44 -0000 On 2/12/19 8:53 AM, Pete French wrote: > I found my panic. If I take everything out of rc.conf and loader.conf > and sysctl.conf and boot the system it works fine when I add an IP > address. If I add this one line to sysctl.conf > >     net.link.ether.inet.garp_rexmit_count=2 > > Then I get a panic when I configure the interface: > > root@serpentine-passive:~ #  ifconfig igb0 inet 10.32.10.4/16 up > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address   = 0x28 > fault code              = supervisor read data, page not present > instruction pointer     = 0x20:0xffffffff80c987f1 > stack pointer           = 0x28:0xfffffe00004d5730 > frame pointer           = 0x28:0xfffffe00004d5750 > code segment            = base 0x0, limit 0xfffff, type 0x1b >                         = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags        = interrupt enabled, resume, IOPL = 0 > current process         = 12 (swi4: clock (0)) > trap number             = 12 > panic: page fault > cpuid = 0 > time = 1549981620 > KDB: stack backtrace: > #0 0xffffffff80bdfdc7 at kdb_backtrace+0x67 > #1 0xffffffff80b93fa3 at vpanic+0x1a3 > #2 0xffffffff80b93df3 at panic+0x43 > #3 0xffffffff8106a7bf at trap_fatal+0x35f > #4 0xffffffff8106a819 at trap_pfault+0x49 > #5 0xffffffff81069e3e at trap+0x29e > #6 0xffffffff810450c5 at calltrap+0x8 > #7 0xffffffff80c986f6 at ether_output+0x6b6 > #8 0xffffffff80d03354 at arprequest+0x4c4 > #9 0xffffffff80d0515c at garp_rexmit+0xbc > #10 0xffffffff80bade19 at softclock_call_cc+0x129 > #11 0xffffffff80bae2f9 at softclock+0x79 > #12 0xffffffff80b57c57 at ithread_loop+0x1a7 > #13 0xffffffff80b54da2 at fork_exit+0x82 > #14 0xffffffff810460be at fork_trampoline+0xe I see the same behavior on head (and stable/12). (kgdb) f #16 0xffffffff80ce5331 in ether_output_frame (ifp=0xfffff80003672800, m=0xfffff8000c88b100) at /usr/src/sys/net/if_ethersubr.c:468 468 switch (pfil_run_hooks(V_link_pfil_head, &m, ifp, PFIL_OUT, 0xffffffff80ce5321 <+81>: mov %gs:0x0,%rax 0xffffffff80ce532a <+90>: mov 0x500(%rax),%rax => 0xffffffff80ce5331 <+97>: mov 0x28(%rax),%rax I think this is part of the V_link_pfil_head. I'm not very familiar with vnet. Does this need a CURVNET_SET(), maybe in garp_rexmit()? Eric From owner-freebsd-stable@freebsd.org Tue Feb 12 20:13:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9968514CF34A; Tue, 12 Feb 2019 20:13:40 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 493AC807AB; Tue, 12 Feb 2019 20:13:38 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id C02AC10AB7; Tue, 12 Feb 2019 21:13:30 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id BDA152C2CA; Tue, 12 Feb 2019 21:13:30 +0100 (CET) Date: Tue, 12 Feb 2019 21:13:30 +0100 From: Kristof Provost To: Eric van Gyzen Cc: Pete French , "Andrey V. Elsukov" , freebsd-stable@freebsd.org, freebsd-current Subject: Re: Networking panic on 12 - found the cause Message-ID: <20190212201330.GK8450@vega.codepro.be> References: <4a350f47-eaca-5aca-9268-bf7a6618e71c@yandex.ru> <0a0f2668-19f7-ad62-dff4-e2997e57ea5c@vangyzen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <0a0f2668-19f7-ad62-dff4-e2997e57ea5c@vangyzen.net> X-Checked-By-NSA: Probably User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 493AC807AB X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of srs0=cpp7=qt=vega.codepro.be=kp@codepro.be designates 5.9.86.228 as permitted sender) smtp.mailfrom=srs0=cpp7=qt=vega.codepro.be=kp@codepro.be X-Spamd-Result: default: False [-5.62 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.86.228]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[228.86.9.5.list.dnswl.org : 127.0.9.2]; MX_GOOD(-0.01)[mx2.codepro.be,mx1.codepro.be]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; IP_SCORE(-2.43)[ip: (-8.04), ipnet: 5.9.0.0/16(-1.87), asn: 24940(-2.24), country: DE(-0.01)]; FORGED_SENDER(0.30)[kp@freebsd.org,srs0=cpp7=qt=vega.codepro.be=kp@codepro.be]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; FROM_NEQ_ENVFROM(0.00)[kp@freebsd.org, srs0=cpp7=qt=vega.codepro.be=kp@codepro.be] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 20:13:40 -0000 On 2019-02-12 13:54:21 (-0600), Eric van Gyzen wrote: > I see the same behavior on head (and stable/12). > > (kgdb) f > #16 0xffffffff80ce5331 in ether_output_frame (ifp=0xfffff80003672800, > m=0xfffff8000c88b100) at /usr/src/sys/net/if_ethersubr.c:468 > 468 switch (pfil_run_hooks(V_link_pfil_head, &m, ifp, PFIL_OUT, > > 0xffffffff80ce5321 <+81>: mov %gs:0x0,%rax > 0xffffffff80ce532a <+90>: mov 0x500(%rax),%rax > => 0xffffffff80ce5331 <+97>: mov 0x28(%rax),%rax > > I think this is part of the V_link_pfil_head. I'm not very familiar > with vnet. Does this need a CURVNET_SET(), maybe in garp_rexmit()? > Yes. I posted a proposed patch in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235699 Basically we get called through a timer, so there's no vnet context. It needs to be set, and then we can safely use any V_ variables. Regards, Kristof From owner-freebsd-stable@freebsd.org Tue Feb 12 20:25:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B81CC14CF99A for ; Tue, 12 Feb 2019 20:25:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 47AE780E67 for ; Tue, 12 Feb 2019 20:25:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0813214CF997; Tue, 12 Feb 2019 20:25:34 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9FE514CF994 for ; Tue, 12 Feb 2019 20:25:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8613280E63 for ; Tue, 12 Feb 2019 20:25:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C48D28107 for ; Tue, 12 Feb 2019 20:25:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CKPWub032918 for ; Tue, 12 Feb 2019 20:25:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CKPWbY032917 for stable@FreeBSD.org; Tue, 12 Feb 2019 20:25:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 20:25:31 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 20:25:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #9 from Andrey V. Elsukov --- Can you try to remove `option VIMAGE` from your kernel config? It looks like the problem is similar to the one described in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235699 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 21:02:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFBB514D0E63 for ; Tue, 12 Feb 2019 21:02:16 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B36A382444; Tue, 12 Feb 2019 21:02:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 3685714B57; Tue, 12 Feb 2019 21:02:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:71eb:28be:92e:f722]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 25213BB71; Wed, 13 Feb 2019 00:02:08 +0300 (MSK) Date: Wed, 13 Feb 2019 00:02:07 +0300 From: Lev Serebryakov Reply-To: Lev Serebryakov Organization: FreeBSD Message-ID: <1294306375.20190213000207@serebryakov.spb.ru> To: Eugene Grosbein , Kurt Jaeger CC: freebsd-stable@freebsd.org, Garrett Wollman Subject: Re: 11.2-STABLE kernel wired memory leak In-Reply-To: <3d7f4ac4-67e4-b010-64c6-0d84a4451246@grosbein.net> References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> <3d7f4ac4-67e4-b010-64c6-0d84a4451246@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: B36A382444 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.93 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_SHORT(-0.94)[-0.938,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 21:02:17 -0000 Hello Eugene, Tuesday, February 12, 2019, 10:18:09 PM, you wrote: >> I'm have same problem. >> >> According to top(1) I have 29G Wired, but only 17G Total ARC (12G >> difference! System has 32G of RAM), and this statistic shows: >> >> 5487.5 zio_data_buf_524288 >> 920.125 zio_data_buf_131072 >> 626 zio_buf_131072 >> 468 zio_data_buf_1048576 >> 398.391 zio_buf_16384 >> 305.464 dnode_t >> 227.989 zio_buf_512 >> 171.5 zio_data_buf_458752 >> 141.75 zio_data_buf_393216 >> 116.456 dmu_buf_impl_t >> >> So, more than 6G (!) is not used in ARC, but hold by ZFS anyway. > dnode_t and dmu_buf_impl_t are parts of ZFS too, > so these numbers represent about 9G, not 6G. > Do you have/had some memory pressure here? Growth of swap usage? I don't have memory pressure right now, but according to my previous experience, ARC will not grow anymore even under heavy disk load (I don't have vfs.zfs.arc_max set). Before new ARC (vfs.zfs.abd_scatter_enabled) I had typically ALL memory occuped by ARC, Wired memory was almost exactly equal to ARC, and ARC hitrate was higher (but I have not exact numbers, unfortunately). Now I have "vfs.zfs.abd_scatter_enabled=0", but still Wired is much larger that ARC under any disk load (it is mostly torrent box). -- Best regards, Lev mailto:lev@FreeBSD.org From owner-freebsd-stable@freebsd.org Tue Feb 12 22:18:11 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 616D414D4574 for ; Tue, 12 Feb 2019 22:18:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E206F85A62 for ; Tue, 12 Feb 2019 22:18:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A10A514D4570; Tue, 12 Feb 2019 22:18:10 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79C8D14D456E for ; Tue, 12 Feb 2019 22:18:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EB1485A5E for ; Tue, 12 Feb 2019 22:18:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 02C41912B for ; Tue, 12 Feb 2019 22:18:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMI8MY028495 for ; Tue, 12 Feb 2019 22:18:08 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMI8wJ028494 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:18:08 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 22:18:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:18:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #10 from Sergey Anokhin --- (In reply to Jan Bramkamp from comment #6) Will it ok? (pts/1)[root@server:~]# sysctl kern.maxssiz=3D1073741824 kern.maxssiz: 536870912 -> 1073741824 (pts/1)[root@server:~]# /usr/local/etc/rc.d/racoon onestart Starting racoon. (pts/1)[root@server:~]# /usr/local/etc/rc.d/racoon onestop Stopping racoon. Waiting for PIDS: 5662 kernel panic btw, I've noticed that kernel panic during stopping racoon. # kgdb kernel /var/crash/vmcore.last GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD] Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd12.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from kernel...Reading symbols from /usr/obj/usr/src/amd64.amd64/sys/SERVER/kernel.debug...done. done. Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 2; apic id =3D 02 fault virtual address =3D 0x28 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80ecd31d stack pointer =3D 0x28:0xfffffe003fca7a40 frame pointer =3D 0x28:0xfffffe003fca7a60 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 0 (softirq_2) trap number =3D 12 panic: page fault cpuid =3D 2 time =3D 1550009599 KDB: stack backtrace: #0 0xffffffff80c531c7 at kdb_backtrace+0x67 #1 0xffffffff80c07143 at vpanic+0x1a3 #2 0xffffffff80c06f93 at panic+0x43 #3 0xffffffff8118d9ff at trap_fatal+0x35f #4 0xffffffff8118da59 at trap_pfault+0x49 #5 0xffffffff8118d07e at trap+0x29e #6 0xffffffff81168ac5 at calltrap+0x8 #7 0xffffffff80eca240 at ipsec_delete_pcbpolicy+0x20 #8 0xffffffff80dbaeec at in_pcbfree_deferred+0x6c #9 0xffffffff80c4db1a at epoch_call_task+0x1ca #10 0xffffffff80c51a54 at gtaskqueue_run_locked+0x144 #11 0xffffffff80c516b8 at gtaskqueue_thread_loop+0x98 #12 0xffffffff80bc6f23 at fork_exit+0x83 #13 0xffffffff81169abe at fork_trampoline+0xe Uptime: 8m33s Dumping 950 out of 8077 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (OFFSETOF_CURTHREAD)); (kgdb) bt #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=3D) at /usr/src/sys/kern/kern_shutdown= .c:366 #2 0xffffffff80c06d2b in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:446 #3 0xffffffff80c071a3 in vpanic (fmt=3D, ap=3D0xfffffe003fc= a7790) at /usr/src/sys/kern/kern_shutdown.c:872 #4 0xffffffff80c06f93 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:799 #5 0xffffffff8118d9ff in trap_fatal (frame=3D0xfffffe003fca7980, eva=3D40)= at /usr/src/sys/amd64/amd64/trap.c:929 #6 0xffffffff8118da59 in trap_pfault (frame=3D0xfffffe003fca7980, usermode= =3D0) at /usr/src/sys/amd64/amd64/trap.c:765 #7 0xffffffff8118d07e in trap (frame=3D0xfffffe003fca7980) at /usr/src/sys/amd64/amd64/trap.c:441 #8 #9 0xffffffff80ecd31d in key_freesp (spp=3D0xfffff80211241880) at /usr/src/sys/netipsec/key.c:1199 #10 0xffffffff80eca240 in ipsec_delete_pcbpolicy (inp=3D0xfffff800151aa1e8)= at /usr/src/sys/netipsec/ipsec_pcb.c:176 #11 0xffffffff80dbaeec in in_pcbfree_deferred (ctx=3D0xfffff800151aa3c0) at /usr/src/sys/netinet/in_pcb.c:1576 #12 0xffffffff80c4db1a in epoch_call_task (arg=3D) at /usr/src/sys/kern/subr_epoch.c:507 #13 0xffffffff80c51a54 in gtaskqueue_run_locked (queue=3D0xfffff80003363c00= ) at /usr/src/sys/kern/subr_gtaskqueue.c:376 #14 0xffffffff80c516b8 in gtaskqueue_thread_loop (arg=3D) at /usr/src/sys/kern/subr_gtaskqueue.c:557 #15 0xffffffff80bc6f23 in fork_exit (callout=3D0xffffffff80c51620 , arg=3D0xfffffe00025f5038, frame=3D0xfffffe003fca7= c00) at /usr/src/sys/kern/kern_fork.c:1059 #16 (kgdb) frame 9 #9 0xffffffff80ecd31d in key_freesp (spp=3D0xfffff80211241880) at /usr/src/sys/netipsec/key.c:1199 1199 KEYDBG(IPSEC_STAMP, (kgdb) --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:21:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA24C14D48D7 for ; Tue, 12 Feb 2019 22:21:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6A13D85DD0 for ; Tue, 12 Feb 2019 22:21:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2910B14D48D3; Tue, 12 Feb 2019 22:21:25 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1669414D48D0 for ; Tue, 12 Feb 2019 22:21:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0F3985DC9 for ; Tue, 12 Feb 2019 22:21:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id DBD469269 for ; Tue, 12 Feb 2019 22:21:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMLNFm034259 for ; Tue, 12 Feb 2019 22:21:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMLNvK034256 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:21:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 22:21:22 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:21:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #11 from Andrey V. Elsukov --- Created attachment 201968 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D201968&action= =3Dedit Proposed patch Also, you can test this patch instead, it should fix panic with VIMAGE opti= on. The problem is due to introduced deferred PCB destroying via epoch_call(). Since this code is executed from gtaskqueue, it has no VNET context. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:30:41 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 254D214D4D3E for ; Tue, 12 Feb 2019 22:30:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AF1AB86459 for ; Tue, 12 Feb 2019 22:30:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6618314D4D3D; Tue, 12 Feb 2019 22:30:40 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43AF314D4D3B for ; Tue, 12 Feb 2019 22:30:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C095186450 for ; Tue, 12 Feb 2019 22:30:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 037E292B3 for ; Tue, 12 Feb 2019 22:30:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMUcx0050897 for ; Tue, 12 Feb 2019 22:30:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMUcQ1050896 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:30:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 233006] [regression] [patch] bsdtar aborts creation of archive on ENOENT Date: Tue, 12 Feb 2019 22:30:39 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: patch, regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: mm@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? mfc-stable12? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:30:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233006 --- Comment #14 from commit-hook@freebsd.org --- A commit references this bug: Author: mm Date: Tue Feb 12 22:29:45 UTC 2019 New revision: 344063 URL: https://svnweb.freebsd.org/changeset/base/344063 Log: Update vendor/libarchive/dist to git 31c0a517c91f44eeee717a04db8b075cadda= 83d8 Relevant vendor changes: PR #1085: Fix a null pointer dereference bug in zip writer PR #1110: ZIP reader added support for XZ, LZMA, PPMD8 and BZIP2 decopmpression PR #1116: Add support for 64-bit ar format PR #1120: Fix a 7zip crash [1] and a ISO9660 infinite loop [2] PR #1125: RAR5 reader - fix an invalid read and a memory leak PR #1131: POSIX reader - do not fail when tree_current_lstat() fails due to ENOENT [3] PR #1134: Delete unnecessary null pointer checks before calls of free() OSS-Fuzz 10843: Force intermediate to uint64_t to make UBSAN happy. OSS-Fuzz 11011: Avoid buffer overflow in rar5 reader PR: 233006 [3] Security: CVE-2019-1000019 [1], CVE-2019-1000020 [2] Changes: vendor/libarchive/dist/.cirrus.yml vendor/libarchive/dist/.travis.yml vendor/libarchive/dist/CMakeLists.txt vendor/libarchive/dist/Makefile.am vendor/libarchive/dist/build/ci/ vendor/libarchive/dist/build/ci/build.sh vendor/libarchive/dist/build/ci/cirrus_ci.sh vendor/libarchive/dist/build/ci/test_driver vendor/libarchive/dist/build/ci_build.sh vendor/libarchive/dist/build/ci_test_driver vendor/libarchive/dist/contrib/shar/tree.c vendor/libarchive/dist/cpio/test/test_option_t.c vendor/libarchive/dist/libarchive/CMakeLists.txt vendor/libarchive/dist/libarchive/archive_acl.c vendor/libarchive/dist/libarchive/archive_disk_acl_sunos.c vendor/libarchive/dist/libarchive/archive_entry.c vendor/libarchive/dist/libarchive/archive_pack_dev.c vendor/libarchive/dist/libarchive/archive_ppmd8.c vendor/libarchive/dist/libarchive/archive_ppmd8_private.h vendor/libarchive/dist/libarchive/archive_read_disk_posix.c vendor/libarchive/dist/libarchive/archive_read_open_file.c vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c vendor/libarchive/dist/libarchive/archive_read_support_format_ar.c vendor/libarchive/dist/libarchive/archive_read_support_format_cpio.c vendor/libarchive/dist/libarchive/archive_read_support_format_iso9660.c vendor/libarchive/dist/libarchive/archive_read_support_format_rar5.c vendor/libarchive/dist/libarchive/archive_read_support_format_xar.c vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c vendor/libarchive/dist/libarchive/archive_write_disk_posix.c vendor/libarchive/dist/libarchive/archive_write_disk_set_standard_lookup.c vendor/libarchive/dist/libarchive/archive_write_disk_windows.c vendor/libarchive/dist/libarchive/archive_write_set_format_ar.c vendor/libarchive/dist/libarchive/archive_write_set_format_cpio.c vendor/libarchive/dist/libarchive/archive_write_set_format_cpio_newc.c vendor/libarchive/dist/libarchive/archive_write_set_format_gnutar.c vendor/libarchive/dist/libarchive/archive_write_set_format_shar.c vendor/libarchive/dist/libarchive/archive_write_set_format_ustar.c vendor/libarchive/dist/libarchive/archive_write_set_format_v7tar.c vendor/libarchive/dist/libarchive/archive_write_set_format_zip.c vendor/libarchive/dist/libarchive/test/CMakeLists.txt vendor/libarchive/dist/libarchive/test/test_compat_pax_libarchive_2x.c vendor/libarchive/dist/libarchive/test/test_compat_pax_libarchive_2x.tar.= Z.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip.c vendor/libarchive/dist/libarchive/test/test_read_format_zip_bzip2.zipx.uu =20 vendor/libarchive/dist/libarchive/test/test_read_format_zip_bzip2_multi.zip= x.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_lzma.zipx.uu =20 vendor/libarchive/dist/libarchive/test/test_read_format_zip_lzma_multi.zipx= .uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_ppmd8.zipx.uu =20 vendor/libarchive/dist/libarchive/test/test_read_format_zip_ppmd8_multi.zip= x.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_xz_multi.zipx= .uu vendor/libarchive/dist/test_utils/test_main.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:32:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF50714D4F99 for ; Tue, 12 Feb 2019 22:32:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4A6A58679B for ; Tue, 12 Feb 2019 22:32:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 096C714D4F93; Tue, 12 Feb 2019 22:32:47 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB1A114D4F90 for ; Tue, 12 Feb 2019 22:32:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81BA186795 for ; Tue, 12 Feb 2019 22:32:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 943499414 for ; Tue, 12 Feb 2019 22:32:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMWjDC059575 for ; Tue, 12 Feb 2019 22:32:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMWj57059574 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:32:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 22:32:45 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:32:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #12 from Sergey Anokhin --- (In reply to Andrey V. Elsukov from comment #9) Sure, now I'm building kernel without VIMAGE. I'll let you know about testi= ng result --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:39:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2FFF14D53BD for ; Tue, 12 Feb 2019 22:39:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3960886C5B for ; Tue, 12 Feb 2019 22:39:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E451A14D53B1; Tue, 12 Feb 2019 22:39:12 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D213314D53B0 for ; Tue, 12 Feb 2019 22:39:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AF2B86C54 for ; Tue, 12 Feb 2019 22:39:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8999B9423 for ; Tue, 12 Feb 2019 22:39:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMdBf5065856 for ; Tue, 12 Feb 2019 22:39:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMdBur065855 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:39:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 22:39:11 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@5034.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:39:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #13 from Sergey Anokhin --- (In reply to Andrey V. Elsukov from comment #11) I'd preferred to try to rebuild kernel if it's no difference between turning off VIMAGE from kernel config and applying patch because kernel building mo= re faster then "world" building. As far as I understand, you are propose patch= for "world" component, right? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:45:11 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AC3514D588E for ; Tue, 12 Feb 2019 22:45:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F13638725A for ; Tue, 12 Feb 2019 22:45:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id AD2A914D5885; Tue, 12 Feb 2019 22:45:10 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9936D14D5884 for ; Tue, 12 Feb 2019 22:45:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F54387251 for ; Tue, 12 Feb 2019 22:45:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 44C49958B for ; Tue, 12 Feb 2019 22:45:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CMj9eu082755 for ; Tue, 12 Feb 2019 22:45:09 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CMj9or082754 for stable@FreeBSD.org; Tue, 12 Feb 2019 22:45:09 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Tue, 12 Feb 2019 22:45:09 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:45:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 --- Comment #14 from Andrey V. Elsukov --- (In reply to Sergey Anokhin from comment #13) > (In reply to Andrey V. Elsukov from comment #11) >=20 > I'd preferred to try to rebuild kernel if it's no difference between turn= ing > off VIMAGE from kernel config and applying patch because kernel building > more faster then "world" building. As far as I understand, you are propose > patch for "world" component, right? No, the patch is for kernel. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Tue Feb 12 22:51:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D33014D60EA; Tue, 12 Feb 2019 22:51:58 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 429EF87B97; Tue, 12 Feb 2019 22:51:56 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.drayhouse.twisted.org.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1gtgu9-0008dR-N2; Tue, 12 Feb 2019 22:51:53 +0000 Subject: Re: Networking panic on 12 - found the cause To: Kristof Provost , Eric van Gyzen Cc: "Andrey V. Elsukov" , freebsd-stable@freebsd.org, freebsd-current References: <4a350f47-eaca-5aca-9268-bf7a6618e71c@yandex.ru> <0a0f2668-19f7-ad62-dff4-e2997e57ea5c@vangyzen.net> <20190212201330.GK8450@vega.codepro.be> From: Pete French Message-ID: <50301a70-9c2e-b340-100c-cd1e6429c134@ingresso.co.uk> Date: Tue, 12 Feb 2019 22:52:07 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:65.0) Gecko/20100101 Thunderbird/65.0 MIME-Version: 1.0 In-Reply-To: <20190212201330.GK8450@vega.codepro.be> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 429EF87B97 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 2a02:b90:3002:411::3 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [4.27 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:b90:3002:411::3]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.02)[country: GB(-0.09)]; RDNS_NONE(1.00)[]; NEURAL_SPAM_MEDIUM(0.35)[0.348,0]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.45)[0.453,0]; MX_GOOD(-0.01)[cached: us-smtp-inbound-1.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; NEURAL_SPAM_LONG(0.80)[0.801,0]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_CC(0.00)[yandex.ru]; ASN(0.00)[asn:16082, ipnet:2a02:b90::/32, country:GB]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 22:51:58 -0000 Thanks guys! That was fast.... On 12/02/2019 20:13, Kristof Provost wrote: > On 2019-02-12 13:54:21 (-0600), Eric van Gyzen wrote: > > I see the same behavior on head (and stable/12). > > > > (kgdb) f > > #16 0xffffffff80ce5331 in ether_output_frame (ifp=0xfffff80003672800, > > m=0xfffff8000c88b100) at /usr/src/sys/net/if_ethersubr.c:468 > > 468 switch (pfil_run_hooks(V_link_pfil_head, &m, ifp, PFIL_OUT, > > > > 0xffffffff80ce5321 <+81>: mov %gs:0x0,%rax > > 0xffffffff80ce532a <+90>: mov 0x500(%rax),%rax > > => 0xffffffff80ce5331 <+97>: mov 0x28(%rax),%rax > > > > I think this is part of the V_link_pfil_head. I'm not very familiar > > with vnet. Does this need a CURVNET_SET(), maybe in garp_rexmit()? > > > Yes. I posted a proposed patch in > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235699 > > Basically we get called through a timer, so there's no vnet context. It > needs to be set, and then we can safely use any V_ variables. > > Regards, > Kristof From owner-freebsd-stable@freebsd.org Tue Feb 12 23:25:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4F7414D6EB1 for ; Tue, 12 Feb 2019 23:25:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 775B3890E3 for ; Tue, 12 Feb 2019 23:25:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 34A2914D6EB0; Tue, 12 Feb 2019 23:25:27 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FBD514D6EAF for ; Tue, 12 Feb 2019 23:25:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C652890DC for ; Tue, 12 Feb 2019 23:25:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C966F9B57 for ; Tue, 12 Feb 2019 23:25:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1CNPPGA095244 for ; Tue, 12 Feb 2019 23:25:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1CNPPoc095242 for stable@FreeBSD.org; Tue, 12 Feb 2019 23:25:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 233006] [regression] [patch] bsdtar aborts creation of archive on ENOENT Date: Tue, 12 Feb 2019 23:25:24 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: patch, regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: mm@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? mfc-stable12? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 23:25:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233006 --- Comment #15 from commit-hook@freebsd.org --- A commit references this bug: Author: mm Date: Tue Feb 12 23:24:47 UTC 2019 New revision: 344065 URL: https://svnweb.freebsd.org/changeset/base/344065 Log: MFV r344063: Sync libarchive with vendor. Relevant vendor changes: PR #1085: Fix a null pointer dereference bug in zip writer PR #1110: ZIP reader added support for XZ, LZMA, PPMD8 and BZIP2 decopmpression PR #1116: Add support for 64-bit ar format PR #1120: Fix a 7zip crash [1] and a ISO9660 infinite loop [2] PR #1125: RAR5 reader - fix an invalid read and a memory leak PR #1131: POSIX reader - do not fail when tree_current_lstat() fails due to ENOENT [3] PR #1134: Delete unnecessary null pointer checks before calls of free() OSS-Fuzz 10843: Force intermediate to uint64_t to make UBSAN happy. OSS-Fuzz 11011: Avoid buffer overflow in rar5 reader PR: 233006 [3] Security: CVE-2019-1000019 [1], CVE-2019-1000020 [2] MFC after: 2 weeks Changes: _U head/contrib/libarchive/ head/contrib/libarchive/cpio/test/test_option_t.c head/contrib/libarchive/libarchive/archive_acl.c head/contrib/libarchive/libarchive/archive_entry.c head/contrib/libarchive/libarchive/archive_pack_dev.c head/contrib/libarchive/libarchive/archive_ppmd8.c head/contrib/libarchive/libarchive/archive_ppmd8_private.h head/contrib/libarchive/libarchive/archive_read_disk_posix.c head/contrib/libarchive/libarchive/archive_read_open_file.c head/contrib/libarchive/libarchive/archive_read_support_format_7zip.c head/contrib/libarchive/libarchive/archive_read_support_format_ar.c head/contrib/libarchive/libarchive/archive_read_support_format_cpio.c head/contrib/libarchive/libarchive/archive_read_support_format_iso9660.c head/contrib/libarchive/libarchive/archive_read_support_format_rar5.c head/contrib/libarchive/libarchive/archive_read_support_format_xar.c head/contrib/libarchive/libarchive/archive_read_support_format_zip.c head/contrib/libarchive/libarchive/archive_write_disk_posix.c head/contrib/libarchive/libarchive/archive_write_disk_set_standard_lookup= .c head/contrib/libarchive/libarchive/archive_write_set_format_ar.c head/contrib/libarchive/libarchive/archive_write_set_format_cpio.c head/contrib/libarchive/libarchive/archive_write_set_format_cpio_newc.c head/contrib/libarchive/libarchive/archive_write_set_format_gnutar.c head/contrib/libarchive/libarchive/archive_write_set_format_shar.c head/contrib/libarchive/libarchive/archive_write_set_format_ustar.c head/contrib/libarchive/libarchive/archive_write_set_format_v7tar.c head/contrib/libarchive/libarchive/archive_write_set_format_zip.c head/contrib/libarchive/libarchive/test/test_compat_pax_libarchive_2x.c =20 head/contrib/libarchive/libarchive/test/test_compat_pax_libarchive_2x.tar.Z= .uu head/contrib/libarchive/libarchive/test/test_read_format_zip.c head/contrib/libarchive/libarchive/test/test_read_format_zip_bzip2.zipx.uu =20 head/contrib/libarchive/libarchive/test/test_read_format_zip_bzip2_multi.zi= px.uu head/contrib/libarchive/libarchive/test/test_read_format_zip_lzma.zipx.uu =20 head/contrib/libarchive/libarchive/test/test_read_format_zip_lzma_multi.zip= x.uu head/contrib/libarchive/libarchive/test/test_read_format_zip_ppmd8.zipx.uu =20 head/contrib/libarchive/libarchive/test/test_read_format_zip_ppmd8_multi.zi= px.uu head/contrib/libarchive/libarchive/test/test_read_format_zip_xz_multi.zip= x.uu head/contrib/libarchive/test_utils/test_main.c head/lib/libarchive/Makefile head/lib/libarchive/tests/Makefile --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:45:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91F6714DCF8C for ; Wed, 13 Feb 2019 01:45:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2B94F8F4F1 for ; Wed, 13 Feb 2019 01:45:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DE6FE14DCF8B; Wed, 13 Feb 2019 01:45:58 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCD7514DCF8A for ; Wed, 13 Feb 2019 01:45:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69E6E8F4EE for ; Wed, 13 Feb 2019 01:45:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B052AAF96 for ; Wed, 13 Feb 2019 01:45:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1jvXF034712 for ; Wed, 13 Feb 2019 01:45:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1jvuv034710 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:45:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 233006] [regression] [patch] bsdtar aborts creation of archive on ENOENT Date: Wed, 13 Feb 2019 01:45:56 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: patch, regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: mm@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? mfc-stable12? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:45:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233006 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --- Comment #16 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:46:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FD4314DCFD2 for ; Wed, 13 Feb 2019 01:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 275D58F542 for ; Wed, 13 Feb 2019 01:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DA93F14DCFCF; Wed, 13 Feb 2019 01:46:33 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C84DB14DCFCE for ; Wed, 13 Feb 2019 01:46:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 647A28F538 for ; Wed, 13 Feb 2019 01:46:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A0019AF9A for ; Wed, 13 Feb 2019 01:46:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1kWEZ035307 for ; Wed, 13 Feb 2019 01:46:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1kWp8035306 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:46:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235684] security/ipsec-tools kernel panic Date: Wed, 13 Feb 2019 01:46:31 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:46:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235684 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --- Comment #15 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:51:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9A1714DD5DF for ; Wed, 13 Feb 2019 01:51:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 403258FABE for ; Wed, 13 Feb 2019 01:51:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id F2DB314DD5DC; Wed, 13 Feb 2019 01:51:35 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E160214DD5D7 for ; Wed, 13 Feb 2019 01:51:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F2918FABA for ; Wed, 13 Feb 2019 01:51:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B3A57B0DB for ; Wed, 13 Feb 2019 01:51:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1pYP8048066 for ; Wed, 13 Feb 2019 01:51:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1pYMG048065 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:51:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221376] 12.0-RELEASE amd64: any kernel fail to boot - freezes just after run kernel Date: Wed, 13 Feb 2019 01:51:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:51:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221376 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rgrimes@FreeBSD.org Assignee|stable@FreeBSD.org |bugs@FreeBSD.org --- Comment #6 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc Placing this bug back on bugs@ to be triaged properly to a technical list --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:53:01 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C25714DD713 for ; Wed, 13 Feb 2019 01:53:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E43098FC8C for ; Wed, 13 Feb 2019 01:53:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A7D7E14DD70C; Wed, 13 Feb 2019 01:53:00 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 958F614DD70A for ; Wed, 13 Feb 2019 01:53:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31C698FC88 for ; Wed, 13 Feb 2019 01:53:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 5485BB111 for ; Wed, 13 Feb 2019 01:52:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1qxDU052627 for ; Wed, 13 Feb 2019 01:52:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1qxRP052626 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:52:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 235683] [zfs] Panic during data access or scrub on 12.0-STABLE r343904 (blkptr at DVA 0 has invalid OFFSET) Date: Wed, 13 Feb 2019 01:52:59 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:53:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235683 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org |rgrimes@FreeBSD.org --- Comment #1 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:53:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA21114DD828 for ; Wed, 13 Feb 2019 01:53:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 83B5F8FE4F for ; Wed, 13 Feb 2019 01:53:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3D83114DD80D; Wed, 13 Feb 2019 01:53:56 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A27314DD80A for ; Wed, 13 Feb 2019 01:53:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F7AB8FE36 for ; Wed, 13 Feb 2019 01:53:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id CF534B113 for ; Wed, 13 Feb 2019 01:53:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1rnBE053451 for ; Wed, 13 Feb 2019 01:53:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1rn3e053450 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:53:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 193360] [panic] [syscons] random syscons panic Date: Wed, 13 Feb 2019 01:53:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 9.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Not Enough Information X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:53:57 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D193360 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:55:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D55F514DD9A8 for ; Wed, 13 Feb 2019 01:55:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6E9858FF78 for ; Wed, 13 Feb 2019 01:55:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2D86C14DD9A6; Wed, 13 Feb 2019 01:55:17 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B5A314DD9A5 for ; Wed, 13 Feb 2019 01:55:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFC178FF74 for ; Wed, 13 Feb 2019 01:55:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1EE31B129 for ; Wed, 13 Feb 2019 01:55:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1tFMT055084 for ; Wed, 13 Feb 2019 01:55:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1tFgP055083 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:55:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 221146] [ixgbe] Problem with second laggport Date: Wed, 13 Feb 2019 01:55:11 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: IntelNetworking, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:55:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --- Comment #25 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:56:19 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6007B14DDE06 for ; Wed, 13 Feb 2019 01:56:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EA9416809A for ; Wed, 13 Feb 2019 01:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id AB24914DDE01; Wed, 13 Feb 2019 01:56:18 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9737E14DDDFF for ; Wed, 13 Feb 2019 01:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DD4668093 for ; Wed, 13 Feb 2019 01:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 65D0BB12D for ; Wed, 13 Feb 2019 01:56:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1uHaQ056146 for ; Wed, 13 Feb 2019 01:56:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1uHNd056145 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:56:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 227213] FreeBSD 10.4 kernel deadlocks on sysctlmemlock Date: Wed, 13 Feb 2019 01:56:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.4-STABLE X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:56:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227213 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|stable@FreeBSD.org |bugs@FreeBSD.org --- Comment #38 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc Placing this bug back on bugs@ to be triaged properly to a technical list --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:56:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5981114DDED9 for ; Wed, 13 Feb 2019 01:56:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BAF346812B for ; Wed, 13 Feb 2019 01:56:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 74AE914DDED4; Wed, 13 Feb 2019 01:56:53 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6285314DDED0 for ; Wed, 13 Feb 2019 01:56:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E485968122 for ; Wed, 13 Feb 2019 01:56:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3A30DB142 for ; Wed, 13 Feb 2019 01:56:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1uqrH056763 for ; Wed, 13 Feb 2019 01:56:52 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1uqYv056761 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:56:52 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 213903] Kernel crashes from turnstile_broadcast (/usr/src/sys/kern/subr_turnstile.c:837) Date: Wed, 13 Feb 2019 01:56:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: crash, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: mjg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10+ mfc-stable11+ X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:56:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213903 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --- Comment #57 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 01:58:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DD6014DE13C for ; Wed, 13 Feb 2019 01:58:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1511B68336 for ; Wed, 13 Feb 2019 01:58:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id CA3C314DE13A; Wed, 13 Feb 2019 01:58:47 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B79A314DE139 for ; Wed, 13 Feb 2019 01:58:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FFF56832F for ; Wed, 13 Feb 2019 01:58:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6BB98B14D for ; Wed, 13 Feb 2019 01:58:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D1wkTX058780 for ; Wed, 13 Feb 2019 01:58:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D1wkw8058778 for stable@FreeBSD.org; Wed, 13 Feb 2019 01:58:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 230620] "install -d" issue Date: Wed, 13 Feb 2019 01:58:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 01:58:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230620 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|stable@FreeBSD.org |bugs@FreeBSD.org --- Comment #3 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc Placing this bug back on bugs@ to be triaged properly to a technical list --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 02:00:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E639214DE39C for ; Wed, 13 Feb 2019 02:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 74832699F5 for ; Wed, 13 Feb 2019 02:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3077E14DE399; Wed, 13 Feb 2019 02:00:53 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E33114DE398 for ; Wed, 13 Feb 2019 02:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2718699EE for ; Wed, 13 Feb 2019 02:00:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id CD16FB16E for ; Wed, 13 Feb 2019 02:00:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D20pl3075080 for ; Wed, 13 Feb 2019 02:00:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D20prt075077 for stable@FreeBSD.org; Wed, 13 Feb 2019 02:00:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 229694] [zfs] unkillable "zpool scrub" in [tx->tx_sync_done_cv] state for damaged data Date: Wed, 13 Feb 2019 02:00:51 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 02:00:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229694 --- Comment #4 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 02:02:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29F0614DEBCD for ; Wed, 13 Feb 2019 02:02:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8A4F569D5C for ; Wed, 13 Feb 2019 02:02:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4E0F714DEBC7; Wed, 13 Feb 2019 02:02:29 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B4AD14DEBC6 for ; Wed, 13 Feb 2019 02:02:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C84B069D52 for ; Wed, 13 Feb 2019 02:02:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0D6AEB2D1 for ; Wed, 13 Feb 2019 02:02:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D22RN8004096 for ; Wed, 13 Feb 2019 02:02:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D22RNW004086 for stable@FreeBSD.org; Wed, 13 Feb 2019 02:02:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 169898] ifconfig(8) fails to set MTU on multiple interfaces. Date: Wed, 13 Feb 2019 02:02:27 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 02:02:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D169898 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |net@FreeBSD.org --- Comment #3 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc Placing this bug on net@ as was done before eadler reset it --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 02:02:56 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABB0C14DED01 for ; Wed, 13 Feb 2019 02:02:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 355FF69DFF for ; Wed, 13 Feb 2019 02:02:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id ED06614DECFD; Wed, 13 Feb 2019 02:02:55 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA50214DECFC for ; Wed, 13 Feb 2019 02:02:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64CF269DF6 for ; Wed, 13 Feb 2019 02:02:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 95D64B2D8 for ; Wed, 13 Feb 2019 02:02:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D22sAZ006520 for ; Wed, 13 Feb 2019 02:02:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D22soM006518 for stable@FreeBSD.org; Wed, 13 Feb 2019 02:02:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 169898] ifconfig(8) fails to set MTU on multiple interfaces. Date: Wed, 13 Feb 2019 02:02:54 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 02:02:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D169898 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org |rgrimes@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 02:03:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 015AB14DEDCA for ; Wed, 13 Feb 2019 02:03:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8B36069EDD for ; Wed, 13 Feb 2019 02:03:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 49E2C14DEDB9; Wed, 13 Feb 2019 02:03:33 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3717C14DEDB8 for ; Wed, 13 Feb 2019 02:03:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B191E69ED2 for ; Wed, 13 Feb 2019 02:03:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8F23AB2DD for ; Wed, 13 Feb 2019 02:03:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D23Vp5007279 for ; Wed, 13 Feb 2019 02:03:31 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D23Vn0007275 for stable@FreeBSD.org; Wed, 13 Feb 2019 02:03:31 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 193364] [panic] ffs_blkfree_cg: freeing free block Date: Wed, 13 Feb 2019 02:03:31 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 02:03:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D193364 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC|stable@FreeBSD.org | --- Comment #5 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 02:04:37 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4A8814DEF2B for ; Wed, 13 Feb 2019 02:04:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9C9646A0A6 for ; Wed, 13 Feb 2019 02:04:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 5BC7914DEF21; Wed, 13 Feb 2019 02:04:35 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 499EA14DEF20 for ; Wed, 13 Feb 2019 02:04:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D324B6A0A0 for ; Wed, 13 Feb 2019 02:04:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 23272B2E5 for ; Wed, 13 Feb 2019 02:04:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1D24YD3008568 for ; Wed, 13 Feb 2019 02:04:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1D24Yla008567 for stable@FreeBSD.org; Wed, 13 Feb 2019 02:04:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 193386] [panic] resource_list_alloc: resource entry is busy Date: Wed, 13 Feb 2019 02:04:33 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 02:04:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D193386 --- Comment #1 from Rodney W. Grimes --- Please do not put bugs on stable@, current@, hackers@, etc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 10:10:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3401714EA411 for ; Wed, 13 Feb 2019 10:10:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B790382E70 for ; Wed, 13 Feb 2019 10:10:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 74C3B14EA40C; Wed, 13 Feb 2019 10:10:47 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6213B14EA40B for ; Wed, 13 Feb 2019 10:10:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA9CE82E6B for ; Wed, 13 Feb 2019 10:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 25BBFFA8D for ; Wed, 13 Feb 2019 10:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1DAAkh5038030 for ; Wed, 13 Feb 2019 10:10:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1DAAkXY038029 for stable@FreeBSD.org; Wed, 13 Feb 2019 10:10:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: stable@FreeBSD.org Subject: [Bug 229694] [zfs] unkillable "zpool scrub" in [tx->tx_sync_done_cv] state for damaged data Date: Wed, 13 Feb 2019 10:10:45 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 10:10:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229694 --- Comment #5 from Andriy Gapon --- (In reply to Eugene Grosbein from comment #3) 5 101937 zfskern txg_thread_enter mi_switch+0xc5 sleepq_wait+0x2c _cv_wait+0x160 zio_resume_wait+0x4b spa_sync+0xd46 txg_sync_thread+0x25e fork_exit+0x75 fork_trampoline+0xe=20 3249 101681 zpool - mi_switch+0xc5 sleepq_wait+0x2c _cv_wait+0x160 txg_wait_synced+0xa5 dsl_sync_task_common+0= x219 dsl_sync_task+0x14 dsl_scan+0x9e zfs_ioc_pool_scan+0x5a zfsdev_ioctl+0x6c2 devfs_ioctl_f+0x12d kern_ioctl+0x212 sys_ioctl+0x15c amd64_syscall+0x25c fast_syscall_common+0x101 So, unfortunately, this is how ZFS works now. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Feb 13 10:22:09 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BC0114EAB40 for ; Wed, 13 Feb 2019 10:22:09 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44F2583886 for ; Wed, 13 Feb 2019 10:22:08 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf1-f54.google.com with SMTP id q12so1375218lfm.0 for ; Wed, 13 Feb 2019 02:22:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=+wBl0oQwFvuETX4xmEojv8k0VQAkFn86arCK0ozUblU=; b=spBp8SiZUA9YGRL/oumSK+hDIR2qO4BudD8VS4CFwA+32VWxB2fYDjfDrBgV+u+iGd WQr1l+VD96qcj+5BC4/HvKTmXIM/RRY/6ifar7cJPOS4ybK4N4+Y603VcFkEEwlhckDT 54L6wGUPvNpYscoDuYXKMZWTwwUyy7xMIvopypJdEJzJjbOVfb7I31gjrPJ8SBLZmneS ZTLpqjJAiSvb9pWRAAjO8eiM3vp1xxoxqZ7KVTgWuVWFg0WG1P33Oqhc81kJ08p8q+gr NYS7UITuoHf2CfM3au7I+5ZiZuFsTVe5QMvTsxjaFucPtrlwMGHibYtMRs6wQXDHSe+J wmAg== X-Gm-Message-State: AHQUAuYlUIPj5ckDMxFbiLz4MLAQWSF6L6/9Wr68TmM8tJC8pR7oDlsT mBvWG9gmQE2I2+xdQOvXmRUeHJcH X-Google-Smtp-Source: AHgI3IZdLDzWR8qkVZhlexSpVrbbHdfFWZBBxbTtAKib297IzMV0bFBnqm5XA7E34OVvA5VA5hV7Fg== X-Received: by 2002:a19:c48e:: with SMTP id u136mr5021780lff.167.1550051942060; Wed, 13 Feb 2019 01:59:02 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id d70sm3648480lfe.14.2019.02.13.01.59.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 01:59:01 -0800 (PST) Subject: Re: 11.2-STABLE kernel wired memory leak To: Eugene Grosbein , Garrett Wollman , freebsd-stable@freebsd.org References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <0710bc0a-7691-b29f-3331-da94412ab737@FreeBSD.org> Date: Wed, 13 Feb 2019 11:59:00 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 44F2583886 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.167.54 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-4.26 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; IP_SCORE(-1.29)[ip: (-0.65), ipnet: 209.85.128.0/17(-3.77), asn: 15169(-1.97), country: US(-0.07)]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.96)[-0.960,0]; RCVD_IN_DNSWL_NONE(0.00)[54.167.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 10:22:09 -0000 On 12/02/2019 20:17, Eugene Grosbein wrote: > 13.02.2019 1:14, Eugene Grosbein wrote: > >> Use following command to see how much memory is wasted in your case: >> >> vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > > Oops, small correction: > > vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head I have a much uglier but somewhat more informative "one-liner" for post-processing vmstat -z output: vmstat -z | tail +3 | awk -F '[:,] *' 'BEGIN { total=0; cache=0; used=0 } {u = $2 * $4; c = $2 * $5; t = u + c; cache += c; used += u; total += t; name=$1; gsub(" ", "_", name); print t, name, u, c} END { print total, "TOTAL", used, cache } ' | sort -n | perl -a -p -e 'while (($j, $_) = each(@F)) { 1 while s/^(-?\d+)(\d{3})/$1,$2/; print $_, " "} print "\n"' | column -t This would be much nicer as a small python script. Or, even, we could add a sort option for vmstat -z / -m. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Wed Feb 13 12:04:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4612914EDA7E for ; Wed, 13 Feb 2019 12:04:39 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 712CE8744C; Wed, 13 Feb 2019 12:04:28 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1DC4G5d080089 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Feb 2019 13:04:16 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x1DC4Fev061720 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 13 Feb 2019 19:04:15 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <9a263536-45e3-c690-e45e-d8ece7d1f388@grosbein.net> Date: Wed, 13 Feb 2019 19:04:14 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190212163446.GA29847@raichu> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 712CE8744C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.70 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; URIBL_BLOCKED(0.00)[grosbein.net.multi.uribl.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; MX_INVALID(0.50)[greylisted]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.36)[ip: (-2.18), ipnet: 2a01:4f8::/29(-2.39), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.74)[-0.744,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 12:04:39 -0000 12.02.2019 23:34, Mark Johnston wrote: > On Tue, Feb 12, 2019 at 11:14:31PM +0700, Eugene Grosbein wrote: >> Hi! >> >> Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel wired memory over 81 days uptime >> out of 8GB total RAM. >> >> Details follow. >> >> I have a workstation running Xorg, Firefox, Thunderbird, LibreOffice and occasionally VirtualBox for single VM. >> >> It has two identical 320GB HDDs combined with single graid-based array with "Intel" >> on-disk format having 3 volumes: >> - one "RAID1" volume /dev/raid/r0 occupies first 10GB or each HDD; >> - two "SINGLE" volumes /dev/raid/r1 and /dev/raid/r2 that utilize "tails" of HDDs (310GB each). >> >> /dev/raid/r0 (10GB) has MBR partitioning and two slices: >> - /dev/raid/r0s1 (8GB) is used for swap; >> - /dev/raid/r0s2 (2GB) is used by non-redundant ZFS pool named "os" that contains only >> root file system (177M used) and /usr file system (340M used). >> >> There is also second pool (ZMIRROR) named "z" built directly on top of /dev/raid/r[12] volumes, >> this pool contains all other file systems including /var, /home, /usr/ports, /usr/local, /usr/{src|obj} etc. >> >> # zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT >> os 1,98G 520M 1,48G - - 55% 25% 1.00x ONLINE - >> z 288G 79,5G 209G - - 34% 27% 1.00x ONLINE - >> >> This way I have swap outside of ZFS, boot blocks and partitioning mirrored by means of GEOM_RAID and >> can use local console to break to single user mode to unmount all file system other than root and /usr >> and can even export bigger ZFS pool "z". And I did that to see that ARC usage >> (limited with vfs.zfs.arc_max="3G" in /boot/loader.conf) dropped from over 2500MB >> down to 44MB but "Wired" stays high. Now after I imported "z" back and booted to multiuser mode >> top(1) shows: >> >> last pid: 51242; load averages: 0.24, 0.16, 0.13 up 81+02:38:38 22:59:18 >> 104 processes: 1 running, 103 sleeping >> CPU: 0.0% user, 0.0% nice, 0.4% system, 0.2% interrupt, 99.4% idle >> Mem: 84M Active, 550M Inact, 4K Laundry, 4689M Wired, 2595M Free >> ARC: 273M Total, 86M MFU, 172M MRU, 64K Anon, 1817K Header, 12M Other >> 117M Compressed, 333M Uncompressed, 2.83:1 Ratio >> Swap: 8192M Total, 940K Used, 8191M Free >> >> I have KDB and DDB in my custom kernel also. How do I debug the leak further? >> >> I use nvidia-driver-340-340.107 driver for GK208 [GeForce GT 710B] video card. >> Here are outputs of "vmstat -m": http://www.grosbein.net/freebsd/leak/vmstat-m.txt >> and "vmstat -z": http://www.grosbein.net/freebsd/leak/vmstat-z.txt > > I suspect that the "leaked" memory is simply being used to cache UMA > items. Note that the values in the FREE column of vmstat -z output are > quite large. The cached items are reclaimed only when the page daemon > wakes up to reclaim memory; if there are no memory shortages, large > amounts of memory may accumulate in UMA caches. In this case, the sum > of the product of columns 2 and 5 gives a total of roughly 4GB cached. After another day with mostly idle system, "Wired" increased to more than 6GB out of 8GB total. I've tried to increase sysctl vm.v_free_min from default 12838 (50MB) upto 1048576 (4GB) and "Wired" dropped a bit but it is still huge, 5060M: last pid: 61619; load averages: 1.05, 0.78, 0.40 up 81+22:33:09 18:53:49 119 processes: 1 running, 118 sleeping CPU: 0.0% user, 0.0% nice, 50.0% system, 0.0% interrupt, 50.0% idle Mem: 47M Active, 731M Inact, 4K Laundry, 5060M Wired, 2080M Free ARC: 3049M Total, 216M MFU, 2428M MRU, 64K Anon, 80M Header, 325M Other 2341M Compressed, 5874M Uncompressed, 2.51:1 Ratio Swap: 8192M Total, 940K Used, 8191M Free # sysctl vm.v_free_min vm.v_free_min: 1048576 # sysctl vm.stats.vm.v_free_count vm.stats.vm.v_free_count: 533232 ARC probably cached results of nightly periodic jobs traversing file system trees and hit its limit (3G). Still cannot understand where have another 2G (5G-3G) of wired memory leaked to? USED: # vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$4/1024/1024, $1}' | sort -k1,1 -rn | head 2763,2 abd_chunk 196,547 zio_buf_16384 183,711 dnode_t 128,304 zio_buf_512 96,3062 VNODE 79,0076 arc_buf_hdr_t_full 66,5 zio_data_buf_131072 63,0772 UMA Slabs 61,6484 256 61,2564 dmu_buf_impl_t FREE: # vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head 245,301 dnode_t 209,086 zio_buf_512 110,163 dmu_buf_impl_t 31,2598 64 21,656 256 10,9262 swblk 10,6295 128 9,0379 RADIX NODE 8,54521 L VFS Cache 7,4917 512 From owner-freebsd-stable@freebsd.org Wed Feb 13 13:28:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E911914EF5D0 for ; Wed, 13 Feb 2019 13:28:34 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0EBF089C1F for ; Wed, 13 Feb 2019 13:28:33 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 7DCC9211098 for ; Wed, 13 Feb 2019 08:28:02 -0500 (EST) Received: from [192.168.10.19] (D9.Denninger.Net [192.168.10.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id D990AC4949 for ; Wed, 13 Feb 2019 07:28:01 -0600 (CST) Subject: Re: 11.2-STABLE kernel wired memory leak To: freebsd-stable@freebsd.org References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <0710bc0a-7691-b29f-3331-da94412ab737@FreeBSD.org> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <116f047d-8c1b-88d2-f914-0eef50ca281d@denninger.net> Date: Wed, 13 Feb 2019 07:28:02 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <0710bc0a-7691-b29f-3331-da94412ab737@FreeBSD.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms030509060104090502030302" X-Rspamd-Queue-Id: 0EBF089C1F X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; MIME_BASE64_TEXT(0.10)[]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.34)[ip: (-9.89), ipnet: 104.236.64.0/18(-4.41), asn: 14061(2.69), country: US(-0.07)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 13:28:35 -0000 This is a cryptographically signed message in MIME format. --------------ms030509060104090502030302 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 DQpPbiAyLzEzLzIwMTkgMDM6NTksIEFuZHJpeSBHYXBvbiB3cm90ZToNCj4gT24gMTIvMDIv MjAxOSAyMDoxNywgRXVnZW5lIEdyb3NiZWluIHdyb3RlOg0KPj4gMTMuMDIuMjAxOSAxOjE0 LCBFdWdlbmUgR3Jvc2JlaW4gd3JvdGU6DQo+Pg0KPj4+IFVzZSBmb2xsb3dpbmcgY29tbWFu ZCB0byBzZWUgaG93IG11Y2ggbWVtb3J5IGlzIHdhc3RlZCBpbiB5b3VyIGNhc2U6DQo+Pj4N Cj4+PiB2bXN0YXQgLXogfCBhd2sgLUYsICd7cHJpbnRmICIlMTBzICVzXG4iLCAkMiokNS8x MDI0LzEwMjQsICQxfScgfCBzb3J0IC1rMSwxIC1ybiB8IGhlYWQNCj4+IE9vcHMsIHNtYWxs IGNvcnJlY3Rpb246DQo+Pg0KPj4gdm1zdGF0IC16IHwgc2VkICdzLzovLC8nIHwgYXdrIC1G LCAne3ByaW50ZiAiJTEwcyAlc1xuIiwgJDIqJDUvMTAyNC8xMDI0LCAkMX0nIHwgc29ydCAt azEsMSAtcm4gfCBoZWFkDQo+IEkgaGF2ZSBhIG11Y2ggdWdsaWVyIGJ1dCBzb21ld2hhdCBt b3JlIGluZm9ybWF0aXZlICJvbmUtbGluZXIiIGZvcg0KPiBwb3N0LXByb2Nlc3Npbmcgdm1z dGF0IC16IG91dHB1dDoNCj4NCj4gdm1zdGF0IC16IHwgdGFpbCArMyB8IGF3ayAtRiAnWzos XSAqJyAnQkVHSU4geyB0b3RhbD0wOyBjYWNoZT0wOyB1c2VkPTAgfSB7dSA9DQo+ICQyICog JDQ7IGMgPSAkMiAqICQ1OyB0ID0gdSArIGM7IGNhY2hlICs9IGM7IHVzZWQgKz0gdTsgdG90 YWwgKz0gdDsgbmFtZT0kMTsNCj4gZ3N1YigiICIsICJfIiwgbmFtZSk7IHByaW50IHQsIG5h bWUsIHUsIGN9IEVORCB7IHByaW50IHRvdGFsLCAiVE9UQUwiLCB1c2VkLA0KPiBjYWNoZSB9 ICcgfCBzb3J0IC1uIHwgcGVybCAtYSAtcCAtZSAnd2hpbGUgKCgkaiwgJF8pID0gZWFjaChA RikpIHsgMSB3aGlsZQ0KPiBzL14oLT9cZCspKFxkezN9KS8kMSwkMi87IHByaW50ICRfLCAi ICJ9IHByaW50ICJcbiInIHwgY29sdW1uIC10DQo+DQo+IFRoaXMgd291bGQgYmUgbXVjaCBu aWNlciBhcyBhIHNtYWxsIHB5dGhvbiBzY3JpcHQuDQo+IE9yLCBldmVuLCB3ZSBjb3VsZCBh ZGQgYSBzb3J0IG9wdGlvbiBmb3Igdm1zdGF0IC16IC8gLW0uDQoNCk9uIDEyLVNUQUJMRSwg cjM0MzgwOSwgL3dpdGhvdXQgL215IHBhdGNoIHNldCAoYW5kIEkndmUgeWV0IHRvIHNlZSB0 aGUNCnNvcnQgb2Ygc3RhbGwgcHJvYmxlbXMgSSB1c2VkIHRvIGhhdmUgcmVndWxhcmx5IHdp dGggMTEuMSkNCg0Kcm9vdEBOZXdGUzovZGlzay9rYXJsICMgc2ggdm1zdGF0LnNjcmlwdA0K MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCBBSU/CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCBBSU9DQsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgQUlPTElPwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgQUlPUMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCBETUFSX01BUF9FTlRSWcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgRkZTMV9kaW5vZGXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIEZQ VV9zYXZlX2FyZWHCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgSVBGV19JUHY2 X2R5bmFtaWNfc3RhdGVzwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQow wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgSVBGV19wYXJlbnRfZHluYW1pY19zdGF0ZXPC oCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIElQc2VjX1NBX2xmdF9jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBM VFNfVkZTX0NhY2hlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIE1BQ19sYWJl bHPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgTkNMTk9ERcKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBTVFNfVkZTX0NhY2hl wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGFyY19idWZfaGRyX3RfbDJvbmx5 wqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCBhdWRpdF9yZWNvcmTCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCBjcnlwdG9kZXNjwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIGNyeXB0b3DCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgZG9tYWluc2V0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgaXRpbWVywqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgbWJ1Zl9qdW1ib18xNmvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIG1i dWZfanVtYm9fOWvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgbnZtZV9yZXF1 ZXN0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcHJvY2Rlc2PCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJlbnRywqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzY3RwX2FzY29uZsKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2N0cF9hc2NvbmZfYWNrwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2N0cF9hc29jwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgc2N0cF9jaHVua8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBzY3RwX2VwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIHNjdHBfcmFkZHLCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgc2N0cF9yZWFkccKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCBzY3RwX3N0cmVhbV9tc2dfb3V0wqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHNpb19j YWNoZcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRjcF9sb2fC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdGNwX2xvZ19i dWNrZXTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRjcF9sb2dfbm9kZcKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg MA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRmb8KgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRmb19jY2FjaGVfZW50cmllc8Kg wqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVkcGxpdGVfaW5wY2LCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgdW10eF9wacKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCB2dG5ldF90eF9oZHLCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCB6aW9fYnVmXzEwNDg1NzYwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHpp b19idWZfMTI1ODI5MTLCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2J1Zl8xNDY4 MDA2NMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB6aW9fYnVmXzE1NzI4NjTCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB6aW9fYnVmXzE2Nzc3MjE2wqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIHppb19idWZfMTgzNTAwOMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAw wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIHppb19idWZfMjA5NzE1MsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19i dWZfMjYyMTQ0MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19idWZfMzE0NTcy OMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg MA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19idWZfMzY3MDAxNsKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19idWZfNDE5NDMwNMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIHppb19idWZfNTI0Mjg4MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAw wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIHppb19idWZfNjI5MTQ1NsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19i dWZfNzM0MDAzMsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19idWZfODM4ODYw OMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg MA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl8xMDQ4NTc2wqDC oMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfMTA0ODU3NjDCoMKgwqDCoMKgwqAgMMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCB6aW9fZGF0YV9idWZfMTI1ODI5MTLCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZf MTMxMDcyMMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQow wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzE0NjgwMDY0wqDCoMKg wqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzE1NzI4NjTCoMKgwqDCoMKgwqDCoCAwwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHpp b19kYXRhX2J1Zl8xNjM4NDDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzE2 Nzc3MjE2wqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzE4MzUwMDjCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIHppb19kYXRhX2J1Zl8xOTY2MDjCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlv X2RhdGFfYnVmXzIwOTcxNTLCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl8yMjkz NzbCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzI2MjE0NMKgwqDCoMKgwqDC oMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfMjYyMTQ0MMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlv X2RhdGFfYnVmXzMxNDU3MjjCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl8zMjc2 ODDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzM2NzAwMTbCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIHppb19kYXRhX2J1Zl8zOTMyMTbCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlv X2RhdGFfYnVmXzQxOTQzMDTCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl80NTg3 NTLCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzUyNDI4OMKgwqDCoMKgwqDC oMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDANCjDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfNTI0Mjg4MMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlv X2RhdGFfYnVmXzYyOTE0NTbCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl82NTUz NjDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzczNDAwMzLCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIHppb19kYXRhX2J1Zl83ODY0MzLCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAwDQowwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgemlv X2RhdGFfYnVmXzgzODg2MDjCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMA0KMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl85MTc1 MDTCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwDQo3LDE2 OMKgwqDCoMKgwqDCoMKgwqDCoCB2bWVtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCA1LDM3NsKgwqDCoMKgwqDCoMKgwqDCoCAxLDc5Mg0KMTUsNzc2 wqDCoMKgwqDCoMKgwqDCoCBNQVDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgNjk2wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxNSwwODANCjE2LDM4 NMKgwqDCoMKgwqDCoMKgwqAgZXBvY2hfcmVjb3JkX3BjcHXCoMKgwqDCoMKgwqDCoMKgwqDC oCAxLDAyNMKgwqDCoMKgwqDCoMKgwqDCoCAxNSwzNjANCjIwLDAwMMKgwqDCoMKgwqDCoMKg wqAgc2Fja2hvbGXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMjAsMDAwDQoyMCwwNDDCoMKgwqDCoMKgwqDCoMKg IG10X3pvbmXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEwLDUx MsKgwqDCoMKgwqDCoMKgwqAgOSw1MjgNCjI2LDg4MMKgwqDCoMKgwqDCoMKgwqAgVU1BX0hh c2jCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxMywzMTLCoMKgwqDC oMKgwqDCoMKgIDEzLDU2OA0KMjcsODg4wqDCoMKgwqDCoMKgwqDCoCB0Y3ByZWFzc8KgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAyNyw4ODgNCjMxLDgwOMKgwqDCoMKgwqDCoMKgwqAgaXBxwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAzMSw4MDgNCjQwLDk2MMKgwqDCoMKgwqDCoMKgwqAgNjRfcGNwdcKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMjYsMDQwwqDCoMKgwqDCoMKg wqDCoCAxNCw5MjANCjQ5LDE1MsKgwqDCoMKgwqDCoMKgwqAgSVBGV19jb3VudGVyc8KgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOTkywqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA0OCwx NjANCjQ5LDE1MsKgwqDCoMKgwqDCoMKgwqAgbXRfc3RhdHNfem9uZcKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgMjgsMDMywqDCoMKgwqDCoMKgwqDCoCAyMSwxMjANCjUxLDc5MsKg wqDCoMKgwqDCoMKgwqAgc2N0cF9sYWRkcsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDUxLDc5Mg0KNTIsMTA0wqDCoMKg wqDCoMKgwqDCoCBjcnlwdG9fc2Vzc2lvbsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEs MzQ0wqDCoMKgwqDCoMKgwqDCoMKgIDUwLDc2MA0KNjAsOTI4wqDCoMKgwqDCoMKgwqDCoCBV TUFfS2Vnc8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDU3LDU2OMKg wqDCoMKgwqDCoMKgwqAgMywzNjANCjYwLDkyOMKgwqDCoMKgwqDCoMKgwqAgemlvX2J1Zl8z NTg0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCA2MCw5MjgNCjc0LDc4NMKgwqDCoMKgwqDCoMKgwqAgaG9zdGNhY2hlwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEsMDU2wqDCoMKgwqDCoMKgwqDCoMKg IDczLDcyOA0KNzUsMjQwwqDCoMKgwqDCoMKgwqDCoCBLTUFQX0VOVFJZwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxLDQ0MMKgwqDCoMKgwqDCoMKgwqDCoCA3Myw4MDAN CjgxLDkyMMKgwqDCoMKgwqDCoMKgwqAgemlvX2J1Zl81MTIwwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIDEwLDI0MMKgwqDCoMKgwqDCoMKgwqAgNzEsNjgwDQo5NCw4NDjCoMKg wqDCoMKgwqDCoMKgIGZha2VwZ8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOTQsODQ4DQoxMDgsODY0wqDC oMKgwqDCoMKgwqAgc3dwY3RyaWXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAzMiwxMTLCoMKgwqDCoMKgwqDCoMKgIDc2LDc1Mg0KMTEwLDU5MsKgwqDCoMKgwqDC oMKgIHppb19idWZfNjE0NMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxMiwyODjC oMKgwqDCoMKgwqDCoMKgIDk4LDMwNA0KMTExLDM2MMKgwqDCoMKgwqDCoMKgIHppbF9sd2Jf Y2FjaGXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEsOTIwwqDCoMKgwqDCoMKgwqDC oMKgIDEwOSw0NDANCjExOSw4MDjCoMKgwqDCoMKgwqDCoCB6aW9fYnVmXzMwNzLCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMTUsMzYwwqDCoMKgwqDCoMKgwqDCoCAxMDQsNDQ4 DQoxMjEsODU2wqDCoMKgwqDCoMKgwqAgemlvX2J1Zl83MTY4wqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxMjEsODU2DQoxMjMs NjQ4wqDCoMKgwqDCoMKgwqAgc3luY2FjaGXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMTIzLDY0OA0KMTI0LDAw MMKgwqDCoMKgwqDCoMKgIHJsX2VudHJ5wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMjgsMDAwwqDCoMKgwqDCoMKgwqDCoCA5NiwwMDANCjEzMCw0MTbCoMKgwqDC oMKgwqDCoCBydGVudHJ5wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCAzLDUzNsKgwqDCoMKgwqDCoMKgwqDCoCAxMjYsODgwDQoxMzIsMDAwwqDCoMKgwqDCoMKg wqAgSVBGV19JUHY0X2R5bmFtaWNfc3RhdGVzwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAxMzIsMDAwDQoxMzIsMDAwwqDCoMKgwqDCoMKgwqAgdWRwY2LCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA4MDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIDEzMSwyMDANCjEzMiwwMDDCoMKgwqDCoMKgwqDCoCB6aWxfemN3X2NhY2hlwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg MTMyLDAwMA0KMTM4LDYwMMKgwqDCoMKgwqDCoMKgIG1ldGFzbGFiX2FsbG9jX3RyYWNlX2Nh Y2hlwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEzOCw2MDANCjE1NCw0NDDCoMKg wqDCoMKgwqDCoCBJUEZXX2R5bmFtaWNfc3RhdGVzX2RhdGHCoMKgwqAgMMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIDE1NCw0NDANCjE1NiwwMDDCoMKgwqDCoMKgwqDCoCB0dHlpbnHC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMzAsMDgwwqDCoMKg wqDCoMKgwqDCoCAxMjUsOTIwDQoxNTcsOTc2wqDCoMKgwqDCoMKgwqAgY3B1c2V0wqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDIsODA4wqDCoMKgwqDCoMKg wqDCoMKgIDE1NSwxNjgNCjE3MSw3NzbCoMKgwqDCoMKgwqDCoCByaXBjYsKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDk3NsKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMTcwLDgwMA0KMTgzLDI2NMKgwqDCoMKgwqDCoMKgIHRhc2txX3pvbmXCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDQ4wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDE4MywyMTYNCjE4OSw0NDDCoMKgwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfNTEyMMKgwqDC oMKgwqDCoMKgwqDCoMKgIDExMiw2NDDCoMKgwqDCoMKgwqDCoCA3Niw4MDANCjE5NCw1NjDC oMKgwqDCoMKgwqDCoCB6aW9fYnVmXzEwMjQwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCA1MSwyMDDCoMKgwqDCoMKgwqDCoMKgIDE0MywzNjANCjE5OCwwMDDCoMKgwqDCoMKgwqDC oCB0Y3B0d8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxOTgsMDAwDQoyMDAsNzA0wqDCoMKgwqDCoMKg wqAgemlvX2RhdGFfYnVmXzM1ODTCoMKgwqDCoMKgwqDCoMKgwqDCoCA3MSw2ODDCoMKgwqDC oMKgwqDCoMKgIDEyOSwwMjQNCjIwNCw4MDDCoMKgwqDCoMKgwqDCoCBOQU1FScKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAyMDQsODAwDQoyMTUsMDQwwqDCoMKgwqDCoMKgwqAgemlvX2J1Zl8xNDMz NsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDIxNSwwNDANCjIxNSwwNDDCoMKgwqDCoMKgwqDCoCB6aW9fYnVmXzI1NjDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMTEyLDY0MMKgwqDCoMKgwqDCoMKgIDEwMiw0MDAN CjIyNSw3MjDCoMKgwqDCoMKgwqDCoCBWTk9ERVBPTEzCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgMiw2NDDCoMKgwqDCoMKgwqDCoMKgwqAgMjIzLDA4MA0KMjI3LDg0 MMKgwqDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl8yNTYwwqDCoMKgwqDCoMKgwqDCoMKgwqAg NDAsOTYwwqDCoMKgwqDCoMKgwqDCoCAxODYsODgwDQoyMzAsMzM2wqDCoMKgwqDCoMKgwqAg dWRwX2lucGNiwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEyLDIwMMKg wqDCoMKgwqDCoMKgwqAgMjE4LDEzNg0KMjMwLDY0MMKgwqDCoMKgwqDCoMKgIFNMRUVQUVVF VUXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDE3MSw0NDDCoMKgwqDCoMKg wqDCoCA1OSwyMDANCjIzNCwyNDDCoMKgwqDCoMKgwqDCoCB0dHlvdXRxwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyNiwzNjjCoMKgwqDCoMKgwqDCoMKgIDIw Nyw4NzINCjI0MiwwNDjCoMKgwqDCoMKgwqDCoCB0Y3BfaW5wY2LCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMjcsMzI4wqDCoMKgwqDCoMKgwqDCoCAyMTQsNzIwDQoy NDIsMDQ4wqDCoMKgwqDCoMKgwqAgdGNwY2LCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCA1NCw2NTbCoMKgwqDCoMKgwqDCoMKgIDE4NywzOTINCjI0OCww MDDCoMKgwqDCoMKgwqDCoCBLTk9URcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIDY5LDYwMMKgwqDCoMKgwqDCoMKgwqAgMTc4LDQwMA0KMjUxLDkwNMKg wqDCoMKgwqDCoMKgIDEyX0J1Y2tldMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAzNSwzMjjCoMKgwqDCoMKgwqDCoMKgIDIxNiw1NzYNCjI1NCw4MDDCoMKgwqDCoMKg wqDCoCBrc2lnaW5mb8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDQw LDQzMsKgwqDCoMKgwqDCoMKgwqAgMjE0LDM2OA0KMjU4LDA0OMKgwqDCoMKgwqDCoMKgIHpp b19kYXRhX2J1Zl8zMDcywqDCoMKgwqDCoMKgwqDCoMKgwqAgNTIsMjI0wqDCoMKgwqDCoMKg wqDCoCAyMDUsODI0DQoyNTgsNDAwwqDCoMKgwqDCoMKgwqAgcGlwZcKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMTI4LDQ0MMKgwqDCoMKgwqDCoMKg IDEyOSw5NjANCjI2NSwyMTbCoMKgwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfNzE2OMKgwqDC oMKgwqDCoMKgwqDCoMKgIDEwMCwzNTLCoMKgwqDCoMKgwqDCoCAxNjQsODY0DQoyNzIsNjQw wqDCoMKgwqDCoMKgwqAgdW5wY2LCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCA0NSw4MjTCoMKgwqDCoMKgwqDCoMKgIDIyNiw4MTYNCjI3Nyw3NjDCoMKg wqDCoMKgwqDCoCBzZWxmZMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIDUxLDAwOMKgwqDCoMKgwqDCoMKgwqAgMjI2LDc1Mg0KMjc4LDUyOMKgwqDCoMKg wqDCoMKgIERJUkhBU0jCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyNzgsNTI4DQoyODAsMDAwwqDCoMKgwqDC oMKgwqAgRmlsZXPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCA4MSw4NDDCoMKgwqDCoMKgwqDCoMKgIDE5OCwxNjANCjI4Miw2MjTCoMKgwqDCoMKgwqDC oCB6aW9fYnVmXzE1MzbCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNTMsNzYwwqDC oMKgwqDCoMKgwqDCoCAyMjgsODY0DQoyODgsNzY4wqDCoMKgwqDCoMKgwqAgemlvX2RhdGFf YnVmXzYxNDTCoMKgwqDCoMKgwqDCoMKgwqDCoCAxNTMsNjAwwqDCoMKgwqDCoMKgwqAgMTM1 LDE2OA0KMzQyLDcyMMKgwqDCoMKgwqDCoMKgIFRVUk5TVElMRcKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAyOTEsNDQ4wqDCoMKgwqDCoMKgwqAgNTEsMjcyDQozNTcs Njk2wqDCoMKgwqDCoMKgwqAgZmlsZWRlc2MwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDIwOSw3NjDCoMKgwqDCoMKgwqDCoCAxNDcsOTM2DQozNjYsNzkywqDCoMKg wqDCoMKgwqAgc3dibGvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAzMDMsMTQ0wqDCoMKgwqDCoMKgwqAgNjMsNjQ4DQozNjgsNjQwwqDCoMKgwqDCoMKg wqAgemlvX2RhdGFfYnVmXzE1MzbCoMKgwqDCoMKgwqDCoMKgwqDCoCA0NCw1NDTCoMKgwqDC oMKgwqDCoMKgIDMyNCwwOTYNCjM3MCw0NDDCoMKgwqDCoMKgwqDCoCBNb3VudHBvaW50c8Kg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDI5Myw2MDjCoMKgwqDCoMKgwqDCoCA3 Niw4MzINCjM4NywwNzLCoMKgwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfMTQzMzbCoMKgwqDC oMKgwqDCoMKgwqAgMjAwLDcwNMKgwqDCoMKgwqDCoMKgIDE4NiwzNjgNCjQxMyw2OTbCoMKg wqDCoMKgwqDCoCB6aW9fYnVmXzEwMjTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg NDgsMTI4wqDCoMKgwqDCoMKgwqDCoCAzNjUsNTY4DQo0NDgsMzg0wqDCoMKgwqDCoMKgwqAg MTZfQnVja2V0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDMzLDI4MMKg wqDCoMKgwqDCoMKgwqAgNDE1LDEwNA0KNDkxLDUyMMKgwqDCoMKgwqDCoMKgIHppb19kYXRh X2J1Zl8xMDI0MMKgwqDCoMKgwqDCoMKgwqDCoCAyNjYsMjQwwqDCoMKgwqDCoMKgwqAgMjI1 LDI4MA0KNDkyLDAzMsKgwqDCoMKgwqDCoMKgIDhfQnVja2V0wqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMTksNDU2wqDCoMKgwqDCoMKgwqDCoCA0NzIsNTc2DQo0 OTQsMDE2wqDCoMKgwqDCoMKgwqAgNl9CdWNrZXTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAzNiw1NzbCoMKgwqDCoMKgwqDCoMKgIDQ1Nyw0NDANCjUxMyw5MzbC oMKgwqDCoMKgwqDCoCBQUk9DwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAyNTAsOTkywqDCoMKgwqDCoMKgwqAgMjYyLDk0NA0KNTE2LDA5NsKgwqDC oMKgwqDCoMKgIHppb19kYXRhX2J1Zl8yMDQ4wqDCoMKgwqDCoMKgwqDCoMKgwqAgNjEsNDQw wqDCoMKgwqDCoMKgwqDCoCA0NTQsNjU2DQo1MjMsMjAwwqDCoMKgwqDCoMKgwqAgc29ja2V0 wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDIzMSw5NTLCoMKg wqDCoMKgwqDCoCAyOTEsMjQ4DQo1MzcsNjgwwqDCoMKgwqDCoMKgwqAgZ19iaW/CoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA2LDc2OMKgwqDCoMKgwqDC oMKgwqDCoCA1MzAsOTEyDQo1NDAsOTYwwqDCoMKgwqDCoMKgwqAgVU1BX1pvbmVzwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDUzOSwwMDDCoMKgwqDCoMKgwqDCoCAx LDk2MA0KNTU3LDA1NsKgwqDCoMKgwqDCoMKgIG1idWZfanVtYm9fcGFnZcKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNTU3LDA1Ng0KNTY1 LDI0OMKgwqDCoMKgwqDCoMKgIHppb19idWZfMTIyODjCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIDEyLDI4OMKgwqDCoMKgwqDCoMKgwqAgNTUyLDk2MA0KNTgxLDYzMsKgwqDCoMKg wqDCoMKgIHppb19idWZfMjA0OMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyNzYs NDgwwqDCoMKgwqDCoMKgwqAgMzA1LDE1Mg0KNTk4LDAxNsKgwqDCoMKgwqDCoMKgIHppb19k YXRhX2J1Zl8xMDI0wqDCoMKgwqDCoMKgwqDCoMKgwqAgMzMsNzkywqDCoMKgwqDCoMKgwqDC oCA1NjQsMjI0DQo2MDUsMzkywqDCoMKgwqDCoMKgwqAgVk1TUEFDRcKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMjUxLDIwMMKgwqDCoMKgwqDCoMKgIDM1NCwx OTINCjY1OSw0NTbCoMKgwqDCoMKgwqDCoCB6aW9fYnVmXzI4NjcywqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAyOCw2NzLCoMKgwqDCoMKgwqDCoMKgIDYzMCw3ODQNCjc2MSw4NTbC oMKgwqDCoMKgwqDCoCA2NF9CdWNrZXTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgMjgzLDEzNsKgwqDCoMKgwqDCoMKgIDQ3OCw3MjANCjgxOSwyMDDCoMKgwqDCoMKg wqDCoCB6aW9fYnVmXzIwNDgwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyMCw0ODDC oMKgwqDCoMKgwqDCoMKgIDc5OCw3MjANCjg0NCwwMDDCoMKgwqDCoMKgwqDCoCA0X0J1Y2tl dMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDQ2Niw0OTbCoMKgwqDC oMKgwqDCoCAzNzcsNTA0DQo5MDYsNzUywqDCoMKgwqDCoMKgwqAgbWJ1Zl9wYWNrZXTCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA1MTLCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDkwNiwyNDANCjkxNyw1MDTCoMKgwqDCoMKgwqDCoCBsejRfY3R4wqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg OTE3LDUwNA0KOTM3LDk4NMKgwqDCoMKgwqDCoMKgIDEyOF9CdWNrZXTCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIDcxOSw4NzLCoMKgwqDCoMKgwqDCoCAyMTgsMTEyDQo5 NzIsMDk2wqDCoMKgwqDCoMKgwqAgemlvX2xpbmtfY2FjaGXCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAxNCw1OTLCoMKgwqDCoMKgwqDCoMKgIDk1Nyw1MDQNCjEsMTI4LDk2MMKgwqDC oMKgwqAgMzJfQnVja2V0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDYx MCwzMDTCoMKgwqDCoMKgwqDCoCA1MTgsNjU2DQoxLDEzMCw0OTbCoMKgwqDCoMKgIDgsMTky wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNzM3LDI4MMKg wqDCoMKgwqDCoMKgIDM5MywyMTYNCjEsMzc2LDI1NsKgwqDCoMKgwqAgemlvX2J1Zl81NzM0 NMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDEsMzc2LDI1Ng0KMSwzNzYsMjU2wqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfNTEywqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAyMSw1MDTCoMKgwqDCoMKgwqDCoMKgIDEsMzU0LDc1Mg0K MSw0MDAsODMywqDCoMKgwqDCoCB6aW9fYnVmXzI0NTc2wqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCA0OSwxNTLCoMKgwqDCoMKgwqDCoMKgIDEsMzUxLDY4MA0KMSw0MDksMDI0wqDC oMKgwqDCoCB6aW9fYnVmXzMyNzY4wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMSw0MDksMDI0DQoxLDQ3NCw1NjDCoMKgwqDCoMKg IHppb19idWZfODE5MjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCAxLDQ3NCw1NjANCjEsNTgwLDAwMMKgwqDCoMKgwqAgRkZTX2lu b2RlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoCAxLDU4MCwwMDANCjEsNjA0LDYwOMKgwqDCoMKgwqAgbWJ1ZsKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNTM1LDgwOMKgwqDC oMKgwqDCoMKgIDEsMDY4LDgwMA0KMSw2MjIsMDE2wqDCoMKgwqDCoCAxNiwzODTCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMSwwNDgsNTc2wqDCoMKgwqDC oCA1NzMsNDQwDQoxLDgwMiwyNDDCoMKgwqDCoMKgIHppb19kYXRhX2J1Zl8xNjM4NMKgwqDC oMKgwqDCoMKgwqDCoCAzMjcsNjgwwqDCoMKgwqDCoMKgwqAgMSw0NzQsNTYwDQoxLDg3MCww ODDCoMKgwqDCoMKgIEZGUzJfZGlub2RlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEsODcwLDA4MA0KMiwwNjQsMzg0wqDC oMKgwqDCoCB6aW9fZGF0YV9idWZfMjg2NzLCoMKgwqDCoMKgwqDCoMKgwqAgNTE2LDA5NsKg wqDCoMKgwqDCoMKgIDEsNTQ4LDI4OA0KMiwwOTcsMTUywqDCoMKgwqDCoCB6aW9fYnVmXzgx OTLCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOCwxOTLCoMKgwqDCoMKgwqDCoMKg wqAgMiwwODgsOTYwDQoyLDEyOSw5MjDCoMKgwqDCoMKgIHppb19idWZfNDA5NjDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEyMiw4ODDCoMKgwqDCoMKgwqDCoCAyLDAwNywwNDAN CjIsMjYwLDk5MsKgwqDCoMKgwqAgemlvX2J1Zl80OTE1MsKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDIsMjYwLDk5Mg0KMiw3Mjcs OTM2wqDCoMKgwqDCoCAyLDA0OMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDEsNjQ2LDU5MsKgwqDCoMKgwqAgMSwwODEsMzQ0DQoyLDg5NCw3NjDCoMKg wqDCoMKgIHVtdHhfc2htwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg Miw3NTgsODAwwqDCoMKgwqDCoCAxMzUsOTYwDQoyLDk0OSwxMjDCoMKgwqDCoMKgIHppb19i dWZfMTYzODQwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDIsOTQ5LDEyMA0KMiw5OTgsODAwwqDCoMKgwqDCoCBUSFJFQUTCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMiw3OTUsODAwwqDCoMKgwqDC oCAyMDMsMDAwDQozLDE0NSw3MjjCoMKgwqDCoMKgIHppb19idWZfMTk2NjA4wqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDMsMTQ1LDcy OA0KMywyNDQsMDMywqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfMjQ1NzbCoMKgwqDCoMKgwqDC oMKgwqAgNjg4LDEyOMKgwqDCoMKgwqDCoMKgIDIsNTU1LDkwNA0KMyw3MDIsNzg0wqDCoMKg wqDCoCB6aW9fZGF0YV9idWZfMzI3NjjCoMKgwqDCoMKgwqDCoMKgwqAgNDkxLDUyMMKgwqDC oMKgwqDCoMKgIDMsMjExLDI2NA0KMyw4OTksMzkywqDCoMKgwqDCoCB6aW9fYnVmXzIyOTM3 NsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCAzLDg5OSwzOTINCjMsOTMyLDE2MMKgwqDCoMKgwqAgemlvX2J1Zl85ODMwNMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDMsOTMy LDE2MA0KNCwxMjgsNzY4wqDCoMKgwqDCoCA2NSw1MzbCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgMiw2ODYsOTc2wqDCoMKgwqDCoCAxLDQ0MSw3OTINCjQs NDcyLDgzMsKgwqDCoMKgwqAgemlvX2J1Zl8xMTQ2ODjCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNCw0NzIsODMyDQo0LDc0NywyNjTC oMKgwqDCoMKgIHppb19idWZfNDA5NsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAy LDQ2OSw4ODjCoMKgwqDCoMKgIDIsMjc3LDM3Ng0KNCw4OTEsNDg4wqDCoMKgwqDCoCAxNsKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDQsMjUy LDM2OMKgwqDCoMKgwqAgNjM5LDEyMA0KNSwxNDQsNTc2wqDCoMKgwqDCoCAzMiw3NjjCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMyw1MDYsMTc2wqDCoMKg wqDCoCAxLDYzOCw0MDANCjUsMTQ4LDY3MsKgwqDCoMKgwqAgemlvX2RhdGFfYnVmXzEyMjg4 wqDCoMKgwqDCoMKgwqDCoMKgIDI3MCwzMzbCoMKgwqDCoMKgwqDCoCA0LDg3OCwzMzYNCjUs MjQyLDg4MMKgwqDCoMKgwqAgemlvX2J1Zl8yNjIxNDTCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNSwyNDIsODgwDQo1LDI0Miw4ODDC oMKgwqDCoMKgIHppb19idWZfMzI3NjgwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDUsMjQyLDg4MA0KNSwzMDQsMzIwwqDCoMKgwqDC oCB6aW9fZGF0YV9idWZfMjA0ODDCoMKgwqDCoMKgwqDCoMKgwqAgMzY4LDY0MMKgwqDCoMKg wqDCoMKgIDQsOTM1LDY4MA0KNSw0MDYsNzIwwqDCoMKgwqDCoCB6aW9fZGF0YV9idWZfNDA5 NjDCoMKgwqDCoMKgwqDCoMKgwqAgMSwwMjQsMDAwwqDCoMKgwqDCoCA0LDM4Miw3MjANCjUs ODE3LDA4OMKgwqDCoMKgwqAgYXJjX2J1Zl90wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDIsOTMyLDE2MMKgwqDCoMKgwqAgMiw4ODQsOTI4DQo1LDg1MywxODTCoMKg wqDCoMKgIDUxMsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAxLDUyOSwzNDTCoMKgwqDCoMKgIDQsMzIzLDg0MA0KNSw5MzUsMTA0wqDCoMKgwqDC oCAyNTZfQnVja2V0wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA1LDgzNiw4 MDDCoMKgwqDCoMKgIDk4LDMwNA0KNiwwMzksNTQ0wqDCoMKgwqDCoCB2bWVtX2J0YWfCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNCw5ODAsNTI4wqDCoMKgwqDCoCAx LDA1OSwwMTYNCjYsNTAyLDI3MsKgwqDCoMKgwqAgTF9WRlNfQ2FjaGXCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAyLDEwNiwwODjCoMKgwqDCoMKgIDQsMzk2LDE4NA0KNyw0 NzEsMTA0wqDCoMKgwqDCoCB6aW9fYnVmXzM5MzIxNsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA3LDQ3MSwxMDQNCjgsNTE5LDY4MMKg wqDCoMKgwqAgemlvX2J1Zl82NTUzNsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDgsNTE5LDY4MA0KOCw5ODgsMDAwwqDCoMKgwqDC oCAzMsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDgsMjc5LDQ1NsKgwqDCoMKgwqAgNzA4LDU0NA0KOSwxNzUsMDQwwqDCoMKgwqDCoCB6aW9f YnVmXzQ1ODc1MsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCA5LDE3NSwwNDANCjksNTM1LDQ4OMKgwqDCoMKgwqAgMSwwMjTCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA5LDE3NCwwMTbCoMKgwqDC oMKgIDM2MSw0NzINCjExLDM3NiwyODjCoMKgwqDCoCBCVUZfVFJJRcKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAx MSwzNzYsMjg4DQoxMSw2NDAsODMywqDCoMKgwqAgemlvX2RhdGFfYnVmXzU3MzQ0wqDCoMKg wqDCoMKgwqDCoMKgIDg2MCwxNjDCoMKgwqDCoMKgwqDCoCAxMCw3ODAsNjcyDQoxMSw2NTcs MjE2wqDCoMKgwqAgbWJ1Zl9jbHVzdGVywqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDExLDQ0NiwyNzLCoMKgwqDCoCAyMTAsOTQ0DQoxMSw3OTYsNDgwwqDCoMKgwqAgemlvX2J1 Zl82NTUzNjDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMTEsNzk2LDQ4MA0KMTMsMjcxLDA0MMKgwqDCoMKgIHppb19kYXRhX2J1Zl84 MTkyMMKgwqDCoMKgwqDCoMKgwqDCoCA3MzcsMjgwwqDCoMKgwqDCoMKgwqAgMTIsNTMzLDc2 MA0KMTQsMDI0LDcwNMKgwqDCoMKgIHppb19kYXRhX2J1Zl82NTUzNsKgwqDCoMKgwqDCoMKg wqDCoCA5MTcsNTA0wqDCoMKgwqDCoMKgwqAgMTMsMTA3LDIwMA0KMTcsMDM5LDM2MMKgwqDC oMKgIHppb19idWZfMTMxMDcyMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgMTcsMDM5LDM2MA0KMTcsMzAxLDUwNMKgwqDCoMKgIHppb19i dWZfNTI0Mjg4wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIDE3LDMwMSw1MDQNCjE4LDA4Nyw5MzbCoMKgwqDCoCB6aW9fZGF0YV9idWZf OTgzMDTCoMKgwqDCoMKgwqDCoMKgwqAgMSwyNzcsOTUywqDCoMKgwqDCoCAxNiw4MDksOTg0 DQoxOCwxNTMsNDU2wqDCoMKgwqAgemlvX2NhY2hlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIDM4OCw4MDjCoMKgwqDCoMKgwqDCoCAxNyw3NjQsNjQ4DQoyNCwxNDQs MTIwwqDCoMKgwqAgTUFQX0VOVFJZwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDE2LDEyMCwyMDDCoMKgwqDCoCA4LDAyMyw5MjANCjI2LDIxNCw0MDDCoMKgwqDCoCB6 aW9fYnVmXzEwNDg1NzbCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMiwwOTcsMTUywqDCoMKg wqDCoCAyNCwxMTcsMjQ4DQoyOSwzNzksMjQwwqDCoMKgwqAgcmFuZ2Vfc2VnX2NhY2hlwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIDIxLDMwMiw4NTbCoMKgwqDCoCA4LDA3NiwzODQNCjI5 LDc4MiwwODDCoMKgwqDCoCBSQURJWF9OT0RFwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAxNyw3NjEsMTA0wqDCoMKgwqAgMTIsMDIwLDk3Ng0KMzQsNTExLDQwMMKgwqDC oMKgIFNfVkZTX0NhY2hlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMzEsNTEy LDY3MsKgwqDCoMKgIDIsOTk4LDcyOA0KMzgsNTM1LDE2OMKgwqDCoMKgIHppb19idWZfNzg2 NDMywqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIDM4LDUzNSwxNjgNCjQwLDY4MCwxNDTCoMKgwqDCoCBzYV9jYWNoZcKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDQwLDU0OCw4MTbCoMKgwqDCoCAxMzEsMzI4 DQo0MSw1MTcsMDU2wqDCoMKgwqAgemlvX2RhdGFfYnVmXzExNDY4OMKgwqDCoMKgwqDCoMKg wqAgMSw0OTAsOTQ0wqDCoMKgwqDCoCA0MCwwMjYsMTEyDQo0MiwyMDUsMTg0wqDCoMKgwqAg emlvX2J1Zl85MTc1MDTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAwwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgNDIsMjA1LDE4NA0KNTAsMTQ3LDMyOMKgwqDCoMKgIHppb19kYXRh X2J1Zl80MDk2wqDCoMKgwqDCoMKgwqDCoMKgwqAgOTgsMzA0wqDCoMKgwqDCoMKgwqDCoCA1 MCwwNDksMDI0DQo1MCw2NzUsNzEywqDCoMKgwqAgemlvX2RhdGFfYnVmXzQ5MTUywqDCoMKg wqDCoMKgwqDCoMKgIDk4MywwNDDCoMKgwqDCoMKgwqDCoCA0OSw2OTIsNjcyDQo1Myw5NzIs NzM2wqDCoMKgwqAgNjTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAyOSw4NzcsODg4wqDCoMKgwqAgMjQsMDk0LDg0OA0KNjEsMzQxLDY5NsKg wqDCoMKgIHppb19idWZfMTMxMDcywqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNDIsMjA1 LDE4NMKgwqDCoMKgIDE5LDEzNiw1MTINCjcyLDAxOSwyMDDCoMKgwqDCoCBWTV9PQkpFQ1TC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgNzEsNTk3LDA1NsKgwqDCoMKg IDQyMiwxNDQNCjc2LDczMSwyMDDCoMKgwqDCoCB6ZnNfem5vZGVfY2FjaGXCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgNzYsNTkyLDIwOMKgwqDCoMKgIDEzOCw5OTINCjg4LDk3Miw4MDDC oMKgwqDCoCAyNTbCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgODIsOTI1LDU2OMKgwqDCoMKgIDYsMDQ3LDIzMg0KOTAsMzkwLDUyOMKgwqDCoMKg IDQsMDk2wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgODks OTExLDI5NsKgwqDCoMKgIDQ3OSwyMzINCjk0LDAzNiwwMDDCoMKgwqDCoCBVTUFfU2xhYnPC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOTQsMDMzLDI4MMKgwqDCoMKg IDIsNzIwDQoxMzUsNDU2LDAwMMKgwqDCoCBWTk9ERcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIDEzNSwyNzMsNjAwwqDCoMKgIDE4Miw0MDANCjE3MSw5 MjgsMzIwwqDCoMKgIGFyY19idWZfaGRyX3RfZnVsbMKgwqDCoMKgwqDCoMKgwqDCoCAxMTks NzM3LDM0NMKgwqDCoCA1MiwxOTAsOTc2DQoyMjEsOTcwLDQzMsKgwqDCoCB6aW9fZGF0YV9i dWZfODE5MsKgwqDCoMKgwqDCoMKgwqDCoMKgIDE2NiwwNzYsNDE2wqDCoMKgIDU1LDg5NCww MTYNCjI3Nyw5MjMsODQwwqDCoMKgIGRtdV9idWZfaW1wbF90wqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMTMwLDc4OCwyNDDCoMKgwqAgMTQ3LDEzNSw2MDANCjM3Niw1ODYsMjQwwqDC oMKgIHppb19idWZfMTYzODTCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDM3Miw3MTks NjE2wqDCoMKgIDMsODY2LDYyNA0KMzk3LDY4MCw4OTbCoMKgwqAgMTI4wqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDE5NSw5NDQsNDQ4wqDCoMKg IDIwMSw3MzYsNDQ4DQo0NDMsMDIzLDM2MMKgwqDCoCB6aW9fZGF0YV9idWZfMTMxMDcywqDC oMKgwqDCoMKgwqDCoCAxNTgsMDcyLDgzMsKgwqDCoCAyODQsOTUwLDUyOA0KNTM1LDU4NCw3 NjjCoMKgwqAgemlvX2J1Zl81MTLCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAy NTUsNjQxLDA4OMKgwqDCoCAyNzksOTQzLDY4MA0KNzEzLDU1Miw4NDDCoMKgwqAgZG5vZGVf dMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMzczLDc1Niw2NTbC oMKgwqAgMzM5LDc5NiwxODQNCjMsODQ5LDc0NCwzODTCoCBhYmRfY2h1bmvCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgMyw4NDgsMzEwLDc4NMKgIDEsNDMzLDYwMA0K OCw0MTgsNzY5LDg0OMKgIFRPVEFMwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgNiw0NTksNTA3LDkzNsKgIDEsOTU5LDI2MSw5MTINCg0KU28gZmFyIHJ1 bm5pbmcgMTItU1RBQkxFICJuZWF0IiBpcyBiZWhhdmluZyB3ZWxsIGZvciBtZS4uLi4NCg0K LS0gDQpLYXJsIERlbm5pbmdlcg0Ka2FybEBkZW5uaW5nZXIubmV0IDxtYWlsdG86a2FybEBk ZW5uaW5nZXIubmV0Pg0KL1RoZSBNYXJrZXQgVGlja2VyLw0KL1tTL01JTUUgZW5jcnlwdGVk IGVtYWlsIHByZWZlcnJlZF0vDQo= --------------ms030509060104090502030302 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEzMTMyODAy WjBPBgkqhkiG9w0BCQQxQgRA05E84bhtdM3QtqcvmcKvTNNWK+/IcFwmQkGUwMiTRXugN0yw oKaOYdMht6PZ5wiI2ijJsoH7vOzoWdcEPk++KzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgAKnZozQ9gVfjPndvz5Er5pj8TykKZoKRN3+rtMSmiv8Fy1pjeR6dMvv2IATe2c9ulh 86jE+HrZouvAjQa8SUtELyVyYi22bsLO2VD/GfR0lh1MCcDLbUmPSf/0yqg+/KMXdOCEmFle Z8WA9lzxAhHFpCO0c/PBF4wusw9gbvKlZEzWL0BQ4iXtEZk02teg1gpgHXB3Q0VjpzkZWjx4 1GNln5LK18O4Qh5nirJy1Jj5MUs/jnmKv0s04C9HySsTlwXb91s/J2VKiVVENXXZoURXX3/n 2nGVRSF48tnDWwW+n3Teiy/gAOHBOnxrLdw0dQiZcQ+r7JTRK5u29wu1ChxXsB12tx/vR7p5 A89WYhZXx/zptB1UyyGv38iTVeUWCAel1foevXWpG/qwf6qYlyNgMCiqEw+wV+LgO84q38Z0 XHIuZhlXWZYUIAf5ASsf4xGXz6yRxHsLv1ikq26g5tVIzn8zzyLdo6xwIgjsknfRRPPk4PK+ YwDdAlpxGHaX/7wwXMu1bVRwgZdvOI0o4FJJY8UTI+d+tS63ZZlS1zr+1o8ncC34i64IryGd ERz4bFGxtcD0nuL1n/otk3ldMKjidAFS69LBZ+rdWo9NnOsJ9ZnYHy5pilHLzeHRWiGPlAF6 SxAJ6WHHVoH8apG3yDgd+LjG4sRa4zda+ZhW1RqYJQAAAAAAAA== --------------ms030509060104090502030302-- From owner-freebsd-stable@freebsd.org Wed Feb 13 13:41:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2DDD14EF986 for ; Wed, 13 Feb 2019 13:41:43 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37A638A3D9 for ; Wed, 13 Feb 2019 13:41:43 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id D1FC2211098 for ; Wed, 13 Feb 2019 08:41:36 -0500 (EST) Received: from [192.168.10.19] (D9.Denninger.Net [192.168.10.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 6B1C4C49B7 for ; Wed, 13 Feb 2019 07:41:36 -0600 (CST) To: freebsd-stable@freebsd.org From: Karl Denninger Subject: Binary update to -STABLE? And if so, what do I get? Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> Date: Wed, 13 Feb 2019 07:41:37 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020706040803080401030104" X-Rspamd-Queue-Id: 37A638A3D9 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.51 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[px.denninger.net]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.34)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.41), asn: 14061(2.69), country: US(-0.07)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:+]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 13:41:44 -0000 This is a cryptographically signed message in MIME format. --------------ms020706040803080401030104 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I know (and have done) binary updates between -RELEASE versions But 12 has a problem with -RELEASE and IPv6, which was recently fixed and MFC'd.=C2=A0 So now I have an interesting situation in that I have tw= o machines in the field running 11.2 that do things for me at one of the "shared colo" joints, and I would like to roll them forward -- but they have to roll forward to a reasonably-recent -STABLE. How do I do this, say, coming from 11.2 and wanting to target 12 post the IPv6 fix MFC?=C2=A0 (e.g. how do I specify the target, since it would= n't be "12-RELEASE"?)=C2=A0 I'm assuming (perhaps incorrectly) that "12-STABL= E" is not the correct means to do so. Or is it, since at first blush it doesn't blow up if I use that... but I'm hesitant to say "yeah, go at it." # freebsd-update -r 12-STABLE upgrade src component not installed, skipped Looking up update.FreeBSD.org mirrors... 3 mirrors found. Fetching metadata signature for 11.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic world/base world/lib32 The following components of FreeBSD do not seem to be installed: kernel/generic-dbg world/base-dbg world/doc world/lib32-dbg Does this look reasonable (y/n)? --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020706040803080401030104 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEzMTM0MTM3 WjBPBgkqhkiG9w0BCQQxQgRA0rQTpWcvDVlIQpNAmN49QV/Y4aFKp0ez/9pKz+zP0BPNbisA 13XGB8Zj5rxTBHhG18oUsDgNztH1I/nAhmGADDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCnSNRARSAruheVSxHCt7ToVQkTutke0suvrOPi3zmwRLnxoYPDoKVdjsvw3FFxoMW8 pRgenlUmZDj4VaqtABHiAXKTOVGrtHEqMUsyM0+LZ5pk+GwWCPKFDL5AYiOkbORaWqJr0tJa yETcQ8d2GlPHgzJxbMxkDGbEkElhSPrWb7tlRMHl0hOwf+/hXoVBD5kdroiD0UWSRDS/VjG+ AJzyRubzg+MzLBMvzqN/5caT0Vunk7Brb7QfZzsD1DfSMa1VVYZd3W/84XGfos37kcFvq6GP RakdtzZ/F9e9tncT880Ux7Nr9THtX7zSUTvF5EknfQi7m7Wi/KhVv6+uEdWqOQS8FRkxSa3V bLPReHuSDgGndLUXcmIog2N8soYdDmeMacD4sFA+BLIzLf1CfvjSccrof+GRMVVY8m4SoKzd sucl9v353t/r6CXfQfQLPJl4vd4ODbbQRNLWtN3d3AfkXBVqu6DqABGbmUxwZ7tTeir+7EVd 0+cdXYjZoLAg6ere1hI56B39vjUkf32x11gfn4xQ+vgd99swQVlJ5z1ERDRqgvAD2PFaec0D 4o0PCvlQbd05zca4fjcpdOjobcDgJhTN89soiLdPTrUeKKJ20hPxjMu8ZPpypVuahKVJg5ZK vaDQXd5HmI9aFrx++sL5TdZYb0nhBr/c2qOB4+DJtgAAAAAAAA== --------------ms020706040803080401030104-- From owner-freebsd-stable@freebsd.org Wed Feb 13 13:49:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21B6414EFCE3 for ; Wed, 13 Feb 2019 13:49:39 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A60BC8A6EE for ; Wed, 13 Feb 2019 13:49:38 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from pi by home.opsec.eu with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1gtuux-0002GV-Mr; Wed, 13 Feb 2019 14:49:39 +0100 Date: Wed, 13 Feb 2019 14:49:39 +0100 From: Kurt Jaeger To: Karl Denninger Cc: freebsd-stable@freebsd.org Subject: Re: Binary update to -STABLE? And if so, what do I get? Message-ID: <20190213134939.GO2748@home.opsec.eu> References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 13:49:39 -0000 Hi! > I know (and have done) binary updates between -RELEASE versions [...] > How do I do this, say, coming from 11.2 and wanting to target 12 post > the IPv6 fix MFC? You can't. Either wait until a 12.0 with the fix included or 12.1 is released, or you fetch the source with the fix included, and build from source. -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-stable@freebsd.org Wed Feb 13 13:53:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B00714F0002 for ; Wed, 13 Feb 2019 13:53:48 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC4938AC59 for ; Wed, 13 Feb 2019 13:53:47 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id B5F4B211098 for ; Wed, 13 Feb 2019 08:53:46 -0500 (EST) Received: from [192.168.10.19] (D9.Denninger.Net [192.168.10.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 4B0ACC4A36 for ; Wed, 13 Feb 2019 07:53:46 -0600 (CST) Subject: Re: Binary update to -STABLE? And if so, what do I get? To: freebsd-stable@freebsd.org References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> Date: Wed, 13 Feb 2019 07:53:46 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <20190213134939.GO2748@home.opsec.eu> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms080508030405090401020601" X-Rspamd-Queue-Id: CC4938AC59 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.993,0] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2019 13:53:48 -0000 This is a cryptographically signed message in MIME format. --------------ms080508030405090401020601 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2/13/2019 07:49, Kurt Jaeger wrote: > Hi! > >> I know (and have done) binary updates between -RELEASE versions > [...] >> How do I do this, say, coming from 11.2 and wanting to target 12 post >> the IPv6 fix MFC? > You can't. Either wait until a 12.0 with the fix included or > 12.1 is released, or you fetch the source with the fix included, > and build from source. Got it -- thanks.=C2=A0 Wait it shall be. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms080508030405090401020601 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMjEzMTM1MzQ2 WjBPBgkqhkiG9w0BCQQxQgRAb24KP8tyrYd53JuxRRkF1ZekgsY1bixgsHS71nt/ofIown8u +lrMIsuAGg2/XEgwTD9SMizHjBoNvcE2AkFdQDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBGjx/jtrOwT3QPbV+95RhPE4BFoiy2jgLO8hXmHdJscHObB224khAEfm7xJA8RHsPJ boqK4pGFn7KvNwTYql1jj40R8VZlNMSCNy3XUDxFZ7snX6mdgqB+DmmA1SpZRTgTpNwZgS7T V5/zCoUxa0hG+6gRpKSAG5q7BPeyuYm1M5NZ79DvszMUh9NMr0h5DW7X6riZiwrF2bZo39z+ 33LmzKf5Zy+2Zk9d+UwfTsicJAhzOVG9c9oi5Y2c18i178syMEIMBC/6a/OHqzYz9uR0c/Ms /SVbrJlBM7kKAEOFpQUK1E/ScQ8c6ZH2X2E0DOAROM1MwpzQ34gfqdxMezBUbMs6p3sB8QhD dPU0XsRYhXW0GZ+p6mbNF4eKeTBx/NprvMoWqxW/SDLKEh7tkMCpSHIzwJmapW8rcpBEl9Ci 7FcczbomGuFRlTzRMxLJU5g+aKqOaOzm7Mhv3XMPEPs3foHeaZuyqpeSGA+3ZuzDZfoL46Ux 8nBrNLFC9nDJTsiH55SlA0x97V99Uv3wAqMVdSZDK831EHd6Ll216VgxwW4R8N44IuGo3xiR 89nB4BugDgr9ehFOScO2rxDcvUrYhkRMEHVHYzt02RldEPCWfn0Fkji0fg2f+a0MR/5JQETQ qp18ZFdMJOKpiP5MktygRow97csoLKFjkV4nGksDSAAAAAAAAA== --------------ms080508030405090401020601-- From owner-freebsd-stable@freebsd.org Thu Feb 14 02:07:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AE2A14E1DD0 for ; Thu, 14 Feb 2019 02:07:36 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5230D8886A for ; Thu, 14 Feb 2019 02:07:35 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-ot1-f54.google.com with SMTP id m1so7959288otf.5 for ; Wed, 13 Feb 2019 18:07:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8tDxnHoV9Kq/yya63u4ckWIylLz9TGlAG/TxOCUvJD4=; b=G33jqvyJS1VoICVA71m0/WrGyip/GGWH4FFc1KvDpW2HHlts256M1ps5IBq7EN3+an udVzXejwQTB8tS45PXW8W/xMX5W0XCbNmldh3um1Qs+O4WOxwzSjE0/VppR6qZ/p4nwO uPIy1F03bpsSKoxritcyBrHA94Q+cnEf1qQc81DQJtDFez3YxOwP1gU6mBlk6KDEdXx5 T4mdGm7TpHk65qPkGqHsqH3omcDeyRR9H8KkBBwjP4uY1GpZ+QFbfZxMQ4mLwc7A/BSU 54hVxtgWQ7HtepnQ6lV1J6XKA8265vVIfStH4Vk/j5t5z3HgWEe/TJ9GVz+BjpRKVr5/ DSzQ== X-Gm-Message-State: AHQUAuZ1TRDyaEPU2mcQ9xqHvnQBUr8wA99DChFW6SFRJlGXx6TYtcbG FSBniYnbjQTQf3I5RKSXB8ZrBB1FkCOHhMFI9vT1kg== X-Google-Smtp-Source: AHgI3IbPJQd9SbcBJuuSshABzx52N/li8l7fUqS21b9cpQ12XDXDobPBnFda+/v26j75qITvyGoJAPXXF+jFjpiyBD8= X-Received: by 2002:a9d:6d04:: with SMTP id o4mr676746otp.243.1550108606553; Wed, 13 Feb 2019 17:43:26 -0800 (PST) MIME-Version: 1.0 References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> In-Reply-To: <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> From: Jason Tubnor Date: Thu, 14 Feb 2019 12:43:14 +1100 Message-ID: Subject: Re: Binary update to -STABLE? And if so, what do I get? To: Karl Denninger Cc: freebsd-stable X-Rspamd-Queue-Id: 5230D8886A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.210.54 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-2.96 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; IP_SCORE(-1.22)[ip: (-0.29), ipnet: 209.85.128.0/17(-3.77), asn: 15169(-1.98), country: US(-0.07)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_TRACE(0.00)[0:+,1:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.210.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.73)[-0.735,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.210.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 02:07:36 -0000 On Thu, 14 Feb 2019 at 00:53, Karl Denninger wrote: > > > You can't. Either wait until a 12.0 with the fix included or > > 12.1 is released, or you fetch the source with the fix included, > > and build from source. > Got it -- thanks. Wait it shall be. > > > I also have hit this IPv6 issue (I thought I was going crazy until I worked it out) and other iflib issues in 12.0, which have been fixed in -STABLE that really should be patched in 12.0 or bring forward an early 12.1 release. For our use case, 12.0 is just too buggy for production at this rate and we won't touch it, which is a shame because there is a lot of good work in there that we would like to use but it is trumped by the breakages. From owner-freebsd-stable@freebsd.org Thu Feb 14 06:22:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D04914EB3AF for ; Thu, 14 Feb 2019 06:22:24 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 83B8191131; Thu, 14 Feb 2019 06:22:13 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x1E6M0nv088735 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Feb 2019 07:22:01 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: markj@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id x1E6Lxiv073128; Thu, 14 Feb 2019 13:21:59 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 11.2-STABLE kernel wired memory leak To: Mark Johnston References: <20190212163446.GA29847@raichu> <9a263536-45e3-c690-e45e-d8ece7d1f388@grosbein.net> Cc: FreeBSD stable From: Eugene Grosbein X-Enigmail-Draft-Status: N1110 Message-ID: <5C650907.7040703@grosbein.net> Date: Thu, 14 Feb 2019 13:21:59 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <9a263536-45e3-c690-e45e-d8ece7d1f388@grosbein.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00, LOCAL_FROM, SPF_PASS, T_DATE_IN_FUTURE_96_Q autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 T_DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after * Received: date * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 83B8191131 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.75 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; URIBL_BLOCKED(0.00)[grosbein.net.multi.uribl.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[grosbein.net]; MX_INVALID(0.50)[greylisted]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.36)[ip: (-2.15), ipnet: 2a01:4f8::/29(-2.40), asn: 24940(-2.24), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.79)[-0.794,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 06:22:24 -0000 On 13.02.2019 19:04, Eugene Grosbein wrote: > 12.02.2019 23:34, Mark Johnston wrote: > >> On Tue, Feb 12, 2019 at 11:14:31PM +0700, Eugene Grosbein wrote: >>> Hi! >>> >>> Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel wired memory over 81 days uptime >>> out of 8GB total RAM. >>> >>> Details follow. >>> >>> I have a workstation running Xorg, Firefox, Thunderbird, LibreOffice and occasionally VirtualBox for single VM. >>> >>> It has two identical 320GB HDDs combined with single graid-based array with "Intel" >>> on-disk format having 3 volumes: >>> - one "RAID1" volume /dev/raid/r0 occupies first 10GB or each HDD; >>> - two "SINGLE" volumes /dev/raid/r1 and /dev/raid/r2 that utilize "tails" of HDDs (310GB each). >>> >>> /dev/raid/r0 (10GB) has MBR partitioning and two slices: >>> - /dev/raid/r0s1 (8GB) is used for swap; >>> - /dev/raid/r0s2 (2GB) is used by non-redundant ZFS pool named "os" that contains only >>> root file system (177M used) and /usr file system (340M used). >>> >>> There is also second pool (ZMIRROR) named "z" built directly on top of /dev/raid/r[12] volumes, >>> this pool contains all other file systems including /var, /home, /usr/ports, /usr/local, /usr/{src|obj} etc. >>> >>> # zpool list >>> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT >>> os 1,98G 520M 1,48G - - 55% 25% 1.00x ONLINE - >>> z 288G 79,5G 209G - - 34% 27% 1.00x ONLINE - >>> >>> This way I have swap outside of ZFS, boot blocks and partitioning mirrored by means of GEOM_RAID and >>> can use local console to break to single user mode to unmount all file system other than root and /usr >>> and can even export bigger ZFS pool "z". And I did that to see that ARC usage >>> (limited with vfs.zfs.arc_max="3G" in /boot/loader.conf) dropped from over 2500MB >>> down to 44MB but "Wired" stays high. Now after I imported "z" back and booted to multiuser mode >>> top(1) shows: >>> >>> last pid: 51242; load averages: 0.24, 0.16, 0.13 up 81+02:38:38 22:59:18 >>> 104 processes: 1 running, 103 sleeping >>> CPU: 0.0% user, 0.0% nice, 0.4% system, 0.2% interrupt, 99.4% idle >>> Mem: 84M Active, 550M Inact, 4K Laundry, 4689M Wired, 2595M Free >>> ARC: 273M Total, 86M MFU, 172M MRU, 64K Anon, 1817K Header, 12M Other >>> 117M Compressed, 333M Uncompressed, 2.83:1 Ratio >>> Swap: 8192M Total, 940K Used, 8191M Free >>> >>> I have KDB and DDB in my custom kernel also. How do I debug the leak further? >>> >>> I use nvidia-driver-340-340.107 driver for GK208 [GeForce GT 710B] video card. >>> Here are outputs of "vmstat -m": http://www.grosbein.net/freebsd/leak/vmstat-m.txt >>> and "vmstat -z": http://www.grosbein.net/freebsd/leak/vmstat-z.txt >> >> I suspect that the "leaked" memory is simply being used to cache UMA >> items. Note that the values in the FREE column of vmstat -z output are >> quite large. The cached items are reclaimed only when the page daemon >> wakes up to reclaim memory; if there are no memory shortages, large >> amounts of memory may accumulate in UMA caches. In this case, the sum >> of the product of columns 2 and 5 gives a total of roughly 4GB cached. > > After another day with mostly idle system, "Wired" increased to more than 6GB out of 8GB total. > I've tried to increase sysctl vm.v_free_min from default 12838 (50MB) > upto 1048576 (4GB) and "Wired" dropped a bit but it is still huge, 5060M: > > last pid: 61619; load averages: 1.05, 0.78, 0.40 up 81+22:33:09 18:53:49 > 119 processes: 1 running, 118 sleeping > CPU: 0.0% user, 0.0% nice, 50.0% system, 0.0% interrupt, 50.0% idle > Mem: 47M Active, 731M Inact, 4K Laundry, 5060M Wired, 2080M Free > ARC: 3049M Total, 216M MFU, 2428M MRU, 64K Anon, 80M Header, 325M Other > 2341M Compressed, 5874M Uncompressed, 2.51:1 Ratio > Swap: 8192M Total, 940K Used, 8191M Free > > # sysctl vm.v_free_min > vm.v_free_min: 1048576 > # sysctl vm.stats.vm.v_free_count > vm.stats.vm.v_free_count: 533232 > > ARC probably cached results of nightly periodic jobs traversing file system trees > and hit its limit (3G). > > Still cannot understand where have another 2G (5G-3G) of wired memory leaked to? After several more hours of pretty idle time with quoted settings memory finally got moved from "Wired" " to "Free" category. I got the system to single user mode again, exported bigger ZFS pool again and ARC dropped down to 18M and Wired to less than 500M. After re-import and boot to multiuser with Xorg running, Wired is less than 800M. So, there seems to be no leak but insanely long time for Wired memory to be freed, even with extreme value for vm.v_free_min. Even heavy memory pressure does not speed things up. It seems, page daemon is somewhat broken and fails to free wired memory timely. From owner-freebsd-stable@freebsd.org Thu Feb 14 11:08:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAE9F14F303E for ; Thu, 14 Feb 2019 11:08:46 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AFB56BC0F for ; Thu, 14 Feb 2019 11:08:44 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6]) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1guEsj-000NxR-VP for freebsd-stable@freebsd.org; Thu, 14 Feb 2019 11:08:41 +0000 Subject: Re: Binary update to -STABLE? And if so, what do I get? To: freebsd-stable@freebsd.org References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> From: Pete French Message-ID: <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> Date: Thu, 14 Feb 2019 11:08:41 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0AFB56BC0F X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 2a02:b90:3002:411::3 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [4.35 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:b90:3002:411::3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RDNS_NONE(1.00)[]; NEURAL_SPAM_SHORT(0.11)[0.106,0]; MX_GOOD(-0.01)[us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; NEURAL_SPAM_LONG(0.86)[0.864,0]; NEURAL_HAM_MEDIUM(-0.29)[-0.291,0]; IP_SCORE(-0.02)[country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:16082, ipnet:2a02:b90::/32, country:GB]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 11:08:47 -0000 On 14/02/2019 01:43, Jason Tubnor wrote: > I also have hit this IPv6 issue (I thought I was going crazy until I worked > it out) and other iflib issues in 12.0, which have been fixed in -STABLE > that really should be patched in 12.0 or bring forward an early 12.1 > release. For our use case, 12.0 is just too buggy for production at this > rate and we won't touch it, which is a shame because there is a lot of good > work in there that we would like to use but it is trumped by the breakages. Any reason behind not running STBLE out of interest ? Yes, 12 has been buggy with regards to networking, but these things get fixed very fast and I now have all my machines on the lattest STABLE in production, as of yesterday. -pete. From owner-freebsd-stable@freebsd.org Thu Feb 14 10:20:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D0C314F1C16 for ; Thu, 14 Feb 2019 10:20:15 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: from mail-yb1-f194.google.com (mail-yb1-f194.google.com [209.85.219.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96AA86A30A for ; Thu, 14 Feb 2019 10:20:14 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: by mail-yb1-f194.google.com with SMTP id e131so2151821ybh.12 for ; Thu, 14 Feb 2019 02:20:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=fJuXN1JdSLXVzVoSL9vsjeDe2em7NbTEsA1ELPG2MbQ=; b=Vo+H4mIwPhljLcX8ypeKlMxurEhKmu3KXNC30JoP+4wRldIBXWlSecyFeTAUR3VGOt XWNvR7L0uF2AnkHP8pM86tA/W3MfFwqtML4h5TVdJNUSVTOD1iYzal1iBDJn14z7Seg6 u76dElCsZl8e2uAlNq187sRRZaVKEvZPtipvTqFSxgTHBO/q/eav7BeTLAP/hnvoB4w/ AgE1yHtDPIarEvIYDEwUvpLgQfF1+8Qycl+StKfaWEp9Tiu6X3rk0E39R2R9DQo2VFRZ OEHn3w6sdjM8E/a5Ixmt36TDdCKK2cA0Mfw3cvKbugRrUGuRX/nrs7dFDTvBPAVUxV8d p3vw== X-Gm-Message-State: AHQUAuZsIXoNLK+DvYTpTAtofFmnA7SUE1VtaikoJjcXoQbU+WfCs2io cRaFdc90cgfkuglvwrhoJxjjXpKrM+2Ofk0u+KA= X-Google-Smtp-Source: AHgI3IbWHfCrjB1fLVz591R87fu5/fV1kwRnc2XOADC7bqsEGo1OPfoNz6wcwFKHti1jGDksr+wTSTK/flAqqcyzjZI= X-Received: by 2002:a25:6d05:: with SMTP id i5mr2150466ybc.489.1550138072896; Thu, 14 Feb 2019 01:54:32 -0800 (PST) MIME-Version: 1.0 From: Li-Wen Hsu Date: Thu, 14 Feb 2019 17:54:15 +0800 Message-ID: Subject: FreeBSD CI Weekly Report 2019-02-10 To: freebsd-testing@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 96AA86A30A X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of lwhsufreebsd@gmail.com designates 209.85.219.194 as permitted sender) smtp.mailfrom=lwhsufreebsd@gmail.com X-Spamd-Result: default: False [-3.73 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_NONE(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.55)[-0.552,0]; FORGED_SENDER(0.30)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FORGED_RECIPIENTS(0.00)[freebsd-testing@freebsd.org,freebsd-stable@freebsd.org]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_NEQ_ENVFROM(0.00)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; URIBL_BLOCKED(0.00)[hackmd.io.multi.uribl.com,hackfoldr.org.multi.uribl.com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[194.219.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.17)[ipnet: 209.85.128.0/17(-3.77), asn: 15169(-1.98), country: US(-0.07)]; RWL_MAILSPIKE_POSSIBLE(0.00)[194.219.85.209.rep.mailspike.net : 127.0.0.17]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Thu, 14 Feb 2019 11:24:11 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 10:20:15 -0000 (bcc -current and -stable for more audience) FreeBSD CI Weekly Report 2019-02-10 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-02-04 to 2019-02-10. During this period, we have: * 2564 builds (97.4% passed, 2.6% failed) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 546 test runs (21.2% passed, 63.4% unstable, 15.4% exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 12 doc buils (100% passed) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. Web version of this report is available at https://hackmd.io/s/ryA8Crr4N and archive is available at http://hackfoldr.org/freebsd-ci-report/, any help is welcome. ## Fixed Jobs * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test r343713 enabled COVERAGE and KCOV by default causes the test VM exits while executing dtrace test cases, it then got disabled by r343949. * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc Error message: `x86_64-unknown-freebsd12.0-gcc: error: unrecognized command line option '-fsanitize-coverage=trace-pc,trace-cmp'; did you mean '-fsanitize-coverage=trace-pc'?` Fixed in : https://svnweb.freebsd.org/changeset/base/343746 ## Fixed Tests * lib.msun.{cbrt_test.cbrtl_powl,trig_test.reduction} * https://bugs.freebsd.org/234040 * https://svnweb.freebsd.org/changeset/base/343916 * https://svnweb.freebsd.org/changeset/base/343917 ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ * lib.libc.sys.sendfile_test.hdtr_positive_v4 * lib.libc.sys.sendfile_test.hdtr_positive_v6 See https://bugs.freebsd.org/235200 and https://bugs.freebsd.org/234809 for deails. WIP: https://bugs.freebsd.org/234809 * lib.libc.regex.exhaust_test.regcomp_too_big * lib.libregex.exhaust_test.regcomp_too_big These two began failing since r343964, further examination is needed. * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * There are 60 (-1 since last report) failing cases, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ * sys.netmap.ctrl-api-test.main * sys.opencrypto.runtests.main * lib.libc.regex.exhaust_test.regcomp_too_big * lib.libregex.exhaust_test.regcomp_too_big * sys.kern.coredump_phnum_test.coredump_phnum WIP: https://reviews.freebsd.org/D18495 * lib.libc.sys.sendfile_test.hdtr_positive_v4 * lib.libc.sys.sendfile_test.hdtr_positive_v6 see https://bugs.freebsd.org/235200 and https://bugs.freebsd.org/234809 for deails. WIP: https://bugs.freebsd.org/234809 * https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/ * sbin.bectl.bectl_test.bectl_mount * sys.netmap.ctrl-api-test.main * sys.opencrypto.runtests.main * lib.libc.regex.exhaust_test.regcomp_too_big * lib.libregex.exhaust_test.regcomp_too_big * sys.kern.coredump_phnum_test.coredump_phnum WIP: https://reviews.freebsd.org/D18495 * https://ci.freebsd.org/job/FreeBSD-stable-11-amd64-test/ * usr.bin.procstat.procstat_test.kernel_stacks * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * sys.netmap.ctrl-api-test.main * sys.opencrypto.runtests.main * usr.bin.procstat.procstat_test.kernel_stacks * local.kyua.* (31 cases) * local.lutok.* (3 cases) ## Disabled Tests * lib.libc.sys.mmap_test.mmap_truncate_signal https://bugs.freebsd.org/211924 * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * usr.bin.procstat.procstat_test.command_line_arguments https://bugs.freebsd.org/233587 * usr.bin.procstat.procstat_test.environment https://bugs.freebsd.org/233588 ## Open Issues ### Cause build fails * [233339: genassym.o build race](https://bugs.freebsd.org/233339) * (Updated) patch available: https://people.freebsd.org/~bdrewery/patches/PR233339.diff * [233735: Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory](https://bugs.freebsd.org/233735) * [233769: Possible build race: ld: error: unable to find library -lgcc_s](https://bugs.freebsd.org/233769) ### Others [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) ## Other News * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/1131/ zfs test became very slow (~100 mins -> ~200 mins) after enabling COVERAGE and KCOV by default (then disabled in r343949) From owner-freebsd-stable@freebsd.org Thu Feb 14 11:30:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC6BB14F3CC9 for ; Thu, 14 Feb 2019 11:30:35 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69D2A6C9E6; Thu, 14 Feb 2019 11:30:35 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-402.fritz.box (p5DC83C19.dip0.t-ipconnect.de [93.200.60.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "st_esser@t-online.de", Issuer "WISeKey CertifyID Standard Services CA 2" (verified OK)) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 9DDBD4A90; Thu, 14 Feb 2019 11:30:34 +0000 (UTC) (envelope-from se@freebsd.org) Subject: Re: 11.2-STABLE kernel wired memory leak To: Andriy Gapon , freebsd-stable@freebsd.org References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <0710bc0a-7691-b29f-3331-da94412ab737@FreeBSD.org> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= mQENBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAG0J1N0ZWZhbiBFw59lciAoRnJlZUJTRCkgPHNlQGZyZWVic2Qub3JnPokBVAQTAQoAPgIb AwULCQgHAwUVCgkICwUWAwIBAAIeAQIXgBYhBKNx6mWcC+zIK3FTE0frte9a/fVEBQJa8u+q BQkLJQETAAoJEEfrte9a/fVEOeMH/icmdK1eZQvB3U8quJo9VMaZsaTuCMbUE4NThyfsIvIm MCd+rb/yULmMYwqNfjyKB1x4ikR4x+94l+yJoz7K0Usks+eNKDmMGJM6pWWssTigaJubFdVd hVVC+C1QJi7JshYSib08uONoPmO4lv5Az0TDYGtsMzsES2sIlc62c9go5WPGYhQFRbX3Lk6y V6m8OHh+G9XGSj3oPO4UteRwu+SzTdOLunZBWG1wu34+IeZm663D+2gOppQLWpLa2qaTerqw THu377ayZ2B2LPJ5JkvkZeHYPkwDQ+b5PGn0UhfkxPnDVYki5F7qKxvQ5uq1/q9YaCX7mmOl H2yO7tgVsrW5AQ0EVXGJEgEIALEj9qCXMZVucjpcd3QxM/TlUr98m5viEd1z4tCnPUyRWcIC EVtj2h5xMH+2iB0q1+KWhq+NsWtvScmEmfHnsr7dJ1K677OdpDhKVaJk61eeRulFY1R4yb6C 1MMxK+WgYB+vvpG0UeyR0M4uBewcPvRsq4yGUHFQKtLAbMdoPTSryJA+ElnmK1vdY+rPcHgi OIMBZM7ahsPXC0C9K4e5SP9clGyIoMpbfHXdx9q+Rp3zVtlbhyk3BS/xccu/+9pk9ICXL6GR js2sNnJ0wxdU1DsAlC59a5MnSruwiZFwRnkQhr3x6wk97Lg7sLS9jjTnCN7LGlVmSmpOEMy6 uq1AWfUAEQEAAYkBPAQYAQoAJgIbDBYhBKNx6mWcC+zIK3FTE0frte9a/fVEBQJa8u+rBQkL JQEZAAoJEEfrte9a/fVEuesH/2DNxGWnHvWwMyiyhlQtafvDKwEn/wAgR8gHJFodB7emf8rA TnukH7MVttCoHtjN5lvv9RSBHjNTZls5wR/ANlwdRuPQHd8ZGxLe3S6IuUB3zDSwFltLGurO N2kOMhs5mTGyypSa+uw3rtQbUAVYf1oPbiR4FLtiM8FLyEvE95hX5fPq9Qvx9FmN79kmCIEw jDKPqDaUf/OR2fEF0LSIbXHEk4tNqCEwx5DIJ0fp5/z5UzICUAmwxyRs5O/Hre1jzPsMVyud Ml9t7UTOJGKVWwRory1PMnOFxN+iz5/d4FhYSKXF7kfMiFgol4LuWaxJRwbBrr71VGBrRy2a L1nw6Bc= Cc: Eugene Grosbein , Garrett Wollman Message-ID: Date: Thu, 14 Feb 2019 12:30:30 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <0710bc0a-7691-b29f-3331-da94412ab737@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 69D2A6C9E6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.92 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.99)[-0.990,0]; NEURAL_HAM_SHORT(-0.94)[-0.936,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 11:30:36 -0000 Am 13.02.19 um 10:59 schrieb Andriy Gapon: > On 12/02/2019 20:17, Eugene Grosbein wrote: >> 13.02.2019 1:14, Eugene Grosbein wrote: >> >>> Use following command to see how much memory is wasted in your case: >>> >>> vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head >> >> Oops, small correction: >> >> vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort -k1,1 -rn | head > > I have a much uglier but somewhat more informative "one-liner" for > post-processing vmstat -z output: > > vmstat -z | tail +3 | awk -F '[:,] *' 'BEGIN { total=0; cache=0; used=0 } {u = > $2 * $4; c = $2 * $5; t = u + c; cache += c; used += u; total += t; name=$1; > gsub(" ", "_", name); print t, name, u, c} END { print total, "TOTAL", used, > cache } ' | sort -n | perl -a -p -e 'while (($j, $_) = each(@F)) { 1 while > s/^(-?\d+)(\d{3})/$1,$2/; print $_, " "} print "\n"' | column -t > > This would be much nicer as a small python script. Or, since you are already using perl: ---------------------------------------------------------------- #!/usr/local/bin/perl5 open STDIN, "vmstat -z |" or die "Failed to run vmstat program"; open STDOUT, "| sort -n @ARGV -k 2" or die "Failed to pipe through sort"; $fmt="%-30s %8.3f %8.3f %8.3f %6.1f%%\n"; while () { ($n, $s, undef, $u, $c) = split /[:,] */; next unless $s > 0; $n =~ s/ /_/g; $s /= 1024 * 1024; $u *= $s; $c *= $s; $t = $u + $c; next unless $t > 0; printf $fmt, $n, $t, $u, $c, 100 * $c / $t; $cache += $c; $used += $u; $total += $t; } printf $fmt, TOTAL, $total, $used, $cache, 100 * $cached / $total; close STDOUT; ---------------------------------------------------------------- This script takes additional parameters, e.g. "-r" to reverse the sort or "-r -k5" to print those entries with the highest percentage of unused memory at the top. (I chose to suppress lines with a current "total" of 0 - you may want to remove the "next" command above the printf in the loop to see them, but they did not seem useful to me.) > Or, even, we could add a sort option for vmstat -z / -m. Yes, and the hardest part might be to select option characters for the various useful report formats. ;-) Regards, STefan From owner-freebsd-stable@freebsd.org Thu Feb 14 18:11:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 803E714DC6F9 for ; Thu, 14 Feb 2019 18:11:32 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 83EF884C29 for ; Thu, 14 Feb 2019 18:11:31 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x336.google.com with SMTP id i20so12142302otl.0 for ; Thu, 14 Feb 2019 10:11:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VfMG06Jlfix7MZ44B7v0lYMhhPaEH8masNHZe5LSmtc=; b=UpFVf0xLZ2PPsR83BPqYTeb0/KMV+Esw5BnxRKl/cVD+Wt/NVIC6yEgNlPl4qx14aP Xnuq25gx5/LjSSwtMOk15Xixqx2IWPwWlRedYVO5ciJh8MhBlzChYks3rzUfNN7FVpNA IcqmGGbI+qtMhRwFJPuTHmVKhGaYqJ9Qz/g/SrhgUhXa7sG8YRrwhqBY2G/yA0P5WrPR lUzWYzMAEc+xI8vbDVN5oAs84GAB+LgmwOhQC6LL+617wVrN54hBZE3ZmbG3zlOCj0ut /yBi6VANhClIre08ndTbErcjfdP/GmpvmnGOCWij/uTAdXVxPOKyfvLz3Fj/JFjEQcMB m8Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VfMG06Jlfix7MZ44B7v0lYMhhPaEH8masNHZe5LSmtc=; b=cN7tNT3JgksCWK0SoqWd8Lb6GEQfXGwZYiM4qkuYEAg04+RIYi5wdPYYYgvfqq8y49 xMa4MPiPXTpPIxVDUJ9cTGyu8ycLbbjCGycnWKjSp1DcMyvyO9JL8uQkAOf4SaSLDddp P6emxevRQCWfHyWIuEpaDksiI84MuBcEODKVkubSvdxkXnqFsiwPaCPeBrFH06Yw56kD +XATNNX2p9KZaJR9LnfIqZ95XMw1uYWeiPysDwEm/0kMX40ReAgT99H41UaeDoWkVJV+ So5ER659HhvYKA5HuojO0qEGqkEWOI//A1iLn1mHxMsZ06hD9xXYBB/QD2LGsr/InAKp NZvg== X-Gm-Message-State: AHQUAuYP5X7lhlXAyrqO4Y/KTKogUPtagFqsNtC2vSijGhL2/TrVKrZa xGa0BG4E03jyhz1nfMoAjtElqj2yfl//q14expB0MPPc X-Google-Smtp-Source: AHgI3IbpM8cmb6ve7U3Dw7ZV7XTTgs0U2CL+I1GKviX8hlTNhRehk1kX9U863bF3HtGGpkMyMBfZqVmgGhpFzWdaXxw= X-Received: by 2002:a9d:5e8c:: with SMTP id f12mr3200136otl.343.1550167890297; Thu, 14 Feb 2019 10:11:30 -0800 (PST) MIME-Version: 1.0 References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> In-Reply-To: <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> From: Kevin Oberman Date: Thu, 14 Feb 2019 10:11:14 -0800 Message-ID: Subject: Re: Binary update to -STABLE? And if so, what do I get? To: Pete French Cc: FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 83EF884C29 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=UpFVf0xL; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::336 as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-4.82 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; IP_SCORE(-2.69)[ip: (-8.85), ipnet: 2607:f8b0::/32(-2.56), asn: 15169(-1.98), country: US(-0.07)]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.42)[-0.415,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 18:11:32 -0000 On Thu, Feb 14, 2019 at 3:10 AM Pete French wrote: > > > On 14/02/2019 01:43, Jason Tubnor wrote: > > I also have hit this IPv6 issue (I thought I was going crazy until I > worked > > it out) and other iflib issues in 12.0, which have been fixed in -STABLE > > that really should be patched in 12.0 or bring forward an early 12.1 > > release. For our use case, 12.0 is just too buggy for production at this > > rate and we won't touch it, which is a shame because there is a lot of > good > > work in there that we would like to use but it is trumped by the > breakages. > > Any reason behind not running STBLE out of interest ? Yes, 12 has been > buggy with regards to networking, but these things get fixed very fast > and I now have all my machines on the lattest STABLE in production, as > of yesterday. > > -pete. > Generally, not many. Far and away the biggest is the requirement to build from sources. It's not a big deal for me, but if I still had many systems to deal with, that would be a pain. I might also mention that just before the pre-release freeze and after a release, STABLE and be a bit unstable as developers rush to get things in before the freeze or to add things that they did not want to MFC with little test time before a release. In recent years this issue has significantly improved, though. I run either HEAD or STABLE on my personal system and RELEASE on my only server, the latter just so I can do binary updates. My server is on hold at 11.2 due to the IPv6 issue and I am amazed that this BIG oops did not result in an EN and a patched release. Lack of IPv6 is not, for many people, a minor issue. The bottom line is that the only real reasons I see for not running stable is the lack of binary updates, and issues with systems being slightly out of sync if all are not updated to the same SVN revision at all times. Those are very big reasons for many. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Thu Feb 14 18:22:20 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A223614DCC7C for ; Thu, 14 Feb 2019 18:22:20 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2621854D0 for ; Thu, 14 Feb 2019 18:22:19 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-lf1-x134.google.com with SMTP id l10so5248906lfh.9 for ; Thu, 14 Feb 2019 10:22:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3Q5GooYkOfwqS1+y+LQZuR4KbYNKSRRKC7UYkdscYeY=; b=uP1/7ufYMguXwymmSsxOl3X2dcnJPrNH1eGkvsKl9UqJkdS2QoHyG5ZYcF7kcOSFWF Tt3SKR/KnaNr2wtmHQXDDj3AyXadyGNIJEueq8mWtpLx3iFoM4VTxti+B4NBzjJ2qS5T S1QeYnhT1+iWTF8R24GnOXHsp2QvUjhbyleJ4qIWHFUjkdwLVPvs/3r28L+csChQrouR gKC5XJ3YiuIZlnStmeOEbKvk8sI+dhMCIkQb/W3oJt0C/KAgCNvwkPfWVuMIp0GLl3UD Y5YJxJaR2xlpPXyRq43eYtZ+jMNDVjsYgxpmTuVp9Lcri8oERkc0n6POAtn5jR/Nl1XZ 2z3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3Q5GooYkOfwqS1+y+LQZuR4KbYNKSRRKC7UYkdscYeY=; b=CP0rwUFeZ7+myJDV8lDUF9aWr64dD77mNuXGL8ys8Moj0UXGi043z7Jo5ebTUqceLK /80ywt09lZnDLJRJ2+DkXRnUhlogt9DpbafB4bQ5huKBLyyjdeH6tg+EmpOsuI6HTB7h seBdWeUF4reNLcsj8QRtU8Hm1E3kmCn6BweBhMmUHyHpoP7YLvorbSQEh9NEUevXgRoN DvbKWYmvvu6RPRbddZQC/qcwOuNfzQYR5+d1HbmgtoSFBP+LnKS9oLFCNUPeZyAaK8rJ MNV6kSrvVNYpuN4HM6zIspnajylHmkJI6QdBPqzpClC651G5XJEilzH/IpjKKCwFCUWf 5rVw== X-Gm-Message-State: AHQUAuZ3zQFCSz5Bkfw3PfPRe2rL8voCzz9q6fz7MwTRaCy1QGxEpopA h27TF0iPK0tUcTnmoj9F+ol1zAdcs9viD0B8tV2R5g== X-Google-Smtp-Source: AHgI3IY4a0Xrs5ZcNbazVbD3nxYC+moqmLPj+r0TNrbSHXShGSvP6XZ29+WM6s8L3j/YrCQM+/0oJ3+HToCxwiy8tnM= X-Received: by 2002:a19:41ca:: with SMTP id o193mr2865074lfa.121.1550168538001; Thu, 14 Feb 2019 10:22:18 -0800 (PST) MIME-Version: 1.0 References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> In-Reply-To: From: Freddie Cash Date: Thu, 14 Feb 2019 10:22:06 -0800 Message-ID: Subject: Re: Binary update to -STABLE? And if so, what do I get? To: Kevin Oberman Cc: FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: A2621854D0 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uP1/7ufY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2a00:1450:4864:20::134 as permitted sender) smtp.mailfrom=fjwcash@gmail.com X-Spamd-Result: default: False [-5.26 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.74)[ip: (-9.33), ipnet: 2a00:1450::/32(-2.30), asn: 15169(-1.98), country: US(-0.07)]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.52)[-0.518,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[4.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; SUBJECT_ENDS_QUESTION(1.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 18:22:20 -0000 On Thu, Feb 14, 2019 at 10:13 AM Kevin Oberman wrote: > On Thu, Feb 14, 2019 at 3:10 AM Pete French > wrote: > > On 14/02/2019 01:43, Jason Tubnor wrote: > > > I also have hit this IPv6 issue (I thought I was going crazy until I > > worked > > > it out) and other iflib issues in 12.0, which have been fixed in > -STABLE > > > that really should be patched in 12.0 or bring forward an early 12.1 > > > release. For our use case, 12.0 is just too buggy for production at > this > > > rate and we won't touch it, which is a shame because there is a lot of > > good > > > work in there that we would like to use but it is trumped by the > > breakages. > > > > Any reason behind not running STBLE out of interest ? Yes, 12 has been > > buggy with regards to networking, but these things get fixed very fast > > and I now have all my machines on the lattest STABLE in production, as > > of yesterday. > > > > -pete. > > > > Generally, not many. > > Far and away the biggest is the requirement to build from sources. It's not > a big deal for me, but if I still had many systems to deal with, that would > be a pain. > Just as one can setup a poudriere/synth system for building custom binary package repositories (so one builds packages on one system for easy installation on multiple systems using binary packages), one can also setup a custom freebsd-update server (so one builds the OS on one system, for easy installation on multiple servers using binary updates). And that can be done to track -STABLE or -CURRENT, I believe. Granted, I have never done it, nor looked too deeply into the documentation around it, but I do know it's possible. :) At least in theory. :D IOW, the days of needing to compile everything on each individual machine are behind us. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@freebsd.org Fri Feb 15 08:06:51 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B1B214F3A7E for ; Fri, 15 Feb 2019 08:06:51 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDD4E8422A for ; Fri, 15 Feb 2019 08:06:44 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id x1F86aLK051390; Fri, 15 Feb 2019 09:06:36 +0100 (CET) Received: from [217.29.44.250] ([217.29.44.250]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x1F86avK086425; Fri, 15 Feb 2019 09:06:36 +0100 (CET) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: Binary update to -STABLE? And if so, what do I get? From: "Patrick M. Hausen" In-Reply-To: Date: Fri, 15 Feb 2019 09:06:36 +0100 Cc: Pete French , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <3B4C0FF6-32C2-4DBB-BE5B-8C5F2EB59430@punkt.de> References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> To: Kevin Oberman X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: CDD4E8422A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 217.29.33.131 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-2.19 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:217.29.32.0/20]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[punkt.de]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[mailin.pluspunkthosting.de,mailin.pluspunkthosting.de]; NEURAL_HAM_SHORT(-0.78)[-0.776,0]; RCVD_IN_DNSWL_NONE(0.00)[131.33.29.217.list.dnswl.org : 127.0.10.0]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:16188, ipnet:217.29.32.0/20, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-0.61)[ipnet: 217.29.32.0/20(-1.68), asn: 16188(-1.35), country: DE(-0.01)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2019 08:06:51 -0000 Good morning, > Am 14.02.2019 um 19:11 schrieb Kevin Oberman : > Far and away the biggest is the requirement to build from sources. = It's not > a big deal for me, but if I still had many systems to deal with, that = would > be a pain. > [=E2=80=A6] > The bottom line is that the only real reasons I see for not running = stable > is the lack of binary updates, and issues with systems being slightly = out > of sync if all are not updated to the same SVN revision at all times. = Those > are very big reasons for many. We build from sources centrally, then zfs send/receive /usr/src and /usr/obj to all of the machines, then just do the install(kernel|world) part on all of them. Kind regards Patrick --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling From owner-freebsd-stable@freebsd.org Fri Feb 15 08:13:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C9E014F3FA0 for ; Fri, 15 Feb 2019 08:13:42 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Received: from v1.leiden.byshenk.net (v1.leiden.byshenk.net [37.97.209.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80B9584A4F for ; Fri, 15 Feb 2019 08:13:40 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Received: by v1.leiden.byshenk.net (Postfix, from userid 1002) id AE96275932; Fri, 15 Feb 2019 09:13:32 +0100 (CET) Date: Fri, 15 Feb 2019 09:13:32 +0100 From: Gregory Byshenk To: FreeBSD-STABLE Mailing List Cc: Kevin Oberman , Freddie Cash Subject: Re: Binary update to -STABLE? And if so, what do I get? Message-ID: <20190215081332.GB85865@v1.leiden.byshenk.net> References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.1 (2018-12-01) X-Rspamd-Queue-Id: 80B9584A4F X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [1.46 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.12)[0.120,0]; IP_SCORE(-0.36)[asn: 20857(-1.83), country: NL(0.02)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[byshenk.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.21)[0.210,0]; NEURAL_HAM_LONG(-0.69)[-0.692,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[v2.leiden.byshenk.net,v1.leiden.byshenk.net]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[freebsd@byshenk.net,byshenknet@byshenk.net]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20857, ipnet:37.97.128.0/17, country:NL]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_NEQ_ENVFROM(0.00)[freebsd@byshenk.net,byshenknet@byshenk.net]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2019 08:13:42 -0000 On Thu, Feb 14, 2019 at 10:22:06AM -0800, Freddie Cash wrote: > On Thu, Feb 14, 2019 at 10:13 AM Kevin Oberman wrote: > > Far and away the biggest is the requirement to build from sources. It's not > > a big deal for me, but if I still had many systems to deal with, that would > > be a pain. > Just as one can setup a poudriere/synth system for building custom binary > package repositories (so one builds packages on one system for easy > installation on multiple systems using binary packages), one can also setup > a custom freebsd-update server (so one builds the OS on one system, for > easy installation on multiple servers using binary updates). And that can > be done to track -STABLE or -CURRENT, I believe. > > Granted, I have never done it, nor looked too deeply into the documentation > around it, but I do know it's possible. :) At least in theory. :D > > IOW, the days of needing to compile everything on each individual machine > are behind us. They have been behind us for quite a long time. Even more than ten years ago already one could use a build machine for builds, and then just install on a collection of different servers -- thus using the same tested version everywhere. I'm doing that even now for a tiny number of "special" machines where I want to run STABLE instead of just using freebsd-update. -- gregory byshenk - gbyshenk@byshenk.net - Leiden, NL From owner-freebsd-stable@freebsd.org Fri Feb 15 10:17:25 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 189BB14D4525 for ; Fri, 15 Feb 2019 10:17:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 93CA28960F for ; Fri, 15 Feb 2019 10:17:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 53EA214D4524; Fri, 15 Feb 2019 10:17:24 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4169514D4522 for ; Fri, 15 Feb 2019 10:17:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5763E8960E for ; Fri, 15 Feb 2019 10:17:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x1FAHGsE031223 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 15 Feb 2019 12:17:19 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x1FAHGsE031223 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x1FAHGXc031222 for stable@freebsd.org; Fri, 15 Feb 2019 12:17:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Feb 2019 12:17:16 +0200 From: Konstantin Belousov To: stable@freebsd.org Subject: HEADS-UP: em users Message-ID: <20190215101716.GM24863@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2019 10:17:25 -0000 Please note the commit below, in particular the UPDATING entry. In short, if you use em ix ixv ixl iavf vmx drivers and compile them into kernel statically, you need to add device iflib to your configs. Standard in-tree configs were updated, so if you do e.g. 'include GENERIC' you do not need to do anything. ----- Forwarded message from Konstantin Belousov ----- Date: Fri, 15 Feb 2019 09:49:09 +0000 (UTC) From: Konstantin Belousov To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r344149 - in stable/12: . share/man/man4 sys/amd64/conf sys/arm64/conf sys/conf sys/dev/ixgbe sys/i386/conf sys/mips/conf sys/modules sys/modules/iflib sys/powerpc/conf sys/powerpc/conf... Author: kib Date: Fri Feb 15 09:49:09 2019 New Revision: 344149 URL: https://svnweb.freebsd.org/changeset/base/344149 Log: MFC r343617, r343618: Make iflib a loadable module. Modified: stable/12/UPDATING ============================================================================== --- stable/12/UPDATING Fri Feb 15 09:45:17 2019 (r344148) +++ stable/12/UPDATING Fri Feb 15 09:49:09 2019 (r344149) @@ -16,6 +16,13 @@ from older versions of FreeBSD, try WITHOUT_CLANG and the tip of head, and then rebuild without this option. The bootstrap process from older version of current across the gcc/clang cutover is a bit fragile. +20190214: + Iflib is no longer unconditionally compiled into the kernel. Drivers + using iflib and statically compiled into the kernel, now require + the 'device iflib' config option. For the same drivers loaded as + modules on kernels not having 'device iflib', the iflib.ko module + is loaded automatically. + From owner-freebsd-stable@freebsd.org Fri Feb 15 20:51:25 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0ACB14E8E8A for ; Fri, 15 Feb 2019 20:51:25 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DC9884EF5; Fri, 15 Feb 2019 20:51:25 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 221F11248A; Fri, 15 Feb 2019 20:51:25 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:195:8843:f798:dbbc]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 129E9C132; Fri, 15 Feb 2019 23:51:20 +0300 (MSK) Date: Fri, 15 Feb 2019 23:51:20 +0300 From: Lev Serebryakov Reply-To: Lev Serebryakov Organization: FreeBSD Message-ID: <107191835.20190215235120@serebryakov.spb.ru> To: Eugene Grosbein , Kurt Jaeger CC: freebsd-stable@freebsd.org, Garrett Wollman Subject: Re: 11.2-STABLE kernel wired memory leak In-Reply-To: <3d7f4ac4-67e4-b010-64c6-0d84a4451246@grosbein.net> References: <201902121757.x1CHve0h056876@hergotha.csail.mit.edu> <7125e053-5adf-929a-bde6-a64fceae2aaa@grosbein.net> <8a9361fd-4701-4e33-33f0-e4800af7637c@grosbein.net> <20190212182915.GM2748@home.opsec.eu> <80e414af-058c-5f84-0f9b-36529d5e06b4@grosbein.net> <331bbb5f-6627-d73c-d174-0865234d296f@FreeBSD.org> <3d7f4ac4-67e4-b010-64c6-0d84a4451246@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 5DC9884EF5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_SHORT(-0.95)[-0.952,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2019 20:51:25 -0000 Hello Eugene, Tuesday, February 12, 2019, 10:18:09 PM, you wrote: > Do you have/had some memory pressure here? Growth of swap usage? After several days, ARC is even smaller, but Wired is the same: Mem: 88M Active, 904M Inact, 29G Wired, 1121M Free ARC: 13G Total, 5288M MFU, 6937M MRU, 2880K Anon, 44M Header, 1036M Other 11G Compressed, 15G Uncompressed, 1.32:1 Ratio :-\ Strill "lowmem_uptime" is zero, so no "low memory" situation, but ARC becomes smaller and smaller (and hit rate goes down too) and it is not clear where are all this "Wired" RAM :-( I had 16G on this system with same load and about half-year ago 14G was typical ARC with 97% hit rate and now it is 32G (other things are unchanged) and it is 13G ARC and 94% hit rate. And all memory is still Wired! -- Best regards, Lev mailto:lev@FreeBSD.org From owner-freebsd-stable@freebsd.org Sat Feb 16 11:37:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EF6B14E10A0 for ; Sat, 16 Feb 2019 11:37:15 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A8E5E86C04 for ; Sat, 16 Feb 2019 11:37:12 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.drayhouse.twisted.org.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1guyHO-000Ls9-BO for freebsd-stable@freebsd.org; Sat, 16 Feb 2019 11:37:10 +0000 Subject: Re: Binary update to -STABLE? And if so, what do I get? To: freebsd-stable@freebsd.org References: <80937c57-7757-3c70-5198-4da12c4f23d9@denninger.net> <20190213134939.GO2748@home.opsec.eu> <29563d3e-608d-591e-89bb-bf428b52bdc8@denninger.net> <1656e7f5-ef76-55d8-f46c-416223c5f975@ingresso.co.uk> <3B4C0FF6-32C2-4DBB-BE5B-8C5F2EB59430@punkt.de> From: Pete French Message-ID: <6629994a-d8df-548a-4163-6ab9f4847d0f@ingresso.co.uk> Date: Sat, 16 Feb 2019 11:37:31 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:66.0) Gecko/20100101 Thunderbird/66.0 MIME-Version: 1.0 In-Reply-To: <3B4C0FF6-32C2-4DBB-BE5B-8C5F2EB59430@punkt.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A8E5E86C04 X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 2a02:b90:3002:411::3 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [5.92 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:b90:3002:411::3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.45)[0.447,0]; RCPT_COUNT_ONE(0.00)[1]; RDNS_NONE(1.00)[]; NEURAL_SPAM_SHORT(0.85)[0.854,0]; MX_GOOD(-0.01)[us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mi mecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; NEURAL_SPAM_LONG(0.95)[0.947,0]; IP_SCORE(-0.02)[country: GB(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:16082, ipnet:2a02:b90::/32, country:GB]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Feb 2019 11:37:15 -0000 On 15/02/2019 08:06, Patrick M. Hausen wrote: > We build from sources centrally, then zfs send/receive /usr/src and > /usr/obj to all of the machines, then just do the install(kernel|world) > part on all of them. I do this occasionally, but in the main I sumply NFS mount /usr/src and /usr/obj and do the upgrade over NFS. Only time I have slight problems with this is major version upgrades sometimes. When that happens i zfs send or rsync to move src and obj.