Date: Thu, 20 Aug 2015 23:46:57 -0700 From: Andreas Ott <andreas@naund.org> To: Glen Barber <gjb@freebsd.org> Cc: Slawa Olhovchenkov <slw@zxy.spb.ru>, Christian Kratzer <ck@cksoft.de>, freebsd-stable@freebsd.org, FreeBSD Security Team <secteam@freebsd.org> Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150820234657.A23228@naund.org> In-Reply-To: <20150817155434.GT24069@FreeBSD.org>; from gjb@freebsd.org on Mon, Aug 17, 2015 at 03:54:34PM %2B0000 References: <alpine.BSF.2.20.1508161911450.49345@noc1.cksoft.de> <20150817155022.GD3158@zxy.spb.ru> <20150817155434.GT24069@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, On Mon, Aug 17, 2015 at 03:54:34PM +0000, Glen Barber wrote: [...] > Secteam. I've cc'd them. the issue persists even when forcing to a single update server, update2.freebsd.org is very close to this server. The DNS (?) response of "Looking up update2.freebsd.org mirrors... none found" is also still there. I end up with files where name and hash don't match. It appears to be an issue how the filename is generated from the hash, while the fact that the file can be unzipped from .gz format tells me it is not really corrupted. Or perhaps, how the gzip compression gets handled on small files, with certain content and padding. For many files, the SHA256 over the ascii content after gunzip is equal to the filename. This is not the case on the files that are flagged as mismatch. I have not looked at the code, but I think it will exit after the first mismatch, even if there would be more mismatched files/checksums. This server is starting from 10.1-RELEASE-p18, fully updated. I removed all files in /var/db/freebsd-update/* , rebooted, then ran freebsd-update fetch again, and got the meta files. I observe, that when running the freebsd-update "upgrade" again after the first failure, I end up with less patches, less downloads, presumably because a large portion got patched in the previous round, but the hash issue exists on a different file. I did a simple checksum verification on the 809 *.gz files after the second run # for f in `ls *gz`; do ls -la $f; echo $f; gunzip -c $f |sha256; done and the output is deposited here: https://files.naund.org/andreas/freebsd-update-SHA256-mismatch.txt Eventually, in the third run, the upgrade completed. First run: [root@dev1 /usr/home/andreas]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 41142 patches.....10....20....30....40....50....60....70....80....90....100.... [... you all can count to 41030....] 41040....41050....41060....41070....41080....41090....41100....41110....41120....41130....41140. done. Applying patches... done. Fetching 5820 files... a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955 has incorrect hash. [root@dev1 /usr/home/andreas]# [root@dev1 /usr/home/andreas]# cd /var/db/freebsd-update/ [root@dev1 /var/db/freebsd-update]# ls -la a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955* -rw-r--r-- 1 root wheel 151 Aug 21 05:38 a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz [root@dev1 /var/db/freebsd-update]# gunzip -c a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz |sha256 a3649107fd11187af3797b596807f82cbab6f0ccae026b26a3eea3669a9223e5 [root@dev1 /var/db/freebsd-update]# [root@dev1 /var/db/freebsd-update]# gunzip -c a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz .\" $FreeBSD: releng/10.2/tools/build/options/WITHOUT_FILE 279506 2015-03-01 22:07:54Z ngie $ Set to not build .Xr file 1 and related programs. [root@dev1 /var/db/freebsd-update]# Second run: [root@dev1 /var/db/freebsd-update]# date Fri Aug 21 05:52:14 UTC 2015 [root@dev1 /var/db/freebsd-update]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 354 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150....160....170....180....190....200....210....220....230....240....250....260....270....280....290....300....310....320....330....340....350.. done. Applying patches... done. Fetching 1810 files... e663aaaca813b1ffebc92189b0f209a413806d0faf5a700bab9c9326e6e5b556 has incorrect hash. [root@dev1 /var/db/freebsd-update]# Third run: [root@dev1 /var/db/freebsd-update]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 1 patches. done. Applying patches... done. Fetching 521 files... done. Attempting to automatically merge changes in files... done. The following file could not be merged automatically: /etc/ntp.conf Press Enter to edit this file in /usr/bin/vi and resolve the conflicts manually... [manually fix diff and write file], then acknowledge the change log of all updated files, proceed with install (kernel), reboot and one more install (user land). Additional debug output available, just ask for it. I have a second server of the same specs awaiting upgrade as well, and then some more. Thanks, andreas -- Andreas Ott K6OTT +1.408.431.8727 andreas@naund.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150820234657.A23228>