Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 25 Jun 2013 02:14:50 -0600 (MDT)
From:      Mike Brown <mike@skew.org>
To:        freebsd-questions@freebsd.org
Subject:   Re: Should I be able to use mergemaster with freebsd-update?
Message-ID:  <201306250814.r5P8Eo4Z077311@chilled.skew.org>
In-Reply-To: <201306230420.r5N4K9Yr054286@chilled.skew.org>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
> I'm using freebsd-update to upgrade my system to the latest minor version
> (from 8.3-RELEASE to 8.4-RELEASE).
> 
> I'm surprised that the merge handling system isn't more robust. When upgrading 
> the old way, from source, I was used to using mergemaster to handle any 
> merges that couldn't be done automatically.
> 
> But when using freebsd-update, it seems that any failed merges require that 
> you get dumped into an empty text editor for each file.
> [...]

As I continue with this process, doing all the mergemaster tasks manually, I'm 
finding that the situation is even worse than I first realized.


First, the relatively painless part. As I mentioned, after running 
'freebsd-update -r 8.4-RELEASE upgrade', I had to deal with the un-mergeable 
files.

Although mergemaster apparently isn't an option, its interactive merge 
function is really just a front-end for sdiff, so I found that I could 
replicate that part of its functionality by doing this in a separate window 
(-w 100 because I use a 100-column terminal):

cd /var/db/freebsd-update/merge/8.4-RELEASE
find -X . -type f | xargs -n 1 -o -I % sh -c '{ echo Now processing %. left=current, right=new, help="?"; sdiff -d -w 100 -o ../new/% ../old/% %; }'

This populated my 'new' directory with merged files, so that (in the first 
window) when I opened the text editor for each one, I only had to just give it 
a once-over and exit the editor.

Among the diffs in this 8.3 to 8.4 upgrade were changes to /etc/master.passwd 
and /etc/passwd, to add the 'auditdistd' and 'hast' users. As reported in 
March 2012 [1] in relation to 8.x to 9.x upgrades, this won't work as 
expected, because freebsd-update doesn't run pwd_mkdb after the master.passwd 
update.



Now the real hurt begins; in the 8.3 to 8.4 upgrade, it's even worse.

Once I saved all the files in the editor, I was prompted to approve a diff for 
each one. I had to answer "y" or the entire process aborts.

Among the changes I was asked to approve, besides visible diffs, were 
unspecified differences in /etc/pwd.db and /etc/spwd.db, the binary files that 
contain the password database.  There's no choice but to answer "y" and 
approve them, and I don't get any opportunity to rebuild them properly.

So apparently, freebsd-update wants to install new, stock password databases, 
which are out-of-sync with my customized, merged master.passwd & passwd files. 
(And because of the way the freebsd-update system works, what I actually 
approved were empty, 0-byte files, the result of the failed merges.)

What would happen if I just let it do this? Surely I wouldn't be able to log 
in, after the reboot, right?



After approving the files (again, I had no choice!), I was presented with 
lists of all the files that would be deleted, added, and modified. Sure 
enough, bad /etc/pwd.db and /etc/spwd.db files were in the list.

At this point, the merge folders were now gone; I no longer had the new 
master.passwd in a recognizable place. So I thought, OK, I'll run 
'freebsd-update install' and hope that the new files end up in /etc. Then I 
could just run 'pwd_mkdb -p /etc/master.passwd' to regenerate passwd, pwd.db 
and spwd.db before my reboot.

But the 'freebsd-update install' didn't put them there yet; I guess that 
doesn't happen till after the reboot. So they're still sitting in a staging 
folder, now gzipped and with obfuscated names, indexed in a separate file.

Averting this disaster-in-the-making is not at all straightforward:

cd /var/db/freebsd-update
mkdir -m 0700 /tmp/oldpwdfiles
zcat files/`grep '^/etc/master\.passwd' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz > /tmp/oldpwdfiles/master.passwd
zcat files/`grep '^/etc/passwd' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz > /tmp/oldpwdfiles/passwd
zcat files/`grep '^/etc/pwd\.db' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz > /tmp/oldpwdfiles/pwd.db
zcat files/`grep '^/etc/spwd\.db' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz > /tmp/oldpwdfiles/spwd.db
mkdir -m 0700 /tmp/newpwdfiles
pwd_mkdb -d /tmp/newpwdfiles -p /tmp/oldpwdfiles/master.passwd
gzip /tmp/newpwdfiles/*
mv /tmp/newpwdfiles/master.passwd.gz files/`grep '^/etc/master\.passwd' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz
mv /tmp/newpwdfiles/passwd.gz files/`grep '^/etc/passwd' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz
mv /tmp/newpwdfiles/pwd.db.gz files/`grep '^/etc/pwd\.db' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz
mv /tmp/newpwdfiles/spwd.db.gz files/`grep '^/etc/spwd\.db' install.LYQAJQ/INDEX-NEW | cut -d \| -f 7`.gz
rm -fr /tmp/oldpwdfiles /tmp/newpwdfiles


I'm really shocked that it came to this. Did I just overlook the 
"--no-surprises" option in freebsd-update?



And now, before I reboot, I have to figure out how to handle the changes that 
got made in /etc/mail ... ordinarily I'd run 'make all install restart' in 
there. Not an option till after reboot, though. At least it's not crucial for 
the reboot to work.

Again, this is something that mergemaster was really good for. But the 
freebsd-update system really makes it difficult to do that kind of pre-reboot 
merging and database-regeneration that's an essential part of the upgrade 
process.


Well, thanks for reading this far. I'm scared to death to reboot now, since my
server is in another city, but we'll see how it goes.


[1] <http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/165954>;



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?201306250814.r5P8Eo4Z077311>