From owner-freebsd-doc@FreeBSD.ORG Fri Nov 20 20:27:53 2009 Return-Path: Delivered-To: doc@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89FFB106566B; Fri, 20 Nov 2009 20:27:53 +0000 (UTC) (envelope-from jhelfman@e-e.com) Received: from mail.experts-exchange.com (mail.experts-exchange.com [64.156.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id 5FF788FC1A; Fri, 20 Nov 2009 20:27:53 +0000 (UTC) Received: from eggman.experts-exchange.com (unknown [72.29.180.81]) by mail.experts-exchange.com (Postfix) with ESMTP id E69BE4A3085D; Fri, 20 Nov 2009 12:25:32 -0800 (PST) Received: by eggman.experts-exchange.com (sSMTP sendmail emulation); Fri, 20 Nov 2009 12:26:44 -0800 Date: Fri, 20 Nov 2009 12:26:44 -0800 From: Jason To: Giorgos Keramidas Message-ID: <20091120202642.GK6241@eggman.experts-exchange.com> References: <4B05BA06.3010303@FreeBSD.org> <87ws1luqmx.fsf@kobe.laptop> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="x4pBfXISqBoDm8sr" Content-Disposition: inline In-Reply-To: <87ws1luqmx.fsf@kobe.laptop> X-Operating-System: FreeBSD 7.2-RELEASE-p4 X-Living-The-Dream: I love the SLO Life! User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Tom Rhodes , "doc@FreeBSD.org" , Gabor PALI , Gabor Kovesdan , Rene Ladan , Manolis Kiagias Subject: Re: [RFC] Article on freebsd-update-server X-BeenThere: freebsd-doc@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Documentation project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2009 20:27:53 -0000 --x4pBfXISqBoDm8sr Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Hi Everyone, I've made comments inline to the patch. I would not want to make any changes to the formatting, as I believe it is good as it stands now. I very much appreciate everyones input, time and comments on my article, and have learned so much in this process. I have attached the diff with comments. Thanks, again Jason On Fri, Nov 20, 2009 at 06:07:02PM +0200, Giorgos Keramidas thus spake: >On Thu, 19 Nov 2009 23:35:02 +0200, Manolis Kiagias wrote: >> Hi all, >> >> I am sending this to my usual reviewers and the doc@ list, everyone is >> welcome to comment. >> >> Jason Helfman (jhelfman@e-e.com) is actively involved with our >> documentation set and you may have seen some of his patches. >> >> His initial contribution however is the article in this PR: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=139095 >> >> I've been working on this article with Jason for quite some time, and >> we think it's about time it gets a broader review. >> >> The source is here: >> >> http://people.freebsd.org/~manolis/patches/freebsd-update-server/article.sgml.txt > >Some of the changes I made locally while reading the source are shown in >diff format below. They are based on the latest revision of the article >source at http://mercurial.dyndns.org/mercurial/freebsd-doc-el (the last >change I pulled was 6d8923ea5ca3 from 2009-11-20 10:05:02 +0200). > >Some of the changes I made to the article are rewording, formatting, >spell checking fixes, or other tiny nits I noticed. Others are merely >adding inline SGML comments. Most of them are things we can fix before >committing this to CVS. There are also a few that are probably ok to >fix later. > >: diff -r 6d8923ea5ca3 en_US.ISO8859-1/articles/freebsd-update-server/article.sgml >: --- a/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 10:05:02 2009 +0200 >: +++ b/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 17:47:40 2009 +0200 >: @@ -80,11 +80,11 @@ >: >: >: >: - An : - url="&url.books.handbook;/network-apache.html">Apache >: - web server, with over half of the space required for the >: - build. For instance, our test builds total 4 GB, and the >: - webserver space needed to distribute updates is 2.6 GB. >: + A web server, like : + url="&url.books.handbook;/network-apache.html">Apache, >: + with over half of the space required for the build. For instance, >: + our test builds total 4 GB, and the webserver space needed to >: + distribute updates is 2.6 GB. >: > >If freebsd-update can work with other HTTP servers, it is a good idea to avoid >writing Apache-specific instructions. I know a few installations of FreeBSD >that prefer lighttpd or nginx, for example. > >: Configuration: Installation & Setup >: >: - Download freebsd-update-server >: + Download >: + the freebsd-update-server >: software. A tarball may be downloaded, or use &man.csup.1; and the > >Tiny nit. > >: >: - This is where the subroutine fetchiso() >: - declared in scripts/build.subr will contact >: - the configured source for downloading the &os; ISO. This setting >: - is not limited to ftp, it may also be >: - a web (httpd) address as well. >: - For our purposes, ISO's are on the same server as our internal >: - http server that will be serving updates. The software has been >: - configured to look in that location. For this setup, we have to >: - alter the routine to fetch the ISO. By copying the source >: - build.subr to >: - scripts/RELEASE/ARCHITECTURE/build.subr this >: - file will be sourced instead of the released source for >: - build.subr. >: + This is the location where ISO images are downloaded from (by >: + the fetchiso() subroutine >: + of scripts/build.subr). The location >: + configured here is not limited to FTP URIs. Any URI scheme >: + supported by the standard &man.fetch.1; utility should work fine. >: + In our own setup the ISO images are on the same internal http >: + server that will be serving the updates. >: + >: + Customizations to the fetchiso() code can >: + be installed by copying the >: + default build.subr script to the release- and >: + architecture-specific area >: + at scripts/RELEASE/ARCHITECTURE/build.subr >: + and applying local edits. > >I tried to reword the original text here. If you like the new version, please >feel free to keep it. If not, we have to find a good way to simplify the >large sentences of the original. > >: BUILDHOSTNAME >: >: >: - Host where software will be built. Coincidentally, this >: - information will be displayed on updated systems when >: - issuing: >: + The name of the build host. This information will be >: + displayed on updated systems when issuing: > >"Coincidentally" seems superfluous here. > >: SSHKEY >: >: >: - Key used when uploading data to an update server which will >: - provide patches or upgrades to clients. A key pair may be >: - created using ssh-keygen -t dsa. Alteration >: - of this parameter is not required as stadnard password >: - authentication through ssh will be >: - used. >: + The SSH key for uploading files to >: + the update server. A key pair can be created by >: + typing ssh-keygen -t dsa. This parameter is >: + optional; standard password authentication will be used as a >: + fallback authentication method when SSHKEY is >: + not defined. >: >: - &man.ssh-keygen.1; has more detailed information in creating >: - a key pair. >: + &man.ssh-keygen.1; has more detailed information >: + about SSH and the appropriate steps for >: + creating and using one. > >Another suggestion for rewording the text, writing smaller sentences, and >other minor wording nits. > >: >: - Account that files are uploaded to on remote system. >: + Account that files are uploaded to on the update >: + server. >: >: >: >: @@ -194,54 +197,68 @@ MASTERDIR=update-master.freebsd.org: MASTERDIR >: >: >: - Directory where files are uploaded to on remote >: - system. >: + Directory where files are uploaded to on the update >: + server. > >I think I like "update server" better than "remote system" here. The update >server may be the same machine that builds the snapshots. By removing the >unstated assumption that it is *not*, the text sounds more accurate. > >: >: >: >: - Now that build directives are set, the installation files are >: - configured for a build. For this example, we will use >: - RELEASE-7.2 under amd64 >: - architecture. Configuration files for &i386; >: - architecture are available with downloaded source. >: + The default build.conf file shipped with >: + the freebsd-update-server sources are >: + suitable for building &i386; releases of &os;. As an example of >: + building an update server for other architectures we will show in >: + the following paragraphs the configuration changes needed for an >: + AMD64 update server: > >We haven't actually described *any* installation so far. So I changed this >part to describe what freebsd-update-server defaults seem to be, and switched >the rest of the section from alternating text and samples to a > with separate steps: > >: - Create build environment directory under : - class="directory">scripts/RELEASE-7.2/amd64. >: + >: + >: + Create a build environment for AMD64: >: >: - >: - &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 >: - >: + >: + &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 >: + >: + >: >: - This is the build.conf file that should be >: - placed in the directory just created. >: + >: >: - # SHA256 hash of RELEASE disc1.iso image. >: + Install a build.conf file in the >: + newly created build directory. The build configuration >: + options for &os; 7.2-RELEASE on AMD64 should be similar >: + to: >: + >: + # SHA256 hash of RELEASE disc1.iso image. >: export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 >: >: # Components of the world, source, and kernels >: export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32" >: -export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ >: - lib libexec release rescue sbin secure share sys tools \ >: - ubin usbin cddl" >: +export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ >: + lib libexec release rescue sbin secure share sys tools \ >: + ubin usbin cddl" >: export KERNELPARTS="generic" >: >: # EOL date >: export EOL=1275289200 >: >: - >: - To generate the "End of Life" number for >: - build.conf, refer to the "Estimated EOL" posted >: - on the &os; >: - Security Website. Based on this date, you can issue >: - date -j -f '%Y%m%d-%H%M%S' '20090401-000000' +%s, >: - and substitute actual date parameters for those stated by >: - &os;. >: + >: + To generate the "End of Life" number for >: + build.conf, refer to the "Estimated >: + EOL" posted on >: + the &os; >: + Security Website. You can derive the value >: + of EOL from the date listed on the web >: + site, using the &man.date.1; utility, e.g.: >: >: - The &man.sha256.1; hash key for the desired release, is >: - published within the respective : - url="&url.base;/releases/">release announcement . >: - >: + &prompt.user; date -j -f '%Y%m%d-%H%M%S' '20090401-000000' '+%s' >: + >: + >: + >: + The &man.sha256.1; hash key for the desired release, is >: + published within the >: + respective release >: + announcement. >: + >: + >: + >: > >NOTE: The above can probably stand on their own, outside of the > element. > >: >: @@ -273,9 +290,9 @@ enter aes-256-cbc encryption password: >: Verifying - enter aes-256-cbc encryption password: >: >: >: - Note down the generated KeyPrint; this value is entered into >: - /etc/freebsd-update.conf for binary >: - updates. >: + Keep a note of the generated key fingerpring. This value is >: + entered into /etc/freebsd-update.conf for >: + binary updates. >: > >There are various places that the article refers to "KeyPrint". I think it >means "key fingerpring", but I am not sure. If that's what the real meaning >should be, please use "key fingerprint". > >: Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE >: @@ -411,10 +428,7 @@ to sign the release. >: file named USAGE. Execute >: scripts/approve.sh, as directed. This will sign >: the release, and move components into a staging area suitable for >: - uploading. It is important to make sure that your key is mounted >: - during this process. A simple df will show if it >: - is mounted. If not mounted, mount the key with the passphrase supplied >: - when creating it earlier. >: + uploading. > >I don't know where the key mounting bits come from. It seems to refer to >those FreeBSD installations where PGP keys are stored in removable media, like >a USB flash disk. Why do we have to explicitly mention this here? After all, >we don't describe how gpg-agent(1) works, or how seahorse(1) integrates PGP >with Gnome, or any other case of the dozens of PGP setups possible... > >: @@ -524,9 +547,11 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st >: >: When running a patch level build, we are assuming that previous >: patches are in place. When a patch build is run, it will run all >: - patches less than or equal to the number specified. Beyond this, >: - you will have to take appropriate measures to verify authenticity >: - of the patch. >: + patches less than or equal to the number specified. >: + >: + It is up to the administrator of the freebsd-update >: + server to take appropriate measures to verify the authenticity of >: + every patch. > >I think we ought to emphasize a bit the part about patch authenticity, but I >am not sure if I chose the right way to do this. > >: - Follow the same process as noted before for appoving a build. >: + Follow the same process as noted before for approving a build: > >Typo. > >There are more changes, in the attached patch. Most of them are attempts to >improve the wording of various small parts of the article. Please see the >attached diff for all of them. > >One more important detail. We are still discussing at doceng@ how we can >bring the final article into CVS. So, please hold from committing this, until >we have resolved all the remaining details. > >I'm sure that a lot of people will love reading an article that describes in >detail how to set up a local freebsd-update server. Thanks for all the work >done so far on what seems to be an excellent article! :-D > >diff -r 6d8923ea5ca3 en_US.ISO8859-1/articles/freebsd-update-server/article.sgml >--- a/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 10:05:02 2009 +0200 >+++ b/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 17:48:23 2009 +0200 >@@ -80,11 +80,11 @@ > > > >- An - url="&url.books.handbook;/network-apache.html">Apache >- web server, with over half of the space required for the >- build. For instance, our test builds total 4 GB, and the >- webserver space needed to distribute updates is 2.6 GB. >+ A web server, like + url="&url.books.handbook;/network-apache.html">Apache, >+ with over half of the space required for the build. For instance, >+ our test builds total 4 GB, and the webserver space needed to >+ distribute updates is 2.6 GB. > > > >@@ -97,7 +97,8 @@ > > Configuration: Installation & Setup > >- Download freebsd-update-server >+ Download >+ the freebsd-update-server > software. A tarball may be downloaded, or use &man.csup.1; and the > projects-all collection. > >@@ -138,19 +139,20 @@ MASTERDIR=update-master.freebsd.org FTP > > >- This is where the subroutine fetchiso() >- declared in scripts/build.subr will contact >- the configured source for downloading the &os; ISO. This setting >- is not limited to ftp, it may also be >- a web (httpd) address as well. >- For our purposes, ISO's are on the same server as our internal >- http server that will be serving updates. The software has been >- configured to look in that location. For this setup, we have to >- alter the routine to fetch the ISO. By copying the source >- build.subr to >- scripts/RELEASE/ARCHITECTURE/build.subr this >- file will be sourced instead of the released source for >- build.subr. >+ This is the location where ISO images are downloaded from (by >+ the fetchiso() subroutine >+ of scripts/build.subr). The location >+ configured here is not limited to FTP URIs. Any URI scheme >+ supported by the standard &man.fetch.1; utility should work fine. >+ In our own setup the ISO images are on the same internal http >+ server that will be serving the updates. >+ >+ Customizations to the fetchiso() code can >+ be installed by copying the >+ default build.subr script to the release- and >+ architecture-specific area >+ at scripts/RELEASE/ARCHITECTURE/build.subr >+ and applying local edits. > > > >@@ -158,9 +160,8 @@ MASTERDIR=update-master.freebsd.org BUILDHOSTNAME > > >- Host where software will be built. Coincidentally, this >- information will be displayed on updated systems when >- issuing: >+ The name of the build host. This information will be >+ displayed on updated systems when issuing: > > &prompt.user; uname -v > >@@ -170,15 +171,16 @@ MASTERDIR=update-master.freebsd.org SSHKEY > > >- Key used when uploading data to an update server which will >- provide patches or upgrades to clients. A key pair may be >- created using ssh-keygen -t dsa. Alteration >- of this parameter is not required as stadnard password >- authentication through ssh will be >- used. >+ The SSH key for uploading files to >+ the update server. A key pair can be created by >+ typing ssh-keygen -t dsa. This parameter is >+ optional; standard password authentication will be used as a >+ fallback authentication method when SSHKEY is >+ not defined. > >- &man.ssh-keygen.1; has more detailed information in creating >- a key pair. >+ &man.ssh-keygen.1; has more detailed information >+ about SSH and the appropriate steps for >+ creating and using one. > > > >@@ -186,7 +188,8 @@ MASTERDIR=update-master.freebsd.org MASTERACCT > > >- Account that files are uploaded to on remote system. >+ Account that files are uploaded to on the update >+ server. > > > >@@ -194,54 +197,68 @@ MASTERDIR=update-master.freebsd.org MASTERDIR > > >- Directory where files are uploaded to on remote >- system. >+ Directory where files are uploaded to on the update >+ server. > > > > >- Now that build directives are set, the installation files are >- configured for a build. For this example, we will use >- RELEASE-7.2 under amd64 >- architecture. Configuration files for &i386; >- architecture are available with downloaded source. >+ The default build.conf file shipped with >+ the freebsd-update-server sources are >+ suitable for building &i386; releases of &os;. As an example of >+ building an update server for other architectures we will show in >+ the following paragraphs the configuration changes needed for an >+ AMD64 update server: > >- Create build environment directory under - class="directory">scripts/RELEASE-7.2/amd64. >+ >+ >+ Create a build environment for AMD64: > >- >- &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 >- >+ >+ &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 >+ >+ > >- This is the build.conf file that should be >- placed in the directory just created. >+ > >- # SHA256 hash of RELEASE disc1.iso image. >+ Install a build.conf file in the >+ newly created build directory. The build configuration >+ options for &os; 7.2-RELEASE on AMD64 should be similar >+ to: >+ >+ # SHA256 hash of RELEASE disc1.iso image. > export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 > > # Components of the world, source, and kernels > export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32" >-export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ >- lib libexec release rescue sbin secure share sys tools \ >- ubin usbin cddl" >+export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ >+ lib libexec release rescue sbin secure share sys tools \ >+ ubin usbin cddl" > export KERNELPARTS="generic" > > # EOL date > export EOL=1275289200 > >- >- To generate the "End of Life" number for >- build.conf, refer to the "Estimated EOL" posted >- on the &os; >- Security Website. Based on this date, you can issue >- date -j -f '%Y%m%d-%H%M%S' '20090401-000000' +%s, >- and substitute actual date parameters for those stated by >- &os;. >+ >+ To generate the "End of Life" number for >+ build.conf, refer to the "Estimated >+ EOL" posted on >+ the &os; >+ Security Website. You can derive the value >+ of EOL from the date listed on the web >+ site, using the &man.date.1; utility, e.g.: > >- The &man.sha256.1; hash key for the desired release, is >- published within the respective - url="&url.base;/releases/">release announcement . >- >+ &prompt.user; date -j -f '%Y%m%d-%H%M%S' '20090401-000000' '+%s' >+ >+ >+ >+ The &man.sha256.1; hash key for the desired release, is >+ published within the >+ respective release >+ announcement. >+ >+ >+ > > > >@@ -273,9 +290,9 @@ enter aes-256-cbc encryption password: > Verifying - enter aes-256-cbc encryption password: > > >- Note down the generated KeyPrint; this value is entered into >- /etc/freebsd-update.conf for binary >- updates. >+ Keep a note of the generated key fingerpring. This value is >+ entered into /etc/freebsd-update.conf for >+ binary updates. > > > At this point, we are ready to stage a build. >@@ -330,8 +347,8 @@ world|base|/usr/lib/libalias_ftp.a > > > Then the build of the world is performed again, with world >- patches. A more detailed explanation may be found in >- scripts/build.subr. >+ patches. A more detailed explanation may be found >+ in scripts/build.subr. > > > Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE >@@ -411,10 +428,7 @@ to sign the release. > file named USAGE. Execute > scripts/approve.sh, as directed. This will sign > the release, and move components into a staging area suitable for >- uploading. It is important to make sure that your key is mounted >- during this process. A simple df will show if it >- is mounted. If not mounted, mount the key with the passphrase supplied >- when creating it earlier. >+ uploading. > > > &prompt.root; cd /usr/local/freebsd-update-server >@@ -436,6 +450,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > &prompt.root; sh scripts/upload.sh amd64 RELEASE-7.2 > > >+ > The uploaded files will need to be in the > DocumentRoot of the webserver in order for updates > to be distributed. For further explanation, please refer to the >@@ -443,6 +459,10 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > url="&url.books.handbook;/network-apache.html">Apache > documentation. > >+ > > Updates for the current release of the &os; system you are > updating, and what you want to upgrade to need >@@ -453,11 +473,14 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > &os; 7.2-RELEASE. > > >+ > Update client's KeyPrint and > ServerName in > /etc/freebsd-update.conf, and perform updates as > instructed in the url="&url.books.handbook;/updating-freebsdupdate.html">&os; Update >+ > instructions in the Handbook. > > For reference, here is the entire run of @@ -467,9 +490,9 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > > Building a Patch > >- In the event a + Every time a url="&url.base;/security/advisories.html">security advisory >- is posted, a patch update can be built. >+ is announced, a patch update can be built. > > For this example, we will be using 7.1-RELEASE. > >@@ -487,8 +510,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > > > >- Create patch directory under >- /usr/local/freebsd-update-server/patches/ for the respective release. >+ Create the patch directory of the respective release >+ under /usr/local/freebsd-update-server/patches/. > > > &prompt.user; mkdir -p /usr/local/freebsd-update-server/patches/RELEASE-7.1/ >@@ -507,8 +530,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > we can see it is called SA-09:12.bind. After > downloading the file, it is required to rename the file to an > appropriate patch level. It is suggested to keep this inline with >- official &os; patch levels, however, this is just a suggestion. >- For this build, let us follow the brief and call this >+ official &os; patch levels, but you are free to choose any name you prefer. >+ For this build, let us follow the currently established practice of &os; and call this > p7. Rename the file: > > >@@ -524,9 +547,11 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st > > When running a patch level build, we are assuming that previous > patches are in place. When a patch build is run, it will run all >- patches less than or equal to the number specified. Beyond this, >- you will have to take appropriate measures to verify authenticity >- of the patch. >+ patches less than or equal to the number specified. >+ >+ It is up to the administrator of the freebsd-update >+ server to take appropriate measures to verify the authenticity of >+ every patch. > > You can also add your own patches to any build. Use the number > zero, or any other number. >@@ -642,7 +667,7 @@ files to confirm that they look sensible > # sh -e approve.sh amd64 7.1-RELEASE > to sign the build. > >- Follow the same process as noted before for appoving a build. >+ Follow the same process as noted before for approving a build: > > &prompt.root; sh -e scripts/approve.sh amd64 7.1-RELEASE > Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.1-RELEASE >@@ -657,7 +682,7 @@ ready to be uploaded. Remember to run > to unmount the decrypted key once you have finished signing all > the new builds. > >- After approving the build, upload the software. >+ After approving the build, upload the software: > > > &prompt.root; cd /usr/local/freebsd-update-server >@@ -671,21 +696,40 @@ the new builds. > > Tips > >+ >+ > > > If you build your own release using the native >- make release, >+ make release procedure, the > freebsd-update-server code will work >+ > from your release. As an example, you may build a release without > ports or documentation and add a custom kernel. After removing > functionality pertaining to the documentation subroutine and >- altering the buildworld() subroutine in >- scripts/build.subr the >+ altering the buildworld() subroutine in >+ scripts/build.subr, the > freebsd-update-code will successfully > build update code on this release. > > > >+ > Add make -j NUMBER > to scripts/build.subr to speed up processing. > Adding flags to anything other than >@@ -695,6 +739,10 @@ the new builds. > > > >+ > Create a firewall rule to block outgoing RST packets. Due to > a bug noted url="http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2009-04/msg00365.html">in this posting, -- i am a mutthead --x4pBfXISqBoDm8sr Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="freebsd-update.6d8923ea5ca3.diff" I have made my notes and commments inline, and would rather not create another patch based on my thoughts. diff -r 6d8923ea5ca3 en_US.ISO8859-1/articles/freebsd-update-server/article.sgml --- a/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 10:05:02 2009 +0200 +++ b/en_US.ISO8859-1/articles/freebsd-update-server/article.sgml Fri Nov 20 17:48:23 2009 +0200 @@ -80,11 +80,11 @@ - An Apache - web server, with over half of the space required for the - build. For instance, our test builds total 4 GB, and the - webserver space needed to distribute updates is 2.6 GB. + A web server, like Apache, + with over half of the space required for the build. For instance, + our test builds total 4 GB, and the webserver space needed to + distribute updates is 2.6 GB. I agree with this, as Apache is a choice, however there are configuration files that are uploaded as part of the code that are Apache specific. I tried using lighttpd, however RST resets prevented updates. @@ -97,7 +97,8 @@ Configuration: Installation & Setup - Download freebsd-update-server + Download + the freebsd-update-server software. A tarball may be downloaded, or use &man.csup.1; and the projects-all collection. The word "the" really doesn't add any special clarity here, from what I can see. As an English and Journalism major, I was told that the word the is one of the most overused words in the language, and to use it as sparsely as possible. :) @@ -138,19 +139,20 @@ MASTERDIR=update-master.freebsd.orgFTP - This is where the subroutine fetchiso() - declared in scripts/build.subr will contact - the configured source for downloading the &os; ISO. This setting - is not limited to ftp, it may also be - a web (httpd) address as well. - For our purposes, ISO's are on the same server as our internal - http server that will be serving updates. The software has been - configured to look in that location. For this setup, we have to - alter the routine to fetch the ISO. By copying the source - build.subr to - scripts/RELEASE/ARCHITECTURE/build.subr this - file will be sourced instead of the released source for - build.subr. + This is the location where ISO images are downloaded from (by + the fetchiso() subroutine + of scripts/build.subr). The location + configured here is not limited to FTP URIs. Any URI scheme + supported by the standard &man.fetch.1; utility should work fine. + In our own setup the ISO images are on the same internal http + server that will be serving the updates. + + Customizations to the fetchiso() code can + be installed by copying the + default build.subr script to the release- and + architecture-specific area + at scripts/RELEASE/ARCHITECTURE/build.subr + and applying local edits. I agree with this change. A symlink can also work. @@ -158,9 +160,8 @@ MASTERDIR=update-master.freebsd.orgBUILDHOSTNAME - Host where software will be built. Coincidentally, this - information will be displayed on updated systems when - issuing: + The name of the build host. This information will be + displayed on updated systems when issuing: I agree with this. &prompt.user; uname -v @@ -170,15 +171,16 @@ MASTERDIR=update-master.freebsd.orgSSHKEY - Key used when uploading data to an update server which will - provide patches or upgrades to clients. A key pair may be - created using ssh-keygen -t dsa. Alteration - of this parameter is not required as stadnard password - authentication through ssh will be - used. + The SSH key for uploading files to + the update server. A key pair can be created by + typing ssh-keygen -t dsa. This parameter is + optional; standard password authentication will be used as a + fallback authentication method when SSHKEY is + not defined. - &man.ssh-keygen.1; has more detailed information in creating - a key pair. + &man.ssh-keygen.1; has more detailed information + about SSH and the appropriate steps for + creating and using one. @@ -186,7 +188,8 @@ MASTERDIR=update-master.freebsd.orgMASTERACCT - Account that files are uploaded to on remote system. + Account that files are uploaded to on the update + server. @@ -194,54 +197,68 @@ MASTERDIR=update-master.freebsd.orgMASTERDIR - Directory where files are uploaded to on remote - system. + Directory where files are uploaded to on the update + server. No need for the word "the," here. Good change. - Now that build directives are set, the installation files are - configured for a build. For this example, we will use - RELEASE-7.2 under amd64 - architecture. Configuration files for &i386; - architecture are available with downloaded source. + The default build.conf file shipped with + the freebsd-update-server sources are + suitable for building &i386; releases of &os;. As an example of + building an update server for other architectures we will show in + the following paragraphs the configuration changes needed for an + AMD64 update server: As note before, these are build directives, and other directives could be seen as release directives. - Create build environment directory under scripts/RELEASE-7.2/amd64. + + + Create a build environment for AMD64: - - &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 - + + &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 + + I like the "steps." Wasn't aware of this function. - This is the build.conf file that should be - placed in the directory just created. + - # SHA256 hash of RELEASE disc1.iso image. + Install a build.conf file in the + newly created build directory. The build configuration + options for &os; 7.2-RELEASE on AMD64 should be similar + to: + + # SHA256 hash of RELEASE disc1.iso image. export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 # Components of the world, source, and kernels export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32" -export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ - lib libexec release rescue sbin secure share sys tools \ - ubin usbin cddl" +export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ + lib libexec release rescue sbin secure share sys tools \ + ubin usbin cddl" export KERNELPARTS="generic" # EOL date export EOL=1275289200 - - To generate the "End of Life" number for - build.conf, refer to the "Estimated EOL" posted - on the &os; - Security Website. Based on this date, you can issue - date -j -f '%Y%m%d-%H%M%S' '20090401-000000' +%s, - and substitute actual date parameters for those stated by - &os;. + + To generate the "End of Life" number for + build.conf, refer to the "Estimated + EOL" posted on + the &os; + Security Website. You can derive the value + of EOL from the date listed on the web + site, using the &man.date.1; utility, e.g.: - The &man.sha256.1; hash key for the desired release, is - published within the respective release announcement . - + &prompt.user; date -j -f '%Y%m%d-%H%M%S' '20090401-000000' '+%s' + + + + The &man.sha256.1; hash key for the desired release, is + published within the + respective release + announcement. + + + I like this modification. @@ -273,9 +290,9 @@ enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: - Note down the generated KeyPrint; this value is entered into - /etc/freebsd-update.conf for binary - updates. + Keep a note of the generated key fingerpring. This value is + entered into /etc/freebsd-update.conf for + binary updates. Typo. "fingerprint" At this point, we are ready to stage a build. @@ -330,8 +347,8 @@ world|base|/usr/lib/libalias_ftp.a Then the build of the world is performed again, with world - patches. A more detailed explanation may be found in - scripts/build.subr. + patches. A more detailed explanation may be found + in scripts/build.subr. Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE @@ -411,10 +428,7 @@ to sign the release. file named USAGE. Execute scripts/approve.sh, as directed. This will sign the release, and move components into a staging area suitable for - uploading. It is important to make sure that your key is mounted - during this process. A simple df will show if it - is mounted. If not mounted, mount the key with the passphrase supplied - when creating it earlier. + uploading. Your build will not upload if it isn't approved, and it can't be approved if it isn't signed. It can't be signed if your key is not mounted. The key is essential for the building process. &prompt.root; cd /usr/local/freebsd-update-server @@ -436,6 +450,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st &prompt.root; sh scripts/upload.sh amd64 RELEASE-7.2 + It may work, however there are specific files uploaded during the upload process that are for Apache. If the functionality can be adapted into other http servers, the end-user may choose to use another webserver. I experimented with lighttpd, however it did not work. Colin notes that Apache handles memory better in the source. The uploaded files will need to be in the DocumentRoot of the webserver in order for updates to be distributed. For further explanation, please refer to the @@ -443,6 +459,10 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st url="&url.books.handbook;/network-apache.html">Apache documentation. + Updates for the current release of the &os; system you are updating, and what you want to upgrade to need @@ -453,11 +473,14 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st &os; 7.2-RELEASE. I re-worded this for clarity, however I thought it was a good place to put it. Right after a build. If you want to update from 7.1 to 7.2, you will need to have build code on your update server for both releases. + KeyPrint is the key fingerprint that needs to be updated in /etc/freebsd-update.conf in order to install updates. Update client's KeyPrint and ServerName in /etc/freebsd-update.conf, and perform updates as instructed in the &os; Update + instructions in the Handbook. For reference, here is the entire run of Building a Patch - In the event a Every time a security advisory - is posted, a patch update can be built. + is announced, a patch update can be built. For this example, we will be using 7.1-RELEASE. @@ -487,8 +510,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st - Create patch directory under - /usr/local/freebsd-update-server/patches/ for the respective release. + Create the patch directory of the respective release + under /usr/local/freebsd-update-server/patches/. &prompt.user; mkdir -p /usr/local/freebsd-update-server/patches/RELEASE-7.1/ @@ -507,8 +530,8 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st we can see it is called SA-09:12.bind. After downloading the file, it is required to rename the file to an appropriate patch level. It is suggested to keep this inline with - official &os; patch levels, however, this is just a suggestion. - For this build, let us follow the brief and call this + official &os; patch levels, but you are free to choose any name you prefer. + For this build, let us follow the currently established practice of &os; and call this p7. Rename the file: @@ -524,9 +547,11 @@ Wed Aug 26 12:50:07 PDT 2009 Cleaning st When running a patch level build, we are assuming that previous patches are in place. When a patch build is run, it will run all - patches less than or equal to the number specified. Beyond this, - you will have to take appropriate measures to verify authenticity - of the patch. + patches less than or equal to the number specified. + + It is up to the administrator of the freebsd-update + server to take appropriate measures to verify the authenticity of + every patch. You can also add your own patches to any build. Use the number zero, or any other number. @@ -642,7 +667,7 @@ files to confirm that they look sensible # sh -e approve.sh amd64 7.1-RELEASE to sign the build. - Follow the same process as noted before for appoving a build. + Follow the same process as noted before for approving a build: &prompt.root; sh -e scripts/approve.sh amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.1-RELEASE @@ -657,7 +682,7 @@ ready to be uploaded. Remember to run to unmount the decrypted key once you have finished signing all the new builds. - After approving the build, upload the software. + After approving the build, upload the software: &prompt.root; cd /usr/local/freebsd-update-server @@ -671,21 +696,40 @@ the new builds. Tips + Agreed. + I agree with the above. If you build your own release using the native - make release, + make release procedure, the freebsd-update-server code will work + from your release. As an example, you may build a release without ports or documentation and add a custom kernel. After removing functionality pertaining to the documentation subroutine and - altering the buildworld() subroutine in - scripts/build.subr the + altering the buildworld() subroutine in + scripts/build.subr, the freebsd-update-code will successfully build update code on this release. This allows for you to maintain an update server with a custom and generic kernel, and a modified world, if you wish. + Add make -j NUMBER to scripts/build.subr to speed up processing. Adding flags to anything other than @@ -695,6 +739,10 @@ the new builds. It is safe to add it to the buildworld and obj process, but not others. Update code will fail on anything besides this. + Updates will fail if this isn't put in place. Create a firewall rule to block outgoing RST packets. Due to a bug noted in this posting, --x4pBfXISqBoDm8sr--