From owner-freebsd-announce Tue Feb 9 18:30:07 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA21802 for freebsd-announce-outgoing; Tue, 9 Feb 1999 18:30:07 -0800 (PST) (envelope-from owner-freebsd-announce@FreeBSD.ORG) Received: from wall.polstra.com (rtrwan160.accessone.com [206.213.115.74]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA21714 for ; Tue, 9 Feb 1999 18:29:59 -0800 (PST) (envelope-from jdp@polstra.com) Received: from vashon.polstra.com (vashon.polstra.com [206.213.73.13]) by wall.polstra.com (8.9.1/8.9.1) with ESMTP id SAA27307 for ; Tue, 9 Feb 1999 18:29:58 -0800 (PST) (envelope-from jdp@polstra.com) Received: (from jdp@localhost) by vashon.polstra.com (8.9.1/8.9.1) id SAA52058 for freebsd-announce@freebsd.org; Tue, 9 Feb 1999 18:29:57 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Tue, 09 Feb 1999 18:29:57 -0800 (PST) Organization: Polstra & Co., Inc. From: John Polstra To: freebsd-announce@FreeBSD.ORG Subject: CVSup 16.0 Is Now Available Sender: owner-freebsd-announce@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Announcing CVSup 16.0 --------------------- Release 16.0 of CVSup, the CVS-aware network file distribution system, is now available. What Has Changed Since the Previous Release? -------------------------------------------- Changes marked with "(*)" were also included in release 15.5, a Linux portability release that saw only limited distribution. Implemented strong authentication to control access to the server when desired. It uses a challenge-response protocol which is immune to packet sniffing and replay attacks. No passwords are sent over the network in either direction. Both the client and the server can independently verify the identities of each other. A new "-a" option on the client allows it to force the server to prove its identity. There is a new "cvpasswd" utility which is used for creating scrambled passwords for the server's authentication database. Added a new "-s" client option to greatly reduce the client's disk load, at significant cost in safety. The option suppresses the usual stat(2) calls that check the status of each file against what is recorded in the list (checkouts) file. It makes updates run quite a bit faster, but it is not safe if there is any possibility that the client's files might be modified locally in between CVSup updates. See the caveats in cvsup(1) for more details. (*) Extended the update algorithm for RCS files to handle arbitrary "newphrases" attached to deltas. (See rcsfile(5) if you don't know what "newphrases" means.) Newer versions of CVS use them when the "PreservePermissions" feature is turned on. This caused previous versions of CVSup to get checksum mismatches and perform time-consuming fixups. Note, checkout mode doesn't actually honor these newphrases yet; but CVS mode mirrors them properly now. Added a "umask=nnn" option in cvsupfiles to specify the umask to use when updating a collection. It can be different for each collection if desired. Made CVSup preserve file permissions more strictly. It used to ignore the mode bits masked off by the umask. That was wrong. Now it enforces (client mode) == ((server mode) & ~(client umask)) exactly. As a result, you may see a few zillion "SetAttrs" messages the first time you update using this version of CVSup. This is nothing to be alarmed about. Note: if "preserve" is in your cvsupfile, the effective umask is forced to 0. That causes the client's file permissions to be set exactly equal to those on the server. Added a new "-D" client option to skip all updates except file deletions. This is useful if the client's disk space is very limited. You can make one run with "-D" to free up as much space as possible. Then make another run without "-D" to update the remaining files. If files or directories have been renamed on the server, this ensures that all of the old versions are deleted before any of the new ones are created. NOTE: This option is not implemented yet for checkout mode. (*) Fixed a signal race in the server. Under extremely rare conditions, the race caused the master server to hang in an infinite loop. (*) Fixed the client so that it no longer dies if it fails to delete a directory. The usual cause is that the user has put an extraneous file in the directory, making it non-empty even though CVSup thinks it should be empty. Now the client issues a warning but continues on. (*) Fixed a bug that caused the client to loop if a collection contained a top-level directory with a one-character name. Fixed a bug that could cause the client to die if its version of an RCS file was in fact completely unrelated to the one on the server even though they had the same name. Now that situation is detected, and CVSup simply replaces the file wholesale rather than attempting to edit it. Fixed the client to clean up better if it is killed by a signal. Added Joseph Koshy's "cvsupchk" script, which finds extra unwanted files that may be littering your source tree. Look for it in the new "contrib" subdirectory of the release. Added Wolfram Schneider's "cvsup2httplog" script. It converts a cvsupd log file to standard httpd log file format, so that it can be analyzed by tools such as webalizer. You'll find it in the new "contrib" subdirectory. Added a "cvsupwho" script, inspired by Garrett Wollman, to list the currently active clients of a CVSup server. Look for it in the new "contrib" subdirectory. Changed the default connection mode to multiplexed, since it can handle all situations that the other modes can handle. Deprecated active mode, passive mode, and SOCKS mode. Added a setsockopt(... SO_REUSEADDR ...) call in the server just before the point where it tries to bind the listen socket on start-up. This allows the server to be re-started immediately after you have killed it, even if some of the old connections are still lingering. Formerly, a "Port in use" error would be reported, and you would have no choice but to wait until the old connections timed out. Added network I/O statistics to the server's final log message for each client. Also added logging of I/O statistics on a per-collection basis. The per-collection messages are at "info" priority, while the others are at "notice" level. Made some portability fixes for Linux and Digital Unix. The software now builds and runs whether using SRC Modula-3 or the newer PM3 release. Added pre-formatted manual pages to the source distribution. They are named "*.cat". Where to Get CVSup ------------------ CVSup is free software. It is available from: ftp://ftp.FreeBSD.org/pub/FreeBSD/development/CVSup/ and from all of the many FreeBSD FTP mirrors. You can find a complete list of them in the FreeBSD Handbook at: http://www.freebsd.org/handbook/mirrors.html CVSup is also available from the author's FTP server: ftp://ftp.polstra.com/pub/FreeBSD/CVSup/ Please avoid this server if possible. It has a wimpy link to the Internet. Full sources as well as FreeBSD binaries are available: cvsup-bin-16.0.tar.gz FreeBSD static binaries for the client (includes GUI) cvsup.nogui-bin-16.0.tar.gz FreeBSD static binaries for the client (no GUI) cvsupd-bin-16.0.tar.gz FreeBSD static binaries for the server cvsup-16.0.tar.gz Sources ** The MD5 file signatures for these files are: MD5 (cvsup-bin-16.0.tar.gz) = 57c25981d3c1d82a79b9ae18aaea715b MD5 (cvsup.nogui-bin-16.0.tar.gz) = 02497824fb097c6fe01424106b073f89 MD5 (cvsupd-bin-16.0.tar.gz) = c03347b4c0dbfc2a911cc1b5a045c3bb MD5 (cvsup-16.0.tar.gz) = bad884ccbd4ed129d360487c87c089a4 Binaries for other platforms are also available. See the CVSup FAQ for their locations: http://www.polstra.com/projects/freeware/CVSup/ An updated port will appear soon in the "net" category of the FreeBSD ports and packages collection: http://www.freebsd.org/ports/ If you want SOCKS support, you must also install the "modula-3-socks" port or package. You'll find it in the "lang" category of the FreeBSD ports and packages collection at the URL above. SOCKS is supported only under FreeBSD, and only with dynamically linked executables. The static binary distributions do not support SOCKS. ** If you wish to build CVSup from the sources, be sure to read the discussion further on in this announcement. Compatibility with Previous Releases ------------------------------------ This version is believed to interoperate properly with all earlier public releases of CVSup. What Is CVSup? -------------- CVSup is a software package for distributing and updating collections of files across a network. It can efficiently and accurately mirror all types of Unix files, including sources, binaries, hard links, symbolic links, and even device nodes. CVSup's streaming communication protocol and multithreaded architecture make it most likely the fastest mirroring tool in existence today. In addition to being a great general-purpose mirroring tool, CVSup includes special features and optimizations specifically tailored to CVS repositories. By taking advantage of the special properties of the files contained in CVS repositories, CVSup is able to perform updates much faster than traditional systems. CVSup parses and understands the RCS files making up a CVS repository. When updates occur, CVSup extracts new deltas directly from the RCS files on the server and edits them into the client's RCS files. Likewise, CVSup notes the addition of new symbolic tags to the files on the server and sends only the new tags to the client. CVSup is able to merge new deltas and tags from the server with deltas and tags added locally on the client machine. This makes it possible for the client to check local modifications into his repository without their being obliterated by subsequent updates from the server. In addition to distributing the RCS files themselves, CVSup is able to distribute specific checked-out versions. The client can specify a symbolic tag, a date, or both and CVSup will extract the appropriate versions from the server's CVS repository. Checked-out versions do not need to be stored on the server, since CVSup can extract any version directly from the CVS repository. If the client has an existing checked-out tree, CVSup will apply the appropriate edits to update the tree or transform it into the requested version. Only the differences between the existing version and the desired version are sent across the network. To update non-RCS files, CVSup uses the highly efficient rsync algorithm, developed by Andrew Tridgell and Paul Mackerras. CVSup uses lightweight processes (threads) to implement a streaming protocol across the network. This completely eliminates the delays associated with the lock-step, request-reply form of communication used by many existing protocols, such as sup and NNTP. Information is transferred at the full available speed of the network in both directions at once. Network latency and server response delays are rendered practically irrelevant. CVSup uses the "zlib" compression package to optionally compress all communications. This provides an additional 65-75% compression, on top of the diff-based compression already built into CVSup. For efficiency, all processing is built into the CVSup package itself. Neither the client nor the server execute any other programs. For further information about how CVSup works, see the (somewhat out of date) "Blurb" document in the CVSup distribution. Using CVSup to Maintain FreeBSD Sources --------------------------------------- CVSup servers for the FreeBSD sources are currently running at about 35 mirror sites around the world. For an up-to-date list of them, see: http://www.freebsd.org/handbook/mirrors-cvsup.html Using CVSup, you can easily receive or update any of the standard FreeBSD source releases, namely, "cvs", "current", and "stable". The manual page for cvsup(1) describes how to do that. For more detailed instructions, see the section on CVSup in the FreeBSD Handbook: http://www.FreeBSD.org/handbook/cvsup.html Building CVSup from the Sources ------------------------------- CVSup is written in Modula-3, a modern, compiled, object-oriented language. Modula-3 integrates threads, exceptions, and garbage collection, providing an ideal vehicle for this sort of application. Without Modula-3, CVSup would almost certainly not exist today. If you wish to build CVSup from the sources, you will first need to install the free Modula-3 compiler and runtime libraries. This release of CVSup has been tested using both SRC Modula-3 release 3.6 and the PM3-1.1.9 release. They are available from the following locations: SRC: http://www.research.digital.com/SRC/modula-3/html/srcm3.html PM3: http://m3.polymtl.ca/m3/ The PM3 release is more actively maintained, and it includes fixes for a number of bugs that are present in SRC Modula-3. Patches to fix the bugs are available from the SRC web site, but they must be applied manually. The PM3 release is available as Linux RPMs as well as in source form. A FreeBSD port of SRC Modula-3 (including the necessary patches) is available in the FreeBSD ports collection, in "lang/modula-3". The corresponding package is, of course, available in the packages collection. You will also need version 1.0.4 or later of the "zlib" library. In FreeBSD-2.1.6 and later releases, this library has been incorporated into the system sources, in "src/lib/libz". Prior to that, a FreeBSD port was available in "devel/libz" of the FreeBSD ports collection. For other sources of this library, see the "Install" file. Do not try to use versions earlier than 1.0.4. To build the entire system from source under FreeBSD, simply type "make" in the top-level directory. (That's the directory that has sub-directories named "client" and "server", among others.) To force the executables to be statically linked, add M3FLAGS=-DSTATIC to the command line. To build the client without the GUI (e.g., if you don't have the X Window System installed on your machine), add M3FLAGS=-DNOGUI To do both of these things, use quotes like this: M3FLAGS="-DSTATIC -DNOGUI" The Makefiles should work on any reasonable Unix system. If you have trouble with them, it is easy to build the components manually. Simply chdir into each of the following subdirectories in the given order and type "m3build": suptcp suplib client server cvpasswd To build statically linked executables, add "-DSTATIC" to each "m3build" command. To build the client without the GUI, add "-DNOGUI". Notes for Specific Platforms ---------------------------- The following hints have been found to work. They don't exclude other possibilities, of course. FreeBSD: Use the SRC compiler from the FreeBSD ports collection. Linux: Use the PM3 RPMs (version 1.1.9 or later). Solaris: Use the SRC compiler (the "SOLgnu" target). Be sure to apply these patches first: http://www.research.digital.com/SRC/modula-3/html/patches36.html#sol2exec http://www.research.digital.com/SRC/modula-3/html/patches36.html#libaio Digital Unix: Use the SRC compiler (the "ALPHA_OSF" target). Be sure to apply this patch first: http://www.research.digital.com/SRC/modula-3/html/patches36.html#randomReal64 When you run cvsup or cvsupd, you must add the argument "@M3nopreemption" to the command line. NetBSD, OpenBSD, BSD/OS: Use the FreeBSD static binaries. When you run cvsup or cvsupd, you must add the argument "@M3novm" to the command line. Portability Issues ------------------ I intend for CVSup to be portable to most POSIX systems. Earlier releases have been run on a number of different platforms, including FreeBSD, Linux, HP-UX, SunOS, Solaris, and DEC OSF/1 ALPHA. In the current release I have attempted to increase rather than decrease portability. Anybody who succeeds in porting CVSup to other systems is encouraged to send his changes to . As long as the changes are reasonably palatable, they will be incorporated into future CVSup releases. CVSup uses several POSIX-specific functions which may make it more of an effort to port the package to non-POSIX systems such as Win32. These functions include mmap, fork, syslog, stat, and chmod, among others. Status of this Release ---------------------- CVSup was first released publicly in August of 1996. Since then it has seen heavy use, and it has been quite stable. Like all software, though, it is not perfect. Please be prepared to find bugs -- without a doubt, there are some. Problems? --------- Please report bugs to . For More Information -------------------- http://www.polstra.com/projects/freeware/CVSup/ John Polstra, Copyright 1996, 1997, 1998, 1999 John D. Polstra $Id: Announce,v 1.46 1999/02/10 01:15:56 jdp Exp $ This is the moderated mailing list freebsd-announce. The list contains announcements of new FreeBSD capabilities, important events and project milestones. See also the FreeBSD Web pages at http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-announce" in the body of the message