From owner-freebsd-hackers@FreeBSD.ORG  Sun Oct  7 17:51:55 2012
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id DE47B1065728
	for <freebsd-hackers@freebsd.org>; Sun,  7 Oct 2012 17:51:55 +0000 (UTC)
	(envelope-from tim@kientzle.com)
Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net
	[99.115.135.74])
	by mx1.freebsd.org (Postfix) with ESMTP id B86A48FC1C
	for <freebsd-hackers@freebsd.org>; Sun,  7 Oct 2012 17:51:55 +0000 (UTC)
Received: (from root@localhost)
	by monday.kientzle.com (8.14.4/8.14.4) id q97HpkUQ017185;
	Sun, 7 Oct 2012 17:51:46 GMT (envelope-from tim@kientzle.com)
Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65])
	by kientzle.com with SMTP id 64ke244kk3upjc7r2g7je68tq6;
	Sun, 07 Oct 2012 17:51:46 +0000 (UTC)
	(envelope-from tim@kientzle.com)
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: text/plain; charset=us-ascii
From: Tim Kientzle <tim@kientzle.com>
In-Reply-To: <alpine.BSF.2.00.1210071859430.15957@wojtek.tensor.gdynia.pl>
Date: Sun, 7 Oct 2012 10:52:27 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <324B736D-8961-4E44-A212-2ECF3E60F2A0@kientzle.com>
References: <5069C9FC.6020400@brandonfa.lk>
	<alpine.BSF.2.00.1210071859430.15957@wojtek.tensor.gdynia.pl>
To: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl>
X-Mailer: Apple Mail (2.1278)
Cc: freebsd-hackers@freebsd.org, Brandon Falk <bfalk_bsd@brandonfa.lk>
Subject: Re: SMP Version of tar
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, 
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 07 Oct 2012 17:51:56 -0000

On Oct 7, 2012, at 10:00 AM, Wojciech Puchar wrote:

>> I would be willing to work on a SMP version of tar (initially just =
gzip or something).
>>=20
>> I don't have the best experience in compression, and how to =
multi-thread it, but I think I would be able to learn and help out.
>=20
> gzip cannot - it is single stream.

gunzip commutes with cat, so gzip
compression can be multi-threaded
by compressing separate blocks and
concatenating the result.

For proof, look at Mark Adler's pigz
program, which does exactly this.

GZip decompression is admittedly trickier.


> bzip2 - no idea

bzip2 is block oriented and can be multi-threaded for both compression =
and decompression.


Tim