Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 31 May 2017 02:27:47 +0000
From:      meta <meta@vmeta.jp>
To:        "Chad J. Milios" <milios@ccsys.com>
Cc:        freebsd-questions <freebsd-questions@freebsd.org>
Subject:   Re: Is it possible to use more than 1 CPUs during pkg create?
Message-ID:  <0101015c5c5498c3-84377171-9e30-40da-a58b-1c4af755f602-000000@us-west-2.amazonses.com>
In-Reply-To: <D75F743F-D5F9-42EB-BE24-7A7A3C14B96E@ccsys.com>
References:  <0101015c58b3dc58-1b936a1a-8412-472b-a257-c436a40e89f2-000000@us-west-2.amazonses.com> <D75F743F-D5F9-42EB-BE24-7A7A3C14B96E@ccsys.com>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
On Tue, May 30, 2017 at 06:55:02AM -0400, Chad J. Milios wrote:
> Threading forfeits a bit of xz’s great compression ratio. An xz file created with -T greater than 1 will necessarily be some amount larger. Operating with just one thread achieves the maximal compression ratio. (The multiple compression threads fail to share the dictionaries they are currently building.)
> 

Yes, compressing with one thread achieves the best ratio. However, it is
really a little bit.

I created a package of x11-fonts as example of a big package. Its size
is 813230580 bytes. Then I recompressed the package with -T0 option,
the archive is just 1% larger.

# make -C /usr/ports/x11-fonts/noto package
$ ls noto*
-rw-r--r--  1 meta  wheel  813230580 May 30 21:30 noto-1.0.5_1.txz

$ zcat noto-1.0.5_txz | xz -T0 -c > noto-1.0.5_1.txz.T0
$ ls noto*
-rw-r--r--  1 meta  wheel  813230580 May 30 21:30 noto-1.0.5_1.txz
-rw-r--r--  1 meta  wheel  821921768 May 30 21:45 noto-1.0.5_1.txz.T0

I understand the merit of compressing package in single thread distributed
by pkg.freebsd.org. However, I'd rather save time than storage or
network bandwidth. I want to make `make package` faster on my host.
The build can use multi cores but pkg create process uses only one so
package stage is the bottle neck.

> When you are transferring GBytes to yourself once, use -T0. When you are compressing something one time that hundreds or thousands of people will potentially download and/or store, use -T1 (the default).

Do you mean I can use -T0 during pkg create? AFAIK it is not possible
but if possible, how?

-- 
`whois vmeta.jp | nkf -w`
meta <meta@vmeta.jp>



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?0101015c5c5498c3-84377171-9e30-40da-a58b-1c4af755f602-000000>