Date: Wed, 31 May 2017 02:27:47 +0000 From: meta <meta@vmeta.jp> To: "Chad J. Milios" <milios@ccsys.com> Cc: freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: Is it possible to use more than 1 CPUs during pkg create? Message-ID: <0101015c5c5498c3-84377171-9e30-40da-a58b-1c4af755f602-000000@us-west-2.amazonses.com> In-Reply-To: <D75F743F-D5F9-42EB-BE24-7A7A3C14B96E@ccsys.com> References: <0101015c58b3dc58-1b936a1a-8412-472b-a257-c436a40e89f2-000000@us-west-2.amazonses.com> <D75F743F-D5F9-42EB-BE24-7A7A3C14B96E@ccsys.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, May 30, 2017 at 06:55:02AM -0400, Chad J. Milios wrote: > Threading forfeits a bit of xz’s great compression ratio. An xz file created with -T greater than 1 will necessarily be some amount larger. Operating with just one thread achieves the maximal compression ratio. (The multiple compression threads fail to share the dictionaries they are currently building.) > Yes, compressing with one thread achieves the best ratio. However, it is really a little bit. I created a package of x11-fonts as example of a big package. Its size is 813230580 bytes. Then I recompressed the package with -T0 option, the archive is just 1% larger. # make -C /usr/ports/x11-fonts/noto package $ ls noto* -rw-r--r-- 1 meta wheel 813230580 May 30 21:30 noto-1.0.5_1.txz $ zcat noto-1.0.5_txz | xz -T0 -c > noto-1.0.5_1.txz.T0 $ ls noto* -rw-r--r-- 1 meta wheel 813230580 May 30 21:30 noto-1.0.5_1.txz -rw-r--r-- 1 meta wheel 821921768 May 30 21:45 noto-1.0.5_1.txz.T0 I understand the merit of compressing package in single thread distributed by pkg.freebsd.org. However, I'd rather save time than storage or network bandwidth. I want to make `make package` faster on my host. The build can use multi cores but pkg create process uses only one so package stage is the bottle neck. > When you are transferring GBytes to yourself once, use -T0. When you are compressing something one time that hundreds or thousands of people will potentially download and/or store, use -T1 (the default). Do you mean I can use -T0 during pkg create? AFAIK it is not possible but if possible, how? -- `whois vmeta.jp | nkf -w` meta <meta@vmeta.jp>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0101015c5c5498c3-84377171-9e30-40da-a58b-1c4af755f602-000000>