From owner-freebsd-arch@freebsd.org Sat May 4 20:53:08 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53B3915989D3 for ; Sat, 4 May 2019 20:53:08 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from mail01.asahi-net.or.jp (mail01.asahi-net.or.jp [202.224.55.13]) by mx1.freebsd.org (Postfix) with ESMTP id 64B37710DE; Sat, 4 May 2019 20:53:03 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from em250.advok.com (ool-435706a0.dyn.optonline.net [67.87.6.160]) (Authenticated sender: NR2Y-OOT) by mail01.asahi-net.or.jp (Postfix) with ESMTPSA id E0D8F11A50C; Sun, 5 May 2019 05:52:48 +0900 (JST) Date: Sat, 4 May 2019 16:52:59 -0400 From: Yoshihiro Ota To: freebsd-arch@freebsd.org Cc: delphij@freebsd.org, kib@freebsd.org, imp@freebsd.org, jmg@freebsd.org, markj@freebsd.org, sobomax@freebsd.org Subject: Re: Zlib Upgrade in kernel Message-Id: <20190504165259.68a44a451dbf0072814d923d@j.email.ne.jp> In-Reply-To: <20190326000956.83bd72411661511656ea1b5d@j.email.ne.jp> References: <20190326000956.83bd72411661511656ea1b5d@j.email.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; i386-portbld-freebsd12.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 64B37710DE X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of ota@j.email.ne.jp designates 202.224.55.13 as permitted sender) smtp.mailfrom=ota@j.email.ne.jp X-Spamd-Result: default: False [-0.48 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:202.224.55.0/24]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[email.ne.jp]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.77)[-0.771,0]; IP_SCORE(0.10)[asn: 4685(0.54), country: JP(-0.06)]; NEURAL_SPAM_SHORT(0.60)[0.601,0]; MX_GOOD(-0.01)[sbmx.asahi-net.or.jp]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_MEDIUM(-0.60)[-0.596,0]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[13.55.224.202.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:4685, ipnet:202.224.32.0/19, country:JP]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[160.6.87.67.zen.spamhaus.org : 127.0.0.10] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 20:53:08 -0000 Hello, After I posted zlib upgrade effort about a month ago, I got feedbacks and went through iterations of changes. https://reviews.freebsd.org/D19706 is ready for another review. I have few unfamilar peices. If anyone can comment, that will be great. Q.1 Can zfs boot loader start from gzipped file system? Q.2 Does anyone run gzipped a.out format on the latest systems? I was able to run a.out and gzipped a.out on FreeBSD 2.2.9. I ran a.out grabbed fron 2.2.9 on 12-STABLE and 13-CURRENT but gzipped a.out didn't start. Q.3 Does anyone have/use arm5 with compressed kernel? These are short summaries of zlib and unzip related code at the phabricator: zlib - FreeBSD adjusted kernel zlib is moved into netgraph. - contrib/zlib is moved to sys/contrib/zlib and adjusted in both user land and kernel. - zlib copy in zfs is removed and uses sys/contrib/zlib. - CTF is modified to use uncompress(). unzip - Cleaned up sys/mips. - arm5 is untouched. - gzipped a.out execution is untouched although it is broken in 12-stable/13-current. - kgzip and kgzldr are removed. Regards, Hiro On Tue, 26 Mar 2019 00:09:56 -0400 Yoshihiro Ota wrote: > Hello, > > Our zlib in kernel has been quite out-dated. > I looked in and started updating. > > I use contrib/zlib files and sys/contrib/zstd to compile zlib with least amount of changes to > zlib side. I split crc32 functions and also added few compile options to distinguish zlib calls. > I have to relocate current one as netgraph uses FreeBSD specific implementation. > > I started wondering if I should move contrib/zlib to sys/contrib/zlib and wanted to get some feed-dated. > I looked in and started updating. > > I use contrib/zlib files and sys/contrib/zstd to compile zlib with least amount of changes to > zlib side. I split crc32 functions and also added few compile options to distinguish zlib calls. > I have to relocate current one as netgraph uses FreeBSD specific implementation. > > I started wondering if I should move contrib/zlib to sys/contrib/zlib and wanted to get some feed > back on some of coding policies. > > I created https://reviews.freebsd.org/D19706, and noted in new sys/sys/zlib.h: > /* > * #1 - sys/crc32.h is split out of sys/libkern.h to avoid conflicts > * between zlib's crc32 and system crc32. > * #2 - Use contrib/zstd/lib/freebsd's stdlib compatible includes. > * #3 - ZLIB_C is created con/kern.pre.mk to share compile paths. > * -I contrib/zlib > * -I sys/contrib/zstd/lib/freebsd > * #4 - sys/zlib.h defines these. > * Z_PREIFX > * NO_GZIP > * Z_SOLO - this is for libkern/zlib.c and not needed for clients of zlib. > * #5 - opencryptodeflate.c > * zfs.state->dummy is an address and doesn't seem to be useful. > * Its DTRACE probe is removed. > * #6 - netgraph/deflate.c needs and uses FreeBSD enhancements to zlib. > * Moved sys/zlib.h to netgraph/zlib.h, sys/libkern/zlib.c to > * netgraph/zlib.c, and netgraph/deflate.c includes netgraph/zlib.c > * to compile as a part of deflate.c. > */ > > Regards, > Hiro > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@freebsd.org Sat May 4 21:10:36 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 433B71598E8F for ; Sat, 4 May 2019 21:10:36 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail106.syd.optusnet.com.au (mail106.syd.optusnet.com.au [211.29.132.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5B8D8716F7; Sat, 4 May 2019 21:10:28 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail106.syd.optusnet.com.au (Postfix) with ESMTPS id 57A8A3D9245; Sun, 5 May 2019 07:10:16 +1000 (AEST) Date: Sun, 5 May 2019 07:10:15 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Yoshihiro Ota cc: freebsd-arch@freebsd.org, sobomax@freebsd.org, markj@freebsd.org, kib@freebsd.org, jmg@freebsd.org, delphij@freebsd.org, imp@freebsd.org Subject: Re: Zlib Upgrade in kernel In-Reply-To: <20190504165259.68a44a451dbf0072814d923d@j.email.ne.jp> Message-ID: <20190505070145.Y11459@besplex.bde.org> References: <20190326000956.83bd72411661511656ea1b5d@j.email.ne.jp> <20190504165259.68a44a451dbf0072814d923d@j.email.ne.jp> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=FNpr/6gs c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=_evYC6HVPZ0pAdLb7W0A:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 X-Rspamd-Queue-Id: 5B8D8716F7 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of brde@optusnet.com.au designates 211.29.132.42 as permitted sender) smtp.mailfrom=brde@optusnet.com.au X-Spamd-Result: default: False [-5.83 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:211.29.132.0/23]; FREEMAIL_FROM(0.00)[optusnet.com.au]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[optusnet.com.au]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[extmail.optusnet.com.au]; NEURAL_HAM_SHORT(-0.74)[-0.738,0]; RCPT_COUNT_SEVEN(0.00)[8]; IP_SCORE(-2.78)[ip: (-7.83), ipnet: 211.28.0.0/14(-3.37), asn: 4804(-2.69), country: AU(-0.01)]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[42.132.29.211.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[optusnet.com.au]; ASN(0.00)[asn:4804, ipnet:211.28.0.0/14, country:AU]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 21:10:36 -0000 On Sat, 4 May 2019, Yoshihiro Ota wrote: > After I posted zlib upgrade effort about a month ago, I got feedbacks and went through iterations of changes. > https://reviews.freebsd.org/D19706 is ready for another review. > > I have few unfamilar peices. If anyone can comment, that will be great. > > ... > Q.2 Does anyone run gzipped a.out format on the latest systems? > I was able to run a.out and gzipped a.out on FreeBSD 2.2.9. > I ran a.out grabbed fron 2.2.9 on 12-STABLE and 13-CURRENT but gzipped a.out didn't start. Seems unlikely. imgact_gzip only ever worked for aout, and has been broken for that for many years. Last time I looked (in 2017), the first failure in imgact_gzip.c was in vm_mmap(exec_map, ...) last changed in r231885 in 2012. This failed with error 11 (EDEADLK). I use old aout utilities often, but don't want gzip (or zlib) in the kernel for aout (or anything). Bruce