Date: Sat, 20 Apr 1996 09:42:56 +0200 (MET DST) From: "Christoph P. Kukulies" <kuku@gilberto.physik.rwth-aachen.de> To: terry@lambert.org (Terry Lambert) Cc: kuku@gilberto.physik.rwth-aachen.de, freebsd-current@freefall.freebsd.org Subject: Re: gzipped executables Message-ID: <199604200742.JAA16759@gilberto.physik.rwth-aachen.de> In-Reply-To: <199604191927.MAA08783@phaeton.artisoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > Is anyone working on fixing the broken gzip executable feature in > > -current? > > It's a cache interaction. Pentium caches are written back inside > the cache queue depth, whereas pre-Pentium processors have > immutable cache queues. Jordan said it already and I can confirm it: It doesn't work on my DX-2/66 anymore where it used to work under 2.1.0. I appended an rcslog of /sys/kern/imgact_gzip.c I don't know if I'm enough a kernel expert to find out if it was one of the VM changes that caused it and whether imgact_gzip.c is the culprit anyway. [considerations about P5 L2 cache removed] > > > Terry Lambert > terry@lambert.org > --- > Any opinions in this posting are my own and not those of my present > or previous employers. --Chris Christoph P. U. Kukulies kuku@gil.physik.rwth-aachen.de RCS file: imgact_gzip.c,v Working file: imgact_gzip.c head: 1.20 branch: locks: strict access list: symbolic names: wollman_polling: 1.20.0.2 RELENG_2_1_0_RELEASE: 1.14 RELENG_2_1_0: 1.14.0.4 RELENG_2_1_0_BP: 1.14 RELENG_2_0_5_RELEASE: 1.14 RELENG_2_0_5: 1.14.0.2 RELENG_2_0_5_BP: 1.14 RELENG_2_0_5_ALPHA: 1.13 OLAH_TTCP: 1.10.0.4 RELEASE_2_0: 1.10 BETA_2_0: 1.10 ALPHA_2_0: 1.10.0.2 keyword substitution: kv total revisions: 20; selected revisions: 20 description: ---------------------------- revision 1.20 date: 1996/03/19 15:02:47; author: bde; state: Exp; lines: +2 -2 Fixed unsigned longs that should have been vm_offset_t. vm_offset_t is currently unsigned long but should probably be plain unsigned for i386's to match the choice of minimal types to represent for fixed-width types in Lite2. Anyway, it shouldn't be assumed to be unsigned long. I only fixed the type mismatches that were detected when I changed vm_offset_t to unsigned. Only pointer type mismatches were detected. ---------------------------- revision 1.19 date: 1996/02/13 14:16:36; author: phk; state: Exp; lines: +8 -6 rewrap some long lines. ---------------------------- revision 1.18 date: 1996/01/19 03:57:56; author: dyson; state: Exp; lines: +2 -2 Eliminated many redundant vm_map_lookup operations for vm_mmap. Speed up for vfs_bio -- addition of a routine bqrelse to greatly diminish overhead for merged cache. Efficiency improvement for vfs_cluster. It used to do alot of redundant calls to cluster_rbuild. Correct the ordering for vrele of .text and release of credentials. Use the selective tlb update for 486/586/P6. Numerous fixes to the size of objects allocated for files. Additionally, fixes in the various pagers. Fixes for proper positioning of vnode_pager_setsize in msdosfs and ext2fs. Fixes in the swap pager for exhausted resources. The pageout code will not as readily thrash. Change the page queue flags (PG_ACTIVE, PG_INACTIVE, PG_FREE, PG_CACHE) into page queue indices (PQ_ACTIVE, PQ_INACTIVE, PQ_FREE, PQ_CACHE), thereby improving efficiency of several routines. Eliminate even more unnecessary vm_page_protect operations. Significantly speed up process forks. Make vm_object_page_clean more efficient, thereby eliminating the pause that happens every 30seconds. Make sequential clustered writes B_ASYNC instead of B_DELWRI even in the case of filesystems mounted async. Fix a panic with busy pages when write clustering is done for non-VMIO buffers. ---------------------------- revision 1.17 date: 1995/12/07 12:46:35; author: davidg; state: Exp; lines: +7 -1 Untangled the vm.h include file spaghetti. ---------------------------- revision 1.16 date: 1995/12/02 16:32:01; author: bde; state: Exp; lines: +3 -2 Staticized. Added prototypes. ---------------------------- revision 1.15 date: 1995/11/06 12:52:30; author: davidg; state: Exp; lines: +6 -6 All: Changed vnodep -> vp for consistency with the rest of the kernel, and changed iparams -> imgp for brevity. kern_exec.c: Explicitly initialized some additional parts of the image_params struct to avoid bzeroing it. Rewrote the set-id code to reduce the number of logical tests. The rewrite exposed a mostly benign bug in the algorithm: traced set-id images would get ktracing disabled even if the set-id didn't happen for other reasons. ---------------------------- revision 1.14 date: 1995/05/30 08:05:18; author: rgrimes; state: Exp; lines: +2 -2 Remove trailing whitespace. ---------------------------- revision 1.13 date: 1995/03/16 18:12:27; author: bde; state: Exp; lines: +1 -3 Add and move declarations to fix all of the warnings from `gcc -Wimplicit' (except in netccitt, netiso and netns) and most of the warnings from `gcc -Wnested-externs'. Fix all the bugs found. There were no serious ones. ---------------------------- revision 1.12 date: 1995/02/20 22:23:07; author: davidg; state: Exp; lines: +17 -15 Use of vm_allocate() and vm_deallocate() has been deprecated. ---------------------------- revision 1.11 date: 1995/02/13 07:40:33; author: phk; state: Exp; lines: +2 -2 Actually access the right first page if the file. Bruce finally caught this bogon for me, Thank you Bruce ! Due to some part of the VM/buffer/pmap magic doing clustering, this bogon managed to work better than 99.9% of the time. Amazing. If You ever again see a weird message from the gzip code, please tell me. ---------------------------- revision 1.10 date: 1994/10/22 11:55:16; author: phk; state: Exp; lines: +3 -2 Make the diagnostics a little more useful. A word of wisdom, don't do this: | cd /usr/bin | for i in * | do | cp $i /tmp/a | gzip -9 < /tmp/a > $i | done It will compress files with multiple links several times. do it this way: | cd /usr/bin | for i in * | do | gunzip -f < $i > /tmp/a | gzip -9 < /tmp/a > $i | done ---------------------------- revision 1.9 date: 1994/10/22 11:40:27; author: phk; state: Exp; lines: +301 -220 I belive imgact_gzip is finally reentrant. It is also a whole lot more readable. inflate is now much more general, and is there if anybody feels like making a uncompressing filesystem or something like that (hint hint !) ---------------------------- revision 1.8 date: 1994/10/11 11:29:09; author: csgr; state: Exp; lines: +9 -7 - remove unnecessary #includes (I think a couple of redundant ones remain) - excise some unused code (#if 0'd out - don't want to nuke it yet) - fix problems with "make depend" - some macros were screwing it up - get rid of some static local variables There still seems to be a small reentrancy problem somewhere. ---------------------------- revision 1.7 date: 1994/10/07 22:26:51; author: csgr; state: Exp; lines: +9 -1211 First stage of getting imgact_gzip reentrant: 1) cut this up into /sys/sys/inflate.h, sys/kern/inflate.c sys/kern/ingact_gzip.c 2) make a lot more things static 3) make a lot of globals const 4) make some args const 5) first stage of making globals into a struct (not used yet) The vm_allocate() call which was introduced between revisions 1.4 and 1.5 of imagact_gzip.c broke things. I have backed that out for the time being. (Davidg: help please) WARNING: if you have gzip enabled in your kernel, you must now run config again, as another source file has been added. Otherwise your kernel compile will fall over. This is all still WIP. More commits to come. Suggestions from: phk. ---------------------------- revision 1.6 date: 1994/10/06 18:22:24; author: phk; state: Exp; lines: +8 -1 Steven Wallace provided a program which broke this stuff. I guess there are more weird kinds of a.out than anyone can argue for. This code failed to load the first 28K of the text-segment, in the case where the first page of the a.out contains only the a.out-header, and the text is still at 0x0. Thanks Steven ! ---------------------------- revision 1.5 date: 1994/10/05 00:58:33; author: phk; state: Exp; lines: +5 -5 David Greenman told me to do this: (Thanks!) use vm_allocate to allocate the uncompression buffer. Now malloc(M_GZIP) is used for all the Huffman- tree stuff only. Numbers so far indicate < 15Kb Malloc use + 32 Kb for the abovementioned buffer while uncompressing. ---------------------------- revision 1.4 date: 1994/10/04 06:51:42; author: phk; state: Exp; lines: +13 -6 Added M_GZIP for the imgact_gzip code. The gzip-code is likely to be used for other weird things in the future (hint, hint!) ---------------------------- revision 1.3 date: 1994/10/04 03:09:13; author: phk; state: Exp; lines: +68 -54 Based on the applause (in this case: not downright rejection :-) I have cleaned up much of the cruft in this thing. No printf's in the case where things go well. Gzip-headers can contain filenames and comments (as long as they're shorter than the page-size.) I don't think we leak memory, in the "exec/aout" code. I'm not quite sure about the inflate code yet, but I don't think memory is lost. Q: Can I add a class M_GZIP to <sys/malloc.h> and bump M_LAST one up without any thing else needing tweaking ? Poul-Henning ---------------------------- revision 1.2 date: 1994/10/03 23:14:48; author: phk; state: Exp; lines: +9 -3 First bug-fix. This this depends on something odd. I am looking at it, but every now and then it will fail without an explanation :-( ---------------------------- revision 1.1 date: 1994/10/03 05:23:01; author: phk; state: Exp; *** WARNING: THIS MATERIAL MIGHT GO AWAY! This material needs the core-groups approval to stay here for the 2.0 release. If the core-group does not concent to this commit, it will be backed out. *** It is a non-gpl'ed "unzip" which will allow execution of a.out files which have been sent through "gzip -9". The idea being saved disk-space. Just now this code has quality rating: "working prototype". To compress a file to be used with this, do it exactly this way: gzip -9 -v < /bin/FOO > /tmp/FOO remember to chmod /tmp/FOO as needed. DON'T compress all of you binaries right away ! There are several things which you should consider first: 1. Using compressed binaries, you use >MUCH< more VM, and thus swap-space. 2. It is slow. 3. It might crash your machine. Apart from that, I welcome comments... NB: There is also a change to sys/conf/files, but cvs core-dumped on me, so it didn't get into the logs or emailed, but the commit seems to have happended OK. ---------------------------- =============================================================================
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199604200742.JAA16759>