Date: Fri, 16 May 2014 14:28:56 +0000 (UTC) From: Luiz Otavio O Souza <loos@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r266220 - in stable/10: share/man/man4 sys/geom/uncompress sys/geom/uzip sys/modules/geom sys/modules/geom/geom_uncompress Message-ID: <201405161428.s4GESudO069528@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: loos Date: Fri May 16 14:28:55 2014 New Revision: 266220 URL: http://svnweb.freebsd.org/changeset/base/266220 Log: MFC r260522, r260523, r261439, r261440, r261586, r264504, r264769, r265193, r265194, r265197 r260522: Add the manual page for geom_uncompress(4). r260523: Build the geom_uncompress(4) module by default. Fix geom_uncompress(4) module loading. Don't link zlib.c (which is a module itself) directly. r261439: Remove some unnecessary code. The offsets read from the first block are overwritten a few lines bellow. r261440: Fix a logic error. Because of this inflateReset() wasn't being called and the output buffer wasn't being cleared between the inflate() calls, producing zeroed output after the first inflate() call. This fixes the read of mkuzip(8) images with geom_uncompress(4). r261586: Fix the build with DEBUG enabled. Where possible, fix style(9) issues. r264504: Make sure not to do I/O for more than MAXPHYS bytes. Doing so can cause problems in our providers, such as a KASSERT in md(4). We can initiate I/O for more than MAXPHYS bytes if we've been given a BIO for MAXPHYS bytes, the blocks from which we're reading couldn't be compressed and we had compression in preceeding blocks resulting in misalignment of the blocks we're trying to read relative to the sector. We're forced to round up the I/O length to make it an multiple of the sector size. When we detect the condition, we'll reduce the block count and perform a "short" read. In g_uzip_done() we need to consider the original I/O length and stop early if we're about to deflate a block that we didn't read. By using bio_completed in the cloned BIO and not bio_length to check for this, we automatically and gracefully handle short reads that our providers may be doing on top of the short reads we may initiate ourselves. r264769: Keep geom_uncompress(4) in line with geom_uzip(4), bring in the r264504 fix. Make sure not to start I/O bigger than MAXPHYS bytes. r265193: Some style and whitespace fixes. Reduce the difference between geom_uzip(4) and geom_uncompress(4). Now, they produce an almost clean diff(1) output. Remove a duplicated variable from g_uncompress.c and an unnecessary header from g_uzip.c. r265194: Actually the FEATURE() macro is defined on sys/sysctl.h. r265197: Fix a leak in g_uzip_taste(). After retrieve all the block offsets from the uzip image, free the last data read. Added: stable/10/share/man/man4/geom_uncompress.4 - copied unchanged from r260522, head/share/man/man4/geom_uncompress.4 Modified: stable/10/share/man/man4/Makefile stable/10/sys/geom/uncompress/g_uncompress.c stable/10/sys/geom/uzip/g_uzip.c stable/10/sys/modules/geom/Makefile stable/10/sys/modules/geom/geom_uncompress/Makefile Directory Properties: stable/10/ (props changed) Modified: stable/10/share/man/man4/Makefile ============================================================================== --- stable/10/share/man/man4/Makefile Fri May 16 14:09:51 2014 (r266219) +++ stable/10/share/man/man4/Makefile Fri May 16 14:28:55 2014 (r266220) @@ -153,6 +153,7 @@ MAN= aac.4 \ geom_fox.4 \ geom_linux_lvm.4 \ geom_map.4 \ + geom_uncompress.4 \ geom_uzip.4 \ gif.4 \ gpib.4 \ Copied: stable/10/share/man/man4/geom_uncompress.4 (from r260522, head/share/man/man4/geom_uncompress.4) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/10/share/man/man4/geom_uncompress.4 Fri May 16 14:28:55 2014 (r266220, copy of r260522, head/share/man/man4/geom_uncompress.4) @@ -0,0 +1,107 @@ +.\" Copyright (c) 2014, Luiz Otavio O Souza <loos@FreeBSD.org> +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd January 9, 2014 +.Dt GEOM_UNCOMPRESS 4 +.Os +.Sh NAME +.Nm geom_uncompress +.Nd "GEOM based compressed disk images" +.Sh SYNOPSIS +To compile this driver into the kernel, place the following line in your +kernel configuration file: +.Bd -ragged -offset indent +.Cd "options GEOM_UNCOMPRESS" +.Ed +.Pp +Alternatively, to load the driver as a module at boot time, place the +following line in +.Xr loader.conf 5 : +.Bd -literal -offset indent +geom_uncompress_load="YES" +.Ed +.Sh DESCRIPTION +The +.Nm +framework provides support for compressed read only disk images. +This allows significant storage savings at the expense of a little CPU +time on each read. +Data written in the GEOM label area allows +.Nm +to detect compressed images which have been created with +.Xr mkulzma 8 +or +.Xr mkuzip 8 +and presented to the kernel as a logical disk device via +.Xr md 4 . +.Nm +creates a unique +.Pa md#.uncompress +device for each image. +.Pp +The +.Nm +device is subsequently used by the +.Fx +kernel to access the disk images. +The +.Nm +driver does not allow write operations to the underlying disk image. +To check which +.Xr md 4 +devices match a given +.Nm +device: +.Bd -literal -offset indent +# geom uncompress list +Geom name: md0.uncompress +Providers: +1. Name: md0.uncompress + Mediasize: 52428800 (50M) + Sectorsize: 512 + Mode: r1w0e0 +Consumers: +1. Name: md0 + Mediasize: 20864000 (20M) + Sectorsize: 512 + Mode: r1w0e0 +.Ed +.Sh SEE ALSO +.Xr GEOM 4 , +.Xr md 4 , +.Xr geom 8 , +.Xr mkulzma 8 , +.Xr mkuzip 8 +.Sh AUTHORS +.An -nosplit +The +.Nm +driver was written by +.An "Maxim Sobolev" Aq sobomax@FreeBSD.org +and +.An "Aleksandr Rybalko" Aq ray@FreeBSD.org . +This manual page was written by +.An "Luiz Otavio O Souza" Aq loos@FreeBSD.org . Modified: stable/10/sys/geom/uncompress/g_uncompress.c ============================================================================== --- stable/10/sys/geom/uncompress/g_uncompress.c Fri May 16 14:09:51 2014 (r266219) +++ stable/10/sys/geom/uncompress/g_uncompress.c Fri May 16 14:28:55 2014 (r266220) @@ -116,7 +116,7 @@ g_uncompress_softc_free(struct g_uncompr } if (sc->offsets != NULL) { free(sc->offsets, M_GEOM_UNCOMPRESS); - sc->offsets = 0; + sc->offsets = NULL; } switch (sc->type) { @@ -150,6 +150,7 @@ z_alloc(void *nil, u_int type, u_int siz void *ptr; ptr = malloc(type * size, M_GEOM_UNCOMPRESS, M_NOWAIT); + return (ptr); } @@ -169,7 +170,7 @@ g_uncompress_done(struct bio *bp) struct g_geom *gp; struct bio *bp2; uint32_t start_blk, i; - off_t pos, upos; + off_t iolen, pos, upos; size_t bsize; int err; @@ -210,24 +211,27 @@ g_uncompress_done(struct bio *bp) */ start_blk = bp2->bio_offset / sc->blksz; bsize = pp2->sectorsize; + iolen = bp->bio_completed; pos = sc->offsets[start_blk] % bsize; upos = 0; - DPRINTF(("%s: done: bio_length %lld bio_completed %lld start_blk %d, " - "pos %lld, upos %lld (%lld, %d, %d)\n", - gp->name, bp->bio_length, bp->bio_completed, start_blk, pos, upos, - bp2->bio_offset, sc->blksz, bsize)); + DPRINTF(("%s: done: bio_length %jd bio_completed %jd start_blk %d, " + "pos %jd, upos %jd (%jd, %d, %zu)\n", + gp->name, (intmax_t)bp->bio_length, (intmax_t)bp->bio_completed, + start_blk, (intmax_t)pos, (intmax_t)upos, + (intmax_t)bp2->bio_offset, sc->blksz, bsize)); for (i = start_blk; upos < bp2->bio_length; i++) { - off_t len, dlen, ulen, uoff; + off_t len, ulen, uoff; uoff = i == start_blk ? bp2->bio_offset % sc->blksz : 0; ulen = MIN(sc->blksz - uoff, bp2->bio_length - upos); - dlen = len = sc->offsets[i + 1] - sc->offsets[i]; + len = sc->offsets[i + 1] - sc->offsets[i]; - DPRINTF(("%s: done: inflate block %d, start %lld, end %lld " - "len %lld\n", - gp->name, i, sc->offsets[i], sc->offsets[i + 1], len)); + DPRINTF(( + "%s: done: inflate block %d, start %ju, end %ju len %jd\n", + gp->name, i, (uintmax_t)sc->offsets[i], + (uintmax_t)sc->offsets[i + 1], (intmax_t)len)); if (len == 0) { /* All zero block: no cache update */ @@ -236,12 +240,17 @@ g_uncompress_done(struct bio *bp) bp2->bio_completed += ulen; continue; } - + if (len > iolen) { + DPRINTF(("%s: done: early termination: len (%jd) > " + "iolen (%jd)\n", + gp->name, (intmax_t)len, (intmax_t)iolen)); + break; + } mtx_lock(&sc->last_mtx); #ifdef GEOM_UNCOMPRESS_DEBUG if (g_debugflags & 32) - hexdump(bp->bio_data + pos, dlen, 0, 0); + hexdump(bp->bio_data + pos, len, 0, 0); #endif switch (sc->type) { @@ -249,7 +258,7 @@ g_uncompress_done(struct bio *bp) sc->b->in = bp->bio_data + pos; sc->b->out = sc->last_buf; sc->b->in_pos = sc->b->out_pos = 0; - sc->b->in_size = dlen; + sc->b->in_size = len; sc->b->out_size = (size_t)-1; err = (xz_dec_run(sc->s, sc->b) != XZ_STREAM_END) ? @@ -258,13 +267,13 @@ g_uncompress_done(struct bio *bp) break; case GEOM_UZIP: sc->zs->next_in = bp->bio_data + pos; - sc->zs->avail_in = dlen; + sc->zs->avail_in = len; sc->zs->next_out = sc->last_buf; sc->zs->avail_out = sc->blksz; err = (inflate(sc->zs, Z_FINISH) != Z_STREAM_END) ? 1 : 0; - if ((err) && (inflateReset(sc->zs) != Z_OK)) + if ((err) || (inflateReset(sc->zs) != Z_OK)) printf("%s: UZIP decoder reset failed\n", gp->name); break; @@ -290,6 +299,7 @@ g_uncompress_done(struct bio *bp) mtx_unlock(&sc->last_mtx); pos += len; + iolen -= len; upos += ulen; bp2->bio_completed += ulen; } @@ -298,8 +308,9 @@ done: /* * Finish processing the request. */ - DPRINTF(("%s: done: (%d, %lld, %ld)\n", - gp->name, bp2->bio_error, bp2->bio_completed, bp2->bio_resid)); + DPRINTF(("%s: done: (%d, %jd, %ld)\n", + gp->name, bp2->bio_error, (intmax_t)bp2->bio_completed, + bp2->bio_resid)); free(bp->bio_data, M_GEOM_UNCOMPRESS); g_destroy_bio(bp); g_io_deliver(bp2, bp2->bio_error); @@ -316,12 +327,12 @@ g_uncompress_start(struct bio *bp) uint32_t start_blk, end_blk; size_t bsize; - pp = bp->bio_to; gp = pp->geom; - DPRINTF(("%s: start (%s) to %s off=%lld len=%lld\n", gp->name, - (bp->bio_cmd==BIO_READ) ? "BIO_READ" : "BIO_WRITE*", - pp->name, bp->bio_offset, bp->bio_length)); + DPRINTF(("%s: start (%d:%s) to %s off=%jd len=%jd\n", + gp->name, bp->bio_cmd, + (bp->bio_cmd == BIO_READ) ? "BIO_READ" : "NOTSUPPORTED", + pp->name, (intmax_t)bp->bio_offset, (intmax_t)bp->bio_length)); if (bp->bio_cmd != BIO_READ) { g_io_deliver(bp, EOPNOTSUPP); @@ -334,10 +345,8 @@ g_uncompress_start(struct bio *bp) start_blk = bp->bio_offset / sc->blksz; end_blk = howmany(bp->bio_offset + bp->bio_length, sc->blksz); - KASSERT(start_blk < sc->nblocks, - ("start_blk out of range")); - KASSERT(end_blk <= sc->nblocks, - ("end_blk out of range")); + KASSERT(start_blk < sc->nblocks, ("start_blk out of range")); + KASSERT(end_blk <= sc->nblocks, ("end_blk out of range")); sc->req_total++; if (start_blk + 1 == end_blk) { @@ -353,9 +362,9 @@ g_uncompress_start(struct bio *bp) sc->req_cached++; mtx_unlock(&sc->last_mtx); - DPRINTF(("%s: start: cached 0 + %lld, " - "%lld + 0 + %lld\n", - gp->name, bp->bio_length, uoff, bp->bio_length)); + DPRINTF(("%s: start: cached 0 + %jd, %jd + 0 + %jd\n", + gp->name, (intmax_t)bp->bio_length, (intmax_t)uoff, + (intmax_t)bp->bio_length)); bp->bio_completed = bp->bio_length; g_io_deliver(bp, 0); return; @@ -368,26 +377,31 @@ g_uncompress_start(struct bio *bp) g_io_deliver(bp, ENOMEM); return; } - DPRINTF(("%s: start (%d..%d), %s: %d + %llu, %s: %d + %llu\n", + DPRINTF(("%s: start (%d..%d), %s: %d + %jd, %s: %d + %jd\n", gp->name, start_blk, end_blk, - pp->name, pp->sectorsize, pp->mediasize, - pp2->name, pp2->sectorsize, pp2->mediasize)); - + pp->name, pp->sectorsize, (intmax_t)pp->mediasize, + pp2->name, pp2->sectorsize, (intmax_t)pp2->mediasize)); bsize = pp2->sectorsize; - bp2->bio_done = g_uncompress_done; - bp2->bio_offset = rounddown(sc->offsets[start_blk],bsize); - bp2->bio_length = roundup(sc->offsets[end_blk],bsize) - - bp2->bio_offset; - bp2->bio_data = malloc(bp2->bio_length, M_GEOM_UNCOMPRESS, M_NOWAIT); + bp2->bio_offset = rounddown(sc->offsets[start_blk], bsize); + while (1) { + bp2->bio_length = roundup(sc->offsets[end_blk], bsize) - + bp2->bio_offset; + if (bp2->bio_length < MAXPHYS) + break; - DPRINTF(("%s: start %lld + %lld -> %lld + %lld -> %lld + %lld\n", + end_blk--; + DPRINTF(( + "%s: bio_length (%jd) > MAXPHYS: lowering end_blk to %u\n", + gp->name, (intmax_t)bp2->bio_length, end_blk)); + } + DPRINTF(("%s: start %jd + %jd -> %ju + %ju -> %jd + %jd\n", gp->name, - bp->bio_offset, bp->bio_length, - sc->offsets[start_blk], - sc->offsets[end_blk] - sc->offsets[start_blk], - bp2->bio_offset, bp2->bio_length)); - + (intmax_t)bp->bio_offset, (intmax_t)bp->bio_length, + (uintmax_t)sc->offsets[start_blk], + (uintmax_t)sc->offsets[end_blk] - sc->offsets[start_blk], + (intmax_t)bp2->bio_offset, (intmax_t)bp2->bio_length)); + bp2->bio_data = malloc(bp2->bio_length, M_GEOM_UNCOMPRESS, M_NOWAIT); if (bp2->bio_data == NULL) { g_destroy_bio(bp2); g_io_deliver(bp, ENOMEM); @@ -403,8 +417,7 @@ g_uncompress_orphan(struct g_consumer *c { struct g_geom *gp; - g_trace(G_T_TOPOLOGY, "%s(%p/%s)", __func__, cp, - cp->provider->name); + g_trace(G_T_TOPOLOGY, "%s(%p/%s)", __func__, cp, cp->provider->name); g_topology_assert(); gp = cp->geom; @@ -451,7 +464,7 @@ g_uncompress_taste(struct g_class *mp, s struct g_provider *pp2; struct g_consumer *cp; struct g_geom *gp; - uint32_t i, total_offsets, offsets_read, type; + uint32_t i, total_offsets, type; uint8_t *buf; int error; @@ -484,17 +497,15 @@ g_uncompress_taste(struct g_class *mp, s * Read cloop header, look for CLOOP magic, perform * other validity checks. */ - DPRINTF(("%s: media sectorsize %u, mediasize %lld\n", - gp->name, pp->sectorsize, pp->mediasize)); - + DPRINTF(("%s: media sectorsize %u, mediasize %jd\n", + gp->name, pp->sectorsize, (intmax_t)pp->mediasize)); i = roundup(sizeof(struct cloop_header), pp->sectorsize); buf = g_read_data(cp, 0, i, NULL); if (buf == NULL) goto err; - header = (struct cloop_header *) buf; if (strncmp(header->magic, CLOOP_MAGIC_START, - sizeof(CLOOP_MAGIC_START) - 1) != 0) { + sizeof(CLOOP_MAGIC_START) - 1) != 0) { DPRINTF(("%s: no CLOOP magic\n", gp->name)); goto err; } @@ -546,25 +557,20 @@ g_uncompress_taste(struct g_class *mp, s gp->name, sc->nblocks); goto err; } - sc->offsets = malloc( - total_offsets * sizeof(uint64_t), M_GEOM_UNCOMPRESS, M_WAITOK); - offsets_read = MIN(total_offsets, - (pp->sectorsize - sizeof(*header)) / sizeof(uint64_t)); - for (i = 0; i < offsets_read; i++) - sc->offsets[i] = be64toh(((uint64_t *) (header + 1))[i]); - DPRINTF(("%s: %u offsets in the first sector\n", - gp->name, offsets_read)); - free(buf, M_GEOM); + i = roundup((sizeof(struct cloop_header) + - total_offsets * sizeof(uint64_t)),pp->sectorsize); + total_offsets * sizeof(uint64_t)), pp->sectorsize); buf = g_read_data(cp, 0, i, NULL); if (buf == NULL) goto err; + sc->offsets = malloc(total_offsets * sizeof(uint64_t), + M_GEOM_UNCOMPRESS, M_WAITOK); for (i = 0; i <= total_offsets; i++) { sc->offsets[i] = be64toh(((uint64_t *) (buf+sizeof(struct cloop_header)))[i]); } + free(buf, M_GEOM); DPRINTF(("%s: done reading offsets\n", gp->name)); mtx_init(&sc->last_mtx, "geom_uncompress cache", NULL, MTX_DEF); sc->last_blk = -1; @@ -599,15 +605,13 @@ g_uncompress_taste(struct g_class *mp, s pp2->stripeoffset = pp->stripeoffset; } g_error_provider(pp2, 0); - free(buf, M_GEOM); g_access(cp, -1, 0, 0); - DPRINTF(("%s: taste ok (%d, %lld), (%d, %d), %x\n", + DPRINTF(("%s: taste ok (%d, %jd), (%d, %d), %x\n", gp->name, - pp2->sectorsize, pp2->mediasize, + pp2->sectorsize, (intmax_t)pp2->mediasize, pp2->stripeoffset, pp2->stripesize, pp2->flags)); - printf("%s: %u x %u blocks\n", - gp->name, sc->nblocks, sc->blksz); + printf("%s: %u x %u blocks\n", gp->name, sc->nblocks, sc->blksz); return (gp); err: @@ -622,6 +626,7 @@ err: g_detach(cp); g_destroy_consumer(cp); g_destroy_geom(gp); + return (NULL); } @@ -648,6 +653,7 @@ g_uncompress_destroy_geom(struct gctl_re g_uncompress_softc_free(gp->softc, gp); gp->softc = NULL; g_wither_geom(gp, ENXIO); + return (0); } @@ -664,4 +670,4 @@ static struct g_class g_uncompress_class }; DECLARE_GEOM_CLASS(g_uncompress_class, g_uncompress); - +MODULE_DEPEND(g_uncompress, zlib, 1, 1, 1); Modified: stable/10/sys/geom/uzip/g_uzip.c ============================================================================== --- stable/10/sys/geom/uzip/g_uzip.c Fri May 16 14:09:51 2014 (r266219) +++ stable/10/sys/geom/uzip/g_uzip.c Fri May 16 14:28:55 2014 (r266220) @@ -35,8 +35,8 @@ __FBSDID("$FreeBSD$"); #include <sys/lock.h> #include <sys/mutex.h> #include <sys/malloc.h> -#include <sys/systm.h> #include <sys/sysctl.h> +#include <sys/systm.h> #include <geom/geom.h> #include <net/zlib.h> @@ -45,19 +45,19 @@ FEATURE(geom_uzip, "GEOM uzip read-only #undef GEOM_UZIP_DEBUG #ifdef GEOM_UZIP_DEBUG -#define DPRINTF(a) printf a +#define DPRINTF(a) printf a #else -#define DPRINTF(a) +#define DPRINTF(a) #endif static MALLOC_DEFINE(M_GEOM_UZIP, "geom_uzip", "GEOM UZIP data structures"); -#define UZIP_CLASS_NAME "UZIP" +#define UZIP_CLASS_NAME "UZIP" /* * Maximum allowed valid block size (to prevent foot-shooting) */ -#define MAX_BLKSZ (MAXPHYS - MAXPHYS / 1000 - 12) +#define MAX_BLKSZ (MAXPHYS - MAXPHYS / 1000 - 12) /* * Integer values (block size, number of blocks, offsets) @@ -65,7 +65,7 @@ static MALLOC_DEFINE(M_GEOM_UZIP, "geom_ * and in native order in struct g_uzip_softc */ -#define CLOOP_MAGIC_LEN 128 +#define CLOOP_MAGIC_LEN 128 static char CLOOP_MAGIC_START[] = "#!/bin/sh\n"; struct cloop_header { @@ -89,12 +89,15 @@ struct g_uzip_softc { static void g_uzip_softc_free(struct g_uzip_softc *sc, struct g_geom *gp) { + if (gp != NULL) { printf("%s: %d requests, %d cached\n", gp->name, sc->req_total, sc->req_cached); } - if (sc->offsets != NULL) + if (sc->offsets != NULL) { free(sc->offsets, M_GEOM_UZIP); + sc->offsets = NULL; + } mtx_destroy(&sc->last_mtx); free(sc->last_buf, M_GEOM_UZIP); free(sc, M_GEOM_UZIP); @@ -106,12 +109,14 @@ z_alloc(void *nil, u_int type, u_int siz void *ptr; ptr = malloc(type * size, M_GEOM_UZIP, M_NOWAIT); - return ptr; + + return (ptr); } static void z_free(void *nil, void *ptr) { + free(ptr, M_GEOM_UZIP); } @@ -125,7 +130,7 @@ g_uzip_done(struct bio *bp) struct g_consumer *cp; struct g_geom *gp; struct g_uzip_softc *sc; - off_t pos, upos; + off_t iolen, pos, upos; uint32_t start_blk, i; size_t bsize; @@ -153,11 +158,13 @@ g_uzip_done(struct bio *bp) } start_blk = bp2->bio_offset / sc->blksz; bsize = pp2->sectorsize; + iolen = bp->bio_completed; pos = sc->offsets[start_blk] % bsize; upos = 0; - DPRINTF(("%s: done: start_blk %d, pos %lld, upos %lld (%lld, %d, %d)\n", - gp->name, start_blk, pos, upos, - bp2->bio_offset, sc->blksz, bsize)); + DPRINTF(("%s: done: start_blk %d, pos %jd, upos %jd, iolen %jd " + "(%jd, %d, %zd)\n", + gp->name, start_blk, (intmax_t)pos, (intmax_t)upos, + (intmax_t)iolen, (intmax_t)bp2->bio_offset, sc->blksz, bsize)); for (i = start_blk; upos < bp2->bio_length; i++) { off_t len, ulen, uoff; @@ -172,6 +179,12 @@ g_uzip_done(struct bio *bp) bp2->bio_completed += ulen; continue; } + if (len > iolen) { + DPRINTF(("%s: done: early termination: len (%jd) > " + "iolen (%jd)\n", + gp->name, (intmax_t)len, (intmax_t)iolen)); + break; + } zs.next_in = bp->bio_data + pos; zs.avail_in = len; zs.next_out = sc->last_buf; @@ -181,21 +194,22 @@ g_uzip_done(struct bio *bp) if (err != Z_STREAM_END) { sc->last_blk = -1; mtx_unlock(&sc->last_mtx); - DPRINTF(("%s: done: inflate failed (%lld + %lld -> %lld + %lld + %lld)\n", - gp->name, pos, len, uoff, upos, ulen)); + DPRINTF(("%s: done: inflate failed (%jd + %jd -> %jd + %jd + %jd)\n", + gp->name, (intmax_t)pos, (intmax_t)len, + (intmax_t)uoff, (intmax_t)upos, (intmax_t)ulen)); inflateEnd(&zs); bp2->bio_error = EIO; goto done; } sc->last_blk = i; - DPRINTF(("%s: done: inflated %lld + %lld -> %lld + %lld + %lld\n", - gp->name, - pos, len, - uoff, upos, ulen)); + DPRINTF(("%s: done: inflated %jd + %jd -> %jd + %jd + %jd\n", + gp->name, (intmax_t)pos, (intmax_t)len, (intmax_t)uoff, + (intmax_t)upos, (intmax_t)ulen)); memcpy(bp2->bio_data + upos, sc->last_buf + uoff, ulen); mtx_unlock(&sc->last_mtx); pos += len; + iolen -= len; upos += ulen; bp2->bio_completed += ulen; err = inflateReset(&zs); @@ -215,8 +229,9 @@ done: /* * Finish processing the request. */ - DPRINTF(("%s: done: (%d, %lld, %ld)\n", - gp->name, bp2->bio_error, bp2->bio_completed, bp2->bio_resid)); + DPRINTF(("%s: done: (%d, %jd, %ld)\n", + gp->name, bp2->bio_error, (intmax_t)bp2->bio_completed, + bp2->bio_resid)); free(bp->bio_data, M_GEOM_UZIP); g_destroy_bio(bp); g_io_deliver(bp2, bp2->bio_error); @@ -248,10 +263,8 @@ g_uzip_start(struct bio *bp) start_blk = bp->bio_offset / sc->blksz; end_blk = (bp->bio_offset + bp->bio_length + sc->blksz - 1) / sc->blksz; - KASSERT(start_blk < sc->nblocks, - ("start_blk out of range")); - KASSERT(end_blk <= sc->nblocks, - ("end_blk out of range")); + KASSERT(start_blk < sc->nblocks, ("start_blk out of range")); + KASSERT(end_blk <= sc->nblocks, ("end_blk out of range")); sc->req_total++; if (start_blk + 1 == end_blk) { @@ -267,8 +280,9 @@ g_uzip_start(struct bio *bp) sc->req_cached++; mtx_unlock(&sc->last_mtx); - DPRINTF(("%s: start: cached 0 + %lld, %lld + 0 + %lld\n", - gp->name, bp->bio_length, uoff, bp->bio_length)); + DPRINTF(("%s: start: cached 0 + %jd, %jd + 0 + %jd\n", + gp->name, (intmax_t)bp->bio_length, (intmax_t)uoff, + (intmax_t)bp->bio_length)); bp->bio_completed = bp->bio_length; g_io_deliver(bp, 0); return; @@ -282,19 +296,28 @@ g_uzip_start(struct bio *bp) return; } bp2->bio_done = g_uzip_done; - DPRINTF(("%s: start (%d..%d), %s: %d + %lld, %s: %d + %lld\n", + DPRINTF(("%s: start (%d..%d), %s: %d + %jd, %s: %d + %jd\n", gp->name, start_blk, end_blk, - pp->name, pp->sectorsize, pp->mediasize, - pp2->name, pp2->sectorsize, pp2->mediasize)); + pp->name, pp->sectorsize, (intmax_t)pp->mediasize, + pp2->name, pp2->sectorsize, (intmax_t)pp2->mediasize)); bsize = pp2->sectorsize; bp2->bio_offset = sc->offsets[start_blk] - sc->offsets[start_blk] % bsize; - bp2->bio_length = sc->offsets[end_blk] - bp2->bio_offset; - bp2->bio_length = (bp2->bio_length + bsize - 1) / bsize * bsize; - DPRINTF(("%s: start %lld + %lld -> %lld + %lld -> %lld + %lld\n", + while (1) { + bp2->bio_length = sc->offsets[end_blk] - bp2->bio_offset; + bp2->bio_length = (bp2->bio_length + bsize - 1) / bsize * bsize; + if (bp2->bio_length < MAXPHYS) + break; + + end_blk--; + DPRINTF(("%s: bio_length (%jd) > MAXPHYS: lowering end_blk " + "to %u\n", gp->name, (intmax_t)bp2->bio_length, end_blk)); + } + DPRINTF(("%s: start %jd + %jd -> %ju + %ju -> %jd + %jd\n", gp->name, - bp->bio_offset, bp->bio_length, - sc->offsets[start_blk], sc->offsets[end_blk] - sc->offsets[start_blk], - bp2->bio_offset, bp2->bio_length)); + (intmax_t)bp->bio_offset, (intmax_t)bp->bio_length, + (uintmax_t)sc->offsets[start_blk], + (uintmax_t)sc->offsets[end_blk] - sc->offsets[start_blk], + (intmax_t)bp2->bio_offset, (intmax_t)bp2->bio_length)); bp2->bio_data = malloc(bp2->bio_length, M_GEOM_UZIP, M_NOWAIT); if (bp2->bio_data == NULL) { g_destroy_bio(bp2); @@ -311,7 +334,7 @@ g_uzip_orphan(struct g_consumer *cp) { struct g_geom *gp; - g_trace(G_T_TOPOLOGY, "g_uzip_orphan(%p/%s)", cp, cp->provider->name); + g_trace(G_T_TOPOLOGY, "%s(%p/%s)", __func__, cp, cp->provider->name); g_topology_assert(); gp = cp->geom; @@ -331,7 +354,7 @@ g_uzip_access(struct g_provider *pp, int KASSERT (cp != NULL, ("g_uzip_access but no consumer")); if (cp->acw + dw > 0) - return EROFS; + return (EROFS); return (g_access(cp, dr, dw, de)); } @@ -342,7 +365,7 @@ g_uzip_spoiled(struct g_consumer *cp) struct g_geom *gp; gp = cp->geom; - g_trace(G_T_TOPOLOGY, "g_uzip_spoiled(%p/%s)", cp, gp->name); + g_trace(G_T_TOPOLOGY, "%s(%p/%s)", __func__, cp, gp->name); g_topology_assert(); g_uzip_softc_free(gp->softc, gp); @@ -362,7 +385,7 @@ g_uzip_taste(struct g_class *mp, struct struct g_provider *pp2; struct g_uzip_softc *sc; - g_trace(G_T_TOPOLOGY, "g_uzip_taste(%s,%s)", mp->name, pp->name); + g_trace(G_T_TOPOLOGY, "%s(%s,%s)", __func__, mp->name, pp->name); g_topology_assert(); /* Skip providers that are already open for writing. */ @@ -391,14 +414,14 @@ g_uzip_taste(struct g_class *mp, struct * Read cloop header, look for CLOOP magic, perform * other validity checks. */ - DPRINTF(("%s: media sectorsize %u, mediasize %lld\n", - gp->name, pp->sectorsize, pp->mediasize)); + DPRINTF(("%s: media sectorsize %u, mediasize %jd\n", + gp->name, pp->sectorsize, (intmax_t)pp->mediasize)); buf = g_read_data(cp, 0, pp->sectorsize, NULL); if (buf == NULL) goto err; header = (struct cloop_header *) buf; if (strncmp(header->magic, CLOOP_MAGIC_START, - sizeof(CLOOP_MAGIC_START) - 1) != 0) { + sizeof(CLOOP_MAGIC_START) - 1) != 0) { DPRINTF(("%s: no CLOOP magic\n", gp->name)); goto err; } @@ -427,7 +450,7 @@ g_uzip_taste(struct g_class *mp, struct if (sizeof(struct cloop_header) + total_offsets * sizeof(uint64_t) > pp->mediasize) { printf("%s: media too small for %u blocks\n", - gp->name, sc->nblocks); + gp->name, sc->nblocks); goto err; } sc->offsets = malloc( @@ -456,6 +479,7 @@ g_uzip_taste(struct g_class *mp, struct } offsets_read += nread; } + free(buf, M_GEOM); DPRINTF(("%s: done reading offsets\n", gp->name)); mtx_init(&sc->last_mtx, "geom_uzip cache", NULL, MTX_DEF); sc->last_blk = -1; @@ -467,17 +491,16 @@ g_uzip_taste(struct g_class *mp, struct pp2 = g_new_providerf(gp, "%s", gp->name); pp2->sectorsize = 512; pp2->mediasize = (off_t)sc->nblocks * sc->blksz; - pp2->stripesize = pp->stripesize; - pp2->stripeoffset = pp->stripeoffset; + pp2->stripesize = pp->stripesize; + pp2->stripeoffset = pp->stripeoffset; g_error_provider(pp2, 0); g_access(cp, -1, 0, 0); - DPRINTF(("%s: taste ok (%d, %lld), (%d, %d), %x\n", + DPRINTF(("%s: taste ok (%d, %jd), (%d, %d), %x\n", gp->name, - pp2->sectorsize, pp2->mediasize, + pp2->sectorsize, (intmax_t)pp2->mediasize, pp2->stripeoffset, pp2->stripesize, pp2->flags)); - printf("%s: %u x %u blocks\n", - gp->name, sc->nblocks, sc->blksz); + printf("%s: %u x %u blocks\n", gp->name, sc->nblocks, sc->blksz); return (gp); err: @@ -492,6 +515,7 @@ err: g_detach(cp); g_destroy_consumer(cp); g_destroy_geom(gp); + return (NULL); } @@ -500,7 +524,7 @@ g_uzip_destroy_geom(struct gctl_req *req { struct g_provider *pp; - g_trace(G_T_TOPOLOGY, "g_uzip_destroy_geom(%s, %s)", mp->name, gp->name); + g_trace(G_T_TOPOLOGY, "%s(%s, %s)", __func__, mp->name, gp->name); g_topology_assert(); if (gp->softc == NULL) { @@ -517,6 +541,7 @@ g_uzip_destroy_geom(struct gctl_req *req g_uzip_softc_free(gp->softc, gp); gp->softc = NULL; g_wither_geom(gp, ENXIO); + return (0); } Modified: stable/10/sys/modules/geom/Makefile ============================================================================== --- stable/10/sys/modules/geom/Makefile Fri May 16 14:09:51 2014 (r266219) +++ stable/10/sys/modules/geom/Makefile Fri May 16 14:28:55 2014 (r266220) @@ -24,6 +24,7 @@ SUBDIR= geom_bde \ geom_shsec \ geom_stripe \ geom_sunlabel \ + geom_uncompress \ geom_uzip \ geom_vinum \ geom_virstor \ Modified: stable/10/sys/modules/geom/geom_uncompress/Makefile ============================================================================== --- stable/10/sys/modules/geom/geom_uncompress/Makefile Fri May 16 14:09:51 2014 (r266219) +++ stable/10/sys/modules/geom/geom_uncompress/Makefile Fri May 16 14:28:55 2014 (r266220) @@ -7,11 +7,11 @@ ${.CURDIR}/../../../net KMOD= geom_uncompress -CFLAGS= -I${.CURDIR}/../../../geom/uncompress/ \ +CFLAGS+= -I${.CURDIR}/../../../geom/uncompress/ \ -I${.CURDIR}/../../../contrib/xz-embedded/freebsd \ -I${.CURDIR}/../../../contrib/xz-embedded/linux/lib/xz/ SRCS= g_uncompress.c xz_crc32.c xz_dec_bcj.c xz_dec_lzma2.c xz_dec_stream.c \ - xz_malloc.c zlib.c + xz_malloc.c SRCS+= xz.h xz_config.h xz_lzma2.h xz_malloc.h xz_private.h xz_stream.h zlib.h .include <bsd.kmod.mk>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201405161428.s4GESudO069528>