Date: Wed, 5 Nov 2008 03:39:51 +0100 From: Max Laier <max@love2party.net> To: Giorgos Keramidas <keramida@freebsd.org> Cc: freebsd-hackers@freebsd.org, koitsu@freebsd.org, pjd@freebsd.org, Pete French <petefrench@ticketswitch.com>, lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] Message-ID: <200811050339.52088.max@love2party.net> In-Reply-To: <87prlagayp.fsf@kobe.laptop> References: <E1KnA2b-0007ls-8u@dilbert.ticketswitch.com> <200811042342.49827.max@love2party.net> <87prlagayp.fsf@kobe.laptop>
next in thread | previous in thread | raw e-mail | index | archive | help
--Boundary-00=_4dQEJnq7CkB11C+ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Wednesday 05 November 2008 03:31:26 Giorgos Keramidas wrote: > On Tue, 4 Nov 2008 23:42:49 +0100, Max Laier <max@love2party.net> wrote: > > Hi again, > > > > On Saturday 01 November 2008 21:14:42 I wrote: > >> a thread on freebsd-stable@ [1] about problems with du(1) and compressed > >> zfs filesystems got me looking for a possible solution. Attached is a > >> diff for du(1) that adds two new options: > >> > >> -A to display the apparent size of the file instead of the used blocks. > >> -B bsize to specify a custom blocksize. In particular one <512byte > >> > >> The GNU du(1) has --apparent-size for -A, but we don't like long > >> options. That's not to say that it couldn't be added for script compat. > >> -B is probably not that interesting, but it can be helpful and came for > >> free. > > > > Attached is an updated patch. This refines the -B option to something > > more useful (and fixes a bug in the original patch). > > > > From the man page: > > > > -B blocksize > > Calculate block counts in blocksize byte blocks. This is > > differ- ent from the -k, -m options or setting BLOCKSIZE and gives an > > estimate of how many space the examined file hierachy would require on a > > filesystem with the given blocksize. Unless in -A mode, blocksize is > > rounded up to the next multiple of 512. > > That looks nice! With a small fix (``how _much_ space'') I like the > idea a lot :) > > The patch fails to apply on a recent /head snapshot of du though: ... > Can you please refresh and repost it? Hum ... are you sure your snapshot is up to date? I did some style(9) cleanup in r184654/6 that you might not have yet. Anyways ... here is a diff with those revisions included. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --Boundary-00=_4dQEJnq7CkB11C+ Content-Type: text/plain; charset="iso-8859-1"; name="du.AB.earlier.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="du.AB.earlier.diff" Index: du.c =================================================================== --- du.c (revision 176561) +++ du.c (working copy) @@ -73,20 +73,21 @@ static int linkchk(FTSENT *); static void usage(void); -void prthumanval(int64_t); -void ignoreadd(const char *); -void ignoreclean(void); -int ignorep(FTSENT *); +static void prthumanval(int64_t); +static void ignoreadd(const char *); +static void ignoreclean(void); +static int ignorep(FTSENT *); -int nodumpflag = 0; +static int nodumpflag = 0; +static int Aflag; +static long blocksize, cblocksize; int main(int argc, char *argv[]) { FTS *fts; FTSENT *p; - off_t savednumber = 0; - long blocksize; + off_t savednumber, curblocks; int ftsoptions; int listall; int depth; @@ -98,79 +99,91 @@ setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + lflag = Aflag = 0; save = argv; ftsoptions = 0; + savednumber = 0; + cblocksize = DEV_BSIZE; + blocksize = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { - case 'H': - Hflag = 1; - break; - case 'I': - ignoreadd(optarg); - break; - case 'L': - if (Pflag) - usage(); - Lflag = 1; - break; - case 'P': - if (Lflag) - usage(); - Pflag = 1; - break; - case 'a': - aflag = 1; - break; - case 's': - sflag = 1; - break; - case 'd': - dflag = 1; - errno = 0; - depth = atoi(optarg); - if (errno == ERANGE || depth < 0) { - warnx("invalid argument to option d: %s", optarg); - usage(); - } - break; - case 'c': - cflag = 1; - break; - case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn( - "setenv: cannot set BLOCKSIZE=512"); - hflag = 1; - break; - case 'k': - hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); - break; - case 'l': - lflag = 1; - break; - case 'm': - hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); - break; - case 'n': - nodumpflag = 1; - break; - case 'r': /* Compatibility. */ - break; - case 'x': - ftsoptions |= FTS_XDEV; - break; - case '?': - default: + case 'A': + Aflag = 1; + break; + case 'B': + errno = 0; + cblocksize = atoi(optarg); + if (errno == ERANGE || cblocksize < 0) { + warnx("invalid argument to option B: %s", + optarg); usage(); + } + break; + case 'H': + Hflag = 1; + break; + case 'I': + ignoreadd(optarg); + break; + case 'L': + if (Pflag) + usage(); + Lflag = 1; + break; + case 'P': + if (Lflag) + usage(); + Pflag = 1; + break; + case 'a': + aflag = 1; + break; + case 's': + sflag = 1; + break; + case 'd': + dflag = 1; + errno = 0; + depth = atoi(optarg); + if (errno == ERANGE || depth < 0) { + warnx("invalid argument to option d: %s", + optarg); + usage(); + } + break; + case 'c': + cflag = 1; + break; + case 'h': + hflag = 1; + break; + case 'k': + hflag = 0; + blocksize = 1024; + break; + case 'l': + lflag = 1; + break; + case 'm': + hflag = 0; + blocksize = 1048576; + break; + case 'n': + nodumpflag = 1; + break; + case 'r': /* Compatibility. */ + break; + case 'x': + ftsoptions |= FTS_XDEV; + break; + case '?': + default: + usage(); + /* NOTREACHED */ } argc -= optind; @@ -204,6 +217,9 @@ if (Pflag) ftsoptions |= FTS_PHYSICAL; + if (!Aflag && (cblocksize % DEV_BSIZE) != 0) + cblocksize = howmany(cblocksize, DEV_BSIZE) * DEV_BSIZE; + listall = 0; if (aflag) { @@ -222,9 +238,14 @@ argv[1] = NULL; } - (void) getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void)getbsize(¬used, &blocksize); + if (!Aflag) { + cblocksize /= DEV_BSIZE; + blocksize /= DEV_BSIZE; + } + rval = 0; if ((fts = fts_open(argv, ftsoptions, NULL)) == NULL) @@ -232,57 +253,63 @@ while ((p = fts_read(fts)) != NULL) { switch (p->fts_info) { - case FTS_D: /* Ignore. */ - if (ignorep(p)) - fts_set(fts, p, FTS_SKIP); + case FTS_D: /* Ignore. */ + if (ignorep(p)) + fts_set(fts, p, FTS_SKIP); + break; + case FTS_DP: + if (ignorep(p)) break; - case FTS_DP: - if (ignorep(p)) - break; - p->fts_parent->fts_bignum += - p->fts_bignum += p->fts_statp->st_blocks; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); + p->fts_parent->fts_bignum += p->fts_bignum += + curblocks; - if (p->fts_level <= depth) { - if (hflag) { - (void) prthumanval(howmany(p->fts_bignum, blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, blocksize), - p->fts_path); - } + if (p->fts_level <= depth) { + if (hflag) { + prthumanval(p->fts_bignum); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(p->fts_bignum * cblocksize, + blocksize), p->fts_path); } + } + break; + case FTS_DC: /* Ignore. */ + break; + case FTS_DNR: /* Warn, continue. */ + case FTS_ERR: + case FTS_NS: + warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); + rval = 1; + break; + default: + if (ignorep(p)) break; - case FTS_DC: /* Ignore. */ + + if (lflag == 0 && p->fts_statp->st_nlink > 1 && + linkchk(p)) break; - case FTS_DNR: /* Warn, continue. */ - case FTS_ERR: - case FTS_NS: - warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); - rval = 1; - break; - default: - if (ignorep(p)) - break; - if (lflag == 0 && - p->fts_statp->st_nlink > 1 && linkchk(p)) - break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); - if (listall || p->fts_level == 0) { - if (hflag) { - (void) prthumanval(howmany(p->fts_statp->st_blocks, - blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_statp->st_blocks, blocksize), - p->fts_path); - } + if (listall || p->fts_level == 0) { + if (hflag) { + prthumanval(curblocks); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(curblocks * cblocksize, + blocksize), p->fts_path); } + } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += curblocks; } savednumber = p->fts_parent->fts_bignum; } @@ -292,10 +319,11 @@ if (cflag) { if (hflag) { - (void) prthumanval(howmany(savednumber, blocksize)); - (void) printf("\ttotal\n"); + prthumanval(savednumber); + (void)printf("\ttotal\n"); } else { - (void) printf("%jd\ttotal\n", (intmax_t)howmany(savednumber, blocksize)); + (void)printf("%jd\ttotal\n", (intmax_t)howmany( + savednumber * cblocksize, blocksize)); } } @@ -348,7 +376,8 @@ free_list = le->next; free(le); } - new_buckets = malloc(new_size * sizeof(new_buckets[0])); + new_buckets = malloc(new_size * + sizeof(new_buckets[0])); } if (new_buckets == NULL) { @@ -436,12 +465,14 @@ return (0); } -void +static void prthumanval(int64_t bytes) { char buf[5]; - bytes *= DEV_BSIZE; + bytes *= cblocksize; + if (!Aflag) + bytes *= DEV_BSIZE; humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -453,12 +484,13 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } -void +static void ignoreadd(const char *mask) { struct ignentry *ign; @@ -472,7 +504,7 @@ SLIST_INSERT_HEAD(&ignores, ign, next); } -void +static void ignoreclean(void) { struct ignentry *ign; @@ -485,7 +517,7 @@ } } -int +static int ignorep(FTSENT *ent) { struct ignentry *ign; Index: du.1 =================================================================== --- du.1 (revision 184666) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,25 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the disk usage. +This can be helpful when operating on compressed volumes or sparse files. +.It Fl B Ar blocksize +Calculate block counts in +.Ar blocksize +byte blocks. +This is different from the +.Fl k, m +options or setting +.Ev BLOCKSIZE +and gives an estimate of how many space the examined file hierachy would +require on a filesystem with the given +.Ar blocksize . +Unless in +.Fl A +mode, +.Ar blocksize +is rounded up to the next multiple of 512. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +156,19 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that -size block. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in units of +that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in 512-byte +blocks. .El .Sh SEE ALSO .Xr df 1 , --Boundary-00=_4dQEJnq7CkB11C+--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200811050339.52088.max>