From owner-svn-src-head@FreeBSD.ORG Wed Feb 10 20:17:46 2010 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 937AE1065679; Wed, 10 Feb 2010 20:17:46 +0000 (UTC) (envelope-from mckusick@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 816E68FC1D; Wed, 10 Feb 2010 20:17:46 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o1AKHkVH097464; Wed, 10 Feb 2010 20:17:46 GMT (envelope-from mckusick@svn.freebsd.org) Received: (from mckusick@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o1AKHkrp097460; Wed, 10 Feb 2010 20:17:46 GMT (envelope-from mckusick@svn.freebsd.org) Message-Id: <201002102017.o1AKHkrp097460@svn.freebsd.org> From: Kirk McKusick Date: Wed, 10 Feb 2010 20:17:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r203764 - head/sbin/newfs X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2010 20:17:46 -0000 Author: mckusick Date: Wed Feb 10 20:17:46 2010 New Revision: 203764 URL: http://svn.freebsd.org/changeset/base/203764 Log: Ensure that newfs will never create a filesystem with more than 2^32 inodes by cutting back on the number of inodes per cylinder group if necessary to stay under the limit. For a default (16K block) file system, this limit begins to take effect for file systems above 32Tb. This fix is in addition to -r203763 which corrected a problem in the kernel that treated large inode numbers as negative rather than unsigned. For a default (16K block) file system, this bug began to show up at a file system size above about 16Tb. Reported by: Scott Burns, John Kilburg, Bruce Evans Followup by: Jeff Roberson PR: 133980 MFC after: 2 weeks Modified: head/sbin/newfs/mkfs.c head/sbin/newfs/newfs.c head/sbin/newfs/newfs.h Modified: head/sbin/newfs/mkfs.c ============================================================================== --- head/sbin/newfs/mkfs.c Wed Feb 10 20:10:35 2010 (r203763) +++ head/sbin/newfs/mkfs.c Wed Feb 10 20:17:46 2010 (r203764) @@ -118,6 +118,8 @@ mkfs(struct partition *pp, char *fsys) time_t utime; quad_t sizepb; int width; + ino_t maxinum; + int minfragsperinode; /* minimum ratio of frags to inodes */ char tmpbuf[100]; /* XXX this will break in about 2,500 years */ union { struct fs fdummy; @@ -170,6 +172,8 @@ mkfs(struct partition *pp, char *fsys) if (sblock.fs_avgfpdir <= 0) printf("illegal expected number of files per directory %d\n", sblock.fs_avgfpdir), exit(15); + +restart: /* * collect and verify the block and fragment sizes */ @@ -216,6 +220,8 @@ mkfs(struct partition *pp, char *fsys) sblock.fs_fsize, MAXFRAG, sblock.fs_bsize / MAXFRAG); sblock.fs_fsize = sblock.fs_bsize / MAXFRAG; } + if (maxbsize == 0) + maxbsize = bsize; if (maxbsize < bsize || !POWEROF2(maxbsize)) { sblock.fs_maxbsize = sblock.fs_bsize; printf("Extent size set to %d\n", sblock.fs_maxbsize); @@ -225,6 +231,14 @@ mkfs(struct partition *pp, char *fsys) } else { sblock.fs_maxbsize = maxbsize; } + /* + * Maxcontig sets the default for the maximum number of blocks + * that may be allocated sequentially. With file system clustering + * it is possible to allocate contiguous blocks up to the maximum + * transfer size permitted by the controller or buffering. + */ + if (maxcontig == 0) + maxcontig = MAX(1, MAXPHYS / bsize); sblock.fs_maxcontig = maxcontig; if (sblock.fs_maxcontig < sblock.fs_maxbsize / sblock.fs_bsize) { sblock.fs_maxcontig = sblock.fs_maxbsize / sblock.fs_bsize; @@ -315,9 +329,26 @@ mkfs(struct partition *pp, char *fsys) * can put into each cylinder group. If this is too big, we reduce * the density until it fits. */ + maxinum = (((int64_t)(1)) << 32) - INOPB(&sblock); + minfragsperinode = 1 + fssize / maxinum; + if (density == 0) { + density = MAX(NFPI, minfragsperinode) * fsize; + } else if (density < minfragsperinode * fsize) { + origdensity = density; + density = minfragsperinode * fsize; + fprintf(stderr, "density increased from %d to %d\n", + origdensity, density); + } origdensity = density; for (;;) { fragsperinode = MAX(numfrags(&sblock, density), 1); + if (fragsperinode < minfragsperinode) { + bsize <<= 1; + fsize <<= 1; + printf("Block size too small for a file system %s %d\n", + "of this size. Increasing blocksize to", bsize); + goto restart; + } minfpg = fragsperinode * INOPB(&sblock); if (minfpg > sblock.fs_size) minfpg = sblock.fs_size; @@ -406,7 +437,10 @@ mkfs(struct partition *pp, char *fsys) if (sblock.fs_sbsize > SBLOCKSIZE) sblock.fs_sbsize = SBLOCKSIZE; sblock.fs_minfree = minfree; - sblock.fs_maxbpg = maxbpg; + if (maxbpg == 0) + sblock.fs_maxbpg = MAXBLKPG(sblock.fs_bsize); + else + sblock.fs_maxbpg = maxbpg; sblock.fs_optim = opt; sblock.fs_cgrotor = 0; sblock.fs_pendingblocks = 0; Modified: head/sbin/newfs/newfs.c ============================================================================== --- head/sbin/newfs/newfs.c Wed Feb 10 20:10:35 2010 (r203763) +++ head/sbin/newfs/newfs.c Wed Feb 10 20:17:46 2010 (r203764) @@ -79,38 +79,6 @@ __FBSDID("$FreeBSD$"); #include "newfs.h" -/* - * The following two constants set the default block and fragment sizes. - * Both constants must be a power of 2 and meet the following constraints: - * MINBSIZE <= DESBLKSIZE <= MAXBSIZE - * sectorsize <= DESFRAGSIZE <= DESBLKSIZE - * DESBLKSIZE / DESFRAGSIZE <= 8 - */ -#define DFL_FRAGSIZE 2048 -#define DFL_BLKSIZE 16384 - -/* - * Cylinder groups may have up to MAXBLKSPERCG blocks. The actual - * number used depends upon how much information can be stored - * in a cylinder group map which must fit in a single file system - * block. The default is to use as many as possible blocks per group. - */ -#define MAXBLKSPERCG 0x7fffffff /* desired fs_fpg ("infinity") */ - -/* - * MAXBLKPG determines the maximum number of data blocks which are - * placed in a single cylinder group. The default is one indirect - * block worth of data blocks. - */ -#define MAXBLKPG(bsize) ((bsize) / sizeof(ufs2_daddr_t)) - -/* - * Each file system has a number of inodes statically allocated. - * We allocate one inode slot per NFPI fragments, expecting this - * to be far more than we will ever need. - */ -#define NFPI 4 - int Eflag; /* Erase previous disk contents */ int Lflag; /* add a volume label */ int Nflag; /* run without writing file system */ @@ -387,25 +355,11 @@ main(int argc, char *argv[]) fsize = MAX(DFL_FRAGSIZE, sectorsize); if (bsize <= 0) bsize = MIN(DFL_BLKSIZE, 8 * fsize); - if (maxbsize == 0) - maxbsize = bsize; - /* - * Maxcontig sets the default for the maximum number of blocks - * that may be allocated sequentially. With file system clustering - * it is possible to allocate contiguous blocks up to the maximum - * transfer size permitted by the controller or buffering. - */ - if (maxcontig == 0) - maxcontig = MAX(1, MAXPHYS / bsize); - if (density == 0) - density = NFPI * fsize; if (minfree < MINFREE && opt != FS_OPTSPACE) { fprintf(stderr, "Warning: changing optimization to space "); fprintf(stderr, "because minfree is less than %d%%\n", MINFREE); opt = FS_OPTSPACE; } - if (maxbpg == 0) - maxbpg = MAXBLKPG(bsize); realsectorsize = sectorsize; if (sectorsize != DEV_BSIZE) { /* XXX */ int secperblk = sectorsize / DEV_BSIZE; Modified: head/sbin/newfs/newfs.h ============================================================================== --- head/sbin/newfs/newfs.h Wed Feb 10 20:10:35 2010 (r203763) +++ head/sbin/newfs/newfs.h Wed Feb 10 20:17:46 2010 (r203764) @@ -41,6 +41,38 @@ #include /* + * The following two constants set the default block and fragment sizes. + * Both constants must be a power of 2 and meet the following constraints: + * MINBSIZE <= DESBLKSIZE <= MAXBSIZE + * sectorsize <= DESFRAGSIZE <= DESBLKSIZE + * DESBLKSIZE / DESFRAGSIZE <= 8 + */ +#define DFL_FRAGSIZE 2048 +#define DFL_BLKSIZE 16384 + +/* + * Cylinder groups may have up to MAXBLKSPERCG blocks. The actual + * number used depends upon how much information can be stored + * in a cylinder group map which must fit in a single file system + * block. The default is to use as many as possible blocks per group. + */ +#define MAXBLKSPERCG 0x7fffffff /* desired fs_fpg ("infinity") */ + +/* + * MAXBLKPG determines the maximum number of data blocks which are + * placed in a single cylinder group. The default is one indirect + * block worth of data blocks. + */ +#define MAXBLKPG(bsize) ((bsize) / sizeof(ufs2_daddr_t)) + +/* + * Each file system has a number of inodes statically allocated. + * We allocate one inode slot per NFPI fragments, expecting this + * to be far more than we will ever need. + */ +#define NFPI 4 + +/* * variables set up by front end. */ extern int Eflag; /* Erase previous disk contents */