Date: Wed, 5 Aug 1998 02:15:04 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: maex-freebsd-hackers@Space.Net (Markus Stumpf) Cc: brett@peloton.physics.montana.edu, freebsd-hackers@FreeBSD.ORG Subject: Re: couple of questions Message-ID: <199808050215.TAA01844@usr02.primenet.com> In-Reply-To: <19980805004942.L12619@space.net> from "Markus Stumpf" at Aug 5, 98 00:49:42 am
next in thread | previous in thread | raw e-mail | index | archive | help
> > - maximum file/partition size: > > > > - I've seen files (from an ls -l emailed to me) of ~ 7 GB, but > > I'm not sure what the absolute maximum file size is. Is there > > a hard limit? > > Hmmm ... this is from man 2 intro (2.2.6-RELEASE) > > > 27 EFBIG File too large. The size of a file exceeded the maximum (about > > 2.1E9 bytes). > > This is 2 GB, right? But I've heard NetBSD has bigger files. > > \Maex > > N.B. This 2 GB always hits me with backups of large partitions that don't > compress well. Still using amanda-2.3.0, however, amanda-2.4.0 is able > to handle this with artificial splits. The size limit is 549755813888 -- that's (u_quad_t) 1 << 39, or 512 Gig. The limit is a result of a 32 bit block number limitation in the file /usr/src/sbin/newfs.c.\, which in turn is because the on disk fs_size and fs_dsize are stored as int32_t. The size limit is also the size limit for an FS: /* XXX - hack to prevent overflow of a 32bit block number */ sblock.fs_maxfilesize = MIN(sblock.fs_maxfilesize, (u_quad_t) 1 << 39); The size may be further reduced if there are insufficient blocks available to support triple indirect blocks, or double indirect blocks. This number is actually incorrect; 2^31 (1 bit for sign; indirect blocks are internally implemented using negative block offsets to indicate they reference an indirect block) times 2^12 (blocks in this case are 4096 bytes, the minimum FS block size). Assuming this change, and a number of small patches to prevent 32 bit field overflow in the physical block calculation, this puts the number at 8TB for a 4k block size and 16TB for an 8k block size (2^33 and 2^34, respectively). The patches have been available for some time; I believe they were first posted a year or so ago by someone building a huge CCD array at UC Berkeley (Satoshi Asami, maybe?). I believe they haven't been incorporated because > 32 bit calculations for physical (not FS) blocks would slow things down (even then, the 39 is 2^31 * 2^8; physical blocks are 2^9, which should put the FS smack on the 1TB mark). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199808050215.TAA01844>