Date: Sat, 20 Aug 2005 15:09:46 +1000 (EST) From: Bruce Evans <bde@zeta.org.au> To: Mikhail Teterin <mi+mx@aldan.algebra.com> Cc: standards@FreeBSD.org, questions@FreeBSD.org Subject: Re: very big files on cd9660 file system Message-ID: <20050820142802.E60211@delplex.bde.org> In-Reply-To: <200508191942.26723.mi%2Bmx@aldan.algebra.com> References: <200508191942.26723.mi%2Bmx@aldan.algebra.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 19 Aug 2005, Mikhail Teterin wrote:
> I have a cd9660 image with several files on it. One of the files is very large
> (above 4Gb). When I mount the image, the size of this file is shown as
> realsize % 4Gb -- 758876749 bytes instead of 5053844045.
>
> What should I blame:
>
> 1) The software, that created the image (modified mkisofs)
> 2) cd9660 part of the FreeBSD kernel
> 3) ISO-9660 standard
Mostly (b). Sizes are 64 bits in the standard, but FreeBSD has always
silently discarded the highest 32 bits and corrupted the next highest
bit to a sign bit, so the file size limit is at most 2GB or 4GB
(depending on whether the sign bit gets corrupted back to a value bit).
>From cd9660_vfsops.c:
% ip->i_size = isonum_733(isodir->size);
This reads the size from the directory entry.
>From iso.h:
% u_char size [ISODCL (11, 18)]; /* 733 */
This says that the size is in bytes 11-18 (option base 1) in the directory
entry. All "733" entries are 8 bytes. The others are for other sizes and
the extent (the starting block number for a file).
% static __inline int
% isonum_733(p)
% u_char *p;
% {
% return *p|(p[1] << 8)|(p[2] << 16)|(p[3] << 24);
% }
This says that the the highest 32 bits are discarded for all "733" entries
and the sign bit in p[3] is corrupted, first by shifting it and then by
assigning the result to an int.
i_size has type long, unlike in most file systems in FreeBSD where it is
uint64_t or uint32_t, so I think the sign bit stays corrupted but doesn't
cause further problems by being converted to 33 top unsigned bits, giving
a limit of 2GB.
The file size limit is hit before the others. 31-bit block numbers with
2K-blocks work up to 4TB. There are likely to be overflow bugs at 1TB
before the 4TB limit is hit.
We still have the even closer limit of 4GB on media sizes. From
cd9660_node.c:
% ino_t
% isodirino(isodir, imp)
% struct iso_directory_record *isodir;
% struct iso_mnt *imp;
% {
% ino_t ino;
%
% ino = (isonum_733(isodir->extent) + isonum_711(isodir->ext_attr_length))
% << imp->im_bshift;
% return (ino);
% }
This fakes the inode number as the byte offset of the directory entry.
ino_t is uint32_t, so this fails if the byte offset exceeds 4GB. The
eventual 32nd bit overflows to become a sign bit in the shift but then
gets overflows back to a correct bit in the assignent, so offsets
between 2GB and 4GB work accidentally.
Since the limit is on the offsets of directory entries, media larger
than 4GB can be used for cd9660 under FreeBSD iff all directroy entries
are below the limit, which happens automatically for the non-multi-session
case only.
See revs.1.77 and 1.99 for other bugs caused by isodirino().
Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050820142802.E60211>