Date: Sun, 21 Apr 2019 03:43:28 +0000 (UTC) From: Kyle Evans <kevans@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r346477 - in stable/11/stand/libsa: . zfs Message-ID: <201904210343.x3L3hSZR026170@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: kevans Date: Sun Apr 21 03:43:27 2019 New Revision: 346477 URL: https://svnweb.freebsd.org/changeset/base/346477 Log: MFC r337871, r339970, r342151, r342161, r343123-r343124, r344226, r344234, r344248, r344387 r337871: pkgfs_init: Initialize pkg new_package may not set *pp if it errors out, leaving pkg uninitialized. r339970: Remove unnecessary include from libstand. r342151: loader: zfs reader should not probe partitionless disks First of all, normal setups can not boot such pools as the tools do not support installing boot programs. Secondly, for proper pool configuration detection, we need to checks all four label copies on disk, 2 from front and 2 from the end of the disk, but zfs label does not contain the size of the disk - so we depend on firmware to report the correct disk size or use information from the partition table. Without partition table, we only can rely on firmware to report and support disk IO properly. There is a specific case: 8TB disks are reported by BIOS to have 4294967295 sectors (0x00000000ffffffff), the sectors reported by OS is 15628053168 (0x00000003a3812ab0), so the reported size is less than actual but is hitting 32-bit max. Unfortuantely the real limit must be even lower because probing this disk in this system will wnd up with hung system. UEFI boot of this system seems not to be affected. r342161: loader: zfs reader should not probe partitionless disks (UEFI case) With r342151 I did fix the BIOS version of zfs_probe_dev() from accessing the whole disk, but the fix was not complete - we actually did not check if the device name was really for whole disk. Since UEFI version is only calling the zfs_probe_dev() with partitions and not with whole disk, the UEFI loader was not able to find the zfs pools. This update does correct the issue by calling archsw.arch_getdev() to translate the device name back to dev_desc, and we have whole disk when both partition and slice values are -1. r343123: loader should ignore active multi_vdev_crash_dump feature on zpool Since the loader zfs reader does not need to read the dump zvol, we can just enable the feature. illumos issue #9051 https://www.illumos.org/issues/9051 r343124: libsa: add asprintf() asprintf() is a nice tool for string processing. r344226: Fix memory corruption bug introduced in r325310 The bug occurred when a bounce buffer was used and the requested read size was greater than the size of the bounce buffer. This commit also rewrites the read logic so that it is easier to systematically verify all alignment and size cases. r344234: It turns out r344226 narrowed the overrun bug but did not eliminate it entirely This commit fixes a remaining output buffer overrun in the single-sector case when there is a non-zero tail. r344248: cd9660: dirmatch fails to unmatch when name is prefix for directory record Loader does fail to properly match the file name in directory record and does open file based on prefix match. For fix, we check the name lengths first. r344387: loader: really fix cd9660 dirmatch The cd9660_open() does pass whole path to dirmatch() and we need to compare only the current path component, not full path. Additinally, skip over duplicate / (if any) and check if the last component in the path was meant to be directory (having trailing /). If it is in fact a file, error out. Modified: stable/11/stand/libsa/cd9660.c stable/11/stand/libsa/ip.c stable/11/stand/libsa/net.c stable/11/stand/libsa/pkgfs.c stable/11/stand/libsa/printf.c stable/11/stand/libsa/stand.h stable/11/stand/libsa/udp.c stable/11/stand/libsa/zfs/zfs.c stable/11/stand/libsa/zfs/zfsimpl.c Directory Properties: stable/11/ (props changed) Modified: stable/11/stand/libsa/cd9660.c ============================================================================== --- stable/11/stand/libsa/cd9660.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/cd9660.c Sun Apr 21 03:43:27 2019 (r346477) @@ -42,6 +42,7 @@ __FBSDID("$FreeBSD$"); */ #include <sys/param.h> #include <string.h> +#include <stdbool.h> #include <sys/dirent.h> #include <fs/cd9660/iso.h> #include <fs/cd9660/cd9660_rrip.h> @@ -227,8 +228,8 @@ static int dirmatch(struct open_file *f, const char *path, struct iso_directory_record *dp, int use_rrip, int lenskip) { - size_t len; - char *cp; + size_t len, plen; + char *cp, *sep; int i, icase; if (use_rrip) @@ -241,6 +242,17 @@ dirmatch(struct open_file *f, const char *path, struct icase = 1; } else icase = 0; + + sep = strchr(path, '/'); + if (sep != NULL) { + plen = sep - path; + } else { + plen = strlen(path); + } + + if (plen != len) + return (0); + for (i = len; --i >= 0; path++, cp++) { if (!*path || *path == '/') break; @@ -279,6 +291,7 @@ cd9660_open(const char *path, struct open_file *f) struct iso_directory_record rec; struct iso_directory_record *dp = NULL; int rc, first, use_rrip, lenskip; + bool isdir = false; /* First find the volume descriptor */ buf = malloc(buf_size = ISO_DEFAULT_BLOCK_SIZE); @@ -368,7 +381,24 @@ cd9660_open(const char *path, struct open_file *f) rec = *dp; while (*path && *path != '/') /* look for next component */ path++; - if (*path) path++; /* skip '/' */ + + if (*path) /* this component was directory */ + isdir = true; + + while (*path == '/') + path++; /* skip '/' */ + + if (*path) /* We do have next component. */ + isdir = false; + } + + /* + * if the path had trailing / but the path does point to file, + * report the error ENOTDIR. + */ + if (isdir == true && (isonum_711(rec.flags) & 2) == 0) { + rc = ENOTDIR; + goto out; } /* allocate file system specific data structure */ Modified: stable/11/stand/libsa/ip.c ============================================================================== --- stable/11/stand/libsa/ip.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/ip.c Sun Apr 21 03:43:27 2019 (r346477) @@ -51,7 +51,6 @@ __FBSDID("$FreeBSD$"); #include <netinet/if_ether.h> #include <netinet/in_systm.h> -#include <netinet/in_pcb.h> #include <netinet/ip.h> #include <netinet/ip_var.h> #include <netinet/udp.h> Modified: stable/11/stand/libsa/net.c ============================================================================== --- stable/11/stand/libsa/net.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/net.c Sun Apr 21 03:43:27 2019 (r346477) @@ -48,7 +48,6 @@ __FBSDID("$FreeBSD$"); #include <netinet/if_ether.h> #include <netinet/in_systm.h> -#include <netinet/in_pcb.h> #include <netinet/ip.h> #include <netinet/ip_var.h> #include <netinet/udp.h> Modified: stable/11/stand/libsa/pkgfs.c ============================================================================== --- stable/11/stand/libsa/pkgfs.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/pkgfs.c Sun Apr 21 03:43:27 2019 (r346477) @@ -163,6 +163,7 @@ pkgfs_init(const char *pkgname, struct fs_ops *proto) struct package *pkg; int error, fd; + pkg = NULL; if (proto != &pkgfs_fsops) pkgfs_cleanup(); Modified: stable/11/stand/libsa/printf.c ============================================================================== --- stable/11/stand/libsa/printf.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/printf.c Sun Apr 21 03:43:27 2019 (r346477) @@ -122,6 +122,34 @@ snprint_func(int ch, void *arg) } int +asprintf(char **buf, const char *cfmt, ...) +{ + int retval; + struct print_buf arg; + va_list ap; + + *buf = NULL; + va_start(ap, cfmt); + retval = kvprintf(cfmt, NULL, NULL, 10, ap); + va_end(ap); + if (retval <= 0) + return (-1); + + arg.size = retval + 1; + arg.buf = *buf = malloc(arg.size); + if (*buf == NULL) + return (-1); + + va_start(ap, cfmt); + retval = kvprintf(cfmt, &snprint_func, &arg, 10, ap); + va_end(ap); + + if (arg.size >= 1) + *(arg.buf)++ = 0; + return (retval); +} + +int snprintf(char *buf, size_t size, const char *cfmt, ...) { int retval; Modified: stable/11/stand/libsa/stand.h ============================================================================== --- stable/11/stand/libsa/stand.h Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/stand.h Sun Apr 21 03:43:27 2019 (r346477) @@ -268,6 +268,7 @@ extern void *reallocf(void *ptr, size_t size); extern void mallocstats(void); extern int printf(const char *fmt, ...) __printflike(1, 2); +extern int asprintf(char **buf, const char *cfmt, ...) __printflike(2, 3); extern int sprintf(char *buf, const char *cfmt, ...) __printflike(2, 3); extern int snprintf(char *buf, size_t size, const char *cfmt, ...) __printflike(3, 4); extern int vprintf(const char *fmt, __va_list); Modified: stable/11/stand/libsa/udp.c ============================================================================== --- stable/11/stand/libsa/udp.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/udp.c Sun Apr 21 03:43:27 2019 (r346477) @@ -48,7 +48,6 @@ __FBSDID("$FreeBSD$"); #include <netinet/if_ether.h> #include <netinet/in_systm.h> -#include <netinet/in_pcb.h> #include <netinet/ip.h> #include <netinet/ip_var.h> #include <netinet/udp.h> Modified: stable/11/stand/libsa/zfs/zfs.c ============================================================================== --- stable/11/stand/libsa/zfs/zfs.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/zfs/zfs.c Sun Apr 21 03:43:27 2019 (r346477) @@ -33,15 +33,16 @@ __FBSDID("$FreeBSD$"); * Stand-alone file reading package. */ +#include <stand.h> #include <sys/disk.h> #include <sys/param.h> #include <sys/time.h> #include <sys/queue.h> +#include <disk.h> #include <part.h> #include <stddef.h> #include <stdarg.h> #include <string.h> -#include <stand.h> #include <bootstrap.h> #include "libzfs.h" @@ -362,53 +363,102 @@ static int vdev_read(vdev_t *vdev, void *priv, off_t offset, void *buf, size_t bytes) { int fd, ret; - size_t res, size, remainder, rb_size, blksz; - unsigned secsz; - off_t off; - char *bouncebuf, *rb_buf; + size_t res, head, tail, total_size, full_sec_size; + unsigned secsz, do_tail_read; + off_t start_sec; + char *outbuf, *bouncebuf; fd = (uintptr_t) priv; + outbuf = (char *) buf; bouncebuf = NULL; ret = ioctl(fd, DIOCGSECTORSIZE, &secsz); if (ret != 0) return (ret); - off = offset / secsz; - remainder = offset % secsz; - if (lseek(fd, off * secsz, SEEK_SET) == -1) - return (errno); + /* + * Handling reads of arbitrary offset and size - multi-sector case + * and single-sector case. + * + * Multi-sector Case + * (do_tail_read = true if tail > 0) + * + * |<----------------------total_size--------------------->| + * | | + * |<--head-->|<--------------bytes------------>|<--tail-->| + * | | | | + * | | |<~full_sec_size~>| | | + * +------------------+ +------------------+ + * | |0101010| . . . |0101011| | + * +------------------+ +------------------+ + * start_sec start_sec + n + * + * + * Single-sector Case + * (do_tail_read = false) + * + * |<------total_size = secsz----->| + * | | + * |<-head->|<---bytes--->|<-tail->| + * +-------------------------------+ + * | |0101010101010| | + * +-------------------------------+ + * start_sec + */ + start_sec = offset / secsz; + head = offset % secsz; + total_size = roundup2(head + bytes, secsz); + tail = total_size - (head + bytes); + do_tail_read = ((tail > 0) && (head + bytes > secsz)); + full_sec_size = total_size; + if (head > 0) + full_sec_size -= secsz; + if (do_tail_read) + full_sec_size -= secsz; - rb_buf = buf; - rb_size = bytes; - size = roundup2(bytes + remainder, secsz); - blksz = size; - if (remainder != 0 || size != bytes) { + /* Return of partial sector data requires a bounce buffer. */ + if ((head > 0) || do_tail_read) { bouncebuf = zfs_alloc(secsz); if (bouncebuf == NULL) { printf("vdev_read: out of memory\n"); return (ENOMEM); } - rb_buf = bouncebuf; - blksz = rb_size - remainder; } - while (bytes > 0) { - res = read(fd, rb_buf, rb_size); - if (res != rb_size) { + if (lseek(fd, start_sec * secsz, SEEK_SET) == -1) + return (errno); + + /* Partial data return from first sector */ + if (head > 0) { + res = read(fd, bouncebuf, secsz); + if (res != secsz) { ret = EIO; goto error; } - if (bytes < blksz) - blksz = bytes; - if (bouncebuf != NULL) - memcpy(buf, rb_buf + remainder, blksz); - buf = (void *)((uintptr_t)buf + blksz); - bytes -= blksz; - remainder = 0; - blksz = rb_size; + memcpy(outbuf, bouncebuf + head, min(secsz - head, bytes)); + outbuf += min(secsz - head, bytes); } + /* Full data return from read sectors */ + if (full_sec_size > 0) { + res = read(fd, outbuf, full_sec_size); + if (res != full_sec_size) { + ret = EIO; + goto error; + } + outbuf += full_sec_size; + } + + /* Partial data return from last sector */ + if (do_tail_read) { + res = read(fd, bouncebuf, secsz); + if (res != secsz) { + ret = EIO; + goto error; + } + memcpy(outbuf, bouncebuf, secsz - tail); + } + ret = 0; error: if (bouncebuf != NULL) @@ -517,6 +567,7 @@ zfs_probe_partition(void *arg, const char *partname, int zfs_probe_dev(const char *devname, uint64_t *pool_guid) { + struct disk_devdesc *dev; struct ptable *table; struct zfs_probe_args pa; uint64_t mediasz; @@ -527,10 +578,22 @@ zfs_probe_dev(const char *devname, uint64_t *pool_guid pa.fd = open(devname, O_RDONLY); if (pa.fd == -1) return (ENXIO); - /* Probe the whole disk */ - ret = zfs_probe(pa.fd, pool_guid); - if (ret == 0) - return (0); + /* + * We will not probe the whole disk, we can not boot from such + * disks and some systems will misreport the disk sizes and will + * hang while accessing the disk. + */ + if (archsw.arch_getdev((void **)&dev, devname, NULL) == 0) { + int partition = dev->d_partition; + int slice = dev->d_slice; + + free(dev); + if (partition != -1 && slice != -1) { + ret = zfs_probe(pa.fd, pool_guid); + if (ret == 0) + return (0); + } + } /* Probe each partition */ ret = ioctl(pa.fd, DIOCGMEDIASIZE, &mediasz); Modified: stable/11/stand/libsa/zfs/zfsimpl.c ============================================================================== --- stable/11/stand/libsa/zfs/zfsimpl.c Sun Apr 21 03:36:05 2019 (r346476) +++ stable/11/stand/libsa/zfs/zfsimpl.c Sun Apr 21 03:43:27 2019 (r346477) @@ -62,6 +62,7 @@ static const char *features_for_read[] = { "org.illumos:sha512", "org.illumos:skein", "org.zfsonlinux:large_dnode", + "com.joyent:multi_vdev_crash_dump", NULL };
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201904210343.x3L3hSZR026170>