From owner-svn-src-all@freebsd.org Mon Apr 29 18:34:12 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B69621598BBB; Mon, 29 Apr 2019 18:34:12 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B62368684; Mon, 29 Apr 2019 18:34:12 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2684A25F2E; Mon, 29 Apr 2019 18:34:12 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x3TIYB5r045410; Mon, 29 Apr 2019 18:34:12 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x3TIYBav045405; Mon, 29 Apr 2019 18:34:11 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201904291834.x3TIYBav045405@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 29 Apr 2019 18:34:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r346902 - stable/12/usr.bin/ar X-SVN-Group: stable-12 X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: stable/12/usr.bin/ar X-SVN-Commit-Revision: 346902 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 5B62368684 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.95)[-0.955,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 18:34:13 -0000 Author: emaste Date: Mon Apr 29 18:34:11 2019 New Revision: 346902 URL: https://svnweb.freebsd.org/changeset/base/346902 Log: MFC ar: implement support for /SYM64/ 64-bit archives r346079: ar: implement support for /SYM64/ 64-bit archives Submitted by: Gerald Aryeetey Reviewed by: imp (earlier) r346568: ar: test for writing 64-bit format only if symbol count is nonzero This is a minor simplification; if we do not have any symbols the empty symbol table can be in 32-bit format. r346569: ar: use array notation to access s_so This is somewhat more readable than pointer arithmetic. Also remove an unnecessary cast while here. r346582: ar: shuffle symbol offsets during conversion for 32-bit ar archives During processing we maintain symbol offsets in the 64-bit s_so array, and when writing the archive convert to 32-bit if no offsets are greater than 4GB. However, this was somewhat inefficient as we looped over the array twice: first, converting to big endian and second, writing each 32-bit value one at a time (and incorrectly so on big-endian platforms). Instead, when writing a 32-bit archive shuffle convert symbol data to big endian (as required by the ar format) and shuffle to the beginning of the allocation at the same time. Also correct emission of the symbol count on big endian platforms. Further changes are planned, but this should fix powerpc64. Reported by: jhibbits, mlinimon Reviewed by: jhibbits, Gerald Aryeetey (earlier) Tested by: jhibbits PR: 234454 Sponsored by: The FreeBSD Foundation Modified: stable/12/usr.bin/ar/ar.h stable/12/usr.bin/ar/read.c stable/12/usr.bin/ar/write.c Directory Properties: stable/12/ (props changed) Modified: stable/12/usr.bin/ar/ar.h ============================================================================== --- stable/12/usr.bin/ar/ar.h Mon Apr 29 18:28:34 2019 (r346901) +++ stable/12/usr.bin/ar/ar.h Mon Apr 29 18:34:11 2019 (r346902) @@ -100,9 +100,11 @@ struct bsdar { /* * Fields for the archive symbol table. */ - uint32_t s_cnt; /* current number of symbols. */ - uint32_t *s_so; /* symbol offset table. */ + uint64_t s_cnt; /* current number of symbols. */ + uint64_t *s_so; /* symbol offset table. */ + uint64_t s_so_max; /* maximum symbol offset. */ size_t s_so_cap; /* capacity of so table buffer. */ + char *s_sn; /* symbol name table */ size_t s_sn_cap; /* capacity of sn table buffer. */ size_t s_sn_sz; /* current size of sn table. */ Modified: stable/12/usr.bin/ar/read.c ============================================================================== --- stable/12/usr.bin/ar/read.c Mon Apr 29 18:28:34 2019 (r346901) +++ stable/12/usr.bin/ar/read.c Mon Apr 29 18:34:11 2019 (r346902) @@ -109,7 +109,8 @@ read_archive(struct bsdar *bsdar, char mode) break; /* Skip pseudo members. */ - if (strcmp(name, "/") == 0 || strcmp(name, "//") == 0) + if (strcmp(name, "/") == 0 || strcmp(name, "//") == 0 || + strcmp(name, "/SYM64/") == 0) continue; if (bsdar->argc > 0) { Modified: stable/12/usr.bin/ar/write.c ============================================================================== --- stable/12/usr.bin/ar/write.c Mon Apr 29 18:28:34 2019 (r346901) +++ stable/12/usr.bin/ar/write.c Mon Apr 29 18:34:11 2019 (r346902) @@ -50,7 +50,7 @@ __FBSDID("$FreeBSD$"); #define _ARMAG_LEN 8 /* length of ar magic string */ #define _ARHDR_LEN 60 /* length of ar header */ #define _INIT_AS_CAP 128 /* initial archive string table size */ -#define _INIT_SYMOFF_CAP (256*(sizeof(uint32_t))) /* initial so table size */ +#define _INIT_SYMOFF_CAP (256*(sizeof(uint64_t))) /* initial so table size */ #define _INIT_SYMNAME_CAP 1024 /* initial sn table size */ #define _MAXNAMELEN_SVR4 15 /* max member name length in svr4 variant */ #define _TRUNCATE_LEN 15 /* number of bytes to keep for member name */ @@ -557,6 +557,7 @@ write_cleanup(struct bsdar *bsdar) free(bsdar->s_sn); bsdar->as = NULL; bsdar->s_so = NULL; + bsdar->s_so_max = 0; bsdar->s_sn = NULL; } @@ -613,7 +614,10 @@ write_objs(struct bsdar *bsdar) struct archive_entry *entry; size_t s_sz; /* size of archive symbol table. */ size_t pm_sz; /* size of pseudo members */ - int i, nr; + size_t w_sz; /* size of words in symbol table */ + uint64_t nr; + uint32_t nr32; + int i; if (elf_version(EV_CURRENT) == EV_NONE) bsdar_errc(bsdar, EX_SOFTWARE, 0, @@ -628,9 +632,6 @@ write_objs(struct bsdar *bsdar) if (strlen(obj->name) > _MAXNAMELEN_SVR4) add_to_ar_str_table(bsdar, obj->name); bsdar->rela_off += _ARHDR_LEN + obj->size + obj->size % 2; - if (bsdar->rela_off > UINT32_MAX) - bsdar_errc(bsdar, EX_SOFTWARE, 0, - "Symbol table offset overflow"); } /* @@ -656,18 +657,31 @@ write_objs(struct bsdar *bsdar) * * absolute_offset = htobe32(relative_offset + size_of_pseudo_members) */ - + w_sz = sizeof(uint32_t); if (bsdar->s_cnt != 0) { s_sz = (bsdar->s_cnt + 1) * sizeof(uint32_t) + bsdar->s_sn_sz; pm_sz = _ARMAG_LEN + (_ARHDR_LEN + s_sz); if (bsdar->as != NULL) pm_sz += _ARHDR_LEN + bsdar->as_sz; - for (i = 0; (size_t)i < bsdar->s_cnt; i++) { - if (*(bsdar->s_so + i) > UINT32_MAX - pm_sz) - bsdar_errc(bsdar, EX_SOFTWARE, 0, - "Symbol table offset overflow"); - *(bsdar->s_so + i) = htobe32(*(bsdar->s_so + i) + - pm_sz); + /* Use the 64-bit word size format if necessary. */ + if (bsdar->s_so_max > UINT32_MAX - pm_sz) { + w_sz = sizeof(uint64_t); + pm_sz -= s_sz; + s_sz = (bsdar->s_cnt + 1) * sizeof(uint64_t) + + bsdar->s_sn_sz; + pm_sz += s_sz; + /* Convert to big-endian. */ + for (i = 0; (size_t)i < bsdar->s_cnt; i++) + bsdar->s_so[i] = + htobe64(bsdar->s_so[i] + pm_sz); + } else { + /* + * Convert to big-endian and shuffle in-place to + * the front of the allocation. XXX UB + */ + for (i = 0; (size_t)i < bsdar->s_cnt; i++) + ((uint32_t *)(bsdar->s_so))[i] = + htobe32(bsdar->s_so[i] + pm_sz); } } @@ -689,16 +703,23 @@ write_objs(struct bsdar *bsdar) if (entry == NULL) bsdar_errc(bsdar, EX_SOFTWARE, 0, "archive_entry_new failed"); - archive_entry_copy_pathname(entry, "/"); + if (w_sz == sizeof(uint64_t)) + archive_entry_copy_pathname(entry, "/SYM64/"); + else + archive_entry_copy_pathname(entry, "/"); if ((bsdar->options & AR_D) == 0) archive_entry_set_mtime(entry, time(NULL), 0); - archive_entry_set_size(entry, (bsdar->s_cnt + 1) * - sizeof(uint32_t) + bsdar->s_sn_sz); + archive_entry_set_size(entry, (bsdar->s_cnt + 1) * w_sz + + bsdar->s_sn_sz); AC(archive_write_header(a, entry)); - nr = htobe32(bsdar->s_cnt); - write_data(bsdar, a, &nr, sizeof(uint32_t)); - write_data(bsdar, a, bsdar->s_so, sizeof(uint32_t) * - bsdar->s_cnt); + if (w_sz == sizeof(uint64_t)) { + nr = htobe64(bsdar->s_cnt); + write_data(bsdar, a, &nr, sizeof(nr)); + } else { + nr32 = htobe32((uint32_t)bsdar->s_cnt); + write_data(bsdar, a, &nr32, sizeof(nr32)); + } + write_data(bsdar, a, bsdar->s_so, w_sz * bsdar->s_cnt); write_data(bsdar, a, bsdar->s_sn, bsdar->s_sn_sz); archive_entry_free(entry); } @@ -904,13 +925,15 @@ add_to_ar_sym_table(struct bsdar *bsdar, const char *n bsdar->s_sn_sz = 0; } - if (bsdar->s_cnt * sizeof(uint32_t) >= bsdar->s_so_cap) { + if (bsdar->s_cnt * sizeof(uint64_t) >= bsdar->s_so_cap) { bsdar->s_so_cap *= 2; bsdar->s_so = realloc(bsdar->s_so, bsdar->s_so_cap); if (bsdar->s_so == NULL) bsdar_errc(bsdar, EX_SOFTWARE, errno, "realloc failed"); } bsdar->s_so[bsdar->s_cnt] = bsdar->rela_off; + if ((uint64_t)bsdar->rela_off > bsdar->s_so_max) + bsdar->s_so_max = (uint64_t)bsdar->rela_off; bsdar->s_cnt++; /*