From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:28:02 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B976ADD1765; Tue, 8 Aug 2017 10:28:02 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 871CE7D4D7; Tue, 8 Aug 2017 10:28:02 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AS1JV022323; Tue, 8 Aug 2017 10:28:01 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78AS1Eg022322; Tue, 8 Aug 2017 10:28:01 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081028.v78AS1Eg022322@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:28:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322217 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 322217 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:28:02 -0000 Author: avg Date: Tue Aug 8 10:28:01 2017 New Revision: 322217 URL: https://svnweb.freebsd.org/changeset/base/322217 Log: 8418 zfs_prop_get_table() call in zfs_validate_name() is a no-op illumos/illumos-gate@e09ba01dcda5e24964b8632718777b39166d86e4 https://github.com/illumos/illumos-gate/commit/e09ba01dcda5e24964b8632718777b39166d86e4 https://www.illumos.org/issues/8418 The following line in zfs_validate_name() is just a no-op and it should be removed: 108 (void) zfs_prop_get_table(); Reviewed by: Vitaliy Gusev Approved by: Matthew Ahrens Author: Marcel Telka Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Tue Aug 8 05:19:45 2017 (r322216) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Tue Aug 8 10:28:01 2017 (r322217) @@ -105,7 +105,6 @@ zfs_validate_name(libzfs_handle_t *hdl, const char *pa namecheck_err_t why; char what; - (void) zfs_prop_get_table(); if (entity_namecheck(path, &why, &what) != 0) { if (hdl != NULL) { switch (why) { From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:31:44 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13C5BDD1C04; Tue, 8 Aug 2017 10:31:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE1397D96A; Tue, 8 Aug 2017 10:31:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AVgkJ023945; Tue, 8 Aug 2017 10:31:42 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78AVgTo023944; Tue, 8 Aug 2017 10:31:42 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081031.v78AVgTo023944@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:31:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322219 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Commit-Revision: 322219 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:31:44 -0000 Author: avg Date: Tue Aug 8 10:31:42 2017 New Revision: 322219 URL: https://svnweb.freebsd.org/changeset/base/322219 Log: 8416 abd.h is not C++ friendly illumos/illumos-gate@5e2a074725cb7c16ea1c6554da11ab4d6b4e7aee https://github.com/illumos/illumos-gate/commit/5e2a074725cb7c16ea1c6554da11ab4d6b4e7aee https://www.illumos.org/issues/8416 A C++ compiler fails to compile abd_is_linear(), which is an inline function defined in abd.h, with the following error: error: cannot initialize return object of type 'boolean_t' with an rvalue of type 'bool' That happens because a bool can not be converted to an enum in C++. That's a problem because abd.h can be visible through other header files that a C++ program that works with ZFS can include. Reviewed by: Igor Kozhukhov Reviewed by: Dan Kimmel Reviewed by: Alek Pinchuk Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Tue Aug 8 10:30:49 2017 (r322218) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Tue Aug 8 10:31:42 2017 (r322219) @@ -60,7 +60,7 @@ extern boolean_t zfs_abd_scatter_enabled; inline boolean_t abd_is_linear(abd_t *abd) { - return ((abd->abd_flags & ABD_FLAG_LINEAR) != 0); + return ((abd->abd_flags & ABD_FLAG_LINEAR) != 0 ? B_TRUE : B_FALSE); } /* From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:37:05 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1086CDD1FEB; Tue, 8 Aug 2017 10:37:05 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD8F07DD6F; Tue, 8 Aug 2017 10:37:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Ab47h026557; Tue, 8 Aug 2017 10:37:04 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Ab4Kl026556; Tue, 8 Aug 2017 10:37:04 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081037.v78Ab4Kl026556@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:37:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322221 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322221 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:37:05 -0000 Author: avg Date: Tue Aug 8 10:37:03 2017 New Revision: 322221 URL: https://svnweb.freebsd.org/changeset/base/322221 Log: 7910 l2arc_write_buffers() may write beyond target_sz illumos/illumos-gate@16a7e5ac116c85d965007a5f201104b564e82210 https://github.com/illumos/illumos-gate/commit/16a7e5ac116c85d965007a5f201104b564e82210 https://www.illumos.org/issues/7910 It seems that the change in issue #6950 resurrected the problem that was earlier fixed by the change in issue #5219. Please also see the following FreeBSD bug report: https://bugs.freebsd.org/ bugzilla/show_bug.cgi?id=216178 Reviewed by: George Wilson Reviewed by: Dan Kimmel Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Aug 8 10:36:07 2017 (r322220) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Aug 8 10:37:03 2017 (r322221) @@ -631,8 +631,8 @@ typedef struct arc_stats { kstat_named_t arcstat_l2_abort_lowmem; kstat_named_t arcstat_l2_cksum_bad; kstat_named_t arcstat_l2_io_error; - kstat_named_t arcstat_l2_size; - kstat_named_t arcstat_l2_asize; + kstat_named_t arcstat_l2_lsize; + kstat_named_t arcstat_l2_psize; kstat_named_t arcstat_l2_hdr_size; kstat_named_t arcstat_memory_throttle_count; kstat_named_t arcstat_meta_used; @@ -3048,19 +3048,19 @@ arc_hdr_l2hdr_destroy(arc_buf_hdr_t *hdr) { l2arc_buf_hdr_t *l2hdr = &hdr->b_l2hdr; l2arc_dev_t *dev = l2hdr->b_dev; - uint64_t asize = arc_hdr_size(hdr); + uint64_t psize = arc_hdr_size(hdr); ASSERT(MUTEX_HELD(&dev->l2ad_mtx)); ASSERT(HDR_HAS_L2HDR(hdr)); list_remove(&dev->l2ad_buflist, hdr); - ARCSTAT_INCR(arcstat_l2_asize, -asize); - ARCSTAT_INCR(arcstat_l2_size, -HDR_GET_LSIZE(hdr)); + ARCSTAT_INCR(arcstat_l2_psize, -psize); + ARCSTAT_INCR(arcstat_l2_lsize, -HDR_GET_LSIZE(hdr)); - vdev_space_update(dev->l2ad_vdev, -asize, 0, 0); + vdev_space_update(dev->l2ad_vdev, -psize, 0, 0); - (void) refcount_remove_many(&dev->l2ad_alloc, asize, hdr); + (void) refcount_remove_many(&dev->l2ad_alloc, psize, hdr); arc_hdr_clear_flags(hdr, ARC_FLAG_HAS_L2HDR); } @@ -6522,8 +6522,8 @@ top: list_remove(buflist, hdr); arc_hdr_clear_flags(hdr, ARC_FLAG_HAS_L2HDR); - ARCSTAT_INCR(arcstat_l2_asize, -arc_hdr_size(hdr)); - ARCSTAT_INCR(arcstat_l2_size, -HDR_GET_LSIZE(hdr)); + ARCSTAT_INCR(arcstat_l2_psize, -arc_hdr_size(hdr)); + ARCSTAT_INCR(arcstat_l2_lsize, -HDR_GET_LSIZE(hdr)); bytes_dropped += arc_hdr_size(hdr); (void) refcount_remove_many(&dev->l2ad_alloc, @@ -6782,7 +6782,7 @@ top: /* * This doesn't exist in the ARC. Destroy. * arc_hdr_destroy() will call list_remove() - * and decrement arcstat_l2_size. + * and decrement arcstat_l2_lsize. */ arc_change_state(arc_anon, hdr, hash_lock); arc_hdr_destroy(hdr); @@ -6824,7 +6824,7 @@ static uint64_t l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint64_t target_sz) { arc_buf_hdr_t *hdr, *hdr_prev, *head; - uint64_t write_asize, write_psize, write_sz, headroom; + uint64_t write_asize, write_psize, write_lsize, headroom; boolean_t full; l2arc_write_callback_t *cb; zio_t *pio, *wzio; @@ -6833,7 +6833,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint ASSERT3P(dev->l2ad_vdev, !=, NULL); pio = NULL; - write_sz = write_asize = write_psize = 0; + write_lsize = write_asize = write_psize = 0; full = B_FALSE; head = kmem_cache_alloc(hdr_l2only_cache, KM_PUSHPAGE); arc_hdr_set_flags(head, ARC_FLAG_L2_WRITE_HEAD | ARC_FLAG_HAS_L2HDR); @@ -6890,7 +6890,22 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint continue; } - if ((write_asize + HDR_GET_LSIZE(hdr)) > target_sz) { + /* + * We rely on the L1 portion of the header below, so + * it's invalid for this header to have been evicted out + * of the ghost cache, prior to being written out. The + * ARC_FLAG_L2_WRITING bit ensures this won't happen. + */ + ASSERT(HDR_HAS_L1HDR(hdr)); + + ASSERT3U(HDR_GET_PSIZE(hdr), >, 0); + ASSERT3P(hdr->b_l1hdr.b_pabd, !=, NULL); + ASSERT3U(arc_hdr_size(hdr), >, 0); + uint64_t psize = arc_hdr_size(hdr); + uint64_t asize = vdev_psize_to_asize(dev->l2ad_vdev, + psize); + + if ((write_asize + asize) > target_sz) { full = B_TRUE; mutex_exit(hash_lock); break; @@ -6923,21 +6938,8 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint list_insert_head(&dev->l2ad_buflist, hdr); mutex_exit(&dev->l2ad_mtx); - /* - * We rely on the L1 portion of the header below, so - * it's invalid for this header to have been evicted out - * of the ghost cache, prior to being written out. The - * ARC_FLAG_L2_WRITING bit ensures this won't happen. - */ - ASSERT(HDR_HAS_L1HDR(hdr)); + (void) refcount_add_many(&dev->l2ad_alloc, psize, hdr); - ASSERT3U(HDR_GET_PSIZE(hdr), >, 0); - ASSERT3P(hdr->b_l1hdr.b_pabd, !=, NULL); - ASSERT3U(arc_hdr_size(hdr), >, 0); - uint64_t size = arc_hdr_size(hdr); - - (void) refcount_add_many(&dev->l2ad_alloc, size, hdr); - /* * Normally the L2ARC can use the hdr's data, but if * we're sharing data between the hdr and one of its @@ -6952,20 +6954,18 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint * lifetime of the ZIO and be cleaned up afterwards, we * add it to the l2arc_free_on_write queue. */ - uint64_t asize = vdev_psize_to_asize(dev->l2ad_vdev, - size); abd_t *to_write; - if (!HDR_SHARED_DATA(hdr) && size == asize) { + if (!HDR_SHARED_DATA(hdr) && psize == asize) { to_write = hdr->b_l1hdr.b_pabd; } else { to_write = abd_alloc_for_io(asize, HDR_ISTYPE_METADATA(hdr)); - abd_copy(to_write, hdr->b_l1hdr.b_pabd, size); - if (asize != size) { - abd_zero_off(to_write, size, - asize - size); + abd_copy(to_write, hdr->b_l1hdr.b_pabd, psize); + if (asize != psize) { + abd_zero_off(to_write, psize, + asize - psize); } - l2arc_free_abd_on_write(to_write, size, + l2arc_free_abd_on_write(to_write, asize, arc_buf_type(hdr)); } wzio = zio_write_phys(pio, dev->l2ad_vdev, @@ -6974,12 +6974,12 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint ZIO_PRIORITY_ASYNC_WRITE, ZIO_FLAG_CANFAIL, B_FALSE); - write_sz += HDR_GET_LSIZE(hdr); + write_lsize += HDR_GET_LSIZE(hdr); DTRACE_PROBE2(l2arc__write, vdev_t *, dev->l2ad_vdev, zio_t *, wzio); - write_asize += size; - write_psize += asize; + write_psize += psize; + write_asize += asize; dev->l2ad_hand += asize; mutex_exit(hash_lock); @@ -6995,7 +6995,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint /* No buffers selected for writing? */ if (pio == NULL) { - ASSERT0(write_sz); + ASSERT0(write_lsize); ASSERT(!HDR_HAS_L1HDR(head)); kmem_cache_free(hdr_l2only_cache, head); return (0); @@ -7003,10 +7003,10 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint ASSERT3U(write_asize, <=, target_sz); ARCSTAT_BUMP(arcstat_l2_writes_sent); - ARCSTAT_INCR(arcstat_l2_write_bytes, write_asize); - ARCSTAT_INCR(arcstat_l2_size, write_sz); - ARCSTAT_INCR(arcstat_l2_asize, write_asize); - vdev_space_update(dev->l2ad_vdev, write_asize, 0, 0); + ARCSTAT_INCR(arcstat_l2_write_bytes, write_psize); + ARCSTAT_INCR(arcstat_l2_lsize, write_lsize); + ARCSTAT_INCR(arcstat_l2_psize, write_psize); + vdev_space_update(dev->l2ad_vdev, write_psize, 0, 0); /* * Bump device hand to the device start if it is approaching the end. From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:44:50 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32149DD29B5; Tue, 8 Aug 2017 10:44:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C2907E519; Tue, 8 Aug 2017 10:44:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AinUp030454; Tue, 8 Aug 2017 10:44:49 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Aimom030451; Tue, 8 Aug 2017 10:44:48 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081044.v78Aimom030451@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:44:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322223 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/ztest X-SVN-Commit-Revision: 322223 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:44:50 -0000 Author: avg Date: Tue Aug 8 10:44:48 2017 New Revision: 322223 URL: https://svnweb.freebsd.org/changeset/base/322223 Log: 8378 crash due to bp in-memory modification of nopwrite block illumos/illumos-gate@b7edcb940884114e61382937505433c4c38c0278 https://github.com/illumos/illumos-gate/commit/b7edcb940884114e61382937505433c4c38c0278 https://www.illumos.org/issues/8378 The problem is that zfs_get_data() supplies a stale zgd_bp to dmu_sync(), which we then nopwrite against. zfs_get_data() doesn't hold any DMU-related locks, so after it copies db_blkptr to zgd_bp, dbuf_write_ready() could change db_blkptr, and dbuf_write_done() could remove the dirty record. dmu_sync() then sees the stale BP and that the dbuf it not dirty, so it is eligible for nop-writing. The fix is for dmu_sync() to copy db_blkptr to zgd_bp after acquiring the db_mtx. We could still see a stale db_blkptr, but if it is stale then the dirty record will still exist and thus we won't attempt to nopwrite. Reviewed by: Prakash Surya Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/ztest/ztest.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Tue Aug 8 10:43:41 2017 (r322222) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Tue Aug 8 10:44:48 2017 (r322223) @@ -1574,6 +1574,7 @@ dmu_sync_done(zio_t *zio, arc_buf_t *buf, void *varg) uint8_t chksum = BP_GET_CHECKSUM(bp_orig); ASSERT(BP_EQUAL(bp, bp_orig)); + VERIFY(BP_EQUAL(bp, db->db_blkptr)); ASSERT(zio->io_prop.zp_compress != ZIO_COMPRESS_OFF); ASSERT(zio_checksum_table[chksum].ci_flags & ZCHECKSUM_FLAG_NOPWRITE); @@ -1614,19 +1615,11 @@ dmu_sync_late_arrival_done(zio_t *zio) blkptr_t *bp_orig = &zio->io_bp_orig; if (zio->io_error == 0 && !BP_IS_HOLE(bp)) { - /* - * If we didn't allocate a new block (i.e. ZIO_FLAG_NOPWRITE) - * then there is nothing to do here. Otherwise, free the - * newly allocated block in this txg. - */ - if (zio->io_flags & ZIO_FLAG_NOPWRITE) { - ASSERT(BP_EQUAL(bp, bp_orig)); - } else { - ASSERT(BP_IS_HOLE(bp_orig) || !BP_EQUAL(bp, bp_orig)); - ASSERT(zio->io_bp->blk_birth == zio->io_txg); - ASSERT(zio->io_txg > spa_syncing_txg(zio->io_spa)); - zio_free(zio->io_spa, zio->io_txg, zio->io_bp); - } + ASSERT(!(zio->io_flags & ZIO_FLAG_NOPWRITE)); + ASSERT(BP_IS_HOLE(bp_orig) || !BP_EQUAL(bp, bp_orig)); + ASSERT(zio->io_bp->blk_birth == zio->io_txg); + ASSERT(zio->io_txg > spa_syncing_txg(zio->io_spa)); + zio_free(zio->io_spa, zio->io_txg, zio->io_bp); } dmu_tx_commit(dsa->dsa_tx); @@ -1658,6 +1651,29 @@ dmu_sync_late_arrival(zio_t *pio, objset_t *os, dmu_sy dsa->dsa_zgd = zgd; dsa->dsa_tx = tx; + /* + * Since we are currently syncing this txg, it's nontrivial to + * determine what BP to nopwrite against, so we disable nopwrite. + * + * When syncing, the db_blkptr is initially the BP of the previous + * txg. We can not nopwrite against it because it will be changed + * (this is similar to the non-late-arrival case where the dbuf is + * dirty in a future txg). + * + * Then dbuf_write_ready() sets bp_blkptr to the location we will write. + * We can not nopwrite against it because although the BP will not + * (typically) be changed, the data has not yet been persisted to this + * location. + * + * Finally, when dbuf_write_done() is called, it is theoretically + * possible to always nopwrite, because the data that was written in + * this txg is the same data that we are trying to write. However we + * would need to check that this dbuf is not dirty in any future + * txg's (as we do in the normal dmu_sync() path). For simplicity, we + * don't nopwrite in this case. + */ + zp->zp_nopwrite = B_FALSE; + zio_nowait(zio_write(pio, os->os_spa, dmu_tx_get_txg(tx), zgd->zgd_bp, abd_get_from_buf(zgd->zgd_db->db_data, zgd->zgd_db->db_size), zgd->zgd_db->db_size, zgd->zgd_db->db_size, zp, @@ -1695,7 +1711,6 @@ dmu_sync_late_arrival(zio_t *pio, objset_t *os, dmu_sy int dmu_sync(zio_t *pio, uint64_t txg, dmu_sync_cb_t *done, zgd_t *zgd) { - blkptr_t *bp = zgd->zgd_bp; dmu_buf_impl_t *db = (dmu_buf_impl_t *)zgd->zgd_db; objset_t *os = db->db_objset; dsl_dataset_t *ds = os->os_dsl_dataset; @@ -1762,6 +1777,21 @@ dmu_sync(zio_t *pio, uint64_t txg, dmu_sync_cb_t *done ASSERT(dr->dr_next == NULL || dr->dr_next->dr_txg < txg); + if (db->db_blkptr != NULL) { + /* + * We need to fill in zgd_bp with the current blkptr so that + * the nopwrite code can check if we're writing the same + * data that's already on disk. We can only nopwrite if we + * are sure that after making the copy, db_blkptr will not + * change until our i/o completes. We ensure this by + * holding the db_mtx, and only allowing nopwrite if the + * block is not already dirty (see below). This is verified + * by dmu_sync_done(), which VERIFYs that the db_blkptr has + * not changed. + */ + *zgd->zgd_bp = *db->db_blkptr; + } + /* * Assume the on-disk data is X, the current syncing data (in * txg - 1) is Y, and the current in-memory data is Z (currently @@ -1813,7 +1843,7 @@ dmu_sync(zio_t *pio, uint64_t txg, dmu_sync_cb_t *done dsa->dsa_tx = NULL; zio_nowait(arc_write(pio, os->os_spa, txg, - bp, dr->dt.dl.dr_data, DBUF_IS_L2CACHEABLE(db), + zgd->zgd_bp, dr->dt.dl.dr_data, DBUF_IS_L2CACHEABLE(db), &zp, dmu_sync_ready, NULL, NULL, dmu_sync_done, dsa, ZIO_PRIORITY_SYNC_WRITE, ZIO_FLAG_CANFAIL, &zb)); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Tue Aug 8 10:43:41 2017 (r322222) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Tue Aug 8 10:44:48 2017 (r322223) @@ -1053,7 +1053,6 @@ zfs_get_data(void *arg, lr_write_t *lr, char *buf, zio uint64_t object = lr->lr_foid; uint64_t offset = lr->lr_offset; uint64_t size = lr->lr_length; - blkptr_t *bp = &lr->lr_blkptr; dmu_buf_t *db; zgd_t *zgd; int error = 0; @@ -1130,11 +1129,7 @@ zfs_get_data(void *arg, lr_write_t *lr, char *buf, zio DMU_READ_NO_PREFETCH); if (error == 0) { - blkptr_t *obp = dmu_buf_get_blkptr(db); - if (obp) { - ASSERT(BP_IS_HOLE(bp)); - *bp = *obp; - } + blkptr_t *bp = &lr->lr_blkptr; zgd->zgd_db = db; zgd->zgd_bp = bp; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Tue Aug 8 10:43:41 2017 (r322222) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Tue Aug 8 10:44:48 2017 (r322223) @@ -24,7 +24,7 @@ * Portions Copyright 2010 Robert Milkowski * * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2014 Integros [integros.com] */ @@ -991,7 +991,6 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, zi uint64_t object = ZVOL_OBJ; uint64_t offset = lr->lr_offset; uint64_t size = lr->lr_length; /* length of user data */ - blkptr_t *bp = &lr->lr_blkptr; dmu_buf_t *db; zgd_t *zgd; int error; @@ -1019,11 +1018,7 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, zi error = dmu_buf_hold(os, object, offset, zgd, &db, DMU_READ_NO_PREFETCH); if (error == 0) { - blkptr_t *obp = dmu_buf_get_blkptr(db); - if (obp) { - ASSERT(BP_IS_HOLE(bp)); - *bp = *obp; - } + blkptr_t *bp = &lr->lr_blkptr; zgd->zgd_db = db; zgd->zgd_bp = bp; From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:44:50 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 729ACDD29BA; Tue, 8 Aug 2017 10:44:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 411AB7E51A; Tue, 8 Aug 2017 10:44:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AinQi030460; Tue, 8 Aug 2017 10:44:49 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Ain21030459; Tue, 8 Aug 2017 10:44:49 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081044.v78Ain21030459@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:44:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322223 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/ztest X-SVN-Commit-Revision: 322223 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:44:50 -0000 Author: avg Date: Tue Aug 8 10:44:48 2017 New Revision: 322223 URL: https://svnweb.freebsd.org/changeset/base/322223 Log: 8378 crash due to bp in-memory modification of nopwrite block illumos/illumos-gate@b7edcb940884114e61382937505433c4c38c0278 https://github.com/illumos/illumos-gate/commit/b7edcb940884114e61382937505433c4c38c0278 https://www.illumos.org/issues/8378 The problem is that zfs_get_data() supplies a stale zgd_bp to dmu_sync(), which we then nopwrite against. zfs_get_data() doesn't hold any DMU-related locks, so after it copies db_blkptr to zgd_bp, dbuf_write_ready() could change db_blkptr, and dbuf_write_done() could remove the dirty record. dmu_sync() then sees the stale BP and that the dbuf it not dirty, so it is eligible for nop-writing. The fix is for dmu_sync() to copy db_blkptr to zgd_bp after acquiring the db_mtx. We could still see a stale db_blkptr, but if it is stale then the dirty record will still exist and thus we won't attempt to nopwrite. Reviewed by: Prakash Surya Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor/illumos/dist/cmd/ztest/ztest.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Tue Aug 8 10:43:41 2017 (r322222) +++ vendor/illumos/dist/cmd/ztest/ztest.c Tue Aug 8 10:44:48 2017 (r322223) @@ -1842,7 +1842,6 @@ ztest_get_data(void *arg, lr_write_t *lr, char *buf, z uint64_t object = lr->lr_foid; uint64_t offset = lr->lr_offset; uint64_t size = lr->lr_length; - blkptr_t *bp = &lr->lr_blkptr; uint64_t txg = lr->lr_common.lrc_txg; uint64_t crtxg; dmu_object_info_t doi; @@ -1896,11 +1895,7 @@ ztest_get_data(void *arg, lr_write_t *lr, char *buf, z DMU_READ_NO_PREFETCH); if (error == 0) { - blkptr_t *obp = dmu_buf_get_blkptr(db); - if (obp) { - ASSERT(BP_IS_HOLE(bp)); - *bp = *obp; - } + blkptr_t *bp = &lr->lr_blkptr; zgd->zgd_db = db; zgd->zgd_bp = bp; From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:47:57 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAC62DD2D34; Tue, 8 Aug 2017 10:47:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9EE8C7EB3C; Tue, 8 Aug 2017 10:47:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AluPO030772; Tue, 8 Aug 2017 10:47:56 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78AluD1030771; Tue, 8 Aug 2017 10:47:56 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081047.v78AluD1030771@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:47:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322227 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322227 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:47:58 -0000 Author: avg Date: Tue Aug 8 10:47:56 2017 New Revision: 322227 URL: https://svnweb.freebsd.org/changeset/base/322227 Log: 8377 Panic in bookmark deletion illumos/illumos-gate@42418f9e73f0d007aa87675ecc206c26fc8e073e https://github.com/illumos/illumos-gate/commit/42418f9e73f0d007aa87675ecc206c26fc8e073e https://www.illumos.org/issues/8377 The problem is that when dsl_bookmark_destroy_check() is executed from open context (the pre-check), it fills in dbda_success based on the existence of the bookmark. But the bookmark (or containing filesystem as in this case) can be destroyed before we get to syncing context. When we re-run dsl_bookmark_destroy_check() in syncing context, it will not add the deleted bookmark to dbda_success, intending for dsl_bookmark_destroy_sync() to not process it. But because the bookmark is still in dbda_success from the open-context call, we do try to destroy it. The fix is that dsl_bookmark_destroy_check() should not modify dbda_success when called from open context. Reviewed by: Paul Dagnelie Reviewed by: Pavel Zakharov Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_bookmark.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_bookmark.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_bookmark.c Tue Aug 8 10:46:51 2017 (r322226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_bookmark.c Tue Aug 8 10:47:56 2017 (r322227) @@ -354,6 +354,9 @@ dsl_bookmark_destroy_check(void *arg, dmu_tx_t *tx) dsl_pool_t *dp = dmu_tx_pool(tx); int rv = 0; + ASSERT(nvlist_empty(dbda->dbda_success)); + ASSERT(nvlist_empty(dbda->dbda_errors)); + if (!spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_BOOKMARKS)) return (0); @@ -383,7 +386,10 @@ dsl_bookmark_destroy_check(void *arg, dmu_tx_t *tx) } } if (error == 0) { - fnvlist_add_boolean(dbda->dbda_success, fullname); + if (dmu_tx_is_syncing(tx)) { + fnvlist_add_boolean(dbda->dbda_success, + fullname); + } } else { fnvlist_add_int32(dbda->dbda_errors, fullname, error); rv = error; From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:49:57 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA5E3DD2F09; Tue, 8 Aug 2017 10:49:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9489E7EDC1; Tue, 8 Aug 2017 10:49:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Anusg030934; Tue, 8 Aug 2017 10:49:56 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78AnuJW030931; Tue, 8 Aug 2017 10:49:56 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081049.v78AnuJW030931@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:49:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322229 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib/libzfs_core/common X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib/libzfs_core/common X-SVN-Commit-Revision: 322229 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:49:57 -0000 Author: avg Date: Tue Aug 8 10:49:56 2017 New Revision: 322229 URL: https://svnweb.freebsd.org/changeset/base/322229 Log: 7600 zfs rollback should pass target snapshot to kernel illumos/illumos-gate@77b171372ed21642e04c873ef1e87fe2365520df https://github.com/illumos/illumos-gate/commit/77b171372ed21642e04c873ef1e87fe2365520df https://www.illumos.org/issues/7600 At present, the kernel side code seems to blindly rollback to whatever happens to be the latest snapshot at the time when the rollback task is processed. The expected target's name should be passed to the kernel driver and the sync task should validate that the target exists and that it is the latest snapshot indeed. Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Tue Aug 8 10:48:52 2017 (r322228) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Tue Aug 8 10:49:56 2017 (r322229) @@ -2173,6 +2173,7 @@ dsl_dataset_handoff_check(dsl_dataset_t *ds, void *own typedef struct dsl_dataset_rollback_arg { const char *ddra_fsname; + const char *ddra_tosnap; void *ddra_owner; nvlist_t *ddra_result; } dsl_dataset_rollback_arg_t; @@ -2214,6 +2215,18 @@ dsl_dataset_rollback_check(void *arg, dmu_tx_t *tx) return (SET_ERROR(EAGAIN)); } + /* + * If the expected target snapshot is specified, then check that + * the latest snapshot is it. + */ + if (ddra->ddra_tosnap != NULL) { + char namebuf[ZFS_MAX_DATASET_NAME_LEN]; + + dsl_dataset_name(ds->ds_prev, namebuf); + if (strcmp(namebuf, ddra->ddra_tosnap) != 0) + return (SET_ERROR(EXDEV)); + } + /* must not have any bookmarks after the most recent snapshot */ nvlist_t *proprequest = fnvlist_alloc(); fnvlist_add_boolean(proprequest, zfs_prop_to_name(ZFS_PROP_CREATETXG)); @@ -2315,11 +2328,13 @@ dsl_dataset_rollback_sync(void *arg, dmu_tx_t *tx) * notes above zfs_suspend_fs() for further details. */ int -dsl_dataset_rollback(const char *fsname, void *owner, nvlist_t *result) +dsl_dataset_rollback(const char *fsname, const char *tosnap, void *owner, + nvlist_t *result) { dsl_dataset_rollback_arg_t ddra; ddra.ddra_fsname = fsname; + ddra.ddra_tosnap = tosnap; ddra.ddra_owner = owner; ddra.ddra_result = result; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Tue Aug 8 10:48:52 2017 (r322228) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Tue Aug 8 10:49:56 2017 (r322229) @@ -334,7 +334,8 @@ void dsl_dataset_set_refreservation_sync_impl(dsl_data void dsl_dataset_zapify(dsl_dataset_t *ds, dmu_tx_t *tx); boolean_t dsl_dataset_is_zapified(dsl_dataset_t *ds); boolean_t dsl_dataset_has_resume_receive_state(dsl_dataset_t *ds); -int dsl_dataset_rollback(const char *fsname, void *owner, nvlist_t *result); +int dsl_dataset_rollback(const char *fsname, const char *tosnap, void *owner, + nvlist_t *result); void dsl_dataset_deactivate_feature(uint64_t dsobj, spa_feature_t f, dmu_tx_t *tx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Tue Aug 8 10:48:52 2017 (r322228) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Tue Aug 8 10:49:56 2017 (r322229) @@ -3636,18 +3636,29 @@ zfs_ioc_destroy(zfs_cmd_t *zc) /* * fsname is name of dataset to rollback (to most recent snapshot) * - * innvl is not used. + * innvl may contain name of expected target snapshot * * outnvl: "target" -> name of most recent snapshot * } */ /* ARGSUSED */ static int -zfs_ioc_rollback(const char *fsname, nvlist_t *args, nvlist_t *outnvl) +zfs_ioc_rollback(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) { zfsvfs_t *zfsvfs; + char *target = NULL; int error; + (void) nvlist_lookup_string(innvl, "target", &target); + if (target != NULL) { + int fslen = strlen(fsname); + + if (strncmp(fsname, target, fslen) != 0) + return (SET_ERROR(EINVAL)); + if (target[fslen] != '@') + return (SET_ERROR(EINVAL)); + } + if (getzfsvfs(fsname, &zfsvfs) == 0) { dsl_dataset_t *ds; @@ -3656,13 +3667,14 @@ zfs_ioc_rollback(const char *fsname, nvlist_t *args, n if (error == 0) { int resume_err; - error = dsl_dataset_rollback(fsname, zfsvfs, outnvl); + error = dsl_dataset_rollback(fsname, target, zfsvfs, + outnvl); resume_err = zfs_resume_fs(zfsvfs, ds); error = error ? error : resume_err; } VFS_RELE(zfsvfs->z_vfs); } else { - error = dsl_dataset_rollback(fsname, NULL, outnvl); + error = dsl_dataset_rollback(fsname, target, NULL, outnvl); } return (error); } From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:49:58 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D2CDDD2F10; Tue, 8 Aug 2017 10:49:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 06CFF7EDC2; Tue, 8 Aug 2017 10:49:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78AnvsQ030943; Tue, 8 Aug 2017 10:49:57 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Anv3f030940; Tue, 8 Aug 2017 10:49:57 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081049.v78Anv3f030940@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:49:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322229 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib/libzfs_core/common X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib/libzfs_core/common X-SVN-Commit-Revision: 322229 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:49:58 -0000 Author: avg Date: Tue Aug 8 10:49:56 2017 New Revision: 322229 URL: https://svnweb.freebsd.org/changeset/base/322229 Log: 7600 zfs rollback should pass target snapshot to kernel illumos/illumos-gate@77b171372ed21642e04c873ef1e87fe2365520df https://github.com/illumos/illumos-gate/commit/77b171372ed21642e04c873ef1e87fe2365520df https://www.illumos.org/issues/7600 At present, the kernel side code seems to blindly rollback to whatever happens to be the latest snapshot at the time when the rollback task is processed. The expected target's name should be passed to the kernel driver and the sync task should validate that the target exists and that it is the latest snapshot indeed. Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Tue Aug 8 10:48:52 2017 (r322228) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Tue Aug 8 10:49:56 2017 (r322229) @@ -3912,14 +3912,19 @@ zfs_rollback(zfs_handle_t *zhp, zfs_handle_t *snap, bo } /* - * We rely on zfs_iter_children() to verify that there are no - * newer snapshots for the given dataset. Therefore, we can - * simply pass the name on to the ioctl() call. There is still - * an unlikely race condition where the user has taken a - * snapshot since we verified that this was the most recent. + * Pass both the filesystem and the wanted snapshot names, + * we would get an error back if the snapshot is destroyed or + * a new snapshot is created before this request is processed. */ - err = lzc_rollback(zhp->zfs_name, NULL, 0); - if (err != 0) { + err = lzc_rollback_to(zhp->zfs_name, snap->zfs_name); + if (err == EXDEV) { + zfs_error_aux(zhp->zfs_hdl, dgettext(TEXT_DOMAIN, + "'%s' is not the latest snapshot"), snap->zfs_name); + (void) zfs_error_fmt(zhp->zfs_hdl, EZFS_BUSY, + dgettext(TEXT_DOMAIN, "cannot rollback '%s'"), + zhp->zfs_name); + return (err); + } else if (err != 0) { (void) zfs_standard_error_fmt(zhp->zfs_hdl, errno, dgettext(TEXT_DOMAIN, "cannot rollback '%s'"), zhp->zfs_name); Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Tue Aug 8 10:48:52 2017 (r322228) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Tue Aug 8 10:49:56 2017 (r322229) @@ -728,6 +728,9 @@ lzc_receive_with_header(const char *snapname, nvlist_t * Roll back this filesystem or volume to its most recent snapshot. * If snapnamebuf is not NULL, it will be filled in with the name * of the most recent snapshot. + * Note that the latest snapshot may change if a new one is concurrently + * created or the current one is destroyed. lzc_rollback_to can be used + * to roll back to a specific latest snapshot. * * Return 0 on success or an errno on failure. */ @@ -747,6 +750,27 @@ lzc_rollback(const char *fsname, char *snapnamebuf, in } nvlist_free(result); + return (err); +} + +/* + * Roll back this filesystem or volume to the specified snapshot, + * if possible. + * + * Return 0 on success or an errno on failure. + */ +int +lzc_rollback_to(const char *fsname, const char *snapname) +{ + nvlist_t *args; + nvlist_t *result; + int err; + + args = fnvlist_alloc(); + fnvlist_add_string(args, "target", snapname); + err = lzc_ioctl(ZFS_IOC_ROLLBACK, fsname, args, &result); + nvlist_free(args); + nvlist_free(result); return (err); } Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Tue Aug 8 10:48:52 2017 (r322228) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Tue Aug 8 10:49:56 2017 (r322229) @@ -84,6 +84,7 @@ int lzc_receive_with_header(const char *, nvlist_t *, boolean_t lzc_exists(const char *); int lzc_rollback(const char *, char *, int); +int lzc_rollback_to(const char *, const char *); #ifdef __cplusplus } From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:55:43 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2C3FDD34DC; Tue, 8 Aug 2017 10:55:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD3257F484; Tue, 8 Aug 2017 10:55:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Atgnr034781; Tue, 8 Aug 2017 10:55:42 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78AtgCF034780; Tue, 8 Aug 2017 10:55:42 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081055.v78AtgCF034780@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:55:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322231 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 322231 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:55:44 -0000 Author: avg Date: Tue Aug 8 10:55:42 2017 New Revision: 322231 URL: https://svnweb.freebsd.org/changeset/base/322231 Log: 8430 dir_is_empty_readdir() doesn't properly handle error from fdopendir() illumos/illumos-gate@ba6e7e6505150388de6dc6a88741164118a421bf https://github.com/illumos/illumos-gate/commit/ba6e7e6505150388de6dc6a88741164118a421bf https://www.illumos.org/issues/8430 we should close dirfd if fdopendir() fails. Reviewed by: Serapheim Dimitropoulos Reviewed by: Matthew Ahrens Reviewed by: Dan Kimmel Reviewed by: Yuri Pankov Reviewed by: Igor Kozhukhov Approved by: Robert Mustacchi Author: Sowrabha Gopal Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Tue Aug 8 10:52:01 2017 (r322230) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Tue Aug 8 10:55:42 2017 (r322231) @@ -209,6 +209,7 @@ dir_is_empty_readdir(const char *dirname) } if ((dirp = fdopendir(dirfd)) == NULL) { + (void) close(dirfd); return (B_TRUE); } From owner-svn-src-vendor@freebsd.org Tue Aug 8 10:58:03 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 434B9DD36CB; Tue, 8 Aug 2017 10:58:03 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FE8B7F5EA; Tue, 8 Aug 2017 10:58:02 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Aw2Z1034905; Tue, 8 Aug 2017 10:58:02 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Aw2Sr034904; Tue, 8 Aug 2017 10:58:02 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081058.v78Aw2Sr034904@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 10:58:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322232 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Commit-Revision: 322232 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 10:58:03 -0000 Author: avg Date: Tue Aug 8 10:58:01 2017 New Revision: 322232 URL: https://svnweb.freebsd.org/changeset/base/322232 Log: 8426 mark immutable buffer arguments as such in abd.h illumos/illumos-gate@9b195260e22529ac0e2580faaf89402420589c1c https://github.com/illumos/illumos-gate/commit/9b195260e22529ac0e2580faaf89402420589c1c https://www.illumos.org/issues/8426 abd_copy_from_buf and abd_cmp_buf do not modify their void *buf arguments, so qualify them with const. abd_copy_from_buf_off and abd_cmp_buf_off already had that type for the corresponding arguments. Reviewed by: Matt Ahrens Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Tue Aug 8 10:55:42 2017 (r322231) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/abd.h Tue Aug 8 10:58:01 2017 (r322232) @@ -113,7 +113,7 @@ abd_copy(abd_t *dabd, abd_t *sabd, size_t size) } inline void -abd_copy_from_buf(abd_t *abd, void *buf, size_t size) +abd_copy_from_buf(abd_t *abd, const void *buf, size_t size) { abd_copy_from_buf_off(abd, buf, 0, size); } @@ -125,7 +125,7 @@ abd_copy_to_buf(void* buf, abd_t *abd, size_t size) } inline int -abd_cmp_buf(abd_t *abd, void *buf, size_t size) +abd_cmp_buf(abd_t *abd, const void *buf, size_t size) { return (abd_cmp_buf_off(abd, buf, 0, size)); } From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:10:39 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26B47DD4133; Tue, 8 Aug 2017 11:10:39 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3F8E7FEEE; Tue, 8 Aug 2017 11:10:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BAc37040874; Tue, 8 Aug 2017 11:10:38 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BAbEI040622; Tue, 8 Aug 2017 11:10:37 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081110.v78BAbEI040622@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:10:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322235 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 322235 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:10:39 -0000 Author: avg Date: Tue Aug 8 11:10:37 2017 New Revision: 322235 URL: https://svnweb.freebsd.org/changeset/base/322235 Log: 8067 zdb should be able to dump literal embedded block pointer illumos/illumos-gate@4923c69fddc0887da5604a262585af3efd82ee20 https://github.com/illumos/illumos-gate/commit/4923c69fddc0887da5604a262585af3efd82ee20 https://www.illumos.org/issues/8067 Add an option to zdb to print a literal embedded block pointer supplied on the command line: zdb -E [-A] word0:word1:...:word15 Reviewed by: George Wilson Reviewed by: Alex Reece Reviewed by: Yuri Pankov Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/man/man1m/zdb.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/blkptr.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/blkptr.h Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Tue Aug 8 11:07:34 2017 (r322234) +++ vendor/illumos/dist/cmd/zdb/zdb.c Tue Aug 8 11:10:37 2017 (r322235) @@ -61,6 +61,7 @@ #include #include #include +#include #include #undef verify #include @@ -134,10 +135,11 @@ usage(void) "\t%s -O \n" "\t%s -R [-A] [-e [-V] [-p ...]] [-U ]\n" "\t\t ::[:]\n" + "\t%s -E [-A] word0:word1:...:word15\n" "\t%s -S [-AP] [-e [-V] [-p ...]] [-U ] " "\n\n", cmdname, cmdname, cmdname, cmdname, cmdname, cmdname, cmdname, - cmdname); + cmdname, cmdname); (void) fprintf(stderr, " Dataset name must include at least one " "separator character '/' or '@'\n"); @@ -152,6 +154,8 @@ usage(void) (void) fprintf(stderr, " -C config (or cachefile if alone)\n"); (void) fprintf(stderr, " -d dataset(s)\n"); (void) fprintf(stderr, " -D dedup statistics\n"); + (void) fprintf(stderr, " -E decode and display block from an " + "embedded block pointer\n"); (void) fprintf(stderr, " -h pool history\n"); (void) fprintf(stderr, " -i intent logs\n"); (void) fprintf(stderr, " -l read label contents\n"); @@ -3625,6 +3629,33 @@ out: free(dup); } +static void +zdb_embedded_block(char *thing) +{ + blkptr_t bp = { 0 }; + unsigned long long *words = (void *)&bp; + char buf[SPA_MAXBLOCKSIZE]; + int err; + + err = sscanf(thing, "%llx:%llx:%llx:%llx:%llx:%llx:%llx:%llx:" + "%llx:%llx:%llx:%llx:%llx:%llx:%llx:%llx", + words + 0, words + 1, words + 2, words + 3, + words + 4, words + 5, words + 6, words + 7, + words + 8, words + 9, words + 10, words + 11, + words + 12, words + 13, words + 14, words + 15); + if (err != 16) { + (void) printf("invalid input format\n"); + exit(1); + } + ASSERT3U(BPE_GET_LSIZE(&bp), <=, SPA_MAXBLOCKSIZE); + err = decode_embedded_bp(&bp, buf, BPE_GET_LSIZE(&bp)); + if (err != 0) { + (void) printf("decode failed: %u\n", err); + exit(1); + } + zdb_dump_block_raw(buf, BPE_GET_LSIZE(&bp), 0); +} + static boolean_t pool_match(nvlist_t *cfg, char *tgt) { @@ -3743,13 +3774,14 @@ main(int argc, char **argv) spa_config_path = spa_config_path_env; while ((c = getopt(argc, argv, - "AbcCdDeFGhiI:lLmMo:Op:PqRsSt:uU:vVx:X")) != -1) { + "AbcCdDeEFGhiI:lLmMo:Op:PqRsSt:uU:vVx:X")) != -1) { switch (c) { case 'b': case 'c': case 'C': case 'd': case 'D': + case 'E': case 'G': case 'h': case 'i': @@ -3813,6 +3845,12 @@ main(int argc, char **argv) break; case 'U': spa_config_path = optarg; + if (spa_config_path[0] != '/') { + (void) fprintf(stderr, + "cachefile must be an absolute path " + "(i.e. start with a slash)\n"); + usage(); + } break; case 'v': verbose++; @@ -3860,7 +3898,7 @@ main(int argc, char **argv) verbose = MAX(verbose, 1); for (c = 0; c < 256; c++) { - if (dump_all && strchr("AeFlLOPRSX", c) == NULL) + if (dump_all && strchr("AeEFlLOPRSX", c) == NULL) dump_opt[c] = 1; if (dump_opt[c]) dump_opt[c] += verbose; @@ -3874,6 +3912,14 @@ main(int argc, char **argv) if (argc < 2 && dump_opt['R']) usage(); + + if (dump_opt['E']) { + if (argc != 1) + usage(); + zdb_embedded_block(argv[0]); + return (0); + } + if (argc < 1) { if (!dump_opt['e'] && dump_opt['C']) { dump_cachefile(spa_config_path); Modified: vendor/illumos/dist/man/man1m/zdb.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zdb.1m Tue Aug 8 11:07:34 2017 (r322234) +++ vendor/illumos/dist/man/man1m/zdb.1m Tue Aug 8 11:10:37 2017 (r322235) @@ -10,10 +10,10 @@ .\" .\" .\" Copyright 2012, Richard Lowe. -.\" Copyright (c) 2012, 2016 by Delphix. All rights reserved. +.\" Copyright (c) 2012, 2017 by Delphix. All rights reserved. .\" Copyright 2017 Nexenta Systems, Inc. .\" -.Dd January 14, 2017 +.Dd April 14, 2017 .Dt ZDB 1M .Os .Sh NAME @@ -39,6 +39,10 @@ .Op Fl A .Op Fl U Ar cache .Nm +.Fl E +.Op Fl A +.Ar word0 Ns : Ns Ar word1 Ns :...: Ns Ar word15 +.Nm .Fl l .Op Fl Aqu .Ar device @@ -153,6 +157,10 @@ Display the statistics independently for each deduplic Dump the contents of the deduplication tables describing duplicate blocks. .It Fl DDDDD Also dump the contents of the deduplication tables describing unique blocks. +.It Fl E Ar word0 Ns : Ns Ar word1 Ns :...: Ns Ar word15 +Decode and display block from an embedded block pointer specified by the +.Ar word +arguments. .It Fl h Display pool history similar to .Nm zpool Cm history , From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:10:38 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABC97DD412B; Tue, 8 Aug 2017 11:10:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8847D7FEED; Tue, 8 Aug 2017 11:10:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BAbam040242; Tue, 8 Aug 2017 11:10:37 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BAbbc040031; Tue, 8 Aug 2017 11:10:37 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081110.v78BAbbc040031@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:10:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322235 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 322235 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:10:38 -0000 Author: avg Date: Tue Aug 8 11:10:37 2017 New Revision: 322235 URL: https://svnweb.freebsd.org/changeset/base/322235 Log: 8067 zdb should be able to dump literal embedded block pointer illumos/illumos-gate@4923c69fddc0887da5604a262585af3efd82ee20 https://github.com/illumos/illumos-gate/commit/4923c69fddc0887da5604a262585af3efd82ee20 https://www.illumos.org/issues/8067 Add an option to zdb to print a literal embedded block pointer supplied on the command line: zdb -E [-A] word0:word1:...:word15 Reviewed by: George Wilson Reviewed by: Alex Reece Reviewed by: Yuri Pankov Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/blkptr.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/blkptr.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/man/man1m/zdb.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/blkptr.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/blkptr.c Tue Aug 8 11:07:34 2017 (r322234) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/blkptr.c Tue Aug 8 11:10:37 2017 (r322235) @@ -117,3 +117,36 @@ decode_embedded_bp_compressed(const blkptr_t *bp, void buf8[i] = BF64_GET(w, (i % sizeof (w)) * NBBY, NBBY); } } + +/* + * Fill in the buffer with the (decompressed) payload of the embedded + * blkptr_t. Takes into account compression and byteorder (the payload is + * treated as a stream of bytes). + * Return 0 on success, or ENOSPC if it won't fit in the buffer. + */ +int +decode_embedded_bp(const blkptr_t *bp, void *buf, int buflen) +{ + int lsize, psize; + + ASSERT(BP_IS_EMBEDDED(bp)); + + lsize = BPE_GET_LSIZE(bp); + psize = BPE_GET_PSIZE(bp); + + if (lsize > buflen) + return (ENOSPC); + ASSERT3U(lsize, ==, buflen); + + if (BP_GET_COMPRESS(bp) != ZIO_COMPRESS_OFF) { + uint8_t dstbuf[BPE_PAYLOAD_SIZE]; + decode_embedded_bp_compressed(bp, dstbuf); + VERIFY0(zio_decompress_data_buf(BP_GET_COMPRESS(bp), + dstbuf, buf, psize, buflen)); + } else { + ASSERT3U(lsize, ==, psize); + decode_embedded_bp_compressed(bp, buf); + } + + return (0); +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/blkptr.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/blkptr.h Tue Aug 8 11:07:34 2017 (r322234) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/blkptr.h Tue Aug 8 11:10:37 2017 (r322235) @@ -30,6 +30,7 @@ extern "C" { void encode_embedded_bp_compressed(blkptr_t *, void *, enum zio_compress, int, int); void decode_embedded_bp_compressed(const blkptr_t *, void *); +int decode_embedded_bp(const blkptr_t *, void *, int); #ifdef __cplusplus } From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:13:29 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 092EDDD454A; Tue, 8 Aug 2017 11:13:29 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CACB280578; Tue, 8 Aug 2017 11:13:28 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BDR3O043187; Tue, 8 Aug 2017 11:13:27 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BDRba043186; Tue, 8 Aug 2017 11:13:27 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081113.v78BDRba043186@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:13:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322236 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322236 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:13:29 -0000 Author: avg Date: Tue Aug 8 11:13:27 2017 New Revision: 322236 URL: https://svnweb.freebsd.org/changeset/base/322236 Log: 8126 ztest assertion failed in dbuf_dirty due to dn_nlevels changing illumos/illumos-gate@dcb6872c565819ac88acbc2ece999ef241c8b982 https://github.com/illumos/illumos-gate/commit/dcb6872c565819ac88acbc2ece999ef241c8b982 https://www.illumos.org/issues/8126 The sync thread is concurrently modifying dn_phys->dn_nlevels while dbuf_dirty() is trying to assert something about it, without holding the necessary lock. We need to move this assertion further down in the function, after we have acquired the dn_struct_rwlock. Reviewed by: Pavel Zakharov Reviewed by: Serapheim Dimitropoulos Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Aug 8 11:10:37 2017 (r322235) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Aug 8 11:13:27 2017 (r322236) @@ -1542,11 +1542,6 @@ dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t *tx) (dmu_tx_is_syncing(tx) ? DN_DIRTY_SYNC : DN_DIRTY_OPEN)); ASSERT3U(dn->dn_nlevels, >, db->db_level); - ASSERT((dn->dn_phys->dn_nlevels == 0 && db->db_level == 0) || - dn->dn_phys->dn_nlevels > db->db_level || - dn->dn_next_nlevels[txgoff] > db->db_level || - dn->dn_next_nlevels[(tx->tx_txg-1) & TXG_MASK] > db->db_level || - dn->dn_next_nlevels[(tx->tx_txg-2) & TXG_MASK] > db->db_level); /* * We should only be dirtying in syncing context if it's the @@ -1662,6 +1657,16 @@ dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t *tx) rw_enter(&dn->dn_struct_rwlock, RW_READER); drop_struct_lock = TRUE; } + + /* + * We need to hold the dn_struct_rwlock to make this assertion, + * because it protects dn_phys / dn_next_nlevels from changing. + */ + ASSERT((dn->dn_phys->dn_nlevels == 0 && db->db_level == 0) || + dn->dn_phys->dn_nlevels > db->db_level || + dn->dn_next_nlevels[txgoff] > db->db_level || + dn->dn_next_nlevels[(tx->tx_txg-1) & TXG_MASK] > db->db_level || + dn->dn_next_nlevels[(tx->tx_txg-2) & TXG_MASK] > db->db_level); /* * If we are overwriting a dedup BP, then unless it is snapshotted, From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:15:37 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7EEEDD47AC; Tue, 8 Aug 2017 11:15:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B8C78088F; Tue, 8 Aug 2017 11:15:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BFaKk043360; Tue, 8 Aug 2017 11:15:36 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BFaZu043359; Tue, 8 Aug 2017 11:15:36 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081115.v78BFaZu043359@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:15:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322238 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322238 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:15:37 -0000 Author: avg Date: Tue Aug 8 11:15:36 2017 New Revision: 322238 URL: https://svnweb.freebsd.org/changeset/base/322238 Log: 7915 checks in l2arc_evict could use some cleaning up illumos/illumos-gate@267ae6c3a88d2fc39276af66caafa978b0935b82 https://github.com/illumos/illumos-gate/commit/267ae6c3a88d2fc39276af66caafa978b0935b82 https://www.illumos.org/issues/7915 l2arc_evict() is strictly serialized with respect to l2arc_write_buffers() and l2arc_write_done(). Normally, l2arc_evict() and l2arc_write_buffers() are called from the same thread, so they can not be concurrent. Also, l2arc_write_buffers() uses zio_wait() on the parent zio of all cache zio- s. That ensures that l2arc_write_done() is completed before l2arc_write_buffers() returns. Finally, if a cache device is removed, then l2arc_evict() is called under SCL_ALL in the exclusive mode. That ensures that it can not be concurrent with the normal L2ARC accesses to the device (including writing and evicting buffers). Given the above, some checks and actions in l2arc_evict() do not make sense. For instance, it must never encounter the write head header let alone remove it from the buffer list. Reviewed by: Dan Kimmel Reviewed by: Prakash Surya Approved by: Matthew Ahrens Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Aug 8 11:14:40 2017 (r322237) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Aug 8 11:15:36 2017 (r322238) @@ -6755,18 +6755,16 @@ top: goto top; } - if (HDR_L2_WRITE_HEAD(hdr)) { - /* - * We hit a write head node. Leave it for - * l2arc_write_done(). - */ - list_remove(buflist, hdr); - mutex_exit(hash_lock); - continue; - } + /* + * A header can't be on this list if it doesn't have L2 header. + */ + ASSERT(HDR_HAS_L2HDR(hdr)); - if (!all && HDR_HAS_L2HDR(hdr) && - (hdr->b_l2hdr.b_daddr > taddr || + /* Ensure this header has finished being written. */ + ASSERT(!HDR_L2_WRITING(hdr)); + ASSERT(!HDR_L2_WRITE_HEAD(hdr)); + + if (!all && (hdr->b_l2hdr.b_daddr >= taddr || hdr->b_l2hdr.b_daddr < dev->l2ad_hand)) { /* * We've evicted to the target address, @@ -6776,7 +6774,6 @@ top: break; } - ASSERT(HDR_HAS_L2HDR(hdr)); if (!HDR_HAS_L1HDR(hdr)) { ASSERT(!HDR_L2_READING(hdr)); /* @@ -6798,9 +6795,6 @@ top: ARCSTAT_BUMP(arcstat_l2_evict_reading); arc_hdr_set_flags(hdr, ARC_FLAG_L2_EVICTED); } - - /* Ensure this header has finished being written */ - ASSERT(!HDR_L2_WRITING(hdr)); arc_hdr_l2hdr_destroy(hdr); } From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:19:58 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 400A0DD4B0E; Tue, 8 Aug 2017 11:19:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A4CF80C02; Tue, 8 Aug 2017 11:19:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BJvIV043590; Tue, 8 Aug 2017 11:19:57 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BJvjL043589; Tue, 8 Aug 2017 11:19:57 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081119.v78BJvjL043589@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:19:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322240 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Commit-Revision: 322240 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:19:58 -0000 Author: avg Date: Tue Aug 8 11:19:56 2017 New Revision: 322240 URL: https://svnweb.freebsd.org/changeset/base/322240 Log: 8491 uberblock on-disk padding to reserve space for smoothly merging zpool checkpoint & MMP in ZFS illumos/illumos-gate@79c2b812ee2010ebf20fdd92dc5f06b59000a94c https://github.com/illumos/illumos-gate/commit/79c2b812ee2010ebf20fdd92dc5f06b59000a94c https://www.illumos.org/issues/8491 The zpool checkpoint feature in DxOS added a new field in the uberblock. The Multi-Modifier Protection Pull Request from ZoL adds two new fields in the uberblock (Reference: https://github.com/zfsonlinux/zfs/pull/6279). As these two changes come from two different sources and once upstreamed and deployed will introduce an incompatibility with each other we want to upstream a change that will reserve the padding for both of them so integration goes smoothly and everyone gets both features. Reviewed by: Matthew Ahrens Reviewed by: Brian Behlendorf Reviewed by: Olaf Faaland Approved by: Gordon Ross Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/uberblock_impl.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/uberblock_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/uberblock_impl.h Tue Aug 8 11:19:14 2017 (r322239) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/uberblock_impl.h Tue Aug 8 11:19:56 2017 (r322240) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2017 by Delphix. All rights reserved. */ #ifndef _SYS_UBERBLOCK_IMPL_H @@ -54,6 +55,12 @@ struct uberblock { /* highest SPA_VERSION supported by software that wrote this txg */ uint64_t ub_software_version; + + /* These fields are reserved for features that are under development: */ + uint64_t ub_mmp_magic; + uint64_t ub_mmp_delay; + uint64_t ub_mmp_seq; + uint64_t ub_checkpoint_txg; }; #ifdef __cplusplus From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:24:15 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23FF4DD5041; Tue, 8 Aug 2017 11:24:15 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E18A4811DE; Tue, 8 Aug 2017 11:24:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BOEhV047367; Tue, 8 Aug 2017 11:24:14 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BOEwB047366; Tue, 8 Aug 2017 11:24:14 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081124.v78BOEwB047366@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:24:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322242 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322242 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:24:15 -0000 Author: avg Date: Tue Aug 8 11:24:13 2017 New Revision: 322242 URL: https://svnweb.freebsd.org/changeset/base/322242 Log: 8373 TXG_WAIT in ZIL commit path illumos/illumos-gate@d28671a3b094af696bea87f52272d4c4d89321c7 https://github.com/illumos/illumos-gate/commit/d28671a3b094af696bea87f52272d4c4d89321c7 https://www.illumos.org/issues/8373 The code that writes ZIL blocks uses dmu_tx_assign(TXG_WAIT) to assign a transaction to a transaction group. That seems to be logically incorrect as writing of the ZIL block does not introduce any new dirty data. Also, when there is a lot of dirty data, the call can introduce significant delays into the ZIL commit path, thus affecting all synchronous writes. Additionally, ARC throttling may affect the ZIL writing. We probably need a new mechanism similar to dmu_tx_create_assigned to assign ZIL transactions. (Ab)using TXG_WAITED does not seem to be sufficient. Reviewed by: Matthew Ahrens Reviewed by: Prakash Surya Approved by: Dan McDonald Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Tue Aug 8 11:21:58 2017 (r322241) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Tue Aug 8 11:24:13 2017 (r322242) @@ -974,7 +974,24 @@ zil_lwb_write_start(zilog_t *zilog, lwb_t *lwb) * to clean up in the event of allocation failure or I/O failure. */ tx = dmu_tx_create(zilog->zl_os); - VERIFY(dmu_tx_assign(tx, TXG_WAIT) == 0); + + /* + * Since we are not going to create any new dirty data and we can even + * help with clearing the existing dirty data, we should not be subject + * to the dirty data based delays. + * We (ab)use TXG_WAITED to bypass the delay mechanism. + * One side effect from using TXG_WAITED is that dmu_tx_assign() can + * fail if the pool is suspended. Those are dramatic circumstances, + * so we return NULL to signal that the normal ZIL processing is not + * possible and txg_wait_synced() should be used to ensure that the data + * is on disk. + */ + error = dmu_tx_assign(tx, TXG_WAITED); + if (error != 0) { + ASSERT3S(error, ==, EIO); + dmu_tx_abort(tx); + return (NULL); + } dsl_dataset_dirty(dmu_objset_ds(zilog->zl_os), tx); txg = dmu_tx_get_txg(tx); From owner-svn-src-vendor@freebsd.org Tue Aug 8 11:27:20 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7565ADD5428; Tue, 8 Aug 2017 11:27:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4379F81775; Tue, 8 Aug 2017 11:27:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78BRJSr047659; Tue, 8 Aug 2017 11:27:19 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78BRJtu047658; Tue, 8 Aug 2017 11:27:19 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201708081127.v78BRJtu047658@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Tue, 8 Aug 2017 11:27:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322246 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 322246 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 11:27:20 -0000 Author: avg Date: Tue Aug 8 11:27:19 2017 New Revision: 322246 URL: https://svnweb.freebsd.org/changeset/base/322246 Log: 8508 Mounting a zpool on 32-bit platforms panics illumos/illumos-gate@b11fe8c01471a5bff68e83e1fe5f809ad16b3be8 https://github.com/illumos/illumos-gate/commit/b11fe8c01471a5bff68e83e1fe5f809ad16b3be8 https://www.illumos.org/issues/8508 Mounting a zpool on a 32-bit system triggers a panic in spa_history_log_version () due to a type format mismatch for ZPL_VERSION. ZPL_VERSION is a unsigned long long, but the format expects an integer. On 64-bit platforms this may not be an issue due to word size and alignment. On 32-bit platforms a word size is half that of a long long, causing the second word of the long long to be seen as the string pointer for utsname.nodename. Reviewed by: Matt Ahrens Approved by: Dan McDonald Author: Justin Hibbits Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_history.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_history.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_history.c Tue Aug 8 11:26:03 2017 (r322245) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_history.c Tue Aug 8 11:27:19 2017 (r322246) @@ -614,7 +614,7 @@ void spa_history_log_version(spa_t *spa, const char *operation) { spa_history_log_internal(spa, operation, NULL, - "pool version %llu; software version %llu/%d; uts %s %s %s %s", + "pool version %llu; software version %llu/%llu; uts %s %s %s %s", (u_longlong_t)spa_version(spa), SPA_VERSION, ZPL_VERSION, utsname.nodename, utsname.release, utsname.version, utsname.machine); From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:52:58 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5111DC2F7D; Tue, 8 Aug 2017 16:52:58 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3EF7069A9A; Tue, 8 Aug 2017 16:52:58 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Gqvn2082492; Tue, 8 Aug 2017 16:52:57 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Gqre2082459; Tue, 8 Aug 2017 16:52:53 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081652.v78Gqre2082459@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:52:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322259 - in vendor/llvm/dist: bindings/ocaml/llvm cmake/modules docs include/llvm/Analysis include/llvm/CodeGen lib/Analysis lib/CodeGen lib/CodeGen/SelectionDAG lib/IR lib/Object lib/... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/llvm/dist: bindings/ocaml/llvm cmake/modules docs include/llvm/Analysis include/llvm/CodeGen lib/Analysis lib/CodeGen lib/CodeGen/SelectionDAG lib/IR lib/Object lib/Target/AArch64 lib/Target... X-SVN-Commit-Revision: 322259 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:52:58 -0000 Author: dim Date: Tue Aug 8 16:52:53 2017 New Revision: 322259 URL: https://svnweb.freebsd.org/changeset/base/322259 Log: Vendor import of llvm release_50 branch r310316: https://llvm.org/svn/llvm-project/llvm/branches/release_50@310316 Added: vendor/llvm/dist/test/CodeGen/Mips/cconv/pr33883.ll vendor/llvm/dist/test/CodeGen/Mips/pr33978.ll vendor/llvm/dist/test/CodeGen/X86/conditional-tailcall-samedest.mir vendor/llvm/dist/test/CodeGen/X86/pause.ll vendor/llvm/dist/test/CodeGen/X86/tail-call-mutable-memarg.ll vendor/llvm/dist/test/Instrumentation/AddressSanitizer/force-dynamic-shadow.ll vendor/llvm/dist/test/Transforms/InstSimplify/pr33957.ll vendor/llvm/dist/test/Transforms/SCCP/definite-initializer.ll Modified: vendor/llvm/dist/bindings/ocaml/llvm/llvm.ml vendor/llvm/dist/bindings/ocaml/llvm/llvm.mli vendor/llvm/dist/bindings/ocaml/llvm/llvm_ocaml.c vendor/llvm/dist/cmake/modules/AddOCaml.cmake vendor/llvm/dist/docs/ReleaseNotes.rst vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h vendor/llvm/dist/include/llvm/CodeGen/MachineFunction.h vendor/llvm/dist/include/llvm/CodeGen/MachineInstr.h vendor/llvm/dist/lib/Analysis/ValueTracking.cpp vendor/llvm/dist/lib/CodeGen/BranchFolding.cpp vendor/llvm/dist/lib/CodeGen/MachineFunction.cpp vendor/llvm/dist/lib/CodeGen/MachineInstr.cpp vendor/llvm/dist/lib/CodeGen/ScheduleDAGInstrs.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp vendor/llvm/dist/lib/CodeGen/StackColoring.cpp vendor/llvm/dist/lib/IR/ConstantFold.cpp vendor/llvm/dist/lib/Object/COFFImportFile.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64FrameLowering.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.td vendor/llvm/dist/lib/Target/AArch64/AArch64MacroFusion.cpp vendor/llvm/dist/lib/Target/X86/X86ISelLowering.cpp vendor/llvm/dist/lib/Target/X86/X86InstrSSE.td vendor/llvm/dist/lib/Transforms/IPO/ArgumentPromotion.cpp vendor/llvm/dist/lib/Transforms/Instrumentation/AddressSanitizer.cpp vendor/llvm/dist/lib/Transforms/Scalar/SCCP.cpp vendor/llvm/dist/lib/Transforms/Utils/CloneModule.cpp vendor/llvm/dist/test/CodeGen/AArch64/aarch64_win64cc_vararg.ll vendor/llvm/dist/test/CodeGen/AArch64/misched-fusion-aes.ll vendor/llvm/dist/test/CodeGen/AArch64/tbi.ll vendor/llvm/dist/test/CodeGen/AArch64/win64_vararg.ll vendor/llvm/dist/test/DllTool/coff-weak-exports.def vendor/llvm/dist/test/Instrumentation/AddressSanitizer/stack-poisoning-byval-args.ll vendor/llvm/dist/test/MC/AArch64/arm64-crypto.s vendor/llvm/dist/test/Transforms/ArgumentPromotion/byval.ll vendor/llvm/dist/unittests/Transforms/Utils/Cloning.cpp Modified: vendor/llvm/dist/bindings/ocaml/llvm/llvm.ml ============================================================================== --- vendor/llvm/dist/bindings/ocaml/llvm/llvm.ml Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/bindings/ocaml/llvm/llvm.ml Tue Aug 8 16:52:53 2017 (r322259) @@ -20,6 +20,10 @@ type llattribute type llmemorybuffer type llmdkind +exception FeatureDisabled of string + +let () = Callback.register_exception "Llvm.FeatureDisabled" (FeatureDisabled "") + module TypeKind = struct type t = | Void Modified: vendor/llvm/dist/bindings/ocaml/llvm/llvm.mli ============================================================================== --- vendor/llvm/dist/bindings/ocaml/llvm/llvm.mli Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/bindings/ocaml/llvm/llvm.mli Tue Aug 8 16:52:53 2017 (r322259) @@ -371,6 +371,8 @@ type ('a, 'b) llrev_pos = (** {6 Exceptions} *) +exception FeatureDisabled of string + exception IoError of string Modified: vendor/llvm/dist/bindings/ocaml/llvm/llvm_ocaml.c ============================================================================== --- vendor/llvm/dist/bindings/ocaml/llvm/llvm_ocaml.c Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/bindings/ocaml/llvm/llvm_ocaml.c Tue Aug 8 16:52:53 2017 (r322259) @@ -336,7 +336,12 @@ CAMLprim LLVMContextRef llvm_type_context(LLVMTypeRef /* lltype -> unit */ CAMLprim value llvm_dump_type(LLVMTypeRef Val) { +#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) LLVMDumpType(Val); +#else + caml_raise_with_arg(*caml_named_value("Llvm.FeatureDisabled"), + caml_copy_string("dump")); +#endif return Val_unit; } Modified: vendor/llvm/dist/cmake/modules/AddOCaml.cmake ============================================================================== --- vendor/llvm/dist/cmake/modules/AddOCaml.cmake Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/cmake/modules/AddOCaml.cmake Tue Aug 8 16:52:53 2017 (r322259) @@ -87,6 +87,11 @@ function(add_ocaml_library name) foreach( include_dir ${LLVM_INCLUDE_DIR} ${LLVM_MAIN_INCLUDE_DIR} ) set(c_flags "${c_flags} -I${include_dir}") endforeach() + # include -D/-UNDEBUG to match dump function visibility + # regex from HandleLLVMOptions.cmake + string(REGEX MATCH "(^| )[/-][UD] *NDEBUG($| )" flag_matches + "${CMAKE_C_FLAGS_${uppercase_CMAKE_BUILD_TYPE}} ${CMAKE_C_FLAGS}") + set(c_flags "${c_flags} ${flag_matches}") foreach( ocaml_file ${ARG_OCAML} ) list(APPEND sources "${ocaml_file}.mli" "${ocaml_file}.ml") @@ -199,7 +204,7 @@ function(add_ocaml_library name) PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE - DESTINATION "${LLVM_OCAML_INSTALL_PATH}/llvm") + DESTINATION "${LLVM_OCAML_INSTALL_PATH}/stublibs") foreach( install_file ${install_files} ${install_shlibs} ) get_filename_component(filename "${install_file}" NAME) Modified: vendor/llvm/dist/docs/ReleaseNotes.rst ============================================================================== --- vendor/llvm/dist/docs/ReleaseNotes.rst Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/docs/ReleaseNotes.rst Tue Aug 8 16:52:53 2017 (r322259) @@ -125,7 +125,22 @@ Changes to the AMDGPU Target Changes to the AVR Target ----------------------------- - During this release ... +This release consists mainly of bugfixes and implementations of features +required for compiling basic Rust programs. + +* Enable the branch relaxation pass so that we don't crash on large + stack load/stores + +* Add support for lowering bit-rotations to the native `ror` and `rol` + instructions + +* Fix bug where function pointers were treated as pointers to RAM and not + pointers to program memory + +* Fix broken code generaton for shift-by-variable expressions + +* Support zero-sized types in argument lists; this is impossible in C, + but possible in Rust Changes to the OCaml bindings ----------------------------- Modified: vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h Tue Aug 8 16:52:53 2017 (r322259) @@ -312,6 +312,12 @@ template class ArrayRef; const DataLayout &DL, LoopInfo *LI = nullptr, unsigned MaxLookup = 6); + /// This is a wrapper around GetUnderlyingObjects and adds support for basic + /// ptrtoint+arithmetic+inttoptr sequences. + void getUnderlyingObjectsForCodeGen(const Value *V, + SmallVectorImpl &Objects, + const DataLayout &DL); + /// Return true if the only users of this pointer are lifetime markers. bool onlyUsedByLifetimeMarkers(const Value *V); Modified: vendor/llvm/dist/include/llvm/CodeGen/MachineFunction.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/MachineFunction.h Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/include/llvm/CodeGen/MachineFunction.h Tue Aug 8 16:52:53 2017 (r322259) @@ -661,6 +661,12 @@ class MachineFunction { (public) MachineMemOperand *getMachineMemOperand(const MachineMemOperand *MMO, int64_t Offset, uint64_t Size); + /// Allocate a new MachineMemOperand by copying an existing one, + /// replacing only AliasAnalysis information. MachineMemOperands are owned + /// by the MachineFunction and need not be explicitly deallocated. + MachineMemOperand *getMachineMemOperand(const MachineMemOperand *MMO, + const AAMDNodes &AAInfo); + using OperandCapacity = ArrayRecycler::Capacity; /// Allocate an array of MachineOperands. This is only intended for use by Modified: vendor/llvm/dist/include/llvm/CodeGen/MachineInstr.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/MachineInstr.h Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/include/llvm/CodeGen/MachineInstr.h Tue Aug 8 16:52:53 2017 (r322259) @@ -379,6 +379,9 @@ class MachineInstr (public) return NumMemRefs == 1; } + /// Return the number of memory operands. + unsigned getNumMemOperands() const { return NumMemRefs; } + /// API for querying MachineInstr properties. They are the same as MCInstrDesc /// queries but they are bundle aware. Modified: vendor/llvm/dist/lib/Analysis/ValueTracking.cpp ============================================================================== --- vendor/llvm/dist/lib/Analysis/ValueTracking.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Analysis/ValueTracking.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -3277,6 +3277,69 @@ void llvm::GetUnderlyingObjects(Value *V, SmallVectorI } while (!Worklist.empty()); } +/// This is the function that does the work of looking through basic +/// ptrtoint+arithmetic+inttoptr sequences. +static const Value *getUnderlyingObjectFromInt(const Value *V) { + do { + if (const Operator *U = dyn_cast(V)) { + // If we find a ptrtoint, we can transfer control back to the + // regular getUnderlyingObjectFromInt. + if (U->getOpcode() == Instruction::PtrToInt) + return U->getOperand(0); + // If we find an add of a constant, a multiplied value, or a phi, it's + // likely that the other operand will lead us to the base + // object. We don't have to worry about the case where the + // object address is somehow being computed by the multiply, + // because our callers only care when the result is an + // identifiable object. + if (U->getOpcode() != Instruction::Add || + (!isa(U->getOperand(1)) && + Operator::getOpcode(U->getOperand(1)) != Instruction::Mul && + !isa(U->getOperand(1)))) + return V; + V = U->getOperand(0); + } else { + return V; + } + assert(V->getType()->isIntegerTy() && "Unexpected operand type!"); + } while (true); +} + +/// This is a wrapper around GetUnderlyingObjects and adds support for basic +/// ptrtoint+arithmetic+inttoptr sequences. +void llvm::getUnderlyingObjectsForCodeGen(const Value *V, + SmallVectorImpl &Objects, + const DataLayout &DL) { + SmallPtrSet Visited; + SmallVector Working(1, V); + do { + V = Working.pop_back_val(); + + SmallVector Objs; + GetUnderlyingObjects(const_cast(V), Objs, DL); + + for (Value *V : Objs) { + if (!Visited.insert(V).second) + continue; + if (Operator::getOpcode(V) == Instruction::IntToPtr) { + const Value *O = + getUnderlyingObjectFromInt(cast(V)->getOperand(0)); + if (O->getType()->isPointerTy()) { + Working.push_back(O); + continue; + } + } + // If GetUnderlyingObjects fails to find an identifiable object, + // getUnderlyingObjectsForCodeGen also fails for safety. + if (!isIdentifiedObject(V)) { + Objects.clear(); + return; + } + Objects.push_back(const_cast(V)); + } + } while (!Working.empty()); +} + /// Return true if the only users of this pointer are lifetime markers. bool llvm::onlyUsedByLifetimeMarkers(const Value *V) { for (const User *U : V->users()) { Modified: vendor/llvm/dist/lib/CodeGen/BranchFolding.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/BranchFolding.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/BranchFolding.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -1475,13 +1475,14 @@ ReoptimizeBlock: bool PredAnalyzable = !TII->analyzeBranch(*Pred, PredTBB, PredFBB, PredCond, true); - if (PredAnalyzable && !PredCond.empty() && PredTBB == MBB) { + if (PredAnalyzable && !PredCond.empty() && PredTBB == MBB && + PredTBB != PredFBB) { // The predecessor has a conditional branch to this block which consists // of only a tail call. Try to fold the tail call into the conditional // branch. if (TII->canMakeTailCallConditional(PredCond, TailCall)) { // TODO: It would be nice if analyzeBranch() could provide a pointer - // to the branch insturction so replaceBranchWithTailCall() doesn't + // to the branch instruction so replaceBranchWithTailCall() doesn't // have to search for it. TII->replaceBranchWithTailCall(*Pred, PredCond, TailCall); ++NumTailCalls; Modified: vendor/llvm/dist/lib/CodeGen/MachineFunction.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/MachineFunction.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/MachineFunction.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -330,6 +330,20 @@ MachineFunction::getMachineMemOperand(const MachineMem MMO->getOrdering(), MMO->getFailureOrdering()); } +MachineMemOperand * +MachineFunction::getMachineMemOperand(const MachineMemOperand *MMO, + const AAMDNodes &AAInfo) { + MachinePointerInfo MPI = MMO->getValue() ? + MachinePointerInfo(MMO->getValue(), MMO->getOffset()) : + MachinePointerInfo(MMO->getPseudoValue(), MMO->getOffset()); + + return new (Allocator) + MachineMemOperand(MPI, MMO->getFlags(), MMO->getSize(), + MMO->getBaseAlignment(), AAInfo, + MMO->getRanges(), MMO->getSyncScopeID(), + MMO->getOrdering(), MMO->getFailureOrdering()); +} + MachineInstr::mmo_iterator MachineFunction::allocateMemRefsArray(unsigned long Num) { return Allocator.Allocate(Num); Modified: vendor/llvm/dist/lib/CodeGen/MachineInstr.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/MachineInstr.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/MachineInstr.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -578,10 +578,8 @@ bool MachinePointerInfo::isDereferenceable(unsigned Si if (BasePtr == nullptr) return false; - return isDereferenceableAndAlignedPointer(BasePtr, 1, - APInt(DL.getPointerSize(), - Offset + Size), - DL); + return isDereferenceableAndAlignedPointer( + BasePtr, 1, APInt(DL.getPointerSizeInBits(), Offset + Size), DL); } /// getConstantPool - Return a MachinePointerInfo record that refers to the Modified: vendor/llvm/dist/lib/CodeGen/ScheduleDAGInstrs.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/ScheduleDAGInstrs.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/ScheduleDAGInstrs.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -121,63 +121,6 @@ ScheduleDAGInstrs::ScheduleDAGInstrs(MachineFunction & SchedModel.init(ST.getSchedModel(), &ST, TII); } -/// This is the function that does the work of looking through basic -/// ptrtoint+arithmetic+inttoptr sequences. -static const Value *getUnderlyingObjectFromInt(const Value *V) { - do { - if (const Operator *U = dyn_cast(V)) { - // If we find a ptrtoint, we can transfer control back to the - // regular getUnderlyingObjectFromInt. - if (U->getOpcode() == Instruction::PtrToInt) - return U->getOperand(0); - // If we find an add of a constant, a multiplied value, or a phi, it's - // likely that the other operand will lead us to the base - // object. We don't have to worry about the case where the - // object address is somehow being computed by the multiply, - // because our callers only care when the result is an - // identifiable object. - if (U->getOpcode() != Instruction::Add || - (!isa(U->getOperand(1)) && - Operator::getOpcode(U->getOperand(1)) != Instruction::Mul && - !isa(U->getOperand(1)))) - return V; - V = U->getOperand(0); - } else { - return V; - } - assert(V->getType()->isIntegerTy() && "Unexpected operand type!"); - } while (true); -} - -/// This is a wrapper around GetUnderlyingObjects and adds support for basic -/// ptrtoint+arithmetic+inttoptr sequences. -static void getUnderlyingObjects(const Value *V, - SmallVectorImpl &Objects, - const DataLayout &DL) { - SmallPtrSet Visited; - SmallVector Working(1, V); - do { - V = Working.pop_back_val(); - - SmallVector Objs; - GetUnderlyingObjects(const_cast(V), Objs, DL); - - for (Value *V : Objs) { - if (!Visited.insert(V).second) - continue; - if (Operator::getOpcode(V) == Instruction::IntToPtr) { - const Value *O = - getUnderlyingObjectFromInt(cast(V)->getOperand(0)); - if (O->getType()->isPointerTy()) { - Working.push_back(O); - continue; - } - } - Objects.push_back(const_cast(V)); - } - } while (!Working.empty()); -} - /// If this machine instr has memory reference information and it can be tracked /// to a normal reference to a known object, return the Value for that object. static void getUnderlyingObjectsForInstr(const MachineInstr *MI, @@ -208,12 +151,10 @@ static void getUnderlyingObjectsForInstr(const Machine Objects.push_back(UnderlyingObjectsVector::value_type(PSV, MayAlias)); } else if (const Value *V = MMO->getValue()) { SmallVector Objs; - getUnderlyingObjects(V, Objs, DL); + getUnderlyingObjectsForCodeGen(V, Objs, DL); for (Value *V : Objs) { - if (!isIdentifiedObject(V)) - return false; - + assert(isIdentifiedObject(V)); Objects.push_back(UnderlyingObjectsVector::value_type(V, true)); } } else Modified: vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -99,6 +99,27 @@ LimitFPPrecision("limit-float-precision", // store [4096 x i8] %data, [4096 x i8]* %buffer static const unsigned MaxParallelChains = 64; +// True if the Value passed requires ABI mangling as it is a parameter to a +// function or a return value from a function which is not an intrinsic. +static bool isABIRegCopy(const Value * V) { + const bool IsRetInst = V && isa(V); + const bool IsCallInst = V && isa(V); + const bool IsInLineAsm = + IsCallInst && static_cast(V)->isInlineAsm(); + const bool IsIndirectFunctionCall = + IsCallInst && !IsInLineAsm && + !static_cast(V)->getCalledFunction(); + // It is possible that the call instruction is an inline asm statement or an + // indirect function call in which case the return value of + // getCalledFunction() would be nullptr. + const bool IsInstrinsicCall = + IsCallInst && !IsInLineAsm && !IsIndirectFunctionCall && + static_cast(V)->getCalledFunction()->getIntrinsicID() != + Intrinsic::not_intrinsic; + + return IsRetInst || (IsCallInst && (!IsInLineAsm && !IsInstrinsicCall)); +} + static SDValue getCopyFromPartsVector(SelectionDAG &DAG, const SDLoc &DL, const SDValue *Parts, unsigned NumParts, MVT PartVT, EVT ValueVT, const Value *V, @@ -1026,13 +1047,9 @@ SDValue SelectionDAGBuilder::getCopyFromRegs(const Val if (It != FuncInfo.ValueMap.end()) { unsigned InReg = It->second; - bool IsABIRegCopy = - V && ((isa(V) && - !(static_cast(V))->isInlineAsm()) || - isa(V)); RegsForValue RFV(*DAG.getContext(), DAG.getTargetLoweringInfo(), - DAG.getDataLayout(), InReg, Ty, IsABIRegCopy); + DAG.getDataLayout(), InReg, Ty, isABIRegCopy(V)); SDValue Chain = DAG.getEntryNode(); Result = RFV.getCopyFromRegs(DAG, FuncInfo, getCurSDLoc(), Chain, nullptr, V); @@ -1221,13 +1238,9 @@ SDValue SelectionDAGBuilder::getValueImpl(const Value // If this is an instruction which fast-isel has deferred, select it now. if (const Instruction *Inst = dyn_cast(V)) { unsigned InReg = FuncInfo.InitializeRegForValue(Inst); - bool IsABIRegCopy = - V && ((isa(V) && - !(static_cast(V))->isInlineAsm()) || - isa(V)); RegsForValue RFV(*DAG.getContext(), TLI, DAG.getDataLayout(), InReg, - Inst->getType(), IsABIRegCopy); + Inst->getType(), isABIRegCopy(V)); SDValue Chain = DAG.getEntryNode(); return RFV.getCopyFromRegs(DAG, FuncInfo, getCurSDLoc(), Chain, nullptr, V); } @@ -8281,13 +8294,9 @@ SelectionDAGBuilder::CopyValueToVirtualRegister(const const TargetLowering &TLI = DAG.getTargetLoweringInfo(); // If this is an InlineAsm we have to match the registers required, not the // notional registers required by the type. - bool IsABIRegCopy = - V && ((isa(V) && - !(static_cast(V))->isInlineAsm()) || - isa(V)); RegsForValue RFV(V->getContext(), TLI, DAG.getDataLayout(), Reg, - V->getType(), IsABIRegCopy); + V->getType(), isABIRegCopy(V)); SDValue Chain = DAG.getEntryNode(); ISD::NodeType ExtendType = (FuncInfo.PreferredExtendType.find(V) == Modified: vendor/llvm/dist/lib/CodeGen/StackColoring.cpp ============================================================================== --- vendor/llvm/dist/lib/CodeGen/StackColoring.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/CodeGen/StackColoring.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -37,6 +37,7 @@ #include "llvm/CodeGen/MachineRegisterInfo.h" #include "llvm/CodeGen/Passes.h" #include "llvm/CodeGen/PseudoSourceValue.h" +#include "llvm/CodeGen/SelectionDAGNodes.h" #include "llvm/CodeGen/SlotIndexes.h" #include "llvm/CodeGen/StackProtector.h" #include "llvm/CodeGen/WinEHFuncInfo.h" @@ -889,6 +890,10 @@ void StackColoring::remapInstructions(DenseMap Allocas; + + // Keep a list of allocas which has been affected by the remap. + SmallPtrSet MergedAllocas; + for (const std::pair &SI : SlotRemap) { const AllocaInst *From = MFI->getObjectAllocation(SI.first); const AllocaInst *To = MFI->getObjectAllocation(SI.second); @@ -908,6 +913,10 @@ void StackColoring::remapInstructions(DenseMapadjustForColoring(From, To); @@ -939,13 +948,6 @@ void StackColoring::remapInstructions(DenseMap(MMO->getValue()); @@ -997,6 +999,48 @@ void StackColoring::remapInstructions(DenseMapallocateMemRefsArray(I.getNumMemOperands()); + unsigned MemOpIdx = 0; + bool ReplaceMemOps = false; + for (MachineMemOperand *MMO : I.memoperands()) { + // If this memory location can be a slot remapped here, + // we remove AA information. + bool MayHaveConflictingAAMD = false; + if (MMO->getAAInfo()) { + if (const Value *MMOV = MMO->getValue()) { + SmallVector Objs; + getUnderlyingObjectsForCodeGen(MMOV, Objs, MF->getDataLayout()); + + if (Objs.empty()) + MayHaveConflictingAAMD = true; + else + for (Value *V : Objs) { + // If this memory location comes from a known stack slot + // that is not remapped, we continue checking. + // Otherwise, we need to invalidate AA infomation. + const AllocaInst *AI = dyn_cast_or_null(V); + if (AI && MergedAllocas.count(AI)) { + MayHaveConflictingAAMD = true; + break; + } + } + } + } + if (MayHaveConflictingAAMD) { + NewMemOps[MemOpIdx++] = MF->getMachineMemOperand(MMO, AAMDNodes()); + ReplaceMemOps = true; + } + else + NewMemOps[MemOpIdx++] = MMO; + } + + // If any memory operand is updated, set memory references of + // this instruction. + if (ReplaceMemOps) + I.setMemRefs(std::make_pair(NewMemOps, I.getNumMemOperands())); } // Update the location of C++ catch objects for the MSVC personality routine. Modified: vendor/llvm/dist/lib/IR/ConstantFold.cpp ============================================================================== --- vendor/llvm/dist/lib/IR/ConstantFold.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/IR/ConstantFold.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -2097,15 +2097,19 @@ Constant *llvm::ConstantFoldGetElementPtr(Type *Pointe // Subsequent evaluation would get confused and produce erroneous results. // // The following prohibits such a GEP from being formed by checking to see - // if the index is in-range with respect to an array or vector. + // if the index is in-range with respect to an array. + // TODO: This code may be extended to handle vectors as well. bool PerformFold = false; if (Idx0->isNullValue()) PerformFold = true; else if (LastI.isSequential()) if (ConstantInt *CI = dyn_cast(Idx0)) - PerformFold = - !LastI.isBoundedSequential() || - isIndexInRangeOfArrayType(LastI.getSequentialNumElements(), CI); + PerformFold = (!LastI.isBoundedSequential() || + isIndexInRangeOfArrayType( + LastI.getSequentialNumElements(), CI)) && + !CE->getOperand(CE->getNumOperands() - 1) + ->getType() + ->isVectorTy(); if (PerformFold) { SmallVector NewIndices; Modified: vendor/llvm/dist/lib/Object/COFFImportFile.cpp ============================================================================== --- vendor/llvm/dist/lib/Object/COFFImportFile.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Object/COFFImportFile.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -542,15 +542,12 @@ NewArchiveMember ObjectFactory::createWeakExternal(Str SymbolTable[2].Name.Offset.Offset = sizeof(uint32_t); //__imp_ String Table - if (Imp) { - SymbolTable[3].Name.Offset.Offset = sizeof(uint32_t) + Sym.size() + 7; - writeStringTable(Buffer, {std::string("__imp_").append(Sym), - std::string("__imp_").append(Weak)}); - } else { - SymbolTable[3].Name.Offset.Offset = sizeof(uint32_t) + Sym.size() + 1; - writeStringTable(Buffer, {Sym, Weak}); - } + StringRef Prefix = Imp ? "__imp_" : ""; + SymbolTable[3].Name.Offset.Offset = + sizeof(uint32_t) + Sym.size() + Prefix.size() + 1; append(Buffer, SymbolTable); + writeStringTable(Buffer, {(Prefix + Sym).str(), + (Prefix + Weak).str()}); // Copied here so we can still use writeStringTable char *Buf = Alloc.Allocate(Buffer.size()); Modified: vendor/llvm/dist/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp ============================================================================== --- vendor/llvm/dist/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -946,6 +946,18 @@ bool AArch64ExpandPseudo::expandMI(MachineBasicBlock & case AArch64::CMP_SWAP_128: return expandCMP_SWAP_128(MBB, MBBI, NextMBBI); + case AArch64::AESMCrrTied: + case AArch64::AESIMCrrTied: { + MachineInstrBuilder MIB = + BuildMI(MBB, MBBI, MI.getDebugLoc(), + TII->get(Opcode == AArch64::AESMCrrTied ? AArch64::AESMCrr : + AArch64::AESIMCrr)) + .add(MI.getOperand(0)) + .add(MI.getOperand(1)); + transferImpOps(MI, MIB, MIB); + MI.eraseFromParent(); + return true; + } } return false; } Modified: vendor/llvm/dist/lib/Target/AArch64/AArch64FrameLowering.cpp ============================================================================== --- vendor/llvm/dist/lib/Target/AArch64/AArch64FrameLowering.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/AArch64/AArch64FrameLowering.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -506,19 +506,23 @@ void AArch64FrameLowering::emitPrologue(MachineFunctio return; } - auto CSStackSize = AFI->getCalleeSavedStackSize(); + bool IsWin64 = + Subtarget.isCallingConvWin64(MF.getFunction()->getCallingConv()); + unsigned FixedObject = IsWin64 ? alignTo(AFI->getVarArgsGPRSize(), 16) : 0; + + auto PrologueSaveSize = AFI->getCalleeSavedStackSize() + FixedObject; // All of the remaining stack allocations are for locals. - AFI->setLocalStackSize(NumBytes - CSStackSize); + AFI->setLocalStackSize(NumBytes - PrologueSaveSize); bool CombineSPBump = shouldCombineCSRLocalStackBump(MF, NumBytes); if (CombineSPBump) { emitFrameOffset(MBB, MBBI, DL, AArch64::SP, AArch64::SP, -NumBytes, TII, MachineInstr::FrameSetup); NumBytes = 0; - } else if (CSStackSize != 0) { + } else if (PrologueSaveSize != 0) { MBBI = convertCalleeSaveRestoreToSPPrePostIncDec(MBB, MBBI, DL, TII, - -CSStackSize); - NumBytes -= CSStackSize; + -PrologueSaveSize); + NumBytes -= PrologueSaveSize; } assert(NumBytes >= 0 && "Negative stack allocation size!?"); @@ -532,8 +536,9 @@ void AArch64FrameLowering::emitPrologue(MachineFunctio ++MBBI; } if (HasFP) { - // Only set up FP if we actually need to. Frame pointer is fp = sp - 16. - int FPOffset = CSStackSize - 16; + // Only set up FP if we actually need to. Frame pointer is fp = + // sp - fixedobject - 16. + int FPOffset = AFI->getCalleeSavedStackSize() - 16; if (CombineSPBump) FPOffset += AFI->getLocalStackSize(); @@ -672,8 +677,8 @@ void AArch64FrameLowering::emitPrologue(MachineFunctio if (HasFP) { // Define the current CFA rule to use the provided FP. unsigned Reg = RegInfo->getDwarfRegNum(FramePtr, true); - unsigned CFIIndex = MF.addFrameInst( - MCCFIInstruction::createDefCfa(nullptr, Reg, 2 * StackGrowth)); + unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::createDefCfa( + nullptr, Reg, 2 * StackGrowth - FixedObject)); BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION)) .addCFIIndex(CFIIndex) .setMIFlags(MachineInstr::FrameSetup); @@ -759,12 +764,16 @@ void AArch64FrameLowering::emitEpilogue(MachineFunctio // AArch64TargetLowering::LowerCall figures out ArgumentPopSize and keeps // it as the 2nd argument of AArch64ISD::TC_RETURN. - auto CSStackSize = AFI->getCalleeSavedStackSize(); + bool IsWin64 = + Subtarget.isCallingConvWin64(MF.getFunction()->getCallingConv()); + unsigned FixedObject = IsWin64 ? alignTo(AFI->getVarArgsGPRSize(), 16) : 0; + + auto PrologueSaveSize = AFI->getCalleeSavedStackSize() + FixedObject; bool CombineSPBump = shouldCombineCSRLocalStackBump(MF, NumBytes); - if (!CombineSPBump && CSStackSize != 0) + if (!CombineSPBump && PrologueSaveSize != 0) convertCalleeSaveRestoreToSPPrePostIncDec( - MBB, std::prev(MBB.getFirstTerminator()), DL, TII, CSStackSize); + MBB, std::prev(MBB.getFirstTerminator()), DL, TII, PrologueSaveSize); // Move past the restores of the callee-saved registers. MachineBasicBlock::iterator LastPopI = MBB.getFirstTerminator(); @@ -786,7 +795,7 @@ void AArch64FrameLowering::emitEpilogue(MachineFunctio return; } - NumBytes -= CSStackSize; + NumBytes -= PrologueSaveSize; assert(NumBytes >= 0 && "Negative stack allocation size!?"); if (!hasFP(MF)) { @@ -796,7 +805,7 @@ void AArch64FrameLowering::emitEpilogue(MachineFunctio if (RedZone && ArgumentPopSize == 0) return; - bool NoCalleeSaveRestore = CSStackSize == 0; + bool NoCalleeSaveRestore = PrologueSaveSize == 0; int StackRestoreBytes = RedZone ? 0 : NumBytes; if (NoCalleeSaveRestore) StackRestoreBytes += ArgumentPopSize; @@ -815,7 +824,8 @@ void AArch64FrameLowering::emitEpilogue(MachineFunctio // be able to save any instructions. if (MFI.hasVarSizedObjects() || AFI->isStackRealigned()) emitFrameOffset(MBB, LastPopI, DL, AArch64::SP, AArch64::FP, - -CSStackSize + 16, TII, MachineInstr::FrameDestroy); + -AFI->getCalleeSavedStackSize() + 16, TII, + MachineInstr::FrameDestroy); else if (NumBytes) emitFrameOffset(MBB, LastPopI, DL, AArch64::SP, AArch64::SP, NumBytes, TII, MachineInstr::FrameDestroy); @@ -845,7 +855,11 @@ int AArch64FrameLowering::resolveFrameIndexReference(c const AArch64RegisterInfo *RegInfo = static_cast( MF.getSubtarget().getRegisterInfo()); const AArch64FunctionInfo *AFI = MF.getInfo(); - int FPOffset = MFI.getObjectOffset(FI) + 16; + const AArch64Subtarget &Subtarget = MF.getSubtarget(); + bool IsWin64 = + Subtarget.isCallingConvWin64(MF.getFunction()->getCallingConv()); + unsigned FixedObject = IsWin64 ? alignTo(AFI->getVarArgsGPRSize(), 16) : 0; + int FPOffset = MFI.getObjectOffset(FI) + FixedObject + 16; int Offset = MFI.getObjectOffset(FI) + MFI.getStackSize(); bool isFixed = MFI.isFixedObjectIndex(FI); @@ -955,12 +969,6 @@ static void computeCalleeSaveRegisterPairs( (Count & 1) == 0) && "Odd number of callee-saved regs to spill!"); int Offset = AFI->getCalleeSavedStackSize(); - - unsigned GPRSaveSize = AFI->getVarArgsGPRSize(); - const AArch64Subtarget &Subtarget = MF.getSubtarget(); - bool IsWin64 = Subtarget.isCallingConvWin64(MF.getFunction()->getCallingConv()); - if (IsWin64) - Offset -= alignTo(GPRSaveSize, 16); for (unsigned i = 0; i < Count; ++i) { RegPairInfo RPI; Modified: vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.cpp ============================================================================== --- vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -9586,8 +9586,8 @@ static bool performTBISimplification(SDValue Addr, SelectionDAG &DAG) { APInt DemandedMask = APInt::getLowBitsSet(64, 56); KnownBits Known; - TargetLowering::TargetLoweringOpt TLO(DAG, DCI.isBeforeLegalize(), - DCI.isBeforeLegalizeOps()); + TargetLowering::TargetLoweringOpt TLO(DAG, !DCI.isBeforeLegalize(), + !DCI.isBeforeLegalizeOps()); const TargetLowering &TLI = DAG.getTargetLoweringInfo(); if (TLI.SimplifyDemandedBits(Addr, DemandedMask, Known, TLO)) { DCI.CommitTargetLoweringOpt(TLO); Modified: vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.td ============================================================================== --- vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.td Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.td Tue Aug 8 16:52:53 2017 (r322259) @@ -37,6 +37,9 @@ def HasFullFP16 : Predicate<"Subtarget->hasFullFP AssemblerPredicate<"FeatureFullFP16", "fullfp16">; def HasSPE : Predicate<"Subtarget->hasSPE()">, AssemblerPredicate<"FeatureSPE", "spe">; +def HasFuseAES : Predicate<"Subtarget->hasFuseAES()">, + AssemblerPredicate<"FeatureFuseAES", + "fuse-aes">; def HasSVE : Predicate<"Subtarget->hasSVE()">, AssemblerPredicate<"FeatureSVE", "sve">; @@ -5303,6 +5306,31 @@ def AESErr : AESTiedInst<0b0100, "aese", int_aarch def AESDrr : AESTiedInst<0b0101, "aesd", int_aarch64_crypto_aesd>; def AESMCrr : AESInst< 0b0110, "aesmc", int_aarch64_crypto_aesmc>; def AESIMCrr : AESInst< 0b0111, "aesimc", int_aarch64_crypto_aesimc>; + +// Pseudo instructions for AESMCrr/AESIMCrr with a register constraint required +// for AES fusion on some CPUs. +let hasSideEffects = 0, mayStore = 0, mayLoad = 0 in { +def AESMCrrTied: Pseudo<(outs V128:$Rd), (ins V128:$Rn), [], "$Rn = $Rd">, + Sched<[WriteV]>; +def AESIMCrrTied: Pseudo<(outs V128:$Rd), (ins V128:$Rn), [], "$Rn = $Rd">, + Sched<[WriteV]>; +} + +// Only use constrained versions of AES(I)MC instructions if they are paired with +// AESE/AESD. +def : Pat<(v16i8 (int_aarch64_crypto_aesmc + (v16i8 (int_aarch64_crypto_aese (v16i8 V128:$src1), + (v16i8 V128:$src2))))), + (v16i8 (AESMCrrTied (v16i8 (AESErr (v16i8 V128:$src1), + (v16i8 V128:$src2)))))>, + Requires<[HasFuseAES]>; + +def : Pat<(v16i8 (int_aarch64_crypto_aesimc + (v16i8 (int_aarch64_crypto_aesd (v16i8 V128:$src1), + (v16i8 V128:$src2))))), + (v16i8 (AESIMCrrTied (v16i8 (AESDrr (v16i8 V128:$src1), + (v16i8 V128:$src2)))))>, + Requires<[HasFuseAES]>; def SHA1Crrr : SHATiedInstQSV<0b000, "sha1c", int_aarch64_crypto_sha1c>; def SHA1Prrr : SHATiedInstQSV<0b001, "sha1p", int_aarch64_crypto_sha1p>; Modified: vendor/llvm/dist/lib/Target/AArch64/AArch64MacroFusion.cpp ============================================================================== --- vendor/llvm/dist/lib/Target/AArch64/AArch64MacroFusion.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/AArch64/AArch64MacroFusion.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -118,11 +118,13 @@ static bool shouldScheduleAdjacent(const TargetInstrIn // Fuse AES crypto operations. switch(SecondOpcode) { // AES encode. - case AArch64::AESMCrr : + case AArch64::AESMCrr: + case AArch64::AESMCrrTied: return FirstOpcode == AArch64::AESErr || FirstOpcode == AArch64::INSTRUCTION_LIST_END; // AES decode. case AArch64::AESIMCrr: + case AArch64::AESIMCrrTied: return FirstOpcode == AArch64::AESDrr || FirstOpcode == AArch64::INSTRUCTION_LIST_END; } Modified: vendor/llvm/dist/lib/Target/X86/X86ISelLowering.cpp ============================================================================== --- vendor/llvm/dist/lib/Target/X86/X86ISelLowering.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/X86/X86ISelLowering.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -3984,6 +3984,13 @@ bool MatchingStackOffset(SDValue Arg, unsigned Offset, if (Offset != MFI.getObjectOffset(FI)) return false; + // If this is not byval, check that the argument stack object is immutable. + // inalloca and argument copy elision can create mutable argument stack + // objects. Byval objects can be mutated, but a byval call intends to pass the + // mutated memory. + if (!Flags.isByVal() && !MFI.isImmutableObjectIndex(FI)) + return false; + if (VA.getLocVT().getSizeInBits() > Arg.getValueSizeInBits()) { // If the argument location is wider than the argument type, check that any // extension flags match. @@ -30605,8 +30612,8 @@ static SDValue combineSelect(SDNode *N, SelectionDAG & assert(BitWidth >= 8 && BitWidth <= 64 && "Invalid mask size"); APInt DemandedMask(APInt::getSignMask(BitWidth)); KnownBits Known; - TargetLowering::TargetLoweringOpt TLO(DAG, DCI.isBeforeLegalize(), - DCI.isBeforeLegalizeOps()); + TargetLowering::TargetLoweringOpt TLO(DAG, !DCI.isBeforeLegalize(), + !DCI.isBeforeLegalizeOps()); if (TLI.ShrinkDemandedConstant(Cond, DemandedMask, TLO) || TLI.SimplifyDemandedBits(Cond, DemandedMask, Known, TLO)) { // If we changed the computation somewhere in the DAG, this change will Modified: vendor/llvm/dist/lib/Target/X86/X86InstrSSE.td ============================================================================== --- vendor/llvm/dist/lib/Target/X86/X86InstrSSE.td Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Target/X86/X86InstrSSE.td Tue Aug 8 16:52:53 2017 (r322259) @@ -3697,8 +3697,7 @@ let SchedRW = [WriteNop] in { // Pause. This "instruction" is encoded as "rep; nop", so even though it // was introduced with SSE2, it's backward compatible. def PAUSE : I<0x90, RawFrm, (outs), (ins), - "pause", [(int_x86_sse2_pause)], IIC_SSE_PAUSE>, - OBXS, Requires<[HasSSE2]>; + "pause", [(int_x86_sse2_pause)], IIC_SSE_PAUSE>, OBXS; } let SchedRW = [WriteFence] in { Modified: vendor/llvm/dist/lib/Transforms/IPO/ArgumentPromotion.cpp ============================================================================== --- vendor/llvm/dist/lib/Transforms/IPO/ArgumentPromotion.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Transforms/IPO/ArgumentPromotion.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -356,7 +356,7 @@ doPromotion(Function *F, SmallPtrSetImpl & // Just add all the struct element types. Type *AgTy = cast(I->getType())->getElementType(); Value *TheAlloca = new AllocaInst(AgTy, DL.getAllocaAddrSpace(), nullptr, - "", InsertPt); + I->getParamAlignment(), "", InsertPt); StructType *STy = cast(AgTy); Value *Idxs[2] = {ConstantInt::get(Type::getInt32Ty(F->getContext()), 0), nullptr}; Modified: vendor/llvm/dist/lib/Transforms/Instrumentation/AddressSanitizer.cpp ============================================================================== --- vendor/llvm/dist/lib/Transforms/Instrumentation/AddressSanitizer.cpp Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/lib/Transforms/Instrumentation/AddressSanitizer.cpp Tue Aug 8 16:52:53 2017 (r322259) @@ -756,7 +756,8 @@ struct FunctionStackPoisoner : public InstVisitor llvm::CloneModule( SmallVector, 1> MDs; I->getAllMetadata(MDs); for (auto MD : MDs) - GV->addMetadata(MD.first, *MapMetadata(MD.second, VMap)); + GV->addMetadata(MD.first, + *MapMetadata(MD.second, VMap, RF_MoveDistinctMDs)); copyComdat(GV, &*I); } Modified: vendor/llvm/dist/test/CodeGen/AArch64/aarch64_win64cc_vararg.ll ============================================================================== --- vendor/llvm/dist/test/CodeGen/AArch64/aarch64_win64cc_vararg.ll Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/test/CodeGen/AArch64/aarch64_win64cc_vararg.ll Tue Aug 8 16:52:53 2017 (r322259) @@ -2,14 +2,14 @@ define win64cc void @pass_va(i32 %count, ...) nounwind { entry: -; CHECK: sub sp, sp, #80 +; CHECK: str x30, [sp, #-80]! ; CHECK: add x8, sp, #24 ; CHECK: add x0, sp, #24 ; CHECK: stp x6, x7, [sp, #64] ; CHECK: stp x4, x5, [sp, #48] ; CHECK: stp x2, x3, [sp, #32] ; CHECK: str x1, [sp, #24] -; CHECK: stp x30, x8, [sp] +; CHECK: str x8, [sp, #8] ; CHECK: bl other_func ; CHECK: ldr x30, [sp], #80 ; CHECK: ret Modified: vendor/llvm/dist/test/CodeGen/AArch64/misched-fusion-aes.ll ============================================================================== --- vendor/llvm/dist/test/CodeGen/AArch64/misched-fusion-aes.ll Tue Aug 8 16:14:31 2017 (r322258) +++ vendor/llvm/dist/test/CodeGen/AArch64/misched-fusion-aes.ll Tue Aug 8 16:52:53 2017 (r322259) @@ -1,10 +1,10 @@ -; RUN: llc %s -o - -mtriple=aarch64-unknown -mattr=+fuse-aes,+crypto | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=generic -mattr=+crypto | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a53 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a57 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a72 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a73 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKFUSEALLPAIRS -; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=exynos-m1 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECKM1 +; RUN: llc %s -o - -mtriple=aarch64-unknown -mattr=+fuse-aes,+crypto | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=generic -mattr=+crypto | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a53 | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a57 | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a72 | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=cortex-a73 | FileCheck %s +; RUN: llc %s -o - -mtriple=aarch64-unknown -mcpu=exynos-m1 | FileCheck %s declare <16 x i8> @llvm.aarch64.crypto.aese(<16 x i8> %d, <16 x i8> %k) declare <16 x i8> @llvm.aarch64.crypto.aesmc(<16 x i8> %d) @@ -76,41 +76,23 @@ define void @aesea(<16 x i8>* %a0, <16 x i8>* %b0, <16 ret void ; CHECK-LABEL: aesea: -; CHECKFUSEALLPAIRS: aese [[VA:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VA]] -; CHECKFUSEALLPAIRS: aese [[VB:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VB]] -; CHECKFUSEALLPAIRS: aese [[VC:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VC]] -; CHECKFUSEALLPAIRS: aese [[VD:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VD]] -; CHECKFUSEALLPAIRS: aese [[VE:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VE]] -; CHECKFUSEALLPAIRS: aese [[VF:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VF]] -; CHECKFUSEALLPAIRS: aese [[VG:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VG]] -; CHECKFUSEALLPAIRS: aese [[VH:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesmc {{v[0-7].16b}}, [[VH]] -; CHECKFUSEALLPAIRS-NOT: aesmc - -; CHECKM1: aese [[VA:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VA]] -; CHECKM1: aese [[VH:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1: aese [[VB:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VB]] -; CHECKM1: aese {{v[0-7].16b}}, {{v[0-7].16b}} -; CHECKM1: aese [[VC:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VC]] -; CHECKM1: aese [[VD:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VD]] -; CHECKM1: aesmc {{v[0-7].16b}}, [[VH]] -; CHECKM1: aese [[VE:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VE]] -; CHECKM1: aese [[VF:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VF]] -; CHECKM1: aese [[VG:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesmc {{v[0-7].16b}}, [[VG]] +; CHECK: aese [[VA:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VA]], [[VA]] +; CHECK: aese [[VB:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VB]], [[VB]] +; CHECK: aese [[VC:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VC]], [[VC]] +; CHECK: aese [[VD:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VD]], [[VD]] +; CHECK: aese [[VE:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VE]], [[VE]] +; CHECK: aese [[VF:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VF]], [[VF]] +; CHECK: aese [[VG:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VG]], [[VG]] +; CHECK: aese [[VH:v[0-7].16b]], {{v[0-7].16b}} +; CHECK-NEXT: aesmc [[VH]], [[VH]] +; CHECK-NOT: aesmc } define void @aesda(<16 x i8>* %a0, <16 x i8>* %b0, <16 x i8>* %c0, <16 x i8> %d, <16 x i8> %e) { @@ -178,41 +160,23 @@ define void @aesda(<16 x i8>* %a0, <16 x i8>* %b0, <16 ret void ; CHECK-LABEL: aesda: -; CHECKFUSEALLPAIRS: aesd [[VA:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VA]] -; CHECKFUSEALLPAIRS: aesd [[VB:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VB]] -; CHECKFUSEALLPAIRS: aesd [[VC:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VC]] -; CHECKFUSEALLPAIRS: aesd [[VD:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VD]] -; CHECKFUSEALLPAIRS: aesd [[VE:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VE]] -; CHECKFUSEALLPAIRS: aesd [[VF:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VF]] -; CHECKFUSEALLPAIRS: aesd [[VG:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VG]] -; CHECKFUSEALLPAIRS: aesd [[VH:v[0-7].16b]], {{v[0-7].16b}} -; CHECKFUSEALLPAIRS-NEXT: aesimc {{v[0-7].16b}}, [[VH]] -; CHECKFUSEALLPAIRS-NOT: aesimc - -; CHECKM1: aesd [[VA:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesimc {{v[0-7].16b}}, [[VA]] -; CHECKM1: aesd [[VH:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1: aesd [[VB:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesimc {{v[0-7].16b}}, [[VB]] -; CHECKM1: aesd {{v[0-7].16b}}, {{v[0-7].16b}} -; CHECKM1: aesd [[VC:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesimc {{v[0-7].16b}}, [[VC]] -; CHECKM1: aesd [[VD:v[0-7].16b]], {{v[0-7].16b}} -; CHECKM1-NEXT: aesimc {{v[0-7].16b}}, [[VD]] -; CHECKM1: aesimc {{v[0-7].16b}}, [[VH]] -; CHECKM1: aesd [[VE:v[0-7].16b]], {{v[0-7].16b}} *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:04 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 186B2DC2FB2; Tue, 8 Aug 2017 16:53:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B780069AA5; Tue, 8 Aug 2017 16:53:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Gr2mE082543; Tue, 8 Aug 2017 16:53:02 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78Gr24s082542; Tue, 8 Aug 2017 16:53:02 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78Gr24s082542@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322260 - vendor/llvm/llvm-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/llvm/llvm-release_50-r310316 X-SVN-Commit-Revision: 322260 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:04 -0000 Author: dim Date: Tue Aug 8 16:53:02 2017 New Revision: 322260 URL: https://svnweb.freebsd.org/changeset/base/322260 Log: Tag llvm release_50 branch r310316. Added: vendor/llvm/llvm-release_50-r310316/ - copied from r322259, vendor/llvm/dist/ From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:27 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04385DC3079; Tue, 8 Aug 2017 16:53:27 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8ABAA69C86; Tue, 8 Aug 2017 16:53:26 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78GrPLx082623; Tue, 8 Aug 2017 16:53:25 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GrNqZ082597; Tue, 8 Aug 2017 16:53:23 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GrNqZ082597@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322261 - in vendor/clang/dist: include/clang/AST include/clang/Basic include/clang/Driver include/clang/Serialization lib/AST lib/Basic lib/Driver/ToolChains lib/Driver/ToolChains/Arch... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/clang/dist: include/clang/AST include/clang/Basic include/clang/Driver include/clang/Serialization lib/AST lib/Basic lib/Driver/ToolChains lib/Driver/ToolChains/Arch lib/Headers lib/Lex lib/... X-SVN-Commit-Revision: 322261 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:27 -0000 Author: dim Date: Tue Aug 8 16:53:22 2017 New Revision: 322261 URL: https://svnweb.freebsd.org/changeset/base/322261 Log: Vendor import of clang release_50 branch r310316: https://llvm.org/svn/llvm-project/cfe/branches/release_50@310316 Added: vendor/clang/dist/test/CodeGen/pause.c (contents, props changed) vendor/clang/dist/test/CodeGen/x86-nontemporal.c (contents, props changed) vendor/clang/dist/test/PCH/coroutines.cpp (contents, props changed) Deleted: vendor/clang/dist/test/CodeGen/mips-madd4.c Modified: vendor/clang/dist/include/clang/AST/StmtCXX.h vendor/clang/dist/include/clang/Basic/BuiltinsX86.def vendor/clang/dist/include/clang/Driver/Options.td vendor/clang/dist/include/clang/Serialization/ASTBitCodes.h vendor/clang/dist/lib/AST/ODRHash.cpp vendor/clang/dist/lib/AST/StmtCXX.cpp vendor/clang/dist/lib/Basic/Targets.cpp vendor/clang/dist/lib/Driver/ToolChains/Arch/Mips.cpp vendor/clang/dist/lib/Driver/ToolChains/OpenBSD.cpp vendor/clang/dist/lib/Headers/avx2intrin.h vendor/clang/dist/lib/Headers/avx512fintrin.h vendor/clang/dist/lib/Headers/avxintrin.h vendor/clang/dist/lib/Headers/float.h vendor/clang/dist/lib/Lex/ScratchBuffer.cpp vendor/clang/dist/lib/Sema/Sema.cpp vendor/clang/dist/lib/Sema/SemaExpr.cpp vendor/clang/dist/lib/Serialization/ASTReaderStmt.cpp vendor/clang/dist/lib/Serialization/ASTWriterStmt.cpp vendor/clang/dist/test/Driver/openbsd.c vendor/clang/dist/test/Misc/caret-diags-multiline.cpp vendor/clang/dist/test/Preprocessor/init.c vendor/clang/dist/test/Sema/vector-cast.c vendor/clang/dist/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp Modified: vendor/clang/dist/include/clang/AST/StmtCXX.h ============================================================================== --- vendor/clang/dist/include/clang/AST/StmtCXX.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/include/clang/AST/StmtCXX.h Tue Aug 8 16:53:22 2017 (r322261) @@ -317,6 +317,7 @@ class CoroutineBodyStmt final unsigned NumParams; friend class ASTStmtReader; + friend class ASTReader; friend TrailingObjects; Stmt **getStoredStmts() { return getTrailingObjects(); } @@ -347,6 +348,8 @@ class CoroutineBodyStmt final public: static CoroutineBodyStmt *Create(const ASTContext &C, CtorArgs const &Args); + static CoroutineBodyStmt *Create(const ASTContext &C, EmptyShell, + unsigned NumParams); bool hasDependentPromiseType() const { return getPromiseDecl()->getType()->isDependentType(); @@ -443,6 +446,8 @@ class CoreturnStmt : public Stmt { (public) SubStmts[SubStmt::Operand] = Operand; SubStmts[SubStmt::PromiseCall] = PromiseCall; } + + CoreturnStmt(EmptyShell) : CoreturnStmt({}, {}, {}) {} SourceLocation getKeywordLoc() const { return CoreturnLoc; } Modified: vendor/clang/dist/include/clang/Basic/BuiltinsX86.def ============================================================================== --- vendor/clang/dist/include/clang/Basic/BuiltinsX86.def Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/include/clang/Basic/BuiltinsX86.def Tue Aug 8 16:53:22 2017 (r322261) @@ -338,8 +338,8 @@ TARGET_BUILTIN(__builtin_ia32_lfence, "v", "", "sse2") TARGET_HEADER_BUILTIN(_mm_lfence, "v", "h", "emmintrin.h", ALL_LANGUAGES, "sse2") TARGET_BUILTIN(__builtin_ia32_mfence, "v", "", "sse2") TARGET_HEADER_BUILTIN(_mm_mfence, "v", "h", "emmintrin.h", ALL_LANGUAGES, "sse2") -TARGET_BUILTIN(__builtin_ia32_pause, "v", "", "sse2") -TARGET_HEADER_BUILTIN(_mm_pause, "v", "h", "emmintrin.h", ALL_LANGUAGES, "sse2") +TARGET_BUILTIN(__builtin_ia32_pause, "v", "", "") +TARGET_HEADER_BUILTIN(_mm_pause, "v", "h", "emmintrin.h", ALL_LANGUAGES, "") TARGET_BUILTIN(__builtin_ia32_pmuludq128, "V2LLiV4iV4i", "", "sse2") TARGET_BUILTIN(__builtin_ia32_psraw128, "V8sV8sV8s", "", "sse2") TARGET_BUILTIN(__builtin_ia32_psrad128, "V4iV4iV4i", "", "sse2") Modified: vendor/clang/dist/include/clang/Driver/Options.td ============================================================================== --- vendor/clang/dist/include/clang/Driver/Options.td Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/include/clang/Driver/Options.td Tue Aug 8 16:53:22 2017 (r322261) @@ -2019,10 +2019,6 @@ def mdspr2 : Flag<["-"], "mdspr2">, Group; def mno_dspr2 : Flag<["-"], "mno-dspr2">, Group; def msingle_float : Flag<["-"], "msingle-float">, Group; def mdouble_float : Flag<["-"], "mdouble-float">, Group; -def mmadd4 : Flag<["-"], "mmadd4">, Group, - HelpText<"Enable the generation of 4-operand madd.s, madd.d and related instructions.">; -def mno_madd4 : Flag<["-"], "mno-madd4">, Group, - HelpText<"Disable the generation of 4-operand madd.s, madd.d and related instructions.">; def mmsa : Flag<["-"], "mmsa">, Group, HelpText<"Enable MSA ASE (MIPS only)">; def mno_msa : Flag<["-"], "mno-msa">, Group, Modified: vendor/clang/dist/include/clang/Serialization/ASTBitCodes.h ============================================================================== --- vendor/clang/dist/include/clang/Serialization/ASTBitCodes.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/include/clang/Serialization/ASTBitCodes.h Tue Aug 8 16:53:22 2017 (r322261) @@ -1545,9 +1545,14 @@ namespace clang { // ARC EXPR_OBJC_BRIDGED_CAST, // ObjCBridgedCastExpr - + STMT_MS_DEPENDENT_EXISTS, // MSDependentExistsStmt - EXPR_LAMBDA // LambdaExpr + EXPR_LAMBDA, // LambdaExpr + STMT_COROUTINE_BODY, + STMT_CORETURN, + EXPR_COAWAIT, + EXPR_COYIELD, + EXPR_DEPENDENT_COAWAIT, }; /// \brief The kinds of designators that can occur in a Modified: vendor/clang/dist/lib/AST/ODRHash.cpp ============================================================================== --- vendor/clang/dist/lib/AST/ODRHash.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/AST/ODRHash.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -378,8 +378,12 @@ void ODRHash::AddCXXRecordDecl(const CXXRecordDecl *Re assert(Record && Record->hasDefinition() && "Expected non-null record to be a definition."); - if (isa(Record)) { - return; + const DeclContext *DC = Record; + while (DC) { + if (isa(DC)) { + return; + } + DC = DC->getParent(); } AddDecl(Record); Modified: vendor/clang/dist/lib/AST/StmtCXX.cpp ============================================================================== --- vendor/clang/dist/lib/AST/StmtCXX.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/AST/StmtCXX.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -96,6 +96,20 @@ CoroutineBodyStmt *CoroutineBodyStmt::Create( return new (Mem) CoroutineBodyStmt(Args); } +CoroutineBodyStmt *CoroutineBodyStmt::Create(const ASTContext &C, EmptyShell, + unsigned NumParams) { + std::size_t Size = totalSizeToAlloc( + CoroutineBodyStmt::FirstParamMove + NumParams); + + void *Mem = C.Allocate(Size, alignof(CoroutineBodyStmt)); + auto *Result = new (Mem) CoroutineBodyStmt(CtorArgs()); + Result->NumParams = NumParams; + auto *ParamBegin = Result->getStoredStmts() + SubStmt::FirstParamMove; + std::uninitialized_fill(ParamBegin, ParamBegin + NumParams, + static_cast(nullptr)); + return Result; +} + CoroutineBodyStmt::CoroutineBodyStmt(CoroutineBodyStmt::CtorArgs const &Args) : Stmt(CoroutineBodyStmtClass), NumParams(Args.ParamMoves.size()) { Stmt **SubStmts = getStoredStmts(); Modified: vendor/clang/dist/lib/Basic/Targets.cpp ============================================================================== --- vendor/clang/dist/lib/Basic/Targets.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Basic/Targets.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -8050,7 +8050,6 @@ class MipsTargetInfo : public TargetInfo { NoDSP, DSP1, DSP2 } DspRev; bool HasMSA; - bool DisableMadd4; protected: bool HasFP64; @@ -8061,7 +8060,7 @@ class MipsTargetInfo : public TargetInfo { : TargetInfo(Triple), IsMips16(false), IsMicromips(false), IsNan2008(false), IsSingleFloat(false), IsNoABICalls(false), CanUseBSDABICalls(false), FloatABI(HardFloat), DspRev(NoDSP), - HasMSA(false), DisableMadd4(false), HasFP64(false) { + HasMSA(false), HasFP64(false) { TheCXXABI.set(TargetCXXABI::GenericMIPS); setABI((getTriple().getArch() == llvm::Triple::mips || @@ -8307,9 +8306,6 @@ class MipsTargetInfo : public TargetInfo { if (HasMSA) Builder.defineMacro("__mips_msa", Twine(1)); - if (DisableMadd4) - Builder.defineMacro("__mips_no_madd4", Twine(1)); - Builder.defineMacro("_MIPS_SZPTR", Twine(getPointerWidth(0))); Builder.defineMacro("_MIPS_SZINT", Twine(getIntWidth())); Builder.defineMacro("_MIPS_SZLONG", Twine(getLongWidth())); @@ -8472,8 +8468,6 @@ class MipsTargetInfo : public TargetInfo { DspRev = std::max(DspRev, DSP2); else if (Feature == "+msa") HasMSA = true; - else if (Feature == "+nomadd4") - DisableMadd4 = true; else if (Feature == "+fp64") HasFP64 = true; else if (Feature == "-fp64") Modified: vendor/clang/dist/lib/Driver/ToolChains/Arch/Mips.cpp ============================================================================== --- vendor/clang/dist/lib/Driver/ToolChains/Arch/Mips.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Driver/ToolChains/Arch/Mips.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -297,8 +297,6 @@ void mips::getMIPSTargetFeatures(const Driver &D, cons AddTargetFeature(Args, Features, options::OPT_mno_odd_spreg, options::OPT_modd_spreg, "nooddspreg"); - AddTargetFeature(Args, Features, options::OPT_mno_madd4, options::OPT_mmadd4, - "nomadd4"); AddTargetFeature(Args, Features, options::OPT_mlong_calls, options::OPT_mno_long_calls, "long-calls"); AddTargetFeature(Args, Features, options::OPT_mmt, options::OPT_mno_mt,"mt"); Modified: vendor/clang/dist/lib/Driver/ToolChains/OpenBSD.cpp ============================================================================== --- vendor/clang/dist/lib/Driver/ToolChains/OpenBSD.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Driver/ToolChains/OpenBSD.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -133,6 +133,8 @@ void openbsd::Linker::ConstructJob(Compilation &C, con } } + if (Args.hasArg(options::OPT_pie)) + CmdArgs.push_back("-pie"); if (Args.hasArg(options::OPT_nopie)) CmdArgs.push_back("-nopie"); Modified: vendor/clang/dist/lib/Headers/avx2intrin.h ============================================================================== --- vendor/clang/dist/lib/Headers/avx2intrin.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Headers/avx2intrin.h Tue Aug 8 16:53:22 2017 (r322261) @@ -832,7 +832,8 @@ _mm256_xor_si256(__m256i __a, __m256i __b) static __inline__ __m256i __DEFAULT_FN_ATTRS _mm256_stream_load_si256(__m256i const *__V) { - return (__m256i)__builtin_nontemporal_load((const __v4di *)__V); + typedef __v4di __v4di_aligned __attribute__((aligned(32))); + return (__m256i)__builtin_nontemporal_load((const __v4di_aligned *)__V); } static __inline__ __m128 __DEFAULT_FN_ATTRS Modified: vendor/clang/dist/lib/Headers/avx512fintrin.h ============================================================================== --- vendor/clang/dist/lib/Headers/avx512fintrin.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Headers/avx512fintrin.h Tue Aug 8 16:53:22 2017 (r322261) @@ -4289,7 +4289,7 @@ static __inline__ __m512i __DEFAULT_FN_ATTRS _mm512_maskz_cvtps_epu32 ( __mmask16 __U, __m512 __A) { return (__m512i) __builtin_ia32_cvtps2udq512_mask ((__v16sf) __A, - (__v16si) + (__v16si) _mm512_setzero_si512 (), (__mmask16) __U , _MM_FROUND_CUR_DIRECTION); @@ -9035,25 +9035,29 @@ _mm512_kxor (__mmask16 __A, __mmask16 __B) static __inline__ void __DEFAULT_FN_ATTRS _mm512_stream_si512 (__m512i * __P, __m512i __A) { - __builtin_nontemporal_store((__v8di)__A, (__v8di*)__P); + typedef __v8di __v8di_aligned __attribute__((aligned(64))); + __builtin_nontemporal_store((__v8di_aligned)__A, (__v8di_aligned*)__P); } static __inline__ __m512i __DEFAULT_FN_ATTRS _mm512_stream_load_si512 (void *__P) { - return (__m512i) __builtin_nontemporal_load((const __v8di *)__P); + typedef __v8di __v8di_aligned __attribute__((aligned(64))); + return (__m512i) __builtin_nontemporal_load((const __v8di_aligned *)__P); } static __inline__ void __DEFAULT_FN_ATTRS _mm512_stream_pd (double *__P, __m512d __A) { - __builtin_nontemporal_store((__v8df)__A, (__v8df*)__P); + typedef __v8df __v8df_aligned __attribute__((aligned(64))); + __builtin_nontemporal_store((__v8df_aligned)__A, (__v8df_aligned*)__P); } static __inline__ void __DEFAULT_FN_ATTRS _mm512_stream_ps (float *__P, __m512 __A) { - __builtin_nontemporal_store((__v16sf)__A, (__v16sf*)__P); + typedef __v16sf __v16sf_aligned __attribute__((aligned(64))); + __builtin_nontemporal_store((__v16sf_aligned)__A, (__v16sf_aligned*)__P); } static __inline__ __m512d __DEFAULT_FN_ATTRS @@ -9217,39 +9221,39 @@ _mm512_maskz_moveldup_ps (__mmask16 __U, __m512 __A) static __inline__ __m128 __DEFAULT_FN_ATTRS _mm_mask_move_ss (__m128 __W, __mmask8 __U, __m128 __A, __m128 __B) { - __m128 res = __A; + __m128 res = __A; res[0] = (__U & 1) ? __B[0] : __W[0]; - return res; + return res; } static __inline__ __m128 __DEFAULT_FN_ATTRS _mm_maskz_move_ss (__mmask8 __U, __m128 __A, __m128 __B) { - __m128 res = __A; - res[0] = (__U & 1) ? __B[0] : 0; - return res; + __m128 res = __A; + res[0] = (__U & 1) ? __B[0] : 0; + return res; } static __inline__ __m128d __DEFAULT_FN_ATTRS _mm_mask_move_sd (__m128d __W, __mmask8 __U, __m128d __A, __m128d __B) { - __m128d res = __A; + __m128d res = __A; res[0] = (__U & 1) ? __B[0] : __W[0]; - return res; + return res; } static __inline__ __m128d __DEFAULT_FN_ATTRS _mm_maskz_move_sd (__mmask8 __U, __m128d __A, __m128d __B) { - __m128d res = __A; - res[0] = (__U & 1) ? __B[0] : 0; - return res; + __m128d res = __A; + res[0] = (__U & 1) ? __B[0] : 0; + return res; } static __inline__ void __DEFAULT_FN_ATTRS _mm_mask_store_ss (float * __W, __mmask8 __U, __m128 __A) { - __builtin_ia32_storess128_mask ((__v16sf *)__W, + __builtin_ia32_storess128_mask ((__v16sf *)__W, (__v16sf) _mm512_castps128_ps512(__A), (__mmask16) __U & (__mmask16)1); } @@ -9257,7 +9261,7 @@ _mm_mask_store_ss (float * __W, __mmask8 __U, __m128 _ static __inline__ void __DEFAULT_FN_ATTRS _mm_mask_store_sd (double * __W, __mmask8 __U, __m128d __A) { - __builtin_ia32_storesd128_mask ((__v8df *)__W, + __builtin_ia32_storesd128_mask ((__v8df *)__W, (__v8df) _mm512_castpd128_pd512(__A), (__mmask8) __U & 1); } @@ -9606,7 +9610,7 @@ _mm_mask_cvtsd_ss (__m128 __W, __mmask8 __U, __m128 __ { return __builtin_ia32_cvtsd2ss_round_mask ((__v4sf)(__A), (__v2df)(__B), - (__v4sf)(__W), + (__v4sf)(__W), (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); } @@ -9615,7 +9619,7 @@ _mm_maskz_cvtsd_ss (__mmask8 __U, __m128 __A, __m128d { return __builtin_ia32_cvtsd2ss_round_mask ((__v4sf)(__A), (__v2df)(__B), - (__v4sf)_mm_setzero_ps(), + (__v4sf)_mm_setzero_ps(), (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); } @@ -9680,7 +9684,7 @@ _mm_mask_cvtss_sd (__m128d __W, __mmask8 __U, __m128d return __builtin_ia32_cvtss2sd_round_mask((__v2df)(__A), (__v4sf)(__B), (__v2df)(__W), - (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); + (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); } static __inline__ __m128d __DEFAULT_FN_ATTRS @@ -9688,8 +9692,8 @@ _mm_maskz_cvtss_sd (__mmask8 __U, __m128d __A, __m128 { return __builtin_ia32_cvtss2sd_round_mask((__v2df)(__A), (__v4sf)(__B), - (__v2df)_mm_setzero_pd(), - (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); + (__v2df)_mm_setzero_pd(), + (__mmask8)(__U), _MM_FROUND_CUR_DIRECTION); } static __inline__ __m128d __DEFAULT_FN_ATTRS @@ -9935,7 +9939,7 @@ static __inline__ double __DEFAULT_FN_ATTRS _mm512_red } // Vec512 - Vector with size 512. -// Vec512Neutral - All vector elements set to the identity element. +// Vec512Neutral - All vector elements set to the identity element. // Identity element: {+,0},{*,1},{&,0xFFFFFFFFFFFFFFFF},{|,0} // Operator - Can be one of following: +,*,&,| // Mask - Intrinsic Mask @@ -9965,19 +9969,19 @@ _mm512_mask_reduce_mul_epi64(__mmask8 __M, __m512i __W static __inline__ long long __DEFAULT_FN_ATTRS _mm512_mask_reduce_and_epi64(__mmask8 __M, __m512i __W) { - _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_epi64(0xFFFFFFFFFFFFFFFF), + _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_epi64(0xFFFFFFFFFFFFFFFF), &, __M, i, i, q); } static __inline__ long long __DEFAULT_FN_ATTRS _mm512_mask_reduce_or_epi64(__mmask8 __M, __m512i __W) { - _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_epi64(0), |, __M, + _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_epi64(0), |, __M, i, i, q); } static __inline__ double __DEFAULT_FN_ATTRS _mm512_mask_reduce_add_pd(__mmask8 __M, __m512d __W) { - _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_pd(0), +, __M, + _mm512_mask_reduce_operator_64bit(__W, _mm512_set1_pd(0), +, __M, f, d, pd); } @@ -10039,17 +10043,17 @@ _mm512_reduce_add_epi32(__m512i __W) { _mm512_reduce_operator_32bit(__W, +, i, i); } -static __inline__ int __DEFAULT_FN_ATTRS +static __inline__ int __DEFAULT_FN_ATTRS _mm512_reduce_mul_epi32(__m512i __W) { _mm512_reduce_operator_32bit(__W, *, i, i); } -static __inline__ int __DEFAULT_FN_ATTRS +static __inline__ int __DEFAULT_FN_ATTRS _mm512_reduce_and_epi32(__m512i __W) { _mm512_reduce_operator_32bit(__W, &, i, i); } -static __inline__ int __DEFAULT_FN_ATTRS +static __inline__ int __DEFAULT_FN_ATTRS _mm512_reduce_or_epi32(__m512i __W) { _mm512_reduce_operator_32bit(__W, |, i, i); } @@ -10065,7 +10069,7 @@ _mm512_reduce_mul_ps(__m512 __W) { } // Vec512 - Vector with size 512. -// Vec512Neutral - All vector elements set to the identity element. +// Vec512Neutral - All vector elements set to the identity element. // Identity element: {+,0},{*,1},{&,0xFFFFFFFF},{|,0} // Operator - Can be one of following: +,*,&,| // Mask - Intrinsic Mask @@ -10095,7 +10099,7 @@ _mm512_mask_reduce_mul_epi32( __mmask16 __M, __m512i _ static __inline__ int __DEFAULT_FN_ATTRS _mm512_mask_reduce_and_epi32( __mmask16 __M, __m512i __W) { - _mm512_mask_reduce_operator_32bit(__W, _mm512_set1_epi32(0xFFFFFFFF), &, __M, + _mm512_mask_reduce_operator_32bit(__W, _mm512_set1_epi32(0xFFFFFFFF), &, __M, i, i, d); } @@ -10158,7 +10162,7 @@ _mm512_mask_reduce_mul_ps(__mmask16 __M, __m512 __W) { return Vec512[0]; \ }) -static __inline__ long long __DEFAULT_FN_ATTRS +static __inline__ long long __DEFAULT_FN_ATTRS _mm512_reduce_max_epi64(__m512i __V) { _mm512_reduce_maxMin_64bit(__V, max_epi64, i, i); } @@ -10168,7 +10172,7 @@ _mm512_reduce_max_epu64(__m512i __V) { _mm512_reduce_maxMin_64bit(__V, max_epu64, i, i); } -static __inline__ double __DEFAULT_FN_ATTRS +static __inline__ double __DEFAULT_FN_ATTRS _mm512_reduce_max_pd(__m512d __V) { _mm512_reduce_maxMin_64bit(__V, max_pd, d, f); } @@ -10183,7 +10187,7 @@ _mm512_reduce_min_epu64(__m512i __V) { _mm512_reduce_maxMin_64bit(__V, min_epu64, i, i); } -static __inline__ double __DEFAULT_FN_ATTRS +static __inline__ double __DEFAULT_FN_ATTRS _mm512_reduce_min_pd(__m512d __V) { _mm512_reduce_maxMin_64bit(__V, min_pd, d, f); } Modified: vendor/clang/dist/lib/Headers/avxintrin.h ============================================================================== --- vendor/clang/dist/lib/Headers/avxintrin.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Headers/avxintrin.h Tue Aug 8 16:53:22 2017 (r322261) @@ -3590,7 +3590,8 @@ _mm_maskstore_ps(float *__p, __m128i __m, __m128 __a) static __inline void __DEFAULT_FN_ATTRS _mm256_stream_si256(__m256i *__a, __m256i __b) { - __builtin_nontemporal_store((__v4di)__b, (__v4di*)__a); + typedef __v4di __v4di_aligned __attribute__((aligned(32))); + __builtin_nontemporal_store((__v4di_aligned)__b, (__v4di_aligned*)__a); } /// \brief Moves double-precision values from a 256-bit vector of [4 x double] @@ -3609,7 +3610,8 @@ _mm256_stream_si256(__m256i *__a, __m256i __b) static __inline void __DEFAULT_FN_ATTRS _mm256_stream_pd(double *__a, __m256d __b) { - __builtin_nontemporal_store((__v4df)__b, (__v4df*)__a); + typedef __v4df __v4df_aligned __attribute__((aligned(32))); + __builtin_nontemporal_store((__v4df_aligned)__b, (__v4df_aligned*)__a); } /// \brief Moves single-precision floating point values from a 256-bit vector @@ -3629,7 +3631,8 @@ _mm256_stream_pd(double *__a, __m256d __b) static __inline void __DEFAULT_FN_ATTRS _mm256_stream_ps(float *__p, __m256 __a) { - __builtin_nontemporal_store((__v8sf)__a, (__v8sf*)__p); + typedef __v8sf __v8sf_aligned __attribute__((aligned(32))); + __builtin_nontemporal_store((__v8sf_aligned)__a, (__v8sf_aligned*)__p); } /* Create vectors */ Modified: vendor/clang/dist/lib/Headers/float.h ============================================================================== --- vendor/clang/dist/lib/Headers/float.h Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Headers/float.h Tue Aug 8 16:53:22 2017 (r322261) @@ -33,6 +33,15 @@ */ #if (defined(__APPLE__) || (defined(__MINGW32__) || defined(_MSC_VER))) && \ __STDC_HOSTED__ && __has_include_next() + +/* Prior to Apple's 10.7 SDK, float.h SDK header used to apply an extra level + * of #include_next to keep Metrowerks compilers happy. Avoid this + * extra indirection. + */ +#ifdef __APPLE__ +#define _FLOAT_H_ +#endif + # include_next /* Undefine anything that we'll be redefining below. */ Modified: vendor/clang/dist/lib/Lex/ScratchBuffer.cpp ============================================================================== --- vendor/clang/dist/lib/Lex/ScratchBuffer.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Lex/ScratchBuffer.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -35,6 +35,14 @@ SourceLocation ScratchBuffer::getToken(const char *Buf const char *&DestPtr) { if (BytesUsed+Len+2 > ScratchBufSize) AllocScratchBuffer(Len+2); + else { + // Clear out the source line cache if it's already been computed. + // FIXME: Allow this to be incrementally extended. + auto *ContentCache = const_cast( + SourceMgr.getSLocEntry(SourceMgr.getFileID(BufferStartLoc)) + .getFile().getContentCache()); + ContentCache->SourceLineCache = nullptr; + } // Prefix the token with a \n, so that it looks like it is the first thing on // its own virtual line in caret diagnostics. Modified: vendor/clang/dist/lib/Sema/Sema.cpp ============================================================================== --- vendor/clang/dist/lib/Sema/Sema.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Sema/Sema.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -542,6 +542,9 @@ void Sema::getUndefinedButUsed( // __attribute__((weakref)) is basically a definition. if (ND->hasAttr()) continue; + if (isa(ND)) + continue; + if (FunctionDecl *FD = dyn_cast(ND)) { if (FD->isDefined()) continue; Modified: vendor/clang/dist/lib/Sema/SemaExpr.cpp ============================================================================== --- vendor/clang/dist/lib/Sema/SemaExpr.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Sema/SemaExpr.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -8288,7 +8288,7 @@ QualType Sema::CheckVectorOperands(ExprResult &LHS, Ex // type. Note that this is already done by non-compound assignments in // CheckAssignmentConstraints. If it's a scalar type, only bitcast for // <1 x T> -> T. The result is also a vector type. - } else if (OtherType->isExtVectorType() || + } else if (OtherType->isExtVectorType() || OtherType->isVectorType() || (OtherType->isScalarType() && VT->getNumElements() == 1)) { ExprResult *RHSExpr = &RHS; *RHSExpr = ImpCastExprToType(RHSExpr->get(), LHSType, CK_BitCast); Modified: vendor/clang/dist/lib/Serialization/ASTReaderStmt.cpp ============================================================================== --- vendor/clang/dist/lib/Serialization/ASTReaderStmt.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Serialization/ASTReaderStmt.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -367,28 +367,45 @@ void ASTStmtReader::VisitMSAsmStmt(MSAsmStmt *S) { } void ASTStmtReader::VisitCoroutineBodyStmt(CoroutineBodyStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + assert(Record.peekInt() == S->NumParams); + Record.skipInts(1); + auto *StoredStmts = S->getStoredStmts(); + for (unsigned i = 0; + i < CoroutineBodyStmt::SubStmt::FirstParamMove + S->NumParams; ++i) + StoredStmts[i] = Record.readSubStmt(); } void ASTStmtReader::VisitCoreturnStmt(CoreturnStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + S->CoreturnLoc = Record.readSourceLocation(); + for (auto &SubStmt: S->SubStmts) + SubStmt = Record.readSubStmt(); + S->IsImplicit = Record.readInt() != 0; } -void ASTStmtReader::VisitCoawaitExpr(CoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtReader::VisitCoawaitExpr(CoawaitExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); + E->OpaqueValue = cast_or_null(Record.readSubStmt()); + E->setIsImplicit(Record.readInt() != 0); } -void ASTStmtReader::VisitDependentCoawaitExpr(DependentCoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtReader::VisitCoyieldExpr(CoyieldExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); + E->OpaqueValue = cast_or_null(Record.readSubStmt()); } -void ASTStmtReader::VisitCoyieldExpr(CoyieldExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtReader::VisitDependentCoawaitExpr(DependentCoawaitExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); } void ASTStmtReader::VisitCapturedStmt(CapturedStmt *S) { @@ -3947,6 +3964,29 @@ Stmt *ASTReader::ReadStmtFromStream(ModuleFile &F) { S = LambdaExpr::CreateDeserialized(Context, NumCaptures); break; } + + case STMT_COROUTINE_BODY: { + unsigned NumParams = Record[ASTStmtReader::NumStmtFields]; + S = CoroutineBodyStmt::Create(Context, Empty, NumParams); + break; + } + + case STMT_CORETURN: + S = new (Context) CoreturnStmt(Empty); + break; + + case EXPR_COAWAIT: + S = new (Context) CoawaitExpr(Empty); + break; + + case EXPR_COYIELD: + S = new (Context) CoyieldExpr(Empty); + break; + + case EXPR_DEPENDENT_COAWAIT: + S = new (Context) DependentCoawaitExpr(Empty); + break; + } // We hit a STMT_STOP, so we're done with this expression. Modified: vendor/clang/dist/lib/Serialization/ASTWriterStmt.cpp ============================================================================== --- vendor/clang/dist/lib/Serialization/ASTWriterStmt.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/lib/Serialization/ASTWriterStmt.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -286,7 +286,7 @@ void ASTStmtWriter::VisitMSAsmStmt(MSAsmStmt *S) { } // Outputs - for (unsigned I = 0, N = S->getNumOutputs(); I != N; ++I) { + for (unsigned I = 0, N = S->getNumOutputs(); I != N; ++I) { Record.AddStmt(S->getOutputExpr(I)); Record.AddString(S->getOutputConstraint(I)); } @@ -300,29 +300,48 @@ void ASTStmtWriter::VisitMSAsmStmt(MSAsmStmt *S) { Code = serialization::STMT_MSASM; } -void ASTStmtWriter::VisitCoroutineBodyStmt(CoroutineBodyStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoroutineBodyStmt(CoroutineBodyStmt *CoroStmt) { + VisitStmt(CoroStmt); + Record.push_back(CoroStmt->getParamMoves().size()); + for (Stmt *S : CoroStmt->children()) + Record.AddStmt(S); + Code = serialization::STMT_COROUTINE_BODY; } void ASTStmtWriter::VisitCoreturnStmt(CoreturnStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + Record.AddSourceLocation(S->getKeywordLoc()); + Record.AddStmt(S->getOperand()); + Record.AddStmt(S->getPromiseCall()); + Record.push_back(S->isImplicit()); + Code = serialization::STMT_CORETURN; } -void ASTStmtWriter::VisitCoawaitExpr(CoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoroutineSuspendExpr(CoroutineSuspendExpr *E) { + VisitExpr(E); + Record.AddSourceLocation(E->getKeywordLoc()); + for (Stmt *S : E->children()) + Record.AddStmt(S); + Record.AddStmt(E->getOpaqueValue()); } -void ASTStmtWriter::VisitDependentCoawaitExpr(DependentCoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoawaitExpr(CoawaitExpr *E) { + VisitCoroutineSuspendExpr(E); + Record.push_back(E->isImplicit()); + Code = serialization::EXPR_COAWAIT; } -void ASTStmtWriter::VisitCoyieldExpr(CoyieldExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoyieldExpr(CoyieldExpr *E) { + VisitCoroutineSuspendExpr(E); + Code = serialization::EXPR_COYIELD; +} + +void ASTStmtWriter::VisitDependentCoawaitExpr(DependentCoawaitExpr *E) { + VisitExpr(E); + Record.AddSourceLocation(E->getKeywordLoc()); + for (Stmt *S : E->children()) + Record.AddStmt(S); + Code = serialization::EXPR_DEPENDENT_COAWAIT; } void ASTStmtWriter::VisitCapturedStmt(CapturedStmt *S) { Added: vendor/clang/dist/test/CodeGen/pause.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist/test/CodeGen/pause.c Tue Aug 8 16:53:22 2017 (r322261) @@ -0,0 +1,11 @@ +// RUN: %clang_cc1 -ffreestanding %s -triple=i386-pc-win32 -target-feature -sse2 -emit-llvm -o - -Wall -Werror | FileCheck %s +// RUN: %clang_cc1 -ffreestanding %s -triple=i386-pc-win32 -target-feature +sse2 -emit-llvm -o - -Wall -Werror | FileCheck %s + + +#include + +void test_mm_pause() { + // CHECK-LABEL: test_mm_pause + // CHECK: call void @llvm.x86.sse2.pause() + return _mm_pause(); +} Added: vendor/clang/dist/test/CodeGen/x86-nontemporal.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist/test/CodeGen/x86-nontemporal.c Tue Aug 8 16:53:22 2017 (r322261) @@ -0,0 +1,93 @@ +// RUN: %clang_cc1 -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -target-feature +avx -target-feature +avx2 -target-feature +avx512f -emit-llvm -o - -Wall -Werror | FileCheck %s --check-prefix=CHECK +// RUN: %clang_cc1 -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -target-feature +avx -target-feature +avx2 -target-feature +avx512f -fno-signed-char -emit-llvm -o - -Wall -Werror | FileCheck %s --check-prefix=CHECK + +// RUN: %clang_cc1 -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -target-feature +avx -target-feature +avx2 -target-feature +avx512f -emit-llvm -o - -Wall -Werror -fmax-type-align=16 | FileCheck %s --check-prefix=CHECK +// RUN: %clang_cc1 -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -target-feature +avx -target-feature +avx2 -target-feature +avx512f -fno-signed-char -emit-llvm -o - -Wall -Werror -fmax-type-align=16 | FileCheck %s --check-prefix=CHECK + +#include + +// (PR33830) Tests ensure the correct alignment of non-temporal load/stores on darwin targets where fmax-type-align is set to 16. + +// +// 128-bit vectors +// + +void test_mm_stream_pd(double* A, __m128d B) { + // CHECK-LABEL: test_mm_stream_pd + // CHECK: store <2 x double> %{{.*}}, <2 x double>* %{{.*}}, align 16, !nontemporal + _mm_stream_pd(A, B); +} + +void test_mm_stream_ps(float* A, __m128 B) { + // CHECK16-LABEL: test_mm_stream_ps + // CHECK16: store <4 x float> %{{.*}}, <4 x float>* %{{.*}}, align 16, !nontemporal + _mm_stream_ps(A, B); +} + +void test_mm_stream_si128(__m128i* A, __m128i B) { + // CHECK-LABEL: test_mm_stream_si128 + // CHECK: store <2 x i64> %{{.*}}, <2 x i64>* %{{.*}}, align 16, !nontemporal + _mm_stream_si128(A, B); +} + +__m128i test_mm_stream_load_si128(__m128i const *A) { + // CHECK-LABEL: test_mm_stream_load_si128 + // CHECK: load <2 x i64>, <2 x i64>* %{{.*}}, align 16, !nontemporal + return _mm_stream_load_si128(A); +} + +// +// 256-bit vectors +// + +void test_mm256_stream_pd(double* A, __m256d B) { + // CHECK-LABEL: test_mm256_stream_pd + // CHECK: store <4 x double> %{{.*}}, <4 x double>* %{{.*}}, align 32, !nontemporal + _mm256_stream_pd(A, B); +} + +void test_mm256_stream_ps(float* A, __m256 B) { + // CHECK-LABEL: test_mm256_stream_ps + // CHECK: store <8 x float> %{{.*}}, <8 x float>* %{{.*}}, align 32, !nontemporal + _mm256_stream_ps(A, B); +} + +void test_mm256_stream_si256(__m256i* A, __m256i B) { + // CHECK-LABEL: test_mm256_stream_si256 + // CHECK: store <4 x i64> %{{.*}}, <4 x i64>* %{{.*}}, align 32, !nontemporal + _mm256_stream_si256(A, B); +} + +__m256i test_mm256_stream_load_si256(__m256i const *A) { + // CHECK-LABEL: test_mm256_stream_load_si256 + // CHECK: load <4 x i64>, <4 x i64>* %{{.*}}, align 32, !nontemporal + return _mm256_stream_load_si256(A); +} + +// +// 512-bit vectors +// + +void test_mm512_stream_pd(double* A, __m512d B) { + // CHECK-LABEL: test_mm512_stream_pd + // CHECK: store <8 x double> %{{.*}}, <8 x double>* %{{.*}}, align 64, !nontemporal + _mm512_stream_pd(A, B); +} + +void test_mm512_stream_ps(float* A, __m512 B) { + // CHECK-LABEL: test_mm512_stream_ps + // CHECK: store <16 x float> %{{.*}}, <16 x float>* %{{.*}}, align 64, !nontemporal + _mm512_stream_ps(A, B); +} + +void test_mm512_stream_si512(__m512i* A, __m512i B) { + // CHECK-LABEL: test_mm512_stream_si512 + // CHECK: store <8 x i64> %{{.*}}, <8 x i64>* %{{.*}}, align 64, !nontemporal + _mm512_stream_si512(A, B); +} + +__m512i test_mm512_stream_load_si512(void *A) { + // CHECK-LABEL: test_mm512_stream_load_si512 + // CHECK: load <8 x i64>, <8 x i64>* %{{.*}}, align 64, !nontemporal + return _mm512_stream_load_si512(A); +} Modified: vendor/clang/dist/test/Driver/openbsd.c ============================================================================== --- vendor/clang/dist/test/Driver/openbsd.c Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/test/Driver/openbsd.c Tue Aug 8 16:53:22 2017 (r322261) @@ -77,7 +77,9 @@ // Check linking against correct startup code when (not) using PIE // RUN: %clang -no-canonical-prefixes -target i686-pc-openbsd %s -### 2>&1 \ // RUN: | FileCheck -check-prefix=CHECK-PIE %s -// RUN: %clang -no-canonical-prefixes -target i686-pc-openbsd %s -fno-pie %s -### 2>&1 \ +// RUN: %clang -no-canonical-prefixes -target i686-pc-openbsd -pie %s -### 2>&1 \ +// RUN: | FileCheck -check-prefix=CHECK-PIE-FLAG %s +// RUN: %clang -no-canonical-prefixes -target i686-pc-openbsd -fno-pie %s -### 2>&1 \ // RUN: | FileCheck -check-prefix=CHECK-PIE %s // RUN: %clang -no-canonical-prefixes -target i686-pc-openbsd -static %s -### 2>&1 \ // RUN: | FileCheck -check-prefix=CHECK-STATIC-PIE %s @@ -93,6 +95,7 @@ // RUN: | FileCheck -check-prefix=CHECK-NOPIE %s // CHECK-PIE: "{{.*}}crt0.o" // CHECK-PIE-NOT: "-nopie" +// CHECK-PIE-FLAG: "-pie" // CHECK-STATIC-PIE: "{{.*}}rcrt0.o" // CHECK-STATIC-PIE-NOT: "-nopie" // CHECK-NOPIE: "-nopie" "{{.*}}crt0.o" Modified: vendor/clang/dist/test/Misc/caret-diags-multiline.cpp ============================================================================== --- vendor/clang/dist/test/Misc/caret-diags-multiline.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/test/Misc/caret-diags-multiline.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -232,3 +232,8 @@ void multiple_ranges(int a, int b) { b ); } + +#define pr33902_a(b) #b +#define pr33902_c(d) _Pragma(pr33902_a(d)) +#define pr33902_e(f) pr33902_c(GCC warning #f) +pr33902_e() pr33902_e() Added: vendor/clang/dist/test/PCH/coroutines.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist/test/PCH/coroutines.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -0,0 +1,77 @@ +// Test this without pch. +// RUN: %clang_cc1 -include %s -verify -std=c++1z -fcoroutines-ts %s + +// Test with pch. +// RUN: %clang_cc1 -std=c++1z -fcoroutines-ts -emit-pch -o %t %s +// RUN: %clang_cc1 -include-pch %t -verify -std=c++1z -fcoroutines-ts %s + +#ifndef HEADER +#define HEADER + +namespace std::experimental { +template struct coroutine_traits; + +template struct coroutine_handle { + coroutine_handle() = default; + static coroutine_handle from_address(void *) noexcept; +}; +template <> struct coroutine_handle { + static coroutine_handle from_address(void *) noexcept; + coroutine_handle() = default; + template + coroutine_handle(coroutine_handle) noexcept; +}; +} + +struct suspend_always { + bool await_ready() noexcept; + void await_suspend(std::experimental::coroutine_handle<>) noexcept; + void await_resume() noexcept; +}; + +template struct std::experimental::coroutine_traits { + struct promise_type { + void get_return_object() noexcept; + suspend_always initial_suspend() noexcept; + suspend_always final_suspend() noexcept; + void return_void() noexcept; + suspend_always yield_value(int) noexcept; + promise_type(); + ~promise_type() noexcept; + void unhandled_exception() noexcept; + }; +}; + +template struct std::experimental::coroutine_traits { + struct promise_type { + int get_return_object() noexcept; + suspend_always initial_suspend() noexcept; + suspend_always final_suspend() noexcept; + void return_value(int) noexcept; + promise_type(); + ~promise_type() noexcept; + void unhandled_exception() noexcept; + }; +}; + +template +void f(T x) { // checks coawait_expr and coroutine_body_stmt + co_yield 42; // checks coyield_expr + co_await x; // checks dependent_coawait + co_return; // checks coreturn_stmt +} + +template +int f2(T x) { // checks coawait_expr and coroutine_body_stmt + co_return x; // checks coreturn_stmt with expr +} + +#else + +// expected-no-diagnostics +void g() { + f(suspend_always{}); + f2(42); +} + +#endif Modified: vendor/clang/dist/test/Preprocessor/init.c ============================================================================== --- vendor/clang/dist/test/Preprocessor/init.c Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/test/Preprocessor/init.c Tue Aug 8 16:53:22 2017 (r322261) @@ -4686,16 +4686,6 @@ // RUN: | FileCheck -match-full-lines -check-prefix MIPS-MSA %s // MIPS-MSA:#define __mips_msa 1 // -// RUN: %clang_cc1 -target-feature +nomadd4 \ -// RUN: -E -dM -triple=mips-none-none < /dev/null \ -// RUN: | FileCheck -match-full-lines -check-prefix MIPS-NOMADD4 %s -// MIPS-NOMADD4:#define __mips_no_madd4 1 -// -// RUN: %clang_cc1 \ -// RUN: -E -dM -triple=mips-none-none < /dev/null \ -// RUN: | FileCheck -match-full-lines -check-prefix MIPS-MADD4 %s -// MIPS-MADD4-NOT:#define __mips_no_madd4 1 -// // RUN: %clang_cc1 -target-cpu mips32r3 -target-feature +nan2008 \ // RUN: -E -dM -triple=mips-none-none < /dev/null \ // RUN: | FileCheck -match-full-lines -check-prefix MIPS-NAN2008 %s Modified: vendor/clang/dist/test/Sema/vector-cast.c ============================================================================== --- vendor/clang/dist/test/Sema/vector-cast.c Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/test/Sema/vector-cast.c Tue Aug 8 16:53:22 2017 (r322261) @@ -48,6 +48,9 @@ typedef float float2 __attribute__ ((vector_size (8))) typedef __attribute__((vector_size(8))) double float64x1_t; typedef __attribute__((vector_size(16))) double float64x2_t; float64x1_t vget_low_f64(float64x2_t __p0); +typedef float float16 __attribute__((__vector_size__(16))); +typedef signed int vSInt32 __attribute__((__vector_size__(16))); +typedef unsigned int vUInt32 __attribute__((__vector_size__(16))); void f4() { float2 f2; @@ -72,4 +75,9 @@ void f5() { void *ptr; v = ptr; // expected-error-re {{assigning to 'short_sizeof_pointer' (vector of {{[0-9]+}} 'short' values) from incompatible type 'void *'}} ptr = v; // expected-error {{assigning to 'void *' from incompatible type 'short_sizeof_pointer'}} +} + +void f6(vSInt32 a0) { + vUInt32 counter = (float16){0.0f, 0.0f, 0.0f, 0.0f}; // expected-warning {{incompatible vector types initializing 'vUInt32' (vector of 4 'unsigned int' values) with an expression of type 'float16' (vector of 4 'float' values)}} + counter -= a0; } Modified: vendor/clang/dist/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp ============================================================================== --- vendor/clang/dist/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp Tue Aug 8 16:53:02 2017 (r322260) +++ vendor/clang/dist/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp Tue Aug 8 16:53:22 2017 (r322261) @@ -1,5 +1,10 @@ -// RUN: %clang_cc1 -std=c++1z -verify %s +// RUN: %clang_cc1 -std=c++1z -verify %s -DERRORS +// RUN: %clang_cc1 -std=c++1z -verify %s -UERRORS +// This test is split into two because we only produce "undefined internal" +// warnings if we didn't produce any errors. +#if ERRORS + namespace std { using size_t = decltype(sizeof(0)); template struct initializer_list { @@ -280,3 +285,21 @@ namespace tuple_tests { scoped_lock l = {}; } } + +#else + +// expected-no-diagnostics +namespace undefined_warnings { + // Make sure we don't get an "undefined but used internal symbol" warning for the deduction guide here. + namespace { + template + struct TemplDObj { + explicit TemplDObj(T func) noexcept {} + }; + auto test1 = TemplDObj(0); + + TemplDObj(float) -> TemplDObj; + auto test2 = TemplDObj(.0f); + } +} +#endif From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:32 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2282DC30AF; Tue, 8 Aug 2017 16:53:32 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A70D369CF1; Tue, 8 Aug 2017 16:53:32 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78GrVdn082672; Tue, 8 Aug 2017 16:53:31 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GrV6A082671; Tue, 8 Aug 2017 16:53:31 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GrV6A082671@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322262 - vendor/clang/clang-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/clang/clang-release_50-r310316 X-SVN-Commit-Revision: 322262 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:33 -0000 Author: dim Date: Tue Aug 8 16:53:31 2017 New Revision: 322262 URL: https://svnweb.freebsd.org/changeset/base/322262 Log: Tag clang release_50 branch r310316. Added: vendor/clang/clang-release_50-r310316/ - copied from r322261, vendor/clang/dist/ From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:36 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 52E2FDC30F6; Tue, 8 Aug 2017 16:53:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0779569D25; Tue, 8 Aug 2017 16:53:35 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78GrZWk082718; Tue, 8 Aug 2017 16:53:35 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GrZKx082717; Tue, 8 Aug 2017 16:53:35 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GrZKx082717@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322263 - vendor/compiler-rt/compiler-rt-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/compiler-rt-release_50-r310316 X-SVN-Commit-Revision: 322263 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:36 -0000 Author: dim Date: Tue Aug 8 16:53:34 2017 New Revision: 322263 URL: https://svnweb.freebsd.org/changeset/base/322263 Log: Tag compiler-rt release_50 branch r310316. Added: vendor/compiler-rt/compiler-rt-release_50-r310316/ - copied from r322262, vendor/compiler-rt/dist/ From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:42 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DA0DDC317A; Tue, 8 Aug 2017 16:53:42 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5DC8669DDB; Tue, 8 Aug 2017 16:53:42 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Grfct082777; Tue, 8 Aug 2017 16:53:41 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GreBB082767; Tue, 8 Aug 2017 16:53:40 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GreBB082767@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322264 - in vendor/libc++/dist: include test/std/input.output/string.streams/stringstream.cons test/std/thread/thread.mutex/thread.lock/thread.lock.scoped test/std/utilities/variant/va... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/libc++/dist: include test/std/input.output/string.streams/stringstream.cons test/std/thread/thread.mutex/thread.lock/thread.lock.scoped test/std/utilities/variant/variant.variant/variant.cto... X-SVN-Commit-Revision: 322264 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:42 -0000 Author: dim Date: Tue Aug 8 16:53:40 2017 New Revision: 322264 URL: https://svnweb.freebsd.org/changeset/base/322264 Log: Vendor import of libc++ release_50 branch r310316: https://llvm.org/svn/llvm-project/libcxx/branches/release_50@310316 Modified: vendor/libc++/dist/include/__bsd_locale_defaults.h vendor/libc++/dist/include/__bsd_locale_fallbacks.h vendor/libc++/dist/include/__locale vendor/libc++/dist/include/mutex vendor/libc++/dist/include/sstream vendor/libc++/dist/test/std/input.output/string.streams/stringstream.cons/string.pass.cpp vendor/libc++/dist/test/std/thread/thread.mutex/thread.lock/thread.lock.scoped/adopt_lock.pass.cpp vendor/libc++/dist/test/std/utilities/variant/variant.variant/variant.ctor/copy.pass.cpp vendor/libc++/dist/www/cxx1z_status.html vendor/libc++/dist/www/cxx2a_status.html Modified: vendor/libc++/dist/include/__bsd_locale_defaults.h ============================================================================== --- vendor/libc++/dist/include/__bsd_locale_defaults.h Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/include/__bsd_locale_defaults.h Tue Aug 8 16:53:40 2017 (r322264) @@ -15,6 +15,10 @@ #ifndef _LIBCPP_BSD_LOCALE_DEFAULTS_H #define _LIBCPP_BSD_LOCALE_DEFAULTS_H +#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) +#pragma GCC system_header +#endif + #define __libcpp_mb_cur_max_l(loc) MB_CUR_MAX_L(loc) #define __libcpp_btowc_l(ch, loc) btowc_l(ch, loc) #define __libcpp_wctob_l(wch, loc) wctob_l(wch, loc) Modified: vendor/libc++/dist/include/__bsd_locale_fallbacks.h ============================================================================== --- vendor/libc++/dist/include/__bsd_locale_fallbacks.h Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/include/__bsd_locale_fallbacks.h Tue Aug 8 16:53:40 2017 (r322264) @@ -18,6 +18,10 @@ #include #include +#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) +#pragma GCC system_header +#endif + _LIBCPP_BEGIN_NAMESPACE_STD inline _LIBCPP_ALWAYS_INLINE Modified: vendor/libc++/dist/include/__locale ============================================================================== --- vendor/libc++/dist/include/__locale Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/include/__locale Tue Aug 8 16:53:40 2017 (r322264) @@ -34,7 +34,7 @@ # include #elif defined(_NEWLIB_VERSION) # include -#elif (defined(__GLIBC__) || defined(__APPLE__) || defined(__FreeBSD__) \ +#elif (defined(__APPLE__) || defined(__FreeBSD__) \ || defined(__EMSCRIPTEN__) || defined(__IBMCPP__)) # include #elif defined(__Fuchsia__) Modified: vendor/libc++/dist/include/mutex ============================================================================== --- vendor/libc++/dist/include/mutex Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/include/mutex Tue Aug 8 16:53:40 2017 (r322264) @@ -116,7 +116,7 @@ class scoped_lock // C++17 (public) using mutex_type = Mutex; // If MutexTypes... consists of the single type Mutex explicit scoped_lock(MutexTypes&... m); - scoped_lock(MutexTypes&... m, adopt_lock_t); + scoped_lock(adopt_lock_t, MutexTypes&... m); ~scoped_lock(); scoped_lock(scoped_lock const&) = delete; scoped_lock& operator=(scoped_lock const&) = delete; @@ -500,7 +500,7 @@ class _LIBCPP_TEMPLATE_VIS scoped_lock<_Mutex> { (publ ~scoped_lock() _LIBCPP_THREAD_SAFETY_ANNOTATION(release_capability()) {__m_.unlock();} _LIBCPP_INLINE_VISIBILITY - explicit scoped_lock(mutex_type& __m, adopt_lock_t) _LIBCPP_THREAD_SAFETY_ANNOTATION(requires_capability(__m)) + explicit scoped_lock(adopt_lock_t, mutex_type& __m) _LIBCPP_THREAD_SAFETY_ANNOTATION(requires_capability(__m)) : __m_(__m) {} scoped_lock(scoped_lock const&) = delete; @@ -522,7 +522,7 @@ class _LIBCPP_TEMPLATE_VIS scoped_lock (public) } _LIBCPP_INLINE_VISIBILITY - scoped_lock(_MArgs&... __margs, adopt_lock_t) + scoped_lock(adopt_lock_t, _MArgs&... __margs) : __t_(__margs...) { } Modified: vendor/libc++/dist/include/sstream ============================================================================== --- vendor/libc++/dist/include/sstream Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/include/sstream Tue Aug 8 16:53:40 2017 (r322264) @@ -249,7 +249,8 @@ basic_stringbuf<_CharT, _Traits, _Allocator>::basic_st template basic_stringbuf<_CharT, _Traits, _Allocator>::basic_stringbuf(const string_type& __s, ios_base::openmode __wch) - : __hm_(0), + : __str_(__s.get_allocator()), + __hm_(0), __mode_(__wch) { str(__s); Modified: vendor/libc++/dist/test/std/input.output/string.streams/stringstream.cons/string.pass.cpp ============================================================================== --- vendor/libc++/dist/test/std/input.output/string.streams/stringstream.cons/string.pass.cpp Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/test/std/input.output/string.streams/stringstream.cons/string.pass.cpp Tue Aug 8 16:53:40 2017 (r322264) @@ -18,6 +18,16 @@ #include #include +template +struct NoDefaultAllocator : std::allocator +{ + template struct rebind { using other = NoDefaultAllocator; }; + NoDefaultAllocator(int id_) : id(id_) { } + template NoDefaultAllocator(const NoDefaultAllocator& a) : id(a.id) { } + int id; +}; + + int main() { { @@ -45,5 +55,14 @@ int main() assert(i == 456); ss << i << ' ' << 123; assert(ss.str() == L"456 1236 "); + } + { // This is https://bugs.llvm.org/show_bug.cgi?id=33727 + typedef std::basic_string , NoDefaultAllocator > S; + typedef std::basic_stringbuf, NoDefaultAllocator > SB; + + S s(NoDefaultAllocator(1)); + SB sb(s); + // This test is not required by the standard, but *where else* could it get the allocator? + assert(sb.str().get_allocator() == s.get_allocator()); } } Modified: vendor/libc++/dist/test/std/thread/thread.mutex/thread.lock/thread.lock.scoped/adopt_lock.pass.cpp ============================================================================== --- vendor/libc++/dist/test/std/thread/thread.mutex/thread.lock/thread.lock.scoped/adopt_lock.pass.cpp Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/test/std/thread/thread.mutex/thread.lock/thread.lock.scoped/adopt_lock.pass.cpp Tue Aug 8 16:53:40 2017 (r322264) @@ -14,7 +14,7 @@ // template class scoped_lock; -// scoped_lock(Mutex&..., adopt_lock_t); +// scoped_lock(adopt_lock_t, Mutex&...); #include #include @@ -43,7 +43,7 @@ int main() using LG = std::scoped_lock; m1.lock(); { - LG lg(m1, std::adopt_lock); + LG lg(std::adopt_lock, m1); assert(m1.locked); } assert(!m1.locked); @@ -53,7 +53,7 @@ int main() using LG = std::scoped_lock; m1.lock(); m2.lock(); { - LG lg(m1, m2, std::adopt_lock); + LG lg(std::adopt_lock, m1, m2); assert(m1.locked && m2.locked); } assert(!m1.locked && !m2.locked); @@ -63,7 +63,7 @@ int main() using LG = std::scoped_lock; m1.lock(); m2.lock(); m3.lock(); { - LG lg(m1, m2, m3, std::adopt_lock); + LG lg(std::adopt_lock, m1, m2, m3); assert(m1.locked && m2.locked && m3.locked); } assert(!m1.locked && !m2.locked && !m3.locked); Modified: vendor/libc++/dist/test/std/utilities/variant/variant.variant/variant.ctor/copy.pass.cpp ============================================================================== --- vendor/libc++/dist/test/std/utilities/variant/variant.variant/variant.ctor/copy.pass.cpp Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/test/std/utilities/variant/variant.variant/variant.ctor/copy.pass.cpp Tue Aug 8 16:53:40 2017 (r322264) @@ -261,4 +261,13 @@ int main() { test_copy_ctor_valueless_by_exception(); test_copy_ctor_sfinae(); test_constexpr_copy_ctor_extension(); +#if 0 +// disable this for the moment; it fails on older compilers. +// Need to figure out which compilers will support it. +{ // This is the motivating example from P0739R0 + std::variant v1(3); + std::variant v2 = v1; + (void) v2; +} +#endif } Modified: vendor/libc++/dist/www/cxx1z_status.html ============================================================================== --- vendor/libc++/dist/www/cxx1z_status.html Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/www/cxx1z_status.html Tue Aug 8 16:53:40 2017 (r322264) @@ -39,6 +39,8 @@

In February 2017, the C++ standard committee approved this draft, and sent it to ISO for approval as C++17

This page shows the status of libc++; the status of clang's support of the language features is here.

+

Reminder: Features in unreleased drafts of the standard are subject to change.

+

The groups that have contributed papers:

  • LWG - Library working group
    • @@ -163,7 +165,7 @@ P0623R0LWGFinal C++17 Parallel Algorithms FixesKona P0682R1LWGRepairing elementary string conversionsToronto - P0739R0LWGSome improvements to class template argument deduction integration into the standard libraryToronto + P0739R0LWGSome improvements to class template argument deduction integration into the standard libraryTorontoComplete5.0 @@ -497,7 +499,7 @@ -

    Last Updated: 25-May-2017

    +

    Last Updated: 27-Jul-2017

    Modified: vendor/libc++/dist/www/cxx2a_status.html ============================================================================== --- vendor/libc++/dist/www/cxx2a_status.html Tue Aug 8 16:53:34 2017 (r322263) +++ vendor/libc++/dist/www/cxx2a_status.html Tue Aug 8 16:53:40 2017 (r322264) @@ -38,6 +38,8 @@

    In July 2017, the C++ standard committee created a draft for the next version of the C++ standard, known here as "C++2a" (probably to be C++20).

    This page shows the status of libc++; the status of clang's support of the language features is here.

    +

    Reminder: Features in unreleased drafts of the standard are subject to change.

    +

    The groups that have contributed papers:

    • LWG - Library working group
      • From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:45 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3167DC319F; Tue, 8 Aug 2017 16:53:45 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91D6D69E2C; Tue, 8 Aug 2017 16:53:45 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78GriYs082827; Tue, 8 Aug 2017 16:53:44 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GriXe082826; Tue, 8 Aug 2017 16:53:44 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GriXe082826@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322265 - vendor/libc++/libc++-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/libc++-release_50-r310316 X-SVN-Commit-Revision: 322265 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:46 -0000 Author: dim Date: Tue Aug 8 16:53:44 2017 New Revision: 322265 URL: https://svnweb.freebsd.org/changeset/base/322265 Log: Tag libc++ release_50 branch r310316. Added: vendor/libc++/libc++-release_50-r310316/ - copied from r322264, vendor/libc++/dist/ From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:50 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C806DC320F; Tue, 8 Aug 2017 16:53:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0971B69EA5; Tue, 8 Aug 2017 16:53:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78GrnUV082874; Tue, 8 Aug 2017 16:53:49 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GrnL0082873; Tue, 8 Aug 2017 16:53:49 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GrnL0082873@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322266 - vendor/lld/lld-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lld/lld-release_50-r310316 X-SVN-Commit-Revision: 322266 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:50 -0000 Author: dim Date: Tue Aug 8 16:53:48 2017 New Revision: 322266 URL: https://svnweb.freebsd.org/changeset/base/322266 Log: Tag lld release_50 branch r310316. Added: vendor/lld/lld-release_50-r310316/ - copied from r322265, vendor/lld/dist/ From owner-svn-src-vendor@freebsd.org Tue Aug 8 16:53:54 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3206ADC326D; Tue, 8 Aug 2017 16:53:54 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D447369F21; Tue, 8 Aug 2017 16:53:53 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v78Grrgi082923; Tue, 8 Aug 2017 16:53:53 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v78GrrHQ082922; Tue, 8 Aug 2017 16:53:53 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201708081653.v78GrrHQ082922@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Tue, 8 Aug 2017 16:53:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322267 - vendor/lldb/lldb-release_50-r310316 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/lldb-release_50-r310316 X-SVN-Commit-Revision: 322267 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 16:53:54 -0000 Author: dim Date: Tue Aug 8 16:53:52 2017 New Revision: 322267 URL: https://svnweb.freebsd.org/changeset/base/322267 Log: Tag lldb release_50 branch r310316. Added: vendor/lldb/lldb-release_50-r310316/ - copied from r322266, vendor/lldb/dist/ From owner-svn-src-vendor@freebsd.org Thu Aug 10 21:48:37 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74E3CDDDB8E; Thu, 10 Aug 2017 21:48:37 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1EA9072A10; Thu, 10 Aug 2017 21:48:37 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7ALma4i014091; Thu, 10 Aug 2017 21:48:36 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7ALmY8i014074; Thu, 10 Aug 2017 21:48:34 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201708102148.v7ALmY8i014074@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Thu, 10 Aug 2017 21:48:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322377 - in vendor/subversion/dist: . subversion/include subversion/libsvn_client subversion/libsvn_fs_fs subversion/libsvn_fs_x subversion/libsvn_ra_svn subversion/libsvn_repos subver... X-SVN-Group: vendor X-SVN-Commit-Author: peter X-SVN-Commit-Paths: in vendor/subversion/dist: . subversion/include subversion/libsvn_client subversion/libsvn_fs_fs subversion/libsvn_fs_x subversion/libsvn_ra_svn subversion/libsvn_repos subversion/libsvn_subr subversi... X-SVN-Commit-Revision: 322377 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 21:48:37 -0000 Author: peter Date: Thu Aug 10 21:48:34 2017 New Revision: 322377 URL: https://svnweb.freebsd.org/changeset/base/322377 Log: Import subversion-1.9.7 Modified: vendor/subversion/dist/CHANGES vendor/subversion/dist/NOTICE vendor/subversion/dist/build-outputs.mk vendor/subversion/dist/configure vendor/subversion/dist/subversion/include/svn_version.h vendor/subversion/dist/subversion/libsvn_client/copy.c vendor/subversion/dist/subversion/libsvn_client/merge.c vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.c vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.h vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.h vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.c vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.c vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.h vendor/subversion/dist/subversion/libsvn_ra_svn/client.c vendor/subversion/dist/subversion/libsvn_repos/dump.c vendor/subversion/dist/subversion/libsvn_subr/config_file.c vendor/subversion/dist/subversion/libsvn_subr/internal_statements.h vendor/subversion/dist/subversion/libsvn_subr/io.c vendor/subversion/dist/subversion/libsvn_subr/version.c vendor/subversion/dist/subversion/libsvn_wc/wc-checks.h vendor/subversion/dist/subversion/libsvn_wc/wc-metadata.h vendor/subversion/dist/subversion/libsvn_wc/wc-queries.h vendor/subversion/dist/subversion/svnadmin/svnadmin.c vendor/subversion/dist/win-tests.py Modified: vendor/subversion/dist/CHANGES ============================================================================== --- vendor/subversion/dist/CHANGES Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/CHANGES Thu Aug 10 21:48:34 2017 (r322377) @@ -1,3 +1,63 @@ +Version 1.9.7 +(10 Aug 2017, from /branches/1.9.x) +http://svn.apache.org/repos/asf/subversion/tags/1.9.7 + + User-visible changes: + - Client-side bugfixes: + * Fix arbitrary code execution vulnerability CVE-2017-9800 + See + for details. + + - Server-side bugfixes: + (none) + + - Bindings bugfixes: + (none) + + Developer-visible changes: + - General: + (none) + + - API changes: + (none) + + +Version 1.9.6 +(5 Jul 2017, from /branches/1.9.x) +http://svn.apache.org/repos/asf/subversion/tags/1.9.6 + + User-visible changes: + - Client-side bugfixes: + * cp/mv: improve error message when target is an unversioned dir (r1779948) + * merge: reduce memory usage with large amounts of mergeinfo (issue #4667) + + - Server-side bugfixes: + * 'svnadmin freeze': document the purpose more clearly (r1774109) + * dump: fix segfault when a revision has no revprops (r1781507) + * fsfs: improve error message upon failure to open rep-cache (r1781655) + * fsfs: never attempt to share directory representations (r1785053) + * fsfs: make consistency independent of hash algorithms (r1785737 et al) + This change makes Subversion resilient to collision attacks, including + SHA-1 collision attacks such as . See also our + documentation at and + . + + - Client-side and server-side bugfixes: + * work around an APR bug related to file truncation (r1759116) + + - Bindings bugfixes: + * javahl: follow redirects when opening a connection (r1667738, r1796720) + + Developer-visible changes: + - General: + * win_tests.py: make the --bin option work, rather than abort (r1706432) + (regression introduced in 1.9.2) + * windows: support building with 'zlibstat.lib' in install-layout (r1783704) + + - API changes: + (none) + + Version 1.9.5 (29 Nov 2016, from /branches/1.9.x) http://svn.apache.org/repos/asf/subversion/tags/1.9.5 @@ -19,7 +79,7 @@ http://svn.apache.org/repos/asf/subversion/tags/1.9.5 * fsfs: fix "offset too large" error during pack (issue #4657) * svnserve: enable hook script environments (r1769152) * fsfs: fix possible data reconstruction error (issue #4658) - * fix source of spurious 'incoming edit' tree conflicts (r1770108) + * fix source of spurious 'incoming edit' tree conflicts (r1760570) * fsfs: improve caching for large directories (r1721285) * fsfs: fix crash when encountering all-zero checksums (r1759686) * fsfs: fix potential source of repository corruptions (r1756266) @@ -34,19 +94,19 @@ http://svn.apache.org/repos/asf/subversion/tags/1.9.5 - Bindings bugfixes: * swig-pl: do not corrupt "{DATE}" revision variable (r1767768) - * javahl: fix temporary accepting SSL server certificates (r1764851) + * javahl: fix temporarily accepting SSL server certificates (r1764851) * swig-pl: fix possible stack corruption (r1683266, r1683267) Developer-visible changes: - General: * add zlib discovery through pkg-config (issue #4655) * fix potential build issue with invalid SVN_LOCALE_DIR (issue #4653) - * ruby: fix test failures with ruby >= 2.2 (r1766621) - * fix link error with --disable-keychain on OS X (r1765385) + * ruby: fix test failures with ruby >= 2.2 (r1766240) + * fix link error with --disable-keychain on OS X (r1761755) * swig: enable building with SWIG >= 3.0.6 (r1721488 et al) * swig: fix building with -Wdate-time in $CPPFLAGS (r1722164) * update serf download URI in build scripts (r1700130 et al) - * raise minimal httpd version from 2.0 to 2.2 (r1754193) + * raise minimal httpd version from 2.0 to 2.2 (r1754190) Version 1.9.4 @@ -105,7 +165,6 @@ http://svn.apache.org/repos/asf/subversion/tags/1.9.3 * svn: report lock/unlock errors as failures (r1701598 et al) * svn: cleanup user deleted external registrations (r1705843, r1710558) * svn: allow simple resolving of binary file text conflicts (r1703581) - * svnlook: properly remove tempfiles on diff errors (r1711346) * ra_serf: report built- and run-time versions of libserf (r1704847) * ra_serf: set Content-Type header in outgoing requests (r1715224 et al) * svn: fix merging deletes of svn:eol-style CRLF/CR files (r1703689 et al) @@ -118,6 +177,7 @@ http://svn.apache.org/repos/asf/subversion/tags/1.9.3 * mod_dav_svn: use LimitXMLRequestBody for skel-encoded requests (r1687812) * svnadmin dump: preserve no-op changes (r1709388 et al, issue #4598) * fsfs: avoid unneeded I/O when opening transactions (r1715793) + * svnlook: properly remove tempfiles on diff errors (r1711346) - Client-side and server-side bugfixes: * fix heap overflow in svn:// protocol parser (CVE-2015-5259) @@ -942,18 +1002,18 @@ http://svn.apache.org/repos/asf/subversion/tags/1.8.17 - Client-side bugfixes: * fix handling of newly secured subdirectories in working copy (r1724448) * ra_serf: fix deleting directories with many files (issue #4557) - * svnlook: properly remove tempfiles on diff errors (r1711346) * gpg-agent: properly handle passwords with percent characters (issue #4611) * merge: fix crash when merging to a local add (r1702299 et al) - Server-side bugfixes: * fsfs: fix possible data reconstruction error (issue #4658) + * svnlook: properly remove tempfiles on diff errors (r1711346) - Client-side and server-side bugfixes: * fix potential memory access bugs (r1722860 et al) - Bindings bugfixes: - * javahl: fix temporary accepting SSL server certificates (r1764851) + * javahl: fix temporarily accepting SSL server certificates (r1764851) * swig-pl: do not corrupt "{DATE}" revision variable (r1767768) * swig-pl: fix possible stack corruption (r1683266) Modified: vendor/subversion/dist/NOTICE ============================================================================== --- vendor/subversion/dist/NOTICE Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/NOTICE Thu Aug 10 21:48:34 2017 (r322377) @@ -1,5 +1,5 @@ Apache Subversion -Copyright 2016 The Apache Software Foundation +Copyright 2017 The Apache Software Foundation This product includes software developed by many people, and distributed under Contributor License Agreements to The Apache Software Foundation Modified: vendor/subversion/dist/build-outputs.mk ============================================================================== --- vendor/subversion/dist/build-outputs.mk Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/build-outputs.mk Thu Aug 10 21:48:34 2017 (r322377) @@ -2782,7 +2782,7 @@ subversion/libsvn_ra_serf/util_error.lo: subversion/li subversion/libsvn_ra_serf/xml.lo: subversion/libsvn_ra_serf/xml.c subversion/include/private/svn_dav_protocol.h subversion/include/private/svn_debug.h subversion/include/private/svn_editor.h subversion/include/private/svn_ra_private.h subversion/include/private/svn_string_private.h subversion/include/private/svn_subr_private.h subversion/include/svn_auth.h subversion/include/svn_checksum.h subversion/include/svn_config.h subversion/include/svn_dav.h subversion/include/svn_delta.h subversion/include/svn_dirent_uri.h subversion/include/svn_error.h subversion/include/svn_error_codes.h subversion/include/svn_hash.h subversion/include/svn_io.h subversion/include/svn_mergeinfo.h subversion/include/svn_path.h subversion/include/svn_pools.h subversion/include/svn_ra.h subversion/include/svn_string.h subversion/include/svn_types.h subversion/include/svn_version.h subversion/include/svn_xml.h subversion/libsvn_ra/ra_loader.h subversion/libsvn_ra_serf/blncache.h subversion/libsvn_ra_serf/ra_se rf.h subversion/svn_private_config.h -subversion/libsvn_ra_svn/client.lo: subversion/libsvn_ra_svn/client.c subversion/include/private/svn_debug.h subversion/include/private/svn_editor.h subversion/include/private/svn_fspath.h subversion/include/private/svn_ra_private.h subversion/include/private/svn_ra_svn_private.h subversion/include/private/svn_subr_private.h subversion/include/svn_auth.h subversion/include/svn_checksum.h subversion/include/svn_compat.h subversion/include/svn_config.h subversion/include/svn_delta.h subversion/include/svn_dirent_uri.h subversion/include/svn_error.h subversion/include/svn_error_codes.h subversion/include/svn_hash.h subversion/include/svn_io.h subversion/include/svn_mergeinfo.h subversion/include/svn_path.h subversion/include/svn_pools.h subversion/include/svn_props.h subversion/include/svn_ra.h subversion/include/svn_ra_svn.h subversion/include/svn_string.h subversion/include/svn_time.h subversion/include/svn_types.h subversion/include/svn_version.h subversion/libsvn_ra/ra_loader.h sub version/libsvn_ra/wrapper_template.h subversion/libsvn_ra_svn/ra_svn.h subversion/svn_private_config.h +subversion/libsvn_ra_svn/client.lo: subversion/libsvn_ra_svn/client.c subversion/include/private/svn_debug.h subversion/include/private/svn_editor.h subversion/include/private/svn_fspath.h subversion/include/private/svn_ra_private.h subversion/include/private/svn_ra_svn_private.h subversion/include/private/svn_subr_private.h subversion/include/svn_auth.h subversion/include/svn_checksum.h subversion/include/svn_compat.h subversion/include/svn_config.h subversion/include/svn_ctype.h subversion/include/svn_delta.h subversion/include/svn_dirent_uri.h subversion/include/svn_error.h subversion/include/svn_error_codes.h subversion/include/svn_hash.h subversion/include/svn_io.h subversion/include/svn_mergeinfo.h subversion/include/svn_path.h subversion/include/svn_pools.h subversion/include/svn_props.h subversion/include/svn_ra.h subversion/include/svn_ra_svn.h subversion/include/svn_string.h subversion/include/svn_time.h subversion/include/svn_types.h subversion/include/svn_version.h subve rsion/libsvn_ra/ra_loader.h subversion/libsvn_ra/wrapper_template.h subversion/libsvn_ra_svn/ra_svn.h subversion/svn_private_config.h subversion/libsvn_ra_svn/cram.lo: subversion/libsvn_ra_svn/cram.c subversion/include/private/svn_debug.h subversion/include/private/svn_editor.h subversion/include/private/svn_ra_svn_private.h subversion/include/svn_auth.h subversion/include/svn_checksum.h subversion/include/svn_config.h subversion/include/svn_delta.h subversion/include/svn_error.h subversion/include/svn_error_codes.h subversion/include/svn_io.h subversion/include/svn_mergeinfo.h subversion/include/svn_ra.h subversion/include/svn_ra_svn.h subversion/include/svn_string.h subversion/include/svn_types.h subversion/libsvn_ra_svn/ra_svn.h subversion/svn_private_config.h Modified: vendor/subversion/dist/configure ============================================================================== --- vendor/subversion/dist/configure Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/configure Thu Aug 10 21:48:34 2017 (r322377) @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for subversion 1.9.5. +# Generated by GNU Autoconf 2.69 for subversion 1.9.7. # # Report bugs to . # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='subversion' PACKAGE_TARNAME='subversion' -PACKAGE_VERSION='1.9.5' -PACKAGE_STRING='subversion 1.9.5' +PACKAGE_VERSION='1.9.7' +PACKAGE_STRING='subversion 1.9.7' PACKAGE_BUGREPORT='http://subversion.apache.org/' PACKAGE_URL='' @@ -1471,7 +1471,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures subversion 1.9.5 to adapt to many kinds of systems. +\`configure' configures subversion 1.9.7 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1537,7 +1537,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of subversion 1.9.5:";; + short | recursive ) echo "Configuration of subversion 1.9.7:";; esac cat <<\_ACEOF @@ -1751,7 +1751,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -subversion configure 1.9.5 +subversion configure 1.9.7 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2295,7 +2295,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by subversion $as_me 1.9.5, which was +It was created by subversion $as_me 1.9.7, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2675,8 +2675,8 @@ ac_configure="$SHELL $ac_aux_dir/configure" # Please -{ $as_echo "$as_me:${as_lineno-$LINENO}: Configuring Subversion 1.9.5" >&5 -$as_echo "$as_me: Configuring Subversion 1.9.5" >&6;} +{ $as_echo "$as_me:${as_lineno-$LINENO}: Configuring Subversion 1.9.7" >&5 +$as_echo "$as_me: Configuring Subversion 1.9.7" >&6;} abs_srcdir="`cd $srcdir && pwd`" @@ -26756,7 +26756,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by subversion $as_me 1.9.5, which was +This file was extended by subversion $as_me 1.9.7, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -26822,7 +26822,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -subversion config.status 1.9.5 +subversion config.status 1.9.7 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Modified: vendor/subversion/dist/subversion/include/svn_version.h ============================================================================== --- vendor/subversion/dist/subversion/include/svn_version.h Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/include/svn_version.h Thu Aug 10 21:48:34 2017 (r322377) @@ -70,7 +70,7 @@ extern "C" { * * @since New in 1.1. */ -#define SVN_VER_PATCH 5 +#define SVN_VER_PATCH 7 /** @deprecated Provided for backward compatibility with the 1.0 API. */ @@ -93,7 +93,7 @@ extern "C" { * * Always change this at the same time as SVN_VER_NUMTAG. */ -#define SVN_VER_TAG " (r1770682)" +#define SVN_VER_TAG " (r1800392)" /** Number tag: a string describing the version. @@ -117,7 +117,7 @@ extern "C" { * file version. Its value remains 0 in the repository except in release * tags where it is the revision from which the tag was created. */ -#define SVN_VER_REVISION 1770682 +#define SVN_VER_REVISION 1800392 /* Version strings composed from the above definitions. */ Modified: vendor/subversion/dist/subversion/libsvn_client/copy.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_client/copy.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_client/copy.c Thu Aug 10 21:48:34 2017 (r322377) @@ -1057,10 +1057,24 @@ verify_wc_dsts(const apr_array_header_t *copy_pairs, ctx->wc_ctx, pair->dst_parent_abspath, FALSE, TRUE, iterpool)); - if (make_parents && dst_parent_kind == svn_node_none) + if (dst_parent_kind == svn_node_none) { - SVN_ERR(svn_client__make_local_parents(pair->dst_parent_abspath, - TRUE, ctx, iterpool)); + if (make_parents) + SVN_ERR(svn_client__make_local_parents(pair->dst_parent_abspath, + TRUE, ctx, iterpool)); + else + { + SVN_ERR(svn_io_check_path(pair->dst_parent_abspath, + &dst_parent_kind, scratch_pool)); + return svn_error_createf(SVN_ERR_WC_PATH_NOT_FOUND, NULL, + (dst_parent_kind == svn_node_dir) + ? _("Directory '%s' is not under " + "version control") + : _("Path '%s' is not a directory"), + svn_dirent_local_style( + pair->dst_parent_abspath, + scratch_pool)); + } } else if (dst_parent_kind != svn_node_dir) { Modified: vendor/subversion/dist/subversion/libsvn_client/merge.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_client/merge.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_client/merge.c Thu Aug 10 21:48:34 2017 (r322377) @@ -6465,6 +6465,7 @@ get_mergeinfo_paths(apr_array_header_t *children_with_ { int i; apr_pool_t *iterpool = svn_pool_create(scratch_pool); + apr_pool_t *swmi_pool; apr_hash_t *subtrees_with_mergeinfo; apr_hash_t *excluded_subtrees; apr_hash_t *switched_subtrees; @@ -6473,10 +6474,13 @@ get_mergeinfo_paths(apr_array_header_t *children_with_ struct pre_merge_status_baton_t pre_merge_status_baton; /* Case 1: Subtrees with explicit mergeinfo. */ + /* Use a subpool for subtrees_with_mergeinfo, as it can be very large + and is temporary. */ + swmi_pool = svn_pool_create(scratch_pool); SVN_ERR(get_wc_explicit_mergeinfo_catalog(&subtrees_with_mergeinfo, target->abspath, depth, ctx, - result_pool, scratch_pool)); + swmi_pool, swmi_pool)); if (subtrees_with_mergeinfo) { apr_hash_index_t *hi; @@ -6513,6 +6517,7 @@ get_mergeinfo_paths(apr_array_header_t *children_with_ children_with_mergeinfo->elt_size, compare_merge_path_t_as_paths); } + svn_pool_destroy(swmi_pool); /* Case 2: Switched subtrees Case 10: Paths at depths of 'empty' or 'files' @@ -12331,6 +12336,10 @@ find_last_merged_location(svn_client__pathrev_t **base svn_revnum_t youngest_merged_rev = SVN_INVALID_REVNUM; svn_mergeinfo_catalog_t target_mergeinfo_cat = NULL; + /* Using a local subpool for 'target_mergeinfo_cat' can make a big + reduction in overall memory usage. */ + apr_pool_t *tmic_pool = svn_pool_create(scratch_pool); + source_peg_rev.kind = svn_opt_revision_number; source_peg_rev.value.number = source_branch->tip->rev; source_start_rev.kind = svn_opt_revision_number; @@ -12351,7 +12360,7 @@ find_last_merged_location(svn_client__pathrev_t **base operative_rev_receiver, &youngest_merged_rev, ctx, ra_session, - result_pool, scratch_pool)); + tmic_pool, tmic_pool)); if (!SVN_IS_VALID_REVNUM(youngest_merged_rev)) { @@ -12387,7 +12396,7 @@ find_last_merged_location(svn_client__pathrev_t **base operative_rev_receiver, &oldest_eligible_rev, ctx, ra_session, - scratch_pool, scratch_pool)); + tmic_pool, tmic_pool)); /* If there are revisions eligible for merging, use the oldest one to calculate the base. Otherwise there are no operative revisions @@ -12409,6 +12418,7 @@ find_last_merged_location(svn_client__pathrev_t **base result_pool, scratch_pool)); } + svn_pool_destroy(tmic_pool); return SVN_NO_ERROR; } Modified: vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.c Thu Aug 10 21:48:34 2017 (r322377) @@ -2029,8 +2029,13 @@ rep_read_contents(void *baton, SVN_ERR(skip_contents(rb, rb->fulltext_delivered)); } - /* Get the next block of data. */ - SVN_ERR(get_contents_from_windows(rb, buf, len)); + /* Get the next block of data. + * Keep in mind that the representation might be empty and leave us + * already positioned at the end of the rep. */ + if (rb->off == rb->len) + *len = 0; + else + SVN_ERR(get_contents_from_windows(rb, buf, len)); if (rb->current_fulltext) svn_stringbuf_appendbytes(rb->current_fulltext, buf, *len); @@ -2119,6 +2124,96 @@ svn_fs_fs__get_contents(svn_stream_t **contents_p, rep_read_contents); svn_stream_set_close(*contents_p, rep_read_contents_close); } + + return SVN_NO_ERROR; +} + +svn_error_t * +svn_fs_fs__get_contents_from_file(svn_stream_t **contents_p, + svn_fs_t *fs, + representation_t *rep, + apr_file_t *file, + apr_off_t offset, + apr_pool_t *pool) +{ + struct rep_read_baton *rb; + pair_cache_key_t fulltext_cache_key = { SVN_INVALID_REVNUM, 0 }; + rep_state_t *rs = apr_pcalloc(pool, sizeof(*rs)); + svn_fs_fs__rep_header_t *rh; + + /* Initialize the reader baton. Some members may added lazily + * while reading from the stream. */ + SVN_ERR(rep_read_get_baton(&rb, fs, rep, fulltext_cache_key, pool)); + + /* Continue constructing RS. Leave caches as NULL. */ + rs->size = rep->size; + rs->revision = SVN_INVALID_REVNUM; + rs->item_index = 0; + rs->ver = -1; + rs->start = -1; + + /* Provide just enough file access info to allow for a basic read from + * FILE but leave all index / footer info with empty values b/c FILE + * probably is not a complete revision file. */ + rs->sfile = apr_pcalloc(pool, sizeof(*rs->sfile)); + rs->sfile->revision = rep->revision; + rs->sfile->pool = pool; + rs->sfile->fs = fs; + rs->sfile->rfile = apr_pcalloc(pool, sizeof(*rs->sfile->rfile)); + rs->sfile->rfile->start_revision = SVN_INVALID_REVNUM; + rs->sfile->rfile->file = file; + rs->sfile->rfile->stream = svn_stream_from_aprfile2(file, TRUE, pool); + + /* Read the rep header. */ + SVN_ERR(aligned_seek(fs, file, NULL, offset, pool)); + SVN_ERR(svn_fs_fs__read_rep_header(&rh, rs->sfile->rfile->stream, + pool, pool)); + SVN_ERR(get_file_offset(&rs->start, rs, pool)); + rs->header_size = rh->header_size; + + /* Log the access. */ + SVN_ERR(dbg_log_access(fs, SVN_INVALID_REVNUM, 0, rh, + SVN_FS_FS__ITEM_TYPE_ANY_REP, pool)); + + /* Build the representation list (delta chain). */ + if (rh->type == svn_fs_fs__rep_plain) + { + rb->rs_list = apr_array_make(pool, 0, sizeof(rep_state_t *)); + rb->src_state = rs; + } + else if (rh->type == svn_fs_fs__rep_self_delta) + { + rb->rs_list = apr_array_make(pool, 1, sizeof(rep_state_t *)); + APR_ARRAY_PUSH(rb->rs_list, rep_state_t *) = rs; + rb->src_state = NULL; + } + else + { + representation_t next_rep = { 0 }; + + /* skip "SVNx" diff marker */ + rs->current = 4; + + /* REP's base rep is inside a proper revision. + * It can be reconstructed in the usual way. */ + next_rep.revision = rh->base_revision; + next_rep.item_index = rh->base_item_index; + next_rep.size = rh->base_length; + svn_fs_fs__id_txn_reset(&next_rep.txn_id); + + SVN_ERR(build_rep_list(&rb->rs_list, &rb->base_window, + &rb->src_state, &rb->len, rb->fs, &next_rep, + rb->filehandle_pool)); + + /* Insert the access to REP as the first element of the delta chain. */ + svn_sort__array_insert(rb->rs_list, &rs, 0); + } + + /* Now, the baton is complete and we can assemble the stream around it. */ + *contents_p = svn_stream_create(rb, pool); + svn_stream_set_read2(*contents_p, NULL /* only full read support */, + rep_read_contents); + svn_stream_set_close(*contents_p, rep_read_contents_close); return SVN_NO_ERROR; } Modified: vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.h ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.h Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.h Thu Aug 10 21:48:34 2017 (r322377) @@ -81,6 +81,18 @@ svn_fs_fs__get_contents(svn_stream_t **contents_p, svn_boolean_t cache_fulltext, apr_pool_t *pool); +/* Set *CONTENTS_P to be a readable svn_stream_t that receives the text + representation REP as seen in filesystem FS. Read the latest element + of the delta chain from FILE at offset OFFSET. + Use POOL for allocations. */ +svn_error_t * +svn_fs_fs__get_contents_from_file(svn_stream_t **contents_p, + svn_fs_t *fs, + representation_t *rep, + apr_file_t *file, + apr_off_t offset, + apr_pool_t *pool); + /* Attempt to fetch the text representation of node-revision NODEREV as seen in filesystem FS and pass it along with the BATON to the PROCESSOR. Set *SUCCESS only of the data could be provided and the processing Modified: vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.h ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.h Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.h Thu Aug 10 21:48:34 2017 (r322377) @@ -1,4 +1,4 @@ -/* This file is automatically generated from rep-cache-db.sql and .dist_sandbox/subversion-1.9.5/subversion/libsvn_fs_fs/token-map.h. +/* This file is automatically generated from rep-cache-db.sql and .dist_sandbox/subversion-1.9.7/subversion/libsvn_fs_fs/token-map.h. * Do not edit this file -- edit the source and rerun gen-make.py */ #define STMT_CREATE_SCHEMA 0 Modified: vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.c Thu Aug 10 21:48:34 2017 (r322377) @@ -128,7 +128,10 @@ svn_fs_fs__open_rep_cache(svn_fs_t *fs, fs_fs_data_t *ffd = fs->fsap_data; svn_error_t *err = svn_atomic__init_once(&ffd->rep_cache_db_opened, open_rep_cache, fs, pool); - return svn_error_quick_wrap(err, _("Couldn't open rep-cache database")); + return svn_error_quick_wrapf(err, + _("Couldn't open rep-cache database '%s'"), + svn_dirent_local_style( + path_rep_cache_db(fs->path, pool), pool)); } svn_error_t * Modified: vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.c Thu Aug 10 21:48:34 2017 (r322377) @@ -2128,6 +2128,11 @@ rep_write_get_baton(struct rep_write_baton **wb_p, there may be new duplicate representations within the same uncommitted revision, those can be passed in REPS_HASH (maps a sha1 digest onto representation_t*), otherwise pass in NULL for REPS_HASH. + + The content of both representations will be compared, taking REP's content + from FILE at OFFSET. Only if they actually match, will *OLD_REP not be + NULL. + Use RESULT_POOL for *OLD_REP allocations and SCRATCH_POOL for temporaries. The lifetime of *OLD_REP is limited by both, RESULT_POOL and REP lifetime. */ @@ -2135,6 +2140,8 @@ static svn_error_t * get_shared_rep(representation_t **old_rep, svn_fs_t *fs, representation_t *rep, + apr_file_t *file, + apr_off_t offset, apr_hash_t *reps_hash, apr_pool_t *result_pool, apr_pool_t *scratch_pool) @@ -2142,6 +2149,10 @@ get_shared_rep(representation_t **old_rep, svn_error_t *err; fs_fs_data_t *ffd = fs->fsap_data; + svn_checksum_t checksum; + checksum.digest = rep->sha1_digest; + checksum.kind = svn_checksum_sha1; + /* Return NULL, if rep sharing has been disabled. */ *old_rep = NULL; if (!ffd->rep_sharing_allowed) @@ -2158,9 +2169,6 @@ get_shared_rep(representation_t **old_rep, /* If we haven't found anything yet, try harder and consult our DB. */ if (*old_rep == NULL) { - svn_checksum_t checksum; - checksum.digest = rep->sha1_digest; - checksum.kind = svn_checksum_sha1; err = svn_fs_fs__get_rep_reference(old_rep, fs, &checksum, result_pool); /* ### Other error codes that we shouldn't mask out? */ if (err == SVN_NO_ERROR) @@ -2235,6 +2243,72 @@ get_shared_rep(representation_t **old_rep, (*old_rep)->uniquifier = rep->uniquifier; } + /* If we (very likely) found a matching representation, compare the actual + * contents such that we can be sure that no rep-cache.db corruption or + * hash collision produced a false positive. */ + if (*old_rep) + { + apr_off_t old_position; + svn_stream_t *contents; + svn_stream_t *old_contents; + svn_boolean_t same; + + /* The existing representation may itself be part of the current + * transaction. In that case, it may be in different stages of + * the commit finalization process. + * + * OLD_REP_NORM is the same as that OLD_REP but it is assigned + * explicitly to REP's transaction if OLD_REP does not point + * to an already committed revision. This then prevents the + * revision lookup and the txn data will be accessed. + */ + representation_t old_rep_norm = **old_rep; + if ( !SVN_IS_VALID_REVNUM(old_rep_norm.revision) + || old_rep_norm.revision > ffd->youngest_rev_cache) + old_rep_norm.txn_id = rep->txn_id; + + /* Make sure we can later restore FILE's current position. */ + SVN_ERR(svn_fs_fs__get_file_offset(&old_position, file, scratch_pool)); + + /* Compare the two representations. + * Note that the stream comparison might also produce MD5 checksum + * errors or other failures in case of SHA1 collisions. */ + SVN_ERR(svn_fs_fs__get_contents_from_file(&contents, fs, rep, file, + offset, scratch_pool)); + SVN_ERR(svn_fs_fs__get_contents(&old_contents, fs, &old_rep_norm, + FALSE, scratch_pool)); + err = svn_stream_contents_same2(&same, contents, old_contents, + scratch_pool); + + /* A mismatch should be extremely rare. + * If it does happen, reject the commit. */ + if (!same || err) + { + /* SHA1 collision or worse. */ + svn_stringbuf_t *old_rep_str + = svn_fs_fs__unparse_representation(*old_rep, + ffd->format, FALSE, + scratch_pool, + scratch_pool); + svn_stringbuf_t *rep_str + = svn_fs_fs__unparse_representation(rep, + ffd->format, FALSE, + scratch_pool, + scratch_pool); + const char *checksum__str + = svn_checksum_to_cstring_display(&checksum, scratch_pool); + + return svn_error_createf(SVN_ERR_FS_GENERAL, + err, "SHA1 of reps '%s' and '%s' " + "matches (%s) but contents differ", + old_rep_str->data, rep_str->data, + checksum__str); + } + + /* Restore FILE's read / write position. */ + SVN_ERR(svn_io_file_seek(file, APR_SET, &old_position, scratch_pool)); + } + return SVN_NO_ERROR; } @@ -2293,8 +2367,8 @@ rep_write_contents_close(void *baton) /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ - SVN_ERR(get_shared_rep(&old_rep, b->fs, rep, NULL, b->result_pool, - b->scratch_pool)); + SVN_ERR(get_shared_rep(&old_rep, b->fs, rep, b->file, b->rep_offset, NULL, + b->result_pool, b->scratch_pool)); if (old_rep) { @@ -2519,11 +2593,16 @@ write_directory_to_stream(svn_stream_t *stream, /* Write out the COLLECTION as a text representation to file FILE using WRITER. In the process, record position, the total size of the dump and MD5 as well as SHA1 in REP. Add the representation of type ITEM_TYPE to - the indexes if necessary. If rep sharing has been enabled and REPS_HASH - is not NULL, it will be used in addition to the on-disk cache to find - earlier reps with the same content. When such existing reps can be - found, we will truncate the one just written from the file and return - the existing rep. Perform temporary allocations in SCRATCH_POOL. */ + the indexes if necessary. + + If ALLOW_REP_SHARING is FALSE, rep-sharing will not be used, regardless + of any other option and rep-sharing settings. If rep sharing has been + enabled and REPS_HASH is not NULL, it will be used in addition to the + on-disk cache to find earlier reps with the same content. If such + existing reps can be found, we will truncate the one just written from + the file and return the existing rep. + + Perform temporary allocations in SCRATCH_POOL. */ static svn_error_t * write_container_rep(representation_t *rep, apr_file_t *file, @@ -2531,14 +2610,15 @@ write_container_rep(representation_t *rep, collection_writer_t writer, svn_fs_t *fs, apr_hash_t *reps_hash, + svn_boolean_t allow_rep_sharing, apr_uint32_t item_type, apr_pool_t *scratch_pool) { svn_stream_t *stream; struct write_container_baton *whb; svn_checksum_ctx_t *fnv1a_checksum_ctx; - representation_t *old_rep; apr_off_t offset = 0; + svn_fs_fs__p2l_entry_t entry; SVN_ERR(svn_fs_fs__get_file_offset(&offset, file, scratch_pool)); @@ -2562,58 +2642,62 @@ write_container_rep(representation_t *rep, /* Store the results. */ SVN_ERR(digests_final(rep, whb->md5_ctx, whb->sha1_ctx, scratch_pool)); + /* Update size info. */ + rep->expanded_size = whb->size; + rep->size = whb->size; + /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ - SVN_ERR(get_shared_rep(&old_rep, fs, rep, reps_hash, scratch_pool, - scratch_pool)); - - if (old_rep) + if (allow_rep_sharing) { - /* We need to erase from the protorev the data we just wrote. */ - SVN_ERR(svn_io_file_trunc(file, offset, scratch_pool)); + representation_t *old_rep; + SVN_ERR(get_shared_rep(&old_rep, fs, rep, file, offset, reps_hash, + scratch_pool, scratch_pool)); - /* Use the old rep for this content. */ - memcpy(rep, old_rep, sizeof (*rep)); + if (old_rep) + { + /* We need to erase from the protorev the data we just wrote. */ + SVN_ERR(svn_io_file_trunc(file, offset, scratch_pool)); + + /* Use the old rep for this content. */ + memcpy(rep, old_rep, sizeof (*rep)); + return SVN_NO_ERROR; + } } - else - { - svn_fs_fs__p2l_entry_t entry; - /* Write out our cosmetic end marker. */ - SVN_ERR(svn_stream_puts(whb->stream, "ENDREP\n")); + /* Write out our cosmetic end marker. */ + SVN_ERR(svn_stream_puts(whb->stream, "ENDREP\n")); - SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id, - offset, scratch_pool)); + SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id, + offset, scratch_pool)); - entry.offset = offset; - SVN_ERR(svn_fs_fs__get_file_offset(&offset, file, scratch_pool)); - entry.size = offset - entry.offset; - entry.type = item_type; - entry.item.revision = SVN_INVALID_REVNUM; - entry.item.number = rep->item_index; - SVN_ERR(fnv1a_checksum_finalize(&entry.fnv1_checksum, - fnv1a_checksum_ctx, - scratch_pool)); + entry.offset = offset; + SVN_ERR(svn_fs_fs__get_file_offset(&offset, file, scratch_pool)); + entry.size = offset - entry.offset; + entry.type = item_type; + entry.item.revision = SVN_INVALID_REVNUM; + entry.item.number = rep->item_index; + SVN_ERR(fnv1a_checksum_finalize(&entry.fnv1_checksum, + fnv1a_checksum_ctx, + scratch_pool)); - SVN_ERR(store_p2l_index_entry(fs, &rep->txn_id, &entry, scratch_pool)); + SVN_ERR(store_p2l_index_entry(fs, &rep->txn_id, &entry, scratch_pool)); - /* update the representation */ - rep->size = whb->size; - rep->expanded_size = whb->size; - } - return SVN_NO_ERROR; } /* Write out the COLLECTION pertaining to the NODEREV in FS as a deltified text representation to file FILE using WRITER. In the process, record the total size and the md5 digest in REP and add the representation of type - ITEM_TYPE to the indexes if necessary. If rep sharing has been enabled and - REPS_HASH is not NULL, it will be used in addition to the on-disk cache to - find earlier reps with the same content. When such existing reps can be - found, we will truncate the one just written from the file and return the - existing rep. + ITEM_TYPE to the indexes if necessary. + If ALLOW_REP_SHARING is FALSE, rep-sharing will not be used, regardless + of any other option and rep-sharing settings. If rep sharing has been + enabled and REPS_HASH is not NULL, it will be used in addition to the + on-disk cache to find earlier reps with the same content. If such + existing reps can be found, we will truncate the one just written from + the file and return the existing rep. + If ITEM_TYPE is IS_PROPS equals SVN_FS_FS__ITEM_TYPE_*_PROPS, assume that we want to a props representation as the base for our delta. Perform temporary allocations in SCRATCH_POOL. @@ -2626,6 +2710,7 @@ write_container_delta_rep(representation_t *rep, svn_fs_t *fs, node_revision_t *noderev, apr_hash_t *reps_hash, + svn_boolean_t allow_rep_sharing, apr_uint32_t item_type, apr_pool_t *scratch_pool) { @@ -2635,10 +2720,10 @@ write_container_delta_rep(representation_t *rep, svn_stream_t *file_stream; svn_stream_t *stream; representation_t *base_rep; - representation_t *old_rep; svn_checksum_ctx_t *fnv1a_checksum_ctx; svn_stream_t *source; svn_fs_fs__rep_header_t header = { 0 }; + svn_fs_fs__p2l_entry_t entry; apr_off_t rep_end = 0; apr_off_t delta_start = 0; @@ -2701,47 +2786,48 @@ write_container_delta_rep(representation_t *rep, /* Store the results. */ SVN_ERR(digests_final(rep, whb->md5_ctx, whb->sha1_ctx, scratch_pool)); + /* Update size info. */ + SVN_ERR(svn_fs_fs__get_file_offset(&rep_end, file, scratch_pool)); + rep->size = rep_end - delta_start; + rep->expanded_size = whb->size; + /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ - SVN_ERR(get_shared_rep(&old_rep, fs, rep, reps_hash, scratch_pool, - scratch_pool)); - - if (old_rep) + if (allow_rep_sharing) { - /* We need to erase from the protorev the data we just wrote. */ - SVN_ERR(svn_io_file_trunc(file, offset, scratch_pool)); + representation_t *old_rep; + SVN_ERR(get_shared_rep(&old_rep, fs, rep, file, offset, reps_hash, + scratch_pool, scratch_pool)); - /* Use the old rep for this content. */ - memcpy(rep, old_rep, sizeof (*rep)); + if (old_rep) + { + /* We need to erase from the protorev the data we just wrote. */ + SVN_ERR(svn_io_file_trunc(file, offset, scratch_pool)); + + /* Use the old rep for this content. */ + memcpy(rep, old_rep, sizeof (*rep)); + return SVN_NO_ERROR; + } } - else - { - svn_fs_fs__p2l_entry_t entry; - /* Write out our cosmetic end marker. */ - SVN_ERR(svn_fs_fs__get_file_offset(&rep_end, file, scratch_pool)); - SVN_ERR(svn_stream_puts(file_stream, "ENDREP\n")); + /* Write out our cosmetic end marker. */ + SVN_ERR(svn_stream_puts(file_stream, "ENDREP\n")); - SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id, - offset, scratch_pool)); + SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id, + offset, scratch_pool)); - entry.offset = offset; - SVN_ERR(svn_fs_fs__get_file_offset(&offset, file, scratch_pool)); - entry.size = offset - entry.offset; - entry.type = item_type; - entry.item.revision = SVN_INVALID_REVNUM; - entry.item.number = rep->item_index; - SVN_ERR(fnv1a_checksum_finalize(&entry.fnv1_checksum, - fnv1a_checksum_ctx, - scratch_pool)); + entry.offset = offset; + SVN_ERR(svn_fs_fs__get_file_offset(&offset, file, scratch_pool)); + entry.size = offset - entry.offset; + entry.type = item_type; + entry.item.revision = SVN_INVALID_REVNUM; + entry.item.number = rep->item_index; + SVN_ERR(fnv1a_checksum_finalize(&entry.fnv1_checksum, + fnv1a_checksum_ctx, + scratch_pool)); - SVN_ERR(store_p2l_index_entry(fs, &rep->txn_id, &entry, scratch_pool)); + SVN_ERR(store_p2l_index_entry(fs, &rep->txn_id, &entry, scratch_pool)); - /* update the representation */ - rep->expanded_size = whb->size; - rep->size = rep_end - delta_start; - } - return SVN_NO_ERROR; } @@ -2920,13 +3006,14 @@ write_final_rev(const svn_fs_id_t **new_id_p, SVN_ERR(write_container_delta_rep(noderev->data_rep, file, entries, write_directory_to_stream, - fs, noderev, NULL, + fs, noderev, NULL, FALSE, SVN_FS_FS__ITEM_TYPE_DIR_REP, pool)); else SVN_ERR(write_container_rep(noderev->data_rep, file, entries, write_directory_to_stream, fs, NULL, - SVN_FS_FS__ITEM_TYPE_DIR_REP, pool)); + FALSE, SVN_FS_FS__ITEM_TYPE_DIR_REP, + pool)); reset_txn_in_rep(noderev->data_rep); } @@ -2971,11 +3058,11 @@ write_final_rev(const svn_fs_id_t **new_id_p, if (ffd->deltify_properties) SVN_ERR(write_container_delta_rep(noderev->prop_rep, file, proplist, write_hash_to_stream, fs, noderev, - reps_hash, item_type, pool)); + reps_hash, TRUE, item_type, pool)); else SVN_ERR(write_container_rep(noderev->prop_rep, file, proplist, write_hash_to_stream, fs, reps_hash, - item_type, pool)); + TRUE, item_type, pool)); reset_txn_in_rep(noderev->prop_rep); } Modified: vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.h ============================================================================== --- vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.h Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.h Thu Aug 10 21:48:34 2017 (r322377) @@ -1,4 +1,4 @@ -/* This file is automatically generated from rep-cache-db.sql and .dist_sandbox/subversion-1.9.5/subversion/libsvn_fs_x/token-map.h. +/* This file is automatically generated from rep-cache-db.sql and .dist_sandbox/subversion-1.9.7/subversion/libsvn_fs_x/token-map.h. * Do not edit this file -- edit the source and rerun gen-make.py */ #define STMT_CREATE_SCHEMA 0 Modified: vendor/subversion/dist/subversion/libsvn_ra_svn/client.c ============================================================================== --- vendor/subversion/dist/subversion/libsvn_ra_svn/client.c Thu Aug 10 21:39:22 2017 (r322376) +++ vendor/subversion/dist/subversion/libsvn_ra_svn/client.c Thu Aug 10 21:48:34 2017 (r322377) @@ -46,6 +46,7 @@ #include "svn_props.h" #include "svn_mergeinfo.h" #include "svn_version.h" +#include "svn_ctype.h" #include "svn_private_config.h" @@ -396,7 +397,7 @@ static svn_error_t *find_tunnel_agent(const char *tunn * versions have it too. If the user is using some other ssh * implementation that doesn't accept it, they can override it * in the [tunnels] section of the config. */ - val = "$SVN_SSH ssh -q"; + val = "$SVN_SSH ssh -q --"; } if (!val || !*val) @@ -441,7 +442,7 @@ static svn_error_t *find_tunnel_agent(const char *tunn for (n = 0; cmd_argv[n] != NULL; n++) argv[n] = cmd_argv[n]; - argv[n++] = svn_path_uri_decode(hostinfo, pool); + argv[n++] = hostinfo; argv[n++] = "svnserve"; argv[n++] = "-t"; argv[n] = NULL; @@ -802,7 +803,33 @@ ra_svn_get_schemes(apr_pool_t *pool) } +/* A simple whitelist to ensure the following are valid: + * user@server + * [::1]:22 + * server-name + * server_name + * 127.0.0.1 + * with an extra restriction that a leading '-' is invalid. + */ +static svn_boolean_t +is_valid_hostinfo(const char *hostinfo) +{ + const char *p = hostinfo; + if (p[0] == '-') + return FALSE; + + while (*p) + { + if (!svn_ctype_isalnum(*p) && !strchr(":.-_[]@", *p)) + return FALSE; + + ++p; + } + + return TRUE; +} + static svn_error_t *ra_svn_open(svn_ra_session_t *session, const char **corrected_url, const char *url, @@ -835,8 +862,18 @@ static svn_error_t *ra_svn_open(svn_ra_session_t *sess || (callbacks->check_tunnel_func && callbacks->open_tunnel_func && !callbacks->check_tunnel_func(callbacks->tunnel_baton, tunnel)))) - SVN_ERR(find_tunnel_agent(tunnel, uri.hostinfo, &tunnel_argv, config, *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Thu Aug 10 22:04:56 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 467E6DDDFB3; Thu, 10 Aug 2017 22:04:56 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED1A67345F; Thu, 10 Aug 2017 22:04:55 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7AM4t1Y022172; Thu, 10 Aug 2017 22:04:55 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7AM4tgg022171; Thu, 10 Aug 2017 22:04:55 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201708102204.v7AM4tgg022171@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Thu, 10 Aug 2017 22:04:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322381 - vendor/subversion/subversion-1.9.7 X-SVN-Group: vendor X-SVN-Commit-Author: peter X-SVN-Commit-Paths: vendor/subversion/subversion-1.9.7 X-SVN-Commit-Revision: 322381 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 22:04:56 -0000 Author: peter Date: Thu Aug 10 22:04:54 2017 New Revision: 322381 URL: https://svnweb.freebsd.org/changeset/base/322381 Log: Tag subversion-1.9.7 import. Added: vendor/subversion/subversion-1.9.7/ - copied from r322380, vendor/subversion/dist/ From owner-svn-src-vendor@freebsd.org Thu Aug 10 22:15:44 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD40BC0827B; Thu, 10 Aug 2017 22:15:44 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4383473A03; Thu, 10 Aug 2017 22:15:44 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7AMFhJc026252; Thu, 10 Aug 2017 22:15:43 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7AMFgDw026247; Thu, 10 Aug 2017 22:15:42 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201708102215.v7AMFgDw026247@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Thu, 10 Aug 2017 22:15:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322382 - in vendor/sqlite3/dist: . tea tea/generic X-SVN-Group: vendor X-SVN-Commit-Author: peter X-SVN-Commit-Paths: in vendor/sqlite3/dist: . tea tea/generic X-SVN-Commit-Revision: 322382 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 22:15:44 -0000 Author: peter Date: Thu Aug 10 22:15:42 2017 New Revision: 322382 URL: https://svnweb.freebsd.org/changeset/base/322382 Log: Import sqlite3-3.20.0 Modified: vendor/sqlite3/dist/Makefile.msc vendor/sqlite3/dist/configure vendor/sqlite3/dist/configure.ac vendor/sqlite3/dist/shell.c vendor/sqlite3/dist/sqlite3.c vendor/sqlite3/dist/sqlite3.h vendor/sqlite3/dist/sqlite3ext.h vendor/sqlite3/dist/tea/configure vendor/sqlite3/dist/tea/configure.ac vendor/sqlite3/dist/tea/generic/tclsqlite3.c Modified: vendor/sqlite3/dist/Makefile.msc ============================================================================== --- vendor/sqlite3/dist/Makefile.msc Thu Aug 10 22:04:54 2017 (r322381) +++ vendor/sqlite3/dist/Makefile.msc Thu Aug 10 22:15:42 2017 (r322382) @@ -21,9 +21,16 @@ TOP = . # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN -USE_FULLWARN = 0 +USE_FULLWARN = 1 !ENDIF +# Set this non-0 to enable treating warnings as errors (-WX, etc) when +# compiling. +# +!IFNDEF USE_FATAL_WARN +USE_FATAL_WARN = 0 +!ENDIF + # Set this non-0 to enable full runtime error checks (-RTC1, etc). This # has no effect if (any) optimizations are enabled. # @@ -31,6 +38,13 @@ USE_FULLWARN = 0 USE_RUNTIME_CHECKS = 0 !ENDIF +# Set this non-0 to create a SQLite amalgamation file that excludes the +# various built-in extensions. +# +!IFNDEF MINIMAL_AMALGAMATION +MINIMAL_AMALGAMATION = 0 +!ENDIF + # Set this non-0 to use "stdcall" calling convention for the core library # and shell executable. # @@ -255,12 +269,15 @@ SQLITE3EXEPDB = /pdb:sqlite3sh.pdb !ENDIF !ENDIF + # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS +!IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 +!ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF @@ -444,6 +461,12 @@ TCC = $(CC) -nologo -W4 -DINCLUDE_MSVC_H=1 $(CCOPTS) $ TCC = $(CC) -nologo -W3 $(CCOPTS) $(TCCOPTS) !ENDIF +# Check if warnings should be treated as errors when compiling. +# +!IF $(USE_FATAL_WARN)!=0 +TCC = $(TCC) -WX +!ENDIF + TCC = $(TCC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -fp:precise RCC = $(RC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) $(RCOPTS) $(RCCOPTS) @@ -622,7 +645,11 @@ RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !IF $(DEBUG)>2 TCC = $(TCC) -DSQLITE_DEBUG=1 RCC = $(RCC) -DSQLITE_DEBUG=1 +!IF $(DYNAMIC_SHELL)==0 +TCC = $(TCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE +RCC = $(RCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE !ENDIF +!ENDIF !IF $(DEBUG)>4 || $(OSTRACE)!=0 TCC = $(TCC) -DSQLITE_FORCE_OS_TRACE=1 -DSQLITE_DEBUG_OS_TRACE=1 @@ -900,7 +927,7 @@ LIBRESOBJS = # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 -SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_SHELL_JSON1 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_EXPLAIN_COMMENTS +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_SHELL_JSON1 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_STMTVTAB !ENDIF @@ -927,7 +954,7 @@ Replace.exe: sqlite3.def: Replace.exe $(LIBOBJ) echo EXPORTS > sqlite3.def dumpbin /all $(LIBOBJ) \ - | .\Replace.exe "^\s+/EXPORT:_?(sqlite3_[^@,]*)(?:@\d+|,DATA)?$$" $$1 true \ + | .\Replace.exe "^\s+/EXPORT:_?(sqlite3(?:session|changeset|changegroup)?_[^@,]*)(?:@\d+|,DATA)?$$" $$1 true \ | sort >> sqlite3.def $(SQLITE3EXE): $(TOP)\shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) Modified: vendor/sqlite3/dist/configure ============================================================================== --- vendor/sqlite3/dist/configure Thu Aug 10 22:04:54 2017 (r322381) +++ vendor/sqlite3/dist/configure Thu Aug 10 22:15:42 2017 (r322382) @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for sqlite 3.14.1. +# Generated by GNU Autoconf 2.69 for sqlite 3.20.0. # # Report bugs to . # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='sqlite' PACKAGE_TARNAME='sqlite' -PACKAGE_VERSION='3.14.1' -PACKAGE_STRING='sqlite 3.14.1' +PACKAGE_VERSION='3.20.0' +PACKAGE_STRING='sqlite 3.20.0' PACKAGE_BUGREPORT='http://www.sqlite.org' PACKAGE_URL='' @@ -1330,7 +1330,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures sqlite 3.14.1 to adapt to many kinds of systems. +\`configure' configures sqlite 3.20.0 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1400,7 +1400,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of sqlite 3.14.1:";; + short | recursive ) echo "Configuration of sqlite 3.20.0:";; esac cat <<\_ACEOF @@ -1521,7 +1521,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -sqlite configure 3.14.1 +sqlite configure 3.20.0 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -1936,7 +1936,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by sqlite $as_me 3.14.1, which was +It was created by sqlite $as_me 3.20.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2802,7 +2802,7 @@ fi # Define the identity of the package. PACKAGE='sqlite' - VERSION='3.14.1' + VERSION='3.20.0' cat >>confdefs.h <<_ACEOF @@ -13108,7 +13108,7 @@ for ac_lib in '' edit; do ac_res="none required" else ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" + LIBS="-l$ac_lib -ltinfo $ac_func_search_save_LIBS" fi if ac_fn_c_try_link "$LINENO"; then : ac_cv_search_readline=$ac_res @@ -13136,7 +13136,7 @@ if test "$ac_res" != no; then : $as_echo "#define HAVE_EDITLINE 1" >>confdefs.h - READLINE_LIBS=$LIBS + READLINE_LIBS="$LIBS -ltinfo" enable_readline=no fi @@ -14227,7 +14227,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by sqlite $as_me 3.14.1, which was +This file was extended by sqlite $as_me 3.20.0, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -14284,7 +14284,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -sqlite config.status 3.14.1 +sqlite config.status 3.20.0 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Modified: vendor/sqlite3/dist/configure.ac ============================================================================== --- vendor/sqlite3/dist/configure.ac Thu Aug 10 22:04:54 2017 (r322381) +++ vendor/sqlite3/dist/configure.ac Thu Aug 10 22:15:42 2017 (r322382) @@ -10,7 +10,7 @@ # AC_PREREQ(2.61) -AC_INIT(sqlite, 3.14.1, http://www.sqlite.org) +AC_INIT(sqlite, 3.20.0, http://www.sqlite.org) AC_CONFIG_SRCDIR([sqlite3.c]) # Use automake. @@ -55,9 +55,9 @@ AS_IF([ test x"$enable_editline" != xno ],[ LIBS="" AC_SEARCH_LIBS([readline],[edit],[ AC_DEFINE([HAVE_EDITLINE],1,Define to use BSD editline) - READLINE_LIBS=$LIBS + READLINE_LIBS="$LIBS -ltinfo" enable_readline=no - ]) + ],[],[-ltinfo]) AS_UNSET(ac_cv_search_readline) LIBS=$sLIBS ]) Modified: vendor/sqlite3/dist/shell.c ============================================================================== --- vendor/sqlite3/dist/shell.c Thu Aug 10 22:04:54 2017 (r322381) +++ vendor/sqlite3/dist/shell.c Thu Aug 10 22:15:42 2017 (r322382) @@ -1,3 +1,21 @@ +/* DO NOT EDIT! +** This file is automatically generated by the script in the canonical +** SQLite source tree at tool/mkshellc.tcl. That script combines source +** code from various constituent source files of SQLite into this single +** "shell.c" file used to implement the SQLite command-line shell. +** +** Most of the code found below comes from the "src/shell.c.in" file in +** the canonical SQLite source tree. That main file contains "INCLUDE" +** lines that specify other files in the canonical source tree that are +** inserted to getnerate this complete program source file. +** +** The code from multiple files is combined into this single "shell.c" +** source file to help make the command-line program easier to compile. +** +** To modify this program, get a copy of the canonical SQLite source tree, +** edit the src/shell.c.in" and/or some of the other files that are included +** by "src/shell.c.in", then rerun the tool/mkshellc.tcl script. +*/ /* ** 2001 September 15 ** @@ -18,11 +36,25 @@ #endif /* -** If requested, include the SQLite compiler options file for MSVC. +** Warning pragmas copied from msvc.h in the core. */ -#if defined(INCLUDE_MSVC_H) -#include "msvc.h" -#endif +#if defined(_MSC_VER) +#pragma warning(disable : 4054) +#pragma warning(disable : 4055) +#pragma warning(disable : 4100) +#pragma warning(disable : 4127) +#pragma warning(disable : 4130) +#pragma warning(disable : 4152) +#pragma warning(disable : 4189) +#pragma warning(disable : 4206) +#pragma warning(disable : 4210) +#pragma warning(disable : 4232) +#pragma warning(disable : 4244) +#pragma warning(disable : 4305) +#pragma warning(disable : 4306) +#pragma warning(disable : 4702) +#pragma warning(disable : 4706) +#endif /* defined(_MSC_VER) */ /* ** No support for loadable extensions in VxWorks. @@ -143,6 +175,7 @@ extern char *sqlite3_win32_unicode_to_utf8(LPCWSTR); extern char *sqlite3_win32_mbcs_to_utf8_v2(const char *, int); extern char *sqlite3_win32_utf8_to_mbcs_v2(const char *, int); +extern LPWSTR sqlite3_win32_utf8_to_unicode(const char *zText); #endif /* On Windows, we normally run with output mode of TEXT so that \n characters @@ -426,7 +459,37 @@ static void SQLITE_CDECL iotracePrintf(const char *zFo } #endif +/* +** Output string zUtf to stream pOut as w characters. If w is negative, +** then right-justify the text. W is the width in UTF-8 characters, not +** in bytes. This is different from the %*.*s specification in printf +** since with %*.*s the width is measured in bytes, not characters. +*/ +static void utf8_width_print(FILE *pOut, int w, const char *zUtf){ + int i; + int n; + int aw = w<0 ? -w : w; + char zBuf[1000]; + if( aw>(int)sizeof(zBuf)/3 ) aw = (int)sizeof(zBuf)/3; + for(i=n=0; zUtf[i]; i++){ + if( (zUtf[i]&0xc0)!=0x80 ){ + n++; + if( n==aw ){ + do{ i++; }while( (zUtf[i]&0xc0)==0x80 ); + break; + } + } + } + if( n>=aw ){ + utf8_printf(pOut, "%.*s", i, zUtf); + }else if( w<0 ){ + utf8_printf(pOut, "%*s%s", aw-n, "", zUtf); + }else{ + utf8_printf(pOut, "%s%*s", zUtf, aw-n, ""); + } +} + /* ** Determines if a string is a number of not. */ @@ -455,28 +518,6 @@ static int isNumber(const char *z, int *realnum){ } /* -** A global char* and an SQL function to access its current value -** from within an SQL statement. This program used to use the -** sqlite_exec_printf() API to substitue a string into an SQL statement. -** The correct way to do this with sqlite3 is to use the bind API, but -** since the shell is built around the callback paradigm it would be a lot -** of work. Instead just use this hack, which is quite harmless. -*/ -static const char *zShellStatic = 0; -static void shellstaticFunc( - sqlite3_context *context, - int argc, - sqlite3_value **argv -){ - assert( 0==argc ); - assert( zShellStatic ); - UNUSED_PARAMETER(argc); - UNUSED_PARAMETER(argv); - sqlite3_result_text(context, zShellStatic, -1, SQLITE_STATIC); -} - - -/* ** Compute a string length that is limited to what can be stored in ** lower 30 bits of a 32-bit signed integer. */ @@ -487,6 +528,18 @@ static int strlen30(const char *z){ } /* +** Return the length of a string in characters. Multibyte UTF8 characters +** count as a single character. +*/ +static int strlenChar(const char *z){ + int n = 0; + while( *z ){ + if( (0xc0&*(z++))!=0x80 ) n++; + } + return n; +} + +/* ** This routine reads a line of text from FILE in, stores ** the text in memory obtained from malloc() and returns a pointer ** to the text. NULL is returned at end of file, or if malloc() @@ -524,7 +577,7 @@ static char *local_getline(char *zLine, FILE *in){ #if defined(_WIN32) || defined(WIN32) /* For interactive input on Windows systems, translate the ** multi-byte characterset characters into UTF-8. */ - if( stdin_is_interactive ){ + if( stdin_is_interactive && in==stdin ){ char *zTrans = sqlite3_win32_mbcs_to_utf8_v2(zLine, 0); if( zTrans ){ int nTrans = strlen30(zTrans)+1; @@ -576,7 +629,1534 @@ static char *one_input_line(FILE *in, char *zPrior, in } return zResult; } +/* +** A variable length string to which one can append text. +*/ +typedef struct ShellText ShellText; +struct ShellText { + char *z; + int n; + int nAlloc; +}; +/* +** Initialize and destroy a ShellText object +*/ +static void initText(ShellText *p){ + memset(p, 0, sizeof(*p)); +} +static void freeText(ShellText *p){ + free(p->z); + initText(p); +} + +/* zIn is either a pointer to a NULL-terminated string in memory obtained +** from malloc(), or a NULL pointer. The string pointed to by zAppend is +** added to zIn, and the result returned in memory obtained from malloc(). +** zIn, if it was not NULL, is freed. +** +** If the third argument, quote, is not '\0', then it is used as a +** quote character for zAppend. +*/ +static void appendText(ShellText *p, char const *zAppend, char quote){ + int len; + int i; + int nAppend = strlen30(zAppend); + + len = nAppend+p->n+1; + if( quote ){ + len += 2; + for(i=0; in+len>=p->nAlloc ){ + p->nAlloc = p->nAlloc*2 + len + 20; + p->z = realloc(p->z, p->nAlloc); + if( p->z==0 ){ + memset(p, 0, sizeof(*p)); + return; + } + } + + if( quote ){ + char *zCsr = p->z+p->n; + *zCsr++ = quote; + for(i=0; in = (int)(zCsr - p->z); + *zCsr = '\0'; + }else{ + memcpy(p->z+p->n, zAppend, nAppend); + p->n += nAppend; + p->z[p->n] = '\0'; + } +} + +/* +** Attempt to determine if identifier zName needs to be quoted, either +** because it contains non-alphanumeric characters, or because it is an +** SQLite keyword. Be conservative in this estimate: When in doubt assume +** that quoting is required. +** +** Return '"' if quoting is required. Return 0 if no quoting is required. +*/ +static char quoteChar(const char *zName){ + /* All SQLite keywords, in alphabetical order */ + static const char *azKeywords[] = { + "ABORT", "ACTION", "ADD", "AFTER", "ALL", "ALTER", "ANALYZE", "AND", "AS", + "ASC", "ATTACH", "AUTOINCREMENT", "BEFORE", "BEGIN", "BETWEEN", "BY", + "CASCADE", "CASE", "CAST", "CHECK", "COLLATE", "COLUMN", "COMMIT", + "CONFLICT", "CONSTRAINT", "CREATE", "CROSS", "CURRENT_DATE", + "CURRENT_TIME", "CURRENT_TIMESTAMP", "DATABASE", "DEFAULT", "DEFERRABLE", + "DEFERRED", "DELETE", "DESC", "DETACH", "DISTINCT", "DROP", "EACH", + "ELSE", "END", "ESCAPE", "EXCEPT", "EXCLUSIVE", "EXISTS", "EXPLAIN", + "FAIL", "FOR", "FOREIGN", "FROM", "FULL", "GLOB", "GROUP", "HAVING", "IF", + "IGNORE", "IMMEDIATE", "IN", "INDEX", "INDEXED", "INITIALLY", "INNER", + "INSERT", "INSTEAD", "INTERSECT", "INTO", "IS", "ISNULL", "JOIN", "KEY", + "LEFT", "LIKE", "LIMIT", "MATCH", "NATURAL", "NO", "NOT", "NOTNULL", + "NULL", "OF", "OFFSET", "ON", "OR", "ORDER", "OUTER", "PLAN", "PRAGMA", + "PRIMARY", "QUERY", "RAISE", "RECURSIVE", "REFERENCES", "REGEXP", + "REINDEX", "RELEASE", "RENAME", "REPLACE", "RESTRICT", "RIGHT", + "ROLLBACK", "ROW", "SAVEPOINT", "SELECT", "SET", "TABLE", "TEMP", + "TEMPORARY", "THEN", "TO", "TRANSACTION", "TRIGGER", "UNION", "UNIQUE", + "UPDATE", "USING", "VACUUM", "VALUES", "VIEW", "VIRTUAL", "WHEN", "WHERE", + "WITH", "WITHOUT", + }; + int i, lwr, upr, mid, c; + if( !isalpha((unsigned char)zName[0]) && zName[0]!='_' ) return '"'; + for(i=0; zName[i]; i++){ + if( !isalnum((unsigned char)zName[i]) && zName[i]!='_' ) return '"'; + } + lwr = 0; + upr = sizeof(azKeywords)/sizeof(azKeywords[0]) - 1; + while( lwr<=upr ){ + mid = (lwr+upr)/2; + c = sqlite3_stricmp(azKeywords[mid], zName); + if( c==0 ) return '"'; + if( c<0 ){ + lwr = mid+1; + }else{ + upr = mid-1; + } + } + return 0; +} + +/* +** SQL function: shell_add_schema(S,X) +** +** Add the schema name X to the CREATE statement in S and return the result. +** Examples: +** +** CREATE TABLE t1(x) -> CREATE TABLE xyz.t1(x); +** +** Also works on +** +** CREATE INDEX +** CREATE UNIQUE INDEX +** CREATE VIEW +** CREATE TRIGGER +** CREATE VIRTUAL TABLE +** +** This UDF is used by the .schema command to insert the schema name of +** attached databases into the middle of the sqlite_master.sql field. +*/ +static void shellAddSchemaName( + sqlite3_context *pCtx, + int nVal, + sqlite3_value **apVal +){ + static const char *aPrefix[] = { + "TABLE", + "INDEX", + "UNIQUE INDEX", + "VIEW", + "TRIGGER", + "VIRTUAL TABLE" + }; + int i = 0; + const char *zIn = (const char*)sqlite3_value_text(apVal[0]); + const char *zSchema = (const char*)sqlite3_value_text(apVal[1]); + assert( nVal==2 ); + if( zIn!=0 && strncmp(zIn, "CREATE ", 7)==0 ){ + for(i=0; i<(int)(sizeof(aPrefix)/sizeof(aPrefix[0])); i++){ + int n = strlen30(aPrefix[i]); + if( strncmp(zIn+7, aPrefix[i], n)==0 && zIn[n+7]==' ' ){ + char cQuote = quoteChar(zSchema); + char *z; + if( cQuote ){ + z = sqlite3_mprintf("%.*s \"%w\".%s", n+7, zIn, zSchema, zIn+n+8); + }else{ + z = sqlite3_mprintf("%.*s %s.%s", n+7, zIn, zSchema, zIn+n+8); + } + sqlite3_result_text(pCtx, z, -1, sqlite3_free); + return; + } + } + } + sqlite3_result_value(pCtx, apVal[0]); +} + +/* +** The source code for several run-time loadable extensions is inserted +** below by the ../tool/mkshellc.tcl script. Before processing that included +** code, we need to override some macros to make the included program code +** work here in the middle of this regular program. +*/ +#define SQLITE_EXTENSION_INIT1 +#define SQLITE_EXTENSION_INIT2(X) (void)(X) + +/************************* Begin ../ext/misc/shathree.c ******************/ +/* +** 2017-03-08 +** +** The author disclaims copyright to this source code. In place of +** a legal notice, here is a blessing: +** +** May you do good and not evil. +** May you find forgiveness for yourself and forgive others. +** May you share freely, never taking more than you give. +** +****************************************************************************** +** +** This SQLite extension implements a functions that compute SHA1 hashes. +** Two SQL functions are implemented: +** +** sha3(X,SIZE) +** sha3_query(Y,SIZE) +** +** The sha3(X) function computes the SHA3 hash of the input X, or NULL if +** X is NULL. +** +** The sha3_query(Y) function evalutes all queries in the SQL statements of Y +** and returns a hash of their results. +** +** The SIZE argument is optional. If omitted, the SHA3-256 hash algorithm +** is used. If SIZE is included it must be one of the integers 224, 256, +** 384, or 512, to determine SHA3 hash variant that is computed. +*/ +SQLITE_EXTENSION_INIT1 +#include +#include +#include +typedef sqlite3_uint64 u64; + +/****************************************************************************** +** The Hash Engine +*/ +/* +** Macros to determine whether the machine is big or little endian, +** and whether or not that determination is run-time or compile-time. +** +** For best performance, an attempt is made to guess at the byte-order +** using C-preprocessor macros. If that is unsuccessful, or if +** -DSHA3_BYTEORDER=0 is set, then byte-order is determined +** at run-time. +*/ +#ifndef SHA3_BYTEORDER +# if defined(i386) || defined(__i386__) || defined(_M_IX86) || \ + defined(__x86_64) || defined(__x86_64__) || defined(_M_X64) || \ + defined(_M_AMD64) || defined(_M_ARM) || defined(__x86) || \ + defined(__arm__) +# define SHA3_BYTEORDER 1234 +# elif defined(sparc) || defined(__ppc__) +# define SHA3_BYTEORDER 4321 +# else +# define SHA3_BYTEORDER 0 +# endif +#endif + + +/* +** State structure for a SHA3 hash in progress +*/ +typedef struct SHA3Context SHA3Context; +struct SHA3Context { + union { + u64 s[25]; /* Keccak state. 5x5 lines of 64 bits each */ + unsigned char x[1600]; /* ... or 1600 bytes */ + } u; + unsigned nRate; /* Bytes of input accepted per Keccak iteration */ + unsigned nLoaded; /* Input bytes loaded into u.x[] so far this cycle */ + unsigned ixMask; /* Insert next input into u.x[nLoaded^ixMask]. */ +}; + +/* +** A single step of the Keccak mixing function for a 1600-bit state +*/ +static void KeccakF1600Step(SHA3Context *p){ + int i; + u64 B0, B1, B2, B3, B4; + u64 C0, C1, C2, C3, C4; + u64 D0, D1, D2, D3, D4; + static const u64 RC[] = { + 0x0000000000000001ULL, 0x0000000000008082ULL, + 0x800000000000808aULL, 0x8000000080008000ULL, + 0x000000000000808bULL, 0x0000000080000001ULL, + 0x8000000080008081ULL, 0x8000000000008009ULL, + 0x000000000000008aULL, 0x0000000000000088ULL, + 0x0000000080008009ULL, 0x000000008000000aULL, + 0x000000008000808bULL, 0x800000000000008bULL, + 0x8000000000008089ULL, 0x8000000000008003ULL, + 0x8000000000008002ULL, 0x8000000000000080ULL, + 0x000000000000800aULL, 0x800000008000000aULL, + 0x8000000080008081ULL, 0x8000000000008080ULL, + 0x0000000080000001ULL, 0x8000000080008008ULL + }; +# define A00 (p->u.s[0]) +# define A01 (p->u.s[1]) +# define A02 (p->u.s[2]) +# define A03 (p->u.s[3]) +# define A04 (p->u.s[4]) +# define A10 (p->u.s[5]) +# define A11 (p->u.s[6]) +# define A12 (p->u.s[7]) +# define A13 (p->u.s[8]) +# define A14 (p->u.s[9]) +# define A20 (p->u.s[10]) +# define A21 (p->u.s[11]) +# define A22 (p->u.s[12]) +# define A23 (p->u.s[13]) +# define A24 (p->u.s[14]) +# define A30 (p->u.s[15]) +# define A31 (p->u.s[16]) +# define A32 (p->u.s[17]) +# define A33 (p->u.s[18]) +# define A34 (p->u.s[19]) +# define A40 (p->u.s[20]) +# define A41 (p->u.s[21]) +# define A42 (p->u.s[22]) +# define A43 (p->u.s[23]) +# define A44 (p->u.s[24]) +# define ROL64(a,x) ((a<>(64-x))) + + for(i=0; i<24; i+=4){ + C0 = A00^A10^A20^A30^A40; + C1 = A01^A11^A21^A31^A41; + C2 = A02^A12^A22^A32^A42; + C3 = A03^A13^A23^A33^A43; + C4 = A04^A14^A24^A34^A44; + D0 = C4^ROL64(C1, 1); + D1 = C0^ROL64(C2, 1); + D2 = C1^ROL64(C3, 1); + D3 = C2^ROL64(C4, 1); + D4 = C3^ROL64(C0, 1); + + B0 = (A00^D0); + B1 = ROL64((A11^D1), 44); + B2 = ROL64((A22^D2), 43); + B3 = ROL64((A33^D3), 21); + B4 = ROL64((A44^D4), 14); + A00 = B0 ^((~B1)& B2 ); + A00 ^= RC[i]; + A11 = B1 ^((~B2)& B3 ); + A22 = B2 ^((~B3)& B4 ); + A33 = B3 ^((~B4)& B0 ); + A44 = B4 ^((~B0)& B1 ); + + B2 = ROL64((A20^D0), 3); + B3 = ROL64((A31^D1), 45); + B4 = ROL64((A42^D2), 61); + B0 = ROL64((A03^D3), 28); + B1 = ROL64((A14^D4), 20); + A20 = B0 ^((~B1)& B2 ); + A31 = B1 ^((~B2)& B3 ); + A42 = B2 ^((~B3)& B4 ); + A03 = B3 ^((~B4)& B0 ); + A14 = B4 ^((~B0)& B1 ); + + B4 = ROL64((A40^D0), 18); + B0 = ROL64((A01^D1), 1); + B1 = ROL64((A12^D2), 6); + B2 = ROL64((A23^D3), 25); + B3 = ROL64((A34^D4), 8); + A40 = B0 ^((~B1)& B2 ); + A01 = B1 ^((~B2)& B3 ); + A12 = B2 ^((~B3)& B4 ); + A23 = B3 ^((~B4)& B0 ); + A34 = B4 ^((~B0)& B1 ); + + B1 = ROL64((A10^D0), 36); + B2 = ROL64((A21^D1), 10); + B3 = ROL64((A32^D2), 15); + B4 = ROL64((A43^D3), 56); + B0 = ROL64((A04^D4), 27); + A10 = B0 ^((~B1)& B2 ); + A21 = B1 ^((~B2)& B3 ); + A32 = B2 ^((~B3)& B4 ); + A43 = B3 ^((~B4)& B0 ); + A04 = B4 ^((~B0)& B1 ); + + B3 = ROL64((A30^D0), 41); + B4 = ROL64((A41^D1), 2); + B0 = ROL64((A02^D2), 62); + B1 = ROL64((A13^D3), 55); + B2 = ROL64((A24^D4), 39); + A30 = B0 ^((~B1)& B2 ); + A41 = B1 ^((~B2)& B3 ); + A02 = B2 ^((~B3)& B4 ); + A13 = B3 ^((~B4)& B0 ); + A24 = B4 ^((~B0)& B1 ); + + C0 = A00^A20^A40^A10^A30; + C1 = A11^A31^A01^A21^A41; + C2 = A22^A42^A12^A32^A02; + C3 = A33^A03^A23^A43^A13; + C4 = A44^A14^A34^A04^A24; + D0 = C4^ROL64(C1, 1); + D1 = C0^ROL64(C2, 1); + D2 = C1^ROL64(C3, 1); + D3 = C2^ROL64(C4, 1); + D4 = C3^ROL64(C0, 1); + + B0 = (A00^D0); + B1 = ROL64((A31^D1), 44); + B2 = ROL64((A12^D2), 43); + B3 = ROL64((A43^D3), 21); + B4 = ROL64((A24^D4), 14); + A00 = B0 ^((~B1)& B2 ); + A00 ^= RC[i+1]; + A31 = B1 ^((~B2)& B3 ); + A12 = B2 ^((~B3)& B4 ); + A43 = B3 ^((~B4)& B0 ); + A24 = B4 ^((~B0)& B1 ); + + B2 = ROL64((A40^D0), 3); + B3 = ROL64((A21^D1), 45); + B4 = ROL64((A02^D2), 61); + B0 = ROL64((A33^D3), 28); + B1 = ROL64((A14^D4), 20); + A40 = B0 ^((~B1)& B2 ); + A21 = B1 ^((~B2)& B3 ); + A02 = B2 ^((~B3)& B4 ); + A33 = B3 ^((~B4)& B0 ); + A14 = B4 ^((~B0)& B1 ); + + B4 = ROL64((A30^D0), 18); + B0 = ROL64((A11^D1), 1); + B1 = ROL64((A42^D2), 6); + B2 = ROL64((A23^D3), 25); + B3 = ROL64((A04^D4), 8); + A30 = B0 ^((~B1)& B2 ); + A11 = B1 ^((~B2)& B3 ); + A42 = B2 ^((~B3)& B4 ); + A23 = B3 ^((~B4)& B0 ); + A04 = B4 ^((~B0)& B1 ); + + B1 = ROL64((A20^D0), 36); + B2 = ROL64((A01^D1), 10); + B3 = ROL64((A32^D2), 15); + B4 = ROL64((A13^D3), 56); + B0 = ROL64((A44^D4), 27); + A20 = B0 ^((~B1)& B2 ); + A01 = B1 ^((~B2)& B3 ); + A32 = B2 ^((~B3)& B4 ); + A13 = B3 ^((~B4)& B0 ); + A44 = B4 ^((~B0)& B1 ); + + B3 = ROL64((A10^D0), 41); + B4 = ROL64((A41^D1), 2); + B0 = ROL64((A22^D2), 62); + B1 = ROL64((A03^D3), 55); + B2 = ROL64((A34^D4), 39); + A10 = B0 ^((~B1)& B2 ); + A41 = B1 ^((~B2)& B3 ); + A22 = B2 ^((~B3)& B4 ); + A03 = B3 ^((~B4)& B0 ); + A34 = B4 ^((~B0)& B1 ); + + C0 = A00^A40^A30^A20^A10; + C1 = A31^A21^A11^A01^A41; + C2 = A12^A02^A42^A32^A22; + C3 = A43^A33^A23^A13^A03; + C4 = A24^A14^A04^A44^A34; + D0 = C4^ROL64(C1, 1); + D1 = C0^ROL64(C2, 1); + D2 = C1^ROL64(C3, 1); + D3 = C2^ROL64(C4, 1); + D4 = C3^ROL64(C0, 1); + + B0 = (A00^D0); + B1 = ROL64((A21^D1), 44); + B2 = ROL64((A42^D2), 43); + B3 = ROL64((A13^D3), 21); + B4 = ROL64((A34^D4), 14); + A00 = B0 ^((~B1)& B2 ); + A00 ^= RC[i+2]; + A21 = B1 ^((~B2)& B3 ); + A42 = B2 ^((~B3)& B4 ); + A13 = B3 ^((~B4)& B0 ); + A34 = B4 ^((~B0)& B1 ); + + B2 = ROL64((A30^D0), 3); + B3 = ROL64((A01^D1), 45); + B4 = ROL64((A22^D2), 61); + B0 = ROL64((A43^D3), 28); + B1 = ROL64((A14^D4), 20); + A30 = B0 ^((~B1)& B2 ); + A01 = B1 ^((~B2)& B3 ); + A22 = B2 ^((~B3)& B4 ); + A43 = B3 ^((~B4)& B0 ); + A14 = B4 ^((~B0)& B1 ); + + B4 = ROL64((A10^D0), 18); + B0 = ROL64((A31^D1), 1); + B1 = ROL64((A02^D2), 6); + B2 = ROL64((A23^D3), 25); + B3 = ROL64((A44^D4), 8); + A10 = B0 ^((~B1)& B2 ); + A31 = B1 ^((~B2)& B3 ); + A02 = B2 ^((~B3)& B4 ); + A23 = B3 ^((~B4)& B0 ); + A44 = B4 ^((~B0)& B1 ); + + B1 = ROL64((A40^D0), 36); + B2 = ROL64((A11^D1), 10); + B3 = ROL64((A32^D2), 15); + B4 = ROL64((A03^D3), 56); + B0 = ROL64((A24^D4), 27); + A40 = B0 ^((~B1)& B2 ); + A11 = B1 ^((~B2)& B3 ); + A32 = B2 ^((~B3)& B4 ); + A03 = B3 ^((~B4)& B0 ); + A24 = B4 ^((~B0)& B1 ); + + B3 = ROL64((A20^D0), 41); + B4 = ROL64((A41^D1), 2); + B0 = ROL64((A12^D2), 62); + B1 = ROL64((A33^D3), 55); + B2 = ROL64((A04^D4), 39); + A20 = B0 ^((~B1)& B2 ); + A41 = B1 ^((~B2)& B3 ); + A12 = B2 ^((~B3)& B4 ); + A33 = B3 ^((~B4)& B0 ); + A04 = B4 ^((~B0)& B1 ); + + C0 = A00^A30^A10^A40^A20; + C1 = A21^A01^A31^A11^A41; + C2 = A42^A22^A02^A32^A12; + C3 = A13^A43^A23^A03^A33; + C4 = A34^A14^A44^A24^A04; + D0 = C4^ROL64(C1, 1); + D1 = C0^ROL64(C2, 1); + D2 = C1^ROL64(C3, 1); + D3 = C2^ROL64(C4, 1); + D4 = C3^ROL64(C0, 1); + + B0 = (A00^D0); + B1 = ROL64((A01^D1), 44); + B2 = ROL64((A02^D2), 43); + B3 = ROL64((A03^D3), 21); + B4 = ROL64((A04^D4), 14); + A00 = B0 ^((~B1)& B2 ); + A00 ^= RC[i+3]; + A01 = B1 ^((~B2)& B3 ); + A02 = B2 ^((~B3)& B4 ); + A03 = B3 ^((~B4)& B0 ); + A04 = B4 ^((~B0)& B1 ); + + B2 = ROL64((A10^D0), 3); + B3 = ROL64((A11^D1), 45); + B4 = ROL64((A12^D2), 61); + B0 = ROL64((A13^D3), 28); + B1 = ROL64((A14^D4), 20); + A10 = B0 ^((~B1)& B2 ); + A11 = B1 ^((~B2)& B3 ); + A12 = B2 ^((~B3)& B4 ); + A13 = B3 ^((~B4)& B0 ); + A14 = B4 ^((~B0)& B1 ); + + B4 = ROL64((A20^D0), 18); + B0 = ROL64((A21^D1), 1); + B1 = ROL64((A22^D2), 6); + B2 = ROL64((A23^D3), 25); + B3 = ROL64((A24^D4), 8); + A20 = B0 ^((~B1)& B2 ); + A21 = B1 ^((~B2)& B3 ); + A22 = B2 ^((~B3)& B4 ); + A23 = B3 ^((~B4)& B0 ); + A24 = B4 ^((~B0)& B1 ); + + B1 = ROL64((A30^D0), 36); + B2 = ROL64((A31^D1), 10); + B3 = ROL64((A32^D2), 15); + B4 = ROL64((A33^D3), 56); + B0 = ROL64((A34^D4), 27); + A30 = B0 ^((~B1)& B2 ); + A31 = B1 ^((~B2)& B3 ); + A32 = B2 ^((~B3)& B4 ); + A33 = B3 ^((~B4)& B0 ); + A34 = B4 ^((~B0)& B1 ); + + B3 = ROL64((A40^D0), 41); + B4 = ROL64((A41^D1), 2); + B0 = ROL64((A42^D2), 62); + B1 = ROL64((A43^D3), 55); + B2 = ROL64((A44^D4), 39); + A40 = B0 ^((~B1)& B2 ); + A41 = B1 ^((~B2)& B3 ); + A42 = B2 ^((~B3)& B4 ); + A43 = B3 ^((~B4)& B0 ); + A44 = B4 ^((~B0)& B1 ); + } +} + +/* +** Initialize a new hash. iSize determines the size of the hash +** in bits and should be one of 224, 256, 384, or 512. Or iSize +** can be zero to use the default hash size of 256 bits. +*/ +static void SHA3Init(SHA3Context *p, int iSize){ + memset(p, 0, sizeof(*p)); + if( iSize>=128 && iSize<=512 ){ + p->nRate = (1600 - ((iSize + 31)&~31)*2)/8; + }else{ + p->nRate = (1600 - 2*256)/8; + } +#if SHA3_BYTEORDER==1234 + /* Known to be little-endian at compile-time. No-op */ +#elif SHA3_BYTEORDER==4321 + p->ixMask = 7; /* Big-endian */ +#else + { + static unsigned int one = 1; + if( 1==*(unsigned char*)&one ){ + /* Little endian. No byte swapping. */ + p->ixMask = 0; + }else{ + /* Big endian. Byte swap. */ + p->ixMask = 7; + } + } +#endif +} + +/* +** Make consecutive calls to the SHA3Update function to add new content +** to the hash +*/ +static void SHA3Update( + SHA3Context *p, + const unsigned char *aData, + unsigned int nData +){ + unsigned int i = 0; +#if SHA3_BYTEORDER==1234 + if( (p->nLoaded % 8)==0 && ((aData - (const unsigned char*)0)&7)==0 ){ + for(; i+7u.s[p->nLoaded/8] ^= *(u64*)&aData[i]; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Fri Aug 11 00:01:23 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6DCAC7ACFC; Fri, 11 Aug 2017 00:01:23 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AD8476E2D; Fri, 11 Aug 2017 00:01:23 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7B01Mjl067201; Fri, 11 Aug 2017 00:01:22 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7B01M1m067200; Fri, 11 Aug 2017 00:01:22 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201708110001.v7B01M1m067200@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Fri, 11 Aug 2017 00:01:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r322387 - vendor/sqlite3/sqlite-3200000 X-SVN-Group: vendor X-SVN-Commit-Author: peter X-SVN-Commit-Paths: vendor/sqlite3/sqlite-3200000 X-SVN-Commit-Revision: 322387 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2017 00:01:23 -0000 Author: peter Date: Fri Aug 11 00:01:22 2017 New Revision: 322387 URL: https://svnweb.freebsd.org/changeset/base/322387 Log: Tag import of sqlite-3.20.0 Added: vendor/sqlite3/sqlite-3200000/ - copied from r322386, vendor/sqlite3/dist/