From owner-svn-src-all@freebsd.org Wed May 24 21:29:33 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB65BD7CAD0; Wed, 24 May 2017 21:29:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A85618B5; Wed, 24 May 2017 21:29:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v4OLTWiV027894; Wed, 24 May 2017 21:29:32 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v4OLTVbx027885; Wed, 24 May 2017 21:29:31 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201705242129.v4OLTVbx027885@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 24 May 2017 21:29:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r318818 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 May 2017 21:29:33 -0000 Author: avg Date: Wed May 24 21:29:31 2017 New Revision: 318818 URL: https://svnweb.freebsd.org/changeset/base/318818 Log: MFC r316907: 1300 filename normalization doesn't work for removes illumos/illumos-gate@1c17160ac558f98048951327f4e9248d8f46acc0 https://github.com/illumos/illumos-gate/commit/1c17160ac558f98048951327f4e9248d8f46acc0 https://www.illumos.org/issues/1300 FreeBSD note: recent FreeBSD was not affected by the issue fixed as the name cache is completely bypassed when normalization is enabled. The change is imported for the sake of ZAP infrastructure modifications. Reviewed by: Yuri Pankov Reviewed by: Pavel Zakharov Reviewed by: Matt Ahrens Approved by: Dan McDonald Author: Kevin Crowe MFC after: 3 weeks Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_bookmark.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Wed May 24 21:29:31 2017 (r318818) @@ -18,15 +18,16 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012, 2016 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. - * Copyright 2015 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2015, STRATO AG, Inc. All rights reserved. * Copyright (c) 2014 Integros [integros.com] + * Copyright 2017 Nexenta Systems, Inc. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -1622,7 +1623,7 @@ dmu_snapshot_realname(objset_t *os, char return (zap_lookup_norm(ds->ds_dir->dd_pool->dp_meta_objset, dsl_dataset_phys(ds)->ds_snapnames_zapobj, name, 8, 1, &ignored, - MT_FIRST, real, maxlen, conflict)); + MT_NORMALIZE, real, maxlen, conflict)); } int Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_bookmark.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_bookmark.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_bookmark.c Wed May 24 21:29:31 2017 (r318818) @@ -12,8 +12,10 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2013, 2014 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. */ #include @@ -59,16 +61,14 @@ dsl_dataset_bmark_lookup(dsl_dataset_t * { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; uint64_t bmark_zapobj = ds->ds_bookmarks; - matchtype_t mt; + matchtype_t mt = 0; int err; if (bmark_zapobj == 0) return (SET_ERROR(ESRCH)); if (dsl_dataset_phys(ds)->ds_flags & DS_FLAG_CI_DATASET) - mt = MT_FIRST; - else - mt = MT_EXACT; + mt = MT_NORMALIZE; err = zap_lookup_norm(mos, bmark_zapobj, shortname, sizeof (uint64_t), sizeof (*bmark_phys) / sizeof (uint64_t), bmark_phys, mt, @@ -339,12 +339,10 @@ dsl_dataset_bookmark_remove(dsl_dataset_ { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; uint64_t bmark_zapobj = ds->ds_bookmarks; - matchtype_t mt; + matchtype_t mt = 0; if (dsl_dataset_phys(ds)->ds_flags & DS_FLAG_CI_DATASET) - mt = MT_FIRST; - else - mt = MT_EXACT; + mt = MT_NORMALIZE; return (zap_remove_norm(mos, bmark_zapobj, name, mt, tx)); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c Wed May 24 21:29:31 2017 (r318818) @@ -18,6 +18,7 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Portions Copyright (c) 2011 Martin Matuska @@ -27,6 +28,7 @@ * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2016, OmniTI Computer Consulting, Inc. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. */ #include @@ -364,17 +366,15 @@ dsl_dataset_snap_lookup(dsl_dataset_t *d { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; uint64_t snapobj = dsl_dataset_phys(ds)->ds_snapnames_zapobj; - matchtype_t mt; + matchtype_t mt = 0; int err; if (dsl_dataset_phys(ds)->ds_flags & DS_FLAG_CI_DATASET) - mt = MT_FIRST; - else - mt = MT_EXACT; + mt = MT_NORMALIZE; err = zap_lookup_norm(mos, snapobj, name, 8, 1, value, mt, NULL, 0, NULL); - if (err == ENOTSUP && mt == MT_FIRST) + if (err == ENOTSUP && (mt & MT_NORMALIZE)) err = zap_lookup(mos, snapobj, name, 8, 1, value); return (err); } @@ -385,18 +385,16 @@ dsl_dataset_snap_remove(dsl_dataset_t *d { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; uint64_t snapobj = dsl_dataset_phys(ds)->ds_snapnames_zapobj; - matchtype_t mt; + matchtype_t mt = 0; int err; dsl_dir_snap_cmtime_update(ds->ds_dir); if (dsl_dataset_phys(ds)->ds_flags & DS_FLAG_CI_DATASET) - mt = MT_FIRST; - else - mt = MT_EXACT; + mt = MT_NORMALIZE; err = zap_remove_norm(mos, snapobj, name, mt, tx); - if (err == ENOTSUP && mt == MT_FIRST) + if (err == ENOTSUP && (mt & MT_NORMALIZE)) err = zap_remove(mos, snapobj, name, tx); if (err == 0 && adj_cnt) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap.h Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap.h Wed May 24 21:29:31 2017 (r318818) @@ -18,9 +18,11 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. */ #ifndef _SYS_ZAP_H @@ -88,22 +90,15 @@ extern "C" { /* * Specifies matching criteria for ZAP lookups. - */ -typedef enum matchtype -{ - /* Only find an exact match (non-normalized) */ - MT_EXACT, - /* - * If there is an exact match, find that, otherwise find the - * first normalized match. - */ - MT_BEST, - /* - * Find the "first" normalized (case and Unicode form) match; - * the designated "first" match will not change as long as the - * set of entries with this normalization doesn't change. - */ - MT_FIRST + * MT_NORMALIZE Use ZAP normalization flags, which can include both + * unicode normalization and case-insensitivity. + * MT_MATCH_CASE Do case-sensitive lookups even if MT_NORMALIZE is + * specified and ZAP normalization flags include + * U8_TEXTPREP_TOUPPER. + */ +typedef enum matchtype { + MT_NORMALIZE = 1 << 0, + MT_MATCH_CASE = 1 << 1, } matchtype_t; typedef enum zap_flags { @@ -120,16 +115,6 @@ typedef enum zap_flags { /* * Create a new zapobj with no attributes and return its object number. - * MT_EXACT will cause the zap object to only support MT_EXACT lookups, - * otherwise any matchtype can be used for lookups. - * - * normflags specifies what normalization will be done. values are: - * 0: no normalization (legacy on-disk format, supports MT_EXACT matching - * only) - * U8_TEXTPREP_TOLOWER: case normalization will be performed. - * MT_FIRST/MT_BEST matching will find entries that match without - * regard to case (eg. looking for "foo" can find an entry "Foo"). - * Eventually, other flags will permit unicode normalization as well. */ uint64_t zap_create(objset_t *ds, dmu_object_type_t ot, dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap_impl.h Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zap_impl.h Wed May 24 21:29:31 2017 (r318818) @@ -18,11 +18,13 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013, 2016 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright (c) 2014 Integros [integros.com] + * Copyright 2017 Nexenta Systems, Inc. */ #ifndef _SYS_ZAP_IMPL_H @@ -189,6 +191,7 @@ typedef struct zap_name { int zn_key_norm_numints; uint64_t zn_hash; matchtype_t zn_matchtype; + int zn_normflags; char zn_normbuf[ZAP_MAXNAMELEN]; } zap_name_t; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c Wed May 24 21:29:31 2017 (r318818) @@ -18,9 +18,11 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. */ /* @@ -361,7 +363,7 @@ zap_leaf_array_match(zap_leaf_t *l, zap_ } ASSERT(zn->zn_key_intlen == 1); - if (zn->zn_matchtype == MT_FIRST) { + if (zn->zn_matchtype & MT_NORMALIZE) { char *thisname = kmem_alloc(array_numints, KM_SLEEP); boolean_t match; @@ -403,7 +405,6 @@ zap_leaf_lookup(zap_leaf_t *l, zap_name_ ASSERT3U(zap_leaf_phys(l)->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); -again: for (chunkp = LEAF_HASH_ENTPTR(l, zn->zn_hash); *chunkp != CHAIN_END; chunkp = &le->le_next) { uint16_t chunk = *chunkp; @@ -418,9 +419,9 @@ again: /* * NB: the entry chain is always sorted by cd on * normalized zap objects, so this will find the - * lowest-cd match for MT_FIRST. + * lowest-cd match for MT_NORMALIZE. */ - ASSERT(zn->zn_matchtype == MT_EXACT || + ASSERT((zn->zn_matchtype == 0) || (zap_leaf_phys(l)->l_hdr.lh_flags & ZLF_ENTRIES_CDSORTED)); if (zap_leaf_array_match(l, zn, le->le_name_chunk, le->le_name_numints)) { @@ -434,15 +435,6 @@ again: } } - /* - * NB: we could of course do this in one pass, but that would be - * a pain. We'll see if MT_BEST is even used much. - */ - if (zn->zn_matchtype == MT_BEST) { - zn->zn_matchtype = MT_FIRST; - goto again; - } - return (SET_ERROR(ENOENT)); } @@ -697,7 +689,7 @@ zap_entry_normalization_conflict(zap_ent continue; if (zn == NULL) { - zn = zap_name_alloc(zap, name, MT_FIRST); + zn = zap_name_alloc(zap, name, MT_NORMALIZE); allocdzn = B_TRUE; } if (zap_leaf_array_match(zeh->zeh_leaf, zn, Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c Wed May 24 21:29:31 2017 (r318818) @@ -18,11 +18,13 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2011, 2016 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright (c) 2014 Integros [integros.com] + * Copyright 2017 Nexenta Systems, Inc. */ #include @@ -133,7 +135,7 @@ zap_hash(zap_name_t *zn) } static int -zap_normalize(zap_t *zap, const char *name, char *namenorm) +zap_normalize(zap_t *zap, const char *name, char *namenorm, int normflags) { size_t inlen, outlen; int err; @@ -145,8 +147,8 @@ zap_normalize(zap_t *zap, const char *na err = 0; (void) u8_textprep_str((char *)name, &inlen, namenorm, &outlen, - zap->zap_normflags | U8_TEXTPREP_IGNORE_NULL | - U8_TEXTPREP_IGNORE_INVALID, U8_UNICODE_LATEST, &err); + normflags | U8_TEXTPREP_IGNORE_NULL | U8_TEXTPREP_IGNORE_INVALID, + U8_UNICODE_LATEST, &err); return (err); } @@ -156,15 +158,15 @@ zap_match(zap_name_t *zn, const char *ma { ASSERT(!(zap_getflags(zn->zn_zap) & ZAP_FLAG_UINT64_KEY)); - if (zn->zn_matchtype == MT_FIRST) { + if (zn->zn_matchtype & MT_NORMALIZE) { char norm[ZAP_MAXNAMELEN]; - if (zap_normalize(zn->zn_zap, matchname, norm) != 0) + if (zap_normalize(zn->zn_zap, matchname, norm, + zn->zn_normflags) != 0) return (B_FALSE); return (strcmp(zn->zn_key_norm, norm) == 0); } else { - /* MT_BEST or MT_EXACT */ return (strcmp(zn->zn_key_orig, matchname) == 0); } } @@ -185,15 +187,30 @@ zap_name_alloc(zap_t *zap, const char *k zn->zn_key_orig = key; zn->zn_key_orig_numints = strlen(zn->zn_key_orig) + 1; zn->zn_matchtype = mt; + zn->zn_normflags = zap->zap_normflags; + + /* + * If we're dealing with a case sensitive lookup on a mixed or + * insensitive fs, remove U8_TEXTPREP_TOUPPER or the lookup + * will fold case to all caps overriding the lookup request. + */ + if (mt & MT_MATCH_CASE) + zn->zn_normflags &= ~U8_TEXTPREP_TOUPPER; + if (zap->zap_normflags) { - if (zap_normalize(zap, key, zn->zn_normbuf) != 0) { + /* + * We *must* use zap_normflags because this normalization is + * what the hash is computed from. + */ + if (zap_normalize(zap, key, zn->zn_normbuf, + zap->zap_normflags) != 0) { zap_name_free(zn); return (NULL); } zn->zn_key_norm = zn->zn_normbuf; zn->zn_key_norm_numints = strlen(zn->zn_key_norm) + 1; } else { - if (mt != MT_EXACT) { + if (mt != 0) { zap_name_free(zn); return (NULL); } @@ -202,6 +219,20 @@ zap_name_alloc(zap_t *zap, const char *k } zn->zn_hash = zap_hash(zn); + + if (zap->zap_normflags != zn->zn_normflags) { + /* + * We *must* use zn_normflags because this normalization is + * what the matching is based on. (Not the hash!) + */ + if (zap_normalize(zap, key, zn->zn_normbuf, + zn->zn_normflags) != 0) { + zap_name_free(zn); + return (NULL); + } + zn->zn_key_norm_numints = strlen(zn->zn_key_norm) + 1; + } + return (zn); } @@ -215,7 +246,7 @@ zap_name_alloc_uint64(zap_t *zap, const zn->zn_key_intlen = sizeof (*key); zn->zn_key_orig = zn->zn_key_norm = key; zn->zn_key_orig_numints = zn->zn_key_norm_numints = numints; - zn->zn_matchtype = MT_EXACT; + zn->zn_matchtype = 0; zn->zn_hash = zap_hash(zn); return (zn); @@ -305,7 +336,6 @@ mze_find(zap_name_t *zn) mze_tofind.mze_hash = zn->zn_hash; mze_tofind.mze_cd = 0; -again: mze = avl_find(avl, &mze_tofind, &idx); if (mze == NULL) mze = avl_nearest(avl, idx, AVL_AFTER); @@ -314,10 +344,7 @@ again: if (zap_match(zn, MZE_PHYS(zn->zn_zap, mze)->mze_name)) return (mze); } - if (zn->zn_matchtype == MT_BEST) { - zn->zn_matchtype = MT_FIRST; - goto again; - } + return (NULL); } @@ -422,8 +449,7 @@ mzap_open(objset_t *os, uint64_t obj, dm if (mze->mze_name[0]) { zap_name_t *zn; - zn = zap_name_alloc(zap, mze->mze_name, - MT_EXACT); + zn = zap_name_alloc(zap, mze->mze_name, 0); if (mze_insert(zap, i, zn->zn_hash) == 0) zap->zap_m.zap_num_entries++; else { @@ -629,7 +655,7 @@ mzap_upgrade(zap_t **zapp, void *tag, dm continue; dprintf("adding %s=%llu\n", mze->mze_name, mze->mze_value); - zn = zap_name_alloc(zap, mze->mze_name, MT_EXACT); + zn = zap_name_alloc(zap, mze->mze_name, 0); err = fzap_add_cd(zn, 8, 1, &mze->mze_value, mze->mze_cd, tag, tx); zap = zn->zn_zap; /* fzap_add_cd() may change zap */ @@ -642,6 +668,23 @@ mzap_upgrade(zap_t **zapp, void *tag, dm return (err); } +/* + * The "normflags" determine the behavior of the matchtype_t which is + * passed to zap_lookup_norm(). Names which have the same normalized + * version will be stored with the same hash value, and therefore we can + * perform normalization-insensitive lookups. We can be Unicode form- + * insensitive and/or case-insensitive. The following flags are valid for + * "normflags": + * + * U8_TEXTPREP_NFC + * U8_TEXTPREP_NFD + * U8_TEXTPREP_NFKC + * U8_TEXTPREP_NFKD + * U8_TEXTPREP_TOUPPER + * + * The *_NF* (Normalization Form) flags are mutually exclusive; at most one + * of them may be supplied. + */ void mzap_create_impl(objset_t *os, uint64_t obj, int normflags, zap_flags_t flags, dmu_tx_t *tx) @@ -800,7 +843,7 @@ again: if (zn == NULL) { zn = zap_name_alloc(zap, MZE_PHYS(zap, mze)->mze_name, - MT_FIRST); + MT_NORMALIZE); allocdzn = B_TRUE; } if (zap_match(zn, MZE_PHYS(zap, other)->mze_name)) { @@ -829,7 +872,7 @@ zap_lookup(objset_t *os, uint64_t zapobj uint64_t integer_size, uint64_t num_integers, void *buf) { return (zap_lookup_norm(os, zapobj, name, integer_size, - num_integers, buf, MT_EXACT, NULL, 0, NULL)); + num_integers, buf, 0, NULL, 0, NULL)); } static int @@ -897,7 +940,7 @@ zap_lookup_by_dnode(dnode_t *dn, const c uint64_t integer_size, uint64_t num_integers, void *buf) { return (zap_lookup_norm_by_dnode(dn, name, integer_size, - num_integers, buf, MT_EXACT, NULL, 0, NULL)); + num_integers, buf, 0, NULL, 0, NULL)); } int @@ -970,7 +1013,7 @@ int zap_contains(objset_t *os, uint64_t zapobj, const char *name) { int err = zap_lookup_norm(os, zapobj, name, 0, - 0, NULL, MT_EXACT, NULL, 0, NULL); + 0, NULL, 0, NULL, 0, NULL); if (err == EOVERFLOW || err == EINVAL) err = 0; /* found, but skipped reading the value */ return (err); @@ -988,7 +1031,7 @@ zap_length(objset_t *os, uint64_t zapobj err = zap_lockdir(os, zapobj, NULL, RW_READER, TRUE, FALSE, FTAG, &zap); if (err) return (err); - zn = zap_name_alloc(zap, name, MT_EXACT); + zn = zap_name_alloc(zap, name, 0); if (zn == NULL) { zap_unlockdir(zap, FTAG); return (SET_ERROR(ENOTSUP)); @@ -1091,7 +1134,7 @@ zap_add(objset_t *os, uint64_t zapobj, c err = zap_lockdir(os, zapobj, tx, RW_WRITER, TRUE, TRUE, FTAG, &zap); if (err) return (err); - zn = zap_name_alloc(zap, key, MT_EXACT); + zn = zap_name_alloc(zap, key, 0); if (zn == NULL) { zap_unlockdir(zap, FTAG); return (SET_ERROR(ENOTSUP)); @@ -1170,7 +1213,7 @@ zap_update(objset_t *os, uint64_t zapobj err = zap_lockdir(os, zapobj, tx, RW_WRITER, TRUE, TRUE, FTAG, &zap); if (err) return (err); - zn = zap_name_alloc(zap, name, MT_EXACT); + zn = zap_name_alloc(zap, name, 0); if (zn == NULL) { zap_unlockdir(zap, FTAG); return (SET_ERROR(ENOTSUP)); @@ -1233,7 +1276,7 @@ zap_update_uint64(objset_t *os, uint64_t int zap_remove(objset_t *os, uint64_t zapobj, const char *name, dmu_tx_t *tx) { - return (zap_remove_norm(os, zapobj, name, MT_EXACT, tx)); + return (zap_remove_norm(os, zapobj, name, 0, tx)); } int @@ -1525,7 +1568,7 @@ zap_count_write_by_dnode(dnode_t *dn, co return (err); if (!zap->zap_ismicro) { - zap_name_t *zn = zap_name_alloc(zap, name, MT_EXACT); + zap_name_t *zn = zap_name_alloc(zap, name, 0); if (zn) { err = fzap_count_write(zn, add, towrite, tooverwrite); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c Wed May 24 21:29:31 2017 (r318818) @@ -18,9 +18,11 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. */ #include @@ -63,12 +65,11 @@ */ static int zfs_match_find(zfsvfs_t *zfsvfs, znode_t *dzp, const char *name, - boolean_t exact, uint64_t *zoid) + matchtype_t mt, uint64_t *zoid) { int error; if (zfsvfs->z_norm) { - matchtype_t mt = exact? MT_EXACT : MT_FIRST; /* * In the non-mixed case we only expect there would ever @@ -108,7 +109,7 @@ int zfs_dirent_lookup(znode_t *dzp, const char *name, znode_t **zpp, int flag) { zfsvfs_t *zfsvfs = dzp->z_zfsvfs; - boolean_t exact; + matchtype_t mt = 0; uint64_t zoid; vnode_t *vp = NULL; int error = 0; @@ -131,15 +132,39 @@ zfs_dirent_lookup(znode_t *dzp, const ch * zfsvfs->z_case and zfsvfs->z_norm fields. These choices * affect how we perform zap lookups. * - * Decide if exact matches should be requested when performing - * a zap lookup on file systems supporting case-insensitive - * access. + * When matching we may need to normalize & change case according to + * FS settings. + * + * Note that a normalized match is necessary for a case insensitive + * filesystem when the lookup request is not exact because normalization + * can fold case independent of normalizing code point sequences. + * + * See the table above zfs_dropname(). + */ + if (zfsvfs->z_norm != 0) { + mt = MT_NORMALIZE; + + /* + * Determine if the match needs to honor the case specified in + * lookup, and if so keep track of that so that during + * normalization we don't fold case. + */ + if (zfsvfs->z_case == ZFS_CASE_MIXED) { + mt |= MT_MATCH_CASE; + } + } + + /* + * Only look in or update the DNLC if we are looking for the + * name on a file system that does not require normalization + * or case folding. We can also look there if we happen to be + * on a non-normalizing, mixed sensitivity file system IF we + * are looking for the exact name. * * NB: we do not need to worry about this flag for ZFS_CASE_SENSITIVE * because in that case MT_EXACT and MT_FIRST should produce exactly * the same result. */ - exact = zfsvfs->z_case == ZFS_CASE_MIXED; if (dzp->z_unlinked && !(flag & ZXATTR)) return (ENOENT); @@ -149,7 +174,7 @@ zfs_dirent_lookup(znode_t *dzp, const ch if (error == 0) error = (zoid == 0 ? ENOENT : 0); } else { - error = zfs_match_find(zfsvfs, dzp, name, exact, &zoid); + error = zfs_match_find(zfsvfs, dzp, name, mt, &zoid); } if (error) { if (error != ENOENT || (flag & ZEXISTS)) { @@ -566,6 +591,28 @@ zfs_link_create(znode_t *dzp, const char return (0); } +/* + * The match type in the code for this function should conform to: + * + * ------------------------------------------------------------------------ + * fs type | z_norm | lookup type | match type + * ---------|-------------|-------------|---------------------------------- + * CS !norm | 0 | 0 | 0 (exact) + * CS norm | formX | 0 | MT_NORMALIZE + * CI !norm | upper | !ZCIEXACT | MT_NORMALIZE + * CI !norm | upper | ZCIEXACT | MT_NORMALIZE | MT_MATCH_CASE + * CI norm | upper|formX | !ZCIEXACT | MT_NORMALIZE + * CI norm | upper|formX | ZCIEXACT | MT_NORMALIZE | MT_MATCH_CASE + * CM !norm | upper | !ZCILOOK | MT_NORMALIZE | MT_MATCH_CASE + * CM !norm | upper | ZCILOOK | MT_NORMALIZE + * CM norm | upper|formX | !ZCILOOK | MT_NORMALIZE | MT_MATCH_CASE + * CM norm | upper|formX | ZCILOOK | MT_NORMALIZE + * + * Abbreviations: + * CS = Case Sensitive, CI = Case Insensitive, CM = Case Mixed + * upper = case folding set by fs type on creation (U8_TEXTPREP_TOUPPER) + * formX = unicode normalization form set on fs creation + */ static int zfs_dropname(znode_t *dzp, const char *name, znode_t *zp, dmu_tx_t *tx, int flag) @@ -573,15 +620,16 @@ zfs_dropname(znode_t *dzp, const char *n int error; if (zp->z_zfsvfs->z_norm) { - if (zp->z_zfsvfs->z_case == ZFS_CASE_MIXED) - error = zap_remove_norm(zp->z_zfsvfs->z_os, - dzp->z_id, name, MT_EXACT, tx); - else - error = zap_remove_norm(zp->z_zfsvfs->z_os, - dzp->z_id, name, MT_FIRST, tx); + matchtype_t mt = MT_NORMALIZE; + + if (zp->z_zfsvfs->z_case == ZFS_CASE_MIXED) { + mt |= MT_MATCH_CASE; + } + + error = zap_remove_norm(zp->z_zfsvfs->z_os, dzp->z_id, + name, mt, tx); } else { - error = zap_remove(zp->z_zfsvfs->z_os, - dzp->z_id, name, tx); + error = zap_remove(zp->z_zfsvfs->z_os, dzp->z_id, name, tx); } return (error); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Wed May 24 21:18:13 2017 (r318817) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Wed May 24 21:29:31 2017 (r318818) @@ -18,11 +18,12 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012, 2015 by Delphix. All rights reserved. - * Copyright 2014 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2014 Integros [integros.com] + * Copyright 2017 Nexenta Systems, Inc. */ /* Portions Copyright 2007 Jeremy Teo */ @@ -1535,7 +1536,15 @@ zfs_lookup(vnode_t *dvp, char *nm, vnode zfsvfs_t *zfsvfs = zdp->z_zfsvfs; int error = 0; - /* fast path (should be redundant with vfs namecache) */ + /* + * Fast path lookup, however we must skip DNLC lookup + * for case folding or normalizing lookups because the + * DNLC code only stores the passed in name. This means + * creating 'a' and removing 'A' on a case insensitive + * file system would work, but DNLC still thinks 'a' + * exists and won't let you create it again on the next + * pass through fast path. + */ if (!(flags & LOOKUP_XATTR)) { if (dvp->v_type != VDIR) { return (SET_ERROR(ENOTDIR));