From owner-svn-src-vendor@FreeBSD.ORG Thu Jul 17 21:58:41 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40FAA953; Thu, 17 Jul 2014 21:58:41 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2AD6725BC; Thu, 17 Jul 2014 21:58:41 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6HLwfeg021351; Thu, 17 Jul 2014 21:58:41 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6HLwM4O021220; Thu, 17 Jul 2014 21:58:22 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407172158.s6HLwM4O021220@svn.freebsd.org> From: Baptiste Daroussin Date: Thu, 17 Jul 2014 21:58:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268809 - in vendor/byacc/dist: . package package/debian package/pkgsrc test test/btyacc test/yacc X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2014 21:58:41 -0000 Author: bapt Date: Thu Jul 17 21:58:22 2014 New Revision: 268809 URL: http://svnweb.freebsd.org/changeset/base/268809 Log: import byacc 20140715 Deleted: vendor/byacc/dist/NOTES-btyacc-Changes vendor/byacc/dist/NOTES-btyacc-Disposition Modified: vendor/byacc/dist/CHANGES vendor/byacc/dist/MANIFEST vendor/byacc/dist/VERSION vendor/byacc/dist/aclocal.m4 vendor/byacc/dist/main.c vendor/byacc/dist/package/byacc.spec vendor/byacc/dist/package/debian/changelog vendor/byacc/dist/package/mingw-byacc.spec vendor/byacc/dist/package/pkgsrc/Makefile vendor/byacc/dist/test/btyacc/big_b.output vendor/byacc/dist/test/btyacc/big_l.output vendor/byacc/dist/test/btyacc/err_inherit1.error vendor/byacc/dist/test/btyacc/err_inherit2.error vendor/byacc/dist/test/btyacc/err_inherit3.error vendor/byacc/dist/test/btyacc/err_inherit4.error vendor/byacc/dist/test/btyacc/err_inherit5.error vendor/byacc/dist/test/btyacc/err_syntax1.error vendor/byacc/dist/test/btyacc/err_syntax10.error vendor/byacc/dist/test/btyacc/err_syntax11.error vendor/byacc/dist/test/btyacc/err_syntax12.error vendor/byacc/dist/test/btyacc/err_syntax13.error vendor/byacc/dist/test/btyacc/err_syntax14.error vendor/byacc/dist/test/btyacc/err_syntax15.error vendor/byacc/dist/test/btyacc/err_syntax16.error vendor/byacc/dist/test/btyacc/err_syntax17.error vendor/byacc/dist/test/btyacc/err_syntax18.error vendor/byacc/dist/test/btyacc/err_syntax19.error vendor/byacc/dist/test/btyacc/err_syntax2.error vendor/byacc/dist/test/btyacc/err_syntax21.error vendor/byacc/dist/test/btyacc/err_syntax22.error vendor/byacc/dist/test/btyacc/err_syntax23.error vendor/byacc/dist/test/btyacc/err_syntax24.error vendor/byacc/dist/test/btyacc/err_syntax25.error vendor/byacc/dist/test/btyacc/err_syntax26.error vendor/byacc/dist/test/btyacc/err_syntax27.error vendor/byacc/dist/test/btyacc/err_syntax3.error vendor/byacc/dist/test/btyacc/err_syntax4.error vendor/byacc/dist/test/btyacc/err_syntax5.error vendor/byacc/dist/test/btyacc/err_syntax6.error vendor/byacc/dist/test/btyacc/err_syntax7.error vendor/byacc/dist/test/btyacc/err_syntax7a.error vendor/byacc/dist/test/btyacc/err_syntax7b.error vendor/byacc/dist/test/btyacc/err_syntax8.error vendor/byacc/dist/test/btyacc/err_syntax8a.error vendor/byacc/dist/test/btyacc/err_syntax9.error vendor/byacc/dist/test/btyacc/help.output vendor/byacc/dist/test/btyacc/no_b_opt.output vendor/byacc/dist/test/btyacc/no_output2.output vendor/byacc/dist/test/btyacc/no_p_opt.output vendor/byacc/dist/test/btyacc/nostdin.output vendor/byacc/dist/test/run_test.sh vendor/byacc/dist/test/yacc/big_b.output vendor/byacc/dist/test/yacc/big_l.output vendor/byacc/dist/test/yacc/err_syntax1.error vendor/byacc/dist/test/yacc/err_syntax10.error vendor/byacc/dist/test/yacc/err_syntax11.error vendor/byacc/dist/test/yacc/err_syntax12.error vendor/byacc/dist/test/yacc/err_syntax13.error vendor/byacc/dist/test/yacc/err_syntax14.error vendor/byacc/dist/test/yacc/err_syntax15.error vendor/byacc/dist/test/yacc/err_syntax16.error vendor/byacc/dist/test/yacc/err_syntax17.error vendor/byacc/dist/test/yacc/err_syntax18.error vendor/byacc/dist/test/yacc/err_syntax19.error vendor/byacc/dist/test/yacc/err_syntax2.error vendor/byacc/dist/test/yacc/err_syntax21.error vendor/byacc/dist/test/yacc/err_syntax22.error vendor/byacc/dist/test/yacc/err_syntax23.error vendor/byacc/dist/test/yacc/err_syntax24.error vendor/byacc/dist/test/yacc/err_syntax25.error vendor/byacc/dist/test/yacc/err_syntax26.error vendor/byacc/dist/test/yacc/err_syntax27.error vendor/byacc/dist/test/yacc/err_syntax3.error vendor/byacc/dist/test/yacc/err_syntax4.error vendor/byacc/dist/test/yacc/err_syntax5.error vendor/byacc/dist/test/yacc/err_syntax6.error vendor/byacc/dist/test/yacc/err_syntax7.error vendor/byacc/dist/test/yacc/err_syntax7a.error vendor/byacc/dist/test/yacc/err_syntax7b.error vendor/byacc/dist/test/yacc/err_syntax8.error vendor/byacc/dist/test/yacc/err_syntax8a.error vendor/byacc/dist/test/yacc/err_syntax9.error vendor/byacc/dist/test/yacc/help.output vendor/byacc/dist/test/yacc/no_b_opt.output vendor/byacc/dist/test/yacc/no_output2.output vendor/byacc/dist/test/yacc/no_p_opt.output vendor/byacc/dist/test/yacc/nostdin.output Modified: vendor/byacc/dist/CHANGES ============================================================================== --- vendor/byacc/dist/CHANGES Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/CHANGES Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,38 @@ +2014-07-15 Thomas E. Dickey + + * aclocal.m4: resync with my-autoconf (no change to configure script) + + * VERSION, package/byacc.spec, package/debian/changelog, package/mingw-byacc.spec, package/pkgsrc/Makefile: + bump + + * test/run_test.sh: + make top-level "make check" work again, by adding another step to filtering + the test results. + +2014-07-14 Thomas E. Dickey + + * test/run_test.sh: changes from Garrett Cooper's patch: + a) ensure that the script returns an error-code if there are differences + b) escape "." character in left side of sed expression for $YACC + c) ensure that $ifBTYACC has a value + + * test/btyacc/big_b.output, test/btyacc/big_l.output, test/btyacc/help.output, test/btyacc/no_b_opt.output, test/btyacc/no_output2.output, test/btyacc/no_p_opt.output, test/btyacc/nostdin.output: + regen (reminder by Garrett Cooper) + +2014-07-14 Garrett.Cooper + + * test/btyacc/err_inherit1.error, test/btyacc/err_inherit2.error, test/btyacc/err_inherit3.error, test/btyacc/err_inherit4.error, test/btyacc/err_inherit5.error, test/btyacc/err_syntax1.error, test/btyacc/err_syntax10.error, test/btyacc/err_syntax11.error, test/btyacc/err_syntax12.error, test/btyacc/err_syntax13.error, test/btyacc/err_syntax14.error, test/btyacc/err_syntax15.error, test/btyacc/err_syntax16.error, test/btyacc/err_syntax17.error, test/btyacc/err_syntax18.error, test/btyacc/err_syntax19.error, test/btyacc/err_syntax2.error, test/btyacc/err_syntax21.error, test/btyacc/err_syntax22.error, test/btyacc/err_syntax23.error, test/btyacc/err_syntax24.error, test/btyacc/err_syntax25.error, test/btyacc/err_syntax26.error, test/btyacc/err_syntax27.error, test/btyacc/err_syntax3.error, test/btyacc/err_syntax4.error, test/btyacc/err_syntax5.error, test/btyacc/err_syntax6.error, test/btyacc/err_syntax7.error, test/btyacc/err_syntax7a.error, test/btyacc/err_syntax7b.error, test/btya cc/err_syntax8.error, test/btyacc/err_syntax8a.error, test/btyacc/err_syntax9.error, test/yacc/err_syntax1.error, test/yacc/err_syntax10.error, test/yacc/err_syntax11.error, test/yacc/err_syntax12.error, test/yacc/err_syntax13.error, test/yacc/err_syntax14.error, test/yacc/err_syntax15.error, test/yacc/err_syntax16.error, test/yacc/err_syntax17.error, test/yacc/err_syntax18.error, test/yacc/err_syntax19.error, test/yacc/err_syntax2.error, test/yacc/err_syntax21.error, test/yacc/err_syntax22.error, test/yacc/err_syntax23.error, test/yacc/err_syntax24.error, test/yacc/err_syntax25.error, test/yacc/err_syntax26.error, test/yacc/err_syntax27.error, test/yacc/err_syntax3.error, test/yacc/err_syntax4.error, test/yacc/err_syntax5.error, test/yacc/err_syntax6.error, test/yacc/err_syntax7.error, test/yacc/err_syntax7a.error, test/yacc/err_syntax7b.error, test/yacc/err_syntax8.error, test/yacc/err_syntax8a.error, test/yacc/err_syntax9.error: + regen + +2014-05-27 Tom.Shields + + * main.c: remove obsolete -D option from usage message + +2014-05-27 Thomas E. Dickey + + * VERSION, package/byacc.spec, package/debian/changelog, test/yacc/big_b.output, test/yacc/big_l.output, test/yacc/help.output, test/yacc/no_b_opt.output, test/yacc/no_output2.output, test/yacc/no_p_opt.output, test/yacc/nostdin.output: + bump + 2014-04-22 Thomas E. Dickey * mstring.c: Modified: vendor/byacc/dist/MANIFEST ============================================================================== --- vendor/byacc/dist/MANIFEST Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/MANIFEST Thu Jul 17 21:58:22 2014 (r268809) @@ -1,4 +1,4 @@ -MANIFEST for byacc-20140422, version t20140422 +MANIFEST for byacc-20140715, version t20140715 -------------------------------------------------------------------------------- MANIFEST this file ACKNOWLEDGEMENTS original version of byacc - 1993 Modified: vendor/byacc/dist/VERSION ============================================================================== --- vendor/byacc/dist/VERSION Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/VERSION Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -20140422 +20140715 Modified: vendor/byacc/dist/aclocal.m4 ============================================================================== --- vendor/byacc/dist/aclocal.m4 Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/aclocal.m4 Thu Jul 17 21:58:22 2014 (r268809) @@ -1,4 +1,4 @@ -dnl $Id: aclocal.m4,v 1.34 2014/04/06 19:16:18 tom Exp $ +dnl $Id: aclocal.m4,v 1.35 2014/07/15 19:38:05 tom Exp $ dnl Macros for byacc configure script (Thomas E. Dickey) dnl --------------------------------------------------------------------------- dnl Copyright 2004-2013,2014 Thomas E. Dickey @@ -28,7 +28,7 @@ dnl sale, use or other dealings in this dnl authorization. dnl --------------------------------------------------------------------------- dnl --------------------------------------------------------------------------- -dnl CF_ACVERSION_CHECK version: 4 updated: 2013/03/04 19:52:56 +dnl CF_ACVERSION_CHECK version: 5 updated: 2014/06/04 19:11:49 dnl ------------------ dnl Conditionally generate script according to whether we're using a given autoconf. dnl @@ -37,7 +37,7 @@ dnl $2 = code to use if AC_ACVERSION is dnl $3 = code to use if AC_ACVERSION is older than $1. define([CF_ACVERSION_CHECK], [ -ifdef([AC_ACVERSION], ,[m4_copy([m4_PACKAGE_VERSION],[AC_ACVERSION])])dnl +ifdef([AC_ACVERSION], ,[ifdef([AC_AUTOCONF_VERSION],[m4_copy([AC_AUTOCONF_VERSION],[AC_ACVERSION])],[m4_copy([m4_PACKAGE_VERSION],[AC_ACVERSION])])])dnl ifdef([m4_version_compare], [m4_if(m4_version_compare(m4_defn([AC_ACVERSION]), [$1]), -1, [$3], [$2])], [CF_ACVERSION_COMPARE( @@ -930,7 +930,7 @@ AC_PROG_AWK test -z "$AWK" && AC_MSG_ERROR(No awk program found) ])dnl dnl --------------------------------------------------------------------------- -dnl CF_PROG_CC version: 3 updated: 2012/10/06 15:31:55 +dnl CF_PROG_CC version: 4 updated: 2014/07/12 18:57:58 dnl ---------- dnl standard check for CC, plus followup sanity checks dnl $1 = optional parameter to pass to AC_PROG_CC to specify compiler name @@ -940,7 +940,7 @@ CF_GCC_VERSION CF_ACVERSION_CHECK(2.52, [AC_PROG_CC_STDC], [CF_ANSI_CC_REQD]) -CF_CC_ENV_FLAGS +CF_CC_ENV_FLAGS ])dnl dnl --------------------------------------------------------------------------- dnl CF_PROG_LINT version: 2 updated: 2009/08/12 04:43:14 Modified: vendor/byacc/dist/main.c ============================================================================== --- vendor/byacc/dist/main.c Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/main.c Thu Jul 17 21:58:22 2014 (r268809) @@ -1,4 +1,4 @@ -/* $Id: main.c,v 1.50 2014/04/22 23:34:47 tom Exp $ */ +/* $Id: main.c,v 1.51 2014/05/28 02:01:55 Tom.Shields Exp $ */ #include #ifndef _WIN32 @@ -208,7 +208,6 @@ usage(void) ," -b file_prefix set filename prefix (default \"y.\")" ," -B create a backtracking parser" ," -d write definitions (" DEFINES_SUFFIX ")" - ," -D enable value stack memory reclamation" ," -i write interface (y.tab.i)" ," -g write a graphical description" ," -l suppress #line directives" Modified: vendor/byacc/dist/package/byacc.spec ============================================================================== --- vendor/byacc/dist/package/byacc.spec Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/package/byacc.spec Thu Jul 17 21:58:22 2014 (r268809) @@ -1,8 +1,8 @@ Summary: byacc - public domain Berkeley LALR Yacc parser generator %define AppProgram byacc -%define AppVersion 20140422 +%define AppVersion 20140715 %define UseProgram yacc -# $XTermId: byacc.spec,v 1.20 2014/04/22 08:13:20 tom Exp $ +# $XTermId: byacc.spec,v 1.22 2014/07/15 19:36:54 tom Exp $ Name: %{AppProgram} Version: %{AppVersion} Release: 1 Modified: vendor/byacc/dist/package/debian/changelog ============================================================================== --- vendor/byacc/dist/package/debian/changelog Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/package/debian/changelog Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,15 @@ +byacc (20140715) unstable; urgency=low + + * maintenance updates + + -- Thomas E. Dickey Tue, 15 Jul 2014 15:36:54 -0400 + +byacc (20140527) unstable; urgency=low + + * remove obsolete option from usage message + + -- Thomas E. Dickey Tue, 27 May 2014 22:01:55 -0400 + byacc (20140422) unstable; urgency=low * maintenance updates Modified: vendor/byacc/dist/package/mingw-byacc.spec ============================================================================== --- vendor/byacc/dist/package/mingw-byacc.spec Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/package/mingw-byacc.spec Thu Jul 17 21:58:22 2014 (r268809) @@ -1,8 +1,8 @@ Summary: byacc - public domain Berkeley LALR Yacc parser generator %define AppProgram byacc -%define AppVersion 20140422 +%define AppVersion 20140715 %define UseProgram yacc -# $XTermId: mingw-byacc.spec,v 1.3 2014/04/22 08:13:20 tom Exp $ +# $XTermId: mingw-byacc.spec,v 1.4 2014/07/15 19:36:54 tom Exp $ Name: %{AppProgram} Version: %{AppVersion} Release: 1 Modified: vendor/byacc/dist/package/pkgsrc/Makefile ============================================================================== --- vendor/byacc/dist/package/pkgsrc/Makefile Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/package/pkgsrc/Makefile Thu Jul 17 21:58:22 2014 (r268809) @@ -1,7 +1,7 @@ # $NetBSD: Makefile,v 1.9 2008/07/24 17:13:00 tonnerre Exp $ # -DISTNAME= byacc-20140422 +DISTNAME= byacc-20140715 PKGREVISION= 1 CATEGORIES= devel MASTER_SITES= ftp://invisible-island.net/byacc/ Modified: vendor/byacc/dist/test/btyacc/big_b.output ============================================================================== --- vendor/byacc/dist/test/btyacc/big_b.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/big_b.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/big_l.output ============================================================================== --- vendor/byacc/dist/test/btyacc/big_l.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/big_l.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/err_inherit1.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_inherit1.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_inherit1.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 64 of "./test/err_inherit1.y", unterminated argument list +YACC: e - line 64 of "./err_inherit1.y", unterminated argument list namelist($c, $t ^ Modified: vendor/byacc/dist/test/btyacc/err_inherit2.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_inherit2.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_inherit2.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,5 +1,5 @@ -YACC: w - line 64 of "./test/err_inherit2.y", number of arguments of namelist doesn't agree with previous declaration -YACC: w - line 64 of "./test/err_inherit2.y", type of argument 1 to namelist doesn't agree with previous declaration -YACC: e - line 64 of "./test/err_inherit2.y", bad formal argument list +YACC: w - line 64 of "./err_inherit2.y", number of arguments of namelist doesn't agree with previous declaration +YACC: w - line 64 of "./err_inherit2.y", type of argument 1 to namelist doesn't agree with previous declaration +YACC: e - line 64 of "./err_inherit2.y", bad formal argument list namelist($c, $t, extra): namelist NAME ^ Modified: vendor/byacc/dist/test/btyacc/err_inherit3.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_inherit3.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_inherit3.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,23 +1,23 @@ -YACC: w - line 64 of "./test/err_inherit3.y", number of arguments of namelist doesn't agree with previous declaration -YACC: w - line 64 of "./test/err_inherit3.y", wrong number of arguments for namelist +YACC: w - line 64 of "./err_inherit3.y", number of arguments of namelist doesn't agree with previous declaration +YACC: w - line 64 of "./err_inherit3.y", wrong number of arguments for namelist namelist: namelist($c) NAME ^ -YACC: w - line 64 of "./test/err_inherit3.y", unknown argument $c -YACC: w - line 64 of "./test/err_inherit3.y", untyped argument $c -YACC: w - line 65 of "./test/err_inherit3.y", unknown argument $t +YACC: w - line 64 of "./err_inherit3.y", unknown argument $c +YACC: w - line 64 of "./err_inherit3.y", untyped argument $c +YACC: w - line 65 of "./err_inherit3.y", unknown argument $t { $$->s = mksymbol($t, $c, $2); ^ -YACC: w - line 65 of "./test/err_inherit3.y", unknown argument $c +YACC: w - line 65 of "./err_inherit3.y", unknown argument $c { $$->s = mksymbol($t, $c, $2); ^ -YACC: w - line 69 of "./test/err_inherit3.y", unknown argument $t +YACC: w - line 69 of "./err_inherit3.y", unknown argument $t { $$->s = mksymbol($t, $c, $1); ^ -YACC: w - line 69 of "./test/err_inherit3.y", untyped argument $t -YACC: w - line 69 of "./test/err_inherit3.y", unknown argument $c +YACC: w - line 69 of "./err_inherit3.y", untyped argument $t +YACC: w - line 69 of "./err_inherit3.y", unknown argument $c { $$->s = mksymbol($t, $c, $1); ^ -YACC: w - line 69 of "./test/err_inherit3.y", untyped argument $c -YACC: w - line 0 of "./test/err_inherit3.y", start symbol declaration requires arguments +YACC: w - line 69 of "./err_inherit3.y", untyped argument $c +YACC: w - line 0 of "./err_inherit3.y", start symbol declaration requires arguments YACC: 1 rule never reduced YACC: 3 shift/reduce conflicts. Modified: vendor/byacc/dist/test/btyacc/err_inherit4.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_inherit4.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_inherit4.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,13 +1,13 @@ -YACC: w - line 32 of "./test/err_inherit4.y", destructor redeclared +YACC: w - line 32 of "./err_inherit4.y", destructor redeclared %destructor { ^ -YACC: w - line 77 of "./test/err_inherit4.y", wrong number of default arguments for namelist +YACC: w - line 77 of "./err_inherit4.y", wrong number of default arguments for namelist { $$ = $1; @$ = @2; } ^ -YACC: w - line 77 of "./test/err_inherit4.y", wrong type for default argument 2 to namelist +YACC: w - line 77 of "./err_inherit4.y", wrong type for default argument 2 to namelist { $$ = $1; @$ = @2; } ^ -YACC: w - line 77 of "./test/err_inherit4.y", wrong type for default argument 1 to namelist +YACC: w - line 77 of "./err_inherit4.y", wrong type for default argument 1 to namelist { $$ = $1; @$ = @2; } ^ -YACC: w - line 77 of "./test/err_inherit4.y", @2 references beyond the end of the current rule +YACC: w - line 77 of "./err_inherit4.y", @2 references beyond the end of the current rule Modified: vendor/byacc/dist/test/btyacc/err_inherit5.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_inherit5.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_inherit5.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 74 of "./test/err_inherit5.y", illegal @$ or @N reference +YACC: e - line 74 of "./err_inherit5.y", illegal @$ or @N reference Modified: vendor/byacc/dist/test/btyacc/err_syntax1.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax1.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax1.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax1.y", syntax error +YACC: e - line 1 of "./err_syntax1.y", syntax error ?% { ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax10.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax10.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax10.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: w - line 7 of "./test/err_syntax10.y", the type of '(' has been redeclared -YACC: w - line 7 of "./test/err_syntax10.y", the type of '*' has been redeclared -YACC: w - line 7 of "./test/err_syntax10.y", the type of '&' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '(' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '*' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '&' has been redeclared Modified: vendor/byacc/dist/test/btyacc/err_syntax11.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax11.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax11.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 7 of "./test/err_syntax11.y", the precedence of '|' has been redeclared +YACC: w - line 7 of "./err_syntax11.y", the precedence of '|' has been redeclared Modified: vendor/byacc/dist/test/btyacc/err_syntax12.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax12.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax12.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 7 of "./test/err_syntax12.y", the value of text has been redeclared +YACC: w - line 7 of "./err_syntax12.y", the value of text has been redeclared Modified: vendor/byacc/dist/test/btyacc/err_syntax13.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax13.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax13.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 7 of "./test/err_syntax13.y", the start symbol text is a token +YACC: e - line 7 of "./err_syntax13.y", the start symbol text is a token Modified: vendor/byacc/dist/test/btyacc/err_syntax14.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax14.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax14.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,2 +1,2 @@ -YACC: w - line 7 of "./test/err_syntax14.y", the start symbol has been redeclared +YACC: w - line 7 of "./err_syntax14.y", the start symbol has been redeclared YACC: e - the start symbol text2 is undefined Modified: vendor/byacc/dist/test/btyacc/err_syntax15.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax15.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax15.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 9 of "./test/err_syntax15.y", no grammar has been specified +YACC: e - line 9 of "./err_syntax15.y", no grammar has been specified Modified: vendor/byacc/dist/test/btyacc/err_syntax16.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax16.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax16.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 14 of "./test/err_syntax16.y", a token appears on the lhs of a production +YACC: e - line 14 of "./err_syntax16.y", a token appears on the lhs of a production Modified: vendor/byacc/dist/test/btyacc/err_syntax17.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax17.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax17.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 8 of "./test/err_syntax17.y", unterminated action +YACC: e - line 8 of "./err_syntax17.y", unterminated action S: { error ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax18.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax18.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax18.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 9 of "./test/err_syntax18.y", $4 references beyond the end of the current rule +YACC: w - line 9 of "./err_syntax18.y", $4 references beyond the end of the current rule Modified: vendor/byacc/dist/test/btyacc/err_syntax19.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax19.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax19.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 9 of "./test/err_syntax19.y", illegal $-name +YACC: e - line 9 of "./err_syntax19.y", illegal $-name { $$ = $; } ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax2.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax2.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax2.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax2.y", unmatched /* +YACC: e - line 1 of "./err_syntax2.y", unmatched /* %{ /* ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax21.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax21.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax21.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 12 of "./test/err_syntax21.y", $0 is untyped +YACC: e - line 12 of "./err_syntax21.y", $0 is untyped Modified: vendor/byacc/dist/test/btyacc/err_syntax22.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax22.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax22.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 17 of "./test/err_syntax22.y", $2 (recur) is untyped +YACC: e - line 17 of "./err_syntax22.y", $2 (recur) is untyped Modified: vendor/byacc/dist/test/btyacc/err_syntax23.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax23.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax23.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 18 of "./test/err_syntax23.y", $$ is untyped +YACC: e - line 18 of "./err_syntax23.y", $$ is untyped Modified: vendor/byacc/dist/test/btyacc/err_syntax24.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax24.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax24.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,2 +1,2 @@ -YACC: w - line 21 of "./test/err_syntax24.y", the default action assigns an undefined value to $$ -YACC: e - line 22 of "./test/err_syntax24.y", $$ is untyped +YACC: w - line 21 of "./err_syntax24.y", the default action assigns an undefined value to $$ +YACC: e - line 22 of "./err_syntax24.y", $$ is untyped Modified: vendor/byacc/dist/test/btyacc/err_syntax25.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax25.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax25.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 11 of "./test/err_syntax25.y", too many %union declarations +YACC: e - line 11 of "./err_syntax25.y", too many %union declarations %union { ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax26.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax26.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax26.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax26.y", unexpected end-of-file +YACC: e - line 6 of "./err_syntax26.y", unexpected end-of-file Modified: vendor/byacc/dist/test/btyacc/err_syntax27.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax27.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax27.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 3 of "./test/err_syntax27.y", missing '}' +YACC: e - line 3 of "./err_syntax27.y", missing '}' Modified: vendor/byacc/dist/test/btyacc/err_syntax3.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax3.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax3.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax3.y", unterminated string +YACC: e - line 6 of "./err_syntax3.y", unterminated string %token '(' '*' '& ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax4.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax4.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax4.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax4.y", unmatched %{ +YACC: e - line 1 of "./err_syntax4.y", unmatched %{ %{ ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax5.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax5.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax5.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax5.y", unterminated %union declaration +YACC: e - line 6 of "./err_syntax5.y", unterminated %union declaration %union { ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax6.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax6.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax6.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax6.y", illegal tag +YACC: e - line 6 of "./err_syntax6.y", illegal tag %token '\777' ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax7a.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax7a.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax7a.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax7a.y", illegal character +YACC: e - line 6 of "./err_syntax7a.y", illegal character %token '\xfff' ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax7b.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax7b.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax7b.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax7b.y", illegal character +YACC: e - line 6 of "./err_syntax7b.y", illegal character %token '\x.' ^ Modified: vendor/byacc/dist/test/btyacc/err_syntax8.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax8.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax8.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax8.y", illegal use of reserved symbol . +YACC: e - line 6 of "./err_syntax8.y", illegal use of reserved symbol . Modified: vendor/byacc/dist/test/btyacc/err_syntax8a.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax8a.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax8a.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax8a.y", illegal use of reserved symbol $$123 +YACC: e - line 6 of "./err_syntax8a.y", illegal use of reserved symbol $$123 Modified: vendor/byacc/dist/test/btyacc/err_syntax9.error ============================================================================== --- vendor/byacc/dist/test/btyacc/err_syntax9.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/err_syntax9.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 7 of "./test/err_syntax9.y", the start symbol text cannot be declared to be a token +YACC: e - line 7 of "./err_syntax9.y", the start symbol text cannot be declared to be a token Modified: vendor/byacc/dist/test/btyacc/help.output ============================================================================== --- vendor/byacc/dist/test/btyacc/help.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/help.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/no_b_opt.output ============================================================================== --- vendor/byacc/dist/test/btyacc/no_b_opt.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/no_b_opt.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/no_output2.output ============================================================================== --- vendor/byacc/dist/test/btyacc/no_output2.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/no_output2.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/no_p_opt.output ============================================================================== --- vendor/byacc/dist/test/btyacc/no_p_opt.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/no_p_opt.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/btyacc/nostdin.output ============================================================================== --- vendor/byacc/dist/test/btyacc/nostdin.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/btyacc/nostdin.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/run_test.sh ============================================================================== --- vendor/byacc/dist/test/run_test.sh Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/run_test.sh Thu Jul 17 21:58:22 2014 (r268809) @@ -1,7 +1,9 @@ #!/bin/sh -# $Id: run_test.sh,v 1.22 2014/04/09 11:00:45 tom Exp $ +# $Id: run_test.sh,v 1.24 2014/07/15 19:21:10 tom Exp $ # vi:ts=4 sw=4: +errors=0 + # NEW is the file created by the testcase # REF is the reference file against which to compare test_diffs() { @@ -11,13 +13,15 @@ test_diffs() { if test ! -f $CMP then echo "...not found $CMP" + errors=1 else sed -e s,$NEW,$REF, \ - -e "s%$YACC%YACC%" \ + -e "s%$YACC_escaped%YACC%" \ -e '/YYPATCH/s/[0-9][0-9]*/"yyyymmdd"/' \ -e '/#define YYPATCH/s/PATCH/CHECK/' \ -e 's,#line \([1-9][0-9]*\) "'$REF_DIR'/,#line \1 ",' \ -e 's,#line \([1-9][0-9]*\) "'$TEST_DIR'/,#line \1 ",' \ + -e 's,\(YACC:.* line [0-9][0-9]* of "\)'$TEST_DIR/',\1./,' \ < $CMP >$tmpfile \ && mv $tmpfile $CMP if test ! -f $REF @@ -31,6 +35,7 @@ test_diffs() { else echo "...diff $REF" diff -u $REF $CMP + errors=1 fi fi } @@ -62,10 +67,11 @@ else fi YACC=$PROG_DIR/yacc +YACC_escaped=`echo "$PROG_DIR/yacc" | sed -e 's/\./\\\./g'` tmpfile=temp$$ -ifBTYACC=`fgrep -l 'define YYBTYACC' config.h > /dev/null; test $? != 0; echo $?` +ifBTYACC=`fgrep -l 'define YYBTYACC' $PROG_DIR/config.h > /dev/null; test $? != 0; echo $?` if test $ifBTYACC = 0; then REF_DIR=${TEST_DIR}/yacc @@ -236,3 +242,5 @@ do ;; esac done + +exit $errors Modified: vendor/byacc/dist/test/yacc/big_b.output ============================================================================== --- vendor/byacc/dist/test/yacc/big_b.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/big_b.output Thu Jul 17 21:58:22 2014 (r268809) @@ -5,7 +5,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/big_l.output ============================================================================== --- vendor/byacc/dist/test/yacc/big_l.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/big_l.output Thu Jul 17 21:58:22 2014 (r268809) @@ -5,7 +5,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/err_syntax1.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax1.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax1.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax1.y", syntax error +YACC: e - line 1 of "./err_syntax1.y", syntax error ?% { ^ Modified: vendor/byacc/dist/test/yacc/err_syntax10.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax10.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax10.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: w - line 7 of "./test/err_syntax10.y", the type of '(' has been redeclared -YACC: w - line 7 of "./test/err_syntax10.y", the type of '*' has been redeclared -YACC: w - line 7 of "./test/err_syntax10.y", the type of '&' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '(' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '*' has been redeclared +YACC: w - line 7 of "./err_syntax10.y", the type of '&' has been redeclared Modified: vendor/byacc/dist/test/yacc/err_syntax11.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax11.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax11.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 7 of "./test/err_syntax11.y", the precedence of '|' has been redeclared +YACC: w - line 7 of "./err_syntax11.y", the precedence of '|' has been redeclared Modified: vendor/byacc/dist/test/yacc/err_syntax12.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax12.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax12.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 7 of "./test/err_syntax12.y", the value of text has been redeclared +YACC: w - line 7 of "./err_syntax12.y", the value of text has been redeclared Modified: vendor/byacc/dist/test/yacc/err_syntax13.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax13.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax13.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 7 of "./test/err_syntax13.y", the start symbol text is a token +YACC: e - line 7 of "./err_syntax13.y", the start symbol text is a token Modified: vendor/byacc/dist/test/yacc/err_syntax14.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax14.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax14.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,2 +1,2 @@ -YACC: w - line 7 of "./test/err_syntax14.y", the start symbol has been redeclared +YACC: w - line 7 of "./err_syntax14.y", the start symbol has been redeclared YACC: e - the start symbol text2 is undefined Modified: vendor/byacc/dist/test/yacc/err_syntax15.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax15.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax15.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 9 of "./test/err_syntax15.y", no grammar has been specified +YACC: e - line 9 of "./err_syntax15.y", no grammar has been specified Modified: vendor/byacc/dist/test/yacc/err_syntax16.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax16.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax16.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 14 of "./test/err_syntax16.y", a token appears on the lhs of a production +YACC: e - line 14 of "./err_syntax16.y", a token appears on the lhs of a production Modified: vendor/byacc/dist/test/yacc/err_syntax17.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax17.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax17.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 8 of "./test/err_syntax17.y", unterminated action +YACC: e - line 8 of "./err_syntax17.y", unterminated action S: { error ^ Modified: vendor/byacc/dist/test/yacc/err_syntax18.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax18.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax18.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: w - line 9 of "./test/err_syntax18.y", $4 references beyond the end of the current rule +YACC: w - line 9 of "./err_syntax18.y", $4 references beyond the end of the current rule Modified: vendor/byacc/dist/test/yacc/err_syntax19.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax19.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax19.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 9 of "./test/err_syntax19.y", illegal $-name +YACC: e - line 9 of "./err_syntax19.y", illegal $-name { $$ = $; } ^ Modified: vendor/byacc/dist/test/yacc/err_syntax2.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax2.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax2.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax2.y", unmatched /* +YACC: e - line 1 of "./err_syntax2.y", unmatched /* %{ /* ^ Modified: vendor/byacc/dist/test/yacc/err_syntax21.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax21.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax21.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 12 of "./test/err_syntax21.y", $0 is untyped +YACC: e - line 12 of "./err_syntax21.y", $0 is untyped Modified: vendor/byacc/dist/test/yacc/err_syntax22.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax22.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax22.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 17 of "./test/err_syntax22.y", $2 (recur) is untyped +YACC: e - line 17 of "./err_syntax22.y", $2 (recur) is untyped Modified: vendor/byacc/dist/test/yacc/err_syntax23.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax23.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax23.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 18 of "./test/err_syntax23.y", $$ is untyped +YACC: e - line 18 of "./err_syntax23.y", $$ is untyped Modified: vendor/byacc/dist/test/yacc/err_syntax24.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax24.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax24.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,2 +1,2 @@ -YACC: w - line 21 of "./test/err_syntax24.y", the default action assigns an undefined value to $$ -YACC: e - line 22 of "./test/err_syntax24.y", $$ is untyped +YACC: w - line 21 of "./err_syntax24.y", the default action assigns an undefined value to $$ +YACC: e - line 22 of "./err_syntax24.y", $$ is untyped Modified: vendor/byacc/dist/test/yacc/err_syntax25.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax25.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax25.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 11 of "./test/err_syntax25.y", too many %union declarations +YACC: e - line 11 of "./err_syntax25.y", too many %union declarations %union { ^ Modified: vendor/byacc/dist/test/yacc/err_syntax26.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax26.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax26.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax26.y", unexpected end-of-file +YACC: e - line 6 of "./err_syntax26.y", unexpected end-of-file Modified: vendor/byacc/dist/test/yacc/err_syntax27.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax27.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax27.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 3 of "./test/err_syntax27.y", missing '}' +YACC: e - line 3 of "./err_syntax27.y", missing '}' Modified: vendor/byacc/dist/test/yacc/err_syntax3.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax3.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax3.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax3.y", unterminated string +YACC: e - line 6 of "./err_syntax3.y", unterminated string %token '(' '*' '& ^ Modified: vendor/byacc/dist/test/yacc/err_syntax4.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax4.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax4.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 1 of "./test/err_syntax4.y", unmatched %{ +YACC: e - line 1 of "./err_syntax4.y", unmatched %{ %{ ^ Modified: vendor/byacc/dist/test/yacc/err_syntax5.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax5.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax5.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax5.y", unterminated %union declaration +YACC: e - line 6 of "./err_syntax5.y", unterminated %union declaration %union { ^ Modified: vendor/byacc/dist/test/yacc/err_syntax6.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax6.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax6.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax6.y", illegal tag +YACC: e - line 6 of "./err_syntax6.y", illegal tag %token '\777' ^ Modified: vendor/byacc/dist/test/yacc/err_syntax7a.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax7a.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax7a.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax7a.y", illegal character +YACC: e - line 6 of "./err_syntax7a.y", illegal character %token '\xfff' ^ Modified: vendor/byacc/dist/test/yacc/err_syntax7b.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax7b.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax7b.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1,3 +1,3 @@ -YACC: e - line 6 of "./test/err_syntax7b.y", illegal character +YACC: e - line 6 of "./err_syntax7b.y", illegal character %token '\x.' ^ Modified: vendor/byacc/dist/test/yacc/err_syntax8.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax8.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax8.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax8.y", illegal use of reserved symbol . +YACC: e - line 6 of "./err_syntax8.y", illegal use of reserved symbol . Modified: vendor/byacc/dist/test/yacc/err_syntax8a.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax8a.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax8a.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 6 of "./test/err_syntax8a.y", illegal use of reserved symbol $$123 +YACC: e - line 6 of "./err_syntax8a.y", illegal use of reserved symbol $$123 Modified: vendor/byacc/dist/test/yacc/err_syntax9.error ============================================================================== --- vendor/byacc/dist/test/yacc/err_syntax9.error Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/err_syntax9.error Thu Jul 17 21:58:22 2014 (r268809) @@ -1 +1 @@ -YACC: e - line 7 of "./test/err_syntax9.y", the start symbol text cannot be declared to be a token +YACC: e - line 7 of "./err_syntax9.y", the start symbol text cannot be declared to be a token Modified: vendor/byacc/dist/test/yacc/help.output ============================================================================== --- vendor/byacc/dist/test/yacc/help.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/help.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/no_b_opt.output ============================================================================== --- vendor/byacc/dist/test/yacc/no_b_opt.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/no_b_opt.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/no_output2.output ============================================================================== --- vendor/byacc/dist/test/yacc/no_output2.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/no_output2.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/no_p_opt.output ============================================================================== --- vendor/byacc/dist/test/yacc/no_p_opt.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/no_p_opt.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives Modified: vendor/byacc/dist/test/yacc/nostdin.output ============================================================================== --- vendor/byacc/dist/test/yacc/nostdin.output Thu Jul 17 21:16:52 2014 (r268808) +++ vendor/byacc/dist/test/yacc/nostdin.output Thu Jul 17 21:58:22 2014 (r268809) @@ -4,7 +4,6 @@ Options: -b file_prefix set filename prefix (default "y.") -B create a backtracking parser -d write definitions (.tab.h) - -D enable value stack memory reclamation -i write interface (y.tab.i) -g write a graphical description -l suppress #line directives From owner-svn-src-vendor@FreeBSD.ORG Thu Jul 17 21:59:14 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33382A7D; Thu, 17 Jul 2014 21:59:14 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0738025C5; Thu, 17 Jul 2014 21:59:14 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6HLxDM8021462; Thu, 17 Jul 2014 21:59:13 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6HLxD7T021461; Thu, 17 Jul 2014 21:59:13 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407172159.s6HLxD7T021461@svn.freebsd.org> From: Baptiste Daroussin Date: Thu, 17 Jul 2014 21:59:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268810 - vendor/byacc/20140715 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2014 21:59:14 -0000 Author: bapt Date: Thu Jul 17 21:59:13 2014 New Revision: 268810 URL: http://svnweb.freebsd.org/changeset/base/268810 Log: tag import of byacc 20140715 Added: vendor/byacc/20140715/ - copied from r268809, vendor/byacc/dist/ From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 06:49:20 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19FD4A38; Fri, 18 Jul 2014 06:49:20 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 047062E21; Fri, 18 Jul 2014 06:49:20 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6I6nJig075949; Fri, 18 Jul 2014 06:49:19 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6I6nIVn075940; Fri, 18 Jul 2014 06:49:18 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407180649.s6I6nIVn075940@svn.freebsd.org> From: Baptiste Daroussin Date: Fri, 18 Jul 2014 06:49:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268828 - in vendor/libucl/dist: . doc include src tests uthash X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 06:49:20 -0000 Author: bapt Date: Fri Jul 18 06:49:18 2014 New Revision: 268828 URL: http://svnweb.freebsd.org/changeset/base/268828 Log: Import libucl 2014-07-16 Added: vendor/libucl/dist/ChangeLog.md vendor/libucl/dist/src/ucl_emitter_streamline.c (contents, props changed) vendor/libucl/dist/src/ucl_emitter_utils.c (contents, props changed) vendor/libucl/dist/tests/streamline.res vendor/libucl/dist/tests/streamline.test (contents, props changed) vendor/libucl/dist/tests/test_streamline.c (contents, props changed) Modified: vendor/libucl/dist/configure.ac vendor/libucl/dist/doc/api.md vendor/libucl/dist/doc/libucl.3 vendor/libucl/dist/doc/pandoc.template vendor/libucl/dist/include/ucl.h vendor/libucl/dist/src/Makefile.am vendor/libucl/dist/src/ucl_emitter.c vendor/libucl/dist/src/ucl_internal.h vendor/libucl/dist/src/ucl_parser.c vendor/libucl/dist/tests/Makefile.am vendor/libucl/dist/tests/test_basic.c vendor/libucl/dist/uthash/utstring.h Added: vendor/libucl/dist/ChangeLog.md ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/dist/ChangeLog.md Fri Jul 18 06:49:18 2014 (r268828) @@ -0,0 +1,6 @@ +# Version history + +## Libucl 0.5 + +- Streamline emitter has been added, so it is now possible to output partial `ucl` objects +- Emitter now is more flexible due to emitter_context structure Modified: vendor/libucl/dist/configure.ac ============================================================================== --- vendor/libucl/dist/configure.ac Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/configure.ac Fri Jul 18 06:49:18 2014 (r268828) @@ -1,7 +1,7 @@ m4_define([maj_ver], [0]) -m4_define([med_ver], [4]) -m4_define([min_ver], [1]) -m4_define([so_version], [1:0:0]) +m4_define([med_ver], [5]) +m4_define([min_ver], [0]) +m4_define([so_version], [2:0:0]) m4_define([ucl_version], [maj_ver.med_ver.min_ver]) AC_INIT([libucl],[ucl_version],[https://github.com/vstakhov/libucl],[libucl]) Modified: vendor/libucl/dist/doc/api.md ============================================================================== --- vendor/libucl/dist/doc/api.md Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/doc/api.md Fri Jul 18 06:49:18 2014 (r268828) @@ -52,15 +52,15 @@ Used to parse `ucl` files and provide in ### Emitting functions Convert `ucl` objects to some textual or binary representation. Currently, libucl supports the following exports: -- `JSON` - valid json format (can possibly loose some original data, such as implicit arrays) -- `Config` - human-readable configuration format (losseless) +- `JSON` - valid json format (can possibly lose some original data, such as implicit arrays) +- `Config` - human-readable configuration format (lossless) - `YAML` - embedded yaml format (has the same limitations as `json` output) ### Conversion functions Help to convert `ucl` objects to C types. These functions are used to convert `ucl_object_t` to C primitive types, such as numbers, strings or boolean values. ### Generation functions -Allow creating of `ucl` objects from C types and creating of complex `ucl` objects, such as hashes or arrays from primitive `ucl` objects, such as numbers or strings. +Allow creation of `ucl` objects from C types and creating of complex `ucl` objects, such as hashes or arrays from primitive `ucl` objects, such as numbers or strings. ### Iteration functions Iterate over `ucl` complex objects or over a chain of values, for example when a key in an object has multiple values (that can be treated as implicit array or implicit consolidation). @@ -148,7 +148,7 @@ bool ucl_parser_add_file (struct ucl_par const char *filename); ~~~ -Load file `filename` and parse it with the specified `parser`. This function uses `mmap` call to load file, therefore, it should not be `shrinked` during parsing. Otherwise, `libucl` can cause memory corruption and terminate the calling application. This function is also used by the internal handler of `include` macro, hence, this macro has the same limitation. +Load file `filename` and parse it with the specified `parser`. This function uses `mmap` call to load file, therefore, it should not be `shrunk` during parsing. Otherwise, `libucl` can cause memory corruption and terminate the calling application. This function is also used by the internal handler of `include` macro, hence, this macro has the same limitation. ### ucl_parser_get_object @@ -225,7 +225,7 @@ ucl_parser_add_chunk (parser, inbuf, r); fclose (in); if (ucl_parser_get_error (parser)) { - printf ("Error occured: %s\n", ucl_parser_get_error (parser)); + printf ("Error occurred: %s\n", ucl_parser_get_error (parser)); ret = 1; } else { @@ -251,7 +251,7 @@ Libucl can transform UCL objects to a nu - compact yaml: `UCL_EMIT_YAML` - compact YAML output Moreover, libucl API allows to select a custom set of emitting functions allowing -efficent and zero-copy output of libucl objects. Libucl uses the following structure to support this feature: +efficient and zero-copy output of libucl objects. Libucl uses the following structure to support this feature: ~~~C struct ucl_emitter_functions { @@ -298,12 +298,12 @@ This function is similar to the previous # Conversion functions -Conversion functions are used to convert UCL objects to primitive types, such as strings, numbers or boolean values. There are two types of conversion functions: +Conversion functions are used to convert UCL objects to primitive types, such as strings, numbers, or boolean values. There are two types of conversion functions: - safe: try to convert an ucl object to a primitive type and fail if such a conversion is not possible - unsafe: return primitive type without additional checks, if the object cannot be converted then some reasonable default is returned (NULL for strings and 0 for numbers) -Also there is a single `ucl_object_tostring_forced` function that converts any UCL object (including compound types - arrays and objects) to a string representation. For compound and numeric types this function performs emitting to a compact json format actually. +Also there is a single `ucl_object_tostring_forced` function that converts any UCL object (including compound types - arrays and objects) to a string representation. For objects, arrays, booleans and numeric types this function performs emitting to a compact json format actually. Here is a list of all conversion functions: @@ -311,14 +311,14 @@ Here is a list of all conversion functio - `ucl_object_todouble` - returns `double` of UCL object - `ucl_object_toboolean` - returns `bool` of UCL object - `ucl_object_tostring` - returns `const char *` of UCL object (this string is NULL terminated) -- `ucl_object_tolstring` - returns `const char *` and `size_t` len of UCL object (string can be not NULL terminated) +- `ucl_object_tolstring` - returns `const char *` and `size_t` len of UCL object (string does not need to be NULL terminated) - `ucl_object_tostring_forced` - returns string representation of any UCL object Strings returned by these pointers are associated with the UCL object and exist over its lifetime. A caller should not free this memory. # Generation functions -It is possible to generate UCL objects from C primitive types. Moreover, libucl permits to create and modify complex UCL objects, such as arrays or associative objects. +It is possible to generate UCL objects from C primitive types. Moreover, libucl allows creation and modifying complex UCL objects, such as arrays or associative objects. ## ucl_object_new ~~~C @@ -350,8 +350,8 @@ Libucl provides the functions similar to - `ucl_object_fromint` - converts `int64_t` to UCL object - `ucl_object_fromdouble` - converts `double` to UCL object - `ucl_object_fromboolean` - converts `bool` to UCL object -- `ucl_object_fromstring` - converts `const char *` to UCL object (this string is NULL terminated) -- `ucl_object_fromlstring` - converts `const char *` and `size_t` len to UCL object (string can be not NULL terminated) +- `ucl_object_fromstring` - converts `const char *` to UCL object (this string should be NULL terminated) +- `ucl_object_fromlstring` - converts `const char *` and `size_t` len to UCL object (string does not need to be NULL terminated) Also there is a function to generate UCL object from a string performing various parsing or conversion operations called `ucl_object_fromstring_common`. @@ -361,7 +361,7 @@ ucl_object_t * ucl_object_fromstring_com size_t len, enum ucl_string_flags flags) ~~~ -This function is used to convert a string `str` of size `len` to an UCL objects applying `flags` conversions. If `len` is equal to zero then a `str` is assumed as NULL-terminated. This function supports the following flags (a set of flags can be specified using logical `OR` operation): +This function is used to convert a string `str` of size `len` to a UCL object applying `flags` conversions. If `len` is equal to zero then a `str` is assumed as NULL-terminated. This function supports the following flags (a set of flags can be specified using logical `OR` operation): - `UCL_STRING_ESCAPE` - perform JSON escape - `UCL_STRING_TRIM` - trim leading and trailing whitespaces @@ -385,7 +385,7 @@ const ucl_object_t* ucl_iterate_object ( ucl_object_iter_t *iter, bool expand_values); ~~~ -This function accept opaque iterator pointer `iter`. In the first call this iterator *must* be initialized to `NULL`. Iterator is changed by this function call. `ucl_iterate_object` returns the next UCL object in the compound object `obj` or `NULL` if all objects have been iterated. The reference count of the object returned is not increased, so a caller should not unref the object or modify its content (e.g. by inserting to another compound object). The object `obj` should not be changed during the iteration process as well. `expand_values` flag speicifies whether `ucl_iterate_object` should expand keys with multiple values. The general rule is that if you need to iterate throught the *object* or *explicit array*, then you always need to set this flag to `true`. However, if you get some key in the object and want to extract all its values then you should set `expand_values` to `false`. Mixing of iteration types are not permitted since the iterator is set according to the iteration type and cannot be reused. Here is an example of iteration over the objects using libucl API (assuming that `top` is `UCL_OBJECT` in this example): +This function accepts opaque iterator pointer `iter`. In the first call this iterator *must* be initialized to `NULL`. Iterator is changed by this function call. `ucl_iterate_object` returns the next UCL object in the compound object `obj` or `NULL` if all objects have been iterated. The reference count of the object returned is not increased, so a caller should not unref the object or modify its content (e.g. by inserting to another compound object). The object `obj` should not be changed during the iteration process as well. `expand_values` flag speicifies whether `ucl_iterate_object` should expand keys with multiple values. The general rule is that if you need to iterate through the *object* or *explicit array*, then you always need to set this flag to `true`. However, if you get some key in the object and want to extract all its values then you should set `expand_values` to `false`. Mixing of iteration types is not permitted since the iterator is set according to the iteration t ype and cannot be reused. Here is an example of iteration over the objects using libucl API (assuming that `top` is `UCL_OBJECT` in this example): ~~~C ucl_object_iter_t it = NULL, it_obj = NULL; @@ -412,13 +412,13 @@ bool ucl_object_validate (const ucl_obje const ucl_object_t *obj, struct ucl_schema_error *err); ~~~ -This function uses ucl object `schema`, that must be valid in terms of `json-schema` draft v4, to validate input object `obj`. If this function returns `true` then validation procedure has been succeed. Otherwise, `false` is returned and `err` is set to a specific value. If caller set `err` to NULL then this function does not set any error just returning `false`. Error is the structure defined as following: +This function uses ucl object `schema`, that must be valid in terms of `json-schema` draft v4, to validate input object `obj`. If this function returns `true` then validation procedure has been succeed. Otherwise, `false` is returned and `err` is set to a specific value. If a caller sets `err` to NULL then this function does not set any error just returning `false`. Error is the structure defined as following: ~~~C struct ucl_schema_error { enum ucl_schema_error_code code; /* error code */ char msg[128]; /* error message */ - ucl_object_t *obj; /* object where error occured */ + ucl_object_t *obj; /* object where error occurred */ }; ~~~ @@ -436,4 +436,4 @@ enum ucl_schema_error_code { }; ~~~ -`msg` is a stiring description of an error and `obj` is an object where error has been occurred. Error object is not allocated by libucl, so there is no need to free it after validation (a static object should thus be used). \ No newline at end of file +`msg` is a string description of an error and `obj` is an object where error has occurred. Error object is not allocated by libucl, so there is no need to free it after validation (a static object should thus be used). Modified: vendor/libucl/dist/doc/libucl.3 ============================================================================== --- vendor/libucl/dist/doc/libucl.3 Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/doc/libucl.3 Fri Jul 18 06:49:18 2014 (r268828) @@ -1,4 +1,4 @@ -.TH LIBUCL 5 "March 20, 2014" "Libucl manual" +.TH "LIBUCL" "3" "July 26, 2014" "Libucl manual" "" .SH NAME .PP \f[B]ucl_parser_new\f[], \f[B]ucl_parser_register_macro\f[], @@ -32,10 +32,10 @@ In future, this limitation can be remove Convert \f[C]ucl\f[] objects to some textual or binary representation. Currently, libucl supports the following exports: .IP \[bu] 2 -\f[C]JSON\f[] \- valid json format (can possibly loose some original +\f[C]JSON\f[] \- valid json format (can possibly lose some original data, such as implicit arrays) .IP \[bu] 2 -\f[C]Config\f[] \- human\-readable configuration format (losseless) +\f[C]Config\f[] \- human\-readable configuration format (lossless) .IP \[bu] 2 \f[C]YAML\f[] \- embedded yaml format (has the same limitations as \f[C]json\f[] output) @@ -46,7 +46,7 @@ These functions are used to convert \f[C types, such as numbers, strings or boolean values. .SS Generation functions .PP -Allow creating of \f[C]ucl\f[] objects from C types and creating of +Allow creation of \f[C]ucl\f[] objects from C types and creating of complex \f[C]ucl\f[] objects, such as hashes or arrays from primitive \f[C]ucl\f[] objects, such as numbers or strings. .SS Iteration functions @@ -175,7 +175,7 @@ bool\ ucl_parser_add_file\ (struct\ ucl_ Load file \f[C]filename\f[] and parse it with the specified \f[C]parser\f[]. This function uses \f[C]mmap\f[] call to load file, therefore, it should -not be \f[C]shrinked\f[] during parsing. +not be \f[C]shrunk\f[] during parsing. Otherwise, \f[C]libucl\f[] can cause memory corruption and terminate the calling application. This function is also used by the internal handler of \f[C]include\f[] @@ -290,7 +290,7 @@ ucl_parser_add_chunk\ (parser,\ inbuf,\ fclose\ (in); if\ (ucl_parser_get_error\ (parser))\ { -\ \ \ \ printf\ ("Error\ occured:\ %s\\n",\ ucl_parser_get_error\ (parser)); +\ \ \ \ printf\ ("Error\ occurred:\ %s\\n",\ ucl_parser_get_error\ (parser)); \ \ \ \ ret\ =\ 1; } else\ { @@ -323,7 +323,7 @@ newlines and spaces compact yaml: \f[C]UCL_EMIT_YAML\f[] \- compact YAML output .PP Moreover, libucl API allows to select a custom set of emitting functions -allowing efficent and zero\-copy output of libucl objects. +allowing efficient and zero\-copy output of libucl objects. Libucl uses the following structure to support this feature: .IP .nf @@ -390,7 +390,7 @@ emitters (including C++ ones, for exampl .SH CONVERSION FUNCTIONS .PP Conversion functions are used to convert UCL objects to primitive types, -such as strings, numbers or boolean values. +such as strings, numbers, or boolean values. There are two types of conversion functions: .IP \[bu] 2 safe: try to convert an ucl object to a primitive type and fail if such @@ -403,8 +403,8 @@ strings and 0 for numbers) Also there is a single \f[C]ucl_object_tostring_forced\f[] function that converts any UCL object (including compound types \- arrays and objects) to a string representation. -For compound and numeric types this function performs emitting to a -compact json format actually. +For objects, arrays, booleans and numeric types this function performs +emitting to a compact json format actually. .PP Here is a list of all conversion functions: .IP \[bu] 2 @@ -418,7 +418,8 @@ Here is a list of all conversion functio object (this string is NULL terminated) .IP \[bu] 2 \f[C]ucl_object_tolstring\f[] \- returns \f[C]const\ char\ *\f[] and -\f[C]size_t\f[] len of UCL object (string can be not NULL terminated) +\f[C]size_t\f[] len of UCL object (string does not need to be NULL +terminated) .IP \[bu] 2 \f[C]ucl_object_tostring_forced\f[] \- returns string representation of any UCL object @@ -429,7 +430,7 @@ A caller should not free this memory. .SH GENERATION FUNCTIONS .PP It is possible to generate UCL objects from C primitive types. -Moreover, libucl permits to create and modify complex UCL objects, such +Moreover, libucl allows creation and modifying complex UCL objects, such as arrays or associative objects. .SS ucl_object_new .IP @@ -467,10 +468,10 @@ converts \f[C]int64_t\f[] to UCL object \f[C]ucl_object_fromdouble\f[] \- converts \f[C]double\f[] to UCL object \- \f[C]ucl_object_fromboolean\f[] \- converts \f[C]bool\f[] to UCL object \- \f[C]ucl_object_fromstring\f[] \- converts -\f[C]const\ char\ *\f[] to UCL object (this string is NULL terminated) -\- \f[C]ucl_object_fromlstring\f[] \- converts \f[C]const\ char\ *\f[] -and \f[C]size_t\f[] len to UCL object (string can be not NULL -terminated) +\f[C]const\ char\ *\f[] to UCL object (this string should be NULL +terminated) \- \f[C]ucl_object_fromlstring\f[] \- converts +\f[C]const\ char\ *\f[] and \f[C]size_t\f[] len to UCL object (string +does not need to be NULL terminated) .PP Also there is a function to generate UCL object from a string performing various parsing or conversion operations called @@ -485,7 +486,7 @@ ucl_object_t\ *\ ucl_object_fromstring_c .fi .PP This function is used to convert a string \f[C]str\f[] of size -\f[C]len\f[] to an UCL objects applying \f[C]flags\f[] conversions. +\f[C]len\f[] to a UCL object applying \f[C]flags\f[] conversions. If \f[C]len\f[] is equal to zero then a \f[C]str\f[] is assumed as NULL\-terminated. This function supports the following flags (a set of flags can be @@ -538,7 +539,7 @@ const\ ucl_object_t*\ ucl_iterate_object \f[] .fi .PP -This function accept opaque iterator pointer \f[C]iter\f[]. +This function accepts opaque iterator pointer \f[C]iter\f[]. In the first call this iterator \f[I]must\f[] be initialized to \f[C]NULL\f[]. Iterator is changed by this function call. @@ -551,12 +552,12 @@ The object \f[C]obj\f[] should not be ch process as well. \f[C]expand_values\f[] flag speicifies whether \f[C]ucl_iterate_object\f[] should expand keys with multiple values. -The general rule is that if you need to iterate throught the +The general rule is that if you need to iterate through the \f[I]object\f[] or \f[I]explicit array\f[], then you always need to set this flag to \f[C]true\f[]. However, if you get some key in the object and want to extract all its values then you should set \f[C]expand_values\f[] to \f[C]false\f[]. -Mixing of iteration types are not permitted since the iterator is set +Mixing of iteration types is not permitted since the iterator is set according to the iteration type and cannot be reused. Here is an example of iteration over the objects using libucl API (assuming that \f[C]top\f[] is \f[C]UCL_OBJECT\f[] in this example): @@ -599,8 +600,8 @@ If this function returns \f[C]true\f[] t been succeed. Otherwise, \f[C]false\f[] is returned and \f[C]err\f[] is set to a specific value. -If caller set \f[C]err\f[] to NULL then this function does not set any -error just returning \f[C]false\f[]. +If a caller sets \f[C]err\f[] to NULL then this function does not set +any error just returning \f[C]false\f[]. Error is the structure defined as following: .IP .nf @@ -608,7 +609,7 @@ Error is the structure defined as follow struct\ ucl_schema_error\ { \ \ \ \ enum\ ucl_schema_error_code\ code;\ \ \ \ /*\ error\ code\ */ \ \ \ \ char\ msg[128];\ \ \ \ \ \ \ \ \ \ \ \ \ \ /*\ error\ message\ */ -\ \ \ \ ucl_object_t\ *obj;\ \ \ \ \ \ \ \ \ \ /*\ object\ where\ error\ occured\ */ +\ \ \ \ ucl_object_t\ *obj;\ \ \ \ \ \ \ \ \ \ /*\ object\ where\ error\ occurred\ */ }; \f[] .fi @@ -629,8 +630,8 @@ enum\ ucl_schema_error_code\ { \f[] .fi .PP -\f[C]msg\f[] is a stiring description of an error and \f[C]obj\f[] is an -object where error has been occurred. +\f[C]msg\f[] is a string description of an error and \f[C]obj\f[] is an +object where error has occurred. Error object is not allocated by libucl, so there is no need to free it after validation (a static object should thus be used). .SH AUTHORS Modified: vendor/libucl/dist/doc/pandoc.template ============================================================================== --- vendor/libucl/dist/doc/pandoc.template Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/doc/pandoc.template Fri Jul 18 06:49:18 2014 (r268828) @@ -1,6 +1,6 @@ -% LIBUCL(5) Libucl manual +% LIBUCL(3) Libucl manual % Vsevolod Stakhov -% March 20, 2014 +% July 26, 2014 # Name Modified: vendor/libucl/dist/include/ucl.h ============================================================================== --- vendor/libucl/dist/include/ucl.h Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/include/ucl.h Fri Jul 18 06:49:18 2014 (r268828) @@ -786,6 +786,7 @@ UCL_EXTERN bool ucl_parser_set_filevars * @{ */ +struct ucl_emitter_context; /** * Structure using for emitter callbacks */ @@ -798,10 +799,49 @@ struct ucl_emitter_functions { int (*ucl_emitter_append_int) (int64_t elt, void *ud); /** Append floating point element */ int (*ucl_emitter_append_double) (double elt, void *ud); + /** Free userdata */ + void (*ucl_emitter_free_func)(void *ud); /** Opaque userdata pointer */ void *ud; }; +struct ucl_emitter_operations { + /** Write a primitive element */ + void (*ucl_emitter_write_elt) (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool first, bool print_key); + /** Start ucl object */ + void (*ucl_emitter_start_object) (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool print_key); + /** End ucl object */ + void (*ucl_emitter_end_object) (struct ucl_emitter_context *ctx, + const ucl_object_t *obj); + /** Start ucl array */ + void (*ucl_emitter_start_array) (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool print_key); + void (*ucl_emitter_end_array) (struct ucl_emitter_context *ctx, + const ucl_object_t *obj); +}; + +/** + * Structure that defines emitter functions + */ +struct ucl_emitter_context { + /** Name of emitter (e.g. json, compact_json) */ + const char *name; + /** Unique id (e.g. UCL_EMIT_JSON for standard emitters */ + int id; + /** A set of output functions */ + const struct ucl_emitter_functions *func; + /** A set of output operations */ + const struct ucl_emitter_operations *ops; + /** Current amount of indent tabs */ + unsigned int ident; + /** Top level object */ + const ucl_object_t *top; + /** The rest of context */ + unsigned char data[1]; +}; + /** * Emit object to a string * @param obj object @@ -817,11 +857,81 @@ UCL_EXTERN unsigned char *ucl_object_emi * @param obj object * @param emit_type if type is #UCL_EMIT_JSON then emit json, if type is * #UCL_EMIT_CONFIG then emit config like object + * @param emitter a set of emitter functions * @return dump of an object (must be freed after using) or NULL in case of error */ UCL_EXTERN bool ucl_object_emit_full (const ucl_object_t *obj, enum ucl_emitter emit_type, struct ucl_emitter_functions *emitter); + +/** + * Start streamlined UCL object emitter + * @param obj top UCL object + * @param emit_type emit type + * @param emitter a set of emitter functions + * @return new streamlined context that should be freed by + * `ucl_object_emit_streamline_finish` + */ +UCL_EXTERN struct ucl_emitter_context* ucl_object_emit_streamline_new ( + const ucl_object_t *obj, enum ucl_emitter emit_type, + struct ucl_emitter_functions *emitter); + +/** + * Start object or array container for the streamlined output + * @param ctx streamlined context + * @param obj container object + */ +UCL_EXTERN void ucl_object_emit_streamline_start_container ( + struct ucl_emitter_context *ctx, const ucl_object_t *obj); +/** + * Add a complete UCL object to streamlined output + * @param ctx streamlined context + * @param obj object to output + */ +UCL_EXTERN void ucl_object_emit_streamline_add_object ( + struct ucl_emitter_context *ctx, const ucl_object_t *obj); +/** + * End previously added container + * @param ctx streamlined context + */ +UCL_EXTERN void ucl_object_emit_streamline_end_container ( + struct ucl_emitter_context *ctx); +/** + * Terminate streamlined container finishing all containers in it + * @param ctx streamlined context + */ +UCL_EXTERN void ucl_object_emit_streamline_finish ( + struct ucl_emitter_context *ctx); + +/** + * Returns functions to emit object to memory + * @param pmem target pointer (should be freed by caller) + * @return emitter functions structure + */ +UCL_EXTERN struct ucl_emitter_functions* ucl_object_emit_memory_funcs ( + void **pmem); + +/** + * Returns functions to emit object to FILE * + * @param fp FILE * object + * @return emitter functions structure + */ +UCL_EXTERN struct ucl_emitter_functions* ucl_object_emit_file_funcs ( + FILE *fp); +/** + * Returns functions to emit object to a file descriptor + * @param fd file descriptor + * @return emitter functions structure + */ +UCL_EXTERN struct ucl_emitter_functions* ucl_object_emit_fd_funcs ( + int fd); + +/** + * Free emitter functions + * @param f pointer to functions + */ +UCL_EXTERN void ucl_object_emit_funcs_free (struct ucl_emitter_functions *f); + /** @} */ /** Modified: vendor/libucl/dist/src/Makefile.am ============================================================================== --- vendor/libucl/dist/src/Makefile.am Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/src/Makefile.am Fri Jul 18 06:49:18 2014 (r268828) @@ -4,6 +4,8 @@ libucl_common_cflags= -I$(top_srcdir)/sr -Wall -W -Wno-unused-parameter -Wno-pointer-sign lib_LTLIBRARIES= libucl.la libucl_la_SOURCES= ucl_emitter.c \ + ucl_emitter_streamline.c \ + ucl_emitter_utils.c \ ucl_hash.c \ ucl_parser.c \ ucl_schema.c \ Modified: vendor/libucl/dist/src/ucl_emitter.c ============================================================================== --- vendor/libucl/dist/src/ucl_emitter.c Fri Jul 18 06:48:02 2014 (r268827) +++ vendor/libucl/dist/src/ucl_emitter.c Fri Jul 18 06:49:18 2014 (r268828) @@ -36,38 +36,55 @@ #endif /** - * @file rcl_emitter.c + * @file ucl_emitter.c * Serialise UCL object to various of output formats */ +static void ucl_emitter_common_elt (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool first, bool print_key, bool compact); + +#define UCL_EMIT_TYPE_OPS(type) \ + static void ucl_emit_ ## type ## _elt (struct ucl_emitter_context *ctx, \ + const ucl_object_t *obj, bool first, bool print_key); \ + static void ucl_emit_ ## type ## _start_obj (struct ucl_emitter_context *ctx, \ + const ucl_object_t *obj, bool print_key); \ + static void ucl_emit_ ## type## _start_array (struct ucl_emitter_context *ctx, \ + const ucl_object_t *obj, bool print_key); \ + static void ucl_emit_ ##type## _end_object (struct ucl_emitter_context *ctx, \ + const ucl_object_t *obj); \ + static void ucl_emit_ ##type## _end_array (struct ucl_emitter_context *ctx, \ + const ucl_object_t *obj) + +/* + * JSON format operations + */ +UCL_EMIT_TYPE_OPS(json); +UCL_EMIT_TYPE_OPS(json_compact); +UCL_EMIT_TYPE_OPS(config); +UCL_EMIT_TYPE_OPS(yaml); + +#define UCL_EMIT_TYPE_CONTENT(type) { \ + .ucl_emitter_write_elt = ucl_emit_ ## type ## _elt, \ + .ucl_emitter_start_object = ucl_emit_ ## type ##_start_obj, \ + .ucl_emitter_start_array = ucl_emit_ ## type ##_start_array, \ + .ucl_emitter_end_object = ucl_emit_ ## type ##_end_object, \ + .ucl_emitter_end_array = ucl_emit_ ## type ##_end_array \ +} + + +const struct ucl_emitter_operations ucl_standartd_emitter_ops[] = { + [UCL_EMIT_JSON] = UCL_EMIT_TYPE_CONTENT(json), + [UCL_EMIT_JSON_COMPACT] = UCL_EMIT_TYPE_CONTENT(json_compact), + [UCL_EMIT_CONFIG] = UCL_EMIT_TYPE_CONTENT(config), + [UCL_EMIT_YAML] = UCL_EMIT_TYPE_CONTENT(yaml) +}; + +/* + * Utility to check whether we need a top object + */ +#define UCL_EMIT_IDENT_TOP_OBJ(ctx, obj) ((ctx)->top != (obj) || \ + ((ctx)->id == UCL_EMIT_JSON_COMPACT || (ctx)->id == UCL_EMIT_JSON)) -static void ucl_obj_write_json (const ucl_object_t *obj, - struct ucl_emitter_functions *func, - unsigned int tabs, - bool start_tabs, - bool compact); -static void ucl_elt_write_json (const ucl_object_t *obj, - struct ucl_emitter_functions *func, - unsigned int tabs, - bool start_tabs, - bool compact); -static void ucl_elt_write_config (const ucl_object_t *obj, - struct ucl_emitter_functions *func, - unsigned int tabs, - bool start_tabs, - bool is_top, - bool expand_array); -static void ucl_elt_write_yaml (const ucl_object_t *obj, - struct ucl_emitter_functions *func, - unsigned int tabs, - bool start_tabs, - bool compact, - bool expand_array); -static void ucl_elt_array_write_yaml (const ucl_object_t *obj, - struct ucl_emitter_functions *func, - unsigned int tabs, - bool start_tabs, - bool is_top); /** * Add tabulation to the output buffer @@ -75,689 +92,358 @@ static void ucl_elt_array_write_yaml (co * @param tabs number of tabs to add */ static inline void -ucl_add_tabs (struct ucl_emitter_functions *func, unsigned int tabs, bool compact) +ucl_add_tabs (const struct ucl_emitter_functions *func, unsigned int tabs, + bool compact) { - if (!compact) { + if (!compact && tabs > 0) { func->ucl_emitter_append_character (' ', tabs * 4, func->ud); } } /** - * Serialise string - * @param str string to emit - * @param buf target buffer + * Print key for the element + * @param ctx + * @param obj */ static void -ucl_elt_string_write_json (const char *str, size_t size, - struct ucl_emitter_functions *func) +ucl_emitter_print_key (bool print_key, struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool compact) { - const char *p = str, *c = str; - size_t len = 0; + const struct ucl_emitter_functions *func = ctx->func; - func->ucl_emitter_append_character ('"', 1, func->ud); - while (size) { - if (ucl_test_character (*p, UCL_CHARACTER_JSON_UNSAFE)) { - if (len > 0) { - func->ucl_emitter_append_len (c, len, func->ud); - } - switch (*p) { - case '\n': - func->ucl_emitter_append_len ("\\n", 2, func->ud); - break; - case '\r': - func->ucl_emitter_append_len ("\\r", 2, func->ud); - break; - case '\b': - func->ucl_emitter_append_len ("\\b", 2, func->ud); - break; - case '\t': - func->ucl_emitter_append_len ("\\t", 2, func->ud); - break; - case '\f': - func->ucl_emitter_append_len ("\\f", 2, func->ud); - break; - case '\\': - func->ucl_emitter_append_len ("\\\\", 2, func->ud); - break; - case '"': - func->ucl_emitter_append_len ("\\\"", 2, func->ud); - break; - } - len = 0; - c = ++p; + if (!print_key) { + return; + } + + if (ctx->id == UCL_EMIT_CONFIG) { + if (obj->flags & UCL_OBJECT_NEED_KEY_ESCAPE) { + ucl_elt_string_write_json (obj->key, obj->keylen, ctx); } else { - p ++; - len ++; + func->ucl_emitter_append_len (obj->key, obj->keylen, func->ud); } - size --; - } - if (len > 0) { - func->ucl_emitter_append_len (c, len, func->ud); - } - func->ucl_emitter_append_character ('"', 1, func->ud); -} -/** - * Write a single object to the buffer - * @param obj object to write - * @param buf target buffer - */ -static void -ucl_elt_obj_write_json (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool compact) -{ - const ucl_object_t *cur; - ucl_hash_iter_t it = NULL; - - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - if (compact) { - func->ucl_emitter_append_character ('{', 1, func->ud); + if (obj->type != UCL_OBJECT && obj->type != UCL_ARRAY) { + func->ucl_emitter_append_len (" = ", 3, func->ud); + } + else { + func->ucl_emitter_append_character (' ', 1, func->ud); + } } else { - func->ucl_emitter_append_len ("{\n", 2, func->ud); - } - while ((cur = ucl_hash_iterate (obj->value.ov, &it))) { - ucl_add_tabs (func, tabs + 1, compact); - if (cur->keylen > 0) { - ucl_elt_string_write_json (cur->key, cur->keylen, func); + if (obj->keylen > 0) { + ucl_elt_string_write_json (obj->key, obj->keylen, ctx); } else { func->ucl_emitter_append_len ("null", 4, func->ud); } + if (compact) { func->ucl_emitter_append_character (':', 1, func->ud); } else { func->ucl_emitter_append_len (": ", 2, func->ud); } - ucl_obj_write_json (cur, func, tabs + 1, false, compact); - if (ucl_hash_iter_has_next (it)) { - if (compact) { - func->ucl_emitter_append_character (',', 1, func->ud); - } - else { - func->ucl_emitter_append_len (",\n", 2, func->ud); - } - } - else if (!compact) { - func->ucl_emitter_append_character ('\n', 1, func->ud); - } } - ucl_add_tabs (func, tabs, compact); - func->ucl_emitter_append_character ('}', 1, func->ud); } -/** - * Write a single array to the buffer - * @param obj array to write - * @param buf target buffer - */ static void -ucl_elt_array_write_json (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool compact) +ucl_emitter_finish_object (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool compact, bool is_array) { - const ucl_object_t *cur = obj; + const struct ucl_emitter_functions *func = ctx->func; - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - if (compact) { - func->ucl_emitter_append_character ('[', 1, func->ud); - } - else { - func->ucl_emitter_append_len ("[\n", 2, func->ud); - } - while (cur) { - ucl_elt_write_json (cur, func, tabs + 1, true, compact); - if (cur->next != NULL) { - if (compact) { - func->ucl_emitter_append_character (',', 1, func->ud); + if (ctx->id == UCL_EMIT_CONFIG && obj != ctx->top) { + if (obj->type != UCL_OBJECT && obj->type != UCL_ARRAY) { + if (!is_array) { + /* Objects are split by ';' */ + func->ucl_emitter_append_len (";\n", 2, func->ud); } else { + /* Use commas for arrays */ func->ucl_emitter_append_len (",\n", 2, func->ud); } } - else if (!compact) { - func->ucl_emitter_append_character ('\n', 1, func->ud); - } - cur = cur->next; - } - ucl_add_tabs (func, tabs, compact); - func->ucl_emitter_append_character (']', 1, func->ud); -} - -/** - * Emit a single element - * @param obj object - * @param buf buffer - */ -static void -ucl_elt_write_json (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool compact) -{ - bool flag; - - switch (obj->type) { - case UCL_INT: - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - func->ucl_emitter_append_int (ucl_object_toint (obj), func->ud); - break; - case UCL_FLOAT: - case UCL_TIME: - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - func->ucl_emitter_append_double (ucl_object_todouble (obj), func->ud); - break; - case UCL_BOOLEAN: - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - flag = ucl_object_toboolean (obj); - if (flag) { - func->ucl_emitter_append_len ("true", 4, func->ud); - } else { - func->ucl_emitter_append_len ("false", 5, func->ud); - } - break; - case UCL_STRING: - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } - ucl_elt_string_write_json (obj->value.sv, obj->len, func); - break; - case UCL_NULL: - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); + func->ucl_emitter_append_character ('\n', 1, func->ud); } - func->ucl_emitter_append_len ("null", 4, func->ud); - break; - case UCL_OBJECT: - ucl_elt_obj_write_json (obj, func, tabs, start_tabs, compact); - break; - case UCL_ARRAY: - ucl_elt_array_write_json (obj->value.av, func, tabs, start_tabs, compact); - break; - case UCL_USERDATA: - break; } } /** - * Write a single object to the buffer - * @param obj object - * @param buf target buffer + * End standard ucl object + * @param ctx emitter context + * @param compact compact flag */ static void -ucl_obj_write_json (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool compact) +ucl_emitter_common_end_object (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool compact) { - const ucl_object_t *cur; - bool is_array = (obj->next != NULL); - - if (is_array) { - /* This is an array actually */ - if (start_tabs) { - ucl_add_tabs (func, tabs, compact); - } + const struct ucl_emitter_functions *func = ctx->func; + if (UCL_EMIT_IDENT_TOP_OBJ(ctx, obj)) { + ctx->ident --; if (compact) { - func->ucl_emitter_append_character ('[', 1, func->ud); + func->ucl_emitter_append_character ('}', 1, func->ud); } else { - func->ucl_emitter_append_len ("[\n", 2, func->ud); - } - cur = obj; - while (cur != NULL) { - ucl_elt_write_json (cur, func, tabs + 1, true, compact); - if (cur->next) { - func->ucl_emitter_append_character (',', 1, func->ud); - } - if (!compact) { + if (ctx->id != UCL_EMIT_CONFIG) { + /* newline is already added for this format */ func->ucl_emitter_append_character ('\n', 1, func->ud); } - cur = cur->next; + ucl_add_tabs (func, ctx->ident, compact); + func->ucl_emitter_append_character ('}', 1, func->ud); } - ucl_add_tabs (func, tabs, compact); - func->ucl_emitter_append_character (']', 1, func->ud); - } - else { - ucl_elt_write_json (obj, func, tabs, start_tabs, compact); } + ucl_emitter_finish_object (ctx, obj, compact, false); } /** - * Emit an object to json - * @param obj object - * @return json output (should be freed after using) + * End standard ucl array + * @param ctx emitter context + * @param compact compact flag */ static void -ucl_object_emit_json (const ucl_object_t *obj, bool compact, - struct ucl_emitter_functions *func) +ucl_emitter_common_end_array (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool compact) { - ucl_obj_write_json (obj, func, 0, false, compact); -} - -/** - * Write a single object to the buffer - * @param obj object to write - * @param buf target buffer - */ -static void -ucl_elt_obj_write_config (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool is_top) -{ - const ucl_object_t *cur, *cur_obj; - ucl_hash_iter_t it = NULL; + const struct ucl_emitter_functions *func = ctx->func; - if (start_tabs) { - ucl_add_tabs (func, tabs, is_top); - } - if (!is_top) { - func->ucl_emitter_append_len ("{\n", 2, func->ud); + ctx->ident --; + if (compact) { + func->ucl_emitter_append_character (']', 1, func->ud); } - - while ((cur = ucl_hash_iterate (obj->value.ov, &it))) { - LL_FOREACH (cur, cur_obj) { - ucl_add_tabs (func, tabs + 1, is_top); - if (cur_obj->flags & UCL_OBJECT_NEED_KEY_ESCAPE) { - ucl_elt_string_write_json (cur_obj->key, cur_obj->keylen, func); - } - else { - func->ucl_emitter_append_len (cur_obj->key, cur_obj->keylen, func->ud); - } - if (cur_obj->type != UCL_OBJECT && cur_obj->type != UCL_ARRAY) { - func->ucl_emitter_append_len (" = ", 3, func->ud); - } - else { - func->ucl_emitter_append_character (' ', 1, func->ud); - } - ucl_elt_write_config (cur_obj, func, - is_top ? tabs : tabs + 1, - false, false, false); - if (cur_obj->type != UCL_OBJECT && cur_obj->type != UCL_ARRAY) { - func->ucl_emitter_append_len (";\n", 2, func->ud); - } - else { - func->ucl_emitter_append_character ('\n', 1, func->ud); - } + else { + if (ctx->id != UCL_EMIT_CONFIG) { + /* newline is already added for this format */ + func->ucl_emitter_append_character ('\n', 1, func->ud); } + ucl_add_tabs (func, ctx->ident, compact); + func->ucl_emitter_append_character (']', 1, func->ud); } - ucl_add_tabs (func, tabs, is_top); - if (!is_top) { - func->ucl_emitter_append_character ('}', 1, func->ud); - } + ucl_emitter_finish_object (ctx, obj, compact, true); } /** - * Write a single array to the buffer - * @param obj array to write - * @param buf target buffer + * Start emit standard UCL array + * @param ctx emitter context + * @param obj object to write + * @param compact compact flag */ static void -ucl_elt_array_write_config (const ucl_object_t *obj, struct ucl_emitter_functions *func, - unsigned int tabs, bool start_tabs, bool is_top) +ucl_emitter_common_start_array (struct ucl_emitter_context *ctx, + const ucl_object_t *obj, bool print_key, bool compact) { - const ucl_object_t *cur = obj; - - if (start_tabs) { - ucl_add_tabs (func, tabs, false); - } + const ucl_object_t *cur; + const struct ucl_emitter_functions *func = ctx->func; + bool first = true; - func->ucl_emitter_append_len ("[\n", 2, func->ud); - while (cur) { - ucl_elt_write_config (cur, func, tabs + 1, true, false, false); - func->ucl_emitter_append_len (",\n", 2, func->ud); - cur = cur->next; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 06:50:34 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 657CFB8E; Fri, 18 Jul 2014 06:50:34 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 393732E96; Fri, 18 Jul 2014 06:50:34 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6I6oYQD077035; Fri, 18 Jul 2014 06:50:34 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6I6oY0E077034; Fri, 18 Jul 2014 06:50:34 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407180650.s6I6oY0E077034@svn.freebsd.org> From: Baptiste Daroussin Date: Fri, 18 Jul 2014 06:50:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268829 - vendor/libucl/20140716 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 06:50:34 -0000 Author: bapt Date: Fri Jul 18 06:50:33 2014 New Revision: 268829 URL: http://svnweb.freebsd.org/changeset/base/268829 Log: Tag import of libucl 20140716 Added: vendor/libucl/20140716/ - copied from r268828, vendor/libucl/dist/ From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 17:55:14 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0D23900; Fri, 18 Jul 2014 17:55:14 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91A602D2D; Fri, 18 Jul 2014 17:55:14 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6IHtEep096703; Fri, 18 Jul 2014 17:55:14 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6IHtDvN096689; Fri, 18 Jul 2014 17:55:13 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201407181755.s6IHtDvN096689@svn.freebsd.org> From: Xin LI Date: Fri, 18 Jul 2014 17:55:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268848 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 17:55:14 -0000 Author: delphij Date: Fri Jul 18 17:55:12 2014 New Revision: 268848 URL: http://svnweb.freebsd.org/changeset/base/268848 Log: 4958 zdb trips assert on pools with ashift >= 0xe Reviewed by: Matthew Ahrens Reviewed by: Max Grossman Reviewed by: George Wilson Reviewed by: Christopher Siden Approved by: Garrett D'Amore illumos/illumos-gate@2a104a5236475eb73aa41eaaf3ed9f3ccbe0ca55 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_debug.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/ztest/ztest.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c Fri Jul 18 17:55:12 2014 (r268848) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2011, 2014 by Delphix. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Fri Jul 18 17:55:12 2014 (r268848) @@ -62,6 +62,21 @@ uint64_t metaslab_gang_bang = SPA_MAXBLO int zfs_condense_pct = 200; /* + * Condensing a metaslab is not guaranteed to actually reduce the amount of + * space used on disk. In particular, a space map uses data in increments of + * MAX(1 << ashift, SPACE_MAP_INITIAL_BLOCKSIZE), so a metaslab might use the + * same number of blocks after condensing. Since the goal of condensing is to + * reduce the number of IOPs required to read the space map, we only want to + * condense when we can be sure we will reduce the number of blocks used by the + * space map. Unfortunately, we cannot precisely compute whether or not this is + * the case in metaslab_should_condense since we are holding ms_lock. Instead, + * we apply the following heuristic: do not condense a spacemap unless the + * uncondensed size consumes greater than zfs_metaslab_condense_block_threshold + * blocks. + */ +int zfs_metaslab_condense_block_threshold = 4; + +/* * The zfs_mg_noalloc_threshold defines which metaslab groups should * be eligible for allocation. The value is defined as a percentage of * a free space. Metaslab groups that have more free space than @@ -1279,6 +1294,8 @@ metaslab_group_preload(metaslab_group_t * times the size than the free space range tree representation * (i.e. zfs_condense_pct = 110 and in-core = 1MB, minimal = 1.1.MB). * + * 3. The on-disk size of the space map should actually decrease. + * * Checking the first condition is tricky since we don't want to walk * the entire AVL tree calculating the estimated on-disk size. Instead we * use the size-ordered range tree in the metaslab and calculate the @@ -1289,13 +1306,21 @@ metaslab_group_preload(metaslab_group_t * To determine the second criterion we use a best-case estimate and assume * each segment can be represented on-disk as a single 64-bit entry. We refer * to this best-case estimate as the space map's minimal form. + * + * Unfortunately, we cannot compute the on-disk size of the space map in this + * context because we cannot accurately compute the effects of compression, etc. + * Instead, we apply the heuristic described in the block comment for + * zfs_metaslab_condense_block_threshold - we only condense if the space used + * is greater than a threshold number of blocks. */ static boolean_t metaslab_should_condense(metaslab_t *msp) { space_map_t *sm = msp->ms_sm; range_seg_t *rs; - uint64_t size, entries, segsz; + uint64_t size, entries, segsz, object_size, optimal_size, record_size; + dmu_object_info_t doi; + uint64_t vdev_blocksize = 1 << msp->ms_group->mg_vd->vdev_ashift; ASSERT(MUTEX_HELD(&msp->ms_lock)); ASSERT(msp->ms_loaded); @@ -1319,9 +1344,15 @@ metaslab_should_condense(metaslab_t *msp entries = size / (MIN(size, SM_RUN_MAX)); segsz = entries * sizeof (uint64_t); - return (segsz <= space_map_length(msp->ms_sm) && - space_map_length(msp->ms_sm) >= (zfs_condense_pct * - sizeof (uint64_t) * avl_numnodes(&msp->ms_tree->rt_root)) / 100); + optimal_size = sizeof (uint64_t) * avl_numnodes(&msp->ms_tree->rt_root); + object_size = space_map_length(msp->ms_sm); + + dmu_object_info_from_db(sm->sm_dbuf, &doi); + record_size = MAX(doi.doi_data_block_size, vdev_blocksize); + + return (segsz <= object_size && + object_size >= (optimal_size * zfs_condense_pct / 100) && + object_size > zfs_metaslab_condense_block_threshold * record_size); } /* Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Jul 18 17:55:12 2014 (r268848) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2011, 2014 by Delphix. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Fri Jul 18 17:55:12 2014 (r268848) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. */ #ifndef _SYS_VDEV_IMPL_H @@ -229,8 +229,11 @@ struct vdev { #define VDEV_PHYS_SIZE (112 << 10) #define VDEV_UBERBLOCK_RING (128 << 10) +/* The largest uberblock we support is 8k. */ +#define MAX_UBERBLOCK_SHIFT (13) #define VDEV_UBERBLOCK_SHIFT(vd) \ - MAX((vd)->vdev_top->vdev_ashift, UBERBLOCK_SHIFT) + MIN(MAX((vd)->vdev_top->vdev_ashift, UBERBLOCK_SHIFT), \ + MAX_UBERBLOCK_SHIFT) #define VDEV_UBERBLOCK_COUNT(vd) \ (VDEV_UBERBLOCK_RING >> VDEV_UBERBLOCK_SHIFT(vd)) #define VDEV_UBERBLOCK_OFFSET(vd, n) \ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Fri Jul 18 17:55:12 2014 (r268848) @@ -165,19 +165,20 @@ enum zio_flag { ZIO_FLAG_RESILVER = 1 << 3, ZIO_FLAG_SCRUB = 1 << 4, ZIO_FLAG_SCAN_THREAD = 1 << 5, + ZIO_FLAG_PHYSICAL = 1 << 6, #define ZIO_FLAG_AGG_INHERIT (ZIO_FLAG_CANFAIL - 1) /* * Flags inherited by ddt, gang, and vdev children. */ - ZIO_FLAG_CANFAIL = 1 << 6, /* must be first for INHERIT */ - ZIO_FLAG_SPECULATIVE = 1 << 7, - ZIO_FLAG_CONFIG_WRITER = 1 << 8, - ZIO_FLAG_DONT_RETRY = 1 << 9, - ZIO_FLAG_DONT_CACHE = 1 << 10, - ZIO_FLAG_NODATA = 1 << 11, - ZIO_FLAG_INDUCE_DAMAGE = 1 << 12, + ZIO_FLAG_CANFAIL = 1 << 7, /* must be first for INHERIT */ + ZIO_FLAG_SPECULATIVE = 1 << 8, + ZIO_FLAG_CONFIG_WRITER = 1 << 9, + ZIO_FLAG_DONT_RETRY = 1 << 10, + ZIO_FLAG_DONT_CACHE = 1 << 11, + ZIO_FLAG_NODATA = 1 << 12, + ZIO_FLAG_INDUCE_DAMAGE = 1 << 13, #define ZIO_FLAG_DDT_INHERIT (ZIO_FLAG_IO_RETRY - 1) #define ZIO_FLAG_GANG_INHERIT (ZIO_FLAG_IO_RETRY - 1) @@ -185,27 +186,27 @@ enum zio_flag { /* * Flags inherited by vdev children. */ - ZIO_FLAG_IO_RETRY = 1 << 13, /* must be first for INHERIT */ - ZIO_FLAG_PROBE = 1 << 14, - ZIO_FLAG_TRYHARD = 1 << 15, - ZIO_FLAG_OPTIONAL = 1 << 16, + ZIO_FLAG_IO_RETRY = 1 << 14, /* must be first for INHERIT */ + ZIO_FLAG_PROBE = 1 << 15, + ZIO_FLAG_TRYHARD = 1 << 16, + ZIO_FLAG_OPTIONAL = 1 << 17, #define ZIO_FLAG_VDEV_INHERIT (ZIO_FLAG_DONT_QUEUE - 1) /* * Flags not inherited by any children. */ - ZIO_FLAG_DONT_QUEUE = 1 << 17, /* must be first for INHERIT */ - ZIO_FLAG_DONT_PROPAGATE = 1 << 18, - ZIO_FLAG_IO_BYPASS = 1 << 19, - ZIO_FLAG_IO_REWRITE = 1 << 20, - ZIO_FLAG_RAW = 1 << 21, - ZIO_FLAG_GANG_CHILD = 1 << 22, - ZIO_FLAG_DDT_CHILD = 1 << 23, - ZIO_FLAG_GODFATHER = 1 << 24, - ZIO_FLAG_NOPWRITE = 1 << 25, - ZIO_FLAG_REEXECUTED = 1 << 26, - ZIO_FLAG_DELEGATED = 1 << 27, + ZIO_FLAG_DONT_QUEUE = 1 << 18, /* must be first for INHERIT */ + ZIO_FLAG_DONT_PROPAGATE = 1 << 19, + ZIO_FLAG_IO_BYPASS = 1 << 20, + ZIO_FLAG_IO_REWRITE = 1 << 21, + ZIO_FLAG_RAW = 1 << 22, + ZIO_FLAG_GANG_CHILD = 1 << 23, + ZIO_FLAG_DDT_CHILD = 1 << 24, + ZIO_FLAG_GODFATHER = 1 << 25, + ZIO_FLAG_NOPWRITE = 1 << 26, + ZIO_FLAG_REEXECUTED = 1 << 27, + ZIO_FLAG_DELEGATED = 1 << 28, }; #define ZIO_FLAG_MUSTSUCCEED 0 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_debug.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_debug.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_debug.c Fri Jul 18 17:55:12 2014 (r268848) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. */ #include @@ -57,7 +57,10 @@ zfs_dbgmsg_fini(void) * echo ::zfs_dbgmsg | mdb -k * * Monitor these messages by running: - * dtrace -q -n 'zfs-dbgmsg{printf("%s\n", stringof(arg0))}' + * dtrace -qn 'zfs-dbgmsg{printf("%s\n", stringof(arg0))}' + * + * When used with libzpool, monitor with: + * dtrace -qn 'zfs$pid::zfs_dbgmsg:probe1{printf("%s\n", copyinstr(arg1))}' */ void zfs_dbgmsg(const char *fmt, ...) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Fri Jul 18 17:55:12 2014 (r268848) @@ -815,8 +815,8 @@ zio_read_phys(zio_t *pio, vdev_t *vd, ui ASSERT3U(offset + size, <=, vd->vdev_psize); zio = zio_create(pio, vd->vdev_spa, 0, NULL, data, size, done, private, - ZIO_TYPE_READ, priority, flags, vd, offset, NULL, - ZIO_STAGE_OPEN, ZIO_READ_PHYS_PIPELINE); + ZIO_TYPE_READ, priority, flags | ZIO_FLAG_PHYSICAL, vd, offset, + NULL, ZIO_STAGE_OPEN, ZIO_READ_PHYS_PIPELINE); zio->io_prop.zp_checksum = checksum; @@ -836,8 +836,8 @@ zio_write_phys(zio_t *pio, vdev_t *vd, u ASSERT3U(offset + size, <=, vd->vdev_psize); zio = zio_create(pio, vd->vdev_spa, 0, NULL, data, size, done, private, - ZIO_TYPE_WRITE, priority, flags, vd, offset, NULL, - ZIO_STAGE_OPEN, ZIO_WRITE_PHYS_PIPELINE); + ZIO_TYPE_WRITE, priority, flags | ZIO_FLAG_PHYSICAL, vd, offset, + NULL, ZIO_STAGE_OPEN, ZIO_WRITE_PHYS_PIPELINE); zio->io_prop.zp_checksum = checksum; @@ -2508,7 +2508,9 @@ zio_vdev_io_start(zio_t *zio) align = 1ULL << vd->vdev_top->vdev_ashift; - if (P2PHASE(zio->io_size, align) != 0) { + if (!(zio->io_flags & ZIO_FLAG_PHYSICAL) && + P2PHASE(zio->io_size, align) != 0) { + /* Transform logical writes to be a full physical block size. */ uint64_t asize = P2ROUNDUP(zio->io_size, align); char *abuf = zio_buf_alloc(asize); ASSERT(vd == vd->vdev_top); @@ -2519,8 +2521,22 @@ zio_vdev_io_start(zio_t *zio) zio_push_transform(zio, abuf, asize, asize, zio_subblock); } - ASSERT(P2PHASE(zio->io_offset, align) == 0); - ASSERT(P2PHASE(zio->io_size, align) == 0); + /* + * If this is not a physical io, make sure that it is properly aligned + * before proceeding. + */ + if (!(zio->io_flags & ZIO_FLAG_PHYSICAL)) { + ASSERT0(P2PHASE(zio->io_offset, align)); + ASSERT0(P2PHASE(zio->io_size, align)); + } else { + /* + * For physical writes, we allow 512b aligned writes and assume + * the device will perform a read-modify-write as necessary. + */ + ASSERT0(P2PHASE(zio->io_offset, SPA_MINBLOCKSIZE)); + ASSERT0(P2PHASE(zio->io_size, SPA_MINBLOCKSIZE)); + } + VERIFY(zio->io_type != ZIO_TYPE_WRITE || spa_writeable(spa)); /* From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 17:55:13 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 251BE8FF; Fri, 18 Jul 2014 17:55:13 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 10C782D2C; Fri, 18 Jul 2014 17:55:13 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6IHtChh096684; Fri, 18 Jul 2014 17:55:12 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6IHtCko096683; Fri, 18 Jul 2014 17:55:12 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201407181755.s6IHtCko096683@svn.freebsd.org> From: Xin LI Date: Fri, 18 Jul 2014 17:55:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268848 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 17:55:13 -0000 Author: delphij Date: Fri Jul 18 17:55:12 2014 New Revision: 268848 URL: http://svnweb.freebsd.org/changeset/base/268848 Log: 4958 zdb trips assert on pools with ashift >= 0xe Reviewed by: Matthew Ahrens Reviewed by: Max Grossman Reviewed by: George Wilson Reviewed by: Christopher Siden Approved by: Garrett D'Amore illumos/illumos-gate@2a104a5236475eb73aa41eaaf3ed9f3ccbe0ca55 Modified: vendor/illumos/dist/cmd/ztest/ztest.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_debug.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Fri Jul 18 16:25:35 2014 (r268847) +++ vendor/illumos/dist/cmd/ztest/ztest.c Fri Jul 18 17:55:12 2014 (r268848) @@ -808,7 +808,7 @@ static uint64_t ztest_get_ashift(void) { if (ztest_opts.zo_ashift == 0) - return (SPA_MINBLOCKSHIFT + ztest_random(3)); + return (SPA_MINBLOCKSHIFT + ztest_random(5)); return (ztest_opts.zo_ashift); } @@ -967,11 +967,28 @@ ztest_random_spa_version(uint64_t initia return (version); } +/* + * Find the largest ashift used + */ +static uint64_t +ztest_spa_get_ashift() { + uint64_t i; + uint64_t ashift = SPA_MINBLOCKSHIFT; + vdev_t *rvd = ztest_spa->spa_root_vdev; + + for (i = 0; i < rvd->vdev_children; i++) { + ashift = MAX(ashift, rvd->vdev_child[i]->vdev_ashift); + } + return (ashift); +} + static int ztest_random_blocksize(void) { - return (1 << (SPA_MINBLOCKSHIFT + - ztest_random(SPA_MAXBLOCKSHIFT - SPA_MINBLOCKSHIFT + 1))); + // Choose a block size >= the ashift. + uint64_t block_shift = + ztest_random(SPA_MAXBLOCKSHIFT - ztest_spa_get_ashift() + 1); + return (1 << (SPA_MINBLOCKSHIFT + block_shift)); } static int @@ -5765,16 +5782,30 @@ ztest_freeze(void) spa_freeze(spa); /* + * Because it is hard to predict how much space a write will actually + * require beforehand, we leave ourselves some fudge space to write over + * capacity. + */ + uint64_t capacity = metaslab_class_get_space(spa_normal_class(spa)) / 2; + + /* * Run tests that generate log records but don't alter the pool config * or depend on DSL sync tasks (snapshots, objset create/destroy, etc). * We do a txg_wait_synced() after each iteration to force the txg * to increase well beyond the last synced value in the uberblock. * The ZIL should be OK with that. + * + * Run a random number of times less than zo_maxloops and ensure we do + * not run out of space on the pool. */ while (ztest_random(10) != 0 && - numloops++ < ztest_opts.zo_maxloops) { - ztest_dmu_write_parallel(zd, 0); - ztest_dmu_object_alloc_free(zd, 0); + numloops++ < ztest_opts.zo_maxloops && + metaslab_class_get_alloc(spa_normal_class(spa)) < capacity) { + ztest_od_t od; + ztest_od_init(&od, 0, FTAG, 0, DMU_OT_UINT64_OTHER, 0, 0); + VERIFY0(ztest_object_init(zd, &od, sizeof (od), B_FALSE)); + ztest_io(zd, od.od_object, + ztest_random(ZTEST_RANGE_LOCKS) << SPA_MAXBLOCKSHIFT); txg_wait_synced(spa_get_dsl(spa), 0); } From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 18:02:35 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEB7ADC0; Fri, 18 Jul 2014 18:02:35 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F8012DFE; Fri, 18 Jul 2014 18:02:35 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6II2Z7K001452; Fri, 18 Jul 2014 18:02:35 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6II2ZW3001448; Fri, 18 Jul 2014 18:02:35 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201407181802.s6II2ZW3001448@svn.freebsd.org> From: Xin LI Date: Fri, 18 Jul 2014 18:02:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268850 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 18:02:35 -0000 Author: delphij Date: Fri Jul 18 18:02:34 2014 New Revision: 268850 URL: http://svnweb.freebsd.org/changeset/base/268850 Log: 4631 zvol_get_stats triggering too many reads Reviewed by: Adam Leventhal Reviewed by: Sebastien Roy Reviewed by: Matt Ahrens Approved by: Dan McDonald illumos/illumos-gate@bbfa8ea8bb4168c969ba27d632dfe0aeec3fc0da Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Jul 18 18:00:00 2014 (r268849) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Jul 18 18:02:34 2014 (r268850) @@ -105,7 +105,7 @@ * with the buffer may be evicted prior to the callback. The callback * must be made with *no locks held* (to prevent deadlock). Additionally, * the users of callbacks must ensure that their private data is - * protected from simultaneous callbacks from arc_buf_evict() + * protected from simultaneous callbacks from arc_clear_callback() * and arc_do_user_evicts(). * * Note that the majority of the performance stats are manipulated @@ -1495,8 +1495,12 @@ arc_buf_data_free(arc_buf_t *buf, void ( } } +/* + * Free up buf->b_data and if 'remove' is set, then pull the + * arc_buf_t off of the the arc_buf_hdr_t's list and free it. + */ static void -arc_buf_destroy(arc_buf_t *buf, boolean_t recycle, boolean_t all) +arc_buf_destroy(arc_buf_t *buf, boolean_t recycle, boolean_t remove) { arc_buf_t **bufp; @@ -1547,7 +1551,7 @@ arc_buf_destroy(arc_buf_t *buf, boolean_ } /* only remove the buf if requested */ - if (!all) + if (!remove) return; /* remove the buf from the hdr list */ @@ -2121,7 +2125,7 @@ arc_do_user_evicts(void) mutex_exit(&arc_eviction_mtx); if (buf->b_efunc != NULL) - VERIFY(buf->b_efunc(buf) == 0); + VERIFY0(buf->b_efunc(buf->b_private)); buf->b_efunc = NULL; buf->b_private = NULL; @@ -3240,16 +3244,25 @@ arc_freed(spa_t *spa, const blkptr_t *bp } /* - * This is used by the DMU to let the ARC know that a buffer is - * being evicted, so the ARC should clean up. If this arc buf - * is not yet in the evicted state, it will be put there. + * Clear the user eviction callback set by arc_set_callback(), first calling + * it if it exists. Because the presence of a callback keeps an arc_buf cached + * clearing the callback may result in the arc_buf being destroyed. However, + * it will not result in the *last* arc_buf being destroyed, hence the data + * will remain cached in the ARC. We make a copy of the arc buffer here so + * that we can process the callback without holding any locks. + * + * It's possible that the callback is already in the process of being cleared + * by another thread. In this case we can not clear the callback. + * + * Returns B_TRUE if the callback was successfully called and cleared. */ -int -arc_buf_evict(arc_buf_t *buf) +boolean_t +arc_clear_callback(arc_buf_t *buf) { arc_buf_hdr_t *hdr; kmutex_t *hash_lock; - arc_buf_t **bufp; + arc_evict_func_t *efunc = buf->b_efunc; + void *private = buf->b_private; mutex_enter(&buf->b_evict_lock); hdr = buf->b_hdr; @@ -3259,17 +3272,16 @@ arc_buf_evict(arc_buf_t *buf) */ ASSERT(buf->b_data == NULL); mutex_exit(&buf->b_evict_lock); - return (0); + return (B_FALSE); } else if (buf->b_data == NULL) { - arc_buf_t copy = *buf; /* structure assignment */ /* * We are on the eviction list; process this buffer now * but let arc_do_user_evicts() do the reaping. */ buf->b_efunc = NULL; mutex_exit(&buf->b_evict_lock); - VERIFY(copy.b_efunc(©) == 0); - return (1); + VERIFY0(efunc(private)); + return (B_TRUE); } hash_lock = HDR_LOCK(hdr); mutex_enter(hash_lock); @@ -3279,48 +3291,21 @@ arc_buf_evict(arc_buf_t *buf) ASSERT3U(refcount_count(&hdr->b_refcnt), <, hdr->b_datacnt); ASSERT(hdr->b_state == arc_mru || hdr->b_state == arc_mfu); - /* - * Pull this buffer off of the hdr - */ - bufp = &hdr->b_buf; - while (*bufp != buf) - bufp = &(*bufp)->b_next; - *bufp = buf->b_next; - - ASSERT(buf->b_data != NULL); - arc_buf_destroy(buf, FALSE, FALSE); - - if (hdr->b_datacnt == 0) { - arc_state_t *old_state = hdr->b_state; - arc_state_t *evicted_state; - - ASSERT(hdr->b_buf == NULL); - ASSERT(refcount_is_zero(&hdr->b_refcnt)); - - evicted_state = - (old_state == arc_mru) ? arc_mru_ghost : arc_mfu_ghost; - - mutex_enter(&old_state->arcs_mtx); - mutex_enter(&evicted_state->arcs_mtx); - - arc_change_state(evicted_state, hdr, hash_lock); - ASSERT(HDR_IN_HASH_TABLE(hdr)); - hdr->b_flags |= ARC_IN_HASH_TABLE; - hdr->b_flags &= ~ARC_BUF_AVAILABLE; + buf->b_efunc = NULL; + buf->b_private = NULL; - mutex_exit(&evicted_state->arcs_mtx); - mutex_exit(&old_state->arcs_mtx); + if (hdr->b_datacnt > 1) { + mutex_exit(&buf->b_evict_lock); + arc_buf_destroy(buf, FALSE, TRUE); + } else { + ASSERT(buf == hdr->b_buf); + hdr->b_flags |= ARC_BUF_AVAILABLE; + mutex_exit(&buf->b_evict_lock); } - mutex_exit(hash_lock); - mutex_exit(&buf->b_evict_lock); - VERIFY(buf->b_efunc(buf) == 0); - buf->b_efunc = NULL; - buf->b_private = NULL; - buf->b_hdr = NULL; - buf->b_next = NULL; - kmem_cache_free(buf_cache, buf); - return (1); + mutex_exit(hash_lock); + VERIFY0(efunc(private)); + return (B_TRUE); } /* @@ -3466,17 +3451,6 @@ arc_released(arc_buf_t *buf) return (released); } -int -arc_has_callback(arc_buf_t *buf) -{ - int callback; - - mutex_enter(&buf->b_evict_lock); - callback = (buf->b_efunc != NULL); - mutex_exit(&buf->b_evict_lock); - return (callback); -} - #ifdef ZFS_DEBUG int arc_referenced(arc_buf_t *buf) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Fri Jul 18 18:00:00 2014 (r268849) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Fri Jul 18 18:02:34 2014 (r268850) @@ -181,8 +181,7 @@ dbuf_hash_insert(dmu_buf_impl_t *db) } /* - * Remove an entry from the hash table. This operation will - * fail if there are any existing holds on the db. + * Remove an entry from the hash table. It must be in the EVICTING state. */ static void dbuf_hash_remove(dmu_buf_impl_t *db) @@ -194,7 +193,7 @@ dbuf_hash_remove(dmu_buf_impl_t *db) dmu_buf_impl_t *dbf, **dbp; /* - * We musn't hold db_mtx to maintin lock ordering: + * We musn't hold db_mtx to maintain lock ordering: * DBUF_HASH_MUTEX > db_mtx. */ ASSERT(refcount_is_zero(&db->db_holds)); @@ -431,7 +430,6 @@ static void dbuf_set_data(dmu_buf_impl_t *db, arc_buf_t *buf) { ASSERT(MUTEX_HELD(&db->db_mtx)); - ASSERT(db->db_buf == NULL || !arc_has_callback(db->db_buf)); db->db_buf = buf; if (buf != NULL) { ASSERT(buf->b_data != NULL); @@ -1544,12 +1542,15 @@ dbuf_assign_arcbuf(dmu_buf_impl_t *db, a * when we are not holding the dn_dbufs_mtx, we can't clear the * entry in the dn_dbufs list. We have to wait until dbuf_destroy() * in this case. For callers from the DMU we will usually see: - * dbuf_clear()->arc_buf_evict()->dbuf_do_evict()->dbuf_destroy() + * dbuf_clear()->arc_clear_callback()->dbuf_do_evict()->dbuf_destroy() * For the arc callback, we will usually see: * dbuf_do_evict()->dbuf_clear();dbuf_destroy() * Sometimes, though, we will get a mix of these two: - * DMU: dbuf_clear()->arc_buf_evict() + * DMU: dbuf_clear()->arc_clear_callback() * ARC: dbuf_do_evict()->dbuf_destroy() + * + * This routine will dissociate the dbuf from the arc, by calling + * arc_clear_callback(), but will not evict the data from the ARC. */ void dbuf_clear(dmu_buf_impl_t *db) @@ -1557,7 +1558,7 @@ dbuf_clear(dmu_buf_impl_t *db) dnode_t *dn; dmu_buf_impl_t *parent = db->db_parent; dmu_buf_impl_t *dndb; - int dbuf_gone = FALSE; + boolean_t dbuf_gone = B_FALSE; ASSERT(MUTEX_HELD(&db->db_mtx)); ASSERT(refcount_is_zero(&db->db_holds)); @@ -1603,7 +1604,7 @@ dbuf_clear(dmu_buf_impl_t *db) } if (db->db_buf) - dbuf_gone = arc_buf_evict(db->db_buf); + dbuf_gone = arc_clear_callback(db->db_buf); if (!dbuf_gone) mutex_exit(&db->db_mtx); @@ -1771,8 +1772,7 @@ dbuf_create(dnode_t *dn, uint8_t level, static int dbuf_do_evict(void *private) { - arc_buf_t *buf = private; - dmu_buf_impl_t *db = buf->b_private; + dmu_buf_impl_t *db = private; if (!MUTEX_HELD(&db->db_mtx)) mutex_enter(&db->db_mtx); @@ -2135,11 +2135,23 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, * block on-disk. If so, then we simply evict * ourselves. */ - if (!DBUF_IS_CACHEABLE(db) || - arc_buf_eviction_needed(db->db_buf)) + if (!DBUF_IS_CACHEABLE(db)) { + if (db->db_blkptr != NULL && + !BP_IS_HOLE(db->db_blkptr) && + !BP_IS_EMBEDDED(db->db_blkptr)) { + spa_t *spa = + dmu_objset_spa(db->db_objset); + blkptr_t bp = *db->db_blkptr; + dbuf_clear(db); + arc_freed(spa, &bp); + } else { + dbuf_clear(db); + } + } else if (arc_buf_eviction_needed(db->db_buf)) { dbuf_clear(db); - else + } else { mutex_exit(&db->db_mtx); + } } } else { mutex_exit(&db->db_mtx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Fri Jul 18 18:00:00 2014 (r268849) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Fri Jul 18 18:02:34 2014 (r268850) @@ -95,7 +95,6 @@ boolean_t arc_buf_remove_ref(arc_buf_t * int arc_buf_size(arc_buf_t *buf); void arc_release(arc_buf_t *buf, void *tag); int arc_released(arc_buf_t *buf); -int arc_has_callback(arc_buf_t *buf); void arc_buf_freeze(arc_buf_t *buf); void arc_buf_thaw(arc_buf_t *buf); boolean_t arc_buf_eviction_needed(arc_buf_t *buf); @@ -114,7 +113,7 @@ zio_t *arc_write(zio_t *pio, spa_t *spa, void arc_freed(spa_t *spa, const blkptr_t *bp); void arc_set_callback(arc_buf_t *buf, arc_evict_func_t *func, void *private); -int arc_buf_evict(arc_buf_t *buf); +boolean_t arc_clear_callback(arc_buf_t *buf); void arc_flush(spa_t *spa); void arc_tempreserve_clear(uint64_t reserve); From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 18:05:10 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9CE6F23; Fri, 18 Jul 2014 18:05:10 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A5D5B2E1C; Fri, 18 Jul 2014 18:05:10 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6II5AJe002006; Fri, 18 Jul 2014 18:05:10 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6II59HW001998; Fri, 18 Jul 2014 18:05:09 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201407181805.s6II59HW001998@svn.freebsd.org> From: Xin LI Date: Fri, 18 Jul 2014 18:05:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268851 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 18:05:10 -0000 Author: delphij Date: Fri Jul 18 18:05:09 2014 New Revision: 268851 URL: http://svnweb.freebsd.org/changeset/base/268851 Log: 4753 increase number of outstanding async writes when sync task is waiting Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Adam Leventhal Reviewed by: Christopher Siden Reviewed by: Dan McDonald Approved by: Garrett D'Amore illumos/illumos-gate@73527f441cbbd953fa42cc5a30a413bad75f24a9 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg.h vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Jul 18 18:02:34 2014 (r268850) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Jul 18 18:05:09 2014 (r268851) @@ -1876,6 +1876,16 @@ spa_writeable(spa_t *spa) return (!!(spa->spa_mode & FWRITE)); } +/* + * Returns true if there is a pending sync task in any of the current + * syncing txg, the current quiescing txg, or the current open txg. + */ +boolean_t +spa_has_pending_synctask(spa_t *spa) +{ + return (!txg_all_lists_empty(&spa->spa_dsl_pool->dp_sync_tasks)); +} + int spa_mode(spa_t *spa) { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Fri Jul 18 18:02:34 2014 (r268850) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Fri Jul 18 18:05:09 2014 (r268851) @@ -780,6 +780,7 @@ extern uint64_t bp_get_dsize(spa_t *spa, extern boolean_t spa_has_slogs(spa_t *spa); extern boolean_t spa_is_root(spa_t *spa); extern boolean_t spa_writeable(spa_t *spa); +extern boolean_t spa_has_pending_synctask(spa_t *spa); extern int spa_mode(spa_t *spa); extern uint64_t strtonum(const char *str, char **nptr); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg.h Fri Jul 18 18:02:34 2014 (r268850) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg.h Fri Jul 18 18:05:09 2014 (r268851) @@ -23,7 +23,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. */ #ifndef _SYS_TXG_H @@ -112,6 +112,7 @@ extern boolean_t txg_sync_waiting(struct extern void txg_list_create(txg_list_t *tl, size_t offset); extern void txg_list_destroy(txg_list_t *tl); extern boolean_t txg_list_empty(txg_list_t *tl, uint64_t txg); +extern boolean_t txg_all_lists_empty(txg_list_t *tl); extern boolean_t txg_list_add(txg_list_t *tl, void *p, uint64_t txg); extern boolean_t txg_list_add_tail(txg_list_t *tl, void *p, uint64_t txg); extern void *txg_list_remove(txg_list_t *tl, uint64_t txg); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c Fri Jul 18 18:02:34 2014 (r268850) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c Fri Jul 18 18:05:09 2014 (r268851) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Portions Copyright 2011 Martin Matuska - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. */ #include @@ -718,6 +718,24 @@ txg_list_empty(txg_list_t *tl, uint64_t } /* + * Returns true if all txg lists are empty. + * + * Warning: this is inherently racy (an item could be added immediately after this + * function returns). We don't bother with the lock because it wouldn't change the + * semantics. + */ +boolean_t +txg_all_lists_empty(txg_list_t *tl) +{ + for (int i = 0; i < TXG_SIZE; i++) { + if (!txg_list_empty(tl, i)) { + return (B_FALSE); + } + } + return (B_TRUE); +} + +/* * Add an entry to the list (unless it's already on the list). * Returns B_TRUE if it was actually added. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Fri Jul 18 18:02:34 2014 (r268850) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Fri Jul 18 18:05:09 2014 (r268851) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. */ #include @@ -350,14 +350,23 @@ vdev_queue_class_min_active(zio_priority } static int -vdev_queue_max_async_writes(uint64_t dirty) +vdev_queue_max_async_writes(spa_t *spa) { int writes; + uint64_t dirty = spa->spa_dsl_pool->dp_dirty_total; uint64_t min_bytes = zfs_dirty_data_max * zfs_vdev_async_write_active_min_dirty_percent / 100; uint64_t max_bytes = zfs_dirty_data_max * zfs_vdev_async_write_active_max_dirty_percent / 100; + /* + * Sync tasks correspond to interactive user actions. To reduce the + * execution time of those actions we push data out as fast as possible. + */ + if (spa_has_pending_synctask(spa)) { + return (zfs_vdev_async_write_max_active); + } + if (dirty < min_bytes) return (zfs_vdev_async_write_min_active); if (dirty > max_bytes) @@ -390,8 +399,7 @@ vdev_queue_class_max_active(spa_t *spa, case ZIO_PRIORITY_ASYNC_READ: return (zfs_vdev_async_read_max_active); case ZIO_PRIORITY_ASYNC_WRITE: - return (vdev_queue_max_async_writes( - spa->spa_dsl_pool->dp_dirty_total)); + return (vdev_queue_max_async_writes(spa)); case ZIO_PRIORITY_SCRUB: return (zfs_vdev_scrub_max_active); default: From owner-svn-src-vendor@FreeBSD.ORG Fri Jul 18 18:09:22 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83D42184; Fri, 18 Jul 2014 18:09:22 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64F2E2E7B; Fri, 18 Jul 2014 18:09:22 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6II9Mwn002819; Fri, 18 Jul 2014 18:09:22 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6II9KkB002810; Fri, 18 Jul 2014 18:09:20 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201407181809.s6II9KkB002810@svn.freebsd.org> From: Xin LI Date: Fri, 18 Jul 2014 18:09:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268852 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 18:09:22 -0000 Author: delphij Date: Fri Jul 18 18:09:20 2014 New Revision: 268852 URL: http://svnweb.freebsd.org/changeset/base/268852 Log: 5008 lock contention (rrw_exit) while running a read only load Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Alex Reece Reviewed by: Christopher Siden Reviewed by: Richard Yao Reviewed by: Saso Kiselkov Approved by: Garrett D'Amore illumos/illumos-gate@c9030f6c93613fe30ee0c16f92b96da7816ac052 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c Fri Jul 18 18:09:20 2014 (r268852) @@ -286,3 +286,91 @@ rrw_tsd_destroy(void *arg) (void *)curthread, (void *)rn->rn_rrl); } } + +/* + * A reader-mostly lock implementation, tuning above reader-writer locks + * for hightly parallel read acquisitions, while pessimizing writes. + * + * The idea is to split single busy lock into array of locks, so that + * each reader can lock only one of them for read, depending on result + * of simple hash function. That proportionally reduces lock congestion. + * Writer same time has to sequentially aquire write on all the locks. + * That makes write aquisition proportionally slower, but in places where + * it is used (filesystem unmount) performance is not critical. + * + * All the functions below are direct wrappers around functions above. + */ +void +rrm_init(rrmlock_t *rrl, boolean_t track_all) +{ + int i; + + for (i = 0; i < RRM_NUM_LOCKS; i++) + rrw_init(&rrl->locks[i], track_all); +} + +void +rrm_destroy(rrmlock_t *rrl) +{ + int i; + + for (i = 0; i < RRM_NUM_LOCKS; i++) + rrw_destroy(&rrl->locks[i]); +} + +void +rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag) +{ + if (rw == RW_READER) + rrm_enter_read(rrl, tag); + else + rrm_enter_write(rrl); +} + +/* + * This maps the current thread to a specific lock. Note that the lock + * must be released by the same thread that acquired it. We do this + * mapping by taking the thread pointer mod a prime number. We examine + * only the low 32 bits of the thread pointer, because 32-bit division + * is faster than 64-bit division, and the high 32 bits have little + * entropy anyway. + */ +#define RRM_TD_LOCK() (((uint32_t)(uintptr_t)(curthread)) % RRM_NUM_LOCKS) + +void +rrm_enter_read(rrmlock_t *rrl, void *tag) +{ + rrw_enter_read(&rrl->locks[RRM_TD_LOCK()], tag); +} + +void +rrm_enter_write(rrmlock_t *rrl) +{ + int i; + + for (i = 0; i < RRM_NUM_LOCKS; i++) + rrw_enter_write(&rrl->locks[i]); +} + +void +rrm_exit(rrmlock_t *rrl, void *tag) +{ + int i; + + if (rrl->locks[0].rr_writer == curthread) { + for (i = 0; i < RRM_NUM_LOCKS; i++) + rrw_exit(&rrl->locks[i], tag); + } else { + rrw_exit(&rrl->locks[RRM_TD_LOCK()], tag); + } +} + +boolean_t +rrm_held(rrmlock_t *rrl, krw_t rw) +{ + if (rw == RW_WRITER) { + return (rrw_held(&rrl->locks[0], rw)); + } else { + return (rrw_held(&rrl->locks[RRM_TD_LOCK()], rw)); + } +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h Fri Jul 18 18:09:20 2014 (r268852) @@ -80,6 +80,31 @@ void rrw_tsd_destroy(void *arg); #define RRW_LOCK_HELD(x) \ (rrw_held(x, RW_WRITER) || rrw_held(x, RW_READER)) +/* + * A reader-mostly lock implementation, tuning above reader-writer locks + * for hightly parallel read acquisitions, pessimizing write acquisitions. + * + * This should be a prime number. See comment in rrwlock.c near + * RRM_TD_LOCK() for details. + */ +#define RRM_NUM_LOCKS 17 +typedef struct rrmlock { + rrwlock_t locks[RRM_NUM_LOCKS]; +} rrmlock_t; + +void rrm_init(rrmlock_t *rrl, boolean_t track_all); +void rrm_destroy(rrmlock_t *rrl); +void rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag); +void rrm_enter_read(rrmlock_t *rrl, void *tag); +void rrm_enter_write(rrmlock_t *rrl); +void rrm_exit(rrmlock_t *rrl, void *tag); +boolean_t rrm_held(rrmlock_t *rrl, krw_t rw); + +#define RRM_READ_HELD(x) rrm_held(x, RW_READER) +#define RRM_WRITE_HELD(x) rrm_held(x, RW_WRITER) +#define RRM_LOCK_HELD(x) \ + (rrm_held(x, RW_WRITER) || rrm_held(x, RW_READER)) + #ifdef __cplusplus } #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h Fri Jul 18 18:09:20 2014 (r268852) @@ -64,7 +64,7 @@ struct zfsvfs { int z_norm; /* normalization flags */ boolean_t z_atime; /* enable atimes mount option */ boolean_t z_unmounted; /* unmounted */ - rrwlock_t z_teardown_lock; + rrmlock_t z_teardown_lock; krwlock_t z_teardown_inactive_lock; list_t z_all_znodes; /* all vnodes in the fs */ kmutex_t z_znodes_lock; /* lock for z_all_znodes */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h Fri Jul 18 18:09:20 2014 (r268852) @@ -238,7 +238,7 @@ typedef struct znode { /* Called on entry to each ZFS vnode and vfs operation */ #define ZFS_ENTER(zfsvfs) \ { \ - rrw_enter_read(&(zfsvfs)->z_teardown_lock, FTAG); \ + rrm_enter_read(&(zfsvfs)->z_teardown_lock, FTAG); \ if ((zfsvfs)->z_unmounted) { \ ZFS_EXIT(zfsvfs); \ return (EIO); \ @@ -246,7 +246,7 @@ typedef struct znode { } /* Must be called before exiting the vop */ -#define ZFS_EXIT(zfsvfs) rrw_exit(&(zfsvfs)->z_teardown_lock, FTAG) +#define ZFS_EXIT(zfsvfs) rrm_exit(&(zfsvfs)->z_teardown_lock, FTAG) /* Verifies the znode is valid */ #define ZFS_VERIFY_ZP(zp) \ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Jul 18 18:09:20 2014 (r268852) @@ -1420,7 +1420,7 @@ zfsvfs_hold(const char *name, void *tag, if (getzfsvfs(name, zfvp) != 0) error = zfsvfs_create(name, zfvp); if (error == 0) { - rrw_enter(&(*zfvp)->z_teardown_lock, (writer) ? RW_WRITER : + rrm_enter(&(*zfvp)->z_teardown_lock, (writer) ? RW_WRITER : RW_READER, tag); if ((*zfvp)->z_unmounted) { /* @@ -1428,7 +1428,7 @@ zfsvfs_hold(const char *name, void *tag, * thread should be just about to disassociate the * objset from the zfsvfs. */ - rrw_exit(&(*zfvp)->z_teardown_lock, tag); + rrm_exit(&(*zfvp)->z_teardown_lock, tag); return (SET_ERROR(EBUSY)); } } @@ -1438,7 +1438,7 @@ zfsvfs_hold(const char *name, void *tag, static void zfsvfs_rele(zfsvfs_t *zfsvfs, void *tag) { - rrw_exit(&zfsvfs->z_teardown_lock, tag); + rrm_exit(&zfsvfs->z_teardown_lock, tag); if (zfsvfs->z_vfs) { VFS_RELE(zfsvfs->z_vfs); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Fri Jul 18 18:09:20 2014 (r268852) @@ -1004,7 +1004,7 @@ zfsvfs_create(const char *osname, zfsvfs mutex_init(&zfsvfs->z_lock, NULL, MUTEX_DEFAULT, NULL); list_create(&zfsvfs->z_all_znodes, sizeof (znode_t), offsetof(znode_t, z_link_node)); - rrw_init(&zfsvfs->z_teardown_lock, B_FALSE); + rrm_init(&zfsvfs->z_teardown_lock, B_FALSE); rw_init(&zfsvfs->z_teardown_inactive_lock, NULL, RW_DEFAULT, NULL); rw_init(&zfsvfs->z_fuid_lock, NULL, RW_DEFAULT, NULL); for (i = 0; i != ZFS_OBJ_MTX_SZ; i++) @@ -1119,7 +1119,7 @@ zfsvfs_free(zfsvfs_t *zfsvfs) mutex_destroy(&zfsvfs->z_znodes_lock); mutex_destroy(&zfsvfs->z_lock); list_destroy(&zfsvfs->z_all_znodes); - rrw_destroy(&zfsvfs->z_teardown_lock); + rrm_destroy(&zfsvfs->z_teardown_lock); rw_destroy(&zfsvfs->z_teardown_inactive_lock); rw_destroy(&zfsvfs->z_fuid_lock); for (i = 0; i != ZFS_OBJ_MTX_SZ; i++) @@ -1784,7 +1784,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea { znode_t *zp; - rrw_enter(&zfsvfs->z_teardown_lock, RW_WRITER, FTAG); + rrm_enter(&zfsvfs->z_teardown_lock, RW_WRITER, FTAG); if (!unmounting) { /* @@ -1814,7 +1814,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea */ if (!unmounting && (zfsvfs->z_unmounted || zfsvfs->z_os == NULL)) { rw_exit(&zfsvfs->z_teardown_inactive_lock); - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); return (SET_ERROR(EIO)); } @@ -1841,7 +1841,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea */ if (unmounting) { zfsvfs->z_unmounted = B_TRUE; - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); rw_exit(&zfsvfs->z_teardown_inactive_lock); } @@ -2073,7 +2073,7 @@ zfs_resume_fs(zfsvfs_t *zfsvfs, const ch znode_t *zp; uint64_t sa_obj = 0; - ASSERT(RRW_WRITE_HELD(&zfsvfs->z_teardown_lock)); + ASSERT(RRM_WRITE_HELD(&zfsvfs->z_teardown_lock)); ASSERT(RW_WRITE_HELD(&zfsvfs->z_teardown_inactive_lock)); /* @@ -2129,7 +2129,7 @@ zfs_resume_fs(zfsvfs_t *zfsvfs, const ch bail: /* release the VOPs */ rw_exit(&zfsvfs->z_teardown_inactive_lock); - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); if (err) { /* Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Fri Jul 18 18:05:09 2014 (r268851) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Fri Jul 18 18:09:20 2014 (r268852) @@ -276,7 +276,7 @@ zfs_znode_move(void *buf, void *newbuf, * can safely ensure that the filesystem is not and will not be * unmounted. The next statement is equivalent to ZFS_ENTER(). */ - rrw_enter(&zfsvfs->z_teardown_lock, RW_READER, FTAG); + rrm_enter(&zfsvfs->z_teardown_lock, RW_READER, FTAG); if (zfsvfs->z_unmounted) { ZFS_EXIT(zfsvfs); rw_exit(&zfsvfs_lock); From owner-svn-src-vendor@FreeBSD.ORG Sat Jul 19 10:42:38 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2863DC6; Sat, 19 Jul 2014 10:42:38 +0000 (UTC) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id 68B392F9E; Sat, 19 Jul 2014 10:42:38 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [198.74.231.63]) by cyrus.watson.org (Postfix) with ESMTPS id ED07646B09; Sat, 19 Jul 2014 06:42:36 -0400 (EDT) Date: Sat, 19 Jul 2014 11:42:36 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Xin LI Subject: Re: svn commit: r268852 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys In-Reply-To: <201407181809.s6II9KkB002810@svn.freebsd.org> Message-ID: References: <201407181809.s6II9KkB002810@svn.freebsd.org> User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-all@freebsd.org, src-committers@freebsd.org, svn-src-vendor@freebsd.org X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 10:42:38 -0000 On Fri, 18 Jul 2014, Xin LI wrote: > Log: > 5008 lock contention (rrw_exit) while running a read only load > Reviewed by: Matthew Ahrens > Reviewed by: George Wilson > Reviewed by: Alex Reece > Reviewed by: Christopher Siden > Reviewed by: Richard Yao > Reviewed by: Saso Kiselkov > Approved by: Garrett D'Amore > > illumos/illumos-gate@c9030f6c93613fe30ee0c16f92b96da7816ac052 Is there an opportunity to use our own read-mostly lock implementation here? It should be substantially more scalable, has integration with WITNESS, proper priority propagation, etc. Robert > > Modified: > vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c > vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h > vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h > vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h > vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c > vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c > vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/rrwlock.c Fri Jul 18 18:09:20 2014 (r268852) > @@ -286,3 +286,91 @@ rrw_tsd_destroy(void *arg) > (void *)curthread, (void *)rn->rn_rrl); > } > } > + > +/* > + * A reader-mostly lock implementation, tuning above reader-writer locks > + * for hightly parallel read acquisitions, while pessimizing writes. > + * > + * The idea is to split single busy lock into array of locks, so that > + * each reader can lock only one of them for read, depending on result > + * of simple hash function. That proportionally reduces lock congestion. > + * Writer same time has to sequentially aquire write on all the locks. > + * That makes write aquisition proportionally slower, but in places where > + * it is used (filesystem unmount) performance is not critical. > + * > + * All the functions below are direct wrappers around functions above. > + */ > +void > +rrm_init(rrmlock_t *rrl, boolean_t track_all) > +{ > + int i; > + > + for (i = 0; i < RRM_NUM_LOCKS; i++) > + rrw_init(&rrl->locks[i], track_all); > +} > + > +void > +rrm_destroy(rrmlock_t *rrl) > +{ > + int i; > + > + for (i = 0; i < RRM_NUM_LOCKS; i++) > + rrw_destroy(&rrl->locks[i]); > +} > + > +void > +rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag) > +{ > + if (rw == RW_READER) > + rrm_enter_read(rrl, tag); > + else > + rrm_enter_write(rrl); > +} > + > +/* > + * This maps the current thread to a specific lock. Note that the lock > + * must be released by the same thread that acquired it. We do this > + * mapping by taking the thread pointer mod a prime number. We examine > + * only the low 32 bits of the thread pointer, because 32-bit division > + * is faster than 64-bit division, and the high 32 bits have little > + * entropy anyway. > + */ > +#define RRM_TD_LOCK() (((uint32_t)(uintptr_t)(curthread)) % RRM_NUM_LOCKS) > + > +void > +rrm_enter_read(rrmlock_t *rrl, void *tag) > +{ > + rrw_enter_read(&rrl->locks[RRM_TD_LOCK()], tag); > +} > + > +void > +rrm_enter_write(rrmlock_t *rrl) > +{ > + int i; > + > + for (i = 0; i < RRM_NUM_LOCKS; i++) > + rrw_enter_write(&rrl->locks[i]); > +} > + > +void > +rrm_exit(rrmlock_t *rrl, void *tag) > +{ > + int i; > + > + if (rrl->locks[0].rr_writer == curthread) { > + for (i = 0; i < RRM_NUM_LOCKS; i++) > + rrw_exit(&rrl->locks[i], tag); > + } else { > + rrw_exit(&rrl->locks[RRM_TD_LOCK()], tag); > + } > +} > + > +boolean_t > +rrm_held(rrmlock_t *rrl, krw_t rw) > +{ > + if (rw == RW_WRITER) { > + return (rrw_held(&rrl->locks[0], rw)); > + } else { > + return (rrw_held(&rrl->locks[RRM_TD_LOCK()], rw)); > + } > +} > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/rrwlock.h Fri Jul 18 18:09:20 2014 (r268852) > @@ -80,6 +80,31 @@ void rrw_tsd_destroy(void *arg); > #define RRW_LOCK_HELD(x) \ > (rrw_held(x, RW_WRITER) || rrw_held(x, RW_READER)) > > +/* > + * A reader-mostly lock implementation, tuning above reader-writer locks > + * for hightly parallel read acquisitions, pessimizing write acquisitions. > + * > + * This should be a prime number. See comment in rrwlock.c near > + * RRM_TD_LOCK() for details. > + */ > +#define RRM_NUM_LOCKS 17 > +typedef struct rrmlock { > + rrwlock_t locks[RRM_NUM_LOCKS]; > +} rrmlock_t; > + > +void rrm_init(rrmlock_t *rrl, boolean_t track_all); > +void rrm_destroy(rrmlock_t *rrl); > +void rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag); > +void rrm_enter_read(rrmlock_t *rrl, void *tag); > +void rrm_enter_write(rrmlock_t *rrl); > +void rrm_exit(rrmlock_t *rrl, void *tag); > +boolean_t rrm_held(rrmlock_t *rrl, krw_t rw); > + > +#define RRM_READ_HELD(x) rrm_held(x, RW_READER) > +#define RRM_WRITE_HELD(x) rrm_held(x, RW_WRITER) > +#define RRM_LOCK_HELD(x) \ > + (rrm_held(x, RW_WRITER) || rrm_held(x, RW_READER)) > + > #ifdef __cplusplus > } > #endif > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h Fri Jul 18 18:09:20 2014 (r268852) > @@ -64,7 +64,7 @@ struct zfsvfs { > int z_norm; /* normalization flags */ > boolean_t z_atime; /* enable atimes mount option */ > boolean_t z_unmounted; /* unmounted */ > - rrwlock_t z_teardown_lock; > + rrmlock_t z_teardown_lock; > krwlock_t z_teardown_inactive_lock; > list_t z_all_znodes; /* all vnodes in the fs */ > kmutex_t z_znodes_lock; /* lock for z_all_znodes */ > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_znode.h Fri Jul 18 18:09:20 2014 (r268852) > @@ -238,7 +238,7 @@ typedef struct znode { > /* Called on entry to each ZFS vnode and vfs operation */ > #define ZFS_ENTER(zfsvfs) \ > { \ > - rrw_enter_read(&(zfsvfs)->z_teardown_lock, FTAG); \ > + rrm_enter_read(&(zfsvfs)->z_teardown_lock, FTAG); \ > if ((zfsvfs)->z_unmounted) { \ > ZFS_EXIT(zfsvfs); \ > return (EIO); \ > @@ -246,7 +246,7 @@ typedef struct znode { > } > > /* Must be called before exiting the vop */ > -#define ZFS_EXIT(zfsvfs) rrw_exit(&(zfsvfs)->z_teardown_lock, FTAG) > +#define ZFS_EXIT(zfsvfs) rrm_exit(&(zfsvfs)->z_teardown_lock, FTAG) > > /* Verifies the znode is valid */ > #define ZFS_VERIFY_ZP(zp) \ > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Jul 18 18:09:20 2014 (r268852) > @@ -1420,7 +1420,7 @@ zfsvfs_hold(const char *name, void *tag, > if (getzfsvfs(name, zfvp) != 0) > error = zfsvfs_create(name, zfvp); > if (error == 0) { > - rrw_enter(&(*zfvp)->z_teardown_lock, (writer) ? RW_WRITER : > + rrm_enter(&(*zfvp)->z_teardown_lock, (writer) ? RW_WRITER : > RW_READER, tag); > if ((*zfvp)->z_unmounted) { > /* > @@ -1428,7 +1428,7 @@ zfsvfs_hold(const char *name, void *tag, > * thread should be just about to disassociate the > * objset from the zfsvfs. > */ > - rrw_exit(&(*zfvp)->z_teardown_lock, tag); > + rrm_exit(&(*zfvp)->z_teardown_lock, tag); > return (SET_ERROR(EBUSY)); > } > } > @@ -1438,7 +1438,7 @@ zfsvfs_hold(const char *name, void *tag, > static void > zfsvfs_rele(zfsvfs_t *zfsvfs, void *tag) > { > - rrw_exit(&zfsvfs->z_teardown_lock, tag); > + rrm_exit(&zfsvfs->z_teardown_lock, tag); > > if (zfsvfs->z_vfs) { > VFS_RELE(zfsvfs->z_vfs); > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Fri Jul 18 18:09:20 2014 (r268852) > @@ -1004,7 +1004,7 @@ zfsvfs_create(const char *osname, zfsvfs > mutex_init(&zfsvfs->z_lock, NULL, MUTEX_DEFAULT, NULL); > list_create(&zfsvfs->z_all_znodes, sizeof (znode_t), > offsetof(znode_t, z_link_node)); > - rrw_init(&zfsvfs->z_teardown_lock, B_FALSE); > + rrm_init(&zfsvfs->z_teardown_lock, B_FALSE); > rw_init(&zfsvfs->z_teardown_inactive_lock, NULL, RW_DEFAULT, NULL); > rw_init(&zfsvfs->z_fuid_lock, NULL, RW_DEFAULT, NULL); > for (i = 0; i != ZFS_OBJ_MTX_SZ; i++) > @@ -1119,7 +1119,7 @@ zfsvfs_free(zfsvfs_t *zfsvfs) > mutex_destroy(&zfsvfs->z_znodes_lock); > mutex_destroy(&zfsvfs->z_lock); > list_destroy(&zfsvfs->z_all_znodes); > - rrw_destroy(&zfsvfs->z_teardown_lock); > + rrm_destroy(&zfsvfs->z_teardown_lock); > rw_destroy(&zfsvfs->z_teardown_inactive_lock); > rw_destroy(&zfsvfs->z_fuid_lock); > for (i = 0; i != ZFS_OBJ_MTX_SZ; i++) > @@ -1784,7 +1784,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea > { > znode_t *zp; > > - rrw_enter(&zfsvfs->z_teardown_lock, RW_WRITER, FTAG); > + rrm_enter(&zfsvfs->z_teardown_lock, RW_WRITER, FTAG); > > if (!unmounting) { > /* > @@ -1814,7 +1814,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea > */ > if (!unmounting && (zfsvfs->z_unmounted || zfsvfs->z_os == NULL)) { > rw_exit(&zfsvfs->z_teardown_inactive_lock); > - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); > + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); > return (SET_ERROR(EIO)); > } > > @@ -1841,7 +1841,7 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolea > */ > if (unmounting) { > zfsvfs->z_unmounted = B_TRUE; > - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); > + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); > rw_exit(&zfsvfs->z_teardown_inactive_lock); > } > > @@ -2073,7 +2073,7 @@ zfs_resume_fs(zfsvfs_t *zfsvfs, const ch > znode_t *zp; > uint64_t sa_obj = 0; > > - ASSERT(RRW_WRITE_HELD(&zfsvfs->z_teardown_lock)); > + ASSERT(RRM_WRITE_HELD(&zfsvfs->z_teardown_lock)); > ASSERT(RW_WRITE_HELD(&zfsvfs->z_teardown_inactive_lock)); > > /* > @@ -2129,7 +2129,7 @@ zfs_resume_fs(zfsvfs_t *zfsvfs, const ch > bail: > /* release the VOPs */ > rw_exit(&zfsvfs->z_teardown_inactive_lock); > - rrw_exit(&zfsvfs->z_teardown_lock, FTAG); > + rrm_exit(&zfsvfs->z_teardown_lock, FTAG); > > if (err) { > /* > > Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c > ============================================================================== > --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Fri Jul 18 18:05:09 2014 (r268851) > +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Fri Jul 18 18:09:20 2014 (r268852) > @@ -276,7 +276,7 @@ zfs_znode_move(void *buf, void *newbuf, > * can safely ensure that the filesystem is not and will not be > * unmounted. The next statement is equivalent to ZFS_ENTER(). > */ > - rrw_enter(&zfsvfs->z_teardown_lock, RW_READER, FTAG); > + rrm_enter(&zfsvfs->z_teardown_lock, RW_READER, FTAG); > if (zfsvfs->z_unmounted) { > ZFS_EXIT(zfsvfs); > rw_exit(&zfsvfs_lock); > > From owner-svn-src-vendor@FreeBSD.ORG Sat Jul 19 14:07:50 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BCE764C3; Sat, 19 Jul 2014 14:07:50 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9ED92ED1; Sat, 19 Jul 2014 14:07:50 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6JE7om2074222; Sat, 19 Jul 2014 14:07:50 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6JE7oPJ074219; Sat, 19 Jul 2014 14:07:50 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407191407.s6JE7oPJ074219@svn.freebsd.org> From: Baptiste Daroussin Date: Sat, 19 Jul 2014 14:07:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268874 - in vendor/libucl/dist: src tests/basic X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 14:07:50 -0000 Author: bapt Date: Sat Jul 19 14:07:49 2014 New Revision: 268874 URL: http://svnweb.freebsd.org/changeset/base/268874 Log: Import libucl 20140718 (fixes a bug in the parser) Added: vendor/libucl/dist/tests/basic/11.in (contents, props changed) vendor/libucl/dist/tests/basic/11.res Modified: vendor/libucl/dist/src/ucl_parser.c Modified: vendor/libucl/dist/src/ucl_parser.c ============================================================================== --- vendor/libucl/dist/src/ucl_parser.c Sat Jul 19 14:06:23 2014 (r268873) +++ vendor/libucl/dist/src/ucl_parser.c Sat Jul 19 14:07:49 2014 (r268874) @@ -1661,6 +1661,11 @@ ucl_state_machine (struct ucl_parser *pa return false; } else { + /* Skip any spaces */ + while (p < chunk->end && ucl_test_character (*p, + UCL_CHARACTER_WHITESPACE_UNSAFE)) { + ucl_chunk_skipc (chunk, p); + } p = chunk->pos; if (*p == '[') { parser->state = UCL_STATE_VALUE; Added: vendor/libucl/dist/tests/basic/11.in ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/dist/tests/basic/11.in Sat Jul 19 14:07:49 2014 (r268874) @@ -0,0 +1,3 @@ + + + {"key": "value"} Added: vendor/libucl/dist/tests/basic/11.res ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/dist/tests/basic/11.res Sat Jul 19 14:07:49 2014 (r268874) @@ -0,0 +1,2 @@ +key = "value"; + From owner-svn-src-vendor@FreeBSD.ORG Sat Jul 19 14:08:16 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D14E5EC; Sat, 19 Jul 2014 14:08:16 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 714022ED8; Sat, 19 Jul 2014 14:08:16 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6JE8GdV074357; Sat, 19 Jul 2014 14:08:16 GMT (envelope-from bapt@svn.freebsd.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6JE8GTY074356; Sat, 19 Jul 2014 14:08:16 GMT (envelope-from bapt@svn.freebsd.org) Message-Id: <201407191408.s6JE8GTY074356@svn.freebsd.org> From: Baptiste Daroussin Date: Sat, 19 Jul 2014 14:08:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r268875 - vendor/libucl/20140718 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 14:08:16 -0000 Author: bapt Date: Sat Jul 19 14:08:15 2014 New Revision: 268875 URL: http://svnweb.freebsd.org/changeset/base/268875 Log: Tag import of libucl 20140718 Added: vendor/libucl/20140718/ - copied from r268874, vendor/libucl/dist/ From owner-svn-src-vendor@FreeBSD.ORG Sat Jul 19 18:32:32 2014 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23D7D791; Sat, 19 Jul 2014 18:32:32 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF4EA229F; Sat, 19 Jul 2014 18:32:30 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id el20so3663623lab.41 for ; Sat, 19 Jul 2014 11:32:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/P8bDR/C8vDQkyKo5b8FgBE0ZxPQEL+Mn2H85N/mg/A=; b=V/7cW/aijadiijpphLvLlw6yLbrndJFheqzjukXhwbV7mzYklOzZBGr+NrpTaVpQ6K PTwqSmLRjIa9TQ9EI2QjBOcekuAqhVRl7rbtkgn/6bdaSUt8B2JuZvEUZuu+NgQnMDF/ xvKGze6wl+F/kRhtWfCHiVGbN+brpeDe3vPj75qd9fFwxWhj9a6uanZ+rQNVFs4zN2po fNSZAKkTIVLfGqgfcDfhzwn3GuT2aSXJwcaKtA55ieGEpa5mvqXd4bM3AF6scGHBpg6m 8WGZ9LYwxOZPYP2eSTmMR1EGIaq3RDKEbBUsYkoA4CBYN9sk4FGIIZLghtAqJhsQdwsl krxA== X-Received: by 10.152.5.105 with SMTP id r9mr13570231lar.37.1405794748460; Sat, 19 Jul 2014 11:32:28 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([91.198.175.1]) by mx.google.com with ESMTPSA id qv5sm15697162lbb.19.2014.07.19.11.32.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Jul 2014 11:32:27 -0700 (PDT) Sender: Alexander Motin Message-ID: <53CAB9B9.6010504@FreeBSD.org> Date: Sat, 19 Jul 2014 21:32:25 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Robert Watson , Xin LI Subject: Re: svn commit: r268852 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys References: <201407181809.s6II9KkB002810@svn.freebsd.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-all@freebsd.org, src-committers@freebsd.org, svn-src-vendor@freebsd.org X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 18:32:32 -0000 On 19.07.2014 13:42, Robert Watson wrote: > On Fri, 18 Jul 2014, Xin LI wrote: > >> Log: >> 5008 lock contention (rrw_exit) while running a read only load >> Reviewed by: Matthew Ahrens >> Reviewed by: George Wilson >> Reviewed by: Alex Reece >> Reviewed by: Christopher Siden >> Reviewed by: Richard Yao >> Reviewed by: Saso Kiselkov >> Approved by: Garrett D'Amore >> >> illumos/illumos-gate@c9030f6c93613fe30ee0c16f92b96da7816ac052 > > Is there an opportunity to use our own read-mostly lock implementation > here? It should be substantially more scalable, has integration with > WITNESS, proper priority propagation, etc. Our rmlock(4) tells: "Writers are permitted to sleep while holding a read-mostly lock, but readers are not." That is not acceptable for ZFS, which allocates memory in many places. Also the rrw locks used there have specific recursion semantics: readers are allowed to recurse on the lock even when there is waiting writer. But first read acquisition at the same situation is not allowed. I am not sure whether out rmlocks can do the same. -- Alexander Motin