From owner-svn-src-all@FreeBSD.ORG Mon Mar 2 21:37:58 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D2EC1CE; Mon, 2 Mar 2015 21:37:58 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05C12B38; Mon, 2 Mar 2015 21:37:58 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t22LbvFv015586; Mon, 2 Mar 2015 21:37:57 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t22Lbu9T015573; Mon, 2 Mar 2015 21:37:56 GMT (envelope-from bapt@FreeBSD.org) Message-Id: <201503022137.t22Lbu9T015573@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f From: Baptiste Daroussin Date: Mon, 2 Mar 2015 21:37:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r279548 - in vendor/libucl/20150302: . cmake doc include klib m4 src tests tests/basic uthash utils X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 21:37:58 -0000 Author: bapt Date: Mon Mar 2 21:37:55 2015 New Revision: 279548 URL: https://svnweb.freebsd.org/changeset/base/279548 Log: Tag import of libucl from git as of 20150302 Added: vendor/libucl/20150302/ - copied from r279545, vendor/libucl/dist/ vendor/libucl/20150302/klib/ - copied from r279546, vendor/libucl/dist/klib/ vendor/libucl/20150302/tests/basic/14.in - copied unchanged from r279546, vendor/libucl/dist/tests/basic/14.in vendor/libucl/20150302/tests/basic/14.res - copied unchanged from r279546, vendor/libucl/dist/tests/basic/14.res Replaced: vendor/libucl/20150302/ChangeLog.md - copied unchanged from r279546, vendor/libucl/dist/ChangeLog.md vendor/libucl/20150302/Makefile.am - copied unchanged from r279546, vendor/libucl/dist/Makefile.am vendor/libucl/20150302/README.md - copied unchanged from r279546, vendor/libucl/dist/README.md vendor/libucl/20150302/cmake/CMakeLists.txt - copied unchanged from r279546, vendor/libucl/dist/cmake/CMakeLists.txt vendor/libucl/20150302/configure.ac - copied unchanged from r279546, vendor/libucl/dist/configure.ac vendor/libucl/20150302/doc/Makefile.am - copied unchanged from r279546, vendor/libucl/dist/doc/Makefile.am vendor/libucl/20150302/doc/api.md - copied unchanged from r279546, vendor/libucl/dist/doc/api.md vendor/libucl/20150302/doc/libucl.3 - copied unchanged from r279546, vendor/libucl/dist/doc/libucl.3 vendor/libucl/20150302/doc/pandoc.template - copied unchanged from r279546, vendor/libucl/dist/doc/pandoc.template vendor/libucl/20150302/include/ucl.h - copied unchanged from r279546, vendor/libucl/dist/include/ucl.h vendor/libucl/20150302/src/Makefile.am - copied unchanged from r279546, vendor/libucl/dist/src/Makefile.am vendor/libucl/20150302/src/ucl_emitter.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_emitter.c vendor/libucl/20150302/src/ucl_emitter_utils.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_emitter_utils.c vendor/libucl/20150302/src/ucl_hash.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_hash.c vendor/libucl/20150302/src/ucl_hash.h - copied unchanged from r279546, vendor/libucl/dist/src/ucl_hash.h vendor/libucl/20150302/src/ucl_internal.h - copied unchanged from r279546, vendor/libucl/dist/src/ucl_internal.h vendor/libucl/20150302/src/ucl_parser.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_parser.c vendor/libucl/20150302/src/ucl_schema.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_schema.c vendor/libucl/20150302/src/ucl_util.c - copied unchanged from r279546, vendor/libucl/dist/src/ucl_util.c vendor/libucl/20150302/tests/schema.test - copied unchanged from r279546, vendor/libucl/dist/tests/schema.test vendor/libucl/20150302/tests/test_generate.c - copied unchanged from r279546, vendor/libucl/dist/tests/test_generate.c vendor/libucl/20150302/tests/test_schema.c - copied unchanged from r279546, vendor/libucl/dist/tests/test_schema.c vendor/libucl/20150302/uthash/utstring.h - copied unchanged from r279546, vendor/libucl/dist/uthash/utstring.h vendor/libucl/20150302/utils/objdump.c - copied unchanged from r279547, vendor/libucl/dist/utils/objdump.c Deleted: vendor/libucl/20150302/m4/ Copied: vendor/libucl/20150302/ChangeLog.md (from r279546, vendor/libucl/dist/ChangeLog.md) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/20150302/ChangeLog.md Mon Mar 2 21:37:55 2015 (r279548, copy of r279546, vendor/libucl/dist/ChangeLog.md) @@ -0,0 +1,34 @@ +# Version history + +## Libucl 0.5 + +- Streamline emitter has been added, so it is now possible to output partial `ucl` objects +- Emitter now is more flexible due to emitter_context structure + +### 0.5.1 +- Fixed number of bugs and memory leaks + +### 0.5.2 + +- Allow userdata objects to be emitted and destructed +- Use userdata objects to store lua function references + +### Libucl 0.6 + +- Reworked macro interface + +### Libucl 0.6.1 + +- Various utilities fixes + +### Libucl 0.7.0 + +- Move to klib library from uthash to reduce memory overhead and increase performance + +### Libucl 0.7.1 + +- Added safe iterators API + +### Libucl 0.7.2 + +- Fixed serious bugs in schema and arrays iteration Copied: vendor/libucl/20150302/Makefile.am (from r279546, vendor/libucl/dist/Makefile.am) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/20150302/Makefile.am Mon Mar 2 21:37:55 2015 (r279548, copy of r279546, vendor/libucl/dist/Makefile.am) @@ -0,0 +1,11 @@ +ACLOCAL_AMFLAGS = -I m4 +EXTRA_DIST = uthash klib README.md + +pkgconfigdir = $(libdir)/pkgconfig +pkgconfig_DATA = libucl.pc + +if LUA_SUB + LUA_SUBDIR = lua +endif + +SUBDIRS = src tests utils doc $(LUA_SUBDIR) \ No newline at end of file Copied: vendor/libucl/20150302/README.md (from r279546, vendor/libucl/dist/README.md) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/20150302/README.md Mon Mar 2 21:37:55 2015 (r279548, copy of r279546, vendor/libucl/dist/README.md) @@ -0,0 +1,369 @@ +# LIBUCL + +[![Build Status](https://travis-ci.org/vstakhov/libucl.svg?branch=master)](https://travis-ci.org/vstakhov/libucl)[![Coverity](https://scan.coverity.com/projects/4138/badge.svg)](https://scan.coverity.com/projects/4138) + +**Table of Contents** *generated with [DocToc](http://doctoc.herokuapp.com/)* + +- [Introduction](#introduction) +- [Basic structure](#basic-structure) +- [Improvements to the json notation](#improvements-to-the-json-notation) + - [General syntax sugar](#general-syntax-sugar) + - [Automatic arrays creation](#automatic-arrays-creation) + - [Named keys hierarchy](#named-keys-hierarchy) + - [Convenient numbers and booleans](#convenient-numbers-and-booleans) +- [General improvements](#general-improvements) + - [Commments](#commments) + - [Macros support](#macros-support) + - [Variables support](#variables-support) + - [Multiline strings](#multiline-strings) +- [Emitter](#emitter) +- [Validation](#validation) +- [Performance](#performance) +- [Conclusion](#conclusion) + +## Introduction + +This document describes the main features and principles of the configuration +language called `UCL` - universal configuration language. + +If you are looking for the libucl API documentation you can find it at [this page](doc/api.md). + +## Basic structure + +UCL is heavily infused by `nginx` configuration as the example of a convenient configuration +system. However, UCL is fully compatible with `JSON` format and is able to parse json files. +For example, you can write the same configuration in the following ways: + +* in nginx like: + +```nginx +param = value; +section { + param = value; + param1 = value1; + flag = true; + number = 10k; + time = 0.2s; + string = "something"; + subsection { + host = { + host = "hostname"; + port = 900; + } + host = { + host = "hostname"; + port = 901; + } + } +} +``` + +* or in JSON: + +```json +{ + "param": "value", + "param1": "value1", + "flag": true, + "subsection": { + "host": [ + { + "host": "hostname", + "port": 900 + }, + { + "host": "hostname", + "port": 901 + } + ] + } +} +``` + +## Improvements to the json notation. + +There are various things that make ucl configuration more convenient for editing than strict json: + +### General syntax sugar + +* Braces are not necessary to enclose a top object: it is automatically treated as an object: + +```json +"key": "value" +``` +is equal to: +```json +{"key": "value"} +``` + +* There is no requirement of quotes for strings and keys, moreover, `:` may be replaced `=` or even be skipped for objects: + +```nginx +key = value; +section { + key = value; +} +``` +is equal to: +```json +{ + "key": "value", + "section": { + "key": "value" + } +} +``` + +* No commas mess: you can safely place a comma or semicolon for the last element in an array or an object: + +```json +{ + "key1": "value", + "key2": "value", +} +``` +### Automatic arrays creation + +* Non-unique keys in an object are allowed and are automatically converted to the arrays internally: + +```json +{ + "key": "value1", + "key": "value2" +} +``` +is converted to: +```json +{ + "key": ["value1", "value2"] +} +``` + +### Named keys hierarchy + +UCL accepts named keys and organize them into objects hierarchy internally. Here is an example of this process: +```nginx +section "blah" { + key = value; +} +section foo { + key = value; +} +``` + +is converted to the following object: + +```nginx +section { + blah { + key = value; + } + foo { + key = value; + } +} +``` + +Plain definitions may be more complex and contain more than a single level of nested objects: + +```nginx +section "blah" "foo" { + key = value; +} +``` + +is presented as: + +```nginx +section { + blah { + foo { + key = value; + } + } +} +``` + +### Convenient numbers and booleans + +* Numbers can have suffixes to specify standard multipliers: + + `[kKmMgG]` - standard 10 base multipliers (so `1k` is translated to 1000) + + `[kKmMgG]b` - 2 power multipliers (so `1kb` is translated to 1024) + + `[s|min|d|w|y]` - time multipliers, all time values are translated to float number of seconds, for example `10min` is translated to 600.0 and `10ms` is translated to 0.01 +* Hexadecimal integers can be used by `0x` prefix, for example `key = 0xff`. However, floating point values can use decimal base only. +* Booleans can be specified as `true` or `yes` or `on` and `false` or `no` or `off`. +* It is still possible to treat numbers and booleans as strings by enclosing them in double quotes. + +## General improvements + +### Commments + +UCL supports different style of comments: + +* single line: `#` +* multiline: `/* ... */` + +Multiline comments may be nested: +```c +# Sample single line comment +/* + some comment + /* nested comment */ + end of comment +*/ +``` + +### Macros support + +UCL supports external macros both multiline and single line ones: +```nginx +.macro "sometext"; +.macro { + Some long text + .... +}; +``` + +Moreover, each macro can accept an optional list of arguments in braces. These +arguments themselves are the UCL object that is parsed and passed to a macro as +options: + +```nginx +.macro(param=value) "something"; +.macro(param={key=value}) "something"; +.macro(.include "params.conf") "something"; +.macro(#this is multiline macro +param = [value1, value2]) "something"; +.macro(key="()") "something"; +``` + +UCL also provide a convenient `include` macro to load content from another files +to the current UCL object. This macro accepts either path to file: + +```nginx +.include "/full/path.conf" +.include "./relative/path.conf" +.include "${CURDIR}/path.conf" +``` + +or URL (if ucl is built with url support provided by either `libcurl` or `libfetch`): + + .include "http://example.com/file.conf" + +`.include` macro supports a set of options: + +* `try` (default: **false**) - if this option is `true` than UCL treats errors on loading of +this file as non-fatal. For example, such a file can be absent but it won't stop the parsing +of the top-level document. +* `sign` (default: **false**) - if this option is `true` UCL loads and checks the signature for +a file from path named `.sig`. Trusted public keys should be provided for UCL API after +parser is created but before any configurations are parsed. +* `glob` (default: **false**) - if this option is `true` UCL treats the filename as GLOB pattern and load +all files that matches the specified pattern (normally the format of patterns is defined in `glob` manual page +for your operating system). This option is meaningless for URL includes. +* `url` (default: **true**) - allow URL includes. +* `priority` (default: 0) - specify priority for the include (see below). + +Priorities are used by UCL parser to manage the policy of objects rewriting during including other files +as following: + +* If we have two objects with the same priority then we form an implicit array +* If a new object has bigger priority then we overwrite an old one +* If a new object has lower priority then we ignore it + +By default, the priority of top-level object is set to zero (lowest priority). Currently, +you can define up to 16 priorities (from 0 to 15). Includes with bigger priorities will +rewrite keys from the objects with lower priorities as specified by the policy. + +### Variables support + +UCL supports variables in input. Variables are registered by a user of the UCL parser and can be presented in the following forms: + +* `${VARIABLE}` +* `$VARIABLE` + +UCL currently does not support nested variables. To escape variables one could use double dollar signs: + +* `$${VARIABLE}` is converted to `${VARIABLE}` +* `$$VARIABLE` is converted to `$VARIABLE` + +However, if no valid variables are found in a string, no expansion will be performed (and `$$` thus remains unchanged). This may be a subject +to change in future libucl releases. + +### Multiline strings + +UCL can handle multiline strings as well as single line ones. It uses shell/perl like notation for such objects: +``` +key = <@]), [], + [enable_urls=no]) +AC_ARG_ENABLE([regex], AS_HELP_STRING([--enable-regex], + [Enable regex checking for schema @<:@default=yes@:>@]), [], + [enable_regex=yes]) +AC_ARG_ENABLE([signatures], AS_HELP_STRING([--enable-signatures], + [Enable signatures check (requires openssl) @<:@default=no@:>@]), [], + [enable_signatures=no]) +AC_ARG_ENABLE([lua], AS_HELP_STRING([--enable-lua], + [Enable lua API build (requires lua libraries and headers) @<:@default=no@:>@]), [], + [enable_lua=no]) +AC_ARG_ENABLE([utils], + AS_HELP_STRING([--enable-utils], [Build and install utils @<:@default=no@:>@]), + [case "${enableval}" in + yes) utils=true ;; + no) utils=false ;; + *) AC_MSG_ERROR([bad value ${enableval} for --enable-utils]) ;; + esac],[utils=false]) +AM_CONDITIONAL([UTILS], [test x$utils = xtrue]) + +AS_IF([test "x$enable_signatures" = "xyes"], [ + AC_SEARCH_LIBS([EVP_MD_CTX_create], [crypto], [ + AC_DEFINE(HAVE_OPENSSL, 1, [Define to 1 if you have the 'crypto' library (-lcrypto).]) + LIBCRYPTO_LIB="-lcrypto" + LIBS_EXTRA="${LIBS_EXTRA} -lcrypto" + ], [AC_MSG_ERROR([unable to find the EVP_MD_CTX_create() function])]) +]) +AC_SUBST(LIBCRYPTO_LIB) +AC_PATH_PROG(PANDOC, pandoc, [/non/existent]) + +AC_SEARCH_LIBS([clock_gettime], [rt], [], [ + AC_CHECK_HEADER([mach/mach_time.h], [ + AC_DEFINE(HAVE_MACH_MACH_TIME_H, 1, [Define to 1 on Darwin]) + ], [AC_MSG_ERROR([unable to find clock_gettime or mach_absolute_time])]) +]) +AC_SEARCH_LIBS([remainder], [m], [], [AC_MSG_ERROR([unable to find remainder() function])]) + +AS_IF([test "x$enable_regex" = "xyes"], [ + AC_CHECK_HEADER([regex.h], [ + AC_DEFINE(HAVE_REGEX_H, 1, [Define to 1 if you have the header file.]) + AC_SEARCH_LIBS([regexec], [regex], [ + AS_IF([test "x$ac_cv_search_regexec" = "x-lregex"], [ + LIBREGEX_LIB="-lregex" + LIBS_EXTRA="${LIBS_EXTRA} -lregex" + ] + )], + [AC_MSG_ERROR([unable to find the regexec() function])])], + [AC_MSG_ERROR([unable to find the regex.h header]) + ], + [#include ]) +]) +AC_SUBST(LIBREGEX_LIB) + +AS_IF([test "x$enable_lua" = "xyes"], [ + AX_PROG_LUA([5.1], [], [ + AX_LUA_HEADERS([ + AX_LUA_LIBS([ + AC_DEFINE(HAVE_LUA, 1, [Define to 1 for lua support.]) + with_lua="yes" + ], [AC_MSG_ERROR([unable to find the lua libraries]) + ]) + ], [AC_MSG_ERROR([unable to find the lua header files]) + ]) + ], [AC_MSG_ERROR([unable to find the lua interpreter])]) +], [with_lua="no"]) + +AM_CONDITIONAL([LUA_SUB], [test "$with_lua" = "yes"]) + +AS_IF([test "x$enable_urls" = "xyes"], [ + AC_CHECK_HEADER([fetch.h], [ + AC_DEFINE(HAVE_FETCH_H, 1, [Define to 1 if you have the header file.]) + AC_CHECK_LIB(fetch, fetchXGet, [ + AC_DEFINE(HAVE_LIBFETCH, 1, [Define to 1 if you have the 'fetch' library (-lfetch).]) + LIBFETCH_LIBS="-lfetch" + have_libfetch="yes" + LIBS_EXTRA="${LIBS_EXTRA} -lfetch" + ]) + ], [],[ + #include + #ifdef HAVE_SYS_PARAM_H + #include + #endif + ]) + AC_SUBST(LIBFETCH_LIBS) + + AS_IF([ test "x$have_libfetch" != "xyes"], [ + dnl Fallback to libcurl + PKG_CHECK_MODULES([CURL], [libcurl], [ + AC_DEFINE(CURL_FOUND, 1, [Use libcurl]) + LIBS_EXTRA="${LIBS_EXTRA} -lcurl"], + [AC_MSG_ERROR([unable to find neither libfetch nor libcurl])]) + ]) + AC_SUBST(CURL_FOUND) + AC_SUBST(CURL_LIBS) + AC_SUBST(CURL_CFLAGS) +]) + +AC_SUBST(LIBS_EXTRA) + +AC_MSG_CHECKING(for GCC atomic builtins) +AC_LINK_IFELSE([ + AC_LANG_SOURCE([[ + int main() { + volatile unsigned long val = 1; + __sync_synchronize(); + __sync_val_compare_and_swap(&val, 1, 0); + __sync_add_and_fetch(&val, 1); + __sync_sub_and_fetch(&val, 1); + return 0; + } + ]]) +], +[ + AC_MSG_RESULT([yes]) + AC_DEFINE([HAVE_ATOMIC_BUILTINS], [1], [Has gcc/MSVC atomic intrinsics]) +], +[ + AC_MSG_RESULT([no]) + AC_DEFINE([HAVE_ATOMIC_BUILTINS], [0], [Has gcc/MSVC atomic intrinsics]) + AC_MSG_WARN([Libucl references could be thread-unsafe because atomic builtins are missing]) +]) + +AC_CONFIG_FILES(Makefile \ + src/Makefile \ + lua/Makefile + tests/Makefile \ + utils/Makefile \ + doc/Makefile \ + lua/libucl.rockspec \ + libucl.pc) +AC_CONFIG_FILES([stamp-h], [echo timestamp > stamp-h]) +AC_OUTPUT Copied: vendor/libucl/20150302/doc/Makefile.am (from r279546, vendor/libucl/dist/doc/Makefile.am) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/20150302/doc/Makefile.am Mon Mar 2 21:37:55 2015 (r279548, copy of r279546, vendor/libucl/dist/doc/Makefile.am) @@ -0,0 +1,9 @@ +EXTRA_DIST = api.md + +dist_man_MANS = libucl.3 + +gen-man: @PANDOC@ + tail -n +$$(grep -n '# Synopsis' api.md | cut -d':' -f1) api.md | \ + cat pandoc.template - | sed -e 's/^# \(.*\)/# \U\1/' \ + -e "s/%%date%%/$$(LANG=C date +'%d %B, %Y')/" | \ + @PANDOC@ -s -f markdown -t man -o libucl.3 \ No newline at end of file Copied: vendor/libucl/20150302/doc/api.md (from r279546, vendor/libucl/dist/doc/api.md) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libucl/20150302/doc/api.md Mon Mar 2 21:37:55 2015 (r279548, copy of r279546, vendor/libucl/dist/doc/api.md) @@ -0,0 +1,495 @@ +# API documentation + +**Table of Contents** *generated with [DocToc](http://doctoc.herokuapp.com/)* + +- [Synopsis](#synopsis) +- [Description](#description) + - [Parser functions](#parser-functions) + - [Emitting functions](#emitting-functions) + - [Conversion functions](#conversion-functions) + - [Generation functions](#generation-functions) + - [Iteration functions](#iteration-functions) + - [Validation functions](#validation-functions) + - [Utility functions](#utility-functions) +- [Parser functions](#parser-functions-1) + - [ucl_parser_new](#ucl_parser_new) + - [ucl_parser_register_macro](#ucl_parser_register_macro) + - [ucl_parser_register_variable](#ucl_parser_register_variable) + - [ucl_parser_add_chunk](#ucl_parser_add_chunk) + - [ucl_parser_add_string](#ucl_parser_add_string) + - [ucl_parser_add_file](#ucl_parser_add_file) + - [ucl_parser_get_object](#ucl_parser_get_object) + - [ucl_parser_get_error](#ucl_parser_get_error) + - [ucl_parser_free](#ucl_parser_free) + - [ucl_pubkey_add](#ucl_pubkey_add) + - [ucl_parser_set_filevars](#ucl_parser_set_filevars) + - [Parser usage example](#parser-usage-example) +- [Emitting functions](#emitting-functions-1) + - [ucl_object_emit](#ucl_object_emit) + - [ucl_object_emit_full](#ucl_object_emit_full) +- [Conversion functions](#conversion-functions-1) +- [Generation functions](#generation-functions-1) + - [ucl_object_new](#ucl_object_new) + - [ucl_object_typed_new](#ucl_object_typed_new) + - [Primitive objects generation](#primitive-objects-generation) + - [ucl_object_fromstring_common](#ucl_object_fromstring_common) +- [Iteration functions](#iteration-functions-1) + - [ucl_iterate_object](#ucl_iterate_object) +- [Validation functions](#validation-functions-1) + - [ucl_object_validate](#ucl_object_validate) + +# Synopsis + +`#include ` + +# Description + +Libucl is a parser and `C` API to parse and generate `ucl` objects. Libucl consist of several groups of functions: + +### Parser functions +Used to parse `ucl` files and provide interface to extract `ucl` object. Currently, `libucl` can parse only full `ucl` documents, for instance, it is impossible to parse a part of document and therefore it is impossible to use `libucl` as a streaming parser. In future, this limitation can be removed. + +### Emitting functions +Convert `ucl` objects to some textual or binary representation. Currently, libucl supports the following exports: + +- `JSON` - valid json format (can possibly lose some original data, such as implicit arrays) +- `Config` - human-readable configuration format (lossless) +- `YAML` - embedded yaml format (has the same limitations as `json` output) + +### Conversion functions +Help to convert `ucl` objects to C types. These functions are used to convert `ucl_object_t` to C primitive types, such as numbers, strings or boolean values. + +### Generation functions +Allow creation of `ucl` objects from C types and creating of complex `ucl` objects, such as hashes or arrays from primitive `ucl` objects, such as numbers or strings. + +### Iteration functions +Iterate over `ucl` complex objects or over a chain of values, for example when a key in an object has multiple values (that can be treated as implicit array or implicit consolidation). + +### Validation functions +Validation functions are used to validate some object `obj` using json-schema compatible object `schema`. Both input and schema must be UCL objects to perform validation. + +### Utility functions +Provide basic utilities to manage `ucl` objects: creating, removing, retaining and releasing reference count and so on. + +# Parser functions + +Parser functions operates with `struct ucl_parser`. + +### ucl_parser_new + +~~~C +struct ucl_parser* ucl_parser_new (int flags); +~~~ + +Creates new parser with the specified flags: + +- `UCL_PARSER_KEY_LOWERCASE` - lowercase keys parsed +- `UCL_PARSER_ZEROCOPY` - try to use zero-copy mode when reading files (in zero-copy mode text chunk being parsed without copying strings so it should exist till any object parsed is used) +- `UCL_PARSER_NO_TIME` - treat time values as strings without parsing them as floats + +### ucl_parser_register_macro + +~~~C +void ucl_parser_register_macro (struct ucl_parser *parser, + const char *macro, ucl_macro_handler handler, void* ud); +~~~ + +Register new macro with name .`macro` parsed by handler `handler` that accepts opaque data pointer `ud`. Macro handler should be of the following type: + +~~~C +bool (*ucl_macro_handler) (const unsigned char *data, + size_t len, void* ud);` +~~~ + +Handler function accepts macro text `data` of length `len` and the opaque pointer `ud`. If macro is parsed successfully the handler should return `true`. `false` indicates parsing failure and the parser can be terminated. + +### ucl_parser_register_variable + +~~~C +void ucl_parser_register_variable (struct ucl_parser *parser, + const char *var, const char *value); +~~~ + +Register new variable $`var` that should be replaced by the parser to the `value` string. + +### ucl_parser_add_chunk + +~~~C +bool ucl_parser_add_chunk (struct ucl_parser *parser, + const unsigned char *data, size_t len); +~~~ + +Add new text chunk with `data` of length `len` to the parser. At the moment, `libucl` parser is not a streamlined parser and chunk *must* contain the *valid* ucl object. For example, this object should be valid: + +~~~json +{ "var": "value" } +~~~ + +while this one won't be parsed correctly: + +~~~json +{ "var": +~~~ + +This limitation may possible be removed in future. + +### ucl_parser_add_string +~~~C +bool ucl_parser_add_string (struct ucl_parser *parser, + const char *data, size_t len); +~~~ + +This function acts exactly like `ucl_parser_add_chunk` does but if `len` argument is zero, then the string `data` must be zero-terminated and the actual length is calculated up to `\0` character. + +### ucl_parser_add_file + +~~~C +bool ucl_parser_add_file (struct ucl_parser *parser, + const char *filename); +~~~ + +Load file `filename` and parse it with the specified `parser`. This function uses `mmap` call to load file, therefore, it should not be `shrunk` during parsing. Otherwise, `libucl` can cause memory corruption and terminate the calling application. This function is also used by the internal handler of `include` macro, hence, this macro has the same limitation. + +### ucl_parser_get_object + +~~~C +ucl_object_t* ucl_parser_get_object (struct ucl_parser *parser); +~~~ + +If the `ucl` data has been parsed correctly this function returns the top object for the parser. Otherwise, this function returns the `NULL` pointer. The reference count for `ucl` object returned is increased by one, therefore, a caller should decrease reference by using `ucl_object_unref` to free object after usage. + +### ucl_parser_get_error + +~~~C +const char *ucl_parser_get_error(struct ucl_parser *parser); +~~~ + +Returns the constant error string for the parser object. If no error occurred during parsing a `NULL` object is returned. A caller should not try to free or modify this string. + +### ucl_parser_free + +~~~C +void ucl_parser_free (struct ucl_parser *parser); +~~~ + +Frees memory occupied by the parser object. The reference count for top object is decreased as well, however if the function `ucl_parser_get_object` was called previously then the top object won't be freed. + +### ucl_pubkey_add + +~~~C +bool ucl_pubkey_add (struct ucl_parser *parser, + const unsigned char *key, size_t len); +~~~ + +This function adds a public key from text blob `key` of length `len` to the `parser` object. This public key should be in the `PEM` format and can be used by `.includes` macro for checking signatures of files included. `Openssl` support should be enabled to make this function working. If a key cannot be added (e.g. due to format error) or `openssl` was not linked to `libucl` then this function returns `false`. + +### ucl_parser_set_filevars + +~~~C +bool ucl_parser_set_filevars (struct ucl_parser *parser, + const char *filename, bool need_expand); +~~~ + +Add the standard file variables to the `parser` based on the `filename` specified: + +- `$FILENAME` - a filename of `ucl` input +- `$CURDIR` - a current directory of the input + +For example, if a `filename` param is `../something.conf` then the variables will have the following values: + +- `$FILENAME` - "../something.conf" +- `$CURDIR` - ".." + +if `need_expand` parameter is `true` then all relative paths are expanded using `realpath` call. In this example if `..` is `/etc/dir` then variables will have these values: + +- `$FILENAME` - "/etc/something.conf" +- `$CURDIR` - "/etc" + +## Parser usage example + +The following example loads, parses and extracts `ucl` object from stdin using `libucl` parser functions (the length of input is limited to 8K): + +~~~C +char inbuf[8192]; +struct ucl_parser *parser = NULL; +int ret = 0, r = 0; +ucl_object_t *obj = NULL; +FILE *in; + +in = stdin; +parser = ucl_parser_new (0); +while (!feof (in) && r < (int)sizeof (inbuf)) { + r += fread (inbuf + r, 1, sizeof (inbuf) - r, in); +} +ucl_parser_add_chunk (parser, inbuf, r); +fclose (in); + +if (ucl_parser_get_error (parser)) { + printf ("Error occurred: %s\n", ucl_parser_get_error (parser)); + ret = 1; +} +else { + obj = ucl_parser_get_object (parser); +} + +if (parser != NULL) { + ucl_parser_free (parser); +} +if (obj != NULL) { + ucl_object_unref (obj); +} +return ret; +~~~ + *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***