Date: Tue, 16 Jul 2013 22:49:02 +0000 From: "Teske, Devin" <Devin.Teske@fisglobal.com> To: Steve Kiernan <stevek@juniper.net> Cc: "arch@freebsd.org" <arch@freebsd.org>, Devin Teske <dteske@freebsd.org>, Jordan Hubbard <jkh@mail.turbofuzz.com> Subject: Re: General purpose library for name/value pairs. Message-ID: <13CA24D6AB415D428143D44749F57D7201FC9F3C@ltcfiswmsgmb21> In-Reply-To: <13CA24D6AB415D428143D44749F57D7201FC9EED@ltcfiswmsgmb21> References: <20130704215329.GG1402@garage.freebsd.pl> <4818.1373008073@critter.freebsd.dk> <20130705195255.GB25842@garage.freebsd.pl> <60317.1373055040@critter.freebsd.dk> <20130708150308.GE1383@garage.freebsd.pl> <717D098F-D07E-45B0-B9F0-8D8BCEF06923@mail.turbofuzz.com> <20130716180606.55ec081c@stevek-ubuntu> <13CA24D6AB415D428143D44749F57D7201FC9EED@ltcfiswmsgmb21>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jul 16, 2013, at 3:39 PM, Teske, Devin wrote:
On Jul 16, 2013, at 3:06 PM, Steve Kiernan wrote:
On Mon, 8 Jul 2013 10:57:17 -0700
Jordan Hubbard <jkh@mail.turbofuzz.com<mailto:jkh@mail.turbofuzz.com>> wrot=
e:
On Jul 8, 2013, at 8:03 AM, Pawel Jakub Dawidek <pjd@freebsd.org<mailto:pjd=
@freebsd.org>> wrote:
How about instead of supporting int8, uint8, int16, uint16, int32,
uint32, int64 and uint64 I'd just support 'number' type, which would be
uint64_t. This shouldn't break transporting signed values and because I
don't support basic types like int, long, etc. casting or conversion has
to be done anyway. This would reduce number of supported types to:
That's a good idea. Since you're re-inventing Apple's XML property list AP=
I (but without serialization and quite a few other things), keeping the num=
ber of supported types down is much better than the converse - exposing the=
details of 16/32/64 bit numbers and their signedness will hang you over th=
e long term, and you can always provide explicit conversion functions to/fr=
om Number for those who actually care.
String, Number, Boolean, Data, Date, Array and Dictionary are all plists su=
pport, and Apple developers have gotten along pretty well for many years wi=
th that set (not supporting Dictionaries, btw, is a pretty fundamental loss=
IMHO - it means you have to always iterate through lists to find your stuf=
f, which is meh!).
When Apple extended the Plist metaphor for IPC (to create XPC), they also a=
dded out-of-band types like file and shared memory descriptors. You have f=
ile descriptors, but not shared memory. The latter is kind of important if=
you want to do larger IPCs with this mechanism someday and want to support=
"big payloads" transparently without passing the burden of the data manage=
ment to the application programmer.
Before you also come back with "but but this is a kernel API! For just one=
purpose!" let me just say that it's absolutely achievable to have ONE API =
for talking userland<->kernel and userland<->userland with the same types a=
nd the transport mechanism(s) largely abstracted away. You don't need two =
- it's already been done with one, just look next door. :)
If you add file serialization of this format to your picture (and add the d=
ictionary type) , bingo, now you have a preferences API that FreeBSD has la=
cked from day one ("just roll your own format and stick the file in /etc" b=
eing the canonical response to that need).
What about looking at NetBSD's libprop?
PROPLIB(3) FreeBSD Library Functions Manual PROPLIB=
(3)
NAME
proplib -- property container object library
LIBRARY
library ``libprop''
SYNOPSIS
#include <prop/proplib.h>
DESCRIPTION
The proplib library provides an abstract interface for creating and
manipulating property lists. Property lists have object types for
boolean values, opaque data, numbers, and strings. Structure is provid=
ed
by the array and dictionary collection types.
Property lists can be passed across protection boundaries by translating
them to an external representation. This external representation is an
XML document whose format is described by the following DTD:
http://www.apple.com/DTDs/PropertyList-1.0.dtd
Property container objects are reference counted. When an object is cr=
e-
ated, its reference count is set to 1. Any code that keeps a reference
to an object, including the collection types (arrays and dictionaries),
must ``retain'' the object (increment its reference count). When that
reference is dropped, the object must be ``released'' (reference count
decremented). When an object's reference count drops to 0, it is auto-
matically freed.
The rules for managing reference counts are very simple:
o If you create an object and do not explicitly maintain a reference =
to
it, you must release it.
o If you get a reference to an object from other code and wish to mai=
n-
tain a reference to it, you must retain the object. You are respon-
sible for releasing the object once you drop that reference.
o You must never release an object unless you create it or retain it.
Object collections may be iterated by creating a special iterator objec=
t.
Iterator objects are special; they may not be retained, and they are
released using an iterator-specific release function.
There's also my own "figpar" free for the taking...
http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/druidbsd/figpar/
>From figpar.c
Sorry, meant to paste the explanation excerpt from figpar.c right there...
/* figpar.c
--------
figpar, the con-fig par-ser. heh
This file contains code to parse complex configuration
files. It steps through the file, reading in values for
directives declared in an array of pre-defined structur-
es outlining the anatomy of configuration options. It
uses function pointers to allow extensibility. Rather
than re-programming the function that reads and parses
the file itself, you can make functions that perform sp-
ecific tasks per directive. You can also perform a task
for unknown directives not declared in your array.
The way that this parser is written is very simple, and
how complex of a set of features you want to allow in
your file is dependent on the functions you have it exe-
cute.
It is possible to read apache style config files accura-
tely, .htaccess files, and other styles.
--
Devin
dteske@scribe9.vicor.com<mailto:dteske@scribe9.vicor.com> figpar $ wc -l *.=
h *.c
123 figpar.h
40 string_m.h
376 figpar.c
292 string_m.c
831 total
I recently had to resurrect string_m.{c,h} for another project, and made th=
em suitable for the FreeBSD tree:
http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/fdpv/
(if you're interested; really just pointing at string_m.{c,h})
I wrote figpar back in 2002 and it's pretty simple.
It parses any POSIX style file and lets *you* (as the user of figpar) decid=
e how different matches are handled. It indeed has what the OP was interest=
ed in...
Do X when you get NUM (instead of all separate types, etc.).
>From figpar.h:
=3D=3D=3D
/* cfgvalue_t is a 4 byte union that allows you to store
varied types of data in a single common container. */
typedef union {
char *str; /* a pointer to a null terminated string */
int32_t num; /* a signed 32-bit integer value */
u_int32_t boolean:1; /* boolean integer value (0 or 1) */
u_int32_t u_num; /* an unsigned 32-bit integer value */
char **strarray; /* pointer to an array of strings */
} cfgvalue_t;
=3D=3D=3D
The OP would simply bump that union to 8 bytes and make the default "num" b=
e a u_int64_t (not how there is also "u_num" etc.).
The rest is pretty simple. Of the 499 lines that comprise figpar.h and figp=
ar.c, there's only two typedefs to learn and two funtions to utilize. The r=
est you bring to the table yourself (you add functions for handling directi=
ves by "match" -- uses fnmatch(3) to match a configuration directive and fi=
re your function).
I've used this to parse pretty much any config file ever seen (from apache =
httpd.conf to jail.conf-style configuration files). HINT: You can match on =
"{" and implement a state/identifier-machine to implement groupings.
What it does is remove all the nasty details of how to parse a file and let=
you worry about what gets fired when it deems it's found a match to one of=
your definitions.
It also tries to keep things nice, as you pass it a struct-array of definit=
ions (which I find helpful because I can have a nice block-definition that'=
s centrally located for continuous munging).
Sure, it'd need some style(9) application (I wrote it in 2002), but it's al=
so extremely portable (tested successfully on over 4-dozen platforms).
As you can see from the union above, it supports most types and you can ext=
end it to support more. But in my experience, I've never needed more than t=
hat basic set.
Of course, as it is under 500 lines, it may not be as flexible as bison+yac=
c, but it's never done me wrong (and pretty easy for any person on a distri=
buted project to grasp and run-with).
My 2 cents.
--
Devin
_____________
The information contained in this message is proprietary and/or confidentia=
l. If you are not the intended recipient, please: (i) delete the message an=
d all copies; (ii) do not disclose, distribute or use the message in any ma=
nner; and (iii) notify the sender immediately. In addition, please be aware=
that any message addressed to our domain is subject to archiving and revie=
w by persons other than the intended recipient. Thank you.
_____________
The information contained in this message is proprietary and/or confidentia=
l. If you are not the intended recipient, please: (i) delete the message an=
d all copies; (ii) do not disclose, distribute or use the message in any ma=
nner; and (iii) notify the sender immediately. In addition, please be aware=
that any message addressed to our domain is subject to archiving and revie=
w by persons other than the intended recipient. Thank you.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?13CA24D6AB415D428143D44749F57D7201FC9F3C>