Date: Thu, 4 Jul 2013 23:53:30 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: arch@FreeBSD.org Subject: General purpose library for name/value pairs. Message-ID: <20130704215329.GG1402@garage.freebsd.pl>
next in thread | raw e-mail | index | archive | help
--pe+tqlI1iYzVj1X/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. During my last project I took time to implement general purpose name/value pair library similar to what exists in Solaris/IllumOS. Currently we have plenty of random methods to pass data around: - nmount(2) implements its own name/value pair approach, - GEOM uses XML to pass data to userland, - various sysctls and ioctls use binary structures to export data to userland. The libnv library I implemented operates around two types: - nvpair_t that describes single name/value pair, - nvlist_t that describes a list of name/value pairs. In most cases you can do without nvpair_t. Let's try an example: nvlist_t *nvl; nvl =3D nvlist_create(0); nvlist_add_string(nvl, "filename", "/tmp/foo"); nvlist_add_int32(nvl, "flags", O_CREAT | O_WRONLY); nvlist_add_uint16(nvl, "mode", 0600); if (nvlist_send(sock, nvl) < 0) { nvlist_destroy(nvl); warn(1, "nvlist_send() failed"); return (-1); } nvlist_destroy(nvl); return (0); As you can see we first allocate nvlist and add three elements to it. Note that we neither check if allocation succeeded nor individual additions. The library is designed so that write-like operations can gracefully handle previous failures. For example if nvlist_create() fails, it returns NULL, but nvlist_add_string() detects if nvl is NULL and does nothing. If nvlist_create() succeeded, but nvlist_add_string() failed, nvlist_add_string() will set an error within nvlist structure. Once we build out nvlist we can either call nvlist_send(), which will fail (if nvl is NULL or has an error set) or we can call nvlist_error() to check for error (if nvl is NULL, ENOMEM will be returned). nvlist_destroy() also accepts NULL nvl and it never modifies global errno value for convenience (that's why warn() above will show correct error message). The API doesn't support types like 'int', 'long' or any other type with arch-dependent size. This is on purpose. I want this API to be used for communication between userland and kernel, even if userland is 32bit and kernel is 64bit, as well as for network communication. nvlist_send() internally uses nvlist_pack() function that produce binary blob to send over the network or to the kernel. It implements adaptive endianess concept known from ZFS - it doesn't convert all values to the network byte order, but it records machine's order in the header, so if receiver uses different byte order, only then swapping will be done. If both machines are either little or big endian no extra swapping will occur. The library allows to send and receive descriptors, of course only over UNIX domain sockets: nvlist_t *nvl; int fd; fd =3D open("/etc/passwd", O_RDONLY); if (fd < 0) err(1, "open(/etc/passwd) failed"); nvl =3D nvlist_create(0); nvlist_add_string(nvl, "filename", "/etc/passwd"); nvlist_move_descriptor(nvl, "fd", fd); if (nvlist_send(sock, nvl) < 0) err(1, "nvlist_send() failed"); nvlist_destroy(nvl); Also note that I used nvlist_move_descriptor() function and not nvlist_add_descriptor(). The former will allow nvlist to consume the given descriptor, so we don't have to close it, the latter will dup(2) the given descriptor and then add it to the nvlist. So, to add elements to the list one can use either nvlist_add_<type>() or nvlist_move_<type>() functions. To get the values one also have two choices: nvlist_get_<type>() and nvlist_take_<type>() - the former just returns the value (but the value is still part of the nvlist) and the latter removes associated nvpair from nvlist and returns its value. For example: nvlist_t *nvl; const char *command; char *filename; int fd; nvl =3D nvlist_recv(sock); if (nvl =3D=3D NULL) err(1, "nvlist_recv() failed"); command =3D nvlist_get_string(nvl, "command"); filename =3D nvlist_take_string(nvl, "filename"); fd =3D nvlist_take_descriptor(nvl, "fd"); printf("command=3D%s filename=3D%s fd=3D%d\n", command, filename, fd); nvlist_destroy(nvl); free(filename); close(fd); /* command was freed by nvlist_destroy() */ Of course nvlist_move_<type>() and nvlist_take_<type>() functions are only available for types that require some resources (memory or descriptor), so there is nvlist_move_string(), nvlist_move_int8_array(), but there is no nvlist_move_int8(), as it doesn't make sense. Also note that if there is no element matching the given name and type, the function will fail on internal assertion. This is caller's responsibility to verify if the given element is on the list before trying to get it using nvlist_exists_<type>() API: if (nvlist_exists_string(nvl, "command")) command =3D nvlist_get_string(nvl, "command"); else command =3D NULL; To iterate over all elements one can do the following: nvlist_t *nvl; nvpair_t *nvp; nvl =3D nvlist_recv(sock); if (nvl =3D=3D NULL) err(1, "nvlist_recv() failed"); for (nvp =3D nvlist_first_nvpair(nvl); nvp !=3D NULL; nvp =3D nvlist_next_nvpair(nvl, nvp)) { printf("name=3D%s type=3D%d\n", nvpair_name(nvp), nvpair_type(nvp)); } For every function that takes name as an argument there are two more functions that allow to provide name as format string and arguments or as va_list, for example: int16_t nvlist_get_int16(const nvlist_t *nvl, const char *name); int16_t nvlist_getf_int16(const nvlist_t *nvl, const char *namefmt, ...) _= _printflike(2, 3); int16_t nvlist_getv_int16(const nvlist_t *nvl, const char *namefmt, va_lis= t nameap) __printflike(2, 0); The whole implementation can be found here: http://people.freebsd.org/~pjd/libnv.tgz Only the nv.h header file is public, so yes, nvlist and nvpair structures are not exposed to callers. I'd be grateful for opinions. This work was sponsored by the FreeBSD Foundation. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --pe+tqlI1iYzVj1X/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iEYEARECAAYFAlHV7tkACgkQForvXbEpPzQ/0gCgscOMG5L/qrK0k+cVIvr4wguR OYEAoOPt682UHpnFIUfkeTrRJ99sciAK =gw5Q -----END PGP SIGNATURE----- --pe+tqlI1iYzVj1X/--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130704215329.GG1402>