From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 09:10:00 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6184CDB; Tue, 3 Mar 2015 09:09:59 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A8C91952; Tue, 3 Mar 2015 09:09:58 +0000 (UTC) Received: from [192.168.0.7] (cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t2399nsb032154 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 3 Mar 2015 09:09:51 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <54F50F15.4050300@freebsd.org> Date: Tue, 3 Mar 2015 09:09:43 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <54F504EF.3030303@freebsd.org> <54F50F15.4050300@freebsd.org> To: Andrey Chernov X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current@freebsd.org, Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 09:10:00 -0000 On 3 Mar 2015, at 01:32, Andrey Chernov wrote: >=20 > So, why you ever need to modify wc? Just load wc inside your > json/xml/etc writer, replacing its printf at the ld-elf.so level. You can't get structured output from printf() because printf() takes = unstructured input. It's a string with some variables pasted in, but no = awareness of context. The libxo changes to the tools are simply marking the output as having a = defined structure. The library then translates this abstract structure = into something that can be parsed easily by external tools. If your argument is that the UNIX philosophy is simple tools doing one = thing well, then please remember the context of this philosophy: It = dates back to the original UNIX systems *that did not support shared = libraries* and was an argument used to justify not implementing them. = This is why globbing is in the shell instead of a shared library and why = some variant of mv *.a *.b works on every command-line interface except = for a UNIX shell. Even with that in mind, small changes to individual tools are a *lot* = simpler than one massive monolithic tool that understands the output = formats of every other tool in the base system and can transform them. = Why do you think a few library calls in each application is more complex = *than an entire parser per tool*? I am reminded of the time about ten years ago when I was writing a = system info widget for a DE. Getting the total amount of RAM, number of = cores, and CPU speed on FreeBSD, OpenBSD, NetBSD, and Solaris proved to = be a single sysctl for each attribute. On Linux, the information was = all there, in /proc/cpuinfo, so I had to write a parser. Unfortunately, = the format changed across kernel revisions and across architectures. In = the end, I had to link in a 300KB library to parse information that the = kernel already had in a machine-readable format and present it to me in = a machine-readable format. Hopefully there's a lesson here that we can learn from: human-readable = formats do not make good intermediate representations when communicating = between tools. If your argument is about maintainability of these changes, then please = point to concrete instances where the changes are complex and difficult = to maintain. If your argument is about binary size, then it would be relatively easy = for us to add a version of libxo for static linking into the versions in = /rescue that only supported plain-text output, but again, please = quantify your objections: What is the size increase that you're seeing = and what kind of devices is this causing a problem for? The general approach of making tools produce machine-readable output has = been under discussion for *years* and the integration of libxo has been = underway for several months, prior to which it was used inside Juniper. = If you had a better approach, then the correct time to raise it was back = when the requirement to produce machine-readable output was raised. The = subject of this thread is just plain trolling (is everything broken by = libxo? No. Is anything broken by libxo? Occasionally, but new = features often come with temporary breakage that's found and fixed in = testing: that's why we have -CURRENT). David