From owner-freebsd-current@freebsd.org Tue Nov 3 15:05:51 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B768A25B65 for ; Tue, 3 Nov 2015 15:05:51 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 044EE1A8C for ; Tue, 3 Nov 2015 15:05:51 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 03C35A25B64; Tue, 3 Nov 2015 15:05:51 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 035EEA25B63 for ; Tue, 3 Nov 2015 15:05:51 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1.freebsd.org (Postfix) with SMTP id B818B1A8B for ; Tue, 3 Nov 2015 15:05:50 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: (qmail 20279 invoked by uid 99); 3 Nov 2015 15:05:49 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Nov 2015 15:05:49 +0000 Received: from [192.168.0.103] (unknown [181.55.232.163]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id CE9EF1A01DC; Tue, 3 Nov 2015 15:05:48 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: [CFT] Unicode collation string and reworked locale definitions From: Pedro Giffuni In-Reply-To: <20151103071758.GC31432@ivaldir.etoilebsd.net> Date: Tue, 3 Nov 2015 10:05:45 -0500 Cc: current@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20151103071758.GC31432@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2015 15:05:51 -0000 Hi Baptiste; > Il giorno 03/nov/2015, alle ore 02:17, Baptiste Daroussin = ha scritto: >=20 > On Mon, Nov 02, 2015 at 06:59:15PM -0500, Pedro Giffuni wrote: >> First of all, congratulations to Baptiste and Marino for succeeding = where >> I failed many moons ago. Also huge thanks to Nexenta and Garret = D=E2=80=99Amore >> for relicensing localedef for us. >>=20 >> Concerning regex; >>=20 >> Gabor@ did a lot of work on libtre but according to him it was not up = to the >> task performancewise. We would also lose features if we move to = libtre. >>=20 >> I think our regex code actually has most of what is needed for = multibyte >> already. I have a patch that turns on the functionality but I = haven=E2=80=99t found >> any brave soul that will do the testing: >>=20 >> https://people.freebsd.org/~pfg/patches/regex-multibyte.diff >>=20 > I think it this can be tested once the collation branch is merged. Absolutely: support for collation is critical and badly needed even = without resolving the regex issues. > Note that > dragonfly and musl libc both uses a patched version of libtre for the = regex > implementation. >=20 I am aware. Also note that Gabor had some patches too, in order to make it usable for bsdgrep: https://wiki.freebsd.org/Regex > =46rom my non scientific testing libtre was more reliable and = performant then our > current regex. According to Gabor, the general performance was better until you take = into account multibyte support where it was clearly inferior to GNU regex. > Anyway it will be relatively "easy" to test using the AT&T > testsuite the reliability and performance of both implementations: = ours + your > patch and patched libtre. >=20 What worries me about libtre is that it lacks important functionality = like word delimiters. We even brought the sysv delimiters to be more compatible = with Solaris and GNU and we can=E2=80=99t back those out now: https://svnweb.freebsd.org/base?view=3Drevision&revision=3D268066 Pedro.