From owner-freebsd-doc@freebsd.org Sun May 21 22:50:51 2017 Return-Path: Delivered-To: freebsd-doc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0267FD7773E for ; Sun, 21 May 2017 22:50:51 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E62A81378 for ; Sun, 21 May 2017 22:50:50 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id DCA3DD77739; Sun, 21 May 2017 22:50:50 +0000 (UTC) Delivered-To: doc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBFA8D77736; Sun, 21 May 2017 22:50:50 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id A955B1375; Sun, 21 May 2017 22:50:50 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from ford.home.vangyzen.net (unknown [76.164.15.242]) by smtp.vangyzen.net (Postfix) with ESMTPSA id A5A925646B; Sun, 21 May 2017 17:50:49 -0500 (CDT) Subject: Re: The futur of the roff toolchain To: Baptiste Daroussin , arch@freebsd.org Cc: doceng@freebsd.org, doc@FreeBSD.org, current@FreeBSD.org References: <20170521125733.bmd4tmq6iqpsbvgl@ivaldir.net> From: Eric van Gyzen Message-ID: Date: Sun, 21 May 2017 17:50:46 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0 MIME-Version: 1.0 In-Reply-To: <20170521125733.bmd4tmq6iqpsbvgl@ivaldir.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-doc@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Documentation project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 May 2017 22:50:51 -0000 I like all of this. Thanks for your very thorough research and effort. Eric On 05/21/2017 07:57, Baptiste Daroussin wrote: > Hi all, > > I have been working for a while to try to import a modern roff toolchain into > base. > > I didn't like the initial approach that consisted in simply removing all roff > toolchain in base. > > Recap of the situation in base: > * We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug > fixes has been made upstream in newer version (GPLv3) in particular regarding > unicode but not only. (and we cannot update it anymore) > * GNU roff is now only used to generate the documentation in share/doc and as a > fallback for manpages which mandoc does not support. > > On the manpages front: > * No manpages in base are not supported by mandoc except groff manpages > themselves > * man(1) can fallback on ports version of groff if installed (for ports not > providing manpages not compatible with mandoc) > > Alternatives to GNU roff: > * Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C) > * neartoff http://litcave.rudi.ir/ BSD licensed (in C) > > I went the road of using heirloom doctools it is 90% compatible with GNU roff, > good enough for all our base roff based documents. > > After getting down that road for a while, including lots of patches sent > upstream (thanks them for being so reactive and integrating them quickly as well > as fixing the issues I wasn't able to fix myself quickly). > > The problem is there are lot of corner small corner cases where heirloom is > different from GNU roff and hard to make it compatible. While this is corner > cases, it breaks document generation for some large documents people are > writing. Those users could use (and actually would benefit a lot from it) GNU > roff from the ports tree, but have to be careful about the path of the tool they > call to ensure only calling the one from GNU roff and not the one (with the same > name) from heirloom doctools. > > Concerning neatroff it is barely compatible with GNU roff, so not an option > (last I tested at least). > > I would like to change this approach and get back to the initial approach taken > by others before I jumped in and I would like just entirely remove the roff > toolchain from base and let people rely on GNU roff from ports. > > man(1) is already asking the user to install groff from ports if the manpage > cannot be read with mandoc. > > No the problem left is documentations available in share/doc. > > I would like to push them elsewhere. Those documents are mostly useful for > historical reason (hence we want to keep them) but not really for daily use of > modern FreeBSD. > Another issue with those documentation, they are installed as text/ascii version > in base, which makes most of them not really readable (as the documents has not > be written for a ascii/text target but more for a PDF/html view - using pic(1) > for example) > > A plan was to push as sources in the svn doc repository and continue to build > them. This approach also have an issue: over the time roff evolved a bit and > while working on heirloom doctools import I had to fix a bunch of markup to make > the rendering of those documents clean (also meaning almost noone should read > them considering some were not really readable). > > What I want to propose now, it to render them as PDF (html?) once and push them > somewhere (to be defined) as static document on our documentation website. > Please doceng@ provide me a location where to push them. > > And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff. > I also want to remove most of roff related tools (the one provided by toolchains > available in ports) for which we kept a BSD version (not really maintained in > base): > namely: > - checknr > - vgrind > - colcrt > > Only keeping: > - col (useful in other places than roff) > - soelim (also used for manpages and we have a clean BSD licensed version which > is also now parts of mandoc) > > Best regards, > Bapt