Date: Sun, 15 Aug 2010 15:07:51 -0700 From: Tim Kientzle <tim@kientzle.com> To: Dimitry Andric <dimitry@andric.com> Cc: Doug Barton <dougb@FreeBSD.org>, Justin Hibbits <chmeeedalf@gmail.com>, delphij@freebsd.org, Gabor Kovesdan <gabor@freebsd.org>, Steve Kargl <sgk@troutmask.apl.washington.edu>, =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= <des@des.no>, current@freebsd.org Subject: Re: Official request: Please make GNU grep the default Message-ID: <9C0F9422-439B-4DB8-A1C4-9F1749407FC5@kientzle.com> In-Reply-To: <4C6844D8.5070602@andric.com> References: <4C6505A4.9060203@FreeBSD.org> <20100813085235.GA16268@freebsd.org> <4C66C010.3040308@FreeBSD.org> <4C673F02.8000805@FreeBSD.org> <20100815013438.GA8958@troutmask.apl.washington.edu> <4C67492C.5020206@FreeBSD.org> <B7A05068-9578-4341-851B-86BD9BC7A2DA@gmail.com> <8639ufd78w.fsf@ds4.des.no> <4C6844D8.5070602@andric.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Aug 15, 2010, at 12:49 PM, Dimitry Andric wrote: > So my first quick fix attempt was to replace the home-grown grep_fgetln > with fgetln(3), which is in libc. This does not support gzip and bzip2 > files, but just to prove the point, it is enough. It gave the following > profiling result: FYI: libarchive has some pretty heavily-optimized bulk I/O routines and handles automatic decompression (including gzip, bzip2, lzma, xz, lzip, compress, and soon uuencode). There's a trick supported in libarchive now that will let you just use it's automatic decompression features on non-archive files (via "format_raw"). Unfortunately, it provides binary blocks of data; there's no nice line-reader interface. There's an effort afoot to refactor libarchive so that the stream I/O and compression/decompression support is actually a separate library that should be very useful for this sort of usage. As part of that, we plan to add some line-oriented I/O features that should be noticeably more efficient than stdio. Cheers, Tim
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9C0F9422-439B-4DB8-A1C4-9F1749407FC5>