From owner-freebsd-stable@FreeBSD.ORG Fri Oct 29 14:53:55 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EED31065674 for ; Fri, 29 Oct 2010 14:53:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 932E48FC14 for ; Fri, 29 Oct 2010 14:53:53 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o9TErnvd009933 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 29 Oct 2010 17:53:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o9TErnX8028682; Fri, 29 Oct 2010 17:53:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o9TErnIO028681; Fri, 29 Oct 2010 17:53:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 29 Oct 2010 17:53:49 +0300 From: Kostik Belousov To: Alexander Zagrebin Message-ID: <20101029145349.GX2392@deviant.kiev.zoral.com.ua> References: <4CCACDC0.7050802@icyb.net.ua> <1BDB4D1B02274CC8AA2DD5E68190CB5D@vosz.local> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6G52ZTUPzPgnX6I2" Content-Disposition: inline In-Reply-To: <1BDB4D1B02274CC8AA2DD5E68190CB5D@vosz.local> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_05, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org, 'Andriy Gapon' Subject: Re: 8.1-STABLE: zfs and sendfile: problem still exists X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Oct 2010 14:53:55 -0000 --6G52ZTUPzPgnX6I2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 29, 2010 at 06:31:21PM +0400, Alexander Zagrebin wrote: > > > I've tried the nginx with > > > disabled sendfile (the nginx.conf contains "sendfile off;"): > > >=20 > > > $ dd if=3D/dev/random of=3Dtest bs=3D1m count=3D100 > > > 100+0 records in > > > 100+0 records out > > > 104857600 bytes transferred in 5.892504 secs (17795083 bytes/sec) > > > $ fetch -o /dev/null http://localhost/test > > > /dev/null 100% of 100=20 > > MB 41 MBps > > > $ fetch -o /dev/null http://localhost/test > > > /dev/null 100% of 100=20 > > MB 44 MBps > > > $ fetch -o /dev/null http://localhost/test > > > /dev/null 100% of 100=20 > > MB 44 MBps > > >=20 > >=20 > > I am really surprised with such a bad performance of sendfile. > > Will you be able to profile the issue further? >=20 > Yes. >=20 > > I will also try to think of some measurements. >=20 > A transfer rate is too low for the _first_ attempt only. > Further attempts demonstrates a reasonable transfer rate. > For example, nginx with "sendfile on;": >=20 > $ dd if=3D/dev/random of=3Dtest bs=3D1m count=3D100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 5.855305 secs (17908136 bytes/sec) > $ fetch -o /dev/null http://localhost/test > /dev/null 3% of 100 MB 118 kBps > 13m50s^C > fetch: transfer interrupted > $ fetch -o /dev/null http://localhost/test > /dev/null 100% of 100 MB 39 MBps >=20 > If there was no access to the file during some time, then everything > repeats: > The first attempt - transfer rate is too low > A further attempts - no problems >=20 > Can you reproduce the problem on your system? Could it be the priming of the vm object pages content ? Due to double-buffering, and (possibly false) optimization to only perform double-buffering when vm object already has some data cached, reads can prime vm object page list before file is mmapped or sendfile-ed. --6G52ZTUPzPgnX6I2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzK3/0ACgkQC3+MBN1Mb4i4xQCeNNxBh8xeQGRogNWqW+x51KCJ x10Ani8du1Qb6EW2alL0FWYvArUXkzU4 =WGoG -----END PGP SIGNATURE----- --6G52ZTUPzPgnX6I2--