From owner-freebsd-current@FreeBSD.ORG  Thu Jul 11 10:17:29 2013
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id 05554751;
 Thu, 11 Jul 2013 10:17:29 +0000 (UTC)
 (envelope-from glebius@FreeBSD.org)
Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10])
 by mx1.freebsd.org (Postfix) with ESMTP id 827351FDE;
 Thu, 11 Jul 2013 10:17:27 +0000 (UTC)
Received: from cell.glebius.int.ru (localhost [127.0.0.1])
 by cell.glebius.int.ru (8.14.7/8.14.7) with ESMTP id r6BAHKt7007658;
 Thu, 11 Jul 2013 14:17:20 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
Received: (from glebius@localhost)
 by cell.glebius.int.ru (8.14.7/8.14.7/Submit) id r6BAHKVX007657;
 Thu, 11 Jul 2013 14:17:20 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to
 glebius@FreeBSD.org using -f
Date: Thu, 11 Jul 2013 14:17:20 +0400
From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Adrian Chadd <adrian@freebsd.org>
Subject: Re: hacking - aio_sendfile()
Message-ID: <20130711101720.GZ67810@FreeBSD.org>
References: <CAJ-Vmo=icr6bda+WMNUarc3WbdqJ+MdauX6kByxxdTx8oSovBg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline
In-Reply-To: <CAJ-Vmo=icr6bda+WMNUarc3WbdqJ+MdauX6kByxxdTx8oSovBg@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-current <freebsd-current@freebsd.org>
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 11 Jul 2013 10:17:29 -0000

  Adrian,

On Wed, Jul 10, 2013 at 04:36:23PM -0700, Adrian Chadd wrote:
A> I've started writing an aio_sendfile() syscall.
A> 
A> http://people.freebsd.org/~adrian/ath/20130710-aio-sendfile-3.diff
A> 
A> Yes, the diff is against -HEAD and not stable/9.
A> 
A> It's totally horrible, hackish and likely bad. I've only done some
A> very, very basic testing to ensure it actually works; i haven't at all
A> stress tested it out yet. It's also very naive - I'm not at all doing
A> any checks to see whether I can short-cut to do the aio there and
A> then; I'm always queuing the sendfile() op through the worker threads.
A> That's likely stupid and inefficient in a lot of cases, but it at
A> least gets the syscall up and working.
A> 
A> I'd like some feedback and possibly some help in stress testing it to
A> make sure it's functioning well.

Apart from problem pointed out by Kostik, there is a race between
aio thread starting with aio_process_sendfile() and file descriptor
(or socket descriptor) going away.

Thus, kern_sendfile() needs to be split into two parts: kern_sendfile_pre()
and kern_sendfile() that should contain only the sending cycle.

The kern_sendfile_pre() should contain:

fgetvp_read(uap->fd, &vp)
vm_object_reference_locked(vp->v_object)

Referencing the socket is probably also required. Current synchronous
code doesn't do it.

The do_sendfile() function should call kern_sendfile_pre() and then
kern_sendfile(). The aio code should perform kern_sendfile_pre() in the
new syscall itself in context of calling process, and kern_sendfile()
in async context.

P.S. Some time ago I have started hacking on the above.

-- 
Totus tuus, Glebius.