From owner-freebsd-stable@freebsd.org Tue Sep 6 16:27:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76E39BC7341 for ; Tue, 6 Sep 2016 16:27:40 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x231.google.com (mail-vk0-x231.google.com [IPv6:2607:f8b0:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DE25E14; Tue, 6 Sep 2016 16:27:40 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x231.google.com with SMTP id w64so16086510vkh.3; Tue, 06 Sep 2016 09:27:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=uMyT/+QjiX51DGmr9I03IPSLn3253prXogjbqkuGBBI=; b=ntEp5Guxu2U1cUVvywYCrvZ8hFQDrFeVLaFFlZeC/e4ZLvp50JAMuavAyo/pXkVVBI M8zsSRIa2W5fdOu9cFcX/jlYYRs8Sky2YXgU96VQbtX1Zmok0cXVp0MZMOywdGOlYVGF Zz27X74OYMY1ISXafcFN3OWUR3lMXhrlnMw2/LI/nv0KNjDN326fReMFQOrGwRmBly+/ dbtedpbft/5t18ESrvCNc6kzUo7IOukEOU4pLwpmVhRznOXP+K4VKnWbyjxW9oPWUhs4 x8GnuQXRbVWlvLVDNRu4qFGdYdDaQCxZHQ1bzIgPVZz22BSooKG4telIoQa7cXz8osDC QuGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=uMyT/+QjiX51DGmr9I03IPSLn3253prXogjbqkuGBBI=; b=eSSpg6IaVYMzQdyB54WjcjTrHi/eeyZmy/fKzPT2S8MTSD5HLt9rA+rQGB+a2/t62V 3IjW8q3INsRXp86AvaZtkSyralojdBd9znEC2HTw9rJJZDarrVlzzsDR4naW+9TuWCkW 1tGbHB+RZTq2jtWvLDVu3U5+hIDzh7UdXM4g7Kl474fRrhOU9uIcU+8aa7hxU515TUpQ fnVupSYa63nE+p51+DhrxANo5IbTE+ENCjgRLLi1dKcLitwDaKPLZkKOIcnuXE0+B3Ov 2PvOiAjK6n1ygsipEd6bXt974Yf+L2TNFdQT+s1V2ExKrRa8JGR3M7lYwIuj7QzshHKU NuzA== X-Gm-Message-State: AE9vXwPl/G7uAMNDtL/Ozkjx2xKlM0hJTVbQSNBPJ2AKoag4GD1nS9ArXux59o+HFrEN+PXXeonksL+RHIhxeA== X-Received: by 10.31.206.68 with SMTP id e65mr21810667vkg.129.1473179259338; Tue, 06 Sep 2016 09:27:39 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.35.200 with HTTP; Tue, 6 Sep 2016 09:27:38 -0700 (PDT) In-Reply-To: <87af720e-7506-d1de-9f93-f9d5a2589f7a@ohlste.in> References: <6c9093cd-1323-77b1-b27e-f53b58ec86e4@ohlste.in> <20160905203412.GQ83214@kib.kiev.ua> <87af720e-7506-d1de-9f93-f9d5a2589f7a@ohlste.in> From: Kevin Oberman Date: Tue, 6 Sep 2016 09:27:38 -0700 X-Google-Sender-Auth: Gkys6JrtE-e7amkqINyzs0ttHXM Message-ID: Subject: Re: AIO not working on NFS share - ? Regression To: Jim Ohlstein Cc: Konstantin Belousov , Alan Somers , FreeBSD Stable ML , np@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 16:27:40 -0000 On Mon, Sep 5, 2016 at 1:41 PM, Jim Ohlstein wrote: > Hello, > > On 09/05/2016 04:34 PM, Konstantin Belousov wrote: > >> On Mon, Sep 05, 2016 at 02:26:30PM -0600, Alan Somers wrote: >> >>> Just to get your webserver out of the picture, could you please try >>> running the aio test suite on an NFS share? Do it like this: >>> $ cd /usr/tests/sys/aio >>> $ export TMPFS=/path/to/some/nfs/share >>> $ kyua test >>> >> By default, unsafe aio is disabled, i.e. aio where the operation is not >> really async and can usurp kernel thread for indefinite amount of time. >> Non-local filesystems accesses are classified as unsafe, since their >> completion is controlled by remote host. >> > > Setting vfs.aio.enable_unsafe to 1 solved the problem, though I don't > recall this being necessary in the past. > > Thanks for the quick responses and fix. > > > >> >>> -Alan >>> >>> On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein wrote: >>> >>>> Hello, >>>> >>>> For sometime now we've used a backend (non-internet accessible) server >>>> to >>>> generate static content that is shared via NFS to a front-end server for >>>> internet access. Both machines use ZFS exclusively. >>>> >>>> The webserver is nginx and we had AIO set to on, with the AIO module >>>> built >>>> into the kernel.In 10.x it "just worked". >>>> >>>> About two weeks ago I upgraded that backend machine to 11-STABLE in >>>> order to >>>> start building packages for 11. I've just updated it again to r305417. I >>>> don't recall it not working as expected. >>>> >>>> More recently I upgraded the front-end machine to 11-STABLE @r305311. >>>> Since >>>> then, AIO is not working and I get a critical error in the nginx logs >>>> like >>>> this: >>>> >>>> 2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") >>>> failed >>>> (45: Operation not supported) while sending response to client ... >>>> >>>> In 11 AIO is built into the kernel by default, and I have confirmed >>>> that it >>>> works as expected in files from a local file system. Since the webserver >>>> isn't aware that the file system in question is shared via NFS, it seems >>>> this is an operating system issue. >>>> >>> > -- > Jim Ohlstein > Could this also be related to AIO hanging VBox VMs? See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212128 -- Kevin Oberman, Retired Network Engineer