From owner-freebsd-stable@freebsd.org Mon Sep 5 20:41:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E770B96785 for ; Mon, 5 Sep 2016 20:41:30 +0000 (UTC) (envelope-from jim@ohlste.in) Received: from mail-qt0-x229.google.com (mail-qt0-x229.google.com [IPv6:2607:f8b0:400d:c0d::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4870FFD9 for ; Mon, 5 Sep 2016 20:41:30 +0000 (UTC) (envelope-from jim@ohlste.in) Received: by mail-qt0-x229.google.com with SMTP id p30so29717291qte.3 for ; Mon, 05 Sep 2016 13:41:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ohlste-in.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=b0tB8FLCsLXr9XHM4Dbj3e7ypdxzf7hYh+08BVysWhI=; b=EPHDP+217u6Ufxt2C0LJvD810clKmagHIc78y54bFUWxSziY7FR7qXA5OdEXYmnN+A vOqOU+zvGi4i6RJTsMrB26THNxyPsgaYNQLjp/d0E2hJ1N4JOOXvwPEqfLWYG5QSeFjK pdmsNpKD5uVFVLccYKa/f12xFo6BIZHV7HwRZahSOlGAb14M2VlQyqGn/naP7AvodEV1 1bK15cs9P1Egv43dhfeUhs4J2EYitm49Vf7ltEhlMRY2qg2oaG8bBYztUF1Q2QoPCSrZ Gn0mJipEOWb3Vgyawb5mUIs3qfyfGCN7PYxiXNpKM6eeMMm3s5fBa5DITIyfm9VQKWAl PF3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=b0tB8FLCsLXr9XHM4Dbj3e7ypdxzf7hYh+08BVysWhI=; b=Ng0oodcztq99UN700PX34FCM7qpyGU3Ur6am6XcE6tRysINP4tJe452HbiXcydxxBa Ywkh9ve8jqZXqR6Fs/FsFWJLv4RAZ3ghq9aQzyB+63tg4whI4Y0oF/DDI+Zc1A95PbfM 3fjKX+CpYU1hHzc0nPQ3b+adPwDuG0duB/GDRVdoe4cRxSxdZ7bWwcbXALS2L0JlvZ24 YGYihBg5ijDMirw6NEpOd/8+NFI9pnXgohGs+iWwUIATZo82ghx6A+sFevWHzIczBX5W YdkFmK7m6or3+bf5hOcFpOUh7JrvcpsZ5JilwW7V2eJ5fC5o82uZ5bhAQ6ObbY5uU5ra cdqg== X-Gm-Message-State: AE9vXwNzmGVxgt67R6neF4qayS59AnQ/IU5eO/xv3T2US+NHn7BqDGBhGH+uLFP6Wg4p4w== X-Received: by 10.200.51.226 with SMTP id d31mr2283614qtb.43.1473108089400; Mon, 05 Sep 2016 13:41:29 -0700 (PDT) Received: from [192.168.1.6] (pool-108-39-64-29.nrflva.fios.verizon.net. [108.39.64.29]) by smtp.gmail.com with ESMTPSA id m201sm15620201qke.44.2016.09.05.13.41.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Sep 2016 13:41:28 -0700 (PDT) Subject: Re: AIO not working on NFS share - ? Regression To: Konstantin Belousov , Alan Somers References: <6c9093cd-1323-77b1-b27e-f53b58ec86e4@ohlste.in> <20160905203412.GQ83214@kib.kiev.ua> Cc: FreeBSD Stable ML , np@FreeBSD.org From: Jim Ohlstein Message-ID: <87af720e-7506-d1de-9f93-f9d5a2589f7a@ohlste.in> Date: Mon, 5 Sep 2016 16:41:27 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160905203412.GQ83214@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2016 20:41:30 -0000 Hello, On 09/05/2016 04:34 PM, Konstantin Belousov wrote: > On Mon, Sep 05, 2016 at 02:26:30PM -0600, Alan Somers wrote: >> Just to get your webserver out of the picture, could you please try >> running the aio test suite on an NFS share? Do it like this: >> $ cd /usr/tests/sys/aio >> $ export TMPFS=/path/to/some/nfs/share >> $ kyua test > By default, unsafe aio is disabled, i.e. aio where the operation is not > really async and can usurp kernel thread for indefinite amount of time. > Non-local filesystems accesses are classified as unsafe, since their > completion is controlled by remote host. Setting vfs.aio.enable_unsafe to 1 solved the problem, though I don't recall this being necessary in the past. Thanks for the quick responses and fix. > >> >> -Alan >> >> On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein wrote: >>> Hello, >>> >>> For sometime now we've used a backend (non-internet accessible) server to >>> generate static content that is shared via NFS to a front-end server for >>> internet access. Both machines use ZFS exclusively. >>> >>> The webserver is nginx and we had AIO set to on, with the AIO module built >>> into the kernel.In 10.x it "just worked". >>> >>> About two weeks ago I upgraded that backend machine to 11-STABLE in order to >>> start building packages for 11. I've just updated it again to r305417. I >>> don't recall it not working as expected. >>> >>> More recently I upgraded the front-end machine to 11-STABLE @r305311. Since >>> then, AIO is not working and I get a critical error in the nginx logs like >>> this: >>> >>> 2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") failed >>> (45: Operation not supported) while sending response to client ... >>> >>> In 11 AIO is built into the kernel by default, and I have confirmed that it >>> works as expected in files from a local file system. Since the webserver >>> isn't aware that the file system in question is shared via NFS, it seems >>> this is an operating system issue. -- Jim Ohlstein