From owner-freebsd-current@freebsd.org Tue Jan 5 06:05:25 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9B8FA62B70 for ; Tue, 5 Jan 2016 06:05:25 +0000 (UTC) (envelope-from benno@freebsd.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C6041241 for ; Tue, 5 Jan 2016 06:05:25 +0000 (UTC) (envelope-from benno@freebsd.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id E379720C15 for ; Tue, 5 Jan 2016 01:05:23 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute6.internal (MEProxy); Tue, 05 Jan 2016 01:05:23 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:message-id :mime-version:subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=w2 JdFCKEwiW2lCZnn9jPazs9M6I=; b=XuwMXl5+dMQZGCSlRii5nG79IRg01zcHeU kBZ50SaJidDzTPh3ckhcZsjKpW9l757X2gt6Y2SKkY9Xewum/zNKBQae448Y5K+n kQxA/MweIkxWHT5jzwiR4xi1oteljadHYqDswJnWJOPAwv8aa7cOX1NavKDqmOUD lXNUSV8BQ= X-Sasl-enc: IC0W/vNGObJ13L1hYknrnsG/d5X2PHYv2lAsgX6k70Xn 1451973923 Received: from itmkpacimm1.corp.emc.com (c-67-160-26-243.hsd1.wa.comcast.net [67.160.26.243]) by mail.messagingengine.com (Postfix) with ESMTPA id 494B16800F2; Tue, 5 Jan 2016 01:05:23 -0500 (EST) From: Benno Rice Subject: Possible bug in or around posix_fadvise after r292326 Date: Mon, 4 Jan 2016 22:05:21 -0800 Message-Id: Cc: freebsd-current To: Konstantin Belousov Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) X-Mailer: Apple Mail (2.3112) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 06:05:25 -0000 Hi Konstantin, I recently updated my dev box to r292962. After doing this I attempted = to set up PostgreSQL 9.4. When I ran initdb the last phase hung. Using = procstat -kk I found it appeared to be stuck in a loop inside a = posix_fadvise syscall. I could not ^C or ^Z the initdb process. I could = kill it but a subsequent attempt to rm -rf the /usr/local/pgsql/data = directory also got stuck and was unkillable by any means. Rebooting = allowed me to remove the directory but the initdb process still hung = when I re-ran it. I tried PostgreSQL 9.3 with similar results. Looking at the source code for initdb I found that it calls = posix_fadvise like so[1]: /* * We do what pg_flush_data() would do in the backend: prefer to = use * sync_file_range, but fall back to posix_fadvise. We ignore = errors * because this is only a hint. */ #if defined(HAVE_SYNC_FILE_RANGE) (void) sync_file_range(fd, 0, 0, SYNC_FILE_RANGE_WRITE); #elif defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_DONTNEED) (void) posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED); #else #error PG_FLUSH_DATA_WORKS should not have been defined #endif Looking for recent commits involving POSIX_FADV_DONTNEED I found = r292326: https://svnweb.freebsd.org/changeset/base/292326 = Backing this revision out allowed the initdb process to complete. My current theory is that some how we=E2=80=99re getting ENOLCK or = EAGAIN from the BUF_TIMELOCK call in bnoreuselist: = https://svnweb.freebsd.org/base/head/sys/kern/vfs_subr.c?view=3Dannotate#l= 1676 = Leading to an infinite loop in vop_stdadvise: = https://svnweb.freebsd.org/base/head/sys/kern/vfs_default.c?annotate=3D292= 373#l1083 = I haven=E2=80=99t managed to dig any deeper than that yet. Is there any other information I could give you to help narrow this = down? Thanks, Benno. [1] = http://git.postgresql.org/gitweb/?p=3Dpostgresql.git;a=3Dblob;f=3Dsrc/bin/= initdb/initdb.c;h=3D35e39ce4b31b2f437d6e28eaf90500a22d229c6a;hb=3DHEAD#l63= 1=