From owner-freebsd-current@freebsd.org Wed Jun 6 23:57:57 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51944FE7CD4; Wed, 6 Jun 2018 23:57:57 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.zefox.org", Issuer "www.zefox.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A6B6172675; Wed, 6 Jun 2018 23:57:56 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.15.2/8.15.2) with ESMTPS id w56Nw2eg091352 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 6 Jun 2018 16:58:03 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.15.2/8.15.2/Submit) id w56Nw2fY091351; Wed, 6 Jun 2018 16:58:02 -0700 (PDT) (envelope-from fbsd) Date: Wed, 6 Jun 2018 16:58:01 -0700 From: bob prohaska To: Ronald Klop Cc: freebsd-arm@freebsd.org, FreeBSD Current , bob prohaska Subject: Re: panic: Assertion td->td_lock == TDQ_LOCKPTR(tdq) failed at /usr/src/sys/kern/sched_ule.c:2137 Message-ID: <20180606235801.GA87339@www.zefox.net> References: <20180601154153.GA62632@www.zefox.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2018 23:57:57 -0000 On Wed, Jun 06, 2018 at 08:55:39PM +0200, Ronald Klop wrote: > On Sat, 02 Jun 2018 13:40:27 +0200, Ronald Klop > wrote: > > > How do you ever run a -j4 buildworld? My RPI3 starts building clang/llvm > with sometimes 500 MB+ per process so everything starts swapping like hell > and takes forever to run. > Lately, never 8-) When I started playing with an RPI3, in late 2016, -j4 buildworlds worked usably well. Early in 2018 problems appeared, including Assertion td->td_lock == TDQ_LOCKPTR(tdq) failed, among others. Things didn't really go to pot until somewhat later when the swap frenzy issue reared its head and haven't improved much. Sadly, when the swap frenzy workaround of using sysctl vm.pageout_update_period=0 was suggested, a -j4 buildworld then resorted to the old td_lock issue, so it looks as if both bugs are alive and kicking. Just to complicate matters, I was in the habit of using a USB flash drive as both an outboard file system (/usr/, /var/ and /tmp/) and as a swap device. A very common reaction was to blame the flash device for the trouble, though so far as I can tell a Sandisk Extreme USB flash drive isn't much slower, if any, than a mechanical hard disk for random writes. The same USB flash devices on a Pi2 running 11-Stable seems to be fine. However, turning off the USB flash swap device does seem to reduce the number of "indefinite wait buffer" messages on the console (they're usually not fatal) so I think there is still something amiss. Whether it's the flash, the USB or the VM system is unclear to me. For now the workarounds are to run buildworld with no explicit -j value (presumably equivalent to -j1), to use only swap on the microSD card and to use the -DNO_CLEAN option for most buildworld sessions, doing an explict "make clean" or "rm -rf /usr/obj/usr" when necessary. In a few cases it seemed helpful to start with "make kernel-toolchain" then follow with make -DNO_CLEAN buildworld" but I didn't keep good enough records to be certain of the benefits. Apologies for the length, HTH bob prohaska