From owner-freebsd-arm@freebsd.org Tue Jun 19 19:10:17 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAAEC100892E for ; Tue, 19 Jun 2018 19:10:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E99B71266 for ; Tue, 19 Jun 2018 19:10:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x232.google.com with SMTP id j135-v6so2007071itj.1 for ; Tue, 19 Jun 2018 12:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=PEFz7wWXV2twl9zgX/ThTl4OwbiL+qoQCkRD7u4kcdE=; b=s4h2/nexdCR6PNXqZyoR8Gwj4dlXUYWgIfL627QBWAXIg+hedEnzFf90Xy58LgUyD3 B2psKw6fjB4+EPmWNE4qG9QRnz3p2ITNBJLUV5usak9CbWnKiOcPMI/WCb1hSUKT+Xr+ eN/tsirQArFy8Zi9NPmkUJj+objE/CkDvMw7b8x9Ts6WbdS2SUDYVwT8TMgeWRy1BBrd hkN7Z+7KRKviHb65KcFevZXTFTGRe2wxwBzv3Bihqva3ia2Jp3FasnmL6lbe1Z750Mts YmtLvbpq8JXgxgnHS5Q3uhP+4TTc8G9ZbRoLswwuYWgmrrvxTHfXYtD2iJBz4+9+HsJf L13g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=PEFz7wWXV2twl9zgX/ThTl4OwbiL+qoQCkRD7u4kcdE=; b=AGRskem8q+RuayCeZu1eDdKU2cYkqCP8NbdaBrz6/ApzKz6d34At2Id3tdnhJr7fbt 7ejU7UbkKwr7rK6VsdEwKHJgYEWkJF2A5gxPj83z7r1Hz6Qqf58PRsK2wjcdyhHGf2Ut Z0pid4zLZ4oS8HEzMnRJjfWmr5410ZHDQcfAFb8ka8p49cIcSn2lUsl9uw9TAOJYdTBA 67zISlptpUi83BQy8zfPSl5nIbfN7a7CWMTgrR0Qk9u5G6m9/X2tlAduPKobxMIOVnir aCZsa/O/Mqs26E/3Rcc2KsAaW46C9Fti5Yt18lMqYF/KT8MGoy46hNweYod6KmM3DY60 1P0Q== X-Gm-Message-State: APt69E02S8t53DYBO06YrhxqLqi2HJaDg4BNk9rZyifmTknBWzP49WZF XFUYJYExHRt8MlyBHi59saR3DOK7u11nl8TZC/ARRA== X-Google-Smtp-Source: ADUXVKJvL+QL8elY/UDBS0sv76+DiyJrbl3aQ2DH6rGhA2YC9NrS3dyKvZagN+DEM0mrw5GtDTeDGLqqyCnXpvo7Kgc= X-Received: by 2002:a02:6348:: with SMTP id j69-v6mr15146346jac.45.1529435416460; Tue, 19 Jun 2018 12:10:16 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:5945:0:0:0:0:0 with HTTP; Tue, 19 Jun 2018 12:10:15 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: <20180614175622.GC35161@www.zefox.net> <201806142110.w5ELAL0N046840@pdx.rh.CN85.dnsmgr.net> <20180615035225.GA37370@www.zefox.net> From: Warner Losh Date: Tue, 19 Jun 2018 13:10:15 -0600 X-Google-Sender-Auth: uKGH4OpjYjdAG2cyAZzZtZn1BdQ Message-ID: Subject: Re: GPT vs MBR for swap devices To: bob prohaska Cc: "Rodney W. Grimes" , "freebsd-arm@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jun 2018 19:10:18 -0000 On Thu, Jun 14, 2018 at 10:00 PM, Warner Losh wrote: > > > On Thu, Jun 14, 2018, 9:52 PM bob prohaska wrote: > >> On Thu, Jun 14, 2018 at 02:10:21PM -0700, Rodney W. Grimes wrote: >> > >> > It might be interesting to do in order the swapon >> > commands to 1G USB flash, 1G SD flash, 2G SD flash, >> >> It seems clear that USB flash swap, alone or in any >> combination, fails early in buildworld. > > > I think that's because USB flash can't swap fast enough to keep up with > the page demand. You might be able to confirm this by looking at the write > rates to the swap portions for the various other media with gstat. I > suspect it's FTL is doing more expensive garbage collection under a swap > work load leading to long pauses from time to time that the VM system > responds to by starting OOM too soon. > Looking at the data posted, I see that we have a 2s latency averaged over 10s. This means that the drive is basically unresponsive. So the average latency is 2s. That means that the max latency is likely way more than that (likely as much as 10-20s if my experience with SSD latency distributions can be trusted). The latency bounces around a bit (and there appears to be some missing data), but this is what I expected to see. Now, maybe we have an issue with the USB stack that's causing this (missed interrupts leading to polling 'saving' the day after a massively long time, for example), or the USB drives are as bad as I've experiences them to be. In any event, if we can't retire the dirty pages fast enough, we'll get into OOM to try to cope with too many dirty pages. The different control loops in the system to moderate these things have some 'hidden' assumptions that we can launder pages faster than this, so I think we're falling off the rails when we can't, even when we have available swap space. Warner