Date: Sun, 4 Mar 2018 12:48:28 -0700 From: Warner Losh <imp@bsdimp.com> To: Ian Lepore <ian@freebsd.org> Cc: bob prohaska <fbsd@www.zefox.net>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: Is maximum swap usage tunable? Message-ID: <CANCZdfrw_YSELHMLLykKJfG4FjTPaFiRJYePasrF%2B7Ma5J%2BZ1Q@mail.gmail.com> In-Reply-To: <1520189171.38056.2.camel@freebsd.org> References: <20180228170311.GA26187@www.zefox.net> <a759ecea-83f4-b0b2-7113-c39633f68637@mgm51.com> <20180228185517.GB26187@www.zefox.net> <8f422161-885e-aa91-eacd-018540222d65@mgm51.com> <20180228214301.GA29481@www.zefox.net> <b82801b8-bc29-414c-1170-621bb4a5d937@mgm51.com> <CANCZdfq_MpxjUyVd-=%2BMiAQAER4TeDh9irhx_evdXwa3yt3h0g@mail.gmail.com> <20180303162605.GA41874@www.zefox.net> <20180304182831.GA44154@www.zefox.net> <CANCZdfpcccDuhJh4EF0C4KFz=dcN5mWvQ9aMiyF53kD=hMzy3Q@mail.gmail.com> <1520189171.38056.2.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Mar 4, 2018 at 11:46 AM, Ian Lepore <ian@freebsd.org> wrote: > On Sun, 2018-03-04 at 11:35 -0700, Warner Losh wrote: > > On Sun, Mar 4, 2018 at 11:28 AM, bob prohaska <fbsd@www.zefox.net> > wrote: > > > > > > > > On Sat, Mar 03, 2018 at 08:26:05AM -0800, bob prohaska wrote: > > > > > > > > > > > > Is there some sort of experiment which can distinguish hardware > delays > > > > from software delays? For example, would logging gstat output shed > any > > > > light? > > > > > > > For lack of any better ideas, I tried running > > > make -j2 -DNO_CLEAN buildworld > buildworld.log && make -j2 -DNO_CLEAN > > > KERNCONF= > > > ZEFOX buildkernel > buildkernel.log > > > > > > while also running > > > gstat -a -B -I 10s > j2_gstat.log & in another ssh session > > > > > > In due course the console reported > > > > > > FreeBSD/arm64 (www.zefox.org) (ttyu0) > > > > > > login: Mar 4 09:28:30 www kernel: pid 9310 (c++), uid 0, was killed: > out > > > of swap space > > > > > > as expected. > > > > > > However, a grep of j2_gstat revealed a maximum write delay of 30ms/w > > > for swap on microSD. > > > > > > Swap on USB flash is slower, but still generally under 100 ms. > > > Only a handful of widely spaced delays exceeded 200 ms/w. > > > > > gstat doesn't tell you the worst-case. It tells you the average of the > > requests. This can (and does) hide very long outlier behavior which > drives > > the crazy delayed swap messages. > > > > The worst-case events were > > > > > > dT: 10.002s w: 10.000s > > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > > > > > > 0 6 0 0 0.0 6 113 14.6 3.3 da0b > > > 0 4 0 0 0.0 4 48 29.0 3.1 da0b > > > 0 9 5 79 3.0 5 47 7.9 2.6 da0b > > > 4 8 0 0 0.0 8 99 67.5 32.5 da0b > > > 0 1 0 13 5.6 1 28 5674 88.3 da0b > > > 0 0 0 0 0.0 0 38 18.6 0.3 da0b > > > 0 1 1 9 2.8 0 0 0.0 0.2 da0b > > > 0 1 1 26 5.1 0 0 0.0 0.4 da0b > > > 0 0 0 3 2.6 0 0 0.0 0.1 da0b > > > 0 1 1 9 161.8 0 0 0.0 14.3 da0b > > > > > > No "indefinite delay" warnings were presented on the console. > > > uname -a reports r329893, sources are at 330383. > > > > > > I hope this is useful information, > > > > > Despite my quibble over what you're measuring (I look at this stuff all > the > > time for Netflix), I think this is quite useful.... Thanks! > > > > I think it would be more useful with a -I1 instead of 10. Including > the -d flag might also be useful, since deleting is what's so slow on > flash-based devices (I'm not sure if swapping triggers BIO_DELETE > stuff, but other filesystem activity does). > BIO_DELETE is generated only by filesystems. Nobody has thought through the best strategies to swap to flash. The swap pager doesn't issue BIO_DELETE at all. Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrw_YSELHMLLykKJfG4FjTPaFiRJYePasrF%2B7Ma5J%2BZ1Q>