Date: Sat, 17 Feb 2018 10:26:43 -0800 From: bob prohaska <fbsd@www.zefox.net> To: Ian Lepore <ian@freebsd.org> Cc: freebsd-arm@freebsd.org, bob prohaska <fbsd@www.zefox.net> Subject: Re: Pi3 out of swap at < 50% Message-ID: <20180217182643.GC93736@www.zefox.net> In-Reply-To: <1518889781.91697.3.camel@freebsd.org> References: <20180217162732.GA93736@www.zefox.net> <1518885801.91697.2.camel@freebsd.org> <20180217173414.GB93736@www.zefox.net> <1518889781.91697.3.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Feb 17, 2018 at 10:49:41AM -0700, Ian Lepore wrote: > On Sat, 2018-02-17 at 09:34 -0800, bob prohaska wrote: > > On Sat, Feb 17, 2018 at 09:43:21AM -0700, Ian Lepore wrote: > > > > > > On Sat, 2018-02-17 at 08:27 -0800, bob prohaska wrote: > > > > > > > > Running make -j4 buildworld on a Pi3 at r329360 tends to result > > > > in??messages similar to: > > > > > > > > pid 33492 (llvm-tblgen), uid 0, was killed: out of swap space > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 11764, size: 28672 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 16080, size: 4096 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 15721, size: 20480 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28139, size: 65536 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 40544, size: 4096 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 58384, size: 65536 > > > > pid 49735 (c++), uid 0, was killed: out of swap space > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28470, size: 4096 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 24889, size: 8192 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28736, size: 4096 > > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 29736, size: 4096 > > > > > > > > even though swap usage appears to be less than one GB out of two available. > > > > > > > > So far it's been possible to use the -DNO_CLEAN option with make to pick > > > > up where things left off and I've backed down to -j3 to see if that helps. > > > > > > > > There are two 1 GB swap partitions, one on USB flash and one on the microSD > > > > card and both appear to be in use according to swapinfo. There are no warnings > > > > in the boot messages and no explicit changes have been made to configuration > > > > files apart from adding the swap entries to /etc/fstab. > > > > > > > > World and kernel are about two weeks out of sync, so top isn't perfectly > > > > up-to-date with this kernel; could that account for the mismatch between > > > > apparent swap usage and the "out of swap" messages? > > > > > > > > Thanks for reading, and any ideas! > > > > > > > > bob prohaska > > > I suspect your swap devices are too slow to keep up with instantaneous > > > demands. ?An sdcard can easily have read and write latencies as big as > > > 30 seconds; they make pretty poor swap devices (have gstat running in > > > another window during the build to see what I mean, look at the ms/r > > > and ms/w columns). ?USB flash drives may not be much better. > > > > > I'm watching gstat now. With make -j3 running and three cc processes in > > the top of top the usual value of ms/r and ms/w is less than 5 ms. An > > eyeball average is probably less than 10 ms. Before cc got going (make > > dominant in top) there were very brief excursions to 2000 ms, as you posit, > > but they didn't last and didn't put any errors on the console.? > > > > The experiment ended when make stopped with: > > > > /usr/src/lib/libc/rpc/auth_time.c:115:10: error: unknown type name 'endpoint' > > free_eps(endpoint eps[], int num) > > ?????????^ > > /usr/src/lib/libc/rpc/auth_time.c:143:8: error: unknown type name 'nis_server' > > static nis_server * > > ???????^ > > /usr/src/lib/libc/rpc/auth_time.c:144:49: error: unknown type name 'nis_server' > > get_server(struct sockaddr_in *sin, char *host, nis_server *srv, > > ????????????????????????????????????????????????^ > > /usr/src/lib/libc/rpc/auth_time.c:145:5: error: unknown type name 'endpoint' > > ????endpoint eps[], int maxep)? > > > > among others. > > > > I'll update sources and restart buildworld with -j4, watching gstat more > > carefully.? > > > > Thanks very much for the hint! > > > > bob prohaska > > Part of the problem is clang-tblgen, which requires HUGE amounts of > ram; if you've gotten beyond that in the build, you may not see any > more problems. > A Pi2 with 2 GB of swap on a usb flash drive got all the way into building libraries before running out of swap with a little over 1.5 GB in use, which is the actual limit. I realize that's armv7, not arm64, but it makes me wonder if perhaps swap isn't being freed when no longer needed. I've restarted buildworld with -DNO_CLEAN and am simply hoping for further progress. Meanwhile, restarting buildworld at j6 on the Pi3 with sources at 329462 didn't clear the compiler error. Can you hazard a guess whether 329462 should compile correctly? If so I'll clean manually and start over, if not I'll wait for an update or a pointer to a self-consistent revision. Thanks very much! bob prohaska
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180217182643.GC93736>