Date: Tue, 3 Apr 2018 19:47:21 -0600 From: Cy Schubert <Cy.Schubert@cschubert.com> To: Andriy Gapon <avg@FreeBSD.org>, Bryan Drewery <bdrewery@FreeBSD.org>, Peter Jeremy <peter@rulingia.com>, Jeff Roberson <jroberson@jroberson.net> Cc: FreeBSD current <freebsd-current@FreeBSD.org> Subject: RE: Strange ARC/Swap/CPU on yesterday's -CURRENT Message-ID: <20180404014720.A64BC1101@spqr.komquats.com>
next in thread | raw e-mail | index | archive | help
Agreed. I've come to the same conclusion that there may be multiple issues. --- Sent using a tiny phone keyboard. Apologies for any typos and autocorrect. Also, this old phone only supports top post. Apologies. Cy Schubert <Cy.Schubert@cschubert.com> or <cy@freebsd.org> The need of the many outweighs the greed of the few. --- -----Original Message----- From: Andriy Gapon Sent: 27/03/2018 08:02 To: Bryan Drewery; Peter Jeremy; Jeff Roberson Cc: FreeBSD current Subject: Re: Strange ARC/Swap/CPU on yesterday's -CURRENT On 24/03/2018 01:21, Bryan Drewery wrote: > On 3/20/2018 12:07 AM, Peter Jeremy wrote: >> >> On 2018-Mar-11 10:43:58 -1000, Jeff Roberson <jroberson@jroberson.net> w= rote: >>> Also, if you could try going back to r328953 or r326346 and let me know= if=20 >>> the problem exists in either. That would be very helpful. If anyone i= s=20 >>> willing to debug this with me contact me directly and I will send some= =20 >>> test patches or debugging info after you have done the above steps. >> >> I ran into this on 11-stable and tracked it to r326619 (MFC of r325851). >> I initially got around the problem by reverting that commit but either >> it or something very similar is still present in 11-stable r331053. >> >> I've seen it in my main server (32GB RAM) but haven't managed to reprodu= ce >> it in smaller VBox guests - one difficulty I faced was artificially fill= ing >> ARC. First, it looks like maybe several different issues are being discussed and possibly conflated in this thread. I see reports related to ZFS, I see rep= orts where ZFS is not used at all. Some people report problems that appeared ve= ry recently while others chime in with "yes, yes, I've always had this problem= ". This does not help to differentiate between problems and to analyze them. > Looking at the ARC change you referred to from r325851 > https://reviews.freebsd.org/D12163, I am convinced that ARC backpressure > is completely broken. Does your being convinced come from the code review or from experiments? If the former, could you please share your analysis? > On my 78GB RAM system with ARC limited to 40GB and > doing a poudriere build of all LLVM and GCC packages at once in tmpfs I > can get swap up near 50GB and yet the ARC remains at 40GB through it > all. It's always been slow to give up memory for package builds but it > really seems broken right now. Well, there are multiple angles. Maybe it's ARC that does not react proper= ly, or maybe it's not being asked properly. It would be useful to monitor the system during its transition to the state= that you reported. There are some interesting DTrace probes in ARC, specificall= y arc-available_memory and arc-needfree are first that come to mind. Their parameters and how frequently they are called are of interest. VM paramete= rs could be of interest as well. A rant. Basically, posting some numbers and jumping to conclusions does not help at= all. Monitoring, graphing, etc does help. ARC is a complex dynamic system. VM (pagedaemon, UMA caches) is a complex dynamic system. They interact in a complex dynamic ways. Sometimes a change in ARC is incorrect and requires a fix. Sometimes a change in VM is incorrect and requires a fix. Sometimes a change in VM requires a change in ARC. These three kinds of problems can all appear as a "problem with ARC". For instance, when vm.lowmem_period was introduced you wouldn't find any me= ntion of ZFS/ARC. But it does affect period between arc_lowmem() calls. Also, pin-pointing a specific commit requires proper bisecting and proper testing to correctly attribute systemic behavior changes to code changes. --=20 Andriy Gapon _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180404014720.A64BC1101>