Date: Wed, 6 Jul 2016 21:35:59 +0100 From: Steven Hartland <killing@multiplay.co.uk> To: freebsd-current@freebsd.org Subject: Re: Setting sysctl vfs.zfs.arc_max failed: 22 Message-ID: <4d83c12e-e9d9-b5a8-e6cb-733924d6fd7d@multiplay.co.uk> In-Reply-To: <CAB%2BRvm63pnLSx-fX8kiZp5Q%2BMB424xnAZeXRAr-de%2BEHiW1XCQ@mail.gmail.com> References: <ccdd6a5b-4771-6207-adb3-ce9a7960485c@FreeBSD.org> <975e5ae2-2090-94cd-49c0-eee9866ff02b@freebsd.org> <CAB%2BRvm7c97ionaM02UgTLNt-aqQJLZ8%2B5ZoFL9gHDhmmx=cCcg@mail.gmail.com> <e1fd9ae9-71de-ec0f-53d2-1c5d2f9ccc87@multiplay.co.uk> <145180d1-43c9-d894-9221-b210d641955d@freebsd.org> <CAB%2BRvm6OpDQMz4vE7d0D76wDuPjzh597UaDXjNfrHLP59st_ag@mail.gmail.com> <7255f240-8cf0-0a62-89be-2a0970b67063@freebsd.org> <CAB%2BRvm4V88ppSdFJ-DsUf6AKx6UC=hdbA_r4LjMdtwLCRa9pVw@mail.gmail.com> <CAB%2BRvm63pnLSx-fX8kiZp5Q%2BMB424xnAZeXRAr-de%2BEHiW1XCQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------52F902E6224F8A286A14620A Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit The ARC settings and kmem aren't initialised when tunables are loaded so the tests fail. I've fixed this locally by blindly setting if ARC is not configured. Request to commit the fix is with re@ In the mean time the patch is attached. Thanks for the report and sorry about the breakage. On 06/07/2016 07:20, Nathan Bosley wrote: > Maybe I misunderstood after all. > I took this: > "You can work around it temporarily by setting a lower arc_min first." > > To mean that I could do something like: > vfs.zfs.arc_min="1073741824" > vfs.zfs.arc_max="8589934592" > > in loader.conf, which would circumvent the problem. > But with the above, in that order, I still get: > > Setting sysctl vfs.zfs.arc_max failed: 22 > Setting sysctl vfs.zfs.arc_min failed: 22 > > As an FYI, WITHOUT any tunables set, my defaults are: > vfs.zfs.arc_meta_limit: 3903459328 > vfs.zfs.arc_min: 1951729664 > vfs.zfs.arc_max: 15613837312 > > So even if I only specified: > vfs.zfs.arc_max="8589934592" > > in loader.conf, that's still not below my default min. > > However, if I make the changes with 'sysctl' after boot, it works fine: > > root@athlonbsd:~ # sysctl vfs.zfs.arc_min="1073741824" > vfs.zfs.arc_min: 1951729664 -> 1073741824 > > root@athlonbsd:~ # sysctl vfs.zfs.arc_max="8589934592" > > vfs.zfs.arc_max: 15613837312 -> 8589934592 > > They also work fine sysctl.conf. > > Sorry if this is a silly question: > I should still be able to set these max/min values in loader.conf, > right--not just in sysctl.conf? > > Thanks. > > > On Tue, Jul 5, 2016 at 10:16 PM, Nathan Bosley <nathan.bosley@gmail.com> > wrote: > >> OK, I follow you now. >> Thanks for the explanation. >> I will try that later tonight or tomorrow. >> >> On Tue, Jul 5, 2016 at 9:45 PM, Allan Jude <allanjude@freebsd.org> wrote: >> >>> On 2016-07-05 21:32, Nathan Bosley wrote: >>>> I think in about 4 - 5 hours I can show what values I'm using in >>>> loader.conf under, say, r302264 and r302265 for comparison. I'm not 100% >>>> sure that the problem arose for me in r302265; I merely suspect it. >>>> >>>> On Tue, Jul 5, 2016 at 9:25 PM, Allan Jude <allanjude@freebsd.org >>>> <mailto:allanjude@freebsd.org>> wrote: >>>> >>>> On 2016-07-05 20:27, Steven Hartland wrote: >>>> > Ahh right, let me check that. >>>> > >>>> > On 06/07/2016 00:51, Nathan Bosley wrote: >>>> >> I actually have this same problem. >>>> >> I'll send more details when I get home later. >>>> >> >>>> >> I think the problem started for me after r302265. >>>> >> Before that, I can set vfs.zfs.arc_max and vfs.zfs.arc_min in >>>> >> loader.conf. >>>> >> After r302265, setting either vfs.zfs.arc_max or vfs.zfs.arc_min >>> in >>>> >> loader.conf results in the EINVAL errors in 'dmesg': >>>> >> >>>> >> Setting sysctl vfs.zfs.arc_max failed: 22 >>>> >> Setting sysctl vfs.zfs.arc_min failed: 22 >>>> >> >>>> >> But setting vfs.zfs.arc_meta_limit in loader.conf works fine. >>>> >> >>>> >> But I did notice that using 'sysct' or sysctl.conf for >>> vfs.zfs.arc_max >>>> >> and vfs.zfs.arc_min works. >>>> >> I only have problems with setting them now in loader.conf. >>>> >> >>>> >> Like I said, I'll try to send output from my setup later. >>>> >> >>>> >> Thanks. >>>> >> >>>> >> On Tue, Jul 5, 2016 at 6:10 PM, Steven Hartland >>>> >> <steven@multiplay.co.uk <mailto:steven@multiplay.co.uk> >>>> <mailto:steven@multiplay.co.uk <mailto:steven@multiplay.co.uk>>> >>> wrote: >>>> >> >>>> >> What is it currently? >>>> >> >>>> >> Just had a quick play here: >>>> >> sysctl vfs.zfs.arc_max >>>> >> vfs.zfs.arc_max: 32283127808 >>>> >> sysctl vfs.zfs.arc_max=32283127807 >>>> >> vfs.zfs.arc_max: 32283127808 -> 32283127807 >>>> >> sysctl vfs.zfs.arc_max=32283127808 >>>> >> vfs.zfs.arc_max: 32283127807 -> 32283127808 >>>> >> >>>> >> Error 22 = EINVAL so I suspect you're requesting a value >>>> which one >>>> >> of the following: >>>> >> * < arc_abs_min >>>> >> * > kmem_size >>>> >> * < arc_c_min >>>> >> * < zfs_arc_meta_limit >>>> >> >>>> >> Regards >>>> >> Steve >>>> >> >>>> >> On 05/07/2016 22:56, Eric van Gyzen wrote: >>>> >> >>>> >> Steven and -current: >>>> >> >>>> >> I just updated to r302350 with a GENERIC kernel config. >>>> I see >>>> >> this in >>>> >> dmesg: >>>> >> >>>> >> VT(efifb): resolution 1024x768 >>>> >> Setting sysctl vfs.zfs.arc_max failed: 22 >>>> >> CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz >>>> >> (3491.98-MHz K8-class >>>> >> CPU) >>>> >> >>>> >> The relevant parts of /boot/loader.conf are: >>>> >> >>>> >> zfs_load="YES" >>>> >> vfs.zfs.arc_max="6442450944" >>>> >> >>>> >> Let me know what other information you need. >>>> >> >>>> >> Cheers, >>>> >> >>>> >> Eric >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> freebsd-current@freebsd.org >>>> <mailto:freebsd-current@freebsd.org> >>>> <mailto:freebsd-current@freebsd.org >>>> <mailto:freebsd-current@freebsd.org>> >>>> >> mailing list >>>> >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> >> To unsubscribe, send any mail to >>>> >> "freebsd-current-unsubscribe@freebsd.org >>>> <mailto:freebsd-current-unsubscribe@freebsd.org> >>>> >> <mailto:freebsd-current-unsubscribe@freebsd.org >>>> <mailto:freebsd-current-unsubscribe@freebsd.org>>" >>>> >> >>>> >> >>>> > >>>> > _______________________________________________ >>>> > freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org> >>>> mailing list >>>> > https://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> > To unsubscribe, send any mail to " >>> freebsd-current-unsubscribe@freebsd.org >>>> <mailto:freebsd-current-unsubscribe@freebsd.org>" >>>> >>>> >>>> I think the issue might be that the default value of arc_min is >>> higher >>>> than when the user is trying to set arc_max to. In that case we >>> might >>>> want sysctl to lower arc_min instead of giving an error? >>>> >>>> It would definitely be a POLA violation to have to set arc_min >>> lower to >>>> be able to have existing lines that set arc_max in loader.conf work >>>> correctly. >>>> >>>> -- >>>> Allan Jude >>>> >>>> >>> The problem is related to Steven's commit, which makes it so you can >>> adjust arc_max and arc_min at runtime. Part of this change put in a >>> requirement that the new arc_max you are trying to set, must be less >>> than the current arc_min. When you first boot, i think the default is >>> arc_max = all ram save 1GB, and arc_min = 1/8 of that. >>> >>> So if you try to set your arc_max to 1/8th or less, it will fail with >>> that error 22 (this should be fixed). >>> >>> You can work around it temporarily by setting a lower arc_min first. >>> >>> >>> -- >>> Allan Jude >>> >>> > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" --------------52F902E6224F8A286A14620A Content-Type: text/plain; charset=UTF-8; name="zfs-arc-fix.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs-arc-fix.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 302360) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -919,6 +919,12 @@ sysctl_vfs_zfs_arc_max(SYSCTL_HANDLER_ARGS) if (err != 0 || req->newptr == NULL) return (err); + if (zfs_arc_max == 0) { + /* Loader tunable so blindly set */ + zfs_arc_max = val; + return (0); + } + if (val < arc_abs_min || val > kmem_size()) return (EINVAL); if (val < arc_c_min) @@ -956,6 +962,12 @@ sysctl_vfs_zfs_arc_min(SYSCTL_HANDLER_ARGS) if (err != 0 || req->newptr == NULL) return (err); + if (zfs_arc_min == 0) { + /* Loader tunable so blindly set */ + zfs_arc_min = val; + return (0); + } + if (val < arc_abs_min || val > arc_c_max) return (EINVAL); --------------52F902E6224F8A286A14620A--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4d83c12e-e9d9-b5a8-e6cb-733924d6fd7d>