Date: Sat, 20 Apr 2002 20:00:57 -0300 (ADT) From: "Marc G. Fournier" <scrappy@hub.org> To: freebsd-current@freebsd.org Cc: freebsd-stable@freebsd.org Subject: Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ? Message-ID: <20020420195845.P1721-100000@mail1.hub.org> In-Reply-To: <20020420190408.O30724-100000@mail1.hub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
As a quick follow-up to this, doing more searching on the web, I came across a few suggested 'sysctl' settings, which I've added to what I had before, for a total of: kern.maxfiles=65534 jail.sysvipc_allowed=1 vm.swap_idle_enabled=1 vfs.vmiodirenable=1 kern.ipc.somaxconn=4096 I've also just reduced my maxusers to 256 from 1024, since 1024 was crashing worse then 512, and I ran across the 'tuning' man page that stated that you shouldn't go above 256 :( Just a bit more detail on the setup ... On Sat, 20 Apr 2002, Marc G. Fournier wrote: > > Over the past week, I've been trying to get information on how to fix a > server that panics with: > > | panic: vm_map_entry_create: kernel resources exhausted > | mp_lock = 01000001; cpuid = 1; lapic.id = 01000000 > | boot() called on cpu#1 > > Great ... but, how do I determine what 'resources' I need to increase to > avoid that crash? I've tried increasing maxusers from 512->1024, but *if* > that works, I imagine I'm raising a bunch of limits (and using memory) > that I don't have to ... > > The server is a Dual-CPU PIII-1Ghz with 3Gig of RAM and ~3Gig of swap > space right now ... the data drive is 5x18gig drives in a RAID5 > configuration (hardware RAID, not vinum) ... > > I ran top in an xterm so that I could see what was up just before the > crash, and the results were: > > last pid: 84988; load averages: 19.82, 57.35, 44.426 up 0+23:33:12 02:05:00 > 5021 processes:16 running, 5005 sleeping > CPU states: 8.7% user, 0.0% nice, 24.3% system, 2.2% interrupt, 64.7% idle > Mem: 2320M Active, 211M Inact, 390M Wired, 92M Cache, 199M Buf, 4348K Free > Swap: 3072M Total, 1048M Used, 2024M Free, 34% Inuse, 448K Out > > So, I have plenty of swapspace left, lots of idle CPU and a whole > whack of processes ... > > Now, looking at the LINT file, there appears to be *alot* of > things I *could* change ... for instance, NSFBUFS, KVA_FILES, etc ... but > I don't imagine that changing these blindly is particularly wise ... so, > how do you determine what to change? For instance, at a maxusers of 512, > NSFBUFS should be ~8704, and if I've only got 5000 processes running, > chances are I'm still safe at that value, no? But sysctl doesn't show any > 'sf_buf' value, so how do I figure out what I'm using? > > Basically, are there any commands similar to "netstat -m" for > nmbclusters that I can run to 'monitor' and isolate where I'm exhausting > these resources? > > Is there a doc on this sort of stuff that I should be reading for > this? Something that talks about kernel tuning for high-load/processes > servers? > > Thanks for any help in advance .. > > ------------------- > machine i386 > cpu I686_CPU > ident kernel > maxusers 1024 > > options NMBCLUSTERS=15360 > > options INET #InterNETworking > options INET6 #IPv6 communications protocols > options FFS #Berkeley Fast Filesystem > options FFS_ROOT #FFS usable as root device [keep this!] > options SOFTUPDATES #Enable FFS soft updates support > options PROCFS #Process filesystem > options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] > options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI > options KTRACE #ktrace(1) support > > options SYSVSHM > options SHMMAXPGS=98304 > options SHMMAX=(SHMMAXPGS*PAGE_SIZE+1) > > options SYSVSEM > options SEMMNI=2048 > options SEMMNS=4096 > > options SYSVMSG #SYSV-style message queues > > options P1003_1B #Posix P1003_1B real-time extensions > options _KPOSIX_PRIORITY_SCHEDULING > options ICMP_BANDLIM #Rate limit bad replies > > options SMP # Symmetric MultiProcessor Kernel > options APIC_IO # Symmetric (APIC) I/O > > device isa > device pci > > device scbus # SCSI bus (required) > device da # Direct Access (disks) > device sa # Sequential Access (tape etc) > device cd # CD > device pass # Passthrough device (direct SCSI access) > > device amr # AMI MegaRAID > device sym > > device atkbdc0 at isa? port IO_KBD > device atkbd0 at atkbdc? irq 1 flags 0x1 > device psm0 at atkbdc? irq 12 > > device vga0 at isa? > > pseudo-device splash > > device sc0 at isa? flags 0x100 > > device npx0 at nexus? port IO_NPX irq 13 > > device sio0 at isa? port IO_COM1 flags 0x10 irq 4 > device sio1 at isa? port IO_COM2 irq 3 > > device miibus # MII bus support > device fxp # Intel EtherExpress PRO/100B (82557, 82558) > > pseudo-device loop # Network loopback > pseudo-device ether # Ethernet support > pseudo-device pty 256 # Pseudo-ttys (telnet etc) > pseudo-device gif # IPv6 and IPv4 tunneling > pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation) > > pseudo-device bpf #Berkeley packet filter > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-stable" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020420195845.P1721-100000>