Date: Sat, 20 Apr 2002 15:35:29 -0700 From: "Mike Grissom" <mikeyg@speedfusion.net> To: <freebsd-stable@freebsd.org> Subject: Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ? Message-ID: <060a01c1e8bb$ad3059c0$0301a8c0@mikeyg> References: <20020420190408.O30724-100000@mail1.hub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
If you are using 4.5 then you should probably use MAXUSERS 0 and remove the NMBCLUSTERS that enabled auto scalling and should up the NMBCLUSTERS and other sysctl parms when they are needed to be higher and it bases the settings on how much ram you have. ----- Original Message ----- From: "Marc G. Fournier" <scrappy@hub.org> To: <freebsd-current@freebsd.org> Cc: <freebsd-stable@freebsd.org> Sent: Saturday, April 20, 2002 3:14 PM Subject: FreeBSD 4.5-STABLE not easily scalable to large servers ... ? > > Over the past week, I've been trying to get information on how to fix a > server that panics with: > > | panic: vm_map_entry_create: kernel resources exhausted > | mp_lock = 01000001; cpuid = 1; lapic.id = 01000000 > | boot() called on cpu#1 > > Great ... but, how do I determine what 'resources' I need to increase to > avoid that crash? I've tried increasing maxusers from 512->1024, but *if* > that works, I imagine I'm raising a bunch of limits (and using memory) > that I don't have to ... > > The server is a Dual-CPU PIII-1Ghz with 3Gig of RAM and ~3Gig of swap > space right now ... the data drive is 5x18gig drives in a RAID5 > configuration (hardware RAID, not vinum) ... > > I ran top in an xterm so that I could see what was up just before the > crash, and the results were: > > last pid: 84988; load averages: 19.82, 57.35, 44.426 up 0+23:33:12 02:05:00 > 5021 processes:16 running, 5005 sleeping > CPU states: 8.7% user, 0.0% nice, 24.3% system, 2.2% interrupt, 64.7% idle > Mem: 2320M Active, 211M Inact, 390M Wired, 92M Cache, 199M Buf, 4348K Free > Swap: 3072M Total, 1048M Used, 2024M Free, 34% Inuse, 448K Out > > So, I have plenty of swapspace left, lots of idle CPU and a whole > whack of processes ... > > Now, looking at the LINT file, there appears to be *alot* of > things I *could* change ... for instance, NSFBUFS, KVA_FILES, etc ... but > I don't imagine that changing these blindly is particularly wise ... so, > how do you determine what to change? For instance, at a maxusers of 512, > NSFBUFS should be ~8704, and if I've only got 5000 processes running, > chances are I'm still safe at that value, no? But sysctl doesn't show any > 'sf_buf' value, so how do I figure out what I'm using? > > Basically, are there any commands similar to "netstat -m" for > nmbclusters that I can run to 'monitor' and isolate where I'm exhausting > these resources? > > Is there a doc on this sort of stuff that I should be reading for > this? Something that talks about kernel tuning for high-load/processes > servers? > > Thanks for any help in advance .. > > ------------------- > machine i386 > cpu I686_CPU > ident kernel > maxusers 1024 > > options NMBCLUSTERS=15360 > > options INET #InterNETworking > options INET6 #IPv6 communications protocols > options FFS #Berkeley Fast Filesystem > options FFS_ROOT #FFS usable as root device [keep this!] > options SOFTUPDATES #Enable FFS soft updates support > options PROCFS #Process filesystem > options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] > options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI > options KTRACE #ktrace(1) support > > options SYSVSHM > options SHMMAXPGS=98304 > options SHMMAX=(SHMMAXPGS*PAGE_SIZE+1) > > options SYSVSEM > options SEMMNI=2048 > options SEMMNS=4096 > > options SYSVMSG #SYSV-style message queues > > options P1003_1B #Posix P1003_1B real-time extensions > options _KPOSIX_PRIORITY_SCHEDULING > options ICMP_BANDLIM #Rate limit bad replies > > options SMP # Symmetric MultiProcessor Kernel > options APIC_IO # Symmetric (APIC) I/O > > device isa > device pci > > device scbus # SCSI bus (required) > device da # Direct Access (disks) > device sa # Sequential Access (tape etc) > device cd # CD > device pass # Passthrough device (direct SCSI access) > > device amr # AMI MegaRAID > device sym > > device atkbdc0 at isa? port IO_KBD > device atkbd0 at atkbdc? irq 1 flags 0x1 > device psm0 at atkbdc? irq 12 > > device vga0 at isa? > > pseudo-device splash > > device sc0 at isa? flags 0x100 > > device npx0 at nexus? port IO_NPX irq 13 > > device sio0 at isa? port IO_COM1 flags 0x10 irq 4 > device sio1 at isa? port IO_COM2 irq 3 > > device miibus # MII bus support > device fxp # Intel EtherExpress PRO/100B (82557, 82558) > > pseudo-device loop # Network loopback > pseudo-device ether # Ethernet support > pseudo-device pty 256 # Pseudo-ttys (telnet etc) > pseudo-device gif # IPv6 and IPv4 tunneling > pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation) > > pseudo-device bpf #Berkeley packet filter > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-stable" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?060a01c1e8bb$ad3059c0$0301a8c0>