Date: Mon, 4 Jul 2011 23:33:35 +0300 From: "Adrian Minta" <gygy@stsnet.ro> To: "Eugene Grosbein" <egrosbein@rdtc.ru> Cc: freebsd-net@freebsd.org, Adrian Minta <gygy@stsnet.ro> Subject: Re: FreeBSD 8.2 and MPD5 stability issues - update Message-ID: <c4495c407039b9281c1c98fc2806d0ab.squirrel@mail.stsnet.ro> In-Reply-To: <20110704191451.GA12372@rdtc.ru> References: <813678a855c90c49bf66c7084f88b45d.squirrel@mail.stsnet.ro> <20110704191451.GA12372@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
> You should study why existing connections break, > do clients disconnect themselves or server disconnect them? > You'll need turn off detailed logs, read mpd's documentation. > > Also, there are system-wide queues for NETGRAPH messages that can overflow > and that's bad thing. Check them out with command: > > vmstat -z | egrep 'ITEM|NetGraph' > > FAILURES column shows how many times NETGRAPH queues have been overflowed. > One may increase their LIMIT (second column in vmstat's output) > with /boot/loader.conf: > > net.graph.maxdata=65536 > net.graph.maxalloc=65536 > > Eugene Grosbein > > Back to "default" mpd for now. --- /boot/loader.conf --- kern.maxusers=1024 net.graph.maxdata=65536 net.graph.maxalloc=65536 kern.ipc.maxpipekva=320000000 ###8 cores maxthreads=8 net.isr.maxthreads=8 net.isr.bindthreads=1 net.isr.defaultqlimit=8192 net.isr.maxqlimit=10240 console="comconsole,vidconsole" hw.igb.rxd=4096 # IGB Tuning hw.igb.txd=4096 # IGB Tuning if_lagg_load="YES" --- /etc/sysctl.conf --- kern.maxfiles=250000 net.inet.ip.intr_queue_maxlen=10240 kern.ipc.nmbclusters=1280000 kern.ipc.somaxconn=32768 kern.ipc.maxsockbuf=128000000 kern.ipc.maxsockets=128000000 net.local.stream.recvspace=65536 net.local.stream.sendspace=65536 net.local.dgram.recvspace=8000 net.inet.udp.recvspace=262144 net.graph.maxdgram=10240000 net.graph.recvspace=10240000 kern.random.sys.harvest.ethernet=0 net.isr.direct=1 net.isr.direct_force=0 --- When failures start (1 sec interval): --- sessions: 5697 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 14, 8512, 11679211, 0 NetGraph data items: 104, 65540, 0, 899, 18656500, 0 sessions: 5695 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 14, 8512, 11698144, 0 NetGraph data items: 104, 65540, 0, 899, 18690476, 0 sessions: 5696 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 14, 8512, 11717921, 0 NetGraph data items: 104, 65540, 0, 899, 18726539, 0 sessions: 5697 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 8, 8518, 11736938, 0 NetGraph data items: 104, 65540, 0, 899, 18760816, 0 sessions: 5697 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 14, 8512, 11756221, 0 NetGraph data items: 104, 65540, 0, 899, 18796218, 0 sessions: 5697 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 11, 8515, 11775151, 0 NetGraph data items: 104, 65540, 0, 899, 18830631, 0 sessions: 5696 ITEM SIZE LIMIT USED FREE REQUESTS FAILURES NetGraph items: 104, 65540, 9, 8517, 11794388, 0 NetGraph data items: 104, 65540, 0, 899, 18865673, 0 --- mpd.log (via "log -ccp -chat -lcp") --- Jul 4 23:23:37 lns mpd: L2TP: Control connection 0x80a2aaf10 terminated: 0 (no more sessions exist in this tunnel) Jul 4 23:23:37 lns mpd: L2TP: Control connection 0x80a2a8810 terminated: 0 (no more sessions exist in this tunnel) Jul 4 23:23:37 lns mpd: [L19-6291] L2TP: Call #19 connected Jul 4 23:23:37 lns mpd: [L19-6291] Link: UP event Jul 4 23:23:37 lns mpd: [L19-6291] LCP: Up event Jul 4 23:23:37 lns mpd: [L19-6291] LCP: state change Starting --> Req-Sent Jul 4 23:23:37 lns mpd: [L19-6291] LCP: SendConfigReq #1 Jul 4 23:23:37 lns mpd: [L50-6292] L2TP: Call #50 connected Jul 4 23:23:37 lns mpd: [L50-6292] Link: UP event Jul 4 23:23:37 lns mpd: [L50-6292] LCP: Up event Jul 4 23:23:37 lns mpd: [L50-6292] LCP: state change Starting --> Req-Sent Jul 4 23:23:37 lns mpd: [L50-6292] LCP: SendConfigReq #1 Jul 4 23:23:37 lns mpd: [L7-6161] LCP: rec'd Configure Reject #1 (Req-Sent) Jul 4 23:23:37 lns mpd: [L7-6161] Wrong id#, expecting 4 Jul 4 23:23:37 lns mpd: [L50-5752] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L50-5752] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L6-6107] LCP: rec'd Configure Request #1 (Req-Sent) Jul 4 23:23:37 lns mpd: [L6-6107] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L6-6107] LCP: state change Req-Sent --> Ack-Sent Jul 4 23:23:37 lns mpd: [L25-6047] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L25-6047] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L1-5754] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L1-5754] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L26-6108] LCP: rec'd Configure Request #1 (Req-Sent) Jul 4 23:23:37 lns mpd: [L26-6108] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L26-6108] LCP: state change Req-Sent --> Ack-Sent Jul 4 23:23:37 lns mpd: [L1-6109] LCP: rec'd Configure Request #1 (Req-Sent) Jul 4 23:23:37 lns mpd: [L1-6109] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L1-6109] LCP: state change Req-Sent --> Ack-Sent Jul 4 23:23:37 lns mpd: [L3-5880] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L3-5880] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L50-6048] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L50-6048] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L19-5921] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L19-5921] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L25-5753] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L25-5753] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L35-6049] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L35-6049] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L3-5923] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L3-5923] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: L2TP: Control connection 0x80b15f610 10.42.1.19 1701 <-> 10.42.10.4 1701 connected Jul 4 23:23:37 lns mpd: Incoming L2TP packet from 10.42.10.1 1701 Jul 4 23:23:37 lns mpd: [L41-6185] LCP: SendConfigReq #4 Jul 4 23:23:37 lns mpd: [L48-6184] LCP: SendConfigReq #4 Jul 4 23:23:37 lns mpd: [L49-6130] LCP: SendConfigReq #5 Jul 4 23:23:37 lns mpd: [L26-6131] LCP: SendConfigReq #5 Jul 4 23:23:37 lns mpd: [L41-6127] LCP: SendConfigReq #5 Jul 4 23:23:37 lns mpd: [L30-6128] LCP: SendConfigReq #5 Jul 4 23:23:37 lns mpd: [L6-6129] LCP: SendConfigReq #5 Jul 4 23:23:37 lns mpd: [L31-6091] LCP: SendConfigReq #6 Jul 4 23:23:37 lns mpd: [L10-6050] LCP: SendConfigReq #7 Jul 4 23:23:37 lns mpd: [L21-6051] LCP: SendConfigReq #7 Jul 4 23:23:37 lns mpd: [L17-6014] LCP: SendConfigReq #8 Jul 4 23:23:37 lns mpd: [L47-6016] LCP: SendConfigReq #8 Jul 4 23:23:37 lns mpd: [L29-6015] LCP: SendConfigReq #8 Jul 4 23:23:37 lns mpd: [L3-5858] LCP: SendConfigReq #11 Jul 4 23:23:37 lns mpd: [L15-5937] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L26-5942] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L30-5944] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L22-5822] LCP: SendConfigReq #12 Jul 4 23:23:37 lns mpd: [L2-5821] LCP: SendConfigReq #12 Jul 4 23:23:37 lns mpd: [L50-5943] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L35-5941] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L25-5760] LCP: SendConfigReq #13 Jul 4 23:23:37 lns mpd: [L11-5945] LCP: SendConfigReq #14 Jul 4 23:23:37 lns mpd: [L4-5963] LCP: SendConfigReq #10 Jul 4 23:23:37 lns mpd: [L13-5962] LCP: SendConfigReq #10 Jul 4 23:23:37 lns mpd: [L35-6089] LCP: SendConfigReq #6 Jul 4 23:23:37 lns mpd: [L10-6088] LCP: SendConfigReq #6 Jul 4 23:23:37 lns mpd: [L43-6216] LCP: SendConfigReq #3 Jul 4 23:23:37 lns mpd: [L41-6217] LCP: SendConfigReq #3 Jul 4 23:23:37 lns mpd: L2TP: Incoming call #19 via connection 0x80b15f610 received Jul 4 23:23:37 lns mpd: [L19-6294] L2TP: Incoming call #19 via control connection 0x80b15f610 accepted Jul 4 23:23:37 lns mpd: [L14-6293] L2TP: Call #14 connected Jul 4 23:23:37 lns mpd: [L14-6293] Link: UP event Jul 4 23:23:37 lns mpd: [L14-6293] LCP: Up event Jul 4 23:23:37 lns mpd: [L14-6293] LCP: state change Starting --> Req-Sent Jul 4 23:23:37 lns mpd: [L14-6293] LCP: SendConfigReq #1 Jul 4 23:23:37 lns mpd: [L49-5991] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L49-5991] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L10-6050] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L10-6050] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L8-5924] LCP: rec'd Configure Request #1 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L8-5924] LCP: SendConfigAck #1 Jul 4 23:23:37 lns mpd: [L37-6123] LCP: rec'd Configure Reject #2 (Req-Sent) Jul 4 23:23:37 lns mpd: [L37-6123] Wrong id#, expecting 5 Jul 4 23:23:37 lns mpd: [L47-5867] LCP: rec'd Configure Reject #7 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L47-5867] Wrong id#, expecting 10 Jul 4 23:23:37 lns mpd: [L38-5964] LCP: rec'd Configure Reject #6 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L38-5964] Wrong id#, expecting 9 Jul 4 23:23:37 lns mpd: [L29-5965] LCP: rec'd Configure Reject #6 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L29-5965] Wrong id#, expecting 9 Jul 4 23:23:37 lns mpd: [L1-5783] LCP: rec'd Configure Reject #9 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L1-5783] Wrong id#, expecting 12 Jul 4 23:23:37 lns mpd: [L7-5828] LCP: rec'd Configure Reject #8 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L7-5828] Wrong id#, expecting 11 Jul 4 23:23:37 lns mpd: [L13-5868] LCP: rec'd Configure Reject #7 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L13-5868] Wrong id#, expecting 10 Jul 4 23:23:37 lns mpd: [L14-6003] LCP: rec'd Configure Reject #5 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L14-6003] Wrong id#, expecting 8 Jul 4 23:23:37 lns mpd: [L34-5947] LCP: rec'd Configure Reject #10 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L34-5947] Wrong id#, expecting 13 Jul 4 23:23:37 lns mpd: [L1-5948] LCP: rec'd Configure Reject #10 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L1-5948] Wrong id#, expecting 13 Jul 4 23:23:37 lns mpd: [L32-5826] LCP: rec'd Configure Reject #8 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L32-5826] Wrong id#, expecting 11 Jul 4 23:23:37 lns mpd: [L45-5946] LCP: rec'd Configure Reject #10 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L45-5946] Wrong id#, expecting 13 Jul 4 23:23:37 lns mpd: [L43-5830] LCP: rec'd Configure Reject #8 (Ack-Sent) Jul 4 23:23:37 lns mpd: [L43-5830] Wrong id#, expecting 11 Jul 4 23:23:37 lns mpd: L2TP: Control connection 0x80b15f010 10.42.1.50 1701 <-> 10.42.9.221 1701 connected Jul 4 23:23:37 lns mpd: [L23-6254] LCP: SendConfigReq #2 Jul 4 23:23:37 lns mpd: Incoming L2TP packet from 10.42.9.244 1701
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c4495c407039b9281c1c98fc2806d0ab.squirrel>