Date: Sun, 18 Dec 2011 18:26:24 -0800 From: YongHyeon PYUN <pyunyh@gmail.com> To: Sami Halabi <sodynet1@gmail.com> Cc: freebsd-net@freebsd.org, Arnaud Lacombe <lacombar@gmail.com> Subject: Re: Problem on re0 Message-ID: <20111219022624.GB1704@michelle.cdnetworks.com> In-Reply-To: <CAEW%2BogZOyBDN36t0BMEFcgKgWydShuC82Ym5=eDUAGNCvU0w5Q@mail.gmail.com> References: <CAEW%2BogauicnjsNdMCBBtoYzOLKVQoxP92Lw9Q_YgYPAe96dS5g@mail.gmail.com> <20111205022730.GE1729@michelle.cdnetworks.com> <CAEW%2BogbPwPY8r=uHhpkkp9-_K_gNsAVzKhVmgWScAMsHt2SJKQ@mail.gmail.com> <20111207005958.GE5825@michelle.cdnetworks.com> <CAEW%2BogZ7wivLcNm8rpcDR-ekP7Z6iEBZ0DtExhg5NVGfho0Ckw@mail.gmail.com> <20111207235810.GB9211@michelle.cdnetworks.com> <CAEW%2BogZKetpmwuCuyQo-g5DVZ4jCFM0L-mRApV9RTQ4S=Yi3hg@mail.gmail.com> <20111208215256.GB13605@michelle.cdnetworks.com> <CACqU3MVyDD=Vj1wK%2BNmrjjauzq85JSqrj2WEjLj878fSqhzT%2BQ@mail.gmail.com> <CAEW%2BogZOyBDN36t0BMEFcgKgWydShuC82Ym5=eDUAGNCvU0w5Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Dec 17, 2011 at 06:49:48PM +0200, Sami Halabi wrote: > Hi, > > I've moved to a new server with bge card driver, similar configuration: > /etc/sysctl.conf > net.inet.flowtable.enable=0 > net.inet.ip.fastforwarding=1 > kern.ipc.somaxconn=8192 > kern.ipc.shmmax=2147483648 > kern.ipc.maxsockets=204800 > kern.ipc.maxsockbuf=2097152 > hw.intr_storm_threshold=9000 > kern.maxfiles=256000 > kern.maxfilesperproc=230400 > net.inet.ip.dummynet.pipe_slot_limit=1000 > #net.inet.ip.dummynet.io_fast=1 > net.link.ether.ipfw=1 > kern.ipc.nmbclusters=409600 > net.graph.recvspace=40960 > net.graph.maxdgram=40960 > > Kernel > ----------- > device lagg > options IPFIREWALL > options IPFIREWALL_FORWARD > options IPFIREWALL_VERBOSE > options DUMMYNET > options HZ=1000 > options TCP_SIGNATURE > device crypto # core crypto support > device cryptodev # /dev/crypto for access to h/w > options IPSEC > options DEVICE_POLLING > device if_bridge > > /boot/loader.conf > net.graph.maxalloc=128000 > net.graph.maxdata=128000 > net.graph.threads=4 > > and i have similar problem.. i check each minute for ping and i see > sometimes loss of 50% or 100% (ping -c 2 -t 4 otherside.ip.com), > here are the sysctl dev.bge.1.stats: > dev.bge.1.stats.FramesDroppedDueToFilters: 0 > dev.bge.1.stats.DmaWriteQueueFull: 2291350 > dev.bge.1.stats.DmaWriteHighPriQueueFull: 0 > dev.bge.1.stats.NoMoreRxBDs: 0 > dev.bge.1.stats.InputDiscards: 0 > dev.bge.1.stats.InputErrors: 0 > dev.bge.1.stats.RecvThresholdHit: 124120300 > dev.bge.1.stats.DmaReadQueueFull: 13470948 > dev.bge.1.stats.DmaReadHighPriQueueFull: 962 > dev.bge.1.stats.SendDataCompQueueFull: 0 > dev.bge.1.stats.RingSetSendProdIndex: 349289174 > dev.bge.1.stats.RingStatusUpdate: 260304688 > dev.bge.1.stats.Interrupts: 260304688 > dev.bge.1.stats.AvoidedInterrupts: 0 > dev.bge.1.stats.SendThresholdHit: 0 > dev.bge.1.stats.rx.ifHCInOctets: 2165908673 > dev.bge.1.stats.rx.Fragments: 0 > dev.bge.1.stats.rx.UnicastPkts: 237503495 > dev.bge.1.stats.rx.MulticastPkts: 0 > dev.bge.1.stats.rx.FCSErrors: 0 > dev.bge.1.stats.rx.AlignmentErrors: 0 > dev.bge.1.stats.rx.xonPauseFramesReceived: 0 > dev.bge.1.stats.rx.xoffPauseFramesReceived: 0 > dev.bge.1.stats.rx.ControlFramesReceived: 0 > dev.bge.1.stats.rx.xoffStateEntered: 0 > dev.bge.1.stats.rx.FramesTooLong: 0 > dev.bge.1.stats.rx.Jabbers: 0 > dev.bge.1.stats.rx.UndersizePkts: 0 > dev.bge.1.stats.rx.inRangeLengthError: 0 > dev.bge.1.stats.rx.outRangeLengthError: 0 > dev.bge.1.stats.tx.ifHCOutOctets: 1956421618 > dev.bge.1.stats.tx.Collisions: 0 > dev.bge.1.stats.tx.XonSent: 0 > dev.bge.1.stats.tx.XoffSent: 0 > dev.bge.1.stats.tx.flowControlDone: 0 > dev.bge.1.stats.tx.InternalMacTransmitErrors: 0 > dev.bge.1.stats.tx.SingleCollisionFrames: 0 > dev.bge.1.stats.tx.MultipleCollisionFrames: 0 > dev.bge.1.stats.tx.DeferredTransmissions: 0 > dev.bge.1.stats.tx.ExcessiveCollisions: 0 > dev.bge.1.stats.tx.LateCollisions: 0 > dev.bge.1.stats.tx.UnicastPkts: 347260508 > dev.bge.1.stats.tx.MulticastPkts: 0 > dev.bge.1.stats.tx.BroadcastPkts: 30306 > dev.bge.1.stats.tx.CarrierSenseErrors: 0 > dev.bge.1.stats.tx.Discards: 0 > dev.bge.1.stats.tx.Errors: 0 > > this driver gives more statics... > > please help me, this causes us serious problems with customers. > Given that you see the same issue with bge(4) it looks like the root cause is not in ethernet driver. I also see no evidence of dropped frames from bge(4) hardware MAC statistics. It seems you have really complex network configuration(ipfw(4), lagg(4), dummynet(4), bridge(4) and ipsec(4) etc). Finding out simplest network setup that shows the issue would be required here.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111219022624.GB1704>