Date: Fri, 14 Mar 2008 14:56:01 +0200 From: hugoboy@inbox.lv To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 7.0 bridge tuning Message-ID: <1205499361.47da75e1ac733@www.inbox.lv>
next in thread | raw e-mail | index | archive | help
I chose pf+altq for traffic shaper solution, because it seems to=0Abetter m= atch my needs. I use ipfw where FW is needed, but from the=0Apoint of easy = administration pf+altq is better for traffic shaper. =0ASo I have not teste= d shaping performance with ipfw as I have chosen=0Apf this time. If it will= turn out that it is not possible to achieve=0Agood enough results this way= - I'll try ipfw+dummynet.=0AServer is i386 based.=0A=0AStill I am quite su= re that it is possible to tune this configuration,=0Abut need to find bottl= eneck...=0A=0AUgis=0A>Hi,=0A>Just for my information, what performance if y= ou replace pf to ipfw=0A?=0A>and what freebsd v7.0 version ? i386 or amd64 = ?=0A>Regards=0A>Rmkml=0A On Fri, 14 Mar 2008, hugoboy@inbox.lv wrote:=0A=0A= > Date: Fri, 14 Mar 2008 12:51:50 +0200=0A> From: hugoboy@inbox.lv=0A> To: = freebsd-performance@freebsd.org=0A> Subject: FreeBSD 7.0 bridge tuning=0A> = =0A> Hello!=0A>=0A> I'm trying to tune FreeBSD 7.0 bridge.=0A>=0A> Environm= ent:=0A> Server - 2 x Xeon 3GHz, 2 x Gb LAN(em driver) + 1 LAN for=0Amanage= ment,=0A> 1GB RAM.=0A> Testers -2 x Sunrise Telecom 100Mbit Ethernet tester= s for traffic=0A> generation.=0A>=0A> What I have intended to achieve is to= substitute proprietary=0Atraffic=0A> shaper Allot with FreeBSD traffic sha= per(Bridge + PF + ALTQ).=0A> The minimum task is to make FreeBSD shaper to = perform perfectly=0Awith=0A> 100Mbit traffic in all spectrum of packet leng= ths (from 64 bytes to=0A> at least 1518 bytes)=0A>=0A> The situation now:= =0A> with pf turned off - there is no problem, bridge throughput is=0A> 100= Mbit/s no packet loss (starting from 64 byte packets)=0A>=0A> With pf on I = have statistics:=0A> packet lengt -> Mbit/s without packet loss=0A> 64 -> 4= 6=0A> 100 -> 66=0A> 150 -> 94=0A>> 200 -> 100=0A>=0A> Lower configuration o= f kernel/sysctl is displayed.=0A>=0A> I don't know what else can I tune?=0A= >=0A> It seems to me that bottleneck is somewhere around pf/kernel=0Abuffer= s=0A> of packet headers. I read somewhere that in bridging packet payload= =0A> does not travel through all stack - just header is evaluated.=0A> In c= ase of 64 byte packets in the same time unit there are more=0A> packets for= the same bandwith on interfaces and as plain layer2=0A> bridge performs 10= 0Mbit/s with no problem=0A> the problem is above layer2 :)=0A>=0A> btw: ker= n.polling.enable=3D1 does not help - at packetlength 64 bytes=0A> performan= ce is 2x worse than with interrupts.=0A> kernel:=0A> ----------------------= -----=0A>=0A> cpu I686_CPU=0A> ident ALLOT =0A>=0A> #= To statically compile in device wiring instead of=0A> /boot/device.hints= =0A> #hints "GENERIC.hints" # Default places to look=0Afor= =0A> devices.=0A>=0A> makeoptions DEBUG=3D-g # Build ker= nel with gdb(1)=0A> debug symbols=0A>=0A> options SCHED_ULE = # ULE scheduler=0A> #options SCHED_4BSD # 4BSD s= cheduler=0A> options PREEMPTION # Enable kernel thread= =0A> preemption=0A> options INET # InterNETworki= ng=0A> #options INET6 # IPv6 communications=0A> pr= otocols=0A> #options SCTP # Stream Control=0ATran= smission=0A> Protocol=0A> options FFS # Berkele= y Fast Filesystem=0A> options SOFTUPDATES # Enable FFS = soft updates=0A> support=0A> options UFS_ACL # Supp= ort for access=0Acontrol=0A> lists=0A> options UFS_DIRHASH = # Improve performance on=0Abig=0A> directories=0A> options UFS_= GJOURNAL # Enable gjournal-based UFS=0A> journaling=0A> options = MD_ROOT # MD is a potential root=0A> device=0A> opt= ions NFSCLIENT # Network Filesystem Client=0A> option= s NFSSERVER # Network Filesystem Server=0A> options = NFS_ROOT # NFS usable as /, requires=0A> NFSCLIENT=0A= > options MSDOSFS # MSDOS Filesystem=0A> options = CD9660 # ISO 9660 Filesystem=0A> options PRO= CFS # Process filesystem=0A> (requires PSEUDOFS)=0A> optio= ns PSEUDOFS # Pseudo-filesystem=0Aframework=0A> opti= ons GEOM_PART_GPT # GUID Partition Tables.=0A> options = GEOM_LABEL # Provides labelization=0A> options C= OMPAT_43TTY # BSD 4.3 TTY compat [KEEP=0A> THIS!]=0A> options = COMPAT_FREEBSD4 # Compatible with FreeBSD4=0A> options = COMPAT_FREEBSD5 # Compatible with FreeBSD5=0A> options CO= MPAT_FREEBSD6 # Compatible with FreeBSD6=0A> options SCSI_D= ELAY=3D5000 # Delay (in ms) before=0A> probing SCSI=0A> options = KTRACE # ktrace(1) support=0A> options SYSVSH= M # SYSV-style shared memory=0A> options SYSVMSG = # SYSV-style message queues=0A> options SYSVSEM = # SYSV-style semaphores=0A> options _KPOSIX_PRIORITY_SCH= EDULING # POSIX P1003_1B=0A> real-time extensions=0A> options KBD_I= NSTALL_CDEV # install a CDEV entry in=0A> /dev=0A> options A= DAPTIVE_GIANT # Giant mutex is adaptive.=0A> options STOP_= NMI # Stop CPUS using NMI=0Ainstead=0A> of IPI=0A> options = AUDIT # Security event auditing=0A>=0A> options AL= TQ=0A> options ALTQ_CBQ=0A> options ALTQ_RED=0A> options ALTQ_RIO=0A> optio= ns ALTQ_HFSC=0A> options ALTQ_CDNR=0A> options ALTQ_PRIQ=0A> options ALTQ_N= OPCC=0A> options HZ=3D1000=0A> options DEVICE_POLLING=0A> options IPSTEALTH= =0A> options ZERO_COPY_SOCKETS=0A> options MPTABLE_FORCE_HTT # Enable= HTT CPUs with the MP Table=0A> options IPI_PREEMPTION=0A>=0A> # To make an= SMP kernel, the next two lines are needed=0A> options SMP = # Symmetric MultiProcessor=0A> Kernel=0A> device apic = # I/O APIC=0A> --------------------------------=0A>=0A> = /etc/sysctl.conf=0A> #kern.polling.enable=3D1=0A> kern.ipc.nmbcluster=3D327= 68=0A> kern.ipc.maxsockbufs=3D2097152=0A> kern.ipc.somaxconn=3D8192=0A> ker= n.maxfiles=3D65536=0A> kern.maxfilesperproc=3D32768=0A> net.inet.tcp.delaye= d_ack=3D0=0A> net.inet.tcp.sendspace=3D65535=0A> net.inet.udp.recvspace=3D6= 5535=0A> net.inet.udp.maxdgram=3D57344=0A> net.local.stream.recvspace=3D655= 35=0A> net.local.stream.sendspace=3D65535=0A> kern.polling.user_frac=3D20= =0A> net.isr.direct=3D0=0A> net.inet.ip.forwarding=3D1=0A> ----------------= ---------------=0A>=0A> P.S. I tried pfSense, but as we have used Allot bef= ore - we need to=0A> see queue statistics in graphs per queue, pfSense just= offers=0A> numbers..=0A> Seems to me that pFsense is good for many things = but not for=0A> bridge+traffic shapeing - correct me if I'm wrong.=0A>=0A> = Best regards,=0A> Ugis
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1205499361.47da75e1ac733>