Date: Tue, 01 Jul 2008 14:58:13 -0400 From: Paul <paul@gtcomm.net> To: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] Message-ID: <486A7E45.3030902@gtcomm.net> In-Reply-To: <4869B025.9080006@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <alpine.LFD.1.10.0807010257570.19444@filebunker.xip.at> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <alpine.LFD.1.10.0807010308320.19444@filebunker.xip.at> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <ea7b9c170806302005n2a66f592h2127f87a0ba2c6d2@mail.gmail.com> <20080701033117.GH83626@cdnetworks.co.kr> <ea7b9c170806302050p2a3a5480t29923a4ac2d7c852@mail.gmail.com> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net>
next in thread | previous in thread | raw e-mail | index | archive | help
I can't reproduce the 580kpps maximum that I saw when I first compiled for some reason, I don't understand, the max I get even with ULE and preemption is now about 530 and it dips to 480 a lot.. The first time I tried it it was at 580 and dipped to 520...what the?.. (kernel config attached at end) * noticed that SOMETIMES the em0 taskq jumps around cpus and doesn't use 100% of any one cpu * noticed that the netstat packets per second rate varies explicitly with the CPU usage of em0 taskq (top output with ULE/PREEMPTION compiled in): PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 10 root 171 ki31 0K 16K RUN 3 64:12 94.09% idle: cpu3 36 root -68 - 0K 16K CPU1 1 5:43 89.75% em0 taskq 13 root 171 ki31 0K 16K CPU0 0 63:21 87.30% idle: cpu0 12 root 171 ki31 0K 16K RUN 1 62:44 66.75% idle: cpu1 11 root 171 ki31 0K 16K CPU2 2 62:17 56.49% idle: cpu2 39 root -68 - 0K 16K - 0 0:54 10.64% em3 taskq this is about 480-500kpps rate......... now I wait a minute and PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 10 root 171 ki31 0K 16K CPU3 3 64:56 100.00% idle: cpu3 36 root -68 - 0K 16K CPU2 2 6:21 94.14% em0 taskq 13 root 171 ki31 0K 16K RUN 0 63:55 80.18% idle: cpu0 11 root 171 ki31 0K 16K RUN 2 62:48 67.38% idle: cpu2 12 root 171 ki31 0K 16K CPU1 1 63:04 58.40% idle: cpu1 39 root -68 - 0K 16K - 1 1:00 10.21% em3 taskq 530kpps rate....... drops to 85%.. 480kpps rate goes back up to 95% 530kpps it keeps flopping like this........... none of the CPUs are 100% use and none of the cpus add up , like the cpu time of em0 taskq is 94% so one of the cpus should be 6% idle but it's not. This is with ULE/PREEMPTION.. I see different behavior without preemption and with 4bsd.. and I also see different behavior depending on the time of day lol :) Figure that one out I'll post back without preemption and with 4bsd in a min then i'll move on to the 32 bit platform tests
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?486A7E45.3030902>