Date: Sun, 7 Aug 2016 01:03:20 +0700 From: Eugene Grosbein <eugen@grosbein.net> To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: 40Gbps http client benchmark Message-ID: <57A62668.7020309@grosbein.net>
next in thread | raw e-mail | index | archive | help
Hi! Is there any high performance benchmark acting as http client for outer http server capable to receive 40Gbps without overwhelming CPU with insane number of syscalls? I've tried benchmarks/wrk version 4.0.2 and it works just fine upto 20Gbps for my hardware: two 6-core (HT disabled) Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz with two dual-port ix(4) 82599ES 10-Gigabit SFI/SFP+ Network Connection combined to single lagg interface (lagghash l4). But each worker pthread of wrk generates too many kqueue() system calls polling for incoming data and eats 100% of its CPU core and cannot receive more. Or, it may be some kqueue() kernel level lock contention, I do not know. More worker threads, more overloaded CPU cores, no increase of transfer over about 20Gbps. I transfer 1MByte-sized files for the benchmark (tried 8MB-sized too). I set sysctl net.inet.tcp.recvspace=4194304 (4MB). I've even patched wrk to to use read buffer sized 4MB instead of its compiled-in default 8k, as it does not change default socket options other than TCP_NODELAY: --- src/wrk.h.orig 2016-08-06 23:20:16.205906000 +0700 +++ src/wrk.h 2016-08-06 23:20:20.460579000 +0700 @@ -18,7 +18,7 @@ #include "ae.h" #include "http_parser.h" -#define RECVBUF 8192 +#define RECVBUF 4194304 #define MAX_THREAD_RATE_S 10000000 #define SOCKET_TIMEOUT_MS 2000 Nothing helps to decrease consumed system time and ktrace(1) assures that's because of kevent() calls.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?57A62668.7020309>