Date: Wed, 19 Sep 2018 09:33:00 +0900 From: KIRIYAMA Kazuhiko <kiri@kx.openedu.org> To: "Andrey V. Elsukov" <bu7cher@yandex.ru> Cc: KIRIYAMA Kazuhiko <kiri@kx.openedu.org>, freebsd-net@freebsd.org Subject: Re: NFS poor performance in ipfw_nat Message-ID: <201809190033.w8J0X0J5051781@kx.openedu.org> In-Reply-To: <8315728b-afe9-7631-d2ad-2d9b06c3d72d@yandex.ru> References: <201809172253.w8HMrXSS025987@kx.openedu.org> <8315728b-afe9-7631-d2ad-2d9b06c3d72d@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
At Tue, 18 Sep 2018 14:02:18 +0300, Andrey V. Elsukov wrote: > > [1 Re: NFS poor performance in ipfw_nat <multipart/mixed (7bit)>] > [1.1 <text/plain; utf-8 (quoted-printable)>] > On 18.09.2018 01:53, KIRIYAMA Kazuhiko wrote: > > Hi, all > > > > I'm working on ipfw_nat box with port redirect for sunrpc > > (111) and nfsd (2049): > > > > # ifconfig > > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > > options=85259b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO> > > igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > > options=e505bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> > > # ipfw nat show config > > ipfw nat 123 config if em0 log deny_in same_ports unreg_only reset redirect_port tcp 192.168.1.253:22 22253 redirect_port tcp 192.168.1.252:22 22252 redirect_port tcp 192.168.1.251:22 22251 redirect_port tcp 192.168.1.250:22 22250 redirect_port tcp 192.168.1.249:22 22249 redirect_port tcp 192.168.1.248:22 22248 redirect_port tcp 192.168.1.247:22 22247 redirect_port tcp 192.168.1.246:22 22246 redirect_port tcp 192.168.1.245:22 22245 redirect_port tcp 192.168.1.244:22 22244 redirect_port tcp 192.168.1.243:22 22243 > > > Is there any suggestions ? > > > > Hi, > > try to disable TSO on your NICs. Thanks for your advice. Add '-lro' and '-tso' to ifconfig, transfer rate up to almost native NIC speed: # dd if=/dev/zero of=/.dake/tmp/foo.img bs=1k count=1m 1048576+0 records in 1048576+0 records out 1073741824 bytes transferred in 10.688162 secs (100460852 bytes/sec) # BTW in VM on behyve, transfer rate to NFS mount of VM server (bhyve) is appreciably low level: # dd if=/dev/zero of=/.dake/tmp/foo.img bs=1k count=1m 1048576+0 records in 1048576+0 records out 1073741824 bytes transferred in 32.094448 secs (33455687 bytes/sec) # ifconfig vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE> ether 58:9c:fc:0c:10:58 inet 192.168.1.193 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet 10Gbase-T <full-duplex> status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> # df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/vtbd0p2 5060988 4244412 411700 91% / devfs 1 1 0 100% /dev vm.pis:/.dake 14773545597 51624777 14721920820 0% /.dake vm.pis:/ds/src/current/12.0/r338547 14749697607 27776787 14721920820 0% /usr/src vm.pis:/ds/obj/current/12.0/r338547 14772357057 50436237 14721920820 0% /usr/obj # cat /etc/rc.conf hostname="tbedfc" ifconfig_vtnet0="inet 192.168.1.193 netmask 255.255.255.0" defaultrouter="192.168.1.254" sshd_enable="YES" nfs_client_enable="YES" rpc_lockd_enable="YES" rpc_statd_enable="YES" moused_enable="YES" ntpd_enable="YES" powerd_enable="YES" dumpdev="AUTO" # uname -a FreeBSD tbedfc 12.0-ALPHA5 FreeBSD 12.0-ALPHA5 #0: Mon Sep 10 22:19:36 JST 2018 root@tbedfc:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 # It seems to be unconcerned to TSO or LRO ... > > -- > WBR, Andrey V. Elsukov > > [2 OpenPGP digital signature <application/pgp-signature (7bit)>] > --- KIRIYAMA Kazuhiko
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201809190033.w8J0X0J5051781>