From owner-freebsd-virtualization@FreeBSD.ORG Mon Jan 26 03:34:00 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63AA9DD6 for ; Mon, 26 Jan 2015 03:34:00 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 20C9FF76 for ; Mon, 26 Jan 2015 03:34:00 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 11D4B280F61 for ; Mon, 26 Jan 2015 13:33:51 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fubPlm9t7M8w for ; Mon, 26 Jan 2015 13:33:50 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 6DAFE280F5C; Mon, 26 Jan 2015 13:33:47 +1000 (EST) Message-ID: <54C5B59B.2070100@freebsd.org> Date: Sun, 25 Jan 2015 19:33:47 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Tiwei Bie Subject: Re: [PATCH] Add netmap support to bhyve References: <1422045440-9410-1-git-send-email-btw@mail.ustc.edu.cn> In-Reply-To: <1422045440-9410-1-git-send-email-btw@mail.ustc.edu.cn> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2015 03:34:00 -0000 Hi, > I saw that there is an open task of adding Netmap support to bhyve in > the latest status report of FreeBSD [1]. So I implement the netmap > support for bhyve, and following is the patch. Hope this can be helpful! Nice work ! Did you manage to get any throughput numbers ? We've had another submission for this as well. I'll do my best to merge the best bits of both into bhyve, though it will have to wait until some restructuring work is done to support multiple network device emulations. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jan 26 07:00:57 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90ED470E; Mon, 26 Jan 2015 07:00:57 +0000 (UTC) Received: from ustc.edu.cn (email6.ustc.edu.cn [IPv6:2001:da8:d800::8]) by mx1.freebsd.org (Postfix) with ESMTP id C712A3EB; Mon, 26 Jan 2015 07:00:56 +0000 (UTC) Received: from freebsd (unknown [58.211.218.74]) by newmailweb.ustc.edu.cn (Coremail) with SMTP id LkAmygCnVzwc5sVUXUm7Ag--.39613S2; Mon, 26 Jan 2015 15:00:52 +0800 (CST) Date: Mon, 26 Jan 2015 15:00:44 +0800 From: Tiwei Bie To: Peter Grehan Subject: Re: [PATCH] Add netmap support to bhyve Message-ID: <20150126070044.GA54990@freebsd> References: <1422045440-9410-1-git-send-email-btw@mail.ustc.edu.cn> <54C5B59B.2070100@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <54C5B59B.2070100@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-CM-TRANSID: LkAmygCnVzwc5sVUXUm7Ag--.39613S2 X-Coremail-Antispam: 1UD129KBjvJXoWxtFWkWF1kWr1rXFWfXFyUtrb_yoWxurykpF W2gF15tr4kZwnxAa9xK3Z0gryfKF1rXFZFg3y5Kry5Kw1UW3yY9ay0yFy5A3y3urZ7AF1U JwsIqry8KrWktF7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUk0b7Iv0xC_Kw4lb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rw A2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xII jxv20xvEc7CjxVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwV C2z280aVCY1x0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC 0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUXVWUAwAv7VC2z280aVAFwI0_Gr0_Cr 1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcVAKI48JMxkIecxEwVAFwVW8CwCF04k20xvY 0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I 0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jrv_JF1lIxkGc2Ij64vIr41lIxAI cVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1lIxAIcV CF04k26cxKx2IYs7xG6rW3Jr0E3s1lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280 aVCY1x0267AKxVWUJVW8JbIYCTnIWIevJa73UjIFyTuYvjxUy3kuDUUUU X-CM-SenderInfo: xewzqzxdloh3xvwfhvlgxou0/1tbiAQUBAVQhl+sX7QABs2 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2015 07:00:57 -0000 On Sun, Jan 25, 2015 at 07:33:47PM -0800, Peter Grehan wrote: > Hi, > > > I saw that there is an open task of adding Netmap support to bhyve in > > the latest status report of FreeBSD [1]. So I implement the netmap > > support for bhyve, and following is the patch. Hope this can be helpful! > > Nice work ! Did you manage to get any throughput numbers ? Thanks! Yeah, I have done some throughput tests. Here are the details: #1. The parameters I used to launch the VMs: $ sudo /home/btw/src/usr.sbin/bhyve/bhyve \ -A -H -P \ -s 0:0,hostbridge \ -s 1:0,lpc \ -s 2,virtio-blk,./disk-vm1 \ -s 3:0,virtio-net,vale0:vm1 \ -s 3:1,virtio-net,tap1 \ -l com1,stdio \ -m 2G -c 2 vm1 $ sudo /home/btw/src/usr.sbin/bhyve/bhyve \ -A -H -P \ -s 0:0,hostbridge \ -s 1:0,lpc \ -s 2,virtio-blk,./disk-vm2 \ -s 3:0,virtio-net,vale0:vm2 \ -s 3:1,virtio-net,tap2 \ -l com1,stdio \ -m 2G -c 1 vm2 #2. The outputs of pkt-gen as the receiver when using tap+bridge: $ ./pkt-gen -i vtnet1 -f rx 075.499873 main [1686] interface is vtnet1 075.500516 extract_ip_range [289] range is 10.0.0.1:0 to 10.0.0.1:0 075.500677 extract_ip_range075.501408 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 [289] range is 075.503540 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 10.1.0.1:0 to 10.1.0.1:0 075.505840 [ 79] vtnet_netmap_free_bufs freed 1024 mbufs, 0 netmap bufs on 1 queues 075.507838 main [1877] mapped 334980KB at 0x801dff000 Receiving from netmap:vtnet1: 1 queues, 1 threads and 1 cpus. 075.509298 main [1963] Wait 2 secs for phy reset 077.553231 main [1965] Ready... 077.553787 nm_open [456] overriding ifname vtnet1 ringid 0x0 flags 0x1 077.554511 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 077.556006 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 077.557663 receiver_body [1219] reading from netmap:vtnet1 fd 4 main_fd 3 078.563897 main_thread [1483] 311989 pps (313942 pkts in 1006261 usec) 079.574224 main_thread [1483] 333345 pps (336771 pkts in 1010278 usec) 080.584718 main_thread [1483] 307455 pps (310688 pkts in 1010514 usec) 081.600552 main_thread [1483] 332357 pps (337629 pkts in 1015862 usec) 082.604157 main_thread [1483] 342985 pps (344222 pkts in 1003606 usec) 083.614341 main_thread [1483] 327806 pps (331144 pkts in 1010184 usec) 084.634725 main_thread [1483] 320947 pps (327489 pkts in 1020385 usec) 085.645380 main_thread [1483] 328959 pps (332456 pkts in 1010630 usec) 086.654096 main_thread [1483] 313482 pps (316222 pkts in 1008740 usec) 087.671256 main_thread [1483] 307234 pps (312506 pkts in 1017161 usec) 088.673871 main_thread [1483] 311188 pps (312002 pkts in 1002615 usec) 089.686453 main_thread [1483] 322132 pps (326185 pkts in 1012582 usec) 090.694224 main_thread [1483] 331368 pps (333922 pkts in 1007708 usec) 091.714343 main_thread [1483] 290061 pps (295915 pkts in 1020181 usec) 092.724065 main_thread [1483] 316366 pps (319442 pkts in 1009722 usec) 093.749938 main_thread [1483] 322501 pps (330845 pkts in 1025874 usec) 094.758012 main_thread [1483] 292125 pps (294484 pkts in 1008074 usec) 095.765606 main_thread [1483] 338889 pps (341463 pkts in 1007594 usec) 096.781560 main_thread [1483] 327785 pps (333013 pkts in 1015949 usec) 097.789245 main_thread [1483] 330064 pps (332602 pkts in 1007690 usec) 098.794141 main_thread [1483] 328757 pps (330367 pkts in 1004896 usec) 099.804160 main_thread [1483] 302752 pps (305785 pkts in 1010019 usec) ^C100.400690 sigint_h [326] received control-C on thread 0x801806800 100.820895 main_thread [1483] 178717 pps (181708 pkts in 1016735 usec) Received 7300802 packets, in 22.84 seconds. Speed: 319.61 Kpps 100.822469 [ 79] vtnet_netmap_free_bufs freed 0 mbufs, 506 netmap bufs on 1 queues #3. The outputs of pkt-gen as the receiver when using netmap+vale: $ ./pkt-gen -i vtnet0 -f rx 139.084350 main [1686] interface is vtnet0 139.085422 extract_ip_range [289] range is 10.0.0.1:0 to 10.0.0.139.086441 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 1:0 139.085705 139.090250 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 extract_ip_range [289] range is 10.1.0.1:0 to 10.1.0.1:0 139.094911 [ 79] vtnet_netmap_free_bufs freed 1024 mbufs, 0 netmap bufs on 1 queues 139.098783 main [1877] mapped 334980KB at 0x801dff000 Receiving from netmap:vtnet0: 1 queues, 1 threads and 1 cpus. 139.099642 main [1963] Wait 2 secs for phy reset 141.142968 main [1965] Ready... 141.143439 nm_op141.143596 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 en [456] overrid141.145487 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 ing ifname vtnet0 ringid 0x0 flags 0x1 141.147778 receiver_body [1219] reading from netmap:vtnet0 fd 4 main_fd 3 142.165927 main_thread [1483] 616128 pps (627330 pkts in 1018181 usec) 143.195891 main_thread [1483] 617845 pps (636357 pkts in 1029963 usec) 144.226279 main_thread [1483] 640422 pps (659884 pkts in 1030389 usec) 145.234087 main_thread [1483] 642099 pps (647113 pkts in 1007808 usec) 146.244540 main_thread [1483] 642227 pps (648839 pkts in 1010296 usec) 147.286503 main_thread [1483] 642392 pps (669410 pkts in 1042058 usec) 148.301086 main_thread [1483] 616898 pps (625932 pkts in 1014645 usec) 149.314164 main_thread [1483] 643629 pps (652046 pkts in 1013078 usec) 150.324461 main_thread [1483] 644681 pps (651319 pkts in 1010297 usec) 151.341187 main_thread [1483] 615727 pps (626025 pkts in 1016725 usec) 152.350815 main_thread [1483] 617538 pps (623484 pkts in 1009629 usec) 153.382287 main_thread [1483] 641915 pps (662117 pkts in 1031472 usec) 154.384370 main_thread [1483] 642324 pps (643662 pkts in 1002083 usec) 155.395499 main_thread [1483] 590924 pps (597500 pkts in 1011129 usec) 156.412232 main_thread [1483] 616879 pps (627201 pkts in 1016733 usec) 157.415682 main_thread [1483] 627524 pps (629689 pkts in 1003450 usec) 158.453857 main_thread [1483] 617421 pps (640991 pkts in 1038175 usec) 159.465732 main_thread [1483] 617130 pps (624458 pkts in 1011875 usec) 160.478797 main_thread [1483] 643278 pps (651682 pkts in 1013065 usec) 161.494513 main_thread [1483] 642237 pps (652330 pkts in 1015716 usec) 162.504104 main_thread [1483] 642578 pps (648740 pkts in 1009589 usec) 163.526906 main_thread [1483] 616442 pps (630460 pkts in 1022741 usec) 164.542778 main_thread [1483] 644121 pps (654385 pkts in 1015935 usec) ^C165.214479 sigint_h [326] received control-C on thread 0x801806800 165.580379 main_thread [1483] 415359 pps (430977 pkts in 1037601 usec) Received 15161931 packets, in 24.07 seconds. Speed: 630.00 Kpps 165.581943 [ 79] vtnet_netmap_free_bufs freed 0 mbufs, 488 netmap bufs on 1 queues Tiwei Bie From owner-freebsd-virtualization@FreeBSD.ORG Fri Jan 30 08:25:25 2015 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C2C6AF6 for ; Fri, 30 Jan 2015 08:25:25 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 636CB610 for ; Fri, 30 Jan 2015 08:25:25 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t0U8PP4Z044594 for ; Fri, 30 Jan 2015 08:25:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 195238] [Hyper-v] Enhance VMBUS and improve storage performance Date: Fri, 30 Jan 2015 08:25:25 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: feature, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kyliel@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2015 08:25:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195238 Kylie changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd-virtualization@Free | |BSD.org -- You are receiving this mail because: You are on the CC list for the bug.