From owner-freebsd-net@FreeBSD.ORG Tue Dec 30 18:40:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93FDBD45 for ; Tue, 30 Dec 2014 18:40:37 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02E532D13 for ; Tue, 30 Dec 2014 18:40:37 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id w7so12574303lbi.2 for ; Tue, 30 Dec 2014 10:40:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gNf+iTaLWAaNsoe07TmUNlbU4B+3ceY8FmvG4vvpd7o=; b=H/NfgXQH9cw2g4fJ55hmfgWL+hcHyVoFyqxnlV+R81HObJy0ziE5OEyvGQL+j0Nlej PUFLtjs4wgkq5MNm1ikpqrqDehqIJWSFn13C0ieqCrJxjUVwGutuHyr98VCKGAVuEe3u U3O0JhCey8HAdQdTk2zBsaxMgP/r7egtw0lrgENipjtaQOyAgWZ6BPrj0YuY60K6VrxU YRcBbYPi7v5s8VhR7jHxELlwbiC8BMDDEQOlfp6TM7E7r7caReZZnSDX+PacmTKmQ6mK ZjdpJloX0sex+09CschQmNihkJeZXGJqFyBlhS538vOKlQauSFVLvbyhio9nvvipsxpv 273w== MIME-Version: 1.0 X-Received: by 10.112.89.232 with SMTP id br8mr24107864lbb.69.1419964834945; Tue, 30 Dec 2014 10:40:34 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.174.169 with HTTP; Tue, 30 Dec 2014 10:40:34 -0800 (PST) In-Reply-To: References: Date: Tue, 30 Dec 2014 19:40:34 +0100 X-Google-Sender-Auth: bx3Ny4GE4NXwmsOvkh13riWiMbk Message-ID: Subject: Re: Regarding Netmap internal memory allocation. From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2014 18:40:37 -0000 On Tue, Dec 30, 2014 at 6:38 PM, Carlos Ferreira wrote: > Ok, I'm having some trouble in tuning the amount of memory for netmap. > > I have been following the man page from FreeBSD in other to understand the > values at /sys/modules/netmap/parameters for linux but I'm having some > trouble in understanding what each value actually means. > For the following values: > > dev.netmap.ring_num: 200 -> Is this the number of rings in the Ring Buffer > Pool? yes. For interfaces with a single queue you need 4 rings (rx+tx for the nic and another rx+tx for the host port) > dev.netmap.ring_size: 36864 -> Is this value, the number of slots per ring? this is the size in bytes of each ring. The number of slots is set by the hardware (low end devices as in the openwrt devices will probably use 256 or 512 slots, so 10-12k should suffice. But this is not worth changing. Instead, you should reduce the number of buffers, though 8MB is only 4000 buffers and it is a bit on the low side for 5 ports. However, as far as I know most openwrt devices only have one physical NIC, and a switch implementing various vlans. cheers luigi > > I'm trying to keep the amount of memory used by netmap as low as 4MB - 8MB > since I'm going to use only up to 4 NICs and one TAP. > > Thanks for the help! > > > On 30 December 2014 at 16:16, Luigi Rizzo wrote: >> >> you can #undefine WITH_VALE. >> But it is only 20K of code (and 150K of data structures, which you >> can further reduce by lowering NM_BRIDGS). >> The saving is probably not worth the effort. >> >> cheers >> luigi >> >> On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira >> wrote: >> > By the way, another question. >> > Is there a way to not compile the code regarding the VALE switch? I'm >> > only >> > interested in using netmap with Tap Devices and NICs, so I was hoping to >> > save some memory. >> > >> > On 30 December 2014 at 15:47, Carlos Ferreira >> > wrote: >> >> >> >> You mean netmap_mem2.c ? It was there where I found the >> >> NETMAP_BUF_MAX_NUM >> >> define. >> >> >> >> >> >> >> >> On 30 December 2014 at 15:43, Carlos Ferreira >> >> wrote: >> >>> >> >>> Ok thanks. I was hoping not having to recompile the module, but it's >> >>> ok. >> >>> Thank you for the info! >> >>> >> >>> >> >>> On 30 December 2014 at 15:38, Luigi Rizzo wrote: >> >>>> >> >>>> you can reduce the amount of ram (buffers, mostly) by >> >>>> tweaking the values in netmap_mem2.c :: >> >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = { >> >>>> ... >> >>>> } >> >>>> >> >>>> or you can simply modify the constant >> >>>> >> >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2 >> >>>> >> >>>> to something smaller that suits an openwrt box >> >>>> (in which i am very interested, as I'd like to deploy one of these >> >>>> soon) >> >>>> >> >>>> cheers >> >>>> luigi >> >>>> >> >>>> >> >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira >> >>>> >> >>>> wrote: >> >>>> > Update: >> >>>> > >> >>>> > I noticed that the netmap module was still crashing, after >> >>>> > changing >> >>>> > the >> >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer >> >>>> > crashed. >> >>>> > The >> >>>> > netmap module is now consuming about 350MB of Ram, which for my >> >>>> > objectives >> >>>> > is just too much... >> >>>> > >> >>>> > On 30 December 2014 at 14:06, Carlos Ferreira >> >>>> > >> >>>> > wrote: >> >>>> > >> >>>> >> To Luigi and to whom may be able to help >> >>>> >> >> >>>> >> Hello all. >> >>>> >> >> >>>> >> Is it possible to reduce the size of the memory buffer allocated >> >>>> >> by >> >>>> >> the >> >>>> >> netmap module? >> >>>> >> I'm asking this because I was implementing some testing code, >> >>>> >> using >> >>>> >> NICs >> >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM. >> >>>> >> Because of the small RAM amount, the nm_open crashed when the >> >>>> >> program >> >>>> >> tried to netmap the tap device, after I previously netmapped one >> >>>> >> NIC >> >>>> >> successfully. >> >>>> >> After the crash, I bumped the VM RAM to 256MB and the test program >> >>>> >> ran >> >>>> >> well, but not without me noticing that the VM RAM consumption was >> >>>> >> increased about 90 MB by netmap. >> >>>> >> >> >>>> >> Resuming, I want to know if there is a way to reduce the memory >> >>>> >> buffer >> >>>> >> allocation, without recompiling the netmap kernel module. >> >>>> >> >> >>>> >> Thank you for the attention. >> >>>> >> >> >>>> >> -- >> >>>> >> >> >>>> >> Carlos Miguel Ferreira >> >>>> >> Researcher at Telecommunications Institute >> >>>> >> Aveiro - Portugal >> >>>> >> Work E-mail - cmf@av.it.pt >> >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >>>> >> >> >>>> > >> >>>> > >> >>>> > >> >>>> > -- >> >>>> > >> >>>> > Carlos Miguel Ferreira >> >>>> > Researcher at Telecommunications Institute >> >>>> > Aveiro - Portugal >> >>>> > Work E-mail - cmf@av.it.pt >> >>>> > Skype & GTalk -> carlosmf.pt@gmail.com >> >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >>>> > _______________________________________________ >> >>>> > freebsd-net@freebsd.org mailing list >> >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >>>> > To unsubscribe, send any mail to >> >>>> > "freebsd-net-unsubscribe@freebsd.org" >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> >> >>>> >> >>>> -----------------------------------------+------------------------------- >> >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> >>>> dell'Informazione >> >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> >>>> TEL +39-050-2211611 . via Diotisalvi 2 >> >>>> Mobile +39-338-6809875 . 56122 PISA (Italy) >> >>>> >> >>>> >> >>>> -----------------------------------------+------------------------------- >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> >> >>> Carlos Miguel Ferreira >> >>> Researcher at Telecommunications Institute >> >>> Aveiro - Portugal >> >>> Work E-mail - cmf@av.it.pt >> >>> Skype & GTalk -> carlosmf.pt@gmail.com >> >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Carlos Miguel Ferreira >> >> Researcher at Telecommunications Institute >> >> Aveiro - Portugal >> >> Work E-mail - cmf@av.it.pt >> >> Skype & GTalk -> carlosmf.pt@gmail.com >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > >> > >> > >> > >> > -- >> > >> > Carlos Miguel Ferreira >> > Researcher at Telecommunications Institute >> > Aveiro - Portugal >> > Work E-mail - cmf@av.it.pt >> > Skype & GTalk -> carlosmf.pt@gmail.com >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------