Date: Thu, 9 Sep 2010 05:02:29 +0530 From: Paul Joe <apauljoe@gmail.com> To: Luigi Rizzo <rizzo@iet.unipi.it>, julian <julian@elischer.org>, apauljoe@gmail.com Cc: freebsd-ipfw@freebsd.org Subject: Re: Extension of dummynet/ipfw to support userspace packet classification Message-ID: <AANLkTim4-cdAUzwKbwttRjyy=-YTT3Bh6=1OT4R%2B%2BAMb@mail.gmail.com> In-Reply-To: <20091007230909.GB37005@onelab2.iet.unipi.it> References: <286e18280910071246r33d33476ya9dd846cd1de6062@mail.gmail.com> <20091007225452.GA37005@onelab2.iet.unipi.it> <20091007230909.GB37005@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
--001636164a2b5543c3048fc7ed55 Content-Type: text/plain; charset=ISO-8859-1 On 10/8/09, Luigi Rizzo <rizzo@iet.unipi.it> wrote: > On Thu, Oct 08, 2009 at 12:54:52AM +0200, Luigi Rizzo wrote: >> On Wed, Oct 07, 2009 at 12:46:24PM -0700, Joe R wrote: >> > We at ironport have a requirement to do bandwidth management, but the >> > traffic classification (and selection of bandwidth pipes) is done in >> > userspace. The reason classification is done in userspace is because the >> > traffic classifications are something like streaming audio traffic, >> > video >> > traffic, based on website categories etc. >> > >> > >> > >> > Our appliance is based on FreeBSD, and so we decided to look at dummynet >> > to >> > support our requirement. We could not use dummynet as such because it >> > uses >> > ipfw for packet classification, where packet classification (and pipe >> > selection) is done in kernel based on tcp/ip parameters like IP and >> > port. >> > >> > >> > >> > So we decided to extended dummynet/ipfw to support packet classification >> > in >> > userspace. >> > >> > Our idea is to extended socket structure to have a pipe number and have >> > a >> > setsockoption to associate the pipe number to a socket structure. Then >> > have >> > a new ipfw target (mappedpipe), which will pass the packet to dummynet >> > (similar to pipe target) but with the pipe number in the socket >> > structure if >> > it is non-zero. >> > >> > >> > >> > I would like to know your comments on this proposal and if people are >> > interested, I will be happy to submit a patch on this. >> >> i think the feature is useful. However I would implement it as an >> ipfw 'option' called "sockarg" (or similar) as follows: >> >> ipfw pipe tablearg sockarg >> >> where 'sockarg' succeeds ONLY if the packet is associated to a socket >> for which the special setsockoption has been issued, and in this >> case sets the 'tablearg' to the value of the setsockopt. This is >> somewhat similar to the 'uid' and 'gid' options (except for setting >> tablearg). This way the mechanism can be very general (not limited >> to pipes) and the implementation is probably >> simpler than the one you propose. >> >> In terms of runtime costs, we can look at check_uidgid() function, >> and there are two ways to implement this feature: >> - as in check_uidgid() , actively lookup for a matching socket if one >> is not available. This is expensive but would allow the feature to >> match also incoming packets; >> - only match if the args->inp parameter is non-null, otherwise do not >> call in_pcblookup_hash(). This is cheaper but clearly only works >> for locally generated packets. >> Perhaps we could use an argument for 'sockarg' so we can decide >> whether to call or not the in_pcblookup_hash() on a case-by-case >> basis. > > To complete the analysis, I must say that I don't know how intrusive > is the setsockopt that can attach a classification tag to the socket. > This is my main concern for merging your proposal into the system > (and i am only concerned about the socket part, the ipfw change is > trivial). > > Also for completeness, there is also another possible approach to > address your problem, which is more general and fully contained in > ipfw (so less intrusive for the OS): > > add a 'hashtable' structure to ipfw, which works in a way similar > to the 'table' with the difference that entries would be the whole > 5-tuple of the packet. > > There is already a hash table in ipfw (used for dynamic rules) so > it would be only a matter of adding the necessary glue to manipulate > the hash table from /sbin/ipfw. An additional bonus of this approach > is that one could use this new code to 'prime' the dynamic rule table > after a reboot, which is a feature that people ask from time to time. > > cheers > luigi > Hi, I am attaching a patch taken against HEAD today which implements the socket and ipfw sockarg option as discussed in the thread. Applying this patch, you can associate a pipe to the socket using the setsocket option(in userspace) and an ipfw rule similar to ipfw add 100 pipe tablearg sockarg will forward the traffic to the pipe associated with the socket. Please let me know your comments. Regards, Joe. --001636164a2b5543c3048fc7ed55 Content-Type: text/x-diff; charset=US-ASCII; name=patch1 Content-Disposition: attachment; filename=patch1 Content-Transfer-Encoding: base64 X-Attachment-Id: file0 SW5kZXg6IHNyYy9zYmluL2lwZncvaXBmdzIuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvaG9tZS9u Y3ZzL3NyYy9zYmluL2lwZncvaXBmdzIuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS4xNTkKZGlm ZiAtYyAtdSAtcjEuMTU5IGlwZncyLmMKLS0tIHNyYy9zYmluL2lwZncvaXBmdzIuYwkxOSBBcHIg MjAxMCAxNjozNTo0NyAtMDAwMAkxLjE1OQorKysgc3JjL3NiaW4vaXBmdy9pcGZ3Mi5jCTggU2Vw IDIwMTAgMjI6Mjk6NDggLTAwMDAKQEAgLTI2Niw2ICsyNjYsNyBAQAogCXsgImVzdGFiIiwJCVRP S19FU1RBQiB9LAogCXsgImVzdGFibGlzaGVkIiwJVE9LX0VTVEFCIH0sCiAJeyAic2V0dXAiLAkJ VE9LX1NFVFVQIH0sCisJeyAic29ja2FyZyIsCQlUT0tfU09DS0FSRyB9LAogCXsgInRjcGRhdGFs ZW4iLAkJVE9LX1RDUERBVEFMRU4gfSwKIAl7ICJ0Y3BmbGFncyIsCQlUT0tfVENQRkxBR1MgfSwK IAl7ICJ0Y3BmbGdzIiwJCVRPS19UQ1BGTEFHUyB9LApAQCAtMTMzOCw2ICsxMzM5LDkgQEAKIAkJ CWNhc2UgT19GSUI6CiAJCQkJcHJpbnRmKCIgZmliICV1IiwgY21kLT5hcmcxICk7CiAJCQkJYnJl YWs7CisJCQljYXNlIE9fU09DS0FSRzoKKwkJCQlwcmludGYoIiBzb2NrYXJnIik7CisJCQkJYnJl YWs7CiAKIAkJCWNhc2UgT19JTjoKIAkJCQlwcmludGYoY21kLT5sZW4gJiBGX05PVCA/ICIgb3V0 IiA6ICIgaW4iKTsKQEAgLTM1MzEsNiArMzUzNSw5IEBACiAJCQlmaWxsX2NtZChjbWQsIE9fRklC LCAwLCBzdHJ0b3VsKCphdiwgTlVMTCwgMCkpOwogCQkJYXYrKzsKIAkJCWJyZWFrOworCQljYXNl IFRPS19TT0NLQVJHOgorCQkJZmlsbF9jbWQoY21kLCBPX1NPQ0tBUkcsIDAsIDApOworCQkJYnJl YWs7CiAKIAkJY2FzZSBUT0tfTE9PS1VQOiB7CiAJCQlpcGZ3X2luc25fdTMyICpjID0gKGlwZndf aW5zbl91MzIgKiljbWQ7CkluZGV4OiBzcmMvc2Jpbi9pcGZ3L2lwZncyLmgKPT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpS Q1MgZmlsZTogL2hvbWUvbmN2cy9zcmMvc2Jpbi9pcGZ3L2lwZncyLmgsdgpyZXRyaWV2aW5nIHJl dmlzaW9uIDEuMTMKZGlmZiAtYyAtdSAtcjEuMTMgaXBmdzIuaAotLS0gc3JjL3NiaW4vaXBmdy9p cGZ3Mi5oCTE5IEFwciAyMDEwIDE1OjExOjQ1IC0wMDAwCTEuMTMKKysrIHNyYy9zYmluL2lwZncv aXBmdzIuaAk4IFNlcCAyMDEwIDIyOjI5OjQ4IC0wMDAwCkBAIC0xOTksNiArMTk5LDcgQEAKIAlU T0tfRklCLAogCVRPS19TRVRGSUIsCiAJVE9LX0xPT0tVUCwKKwlUT0tfU09DS0FSRywKIH07CiAv KgogICogdGhlIGZvbGxvd2luZyBtYWNybyByZXR1cm5zIGFuIGVycm9yIG1lc3NhZ2UgaWYgd2Ug cnVuIG91dCBvZgpJbmRleDogc3JjL3N5cy9rZXJuL3VpcGNfc29ja2V0LmMKPT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpS Q1MgZmlsZTogL2hvbWUvbmN2cy9zcmMvc3lzL2tlcm4vdWlwY19zb2NrZXQuYyx2CnJldHJpZXZp bmcgcmV2aXNpb24gMS4zNDkKZGlmZiAtYyAtdSAtcjEuMzQ5IHVpcGNfc29ja2V0LmMKLS0tIHNy Yy9zeXMva2Vybi91aXBjX3NvY2tldC5jCTcgQXVnIDIwMTAgMTc6NTc6NTggLTAwMDAJMS4zNDkK KysrIHNyYy9zeXMva2Vybi91aXBjX3NvY2tldC5jCTggU2VwIDIwMTAgMjI6Mjk6NTQgLTAwMDAK QEAgLTEyMyw2ICsxMjMsOCBAQAogI2luY2x1ZGUgPHN5cy9zb2NrZXR2YXIuaD4KICNpbmNsdWRl IDxzeXMvcmVzb3VyY2V2YXIuaD4KICNpbmNsdWRlIDxuZXQvcm91dGUuaD4KKyNpbmNsdWRlIDxu ZXRpbmV0L2luLmg+CisjaW5jbHVkZSA8bmV0aW5ldC9pcF92YXIuaD4KICNpbmNsdWRlIDxzeXMv c2lnbmFsdmFyLmg+CiAjaW5jbHVkZSA8c3lzL3N0YXQuaD4KICNpbmNsdWRlIDxzeXMvc3guaD4K QEAgLTI0NjEsNiArMjQ2MywyNiBAQAogCQkJCXNvLT5zb19maWJudW0gPSAwOwogCQkJfQogCQkJ YnJlYWs7CisKKwkJY2FzZSBTT19TRVRCUElQRToKKwkJCWlmKGlwX2RuX2lvX3B0ciA9PSBOVUxM KXsKKwkJCQllcnJvciA9IEVOT1BST1RPT1BUOworCQkJCWdvdG8gYmFkOworCQkJfQorCisJCQll cnJvciA9IHNvb3B0Y29weWluKHNvcHQsICZvcHR2YWwsIHNpemVvZiBvcHR2YWwsCisJCQkJCQlz aXplb2Ygb3B0dmFsKTsKKwkJCXByaW50Zigib3B0IHZhbCBpcyAlZCBcbiIsIG9wdHZhbCk7CisJ CQlpZiAob3B0dmFsIDwgMCB8fCBlcnJvciApeworCQkJCWVycm9yPSBFSU5WQUw7IAorCQkJCWdv dG8gYmFkOworCQkJfQorCQorCQkJaWYoc28tPnNvX3Byb3RvLT5wcl9kb21haW4tPmRvbV9mYW1p bHkgPT0gUEZfSU5FVCkgCisJCQkJc28tPnNvX3BpcGVudW0gPSBvcHR2YWw7CisJCQkKKwkJCWJy ZWFrOworCiAJCWNhc2UgU09fU05EQlVGOgogCQljYXNlIFNPX1JDVkJVRjoKIAkJY2FzZSBTT19T TkRMT1dBVDoKSW5kZXg6IHNyYy9zeXMvbmV0aW5ldC9pcF9mdy5oCj09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZp bGU6IC9ob21lL25jdnMvc3JjL3N5cy9uZXRpbmV0L2lwX2Z3LmgsdgpyZXRyaWV2aW5nIHJldmlz aW9uIDEuMTM4CmRpZmYgLWMgLXUgLXIxLjEzOCBpcF9mdy5oCi0tLSBzcmMvc3lzL25ldGluZXQv aXBfZncuaAkxNSBNYXIgMjAxMCAxNzoxNDoyNyAtMDAwMAkxLjEzOAorKysgc3JjL3N5cy9uZXRp bmV0L2lwX2Z3LmgJOCBTZXAgMjAxMCAyMjoyOTo1OCAtMDAwMApAQCAtMTkyLDEwICsxOTIsMTMg QEAKIAogCU9fU0VURklCLAkJLyogYXJnMT1GSUIgbnVtYmVyICovCiAJT19GSUIsCQkJLyogYXJn MT1GSUIgZGVzaXJlZCBmaWIgbnVtYmVyICovCisJCisJT19TT0NLQVJHLAkJLyogc29ja2V0IGFy Z3VtZW50ICovCiAKIAlPX0xBU1RfT1BDT0RFCQkvKiBub3QgYW4gb3Bjb2RlIQkJKi8KIH07CiAK KwogLyoKICAqIFRoZSBleHRlbnNpb24gaGVhZGVyIGFyZSBmaWx0ZXJlZCBvbmx5IGZvciBwcmVz ZW5jZSB1c2luZyBhIGJpdAogICogdmVjdG9yIHdpdGggYSBmbGFnIGZvciBlYWNoIGhlYWRlci4K SW5kZXg6IHNyYy9zeXMvbmV0aW5ldC9pcGZ3L2lwX2Z3Mi5jCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6 IC9ob21lL25jdnMvc3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZncyLmMsdgpyZXRyaWV2aW5nIHJl dmlzaW9uIDEuNDUKZGlmZiAtYyAtdSAtcjEuNDUgaXBfZncyLmMKLS0tIHNyYy9zeXMvbmV0aW5l dC9pcGZ3L2lwX2Z3Mi5jCTI3IEp1bCAyMDEwIDE0OjI2OjM0IC0wMDAwCTEuNDUKKysrIHNyYy9z eXMvbmV0aW5ldC9pcGZ3L2lwX2Z3Mi5jCTggU2VwIDIwMTAgMjI6MzA6MDUgLTAwMDAKQEAgLTE4 MDEsNiArMTgwMSwzOSBAQAogCQkJCQltYXRjaCA9IDE7CiAJCQkJYnJlYWs7CiAKKwkJCWNhc2Ug T19TT0NLQVJHOgl7CisJCQkJc3RydWN0IGlucGNiICppbnAgPSBhcmdzLT5pbnA7CisJCQkJc3Ry dWN0IGlucGNiaW5mbyAqcGk7CisJCQkJCisJCQkJaWYoaXNfaXB2NikKKwkJCQkJYnJlYWs7CisK KwkJCQlpZihwcm90byA9PSBJUFBST1RPX1RDUCkKKwkJCQkJcGkgPSAmVl90Y2JpbmZvOworCQkJ CWVsc2UgaWYgKHByb3RvID09IElQUFJPVE9fVURQKQorCQkJCQlwaSA9ICZWX3VkYmluZm87CisJ CQkJZWxzZQorCQkJCQlicmVhazsKKworCQkJCS8qIEZvciBpbmNvbW1pbmcgcGFja2V0LCBsb29r dXAgdXAgdGhlIAorCQkJCWlucGNiIHVzaW5nIHRoZSBzcmMvZGVzdCBpcC9wb3J0IHR1cGxlICov CisJCQkJaWYoaW5wID09IE5VTEwpIHsKKwkJCQkJSU5QX0lORk9fUkxPQ0socGkpOworCQkJCQlp bnAgPSBpbl9wY2Jsb29rdXBfaGFzaChwaSwgCisJCQkJCQlzcmNfaXAsIGh0b25zKHNyY19wb3J0 KSwKKwkJCQkJCWRzdF9pcCwgaHRvbnMoZHN0X3BvcnQpLAorCQkJCQkJMCwgTlVMTCk7CisJCQkJ CUlOUF9JTkZPX1JVTkxPQ0socGkpOworCQkJCX0KKwkJCQkKKwkJCQlpZihpbnAgJiYgaW5wLT5p bnBfc29ja2V0KSB7CisJCQkJCXRhYmxlYXJnID0gaW5wLT5pbnBfc29ja2V0LT5zb19waXBlbnVt OworCQkJCQlpZih0YWJsZWFyZykKKwkJCQkJCW1hdGNoID0gMTsKKwkJCQl9CisJCQkJYnJlYWs7 CisJCQl9CisKIAkJCWNhc2UgT19UQUdHRUQ6IHsKIAkJCQlzdHJ1Y3QgbV90YWcgKm10YWc7CiAJ CQkJdWludDMyX3QgdGFnID0gKGNtZC0+YXJnMSA9PSBJUF9GV19UQUJMRUFSRykgPwpJbmRleDog c3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZndfc29ja29wdC5jCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6 IC9ob21lL25jdnMvc3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZndfc29ja29wdC5jLHYKcmV0cmll dmluZyByZXZpc2lvbiAxLjE3CmRpZmYgLWMgLXUgLXIxLjE3IGlwX2Z3X3NvY2tvcHQuYwotLS0g c3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZndfc29ja29wdC5jCTcgQXByIDIwMTAgMDg6MjM6NTgg LTAwMDAJMS4xNworKysgc3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZndfc29ja29wdC5jCTggU2Vw IDIwMTAgMjI6MzA6MDYgLTAwMDAKQEAgLTU3Miw2ICs1NzIsNyBAQAogCQljYXNlIE9fSVBUT1M6 CiAJCWNhc2UgT19JUFBSRUNFREVOQ0U6CiAJCWNhc2UgT19JUFZFUjoKKwkJY2FzZSBPX1NPQ0tB Ukc6CiAJCWNhc2UgT19UQ1BXSU46CiAJCWNhc2UgT19UQ1BGTEFHUzoKIAkJY2FzZSBPX1RDUE9Q VFM6CkluZGV4OiBzcmMvc3lzL3N5cy9zb2NrZXQuaAo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvaG9t ZS9uY3ZzL3NyYy9zeXMvc3lzL3NvY2tldC5oLHYKcmV0cmlldmluZyByZXZpc2lvbiAxLjEwNQpk aWZmIC1jIC11IC1yMS4xMDUgc29ja2V0LmgKLS0tIHNyYy9zeXMvc3lzL3NvY2tldC5oCTkgSmFu IDIwMTAgMjM6MjQ6NDkgLTAwMDAJMS4xMDUKKysrIHNyYy9zeXMvc3lzL3NvY2tldC5oCTggU2Vw IDIwMTAgMjI6MzA6MDcgLTAwMDAKQEAgLTEzNyw2ICsxMzcsNyBAQAogI2RlZmluZQlTT19MSVNU RU5RTEVOCTB4MTAxMgkJLyogc29ja2V0J3MgY29tcGxldGUgcXVldWUgbGVuZ3RoICovCiAjZGVm aW5lCVNPX0xJU1RFTklOQ1FMRU4JMHgxMDEzCS8qIHNvY2tldCdzIGluY29tcGxldGUgcXVldWUg bGVuZ3RoICovCiAjZGVmaW5lCVNPX1NFVEZJQgkweDEwMTQJCS8qIHVzZSB0aGlzIEZJQiB0byBy b3V0ZSAqLworI2RlZmluZSBTT19TRVRCUElQRQkweDEwMTUJCS8qIHVzZSB0aGlzIHBpcGUgdG8g dGhyb3R0bGUgKi8KICNlbmRpZgogCiAvKgpJbmRleDogc3JjL3N5cy9zeXMvc29ja2V0dmFyLmgK PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQpSQ1MgZmlsZTogL2hvbWUvbmN2cy9zcmMvc3lzL3N5cy9zb2NrZXR2YXIuaCx2 CnJldHJpZXZpbmcgcmV2aXNpb24gMS4xNzMKZGlmZiAtYyAtdSAtcjEuMTczIHNvY2tldHZhci5o Ci0tLSBzcmMvc3lzL3N5cy9zb2NrZXR2YXIuaAkxOCBKdWwgMjAxMCAyMDo1Nzo1MyAtMDAwMAkx LjE3MworKysgc3JjL3N5cy9zeXMvc29ja2V0dmFyLmgJOCBTZXAgMjAxMCAyMjozMDowNyAtMDAw MApAQCAtMTE4LDYgKzExOCw3IEBACiAJCWNoYXIJKnNvX2FjY2VwdF9maWx0ZXJfc3RyOwkvKiBz YXZlZCB1c2VyIGFyZ3MgKi8KIAl9ICpzb19hY2NmOwogCWludCBzb19maWJudW07CQkvKiByb3V0 aW5nIGRvbWFpbiBmb3IgdGhpcyBzb2NrZXQgKi8KKwlpbnQgc29fcGlwZW51bTsKIH07CiAKIC8q Cg== --001636164a2b5543c3048fc7ed55--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTim4-cdAUzwKbwttRjyy=-YTT3Bh6=1OT4R%2B%2BAMb>