From owner-freebsd-arch@FreeBSD.ORG Wed Aug 17 23:25:37 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01AF7106564A for ; Wed, 17 Aug 2011 23:25:37 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 03EE88FC15 for ; Wed, 17 Aug 2011 23:25:35 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QtpFo-0006jO-5x for freebsd-arch@freebsd.org; Thu, 18 Aug 2011 01:10:32 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 01:10:32 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 01:10:32 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Wed, 17 Aug 2011 23:10:19 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 733 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org Summary: Project is experiencing serious social/technical problems, we must solve them or die X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: All User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2011 23:25:37 -0000 Hi, I have bad news. A month ago techlead of Rambler's Mail department declared in his blog that they begin migration from FreeBSD to Debian/Ubuntu. Comments to the blog entry (there was much flame) revealed that search department of Russian search engine #1 Yandex.com also plans to migrate their search cluster - about 30,000 servers in several DCs (~60% of total their servers - to Linux. The problem here is that both big companies were using FreeBSD from the beginning (~1997), so migration will be rather expensive. The official reasons (really semi-official, as these are individual blogs), in short, were: * inadequate package manager and huge monolithic base system * lack of OpenVZ-like virtualization (need CPU limiting) * a FreeBSD marketshare forecast for 5 years Despite of several our committers working for them, the operating expenses for FreeBSD are considered too high by these companies. They've not confirmed, but the key problem is probably shortage of FreeBSD specialists on the market to hire (e.g. Yandex has about 20 FreeBSD admins and about 84 Linux admins, for all other their services and 40% of servers). So this is problem of FreeBSD's too low userbase/marketshare, caused, in turn, by other FreeBSD's problems. Given that they is just planning today, but not yet started, we have to solve our problems or FreeBSD will effectively die[1]. Currently https://ssl.netcraft.com/ssl-sample-report//CMatch/oscnt_all shows that still 3% of servers are running FreeBSD. We are currently probably at the tip of 'hype curve'[2] and then our userbase will begin to shrink, and the task is to solve problems, so after hard time it will rise again (hopefully more than it was before). [1] 'Die' means shrinking userbase size to those of NetBSD/OpenBSD. [2] http://www.metodolog.ru/01493/2.gif Earlier this year in this list marcel@ have said about objective view: http://lists.freebsd.org/pipermail/freebsd-arch/2011-January/010999.html from the outside, not FreeBSD people. I have posted a call to Russian community in my blog (http://nuclight.livejournal.com/128712.html) and gathered a list of problems along with possible solutions to some of them. The rest of this message is summary of them. In general, people sympathizing FreeBSD agree that there are no fatal technical problems and that FreeBSD could compete with mainstream Linux. That said, the 80/20 principle is in effect and 80% losed users for FreeBSD are caused by 20% problems. The trouble here is that what "outside" people view as considerable problems, we here inside FreeBSD view as something insignificant ("do it yourself"). Thus 80% could be achieved by 20% of work. I have sorted them in order of decreased importance (priority): 1. Social problems of community (and marketing, docs, ...) 2. Lack of drivers and virtualization. 3. Ports and packages. 4. Base system, closely related with packages. 5. Too short major releases' cycle. 6. Bug tracker, unicode and other less important trivia. (bonus) FreeBSD strengths to be concentrated on. Now all of them more detailed. > 1. Social (psychologic) problems of community (marketing, docs, ...). This is the most important one, because all technical problems are just won't get solved because are even not viewed as problems. The FreeBSD Project does not listen to users' needs. The typical response when poor user want something is: "we don't need this, we won't change for you", with "where are your patches?" at best. Then many users go out when see such attitude toward them. The key points are: 1) *The competent user is not zealot*. 2) The system is *for users, not for developers*. With first: when user sees the system costs too much time for him, and that those problems won't get fixed and even considered such - he will switch to another OS. This may mean that he is able to follow our advice hoe to do some or another thing (e.g. to recompile all ports), but this is unacceptable to him because this is too much maintenance (another system by objectivity requires less work). Many businesses fall into this category. They will not with FreeBSD by any price just because they love OS. Unlike ourselves. With second: that's simple and not simple. It is simple in that by nature each person don't make a work just for himself but for all others, who are now "users" to him, too - just by induction that's not for developers only. It is not simple due to question "Why do we need those who don't contribute anything back to us?" which random committer has right to ask. The answer, in short, is: there tasks which could be only done by large groups of people, e.g. big corporations. For corporations to invest and contribute back - the system must be popular enough. To be popular enough means there must be a big amount of users who is not contributing anything. It is useful in itself, though, that some of those users, say 1%, will become contributors, that is, absolute number of our developers will also benefit from FreeBSD being much more popular. There are opinions in our community like: | > Personally I'd like to think that that we write an OS for users. | | We write an OS for the people who can and will use an OS written by | us. | FreeBSD is whatever its developers make it be These are wrong and harmful nowadays. The world has changed. We have to admit it or we will die. We have to admit mistakes and *change* some of the ways we're doing things: any answer like "I don't agree, we don't need these users/features/etc., we *won't* do this for someone" - is just another step to grave. Of course that doesn't mean to do everything - we often have not enough time/resources to do. But that's another question and should not be mixed with attitude - "it's good but we have no resources, will you do?" vs "we don't need this, go away" (that's not only about features but about keeping something too). So ("system is for users") we need to (re)define who our user is. I view the current situation as following: 100% .---------------. The stratification of overall | Ubuntu | our userbase. Areas are: | target | 6 | audience | 1 - official developers expand -> |---------------| to here | not our, but | 2 - actively participating and | competent | 5 sending patches, contributors | users | and other "zealots" (whose |===============| "soul is with FreeBSD") |announce@ only | 4 involved |---------------| 3 - not so active but discussing | subscribed to | in mail lists, patches go | our mailing | 3 sometimes | lists | committed |- - - - - - - -| 4 - busy with work to read lists | | 2 |_______________| 5 - can use, but costs are now high | *@freebsd.org | 1 0% `---------------' 6 - we'll never want them The terms "involved" and "committed" were taken from http://en.wikipedia.org/wiki/The_Chicken_and_the_Pig for those who are "at the heart" of the community, at "periphery" and all others. Currently, only those in (1) and partially (2) are listened to, sometimes with "call for ..." in (3). Meanwhile, the most important of our users, who are using FreeBSD in production, who are busy with real work in real world, are in (4) and sometimes (5). Those are largely ignored by the Project - even messages to announce@ last years are less about important events and calls for volunteers/money. The other social problem is lack of companies which offer commercial support of FreeBSD like RedHat does. As a possible solution to find out what user needs are, and to compensate the fact that not all users in (4) and (5) care enough about our principles and spirit, I propose a system for "weighted democracy". This is a voting system (I could implement it), say, there is a mail voting@freebsd.org to which users send filled in vote forms, with selected answers from a survey published in announce@. The system has a database where users are recorded given their FreeBSD activity in mail lists etc., and votes are summed as follows: + 1.00 vote for anyone not in database (not involved to FreeBSD) + a decimal logarithm of number of posts to mail lists (2 votes for 100 posts, 3 votes for 1000 posts) + a binary logarithm of num PRs in GNATS db (6 votes for 64 send-pr's) + a proportion (say, N/2) of entries for this person in "svn log" (e.g. in "Submitted by:") + an assigned (by core@) number of votes in special exceptional DB (for corporation and the like) The system then presents results for each answer: 1) how many users voted, 2) how many votes summed. This is by no mean to measure "exact contribution", but to defend from anonymous users and trolls who not care about amount of work developers will need to. The results are viewed as a feedback to core team, not as a final decision - that's all for what marketing is solely exist. There no adequate feedback from users to developers currently at all - individual posts in mail lists are too small for statistics (what about ministat(1) for users, huh?..). What is also in this part? Documentation. No, not in that sense. The observations show that while there are described solutions to problems in system, it's difficult for users to find it. It's somehow organized or misorganized that solution is not intuitive for them (e.g. someone migrating from Windows complained that it was difficult to find 'simply how to format a flash drive' in Handbook). I don't know how to handle this properly. May be catalogues with Best Practices howto's pointing to exact sections of Handbook, FAQ, articles etc.? May be better search mechanisms?.. And the last here about too much conservatism and rigidness in our camp. As one of the opponents said (roughly translated): | FreeBSD, historically, was good system, let's say, "valid system | for solid people". Pureness, strictness, etc. The base system is | consequence of this approach. The trouble is, that in 2011 nobody | needs pureness and strictness. Customers need transparency and fast | reaction to their requests. If there is no transparency and speed, | you get CentOS 6 (just released, at last). Or FreeBSD (Gentoo, | Solaris, you name it). I don't agree that those certainly contradicts each other, but we definitely need to change, er, something. And don't keep something just because it is tradition, if that tradition has no more technical reasons for our users. Let's shift conservatism to another field where it is really needed (e.g. legacy support and other examples below). Finally, two more quotes from the arch@freebsd.org archives of last 2 years: | From: Cyrille Lefevre | [...] however, you answer remember me why I quit FreeBSD, cease | fire, courtesy is the key and | From: Julian H. Stacey | Though I & many others have sent lots of send-pr, contributing even | a spelling correction to FreeBSD is much harder than to e.g. | http://wikipedia.org The threshold of entering to FreeBSD is too high and should be lowered. I can confirm the last quote on my own example, 2 months ago. I've submitted a patch to ipfw, adding call/return rule actions. It allows to organize rules in somewhat like pf anchors, and, more importantly, iptables chains, enabling FreeBSD ipfw to compete with Linux at least partially in this sphere. The patch wasn't taken in maillists, I had to push it in IRC, and push hard: the whole needed by many big users thing was deferred (and still not MFCed to 8.x) just because some #define's and printf's were Not So Proper & Right Way! F*cking shame. > 2. Lack of drivers and poor guest virtualization. It is known that FreeBSD supports less hardware than competitors. It is, however, a matter of popularity - the more system will have users, the more drivers will be created by 3rd parties, so we cannot directly affect this. It's kind of exclusive circle. But there is one area - virtualization - without supporting it the FreeBSD niche will collapse very fast. It is necessary to say that it is about guest (para-)virtualization only, not host mode. Host mode market is already lost for FreeBSD for quite some time, may be something in the future, but today it will waste of resources. What is needed here is just analogue to OpenVZ (and jails/rctl already done more than half of the work here). And in guest mode FreeBSD *urgently needs* working drivers/utilities for all common suits, Citrix XenServer, VMWare vSphere (ESXi), Xen*, Microsoft Hyper-V, etc. This must be the primary direction for developers' forces and FreeBSD Foundation's money. We have may be about 1 year here. Why? Because of cloud computing and VPS/VDS. It's no matter what OS runs hoster if clients will use FreeBSD and we still have users. I've been already asked by a customer for which I wrote a non-portable kqueue-based daemon to rewrite for Linux because they want to go for Amazon EC2 (it's cheaper as only used resources are accounted) and FreeBSD there has still many in ISSUES scaring them... > 3. Ports and packages. What was the main problems with large-scale installations of FreeBSD in that businesses? In short, that binary packages are not equal in rights to ports, and that complicates things (i.e. requires too much work) when one have many (> 10) servers. This was listed to me as: 1) No pkg and pkg-devel versions. The -devel version is headers, static libs, programmer examples, etc. not needed in production (we could say this part is what is actually depended on in B-deps). 2) Package name is dependent on options, so packages with another opts don't work well when dependencies are rebuilt. 3) Conflicts: no way to have apache13 and apache22 the same time. 4) No dependence on base system. You may cut out something, recompile world, deploy it on cluster and just then see that some packages are now don't work. 5) Dependencies are badly designed. No version ranges in dependencies, no alternative packages, no priorities in package search. 6) Update problems. The version is just coded into name of package, and dependencies are on the entire name, so there are situations when install/upgrade of just one package may require rebuild 3/4 of all pkgs. You cannot easiy modify installed package without editing pkgdb manually. It is impossible to upgrade/replace package by out of the box tools. 7) Base system has no "out of the box" tools for package upgrade. Our business opponents say this the least problem as one can always install portupgrade, but conclude that overall base system concept does not play well with full-featured packages (see also next part about base system). 8) There is no -STABLE supported branches in ports. All of this could be avoided (they know about tinderbox etc.) but just requires too much work, for their basic tasks like automated upgrade of entire system & packages or reinstall of needed packages. That's problems. Next, possible (gathered) solutions. It is obvious that current packages are not first-class citizens, in comparison with ports. They want ba able to run most machines without a compiling at all (BTW, our desktop users need the same), but setting build farms when there are many machine roles is hard. So packages need to be "equal in rights" in ports. The ports can have things like this: .if ${OSVERSION} < 700104 || ${OSVERSION} >= 900000 or this: LIB_DEPENDS+= profiler.1:${PORTSDIR}/devel/google-perftools but packages are not so flexible, all you have is: @pkgdep perl-5.8.9_2 skv@ proposed the following changes: * OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL, SQLite) and dialog(1) supports it. * Options must be included and installed to /var/db/ports/*/options (this will allow to rebuild installed binary pkg as port) * Info about options must be included to /var/db/pkg/*/+CONTENTS like: @option WITH_SSL @option WITHOUT_DEBUG * Dependencies must be able to specify needed OPTIONS, both required to set and required to be unset, somewaht like: RUN_DEPENDS+= foobar:${PORTSDIR}/devel/foobar:+SSL:-SQLITE This will allow to detect conflicts with installed packages with incompatible options. * For the package file names, introduce presets, e.g.: OPTIONS_PRESETS= default "+SSL:-DEBUG" \ lite "-SSL:-DEBUG" And preset name could be put to pkg name (may be "" for default). * (internal) move away from CVS, rebalance to category-subcategory. These ideas in later discussion evolved to another additions. Let say we are able to use multiple repositories, where "repository" is a variant of /usr/ports tree and packages built from it. Then, each port allows to build several packages from it, with different options. Now, if we have a port called "softina" and user does pkg_add -r softina then dependency search must be made given needed options. So @pkgdep consists just of "softina" and versions like ">=1.1" and options. Also, if packages are equal in rights to ports, they need integrity/security check. So, package file name is now like: softina-1.2.3_1:repo.id:1312929890.tbz # chars allowed by windows Here is a repository id, just a hostname like "freebsd.org" for official ports, and an unique build id in that repo, though it were suggested that option preset name instead of that name will be better (because human-readable). And `pkg_add -r' fetches a single file softina.pkgmeta consisting of sections for each actual package file. Each section has a copy of what already is in .tbz file: name, comment, options, dependencies (and their versions and options). And one thing not present in .tbz - it's digital signature. Fetching pkg_add looks up local key for given repository id to check. Fetching .pkgmeta beforehand also allows to calculate if all needed dependencies are present in repo to not fail in the middle of 100 packages as current pkg_add may do. The signature is in another file and optional, so user could install the plain .tbz file manually (it still contains all needed information, .pkgmeta is only a copy except a signature). The one other thing which is also optional to @pkgdep is again repository id. This is to allow following situation: * company has 10 it's own internal projects * it also has 20 modified ports from original tree * internal pkgs depend on modified ports, not original * it's local port tree/pkg repo may hand only those, not full tree Then internal pkgs may be depended on modified ports to be not intermixed by mistake with original versions from official tree. Repository IDs are used for that, this is optional mechanism, though. End machines have two repositories in their config files, local and offical, and all works correct, and local repo doesn't need to be a full clone of ports tree. The problem of -devel ports could be solved by using a global knob like WITH_DEVEL (analogue of WITHOUT_X11), and options affect parts of pkg-plist. The solved problem: glib has perl in R-deps, just for one script, but this script is needed only for _build_ of dependent ports. Now, if you install irssi, you will always have glib and perl, but you don't need perl to run irssi. And irssi could have it's build dependence of glib:+DEVEL and run of just plain glib. Of course we don't want to split ports to perl, perl-base, perl-modules, perl-doc, etc. like in Debian, and options could be solution here - one port, several packages. Build farm may now build not one, but 2-3 common option presets. Here we go to another related problem, though - ports infrastructure. I've read 16 chapters of http://www.netbsd.org/docs/pkgsrc/ and found several interesting moments (let's concentrate on most close sibling of our ports, though e.g. slots in Gentoo and Arch Linux system are also worth looking too). They already have many of what we need, and since pkg_install/ports by Jordan Hubbard is common ancestor, it may be easier to port from them to us needed features. For example, there is problem generating plist for our porters. And what they have, a program to create port from distfile: | Run the program url2pkg, which will ask you for a URL. Enter | the URL of the distribution file (in most cases a .tar.gz file) and | watch how the basic ingredients of your package are created | automatically. Just fix oddities later manually, but saves from all boilerplate work! Next, skv@ said about radio-buttons, and thay also have it, in two favors: first with one from group always set, second allowin all clear: "Exactly one of the following gecko options is required" "At most one of the following database options may be selected" | PKG_OPTIONS_REQUIRED_GROUPS is like PKG_OPTIONS_OPTIONAL_GROUPS, but | building the packages will fail if no option from the group is | selected. PKG_OPTIONS_NONEMPTY_SETS is a list of names of sets of | options. At least one option from each set must be selected. The OPTIONS, however, are handled in different way: $ grep "PKG.*OPTION" mk.conf PKG_DEFAULT_OPTIONS= -arts -dvdread -esound PKG_OPTIONS.kdebase= debug -sasl PKG_OPTIONS.apache= suexec Dependencies also allow versions: BUILD_DEPENDS+= lua>=5.0:../../lang/lua DEPENDS+= screen-[0-9]*:../../misc/screen DEPENDS+= screen>=4.0:../../misc/screen Checksum files for packages exist on their own, and only those may be signed not the packages itself (an alternative to .pkgmeta approach described above). Another useful thing to borrow is patches/* separately from files/* The most visible thing in pkgsrc superior to ports, is buildlink3 framework: | Buildlink is a framework in pkgsrc that controls what headers and | libraries are seen by a package's configure and build processes. | [...] Please note that the normal system header and library paths, | e.g. /usr/include, /usr/lib, etc., are always searched -- buildlink3 | is designed to insulate the package build from non-system-supplied | software. | [...] | Some packages in pkgsrc install headers and libraries that coincide | with headers and libraries present in the base system. Aside from | a buildlink3.mk file, these packages should also include a | builtin.mk file that includes the necessary checks to decide whether | using the built-in software or the pkgsrc software is appropriate. | [...] When building packages, it's possible to choose whether to set | a global preference for using either the built-in (native) version | or the pkgsrc version of software to satisfy a dependency. This is | controlled by setting PREFER_PKGSRC and PREFER_NATIVE. | PREFER_PKGSRC= yes | PREFER_NATIVE= getopt skey tcp_wrappers There are more automation: | Up to now, the file PLIST, which contains a list of the files that | are installed by the package, is nearly empty. Run | bmake print-PLIST >PLIST | to generate a probably correct list. Yes, they rely on their derivative of BSD make, bmake, which it itself worth merging deifferencies to our make (for example, there is a clean and small alternative to autotools, mk-configure, using bmake and written in style of .include ). There may be other examples of userland tools worth porting from NetBSD to us instead of directly upstream, e.g. awk... but that's another story, let's continue pkg. Their developments of pkg_* tools contain facilities to implement a good package manager out-of-the-box, e.g. pkg_add there has flags: | -A Mark package as installed automatically, as dependency of | another package. You can use | pkg_admin set automatic=YES | to mark packages this way after installation, and | pkg_admin unset automatic | to remove the mark. If you pkg_add a package without | specifying -A after it had already been automatically | installed, the mark is removed. | -U Replace an already installed version from a package. | Implies -u. | -u If the package that's being installed is already installed, | an update is performed. These are crucial to effective binary package management. > 4. Base system, closely related with packages. The next in turn was concept of monolithic base system. The troubles with base system were: 1. To change the components of the base you may only recompile it with corresponding src.conf(5) options, but there is no track in base system which are actually installed now. You cannot binary upgrade a custom world, even if it is just unmodified -STABLE instead of -RELEASE. 2. Consequently, there is no way to check integrity (MD5 etc.) of any non-RELEASE variant (freebsd-update IDS is very limited). 3. No ties between base system and packages: who knows what previous admin has installed, you may have compiler or may not have, etc. Packages may silently broke if some part of base system SUDDENLY disappears, as no dependency information is recorded. 4. Base system is monolithic, so there is no easy way to replace one component with another - ports replacing base parts are hemorrhoids. So, they conclude, the base system concept should be eliminated and be split to bare kernel and gazillion of packages. But we all know what benefits base system gives to us: it is actually a *system* where all components are in concordance to each other, not just a bunch of packages. Also, if it will be all split, this will require *much more* efforts from our committers to keep everything in sync. And our resources (efforts) are limited. Actually, when you face two contradictory ways, all-or-nothing, both with pros and cons - you should select neither of them, but try to find something which will *synthesize* both of them. Such finding is an interesting engineering (often inventor) task. And dougb@ already said once that between base as-is and splitting all must be something middle. It's Cathedral vs Bazaar, after all, and our Cathedral should be updated and still use it's cathedral benefits. After all, NetBSD has proposal of syspkg in 2004, and still has older format - just because it is not BSD way, something new has to be invented. The most obvious solution is to split base not to ~500 packages, but to, say, ~50, and change boundaries. Why should freebsd.org developers place boundaries in packages between themselves? So most natural solution is to split packages out of base by the vendor criteria. Luckily, this work is *already done* to some extent. So this will be minimal efforts overhead, if any. Looking at /usr/src/sontrib and http://wiki.freebsd.org/ContribSoftware we can identify many of what could became a package. There can be different approaches to criteria "what is in base system": 1. Only what is done on freebsd.org: all contrib must go to pkgs. 2. Whose effective vendor is now *@*bsd.org: contrib from other BSDs may still live, and those with ceased upstream or renamed (non-conflicting with ports) soft like libbsdxml, too. 3. Axe out only the most odious contrib parts: BIND (as Peter said in archives, host/dig could be resurrected from Attic), sendmail (could be replaced by dma) and several others, presumably GCC/binutils & CVS (I've also heard about painful Kereberos interferencing with Samba). The latter is most probable :-) given known conservatism and parts inter-dependencies (openssl is required by freebsd-update, and it's not *@*bsd.org - already an exception). But it still needs clear definition of what base system is destined for. A philosophical observation. The more "fat" base system is, the more things are supposed about end user (the "WHEREs" in "SELECT"), so the more narrow target audience is. The more thin base system is, the more popular OS can be. You elitists may sleep well: FreeBSD will never be as popular of Linux whose "rod" is just kernel, not base sys. So each component of the base must be justified for the task, and for task we may be should define to which user FreeBSD is targeted. Let it be, say, task "the fresh setuped machine must be able to connect to network to download and install binary packages". Then you don't need GCC but you need editor to edit resolv.conf, less to view man pages, tcpdump to debug network problems, send-pr to report a bug about non-installed packages and thus a mail agent able to handle outgoing report mail and daily periodic scripts to root while you have sex with this (dma is suitable, sendmail is overkill). And so on. Axing out GCC to packages has another benefit: the newer GCC could be used, and base could be polished to be more compiler-agnostic (hello, clang). But this requires binary packages to have equal rights with ports, of course. A base without a compiler is not a dramatic - you already need some ports to do "make release", and doc team already began to split docs to packages, that's a right direction. For POLA reasons all axed out packages (and sendmail too, respect traditions) should be just packaged on CD1. There may be another approach, not package one, but it is still in a fog (and don't solve ports overriding base well). It's like the kernel and modules: monolithic base system is just as monolithic kernel before invention of modules - you the same way don't know what's in. Let's dream a little - imagine we have our own DVCS, combining benefits of centralized and distributed one, and not requiring splitting to multiple repos like git, but allowing to use any given subset of files (not dirs and subtrees as svn, *any* "inode"-based subset) from central repo (this one "more equal" from all equal distributed repos). Such system could be placed instead freebsd-update, binary updates of STABLE etc. go to special branch, and user updates only those files which he have on his system, VCS automatically tracks it. > 5. Too short major releases' cycle. I've once read a thread: From: mdf@FreeBSD.org Subject: Schedule for releases Date: Tue, 21 Dec 2010 09:47:08 -0800 where e.g. julian@ have said: | Generally a company wouldn't want to go through the pain of an OS | upgrade more than about once in three years and often it's longer.. | It IS a pain for them. And many business people reported the same. And it is known many people are doing backports, testing etc., they should be motivated to give work back to FreeBSD. While it is more social problem, it is also a DVCS/bugtracker problem, but more rare major releases would still help on it's own. It is known why the current scheme was adopted: the 4.11/5.3 case, a horror. But between X.4 and X.11 there are even *several* intermediate choices. What happens today: many conservative users (or builders of products) consider -STABLE is really stable at the X.2 release. But just after than an (X+1).0 is forked and all developers' attention goes to new branch, X.3 and X.4 are not seriously developed. And due to timelines described above, many users will then upgrade right away to (X+2).Y, not (X+1). There are many 6.x and 7.x users in the wild, many of 7.x users will upgrade directly to 9.x skipping 8.x. Is this good? No. Aside of many branches receive not enought production testing, our committers must do MFCs to TWO branches, stable/7 and stable/8. While usualy having no real possibility to test properly. Nonsense. Is supporting two stable branches not using more efforts that it was in 4.x times? Not so? Still could be made better. Proposed solution: prolonging major releases fork time a little. Just to time so only one stable branch will exist. I hope increasing branch lifetime to one minor release will help: last will be not .4, but .5, and new .0 fork should occur at least after .3, not .2. We are not Ubuntu. I understand that pace of changes is high, 3 years for each .0 may be unacceptable, but waht about at least 2.5 years? Not like 1.8 years currently. Rememeber, .5 and even .6 is still far from .11. > 6. Bug tracker, unicode and other less important trivia. GNATS is too old and unfriendly e.g. to user attachments. It has only one adavantage: user is able to send report from CLI by mail without any registration. This is essential. May be the good alternative will be RT used at cpan.org (it also accepts by mail). Hopefully this will help a little to solving problems. Not sure, though - and unsolved bugs are also make users to go away from FreeBSD. Users also want properly working UTF-8 out-of-the-box. Not only console, but full collation support, etc. There was suggestion about automatic kldloading of drivers, but this work already began (for USB) in June - more matte of our documentation and press relations, huh. Installer. Must be more featureful and more user friendly :-) This is often may give negative first impression if installer was unable to do something even if system after this could work well. Other problems, like broken cvsup, are exist, but are not critical to Project's survival. > FreeBSD strengths to be concentrated on. The paragraph about virtualization above, as long as points below, are roughly translated words of one small business representative in Russia, who often acts as integrator. 1. BSD License. Very good for embedded vendors, we should be able to compile and work both base and ports without licenses requiring to open the sources. It would be good to not loose functionality without them. 2. Kernel features for storage (ZFS, HAST, GEOM). We need to go to NAS/SAN solutions niche while we have ability - it is still, because Solaris etc. in unknown state with Oracle, BTRFS still beta. We have about 1 year max. It would be nice to roll out "box" solution for a failover iSCSI storage (2 PCs with ZFS raids replicated by HAST and accessible by iSCSI with automatic role change). 3. Kernel features for complex network solutions (netgraph, carp, ipfw). The niche for routers & traffic analysis is still ours. It would be nice to take e.g. pfSense and agree with some vendor (Netgear, D-Link, etc) to put on sale hardware with FreeBSD inside. So these are main ways - embedded (NAS & routers). Other market is still not our, e.g. not an application server until there will be a killer-app for SCTP. May be also for a DB: Solaris was first recommended for PostgreSQL, with some efforts we may become the first. Of course, the main way - if some commercial company will offer FreeBSD. We still have some time, but almost no time. We need to take decisions right now. That's all for today. Thanks to everyone who has patience to carefully read this all entirely. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 08:07:25 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E488106564A for ; Thu, 18 Aug 2011 08:07:25 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from xiurhn.etoilebsd.net (xiurhn.etoilebsd.net [94.23.37.58]) by mx1.freebsd.org (Postfix) with ESMTP id F3B938FC1A for ; Thu, 18 Aug 2011 08:07:24 +0000 (UTC) Received: by xiurhn.etoilebsd.net (Postfix, from userid 80) id B50587E832; Thu, 18 Aug 2011 09:49:30 +0200 (CEST) To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 18 Aug 2011 07:49:30 +0000 From: Baptiste Daroussin In-Reply-To: References: Message-ID: X-Sender: bapt@FreeBSD.org User-Agent: Roundcube Webmail/0.5.3 Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 08:07:25 -0000 Hi, My reply only concerns package/ports management. Most of what you are expecting from the ports tree is coming soon (new option framework) and most of what you are expecting from binary packages will be done with pkgng. Just have a look https://github.com/pkgng/pkgng for pkgng it is still experimental but works quite well :). > >> 3. Ports and packages. > > What was the main problems with large-scale installations of FreeBSD > in > that businesses? In short, that binary packages are not equal in > rights > to ports, and that complicates things (i.e. requires too much work) > when > one have many (> 10) servers. This was listed to me as: > > 2) Package name is dependent on options, so packages with another > opts > don't work well when dependencies are rebuilt. They will with pkgng. > > 3) Conflicts: no way to have apache13 and apache22 the same time. > This is not a problem of the infrastructure nor of the package tools, this has to deal with the said ports. I'm pretty sure if you come with a path to solve this (for example avoid file conflicts) > 4) No dependence on base system. You may cut out something, recompile > world, deploy it on cluster and just then see that some packages > are > now don't work. > > 6) Update problems. The version is just coded into name of package, > and > dependencies are on the entire name, so there are situations when > install/upgrade of just one package may require rebuild 3/4 of all > pkgs. You cannot easiy modify installed package without editing > pkgdb > manually. It is impossible to upgrade/replace package by out of > the > box tools. > This will be solve with pkgng > 8) There is no -STABLE supported branches in ports. > > All of this could be avoided (they know about tinderbox etc.) but > just > requires too much work, for their basic tasks like automated upgrade > of > entire system & packages or reinstall of needed packages. > > That's problems. Next, possible (gathered) solutions. > > It is obvious that current packages are not first-class citizens, in > comparison with ports. They want ba able to run most machines without > a compiling at all (BTW, our desktop users need the same), but > setting > build farms when there are many machine roles is hard. > > So packages need to be "equal in rights" in ports. The ports can have > things like this: > > .if ${OSVERSION} < 700104 || ${OSVERSION} >= 900000 > > or this: > > LIB_DEPENDS+= profiler.1:${PORTSDIR}/devel/google-perftools > > but packages are not so flexible, all you have is: > > @pkgdep perl-5.8.9_2 > > skv@ proposed the following changes: > > * OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL, > SQLite) and dialog(1) supports it. > This is coming soon to the ports tree, time to do more testings > * Options must be included and installed to /var/db/ports/*/options > (this will allow to rebuild installed binary pkg as port) > This could be a good idea :) I'll keep it for pkgng > * Info about options must be included to /var/db/pkg/*/+CONTENTS > like: > > @option WITH_SSL > @option WITHOUT_DEBUG > There is an equivalent of this in pkgng. > * (internal) move away from CVS, rebalance to category-subcategory. > > These ideas in later discussion evolved to another additions. Let say > we > are able to use multiple repositories, where "repository" is a > variant > of /usr/ports tree and packages built from it. Then, each port allows > to > build several packages from it, with different options. Now, if we > have > a port called "softina" and user does > > pkg_add -r softina > > then dependency search must be made given needed options. So @pkgdep > consists just of "softina" and versions like ">=1.1" and options. > Also, > if packages are equal in rights to ports, they need > integrity/security > check. So, package file name is now like: > > softina-1.2.3_1:repo.id:1312929890.tbz # chars allowed by > windows > pkgng have an experimental multi repository support, even if we focus on a clean single repository support. > Next, skv@ said about radio-buttons, and thay also have it, in two > favors: first with one from group always set, second allowin all > clear: > "Exactly one of the following gecko options is required" > "At most one of the following database options may be selected" > > | PKG_OPTIONS_REQUIRED_GROUPS is like PKG_OPTIONS_OPTIONAL_GROUPS, > but > | building the packages will fail if no option from the group is > | selected. PKG_OPTIONS_NONEMPTY_SETS is a list of names of sets of > | options. At least one option from each set must be selected. > > The OPTIONS, however, are handled in different way: > > $ grep "PKG.*OPTION" mk.conf > PKG_DEFAULT_OPTIONS= -arts -dvdread -esound > PKG_OPTIONS.kdebase= debug -sasl > PKG_OPTIONS.apache= suexec > This is coming along with the radio options. > > Their developments of pkg_* tools contain facilities to implement a > good > package manager out-of-the-box, e.g. pkg_add there has flags: > > | -A Mark package as installed automatically, as dependency of > | another package. You can use > | pkg_admin set automatic=YES > | to mark packages this way after installation, and > | pkg_admin unset automatic > | to remove the mark. If you pkg_add a package without > | specifying -A after it had already been automatically > | installed, the mark is removed. > | -U Replace an already installed version from a package. > | Implies -u. > | -u If the package that's being installed is already > installed, > | an update is performed. > > These are crucial to effective binary package management. > All of this can be done a better way with pkgng regars, Bapt From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 15:42:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8894F1065673 for ; Thu, 18 Aug 2011 15:42:16 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 47D178FC08 for ; Thu, 18 Aug 2011 15:42:16 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id DC60A46B2A; Thu, 18 Aug 2011 11:42:15 -0400 (EDT) Date: Thu, 18 Aug 2011 16:42:15 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Vadim Goncharov In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 15:42:16 -0000 On Wed, 17 Aug 2011, Vadim Goncharov wrote: >> 2. Lack of drivers and poor guest virtualization. > > It is known that FreeBSD supports less hardware than competitors. It is, > however, a matter of popularity - the more system will have users, the more > drivers will be created by 3rd parties, so we cannot directly affect this. > It's kind of exclusive circle. > > But there is one area - virtualization - without supporting it the FreeBSD > niche will collapse very fast. It is necessary to say that it is about guest > (para-)virtualization only, not host mode. Host mode market is already lost > for FreeBSD for quite some time, may be something in the future, but today > it will waste of resources. What is needed here is just analogue to OpenVZ > (and jails/rctl already done more than half of the work here). > > And in guest mode FreeBSD *urgently needs* working drivers/utilities for all > common suits, Citrix XenServer, VMWare vSphere (ESXi), Xen*, Microsoft > Hyper-V, etc. This must be the primary direction for developers' forces and > FreeBSD Foundation's money. We have may be about 1 year here. Why? Because > of cloud computing and VPS/VDS. It's no matter what OS runs hoster if > clients will use FreeBSD and we still have users. I've been already asked by > a customer for which I wrote a non-portable kqueue-based daemon to rewrite > for Linux because they want to go for Amazon EC2 (it's cheaper as only used > resources are accounted) and FreeBSD there has still many in ISSUES scaring > them... Thanks for your post -- it has made a very interesting read! I'd also agree that failurely to rapidly adopt full-machine virtualisation (especially in light of our early jail success in the OS virtualisation space) has been a big problem. However, I think it's worth observing why we're now improving that support even though it took a while to do it: as embedded "grew up" into the MMU-capable CPU space, increasing numbers of FreeBSD developers were employed by embedded and appliance companies. It's only as those companies begin to realise that they want virtualisation in their products that we're seeing dramatic improvements in Xen support, the arrival of bHyve, etc. This is unfortunate in two senses: first of all, it would have been great to provide those features that our ISP users really wanted, and second, the features would have been much more mature by the time the embedded/appliance space got to them. I think we now are running in the right direction but could use a more focused effort targeted more at EC2 and the ISP space to supplement the appliance world. (Case in point: for appliances, limiting Xen support to guest HVM with amd64, but with a nice set of both front and back drivers, meets most needs. EC2 requires full PV, which we don't support on amd64, but not back drivers.) One potential direction to think about here is improved distributed system support -- integrated OpenAFS, easy-to-deploy Kerberos, a distributed shared memory scheme of some sort, large-scale management tools, improved support for caching distributed credentials, and easy-to-install distributed computation frameworks. Provide EC2 images that make it really easy to manage a large number -- not just because EC2 makes it easy to clone VMs, but also by providing OS-side management tools that DTRT. Robert From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 18:37:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AAE1106566B for ; Thu, 18 Aug 2011 18:37:51 +0000 (UTC) (envelope-from gelraen.ua@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id C45A68FC16 for ; Thu, 18 Aug 2011 18:37:50 +0000 (UTC) Received: by fxe4 with SMTP id 4so1966823fxe.13 for ; Thu, 18 Aug 2011 11:37:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:in-reply-to:mime-version:message-id :content-type:content-transfer-encoding; bh=8vZbG27+DFWJ5rrlxg2ri41Yk8qiw3pzFusigN/qPu4=; b=MNFajTOyOQQeW4f8Nt6ngg63AGpyUAEzvRxo7Vi4lf9qy55CKAAs2yuwfM9m7B6Wpw BIKx0Cj0Jzv4C9J10xZUXCSADpVJUAA4eG9hMCfasrwXhFTcJh2L6OjhCxwxXeoAG88Q PvlPixIb4DHTcq8bcrGsJ33RhADdA9+3xFErs= Received: by 10.223.159.2 with SMTP id h2mr159774fax.5.1313691014561; Thu, 18 Aug 2011 11:10:14 -0700 (PDT) Received: from imax.localnet (187-94-133-95.pool.ukrtel.net [95.133.94.187]) by mx.google.com with ESMTPS id 6sm1924083fas.2.2011.08.18.11.10.11 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Aug 2011 11:10:12 -0700 (PDT) From: Maxim Ignatenko To: freebsd-arch@freebsd.org Date: Thu, 18 Aug 2011 21:10:03 +0300 User-Agent: KMail/1.13.7 (FreeBSD/8.2-STABLE; KDE/4.6.5; i386; ; ) In-Reply-To: dae4b29c363977d83e857bed8e253ae8@etoilebsd.net MIME-Version: 1.0 Message-Id: <201108182110.03525.gelraen.ua@gmail.com> Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 18:37:51 -0000 Hi, pkgng looks great, but I suggest to add in-place upgrade feature. Now packages upgraded with just deleting and installing new version, which is not so fast and requires some "dirty" tricks with config files to correctly delete unmodified configs on pkg_delete and keep modified. Instead, we can simply overwrite only those files, that differs with previous version and even keep any user-modified files without any special tricks in port's Makefile and package metadata. Also, in this case upgrading package between minor versions will generate much less write requests to FS. But this requires to make some changes in ports infrastructure. Each port need to be installed to some temporary location first, and only then changed files should be moved in place. Using tmpfs as temporary location will slightly reduce "make upgrade" time, comparing to "make deinstall install". Another way is to create binary package directly and leave all upgrading logic in pkgng. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 19:50:17 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14F32106564A for ; Thu, 18 Aug 2011 19:50:17 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id CFEB58FC14 for ; Thu, 18 Aug 2011 19:50:16 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 213124AC58; Thu, 18 Aug 2011 23:50:15 +0400 (MSD) Date: Thu, 18 Aug 2011 23:50:11 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1144162985.20110818235011@serebryakov.spb.ru> To: Vadim Goncharov In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 19:50:17 -0000 Hello, Vadim. You wrote 18 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 3:10:19: > 8) There is no -STABLE supported branches in ports. I want to be more precise here: not -STABLE, but all -RELEASE branches, where "upstream" version of ports/packages never changes, and only security bugfixes are backported. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 21:24:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52692106564A for ; Thu, 18 Aug 2011 21:24:27 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id E4C088FC14 for ; Thu, 18 Aug 2011 21:24:26 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 3F7434AC1C; Fri, 19 Aug 2011 01:24:25 +0400 (MSD) Date: Fri, 19 Aug 2011 01:24:21 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <705869186.20110819012421@serebryakov.spb.ru> To: Vadim Goncharov In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 21:24:27 -0000 Hello, Vadim. You wrote 18 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 3:10:19: > The other social problem is lack of companies which offer commercial > support of FreeBSD like RedHat does. Main social problem, IMHO, that there WAS NOT (I forgot Linux history and don't rememberfirst uf-distributive) and, later, Ubuntu-like versions of FeeBSD. Even if these doen;t replace Windows on many desktops (1% now?), they prepare Linux-aware users, and some of these users becomes admins or people, who decide which OS should be used in their business. And, I think, it is too late. Why somebody should now choose FreeBSD when here is fancy Linux with bells and whistles? :( Yes, I'm pessimistic :( I don't say, that people need all these B&W on servers, for example. No. It works like this: user choose to try something fancy and trendy, and even if he don't start to use it now, after evaluation, he'll return to this system later, if he need to choose something for real task. Of course, system should be suitable for this task. But almost everything is ``suitable'' for common tasks. And it is NOT ENOUGH to be technically better. System should be far more superior to be chosen, if it is not fancy/trendy. Yes, I belive, that FreeBSD is better than Linux (at least on supported hardware) in server tasks, more clear, more solid, etc. But it is ``only'' better, and is not enough. Other factors are hardware certification and hosting providers. And, yes, commercial software. I mean Oracle and (not-so-commercial but very important) Java :) BTW, I belive that Solaris is better than FreeBSD and much, much better that Linux for many server tasks. Now Solaris future is unclear, but before Sun/Oracle acquisition it looks TECHNICALLY very good. But it was not fancy... Ooops... And, even more, I've worked with "small" Sun's servers (like SunFire X4xxx), and with Supermicro-based servers and with Dell servers in same class. Sun's was TECHNICALLY much better, and cost was almost the same. But many of my friends buy Dell or Supermicro for their businesses. Why?! Because ``Sun makes very expensive stuff for very big companies''. And it was Sun, with all marketing money, etc! I don't think, that last paragraph is off-topic -- it is example of system with exactly same non-techincal problems. And even best-in-class or best-in-world package management system and streamlined base system DON'T SOLVE non-technical problems. They could help don't lose current users, but they can not help find new ones! --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 21:45:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 059E6106564A for ; Thu, 18 Aug 2011 21:45:38 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 886158FC08 for ; Thu, 18 Aug 2011 21:45:37 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QuAP9-0004PP-J7 for freebsd-arch@freebsd.org; Thu, 18 Aug 2011 23:45:35 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 23:45:35 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 23:45:35 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 18 Aug 2011 21:45:23 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 87 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Baptiste Daroussin User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 21:45:38 -0000 Hi Baptiste Daroussin! On Thu, 18 Aug 2011 07:49:30 +0000; Baptiste Daroussin wrote about 'Re: FreeBSD problems and preliminary ways to solve': > My reply only concerns package/ports management. > Most of what you are expecting from the ports tree is coming soon (new > option framework) and most of what you are expecting from binary > packages will be done with pkgng. Good news! > Just have a look https://github.com/pkgng/pkgng for pkgng it is still > experimental but works quite well :). I've looked to wiki.freebsd.org before that and there was almost empty page. An illustration of another social problem - more should be known to the user in the first place... >> 3) Conflicts: no way to have apache13 and apache22 the same time. > This is not a problem of the infrastructure nor of the package tools, > this has to deal with the said ports. > I'm pretty sure if you come with a path to solve this (for example > avoid file conflicts) Separate PREFIX? How is it solved on another systems? >> 4) No dependence on base system. You may cut out something, recompile >> world, deploy it on cluster and just then see that some packages are >> now don't work. >> >> 6) Update problems. The version is just coded into name of package, and >> dependencies are on the entire name, so there are situations when >> install/upgrade of just one package may require rebuild 3/4 of all >> pkgs. You cannot easiy modify installed package without editing pkgdb >> manually. It is impossible to upgrade/replace package by out of the >> box tools. >> > This will be solve with pkgng And base system too? I've forgotten to write another idea in previous letter: the kernel nowadays has FEATURE() macros which application could check, so for packages something may be deployed, e.g. @basefeature GCC etc. Same with $OSVERSION. >> * OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL, >> SQLite) and dialog(1) supports it. >> > This is coming soon to the ports tree, time to do more testings The both versions "exactly one must be set" & "zero or one must be set" ? >> Also, if packages are equal in rights to ports, they need >> integrity/security check. So, package file name is now like: >> >> softina-1.2.3_1:repo.id:1312929890.tbz # chars allowed by windows > pkgng have an experimental multi repository support, even if we focus > on a clean single repository support. Are there digital signatures for them in TODO ? :) >> Their developments of pkg_* tools contain facilities to implement a >> good package manager out-of-the-box, e.g. pkg_add there has flags: >> >> | -A Mark package as installed automatically, as dependency of >> | another package. You can use >> | pkg_admin set automatic=YES >> | to mark packages this way after installation, and >> | pkg_admin unset automatic >> | to remove the mark. If you pkg_add a package without >> | specifying -A after it had already been automatically >> | installed, the mark is removed. >> | -U Replace an already installed version from a package. >> | Implies -u. >> | -u If the package that's being installed is already installed, >> | an update is performed. >> >> These are crucial to effective binary package management. >> > All of this can be done a better way with pkgng -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 21:58:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA260106566B; Thu, 18 Aug 2011 21:58:49 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe04.c2i.net [212.247.154.98]) by mx1.freebsd.org (Postfix) with ESMTP id 373C08FC13; Thu, 18 Aug 2011 21:58:48 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=Ki88u+3nwejNRg6MOseuLmL2vomBRNHvHPpCRMZep5s= c=1 sm=1 a=SvYTsOw2Z4kA:10 a=Dyoqhi_TatcA:10 a=GIYlNEbclosA:10 a=WQU8e4WWZSUA:10 a=Cfj4BQAnxiAA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=wX8SDCiys2fo2iJbRZ8A:9 a=Ft8UYL4EG9YA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe04.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 167494647; Thu, 18 Aug 2011 23:48:45 +0200 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Thu, 18 Aug 2011 23:46:19 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <705869186.20110819012421@serebryakov.spb.ru> In-Reply-To: <705869186.20110819012421@serebryakov.spb.ru> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq,NwSZ4V" =?iso-8859-1?q?=7CLR=2E+tj=7Dg5=0A=09=25V?=,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( =?iso-8859-1?q?=0A=09=3AAuzV9=3A=2EhESm-x4h240C=609=3Dw?= MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1251" Content-Transfer-Encoding: 7bit Message-Id: <201108182346.19470.hselasky@c2i.net> Cc: Vadim Goncharov , Lev Serebryakov Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 21:58:49 -0000 On Thursday 18 August 2011 23:24:21 Lev Serebryakov wrote: > And, I think, it is too late. Why somebody should now choose FreeBSD > when here is fancy Linux with bells and whistles? :( Yes, I'm > pessimistic :( Hi, I think, that we now have a chance to make a better bells and whistles solution than in Linux. I personally have big plans to enhance the multimedia framework under FreeBSD, like /dev/dsp, /dev/midi, /dev/video0, /dev/dvb by using the recently invented cuse4bsd. For example I want to make a system global Audio daemon that can do more advanced audio processing and not at least echo cancelling for audio devices, seamless and transparent to the client applications, using standard /dev/dspX nodes, which eventually will support Bluetooth Audio headsets aswell and 3G modems. The only part which is missing is time and a good sponsor for this project. --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 21:59:55 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EBA1106564A for ; Thu, 18 Aug 2011 21:59:55 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 08FDC8FC0A for ; Thu, 18 Aug 2011 21:59:54 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QuAcz-00026E-P7 for freebsd-arch@freebsd.org; Thu, 18 Aug 2011 23:59:53 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 23:59:53 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Aug 2011 23:59:53 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 18 Aug 2011 21:59:37 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 18 Message-ID: References: <1144162985.20110818235011@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Lev Serebryakov User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 21:59:55 -0000 Hi Lev Serebryakov! On Thu, 18 Aug 2011 23:50:11 +0400; Lev Serebryakov wrote about 'Re: FreeBSD problems and preliminary ways to solve': >> 8) There is no -STABLE supported branches in ports. > I want to be more precise here: not -STABLE, but all -RELEASE > branches, where "upstream" version of ports/packages never changes, > and only security bugfixes are backported. To be even more precise, they need a guarantee that automatic updates will not break anything so that it could be put to cron like "apt-cron". This goal could be satisfied by another means, I hope: FreeBSD developers unlikely to have enough time/efforts to keep it for *all* -RELEASE branches, but for only chosen ones (e.g. extended security support) - may be. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 22:17:41 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 709D51065675 for ; Thu, 18 Aug 2011 22:17:41 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 017838FC16 for ; Thu, 18 Aug 2011 22:17:40 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QuAuB-0000t1-Ab for freebsd-arch@freebsd.org; Fri, 19 Aug 2011 00:17:39 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Aug 2011 00:17:39 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Aug 2011 00:17:39 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 18 Aug 2011 22:17:25 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 57 Message-ID: References: <705869186.20110819012421@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: Lev Serebryakov User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 22:17:41 -0000 Hi Lev Serebryakov! On Fri, 19 Aug 2011 01:24:21 +0400; Lev Serebryakov wrote about 'Re: FreeBSD problems and preliminary ways to solve': >> The other social problem is lack of companies which offer commercial >> support of FreeBSD like RedHat does. > Main social problem, IMHO, that there WAS NOT (I forgot Linux > history and don't rememberfirst uf-distributive) and, later, > Ubuntu-like versions of FeeBSD. Even if these doen;t replace Windows > on many desktops (1% now?), they prepare Linux-aware users, and some > of these users becomes admins or people, who decide which OS should be > used in their business. PC-BSD is exactly for that. > And, I think, it is too late. Why somebody should now choose FreeBSD > when here is fancy Linux with bells and whistles? :( Yes, I'm > pessimistic :( [...] > better that Linux for many server tasks. Now Solaris future is > unclear, but before Sun/Oracle acquisition it looks TECHNICALLY very It's wrong and harmful to be pessimistic these days, as FreeBSD has prospects just exactly because of what happened to Sun. And because Linux /feels itself a winner/ now - and you know what happens with monopolies which stop to develop. But FreeBSD needs to invest efforts (and serious efforts) to use the opportunities, of course. > everything is ``suitable'' for common tasks. And it is NOT ENOUGH > to be technically better. System should be far more superior to be > chosen, if it is not fancy/trendy. Yes, I belive, that FreeBSD is > better than Linux (at least on supported hardware) in server tasks, > more clear, more solid, etc. But it is ``only'' better, and is not > enough. No. The whole message was about that FreeBSD is worse in several areas, which masks out areas where it is better. If that will get fixed, we could talk about is it needing to be "far more" superior or not. But not before that as excuse to do nothing - no such excuse exists. > Other factors are hardware certification and hosting providers. > And, yes, commercial software. I mean Oracle and (not-so-commercial > but very important) Java :) True. > And even best-in-class or best-in-world package management system > and streamlined base system DON'T SOLVE non-technical problems. They > could help don't lose current users, but they can not help find new > ones! Yes, that's the first task - to not lose current users, the second - to dig in our rock solid niche, and only then - to attack and find new ones. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-arch@FreeBSD.ORG Thu Aug 18 23:50:35 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44E571065670 for ; Thu, 18 Aug 2011 23:50:35 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id CD90E8FC0C for ; Thu, 18 Aug 2011 23:50:34 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QuCM4-0000E5-JX for freebsd-arch@freebsd.org; Fri, 19 Aug 2011 01:50:32 +0200 Received: from 208.88.188.90.adsl.tomsknet.ru ([90.188.88.208]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Aug 2011 01:50:32 +0200 Received: from vadim_nuclight by 208.88.188.90.adsl.tomsknet.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Aug 2011 01:50:32 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Vadim Goncharov Date: Thu, 18 Aug 2011 23:50:20 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 142 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 208.88.188.90.adsl.tomsknet.ru X-Comment-To: All User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: FreeBSD problems/solutions: voting system & marketing surveys X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 23:50:35 -0000 Hi! The previous message "FreeBSD problems and preliminary ways to solve" may have been a little long to cover all problems/ideas/solutions on the first reading, but all problems could be essentially tracked down to one. The ultimate problem. > Nobody wants to do more and/or another way than he already does. < That is, it is not known who and how will solve those FreeBSD problems. Everyone thinks he's not "must" and will continue to do "interesting". Passivity. (and FreeBSD, meanwhile, slowly bowls along to grave, as a result) The obvious solution: Foundation hires Someone(TM). There are sponsored projects already. Does it work? Do people donate well for it? No. Why? Because there are no *concrete clear goals and tasks* for the work. Foundation (core@, etc., whoever may be responsible) passively awaits like a girl, who will come and takes. This almost always fails. "Imagine something, we'll pay" vs "Here is the job, let's negotiate a price". Donating people, from the other side, will not give much money for something blurred - where exactly will it go?.. What useful there will be for me (donator)?.. ...If this works bad, isn't it a time to experiment & try another approach? > Working hypothesis: \ people will more willingly donate and support for a project with \ clear goals. Calls for helpers, may be even more broad than \ announce@ subscribers, will succeed more and total donations with \ all such projects will exceed current ones. In other words, Foundation must act as an intermediate party between those who will work and those who will pay for concrete tasks set up by e.g. core@ or by user suggestions/need. Why it was not done earlier? Because many believe it's easy for users to find an implementor without such actions from Foundation, directly, or users don't need something, or it is easy for committer to get sponsored. So why we see hselasky@ saying "The only part which is missing is time and a good sponsor for this project" only here? Who ever heard about it in broad masses? Who ever checked this to be in reality? Who knows what our users think? > How to verify? We need marketing surveys for this. < So in previous message I've proposed a voting system, which could also be used for plain non-weighted surveys. An excerpt from previous msg: === This is a voting system (I could implement it), say, there is a mail voting@freebsd.org to which users send filled in vote forms, with selected answers from a survey published in announce@. The system has a database where users are recorded given their FreeBSD activity in mail lists etc., and votes are summed as follows: + 1.00 vote for anyone not in database (not involved to FreeBSD) + a decimal logarithm of number of posts to mail lists (2 votes for 100 posts, 3 votes for 1000 posts) + a binary logarithm of num PRs in GNATS db (6 votes for 64 send-pr's) + a proportion (say, N/2) of entries for this person in "svn log" (e.g. in "Submitted by:") + an assigned (by core@) number of votes in special exceptional DB (for corporation and the like) The system then presents results for each answer: 1) how many users voted, 2) how many votes summed. This is by no mean to measure "exact contribution", but to defend from anonymous users and trolls who not care about amount of work developers will need to. The results are viewed as a feedback to core team, not as a final decision - that's all for what marketing is solely exist. There no adequate feedback from users to developers currently at all - individual posts in mail lists are too small for statistics (what about ministat(1) for users, huh?..). === The more specifically, this is a Web form with a question and variants of answer, and a message to announce@ with the same and poll ID. The user enters answer and his e-mail on Web, or sends it directly to voting@ with poll ID. The system responds with UUID to verify e-mail is valid, the users just replies, and vote now sits in the DB. An example of a voting where "weighted democracy" could be used: | What you don't need so it should be axed out from base system? | | 0) I'm pretty happy right now, don't do axe anything | 1) BIND only | 2) BIND and Sendmail (provided it will be replaced by dma) | 3) Same as above, but also GCC | ... An example of plain user number counting survey: | There is a page with list of projects seeking sponsors (you can | add your wanted project there), how many you will donate for | implementor of the project you are in need for? | | 1) 1 - 10 $ | 2) 10 - 50 $ | 3) 50 - 100 $ | 4) 100 - 500 $ | 5) I'm money-bag ISP really needing it, I could afford even 50,000$ The latter is example of likely the first survey (wording needs clarification, of course). > So, I'm proposing to raise funds this way, \ for which marketing sureveys are needed, \ for which I propose this system. *I will implement this system if this will be really officially used* Officially means installed at freebsd.org with calls to vote made by authoritative Project members. (this letter is a technical sketch and you may correct me in the way how it should done (UI), I also don't know how mail is stored there on lists.freebsd.org but all this are details and are solvable. ) Please provide an official reply with opinion of the FreeBSD Project. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 03:24:31 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F9FB1065672 for ; Fri, 19 Aug 2011 03:24:31 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 165468FC13 for ; Fri, 19 Aug 2011 03:24:30 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p7J2sAvP079656 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 18 Aug 2011 19:54:14 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4E4DD059.50403@freebsd.org> Date: Thu, 18 Aug 2011 19:54:17 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 MIME-Version: 1.0 To: vadim_nuclight@mail.ru References: <1144162985.20110818235011@serebryakov.spb.ru> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 03:24:31 -0000 On 8/18/11 2:59 PM, Vadim Goncharov wrote: > Hi Lev Serebryakov! > > On Thu, 18 Aug 2011 23:50:11 +0400; Lev Serebryakov wrote about 'Re: FreeBSD problems and preliminary ways to solve': > >>> 8) There is no -STABLE supported branches in ports. >> I want to be more precise here: not -STABLE, but all -RELEASE >> branches, where "upstream" version of ports/packages never changes, >> and only security bugfixes are backported. > To be even more precise, they need a guarantee that automatic updates > will not break anything so that it could be put to cron like "apt-cron". > This goal could be satisfied by another means, I hope: FreeBSD developers > unlikely to have enough time/efforts to keep it for *all* -RELEASE branches, > but for only chosen ones (e.g. extended security support) - may be. > while all the talk about new ports frameworks etc is nice, it is still annoying that the ports and FreeBSD crews don't take the *new* PBI infrastructure that is being pused out with PCBSD-9 as an important move. The new PBI infrastructure should be taken into the ports system as an important factor. For those who do not know it, it give a facility somewhat like the what that APPLE applications work. At the potential (not always) for having redundant libraries, every PBI package comes with EVERYTHING IT NEEDS. there are no 'dependnet packages' as such. On install a survey is made so that if anything is found to be truly duplicated (different versions of the same library are NOT considered a duplicate) then they share, but if not then each package installs and ONLY USES the stuff that came with it. The ramifications of this (in this era of large disks) are immense. If you unstall all your main applications using PBI, then if you screw up your ports installed libraries and development environment when you install some new version of the XXX runtime, *your applications keep working*. A case of "it just works". For the life of me I don't understand WHY there is this resistance to taking it into the fold. Especially when all the work has already been done. It won't replace pkgng and it it won't replace ports because it actually uses ports to generate the PBI packages. But it should be teh default delivery mechanism for binary basic packages. As I said.. go run an apple for a while and see what it is supposed to be like. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 04:02:17 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 039C6106564A for ; Fri, 19 Aug 2011 04:02:17 +0000 (UTC) (envelope-from star_guardian@inbox.ru) Received: from fallback4.mail.ru (fallback4.mail.ru [94.100.176.42]) by mx1.freebsd.org (Postfix) with ESMTP id 71F8D8FC08 for ; Fri, 19 Aug 2011 04:02:16 +0000 (UTC) Received: from f294.mail.ru (f294.mail.ru [217.69.129.76]) by fallback4.mail.ru (mPOP.Fallback_MX) with ESMTP id 59555421D4CC for ; Fri, 19 Aug 2011 07:48:51 +0400 (MSD) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail; h=Message-Id:Content-Transfer-Encoding:Content-Type:Reply-To:Date:Mime-Version:Subject:To:From; bh=aaCFeLD29uMysiwDFr1NlWEG5O0XIfHY4UibLetZAMM=; b=Kl3UE7kxVPy8IVO8vQ5DQFfeha02c6WvcnpWCQAVTx4HJYoZwVEAOVaVTjg5cXUjKRMqe9tvEskRjDoHKeu+h6xDaNKq5RHZZ9x698wyJSSdxeWprRtP22C6rj4JoaCK; Received: from mail by f294.mail.ru with local id 1QuG4e-00005r-00 for freebsd-arch@freebsd.org; Fri, 19 Aug 2011 07:48:48 +0400 Received: from [193.105.72.1] by e.mail.ru with HTTP; Fri, 19 Aug 2011 07:48:48 +0400 From: =?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=?= To: freebsd-arch@freebsd.org Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [193.105.72.1] Date: Fri, 19 Aug 2011 07:48:48 +0400 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Message-Id: X-Spam: Not detected X-Mras: Ok X-Mailman-Approved-At: Fri, 19 Aug 2011 04:13:37 +0000 Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?UTF-8?B?TWljaGFlbCBWLiBCdXp1dmVyb3Y=?= List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 04:02:17 -0000 SGVsbG8KCjE4IEF1Z3VzdCAyMDExLCAyMjozOCBNYXhpbSBJZ25hdGVua28gPGdlbHJhZW4udWFA Z21haWwuY29tPiBoYXMgd3JpdHRlbjoKPiBIaSwKPiAKPiBBbm90aGVyIHdheSBpcyB0byBjcmVh dGUgYmluYXJ5IHBhY2thZ2UgZGlyZWN0bHkgYW5kIGxlYXZlIGFsbCB1cGdyYWRpbmcgbG9naWMg aW4gcGtnbmcuCj4gCgpJIHRoaW5rIG5leHQgc3RlcCBpbiBkZXZlbG9wbWVudCBvZiBwb3J0cyBz eXN0ZW0gc2hvdWxkIGJlIGJ1aWxkaW5nIHBhY2thZ2VzIHdpdGhvdXQgaW5zdGFsbGluZyBwb3J0 cy4KCk1haW4gaWRlYSBpcyBydW50aW1lIGRlcGVuZGVuY2llcyBzaG91bGQgYmUgY2hlY2tlZCBh dCBwYWNrYWdlIChwb3J0KSBpbnN0YWxsYXRpb24gb25seS4gU28sIHdlIGNhbiBidWlsZCBzb21l CmNvbmZsaWN0aW5nIHBhY2thZ2VzIChhcGFjaGUxMywgYXBhY2hlMjAgYW5kIGFwYWNoZTIyLCBm b3IgZXhhbXBsZSkgYW5kIGNob29zZSB3aGljaCB3aWxsIGJlIHVzZWQgYXQgCmluc3RhbGxhdGlv biB0aW1lLgoKTm93LCBwYWNrYWdlIGNyZWF0aW9uIHByb2Nlc3MgaXMgImJ1aWxkIC0+IGluc3Rh bGwgLT4gcGFja2FnZSIuIEkgYmVsaWV2ZSB0aGF0IHNlcXVlbmNlICJidWlsZCAtPiBwYWNrYWdl IC0+IGluc3RhbGwiCmlzIG1vcmUgY29ycmVjdCBhbmQgZWZmaWNpZW50LiBJdCBhbGxvd3MgYXZv aWQgY29uZmxpY3RzIGJldHdlZW4gb2xkIGFuZCBuZXcgcGFja2FnZXMsIHlvdSBtYXkgb25seSBi dWlsZCBzZXQgb2YgbmV3CnBhY2thZ2VzIGFuZCBpbnN0YWxsIHRoZW0gbGF0ZXIuIE9mIGNvdXJj ZSwgaXQgY2FuIGJlIGRvbmUgdXNpbmcgY2hyb290LCBidXQgaXQgaXMgd29ya2Fyb3VuZCwgSSBi ZWxpZXZlIGl0IHNob3VsZCBiZSAKc3RhbmRhcnQgcG9zc2liaWxpdHkgb2YgcG9ydHMgc3lzdGVt LgoKSSB0aGluaywgdGVjaG5pY2FsbHkgaXQgbWF5IGJlIGltcGxlbWVudGVkIGFzIGluc3RhbGxh dGlvbiBwb3J0IGludG8gdGVtcG9yYXJ5IGRpcmVjdG9yeSBhbmQgY3JlYXRpb24gcGFja2FnZQpm cm9tIHRoZXJlLiBPZiBjb3Vyc2UsIHdlIG5lZWQgc29sdmUgc29tZSBwcm9ibGVtcyB3aXRoIGRl cGVuZGVuY2llcywgYnV0IEkgY2FuJ3Qgc2VlIHVuc29sdmFibGUgcHJvYmxlbXMuCgpEbyB5b3Ug dGhpbmsgdGhpcyBmZWF0dXJlIGNvdWxkIGJlIHVzZWZ1bD8KCkJ1enV2ZXJvdiBNaWNoYWVsCg== From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 06:13:22 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D746E106564A for ; Fri, 19 Aug 2011 06:13:22 +0000 (UTC) (envelope-from lme@FreeBSD.org) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) by mx1.freebsd.org (Postfix) with ESMTP id 6A80B8FC08 for ; Fri, 19 Aug 2011 06:13:22 +0000 (UTC) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) by mail.0x20.net (Postfix) with ESMTP id C42016A6C44 for ; Fri, 19 Aug 2011 07:56:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.0x20.net Received: from mail.0x20.net ([217.69.76.211]) by mail.0x20.net (mail.0x20.net [217.69.76.211]) (amavisd-new, port 10024) with ESMTP id Hu0PW4ice--u for ; Fri, 19 Aug 2011 07:56:13 +0200 (CEST) Received: from 0x20.net (0x20.net [217.69.76.212]) (Authenticated sender: lala) by mail.0x20.net (Postfix) with ESMTPA id 667306A6C40 for ; Fri, 19 Aug 2011 07:56:13 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 19 Aug 2011 07:56:13 +0200 From: Lars Engels To: Organization: FreeBSD In-Reply-To: <4E4DD059.50403@freebsd.org> References: "" <1144162985.20110818235011@serebryakov.spb.ru> <4E4DD059.50403@freebsd.org> Message-ID: <37fed82f77901f0e44abddc6d86895c3@mail.0x20.net> X-Sender: lme@FreeBSD.org User-Agent: Roundcube Webmail/0.5.4 Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 06:13:23 -0000 On Thu, 18 Aug 2011 19:54:17 -0700, Julian Elischer wrote: > On 8/18/11 2:59 PM, Vadim Goncharov wrote: >> Hi Lev Serebryakov! >> >> On Thu, 18 Aug 2011 23:50:11 +0400; Lev Serebryakov wrote about 'Re: >> FreeBSD problems and preliminary ways to solve': >> >>>> 8) There is no -STABLE supported branches in ports. >>> I want to be more precise here: not -STABLE, but all -RELEASE >>> branches, where "upstream" version of ports/packages never changes, >>> and only security bugfixes are backported. >> To be even more precise, they need a guarantee that automatic >> updates >> will not break anything so that it could be put to cron like >> "apt-cron". >> This goal could be satisfied by another means, I hope: FreeBSD >> developers >> unlikely to have enough time/efforts to keep it for *all* -RELEASE >> branches, >> but for only chosen ones (e.g. extended security support) - may be. >> > while all the talk about new ports frameworks etc is nice, it is > still annoying that the ports and FreeBSD crews don't take > the *new* PBI infrastructure that is being pused out with PCBSD-9 > as an important move. The new PBI infrastructure should be taken > into the ports system as an important factor. For those who do not > know it, it give a facility somewhat like the what that APPLE > applications work. At the potential (not always) for having redundant > libraries, every PBI package comes with EVERYTHING IT NEEDS. > there are no 'dependnet packages' as such. On install a > survey is made so that if anything is found to be truly duplicated > (different versions of the same library are NOT considered a > duplicate) > then they share, but if not then each package installs and ONLY USES > the stuff that came with it. > > The ramifications of this (in this era of large disks) are immense. > If you unstall all your main applications using PBI, then if you > screw > up your ports installed libraries and development environment when > you > install some new version of the XXX runtime, *your applications keep > working*. > > A case of "it just works". For the life of me I don't understand WHY > there > is this resistance to taking it into the fold. Especially when all > the > work has already been done. It won't replace pkgng and it it won't > replace > ports because it actually uses ports to generate the PBI packages. > But it should be teh default delivery mechanism for binary basic > packages. > > > As I said.. go run an apple for a while and see what it is supposed > to be like. > PBIs are a nice thing but.... The thing that sucked about PBIs at least in PCBSD 8.x is that the PBIs are too big to download. We may all have big disks but there are many people around who don't have fast internet access. E.g. the Firefox PBI is about 100 MB of size and there's a new version of it every few days. Add Thunderbird, VLC and OpenOffice to that list and your're downloading some GBs per month just to get your security holes closed. What is the situation like in PCBSD 9.x? I heard that it was planned to offer update deltas which should be much smaller. If that's so, I'm all for PBIs. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 07:38:23 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D841F106564A for ; Fri, 19 Aug 2011 07:38:23 +0000 (UTC) (envelope-from lme@FreeBSD.org) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) by mx1.freebsd.org (Postfix) with ESMTP id 6D5168FC13 for ; Fri, 19 Aug 2011 07:38:23 +0000 (UTC) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) by mail.0x20.net (Postfix) with ESMTP id 2D69C6A6C4B for ; Fri, 19 Aug 2011 09:38:22 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.0x20.net Received: from mail.0x20.net ([217.69.76.211]) by mail.0x20.net (mail.0x20.net [217.69.76.211]) (amavisd-new, port 10024) with ESMTP id M4vn+5Mi8gdw for ; Fri, 19 Aug 2011 09:38:22 +0200 (CEST) Received: from 0x20.net (0x20.net [217.69.76.212]) (Authenticated sender: lala) by mail.0x20.net (Postfix) with ESMTPA id D7D106A6C49 for ; Fri, 19 Aug 2011 09:38:21 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 19 Aug 2011 09:38:21 +0200 From: Lars Engels To: Organization: FreeBSD In-Reply-To: References: <705869186.20110819012421@serebryakov.spb.ru> Message-ID: <18ea84978d3705e47de87a5d9a0dd7ea@mail.0x20.net> X-Sender: lme@FreeBSD.org User-Agent: Roundcube Webmail/0.5.4 Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 07:38:24 -0000 On Thu, 18 Aug 2011 22:17:25 +0000 (UTC), Vadim Goncharov wrote: > Hi Lev Serebryakov! > > On Fri, 19 Aug 2011 01:24:21 +0400; Lev Serebryakov wrote about 'Re: > FreeBSD problems and preliminary ways to solve': > >>> The other social problem is lack of companies which offer >>> commercial >>> support of FreeBSD like RedHat does. >> Main social problem, IMHO, that there WAS NOT (I forgot Linux >> history and don't rememberfirst uf-distributive) and, later, >> Ubuntu-like versions of FeeBSD. Even if these doen;t replace Windows >> on many desktops (1% now?), they prepare Linux-aware users, and some >> of these users becomes admins or people, who decide which OS should >> be >> used in their business. > > PC-BSD is exactly for that. > And IMHO PC-BSD should get pushed more. The changes in 9.0 are a great opportunity for this. We should write mails to PC magazines and ask them to include a version of PC-BSD on their DVD, ask IT websites to write some news about the release, tell the people that it's a system which is actually easy to use, has some unique features and is definitively worth a try. To gain market share, PC-BSDand FreeBSD need to be known! In my experience, maybe one out of ten IT admins have ever heard of FreeBSD, and one out of hundred has actually tried it. Of them, maybe 10 percent actually used it for some time but for whatever reason eventually went back to Windows or Linux where they came from. So there's only a very small amount of fresh users with every release, while others go ahead and switch to different OSes because of the reasons Vadmin wrote about. But that was only the situation of our fellow admins. What about their bosses who are the ones who decide which way to go and which OS is allowed to run on their servers? I know answers like "Free-what? Ah, some Linux-like OS. Never heard of it. Why don't we use Linux if it's similar? Everybody uses Linux, it's standard! Do you get support from Big Companies for FreeBSD? Who do we call if we have a problem with it on our servers? Doesn't HP hang up the phone if we tell them that we installed that Free.. what was it called? FreeBSD??? on it?" So FreeBSD urgently needs a wider awareness level. People have heard of FreeBSD -> People are more willing to try it -> Some people try it -> Some people stay with FreeBSD and like it -> Some people spread the word -> More people try it -> At the end of the chain we have some new contributors, devs, donors, and new installations For the people who may say now: "We don't want Ubuntu-like users, they only ask stupid questions I don't want to answer again and again." We need them! Everybody once started by asking dumb question but eventually we began to answer other people's question. Take a look at FreeBSD Forums: 26,000 members, 137,000 posts in 20,000 threads. Only a small fraction of theses posts are from FreeBSD developers, the others are good and valid answers to the questions you don't want to answer any more. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 08:23:47 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 518B8106566C for ; Fri, 19 Aug 2011 08:23:47 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0A5E98FC13 for ; Fri, 19 Aug 2011 08:23:47 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 9E72246B23; Fri, 19 Aug 2011 04:23:46 -0400 (EDT) Date: Fri, 19 Aug 2011 09:23:46 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Vadim Goncharov In-Reply-To: Message-ID: References: <705869186.20110819012421@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 08:23:47 -0000 On Thu, 18 Aug 2011, Vadim Goncharov wrote: >> everything is ``suitable'' for common tasks. And it is NOT ENOUGH >> to be technically better. System should be far more superior to be >> chosen, if it is not fancy/trendy. Yes, I belive, that FreeBSD is >> better than Linux (at least on supported hardware) in server tasks, >> more clear, more solid, etc. But it is ``only'' better, and is not >> enough. > > No. The whole message was about that FreeBSD is worse in several areas, > which masks out areas where it is better. If that will get fixed, we could > talk about is it needing to be "far more" superior or not. But not before > that as excuse to do nothing - no such excuse exists. I'd point out that the key thing here is to produce, distribute, and as you rightly point out *make easy*, new technologies that are transformative in a way that makes FreeBSD compelling. So compelling that you'd rather switch OS than not have it. It's worth observing that the success of Linux (and FreeBSD) to date comes out of a few very basic but fundamental improvements in OS design: - Tightly integrated networking - Much cheaper than any competition - Extremely stable compared to the competition It's clear that Windows has long since caught up in the server world with respect to these in every practical sense (not an invitation for discussion -- we could talk for days) in that it is now a viable server solution in all of the above senses. Which isn't to say that my multi-year uptimes for FreeBSD don't beat the competition, but it's clear FreeBSD/Linux no longer give you orders of magnitude more uptime between crashes. I think people are also now much more aware of the TCO issue with operating systems, and understand that although open source is often better, and therefore cheaper, the lack of a license fee isn't the biggest issue in cost. However, I think the lesson is clear: compelling features required (including things like cost and stability). My own interests are largely in security, and this is what we're trying to do with Capsicum: make it possible to sandbox applications in a way that simply has never been done before, giving security that has never been had before. This required a new solution (although if you read our USENIX Security or forthcoming CACM paper, it is grounded in some quite old but promising ideas) that you can't find anywhere else. It will take several years for Capsicum to meet this promise, but the first parts arrive in FreeBSD 9.0. FreeBSD 9.1 is where it should get really exciting -- even (and especially) tools like tcpdump will run automatically and easily in sandboxes, something that just isn't plausibe with other existing sandboxing technologies. Obviously, we hope that the rest of the world will adopt our APIs (and have spotted the OpenBSD folk working on this already, and there's a Linux port out of Google), but I hope for a bit of a run where you have to come to us to get this! And being the place APIs and ideas like this come from is important. There are lots of other exciting things in 9.x, and we need to make sure we promote them well. I'd point out that this is an area where we also need to do a lot of work. There was a lot of buzz around the release of FreeBSD 8 when Kris was running regular competitive SMP benchmarks and showing us walking all over the competition. Buzz is a critical part of selling ideas in open source (for better or worse), and there's no reason we can't play in that game a bit while maintaining our boring and staid personalities :-). Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 08:37:06 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B79A5106566B for ; Fri, 19 Aug 2011 08:37:06 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 575858FC13 for ; Fri, 19 Aug 2011 08:37:06 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 775C44AC1C; Fri, 19 Aug 2011 12:37:04 +0400 (MSD) Date: Fri, 19 Aug 2011 12:37:00 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <810527321.20110819123700@serebryakov.spb.ru> To: Vadim Goncharov In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 08:37:06 -0000 Hello, Vadim. You wrote 18 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 3:10:19: > 3. Kernel features for complex network solutions (netgraph, carp, ipfw). > The niche for routers & traffic analysis is still ours. It would be > nice to take e.g. pfSense and agree with some vendor (Netgear, > D-Link, etc) to put on sale hardware with FreeBSD inside. What about 10G routing? Here are reports about full-bandwidth 10G routing= on modern Intel NICs with Linux (and multi-core server), but I didn't see any such data for FreeBSD, and somebody says, that Intel drivers and network stack is not so good parallel in FreeBSD. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 08:41:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D2E11065673; Fri, 19 Aug 2011 08:41:42 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 77DCD8FC16; Fri, 19 Aug 2011 08:41:42 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 6574546B06; Fri, 19 Aug 2011 04:41:41 -0400 (EDT) Date: Fri, 19 Aug 2011 09:41:41 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Lev Serebryakov In-Reply-To: <810527321.20110819123700@serebryakov.spb.ru> Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Vadim Goncharov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 08:41:42 -0000 On Fri, 19 Aug 2011, Lev Serebryakov wrote: >> 3. Kernel features for complex network solutions (netgraph, carp, ipfw). >> The niche for routers & traffic analysis is still ours. It would be >> nice to take e.g. pfSense and agree with some vendor (Netgear, >> D-Link, etc) to put on sale hardware with FreeBSD inside. > > What about 10G routing? Here are reports about full-bandwidth 10G routing > on modern Intel NICs with Linux (and multi-core server), but I didn't see > any such data for FreeBSD, and somebody says, that Intel drivers and network > stack is not so good parallel in FreeBSD. Our network stack is actually pretty parallel as such things go, and there are a number of changes in FreeBSD 9.x that extend this work. Most of the performance work is being done on edge nodes rather than middle nodes -- i.e., maxing out multiple 10gbps links serving content, etc, rather than in routing configurations, though. We also have a strong and growing collection of 10gbps drivers. You'll find our drivers lifted for many other systems, including Solaris :-). There are a few known issues in terms of parallelism -- one is contention between the ithread and user thread on per-TCP connection locks. Another is that we still haven't managed to switch to per-CPU statistics for the network stack (which is fairly straight forward in a specific sense, but we'd like a proper abstraction for it so we can generalise). Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 08:50:11 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96EAB106564A; Fri, 19 Aug 2011 08:50:11 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5BE648FC08; Fri, 19 Aug 2011 08:50:11 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 3361D4AC58; Fri, 19 Aug 2011 12:50:10 +0400 (MSD) Date: Fri, 19 Aug 2011 12:50:05 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <319607032.20110819125005@serebryakov.spb.ru> To: Robert Watson In-Reply-To: References: <810527321.20110819123700@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 08:50:11 -0000 Hello, Robert. You wrote 19 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 12:41:41: > Our network stack is actually pretty parallel as such things go, and ther= e are > a number of changes in FreeBSD 9.x that extend this work. Most of the > performance work is being done on edge nodes rather than middle nodes -- = i.e., > maxing out multiple 10gbps links serving content, etc, rather than in rou= ting > configurations, though. We also have a strong and growing collection of > 10gbps drivers. You'll find our drivers lifted for many other systems, > including Solaris :-). I need to bribe our admins (OPs) on my paid work to try FreeBSD instead of Solaris on new servers :) We are processing huge amount of multicast streams (up to 2.5-3Gbit/s with 500-1000 bytes packets) and have difficulties not to lose any packets on Solaris :) They tried Linux without success, but FreeBSD is unknown to them. One problem for FreeBSD is that our applications are Java-based... Problems are not in Java, but in Intel drivers (igb / ixgb in FreeBSD terms), which sometimes lose packets with "buffer is not available" diagnostics when consumer is heavily-multithreaded. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 08:56:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B467106566B; Fri, 19 Aug 2011 08:56:03 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 627578FC13; Fri, 19 Aug 2011 08:56:03 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 03E4246B06; Fri, 19 Aug 2011 04:56:03 -0400 (EDT) Date: Fri, 19 Aug 2011 09:56:02 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Lev Serebryakov In-Reply-To: <319607032.20110819125005@serebryakov.spb.ru> Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 08:56:03 -0000 On Fri, 19 Aug 2011, Lev Serebryakov wrote: >> Our network stack is actually pretty parallel as such things go, and there are >> a number of changes in FreeBSD 9.x that extend this work. Most of the >> performance work is being done on edge nodes rather than middle nodes -- i.e., >> maxing out multiple 10gbps links serving content, etc, rather than in routing >> configurations, though. We also have a strong and growing collection of >> 10gbps drivers. You'll find our drivers lifted for many other systems, >> including Solaris :-). > > I need to bribe our admins (OPs) on my paid work to try FreeBSD instead of > Solaris on new servers :) We are processing huge amount of multicast streams > (up to 2.5-3Gbit/s with 500-1000 bytes packets) and have difficulties not to > lose any packets on Solaris :) They tried Linux without success, but FreeBSD > is unknown to them. > > One problem for FreeBSD is that our applications are Java-based... Problems > are not in Java, but in Intel drivers (igb / ixgb in FreeBSD terms), which > sometimes lose packets with "buffer is not available" diagnostics when > consumer is heavily-multithreaded. Is the issue here that FreeBSD is dropping more packes, or just that FreeBSD is reporting that it drops packets? Historically, we've returned ENOBUFS from datagram sockets when the interface queue is overflowed, but some other systems (most noticeably Linux) simply return success when they drop a packet on an outgoing interface queue. You can debate which is the better model, but one impact is that sometimes people report errors on FreeBSD that they don't see on Linux -- when actually, the same failure is present, we just allow the application to learn about it. Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:05:47 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F88B106566B; Fri, 19 Aug 2011 09:05:47 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 1339B8FC0C; Fri, 19 Aug 2011 09:05:47 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id E3AD14AC1C; Fri, 19 Aug 2011 13:05:45 +0400 (MSD) Date: Fri, 19 Aug 2011 13:05:41 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <319640694.20110819130541@serebryakov.spb.ru> To: Robert Watson In-Reply-To: References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:05:47 -0000 Hello, Robert. You wrote 19 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 12:56:02: > Is the issue here that FreeBSD is dropping more packes, or just that Free= BSD > is reporting that it drops packets? Historically, we've returned ENOBUFS= from I wasn't clear enough: we didn't try FreeBSD, as I'm developer in this project, and our admins (ops) know nothing about FreeBSD and have enough current tasks to don't have any spare time to try new system. Maybe, I'll be able to "borrow" one of test servers in _my_ spare time and try to play with FreeBSD in this environment. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:06:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C33C106564A for ; Fri, 19 Aug 2011 09:06:28 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 46A7A8FC08 for ; Fri, 19 Aug 2011 09:06:27 +0000 (UTC) Received: by gwb15 with SMTP id 15so1742520gwb.13 for ; Fri, 19 Aug 2011 02:06:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Q3I/Ay3cDgwjKG3X4zyM/DATPRpxatAjSXye9eTbJ8o=; b=MSHQMmdKDCXENrIpGdDDzZYgD/DxD1YvODkLbg5K8OUrM6XQNXauPeAUA3A/1gzXG8 MRs5fhweLkVjuB0yTLXwfW3Hjcp3uimWWwRVv9nYGUqS0q3fvcVrMMD6tF1Oapmga0ar 5JdreY4UivJACpFaDqLtxfjYCrHa6NPVljx1I= MIME-Version: 1.0 Received: by 10.150.158.17 with SMTP id g17mr738157ybe.386.1313744787388; Fri, 19 Aug 2011 02:06:27 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 19 Aug 2011 02:06:27 -0700 (PDT) In-Reply-To: <810527321.20110819123700@serebryakov.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> Date: Fri, 19 Aug 2011 17:06:27 +0800 X-Google-Sender-Auth: -24WyXkrqLlQqQDFM843TtrvuIE Message-ID: From: Adrian Chadd To: Lev Serebryakov Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable Cc: Vadim Goncharov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:06:28 -0000 Like a lot of things, I think something that's sorely missing here from this discussion is the users needs versus the developers needs. You've given me a great example. 2011/8/19 Lev Serebryakov : > Hello, Vadim. > You wrote 18 =C1=D7=C7=D5=D3=D4=C1 2011 =C7., 3:10:19: > > >> 3. Kernel features for complex network solutions (netgraph, carp, ipfw). >> =9A =9AThe niche for routers & traffic analysis is still ours. It would = be >> =9A =9Anice to take e.g. pfSense and agree with some vendor (Netgear, >> =9A =9AD-Link, etc) to put on sale hardware with FreeBSD inside. > =9AWhat about 10G routing? Here are reports about full-bandwidth 10G rout= ing on modern > Intel NICs with Linux (and multi-core server), but I didn't see any > such data for FreeBSD, and somebody says, that Intel drivers and > network stack is not so good parallel in FreeBSD. Can freebsd do it now? Maybe not. Do FreeBSD developers know what needs to occur? (eg someone like Kip?) Yes. What's missing? There's a lot of users that say "yes, I'd like X, but FreeBSD doesn't do it= ." There's a lot of developers who know what to do, or could figure it out, but it's not within their scope of work. There's only so much one can do for fun. Linux works here because of two things: * there's a heavy following of -users- who are -developers-, and * there's enough developers who want these features in Linux, so they get d= one. FreeBSD suffers here because the developers are working on what they're paid to or or what they'd like to do, and this doesn't at all fit in with what some users (even large ones, like said large russian ISPs) would like to see, right now. Xen is a great example of how this failed. Is there a lot of interest in Xen support for FreeBSD? Yes. The companies that got it done internally made it work for their needs - eg Yahoo! with Xen HVM support. They didn't need PVM, so it didn't get done. Lots of people would pop up and ask about PVM support to run FreeBSD in EC2 instances. But since there wasn't any developers working on it, nothing happened. The moment I started poking Xen PVM (just to see how much effort it'd take to get -HEAD into shape), I had a whole lot of users ask me about it, email me about how to build images, how to test images, why things weren't working, etc. When it was obvious that it would need someone to do VM surgery to fix things up, I stopped working on it. I wasn't getting paid to do it, I'm a self-employed contractor who was also finishing a degree, so "fix it up for fun" wasn't on my TODO list. When Colin got involved, and managed to get Amazon's interest (yay!), things moved along a bit more. I don't think -HEAD is stable in PVM still. That again, needs developers. Another example is 11n support. Lots of users would love 11n support. I've had plenty of people ask me when it'll be ready. But where are the 802.11 developers? All you see/hear are crickets. That big rush of 11n work that I've been doing lately? It's because a couple of companies stood up and asked for the atheros and net80211 DFS support to be improved - and they were willing to pay for it. Since there's not enough (almost none) people hacking on it in public, the code languished for years. Having lots of users email me privately (and a couple of companies) is nice but it doesn't at all demonstrate to people who can fund these things (eg the Foundation) what the users are after. If you'd like to see this feature in FreeBSD, and you can find a bunch of like-minded companies, then please get together and talk to the foundation about how to get this put on their agenda. Chase it up, get it done. Find a developer, get him on board, get it done. Be loud. Be supportive. But please, don't just say "but where's X?" "X" is wherever the like-minded developer with the equipment needed and the time/money needed is. Finally - 10G line rate support isn't as easy as you think. Well, it is, but it's costly. 11n hacking is (comparatively) cheap because someone's already written mostly-working code, and the hardware is cheap. I don't need a $20k spectrum analyser or a $80k signal generator to get the basics done. But 10GE stuff requires 10GE NICs, 10GE switches. Those don't come cheap. So it's not going to be done by someone who is bored and at home. It's going to be done by someone who has access to the right kit. Luigi's netmap support shows what's possible. The question is, can someone pick it up and run with it. Note, he's also hit all those corner/edge cases with NICs because of the sheer rate of transactions going on (and he's emailed questions about it publicly.) 2c, Adrian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:22:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 703F4106566B; Fri, 19 Aug 2011 09:22:54 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id C477E8FC0A; Fri, 19 Aug 2011 09:22:53 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QuLI8-000P4o-IC; Fri, 19 Aug 2011 13:23:04 +0400 Date: Fri, 19 Aug 2011 13:23:04 +0400 From: Slawa Olhovchenkov To: Robert Watson Message-ID: <20110819092304.GB92576@zxy.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Vadim Goncharov , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:22:54 -0000 On Fri, Aug 19, 2011 at 09:41:41AM +0100, Robert Watson wrote: > There are a few known issues in terms of parallelism -- one is contention > between the ithread and user thread on per-TCP connection locks. Another is > that we still haven't managed to switch to per-CPU statistics for the network > stack (which is fairly straight forward in a specific sense, but we'd like a > proper abstraction for it so we can generalise). After discussion in other place: may be perfomace improve if application resheduling on the same core, as processing incoming IP packet (hot CPU caÓhe)? -- Slawa Olhovchenkov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:26:13 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D25E8106564A; Fri, 19 Aug 2011 09:26:13 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 971018FC08; Fri, 19 Aug 2011 09:26:13 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 5B3904AC1C; Fri, 19 Aug 2011 13:26:12 +0400 (MSD) Date: Fri, 19 Aug 2011 13:26:07 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1923391409.20110819132607@serebryakov.spb.ru> To: Adrian Chadd In-Reply-To: References: <810527321.20110819123700@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:26:13 -0000 Hello, Adrian. You wrote 19 =C1=D7=C7=D5=D3=D4=C1 2011 =C7., 13:06:27: >>> 3. Kernel features for complex network solutions (netgraph, carp, ipfw). >>> =9A =9AThe niche for routers & traffic analysis is still ours. It would= be >>> =9A =9Anice to take e.g. pfSense and agree with some vendor (Netgear, >>> =9A =9AD-Link, etc) to put on sale hardware with FreeBSD inside. >> =9AWhat about 10G routing? Here are reports about full-bandwidth 10G rou= ting on modern >> Intel NICs with Linux (and multi-core server), but I didn't see any >> such data for FreeBSD, and somebody says, that Intel drivers and >> network stack is not so good parallel in FreeBSD. I could say, that I have nothing to object to your message, I agree with you almost completely. Some notes below, but they are minor ones. > Linux works here because of two things: > * there's a heavy following of -users- who are -developers-, and As far as I understand (I could be wrong), there's many COMPANIES, which are USERS and hires (on-site, off-site -- it doesn't matter) DEVELOPERS. It is not a situation when users are exactly same persons as developers or vice versa, but they are linked together directly. > * there's enough developers who want these features in Linux, so they get= done. And there is enough users, who want and can pay to these developers. I don't think, that "independent (private, hobby, whatever) developers to independent users" ratio is better for Linux, that for FreeBSD :) But number of payed developers, of course, is much, much larger for Linux now. I have several FreeBSD users around me (they are not payed to be DEVELOPERS, sometimes they are payed to be ADMINS, sometimes it is their own non-profit systems, like mine), and everybody sent PR at least once, often with patches to solve problem. About 1/4 of my friends (yes, I have non-typical friends, I know) uses Linux on desktops, notebooks or servers, and most of them didn't send any bugreports, not to mention patches. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:28:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8C5B1065672 for ; Fri, 19 Aug 2011 09:28:24 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9DEC18FC0C for ; Fri, 19 Aug 2011 09:28:24 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id D3BF74AC1C; Fri, 19 Aug 2011 13:28:23 +0400 (MSD) Date: Fri, 19 Aug 2011 13:28:19 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1156334334.20110819132819@serebryakov.spb.ru> To: Slawa Olhovchenkov In-Reply-To: <20110819092304.GB92576@zxy.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> <20110819092304.GB92576@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:28:24 -0000 Hello, Slawa. You wrote 19 =C1=D7=C7=D5=D3=D4=C1 2011 =C7., 13:23:04: >> There are a few known issues in terms of parallelism -- one is contention >> between the ithread and user thread on per-TCP connection locks. Anothe= r is=20 >> that we still haven't managed to switch to per-CPU statistics for the ne= twork=20 >> stack (which is fairly straight forward in a specific sense, but we'd li= ke a=20 >> proper abstraction for it so we can generalise). > After discussion in other place: may be perfomace improve if > application resheduling on the same core, as processing incoming > IP packet (hot CPU ca=D3he)? Yes, it is exactly what our admins done, and it helps, but if streams will grow (and they WILL) here will be a little space to grow up performance as well. But it is completely offtopic already. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:29:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6860F1065672; Fri, 19 Aug 2011 09:29:16 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id BBFEE8FC16; Fri, 19 Aug 2011 09:29:15 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QuL1E-000OJq-Ih; Fri, 19 Aug 2011 13:05:36 +0400 Date: Fri, 19 Aug 2011 13:05:36 +0400 From: Slawa Olhovchenkov To: Robert Watson Message-ID: <20110819090536.GA92576@zxy.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:29:16 -0000 On Fri, Aug 19, 2011 at 09:56:02AM +0100, Robert Watson wrote: > > On Fri, 19 Aug 2011, Lev Serebryakov wrote: > > >> Our network stack is actually pretty parallel as such things go, and there are > >> a number of changes in FreeBSD 9.x that extend this work. Most of the > >> performance work is being done on edge nodes rather than middle nodes -- i.e., > >> maxing out multiple 10gbps links serving content, etc, rather than in routing > >> configurations, though. We also have a strong and growing collection of > >> 10gbps drivers. You'll find our drivers lifted for many other systems, > >> including Solaris :-). > > > > I need to bribe our admins (OPs) on my paid work to try FreeBSD instead of > > Solaris on new servers :) We are processing huge amount of multicast streams > > (up to 2.5-3Gbit/s with 500-1000 bytes packets) and have difficulties not to > > lose any packets on Solaris :) They tried Linux without success, but FreeBSD > > is unknown to them. > > > > One problem for FreeBSD is that our applications are Java-based... Problems > > are not in Java, but in Intel drivers (igb / ixgb in FreeBSD terms), which > > sometimes lose packets with "buffer is not available" diagnostics when > > consumer is heavily-multithreaded. > > Is the issue here that FreeBSD is dropping more packes, or just that FreeBSD > is reporting that it drops packets? Historically, we've returned ENOBUFS from > datagram sockets when the interface queue is overflowed, but some other > systems (most noticeably Linux) simply return success when they drop a packet > on an outgoing interface queue. You can debate which is the better model, but > one impact is that sometimes people report errors on FreeBSD that they don't > see on Linux -- when actually, the same failure is present, we just allow the > application to learn about it. Historically, Linux on datagram (UDP) socket allow use select, FreeBSD -- don't allow. FreeBSD always report 'UDP socket ready to transmit'. And after try to send packet -- 'oops, ENOBUFS'. -- Slawa Olhovchenkov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:36:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F955106566B; Fri, 19 Aug 2011 09:36:33 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DBE1D8FC08; Fri, 19 Aug 2011 09:36:32 +0000 (UTC) Received: from [192.168.2.124] (host81-151-180-177.range81-151.btcentralplus.com [81.151.180.177]) by cyrus.watson.org (Postfix) with ESMTPSA id D643746B06; Fri, 19 Aug 2011 05:36:31 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <20110819090536.GA92576@zxy.spb.ru> Date: Fri, 19 Aug 2011 10:36:29 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> <20110819090536.GA92576@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.1084) Cc: Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:36:33 -0000 On 19 Aug 2011, at 10:05, Slawa Olhovchenkov wrote: >> Is the issue here that FreeBSD is dropping more packes, or just that = FreeBSD=20 >> is reporting that it drops packets? Historically, we've returned = ENOBUFS from=20 >> datagram sockets when the interface queue is overflowed, but some = other=20 >> systems (most noticeably Linux) simply return success when they drop = a packet=20 >> on an outgoing interface queue. You can debate which is the better = model, but=20 >> one impact is that sometimes people report errors on FreeBSD that = they don't=20 >> see on Linux -- when actually, the same failure is present, we just = allow the=20 >> application to learn about it. >=20 > Historically, Linux on datagram (UDP) socket allow use select, FreeBSD > -- don't allow. FreeBSD always report 'UDP socket ready to transmit'. > And after try to send packet -- 'oops, ENOBUFS'. And if you have two consumers sending UDP on Linux, they both get = unreported 50% packet loss, to my understanding? Robert= From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:36:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7245F106564A; Fri, 19 Aug 2011 09:36:43 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 26B848FC08; Fri, 19 Aug 2011 09:36:43 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QuLVW-000PEY-GV; Fri, 19 Aug 2011 13:36:54 +0400 Date: Fri, 19 Aug 2011 13:36:54 +0400 From: Slawa Olhovchenkov To: Lev Serebryakov Message-ID: <20110819093654.GC92576@zxy.spb.ru> References: <1144162985.20110818235011@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1144162985.20110818235011@serebryakov.spb.ru> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Vadim Goncharov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:36:43 -0000 On Thu, Aug 18, 2011 at 11:50:11PM +0400, Lev Serebryakov wrote: > > 8) There is no -STABLE supported branches in ports. > I want to be more precise here: not -STABLE, but all -RELEASE > branches, where "upstream" version of ports/packages never changes, > and only security bugfixes are backported. But when? "Soft" version depend is best, IMHO. -- Slawa Olhovchenkov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:38:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 386C5106564A; Fri, 19 Aug 2011 09:38:02 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0E54E8FC1A; Fri, 19 Aug 2011 09:38:02 +0000 (UTC) Received: from [192.168.2.124] (host81-151-180-177.range81-151.btcentralplus.com [81.151.180.177]) by cyrus.watson.org (Postfix) with ESMTPSA id F0CF146B2D; Fri, 19 Aug 2011 05:38:00 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=koi8-r From: "Robert N. M. Watson" In-Reply-To: <20110819092304.GB92576@zxy.spb.ru> Date: Fri, 19 Aug 2011 10:37:59 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <07024C3C-C6C7-47C1-9763-E3A62CA37B0C@FreeBSD.org> References: <810527321.20110819123700@serebryakov.spb.ru> <20110819092304.GB92576@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.1084) Cc: Vadim Goncharov , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:38:02 -0000 On 19 Aug 2011, at 10:23, Slawa Olhovchenkov wrote: > On Fri, Aug 19, 2011 at 09:41:41AM +0100, Robert Watson wrote: >=20 >> There are a few known issues in terms of parallelism -- one is = contention=20 >> between the ithread and user thread on per-TCP connection locks. = Another is=20 >> that we still haven't managed to switch to per-CPU statistics for the = network=20 >> stack (which is fairly straight forward in a specific sense, but we'd = like a=20 >> proper abstraction for it so we can generalise). >=20 > After discussion in other place: may be perfomace improve if > application resheduling on the same core, as processing incoming > IP packet (hot CPU ca=D3he)? We have a GSoC student working on this currently (measuring rough socket = affinity for applications a la RPS and redirecting work on the software = side). I also have some related work, which can supplement his, which = will push those application affinities into programmable TCAMs/hardware = hash tables in 10gbs cards. We do need some general improvements in = scheduling to allow the scheduler to take into account notions of = vertically aligning affinity, however. Robert= From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:50:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20A5C1065670; Fri, 19 Aug 2011 09:50:33 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 76E218FC19; Fri, 19 Aug 2011 09:50:32 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QuLit-000PTZ-EC; Fri, 19 Aug 2011 13:50:43 +0400 Date: Fri, 19 Aug 2011 13:50:43 +0400 From: Slawa Olhovchenkov To: "Robert N. M. Watson" Message-ID: <20110819095043.GB2362@zxy.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> <20110819090536.GA92576@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:50:33 -0000 On Fri, Aug 19, 2011 at 10:36:29AM +0100, Robert N. M. Watson wrote: > > On 19 Aug 2011, at 10:05, Slawa Olhovchenkov wrote: > > >> Is the issue here that FreeBSD is dropping more packes, or just that FreeBSD > >> is reporting that it drops packets? Historically, we've returned ENOBUFS from > >> datagram sockets when the interface queue is overflowed, but some other > >> systems (most noticeably Linux) simply return success when they drop a packet > >> on an outgoing interface queue. You can debate which is the better model, but > >> one impact is that sometimes people report errors on FreeBSD that they don't > >> see on Linux -- when actually, the same failure is present, we just allow the > >> application to learn about it. > > > > Historically, Linux on datagram (UDP) socket allow use select, FreeBSD > > -- don't allow. FreeBSD always report 'UDP socket ready to transmit'. > > And after try to send packet -- 'oops, ENOBUFS'. > > > And if you have two consumers sending UDP on Linux, they both get unreported 50% packet loss, to my understanding? In my test I use netperf. netperf/UDP on linux send only packets can be proccessed by box (and NIC) and packets don't drop. netperf don't allocate all CPU cycles. netperf/UDP on FreeBSD get all CPU, try to send HUGE UDP flow and report about massive packets drop. select on UDP socket report 'ready' allways. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 10:06:18 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48DAE1065672; Fri, 19 Aug 2011 10:06:18 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by mx1.freebsd.org (Postfix) with ESMTP id A33C88FC19; Fri, 19 Aug 2011 10:06:17 +0000 (UTC) Received: from [194.32.164.28] (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id p7J9plIL096075; Fri, 19 Aug 2011 10:51:47 +0100 (BST) (envelope-from rb@gid.co.uk) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Bob Bishop In-Reply-To: Date: Fri, 19 Aug 2011 10:51:42 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <50D0583B-BF23-4CD5-ACDB-DEEBDD5B465B@gid.co.uk> References: <705869186.20110819012421@serebryakov.spb.ru> To: Robert Watson X-Mailer: Apple Mail (2.1084) Cc: Vadim Goncharov , freebsd-arch@FreeBSD.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 10:06:18 -0000 Hi, On 19 Aug 2011, at 09:23, Robert Watson wrote: > ...it's clear FreeBSD/Linux no longer give you orders of magnitude = more uptime between crashes. ... Here's a data point that suggests otherwise: http://www.networksolutions.com/web-hosting/popup-uptime.jsp -- Bob Bishop rb@gid.co.uk From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 12:35:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EF2D106567A; Fri, 19 Aug 2011 12:35:58 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from mx.utwente.nl (mx1.utsp.utwente.nl [130.89.2.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7B5A38FC0A; Fri, 19 Aug 2011 12:35:57 +0000 (UTC) Received: from pieter-dev.localnet (lux.student.utwente.nl [130.89.161.112]) by mx.utwente.nl (8.12.10/SuSE Linux 0.7) with ESMTP id p7JC1NM7013018; Fri, 19 Aug 2011 14:01:23 +0200 From: Pieter de Goeje To: freebsd-arch@freebsd.org Date: Fri, 19 Aug 2011 14:01:23 +0200 User-Agent: KMail/1.13.7 (Linux/2.6.39-2-amd64; KDE/4.6.5; x86_64; ; ) References: <810527321.20110819123700@serebryakov.spb.ru> In-Reply-To: <810527321.20110819123700@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable Message-Id: <201108191401.23083.pieter@degoeje.nl> X-UTwente-MailScanner-Information: Scanned by MailScanner. Contact icts.servicedesk@utwente.nl for more information. X-UTwente-MailScanner: Found to be clean X-UTwente-MailScanner-From: pieter@degoeje.nl X-Spam-Status: No Cc: Vadim Goncharov , Lev Serebryakov Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 12:35:58 -0000 On Friday, August 19, 2011 10:37:00 AM Lev Serebryakov wrote: > Hello, Vadim. >=20 > You wrote 18 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 3:10:19: > > 3. Kernel features for complex network solutions (netgraph, carp, ipfw). > >=20 > > The niche for routers & traffic analysis is still ours. It would be > > nice to take e.g. pfSense and agree with some vendor (Netgear, > > D-Link, etc) to put on sale hardware with FreeBSD inside. >=20 > What about 10G routing? Here are reports about full-bandwidth 10G routi= ng > on modern Intel NICs with Linux (and multi-core server), but I didn't see > any such data for FreeBSD, and somebody says, that Intel drivers and > network stack is not so good parallel in FreeBSD. With regards to high speed packet forwarding and routing, check out this wo= rk=20 by Luigi Rizzo: http://info.iet.unipi.it/~luigi/netmap/ =2D Pieter From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 12:41:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 473B41065670; Fri, 19 Aug 2011 12:41:02 +0000 (UTC) (envelope-from kris@pcbsd.org) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 238278FC17; Fri, 19 Aug 2011 12:41:01 +0000 (UTC) Received: from mail.ixsystems.com (localhost [127.0.0.1]) by mail.iXsystems.com (Postfix) with ESMTP id 0967BA6643E; Fri, 19 Aug 2011 05:21:07 -0700 (PDT) Received: from mail.iXsystems.com ([127.0.0.1]) by mail.ixsystems.com (mail.ixsystems.com [127.0.0.1]) (amavisd-maia, port 10024) with ESMTP id 98126-02; Fri, 19 Aug 2011 05:21:06 -0700 (PDT) Received: from [10.232.125.226] (111.sub-174-252-159.myvzw.com [174.252.159.111]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id F066AA66437; Fri, 19 Aug 2011 05:21:05 -0700 (PDT) References: "" <1144162985.20110818235011@serebryakov.spb.ru> <4E4DD059.50403@freebsd.org> <37fed82f77901f0e44abddc6d86895c3@mail.0x20.net> User-Agent: K-9 Mail for Android In-Reply-To: <37fed82f77901f0e44abddc6d86895c3@mail.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Kris Moore Date: Fri, 19 Aug 2011 08:21:14 -0400 To: Lars Engels ,freebsd-arch@freebsd.org Message-ID: <428ef630-c2b4-4a42-b2c6-c34d63a8f353@email.android.com> Cc: Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 12:41:02 -0000 Lars Engels wrote: >On Thu, 18 Aug 2011 19:54:17 -0700, Julian Elischer wrote: >> On 8/18/11 2:59 PM, Vadim Goncharov wrote: >>> Hi Lev Serebryakov! >>> >>> On Thu, 18 Aug 2011 23:50:11 +0400; Lev Serebryakov wrote about 'Re: > >>> FreeBSD problems and preliminary ways to solve': >>> >>>>> 8) There is no -STABLE supported branches in ports. >>>> I want to be more precise here: not -STABLE, but all -RELEASE >>>> branches, where "upstream" version of ports/packages never changes, >>>> and only security bugfixes are backported. >>> To be even more precise, they need a guarantee that automatic >>> updates >>> will not break anything so that it could be put to cron like >>> "apt-cron". >>> This goal could be satisfied by another means, I hope: FreeBSD >>> developers >>> unlikely to have enough time/efforts to keep it for *all* -RELEASE >>> branches, >>> but for only chosen ones (e.g. extended security support) - may be. >>> >> while all the talk about new ports frameworks etc is nice, it is >> still annoying that the ports and FreeBSD crews don't take >> the *new* PBI infrastructure that is being pused out with PCBSD-9 >> as an important move. The new PBI infrastructure should be taken >> into the ports system as an important factor. For those who do not >> know it, it give a facility somewhat like the what that APPLE >> applications work. At the potential (not always) for having redundant >> libraries, every PBI package comes with EVERYTHING IT NEEDS. >> there are no 'dependnet packages' as such. On install a >> survey is made so that if anything is found to be truly duplicated >> (different versions of the same library are NOT considered a >> duplicate) >> then they share, but if not then each package installs and ONLY USES >> the stuff that came with it. >> >> The ramifications of this (in this era of large disks) are immense. >> If you unstall all your main applications using PBI, then if you >> screw >> up your ports installed libraries and development environment when >> you >> install some new version of the XXX runtime, *your applications keep >> working*. >> >> A case of "it just works". For the life of me I don't understand WHY > >> there >> is this resistance to taking it into the fold. Especially when all >> the >> work has already been done. It won't replace pkgng and it it won't >> replace >> ports because it actually uses ports to generate the PBI packages. >> But it should be teh default delivery mechanism for binary basic >> packages. >> >> >> As I said.. go run an apple for a while and see what it is supposed >> to be like. >> > >PBIs are a nice thing but.... > >The thing that sucked about PBIs at least in PCBSD 8.x is that the PBIs > >are >too big to download. We may all have big disks but there are many >people around >who don't have fast internet access. E.g. the Firefox PBI is about 100 >MB of size >and there's a new version of it every few days. Add Thunderbird, VLC >and OpenOffice >to that list and your're downloading some GBs per month just to get >your security >holes closed. >What is the situation like in PCBSD 9.x? I heard that it was planned to > >offer >update deltas which should be much smaller. If that's so, I'm all for >PBIs. FYI, in 9 updates are done via deltas, or more specifically bsdiff, which often makes the download file a fraction of the size. The update to a 100mb firefox pbi may only be 5-6 mb, just depends on the size of the changes. -- Kris Moore From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 13:53:13 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7133106564A for ; Fri, 19 Aug 2011 13:53:13 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 665DF8FC08 for ; Fri, 19 Aug 2011 13:53:13 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id A362D4AC1C; Fri, 19 Aug 2011 17:53:11 +0400 (MSD) Date: Fri, 19 Aug 2011 17:53:07 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <425884435.20110819175307@serebryakov.spb.ru> To: Pieter de Goeje In-Reply-To: <201108191401.23083.pieter@degoeje.nl> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 13:53:13 -0000 Hello, Pieter. You wrote 19 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 16:01:23: >> > 3. Kernel features for complex network solutions (netgraph, carp, ipfw= ). >> >=20 >> > The niche for routers & traffic analysis is still ours. It would be >> > nice to take e.g. pfSense and agree with some vendor (Netgear, >> > D-Link, etc) to put on sale hardware with FreeBSD inside. >>=20 >> What about 10G routing? Here are reports about full-bandwidth 10G rout= ing >> on modern Intel NICs with Linux (and multi-core server), but I didn't see >> any such data for FreeBSD, and somebody says, that Intel drivers and >> network stack is not so good parallel in FreeBSD. > With regards to high speed packet forwarding and routing, check out this = work > by Luigi Rizzo: > http://info.iet.unipi.it/~luigi/netmap/ Yep, I know this work, but it is rather special, not out-of-box (ok, maybe with some changed tunables) routing solution with routing tables, some firewall, etc. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 15:27:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 729111065670; Fri, 19 Aug 2011 15:27:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1F2498FC12; Fri, 19 Aug 2011 15:27:08 +0000 (UTC) Received: by yxn22 with SMTP id 22so1539108yxn.13 for ; Fri, 19 Aug 2011 08:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=mqQLm28diuaB9U8LDx42O2/i+wLny1e2OepCp7YUnIk=; b=uo3Ykc8TLi4bMI9srXQbYCgbEmyOqToDRqclZ854X2MW8UcJd7uxZmQwsj7OK8PoUg nvLSZcGvvrGhrKHjhY+c0XFJ4Je1dMwzmIMlqjEoAGd9dMskvccxv6wmwKtdr6HOyV9l DLIOWXoak3xTkzbTQNLcxzRpOgN8qoQ/MCRy8= MIME-Version: 1.0 Received: by 10.150.238.2 with SMTP id l2mr2574603ybh.44.1313767628603; Fri, 19 Aug 2011 08:27:08 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.145.21 with HTTP; Fri, 19 Aug 2011 08:27:08 -0700 (PDT) In-Reply-To: <425884435.20110819175307@serebryakov.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> Date: Fri, 19 Aug 2011 23:27:08 +0800 X-Google-Sender-Auth: 72Y8uXWscSGSEwTPqYi0iE0t6wc Message-ID: From: Adrian Chadd To: Lev Serebryakov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Pieter de Goeje , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 15:27:09 -0000 2011/8/19 Lev Serebryakov : >> With regards to high speed packet forwarding and routing, check out this= work >> by Luigi Rizzo: >> http://info.iet.unipi.it/~luigi/netmap/ > =A0Yep, I know this work, but it is rather special, not out-of-box (ok, > maybe with some changed tunables) routing solution with routing > tables, some firewall, etc. It proves the basic idea. It exposed Luigi (and others, hopefully) to the realities of doing 10ge line rate (small packet!) forwarding. It shows you CAN do it with FreeBSD; someone just needs to take it and run with it. The tools are there to get the basic job done. Adrian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 16:02:45 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EC4F106564A for ; Fri, 19 Aug 2011 16:02:45 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from thyme.infocus-llc.com (server.infocus-llc.com [206.156.254.44]) by mx1.freebsd.org (Postfix) with ESMTP id 042FB8FC0C for ; Fri, 19 Aug 2011 16:02:44 +0000 (UTC) Received: from draco.over-yonder.net (c-174-50-4-38.hsd1.ms.comcast.net [174.50.4.38]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id BBE9F37B59D; Fri, 19 Aug 2011 10:46:39 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 02A3F61C44; Fri, 19 Aug 2011 10:46:39 -0500 (CDT) Date: Fri, 19 Aug 2011 10:46:38 -0500 From: "Matthew D. Fuller" To: Vadim Goncharov Message-ID: <20110819154638.GA75879@over-yonder.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.21-fullermd.4 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems/solutions: voting system & marketing surveys X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 16:02:45 -0000 On Thu, Aug 18, 2011 at 11:50:20PM +0000 I heard the voice of Vadim Goncharov, and lo! it spake thus: > > In other words, Foundation must act as an intermediate party > between those who will work and those who will pay for concrete > tasks set up by e.g. core@ or by user suggestions/need. > > Why it was not done earlier? Actually, it's my understanding that there are significant legal issues which prevent the Foundation from acting in this way (taking targetted donations, acting as an escrow party on specific bounties). -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 17:22:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5170106566B; Fri, 19 Aug 2011 17:22:54 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id A69B58FC1E; Fri, 19 Aug 2011 17:22:54 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QuSmS-000PT3-LZ; Fri, 19 Aug 2011 13:22:52 -0400 Date: Fri, 19 Aug 2011 13:22:52 -0400 From: Gary Palmer To: Lev Serebryakov Message-ID: <20110819172252.GE88904@in-addr.com> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <425884435.20110819175307@serebryakov.spb.ru> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: Pieter de Goeje , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 17:22:54 -0000 On Fri, Aug 19, 2011 at 05:53:07PM +0400, Lev Serebryakov wrote: > Hello, Pieter. > You wrote 19 ??????? 2011 ?., 16:01:23: > > >> > 3. Kernel features for complex network solutions (netgraph, carp, ipfw). > >> > > >> > The niche for routers & traffic analysis is still ours. It would be > >> > nice to take e.g. pfSense and agree with some vendor (Netgear, > >> > D-Link, etc) to put on sale hardware with FreeBSD inside. > >> > >> What about 10G routing? Here are reports about full-bandwidth 10G routing > >> on modern Intel NICs with Linux (and multi-core server), but I didn't see > >> any such data for FreeBSD, and somebody says, that Intel drivers and > >> network stack is not so good parallel in FreeBSD. > > With regards to high speed packet forwarding and routing, check out this work > > by Luigi Rizzo: > > http://info.iet.unipi.it/~luigi/netmap/ > Yep, I know this work, but it is rather special, not out-of-box (ok, > maybe with some changed tunables) routing solution with routing > tables, some firewall, etc. I think we need to refocus on what would give the most benefit. At least as far as I can see: - port/package management: most users would benefit - improved virtualisation support: a lot of users would benefit - improved driver support: a lot of users would benefit - KMS/GEM for better graphics card support: a lot of users would benefit Can you honestly say the same about handling line rate packet forwarding for multiple 10G cards? I suspect I would rank true multipath storage support above 10G routing, for example. Gary From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 18:49:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3678106566B; Fri, 19 Aug 2011 18:49:37 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C8D538FC1F; Fri, 19 Aug 2011 18:49:37 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 3679B46B23; Fri, 19 Aug 2011 14:49:37 -0400 (EDT) Date: Fri, 19 Aug 2011 19:49:37 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Gary Palmer In-Reply-To: <20110819172252.GE88904@in-addr.com> Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Pieter de Goeje , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 18:49:38 -0000 On Fri, 19 Aug 2011, Gary Palmer wrote: > I think we need to refocus on what would give the most benefit. At least > as far as I can see: > > - port/package management: most users would benefit Definitely -- and there's active work going on here that is very exciting indeed. > - improved virtualisation support: a lot of users would benefit This is an area of very active work as well, although I'd like to see more going on. In particular, I'd like to see (a) significantly more mature Xen HVM PV driver support -- and the ability to load them on a GENERIC kernel so we don't break freebsd-update kernel updates by requiring custom kernels, and (b) bHyve to be fleshed out, including support for suspend/resume and migration. Cambridge may be able to help with non-i386/amd64 virtualisation; I have a local TODO item relating to MIPS virtualisation, for example. > - improved driver support: a lot of users would benefit For server/appliance-centric devices, we're going quite well. For consumer devices, less so. However, it's generally the case that things have dramatically improved in the last ten years: companies come to us with drivers now, asking how to get them merged, and frequently their developers get commit bits and maintain them in-tree even. Compare this to 2000 when we had hacked up Intel device drivers, and other than Adaptec, almost no storage vendors closely involved in the project. > - KMS/GEM for better graphics card support: a lot of users would benefit The FreeBSD Foundation recently funded work on precisely this; it hasn't quite made 9.0, but for Intel driver stuff it should make it for 9.1. Maybe Kostik can say more about this. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 20:31:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD939106566B; Fri, 19 Aug 2011 20:31:16 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9C0288FC08; Fri, 19 Aug 2011 20:31:16 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QuVik-000Pfk-TY; Fri, 19 Aug 2011 16:31:14 -0400 Date: Fri, 19 Aug 2011 16:31:14 -0400 From: Gary Palmer To: Robert Watson Message-ID: <20110819203114.GF88904@in-addr.com> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: Pieter de Goeje , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 20:31:16 -0000 On Fri, Aug 19, 2011 at 07:49:37PM +0100, Robert Watson wrote: > >- improved driver support: a lot of users would benefit > > For server/appliance-centric devices, we're going quite well. For consumer > devices, less so. However, it's generally the case that things have > dramatically improved in the last ten years: companies come to us with > drivers now, asking how to get them merged, and frequently their developers > get commit bits and maintain them in-tree even. Compare this to 2000 when > we had hacked up Intel device drivers, and other than Adaptec, almost no > storage vendors closely involved in the project. While on the most part I agree with the fact that a fair number of server chips and chipsets are supported I think we are still missing key components for truly reliable and scalable systems, including SAN multipath support (yes, we have primitive support but it doesn't know about limitations of different systems, e.g. HP EVA 5000s which do a LUN failover if you query the LUN down the wrong path. Yes, the EVA 5000 is an EOL system but its an example of the issues a proper multipath solution should solve. I mean no offense to the authors of the current code either). IPMI boards/interfaces seems to be a constant problem and I'm not entirely sure we have a good handle on SERDES support for NICs in blade systems. Does the project or the foundation make any effort to directly engage with manufacturers to ensure we have robust support for their products? Gary From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 20:35:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F30B61065673; Fri, 19 Aug 2011 20:35:41 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from mail.icecube.wisc.edu (trout.icecube.wisc.edu [128.104.255.119]) by mx1.freebsd.org (Postfix) with ESMTP id AFDBD8FC0A; Fri, 19 Aug 2011 20:35:41 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 17AB1582CC; Fri, 19 Aug 2011 15:18:20 -0500 (CDT) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from mail.icecube.wisc.edu ([127.0.0.1]) by localhost (trout.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id R5e+4iX4uZfZ; Fri, 19 Aug 2011 15:18:20 -0500 (CDT) Received: from wanderer.tachypleus.net (thistle.icecube.wisc.edu [172.16.223.52]) by mail.icecube.wisc.edu (Postfix) with ESMTP id E9F04582CA; Fri, 19 Aug 2011 15:18:19 -0500 (CDT) Message-ID: <4E4EC50B.2010003@freebsd.org> Date: Fri, 19 Aug 2011 15:18:19 -0500 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.18) Gecko/20110624 Thunderbird/3.1.11 MIME-Version: 1.0 To: Robert Watson References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Pieter de Goeje , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 20:35:42 -0000 On 08/19/11 13:49, Robert Watson wrote: > On Fri, 19 Aug 2011, Gary Palmer wrote: > >> I think we need to refocus on what would give the most benefit. At least >> as far as I can see: >> >> - port/package management: most users would benefit > > Definitely -- and there's active work going on here that is very > exciting indeed. > >> - improved virtualisation support: a lot of users would benefit > > This is an area of very active work as well, although I'd like to see > more going on. In particular, I'd like to see (a) significantly more > mature Xen HVM PV driver support -- and the ability to load them on a > GENERIC kernel so we don't break freebsd-update kernel updates by > requiring custom kernels, and (b) bHyve to be fleshed out, including > support for suspend/resume and migration. Cambridge may be able to help > with non-i386/amd64 virtualisation; I have a local TODO item relating to > MIPS virtualisation, for example. On this subject, we've had a lot of luck doing a potentially-paravirtualized GENERIC on powerpc, where you can boot the same GENERIC kernel on real hardware, on IBM's POWER hypervisor (in a project branch) and under Sony's Cell hypervisor on the PS3. Perhaps some of the infrastructure for that can be reused on x86. -Nathan From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 20:43:06 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43DB21065672 for ; Fri, 19 Aug 2011 20:43:06 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm5-vm0.bullet.mail.sp2.yahoo.com (nm5-vm0.bullet.mail.sp2.yahoo.com [98.139.91.204]) by mx1.freebsd.org (Postfix) with SMTP id 1ED918FC14 for ; Fri, 19 Aug 2011 20:43:06 +0000 (UTC) Received: from [98.139.91.66] by nm5.bullet.mail.sp2.yahoo.com with NNFMP; 19 Aug 2011 20:30:07 -0000 Received: from [98.139.91.36] by tm6.bullet.mail.sp2.yahoo.com with NNFMP; 19 Aug 2011 20:30:07 -0000 Received: from [127.0.0.1] by omp1036.mail.sp2.yahoo.com with NNFMP; 19 Aug 2011 20:30:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 396580.76389.bm@omp1036.mail.sp2.yahoo.com Received: (qmail 69851 invoked by uid 60001); 19 Aug 2011 20:30:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1313785807; bh=PkKpCoioXFv/J/37PDOPkClTPmBPYGy1u998fEW257E=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=fZGDRtYK0P5/xvHXkaaTEhjXrOxVY5EZfN7sPZ+TZUya5LbGqSilI60dp7gzyz7Cr5akkCjjHfEncj1j7Gn8iJWl9hCwgB7Nl4LuLGLu+7CHFu3w3jJCzD2/FuBgvidMsZGi73AxKe0xY2fy9je326HeeHX9N+6+FWFm5IiJf9I= X-YMail-OSG: vivH2Y8VM1mr9hlFUMgCH4CNDUKIxF62.UcK_QEY_O783iJ 0OGioJAqKGD1M2Ly0O06AC0jmBIHMy_eKqPHP_grtvrAXUe7AoIqJ0bguCDZ HlXW3yH7K.hNH8YeB62Ty4dzPhLGy7fqsc3Q9wDszn6Y6nN3ueacvb2Bn3B9 F8yAWqvXCtHMMfQN8oDZ9sH9QPZQZnZRwReOpoWvm6zdoRcMchyM5V3sPEtc 9qyS1A1YE8WijH0j8BlhGTPPyLphHzHwRqkN_nIuS_1j.Q6kFlsnEdl9Y_gh 32wa6JrET6P_KFBt7jyLrIqvCobJo8u1cHOigUErh4Bf_mkc58kGL4frEzI6 vCJzL_PV167c1yvYe1VSsXhdnvsc9k_qF0vkM7cBrDMdmQRYg9.kDFfpVyaT RbcoFjaDul0Jov_TvLXHMYQvUCkPGydf_sl9hofqCGwc9dAmo5kHp2GZxcvR hFdFAejak74fe.XTGdI4CcsBlgAbzysMEgGI.zbQQ67Q- Received: from [190.157.142.22] by web113503.mail.gq1.yahoo.com via HTTP; Fri, 19 Aug 2011 13:30:06 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.4 YahooMailWebService/0.8.113.313619 Message-ID: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> Date: Fri, 19 Aug 2011 13:30:06 -0700 (PDT) From: "Pedro F. Giffuni" To: freebsd-arch@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 20:43:06 -0000 First of all, thanks to Vadim for his initial post: it's=0Aevident he spend= a lot of time preparing it.=0A=0AI just want to give my 0.02$ to this disc= ussion:=0A=0AFreeBSD 9.0 is a huge step in the right direction in many=0Awa= ys. Perhaps what I like most is getting done with the=0Alast GNU toolchain = and ZFS updates and maintaining the well=0Aknown stability and performance = along with the new features.=0A=0AUnfortunately some necessary updates didn= 't make it in time,=0Ain particular pkgng and the X.Org drivers. The new in= staller=0Ais promising but lately installing and configuring XOrg has=0Abec= ome a nightmare so I find myself suggesting people to use=0APC-BSD instead.= =0A=0ASome rather important changes that seem critical for the future=0Alik= e ARM-EABI and a new toolchain seem to be lacking developers=0Aand perhaps = are going on too slowly to expect them to work in=0Athe near future but, al= l in all, I should mention that FreeBSD=0Ais still very influential and com= petitive. I've seen posts not=0Atoo long ago where linux developers praise = FreeBSD for keeping=0Athe development pace despite having a much smaller gr= oup.=0A=0AWhat I think is that perhaps FreeBSD shouldn=B4t be expecting=0At= o be a better linux than linux, simply because there is no=0Acompany relate= d to any BSD in capacity to compete with, say,=0ARedhat. We still can do pr= etty much everything linux does=0Awith a little extras, but we are a actual= ly niche market and=0Ait actually hasn't worked bad at all for us. =0A=0ACo= ncerning BSD sites moving to linux to reduce costs, as a=0Along time Yahoo = user I have to say that doesn't seem to be=0Agoing very well for them. I am= pretty sure such a process=0Ais painful for them as it is for the end user= .=0A=0AIMHO, we need to focus on the two fields where people expect=0AFreeB= SD to shine:=0A=0A- Servers: performance has always been our strong point= =0A=A0 but perhaps we should be focusing on ease of use. Ohh...=0A=A0 BTW, = jails are awesome!=0A=0A- Embedded devices: FreeBSD's license is key here.= =0A=0AI don't think the FreeBSD Foundation has been clueless at all=0Athoug= h, coordinating efforts in a voluntary project like this=0A*is* tough. Some= people complain because we release too fast=0Abut some also complain becau= se the release cycle is too long.=0AWe can't satisfy everyone and developme= nt just goes on.=0A=0APedro.=0A From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 21:42:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88FF91065670; Fri, 19 Aug 2011 21:42:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id DD4CB8FC0A; Fri, 19 Aug 2011 21:42:56 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p7JLQQfi061862 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Aug 2011 00:26:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p7JLQQQK043396; Sat, 20 Aug 2011 00:26:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p7JLQQLN043395; Sat, 20 Aug 2011 00:26:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Aug 2011 00:26:26 +0300 From: Kostik Belousov To: Robert Watson Message-ID: <20110819212626.GC17489@deviant.kiev.zoral.com.ua> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lWojlNcppt5iyI2E" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Pieter de Goeje , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 21:42:57 -0000 --lWojlNcppt5iyI2E Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 19, 2011 at 07:49:37PM +0100, Robert Watson wrote: > On Fri, 19 Aug 2011, Gary Palmer wrote: > >- KMS/GEM for better graphics card support: a lot of users would benefit >=20 > The FreeBSD Foundation recently funded work on precisely this; it hasn't= =20 > quite made 9.0, but for Intel driver stuff it should make it for 9.1. =20 > Maybe Kostik can say more about this. I am very sorry for ever answer in this thread at all. Yes, my expectation is that the driver will be in 9.1. For now, you need a published patch and the xorg-dev ports tree. Putting all the stuff into the repositories without inducing a pain both on users and me is a hard work. I am esp. feel for kwm who, I believe, leads the xorg-dev ports efforts. --lWojlNcppt5iyI2E Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk5O1QIACgkQC3+MBN1Mb4heAACdFPqjKrRGS1tVi/MIUBTIdx1G fFAAn2pzJ8w+u7FnePbVcUmNfPVizb84 =XxAQ -----END PGP SIGNATURE----- --lWojlNcppt5iyI2E-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 22:57:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0742106564A; Fri, 19 Aug 2011 22:57:49 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C9DA78FC17; Fri, 19 Aug 2011 22:57:49 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 654AD46B2A; Fri, 19 Aug 2011 18:57:49 -0400 (EDT) Date: Fri, 19 Aug 2011 23:57:49 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Gary Palmer In-Reply-To: <20110819203114.GF88904@in-addr.com> Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> <20110819203114.GF88904@in-addr.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Pieter de Goeje , Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 22:57:50 -0000 On Fri, 19 Aug 2011, Gary Palmer wrote: >> For server/appliance-centric devices, we're going quite well. For consumer >> devices, less so. However, it's generally the case that things have >> dramatically improved in the last ten years: companies come to us with >> drivers now, asking how to get them merged, and frequently their developers >> get commit bits and maintain them in-tree even. Compare this to 2000 when >> we had hacked up Intel device drivers, and other than Adaptec, almost no >> storage vendors closely involved in the project. > > While on the most part I agree with the fact that a fair number of server > chips and chipsets are supported I think we are still missing key components > for truly reliable and scalable systems, including SAN multipath support > (yes, we have primitive support but it doesn't know about limitations of > different systems, e.g. HP EVA 5000s which do a LUN failover if you query > the LUN down the wrong path. Yes, the EVA 5000 is an EOL system but its an > example of the issues a proper multipath solution should solve. I mean no > offense to the authors of the current code either). IPMI boards/interfaces > seems to be a constant problem and I'm not entirely sure we have a good > handle on SERDES support for NICs in blade systems. > > Does the project or the foundation make any effort to directly engage with > manufacturers to ensure we have robust support for their products? Those specific devices are well outside my area of expertise, so I'll have to leave those to others to address. Let me turn it around, however: have you approached the Foundation to bring weak support there to the board's attention? As far as I'm aware, no one has ever mentioned this to board@ before, and it's something the board would surely work on actively if we realised it was a problem. More generally, yes, we do chat with hardware vendors regularly, and have attempted to organise a number of meetings on behalf of the FreeBSD Project when we're aware of gaps. We also try to help vendors figure out how to get their drivers back into FreeBSD. However, we can only be effective advocates for the project when we're aware of gaps, and we rely on FreeBSD developers and consumers to help us figure out what they are. (We also try to meet with companies that use FreeBSD in their products to help them figure out how to interact with the project better -- so if you're aware of such companies, perhaps one that needs help working out what to do about support for the storage systems you have in mind, do drop us an e-mail!) Robert From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 23:07:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADC37106564A for ; Fri, 19 Aug 2011 23:07:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6A9FF8FC14 for ; Fri, 19 Aug 2011 23:07:14 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AooGALfkTk6DaFvO/2dsb2JhbABBhEuUJpAUgUABAQQBI1YFFhgCAg0ZAlmICqc2kS6BLIQMgRAEkxOREQ X-IronPort-AV: E=Sophos;i="4.68,253,1312171200"; d="scan'208";a="134979033" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 19 Aug 2011 18:38:35 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 6C929B3F24; Fri, 19 Aug 2011 18:38:35 -0400 (EDT) Date: Fri, 19 Aug 2011 18:38:35 -0400 (EDT) From: Rick Macklem To: giffunip@tutopia.com Message-ID: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1313785806.56747.YahooMailClassic@web113503.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 23:07:14 -0000 Pedro F. Giffuni wrote: > First of all, thanks to Vadim for his initial post: it's > evident he spend a lot of time preparing it. >=20 Yes, I'll second that. Being fairly new to FreeBSD, I've found the discussion interesting. One thing I thought I'd bring up (since I haven't seen it mentioned yet) is Debian GNU/kFreeBSD. I haven't tried it, so I'm talking through my hat a bunch, but... It seems to me that FreeBSD should do what it can to support this effort. Why? Well, I suspect a lot of why organizations running servers might choose Linux over FreeBSD has to do with the distros/applications and not the kernels. If they like the style of a typical Linux distro and don't mind the GPL, this could be a nice alternative for them. In saying this, I don't mean to belittle the good work being done on ports and packages and it sounds like more good things are coming down the pipe. It's just that someone that is familiar with a typical Linux distro may find Debian GNU/kFreeBSD less of a transition for them. (And lets face it, everyone is biased towards what they know vs what they don't know.) > I just want to give my 0.02$ to this discussion: >=20 > FreeBSD 9.0 is a huge step in the right direction in many > ways. Perhaps what I like most is getting done with the > last GNU toolchain and ZFS updates and maintaining the well > known stability and performance along with the new features. >=20 > Unfortunately some necessary updates didn't make it in time, > in particular pkgng and the X.Org drivers. The new installer > is promising but lately installing and configuring XOrg has > become a nightmare so I find myself suggesting people to use > PC-BSD instead. >=20 > Some rather important changes that seem critical for the future > like ARM-EABI and a new toolchain seem to be lacking developers > and perhaps are going on too slowly to expect them to work in > the near future but, all in all, I should mention that FreeBSD > is still very influential and competitive. I've seen posts not > too long ago where linux developers praise FreeBSD for keeping > the development pace despite having a much smaller group. >=20 > What I think is that perhaps FreeBSD shouldn=C2=B4t be expecting > to be a better linux than linux, simply because there is no > company related to any BSD in capacity to compete with, say, > Redhat. We still can do pretty much everything linux does > with a little extras, but we are a actually niche market and > it actually hasn't worked bad at all for us. >=20 One of the reasons I brought up GNU/kFreeBSD is that I think it would be nice if folks could easily compare kernels without having to deal with all the userland differences. However, I have no idea how similar GNU/kFreeBSD can come to their Linux distro and whether it will allow users to switch between them easily? Just my $0.00 (not worth 2 cents), rick From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 01:09:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C54A21065670 for ; Sat, 20 Aug 2011 01:09:59 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from nm7-vm2.bullet.mail.ne1.yahoo.com (nm7-vm2.bullet.mail.ne1.yahoo.com [98.138.90.155]) by mx1.freebsd.org (Postfix) with SMTP id 6F2578FC0C for ; Sat, 20 Aug 2011 01:09:59 +0000 (UTC) Received: from [98.138.90.52] by nm7.bullet.mail.ne1.yahoo.com with NNFMP; 20 Aug 2011 00:57:32 -0000 Received: from [98.138.89.234] by tm5.bullet.mail.ne1.yahoo.com with NNFMP; 20 Aug 2011 00:57:32 -0000 Received: from [127.0.0.1] by omp1049.mail.ne1.yahoo.com with NNFMP; 20 Aug 2011 00:57:32 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 151134.9414.bm@omp1049.mail.ne1.yahoo.com Received: (qmail 3706 invoked by uid 60001); 20 Aug 2011 00:57:31 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1313801851; bh=C2kIj1Jgs0JcFsa4IeRJsvzZGQuajANyJeRw/95EAxo=; h=X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=TAp4rBQXVvTsE3EcR4eZJl2ORWMbpUu35h7uHbBAvUzVxxs6+IE33vAUZXfrcbZ+hHBMWcLP9R60GukUhM6ylBGUEtqTyj8pxp0BgIBGA3lbCjoTAWi02c8UVuU4YbzSIz9IKToBiglcy2zhcZhBXngtIYUXm2a6RsUnZVxFbgo= X-YMail-OSG: LG1VtY4VM1nFD6ZE_BmvusOc.DE8hJCHYpDO0ZZ.tXAQHQF va_o8IT04Gp_3iz3uSVhYuRfWBwGaD7HOG_gbHS.yTkcljJxZj6OWexaLEM2 faqY2Ws5ZM.B3uCuPclUANZvtVsfj9AdF89Mk8NhC4y0fs3OCk83CcSXqBXD gU7UjjofIFeYmq8iT9fwLWIJ36uX5bYiWzVh9X7.xFsR48P3iDIB4wdEXZ9z JllZFt_JhxL0LBEkoW0YlEHgZZyywbwcwcwJ.9_OepNCrVpK2aBity9K8wof OecIOQWWO8ikwCeb6tr0dL3qVtZApEzPod1xbyZ_KEB0crp0OZ5vARRLy8ws FD.lJ_4aslodYc8sIqT115Dc15Uw9aCI2dVn508E3j4d3kLBEtCKlg3z.mAf 33PeJEdzZbKeo0L4NQBRc.PytMlz2MByJtY_SCuNApyUbqE7Lym_RFvPHHa6 aa7H4ges3wUbxVmFiJyh73QYRIyw83FGURMIEMSv2oHSJfjtfhFDZ_LKZB7F uZfOqaCdOUIZtShCb5Ih.udiUhxqJRclTCbmpLn2di3Zll1KsRdEsZ87zVUI IHWp2Ec__Ni0wBfu3zN4CpSErSc4KWCFDY9k7VNaIqNrM2rTv2RLSPoF7Y1A bQnZQ1HvoYUQdfITXFg64fjdfURIhrHQl8avG16extXXph8F1V4pE1leG5CC UetclggB0dfqEpbvaqTocDJRZqztq1pkYLAZA5POV.rc_qJ3AIiactXmJkw- - Received: from [190.157.142.22] by web113505.mail.gq1.yahoo.com via HTTP; Fri, 19 Aug 2011 17:57:31 PDT X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/14.0.4 YahooMailWebService/0.8.113.313619 Message-ID: <1313801851.3688.YahooMailClassic@web113505.mail.gq1.yahoo.com> Date: Fri, 19 Aug 2011 17:57:31 -0700 (PDT) From: "Pedro F. Giffuni" To: Rick Macklem In-Reply-To: <175270279.98941.1313793515391.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-arch@freebsd.org Subject: Working with Debian GNU/kFreeBSD (was Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 01:09:59 -0000 --- On Fri, 8/19/11, Rick Macklem wrote: ... > > One thing I thought I'd bring up (since I haven't seen it > mentioned yet) is Debian GNU/kFreeBSD. I haven't tried it, > so I'm talking through my hat a bunch, but... > Oh, yes ... the "others" ... ;). This kFreeBSD vs. Linux comparison is very interesting: http://www.phoronix.com/scan.php?page=article&item=debian_kfreebsd_h210&num=1 They seem to be doing pretty well but they still have some issues that are important for some debian packages: - No alsa - Still using the older userland linux threads. - Systemd is becoming popular but is linux specific. Having the kernel X.Org stuff would help them too. The guys there are actually eager to work with us but they don't know well the ways around our standards and development process. For some stuff I was working on, Robert Millan pointed me to these patches for system headers: http://anonscm.debian.org/viewvc/glibc-bsd/trunk/kfreebsd-kernel-headers/debian/patches/ They require a lot of cleaning but it would be a good starting point. Another possibility could be to work together on a launchd port, but that's a completely different topic and I am not sure there's interest. cheers, Pedro. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 06:15:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A59DE106564A; Sat, 20 Aug 2011 06:15:14 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 42C7C8FC0C; Sat, 20 Aug 2011 06:15:14 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 486E44AC1C; Sat, 20 Aug 2011 10:15:12 +0400 (MSD) Date: Sat, 20 Aug 2011 10:15:06 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <368496955.20110820101506@serebryakov.spb.ru> To: Gary Palmer In-Reply-To: <20110819172252.GE88904@in-addr.com> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 06:15:14 -0000 Hello, Gary. You wrote 19 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 21:22:52: >> >> > The niche for routers & traffic analysis is still ours. It would= be >> >> > nice to take e.g. pfSense and agree with some vendor (Netgear, >> >> > D-Link, etc) to put on sale hardware with FreeBSD inside. >> >>=20 >> >> What about 10G routing? Here are reports about full-bandwidth 10G r= outing >> >> on modern Intel NICs with Linux (and multi-core server), but I didn't= see >> >> any such data for FreeBSD, and somebody says, that Intel drivers and >> >> network stack is not so good parallel in FreeBSD. >> > With regards to high speed packet forwarding and routing, check out th= is work >> > by Luigi Rizzo: >> > http://info.iet.unipi.it/~luigi/netmap/ >> Yep, I know this work, but it is rather special, not out-of-box (ok, >> maybe with some changed tunables) routing solution with routing >> tables, some firewall, etc. > I think we need to refocus on what would give the most benefit. At least > as far as I can see: > - port/package management: most users would benefit > - improved virtualisation support: a lot of users would benefit > - improved driver support: a lot of users would benefit > - KMS/GEM for better graphics card support: a lot of users would benefit > Can you honestly say the same about handling line rate packet forwarding > for multiple 10G cards? I agree with you. I've not say, that 10G routing is very important for many users. My comment about 10G was answer to statement, that "The niche for routers & traffic analysis is still ours.". I wanted to say, that it is so may be now, but not for long. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 11:38:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38E98106566B; Sat, 20 Aug 2011 11:38:27 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0DC048FC16; Sat, 20 Aug 2011 11:38:27 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id A825146B06; Sat, 20 Aug 2011 07:38:26 -0400 (EDT) Date: Sat, 20 Aug 2011 12:38:26 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Lev Serebryakov In-Reply-To: <368496955.20110820101506@serebryakov.spb.ru> Message-ID: References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> <368496955.20110820101506@serebryakov.spb.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 11:38:27 -0000 On Sat, 20 Aug 2011, Lev Serebryakov wrote: >> Can you honestly say the same about handling line rate packet forwarding >> for multiple 10G cards? > > I agree with you. I've not say, that 10G routing is very important for many > users. My comment about 10G was answer to statement, that "The niche for > routers & traffic analysis is still ours.". I wanted to say, that it is so > may be now, but not for long. Part of the key here will be reworking things like ipfw(4) and pf(4) to scale better than they do currently. For pf(4), it's particularly important that we align hardware work distribution via RSS with state management for TCP connections. I've been working on this for the base system TCP implementation over the last few years, and got most of it into 9.x (but not the actual RSS driver interface, as I wasn't convinced it was a stable KPI in the form I prototyped it in). Post-9.0, I'll try to get the RSS KPI cleaned up so that we can merge it and get our device drivers updated. There's also a related work-in-progress I have that teaches the network stack how to program NIC filters, usually implemented as TCAMs (Chelsio) or hardware hash tables (Solarflare) about network stack connection affinity. My plan is to work on making this substantially more real once the RSS patches are in. (Those are, themselves, fairly minor: we have connection groups already in 9.0, and the RSS changes simply cause existing software-side hash tables to align with hardware-side hashing: the tricky bit is a sustainable KPI for device driver writers). These are closely related to the issue of userspace networking, which Luigi is starting to explore with netmap. Ideally, you could use the same NIC for both kernel network stack stuff and userspace applications, using hardware filters to decide whether individual packets go to a descriptor ring in the kernel or userspace. Solarflare's Open Onload is an interesting potential model there, although perhaps not the exact model we want (they rely on shared network stacks between kernel and userspace, and for most of our purposes, less sharing is not only sufficient, but perhaps better). Robert From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 13:43:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ECA831065670; Sat, 20 Aug 2011 13:43:17 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 7922F8FC0A; Sat, 20 Aug 2011 13:43:17 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 38CAB7300A; Sat, 20 Aug 2011 15:45:30 +0200 (CEST) Date: Sat, 20 Aug 2011 15:45:30 +0200 From: Luigi Rizzo To: Robert Watson Message-ID: <20110820134530.GA42942@onelab2.iet.unipi.it> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> <368496955.20110820101506@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 13:43:18 -0000 On Sat, Aug 20, 2011 at 12:38:26PM +0100, Robert Watson wrote: > > On Sat, 20 Aug 2011, Lev Serebryakov wrote: > > >>Can you honestly say the same about handling line rate packet forwarding > >>for multiple 10G cards? > > > >I agree with you. I've not say, that 10G routing is very important for > >many users. My comment about 10G was answer to statement, that "The niche > >for routers & traffic analysis is still ours.". I wanted to say, that it > >is so may be now, but not for long. > > Part of the key here will be reworking things like ipfw(4) and pf(4) to > scale better than they do currently. For pf(4), it's particularly ... > These are closely related to the issue of userspace networking, which Luigi > is starting to explore with netmap. Ideally, you could use the same NIC > for both kernel network stack stuff and userspace applications, using > hardware filters to decide whether individual packets go to a descriptor > ring in the kernel or userspace. Solarflare's Open Onload is an ... Thanks to Robert for changing the subject (because i believe that 10G operation is at the bottom of the list of issues that Vadim brought up). Regarding netmap i wanted to mention that, since the announce at the beginning of june, we now have a lot more stuff: - an initial libpcap library, so a number of apps can run at much higher speed; - OpenvSwitch support, which mean that you can do userspace bridging much faster than - the Click modular router now runs (in userspace) at up to 4Mpps per core, which is faster than in-kernel linux; A userspace version of ipfw should be available in a short time, and i have some work in progress to bring the forwarding tables in userspace (but of course you can do the same with Click). I also see people start using it, which is a good thing because i am getting useful feedback on features and bugs and patches for more device drivers. More (including a recently posted GoogleTechTalk) at http://info.iet.unipi.it/~luigi/netmap/ http://www.youtube.com/watch?v=SPtoXNW9yEQ I still think that it would have been nice (especially to compare FreeBSD to Linux) to have netmap into 9.0, as it would have given us the lead for sw packet processing solutions. I understand that the timing of the netmap release was unfortunate (due to the impending code freeze and summer and holidays), but probably we could have given it a chance, since the code does not make a single change to the kernel code except for device drivers, and even those are small and #ifdef'ed out if you don't want a netmap-enabled kernel. Let's hope we find a way to import it into RELENG_9 and i will do my best to distribute patches compatible with recent OS versions. On the general issue of improving performance of the network stack, I feel that to achieve significant speed improvements we should really reconsider the way things are done in the network stack. And that comes before support for special HW features. In netmap at least, a large performance improvement came from getting rid of mbufs. Per-packet allocation and deallocation are a huge cost, and really an unnecessary one if the consumer of the packet can do the processing inline instead of storing the packet and then work on it a week after. Think for instance of TCP acks, which could really be processed inline. Same goes for firewalled traffic. For high speed TCP (i.e. sessions trying to stream data) we have a lot of issues, two of which are below: - we still have linear lists of buffers, which means that the cost of out-of-order incoming segments is O(N) (with N large at 1..10Gbps). Fixing that is way more important than improving the locking. - on the outgoing side, the code makes no assumption on what happens on the MTU and incoming acks, so every transmission recomputes the boundaries of the segment to be sent. Never mind that in the real world the MTU is normally stable, and it would be a lot more efficient to store (in the socket buffer) and manage (in the stack) data as an array of MTU-sized buffers, optimize the fast path for that, and trap to a slowpath if something changes. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 14:21:36 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0669210656D8; Sat, 20 Aug 2011 14:21:36 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D75FE8FC12; Sat, 20 Aug 2011 14:21:35 +0000 (UTC) Received: from [192.168.2.124] (host81-151-180-177.range81-151.btcentralplus.com [81.151.180.177]) by cyrus.watson.org (Postfix) with ESMTPSA id ABBC646B0A; Sat, 20 Aug 2011 10:21:34 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <5299.1313849459@critter.freebsd.dk> Date: Sat, 20 Aug 2011 15:21:33 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5299.1313849459@critter.freebsd.dk> To: "Poul-Henning Kamp" X-Mailer: Apple Mail (2.1084) Cc: Lev Serebryakov , freebsd-arch@FreeBSD.org Subject: Re: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 14:21:36 -0000 On 20 Aug 2011, at 15:10, Poul-Henning Kamp wrote: > In message , = Robert Watso > n writes: >=20 >> Part of the key here will be reworking things like ipfw(4)=20 >=20 > Here is how to do it: >=20 > Compile IPFW rules to C-code, compile C-code to KLD, load KLD and hook > the firewall rules. >=20 > If the C-code is designed smartly, the C-compiler can optimize to > insanely efficient code. >=20 > The same semantics as today can be preserved with respect to counters > and dynamic addition/removal of rules, with a little bit of creative > thinking about data structures. >=20 > Somebody[tm] did that long ago, but never contributed the patches back > once The Mgt[tm] realized what performance we were talking about. I'm actually slightly less concerned about this aspect of it, although = some sort of JIT/etc, perhaps grounded in LLVM, would make sense. I'm = more concerned with the management of firewall state in the presence of = multiple network queues and SMP. We should be able to build = substantially on the approaches we've been using higher in the network = stack to align NIC-level work distribution with network stack processing = and application process affinity. (These ideas are still coming to = maturity, but there's useful stuff to be found there.) Robert= From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 14:27:40 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B44E106564A; Sat, 20 Aug 2011 14:27:40 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1C1BC8FC13; Sat, 20 Aug 2011 14:27:39 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 15A5E5E44; Sat, 20 Aug 2011 14:11:00 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id p7KEAxWA005300; Sat, 20 Aug 2011 14:10:59 GMT (envelope-from phk@phk.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 20 Aug 2011 12:38:26 +0100." Content-Type: text/plain; charset=ISO-8859-1 Date: Sat, 20 Aug 2011 14:10:59 +0000 Message-ID: <5299.1313849459@critter.freebsd.dk> Cc: Lev Serebryakov , freebsd-arch@FreeBSD.org Subject: Re: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 14:27:40 -0000 In message , Robert Watso n writes: >Part of the key here will be reworking things like ipfw(4) Here is how to do it: Compile IPFW rules to C-code, compile C-code to KLD, load KLD and hook the firewall rules. If the C-code is designed smartly, the C-compiler can optimize to insanely efficient code. The same semantics as today can be preserved with respect to counters and dynamic addition/removal of rules, with a little bit of creative thinking about data structures. Somebody[tm] did that long ago, but never contributed the patches back once The Mgt[tm] realized what performance we were talking about. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 21:10:08 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27815106564A for ; Sat, 20 Aug 2011 21:10:08 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id AC29F8FC12 for ; Sat, 20 Aug 2011 21:10:07 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:248b:b3cd:918d:d0d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id EF9FE4AC58; Sun, 21 Aug 2011 01:10:05 +0400 (MSD) Date: Sun, 21 Aug 2011 01:10:05 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1361908410.20110821011005@serebryakov.spb.ru> To: Luigi Rizzo In-Reply-To: <20110820134530.GA42942@onelab2.iet.unipi.it> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> <368496955.20110820101506@serebryakov.spb.ru> <20110820134530.GA42942@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 21:10:08 -0000 Hello, Luigi. You wrote 20 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 17:45:30: > - the Click modular router now runs (in userspace) at up to 4Mpps > per core, which is faster than in-kernel linux; > A userspace version of ipfw should be available in a short time, > and i have some work in progress to bring the forwarding tables > in userspace (but of course you can do the same with Click). > I also see people start using it, which is a good thing because > i am getting useful feedback on features and bugs and patches > for more device drivers. [SKIPPED] > On the general issue of improving performance of the network stack, > I feel that to achieve significant speed improvements we should > really reconsider the way things are done in the network stack.=20 > And that comes before support for special HW features.=20 Could you please explain (I don't mean, that you are wrong, I really don't understand), how netmap and other user-level processing could help for ROUTING (with firewalling, different routes, etc) and software switching? I understand very well, why this help user-level applications, which need to process huge PPS rates. Less memcpy, less allocations, less context switches (and TLB/cache flushes) -- all these things is very clear to me. But why user-level software swithcing is faster than in-kernel one, hwcih should wotk without memory context switches AT ALL?! Or netmap is used for prototyping code, which will be moved into kernel later? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Sat Aug 20 21:37:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8388E106564A; Sat, 20 Aug 2011 21:37:07 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2B84F8FC0A; Sat, 20 Aug 2011 21:37:06 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 1A0217300A; Sat, 20 Aug 2011 23:55:03 +0200 (CEST) Date: Sat, 20 Aug 2011 23:55:03 +0200 From: Luigi Rizzo To: Lev Serebryakov Message-ID: <20110820215503.GA45984@onelab2.iet.unipi.it> References: <810527321.20110819123700@serebryakov.spb.ru> <201108191401.23083.pieter@degoeje.nl> <425884435.20110819175307@serebryakov.spb.ru> <20110819172252.GE88904@in-addr.com> <368496955.20110820101506@serebryakov.spb.ru> <20110820134530.GA42942@onelab2.iet.unipi.it> <1361908410.20110821011005@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1361908410.20110821011005@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: 10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 21:37:07 -0000 On Sun, Aug 21, 2011 at 01:10:05AM +0400, Lev Serebryakov wrote: > Hello, Luigi. > You wrote 20 ??????? 2011 ?., 17:45:30: > > > - the Click modular router now runs (in userspace) at up to 4Mpps > > per core, which is faster than in-kernel linux; > > A userspace version of ipfw should be available in a short time, > > and i have some work in progress to bring the forwarding tables > > in userspace (but of course you can do the same with Click). > > I also see people start using it, which is a good thing because > > i am getting useful feedback on features and bugs and patches > > for more device drivers. > [SKIPPED] > > On the general issue of improving performance of the network stack, > > I feel that to achieve significant speed improvements we should > > really reconsider the way things are done in the network stack. > > And that comes before support for special HW features. > Could you please explain (I don't mean, that you are wrong, I really > don't understand), how netmap and other user-level processing could > help for ROUTING (with firewalling, different routes, etc) and > software switching? I understand very well, why this help user-level i am working on the following now: - routing daemons and the like still work as usual, adding and modifying routes with the standard mechanisms (routing sockets etc.) - the kernel updates its own forwarding tables (FIB) as usual But: - a netmap client (userspace) listens for FIB updates on a routing socket, and builds its own copy of the FIB in userspace (call it uFIB) - the same process sets interfaces in netmap mode, and uses the uFIB to do forwarding, injecting back into the kernel those packets that have a local destination. > applications, which need to process huge PPS rates. Less memcpy, less > allocations, less context switches (and TLB/cache flushes) -- all > these things is very clear to me. But why user-level software > swithcing is faster than in-kernel one, hwcih should wotk without > memory context switches AT ALL?! essentially, the driver in netmap mode is way more efficient and this offsets the cost of the few syscalls. As an example, currently with netmap one core can forward packets between interfaces at a rate between 3 and 10 Mpps depending on the amount of processing on the packet, and there are significant optimizations that are still possible especially at the lower speeds (if 3 Mpps can be called so) > Or netmap is used for prototyping code, which will be moved into > kernel later? Nothing prevents, of course, that kernel subsystems use the interface directly in netmap mode. But i think that now that we have the option, it makes sense to spend some time to experiment with newer solutions (FIB data structures, firewalls, memory aligment, possibly even tcp buffer management) in userspace and then move stuff back into the kernel once we have a good solution. i am using it for prototyping and testing subsystems in userspace, whether it makes sense to move them depends on the performance we manage to get. cheers luigi