From owner-freebsd-advocacy@FreeBSD.ORG Tue Jan 13 23:14:45 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02496B57 for ; Tue, 13 Jan 2015 23:14:45 +0000 (UTC) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA8D6C10 for ; Tue, 13 Jan 2015 23:14:44 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id gq1so5253573obb.12 for ; Tue, 13 Jan 2015 15:14:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=je1muvQEuzwnGucXavb3cXS1pjgkFrb5DTYkgCucwdM=; b=GSzn+2kJz/8Ch6ibnK/4Xplt/vj4A3qsbTCbgL/SfXSSplMwEIMKpwsvTgYmAJI41M nivLipnIHzIZkJFY2NOo7Knu8bSlbt9O1gWJN3Ju6jrnjMRvQg/ZZMGaD/7nzkngzgqT CNDqo1n00l0WbSA70n1YOVHhnyIGmY0DihPofdz68rPum1J/OeFCcr4P4zNHZ41mLa4Y RlMlZkibmmvxESzKAV26mo1MpSKa1/uuXq1aUyHkG0yA3C8tR8I1JXicQyhgKFpgXsgM wt/J0h3iTE9bMIprw1vZ5Uo0oZ0a3YdHwGy+1Nq9w9JhrqzdClbChKqdcczx3UWtvCyo 1B0A== X-Received: by 10.202.72.151 with SMTP id v145mr525880oia.122.1421190884178; Tue, 13 Jan 2015 15:14:44 -0800 (PST) MIME-Version: 1.0 Sender: royce.williams@gmail.com Received: by 10.202.89.132 with HTTP; Tue, 13 Jan 2015 15:14:24 -0800 (PST) From: Royce Williams Date: Tue, 13 Jan 2015 14:14:24 -0900 X-Google-Sender-Auth: T16T1fk7qWTmVMLb5ITZVfhauBE Message-ID: Subject: projects to better support FreeBSD sysadmins To: "freebsd-advocacy@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 23:14:45 -0000 At Craig Rodrigues' request, I'm starting a new thread here branched from a freebsd-ports@ thread. For those who want more context, the original thread starts here: https://lists.freebsd.org/pipermail/freebsd-ports/2015-January/097462.html It was initially about BIND REPLACE_BASE, but branched off into general sysadmin concerns that Craig wanted to respond to. Royce ---------- Forwarded message ---------- From: Royce Williams Date: Mon, Jan 12, 2015 at 7:10 AM Subject: Re: BIND REPLACE_BASE option To: ports Cc: Deb Goodkin On Mon, Jan 12, 2015 at 4:08 AM, Kurt Jaeger wrote: >> No disputing that, just thinking, is FreeBSD being driven by user need, >> financial contributer need, developer need, security need, making things >> 'better' or just by people wanting to make their mark in a warped sense >> of "it'll all get better"...? > > Probably by developer *capacity* (not need) and fire-fighting, > like most IT stuff 8-( But like most IT stuff, resources are being asymmetrically applied to the root causes of the fires. Read the list of projects from last quarter: - Address Space Layout Randomization (ASLR) - amd64 Xen Paravirtualization - bhyve - Chelsio iSCSI Offload Support - Debian GNU/kFreeBSD - FreeBSD Preseed Installation (PXE) - Jenkins Continuous Integration for FreeBSD - New Automounter - QEMU bsd-user-Enabled Ports Building - VMWare VAAI and Microsoft ODX Acceleration in CTL - ZFSguru - Intel GPU Driver Update - SDIO Driver - UEFI Boot - Updated vt(4) System Console - Updating OpenCrypto - FreeBSD on Newer ARM Boards - FreeBSD/arm64 - LLDB Debugger Port - LLVM Address Sanitizer (Asan) - SSE Variants of libc Routines for amd64 - FreeBSD Python Ports - GNOME/FreeBSD - KDE on FreeBSD - The Graphics Stack on FreeBSD - Xfce The Foundation section also lists these items not overlapping with the above: - FreeBSD Journal - PostgreSQL performance improvements - Ongoing release process - Development snapshots - VM images for releases - Secure Boot planning - Infrastructure hardware - Java licensing - Summits and summit sponsorship - Travel grants, tutorials, and talks - New Design and Implementation book - Recruitment flyers Are there long-term improvement projects that aren't being listed? If so, they should be. At face value, the main project list is heavily weighted towards relatively esoteric OS features. The Foundation list is heavily weighted towards advocacy and communication (as it should be). What is missing are high-level projects to help sysadmins maintain and use FreeBSD on an ongoing basis. Here are some projects that would help to close the sysadmin gap: - Automatic error reporting and analysis - OS and port debugging tools for sysadmins - Independent project-wide usability analysis - Ports dependency isolation and reduction framework - Ports system reliability parity with Linuxes - Searchable, taggable project FAQ - Searchable hardware support matrix integrated with bug tracker - Wiki curation and platform improvements These projects decentralize and improve support for sysadmins and new adopters. As a business case for the Foundation, these projects should also deeply free up developer resources to focus on other major projects. In the past, when I have pointed out this "sysadmin gap", I receive one of two answers: 1. Sounds great. Let us know when you have it finished. 2. We're too busy to do any of those things. ... to which I answer: 1. These projects require technical skill and political capital within the project. They are ideally suited for well-established independent FreeBSD consultants with large blocks of time sponsored by the FreeBSD Foundation. I can help (especially with the wiki work), but cannot tackle these deeper problems in the way that others can. 2. The reason you're busy is that you don't have these things. I applaud recent work on Jenkins and cluster infrastructure. I also appreciate Colin Percival's automated error reporting work, because it directly attacks the sysadmin gap. And I know that getting releases out the door is time-consuming and keeps the lights on. But the overall project list needed to be rebalanced towards system administration. I request that the Foundation consider this when calling for proposals for the next round of funded projects. Royce From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 02:33:30 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1060) id 2E65CBC4; Wed, 14 Jan 2015 02:33:30 +0000 (UTC) Date: Tue, 13 Jan 2015 18:33:30 -0800 From: Craig Rodrigues To: Royce Williams Subject: Re: projects to better support FreeBSD sysadmins Message-ID: <20150114023330.GA80986@FreeBSD.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 02:33:30 -0000 On Tue, Jan 13, 2015 at 02:14:24PM -0900, Royce Williams wrote: > At face value, the main project list is heavily weighted towards > relatively esoteric OS features. The Foundation list is heavily > weighted towards advocacy and communication (as it should be). Royce, Thank you for your post and your analysis. I agree with everything you wrote. My observation is that the FreeBSD developer community is heavily skewed towards kernel developers and systems developers. That's why the project list which you mentioned in the FreeBSD status report has a lot of items for kernel and OS features. However, FreeBSD has always been more than just a kernel. The project sells itself as a provider of a fully usable and integrated operating system. The kernel is only one component of a fully usable system. For a while, I worked for Jordan Hubbard at iXsystems, and when I talked with him, the sense I got is that in the early days of the project, the focus was much more on having a fully usable and integrated operating system than it is today. The early project founders were much more pragmatic about getting things done and having a usable system. They chose the BSD license for practicality, but were not afraid to use GNU things if there was no equivalently functional BSD licensed tool. The project was not just focused on adding esoteric OS and kernel features. For example, things like sysinstall, which tried to have a fully integrated menu for configuring the system, was a big deal in the early 1990's compared to the competition. Today, the state of the art has advanced, and sysinstall looks quite primitive, but the ideas for what it was trying to accomplish are valid. However, it was an attempt at improving usability. Unfortunately, in recent years, when Kris Moore tried to integrate newer installer work that he wrote, he was constantly pushed away because his code depends on 3rd party libraries such as Qt, which are not in the base system. Kris's work is very nice. I've used his installer in PC-BSD both in desktop and server modes. It's a shame that Kris did all this work and was basically told to get lost. The end result is that Kris had to go and form a separate PC-BSD project instead of being able to improve FreeBSD itself. The bsdinstall installer that we have today in the base system does work, but it actually *lacks* features in comparision to sysinstall which is a 1990's era tool!! Unfortunately, I think the project has lots its way and gone away from its roots in the areas of having a usable operating system and has veered towards esoteric OS and system features. I agree with you that refocusing Foundation efforts more towards improving usability would be a very good thing. -- Craig Rodrigues rodrigc@FreeBSD.org From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 02:41:05 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3F37D0C for ; Wed, 14 Jan 2015 02:41:05 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6815421F for ; Wed, 14 Jan 2015 02:41:05 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id w8so4107669qac.13 for ; Tue, 13 Jan 2015 18:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=VTjQ26fEBY+Fm92qbCLCOf69yf6pvzEnk3REkhx8w8c=; b=ymJpW1Lx9Huau4nipmiQxHH/JBTPUCYdFAc2HkpN2AENJ7xzyUv4stanRw/iQ/S1Aj QCQqnXYsHhClUu+9HUe5VZfzzjymfZzPyaW14CfEKzruz6T/uGDV54JMrDufcg5K72m2 FVRJxeJx19CbUAQ15KcisthXb+EdBwzV2aqGc1k3qTHua07Mr77PjpTc+2diXOWKFChN YobH2fzwv4/EX/oxOtcikST6mbAk2ZCCeCaxjkLoDgwD0AzOKBqhaqEgz0h/Bp/LcnY1 T9kpO2xfDfjhYSCQdsYSpyuNfTS3QSHB3gjHRxxiNU5RFrrqRlre/UrxSC3uaBHfrY3x UBAw== X-Received: by 10.224.137.65 with SMTP id v1mr2713817qat.95.1421203264606; Tue, 13 Jan 2015 18:41:04 -0800 (PST) Received: from [10.77.77.30] (c-24-2-112-131.hsd1.pa.comcast.net. [24.2.112.131]) by mx.google.com with ESMTPSA id r9sm19418526qak.2.2015.01.13.18.41.04 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 13 Jan 2015 18:41:04 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: projects to better support FreeBSD sysadmins From: Joshua Smith X-Mailer: iPhone Mail (12B440) In-Reply-To: Date: Tue, 13 Jan 2015 21:41:03 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Royce Williams Cc: "freebsd-advocacy@freebsd.org" X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 02:41:05 -0000 > On Jan 13, 2015, at 6:14 PM, Royce Williams wrote: >=20 > At Craig Rodrigues' request, I'm starting a new thread here branched > from a freebsd-ports@ thread. For those who want more context, the > original thread starts here: >=20 > https://lists.freebsd.org/pipermail/freebsd-ports/2015-January/097462.html= >=20 > It was initially about BIND REPLACE_BASE, but branched off into > general sysadmin concerns that Craig wanted to respond to. >=20 > Royce >=20 > ---------- Forwarded message ---------- > From: Royce Williams > Date: Mon, Jan 12, 2015 at 7:10 AM > Subject: Re: BIND REPLACE_BASE option > To: ports > Cc: Deb Goodkin >=20 > On Mon, Jan 12, 2015 at 4:08 AM, Kurt Jaeger wrote: >=20 >>> No disputing that, just thinking, is FreeBSD being driven by user need, >>> financial contributer need, developer need, security need, making things= >>> 'better' or just by people wanting to make their mark in a warped sense >>> of "it'll all get better"...? >>=20 >> Probably by developer *capacity* (not need) and fire-fighting, >> like most IT stuff 8-( >=20 > But like most IT stuff, resources are being asymmetrically applied to > the root causes of the fires. >=20 > Read the list of projects from last quarter: >=20 > - Address Space Layout Randomization (ASLR) I would hardly consider this esoteric.=20 > - amd64 Xen Paravirtualization > - bhyve The ability for FreeBSD to host VMs is definitely something that I find very= interesting and useful. I am a sysadmin.=20 > - Chelsio iSCSI Offload Support > - Debian GNU/kFreeBSD > - FreeBSD Preseed Installation (PXE) This also fits right in the making a sysadmin a life easier wheel house.=20 > - Jenkins Continuous Integration for FreeBSD > - New Automounter An auto mounter that behaves more like what is in other unixes also improves= my life as a sysadmin.=20 > - QEMU bsd-user-Enabled Ports Building > - VMWare VAAI and Microsoft ODX Acceleration in CTL Not really sysadmin focused but definitely not esoteric.=20 > - ZFSguru > - Intel GPU Driver Update > - SDIO Driver > - UEFI Boot Like it or not UEFI is the future supporting it well is not optional.=20 > - Updated vt(4) System Console > - Updating OpenCrypto > - FreeBSD on Newer ARM Boards > - FreeBSD/arm64 > - LLDB Debugger Port > - LLVM Address Sanitizer (Asan) > - SSE Variants of libc Routines for amd64 > - FreeBSD Python Ports > - GNOME/FreeBSD > - KDE on FreeBSD > - The Graphics Stack on FreeBSD > - Xfce >=20 > The Foundation section also lists these items not overlapping with the abo= ve: >=20 > - FreeBSD Journal > - PostgreSQL performance improvements > - Ongoing release process > - Development snapshots A better release process will likely benefit me as a sysadmin.=20 > - VM images for releases Being able to boot the base system on the hyper visor of my choice with out h= aving to muddle through the installer is a huge time saver and a bandit of s= ysadmin a everywhere.=20 > - Secure Boot planning > - Infrastructure hardware > - Java licensing > - Summits and summit sponsorship > - Travel grants, tutorials, and talks > - New Design and Implementation book > - Recruitment flyers >=20 > Are there long-term improvement projects that aren't being listed? If > so, they should be. These are just projects sponsored by the foundation. I'm sure there are many= other developments occurring throughout the project that are not listed her= e because they are not sponsored by the foundation.=20 >=20 > At face value, the main project list is heavily weighted towards > relatively esoteric OS features. See my other comments above. Frankly this is a bullshit statement.=20 > The Foundation list is heavily > weighted towards advocacy and communication (as it should be). >=20 > What is missing are high-level projects to help sysadmins maintain and > use FreeBSD on an ongoing basis. >=20 > Here are some projects that would help to close the sysadmin gap: >=20 > - Automatic error reporting and analysis A crash reporting mechanism already exists.=20 > - OS and port debugging tools for sysadmins > - Independent project-wide usability analysis What does this mean? If you run into a usability or any other sort of proble= m. Submit a PR.=20 > - Ports dependency isolation and reduction framework Doesn't seem like a sysadmin type thing to me.=20 > - Ports system reliability parity with Linuxes Can you provide more details and expand upon this? > - Searchable, taggable project FAQ Any number of the projects above are far more beneficial to sysadmin a every= where than this.=20 > - Searchable hardware support matrix integrated with bug tracker +1 for this.=20 > - Wiki curation and platform improvements >=20 > These projects decentralize and improve support for sysadmins and new > adopters. As a business case for the Foundation, these projects > should also deeply free up developer resources to focus on other major > projects. >=20 > In the past, when I have pointed out this "sysadmin gap", I receive > one of two answers: >=20 > 1. Sounds great. Let us know when you have it finished. Perhaps just getting started with something would entice support.=20 >=20 > 2. We're too busy to do any of those things. >=20 > ... to which I answer: >=20 > 1. These projects require technical skill and political capital within > the project. They are ideally suited for well-established independent > FreeBSD consultants with large blocks of time sponsored by the FreeBSD > Foundation. I can help (especially with the wiki work), but cannot > tackle these deeper problems in the way that others can. >=20 > 2. The reason you're busy is that you don't have these things. >=20 > I applaud recent work on Jenkins and cluster infrastructure. I also > appreciate Colin Percival's automated error reporting work, because > it directly attacks the sysadmin gap. And I know that getting > releases out the door is time-consuming and keeps the lights on. >=20 > But the overall project list needed to be rebalanced towards system > administration. I request that the Foundation consider this when > calling for proposals for the next round of funded projects. >=20 > Royce > _______________________________________________ > freebsd-advocacy@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-advocacy > To unsubscribe, send any mail to "freebsd-advocacy-unsubscribe@freebsd.org= " From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 02:45:26 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31284E93 for ; Wed, 14 Jan 2015 02:45:26 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EFEA24E for ; Wed, 14 Jan 2015 02:45:25 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id gd6so5950826lab.3 for ; Tue, 13 Jan 2015 18:45:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=09x5LRzxylWdgH+xNQB5s2MLET/VXAfamedwdWhbsSw=; b=Vm9NcSIW/jgAmruIIImmhKS+Q4ROAxfYuPKSKZIAsg4XvESQHBN/wywXPAeP6WwWX1 8FqSUdfX38q/XEFAn7TmxI/D/3vvEENQT22EDDDneunC3EgseK40juzvBC9hw9tT93L1 w6tHg46YQl14yVj9lT9rnZLgRGbNNqnRLzmBaRDzF2or+ZwPBUlYfdk3wx7oH22cvLqD 9AoWO/4VSFPvH0qhBdHCjcyY1/IBzEAeVAIPX4R5lTDYsarDsojWtU//GhuLI9arOIvS 4dBwsrtsFfaB7Cvw2Y3h8tEu6pmDDnBQRjmDoqWYUkERdTfMbmWmkqUw0n7m6JP1zK5E Scmw== MIME-Version: 1.0 X-Received: by 10.112.159.136 with SMTP id xc8mr1441001lbb.98.1421203523279; Tue, 13 Jan 2015 18:45:23 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.129.3 with HTTP; Tue, 13 Jan 2015 18:45:23 -0800 (PST) In-Reply-To: <20150114023330.GA80986@FreeBSD.org> References: <20150114023330.GA80986@FreeBSD.org> Date: Tue, 13 Jan 2015 18:45:23 -0800 X-Google-Sender-Auth: t3Ny5Fs67Lil_Vd49J21uyOQOe0 Message-ID: Subject: Re: projects to better support FreeBSD sysadmins From: Craig Rodrigues To: Royce Williams Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 02:45:26 -0000 On Tue, Jan 13, 2015 at 02:14:24PM -0900, Royce Williams wrote: > But the overall project list needed to be rebalanced towards system > administration. I request that the Foundation consider this when > calling for proposals for the next round of funded projects. Royce, I agree with what you wrote, but I want to add to it. I would like there to be a focus on making FreeBSD more "devops-friendly" so that the current generation of devops engineers can pick it up and easily deploy it in large cluster and cloud environments. We need to move beyond the 1990's era view of Unix administration where we only have a few Unix servers administered by hand by a few sysadmins. Today's datacenter has hundreds or thousands of nodes. These nodes need to be installed, maintained, and upgraded. These nodes are more and more maintained by devops teams who use automation frameworks (Puppet, Chef, Ansible, Saltstack, CFEngine, etc.) to accomplish these tasks. devops teams will write C and shell script if they have to, but are very pragmatic about using newer scripting languages like Python, Ruby, etc. if it is necessary to get the job done. What is FreeBSD doing to be more devops friendly? How can we make FreeBSD friendlier to people who are trying to deploy hundreds and thousands of FreeBSD nodes, especially in datacenter and cloud environments? I would like to see the project and Foundation focus on these types of problems. -- Craig From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 03:13:36 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55EBB570 for ; Wed, 14 Jan 2015 03:13:36 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1EB80883 for ; Wed, 14 Jan 2015 03:13:36 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id rd18so6653520iec.1 for ; Tue, 13 Jan 2015 19:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=webassign.net; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wXZnyiCYtQoVwPom6uP8HehNwGtDnN/S84dad31RQKI=; b=Ozw7IE4yJShlBG/y+NmiokASsliXuP0JLF8vDDIiDBk7JQRAOn+4eAbMdbqZze0QYa J3q8+v0aDwjTTeI3RFU8SSLdRVa1vtXrukn28hfa4meK/M0V0FZ0x4x7LRKzVZKJOVrg T1vfa6eEmKSe2WKYmSSbZZpMma/DyWXItUePE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=wXZnyiCYtQoVwPom6uP8HehNwGtDnN/S84dad31RQKI=; b=YpBhjxVUJcr283RQqj5BYqGRiP+Cg2xLkMtTKlFsp+GuFM0gPMqZVzsRNJd/fclVap 5/PTbVk4pVs5Yy08S0Rsb5QKNlAoPk2jnKtcmiTMzRE2omlxpi67wQkOxEqmOFe5Vvkm JyDAMh0P2sZANdJS1ryfi2VKmngJifINSosHX33kDAePgUDv2qiC67XUgaTLBtQBI9dy HBMW34YQm0Z9RydXHd4MIbrN9h+4y5iUIzJ0k+vQJd9SJWGTitB+dD0ppAKzEsxKv+se VZgKXAa6Y+qg0JB/k9/vw4P25Z4Wu03tEY/fhxZ+4NjnjVJ39jC6G/FcKx4NmEvwOWeC 1Vvw== X-Gm-Message-State: ALoCoQl80QLxQ9id16FzdvOA1EfPmBjKxJ7XKZ9xe4Zcw8ZcJPTBou5ZOsIvnD/Jzjrko73iM6mq X-Received: by 10.107.19.88 with SMTP id b85mr1849381ioj.66.1421205214793; Tue, 13 Jan 2015 19:13:34 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.106.102 with HTTP; Tue, 13 Jan 2015 19:13:14 -0800 (PST) In-Reply-To: References: <20150114023330.GA80986@FreeBSD.org> From: Hunter Satterwhite Date: Tue, 13 Jan 2015 22:13:14 -0500 Message-ID: Subject: Re: projects to better support FreeBSD sysadmins To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 03:13:36 -0000 Craig, Could you elaborate on these "problems"? Our data center is ~400 nodes and 99% FreeBSD. We've used CFengine, we're implementing Puppet (and its going great!), we use Ansible, and we also use languages such as Python, Ruby, and Google's Go. Oh and not to mention we have a RESTful application running on FreeBSD + node.js + MongoBD/MySQL. I think the project's focus is fine. Year after year we're given a complete, enterprise Unix operating system and it's only getting better. Best, Hunter On Tue, Jan 13, 2015 at 9:45 PM, Craig Rodrigues wrote: > On Tue, Jan 13, 2015 at 02:14:24PM -0900, Royce Williams wrote: > > But the overall project list needed to be rebalanced towards system > > administration. I request that the Foundation consider this when > > calling for proposals for the next round of funded projects. > > Royce, > > I agree with what you wrote, but I want to add to it. > > I would like there to be a focus on making FreeBSD more "devops-friendly" > so that the current generation of devops engineers can pick it up > and easily deploy it in large cluster and cloud environments. We need > to move beyond the 1990's era view of Unix administration where we only > have a few Unix servers administered by hand by a few sysadmins. > > Today's datacenter has hundreds or thousands of nodes. These nodes > need to be installed, maintained, and upgraded. These nodes are more and > more > maintained by devops teams who use automation frameworks (Puppet, Chef, > Ansible, > Saltstack, CFEngine, etc.) to accomplish these tasks. devops teams will > write C and shell script > if they have to, but are very pragmatic about using newer scripting > languages like Python, Ruby, etc. > if it is necessary to get the job done. > > What is FreeBSD doing to be more devops friendly? > How can we make FreeBSD friendlier to people who are trying to deploy > hundreds and thousands of > FreeBSD nodes, especially in datacenter and cloud environments? > > I would like to see the project and Foundation focus on these types of > problems. > > -- > Craig > _______________________________________________ > freebsd-advocacy@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-advocacy > To unsubscribe, send any mail to "freebsd-advocacy-unsubscribe@freebsd.org > " > -- Hunter Satterwhite Systems Engineer, Technical Operations (TechOps) From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 03:30:54 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE882941 for ; Wed, 14 Jan 2015 03:30:53 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59F30A5B for ; Wed, 14 Jan 2015 03:30:53 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id gd6so6069887lab.1 for ; Tue, 13 Jan 2015 19:30:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=NkmcfrsM5b8ffilaG6fqCqE+oBHEIebhLARNBWMCeag=; b=Zyz55oy7KuPH50HPRq5DtX9c9gXFzglij6Bt2TFrIvhMxDg/2okZ8OjnesSf5maqxT JhGkxLj7EVpTeDVGUofm0gkNIJU9ZLGE8yqzj/oPHAMRe9qIBQzjWsE6eoHSoEDvIWFK qQy50eTJzp0ceSsBhHocH4jan0ov4gc8O0FJmzPHqtiyVGtvRCnOK9uGO0Yh1D4oJl8K smC3AN6LUdxLAeer9Q50Qf6BMyj0P3N2kwg3YNUwuTV9q2yCiO4LvPnbZnywr14Qt8BQ GmPnO0zk7qeBmcO9tDLam4Guv5rVAG9Qi9d+GoQ19s6ZvphEWhZehUZ0AeSXSn8/o9vk rJ2A== MIME-Version: 1.0 X-Received: by 10.152.203.201 with SMTP id ks9mr1487774lac.57.1421206251156; Tue, 13 Jan 2015 19:30:51 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.129.3 with HTTP; Tue, 13 Jan 2015 19:30:51 -0800 (PST) In-Reply-To: References: <20150114023330.GA80986@FreeBSD.org> Date: Tue, 13 Jan 2015 19:30:51 -0800 X-Google-Sender-Auth: dC5xeudJBfKxMOpqr1o-B3fhiJc Message-ID: Subject: Re: projects to better support FreeBSD sysadmins From: Craig Rodrigues To: Hunter Satterwhite Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 03:30:54 -0000 On Tue, Jan 13, 2015 at 7:13 PM, Hunter Satterwhite < hsatterwhite@webassign.net> wrote: > Craig, > > Could you elaborate on these "problems"? Our data center is ~400 nodes and > 99% FreeBSD. We've used CFengine, we're implementing Puppet (and its going > great!), we use Ansible, and we also use languages such as Python, Ruby, > and Google's Go. Oh and not to mention we have a RESTful application > running on FreeBSD + node.js + MongoBD/MySQL. > > I think the project's focus is fine. Year after year we're given a > complete, enterprise Unix operating system and it's only getting better. > > I can point to two problems which I found today: (1) freebsd-update doesn't work so well in an automation environment without a real tty: https://lists.freebsd.org/pipermail/freebsd-current/2015-January/053982.html This was pointed out to me by a devops expert who is helping me with automation for the http://jenkins.freebsd.org. (2) documentation for doing "kickstart" installs of FreeBSD is not as easy to find as for Linux: https://lists.freebsd.org/pipermail/freebsd-current/2015-January/053970.html This was pointed out to me by another devops person I am working with who is familiar with setting up kickstart installs for Linux, but couldn't easily figure out how to do it for FreeBSD. These are very basic things and can be solved on their own, but I would like to see more of a focus on this kind of stuff at a project level, so that these problems don't exist in the first place, and things *just work*. For many people, the perception is that Linux is easier for devops people to work with than FreeBSD, and they can install/maintain many nodes in large cloud and datacenter environments more easily. I have seen in two companies where hundreds of FreeBSD nodes were migrated to Linux, because the IT/devops staff found Linux worked better at large scale than FreeBSD in the modern datacenter. I think the FreeBSD project is improving, but we can do better. -- Craig From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 03:53:34 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 591FEF74 for ; Wed, 14 Jan 2015 03:53:34 +0000 (UTC) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AF74C54 for ; Wed, 14 Jan 2015 03:53:33 +0000 (UTC) Received: by mail-ig0-f176.google.com with SMTP id b16so4983881igk.3 for ; Tue, 13 Jan 2015 19:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=webassign.net; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=9LgtUAvBzGcj25M7591QOs6RmhHALY1hmcjxV5wFmX8=; b=HxzmJm1ITZtv9nLWrNj4VpPllCMJ80rSwqmslO20eEL/9lviklP4Ze1k+WITlhn5ql wuOwkk7QMcYCfpszt+nUunQ5qBHQ0H7E2tOAs9c2QRNuNpXNIRpOpABH8stwQ7noa3NY 1fs+2tM4aKEMdtpe0Q/czvo2C9kiKYPZwU/Gw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=9LgtUAvBzGcj25M7591QOs6RmhHALY1hmcjxV5wFmX8=; b=R9ftU27PSK9zMoqVxcJQ2m/7g8pl20pFSXInBlzJxGWMNvGD86xxU8KiYVgJR8vzrF rNznLDM22mzCvIOHcMgDZMHW3UZ0h5QvM/0d1qeZu88w57zqdwsIn6ac6uos4R7Xrzd+ p+rDOFQUABkwAlyzC6jyzk8r7O0WwYwvF5dG89IWlDjvrlvHUtzWUrKb2Lgbaw525+A8 xF886s+Tfd31SgXznuaBAnLKYgIflrZw/+mmsIakWF09ZY2eJPy7+2SNYeNwtd5mMu7R 8PFwWSdKmubagcX01LQYoOu/ieJJcx5oCcGxEEr7Dp2tYphppGP/ukHDdh9eyucnxUBA yYLQ== X-Gm-Message-State: ALoCoQmivt2u6LwJnh7YTJfJxXUdZFjzwq2cI2NZxS1kIiEfNh4oZ2sEAmAw/6mgrmdRUh0j9j/o X-Received: by 10.43.33.1 with SMTP id sm1mr2625890icb.43.1421207613284; Tue, 13 Jan 2015 19:53:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.106.102 with HTTP; Tue, 13 Jan 2015 19:53:12 -0800 (PST) In-Reply-To: References: <20150114023330.GA80986@FreeBSD.org> From: Hunter Satterwhite Date: Tue, 13 Jan 2015 22:53:12 -0500 Message-ID: Subject: Re: projects to better support FreeBSD sysadmins To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 03:53:34 -0000 Thanks for providing more detail Craig. Fair enough, but these two items are minor at best and I don't feel like they do much in the way of supporting your previous claim. While I do wholeheartedly agree with the fact that freebsd-update should "just work" it's still easy to work around. To automate the installation of FreeBSD without the use of any other 3rd-party tools you would write your own shell script for bsdinstall. It's pretty straight forward and easy to do. However, I'd argue that if you want to operate at the scale you keep referring to and do it full life cycle, then you're likely not going to be doing this. Instead you'll be using tools, like Foreman and Puppet, which will make provisioning systems a cinch. FWIW, I've had both inexperienced and experienced Linux system administrators who want to employ the use of DevOps and have had both at some point and time state, "You can't do that with FreeBSD" or "FreeBSD makes it very difficult to do X". Each and every time they were incorrect and it was, because FreeBSD is not their wheel house and unfortunately they didn't take the time to do much research on their own. No one administrator can be an expert in everything, but part of what we do requires us to be inquisitive and investigative. Two traits that are fading fast in Linux administrators. - Hunter On Tue, Jan 13, 2015 at 10:30 PM, Craig Rodrigues wrote: > On Tue, Jan 13, 2015 at 7:13 PM, Hunter Satterwhite < > hsatterwhite@webassign.net> wrote: > >> Craig, >> >> Could you elaborate on these "problems"? Our data center is ~400 nodes >> and 99% FreeBSD. We've used CFengine, we're implementing Puppet (and its >> going great!), we use Ansible, and we also use languages such as Python, >> Ruby, and Google's Go. Oh and not to mention we have a RESTful application >> running on FreeBSD + node.js + MongoBD/MySQL. >> >> I think the project's focus is fine. Year after year we're given a >> complete, enterprise Unix operating system and it's only getting better. >> >> > > I can point to two problems which I found today: > > (1) freebsd-update doesn't work so well in an automation environment > without a real tty: > > > https://lists.freebsd.org/pipermail/freebsd-current/2015-January/053982.html > > This was pointed out to me by a devops expert who is helping me with > automation > for the http://jenkins.freebsd.org. > > (2) documentation for doing "kickstart" installs of FreeBSD is not as > easy to find as for Linux: > > > https://lists.freebsd.org/pipermail/freebsd-current/2015-January/053970.html > > This was pointed out to me by another devops person I am working with > who is familiar > with setting up kickstart installs for Linux, but couldn't easily > figure out how to do it for FreeBSD. > > These are very basic things and can be solved on their own, > but I would like to see more of a focus on this kind of stuff at a project > level, so that > these problems don't exist in the first place, and things *just work*. > > For many people, the perception is that Linux is easier for devops people > to work > with than FreeBSD, and they can install/maintain many nodes in large cloud > and datacenter environments > more easily. I have seen in two companies where hundreds of FreeBSD nodes > were migrated to Linux, > because the IT/devops staff found Linux worked better at large scale than > FreeBSD in the modern datacenter. > > I think the FreeBSD project is improving, but we can do better. > > -- > Craig > > -- Hunter Satterwhite Systems Engineer, Technical Operations (TechOps) From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 05:32:24 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40B68629 for ; Wed, 14 Jan 2015 05:32:24 +0000 (UTC) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 023138BF for ; Wed, 14 Jan 2015 05:32:24 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wo20so6349139obc.13 for ; Tue, 13 Jan 2015 21:32:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=wQLmu4iZMxfxtLO+LCVh3ToGTyYPAcsNWUfBYrk7Nu0=; b=EXMSfNH+60GZMtrz5fcVZy4/w3082N/u064VC9Yj4/xXX7Yl9DJNVI5X1lT6FiWrlK Q0VbDmMgmjWP5q1OO9BGYWNfMqGG+qDZjGBnOM27g06qW7UqFm6JsCUxwZ2Lz2PEAUo1 RNbOVAOqPYsTDgiQZN4zrlB6riPGYNrU50GmZafurYoxcfwGClTdqFsB4kpQbisuCA6O +XL3Lbqs3hys7Yq6zDmE12h/wm/sJO4GDt17hCUXW3SVYdKZpoxkPdRUu48zOAfgTVxk A2rwRtT3kvY82i2MF3KtJ0fJNwDHvoSudfCuz5Ndv0dGzrJ3XA8yAvx8On1DyCbDbOfd wV4w== X-Received: by 10.202.222.214 with SMTP id v205mr1160252oig.103.1421213543321; Tue, 13 Jan 2015 21:32:23 -0800 (PST) MIME-Version: 1.0 Sender: royce.williams@gmail.com Received: by 10.202.89.132 with HTTP; Tue, 13 Jan 2015 21:32:02 -0800 (PST) In-Reply-To: References: From: Royce Williams Date: Tue, 13 Jan 2015 20:32:02 -0900 X-Google-Sender-Auth: L3iLFswuK6tEwGXFsOV2N2K9lcI Message-ID: Subject: Re: projects to better support FreeBSD sysadmins To: Joshua Smith , "freebsd-advocacy@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 05:32:24 -0000 On Tue, Jan 13, 2015 at 5:41 PM, Joshua Smith wrote: >> On Jan 13, 2015, at 6:14 PM, Royce Williams wrote: >> >> At Craig Rodrigues' request, I'm starting a new thread here branched >> from a freebsd-ports@ thread. For those who want more context, the >> original thread starts here: >> >> https://lists.freebsd.org/pipermail/freebsd-ports/2015-January/097462.html >> >> It was initially about BIND REPLACE_BASE, but branched off into >> general sysadmin concerns that Craig wanted to respond to. >> >> Royce >> >> ---------- Forwarded message ---------- >> From: Royce Williams >> Date: Mon, Jan 12, 2015 at 7:10 AM >> Subject: Re: BIND REPLACE_BASE option >> To: ports >> Cc: Deb Goodkin >> >> On Mon, Jan 12, 2015 at 4:08 AM, Kurt Jaeger wrote: >> >>>> No disputing that, just thinking, is FreeBSD being driven by user need, >>>> financial contributer need, developer need, security need, making things >>>> 'better' or just by people wanting to make their mark in a warped sense >>>> of "it'll all get better"...? >>> >>> Probably by developer *capacity* (not need) and fire-fighting, >>> like most IT stuff 8-( >> >> But like most IT stuff, resources are being asymmetrically applied to >> the root causes of the fires. >> >> Read the list of projects from last quarter: I did not intend to pick on each of these projects, though you've responded as though I had. Rather, I am trying to encourage the Foundation to look at them in the aggregate, and think about other entire families of project ideas that could be encouraged. >> - Address Space Layout Randomization (ASLR) > > I would hardly consider this esoteric. I used the phrase "relatively esoteric" on purpose. "Esoteric" means "intended for or likely to be understood by only a small number of people with a specialized knowledge or interest," and "relatively" here means relative to other broad areas of high-level FreeBSD improvement. I stand by the assessment. I believe that small-shop admins would value stable port management and other basics like my list below before most of the projects on the list. >> - amd64 Xen Paravirtualization >> - bhyve > > The ability for FreeBSD to host VMs is definitely something that I find very interesting and useful. I am a sysadmin. My intent was not to say that these things aren't useful. It was that there are some other basic things that could use some attention as well. >> - Chelsio iSCSI Offload Support >> - Debian GNU/kFreeBSD >> - FreeBSD Preseed Installation (PXE) > > This also fits right in the making a sysadmin a life easier wheel house. When sysadmins are afraid to upgrade ports because of a hidden cascade of dependencies that will result in an unusable system, PXE is a great way to recover. What would be even better is a way to reduce the chances of that cascade in the first place. >> - Jenkins Continuous Integration for FreeBSD >> - New Automounter > > An auto mounter that behaves more like what is in other unixes also improves my life as a sysadmin. > >> - QEMU bsd-user-Enabled Ports Building >> - VMWare VAAI and Microsoft ODX Acceleration in CTL > > Not really sysadmin focused but definitely not esoteric. I was going to try to respond to this in some way, but I'd like to just restate my broader point that the project has opportunities in other kinds of areas that would make it so that more core resources become available to work on needed improvements like this one. >> - ZFSguru >> - Intel GPU Driver Update >> - SDIO Driver >> - UEFI Boot > > Like it or not UEFI is the future supporting it well is not optional. I agree. >> - Updated vt(4) System Console >> - Updating OpenCrypto >> - FreeBSD on Newer ARM Boards > >> - FreeBSD/arm64 >> - LLDB Debugger Port >> - LLVM Address Sanitizer (Asan) >> - SSE Variants of libc Routines for amd64 >> - FreeBSD Python Ports >> - GNOME/FreeBSD >> - KDE on FreeBSD >> - The Graphics Stack on FreeBSD >> - Xfce >> >> The Foundation section also lists these items not overlapping with the above: >> >> - FreeBSD Journal >> - PostgreSQL performance improvements >> - Ongoing release process >> - Development snapshots > > A better release process will likely benefit me as a sysadmin. Agreed. >> - VM images for releases > > Being able to boot the base system on the hyper visor of my choice with out having to muddle through the installer is a huge time saver and a bandit of sysadmin a everywhere. > >> - Secure Boot planning >> - Infrastructure hardware >> - Java licensing >> - Summits and summit sponsorship >> - Travel grants, tutorials, and talks >> - New Design and Implementation book >> - Recruitment flyers >> >> Are there long-term improvement projects that aren't being listed? If >> so, they should be. > > These are just projects sponsored by the foundation. I'm sure there are many other developments occurring throughout the project that are not listed here because they are not sponsored by the foundation. The purpose of my message was to ask the Foundation to consider shifting the balance of sponsorship a little to include a different class of projects, so I worked from the latest announced list of Foundation projects. >> At face value, the main project list is heavily weighted towards >> relatively esoteric OS features. > > See my other comments above. Frankly this is a bullshit statement. I worked very hard to make this message constructive, and you are making it difficult to remain so. What I'm saying is that just like an investment portfolio, the FreeBSD project portfolio needs to be examined for rebalancing once in a while. >> The Foundation list is heavily >> weighted towards advocacy and communication (as it should be). >> >> What is missing are high-level projects to help sysadmins maintain and >> use FreeBSD on an ongoing basis. >> >> Here are some projects that would help to close the sysadmin gap: >> >> - Automatic error reporting and analysis > > A crash reporting mechanism already exists. Is there a way to publicly view aggregated results, or see a list of bugs that were fixed because the mechanism is working? If so, that would be useful to know. >> - OS and port debugging tools for sysadmins >> - Independent project-wide usability analysis > > What does this mean? If you run into a usability or any other sort of problem. Submit a PR. It may be hard to imagine what it might be like to be, say, a Solaris admin all your life, with no other Unix-like exposure ... when one day, someone comes to you and tells you that you have to upgrade the FreeBSD Apache server between now and when you have to go home and feed your kids. But I encourage you to think about it. There's no way to file a PR for this. It's the kind of thing that requires a champion within the project to build bridges, tear down walls, challenge assumptions, and tackle the problem. >> - Ports dependency isolation and reduction framework > > Doesn't seem like a sysadmin type thing to me. This would help sysadmins by supporting improving end-result reliability of the ports system generally, which I see as a cornerstone of why FreeBSD is hard today. >> - Ports system reliability parity with Linuxes > > Can you provide more details and expand upon this? Please bear in mind that what I'm about to say pains me greatly, as someone who has been a passionate advocate for FreeBSD at companies I work at for the past fourteen years. If a small or medium-sized shop wanted to set up a stable, basic web server, and feel confident that they can move smoothly through OS and server/application upgrades, I would recommend an Ubuntu-derived "LTS" (Long Term Support) release before I would recommend FreeBSD. When I try to explain to other OS' admins that we have to release a quarterly stable snapshot set of ports because regular rolling port updates are too fragile, or that we have to manually remember to check a text file every time we want to upgrade a port, and regularly have to manually deinstall the old port, they look at me like I'm insane. They take it for granted that basic, everyday port dependency management is a largely solved problem. That's what I meant by parity. >> - Searchable, taggable project FAQ > > Any number of the projects above are far more beneficial to sysadmin a everywhere than this. Unless you're a sysadmin strapped for time. Or new to the project. Or split between multiple platforms. Or also assigned non-sysadmin duties. Or picked the wrong day to be too tired from the outage the night before and accidentally not read UPDATING. The FreeBSD forums and lists have people asking common questions. They are often answered by the same people -- people who are skilled, eager to help, and in danger of burning out by having to answer the same questions all the time. A FAQ with some self-organizing features -- perhaps even Stack Overflow-style, but with more curation -- could help with that. >> - Searchable hardware support matrix integrated with bug tracker > > +1 for this. Well, at least something was a good idea. :-) >> - Wiki curation and platform improvements >> >> These projects decentralize and improve support for sysadmins and new >> adopters. As a business case for the Foundation, these projects >> should also deeply free up developer resources to focus on other major >> projects. >> >> In the past, when I have pointed out this "sysadmin gap", I receive >> one of two answers: >> >> 1. Sounds great. Let us know when you have it finished. > > Perhaps just getting started with something would entice support. See answer #1 below. >> 2. We're too busy to do any of those things. >> >> ... to which I answer: >> >> 1. These projects require technical skill and political capital within >> the project. They are ideally suited for well-established independent >> FreeBSD consultants with large blocks of time sponsored by the FreeBSD >> Foundation. I can help (especially with the wiki work), but cannot >> tackle these deeper problems in the way that others can. >> >> 2. The reason you're busy is that you don't have these things. >> >> I applaud recent work on Jenkins and cluster infrastructure. I also >> appreciate Colin Percival's automated error reporting work, because >> it directly attacks the sysadmin gap. And I know that getting >> releases out the door is time-consuming and keeps the lights on. >> >> But the overall project list needed to be rebalanced towards system >> administration. I request that the Foundation consider this when >> calling for proposals for the next round of funded projects. From owner-freebsd-advocacy@FreeBSD.ORG Wed Jan 14 17:28:10 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F525C80 for ; Wed, 14 Jan 2015 17:28:10 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A08CEEF8 for ; Wed, 14 Jan 2015 17:28:09 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id u10so9055195lbd.13 for ; Wed, 14 Jan 2015 09:28:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=c2l04Nak0ArWBPC0kjAPWhLGsfE4CN/aO8GXJnb/EGU=; b=u/pCSF74cJxoOzwjTErB2mbIzDoa3BWU5rkPjn9xf5gk8qz6hItYqJ23AQ0pBOO+6H tfzTv0drd4mo3fw1I2qeJ7OTyobLaDn1X/yz3OS2e7pPrJi0lIqNJZKh9YgwbYZ7YY83 sdOjY94BF4DqF16P1PTia2yD9BK3L7rdwjqeRHNsRPKMk2WYFsDH/RQypby8NtwASZPW ulgzsRBPaDxTsaZibYWojF14YQ+xjcU/f7s3s6FTAuGE3vDC966JiJP/NAYj2I+oPKGK VvPF9OihhBmIy1EAzJmq+D+hVq3b4hOxUEUkegiqUl9GlqIiT3CjixSblTlc7KlPnj3O 7zCg== MIME-Version: 1.0 X-Received: by 10.152.23.38 with SMTP id j6mr5173135laf.81.1421256487647; Wed, 14 Jan 2015 09:28:07 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.129.3 with HTTP; Wed, 14 Jan 2015 09:28:07 -0800 (PST) In-Reply-To: References: <20150114023330.GA80986@FreeBSD.org> Date: Wed, 14 Jan 2015 09:28:07 -0800 X-Google-Sender-Auth: egXKghVC1n3F9ohT9npEjve1SEY Message-ID: Subject: Re: projects to better support FreeBSD sysadmins From: Craig Rodrigues To: Hunter Satterwhite Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 17:28:10 -0000 On Tue, Jan 13, 2015 at 7:53 PM, Hunter Satterwhite < hsatterwhite@webassign.net> wrote: > Thanks for providing more detail Craig. > > Fair enough, but these two items are minor at best and I don't feel like > they do much in the way of supporting your previous claim. While I do > wholeheartedly agree with the fact that freebsd-update should "just work" > it's still easy to work around. > kickstart and freebsd-update are just two examples, found very quickly after trying to set this stuff up. freebsd-update has been around since 2006. The problem I mentioned and workarounds have been mentioned on the web since then. To go back to Royce's original posting, I am seeing that there is a definite disconnect between developers who work on the source tree, and people who are deploying FreeBSD in modern datacenter and cloud environments. I'm actually not the only one who has run into these types of problems. I've talked to a friend in a company making a product based on FreeBSD, who has run into similar problems when trying to do kickstart and mass deployment of FreeBSD nodes. If you are willing to code your own stuff up, it is doable, but things are definitely not as well documented and turnkey as the Linux equivalet solutions. > > To automate the installation of FreeBSD without the use of any other > 3rd-party tools you would write your own shell script for bsdinstall. It's > pretty straight forward and easy to do. However, I'd argue that if you want > to operate at the scale you keep referring to and do it full life cycle, > then you're likely not going to be doing this. Instead you'll be using > tools, like Foreman and Puppet, which will make provisioning systems a > cinch. > For the http://jenkins.freebsd.org cluster, I quickly came to the conclusion that you have described. I put out a Call for Help for some devops assistance: https://lists.freebsd.org/pipermail/freebsd-current/2014-December/053584.html and got one volunteer from Ahmed Kamal, a devops expert who works for a company specializing in cloud/devops ( http://www.cloud9ers.com ): https://lists.freebsd.org/pipermail/freebsd-testing/2015-January/000723.html Ahmed is new to FreeBSD, but he definitely knows his stuff with devops and cloud, and has started providing code and scripts to help: https://github.com/freebsd/freebsd-ci/commits/master Any issues that Ahmed is finding in FreeBSD itself (whether it is src, ports, or docs), I am trying to push fixes back into FreeBSD itself to improve things and smooth things over. > > FWIW, I've had both inexperienced and experienced Linux system > administrators who want to employ the use of DevOps and have had both at > some point and time state, "You can't do that with FreeBSD" or "FreeBSD > makes it very difficult to do X". Each and every time they were incorrect > and it was, because FreeBSD is not their wheel house and unfortunately they > didn't take the time to do much research on their own. No one administrator > can be an expert in everything, but part of what we do requires us to be > inquisitive and investigative. Two traits that are fading fast in Linux > administrators. > > Well, like it or not, Linux (in its various distributions) has succeeded in becoming the dominant Unix platform in the modern datacenter. The 3rd party tools, documentation, and skillset of people who are available for hire reflect this, and FreeBSD is an afterthought. Anything that we can do in FreeBSD to change things in the base system, ports, and documentation to make things easier for sysadmins, as Royce pointed out, would be a great focus for the project and Foundation. Having people like Ahmed, who are familiar with the Linux ways of doing things, but are open to pointing out where FreeBSD lags behind and helping improve things, is as good a place to start as any. -- Craig From owner-freebsd-advocacy@FreeBSD.ORG Thu Jan 15 05:48:44 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E16FAA65; Thu, 15 Jan 2015 05:48:44 +0000 (UTC) Received: from ns0.genyosha.net (ns0.genyosha.net [108.86.149.90]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF835AF9; Thu, 15 Jan 2015 05:48:44 +0000 (UTC) Received: from dragon.genyosha.home (dragon.genyosha.net [108.86.149.92]) by ns0.genyosha.net (8.14.9/8.14.9) with ESMTP id t0F5mfMv048176; Wed, 14 Jan 2015 21:48:42 -0800 (PST) (envelope-from sr@genyosha.net) Received: from dragon.genyosha.home (localhost.localdomain [127.0.0.1]) by dragon.genyosha.home (8.14.4/8.14.4) with ESMTP id t0F5madC002321; Wed, 14 Jan 2015 21:48:36 -0800 Received: (from sr@localhost) by dragon.genyosha.home (8.14.4/8.14.4/Submit) id t0F5maLA002319; Wed, 14 Jan 2015 21:48:36 -0800 Date: Wed, 14 Jan 2015 21:48:36 -0800 From: Steve Rikli To: Craig Rodrigues Subject: Re: projects to better support FreeBSD sysadmins Message-ID: <20150115054836.GA2307@dragon.genyosha.home> References: <20150114023330.GA80986@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (ns0.genyosha.net [108.86.149.90]); Wed, 14 Jan 2015 21:48:42 -0800 (PST) Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 05:48:45 -0000 On Wed, Jan 14, 2015 at 09:28:07AM -0800, Craig Rodrigues wrote: > > kickstart and freebsd-update are just two examples, found very quickly > after trying to set this > stuff up. freebsd-update has been around since 2006. The problem > I mentioned and workarounds have been mentioned on the web since then. > To go back to Royce's original posting, I am seeing that there is a > definite disconnect between developers who work on the source tree, and > people > who are deploying FreeBSD in modern datacenter and cloud environments. > > I'm actually not the only one who has run into these types of problems. > I've talked to a friend in a company making a product based on FreeBSD, > who has run into similar problems when trying to do kickstart and mass > deployment of FreeBSD nodes. If you are willing to code your own stuff up, > it is doable, but things are definitely not as well documented and turnkey > as the Linux equivalet solutions. I think Craig's comments capture my own experience with FreeBSD Kickstart pretty well. I setup Kickstart/Jumpstart for FreeBSD 6.* long ago at $WORK, and it was a fair amount of effort putting all the pieces together from various docs and websearching, plus some scripting on my own for postinstall (which is fine, and expected -- not unlike Linux). The end results were functional, but it wasn't as flexible or easy to do as Linux Kickstart with PXElinux. IIRC I ended up having to recompile the FreeBSD pxeboot loader, since it hardcoded "/pxeroot" as the NFS root path, and didn't support TFTP (I think); I had to do that for all versions and architectures of FreeBSD we ran at the time -- so it had a relatively high "start-up cost" to get a new/additional version going, compared to a new version of Linux CentOS or what have you. Nowdays I'm not sure where to start with modern FreeBSD 9 or 10. I keep an eye out for sysinstall- and PXE-related activity in modern FreeBSD, and I gather there have been changes in those areas, but I confess I haven't pursued any of them yet. For my own admittedly selfish needs, in the context of this thread I'm less interested in Puppet and the other configuration management orchestration schemes -- there are already howto recipes and docs and other help resources for those, and I don't think FreeBSD needs to reinvent the wheel to get FreeBSD-flavored docs. Nor do I think we need another FreeBSD howto on setting up an NFS, DHCP, TFTP, HTTP, etc. server, e.g. to provide the OS images to Kickstart -- again, documentation for that already exists, and even the Linux docs are not hard to adapt to FreeBSD. Plus the Ports Collection is great for whatever services don't come along natively with the base FreeBSD. Instead, I think there needs to be more focus on the parts of the OS automated installation which are FreeBSD-specific and different from the Linux Kickstart equivalents; e.g. just off the top of my head: - how is the FreeBSD pxeboot loader different from Linux? E.g. what args/options will it accept? Can it play nicely with PXElinux these days? Example pxe.cfg files? What if you need to have multiple FreeBSD versions and architectures Kickstarted from the same server? - what is the modern FreeBSD equivalent of a Linux Kickstart ks.cfg file, if any? - how does one script/automate the postinstall configuration with sysinstall or PC-BSD's installer or ??? - likewise for preinstall steps, if applicable (Linux Kickstart has sections for both in the kickstart config file) e.g. for disk partitioning or other early actions during an automated OS install As others have mentioned in this thread, the RedHat/CentOS et al docs for those areas are pretty good and pretty easily found. I'd love to see something similar for FreeBSD instead of my very old cobbled- together notes which probably aren't applicable anymore. Maybe I'm wrong (always a distinct possibility :-) ) but it seems to me that clouds and VMs already have their own deployment mechanisms (the AWS Store or VMware templates and clones etc.), so again that's an area where FreeBSD maybe shouldn't spend a lot of resources to reinvent wheels and documentation. Whereas for better or worse, Linux Kickstart and PXElinux (or SYSlinux etc.) seems to be the defacto standard for typical OS deployments, until you get to cloud-y things and cloning VMs and whatnot. But even in cloud/vm areas, you still may want to Kickstart at least the 1st instance, right? In any case, thanks for having the conversation. Cheers, sr. From owner-freebsd-advocacy@FreeBSD.ORG Thu Jan 15 13:33:18 2015 Return-Path: Delivered-To: freebsd-advocacy@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95AF9D59 for ; Thu, 15 Jan 2015 13:33:18 +0000 (UTC) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00730235 for ; Thu, 15 Jan 2015 13:33:18 +0000 (UTC) Received: by mail-lb0-f169.google.com with SMTP id p9so13270047lbv.0 for ; Thu, 15 Jan 2015 05:33:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=fvm/UswZx9ikhD85djGu1mY8iGjQJHVfMfMCkuEOunA=; b=y7sPsKaEVfzHX3upqBbsaBoyZCwis3cPayRM1pL1HW4wUSQKvlcShTvn12GGpEJ1AY sbJcvQi4O8s2ki1Ls2XEdPSubCjvCV/QzMQvMsvvwqXxkdszPt8YcdgcrNjTFJUcxafF d7ZtcaUcWgJd/rnr+6sL5a6597h1IAP/bfnCPnXRRwLLDv8Pp+/ClZ48MhNqmOCD5dXr 3SyzsmhDK8S8Ia1iTryxP+puIzvaBrzp+gEoVrv7i4WnEDbs7yK7srp3fI2ouIE8UN5+ 49Lakq2Z6CZVoDwnvxMUqp7n8sRp5RfzDPQusMYHvsdp+0GFtFbanRNbVgrc4earRlYm BDZg== MIME-Version: 1.0 X-Received: by 10.112.97.163 with SMTP id eb3mr3545584lbb.47.1421328794928; Thu, 15 Jan 2015 05:33:14 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.129.3 with HTTP; Thu, 15 Jan 2015 05:33:14 -0800 (PST) In-Reply-To: <20150115054836.GA2307@dragon.genyosha.home> References: <20150114023330.GA80986@FreeBSD.org> <20150115054836.GA2307@dragon.genyosha.home> Date: Thu, 15 Jan 2015 05:33:14 -0800 X-Google-Sender-Auth: 6ffEs7u7KOG9_4ud6XV9aE79C4c Message-ID: Subject: Re: projects to better support FreeBSD sysadmins From: Craig Rodrigues To: Steve Rikli Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-advocacy@freebsd.org, Deb Goodkin X-BeenThere: freebsd-advocacy@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD Evangelism List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 13:33:18 -0000 On Wed, Jan 14, 2015 at 9:48 PM, Steve Rikli wrote: > > Instead, I think there needs to be more focus on the parts of the OS > automated installation which are FreeBSD-specific and different from > the Linux Kickstart equivalents; e.g. just off the top of my head: > > - how is the FreeBSD pxeboot loader different from Linux? E.g. > what args/options will it accept? Can it play nicely with > PXElinux these days? Example pxe.cfg files? What if you need > to have multiple FreeBSD versions and architectures Kickstarted > from the same server? > > - what is the modern FreeBSD equivalent of a Linux Kickstart > ks.cfg file, if any? > > - how does one script/automate the postinstall configuration with > sysinstall or PC-BSD's installer or ??? > > - likewise for preinstall steps, if applicable (Linux Kickstart > has sections for both in the kickstart config file) e.g. for > disk partitioning or other early actions during an automated > OS install > I just found out about something today. Can you review this work by Google Summer of Code student Kamil Czekirda and see how functional it is compared to Linux kickstart: https://lists.freebsd.org/pipermail/freebsd-current/2015-January/053994.html If this work fits the needs, then maybe we can focus on getting it into the FreeBSD tree, improving the docs, and making sure that web searches for "FreeBSD kickstart" show this stuff. > > Whereas for better or worse, Linux Kickstart and PXElinux (or SYSlinux > etc.) seems to be the defacto standard for typical OS deployments, until > you get to cloud-y things and cloning VMs and whatnot. But even in > cloud/vm areas, you still may want to Kickstart at least the 1st > instance, right? > I definitely need to bootstrap/kickstart the first initial instance of VM's for things I am working on. I think other people need to do the same thing also. -- Craig