Date: Tue, 1 May 2018 13:38:17 -0400 From: Mark Johnston <markj@FreeBSD.org> To: freebsd-arch@FreeBSD.org Subject: Re: CFT: netdump Message-ID: <20180501173817.GB47315@raichu> In-Reply-To: <20180411210445.GC43015@raichu> References: <20180411210445.GC43015@raichu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Apr 11, 2018 at 05:04:45PM -0400, Mark Johnston wrote: > Hi, > > In user/markj/netdump I've been working on a feature which allows the > kernel to transmit kernel dumps over a network rather than writing to a > local storage device. This is convenient for VMs, diskless or embedded > devices, and devices with limited storage space. The server component of > netdump is a UDP daemon which is in ports as ftp/netdumpd. > > The original code has been around for a long time and wasn't authored by > me. However, I've fixed a number of bugs, made it more reliable, and > integrated it with dumpon(8). I also largely rewrote the userland > daemon. At this point I'd like to propose committing it to HEAD. > > The netdump(4) man page included in the patch lists the drivers > currently supported by netdump[*]. To be supported, a driver must be > augmented with a set of routines for use by the generic code. These > allow the generic code to transmit packets and poll for completions. I > added a mechanism to overwrite the mbuf/cluster UMA zone pointers at > panic time so that they instead fetch mbufs from a pre-allocated pool. > This way, existing driver code can be used without modification, and we > don't impose any runtime overhead in the regular mbuf allocation path. > > Of course, the netdump data path is not fully reliable since the panic > may have interrupted a thread as it was processing tx/rx ring entries > and thus left them in an inconsistent state. In my experience this > hasn't proved to be an issue, and when testing I will typically have > several iperf streams running over the interface at panic time. That > said, the same issue exists in principle with local storage devices > as well. > > A kernel dump will generally contain sensitive data. Thus, netdump > should only be used on trusted networks. It is possible to use the > kernel dump encryption feature (dumpon -k) with netdump, however I > am not certain that the encryption scheme used is well-suited for > this situation. Any feedback on this would be appreciated. Please note > that you need netdumpd-20180411 or later when using this feature, as the > daemon needs to accept and save the symmetric key in addition to the > dump itself and support for this isn't present in earlier versions. > > A single large patch for the src tree is available here: > https://people.freebsd.org/~markj/patches/netdump/netdump-20180411.diff > Of course, the user/markj/netdump branch in SVN or its clone in the > GitHub mirror may be used as well. Documentation for configuration is in > the dumpon(8) man page. The gist of it is that one specifies some > additional network parameters to dumpon(8), along with an interface > name. For instance, > > # dumpon -c 192.168.2.11 -s 192.168.2.1 vtnet0 > > tells the kernel to transmit to netdumpd at 192.168.2.1 using a source > address of 192.168.2.11. If there is a router between the two, the > gateway IP for the client must be specified as well, with -g. > > I'm interested in any testing reports that folks may have, especially > with ixgbe or bnxt, or with NICs under load. If you have a favourite > NIC driver which does not yet support netdump, I'm happy to help add > support or review patches. Depending on how testing goes, I plan to > create reviews for various components of the patch over the next several > days. I've created reviews for various components of the change: https://reviews.freebsd.org/D15251 (post-panic mbuf allocator) https://reviews.freebsd.org/D15252 (kernel dump refactoring for netdump) https://reviews.freebsd.org/D15253 (core netdump client) https://reviews.freebsd.org/D15254 (dumpon(8) extension for netdump) https://reviews.freebsd.org/D15255 (alc(4) hooks) https://reviews.freebsd.org/D15256 (bge(4) hooks) https://reviews.freebsd.org/D15257 (bxe(4) hooks) https://reviews.freebsd.org/D15258 (cxgb(4) hooks) https://reviews.freebsd.org/D15259 (mlxen(4) hooks) https://reviews.freebsd.org/D15260 (re(4) hooks) https://reviews.freebsd.org/D15261 (vtnet(4) hooks) https://reviews.freebsd.org/D15262 (iflib hooks) If anyone is interested in reviewing some or all of these diffs, please feel free to add yourself to the corresponding review. I plan to add driver maintainers as reviewers to the corresponding driver changes.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180501173817.GB47315>