From owner-freebsd-arch@freebsd.org Wed Apr 11 21:04:49 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9BAE4F916C7 for ; Wed, 11 Apr 2018 21:04:49 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B5607F0DF for ; Wed, 11 Apr 2018 21:04:49 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io0-x229.google.com with SMTP id p139so3950642iod.0 for ; Wed, 11 Apr 2018 14:04:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=JWzm5xyqO9oNsCa5RsH3lRMRYWMDv6+Gxki5zg4KeD4=; b=ihh4OWdQQrf8kUZAfvOBAYj26Q+XZMJglhjuJsiLcDws7d5dEn9T+G9tWYZKKFL5Ek yu5hNZu2oKQt1REIkGYGCdUxSWGaaDnRKusXTyDLA/R7cs4LyV/2YlYmIezuscXYDGHz qPof+/eZrFNEgbJuVU7XXMmForrdd/NlWtFrMBzylPhjMuRGIND8nUTcwoqDT3qShStd DpLnHAt/BCQjcESiTXQkA0dojPZEkmYZeQoGOt9GfYWrh2vDbzaJmErye/KseuIXTXtc 86xArPAAGOJtK+QmVDGVSCCgRmxfvlTAihxHpyclFKeyI03EeO+s5SSOlig7Y37Ybmlb 1Uwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-disposition:user-agent; bh=JWzm5xyqO9oNsCa5RsH3lRMRYWMDv6+Gxki5zg4KeD4=; b=eF2Ep1L2Dpg5jFHJSb2xJifjxhY2jBCPY9VpxVMgA/2/YdFecBmO4nvDMgIVktIsvE rcIu454h+uLzdtPIpvoTQlLWinQpLnc3r42IV6nFftEAeP7QhmYGNAziTDnp/TwQejme Iz9PEWxR3wvhcfNmB4SWjH31X1/uKyeAyoTDnjo4XpYqOdOQPejAn9KpMRc4t94UHjEo rRhCvz9ODZr0PTmRuS8buaDqaRZMq4ujz1Sm9C7eJdhj/O5paH37nNtLlljHyk3wRmtp IN+nEjdvEZujabEGHXbwnjFfsMyDIFVVZhU00rAvi47QOgGg5EG6nioVladlx77y/geS rorw== X-Gm-Message-State: ALQs6tC7K4zGgIceCDrNnRD7Khcc39MUghqjWgEIcmk5S82V+GuoI+iL powT4AJql7QTtaCHGvXLsCbKfA== X-Google-Smtp-Source: AIpwx4/YSpX+GA/vWCLoyereGrwhxfZ8jVeSonLwkEIHPTx8bpTjBC/mJaPy6VBnqPjlfF90mrs/uA== X-Received: by 10.107.144.11 with SMTP id s11mr6427905iod.37.1523480688246; Wed, 11 Apr 2018 14:04:48 -0700 (PDT) Received: from raichu (toroon0560w-lp130-04-184-145-252-74.dsl.bell.ca. [184.145.252.74]) by smtp.gmail.com with ESMTPSA id k7sm887867ioc.44.2018.04.11.14.04.47 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Apr 2018 14:04:47 -0700 (PDT) Sender: Mark Johnston Date: Wed, 11 Apr 2018 17:04:45 -0400 From: Mark Johnston To: freebsd-arch@FreeBSD.org Subject: CFT: netdump Message-ID: <20180411210445.GC43015@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 21:04:49 -0000 Hi, In user/markj/netdump I've been working on a feature which allows the kernel to transmit kernel dumps over a network rather than writing to a local storage device. This is convenient for VMs, diskless or embedded devices, and devices with limited storage space. The server component of netdump is a UDP daemon which is in ports as ftp/netdumpd. The original code has been around for a long time and wasn't authored by me. However, I've fixed a number of bugs, made it more reliable, and integrated it with dumpon(8). I also largely rewrote the userland daemon. At this point I'd like to propose committing it to HEAD. The netdump(4) man page included in the patch lists the drivers currently supported by netdump[*]. To be supported, a driver must be augmented with a set of routines for use by the generic code. These allow the generic code to transmit packets and poll for completions. I added a mechanism to overwrite the mbuf/cluster UMA zone pointers at panic time so that they instead fetch mbufs from a pre-allocated pool. This way, existing driver code can be used without modification, and we don't impose any runtime overhead in the regular mbuf allocation path. Of course, the netdump data path is not fully reliable since the panic may have interrupted a thread as it was processing tx/rx ring entries and thus left them in an inconsistent state. In my experience this hasn't proved to be an issue, and when testing I will typically have several iperf streams running over the interface at panic time. That said, the same issue exists in principle with local storage devices as well. A kernel dump will generally contain sensitive data. Thus, netdump should only be used on trusted networks. It is possible to use the kernel dump encryption feature (dumpon -k) with netdump, however I am not certain that the encryption scheme used is well-suited for this situation. Any feedback on this would be appreciated. Please note that you need netdumpd-20180411 or later when using this feature, as the daemon needs to accept and save the symmetric key in addition to the dump itself and support for this isn't present in earlier versions. A single large patch for the src tree is available here: https://people.freebsd.org/~markj/patches/netdump/netdump-20180411.diff Of course, the user/markj/netdump branch in SVN or its clone in the GitHub mirror may be used as well. Documentation for configuration is in the dumpon(8) man page. The gist of it is that one specifies some additional network parameters to dumpon(8), along with an interface name. For instance, # dumpon -c 192.168.2.11 -s 192.168.2.1 vtnet0 tells the kernel to transmit to netdumpd at 192.168.2.1 using a source address of 192.168.2.11. If there is a router between the two, the gateway IP for the client must be specified as well, with -g. I'm interested in any testing reports that folks may have, especially with ixgbe or bnxt, or with NICs under load. If you have a favourite NIC driver which does not yet support netdump, I'm happy to help add support or review patches. Depending on how testing goes, I plan to create reviews for various components of the patch over the next several days. [*] Currently, alc(4), bge(4), bxe(4), cxgb(4), mlx4en(4), re(4) and vtnet(4) are supported. Drivers implemented using iflib are also supported in principle, though I've only tested em(4) and igb(4).