From owner-svn-src-all@freebsd.org Mon Aug 7 02:49:27 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC362DD93A3; Mon, 7 Aug 2017 02:49:27 +0000 (UTC) (envelope-from sephe@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99EA0777D0; Mon, 7 Aug 2017 02:49:27 +0000 (UTC) (envelope-from sephe@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v772nQYc047057; Mon, 7 Aug 2017 02:49:26 GMT (envelope-from sephe@FreeBSD.org) Received: (from sephe@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v772nQba047053; Mon, 7 Aug 2017 02:49:26 GMT (envelope-from sephe@FreeBSD.org) Message-Id: <201708070249.v772nQba047053@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: sephe set sender to sephe@FreeBSD.org using -f From: Sepherosa Ziehau Date: Mon, 7 Aug 2017 02:49:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r322134 - in stable/11: contrib/hyperv/tools/scripts etc/devd libexec/hyperv X-SVN-Group: stable-11 X-SVN-Commit-Author: sephe X-SVN-Commit-Paths: in stable/11: contrib/hyperv/tools/scripts etc/devd libexec/hyperv X-SVN-Commit-Revision: 322134 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2017 02:49:27 -0000 Author: sephe Date: Mon Aug 7 02:49:26 2017 New Revision: 322134 URL: https://svnweb.freebsd.org/changeset/base/322134 Log: MFC 321762 hyperv: Add VF bringup scripts and devd rules. How network VF works with hn(4) on Hyper-V in non-transparent mode: - Each network VF has a cooresponding hn(4). - The network VF and the it's cooresponding hn(4) have the same hardware address. - Once the network VF is up, e.g. ifconfig VF up: o All of the transmission should go through the network VF. o Most of the reception goes through the network VF. o Small amount of reception may go through the cooresponding hn(4). This reception will happen, even if the the cooresponding hn(4) is down. The cooresponding hn(4) will change the reception interface to the network VF, so that network layer and application layer will be tricked into thinking that these packets were received by the network VF. o The cooresponding hn(4) pretends the physical link is down. - Once the network VF is down or detached: o All of the transmission should go through the cooresponding hn(4). o All of the reception goes through the cooresponding hn(4). o The cooresponding hn(4) fallbacks to the original physical link detection logic. All these features are mainly used to help live migration, during which the network VF will be detached, while the network communication to the VM must not be cut off. In order to reach this level of live migration transparency, we use failover mode lagg(4) with the network VF and the cooresponding hn(4) attached to it. To ease user configuration for both network VF and non-network VF, the lagg(4) will be created by the following rules, and the configuration of the cooresponding hn(4) will be applied to the lagg(4) automatically. Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D11635 Added: stable/11/contrib/hyperv/tools/scripts/hyperv_vfattach - copied unchanged from r321762, head/contrib/hyperv/tools/scripts/hyperv_vfattach stable/11/contrib/hyperv/tools/scripts/hyperv_vfup - copied unchanged from r321762, head/contrib/hyperv/tools/scripts/hyperv_vfup Modified: stable/11/etc/devd/hyperv.conf stable/11/libexec/hyperv/Makefile Directory Properties: stable/11/ (props changed) Copied: stable/11/contrib/hyperv/tools/scripts/hyperv_vfattach (from r321762, head/contrib/hyperv/tools/scripts/hyperv_vfattach) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/contrib/hyperv/tools/scripts/hyperv_vfattach Mon Aug 7 02:49:26 2017 (r322134, copy of r321762, head/contrib/hyperv/tools/scripts/hyperv_vfattach) @@ -0,0 +1,79 @@ +#!/bin/sh + +# +# If transparent VF is enabled, don't do anything. +# + +sysctl -n hw.hn.vf_transparent > /dev/null 2>&1 +if [ $? -ne 0 ] +then + # Old kernel; no transparent VF. + vf_transparent=0 +else + vf_transparent=`sysctl -n hw.hn.vf_transparent` +fi + +if [ $vf_transparent -ne 0 ] +then + # Transparent VF; done! + exit 0 +fi + +iface=$1 +delay=$2 + +if [ $delay -gt 0 ] +then + # + # Delayed VF up. + # + sleep $delay + ifconfig $iface up + # Done! + exit $? +fi + +# +# Check to see whether $iface is a VF or not. +# If $iface is a VF, bring it up now. +# + +# for hyperv_vf_delay +. /etc/rc.conf + +sysctl -n hw.hn.vflist > /dev/null 2>&1 +if [ $? -ne 0 ] +then + # Old kernel; nothing could be done properly. + exit 0 +fi +vf_list=`sysctl -n hw.hn.vflist` + +for vf in $vf_list +do + if [ $vf = $iface ] + then + # + # Linger a little bit (at least 2 seconds) mainly to + # make sure that $iface is fully attached. + # + # NOTE: + # In Azure hyperv_vf_delay should be configured to a + # large value, e.g. 120 seconds, to avoid racing cloud + # agent goofs. + # + test $hyperv_vf_delay -ge 2 > /dev/null 2>&1 + if [ $? -ne 0 ] + then + hyperv_vf_delay=2 + fi + # + # NOTE: + # "(sleep ..; ifconfig .. up) > /dev/null 2>&1 &" + # does _not_ work. + # + daemon -f /usr/libexec/hyperv/hyperv_vfattach \ + $iface $hyperv_vf_delay + break + fi +done Copied: stable/11/contrib/hyperv/tools/scripts/hyperv_vfup (from r321762, head/contrib/hyperv/tools/scripts/hyperv_vfup) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/contrib/hyperv/tools/scripts/hyperv_vfup Mon Aug 7 02:49:26 2017 (r322134, copy of r321762, head/contrib/hyperv/tools/scripts/hyperv_vfup) @@ -0,0 +1,119 @@ +#!/bin/sh + +. /etc/rc.subr +. /etc/network.subr + +load_rc_config netif + +# +# Customized per-interface setup, e.g. hyperv_vfup.hn1 +# +# NOTE-CUSTOMIZE: +# Comment this out, if this script is used as template +# for the customized per-interface setup. +# +if [ -f /usr/libexec/hyperv/hyperv_vfup.$1 ] +then + /usr/libexec/hyperv/hyperv_vfup.$1 + exit $? +fi + +# NOTE-CUSTOMIZE: +#hn=${0##*.} +hn=$1 +hn_unit=`echo $hn | sed 's/[^0-9]*//g'` + +vf=`sysctl -n dev.hn.$hn_unit.vf` +if [ ! $vf ] +then + # Race happened; VF was removed, before we ran. + echo "$hn: VF was detached" + exit 0 +fi + +# +# Create laggX for hnX. +# Add VF and hnX to laggX. +# + +lagg=lagg$hn_unit + +ifconfig $lagg > /dev/null 2>&1 +if [ $? -ne 0 ] +then + # + # No laggX, create it now. + # + ifconfig $lagg create > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "$lagg creation failed" + exit 1 + fi + + # + # Configure laggX (failover), add hnX and VF to it. + # + ifconfig $lagg laggproto failover laggport $hn laggport $vf + ifconfig $lagg inet6 no_dad + + # + # Stop dhclient on hnX, if any. + # + pidfile=/var/run/dhclient.$hn.pid + if [ -f $pidfile ] + then + kill -TERM `cat $pidfile` + fi + + # + # Remove all configured IPv4 addresses on hnX, e.g. + # configured by dhclient. laggX will take over the + # network operations. + # + while true + do + ifconfig $hn -alias > /dev/null 2>&1 + if [ $? -ne 0 ] + then + break + fi + done + + # TODO: Remove IPv6 addresses on hnX + + # + # Use hnX's configuration for laggX + # + # NOTE-CUSTOMIZE: + # If this script is used as template for the customized + # per-interface setup, replace this with whatever you + # want to do with the laggX. + # + if dhcpif $hn; + then + ifconfig $lagg up + if syncdhcpif $hn; + then + dhclient $lagg + else + dhclient -b $lagg + fi + else + ifconfig_args=`ifconfig_getargs $hn` + if [ -n "$ifconfig_args" ] + then + ifconfig $lagg $ifconfig_args + fi + fi +else + # + # laggX exists. Check whether VF was there or not. + # If VF was not added to laggX, add it now. + # + ifconfig $lagg | grep "laggport: $vf" > /dev/null 2>&1 + if [ $? -ne 0 ] + then + ifconfig $lagg laggport $vf + fi +fi Modified: stable/11/etc/devd/hyperv.conf ============================================================================== --- stable/11/etc/devd/hyperv.conf Mon Aug 7 02:33:22 2017 (r322133) +++ stable/11/etc/devd/hyperv.conf Mon Aug 7 02:49:26 2017 (r322134) @@ -33,3 +33,76 @@ notify 11 { match "cdev" "hv_fsvss_dev"; action "pkill -x hv_vss_daemon"; }; + +# +# Rules for non-transparent network VF. +# +# How network VF works with hn(4) on Hyper-V in non-transparent mode: +# +# - Each network VF has a cooresponding hn(4). +# - The network VF and the it's cooresponding hn(4) have the same hardware +# address. +# - Once the network VF is up, e.g. ifconfig VF up: +# o All of the transmission should go through the network VF. +# o Most of the reception goes through the network VF. +# o Small amount of reception may go through the cooresponding hn(4). +# This reception will happen, even if the the cooresponding hn(4) is +# down. The cooresponding hn(4) will change the reception interface +# to the network VF, so that network layer and application layer will +# be tricked into thinking that these packets were received by the +# network VF. +# o The cooresponding hn(4) pretends the physical link is down. +# - Once the network VF is down or detached: +# o All of the transmission should go through the cooresponding hn(4). +# o All of the reception goes through the cooresponding hn(4). +# o The cooresponding hn(4) fallbacks to the original physical link +# detection logic. +# +# All these features are mainly used to help live migration, during which +# the network VF will be detached, while the network communication to the +# VM must not be cut off. In order to reach this level of live migration +# transparency, we use failover mode lagg(4) with the network VF and the +# cooresponding hn(4) attached to it. +# +# To ease user configuration for both network VF and non-network VF, the +# lagg(4) will be created by the following rules, and the configuration +# of the cooresponding hn(4) will be applied to the lagg(4) automatically. +# +# NOTE: +# If live migration is not needed at all, the following rules could be +# commented out, and the network VF interface could be used exclusively. +# Most often the cooresponding hn(4) could be completely ignored. +# +# +# Default workflow for the network VF bringup: +# 1) ETHERNET/IFATTACH -> VF interface up (delayed by rc.conf hyperv_vf_delay +# seconds). This operation will trigger HYPERV_NIC_VF/VF_UP. +# 2) HYPERV_NIC_VF/VF_UP: +# a) Create laggX coresponding to hnX. +# b) Add hnX and VF to laggX. +# c) Whack all previous network configuration on hnX, including stopping +# dhclient. +# d) Apply rc.conf ifconfig_hnX to laggX; i.e. including starting dhclient. +# +# NOTE: +# HYPERV_NIC_VF/VF_UP action script could be customized per-interface by +# adding /usr/libexec/hyperv/hyperv_vfup.hnX script. +# /usr/libexec/hyperv/hyperv_vfup could be used as the template for the +# customized per-interface script. +# +# NOTE: +# For transparent network VF, hyperv_vfattach does nothing and +# HYPERV_NIC_VF/VF_UP will not be triggered at all. +# + +notify 10 { + match "system" "HYPERV_NIC_VF"; + match "type" "VF_UP"; + action "/usr/libexec/hyperv/hyperv_vfup $subsystem"; +}; + +notify 10 { + match "system" "ETHERNET"; + match "type" "IFATTACH"; + action "/usr/libexec/hyperv/hyperv_vfattach $subsystem 0"; +}; Modified: stable/11/libexec/hyperv/Makefile ============================================================================== --- stable/11/libexec/hyperv/Makefile Mon Aug 7 02:33:22 2017 (r322133) +++ stable/11/libexec/hyperv/Makefile Mon Aug 7 02:49:26 2017 (r322134) @@ -5,5 +5,6 @@ BINDIR= ${LIBEXECDIR}/hyperv SCRIPTS= hv_set_ifconfig hv_get_dns_info hv_get_dhcp_info +SCRIPTS+= hyperv_vfattach hyperv_vfup .include