From owner-svn-src-all@freebsd.org  Thu Oct 24 20:08:39 2019
Return-Path: <owner-svn-src-all@freebsd.org>
Delivered-To: svn-src-all@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5719715932C;
 Thu, 24 Oct 2019 20:08:39 +0000 (UTC) (envelope-from bz@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
 [IPv6:2610:1c1:1:606c::19:3])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "mxrelay.nyi.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46zdbq2JRHz4YtV;
 Thu, 24 Oct 2019 20:08:39 +0000 (UTC) (envelope-from bz@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
 [IPv6:2610:1c1:1:6068::e6a:0])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2DB081B66F;
 Thu, 24 Oct 2019 20:08:39 +0000 (UTC) (envelope-from bz@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
 by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x9OK8d9p037604;
 Thu, 24 Oct 2019 20:08:39 GMT (envelope-from bz@FreeBSD.org)
Received: (from bz@localhost)
 by repo.freebsd.org (8.15.2/8.15.2/Submit) id x9OK8YRY037580;
 Thu, 24 Oct 2019 20:08:34 GMT (envelope-from bz@FreeBSD.org)
Message-Id: <201910242008.x9OK8YRY037580@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: bz set sender to bz@FreeBSD.org
 using -f
From: "Bjoern A. Zeeb" <bz@FreeBSD.org>
Date: Thu, 24 Oct 2019 20:08:34 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
 svn-src-head@freebsd.org
Subject: svn commit: r354045 - head/tests/sys/netinet6/frag6
X-SVN-Group: head
X-SVN-Commit-Author: bz
X-SVN-Commit-Paths: head/tests/sys/netinet6/frag6
X-SVN-Commit-Revision: 354045
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
 user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all/>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 24 Oct 2019 20:08:39 -0000

Author: bz
Date: Thu Oct 24 20:08:33 2019
New Revision: 354045
URL: https://svnweb.freebsd.org/changeset/base/354045

Log:
  frag6 test cases:  check more counters, wait for expiry
  
  When done with tests check that both the per-VNET and the global-fragmented-
  packets-in-system counters are zero to make sure we do not leak counters or
  queue entries.
  
  This implies that for all test cases we either have to check for the ICMPv6
  packet sent in case of TLL=0 expiry (if it is sent) or sleep at least long
  enough for the TTL to expire for all packets (e.g., fragments where we do not
  have the off=0 packet).
  
  This also means that statistics are now updated to include all the expired
  packets.
  
  There are cases when we do not check for counters to be zero and this is
  when testing VNET teardown to behave properly and not panic, when we are
  intentionally leaving fragments in the system.
  
  MFC after:	3 weeks
  Sponsored by:	Netflix

Modified:
  head/tests/sys/netinet6/frag6/frag6_01.sh
  head/tests/sys/netinet6/frag6/frag6_02.sh
  head/tests/sys/netinet6/frag6/frag6_03.py
  head/tests/sys/netinet6/frag6/frag6_03.sh
  head/tests/sys/netinet6/frag6/frag6_04.sh
  head/tests/sys/netinet6/frag6/frag6_05.py
  head/tests/sys/netinet6/frag6/frag6_05.sh
  head/tests/sys/netinet6/frag6/frag6_06.sh
  head/tests/sys/netinet6/frag6/frag6_07.py
  head/tests/sys/netinet6/frag6/frag6_07.sh
  head/tests/sys/netinet6/frag6/frag6_08.py
  head/tests/sys/netinet6/frag6/frag6_08.sh
  head/tests/sys/netinet6/frag6/frag6_09.sh
  head/tests/sys/netinet6/frag6/frag6_10.py
  head/tests/sys/netinet6/frag6/frag6_10.sh
  head/tests/sys/netinet6/frag6/frag6_11.sh
  head/tests/sys/netinet6/frag6/frag6_12.sh
  head/tests/sys/netinet6/frag6/frag6_13.py
  head/tests/sys/netinet6/frag6/frag6_13.sh
  head/tests/sys/netinet6/frag6/frag6_14.py
  head/tests/sys/netinet6/frag6/frag6_14.sh
  head/tests/sys/netinet6/frag6/frag6_15.sh
  head/tests/sys/netinet6/frag6/frag6_16.sh

Modified: head/tests/sys/netinet6/frag6/frag6_01.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_01.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_01.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_01_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_02.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_02.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_02.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_02_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_03.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_03.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_03.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -82,20 +82,21 @@ def main():
 
 	########################################################################
 	#
-	# (1) Atomic fragment.
+	# Atomic fragment.
 	#
 	# A:  Nothing listening on UDP port.
 	# R:  ICMPv6 dst unreach, unreach port.
 	#
 	ip6f01 = sp.Ether() / \
 		sp.IPv6(src=args.src[0], dst=args.to[0]) / \
-		sp.IPv6ExtHdrFragment(offset=0, m=0, id=1) / \
+		sp.IPv6ExtHdrFragment(offset=0, m=0, id=3) / \
 		sp.UDP(dport=3456, sport=6543)
 	if args.debug :
 		ip6f01.display()
 	sp.sendp(ip6f01, iface=args.sendif[0], verbose=False)
 
 	sleep(0.10)
+	sniffer.setEnd()
 	sniffer.join()
 	if not sniffer.foundCorrectPacket:
 		sys.exit(1)

Modified: head/tests/sys/netinet6/frag6/frag6_03.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_03.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_03.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_03_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_04.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_04.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_04.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_04_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_05.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_05.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_05.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -76,6 +76,8 @@ def main():
 			ip6f01.display()
 		sp.sendp(ip6f01, iface=args.sendif[0], verbose=False)
 
+	# Wait for possible expiry to happen.
+	sleep(75)
 	sys.exit(0)
 
 if __name__ == '__main__':

Modified: head/tests/sys/netinet6/frag6/frag6_05.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_05.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_05.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -47,6 +47,17 @@ frag6_05_check_stats_0() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check that the sysctl is set to what we expect.
 	#
@@ -229,6 +240,16 @@ frag6_05_check_stats_1() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check that the sysctl is set to what we expect.
@@ -272,7 +293,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>20</received-fragments>
     <dropped-fragment>10</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>10</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>
@@ -300,7 +321,7 @@ EOF
 	# XXX-TODO check output histogram (just too hard to parse [no multi-line-grep])
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
-    <icmp6-calls>0</icmp6-calls>
+    <icmp6-calls>10</icmp6-calls>
       <no-route>0</no-route>
       <admin-prohibited>0</admin-prohibited>
       <beyond-scope>0</beyond-scope>
@@ -308,7 +329,7 @@ EOF
       <port-unreachable>0</port-unreachable>
       <packet-too-big>0</packet-too-big>
       <time-exceed-transmit>0</time-exceed-transmit>
-      <time-exceed-reassembly>0</time-exceed-reassembly>
+      <time-exceed-reassembly>10</time-exceed-reassembly>
       <bad-header>0</bad-header>
       <bad-next-header>0</bad-next-header>
       <bad-option>0</bad-option>
@@ -372,10 +393,10 @@ EOF
     <received-echo-replies>0</received-echo-replies>
     <received-router-solicitation>0</received-router-solicitation>
     <received-router-advertisement>0</received-router-advertisement>
-    <sent-errors>0</sent-errors>
+    <sent-errors>10</sent-errors>
     <sent-destination-unreachable>0</sent-destination-unreachable>
     <sent-admin-prohibited>0</sent-admin-prohibited>
-    <sent-time-exceeded>0</sent-time-exceeded>
+    <sent-time-exceeded>10</sent-time-exceeded>
     <sent-bad-parameter>0</sent-bad-parameter>
     <sent-packet-too-big>0</sent-packet-too-big>
     <sent-echo-requests>0</sent-echo-requests>

Modified: head/tests/sys/netinet6/frag6/frag6_06.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_06.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_06.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -61,6 +61,17 @@ frag6_06_check_stats_0() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check that the sysctl is set to what we expect.
 	#

Modified: head/tests/sys/netinet6/frag6/frag6_07.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_07.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_07.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -54,6 +54,25 @@ def check_icmp6_error(args, packet):
 	#icmp6.display()
 	return True
 
+def check_icmp6_error_2(args, packet):
+	ip6 = packet.getlayer(sp.IPv6)
+	if not ip6:
+		return False
+	oip6 = sp.IPv6(src=args.src[0], dst=args.to[0])
+	if ip6.dst != oip6.src:
+		return False
+	icmp6 = packet.getlayer(sp.ICMPv6TimeExceeded)
+	if not icmp6:
+		return False
+	# ICMP6_TIME_EXCEED_REASSEMBLY 1
+	if icmp6.code != 1:
+		return False
+	# Should we check the payload as well?
+	# We are running in a very isolated environment and nothing else
+	# should trigger an ICMPv6 Time Exceeded / Frag reassembly so leave it.
+	#icmp6.display()
+	return True
+
 def main():
 	parser = argparse.ArgumentParser("frag6.py",
 		description="IPv6 fragementation test tool")
@@ -78,6 +97,7 @@ def main():
 
 	# Start sniffing on recvif
 	sniffer = Sniffer(args, check_icmp6_error)
+	sniffer2 = Sniffer(args, check_icmp6_error_2)
 
 
 	########################################################################
@@ -88,6 +108,7 @@ def main():
 	#
 	# A:  Reassembly failure.
 	# R:  ICMPv6 param prob, param header.
+	# R:  ICMPv6 timeout (1st frag, off=0)
 	#
 	data = "6" * 1280
 	ip6f01 = \
@@ -142,6 +163,13 @@ def main():
 	sniffer.setEnd()
 	sniffer.join()
 	if not sniffer.foundCorrectPacket:
+		sys.exit(1)
+
+	# Wait for expiry from first test run.
+	sleep(75)
+	sniffer2.setEnd()
+	sniffer2.join()
+	if not sniffer2.foundCorrectPacket:
 		sys.exit(1)
 
 	sys.exit(0)

Modified: head/tests/sys/netinet6/frag6/frag6_07.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_07.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_07.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -47,6 +47,17 @@ frag6_07_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check selection of global UDP stats.
 	#
@@ -81,7 +92,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>3</received-fragments>
     <dropped-fragment>0</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>1</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>
@@ -109,7 +120,7 @@ EOF
 	# XXX-TODO check output histogram (just too hard to parse [no multi-line-grep])
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
-    <icmp6-calls>2</icmp6-calls>
+    <icmp6-calls>3</icmp6-calls>
       <no-route>0</no-route>
       <admin-prohibited>0</admin-prohibited>
       <beyond-scope>0</beyond-scope>
@@ -117,7 +128,7 @@ EOF
       <port-unreachable>0</port-unreachable>
       <packet-too-big>0</packet-too-big>
       <time-exceed-transmit>0</time-exceed-transmit>
-      <time-exceed-reassembly>0</time-exceed-reassembly>
+      <time-exceed-reassembly>1</time-exceed-reassembly>
       <bad-header>2</bad-header>
       <bad-next-header>0</bad-next-header>
       <bad-option>0</bad-option>
@@ -181,10 +192,10 @@ EOF
     <received-echo-replies>0</received-echo-replies>
     <received-router-solicitation>0</received-router-solicitation>
     <received-router-advertisement>0</received-router-advertisement>
-    <sent-errors>2</sent-errors>
+    <sent-errors>3</sent-errors>
     <sent-destination-unreachable>0</sent-destination-unreachable>
     <sent-admin-prohibited>0</sent-admin-prohibited>
-    <sent-time-exceeded>0</sent-time-exceeded>
+    <sent-time-exceeded>1</sent-time-exceeded>
     <sent-bad-parameter>2</sent-bad-parameter>
     <sent-packet-too-big>0</sent-packet-too-big>
     <sent-echo-requests>0</sent-echo-requests>

Modified: head/tests/sys/netinet6/frag6/frag6_08.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_08.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_08.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -54,6 +54,26 @@ def check_icmp6_error(args, packet):
 	#icmp6.display()
 	return True
 
+def check_icmp6_error_2(args, packet):
+	ip6 = packet.getlayer(sp.IPv6)
+	if not ip6:
+		return False
+	oip6 = sp.IPv6(src=args.src[0], dst=args.to[0])
+	if ip6.dst != oip6.src:
+		return False
+	icmp6 = packet.getlayer(sp.ICMPv6TimeExceeded)
+	if not icmp6:
+		return False
+	# ICMP6_TIME_EXCEED_REASSEMBLY 1
+	if icmp6.code != 1:
+		return False
+	# Should we check the payload as well?
+	# We are running in a very isolated environment and nothing else
+	# should trigger an ICMPv6 Time Exceeded / Frag reassembly so leave it.
+	#icmp6.display()
+	return True
+
+
 def main():
 	parser = argparse.ArgumentParser("frag6.py",
 		description="IPv6 fragementation test tool")
@@ -78,15 +98,20 @@ def main():
 
 	# Start sniffing on recvif
 	sniffer = Sniffer(args, check_icmp6_error)
+	sniffer2 = Sniffer(args, check_icmp6_error_2)
 
 
 	########################################################################
 	#
 	# A fragment with payload and offset set to add up to >64k when
 	# another frag with offset=0 arrives and has an unfrag part.
+	# This is us checking for all fragments queued already when the
+	# one with off=0 arrives.  Note:  unless the off=0 has its own problem
+	# it will be queued and off!=0 ones might be expunged with param prob.
 	#
-	# A:  Reassembly failure (timeout) after
-	# R:  ICMPv6 param prob, param header (earlier).
+	# A:  Reassembly failure, timeout after
+	# R:  ICMPv6 param prob, param header (1st frag)
+	# R:  ICMPv6 time exceeded (2nd frag, as off=0)
 	#
 	data = "6" * 15
 	ip6f01 = \
@@ -114,6 +139,11 @@ def main():
 	sniffer.setEnd()
 	sniffer.join()
 	if not sniffer.foundCorrectPacket:
+		sys.exit(1)
+	sleep(75)
+	sniffer2.setEnd()
+	sniffer2.join()
+	if not sniffer2.foundCorrectPacket:
 		sys.exit(1)
 
 	sys.exit(0)

Modified: head/tests/sys/netinet6/frag6/frag6_08.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_08.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_08.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -47,6 +47,17 @@ frag6_08_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check selection of global UDP stats.
 	#
@@ -81,7 +92,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>2</received-fragments>
     <dropped-fragment>0</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>1</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>
@@ -109,7 +120,7 @@ EOF
 	# XXX-TODO check output histogram (just too hard to parse [no multi-line-grep])
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
-    <icmp6-calls>1</icmp6-calls>
+    <icmp6-calls>2</icmp6-calls>
       <no-route>0</no-route>
       <admin-prohibited>0</admin-prohibited>
       <beyond-scope>0</beyond-scope>
@@ -117,7 +128,7 @@ EOF
       <port-unreachable>0</port-unreachable>
       <packet-too-big>0</packet-too-big>
       <time-exceed-transmit>0</time-exceed-transmit>
-      <time-exceed-reassembly>0</time-exceed-reassembly>
+      <time-exceed-reassembly>1</time-exceed-reassembly>
       <bad-header>1</bad-header>
       <bad-next-header>0</bad-next-header>
       <bad-option>0</bad-option>
@@ -181,10 +192,10 @@ EOF
     <received-echo-replies>0</received-echo-replies>
     <received-router-solicitation>0</received-router-solicitation>
     <received-router-advertisement>0</received-router-advertisement>
-    <sent-errors>1</sent-errors>
+    <sent-errors>2</sent-errors>
     <sent-destination-unreachable>0</sent-destination-unreachable>
     <sent-admin-prohibited>0</sent-admin-prohibited>
-    <sent-time-exceeded>0</sent-time-exceeded>
+    <sent-time-exceeded>1</sent-time-exceeded>
     <sent-bad-parameter>1</sent-bad-parameter>
     <sent-packet-too-big>0</sent-packet-too-big>
     <sent-echo-requests>0</sent-echo-requests>

Modified: head/tests/sys/netinet6/frag6/frag6_09.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_09.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_09.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_09_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_10.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_10.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_10.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -72,6 +72,8 @@ def main():
 	sp.sendp(ip6f01, iface=args.sendif[0], verbose=False)
 
 	# We do not generate ICMPv6 for non-off=0-segments.
+	# Wait for expiry.
+	sleep(75)
 
 	sys.exit(0)
 

Modified: head/tests/sys/netinet6/frag6/frag6_10.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_10.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_10.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_10_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.
@@ -77,8 +87,8 @@ EOF
 
 	#
 	# Check selection of global IPv6 stats.
-	# We do not seem to sent a timeout ICMPv6 for this one?
-	# No, as it is not an off=0 segment.
+	# We do not sent a timeout ICMPv6 for this one
+	# as it is not an off=0 segment.
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
     <dropped-below-minimum-size>0</dropped-below-minimum-size>
@@ -87,7 +97,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>1</received-fragments>
     <dropped-fragment>0</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>1</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>

Modified: head/tests/sys/netinet6/frag6/frag6_11.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_11.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_11.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_11_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_12.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_12.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_12.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_12_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.

Modified: head/tests/sys/netinet6/frag6/frag6_13.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_13.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_13.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -114,6 +114,8 @@ def main():
 	sp.sendp(ip6f01, iface=args.sendif[0], verbose=False)
 	sp.sendp(ip6f02, iface=args.sendif[0], verbose=False)
 
+	# Wait for expiry.
+	sleep(75)
 	sys.exit(0)
 
 if __name__ == '__main__':

Modified: head/tests/sys/netinet6/frag6/frag6_13.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_13.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_13.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_13_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.
@@ -85,7 +95,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>4</received-fragments>
     <dropped-fragment>2</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>2</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>
@@ -112,7 +122,7 @@ EOF
 	# Check selection of global ICMPv6 stats.
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
-    <icmp6-calls>0</icmp6-calls>
+    <icmp6-calls>2</icmp6-calls>
       <no-route>0</no-route>
       <admin-prohibited>0</admin-prohibited>
       <beyond-scope>0</beyond-scope>
@@ -120,7 +130,7 @@ EOF
       <port-unreachable>0</port-unreachable>
       <packet-too-big>0</packet-too-big>
       <time-exceed-transmit>0</time-exceed-transmit>
-      <time-exceed-reassembly>0</time-exceed-reassembly>
+      <time-exceed-reassembly>2</time-exceed-reassembly>
       <bad-header>0</bad-header>
       <bad-next-header>0</bad-next-header>
       <bad-option>0</bad-option>
@@ -184,10 +194,10 @@ EOF
     <received-echo-replies>0</received-echo-replies>
     <received-router-solicitation>0</received-router-solicitation>
     <received-router-advertisement>0</received-router-advertisement>
-    <sent-errors>0</sent-errors>
+    <sent-errors>2</sent-errors>
     <sent-destination-unreachable>0</sent-destination-unreachable>
     <sent-admin-prohibited>0</sent-admin-prohibited>
-    <sent-time-exceeded>0</sent-time-exceeded>
+    <sent-time-exceeded>2</sent-time-exceeded>
     <sent-bad-parameter>0</sent-bad-parameter>
     <sent-packet-too-big>0</sent-packet-too-big>
     <sent-echo-requests>0</sent-echo-requests>

Modified: head/tests/sys/netinet6/frag6/frag6_14.py
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_14.py	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_14.py	Thu Oct 24 20:08:33 2019	(r354045)
@@ -129,6 +129,8 @@ def main():
 	sp.sendp(ip6f01, iface=args.sendif[0], verbose=False)
 	sp.sendp(ip6f02, iface=args.sendif[0], verbose=False)
 
+	# Wait for expiry.
+	sleep(75)
 	sys.exit(0)
 
 if __name__ == '__main__':

Modified: head/tests/sys/netinet6/frag6/frag6_14.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_14.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_14.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -52,6 +52,16 @@ frag6_14_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
 
 	#
 	# Check selection of global UDP stats.
@@ -85,7 +95,7 @@ EOF
     <dropped-bad-version>0</dropped-bad-version>
     <received-fragments>6</received-fragments>
     <dropped-fragment>2</dropped-fragment>
-    <dropped-fragment-after-timeout>0</dropped-fragment-after-timeout>
+    <dropped-fragment-after-timeout>4</dropped-fragment-after-timeout>
     <dropped-fragments-overflow>0</dropped-fragments-overflow>
     <atomic-fragments>0</atomic-fragments>
     <reassembled-packets>0</reassembled-packets>
@@ -112,7 +122,7 @@ EOF
 	# Check selection of global ICMPv6 stats.
 	#
 	cat <<EOF > ${HOME}/filter-${jname}.txt
-    <icmp6-calls>0</icmp6-calls>
+    <icmp6-calls>1</icmp6-calls>
       <no-route>0</no-route>
       <admin-prohibited>0</admin-prohibited>
       <beyond-scope>0</beyond-scope>
@@ -120,7 +130,7 @@ EOF
       <port-unreachable>0</port-unreachable>
       <packet-too-big>0</packet-too-big>
       <time-exceed-transmit>0</time-exceed-transmit>
-      <time-exceed-reassembly>0</time-exceed-reassembly>
+      <time-exceed-reassembly>1</time-exceed-reassembly>
       <bad-header>0</bad-header>
       <bad-next-header>0</bad-next-header>
       <bad-option>0</bad-option>
@@ -184,10 +194,10 @@ EOF
     <received-echo-replies>0</received-echo-replies>
     <received-router-solicitation>0</received-router-solicitation>
     <received-router-advertisement>0</received-router-advertisement>
-    <sent-errors>0</sent-errors>
+    <sent-errors>1</sent-errors>
     <sent-destination-unreachable>0</sent-destination-unreachable>
     <sent-admin-prohibited>0</sent-admin-prohibited>
-    <sent-time-exceeded>0</sent-time-exceeded>
+    <sent-time-exceeded>1</sent-time-exceeded>
     <sent-bad-parameter>0</sent-bad-parameter>
     <sent-packet-too-big>0</sent-packet-too-big>
     <sent-echo-requests>0</sent-echo-requests>

Modified: head/tests/sys/netinet6/frag6/frag6_15.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_15.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_15.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -61,6 +61,17 @@ frag6_15_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check that the sysctl is set to what we expect.
 	#

Modified: head/tests/sys/netinet6/frag6/frag6_16.sh
==============================================================================
--- head/tests/sys/netinet6/frag6/frag6_16.sh	Thu Oct 24 20:05:10 2019	(r354044)
+++ head/tests/sys/netinet6/frag6/frag6_16.sh	Thu Oct 24 20:08:33 2019	(r354045)
@@ -48,6 +48,17 @@ frag6_16_check_stats() {
 	# The Python script has to wait for this already to get the ICMPv6
 	# hence we do not sleep here anymore.
 
+	nf=`jexec ${jname} sysctl -n net.inet6.ip6.frag6_nfragpackets`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "VNET frag6_nfragpackets not 0 but: ${nf}" ;;
+	esac
+	nf=`sysctl -n net.inet6.ip6.frag6_nfrags`
+	case ${nf} in
+	0)	break ;;
+	*)	atf_fail "Global frag6_nfrags not 0 but: ${nf}" ;;
+	esac
+
 	#
 	# Check selection of global UDP stats.
 	#