From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 07:29:19 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70807106566B for ; Sun, 22 Jan 2012 07:29:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 28CA98FC0C for ; Sun, 22 Jan 2012 07:29:18 +0000 (UTC) Received: by vcbfl17 with SMTP id fl17so1951858vcb.13 for ; Sat, 21 Jan 2012 23:29:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=JPn6blT3YaXhLXbItrEOrqfVLM0WvapPEOcrsgF/Wxs=; b=uqflAVJ4osyiMuRBRLbq1x+QrJZYISqkhUE+yPyYb6EnDX1l4P+zUYqw1W26nSDem8 WHoMOCHsSBmvkj1zA6jHH10n8TJ1udI+WYXKuk0+0aUGXgwUh+6zv0Mug6Qwcj+w9VJD 5sWHtiijVbTNhYbshafl6fJiFhkmz5/CgPWfA= MIME-Version: 1.0 Received: by 10.220.156.134 with SMTP id x6mr2112064vcw.17.1327217356933; Sat, 21 Jan 2012 23:29:16 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Sat, 21 Jan 2012 23:29:16 -0800 (PST) In-Reply-To: <4F1ABF9C.5010608@gmx.com> References: <4F1ABF9C.5010608@gmx.com> Date: Sat, 21 Jan 2012 23:29:16 -0800 X-Google-Sender-Auth: 1MZ00kHebbQbyedoXMmpwkwwrIs Message-ID: From: Adrian Chadd To: Nikos Vassiliadis Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: STP id selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 07:29:19 -0000 On 21 January 2012 05:37, Nikos Vassiliadis wrote: > Hi, > > The current code in bridgestp.c finds the lower MAC address from all > available ethernets and uses it as the STP id. This is problematic when > more than one STP bridges participate in the same STP domain because > more than one bridges will use the same id. A similar fix was applied > to the OpenBSD version of the code[1]. Could you review the attached > patch? > > 1.http://www.openbsd.org/cgi-**bin/cvsweb/src/sys/net/** > bridgestp.c?rev=1.33 > > Thanks, Nikos > > .. that sounds sensible enough. Sure, please create a PR and then tell me what the ID is. I keep finding myself knee deep in the bridge code so this is something I can test out locally. Adrian From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 11:47:44 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7B281065670 for ; Sun, 22 Jan 2012 11:47:44 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mailout-us.mail.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 136C08FC13 for ; Sun, 22 Jan 2012 11:47:44 +0000 (UTC) Received: (qmail invoked by alias); 22 Jan 2012 11:47:42 -0000 Received: from unknown (EHLO [192.168.73.192]) [91.140.99.89] by mail.gmx.com (mp-us008) with SMTP; 22 Jan 2012 06:47:42 -0500 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX19ZfLpPVRVZ6N+oMnnw00sdLFBAQMe7Ta0FpljBJA /wfBYBjGqVTriS Message-ID: <4F1BF757.7090701@gmx.com> Date: Sun, 22 Jan 2012 13:47:35 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 MIME-Version: 1.0 To: Adrian Chadd References: <4F1ABF9C.5010608@gmx.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: freebsd-net@freebsd.org Subject: Re: STP id selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 11:47:44 -0000 On 1/22/2012 9:29 AM, Adrian Chadd wrote: > On 21 January 2012 05:37, Nikos Vassiliadis wrote: > >> Hi, >> >> The current code in bridgestp.c finds the lower MAC address from all >> available ethernets and uses it as the STP id. This is problematic when >> more than one STP bridges participate in the same STP domain because >> more than one bridges will use the same id. A similar fix was applied >> to the OpenBSD version of the code[1]. Could you review the attached >> patch? >> >> 1.http://www.openbsd.org/cgi-**bin/cvsweb/src/sys/net/** >> bridgestp.c?rev=1.33 >> >> Thanks, Nikos >> >> > .. that sounds sensible enough. Sure, please create a PR and then tell me > what the ID is. > > I keep finding myself knee deep in the bridge code so this is something I > can test out locally. > Here it is: http://www.freebsd.org/cgi/query-pr.cgi?pr=164369 Thanks for looking into this, Nikos From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 15:31:09 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 726AD106564A; Sun, 22 Jan 2012 15:31:09 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id DAFAD8FC08; Sun, 22 Jan 2012 15:31:08 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 7BC85451C; Sun, 22 Jan 2012 15:31:07 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Stefan Bethke In-Reply-To: <20120120221319.ca8b631f.ray@freebsd.org> Date: Sun, 22 Jan 2012 16:31:06 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> To: Aleksandr Rybalko X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-net@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 15:31:09 -0000 Am 20.01.2012 um 21:13 schrieb Aleksandr Rybalko: > It include sys/mips/conf/AR7240, that together with hints file is good > example for typical AR7240 setup. I=C4m heaving trouble getting this to work. The patch applies cleanly = and I can get a kernel compiled and booted, but neither arge0 nor arge1 = appear to be functional. I had to roll my own kernel config as your = AR7240 hangs before printing anything on my TL-MR3420. dmesg and devinfo below. Stefan CPU platform: Atheros AR7241 rev 1 CPU Frequency=3D400 MHz CPU DDR Frequency=3D400 MHz CPU AHB Frequency=3D200 MHz platform frequency: 400000000 arguments:=20 a0 =3D 00000008 a1 =3D a1f87fb0 a2 =3D a1f88470 a3 =3D 00000004 Cmd line:argv is invalid Environment: envp is invalid Cache info: picache_stride =3D 4096 picache_loopcount =3D 16 pdcache_stride =3D 4096 pdcache_loopcount =3D 8 cpu0: MIPS Technologies processor v116.147 MMU: Standard TLB, 16 entries L1 i-cache: 4 ways of 512 sets, 32 bytes per line L1 d-cache: 4 ways of 256 sets, 32 bytes per line Config1=3D0x9ee3519e Config3=3D0x20 KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #1: Thu Jan 1 01:00:00 CET 1970 = stb@dummy:/home/stb/working/fe/obj/mipseb/mips.mipseb/home/stb/working/fe/= freebsd/sys/TL-MR3420D mips WARNING: WITNESS option enabled, expect reduced performance. real memory =3D 33554432 (32768K bytes) avail memory =3D 25567232 (24MB) random device not loaded; using insecure entropy nexus0: nexus0: failed to add child: arge0 nexus0: failed to add child: arge1 clock0: on nexus0 Timecounter "MIPS32" frequency 200000000 Hz quality 800 Event timer "MIPS32" frequency 200000000 Hz quality 800 apb0 at irq 4 on nexus0 uart0: <16550 or compatible> on apb0 uart0: console (115200,n,8,1) gpio0: on apb0 gpio0: [GIANT-LOCKED] gpio0: function_set: 0x0 gpio0: function_clear: 0x0 gpio0: gpio pinmask=3D0x1943 gpioc0: on gpio0 gpiobus0: on gpio0 gpioled0: at pin(s) 0 on gpiobus0 gpioled1: at pin(s) 1 on gpiobus0 gpioled2: at pin(s) 3 on gpiobus0 ehci0: at mem = 0x1b000100-0x1bffffff irq 1 on nexus0 usbus0: set host controller mode usbus0: EHCI version 1.0 usbus0: set host controller mode usbus0: on ehci0 arge0: at mem = 0x19000000-0x19000fff irq 2 on nexus0 arge0: Overriding MAC from EEPROM arge0: No PHY specified, using mask 16 miibus0: on arge0 floatphy0 PHY 0 on miibus0 floatphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, = 1000baseSX, 1000baseSX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, = 1000baseT-FDX-master, auto switch0 PHY 1 on miibus0 switch0: 100baseTX, 100baseTX-FDX, 1000baseSX, 1000baseSX-FDX, = 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master ar8x16_switch0: on switch0 arge0: Ethernet address: ff:ff:ff:ff:ff:ff arge1: at mem = 0x1a000000-0x1a000fff irq 3 on nexus0 arge1: No PHY specified, using mask 15 arge1: Ethernet address: ff:ff:ff:ff:ff:00 spi0: at mem 0x1f000000-0x1f00000f on nexus0 spibus0: on spi0 mx25l0: at cs 0 on spibus0 mx25l0: s25s1032, sector 65536 bytes, 64 sectors ar71xx_wdog0: on nexus0 ar71xx_wdog0: Previous reset was due to watchdog timeout Timecounters tick every 1.000 msec usbus0: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on = usbus0 WARNING: WITNESS option enabled, expect reduced performance. uhub0: 1 port with 1 removable, self powered Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 ugen0.2: at usbus0 umass0: on = usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0x0100 umass0:0:0:-1: Attached to scbus0 Trying to mount root from ufs:map/rootfs.uzip []... mountroot: waiting for device map/rootfs.uzip ... da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: Removable Direct Access SCSI-0 = device=20 da0: 40.000MB/s transfers da0: 15252MB (31236096 512 byte sectors: 255H 63S/T 1944C) Trying to mount root from ufs:da0s1a []... warning: no time-of-day clock registered, system time will not be set = accurately Setting hostuuid: aed4c502-193a-11e1-b662-74ea3ae4d920. Setting hostid: 0x6a714343. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/da0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1a: clean, 3849245 free (1085 frags, 481020 blocks, 0.0% = fragmentation) lock order reversal: 1st 0x8095691c ufs (ufs) @ = /home/stb/working/fe/freebsd/sys/kern/vfs_mount.c:1245 2nd 0x80956e94 devfs (devfs) @ = /home/stb/working/fe/freebsd/sys/kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_thread+30 (?,?,?,?) ra c3f597c000000018 sp 0 sz 0 db_trace_self+1c (?,?,?,?) ra c3f597d800000018 sp 0 sz 0 800776d8+34 (?,?,?,?) ra c3f597f0000001a0 sp 0 sz 0 kdb_backtrace+44 (?,?,?,?) ra c3f5999000000018 sp 0 sz 0 801b3bbc+34 (?,?,?,?) ra c3f599a800000020 sp 0 sz 0 witness_checkorder+9cc (?,?,803b1f7c,877) ra c3f599c800000050 sp 0 sz 1 __lockmgr_args+948 (?,?,80956eb4,?) ra c3f59a1800000070 sp 0 sz 1 vop_stdlock+4c (?,?,?,?) ra c3f59a8800000028 sp 0 sz 0 VOP_LOCK1_APV+e4 (?,?,?,?) ra c3f59ab000000020 sp 0 sz 0 _vn_lock+84 (?,?,?,?) ra c3f59ad000000048 sp 0 sz 0 vget+c8 (?,?,?,?) ra c3f59b1800000030 sp 0 sz 0 devfs_allocv+100 (?,?,?,?) ra c3f59b4800000038 sp 0 sz 0 800ba9b4+4c (?,?,?,?) ra c3f59b8000000028 sp 0 sz 0 vflush+6c (?,?,0,80991300) ra c3f59ba8000000f8 sp 0 sz 1 800baa74+54 (?,?,?,?) ra c3f59ca000000028 sp 0 sz 0 dounmount+3f0 (809e8000,?,?,?) ra c3f59cc800000050 sp 100000000 sz 0 sys_unmount+39c (?,?,?,?) ra c3f59d18000000b0 sp 0 sz 0 trap+7f4 (?,?,?,?) ra c3f59dc8000000b8 sp 0 sz 0 MipsUserGenException+10c (?,?,?,404b53c0) ra c3f59e8000000000 sp 0 sz 0 pid 70 Mounting local file systems:. Setting hostname: whitebox.lassitu.de. miibus0: mii_mediachg: can't handle non-zero PHY instance 1 floatphy0: found master switch0 miibus0: mii_mediachg: can't handle non-zero PHY instance 1 arge0: link state changed to DOWN Starting Network: lo0 arge0 arge1. lo0: flags=3D8049 metric 0 mtu 16384 options=3D3 inet6 ::1 prefixlen 128=20 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4=20 inet 127.0.0.1 netmask 0xff000000=20 nd6 options=3D21 arge0: flags=3D8843 metric 0 mtu = 1500 options=3D80000 ether ff:ff:ff:ff:ff:ff inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255=20 inet6 fe80::481f:1b46:dbf1:bd78%arge0 prefixlen 64 scopeid 0x2=20= nd6 options=3D29 media: Ethernet autoselect (100baseTX ) status: no carrier arge1: flags=3D8843 metric 0 mtu = 1500 ether ff:ff:ff:ff:ff:00 inet 44.128.65.33 netmask 0xffffffc0 broadcast 44.128.65.63=20 inet6 fe80::481f:1b46:dbf1:bd78%arge1 prefixlen 64 scopeid 0x3=20= nd6 options=3D29 media: Ethernet 100baseTX status: active add net default: gateway 44.128.65.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Mounting NFS file systems:mount_nfs: diesel: hostname nor servname = provided, or not known . Creating and/or trimming log files. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat Setting date via ntp. Error : hostname nor servname provided, or not known 22 Jan 15:01:04 ntpdate[566]: can't find host diesel 22 Jan 15:01:04 ntpdate[566]: no servers can be used, exiting Clearing /tmp (X related). Mounting late file systems:mount_nfs: diesel: hostname nor servname = provided, or not known . Mounting /etc/fstab filesystems failed,Jan 22 15:01:12 init: /bin/sh on = /etc/rc terminated abnormally, going to single user mode Enter full pathname of shell or RETURN for /bin/sh:=20 # devinfo nexus0 clock0 apb0 uart0 gpio0 gpioc0 gpiobus0 gpioled0 gpioled1 gpioled2 ehci0 usbus0 uhub0 umass0 arge0 miibus0 floatphy0 switch0 ar8x16_switch0 arge1 spi0 spibus0 mx25l0 ar71xx_wdog0 #=20 --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 17:51:46 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27722106566C; Sun, 22 Jan 2012 17:51:46 +0000 (UTC) (envelope-from ray@freebsd.org) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 6A4FC8FC12; Sun, 22 Jan 2012 17:51:42 +0000 (UTC) Received: from rnote.ddteam.net (162-84-133-95.pool.ukrtel.net [95.133.84.162]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id E7BBCC493A; Sun, 22 Jan 2012 19:51:40 +0200 (EET) Date: Sun, 22 Jan 2012 19:51:30 +0200 From: Aleksandr Rybalko To: Stefan Bethke Message-Id: <20120122195130.360261ce.ray@freebsd.org> In-Reply-To: <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> Organization: FreeBSD.ORG X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) X-Operating-System: FreeBSD Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: freebsd-net@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 17:51:46 -0000 On Sun, 22 Jan 2012 16:31:06 +0100 Stefan Bethke wrote: > Am 20.01.2012 um 21:13 schrieb Aleksandr Rybalko: > > > It include sys/mips/conf/AR7240, that together with hints file is > > good example for typical AR7240 setup. > > IÄm heaving trouble getting this to work. The patch applies cleanly > and I can get a kernel compiled and booted, but neither arge0 nor > arge1 appear to be functional. I had to roll my own kernel config as > your AR7240 hangs before printing anything on my TL-MR3420. Yeah, I know where is problem, to proper attach switch framework to arge, arge must be regular NIC. Here is the patch for that: http://my.ddteam.net/files/2012-01-22_arge.patch Hope it will apply cleanly. Patch have fixed both arge problems (problem for allocation of ring buffer, and stray interrupts) + remove most phymask bits + whitespace cleanup. Thank you for testing that Stefan. P.S. I can't test clear SoC config on my board, because my board id D-Link DIR-615_E4 with modified U-Boot in it, which able to load only FW images, but not ELF kernel. So I test it with ZRouter.org FW image instead. P.P.S. can you also show me network part of your config and hints files. P.P.P.S. still working on your previous question about subj, already begin work on more wide documentation on wiki, but still far enough :) "http://wiki.freebsd.org/AleksandrRybalko/Switch Framework" > > dmesg and devinfo below. > > > Stefan > > CPU platform: Atheros AR7241 rev 1 > CPU Frequency=400 MHz > CPU DDR Frequency=400 MHz > CPU AHB Frequency=200 MHz > platform frequency: 400000000 > arguments: > a0 = 00000008 > a1 = a1f87fb0 > a2 = a1f88470 > a3 = 00000004 > Cmd line:argv is invalid > Environment: > envp is invalid > Cache info: > picache_stride = 4096 > picache_loopcount = 16 > pdcache_stride = 4096 > pdcache_loopcount = 8 > cpu0: MIPS Technologies processor v116.147 > MMU: Standard TLB, 16 entries > L1 i-cache: 4 ways of 512 sets, 32 bytes per line > L1 d-cache: 4 ways of 256 sets, 32 bytes per line > Config1=0x9ee3519e > Config3=0x20 > KDB: debugger backends: ddb > KDB: current backend: ddb > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, > 1994 The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #1: Thu Jan 1 01:00:00 CET 1970 > stb@dummy:/home/stb/working/fe/obj/mipseb/mips.mipseb/home/stb/working/fe/freebsd/sys/TL-MR3420D > mips WARNING: WITNESS option enabled, expect reduced performance. > real memory = 33554432 (32768K bytes) > avail memory = 25567232 (24MB) > random device not loaded; using insecure entropy > nexus0: > nexus0: failed to add child: arge0 > nexus0: failed to add child: arge1 > clock0: on nexus0 > Timecounter "MIPS32" frequency 200000000 Hz quality 800 > Event timer "MIPS32" frequency 200000000 Hz quality 800 > apb0 at irq 4 on nexus0 > uart0: <16550 or compatible> on apb0 > uart0: console (115200,n,8,1) > gpio0: on apb0 > gpio0: [GIANT-LOCKED] > gpio0: function_set: 0x0 > gpio0: function_clear: 0x0 > gpio0: gpio pinmask=0x1943 > gpioc0: on gpio0 > gpiobus0: on gpio0 > gpioled0: at pin(s) 0 on gpiobus0 > gpioled1: at pin(s) 1 on gpiobus0 > gpioled2: at pin(s) 3 on gpiobus0 > ehci0: at mem > 0x1b000100-0x1bffffff irq 1 on nexus0 usbus0: set host controller mode > usbus0: EHCI version 1.0 > usbus0: set host controller mode > usbus0: on ehci0 > arge0: at mem > 0x19000000-0x19000fff irq 2 on nexus0 arge0: Overriding MAC from > EEPROM arge0: No PHY specified, using mask 16 > miibus0: on arge0 > floatphy0 PHY 0 on miibus0 > floatphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, > 1000baseSX, 1000baseSX-FDX, 1000baseT, 1000baseT-master, > 1000baseT-FDX, 1000baseT-FDX-master, auto switch0 PHY 1 on miibus0 > switch0: 100baseTX, 100baseTX-FDX, 1000baseSX, 1000baseSX-FDX, > 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master > ar8x16_switch0: on > switch0 arge0: Ethernet address: ff:ff:ff:ff:ff:ff arge1: AR71xx built-in ethernet interface> at mem 0x1a000000-0x1a000fff irq > 3 on nexus0 arge1: No PHY specified, using mask 15 arge1: Ethernet > address: ff:ff:ff:ff:ff:00 spi0: at mem > 0x1f000000-0x1f00000f on nexus0 spibus0: on spi0 mx25l0: > at cs 0 on spibus0 mx25l0: s25s1032, sector > 65536 bytes, 64 sectors ar71xx_wdog0: > on nexus0 ar71xx_wdog0: Previous reset was due to watchdog timeout > Timecounters tick every 1.000 msec > usbus0: 480Mbps High Speed USB v2.0 > ugen0.1: at usbus0 > uhub0: on > usbus0 WARNING: WITNESS option enabled, expect reduced performance. > uhub0: 1 port with 1 removable, self powered > Root mount waiting for: usbus0 > Root mount waiting for: usbus0 > Root mount waiting for: usbus0 > ugen0.2: at usbus0 > umass0: > on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x0100 > umass0:0:0:-1: Attached to scbus0 > Trying to mount root from ufs:map/rootfs.uzip []... > mountroot: waiting for device map/rootfs.uzip ... > da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 > da0: Removable Direct Access SCSI-0 > device da0: 40.000MB/s transfers > da0: 15252MB (31236096 512 byte sectors: 255H 63S/T 1944C) > Trying to mount root from ufs:da0s1a []... > warning: no time-of-day clock registered, system time will not be set > accurately Setting hostuuid: aed4c502-193a-11e1-b662-74ea3ae4d920. > Setting hostid: 0x6a714343. > Entropy harvesting: interrupts ethernet point_to_point kickstart. > Starting file system checks: > /dev/da0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS > /dev/da0s1a: clean, 3849245 free (1085 frags, 481020 blocks, 0.0% > fragmentation) lock order reversal: > 1st 0x8095691c ufs (ufs) > @ /home/stb/working/fe/freebsd/sys/kern/vfs_mount.c:1245 2nd > 0x80956e94 devfs (devfs) > @ /home/stb/working/fe/freebsd/sys/kern/vfs_subr.c:2167 KDB: stack > backtrace: db_trace_thread+30 (?,?,?,?) ra c3f597c000000018 sp 0 sz 0 > db_trace_self+1c (?,?,?,?) ra c3f597d800000018 sp 0 sz 0 800776d8+34 > (?,?,?,?) ra c3f597f0000001a0 sp 0 sz 0 kdb_backtrace+44 (?,?,?,?) ra > c3f5999000000018 sp 0 sz 0 801b3bbc+34 (?,?,?,?) ra c3f599a800000020 > sp 0 sz 0 witness_checkorder+9cc (?,?,803b1f7c,877) ra > c3f599c800000050 sp 0 sz 1 __lockmgr_args+948 (?,?,80956eb4,?) ra > c3f59a1800000070 sp 0 sz 1 vop_stdlock+4c (?,?,?,?) ra > c3f59a8800000028 sp 0 sz 0 VOP_LOCK1_APV+e4 (?,?,?,?) ra > c3f59ab000000020 sp 0 sz 0 _vn_lock+84 (?,?,?,?) ra c3f59ad000000048 > sp 0 sz 0 vget+c8 (?,?,?,?) ra c3f59b1800000030 sp 0 sz 0 > devfs_allocv+100 (?,?,?,?) ra c3f59b4800000038 sp 0 sz 0 > 800ba9b4+4c (?,?,?,?) ra c3f59b8000000028 sp 0 sz 0 > vflush+6c (?,?,0,80991300) ra c3f59ba8000000f8 sp 0 sz 1 > 800baa74+54 (?,?,?,?) ra c3f59ca000000028 sp 0 sz 0 > dounmount+3f0 (809e8000,?,?,?) ra c3f59cc800000050 sp 100000000 sz 0 > sys_unmount+39c (?,?,?,?) ra c3f59d18000000b0 sp 0 sz 0 > trap+7f4 (?,?,?,?) ra c3f59dc8000000b8 sp 0 sz 0 > MipsUserGenException+10c (?,?,?,404b53c0) ra c3f59e8000000000 sp 0 sz > 0 pid 70 > Mounting local file systems:. > Setting hostname: whitebox.lassitu.de. > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > floatphy0: found master switch0 > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > arge0: link state changed to DOWN > Starting Network: lo0 arge0 arge1. > lo0: flags=8049 metric 0 mtu 16384 > options=3 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > arge0: flags=8843 metric 0 > mtu 1500 options=80000 > ether ff:ff:ff:ff:ff:ff > inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 > inet6 fe80::481f:1b46:dbf1:bd78%arge0 prefixlen 64 scopeid > 0x2 nd6 options=29 > media: Ethernet autoselect (100baseTX ) > status: no carrier > arge1: flags=8843 metric 0 > mtu 1500 ether ff:ff:ff:ff:ff:00 > inet 44.128.65.33 netmask 0xffffffc0 broadcast 44.128.65.63 > inet6 fe80::481f:1b46:dbf1:bd78%arge1 prefixlen 64 scopeid > 0x3 nd6 options=29 > media: Ethernet 100baseTX > status: active > add net default: gateway 44.128.65.1 > add net ::ffff:0.0.0.0: gateway ::1 > add net ::0.0.0.0: gateway ::1 > add net fe80::: gateway ::1 > add net ff02::: gateway ::1 > Mounting NFS file systems:mount_nfs: diesel: hostname nor servname > provided, or not known . > Creating and/or trimming log files. > No core dumps found. > ELF ldconfig path: /lib /usr/lib /usr/lib/compat > Setting date via ntp. > Error : hostname nor servname provided, or not known > 22 Jan 15:01:04 ntpdate[566]: can't find host diesel > > 22 Jan 15:01:04 ntpdate[566]: no servers can be used, exiting > Clearing /tmp (X related). > Mounting late file systems:mount_nfs: diesel: hostname nor servname > provided, or not known . > Mounting /etc/fstab filesystems failed,Jan 22 15:01:12 init: /bin/sh > on /etc/rc terminated abnormally, going to single user mode Enter > full pathname of shell or RETURN for /bin/sh: > # devinfo > nexus0 > clock0 > apb0 > uart0 > gpio0 > gpioc0 > gpiobus0 > gpioled0 > gpioled1 > gpioled2 > ehci0 > usbus0 > uhub0 > umass0 > arge0 > miibus0 > floatphy0 > switch0 > ar8x16_switch0 > arge1 > spi0 > spibus0 > mx25l0 > ar71xx_wdog0 > # > > > -- > Stefan Bethke Fon +49 151 14070811 > > > -- Aleksandr Rybalko From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 18:36:02 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 500BF106564A; Sun, 22 Jan 2012 18:36:02 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 22BFC8FC1D; Sun, 22 Jan 2012 18:36:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0MIa2oP036624; Sun, 22 Jan 2012 18:36:02 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0MIa1qs036620; Sun, 22 Jan 2012 18:36:02 GMT (envelope-from linimon) Date: Sun, 22 Jan 2012 18:36:02 GMT Message-Id: <201201221836.q0MIa1qs036620@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164369: [bridge] [patch] two STP bridges have the same id X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 18:36:02 -0000 Old Synopsis: [patch] two STP bridges have the same id New Synopsis: [bridge] [patch] two STP bridges have the same id Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jan 22 18:35:36 UTC 2012 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=164369 From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 18:55:33 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE7AA106567C for ; Sun, 22 Jan 2012 18:55:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 99BBF8FC16 for ; Sun, 22 Jan 2012 18:55:33 +0000 (UTC) Received: by vbbey12 with SMTP id ey12so2178032vbb.13 for ; Sun, 22 Jan 2012 10:55:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=RiQohVfogS+We1blayAgfnmjc008Dz0IX92ZVz5wmT4=; b=CO11fOrf2D44Lt+Ge9iJktUM7w0Gz3+ci9n15ynJ8P+HhTXEe+ZRYGFbZCakAyoQz9 S/FudYs9u4PYHcIpEYPR/PJLoU1svFKOdb7GnBj+r+5SXgP82mdHYExz+u2DNRDC/Qbb 6oorKnOyHEoQfOVGONwTPG8jWBTZ/3mmOWgDE= MIME-Version: 1.0 Received: by 10.52.30.81 with SMTP id q17mr2528828vdh.115.1327258532836; Sun, 22 Jan 2012 10:55:32 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.184.164 with HTTP; Sun, 22 Jan 2012 10:55:32 -0800 (PST) Date: Sun, 22 Jan 2012 10:55:32 -0800 X-Google-Sender-Auth: 3sbRIwkyXNHrYOKYeonXjTzrN7Y Message-ID: From: Adrian Chadd To: FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: vimage recursions? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 18:55:33 -0000 Hi, I've just started testing vimage and I've received this: CURVNET_SET() recursion in unp_connect() line 1321, prev in soconnect() (ptr -> ptr) are these a big deal? Adrian From owner-freebsd-net@FreeBSD.ORG Sun Jan 22 19:48:59 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F77A1065674; Sun, 22 Jan 2012 19:48:59 +0000 (UTC) (envelope-from adrian@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 40F508FC14; Sun, 22 Jan 2012 19:48:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0MJmxJd000569; Sun, 22 Jan 2012 19:48:59 GMT (envelope-from adrian@freefall.freebsd.org) Received: (from adrian@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0MJmxIK000564; Sun, 22 Jan 2012 19:48:59 GMT (envelope-from adrian) Date: Sun, 22 Jan 2012 19:48:59 GMT Message-Id: <201201221948.q0MJmxIK000564@freefall.freebsd.org> To: adrian@FreeBSD.org, freebsd-net@FreeBSD.org, adrian@FreeBSD.org From: adrian@FreeBSD.org Cc: Subject: Re: kern/164369: [bridge] [patch] two STP bridges have the same id X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2012 19:48:59 -0000 Synopsis: [bridge] [patch] two STP bridges have the same id Responsible-Changed-From-To: freebsd-net->adrian Responsible-Changed-By: adrian Responsible-Changed-When: Sun Jan 22 19:48:49 UTC 2012 Responsible-Changed-Why: I'll poke this. http://www.freebsd.org/cgi/query-pr.cgi?pr=164369 From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 04:33:17 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92092106566B for ; Mon, 23 Jan 2012 04:33:17 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id E14F98FC15 for ; Mon, 23 Jan 2012 04:33:16 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id q0N4X3m3004212; Mon, 23 Jan 2012 13:33:13 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id q0N4WxP0061115; Mon, 23 Jan 2012 13:33:00 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Mon, 23 Jan 2012 13:32:27 +0900 (JST) Message-Id: <20120123.133227.887416479693451234.hrs@allbsd.org> To: dk@neveragain.de From: Hiroki Sato In-Reply-To: <9BBB9AB2-4F60-4DCB-8D25-FDEC83F62FF3@neveragain.de> References: <20120110102405.GA82356@neveragain.de> <20120117.044533.1784742896398431105.hrs@allbsd.org> <9BBB9AB2-4F60-4DCB-8D25-FDEC83F62FF3@neveragain.de> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.4 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Mon_Jan_23_13_32_27_2012_406)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Mon, 23 Jan 2012 13:33:15 +0900 (JST) X-Spam-Status: No, score=-104.2 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT,MIMEQENC,QENCPTR2,RDNS_NONE,SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: freebsd-net@FreeBSD.org Subject: Re: Unnecessary sleep in network.subr: ipv6_up() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 04:33:17 -0000 ----Security_Multipart(Mon_Jan_23_13_32_27_2012_406)-- Content-Type: Text/Plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Dennis K=F6gel wrote in <9BBB9AB2-4F60-4DCB-8D25-FDEC83F62FF3@neveragain.de>: dk> Am 16.01.2012 um 20:45 schrieb Hiroki Sato: dk> > Can you try the attached patch and let me know if it works fine o= n dk> > your system? dk> = dk> It does indeed. Thank you for your report. I committed the change into HEAD. It will be merged to 9-STABLE if there is no major problem with it. -- Hiroki ----Security_Multipart(Mon_Jan_23_13_32_27_2012_406)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk8c4tsACgkQTyzT2CeTzy1dEQCePfXNDTSGkwFvJEMllilokvzn Os0AoN1CK5eac8kroZgDqXcOaCZe6bPr =NWxa -----END PGP SIGNATURE----- ----Security_Multipart(Mon_Jan_23_13_32_27_2012_406)---- From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 04:49:04 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2382106566B for ; Mon, 23 Jan 2012 04:49:04 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 068C88FC12 for ; Mon, 23 Jan 2012 04:49:03 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id q0N4mlcc007836; Mon, 23 Jan 2012 13:48:57 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id q0N4mig2061276; Mon, 23 Jan 2012 13:48:45 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Mon, 23 Jan 2012 13:48:36 +0900 (JST) Message-Id: <20120123.134836.1135493075127784763.hrs@allbsd.org> To: sem@semmy.ru, bra@fsn.hu, kes-kes@yandex.ru From: Hiroki Sato In-Reply-To: <4F196D64.6020508@semmy.ru> References: <4F190F3F.7050302@fsn.hu> <4F196D64.6020508@semmy.ru> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.4 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Mon_Jan_23_13_48_36_2012_481)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Mon, 23 Jan 2012 13:49:02 +0900 (JST) X-Spam-Status: No, score=-102.4 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, FSL_RU_URL, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: freebsd-net@FreeBSD.org Subject: Re: Adding setfib support to rc.d/routing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 04:49:04 -0000 ----Security_Multipart(Mon_Jan_23_13_48_36_2012_481)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sergey Matveychuk wrote in <4F196D64.6020508@semmy.ru>: se> 20.01.2012 10:52, Attila Nagy wrote: se> > Hi, se> > se> > Having multiple routing tables is a very nice and (was a) long awaited se> > capability in FreeBSD. Having it since years is even more cool, se> > because se> > we can assume it's stable now. se> > But not having infrastructure support for it sucks, this makes people se> > hacking with rc.local or various scripts in various places. se> > se> > There is a(t least one) PR about it: conf/145440, which proposes a se> > standard method for setting up different FIBs in a seems to be logical se> > way, which is compatible with the current single routing table method se> > of se> > static_routes. se> > se> > Are there any objections about this PR? Is there something we can do se> > to se> > get it committed? se> > se> se> se> JFYI conf/132476 I'll take a look at these patches and try to merge. -- Hiroki ----Security_Multipart(Mon_Jan_23_13_48_36_2012_481)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk8c5qQACgkQTyzT2CeTzy1n2wCeIR5YsSJTwnpbbeyNRKa9eN8F HXMAn3iyQrdydRwO+0nYE+ba0quM7Zek =Cr6V -----END PGP SIGNATURE----- ----Security_Multipart(Mon_Jan_23_13_48_36_2012_481)---- From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 08:29:42 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D85B106564A; Mon, 23 Jan 2012 08:29:42 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 545D68FC08; Mon, 23 Jan 2012 08:29:42 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0N8TgCE030524; Mon, 23 Jan 2012 08:29:42 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0N8TgZx030520; Mon, 23 Jan 2012 08:29:42 GMT (envelope-from linimon) Date: Mon, 23 Jan 2012 08:29:42 GMT Message-Id: <201201230829.q0N8TgZx030520@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164400: [ipsec] immediate crash after the start of ipsec processing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 08:29:42 -0000 Old Synopsis: immediate crash after the start of ipsec processing New Synopsis: [ipsec] immediate crash after the start of ipsec processing Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Mon Jan 23 08:29:29 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=164400 From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 08:30:15 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AF371065687 for ; Mon, 23 Jan 2012 08:30:15 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CB9DF8FC28 for ; Mon, 23 Jan 2012 08:30:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0N8UB3D030750 for ; Mon, 23 Jan 2012 08:30:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0N8UBX9030744; Mon, 23 Jan 2012 08:30:11 GMT (envelope-from gnats) Date: Mon, 23 Jan 2012 08:30:11 GMT Message-Id: <201201230830.q0N8UBX9030744@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "Eugene M. Zheganin" Cc: Subject: Re: kern/164400: immediate crash after the start of ipsec processing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Eugene M. Zheganin" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 08:30:15 -0000 The following reply was made to PR kern/164400; it has been noted by GNATS. From: "Eugene M. Zheganin" To: bug-followup@FreeBSD.org, eugene@zhegan.in Cc: Subject: Re: kern/164400: immediate crash after the start of ipsec processing Date: Mon, 23 Jan 2012 14:22:59 +0600 I've re-read it and I found one phrase that can cause misunderstanding: 'Without it it runs fine.' I should write 'without switching to master and starting ipsec processing from branches'. From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 11:07:09 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85AF11065674 for ; Mon, 23 Jan 2012 11:07:09 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 72ABA8FC14 for ; Mon, 23 Jan 2012 11:07:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0NB79W4081027 for ; Mon, 23 Jan 2012 11:07:09 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0NB78wc081025 for freebsd-net@FreeBSD.org; Mon, 23 Jan 2012 11:07:08 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 23 Jan 2012 11:07:08 GMT Message-Id: <201201231107.q0NB78wc081025@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 11:07:09 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/164400 net [ipsec] immediate crash after the start of ipsec proce o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162926 net [ipfilter] Infinite loop in ipfilter with fragmented I o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162509 net [re] [panic] Kernel panic may be related to if_re.c (r o kern/162352 net [patch] Enhancement: add SO_PROTO to socket.h o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161899 net [route] ntpd(8): Repeating RTM_MISS packets causing hi o kern/161381 net [re] RTL8169SC - re0: PHY write failed o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160420 net [msk] phy write timeout on HP 5310m o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159795 net [tcp] excessive duplicate ACKs and TCP session freezes o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157429 net [re] Realtek RTL8169 doesn't work with re(4) o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155772 net ifconfig(8): ioctl (SIOCAIFADDR): File exists on direc o kern/155680 net [multicast] problems with multicast s kern/155642 net [request] Add driver for Realtek RTL8191SE/RTL8192SE W o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [request]: Port brcm80211 driver from Linux to FreeBSD o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m o kern/146426 net [mwl] 802.11n rates not possible on mwl o kern/146425 net [mwl] mwl dropping all packets during and after high u f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o bin/145934 net [patch] add count option to netstat(1) o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph o kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138620 net [lagg] [patch] lagg port bpf-writes blocked o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging s kern/90086 net [hang] 5.4p8 on supermicro P8SCT hangs during boot if o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ p kern/85320 net [gre] [patch] possible depletion of kernel stack in ip o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr s kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 383 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 13:05:48 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FEAC1065675; Mon, 23 Jan 2012 13:05:48 +0000 (UTC) (envelope-from coco@executive-computing.de) Received: from mail.moehre.org (mail.moehre.org [195.96.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id AA8D58FC1F; Mon, 23 Jan 2012 13:05:47 +0000 (UTC) Received: from mail.moehre.org (unknown [195.96.35.7]) by mail.moehre.org (Postfix) with ESMTP id C0EA88B141D; Mon, 23 Jan 2012 13:48:25 +0100 (CET) X-Spam-Flag: NO X-Spam-Score: -100.964 X-Spam-Level: X-Spam-Status: No, score=-100.964 tagged_above=-999 required=5 tests=[ALL_TRUSTED=-1, AWL=0.036, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mail.moehre.org ([195.96.35.7]) by mail.moehre.org (mail.moehre.org [195.96.35.7]) (amavisd-new, port 10024) with ESMTP id nP3MTV5rYgXS; Mon, 23 Jan 2012 13:48:20 +0100 (CET) Received: from [192.168.100.30] (p54B0B982.dip.t-dialin.net [84.176.185.130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: coco@executive-computing.de) by mail.moehre.org (Postfix) with ESMTPSA id 6D84D8B141C; Mon, 23 Jan 2012 13:48:20 +0100 (CET) Message-ID: <4F1D56E3.4090109@executive-computing.de> Date: Mon, 23 Jan 2012 13:47:31 +0100 From: Marco Steinbach User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) MIME-Version: 1.0 To: freebsd-net@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Cc: Pyun YongHyeon Subject: ULi/ALi M5263 /dc(4) and release/9.0.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 13:05:48 -0000 Hi, it looks the changes from r226701, which enable dc(4) to handle ALi/ULi M5261/M5263 ethernet controllers, are present in head, stable/7 and stable/8, but not in release/9.0.0, as there's no mention in the release notes, I don't see any of the changes in the repository, and the device isn't recognized. Was it too late to have these included, or were there other reasons ? MfG CoCo From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 15:01:11 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96173106564A for ; Mon, 23 Jan 2012 15:01:11 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 209D88FC13 for ; Mon, 23 Jan 2012 15:01:10 +0000 (UTC) Received: by eaai10 with SMTP id i10so1322961eaa.13 for ; Mon, 23 Jan 2012 07:01:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=TtnxEp0c3IDDJyiYUVXdOSbe9UUMomcq8a/oTWqLlac=; b=J/6//9DZuNmzxoe0xoxrysGk0twgGQGtQcy4paA3iRJD25fp+hOOsuCAMVrdbwybjW WcuFxzwblewxNcuFujBWWWd8pqoRN8Jh2odv0wetygGxyKbsEY5dmTa4Q/lAv7/olAkX l6xrXr7x4CLaXUP18inEP4Xmt0oKqnOkwu7nQ= Received: by 10.213.36.11 with SMTP id r11mr1625923ebd.69.1327330870050; Mon, 23 Jan 2012 07:01:10 -0800 (PST) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id x4sm54794406eeb.4.2012.01.23.07.01.07 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 23 Jan 2012 07:01:08 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=windows-1252 From: Nikolay Denev In-Reply-To: <7D135FA9-6503-4263-AE55-5C80F94CDF5A@gmail.com> Date: Mon, 23 Jan 2012 17:01:06 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4F131A7D.4020006@zonov.org> <733BE6AF-33E0-4C16-A222-B5F5D0519194@gmail.com> <12379405.15603.1326656127893.JavaMail.mobile-sync@vbzh28> <3008402354236887854@unknownmsgid> <7D135FA9-6503-4263-AE55-5C80F94CDF5A@gmail.com> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1251.1) Subject: Re: ICMP attacks against TCP and PMTUD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 15:01:11 -0000 On Jan 20, 2012, at 10:32 AM, Nikolay Denev wrote: > On Jan 15, 2012, at 9:52 PM, Nikolay Denev wrote: >=20 >> On 15.01.2012, at 21:35, Andrey Zonov wrote: >>=20 >>> This helped me: >>> /boot/loader.conf >>> net.inet.tcp.hostcache.hashsizee536 >>> net.inet.tcp.hostcache.cachelimit=1966080 >>>=20 >>> Actually, this is a workaround. As I remember, real problem is in >>> tcp_ctlinput(), it could not update MTU for destination IP if = hostcache >>> allocation fails. tcp_hc_updatemtu() should returns NULL if >>> tcp_hc_insert() returns NULL and tcp_ctlinput() should check this = case >>> and sets updated MTU for this particular connection if >>> tcp_hc_updatemtu() fails. Otherwise we've got infinite loop in MTU >>> discovery. >>>=20 >>>=20 >>> On 15.01.2012 22:59, Nikolay Denev wrote: >>>>=20 >>>> % uptime >>>> 7:57PM up 608 days, 4:06, 1 user, load averages: 0.30, 0.21, 0.17 >>>>=20 >>>> % vmstat -z|grep hostcache >>>> hostcache: 136, 15372, 15136, 236, = 44946965, 10972760 >>>>=20 >>>>=20 >>>> Hmm=85 probably I should increase this=85. >>>>=20 >>>=20 >>> -- >>> Andrey Zonov >>=20 >> Thanks, I will test this asap! >>=20 >> Regards, >> Nikolay >=20 > I've upgraded from 7.3-STABLE to 8.2-STABLE and bumped significantly = the hostcache tunables. > So far so good, I'll report back if I see similar traffic spikes. >=20 Seems like I have been wrong about these traffic spikes being attacks, = and actually the problem seems to be the pmtu infinite loop Andrey = described. I'm now running 8.2-STABLE with hostcache significantly bumped and = regularly have more than 20K hostcache entries, which was more than the default = limit of 15K I was running with before. From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 21:00:23 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C8911065673 for ; Mon, 23 Jan 2012 21:00:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 019198FC16 for ; Mon, 23 Jan 2012 21:00:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0NL0MZP035031 for ; Mon, 23 Jan 2012 21:00:22 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0NL0MrQ035030; Mon, 23 Jan 2012 21:00:22 GMT (envelope-from gnats) Date: Mon, 23 Jan 2012 21:00:22 GMT Message-Id: <201201232100.q0NL0MrQ035030@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "K. Macy" Cc: Subject: kern/144917: [flowtable] [panic] flowtable crashes system [regression] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "K. Macy" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:00:23 -0000 The following reply was made to PR kern/144917; it has been noted by GNATS. From: "K. Macy" To: root@net1.cc Cc: bug-followup@freebsd.org Subject: kern/144917: [flowtable] [panic] flowtable crashes system [regression] Date: Mon, 23 Jan 2012 21:50:08 +0100 Have you tested this workload on 9.0 without flowtable removed / disabled? Is the problem still there or reduced? Thanks From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 21:00:28 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C099D106566B for ; Mon, 23 Jan 2012 21:00:28 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 95BCA8FC19 for ; Mon, 23 Jan 2012 21:00:28 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0NL0SM7035082 for ; Mon, 23 Jan 2012 21:00:28 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0NL0SIF035081; Mon, 23 Jan 2012 21:00:28 GMT (envelope-from gnats) Date: Mon, 23 Jan 2012 21:00:28 GMT Message-Id: <201201232100.q0NL0SIF035081@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "K. Macy" Cc: Subject: kern/146792: [flowtable] flowcleaner 100% cpu's core load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "K. Macy" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:00:28 -0000 The following reply was made to PR kern/146792; it has been noted by GNATS. From: "K. Macy" To: niko@gtelecom.ru Cc: bug-followup@freebsd.org Subject: kern/146792: [flowtable] flowcleaner 100% cpu's core load Date: Mon, 23 Jan 2012 21:48:09 +0100 Have you tested this workload with 9.0 with the flowtable enabled? Is the problem as severe? Thanks From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 21:09:36 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66440106566B for ; Mon, 23 Jan 2012 21:09:36 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id EC3758FC12 for ; Mon, 23 Jan 2012 21:09:35 +0000 (UTC) Received: by bkbc12 with SMTP id c12so3723620bkb.13 for ; Mon, 23 Jan 2012 13:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=reply-to:from:to:subject:date:message-id:mime-version:content-type :content-transfer-encoding:x-mailer:thread-index:content-language; bh=y9Q4hHD4GZd5zmns8nIvLMTUjSyD4jp0OyuhRS4i6PA=; b=cWTd8xqBPz/VQ9VD9Q54lttq+oCDqqNWSw2PzYX3cw4tjllzE73PRLR4GQchgvLa2x aLegowyCe5MlZdjZZxtAnD/RZTi4ChuJYV61sobb3XTB7sWyXtKRRYPPvbbfCaXcWsVa AGTsJ0/vryd5kSXoodBgSEVpdSpmo0tRJcOiY= Received: by 10.204.153.28 with SMTP id i28mr3977037bkw.136.1327352974871; Mon, 23 Jan 2012 13:09:34 -0800 (PST) Received: from rimwks1w7x64 ([31.47.168.205]) by mx.google.com with ESMTPS id ev5sm28487107bkb.4.2012.01.23.13.09.33 (version=SSLv3 cipher=OTHER); Mon, 23 Jan 2012 13:09:34 -0800 (PST) From: rozhuk.im@gmail.com To: Date: Tue, 24 Jan 2012 06:09:30 +0900 Message-ID: <4f1dcc8e.45c8cc0a.2935.2254@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AczaE0xCSiCeVikkS4WctDYr2JN5Yw== Content-Language: ru Cc: Subject: ng_bridge and locks X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Rozhuk.IM@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:09:36 -0000 Hi! I found a comment in the code: /* * This node has all kinds of stuff that could be screwed by SMP. * Until it gets it's own internal protection, we go through in * single file. This could hurt a machine bridging beteen two * GB ethernets so it should be fixed. * When it's fixed the process SHOULD NOT SLEEP, spinlocks please! * (and atomic ops ) */ mtx_init(...., MTX_DEF); How bad to use netgraph node MTX_DEF mutex? From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 21:17:11 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E64E1065673 for ; Mon, 23 Jan 2012 21:17:11 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 88CF08FC08 for ; Mon, 23 Jan 2012 21:17:10 +0000 (UTC) Received: (qmail 8815 invoked from network); 23 Jan 2012 19:39:03 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 23 Jan 2012 19:39:03 -0000 Message-ID: <4F1DCE54.8060107@freebsd.org> Date: Mon, 23 Jan 2012 22:17:08 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: Nikolay Denev References: <4F131A7D.4020006@zonov.org> <733BE6AF-33E0-4C16-A222-B5F5D0519194@gmail.com> <12379405.15603.1326656127893.JavaMail.mobile-sync@vbzh28> <3008402354236887854@unknownmsgid> <7D135FA9-6503-4263-AE55-5C80F94CDF5A@gmail.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org Subject: Re: ICMP attacks against TCP and PMTUD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:17:11 -0000 On 23.01.2012 16:01, Nikolay Denev wrote: > > On Jan 20, 2012, at 10:32 AM, Nikolay Denev wrote: > >> On Jan 15, 2012, at 9:52 PM, Nikolay Denev wrote: >> >>> On 15.01.2012, at 21:35, Andrey Zonov wrote: >>> >>>> This helped me: >>>> /boot/loader.conf >>>> net.inet.tcp.hostcache.hashsizee536 >>>> net.inet.tcp.hostcache.cachelimit66080 >>>> >>>> Actually, this is a workaround. As I remember, real problem is in >>>> tcp_ctlinput(), it could not update MTU for destination IP if hostcache >>>> allocation fails. tcp_hc_updatemtu() should returns NULL if >>>> tcp_hc_insert() returns NULL and tcp_ctlinput() should check this case >>>> and sets updated MTU for this particular connection if >>>> tcp_hc_updatemtu() fails. Otherwise we've got infinite loop in MTU >>>> discovery. >>>> >>>> >>>> On 15.01.2012 22:59, Nikolay Denev wrote: >>>>> >>>>> % uptime >>>>> 7:57PM up 608 days, 4:06, 1 user, load averages: 0.30, 0.21, 0.17 >>>>> >>>>> % vmstat -z|grep hostcache >>>>> hostcache: 136, 15372, 15136, 236, 44946965, 10972760 >>>>> >>>>> >>>>> Hmm… probably I should increase this…. >>>>> >>>> >>>> -- >>>> Andrey Zonov >>> >>> Thanks, I will test this asap! >>> >>> Regards, >>> Nikolay >> >> I've upgraded from 7.3-STABLE to 8.2-STABLE and bumped significantly the hostcache tunables. >> So far so good, I'll report back if I see similar traffic spikes. >> > > Seems like I have been wrong about these traffic spikes being attacks, and > actually the problem seems to be the pmtu infinite loop Andrey described. > I'm now running 8.2-STABLE with hostcache significantly bumped and regularly > have more than 20K hostcache entries, which was more than the default limit of 15K I was running with before. The bug is real. Please try the attached patch to fix the issue for IPv4. It's against current but should apply to 8 or 9 as well. -- Andre http://people.freebsd.org/~andre/tcp_subr.c-pmtud-20120123.diff Index: netinet/tcp_subr.c =================================================================== --- netinet/tcp_subr.c (revision 230489) +++ netinet/tcp_subr.c (working copy) @@ -1410,9 +1410,11 @@ */ if (mtu <= tcp_maxmtu(&inc, NULL)) tcp_hc_updatemtu(&inc, mtu); - } - - inp = (*notify)(inp, inetctlerrmap[cmd]); + /* XXXAO: Slighly hackish. */ + inp = (*notify)(inp, mtu); + } else + inp = (*notify)(inp, + inetctlerrmap[cmd]); } } if (inp != NULL) @@ -1656,12 +1658,15 @@ * based on the new value in the route. Also nudge TCP to send something, * since we know the packet we just sent was dropped. * This duplicates some code in the tcp_mss() function in tcp_input.c. + * + * XXXAO: Slight abuse of 'errno'. */ struct inpcb * tcp_mtudisc(struct inpcb *inp, int errno) { struct tcpcb *tp; struct socket *so; + int mtu; INP_WLOCK_ASSERT(inp); if ((inp->inp_flags & INP_TIMEWAIT) || @@ -1671,7 +1676,12 @@ tp = intotcpcb(inp); KASSERT(tp != NULL, ("tcp_mtudisc: tp == NULL")); - tcp_mss_update(tp, -1, NULL, NULL); + /* Extract the MTU from errno for IPv4. */ + if (errno > PRC_NCMDS) + mtu = errno; + else + mtu = -1; + tcp_mss_update(tp, mtu, NULL, NULL); so = inp->inp_socket; SOCKBUF_LOCK(&so->so_snd); From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 21:39:21 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19C5B106566B for ; Mon, 23 Jan 2012 21:39:21 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id D1F038FC13 for ; Mon, 23 Jan 2012 21:39:20 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id 632D2358C5B; Mon, 23 Jan 2012 22:39:19 +0100 (CET) To: X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 23 Jan 2012 22:39:19 +0100 From: Marcin Markowski Message-ID: <159d4fbce722663a84f3cea12da828a5@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Subject: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:39:21 -0000 Hello, This message has been sent to freebsd-performance@ but got the information that should contact also with freebsd-net@. We use FreeBSD as sniffer (libpcap programs) and we experience performance problems when incoming traffic is greater than 7.5Gbps/s. If we check 'top' we see that first irq from network card is using 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), and both systems behave the same. In logs we see also: interrupt storm detected on "irq268:"; throttling interrupt source Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and NIC Intel X520-DA2. I'm not sure if problem is with NIC or motherboard in SR2600URBRP, because everything is fine when we use other server configuration: Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 My /boot/loader.conf: kern.ipc.nmbclusters=262144 hw.ixgbe.rxd=2048 hw.ixgbe.txd=2048 hw.ixgbe.num_queues=16 /etc/sysctl.conf hw.intr_storm_threshold=10000 -- Marcin Markowski From owner-freebsd-net@FreeBSD.ORG Mon Jan 23 23:38:57 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAC62106566B for ; Mon, 23 Jan 2012 23:38:57 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id 808D48FC14 for ; Mon, 23 Jan 2012 23:38:56 +0000 (UTC) Received: by wgbgn7 with SMTP id gn7so2175790wgb.1 for ; Mon, 23 Jan 2012 15:38:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=7bPGMGoSES+jh9gPn+KLGRz+M6cfTBdN9gW175FzP3Q=; b=qEKW/wLL0zePO5a7QOTF8Rl8GOta+sbnReZ93C9CP57rMNLLMbgHhFtJhPGVkqGIMp Y4vlfvh1jaN3jLxcwfNaz7/XUx4InFmsNU6p6TocjBv+W40aa+PeEtTp6x6gtCFJksY/ M18lBtrpr9HLy4XWVIdDtMIzRV0+/aKgCKOKU= MIME-Version: 1.0 Received: by 10.180.96.161 with SMTP id dt1mr348wib.13.1327360315920; Mon, 23 Jan 2012 15:11:55 -0800 (PST) Received: by 10.216.82.2 with HTTP; Mon, 23 Jan 2012 15:11:51 -0800 (PST) In-Reply-To: <220871327066330@web3.yandex.ru> References: <220871327066330@web3.yandex.ru> Date: Mon, 23 Jan 2012 18:11:51 -0500 Message-ID: From: Arnaud Lacombe To: Dmukha Nikolay Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: MSTP on FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 23:38:58 -0000 Hi, On Fri, Jan 20, 2012 at 8:32 AM, Dmukha Nikolay wrote: > Hello. > Please, help me to setup this sheme: > ******************************************************* > comp1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 comp2 > =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > dgs3627g(24)-----------(12)dgs3612g > =A0|(21) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(9)| > =A0|______freebsd bridge_______| > ******************************************************* > > Devices are: > Dlink DGS3627G (FW 2.84-B25), Dlink DGS3612G (FW 2.84-B25), FreeBSD (8.0 = stable 201105). > > The main target is to reserve the FreeBsd bridge. The priority route for = traffic from comp 1 to comp 2 is freebsd. When the server (freebsd bridge) = is down, traffic should go through the direct link between switches. I want= to reserve 3 freebsd bridges. what do you mean by "I want to reserve 3 freebsd bridges" ? > So it`s necessary to use MSTP technology. Does it ? I only see a single VLAN in your setup, which limits MSTP utility= . FreeBSD does not support MSTP. In presence of MSTP-enabled bridges, it will fall back to RSTP mode, and only see the CIST. I an not sure how the CIST is exactly built in such a mixed environment. - Arnaud From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 05:40:55 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39C07106566C for ; Tue, 24 Jan 2012 05:40:55 +0000 (UTC) (envelope-from rejithomas.d@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id EC1F78FC0C for ; Tue, 24 Jan 2012 05:40:54 +0000 (UTC) Received: by ghbg15 with SMTP id g15so276743ghb.13 for ; Mon, 23 Jan 2012 21:40:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=uyZyUAGFgNK50wePCvuGSLEwtuGVZyY8R9G7pB3rlk0=; b=tPUOSXIij7tSoUzIzh2LDaUE7AxyLrmSUMTtLH+RYx8nYdXENvGa7nW2fg00Yob2De uo/Rh6nXwqlYk3slzCCdaFtqvRWCDMcYeQ7Mr2c6p7m+0CuTbQh2dMREt87PSDYVLMNg Yt8iYCeIOAM5C27UldmCQ3rowskvdIv3J4l+A= MIME-Version: 1.0 Received: by 10.236.165.6 with SMTP id d6mr15400918yhl.44.1327381966013; Mon, 23 Jan 2012 21:12:46 -0800 (PST) Received: by 10.147.51.17 with HTTP; Mon, 23 Jan 2012 21:12:45 -0800 (PST) Date: Tue, 24 Jan 2012 10:42:45 +0530 Message-ID: From: Reji Thomas To: freebsd-net@freebsd.org X-Mailman-Approved-At: Tue, 24 Jan 2012 05:49:44 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Doubt regarding ICMP_UNREACH_NEEDFRAG processing for IPSec X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 05:40:55 -0000 Hi, I dont see the freebsd ip stack processing ICMP_UNREACH_NEEDFRAG where protocol is ESP/AH. ( The ipsec4_common_ctlinput is not implemented). If this is right, how is the PMTU discovery done for ipsec traffic or is it that freebsd is not doing that?. Thanks Reji From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 08:18:43 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB1E2106564A for ; Tue, 24 Jan 2012 08:18:43 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3BECD8FC0C for ; Tue, 24 Jan 2012 08:18:42 +0000 (UTC) Received: by eekb47 with SMTP id b47so1597159eek.13 for ; Tue, 24 Jan 2012 00:18:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=GbSl7/uW0VfMNZvf5Z3TxlWdLa4BjHexZyOOLPxxrPE=; b=ujYQpS04GJRuXcUkPKmSvl9IwM1ANG2maZkPqpbDz1O5OR3W9N+sofzGyjaPD90LAc 5huW5LzqcMMJ92dB2EEpUvoJJNj4jG/u4qf4rCH/JQfqHIXa3ErDBVZ44ZeA8O+utw77 IX+xYJcXSHU7rlB/xeYe159rnahU7zlsKG1k4= Received: by 10.14.127.16 with SMTP id c16mr4294257eei.35.1327393121136; Tue, 24 Jan 2012 00:18:41 -0800 (PST) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id x4sm64696959eeb.4.2012.01.24.00.18.39 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 Jan 2012 00:18:40 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <159d4fbce722663a84f3cea12da828a5@leon.pl> Date: Tue, 24 Jan 2012 10:18:38 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> References: <159d4fbce722663a84f3cea12da828a5@leon.pl> To: Marcin Markowski X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-net@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 08:18:43 -0000 On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote: > Hello, >=20 > This message has been sent to freebsd-performance@ but got > the information that should contact also with freebsd-net@. >=20 > We use FreeBSD as sniffer (libpcap programs) and we experience > performance problems when incoming traffic is greater than 7.5Gbps/s. > If we check 'top' we see that first irq from network card is using > 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE > (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), > and both systems behave the same. In logs we see also: > interrupt storm detected on "irq268:"; throttling interrupt source >=20 > Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and > NIC Intel X520-DA2. >=20 > I'm not sure if problem is with NIC or motherboard in SR2600URBRP, > because everything is fine when we use other server configuration: > Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >=20 > My /boot/loader.conf: > kern.ipc.nmbclusters=3D262144 > hw.ixgbe.rxd=3D2048 > hw.ixgbe.txd=3D2048 > hw.ixgbe.num_queues=3D16 >=20 > /etc/sysctl.conf > hw.intr_storm_threshold=3D10000 >=20 > --=20 > Marcin Markowski Hi, Maybe you want to take a loot at NETMAP : = http://info.iet.unipi.it/~luigi/netmap/ There is a libpcap wrapper library, so you can use it with unchanged = pcap consumers, and get great performance increase. I'm not sure that the patches are updated for 8 and 9 though, since the = initial commit to HEAD there were several related changes. P.S.: It is important also what is you packet rate, since 7.5Gbps with = jumbo packets or 64 byte packets are very different things :) Regards, Nikolay From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 08:20:37 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E37FF106564A; Tue, 24 Jan 2012 08:20:37 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3D4858FC19; Tue, 24 Jan 2012 08:20:36 +0000 (UTC) Received: by eaai10 with SMTP id i10so1625099eaa.13 for ; Tue, 24 Jan 2012 00:20:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=nk3dw3ZCg/prTRJE+uFz79NWIIsXc891uVO+dzaBgUk=; b=lM//O0Wo1s6zd45blTDFxZCErtWKNWJVOhJgQ1CHradvetS2NA47yMAfMBT9EaOKpb QvnW6JOnWVbKNMxGj1HSQy4Uf37BGSvofiDOwmdvpRr1QQ19LSA/WAOn4ciJyjiq8r5r X1FPJdnhkWe40I9J9D2P0gM3ku7fDSLZFt1I8= Received: by 10.213.23.1 with SMTP id p1mr247049ebb.0.1327393235980; Tue, 24 Jan 2012 00:20:35 -0800 (PST) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id c16sm64763938eei.1.2012.01.24.00.20.33 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 Jan 2012 00:20:34 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=windows-1252 From: Nikolay Denev In-Reply-To: <4F1DCE54.8060107@freebsd.org> Date: Tue, 24 Jan 2012 10:20:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3D2ACF41-3CFB-42BE-B89D-06B202AFD2B0@gmail.com> References: <4F131A7D.4020006@zonov.org> <733BE6AF-33E0-4C16-A222-B5F5D0519194@gmail.com> <12379405.15603.1326656127893.JavaMail.mobile-sync@vbzh28> <3008402354236887854@unknownmsgid> <7D135FA9-6503-4263-AE55-5C80F94CDF5A@gmail.com> <4F1DCE54.8060107@freebsd.org> To: Andre Oppermann X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-net@freebsd.org Subject: Re: ICMP attacks against TCP and PMTUD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 08:20:38 -0000 On Jan 23, 2012, at 11:17 PM, Andre Oppermann wrote: > On 23.01.2012 16:01, Nikolay Denev wrote: >>=20 >> On Jan 20, 2012, at 10:32 AM, Nikolay Denev wrote: >>=20 >>> On Jan 15, 2012, at 9:52 PM, Nikolay Denev wrote: >>>=20 >>>> On 15.01.2012, at 21:35, Andrey Zonov wrote: >>>>=20 >>>>> This helped me: >>>>> /boot/loader.conf >>>>> net.inet.tcp.hostcache.hashsizee536 >>>>> net.inet.tcp.hostcache.cachelimit=1966080 >>>>>=20 >>>>> Actually, this is a workaround. As I remember, real problem is in >>>>> tcp_ctlinput(), it could not update MTU for destination IP if = hostcache >>>>> allocation fails. tcp_hc_updatemtu() should returns NULL if >>>>> tcp_hc_insert() returns NULL and tcp_ctlinput() should check this = case >>>>> and sets updated MTU for this particular connection if >>>>> tcp_hc_updatemtu() fails. Otherwise we've got infinite loop in = MTU >>>>> discovery. >>>>>=20 >>>>>=20 >>>>> On 15.01.2012 22:59, Nikolay Denev wrote: >>>>>>=20 >>>>>> % uptime >>>>>> 7:57PM up 608 days, 4:06, 1 user, load averages: 0.30, 0.21, = 0.17 >>>>>>=20 >>>>>> % vmstat -z|grep hostcache >>>>>> hostcache: 136, 15372, 15136, 236, = 44946965, 10972760 >>>>>>=20 >>>>>>=20 >>>>>> Hmm=85 probably I should increase this=85. >>>>>>=20 >>>>>=20 >>>>> -- >>>>> Andrey Zonov >>>>=20 >>>> Thanks, I will test this asap! >>>>=20 >>>> Regards, >>>> Nikolay >>>=20 >>> I've upgraded from 7.3-STABLE to 8.2-STABLE and bumped significantly = the hostcache tunables. >>> So far so good, I'll report back if I see similar traffic spikes. >>>=20 >>=20 >> Seems like I have been wrong about these traffic spikes being = attacks, and >> actually the problem seems to be the pmtu infinite loop Andrey = described. >> I'm now running 8.2-STABLE with hostcache significantly bumped and = regularly >> have more than 20K hostcache entries, which was more than the default = limit of 15K I was running with before. >=20 > The bug is real. Please try the attached patch to fix the issue for = IPv4. > It's against current but should apply to 8 or 9 as well. >=20 > --=20 > Andre >=20 > http://people.freebsd.org/~andre/tcp_subr.c-pmtud-20120123.diff >=20 > Index: netinet/tcp_subr.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- netinet/tcp_subr.c (revision 230489) > +++ netinet/tcp_subr.c (working copy) > @@ -1410,9 +1410,11 @@ > */ > if (mtu <=3D = tcp_maxmtu(&inc, NULL)) > tcp_hc_updatemtu(&inc, = mtu); > - } > - > - inp =3D (*notify)(inp, = inetctlerrmap[cmd]); > + /* XXXAO: Slighly hackish. = */ > + inp =3D (*notify)(inp, mtu); > + } else > + inp =3D (*notify)(inp, > + = inetctlerrmap[cmd]); > } > } > if (inp !=3D NULL) > @@ -1656,12 +1658,15 @@ > * based on the new value in the route. Also nudge TCP to send = something, > * since we know the packet we just sent was dropped. > * This duplicates some code in the tcp_mss() function in tcp_input.c. > + * > + * XXXAO: Slight abuse of 'errno'. > */ > struct inpcb * > tcp_mtudisc(struct inpcb *inp, int errno) > { > struct tcpcb *tp; > struct socket *so; > + int mtu; >=20 > INP_WLOCK_ASSERT(inp); > if ((inp->inp_flags & INP_TIMEWAIT) || > @@ -1671,7 +1676,12 @@ > tp =3D intotcpcb(inp); > KASSERT(tp !=3D NULL, ("tcp_mtudisc: tp =3D=3D NULL")); >=20 > - tcp_mss_update(tp, -1, NULL, NULL); > + /* Extract the MTU from errno for IPv4. */ > + if (errno > PRC_NCMDS) > + mtu =3D errno; > + else > + mtu =3D -1; > + tcp_mss_update(tp, mtu, NULL, NULL); >=20 > so =3D inp->inp_socket; > SOCKBUF_LOCK(&so->so_snd); Hi Andre, Thanks for the patch. I will apply it as soon as possible. I'll probably first try to reproduce the problem locally since I've = increased the hostcache on my nginx balancers already, and changes require reboots which I'm not = able to do at the moment. Will let you know as soon as I have results. Thanks! From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 11:15:01 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCA8F106566B for ; Tue, 24 Jan 2012 11:15:01 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from mail.incore.de (dss.incore.de [195.145.1.138]) by mx1.freebsd.org (Postfix) with ESMTP id 784568FC13 for ; Tue, 24 Jan 2012 11:15:01 +0000 (UTC) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by mail.incore.de (Postfix) with ESMTP id 929FE5EC58; Tue, 24 Jan 2012 11:57:47 +0100 (CET) X-Virus-Scanned: amavisd-new at incore.de Received: from mail.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id YPfIMTYwDkLl; Tue, 24 Jan 2012 11:57:46 +0100 (CET) Received: from mail.incore (fwintern.dmz [10.0.0.253]) by mail.incore.de (Postfix) with ESMTP id D45F45EC56; Tue, 24 Jan 2012 11:57:46 +0100 (CET) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.incore (Postfix) with ESMTP id C6395450A0; Tue, 24 Jan 2012 11:57:46 +0100 (CET) Message-ID: <4F1E8EAA.2020905@incore.de> Date: Tue, 24 Jan 2012 11:57:46 +0100 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: azanar@carrel.org Subject: Re: pf not seeing inbound packets on netgraph interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:15:01 -0000 Hi Ed, > I am running into a roadblock getting PF to filter traffic on > a Netgraph interface representing an L2TP/IPSec connection. > The problem I have is that PF only sees traffic on the outbound > side of the netgraph interface. This happens because the L2TP packets are tagged with an IPSEC-flag for later use by ipfw, and this flag is passed to the packets coming from ng0. Thats done by the netgraph under control of mpd, or better: mpd does nothing to clear this flag. With net.inet.ipsec.filtertunnel=1 you can ignore this IPSEC-flag but only global for all interfaces in the system. Thats probably not what you want, especially not for the real hardware interface the IPSEC-tunnel is going through. I think L2TP under control of mpd should work independent of the existence of an IPSEC-tunnel and therefore clear this flag: --- ng_l2tp.c.orig 2010-04-15 14:40:02.000000000 +0200 +++ ng_l2tp.c 2012-01-23 17:09:41.000000000 +0100 @@ -752,6 +752,7 @@ hookpriv_p hpriv = NULL; hook_p hook = NULL; struct mbuf *m; + struct m_tag *mtag; u_int16_t tid, sid; u_int16_t hdr; u_int16_t ns, nr; @@ -996,6 +997,11 @@ ERROUT(0); } + /* Delete an existing ipsec tag */ + mtag = m_tag_find(m, PACKET_TAG_IPSEC_IN_DONE, NULL); + if (mtag != NULL) + m_tag_delete(m, mtag); + /* Deliver data */ NG_FWD_NEW_DATA(error, item, hook, m); This patch for the l2tp netgraph node does the job and you can use pf on the ng0 interface without any restrections. Regards, -- Dr. Andreas Longwitz Data Service GmbH Beethovenstr. 2A 23617 Stockelsdorf Amtsgericht Lübeck, HRB 318 BS Geschäftsführer: Wilfried Paepcke, Dr. Andreas Longwitz, Josef Flatau From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 11:15:50 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7717F106566C for ; Tue, 24 Jan 2012 11:15:50 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 0A0528FC08 for ; Tue, 24 Jan 2012 11:15:50 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id 91070358C5C; Tue, 24 Jan 2012 12:15:47 +0100 (CET) To: Nikolay Denev X-PHP-Script: mail.leon.pl/index.php for 91.195.135.101 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 12:15:47 +0100 From: Marcin Markowski In-Reply-To: <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> References: <159d4fbce722663a84f3cea12da828a5@leon.pl> <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> Message-ID: <62333c6068c0df493b2e4e9ec86f34d2@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Cc: freebsd-net@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:15:50 -0000 On 24.01.2012 09:18, Nikolay Denev wrote: > On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote: > >> Hello, >> >> This message has been sent to freebsd-performance@ but got >> the information that should contact also with freebsd-net@. >> >> We use FreeBSD as sniffer (libpcap programs) and we experience >> performance problems when incoming traffic is greater than >> 7.5Gbps/s. >> If we check 'top' we see that first irq from network card is using >> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >> CPU), >> and both systems behave the same. In logs we see also: >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and >> NIC Intel X520-DA2. >> >> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >> because everything is fine when we use other server configuration: >> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >> >> My /boot/loader.conf: >> kern.ipc.nmbclusters=262144 >> hw.ixgbe.rxd=2048 >> hw.ixgbe.txd=2048 >> hw.ixgbe.num_queues=16 >> >> /etc/sysctl.conf >> hw.intr_storm_threshold=10000 >> >> -- >> Marcin Markowski > > > Hi, > > Maybe you want to take a loot at NETMAP : > http://info.iet.unipi.it/~luigi/netmap/ > There is a libpcap wrapper library, so you can use it with unchanged > pcap consumers, > and get great performance increase. > I'm not sure that the patches are updated for 8 and 9 though, since > the initial commit to HEAD > there were several related changes. > > P.S.: It is important also what is you packet rate, since 7.5Gbps > with jumbo packets or 64 byte packets > are very different things :) Hi Nikolay, I tried to compile the kernel with NETMAP on FreeBSD 8 and 9, but I get warnings and the compilation ends. cc1: warnings being treated as errors ../../../dev/netmap/netmap.c: In function 'netmap_memory_init': ../../../dev/netmap/netmap.c:1557: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c:1564: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_memory_fini': ../../../dev/netmap/netmap.c:1607: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_init': ../../../dev/netmap/netmap.c:1636: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' *** Error code 1 I'll try HEAD and see if it will be the same. -- Marcin Markowski From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 11:26:32 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78E78106566B for ; Tue, 24 Jan 2012 11:26:32 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 3902F8FC1E for ; Tue, 24 Jan 2012 11:26:32 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id 92DFA358C5D; Tue, 24 Jan 2012 12:26:30 +0100 (CET) To: Nikolay Denev X-PHP-Script: mail.leon.pl/index.php for 91.195.135.101 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 12:26:30 +0100 From: Marcin Markowski In-Reply-To: <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> References: <159d4fbce722663a84f3cea12da828a5@leon.pl> <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> Message-ID: <86d5416f7f55a71fb01fd86e9051d678@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Cc: freebsd-net@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:26:32 -0000 On 24.01.2012 09:18, Nikolay Denev wrote: > On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote: > >> Hello, >> >> This message has been sent to freebsd-performance@ but got >> the information that should contact also with freebsd-net@. >> >> We use FreeBSD as sniffer (libpcap programs) and we experience >> performance problems when incoming traffic is greater than >> 7.5Gbps/s. >> If we check 'top' we see that first irq from network card is using >> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >> CPU), >> and both systems behave the same. In logs we see also: >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and >> NIC Intel X520-DA2. >> >> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >> because everything is fine when we use other server configuration: >> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >> >> My /boot/loader.conf: >> kern.ipc.nmbclusters=262144 >> hw.ixgbe.rxd=2048 >> hw.ixgbe.txd=2048 >> hw.ixgbe.num_queues=16 >> >> /etc/sysctl.conf >> hw.intr_storm_threshold=10000 >> >> -- >> Marcin Markowski > > > Hi, > > Maybe you want to take a loot at NETMAP : > http://info.iet.unipi.it/~luigi/netmap/ > There is a libpcap wrapper library, so you can use it with unchanged > pcap consumers, > and get great performance increase. > I'm not sure that the patches are updated for 8 and 9 though, since > the initial commit to HEAD > there were several related changes. > > P.S.: It is important also what is you packet rate, since 7.5Gbps > with jumbo packets or 64 byte packets > are very different things :) > > Regards, > Nikolay I forgot to answer the P.S. Our switch shows that the peak was 2Mpps. -- Marcin Markowski From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 11:37:30 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DAA4106564A for ; Tue, 24 Jan 2012 11:37:30 +0000 (UTC) (envelope-from prvs=1370443d68=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 8D09A8FC15 for ; Tue, 24 Jan 2012 11:37:29 +0000 (UTC) X-Spam-Processed: mail1.multiplay.co.uk, Tue, 24 Jan 2012 11:26:07 +0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50017683607.msg for ; Tue, 24 Jan 2012 11:26:07 +0000 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1370443d68=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-net@freebsd.org Message-ID: <66BBF95CED1F4B2B8233EC2D5A610ACD@multiplay.co.uk> From: "Steven Hartland" To: "Nikolay Denev" , "Marcin Markowski" References: <159d4fbce722663a84f3cea12da828a5@leon.pl><6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> <62333c6068c0df493b2e4e9ec86f34d2@leon.pl> Date: Tue, 24 Jan 2012 11:26:19 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:37:30 -0000 ----- Original Message ----- From: "Marcin Markowski" > I tried to compile the kernel with NETMAP on FreeBSD 8 and 9, but I get > warnings and > the compilation ends. > > cc1: warnings being treated as errors > ../../../dev/netmap/netmap.c: In function 'netmap_memory_init': > ../../../dev/netmap/netmap.c:1557: warning: format '%d' expects type > 'int', but argument 7 has type 'size_t' > ../../../dev/netmap/netmap.c:1564: warning: format '%d' expects type > 'int', but argument 7 has type 'size_t' > ../../../dev/netmap/netmap.c: In function 'netmap_memory_fini': > ../../../dev/netmap/netmap.c:1607: warning: format '%d' expects type > 'int', but argument 2 has type 'size_t' > ../../../dev/netmap/netmap.c: In function 'netmap_init': > ../../../dev/netmap/netmap.c:1636: warning: format '%d' expects type > 'int', but argument 2 has type 'size_t' > *** Error code 1 > > I'll try HEAD and see if it will be the same. If its just that error, change %d to %ld and that should fix it. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 14:53:12 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F42F106566B for ; Tue, 24 Jan 2012 14:53:12 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.64.117]) by mx1.freebsd.org (Postfix) with ESMTP id A5DA38FC1C for ; Tue, 24 Jan 2012 14:53:11 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.5/8.14.5) with ESMTP id q0OErAjq048699; Tue, 24 Jan 2012 18:53:10 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.5/8.14.5/Submit) id q0OErAJA048698; Tue, 24 Jan 2012 18:53:10 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 24 Jan 2012 18:53:10 +0400 From: Gleb Smirnoff To: rozhuk.im@gmail.com Message-ID: <20120124145310.GC48157@FreeBSD.org> References: <4f1dcc8e.45c8cc0a.2935.2254@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <4f1dcc8e.45c8cc0a.2935.2254@mx.google.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@FreeBSD.org Subject: Re: ng_bridge and locks X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 14:53:12 -0000 On Tue, Jan 24, 2012 at 06:09:30AM +0900, rozhuk.im@gmail.com wrote: r> I found a comment in the code: r> /* r> * This node has all kinds of stuff that could be screwed by SMP. r> * Until it gets it's own internal protection, we go through in r> * single file. This could hurt a machine bridging beteen two r> * GB ethernets so it should be fixed. r> * When it's fixed the process SHOULD NOT SLEEP, spinlocks please! r> * (and atomic ops ) r> */ r> r> mtx_init(...., MTX_DEF); r> How bad to use netgraph node MTX_DEF mutex? It would be correct to use MTX_DEF mutex to lock the ng_bridge node. You need smth like a mutex per hash entry, and if all done correctly, then you can remove NG_NODE_FORCE_WRITER(). -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 15:23:32 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CAE0106566B for ; Tue, 24 Jan 2012 15:23:32 +0000 (UTC) (envelope-from kirk.davis@epsb.ca) Received: from OWA01.EDU.epsb.ca (owa01.epsb.ca [198.161.119.28]) by mx1.freebsd.org (Postfix) with ESMTP id 4FA088FC08 for ; Tue, 24 Jan 2012 15:23:31 +0000 (UTC) Received: from Exchange26.EDU.epsb.ca ([10.0.5.123]) by OWA01.EDU.epsb.ca with Microsoft SMTPSVC(6.0.3790.4675); Tue, 24 Jan 2012 08:11:31 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Date: Tue, 24 Jan 2012 08:07:09 -0700 Message-ID: <529374128DC1B04D9D037911B8E8F0531095BC48@Exchange26.EDU.epsb.ca> In-Reply-To: <86d5416f7f55a71fb01fd86e9051d678@leon.pl> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Performance problem using Intel X520-DA2 Thread-Index: AczaiwssihrpPZryR8WNW2krBHcgRQAHduNw References: <159d4fbce722663a84f3cea12da828a5@leon.pl><6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> <86d5416f7f55a71fb01fd86e9051d678@leon.pl> From: "Kirk Davis" To: X-OriginalArrivalTime: 24 Jan 2012 15:11:31.0205 (UTC) FILETIME=[73FB5B50:01CCDAAA] Subject: RE: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 15:23:32 -0000 DQoNCj4tLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPk9uIDI0LjAxLjIwMTIgMDk6MTgsIE5p a29sYXkgRGVuZXYgd3JvdGU6DQo+PiBPbiBKYW4gMjMsIDIwMTIsIGF0IDExOjM5IFBNLCBNYXJj aW4gTWFya293c2tpIHdyb3RlOg0KPj4NCj4+PiBIZWxsbywNCj4+Pg0KPj4+IFRoaXMgbWVzc2Fn ZSBoYXMgYmVlbiBzZW50IHRvIGZyZWVic2QtcGVyZm9ybWFuY2VAIGJ1dCBnb3QgdGhlIA0KPj4+ IGluZm9ybWF0aW9uIHRoYXQgc2hvdWxkIGNvbnRhY3QgYWxzbyB3aXRoIGZyZWVic2QtbmV0QC4N Cj4+Pg0KPj4+IFdlIHVzZSBGcmVlQlNEIGFzIHNuaWZmZXIgKGxpYnBjYXAgcHJvZ3JhbXMpIGFu ZCB3ZSBleHBlcmllbmNlIA0KPj4+IHBlcmZvcm1hbmNlIHByb2JsZW1zIHdoZW4gaW5jb21pbmcg dHJhZmZpYyBpcyBncmVhdGVyIHRoYW4gNy41R2Jwcy9zLg0KPj4+IElmIHdlIGNoZWNrICd0b3An IHdlIHNlZSB0aGF0IGZpcnN0IGlycSBmcm9tIG5ldHdvcmsgY2FyZCBpcyB1c2luZyANCj4+PiAx MDAlIENQVS4gSSd2ZSB0ZXN0ZWQgdGhpcyBvbiBGcmVlQlNEIDguMi1SRUxFQVNFIGFuZCA5LjAt UkVMRUFTRSAob24gDQo+Pj4gOS4wIHdlIGNhbiBzZWUgYWxzbyBrZXJuZWwgdGhyZWFkIG5hbWVk IHtpeDAgcXVlfSB1c2luZyAxMDAlIENQVSksIA0KPj4+IGFuZCBib3RoIHN5c3RlbXMgYmVoYXZl IHRoZSBzYW1lLiBJbiBsb2dzIHdlIHNlZSBhbHNvOg0KPj4+IGludGVycnVwdCBzdG9ybSBkZXRl Y3RlZCBvbiAiaXJxMjY4OiI7IHRocm90dGxpbmcgaW50ZXJydXB0IHNvdXJjZQ0KPj4+DQo+Pj4g T3VyIHNlcnZlciBwbGF0Zm9ybSBpcyBJbnRlbCBTUjI2MDBVUkJSUCwgMnggWGVvbiBYNTY1MCwg NkdCIFJBTSBhbmQgDQo+Pj4gTklDIEludGVsIFg1MjAtREEyLg0KPj4+DQo+Pj4gSSdtIG5vdCBz dXJlIGlmIHByb2JsZW0gaXMgd2l0aCBOSUMgb3IgbW90aGVyYm9hcmQgaW4gU1IyNjAwVVJCUlAs IA0KPj4+IGJlY2F1c2UgZXZlcnl0aGluZyBpcyBmaW5lIHdoZW4gd2UgdXNlIG90aGVyIHNlcnZl ciBjb25maWd1cmF0aW9uOg0KPj4+IEludGVsIFNSMTYzMEdQLCAxeCBYZW9uIFgzNDUwLCA4R0Ig UkFNLCBOSUMgWDUyMC1EQTINCj4+Pg0KPj4+IE15IC9ib290L2xvYWRlci5jb25mOg0KPj4+IGtl cm4uaXBjLm5tYmNsdXN0ZXJzPTI2MjE0NA0KPj4+IGh3Lml4Z2JlLnJ4ZD0yMDQ4DQo+Pj4gaHcu aXhnYmUudHhkPTIwNDgNCj4+PiBody5peGdiZS5udW1fcXVldWVzPTE2DQo+Pj4NCj4+PiAvZXRj L3N5c2N0bC5jb25mDQo+Pj4gaHcuaW50cl9zdG9ybV90aHJlc2hvbGQ9MTAwMDANCj4+Pg0KDQpJ IGp1c3QgZmluaXNoZWQgYSBidW5jaCBvZiBwZXJmb3JtYW5jZSB0ZXN0cyBvbiB0aGlzIGNhcmQu ICBJbiBteSBjYXNlIEkgd2FzIHRyeWluZyB0byBnZXQgYXMgY2xvc2UgdG8gYSBmdWxsIDEwR2Iv cyBhcyBwb3NzaWJsZSBvbiBhIERlbGwgUjcxMCBidXQgSSBoYXZlbid0IHlldCB0cmllZCBhbnkg c25pZmZpbmcuDQoNCkhhdmUgeW91IHRyaWVkIHR1cm5pbmcgb24gTFJPIG9uIHRoZSBpbnRlcmZh Y2UgKCBpZmNvbmZpZyBscm8gKS4gIEluIG15IGNhc2UgdGhpcyBtYWRlIGEgYmlnIGRpZmZlcmVu Y2UgYW5kIEkgY2FuIG5vdyBnZXQgOS40MUdiL3Mgd2l0aG91dCBoaWdoIENQVSBvciBpbnRlcnJ1 cHQgc3Rvcm1zLiAgSSBhbSBhbHNvIHVzaW5nIEphY2sncyBsYXRlc3QgZHJpdmVyIGRvd25sb2Fk ZWQgZnJvbSBpbnRlbCAodmVyc2lvbiAyLjQuNCkuICBFdmVuIHRoZSBkcml2ZXIgaW4gOS4wIHdh cyBvbGRlci4NCg0KSGVyZSBpcyB3aGF0IEkgaGF2ZSANCi9ldGMvc3lzY3RsLmNvbmYNCiMgSW5j cmVhc2UgdGhlIG5ldHdvcmsgYnVmZmVycw0Ka2Vybi5pcGMubm1iY2x1c3RlcnM9MjYyMTQ0DQpr ZXJuLmlwYy5tYXhzb2NrYnVmPTQxOTQzMDQNCmh3LmludHJfc3Rvcm1fdGhyZXNob2xkPTkwMDAN Cmtlcm4uaXBjLm5tYmp1bWJvcD0yNjIxNDQNCg0KL2Jvb3QvbG9hZGVyLmNvbmYNCml4Z2JlX2xv YWQ9IllFUyINCmh3Lml4Z2JlLnR4ZD00MDk2DQpody5peGdiZS5yeGQ9NDA5Ng0KDQotLS0tIEtp cmsNCg== From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 16:14:33 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7DB21065670 for ; Tue, 24 Jan 2012 16:14:32 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7D6338FC12 for ; Tue, 24 Jan 2012 16:14:32 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rpj0w-0002yh-Iv for freebsd-net@freebsd.org; Tue, 24 Jan 2012 17:14:30 +0100 Received: from office-nat.spylog.net ([193.169.234.6]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Jan 2012 17:14:30 +0100 Received: from citrin by office-nat.spylog.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Jan 2012 17:14:30 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Anton Yuzhaninov Date: Tue, 24 Jan 2012 16:14:17 +0000 (UTC) Lines: 90 Sender: Anton Yuzhaninov Message-ID: X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: office-nat.spylog.net User-Agent: tin/1.9.6-20101126 ("Burnside") (UNIX) (FreeBSD/10.0-CURRENT (i386)) Subject: livelock with full loaded em(4) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 16:14:33 -0000 Hello. I have test boxes with em(4) network card - Intel 82563EB FreeBSD version - 8.2 stable from 2012-01-15, amd64 When this NIC is full loaded livelock occurs - system is unresponsive even from local console. To generate load I use netsend from /usr/src/tools/tools/netrate/ but other traffic source (e. g. TCP instead UDP) cause same problem. There is 2 quese of this livelock: 1. With full load kernel thread "em1 taskq" hogs CPU. top -zISHP for interface load a bit less, than full. Traffic is generated by # netsend 172.16.0.2 9001 8500 14300 3600 where 14300 is packets per second: 112 processes: 10 running, 82 sleeping, 20 waiting CPU 0: 0.0% user, 0.0% nice, 27.1% system, 0.0% interrupt, 72.9% idle CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 2: 2.3% user, 0.0% nice, 97.7% system, 0.0% interrupt, 0.0% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 6: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 26M Active, 378M Inact, 450M Wired, 132K Cache, 63M Buf, 15G Free Swap: 8192M Total, 8192M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 7737 ayuzhaninov 119 0 5832K 1116K CPU2 2 0:04 100.00% netsend 0 root -68 0 0K 144K - 0 2:17 22.27% {em1 taskq} top -zISHP for full interface load (some drops occurs), load is generated by # netsend 172.16.0.2 9001 8500 14400 3600 112 processes: 11 running, 81 sleeping, 20 waiting CPU 0: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle CPU 1: 4.1% user, 0.0% nice, 95.9% system, 0.0% interrupt, 0.0% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 6: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 26M Active, 378M Inact, 450M Wired, 132K Cache, 63M Buf, 15G Free Swap: 8192M Total, 8192M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 0 root -68 0 0K 144K CPU0 0 2:17 100.00% {em1 taskq} 7759 ayuzhaninov 119 0 5832K 1116K CPU1 1 0:01 100.00% netsend So pps increased from 14300 to 14400 (0.7%), but CPU load from "em1 taskq" thread increased from 27.1% to 100.00% This at least strange, but system still works fine unil I run sysctl dev.cpu.0.temperature 2. sysctl handler code for coretemp must be executed on target cpu, e. g. for dev.cpu.0.temperature code executed on CPU0. If CPU0 is fully loaded by "em1 taskq" sysctl handler for dev.cpu.0.temperature acquires Giant mutex lock then tries to run code on CPU0, but it can't - CPU0 is busy. If Giant mutex hold for long time system is unresponsive. In my case Giant mutex acquired when sysctl dev.cpu.0.temperature started and hold all time while netsend is running. This seems to be a scheduler problem: 1. Why "em1 taskq" runs only on CPU0 (there is no affinity for this tread)? # procstat -k 0 | egrep '(PID|em1)' PID TID COMM TDNAME KSTACK 0 100038 kernel em1 taskq # cpuset -g -t 100038 tid 100038 mask: 0, 1, 2, 3, 4, 5, 6, 7 2. Why "em1 taskq" is not preempted to execute sysctl handler code? This is not short term condition - is netsend running for a hour, "em1 taskq" is not preempted for a hour - sysctl all this time in running state but don't have a chance to be executed. -- Anton Yuzhaninov P. S. I tried to use EM_MULTIQUEUE, but this is don't help in this case. From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 17:17:15 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06AF41065672 for ; Tue, 24 Jan 2012 17:17:15 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id BA35A8FC0A for ; Tue, 24 Jan 2012 17:17:14 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id 2DABC358C5C; Tue, 24 Jan 2012 18:17:13 +0100 (CET) To: X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 18:17:13 +0100 From: Marcin Markowski In-Reply-To: <529374128DC1B04D9D037911B8E8F0531095BC48@Exchange26.EDU.epsb.ca> References: <159d4fbce722663a84f3cea12da828a5@leon.pl><6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> <86d5416f7f55a71fb01fd86e9051d678@leon.pl> <529374128DC1B04D9D037911B8E8F0531095BC48@Exchange26.EDU.epsb.ca> Message-ID: X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Subject: RE: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 17:17:15 -0000 On 24.01.2012 16:07, Kirk Davis wrote: >>-----Original Message----- >>On 24.01.2012 09:18, Nikolay Denev wrote: >>> On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote: >>> >>>> Hello, >>>> >>>> This message has been sent to freebsd-performance@ but got the >>>> information that should contact also with freebsd-net@. >>>> >>>> We use FreeBSD as sniffer (libpcap programs) and we experience >>>> performance problems when incoming traffic is greater than >>>> 7.5Gbps/s. >>>> If we check 'top' we see that first irq from network card is using >>>> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >>>> (on >>>> 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), >>>> and both systems behave the same. In logs we see also: >>>> interrupt storm detected on "irq268:"; throttling interrupt source >>>> >>>> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM >>>> and >>>> NIC Intel X520-DA2. >>>> >>>> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >>>> because everything is fine when we use other server configuration: >>>> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >>>> >>>> My /boot/loader.conf: >>>> kern.ipc.nmbclusters=262144 >>>> hw.ixgbe.rxd=2048 >>>> hw.ixgbe.txd=2048 >>>> hw.ixgbe.num_queues=16 >>>> >>>> /etc/sysctl.conf >>>> hw.intr_storm_threshold=10000 >>>> > > I just finished a bunch of performance tests on this card. In my > case I was trying to get as close to a full 10Gb/s as possible on a > Dell R710 but I haven't yet tried any sniffing. > > Have you tried turning on LRO on the interface ( ifconfig lro ). In > my case this made a big difference and I can now get 9.41Gb/s without > high CPU or interrupt storms. I am also using Jack's latest driver > downloaded from intel (version 2.4.4). Even the driver in 9.0 was > older. > > Here is what I have > /etc/sysctl.conf > # Increase the network buffers > kern.ipc.nmbclusters=262144 > kern.ipc.maxsockbuf=4194304 > hw.intr_storm_threshold=9000 > kern.ipc.nmbjumbop=262144 > > /boot/loader.conf > ixgbe_load="YES" > hw.ixgbe.txd=4096 > hw.ixgbe.rxd=4096 > > ---- Kirk Hi Kirk, I did not notice any change after turning on LRO. When checking stats I see that LRO is not used (perhaps because the interface is receiving traffic from port mirror on the switch). sysctl output from dev.ix.0: http://pastebin.com/fkRp7Py5 -- Marcin Markowski From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 06:02:14 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FBCE106564A for ; Wed, 25 Jan 2012 06:02:14 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norman-vivat.ru [89.250.210.68]) by mx1.freebsd.org (Postfix) with ESMTP id 838288FC08 for ; Wed, 25 Jan 2012 06:02:13 +0000 (UTC) Received: from bsdrookie.norma.com. ([IPv6:fd00::780]) by elf.hq.norma.perm.ru (8.14.4/8.14.4) with ESMTP id q0P5Rwwv008270 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 25 Jan 2012 10:27:58 +0500 (YEKT) (envelope-from eugene@zhegan.in) Message-ID: <4F1F92DE.9060200@zhegan.in> Date: Wed, 25 Jan 2012 11:27:58 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0) Gecko/20111001 Thunderbird/7.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [IPv6:fd00::30a]); Wed, 25 Jan 2012 10:27:58 +0500 (YEKT) X-Spam-Status: No hits=1.7 bayes=0.5 testhits RDNS_NONE=0.793, TO_NO_BRKTS_DIRECT=0.904, TO_NO_BRKTS_NORDNS=0.001 autolearn=no version=3.3.2 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru Subject: low network speed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 06:02:14 -0000 Hi. I'm suffering from low network performance on one of my FreeBSDs. I have an i386 8.2-RELEASE machine with an fxp(4) adapter. It's connected though a bunch of catalysts 2950 to another 8.2. While other machines in this server room using the same sequence of switches and the same target source server (which, btw, is equipped with an em(4) and a gigabit link bia catalyst 3750) show sufficient speed, this particular machine while using scp starts with a speed of 200 Kbytes/sec and while copying the file shows speed about 600-800 Kbytes/sec. I've added this tweak to the sysctl: net.local.stream.recvspace=196605 net.local.stream.sendspace=196605 net.inet.tcp.sendspace=196605 net.inet.tcp.recvspace=196605 net.inet.udp.recvspace=196605 kern.ipc.maxsockbuf=2621440 kern.ipc.somaxconn=4096 net.inet.tcp.sendbuf_max=524288 net.inet.tcp.recvbuf_max=524288 With these settings the copying starts at 9.5 Mbytes/sec speed, but then, as file is copying, drops down to 3.5 Megs/sec in about two-three minutes. Is there some way to maintain 9.5 Mbytes/sec (I like this speed more) ? Thanks. Eugene. P.S. This machine also runs zfs, I don't know if it's important but I decided to mention it. From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 07:12:41 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7C4B1065673; Wed, 25 Jan 2012 07:12:41 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4A7D48FC0C; Wed, 25 Jan 2012 07:12:40 +0000 (UTC) Received: by vcmm1 with SMTP id m1so27059vcm.13 for ; Tue, 24 Jan 2012 23:12:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Id9fczfgFoBQWogvc28rJvIOEwkrfZSk4atflRr27Tk=; b=KmvBhU74FqjdgLiqXCa+HucZG1/QZrXyA2S1iaEXmWlm2tIfSM6P3PFsjkKadP3ByR w2RHQXcE/73oXecK0FoVvTSki1lfhiUwr2NxTBAVZemvgjgnTkv7aaC1d03o4d1SG1Ve ihsYNfSWMEJ6nScD2OFTe6rDfAkuI9KoIlZyI= MIME-Version: 1.0 Received: by 10.52.74.163 with SMTP id u3mr8091806vdv.91.1327475560466; Tue, 24 Jan 2012 23:12:40 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.71.241 with HTTP; Tue, 24 Jan 2012 23:12:40 -0800 (PST) In-Reply-To: <20120122195130.360261ce.ray@freebsd.org> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> Date: Tue, 24 Jan 2012 23:12:40 -0800 X-Google-Sender-Auth: JuVWj5UrP9u-xuS7xBtP0RbAsIk Message-ID: From: Adrian Chadd To: Aleksandr Rybalko Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Stefan Bethke , freebsd-arch@freebsd.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 07:12:41 -0000 So when will you two have something consensus-y to commit? :-) What I'm hoping for is: * some traction on the MII bus / MDIO bus split and tidyup from stb, which is nice; * ray's switch API for speaking to userland with; * agreeing on whether the correct place to put the driver(s) is where stb, ray, or a mix of both approaches says so. I've been (mostly) trying to stay out of this to see where both of you have gone. I think we've made some good progress; now it's time to solidify a design for the first pass of what we want in -HEAD and figure out how to move forward. Adrian On 22 January 2012 09:51, Aleksandr Rybalko wrote: > On Sun, 22 Jan 2012 16:31:06 +0100 > Stefan Bethke wrote: > > > Am 20.01.2012 um 21:13 schrieb Aleksandr Rybalko: > > > > > It include sys/mips/conf/AR7240, that together with hints file is > > > good example for typical AR7240 setup. > > > > I=C4m heaving trouble getting this to work. The patch applies cleanly > > and I can get a kernel compiled and booted, but neither arge0 nor > > arge1 appear to be functional. I had to roll my own kernel config as > > your AR7240 hangs before printing anything on my TL-MR3420. > > Yeah, I know where is problem, to proper attach switch framework to > arge, arge must be regular NIC. Here is the patch for that: > http://my.ddteam.net/files/2012-01-22_arge.patch > Hope it will apply cleanly. > > Patch have fixed both arge problems (problem for allocation of ring > buffer, and stray interrupts) + remove most phymask bits + whitespace > cleanup. > > Thank you for testing that Stefan. > > P.S. I can't test clear SoC config on my board, because my board id > D-Link DIR-615_E4 with modified U-Boot in it, which able to load only > FW images, but not ELF kernel. So I test it with ZRouter.org FW image > instead. > > P.P.S. can you also show me network part of your config and hints files. > > P.P.P.S. still working on your previous question about subj, already > begin work on more wide documentation on wiki, but still far enough :) > "http://wiki.freebsd.org/AleksandrRybalko/Switch Framework" > > > > > dmesg and devinfo below. > > > > > > Stefan > > > > CPU platform: Atheros AR7241 rev 1 > > CPU Frequency=3D400 MHz > > CPU DDR Frequency=3D400 MHz > > CPU AHB Frequency=3D200 MHz > > platform frequency: 400000000 > > arguments: > > a0 =3D 00000008 > > a1 =3D a1f87fb0 > > a2 =3D a1f88470 > > a3 =3D 00000004 > > Cmd line:argv is invalid > > Environment: > > envp is invalid > > Cache info: > > picache_stride =3D 4096 > > picache_loopcount =3D 16 > > pdcache_stride =3D 4096 > > pdcache_loopcount =3D 8 > > cpu0: MIPS Technologies processor v116.147 > > MMU: Standard TLB, 16 entries > > L1 i-cache: 4 ways of 512 sets, 32 bytes per line > > L1 d-cache: 4 ways of 256 sets, 32 bytes per line > > Config1=3D0x9ee3519e > > Config3=3D0x20 > > KDB: debugger backends: ddb > > KDB: current backend: ddb > > Copyright (c) 1992-2012 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, > > 1994 The Regents of the University of California. All rights reserved. > > FreeBSD is a registered trademark of The FreeBSD Foundation. > > FreeBSD 10.0-CURRENT #1: Thu Jan 1 01:00:00 CET 1970 > > stb@dummy > :/home/stb/working/fe/obj/mipseb/mips.mipseb/home/stb/working/fe/freebsd/= sys/TL-MR3420D > > mips WARNING: WITNESS option enabled, expect reduced performance. > > real memory =3D 33554432 (32768K bytes) > > avail memory =3D 25567232 (24MB) > > random device not loaded; using insecure entropy > > nexus0: > > nexus0: failed to add child: arge0 > > nexus0: failed to add child: arge1 > > clock0: on nexus0 > > Timecounter "MIPS32" frequency 200000000 Hz quality 800 > > Event timer "MIPS32" frequency 200000000 Hz quality 800 > > apb0 at irq 4 on nexus0 > > uart0: <16550 or compatible> on apb0 > > uart0: console (115200,n,8,1) > > gpio0: on apb0 > > gpio0: [GIANT-LOCKED] > > gpio0: function_set: 0x0 > > gpio0: function_clear: 0x0 > > gpio0: gpio pinmask=3D0x1943 > > gpioc0: on gpio0 > > gpiobus0: on gpio0 > > gpioled0: at pin(s) 0 on gpiobus0 > > gpioled1: at pin(s) 1 on gpiobus0 > > gpioled2: at pin(s) 3 on gpiobus0 > > ehci0: at mem > > 0x1b000100-0x1bffffff irq 1 on nexus0 usbus0: set host controller mode > > usbus0: EHCI version 1.0 > > usbus0: set host controller mode > > usbus0: on ehci0 > > arge0: at mem > > 0x19000000-0x19000fff irq 2 on nexus0 arge0: Overriding MAC from > > EEPROM arge0: No PHY specified, using mask 16 > > miibus0: on arge0 > > floatphy0 PHY 0 on miibus0 > > floatphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, > > 1000baseSX, 1000baseSX-FDX, 1000baseT, 1000baseT-master, > > 1000baseT-FDX, 1000baseT-FDX-master, auto switch0 PHY 1 on miibus0 > > switch0: 100baseTX, 100baseTX-FDX, 1000baseSX, 1000baseSX-FDX, > > 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master > > ar8x16_switch0: on > > switch0 arge0: Ethernet address: ff:ff:ff:ff:ff:ff arge1: > AR71xx built-in ethernet interface> at mem 0x1a000000-0x1a000fff irq > > 3 on nexus0 arge1: No PHY specified, using mask 15 arge1: Ethernet > > address: ff:ff:ff:ff:ff:00 spi0: at mem > > 0x1f000000-0x1f00000f on nexus0 spibus0: on spi0 mx25l0: > > at cs 0 on spibus0 mx25l0: s25s1032, sector > > 65536 bytes, 64 sectors ar71xx_wdog0: > > on nexus0 ar71xx_wdog0: Previous reset was due to watchdog timeout > > Timecounters tick every 1.000 msec > > usbus0: 480Mbps High Speed USB v2.0 > > ugen0.1: at usbus0 > > uhub0: on > > usbus0 WARNING: WITNESS option enabled, expect reduced performance. > > uhub0: 1 port with 1 removable, self powered > > Root mount waiting for: usbus0 > > Root mount waiting for: usbus0 > > Root mount waiting for: usbus0 > > ugen0.2: at usbus0 > > umass0: > > on usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0x0100 > > umass0:0:0:-1: Attached to scbus0 > > Trying to mount root from ufs:map/rootfs.uzip []... > > mountroot: waiting for device map/rootfs.uzip ... > > da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 > > da0: Removable Direct Access SCSI-0 > > device da0: 40.000MB/s transfers > > da0: 15252MB (31236096 512 byte sectors: 255H 63S/T 1944C) > > Trying to mount root from ufs:da0s1a []... > > warning: no time-of-day clock registered, system time will not be set > > accurately Setting hostuuid: aed4c502-193a-11e1-b662-74ea3ae4d920. > > Setting hostid: 0x6a714343. > > Entropy harvesting: interrupts ethernet point_to_point kickstart. > > Starting file system checks: > > /dev/da0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS > > /dev/da0s1a: clean, 3849245 free (1085 frags, 481020 blocks, 0.0% > > fragmentation) lock order reversal: > > 1st 0x8095691c ufs (ufs) > > @ /home/stb/working/fe/freebsd/sys/kern/vfs_mount.c:1245 2nd > > 0x80956e94 devfs (devfs) > > @ /home/stb/working/fe/freebsd/sys/kern/vfs_subr.c:2167 KDB: stack > > backtrace: db_trace_thread+30 (?,?,?,?) ra c3f597c000000018 sp 0 sz 0 > > db_trace_self+1c (?,?,?,?) ra c3f597d800000018 sp 0 sz 0 800776d8+34 > > (?,?,?,?) ra c3f597f0000001a0 sp 0 sz 0 kdb_backtrace+44 (?,?,?,?) ra > > c3f5999000000018 sp 0 sz 0 801b3bbc+34 (?,?,?,?) ra c3f599a800000020 > > sp 0 sz 0 witness_checkorder+9cc (?,?,803b1f7c,877) ra > > c3f599c800000050 sp 0 sz 1 __lockmgr_args+948 (?,?,80956eb4,?) ra > > c3f59a1800000070 sp 0 sz 1 vop_stdlock+4c (?,?,?,?) ra > > c3f59a8800000028 sp 0 sz 0 VOP_LOCK1_APV+e4 (?,?,?,?) ra > > c3f59ab000000020 sp 0 sz 0 _vn_lock+84 (?,?,?,?) ra c3f59ad000000048 > > sp 0 sz 0 vget+c8 (?,?,?,?) ra c3f59b1800000030 sp 0 sz 0 > > devfs_allocv+100 (?,?,?,?) ra c3f59b4800000038 sp 0 sz 0 > > 800ba9b4+4c (?,?,?,?) ra c3f59b8000000028 sp 0 sz 0 > > vflush+6c (?,?,0,80991300) ra c3f59ba8000000f8 sp 0 sz 1 > > 800baa74+54 (?,?,?,?) ra c3f59ca000000028 sp 0 sz 0 > > dounmount+3f0 (809e8000,?,?,?) ra c3f59cc800000050 sp 100000000 sz 0 > > sys_unmount+39c (?,?,?,?) ra c3f59d18000000b0 sp 0 sz 0 > > trap+7f4 (?,?,?,?) ra c3f59dc8000000b8 sp 0 sz 0 > > MipsUserGenException+10c (?,?,?,404b53c0) ra c3f59e8000000000 sp 0 sz > > 0 pid 70 > > Mounting local file systems:. > > Setting hostname: whitebox.lassitu.de. > > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > > floatphy0: found master switch0 > > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > > arge0: link state changed to DOWN > > Starting Network: lo0 arge0 arge1. > > lo0: flags=3D8049 metric 0 mtu 16384 > > options=3D3 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > > inet 127.0.0.1 netmask 0xff000000 > > nd6 options=3D21 > > arge0: flags=3D8843 metric 0 > > mtu 1500 options=3D80000 > > ether ff:ff:ff:ff:ff:ff > > inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 > > inet6 fe80::481f:1b46:dbf1:bd78%arge0 prefixlen 64 scopeid > > 0x2 nd6 options=3D29 > > media: Ethernet autoselect (100baseTX ) > > status: no carrier > > arge1: flags=3D8843 metric 0 > > mtu 1500 ether ff:ff:ff:ff:ff:00 > > inet 44.128.65.33 netmask 0xffffffc0 broadcast 44.128.65.63 > > inet6 fe80::481f:1b46:dbf1:bd78%arge1 prefixlen 64 scopeid > > 0x3 nd6 options=3D29 > > media: Ethernet 100baseTX > > status: active > > add net default: gateway 44.128.65.1 > > add net ::ffff:0.0.0.0: gateway ::1 > > add net ::0.0.0.0: gateway ::1 > > add net fe80::: gateway ::1 > > add net ff02::: gateway ::1 > > Mounting NFS file systems:mount_nfs: diesel: hostname nor servname > > provided, or not known . > > Creating and/or trimming log files. > > No core dumps found. > > ELF ldconfig path: /lib /usr/lib /usr/lib/compat > > Setting date via ntp. > > Error : hostname nor servname provided, or not known > > 22 Jan 15:01:04 ntpdate[566]: can't find host diesel > > > > 22 Jan 15:01:04 ntpdate[566]: no servers can be used, exiting > > Clearing /tmp (X related). > > Mounting late file systems:mount_nfs: diesel: hostname nor servname > > provided, or not known . > > Mounting /etc/fstab filesystems failed,Jan 22 15:01:12 init: /bin/sh > > on /etc/rc terminated abnormally, going to single user mode Enter > > full pathname of shell or RETURN for /bin/sh: > > # devinfo > > nexus0 > > clock0 > > apb0 > > uart0 > > gpio0 > > gpioc0 > > gpiobus0 > > gpioled0 > > gpioled1 > > gpioled2 > > ehci0 > > usbus0 > > uhub0 > > umass0 > > arge0 > > miibus0 > > floatphy0 > > switch0 > > ar8x16_switch0 > > arge1 > > spi0 > > spibus0 > > mx25l0 > > ar71xx_wdog0 > > # > > > > > > -- > > Stefan Bethke Fon +49 151 14070811 > > > > > > > > > -- > Aleksandr Rybalko > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 07:25:59 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40046106566B for ; Wed, 25 Jan 2012 07:25:59 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id C82818FC13 for ; Wed, 25 Jan 2012 07:25:58 +0000 (UTC) Received: by werg1 with SMTP id g1so5749687wer.13 for ; Tue, 24 Jan 2012 23:25:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=psmRduNMyl+F3+zGcHdNvIhlc22qivJn2beBMyPXG2w=; b=R1fSQisQgC3Q1nwlVPio0YSpJSBCO4sO6mipRnXhc0tK9aip4m/xIycZRvM2IP58Dg f205sjt0oYcOXjS5AQI2f73l+JL/+oO4PGFPgyN1FxTBlh9QT3r4YtxhxFnM/8CS7uG4 22ZYtY4iviIdC4B6PPOJb3x0oe/W0clt1T1xs= MIME-Version: 1.0 Received: by 10.216.132.211 with SMTP id o61mr2261016wei.58.1327474461208; Tue, 24 Jan 2012 22:54:21 -0800 (PST) Received: by 10.223.101.196 with HTTP; Tue, 24 Jan 2012 22:54:21 -0800 (PST) In-Reply-To: <4F1F92DE.9060200@zhegan.in> References: <4F1F92DE.9060200@zhegan.in> Date: Tue, 24 Jan 2012 22:54:21 -0800 Message-ID: From: Kevin Oberman To: "Eugene M. Zheganin" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org Subject: Re: low network speed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 07:25:59 -0000 On Tue, Jan 24, 2012 at 9:27 PM, Eugene M. Zheganin wrote: > Hi. > > I'm suffering from low network performance on one of my FreeBSDs. > I have an i386 8.2-RELEASE machine with an fxp(4) adapter. It's connected > though a bunch of catalysts 2950 to another 8.2. While other machines in > this server room using the same sequence of switches and the same target > source server (which, btw, is equipped with an em(4) and a gigabit link bia > catalyst 3750) show sufficient speed, this particular machine while using > scp starts with a speed of 200 Kbytes/sec and while copying the file shows > speed about 600-800 Kbytes/sec. > > I've added this tweak to the sysctl: > > net.local.stream.recvspace=196605 > net.local.stream.sendspace=196605 > net.inet.tcp.sendspace=196605 > net.inet.tcp.recvspace=196605 > net.inet.udp.recvspace=196605 > kern.ipc.maxsockbuf=2621440 > kern.ipc.somaxconn=4096 > net.inet.tcp.sendbuf_max=524288 > net.inet.tcp.recvbuf_max=524288 > > With these settings the copying starts at 9.5 Mbytes/sec speed, but then, as > file is copying, drops down to 3.5 Megs/sec in about two-three minutes. > > Is there some way to maintain 9.5 Mbytes/sec (I like this speed more) ? > > > Thanks. > Eugene. > > P.S. This machine also runs zfs, I don't know if it's important but I > decided to mention it. 9.5 MBytes? That's 76 Mbps which is reasonable. 28 Mbps is not, but it's too good to make be think it's a duplex mis-match, but it's probably worth checking. Look at the output of 'sysctl dev.fxp.0.stats'. See if you are getting framing and CRC errors. If this does not point at something, a packet capture of header and tcptrace may show the cause of the problem, but the output is not easy to understand. tcptrace is in ports. You could also look at the capture with wireshark. It won't tell as much, but will flag errors and "unusual" activity. Both tools are in ports. -- R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 08:57:36 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF67E1065673; Wed, 25 Jan 2012 08:57:36 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 76AF08FC1B; Wed, 25 Jan 2012 08:57:36 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 35843A0D9; Wed, 25 Jan 2012 08:57:33 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: Date: Wed, 25 Jan 2012 09:57:32 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1084) Cc: freebsd-net@freebsd.org, Aleksandr Rybalko , freebsd-arch@freebsd.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 08:57:36 -0000 Am 25.01.2012 um 08:12 schrieb Adrian Chadd: > So when will you two have something consensus-y to commit? :-) >=20 > What I'm hoping for is: >=20 > * some traction on the MII bus / MDIO bus split and tidyup from stb, = which is nice; > * ray's switch API for speaking to userland with; > * agreeing on whether the correct place to put the driver(s) is where = stb, ray, or a mix of both approaches says so. >=20 > I've been (mostly) trying to stay out of this to see where both of you = have gone. I think we've made some good progress; now it's time to = solidify a design for the first pass of what we want in -HEAD and figure = out how to move forward. My suggestion is to take my bus attachment code (incl. mdio and = miiproxy) and ray's ioctl and userland code. Aleksandr's approach for the driver attachment is to have a generic = switch "bus" driver that abstracts the mii, i2c, memory mapped I/O, etc. = busses the devices are physically attached to, and present a uniform = register file to the chip-specific switch driver. I believe that this = is unnecessarily complicated for two reasons: newbus already provides = that abstraction, and chip-specific drivers usually differ in so many = aspects, including their register files, that code sharing between chips = will be somewhat limited anyway. One aspect that I would enjoy looking into in more detail is how = register accesses on, for example, MDIO, can be provided through the bus = space API. =46rom my cursory reading, it seems that the code currently = is tailored towards register accesses that can be implemented through = CPU native instructions, instead of indirectly through a controller. Aleksandr has defined a quite comprehensive ethernet switch control API = that the framework provides towards in-kernel clients as well as = userland. I think it would be really helpful if we could concentrate on = those API functions that can be controlled through the userland utility, = have immediate use cases (for example, VLAN configuration on the = TL-WR1043ND to separate the WAN from the LAN ports), and we have test = hardware for. In short, don't commit dead code. Having a description of the generic switch model that the API assumes = and driver-specific documentation also wouldn't hurt. (Yes, I'm = volunteering.) Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 12:24:53 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D45FB106566C for ; Wed, 25 Jan 2012 12:24:53 +0000 (UTC) (envelope-from citrin@citrin.ru) Received: from mail-chaos.rambler.ru (mail-chaos.rambler.ru [81.19.68.130]) by mx1.freebsd.org (Postfix) with ESMTP id 4CCB48FC18 for ; Wed, 25 Jan 2012 12:24:53 +0000 (UTC) Received: from citrin.office.vega.ru (office-nat.spylog.net [193.169.234.6]) (Authenticated sender: citrin@citrin.ru) by mail-chaos.rambler.ru (Postfix) with ESMTPSA id 23A771702C for ; Wed, 25 Jan 2012 15:14:07 +0300 (MSK) Message-ID: <4F1FF20E.7080108@citrin.ru> Date: Wed, 25 Jan 2012 16:14:06 +0400 From: Anton Yuzhaninov User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:6.0.2) Gecko/20110922 Thunderbird/6.0.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: livelock with full loaded em(4) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 12:24:54 -0000 Hello. I have test boxes with em(4) network card - Intel 82563EB FreeBSD version - 8.2 stable from 2012-01-15, amd64 When this NIC is full loaded livelock occurs - system is unresponsive even from local console. To generate load I use netsend from /usr/src/tools/tools/netrate/ but other traffic source (e. g. TCP instead UDP) cause same problem. There is need 2 conditions for this livelock: 1. With full NIC load, kernel thread "em1 taskq" hogs CPU. top -zISHP for interface load a bit less, than full. Traffic is generated by # netsend 172.16.0.2 9001 8500 14300 3600 where 14300 is packets per second: 112 processes: 10 running, 82 sleeping, 20 waiting CPU 0: 0.0% user, 0.0% nice, 27.1% system, 0.0% interrupt, 72.9% idle CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 2: 2.3% user, 0.0% nice, 97.7% system, 0.0% interrupt, 0.0% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 6: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 26M Active, 378M Inact, 450M Wired, 132K Cache, 63M Buf, 15G Free Swap: 8192M Total, 8192M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 7737 ayuzhaninov 119 0 5832K 1116K CPU2 2 0:04 100.00% netsend 0 root -68 0 0K 144K - 0 2:17 22.27% {em1 taskq} top -zISHP for full interface load (some drops occurs), load is generated by # netsend 172.16.0.2 9001 8500 14400 3600 112 processes: 11 running, 81 sleeping, 20 waiting CPU 0: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle CPU 1: 4.1% user, 0.0% nice, 95.9% system, 0.0% interrupt, 0.0% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 6: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 26M Active, 378M Inact, 450M Wired, 132K Cache, 63M Buf, 15G Free Swap: 8192M Total, 8192M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 0 root -68 0 0K 144K CPU0 0 2:17 100.00% {em1 taskq} 7759 ayuzhaninov 119 0 5832K 1116K CPU1 1 0:01 100.00% netsend So pps increased from 14300 to 14400 (0.7%), but CPU load from "em1 taskq" thread increased from 27.1% to 100.00% This at least strange, but system still works fine until I run sysctl dev.cpu.0.temperature 2. sysctl handler code for coretemp must be executed on target CPU, e. g. for dev.cpu.0.temperature code executed on CPU0. If CPU0 is fully loaded by "em1 taskq" sysctl handler for dev.cpu.0.temperature acquires Giant mutex lock then tries to run code on CPU0, but it can't - CPU0 is busy. If Giant mutex hold for long time system is unresponsive. In my case Giant mutex acquired when sysctl dev.cpu.0.temperature started and hold all time while netsend is running. This seems to be a scheduler problem: 1. Why "em1 taskq" runs only on CPU0 (there is no affinity for this tread)? # procstat -k 0 | egrep '(PID|em1)' PID TID COMM TDNAME KSTACK 0 100038 kernel em1 taskq # cpuset -g -t 100038 tid 100038 mask: 0, 1, 2, 3, 4, 5, 6, 7 2. Why "em1 taskq" is not preempted to execute sysctl handler code? This is not short term condition - is netsend running for a hour, "em1 taskq" is not preempted for a hour - sysctl all this time in running state but don't have a chance to be executed. -- Anton Yuzhaninov P. S. I tried to use EM_MULTIQUEUE, but this is don't help in my case. From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 17:31:49 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65410106566C for ; Wed, 25 Jan 2012 17:31:49 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 243678FC0A for ; Wed, 25 Jan 2012 17:31:48 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rq6hH-0005nG-D0 for freebsd-net@freebsd.org; Wed, 25 Jan 2012 18:31:47 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 Jan 2012 18:31:47 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 Jan 2012 18:31:47 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Ivan Voras Date: Wed, 25 Jan 2012 18:31:29 +0100 Lines: 10 Message-ID: References: <4F1F92DE.9060200@zhegan.in> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120110 Thunderbird/9.0 In-Reply-To: <4F1F92DE.9060200@zhegan.in> Subject: Re: low network speed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 17:31:49 -0000 On 25/01/2012 06:27, Eugene M. Zheganin wrote: > Hi. > > I'm suffering from low network performance on one of my FreeBSDs. > I have an i386 8.2-RELEASE machine with an fxp(4) adapter. It's > connected though a bunch of catalysts 2950 to another 8.2. Another thing to try would be to upgrade both ends to 8-STABLE and try the high-performance network buffer sizing in ssh (enabled by default). From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 19:58:24 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2779106564A; Wed, 25 Jan 2012 19:58:24 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B8B088FC15; Wed, 25 Jan 2012 19:58:24 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0PJwOkA047269; Wed, 25 Jan 2012 19:58:24 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0PJwONt047265; Wed, 25 Jan 2012 19:58:24 GMT (envelope-from linimon) Date: Wed, 25 Jan 2012 19:58:24 GMT Message-Id: <201201251958.q0PJwONt047265@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164490: [pfil] Incorrect IP checksum on pfil pass from ip_output() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 19:58:25 -0000 Old Synopsis: Incorrect IP checksum on pfil pass from ip_output() New Synopsis: [pfil] Incorrect IP checksum on pfil pass from ip_output() Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Jan 25 19:58:14 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=164490 From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 20:36:44 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5085A1065673 for ; Wed, 25 Jan 2012 20:36:44 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 80CEC8FC18 for ; Wed, 25 Jan 2012 20:36:42 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0PKGIDF017723; Wed, 25 Jan 2012 21:16:18 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0PKGICs017722; Wed, 25 Jan 2012 21:16:18 +0100 (CET) (envelope-from marius) Date: Wed, 25 Jan 2012 21:16:18 +0100 From: Marius Strobl To: Stefan Bethke Message-ID: <20120125201618.GO44286@alchemy.franken.de> References: <600A8C6C-DAB4-4E22-A034-38224017166B@lassitu.de> <20111213025041.GF3705@michelle.cdnetworks.com> <45B0B859-207C-4F02-A28F-7E34B775A273@lassitu.de> <20111213185348.GA7546@michelle.cdnetworks.com> <20111214011657.GH36635@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: YongHyeon PYUN , FreeBSD Net Subject: Re: "ifconfig media off"? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 20:36:44 -0000 On Sat, Jan 21, 2012 at 12:58:08AM +0100, Stefan Bethke wrote: > Am 14.12.2011 um 02:16 schrieb Marius Strobl: > > > On Tue, Dec 13, 2011 at 10:53:48AM -0800, YongHyeon PYUN wrote: > >> On Tue, Dec 13, 2011 at 11:04:51AM +0100, Stefan Bethke wrote: > >>> Am 13.12.2011 um 03:50 schrieb YongHyeon PYUN: > >>> > >>>> On Tue, Dec 13, 2011 at 12:56:22AM +0100, Stefan Bethke wrote: > >>>>> I'm currently writing a driver to configure an ethernet switch chip (see TL-WR1043ND on -embedded). > >>>>> > >>>>> I noticed that there doesn't seem to be a way to power down a phy right now through the ifconfig media command. > >>>>> > >>>>> Would there be objections to extend the media subtype definitions to include an "off", "poweroff" or "down" media subtype, and add code to the relevant phy drivers to power down the phy for this media subtype? > >>>>> > >>>>> The difference between media subtype "none" and this new one would be that there will be no link, even if there is a physical connection. With media subtype "none", a 10 MBit/s half-duplex connection is established, potentially confusing the remote end about the availability of this link. On the local side, the link is down, so no packets are exchanged. > >>>>> > >>>> > >>>> I think "none" means "isolated" so should have no established link > >>>> and probably you can also power down the PHY. > >>>> I vaguely guess the PHY of switch chip does not correctly support > >>>> isolated mode so you may have wanted to power down. > >>> > >>> > >>> After looking at the code a bit more, I think the common code just doesn't set the BMCR_PDOWN (but clears it when bringing up the PHY). > >>> > >> > >> Yes, and most PHYs could be powered down when BMCR_ISO is chosen. > >> I'm not sure whether this could be applied to hardwares that > >> support multiple PHYs(i.e. internal and external transceivers) > >> though. Marius may have some opinions on this(CCed). > >> However powering down PHY with BMCR_ISO looks natural to me. > >> > >>> Index: sys/dev/mii/mii_physubr.c > >>> =================================================================== > >>> --- sys/dev/mii/mii_physubr.c (revision 228402) > >>> +++ sys/dev/mii/mii_physubr.c (working copy) > >>> @@ -58,7 +58,7 @@ > >>> */ > >>> static const struct mii_media mii_media_table[MII_NMEDIA] = { > >>> /* None */ > >>> - { BMCR_ISO, ANAR_CSMA, > >>> + { BMCR_ISO | BMCR_PDOWN, ANAR_CSMA, > >>> 0, }, > >>> > >>> /* 10baseT */ > >>> > >>> I've opened kern/163240. > >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=163240 > > I'd like to revisit this. Just to reiterate my motivation for the change: I want to be able to indicate to the remote end that my station is not active. With the PHY just isolated from the MII, the link stays up and functional (and even autoneg continues to work), so the remote has no indication that it's just shouting into a void. Yes, I understand the motivation and generally agree that this should be implemented. IMO the above is just a quick-hack though and no proper solution, on the other hand I neither see a need to grown an "off" media for this. > > > I don't think powering down the PHY along with IFM_NONE especially > > in that way is a good idea for several reasons: > > - It's incomplete as not all PHY drivers use mii_phy_add_media()/ > > mii_phy_setmedia(). > > - Even for those that do IFM_NONE isn't added when the PHY driver > > sets MIIF_NOISOLATE (for some PHYs BMCR_ISO either just doesn't > > work as especially the built-in ones probably have been designed > > with only single-PHY configurations in mind or even wedges the > > chip up to the point that even a reset doesn't get it working > > again). In general though, BMCR_ISO and BMCR_PDOWN are orthogonal > > (even in IEEE 802.3-2008 as far as I can see), i.e. while BMCR_ISO > > might be broken, BMCR_PDOWN could work (actually I'd expect > > BMCR_PDOWN to be less fragile than BMCR_ISO). > > I didn't expect my suggestion to be the be-all end-all, only a quick and easy way to allow compliant PHYs to be powered down, and I'm not sure why a "complete" solution is required. I'd assume that PHYs setting MIIF_NOISOLATE have specific requirements, so it's OK to not have the power-down option available there. (Plus I don't have hardware I could test that case on). I wouldn't call it "specific requirements". The PHY drivers I've flagged with MIIF_NOISOLATE so far fall into one of two categories: a) Setting BMCR_ISO just doesn't have any effect and the PHY happily continues to pass traffic. Setting MIIF_NOISOLATE in this case is done in order to not add an non-working "none" media. b) Upon setting BMCR_ISO the hardware wedges up to a way that a power-cycle is required in order to get it into a working state again. MIIF_NOISOLATE is set here in order to protect the users from such an experience when configuring "none" (think production machines). Some PHYs affected by these quirks are integrated ones with no external MII bus on the MAC so it's actually sort of okay that you can't isolate the integrated PHY but unfortunately some are also stand-alone PHYs. Based on that I expect the same issues with power down as seen with isolation with the two quirk sets not necessarily being correlated. Actually, I know for a fact that at least some Broadcom PHYs are affected by power down issues. Also, see the following if if_age.c: /* * No PME capability, PHY power down. * XXX * Due to an unknown reason powering down PHY resulted * in unexpected results such as inaccessbility of * hardware of freshly rebooted system. Disable * powering down PHY until I got more information for * Attansic/Atheros PHY hardwares. */ #ifdef notyet age_miibus_writereg(sc->age_dev, sc->age_phyaddr, MII_BMCR, BMCR_PDOWN); #endif That's why I don't want to make BMCR_PDOWN available as soon as MIIF_NOISOLATE is not set and generally don't want to combine BMCR_PDOWN with BMCR_ISO. > > > - There should be a way to let a PHY driver do something different > > than setting BMCR_PDOWN when it's told to power down as f.e. at > > least some Broadcom PHYs support a "super isolation" mode. > > Cool, but how does my suggestion stop anyone from implementing that? Why not solve it properly in the first place? I agree that a possibility to power down PHYs should be implemented but given that we've lived for so long without it I don't see a need to get a quick-hack in as fast as possible. > > > - As you already mentioned in general there can be multiple PHYs > > on one MII bus and I'd expect "power down that interface" to > > mean to power all of them down (granted, MACs with multiple > > PHYs are rather uncommon in reality but they do exist, f.e. > > Allied Telesis has several card models that use two PHYs per > > MAC). > > Since there can be only one active PHY at a time, it could make sense to also set PDOWN in mii_mediachg(). How about a new flag MIIF_PDOWN that the phy driver can set to indicate that it would rather be shut off than just isolated? Right now I only care about ukphy anyway. Yes, something like a MIIF_PDOWN (or probably better a MIIF_NOPDWON to be in line with MIIF_NOISOLATE) is needed in order to deal with broken PHYs. On the other hand IEEE 802.3-2008 says that non-active PHYs should just be isolated, not powered down or both. Given that multi-PHY configurations are actually very rare in practice I don't see a good reason to violate the standard and power down non-active PHYs in mii_mediachg(). For the embedded world with multiple (fake) stuff hanging around on the MDIO bus you need to already prevent mii_mediachg() from isolating all but one "PHY" anyway. > > > In generall I agree that there should be a way to power down PHYs > > though. While there might be some merit in additionally adding that > > as a media option I think that the most intuitive way would be for > > `ifconfig foo0 down` to also power down the PHYs on that interface. > > In order implement that the basic way seems to be to make the > > various foo_stop() methods of MAC drivers to call mii_down() (as > > in NetBSD) and to make mii_phy_down() (or a replacement) actually > > do something, with PHY drivers being allowed to opt-out with > > something like a MIIF_NOPOWERDOWN. What I'm not sure about is how > > to implement power down in the individual drivers while keeping > > code duplication for the common case low, i.e. I'm not sure whether > > the NetBSD way of having a MII_DOWN case in the service routines > > is the way to go. > > I'm hesitant to tie IFF_UP to link state. Is there precedent for that? My gut feeling is that IFF_UP has always meant that the interface has been brought up administratively; bringing an interface down "only" means that the system should not transmit on it and discard received frames. > Without investigating all the Ethernet drivers that don't use miibus(4) I'd say yes, in FreeBSD administratively down currently means what you describe. However, the question is whether that behavior makes any sense at all. So far I couldn't think of a single usecase where the current behavior of keeping up the link but ignoring all the traffic when an interface is set to down would be needed (and if there actually is one the same effect likely could be achieved by just up'ing the interface but not setting an address. On the other hand, tying power down and therefore the link state (if the PHY allows to ...) as the advantage of: - no need to grow a hack or another media to power down a PHY, - save power in the default case, - being in line what administratively up/down means on network gear, - doing the right thing when you want to bring a link of an LACP channel up/down, - not reporting false media when the interface is down (this is PHY-dependent though). I'm interested to hear about usecases where the current behavior of keeping up the link when the interface is down actually is required though. Marius From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 20:56:23 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A42D41065670 for ; Wed, 25 Jan 2012 20:56:23 +0000 (UTC) (envelope-from w8hdkim@gmail.com) Received: from mail-qw0-f47.google.com (mail-qw0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 3D21A8FC13 for ; Wed, 25 Jan 2012 20:56:22 +0000 (UTC) Received: by qadz30 with SMTP id z30so1286686qad.13 for ; Wed, 25 Jan 2012 12:56:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=Wl+mPu7TfsrKbRGgEoVjbQ9bN8maf3dhXJdIKJQeohA=; b=QNRNxQNwm9EKPMK3psZw3EmqHdFF1t5wtuw5YRaUSVVZ6Dg0D2bAfjFq+pjSobDfYn 2pSYLV5hFZy69xl8FsJCh0xE2VR426CaRcvkmDdxkGD8Po0q36FxL+uObeXpfuI656X/ DI3sudtRFJdr8vim3I2p37k+VTUeirPadk2mA= MIME-Version: 1.0 Received: by 10.224.180.67 with SMTP id bt3mr1805139qab.6.1327523170093; Wed, 25 Jan 2012 12:26:10 -0800 (PST) Received: by 10.229.223.135 with HTTP; Wed, 25 Jan 2012 12:26:09 -0800 (PST) Date: Wed, 25 Jan 2012 15:26:09 -0500 Message-ID: From: Kim Culhan To: freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary=20cf303b3a2dfc0a6904b76012b9 Subject: msk0: watchdog timeout interface hang X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 20:56:23 -0000 --20cf303b3a2dfc0a6904b76012b9 Content-Type: text/plain; charset=ISO-8859-1 Running 10-curent from 01-20-12 the msk0 interface hung, on the console: msk0: watchdog timeout msk0: prefetch unit stuck? msk0: initialization failed: no memory for Rx buffers Verbose boot dmesg output attached. Any help is greatly appreciated. -kim --20cf303b3a2dfc0a6904b76012b9 Content-Type: application/octet-stream; name="acer5570_2997_msk0_dmesg.out" Content-Disposition: attachment; filename="acer5570_2997_msk0_dmesg.out" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gxusylzl0 TVAgQ29uZmlndXJhdGlvbiBUYWJsZSB2ZXJzaW9uIDEuNCBmb3VuZCBhdCAweGMwMDllNTcxClRh YmxlICdGQUNQJyBhdCAweDdmNjlhYzJhClRhYmxlICdBUElDJyBhdCAweDdmNjlhZDFlCkFQSUM6 IEZvdW5kIHRhYmxlIGF0IDB4N2Y2OWFkMWUKQVBJQzogVXNpbmcgdGhlIE1BRFQgZW51bWVyYXRv ci4KTUFEVDogRm91bmQgQ1BVIEFQSUMgSUQgMCBBQ1BJIElEIDA6IGVuYWJsZWQKU01QOiBBZGRl ZCBDUFUgMCAoQVApCk1BRFQ6IEZvdW5kIENQVSBBUElDIElEIDEgQUNQSSBJRCAxOiBlbmFibGVk ClNNUDogQWRkZWQgQ1BVIDEgKEFQKQpDb3B5cmlnaHQgKGMpIDE5OTItMjAxMiBUaGUgRnJlZUJT RCBQcm9qZWN0LgpDb3B5cmlnaHQgKGMpIDE5NzksIDE5ODAsIDE5ODMsIDE5ODYsIDE5ODgsIDE5 ODksIDE5OTEsIDE5OTIsIDE5OTMsIDE5OTQKCVRoZSBSZWdlbnRzIG9mIHRoZSBVbml2ZXJzaXR5 IG9mIENhbGlmb3JuaWEuIEFsbCByaWdodHMgcmVzZXJ2ZWQuCkZyZWVCU0QgaXMgYSByZWdpc3Rl cmVkIHRyYWRlbWFyayBvZiBUaGUgRnJlZUJTRCBGb3VuZGF0aW9uLgpGcmVlQlNEIDEwLjAtQ1VS UkVOVCAjMDogV2VkIEphbiAyNSAxNDowMToxMyBVVEMgMjAxMgogICAgcm9vdEBidWlsZC1pMzg2 LWZic2QtMi5hbGxic2Qub3JnOi91c3Ivb2JqL2kzODYuaTM4Ni91c3Ivc3JjL3N5cy9HRU5FUklD IGkzODYKV0FSTklORzogV0lUTkVTUyBvcHRpb24gZW5hYmxlZCwgZXhwZWN0IHJlZHVjZWQgcGVy Zm9ybWFuY2UuClByZWxvYWRlZCBlbGYga2VybmVsICIvYm9vdC9rZXJuZWwva2VybmVsIiBhdCAw eGMxNDdlMDAwLgpDYWxpYnJhdGluZyBUU0MgY2xvY2sgLi4uIFRTQyBjbG9jazogMTczMzQzOTY5 NSBIegpDUFU6IEdlbnVpbmUgSW50ZWwoUikgQ1BVICAgICAgICAgICBUMjA4MCAgQCAxLjczR0h6 ICgxNzMzLjQ0LU1IeiA2ODYtY2xhc3MgQ1BVKQogIE9yaWdpbiA9ICJHZW51aW5lSW50ZWwiICBJ ZCA9IDB4NmVjICBGYW1pbHkgPSA2ICBNb2RlbCA9IGUgIFN0ZXBwaW5nID0gMTIKICBGZWF0dXJl cz0weGJmZTlmYmZmPEZQVSxWTUUsREUsUFNFLFRTQyxNU1IsUEFFLE1DRSxDWDgsQVBJQyxTRVAs TVRSUixQR0UsTUNBLENNT1YsUEFULENMRkxVU0gsRFRTLEFDUEksTU1YLEZYU1IsU1NFLFNTRTIs U1MsSFRULFRNLFBCRT4KICBGZWF0dXJlczI9MHhjMTg5PFNTRTMsTU9OLEVTVCxUTTIseFRQUixQ RENNPgogIEFNRCBGZWF0dXJlcz0weDEwMDAwMDxOWD4KICBUU0M6IFAtc3RhdGUgaW52YXJpYW50 LCBwZXJmb3JtYW5jZSBzdGF0aXN0aWNzCgpJbnN0cnVjdGlvbiBUTEI6IDQgS0IgUGFnZXMsIDQt d2F5IHNldCBhc3NvY2lhdGl2ZSwgMTI4IGVudHJpZXMKRGF0YSBUTEI6IDQgS0IgUGFnZXMsIDQt d2F5IHNldCBhc3NvY2lhdGl2ZSwgMTI4IGVudHJpZXMKSW5zdHJ1Y3Rpb24gVExCOiA0IE1CIHBh Z2VzLCBmdWxseSBhc3NvY2lhdGl2ZSwgMiBlbnRyaWVzCjJuZC1sZXZlbCBjYWNoZTogMSBNQiwg NC13YXkgc2V0IGFzc29jaWF0aXZlLCA2NC1ieXRlIGxpbmUgc2l6ZQoxc3QtbGV2ZWwgaW5zdHJ1 Y3Rpb24gY2FjaGU6IDMyIEtCLCA4LXdheSBzZXQgYXNzb2NpYXRpdmUsIDY0IGJ5dGUgbGluZSBz aXplCkRhdGEgVExCOiA0IE1CIFBhZ2VzLCA0LXdheSBzZXQgYXNzb2NpYXRpdmUsIDggZW50cmll cwoxc3QtbGV2ZWwgZGF0YSBjYWNoZTogMzIgS0IsIDgtd2F5IHNldCBhc3NvY2lhdGl2ZSwgNjQg Ynl0ZSBsaW5lIHNpemUKTDIgY2FjaGU6IDEwMjQga2J5dGVzLCA0LXdheSBhc3NvY2lhdGl2ZSwg NjQgYnl0ZXMvbGluZQpyZWFsIG1lbW9yeSAgPSAyMTQ3NDgzNjQ4ICgyMDQ4IE1CKQpQaHlzaWNh bCBtZW1vcnkgY2h1bmsocyk6CjB4MDAwMDAwMDAwMDAwMTAwMCAtIDB4MDAwMDAwMDAwMDA5Y2Zm ZiwgNjM4OTc2IGJ5dGVzICgxNTYgcGFnZXMpCjB4MDAwMDAwMDAwMDEwMDAwMCAtIDB4MDAwMDAw MDAwMDNmZmZmZiwgMzE0NTcyOCBieXRlcyAoNzY4IHBhZ2VzKQoweDAwMDAwMDAwMDE4MjYwMDAg LSAweDAwMDAwMDAwN2QyNGNmZmYsIDIwNzQyNDMwNzIgYnl0ZXMgKDUwNjQwNyBwYWdlcykKYXZh aWwgbWVtb3J5ID0gMjA3MjMyNjE0NCAoMTk3NiBNQikKRXZlbnQgdGltZXIgIkxBUElDIiBxdWFs aXR5IDQwMApBQ1BJIEFQSUMgVGFibGU6IDxJTlRFTCAgQ0FMSVNUR0E+CklOVFI6IEFkZGluZyBs b2NhbCBBUElDIDEgYXMgYSB0YXJnZXQKRnJlZUJTRC9TTVA6IE11bHRpcHJvY2Vzc29yIFN5c3Rl bSBEZXRlY3RlZDogMiBDUFVzCkZyZWVCU0QvU01QOiAxIHBhY2thZ2UocykgeCAyIGNvcmUocykK IGNwdTAgKEJTUCk6IEFQSUMgSUQ6ICAwCiBjcHUxIChBUCk6IEFQSUMgSUQ6ICAxCmJpb3MzMjog Rm91bmQgQklPUzMyIFNlcnZpY2UgRGlyZWN0b3J5IGhlYWRlciBhdCAweGMwMGY2NjAwCmJpb3Mz MjogRW50cnkgPSAweGZkNDYwIChjMDBmZDQ2MCkgIFJldiA9IDAgIExlbiA9IDEKcGNpYmlvczog UENJIEJJT1MgZW50cnkgYXQgMHhmZDQ2MCsweDI2MgpwbnBiaW9zOiBGb3VuZCBQblAgQklPUyBk YXRhIGF0IDB4YzAwZjY2YTAKcG5wYmlvczogRW50cnkgPSBmMDAwMDphY2NkICBSZXYgPSAxLjAK T3RoZXIgQklPUyBzaWduYXR1cmVzIGZvdW5kOgpBUElDOiBDUFUgMCBoYXMgQUNQSSBJRCAwCkFQ SUM6IENQVSAxIGhhcyBBQ1BJIElEIDEKVUxFOiBzZXR1cCBjcHUgMApVTEU6IHNldHVwIGNwdSAx CkFDUEk6IFJTRFAgMHhmNjY1MCAwMDAyNCAodjAyIFBUTFREICkKQUNQSTogWFNEVCAweDdmNjkx MWYzIDAwMDY0ICh2MDEgQUNSU1lTIEFDUlBSRENUIDA2MDQwMDAwICBMVFAgMDAwMDAwMDApCkFD UEk6IEZBQ1AgMHg3ZjY5YWMyYSAwMDBGNCAodjAzIElOVEVMICBDQUxJU1RHQSAwNjA0MDAwMCBB TEFOIDAwMDAwMDAxKQpBQ1BJOiBEU0RUIDB4N2Y2OTFmNzggMDhDM0UgKHYwMiBJTlRFTCAgQ0FM SVNUR0EgMDYwNDAwMDAgTVNGVCAwMzAwMDAwMCkKQUNQSTogRkFDUyAweDdmNjliZmMwIDAwMDQw CkFDUEk6IEFQSUMgMHg3ZjY5YWQxZSAwMDA2OCAodjAxIElOVEVMICBDQUxJU1RHQSAwNjA0MDAw MCBMT0hSIDAwMDAwMDVBKQpBQ1BJOiBIUEVUIDB4N2Y2OWFkODYgMDAwMzggKHYwMSBJTlRFTCAg Q0FMSVNUR0EgMDYwNDAwMDAgTE9IUiAwMDAwMDA1QSkKQUNQSTogTUNGRyAweDdmNjlhZGJlIDAw MDNDICh2MDEgSU5URUwgIENBTElTVEdBIDA2MDQwMDAwIExPSFIgMDAwMDAwNUEpCkFDUEk6IEFQ SUMgMHg3ZjY5YWRmYSAwMDA2OCAodjAxIFBUTFREICA/IEFQSUMgICAwNjA0MDAwMCAgTFRQIDAw MDAwMDAwKQpBQ1BJOiBCT09UIDB4N2Y2OWFlNjIgMDAwMjggKHYwMSBQVExURCAgJFNCRlRCTCQg MDYwNDAwMDAgIExUUCAwMDAwMDAwMSkKQUNQSTogU0xJQyAweDdmNjlhZThhIDAwMTc2ICh2MDEg QUNSU1lTIEFDUlBSRENUIDA2MDQwMDAwIGFjZXIgMDAwMDAwMDApCkFDUEk6IFNTRFQgMHg3ZjY5 MTI1NyAwMDRFNiAodjAxICBQbVJlZiAgICBDcHVQbSAwMDAwMzAwMCBJTlRMIDIwMDUwNjI0KQpN QURUOiBGb3VuZCBJTyBBUElDIElEIDEsIEludGVycnVwdCAwIGF0IDB4ZmVjMDAwMDAKaW9hcGlj MDogQ2hhbmdpbmcgQVBJQyBJRCB0byAxCmlvYXBpYzA6IFJvdXRpbmcgZXh0ZXJuYWwgODI1OUEn cyAtPiBpbnRwaW4gMApNQURUOiBJbnRlcnJ1cHQgb3ZlcnJpZGU6IHNvdXJjZSAwLCBpcnEgMgpp b2FwaWMwOiBSb3V0aW5nIElSUSAwIC0+IGludHBpbiAyCk1BRFQ6IEludGVycnVwdCBvdmVycmlk ZTogc291cmNlIDksIGlycSA5CmlvYXBpYzA6IGludHBpbiA5IHRyaWdnZXI6IGxldmVsCmxhcGlj MDogUm91dGluZyBOTUkgLT4gTElOVDEKbGFwaWMwOiBMSU5UMSB0cmlnZ2VyOiBlZGdlCmxhcGlj MDogTElOVDEgcG9sYXJpdHk6IGhpZ2gKbGFwaWMxOiBSb3V0aW5nIE5NSSAtPiBMSU5UMQpsYXBp YzE6IExJTlQxIHRyaWdnZXI6IGVkZ2UKbGFwaWMxOiBMSU5UMSBwb2xhcml0eTogaGlnaAppb2Fw aWMwIDxWZXJzaW9uIDIuMD4gaXJxcyAwLTIzIG9uIG1vdGhlcmJvYXJkCmNwdTAgQlNQOgogICAg IElEOiAweDAwMDAwMDAwICAgVkVSOiAweDAwMDUwMDE0IExEUjogMHgwMDAwMDAwMCBERlI6IDB4 ZmZmZmZmZmYKICBsaW50MDogMHgwMDAxMDcwMCBsaW50MTogMHgwMDAwMDQwMCBUUFI6IDB4MDAw MDAwMDAgU1ZSOiAweDAwMDAwMWZmCiAgdGltZXI6IDB4MDAwMTAwZWYgdGhlcm06IDB4MDAwMTAw MDAgZXJyOiAweDAwMDAwMGYwIHBtYzogMHgwMDAxMDQwMApzbmRfdW5pdF9pbml0KCkgdT0weDAw ZmY4MDAwIFs1MTJdIGQ9MHgwMDAwN2MwMCBbMzJdIGM9MHgwMDAwMDNmZiBbMTAyNF0KZmVlZGVy X3JlZ2lzdGVyOiBzbmRfdW5pdD0tMSBzbmRfbWF4YXV0b3ZjaGFucz0xNiBsYXRlbmN5PTUgZmVl ZGVyX3JhdGVfbWluPTEgZmVlZGVyX3JhdGVfbWF4PTIwMTYwMDAgZmVlZGVyX3JhdGVfcm91bmQ9 MjUKd2xhbjogPDgwMi4xMSBMaW5rIExheWVyPgprYmQ6IG5ldyBhcnJheSBzaXplIDQKa2JkMSBh dCBrYmRtdXgwCm1lbTogPG1lbW9yeT4KUGVudGl1bSBQcm8gTVRSUiBzdXBwb3J0IGVuYWJsZWQK bmZzbG9jazogcHNldWRvLWRldmljZQpudWxsOiA8bnVsbCBkZXZpY2UsIHplcm8gZGV2aWNlPgpp bzogPEkvTz4KcmFuZG9tOiA8ZW50cm9weSBzb3VyY2UsIFNvZnR3YXJlLCBZYXJyb3c+CmhwdHJy OiBSb2NrZXRSQUlEIDE3eHgvMnh4eCBTQVRBIGNvbnRyb2xsZXIgZHJpdmVyIHYxLjIKY3RsOiBD QU0gVGFyZ2V0IExheWVyIGxvYWRlZAphY3BpMDogPEFDUlNZUyBBQ1JQUkRDVD4gb24gbW90aGVy Ym9hcmQKUENJZTogTWVtb3J5IE1hcHBlZCBjb25maWd1cmF0aW9uIGJhc2UgQCAweGUwMDAwMDAw CnBjaWJpb3M6IEJJT1MgdmVyc2lvbiAyLjEwCmlvYXBpYzA6IHJvdXRpbmcgaW50cGluIDkgKElT QSBJUlEgOSkgdG8gbGFwaWMgMCB2ZWN0b3IgNDgKYWNwaTA6IFBvd2VyIEJ1dHRvbiAoZml4ZWQp CmFjcGkwOiB3YWtldXAgY29kZSB2YSAweGM1YTc4MDAwIHBhIDB4MTAwMAp1bmtub3duOiBJL08g cmFuZ2Ugbm90IHN1cHBvcnRlZApocGV0MDogPEhpZ2ggUHJlY2lzaW9uIEV2ZW50IFRpbWVyPiBp b21lbSAweGZlZDAwMDAwLTB4ZmVkMDAzZmYgaXJxIDAsOCBvbiBhY3BpMApocGV0MDogdmVuZG9y IDB4ODA4NiwgcmV2IDB4MSwgMTQzMTgxODBIeiA2NGJpdCwgMyB0aW1lcnMsIGxlZ2FjeSByb3V0 ZQpocGV0MDogIHQwOiBpcnFzIDB4MDBmMDAwMDAgKDApLCA2NGJpdCwgcGVyaW9kaWMKaHBldDA6 ICB0MTogaXJxcyAweDAwZjAwMDAwICgwKQpocGV0MDogIHQyOiBpcnFzIDB4MDBmMDA4MDAgKDAp ClRpbWVjb3VudGVyICJIUEVUIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSA5NTAKaW9h cGljMDogcm91dGluZyBpbnRwaW4gMjAgKFBDSSBJUlEgMjApIHRvIGxhcGljIDAgdmVjdG9yIDQ5 CkV2ZW50IHRpbWVyICJIUEVUIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSA0NTAKRXZl bnQgdGltZXIgIkhQRVQxIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSA0NDAKRXZlbnQg dGltZXIgIkhQRVQyIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSA0NDAKQUNQSSB0aW1l cjogMS8xIDEvMSAxLzEgMS8xIDEvMSAxLzEgMS8xIDEvMSAxLzEgMS8xIC0+IDEwClRpbWVjb3Vu dGVyICJBQ1BJLWZhc3QiIGZyZXF1ZW5jeSAzNTc5NTQ1IEh6IHF1YWxpdHkgOTAwCmFjcGlfdGlt ZXIwOiA8MjQtYml0IHRpbWVyIGF0IDMuNTc5NTQ1TUh6PiBwb3J0IDB4MTAwOC0weDEwMGIgb24g YWNwaTAKY3B1MDogPEFDUEkgQ1BVPiBvbiBhY3BpMApBQ1BJOiBTU0RUIDB4N2Y2OTFjNzggMDAy MzggKHYwMSAgUG1SZWYgIENwdTBJc3QgMDAwMDMwMDAgSU5UTCAyMDA1MDYyNCkKQUNQSTogRHlu YW1pYyBPRU0gVGFibGUgTG9hZDoKQUNQSTogU1NEVCAwIDAwMjM4ICh2MDEgIFBtUmVmICBDcHUw SXN0IDAwMDAzMDAwIElOVEwgMjAwNTA2MjQpCkFDUEk6IFNTRFQgMHg3ZjY5MTczZCAwMDRCNiAo djAxICBQbVJlZiAgQ3B1MENzdCAwMDAwMzAwMSBJTlRMIDIwMDUwNjI0KQpBQ1BJOiBEeW5hbWlj IE9FTSBUYWJsZSBMb2FkOgpBQ1BJOiBTU0RUIDAgMDA0QjYgKHYwMSAgUG1SZWYgIENwdTBDc3Qg MDAwMDMwMDEgSU5UTCAyMDA1MDYyNCkKY3B1MTogPEFDUEkgQ1BVPiBvbiBhY3BpMApBQ1BJOiBT U0RUIDB4N2Y2OTFlYjAgMDAwQzggKHYwMSAgUG1SZWYgIENwdTFJc3QgMDAwMDMwMDAgSU5UTCAy MDA1MDYyNCkKQUNQSTogRHluYW1pYyBPRU0gVGFibGUgTG9hZDoKQUNQSTogU1NEVCAwIDAwMEM4 ICh2MDEgIFBtUmVmICBDcHUxSXN0IDAwMDAzMDAwIElOVEwgMjAwNTA2MjQpCkFDUEk6IFNTRFQg MHg3ZjY5MWJmMyAwMDA4NSAodjAxICBQbVJlZiAgQ3B1MUNzdCAwMDAwMzAwMCBJTlRMIDIwMDUw NjI0KQpBQ1BJOiBEeW5hbWljIE9FTSBUYWJsZSBMb2FkOgpBQ1BJOiBTU0RUIDAgMDAwODUgKHYw MSAgUG1SZWYgIENwdTFDc3QgMDAwMDMwMDAgSU5UTCAyMDA1MDYyNCkKYWNwaV9lYzA6IDxFbWJl ZGRlZCBDb250cm9sbGVyOiBHUEUgMHgxNz4gcG9ydCAweDYyLDB4NjYgb24gYWNwaTAKcGNpX2xp bmswOiAgICAgICAgSW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMKICBJbml0aWFsIFByb2JlICAg ICAgIDAgICAxMSAgIE4gICAgIDAgIDEgMyA0IDUgNiA3IDEwIDEyIDE0IDE1CiAgVmFsaWRhdGlv biAgICAgICAgICAwICAyNTUgICBOICAgICAwICAxIDMgNCA1IDYgNyAxMCAxMiAxNCAxNQogIEFm dGVyIERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcgMTAgMTIgMTQg MTUKcGNpX2xpbmsxOiAgICAgICAgSW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMKICBJbml0aWFs IFByb2JlICAgICAgIDAgICAxMSAgIE4gICAgIDAgIDEgMyA0IDUgNiA3IDExIDEyIDE0IDE1CiAg VmFsaWRhdGlvbiAgICAgICAgICAwICAgMTEgICBOICAgICAwICAxIDMgNCA1IDYgNyAxMSAxMiAx NCAxNQogIEFmdGVyIERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcg MTEgMTIgMTQgMTUKcGNpX2xpbmsyOiAgICAgICAgSW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMK ICBJbml0aWFsIFByb2JlICAgICAgIDAgICAxMCAgIE4gICAgIDAgIDEgMyA0IDUgNiA3IDEwIDEy IDE0IDE1CiAgVmFsaWRhdGlvbiAgICAgICAgICAwICAgMTAgICBOICAgICAwICAxIDMgNCA1IDYg NyAxMCAxMiAxNCAxNQogIEFmdGVyIERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAz IDQgNSA2IDcgMTAgMTIgMTQgMTUKcGNpX2xpbmszOiAgICAgICAgSW5kZXggIElSUSAgUnRkICBS ZWYgIElSUXMKICBJbml0aWFsIFByb2JlICAgICAgIDAgICAxMCAgIE4gICAgIDAgIDEgMyA0IDUg NiA3IDExIDEyIDE0IDE1CiAgVmFsaWRhdGlvbiAgICAgICAgICAwICAyNTUgICBOICAgICAwICAx IDMgNCA1IDYgNyAxMSAxMiAxNCAxNQogIEFmdGVyIERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAg ICAgMCAgMSAzIDQgNSA2IDcgMTEgMTIgMTQgMTUKcGNpX2xpbms0OiAgICAgICAgSW5kZXggIElS USAgUnRkICBSZWYgIElSUXMKICBJbml0aWFsIFByb2JlICAgICAgIDAgICAxMCAgIE4gICAgIDAg IDEgMyA0IDUgNiA3IDEwIDEyIDE0IDE1CiAgVmFsaWRhdGlvbiAgICAgICAgICAwICAgMTAgICBO ICAgICAwICAxIDMgNCA1IDYgNyAxMCAxMiAxNCAxNQogIEFmdGVyIERpc2FibGUgICAgICAgMCAg MjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcgMTAgMTIgMTQgMTUKcGNpX2xpbms1OiAgICAgICAg SW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMKICBJbml0aWFsIFByb2JlICAgICAgIDAgICAxMSAg IE4gICAgIDAgIDEgMyA0IDUgNiA3IDExIDEyIDE0IDE1CiAgVmFsaWRhdGlvbiAgICAgICAgICAw ICAgMTEgICBOICAgICAwICAxIDMgNCA1IDYgNyAxMSAxMiAxNCAxNQogIEFmdGVyIERpc2FibGUg ICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcgMTEgMTIgMTQgMTUKcGNpX2xpbms2 OiAgICAgICAgSW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMKICBJbml0aWFsIFByb2JlICAgICAg IDAgICAxMCAgIE4gICAgIDAgIDEgMyA0IDUgNiA3IDEwIDEyIDE0IDE1CiAgVmFsaWRhdGlvbiAg ICAgICAgICAwICAgMTAgICBOICAgICAwICAxIDMgNCA1IDYgNyAxMCAxMiAxNCAxNQogIEFmdGVy IERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcgMTAgMTIgMTQgMTUK cGNpX2xpbms3OiAgICAgICAgSW5kZXggIElSUSAgUnRkICBSZWYgIElSUXMKICBJbml0aWFsIFBy b2JlICAgICAgIDAgICAxMSAgIE4gICAgIDAgIDEgMyA0IDUgNiA3IDExIDEyIDE0IDE1CiAgVmFs aWRhdGlvbiAgICAgICAgICAwICAgMTEgICBOICAgICAwICAxIDMgNCA1IDYgNyAxMSAxMiAxNCAx NQogIEFmdGVyIERpc2FibGUgICAgICAgMCAgMjU1ICAgTiAgICAgMCAgMSAzIDQgNSA2IDcgMTEg MTIgMTQgMTUKcGNpYjA6IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gcG9ydCAweGNmOC0weGNmZiBv biBhY3BpMApwY2liMDogZGVjb2RpbmcgNCByYW5nZSAwLTB4Y2Y3CnBjaWIwOiBkZWNvZGluZyA0 IHJhbmdlIDB4ZDAwLTB4ZmZmZgpwY2liMDogZGVjb2RpbmcgMyByYW5nZSAweGEwMDAwLTB4YmZm ZmYKcGNpYjA6IGRlY29kaW5nIDMgcmFuZ2UgMHhkMDAwMC0weGQzZmZmCnBjaWIwOiBkZWNvZGlu ZyAzIHJhbmdlIDB4ZDQwMDAtMHhkN2ZmZgpwY2liMDogZGVjb2RpbmcgMyByYW5nZSAweGQ4MDAw LTB4ZGJmZmYKcGNpYjA6IGRlY29kaW5nIDMgcmFuZ2UgMHhlMDAwMC0weGUzZmZmCnBjaWIwOiBk ZWNvZGluZyAzIHJhbmdlIDB4ODAwMDAwMDAtMHhmZWJmZmZmZgpwY2kwOiA8QUNQSSBQQ0kgYnVz PiBvbiBwY2liMApwY2kwOiBkb21haW49MCwgcGh5c2ljYWwgYnVzPTAKZm91bmQtPgl2ZW5kb3I9 MHg4MDg2LCBkZXY9MHgyN2EwLCByZXZpZD0weDAzCglkb21haW49MCwgYnVzPTAsIHNsb3Q9MCwg ZnVuYz0wCgljbGFzcz0wNi0wMC0wMCwgaGRydHlwZT0weDAwLCBtZmRldj0wCgljbWRyZWc9MHgw MDA2LCBzdGF0cmVnPTB4MjA5MCwgY2FjaGVsbnN6PTAgKGR3b3JkcykKCWxhdHRpbWVyPTB4MDAg KDAgbnMpLCBtaW5nbnQ9MHgwMCAoMCBucyksIG1heGxhdD0weDAwICgwIG5zKQpmb3VuZC0+CXZl bmRvcj0weDgwODYsIGRldj0weDI3YTIsIHJldmlkPTB4MDMKCWRvbWFpbj0wLCBidXM9MCwgc2xv dD0yLCBmdW5jPTAKCWNsYXNzPTAzLTAwLTAwLCBoZHJ0eXBlPTB4MDAsIG1mZGV2PTEKCWNtZHJl Zz0weDAwMDcsIHN0YXRyZWc9MHgwMDkwLCBjYWNoZWxuc3o9MCAoZHdvcmRzKQoJbGF0dGltZXI9 MHgwMCAoMCBucyksIG1pbmdudD0weDAwICgwIG5zKSwgbWF4bGF0PTB4MDAgKDAgbnMpCglpbnRw aW49YSwgaXJxPTExCglwb3dlcnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKCU1T SSBzdXBwb3J0cyAxIG1lc3NhZ2UKCW1hcFsxMF06IHR5cGUgTWVtb3J5LCByYW5nZSAzMiwgYmFz ZSAweGQwMzAwMDAwLCBzaXplIDE5LCBlbmFibGVkCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGQwMzAwMDAwLTB4ZDAzN2ZmZmYpIGZvciByaWQgMTAgb2YgcGNpMDowOjI6MAoJbWFwWzE0XTog dHlwZSBJL08gUG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHgxODAwLCBzaXplICAzLCBlbmFibGVkCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSA0ICgweDE4MDAtMHgxODA3KSBmb3IgcmlkIDE0IG9mIHBjaTA6 MDoyOjAKCW1hcFsxOF06IHR5cGUgUHJlZmV0Y2hhYmxlIE1lbW9yeSwgcmFuZ2UgMzIsIGJhc2Ug MHhjMDAwMDAwMCwgc2l6ZSAyOCwgZW5hYmxlZApwY2liMDogYWxsb2NhdGVkIHR5cGUgMyAoMHhj MDAwMDAwMC0weGNmZmZmZmZmKSBmb3IgcmlkIDE4IG9mIHBjaTA6MDoyOjAKCW1hcFsxY106IHR5 cGUgTWVtb3J5LCByYW5nZSAzMiwgYmFzZSAweGQwNDAwMDAwLCBzaXplIDE4LCBlbmFibGVkCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQwNDAwMDAwLTB4ZDA0M2ZmZmYpIGZvciByaWQgMWMg b2YgcGNpMDowOjI6MApwY2liMDogbWF0Y2hlZCBlbnRyeSBmb3IgMC4yLklOVEEKcGNpYjA6IHNs b3QgMiBJTlRBIGhhcmR3aXJlZCB0byBJUlEgMTYKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9 MHgyN2E2LCByZXZpZD0weDAzCglkb21haW49MCwgYnVzPTAsIHNsb3Q9MiwgZnVuYz0xCgljbGFz cz0wMy04MC0wMCwgaGRydHlwZT0weDAwLCBtZmRldj0xCgljbWRyZWc9MHgwMDA3LCBzdGF0cmVn PTB4MDA5MCwgY2FjaGVsbnN6PTAgKGR3b3JkcykKCWxhdHRpbWVyPTB4MDAgKDAgbnMpLCBtaW5n bnQ9MHgwMCAoMCBucyksIG1heGxhdD0weDAwICgwIG5zKQoJcG93ZXJzcGVjIDIgIHN1cHBvcnRz IEQwIEQzICBjdXJyZW50IEQwCgltYXBbMTBdOiB0eXBlIE1lbW9yeSwgcmFuZ2UgMzIsIGJhc2Ug MHhkMDM4MDAwMCwgc2l6ZSAxOSwgZW5hYmxlZApwY2liMDogYWxsb2NhdGVkIHR5cGUgMyAoMHhk MDM4MDAwMC0weGQwM2ZmZmZmKSBmb3IgcmlkIDEwIG9mIHBjaTA6MDoyOjEKZm91bmQtPgl2ZW5k b3I9MHg4MDg2LCBkZXY9MHgyN2Q4LCByZXZpZD0weDAyCglkb21haW49MCwgYnVzPTAsIHNsb3Q9 MjcsIGZ1bmM9MAoJY2xhc3M9MDQtMDMtMDAsIGhkcnR5cGU9MHgwMCwgbWZkZXY9MAoJY21kcmVn PTB4MDAwNiwgc3RhdHJlZz0weDAwMTAsIGNhY2hlbG5zej0xNiAoZHdvcmRzKQoJbGF0dGltZXI9 MHgwMCAoMCBucyksIG1pbmdudD0weDAwICgwIG5zKSwgbWF4bGF0PTB4MDAgKDAgbnMpCglpbnRw aW49YSwgaXJxPTEwCglwb3dlcnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKCU1T SSBzdXBwb3J0cyAxIG1lc3NhZ2UsIDY0IGJpdAoJbWFwWzEwXTogdHlwZSBNZW1vcnksIHJhbmdl IDY0LCBiYXNlIDB4ZDA0NDAwMDAsIHNpemUgMTQsIGVuYWJsZWQKcGNpYjA6IGFsbG9jYXRlZCB0 eXBlIDMgKDB4ZDA0NDAwMDAtMHhkMDQ0M2ZmZikgZm9yIHJpZCAxMCBvZiBwY2kwOjA6Mjc6MApw Y2liMDogbWF0Y2hlZCBlbnRyeSBmb3IgMC4yNy5JTlRBCnBjaWIwOiBzbG90IDI3IElOVEEgaGFy ZHdpcmVkIHRvIElSUSAyMgpmb3VuZC0+CXZlbmRvcj0weDgwODYsIGRldj0weDI3ZDAsIHJldmlk PTB4MDIKCWRvbWFpbj0wLCBidXM9MCwgc2xvdD0yOCwgZnVuYz0wCgljbGFzcz0wNi0wNC0wMCwg aGRydHlwZT0weDAxLCBtZmRldj0xCgljbWRyZWc9MHgwMDAwLCBzdGF0cmVnPTB4MDAxMCwgY2Fj aGVsbnN6PTE2IChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDQgKDEw MDAgbnMpLCBtYXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1hLCBpcnE9MTEKCXBvd2Vyc3BlYyAy ICBzdXBwb3J0cyBEMCBEMyAgY3VycmVudCBEMAoJTVNJIHN1cHBvcnRzIDEgbWVzc2FnZQpwY2li MDogbWF0Y2hlZCBlbnRyeSBmb3IgMC4yOC5JTlRBCnBjaWIwOiBzbG90IDI4IElOVEEgaGFyZHdp cmVkIHRvIElSUSAxNgpmb3VuZC0+CXZlbmRvcj0weDgwODYsIGRldj0weDI3ZDIsIHJldmlkPTB4 MDIKCWRvbWFpbj0wLCBidXM9MCwgc2xvdD0yOCwgZnVuYz0xCgljbGFzcz0wNi0wNC0wMCwgaGRy dHlwZT0weDAxLCBtZmRldj0xCgljbWRyZWc9MHgwMDAwLCBzdGF0cmVnPTB4MDAxMCwgY2FjaGVs bnN6PTE2IChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDQgKDEwMDAg bnMpLCBtYXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1iLCBpcnE9MTEKCXBvd2Vyc3BlYyAyICBz dXBwb3J0cyBEMCBEMyAgY3VycmVudCBEMAoJTVNJIHN1cHBvcnRzIDEgbWVzc2FnZQpwY2liMDog bWF0Y2hlZCBlbnRyeSBmb3IgMC4yOC5JTlRCCnBjaWIwOiBzbG90IDI4IElOVEIgaGFyZHdpcmVk IHRvIElSUSAxNwpmb3VuZC0+CXZlbmRvcj0weDgwODYsIGRldj0weDI3ZDQsIHJldmlkPTB4MDIK CWRvbWFpbj0wLCBidXM9MCwgc2xvdD0yOCwgZnVuYz0yCgljbGFzcz0wNi0wNC0wMCwgaGRydHlw ZT0weDAxLCBtZmRldj0xCgljbWRyZWc9MHgwMDAwLCBzdGF0cmVnPTB4MDAxMCwgY2FjaGVsbnN6 PTE2IChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDQgKDEwMDAgbnMp LCBtYXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1jLCBpcnE9MTAKCXBvd2Vyc3BlYyAyICBzdXBw b3J0cyBEMCBEMyAgY3VycmVudCBEMAoJTVNJIHN1cHBvcnRzIDEgbWVzc2FnZQpwY2liMDogbWF0 Y2hlZCBlbnRyeSBmb3IgMC4yOC5JTlRDCnBjaWIwOiBzbG90IDI4IElOVEMgaGFyZHdpcmVkIHRv IElSUSAxOApmb3VuZC0+CXZlbmRvcj0weDgwODYsIGRldj0weDI3YzgsIHJldmlkPTB4MDIKCWRv bWFpbj0wLCBidXM9MCwgc2xvdD0yOSwgZnVuYz0wCgljbGFzcz0wYy0wMy0wMCwgaGRydHlwZT0w eDAwLCBtZmRldj0xCgljbWRyZWc9MHgwMDA1LCBzdGF0cmVnPTB4MDI4MCwgY2FjaGVsbnN6PTAg KGR3b3JkcykKCWxhdHRpbWVyPTB4MDAgKDAgbnMpLCBtaW5nbnQ9MHgwMCAoMCBucyksIG1heGxh dD0weDAwICgwIG5zKQoJaW50cGluPWEsIGlycT0xMQoJbWFwWzIwXTogdHlwZSBJL08gUG9ydCwg cmFuZ2UgMzIsIGJhc2UgMHgxODIwLCBzaXplICA1LCBlbmFibGVkCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSA0ICgweDE4MjAtMHgxODNmKSBmb3IgcmlkIDIwIG9mIHBjaTA6MDoyOTowCnBjaWIwOiBt YXRjaGVkIGVudHJ5IGZvciAwLjI5LklOVEEKcGNpYjA6IHNsb3QgMjkgSU5UQSBoYXJkd2lyZWQg dG8gSVJRIDIzCmZvdW5kLT4JdmVuZG9yPTB4ODA4NiwgZGV2PTB4MjdjOSwgcmV2aWQ9MHgwMgoJ ZG9tYWluPTAsIGJ1cz0wLCBzbG90PTI5LCBmdW5jPTEKCWNsYXNzPTBjLTAzLTAwLCBoZHJ0eXBl PTB4MDAsIG1mZGV2PTAKCWNtZHJlZz0weDAwMDUsIHN0YXRyZWc9MHgwMjgwLCBjYWNoZWxuc3o9 MCAoZHdvcmRzKQoJbGF0dGltZXI9MHgwMCAoMCBucyksIG1pbmdudD0weDAwICgwIG5zKSwgbWF4 bGF0PTB4MDAgKDAgbnMpCglpbnRwaW49YiwgaXJxPTEwCgltYXBbMjBdOiB0eXBlIEkvTyBQb3J0 LCByYW5nZSAzMiwgYmFzZSAweDE4NDAsIHNpemUgIDUsIGVuYWJsZWQKcGNpYjA6IGFsbG9jYXRl ZCB0eXBlIDQgKDB4MTg0MC0weDE4NWYpIGZvciByaWQgMjAgb2YgcGNpMDowOjI5OjEKcGNpYjA6 IG1hdGNoZWQgZW50cnkgZm9yIDAuMjkuSU5UQgpwY2liMDogc2xvdCAyOSBJTlRCIGhhcmR3aXJl ZCB0byBJUlEgMTkKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9MHgyN2NhLCByZXZpZD0weDAy Cglkb21haW49MCwgYnVzPTAsIHNsb3Q9MjksIGZ1bmM9MgoJY2xhc3M9MGMtMDMtMDAsIGhkcnR5 cGU9MHgwMCwgbWZkZXY9MAoJY21kcmVnPTB4MDAwNSwgc3RhdHJlZz0weDAyODAsIGNhY2hlbG5z ej0wIChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDAgKDAgbnMpLCBt YXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1jLCBpcnE9MTAKCW1hcFsyMF06IHR5cGUgSS9PIFBv cnQsIHJhbmdlIDMyLCBiYXNlIDB4MTg2MCwgc2l6ZSAgNSwgZW5hYmxlZApwY2liMDogYWxsb2Nh dGVkIHR5cGUgNCAoMHgxODYwLTB4MTg3ZikgZm9yIHJpZCAyMCBvZiBwY2kwOjA6Mjk6MgpwY2li MDogbWF0Y2hlZCBlbnRyeSBmb3IgMC4yOS5JTlRDCnBjaWIwOiBzbG90IDI5IElOVEMgaGFyZHdp cmVkIHRvIElSUSAxOApmb3VuZC0+CXZlbmRvcj0weDgwODYsIGRldj0weDI3Y2IsIHJldmlkPTB4 MDIKCWRvbWFpbj0wLCBidXM9MCwgc2xvdD0yOSwgZnVuYz0zCgljbGFzcz0wYy0wMy0wMCwgaGRy dHlwZT0weDAwLCBtZmRldj0wCgljbWRyZWc9MHgwMDA1LCBzdGF0cmVnPTB4MDI4MCwgY2FjaGVs bnN6PTAgKGR3b3JkcykKCWxhdHRpbWVyPTB4MDAgKDAgbnMpLCBtaW5nbnQ9MHgwMCAoMCBucyks IG1heGxhdD0weDAwICgwIG5zKQoJaW50cGluPWQsIGlycT0xMQoJbWFwWzIwXTogdHlwZSBJL08g UG9ydCwgcmFuZ2UgMzIsIGJhc2UgMHgxODgwLCBzaXplICA1LCBlbmFibGVkCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSA0ICgweDE4ODAtMHgxODlmKSBmb3IgcmlkIDIwIG9mIHBjaTA6MDoyOTozCnBj aWIwOiBtYXRjaGVkIGVudHJ5IGZvciAwLjI5LklOVEQKcGNpYjA6IHNsb3QgMjkgSU5URCBoYXJk d2lyZWQgdG8gSVJRIDE2CmZvdW5kLT4JdmVuZG9yPTB4ODA4NiwgZGV2PTB4MjdjYywgcmV2aWQ9 MHgwMgoJZG9tYWluPTAsIGJ1cz0wLCBzbG90PTI5LCBmdW5jPTcKCWNsYXNzPTBjLTAzLTIwLCBo ZHJ0eXBlPTB4MDAsIG1mZGV2PTAKCWNtZHJlZz0weDAwMDYsIHN0YXRyZWc9MHgwMjkwLCBjYWNo ZWxuc3o9MCAoZHdvcmRzKQoJbGF0dGltZXI9MHgwMCAoMCBucyksIG1pbmdudD0weDAwICgwIG5z KSwgbWF4bGF0PTB4MDAgKDAgbnMpCglpbnRwaW49YSwgaXJxPTExCglwb3dlcnNwZWMgMiAgc3Vw cG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKCW1hcFsxMF06IHR5cGUgTWVtb3J5LCByYW5nZSAzMiwg YmFzZSAweGQwNjQ0MDAwLCBzaXplIDEwLCBlbmFibGVkCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGQwNjQ0MDAwLTB4ZDA2NDQzZmYpIGZvciByaWQgMTAgb2YgcGNpMDowOjI5OjcKcGNpYjA6 IG1hdGNoZWQgZW50cnkgZm9yIDAuMjkuSU5UQQpwY2liMDogc2xvdCAyOSBJTlRBIGhhcmR3aXJl ZCB0byBJUlEgMjMKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9MHgyNDQ4LCByZXZpZD0weGUy Cglkb21haW49MCwgYnVzPTAsIHNsb3Q9MzAsIGZ1bmM9MAoJY2xhc3M9MDYtMDQtMDEsIGhkcnR5 cGU9MHgwMSwgbWZkZXY9MAoJY21kcmVnPTB4MDAwNCwgc3RhdHJlZz0weDAwMTAsIGNhY2hlbG5z ej0wIChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDQgKDEwMDAgbnMp LCBtYXhsYXQ9MHgwMCAoMCBucykKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9MHgyN2I5LCBy ZXZpZD0weDAyCglkb21haW49MCwgYnVzPTAsIHNsb3Q9MzEsIGZ1bmM9MAoJY2xhc3M9MDYtMDEt MDAsIGhkcnR5cGU9MHgwMCwgbWZkZXY9MQoJY21kcmVnPTB4MDAwNywgc3RhdHJlZz0weDAyMTAs IGNhY2hlbG5zej0wIChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4MDAg KDAgbnMpLCBtYXhsYXQ9MHgwMCAoMCBucykKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9MHgy N2M0LCByZXZpZD0weDAyCglkb21haW49MCwgYnVzPTAsIHNsb3Q9MzEsIGZ1bmM9MgoJY2xhc3M9 MDEtMDEtODAsIGhkcnR5cGU9MHgwMCwgbWZkZXY9MAoJY21kcmVnPTB4MDAwNSwgc3RhdHJlZz0w eDAyYjgsIGNhY2hlbG5zej0wIChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250 PTB4MDAgKDAgbnMpLCBtYXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1iLCBpcnE9MjU1Cglwb3dl cnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKcGNpYjA6IGFsbG9jYXRlZCB0eXBl IDQgKDB4MWYwLTB4MWY3KSBmb3IgcmlkIDEwIG9mIHBjaTA6MDozMToyCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSA0ICgweDNmNi0weDNmNikgZm9yIHJpZCAxNCBvZiBwY2kwOjA6MzE6MgpwY2liMDog YWxsb2NhdGVkIHR5cGUgNCAoMHgxNzAtMHgxNzcpIGZvciByaWQgMTggb2YgcGNpMDowOjMxOjIK cGNpYjA6IGFsbG9jYXRlZCB0eXBlIDQgKDB4Mzc2LTB4Mzc2KSBmb3IgcmlkIDFjIG9mIHBjaTA6 MDozMToyCgltYXBbMjBdOiB0eXBlIEkvTyBQb3J0LCByYW5nZSAzMiwgYmFzZSAweDE4YjAsIHNp emUgIDQsIGVuYWJsZWQKcGNpYjA6IGFsbG9jYXRlZCB0eXBlIDQgKDB4MThiMC0weDE4YmYpIGZv ciByaWQgMjAgb2YgcGNpMDowOjMxOjIKZm91bmQtPgl2ZW5kb3I9MHg4MDg2LCBkZXY9MHgyN2Rh LCByZXZpZD0weDAyCglkb21haW49MCwgYnVzPTAsIHNsb3Q9MzEsIGZ1bmM9MwoJY2xhc3M9MGMt MDUtMDAsIGhkcnR5cGU9MHgwMCwgbWZkZXY9MAoJY21kcmVnPTB4MDAwMSwgc3RhdHJlZz0weDAy ODAsIGNhY2hlbG5zej0wIChkd29yZHMpCglsYXR0aW1lcj0weDAwICgwIG5zKSwgbWluZ250PTB4 MDAgKDAgbnMpLCBtYXhsYXQ9MHgwMCAoMCBucykKCWludHBpbj1iLCBpcnE9MTAKCW1hcFsyMF06 IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDB4MThjMCwgc2l6ZSAgNSwgZW5hYmxlZApw Y2liMDogYWxsb2NhdGVkIHR5cGUgNCAoMHgxOGMwLTB4MThkZikgZm9yIHJpZCAyMCBvZiBwY2kw OjA6MzE6MwpwY2liMDogbWF0Y2hlZCBlbnRyeSBmb3IgMC4zMS5JTlRCCnBjaWIwOiBzbG90IDMx IElOVEIgaGFyZHdpcmVkIHRvIElSUSAxOQp2Z2FwY2kwOiA8VkdBLWNvbXBhdGlibGUgZGlzcGxh eT4gcG9ydCAweDE4MDAtMHgxODA3IG1lbSAweGQwMzAwMDAwLTB4ZDAzN2ZmZmYsMHhjMDAwMDAw MC0weGNmZmZmZmZmLDB4ZDA0MDAwMDAtMHhkMDQzZmZmZiBpcnEgMTYgYXQgZGV2aWNlIDIuMCBv biBwY2kwCmFncDA6IDxJbnRlbCA4Mjk0NUdNICg5NDVHTSBHTUNIKSBTVkdBIGNvbnRyb2xsZXI+ IG9uIHZnYXBjaTAKYWdwMDogYXBlcnR1cmUgc2l6ZSBpcyAyNTZNLCBkZXRlY3RlZCA3OTMyayBz dG9sZW4gbWVtb3J5CnZnYXBjaTE6IDxWR0EtY29tcGF0aWJsZSBkaXNwbGF5PiBtZW0gMHhkMDM4 MDAwMC0weGQwM2ZmZmZmIGF0IGRldmljZSAyLjEgb24gcGNpMApoZGFjMDogPEludGVsIDgyODAx RyBIREEgQ29udHJvbGxlcj4gbWVtIDB4ZDA0NDAwMDAtMHhkMDQ0M2ZmZiBpcnEgMjIgYXQgZGV2 aWNlIDI3LjAgb24gcGNpMApoZGFjMDogSERBIERyaXZlciBSZXZpc2lvbjogMjAxMjAxMTFfMDAw MQpoZGFjMDogQ29uZmlnIG9wdGlvbnM6IG9uPTB4MDAwMDAwMDAgb2ZmPTB4MDAwMDAwMDAKaGRh YzA6IGF0dGVtcHRpbmcgdG8gYWxsb2NhdGUgMSBNU0kgdmVjdG9ycyAoMSBzdXBwb3J0ZWQpCm1z aTogcm91dGluZyBNU0kgSVJRIDI1NiB0byBsb2NhbCBBUElDIDAgdmVjdG9yIDUwCmhkYWMwOiB1 c2luZyBJUlEgMjU2IGZvciBNU0kKaGRhYzA6IENhcHM6IE9TUyA0LCBJU1MgNCwgQlNTIDAsIE5T RE8gMSwgNjRiaXQsIENPUkIgMjU2LCBSSVJCIDI1NgpwY2liMTogPEFDUEkgUENJLVBDSSBicmlk Z2U+IGlycSAxNiBhdCBkZXZpY2UgMjguMCBvbiBwY2kwCnBjaWIxOiBmYWlsZWQgdG8gYWxsb2Nh dGUgaW5pdGlhbCBJL08gcG9ydCB3aW5kb3c6IDAtMHhmZmYKcGNpYjE6IGZhaWxlZCB0byBhbGxv Y2F0ZSBpbml0aWFsIG1lbW9yeSB3aW5kb3c6IDAtMHhmZmZmZgpwY2liMTogZmFpbGVkIHRvIGFs bG9jYXRlIGluaXRpYWwgcHJlZmV0Y2ggd2luZG93OiAwLTB4ZmZmZmYKcGNpYjE6ICAgZG9tYWlu ICAgICAgICAgICAgMApwY2liMTogICBzZWNvbmRhcnkgYnVzICAgICAyCnBjaWIxOiAgIHN1Ym9y ZGluYXRlIGJ1cyAgIDIKcGNpYjE6ICAgbm8gcHJlZmV0Y2hlZCBkZWNvZGUKcGNpMjogPEFDUEkg UENJIGJ1cz4gb24gcGNpYjEKcGNpMjogZG9tYWluPTAsIHBoeXNpY2FsIGJ1cz0yCmZvdW5kLT4J dmVuZG9yPTB4MTFhYiwgZGV2PTB4NDM1MiwgcmV2aWQ9MHgxNAoJZG9tYWluPTAsIGJ1cz0yLCBz bG90PTAsIGZ1bmM9MAoJY2xhc3M9MDItMDAtMDAsIGhkcnR5cGU9MHgwMCwgbWZkZXY9MAoJY21k cmVnPTB4MDAwMCwgc3RhdHJlZz0weDQwMTAsIGNhY2hlbG5zej0xNiAoZHdvcmRzKQoJbGF0dGlt ZXI9MHgwMCAoMCBucyksIG1pbmdudD0weDAwICgwIG5zKSwgbWF4bGF0PTB4MDAgKDAgbnMpCglp bnRwaW49YSwgaXJxPTExCglwb3dlcnNwZWMgMiAgc3VwcG9ydHMgRDAgRDEgRDIgRDMgIGN1cnJl bnQgRDAKCU1TSSBzdXBwb3J0cyAyIG1lc3NhZ2VzLCA2NCBiaXQKCW1hcFsxMF06IHR5cGUgTWVt b3J5LCByYW5nZSA2NCwgYmFzZSAwLCBzaXplIDE0LCBtZW1vcnkgZGlzYWJsZWQKCW1hcFsxOF06 IHR5cGUgSS9PIFBvcnQsIHJhbmdlIDMyLCBiYXNlIDAsIHNpemUgIDgsIHBvcnQgZGlzYWJsZWQK cGNpYjE6IG1hdGNoZWQgZW50cnkgZm9yIDIuMC5JTlRBCnBjaWIxOiBzbG90IDAgSU5UQSBoYXJk d2lyZWQgdG8gSVJRIDE2Cm1za2MwOiA8TWFydmVsbCBZdWtvbiA4OEU4MDM4IEZhc3QgRXRoZXJu ZXQ+IGlycSAxNiBhdCBkZXZpY2UgMC4wIG9uIHBjaTIKcGNpYjA6IGFsbG9jYXRlZCB0eXBlIDMg KDB4ODAwMDAwMDAtMHg4MDBmZmZmZikgZm9yIHJpZCAyMCBvZiBwY2liMQpwY2liMTogYWxsb2Nh dGVkIGluaXRpYWwgbWVtb3J5IHdpbmRvdyBvZiAweDgwMDAwMDAwLTB4ODAwZmZmZmYKcGNpYjE6 IGFsbG9jYXRlZCBtZW1vcnkgcmFuZ2UgKDB4ODAwMDAwMDAtMHg4MDAwM2ZmZikgZm9yIHJpZCAx MCBvZiBtc2tjMAptc2tjMDogTGF6eSBhbGxvY2F0aW9uIG9mIDB4NDAwMCBieXRlcyByaWQgMHgx MCB0eXBlIDMgYXQgMHg4MDAwMDAwMAptc2tjMDogTVNJIGNvdW50IDogMgptc2tjMDogYXR0ZW1w dGluZyB0byBhbGxvY2F0ZSAxIE1TSSB2ZWN0b3JzICgyIHN1cHBvcnRlZCkKbXNpOiByb3V0aW5n IE1TSSBJUlEgMjU3IHRvIGxvY2FsIEFQSUMgMCB2ZWN0b3IgNTEKbXNrYzA6IHVzaW5nIElSUSAy NTcgZm9yIE1TSQptc2tjMDogUkFNIGJ1ZmZlciBzaXplIDogNEtCCm1za2MwOiBQb3J0IDAgOiBS eCBRdWV1ZSAyS0IoMHgwMDAwMDAwMDoweDAwMDAwN2ZmKQptc2tjMDogUG9ydCAwIDogVHggUXVl dWUgMktCKDB4MDAwMDA4MDA6MHgwMDAwMGZmZikKbXNrMDogPE1hcnZlbGwgVGVjaG5vbG9neSBH cm91cCBMdGQuIFl1a29uIEZFIElkIDB4YjcgUmV2IDB4MDE+IG9uIG1za2MwCm1zazA6IGRpc2Fi bGluZyBqdW1ibyBmcmFtZSBzdXBwb3J0Cm1zazA6IGJwZiBhdHRhY2hlZAptc2swOiBFdGhlcm5l dCBhZGRyZXNzOiAwMDoxYjoyNDo1OTo1Yjo3YQptaWlidXMwOiA8TUlJIGJ1cz4gb24gbXNrMApl MTAwMHBoeTA6IDxNYXJ2ZWxsIDg4RTMwODIgMTAvMTAwIEZhc3QgRXRoZXJuZXQgUEhZPiBQSFkg MCBvbiBtaWlidXMwCmUxMDAwcGh5MDogT1VJIDB4MDAwYWMyLCBtb2RlbCAweDAwMDgsIHJldi4g MwplMTAwMHBoeTA6ICBub25lLCAxMGJhc2VULCAxMGJhc2VULUZEWCwgMTAwYmFzZVRYLCAxMDBi YXNlVFgtRkRYLCBhdXRvLCBhdXRvLWZsb3cKcGNpYjI6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBp cnEgMTcgYXQgZGV2aWNlIDI4LjEgb24gcGNpMApwY2liMjogZmFpbGVkIHRvIGFsbG9jYXRlIGlu aXRpYWwgSS9PIHBvcnQgd2luZG93OiAwLTB4ZmZmCnBjaWIyOiBmYWlsZWQgdG8gYWxsb2NhdGUg aW5pdGlhbCBtZW1vcnkgd2luZG93OiAwLTB4ZmZmZmYKcGNpYjI6IGZhaWxlZCB0byBhbGxvY2F0 ZSBpbml0aWFsIHByZWZldGNoIHdpbmRvdzogMC0weGZmZmZmCnBjaWIyOiAgIGRvbWFpbiAgICAg ICAgICAgIDAKcGNpYjI6ICAgc2Vjb25kYXJ5IGJ1cyAgICAgMwpwY2liMjogICBzdWJvcmRpbmF0 ZSBidXMgICAzCnBjaWIyOiAgIG5vIHByZWZldGNoZWQgZGVjb2RlCnBjaTM6IDxBQ1BJIFBDSSBi dXM+IG9uIHBjaWIyCnBjaTM6IGRvbWFpbj0wLCBwaHlzaWNhbCBidXM9Mwpmb3VuZC0+CXZlbmRv cj0weDE2OGMsIGRldj0weDAwMWMsIHJldmlkPTB4MDEKCWRvbWFpbj0wLCBidXM9Mywgc2xvdD0w LCBmdW5jPTAKCWNsYXNzPTAyLTAwLTAwLCBoZHJ0eXBlPTB4MDAsIG1mZGV2PTAKCWNtZHJlZz0w eDAwMDAsIHN0YXRyZWc9MHgwMDEwLCBjYWNoZWxuc3o9MTYgKGR3b3JkcykKCWxhdHRpbWVyPTB4 MDAgKDAgbnMpLCBtaW5nbnQ9MHgwMCAoMCBucyksIG1heGxhdD0weDAwICgwIG5zKQoJaW50cGlu PWEsIGlycT0xMQoJcG93ZXJzcGVjIDIgIHN1cHBvcnRzIEQwIEQzICBjdXJyZW50IEQwCglNU0kg c3VwcG9ydHMgMSBtZXNzYWdlCglNU0ktWCBzdXBwb3J0cyAxIG1lc3NhZ2UgaW4gbWFwIDB4MTAK CW1hcFsxMF06IHR5cGUgTWVtb3J5LCByYW5nZSA2NCwgYmFzZSAwLCBzaXplIDE2LCBtZW1vcnkg ZGlzYWJsZWQKcGNpYjI6IG1hdGNoZWQgZW50cnkgZm9yIDMuMC5JTlRBCnBjaWIyOiBzbG90IDAg SU5UQSBoYXJkd2lyZWQgdG8gSVJRIDE3CmF0aDA6IDxBdGhlcm9zIDU0MjQvMjQyND4gaXJxIDE3 IGF0IGRldmljZSAwLjAgb24gcGNpMwpwY2liMDogYWxsb2NhdGVkIHR5cGUgMyAoMHg4MDEwMDAw MC0weDgwMWZmZmZmKSBmb3IgcmlkIDIwIG9mIHBjaWIyCnBjaWIyOiBhbGxvY2F0ZWQgaW5pdGlh bCBtZW1vcnkgd2luZG93IG9mIDB4ODAxMDAwMDAtMHg4MDFmZmZmZgpwY2liMjogYWxsb2NhdGVk IG1lbW9yeSByYW5nZSAoMHg4MDEwMDAwMC0weDgwMTBmZmZmKSBmb3IgcmlkIDEwIG9mIGF0aDAK YXRoMDogTGF6eSBhbGxvY2F0aW9uIG9mIDB4MTAwMDAgYnl0ZXMgcmlkIDB4MTAgdHlwZSAzIGF0 IDB4ODAxMDAwMDAKaW9hcGljMDogcm91dGluZyBpbnRwaW4gMTcgKFBDSSBJUlEgMTcpIHRvIGxh cGljIDAgdmVjdG9yIDUyCmF0aDA6IDExYiByYXRlczogMU1icHMgMk1icHMgNS41TWJwcyAxMU1i cHMKYXRoMDogMTFnIHJhdGVzOiAxTWJwcyAyTWJwcyA1LjVNYnBzIDExTWJwcyA2TWJwcyA5TWJw cyAxMk1icHMgMThNYnBzIDI0TWJwcyAzNk1icHMgNDhNYnBzIDU0TWJwcwphdGgwOiBBUjI0MjUg bWFjIDE0LjIgUkY1NDI0IHBoeSA3LjAKYXRoMDogMkdIeiByYWRpbzogMHgwMDAwOyA1R0h6IHJh ZGlvOiAweDAwYTIKYXRoMDogVXNlIGh3IHF1ZXVlIDEgZm9yIFdNRV9BQ19CRSB0cmFmZmljCmF0 aDA6IFVzZSBodyBxdWV1ZSAwIGZvciBXTUVfQUNfQksgdHJhZmZpYwphdGgwOiBVc2UgaHcgcXVl dWUgMiBmb3IgV01FX0FDX1ZJIHRyYWZmaWMKYXRoMDogVXNlIGh3IHF1ZXVlIDMgZm9yIFdNRV9B Q19WTyB0cmFmZmljCmF0aDA6IFVzZSBodyBxdWV1ZSA4IGZvciBDQUIgdHJhZmZpYwphdGgwOiBV c2UgaHcgcXVldWUgOSBmb3IgYmVhY29ucwphdGgwOiB1c2luZyBtdWx0aWNhc3Qga2V5IHNlYXJj aApwY2liMzogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSAxOCBhdCBkZXZpY2UgMjguMiBvbiBw Y2kwCnBjaWIzOiBmYWlsZWQgdG8gYWxsb2NhdGUgaW5pdGlhbCBJL08gcG9ydCB3aW5kb3c6IDAt MHhmZmYKcGNpYjM6IGZhaWxlZCB0byBhbGxvY2F0ZSBpbml0aWFsIG1lbW9yeSB3aW5kb3c6IDAt MHhmZmZmZgpwY2liMzogZmFpbGVkIHRvIGFsbG9jYXRlIGluaXRpYWwgcHJlZmV0Y2ggd2luZG93 OiAwLTB4ZmZmZmYKcGNpYjM6ICAgZG9tYWluICAgICAgICAgICAgMApwY2liMzogICBzZWNvbmRh cnkgYnVzICAgICA0CnBjaWIzOiAgIHN1Ym9yZGluYXRlIGJ1cyAgIDQKcGNpYjM6ICAgbm8gcHJl ZmV0Y2hlZCBkZWNvZGUKcGNpNDogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjMKcGNpNDogZG9tYWlu PTAsIHBoeXNpY2FsIGJ1cz00CnVoY2kwOiA8SW50ZWwgODI4MDFHIChJQ0g3KSBVU0IgY29udHJv bGxlciBVU0ItQT4gcG9ydCAweDE4MjAtMHgxODNmIGlycSAyMyBhdCBkZXZpY2UgMjkuMCBvbiBw Y2kwCmlvYXBpYzA6IHJvdXRpbmcgaW50cGluIDIzIChQQ0kgSVJRIDIzKSB0byBsYXBpYyAwIHZl Y3RvciA1Mwp1c2J1czA6IDxJbnRlbCA4MjgwMUcgKElDSDcpIFVTQiBjb250cm9sbGVyIFVTQi1B PiBvbiB1aGNpMAp1c2J1czA6IGJwZiBhdHRhY2hlZAp1aGNpMDogdXNicGY6IEF0dGFjaGVkCnVo Y2kxOiA8SW50ZWwgODI4MDFHIChJQ0g3KSBVU0IgY29udHJvbGxlciBVU0ItQj4gcG9ydCAweDE4 NDAtMHgxODVmIGlycSAxOSBhdCBkZXZpY2UgMjkuMSBvbiBwY2kwCmlvYXBpYzA6IHJvdXRpbmcg aW50cGluIDE5IChQQ0kgSVJRIDE5KSB0byBsYXBpYyAwIHZlY3RvciA1NAp1c2J1czE6IDxJbnRl bCA4MjgwMUcgKElDSDcpIFVTQiBjb250cm9sbGVyIFVTQi1CPiBvbiB1aGNpMQp1c2J1czE6IGJw ZiBhdHRhY2hlZAp1aGNpMTogdXNicGY6IEF0dGFjaGVkCnVoY2kyOiA8SW50ZWwgODI4MDFHIChJ Q0g3KSBVU0IgY29udHJvbGxlciBVU0ItQz4gcG9ydCAweDE4NjAtMHgxODdmIGlycSAxOCBhdCBk ZXZpY2UgMjkuMiBvbiBwY2kwCmlvYXBpYzA6IHJvdXRpbmcgaW50cGluIDE4IChQQ0kgSVJRIDE4 KSB0byBsYXBpYyAwIHZlY3RvciA1NQp1c2J1czI6IDxJbnRlbCA4MjgwMUcgKElDSDcpIFVTQiBj b250cm9sbGVyIFVTQi1DPiBvbiB1aGNpMgp1c2J1czI6IGJwZiBhdHRhY2hlZAp1aGNpMjogdXNi cGY6IEF0dGFjaGVkCnVoY2kzOiA8SW50ZWwgODI4MDFHIChJQ0g3KSBVU0IgY29udHJvbGxlciBV U0ItRD4gcG9ydCAweDE4ODAtMHgxODlmIGlycSAxNiBhdCBkZXZpY2UgMjkuMyBvbiBwY2kwCmlv YXBpYzA6IHJvdXRpbmcgaW50cGluIDE2IChQQ0kgSVJRIDE2KSB0byBsYXBpYyAwIHZlY3RvciA1 Ngp1c2J1czM6IDxJbnRlbCA4MjgwMUcgKElDSDcpIFVTQiBjb250cm9sbGVyIFVTQi1EPiBvbiB1 aGNpMwp1c2J1czM6IGJwZiBhdHRhY2hlZAp1aGNpMzogdXNicGY6IEF0dGFjaGVkCmVoY2kwOiA8 SW50ZWwgODI4MDFHQi9SIChJQ0g3KSBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAweGQwNjQ0MDAw LTB4ZDA2NDQzZmYgaXJxIDIzIGF0IGRldmljZSAyOS43IG9uIHBjaTAKdXNidXM0OiBFSENJIHZl cnNpb24gMS4wCnVzYnVzNDogPEludGVsIDgyODAxR0IvUiAoSUNINykgVVNCIDIuMCBjb250cm9s bGVyPiBvbiBlaGNpMAp1c2J1czQ6IGJwZiBhdHRhY2hlZAplaGNpMDogdXNicGY6IEF0dGFjaGVk CnBjaWI0OiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gYXQgZGV2aWNlIDMwLjAgb24gcGNpMApwY2li MDogYWxsb2NhdGVkIHR5cGUgMyAoMHhkMDIwMDAwMC0weGQwMmZmZmZmKSBmb3IgcmlkIDIwIG9m IHBjaWI0CnBjaWI0OiAgIGRvbWFpbiAgICAgICAgICAgIDAKcGNpYjQ6ICAgc2Vjb25kYXJ5IGJ1 cyAgICAgMTAKcGNpYjQ6ICAgc3Vib3JkaW5hdGUgYnVzICAgMTEKcGNpYjQ6ICAgbWVtb3J5IGRl Y29kZSAgICAgMHhkMDIwMDAwMC0weGQwMmZmZmZmCnBjaWI0OiAgIG5vIHByZWZldGNoZWQgZGVj b2RlCnBjaWI0OiAgIFN1YnRyYWN0aXZlbHkgZGVjb2RlZCBicmlkZ2UuCnBjaTEwOiA8QUNQSSBQ Q0kgYnVzPiBvbiBwY2liNApwY2kxMDogZG9tYWluPTAsIHBoeXNpY2FsIGJ1cz0xMApmb3VuZC0+ CXZlbmRvcj0weDEwNGMsIGRldj0weDgwMzksIHJldmlkPTB4MDAKCWRvbWFpbj0wLCBidXM9MTAs IHNsb3Q9OSwgZnVuYz0wCgljbGFzcz0wNi0wNy0wMCwgaGRydHlwZT0weDAyLCBtZmRldj0xCglj bWRyZWc9MHgwMDAwLCBzdGF0cmVnPTB4MDIxMCwgY2FjaGVsbnN6PTE2IChkd29yZHMpCglsYXR0 aW1lcj0weDMxICgxNDcwIG5zKSwgbWluZ250PTB4NDQgKDE3MDAwIG5zKSwgbWF4bGF0PTB4MDMg KDc1MCBucykKCWludHBpbj1hLCBpcnE9MTAKCXBvd2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBEMSBE MiBEMyAgY3VycmVudCBEMAoJbWFwWzEwXTogdHlwZSBNZW1vcnksIHJhbmdlIDMyLCBiYXNlIDB4 ZDAyMDQwMDAsIHNpemUgMTIsIG1lbW9yeSBkaXNhYmxlZApwY2liNDogYWxsb2NhdGVkIG1lbW9y eSByYW5nZSAoMHhkMDIwNDAwMC0weGQwMjA0ZmZmKSBmb3IgcmlkIDEwIG9mIHBjaTA6MTA6OTow CnBjaWI0OiBtYXRjaGVkIGVudHJ5IGZvciAxMC45LklOVEEKcGNpYjQ6IHNsb3QgOSBJTlRBIGhh cmR3aXJlZCB0byBJUlEgMjAKZm91bmQtPgl2ZW5kb3I9MHgxMDRjLCBkZXY9MHg4MDNiLCByZXZp ZD0weDAwCglkb21haW49MCwgYnVzPTEwLCBzbG90PTksIGZ1bmM9MgoJY2xhc3M9MDEtODAtMDAs IGhkcnR5cGU9MHgwMCwgbWZkZXY9MQoJY21kcmVnPTB4MDAwNiwgc3RhdHJlZz0weDAyMTAsIGNh Y2hlbG5zej0xNiAoZHdvcmRzKQoJbGF0dGltZXI9MHgzOSAoMTcxMCBucyksIG1pbmdudD0weDA3 ICgxNzUwIG5zKSwgbWF4bGF0PTB4MDQgKDEwMDAgbnMpCglpbnRwaW49YSwgaXJxPTEwCglwb3dl cnNwZWMgMiAgc3VwcG9ydHMgRDAgRDEgRDIgRDMgIGN1cnJlbnQgRDAKCW1hcFsxMF06IHR5cGUg TWVtb3J5LCByYW5nZSAzMiwgYmFzZSAweGQwMjA1MDAwLCBzaXplIDEyLCBlbmFibGVkCnBjaWI0 OiBhbGxvY2F0ZWQgbWVtb3J5IHJhbmdlICgweGQwMjA1MDAwLTB4ZDAyMDVmZmYpIGZvciByaWQg MTAgb2YgcGNpMDoxMDo5OjIKcGNpYjQ6IG1hdGNoZWQgZW50cnkgZm9yIDEwLjkuSU5UQQpwY2li NDogc2xvdCA5IElOVEEgaGFyZHdpcmVkIHRvIElSUSAyMApjYmIwOiA8UENJLUNhcmRCdXMgQnJp ZGdlPiBtZW0gMHhkMDIwNDAwMC0weGQwMjA0ZmZmIGlycSAyMCBhdCBkZXZpY2UgOS4wIG9uIHBj aTEwCmNhcmRidXMwOiA8Q2FyZEJ1cyBidXM+IG9uIGNiYjAKcGNjYXJkMDogPDE2LWJpdCBQQ0Nh cmQgYnVzPiBvbiBjYmIwCmNiYjA6IFBDSSBDb25maWd1cmF0aW9uIHNwYWNlOgogIDB4MDA6IDB4 ODAzOTEwNGMgMHgwMjEwMDAwNyAweDA2MDcwMDAwIDB4MDA4MjMxMTAgCiAgMHgxMDogMHhkMDIw NDAwMCAweDAyMDAwMGEwIDB4MjAwYjBiMGEgMHhmZmZmZjAwMCAKICAweDIwOiAweDAwMDAwMDAw IDB4ZmZmZmYwMDAgMHgwMDAwMDAwMCAweGZmZmZmZmZjIAogIDB4MzA6IDB4MDAwMDAwMDAgMHhm ZmZmZmZmYyAweDAwMDAwMDAwIDB4MDc0NDAxMTQgCiAgMHg0MDogMHgwMTEwMTAyNSAweDAwMDAw MDAxIDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAKICAweDUwOiAweDAwMDAwMDAwIDB4MDAwMDAwMDAg MHgwMDAwMDAwMCAweDAwMDAwMDAwIAogIDB4NjA6IDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAweDAw MDAwMDAwIDB4MDAwMDAwMDAgCiAgMHg3MDogMHgwMDAwMDAwMCAweDAwMDAwMDAwIDB4MDAwMDAw MDAgMHgwMDAwMDAwMCAKICAweDgwOiAweDE4NDRkMDYwIDB4MDJkODAwMTkgMHgwMDE5MDAxNiAw eDAxMzIxYjIyIAogIDB4OTA6IDB4NjA2NjAyYzAgMHgwMDAwMDAwMCAweDAwMDAwMDAwIDB4MDAw MDAwMDAgCiAgMHhhMDogMHhmZTEyMDAwMSAweDAwYzAwMDAwIDB4MDAwMDAwMDAgMHgwMDAwMDAw MCAKICAweGIwOiAweDAwMDAwMDAwIDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAweDAwMDAwMDAwIAog IDB4YzA6IDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAweDAwMDAwMDAwIDB4MDAwMDAwMDAgCiAgMHhk MDogMHgwMDAwMDAwMCAweDAwMDAwMDAwIDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAKICAweGUwOiAw eDAwMDAwMDAwIDB4MDAwMDAwMDAgMHgwMDAwMDAwMCAweDAwMDAwMDAwIAogIDB4ZjA6IDB4ZjQ4 YjM5MGYgMHg5NzAxOWFkMiAweDAwMDAwMDAwIDB4MDAwMDAwMDAgCnBjaTEwOiA8bWFzcyBzdG9y YWdlPiBhdCBkZXZpY2UgOS4yIChubyBkcml2ZXIgYXR0YWNoZWQpCmlzYWIwOiA8UENJLUlTQSBi cmlkZ2U+IGF0IGRldmljZSAzMS4wIG9uIHBjaTAKaXNhMDogPElTQSBidXM+IG9uIGlzYWIwCmF0 YXBjaTA6IDxJbnRlbCBJQ0g3TSBTQVRBMTUwIGNvbnRyb2xsZXI+IHBvcnQgMHgxZjAtMHgxZjcs MHgzZjYsMHgxNzAtMHgxNzcsMHgzNzYsMHgxOGIwLTB4MThiZiBhdCBkZXZpY2UgMzEuMiBvbiBw Y2kwCmF0YTA6IDxBVEEgY2hhbm5lbD4gYXQgY2hhbm5lbCAwIG9uIGF0YXBjaTAKaW9hcGljMDog cm91dGluZyBpbnRwaW4gMTQgKElTQSBJUlEgMTQpIHRvIGxhcGljIDAgdmVjdG9yIDU3CmF0YTE6 IDxBVEEgY2hhbm5lbD4gYXQgY2hhbm5lbCAxIG9uIGF0YXBjaTAKaW9hcGljMDogcm91dGluZyBp bnRwaW4gMTUgKElTQSBJUlEgMTUpIHRvIGxhcGljIDAgdmVjdG9yIDU4CnBjaTA6IDxzZXJpYWwg YnVzLCBTTUJ1cz4gYXQgZGV2aWNlIDMxLjMgKG5vIGRyaXZlciBhdHRhY2hlZCkKYWNwaV9hY2Fk MDogPEFDIEFkYXB0ZXI+IG9uIGFjcGkwCmJhdHRlcnkwOiA8QUNQSSBDb250cm9sIE1ldGhvZCBC YXR0ZXJ5PiBvbiBhY3BpMAphY3BpX2xpZDA6IDxDb250cm9sIE1ldGhvZCBMaWQgU3dpdGNoPiBv biBhY3BpMAphY3BpX2J1dHRvbjA6IDxQb3dlciBCdXR0b24+IG9uIGFjcGkwCmFjcGlfYnV0dG9u MTogPFNsZWVwIEJ1dHRvbj4gb24gYWNwaTAKYWNwaV90ejA6IDxUaGVybWFsIFpvbmU+IG9uIGFj cGkwCmF0cnRjMDogPEFUIHJlYWx0aW1lIGNsb2NrPiBwb3J0IDB4NzAtMHg3NyBvbiBhY3BpMAph dHJ0YzA6IHJlZ2lzdGVyZWQgYXMgYSB0aW1lLW9mLWRheSBjbG9jayAocmVzb2x1dGlvbiAxMDAw MDAwdXMsIGFkanVzdG1lbnQgMC41MDAwMDAwMDBzKQppb2FwaWMwOiByb3V0aW5nIGludHBpbiA4 IChJU0EgSVJRIDgpIHRvIGxhcGljIDAgdmVjdG9yIDU5CkV2ZW50IHRpbWVyICJSVEMiIGZyZXF1 ZW5jeSAzMjc2OCBIeiBxdWFsaXR5IDAKYXR0aW1lcjA6IDxBVCB0aW1lcj4gcG9ydCAweDQwLTB4 NDMsMHg1MC0weDUzIG9uIGFjcGkwClRpbWVjb3VudGVyICJpODI1NCIgZnJlcXVlbmN5IDExOTMx ODIgSHogcXVhbGl0eSAwCmlvYXBpYzA6IHJvdXRpbmcgaW50cGluIDIgKElTQSBJUlEgMCkgdG8g bGFwaWMgMCB2ZWN0b3IgNjAKRXZlbnQgdGltZXIgImk4MjU0IiBmcmVxdWVuY3kgMTE5MzE4MiBI eiBxdWFsaXR5IDEwMAphdGtiZGMwOiA8S2V5Ym9hcmQgY29udHJvbGxlciAoaTgwNDIpPiBwb3J0 IDB4NjAsMHg2NCBpcnEgMSBvbiBhY3BpMAphdGtiZDA6IDxBVCBLZXlib2FyZD4gaXJxIDEgb24g YXRrYmRjMAphdGtiZDogdGhlIGN1cnJlbnQga2JkIGNvbnRyb2xsZXIgY29tbWFuZCBieXRlIDAw NDcKYXRrYmQ6IGtleWJvYXJkIElEIDB4NDFhYiAoMikKa2JkMCBhdCBhdGtiZDAKa2JkMDogYXRr YmQwLCBBVCAxMDEvMTAyICgyKSwgY29uZmlnOjB4MCwgZmxhZ3M6MHgzZDAwMDAKaW9hcGljMDog cm91dGluZyBpbnRwaW4gMSAoSVNBIElSUSAxKSB0byBsYXBpYyAwIHZlY3RvciA2MQphdGtiZDA6 IFtHSUFOVC1MT0NLRURdCnBzbTA6IHVuYWJsZSB0byBhbGxvY2F0ZSBJUlEKcHNtY3BucDA6IDxQ Uy8yIG1vdXNlIHBvcnQ+IGlycSAxMiBvbiBhY3BpMApwc20wOiBjdXJyZW50IGNvbW1hbmQgYnl0 ZTowMDQ3CnBzbTA6IDxQUy8yIE1vdXNlPiBpcnEgMTIgb24gYXRrYmRjMAppb2FwaWMwOiByb3V0 aW5nIGludHBpbiAxMiAoSVNBIElSUSAxMikgdG8gbGFwaWMgMCB2ZWN0b3IgNjIKcHNtMDogW0dJ QU5ULUxPQ0tFRF0KcHNtMDogbW9kZWwgR2VuZXJpYyBQUy8yIG1vdXNlLCBkZXZpY2UgSUQgMC0w MCwgMiBidXR0b25zCnBzbTA6IGNvbmZpZzowMDAwMDAwMCwgZmxhZ3M6MDAwMDAwMDgsIHBhY2tl dCBzaXplOjMKcHNtMDogc3luY21hc2s6YzAsIHN5bmNiaXRzOjAwCmV4X2lzYV9pZGVudGlmeSgp CmFoY19pc2FfcHJvYmUgMDogaW9wb3J0IDB4YzAwIGFsbG9jIGZhaWxlZAphaGNfaXNhX3Byb2Jl IDE6IGlvcG9ydCAweDFjMDAgYWxsb2MgZmFpbGVkCmFoY19pc2FfcHJvYmUgMjogaW9wb3J0IDB4 MmMwMCBhbGxvYyBmYWlsZWQKYWhjX2lzYV9wcm9iZSAzOiBpb3BvcnQgMHgzYzAwIGFsbG9jIGZh aWxlZAphaGNfaXNhX3Byb2JlIDQ6IGlvcG9ydCAweDRjMDAgYWxsb2MgZmFpbGVkCmFoY19pc2Ff cHJvYmUgNTogaW9wb3J0IDB4NWMwMCBhbGxvYyBmYWlsZWQKYWhjX2lzYV9wcm9iZSA2OiBpb3Bv cnQgMHg2YzAwIGFsbG9jIGZhaWxlZAphaGNfaXNhX3Byb2JlIDc6IGlvcG9ydCAweDdjMDAgYWxs b2MgZmFpbGVkCmFoY19pc2FfcHJvYmUgODogaW9wb3J0IDB4OGMwMCBhbGxvYyBmYWlsZWQKYWhj X2lzYV9wcm9iZSA5OiBpb3BvcnQgMHg5YzAwIGFsbG9jIGZhaWxlZAphaGNfaXNhX3Byb2JlIDEw OiBpb3BvcnQgMHhhYzAwIGFsbG9jIGZhaWxlZAphaGNfaXNhX3Byb2JlIDExOiBpb3BvcnQgMHhi YzAwIGFsbG9jIGZhaWxlZAphaGNfaXNhX3Byb2JlIDEyOiBpb3BvcnQgMHhjYzAwIGFsbG9jIGZh aWxlZAphaGNfaXNhX3Byb2JlIDEzOiBpb3BvcnQgMHhkYzAwIGFsbG9jIGZhaWxlZAphaGNfaXNh X3Byb2JlIDE0OiBpb3BvcnQgMHhlYzAwIGFsbG9jIGZhaWxlZApwbnBfaWRlbnRpZnk6IFRyeWlu ZyBSZWFkX1BvcnQgYXQgMjAzCnBucF9pZGVudGlmeTogVHJ5aW5nIFJlYWRfUG9ydCBhdCAyNDMK cG5wX2lkZW50aWZ5OiBUcnlpbmcgUmVhZF9Qb3J0IGF0IDI4MwpwbnBfaWRlbnRpZnk6IFRyeWlu ZyBSZWFkX1BvcnQgYXQgMmMzCnBucF9pZGVudGlmeTogVHJ5aW5nIFJlYWRfUG9ydCBhdCAzMDMK cG5wX2lkZW50aWZ5OiBUcnlpbmcgUmVhZF9Qb3J0IGF0IDM0MwpwbnBfaWRlbnRpZnk6IFRyeWlu ZyBSZWFkX1BvcnQgYXQgMzgzCnBucF9pZGVudGlmeTogVHJ5aW5nIFJlYWRfUG9ydCBhdCAzYzMK UE5QIElkZW50aWZ5IGNvbXBsZXRlCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGEwMDAwLTB4 YTA3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGEwODAw LTB4YTBmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGEx MDAwLTB4YTE3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGExODAwLTB4YTFmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGEyMDAwLTB4YTI3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSAzICgweGEyODAwLTB4YTJmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSAzICgweGEzMDAwLTB4YTM3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSAzICgweGEzODAwLTB4YTNmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSAzICgweGE0MDAwLTB4YTQ3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBh bGxvY2F0ZWQgdHlwZSAzICgweGE0ODAwLTB4YTRmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIw OiBhbGxvY2F0ZWQgdHlwZSAzICgweGE1MDAwLTB4YTU3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE1ODAwLTB4YTVmZmYpIGZvciByaWQgMCBvZiBvcm0w CnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE2MDAwLTB4YTY3ZmYpIGZvciByaWQgMCBvZiBv cm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE2ODAwLTB4YTZmZmYpIGZvciByaWQgMCBv ZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE3MDAwLTB4YTc3ZmYpIGZvciByaWQg MCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE3ODAwLTB4YTdmZmYpIGZvciBy aWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE4MDAwLTB4YTg3ZmYpIGZv ciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE4ODAwLTB4YThmZmYp IGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE5MDAwLTB4YTk3 ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGE5ODAwLTB4 YTlmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGFhMDAw LTB4YWE3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGFh ODAwLTB4YWFmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGFiMDAwLTB4YWI3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGFiODAwLTB4YWJmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSAzICgweGFjMDAwLTB4YWM3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSAzICgweGFjODAwLTB4YWNmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSAzICgweGFkMDAwLTB4YWQ3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSAzICgweGFkODAwLTB4YWRmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBh bGxvY2F0ZWQgdHlwZSAzICgweGFlMDAwLTB4YWU3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIw OiBhbGxvY2F0ZWQgdHlwZSAzICgweGFlODAwLTB4YWVmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGFmMDAwLTB4YWY3ZmYpIGZvciByaWQgMCBvZiBvcm0w CnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGFmODAwLTB4YWZmZmYpIGZvciByaWQgMCBvZiBv cm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIwMDAwLTB4YjA3ZmYpIGZvciByaWQgMCBv ZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIwODAwLTB4YjBmZmYpIGZvciByaWQg MCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIxMDAwLTB4YjE3ZmYpIGZvciBy aWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIxODAwLTB4YjFmZmYpIGZv ciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIyMDAwLTB4YjI3ZmYp IGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIyODAwLTB4YjJm ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIzMDAwLTB4 YjM3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGIzODAw LTB4YjNmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGI0 MDAwLTB4YjQ3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGI0ODAwLTB4YjRmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGI1MDAwLTB4YjU3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSAzICgweGI1ODAwLTB4YjVmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSAzICgweGI2MDAwLTB4YjY3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSAzICgweGI2ODAwLTB4YjZmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSAzICgweGI3MDAwLTB4Yjc3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBh bGxvY2F0ZWQgdHlwZSAzICgweGI3ODAwLTB4YjdmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIw OiBhbGxvY2F0ZWQgdHlwZSAzICgweGI4MDAwLTB4Yjg3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGI4ODAwLTB4YjhmZmYpIGZvciByaWQgMCBvZiBvcm0w CnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGI5MDAwLTB4Yjk3ZmYpIGZvciByaWQgMCBvZiBv cm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGI5ODAwLTB4YjlmZmYpIGZvciByaWQgMCBv ZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJhMDAwLTB4YmE3ZmYpIGZvciByaWQg MCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJhODAwLTB4YmFmZmYpIGZvciBy aWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJiMDAwLTB4YmI3ZmYpIGZv ciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJiODAwLTB4YmJmZmYp IGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJjMDAwLTB4YmM3 ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJjODAwLTB4 YmNmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJkMDAw LTB4YmQ3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGJk ODAwLTB4YmRmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGJlMDAwLTB4YmU3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGJlODAwLTB4YmVmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSAzICgweGJmMDAwLTB4YmY3ZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSAzICgweGJmODAwLTB4YmZmZmYpIGZvciByaWQgMCBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSAzICgweGQwMDAwLTB4ZDA3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSAzICgweGQwODAwLTB4ZDBmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBh bGxvY2F0ZWQgdHlwZSAzICgweGQxMDAwLTB4ZDE3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIw OiBhbGxvY2F0ZWQgdHlwZSAzICgweGQxODAwLTB4ZDFmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQyMDAwLTB4ZDI3ZmYpIGZvciByaWQgMSBvZiBvcm0w CnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQyODAwLTB4ZDJmZmYpIGZvciByaWQgMSBvZiBv cm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQzMDAwLTB4ZDM3ZmYpIGZvciByaWQgMSBv ZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQzODAwLTB4ZDNmZmYpIGZvciByaWQg MSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ0MDAwLTB4ZDQ3ZmYpIGZvciBy aWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ0ODAwLTB4ZDRmZmYpIGZv ciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ1MDAwLTB4ZDU3ZmYp IGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ1ODAwLTB4ZDVm ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ2MDAwLTB4 ZDY3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ2ODAw LTB4ZDZmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGQ3 MDAwLTB4ZDc3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgw eGQ3ODAwLTB4ZDdmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAz ICgweGQ4MDAwLTB4ZDg3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSAzICgweGQ4ODAwLTB4ZDhmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQg dHlwZSAzICgweGQ5MDAwLTB4ZDk3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0 ZWQgdHlwZSAzICgweGQ5ODAwLTB4ZDlmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSAzICgweGRhMDAwLTB4ZGE3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIwOiBh bGxvY2F0ZWQgdHlwZSAzICgweGRhODAwLTB4ZGFmZmYpIGZvciByaWQgMSBvZiBvcm0wCnBjaWIw OiBhbGxvY2F0ZWQgdHlwZSAzICgweGRiMDAwLTB4ZGI3ZmYpIGZvciByaWQgMSBvZiBvcm0wCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGRiODAwLTB4ZGJmZmYpIGZvciByaWQgMSBvZiBvcm0w CnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUwMDAwLTB4ZTA3ZmYpIGZvciByaWQgMiBvZiBv cm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUwODAwLTB4ZTBmZmYpIGZvciByaWQgMiBv ZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUxMDAwLTB4ZTE3ZmYpIGZvciByaWQg MiBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUxODAwLTB4ZTFmZmYpIGZvciBy aWQgMiBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUyMDAwLTB4ZTI3ZmYpIGZv ciByaWQgMiBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUyODAwLTB4ZTJmZmYp IGZvciByaWQgMiBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUzMDAwLTB4ZTM3 ZmYpIGZvciByaWQgMiBvZiBvcm0wCnBjaWIwOiBhbGxvY2F0ZWQgdHlwZSAzICgweGUzODAwLTB4 ZTNmZmYpIGZvciByaWQgMiBvZiBvcm0wCmlzYV9wcm9iZV9jaGlsZHJlbjogZGlzYWJsaW5nIFBu UCBkZXZpY2VzCnBtdGltZXIwIG9uIGlzYTAKYXRhOiBhdGEwIGFscmVhZHkgZXhpc3RzOyBza2lw cGluZyBpdAphdGE6IGF0YTEgYWxyZWFkeSBleGlzdHM7IHNraXBwaW5nIGl0CmF0a2JkYzogYXRr YmRjMCBhbHJlYWR5IGV4aXN0czsgc2tpcHBpbmcgaXQKYXRydGM6IGF0cnRjMCBhbHJlYWR5IGV4 aXN0czsgc2tpcHBpbmcgaXQKYXR0aW1lcjogYXR0aW1lcjAgYWxyZWFkeSBleGlzdHM7IHNraXBw aW5nIGl0CnNjOiBzYzAgYWxyZWFkeSBleGlzdHM7IHNraXBwaW5nIGl0CmlzYV9wcm9iZV9jaGls ZHJlbjogcHJvYmluZyBub24tUG5QIGRldmljZXMKb3JtMDogPElTQSBPcHRpb24gUk9Ncz4gYXQg aW9tZW0gMHhjZjAwMC0weGNmZmZmLDB4ZGY4MDAtMHhkZmZmZiBwbnBpZCBPUk0wMDAwIG9uIGlz YTAKc2MwOiA8U3lzdGVtIGNvbnNvbGU+IGF0IGZsYWdzIDB4MTAwIG9uIGlzYTAKc2MwOiBWR0Eg PDE2IHZpcnR1YWwgY29uc29sZXMsIGZsYWdzPTB4MzAwPgpzYzA6IGZiMCwga2JkMSwgdGVybWlu YWwgZW11bGF0b3I6IHNjdGVrZW4gKHRla2VuIHRlcm1pbmFsKQp2Z2EwOiA8R2VuZXJpYyBJU0Eg VkdBPiBhdCBwb3J0IDB4M2MwLTB4M2RmIGlvbWVtIDB4YTAwMDAtMHhiZmZmZiBvbiBpc2EwCnBj aWIwOiBhbGxvY2F0ZWQgdHlwZSA0ICgweDNjMC0weDNkZikgZm9yIHJpZCAwIG9mIHZnYTAKcGNp YjA6IGFsbG9jYXRlZCB0eXBlIDMgKDB4YTAwMDAtMHhiZmZmZikgZm9yIHJpZCAwIG9mIHZnYTAK cGNpYjA6IGFsbG9jYXRlZCB0eXBlIDQgKDB4M2YwLTB4M2Y1KSBmb3IgcmlkIDAgb2YgZmRjMApw Y2liMDogYWxsb2NhdGVkIHR5cGUgNCAoMHgzZjctMHgzZjcpIGZvciByaWQgMSBvZiBmZGMwCmZk YzAgZmFpbGVkIHRvIHByb2JlIGF0IHBvcnQgMHgzZjAtMHgzZjUsMHgzZjcgaXJxIDYgZHJxIDIg b24gaXNhMApwcGMwOiBwYXJhbGxlbCBwb3J0IG5vdCBmb3VuZC4KcHBjMDogPFBhcmFsbGVsIHBv cnQ+IGZhaWxlZCB0byBwcm9iZSBhdCBpcnEgNyBvbiBpc2EwCnBjaWIwOiBhbGxvY2F0ZWQgdHlw ZSA0ICgweDNmOC0weDNmZikgZm9yIHJpZCAwIG9mIHVhcnQwCnVhcnQwOiA8bnM4MjUwPiBmYWls ZWQgdG8gcHJvYmUgYXQgcG9ydCAweDNmOC0weDNmZiBpcnEgNCBvbiBpc2EwCnBjaWIwOiBhbGxv Y2F0ZWQgdHlwZSA0ICgweDJmOC0weDJmZikgZm9yIHJpZCAwIG9mIHVhcnQxCnVhcnQxOiA8bnM4 MjUwPiBmYWlsZWQgdG8gcHJvYmUgYXQgcG9ydCAweDJmOC0weDJmZiBpcnEgMyBvbiBpc2EwCmlz YV9wcm9iZV9jaGlsZHJlbjogcHJvYmluZyBQblAgZGV2aWNlcwplc3QwOiA8RW5oYW5jZWQgU3Bl ZWRTdGVwIEZyZXF1ZW5jeSBDb250cm9sPiBvbiBjcHUwCnA0dGNjMDogPENQVSBGcmVxdWVuY3kg VGhlcm1hbCBDb250cm9sPiBvbiBjcHUwCmVzdDE6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJlcXVl bmN5IENvbnRyb2w+IG9uIGNwdTEKcDR0Y2MxOiA8Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENvbnRy b2w+IG9uIGNwdTEKRGV2aWNlIGNvbmZpZ3VyYXRpb24gZmluaXNoZWQuCnByb2NmcyByZWdpc3Rl cmVkClRpbWVjb3VudGVycyB0aWNrIGV2ZXJ5IDEuMDAwIG1zZWMKdmxhbjogaW5pdGlhbGl6ZWQs IHVzaW5nIGhhc2ggdGFibGVzIHdpdGggY2hhaW5pbmcKbG8wOiBicGYgYXR0YWNoZWQKaHB0cnI6 IG5vIGNvbnRyb2xsZXIgZGV0ZWN0ZWQuCmhkYWNjMDogPFJlYWx0ZWsgQUxDODgzIEhEQSBDT0RF Qz4gYXQgY2FkIDAgb24gaGRhYzAKaGRhY2MwOiBSb290IE5vZGUgYXQgbmlkPTA6IDEgc3Vibm9k ZXMgMS0xCmhkYWEwOiA8UmVhbHRlayBBTEM4ODMgSERBIENPREVDIEF1ZGlvIEZ1bmN0aW9uIEdy b3VwPiBhdCBuaWQgMSBvbiBoZGFjYzAKaGRhYTA6IEF1ZGlvIEZ1bmN0aW9uIEdyb3VwIGF0IG5p ZD0xOiAzNyBzdWJub2RlcyAyLTM4CmhkYWEwOiBOdW1HUElPPTIgTnVtR1BPPTAgTnVtR1BJPTAg R1BJV2FrZT0wIEdQSVVuc29sPTEKaGRhYTA6ICBHUElPMDogZGlzYWJsZWQKaGRhYTA6ICBHUElP MTogZGlzYWJsZWQKaGRhYTA6IE9yaWdpbmFsIHBpbnMgY29uZmlndXJhdGlvbjoKaGRhYTA6IG5p ZCAgIDB4ICAgIGFzIHNlcSBkZXZpY2UgICAgICAgY29ubiAgamFjayAgICBsb2MgICAgICAgIGNv bG9yICAgbWlzYwpoZGFhMDogMjAgMDEyMTEwMWYgMSAgMTUgSGVhZHBob25lcyAgICBKYWNrICAx LzggICAgIFJlYXIgICAgICAgQmxhY2sgICAwCmhkYWEwOiAyMSA5OTEzMDExMCAxICAwICBTcGVh a2VyICAgICAgIEZpeGVkIEFUQVBJICAgT25ib2FyZCAgICBVbmtub3duIDEKaGRhYTA6IDIyIDQx MTExMWYwIDE1IDAgIFNwZWFrZXIgICAgICAgTm9uZSAgMS84ICAgICBSZWFyICAgICAgIEJsYWNr ICAgMQpoZGFhMDogMjMgNDExMTExZjAgMTUgMCAgU3BlYWtlciAgICAgICBOb25lICAxLzggICAg IFJlYXIgICAgICAgQmxhY2sgICAxCmhkYWEwOiAyNCAwMWExOTgzMCAzICAwICBNaWMgICAgICAg ICAgIEphY2sgIDEvOCAgICAgUmVhciAgICAgICBQaW5rICAgIDgKaGRhYTA6IDI1IDk5YTMwMTMx IDMgIDEgIE1pYyAgICAgICAgICAgRml4ZWQgQVRBUEkgICBPbmJvYXJkICAgIFVua25vd24gMQpo ZGFhMDogMjYgMDE4MTMwM2YgMyAgMTUgTGluZS1pbiAgICAgICBKYWNrICAxLzggICAgIFJlYXIg ICAgICAgQmx1ZSAgICAwCmhkYWEwOiAyNyA0MTExMTFmMCAxNSAwICBTcGVha2VyICAgICAgIE5v bmUgIDEvOCAgICAgUmVhciAgICAgICBCbGFjayAgIDEKaGRhYTA6IDI4IDQxMTExMWYwIDE1IDAg IFNwZWFrZXIgICAgICAgTm9uZSAgMS84ICAgICBSZWFyICAgICAgIEJsYWNrICAgMQpoZGFhMDog MjkgNDExMTExZjAgMTUgMCAgU3BlYWtlciAgICAgICBOb25lICAxLzggICAgIFJlYXIgICAgICAg QmxhY2sgICAxCmhkYWEwOiAzMCAwMTQ1MTEyMCAyICAwICBTUERJRi1vdXQgICAgIEphY2sgIE9w dGljYWwgUmVhciAgICAgICBCbGFjayAgIDEKaGRhYTA6IDMxIDQxMTExMWYwIDE1IDAgIFNwZWFr ZXIgICAgICAgTm9uZSAgMS84ICAgICBSZWFyICAgICAgIEJsYWNrICAgMQpoZGFhMDogUGF0Y2hp bmcgd2lkZ2V0IGNhcHMgbmlkPTI5IDB4MDA0MDAwMDAgLT4gMHgwMDcwMDAwMApoZGFhMDogUGF0 Y2hlZCBwaW5zIGNvbmZpZ3VyYXRpb246CmhkYWEwOiBuaWQgICAweCAgICBhcyBzZXEgZGV2aWNl ICAgICAgIGNvbm4gIGphY2sgICAgbG9jICAgICAgICBjb2xvciAgIG1pc2MKaGRhYTA6IDIwIDAx MjExMDFmIDEgIDE1IEhlYWRwaG9uZXMgICAgSmFjayAgMS84ICAgICBSZWFyICAgICAgIEJsYWNr ICAgMApoZGFhMDogMjEgOTkxMzAxMTAgMSAgMCAgU3BlYWtlciAgICAgICBGaXhlZCBBVEFQSSAg IE9uYm9hcmQgICAgVW5rbm93biAxCmhkYWEwOiAyMiA0MTExMTFmMCAxNSAwICBTcGVha2VyICAg ICAgIE5vbmUgIDEvOCAgICAgUmVhciAgICAgICBCbGFjayAgIDEgRElTQQpoZGFhMDogMjMgNDEx MTExZjAgMTUgMCAgU3BlYWtlciAgICAgICBOb25lICAxLzggICAgIFJlYXIgICAgICAgQmxhY2sg ICAxIERJU0EKaGRhYTA6IDI0IDAxYTE5ODMwIDMgIDAgIE1pYyAgICAgICAgICAgSmFjayAgMS84 ICAgICBSZWFyICAgICAgIFBpbmsgICAgOApoZGFhMDogMjUgOTlhMzAxMzEgMyAgMSAgTWljICAg ICAgICAgICBGaXhlZCBBVEFQSSAgIE9uYm9hcmQgICAgVW5rbm93biAxCmhkYWEwOiAyNiAwMTgx MzAzZiAzICAxNSBMaW5lLWluICAgICAgIEphY2sgIDEvOCAgICAgUmVhciAgICAgICBCbHVlICAg IDAKaGRhYTA6IDI3IDQxMTExMWYwIDE1IDAgIFNwZWFrZXIgICAgICAgTm9uZSAgMS84ICAgICBS ZWFyICAgICAgIEJsYWNrICAgMSBESVNBCmhkYWEwOiAyOCA0MTExMTFmMCAxNSAwICBTcGVha2Vy ICAgICAgIE5vbmUgIDEvOCAgICAgUmVhciAgICAgICBCbGFjayAgIDEgRElTQQpoZGFhMDogMzAg MDE0NTExMjAgMiAgMCAgU1BESUYtb3V0ICAgICBKYWNrICBPcHRpY2FsIFJlYXIgICAgICAgQmxh Y2sgICAxCmhkYWEwOiAzMSA0MTExMTFmMCAxNSAwICBTcGVha2VyICAgICAgIE5vbmUgIDEvOCAg ICAgUmVhciAgICAgICBCbGFjayAgIDEgRElTQQpoZGFhMDogMyBhc3NvY2lhdGlvbnMgZm91bmQ6 CmhkYWEwOiBBc3NvY2lhdGlvbiAwICgxKSBvdXQ6CmhkYWEwOiAgUGluIG5pZD0yMSBzZXE9MApo ZGFhMDogIFBpbiBuaWQ9MjAgc2VxPTE1CmhkYWEwOiBBc3NvY2lhdGlvbiAxICgyKSBvdXQ6Cmhk YWEwOiAgUGluIG5pZD0zMCBzZXE9MApoZGFhMDogQXNzb2NpYXRpb24gMiAoMykgaW46CmhkYWEw OiAgUGluIG5pZD0yNCBzZXE9MApoZGFhMDogIFBpbiBuaWQ9MjUgc2VxPTEKaGRhYTA6ICBQaW4g bmlkPTI2IHNlcT0xNQpoZGFhMDogVHJhY2luZyBhc3NvY2lhdGlvbiAwICgxKQpoZGFhMDogIFBp biAyMSB0cmFjZWQgdG8gREFDIDIKaGRhYTA6ICBQaW4gMjAgdHJhY2VkIHRvIERBQyAyIGFuZCBo cHJlZGlyIDAKaGRhYTA6IEFzc29jaWF0aW9uIDAgKDEpIHRyYWNlIHN1Y2NlZWRlZApoZGFhMDog VHJhY2luZyBhc3NvY2lhdGlvbiAxICgyKQpoZGFhMDogIFBpbiAzMCB0cmFjZWQgdG8gREFDIDYK aGRhYTA6IEFzc29jaWF0aW9uIDEgKDIpIHRyYWNlIHN1Y2NlZWRlZApoZGFhMDogVHJhY2luZyBh c3NvY2lhdGlvbiAyICgzKQpoZGFhMDogIFBpbiAyNCB0cmFjZWQgdG8gQURDIDgKaGRhYTA6ICBQ aW4gMjUgdHJhY2VkIHRvIEFEQyA4CmhkYWEwOiAgUGluIDI2IHRyYWNlZCB0byBBREMgOApoZGFh MDogQXNzb2NpYXRpb24gMiAoMykgdHJhY2Ugc3VjY2VlZGVkCmhkYWEwOiBMb29raW5nIGZvciBh ZGRpdGlvbmFsIERBQyBmb3IgYXNzb2NpYXRpb24gMCAoMSkKaGRhYTA6IExvb2tpbmcgZm9yIGFk ZGl0aW9uYWwgREFDIGZvciBhc3NvY2lhdGlvbiAxICgyKQpoZGFhMDogTG9va2luZyBmb3IgYWRk aXRpb25hbCBBREMgZm9yIGFzc29jaWF0aW9uIDIgKDMpCmhkYWEwOiAgQURDIDkgY29uc2lkZXJl ZCBlcXVhbCB0byBBREMgOApoZGFhMDogVHJhY2luZyBpbnB1dCBtb25pdG9yCmhkYWEwOiAgVHJh Y2luZyBuaWQgMTEgdG8gb3V0CmhkYWEwOiAgbmlkIDExIGlzIGlucHV0IG1vbml0b3IKaGRhYTA6 ICBUcmFjaW5nIG5pZCAzNCB0byBvdXQKaGRhYTA6ICBUcmFjaW5nIG5pZCAzNSB0byBvdXQKaGRh YTA6IFRyYWNpbmcgb3RoZXIgaW5wdXQgbW9uaXRvcnMKaGRhYTA6ICBUcmFjaW5nIG5pZCAyNCB0 byBvdXQKaGRhYTA6ICBUcmFjaW5nIG5pZCAyNSB0byBvdXQKaGRhYTA6ICBUcmFjaW5nIG5pZCAy NiB0byBvdXQKaGRhYTA6IFRyYWNpbmcgYmVlcGVyCmhkYWEwOiBIZWFkcGhvbmVzIHJlZGlyZWN0 aW9uIGZvciBhcz0wIG5pZD0yMCB1c2luZyB1bnNvbGljaXRlZCByZXNwb25zZXMuCmhkYWEwOiBQ aW4gc2Vuc2U6IG5pZD0yMCBzZW5jZT0weDAwMDAwMDAwIChkaXNjb25uZWN0ZWQpCmhkYWEwOiBG RyBjb25maWcvcXVpcmtzOiBmb3JjZXN0ZXJlbyBpdnJlZjUwIGl2cmVmODAgaXZyZWYxMDAgaXZy ZWYKaGRhYTA6IApoZGFhMDogKy0tLS0tLS0tLS0tLS0tLS0tLS0rCmhkYWEwOiB8IERVTVBJTkcg SERBIE5PREVTIHwKaGRhYTA6ICstLS0tLS0tLS0tLS0tLS0tLS0tKwpoZGFhMDogCmhkYWEwOiBE ZWZhdWx0IFBhcmFtZXRlcgpoZGFhMDogLS0tLS0tLS0tLS0tLS0tLS0KaGRhYTA6ICAgICAgU3Ry ZWFtIGNhcDogMHgwMDAwMDAwMQpoZGFhMDogICAgICAgICAgICAgICAgICBQQ00KaGRhYTA6ICAg ICAgICAgUENNIGNhcDogMHgwMDBlMDU2MApoZGFhMDogICAgICAgICAgICAgICAgICAxNiAyMCAy NCBiaXRzLCA0NCA0OCA5NiAxOTIgS0h6CmhkYWEwOiAgICAgICAgICBJTiBhbXA6IDB4MDAwMDAw MDAKaGRhYTA6ICAgICAgICAgT1VUIGFtcDogMHgwMDAwMDAwMApoZGFhMDogCmhkYWEwOiAgICAg ICAgICAgICBuaWQ6IDIKaGRhYTA6ICAgICAgICAgICAgTmFtZTogYXVkaW8gb3V0cHV0CmhkYWEw OiAgICAgIFdpZGdldCBjYXA6IDB4MDAwMDAwMTEKaGRhYTA6ICAgICAgICAgICAgICAgICAgU1RF UkVPCmhkYWEwOiAgICAgQXNzb2NpYXRpb246IDAgKDB4MDAwMDgwMDEpCmhkYWEwOiAgICAgICAg ICAgICBPU1M6IHBjbSAocGNtKQpoZGFhMDogICAgICBTdHJlYW0gY2FwOiAweDAwMDAwMDAxCmhk YWEwOiAgICAgICAgICAgICAgICAgIFBDTQpoZGFhMDogICAgICAgICBQQ00gY2FwOiAweDAwMGUw NTYwCmhkYWEwOiAgICAgICAgICAgICAgICAgIDE2IDIwIDI0IGJpdHMsIDQ0IDQ4IDk2IDE5MiBL SHoKaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAzIFtESVNBQkxFRF0KaGRhYTA6ICAg ICAgICAgICAgTmFtZTogYXVkaW8gb3V0cHV0CmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDAw MDAwMTEKaGRhYTA6ICAgICAgICAgICAgICAgICAgU1RFUkVPCmhkYWEwOiAgICAgIFN0cmVhbSBj YXA6IDB4MDAwMDAwMDEKaGRhYTA6ICAgICAgICAgICAgICAgICAgUENNCmhkYWEwOiAgICAgICAg IFBDTSBjYXA6IDB4MDAwZTA1NjAKaGRhYTA6ICAgICAgICAgICAgICAgICAgMTYgMjAgMjQgYml0 cywgNDQgNDggOTYgMTkyIEtIegpoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDQgW0RJ U0FCTEVEXQpoZGFhMDogICAgICAgICAgICBOYW1lOiBhdWRpbyBvdXRwdXQKaGRhYTA6ICAgICAg V2lkZ2V0IGNhcDogMHgwMDAwMDAxMQpoZGFhMDogICAgICAgICAgICAgICAgICBTVEVSRU8KaGRh YTA6ICAgICAgU3RyZWFtIGNhcDogMHgwMDAwMDAwMQpoZGFhMDogICAgICAgICAgICAgICAgICBQ Q00KaGRhYTA6ICAgICAgICAgUENNIGNhcDogMHgwMDBlMDU2MApoZGFhMDogICAgICAgICAgICAg ICAgICAxNiAyMCAyNCBiaXRzLCA0NCA0OCA5NiAxOTIgS0h6CmhkYWEwOiAKaGRhYTA6ICAgICAg ICAgICAgIG5pZDogNSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IGF1ZGlvIG91 dHB1dApoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwMDAwMDExCmhkYWEwOiAgICAgICAgICAg ICAgICAgIFNURVJFTwpoZGFhMDogICAgICBTdHJlYW0gY2FwOiAweDAwMDAwMDAxCmhkYWEwOiAg ICAgICAgICAgICAgICAgIFBDTQpoZGFhMDogICAgICAgICBQQ00gY2FwOiAweDAwMGUwNTYwCmhk YWEwOiAgICAgICAgICAgICAgICAgIDE2IDIwIDI0IGJpdHMsIDQ0IDQ4IDk2IDE5MiBLSHoKaGRh YTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiA2CmhkYWEwOiAgICAgICAgICAgIE5hbWU6IGF1 ZGlvIG91dHB1dApoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwMDAwMjExCmhkYWEwOiAgICAg ICAgICAgICAgICAgIERJR0lUQUwgU1RFUkVPCmhkYWEwOiAgICAgQXNzb2NpYXRpb246IDEgKDB4 MDAwMDAwMDEpCmhkYWEwOiAgICAgICAgICAgICBPU1M6IHBjbSAocGNtKQpoZGFhMDogICAgICBT dHJlYW0gY2FwOiAweDAwMDAwMDAxCmhkYWEwOiAgICAgICAgICAgICAgICAgIFBDTQpoZGFhMDog ICAgICAgICBQQ00gY2FwOiAweDAwMWUwNTYwCmhkYWEwOiAgICAgICAgICAgICAgICAgIDE2IDIw IDI0IDMyIGJpdHMsIDQ0IDQ4IDk2IDE5MiBLSHoKaGRhYTA6IApoZGFhMDogICAgICAgICAgICAg bmlkOiA3IFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICAgTmFtZTogdmVuZG9yIHdpZGdldApo ZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwZjAwMDAwCmhkYWEwOiAKaGRhYTA6ICAgICAgICAg ICAgIG5pZDogOApoZGFhMDogICAgICAgICAgICBOYW1lOiBhdWRpbyBpbnB1dApoZGFhMDogICAg ICBXaWRnZXQgY2FwOiAweDAwMTAwMTFiCmhkYWEwOiAgICAgICAgICAgICAgICAgIFNURVJFTwpo ZGFhMDogICAgIEFzc29jaWF0aW9uOiAyICgweDAwMDA4MDAzKQpoZGFhMDogICAgICBTdHJlYW0g Y2FwOiAweDAwMDAwMDAxCmhkYWEwOiAgICAgICAgICAgICAgICAgIFBDTQpoZGFhMDogICAgICAg ICBQQ00gY2FwOiAweDAwMDYwMTYwCmhkYWEwOiAgICAgICAgICAgICAgICAgIDE2IDIwIGJpdHMs IDQ0IDQ4IDk2IEtIegpoZGFhMDogICAgICAgSW5wdXQgYW1wOiAweDgwMDUxZjA4CmhkYWEwOiAg ICAgICAgICAgICAgICAgIG11dGU9MSBzdGVwPTMxIHNpemU9NSBvZmZzZXQ9OApoZGFhMDogICAg IGNvbm5lY3Rpb25zOiAxCmhkYWEwOiAgICAgICAgICAgfApoZGFhMDogICAgICAgICAgICsgPC0g bmlkPTM1IFthdWRpbyBtaXhlcl0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiA5Cmhk YWEwOiAgICAgICAgICAgIE5hbWU6IGF1ZGlvIGlucHV0CmhkYWEwOiAgICAgIFdpZGdldCBjYXA6 IDB4MDAxMDAxMWIKaGRhYTA6ICAgICAgICAgICAgICAgICAgU1RFUkVPCmhkYWEwOiAgICAgQXNz b2NpYXRpb246IDIgKDB4MDAwMDgwMDMpCmhkYWEwOiAgICAgIFN0cmVhbSBjYXA6IDB4MDAwMDAw MDEKaGRhYTA6ICAgICAgICAgICAgICAgICAgUENNCmhkYWEwOiAgICAgICAgIFBDTSBjYXA6IDB4 MDAwNjAxNjAKaGRhYTA6ICAgICAgICAgICAgICAgICAgMTYgMjAgYml0cywgNDQgNDggOTYgS0h6 CmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4ODAwNTFmMDgKaGRhYTA6ICAgICAgICAgICAgICAg ICAgbXV0ZT0xIHN0ZXA9MzEgc2l6ZT01IG9mZnNldD04CmhkYWEwOiAgICAgY29ubmVjdGlvbnM6 IDEKaGRhYTA6ICAgICAgICAgICB8CmhkYWEwOiAgICAgICAgICAgKyA8LSBuaWQ9MzQgW2F1ZGlv IG1peGVyXQpoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDEwIFtESVNBQkxFRF0KaGRh YTA6ICAgICAgICAgICAgTmFtZTogYXVkaW8gaW5wdXQKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDog MHgwMDEwMDM5MQpoZGFhMDogICAgICAgICAgICAgICAgICBESUdJVEFMIFVOU09MIFNURVJFTwpo ZGFhMDogICAgICBTdHJlYW0gY2FwOiAweDAwMDAwMDAxCmhkYWEwOiAgICAgICAgICAgICAgICAg IFBDTQpoZGFhMDogICAgICAgICBQQ00gY2FwOiAweDAwMWUwNTYwCmhkYWEwOiAgICAgICAgICAg ICAgICAgIDE2IDIwIDI0IDMyIGJpdHMsIDQ0IDQ4IDk2IDE5MiBLSHoKaGRhYTA6ICAgICBjb25u ZWN0aW9uczogMQpoZGFhMDogICAgICAgICAgIHwKaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxF RF0gPC0gbmlkPTMxIFtwaW46IFNwZWFrZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEwOiAKaGRh YTA6ICAgICAgICAgICAgIG5pZDogMTEKaGRhYTA6ICAgICAgICAgICAgTmFtZTogYXVkaW8gbWl4 ZXIKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDIwMDEwYgpoZGFhMDogICAgICAgICAgICAg ICAgICBTVEVSRU8KaGRhYTA6ICAgICBBc3NvY2lhdGlvbjogMiAoMHgwMDAwODAwMykKaGRhYTA6 ICAgICAgICAgICAgIE9TUzogbWl4IChtaXgpCmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4ODAw NTFmMTcKaGRhYTA6ICAgICAgICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MzEgc2l6ZT01IG9mZnNl dD0yMwpoZGFhMDogICAgIGNvbm5lY3Rpb25zOiAxMApoZGFhMDogICAgICAgICAgIHwKaGRhYTA6 ICAgICAgICAgICArIDwtIG5pZD0yNCBbcGluOiBNaWMgKFBpbmsgSmFjayldCmhkYWEwOiAgICAg ICAgICAgKyA8LSBuaWQ9MjUgW3BpbjogTWljIChGaXhlZCldCmhkYWEwOiAgICAgICAgICAgKyA8 LSBuaWQ9MjYgW3BpbjogTGluZS1pbiAoQmx1ZSBKYWNrKV0KaGRhYTA6ICAgICAgICAgICArIFtE SVNBQkxFRF0gPC0gbmlkPTI3IFtwaW46IFNwZWFrZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEw OiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0yOCBbcGluOiBTcGVha2VyIChOb25lKV0g W0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTI5IFtiZWVwIHdpZGdldF0KaGRh YTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTIwIFtwaW46IEhlYWRwaG9uZXMgKEJs YWNrIEphY2spXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MjEgW3Bpbjog U3BlYWtlciAoRml4ZWQpXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MjIg W3BpbjogU3BlYWtlciAoTm9uZSldIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNB QkxFRF0gPC0gbmlkPTIzIFtwaW46IFNwZWFrZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEwOiAK aGRhYTA6ICAgICAgICAgICAgIG5pZDogMTIKaGRhYTA6ICAgICAgICAgICAgTmFtZTogYXVkaW8g bWl4ZXIKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDIwMDEwZgpoZGFhMDogICAgICAgICAg ICAgICAgICBTVEVSRU8KaGRhYTA6ICAgICBBc3NvY2lhdGlvbjogMCAoMHgwMDAwODAwMSkKaGRh YTA6ICAgICAgICAgICAgIE9TUzogcGNtLCBtaXgKaGRhYTA6ICAgICAgT3V0cHV0IGFtcDogMHgw MDA1MWYxZgpoZGFhMDogICAgICAgICAgICAgICAgICBtdXRlPTAgc3RlcD0zMSBzaXplPTUgb2Zm c2V0PTMxCmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4ODAwMDAwMDAKaGRhYTA6ICAgICAgICAg ICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zmc2V0PTAKaGRhYTA6ICAgICBjb25uZWN0 aW9uczogMgpoZGFhMDogICAgICAgICAgIHwKaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0yIFth dWRpbyBvdXRwdXRdCmhkYWEwOiAgICAgICAgICAgKyA8LSBuaWQ9MTEgW2F1ZGlvIG1peGVyXQpo ZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDEzIFtESVNBQkxFRF0KaGRhYTA6ICAgICAg ICAgICAgTmFtZTogYXVkaW8gbWl4ZXIKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDIwMDEw ZgpoZGFhMDogICAgICAgICAgICAgICAgICBTVEVSRU8KaGRhYTA6ICAgICBBc3NvY2lhdGlvbjog LTIgKDB4MDAwMDAwMDApCmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4MDAwNTFmMWYKaGRhYTA6 ICAgICAgICAgICAgICAgICAgbXV0ZT0wIHN0ZXA9MzEgc2l6ZT01IG9mZnNldD0zMQpoZGFhMDog ICAgICAgSW5wdXQgYW1wOiAweDgwMDAwMDAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9 MSBzdGVwPTAgc2l6ZT0wIG9mZnNldD0wCmhkYWEwOiAgICAgY29ubmVjdGlvbnM6IDIKaGRhYTA6 ICAgICAgICAgICB8CmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0zIFthdWRp byBvdXRwdXRdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlk PTExIFthdWRpbyBtaXhlcl0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAxNCBbRElT QUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IGF1ZGlvIG1peGVyCmhkYWEwOiAgICAgIFdp ZGdldCBjYXA6IDB4MDAyMDAxMGYKaGRhYTA6ICAgICAgICAgICAgICAgICAgU1RFUkVPCmhkYWEw OiAgICAgQXNzb2NpYXRpb246IC0yICgweDAwMDAwMDAwKQpoZGFhMDogICAgICBPdXRwdXQgYW1w OiAweDAwMDUxZjFmCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9MCBzdGVwPTMxIHNpemU9 NSBvZmZzZXQ9MzEKaGRhYTA6ICAgICAgIElucHV0IGFtcDogMHg4MDAwMDAwMApoZGFhMDogICAg ICAgICAgICAgICAgICBtdXRlPTEgc3RlcD0wIHNpemU9MCBvZmZzZXQ9MApoZGFhMDogICAgIGNv bm5lY3Rpb25zOiAyCmhkYWEwOiAgICAgICAgICAgfApoZGFhMDogICAgICAgICAgICsgW0RJU0FC TEVEXSA8LSBuaWQ9NCBbYXVkaW8gb3V0cHV0XSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAg KyBbRElTQUJMRURdIDwtIG5pZD0xMSBbYXVkaW8gbWl4ZXJdCmhkYWEwOiAKaGRhYTA6ICAgICAg ICAgICAgIG5pZDogMTUgW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICBOYW1lOiBhdWRpbyBt aXhlcgpoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwMjAwMTBmCmhkYWEwOiAgICAgICAgICAg ICAgICAgIFNURVJFTwpoZGFhMDogICAgIEFzc29jaWF0aW9uOiAtMiAoMHgwMDAwMDAwMCkKaGRh YTA6ICAgICAgT3V0cHV0IGFtcDogMHgwMDA1MWYxZgpoZGFhMDogICAgICAgICAgICAgICAgICBt dXRlPTAgc3RlcD0zMSBzaXplPTUgb2Zmc2V0PTMxCmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4 ODAwMDAwMDAKaGRhYTA6ICAgICAgICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zm c2V0PTAKaGRhYTA6ICAgICBjb25uZWN0aW9uczogMgpoZGFhMDogICAgICAgICAgIHwKaGRhYTA6 ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTUgW2F1ZGlvIG91dHB1dF0gW0RJU0FCTEVE XQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTEgW2F1ZGlvIG1peGVyXQpo ZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDE2IFtESVNBQkxFRF0KaGRhYTA6ICAgICAg ICAgICAgTmFtZTogdmVuZG9yIHdpZGdldApoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwZjAw MDAwCmhkYWEwOiAKaGRhYTA6ICAgICAgICAgICAgIG5pZDogMTcgW0RJU0FCTEVEXQpoZGFhMDog ICAgICAgICAgICBOYW1lOiB2ZW5kb3Igd2lkZ2V0CmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4 MDBmMDAwMDAKaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAxOCBbRElTQUJMRURdCmhk YWEwOiAgICAgICAgICAgIE5hbWU6IHZlbmRvciB3aWRnZXQKaGRhYTA6ICAgICAgV2lkZ2V0IGNh cDogMHgwMGYwMDAwMApoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDE5IFtESVNBQkxF RF0KaGRhYTA6ICAgICAgICAgICAgTmFtZTogdmVuZG9yIHdpZGdldApoZGFhMDogICAgICBXaWRn ZXQgY2FwOiAweDAwZjAwMDAwCmhkYWEwOiAKaGRhYTA6ICAgICAgICAgICAgIG5pZDogMjAKaGRh YTA6ICAgICAgICAgICAgTmFtZTogcGluOiBIZWFkcGhvbmVzIChCbGFjayBKYWNrKQpoZGFhMDog ICAgICBXaWRnZXQgY2FwOiAweDAwNDAwMThmCmhkYWEwOiAgICAgICAgICAgICAgICAgIFVOU09M IFNURVJFTwpoZGFhMDogICAgIEFzc29jaWF0aW9uOiAwICgweDAwMDA4MDAwKQpoZGFhMDogICAg ICAgICBQaW4gY2FwOiAweDAwMDAwMDNlCmhkYWEwOiAgICAgICAgICAgICAgICAgIFRSUUQgUERD IEhQIE9VVCBJTgpoZGFhMDogICAgICBQaW4gY29uZmlnOiAweDAxMjExMDFmCmhkYWEwOiAgICAg UGluIGNvbnRyb2w6IDB4MDAwMDAwYzAgSFAgT1VUCmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4 ODAwMDAwMDAKaGRhYTA6ICAgICAgICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zm c2V0PTAKaGRhYTA6ICAgICAgIElucHV0IGFtcDogMHgwMDI3MDMwMApoZGFhMDogICAgICAgICAg ICAgICAgICBtdXRlPTAgc3RlcD0zIHNpemU9Mzkgb2Zmc2V0PTAKaGRhYTA6ICAgICBjb25uZWN0 aW9uczogNQpoZGFhMDogICAgICAgICAgIHwKaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0xMiBb YXVkaW8gbWl4ZXJdIChzZWxlY3RlZCkKaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0g bmlkPTEzIFthdWRpbyBtaXhlcl0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FC TEVEXSA8LSBuaWQ9MTQgW2F1ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAg KyBbRElTQUJMRURdIDwtIG5pZD0xNSBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAg ICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTM4IFthdWRpbyBtaXhlcl0gW0RJU0FCTEVEXQpo ZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDIxCmhkYWEwOiAgICAgICAgICAgIE5hbWU6 IHBpbjogU3BlYWtlciAoRml4ZWQpCmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDA0MDAxOGYK aGRhYTA6ICAgICAgICAgICAgICAgICAgVU5TT0wgU1RFUkVPCmhkYWEwOiAgICAgQXNzb2NpYXRp b246IDAgKDB4MDAwMDAwMDEpCmhkYWEwOiAgICAgICAgIFBpbiBjYXA6IDB4MDAwMDAwM2UKaGRh YTA6ICAgICAgICAgICAgICAgICAgVFJRRCBQREMgSFAgT1VUIElOCmhkYWEwOiAgICAgIFBpbiBj b25maWc6IDB4OTkxMzAxMTAKaGRhYTA6ICAgICBQaW4gY29udHJvbDogMHgwMDAwMDA0MCBPVVQK aGRhYTA6ICAgICAgT3V0cHV0IGFtcDogMHg4MDAwMDAwMApoZGFhMDogICAgICAgICAgICAgICAg ICBtdXRlPTEgc3RlcD0wIHNpemU9MCBvZmZzZXQ9MApoZGFhMDogICAgICAgSW5wdXQgYW1wOiAw eDAwMjcwMzAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9MCBzdGVwPTMgc2l6ZT0zOSBv ZmZzZXQ9MApoZGFhMDogICAgIGNvbm5lY3Rpb25zOiA1CmhkYWEwOiAgICAgICAgICAgfApoZGFh MDogICAgICAgICAgICsgPC0gbmlkPTEyIFthdWRpbyBtaXhlcl0gKHNlbGVjdGVkKQpoZGFhMDog ICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTMgW2F1ZGlvIG1peGVyXSBbRElTQUJMRURd CmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0xNCBbYXVkaW8gbWl4ZXJdIFtE SVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTE1IFthdWRpbyBt aXhlcl0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9Mzgg W2F1ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAKaGRhYTA6ICAgICAgICAgICAgIG5pZDog MjIgW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICBOYW1lOiBwaW46IFNwZWFrZXIgKE5vbmUp CmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDA0MDAxOGYKaGRhYTA6ICAgICAgICAgICAgICAg ICAgVU5TT0wgU1RFUkVPCmhkYWEwOiAgICAgICAgIFBpbiBjYXA6IDB4MDAwMDAwM2UKaGRhYTA6 ICAgICAgICAgICAgICAgICAgVFJRRCBQREMgSFAgT1VUIElOCmhkYWEwOiAgICAgIFBpbiBjb25m aWc6IDB4NDExMTExZjAKaGRhYTA6ICAgICBQaW4gY29udHJvbDogMHgwMDAwMDAwMApoZGFhMDog ICAgICBPdXRwdXQgYW1wOiAweDgwMDAwMDAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9 MSBzdGVwPTAgc2l6ZT0wIG9mZnNldD0wCmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4MDAyNzAz MDAKaGRhYTA6ICAgICAgICAgICAgICAgICAgbXV0ZT0wIHN0ZXA9MyBzaXplPTM5IG9mZnNldD0w CmhkYWEwOiAgICAgY29ubmVjdGlvbnM6IDUKaGRhYTA6ICAgICAgICAgICB8CmhkYWEwOiAgICAg ICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0xMiBbYXVkaW8gbWl4ZXJdIChzZWxlY3RlZCkKaGRh YTA6ICAgICAgICAgICArIDwtIG5pZD0xMyBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6 ICAgICAgICAgICArIDwtIG5pZD0xNCBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAg ICAgICAgICArIDwtIG5pZD0xNSBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAg ICAgICArIDwtIG5pZD0zOCBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6IApoZGFhMDog ICAgICAgICAgICAgbmlkOiAyMyBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IHBp bjogU3BlYWtlciAoTm9uZSkKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDQwMDE4ZgpoZGFh MDogICAgICAgICAgICAgICAgICBVTlNPTCBTVEVSRU8KaGRhYTA6ICAgICAgICAgUGluIGNhcDog MHgwMDAwMDAzZQpoZGFhMDogICAgICAgICAgICAgICAgICBUUlFEIFBEQyBIUCBPVVQgSU4KaGRh YTA6ICAgICAgUGluIGNvbmZpZzogMHg0MTExMTFmMApoZGFhMDogICAgIFBpbiBjb250cm9sOiAw eDAwMDAwMDAwCmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4ODAwMDAwMDAKaGRhYTA6ICAgICAg ICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zmc2V0PTAKaGRhYTA6ICAgICAgIElu cHV0IGFtcDogMHgwMDI3MDMwMApoZGFhMDogICAgICAgICAgICAgICAgICBtdXRlPTAgc3RlcD0z IHNpemU9Mzkgb2Zmc2V0PTAKaGRhYTA6ICAgICBjb25uZWN0aW9uczogNQpoZGFhMDogICAgICAg ICAgIHwKaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTEyIFthdWRpbyBtaXhl cl0gKHNlbGVjdGVkKQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTEzIFthdWRpbyBtaXhlcl0g W0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTE0IFthdWRpbyBtaXhlcl0gW0RJ U0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTE1IFthdWRpbyBtaXhlcl0gW0RJU0FC TEVEXQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTM4IFthdWRpbyBtaXhlcl0gW0RJU0FCTEVE XQpoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDI0CmhkYWEwOiAgICAgICAgICAgIE5h bWU6IHBpbjogTWljIChQaW5rIEphY2spCmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDA0MDAx OGYKaGRhYTA6ICAgICAgICAgICAgICAgICAgVU5TT0wgU1RFUkVPCmhkYWEwOiAgICAgQXNzb2Np YXRpb246IDIgKDB4MDAwMDAwMDEpCmhkYWEwOiAgICAgICAgICAgICBPU1M6IG1pYyAobWljKQpo ZGFhMDogICAgICAgICBQaW4gY2FwOiAweDAwMDAxNzNlCmhkYWEwOiAgICAgICAgICAgICAgICAg IFRSUUQgUERDIEhQIE9VVCBJTiBWUkVGWyA1MCA4MCBHUk9VTkQgSElaIF0KaGRhYTA6ICAgICAg UGluIGNvbmZpZzogMHgwMWExOTgzMApoZGFhMDogICAgIFBpbiBjb250cm9sOiAweDAwMDAwMDI0 IElOIFZSRUZzCmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4ODAwMDAwMDAKaGRhYTA6ICAgICAg ICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zmc2V0PTAKaGRhYTA6ICAgICAgIElu cHV0IGFtcDogMHgwMDI3MDMwMApoZGFhMDogICAgICAgICAgICAgICAgICBtdXRlPTAgc3RlcD0z IHNpemU9Mzkgb2Zmc2V0PTAKaGRhYTA6ICAgICBjb25uZWN0aW9uczogNQpoZGFhMDogICAgICAg ICAgIHwKaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTEyIFthdWRpbyBtaXhl cl0gKHNlbGVjdGVkKQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTMgW2F1 ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5p ZD0xNCBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxF RF0gPC0gbmlkPTE1IFthdWRpbyBtaXhlcl0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsg W0RJU0FCTEVEXSA8LSBuaWQ9MzggW2F1ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAKaGRh YTA6ICAgICAgICAgICAgIG5pZDogMjUKaGRhYTA6ICAgICAgICAgICAgTmFtZTogcGluOiBNaWMg KEZpeGVkKQpoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwNDAwMThmCmhkYWEwOiAgICAgICAg ICAgICAgICAgIFVOU09MIFNURVJFTwpoZGFhMDogICAgIEFzc29jaWF0aW9uOiAyICgweDAwMDAw MDAyKQpoZGFhMDogICAgICAgICAgICAgT1NTOiBtb25pdG9yIChtb25pdG9yKQpoZGFhMDogICAg ICAgICBQaW4gY2FwOiAweDAwMDAxNzNlCmhkYWEwOiAgICAgICAgICAgICAgICAgIFRSUUQgUERD IEhQIE9VVCBJTiBWUkVGWyA1MCA4MCBHUk9VTkQgSElaIF0KaGRhYTA6ICAgICAgUGluIGNvbmZp ZzogMHg5OWEzMDEzMQpoZGFhMDogICAgIFBpbiBjb250cm9sOiAweDAwMDAwMDI0IElOIFZSRUZz CmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4ODAwMDAwMDAKaGRhYTA6ICAgICAgICAgICAgICAg ICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zmc2V0PTAKaGRhYTA6ICAgICAgIElucHV0IGFtcDog MHgwMDI3MDMwMApoZGFhMDogICAgICAgICAgICAgICAgICBtdXRlPTAgc3RlcD0zIHNpemU9Mzkg b2Zmc2V0PTAKaGRhYTA6ICAgICBjb25uZWN0aW9uczogNQpoZGFhMDogICAgICAgICAgIHwKaGRh YTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTEyIFthdWRpbyBtaXhlcl0gKHNlbGVj dGVkKQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTMgW2F1ZGlvIG1peGVy XSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0xNCBbYXVk aW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlk PTE1IFthdWRpbyBtaXhlcl0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVE XSA8LSBuaWQ9MzggW2F1ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAKaGRhYTA6ICAgICAg ICAgICAgIG5pZDogMjYKaGRhYTA6ICAgICAgICAgICAgTmFtZTogcGluOiBMaW5lLWluIChCbHVl IEphY2spCmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDA0MDAxOGYKaGRhYTA6ICAgICAgICAg ICAgICAgICAgVU5TT0wgU1RFUkVPCmhkYWEwOiAgICAgQXNzb2NpYXRpb246IDIgKDB4MDAwMDgw MDApCmhkYWEwOiAgICAgICAgICAgICBPU1M6IGxpbmUgKGxpbmUpCmhkYWEwOiAgICAgICAgIFBp biBjYXA6IDB4MDAwMDE3M2UKaGRhYTA6ICAgICAgICAgICAgICAgICAgVFJRRCBQREMgSFAgT1VU IElOIFZSRUZbIDUwIDgwIEdST1VORCBISVogXQpoZGFhMDogICAgICBQaW4gY29uZmlnOiAweDAx ODEzMDNmCmhkYWEwOiAgICAgUGluIGNvbnRyb2w6IDB4MDAwMDAwMjQgSU4gVlJFRnMKaGRhYTA6 ICAgICAgT3V0cHV0IGFtcDogMHg4MDAwMDAwMApoZGFhMDogICAgICAgICAgICAgICAgICBtdXRl PTEgc3RlcD0wIHNpemU9MCBvZmZzZXQ9MApoZGFhMDogICAgICAgSW5wdXQgYW1wOiAweDAwMjcw MzAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9MCBzdGVwPTMgc2l6ZT0zOSBvZmZzZXQ9 MApoZGFhMDogICAgIGNvbm5lY3Rpb25zOiA1CmhkYWEwOiAgICAgICAgICAgfApoZGFhMDogICAg ICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTIgW2F1ZGlvIG1peGVyXSAoc2VsZWN0ZWQpCmhk YWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0xMyBbYXVkaW8gbWl4ZXJdIFtESVNB QkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTE0IFthdWRpbyBtaXhl cl0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MTUgW2F1 ZGlvIG1peGVyXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5p ZD0zOCBbYXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAg bmlkOiAyNyBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IHBpbjogU3BlYWtlciAo Tm9uZSkKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDQwMDE4ZgpoZGFhMDogICAgICAgICAg ICAgICAgICBVTlNPTCBTVEVSRU8KaGRhYTA6ICAgICAgICAgUGluIGNhcDogMHgwMDAwMTczZQpo ZGFhMDogICAgICAgICAgICAgICAgICBUUlFEIFBEQyBIUCBPVVQgSU4gVlJFRlsgNTAgODAgR1JP VU5EIEhJWiBdCmhkYWEwOiAgICAgIFBpbiBjb25maWc6IDB4NDExMTExZjAKaGRhYTA6ICAgICBQ aW4gY29udHJvbDogMHgwMDAwMDAwMApoZGFhMDogICAgICBPdXRwdXQgYW1wOiAweDgwMDAwMDAw CmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9MSBzdGVwPTAgc2l6ZT0wIG9mZnNldD0wCmhk YWEwOiAgICAgICBJbnB1dCBhbXA6IDB4MDAyNzAzMDAKaGRhYTA6ICAgICAgICAgICAgICAgICAg bXV0ZT0wIHN0ZXA9MyBzaXplPTM5IG9mZnNldD0wCmhkYWEwOiAgICAgY29ubmVjdGlvbnM6IDUK aGRhYTA6ICAgICAgICAgICB8CmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0x MiBbYXVkaW8gbWl4ZXJdIChzZWxlY3RlZCkKaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0xMyBb YXVkaW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0xNCBbYXVk aW8gbWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0xNSBbYXVkaW8g bWl4ZXJdIFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0zOCBbYXVkaW8gbWl4 ZXJdIFtESVNBQkxFRF0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAyOCBbRElTQUJM RURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IHBpbjogU3BlYWtlciAoTm9uZSkKaGRhYTA6ICAg ICAgV2lkZ2V0IGNhcDogMHgwMDQwMDAwMQpoZGFhMDogICAgICAgICAgICAgICAgICBTVEVSRU8K aGRhYTA6ICAgICAgICAgUGluIGNhcDogMHgwMDAwMDAyMApoZGFhMDogICAgICAgICAgICAgICAg ICBJTgpoZGFhMDogICAgICBQaW4gY29uZmlnOiAweDQxMTExMWYwCmhkYWEwOiAgICAgUGluIGNv bnRyb2w6IDB4MDAwMDAwMDAKaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAyOQpoZGFh MDogICAgICAgICAgICBOYW1lOiBiZWVwIHdpZGdldApoZGFhMDogICAgICBXaWRnZXQgY2FwOiAw eDAwNzAwMDAwCmhkYWEwOiAgICAgQXNzb2NpYXRpb246IC0yICgweDAwMDAwMDAwKQpoZGFhMDog ICAgICAgICAgICAgT1NTOiBzcGVha2VyIChzcGVha2VyKQpoZGFhMDogICAgICAgICBQaW4gY2Fw OiAweDAwMDAwMDIwCmhkYWEwOiAgICAgICAgICAgICAgICAgIElOCmhkYWEwOiAgICAgIFBpbiBj b25maWc6IDB4NDExMTExZjAKaGRhYTA6ICAgICBQaW4gY29udHJvbDogMHgwMDAwMDAyMCBJTgpo ZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDMwCmhkYWEwOiAgICAgICAgICAgIE5hbWU6 IHBpbjogU1BESUYtb3V0IChCbGFjayBKYWNrKQpoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAw NDAwMzAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIERJR0lUQUwKaGRhYTA6ICAgICBBc3NvY2lh dGlvbjogMSAoMHgwMDAwMDAwMSkKaGRhYTA6ICAgICAgICAgUGluIGNhcDogMHgwMDAwMDAxMApo ZGFhMDogICAgICAgICAgICAgICAgICBPVVQKaGRhYTA6ICAgICAgUGluIGNvbmZpZzogMHgwMTQ1 MTEyMApoZGFhMDogICAgIFBpbiBjb250cm9sOiAweDAwMDAwMDQwIE9VVApoZGFhMDogICAgIGNv bm5lY3Rpb25zOiAxCmhkYWEwOiAgICAgICAgICAgfApoZGFhMDogICAgICAgICAgICsgPC0gbmlk PTYgW2F1ZGlvIG91dHB1dF0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAzMSBbRElT QUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IHBpbjogU3BlYWtlciAoTm9uZSkKaGRhYTA6 ICAgICAgV2lkZ2V0IGNhcDogMHgwMDQwMDIwMApoZGFhMDogICAgICAgICAgICAgICAgICBESUdJ VEFMCmhkYWEwOiAgICAgICAgIFBpbiBjYXA6IDB4MDAwMDAwMjAKaGRhYTA6ICAgICAgICAgICAg ICAgICAgSU4KaGRhYTA6ICAgICAgUGluIGNvbmZpZzogMHg0MTExMTFmMApoZGFhMDogICAgIFBp biBjb250cm9sOiAweDAwMDAwMDAwCmhkYWEwOiAKaGRhYTA6ICAgICAgICAgICAgIG5pZDogMzIg W0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICBOYW1lOiB2ZW5kb3Igd2lkZ2V0CmhkYWEwOiAg ICAgIFdpZGdldCBjYXA6IDB4MDBmMDAwNDAKaGRhYTA6ICAgICAgICAgICAgICAgICAgUFJPQwpo ZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDMzIFtESVNBQkxFRF0KaGRhYTA6ICAgICAg ICAgICAgTmFtZTogdmVuZG9yIHdpZGdldApoZGFhMDogICAgICBXaWRnZXQgY2FwOiAweDAwZjAw MDAwCmhkYWEwOiAKaGRhYTA6ICAgICAgICAgICAgIG5pZDogMzQKaGRhYTA6ICAgICAgICAgICAg TmFtZTogYXVkaW8gbWl4ZXIKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDIwMDEwZgpoZGFh MDogICAgICAgICAgICAgICAgICBTVEVSRU8KaGRhYTA6ICAgICBBc3NvY2lhdGlvbjogMiAoMHgw MDAwODAwMykKaGRhYTA6ICAgICAgICAgICAgIE9TUzogc3BlYWtlciwgbGluZSwgbWljLCBtaXgs IG1vbml0b3IKaGRhYTA6ICAgICAgIElucHV0IGFtcDogMHg4MDAwMDAwMApoZGFhMDogICAgICAg ICAgICAgICAgICBtdXRlPTEgc3RlcD0wIHNpemU9MCBvZmZzZXQ9MApoZGFhMDogICAgIGNvbm5l Y3Rpb25zOiAxMQpoZGFhMDogICAgICAgICAgIHwKaGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0y NCBbcGluOiBNaWMgKFBpbmsgSmFjayldCmhkYWEwOiAgICAgICAgICAgKyA8LSBuaWQ9MjUgW3Bp bjogTWljIChGaXhlZCldCmhkYWEwOiAgICAgICAgICAgKyA8LSBuaWQ9MjYgW3BpbjogTGluZS1p biAoQmx1ZSBKYWNrKV0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTI3IFtw aW46IFNwZWFrZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJM RURdIDwtIG5pZD0yOCBbcGluOiBTcGVha2VyIChOb25lKV0gW0RJU0FCTEVEXQpoZGFhMDogICAg ICAgICAgICsgPC0gbmlkPTI5IFtiZWVwIHdpZGdldF0KaGRhYTA6ICAgICAgICAgICArIFtESVNB QkxFRF0gPC0gbmlkPTIwIFtwaW46IEhlYWRwaG9uZXMgKEJsYWNrIEphY2spXQpoZGFhMDogICAg ICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MjEgW3BpbjogU3BlYWtlciAoRml4ZWQpXQpoZGFh MDogICAgICAgICAgICsgW0RJU0FCTEVEXSA8LSBuaWQ9MjIgW3BpbjogU3BlYWtlciAoTm9uZSld IFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTIzIFtwaW46 IFNwZWFrZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyA8LSBuaWQ9MTEg W2F1ZGlvIG1peGVyXQpoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBuaWQ6IDM1CmhkYWEwOiAg ICAgICAgICAgIE5hbWU6IGF1ZGlvIG1peGVyCmhkYWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDAy MDAxMGYKaGRhYTA6ICAgICAgICAgICAgICAgICAgU1RFUkVPCmhkYWEwOiAgICAgQXNzb2NpYXRp b246IDIgKDB4MDAwMDgwMDMpCmhkYWEwOiAgICAgICAgICAgICBPU1M6IHNwZWFrZXIsIGxpbmUs IG1pYywgbWl4LCBtb25pdG9yCmhkYWEwOiAgICAgICBJbnB1dCBhbXA6IDB4ODAwMDAwMDAKaGRh YTA6ICAgICAgICAgICAgICAgICAgbXV0ZT0xIHN0ZXA9MCBzaXplPTAgb2Zmc2V0PTAKaGRhYTA6 ICAgICBjb25uZWN0aW9uczogMTEKaGRhYTA6ICAgICAgICAgICB8CmhkYWEwOiAgICAgICAgICAg KyA8LSBuaWQ9MjQgW3BpbjogTWljIChQaW5rIEphY2spXQpoZGFhMDogICAgICAgICAgICsgPC0g bmlkPTI1IFtwaW46IE1pYyAoRml4ZWQpXQpoZGFhMDogICAgICAgICAgICsgPC0gbmlkPTI2IFtw aW46IExpbmUtaW4gKEJsdWUgSmFjayldCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwt IG5pZD0yNyBbcGluOiBTcGVha2VyIChOb25lKV0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAg ICsgW0RJU0FCTEVEXSA8LSBuaWQ9MjggW3BpbjogU3BlYWtlciAoTm9uZSldIFtESVNBQkxFRF0K aGRhYTA6ICAgICAgICAgICArIDwtIG5pZD0yOSBbYmVlcCB3aWRnZXRdCmhkYWEwOiAgICAgICAg ICAgKyBbRElTQUJMRURdIDwtIG5pZD0yMCBbcGluOiBIZWFkcGhvbmVzIChCbGFjayBKYWNrKV0K aGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTIxIFtwaW46IFNwZWFrZXIgKEZp eGVkKV0KaGRhYTA6ICAgICAgICAgICArIFtESVNBQkxFRF0gPC0gbmlkPTIyIFtwaW46IFNwZWFr ZXIgKE5vbmUpXSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5p ZD0yMyBbcGluOiBTcGVha2VyIChOb25lKV0gW0RJU0FCTEVEXQpoZGFhMDogICAgICAgICAgICsg PC0gbmlkPTExIFthdWRpbyBtaXhlcl0KaGRhYTA6IApoZGFhMDogICAgICAgICAgICAgbmlkOiAz NiBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgIE5hbWU6IHZlbmRvciB3aWRnZXQKaGRhYTA6 ICAgICAgV2lkZ2V0IGNhcDogMHgwMGYwMDAwMApoZGFhMDogCmhkYWEwOiAgICAgICAgICAgICBu aWQ6IDM3IFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICAgTmFtZTogYXVkaW8gb3V0cHV0Cmhk YWEwOiAgICAgIFdpZGdldCBjYXA6IDB4MDAwMDAwMTEKaGRhYTA6ICAgICAgICAgICAgICAgICAg U1RFUkVPCmhkYWEwOiAgICAgIFN0cmVhbSBjYXA6IDB4MDAwMDAwMDEKaGRhYTA6ICAgICAgICAg ICAgICAgICAgUENNCmhkYWEwOiAgICAgICAgIFBDTSBjYXA6IDB4MDAwZTA1NjAKaGRhYTA6ICAg ICAgICAgICAgICAgICAgMTYgMjAgMjQgYml0cywgNDQgNDggOTYgMTkyIEtIegpoZGFhMDogCmhk YWEwOiAgICAgICAgICAgICBuaWQ6IDM4IFtESVNBQkxFRF0KaGRhYTA6ICAgICAgICAgICAgTmFt ZTogYXVkaW8gbWl4ZXIKaGRhYTA6ICAgICAgV2lkZ2V0IGNhcDogMHgwMDIwMDEwZgpoZGFhMDog ICAgICAgICAgICAgICAgICBTVEVSRU8KaGRhYTA6ICAgICBBc3NvY2lhdGlvbjogLTIgKDB4MDAw MDAwMDApCmhkYWEwOiAgICAgIE91dHB1dCBhbXA6IDB4MDAwNTFmMWYKaGRhYTA6ICAgICAgICAg ICAgICAgICAgbXV0ZT0wIHN0ZXA9MzEgc2l6ZT01IG9mZnNldD0zMQpoZGFhMDogICAgICAgSW5w dXQgYW1wOiAweDgwMDAwMDAwCmhkYWEwOiAgICAgICAgICAgICAgICAgIG11dGU9MSBzdGVwPTAg c2l6ZT0wIG9mZnNldD0wCmhkYWEwOiAgICAgY29ubmVjdGlvbnM6IDIKaGRhYTA6ICAgICAgICAg ICB8CmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0zNyBbYXVkaW8gb3V0cHV0 XSBbRElTQUJMRURdCmhkYWEwOiAgICAgICAgICAgKyBbRElTQUJMRURdIDwtIG5pZD0xMSBbYXVk aW8gbWl4ZXJdCmhkYWEwOiAKcGNtMDogPFJlYWx0ZWsgQUxDODgzIEhEQSBDT0RFQyBQQ00gKEFu YWxvZyk+IGF0IG5pZCAyMSwyMCBhbmQgMjQsMjUsMjYgb24gaGRhYTAKcGNtMDogKy0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKwpwY20wOiB8IERVTVBJTkcgUENNIFBsYXli YWNrL1JlY29yZCBDaGFubmVscyB8CnBjbTA6ICstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLSsKcGNtMDogCnBjbTA6IFBsYXliYWNrOgpwY20wOiAKcGNtMDogICAgICBTdHJl YW0gY2FwOiAweDAwMDAwMDAxCnBjbTA6ICAgICAgICAgICAgICAgICAgUENNCnBjbTA6ICAgICAg ICAgUENNIGNhcDogMHgwMDBlMDU2MApwY20wOiAgICAgICAgICAgICAgICAgIDE2IDIwIDI0IGJp dHMsIDQ0IDQ4IDk2IDE5MiBLSHoKcGNtMDogICAgICAgICAgICAgREFDOiAyCnBjbTA6IApwY20w OiBSZWNvcmQ6CnBjbTA6IApwY20wOiAgICAgIFN0cmVhbSBjYXA6IDB4MDAwMDAwMDEKcGNtMDog ICAgICAgICAgICAgICAgICBQQ00KcGNtMDogICAgICAgICBQQ00gY2FwOiAweDAwMDYwMTYwCnBj bTA6ICAgICAgICAgICAgICAgICAgMTYgMjAgYml0cywgNDQgNDggOTYgS0h6CnBjbTA6ICAgICAg ICAgICAgIERBQzogOApwY20wOiAgICAgICAgICAgICBEQUM6IDkKcGNtMDogCnBjbTA6ICstLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKwpwY20wOiB8IERVTVBJTkcgUGxheWJhY2svUmVj b3JkIFBhdGhzIHwKcGNtMDogKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rCnBjbTA6 IApwY20wOiBQbGF5YmFjazoKcGNtMDogCnBjbTA6ICAgICBuaWQ9MjEgW3BpbjogU3BlYWtlciAo Rml4ZWQpXQpwY20wOiAgICAgICB8CnBjbTA6ICAgICAgICsgPC0gbmlkPTEyIFthdWRpbyBtaXhl cl0gW3NyYzogcGNtLCBtaXhdCnBjbTA6ICAgICAgICAgICAgICB8CnBjbTA6ICAgICAgICAgICAg ICArIDwtIG5pZD0yIFthdWRpbyBvdXRwdXRdIFtzcmM6IHBjbV0KcGNtMDogICAgICAgICAgICAg ICsgPC0gbmlkPTExIFthdWRpbyBtaXhlcl0gW3NyYzogbWl4XQpwY20wOiAKcGNtMDogICAgIG5p ZD0yMCBbcGluOiBIZWFkcGhvbmVzIChCbGFjayBKYWNrKV0KcGNtMDogICAgICAgfApwY20wOiAg ICAgICArIDwtIG5pZD0xMiBbYXVkaW8gbWl4ZXJdIFtzcmM6IHBjbSwgbWl4XQpwY20wOiAgICAg ICAgICAgICAgfApwY20wOiAgICAgICAgICAgICAgKyA8LSBuaWQ9MiBbYXVkaW8gb3V0cHV0XSBb c3JjOiBwY21dCnBjbTA6ICAgICAgICAgICAgICArIDwtIG5pZD0xMSBbYXVkaW8gbWl4ZXJdIFtz cmM6IG1peF0KcGNtMDogCnBjbTA6IFJlY29yZDoKcGNtMDogCnBjbTA6ICAgICBuaWQ9OCBbYXVk aW8gaW5wdXRdCnBjbTA6ICAgICAgIHwKcGNtMDogICAgICAgKyA8LSBuaWQ9MzUgW2F1ZGlvIG1p eGVyXSBbc3JjOiBzcGVha2VyLCBsaW5lLCBtaWMsIG1peCwgbW9uaXRvcl0KcGNtMDogICAgICAg ICAgICAgIHwKcGNtMDogICAgICAgICAgICAgICsgPC0gbmlkPTI0IFtwaW46IE1pYyAoUGluayBK YWNrKV0gW3NyYzogbWljXQpwY20wOiAgICAgICAgICAgICAgKyA8LSBuaWQ9MjUgW3BpbjogTWlj IChGaXhlZCldIFtzcmM6IG1vbml0b3JdCnBjbTA6ICAgICAgICAgICAgICArIDwtIG5pZD0yNiBb cGluOiBMaW5lLWluIChCbHVlIEphY2spXSBbc3JjOiBsaW5lXQpwY20wOiAgICAgICAgICAgICAg KyA8LSBuaWQ9MjkgW2JlZXAgd2lkZ2V0XSBbc3JjOiBzcGVha2VyXQpwY20wOiAgICAgICAgICAg ICAgKyA8LSBuaWQ9MTEgW2F1ZGlvIG1peGVyXSBbc3JjOiBtaXhdCnBjbTA6IApwY20wOiAgICAg bmlkPTkgW2F1ZGlvIGlucHV0XQpwY20wOiAgICAgICB8CnBjbTA6ICAgICAgICsgPC0gbmlkPTM0 IFthdWRpbyBtaXhlcl0gW3NyYzogc3BlYWtlciwgbGluZSwgbWljLCBtaXgsIG1vbml0b3JdCnBj bTA6ICAgICAgICAgICAgICB8CnBjbTA6ICAgICAgICAgICAgICArIDwtIG5pZD0yNCBbcGluOiBN aWMgKFBpbmsgSmFjayldIFtzcmM6IG1pY10KcGNtMDogICAgICAgICAgICAgICsgPC0gbmlkPTI1 IFtwaW46IE1pYyAoRml4ZWQpXSBbc3JjOiBtb25pdG9yXQpwY20wOiAgICAgICAgICAgICAgKyA8 LSBuaWQ9MjYgW3BpbjogTGluZS1pbiAoQmx1ZSBKYWNrKV0gW3NyYzogbGluZV0KcGNtMDogICAg ICAgICAgICAgICsgPC0gbmlkPTI5IFtiZWVwIHdpZGdldF0gW3NyYzogc3BlYWtlcl0KcGNtMDog ICAgICAgICAgICAgICsgPC0gbmlkPTExIFthdWRpbyBtaXhlcl0gW3NyYzogbWl4XQpwY20wOiAK cGNtMDogSW5wdXQgTWl4OgpwY20wOiAKcGNtMDogICAgIG5pZD0xMSBbYXVkaW8gbWl4ZXJdCnBj bTA6ICAgICAgIHwKcGNtMDogICAgICAgKyA8LSBuaWQ9MjQgW3BpbjogTWljIChQaW5rIEphY2sp XSBbc3JjOiBtaWNdCnBjbTA6ICAgICAgICsgPC0gbmlkPTI1IFtwaW46IE1pYyAoRml4ZWQpXSBb c3JjOiBtb25pdG9yXQpwY20wOiAgICAgICArIDwtIG5pZD0yNiBbcGluOiBMaW5lLWluIChCbHVl IEphY2spXSBbc3JjOiBsaW5lXQpwY20wOiAgICAgICArIDwtIG5pZD0yOSBbYmVlcCB3aWRnZXRd IFtzcmM6IHNwZWFrZXJdCnBjbTA6IApwY20wOiArLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSsK cGNtMDogfCBEVU1QSU5HIFZvbHVtZSBDb250cm9scyB8CnBjbTA6ICstLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tKwpwY20wOiAKcGNtMDogTWFzdGVyIFZvbHVtZSAoT1NTOiB2b2wpCnBjbTA6ICAg IHwKcGNtMDogICAgKy0gY3RsIDEzIChuaWQgIDEyIG91dCk6ICAgIC00Ni8wZEIgKDMyIHN0ZXBz KQpwY20wOiAgICArLSBjdGwgMTQgKG5pZCAgMTIgaW4gICAwKTogbXV0ZQpwY20wOiAgICArLSBj dGwgMTUgKG5pZCAgMTIgaW4gICAxKTogbXV0ZQpwY20wOiAgICArLSBjdGwgMjUgKG5pZCAgMjAg aW4gKTogICAgbXV0ZQpwY20wOiAgICArLSBjdGwgMjcgKG5pZCAgMjEgaW4gKTogICAgbXV0ZQpw Y20wOiAKcGNtMDogUENNIFZvbHVtZSAoT1NTOiBwY20pCnBjbTA6ICAgIHwKcGNtMDogICAgKy0g Y3RsIDE0IChuaWQgIDEyIGluICAgMCk6IG11dGUKcGNtMDogCnBjbTA6IE1pY3JvcGhvbmUgVm9s dW1lIChPU1M6IG1pYykKcGNtMDogICAgfApwY20wOiAgICArLSBjdGwgMzQgKG5pZCAgMjQgb3V0 KTogICAgMC8zMGRCICg0IHN0ZXBzKQpwY20wOiAgICArLSBjdGwgNDEgKG5pZCAgMzQgaW4gICAw KTogbXV0ZQpwY20wOiAgICArLSBjdGwgNTIgKG5pZCAgMzUgaW4gICAwKTogbXV0ZQpwY20wOiAK cGNtMDogTWljcm9waG9uZTIgVm9sdW1lIChPU1M6IG1vbml0b3IpCnBjbTA6ICAgIHwKcGNtMDog ICAgKy0gY3RsIDM2IChuaWQgIDI1IG91dCk6ICAgIDAvMzBkQiAoNCBzdGVwcykKcGNtMDogICAg Ky0gY3RsIDQyIChuaWQgIDM0IGluICAgMSk6IG11dGUKcGNtMDogICAgKy0gY3RsIDUzIChuaWQg IDM1IGluICAgMSk6IG11dGUKcGNtMDogCnBjbTA6IExpbmUtaW4gVm9sdW1lIChPU1M6IGxpbmUp CnBjbTA6ICAgIHwKcGNtMDogICAgKy0gY3RsIDM4IChuaWQgIDI2IG91dCk6ICAgIDAvMzBkQiAo NCBzdGVwcykKcGNtMDogICAgKy0gY3RsIDQzIChuaWQgIDM0IGluICAgMik6IG11dGUKcGNtMDog ICAgKy0gY3RsIDU0IChuaWQgIDM1IGluICAgMik6IG11dGUKcGNtMDogCnBjbTA6IFNwZWFrZXIv QmVlcCBWb2x1bWUgKE9TUzogc3BlYWtlcikKcGNtMDogICAgfApwY20wOiAgICArLSBjdGwgIDgg KG5pZCAgMTEgaW4gICA1KTogLTM0LzEyZEIgKDMyIHN0ZXBzKSArIG11dGUKcGNtMDogICAgKy0g Y3RsIDQ2IChuaWQgIDM0IGluICAgNSk6IG11dGUKcGNtMDogICAgKy0gY3RsIDU3IChuaWQgIDM1 IGluICAgNSk6IG11dGUKcGNtMDogCnBjbTA6IFJlY29yZGluZyBMZXZlbCAoT1NTOiByZWMpCnBj bTA6ICAgIHwKcGNtMDogICAgKy0gY3RsICAxIChuaWQgICA4IGluICAgMCk6IC0xMi8zNGRCICgz MiBzdGVwcykgKyBtdXRlCnBjbTA6ICAgICstIGN0bCAgMiAobmlkICAgOSBpbiAgIDApOiAtMTIv MzRkQiAoMzIgc3RlcHMpICsgbXV0ZQpwY20wOiAgICArLSBjdGwgNDEgKG5pZCAgMzQgaW4gICAw KTogbXV0ZQpwY20wOiAgICArLSBjdGwgNDIgKG5pZCAgMzQgaW4gICAxKTogbXV0ZQpwY20wOiAg ICArLSBjdGwgNDMgKG5pZCAgMzQgaW4gICAyKTogbXV0ZQpwY20wOiAgICArLSBjdGwgNDYgKG5p ZCAgMzQgaW4gICA1KTogbXV0ZQpwY20wOiAgICArLSBjdGwgNTEgKG5pZCAgMzQgaW4gIDEwKTog bXV0ZQpwY20wOiAgICArLSBjdGwgNTIgKG5pZCAgMzUgaW4gICAwKTogbXV0ZQpwY20wOiAgICAr LSBjdGwgNTMgKG5pZCAgMzUgaW4gICAxKTogbXV0ZQpwY20wOiAgICArLSBjdGwgNTQgKG5pZCAg MzUgaW4gICAyKTogbXV0ZQpwY20wOiAgICArLSBjdGwgNTcgKG5pZCAgMzUgaW4gICA1KTogbXV0 ZQpwY20wOiAgICArLSBjdGwgNjIgKG5pZCAgMzUgaW4gIDEwKTogbXV0ZQpwY20wOiAKcGNtMDog SW5wdXQgTWl4IExldmVsIChPU1M6IG1peCkKcGNtMDogICAgfApwY20wOiAgICArLSBjdGwgIDMg KG5pZCAgMTEgaW4gICAwKTogLTM0LzEyZEIgKDMyIHN0ZXBzKSArIG11dGUKcGNtMDogICAgKy0g Y3RsICA0IChuaWQgIDExIGluICAgMSk6IC0zNC8xMmRCICgzMiBzdGVwcykgKyBtdXRlCnBjbTA6 ICAgICstIGN0bCAgNSAobmlkICAxMSBpbiAgIDIpOiAtMzQvMTJkQiAoMzIgc3RlcHMpICsgbXV0 ZQpwY20wOiAgICArLSBjdGwgIDggKG5pZCAgMTEgaW4gICA1KTogLTM0LzEyZEIgKDMyIHN0ZXBz KSArIG11dGUKcGNtMDogICAgKy0gY3RsIDE1IChuaWQgIDEyIGluICAgMSk6IG11dGUKcGNtMDog ICAgKy0gY3RsIDUxIChuaWQgIDM0IGluICAxMCk6IG11dGUKcGNtMDogICAgKy0gY3RsIDYyIChu aWQgIDM1IGluICAxMCk6IG11dGUKcGNtMDogCnBjbTA6IElucHV0IE1vbml0b3JpbmcgTGV2ZWwg KE9TUzogaWdhaW4pCnBjbTA6ICAgIHwKcGNtMDogICAgKy0gY3RsIDE1IChuaWQgIDEyIGluICAg MSk6IG11dGUKcGNtMDogCnBjbTA6IEVuYWJsaW5nIFNvZnQgUENNIHZvbHVtZQpwY20wOiBNaXhl ciAidm9sIjoKcGNtMDogTWl4ZXIgInBjbSI6CnBjbTA6IE1peGVyICJzcGVha2VyIjoKcGNtMDog TWl4ZXIgImxpbmUiOgpwY20wOiBNaXhlciAibWljIjoKcGNtMDogTWl4ZXIgIm1peCI6CnBjbTA6 IE1peGVyICJyZWMiOgpwY20wOiBNaXhlciAiaWdhaW4iOgpwY20wOiBNaXhlciAibW9uaXRvciI6 CnBjbTA6IFNvZnQgUENNIG1peGVyIEVOQUJMRUQKcGNtMDogY2xvbmUgbWFuYWdlcjogZGVhZGxp bmU9NzUwbXMgZmxhZ3M9MHg4MDAwMDAxZQpwY20wOiBzbmRidWZfc2V0bWFwIDQ0NjMwMDAsIDQw MDA7IDB4ZWMwNjMwMDAgLT4gNDQ2MzAwMApwY20wOiBzbmRidWZfc2V0bWFwIDQ0NzMwMDAsIDQw MDA7IDB4ZWMwNzMwMDAgLT4gNDQ3MzAwMApwY20wOiBzbmRidWZfc2V0bWFwIDQ0ODMwMDAsIDQw MDA7IDB4ZWMwODMwMDAgLT4gNDQ4MzAwMApwY20xOiA8UmVhbHRlayBBTEM4ODMgSERBIENPREVD IFBDTSAoUmVhciBEaWdpdGFsKT4gYXQgbmlkIDMwIG9uIGhkYWEwCnBjbTE6ICstLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSsKcGNtMTogfCBEVU1QSU5HIFBDTSBQbGF5YmFj ay9SZWNvcmQgQ2hhbm5lbHMgfApwY20xOiArLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0rCnBjbTE6IApwY20xOiBQbGF5YmFjazoKcGNtMTogCnBjbTE6ICAgICAgU3RyZWFt IGNhcDogMHgwMDAwMDAwNQpwY20xOiAgICAgICAgICAgICAgICAgIEFDMyBQQ00KcGNtMTogICAg ICAgICBQQ00gY2FwOiAweDAwMWUwNTYwCnBjbTE6ICAgICAgICAgICAgICAgICAgMTYgMjAgMjQg MzIgYml0cywgNDQgNDggOTYgMTkyIEtIegpwY20xOiAgICAgICAgICAgICBEQUM6IDYKcGNtMTog CnBjbTE6ICstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKwpwY20xOiB8IERVTVBJTkcg UGxheWJhY2svUmVjb3JkIFBhdGhzIHwKcGNtMTogKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0rCnBjbTE6IApwY20xOiBQbGF5YmFjazoKcGNtMTogCnBjbTE6ICAgICBuaWQ9MzAgW3Bp bjogU1BESUYtb3V0IChCbGFjayBKYWNrKV0KcGNtMTogICAgICAgfApwY20xOiAgICAgICArIDwt IG5pZD02IFthdWRpbyBvdXRwdXRdIFtzcmM6IHBjbV0KcGNtMTogCnBjbTE6ICstLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tKwpwY20xOiB8IERVTVBJTkcgVm9sdW1lIENvbnRyb2xzIHwKcGNtMTog Ky0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rCnBjbTE6IApwY20xOiBGb3JjaW5nIFNvZnQgUENN IHZvbHVtZQpwY20xOiBGb3JjaW5nIG1hc3RlciB2b2x1bWUgd2l0aCBQQ00KcGNtMTogTWl4ZXIg InZvbCIgLT4gIm5vbmUiOiBjaGlsZD0weDAwMDAwMDEwCnBjbTE6IE1peGVyICJwY20iOiBwYXJl bnQ9InZvbCIKcGNtMTogU29mdCBQQ00gbWl4ZXIgRU5BQkxFRApwY20xOiBjbG9uZSBtYW5hZ2Vy OiBkZWFkbGluZT03NTBtcyBmbGFncz0weDgwMDAwMDFlCnBjbTE6IHNuZGJ1Zl9zZXRtYXAgNDQ5 MzAwMCwgNDAwMDsgMHhlYzA5MzAwMCAtPiA0NDkzMDAwCmhkYWNjMTogPEx1Y2VudC9BZ2VyZSBT eXN0ZW1zICgweDEwNDApIEhEQSBDT0RFQz4gYXQgY2FkIDEgb24gaGRhYzAKaGRhY2MxOiBSb290 IE5vZGUgYXQgbmlkPTA6IDEgc3Vibm9kZXMgMS0xCnVua25vd246IDxMdWNlbnQvQWdlcmUgU3lz dGVtcyAoMHgxMDQwKSBIREEgQ09ERUMgTW9kZW0gRnVuY3Rpb24gR3JvdXA+IGF0IG5pZCAxIG9u IGhkYWNjMSAobm8gZHJpdmVyIGF0dGFjaGVkKQp1c2J1czA6IDEyTWJwcyBGdWxsIFNwZWVkIFVT QiB2MS4wCnVzYnVzMTogMTJNYnBzIEZ1bGwgU3BlZWQgVVNCIHYxLjAKdXNidXMyOiAxMk1icHMg RnVsbCBTcGVlZCBVU0IgdjEuMAp1c2J1czM6IDEyTWJwcyBGdWxsIFNwZWVkIFVTQiB2MS4wCnVz YnVzNDogNDgwTWJwcyBIaWdoIFNwZWVkIFVTQiB2Mi4wCnVnZW4wLjE6IDxJbnRlbD4gYXQgdXNi dXMwCnVodWIwOiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMS4wMC8xLjAw LCBhZGRyIDE+IG9uIHVzYnVzMAp1Z2VuMS4xOiA8SW50ZWw+IGF0IHVzYnVzMQp1aHViMTogPElu dGVsIFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1 c2J1czEKdWdlbjIuMTogPEludGVsPiBhdCB1c2J1czIKdWh1YjI6IDxJbnRlbCBVSENJIHJvb3Qg SFVCLCBjbGFzcyA5LzAsIHJldiAxLjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXMyCnVnZW4zLjE6 IDxJbnRlbD4gYXQgdXNidXMzCnVodWIzOiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8w LCByZXYgMS4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMwp1Z2VuNC4xOiA8SW50ZWw+IGF0IHVz YnVzNAp1aHViNDogPEludGVsIEVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAvMS4w MCwgYWRkciAxPiBvbiB1c2J1czQKdWh1YjA6IDIgcG9ydHMgd2l0aCAyIHJlbW92YWJsZSwgc2Vs ZiBwb3dlcmVkCnVodWIxOiAyIHBvcnRzIHdpdGggMiByZW1vdmFibGUsIHNlbGYgcG93ZXJlZAp1 aHViMjogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQKdWh1YjM6IDIgcG9y dHMgd2l0aCAyIHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkCmF0YTA6IFNBVEEgcmVzZXQ6IHBvcnRz IHN0YXR1cz0weDAxCmF0YTA6IHJlc2V0IHRwMSBtYXNrPTAzIG9zdGF0MD01MCBvc3RhdDE9MDAK YXRhMDogc3RhdDA9MHg1MCBlcnI9MHgwMSBsc2I9MHgwMCBtc2I9MHgwMAphdGEwOiBzdGF0MT0w eDAwIGVycj0weDAxIGxzYj0weDAwIG1zYj0weDAwCmF0YTA6IHJlc2V0IHRwMiBzdGF0MD01MCBz dGF0MT0wMCBkZXZpY2VzPTB4MQooYXByb2JlMDphdGEwOjA6MDowKTogU0lHTkFUVVJFOiAwMDAw CmF0YTE6IHJlc2V0IHRwMSBtYXNrPTAzIG9zdGF0MD01MCBvc3RhdDE9MDEKYXRhMTogc3RhdDA9 MHgxMCBlcnI9MHgwMSBsc2I9MHgxNCBtc2I9MHhlYgphdGExOiBzdGF0MT0weDAxIGVycj0weDA0 IGxzYj0weDAwIG1zYj0weDAwCmF0YTE6IHJlc2V0IHRwMiBzdGF0MD0xMCBzdGF0MT0wMSBkZXZp Y2VzPTB4MTAwMDAKKGFwcm9iZTE6YXRhMTowOjA6MCk6IFNJR05BVFVSRTogZWIxNAphY3BpX2Fj YWQwOiBhY2xpbmUgaW5pdGlhbGl6YXRpb24gc3RhcnQKYmF0dGVyeTA6IGJhdHRlcnkgaW5pdGlh bGl6YXRpb24gc3RhcnQKYWNwaV9hY2FkMDogT24gTGluZQphY3BpX2FjYWQwOiBhY2xpbmUgaW5p dGlhbGl6YXRpb24gZG9uZSwgdHJpZWQgMSB0aW1lcwp1aHViNDogOCBwb3J0cyB3aXRoIDggcmVt b3ZhYmxlLCBzZWxmIHBvd2VyZWQKKHByb2JlMDpjdGwyY2FtMDowOjE6MCk6IEVycm9yIDYsIFVu cmV0cnlhYmxlIGVycm9yCkdFT006IG5ldyBkaXNrIGNkMAphZGEwIGF0IGF0YTAgYnVzIDAgc2Ni dXMwIHRhcmdldCAwIGx1biAwCmFkYTA6IChjZDA6YXRhMTowOjA6MCk6IFNDU0kgc3RhdHVzIGVy cm9yCihjZDA6YXRhMTowOjA6MCk6IFJFQUQgQ0FQQUNJVFkuIENEQjogMjUgMCAwIDAgMCAwIDAg MCAwIDAgCihjZDA6YXRhMTowOjA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihj ZDA6YXRhMTowOjA6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGNkMDphdGExOjA6 MDowKTogU0NTSSBzZW5zZTogTk9UIFJFQURZIGFzYzozYSwxIChNZWRpdW0gbm90IHByZXNlbnQg LSB0cmF5IGNsb3NlZCkKKGNkMDphdGExOjA6MDowKTogRXJyb3IgNiwgVW5yZXRyeWFibGUgZXJy b3IKY2QwIGF0IGF0YTEgYnVzIDAgc2NidXMxIHRhcmdldCAwIGx1biAwCmNkMDogPE9wdGlhcmMg RFZEIFJXIEFELTc1MzBBIEVYMzE+IFJlbW92YWJsZSBDRC1ST00gU0NTSS0wIGRldmljZSAKY2Qw OiBTZXJpYWwgTnVtYmVyIDMwNjQ1NDAwIDE0NDY3NDJRMTExCmNkMDogMzMuMzAwTUIvcyB0cmFu c2ZlcnMgKFVETUEyLCBBVEFQSSAxMmJ5dGVzLCBQSU8gNjU1MzRieXRlcykKY2QwOiBBdHRlbXB0 IHRvIHF1ZXJ5IGRldmljZSBzaXplIGZhaWxlZDogTk9UIFJFQURZLCBNZWRpdW0gbm90IHByZXNl bnQgLSB0cmF5IGNsb3NlZAo8U1Q5MTYwODIxQVMgMy5BTEQ+IEFUQS03IFNBVEEgMS54IGRldmlj ZQphZGEwOiBTZXJpYWwgTnVtYmVyIDVNQTMyQ1BOCmFkYTA6IDE1MC4wMDBNQi9zIHRyYW5zZmVy cyAoU0FUQSwgVURNQTUsIFBJTyA4MTkyYnl0ZXMpCmFkYTA6IDE1MjYyN01CICgzMTI1ODE4MDgg NTEyIGJ5dGUgc2VjdG9yczogMTZIIDYzUy9UIDE2MzgzQykKYWRhMDogUHJldmlvdXNseSB3YXMg a25vd24gYXMgYWQwCihjZDA6YXRhMTowOjA6MCk6IFNDU0kgc3RhdHVzIGVycm9yCihjZDA6YXRh MTowOjA6MCk6IFJFQUQgQ0FQQUNJVFkuIENEQjogMjUgMCAwIDAgMCAwIDAgMCAwIDAgCihjZDA6 YXRhMTowOjA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihjZDA6YXRhMTowOjA6 MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGNkMDphdGExOjA6MDowKTogU0NTSSBz ZW5zZTogTk9UIFJFQURZIGFzYzozYSwxIChNZWRpdW0gbm90IHByZXNlbnQgLSB0cmF5IGNsb3Nl ZCkKKGNkMDphdGExOjA6MDowKTogRXJyb3IgNiwgVW5yZXRyeWFibGUgZXJyb3IKcGFzczAgYXQg YXRhMCBidXMgMCBzY2J1czAgdGFyZ2V0IDAgbHVuIDAKcGFzczA6IDxTVDkxNjA4MjFBUyAzLkFM RD4gQVRBLTcgU0FUQSAxLnggZGV2aWNlCnBhc3MwOiBTZXJpYWwgTnVtYmVyIDVNQTMyQ1BOCnBh c3MwOiAxNTAuMDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEsIFVETUE1LCBQSU8gODE5MmJ5dGVzKQpw YXNzMSBhdCBhdGExIGJ1cyAwIHNjYnVzMSB0YXJnZXQgMCBsdW4gMApwYXNzMTogPE9wdGlhcmMg RFZEIFJXIEFELTc1MzBBIEVYMzE+IFJlbW92YWJsZSBDRC1ST00gU0NTSS0wIGRldmljZSAKcGFz czE6IFNlcmlhbCBOdW1iZXIgMzA2NDU0MDAgMTQ0Njc0MlExMTEKcGFzczE6IDMzLjMwME1CL3Mg dHJhbnNmZXJzIChVRE1BMiwgQVRBUEkgMTJieXRlcywgUElPIDY1NTM0Ynl0ZXMpCihjZDA6YXRh MTowOjA6MCk6IFNDU0kgc3RhdHVzIGVycm9yCihjZDA6YXRhMTowOjA6MCk6IFJFQUQgQ0FQQUNJ VFkuIENEQjogMjUgMCAwIDAgMCAwIDAgMCAwIDAgCihjZDA6YXRhMTowOjA6MCk6IENBTSBzdGF0 dXM6IFNDU0kgU3RhdHVzIEVycm9yCihjZDA6YXRhMTowOjA6MCk6IFNDU0kgc3RhdHVzOiBDaGVj ayBDb25kaXRpb24KKGNkMDphdGExOjA6MDowKTogU0NTSSBzZW5zZTogTk9UIFJFQURZIGFzYzoz YSwxIChNZWRpdW0gbm90IHByZXNlbnQgLSB0cmF5IGNsb3NlZCkKKGNkMDphdGExOjA6MDowKTog RXJyb3IgNiwgVW5yZXRyeWFibGUgZXJyb3IKU01QOiBBUCBDUFUgIzEgTGF1bmNoZWQhCmNwdTEg QVA6CiAgICAgSUQ6IDB4MDEwMDAwMDAgICBWRVI6IDB4MDAwNTAwMTQgTERSOiAweDAwMDAwMDAw IERGUjogMHhmZmZmZmZmZgogIGxpbnQwOiAweDAwMDEwNzAwIGxpbnQxOiAweDAwMDAwNDAwIFRQ UjogMHgwMDAwMDAwMCBTVlI6IDB4MDAwMDAxZmYKICB0aW1lcjogMHgwMDAxMDBlZiB0aGVybTog MHgwMDAxMDAwMCBlcnI6IDB4MDAwMDAwZjAgcG1jOiAweDAwMDEwNDAwClRTQyB0aW1lY291bnRl ciBkaXNhYmxlZDogQzMgZW5hYmxlZC4KVGltZWNvdW50ZXIgIlRTQyIgZnJlcXVlbmN5IDE3MzM0 Mzk2OTUgSHogcXVhbGl0eSAtMTAwMApXQVJOSU5HOiBXSVRORVNTIG9wdGlvbiBlbmFibGVkLCBl eHBlY3QgcmVkdWNlZCBwZXJmb3JtYW5jZS4KKGNkMDphdGExOjA6MDowKTogU0NTSSBzdGF0dXMg ZXJyb3IKKGNkMDphdGExOjA6MDowKTogUkVBRCBDQVBBQ0lUWS4gQ0RCOiAyNSAwIDAgMCAwIDAg MCAwIDAgMCAKKGNkMDphdGExOjA6MDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IK KGNkMDphdGExOjA6MDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooY2QwOmF0YTE6 MDowOjApOiBTQ1NJIHNlbnNlOiBOT1QgUkVBRFkgYXNjOjNhLDEgKE1lZGl1bSBub3QgcHJlc2Vu dCAtIHRyYXkgY2xvc2VkKQooY2QwOmF0YTE6MDowOjApOiBFcnJvciA2LCBVbnJldHJ5YWJsZSBl cnJvcgpHRU9NOiBuZXcgZGlzayBhZGEwClRyeWluZyB0byBtb3VudCByb290IGZyb20gdWZzOi9k ZXYvYWRhMHAyIFtyd10uLi4Kc3RhcnRfaW5pdDogdHJ5aW5nIC9zYmluL2luaXQKYmF0dGVyeTA6 IGJhdHRlcnkgaW5pdGlhbGl6YXRpb24gZG9uZSwgdHJpZWQgMiB0aW1lcwptc2swOiBsaW5rIHN0 YXRlIGNoYW5nZWQgdG8gVVAK --20cf303b3a2dfc0a6904b76012b9-- From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 21:06:44 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42314106564A for ; Wed, 25 Jan 2012 21:06:44 +0000 (UTC) (envelope-from mail@chdevelopment.se) Received: from smtpout3.tre.se (smtpout3.tre.se [80.251.192.232]) by mx1.freebsd.org (Postfix) with ESMTP id C3CC48FC08 for ; Wed, 25 Jan 2012 21:06:43 +0000 (UTC) Received: from free.paradise.x (109.58.39.88.bredband.tre.se [109.58.39.88]) by smtpout3.tre.se with ESMTP id q0PKkXA6011477-q0PKkXA7011477 for ; Wed, 25 Jan 2012 21:46:34 +0100 Message-ID: <4F206A27.60006@chdevelopment.se> Date: Wed, 25 Jan 2012 21:46:31 +0100 From: Christer Hermansson User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0.1) Gecko/20111226 Firefox/9.0.1 SeaMonkey/2.6.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Problem with nat traversal X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 21:06:44 -0000 I have problem with nat traversal. The server is directly connected to the Internet, the client is behind a gateway that use nat. The problem is that the server tries to respond to the clients internal private address 192.168.1.10, (and the ISP sends icmp messages back to the server, telling it blocks 192.168 addresses). (I don't have access to the real output from tcpdump right now...) tcpdump on the server shows something like this: client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered windump on the client with win7 shows something like this: client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 4500 client-ext-ip > srv-ext-ip UDP 4500 client-ext-ip > srv-ext-ip UDP 4500 I get the same problem with FreeBSD 8.1R i386 + ipsec-tools 0.8.0 FreeBSD 8.2R amd64 + ipsec-tools 0.7.3 FreeBSD 8.2R amd64 + ipsec-tools 0.8.0 I have compiled the kernel with options IPSEC options IPSEC_DEBUG options IPSEC_FILTERTUNNEL options IPSEC_NAT_T device crypto device enc and I have "nat_traversal on" in racoon.conf. Why is the server trying to send packets to the clients internal address ? From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 21:07:22 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9091106567E for ; Wed, 25 Jan 2012 21:07:22 +0000 (UTC) (envelope-from mail@chdevelopment.se) Received: from smtpout3.tre.se (smtpout3.tre.se [80.251.192.232]) by mx1.freebsd.org (Postfix) with ESMTP id 402428FC1C for ; Wed, 25 Jan 2012 21:07:21 +0000 (UTC) Received: from free.paradise.x (109.58.39.88.bredband.tre.se [109.58.39.88]) by smtpout3.tre.se with ESMTP id q0PL7J9C013796-q0PL7J9D013796 for ; Wed, 25 Jan 2012 22:07:20 +0100 Message-ID: <4F206F04.8080805@chdevelopment.se> Date: Wed, 25 Jan 2012 22:07:16 +0100 From: Christer Hermansson User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0.1) Gecko/20111226 Firefox/9.0.1 SeaMonkey/2.6.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Problem with nat traversal X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 21:07:22 -0000 I have problem with nat traversal. The server is directly connected to the Internet, the client is behind a gateway that use nat. The problem is that the server tries to respond to the clients internal private address 192.168.1.10, (and the ISP sends icmp messages back to the server, telling it blocks 192.168 addresses). (I don't have access to the real output from tcpdump right now...) tcpdump on the server shows something like this: client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered client-ext-ip > srv-ext-ip UDP 4500 srv-ext-ip 4500 > client-INT-ip UDP icmp from isp-router telling client-INT-ip is filtered windump on the client with win7 shows something like this: client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 500 srv-ext-ip UDP 500 > client-ext-ip client-ext-ip > srv-ext-ip UDP 4500 client-ext-ip > srv-ext-ip UDP 4500 client-ext-ip > srv-ext-ip UDP 4500 I get the same problem with FreeBSD 8.1R i386 + ipsec-tools 0.8.0 FreeBSD 8.2R amd64 + ipsec-tools 0.7.3 FreeBSD 8.2R amd64 + ipsec-tools 0.8.0 I have compiled the kernel with options IPSEC options IPSEC_DEBUG options IPSEC_FILTERTUNNEL options IPSEC_NAT_T device crypto device enc and I have "nat_traversal on" in racoon.conf. Why is the server trying to send packets to the clients internal address ? From owner-freebsd-net@FreeBSD.ORG Wed Jan 25 23:37:55 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7819106564A for ; Wed, 25 Jan 2012 23:37:55 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8A08FC18 for ; Wed, 25 Jan 2012 23:37:55 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAMiRIE+DaFvO/2dsb2JhbABDhQmqQYFyAQEBAwEBAQEgKyALGxgCAg0ZAikBCSYGCAcEARwEh1sIpjWRVIEvh2ABBgYJBQEBAgEJAhgKBYJIBgIFAQUKAQMJCgEGAgJdHIIdgRYEiD+JBIE0gieSbw X-IronPort-AV: E=Sophos;i="4.71,571,1320642000"; d="scan'208";a="156817553" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 25 Jan 2012 18:37:38 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EB45CB3F9A; Wed, 25 Jan 2012 18:37:38 -0500 (EST) Date: Wed, 25 Jan 2012 18:37:38 -0500 (EST) From: Rick Macklem To: "Eugene M. Zheganin" Message-ID: <234323947.154683.1327534658917.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4F1F92DE.9060200@zhegan.in> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org Subject: Re: low network speed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 23:37:55 -0000 Eugene M. Zheganin wrote: > Hi. > > I'm suffering from low network performance on one of my FreeBSDs. > I have an i386 8.2-RELEASE machine with an fxp(4) adapter. It's > connected though a bunch of catalysts 2950 to another 8.2. While other > machines in this server room using the same sequence of switches and > the > same target source server (which, btw, is equipped with an em(4) and a > gigabit link bia catalyst 3750) show sufficient speed, this particular > machine while using scp starts with a speed of 200 Kbytes/sec and > while > copying the file shows speed about 600-800 Kbytes/sec. > > I've added this tweak to the sysctl: > > net.local.stream.recvspace=196605 > net.local.stream.sendspace=196605 > net.inet.tcp.sendspace=196605 > net.inet.tcp.recvspace=196605 > net.inet.udp.recvspace=196605 > kern.ipc.maxsockbuf=2621440 > kern.ipc.somaxconn=4096 > net.inet.tcp.sendbuf_max=524288 > net.inet.tcp.recvbuf_max=524288 > > With these settings the copying starts at 9.5 Mbytes/sec speed, but > then, as file is copying, drops down to 3.5 Megs/sec in about > two-three > minutes. > > Is there some way to maintain 9.5 Mbytes/sec (I like this speed more) > ? > You might want to try disabling the hardware checksumming via ifconfig. (I very vaguely recall doing that for a fxp(4) interface some time ago, but am probably completely wrong.:-) rick > > Thanks. > Eugene. > > P.S. This machine also runs zfs, I don't know if it's important but I > decided to mention it. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 02:02:19 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62FB41065675; Thu, 26 Jan 2012 02:02:19 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 4EC0D8FC21; Thu, 26 Jan 2012 02:02:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RqEfK-0008Wg-VW; Thu, 26 Jan 2012 02:02:19 +0000 Date: Thu, 26 Jan 2012 11:02:17 +0900 Message-ID: From: Randy Bush To: FreeBSD Net , FreeBSD Stable User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: Subject: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 02:02:19 -0000 day old i386 current bge1: mem 0xd0200000-0xd020ffff irq 10 at device 0.0 on pci5 bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: OUI 0x001018, model 0x0018, rev. 0 brgphy1: no media present ifmedia_set: no match for 0x0/0xfffffff panic: ifmedia_set KDB: stack backtrace: #0 0xc05bc257 at kdb_backtrace+0x47 #1 0xc058db2f at panic+0xaf #2 0xc063e3d1 at ifmedia_set+0x41 #3 0xc04e94fa at miibus_mediainit+0x8a #4 0xc04e227f at brgphy_attach+0x3bf #5 0xc05b5f6f at device_attach+0x36f #6 0xc05b745c at device_probe_and_attach+0x2c #7 0xc05b7489 at bus_generic_attach+0x19 #8 0xc04e9987 at miibus_attach+0xd7 #9 0xc05b5f6f at device_attach+0x36f #10 0xc05b745c at device_probe_and_attach+0x2c #11 0xc05b7489 at bus_generic_attach+0x19 #12 0xc04e9f0c at mii_attach+0x40c #13 0xc04db0f3 at bge_attach+0x3a93 #14 0xc05b5f6f at device_attach+0x36f #15 0xc05b745c at device_probe_and_attach+0x2c #16 0xc05b7489 at bus_generic_attach+0x19 #17 0xc049e984 at acpi_pci_attach+0x194 Uptime: 1s Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, --> or switch off the system now. randy From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 02:23:36 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A84601065676; Thu, 26 Jan 2012 02:23:36 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7F1118FC16; Thu, 26 Jan 2012 02:23:36 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0Q2NaEI011230; Thu, 26 Jan 2012 02:23:36 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0Q2NaZ5011226; Thu, 26 Jan 2012 02:23:36 GMT (envelope-from linimon) Date: Thu, 26 Jan 2012 02:23:36 GMT Message-Id: <201201260223.q0Q2NaZ5011226@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164495: [igb] connect double head igb to switch cause system to halt X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 02:23:36 -0000 Old Synopsis: connect double head igb to switch cause system to halt New Synopsis: [igb] connect double head igb to switch cause system to halt Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Jan 26 02:23:09 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=164495 From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 02:24:13 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 501401065677; Thu, 26 Jan 2012 02:24:13 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 26F288FC1A; Thu, 26 Jan 2012 02:24:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0Q2ODVZ011295; Thu, 26 Jan 2012 02:24:13 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0Q2ODgn011291; Thu, 26 Jan 2012 02:24:13 GMT (envelope-from linimon) Date: Thu, 26 Jan 2012 02:24:13 GMT Message-Id: <201201260224.q0Q2ODgn011291@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164475: [gre] gre misses RUNNING flag after a reboot X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 02:24:13 -0000 Old Synopsis: gre misses RUNNING flag after a reboot New Synopsis: [gre] gre misses RUNNING flag after a reboot Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Jan 26 02:23:58 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=164475 From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 03:10:13 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDFD71065690; Thu, 26 Jan 2012 03:10:13 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id B16028FC1C; Thu, 26 Jan 2012 03:10:13 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RqFj3-0008dM-3T; Thu, 26 Jan 2012 03:10:13 +0000 Date: Thu, 26 Jan 2012 12:10:12 +0900 Message-ID: From: Randy Bush To: FreeBSD Net , FreeBSD Stable In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 03:10:13 -0000 way cool. a /boot/device.hints entry of hint.acpi.bge.1.disable=1 did disable bge1. but now it's bge0, and i need that interface. and media are present! so i tried /etc/rc.conf ifconfig_bge0="198.180.150.1/25 media 1000baseTX" ifconfig_bge0_ipv6="inet6 2001:418:8006::1/64" ifconfig_bge0_alias0="inet 198.180.150.2/32" ifconfig_bge1="media 1000baseTX" pcib4: irq 12 at device 28.2 on pci0 pcib0: allocated type 3 (0xd0100000-0xd01fffff) for rid 20 of pcib4 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: memory decode 0xd0100000-0xd01fffff pcib4: no prefetched decode ACPI: Found matching pin for 4.0.INTA at func 0: 12 pci4: on pcib4 pci4: domain=0, physical bus=4 found-> vendor=0x14e4, dev=0x1659, revid=0x11 domain=0, bus=4, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=12 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0xd0100000, size 16, enabled pcib4: allocated memory range (0xd0100000-0xd010ffff) for rid 10 of pci0:4:0:0 pcib4: matched entry for 4.0.INTA (src \_SB_.PCI0.LNKC:0) pcib4: slot 0 INTA routed to irq 12 via \_SB_.PCI0.LNKC pci0:4:0:0: bad VPD cksum, remain 14 bge0: mem 0xd0100000-0xd010ffff irq 12 at device 0.0 on pci4 bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: OUI 0x001018, model 0x0018, rev. 0 brgphy0: no media present ifmedia_set: no match for 0x0/0xfffffff panic: ifmedia_set KDB: stack backtrace: #0 0xc05bc257 at kdb_backtrace+0x47 #1 0xc058db2f at panic+0xaf #2 0xc063e3d1 at ifmedia_set+0x41 #3 0xc04e94fa at miibus_mediainit+0x8a #4 0xc04e227f at brgphy_attach+0x3bf #5 0xc05b5f6f at device_attach+0x36f #6 0xc05b745c at device_probe_and_attach+0x2c #7 0xc05b7489 at bus_generic_attach+0x19 #8 0xc04e9987 at miibus_attach+0xd7 #9 0xc05b5f6f at device_attach+0x36f #10 0xc05b745c at device_probe_and_attach+0x2c #11 0xc05b7489 at bus_generic_attach+0x19 #12 0xc04e9f0c at mii_attach+0x40c #13 0xc04db0f3 at bge_attach+0x3a93 #14 0xc05b5f6f at device_attach+0x36f #15 0xc05b745c at device_probe_and_attach+0x2c #16 0xc05b7489 at bus_generic_attach+0x19 #17 0xc049e984 at acpi_pci_attach+0x194 Uptime: 1s Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, --> or switch off the system now. randy From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 03:20:25 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FEFA1065686 for ; Thu, 26 Jan 2012 03:20:25 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id E0CBE8FC15 for ; Thu, 26 Jan 2012 03:20:24 +0000 (UTC) Received: by vcmm1 with SMTP id m1so159180vcm.13 for ; Wed, 25 Jan 2012 19:20:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MAtWoxduYymD0uEU5b74/M9fLJVbcrnEsm8F2XTgu3w=; b=EBuXYZDR6Q7p+BNAwN2+W33uTBapSr4ulaM8SBM5Jd0nJS31Kl1SDSMQ9UBUV5Nq8N bfdaeOAnSBP1fHRQSBH+GE8vuQIp3zwMPoNT70qS/jKjx03PLfValnPPYHGm9/p0dSde 2/xkYqt+9CAY+XzXpJTimAAOppX/zTyhXjI5g= MIME-Version: 1.0 Received: by 10.220.154.135 with SMTP id o7mr210742vcw.2.1327548022491; Wed, 25 Jan 2012 19:20:22 -0800 (PST) Received: by 10.220.6.79 with HTTP; Wed, 25 Jan 2012 19:20:22 -0800 (PST) In-Reply-To: References: Date: Wed, 25 Jan 2012 22:20:22 -0500 Message-ID: From: Arnaud Lacombe To: Kim Culhan Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org Subject: Re: msk0: watchdog timeout interface hang X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 03:20:25 -0000 Hi, On Wed, Jan 25, 2012 at 3:26 PM, Kim Culhan wrote: > Running 10-curent from 01-20-12 > the msk0 interface hung, on the console: > > msk0: watchdog timeout > msk0: prefetch unit stuck? > msk0: initialization failed: no memory for Rx buffers > > Verbose boot dmesg output attached. > known issue affecting at least 8-STABLE, 9-STABLE (assumed) and -current. Already reported in these threads: http://lists.freebsd.org/pipermail/freebsd-net/2011-December/030635.html http://lists.freebsd.org/pipermail/freebsd-questions/2011-November/235646.html - Arnaud > Any help is greatly appreciated. > > -kim > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 03:24:13 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87890106566B; Thu, 26 Jan 2012 03:24:13 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 713EE8FC12; Thu, 26 Jan 2012 03:24:13 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RqFwb-0008fE-3L; Thu, 26 Jan 2012 03:24:13 +0000 Date: Thu, 26 Jan 2012 12:24:11 +0900 Message-ID: From: Randy Bush To: FreeBSD Net , FreeBSD Stable In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 03:24:13 -0000 ok, i o used device.hints to disable both bge interfaces o booted successfully o used serial console o ifconfiged bge0 to the normal addresses o and it is working i suspect that something sucks in bge initialization at startup. insightful, i know. sorry. randy From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 04:16:27 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BC87106566B for ; Thu, 26 Jan 2012 04:16:27 +0000 (UTC) (envelope-from mjl@luckie.org.nz) Received: from hrndva-omtalb.mail.rr.com (hrndva-omtalb.mail.rr.com [71.74.56.122]) by mx1.freebsd.org (Postfix) with ESMTP id 48C528FC15 for ; Thu, 26 Jan 2012 04:16:27 +0000 (UTC) Received: from cdptpa-omtalb.mail.rr.com ([10.127.143.52]) by cdptpa-qmta01.mail.rr.com with ESMTP id <20120126024507686.SVCR16141@cdptpa-qmta01.mail.rr.com> for ; Thu, 26 Jan 2012 02:45:07 +0000 X-Authority-Analysis: v=2.0 cv=adPjbGUt c=1 sm=0 a=aIgkrZOPZtJQBHPFbnUF7Q==:17 a=tm9qpUR5xacA:10 a=07AZlKUsOKkA:10 a=8nJEP1OIZ-IA:10 a=bzY6Fysj7q9iz3SMlDoA:9 a=s7YZLsFBfEiFwCzSuLcA:7 a=wPNLvfGTeEIA:10 a=aIgkrZOPZtJQBHPFbnUF7Q==:117 X-Cloudmark-Score: 0 X-Originating-IP: 76.88.32.44 Received: from [76.88.32.44] ([76.88.32.44:60054] helo=spandex.luckie.org.nz) by cdptpa-oedge02.mail.rr.com (envelope-from ) (ecelerity 2.2.3.46 r()) with ESMTP id 77/6F-15128-4FDB02F4; Thu, 26 Jan 2012 02:44:06 +0000 Received: from mylar.luckie.org.nz ([192.168.2.20]) by spandex.luckie.org.nz with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RqFJk-0000it-Aw for freebsd-net@freebsd.org; Wed, 25 Jan 2012 18:44:04 -0800 Message-ID: <4F20BE24.3050101@luckie.org.nz> Date: Wed, 25 Jan 2012 18:44:52 -0800 From: Matthew Luckie User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:9.0) Gecko/20111231 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: high cpu usage on natd / dhcpd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 04:16:27 -0000 Hi I have a small system running FreeBSD 8.2 that does NAT using ipfw and natd to systems attached to two interfaces: em0 and wlan0. I have a dhcpd daemon issuing leases on those interfaces. The system has an em1 interface plugged into a cable modem where it obtains a DHCP lease from an ISP. For some reason, when traffic from the Internet terminates on the system itself (I scp a file from the computer) the natd and dhcpd processes consume significant CPU, and the throughput is less than I expect. Traffic that passes through to a computer behind the NAT flows without causing the natd or dhcpd processes to measurably consume CPU. From top: CPU: 10.9% user, 0.0% nice, 56.0% system, 21.1% interrupt, 12.0% idle Mem: 225M Active, 92M Inact, 162M Wired, 556K Cache, 112M Buf, 1506M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1222 root 1 104 0 3572K 1448K RUN 1:29 39.36% natd 1676 root 1 62 0 5340K 3544K select 0:59 24.56% dhcpd What is going on? My ipfw ruleset is below, and is based on the example in the FreeBSD handbook. 00001 allow ip from any to any via lo0 00002 allow ip from any to any via em0 00003 allow ip from any to any via wlan0 00101 divert 8668 ip from any to any in via em1 00102 check-state 00110 skipto 500 tcp from any to any out via em1 setup keep-state 00111 skipto 500 udp from any to any out via em1 keep-state 00112 skipto 500 icmp from any to any out via em1 keep-state 00201 allow udp from any to any dst-port 68 in keep-state 00202 allow tcp from any to me dst-port 80 in via em1 setup keep-state 00210 allow tcp from 130.217.250.13 to me in via em1 setup keep-state 00211 allow tcp from 199.109.33.1 to me in via em1 setup keep-state 00212 allow tcp from 192.172.226.78 to me in via em1 setup keep-state 00213 allow tcp from 192.172.226.95 to me in via em1 setup keep-state 00230 allow tcp from any to me dst-port 6984 in via em1 setup keep-state 00231 allow udp from any to me dst-port 6984 in via em1 00240 allow icmp from any to me in via em1 00300 unreach filter-prohib log ip from any to any 00500 divert 8668 ip from any to any out via em1 00501 allow ip from any to any 65535 allow ip from any to any From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 10:46:25 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26908106567A for ; Thu, 26 Jan 2012 10:46:25 +0000 (UTC) (envelope-from w8hdkim@gmail.com) Received: from mail-qw0-f47.google.com (mail-qw0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id D9D9A8FC0A for ; Thu, 26 Jan 2012 10:46:24 +0000 (UTC) Received: by qadz30 with SMTP id z30so47773qad.13 for ; Thu, 26 Jan 2012 02:46:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=uI55+DNZL5fFFokhIopSgv0r0ob18Ek/RIp+QSliAAo=; b=SGaIjRynwITAs7aK1JRvZBu1KbpkCQaZYf+UPIOyKAEdyOPCrxmYd1JKQseZ0m1cGE bcLBy/BFx+1m0tENfq5wd0WQf5wMADozLNeFADiIehyHx3sm9Xb6tfVhnm7AX28wO3z2 GHso5VzHstjO3JV0WxGOY/RyjuGMGuZsnGzLU= MIME-Version: 1.0 Received: by 10.224.212.134 with SMTP id gs6mr1659132qab.32.1327574784070; Thu, 26 Jan 2012 02:46:24 -0800 (PST) Received: by 10.229.223.135 with HTTP; Thu, 26 Jan 2012 02:46:23 -0800 (PST) In-Reply-To: References: Date: Thu, 26 Jan 2012 05:46:23 -0500 Message-ID: From: Kim Culhan To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: msk0: watchdog timeout interface hang X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 10:46:25 -0000 On Wed, Jan 25, 2012 at 3:26 PM, Kim Culhan wrote: > Running 10-curent from 01-20-12 > the msk0 interface hung, on the console: > > msk0: watchdog timeout > msk0: prefetch unit stuck? > msk0: initialization failed: no memory for Rx buffers > > Verbose boot dmesg output attached. This additional datapoint found, at boot after the last line in the verbose dmesg this line was logged to messages: Jan 25 15:21:19 foo kernel: interrupt storm detected on "irq257:"; throttling interrupt source Hope this helps. thanks -kiim From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 13:40:11 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDA73106566B for ; Thu, 26 Jan 2012 13:40:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BE01A8FC14 for ; Thu, 26 Jan 2012 13:40:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0QDeBm7072317 for ; Thu, 26 Jan 2012 13:40:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0QDeBnF072316; Thu, 26 Jan 2012 13:40:11 GMT (envelope-from gnats) Date: Thu, 26 Jan 2012 13:40:11 GMT Message-Id: <201201261340.q0QDeBnF072316@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Dmitry Banschikov Cc: Subject: Re: bin/145934: [patch] add count option to netstat(1) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dmitry Banschikov List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 13:40:12 -0000 The following reply was made to PR bin/145934; it has been noted by GNATS. From: Dmitry Banschikov To: bug-followup@FreeBSD.org Cc: Subject: Re: bin/145934: [patch] add count option to netstat(1) Date: Thu, 26 Jan 2012 16:09:45 +0300 --001636ed74f71f7ceb04b76e18c4 Content-Type: text/plain; charset=ISO-8859-1 It seems, that this PR can be closed, as r202060 introduced similiar changes. -- Dmitry Banshchikov --001636ed74f71f7ceb04b76e18c4 Content-Type: text/html; charset=ISO-8859-1 It seems, that this PR can be closed, as r202060 introduced similiar changes.

--

Dmitry Banshchikov
--001636ed74f71f7ceb04b76e18c4-- From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 13:57:06 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4B931065670; Thu, 26 Jan 2012 13:57:06 +0000 (UTC) (envelope-from pluknet@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B7A108FC14; Thu, 26 Jan 2012 13:57:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0QDv6UF089160; Thu, 26 Jan 2012 13:57:06 GMT (envelope-from pluknet@freefall.freebsd.org) Received: (from pluknet@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0QDv6mj089156; Thu, 26 Jan 2012 13:57:06 GMT (envelope-from pluknet) Date: Thu, 26 Jan 2012 13:57:06 GMT Message-Id: <201201261357.q0QDv6mj089156@freefall.freebsd.org> To: me@ubique.spb.ru, pluknet@FreeBSD.org, freebsd-net@FreeBSD.org From: pluknet@FreeBSD.org Cc: Subject: Re: bin/145934: [patch] add count option to netstat(1) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 13:57:07 -0000 Synopsis: [patch] add count option to netstat(1) State-Changed-From-To: open->closed State-Changed-By: pluknet State-Changed-When: Thu Jan 26 13:54:57 UTC 2012 State-Changed-Why: Close per submitter request. Similar functionality is available since 8.1 using the -q option. http://www.freebsd.org/cgi/query-pr.cgi?pr=145934 From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 17:49:21 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0476106564A for ; Thu, 26 Jan 2012 17:49:21 +0000 (UTC) (envelope-from satishkamara@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 697118FC15 for ; Thu, 26 Jan 2012 17:49:21 +0000 (UTC) Received: by obcwo16 with SMTP id wo16so1268693obc.13 for ; Thu, 26 Jan 2012 09:49:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=0qAyTg2mILlq1FBYBGpcNmK26F1/MSQBzovM03/QCEY=; b=MVDWkG5NJHm8WxKPJ8ooeaQ5J6xS/g1hvjwZUrRcqAAAJ5u3fEPNJZVL7dXhxfbknr qLyqOeIb9bO5rIww2xZb/3qef0j6sVlNSDMNZoFTS/X89V511ZFxXuAqS60iz30bQ16R ePGsfzb9+0kHsPvdmA9qNY9LX1eKvuxwkjxQY= MIME-Version: 1.0 Received: by 10.182.41.98 with SMTP id e2mr1396419obl.68.1327598690284; Thu, 26 Jan 2012 09:24:50 -0800 (PST) Received: by 10.60.46.69 with HTTP; Thu, 26 Jan 2012 09:24:50 -0800 (PST) Date: Thu, 26 Jan 2012 12:24:50 -0500 Message-ID: From: satish amara To: freebsd-net@freebsd.org X-Mailman-Approved-At: Thu, 26 Jan 2012 19:04:16 +0000 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 17:49:21 -0000 Hi, I have question regarding stateful firewall implementation of FreeBSD. IPF has stateful =93keep state=94 option. Stateful filtering treats traffic as a bi-directional exchange of packets comprising a session conversation. When activated, keep-state dynamically generates internal rules for each anticipated packet being exchanged during the bi-directional session conversation. It has sufficient matching capabilities to determine if the session conversation between the originating sender and the destination are following the valid procedure of bi-directional packet exchange. Any packets that do not properly fit the session conversation template are automatically rejected as impostors. I have question regarding the size of the state table kept in FreeBSD for stateful packet inspection. Say we have a valid senario where we have stateful firewall rule for HTTP and we get lot of incoming new HTTP session and state table is filled full. In that case I guess FreeBSD would reject new sessions. Just want to know what is the latest on this. How does FreeBSD would handle if the state table is full and we get valid new HTTP connection. What are options in terms of configuration or new feature in BSD would address this issue. Thanks, Satish K Amara From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 19:44:21 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E8AF1065700 for ; Thu, 26 Jan 2012 19:44:21 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id ECB3D8FC08 for ; Thu, 26 Jan 2012 19:44:20 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.5/8.14.4) with ESMTP id q0QJiJxq003479; Thu, 26 Jan 2012 14:44:19 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <4F21AD09.9010307@sentex.net> Date: Thu, 26 Jan 2012 14:44:09 -0500 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: satish amara References: In-Reply-To: X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Cc: freebsd-net@freebsd.org Subject: Re: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 19:44:21 -0000 On 1/26/2012 12:24 PM, satish amara wrote: > Hi, > I have question regarding stateful firewall implementation of FreeBSD. > IPF has stateful “keep state” option. Hi, Take a look at pf, not ipf. ipf is not really maintained or used much any more under FreeBSD. With respect to dealing with congestion, there are many params you can tune in pf. Take a look at the man pages for pf.conf for details as you can control how this situation is dealt with to some degree. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Thu Jan 26 20:40:39 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7C121065741 for ; Thu, 26 Jan 2012 20:40:39 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout028.mac.com (asmtpout028.mac.com [17.148.16.103]) by mx1.freebsd.org (Postfix) with ESMTP id AEEE48FC16 for ; Thu, 26 Jan 2012 20:40:39 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from cswiger1.apple.com (unknown [17.209.4.71]) by asmtp028.mac.com (Oracle Communications Messaging Server 7u4-23.01 (7.0.4.23.0) 64bit (built Aug 10 2011)) with ESMTPSA id <0LYF00LHG80FW100@asmtp028.mac.com> for freebsd-net@freebsd.org; Thu, 26 Jan 2012 11:41:04 -0800 (PST) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.6.7361,1.0.211,0.0.0000 definitions=2012-01-26_07:2012-01-26, 2012-01-26, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1012030000 definitions=main-1201260219 From: Chuck Swiger In-reply-to: Date: Thu, 26 Jan 2012 11:41:02 -0800 Message-id: References: To: satish amara X-Mailer: Apple Mail (2.1084) Cc: freebsd-net@freebsd.org Subject: Re: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 20:40:39 -0000 Hi-- On Jan 26, 2012, at 9:24 AM, satish amara wrote: > I have question regarding the size of the state table kept in FreeBSD for > stateful packet inspection. Say we have a valid senario where we have > stateful firewall rule for HTTP and we get lot of incoming new HTTP session > and state table is filled full. In that case I guess FreeBSD would reject > new sessions. Just want to know what is the latest on this. How does > FreeBSD would handle if the state table is full and we get valid new HTTP > connection. What are options in terms of configuration or new feature in > BSD would address this issue. A securely designed firewall will drop connections when the state table is full. You can increase the size of the state table by following the IPF FAQ: http://www.phildev.net/ipf/IPFques.html#ques25 ...but in point of fact, keeping state for high-volume traffic is generally a losing game, and you are better off (IMHO) setting up stateless bidirectional rules which permit such high volume traffic. HTTP isn't generally too much of a problem, though-- something like a popular stratum-1 or 2 public NTP timeserver will easily blow out a stateful firewall if you try to keep state for NTP's UDP traffic. Regards, -- -Chuck From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 02:41:56 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C66C3106564A for ; Fri, 27 Jan 2012 02:41:56 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4AFB58FC12 for ; Fri, 27 Jan 2012 02:41:55 +0000 (UTC) Received: by wibhn14 with SMTP id hn14so1408833wib.13 for ; Thu, 26 Jan 2012 18:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=/0/mjQ/B+YuI2yg9j3aftQDSyUnnPxrB8OvEA6WbNR0=; b=wAXdEwnGixjcF8l7F0DUqxzB2dvV5iH0eTupCxOZQp6dSK3BVps6kk3HvtLguGh+am Z8FKOUAcM+tWIAWhDyhfIVfxB9xK/HKvdrQzrPb14cPH+19M6LQ4gxhcCdDhGB5iZHXP wnweawDIAjko7YtE0fu1aGmfI/KHGFnaqEK3E= MIME-Version: 1.0 Received: by 10.181.11.231 with SMTP id el7mr4382046wid.0.1327632114726; Thu, 26 Jan 2012 18:41:54 -0800 (PST) Received: by 10.223.101.196 with HTTP; Thu, 26 Jan 2012 18:41:54 -0800 (PST) In-Reply-To: References: Date: Thu, 26 Jan 2012 18:41:54 -0800 Message-ID: From: Kevin Oberman To: Chuck Swiger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: satish amara , freebsd-net@freebsd.org Subject: Re: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 02:41:56 -0000 On Thu, Jan 26, 2012 at 11:41 AM, Chuck Swiger wrote: > Hi-- > > On Jan 26, 2012, at 9:24 AM, satish amara wrote: >> I have question regarding the size of the state table kept in FreeBSD fo= r >> stateful packet inspection. Say we have a valid senario where we have >> stateful firewall rule for HTTP and we get lot of incoming new HTTP sess= ion >> and state table is filled full. In that case I guess FreeBSD would rejec= t >> new sessions. =A0Just want to know what is the latest on this. How does >> FreeBSD would handle if the state table is full and we get valid new HTT= P >> connection. What are options in terms of configuration or new feature in >> BSD would address this issue. > > A securely designed firewall will drop connections when the state table i= s full. > > You can increase the size of the state table by following the IPF FAQ: > > =A0http://www.phildev.net/ipf/IPFques.html#ques25 > > ...but in point of fact, keeping state for high-volume traffic is general= ly > a losing game, and you are better off (IMHO) setting up stateless bidirec= tional > rules which permit such high volume traffic. > > HTTP isn't generally too much of a problem, though-- something like a pop= ular > stratum-1 or 2 public NTP timeserver will easily blow out a stateful fire= wall > if you try to keep state for NTP's UDP traffic. To put it very clearly, a stateful firewall "protecting" a server is an open invitation to DOS. It is trivial to generate enough UDP traffic to overflow any limit on connections in a stateful firewall. Various tricks have been tried but the reality is that none has really succeeded. Some do help, but nowhere near enough to solve the problem. Stateful firewalls are for clients and systems that don't provide publicly accessible services. Servers require stateless filters along with IDS/IPS for effective protection. And I do expect to get people saying that you HAVE to have a stateful firewall is a basic requirement for a device on the Internet. I can only say htat I know of many well known servers that do not have them and few that do. There is a reason for that. At my old employer we were under government security oversight and I can remember the auditors back a few years ago who had a fit when told that no firewall was employed, just an IDS/IPS with RTBH. The problem is that their red team of attackers never could successfully attack which really annoyed them to the point that they tryed toi order that the IDS be disabled for their attack attempts. (We refused, siting terms of the testing agreement.) Today, auditors still are a bit surprised that they don't use a firewall, but are no longer upset by it as they are seeing it more often. --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 04:43:35 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5B93106564A for ; Fri, 27 Jan 2012 04:43:35 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 606EF8FC0C for ; Fri, 27 Jan 2012 04:43:34 +0000 (UTC) Received: by eaaa14 with SMTP id a14so481820eaa.13 for ; Thu, 26 Jan 2012 20:43:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=17OGqnR9HyHOzLkZB52+lq4BOlwv6r9w+7RlcQjuEG0=; b=kFWmKqvVBAQBh+L8x5FQxn3Bb7OwsxMbeSW5nWqf90F7URTChEZ/A7IRzBZOz+Ibm1 mC+JH89xSisDklQqx10ATI0VKZMOk30+Y9RaOblaHX+Q4OmbbgVtNUtqQCwg/iH4f5YP 3T5DYQLGkG4KbI2YoM8aABRFqrpXtx4zfrQLQ= Received: by 10.213.114.129 with SMTP id e1mr174770ebq.124.1327639413086; Thu, 26 Jan 2012 20:43:33 -0800 (PST) Received: from imba-brutale.totalterror.net (93-152-152-135.ddns.onlinedirect.bg. [93.152.152.135]) by mx.google.com with ESMTPS id s16sm24312761eef.2.2012.01.26.20.43.30 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 26 Jan 2012 20:43:31 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Nikolay Denev In-Reply-To: Date: Fri, 27 Jan 2012 06:43:29 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1A4CBF45-8ABB-4BFB-A83A-2906CBD32667@gmail.com> References: To: Kevin Oberman X-Mailer: Apple Mail (2.1251.1) Cc: satish amara , freebsd-net@freebsd.org Subject: Re: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 04:43:35 -0000 On Jan 27, 2012, at 4:41 AM, Kevin Oberman wrote: > On Thu, Jan 26, 2012 at 11:41 AM, Chuck Swiger = wrote: >> Hi-- >>=20 >> On Jan 26, 2012, at 9:24 AM, satish amara wrote: >>> I have question regarding the size of the state table kept in = FreeBSD for >>> stateful packet inspection. Say we have a valid senario where we = have >>> stateful firewall rule for HTTP and we get lot of incoming new HTTP = session >>> and state table is filled full. In that case I guess FreeBSD would = reject >>> new sessions. Just want to know what is the latest on this. How = does >>> FreeBSD would handle if the state table is full and we get valid new = HTTP >>> connection. What are options in terms of configuration or new = feature in >>> BSD would address this issue. >>=20 >> A securely designed firewall will drop connections when the state = table is full. >>=20 >> You can increase the size of the state table by following the IPF = FAQ: >>=20 >> http://www.phildev.net/ipf/IPFques.html#ques25 >>=20 >> ...but in point of fact, keeping state for high-volume traffic is = generally >> a losing game, and you are better off (IMHO) setting up stateless = bidirectional >> rules which permit such high volume traffic. >>=20 >> HTTP isn't generally too much of a problem, though-- something like a = popular >> stratum-1 or 2 public NTP timeserver will easily blow out a stateful = firewall >> if you try to keep state for NTP's UDP traffic. >=20 > To put it very clearly, a stateful firewall "protecting" a server is > an open invitation to DOS. It is trivial to generate enough UDP > traffic to overflow any limit on connections in a stateful firewall. > Various tricks have been tried but the reality is that none has really > succeeded. Some do help, but nowhere near enough to solve the problem. >=20 > Stateful firewalls are for clients and systems that don't provide > publicly accessible services. Servers require stateless filters along > with IDS/IPS for effective protection. >=20 > And I do expect to get people saying that you HAVE to have a stateful > firewall is a basic requirement for a device on the Internet. I can > only say htat I know of many well known servers that do not have them > and few that do. There is a reason for that. At my old employer we > were under government security oversight and I can remember the > auditors back a few years ago who had a fit when told that no firewall > was employed, just an IDS/IPS with RTBH. The problem is that their red > team of attackers never could successfully attack which really annoyed > them to the point that they tryed toi order that the IDS be disabled > for their attack attempts. (We refused, siting terms of the testing > agreement.) >=20 > Today, auditors still are a bit surprised that they don't use a > firewall, but are no longer upset by it as they are seeing it more > often. > --=20 > R. Kevin Oberman, Network Engineer > E-mail: kob6558@gmail.com >=20 In my experience (and I've had a few DDoS attacks), the state table was = never an issue (unless left at default settings), the machine would either die from interrupt/cpu overload, or the pipe = will be filled. For example the pf(4) firewall can be tuned to have millions of state = entries, then you can configure thresholds which reached will make the existing = state entries expire sooner, leaving room for new ones. P.S.: Stateful firewalls are required by the PCI DSS (requirement 1.3.6) From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 06:04:00 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57143106566B; Fri, 27 Jan 2012 06:04:00 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id D5AE08FC1B; Fri, 27 Jan 2012 06:03:59 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so1618977vbb.13 for ; Thu, 26 Jan 2012 22:03:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=zsOG+Td6olprJiykJJ3eGZ9/lHoDEO0L9yDV6npBZzk=; b=bGapyzKhVbCBtWDUH5sV+NFJX5z9a4FDdWvaNy6akeR1JxtPCnKf+287DLQZU4e+HM p0j/FFreRrN04OFcuRagRwHVWr5myjC2MTAGYnjPg03o0dXC2n82FF/vTRUqNjm9ytrc RWmvFpqIM86MGz32uURo6BaLqYqjnK4BjCjYo= MIME-Version: 1.0 Received: by 10.52.65.100 with SMTP id w4mr2342326vds.3.1327644238987; Thu, 26 Jan 2012 22:03:58 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.73.228 with HTTP; Thu, 26 Jan 2012 22:03:58 -0800 (PST) In-Reply-To: <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> Date: Thu, 26 Jan 2012 22:03:58 -0800 X-Google-Sender-Auth: jNdELBKiWmqVD4lBcH_wkce2LkQ Message-ID: From: Adrian Chadd To: Stefan Bethke Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Aleksandr Rybalko , freebsd-arch@freebsd.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 06:04:00 -0000 Ok, I do like the idea of: * mdiobus/miibus proxy tidyup; * then the switch API; * then the switch devices themselves. Can we get some consensus/agreement from Marius (and others) about the first step? Adrian From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 14:18:23 2012 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B32261065670; Fri, 27 Jan 2012 14:18:23 +0000 (UTC) (envelope-from alexey@kouznetsov.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 590D88FC14; Fri, 27 Jan 2012 14:18:23 +0000 (UTC) Received: by qcse14 with SMTP id e14so1302596qcs.13 for ; Fri, 27 Jan 2012 06:18:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.111.89 with SMTP id r25mr2476219qcp.106.1327672234311; Fri, 27 Jan 2012 05:50:34 -0800 (PST) Received: by 10.229.39.13 with HTTP; Fri, 27 Jan 2012 05:50:34 -0800 (PST) In-Reply-To: References: <4ebc3732.86962a0a.7ab9.ffffa29dSMTPIN_ADDED@mx.google.com> Date: Fri, 27 Jan 2012 17:50:34 +0400 Message-ID: From: Alexey Kouznetsov To: zi@FreeBSD.org, net@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Fwd: [ net-snmp-Bugs-3434824 ] coredump on HUP while disks read X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 14:18:23 -0000 Hello! There are bug in net-snmp. net-snmp die (coredump) on SIGHUP while we have disk command defined in snmpd.*conf. Bellow fix in main net-snmp tree. I think it is good idea to add such small patch to the FreeBSD port unlil they release new version an new version will be included to the ports. You can see path an butom of mail conversation bellow Thank you! With best regards /Alexey ---------- Forwarded message ---------- From: SourceForge.net Date: 2011/11/11 Subject: [ net-snmp-Bugs-3434824 ] coredump on HUP while disks read To: "SourceForge.net" Bugs item #3434824, was opened at 2011-11-07 23:45 Message generated for change (Comment added) made by nba You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3434824&group_id=12694 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: agent Group: None >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Alexey (st-da) Assigned to: Niels Baggesen (nba) Summary: coredump on HUP while disks read Initial Comment: FreeBSD xxx 8.2-STABLE FreeBSD 8.2-STABLE #9: Tue Oct 11 07:07:46 UTC 2011 root@xxx:/usr/obj/usr/src/sys/AAASMP i386 but some error reproduced at amd64 system net-snmp from ports net-snmp-5.7_4 net snmp started as: --- /usr/local/sbin/snmpd -p /var/run/snmpd.pid -Lf /var/log/snmpd.log --- We have commands like: disk / disk /usr disk /var in config. when we kill -HUP ` cat /var/run/snmpd.pid ` we got "kernel: pid 61129 (snmpd), uid 0: exited on signal 11 (core dumped)" If we remove disk commands error disapear. if we add includeAllDisks or any single disk we got an error. snmpd.cons is simple some access lines and this disk lines. snmpd works untill we send sighup. (unlill we logrotate, and logrotate send sighup for reopen log file). If we disable logging problem hides (we do not need sent sighup, so it will work) gdb output bellow: Core was generated by `snmpd'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/libnetsnmpagent.so.30...done. Loaded symbols for /usr/local/lib/libnetsnmpagent.so.30 Reading symbols from /usr/local/lib/libnetsnmpmibs.so.30...done. Loaded symbols for /usr/local/lib/libnetsnmpmibs.so.30 Reading symbols from /usr/lib/libwrap.so.6...done. Loaded symbols for /usr/lib/libwrap.so.6 Reading symbols from /usr/local/lib/libnetsnmp.so.30...done. Loaded symbols for /usr/local/lib/libnetsnmp.so.30 Reading symbols from /lib/libm.so.5...done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libkvm.so.5...done. Loaded symbols for /lib/libkvm.so.5 Reading symbols from /lib/libdevstat.so.7...done. Loaded symbols for /lib/libdevstat.so.7 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libelf.so.1...done. Loaded symbols for /usr/lib/libelf.so.1 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x2814a8a5 in disk_parse_config (token=0xbfbfdb8c "disk", cptr=0xbfbfdf9c "10240") at ucd-snmp/disk_hw.c:193 193 disks[numdisks++] = entry; (gdb) where #0 0x2814a8a5 in disk_parse_config (token=0xbfbfdb8c "disk", cptr=0xbfbfdf9c "10240") at ucd-snmp/disk_hw.c:193 #1 0x28269830 in run_config_handler (lptr=0x286878a0, token=0xbfbfdb8c "disk", cptr=0xbfbfdf91 "/ 10240", when=0) at read_config.c:546 #2 0x2826a70a in read_config (filename=0xbfbfe440 "/usr/local/etc/snmp/snmpd.conf", line_handler=0x2860f080, when=0) at read_config.c:939 #3 0x2826b7ae in read_config_files_in_path (path=Variable "path" is not available. ) at read_config.c:1282 #4 0x2826bb86 in read_config_files_of_type (when=0, ctmp=0x28620040) at read_config.c:1365 #5 0x2826bc14 in read_config_files (when=0) at read_config.c:1406 #6 0x2826c39f in read_configs () at read_config.c:1018 #7 0x280bc928 in update_config () at agent_read_config.c:292 #8 0x0804b91b in SnmpdReconfig () #9 0x0804a127 in ?? () #10 0x00000000 in ?? () #11 0x00000000 in ?? () #12 0xbfbfea08 in ?? () #13 0x0804a127 in ?? () #14 0x00000005 in ?? () #15 0xbfbfea30 in ?? () #16 0xbfbfea48 in ?? () #17 0xbfbfea10 in ?? () #18 0xbfbfea2c in ?? () #19 0x00000000 in ?? () #20 0xbfbfea28 in ?? () #21 0x0804a098 in ?? () Previous frame inner to this frame (corrupt stack?) (gdb) list 292 read_configs(); 293 } 294 295 296 void 297 snmpd_register_config_handler(const char *token, 298 void (*parser) (const char *, char *), 299 void (*releaser) (void), const char *help) 300 { 301 DEBUGMSGTL(("snmpd_register_app_config_handler", ---------------------------------------------------------------------- >Comment By: Niels Baggesen (nba) Date: 2011-11-10 12:42 Message: Best I can tell this fiexed the problem, and I have applied this fix to Net-SNMP Thanks for the bug report. ---------------------------------------------------------------------- Comment By: Niels Baggesen (nba) Date: 2011-11-08 12:55 Message: I think a single missing line is causing the trouble. Please try this: --- a/agent/mibgroup/ucd-snmp/disk_hw.c +++ b/agent/mibgroup/ucd-snmp/disk_hw.c @@ -137,6 +137,7 @@ disk_free_config(void) if (disks) { free( disks ); disks = NULL; + maxdisks = numdisks = 0; } allDisksIncluded = 0; } ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3434824&group_id=12694 From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 15:20:12 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 404B3106564A for ; Fri, 27 Jan 2012 15:20:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 2BDC88FC14 for ; Fri, 27 Jan 2012 15:20:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0RFKCA9029905 for ; Fri, 27 Jan 2012 15:20:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0RFKCax029904; Fri, 27 Jan 2012 15:20:12 GMT (envelope-from gnats) Date: Fri, 27 Jan 2012 15:20:12 GMT Message-Id: <201201271520.q0RFKCax029904@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "Pavel Polyakov" Cc: Subject: Re: kern/159621: [tcp] [panic] panic: soabort: so_count X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Pavel Polyakov List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 15:20:12 -0000 The following reply was made to PR kern/159621; it has been noted by GNATS. From: "Pavel Polyakov" To: bug-followup@freebsd.org, bsd@kobyla.org Cc: Subject: Re: kern/159621: [tcp] [panic] panic: soabort: so_count Date: Fri, 27 Jan 2012 16:48:10 +0200 Dump header from device /dev/ada1p20 Architecture: amd64 Architecture Version: 1 Dump Length: 465408B (0 MB) Blocksize: 512 Dumptime: Wed Jan 25 00:14:21 2012 Hostname: shared Magic: FreeBSD Text Dump Version String: FreeBSD 8.2-STABLE #5 r230331: Mon Jan 23 22:05:26 UTC 2012 Panic String: soabort: so_count Dump Parity: 1378570526 Bounds: 2 Dump Status: good ichwd0: timer reloaded panic: soabort: so_count cpuid = 1 KDB: enter: panic 0xffffff0008d8a3b0: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xffffff0008657000 (pid 20) 0xffffff02104dcce8: tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xffffff009f19a798 ref 0 pages 1 lock type ufs: SHARED (count 1) ino 89806906, on dev mirror/shared0home 0xffffff0085b4b760: tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xffffff013d38c5e8 ref 0 pages 1 lock type ufs: SHARED (count 1) ino 57062185, on dev mirror/shared0home 0xffffff01afaa21d8: tag nfs, type VDIR usecount 2, writecount 0, refcount 10934 mountedhere 0 flags () v_object 0xffffff01a4955000 ref 0 pages 15576 lock type nfs: SHARED (count 1) fileid 43900928 fsid 0x500ff01 db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks exclusive rw tcpinp (tcpinp) r = 0 (0xffffff0042d89670) locked @ /usr/src/sys/netinet/tcp_input.c:946 exclusive rw tcp (tcp) r = 0 (0xffffffff80b35108) locked @ /usr/src/sys/netinet/tcp_usrreq.c:1507 db:1:locks> show alllocks Process 65325 (mysqldump) thread 0xffffff00be85f460 (100712) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01ae596648) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 63966 (exim-4.77-0) thread 0xffffff01ba777000 (100319) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00be7170f8) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 62099 (cpsrvd-ssl) thread 0xffffff01d78de460 (103292) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00255b4b98) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 34226 (pkgacct) thread 0xffffff011936e8c0 (101408) shared lockmgr nfs (nfs) r = 0 (0xffffff01afaa2270) locked @ /usr/src/sys/kern/vfs_syscalls.c:4100 Process 33445 (pkgacct) thread 0xffffff01ba0c5460 (101331) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff81ee9040e0) locked @ /usr/src/sys/kern/vfs_bio.c:1891 shared lockmgr ufs (ufs) r = 0 (0xffffff02104dcd80) locked @ /usr/src/sys/kern/vfs_syscalls.c:4100 Process 26396 (pkgacct) thread 0xffffff01baa80460 (100415) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff81ee66b748) locked @ /usr/src/sys/kern/vfs_bio.c:1891 shared lockmgr ufs (ufs) r = 0 (0xffffff0085b4b7f8) locked @ /usr/src/sys/kern/vfs_syscalls.c:4100 Process 20 (syncer) thread 0xffffff0008657000 (100115) exclusive sleep mutex struct mount mtx (struct mount mtx) r = 0 (0xffffff00087e1bc0) locked @ /usr/src/sys/ufs/ufs/ufs_quota.c:939 exclusive lockmgr syncer (syncer) r = 0 (0xffffff0008d8a448) locked @ /usr/src/sys/kern/vfs_subr.c:1770 Process 0 (kernel) thread 0xffffff0002b07000 (100076) exclusive rw tcpinp (tcpinp) r = 0 (0xffffff0042d89670) locked @ /usr/src/sys/netinet/tcp_input.c:946 exclusive rw tcp (tcp) r = 0 (0xffffffff80b35108) locked @ /usr/src/sys/netinet/tcp_usrreq.c:1507 db:1:alllocks> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid = 1 dynamic pcpu = 0xffffff807f6d5780 curthread = 0xffffff0002b07000: pid 0 "em0 rxq" curpcb = 0xffffff824101dd10 fpcurthread = none idlethread = 0xffffff00023778c0: tid 100017 "idle: cpu1" curpmap = 0xffffffff8096b670 tssp = 0xffffffff80b7a768 commontssp = 0xffffffff80b7a768 rsp0 = 0xffffff824101dd10 gs32p = 0xffffffff80b795a0 ldt = 0xffffffff80b795e0 tss = 0xffffffff80b795d0 spin locks held: db:0:kdb.enter.panic> bt Tracing pid 0 tid 100076 td 0xffffff0002b07000 kdb_enter() at kdb_enter+0x3b panic() at panic+0x17b soabort() at soabort+0x9c syncache_expand() at syncache_expand+0x2ca tcp_input() at tcp_input+0xf05 ip_input() at ip_input+0xb3 netisr_dispatch_src() at netisr_dispatch_src+0x9e ether_demux() at ether_demux+0x176 ether_input() at ether_input+0x198 em_rxeof() at em_rxeof+0x19d em_handle_rx() at em_handle_rx+0x1c taskqueue_run_locked() at taskqueue_run_locked+0x93 taskqueue_thread_loop() at taskqueue_thread_loop+0x3f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff824101dd00, rbp = 0 --- 1 0 1 0 SLs wait 0xffffff00023648e0 [init] 10 0 0 0 SL audit_wo 0xffffffff80b3fdd0 [audit] 0 0 0 0 RLs (threaded) kernel 100321 D - 0xffffff01ba75c500 [aiod_bio taskq] 100107 Run CPU 7 [dummynet] 100105 D - 0xffffffff8096e5c4 [deadlkres] 100083 D - 0xffffff0002b21380 [em1 txq] 100081 D - 0xffffff0002b21500 [em1 rxq] 100078 D - 0xffffff0002b0f980 [em0 txq] 100076 Run CPU 1 [em0 rxq] 100053 D - 0xffffff0002466880 [kqueue taskq] 100049 D - 0xffffff00024a2380 [thread taskq] 100047 D - 0xffffff00024a2500 [ffs_trim taskq] 100046 D - 0xffffff0002444280 [acpi_task_2] 100045 D - 0xffffff0002444280 [acpi_task_1] 100044 D - 0xffffff0002444280 [acpi_task_0] 100040 D - 0xffffff0002363880 [firmware taskq] 100000 D sched 0xffffffff8096aca0 [swapper] From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 15:30:11 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 581D61065676 for ; Fri, 27 Jan 2012 15:30:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 36CA08FC12 for ; Fri, 27 Jan 2012 15:30:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0RFUBEM038236 for ; Fri, 27 Jan 2012 15:30:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0RFUBJa038232; Fri, 27 Jan 2012 15:30:11 GMT (envelope-from gnats) Date: Fri, 27 Jan 2012 15:30:11 GMT Message-Id: <201201271530.q0RFUBJa038232@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "Pavel Polyakov" Cc: Subject: Re: kern/159621: [tcp] [panic] panic: soabort: so_count X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Pavel Polyakov List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 15:30:11 -0000 The following reply was made to PR kern/159621; it has been noted by GNATS. From: "Pavel Polyakov" To: bug-followup@freebsd.org, bsd@kobyla.org Cc: Subject: Re: kern/159621: [tcp] [panic] panic: soabort: so_count Date: Fri, 27 Jan 2012 16:44:28 +0200 Dump header from device /dev/ada1p20 Architecture: amd64 Architecture Version: 1 Dump Length: 223744B (0 MB) Blocksize: 512 Dumptime: Tue Jan 24 05:32:13 2012 Hostname: shared Magic: FreeBSD Text Dump Version String: FreeBSD 8.2-STABLE #5 r230331: Mon Jan 23 22:05:26 UTC 2012 Panic String: soabort: so_count Dump Parity: 1380996126 Bounds: 1 Dump Status: good ichwd0: timer reloaded panic: soabort: so_count cpuid = 9 KDB: enter: panic 0xffffff0008b293b0: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xffffff0008657000 (pid 20) 0xffffff00bb6a8000: tag ufs, type VDIR usecount 1, writecount 0, refcount 758 mountedhere 0 flags () v_object 0xffffff00af22d948 ref 0 pages 6032 lock type ufs: EXCL by thread 0xffffff01c252a8c0 (pid 73631) with exclusive waiters pending ino 70625292, on dev mirror/shared0home 0xffffff00b47c0ce8: tag ufs, type VREG usecount 4, writecount 4, refcount 1128 mountedhere 0 flags () v_object 0xffffff0014183360 ref 0 pages 8984 lock type ufs: SHARED (count 4) ino 6584279, on dev mirror/shared0home 0xffffff00bc8b8938: tag ufs, type VREG usecount 4, writecount 0, refcount 5 mountedhere 0 flags () v_object 0xffffff017f4e9438 ref 2 pages 9 lock type ufs: SHARED (count 1) ino 103787584, on dev mirror/shared0home db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks exclusive rw tcpinp (tcpinp) r = 0 (0xffffff01c2425670) locked @ /usr/src/sys/netinet/tcp_input.c:946 exclusive rw tcp (tcp) r = 0 (0xffffffff80b35108) locked @ /usr/src/sys/netinet/tcp_usrreq.c:1507 db:1:locks> show alllocks Process 75843 (php) thread 0xffffff01c2b9a8c0 (100468) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff81ede351c8) locked @ /usr/src/sys/vm/vm_pager.c:311 shared lockmgr ufs (ufs) r = 0 (0xffffff00bc8b89d0) locked @ /usr/src/sys/kern/vfs_subr.c:2170 Process 75837 (php) thread 0xffffff00b723e000 (100232) exclusive sleep mutex pmap (pmap) r = 0 (0xffffff0008f98d70) locked @ /usr/src/sys/amd64/amd64/pmap.c:4363 exclusive sleep mutex vm page queue mutex (vm page queue mutex) r = 0 (0xffffffff80b41880) locked @ /usr/src/sys/vm/vm_object.c:1101 exclusive sleep mutex vm object (standard object) r = 0 (0xffffff00b41de0d8) locked @ /usr/src/sys/vm/vm_object.c:1048 shared sx user map (user map) r = 0 (0xffffff0008f98cb8) locked @ /usr/src/sys/vm/vm_map.c:2023 Process 75836 (php) thread 0xffffff01c2e228c0 (100490) exclusive sleep mutex vm object (standard object) r = 0 (0xffffff00bb23dbd0) locked @ /usr/src/sys/vm/vm_object.c:1048 shared sx user map (user map) r = 0 (0xffffff0008a44078) locked @ /usr/src/sys/vm/vm_map.c:2023 Process 75148 (php) thread 0xffffff010568c8c0 (100744) shared lockmgr ufs (ufs) r = 0 (0xffffff00b47c0d80) locked @ /usr/src/sys/kern/vfs_vnops.c:549 Process 74926 (exim-4.77-0) thread 0xffffff00b71ea000 (100323) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01fc50c8f0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 74366 (exim-4.77-0) thread 0xffffff00b713c000 (100677) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff0060a908f0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 74205 (php) thread 0xffffff01c2c2f8c0 (100694) shared lockmgr ufs (ufs) r = 0 (0xffffff00b47c0d80) locked @ /usr/src/sys/kern/vfs_vnops.c:549 Process 74118 (exim-4.77-0) thread 0xffffff00b7d748c0 (100685) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01fc990e40) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 74105 (php) thread 0xffffff01c262d460 (100442) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff81ee2c5e90) locked @ /usr/src/sys/kern/vfs_bio.c:1891 shared lockmgr ufs (ufs) r = 0 (0xffffff00b47c0d80) locked @ /usr/src/sys/kern/vfs_vnops.c:549 Process 73987 (exim-4.77-0) thread 0xffffff00b715a460 (100223) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00600c68f0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 73728 (php) thread 0xffffff0008f14000 (100195) shared lockmgr ufs (ufs) r = 0 (0xffffff00b47c0d80) locked @ /usr/src/sys/kern/vfs_vnops.c:549 Process 73631 (exim-4.77-0) thread 0xffffff01c252a8c0 (100353) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff81ee2d62b8) locked @ /usr/src/sys/kern/vfs_bio.c:1891 exclusive sx dirhash (dirhash) r = 0 (0xffffff0092e50000) locked @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:216 exclusive lockmgr ufs (ufs) r = 0 (0xffffff00bb6a8098) locked @ /usr/src/sys/kern/vfs_subr.c:2170 Process 73552 (exim-4.77-0) thread 0xffffff0008bda000 (100206) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff005c0b80f8) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 73547 (exim-4.77-0) thread 0xffffff01c252e000 (100349) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01c27d6648) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 2234 (zabbix_agentd) thread 0xffffff0008c018c0 (100272) exclusive sleep mutex sigio lock (sigio lock) r = 0 (0xffffffff8096bb60) locked @ /usr/src/sys/kern/kern_descrip.c:1095 Process 20 (syncer) thread 0xffffff0008657000 (100115) exclusive lockmgr syncer (syncer) r = 0 (0xffffff0008b29448) locked @ /usr/src/sys/kern/vfs_subr.c:1770 Process 12 (intr) thread 0xffffff0002b068c0 (100077) exclusive sleep mutex em0:tx(0) (em0:tx(0)) r = 0 (0xffffff0002561808) locked @ /usr/src/sys/modules/em/../../dev/e1000/if_em.c:1523 Process 12 (intr) thread 0xffffff0002b07460 (100075) exclusive rw tcpinp (tcpinp) r = 0 (0xffffff01c2425670) locked @ /usr/src/sys/netinet/tcp_input.c:946 exclusive rw tcp (tcp) r = 0 (0xffffffff80b35108) locked @ /usr/src/sys/netinet/tcp_usrreq.c:1507 db:1:alllocks> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid = 9 dynamic pcpu = 0xffffff807f70d780 curthread = 0xffffff0002b07460: pid 12 "irq256: em0:rx 0" curpcb = 0xffffff8241018d10 fpcurthread = none idlethread = 0xffffff0002369460: tid 100009 "idle: cpu9" curpmap = 0xffffffff8096b670 tssp = 0xffffffff80b7aaa8 commontssp = 0xffffffff80b7aaa8 rsp0 = 0xffffff8241018d10 gs32p = 0xffffffff80b798e0 ldt = 0xffffffff80b79920 tss = 0xffffffff80b79910 spin locks held: db:0:kdb.enter.panic> bt Tracing pid 12 tid 100075 td 0xffffff0002b07460 kdb_enter() at kdb_enter+0x3b panic() at panic+0x17b soabort() at soabort+0x9c syncache_expand() at syncache_expand+0x2ca tcp_input() at tcp_input+0xf05 ip_input() at ip_input+0xb3 netisr_dispatch_src() at netisr_dispatch_src+0x9e ether_demux() at ether_demux+0x176 ether_input() at ether_input+0x198 em_rxeof() at em_rxeof+0x19d em_msix_rx() at em_msix_rx+0x24 intr_event_execute_handlers() at intr_event_execute_handlers+0x67 ithread_loop() at ithread_loop+0xad fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8241018d00, rbp = 0 --- 6 0 0 0 SL crypto_r 0xffffffff80e03ce0 [crypto returns] 5 0 0 0 SL crypto_w 0xffffffff80e03ca0 [crypto] 13 0 0 0 SL - 0xffffffff8096e5c4 [yarrow] 4 0 0 0 SL - 0xffffffff8096ab68 [g_down] 3 0 0 0 SL - 0xffffffff8096ab60 [g_up] 2 0 0 0 SL - 0xffffffff8096ab50 [g_event] 12 0 0 0 RL (threaded) intr From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 15:42:16 2012 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 202271065693 for ; Fri, 27 Jan 2012 15:42:16 +0000 (UTC) (envelope-from zi@FreeBSD.org) Received: from fast.rit.edu (fast.rit.edu [129.21.182.30]) by mx1.freebsd.org (Postfix) with ESMTP id D2A7E8FC1F for ; Fri, 27 Jan 2012 15:42:15 +0000 (UTC) Received: from fast.rit.edu (localhost.rit.edu [127.0.0.1]) by fast.rit.edu (Postfix) with ESMTP id 23C391D1A6; Fri, 27 Jan 2012 10:14:54 -0500 (EST) X-Virus-Scanned: by amavisd-new at fast.rit.edu Received: from fast.rit.edu ([127.0.0.1]) by fast.rit.edu (fast.rit.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AyfTuTazrTqS; Fri, 27 Jan 2012 10:14:52 -0500 (EST) Received: from syn.rit.edu (syn.rit.edu [129.21.182.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by fast.rit.edu (Postfix) with ESMTPS id C71C81D17F; Fri, 27 Jan 2012 10:14:52 -0500 (EST) Received: from syn.rit.edu (localhost.rit.edu [127.0.0.1]) by syn.rit.edu (8.14.4/8.14.3) with ESMTP id q0RFEqp0081942; Fri, 27 Jan 2012 10:14:52 -0500 (EST) (envelope-from zi@FreeBSD.org) Received: (from zi@localhost) by syn.rit.edu (8.14.4/8.14.3/Submit) id q0RFEpk2076835; Fri, 27 Jan 2012 10:14:51 -0500 (EST) (envelope-from zi@FreeBSD.org) Date: Fri, 27 Jan 2012 10:14:51 -0500 From: Ryan Steinmetz To: Alexey Kouznetsov Message-ID: <20120127151451.GA20090@fast.rit.edu> References: <4ebc3732.86962a0a.7ab9.ffffa29dSMTPIN_ADDED@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: net@FreeBSD.org Subject: Re: Fwd: [ net-snmp-Bugs-3434824 ] coredump on HUP while disks read X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 15:42:16 -0000 This has been completed and is part of net-snmp-5.7.1_4. -r On (01/27/12 17:50), Alexey Kouznetsov wrote: > Hello! > > There are bug in net-snmp. net-snmp die (coredump) on SIGHUP while we have > disk command defined in snmpd.*conf. Bellow fix in main net-snmp tree. I > think it is good idea to add such small patch to the FreeBSD port unlil > they release new version an new version will be included to the ports. You > can see path an butom of mail conversation bellow > > Thank you! > > With best regards > /Alexey > > ---------- Forwarded message ---------- > From: SourceForge.net > Date: 2011/11/11 > Subject: [ net-snmp-Bugs-3434824 ] coredump on HUP while disks read > To: "SourceForge.net" > > > Bugs item #3434824, was opened at 2011-11-07 23:45 > Message generated for change (Comment added) made by nba > You can respond by visiting: > https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3434824&group_id=12694 > > Please note that this message will contain a full copy of the comment > thread, > including the initial issue submission, for this request, > not just the latest update. > Category: agent > Group: None > >Status: Closed > >Resolution: Fixed > Priority: 5 > Private: No > Submitted By: Alexey (st-da) > Assigned to: Niels Baggesen (nba) > Summary: coredump on HUP while disks read > > Initial Comment: > FreeBSD xxx 8.2-STABLE FreeBSD 8.2-STABLE #9: Tue Oct 11 07:07:46 UTC 2011 > root@xxx:/usr/obj/usr/src/sys/AAASMP i386 > but some error reproduced at amd64 system > net-snmp from ports net-snmp-5.7_4 > net snmp started as: > --- > /usr/local/sbin/snmpd -p /var/run/snmpd.pid -Lf /var/log/snmpd.log > --- > > We have commands like: > > disk / > disk /usr > disk /var > in config. > when we > kill -HUP ` cat /var/run/snmpd.pid ` > we got "kernel: pid 61129 (snmpd), uid 0: exited on signal 11 (core dumped)" > > If we remove disk commands error disapear. if we add includeAllDisks or any > single disk we got an error. snmpd.cons is simple some access lines and > this disk lines. snmpd works untill we send sighup. (unlill we logrotate, > and logrotate send sighup for reopen log file). If we disable logging > problem hides (we do not need sent sighup, so it will work) > > gdb output bellow: > > Core was generated by `snmpd'. > Program terminated with signal 11, Segmentation fault. > Reading symbols from /usr/local/lib/libnetsnmpagent.so.30...done. > Loaded symbols for /usr/local/lib/libnetsnmpagent.so.30 > Reading symbols from /usr/local/lib/libnetsnmpmibs.so.30...done. > Loaded symbols for /usr/local/lib/libnetsnmpmibs.so.30 > Reading symbols from /usr/lib/libwrap.so.6...done. > Loaded symbols for /usr/lib/libwrap.so.6 > Reading symbols from /usr/local/lib/libnetsnmp.so.30...done. > Loaded symbols for /usr/local/lib/libnetsnmp.so.30 > Reading symbols from /lib/libm.so.5...done. > Loaded symbols for /lib/libm.so.5 > Reading symbols from /lib/libkvm.so.5...done. > Loaded symbols for /lib/libkvm.so.5 > Reading symbols from /lib/libdevstat.so.7...done. > Loaded symbols for /lib/libdevstat.so.7 > Reading symbols from /lib/libcrypto.so.6...done. > Loaded symbols for /lib/libcrypto.so.6 > Reading symbols from /usr/lib/libelf.so.1...done. > Loaded symbols for /usr/lib/libelf.so.1 > Reading symbols from /lib/libc.so.7...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from /libexec/ld-elf.so.1...done. > Loaded symbols for /libexec/ld-elf.so.1 > #0 0x2814a8a5 in disk_parse_config (token=0xbfbfdb8c "disk", > cptr=0xbfbfdf9c "10240") at ucd-snmp/disk_hw.c:193 > 193 disks[numdisks++] = entry; > > (gdb) where > #0 0x2814a8a5 in disk_parse_config (token=0xbfbfdb8c "disk", > cptr=0xbfbfdf9c "10240") at ucd-snmp/disk_hw.c:193 > #1 0x28269830 in run_config_handler (lptr=0x286878a0, token=0xbfbfdb8c > "disk", cptr=0xbfbfdf91 "/ 10240", when=0) > at read_config.c:546 > #2 0x2826a70a in read_config (filename=0xbfbfe440 > "/usr/local/etc/snmp/snmpd.conf", line_handler=0x2860f080, when=0) > at read_config.c:939 > #3 0x2826b7ae in read_config_files_in_path (path=Variable "path" is not > available. > ) at read_config.c:1282 > #4 0x2826bb86 in read_config_files_of_type (when=0, ctmp=0x28620040) at > read_config.c:1365 > #5 0x2826bc14 in read_config_files (when=0) at read_config.c:1406 > #6 0x2826c39f in read_configs () at read_config.c:1018 > #7 0x280bc928 in update_config () at agent_read_config.c:292 > #8 0x0804b91b in SnmpdReconfig () > #9 0x0804a127 in ?? () > #10 0x00000000 in ?? () > #11 0x00000000 in ?? () > #12 0xbfbfea08 in ?? () > #13 0x0804a127 in ?? () > #14 0x00000005 in ?? () > #15 0xbfbfea30 in ?? () > #16 0xbfbfea48 in ?? () > #17 0xbfbfea10 in ?? () > #18 0xbfbfea2c in ?? () > #19 0x00000000 in ?? () > #20 0xbfbfea28 in ?? () > #21 0x0804a098 in ?? () > Previous frame inner to this frame (corrupt stack?) > > (gdb) list > 292 read_configs(); > 293 } > 294 > 295 > 296 void > 297 snmpd_register_config_handler(const char *token, > 298 void (*parser) (const char *, char *), > 299 void (*releaser) (void), const char > *help) > 300 { > 301 DEBUGMSGTL(("snmpd_register_app_config_handler", > > > > ---------------------------------------------------------------------- > > >Comment By: Niels Baggesen (nba) > Date: 2011-11-10 12:42 > > Message: > Best I can tell this fiexed the problem, and I have applied this fix to > Net-SNMP > > Thanks for the bug report. > > ---------------------------------------------------------------------- > > Comment By: Niels Baggesen (nba) > Date: 2011-11-08 12:55 > > Message: > I think a single missing line is causing the trouble. Please try this: > > --- a/agent/mibgroup/ucd-snmp/disk_hw.c > +++ b/agent/mibgroup/ucd-snmp/disk_hw.c > @@ -137,6 +137,7 @@ disk_free_config(void) > if (disks) { > free( disks ); > disks = NULL; > + maxdisks = numdisks = 0; > } > allDisksIncluded = 0; > } > > > ---------------------------------------------------------------------- > > You can respond by visiting: > https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3434824&group_id=12694 -- Ryan Steinmetz PGP: EF36 D45A 5CA9 28B1 A550 18CD A43C D111 7AD7 FAF2 From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 18:15:43 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD657106566C; Fri, 27 Jan 2012 18:15:43 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mx1.giulioferro.ch (mx1.giulioferro.ch [217.150.252.208]) by mx1.freebsd.org (Postfix) with ESMTP id 681C78FC0A; Fri, 27 Jan 2012 18:15:43 +0000 (UTC) Received: from mailscan.giulioferro.ch (unknown [192.168.115.2]) by mx1.giulioferro.ch (Postfix) with ESMTP id 7F19736F31; Fri, 27 Jan 2012 18:58:51 +0100 (CET) X-Virus-Scanned: amavisd-new at example.com Received: from mx1.giulioferro.ch ([192.168.114.4]) by mailscan.giulioferro.ch (mailscan.giulioferro.ch [192.168.115.2]) (amavisd-new, port 10024) with ESMTP id iCK4WqAniHtD; Fri, 27 Jan 2012 18:58:48 +0100 (CET) Received: from mail.zirakzigil.org (net-93-70-48-129.cust.dsl.vodafone.it [93.70.48.129]) by mx1.giulioferro.ch (Postfix) with ESMTP id 9929036F1B; Fri, 27 Jan 2012 18:58:48 +0100 (CET) Received: from ext.zirakzigil.org (unknown [192.168.1.2]) by mail.zirakzigil.org (Postfix) with ESMTP id 5BDF519282F; Fri, 27 Jan 2012 18:58:48 +0100 (CET) X-Virus-Scanned: amavisd-new at zirakzigil.org Received: from mail.zirakzigil.org ([192.168.1.2]) by ext.zirakzigil.org (ext.zirakzigil.org [192.168.1.2]) (amavisd-new, port 10024) with ESMTP id otg5R2JXZrzg; Fri, 27 Jan 2012 18:58:47 +0100 (CET) Received: from [192.168.10.83] (ext [192.168.1.2]) (Authenticated sender: auryn@zirakzigil.org) by mail.zirakzigil.org (Postfix) with ESMTPA id C0F3D192829; Fri, 27 Jan 2012 18:58:47 +0100 (CET) Message-ID: <4F22E5D7.4000707@zirakzigil.org> Date: Fri, 27 Jan 2012 18:58:47 +0100 From: Giulio Ferro User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" , freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 18:15:43 -0000 I'm trying to setup a kerberized NFS system made of a server and a client (both freebsd 9 amd64 stable) I've tried to follow this howto: http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup But couldn't get much out of it. First question : is this howto still valid or something more recent should be followed? I've searched with Google but I've come up empty. I've set up kerberos heimdal, created the dns entries for both client and server, set up krb5.keytab and copied it to client, set up nfs4 according to man nfsv4: (server) cat /etc/exports V4: /usr/src -sec=krb5:krb5i:krb5p and then tried to mount it from the client: mount_nfs -o ntfsv4,sec=krb5i,gssname=nfs nfsinternal1.dcssrl.it:/usr/src /usr/src but it failed with : [tcp] nfsinternal1.dcssrl.it:/usr/src: Permission denied Can you point me to something that I might have got wrong? Thanks in advance. From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 18:33:06 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB9B2106566B; Fri, 27 Jan 2012 18:33:06 +0000 (UTC) (envelope-from yuri.pankov@gmail.com) Received: from sirius.xvoid.org (sirius.xvoid.org [IPv6:2001:470:28:4ba:20c:29ff:fe62:9a22]) by mx1.freebsd.org (Postfix) with ESMTP id 8C1718FC0C; Fri, 27 Jan 2012 18:33:06 +0000 (UTC) Received: from sirius.xvoid.org (yuri@sirius.xvoid.org [IPv6:::1]) by sirius.xvoid.org (8.14.5/8.14.5) with ESMTP id q0RIX3tq037687; Fri, 27 Jan 2012 22:33:03 +0400 (MSK) (envelope-from yuri.pankov@gmail.com) Received: (from yuri@localhost) by sirius.xvoid.org (8.14.5/8.14.5/Submit) id q0RIX3M7037686; Fri, 27 Jan 2012 22:33:03 +0400 (MSK) (envelope-from yuri.pankov@gmail.com) X-Authentication-Warning: sirius.xvoid.org: yuri set sender to yuri.pankov@gmail.com using -f Date: Fri, 27 Jan 2012 22:33:03 +0400 From: Yuri Pankov To: Giulio Ferro Message-ID: <20120127183303.GG1070@sirius.xvoid.org> References: <4F22E5D7.4000707@zirakzigil.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SO98HVl1bnMOfKZd" Content-Disposition: inline In-Reply-To: <4F22E5D7.4000707@zirakzigil.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" , freebsd-stable@freebsd.org Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 18:33:07 -0000 --SO98HVl1bnMOfKZd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 27, 2012 at 06:58:47PM +0100, Giulio Ferro wrote: > I'm trying to setup a kerberized NFS system made of a server and a > client (both freebsd 9 amd64 stable) >=20 > I've tried to follow this howto: > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup >=20 > But couldn't get much out of it. >=20 > First question : is this howto still valid or something more recent > should be followed? I've searched with Google but I've come up empty. >=20 > I've set up kerberos heimdal, created the dns entries for both > client and server, set up krb5.keytab and copied it to client, set > up nfs4 according to man nfsv4: >=20 > (server) > cat /etc/exports > V4: /usr/src -sec=3Dkrb5:krb5i:krb5p >=20 > and then tried to mount it from the client: >=20 > mount_nfs -o ntfsv4,sec=3Dkrb5i,gssname=3Dnfs=20 > nfsinternal1.dcssrl.it:/usr/src /usr/src > > but it failed with : > [tcp] nfsinternal1.dcssrl.it:/usr/src: Permission denied >=20 > Can you point me to something that I might have got wrong? Not really related to Kerberos question, but.. Some problems here: - ntfsv4 - probably a typo - more serious one - V4: line specifies the ROOT of NFSv4 exported FS - nfsinternal1.dcssrl.it:/usr/src points to /usr/src/usr/src. What you /etc/exports could look like (the way it works for me, doesn't mean that it's correct though): /usr/src V4: / -sec=3Dkrb5:krb5i:krb5p Yuri --SO98HVl1bnMOfKZd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQIcBAABAgAGBQJPIu3fAAoJEF9SuVmZPGsqs0AP/i5DlKKBjM8r4grf0LkWLJmr p6A+AqhBHRE7Ei3I+XxwKGk1gI3uBYTgNpXFNeVlsv2Qf4R+2LdhDSmCj8Z3X16S Y+Ro+lbMP4++sUm44BCouxzx/a9TGzAeW8P9KZwG7DrdreBuVc5FI/WxbyxVTbrW QeEdh7oNhp/yj5S4AkX0Kd2/w1GcwPX/kK8PvdxSOJ6bzSnRvBRiXHq2A5Lm727g vrl+OmwqKf2ibAQQCqKVVfjr9PTR+UQjPeGJnw3lFokOfz4grqDM11aZEtdTK8WT 4aUaarswptDpHEGp7KM9NePa2AqvatlWjfU6u9n66+yg1QyoSVAwrKVacXnNt81k uAHEk0eoI8PSWyunZ0CjAFf7DNe0KcyCgJ8oWqSZSRhuE9yCQ0dSUQtfA5LpRS0n HM6ZPTlcaBqrMxlpaEGHa1dXoQZ75ZnZz2cG/xRTZAhz86rfmqVA3Rl0NxzWBi/+ RcpR7RmuIvzXP0/OcA4WMCmxUU1mmD0MTJNrg++naTVEBS40ulme1bh/y8KbeQin EwiyeNx9t6EXyG/43EqeYUkkMNxke4uvO4Dt98bRhpUG68/I3pqpClLozD46sFRv ZeKvL7z+yBkk6IsHdX/SgMdV262OnCVLezqntDWVQAR9yd6u62hy4gzGbcTtGvsD pNQLZCdWUYo0gaWIdLFH =OhN3 -----END PGP SIGNATURE----- --SO98HVl1bnMOfKZd-- From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 23:52:26 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99FBB106564A for ; Fri, 27 Jan 2012 23:52:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 37A128FC0C for ; Fri, 27 Jan 2012 23:52:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAIs3I0+DaFvO/2dsb2JhbABEhQuqUIFyAQEBAwEBAQEgKyALBRYOCgICDRkCKQEJJgYIBwQBHASHWwimNpFbgS+CUIURAQUDHAQBCwEIAQYEAwMEEIJ7BQMDAQIHAxUBBQsHAgGBGwkGghiBFgSIP4o0gieSaw X-IronPort-AV: E=Sophos;i="4.71,583,1320642000"; d="scan'208";a="157159609" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 27 Jan 2012 18:52:24 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C9274B3EB2; Fri, 27 Jan 2012 18:52:24 -0500 (EST) Date: Fri, 27 Jan 2012 18:52:24 -0500 (EST) From: Rick Macklem To: Giulio Ferro Message-ID: <1755360318.299423.1327708344781.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4F22E5D7.4000707@zirakzigil.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 23:52:26 -0000 Giulio Ferro wrote: > I'm trying to setup a kerberized NFS system made of a server and a > client (both freebsd 9 amd64 stable) > > I've tried to follow this howto: > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup > > But couldn't get much out of it. > > First question : is this howto still valid or something more recent > should be followed? I've searched with Google but I've come up empty. > It's all there is. I don't think anything has changed since it was written. (I haven't had a kerberos setup for about 2 years, so I know I haven't changed anything recently.) It was a google wiki, since I hoped others would add to it, but I don't think that has happened? > I've set up kerberos heimdal, created the dns entries for both > client and server, set up krb5.keytab and copied it to client, set > up nfs4 according to man nfsv4: > > (server) > cat /etc/exports > V4: /usr/src -sec=krb5:krb5i:krb5p > The V4: line doesn't export any file system. It only defines where the root of the directory tree is for NFSv4 and what authentication can be used for "system operations" which do not take any file handle and, therefore, aren't tied to any server file system. For example, the above would need to be something like: V4: /usr/src -sec=krb5:krb5i:krb5p /usr/src -sec=krb5:krb5i:krb5p - If /usr/src is not the root of a file system on the server, it is less confusing to export the root of the file system, such as "/usr" or "/". > and then tried to mount it from the client: > > mount_nfs -o ntfsv4,sec=krb5i,gssname=nfs > nfsinternal1.dcssrl.it:/usr/src /usr/src > To make the "gssname" case work, you need a couple of things: - You need the patch it refers to applied to the client's kernel, so it can handle "host based initiator credentials". After applying the patch, you also need to have an entry in the client's /etc/keytab that looks like: nfs/client-host.dnsdomain@YOUR.REALM Without the above, the client can only do an NFSv4 mount as a user (not root) that has a valid credential. For example: - non-root mounts enabled via # sysctl vfs.usermount=1 - then a user logs in - gets a kerberos TGT via "kinit" - then does a mount command that looks like: % mount -t nfs -o nfsv4,sec=krb5i :/path - this mount breaks if this user's TGT expires, so it either must be maintained via some utility (there are a couple out there, but I can't remember the name of one offhand) or manually by doing "kinit" again before it expires - this user must umount the file system when done with it (I know, it would be nice if the host based initiator cred. worked, "out of the box", but the patch is ugly and the reviewer understandably didn't agree with it. However, I don't know how to do it another way for the version of Heimdal in FreeBSD. There is a bug that has apparently been fixed for newer Heimdal releases, where it gets confused w.r.t. encryption type for the keytab entry unless it is forced to one encryption type only.) Also, you need the following in the server's /etc/rc.conf: nfsv4_server_enable="YES" gssd_enable="YES" and in the client: nfsuserd_enable="YES" gssd_enable="YES" Finally, I'd suggest that you get NFSv4 mounts over "sys" working first and then you can try Kerberos. > but it failed with : > [tcp] nfsinternal1.dcssrl.it:/usr/src: Permission denied > > Can you point me to something that I might have got wrong? > > Thanks in advance. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Jan 27 23:57:25 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9B9A106566C for ; Fri, 27 Jan 2012 23:57:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7C3848FC17 for ; Fri, 27 Jan 2012 23:57:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EALo4I0+DaFvO/2dsb2JhbABEhQuqUIFyAQEEASNWGxgCAg0ZAlkGiA8IpjaRWIEvglCFEQEFAxwEAQsBCAEGBAMDBBASA4JmBQMDAQIHAxUBBQsHAgGDQoEWBIg/jFuSaw X-IronPort-AV: E=Sophos;i="4.71,583,1320642000"; d="scan'208";a="157159970" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 27 Jan 2012 18:57:24 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 974BEB3F27; Fri, 27 Jan 2012 18:57:24 -0500 (EST) Date: Fri, 27 Jan 2012 18:57:24 -0500 (EST) From: Rick Macklem To: Yuri Pankov Message-ID: <2004862701.299511.1327708644586.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20120127183303.GG1070@sirius.xvoid.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, Giulio Ferro , freebsd-stable@freebsd.org Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2012 23:57:25 -0000 Yuri Pankov wrote: > On Fri, Jan 27, 2012 at 06:58:47PM +0100, Giulio Ferro wrote: > > I'm trying to setup a kerberized NFS system made of a server and a > > client (both freebsd 9 amd64 stable) > > > > I've tried to follow this howto: > > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup > > > > But couldn't get much out of it. > > > > First question : is this howto still valid or something more recent > > should be followed? I've searched with Google but I've come up > > empty. > > > > I've set up kerberos heimdal, created the dns entries for both > > client and server, set up krb5.keytab and copied it to client, set > > up nfs4 according to man nfsv4: > > > > (server) > > cat /etc/exports > > V4: /usr/src -sec=krb5:krb5i:krb5p > > > > and then tried to mount it from the client: > > > > mount_nfs -o ntfsv4,sec=krb5i,gssname=nfs > > nfsinternal1.dcssrl.it:/usr/src /usr/src > > > > but it failed with : > > [tcp] nfsinternal1.dcssrl.it:/usr/src: Permission denied > > > > Can you point me to something that I might have got wrong? > > Not really related to Kerberos question, but.. Some problems here: > - ntfsv4 - probably a typo > - more serious one - V4: line specifies the ROOT of NFSv4 exported FS > - nfsinternal1.dcssrl.it:/usr/src points to /usr/src/usr/src. > > What you /etc/exports could look like (the way it works for me, > doesn't > mean that it's correct though): > > /usr/src > V4: / -sec=krb5:krb5i:krb5p > Yes. If you specify "/", then the tree starts at the root. The main problem with doing this is that, for ZFS, you then have to export all file systems from "/" down to where you want to mount. (Again, these are done by export lines separate from the "V4:" line.) If you specify: V4: /usr/src -sec=krb5:krb5i:krb5p /usr/src -sec=krb5:krb5i:krb5p then the client mounts /usr/src via: % mount -t nfs -o nfsv4,sec=krb5i server:/ /mntpoint rick > > Yuri From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 00:00:48 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 569FF1065670 for ; Sat, 28 Jan 2012 00:00:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id F24B58FC08 for ; Sat, 28 Jan 2012 00:00:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAOY5I0+DaFvO/2dsb2JhbABEhQuqUIFyAQEFI0sLGxgCAg0ZAlkGiBemOpFYgS+CUIURAQUDHAQBCwEIAQYEAwMEEBIDgmYFAwMBAgcDFQEFCwcCAYNCgRYEiD+MW5Jr X-IronPort-AV: E=Sophos;i="4.71,583,1320642000"; d="scan'208";a="157160203" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 27 Jan 2012 19:00:47 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 43A62B3F08; Fri, 27 Jan 2012 19:00:47 -0500 (EST) Date: Fri, 27 Jan 2012 19:00:47 -0500 (EST) From: Rick Macklem To: Yuri Pankov Message-ID: <708626908.299589.1327708847263.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20120127183303.GG1070@sirius.xvoid.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, Giulio Ferro , freebsd-stable@freebsd.org Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 00:00:48 -0000 Yuri Pankov wrote: > On Fri, Jan 27, 2012 at 06:58:47PM +0100, Giulio Ferro wrote: > > I'm trying to setup a kerberized NFS system made of a server and a > > client (both freebsd 9 amd64 stable) > > > > I've tried to follow this howto: > > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup > > > > But couldn't get much out of it. > > > > First question : is this howto still valid or something more recent > > should be followed? I've searched with Google but I've come up > > empty. > > > > I've set up kerberos heimdal, created the dns entries for both > > client and server, set up krb5.keytab and copied it to client, set > > up nfs4 according to man nfsv4: > > > > (server) > > cat /etc/exports > > V4: /usr/src -sec=krb5:krb5i:krb5p > > > > and then tried to mount it from the client: > > > > mount_nfs -o ntfsv4,sec=krb5i,gssname=nfs > > nfsinternal1.dcssrl.it:/usr/src /usr/src > > > > but it failed with : > > [tcp] nfsinternal1.dcssrl.it:/usr/src: Permission denied > > > > Can you point me to something that I might have got wrong? > > Not really related to Kerberos question, but.. Some problems here: > - ntfsv4 - probably a typo > - more serious one - V4: line specifies the ROOT of NFSv4 exported FS > - nfsinternal1.dcssrl.it:/usr/src points to /usr/src/usr/src. > > What you /etc/exports could look like (the way it works for me, > doesn't > mean that it's correct though): > > /usr/src > V4: / -sec=krb5:krb5i:krb5p > > > Yuri Btw, Guilio, your email address bounces for me, so hopefully you read the mailing list and see the previous messages. rick From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 01:35:55 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EE411065672; Sat, 28 Jan 2012 01:35:55 +0000 (UTC) (envelope-from weongyo.jeong@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5E2A98FC0C; Sat, 28 Jan 2012 01:35:55 +0000 (UTC) Received: by dadw1 with SMTP id w1so2285934dad.13 for ; Fri, 27 Jan 2012 17:35:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:reply-to:mail-followup-to :mime-version:content-type:content-disposition:organization :x-operation-sytem:user-agent; bh=EgF/QejoGkAVt0ax6hEA8qv2QDrCWO+5h+2B75z9LOE=; b=tCH1qgOsvqsKW2IiFRrE62DMJrRR6WZeCueeTBuYgAPtCpJOS3hEzZ8zeDm+8YzSNG CBFe+gXj10S5ykeA9wG5GlSCUFyRHsXxlbEZeeTGPwUztxzpTnQFDNiWS1KYNIbeoekT xgei3dq9XQ+8HRRLBHRFABR42H3EgQQgmyTdg= Received: by 10.68.195.73 with SMTP id ic9mr18908577pbc.72.1327713138424; Fri, 27 Jan 2012 17:12:18 -0800 (PST) Received: from wgj.corp.aryaka.com ([209.119.61.195]) by mx.google.com with ESMTPS id n8sm24256023pbq.7.2012.01.27.17.12.16 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 27 Jan 2012 17:12:17 -0800 (PST) Received: by wgj.corp.aryaka.com (Postfix, from userid 1000) id EA80B41019; Fri, 27 Jan 2012 17:12:35 -0800 (PST) Date: Fri, 27 Jan 2012 17:12:35 -0800 From: Weongyo Jeong To: kmacy@FreeBSD.org Message-ID: <20120128011235.GC24242@wgj.corp.aryaka.com> Mail-Followup-To: kmacy@FreeBSD.org, freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: CDNetworks. X-Operation-Sytem: FreeBSD User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org Subject: a question about flowtable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Weongyo Jeong List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 01:35:55 -0000 Hello Kip, I had looked flowtable code briefly and still not sure whether I understand it correctly. At this moment I have a question. Is it possible to apply flowtable techniques for forwarding packets? If I understand it right it looks it's impossible at current status because flowtable is only applied when ro == NULL at ip_output(). Is it intentional one? Regards, Weongyo Jeong From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 04:58:51 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C652F106566C; Sat, 28 Jan 2012 04:58:51 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9D5018FC0A; Sat, 28 Jan 2012 04:58:51 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0S4wpxm089470; Sat, 28 Jan 2012 04:58:51 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0S4wpWB089466; Sat, 28 Jan 2012 04:58:51 GMT (envelope-from linimon) Date: Sat, 28 Jan 2012 04:58:51 GMT Message-Id: <201201280458.q0S4wpWB089466@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/164534: [bpf] net.bpf.zerocopy_enable=1 makes pflogd eat cpu and hang X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 04:58:52 -0000 Old Synopsis: net.bpf.zerocopy_enable=1 makes pflogd eat cpu and hang New Synopsis: [bpf] net.bpf.zerocopy_enable=1 makes pflogd eat cpu and hang Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sat Jan 28 04:58:28 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=164534 From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 05:56:09 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 687511065675 for ; Sat, 28 Jan 2012 05:56:09 +0000 (UTC) (envelope-from jshupe@hermetek.com) Received: from alcor.hermetek.com (unknown [IPv6:2607:f348:1007::2]) by mx1.freebsd.org (Postfix) with ESMTP id 28A238FC15 for ; Sat, 28 Jan 2012 05:56:09 +0000 (UTC) Received: from [10.45.29.30] (adsl-64-219-110-27.dsl.lgvwtx.swbell.net [64.219.110.27]) by alcor.hermetek.com (Postfix) with ESMTPSA id 810E13B711 for ; Wed, 25 Jan 2012 00:14:04 -0600 (CST) Message-ID: <4F1F9DAB.6020908@hermetek.com> Date: Wed, 25 Jan 2012 00:14:03 -0600 From: James Shupe Organization: HermeTek Network Solutions User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111124 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4F1F92DE.9060200@zhegan.in> In-Reply-To: <4F1F92DE.9060200@zhegan.in> X-Enigmail-Version: 1.3.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig8674301CEA752B52E1C425DE" Subject: Re: low network speed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 05:56:09 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8674301CEA752B52E1C425DE Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 01/24/2012 11:27 PM, Eugene M. Zheganin wrote: > Hi. >=20 > I'm suffering from low network performance on one of my FreeBSDs. > I have an i386 8.2-RELEASE machine with an fxp(4) adapter. It's > connected though a bunch of catalysts 2950 to another 8.2. While other > machines in this server room using the same sequence of switches and th= e > same target source server (which, btw, is equipped with an em(4) and a > gigabit link bia catalyst 3750) show sufficient speed, this particular > machine while using scp starts with a speed of 200 Kbytes/sec and while= > copying the file shows speed about 600-800 Kbytes/sec. >=20 > I've added this tweak to the sysctl: >=20 > net.local.stream.recvspace=3D196605 > net.local.stream.sendspace=3D196605 > net.inet.tcp.sendspace=3D196605 > net.inet.tcp.recvspace=3D196605 > net.inet.udp.recvspace=3D196605 > kern.ipc.maxsockbuf=3D2621440 > kern.ipc.somaxconn=3D4096 > net.inet.tcp.sendbuf_max=3D524288 > net.inet.tcp.recvbuf_max=3D524288 >=20 > With these settings the copying starts at 9.5 Mbytes/sec speed, but > then, as file is copying, drops down to 3.5 Megs/sec in about two-three= > minutes. >=20 > Is there some way to maintain 9.5 Mbytes/sec (I like this speed more) ?= >=20 >=20 > Thanks. > Eugene. >=20 > P.S. This machine also runs zfs, I don't know if it's important but I > decided to mention it. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" Test IO independently of the network and come back. ZFS takes up a lot of memory and doesn't run amazingly well on i386. However, I doubt it drops your IO speed to under 10MBps. As for the network speed, try using the "-c blowfish" option and see what speeds you get. You fail to mention the processor, and we need to make sure it's not getting choked at the CPU during encryption before moving to the network. Try using iperf too. Remove that crap from sysctl and don't mess with it again. --------------enig8674301CEA752B52E1C425DE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJPH52rAAoJECPibMsISQ9aLv8QALJgKuLjs3m5D5VCjfgnUKd6 8C4fuk2oBl1zGGwXFuAr8oZvkvg+At0qFhdIsdNkYhdCLfp0L6uGkI2C/RzdoHsg h9nLodLGWqNm88MhYYmVEfgNvsB5S3R33Upn0aDzaP7jtFJPUCW+uJgcTqBfZ+F9 mk3qli6AXBLiAZDuof6AQ1dQUmSFK+Zh+Wa4j8ZB5RRjT/apy8Kuy8/JOD0v9HYl n+X8tSVdyWyOMIeOy0d7ZWTaQYNg9+oljk4WLOasWNnZv2cuyd1BGhEx45jKN2RE +XTczuBVP34VzAcZIFZNX1U5lyqp9f5OHTfJhGBWsC0tvXU55pEpkndOrteiABDX LUTRKBgXgB9BZmPFZapqUfFZsF2HCfHALq5Bb6mqDNy3yzg67g9ylNiBsCACJ0Cf YB97rNmoSUdvU27hkrNS7azsV9y5A9chaOjk7QkMqm/oidyrAjWNAf8phaRVpKa0 NIJfC/XRXBCdGq4GSnqhsdjm86FByH+7WQ9ONdk9wRf8vtexu8ARGvdoUEAvCB7f lQxve1MRz/l9kNiPC8IzTvIeFG2kXzFVDHE3Xhwe5zc6dxgyOtELygRllr0KEf3U 9w1u6M+uIWuTLjO28dPA8v6CDz7dgSHpwrSkTFycbV/tNMX2lqWfrAw+rOqJBiEU /hIIFjmKEGwuvqG6aFJj =y/pT -----END PGP SIGNATURE----- --------------enig8674301CEA752B52E1C425DE-- From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 07:06:09 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F1F41065673 for ; Sat, 28 Jan 2012 07:06:09 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id B021F8FC12 for ; Sat, 28 Jan 2012 07:06:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id q0S7669W038679; Sat, 28 Jan 2012 18:06:07 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 28 Jan 2012 18:06:06 +1100 (EST) From: Ian Smith To: Nikolay Denev In-Reply-To: <1A4CBF45-8ABB-4BFB-A83A-2906CBD32667@gmail.com> Message-ID: <20120128175204.L13367@sola.nimnet.asn.au> References: <1A4CBF45-8ABB-4BFB-A83A-2906CBD32667@gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: satish amara , freebsd-net@freebsd.org, Kevin Oberman Subject: Re: stateful firewall implementation in FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 07:06:09 -0000 On Fri, 27 Jan 2012, Nikolay Denev wrote: > On Jan 27, 2012, at 4:41 AM, Kevin Oberman wrote: > > > On Thu, Jan 26, 2012 at 11:41 AM, Chuck Swiger wrote: > >> Hi-- > >> > >> On Jan 26, 2012, at 9:24 AM, satish amara wrote: > >>> I have question regarding the size of the state table kept in FreeBSD for > >>> stateful packet inspection. Say we have a valid senario where we have > >>> stateful firewall rule for HTTP and we get lot of incoming new HTTP session > >>> and state table is filled full. In that case I guess FreeBSD would reject > >>> new sessions. Just want to know what is the latest on this. How does > >>> FreeBSD would handle if the state table is full and we get valid new HTTP > >>> connection. What are options in terms of configuration or new feature in > >>> BSD would address this issue. > >> > >> A securely designed firewall will drop connections when the state table is full. > >> > >> You can increase the size of the state table by following the IPF FAQ: > >> > >> http://www.phildev.net/ipf/IPFques.html#ques25 > >> > >> ...but in point of fact, keeping state for high-volume traffic is generally > >> a losing game, and you are better off (IMHO) setting up stateless bidirectional > >> rules which permit such high volume traffic. > >> > >> HTTP isn't generally too much of a problem, though-- something like a popular > >> stratum-1 or 2 public NTP timeserver will easily blow out a stateful firewall > >> if you try to keep state for NTP's UDP traffic. > > > > To put it very clearly, a stateful firewall "protecting" a server is > > an open invitation to DOS. It is trivial to generate enough UDP > > traffic to overflow any limit on connections in a stateful firewall. > > Various tricks have been tried but the reality is that none has really > > succeeded. Some do help, but nowhere near enough to solve the problem. > > > > Stateful firewalls are for clients and systems that don't provide > > publicly accessible services. Servers require stateless filters along > > with IDS/IPS for effective protection. > > > > And I do expect to get people saying that you HAVE to have a stateful > > firewall is a basic requirement for a device on the Internet. I can > > only say htat I know of many well known servers that do not have them > > and few that do. There is a reason for that. At my old employer we > > were under government security oversight and I can remember the > > auditors back a few years ago who had a fit when told that no firewall > > was employed, just an IDS/IPS with RTBH. The problem is that their red > > team of attackers never could successfully attack which really annoyed > > them to the point that they tryed toi order that the IDS be disabled > > for their attack attempts. (We refused, siting terms of the testing > > agreement.) > > > > Today, auditors still are a bit surprised that they don't use a > > firewall, but are no longer upset by it as they are seeing it more > > often. > > -- > > R. Kevin Oberman, Network Engineer > > E-mail: kob6558@gmail.com > > > > In my experience (and I've had a few DDoS attacks), the state table > was never an issue (unless left at default settings), the machine > would either die from interrupt/cpu overload, or the pipe will be > filled. For example the pf(4) firewall can be tuned to have millions > of state entries, then you can configure thresholds which reached > will make the existing state entries expire sooner, leaving room for > new ones. > > P.S.: Stateful firewalls are required by the PCI DSS (requirement 1.3.6) Some of us will recall when just about anything .gov (or at least .mil) -related had to be written in ADA. One is left to ponder the relative wisdom of that old and this newer requirement :) cheers, Ian From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 08:05:52 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 903DE106566C; Sat, 28 Jan 2012 08:05:52 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mx1.giulioferro.ch (mx1.giulioferro.ch [217.150.252.208]) by mx1.freebsd.org (Postfix) with ESMTP id 423A48FC0A; Sat, 28 Jan 2012 08:05:52 +0000 (UTC) Received: from mailscan.giulioferro.ch (unknown [192.168.115.2]) by mx1.giulioferro.ch (Postfix) with ESMTP id BB88B3683A; Sat, 28 Jan 2012 09:05:50 +0100 (CET) X-Virus-Scanned: amavisd-new at example.com Received: from mx1.giulioferro.ch ([192.168.114.4]) by mailscan.giulioferro.ch (mailscan.giulioferro.ch [192.168.115.2]) (amavisd-new, port 10024) with ESMTP id Pj1SaB2RaZiG; Sat, 28 Jan 2012 09:05:48 +0100 (CET) Received: from mail.zirakzigil.org (net-93-70-48-129.cust.dsl.vodafone.it [93.70.48.129]) by mx1.giulioferro.ch (Postfix) with ESMTP id 7B9A736825; Sat, 28 Jan 2012 09:05:48 +0100 (CET) Received: from ext.zirakzigil.org (unknown [192.168.1.2]) by mail.zirakzigil.org (Postfix) with ESMTP id 88474192235; Sat, 28 Jan 2012 09:05:47 +0100 (CET) X-Virus-Scanned: amavisd-new at zirakzigil.org Received: from mail.zirakzigil.org ([192.168.1.2]) by ext.zirakzigil.org (ext.zirakzigil.org [192.168.1.2]) (amavisd-new, port 10024) with ESMTP id DSMH4B5f1L6M; Sat, 28 Jan 2012 09:05:46 +0100 (CET) Received: from [192.168.229.41] (ext [192.168.1.2]) (Authenticated sender: auryn@zirakzigil.org) by mail.zirakzigil.org (Postfix) with ESMTPA id 79D5C192230; Sat, 28 Jan 2012 09:05:46 +0100 (CET) Message-ID: <4F23AC5A.3080308@zirakzigil.org> Date: Sat, 28 Jan 2012 09:05:46 +0100 From: Giulio Ferro User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org References: <4F22E5D7.4000707@zirakzigil.org> In-Reply-To: <4F22E5D7.4000707@zirakzigil.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 08:05:52 -0000 I forgot to mentioned that I compiled both servers with option KGSSAPI and device crypto, and I enabled gssd on both. Is there anyone who was able to configure this setup? From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 12:33:27 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01AB7106566B for ; Sat, 28 Jan 2012 12:33:27 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 768AE8FC08 for ; Sat, 28 Jan 2012 12:33:26 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0SCIUKX038543; Sat, 28 Jan 2012 13:18:30 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0SCIUEE038542; Sat, 28 Jan 2012 13:18:30 +0100 (CET) (envelope-from marius) Date: Sat, 28 Jan 2012 13:18:30 +0100 From: Marius Strobl To: Randy Bush Message-ID: <20120128121830.GA38513@alchemy.franken.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 12:33:27 -0000 On Thu, Jan 26, 2012 at 12:24:11PM +0900, Randy Bush wrote: > ok, i > o used device.hints to disable both bge interfaces > o booted successfully > o used serial console > o ifconfiged bge0 to the normal addresses > o and it is working > > i suspect that something sucks in bge initialization at startup. > insightful, i know. sorry. > Has that worked before with FreeBSD and if yes with which reversion? Marius From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 12:34:08 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43953106566B; Sat, 28 Jan 2012 12:34:08 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 22A5F8FC0C; Sat, 28 Jan 2012 12:34:08 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1Rr7Tq-000Gv8-91; Sat, 28 Jan 2012 12:34:06 +0000 Date: Sat, 28 Jan 2012 21:34:04 +0900 Message-ID: From: Randy Bush To: Marius Strobl In-Reply-To: <20120128121830.GA38513@alchemy.franken.de> References: <20120128121830.GA38513@alchemy.franken.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 12:34:08 -0000 >> ok, i >> o used device.hints to disable both bge interfaces >> o booted successfully >> o used serial console >> o ifconfiged bge0 to the normal addresses >> o and it is working >> >> i suspect that something sucks in bge initialization at startup. >> insightful, i know. sorry. > > Has that worked before with FreeBSD and if yes with which reversion? yes, the bge and config worked for a long time on 7.foo and 8.foo, most recently 8.2. randy From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 13:18:31 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4F02106566C; Sat, 28 Jan 2012 13:18:31 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 30D5F8FC08; Sat, 28 Jan 2012 13:18:30 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0SDIUBF038763; Sat, 28 Jan 2012 14:18:30 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0SDIUZw038762; Sat, 28 Jan 2012 14:18:30 +0100 (CET) (envelope-from marius) Date: Sat, 28 Jan 2012 14:18:30 +0100 From: Marius Strobl To: Randy Bush Message-ID: <20120128131830.GZ44286@alchemy.franken.de> References: <20120128121830.GA38513@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 13:18:31 -0000 On Sat, Jan 28, 2012 at 09:34:04PM +0900, Randy Bush wrote: > >> ok, i > >> o used device.hints to disable both bge interfaces > >> o booted successfully > >> o used serial console > >> o ifconfiged bge0 to the normal addresses > >> o and it is working > >> > >> i suspect that something sucks in bge initialization at startup. > >> insightful, i know. sorry. > > > > Has that worked before with FreeBSD and if yes with which reversion? > > yes, the bge and config worked for a long time on 7.foo and 8.foo, most > recently 8.2. > Hrm, the problem apparently is that while when probing, the PHY still knows about the media it supports, it just has forgotten about it after the reset during attach. There was a change prior to 8.2 which would turn this from silently being ignored (which generally might or might not work) into resulting what you see now (the upper layers arguably shouldn't trigger a panic in this case though). I can't remember a change to either bge(4) or brgphy(4) between 8.2 and now which could trigger this though. Have you tried to set the loader-tunable hw.bge.allow_asf to 0? The default for that option still is different between 8 and 9+. Marius From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 15:54:56 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93D831065672; Sat, 28 Jan 2012 15:54:56 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 32CCA8FC08; Sat, 28 Jan 2012 15:54:56 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAMcYJE+DaFvO/2dsb2JhbABDhQuqU4FyAQEFI1YbDgoCAg0ZAlkGrzaRKYEvhw0BBQMcBAELAQgBBgQDAwQQFYJmBQMDAQIHAxUBBQsHAgGBGwmCHoEWBIg/jFuSbw X-IronPort-AV: E=Sophos;i="4.71,585,1320642000"; d="scan'208";a="154113025" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 28 Jan 2012 10:54:55 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3BE97B3F22; Sat, 28 Jan 2012 10:54:55 -0500 (EST) Date: Sat, 28 Jan 2012 10:54:55 -0500 (EST) From: Rick Macklem To: Giulio Ferro Message-ID: <1721865563.311886.1327766095191.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4F23AC5A.3080308@zirakzigil.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 15:54:56 -0000 Giulio Ferro wrote: > I forgot to mentioned that I compiled both servers with > option KGSSAPI and device crypto, and I enabled gssd > on both. > > Is there anyone who was able to configure this setup? > I had a server at the nfsv4 testing event last June and it worked ok. I haven't tried one since then. Step 1: make sure that nfsv4 mounts work over auth_sys. (You'll need to add "sys" to the sec= flavours, so your /etc/exports will look something like: V4: /usr/src -sec=sys:krb5:krb5i:krb5p /usr/src -sec=sys:krb5:krb5i:krb5p Then on the client: # mount -t nfs -o nfsv4 :/ / (Where "<" and ">" indicate "replace this with what yours".) - Then cd / and do an "ls -l" to see that the file ownership looks ok. If it doesn't, it will be related to "nfsuserd", which must be running in both client and server. Once, Step 1 looks fine: Step 2: Check that Kerberos is working ok in the server. - Log into the server as root and do the following: # kinit -k nfs/@ - This should work ok. # klist - This should list a TGT for nfs/@ If this doesn't work, something isn't right in the Kerberos setup on the server. The NFS server (not client) must have a /etc/krb5.keytab file with an entry for: nfs/@ in it. You should create it on your KDC with encryption type DES-CBC_CRC initially and you should specify that as your default enctype in your /etc/krb5.conf. Once that is working, make sure all the daemons are running on the server. mountd, nfsd, nfsuserd and gssd If this all looks good, go to the client: # sysctl vfs.usermount=1 - make sure these daemons are running nfsuserd, gssd - Log in as non-root user: % kinit % klist - there should be a TGT for the user you are logged in as - Now, try a kerberos mount, as follows: % mount -t nfs -o nfsv4,sec=krb5 :/ / - if that works % cd / % ls -l If these last steps fail, it is not easy to figure out why. (Look in /var/log/messages for any errors. If you get what the gssd calls an minor status, that is the kerberos error.) rick From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 16:18:16 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A481E106566C; Sat, 28 Jan 2012 16:18:16 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mx1.giulioferro.ch (mx1.giulioferro.ch [217.150.252.208]) by mx1.freebsd.org (Postfix) with ESMTP id 5223E8FC0C; Sat, 28 Jan 2012 16:18:16 +0000 (UTC) Received: from mailscan.giulioferro.ch (unknown [192.168.115.2]) by mx1.giulioferro.ch (Postfix) with ESMTP id DF38B37CC3; Sat, 28 Jan 2012 17:18:14 +0100 (CET) X-Virus-Scanned: amavisd-new at example.com Received: from mx1.giulioferro.ch ([192.168.114.4]) by mailscan.giulioferro.ch (mailscan.giulioferro.ch [192.168.115.2]) (amavisd-new, port 10024) with ESMTP id FrXwqxLDBs9M; Sat, 28 Jan 2012 17:18:13 +0100 (CET) Received: from mail.zirakzigil.org (net-93-70-48-129.cust.dsl.vodafone.it [93.70.48.129]) by mx1.giulioferro.ch (Postfix) with ESMTP id 1018D37CB9; Sat, 28 Jan 2012 17:18:13 +0100 (CET) Received: from ext.zirakzigil.org (unknown [192.168.1.2]) by mail.zirakzigil.org (Postfix) with ESMTP id BF041192A78; Sat, 28 Jan 2012 17:18:12 +0100 (CET) X-Virus-Scanned: amavisd-new at zirakzigil.org Received: from mail.zirakzigil.org ([192.168.1.2]) by ext.zirakzigil.org (ext.zirakzigil.org [192.168.1.2]) (amavisd-new, port 10024) with ESMTP id 58-_-X-aiwLh; Sat, 28 Jan 2012 17:18:12 +0100 (CET) Received: from [192.168.229.36] (ext [192.168.1.2]) (Authenticated sender: auryn@zirakzigil.org) by mail.zirakzigil.org (Postfix) with ESMTPA id 7C560192A72; Sat, 28 Jan 2012 17:18:12 +0100 (CET) Message-ID: <4F241FC4.20508@zirakzigil.org> Date: Sat, 28 Jan 2012 17:18:12 +0100 From: Giulio Ferro User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org References: <1721865563.311886.1327766095191.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1721865563.311886.1327766095191.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: kerberized NFS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 16:18:16 -0000 Thank you to all of you for your replies. I'll try next week and let you know. My mail server was down for a few hours, but everything should be ok now... Giulio. From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 18:13:43 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C40E1065673 for ; Sat, 28 Jan 2012 18:13:43 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 053098FC08 for ; Sat, 28 Jan 2012 18:13:42 +0000 (UTC) Received: by eaaa14 with SMTP id a14so1070526eaa.13 for ; Sat, 28 Jan 2012 10:13:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:reply-to:user-agent:mime-version:to :subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=vLNitgCl3UNs1PmOEAhJPcI01RxATIfiau8pWR0ZLMI=; b=TnKUN06p6bKwgnd0TtXlBJPL9zbxozziSXC8X3YRoHve//9q4SZUSAMWD2bkKH32hl Hd/nDX50qV40fYwt7iSRQKQ3oiFrVwBJZbXktKeZo3mPJKxC3unTLHHHr0tEYgIkco/f 6Ktyq7mNyMERSCev4C+ULQwew/kyWdN32vzrc= Received: by 10.213.16.75 with SMTP id n11mr1829226eba.106.1327774421945; Sat, 28 Jan 2012 10:13:41 -0800 (PST) Received: from [192.168.0.109] ([95.232.4.130]) by mx.google.com with ESMTPS id b49sm46833916eec.9.2012.01.28.10.13.39 (version=SSLv3 cipher=OTHER); Sat, 28 Jan 2012 10:13:40 -0800 (PST) Sender: K Macy Message-ID: <4F243AD5.6070902@freebsd.org> Date: Sat, 28 Jan 2012 19:13:41 +0100 From: Kip Macy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Thunderbird/3.1.16 MIME-Version: 1.0 To: freebsd-net@freebsd.org, weongyo@freebsd.org References: <20120128011235.GC24242@wgj.corp.aryaka.com> In-Reply-To: <20120128011235.GC24242@wgj.corp.aryaka.com> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: a question about flowtable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: kmacy@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 18:13:43 -0000 On 01/28/2012 02:12 AM, Weongyo Jeong wrote: > Hello Kip, > > I had looked flowtable code briefly and still not sure whether I > understand it correctly. At this moment I have a question. > > Is it possible to apply flowtable techniques for forwarding packets? If > I understand it right it looks it's impossible at current status because > flowtable is only applied when ro == NULL at ip_output(). Is it > intentional one? > You can pass in a struct route filled in by a flowtable lookup in ip_output. I have made this change in a number of branches and I know at least one firewall is seeing good results from doing this. The one thing to be careful about is that the number of cached flows scales with the number of IPs and not the number of prefixes. Cheers, Kip From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 19:56:31 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 245C9106564A; Sat, 28 Jan 2012 19:56:31 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 062A98FC16; Sat, 28 Jan 2012 19:56:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RrENy-000HiH-4x; Sat, 28 Jan 2012 19:56:30 +0000 Date: Sun, 29 Jan 2012 04:56:28 +0900 Message-ID: From: Randy Bush To: Marius Strobl In-Reply-To: <20120128131830.GZ44286@alchemy.franken.de> References: <20120128121830.GA38513@alchemy.franken.de> <20120128131830.GZ44286@alchemy.franken.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 19:56:31 -0000 > Hrm, the problem apparently is that while when probing, the PHY > still knows about the media it supports, it just has forgotten > about it after the reset during attach. There was a change prior > to 8.2 which would turn this from silently being ignored (which > generally might or might not work) into resulting what you see > now (the upper layers arguably shouldn't trigger a panic in this > case though). I can't remember a change to either bge(4) or > brgphy(4) between 8.2 and now which could trigger this though. > Have you tried to set the loader-tunable hw.bge.allow_asf to 0? > The default for that option still is different between 8 and 9+. it no longer panics when booting, but the interface comes up not seeing carrier bge0: flags=8843 metric 0 mtu 1500 options=8009b ether 00:30:48:82:11:a2 inet 198.180.150.1 netmask 0xffffff80 broadcast 198.180.150.127 inet6 fe80::230:48ff:fe82:11a2%bge0 prefixlen 64 scopeid 0x1 inet6 2001:418:8006::1 prefixlen 64 inet 198.180.150.2 netmask 0xffffffff broadcast 198.180.150.2 nd6 options=21 media: Ethernet 1000baseT (none) status: no carrier randy From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 20:45:03 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76C67106566C; Sat, 28 Jan 2012 20:45:03 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 4D8798FC08; Sat, 28 Jan 2012 20:45:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RrF8w-000I19-Q8; Sat, 28 Jan 2012 20:45:02 +0000 Date: Sun, 29 Jan 2012 05:45:01 +0900 Message-ID: From: Randy Bush To: Marius Strobl In-Reply-To: References: <20120128121830.GA38513@alchemy.franken.de> <20120128131830.GZ44286@alchemy.franken.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 20:45:03 -0000 >> Have you tried to set the loader-tunable hw.bge.allow_asf to 0? >> The default for that option still is different between 8 and 9+. > it no longer panics when booting, but the interface comes up not > seeing carrier an additional datum. o with hw.bge.allow_asf untouched, i.e. default o with /boot/device.hints having hint.bge.0.disabled=1 o with /etc/rc.conf having ifconfig_bge0="198.180.150.1/25 media 1000baseTX" ifconfig_bge0_alias0="inet 198.180.150.2/32" etc o it boots but bge0 carrier is down and can not be brought up comment out the bge0 entries in /etc/rc.conf, and it boots, carrier is up, and can be hand configured with the address assignments, and works. randy From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 21:38:59 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BA901065673; Sat, 28 Jan 2012 21:38:59 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id F2B0F8FC12; Sat, 28 Jan 2012 21:38:58 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0SLcv9H040335; Sat, 28 Jan 2012 22:38:57 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0SLcvXQ040334; Sat, 28 Jan 2012 22:38:57 +0100 (CET) (envelope-from marius) Date: Sat, 28 Jan 2012 22:38:57 +0100 From: Marius Strobl To: Randy Bush Message-ID: <20120128213857.GB39861@alchemy.franken.de> References: <20120128121830.GA38513@alchemy.franken.de> <20120128131830.GZ44286@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Stable , FreeBSD Net Subject: Re: 9-stable - ifmedia_set: no match for 0x0/0xfffffff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 21:38:59 -0000 On Sun, Jan 29, 2012 at 04:56:28AM +0900, Randy Bush wrote: > > Hrm, the problem apparently is that while when probing, the PHY > > still knows about the media it supports, it just has forgotten > > about it after the reset during attach. There was a change prior > > to 8.2 which would turn this from silently being ignored (which > > generally might or might not work) into resulting what you see > > now (the upper layers arguably shouldn't trigger a panic in this > > case though). I can't remember a change to either bge(4) or > > brgphy(4) between 8.2 and now which could trigger this though. > > Have you tried to set the loader-tunable hw.bge.allow_asf to 0? > > The default for that option still is different between 8 and 9+. > > it no longer panics when booting, but the interface comes up not seeing > carrier > > bge0: flags=8843 metric 0 mtu 1500 > options=8009b > ether 00:30:48:82:11:a2 > inet 198.180.150.1 netmask 0xffffff80 broadcast 198.180.150.127 > inet6 fe80::230:48ff:fe82:11a2%bge0 prefixlen 64 scopeid 0x1 > inet6 2001:418:8006::1 prefixlen 64 > inet 198.180.150.2 netmask 0xffffffff broadcast 198.180.150.2 > nd6 options=21 > media: Ethernet 1000baseT (none) > status: no carrier > Are you sure that the other end is also forced to 1000baseT half-duplex? What happens if you set hw.bge.allow_asf to 0 and use auto-negotiation on both sides? Marius From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 22:38:36 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99D72106566C; Sat, 28 Jan 2012 22:38:36 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0401D8FC13; Sat, 28 Jan 2012 22:38:35 +0000 (UTC) Received: by eaaa14 with SMTP id a14so1120452eaa.13 for ; Sat, 28 Jan 2012 14:38:34 -0800 (PST) Received: by 10.213.25.78 with SMTP id y14mr1955869ebb.139.1327788787920; Sat, 28 Jan 2012 14:13:07 -0800 (PST) Received: from rnote.ddteam.net (238-239-92-178.pool.ukrtel.net. [178.92.239.238]) by mx.google.com with ESMTPS id n56sm49505525eeh.6.2012.01.28.14.13.04 (version=SSLv3 cipher=OTHER); Sat, 28 Jan 2012 14:13:06 -0800 (PST) Date: Sun, 29 Jan 2012 00:12:51 +0200 From: Aleksandr Rybalko To: Adrian Chadd Message-Id: <20120129001251.7e4cfe83.ray@ddteam.net> In-Reply-To: References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Aleksandr Rybalko , Stefan Bethke , freebsd-arch@freebsd.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 22:38:36 -0000 On Wed, 25 Jan 2012 09:57:32 +0100 Stefan Bethke wrote: > My suggestion is to take my bus attachment code (incl. mdio and > miiproxy) and ray's ioctl and userland code. As I see from your patch, mdio/miiproxy require special bits in MAC driver. When I design switch framework, I keeping in mind that MAC drivers should be standard as possible, that why I send you patch http://my.ddteam.net/files/2012-01-22_arge.patch which clean most phymask features from if_arge driver. You may ask me why I do so? It is because arge H/W is not the single implementation, it is just FPGA design that also used in many other devices, f.e. if_et. Look into dev/et/if_etreg.h, begin from line: #define ET_MAC_CFG1 0x5000 There is the same registers, same logic, just mapped at 0x5000 in device PCI BAR, instead of 0x19000000(arge) and I bet it is not last time when that FPGA design used :) > > Aleksandr's approach for the driver attachment is to have a generic > switch "bus" driver that abstracts the mii, i2c, memory mapped I/O, > etc. busses the devices are physically attached to, and present a > uniform register file to the chip-specific switch driver. I believe > that this is unnecessarily complicated for two reasons: newbus > already provides that abstraction, and chip-specific drivers usually > differ in so many aspects, including their register files, that code > sharing between chips will be somewhat limited anyway. newbus allow attach anything to anything, but bus interfaces implemented in different ways (for mem/mdio we call read/write, for SPI/IIC we call transfer). When we made that interfaces consistent we be able really forget about "bus glue". While we still not done it(even still not doing it), model with single parent (switch0) require bus glue for each supported interface (MDIO, MEM, SPI, etc.). But model with direct attach driver to bus will require bus glue per driver. If only one interface is supported, then glue in driver file, else - separate file per each supported interface. And two words about "complicated": 1. If we about complicated structure of devices - yes switch0 with bcm5325_switch0 more complicated, than just bcm5325_switch0, but device tree will care about it. 2. If we about code size, then I will say my model much smaller even having more drivers. My personal decision - is 2, because - less code better for maintenance. > > One aspect that I would enjoy looking into in more detail is how > register accesses on, for example, MDIO, can be provided through the > bus space API. From my cursory reading, it seems that the code > currently is tailored towards register accesses that can be > implemented through CPU native instructions, instead of indirectly > through a controller. > > Aleksandr has defined a quite comprehensive ethernet switch control > API that the framework provides towards in-kernel clients as well as > userland. I think it would be really helpful if we could concentrate > on those API functions that can be controlled through the userland > utility, have immediate use cases (for example, VLAN configuration on > the TL-WR1043ND to separate the WAN from the LAN ports), and we have > test hardware for. In short, don't commit dead code. It is not dead code, it is TODO :) > > Having a description of the generic switch model that the API assumes > and driver-specific documentation also wouldn't hurt. (Yes, I'm > volunteering.) It is also TODO :) On Thu, 26 Jan 2012 22:03:58 -0800 Adrian Chadd wrote: > Ok, I do like the idea of: > > * mdiobus/miibus proxy tidyup; > * then the switch API; > * then the switch devices themselves. > > Can we get some consensus/agreement from Marius (and others) about the > first step? I think we don't need to "rewrite" miibus now. :) > > > Adrian WBW -- Aleksandr Rybalko From owner-freebsd-net@FreeBSD.ORG Sat Jan 28 23:00:23 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4CDDC1065670 for ; Sat, 28 Jan 2012 23:00:23 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id CECED8FC08 for ; Sat, 28 Jan 2012 23:00:22 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so3190234wgb.31 for ; Sat, 28 Jan 2012 15:00:21 -0800 (PST) Received: by 10.180.89.71 with SMTP id bm7mr14901816wib.20.1327791621197; Sat, 28 Jan 2012 15:00:21 -0800 (PST) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.227.208.210 with HTTP; Sat, 28 Jan 2012 15:00:01 -0800 (PST) In-Reply-To: <20120129001251.7e4cfe83.ray@ddteam.net> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> From: Juli Mallett Date: Sat, 28 Jan 2012 15:00:01 -0800 X-Google-Sender-Auth: -ga7FvK7qbBxsus2YXrZSJahcsA Message-ID: To: Aleksandr Rybalko Content-Type: text/plain; charset=UTF-8 Cc: freebsd-net@freebsd.org, Adrian Chadd , freebsd-arch@freebsd.org, Aleksandr Rybalko , Stefan Bethke Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 23:00:23 -0000 On Sat, Jan 28, 2012 at 14:12, Aleksandr Rybalko wrote: > As I see from your patch, mdio/miiproxy require special bits in MAC > driver. When I design switch framework, I keeping in mind that > MAC drivers should be standard as possible I don't understand why this is desirable in practice. It's a nice theory, but it falls down when one thinks in depth about how Ethernet interfaces are used and administered vs. how switches are used and administered. What should media report? What should media changes do? What is link status? Do you show the CPU-to-switch port, or all switch ports? It makes me wonder if the understanding of the relationship in FreeBSD isn't backwards. Yes, the MAC sits on a bus and is memory-mapped, but you can conceptualize of it as a child of the PHY, rather than the parent of it, especially in systems with switch chipsets. Especially in systems where there is a switch chipset attached to multiple MACs. In that model, it makes sense to semi-generically attach a CPU-to-switch port's pseudo-PHY (or actual PHY, depending on hardware) to a MAC generically, but that doesn't meant that the switch itself is attached generically to the MAC. There are a lot of switches out there that don't look or act much like MII-driven PHYs, but which are connected over MDIO, as I've tried to stress before. I hope there will be as much separation between the MII work that is being done and the switch work that is being done as possible. I think anything else will prove rapidly-obsolete and perhaps even obstructive as soon as anyone seeks to add support for more switch chipsets to FreeBSD. I suppose the most likely reality, though, is that people simply won't add switch support to FreeBSD, and will administer them out-of-band from userland, using gross kernel shims. That is probably true whether any of the currently-outstanding work is committed or not, unfortunately :( I just hope we'll end up with something flexible enough, broad enough in applicability, narrow enough in requirements, etc., that we'll have feature-rich support for a few chipsets in tree that work in sufficiently-different manners that they can be models for other drivers in the future. Thanks, Juli.