From owner-freebsd-current@freebsd.org Sun Sep 24 07:32:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F5A9E2156B for ; Sun, 24 Sep 2017 07:32:14 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B6456CE8A for ; Sun, 24 Sep 2017 07:32:12 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from thor.intern.walstatt.dynvpn.de ([78.53.137.92]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0Ld1CS-1dVnNb3r7o-00iBmj for ; Sun, 24 Sep 2017 09:32:04 +0200 Date: Sun, 24 Sep 2017 09:31:51 +0200 From: "O. Hartmann" To: FreeBSD CURRENT Subject: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem Message-ID: <20170924093151.2edfeeff@thor.intern.walstatt.dynvpn.de> Organization: WALSTATT User-Agent: OutScare 3.1415926 X-Operating-System: ImNotAnOperatingSystem 3.141592527 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/CavIwlhnZjRtdhxPEj_w8as"; protocol="application/pgp-signature" X-Provags-ID: V03:K0:CUaVvU5Ln4Wr4QVxg4LD9QbL2SvDlCdrgbEJjvMktWcpr7fAdJu /sPm9UAOugeLEyKiwjmGLt109Zc7NmgRso3KDO3XBn4SSTvqufS6hdPScOyJUr9M0qlumzT 9zUZw/i5XNcZqWqTsspQOcsDbDBGPoy2TUDKpOWhxTBypj/TAUoBAOp7uVZzvpttVYJcIcH jIzp5rOGhS2lHoK3LSAcQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:d4Aj5uAB2UA=:3ZKF0CqJfO6W8YCZvZ8ghU O4ryUp5XJGAUDoOyTeQR3BL7XtkbGL4Ay0Wln0rVE5D7Gzq3kUZzTeB8rMA4xPRT5UlLZ4T0F pKA6xuEhXhD4VhXpVfTIIVYAkXGHza/r1jsn9nfr4RBAohBF2vieuwwmt0E8Xwp4LPIWti3s4 4SB/f1K0/DGHCA3O8YDSgYW+dErxH4UyjmFe49OnKJqYeRREANcgZlvmoACyDLFp8oxljeeKj 1BLbdnGdh0ORXcDzG9JAmAAZ/W/W+iKtRL3OLLzl65+3p82oBWA6MOs6MpJ3JkN41SzBJ4gjj R6IEvUkeKAloSdsGWg2Gcj7U3PYOmTQ5+lE+N9TsIfkWr1YyGLSaZhNCekJ7yKC0ZfzYg0Iw+ QDUzGzzWbdow4Ob+6rgHzeN4jJD/A0tM/OHhXzQ5KcMzwjMgwkt//oGDYXKzPHP5pFgpFF/9M bTLb4p9oHHJkOQDrex6Vm0KeFPMTNUQ7eIJivU2KSfhZmX/vBfzXOdKnlhe+KB4EmllGV/ul/ GNOxQA7APjJciVme4qYYg8QjS3kH1R1oJea9nNJYsz1H/2Ff7C4kTAhOLd4jQcnTNqDnufNLc YM3rdqptWlz2p63qeMD7+KiFXRBHD+tib2M56F1brl3hLLOU5zD/Vc1lEqMGm9HYflnqkYTaT DG6XW/iDXrmyanutfTpMRXUfwCJqDeQt+jlWMQHv66Csfw7WH19ssv/cPruBtXMi19Hciek9d 2WBTxmF9x8gxKe4/dKPpdZs1MARgO96RODV9HVMGbBrrsJc4/77mM87wRUlVth0pfyJ9biNis s+JFWt493o6QMrY/25vMGn+/XJivg== X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 07:32:14 -0000 --Sig_/CavIwlhnZjRtdhxPEj_w8as Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Running a NanoBSD application on a small PCEngine device, feeding the box w= ith a CURRENT (FreeBSD 12.0-CURRENT #33 r323958: Sat Sep 23 22:19:57 CEST 2017 amd64), bo= oting from a SD memory card. Recently, when saving configuration changes on the SD memory card, I receiv= e these strange errors: [...] checksum failed: cg 0, cgp: 0xb478d1d5 !=3D bp: 0x874608df checksum failed: cg 12, cgp: 0xe626d67c !=3D bp: 0x7360c12a checksum failed: cg 0, cgp: 0xb478d1d5 !=3D bp: 0x874608df handle_workitem_freefile: got error 5 while accessing filesystem checksum failed: cg 12, cgp: 0xe626d67c !=3D bp: 0x7360c12a handle_workitem_freefile: got error 5 while accessing filesystem [...] I do not use MMCCAM in the kernel, but as far as I know, the problem occure= d in the same time frame.=20 The question is: is the SD memory about to die and is to be replaced or is = this a bug introduced with the new MMCCAM stuff recently? Thanks in advance. oh --=20 O. Hartmann Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Abs.= 4 BDSG). --Sig_/CavIwlhnZjRtdhxPEj_w8as Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWcdfZwAKCRDS528fyFhY lKFKAf0eWTym2AYJaphOV0zP0KEqJyvkJLkgSwZEqlAOY3foUOjYkbbmaj5Y1H+c w/ccYvTaKttr+YnSkerXWhGFbJgpAf467fNsd7cbQ+/RwmOxmhBgx5qGmSFG+VxX NyU9dg3npdHTQ4gJwESh1d1qA7aYepNd4xpV2AoL3UdyHFb62Au6 =q6WE -----END PGP SIGNATURE----- --Sig_/CavIwlhnZjRtdhxPEj_w8as-- From owner-freebsd-current@freebsd.org Sun Sep 24 07:52:44 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D4C1E21F16 for ; Sun, 24 Sep 2017 07:52:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8E2F6D7C9 for ; Sun, 24 Sep 2017 07:52:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8O7qces025092 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 24 Sep 2017 10:52:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8O7qces025092 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8O7qcER025091; Sun, 24 Sep 2017 10:52:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 24 Sep 2017 10:52:38 +0300 From: Konstantin Belousov To: "O. Hartmann" Cc: FreeBSD CURRENT Subject: Re: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem Message-ID: <20170924075238.GS2271@kib.kiev.ua> References: <20170924093151.2edfeeff@thor.intern.walstatt.dynvpn.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170924093151.2edfeeff@thor.intern.walstatt.dynvpn.de> User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 07:52:44 -0000 On Sun, Sep 24, 2017 at 09:31:51AM +0200, O. Hartmann wrote: > Running a NanoBSD application on a small PCEngine device, feeding the box with a CURRENT > (FreeBSD 12.0-CURRENT #33 r323958: Sat Sep 23 22:19:57 CEST 2017 amd64), booting from a > SD memory card. > > Recently, when saving configuration changes on the SD memory card, I receive these > strange errors: > > [...] > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > handle_workitem_freefile: got error 5 while accessing filesystem > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > handle_workitem_freefile: got error 5 while accessing filesystem > [...] > > I do not use MMCCAM in the kernel, but as far as I know, the problem occured in the same > time frame. > > The question is: is the SD memory about to die and is to be replaced or is this a bug > introduced with the new MMCCAM stuff recently? The messages you see about 'checksum_failed' are due to recently added UFS feature where cylinder group blocks are checksummed, and the checksum is verified against the known one stored on the volume. Were there kernel messages about disk I/O errors before that (as opposed to UFS errors about checksums and SU complains) ? From owner-freebsd-current@freebsd.org Sun Sep 24 09:14:57 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7F14E23974 for ; Sun, 24 Sep 2017 09:14:57 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A66B06F6CD for ; Sun, 24 Sep 2017 09:14:57 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from fortune.joker.local (124-18-21-125.dz.commufa.jp [124.18.21.125]) (authenticated bits=0) by dec.sakura.ne.jp (8.15.2/8.15.2/[SAKURA-WEB]/20080708) with ESMTPA id v8O8NBn5030152 for ; Sun, 24 Sep 2017 17:23:11 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 24 Sep 2017 17:23:11 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: USB optical drive fails to attach by CAM status 0x50 Message-Id: <20170924172311.3ab66d18a0c80eb92b1e7bcd@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.6.0 (GTK+ 2.24.31; amd64-portbld-freebsd11.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 09:14:58 -0000 Hi. USB Blu-ray drive I-O DATA BRP-UT6SK fails to attach with verbose dmesg below. Maybe some quirks should be added or removed, but no idea. No change with / without genuine optional AC adapter, and I saw the drive itself was successfully playing DVD video on Windoze10 environment, so not physically broken at least for read access. Both head and stable/11 got the same error. IIRC, quirks 0x0100 was set on head instead of 0xc101 on stable/11. Any ideas? This drive is known to have Matsushita or Pioneer drive inside. I couldn't find results of Pioneer drives (I don't have one). *No USB3 port on the host, so attached as USB2 device. [Related verbose dmesg, stable/11, obtained at r323431, amd64] ugen1.4: at usbus1 umass0 on uhub2 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0xc101 umass0:5:0: Attached to scbus5 random: harvesting attach, 8 bytes (4 bits) from umass0 (probe0:umass-sim0:0:0:0): Down reving Protocol Version from 2 to 0? GEOM: new disk cd0 pass3 at umass-sim0 bus 0 scbus5 target 0 lun 0 pass3: Removable CD-ROM SCSI device pass3: 40.000MB/s transfers (cd0:umass-sim0:0:0:0): got CAM status 0x50 (cd0:umass-sim0:0:0:0): fatal error, failed to attach to device Opened disk cd0 -> 6 g_access(918): provider cd0 has error 6 set g_access(918): provider cd0 has error 6 set g_access(918): provider cd0 has error 6 set g_access(918): provider cd0 has error 6 set [usbconfig outputs] #usbconfig -d ugen1.4 dump_info ugen1.4: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA) # usbconfig -d ugen1.4 dump_device_desc ugen1.4: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0210 bDeviceClass = 0x0000 bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x04bb idProduct = 0x022f bcdDevice = 0x0320 iManufacturer = 0x0001 iProduct = 0x0002 iSerialNumber = 0x0003 <001609230892> bNumConfigurations = 0x0001 #usbconfig -d ugen1.4 dump_curr_config_desc ugen1.4: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA) Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x0020 bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 bmAttributes = 0x0080 bMaxPower = 0x00fa Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0002 bInterfaceClass = 0x0008 bInterfaceSubClass = 0x0006 bEndpointAddress = 0x0081 bmAttributes = 0x0002 wMaxPacketSize = 0x0200 bInterval = 0x0000 bRefresh = 0x0000 bSynchAddress = 0x0000 Endpoint 1 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0002 bmAttributes = 0x0002 wMaxPacketSize = 0x0200 bInterval = 0x0000 bRefresh = 0x0000 bSynchAddress = 0x0000 Quirks 0xc101 should be broken down as below: #define NO_TEST_UNIT_READY 0x0001 #define NO_GETMAXLUN 0x0100 #define NO_SYNCHRONIZE_CACHE 0x4000 #define NO_PREVENT_ALLOW 0x8000 Regards. -- Tomoaki AOKI From owner-freebsd-current@freebsd.org Sun Sep 24 12:19:38 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05B94E26ED8 for ; Sun, 24 Sep 2017 12:19:38 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D4FE739E4 for ; Sun, 24 Sep 2017 12:19:36 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from thor.intern.walstatt.dynvpn.de ([77.179.65.254]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M20Jj-1d6btL1KdH-00tyHo; Sun, 24 Sep 2017 14:19:28 +0200 Date: Sun, 24 Sep 2017 14:19:26 +0200 From: "O. Hartmann" To: Konstantin Belousov Cc: "O. Hartmann" , FreeBSD CURRENT Subject: Re: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem Message-ID: <20170924141926.4c6876dd@thor.intern.walstatt.dynvpn.de> In-Reply-To: <20170924075238.GS2271@kib.kiev.ua> References: <20170924093151.2edfeeff@thor.intern.walstatt.dynvpn.de> <20170924075238.GS2271@kib.kiev.ua> Organization: WALSTATT User-Agent: OutScare 3.1415926 X-Operating-System: ImNotAnOperatingSystem 3.141592527 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/+M_b37R/Yaxce8PLz0ruRLA"; protocol="application/pgp-signature" X-Provags-ID: V03:K0:/O6je/GDDv1K9S0ttmypCnaik7pMsnyFll/kLp7OM/Y7rUxJY1A Q/nDHxJplBRtBrvN7XgPSi9oaWYbo+RkwHOgLtyxuWTKKeHXraBqItrHJ38HZjBUbpm8Ztl h8ZyoO9hxrTMlzRyf44QZkUyDLUgFO5ThbBSkwo7PUN6EVvZCnleh1r7ew6VGET16ITH/mM FvO/t3R0bHkcTtjYmdhJQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:pW+K6/rrBeI=:d3mj+vfpc3tF6eETQQonz7 ZNn4GxIJQ74kRPFZ5D7Y2DXRHr/zJSYWCbquPACd4vlNZXevOcJBQWQvGwrpSOLKS4sYS9bAI lDmqkza8bwmzevmKLv2Ziixa1dw9rSBOTPAQeiubA+8KfbJa6L4L1ZdxmhrT2aacE9YTRx+4X kbARyMDINWaNlajENnZKw9QrGCRpOCQLC6TTLIPJDRlAxZfdSa1DUQzN5GlBbBWWNik4w41Ou jEHBf9WUivhC3rUq2yAGTUjAnQHlmi7uJlsigITuwa9higFNPuS57l7Bozo45gyoR6CzbY7ca xNBNqu9/cjp2P+5hww2LhC//y8sZOd4EGHNwDp8r06RTEUSB4fUt0ZBTPKLNRoGWs6eBR6Y08 1I14C9num46VzkCBOtzQcwkWrYmqBqEPlDL5EnLplNg474FcOhCSYIq1/Xh7L/tN3CMtVsdqP sskhmarhifJR2qMx/L/BmB8ZVgUmXiUagVBx9DFwz0cxRiHuv4nlFsHWpw6DZpf5sLmUnYk7m R4sWy+4l+mdH64E40O1xzScCxHIkJJ8N8KroKj1SxthKJG+dnXeZXBjHfKBE5vO81cTxg/W5S Xr1eSVgGsYnn24+j9cuRSaAojsIqfpipjzTIQc5RclmBLLOElt+Ik9INCPZEE/WXQ1+wZ2jlA IhLqovvF6//YbijNRu1gBfhMxn4jIti44Dd0fB9LNwpOJqHKtlLcw6LMv3KP02PyyomaDFXvM +pmmplMi65a95zf4XJhV6cK1PelH/kg/T9DxuRKjGeeMftVJCLl5pQaVcJXwvrTWpQd9JaWbN 3UDGqpvPwEtbgYfqBx7sDF0H3MxTw== X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 12:19:38 -0000 --Sig_/+M_b37R/Yaxce8PLz0ruRLA Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Am Sun, 24 Sep 2017 10:52:38 +0300 Konstantin Belousov schrieb: > On Sun, Sep 24, 2017 at 09:31:51AM +0200, O. Hartmann wrote: > > Running a NanoBSD application on a small PCEngine device, feeding the b= ox with a > > CURRENT (FreeBSD 12.0-CURRENT #33 r323958: Sat Sep 23 22:19:57 CEST 201= 7 amd64), > > booting from a SD memory card. > >=20 > > Recently, when saving configuration changes on the SD memory card, I re= ceive these > > strange errors: > >=20 > > [...] > > checksum failed: cg 0, cgp: 0xb478d1d5 !=3D bp: 0x874608df > > checksum failed: cg 12, cgp: 0xe626d67c !=3D bp: 0x7360c12a > > checksum failed: cg 0, cgp: 0xb478d1d5 !=3D bp: 0x874608df > > handle_workitem_freefile: got error 5 while accessing filesystem > > checksum failed: cg 12, cgp: 0xe626d67c !=3D bp: 0x7360c12a > > handle_workitem_freefile: got error 5 while accessing filesystem > > [...] > >=20 > > I do not use MMCCAM in the kernel, but as far as I know, the problem oc= cured in the > > same time frame.=20 > >=20 > > The question is: is the SD memory about to die and is to be replaced or= is this a bug > > introduced with the new MMCCAM stuff recently? =20 >=20 > The messages you see about 'checksum_failed' are due to recently added > UFS feature where cylinder group blocks are checksummed, and the checksum > is verified against the known one stored on the volume. >=20 > Were there kernel messages about disk I/O errors before that (as opposed > to UFS errors about checksums and SU complains) ? Hello, no, there were no errors before. The small box is updated on a frequent bas= is, so I'm sure there were no errors like those before or any other kind regarding I/O. I recall having read the commit description about the UFS2 enhancement. The NanoBSD image is created using UFS2/GPT partitioning scheme, if this is= important to know.=20 Regards, Oliver --=20 O. Hartmann Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Abs.= 4 BDSG). --Sig_/+M_b37R/Yaxce8PLz0ruRLA Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWceizgAKCRDS528fyFhY lP64Af9yUue8ZFcTHwOw02USilSKJAsYMG53lxiHviJAImBgyzXOeREiL8+5/ebg cSVV7cXwea5WUOrO52u/uzBQrhLeAf0fXrRY0HADNssh2m7tAUyZ/+h8997o1u4+ NnlGXQ+sEJvqCZmty/YTRNtFEmD8HE6X5nAJTktKpuXtWsrjHTnc =pXx6 -----END PGP SIGNATURE----- --Sig_/+M_b37R/Yaxce8PLz0ruRLA-- From owner-freebsd-current@freebsd.org Sun Sep 24 12:44:42 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5652CE27A24 for ; Sun, 24 Sep 2017 12:44:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2CE374913 for ; Sun, 24 Sep 2017 12:44:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8OCiaYV088786 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 24 Sep 2017 15:44:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8OCiaYV088786 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8OCia7U088785; Sun, 24 Sep 2017 15:44:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 24 Sep 2017 15:44:36 +0300 From: Konstantin Belousov To: "O. Hartmann" Cc: FreeBSD CURRENT Subject: Re: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem Message-ID: <20170924124436.GU2271@kib.kiev.ua> References: <20170924093151.2edfeeff@thor.intern.walstatt.dynvpn.de> <20170924075238.GS2271@kib.kiev.ua> <20170924141926.4c6876dd@thor.intern.walstatt.dynvpn.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170924141926.4c6876dd@thor.intern.walstatt.dynvpn.de> User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 12:44:42 -0000 On Sun, Sep 24, 2017 at 02:19:26PM +0200, O. Hartmann wrote: > Am Sun, 24 Sep 2017 10:52:38 +0300 > Konstantin Belousov schrieb: > > > On Sun, Sep 24, 2017 at 09:31:51AM +0200, O. Hartmann wrote: > > > Running a NanoBSD application on a small PCEngine device, feeding the box with a > > > CURRENT (FreeBSD 12.0-CURRENT #33 r323958: Sat Sep 23 22:19:57 CEST 2017 amd64), > > > booting from a SD memory card. > > > > > > Recently, when saving configuration changes on the SD memory card, I receive these > > > strange errors: > > > > > > [...] > > > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > > > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > > > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > > > handle_workitem_freefile: got error 5 while accessing filesystem > > > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > > > handle_workitem_freefile: got error 5 while accessing filesystem > > > [...] > > > > > > I do not use MMCCAM in the kernel, but as far as I know, the problem occured in the > > > same time frame. > > > > > > The question is: is the SD memory about to die and is to be replaced or is this a bug > > > introduced with the new MMCCAM stuff recently? > > > > The messages you see about 'checksum_failed' are due to recently added > > UFS feature where cylinder group blocks are checksummed, and the checksum > > is verified against the known one stored on the volume. > > > > Were there kernel messages about disk I/O errors before that (as opposed > > to UFS errors about checksums and SU complains) ? > > Hello, > > no, there were no errors before. The small box is updated on a frequent basis, so I'm > sure there were no errors like those before or any other kind regarding I/O. > > I recall having read the commit description about the UFS2 enhancement. > > The NanoBSD image is created using UFS2/GPT partitioning scheme, if this is important to > know. Try to read the SD card on pre-checksum kernel. If you can read it, verify that the files are not damaged. This is a way to check whether your SD card or controller provides the wrong data to host. From owner-freebsd-current@freebsd.org Sun Sep 24 16:01:34 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FCB7E2AA27 for ; Sun, 24 Sep 2017 16:01:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C142B7D9AD for ; Sun, 24 Sep 2017 16:01:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8OG1OjJ032873 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 24 Sep 2017 19:01:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8OG1OjJ032873 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8OG1NYt032872; Sun, 24 Sep 2017 19:01:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 24 Sep 2017 19:01:23 +0300 From: Konstantin Belousov To: "Rodney W. Grimes" Cc: "O. Hartmann" , FreeBSD CURRENT , Kirk McKusick Subject: Re: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem Message-ID: <20170924160123.GV2271@kib.kiev.ua> References: <20170924075238.GS2271@kib.kiev.ua> <201709241532.v8OFWsWN008079@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201709241532.v8OFWsWN008079@pdx.rh.CN85.dnsmgr.net> User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 16:01:34 -0000 On Sun, Sep 24, 2017 at 08:32:54AM -0700, Rodney W. Grimes wrote: > Would it be possible to add some robustness to these checksum > failed errors to indicate there source more accurately? Ie > either a /dev entry or a fstab entry, and the fact they are > from the ffs layer? Perhaps this. But untested and somewhat racy. diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index b981b5e5794..bd78b760442 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -2584,6 +2584,15 @@ ffs_mapsearch(fs, cgp, bpref, allocsiz) return (-1); } +static const struct statfs * +ffs_getmntstat(struct vnode *devvp) +{ + + if (devvp->v_type == VCHR) + return (&devvp->v_rdev->si_mountpt->mnt_stat); + return (ffs_getmntstat(VFSTOUFS(devvp->v_mount)->um_devvp)); +} + /* * Fetch and verify a cylinder group. */ @@ -2597,6 +2606,7 @@ ffs_getcg(fs, devvp, cg, bpp, cgpp) { struct buf *bp; struct cg *cgp; + const struct statfs *sfs; int flags, error; *bpp = NULL; @@ -2615,7 +2625,11 @@ ffs_getcg(fs, devvp, cg, bpp, cgpp) (bp->b_flags & B_CKHASH) != 0 && cgp->cg_ckhash != bp->b_ckhash) || !cg_chkmagic(cgp) || cgp->cg_cgx != cg) { - printf("checksum failed: cg %u, cgp: 0x%x != bp: 0x%jx\n", + sfs = ffs_getmntstat(devvp); + printf("UFS %s%s (%s) cylinder checksum failed: cg %u, cgp: " + "0x%x != bp: 0x%jx\n", + devvp->v_type == VCHR ? "" : "snapshot of ", + sfs->f_mntfromname, sfs->f_mntonname, cg, cgp->cg_ckhash, (uintmax_t)bp->b_ckhash); bp->b_flags &= ~B_CKHASH; bp->b_flags |= B_INVAL | B_NOCACHE; From owner-freebsd-current@freebsd.org Sun Sep 24 20:57:22 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC413E01378 for ; Sun, 24 Sep 2017 20:57:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0055.outbound.protection.outlook.com [104.47.42.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67AF330D9 for ; Sun, 24 Sep 2017 20:57:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM (52.132.78.18) by YQXPR0101MB0885.CANPRD01.PROD.OUTLOOK.COM (52.132.76.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Sun, 24 Sep 2017 20:57:20 +0000 Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5]) by YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5%13]) with mapi id 15.20.0077.011; Sun, 24 Sep 2017 20:57:20 +0000 From: Rick Macklem To: "freebsd-current@freebsd.org" Subject: anyone in the Boston area with time this week? Thread-Topic: anyone in the Boston area with time this week? Thread-Index: AQHTNXblbIUo9wT9sEqgu+On9UL5tw== Date: Sun, 24 Sep 2017 20:57:20 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQXPR0101MB0885; 6:I/Dg9HyTH8nOTlIUkMUOR8KAA4MrEOsi5Jb3G8KMA6spHtu4yjLFi4VjwOdyr7pfZmOlPbNLgLMcEhnkpNmq6YpvUAWgk/oAjY8Qqgl9x8xWLvekpEGpcdrVsV6N7M3Odrrp9UWffsGCAGWGmsfN6vnnzf6IrOo8jpjEtSwg53yhRcZznhHdyUoSQ/QXRke2GB4nnVkRzjMgfvqz476NChPi42TZxe3hiVdErNzWRWZnre+WF9qGycKbMuSSc4k33wdgpDgLVIaX9gG05sew9aJztOu+cuQnr9gSftzzruumyRQ/3151SZd5u2OPoQRl8iHnL5RSNs9m29eDOcB8Kw==; 5:Vo1aTucYZTEAtV3rnZ4Lw+83OiCj/JEow0RIvSKBPkKWbmfrZbYLG6vP5rbe5tIwD7vw0nAp2tnKil9npuIfe5PGYMPMy3DxPy+KGTQ6ACNxQ+BgISu+bQfepwenU9R/CT2NIyXWmCH4tic5Bpkvqg==; 24:bl/EBr/UIl/zi+uTjAdjjJj4O8VClPjGCJANPTW/PwGPPcepegd754xadaDhguAqmZAUNqArt2ZN5yWF8NnqjOTVbVOqe9Cj7cL+sTMoce8=; 7:LQ2zsSfKNcwGNylcTypK6GwVIwRD8U61wuWnJQHZUr2mxuSVOcPxz3oj7TTaT92FlJ+kEKcVABPs2X4RBylStMZ2dbwdMf1FAAeNVN7CLl6dMA3YoTcdqcNuBZP1OY9sXHUAVTdvrZsMQ4pP8tNhAiKA9FF+b+phJSihjBV+loxAndrC/mmP+L2YV2fx4HAP3ew7dgkDjrnqSvN9DOoScmEd+1NeHS4A5oFVnl6Fk1I= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 9f91cb2c-2559-4a10-1b8d-08d5038ed95c x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:YQXPR0101MB0885; x-ms-traffictypediagnostic: YQXPR0101MB0885: x-exchange-antispam-report-test: UriScan:(158342451672863); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(6041248)(20161123555025)(20161123562025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:YQXPR0101MB0885; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:YQXPR0101MB0885; x-forefront-prvs: 0440AC9990 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(189002)(199003)(2501003)(53936002)(106356001)(2351001)(9686003)(8936002)(55016002)(105586002)(8676002)(81166006)(33656002)(5640700003)(6506006)(102836003)(6916009)(74482002)(7696004)(101416001)(6436002)(81156014)(68736007)(5660300001)(50986999)(25786009)(3660700001)(189998001)(3280700002)(2900100001)(97736004)(2906002)(74316002)(305945005)(86362001)(478600001)(14454004)(54356999)(786003)(5250100002)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR0101MB0885; H:YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2017 20:57:20.8787 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB0885 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 20:57:22 -0000 Hi, I really doubt that there is anyone out there interested in doing this, but= I figured it can't hurt asking... RedHat is hosting a NFSv4 testing event at their facility at 34 Littleton Rd Westford, MA 01186 next week. There is no fee for attendance, but you need to physically be th= ere to help with testing. (They are actually fairly interesting events, with en= gineers from various vendors testing their code.) Anyhow, I have a pNFS server using Flexible Files Layout (and something cal= led "tightly coupled") that could be tested. You would basically need to show u= p with a couple of FreeBSD systems (or VMs with different IP addresses) set up wit= h this server. If anyone is interested, email and I can fill you in, rick ps: And, yes, I realize this is last minute. I just didn't think to email = sooner.= From owner-freebsd-current@freebsd.org Sun Sep 24 15:33:01 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E4FCE2A242 for ; Sun, 24 Sep 2017 15:33:01 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 490FE7CF1B for ; Sun, 24 Sep 2017 15:33:00 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v8OFWsvR008080; Sun, 24 Sep 2017 08:32:54 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v8OFWsWN008079; Sun, 24 Sep 2017 08:32:54 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201709241532.v8OFWsWN008079@pdx.rh.CN85.dnsmgr.net> Subject: Re: Dying SD memory? handle_workitem_freefile: got error 5 while accessing filesystem In-Reply-To: <20170924075238.GS2271@kib.kiev.ua> To: Konstantin Belousov Date: Sun, 24 Sep 2017 08:32:54 -0700 (PDT) CC: "O. Hartmann" , FreeBSD CURRENT X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 25 Sep 2017 04:13:17 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 15:33:01 -0000 > On Sun, Sep 24, 2017 at 09:31:51AM +0200, O. Hartmann wrote: > > Running a NanoBSD application on a small PCEngine device, feeding the box with a CURRENT > > (FreeBSD 12.0-CURRENT #33 r323958: Sat Sep 23 22:19:57 CEST 2017 amd64), booting from a > > SD memory card. > > > > Recently, when saving configuration changes on the SD memory card, I receive these > > strange errors: > > > > [...] > > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > > checksum failed: cg 0, cgp: 0xb478d1d5 != bp: 0x874608df > > handle_workitem_freefile: got error 5 while accessing filesystem > > checksum failed: cg 12, cgp: 0xe626d67c != bp: 0x7360c12a > > handle_workitem_freefile: got error 5 while accessing filesystem > > [...] Would it be possible to add some robustness to these checksum failed errors to indicate there source more accurately? Ie either a /dev entry or a fstab entry, and the fact they are from the ffs layer? > > I do not use MMCCAM in the kernel, but as far as I know, the problem occured in the same > > time frame. > > > > The question is: is the SD memory about to die and is to be replaced or is this a bug > > introduced with the new MMCCAM stuff recently? > > The messages you see about 'checksum_failed' are due to recently added > UFS feature where cylinder group blocks are checksummed, and the checksum > is verified against the known one stored on the volume. > > Were there kernel messages about disk I/O errors before that (as opposed > to UFS errors about checksums and SU complains) ? > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-current@freebsd.org Mon Sep 25 06:46:43 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B992E0C405 for ; Mon, 25 Sep 2017 06:46:43 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F2C57061B for ; Mon, 25 Sep 2017 06:46:43 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm0-x233.google.com with SMTP id r136so16254533wmf.2 for ; Sun, 24 Sep 2017 23:46:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=E21H6FUYFH+UxklD3TJ7GacyJxeZEe4M31ztGtoSXGA=; b=TJ4Lx9ARDZpa+6d7GOz3XvHv2dMPT57IRNRkoEG6DlVpFZjXKVnZHkmBSFHKdynBtd JKnF+pvQQxQdCW9+nfYMZKzZWS6ByRCKX48nCPNWBV1VI7/XDWqoq9MYjtD7wA62HOXP DKIe+FMjWkkYZ5mRP/wwj8BxAe7wXO5STnNqJe57x4I8vtNzy8xFyS/IzuPWHtySKz0l YUth3fbuYtISO35z6Np3pzyD1sWKBJZwRzQ093dhlwfjMeGUPZuhEZDBlLS1Ely09rsS JhRM8kUJUGJFLIfrNbzldWuxuYgGnIU0wYXiJl+IK8pjqk/EH6y9RUzFIwxIzX5rjImk zOeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=E21H6FUYFH+UxklD3TJ7GacyJxeZEe4M31ztGtoSXGA=; b=KZrus6TEQAANO5Vnv8mixZUeGsFUXkaV6rb02fsJOmmFTQwOKti/2z1X/V6Q3jkxGs j7uIpYeotkQbJzY/UU7YxWdmh8UeSwK1UriFGHu54CXFRLIMWC1cCVr7Si3hrmhoOVrc EKFfb6url3C9y3KEyqv1QqjuNiZKxq5n5YArih7eZljCShv1usKb33y1FxbulAxpxlzO piM+Pwvmu6i6VmFi+WtqIWwZYNspvpc6oqu5HimpiKYOjdjPv4HFIgLeLh9+AmDIQLRU Kp64WX9tAULpvBZR59YD15kkFvp+mApy29LiG+3a0o8Ppu/Y8b8mGZA3AkTZa/Ah60/y M9+Q== X-Gm-Message-State: AHPjjUj5PA1BT4ZB6VRXWs7iNxqO0JwynqzemaAOUkPvgnKEpn1pCSVQ 24ZOgKzym+reqcR4jzUA0McARA== X-Google-Smtp-Source: AOwi7QBQusPT54KmweXyrZ9tPHqLfgyNoJfYfS0EKkdmDWxsPpGs2E98knc9iOhTMyu+ohbIIQTvgg== X-Received: by 10.28.31.140 with SMTP id f134mr8144817wmf.81.1506322001449; Sun, 24 Sep 2017 23:46:41 -0700 (PDT) Received: from ernst.home (p4FCA62DB.dip0.t-ipconnect.de. [79.202.98.219]) by smtp.gmail.com with ESMTPSA id m201sm1975829wma.13.2017.09.24.23.46.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 24 Sep 2017 23:46:40 -0700 (PDT) Date: Mon, 25 Sep 2017 08:46:39 +0200 From: Gary Jennejohn To: Rick Macklem Cc: "freebsd-current@freebsd.org" Subject: Re: anyone in the Boston area with time this week? Message-ID: <20170925084639.4dfab0e6@ernst.home> In-Reply-To: References: Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.15.1 (GTK+ 2.24.31; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 06:46:43 -0000 On Sun, 24 Sep 2017 20:57:20 +0000 Rick Macklem wrote: > Hi, > > I really doubt that there is anyone out there interested in > doing this, but I figured it can't hurt asking... RedHat is > hosting a NFSv4 testing event at their facility at 34 Littleton > Rd Westford, MA 01186 next week. There is no fee for > attendance, but you need to physically be there to help with > testing. (They are actually fairly interesting events, with > engineers from various vendors testing their code.) > Historical note: I actually attended one of these events at Sun in 1985 and it really was interesting. But I now live in Germany, so no way I can attend this one. > Anyhow, I have a pNFS server using Flexible Files Layout (and > something called "tightly coupled") that could be tested. You > would basically need to show up with a couple of FreeBSD > systems (or VMs with different IP addresses) set up with this > server. > > If anyone is interested, email and I can fill you in, rick ps: > And, yes, I realize this is last minute. I just didn't think > to email sooner. -- Gary Jennejohn From owner-freebsd-current@freebsd.org Tue Sep 26 08:35:55 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92670E02E8C for ; Tue, 26 Sep 2017 08:35:55 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15430804DA for ; Tue, 26 Sep 2017 08:35:51 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MTTKZ-1dotco0d3H-00SMjL; Tue, 26 Sep 2017 10:35:49 +0200 Date: Tue, 26 Sep 2017 10:35:43 +0200 From: "O. Hartmann" To: freebsd-current , reebsd-ipfw@freebsd.org Subject: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:ZNHFsDv8EvBt7dofT2gf0RY/dtRvT9G7YmHgsC2V7QIbvAgWKF2 MAc+BPxieX7wusivDLMDM6534hIv5cUGoVv87712Z5OXe2uH8pH3Y4/Wpomh1cyqJecg2d0 kx3LVNFcX4Rj7olMxJxeg/JPHW4RirvfLLa8CS0bXOo8eb+lNbWlkHNd5A4v1HPzmjbaY/Z eYBt3wL9oWtF+u0ZzbxHw== X-UI-Out-Filterresults: notjunk:1;V01:K0:RcDHZO1QyBE=:KxKq+0fmp2KyMBvX1MrxMG oDkxf953BhOqS6TiheoW3SOa6Gt6nulI+tUjjQwp3CWeT6ltkR5FEsPyFpSQKbl7X9uXuKP7c Dwc59AnBrGBz108+xssURCiKAtppesxJn+LiGhrKIv8baH9CiLyqQh0M1Kri7t1/EbR9bwhm8 tdFipgSON3bD66WUoKCoBAPJin8WC88sx4cjx1xJK1paum2z0hOG6aQ0chrju327JFyV6icPN zYBcQrHxIKr3+okdsquJnXb82+4TW4qGcNb7Lnr2JV9kxCV2VOdxU7gGYsWxXDas0c7ZIxJ90 J/pXK1dNKHDwX5WzVkM0Y1iFesAB3RcQZf7kyQHErLxVOwqL0/VYFzy15F3r8tZa328AdknWP CK7YZc6h99BF6SDaILPKx4kCxLN66ymke6wLORy130pXZRne9JU9qrNg/2UDgEKBiGRgMfZT1 0jcgfslJRRv9l3PyRGGWULzghSRM6YZXSU7D2HYvCj2HmiReAxwlgjLm0A8veCiOq6+MaaLNn OA7VTrro7f3OuNH/5QnfuBPYY98SU1b5vGxUXcU2HSflsCUu017a8XgfkDgbyYhZMj/JMyRYZ 6RlVfExLOV+POKKsxQ82E1WCSpahC5YP+t6slEm+3k0PlVDr+mrojmWTjvWnKs8e45weUnBHB RfQXdtZbe2ijxy9pYgBzIfiyzMFVUlwDXfIN8Hibvrty/c36J5nwNL1EUs5smPNVOKsNzFlyz 2ftAPo2WlR2sQcs+ILCIUSSQ6TxO1i5GJOtUqzwd0BKKNMzCjdwJTZ1o6s6yRdVeJ/d/pXIX8 i+o5QJijLeUdyrkBKkUUXqrg6BW5UlVZCsC40EPX5Msn96zDa4= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 08:35:55 -0000 Hello, trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran into several problems. My questions might have a "noobish" character, so my apology, my experiences with IPFW are not as thorough as they should be. Before I'll got into medias res, aquestion about Pine64/AARCH64: - port net/asterisk13 is supposed to build only on armv6, is aarch64 about coming soon also supported? - would a Pine64 running CURRENT (12) be sufficient as a PBX platform (assumed having 2 GB of RAM)? My main concern is about IPFW (we do not use PF for some reasons, I have to stay with IPFW). I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet IPv6. The FreeBSD system acting as a router is supposed to have a jail soon containing the Asterisk 13 IP PBX (at the moment running on the main system). To provide access to the VoIP infrastructure inside/behind the router/NAT system, the in-kernel NAT facility of FreeBSD is forwarding the relevant UPD/TCP ports for VoIP to its destination network, and here I have a problem to solve. While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, it is a mess and pain in the arse to forward a whole range, say 11000/udp - 35000/udp, which is a range one of my providers is sending RTP on. A second provider uses another range for RTP, starting at 5000/udp. So, the logical consequence would be a union set up UDP range to forward, which exapnds then form 5000/udp to 45000/udp - which is much more a pain ... One of the most disturbing and well known problems is that due to the stateful firewall the RTP session very often is half duplex - it seems one direction of the RTP connection doesn't make it through IPFW/NAT. As often I search the net, I always get informed this is a typical problem and solutions are provided by so called ALGs - since SIP protocol's SDP indicates within the payload of the packets on which UDP ports both ends wish to establish their RTP session, it would be "easy" to pinhole the IPFW on exactly those ports for a theoretical large number of sessions, if IPFW could "divert" those packets to an instance inspecting SDP (or whatever is used for the RTP port indication, I'm new to that, sorry for the terminology) and then pinholing the NAT/IPFW for exactly this purpose without the forwarding mess. I came along netgraph() while searching for hints and hooks, but it seems a complete Linux domain, when it somes to appliances like VoIP/IP PBX. Either, the problem is that trivial on FreeBSD, so no further mentioning is necessary (which would explain the vast emptyness of explanations, hints and so on) or FreeBSD is a complete wasteland on this subject - which I also suspect, since pfSense and OPNsense must have come along with such problems and I simply do not know or recognise the software used for those purposes. So, if someone enlightened in this matter stumbles over my question and could delegate me onto the right way (ports, ng_XXX netgraph ficilities to look at, some ipfw techniques relevant to the problem apart from the stupid simple forwarding large ranges of ports) - I'd appreciate this and thanks in advance for patience and help, Oliver From owner-freebsd-current@freebsd.org Tue Sep 26 09:01:08 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28117E03955 for ; Tue, 26 Sep 2017 09:01:08 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-wr0-x233.google.com (mail-wr0-x233.google.com [IPv6:2a00:1450:400c:c0c::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0AFE8110F for ; Tue, 26 Sep 2017 09:01:07 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-wr0-x233.google.com with SMTP id k20so12226447wre.4; Tue, 26 Sep 2017 02:01:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=3ugMHUzx3LieB0WfYtJlhpBKCtPF6gWiVtqlEzD6oIA=; b=jCD1X+WFKG0OkzJtZyNxuU09Q/lhf7x4dQOz7iJgYHoL03BA3NSvhqERQwRn+F87Pu 9Jw7zACW8+uyu8K5UEgYnHlI3P9aY38sd4tpusHMB63qdC4VRi9FB8c150OdespH99jt WSbYEPzmF85iD0XRJMe5pitJ/GUVkEUTX91nYxrdlaNpfY6NoHKtnX0XRbFi8SMuxkdw 3l//WLErEv1/O7GK+TTzjr/9qKkqLLGRMnGuDeQ/vEyqPMkHJqphrgPZaDvl9WDCUxgu Efm7NmvHV/wYaV903EPphE7QpzeEqFmwlbBFGh33tKhKb8rtWQluKJUjXI+RHtG4PLNK vn4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=3ugMHUzx3LieB0WfYtJlhpBKCtPF6gWiVtqlEzD6oIA=; b=R+BGsi52N8JX5gnSfoe/SOIZeU+KhYNyRi//u7JG6TxSvcR5ww2jN5enWeVHIMa8Oi tyjrrlDXw9p/QtymjYm5WBJVSvCURMZt8eP5BwQoHNFKb4gOgQAqcUQNRmf60wosjhXF YjDxUQ8iitW8lKOzNWBsgPBVAKx54+ljcBXuVYHSwWHdDFSsoxAds0n+H0HzmX1iJTpX q9OzOLmj+X0zYuzn70CONfNhXb4djJoJrJv0xrlm2laiWCnDCa4TDhOhj6oO+KAslkGm 5cqGl5zcBm1ssqJXqbdf2WCob3olZGwNwmyohxl4rIjcazHnaZEaMQQrmlsMqFxSS+lJ QCCQ== X-Gm-Message-State: AHPjjUjri7X7WwwbijTAMHDKSMswmkPeLijB3ovF4iNGjfxeb3SfhHk2 NzsHtP33WMCfZ/uoTg8BDpX6bwFr0Y/pIjEp9z4= X-Google-Smtp-Source: AOwi7QBQnzEZaQx6/txXdkY80nsJKAfITHPspa7yZKPAVpVAwyK1fja/bPpYpyaL/oSQV1+H2+d/82EmtxhKGHTNCd8= X-Received: by 10.46.66.145 with SMTP id h17mr3708783ljf.140.1506416465977; Tue, 26 Sep 2017 02:01:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.89.13 with HTTP; Tue, 26 Sep 2017 02:00:45 -0700 (PDT) In-Reply-To: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> From: Damjan Jovanovic Date: Tue, 26 Sep 2017 11:00:45 +0200 Message-ID: Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem To: "O. Hartmann" Cc: freebsd-current , reebsd-ipfw@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 09:01:08 -0000 On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann wrote: > Hello, > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran > into > several problems. My questions might have a "noobish" character, so my > apology, > my experiences with IPFW are not as thorough as they should be. > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 about > coming soon also supported? > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform > (assumed > having 2 GB of RAM)? > > My main concern is about IPFW (we do not use PF for some reasons, I have to > stay with IPFW). > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet > IPv6. > The FreeBSD system acting as a router is supposed to have a jail soon > containing the Asterisk 13 IP PBX (at the moment running on the main > system). > To provide access to the VoIP infrastructure inside/behind the router/NAT > system, the in-kernel NAT facility of FreeBSD is forwarding the relevant > UPD/TCP ports for VoIP to its destination network, and here I have a > problem to > solve. > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, > it > is a mess and pain in the arse to forward a whole range, say 11000/udp - > 35000/udp, which is a range one of my providers is sending RTP on. A second > provider uses another range for RTP, starting at 5000/udp. So, the logical > consequence would be a union set up UDP range to forward, which exapnds > then > form 5000/udp to 45000/udp - which is much more a pain ... > > One of the most disturbing and well known problems is that due to the > stateful > firewall the RTP session very often is half duplex - it seems one direction > of the RTP connection doesn't make it through IPFW/NAT. As often I search > the > net, I always get informed this is a typical problem and solutions are > provided by so called ALGs - since SIP protocol's SDP indicates within the > payload of the packets on which UDP ports both ends wish to establish their > RTP session, it would be "easy" to pinhole the IPFW on exactly those ports > for > a theoretical large number of sessions, if IPFW could "divert" those > packets > to an instance inspecting SDP (or whatever is used for the RTP port > indication, I'm new to that, sorry for the terminology) and then pinholing > the > NAT/IPFW for exactly this purpose without the forwarding mess. I came along > netgraph() while searching for hints and hooks, but it seems a complete > Linux > domain, when it somes to appliances like VoIP/IP PBX. > > Either, the problem is that trivial on FreeBSD, so no further mentioning is > necessary (which would explain the vast emptyness of explanations, hints > and > so on) or FreeBSD is a complete wasteland on this subject - which I also > suspect, since pfSense and OPNsense must have come along with such problems > and I simply do not know or recognise the software used for those purposes. > > So, if someone enlightened in this matter stumbles over my question and > could > delegate me onto the right way (ports, ng_XXX netgraph ficilities to look > at, > some ipfw techniques relevant to the problem apart from the stupid simple > forwarding large ranges of ports) - I'd appreciate this and > > thanks in advance for patience and help, > > Oliver > Hi It might be easier if you just enable STUN on Asterisk, and build FreeBSD from source with my [largely neglected :( ] patch on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918 That way Asterisk should dynamically discover consistent external mappings for connections, making port forwarding RTP unnecessary. Damjan From owner-freebsd-current@freebsd.org Tue Sep 26 09:27:21 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B43CE049AC for ; Tue, 26 Sep 2017 09:27:21 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DDD678278A for ; Tue, 26 Sep 2017 09:27:19 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y1bFW5Gv2zZqh; Tue, 26 Sep 2017 11:27:11 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id Vphw-hdMCdU7; Tue, 26 Sep 2017 11:27:05 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Tue, 26 Sep 2017 11:27:05 +0200 (CEST) Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem To: "O. Hartmann" , freebsd-current , reebsd-ipfw@freebsd.org References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: Date: Tue, 26 Sep 2017 11:27:05 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 09:27:21 -0000 On 09/26/2017 10:35, O. Hartmann wrote: > Hello, > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran into > several problems. My questions might have a "noobish" character, so my apology, > my experiences with IPFW are not as thorough as they should be. > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 about > coming soon also supported? I'm maintaining the asterisk ports. At present I don't have any ARM64 hardware to test it on, but I plan to create an ARM64 jail in poudriere so I can try to make it at least build there. In such a case would you be willing to test port changes on the hardware to actually check it runs? > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform (assumed > having 2 GB of RAM)? That very much depends on the kind of load you are expecting. Asterisk can process a lot of calls. Especially if it can avoid being in the media path. On the other hand if you plan doing a lot of audio transcoding or some video transcoding, load can get up quite fast. Compressed codecs like the "simple" G729 will make your load grow relatively fast even with audio transcoding. Transcoding also lowers call quality so it should anyway be avoided as much as possible. Also load can go up if you're doing many disk operations. Monitoring and saving audio for a bunch of calls can be quite heavy on disk resources AND could require additional transcoding. > > My main concern is about IPFW (we do not use PF for some reasons, I have to > stay with IPFW). > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet IPv6. > The FreeBSD system acting as a router is supposed to have a jail soon > containing the Asterisk 13 IP PBX (at the moment running on the main system). > To provide access to the VoIP infrastructure inside/behind the router/NAT > system, the in-kernel NAT facility of FreeBSD is forwarding the relevant > UPD/TCP ports for VoIP to its destination network, and here I have a problem to > solve. > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, it > is a mess and pain in the arse to forward a whole range, say 11000/udp - > 35000/udp, which is a range one of my providers is sending RTP on. A second > provider uses another range for RTP, starting at 5000/udp. So, the logical > consequence would be a union set up UDP range to forward, which exapnds then > form 5000/udp to 45000/udp - which is much more a pain ... The asterisk project has some suggestions on this here [1] RTP with NAT+FW is a pain. I'm not aware of any IPFW tools able to actively inspect SIP packets (which could also be encrypted if using SIPS, so there would be no clean way to inspect them). Depending on your phone providers you could use a stun and/or turn server by enabling the ICE protocol [2], which are all technologies supported by asterisk, but require support from your provider. Another option is symmetric RTP, which is a trick by creating symmetric port numbers connections which sometimes can trick firewalls properly configured. You setup also forces you to keep asterisk in the media path (direct_media=no in peer configuration) Unluckily none of these technologies is 100% bulletproof. RTP is not made to play well with NAT, so the professional solution is to spare an IP and redirect almost all ports to the SIP/RTP box. Also it's much better to do this with static rules, to avoid load problems on the FW (see later). You can sidestep the whole issue by running a proxy on your firewall machine, if you have control of it. There are a few in the ports tree. Take a look at: net/kamailio - It is really a SIP proxy, but can parse SIP/SDP packets and modify their content on the fly, allowing you to play neat tricks. Requires some knowledge of the protocol and work though. (maybe you can get him to punch holes in the firewall, but I have not checked if it's possible) net/rtpproxy: this is more specific and maybe your best bet. Beware of the load of proxying RTP in userland though. > > One of the most disturbing and well known problems is that due to the stateful > firewall the RTP session very often is half duplex - it seems one direction Depending on how many simultaneous SIP calls you plan to manage keep an eye on your firewall too. each call will create at least 3 states, one for SIP and one for each leg of the RTP stream, so it can pile up quite fast. > of the RTP connection doesn't make it through IPFW/NAT. As often I search the > net, I always get informed this is a typical problem and solutions are > provided by so called ALGs - since SIP protocol's SDP indicates within the This would require coding it in IPFW, and the load on the firewall could be significant. It could be done in userland maybe, leveraging divert(4) and having a daemon listening there and doing the extra work, but this would be quite expensive. Depending on your call volume the load could be too much for your firewall. > payload of the packets on which UDP ports both ends wish to establish their > RTP session, it would be "easy" to pinhole the IPFW on exactly those ports for > a theoretical large number of sessions, if IPFW could "divert" those packets > to an instance inspecting SDP (or whatever is used for the RTP port > indication, I'm new to that, sorry for the terminology) and then pinholing the > NAT/IPFW for exactly this purpose without the forwarding mess. I came along > netgraph() while searching for hints and hooks, but it seems a complete Linux > domain, when it somes to appliances like VoIP/IP PBX. > > Either, the problem is that trivial on FreeBSD, so no further mentioning is > necessary (which would explain the vast emptyness of explanations, hints and > so on) or FreeBSD is a complete wasteland on this subject - which I also > suspect, since pfSense and OPNsense must have come along with such problems > and I simply do not know or recognise the software used for those purposes. I'm not aware of any silver bullets in those products for this. > > So, if someone enlightened in this matter stumbles over my question and could > delegate me onto the right way (ports, ng_XXX netgraph ficilities to look at, > some ipfw techniques relevant to the problem apart from the stupid simple > forwarding large ranges of ports) - I'd appreciate this and Maybe you could also get fancy with netflow, and that could have a lower load but you'd have to create the IP parsing code yourself. Keep in mind that SIP, being used only for signaling, has a relatively low cost on being manipulated(obviously it depends on the load of calls being managed but it gets hundreds of calls starting and stopping every few seconds to create an heavy SIP traffic). RTP on the other hand consists on a continuous UDP packet flow, and in VoIP it means two for each call, which count directly on the PPS (Packets per second) load of you networking equipment, performing manipulation on those imposes a constant, heavy per connection load on the machines performing it, so you should try to use static FW rules and perform no further processing on it unless you have very low call volume(let's say that more that 10 through firewall performing extra manipulation on RTP streams is starting to be non trivial) or appropriate hardware. Hope this all helps. [1] https://wiki.asterisk.org/wiki/display/AST/Configuring+res_pjsip+to+work+through+NAT [2] https://wiki.asterisk.org/wiki/display/AST/Interactive+Connectivity+Establishment+%28ICE%29+in+Asterisk -- Guido Falsi From owner-freebsd-current@freebsd.org Tue Sep 26 12:35:12 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4391AE0984A; Tue, 26 Sep 2017 12:35:12 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD0F0642E1; Tue, 26 Sep 2017 12:35:11 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LdHLB-1dWs2o3hw1-00iUcD; Tue, 26 Sep 2017 14:35:04 +0200 Date: Tue, 26 Sep 2017 14:35:03 +0200 From: "O. Hartmann" To: freebsd-current , freebsd-ipfw@freebsd.org Subject: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170926143503.66f6532c@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:SF2EdouQVPAh1+Mjc4FHM8gH9ziNKTD0RWJhZKJC13xlNzYuO0f GX9mWpeCFGgZ7juf/xDp8yDxNbw78UxBMlLf2ZXCLD2Qq5b2VJsLrQS2DWpcv6p/P9YehHg JqnsCHzd4XjbYc4yYyp902jXsfb4G5/HIQTlHMaqcTT40HKsB1iJ7GQvawr051xtAewyRJA AuiJ9nnU2wHhAOHb4baGg== X-UI-Out-Filterresults: notjunk:1;V01:K0:v21T0mBV0C4=:Qj4N/n9dmtnLC4tGEEKDu9 I88wyUpTxV9Klfs9WDk7XV7O6+WE2wcW6KiNCxGCaB65zc1LiZ4WCA/GMcE1cUrpDi7mryDZz AGbTiUn14tYPefA6RNcK33uafyjdjaArF+5qH/WjOvDskQfPTBOXXSvmzhp2M9e5Al0/J15FN tS7rPxjg65q3VT4bjM9wJrCgqDm/MxcRNp2wGi35gKzKhQqB62h324RQPQii6AtXQaDUBCb6C 2A2fx08sYZGRSLm0BRFhM9T71SUOpl3t718RPsUlSYo27Zi+yXSVjEuyuwGzX25bSlThia0Va wHhj0YB4VCv+w3T7ugxWahEKrA125ONe4B4GeX1vnfWoVAoxgLyuJjwd7fUOcvdYP26qae95Z HI7TdSgWlc3FIH7JkQG707TAFLMu6RM1pFU2Pm9NsasySN82P0737gCrw7NzN78z8XaSYsp1s pL4ZQm2QFU0OxCSQ7jTQ3sJ3RvzafuJ4XFb6urBtCkKXl7eiwh+H2N0usS4GeaPiko0kg3QlL 8+VERP1Sky3jBZDgTz2mV92WyjWHUgGuNdAHj+C6DnMGixoZWmczrb2ziqPN93EtzlS0u3Qp8 R4MJ5DhZ9Gz29GmuKkstNxm5xK34IVvxYyByR9748D78wwOhCCVuUUdzscR5pSF0BqK007Blq YA+Z4tIq7iC9vUAhHGAx5To5E7CshKcB5JPNYs2uCQO27mvNYQlrbxdcAel0yXbPj1g9r93F4 2sJskZGMpvx+kD8YXHwsYWRTzNC4tUt7bzHOVqtISXd2hZOFRp4BA4KS+FmfxZiYG1wlzzOK0 Dlu7pYeyCl/gmP4+kVsYBBoAByRqKIhpQIxo7SnbeQzhmbU8T0= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 12:35:12 -0000 Hello, trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran into several problems. My questions might have a "noobish" character, so my apology, my experiences with IPFW are not as thorough as they should be. Before I'll got into medias res, aquestion about Pine64/AARCH64: - port net/asterisk13 is supposed to build only on armv6, is aarch64 about coming soon also supported? - would a Pine64 running CURRENT (12) be sufficient as a PBX platform (assumed having 2 GB of RAM)? My main concern is about IPFW (we do not use PF for some reasons, I have to stay with IPFW). I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet IPv6. The FreeBSD system acting as a router is supposed to have a jail soon containing the Asterisk 13 IP PBX (at the moment running on the main system). To provide access to the VoIP infrastructure inside/behind the router/NAT system, the in-kernel NAT facility of FreeBSD is forwarding the relevant UPD/TCP ports for VoIP to its destination network, and here I have a problem to solve. While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, it is a mess and pain in the arse to forward a whole range, say 11000/udp - 35000/udp, which is a range one of my providers is sending RTP on. A second provider uses another range for RTP, starting at 5000/udp. So, the logical consequence would be a union set up UDP range to forward, which exapnds then form 5000/udp to 45000/udp - which is much more a pain ... One of the most disturbing and well known problems is that due to the stateful firewall the RTP session very often is half duplex - it seems one direction of the RTP connection doesn't make it through IPFW/NAT. As often I search the net, I always get informed this is a typical problem and solutions are provided by so called ALGs - since SIP protocol's SDP indicates within the payload of the packets on which UDP ports both ends wish to establish their RTP session, it would be "easy" to pinhole the IPFW on exactly those ports for a theoretical large number of sessions, if IPFW could "divert" those packets to an instance inspecting SDP (or whatever is used for the RTP port indication, I'm new to that, sorry for the terminology) and then pinholing the NAT/IPFW for exactly this purpose without the forwarding mess. I came along netgraph() while searching for hints and hooks, but it seems a complete Linux domain, when it somes to appliances like VoIP/IP PBX. Either, the problem is that trivial on FreeBSD, so no further mentioning is necessary (which would explain the vast emptyness of explanations, hints and so on) or FreeBSD is a complete wasteland on this subject - which I also suspect, since pfSense and OPNsense must have come along with such problems and I simply do not know or recognise the software used for those purposes. So, if someone enlightened in this matter stumbles over my question and could delegate me onto the right way (ports, ng_XXX netgraph ficilities to look at, some ipfw techniques relevant to the problem apart from the stupid simple forwarding large ranges of ports) - I'd appreciate this and thanks in advance for patience and help, Oliver From owner-freebsd-current@freebsd.org Tue Sep 26 12:45:31 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB00FE09C7D for ; Tue, 26 Sep 2017 12:45:31 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16FAE647C1 for ; Tue, 26 Sep 2017 12:45:30 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MEFIm-1e7rr90vxA-00FVl7 for ; Tue, 26 Sep 2017 14:45:23 +0200 Date: Tue, 26 Sep 2017 14:45:22 +0200 From: "O. Hartmann" To: freebsd-current Subject: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:MVCegF916ppv2vJJgEYCgZVHKXTBGqbtsw7ewpUGF6CCYRfOu9R 8QNsv3OAoj6Qqt7DWCMa9Sw98pa//DcUrGcUaRek/ALaoddtBjZIt0Q5r8wrVOBW0aTrbLJ WrU2dMG5U8/OLO2cinFCyIO3HirF0S8RHGAMwqBxRXbqZFZ6wVHMpjqyKSy9uJ44vW6X8cy skHLIYRvARQPC4mQvv1+g== X-UI-Out-Filterresults: notjunk:1;V01:K0:u8n/067QGIo=:ZY03tsGOaInC9/E9cTo+hC d9D5K2mTqSiGnh1oDZfAW7d4Iwpwwq2b5MZbbhXFGginiFzvaYzz7bY3G714rz2Vlwfdhd3bf 4GrJa/RGShrZPKvg5geXNvQj313tA/hah5jCibHoTNajqYKu48GTd6T9pARtlyJsqn5hhOpYo CzIfACL8hI1z3n9lygqE4vPyZfDwif9IOltLgOoKMno+imN+JLceT4r+yoWf3JElwDkV1DhgO v38I3ac4NnLI1KRKLkw1APY8oNMtFsYDHuUYgVYvh4kfMSy9VaEOIyx+ZfUdyTV0HTIfjPGiU 3rUt/e4vkce3KXF0n7I0MKwS4+UoBRvDqnLNcCJpD9g4dygAi9BUdgXkAlMwEDtgkAM4pYkil hPDB/M62qXQxrNXU0uRFLq8loZclw91Yka3I39uGxDzX7brXjOMABd3TwEsJ8+sBrvxbNy/+o TQNfRSTPeKaqKauC1oy/vD08LFR3HhDbxoKLhaSVfdxIr0EXukMcY91MijBA81u3fRjxD9Bbw WEvJOQs4JJl17rsMDqWczH/jzn5VFMG6ScLodpT1RtiCCCU2VWdFpYUauof+dmB7CpumTYvA/ Eyq8PGAgjm+UkmKn7PF/SG501oc7d+ugFI3r1Ef5G2qJ3NrldbSi/n3JSSSonUA7ZVJxpKnL8 gBbjidanEb1XtNK1tFSc+pqofRJv7LbULIhyABdTsqZgwN8Owyu6VmQfhLCBonW2J6TufeM+o mpfKR2E1/OoH0dVV94YaEtYT8Jt8u/2nvIWhPmxRmznbypVzR+8ZUzG/ahFJTsi63f0Y4R0zh yb8joxUmUkJh43AeSbQbZi2XgENHPr+KfYFy2jzo9mCT4gCSlg= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 12:45:31 -0000 Befor starting a PR I'd liek to ask for some advice to document a supposedly existent memory leak in net/asterisk13 and 12-CURRENT. Background: Running recent 12-CURRENT (FreeBSD 12.0-CURRENT #58 r323999: Tue Sep 26 06:18:27 CEST 2017 amd64) on a PCengine APU 2C4, equipted with 4GB of RAM and 4080 KB free after the most recent SeaBIOS has been started, I try to utilise a net/asterisk13 as PBX on that platform. Asterisk13 has been build via poudriere and is compiled with CLANG/LLVM, not GCC! When FreeBSD (NanoBSD) has been started, depending on the recent revision of the OS/Kernel, top shows ~3680 - 3705 MBytes of free memory. Starting net/asterisk13 via "service asterisk onestart" indicates an overall usage of up to 100 MBytes of memory. After having now run the Asterisk 13.17.2 daemon for two days resulted in ~ 3100 MBytes of free memory, while the asterisk daemon was still running and doing nothing. But performing several restarts on a freshly started box gives the same result after ~ 10 restarts - and even after stopping asterisk and let the system run for a couple of days doesn't free the memory. I stopped after 15 restarts or so watching the loss of memory after each restart of asterisk, so i came to the conclusion that there must be a memory leak somewhere. now i try to provide more informations as needed for a PR. Can someone help? Thanks in advance, Oliver From owner-freebsd-current@freebsd.org Tue Sep 26 13:06:32 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2568E0A3E2 for ; Tue, 26 Sep 2017 13:06:32 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96B296527B for ; Tue, 26 Sep 2017 13:06:32 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y1h6Y6SCjzZrT; Tue, 26 Sep 2017 15:06:29 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id 8SrtYKeFCaiv; Tue, 26 Sep 2017 15:06:24 +0200 (CEST) Received: from tommy.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Tue, 26 Sep 2017 15:06:24 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" , freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> Date: Tue, 26 Sep 2017 15:06:23 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 13:06:32 -0000 On 09/26/2017 14:45, O. Hartmann wrote: > Befor starting a PR I'd liek to ask for some advice to document a supposedly > existent memory leak in net/asterisk13 and 12-CURRENT. > > Background: > > Running recent 12-CURRENT (FreeBSD 12.0-CURRENT #58 r323999: Tue Sep 26 > 06:18:27 CEST 2017 amd64) on a PCengine APU 2C4, equipted with 4GB of RAM and > 4080 KB free after the most recent SeaBIOS has been started, I try to utilise a > net/asterisk13 as PBX on that platform. Asterisk13 has been build via poudriere > and is compiled with CLANG/LLVM, not GCC! > I'm running asterisk on similar hardware and also building with clang, and have it going for months without any problems. I have disabled many modules in that build though. Problem could be caused by some ancillary library pulled in by some module. > When FreeBSD (NanoBSD) has been started, depending on the recent revision of the > OS/Kernel, top shows ~3680 - 3705 MBytes of free memory. Starting > net/asterisk13 via "service asterisk onestart" indicates an overall usage of up > to 100 MBytes of memory. After having now run the Asterisk 13.17.2 daemon for > two days resulted in ~ 3100 MBytes of free memory, while the asterisk daemon > was still running and doing nothing. But performing several restarts on a > freshly started box gives the same result after ~ 10 restarts - and even after > stopping asterisk and let the system run for a couple of days doesn't free the > memory. I stopped after 15 restarts or so watching the loss of memory after > each restart of asterisk, so i came to the conclusion that there must be a > memory leak somewhere. now i try to provide more informations as needed for a > PR. > > Can someone help? Not sure, restarting the daemon should free any leaked memory the daemon has. If a killed process leaves memory locked at the system level there should be some other cause. Have you tried diagnosing this memory with more tools? Depending on how deep you want to investigate ps, top and vmstat are simple to use and give you some indication. To go deeper you'll need to use more complicated tools. Others are better suited than me for suggesting those. DTrace could be a powerful but not easy to use instrument. Looking just at free memory (which under a UNIX OS can have various valid definitions, depending on how you account for inactive memory, buffers and laundry RAM) is definitely not enough. Have you seen asterisk process allocate more and more memory? You should check also what other processes in the system are doing. A computer software when running never really does "nothing"(except, maybe, when swapped out) and some memory usage can accumulate in many parts of the system. -- Guido Falsi From owner-freebsd-current@freebsd.org Tue Sep 26 13:30:00 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3525BE0AAAB for ; Tue, 26 Sep 2017 13:30:00 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A72EA65BFD; Tue, 26 Sep 2017 13:29:59 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0Lb5nF-1dUhC915uD-00kfI5; Tue, 26 Sep 2017 15:29:56 +0200 Date: Tue, 26 Sep 2017 15:29:55 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current , reebsd-ipfw@freebsd.org Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170926152955.205a012c@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:YttJ/O5PbY3YAGdMk/VYkht9FnrwsFKHiZpJGT0xxSQKDqH86bW 1enN7/MLR6AaqxjaD+5lmmtbpkhdi+Z1kPo+cY+qzIfgufehKwhpM8RMBMS1diX+eieo5Gd WeRoM5+3PvDdY9G89AUlbiPF3HVPqK4V7KqjYGQyCSk/wlnCF8J5tzqVlaQ85uSz65rLyH+ zIGY63oS2pPgqBN4QPWlQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:SMb1vRYGkJk=:k/D5ZEP9AhRsAjDWQlyswQ hik0fDMazAS+lVh+JQmFR/2WE+KVXoBmrX5dk/gdFk/pHoN4j9p9CSHrN3C4fD/NBdUAhFcM2 5wC6FBK0//mjVibSmGrXv4bFJwvvPIeCi3c5BkUSXK6wKId3Tq2AvTv+4BtEpe4qBCMCi26fL lNjJ219hZMcKt0X12lXRKJD04xzJaiY/3h9QSfX9Jd4sof1AkYhOqZ8YzYTpAGgUD8UhW2Ej+ No8XHboCPsgAIjpi4WiAZgu1fLCMDT9VDNBA7dbI8aDZZGX9fOLDwknZolCH656DlIvPHwJk7 XlW7WVJdqp5NfDEI8WEAbtOaPzq5JYlHWV0OZM033b3E/pz9nfBJG4gH51Gq62mpg4JjywhgP 5In/F9+0DjUZTLT+8UxP7JdLV0jmMLFFPRSAohNeD3slDyJ3IuWzVoYbI3hs/wbMI6vksgA26 oex3CPqvTFn1z4J3dew5zRpfvwZ8lgO9E+ZGseRs3kQY2kgTU+kujwBAs0XXV4xmxuCw+FfH8 dlzTKaa2Oq83n+3zqd7SPyzqBeuq79GCFEfCJOP2qi7PMeFLgYtnO4fOfcqraYffS+3cntCQa AZX4jDOTUJXlrqMSXX3IC1bnx6YKx30fwwKnYSGZiAIFCH/mif/CPvNVQf9NMOVmILz4TVwqG rcghxUTwt3YHNGvBTql7XtdZnGq5EwK73yMRUzyjy4vZcf/kQxrIc0iMXCCKPn65EiRT6g7m2 DPEB7oLAGhszCMKRHGjEBunQ3Q44QQCmTJjGEsoPjT0nGb6GhVCzNoHTNzpKivUExjCNTIBua VDzn93zdjTDK55Vu8bgOfo61qr+HfiQi+VlQgbHS/DQnO+ISy0= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 13:30:00 -0000 On Tue, 26 Sep 2017 11:27:05 +0200 Guido Falsi wrote: > On 09/26/2017 10:35, O. Hartmann wrote: > > Hello, > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran > > into several problems. My questions might have a "noobish" character, so my > > apology, my experiences with IPFW are not as thorough as they should be. > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 about > > coming soon also supported? > > I'm maintaining the asterisk ports. At present I don't have any ARM64 > hardware to test it on, but I plan to create an ARM64 jail in poudriere > so I can try to make it at least build there. Hello Guido, I posted by accident the question again to this list as I introduced a typo when sending it also to the IPFW list. I'm sorry. I already tried to build net/asterisk13 on my AARCH64 jail, but since I'd like to have the databases/postgresql96-client aboard and this specific port fails to build, I gave up - it is, by the way, the only port (pgsql) as far as I know that fails in my poudriere cross compiling jail. I did not get further, but I saw that it is supposed to build only for amd64 and armv6. > > In such a case would you be willing to test port changes on the hardware > to actually check it runs? I'd like to if the efforts are not to much time consuming - I do not have a working Pine64 image anymore, but I have a Pine64 with 2GB RAM at hand. Months ago I started playing with cross compiling world/kernel for AARCH64, but I'm not familiar with crochet and preparing the SD image - but willing to do. But beware: my home box preparing the cross cimpilation is not the fastest! > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform > > (assumed having 2 GB of RAM)? > > That very much depends on the kind of load you are expecting. > > Asterisk can process a lot of calls. Especially if it can avoid being > in the media path. For the moment, the ARM based PBX should perform SoHo tasks - three, up to ten lines at maximum. > > On the other hand if you plan doing a lot of audio transcoding or some > video transcoding, load can get up quite fast. Compressed codecs like > the "simple" G729 will make your load grow relatively fast even with > audio transcoding. Transcoding also lowers call quality so it should > anyway be avoided as much as possible. Good to know. But the PBX is more like an experiment for "at home's PBX" and, if applicable, later for some fellows working scientifically in field and in need for some small and neat equipement. Video message/streaming is not so much the focus on the first attempts, but if possible, welcome. if not: not so tragic. > > Also load can go up if you're doing many disk operations. Monitoring and > saving audio for a bunch of calls can be quite heavy on disk resources > AND could require additional transcoding. There are Linux fellows running Asterisk 13 on raspberry Pi3 very successfully and this little box has only 1GB RAM as far as I know. Why should FreeBSD fail? > > > > > My main concern is about IPFW (we do not use PF for some reasons, I have to > > stay with IPFW). > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet > > IPv6. The FreeBSD system acting as a router is supposed to have a jail soon > > containing the Asterisk 13 IP PBX (at the moment running on the main > > system). To provide access to the VoIP infrastructure inside/behind the > > router/NAT system, the in-kernel NAT facility of FreeBSD is forwarding the > > relevant UPD/TCP ports for VoIP to its destination network, and here I have > > a problem to solve. > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, > > it is a mess and pain in the arse to forward a whole range, say 11000/udp - > > 35000/udp, which is a range one of my providers is sending RTP on. A second > > provider uses another range for RTP, starting at 5000/udp. So, the logical > > consequence would be a union set up UDP range to forward, which exapnds then > > form 5000/udp to 45000/udp - which is much more a pain ... > > The asterisk project has some suggestions on this here [1] I know those references as I'm with the problem now for a couple of months ... > > RTP with NAT+FW is a pain. I'm not aware of any IPFW tools able to > actively inspect SIP packets (which could also be encrypted if using > SIPS, so there would be no clean way to inspect them). > > Depending on your phone providers you could use a stun and/or turn > server by enabling the ICE protocol [2], which are all technologies > supported by asterisk, but require support from your provider. Another > option is symmetric RTP, which is a trick by creating symmetric port > numbers connections which sometimes can trick firewalls properly configured. > > You setup also forces you to keep asterisk in the media path > (direct_media=no in peer configuration) I know. It is configured that way and it works well with one of my providers, which has "ingres" calls via 10000/udp through 30000/udp. By simply doing a forwarding of these portranges in the IPFW rules for the NAT section, my firewall is open on that range! And this open range grows larger with the second provider, which has another range on which its RTP communications is attempting to establish ingress calls. > > Unluckily none of these technologies is 100% bulletproof. > > RTP is not made to play well with NAT, so the professional solution is > to spare an IP and redirect almost all ports to the SIP/RTP box. Also > it's much better to do this with static rules, to avoid load problems on > the FW (see later). > > > You can sidestep the whole issue by running a proxy on your firewall > machine, if you have control of it. There are a few in the ports tree. > > Take a look at: > > net/kamailio - It is really a SIP proxy, but can parse SIP/SDP packets > and modify their content on the fly, allowing you to play neat tricks. > Requires some knowledge of the protocol and work though. (maybe you can > get him to punch holes in the firewall, but I have not checked if it's > possible) > > net/rtpproxy: this is more specific and maybe your best bet. Beware of > the load of proxying RTP in userland though. Yes, I'm aware of the problemacy regarding NAT and RTP. The problem is the pinholing of IPFW. > > > > > > One of the most disturbing and well known problems is that due to the > > stateful firewall the RTP session very often is half duplex - it seems one > > direction > > Depending on how many simultaneous SIP calls you plan to manage keep an > eye on your firewall too. each call will create at least 3 states, one > for SIP and one for each leg of the RTP stream, so it can pile up quite > fast. > > > of the RTP connection doesn't make it through IPFW/NAT. As often I search > > the net, I always get informed this is a typical problem and solutions are > > provided by so called ALGs - since SIP protocol's SDP indicates within the > > This would require coding it in IPFW, and the load on the firewall could > be significant. > > It could be done in userland maybe, leveraging divert(4) and having a > daemon listening there and doing the extra work, but this would be quite > expensive. Depending on your call volume the load could be too much for > your firewall. I thought this specific "divert" could be handled by some ng_XXX thingies of the netgraph() suite? As far as I know: egress calls open the IPFW in a stateful manner for the desired UDP ports (two of them if RTCP is not used, otherwise four, in and out) and if the prerequisite symmetric rtp is enforced and respected, the "other side" should send back RTP via the already opened ports. More complicated is the inbound/ingress direction, since the IPFW doesn't know about the desired UDP ports for RTP (or TCP, doesn't matter here), so somehow my(!) PBX has to open the IPFW some ways - and I thought this could be done by an inspection of SDP. I might be wrong here, since I havn't studied the protocol in deep. Linux folks talk about a "tracking of communications" - if I recall right. It sounded to me as they have in IPtables some facility doing exactly this inspecting stuff. > > > payload of the packets on which UDP ports both ends wish to establish their > > RTP session, it would be "easy" to pinhole the IPFW on exactly those ports > > for a theoretical large number of sessions, if IPFW could "divert" those > > packets to an instance inspecting SDP (or whatever is used for the RTP port > > indication, I'm new to that, sorry for the terminology) and then pinholing > > the NAT/IPFW for exactly this purpose without the forwarding mess. I came > > along netgraph() while searching for hints and hooks, but it seems a > > complete Linux domain, when it somes to appliances like VoIP/IP PBX. > > > > Either, the problem is that trivial on FreeBSD, so no further mentioning is > > necessary (which would explain the vast emptyness of explanations, hints and > > so on) or FreeBSD is a complete wasteland on this subject - which I also > > suspect, since pfSense and OPNsense must have come along with such problems > > and I simply do not know or recognise the software used for those > > purposes. > > I'm not aware of any silver bullets in those products for this. > > > > > So, if someone enlightened in this matter stumbles over my question and > > could delegate me onto the right way (ports, ng_XXX netgraph ficilities to > > look at, some ipfw techniques relevant to the problem apart from the stupid > > simple forwarding large ranges of ports) - I'd appreciate this and > > Maybe you could also get fancy with netflow, and that could have a lower > load but you'd have to create the IP parsing code yourself. > > Keep in mind that SIP, being used only for signaling, has a relatively > low cost on being manipulated(obviously it depends on the load of calls > being managed but it gets hundreds of calls starting and stopping every > few seconds to create an heavy SIP traffic). > > RTP on the other hand consists on a continuous UDP packet flow, and in > VoIP it means two for each call, which count directly on the PPS > (Packets per second) load of you networking equipment, performing > manipulation on those imposes a constant, heavy per connection load on > the machines performing it, so you should try to use static FW rules and > perform no further processing on it unless you have very low call > volume(let's say that more that 10 through firewall performing extra > manipulation on RTP streams is starting to be non trivial) or > appropriate hardware. All correct and I'd like to go with static rules as far as I can go (on the other hand, I do not know how IPFW is going to be forced into using dynamic rulesets on this specific matter ...). But I think the trivial concern here is the worst case in the RTP/NAT scenario: how to pinhole (or punchhole?) the IPFW? As I said above, one provider expects/send RTP from 10000/udp-30000/udp. It is easy to tell asterisk to use 10000-30000 via config rtp.conf. Again: as I understand outgoing connections, IPFW is opened "statefully" on the desired port and with symmetric RTP, media data flow on both ends on the same ports. But how is the incoming/ingress case handled? By forwarding the SIP signalling (and this means, the IPF is inherently opened for this port), incoming call request contains the desired RTP ports. In the stateful firewall case, asterisk now should try to contact the other side exactly via those both ports. But one of those ports is supposed to be the receiving port, so the asterisk could never establish a stable outgoing connection - and here is the problem, how to open IPFW (without ALG/inspecting) - or any stateful firewall? Either my understanding is completely bullshit or ... > > > Hope this all helps. > > > [1] > https://wiki.asterisk.org/wiki/display/AST/Configuring+res_pjsip+to+work+through+NAT > > [2] > https://wiki.asterisk.org/wiki/display/AST/Interactive+Connectivity+Establishment+%28ICE%29+in+Asterisk > Thank you very much. Kind regards, Oliver From owner-freebsd-current@freebsd.org Tue Sep 26 14:27:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6838DE0D1A2; Tue, 26 Sep 2017 14:27:14 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-lf0-x234.google.com (mail-lf0-x234.google.com [IPv6:2a00:1450:4010:c07::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC47867BFF; Tue, 26 Sep 2017 14:27:13 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-lf0-x234.google.com with SMTP id m199so4728157lfe.3; Tue, 26 Sep 2017 07:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=U6Nk1dzFPwcJM/CqFUnxO0JibOlKM8qChqzMRRekiKo=; b=Epe44ArCKEQV0KFmiEz5ajTeEWK9VUfT+M+6yihvg3hD3lH9fgQugKm8Nm+/p0T8tC SyfaQzQnQ+O5BxrNZedCGjt+DIcoRm6mnRI+Th6QFkvoKRXe0z2dOHNmecb3OyTfPEhV U1r0UpOvkcuNSS1krWg96Xc+ltOxcnAxoZUk/fcBH+xYBplmEMjpCAhH3Mw/+jFIPfTZ RpUZ1/AiVlV70uVrsU43309dOILgMLl5GTJSuBIpc2dRAwCVKzCyJT/8xGJ7UbCCCGMG epzUsLuuS2UwBp5+uAC3NZBgGMJDIACa7izeBIedcsQrXbdJGMqF2T6ERbVC15xQy2nS 0GmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=U6Nk1dzFPwcJM/CqFUnxO0JibOlKM8qChqzMRRekiKo=; b=dyI8oL2QFUHdhmfsqtK40D88qmSmi9Uaup4x9NJ8fBSXjL4q1xFqmexSQ1OMdAsqs/ ZgOxjLYpTGj3Ve4PT1M8tr2NuCEldVi89+6tI9It2v7OL+O+U6kXrDzOEkr4s0oHazVB 1uNq9UHA4pbxgvwiufI+YykkgRP48JmFkDH2UrUY5+j37Nu7NMvmBYpG4QZx+lMJ10AY kArcOlQynx2JOuqV89EFSh8OoZ1/lf4NrQarnhkbaBIODuH9BqpO0g6oA+eJ1E4YUWvd 7TYGd9FZ4T6pT+sWKQYfWfu7MTTpJ0r/cME23EcIN5DW8WIeWEo25vX00JFO5/jx9N+O 3rog== X-Gm-Message-State: AHPjjUg/QTwqcCRST/wwoPiI7au0vTKQeCVBGuRx8mDAfHeolFmmYB97 UtcGcySWsXN7jovPQxQ2QdrkInHZrKt2Pm4o1QvSdQ== X-Google-Smtp-Source: AOwi7QCwSK04ERIBvKDYhdJzEWEdrV6HUoy7Thmg27ppIrdH7+zkkv/nqcGky48GT0qL0gwBmg6hSo6QDgAT1ZtKDJg= X-Received: by 10.46.56.8 with SMTP id f8mr4319481lja.189.1506436031680; Tue, 26 Sep 2017 07:27:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.89.13 with HTTP; Tue, 26 Sep 2017 07:26:51 -0700 (PDT) In-Reply-To: <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> From: Damjan Jovanovic Date: Tue, 26 Sep 2017 16:26:51 +0200 Message-ID: Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem To: "O. Hartmann" Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 14:27:14 -0000 On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann wrote: > On Tue, 26 Sep 2017 11:00:45 +0200 > Damjan Jovanovic wrote: > > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann > > wrote: > > > > > Hello, > > > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I > ran > > > into > > > several problems. My questions might have a "noobish" character, so my > > > apology, > > > my experiences with IPFW are not as thorough as they should be. > > > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 > about > > > coming soon also supported? > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform > > > (assumed > > > having 2 GB of RAM)? > > > > > > My main concern is about IPFW (we do not use PF for some reasons, I > have to > > > stay with IPFW). > > > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not > yet > > > IPv6. > > > The FreeBSD system acting as a router is supposed to have a jail soon > > > containing the Asterisk 13 IP PBX (at the moment running on the main > > > system). > > > To provide access to the VoIP infrastructure inside/behind the > router/NAT > > > system, the in-kernel NAT facility of FreeBSD is forwarding the > relevant > > > UPD/TCP ports for VoIP to its destination network, and here I have a > > > problem to > > > solve. > > > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other > ports, > > > it > > > is a mess and pain in the arse to forward a whole range, say 11000/udp > - > > > 35000/udp, which is a range one of my providers is sending RTP on. A > second > > > provider uses another range for RTP, starting at 5000/udp. So, the > logical > > > consequence would be a union set up UDP range to forward, which exapnds > > > then > > > form 5000/udp to 45000/udp - which is much more a pain ... > > > > > > One of the most disturbing and well known problems is that due to the > > > stateful > > > firewall the RTP session very often is half duplex - it seems one > direction > > > of the RTP connection doesn't make it through IPFW/NAT. As often I > search > > > the > > > net, I always get informed this is a typical problem and solutions are > > > provided by so called ALGs - since SIP protocol's SDP indicates within > the > > > payload of the packets on which UDP ports both ends wish to establish > their > > > RTP session, it would be "easy" to pinhole the IPFW on exactly those > ports > > > for > > > a theoretical large number of sessions, if IPFW could "divert" those > > > packets > > > to an instance inspecting SDP (or whatever is used for the RTP port > > > indication, I'm new to that, sorry for the terminology) and then > pinholing > > > the > > > NAT/IPFW for exactly this purpose without the forwarding mess. I came > along > > > netgraph() while searching for hints and hooks, but it seems a complete > > > Linux > > > domain, when it somes to appliances like VoIP/IP PBX. > > > > > > Either, the problem is that trivial on FreeBSD, so no further > mentioning is > > > necessary (which would explain the vast emptyness of explanations, > hints > > > and > > > so on) or FreeBSD is a complete wasteland on this subject - which I > also > > > suspect, since pfSense and OPNsense must have come along with such > problems > > > and I simply do not know or recognise the software used for those > purposes. > > > > > > So, if someone enlightened in this matter stumbles over my question and > > > could > > > delegate me onto the right way (ports, ng_XXX netgraph ficilities to > look > > > at, > > > some ipfw techniques relevant to the problem apart from the stupid > simple > > > forwarding large ranges of ports) - I'd appreciate this and > > > > > > thanks in advance for patience and help, > > > > > > Oliver > > > > > > > > > Hi > > > > It might be easier if you just enable STUN on Asterisk, and build FreeBSD > > from source with my [largely neglected :( ] patch on > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918 > > > > That way Asterisk should dynamically discover consistent external > mappings > > for connections, making port forwarding RTP unnecessary. > > > > Damjan > > STUN is enabled, but my providers do not support STUN. > > I try to figure out how SIP works exactly to make my problem more precise. > I > also try to understand the aim of your patch - as far as I know, it does > exactly as it is needed for the IPW/NAT/VoIP case. And I really regret that > there are objections to commit the patch ... > > Firstly, if your providers support NAT, you register to them (as opposed to they register to you, or no registration), and the only VoIP calls are to/from your providers and to/from the same IP:port you register to (as opposed to unknown external addresses), then none of this should be necessary. Just put these on every SIP peer in Asterisk (this is for chan_sip; not sure about chan_pjsip): directmedia=no nat=force_rport,comedia and register to your provider more often than your NAT timeout is (eg. every minute), and you should be good. Why? Every registration opens a NAT mapping that your provider can use to send you calls on. The provider will also send RTP to the source IP:port it received it from, so when you start sending RTP you will get RTP back even if it's arriving from an unexpected IP:port. NAT is not a big problem for SIP clients, only for SIP providers that have receive packets from unknown addresses. Otherwise... Why would your providers need to support STUN? Applications first use STUN to discover the external IP:port of their internal IP:port, and then communicate that IP:port to the other side however they usually would (eg. headers in SIP and SDP packets) - the other side doesn't know or care that they were discovered through STUN. Any STUN server anywhere on the Internet can be used for this and should give the same results; see https://www.voip-info.org/wiki/view/STUN for a list. My patch ensures UDP NAT hole punching logic can be used properly. With it, if a packet was sent from an internal IP:port through an external IP:port (eg. to a STUN server), then any future packet from that internal IP:port to any other external server:port will go out the same external IP:port, and no other internal IP:port will use that external IP:port. It's like the internal IP:port temporarily owns the unique external IP:port and can send and receive through it to and from anywhere. The same source IP:port will be seen by all servers, and they can send back to it. From owner-freebsd-current@freebsd.org Tue Sep 26 13:41:59 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADDFAE0C099 for ; Tue, 26 Sep 2017 13:41:59 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2771966581; Tue, 26 Sep 2017 13:41:58 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LgHvY-1dc6Hw0S9M-00ng1W; Tue, 26 Sep 2017 15:41:56 +0200 Date: Tue, 26 Sep 2017 15:41:55 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current Subject: Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:+088Ni3wqUJaisiHvil2tMlUWXc1J/2xKPR6IcV02e1j7kvoMB1 aulDWSEq504AFFSGT42AQdZ38U6JDZ/nyrgvFSjXHfqy3nnHl/qcUWya061jnjlG/K6aHFu mP3legeFCsmmEPPEXNpZFixQu6npcFDK1PHAXPwAaTBRydXaOCk8IcuSQFiTBdPuQJye14p aS21VTrYScQQY84v7+MbQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:BoFkb/MONhQ=:Qk1a/hP3ltaifHdJzGD6bC sQxXgSy+JXLoet3On4uGdXQcuI6odbuqe3kX5npk5W484FjalQNYD07NUO7R3NqmpcCe5LyH1 8gHfiL+CgY19TAQRbHFCvRz+qTEjW2zNCxrsDiGm+3vIOUi/ajWQMwxKk5/wInO6zsRX91a7t Ys8zakg7O+74os7thWfzVDRIieuJ/KCfNGNLVv1f2YpLvx9g3FXRXC+JtPAa342eYWRO4eiJH MN4yFiNPlLQDLrhCTqiedJujz1aevnkTV91f5JLEt6cCjbMKVRlZD4WHm+7T2/X57AwRY+5V+ HzV3KK0weGcVmgSqczYbTcpuimTsE1wcwp0HXBd8K2MbLIOF9u9rYclAP0mCpJMTaElcjFqHu CoukhgfxWqGupTUEGjPcNjUlZ3VzUfeLGqhp5to2lCp5tl1LOYOwM1DQn+6IlSbilMishb4Zw JuS/nT3v+T1qHWwFUQSth5S1KvBIjrj+7B1HqBpB+NokVpSCcMEyDj/Usk+bwOTUufVMcrK0s 7FSGShHytexzvpiQTqh+1kJAGTjZi4Pzx0x0mckJcO4yYGN5v03wTxD5bR39i/FDlBsrLvXS4 YwArD7+NbNQDAaHBidB+ac2paVA4QzDjVIMHXBVvtmfILwmjqW6Ibea+YjIXh+Nsu0so3EPb+ 1WflYt29UgZnvMbRo4TxJBD6bobhOgDD718Sonb1Q3UsZ5RHocdO9XsPgqivjnjUfg2qg841u cVVG4jeRvnw42bM55zXRQddpj5OYSm+9i8X1afyLQsHsi7gMGT/uxulmDorknZwu6ZxrHb+7B dXW8iOSkkN5hJ+UGReqp3T5mZY1JJUdV2TqWicmRxW03FsHE10= X-Mailman-Approved-At: Tue, 26 Sep 2017 17:23:59 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 13:41:59 -0000 On Tue, 26 Sep 2017 15:06:23 +0200 Guido Falsi wrote: > On 09/26/2017 14:45, O. Hartmann wrote: > > Befor starting a PR I'd liek to ask for some advice to document a supposedly > > existent memory leak in net/asterisk13 and 12-CURRENT. > > > > Background: > > > > Running recent 12-CURRENT (FreeBSD 12.0-CURRENT #58 r323999: Tue Sep 26 > > 06:18:27 CEST 2017 amd64) on a PCengine APU 2C4, equipted with 4GB of RAM > > and 4080 KB free after the most recent SeaBIOS has been started, I try to > > utilise a net/asterisk13 as PBX on that platform. Asterisk13 has been build > > via poudriere and is compiled with CLANG/LLVM, not GCC! > > > > I'm running asterisk on similar hardware and also building with clang, > and have it going for months without any problems. I have disabled many > modules in that build though. Problem could be caused by some ancillary > library pulled in by some module. Since I run net/asterisk with automatic module loading (I'm new to asterisk), this is very likely and might cause the problem somehow. > > > When FreeBSD (NanoBSD) has been started, depending on the recent revision > > of the OS/Kernel, top shows ~3680 - 3705 MBytes of free memory. Starting > > net/asterisk13 via "service asterisk onestart" indicates an overall usage > > of up to 100 MBytes of memory. After having now run the Asterisk 13.17.2 > > daemon for two days resulted in ~ 3100 MBytes of free memory, while the > > asterisk daemon was still running and doing nothing. But performing several > > restarts on a freshly started box gives the same result after ~ 10 restarts > > - and even after stopping asterisk and let the system run for a couple of > > days doesn't free the memory. I stopped after 15 restarts or so watching > > the loss of memory after each restart of asterisk, so i came to the > > conclusion that there must be a memory leak somewhere. now i try to provide > > more informations as needed for a PR. > > > > Can someone help? > > Not sure, restarting the daemon should free any leaked memory the daemon > has. If a killed process leaves memory locked at the system level there > should be some other cause. Even with no runnidng asterisk, memory level drops after the last shutdown of asterisk and keeps that low. Even for weeks! My router never shows that high memory consumption, even under load. > > Have you tried diagnosing this memory with more tools? top so far ... > > > Depending on how deep you want to investigate ps, top and vmstat are > simple to use and give you some indication. To go deeper you'll need to > use more complicated tools. Others are better suited than me for > suggesting those. DTrace could be a powerful but not easy to use instrument. Well, DTrace would be an overkill for me. I'd like to provide more informations in case any FreeBSD developer wants to look at this - in case there is a real bug in either FreeBSD CURRENT - or net/asterisk. > > Looking just at free memory (which under a UNIX OS can have various > valid definitions, depending on how you account for inactive memory, > buffers and laundry RAM) is definitely not enough. Have you seen > asterisk process allocate more and more memory? > > You should check also what other processes in the system are doing. A > computer software when running never really does "nothing"(except, > maybe, when swapped out) and some memory usage can accumulate in many > parts of the system. Well, since it is a well defined NanoBSD system which has run for weeks with the same memory consumptions ebven under load, it seems obvious that there must be a black hole for memory when it somes to loading/unloading asterisk. The memory drain can be reproduced. > The question would be: how to use vmstat to give hints for those familiar with memory subsystems to indicate a real bug? I tried to find some advices, but maybe my English isn't good enough to make google help. Kind regards, Oliver From owner-freebsd-current@freebsd.org Tue Sep 26 13:44:34 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54017E0C204; Tue, 26 Sep 2017 13:44:34 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60D6866617; Tue, 26 Sep 2017 13:44:32 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LiTJE-1dMfvW25pk-00cjZn; Tue, 26 Sep 2017 15:44:30 +0200 Date: Tue, 26 Sep 2017 15:44:29 +0200 From: "O. Hartmann" To: Damjan Jovanovic Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:x6/3FELAFpyMg93sjhTdiuvoU182tuoOefvuYTiMytPfULC6BZM Qy0a3G7FIaBLyx5xjXpdOEoXiQQJvnfQMCf/luEp9prrRcX+f0paFLvyZv0ahTlJR4POxh1 ToGgGAsIf3/cE980TPugvPTyClgRYpou/KEW1a9uAN9PnUyFIDRQVHBxkX/PMdaQa3eL1GC Ce7vMwonkcYQkLZXM60FA== X-UI-Out-Filterresults: notjunk:1;V01:K0:GKDajNlfr5E=:MFqkx33eA1Ydwh5hMgRm4g 80rNpU8C6EoB3V9sNAGFwfkHl4IPQ8cGK23I+Ix5Bsy8grAL+6QyZtMdBD9SCieQLNdwyuIdV G2sDdToqljL3IXhw+7irxBLemfOMT+JXhBIUzgLsZcvh8eRkvqhZo8sVF9Ul59CvZqRLkmh6+ ZwXmk8ug991cdSN/0M/YIlekhUmvEl3AKk3l0ltW2VEEN/q+LQrcBHWSdBuRqAOJZkTcKUZW2 JqCkvMdlpbheunOeiedBPux67D3+Q/NerVzq17SRfylRicesvvIR6DqVoy/o4nrEgVjwT8iN0 GWlraGScE7WrhBr0TAyr096mfLQ67dIKjczqlixcyyuBllCO0BGQmRAYsPFXlbEYCNzove18g ORpwWaFXkzTpMe0VFhfRk7KdlI4RZZJOmQxYU2ZvNEexwkz7fHz8fThvM964XvMxj5xCnAQqM LvS16YguzgvT/ijujR91SQejviAJmQv5QSp+ajshI+Y1m29WcwaE61CErLdtTHsOgiYER+OVP NqBJGdG8eZTj9fTX/+ExQIcFgqipBW/T4ILsFWsQaO/FC3oExFk0rXW12qhzSkvXDIgakGz5y msgKEbSbQVETuUsfI4M3UAFR0vDAD85MBpoxTvn0m7aNqeyuE/gLsergo4lAhthi2S23h1Amn c2yAAx7frckqGlcEOc53u4j2K0+7KpLhGryxoADpn/Y0MkJwfJ/c26sPdH6g0qX2VG6NgqIg7 n8UQs6TYfdxUuCGggIxynV0gu6v3yfYel7ZfHMj1htglkDj89yU9nfav8drrAcZm6lHPuXG8e 0brTUMG3MB4smzfujwJBZfg3HQxDCDhfsbdwUL2fhKweh8czPs= X-Mailman-Approved-At: Tue, 26 Sep 2017 17:24:54 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Sep 2017 13:44:34 -0000 On Tue, 26 Sep 2017 11:00:45 +0200 Damjan Jovanovic wrote: > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann > wrote: > > > Hello, > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran > > into > > several problems. My questions might have a "noobish" character, so my > > apology, > > my experiences with IPFW are not as thorough as they should be. > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 about > > coming soon also supported? > > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform > > (assumed > > having 2 GB of RAM)? > > > > My main concern is about IPFW (we do not use PF for some reasons, I have to > > stay with IPFW). > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet > > IPv6. > > The FreeBSD system acting as a router is supposed to have a jail soon > > containing the Asterisk 13 IP PBX (at the moment running on the main > > system). > > To provide access to the VoIP infrastructure inside/behind the router/NAT > > system, the in-kernel NAT facility of FreeBSD is forwarding the relevant > > UPD/TCP ports for VoIP to its destination network, and here I have a > > problem to > > solve. > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports, > > it > > is a mess and pain in the arse to forward a whole range, say 11000/udp - > > 35000/udp, which is a range one of my providers is sending RTP on. A second > > provider uses another range for RTP, starting at 5000/udp. So, the logical > > consequence would be a union set up UDP range to forward, which exapnds > > then > > form 5000/udp to 45000/udp - which is much more a pain ... > > > > One of the most disturbing and well known problems is that due to the > > stateful > > firewall the RTP session very often is half duplex - it seems one direction > > of the RTP connection doesn't make it through IPFW/NAT. As often I search > > the > > net, I always get informed this is a typical problem and solutions are > > provided by so called ALGs - since SIP protocol's SDP indicates within the > > payload of the packets on which UDP ports both ends wish to establish their > > RTP session, it would be "easy" to pinhole the IPFW on exactly those ports > > for > > a theoretical large number of sessions, if IPFW could "divert" those > > packets > > to an instance inspecting SDP (or whatever is used for the RTP port > > indication, I'm new to that, sorry for the terminology) and then pinholing > > the > > NAT/IPFW for exactly this purpose without the forwarding mess. I came along > > netgraph() while searching for hints and hooks, but it seems a complete > > Linux > > domain, when it somes to appliances like VoIP/IP PBX. > > > > Either, the problem is that trivial on FreeBSD, so no further mentioning is > > necessary (which would explain the vast emptyness of explanations, hints > > and > > so on) or FreeBSD is a complete wasteland on this subject - which I also > > suspect, since pfSense and OPNsense must have come along with such problems > > and I simply do not know or recognise the software used for those purposes. > > > > So, if someone enlightened in this matter stumbles over my question and > > could > > delegate me onto the right way (ports, ng_XXX netgraph ficilities to look > > at, > > some ipfw techniques relevant to the problem apart from the stupid simple > > forwarding large ranges of ports) - I'd appreciate this and > > > > thanks in advance for patience and help, > > > > Oliver > > > > > Hi > > It might be easier if you just enable STUN on Asterisk, and build FreeBSD > from source with my [largely neglected :( ] patch on > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918 > > That way Asterisk should dynamically discover consistent external mappings > for connections, making port forwarding RTP unnecessary. > > Damjan STUN is enabled, but my providers do not support STUN. I try to figure out how SIP works exactly to make my problem more precise. I also try to understand the aim of your patch - as far as I know, it does exactly as it is needed for the IPW/NAT/VoIP case. And I really regret that there are objections to commit the patch ... From owner-freebsd-current@freebsd.org Wed Sep 27 07:05:48 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07B53E2CD20 for ; Wed, 27 Sep 2017 07:05:48 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C1AFB66AB8 for ; Wed, 27 Sep 2017 07:05:46 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y283r4wjBzZqh; Wed, 27 Sep 2017 09:05:44 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id eugQVOsIPVrb; Wed, 27 Sep 2017 09:05:42 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Wed, 27 Sep 2017 09:05:42 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" Cc: "O. Hartmann" , freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: Date: Wed, 27 Sep 2017 09:05:42 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 07:05:48 -0000 On 09/26/2017 15:41, O. Hartmann wrote: > On Tue, 26 Sep 2017 15:06:23 +0200 > Guido Falsi wrote: > Since I run net/asterisk with automatic module loading (I'm new to asterisk), > this is very likely and might cause the problem somehow. > You can exclude single modules from autoloading via modules.conf. >> Not sure, restarting the daemon should free any leaked memory the daemon >> has. If a killed process leaves memory locked at the system level there >> should be some other cause. > > Even with no runnidng asterisk, memory level drops after the last shutdown of > asterisk and keeps that low. Even for weeks! My router never shows that high > memory consumption, even under load. But while asterisk is running does the memory usage increase unbounded till filling all available memory or does it stabilize at some point? Asterisk is relatively memory hungry, especially with all modules enabled. It also caches and logs various information in RAM, even doing "nothing" it will cache and log that "nothing" activity. If memory does stabilize after some point it's not really a leak but it's standard memory usage. To reduce it you should disable all unused modules. > > The question would be: how to use vmstat to give hints for those familiar with > memory subsystems to indicate a real bug? > > I tried to find some advices, but maybe my English isn't good enough to make > google help. I'm not able to give you a correct indication, but if the memory usage is not increasing indefinitely but is stabilizing I'd say it's not really a leak. -- Guido Falsi From owner-freebsd-current@freebsd.org Wed Sep 27 09:23:38 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C58AE2F50D for ; Wed, 27 Sep 2017 09:23:38 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F17BC6AC61 for ; Wed, 27 Sep 2017 09:23:37 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MeMOx-1dhwpL3yV5-00Q8Ee for ; Wed, 27 Sep 2017 11:23:29 +0200 Date: Wed, 27 Sep 2017 11:23:22 +0200 From: "O. Hartmann" To: freebsd-current Subject: r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human Message-ID: <20170927112322.0d762681@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:hTcGfuBEF+/EtbmTYxfMW63cqPcYc87Cbg8mpZXiOnTG7S0MUO1 p+uDNtQ8RkVzWHntusFePHP2fIE72LY6wZZvXgRCNXKZzxBe+ysD6ou0ObKDbwANTAffN0R W3D8E2C3SaP9ycAJBzWCiBLsr4m3fiWN9G2kh/OHSAWGAgosCZcyPMu4DXBjmX53K0HWMpD O3kNZ9ALHe7mvcenn4uIg== X-UI-Out-Filterresults: notjunk:1;V01:K0:Y1dGI1fuChs=:zRGnWjhk6/kytXveShZmhh Rua+KG/bUGE3dcCAnHVZo42wdHooPTgGuAkugTg3MqeQfKJpYQ1MTIGRvWqKYMgVstnnkA94H d8Q0J1A3Wcu3rT52dLVGP7Z4Lp17jxCTpR/8gmqtS8beJrOquWOhqfO5Zp55YCCGEzWaLW3kq qXJ9kExPLWx1wB17dAcQdcfBNikdQMz32cHlzJsBJXDoSls1UL/K68MGtbxEIeJ79kATuEDHB phMzI7h8tJip0i7Ee5bFiEj6eVH6EKKVROFCkRb7nDyoI65OFu9/37orUMadS/6Gkkd4gQsoO xviIuJGRHvahhSOdFA2gUwSs2AlfAGbL7o8v1BrCbnkRqpIaDjHOTFosVOfXOhzKr4sWKTUWM LjoIsWVmxDIcVZoe7QFDitef+O5vn+Y2+gyuPoTLGlGs2pkVgTEG1rf3XhsWMQsygVrr+gtCo xEObHq1VzS3EtTH6r85L0PGAScrp3aMzKrD9zXOcvjXhNvc3rntQx3mG4cvaP0LCdRSBPjnt/ N0VYGgDo1iqyiJvyt71kOhQ+/BKBNAbWS8RyTxodPVjG5aRY9mru/kY4wq7XfMgT6/4esWRlc 9mM7pgLN1o6Uzd3eVmlQwvz9Iyo+dS9odbqdmNQ1cJYsYRmt8wGSF8EVR3mkkqvLpdSYrR7CC 2lc1Sred/a1enbz35WbrIqQvW7CFAsk1HMzQ5yfP94CCRgA+Q8ixGaesG2CCdoNJXei4xk6XE 9O3GswyHKFeLOLRYwRyJ6kR73wr/rehkkiAVssfeQG6fMhpLji00REBWgfts6fyQI/15Ox/t/ dllm5+o/ZE8U7gHY4rfa9SsZBDUvI86tyYhUuzHG7Lm1g52tJE= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 09:23:38 -0000 The installation of a newly, prsitine build kernel fails on CURRENT with the error shown below: [...] kldxref /boot/kernel kldxref: Parse error of description string U32:vendor;U32:device;P;D:human *** [afterinstall] Error code 1 make[3]: stopped in /usr/src/sys/modules From owner-freebsd-current@freebsd.org Wed Sep 27 10:23:33 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B68AE30638 for ; Wed, 27 Sep 2017 10:23:33 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C0B386C976 for ; Wed, 27 Sep 2017 10:23:32 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y2DRv5WsrzZrX; Wed, 27 Sep 2017 12:23:23 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id 6tGYyQdnvMgj; Wed, 27 Sep 2017 12:23:21 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Wed, 27 Sep 2017 12:23:21 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" Cc: "O. Hartmann" , freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170927112706.436500ec@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: <40903499-6cbb-53b7-e25a-1d7d96c47a75@FreeBSD.org> Date: Wed, 27 Sep 2017 12:23:20 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170927112706.436500ec@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 10:23:33 -0000 On 09/27/2017 11:27, O. Hartmann wrote: > On Wed, 27 Sep 2017 09:05:42 +0200 > Guido Falsi wrote: >> But while asterisk is running does the memory usage increase unbounded >> till filling all available memory or does it stabilize at some point? > > As far as I could observe, a three day test run of the router/firewall/asterisk > box drained around 500 MB of memory: starting at boot time with ~3700 MB, > asterisk leaves the box with ~3640 MB after bein started and after three days > the system reached ~3150 MB. Stopping asterisk gave back some memory, so ~3300 > MB then was for days the final result - not recovering anything further. I use > TEMPFS, if it matters, but I checked /tmp and /var/, there were no remnant > files so far. TMPVAR is only allowed to have 256 MB. > These numbers really don't tell us anything. The system has anyway been running for days, depending on configuration daemons like cron and ntp are running and performing tasks, things are being cached and so on, so that difference after three days could be perfectly normal overhead. You need to investigate the amount of memory allocated to asterisk with ps and top and check if that stabilizes. A few days, at most a week would be enough. After that, if it's not stabilizing you can start thinking on a leak, but still can't assume where the leak is happening. > Can't say whether it is stabilising or not - I think the runtime is too short. > I'll check first to disable some modules in the first place and then try to > perform a test with several days of asterisk enabled. > Whatever you prefer, but trying a few days uptime with all modules enabled is zero cost and east to do. Also you started this report with THIS configuration, changing configuration would prevent from comparing results. Let's test one thing at a time. -- Guido Falsi From owner-freebsd-current@freebsd.org Wed Sep 27 10:53:58 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E81E3E30BEF for ; Wed, 27 Sep 2017 10:53:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F1F46D3F4; Wed, 27 Sep 2017 10:53:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 34F552600FD; Wed, 27 Sep 2017 12:53:55 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: Guido Falsi , "O. Hartmann" Cc: "O. Hartmann" , freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> From: Hans Petter Selasky Message-ID: <31faf367-69e2-b7e3-dd14-67bf69a67ec2@selasky.org> Date: Wed, 27 Sep 2017 12:51:18 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 10:53:59 -0000 On 09/27/17 09:05, Guido Falsi wrote: > On 09/26/2017 15:41, O. Hartmann wrote: >> On Tue, 26 Sep 2017 15:06:23 +0200 >> Guido Falsi wrote: > >> Since I run net/asterisk with automatic module loading (I'm new to asterisk), >> this is very likely and might cause the problem somehow. >> > > You can exclude single modules from autoloading via modules.conf. > >>> Not sure, restarting the daemon should free any leaked memory the daemon >>> has. If a killed process leaves memory locked at the system level there >>> should be some other cause. >> >> Even with no runnidng asterisk, memory level drops after the last shutdown of >> asterisk and keeps that low. Even for weeks! My router never shows that high >> memory consumption, even under load. > > But while asterisk is running does the memory usage increase unbounded > till filling all available memory or does it stabilize at some point? > > Asterisk is relatively memory hungry, especially with all modules > enabled. It also caches and logs various information in RAM, even doing > "nothing" it will cache and log that "nothing" activity. If memory does > stabilize after some point it's not really a leak but it's standard > memory usage. To reduce it you should disable all unused modules. > >> >> The question would be: how to use vmstat to give hints for those familiar with >> memory subsystems to indicate a real bug? >> >> I tried to find some advices, but maybe my English isn't good enough to make >> google help. > > I'm not able to give you a correct indication, but if the memory usage > is not increasing indefinitely but is stabilizing I'd say it's not > really a leak. > Did you look at the output from "vmstat -m" and "vmstat -z" ? --HPS From owner-freebsd-current@freebsd.org Wed Sep 27 09:27:16 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9342BE2F6AC for ; Wed, 27 Sep 2017 09:27:16 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 083146AECD; Wed, 27 Sep 2017 09:27:15 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MA9Yv-1e3Q8Y3KOX-00BNVq; Wed, 27 Sep 2017 11:27:07 +0200 Date: Wed, 27 Sep 2017 11:27:06 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current Subject: Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170927112706.436500ec@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:DYGCNeLtIoMEedAbXlWA8tCeUjsjUMvUj7OxhFZ+ZbmOvXOkvSd Kt81KNw7dXDuJqGhHvewlvgIasveyVbsa8jxHbQJoAknHbJeGLyZ7XquUw/kwD3nveYOJ5e NapFWtAMHxuJiaBgG9x496K2bneSjSLIvY6r4/hUCcR+1QcO0TTziTBN+xnmN0Sg8amMH0G NJYHkqDfVvZuE3SCUVPPA== X-UI-Out-Filterresults: notjunk:1;V01:K0:P2rbc7mo0yI=:VxFJBX46Nu6niqpEkFjjdb RiqG3qmf7mTHznp6Pr2JxO+bXzL2auR15lWfpjvvGF1br/A7WT+Qx9d+sXr4MHNyUPP4ARQDe JAaDBWUaFthnGabEEJhmQocLZtlrvEZoa6dUKIPgajVA+POGgXu427s/fgEOkH7r02R6fQ0wC 8IhFfrsUmbA38hllPx09sHO3lycIMysTMl6BGxYJtO014H518fyl/lDAGJz+iXQ5tA0QvbrrJ UZq7R0QOFvGUBhBCs3CZ4dGqlKzMpL6YCbJpplbDfJEeuTXfwrnODikHOCJodXAHRhl4CDYB3 RMd8jZb9LxoAeLrUbW1xe9C3BEB8jg160RGoZERiy3JLiZtH2l/lX/HhJfU0Q015QJD0vvfeT izolnYvrnOC1gP93TWU4vRROrK8AD66ZFsDBXCCVxCGi9Xh4V11M465zDzxmHj/5S/gC9IHew 1zEvqyfLt1jczmAehuOLat29cgqX49JgYhc8H6bravz/ImGuEhv0iJGB/HH2bBQymouYJJ1lO mI4PVjdHiIob4M8S58+HAJq2wMLfeAxS+A8KaBnjPaW3cNG7IqLTdZxFzH68dFP/AypYN/tO4 LTXT/uRv1P4NjQZAEW+LAiEfMdoySt8yb5jn00Ev6tCokLlDfE9WtMIF4mC3iC3c7y561f327 UWROLuQwYBnQQ0CKUtBv5BUdZ2Ipb0NvkgXr3gl9tRifaqjfrK35Z7pLDM74JOO4ECXAR3DFM bqmr4KA/hBAmYJeC/4wGzpTstYGiWERjkpp0suY+JIgpdifPCxW+U3gwOqrlmfMmjpRkDc1Il FEaGwuYi+DPaergqDkpBAVJteVaWWwD756XFnXC0TOkQ4sRuqQ= X-Mailman-Approved-At: Wed, 27 Sep 2017 10:57:18 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 09:27:16 -0000 On Wed, 27 Sep 2017 09:05:42 +0200 Guido Falsi wrote: > On 09/26/2017 15:41, O. Hartmann wrote: > > On Tue, 26 Sep 2017 15:06:23 +0200 > > Guido Falsi wrote: > > > Since I run net/asterisk with automatic module loading (I'm new to > > asterisk), this is very likely and might cause the problem somehow. > > > > You can exclude single modules from autoloading via modules.conf. I tried, but I need to study first more documentation to explore what is prerequisite and what is optional. > > >> Not sure, restarting the daemon should free any leaked memory the daemon > >> has. If a killed process leaves memory locked at the system level there > >> should be some other cause. > > > > Even with no runnidng asterisk, memory level drops after the last shutdown > > of asterisk and keeps that low. Even for weeks! My router never shows that > > high memory consumption, even under load. > > But while asterisk is running does the memory usage increase unbounded > till filling all available memory or does it stabilize at some point? As far as I could observe, a three day test run of the router/firewall/asterisk box drained around 500 MB of memory: starting at boot time with ~3700 MB, asterisk leaves the box with ~3640 MB after bein started and after three days the system reached ~3150 MB. Stopping asterisk gave back some memory, so ~3300 MB then was for days the final result - not recovering anything further. I use TEMPFS, if it matters, but I checked /tmp and /var/, there were no remnant files so far. TMPVAR is only allowed to have 256 MB. > > Asterisk is relatively memory hungry, especially with all modules > enabled. It also caches and logs various information in RAM, even doing > "nothing" it will cache and log that "nothing" activity. If memory does > stabilize after some point it's not really a leak but it's standard > memory usage. To reduce it you should disable all unused modules. > > > > > The question would be: how to use vmstat to give hints for those familiar > > with memory subsystems to indicate a real bug? > > > > I tried to find some advices, but maybe my English isn't good enough to make > > google help. > > I'm not able to give you a correct indication, but if the memory usage > is not increasing indefinitely but is stabilizing I'd say it's not > really a leak. > Can't say whether it is stabilising or not - I think the runtime is too short. I'll check first to disable some modules in the first place and then try to perform a test with several days of asterisk enabled. From owner-freebsd-current@freebsd.org Wed Sep 27 11:26:30 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B18F4E31841 for ; Wed, 27 Sep 2017 11:26:30 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C1046E392 for ; Wed, 27 Sep 2017 11:26:29 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y2Frf6TnkzZrX; Wed, 27 Sep 2017 13:26:26 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id L89Cb7_6eqUK; Wed, 27 Sep 2017 13:26:24 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Wed, 27 Sep 2017 13:26:24 +0200 (CEST) Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> <20170926152955.205a012c@freyja.zeit4.iv.bundesimmobilien.de> Cc: FreeBSD CURRENT To: "O. Hartmann" From: Guido Falsi Message-ID: <38f78ee1-0946-f094-e9a2-42798f656cf4@FreeBSD.org> Date: Wed, 27 Sep 2017 13:26:24 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170926152955.205a012c@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 11:26:30 -0000 Sorry the top posting but Before anything else I need to state a few points. First this post is getting quite VoIP and asterisk specific, so please direct any further replies to me at my email. We're off subject on this list. Also before giving specific answers I need to clarify a few details about SIP/SDP and RTP, since, from your message it looks like you have some misconceptions about some details. In specific how RTP connection endpoints are negotiated and how the connections actually happen after that. (I'll make a general description to be sure everything is clear) SIP accepts connections on only one port (UDP usually but TCP is also supported), 5060 by default, but can be changed. This IP:port combination is what is registered with people wanting to call you, be it SIP REGISTER on a provider, a DNS SRV record, a blind connection on 5060 to an IP returned by an A record or manual registration via your provider website. (these are all method I've seen actually used, there may be others) After the caller connects via SIP to the called party they negotiate the connection. The "media channel" is usually negotiated by embedding SDP packets in the SIP exchange (it's all plain text, you can actually sniff it from the asterisk console, and in fact it's often necessary to be done to understand problems). There is absolutely no rule forcing RTP to be used as the media channel protocol, there are also others, but for phone calls that's what is usually negotiated. But RTP is a different protocol whose only relation to SIP is it being commonly used to actually relay the voice communication between endpoints. RTP is strictly defined to use UDP transport and is a monodirectional protocol, so for VoIP communication you need two RTP channels (or connections if you prefer), one for inbound and one for outbound. In the SDP negotiation each party (there no differentiation between caller and called party) will state where he expects the other party to send it's audio. Various information will be exchanged, including protocol audio format but most important for us IP:port. After negotiation each party performs the connection to the other. Let me rephrase: Your asterisk will be telling your provider to which IP:port is should send it's UDP connection for the audio stream, so you can't control where you are connecting (but that's not a problem, NAT correctly handles that) you have FULL CONTROL of the UDP port range the provider MUST use to connect to you (that is unless they are getting out of their way with a non compliant implementation, in such a case, look for a better provider). That said your situation is quite easy to solve without any special firewall configuration except redirecting a small (50-100) number of UDP ports of your choice. In real world situations this is quite handy since you rarely have full control of the router/firewall and many times all you get is a basic DSL modem/router with very limited FW/NAT functionality. On the other hand if you actually need thousands of UDP ports(that is thousands of simultaneous calls), most probably you also can get a static IP address for your PBX. On 09/26/2017 15:29, O. Hartmann wrote: > On Tue, 26 Sep 2017 11:27:05 +0200 > Guido Falsi wrote: > > I already tried to build net/asterisk13 on my AARCH64 jail, but since I'd like > to have the databases/postgresql96-client aboard and this specific port fails > to build, I gave up - it is, by the way, the only port (pgsql) as far as I > know that fails in my poudriere cross compiling jail. I did not get further, > but I saw that it is supposed to build only for amd64 and armv6. > >> >> In such a case would you be willing to test port changes on the hardware >> to actually check it runs? > > I'd like to if the efforts are not to much time consuming - I do not have a > working Pine64 image anymore, but I have a Pine64 with 2GB RAM at hand. Months > ago I started playing with cross compiling world/kernel for AARCH64, but I'm > not familiar with crochet and preparing the SD image - but willing to do. But > beware: my home box preparing the cross cimpilation is not the fastest! > Actually, I don't have ARM64 hardware available right now and no ARM64 jail right now, maybe in a not to distant future I'll have that and be able to perform specific tests, but things standing as they are I cannot state anything definitive on this subject. > For the moment, the ARM based PBX should perform SoHo tasks - three, up to ten > lines at maximum. So you need at most 1 port for SIP and 20 for RTP...a 50 ports range will have space to spare. > >> >> On the other hand if you plan doing a lot of audio transcoding or some >> video transcoding, load can get up quite fast. Compressed codecs like >> the "simple" G729 will make your load grow relatively fast even with >> audio transcoding. Transcoding also lowers call quality so it should >> anyway be avoided as much as possible. > > Good to know. But the PBX is more like an experiment for "at home's PBX" and, > if applicable, later for some fellows working scientifically in field and in > need for some small and neat equipement. Video message/streaming is not so much > the focus on the first attempts, but if possible, welcome. if not: not so > tragic. For 2-3 calls I think even a PCEngine or arm64 will have horsepower to spare, even for video calls, unless you go too fancy with resolution. >> >> Also load can go up if you're doing many disk operations. Monitoring and >> saving audio for a bunch of calls can be quite heavy on disk resources >> AND could require additional transcoding. > > There are Linux fellows running Asterisk 13 on raspberry Pi3 very successfully > and this little box has only 1GB RAM as far as I know. Why should FreeBSD fail? What do you mean by fail? Having said I don't have such hardware, does asterisk crash on that hardware? Does calls get lost? If the only problem that you mean by "fail" are NAT problems with the media path, that's common in many setups, and there are solutions. > I know. It is configured that way and it works well with one of my providers, > which has "ingres" calls via 10000/udp through 30000/udp. By simply doing a > forwarding of these portranges in the IPFW rules for the NAT section, my > firewall is open on that range! And this open range grows larger with the > second provider, which has another range on which its RTP communications is > attempting to establish ingress calls. As explained in the start of the message you are misunderstanding how things work here. That's the range you must use to connect to them, but, unless they are using non conforming SIP/SDP and RTP implementations, YOU choose the port range they must connect to you. Specifically with asterisk that range is defined in rtp.conf: [general] rtpstart=10000 rtpend=10050 (choose whatever you like, just avoid the low ports below 1024) Only problem is in SDP asterisk sends a IP:port combination, putting you own machine IP in there. That means it's putting there the LAN IP. If your IP address is static this is easy to resolve, in pjsip.conf you have the directives: external_media_address=1.2.3.4 external_signaling_address=1.2.3.4 local_net=172.16.0.0/12 you can put in your transports, and sip.conf has: localnet=172.16.0.0/12 externaddr = 1.2.3.4 With these you are telling asterisk to always announce the "1.2.3.4" IP when asking for inbound connections to IPs outside the local net. With dynamic IPs things are more tricky. There is no easy way to put the correct dynamic address in those variables and changing tham on the fly will require an asterisk restart(which is not really a big problem since by changing IP you already lost any ongoing calls and invalidated any existing registration). The dynamic IP case is where STUN TURN and ICE become very handy. ICE is quite handy, you just enable it, configure a STUN and, in available, TURN server, and it augments the SDP communication with extra information. It can make things work out of the box in quite complicated situations, even with no redirections on the firewall, but it requires support by both parties, so it could not be an option. But, having configured rtp as above and redirected that range on the firewall, not all is lost. You should check the res_stun_monitor module and it's configuration. There are free stun servers around the internet, so you should be able to use this even if your upstream does not provide it. I don't think it is supported by the pjsip module so in that case you should use the old sip module. >> Take a look at: >> >> net/kamailio - It is really a SIP proxy, but can parse SIP/SDP packets >> and modify their content on the fly, allowing you to play neat tricks. >> Requires some knowledge of the protocol and work though. (maybe you can >> get him to punch holes in the firewall, but I have not checked if it's >> possible) >> >> net/rtpproxy: this is more specific and maybe your best bet. Beware of >> the load of proxying RTP in userland though. > > Yes, I'm aware of the problemacy regarding NAT and RTP. The problem is the > pinholing of IPFW. Creating dynamic holes in IPFW is not necessary in 99% of times. As stated if you have low traffic you can just redirect a small range, if you have high traffic you should be able to afford a dedicated IP. Again remember that YOU control the inbound port range for RTP. Forwarding an high range is not really that risky either, those ports will be anyway closed on the asterisk box until it's expecting some connection, the only problem is that forwarding them causes load on the firewall and doing that with dynamic rules exposes you to DoS via dynamic rule exhaustion. So especially with an high range you should use static FW rules, not the other way round. Also note that in a professional settings there rarely is available a firewall with hole punching functionality on the client premises. People who actually need and have that CAN afford an extra IP usually. Not of lesser importance, there is no way to look at SIPS (SIP+TLS, so encrypted packets) content without terminating the SIP connection, that is you can't just watch inside the packets on the fly is using secure communication. That said if you really want to go the dynamic FW hole punching even if it has no professional use except edge cases and is quite more complicated, error prone and easy to break you should really check out the two ports i Suggested, especially kamailio. Kamailio is a SIP proxy, It will terminate your provider SIP connection and perform all the negotiations(so it can work with secure connections too), giving you a language in which you can check what is negotiated and even change it. I'm not sure but I think it will also allow you to launch scripts during it's negotiations. Such scripts could perform the actual hole punching. Kamailio will NEVER touch the RTP stream, so it will connect to asterisk via SIP informing it of the incoming call and your provider while keeping sending SIP packets to kamailio, will directly connect to asterisk and asterisk directly to your provider for the RTP streams. This is all perfectly standard and catered for by the SIP protocol > > All correct and I'd like to go with static rules as far as I can go (on the > other hand, I do not know how IPFW is going to be forced into using dynamic > rulesets on this specific matter ...). > > But I think the trivial concern here is the worst case in the RTP/NAT scenario: > how to pinhole (or punchhole?) the IPFW? As I said above, one provider > expects/send RTP from 10000/udp-30000/udp. It is easy to tell asterisk to use > 10000-30000 via config rtp.conf. Again: as I understand outgoing connections, > IPFW is opened "statefully" on the desired port and with symmetric RTP, media > data flow on both ends on the same ports. But how is the incoming/ingress case > handled? By forwarding the SIP signalling (and this means, the IPF is > inherently opened for this port), incoming call request contains the desired > RTP ports. In the stateful firewall case, asterisk now should try to contact > the other side exactly via those both ports. But one of those ports is supposed > to be the receiving port, so the asterisk could never establish a stable > outgoing connection - and here is the problem, how to open IPFW (without > ALG/inspecting) - or any stateful firewall? Either my understanding is > completely bullshit or ... Again: look at kamailio, it speaks SIP and can be used to do the inspection. Otherwise, for the inspection solution, there is nothing ready that I'm aware of in FreeBSD. There isn't code in IPFW able to parse SIP/SDP, nor there is a netgraph module with such ability. You should write it yourself. But I think they are not really needed. I hope this helps you understand better what is going on and what are your options. Also gives you material to study further maybe. -- Guido Falsi From owner-freebsd-current@freebsd.org Wed Sep 27 11:17:01 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC1EEE3150C; Wed, 27 Sep 2017 11:17:01 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 468EF6E016; Wed, 27 Sep 2017 11:17:00 +0000 (UTC) (envelope-from o.hartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0Lubnw-1dF8Ls1J6e-00znZD; Wed, 27 Sep 2017 13:16:52 +0200 Date: Wed, 27 Sep 2017 13:16:51 +0200 From: "O. Hartmann" To: Damjan Jovanovic Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org, Guido Falsi Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170927131651.7346fd1f@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:v3LzEPxmjXu2Ko9PCPGhR5SGOjVlzUFsXAypXZR7D7o8XVP4P8v FmuocXk4qW/qY9gSNavsPW+3O2NFr30d1HG9XX9JpNFJaQ6ns7DKWwGZQ7PCZSfzZwQPuQw HH4dcOQDdv/gtu+tOnAX4ujK3gaVlj5GgHAMKTzFsEKsDzjGubaqbzZ+jTUsxkMpxMcl4co z8aBVoo3ZwlqO2TfkM41w== X-UI-Out-Filterresults: notjunk:1;V01:K0:FeUxNHqNc70=:U3IkZfj5WUxukGcpcLaiEY RoTGSeMyYh9Jp/Zf5WXB8W4daoIIcZYjNH+98cAvRw91spOOIrCuuU2bdyU/Elw/wUzLPLcVl ZzL0bbQlfFROFoz1DvF7JUyFKxpPfKs0cp9P3ltr77cEz6ESXHLyuNga+zJSVGZ3W/ktb+Uqv tnZ9BXXf8FyH+fPKUw6eVq9RDo8zHdXwrHxMMvf+dX/7+Y1o263j3L9Kcs+aEiMsjyOKeOFqH F4r06VuULHucyhL/4UjYHa/ljjxU73WApWkEansmLy7Kk9v0UWL6c1HgbbQ7+VC0NFMFrVwXe J+LKMLzbHMVDyOv7txGg9RUkZmSyYxQ5zn3Malzt92dnb3dkhJ0jB7n4kNmvZqRFONbrm1e7K CWhqdz4O8SsGCtjp3dYhjwcFlYXjp0atew4N258rBdKfLUnz8qKdYFfmFCwCRecxCvvoQaqOM ODz1YF4tpEDLiyGtBQwn7+UC0spb8wWKXQxC7rJq/+DuGf/DmHF97tbIzUBJiRUXw5BANyCmH EIeazHWZYY7Kpc8VMhDxYsMMqDvi+Ueaqe0g4+MHG7RD/UtSwwh/O/PoNdbpIP4MltvS1ZSDH SA8XivzlMaumMbqQh0N85/WEXgebj2S/+2hApv9w3gwejRPvNLWwNvqLrKuJvXIVM/u1QEbQF qQKemeEF7t9Dt4c1wKc1NhfU70aB7SGGLYlgGUVHUvC/s4vZvuQcODSrSnAFr3e95wSETFaLb PnIlklShd+UKee59C0RW+f3dbXpE5a5Pv2oP/1dUnOd3mupb6XP3rAMXyQY7GoTCGA6kXKMXq uSQJB8bVq87qNhYByV3NkEp8rpYIzIeeSan43eqFjUP76UFwJg= X-Mailman-Approved-At: Wed, 27 Sep 2017 11:41:21 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 11:17:01 -0000 On Tue, 26 Sep 2017 16:26:51 +0200 Damjan Jovanovic wrote: > On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann > wrote: > > > On Tue, 26 Sep 2017 11:00:45 +0200 > > Damjan Jovanovic wrote: > > > > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann > > > wrote: > > > > > > > Hello, > > > > > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I > > ran > > > > into > > > > several problems. My questions might have a "noobish" character, so my > > > > apology, > > > > my experiences with IPFW are not as thorough as they should be. > > > > > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > > > > > - port net/asterisk13 is supposed to build only on armv6, is aarch64 > > about > > > > coming soon also supported? > > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform > > > > (assumed > > > > having 2 GB of RAM)? > > > > > > > > My main concern is about IPFW (we do not use PF for some reasons, I > > have to > > > > stay with IPFW). > > > > > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not > > yet > > > > IPv6. > > > > The FreeBSD system acting as a router is supposed to have a jail soon > > > > containing the Asterisk 13 IP PBX (at the moment running on the main > > > > system). > > > > To provide access to the VoIP infrastructure inside/behind the > > router/NAT > > > > system, the in-kernel NAT facility of FreeBSD is forwarding the > > relevant > > > > UPD/TCP ports for VoIP to its destination network, and here I have a > > > > problem to > > > > solve. > > > > > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other > > ports, > > > > it > > > > is a mess and pain in the arse to forward a whole range, say 11000/udp > > - > > > > 35000/udp, which is a range one of my providers is sending RTP on. A > > second > > > > provider uses another range for RTP, starting at 5000/udp. So, the > > logical > > > > consequence would be a union set up UDP range to forward, which exapnds > > > > then > > > > form 5000/udp to 45000/udp - which is much more a pain ... > > > > > > > > One of the most disturbing and well known problems is that due to the > > > > stateful > > > > firewall the RTP session very often is half duplex - it seems one > > direction > > > > of the RTP connection doesn't make it through IPFW/NAT. As often I > > search > > > > the > > > > net, I always get informed this is a typical problem and solutions are > > > > provided by so called ALGs - since SIP protocol's SDP indicates within > > the > > > > payload of the packets on which UDP ports both ends wish to establish > > their > > > > RTP session, it would be "easy" to pinhole the IPFW on exactly those > > ports > > > > for > > > > a theoretical large number of sessions, if IPFW could "divert" those > > > > packets > > > > to an instance inspecting SDP (or whatever is used for the RTP port > > > > indication, I'm new to that, sorry for the terminology) and then > > pinholing > > > > the > > > > NAT/IPFW for exactly this purpose without the forwarding mess. I came > > along > > > > netgraph() while searching for hints and hooks, but it seems a complete > > > > Linux > > > > domain, when it somes to appliances like VoIP/IP PBX. > > > > > > > > Either, the problem is that trivial on FreeBSD, so no further > > mentioning is > > > > necessary (which would explain the vast emptyness of explanations, > > hints > > > > and > > > > so on) or FreeBSD is a complete wasteland on this subject - which I > > also > > > > suspect, since pfSense and OPNsense must have come along with such > > problems > > > > and I simply do not know or recognise the software used for those > > purposes. > > > > > > > > So, if someone enlightened in this matter stumbles over my question and > > > > could > > > > delegate me onto the right way (ports, ng_XXX netgraph ficilities to > > look > > > > at, > > > > some ipfw techniques relevant to the problem apart from the stupid > > simple > > > > forwarding large ranges of ports) - I'd appreciate this and > > > > > > > > thanks in advance for patience and help, > > > > > > > > Oliver > > > > > > > > > > > > > Hi > > > > > > It might be easier if you just enable STUN on Asterisk, and build FreeBSD > > > from source with my [largely neglected :( ] patch on > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918 > > > > > > That way Asterisk should dynamically discover consistent external > > mappings > > > for connections, making port forwarding RTP unnecessary. > > > > > > Damjan > > > > STUN is enabled, but my providers do not support STUN. > > > > I try to figure out how SIP works exactly to make my problem more precise. > > I > > also try to understand the aim of your patch - as far as I know, it does > > exactly as it is needed for the IPW/NAT/VoIP case. And I really regret that > > there are objections to commit the patch ... > > > > > Firstly, if your providers support NAT, you register to them (as opposed to > they register to you, or no registration), and the only VoIP calls are > to/from your providers and to/from the same IP:port you register to (as > opposed to unknown external addresses), then none of this should be > necessary. Just put these on every SIP peer in Asterisk (this is for > chan_sip; not sure about chan_pjsip): My providers do support NAT, I suppose, I'm sure with one. Not so sure about the iberian Telefonica/O2 - they claim, but they are a kind of not willing to provide substantial informations as they want to force customers to use the (crap) equipment they offer. Very often, calling from the outside through the NAT/firewall to the PBX, I have this half-duplex phenomenon well described in many palces regarding NAT. In most cases I can hear the answering machine/voicemail from the PBX, but I can not send an audio stream inside. When my PBX register to its provider, how is the RTP port port for the ingress media stream (from the view of my PBX) opened? As I understand stateful IPFW, someone from the inside needs first to punchhole the firewall. I must confess, I have a bit of an understanding problem here since I do know know the protocol. Is there anything on the net explaining this scenario? RFC3261 is describing SIP, but not the registration ... > > directmedia=no > nat=force_rport,comedia In chan_pjsip, I found these settings important for NAT on peers in avrious places on the net: rtp_symmetric= yes ;rtp_keepalive= 30 (not sure about ; the timing here, I use this value) force_rport= yes rewrite_contact= yes timers= yes direct_media= no disable_direct_media_on_nat= yes > > and register to your provider more often than your NAT timeout is (eg. > every minute), and you should be good. Why? Every registration opens a NAT > mapping that your provider can use to send you calls on. The provider will > also send RTP to the source IP:port it received it from, so when you start > sending RTP you will get RTP back even if it's arriving from an unexpected > IP:port. NAT is not a big problem for SIP clients, only for SIP providers > that have receive packets from unknown addresses. I tried to find lifetime settings or timeout, the only (documented) values I founf were located in ipfw(8): net.inet.ip.fw.dyn_ack_lifetime: 300 net.inet.ip.fw.dyn_syn_lifetime: 20 net.inet.ip.fw.dyn_fin_lifetime: 1 net.inet.ip.fw.dyn_rst_lifetime: 1 net.inet.ip.fw.dyn_udp_lifetime: 10 net.inet.ip.fw.dyn_short_lifetime: 5 > > Otherwise... > > Why would your providers need to support STUN? Applications first use STUN > to discover the external IP:port of their internal IP:port, and then > communicate that IP:port to the other side however they usually would (eg. > headers in SIP and SDP packets) - the other side doesn't know or care that > they were discovered through STUN. Any STUN server anywhere on the Internet > can be used for this and should give the same results; see > https://www.voip-info.org/wiki/view/STUN for a list. I can use the STUN of an other provider, but not sure this is necessary, since both providers I'm consumer do not offer STUN themselfes. > > My patch ensures UDP NAT hole punching logic can be used properly. With it, > if a packet was sent from an internal IP:port through an external IP:port > (eg. to a STUN server), then any future packet from that internal IP:port > to any other external server:port will go out the same external IP:port, > and no other internal IP:port will use that external IP:port. It's like the > internal IP:port temporarily owns the unique external IP:port and can send > and receive through it to and from anywhere. The same source IP:port will > be seen by all servers, and they can send back to it. That sounds plausible, but implies that, say the PBX behind the NAT at IP1:port, is not guaranteed to send exclusively via external IP2:port? From owner-freebsd-current@freebsd.org Wed Sep 27 12:13:22 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 777F5E01ED1; Wed, 27 Sep 2017 12:13:22 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-wr0-x22f.google.com (mail-wr0-x22f.google.com [IPv6:2a00:1450:400c:c0c::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B9B7700F9; Wed, 27 Sep 2017 12:13:22 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-wr0-x22f.google.com with SMTP id h16so1841339wrf.6; Wed, 27 Sep 2017 05:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=3nnVqT89k7JX7Jg7I5DZqTWoXTq+ldKiYFcWxZX7Kmg=; b=l0Cpie+iCo1BHTo+DbrPNJtAundH6npz3KLoJKPL/547EuxQD7VS1MQBYLAscnxdmp elQM76JNmq1VZ5QXMKigUF2Ocf0XSga8XdMSkPrCpD84RqSpWzsqxEfgDDlk8kC6nPeX 5wrkShZydoccC0BlrBKNhgNFzfK4jGA2XUVqHn7b1QBm1JMfNZk5UB6tBqLnOhuVJcQb q5FTQ3qKyjY/dcWWrbBk+fZ7wM1W3/X0xRhpjr+9iGWkjlIs4zt1AyiHteOboKzC0t+v 9NULyxbWyCbuOQuzFvxZNSA3kMqkUsrq0ajToBumNi8//MpXUcSfmeN89dc2kFqU3cL4 zKvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=3nnVqT89k7JX7Jg7I5DZqTWoXTq+ldKiYFcWxZX7Kmg=; b=X3adIxRHQj+mApdHdPb9f2feuK0zC947dJGhVRhiZmP/9qByqfbn1t+ae6I9gGS/GJ Lrbq2HKYHlLl28A4NAOGkWPkKGPCME3SC1bq6sQGgn+ylhUcZctPZeKSEJrJnNp0gGcv QBPARbjDzR4DxUn1QMKetWqAyY4//SwTSvYduqrcFVrPQsAz9JAj1jb/d6slEbVjFpvA OqiSAFpku6L3JPST4EcmAZ8ZwL8ERXg4EXkcFpF+uluGFgmdmKEpzftdhcZrTHbIu/NC utOLCgj09yTKqgzhbP94Md+8TnO7os9MlsgVkTz7ra7c94tF7M7wPZEItb70tCw9uPeA x8Vg== X-Gm-Message-State: AMCzsaVUXT+/clfCKsW1O1Qu12OvCSXQNySGWDVgeilBDaKnwkmvEpG6 ZQdZpYO8IontU0pdagIQliO+yVCkr5YdvuyIHEu3lw== X-Google-Smtp-Source: AOwi7QAxp7AAYLVu7SYZia2NRNRgM4Ofivlm+6c5T1GzC6P+a9ajqpAX4A3eB0r6Vt2zrQoc3VTYTjQ/PO8z2cCwe2E= X-Received: by 10.25.24.231 with SMTP id 100mr467357lfy.241.1506514399417; Wed, 27 Sep 2017 05:13:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.89.13 with HTTP; Wed, 27 Sep 2017 05:12:58 -0700 (PDT) In-Reply-To: References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> From: Damjan Jovanovic Date: Wed, 27 Sep 2017 14:12:58 +0200 Message-ID: Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem To: Guido Falsi Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 12:13:22 -0000 On Tue, Sep 26, 2017 at 11:27 AM, Guido Falsi wrote: > On 09/26/2017 10:35, O. Hartmann wrote: > > > of the RTP connection doesn't make it through IPFW/NAT. As often I > search the > > net, I always get informed this is a typical problem and solutions are > > provided by so called ALGs - since SIP protocol's SDP indicates within > the > > This would require coding it in IPFW, and the load on the firewall could > be significant. > > It could be done in userland maybe, leveraging divert(4) and having a > daemon listening there and doing the extra work, but this would be quite > expensive. Depending on your call volume the load could be too much for > your firewall. > > SIP headers like Proxy-Authorization need to send a cryptographic quality hash of data that includes the password and the SDP when qop=auth-int, and the ALG needs to change the IP address and port in the SDP, which changes this hash. The ALG would have to know your password to calculate the new hash. A SIP ALG can thus only work with the weaker qop=auth protection, which doesn't hash the SDP and is thus less secure (MITM attacks can capture/modify RTP in transit), and even then it would have to be careful not to change the SIP headers which are included in the hash. Since it is the provider that chooses the allowed qop, a general SIP ALG is impossible unless the ALG knows the password. Linux has a SIP ALG in iptables, and it's full of problems and best disabled. From owner-freebsd-current@freebsd.org Wed Sep 27 12:17:37 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 847D6E020C3; Wed, 27 Sep 2017 12:17:37 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-wr0-x229.google.com (mail-wr0-x229.google.com [IPv6:2a00:1450:400c:c0c::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 031CB7030B; Wed, 27 Sep 2017 12:17:37 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-wr0-x229.google.com with SMTP id l39so16097737wrl.12; Wed, 27 Sep 2017 05:17:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QRoM+U6TjlWHC/2oeXvRicioSbJ/ypD1ZpoHVzLt/fE=; b=Ekv44o+pXOS/vcS0CH0Ji9nzS6qrCz8kIDN4vIfNm8mPV+7+BC1FesgvtSF1aK0R2E XaO3WQaT3Ol5SlpOm7Hz3kKSYSle392bAk+FYMwL24+jKuG5vn40/pWaex+yB9/Xz+Xf wknxe7ZauW2jMHr+4vAKcJ0g8IhoYz/099zzIkMWI3q6BO4HybxiFt3NzADvGotmvCvn qsBdsIIWdf9UCegiGHQDtif/jUjI0hsFCuMYryXi4d/1sXrsAoIYb7gWOLkIA/bDi9kh r1iXWcG/quDvj//dQZxZj1T5rOMDbZLKN5Mki60nuWWxo5Bi3GsGbVaOBTRFCSXuSkzY +Lyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QRoM+U6TjlWHC/2oeXvRicioSbJ/ypD1ZpoHVzLt/fE=; b=dt/Xzy8pxi88y1aIKRrGQJJswIM060g5Jy+yKQJd0A7gCc9ekF1ad5jE8r7WCPMb02 jb7caynP8DQ4a5uHD9LWOk6z4qrGIXAlfrjpom4vPVa60j0J9A9+wZ6g6W3FArSGu2wC Vp2guwVoUoYbi2bHzNHyviuvdnvISG1YSFiVjI+nzMVikBGKKaWnvBmKbLVMDunW7nDc Lcsdd9yp03ppbcFH3/Clvy0Qjr7Z18B5GoXZUs6x5lCkg35OKBiu3ztUT6/VpPukdeuo fD4KKGEBUbNZx6VpqhNVbb0Xdz+Bov/e14omX4emWODgo51I7oxaImF+3zwyABNzXFik N6QQ== X-Gm-Message-State: AMCzsaVHByrOVMssai2uPlrZ4lacGAx/yDbXcZJg/euR6n8iliaCu1ii puCXe/1II5d1YJ+nJYUttYpjBo58ZiJTg7UehoY= X-Google-Smtp-Source: AOwi7QDIjOt1koTdQTbK5sWZxl3XzSd3+LjY4RyVHao8k00OdL15Par7UjYBqA/Jti463PTv0QkbN1TtUOMuHGI0JoQ= X-Received: by 10.25.19.95 with SMTP id j92mr483866lfi.138.1506514655196; Wed, 27 Sep 2017 05:17:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.89.13 with HTTP; Wed, 27 Sep 2017 05:17:14 -0700 (PDT) In-Reply-To: <20170927131651.7346fd1f@freyja.zeit4.iv.bundesimmobilien.de> References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> <20170927131651.7346fd1f@freyja.zeit4.iv.bundesimmobilien.de> From: Damjan Jovanovic Date: Wed, 27 Sep 2017 14:17:14 +0200 Message-ID: Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem To: "O. Hartmann" Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org, Guido Falsi Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 12:17:37 -0000 On Wed, Sep 27, 2017 at 1:16 PM, O. Hartmann wrote: > On Tue, 26 Sep 2017 16:26:51 +0200 > Damjan Jovanovic wrote: > > > On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann > > wrote: > > > > > On Tue, 26 Sep 2017 11:00:45 +0200 > > > Damjan Jovanovic wrote: > > > > > > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann < > ohartmann@walstatt.org> > > > > wrote: > > > > > > > > > Hello, > > > > > > > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) > appliance, I > > > ran > > > > > into > > > > > several problems. My questions might have a "noobish" character, > so my > > > > > apology, > > > > > my experiences with IPFW are not as thorough as they should be. > > > > > > > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > > > > > > > - port net/asterisk13 is supposed to build only on armv6, is > aarch64 > > > about > > > > > coming soon also supported? > > > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX > platform > > > > > (assumed > > > > > having 2 GB of RAM)? > > > > > > > > > > My main concern is about IPFW (we do not use PF for some reasons, I > > > have to > > > > > stay with IPFW). > > > > > > > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and > not > > > yet > > > > > IPv6. > > > > > The FreeBSD system acting as a router is supposed to have a jail > soon > > > > > containing the Asterisk 13 IP PBX (at the moment running on the > main > > > > > system). > > > > > To provide access to the VoIP infrastructure inside/behind the > > > router/NAT > > > > > system, the in-kernel NAT facility of FreeBSD is forwarding the > > > relevant > > > > > UPD/TCP ports for VoIP to its destination network, and here I have > a > > > > > problem to > > > > > solve. > > > > > > > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other > > > ports, > > > > > it > > > > > is a mess and pain in the arse to forward a whole range, say > 11000/udp > > > - > > > > > 35000/udp, which is a range one of my providers is sending RTP on. > A > > > second > > > > > provider uses another range for RTP, starting at 5000/udp. So, the > > > logical > > > > > consequence would be a union set up UDP range to forward, which > exapnds > > > > > then > > > > > form 5000/udp to 45000/udp - which is much more a pain ... > > > > > > > > > > One of the most disturbing and well known problems is that due to > the > > > > > stateful > > > > > firewall the RTP session very often is half duplex - it seems one > > > direction > > > > > of the RTP connection doesn't make it through IPFW/NAT. As often I > > > search > > > > > the > > > > > net, I always get informed this is a typical problem and solutions > are > > > > > provided by so called ALGs - since SIP protocol's SDP indicates > within > > > the > > > > > payload of the packets on which UDP ports both ends wish to > establish > > > their > > > > > RTP session, it would be "easy" to pinhole the IPFW on exactly > those > > > ports > > > > > for > > > > > a theoretical large number of sessions, if IPFW could "divert" > those > > > > > packets > > > > > to an instance inspecting SDP (or whatever is used for the RTP port > > > > > indication, I'm new to that, sorry for the terminology) and then > > > pinholing > > > > > the > > > > > NAT/IPFW for exactly this purpose without the forwarding mess. I > came > > > along > > > > > netgraph() while searching for hints and hooks, but it seems a > complete > > > > > Linux > > > > > domain, when it somes to appliances like VoIP/IP PBX. > > > > > > > > > > Either, the problem is that trivial on FreeBSD, so no further > > > mentioning is > > > > > necessary (which would explain the vast emptyness of explanations, > > > hints > > > > > and > > > > > so on) or FreeBSD is a complete wasteland on this subject - which I > > > also > > > > > suspect, since pfSense and OPNsense must have come along with such > > > problems > > > > > and I simply do not know or recognise the software used for those > > > purposes. > > > > > > > > > > So, if someone enlightened in this matter stumbles over my > question and > > > > > could > > > > > delegate me onto the right way (ports, ng_XXX netgraph ficilities > to > > > look > > > > > at, > > > > > some ipfw techniques relevant to the problem apart from the stupid > > > simple > > > > > forwarding large ranges of ports) - I'd appreciate this and > > > > > > > > > > thanks in advance for patience and help, > > > > > > > > > > Oliver > > > > > > > > > > > > > > > > > Hi > > > > > > > > It might be easier if you just enable STUN on Asterisk, and build > FreeBSD > > > > from source with my [largely neglected :( ] patch on > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918 > > > > > > > > That way Asterisk should dynamically discover consistent external > > > mappings > > > > for connections, making port forwarding RTP unnecessary. > > > > > > > > Damjan > > > > > > STUN is enabled, but my providers do not support STUN. > > > > > > I try to figure out how SIP works exactly to make my problem more > precise. > > > I > > > also try to understand the aim of your patch - as far as I know, it > does > > > exactly as it is needed for the IPW/NAT/VoIP case. And I really regret > that > > > there are objections to commit the patch ... > > > > > > > > Firstly, if your providers support NAT, you register to them (as opposed > to > > they register to you, or no registration), and the only VoIP calls are > > to/from your providers and to/from the same IP:port you register to (as > > opposed to unknown external addresses), then none of this should be > > necessary. Just put these on every SIP peer in Asterisk (this is for > > chan_sip; not sure about chan_pjsip): > > > > My providers do support NAT, I suppose, I'm sure with one. Not so > sure about the iberian Telefonica/O2 - they claim, but they are a kind of > not > willing to provide substantial informations as they want to force > customers to > use the (crap) equipment they offer. > > Very often, calling from the outside through the NAT/firewall to the PBX, I > have this half-duplex phenomenon well described in many palces regarding > NAT. > In most cases I can hear the answering machine/voicemail from the PBX, but > I > can not send an audio stream inside. > > When my PBX register to its provider, how is the RTP port port for the > ingress > media stream (from the view of my PBX) opened? As I understand stateful > IPFW, > someone from the inside needs first to punchhole the firewall. I must > confess, > I have a bit of an understanding problem here since I do know know the > protocol. Is there anything on the net explaining this scenario? RFC3261 is > describing SIP, but not the registration ... > > Both sides usually send RTP to each other. When you send RTP through your NAT to a provider supporting NAT, it should see where you are externally sending from, and send its future RTP packets back there, even if that isn't the (internal) IP:port you previously said you would use in your SDP. This can obviously break in some cases: - If the voice is intentionally one-way throughout the call, such as phoning out into an announcement service that intentionally says "sendonly" in its SDP, so you aren't sending any RTP to it and its RTP can't route back to you. - If you use out of band ringback and transfer out an inbound call before it's answered, so the call hairpins from the provider through you and back. You have to send RTP to open NAT mappings, but you have nothing to send, as you first need to receive it, but can't as the NAT mappings aren't open: a cycle you can't exit. For those cases, NAT traversal can't be transparent, you have use some kind of software negotiated NAT traversal: static port forwarding and set Asterisk's external signaling and media addresses, use STUN with cone NAT (my patch + STUN/ICE settings in Asterisk's rtp.conf, sip.conf, etc.), or a NAT traversal protocol such as UPNP or NAT-PMP with supporting software (which Asterisk currently isn't). > > > > directmedia=no > > nat=force_rport,comedia > > In chan_pjsip, I found these settings important for NAT on peers in > avrious > places on the net: > > rtp_symmetric= yes > ;rtp_keepalive= 30 (not sure about > ; the timing here, I use this > value) > force_rport= yes > rewrite_contact= yes > timers= yes > direct_media= no > disable_direct_media_on_nat= yes > > > > > and register to your provider more often than your NAT timeout is (eg. > > every minute), and you should be good. Why? Every registration opens a > NAT > > mapping that your provider can use to send you calls on. The provider > will > > also send RTP to the source IP:port it received it from, so when you > start > > sending RTP you will get RTP back even if it's arriving from an > unexpected > > IP:port. NAT is not a big problem for SIP clients, only for SIP providers > > that have receive packets from unknown addresses. > > I tried to find lifetime settings or timeout, the only (documented) values > I > founf were located in ipfw(8): > > net.inet.ip.fw.dyn_ack_lifetime: 300 > > net.inet.ip.fw.dyn_syn_lifetime: 20 > > net.inet.ip.fw.dyn_fin_lifetime: 1 > > net.inet.ip.fw.dyn_rst_lifetime: 1 > > net.inet.ip.fw.dyn_udp_lifetime: 10 > > net.inet.ip.fw.dyn_short_lifetime: 5 > > > > Otherwise... > > > > Why would your providers need to support STUN? Applications first use > STUN > > to discover the external IP:port of their internal IP:port, and then > > communicate that IP:port to the other side however they usually would > (eg. > > headers in SIP and SDP packets) - the other side doesn't know or care > that > > they were discovered through STUN. Any STUN server anywhere on the > Internet > > can be used for this and should give the same results; see > > https://www.voip-info.org/wiki/view/STUN for a list. > > I can use the STUN of an other provider, but not sure this is necessary, > since > both providers I'm consumer do not offer STUN themselfes. > > > > > My patch ensures UDP NAT hole punching logic can be used properly. With > it, > > if a packet was sent from an internal IP:port through an external IP:port > > (eg. to a STUN server), then any future packet from that internal IP:port > > to any other external server:port will go out the same external IP:port, > > and no other internal IP:port will use that external IP:port. It's like > the > > internal IP:port temporarily owns the unique external IP:port and can > send > > and receive through it to and from anywhere. The same source IP:port will > > be seen by all servers, and they can send back to it. > > > That sounds plausible, but implies that, say the PBX behind the NAT at > IP1:port, is not guaranteed to send exclusively via external IP2:port? > > With my patch, every packet from IP1:port1 will be routed out of IP2:port2, no matter what the destination. Of course software must be written to detect IP2:port2 for every new socket using something like STUN, and report IP2:port2 to other parties it wants traffic from. From owner-freebsd-current@freebsd.org Wed Sep 27 12:21:37 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EECA3E027E8 for ; Wed, 27 Sep 2017 12:21:37 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BBD5F70948 for ; Wed, 27 Sep 2017 12:21:37 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id v8RC8Osu042188; Wed, 27 Sep 2017 12:08:24 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id v8RC8Nmq042187; Wed, 27 Sep 2017 05:08:23 -0700 (PDT) (envelope-from david) Date: Wed, 27 Sep 2017 05:08:23 -0700 From: David Wolfskill To: "O. Hartmann" Cc: freebsd-current Subject: Re: r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human Message-ID: <20170927120823.GT1165@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , "O. Hartmann" , freebsd-current References: <20170927112322.0d762681@freyja.zeit4.iv.bundesimmobilien.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="CLy2MrvMHpW9mjhY" Content-Disposition: inline In-Reply-To: <20170927112322.0d762681@freyja.zeit4.iv.bundesimmobilien.de> User-Agent: Mutt/1.9.0 (2017-09-02) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 12:21:38 -0000 --CLy2MrvMHpW9mjhY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 27, 2017 at 11:23:22AM +0200, O. Hartmann wrote: > The installation of a newly, prsitine build kernel fails on CURRENT with = the > error shown below: >=20 > [...] > kldxref /boot/kernel > kldxref: Parse error of description string U32:vendor;U32:device;P;D:human > *** [afterinstall] Error code 1 >=20 > make[3]: stopped in /usr/src/sys/modules > .... I also encountered this (though merely with the usual "WITH_META_MODE=3Dyes"), updating from r324007 to r324054. --=20 David H. Wolfskill david@catwhisker.org http://www.cbc.ca/news/opinion/donald-trump-playbook-1.4265374 See http://www.catwhisker.org/~david/publickey.gpg for my public key. --CLy2MrvMHpW9mjhY Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJZy5S3XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4XZW0H/2N63X3Hn7A4nsgXvrEiAyrg BEG+LAcUK0Pd+54j0QguBAWw0BvxrCb2py78OWAca6U6bV578sDdH5zoJWM3juNG SQSjWngKIDwCM/QKXMqbEaEvWXyeuAPTYodNs/+RIjHoKshtcQaJOwlLAfeNPIek ftiPpzzQ2rnS9qtnEF9No/D2oMbgAsxqVx7nx447kVNlKUap4r3UfEgX+VIRK7SY TgYIha2QoGmxJWvefFfA0+94cOBANWY1W7lWV53CYv5jpd5nCmw9oMmFhA+dq1yV /VwIUkDbP2ztuDBwChNwkKlPJxatz/Nvt/BA2rZq2a7wAhfATM65Cx/DqMHc+dc= =HxPf -----END PGP SIGNATURE----- --CLy2MrvMHpW9mjhY-- From owner-freebsd-current@freebsd.org Wed Sep 27 12:25:39 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BAD2E029BD for ; Wed, 27 Sep 2017 12:25:39 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FA3B70BB3; Wed, 27 Sep 2017 12:25:38 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id v8RCPbrq042331; Wed, 27 Sep 2017 12:25:37 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id v8RCPbwo042330; Wed, 27 Sep 2017 05:25:37 -0700 (PDT) (envelope-from david) Date: Wed, 27 Sep 2017 05:25:37 -0700 From: David Wolfskill To: "O. Hartmann" , freebsd-current Subject: Re: r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human Message-ID: <20170927122537.GU1165@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , "O. Hartmann" , freebsd-current References: <20170927112322.0d762681@freyja.zeit4.iv.bundesimmobilien.de> <20170927120823.GT1165@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="+y7rRxfO4bOZsSs1" Content-Disposition: inline In-Reply-To: <20170927120823.GT1165@albert.catwhisker.org> User-Agent: Mutt/1.9.0 (2017-09-02) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 12:25:39 -0000 --+y7rRxfO4bOZsSs1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 27, 2017 at 05:08:23AM -0700, David Wolfskill wrote: > On Wed, Sep 27, 2017 at 11:23:22AM +0200, O. Hartmann wrote: > > The installation of a newly, prsitine build kernel fails on CURRENT wit= h the > > error shown below: > >=20 > > [...] > > kldxref /boot/kernel > > kldxref: Parse error of description string U32:vendor;U32:device;P;D:hu= man > > *** [afterinstall] Error code 1 > >=20 > > make[3]: stopped in /usr/src/sys/modules > > .... >=20 > I also encountered this (though merely with the usual > "WITH_META_MODE=3Dyes"), updating from r324007 to r324054. > .... The code in question appears to have been added in r324038, affecting src/sys/dev/drm2/i915/i915_drv.c:1239 and src/sys/dev/drm2/radeon/radeon_drv.c:404. Peace, david --=20 David H. Wolfskill david@catwhisker.org http://www.cbc.ca/news/opinion/donald-trump-playbook-1.4265374 See http://www.catwhisker.org/~david/publickey.gpg for my public key. --+y7rRxfO4bOZsSs1 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJZy5jBXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4XmHcH/jjXaImqE6j8bgyTfaWD+Z9d Za4tqD20kdOQcdaP9sAOXee3FQShdBdUOFpYxYB4gERVKtW7+a+HZWcRnThXGTuX 0mXqCiukCzLypl+bFsA4s40E2DafwcLSlBP2OnK7uEG4wX0VLVz3pdhdK2S0ds4L Zqz00r7p8/oblPHt4jJNRBFYaFMiUd4/DaZeXq7MZ95An3l+j1sh9pkRJnTO+PrS uTBFcctCmsMke62luKU1Q67aER/soUZ2ovNqrCNVq9s5RoeHALfy1dlxZ/yMyjFz SdMD176HcyFbKu/1vpx1/A10jM26JC0lZnHnwTSzOqFs1IMSC10WwaboiIMUvQ0= =+kMh -----END PGP SIGNATURE----- --+y7rRxfO4bOZsSs1-- From owner-freebsd-current@freebsd.org Wed Sep 27 22:13:27 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3E9EE10AD7 for ; Wed, 27 Sep 2017 22:13:27 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C7333FF0 for ; Wed, 27 Sep 2017 22:13:27 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v8RMDLVx001129 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 27 Sep 2017 15:13:21 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v8RMDL1o001128 for freebsd-current@freebsd.org; Wed, 27 Sep 2017 15:13:21 -0700 (PDT) (envelope-from sgk) Date: Wed, 27 Sep 2017 15:13:21 -0700 From: Steve Kargl To: freebsd-current@freebsd.org Subject: panic: softdep_deallocate_dependencies: dangling deps Message-ID: <20170927221321.GA1023@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 22:13:27 -0000 Just got this panic on FreeBSD troutmask.apl.washington.edu 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r321800: Mon Jul 31 13:48:43 PDT 2017 kargl@:/data/obj/usr/src/sys/SPEW amd64 core.txt.0 contains panic: softdep_deallocate_dependencies: dangling deps cpuid = 7 time = 1506549566 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe023a281710 vpanic() at vpanic+0x19c/frame 0xfffffe023a281790 panic() at panic+0x43/frame 0xfffffe023a2817f0 softdep_deallocate_dependencies() at softdep_deallocate_dependencies+0x76/frame 0xfffffe023a281810 brelse() at brelse+0x149/frame 0xfffffe023a281870 bufwrite() at bufwrite+0x65/frame 0xfffffe023a2818b0 softdep_process_journal() at softdep_process_journal+0x7a8/frame 0xfffffe023a281950 softdep_process_worklist() at softdep_process_worklist+0x80/frame 0xfffffe023a2819b0 softdep_flush() at softdep_flush+0xff/frame 0xfffffe023a2819f0 fork_exit() at fork_exit+0x75/frame 0xfffffe023a281a30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe023a281a30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- __curthread () at ./machine/pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:232 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xffffffff805879eb in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xffffffff80587e66 in vpanic (fmt=, ap=0xfffffe023a2817d0) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xffffffff80587c83 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xffffffff80787f56 in softdep_deallocate_dependencies ( bp=0xfffffe01f008d8b8) at /usr/src/sys/ufs/ffs/ffs_softdep.c:14304 #6 0xffffffff8061dd69 in buf_deallocate (bp=0xfffffe01f008d8b8) at /usr/src/sys/sys/buf.h:429 #7 brelse (bp=0xfffffe01f008d8b8) at /usr/src/sys/kern/vfs_bio.c:2348 #8 0xffffffff8061b9e5 in bufwrite (bp=0xfffffe01f008d8b8) at /usr/src/sys/kern/vfs_bio.c:1914 #9 0xffffffff8079bec8 in softdep_process_journal (mp=, needwk=0x0, flags=) at /usr/src/sys/ufs/ffs/ffs_softdep.c:3559 #10 0xffffffff80785dc0 in softdep_process_worklist (mp=0xfffff80007eef000, full=0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:1592 #11 0xffffffff807894ff in softdep_flush (addr=0xfffff80007eef000) at /usr/src/sys/ufs/ffs/ffs_softdep.c:1397 #12 0xffffffff80555075 in fork_exit ( callout=0xffffffff80789400 , arg=0xfffff80007eef000, frame=0xfffffe023a281a40) at /usr/src/sys/kern/kern_fork.c:1038 #13 (kgdb) Hmmm, % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0 GNU gdb (GDB) 8.0 [GDB v8.0 for FreeBSD] Copyright (C) 2017 Free Software Foundation, Inc. Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done. ABI doesn't support a vmcore target OK, so debugging is broken :-/ -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-current@freebsd.org Wed Sep 27 22:20:00 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15D28E10C6F for ; Wed, 27 Sep 2017 22:20:00 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EA90F63458 for ; Wed, 27 Sep 2017 22:19:59 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v8RMJxwX001208 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 27 Sep 2017 15:19:59 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v8RMJx3U001207 for freebsd-current@freebsd.org; Wed, 27 Sep 2017 15:19:59 -0700 (PDT) (envelope-from sgk) Date: Wed, 27 Sep 2017 15:19:59 -0700 From: Steve Kargl To: freebsd-current@freebsd.org Subject: Re: panic: softdep_deallocate_dependencies: dangling deps Message-ID: <20170927221959.GA1198@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu References: <20170927221321.GA1023@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170927221321.GA1023@troutmask.apl.washington.edu> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 22:20:00 -0000 On Wed, Sep 27, 2017 at 03:13:21PM -0700, Steve Kargl wrote: > > Hmmm, > > % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0 > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done. > ABI doesn't support a vmcore target > > OK, so debugging is broken :-/ > It seems the devel/gdb80 port installs a kgdb symlink to kgdb80. So, I'm not getting the correct kgdb due to my path. Ugh. -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-current@freebsd.org Wed Sep 27 23:06:58 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C993DE12A72; Wed, 27 Sep 2017 23:06:58 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu [18.7.68.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1AC0B6474C; Wed, 27 Sep 2017 23:06:54 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074422-8b1ff70000004e42-dd-59cc2dd90556 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP id 58.1E.20034.9DD2CC95; Wed, 27 Sep 2017 19:01:45 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH-1.MIT.EDU [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id v8RN1ivs018522; Wed, 27 Sep 2017 19:01:44 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v8RN1dKE001435 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 27 Sep 2017 19:01:41 -0400 Date: Wed, 27 Sep 2017 18:01:39 -0500 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org Subject: FreeBSD Quarterly Status Report - Second Quarter 2017 Message-ID: <20170927230138.GB96685@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="HeFlAV5LIbMFYYuh" Content-Disposition: inline User-Agent: Mutt/1.8.3 (2017-05-23) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPKsWRmVeSWpSXmKPExsUixCmqrXtT90ykwYujWha7rp1mt5jz5gOT xfbN/xgtDjcLObB4zPg0nyWAMYrLJiU1J7MstUjfLoEr4/GbQ+wFixazViz9/YCxgXH9RJYu Rg4OCQETicZPaV2MXBxCAouZJL7NnMkM4WxklNjRN58JwrnKJHHu4QugDk4OFgFViefrFrCD 2GwCahLrV1xjBrFFBOQl9jW9B4szC1hL/HvQCBYXFrCTmHFwJROIzQu0rW35PGYIW1Di5Mwn LBD1ZRKf9qxiA7mIWUBaYvk/DpCwqICyxLx9q9gmMPLNQtIxC0nHLIQOiLC6xJ95l5gxhLUl li18zQxh20qsW/eeZQEj+ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJdU73czBK91JTSTYyg8GZ3 UdrBOPGf1yFGAQ5GJR7eCwtORwqxJpYVV+YeYpTkYFIS5W1UPhMpxJeUn1KZkVicEV9UmpNa fIhRBWjXow2rLzBKseTl56UqifCeZwaq401JrKxKLcqHKZPmYFES590WtCtSSCA9sSQ1OzW1 ILUIJivDwaEkwftbB6hRsCg1PbUiLTOnBCHNxMF5iFGCgwdoeCpIDW9xQWJucWY6RP4UYyxH 14qLf5g4jm26DCQPTLgCJPfdvAMkH924CyTn3ASRm8Dkiv/3gOSG7w/+MAmBXSwlzvsJZKgA yNCM0jy4vaB0KJG9v+YVozgwSIR5k4HJUYgHmErhNr8COooJ6KjeqSdAjipJREhJNTC6V1b9 nmBxaOKR6/84T5n9vmm2U3fBI8uXdklxXKtWKd5K/Z3wyz/mb9q2MyeKfyxwty5t+xhwnWnO gvO776woKHkXMm/JOpUkL8s9bAwMTnxPWtSyCxvn6962OZezLztVI7h8Pdvv6+8kVW+e/BIm rvBLbD/7VuuVbFrCVjLl/lJf64rY1m9XYinOSDTUYi4qTgQAcmBcT2IDAAA= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 23:06:58 -0000 --HeFlAV5LIbMFYYuh Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FreeBSD Quarterly Status Report - 2nd Quarter 2017 FreeBSD continues to defy the rumors of its demise. Much of the development work done this quarter was not particularly visible, especially the effort needed to ensure the upcoming 11.1 release has as few regressions as possible. Planning is also well under way for the 10.4 maintenance release which will quickly follow it. Further work focused on moving the arm architectures' support closer to tier-1 status and improving documentation. In addition, large changes were made to the src and ports trees. These projects and others are further detailed below. --Mark Linimon __________________________________________________________________ The deadline for submissions covering the period from July to September 2017 is October 21, 2017. __________________________________________________________________ FreeBSD Team Reports * FreeBSD Release Engineering Team * Ports Collection * The FreeBSD Core Team * The FreeBSD Foundation * The Postmaster Team Projects * 64-bit Inode Numbers * Capability-Based Network Communication for Capsicum/CloudABI * Ceph on FreeBSD * DTS Updates Kernel * Coda revival * FreeBSD Driver for the Annapurna Labs ENA * Intel 10G Driver Update * pNFS Server Plan B Architectures * FreeBSD on Marvell Armada38x * FreeBSD/arm64 Userland Programs * DTC * Using LLVM's LLD Linker as FreeBSD's System Linker Ports * A New USES Macro for Porting Cargo-Based Rust Applications * GCC (GNU Compiler Collection) * GNOME on FreeBSD * KDE on FreeBSD * New Port: FRRouting * PHP Ports: Help Improving QA * Rust * sndio Support in the FreeBSD Ports Collection * TensorFlow * Updating Port Metadata for non-x86 Architectures * Xfce on FreeBSD Documentation * Absolute FreeBSD, 3rd Edition * Doc Version Strings Improved by Their Absence * New Xen Handbook Section Miscellaneous * BSD Meetups at Rennes (France) Third-Party Projects * HardenedBSD __________________________________________________________________ FreeBSD Team Reports FreeBSD Release Engineering Team Links FreeBSD 11.1-RELEASE Schedule URL: https://www.FreeBSD.org/releases/11.1R/schedule.html FreeBSD Development Snapshots URL: https://download.FreeBSD.org/ftp/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes, and maintaining the respective branches, among other things. The FreeBSD 11.1-RELEASE cycle started on May 19, and continued as scheduled. FreeBSD consumers are urged to test whenever possible to help ensure the reliability and stability of the upcoming second release from the stable/11 branch. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Ports Collection Links About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/= ports-contributing.html FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html Ports Management Team Website URL: https://www.freebsd.org/portmgr/index.html FreeBSD portmgr on Twitter (@freebsd_portmgr) URL: https://twitter.com/freebsd_portmgr/ FreeBSD Ports Management Team on Facebook URL: https://www.facebook.com/portmgr FreeBSD Ports Management Team on Google+ URL: https://plus.google.com/communities/108335846196454338383 Contact: Ren=C3=A9 Ladan Contact: FreeBSD Ports Management Team This quarter, 2017Q2, broke the 30,000 ports landmark for the first time. The PR count is currently just under 2,500, with almost 600 of them unassigned. This quarter saw almost 7,400 commits from 171 committers. More PRs got closed this quarter than last quarter, but also more PRs got sent in, both of which are good to see. Over the past three months, we welcomed four new committers: Bradley T. Hughes (bhughes@), Danilo G. Baio (dbaio@), Jochen Neumeister (joneum@), and Richard Gallamore (ultima@). kan@ re-joined us as a ports committer. One commit bit, that of bf@, was taken in for safekeeping after a long period of inactivity. On the management side, the Ports Management Team welcomed back bapt@, who is working on several new features for the Ports Tree. The Ports Management Team also had its annual real-life meeting during BSDCan. On the infrastructure side, three new USES values were introduced: * cargo, to ease the porting of Rust packages or binaries using the cargo command (also covered separately in this report) * groff, to handle a dependency on the groff document formatting system, that has been removed from the base system for FreeBSD 12 * meson, to provide support for projects based on Meson The default version of PostgreSQL switched from 9.3 to 9.5, and that of Python3 from 3.5 to 3.6. The default generator for ports using cmake has been switched to ninja. Some major version updates are: pkg 1.10.1, Firefox 54.0.1, and Chromium 59.0.3071.115. Behind the scenes, antoine@ ran 36 exp-runs to test version updates, make the CRAN ports platform-independent, test installing bsdgrep(1) as /usr/bin/grep, test LLVM updates, test the ino64 project, and perform Makefile cleanups. __________________________________________________________________ The FreeBSD Core Team Contact: FreeBSD Core Team Core's activities during the second quarter culminated in the introduction of two new initiatives during BSDCan: * Extending FreeBSD Project Membership * The FreeBSD Community Process FreeBSD Project Members FreeBSD Project Membership being extended to more than just committers is a step that enables the Project to recognise and reward people who support us in ways other than by writing code. People that organise conferences or user groups; who are prominent supporters on social media; who triage bug reports and who test changes; and many others who contribute in various ways, are deserving of recognition for the support that they give to the Project. Core hopes that this will both encourage more people to volunteer their time and effort on behalf of the Project, and encourage those who already do to stick with the Project, if not become more deeply involved. The naming for the new group of non-committer Project members took a few tries to get right: having tried, and rejected, "Contributor" and then "Associate", Core took the view that since what they were offerring was formal Project Membership, then that was the right thing to call it. Committers thus become those Project Members with access to commit to the Project's code repositories. Project Members receive an @FreeBSD.org e-mail address, access to various Project hardware, access to internal mailing lists and other communications channels, and invitations to attend Developer Summits in their own right. Committers in addition have commit rights in the Subversion repositories and GitHub, and active Committers can vote in Core team elections. The FreeBSD Community Process This is an idea that has a long pedigree within other projects, and FreeBSD is very consciously modelling its implementation on what has worked elsewhere. When a significantly disruptive or wide-scale change is proposed, we should have a formal mechanism for documenting the change and what it implies. Interested parties can then respond and the change can be evolved into the best fit for all users, or else it can be found to be impracticable and withdrawn. The documentation of the change will remain as a point of reference should the same or a similar proposal come up in the future. Creating a more formal process should help avoid endless sterile arguments about what needs to be done, without anyone feeling they have sufficient investment in the idea nor backing from the majority of the project to justify putting in the work to achieve the desired result. The very first FCP -- FCP 0 -- describes the process itself. At the time of this writing, Core is voting on accepting the initial document, which can be viewed in the Project's Github repository. Two new mailing lists have been created: fcp@FreeBSD.org is the channel for receiving notifications of new FCP proposals and discussing their content, whilst fcp-editors@FreeBSD.org exists to provide help with the process of drafting the FCP documents. Other Core activities Core is delighted to announce that Gordon Tetlow has joined the Security Officer team, and will be working on managing the Security Team caseload, freeing up other members to concentrate on the more technical aspects of vulnerability remediation. In addition, Ed Maste has joined the Security Team and is available to assist the Security Officers where necessary. Although Florian Smeets had to step down, the postmaster team has recruited three new members and is now back up to strength. Considering the desirability of a number of fixes that have been merged into 10-STABLE since the 10.3 release, core has approved a 10.4 release to occur shortly after the 11.1 release. This will be a normal support-lifetime release, unlike the extended lifetime of the 10.3 release, so the overall support lifetime for the 10.x branch will not be significantly extended. During this quarter, Core has approved issuing three new commit bits. Please welcome: * Vladimir Kondratyev (wulf@) * Ryan Libby (rlibby@) * Kyle Evans (kevans@) Also, during this quarter, we had one person give up their commit bit: * Jordan Hubbard (jkh@) It is always unsettling when one of the Project's founding members decides to move on, but Jordan's interests have migrated away from FreeBSD-related projects and he has decided to hang up his bit once and for all. Core would like to thank NTTA (formerly Verio) for providing hosting for a cvsup mirror for many years, and also for their kind offer to provide ongoing hosting for a machine in their Seattle facility. Since we have no need for additional North America hosting, we have declined their offer. As usual, a number of questions have been raised about code licensing and other matters related to intellectual property. Ed Maste has registered "freebsd" on behalf of the FreeBSD Foundation on the Mastodon social media network. The "Unlicense" is suitable for code being imported into libc. We still have some code published under the old 4-clause style BSD license, where the extra clause refers specifically to the University of California. While UC has generally approved removing that clause, we need to check with all copyright holders before changing any remaining 4-clause licensing. Core, along with the Security Team, are monitoring developments concerning the "Stack Clash" vulnerability that hit the headlines during June. Changes to the stack-guard mitigation system are underway as a response to the proof-of-concept published by Qualys. __________________________________________________________________ The FreeBSD Foundation Links FreeBSD Foundation Website URL: https://www.FreeBSDFoundation.org/ FreeBSD Foundation Quarterly Newsletter URL: https://www.FreeBSDfoundation.org/wp-content/uploads/2017/06/FreeB= SD-Foundation-Q2-2017-Update.pdf Contact: Deb Goodkin Last quarter the Foundation was busy supporting the FreeBSD Project in so many ways! We brought on two interns from the University of Waterloo who were extremely productive, from working on a continuous integration project to adding MSDOS FAT filesystem support to makefs. We continued helping to accelerate OS changes with our internal staff of software developers, as well as funding outside software development projects, and continued promoting FreeBSD by participating in technology conferences around the world. To encourage more commercial users to donate to the Foundation, we launched a new partnership program. The FreeBSD 11.1 release effort has been led by a full-time Foundation employee, to continue keeping releases timely and reliable. Finally, we led the effort to celebrate the newly declared FreeBSD Day, to help raise awareness of FreeBSD around the world! Below, you can read some of the highlights from our Q2 newsletter, and find writeups throughout this status report from Foundation staff members including Ed Maste, Kostik Belousov, and Glen Barber. Don't forget, we are 100% funded by donations. Please take a moment to donate now, so we can continue supporting the FreeBSD Project and community worldwide! Q2 Development Projects Summary Our hard work continues into the 2nd quarter of 2017. Please take a look at the highlights from our more recent Development Projects summaries. April: FreeBSD USB Mass Storage Target Project Update The Foundation awarded a project grant to Edward Tomasz Napiera=C5=82a to develop a USB mass storage target driver, using the FreeBSD CAM Target Layer (CTL) as a backend. This project allows FreeBSD on an embedded platform, such as a BeagleBone Black or Raspberry Pi Zero, to emulate a USB mass storage target, commonly known as a USB flash stick. Read more at https://www.FreeBSDfoundation.org/blog/april-2017-development-project= s-update/. May: Foundation Brings on Co-Op Students At the beginning of May we embarked on a new path in the FreeBSD Foundation, with the hiring of co-operative education (co-op) students from the University of Waterloo. The University of Waterloo is a pioneer and leader in co-operative education, with 100% of Engineering students and a majority of Computer Science students participating in co-op programs. Read more at https://www.FreeBSDfoundation.org/blog/may-2017-development-projects-upd= ate/. June: FreeBSD Foundation 2017 Project Proposal Solicitation (contributed by Ed Maste) One of the ways the Foundation supports FreeBSD is by providing development grants for work on individual projects. These allow developers to propose projects they would like to undertake to improve FreeBSD and request funding to perform that work. The Foundation is always willing to receive proposals, but will occasionally issue a call for proposals to highlight specific areas of focus and to be able to collect and evaluate a group of proposals. The proposal submission deadline was July 14, 2017, but as mentioned above, people are welcome to submit proposals at any time. Although proposals may address any FreeBSD subsystem or infrastructure, we are particularly interested in receiving proposals related to: * Improvements to the security of FreeBSD itself, or of applications running on FreeBSD * New test cases, improved test infrastructure, and quality assurance * Improved software development tools * Projects to improve community collaboration and communication * Improving the FreeBSD "out of the box" experience for new users on various hardware platforms * Establishing FreeBSD as a leader in advancing projects of shared interest (such as ZFS, LLVM, or libarchive) More details can be found at https://www.FreeBSDfoundation.org/blog/FreeBSD-foundation-2017-project-p= roposal-solicitation/. The full project proposal submission guidelines can be found at http://cts.vresp.com/c/?FreeBSDFoundation/d364934d4d/TEST/1b229d9af7. Please do not hesitate to contact proposals@FreeBSDfoundation.org with any questions. Announcing the New Partnership Program (contributed by Deb Goodkin) I'm excited to announce our new FreeBSD Foundation Partnership Program! Our work is 100% supported by donations from individuals and organizations. With a spending budget of $1,500,000, we rely on large donations from our commercial users to help us sustain and increase our support. Recognizing the value of these donations, and putting together a sustainable funding model, we wanted to institute benefits that highlighted this support, and recognize these donors in productive ways. Partnerships are an avenue to assist commercial users by helping them get on board more quickly with FreeBSD, share their needs with the community, and facilitate collaboration with FreeBSD developers. We believe that building these relationships with commercial users will contribute to keeping FreeBSD relevant and help provide a sustainable and healthy ecosystem. You can check out our updated donor pages to see how we are acknowledging our Partners at https://www.FreeBSDfoundation.org/donors/. You can also find out more about this new program at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program= /. When I was in China last week, I had a chance to talk to a few companies about our new partnership program, and it definitely generated more interest in supporting our efforts. We are continuing to reach out to commercial users for help that will enable us to provide more outreach and support for FreeBSD. This includes funding more projects to improve FreeBSD, providing FreeBSD education and training, and recruiting more contributors to the Project. We can only provide the above support with your donations, and we need your help to connect us with your companies. Please consider notifying your organization about our new Partnership Program and helping to connect us with the appropriate contacts at your company. Your donations will help us: * Accelerate improvements and add new features to FreeBSD * Support release engineering efforts full-time * Create and provide FreeBSD educational and training material * Provide face-to-face opportunities for developers to work together * Improve and support FreeBSD infrastructure We need your support to continue improving FreeBSD. Q2 2017 Conference Recaps From sponsoring events to attending conferences, the Foundation continued its mission of advocacy in the second quarter of 2017. Over the past few weeks, members of the Foundation team represented the Project and the Foundation at events around the world. Below are just a few of the conference recaps. FOSSASIA 2017 (contributed by Philip Paeps) The Foundation kindly funded part of my travel from Tokyo to Singapore to attend FOSSASIA. I gave the "FreeBSD is not a Linux Distribution" presentation that Foundation board member George Neville-Neil wrote for Open Source China in December. My presentation was well-attended, and I got a lot of good questions from the primarily Linux-oriented audience. Read more at https://www.FreeBSDfoundation.org/blog/fossasia-2017-trip-report-philip-= paeps/. OSCON 2017 (contributed by Ed Maste) I represented the FreeBSD Foundation at OSCON 2017, which took place May 8-11, 2017, in Austin, TX: https://conferences.oreilly.com/oscon/oscon-tx . The Foundation booth was also staffed by FreeBSD committer Brad Davis and Doug Mcintire from Netgate. We met up Wednesday morning to set up the table. We were part of a "nonprofit pavilion" which consisted of eight or so tables, located between Open Camps and Operation Code. To help attract booth traffic, I brought a Raspberry Pi 3, with a small LCD display attached. As a demo, the Raspberry Pi showed a video of a Gource rendering of changes to the FreeBSD source tree over time (see example at https://www.youtube.com/watch?v=3DvZ8Sspua0Ks). Read more at https://www.FreeBSDfoundation.org/blog/conference-recap-oscon-2017/. Rootconf 2017 (contributed by Philip Paeps) In mid-May I presented at Rootconf 2017 in Bangalore. Rootconf is India's principal conference where systems and operations engineers share real-world knowledge about building reliable systems: https://rootconf.in/2017/. As always, it was interesting to hear the difficulties people face trying to run reliable systems on less reliable platforms. While many of the presentations were very Linux-specific and not very exciting to me, a couple of talks did catch my eye. I particularly enjoyed the talk by Aruna Sankaranarayanan (https://www.youtube.com/watch?v=3DXQJ7YhVoSWI&feature=3Dyoutu.be) explaining how Mapbox takes advantage of Amazon's "spot pricing" mechanism by spawning and shutting down machines at different price points to optimize for cost without compromising availability. Their spotswap https://github.com/mapbox/spotswap/ software has been released under a BSD license. It sounds as though it should be possible to port this to FreeBSD with minimal effort. Read more at https://www.FreeBSDfoundation.org/blog/rootconf-2017-trip-report-philip-= paeps/. BSDCan 2017/FreeBSD Developers Summit (contributed by Deb Goodkin) One of our initiatives is to assist in providing face-to-face knowledge sharing and development opportunities around the world. One way we do this is by sponsoring BSD-related conferences and FreeBSD Developer and Vendor Summits. We recently sponsored both BSDCan 2017 and the FreeBSD Developer and Vendor Summit in Ottawa, Ontario, Canada, which took place June 7-10, 2017. Many of our board and staff members attended the summit and conference to run tutorials, give presentations, lead sessions, work with developers, give demos, and share knowledge. In addition, this year we were pleased to bring our new University of Waterloo interns to the conference where they had the opportunity to demonstrate some of their projects at the Foundation table. Read more at https://www.FreeBSDfoundation.org/blog/conference-recap-bsdcan-2017Fr= eeBSD-developers-summit/. Open Travel Grant Applications The Foundation recognizes the importance of bringing members of the FreeBSD community face-to-face to both further development of the Project and spread the word about FreeBSD. Travel grants are available to community members who need assistance with travel expenses for attending conferences related to FreeBSD development and advocacy. Please note: the travel grant policy has been recently updated. Please carefully review it before submitting your application. More information about travel grants is available at: https://www.FreeBSDfoundation.org/what-we-do/grants/travel-grants/. FreeBSD Day was June 19! (contributed by Anne Dickison) June 19th was declared FreeBSD Day! Thank you to everyone who joined us in honoring the FreeBSD Project's pioneering legacy and continuing impact on technology. Find out more about FreeBSD Day and how we celebrated here at https://www.FreeBSDfoundation.org/blog/happy-FreeBSD-day/. Upcoming Events Find out about upcoming Foundation events at https://www.FreeBSDfoundation.org/news-and-events/upcoming-events/. FreeBSD Journal The May/June 2017 Issue of the FreeBSD Journal is now available. Don't miss articles on FreeBSD's Firewall Feast, CADETS: Blending Tracing and Security on FreeBSD, Toward Oblivious Sandboxing with Capsicum, and more. (https://www.FreeBSDfoundation.org/past-issues/security/) Did you miss the March/April issue? Check out articles on CFEngine, Puppet on FreeBSD, Vagrant, and more! (https://www.FreeBSDfoundation.org/past-issues/configuration-management/) As a recent addition of functionality, browser-based subscribers now have the ability to download and share PDFs of the articles! Sample Issue! If you've ever wanted to read through an entire issue of the FreeBSD Journal, now's your chance. Download the sample issue from https://mydigitalpublication.com/publication/?i=3D296880#{"issue_id":296= 880,"numpages":1,"page":1} and be sure to share with your friends and colleagues. Not a subscriber? Sign up today at https://www.FreeBSDfoundation.org/journal/. More information about the Foundation's doings and goings-on can be found in our own quarterly newsletter, linked above. __________________________________________________________________ The Postmaster Team Links The Postmaster Team URL: https://www.FreeBSD.org/administration.html#t-postmaster Contact: David Wolfskill Contact: Larry Rosenman Contact: Ryan Steinmetz Contact: Eygene Ryabinkin Contact: Remko Lodder Contact: Kurt Jaeger Postmaster handles the mail flow for the FreeBSD project. Clusteradm provides us with four jails: mailman, mailarchive, mx1, and mx2. In addition, there is some part of the setup running on freefall.FreeBSD.org. The system uses postfix, mailman, spamassassin, and some other tools from the ports tree to handle the mail flow. We use a very small, non-public Subversion repository for parts of the configuration. During Q2, Larry Rosenman, Kurt Jaeger, Eygene Ryabinkin, Remko Lodder and Ryan Steinmetz joined the Postmaster Team, and Florian Smeets left the Postmaster Team. Thanks to Florian for his long service in that role! David Wolfskill is planning to leave the role as soon as the new team members are settled. Vsevolod Stakhov plans to provide us with support to integrate rspamd into the setup, as well. The workload for the Postmaster Team is not high, but the complexity of the setup has its own demands. Open tasks: 1. We need to improve our internal documentation of workflows and processes. 2. We should consider adding some monitoring to provide quarterly numbers on the mail flow. __________________________________________________________________ Projects 64-bit Inode Numbers Links Phabricator Review URL: https://reviews.FreeBSD.org/D10439 Contact: Gleb Kurtsou Contact: Konstantin Belousov Contact: Kirk McKusick The 64-bit inode project was completed and merged into FreeBSD 12 on May 23, 2017. It extends the ino_t, dev_t, and nlink_t types to be 64-bit integers. It modifies the struct dirent layout to add a d_off field, increases the size of d_fileno to 64 bits, increases the size of d_namlen to 16 bits, and changes the required alignment of the structure. It increases the struct statfs f_mntfromname[] and f_mntonname[] array lengths from MNAMELEN to 1024. ABI breakage is mitigated by providing compatibility using versioned symbols, ingenious use of the existing padding in structures, and employing various other tricks. Unfortunately, not everything can be fixed, especially outside the base system. For instance, third-party APIs which pass struct stat as parameters are broken in backward- and forward-incompatible ways. The ABI for kinfo-consuming sysctl MIBs is changed in a backward-compatible way, but there is no general mechanism to handle other sysctl MIBS which return structures where the layout has changed. In our consideration, this breakage is either in management interfaces, where we usually allow ABI slippage, or is not important. The layout of struct xvnode changed, and no compatibility shims are provided. For struct xtty, the dev_t tty device member was reduced to be just uint32_t. It was decided that maintaining ABI compatability in this case is more useful than reporting a 64-bit dev_t value, for the sake of pstat. Updating note: strictly follow the instructions in UPDATING. Build and install the new kernel with the COMPAT_FREEBSD11 option enabled, then reboot, and only then install the new world. Credits: The 64-bit inode project, also known as ino64, started life many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick (mckusick) then picked up and updated the patch, and acted as a flag-waver. Feedback, suggestions, and discussions were carried out by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles), and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial ports investigation followed by an exp-run by Antoine Brodin (antoine). Essential and all-embracing testing was done by Peter Holm (pho). The heavy lifting of coordinating all these efforts and bringing the project to completion were done by Konstantin Belousov (kib). This project was sponsored by The FreeBSD Foundation (emaste, kib). __________________________________________________________________ Capability-Based Network Communication for Capsicum/CloudABI Links ARPC: GRPC-Like RPC Library That Supports File Descriptor Passing URL: https://github.com/NuxiNL/arpc Flower: A Label-Based Network Backplane URL: https://github.com/NuxiNL/flower Contact: Ed Schouten One of the weaknesses of Capsicum and CloudABI is that it is not easy to develop applications that need to make outgoing network connections, since system calls like connect() and sendto() are disabled. Though we can sometimes work around this by ensuring that the sandboxed process already possesses socket file descriptors on startup, this does not allow the destination process to be restarted, moved to a different network address, be load balanced, etc.. Coming up with a solution for this is quite important for me, as I am currently working on making CloudABI work on top of Kubernetes, Google's open source cluster management suite. The idea is that Kubernetes will schedule CloudABI processes instead of Docker containers. All of these CloudABI processes will have their dependencies on other services in the cluster injected explicitly, making internal communication very secure. All of this is intended to work on FreeBSD as well, of course! To solve this problem, I've been working on a daemon called Flower (read: flow-er) that allows software to register services and connect to them. Servers are identified by a set of labels with values (e.g., {datacenter: 'frankfurt', service: 'mysql'}). Clients can connect these servers by providing the corresponding label(s). Flower's security model is capability-based, just like Capsicum. The ability to bind and connect can be limited by permanently constraining labels to certain values. Flower has been designed not to act as a proxy. It does not copy any data. It merely forwards existing socket file descriptors or creates UNIX socket pairs and hands these out to its clients and servers. To realize this, processes communicate with Flower using an RPC library called ARPC. ARPC is a very simple clone of Google's GRPC, with the special feature that messages (Protobufs) can have file descriptors attached. This project was sponsored by Nuxi, the Netherlands. Open tasks: 1. Finish implementing the Flower code. 2. Integrate Flower with the Kubernetes/CloudABI runtime. 3. Release the Kubernetes/CloudABI runtime as open source software. __________________________________________________________________ Ceph on FreeBSD Links Ceph Main Site URL: http://ceph.com Main Repository URL: https://github.com/ceph/ceph My FreeBSD Fork URL: https://github.com/wjwithagen/ceph Contact: Willem Jan Withagen Ceph is a distributed object store and file system designed to provide excellent performance, reliability and scalability. * Object Storage Ceph provides seamless access to objects using native language bindings or radosgw, a REST interface that is compatible with applications written for S3 and Swift. * Block Storage Ceph's RADOS Block Device (RBD) provides access to block device images that are striped and replicated across the entire storage cluster. * File System Ceph provides a POSIX-compliant network file system that aims for high performance, large data storage, and maximum compatibility with legacy applications. I started looking into Ceph because the HAST solution with CARP and ggate did not really do what I was looking for. I aim to run a Ceph storage cluster of storage nodes that are running ZFS, with user workstations running bhyve on RBD disks that are stored in Ceph. Compiling for FreeBSD will now build most of the tools available in Ceph. The most important changes since the last report are: * Ceph has released release candidate v12.1.0 (aka Luminous); the corresponding packaging is sitting in my tree waiting for Luminous to be actually released. * ceph-fuse works, and allows mounting of cephfs filesystems. The speed is not impressive, but it does work. * rbd-ggate is available to create a Ceph rbd backed device. rbd-ggate was submitted by Mykola Golub. It works in a rather simple fashion: once a cluster is functioning, rbd import and rbd-ggate map are used to create ggate-like devices backed by the Ceph cluster. Other improvements since the previous report: * Some bugs in the init-ceph code (needed for rc.d) are being fixed. * RBD and rados are functioning. * The needed compatability code was written so that FreeBSD and Linux daemons can operate together in a single cluster. * More of the awkward dependancies on Linux-isms are deleted -- only /bin/bash is there to stay. The next forthcoming official release of Ceph is called Luminous (v12.1.0). As soon as it is available from upstream, a port will be provided for FreeBSD. To get things running on a FreeBSD system, run pkg install net/ceph-devel or clone https://github.com/wjwithagen/ceph, check out the wip.freebsd.201707 branch, and build manually by running ./do_freebsd.sh in the checkout root. Parts not (yet) included: * KRBD -- but rbd-ggate is usable in its stead. * BlueStore -- FreeBSD and Linux have different AIO APIs, and that incompatibility needs to be resolved somehow. Additionally, there is discussion in FreeBSD about aio_cancel not working for all device types. Open tasks: 1. Run integration tests to see if the FreeBSD daemons will work with a Linux Ceph platform. 2. Investigate the keystore, which can be embedded in the kernel on Linux and currently prevents building Cephfs and some other parts. The first question is whether it is really required, or if only KRBD requires it. 3. Scheduler information is not used at the moment, because the schedulers work rather differently between Linux and FreeBSD. But at a certain point in time, this will need some attention (in src/common/Thread.cc). 4. Improve the FreeBSD init scripts in the Ceph stack, both for testing purposes and for running Ceph on production machines. Work on ceph-disk and ceph-deploy to make it more FreeBSD- and ZFS-compatible. 5. Build a test cluster and start running some of the teuthology integration tests on it. Teuthology wants to build its own libvirt, and that does not quite work with all the packages FreeBSD already has in place. There are many details to work out here. 6. Design a virtual disk implementation that can be used with bhyve and attached to an RBD image. __________________________________________________________________ DTS Updates Contact: Emmanuel Vadot DTS (Device Tree Source) files provide a human-readable source description of the hardware resources for a given computer system (such as ARM- or MIPS-based embedded boards). The DTS source representation must be compiled into a binary format in order to be linked into the kernel and used to locate devices at runtime. The DTS files in FreeBSD were updated to match the versions from Linux 4.11, to represent more modern devices and provide more accurate representations. __________________________________________________________________ Kernel Coda revival Links GitHub Repository URL: https://github.com/trasz/FreeBSD/tree/coda Contact: Edward Tomasz Napiera=C5=82a Coda is a distributed file system developed as a research project at Carnegie Mellon University, descended from a older version of the Andrew File System. It got dropped from FreeBSD some five years ago, due to not having been adopted for a MPSAFE world. The focus for this current project is to bring it back into sufficiently workable shape that it could return to the kernel. It is currently in a working condition. Work is underway to test it better, fix whatever issues are found, and commit it to 12-CURRENT. This project was sponsored by Chalmers University of Technology. Open tasks: 1. Additional testing. 2. Update the userspace components (net/coda_client and net/coda_server). __________________________________________________________________ FreeBSD Driver for the Annapurna Labs ENA Links Enhanced Networking Guide URL: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networ= king.html Contact: Marcin Wojtas Contact: Michal/ Krawczyk The ENA (Elastic Network Adapter) is a 25G SmartNIC developed by Annapurna Labs and is based on a custom ARMv8 chip. This is a high-performance networking card available in the AWS offerings. It introduces enhancements in network utilization scalability on EC2 machines under the control of various operating systems, in particular FreeBSD. The goal of FreeBSD enablement is to provide top performance and a wide range of monitoring and management features such as: * multiple queue modes * hardware offloads (rx and tx checksum) * an admin queue * asynchronous notifications * robust hardware access * a scalable number of MSI-X vectors * hardware counters * watchdog mechanism * LRO * RSS The driver is available in the kernel source tree as of r318647. This project was sponsored by Annapurna Labs -- an Amazon company. Open tasks: 1. Add RSS configuration from userspace (via sysctls). 2. Add support for LLQ mechanisms. __________________________________________________________________ Intel 10G Driver Update Links Commit Adding X553 ix/ixv Support URL: https://reviews.FreeBSD.org/D11232 Contact: Chris Galazka Contact: Jeb Cramer The ix and ixv network interface drivers support a variety of Intel network interfaces, with line speeds at 10 Gbit/second. This quarter, the drivers gained support for the X553 network interface, which is found on System-on-a-Chip devices based on the Denverton platform. This update should allow FreeBSD to be more useful on a new class of hardware platform. Work is also underway to convert these drivers to use the iflib network driver library, which should ease future maintenance of the drivers, as well as the network subsystem as a whole. __________________________________________________________________ pNFS Server Plan B Links Testing Instructions URL: http://people.FreeBSD.org/~rmacklem/pnfs-planb-setup.txt Contact: Rick Macklem Parallel NFS (pNFS) is an extension to the NFSv4 protocol that allows for file accesses within a single logical mount to be performed against multiple file servers, with the potential for data access to occur in parallel. The pNFS "layout" in use specifies how the division occurs, with metadata operations occurring against the main server, and bulk data operations (read/write/setattr/etc.) occurring via a layout-specific scheme between the client and the data servers. My first attempt at a pNFS server using GlusterFS was a dud. It worked, but performance was so poor that it was not usable. This attempt that I call "Plan B", only uses FreeBSD, with one FreeBSD server handling the metadata operations and multiple FreeBSD servers configured to serve data, is now ready for third-party testing. If testing by third parties goes well, I anticipate the code will be merged into FreeBSD head in time for FreeBSD 12. Fairly recent FreeBSD or Linux systems should be usable as pNFS clients for testing. This server supports the File Layout, which is supported by both of these clients. There is no support for the Flex Files Layout or mirroring at this time. I hope to use the Flex Files Layout to add mirroring support over the next year or so. Striping is not supported, and I have no plans for implementing this at the moment. The patched FreeBSD sources may now be accessed for testing via either Subversion or download of a gzipped tarball. They consist of a patched kernel and nfsd and can be used on any FreeBSD 11 or later system. Open tasks: 1. Testing by others will be needed, now that the code is available. __________________________________________________________________ Architectures FreeBSD on Marvell Armada38x Contact: Marcin Wojtas Contact: Zbigniew Bodek Work proceeds to finalize the process of bringing support for the Marvell Armada38x platform into FreeBSD head. The most important parts of the recent effort are: * Add the network driver (NETA) * Enable coherent busdma operation for all ARMv7 SoCs * Add various low-level optimizations, such as L1 cache prefetch and MBUS quirks * Enable PL310 L2 cache controller * Add SDHCI support * Fixes for the e6000sw driver and a rework of its PHY handling * Support multi-port PCIe operation * Various fixes and enhancements of the common Marvell code * Fix and enable support for performance counters (HWPMC) This project was sponsored by Stormshield, Semihalf, and Netgate. __________________________________________________________________ FreeBSD/arm64 Links FreeBSD arm64 Wiki Page URL: https://wiki.FreeBSD.org/arm64 Contact: Andrew Turner Support for the Privilege Access Never (PAN) feature was added. This stops the kernel from accessing userspace memory, except through specific instructions. This helps security by only allowing access to userspace via the correct accessor functions. This is enabled on all supported CPUs that implement ARMv8.1 or later. The pmap code now supports the Unprivileged execute-never (UXN) and Privileged execute-never (PXN) bits in the page tables. These bits stop userspace and the kernel, respectively, from executing instructions on any marked page. The performance of the pmap layer has been improved. Many of the cache handling function calls have been removed. Some were needed early on to work around other bugs that have now been fixed. The removal of these calls has led to a large performance improvement. The kernel now uses crc32c instructions where appropriate. These are an optional set of instructions to perform crc32c checksumming quickly without using a lookup table.c The VM_MEMATTR_WRITE_THROUGH memory attribute is now supported. This is used to allocate memory for the framebuffer. Previously, the kernel would use cached memory; however, this leads to visual artifacts. The write-through flag fixes these by writing data out to RAM. The default linker on arm64 is now lld. This means that FreeBSD is able to build itself with just the components in the base system, a big milestone! __________________________________________________________________ Userland Programs DTC Contact: Emmanuel Vadot The in-tree DTC (Device Tree Compiler) was switched to use the BSD-licensed version by default. (The previous default DTC is licensed under the GPL.) The current version supports overlays and is able to compile every DTS (Device Tree Source) used by the FreeBSD arm releases. The ports GPL version was updated to the latest release (1.4.4). The in-tree GPL version is still present but the goal is to remove it before FreeBSD 12.0. __________________________________________________________________ Using LLVM's LLD Linker as FreeBSD's System Linker Links FreeBSD lld Wiki Page URL: https://wiki.FreeBSD.org/LLD FreeBSD/LLD Tracking PR (LLVM Bugzilla) URL: http://llvm.org/pr23214 Exp-Run Request Using lld as /usr/bin/ld URL: https://bugs.FreeBSD.org/214864 Contact: Rafael Esp=C3=ADndola Contact: Ed Maste LLD is the linker in the LLVM family of projects. It is a high-performance linker that supports the ELF, COFF and Mach-O object formats. It is broadly compatible with the common linkers used for each file format. For ELF this is the GNU Binary File Descriptor (BFD) ld and GNU gold. However, LLD's authors are not constrained by strict compatibility where it would hamper performance or desired functionality. LLD is now used as the default system linker for FreeBSD/arm64 and can link a working kernel, kernel modules, and userland for FreeBSD/amd64. LLD can also link a working kernel and modules (but not userland) for FreeBSD/arm and FreeBSD/i386. Work is ongoing to address ports that do not build with LLD as the system linker (either by fixing the port, or configuring the port to be linked by GNU ld). For FreeBSD 12.0 we expect to use LLD as the system linker for the same set of architectures that use Clang by default: 32- and 64-bit arm and x86. This project was sponsored by The FreeBSD Foundation. Open tasks: 1. Fix libtool to detect LLD and pass the same command line arguments as for GNU ld and gold. 2. Investigate the remaining amd64 and arm64 port build failures. 3. Investigate and improve LLD on i386 and arm, before the creation of the stable/12 branch. 4. Investigate and improve LLD on all other architectures. 5. Extensive testing. __________________________________________________________________ Ports A New USES Macro for Porting Cargo-Based Rust Applications Links Rust Homepage URL: https://www.rust-lang.org/ Cargo Homepage URL: https://crates.io/ Alacritty Homepage URL: https://github.com/jwilm/alacritty Exa Homepage URL: https://the.exa.website/ Ripgrep Homepage URL: https://github.com/BurntSushi/ripgrep Short Screencast About How to Use the USES=3Dcargo Macro URL: https://asciinema.org/a/SM2sOLi6iBUOmGWrxn5W1QI8U Contact: Tobias Kortkamp Support in the Ports Collection for applications written in the Rust programming language that use Rust's package manager Cargo was added, via a new USES=3Dcargo setting. The work is based on the cargo module from the OpenBSD ports tree. This should significantly ease the porting of Rust applications, as previously porters had to create their own tarball of the application's dependencies or find other manual ways of bringing them in. Several new ports were added that use it, for example: * Alacritty, a GPU-accelerated terminal emulator * Exa, a modern replacement for ls * Ripgrep, a line-oriented search tool that combines the usability of The Silver Searcher with the raw speed of GNU grep Open tasks: 1. Add documentation for the new feature. __________________________________________________________________ GCC (GNU Compiler Collection) Links GCC Homepage URL: https://gcc.gnu.org Issue Tracker Entry for the Update to GCC 6 URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D219275 GCC 5 Changelog URL: https://gcc.gnu.org/gcc-5/changes.html GCC 5 Porting Issues URL: https://gcc.gnu.org/gcc-5/porting_to.html Contact: Gerald Pfeifer Contact: Andreas Tobler The default version of GCC in the Ports Collection (the one requested by USE_GCC=3Dyes and various USES=3Dcompiler invocations) has been updat= ed from GCC 4.9.4 to GCC 5.4. This new major version brings many new capabilities and improvements, as well as some changes that may require adjustments. The latter category includes many new compiler warnings, significant improvements to inter-procedural optimizations, and link-time optimization. The default mode for C is now -std=3Dgnu11 instead of -std=3Dgnu89. The = C++ front end has full C++14 language support, including C++14 variable templates, C++14 aggregates with non-static data member initializers, C++14 extended constexpr, and more. The Standard C++ Library (libstdc++) has full C++11 support and experimental full C++14 support. It uses a new ABI by default. The lang/gcc port now is a meta-port that pulls in the respective lang/gccX port (based on the setting of $GCC_DEFAULT) and defines gcc, g++, and gfortran as symlinks to the respective versioned binaries. This is the end of a long journey establishing this infrastructure, which is now similar that used by the python ports, for example. Having the new infrastructure makes upgrading the default, as well as locally adjusting the default version, a lot easier. gcc8-devel has been added, and armv6hf support removed, and we made adjustments for newer versions of FreeBSD. Also of note are various cleanups and changes to improve the robustness of our packages and the addition of support for aarch64 to many ports. Thanks to dim@, jbeich@, tijl@, mat@, miwi@, linimon@ for assisting with this work. Open tasks: 1. The update of the default version of GCC from GCC 5.4 to GCC 6.4 is stalled, unfortunately. The work on the GCC and insfrastructure sides is complete, but unfortunately there are a number of broken ports that need to be adjusted/fixed. Any help is very appreciated; see PR 219275 for details. __________________________________________________________________ GNOME on FreeBSD Links FreeBSD GNOME Website URL: http://www.FreeBSD.org/gnome Development Repository URL: https://github.com/FreeBSD/FreeBSD-ports-gnome Upstream Build Bot URL: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD USE_GNOME Porter's Handbook Chapter URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/porters-handbook= /using-gnome.html Contact: FreeBSD GNOME Team The FreeBSD GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for FreeBSD. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel. After a period of not much activity, this quarter we started a little experiment in how we merge ports from the development repo to the FreeBSD Ports Collection. Instead of merging everything in one big commit, we have been updating the GNOME ports one at a time or in small groups. For example, the GTK+ stack and the Evolution Suite were updated as groups, and all the gnome-games components were done in one commit. It might be a bit more work preparing and testing the updates, but on the plus side, it easy to keep track of what is going on, and allows us to pay attention to the details. It should also make it easier to commit smaller changes. This quarter started with the update of GTK+ 3 to 3.22.15, and the underlying libraries to their latest stable versions. After the GTK+ update, work started on getting newer versions of other GNOME applications updated. The webkit2-gtk3 port was first updated to the 2.14 series and later to 2.16.3, which is the latest stable version. This step was needed because 2.16 couldn't be built on FreeBSD 10.3 without some required framework changes. harfbuzz-icu was split off from the main harfbuzz port. This drops the heavy icu dependency from the main harfbuzz port. A longstanding GLib/gio bug was fixed that had previously caused crashes of gnome-shell and other applications when share/applications was modified, as happens on pkg install or deinstall. Many of these updates are based on work previously done in the Gnome development branch by Ruslan Makhmatkhanov, Gustau Perez and Koop Mast. Open tasks: 1. Porting of Mutter/Gnome-shell/GDM 3.24 is complete. Unfortunately, GDM is blocking the update because of a "handoff" bug to the session after login. 2. Fix the printer submenu in gnome-control-center. As a workaround, system-config-printer can be used to configure printers. 3. MATE 1.18 is being QA tested and should arrive in early July. __________________________________________________________________ KDE on FreeBSD Links KDE on FreeBSD Website URL: https://FreeBSD.kde.org/ KDE Ports Staging Area URL: https://FreeBSD.kde.org/area51.php KDE on FreeBSD Wiki URL: https://wiki.FreeBSD.org/KDE KDE/FreeBSD Mailing List URL: https://mail.kde.org/mailman/listinfo/kde-FreeBSD Development Repository URL: https://github.com/FreeBSD/FreeBSD-ports-kde KDE's Continous Integration Dashboard URL: https://build.kde.org Blog Post on Using the Ninja CMake Generator URL: https://euroquis.nl/bobulate/?p=3D1600 Contact: KDE on FreeBSD Team The KDE on FreeBSD team focuses on packaging KDE and Qt, and making sure that their experience on FreeBSD is as good as possible. This quarter, in addition to the regular updates to the KDE, Qt, and related ports, there have also been some changes behind the scenes: our development repository has moved to GitHub, and FreeBSD is now part of KDE's official continuous integration (CI infrastructure). After the X.Org and GNOME ports teams, the KDE on FreeBSD team has moved its development repository to GitHub. This should make it easier for others to collaborate with us via pull requests, and by basing all our changes on top of the official ports tree we also hope this reduces the amount of conflicts and churn we need to deal with when landing big updates across the tree. We would like to thank iXsystems for hosting and supporting our area51 Subversion repository for many years. FreeBSD has finally joined KDE's CI (Continuous Integration) system as a tier-1 platform. KDE CI builds all the KDE sources -- 70 frameworks, the KDE Plasma Desktop and a plethora of KDE Applications -- continuously, straight from KDE's git repositories. There is strong commitment from upstream and the downstream KDE-FreeBSD team to reduce the amount of patching in the KDE ports to as little as possible. The first effects are being felt in expanding the set of unit tests to include FreeBSD-specific situations, and in extending Qt to handle FreeBSD filesystems better. In addition to the KDE sysadmins, we would also like to extend our thanks to Adriaan de Groot, who is both a KDE committer and part of our KDE on FreeBSD team, for spearheading these efforts. The following big updates landed in the ports tree this quarter: * CMake was updated to 3.8.0 and 3.8.2 * KDE Frameworks was updated to 5.33, 5.34 and 5.35 * The Calligra office suite was updated to 3.0.1, the first release in the ports tree to be based on KDE Frameworks 5, and the latest stable release upstream * The Konversation IRC client was updated to 1.7.2, the latest upstream release and the first ports version based on KDE Frameworks 5 * KchmViewer was updated to 7.7, which is based on KDE Frameworks 5 * LabPlot was updated to 2.3.0 and 2.4.0, and is now based on KDE Frameworks 5 * QtCreator was upated to 4.2.2 and subsequently to 4.3.0 * py-sip was updated to 4.19.2, PyQt4 to 4.12 and PyQt5 to 5.7.1 * Several fixes for ARMv6 landed in the Qt4 and Qt5 ports -- thanks to Mika=C3=ABl Urankar After several review rounds and exp-runs, Tobias Berner (tcberner@) finally made the Ninja generator the default for CMake-based ports, so that devel/ninja is used instead of (g)make in most cases. This should make most builds faster, even if only by a small margin. Adriaan de Groot also wrote a blog post about the change. __________________________________________________________________ New Port: FRRouting Links FRRouting Home Page URL: https://frrouting.org/ Contact: Olivier Cochard-Labb=C3=A9 FRRouting (FRR), a Quagga fork, is an IP routing protocol suite for Linux and Unix platforms which includes protocol daemons for BGP, IS-IS, OSPF and RIP (LPD and PIM support need to be fixed on FreeBSD). FRR is a Linux Foundation Collaborative Project with contributors including 6WIND, Architecture Technology Corporation, Big Switch Networks, Cumulus Networks, LabN Consulting, NetDEF (OpenSourceRouting), Orange, Volta Networks, and other companies. This project was sponsored by Orange. __________________________________________________________________ PHP Ports: Help Improving QA Links My Patreon Page URL: https://www.patreon.com/TorstenZuehlsdorff Contact: Torsten Z=C3=BChlsdorff As maintainer of the PHP ports, I first want to thank you all for the great feedback and patches I receive, in many forms. You keep my life interesting! In the past few months I learned a lot about various configurations, settings and bugs. Also, sadly, there are always PRs, patches and emails left unanswered, because of missing time on my side. I want to improve the situation by adding more automatic QA testing, but I need help to do so. Please send me your non-standard PHP-configurations or describe your exotic setups! These can be as simple as changed default versions, like LibreSSL instead of OpenSSL or the GCC version used for compiling. I, for example, always use another PostgreSQL-version than the default (and always PHP 7.1). Of course, this also covers port options set in an non-default way or setups that change variables to allow for multiple PHP installations, etc.. I plan to test on all supported FreeBSD versions, so you only need to mention if you are using an unsupported version. Note: Since PHP 7.2 is coming (hopefully on schedule), I will test PHP 7.2 from the onset with all the provided configurations, too. Open tasks: 1. Document the various configurations to be tested. 2. Setup the automatic QA infrastructure. __________________________________________________________________ Rust Links Wiki Portal URL: https://wiki.FreeBSD.org/Rust Guide to Bootstrap Rust on FreeBSD URL: https://gist.github.com/dumbbell/b587da50ef014078da9e732a4331ebad Bug Report to Track Progress on Bootstrapping URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D216143 Upstream Discussion of API/ABI-Breaking Changes URL: https://internals.rust-lang.org/t/pre-rfc-target-extension-dealing= -with-breaking-changes-at-os-level/5289 Contact: FreeBSD Rust team Rust was updated to 1.18.0 and Cargo to 0.19.0, the latest versions at the time of this writing. lang/rust was enabled on FreeBSD/aarch64 and work has continued on devel/cargo to achieve the same. We are also making slow progress to add support for even more platforms. Discussion has started upstream to support API/ABI-breaking changes between major releases of operating systems. For instance, this is required to be able to target both FreeBSD 11.x and 12.x, which have ABI changes involving important structures. Once support is added upstream, it will be possible to target a specific ABI and do cross-compilation. lang/rust-nightly was marked as broken for now. We need to revisit how the port is built so we can use the x.py script as recommended by upstream. Tobias Kortkamp (tobik@) created the USES=3Dcargo setting to make it easy to add Rust applications to the Ports Collection. This is further detailed in a separate entry in this quarterly status report. The compiler, rustc, is crashing sometimes when there is a compilation error. Therefore, there is a bit of work to do to improve its stability. There is some code duplication between the lang/rust* and devel/cargo Makefiles. These all deserve a bit of cleanup, and it might be useful to create a USES=3Drust Makefile helper. Open tasks: 1. Bootstrap Rust on more platforms. 2. Investigate compiler crashes. 3. Investigate how to speed up lang/rust* compilation times. __________________________________________________________________ sndio Support in the FreeBSD Ports Collection Links Sndio Homepage URL: http://www.sndio.org Sndio Paper URL: https://www.openbsd.org/papers/asiabsdcon2010_sndio.pdf Comprehensive and Biased Comparison of OpenBSD and FreeBSD (Section 17) URL: https://www.bsdfrog.org/pub/events/my_bsd_sucks_less_than_yours-As= iaBSDCon2017-paper.pdf Contact: Tobias Kortkamp sndio is a small audio and MIDI framework that is part of the OpenBSD project. It provides a lightweight audio and MIDI server, sndiod. It currently supports OpenBSD, FreeBSD, DragonFly BSD, and Linux. The porting effort to FreeBSD and OSS started last year and the sndio backend support in the FreeBSD Ports Collection can now be considered good enough for daily use. Sndio offers network transparency through sndiod, which provides an easy way to share your audio devices with other machines/VMs/jails on your network. However, applications and libraries need to support playing and recording through it. To that end, I submitted several patches to various ports over the course of the last year. Here's a short selection of ports that now support sndio in the FreeBSD Ports Collection: * Most games, via audio/openal-soft, devel/sdl12, and devel/sdl20. * GStreamer-based applications and WebKit-based browsers through two new GStreamer plugins (audio/gstreamer1-plugins-sndio and audio/gstreamer-plugins-sndio). * Firefox, Firefox ESR, Seamonkey, Chromium, and Iridium. The browsers currently lack or have a non-functional OSS backend. Sndio support provides a BSD-native alternative to the ALSA and PulseAudio backends. * Video players like VLC, Totem, mpv, mplayer, etc.. * Audio players like Clementine, cmus, mpd, mpg123, siren, xmp, etc.. * SoX. * Shairport Sync, through a newly implemented backend. * JACK. * PulseAudio, through audio/pulseaudio-module-sndio. Open tasks: 1. Commit a backport of Kodi's new sndio backend to the Ports Collection. 2. If you maintain or use an audio-related port, consider checking whether it includes an sndio backend, and adding an SNDIO option. Thanks to the OpenBSD developers, several open-source projects already include one, so adding it might be very easy to do. __________________________________________________________________ TensorFlow Links TensorFlow PR URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D219609 Phabricator Review URL: https://reviews.FreeBSD.org/D11194 Prebuilt Packages URL: https://github.com/amutu/tf-FreeBSD-pkg TensorFlow Upstream URL: https://www.tensorflow.org Contact: Jov As described on its website, "TensorFlow(TM) is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well." TensorFlow now is the most popular platform/library for machine learning and AI. There are official binaries for Linux, Mac, Windows, and Android, but no official support for FreeBSD. For the last several months, I have done some work to make TensorFlow available on FreeBSD. Some notable items include: * bazel was patched to not depend on /proc at build time. bazel is a build tool made by Google. It uses /proc to get path-to-self when building C++ code, but mounting /proc is usually not allowed when building as an unprivileged user. * TensorFlow can now be built on FreeBSD 10.x by using clang38 as the default bazel cross-build tool. * Patch the bazel workspace files to allow TensorFlow to be built using offline third-party dependencies. This work is needed because the FreeBSD Ports framework does not allow network access except during the fetch stage. * Fix the build on FreeBSD i386. * Make TensorFlow build with either Python 2 or Python 3. * Update to the latest version, which is tensorflow-1.2.0. TensorFlow can now be run on FreeBSD in CPU-only mode. Some functional tests have been performed on some combinations of FreeBSD 10.3-RELEASE and 11.0-RELEASE, amd64 and i386, and Python 2.7 and Python 3.6. This port would not be possible without substantial assistance from bapt@, lwhsu@, mat@, and koobs@ -- thank you for your advice, review, and help! You are very nice and I learned a lot about FreeBSD and the Ports framework from you. Open tasks: 1. Review, test, comment, and most importantly, commit to the Ports Collection. 2. Fix OpenCL (GPU acceleration) support on FreeBSD. 3. Port tensorflow-serving, which is a flexible, high-performance serving system for machine learning models produced by TensorFlow. 4. Set up a CI for TensorFlow on FreeBSD and give early notice to upstream when they break TensorFlow on FreeBSD. __________________________________________________________________ Updating Port Metadata for non-x86 Architectures Links aarch64 Poudriere Machine URL: http://thunderx1.nyi.FreeBSD.org/jail.html?mastername=3D110arm64-d= efault armv6 Poudriere Machine URL: http://beefy8.nyi.FreeBSD.org/jail.html?mastername=3Dhead-armv6-de= fault Contact: Mark Linimon I have been analyzing the error logs from ports builds for all non-x86 architectures, including both the logs published on the package build cluster and also other builds of powerpc64 and sparc64. From this analysis, I have marked almost all the failing ports as either BROKEN or NOT_FOR/ONLY_FOR, as appropriate. The intent of this work is not to make life harder for anyone, but rather, in fact, the opposite. With these definitions in place, it is possible to scan the poudriere bulk build output (the "Ignored ports" portion, in particular) and see quickly what ports are failing to build and why. Previously, finding the exact reason why a build failed needed some research (portsmon only analyzes failure messages on amd64). Additionally, it is extremely difficult to work through several hundred logs that simply say "failed to compile", "failed to link", and so forth. This is part of an effort to identify where we need further work to bring sufficient Ports Collection support to, e.g., armv6 and aarch64 to bring them closer to true Tier-1 status. To further facilitate locating patterns in the Poudriere output, I have begun reworking some existing BROKEN/NOT_FOR/ONLY_FOR messages so that they will sort more easily. This includes sorting the order in which architectures appear in the lists. Many people have been doing great work on fixing the individual ports. I hope that my work makes their jobs somewhat easier. __________________________________________________________________ Xfce on FreeBSD Links FreeBSD Xfce Project URL: https://wiki.FreeBSD.org/Xfce Ports Development Repository URL: https://www.assembla.com/spaces/xfce4/subversion/source Contact: FreeBSD Xfce Team Contact: Olivier Duchateau Xfce is a free software desktop environment for Unix and Unix-like platforms such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use. During this quarter, we have kept these applications up-to-date: * audio/xfce4-pulseaudio-plugin (0.2.5, PR219357) * deskutils/xfce4-tumbler (0.1.32, PR219848) * deskutils/xfce4-xkb-plugin (0.8.0, PR220071) * sysutils/garcon (0.6.1, PR219928, and PR219334 for Mk/Uses/xfce.mk) * textproc/xfce4-dict-plugin (0.8.0, PR220266) * x11/xfce4-terminal (0.8.5.1, PR219312) * x11/xfce4-whiskermenu-plugin (1.7.2, PR219347) * x11-wm/xfce4-desktop (4.12.4, PR220290) We have created a new Subversion tag (4.13) in order to follow the unstable releases. The separate tag was necessary in order to support changes in the USES=3Dxfce infrastucture, and due to some incompatible changes to the xfconf API. Ports following the unstable release are: * deskutils/xfce4-tumbler (0.1.92.1) * multimedia/xfce4-parole (0.9.2) * sysutils/xfce4-settings (4.13.1) * x11/libexo (0.11.3) * x11/libxfce4menu (4.13.2) * x11/libxfce4util (4.13.1) * x11/xfce4-conf (4.13.2) * x11/xfce4-dashboard (0.7.2) * x11/xfce4-screenshooter (1.9.1) * x11/xfce4-whiskermenu-plugin (2.1.2) * x11-wm/xfce4-desktop (4.13.1) * x11-wm/xfce4-panel (4.13.0) * x11-wm/xfce4-session (4.13.0) * x11-wm/xfce4-wm (4.13.0) Open tasks: 1. Make the transition to Gtk3 smoother for end users. __________________________________________________________________ Documentation Absolute FreeBSD, 3rd Edition Links Status as of 30 June URL: https://blather.michaelwlucas.com/archives/2972 Second Edition URL: https://www.michaelwlucas.com/os/af2e Trivial Updates URL: https://twitter.com/search?q=3D%23af3e&src=3Dtypd Contact: Michael Lucas I'm working on a third edition of Absolute FreeBSD. This will be a nearly complete rewrite, thanks to the addition of little details like ZFS, GPT, dma, GELI, new boot procedures, disk labeling, pkg(8), blacklistd, jails, etc.. My current (delusional) plan is to have a first draft finished by the end of October 2017, so we can have print copies for BSDCan 2018. Open tasks: 1. Write the remaining 75% of the book. __________________________________________________________________ Doc Version Strings Improved by Their Absence Links FreeBSD Documentation Project Primer URL: https://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/ Get Version Information from Subversion Metadata URL: https://svnweb.freebsd.org/doc/head/share/mk/doc.docbook.mk?r1=3D5= 0233&r2=3D50232&pathrev=3D50233 Contact: Warren Block In retrospect, our $FreeBSD$ strings in source files are kind of weird, like a vestigial tail. The version control system stores all of that information in metadata. Yet here we are, not only allowing the version control system to alter our source files on every commit, but forcing it to do so. The reason for doing so is that the previous version control system did it. Really. Version control strings are a headache for translators using the new PO toolchain. It is an ever-changing string that offers nothing to the translation, yet can cause conflicts with earlier versions of itself. We also had complaints about how the Handbook was always months out of date. It was not, of course... but looking at just the version string in the main, rarely-changing book.xml file gave that impression. We fixed that problem last year, so the build system checks all the source files for the latest commit, but it seems easier to not have to fix the problem at all. Of course, that was really only one aspect of an ongoing problem. Our documentation build system was checking the version string in the source file, not the metadata. In 1973, metadata, like cars not composed chiefly of rust, had not yet been invented. I modified the build system to extract the information from the metadata (and noted, with some surprise, that this is a task at which Git is much better than Subversion). The next step was to remove the $FreeBSD$ strings from the source files and remove the FreeBSD=3D%H property that forces Subversion, against its better judgement, to substitute text in the actual contents of the file. The version information is not lost. It lives in the metadata, so retrieving it is as simple as svn info -- it does not need to be in the source at all. However, as with anything that touches code or processes which have not been touched in living memory, there was some debate over this. At that point, I offered to remove the version strings from the FreeBSD Documentation Project Primer book as a test. The change allowed the zh_TW translation team to turn off the FreeBSD=3D%H property on their translation and continue their work without fighting with the version strings. Rendered versions of the book still display the name of the last committer and the date and revision number of the last commit, but all of that information comes from metadata. As such, it is also more likely to be correct. Since the change, there have not been any complaints, at least not to me. In fairness, the removal of version strings from the FDP Primer alone is a small change in a tiny corner of the project. Looking at it another way, it might be that some things that seem to be necessary are more about the comfort of familiarity than actual utility. At present, this is strictly a change to the documentation build toolchain and a single documentation book. However, there do not appear to be any reason why it could not be extended to the rest of the documents. It might even serve as tiny test of whether the expansion of $FreeBSD$ tags is needed throughout the rest of the FreeBSD tree. __________________________________________________________________ New Xen Handbook Section Links Handbook Section About FreeBSD as a Xen Host URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/virtual= ization-host-xen.html Original Phabricator Review URL: https://reviews.freebsd.org/D10774 Contact: Benedict Reuschling FreeBSD supports the Xen hypervisor, with DomU (guest) support since FreeBSD 8.0 and Dom0 (host) available since FreeBSD 11.0. The FreeBSD Handbook was lacking instructions on how to run a Xen host and VMs. The steps were outlined in the FreeBSD wiki, but needed some extra bits of text from the upstream Xen wiki in order to form a complete guide. The new handbook section briefly explains what Xen is, how it differs from other hypervisors, and what features are currently available in FreeBSD. It then goes on to describe how to set up the Dom0, as well as detailing the guest VM support known as DomU. Reviewers Nikolai Lifanov, Roger Pau Monn=C3=A9, and Warren Block provid= ed valuable feedback on the initial version in Phabricator. Additional corrections were made by Bj=C3=B6rn Heidotting while translating the sec= tion into German. Open tasks: 1. More options for the Dom0 and DomU could be provided. 2. People should test these instructions on their hardware and provide feedback. This would also help us get better testing of the Xen port for FreeBSD. __________________________________________________________________ Miscellaneous BSD Meetups at Rennes (France) Links First Event URL: https://www.meetup.com/fr-FR/Meetup-BSD-Rennes/events/239248155/ Second Event URL: https://www.meetup.com/fr-FR/Meetup-BSD-Rennes/events/240202297/ Contact: Mathieu Kerjouan Two meetups dedicated to BSD systems were held in Rennes, France. The first one was hosted in the OVH office in Rennes and included presentations on multiple subjects: the non-technical history of FreeNAS (presented by olivier@), how OVH is using ZFS, an introduction to jails, and a use case for BGP/bird on FreeBSD. The second meetup, also hosted in the OVH office, presented these subjects: how to create a FreeBSD port (presented by jadawin@), how OVH is using Finite State Machines for managing their storage system, network high-availability with FreeBSD, and a jail tutorial by means of a demonstration running 200 OSPF (using net/bird) routers using jails and vnets on a small PC Engines APU2 system with only 4 CPU cores (1Ghz AMD) and 4GB RAM). This project was sponsored by OVH. __________________________________________________________________ Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. HardenedBSD Links HardenedBSD Homepage URL: https://hardenedbsd.org/ SafeStack URL: http://clang.llvm.org/docs/SafeStack.html HardenedBSD Tor Hidden Service URL: http://t3a73imee26zfb3d.onion/ Projects HardenedBSD Would Like Help With URL: https://github.com/HardenedBSD/hardenedBSD/issues?q=3Dis%3Aissue+i= s%3Aopen+label%3A%22help+wanted%22 Contact: Shawn Webb Contact: Oliver Pinter HardenedBSD is a derivative of FreeBSD that gives special attention to security-related enhancements and exploit-mitigation technologies. From an initial focus on Address Space Layout Randomization (ASLR), it has now branched out to explore additional exploit mitigation techniques. It has been a long while since HardenedBSD's last entry in a quarterly status report, back in 2015Q4. The intervening year saw HardenedBSD gain new developers Bernard Spil and Franco Fichtner, import LibreSSL and OpenNTPd into base as the default crypto library and NTP client, respectively, and introduce the hbsd-update binary update mechanism for the base system. The secadm application got a rewrite and Trusted Path Execution (TPE). PIE is now enabled for the base system for arm64 and amd64 as well as the bulk of the ports tree, and the ports tree also gained RELRO and BIND_NOW. Integriforce (similar to NetBSD's verified exec, veriexec) was introduced for the base system, as well as SafeStack, a technology for protection against stack-based buffer overflows that's developed by the Clang/LLVM community. SafeStack relies and builds on top of Address Space Layout Randomization (ASLR), and is strengthened by the presence of PaX NOEXEC. Certain high-profile ports also have SafeStack enabled. Extremely generous hardware donations from G2, Inc. have provided for dedicated package building and binary update servers, as well as development and test servers. In March of 2017, we added Control Flow Integrity (CFI) to the base system. CFI is an exploit mitigation technique that helps prevent attackers from modifying the behavior of a program and jumping to undefined or arbitrary memory locations. This type of technique is gaining adoption across the industry -- Microsoft has implemented a variant of CFI, which they term Control Flow Guard, or CFG, and the PaX team has spent the last few years perfecting their Reuse Attack Protector, RAP. Of these, RAP is the most complete and effective implementation, followed by Clang's CFI. RAP would be a great addition to HardenedBSD; however, it requires a GPLv3 toolchain and is patent-pending. CFI can be implemented either on a per-DSO basis, or across all DSOs in a process. Currently only the former is implemented, but we are working hard to enable cross-DSO CFI. As is the case for SafeStack, cross-DSO CFI requires both ASLR and PaX NOEXEC in order to be effective. If an attacker knows the memory layout of an application, the attacker might be able to craft a data-only attack, modifying the CFI control data. The behavior of several system control (sysctl) nodes has been tighened up, limiting write access and introducing additional safety checks for write accesses. Kernel module APIs received a similar treatment. HardenedBSD's PaX SEGVGUARD implementation received a few updates to make it more stable and performant. As of March 2017, HardenedBSD is now accessible through a Tor hidden service. The main website, binary updates, and package distribution are all available over the hidden service. We now maintain our own version of the drm-next branch for updated graphics support. Binary updates are also provided for this branch. HardenedBSD would like to thank all those who have generously donated time, money, or other resources to the project. This project was sponsored by SoldierX, and G2, Inc. Open tasks: 1. Port SafeStack to arm64. 2. Integrate Cross-DSO CFI. 3. Add documentation to the HardenedBSD Handbook. 4. Start porting grsecurity's RBAC. __________________________________________________________________ --HeFlAV5LIbMFYYuh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQG3BAABCgAdFiEE2WGV4E2ARf9BYP0XKNmm82TrdRIFAlnMLcwACgkQKNmm82Tr dRKXUAwgzueor3CCu0VPI+jG5v5NVEewwGB3Q3GMKaykJbRbVQusrHzR1mc3CTLq fIE4UWL/bvnV7IPSbfmX4qhVHU5f/zl100wbke88RT1Og4yzWjgQV/kYVmhrnXPV wFxYuuSoW6XckjHvVEXgSyGKgBSwjy/8/1KCM2tUTaxP1zR+11dFure5BsXKLnvo IVONjg4zQS2qdoB4f+/D6rG4pP+tK/VoMQx8NNPQXuO8L+PzEbs23F8l6WG67I6o eLdVX5q5vgEVZIaCRn0tlipNHTvyJuFaw5dlSU970LOwoZ3ZaUHzHtVehbvB5Jck lgG6gZxbI1ROwy9niyhsWixJ7oAyWt0koo4LxxyT7K3nyzv/r5stHiKeijr5JsRR dDFP95br/4rJ+d41TuU+FAiCeKRI9rywusT4qGJdT5nG8Y2ZbFxHLaBj8+SHx2+j C1zMdnCwv7pPGzmYyeIw1m9CoTf8WdDbdWx/07xXFJ6Pp7Tkn1B+kU4PHNmiUYv3 UzLmeRO29rfdZA== =n+Q+ -----END PGP SIGNATURE----- --HeFlAV5LIbMFYYuh-- From owner-freebsd-current@freebsd.org Thu Sep 28 05:57:08 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5311FE2994A for ; Thu, 28 Sep 2017 05:57:08 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BADB570DFD; Thu, 28 Sep 2017 05:57:06 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LcSWg-1dWOFs1vqo-00jrXV; Thu, 28 Sep 2017 07:57:02 +0200 Date: Thu, 28 Sep 2017 07:57:01 +0200 From: "O. Hartmann" To: Hans Petter Selasky Cc: Guido Falsi , "O. Hartmann" , freebsd-current Subject: Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170928075701.23ca8b10@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <31faf367-69e2-b7e3-dd14-67bf69a67ec2@selasky.org> References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <31faf367-69e2-b7e3-dd14-67bf69a67ec2@selasky.org> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:FwtbhkqV/tNFeuiLvTMpNOzcYgYfcIjz5gg+GI7Juzd7CSBqYgl ZGNcSrXZEUuxZNiTHTWaFS7Wgi16sZf3HI/A2zKYzUYvQAq9r3hXbUfXVG1g7UHpO0SEepH GCACmJYATNef2bwhg0uRhY/6/YVtiVfNsyM3JPrwnEeODocfb4ivhUHCxZj2vQX9A+wDo4w +2NmJC1bjvH/2iRD5s0OA== X-UI-Out-Filterresults: notjunk:1;V01:K0:5C4uEzWwIcU=:02UaAp4A3R6b14c3gq1zWS waBZSeL9DQUVO/qxdLbXFjI+crRmr1UuUtb3UwDBeoT+tveO3DzOs6Y83rkrGuPym7japRqJw IXlkJGiso8jj08VELL+pkSSt14sd6OFqoXO4wWv39CQ/z8raV4FoJOzdPifT6CFI4wAECDkGx xliueOfGYV89K7EgfM3YPPIm5f39ZK5D/3pkTNYnTeXD7JGWMKNKi06ovQZl+C/5+WPxRzQF3 QE2txC4OnW7NMElkz1SGYaOJ9mZTgYbIwss0UARGABbeUVP/fQN+hkZIjynKvahoePVakvawH AfcP5W3xroiQTzOvNM8G8KSdxZKgeogmUnKMkS5Yl0JT39+gZ/5QFX+S3i1LLqOYkdMiYn/O1 wvaDlpQZ5+w5DFeQgVJkBfPVW99fM+/ri7Ia607+/w5ey2Gn0nMyxiIJI3uRAjHsYaFhyj3Ii 4ovqka4MAvV/gw9WLnjTXTcLuJZPPKqtjHhxEIovAaaeDvtzjwpgbZysDp+FMow5atux23mSi OtFcMntwI7fEL+lw5F0uZT7RLoZDbBU+9bAn4hQPQZOFLG2jRObhX9TZEY//4Lw/oKygUfNP9 V7QJbiRK0ycCGagb40yDGLOrfCSYCPTYL3pLnnGY140JybANqSvUsGkOgOvZoHpWmj5pIi/sK yK0/bjzNHCf3dYlhTZtB1RaGVqmvF9+nmr17xCXlmC3QFqVBUsDbs9ESb1QUW76+Bvm7Cma6c /N7XCO6JvNLfp2ydtnWq3rgBXckc5WsOEk9ntiWkNMfc4BcdRCm/UBQzZBISn7PyDChz7/fQc /7D8zogUBYOFglqEITZzkji4JWwWCS519aTgzEmVFhz7UI9qYM= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 05:57:08 -0000 On Wed, 27 Sep 2017 12:51:18 +0200 Hans Petter Selasky wrote: > On 09/27/17 09:05, Guido Falsi wrote: > > On 09/26/2017 15:41, O. Hartmann wrote: > >> On Tue, 26 Sep 2017 15:06:23 +0200 > >> Guido Falsi wrote: > > > >> Since I run net/asterisk with automatic module loading (I'm new to > >> asterisk), this is very likely and might cause the problem somehow. > >> > > > > You can exclude single modules from autoloading via modules.conf. > > > >>> Not sure, restarting the daemon should free any leaked memory the daemon > >>> has. If a killed process leaves memory locked at the system level there > >>> should be some other cause. > >> > >> Even with no runnidng asterisk, memory level drops after the last shutdown > >> of asterisk and keeps that low. Even for weeks! My router never shows that > >> high memory consumption, even under load. > > > > But while asterisk is running does the memory usage increase unbounded > > till filling all available memory or does it stabilize at some point? > > > > Asterisk is relatively memory hungry, especially with all modules > > enabled. It also caches and logs various information in RAM, even doing > > "nothing" it will cache and log that "nothing" activity. If memory does > > stabilize after some point it's not really a leak but it's standard > > memory usage. To reduce it you should disable all unused modules. > > > >> > >> The question would be: how to use vmstat to give hints for those familiar > >> with memory subsystems to indicate a real bug? > >> > >> I tried to find some advices, but maybe my English isn't good enough to > >> make google help. > > > > I'm not able to give you a correct indication, but if the memory usage > > is not increasing indefinitely but is stabilizing I'd say it's not > > really a leak. > > > > Did you look at the output from "vmstat -m" and "vmstat -z" ? > > --HPS I did not, but now I will ;-) Thanks for the hint! Kind regards, Oliver From owner-freebsd-current@freebsd.org Thu Sep 28 06:01:40 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A679E29C68 for ; Thu, 28 Sep 2017 06:01:40 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D6BC71084; Thu, 28 Sep 2017 06:01:39 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MJSuF-1dw5Nb3xDH-0039f7; Thu, 28 Sep 2017 08:01:37 +0200 Date: Thu, 28 Sep 2017 08:01:36 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current Subject: Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170928080136.6a19fd7d@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <40903499-6cbb-53b7-e25a-1d7d96c47a75@FreeBSD.org> References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170927112706.436500ec@freyja.zeit4.iv.bundesimmobilien.de> <40903499-6cbb-53b7-e25a-1d7d96c47a75@FreeBSD.org> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:p3Mejpjnk049Im7TmbAF7vJreDv1PEAPa3ajBKM8h51oMWGV3EX YRbjHQgbzaF73OL/1iBMiafGY7zZ2hi79WOUhh6etdDeqNdmrYbmIksb8OxGFfxWhAfwia+ xInCrO0V+b+ajB5fC+hN2Evy6tDU+7wmatF86basLeOsbV85sGRdUCy6/oMVqSY8zSxf85g XeNlvvfwg5Mcm+AgZlonA== X-UI-Out-Filterresults: notjunk:1;V01:K0:3fXE7jnwv5o=:qlb2GE4fQfBORAKZeHL//k VmdwIx4EgLmLzbs2E8cc1axchdw9FEwZcI22T+blKbIkG8nVfRWFvdIaY0xNAN5X1SBx+RwY0 FaumEwwoh0bgFYlLUzDVx+Oq5JnJd1jvafrwoACo5GvVPxndL3WL+CKSgvuvTqS4d5Hu2OGDs cEamNi+PqVcMDFSQkOBYFgLycAJLv9Vf4GqxkZ5KcpvYO5hWZmsYIApBJZDLHedvJI0TXPXm+ F0o1nZPtREOcPoBm5xu6TDYm5KNxeXKMwuU5fAJSUnNxQw1G4T0opfkyKPhDCZlFlurJbotl4 bxPFONNkKfR6rFH/gdTA9ZVJbyMJ13OmU3HaOV2l0Y9D9gIZJ0gu0gfvC5w4xx3qh1GNNIEzb ymknejRD11qMJRiQ3I8wNj4JMCy9NoWPARBEwIHiUnlxpGDpvqOiXC6EU5RyJdWXkmOJjvWCO XJ7eE4DByWCDfGegvl29Lz903iBZU08tGkM/ttT9DRCx6BLa40pKckXncDv+ksgUoJ8bI0Sbz z1TbLCTYEmkdrcJGZb4HEkOlYNMc67UtphAGx1Icw3diyl8N5Er9Jghg0ywgVYTSzxbG47Nxs StvaHrj/0GzlbTSB5duce7KWaZc+1gfQa55ZgOjM4AEa1tgvgVuGx3spxE5bTN+MLFcZ35eIy 3lpyjnmpPyOZ6EKOd3lbTE+kRagyZkzLCMNTLzFwqNXHc9BCiHRxOnHmlB+twF4ArP49or+GL twj6DqI2n5uFoEX7tp8gwkyVyQYZTLqwy0npw6aIwnH1oug5wR61/UyFHQKmNGtKZkUxQqDly Vge7haiAfe9U6tS4JLNb2N7YBkzQgHuXFR3zILitkd2ehGpD4o= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 06:01:40 -0000 On Wed, 27 Sep 2017 12:23:20 +0200 Guido Falsi wrote: > On 09/27/2017 11:27, O. Hartmann wrote: > > On Wed, 27 Sep 2017 09:05:42 +0200 > > Guido Falsi wrote: > >> But while asterisk is running does the memory usage increase unbounded > >> till filling all available memory or does it stabilize at some point? > > > > As far as I could observe, a three day test run of the > > router/firewall/asterisk box drained around 500 MB of memory: starting at > > boot time with ~3700 MB, asterisk leaves the box with ~3640 MB after bein > > started and after three days the system reached ~3150 MB. Stopping asterisk > > gave back some memory, so ~3300 MB then was for days the final result - not > > recovering anything further. I use TEMPFS, if it matters, but I > > checked /tmp and /var/, there were no remnant files so far. TMPVAR is only > > allowed to have 256 MB. > > These numbers really don't tell us anything. The system has anyway been > running for days, depending on configuration daemons like cron and ntp > are running and performing tasks, things are being cached and so on, so > that difference after three days could be perfectly normal overhead. I think they do, but not in a scientific way. The system in question has to do always the same task and is running for months with never dropping below a certain memory boundary. Then, when asterisk started/stopped/started, memory begins to fade away and is never getting back to "free", even with asterisk off for days, twoo weeks. Does this really tell me nothing? > > You need to investigate the amount of memory allocated to asterisk with > ps and top and check if that stabilizes. A few days, at most a week > would be enough. After that, if it's not stabilizing you can start > thinking on a leak, but still can't assume where the leak is happening. > > > > Can't say whether it is stabilising or not - I think the runtime is too > > short. I'll check first to disable some modules in the first place and then > > try to perform a test with several days of asterisk enabled. > > > > Whatever you prefer, but trying a few days uptime with all modules > enabled is zero cost and east to do. Also you started this report with > THIS configuration, changing configuration would prevent from comparing > results. Let's test one thing at a time. > I will. Now as we have a indication that there is porbably something, I'll go for deeper investiagtions with a static config. From owner-freebsd-current@freebsd.org Thu Sep 28 06:11:57 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A815E29E60 for ; Thu, 28 Sep 2017 06:11:57 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D630716D8; Thu, 28 Sep 2017 06:11:55 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LkCU2-1dMTKt3YXS-00cB2f; Thu, 28 Sep 2017 08:11:54 +0200 Date: Thu, 28 Sep 2017 08:11:52 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current Subject: Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:8nzT1+AdMsb7NZki2q7nOqdXW2LvjdYEVAt25yoEsoDBqkZNiHN 91i36wBYdZxYeFWduGFSSlHqEx34PgRRKrOeLY56Z2ynuIbtr8dP70EiCeaK5IcG2iSGBgO Yu8gynfsboCkIuurHfyBMaKK+gcWfmmOXJmnRiOiUBR+rbNQGD+vS4ZLs6RJnfJbADF+0VK 9wQnZhl+06g5EmBaZE9uw== X-UI-Out-Filterresults: notjunk:1;V01:K0:vW5w82ndmAo=:o1nGaX03Yd6MX+yo42j23t 2SmUei1yyYE1wh1PCJw4RLBBlJiON6KkZA4L3/p15GKoC94dl0bpwFSaX62+2i9NfERMYqgHV m6Ad9GpbxoIC2n7YtZt9bumbJNYs3S9T5KBE9Mhn8PC0yyeQl6yDf+hZ2ujnGXh4MBsthgNN8 JEeMCGYXKDXk521lV8aFwJwTzMd1m9ZULcc84LH+k7AoKGTj0zPcz7/FmnIKtxLml0KZQsrlp Danhq6KBKnEEYap0b5MNdQEfxY9UUBiSJUS5/o/tbCJJeM+lDUC9lvsC8htAlT81Fro5fOFVf 4hNkhrXUjvdwgmqSYMsai7SFHZETWDxJjSzbOoJge1jTKuiLRxaOPkfpkm1JUdV1fQZiC5xBq dNT5RAkshDOkyGRsUDXzxtKvVfhuMC62A/pqAbUuW9B43Ns6VheNXqIWleNUea+E+WawDx1jW hFiNlmzIIkLWqL1bCTBjJNAxlxcmGkhCTzZin/un0L/GDnsDEfjHCrAZxXqG9UCEIEMDrSaaj GWv5QC+uebnRbzUyuMTh/XrYMgcl3gpARQfL6QfNkosioDS9h3BxKMw7OEFFTzLMDDlAB60qI ipwIkvc8Y1bFUXFoncIPep9T8xsPxeNu+kDEIld51+aoPRVjHQJYfy+T6BeRGylbceyQwehbl rtva4hWnJpAqkTGfKjhwV5KsWt5YmiQe47mKVm3olCVdD+aAIFwX6HLptgN0ZNLysHIgbG4tY 5APhGA+0szutvW7lTa1RV1Om3Fka+3upF+dzJ71C1tSDpAAisnG8JVW0V/nZNRqGsEmsiBAuV 5/GX+8nff+sk5PiwimjXlL9u6neDYqS6fYTKY0CD8yYI4KXtJ4= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 06:11:57 -0000 On Wed, 27 Sep 2017 09:05:42 +0200 Guido Falsi wrote: > On 09/26/2017 15:41, O. Hartmann wrote: > > On Tue, 26 Sep 2017 15:06:23 +0200 > > Guido Falsi wrote: > > > Since I run net/asterisk with automatic module loading (I'm new to > > asterisk), this is very likely and might cause the problem somehow. > > > > You can exclude single modules from autoloading via modules.conf. > > >> Not sure, restarting the daemon should free any leaked memory the daemon > >> has. If a killed process leaves memory locked at the system level there > >> should be some other cause. > > > > Even with no runnidng asterisk, memory level drops after the last shutdown > > of asterisk and keeps that low. Even for weeks! My router never shows that > > high memory consumption, even under load. > > But while asterisk is running does the memory usage increase unbounded > till filling all available memory or does it stabilize at some point? While Asterisk is running, it doesn't consume much memory, but stopping Asterisk, I would expect that the claimed memory is freed again. It isn't, not all memory is freed. Starting Asterisk then again from this reduced memory level, it claims its memory, "stabilzes" at a certain point while running and again, stopping Asterisk leaves the free memory now at a much lower level never been leveld out - as I said. I played this game last night ~ 20 times until the free memory dropped beneath 3 GB after asterisk has been shut down. This morning, the level was at the same low level as I left it. The router has nothing special to do, the workload is not memory consuming even for weeks! And if there is load, after the load went away, the memory consumption always leveld out and freed memory. > > Asterisk is relatively memory hungry, especially with all modules > enabled. It also caches and logs various information in RAM, even doing > "nothing" it will cache and log that "nothing" activity. If memory does > stabilize after some point it's not really a leak but it's standard > memory usage. To reduce it you should disable all unused modules. I don't understand here. Even if Asterisk is memory hungry - it has ~ 3 GB to use. But after stopping it, it should free the memory. BUT the system is then after the stop with less memory! that is the point. Not the running asterisk's memory consumption bothers me, but the fact, that after 20 start and stops and waiting for days the memory once gone is never put back. At the moment, I have mpg123 suspect doing nasty things, because the vanishing memory is more prominent and indicated when voicemail system has been used and mpg123 started. Not touching VM subsystem seems to free the whole memory claimed by asterisk after stopping asterisk, apart from maybe buffers claimed by the OS released later (I did no thourough investigations on that). > > > > > The question would be: how to use vmstat to give hints for those familiar > > with memory subsystems to indicate a real bug? > > > > I tried to find some advices, but maybe my English isn't good enough to make > > google help. > > I'm not able to give you a correct indication, but if the memory usage > is not increasing indefinitely but is stabilizing I'd say it's not > really a leak. > From owner-freebsd-current@freebsd.org Thu Sep 28 07:29:34 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37A7CE2B2DE for ; Thu, 28 Sep 2017 07:29:34 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE45173442 for ; Thu, 28 Sep 2017 07:29:32 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y2mXp39MszZrX; Thu, 28 Sep 2017 09:29:30 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id JMYGfTdVJLnK; Thu, 28 Sep 2017 09:29:27 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Thu, 28 Sep 2017 09:29:27 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" Cc: freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170927112706.436500ec@freyja.zeit4.iv.bundesimmobilien.de> <40903499-6cbb-53b7-e25a-1d7d96c47a75@FreeBSD.org> <20170928080136.6a19fd7d@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: <6d582286-d5ce-11ac-a8d4-391d79691d2c@FreeBSD.org> Date: Thu, 28 Sep 2017 09:29:26 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170928080136.6a19fd7d@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 07:29:34 -0000 On 09/28/2017 08:01, O. Hartmann wrote: >> These numbers really don't tell us anything. The system has anyway been >> running for days, depending on configuration daemons like cron and ntp >> are running and performing tasks, things are being cached and so on, so >> that difference after three days could be perfectly normal overhead. > > I think they do, but not in a scientific way. I am unable to work with such data anyway. > > The system in question has to do always the same task and is running for months > with never dropping below a certain memory boundary. > > Then, when asterisk started/stopped/started, memory begins to fade away and is > never getting back to "free", even with asterisk off for days, twoo weeks. Does > this really tell me nothing? > It tells you the OS is caching some data. Which is normal. You should discover what the OS is doing with that memory. The fact that it is in some way allocated does not indicate anything. Maybe the memory used by asterisk is simply going in the inactive pool and never reclaimed by the OS because it's not required, having still lots of Free memory. There are many possibilities, and exact data is needed to investigate. I'm also not an expert about VM systems. Asterisk, even with few modules tends to use a big chunk of memory and after stopping a software which uses much RAM it's normal for the OS to not immediately reclaim that to the Free pool. In fact such reclaiming happens only if there actually is memory pressure. There are so many factors playing into this that just looking at Free RAM answers no question. What problem are you trying to solve? Are you running out of memory? >> Whatever you prefer, but trying a few days uptime with all modules >> enabled is zero cost and east to do. Also you started this report with >> THIS configuration, changing configuration would prevent from comparing >> results. Let's test one thing at a time. >> > > I will. > Now as we have a indication that there is porbably something, I'll go for > deeper investiagtions with a static config. > I disagree with the indication buyt to investigate such a problem you need to use better tools than just looking at the free ram reported in top. Please at least give us the other ram numbers (Used, inactive, laundry, wired, buffers...) -- Guido Falsi From owner-freebsd-current@freebsd.org Thu Sep 28 07:39:03 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74A7BE2B6E6 for ; Thu, 28 Sep 2017 07:39:03 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 04F3D73A45 for ; Thu, 28 Sep 2017 07:39:02 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y2mlm454XzZrX; Thu, 28 Sep 2017 09:39:00 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id RzVoQkxHNwrj; Thu, 28 Sep 2017 09:38:58 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Thu, 28 Sep 2017 09:38:58 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" Cc: freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: Date: Thu, 28 Sep 2017 09:38:58 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 07:39:03 -0000 On 09/28/2017 08:11, O. Hartmann wrote: > On Wed, 27 Sep 2017 09:05:42 +0200 > Guido Falsi wrote: > >> On 09/26/2017 15:41, O. Hartmann wrote: >>> On Tue, 26 Sep 2017 15:06:23 +0200 >>> Guido Falsi wrote: >> >>> Since I run net/asterisk with automatic module loading (I'm new to >>> asterisk), this is very likely and might cause the problem somehow. >>> >> >> You can exclude single modules from autoloading via modules.conf. >> >>>> Not sure, restarting the daemon should free any leaked memory the daemon >>>> has. If a killed process leaves memory locked at the system level there >>>> should be some other cause. >>> >>> Even with no runnidng asterisk, memory level drops after the last shutdown >>> of asterisk and keeps that low. Even for weeks! My router never shows that >>> high memory consumption, even under load. >> >> But while asterisk is running does the memory usage increase unbounded >> till filling all available memory or does it stabilize at some point? > > While Asterisk is running, it doesn't consume much memory, but stopping > Asterisk, I would expect that the claimed memory is freed again. It isn't, not > all memory is freed. Starting Asterisk then again from this reduced memory > level, it claims its memory, "stabilzes" at a certain point while running and > again, stopping Asterisk leaves the free memory now at a much lower level never > been leveld out - as I said. > > I played this game last night ~ 20 times until the free memory dropped beneath > 3 GB after asterisk has been shut down. This morning, the level was at the same > low level as I left it. The router has nothing special to do, the workload is > not memory consuming even for weeks! And if there is load, after the load went > away, the memory consumption always leveld out and freed memory. >> >> Asterisk is relatively memory hungry, especially with all modules >> enabled. It also caches and logs various information in RAM, even doing >> "nothing" it will cache and log that "nothing" activity. If memory does >> stabilize after some point it's not really a leak but it's standard >> memory usage. To reduce it you should disable all unused modules. > > I don't understand here. Even if Asterisk is memory hungry - it has ~ 3 GB to > use. But after stopping it, it should free the memory. BUT the system is then > after the stop with less memory! that is the point. Not the running asterisk's > memory consumption bothers me, but the fact, that after 20 start and stops and > waiting for days the memory once gone is never put back. VM system is not composed only of "free" and "used" ram, there ar3e other categories. Depending on how a program allocates and uses memory it's not automatically sent to the "free" pool after being reclaimed. Allocated memory can be dirty buffers which are reclaimed after time, or other types of buffered data which is never reclaimed until there is memor7y pressure. How do you know the game you were playing has a similar memory usage as asterisk, which, I assure you, has some complex memory usage patterns in it's source. Also asterisk leverages many parts of the kernel which a game does not leverage. I'm quite sure after quitting asterisk you have an higher "wired" memory than before starting it. That memory was not allocated by asterisk itself, but by the kernel while "serving" asterisk requests. The kernel keeps running and does not free that memory right away. in fact will not free it until forced by VM pressure AFAIK. But this is quite a complicated matter and not being a VM expert I can't explain it in detail. What you must understand is that many things are going on at a given time in a memory subsystem and looking at it through a single value is not indicative. > > At the moment, I have mpg123 suspect doing nasty things, because the > vanishing memory is more prominent and indicated when voicemail system has > been used and mpg123 started. Not touching VM subsystem seems to free the > whole memory claimed by asterisk after stopping asterisk, apart from maybe > buffers claimed by the OS released later (I did no thourough investigations on > that). Please note that when I use the "VM" Acronym I mean "Virtual Memory". there is a new version of mpg123 also, and I'm quite sure that mpg123 causes caching of the audio data it reads. Adsterisk Voicemail system also causes caching. Those cached are not going to be freed after program exit because other programs could take advantage of the cached data. I'm not understanding what you are expecting us to do based on circumstantial and partial data. -- Guido Falsi From owner-freebsd-current@freebsd.org Thu Sep 28 07:57:57 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB048E2BD6C for ; Thu, 28 Sep 2017 07:57:57 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7DBEE743C1 for ; Thu, 28 Sep 2017 07:57:56 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y2n9Z55XVzZrX; Thu, 28 Sep 2017 09:57:54 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id yB726oSbISVN; Thu, 28 Sep 2017 09:57:48 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Thu, 28 Sep 2017 09:57:48 +0200 (CEST) Subject: Re: net/asterisk13: memory leak under 12-CURRENT? From: Guido Falsi To: "O. Hartmann" Cc: freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> Message-ID: <17b9c8e6-caf8-5334-ce12-1ee3ac9969e1@FreeBSD.org> Date: Thu, 28 Sep 2017 09:57:47 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 07:57:57 -0000 On 09/28/2017 09:38, Guido Falsi wrote: > I'm not understanding what you are expecting us to do based on > circumstantial and partial data. > I'm clarifying myself here: I mean that I don't know what to do about this based on the data and the details you give. What you are seeing looks normal to me. To understand better what is happening is more data. You say you have data that shows something but in a non scientific way. But this translates to me to you having a feeling, which I don't share based on that data. So if you want help or a better explanation your only way is to give more and more detailed data. At a minimum the while memory line from top not only the "free memory:" column, which by itself gives completely incomplete information. It would be better if you could give the time evolution of that line. before launching asterisk, some samples while it is running and you are doing some tests, and after closing it. Other tools like vmstat can give more detailed information, but I'm not very good at deciphering that data, but it could be very useful anyway for people more expert about the VM system to give you an explanation or be convinced that there is actually something strange going on. In other words I'm sorry to say you need to give more scientific proof of your feeling. There's nothing personal in this. -- Guido Falsi From owner-freebsd-current@freebsd.org Thu Sep 28 10:20:44 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D96FCE2E5E1 for ; Thu, 28 Sep 2017 10:20:44 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63DDE7C7C9 for ; Thu, 28 Sep 2017 10:20:44 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm0-x244.google.com with SMTP id q124so1267223wmb.0 for ; Thu, 28 Sep 2017 03:20:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=artn2XrVr4lMmYeJax5/yl64zToXBg4weXtTHTlmOjE=; b=F3TgJ932T2yHqEv1n+0cf0Dhc9pFJxyrskW/F599pE9Xv5p2V98zJe9idC6+U3scS1 5Xm2fmmkEyIjSzR18LA4+QNdAalukRFKW2On/nzWINtu+xobkAKjFM7EmdEWR9tAt4J7 8+MRGU/6a71UIIALf0F92mMdEWkEQNUYMMOK/jTnGsGVfzA3FfQFjd1NHo++0EVdn3ce bM4wyNXv72HJx6evArr4qqabbtIPqi5wgyYOB7cdLl4DNhb6iFr+wBBlNjKcQgkc6ZpX pG6t7CA5MHIY4dH2yGO8YvTRQJtisZrG6xBbLe+qlpY1+aU0rlZfcWlFDvifu7bq0Ni3 R9GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=artn2XrVr4lMmYeJax5/yl64zToXBg4weXtTHTlmOjE=; b=XrpOxat+o1ZfEuwmCcZMlb+5d3i2pAJHxOHBSqM6xNK6zHrJzO/V8hwnwtaOIFO9mT aueN/I5F1Lp4MMGYSrEbmWeBSCCqkuQztCxb7tWTAy/YPSJIcBr9L4Gubr0trvB6fxm2 Q/wTZAofWDRA+6m8haCc1p8Scp4dPrkzKH9EXfJKu79/hlEWrmCE4c3dbHNGdI9DogT7 m6J3Pir/80Zr9J+qBKihEuZHKLrasSFgiZYqhpxF8ZOF52ewt1hB2XLSLPhd9kwHIESS xbkj/ayerToQ28SJOGEyWSN+gKQYVwtY7FClGwcSC9mrOmzBIWjbuBi9Wu5c0veQhcFL 5WPw== X-Gm-Message-State: AHPjjUhapJ4gfSN1DsiGBuFP6SwqvJ+yw38MJc0LF51rbwzx03ofhjxY wmO7ghuWcxHQIrOsbiwd/2WyNg== X-Google-Smtp-Source: AOwi7QDovx8bCOrIdPqBjD3D4rdqH5WZ4ebSVa+k+SVYhGUHZUFmGmLVMLF3ZLjbklFixOAvuwFT2g== X-Received: by 10.28.0.15 with SMTP id 15mr590700wma.7.1506594042882; Thu, 28 Sep 2017 03:20:42 -0700 (PDT) Received: from ernst.home (p578E321D.dip0.t-ipconnect.de. [87.142.50.29]) by smtp.gmail.com with ESMTPSA id l4sm794847wrb.74.2017.09.28.03.20.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 28 Sep 2017 03:20:42 -0700 (PDT) Date: Thu, 28 Sep 2017 12:20:18 +0200 From: Gary Jennejohn To: Steve Kargl Cc: freebsd-current@freebsd.org Subject: Re: panic: softdep_deallocate_dependencies: dangling deps Message-ID: <20170928122018.3aa3540a@ernst.home> In-Reply-To: <20170927221959.GA1198@troutmask.apl.washington.edu> References: <20170927221321.GA1023@troutmask.apl.washington.edu> <20170927221959.GA1198@troutmask.apl.washington.edu> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.15.1 (GTK+ 2.24.31; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 10:20:45 -0000 On Wed, 27 Sep 2017 15:19:59 -0700 Steve Kargl wrote: > On Wed, Sep 27, 2017 at 03:13:21PM -0700, Steve Kargl wrote: > > > > Hmmm, > > > > % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0 > > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done. > > ABI doesn't support a vmcore target > > > > OK, so debugging is broken :-/ > > > > It seems the devel/gdb80 port installs a kgdb symlink > to kgdb80. So, I'm not getting the correct kgdb due to > my path. Ugh. > Yeah. I think the idea is to delete the old base-system kgdb when you install the port. -- Gary Jennejohn From owner-freebsd-current@freebsd.org Thu Sep 28 14:34:40 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C86CE3335C; Thu, 28 Sep 2017 14:34:40 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-8.mit.edu (dmz-mailsec-scanner-8.mit.edu [18.7.68.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA5D91719; Thu, 28 Sep 2017 14:34:39 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074425-6efff70000001f28-3c-59cd074a895a Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-8.mit.edu (Symantec Messaging Gateway) with SMTP id DB.2B.07976.A470DC95; Thu, 28 Sep 2017 10:29:30 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH-1.MIT.EDU [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id v8SETTil012333; Thu, 28 Sep 2017 10:29:30 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v8SETQuY012052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 28 Sep 2017 10:29:28 -0400 Date: Thu, 28 Sep 2017 09:29:26 -0500 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: Call for 2017Q3 quarterly status reports Message-ID: <20170928142925.GG96685@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.8.3 (2017-05-23) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNIsWRmVeSWpSXmKPExsUixG6nruvFfjbS4PEdSYs5bz4wWWzf/I/R gcljxqf5LAGMUVw2Kak5mWWpRfp2CVwZ737pFXTxVOybv4W9gbGJq4uRk0NCwERi5qwfrCC2 kMBiJonmZiCbC8jeyChxe1cblHOVSWLz0c0sIFUsAqoS/w+/YQex2QRUJBq6LzOD2CIC8hL7 mt6DxZmB7F9bm4BsDg5hAUOJfQcyQMK8QMuuXPvEDGELSpyc+YQFolxL4sa/l0wg5cwC0hLL /3GAhEUFlCXm7VvFNoGRbxaSjllIOmYhdCxgZF7FKJuSW6Wbm5iZU5yarFucnJiXl1qka6GX m1mil5pSuokRHGYuqjsY5/z1OsQowMGoxMN7YcHpSCHWxLLiytxDjJIcTEqivM9YzkYK8SXl p1RmJBZnxBeV5qQWH2KU4GBWEuFdxwaU401JrKxKLcqHSUlzsCiJ824L2hUpJJCeWJKanZpa kFoEk5Xh4FCS4A0HaRQsSk1PrUjLzClBSDNxcIIM5wEa7gA2vLggMbc4Mx0if4pRUUqcdwJI QgAkkVGaB9cLSgMS2ftrXjGKA70izLsUpIoHmELgul8BDWYCGjx54hmQwSWJCCmpBkatY3/P 9m5Rqqr+wRvSrPFDuNsgmX2r4tdZjoU2j2PPJK+T+bmg/WXvmx3Nh7I8JBk3PDXJVr8j3n5p 7fVnNW90jn1y2uTeIq+p4LFWht31VIG9iMzTUovEBsfikJjgTt8f1YX9TDJLJuc33tss8OT5 Mc1t+zRYLaQnnpg9/53Np2M169gSbJRYijMSDbWYi4oTAcrqhj7eAgAA X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 14:34:40 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is October 21, 2017, for work done in July through September. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about work that is underway and completed. Submission of reports is not restricted to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred and easiest submission method is to use the XML generator [1] with the results emailed to the status report team at monthly@FreeBSD.org . (Do be sure, though, to save the form output and not the form itself! In particular, the Google Chrome "save as" function does not save the generated output for some reason.) There is also an XML template [2] that can be filled out manually and attached if preferred. For the expected content and style, please study our guidelines on how to write a good status report [3]. You can also review previous issues [4][5] for ideas on the style and format. We look forward to seeing your 2017Q3 reports! Thanks, Ben (on behalf of monthly@) [1] https://www.FreeBSD.org/cgi/monthly.cgi [2] https://www.FreeBSD.org/news/status/report-sample.xml [3] https://www.FreeBSD.org/news/status/howto.html [4] https://www.FreeBSD.org/news/status/report-2017-01-2017-03.html [5] https://www.FreeBSD.org/news/status/report-2017-04-2017-06.html From owner-freebsd-current@freebsd.org Thu Sep 28 15:31:52 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD518E014B5 for ; Thu, 28 Sep 2017 15:31:52 +0000 (UTC) (envelope-from nick@van-laarhoven.org) Received: from valery.hibma.org (valery.hibma.org [178.21.114.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6BC123903 for ; Thu, 28 Sep 2017 15:31:48 +0000 (UTC) (envelope-from nick@van-laarhoven.org) Received: from [IPv6:2001:980:530a:1:bd28:5ad2:4637:56bc] (unknown [IPv6:2001:980:530a:1:bd28:5ad2:4637:56bc]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by valery.hibma.org (Postfix) with ESMTPSA id 17CB56E0A17 for ; Thu, 28 Sep 2017 17:31:41 +0200 (CEST) From: Nick Hibma Content-Type: multipart/signed; boundary="Apple-Mail=_D1611BBC-3753-4484-8DDB-6D053D1F8C04"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: How do GEOM_PART_* options configure geom_part_* modules?? Message-Id: Date: Thu, 28 Sep 2017 17:31:40 +0200 To: FreeBSD Current Mailing List X-Mailer: Apple Mail (2.3273) X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:31:52 -0000 --Apple-Mail=_D1611BBC-3753-4484-8DDB-6D053D1F8C04 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I created a new kernel config file from scratch, wondered what the = GEOM_PART_MBR option and friends were doing, search for them, didn't = find them in the tree, and deleted them from my config. But... de = resulting disk image didn't boot, because of the fact that it didn't = recognise the MBR partitions (it only had a single diskid entry on the = mount-root prompt). Can anyone explain to me how these kernel options work, as in: they are = defined in kernel configs and as a consequence in opt_geom.h, but how = are they actually used to select which geom_part_* modules/kernel parts = to build? I thought these options were translated to stuff that cpp = would use, but there are not uses of for example GEOM_PART_MBR anywhere = for example! The only thing I was able to come up with, but could not figure out, was = FEATURE() doing some magic. Thanks in advance for any pointers! Nick Hibma nick@van-laarhoven.org -- Open Source: We stand on the shoulders of giants. % grep -r GEOM_PART_ /usr/src/sys/ | grep -Ev '/conf/.*options' /usr/src/sys/geom/part/g_part_mbr.c: "GEOM_PART_MBR Master Boot = Record"); /usr/src/sys/geom/part/g_part_ldm.c: "GEOM_PART_LDM Logical Disk = Manager"); /usr/src/sys/geom/part/g_part_ldm.c: * XXX: We use some knowledge = about GEOM_PART_GPT internal /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT /usr/src/sys/geom/part/g_part.h:#ifndef _GEOM_PART_H_ /usr/src/sys/geom/part/g_part.h:#define _GEOM_PART_H_ /usr/src/sys/geom/part/g_part.h:#endif /* !_GEOM_PART_H_ */ --Apple-Mail=_D1611BBC-3753-4484-8DDB-6D053D1F8C04 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 Comment: GPGTools - https://gpgtools.org iQIzBAEBCgAdFiEETFbRZ/gKjBgO10CrH3Ic7sY0duAFAlnNFdwACgkQH3Ic7sY0 duD7ghAAlgLajw3bUgw4fyq9XVarB3nrKsdidExM/fTcHvK1digiAJ8p34lvzgaM ArMhmkZXWSnACCiXh48x5XUVz7PDepRwo+knp+jrgPe9fMg3BPhh/qLCVE/NtNSb MUI6XJ58NViG2MAkMl2UpJgUo9U3IYdHN21TdgeeYh2jta20D5174X7ADYePaLHy brViS44jCUn63ptylav8BoGzC9cKf6RqptaOBv308WpPHltbYtQtl3RlWsclZWOF 6Gy1aX24A0BtZcQrskYXD8qPJWt1tnqI5H5I3EVw4FSHsp5EY16blirkkZ9v29WR 4TJN+UTjtscsNLGCLWjG15BM7S9EuHWTv33pADA1AFzlBeZUU3jsdnYjTU5R6vng GixxZUvsXm+wUh6Vov5K7PqPdhwK3Lyo6pvdrkZfbtxmmTQVWB67DHCNawGApZvf WC9vecoAUqJO1d4dfnFau0E/w0qPE/Qp6u7aiqDnr0qKhg/9P2IYBt9K8pU9gdxU WA0FX+2fNd9qXXDnyOLLS3oSjJ75SN0X6g5kyfPoXVM5YbuTpNyg/bTrpvNh5gz/ R18qQelFm5XOuJGnUoIxcTcBLBWjUhICYP6pdj8Navnu3m0hkb5vZPXDwfdZmCiZ ujzA+Eb1nE+IrfJcxYWCwd0Z0JzKzB4dLWju6YYAf0uT9mqRR2s= =KCbk -----END PGP SIGNATURE----- --Apple-Mail=_D1611BBC-3753-4484-8DDB-6D053D1F8C04-- From owner-freebsd-current@freebsd.org Thu Sep 28 15:54:40 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CC73E0205A for ; Thu, 28 Sep 2017 15:54:40 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C6F7639F7 for ; Thu, 28 Sep 2017 15:54:39 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22f.google.com with SMTP id g32so1967367ioj.2 for ; Thu, 28 Sep 2017 08:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=yLU/+AMqWcX6f8k5wRj5w4i64L3FUVYyEFhp3UzxKlE=; b=JF9A2WfXb8xasi2fhJ7hGawuNRca8r8YX7uMrOPrhZ6o6ukoTvZWSP6rAFj1PKa2sN 2q5OwLfodpgMq6ha4Px8rU9NhQfEbxxWkJQyIc3Lv+HtIIRP77RgPbSgV9dyWwbVYCs+ 0mRJvd/oq92qQPPWKFbN7FjZeuNN+WjclCz0z9up5ICt5qMttstglw23EhVbf5FzXphO cwvSraFUISr9PF799cq3QG/zgVe7xGEmf1sCiURBznjanoqevJjQXMDlu+8xgDMgRw/V hE+PEtnXH+f3P61yvQa+NaRaLlWeM4Jk1MncJtUbUqygjwyC8vM8/SNTtFzDBpHeD4hs essg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=yLU/+AMqWcX6f8k5wRj5w4i64L3FUVYyEFhp3UzxKlE=; b=V+jNbqELk6Mso4KRkSyUEQn/2C8IaDa5Hu/6AUBPZUSeu7810ubsTtqnrJCsTbAxr8 0x+BYQsK8fpRCHZsjrDQxOUXc5nJF0OAmx02Jn+/zY+RmPNoshdU5odPRAkzpfNY+MZw 9tTn/eFmOT0xPcRbWFZ6/eUbsTLwHtIaGzgTvgcbE5zKiAZjgfH1CSolfwESfpRaGBnN ATYwSqw3c9759pNfzN8i4wGsNXz1UgQ0XaYBsrwAKbvhVbXNz9aRJRyMwce5cuduYJnd Pz8UCY7EAV/2Qr0K7PXwv80BLGbw34H+F6D0y+y+vvpPBAGsLnrV4YY5GzRQTqj40Wlq DixA== X-Gm-Message-State: AMCzsaWOgj4+Z97P0OX8fATC+sfsnDTf6dUerG5V1xkiIq9kgbW09yJR Jn+8acA6QIWZxkFiREGcTiyJmO1c8y32iqwrokWRTVgr X-Google-Smtp-Source: AOwi7QAygVBtJELqmM19Asa9sv3PaCL9jdH11CPFw7mTNn1CbuN4zMUHWiEixysClrLB6bgQlPMNuYEm+TUgbwZyvs4= X-Received: by 10.107.185.7 with SMTP id j7mr8296608iof.221.1506614078557; Thu, 28 Sep 2017 08:54:38 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.2.194 with HTTP; Thu, 28 Sep 2017 08:54:37 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:191c:5e93:2feb:3c70] In-Reply-To: References: From: Warner Losh Date: Thu, 28 Sep 2017 09:54:37 -0600 X-Google-Sender-Auth: DH4my3dQIc3FrbPHp6BEdKKmQpY Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: Nick Hibma Cc: FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:54:40 -0000 On Thu, Sep 28, 2017 at 9:31 AM, Nick Hibma wrote: > I created a new kernel config file from scratch, wondered what the > GEOM_PART_MBR option and friends were doing, search for them, didn't find > them in the tree, and deleted them from my config. But... de resulting disk > image didn't boot, because of the fact that it didn't recognise the MBR > partitions (it only had a single diskid entry on the mount-root prompt). > > Can anyone explain to me how these kernel options work, as in: they are > defined in kernel configs and as a consequence in opt_geom.h, but how are > they actually used to select which geom_part_* modules/kernel parts to > build? I thought these options were translated to stuff that cpp would use, > but there are not uses of for example GEOM_PART_MBR anywhere for example! > The module always build them because they are listed in the module's Makefile. The kernel only sometimes does. Here's the key lines from conf/files: files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd files:geom/part/g_part_apm.c optional geom_part_apm files:geom/part/g_part_bsd.c optional geom_part_bsd files:geom/part/g_part_bsd64.c optional geom_part_bsd64 files:geom/part/g_part_ebr.c optional geom_part_ebr files:geom/part/g_part_gpt.c optional geom_part_gpt files:geom/part/g_part_ldm.c optional geom_part_ldm files:geom/part/g_part_mbr.c optional geom_part_mbr files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 which turn on/off which files get included. config "helpfully" convers the upper case options to lower case for this. Warner Warner > The only thing I was able to come up with, but could not figure out, was > FEATURE() doing some magic. > > Thanks in advance for any pointers! > > Nick Hibma > nick@van-laarhoven.org > > -- Open Source: We stand on the shoulders of giants. > > > % grep -r GEOM_PART_ /usr/src/sys/ | grep -Ev '/conf/.*options' > /usr/src/sys/geom/part/g_part_mbr.c: "GEOM_PART_MBR Master Boot > Record"); > /usr/src/sys/geom/part/g_part_ldm.c: "GEOM_PART_LDM Logical Disk > Manager"); > /usr/src/sys/geom/part/g_part_ldm.c: * XXX: We use some knowledge > about GEOM_PART_GPT internal > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part.h:#ifndef _GEOM_PART_H_ > /usr/src/sys/geom/part/g_part.h:#define _GEOM_PART_H_ > /usr/src/sys/geom/part/g_part.h:#endif /* !_GEOM_PART_H_ */ > From owner-freebsd-current@freebsd.org Thu Sep 28 16:00:46 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64022E0239A for ; Thu, 28 Sep 2017 16:00:46 +0000 (UTC) (envelope-from nick@van-laarhoven.org) Received: from valery.hibma.org (valery.hibma.org [178.21.114.54]) by mx1.freebsd.org (Postfix) with ESMTP id C7F4B63E7A for ; Thu, 28 Sep 2017 16:00:45 +0000 (UTC) (envelope-from nick@van-laarhoven.org) Received: from [IPv6:2001:980:530a:1:d199:f536:37f0:a864] (unknown [IPv6:2001:980:530a:1:d199:f536:37f0:a864]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by valery.hibma.org (Postfix) with ESMTPSA id 3008F6E0A17; Thu, 28 Sep 2017 18:00:44 +0200 (CEST) From: Nick Hibma Message-Id: <30C7B916-C022-432B-BA12-F51536F7CC7D@van-laarhoven.org> Content-Type: multipart/signed; boundary="Apple-Mail=_9577C347-7CF2-41EC-8029-473806168AC8"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? Date: Thu, 28 Sep 2017 18:00:43 +0200 In-Reply-To: Cc: FreeBSD Current Mailing List To: Warner Losh References: X-Mailer: Apple Mail (2.3273) X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 16:00:46 -0000 --Apple-Mail=_9577C347-7CF2-41EC-8029-473806168AC8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii >> I created a new kernel config file from scratch, wondered what the >> GEOM_PART_MBR option and friends were doing, search for them, didn't = find >> them in the tree, and deleted them from my config. But... de = resulting disk >> image didn't boot, because of the fact that it didn't recognise the = MBR >> partitions (it only had a single diskid entry on the mount-root = prompt). >>=20 >> Can anyone explain to me how these kernel options work, as in: they = are >> defined in kernel configs and as a consequence in opt_geom.h, but how = are >> they actually used to select which geom_part_* modules/kernel parts = to >> build? I thought these options were translated to stuff that cpp = would use, >> but there are not uses of for example GEOM_PART_MBR anywhere for = example! >>=20 >=20 > The module always build them because they are listed in the module's > Makefile. >=20 > The kernel only sometimes does. Here's the key lines from conf/files: > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > files:geom/part/g_part_apm.c optional geom_part_apm > files:geom/part/g_part_bsd.c optional geom_part_bsd > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > files:geom/part/g_part_ebr.c optional geom_part_ebr > files:geom/part/g_part_gpt.c optional geom_part_gpt > files:geom/part/g_part_ldm.c optional geom_part_ldm > files:geom/part/g_part_mbr.c optional geom_part_mbr > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 >=20 > which turn on/off which files get included. config "helpfully" = converts the > upper case options to lower case for this. >=20 > Warner *slaps forehead* Goose chase! I actually knew that... and, at the time, thought it was weird = behaviour. ''grep" would not have failed me if those options would be = uppercase there ... Thanks, Warner. Nick --Apple-Mail=_9577C347-7CF2-41EC-8029-473806168AC8 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 Comment: GPGTools - https://gpgtools.org iQIzBAEBCgAdFiEETFbRZ/gKjBgO10CrH3Ic7sY0duAFAlnNHKsACgkQH3Ic7sY0 duBQRg/+PfO2q/GcbDsinZUz9dODAIl29H9Tk/JPYdT2itrd4+MyYKpP0GYLQamE QktYP+3rmonZsK4bw3iTEiZyegb+3gSXR7C6WP/yZT5Nw4oejKeCfibEyauSiIxl 1CvT1ux0aGUQOH3QNBCts3N5ULWYwGzU7KRboSdeyq8Gqe+/fb51/06LXMJSh1Mr cWcrN5OSPXW/fd2zpvb2gb3CCCBxIMzM+FAeIGGcN5gGrckxW6wmwp/JNh58Y3jZ yvHWMicjMrMTGBpxpKy3X0KydrPlI8v1UMOdhgx+A3EzUNS082TQ189thiVE1iqU PreTd06m9+Tex4dr2LJXgECvdKQnNEpNzzciz2BtWtR75vOr4SKLZex6smZ5CQmt pazrTNcXQMoJ/2xWDBeRm6uV53Np8YbcrPHz+Y7VLp1X4PCcKBzcPoGS7iVkr4Up j/HO3ZIWJVDgB4LpU+/OWd5vNUdS5t296PakjfKjYp3d2He4zi74bnHe3h9J+J17 ApSiBTeYa8kzTMVgvaeEPJBGu/gE1fJ2aQNqiq5WvflDzaSaU2DdPw7agUx2MfpL 82YTHimTujJ6TNGMFOzqUP1VgAIq6OKctDwXRmml8yNNzI5U871GxCWVkwnVQYJc r00jIo/JnPYV+32/TIzXmrc7/T5VStWmNR85h8O4kUfXE/JE+wA= =55js -----END PGP SIGNATURE----- --Apple-Mail=_9577C347-7CF2-41EC-8029-473806168AC8-- From owner-freebsd-current@freebsd.org Thu Sep 28 16:13:16 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 046DAE02A42 for ; Thu, 28 Sep 2017 16:13:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC0FF6475A for ; Thu, 28 Sep 2017 16:13:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22d.google.com with SMTP id d16so2011404ioj.3 for ; Thu, 28 Sep 2017 09:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Ar/tgfeW0mxAYhx21zHRX7b31hh3vv7QgOklm9uWtI0=; b=VoPk3cqq9zlg3N+wJZOPmPLTazSRB19x8Wq1zIAtfZ95WvPM7ELSxy2UqMynR0YnZl Sr16W19aPC1sVvtu75uHmAVjN3l7+AYUQSS/0AkTelkRfSTI376sAIdzZLKzpaoEaRfa Xth2NUz69B11jlOsvirZbZ8ibUrL3NwEURyYFQ1OB5YNvRi+MN8UkE3XS4XW6DMyyvZK 9cJlbMUxyNZaF/VMMRmqqGxzKMwuvKpNNEMTDdA/0XP9bXsxelGTep/k5niDf2/AZo+9 VbnhMyazRYtYSYfbrvYsjJAyu8ctDNAf704ZqQUfUzlSyqq6NnUYESEV5qdwmChgpdNS ycBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Ar/tgfeW0mxAYhx21zHRX7b31hh3vv7QgOklm9uWtI0=; b=RaA2JYAGB5wkwAdqn1UBQHKftEn9Vp1QDOpJh88ZXujdoyShy8OO26N5C+bkjaiMh7 d0CaTE9AR6vt3nLR5z3rQa3WVa8eLoiswhT7Eyd63vcC8chJrQB7K7Ex1arZjCJf8XsY 4jCUb1BfbLF5esLenbrNMAp17lqGto5/oDF+YRA3kei7g/zjwNQIP3pdYCx9jrMi1yYa i9i8Rz1OEMU52We6LSyXJ2YFh9oYQVhBe9re1fedowEF+aFpJj89bVK4RRuNqnwiFHPn 9YcEh4JROfd/YvWDnW/rTFC1fn2bK7JPS8UFvT8ySdu3GDsMjnmn0NpY2xx34dNr/e39 xJvA== X-Gm-Message-State: AMCzsaXuhQZ7NHYHsI0zNAB+3DRb390kwPPWEL/JOhrBJWsr1cynvEbL SEWDgpDFZU+L5qgsClyoV0bNINo3rU53iOJWjvmvgA== X-Google-Smtp-Source: AOwi7QAW7b1CYNDjZA6bVqe+WAIYa0QnoGjClSEzDFuAG7mn5TBCHB93RjD07a3e1hrBPxMCaJwYsnT8Ze562SvGdPs= X-Received: by 10.107.185.7 with SMTP id j7mr8397262iof.221.1506615195101; Thu, 28 Sep 2017 09:13:15 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.2.194 with HTTP; Thu, 28 Sep 2017 09:13:14 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:191c:5e93:2feb:3c70] In-Reply-To: <30C7B916-C022-432B-BA12-F51536F7CC7D@van-laarhoven.org> References: <30C7B916-C022-432B-BA12-F51536F7CC7D@van-laarhoven.org> From: Warner Losh Date: Thu, 28 Sep 2017 10:13:14 -0600 X-Google-Sender-Auth: oJzI2Uo5MhCGNbbXGEX570j2StM Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: Nick Hibma Cc: FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 16:13:16 -0000 On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma wrote: > I created a new kernel config file from scratch, wondered what the > GEOM_PART_MBR option and friends were doing, search for them, didn't find > them in the tree, and deleted them from my config. But... de resulting disk > image didn't boot, because of the fact that it didn't recognise the MBR > partitions (it only had a single diskid entry on the mount-root prompt). > > Can anyone explain to me how these kernel options work, as in: they are > defined in kernel configs and as a consequence in opt_geom.h, but how are > they actually used to select which geom_part_* modules/kernel parts to > build? I thought these options were translated to stuff that cpp would use, > but there are not uses of for example GEOM_PART_MBR anywhere for example! > > > The module always build them because they are listed in the module's > Makefile. > > The kernel only sometimes does. Here's the key lines from conf/files: > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > files:geom/part/g_part_apm.c optional geom_part_apm > files:geom/part/g_part_bsd.c optional geom_part_bsd > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > files:geom/part/g_part_ebr.c optional geom_part_ebr > files:geom/part/g_part_gpt.c optional geom_part_gpt > files:geom/part/g_part_ldm.c optional geom_part_ldm > files:geom/part/g_part_mbr.c optional geom_part_mbr > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > > which turn on/off which files get included. config "helpfully" converts the > upper case options to lower case for this. > > Warner > > > *slaps forehead* Goose chase! > > I actually knew that... and, at the time, thought it was weird behaviour. > ''grep" would not have failed me if those options would be uppercase there > ... > I've been nibbled to death by these geese often enough to have a PTSD-like reaction when someone mentions it and habitually add -i to my greps... Warner From owner-freebsd-current@freebsd.org Thu Sep 28 17:10:15 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83F12E041D9 for ; Thu, 28 Sep 2017 17:10:15 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from crab.apple.relay.mailchannels.net (crab.apple.relay.mailchannels.net [23.83.208.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D62E166694 for ; Thu, 28 Sep 2017 17:10:14 +0000 (UTC) (envelope-from ian@freebsd.org) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id EB6FA2091F3 for ; Thu, 28 Sep 2017 15:53:34 +0000 (UTC) Received: from outbound1a.eu.mailhop.org (unknown [100.96.147.85]) (Authenticated sender: duocircle) by relay.mailchannels.net (Postfix) with ESMTPA id 6C57E20893D for ; Thu, 28 Sep 2017 15:53:34 +0000 (UTC) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [172.20.110.49]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Thu, 28 Sep 2017 15:53:34 +0000 X-MC-Relay: Forwarding X-MailChannels-SenderId: _forwarded-from|73.78.92.27 X-MailChannels-Auth-Id: duocircle X-Coil-Battle: 67397e4c3598f8bf_1506614014842_290604630 X-MC-Loop-Signature: 1506614014841:482120150 X-MC-Ingress-Time: 1506614014841 X-MHO-User: 2c091121-a465-11e7-a893-25625093991c X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id 2c091121-a465-11e7-a893-25625093991c; Thu, 28 Sep 2017 15:53:30 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v8SFrR9L001425; Thu, 28 Sep 2017 09:53:27 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1506614007.31939.19.camel@freebsd.org> Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? From: Ian Lepore To: Nick Hibma , FreeBSD Current Mailing List Date: Thu, 28 Sep 2017 09:53:27 -0600 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 17:10:15 -0000 On Thu, 2017-09-28 at 17:31 +0200, Nick Hibma wrote: > I created a new kernel config file from scratch, wondered what the > GEOM_PART_MBR option and friends were doing, search for them, didn't > find them in the tree, and deleted them from my config. But... de > resulting disk image didn't boot, because of the fact that it didn't > recognise the MBR partitions (it only had a single diskid entry on > the mount-root prompt). >=20 > Can anyone explain to me how these kernel options work, as in: they > are defined in kernel configs and as a consequence in opt_geom.h, but > how are they actually used to select which geom_part_* modules/kernel > parts to build? I thought these options were translated to stuff that > cpp would use, but there are not uses of for example GEOM_PART_MBR > anywhere for example! >=20 > The only thing I was able to come up with, but could not figure out, > was FEATURE() doing some magic. >=20 > Thanks in advance for any pointers! >=20 > Nick Hibma > nick@van-laarhoven.org >=20 > -- Open Source: We stand on the shoulders of giants. >=20 >=20 > % grep -r GEOM_PART_ /usr/src/sys/ | grep -Ev '/conf/.*options' > /usr/src/sys/geom/part/g_part_mbr.c:=A0=A0=A0=A0"GEOM_PART_MBR Master B= oot > Record"); > /usr/src/sys/geom/part/g_part_ldm.c:=A0=A0=A0=A0"GEOM_PART_LDM Logical = Disk > Manager"); > /usr/src/sys/geom/part/g_part_ldm.c: =A0* XXX: We use some > knowledge about GEOM_PART_GPT internal > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#if defined(GEOM_PART_EBR_COMPAT) > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part_ebr.c:#ifndef GEOM_PART_EBR_COMPAT > /usr/src/sys/geom/part/g_part.h:#ifndef _GEOM_PART_H_ > /usr/src/sys/geom/part/g_part.h:#define _GEOM_PART_H_ > /usr/src/sys/geom/part/g_part.h:#endif /* !_GEOM_PART_H_ */ If you had added '-i' to your grep command you would have found the missing piece in sys/conf/files. =A0config(8) turns uppercase OPTION_NAME into lowercase option_name for purposes of configuring which files to compile. -- Ian From owner-freebsd-current@freebsd.org Thu Sep 28 17:12:48 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8688BE04496 for ; Thu, 28 Sep 2017 17:12:48 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3CCA366A49 for ; Thu, 28 Sep 2017 17:12:48 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x233.google.com with SMTP id d12so1147327vkf.1 for ; Thu, 28 Sep 2017 10:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=DNVDvAiv9o+eE3Y6xz2UIlW5G8aLQu/BUSqb7Ex7bO0=; b=rN0JyUKuXa619vUkjD0CYTaA1gj1S9AEeO/5YDJuB4qDrs/uSPU7myZuIcX/ZJeBdV lgaMmkWWhJHLSAX7xkscq+vIYDa52gYgmm83maqVWsHx3yUvdkxwyk9+58g81eqRPzkW Xsf6xAEimHVY2vMiDY/3rUL0JDQmIlGzyTNWqyh4IC0tqFT0FIuYs+lGYp9drpACLzSX f2jK/UAAfFlXrjkmUYvVR4tb9qhXoiztjwgkx+oGBjadp/RtpbKVsx0sRTMD3snE0Ins 1YkEbVfTqZGVLCEo2nTtbvuQJvvrOnUz7h40qZtGEg8oHFesACb7M3iymuAxAxWhDzjm +A8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=DNVDvAiv9o+eE3Y6xz2UIlW5G8aLQu/BUSqb7Ex7bO0=; b=BkQFmp4YfyBX75isw+wPYkN4j1uNvCNG1V9VT0BDa560ddnQzNBlscEng0gV1kkh7i y9GVwWfQPrL6sg/pCruCMb4FJGgOjBSmL2xMJuL+I684w20fMVsBQR1NgxH8u5DVVBYE gj68+F1jLKQCuM9N2a8aGDcJ9lRm63t6guIebBcxYgRLhCzNjAQIHz6RES4wiTRgnCZ9 3+Bz8Njh4gu1IxB16jcif+pF8/7aZIW3WZbihznIw5bsCNokwYy6tWvR4dqSySR/BugV V01gIarKxmrV19YKQXbKUSbq+UJmTrRbb14asgXolxVu6QUb/vPZhZhO+K4tJLFFbvr6 OMHQ== X-Gm-Message-State: AHPjjUiagqFTSmd3+r6/vXDS4vHiNpXEWhpmmEtRWlAAbv7fZqGigjGp TZUNh4bFECDnZ3RimO+L7xohw2G8fogBoO9oijfpCLU3 X-Google-Smtp-Source: AOwi7QBsgt0vgg0wh4b6ykif3vKGJy7rVMCSAiEEPPUez0o9KS/enCLFIK6OJ+AhB4+gDbBbMUtGu396xbSC0SJPfq8= X-Received: by 10.31.141.14 with SMTP id p14mr2765323vkd.53.1506618767134; Thu, 28 Sep 2017 10:12:47 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.134.194 with HTTP; Thu, 28 Sep 2017 10:12:46 -0700 (PDT) In-Reply-To: References: <30C7B916-C022-432B-BA12-F51536F7CC7D@van-laarhoven.org> From: Kevin Oberman Date: Thu, 28 Sep 2017 10:12:46 -0700 X-Google-Sender-Auth: n5pqtQx4aKOjcKDc7EOvmTKotHk Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: Warner Losh Cc: Nick Hibma , FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 17:12:48 -0000 On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > wrote: > > > I created a new kernel config file from scratch, wondered what the > > GEOM_PART_MBR option and friends were doing, search for them, didn't find > > them in the tree, and deleted them from my config. But... de resulting > disk > > image didn't boot, because of the fact that it didn't recognise the MBR > > partitions (it only had a single diskid entry on the mount-root prompt). > > > > Can anyone explain to me how these kernel options work, as in: they are > > defined in kernel configs and as a consequence in opt_geom.h, but how are > > they actually used to select which geom_part_* modules/kernel parts to > > build? I thought these options were translated to stuff that cpp would > use, > > but there are not uses of for example GEOM_PART_MBR anywhere for example! > > > > > > The module always build them because they are listed in the module's > > Makefile. > > > > The kernel only sometimes does. Here's the key lines from conf/files: > > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > > files:geom/part/g_part_apm.c optional geom_part_apm > > files:geom/part/g_part_bsd.c optional geom_part_bsd > > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > > files:geom/part/g_part_ebr.c optional geom_part_ebr > > files:geom/part/g_part_gpt.c optional geom_part_gpt > > files:geom/part/g_part_ldm.c optional geom_part_ldm > > files:geom/part/g_part_mbr.c optional geom_part_mbr > > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > > > > which turn on/off which files get included. config "helpfully" converts > the > > upper case options to lower case for this. > > > > Warner > > > > > > *slaps forehead* Goose chase! > > > > I actually knew that... and, at the time, thought it was weird behaviour. > > ''grep" would not have failed me if those options would be uppercase > there > > ... > > > > I've been nibbled to death by these geese often enough to have a PTSD-like > reaction when someone mentions it and habitually add -i to my greps... > > Warner This horrid POLA violation seems to have been in FreeBSD configuration since at least 3.0 and probably goes back to the creation of the configuration process. Any idea why such a horrible POLA was ever introduced? Seems like an obviously bad idea in an OS that is ALMOST always case sensitive. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-current@freebsd.org Thu Sep 28 17:45:56 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E275E04C4D for ; Thu, 28 Sep 2017 17:45:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A24767655 for ; Thu, 28 Sep 2017 17:45:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x234.google.com with SMTP id m123so2417893ita.3 for ; Thu, 28 Sep 2017 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=XnIiO1miMj/eZHSFWm2wYhzGhfWmSOx/riUJ0RkBdlg=; b=oDJycVlV2chWurZzDB7r39ncQosDknkS55jjTsAbREjXZhEGlA3NFGoaHflZ9d8wS1 c/zC9en50YuNrkD41Gma0wmJKngiqPSX7hHQxx4nfAPaWyQDP5EBjnF8VtlaMv1pFXjt xX6fCKxVm7PziVzeYyhEiy1RK46y6tV5kKgirKeGIuqsWwJFdZcdArpFW3KaZAVjwBRZ UFU6U7UERQatRNAhOxqdRrbXFy6Bsjrz9HX447iYEvHOHR5S4peMSrl9E++Ri/oWdAfS hcnTTJipblqtLUh5AjSQmqPoqBzi+fw697iD9925G553SqIQNX2+8PrES/MUMTFNjhXO 2aYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=XnIiO1miMj/eZHSFWm2wYhzGhfWmSOx/riUJ0RkBdlg=; b=b/CpPg6ySq6wtLlxzag80w6JCv61CyyDo6r2UZXSVAzQytLnd+xD6aNh9A/+IzP5vD ptAPmHOKM0h/KFRTaBN4AjHf4d4oERXpsgPSBCgaYv1VwBBiBiFi05/K1hOj6M0KaChi Km5L+Vzz+p47AqNGY22PThQ0TnDsRSi2ZfSdDCkI/xvT8xrJRY0h1YJaUBYbR031AxJE uVqXX15c6Np8lZHmOb6+VfGytftDWLkh9QeYYGPqtTrKVHG+gQ94RpbFrno4hj6dFQsJ d2oYLt4f6g/Cls1ZjF6CKXJfm6yOIxI4mcAjLMcy83bZY3I1L9qtgFHi7XIOhU92hd1D Mwzw== X-Gm-Message-State: AMCzsaVADuOgI/g8y3wP72pPWs+eJ6ICDd2sZED9oi/S46isuRKRB6Nc dt3NhIhzWrApjAyd7k1jITtspu4CBiSK5K/7PONOEQ== X-Google-Smtp-Source: AOwi7QD4/VcQElUYbl2ZnHHv5uu1lnJ+KEB+M4gqHr0TK3RbSbxvgAqo/7bfIoXhENY6qm9I+zzOXYYs+ZHgIqZu18k= X-Received: by 10.36.6.18 with SMTP id 18mr3036559itv.15.1506620754695; Thu, 28 Sep 2017 10:45:54 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.2.194 with HTTP; Thu, 28 Sep 2017 10:45:53 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:191c:5e93:2feb:3c70] In-Reply-To: References: <30C7B916-C022-432B-BA12-F51536F7CC7D@van-laarhoven.org> From: Warner Losh Date: Thu, 28 Sep 2017 11:45:53 -0600 X-Google-Sender-Auth: c49iZY6ntl3bspq-ZgkNZPGt8so Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: Kevin Oberman Cc: Nick Hibma , FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 17:45:56 -0000 On Thu, Sep 28, 2017 at 11:12 AM, Kevin Oberman wrote: > On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > >> On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma >> wrote: >> >> > I created a new kernel config file from scratch, wondered what the >> > GEOM_PART_MBR option and friends were doing, search for them, didn't >> find >> > them in the tree, and deleted them from my config. But... de resulting >> disk >> > image didn't boot, because of the fact that it didn't recognise the MBR >> > partitions (it only had a single diskid entry on the mount-root prompt). >> > >> > Can anyone explain to me how these kernel options work, as in: they are >> > defined in kernel configs and as a consequence in opt_geom.h, but how >> are >> > they actually used to select which geom_part_* modules/kernel parts to >> > build? I thought these options were translated to stuff that cpp would >> use, >> > but there are not uses of for example GEOM_PART_MBR anywhere for >> example! >> > >> > >> > The module always build them because they are listed in the module's >> > Makefile. >> > >> > The kernel only sometimes does. Here's the key lines from conf/files: >> > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd >> > files:geom/part/g_part_apm.c optional geom_part_apm >> > files:geom/part/g_part_bsd.c optional geom_part_bsd >> > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 >> > files:geom/part/g_part_ebr.c optional geom_part_ebr >> > files:geom/part/g_part_gpt.c optional geom_part_gpt >> > files:geom/part/g_part_ldm.c optional geom_part_ldm >> > files:geom/part/g_part_mbr.c optional geom_part_mbr >> > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 >> > >> > which turn on/off which files get included. config "helpfully" converts >> the >> > upper case options to lower case for this. >> > >> > Warner >> > >> > >> > *slaps forehead* Goose chase! >> > >> > I actually knew that... and, at the time, thought it was weird >> behaviour. >> > ''grep" would not have failed me if those options would be uppercase >> there >> > ... >> > >> >> I've been nibbled to death by these geese often enough to have a PTSD-like >> reaction when someone mentions it and habitually add -i to my greps... >> >> Warner > > > This horrid POLA violation seems to have been in FreeBSD configuration > since at least 3.0 and probably goes back to the creation of the > configuration process. > > Any idea why such a horrible POLA was ever introduced? Seems like an > obviously bad idea in an OS that is ALMOST always case sensitive. > It's received code from the old 4.3 BSD config program (or maybe the net-2 config program). Warner From owner-freebsd-current@freebsd.org Thu Sep 28 17:55:36 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7B12E05133 for ; Thu, 28 Sep 2017 17:55:36 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F32667BB4 for ; Thu, 28 Sep 2017 17:55:35 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with SMTP id xd1xdzDirI8mCxd1ydGdYv; Thu, 28 Sep 2017 11:55:28 -0600 X-Authority-Analysis: v=2.2 cv=HahkdmM8 c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=2JCJgTwv5E4A:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=7Qk2ozbKAAAA:8 a=pc8SvQfqAAAA:8 a=pGLkceISAAAA:8 a=EYmTS2ldPSQIW-lwIQAA:9 a=CjuIK1q_8ugA:10 a=a4w0SzYmEskA:10 a=a4zXcJKqs8IJMo8Z-wUA:9 a=ZajDK59Warh2Du5F:21 a=_W_S_7VecoQA:10 a=Fj9iO6pqr7gSyLvOkxId:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=1lyxoWkJIXJV6VJUPhuM:22 a=kM0ClJ1xsE7__0tUH1SI:22 Received: from [25.81.203.182] (S0106d4ca6d8943b0.gv.shawcable.net [24.68.134.59]) by spqr.komquats.com (Postfix) with ESMTPSA id 2D0042D5; Thu, 28 Sep 2017 10:55:25 -0700 (PDT) MIME-Version: 1.0 From: Cy Schubert Subject: RE: How do GEOM_PART_* options configure geom_part_* modules?? Date: Thu, 28 Sep 2017 10:55:28 -0700 To: Kevin Oberman , Warner Losh CC: Nick Hibma , FreeBSD Current Mailing List , "cy.schubert@cschubert.com" Message-Id: <20170928175525.2D0042D5@spqr.komquats.com> X-CMAE-Envelope: MS4wfCeG0U52a5wzlLwA9EieGDZ8vckILouIpMdAjbp/A2pJ/bI83hnSRPcRH77IubOqgJphP84G1EeK/S2W0aW0mUU12a6XMjH6vMBr67sz3xVL14m0ZUKB d6hhnPcqFSPBajcvS3JfeFL/2nLAlfYSxNKFGUmA+35mK4FXqPJC0rNS+aOVvKSUMspJKNBY+oRLjlJjHGxfvZ68lyIkr4Gahy5GJ5REhMlq6TZvVcIzBgyw jbZUNxCCXYHbshtTPZTeEwdh0kkmFsdPCznnQ/dOPnE= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 17:55:36 -0000 Changing grep to default to -i would be the POLA violation. Rather than mes= s with grep like RHEL has I suggest installing textproc/the_silver_searcher= pkg/port. It does everything grep does and more. I've switched to ag (whic= h by default does recursive case insensitive searches) while using grep in = scripts and at times I need simply grep. --- Sent using a tiny phone keyboard. Apologies for any typos and autocorrect. Cy Schubert or -----Original Message----- From: Kevin Oberman Sent: 28/09/2017 10:13 To: Warner Losh Cc: Nick Hibma; FreeBSD Current Mailing List Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > wrote: > > > I created a new kernel config file from scratch, wondered what the > > GEOM_PART_MBR option and friends were doing, search for them, didn't fi= nd > > them in the tree, and deleted them from my config. But... de resulting > disk > > image didn't boot, because of the fact that it didn't recognise the MBR > > partitions (it only had a single diskid entry on the mount-root prompt)= . > > > > Can anyone explain to me how these kernel options work, as in: they are > > defined in kernel configs and as a consequence in opt_geom.h, but how a= re > > they actually used to select which geom_part_* modules/kernel parts to > > build? I thought these options were translated to stuff that cpp would > use, > > but there are not uses of for example GEOM_PART_MBR anywhere for exampl= e! > > > > > > The module always build them because they are listed in the module's > > Makefile. > > > > The kernel only sometimes does. Here's the key lines from conf/files: > > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > > files:geom/part/g_part_apm.c optional geom_part_apm > > files:geom/part/g_part_bsd.c optional geom_part_bsd > > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > > files:geom/part/g_part_ebr.c optional geom_part_ebr > > files:geom/part/g_part_gpt.c optional geom_part_gpt > > files:geom/part/g_part_ldm.c optional geom_part_ldm > > files:geom/part/g_part_mbr.c optional geom_part_mbr > > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > > > > which turn on/off which files get included. config "helpfully" converts > the > > upper case options to lower case for this. > > > > Warner > > > > > > *slaps forehead* Goose chase! > > > > I actually knew that... and, at the time, thought it was weird behaviou= r. > > ''grep" would not have failed me if those options would be uppercase > there > > ... > > > > I've been nibbled to death by these geese often enough to have a PTSD-lik= e > reaction when someone mentions it and habitually add -i to my greps... > > Warner This horrid POLA violation seems to have been in FreeBSD configuration since at least 3.0 and probably goes back to the creation of the configuration process. Any idea why such a horrible POLA was ever introduced? Seems like an obviously bad idea in an OS that is ALMOST always case sensitive. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Thu Sep 28 18:44:08 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FD16E05DD3 for ; Thu, 28 Sep 2017 18:44:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8435694CB for ; Thu, 28 Sep 2017 18:44:07 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id A497310AB01; Thu, 28 Sep 2017 14:44:00 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org, sgk@troutmask.apl.washington.edu Subject: Re: panic: softdep_deallocate_dependencies: dangling deps Date: Thu, 28 Sep 2017 11:03:46 -0700 Message-ID: <25204210.C6pJgIpUy0@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <20170927221321.GA1023@troutmask.apl.washington.edu> References: <20170927221321.GA1023@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 28 Sep 2017 14:44:00 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 18:44:08 -0000 On Wednesday, September 27, 2017 03:13:21 PM Steve Kargl wrote: > Just got this panic on > > FreeBSD troutmask.apl.washington.edu 12.0-CURRENT FreeBSD 12.0-CURRENT > #0 r321800: Mon Jul 31 13:48:43 PDT 2017 > kargl@:/data/obj/usr/src/sys/SPEW amd64 > > core.txt.0 contains > > panic: softdep_deallocate_dependencies: dangling deps > cpuid = 7 > time = 1506549566 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe023a281710 > vpanic() at vpanic+0x19c/frame 0xfffffe023a281790 > panic() at panic+0x43/frame 0xfffffe023a2817f0 > softdep_deallocate_dependencies() at softdep_deallocate_dependencies+0x76/frame 0xfffffe023a281810 > brelse() at brelse+0x149/frame 0xfffffe023a281870 > bufwrite() at bufwrite+0x65/frame 0xfffffe023a2818b0 > softdep_process_journal() at softdep_process_journal+0x7a8/frame 0xfffffe023a281950 > softdep_process_worklist() at softdep_process_worklist+0x80/frame 0xfffffe023a2819b0 > softdep_flush() at softdep_flush+0xff/frame 0xfffffe023a2819f0 > fork_exit() at fork_exit+0x75/frame 0xfffffe023a281a30 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe023a281a30 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > __curthread () at ./machine/pcpu.h:232 > 232 __asm("movq %%gs:%1,%0" : "=r" (td) > (kgdb) #0 __curthread () at ./machine/pcpu.h:232 > #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:318 > #2 0xffffffff805879eb in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:386 > #3 0xffffffff80587e66 in vpanic (fmt=, ap=0xfffffe023a2817d0) > at /usr/src/sys/kern/kern_shutdown.c:779 > #4 0xffffffff80587c83 in panic (fmt=) > at /usr/src/sys/kern/kern_shutdown.c:710 > #5 0xffffffff80787f56 in softdep_deallocate_dependencies ( > bp=0xfffffe01f008d8b8) at /usr/src/sys/ufs/ffs/ffs_softdep.c:14304 > #6 0xffffffff8061dd69 in buf_deallocate (bp=0xfffffe01f008d8b8) > at /usr/src/sys/sys/buf.h:429 > #7 brelse (bp=0xfffffe01f008d8b8) at /usr/src/sys/kern/vfs_bio.c:2348 > #8 0xffffffff8061b9e5 in bufwrite (bp=0xfffffe01f008d8b8) > at /usr/src/sys/kern/vfs_bio.c:1914 > #9 0xffffffff8079bec8 in softdep_process_journal (mp=, > needwk=0x0, flags=) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:3559 > #10 0xffffffff80785dc0 in softdep_process_worklist (mp=0xfffff80007eef000, > full=0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:1592 > #11 0xffffffff807894ff in softdep_flush (addr=0xfffff80007eef000) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:1397 > #12 0xffffffff80555075 in fork_exit ( > callout=0xffffffff80789400 , arg=0xfffff80007eef000, > frame=0xfffffe023a281a40) at /usr/src/sys/kern/kern_fork.c:1038 > #13 > (kgdb) > > Hmmm, > > % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0 > GNU gdb (GDB) 8.0 [GDB v8.0 for FreeBSD] > Copyright (C) 2017 Free Software Foundation, Inc. > Type "apropos word" to search for commands related to "word"... > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done. > ABI doesn't support a vmcore target > > OK, so debugging is broken :-/ Run the debugger on the binary, not the debug symbols: kgdb /boot/kernel/kernel vmcore.0 -- John Baldwin From owner-freebsd-current@freebsd.org Thu Sep 28 20:26:50 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D43B9E08630; Thu, 28 Sep 2017 20:26:50 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 413356CFAD; Thu, 28 Sep 2017 20:26:49 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from thor.intern.walstatt.dynvpn.de ([85.181.38.24]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LnPGI-1dQEWX07AM-00hfFk; Thu, 28 Sep 2017 22:26:47 +0200 Date: Thu, 28 Sep 2017 22:26:38 +0200 From: "O. Hartmann" To: Damjan Jovanovic Cc: "O. Hartmann" , freebsd-current , freebsd-ipfw@freebsd.org, Guido Falsi Subject: Re: FreeBSD, IPFW and the SIP/VoIP NAT problem Message-ID: <20170928222638.49e3f476@thor.intern.walstatt.dynvpn.de> In-Reply-To: References: <20170926103543.0aa03c7a@freyja.zeit4.iv.bundesimmobilien.de> <20170926154429.1c79d842@freyja.zeit4.iv.bundesimmobilien.de> <20170927131651.7346fd1f@freyja.zeit4.iv.bundesimmobilien.de> Organization: WALSTATT User-Agent: OutScare 3.1415926 X-Operating-System: ImNotAnOperatingSystem 3.141592527 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/.Z7l44Ng0_rTXip7PmIAyIk"; protocol="application/pgp-signature" X-Provags-ID: V03:K0:Ud9Lcs6u92rJ4+EVEUEUdjsV41pO3JhEBP3eAONvMMo89xLJ/oE HbsAfAJB3hmlM0RyWhoa+NxN5Q8zoGAz4kgF5uKL1c/YSaqDoQwF3NtkKCZIL4578CtfDik Y0RO0o54u5hYomfzQYqiiGKjKxLy8Hwh1m0o6rygGGrUVDuvBGKFecl62oEtYCgwT+3OjV7 VQaxobDaGkdcqwrmPRrLw== X-UI-Out-Filterresults: notjunk:1;V01:K0:Q3WIA9/2aeI=:oTEv46EO8vBmxo7TwFbQC8 VIQcZ8tM9SHNBbgixWHy37HqJvv1tb1SNny9BAVR/qzz8/TgDTBuayY9405VVy7s+CAzy0llo nH/6NXYguxDODDC7+jovDl/vrbFswgaPdiH1ScNVtB+VbvAsVz8kFGIBkzVqyUiW0lY2a/k5W yu3IZbXqg6ZcYhmhqPTB17kQ/c5oaMT7g5kQSooIsVEN773lobLv+Rf5b3/penCfdeAMbNjf+ Hxd6Ax/Ku+GI+rOIXNq2UX1KnL5AoaB0rJT53namX5sLRDcbf1jEp4wOBfG525FAn/nxlXN5f GTS3i5x/syD62Ooa/i7I+SQJ/48WP+9OKXBBNJGatiBA8jSfOSydiyX2KVLO7EY4OECOqrwGk kK1rnOFxJkX45yYX4ZZK9vAlIx3QQtcSKUN7G1aQkCMOf1DNUEcnyPfKDN7EgrmSQJNccm13q /oqyrEYDjvQM+MWG1XWkX0ewYgtx4ekBfK0DnayZv/9WbiO+z6Gr5hNmaatus+Sq67ZCrpE1h FZHcymmfAAaqNm1q9iD+1JOWbr8JN0TK0aObTn4x61NkbrBtwrD2FGobY/23I/Cl183JSupaN WYMneZyxDZL887yoK7qs4TYcCIvGUgR5ZOIF/HQNAzLH4kOfWmZwpD6aMqFlh8uyJzG1vOrvk nULJgmpSPfbZQSMkjcmw69nTki91UTRDC2V53tsVxdLEG483g/aXbJB8UVkfKDJh5nrdCNvFT jg2nnVkz7o75QlemQr6Z8qYPcLnwV4VJ9VDIgfTbgrbcaZ6U7h23Mdo653KPBBKPySbaNj3ax 5bPjBpwB877u/XveWSqNaexv/rsyQ== X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 20:26:51 -0000 --Sig_/.Z7l44Ng0_rTXip7PmIAyIk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Am Wed, 27 Sep 2017 14:17:14 +0200 Damjan Jovanovic schrieb: > On Wed, Sep 27, 2017 at 1:16 PM, O. Hartmann > wrote: >=20 > > On Tue, 26 Sep 2017 16:26:51 +0200 > > Damjan Jovanovic wrote: > > =20 > > > On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann > > > wrote: > > > =20 > > > > On Tue, 26 Sep 2017 11:00:45 +0200 > > > > Damjan Jovanovic wrote: > > > > =20 > > > > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann < =20 > > ohartmann@walstatt.org> =20 > > > > > wrote: > > > > > =20 > > > > > > Hello, > > > > > > > > > > > > trying to build a FreeBSD based router/PBX (Asterisk 13) =20 > > appliance, I =20 > > > > ran =20 > > > > > > into > > > > > > several problems. My questions might have a "noobish" character= , =20 > > so my =20 > > > > > > apology, > > > > > > my experiences with IPFW are not as thorough as they should be. > > > > > > > > > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64: > > > > > > > > > > > > - port net/asterisk13 is supposed to build only on armv6, is =20 > > aarch64 =20 > > > > about =20 > > > > > > coming soon also supported? > > > > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX =20 > > platform =20 > > > > > > (assumed > > > > > > having 2 GB of RAM)? > > > > > > > > > > > > My main concern is about IPFW (we do not use PF for some reason= s, I =20 > > > > have to =20 > > > > > > stay with IPFW). > > > > > > > > > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT a= nd =20 > > not =20 > > > > yet =20 > > > > > > IPv6. > > > > > > The FreeBSD system acting as a router is supposed to have a jai= l =20 > > soon =20 > > > > > > containing the Asterisk 13 IP PBX (at the moment running on the= =20 > > main =20 > > > > > > system). > > > > > > To provide access to the VoIP infrastructure inside/behind the = =20 > > > > router/NAT =20 > > > > > > system, the in-kernel NAT facility of FreeBSD is forwarding the= =20 > > > > relevant =20 > > > > > > UPD/TCP ports for VoIP to its destination network, and here I h= ave =20 > > a =20 > > > > > > problem to > > > > > > solve. > > > > > > > > > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and o= ther =20 > > > > ports, =20 > > > > > > it > > > > > > is a mess and pain in the arse to forward a whole range, say =20 > > 11000/udp =20 > > > > - =20 > > > > > > 35000/udp, which is a range one of my providers is sending RTP = on. =20 > > A =20 > > > > second =20 > > > > > > provider uses another range for RTP, starting at 5000/udp. So, = the =20 > > > > logical =20 > > > > > > consequence would be a union set up UDP range to forward, which= =20 > > exapnds =20 > > > > > > then > > > > > > form 5000/udp to 45000/udp - which is much more a pain ... > > > > > > > > > > > > One of the most disturbing and well known problems is that due = to =20 > > the =20 > > > > > > stateful > > > > > > firewall the RTP session very often is half duplex - it seems o= ne =20 > > > > direction =20 > > > > > > of the RTP connection doesn't make it through IPFW/NAT. As ofte= n I =20 > > > > search =20 > > > > > > the > > > > > > net, I always get informed this is a typical problem and soluti= ons =20 > > are =20 > > > > > > provided by so called ALGs - since SIP protocol's SDP indicates= =20 > > within =20 > > > > the =20 > > > > > > payload of the packets on which UDP ports both ends wish to =20 > > establish =20 > > > > their =20 > > > > > > RTP session, it would be "easy" to pinhole the IPFW on exactly = =20 > > those =20 > > > > ports =20 > > > > > > for > > > > > > a theoretical large number of sessions, if IPFW could "divert" = =20 > > those =20 > > > > > > packets > > > > > > to an instance inspecting SDP (or whatever is used for the RTP = port > > > > > > indication, I'm new to that, sorry for the terminology) and the= n =20 > > > > pinholing =20 > > > > > > the > > > > > > NAT/IPFW for exactly this purpose without the forwarding mess. = I =20 > > came =20 > > > > along =20 > > > > > > netgraph() while searching for hints and hooks, but it seems a = =20 > > complete =20 > > > > > > Linux > > > > > > domain, when it somes to appliances like VoIP/IP PBX. > > > > > > > > > > > > Either, the problem is that trivial on FreeBSD, so no further = =20 > > > > mentioning is =20 > > > > > > necessary (which would explain the vast emptyness of explanatio= ns, =20 > > > > hints =20 > > > > > > and > > > > > > so on) or FreeBSD is a complete wasteland on this subject - whi= ch I =20 > > > > also =20 > > > > > > suspect, since pfSense and OPNsense must have come along with s= uch =20 > > > > problems =20 > > > > > > and I simply do not know or recognise the software used for tho= se =20 > > > > purposes. =20 > > > > > > > > > > > > So, if someone enlightened in this matter stumbles over my =20 > > question and =20 > > > > > > could > > > > > > delegate me onto the right way (ports, ng_XXX netgraph ficiliti= es =20 > > to =20 > > > > look =20 > > > > > > at, > > > > > > some ipfw techniques relevant to the problem apart from the stu= pid =20 > > > > simple =20 > > > > > > forwarding large ranges of ports) - I'd appreciate this and > > > > > > > > > > > > thanks in advance for patience and help, > > > > > > > > > > > > Oliver > > > > > > =20 > > > > > > > > > > > > > > > Hi > > > > > > > > > > It might be easier if you just enable STUN on Asterisk, and build= =20 > > FreeBSD =20 > > > > > from source with my [largely neglected :( ] patch on > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219918 > > > > > > > > > > That way Asterisk should dynamically discover consistent external= =20 > > > > mappings =20 > > > > > for connections, making port forwarding RTP unnecessary. > > > > > > > > > > Damjan =20 > > > > > > > > STUN is enabled, but my providers do not support STUN. > > > > > > > > I try to figure out how SIP works exactly to make my problem more = =20 > > precise. =20 > > > > I > > > > also try to understand the aim of your patch - as far as I know, it= =20 > > does =20 > > > > exactly as it is needed for the IPW/NAT/VoIP case. And I really reg= ret =20 > > that =20 > > > > there are objections to commit the patch ... > > > > > > > > =20 > > > Firstly, if your providers support NAT, you register to them (as oppo= sed =20 > > to =20 > > > they register to you, or no registration), and the only VoIP calls are > > > to/from your providers and to/from the same IP:port you register to (= as > > > opposed to unknown external addresses), then none of this should be > > > necessary. Just put these on every SIP peer in Asterisk (this is for > > > chan_sip; not sure about chan_pjsip): =20 > > > > > > > > My providers do support NAT, I suppose, I'm sure with one. Not so > > sure about the iberian Telefonica/O2 - they claim, but they are a kind = of > > not > > willing to provide substantial informations as they want to force > > customers to > > use the (crap) equipment they offer. > > > > Very often, calling from the outside through the NAT/firewall to the PB= X, I > > have this half-duplex phenomenon well described in many palces regarding > > NAT. > > In most cases I can hear the answering machine/voicemail from the PBX, = but > > I > > can not send an audio stream inside. > > > > When my PBX register to its provider, how is the RTP port port for the > > ingress > > media stream (from the view of my PBX) opened? As I understand stateful > > IPFW, > > someone from the inside needs first to punchhole the firewall. I must > > confess, > > I have a bit of an understanding problem here since I do know know the > > protocol. Is there anything on the net explaining this scenario? RFC326= 1 is > > describing SIP, but not the registration ... > > > > =20 > Both sides usually send RTP to each other. When you send RTP through your > NAT to a provider supporting NAT, it should see where you are externally > sending from, and send its future RTP packets back there, even if that > isn't the (internal) IP:port you previously said you would use in your SD= P. >=20 > This can obviously break in some cases: > - If the voice is intentionally one-way throughout the call, such as > phoning out into an announcement service that intentionally says "sendonl= y" > in its SDP, so you aren't sending any RTP to it and its RTP can't route > back to you. > - If you use out of band ringback and transfer out an inbound call before > it's answered, so the call hairpins from the provider through you and bac= k. > You have to send RTP to open NAT mappings, but you have nothing to send, = as > you first need to receive it, but can't as the NAT mappings aren't open: a > cycle you can't exit. >=20 > For those cases, NAT traversal can't be transparent, you have use some ki= nd > of software negotiated NAT traversal: static port forwarding and set > Asterisk's external signaling and media addresses, use STUN with cone NAT > (my patch + STUN/ICE settings in Asterisk's rtp.conf, sip.conf, etc.), or= a > NAT traversal protocol such as UPNP or NAT-PMP with supporting software > (which Asterisk currently isn't). >=20 >=20 > > > > > > directmedia=3Dno > > > nat=3Dforce_rport,comedia =20 > > > > In chan_pjsip, I found these settings important for NAT on peers in > > avrious > > places on the net: > > > > rtp_symmetric=3D yes > > ;rtp_keepalive=3D 30 (not sure about > > ; the timing here, I use this > > value) > > force_rport=3D yes > > rewrite_contact=3D yes > > timers=3D yes > > direct_media=3D no > > disable_direct_media_on_nat=3D yes > > =20 > > > > > > and register to your provider more often than your NAT timeout is (eg. > > > every minute), and you should be good. Why? Every registration opens = a =20 > > NAT =20 > > > mapping that your provider can use to send you calls on. The provider= =20 > > will =20 > > > also send RTP to the source IP:port it received it from, so when you = =20 > > start =20 > > > sending RTP you will get RTP back even if it's arriving from an =20 > > unexpected =20 > > > IP:port. NAT is not a big problem for SIP clients, only for SIP provi= ders > > > that have receive packets from unknown addresses. =20 > > > > I tried to find lifetime settings or timeout, the only (documented) val= ues > > I > > founf were located in ipfw(8): > > > > net.inet.ip.fw.dyn_ack_lifetime: 300 > > > > net.inet.ip.fw.dyn_syn_lifetime: 20 > > > > net.inet.ip.fw.dyn_fin_lifetime: 1 > > > > net.inet.ip.fw.dyn_rst_lifetime: 1 > > > > net.inet.ip.fw.dyn_udp_lifetime: 10 > > > > net.inet.ip.fw.dyn_short_lifetime: 5 =20 > > > > > > Otherwise... > > > > > > Why would your providers need to support STUN? Applications first use= =20 > > STUN =20 > > > to discover the external IP:port of their internal IP:port, and then > > > communicate that IP:port to the other side however they usually would= =20 > > (eg. =20 > > > headers in SIP and SDP packets) - the other side doesn't know or care= =20 > > that =20 > > > they were discovered through STUN. Any STUN server anywhere on the =20 > > Internet =20 > > > can be used for this and should give the same results; see > > > https://www.voip-info.org/wiki/view/STUN for a list. =20 > > > > I can use the STUN of an other provider, but not sure this is necessary, > > since > > both providers I'm consumer do not offer STUN themselfes. > > =20 > > > > > > My patch ensures UDP NAT hole punching logic can be used properly. Wi= th =20 > > it, =20 > > > if a packet was sent from an internal IP:port through an external IP:= port > > > (eg. to a STUN server), then any future packet from that internal IP:= port > > > to any other external server:port will go out the same external IP:po= rt, > > > and no other internal IP:port will use that external IP:port. It's li= ke =20 > > the =20 > > > internal IP:port temporarily owns the unique external IP:port and can= =20 > > send =20 > > > and receive through it to and from anywhere. The same source IP:port = will > > > be seen by all servers, and they can send back to it. =20 > > > > > > That sounds plausible, but implies that, say the PBX behind the NAT at > > IP1:port, is not guaranteed to send exclusively via external IP2:port? > > > > =20 > With my patch, every packet from IP1:port1 will be routed out of IP2:port= 2, > no matter what the destination. Of course software must be written to > detect IP2:port2 for every new socket using something like STUN, and repo= rt > IP2:port2 to other parties it wants traffic from. Stupid question here: is this patch some kind of similar to that what the Linux folks call "CONNT= RACK"? Whenever I read about SIP and NAT in conjunction with Linux, this "conntrac= k" module is always a kind of requisite. On a quick search, I found this page : https://voipmagazine.wordpress.com/2015/03/14/linux-nat-using-conntrack-and= -iptables/ and in some places I had the impression that your patch is exactly what "co= nntrack" is in the Linux world. --=20 O. Hartmann Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Abs.= 4 BDSG). --Sig_/.Z7l44Ng0_rTXip7PmIAyIk Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWc1a/gAKCRDS528fyFhY lAI5Af0RcP/GO7dZfA3R267C6Gh4SbfuXrBrWZBsdrqi4y7LvPthCmpFKCNnAvNH 3n9aZAzMX39Z+kg5uSy7+PF5kxDmAf4kaQZz24fZy+DJM7Cx88AHUHs7+afbd8nB Aljxt+knzegKA4muKE4PGMyBofENRzTjm/92Z9XRirOqasu6Z3fw =1oUM -----END PGP SIGNATURE----- --Sig_/.Z7l44Ng0_rTXip7PmIAyIk-- From owner-freebsd-current@freebsd.org Fri Sep 29 01:29:06 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56552E12610 for ; Fri, 29 Sep 2017 01:29:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1954374C83 for ; Fri, 29 Sep 2017 01:29:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22a.google.com with SMTP id v36so316747ioi.1 for ; Thu, 28 Sep 2017 18:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=wUXMC+f/deIMz+0/Ga0yAFzTvOy18I4h30HARYRaZTA=; b=0IcgK+pZiVCQlxEtkweG/mfw+soVzSd9prQcZGxZsLsSHtvmoW4yJ0Vjx1uXbO4Xl+ 6ulD0P4kO/e+yuy9JSY7RXRRvJ+0bL6S7NDH2q0q5UAZZtk6cWCi22C+V7nDV69bsNJ4 j6eC7ZQ0bA0xzoFkV4M85+E8mYY4042RSXjHwWMjqWiAGG3b9qopniKMC7r7uzAMfKhp HlkFz5dDX5CFvZ3vYPlwcGTpRDl+egPW+nv0S+qtS/QZnY22yyILG0FWTM2+Ll5mN3/c KMTl9XFZ96E6IV3I4r5v+yZmxgjhitEWQDqjHCOs90UyNpJu3JneCwpwYjM4xnvaWCyd jnUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=wUXMC+f/deIMz+0/Ga0yAFzTvOy18I4h30HARYRaZTA=; b=Ez0d0E7Lq413Ahylta9adDvStnWXBd/3TADsN94wIBlh20nAKrgxCh/4uelugpCSYJ NzzyCMNXKcrKH93RAipSn06e4PCJXc/sdAhaP0+aFlKeAGGb5ISkld+8QwyZ4DsrxU6m hFvm/gQ/+T6Qovz/bCxr1N+U40Zob1YmDcLyUJ2I/QW/dy0EOujhw2wSKWbYgaGjuTGg 5Wy7zOLOeBHjrZ4AG9DHz16DHGqbRgJSvx0RVIE0tR3ph6VInordOYxvlMfS5O2Buv6j xgMfZwMkPgEn0KfgHLOPeD3a3MbCpjCP9K5YKDR29KP/nR9c3ueyzv1/u0GGoy0ycl1s y2mw== X-Gm-Message-State: AMCzsaWx+sOpVo6I+if4UvPZzuUXYHq6Qk8PNCxkvFzqassBeIxaJort LQkdD25GrCgIEhh001uGoSKl04Nh/8Re636Zpf3XOQ== X-Google-Smtp-Source: AOwi7QD1pVKt5ckFqOJuD3le73P1jaLiIdZHT84/LNk/nPcVzY/ibwoeGB8EqrHZglw2Sw+Iy0+cNvW4xnYLGpaYn6s= X-Received: by 10.107.133.92 with SMTP id h89mr9557828iod.208.1506648545315; Thu, 28 Sep 2017 18:29:05 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.2.194 with HTTP; Thu, 28 Sep 2017 18:29:04 -0700 (PDT) X-Originating-IP: [2607:fb90:6e97:4975:0:11:542e:4401] Received: by 10.79.2.194 with HTTP; Thu, 28 Sep 2017 18:29:04 -0700 (PDT) In-Reply-To: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> References: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> From: Warner Losh Date: Thu, 28 Sep 2017 19:29:04 -0600 X-Google-Sender-Auth: gviUXHTy62pGPEavuEQNnUHeMX8 Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: "Rodney W. Grimes" Cc: FreeBSD Current , Nick Hibma , Kevin Oberman Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 01:29:06 -0000 On Sep 28, 2017 7:09 PM, "Rodney W. Grimes" < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > On Thu, Sep 28, 2017 at 11:12 AM, Kevin Oberman wrote: > > > On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > > > >> On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > >> wrote: > >> > >> > I created a new kernel config file from scratch, wondered what the > >> > GEOM_PART_MBR option and friends were doing, search for them, didn't > >> find > >> > them in the tree, and deleted them from my config. But... de resulting > >> disk > >> > image didn't boot, because of the fact that it didn't recognise the MBR > >> > partitions (it only had a single diskid entry on the mount-root prompt). > >> > > >> > Can anyone explain to me how these kernel options work, as in: they are > >> > defined in kernel configs and as a consequence in opt_geom.h, but how > >> are > >> > they actually used to select which geom_part_* modules/kernel parts to > >> > build? I thought these options were translated to stuff that cpp would > >> use, > >> > but there are not uses of for example GEOM_PART_MBR anywhere for > >> example! > >> > > >> > > >> > The module always build them because they are listed in the module's > >> > Makefile. > >> > > >> > The kernel only sometimes does. Here's the key lines from conf/files: > >> > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > >> > files:geom/part/g_part_apm.c optional geom_part_apm > >> > files:geom/part/g_part_bsd.c optional geom_part_bsd > >> > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > >> > files:geom/part/g_part_ebr.c optional geom_part_ebr > >> > files:geom/part/g_part_gpt.c optional geom_part_gpt > >> > files:geom/part/g_part_ldm.c optional geom_part_ldm > >> > files:geom/part/g_part_mbr.c optional geom_part_mbr > >> > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > >> > > >> > which turn on/off which files get included. config "helpfully" converts > >> the > >> > upper case options to lower case for this. > >> > > >> > Warner > >> > > >> > > >> > *slaps forehead* Goose chase! > >> > > >> > I actually knew that... and, at the time, thought it was weird > >> behaviour. > >> > ''grep" would not have failed me if those options would be uppercase > >> there > >> > ... > >> > > >> > >> I've been nibbled to death by these geese often enough to have a PTSD-like > >> reaction when someone mentions it and habitually add -i to my greps... > >> > >> Warner > > > > > > This horrid POLA violation seems to have been in FreeBSD configuration > > since at least 3.0 and probably goes back to the creation of the > > configuration process. > > > > Any idea why such a horrible POLA was ever introduced? Seems like an > > obviously bad idea in an OS that is ALMOST always case sensitive. > > > > It's received code from the old 4.3 BSD config program (or maybe the net-2 > config program). We had best not have any code direct from 4.3 or net-2, it should be from 4.4BSDLite, any code prior to that is subject of the lawsuit. The code is idental in both. There is also a newconfig in 4.4 we didn't adopt... Warner -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-current@freebsd.org Fri Sep 29 01:10:03 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCD5AE12008 for ; Fri, 29 Sep 2017 01:10:03 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 69952744A9 for ; Fri, 29 Sep 2017 01:10:02 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v8T19pOE028899; Thu, 28 Sep 2017 18:09:52 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v8T19oMO028898; Thu, 28 Sep 2017 18:09:50 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? In-Reply-To: To: Warner Losh Date: Thu, 28 Sep 2017 18:09:50 -0700 (PDT) CC: Kevin Oberman , Nick Hibma , FreeBSD Current Mailing List X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Fri, 29 Sep 2017 01:36:12 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 01:10:03 -0000 > On Thu, Sep 28, 2017 at 11:12 AM, Kevin Oberman wrote: > > > On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > > > >> On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > >> wrote: > >> > >> > I created a new kernel config file from scratch, wondered what the > >> > GEOM_PART_MBR option and friends were doing, search for them, didn't > >> find > >> > them in the tree, and deleted them from my config. But... de resulting > >> disk > >> > image didn't boot, because of the fact that it didn't recognise the MBR > >> > partitions (it only had a single diskid entry on the mount-root prompt). > >> > > >> > Can anyone explain to me how these kernel options work, as in: they are > >> > defined in kernel configs and as a consequence in opt_geom.h, but how > >> are > >> > they actually used to select which geom_part_* modules/kernel parts to > >> > build? I thought these options were translated to stuff that cpp would > >> use, > >> > but there are not uses of for example GEOM_PART_MBR anywhere for > >> example! > >> > > >> > > >> > The module always build them because they are listed in the module's > >> > Makefile. > >> > > >> > The kernel only sometimes does. Here's the key lines from conf/files: > >> > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > >> > files:geom/part/g_part_apm.c optional geom_part_apm > >> > files:geom/part/g_part_bsd.c optional geom_part_bsd > >> > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > >> > files:geom/part/g_part_ebr.c optional geom_part_ebr > >> > files:geom/part/g_part_gpt.c optional geom_part_gpt > >> > files:geom/part/g_part_ldm.c optional geom_part_ldm > >> > files:geom/part/g_part_mbr.c optional geom_part_mbr > >> > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > >> > > >> > which turn on/off which files get included. config "helpfully" converts > >> the > >> > upper case options to lower case for this. > >> > > >> > Warner > >> > > >> > > >> > *slaps forehead* Goose chase! > >> > > >> > I actually knew that... and, at the time, thought it was weird > >> behaviour. > >> > ''grep" would not have failed me if those options would be uppercase > >> there > >> > ... > >> > > >> > >> I've been nibbled to death by these geese often enough to have a PTSD-like > >> reaction when someone mentions it and habitually add -i to my greps... > >> > >> Warner > > > > > > This horrid POLA violation seems to have been in FreeBSD configuration > > since at least 3.0 and probably goes back to the creation of the > > configuration process. > > > > Any idea why such a horrible POLA was ever introduced? Seems like an > > obviously bad idea in an OS that is ALMOST always case sensitive. > > > > It's received code from the old 4.3 BSD config program (or maybe the net-2 > config program). We had best not have any code direct from 4.3 or net-2, it should be from 4.4BSDLite, any code prior to that is subject of the lawsuit. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-current@freebsd.org Fri Sep 29 05:54:54 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2920CE2419B for ; Fri, 29 Sep 2017 05:54:54 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ua0-x229.google.com (mail-ua0-x229.google.com [IPv6:2607:f8b0:400c:c08::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CDB1D7F0F3 for ; Fri, 29 Sep 2017 05:54:53 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ua0-x229.google.com with SMTP id y5so161257uai.2 for ; Thu, 28 Sep 2017 22:54:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=9VwfEa0bQeYl/KXxBWJD2nfxB4J0wNCopHX0Zf+buJQ=; b=LaUal1f/llCmH2SJRsj1rDRnEaV2ecw9y27dAXhyCPKZgFqAO3UZaNSMcSUNx4gPeb iD9ddINwJ4uNtAf0NueWeH7NudfZL4oeWoPVBTJlTliR/eIuUfCjcEEed+7XH4Yw+J4P vUm9yYy41KoBgzceCTH8zbvfzd1tfJM0QsPdRedqIX5bAwITUz8gaSb2oviOUYHkaMNN A1kDCRk7piwhUAqgSC3BuuE3dAAGf43XXn/ONSyODAseUW45Km9nMo5sEthL8xDxWTNC a0hZ+MN6SLFIFqJ96a0Kyky4QxfjkFAoyyJzPRREJElX49sW4o+9HDovl5fjQPxR2O52 yYBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=9VwfEa0bQeYl/KXxBWJD2nfxB4J0wNCopHX0Zf+buJQ=; b=AcQR6bDe/nqDE35y98iZgiu/jWkKTXwtGAmBo8GzDBHOCFqQPaMWOqi2Vf8BmJY4H+ Qk0MooekaGbLET95oixpYbGLrRmcuIR9Q1fkzhsG7uxhPesTT0WAeFUtVLV+vQLRY1yU Mzz4e9lDxMWDf3jeXlZW5CvVv299SAbf5ychriy3P8ayFMYhXKFWOVXb/Qf+kzazUQ/K bPjwBw0QkVZdFK19cRor859YGR7hlCK4MmtNp3tv4PFDh6CRGWhCbJFGRTBibUQ5bd1U JXrssH5eWA2Ne8eeRmRDc9nexu06zijuUMmUGfHus8YlImUhuVh+IL4/oY4RsuvyZrLt ndbg== X-Gm-Message-State: AHPjjUhDcNh2ZYq1Lnr1Yo1IKv43BA84OBpHD9UQI4PXj8N364XhtqpI 35WAD1LF7+m5YWQo3ScENPzeyhln//5k/upDomI= X-Google-Smtp-Source: AOwi7QAZcA9c5NcHM5nCxhkVdbR/vSLsM4aL542OxS88Rw6B/uKIe9J0QrFUzGyHqFlnhsg2q9Y+aVV5rdseH13Iz90= X-Received: by 10.176.4.199 with SMTP id 65mr4062624uaw.47.1506664492687; Thu, 28 Sep 2017 22:54:52 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.134.194 with HTTP; Thu, 28 Sep 2017 22:54:52 -0700 (PDT) In-Reply-To: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> References: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> From: Kevin Oberman Date: Thu, 28 Sep 2017 22:54:52 -0700 X-Google-Sender-Auth: I4FBsSNQJjqflrAWP9ab1TMNV-8 Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: "Rodney W. Grimes" Cc: Warner Losh , Nick Hibma , FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 05:54:54 -0000 On Thu, Sep 28, 2017 at 6:09 PM, Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > On Thu, Sep 28, 2017 at 11:12 AM, Kevin Oberman > wrote: > > > > > On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: > > > > > >> On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > > >> wrote: > > >> > > >> > I created a new kernel config file from scratch, wondered what the > > >> > GEOM_PART_MBR option and friends were doing, search for them, didn't > > >> find > > >> > them in the tree, and deleted them from my config. But... de > resulting > > >> disk > > >> > image didn't boot, because of the fact that it didn't recognise the > MBR > > >> > partitions (it only had a single diskid entry on the mount-root > prompt). > > >> > > > >> > Can anyone explain to me how these kernel options work, as in: they > are > > >> > defined in kernel configs and as a consequence in opt_geom.h, but > how > > >> are > > >> > they actually used to select which geom_part_* modules/kernel parts > to > > >> > build? I thought these options were translated to stuff that cpp > would > > >> use, > > >> > but there are not uses of for example GEOM_PART_MBR anywhere for > > >> example! > > >> > > > >> > > > >> > The module always build them because they are listed in the module's > > >> > Makefile. > > >> > > > >> > The kernel only sometimes does. Here's the key lines from > conf/files: > > >> > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd > > >> > files:geom/part/g_part_apm.c optional geom_part_apm > > >> > files:geom/part/g_part_bsd.c optional geom_part_bsd > > >> > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 > > >> > files:geom/part/g_part_ebr.c optional geom_part_ebr > > >> > files:geom/part/g_part_gpt.c optional geom_part_gpt > > >> > files:geom/part/g_part_ldm.c optional geom_part_ldm > > >> > files:geom/part/g_part_mbr.c optional geom_part_mbr > > >> > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 > > >> > > > >> > which turn on/off which files get included. config "helpfully" > converts > > >> the > > >> > upper case options to lower case for this. > > >> > > > >> > Warner > > >> > > > >> > > > >> > *slaps forehead* Goose chase! > > >> > > > >> > I actually knew that... and, at the time, thought it was weird > > >> behaviour. > > >> > ''grep" would not have failed me if those options would be uppercase > > >> there > > >> > ... > > >> > > > >> > > >> I've been nibbled to death by these geese often enough to have a > PTSD-like > > >> reaction when someone mentions it and habitually add -i to my greps... > > >> > > >> Warner > > > > > > > > > This horrid POLA violation seems to have been in FreeBSD configuration > > > since at least 3.0 and probably goes back to the creation of the > > > configuration process. > > > > > > Any idea why such a horrible POLA was ever introduced? Seems like an > > > obviously bad idea in an OS that is ALMOST always case sensitive. > > > > > > > It's received code from the old 4.3 BSD config program (or maybe the > net-2 > > config program). > > We had best not have any code direct from 4.3 or net-2, it should be from > 4.4BSDLite, any code prior to that is subject of the lawsuit. > > > -- > Rod Grimes > rgrimes@freebsd.org > I believe any code in 4.3 BSD that was developed by CSRG was not subject of the suit. Copyright on that code by the Regents of UC, not AT&T. (So was any code I wrote.) Many utilities were from v8 or descended from AT&T code, but a great deal was not. It was what UC was getting paid for. My first Unix experience was with 4.2 BSD and I was not concerned with copyrights as I was working for UC (at least my paychecks said so) and we had an AT&T license for Unix, so it was simply not an issue. The system was a VAX-11/750 at the UC-Davis Department of Applied Science located at Lawrence Livermore Lab. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-current@freebsd.org Fri Sep 29 08:59:21 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65925E27741 for ; Fri, 29 Sep 2017 08:59:21 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDE008398B; Fri, 29 Sep 2017 08:59:20 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LZzY9-1dUbFt2AyD-00lmRc; Fri, 29 Sep 2017 10:59:12 +0200 Date: Fri, 29 Sep 2017 10:59:06 +0200 From: "O. Hartmann" To: Guido Falsi Cc: "O. Hartmann" , freebsd-current Subject: [SOLVED] Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170929105856.5dbecae7@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:rAQjZzEQZXYQkS2bCLNJU0gJV+4n+jwvHLeJv4v4VAfe+s+N4MO +WRdaxLC1GUNUxcnoUwx9q0tUFJpYkMG74wCYXBxEfM4oOdhAfNpTQJDBCdFwrnQX5q2ghQ ZYM9glX+gF7knjTCBLuqYTSIafQmFrUmmvIrmCztlTXAtbsZvNx1er4w0nx9nHLhsowXYEC HfMRh6ujdgve26z90Xu3A== X-UI-Out-Filterresults: notjunk:1;V01:K0:kzM1fZpFOio=:wn/oNqpiNE6xzUVmgneDXW APetoxVrHfRzyGI4g7eZbZpKRoSkNk3thVKj3szMgwm9FIm/PckNI5MP30M2//3yPkhmLDBIR zbf+Lr4GM3PdDpdBe3JFXoj4QrNveIwBN/DPjkcXWdtdgoNTrOrdjjaxAHFVLlueBipzHai1J vVVid7sgtbo9jOdMqKo7BUJN9NhdNXIbNEl5v0r9Ixa44Z6fcge1gI9zJRYGOsaQJQZyL9mSj sWLBgaYx8TeUAkMXSrOP+309sxeB8HAQkxK25W+9TY51x5BV2BEN2bXF77roaIyi2Rfy4WQsO aRodB8/cfTw7DkRDG0lWGNJHpv8z3iwILO8uCkSBRe2+qc4YYjPHGNusHtCiGoo4z2u3lp6R/ 3osgtlq0vJi6G2XGcyL0nNgYV75/1E6DN/VifvzMjo4k2/mn6vRqMJY2vyMhDSoyPlFpOe2v2 DPL5eCldaGG/cXjWr7eF62GVq/Vdqy/8eMNQ9jh5hztU0HMbjuOeX3Ns60nAnwznvKXTvOtSR xDe5vPwh3aIjoe8Lpfrv7hECJF3gagPM2sCfo4SgrP2Pu907rhMpSK90adecMjzeGPbYTe0u9 D5FidSdWtIzKohkXVoFaatIoPM1k/9t2tLiDt7T0PS4+5xWTrj4pRI5y0wkHBmO4jxWSqitkg ne5wFMkUqT1pOmLRqBX2NvhjL9z+0Qu5R36m4DP/FZLpSp/y4gVg9oeu3CuDUu097i59nvV4C M0NKTKTFqoZ7sk/x+PtKJNQo6z0IlhyhcyIQLws9g1C28dL3rbRNb+R5/xrZ7ocEnTjx+33DY zeTsjvLiZUAXaxlNz/tKs50Jf/hy51y1gsXqVjVyrHROjAYhM0= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 08:59:21 -0000 On Thu, 28 Sep 2017 09:38:58 +0200 Guido Falsi wrote: > On 09/28/2017 08:11, O. Hartmann wrote: > > On Wed, 27 Sep 2017 09:05:42 +0200 > > Guido Falsi wrote: > > > >> On 09/26/2017 15:41, O. Hartmann wrote: > >>> On Tue, 26 Sep 2017 15:06:23 +0200 > >>> Guido Falsi wrote: > >> > >>> Since I run net/asterisk with automatic module loading (I'm new to > >>> asterisk), this is very likely and might cause the problem somehow. > >>> > >> > >> You can exclude single modules from autoloading via modules.conf. > >> > >>>> Not sure, restarting the daemon should free any leaked memory the daemon > >>>> has. If a killed process leaves memory locked at the system level there > >>>> should be some other cause. > >>> > >>> Even with no runnidng asterisk, memory level drops after the last shutdown > >>> of asterisk and keeps that low. Even for weeks! My router never shows that > >>> high memory consumption, even under load. > >> > >> But while asterisk is running does the memory usage increase unbounded > >> till filling all available memory or does it stabilize at some point? > > > > While Asterisk is running, it doesn't consume much memory, but stopping > > Asterisk, I would expect that the claimed memory is freed again. It isn't, > > not all memory is freed. Starting Asterisk then again from this reduced > > memory level, it claims its memory, "stabilzes" at a certain point while > > running and again, stopping Asterisk leaves the free memory now at a much > > lower level never been leveld out - as I said. > > > > I played this game last night ~ 20 times until the free memory dropped > > beneath 3 GB after asterisk has been shut down. This morning, the level was > > at the same low level as I left it. The router has nothing special to do, > > the workload is not memory consuming even for weeks! And if there is load, > > after the load went away, the memory consumption always leveld out and > > freed memory. > >> > >> Asterisk is relatively memory hungry, especially with all modules > >> enabled. It also caches and logs various information in RAM, even doing > >> "nothing" it will cache and log that "nothing" activity. If memory does > >> stabilize after some point it's not really a leak but it's standard > >> memory usage. To reduce it you should disable all unused modules. > > > > I don't understand here. Even if Asterisk is memory hungry - it has ~ 3 GB > > to use. But after stopping it, it should free the memory. BUT the system is > > then after the stop with less memory! that is the point. Not the running > > asterisk's memory consumption bothers me, but the fact, that after 20 start > > and stops and waiting for days the memory once gone is never put back. First of all, my questions weren't ment as an offense, more a question of interest by asking thing that I whitnessed. > > VM system is not composed only of "free" and "used" ram, there ar3e > other categories. Depending on how a program allocates and uses memory > it's not automatically sent to the "free" pool after being reclaimed. > > Allocated memory can be dirty buffers which are reclaimed after time, or > other types of buffered data which is never reclaimed until there is > memor7y pressure. How do you know the game you were playing has a > similar memory usage as asterisk, which, I assure you, has some complex > memory usage patterns in it's source. > > Also asterisk leverages many parts of the kernel which a game does not > leverage. I'm quite sure after quitting asterisk you have an higher > "wired" memory than before starting it. That memory was not allocated by > asterisk itself, but by the kernel while "serving" asterisk requests. > The kernel keeps running and does not free that memory right away. in > fact will not free it until forced by VM pressure AFAIK. After a couple of other starts and stops yesterday, it seems, 12-Current and Asterisk 13.17.2 have a barrier at ~ 3 GB of "free" memory - always having in mind that this is within the very specific parameters of my setup. Maybe it would be better first to nail down all the modules in Asterisk not used and switch them off. > > But this is quite a complicated matter and not being a VM expert I can't > explain it in detail. What you must understand is that many things are > going on at a given time in a memory subsystem and looking at it through > a single value is not indicative. You are completely right. > > > > > At the moment, I have mpg123 suspect doing nasty things, because the > > vanishing memory is more prominent and indicated when voicemail system has > > been used and mpg123 started. Not touching VM subsystem seems to free the > > whole memory claimed by asterisk after stopping asterisk, apart from maybe > > buffers claimed by the OS released later (I did no thourough investigations > > on that). > > Please note that when I use the "VM" Acronym I mean "Virtual Memory". > > there is a new version of mpg123 also, and I'm quite sure that mpg123 > causes caching of the audio data it reads. Adsterisk Voicemail system > also causes caching. Those cached are not going to be freed after > program exit because other programs could take advantage of the cached data. > > I'm not understanding what you are expecting us to do based on > circumstantial and partial data. I expect "in circumstantial occurences of some phenomena" nothing. But my observations triggered this explanation - and I'm glad I did and I'm glad you explained! And if it would have indicated something seriously "indicative", I'd have files a PR. > Many thanks! Kind regards, Oliver From owner-freebsd-current@freebsd.org Fri Sep 29 09:18:44 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B42BE2801E for ; Fri, 29 Sep 2017 09:18:44 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 579BC844CE; Fri, 29 Sep 2017 09:18:43 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MXqaN-1dsCNt3C5f-00Wpfg; Fri, 29 Sep 2017 11:18:39 +0200 Date: Fri, 29 Sep 2017 11:18:37 +0200 From: "O. Hartmann" To: "O. Hartmann" Cc: Hans Petter Selasky , Guido Falsi , freebsd-current Subject: [SOLVED] Re: net/asterisk13: memory leak under 12-CURRENT? Message-ID: <20170929111837.79bb7c32@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <20170928075701.23ca8b10@freyja.zeit4.iv.bundesimmobilien.de> References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <31faf367-69e2-b7e3-dd14-67bf69a67ec2@selasky.org> <20170928075701.23ca8b10@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:11CDzZo09CD2eyE4kd1loSOuMkJcpJx1txwfDGrYN0e1ZdolaKZ NukyKMukwuruxgHREO4dURh7+VFWdvGu2hFDepThZtlTXn1eXyOJww0t96aOmdEOUW7av5l rIREb+yPDRtty59L3AjfYxRX07PgFC01bDP22WreYmJ6hUJRYn68pz9sHjSypia7R57xFQZ 5c4m4E/pF1cndq4wBHFEA== X-UI-Out-Filterresults: notjunk:1;V01:K0:UtEYn3p7rz0=:guooH+oQKs/xCn7l1bV1fY N66H8GIW/HW91DL0C6tYt+vG0xogQ33s7VnHlPdw2dLn7nFwp+hEN91ZKce8zOxBEXBVkqmfG 3zD0Im4eYh80VYOCR/yCjQqtBLs2F7mLxfoUeCMuqcCS96a1KnHwjqf9nP3utdJ237aEgv9YP UYXiBIushEoD3CZkxPMZDRGq8WVxzJP71G2qLHkar76xODb1HDOYvlXxIU27uMEyXQv4seCJ7 +7MqRgip5vQJP5wqRHC+HfJTaSsU6IxxapNsM/jsxh+rYuDp6KOsg0ZXbtpoAixnsYsamg9Ib ft+E1teHmxzl1OH3b0QXqctyDiCkuqKi+wcVROw55T/wGCwRz3uVpVacrQU+vLm0og+M4NgRY wJu4L5MXzbkTm5HSDAiXp+xFhd2zZjfaeyeSzHrZevuCF49LSb2+5SlplGCBtDV7ZZ7clS8Bc h/w0JBRVPW9ZnWCEGBCXylECeu3fah+Co+1FPrTJfEoWSYGvXGwrOEhx0gAhBbUIrSV+L2eE6 wTHAKrr4L6hbbzXKmmxRRRZmDyPElPN9MfRmCbk7ls47c4TaKGCPhvxyxVHtBVfa5BBSzCjfV hEL98yyThZkANjXWpp0XX67LEoZJVReqmoiiGr5a7pKAitGU8CVFQplHmViFx8rRT3eIuOdIC OuLfnKZRB6WtODC1o56Vo/GWV/spv1263gKCsejUtZ1/LMPCRd4HzvFty0sHkGIVx0G9iD/a5 Rqx9dXv/UV6gcIMKNWPF+F1I+LXq8qmsT0CevvVdTn2utzGDFkp4HCxJ6tleVGy7HQC20w4od rTREPRrbEc7s88Niriho2Exuz13+MwKdHZn9eAZoipdNW/bfjA= X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 09:18:44 -0000 On Thu, 28 Sep 2017 07:57:01 +0200 "O. Hartmann" wrote: > On Wed, 27 Sep 2017 12:51:18 +0200 > Hans Petter Selasky wrote: > > > On 09/27/17 09:05, Guido Falsi wrote: > > > On 09/26/2017 15:41, O. Hartmann wrote: > > >> On Tue, 26 Sep 2017 15:06:23 +0200 > > >> Guido Falsi wrote: > > > > > >> Since I run net/asterisk with automatic module loading (I'm new to > > >> asterisk), this is very likely and might cause the problem somehow. > > >> > > > > > > You can exclude single modules from autoloading via modules.conf. > > > > > >>> Not sure, restarting the daemon should free any leaked memory the daemon > > >>> has. If a killed process leaves memory locked at the system level there > > >>> should be some other cause. > > >> > > >> Even with no runnidng asterisk, memory level drops after the last > > >> shutdown of asterisk and keeps that low. Even for weeks! My router never > > >> shows that high memory consumption, even under load. > > > > > > But while asterisk is running does the memory usage increase unbounded > > > till filling all available memory or does it stabilize at some point? > > > > > > Asterisk is relatively memory hungry, especially with all modules > > > enabled. It also caches and logs various information in RAM, even doing > > > "nothing" it will cache and log that "nothing" activity. If memory does > > > stabilize after some point it's not really a leak but it's standard > > > memory usage. To reduce it you should disable all unused modules. > > > > > >> > > >> The question would be: how to use vmstat to give hints for those familiar > > >> with memory subsystems to indicate a real bug? > > >> > > >> I tried to find some advices, but maybe my English isn't good enough to > > >> make google help. > > > > > > I'm not able to give you a correct indication, but if the memory usage > > > is not increasing indefinitely but is stabilizing I'd say it's not > > > really a leak. > > > > > > > Did you look at the output from "vmstat -m" and "vmstat -z" ? > > > > --HPS > > I did not, but now I will ;-) > > Thanks for the hint! > > Kind regards, > Oliver For the completeness. Guido already explained that things I considered "stange' could be normal. I think I missed the point, that in some cases, caching data can also persist even when the process, which I considered to be the consumer, died. The vmstat output is taken after a couple of restarts of asterisk and while asterisk was still running. I had a very simplistic, outdated memory model in mind and Guido made that clear that it isn't. I was concerned about if it comes to the use of a small SoC with only 1 or 2 GB of RAM. Sorry for the noise :-( Kind regards, Oliver [...] last pid: 19557; load averages: 0.22, 0.18, 0.14 up 1+14:17:56 11:14:44 25 processes: 1 running, 24 sleeping CPU: 0.3% user, 0.0% nice, 0.8% system, 0.0% interrupt, 98.9% idle Mem: 27M Active, 104M Inact, 392M Wired, 146M Buf, 3421M Free Swap: 8192M Total, 8192M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 36617 asterisk 70 20 0 147M 74236K piperd 2 3:38 2.42% asterisk 19557 root 1 20 0 13180K 3508K CPU3 3 0:00 0.13% top 93400 root 1 20 0 11444K 2100K select 3 1:34 0.06% syslogd 12569 root 1 20 0 15136K 3576K select 1 4:35 0.04% ppp 88182 administrator 1 20 0 18768K 7980K select 0 0:00 0.02% sshd 24901 root 1 20 0 10840K 1704K select 1 0:19 0.01% powerd 17169 root 1 20 0 18192K 18296K select 2 0:10 0.01% ntpd 10719 root 1 20 0 14724K 4712K bpf 1 0:03 0.00% arpwatch 9691 bind 7 52 0 64552K 38192K sigwai 3 0:19 0.00% named 98424 root 1 20 0 32100K 20948K nanslp 2 0:05 0.00% perl 39431 asterisk 1 52 0 13848K 5072K select 0 0:01 0.00% mpg123 73938 root 1 20 0 11332K 2036K nanslp 3 0:01 0.00% cron 45343 daemon 1 20 0 11316K 1940K select 2 0:00 0.00% rpcbind 41607 root 1 20 0 10976K 1848K kqread 1 0:00 0.00% autounmountd 82722 root 1 22 0 18396K 7920K select 1 0:00 0.00% sshd 97178 root 1 21 0 16216K 6780K select 2 0:00 0.00% sudo 97625 root 1 20 0 13604K 4356K pause 2 0:00 0.00% csh 46309 root 1 20 0 10880K 1784K autofs 3 0:00 0.00% automountd 90914 administrator 1 20 0 13172K 4184K pause 0 0:00 0.00% csh 40858 asterisk 1 52 0 13616K 3040K pipewr 3 0:00 0.00% mpg123 97392 root 1 23 0 11776K 3096K wait 3 0:00 0.00% su 77979 dhcpd 1 20 0 21896K 8340K select 0 0:00 0.00% dhcpd 69003 root 1 20 0 18000K 4904K select 0 0:00 0.00% sshd 52102 root 1 52 0 10880K 1788K ttyin 2 0:00 0.00% getty 97105 root 1 20 0 10868K 2444K ttyin 3 0:00 0.00% getty [vmstat -m] root@gate:~ # vmstat -m Type InUse MemUse HighUse Requests Size(s) acpiintr 1 1K - 1 64 acpica 2689 269K - 53492 16,32,64,128,256,512,1024 vtbuf 8 656K - 14 4096 vt 3 2K - 3 512 autofs 6 9K - 8219485 16,128,8192 acpitask 1 64K - 1 65536 acpisem 24 3K - 24 128 acpidev 32 2K - 32 64 CAM SIM 3 1K - 3 256 CAM XPT 18 2K - 112 32,64,128,256,512,1024,2048,65536 CAM DEV 4 8K - 10 2048 DEVFS3 140 35K - 520 256 DEVFS1 109 55K - 349 512 DEVFS_RULE 55 26K - 55 64,512 DEVFS 11 1K - 12 16,128 DEVFSP 7 1K - 7 64 NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 pfs_nodes 20 5K - 20 256 tmpfs mount 2 1K - 2 128 tmpfs name 1500 30K - 2098 16,32,64 eli data 7 2K - 29 64,256,512,1024 CAM CCB 0 0K - 117024 2048 geom_flashmap 0 0K - 24 256 GEOM 278 42K - 12066 16,32,64,128,256,512,1024,2048,4096,8192,16384 raid_data 0 0K - 2088 32,128,256 isadev 6 1K - 6 128 CAM path 5 1K - 38 32 CAM periph 4 1K - 19 16,32,64,128,256 acpi_perf 4 1K - 4 128 cdev 4 1K - 4 256 filedesc 7 85K - 276 128,4096,65536 sigio 0 0K - 1 64 filecaps 0 0K - 25082 16,32,64 kenv 119 12K - 122 16,32,64,128,8192 kqueue 61 17K - 16840 64,256,512,2048,8192 proc-args 56 3K - 29790 16,32,64,128,256 hhook 13 3K - 13 32,256 ithread 130 26K - 130 32,128,256 linker 127 9K - 129 16,32,64,512 lockf 23 3K - 5019 64,128 loginclass 3 1K - 3 64 cache 1 1K - 1 32 devbuf 16929 33991K - 17350 16,32,64,128,256,512,1024,2048,4096,8192 temp 75 5K - 79184 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 CAM I/O Scheduler 1 1K - 1 1024 module 327 41K - 327 128 mtx_pool 2 16K - 2 8192 osd 3 1K - 9 16,32,64,128,256 pmchooks 1 1K - 1 128 pgrp 23 3K - 922 128 session 19 3K - 359 128 proc 2 32K - 2 16384 subproc 162 267K - 16726 512,4096 cred 111 28K - 16500 256 kbdmux 5 22K - 5 512,1024,2048,16384 plimit 22 6K - 5489 256 uidinfo 9 5K - 287 128,4096 sysctl 0 0K - 21212 16,32,64 sysctloid 5895 303K - 5975 16,32,64,128 sysctltmp 0 0K - 1183 16,32,64,256,1024 tidhash 1 32K - 1 32768 callout 5 2184K - 5 umtx 560 70K - 560 128 p1003.1b 1 1K - 1 16 SWAP 2 549K - 2 32 bus 2646 191K - 5020 16,32,64,128,256,1024 bus-sc 61 1729K - 1214 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 devstat 16 33K - 16 32,4096 eventhandler 107 9K - 107 64,128 gtaskqueue 30 27K - 30 16,32,256,8192 kobj 131 524K - 3199 4096 Per-cpu 1 1K - 1 32 rman 164 19K - 553 32,128 sbuf 1 1K - 9894 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 toponodes 14 2K - 14 128 taskqueue 36 4K - 48 16,32,256 terminal 3 1K - 3 256 Unitno 41 3K - 7477 32,64 vmem 3 200K - 5 4096,8192,65536 ioctlops 1 8K - 297 256,1024,8192 select 181 23K - 280 128,8192 iov 0 0K - 2135314 16,64,128,256 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 1 32K - 1 32768 tty 7 7K - 10 1024 pts 1 1K - 4 256 mbuf_tag 175 11K - 84049978 32,64,256 shmfd 1 8K - 1 8192 soname 4 1K - 2413713 16,32,128 pcb 37 663K - 33716 16,32,64,128,1024,2048,8192 acl 0 0K - 3696 4096 vfscache 4 2097K - 4 256,16384,32768 cl_savebuf 0 0K - 956 64,128,256 vfs_hash 1 1024K - 1 vnodes 1 1K - 1 256 mount 98 5K - 539 16,32,64,128,256 statfs 0 0K - 32609 4096 vnodemarker 0 0K - 33467 512 GSS-API 1 1K - 1 64 chacha20random 1 1K - 1 1024 iconv 2 1K - 2 128 BPF 42 1079K - 43 16,64,128,512,4096 BPF_JIT 7 1K - 15 16,32,64 ifnet 15 29K - 15 128,2048 ifaddr 69 13K - 70 16,32,256,512,4096 ether_multi 36 2K - 36 16,64 clone 17 3K - 17 128 ipsec 1 1K - 1 256 lltable 38 12K - 624 256,512 sppp 1 2K - 1 2048 tun 1 1K - 1 256 vlan 14 2K - 14 64,128 entropy 0 0K - 15571 4096 iflib 132 816K - 144 32,64,128,1024,2048,8192,16384,32768 routetbl 96 19K - 123 32,128,256,512,4096 vnet 1 1K - 1 64 vnet_data 1 64K - 1 65536 vnet_data_free 1 1K - 1 32 netgraph 9 1K - 9 64 netgraph_msg 0 0K - 22 64,128,256,512,1024 netgraph_hook 4 1K - 6 128 netgraph_node 14 4K - 17 128,256 netgraph_pppoe 2 17K - 5 64,512,16384 netgraph_path 0 0K - 9 16 netgraph_sock 3 1K - 6 32,128 igmp 14 2K - 14 128 ipid 2 24K - 2 8192,16384 in_mfilter 1 1K - 1 1024 in_multi 8 2K - 8 256 ip_moptions 2 1K - 2 64,256 encap_export_host 2 2K - 2 1024 mroutetbl 1 1K - 1 256 sctp_a_it 0 0K - 8 16 sctp_vrf 1 1K - 1 64 sctp_ifa 8 1K - 9 128 sctp_ifn 8 1K - 8 128 sctp_iter 0 0K - 8 256 hostcache 1 32K - 1 32768 LRO 24 480K - 24 8192,32768 tcpfunc 1 1K - 1 64 syncache 1 68K - 1 libalias 66 136K - 31040 128,65536 sctpnat 6 80K - 6 8192,16384 inpcbpolicy 32 1K - 37745 32 secasvar 1 1K - 1 1024 sahead 1 1K - 1 1024 ipsecpolicy 2 2K - 2 256,1024 ipsec-saq 2 2K - 2 1024 dummynet 4 4K - 4 512,1024 dummynet 13 6K - 278817 256,512 UART 6 5K - 6 16,1024 IpFw/IpAcct 96 59K - 163 16,32,64,128,256,512,1024,2048,4096,8192,16384 ipfw_tbl 10 2K - 10 128 crypto 1 1K - 1 512 xform 0 0K - 986 16,32,256 rpc 2 8K - 2 4096 audit_evclass 230 8K - 286 32 mactemp 0 0K - 194 8192 pagedep 2 256K - 416 256 inodedep 2 2048K - 4035 512 bmsafemap 2 16K - 3283 256,8192 newblk 2 4096K - 5152 256 indirdep 0 0K - 2368 128,32768 freefrag 0 0K - 183 128 freeblks 0 0K - 367 256 freefile 0 0K - 368 64 diradd 0 0K - 710 128 mkdir 0 0K - 6 128 dirrem 0 0K - 706 128 newdirblk 0 0K - 3 64 freework 2 1K - 772 16,128 freedep 0 0K - 4 64 jaddref 0 0K - 693 128 jremref 0 0K - 684 128 jmvref 0 0K - 42 128 jnewblk 0 0K - 5097 128 jfreefrag 0 0K - 181 128 jseg 0 0K - 4829 128 jsegdep 0 0K - 6655 64 sbdep 0 0K - 1182 64 savedino 0 0K - 1317 256 jblocks 4 1K - 12 128,256 softdep 2 1K - 27 512 ufs_dirhash 213 44K - 243 16,32,64,128,256,512,1024 ufs_quota 1 1024K - 1 ufs_mount 9 30K - 87 512,2048,4096,8192 vm_pgdata 1 1K - 1 128 UMAHash 7 38K - 18 512,1024,2048,4096,8192,16384,32768 CAM queue 7 3K - 26 16,32,512 USB 20 23K - 22 16,128,512,4096 fpukern_ctx 4 4K - 4 1024 memdesc 1 4K - 1 4096 aesni_data 7 6K - 7 32,256,1024 pci_link 12 1K - 12 32,128 cpuctl 1 1K - 1 32 USBdev 15 2K - 15 32,64,128,256 apmdev 1 1K - 1 128 madt_table 0 0K - 2 32,4096 CAM dev queue 3 1K - 3 64 io_apic 2 6K - 2 2048,4096 local_apic 1 4K - 1 4096 MCA 11 2K - 11 32,128 cpus 2 1K - 2 16 msi 16 2K - 16 128 nexusdev 6 1K - 6 16 root@gate:~ # [vmstat -z] root@gate:~ # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 256, 0, 221, 4, 221, 0, 0 UMA Zones: 704, 0, 222, 3, 222, 0, 0 UMA Slabs: 80, 0, 5287, 13, 5512, 0, 0 UMA Hash: 256, 0, 5, 10, 12, 0, 0 4 Bucket: 32, 0, 221, 904, 2102, 0, 0 6 Bucket: 48, 0, 23, 1056, 570, 0, 0 8 Bucket: 64, 0, 42, 1260, 37128, 11, 0 12 Bucket: 96, 0, 74, 705, 469791, 0, 0 16 Bucket: 128, 0, 126, 1207, 70315, 0, 0 32 Bucket: 256, 0, 198, 672, 27016, 135, 0 64 Bucket: 512, 0, 207, 425, 4310, 167, 0 128 Bucket: 1024, 0, 178, 318, 160115, 0, 0 256 Bucket: 2048, 0, 278, 38, 94068, 0, 0 vmem btag: 56, 0, 15673, 586, 17122, 115, 0 VM OBJECT: 240, 0, 7368, 696, 194473, 0, 0 RADIX NODE: 144, 0, 9059, 1147, 409047, 0, 0 MAP: 232, 0, 3, 65, 3, 0, 0 KMAP ENTRY: 120, 0, 5, 259, 5, 0, 0 MAP ENTRY: 120, 0, 2904, 2607, 808977, 0, 0 VMSPACE: 2496, 0, 26, 28, 16590, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 16400, 0, 397, 0, 397, 0, 0 16: 16, 0, 23, 1232, 36289, 0, 0 16: 16, 0, 1211, 797, 25573, 0, 0 16: 16, 0, 25, 1230, 34, 0, 0 16: 16, 0, 222, 1033, 8220542, 0, 0 16: 16, 0, 1905, 856, 20512, 0, 0 16: 16, 0, 122, 1886, 2411743, 0, 0 16: 16, 0, 92, 1163, 70381, 0, 0 16: 16, 0, 0, 1255, 385, 0, 0 32: 32, 0, 105, 1270, 38699, 0, 0 32: 32, 0, 460, 665, 4210, 0, 0 32: 32, 0, 59, 1316, 33644, 0, 0 32: 32, 0, 1107, 768, 27259, 0, 0 32: 32, 0, 2444, 681, 5616, 0, 0 32: 32, 0, 66, 1059, 719, 0, 0 32: 32, 0, 182, 943, 1510, 0, 0 32: 32, 0, 11, 1114, 7866, 0, 0 64: 64, 0, 84, 970, 86, 0, 0 64: 64, 0, 919, 879, 19134, 0, 0 64: 64, 0, 207, 847, 1124, 0, 0 64: 64, 0, 688, 614, 1734, 0, 0 64: 64, 0, 351, 1199, 1545, 0, 0 64: 64, 0, 136, 1414, 24244, 0, 0 64: 64, 0, 8325, 789, 362569, 0, 0 64: 64, 0, 211, 1339,20905924, 0, 0 128: 128, 0, 662, 857, 23856, 0, 0 128: 128, 0, 1690, 573, 33432, 0, 0 128: 128, 0, 37, 490, 85, 0, 0 128: 128, 0, 835, 1304, 37383, 0, 0 128: 128, 0, 1749, 762, 3792, 0, 0 128: 128, 0, 392, 755, 10647, 0, 0 128: 128, 0, 343, 432, 6647, 0, 0 128: 128, 0, 33, 742, 5036, 0, 0 256: 256, 0, 82, 173, 1413, 0, 0 256: 256, 0, 19, 356, 2226, 0, 0 256: 256, 0, 28, 227, 30, 0, 0 256: 256, 0, 250, 365, 6275, 0, 0 256: 256, 0, 181, 314, 24437, 0, 0 256: 256, 0, 137, 358, 5862, 0, 0 256: 256, 0, 106, 329, 1713374, 0, 0 256: 256, 0, 32, 403,63057061, 0, 0 512: 512, 0, 1, 319, 4017, 0, 0 512: 512, 0, 114, 118, 369, 0, 0 512: 512, 0, 121, 55, 122, 0, 0 512: 512, 0, 76, 76, 143, 0, 0 512: 512, 0, 27, 613, 279136, 0, 0 512: 512, 0, 56, 96, 3584, 0, 0 512: 512, 0, 41, 111, 100, 0, 0 512: 512, 0, 4, 148, 33451, 0, 0 1024: 1024, 0, 1, 51, 16875, 0, 0 1024: 1024, 0, 18, 34, 909, 0, 0 1024: 1024, 0, 5, 11, 5, 0, 0 1024: 1024, 0, 13, 143, 600, 0, 0 1024: 1024, 0, 48, 44, 152, 0, 0 1024: 1024, 0, 13, 55, 922, 0, 0 1024: 1024, 0, 19, 49, 28, 0, 0 1024: 1024, 0, 0, 28, 2, 0, 0 2048: 2048, 0, 4, 22, 1622, 0, 0 2048: 2048, 0, 1, 9, 2, 0, 0 2048: 2048, 0, 1, 5, 1, 0, 0 2048: 2048, 0, 2, 4, 25, 0, 0 2048: 2048, 0, 3, 3, 3, 0, 0 2048: 2048, 0, 7, 33, 117156, 0, 0 2048: 2048, 0, 31, 15, 37, 0, 0 2048: 2048, 0, 4, 22, 48, 0, 0 4096: 4096, 0, 1, 2, 9, 0, 0 4096: 4096, 0, 0, 1, 1, 0, 0 4096: 4096, 0, 194, 16, 19831, 0, 0 4096: 4096, 0, 9, 8, 32834, 0, 0 4096: 4096, 0, 2, 7, 19229, 0, 0 4096: 4096, 0, 3, 3, 18, 0, 0 4096: 4096, 0, 28, 3, 34, 0, 0 4096: 4096, 0, 4, 1, 8, 0, 0 8192: 8192, 0, 0, 1, 1, 0, 0 8192: 8192, 0, 0, 1, 1, 0, 0 8192: 8192, 0, 2, 0, 2, 0, 0 8192: 8192, 0, 4, 8, 340, 0, 0 8192: 8192, 0, 41, 5, 140, 0, 0 8192: 8192, 0, 8, 6, 788, 0, 0 8192: 8192, 0, 3, 3, 9, 0, 0 8192: 8192, 0, 15, 1, 19, 0, 0 16384: 16384, 0, 2, 2, 11, 0, 0 16384: 16384, 0, 0, 1, 1, 0, 0 16384: 16384, 0, 0, 0, 0, 0, 0 16384: 16384, 0, 2, 1, 97, 0, 0 16384: 16384, 0, 11, 0, 11, 0, 0 16384: 16384, 0, 1, 9, 164, 0, 0 16384: 16384, 0, 1, 6, 13, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 0, 4, 8, 0, 0 32768: 32768, 0, 2, 0, 2, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 13, 0, 13, 0, 0 32768: 32768, 0, 2, 0, 2, 0, 0 32768: 32768, 0, 0, 6, 30, 0, 0 32768: 32768, 0, 12, 0, 12, 0, 0 65536: 65536, 0, 0, 6, 2062, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 6, 1, 104, 0, 0 65536: 65536, 0, 1, 0, 1, 0, 0 65536: 65536, 0, 1, 2, 13, 0, 0 65536: 65536, 0, 0, 4, 10, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 SLEEPQUEUE: 80, 0, 281, 246, 281, 0, 0 64 pcpu: 8, 0, 1842, 462, 1846, 0, 0 Files: 80, 0, 149, 751, 264053, 0, 0 filedesc0: 1104, 0, 53, 64, 16616, 0, 0 rl_entry: 40, 0, 168, 932, 168, 0, 0 TURNSTILE: 136, 0, 281, 159, 281, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 52353, 1347, 200398, 0, 0 mac_mls: 112, 0, 52528, 1372,21106277, 0, 0 mac_biba: 112, 0, 52528, 847,21106277, 0, 0 PROC: 1344, 0, 52, 56, 16615, 0, 0 THREAD: 1360, 0, 239, 41, 3200, 0, 0 cpuset: 96, 0, 84, 490, 121, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 1615545, 22, 1243, 1054258, 0, 0 mbuf: 256, 1615545, 6228, 1627,20476925, 0, 0 mbuf_cluster: 2048, 757350, 7309, 109, 7108913, 0, 0 mbuf_jumbo_page: 4096, 126213, 0, 46, 300, 0, 0 mbuf_jumbo_9k: 9216, 37396, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 21035, 0, 0, 0, 0, 0 NetGraph items: 72, 4123, 0, 279, 26, 0, 0 NetGraph data items: 72, 4123, 0, 527, 1775955, 0, 0 g_bio: 376, 0, 0, 1720, 600366, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 255, 195, 1410, 0, 0 ttyoutq: 256, 0, 132, 303, 730, 0, 0 nvme_request: 128, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 FPU_save_area: 832, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 51606, 282, 64870, 0, 0 VNODEPOLL: 120, 0, 4, 392, 5, 0, 0 BUF TRIE: 144, 0, 3578, 23098, 28233, 0, 0 NAMEI: 1024, 0, 0, 80, 9028599, 0, 0 rentr: 24, 0, 0, 0, 0, 0, 0 S VFS Cache: 108, 0, 51645, 855, 75910, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 1418, 94, 1538, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 592, 0, 0, 0, 0, 0, 0 nandfs node zone: 312, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 398, 46, 519, 0, 0 fuse_ticket: 224, 0, 0, 0, 0, 0, 0 autofs_request: 5280, 0, 0, 7, 74, 0, 0 autofs_node: 208, 0, 4, 129, 4, 0, 0 Mountpoints: 2744, 0, 7, 14, 33, 0, 0 AIO: 224, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0 AIOCB: 1520, 0, 0, 0, 0, 0, 0 AIOL: 128, 0, 0, 0, 0, 0, 0 AIOLIO: 280, 0, 0, 0, 0, 0, 0 pipe: 760, 0, 8, 72, 2352, 0, 0 procdesc: 136, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 94, 956, 416385, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 bridge_rtnode: 64, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 20, 305, 83900, 0, 0 socket: 856, 129972, 53, 79, 38782, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 ipq: 56, 23714, 0, 781, 29, 0, 0 udp_inpcb: 424, 129978, 20, 160, 37423, 0, 0 udpcb: 32, 130000, 20, 1355, 37423, 0, 0 tcp_inpcb: 424, 129978, 11, 106, 138, 0, 0 tcpcb: 920, 129972, 11, 41, 138, 0, 0 tcptw: 88, 26010, 0, 540, 55, 0, 0 syncache: 168, 15364, 0, 253, 6, 0, 0 hostcache: 96, 15375, 0, 287, 7, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 tcpreass: 40, 47400, 0, 0, 0, 0, 0 sctp_ep: 1208, 129972, 0, 0, 0, 0, 0 sctp_asoc: 2400, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 830, 9, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 152, 400010, 0, 0, 0, 0, 0 sctp_readq: 152, 400010, 0, 0, 0, 0, 0 sctp_stream_msg_out: 112, 400015, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 424, 129978, 0, 0, 0, 0, 0 ripcb: 424, 129978, 1, 116, 88, 0, 0 unpcb: 240, 129984, 17, 239, 1114, 0, 0 rtentry: 208, 0, 56, 191, 70, 0, 0 IPFW counters: 16, 0, 57, 967, 57, 0, 0 IPFW dynamic rule: 128, 16399, 12, 1631, 48491, 0, 0 divcb: 424, 129978, 0, 0, 0, 0, 0 selfd: 64, 0, 233, 1317,23093991, 0, 0 swpctrie: 144, 504873, 0, 0, 0, 0, 0 swblk: 136, 504861, 0, 0, 0, 0, 0 FFS inode: 160, 0, 50104, 421, 62783, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 50104, 416, 62782, 0, 0 TMPFS dirent: 64, 0, 1298, 500, 1298, 0, 0 TMPFS node: 240, 0, 1299, 173, 1299, 0, 0 TMPFS dirent: 64, 0, 158, 896, 716, 0, 0 TMPFS node: 240, 0, 159, 273, 713, 0, 0 From owner-freebsd-current@freebsd.org Fri Sep 29 09:58:54 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 475E8E28B52 for ; Fri, 29 Sep 2017 09:58:54 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E68C89D5 for ; Fri, 29 Sep 2017 09:58:52 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3y3RpX2yPBzZr7; Fri, 29 Sep 2017 11:58:44 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id IkYDDNp8EO3n; Fri, 29 Sep 2017 11:58:42 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Fri, 29 Sep 2017 11:58:42 +0200 (CEST) Subject: Re: [SOLVED] Re: net/asterisk13: memory leak under 12-CURRENT? To: "O. Hartmann" Cc: freebsd-current References: <20170926144522.21e59cfe@freyja.zeit4.iv.bundesimmobilien.de> <979b6cfe-0e38-5df3-7bb5-cdb8de6677bf@FreeBSD.org> <20170926154155.28deb2e1@freyja.zeit4.iv.bundesimmobilien.de> <20170928081152.1b2f039c@freyja.zeit4.iv.bundesimmobilien.de> <20170929105856.5dbecae7@freyja.zeit4.iv.bundesimmobilien.de> From: Guido Falsi Message-ID: Date: Fri, 29 Sep 2017 11:58:42 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170929105856.5dbecae7@freyja.zeit4.iv.bundesimmobilien.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 09:58:54 -0000 On 09/29/2017 10:59, O. Hartmann wrote: > On Thu, 28 Sep 2017 09:38:58 +0200 > Guido Falsi wrote: > >> On 09/28/2017 08:11, O. Hartmann wrote: >>> On Wed, 27 Sep 2017 09:05:42 +0200 >>> Guido Falsi wrote: >>> >>>> On 09/26/2017 15:41, O. Hartmann wrote: >>>>> On Tue, 26 Sep 2017 15:06:23 +0200 >>>>> Guido Falsi wrote: >>>> >>>>> Since I run net/asterisk with automatic module loading (I'm new to >>>>> asterisk), this is very likely and might cause the problem somehow. >>>>> >>>> >>>> You can exclude single modules from autoloading via modules.conf. >>>> >>>>>> Not sure, restarting the daemon should free any leaked memory the daemon >>>>>> has. If a killed process leaves memory locked at the system level there >>>>>> should be some other cause. >>>>> >>>>> Even with no runnidng asterisk, memory level drops after the last shutdown >>>>> of asterisk and keeps that low. Even for weeks! My router never shows that >>>>> high memory consumption, even under load. >>>> >>>> But while asterisk is running does the memory usage increase unbounded >>>> till filling all available memory or does it stabilize at some point? >>> >>> While Asterisk is running, it doesn't consume much memory, but stopping >>> Asterisk, I would expect that the claimed memory is freed again. It isn't, >>> not all memory is freed. Starting Asterisk then again from this reduced >>> memory level, it claims its memory, "stabilzes" at a certain point while >>> running and again, stopping Asterisk leaves the free memory now at a much >>> lower level never been leveld out - as I said. >>> >>> I played this game last night ~ 20 times until the free memory dropped >>> beneath 3 GB after asterisk has been shut down. This morning, the level was >>> at the same low level as I left it. The router has nothing special to do, >>> the workload is not memory consuming even for weeks! And if there is load, >>> after the load went away, the memory consumption always leveld out and >>> freed memory. >>>> >>>> Asterisk is relatively memory hungry, especially with all modules >>>> enabled. It also caches and logs various information in RAM, even doing >>>> "nothing" it will cache and log that "nothing" activity. If memory does >>>> stabilize after some point it's not really a leak but it's standard >>>> memory usage. To reduce it you should disable all unused modules. >>> >>> I don't understand here. Even if Asterisk is memory hungry - it has ~ 3 GB >>> to use. But after stopping it, it should free the memory. BUT the system is >>> then after the stop with less memory! that is the point. Not the running >>> asterisk's memory consumption bothers me, but the fact, that after 20 start >>> and stops and waiting for days the memory once gone is never put back. > > First of all, my questions weren't ment as an offense, more a question of > interest by asking thing that I whitnessed. If I sounded harsh or not polite, sorry, I did not mean to do that! I was trying to get a message through, but did not feel like I was succeeding and that was frustrating me. So sorry for any lack of respect I could have shown. > >> >> VM system is not composed only of "free" and "used" ram, there ar3e >> other categories. Depending on how a program allocates and uses memory >> it's not automatically sent to the "free" pool after being reclaimed. >> >> Allocated memory can be dirty buffers which are reclaimed after time, or >> other types of buffered data which is never reclaimed until there is >> memor7y pressure. How do you know the game you were playing has a >> similar memory usage as asterisk, which, I assure you, has some complex >> memory usage patterns in it's source. >> >> Also asterisk leverages many parts of the kernel which a game does not >> leverage. I'm quite sure after quitting asterisk you have an higher >> "wired" memory than before starting it. That memory was not allocated by >> asterisk itself, but by the kernel while "serving" asterisk requests. >> The kernel keeps running and does not free that memory right away. in >> fact will not free it until forced by VM pressure AFAIK. > > After a couple of other starts and stops yesterday, it seems, 12-Current and > Asterisk 13.17.2 have a barrier at ~ 3 GB of "free" memory - always having in > mind that this is within the very specific parameters of my setup. It's quite possible that on a system with less RAM (for example an older ALIX board) these extra buffers allocated while asterisk was running get freed earlier because memory pressure is felt sooner. > I expect "in circumstantial occurences of some phenomena" nothing. But my > observations triggered this explanation - and I'm glad I did and I'm glad you > explained! I'm glad I was useful! -- Guido Falsi From owner-freebsd-current@freebsd.org Fri Sep 29 07:35:48 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE30BE25F10; Fri, 29 Sep 2017 07:35:48 +0000 (UTC) (envelope-from tommi.pernila@gmail.com) Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com [209.85.220.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73D3181A09; Fri, 29 Sep 2017 07:35:48 +0000 (UTC) (envelope-from tommi.pernila@gmail.com) Received: by mail-qk0-f180.google.com with SMTP id b82so404289qkc.4; Fri, 29 Sep 2017 00:35:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dWux4/4p1Ldwmw9ktqlrmwpRV1cCoje+y3D+2Xry2aw=; b=AHTMM3nDZ/sWiC3FxCzjRqp3ILv8wMz5f3BThkJ17b944aRNLnv4dEwK5IAbh4TfTN OCONKZE2hkweZsLEfHh+pnyJUNYVlRygpdOrPBEfGiZB/Pbc4L/vHo1kByOUyyx9C/TI IUSpxaIexVj209ErdevzdfyL3pS/VVJfGCPEwBrgehxJauNROo5UkGtIT0IECOwI//Lb 06VmfuEDIivZj0icWRMuvYxymP2xnF58UtqNr6x0q/xlWeGm7ZxrdNnTGnntx1qBxwAg ta6mN0EVxwiu+QCt0fR1ksEGmhdyJHnxBemv3+7wRN3yJsNcDRAsFT2vTT0tgJDwqiHT CujA== X-Gm-Message-State: AMCzsaUHWHzVdVDkv/VOLZ26vTz+tG13ZLL5tX2wCb+9F1D3WXF2XdlQ KH9a+OMQ74oMP+eWaABspSXdhiIT7y0TPjfz0F02pg== X-Google-Smtp-Source: AOwi7QBvx7v3PL1Nf0jNh65KzCCgC4YDiJ6fpFt26zmjjrzyPyA7WDXgRcwuGrVa/sfOsTcSb5FfWYJA77rcYHtRc0U= X-Received: by 10.55.159.88 with SMTP id i85mr1872409qke.350.1506666959043; Thu, 28 Sep 2017 23:35:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Tommi Pernila Date: Fri, 29 Sep 2017 06:35:48 +0000 Message-ID: Subject: Re: Cannot install FreeBSD 12.0 on HP Compaq DC5850 To: Elena Mihailescu , freebsd-current Cc: freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org X-Mailman-Approved-At: Fri, 29 Sep 2017 10:38:12 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 07:35:48 -0000 Hi Elena, Thanks for the details. I'm also forwarding this to the Current mailing list as it has more devs following. Br, Tommi On Thu, 28 Sep 2017 at 10.45, Elena Mihailescu wrote: > Hi Tommi, > > > 2017-09-27 21:28 GMT+03:00 Tommi Pernila : > > Hi Elena, > > > > If you can send more details from the hardware, the troubleshooting will > be > > faster. > > E.g. sending a dmesg output (even a screenshot) from the OS. > > > > At the moment, there is no OS on this PC. When I try to install FreeBSD, > this is the only output which is displayed and everything hangs [1]. > > I took a picture of system information from BIOS [2]. Please tell me if > there is any other information from the hardware which I should provide. > > > > > Have you tried booting FreeBSD 11.1-Release? > > > > Also when you mention FreeBSD 12.0-Current, is it how old? Or do you know > > the release version? > > I downloaded FreeBSD-12.0-CURRENT-amd64-20170814-r322508-disc1.iso from > here [3]. I will try the latest version, which is > FreeBSD-12.0-CURRENT-amd64-20170925-r323985-disc1.iso and also, I will try > FreeBSD 11.1-Release. > > Thanks, > Elena > > > [1] https://drive.google.com/open?id=0B8qXIRloK7xeRDZMdnJBTWRCXzA > [2] https://drive.google.com/open?id=0B8qXIRloK7xeRU03R3p4OGdnYlU > [3] > https://download.freebsd.org/ftp/snapshots/amd64/amd64/ISO-IMAGES/12.0/ > From owner-freebsd-current@freebsd.org Fri Sep 29 11:26:30 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B03B2E2ACBC; Fri, 29 Sep 2017 11:26:30 +0000 (UTC) (envelope-from tsoome@me.com) Received: from st13p35im-asmtp002.me.com (st13p35im-asmtp002.me.com [17.164.199.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8680D3EDC; Fri, 29 Sep 2017 11:26:30 +0000 (UTC) (envelope-from tsoome@me.com) Received: from process-dkim-sign-daemon.st13p35im-asmtp002.me.com by st13p35im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) id <0OX100400H12XO00@st13p35im-asmtp002.me.com>; Fri, 29 Sep 2017 11:26:24 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=04042017; t=1506684383; bh=broJjFNrqt3txJyGR7Q4N4hMK8fINHwONoC8QAAX2Z4=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=cavY6bMPvsX0TITGz8aGq1Ju9irXzMra0PRtwKguVxhbxNStobE1sIwSIx9Le8qvP 7nBZxPmSMdzPoDRI5Wl9TLij2dKHur+Oz0qnWbBMEuzQ40MMDSwur6YwnBRlF43CEw YcnkhYUMbmMnevEX4gABpGJ3SxCfPCOxe1EbQ+ANcwCkYfAvZ6+090cRRGlAkUjN4P LiwX6G6EyXsPvWSuZR/qQGieXuJxPcFJtFaStCtxROCKi3Ird7aN8V2V6wf0rsnrP7 cRDXHYApUO59ZotkOO5wvBpBBp7am/UQ2+6V23QpcJR/2xxwLLA3xyy28OdDTcYOjS XKXzEBEA2yqWw== Received: from icloud.com ([127.0.0.1]) by st13p35im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) with ESMTPSA id <0OX1009PGH3V5040@st13p35im-asmtp002.me.com>; Fri, 29 Sep 2017 11:26:23 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-09-29_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709290165 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 11.0 \(3445.1.6\)) Subject: Re: Cannot install FreeBSD 12.0 on HP Compaq DC5850 From: Toomas Soome In-reply-to: Date: Fri, 29 Sep 2017 14:26:18 +0300 Cc: Elena Mihailescu , freebsd-current , freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <2C4ECAC8-1F4E-4C7B-8FE6-18B649F204AB@me.com> References: To: Tommi Pernila X-Mailer: Apple Mail (2.3445.1.6) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 11:26:30 -0000 > On 29 Sep 2017, at 09:35, Tommi Pernila wrote: >=20 > Hi Elena, >=20 > Thanks for the details. > I'm also forwarding this to the Current mailing list as it has more = devs > following. >=20 Two things to test - check for bios update and UEFI boot (with decure = boot disabled). Also make sure any non-essential devices are = disconnected - this hung smells like it is getting stuck on disk device = probing, and one case I know I can happen was about SD card reader = attached. rgds, toomas > Br, >=20 > Tommi >=20 >=20 > On Thu, 28 Sep 2017 at 10.45, Elena Mihailescu = > wrote: >=20 >> Hi Tommi, >>=20 >>=20 >> 2017-09-27 21:28 GMT+03:00 Tommi Pernila : >>> Hi Elena, >>>=20 >>> If you can send more details from the hardware, the troubleshooting = will >> be >>> faster. >>> E.g. sending a dmesg output (even a screenshot) from the OS. >>>=20 >>=20 >> At the moment, there is no OS on this PC. When I try to install = FreeBSD, >> this is the only output which is displayed and everything hangs [1]. >>=20 >> I took a picture of system information from BIOS [2]. Please tell me = if >> there is any other information from the hardware which I should = provide. >>=20 >>=20 >>=20 >>> Have you tried booting FreeBSD 11.1-Release? >>>=20 >>> Also when you mention FreeBSD 12.0-Current, is it how old? Or do you = know >>> the release version? >>=20 >> I downloaded FreeBSD-12.0-CURRENT-amd64-20170814-r322508-disc1.iso = from >> here [3]. I will try the latest version, which is >> FreeBSD-12.0-CURRENT-amd64-20170925-r323985-disc1.iso and also, I = will try >> FreeBSD 11.1-Release. >>=20 >> Thanks, >> Elena >>=20 >>=20 >> [1] https://drive.google.com/open?id=3D0B8qXIRloK7xeRDZMdnJBTWRCXzA >> [2] https://drive.google.com/open?id=3D0B8qXIRloK7xeRU03R3p4OGdnYlU >> [3] >> = https://download.freebsd.org/ftp/snapshots/amd64/amd64/ISO-IMAGES/12.0/ >>=20 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Fri Sep 29 16:42:34 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8D41E3041E for ; Fri, 29 Sep 2017 16:42:34 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic306-3.consmr.mail.bf2.yahoo.com (sonic306-3.consmr.mail.bf2.yahoo.com [74.6.132.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 83DB16CE4C for ; Fri, 29 Sep 2017 16:42:33 +0000 (UTC) (envelope-from filippomore@yahoo.com) X-YMail-OSG: tKFXVCoVM1mlSMnq51LkTOxGzKb_Mb2oxkOwvGc3Ne4oi0VCigO2I3qdO1z2qUQ x6vwBn82XxZE6f3UVp_z_Bgm1vVO3aOXvFeFsBUqd6FxSaZtO_oAun.bGXTLVXf4wqTMOrNX8yKs w.PZ9xwYGPm5uhza.MPl21AYlvRXor3jAnT5Drie4tVcsEj8BGtwam5bIG8Wh0BSoCivJjT8bEYT PXcHtLvYsG3G2B3n75Es2snBZVguDfdV3hfSBdOi8YvIw684jqzROOuvd4SXSZMQ3jErk2LF1kn1 FAdmq1.3AUrDrbRwYcSOCDlnYDKrVqviBlYaA85SVlCaoXDzykGJ9LnbG165LzO3oZ9EQdjF3Y7v Tiap.md.vJLZIQKnJ32J6zptL138fZ5ljByB60T5xvwnbJwlTO8SNHerTonDz31eOA2LFYOykOQ6 JxnsAqGX2ZkwejIvRb8mNR5tXPoC2c68X9yuoWJnt Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.bf2.yahoo.com with HTTP; Fri, 29 Sep 2017 16:42:26 +0000 Date: Fri, 29 Sep 2017 16:42:26 +0000 (UTC) From: Filippo Moretti To: FreeBSD Current Message-ID: <173970722.743739.1506703346081@mail.yahoo.com> Subject: Problem with buildkernel MIME-Version: 1.0 References: <173970722.743739.1506703346081.ref@mail.yahoo.com> X-Mailer: WebService/1.1.10653 YMailNorrin Mozilla/5.0 (X11; FreeBSD i386; rv:55.0) Gecko/20100101 Firefox/55.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 16:42:34 -0000 After buildworld of yesterday the buildkernel fails with the following mess= age:r/src/tmp/usr/bin -c -O2 -pipe -fno-strict-aliasing -nostdinc -I. -I/us= r/src/sys -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEAD= ERS -include opt_global.h -MD -MF.depend.if_hn.o -MTif_hn.o -mno-mmx -mno-= sse -msoft-float -ffreestanding -fwrapv -fstack-protector -Wall -Wredundant= -decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-= arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=3D__freeb= sd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-= pragmas -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-pa= rentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-= error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -mn= o-avx -std=3Diso9899:1999 -Werror /usr/src/sys/dev/hyperv/netvsc/if_hn.c***= Error code 1Stop.make[2]: stopped in /usr/obj/usr/src/sys/STING_VT*** Erro= r code 1Stop.make[1]: stopped in /usr/src*** Error code 1Stop.make: stopped= in /usr/src~ [root@sting /home/filippo]# uname -aFreeBSD sting 12.0-CURRENT FreeBSD 12.0= -CURRENT #4 r322877M: Fri Aug 25 19:11:32 CEST 2017 root@sting:/usr/obj/usr= /src/sys/STING_VT i386sincerelyFilippo From owner-freebsd-current@freebsd.org Fri Sep 29 17:18:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D28DDE310C2 for ; Fri, 29 Sep 2017 17:18:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A35476E7A9 for ; Fri, 29 Sep 2017 17:18:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22a.google.com with SMTP id i197so378081ioe.9 for ; Fri, 29 Sep 2017 10:18:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=vBE0vCY9XcLj7D+cfyzi8/apOavMKxXQM2db/k+NUf8=; b=m2vJHRJ9MdkG/1uF8UcR0kCmr69EvvMo0wJuudJrSTrfCUg2wtA9l4zm2VLd+wXYTS 9/hBXls7Chl4oTxPL1vR3DqTZKM5yFAYPB+YOJo83D4lRMWKooxrELAytuXjIWcWpQJN 3zPWp9XQBQrNLKfN/XhscSUdb7oHc7k1NL7UQ/bMs+NmF5ZYG1vdeDA56AzX5cs4OV5T rV1v5sC5vs9/d0GXOZv7Og7pi0HRIn95+E9+dpkyPcZZ28l/eNGkJNIIDvaEOL5cN2WL cdNtPCe8JLiC72CQxvpGTqIK31mz4ugFSGD6/4nZHItL90e2ok+uGtNRnZtztAdjnAri O+GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=vBE0vCY9XcLj7D+cfyzi8/apOavMKxXQM2db/k+NUf8=; b=BVbiOpRiXvIW2YZklHQw8akc4mJ6prG2uCad1IrLgRtgn/tMWHFjlOuy9j1/2uxa5x RTGLjABCyzac0bju7AThL3gg3chUP/TPPfHqmfrub73PGK6a986xT9QJdOH0QrnOaUIC ndkKEqBQNPN7CAkyl9nZdySXJSRmZMMAJxpwpfbZZwW7LcZR4jPbPXTxkB09ofPTf924 BPoCgJu6FFdMR5Xg+4DzcNIFHEBW9Ck6GEfdZ2DDWY8O+0kQXC7mufkGUishRB1ix9mD R87BZOkF5Wk0bTxs4PcawXXG70zZls1mKfQ/9ju+pAYl2DNfezSZX/av08T2yH/2diXZ 8Xig== X-Gm-Message-State: AMCzsaWNNv2a5+mVm562/fxyzu56nAS3CH8B2zbACOAxx2ZKX5I5mgI7 nU+CMQTSKwBsJZg0l4Q+uITC4rkOg7PeeBlm9wQDqQ== X-Google-Smtp-Source: AOwi7QC35QV2m3yWKKJzIPYbyHrxhs4q8iXM9gKXw72SVwL8tg6gLpH9bvAVkzfiAb81P1P9UflzXwqfsTOrR98/ptk= X-Received: by 10.107.68.6 with SMTP id r6mr13676690ioa.282.1506705493915; Fri, 29 Sep 2017 10:18:13 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.2.194 with HTTP; Fri, 29 Sep 2017 10:18:13 -0700 (PDT) X-Originating-IP: [96.88.77.243] In-Reply-To: References: <201709290109.v8T19oMO028898@pdx.rh.CN85.dnsmgr.net> From: Warner Losh Date: Fri, 29 Sep 2017 11:18:13 -0600 X-Google-Sender-Auth: HHRgV3ZbGuWPIzSpsGFJsIf2hQE Message-ID: Subject: Re: How do GEOM_PART_* options configure geom_part_* modules?? To: Kevin Oberman Cc: "Rodney W. Grimes" , Nick Hibma , FreeBSD Current Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 17:18:14 -0000 On Thu, Sep 28, 2017 at 11:54 PM, Kevin Oberman wrote: > On Thu, Sep 28, 2017 at 6:09 PM, Rodney W. Grimes < > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > >> > On Thu, Sep 28, 2017 at 11:12 AM, Kevin Oberman >> wrote: >> > >> > > On Thu, Sep 28, 2017 at 9:13 AM, Warner Losh wrote: >> > > >> > >> On Thu, Sep 28, 2017 at 10:00 AM, Nick Hibma > > >> > >> wrote: >> > >> >> > >> > I created a new kernel config file from scratch, wondered what the >> > >> > GEOM_PART_MBR option and friends were doing, search for them, >> didn't >> > >> find >> > >> > them in the tree, and deleted them from my config. But... de >> resulting >> > >> disk >> > >> > image didn't boot, because of the fact that it didn't recognise >> the MBR >> > >> > partitions (it only had a single diskid entry on the mount-root >> prompt). >> > >> > >> > >> > Can anyone explain to me how these kernel options work, as in: >> they are >> > >> > defined in kernel configs and as a consequence in opt_geom.h, but >> how >> > >> are >> > >> > they actually used to select which geom_part_* modules/kernel >> parts to >> > >> > build? I thought these options were translated to stuff that cpp >> would >> > >> use, >> > >> > but there are not uses of for example GEOM_PART_MBR anywhere for >> > >> example! >> > >> > >> > >> > >> > >> > The module always build them because they are listed in the >> module's >> > >> > Makefile. >> > >> > >> > >> > The kernel only sometimes does. Here's the key lines from >> conf/files: >> > >> > files:geom/geom_bsd_enc.c optional geom_bsd | geom_part_bsd >> > >> > files:geom/part/g_part_apm.c optional geom_part_apm >> > >> > files:geom/part/g_part_bsd.c optional geom_part_bsd >> > >> > files:geom/part/g_part_bsd64.c optional geom_part_bsd64 >> > >> > files:geom/part/g_part_ebr.c optional geom_part_ebr >> > >> > files:geom/part/g_part_gpt.c optional geom_part_gpt >> > >> > files:geom/part/g_part_ldm.c optional geom_part_ldm >> > >> > files:geom/part/g_part_mbr.c optional geom_part_mbr >> > >> > files:geom/part/g_part_vtoc8.c optional geom_part_vtoc8 >> > >> > >> > >> > which turn on/off which files get included. config "helpfully" >> converts >> > >> the >> > >> > upper case options to lower case for this. >> > >> > >> > >> > Warner >> > >> > >> > >> > >> > >> > *slaps forehead* Goose chase! >> > >> > >> > >> > I actually knew that... and, at the time, thought it was weird >> > >> behaviour. >> > >> > ''grep" would not have failed me if those options would be >> uppercase >> > >> there >> > >> > ... >> > >> > >> > >> >> > >> I've been nibbled to death by these geese often enough to have a >> PTSD-like >> > >> reaction when someone mentions it and habitually add -i to my >> greps... >> > >> >> > >> Warner >> > > >> > > >> > > This horrid POLA violation seems to have been in FreeBSD configuration >> > > since at least 3.0 and probably goes back to the creation of the >> > > configuration process. >> > > >> > > Any idea why such a horrible POLA was ever introduced? Seems like an >> > > obviously bad idea in an OS that is ALMOST always case sensitive. >> > > >> > >> > It's received code from the old 4.3 BSD config program (or maybe the >> net-2 >> > config program). >> >> We had best not have any code direct from 4.3 or net-2, it should be from >> 4.4BSDLite, any code prior to that is subject of the lawsuit. >> >> >> -- >> Rod Grimes >> rgrimes@freebsd.org >> > > I believe any code in 4.3 BSD that was developed by CSRG was not subject > of the suit. Copyright on that code by the Regents of UC, not AT&T. (So was > any code I wrote.) Many utilities were from v8 or descended from AT&T code, > but a great deal was not. It was what UC was getting paid for. > Yea, the suit quickly devolved into the short list of files... v8 was from BSD that was reimported. That's clear from the TUHS archive. v8 is a big break from v7... Warner From owner-freebsd-current@freebsd.org Sat Sep 30 01:09:39 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3115E017F1 for ; Sat, 30 Sep 2017 01:09:39 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from mail.protected-networks.net (mail.protected-networks.net [IPv6:2001:470:8d59:1::8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protected-networks.net", Issuer "Protected Networks CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AEF87F322; Sat, 30 Sep 2017 01:09:39 +0000 (UTC) (envelope-from imb@protected-networks.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= protected-networks.net; h=content-transfer-encoding :content-language:content-type:content-type:mime-version :user-agent:date:date:message-id:subject:subject:from:from; s= 201508; t=1506733776; bh=bX+ZcXaalXx3LAFs1E4uX33izpebAEyYEuCR8dP OwQ4=; b=BTBBAWfCrm2hnNXXFs+0AFpuHGJQ8s354hfDXb1cexDQ46Ks1FpiRbY 0mblITt2wAtNjDSWZYg9sNt4C2pIpQj/Bo7oh6q0rfmzP0qvFD3bO5N7WSUme0DT 9QzANh4ZCIOoQAKh7GxjjhPOMko9BK7OytEZFFYlo8Lpx6ceLvR0= Received: from toshi.auburn.protected-networks.net (toshi.auburn.protected-networks.net [192.168.1.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: imb@mail.protected-networks.net) by mail.protected-networks.net (Postfix) with ESMTPSA id D70AC2E4C8; Fri, 29 Sep 2017 21:09:36 -0400 (EDT) To: freebsd-current Cc: rmacklem@freebsd.org From: Michael Butler Subject: Can't NFS mount ZFS volume Message-ID: <4cbb6150-0fcc-5393-846f-309e19cfb0ea@protected-networks.net> Date: Fri, 29 Sep 2017 21:09:35 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 01:09:39 -0000 Both client and server have been upgraded from SVN r324033 to r324089 Now I can't mount a ZFS dataset over NFS :-( imb@toshi:/home/imb> sudo /sbin/mount -t nfs vm01:/usr/local/exports/ /mnt imb@toshi:/home/imb> mount /dev/ada0s3a on / (ufs, local, soft-updates) devfs on /dev (devfs, local, multilabel) procfs on /proc (procfs, local) linprocfs on /usr/compat/linux/proc (linprocfs, local) linsysfs on /usr/compat/linux/sys (linsysfs, local) vm01:/usr/local/exports on /mnt (nfs) imb@toshi:/home/imb> sudo ls -l /mnt ls: /mnt: Input/output error The server shows the mount as being registered .. imb@vm01:/home/imb> showmount -a All mount points on localhost: toshi.auburn.protected-networks.net:/usr/local/exports It takes some time to complete an umount request but it does complete, Any thoughts? imb From owner-freebsd-current@freebsd.org Sat Sep 30 19:38:25 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D6D3E2D9F3 for ; Sat, 30 Sep 2017 19:38:25 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7EE817D95D for ; Sat, 30 Sep 2017 19:38:25 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7AFFEE2D9F2; Sat, 30 Sep 2017 19:38:25 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A5E9E2D9F1 for ; Sat, 30 Sep 2017 19:38:25 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic314-14.consmr.mail.bf2.yahoo.com (sonic314-14.consmr.mail.bf2.yahoo.com [74.6.132.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 439CF7D95C for ; Sat, 30 Sep 2017 19:38:25 +0000 (UTC) (envelope-from filippomore@yahoo.com) X-YMail-OSG: TdthDFQVM1kUKCb2H7wya40TLeBbOFoBu31y.Sd1PNDiW_JMHjwzxYVfITjYwWk iec9PiMY222LASvjoTjYnvg1KBxe0kquUamUWl1uRVyIPvyNUgOwAEo.g2Ps_Q9_kzGwT37EEGc1 2JznvXX8QGGh70A0PUffew_KHkk5vAL2ryYyUIJFx8m.5Ah8K8_Shu1OsroMlP.0twW_b_espHjy 87iSdnTQViDC8TCw7isHiYE4UPdZ9tW2BZi4akmi7iznRfo15vUB9ecLVUhNziWRLJcQEqbEwKiF UfWftig4l4sjWS8Z1a3VtpHZEdaXRkyV2DyLzaGjhGjK3Z671nQirK8_9Cpan4EIhAxvvQP6XoUM RHsm4nXdXAOAkX1tpcv4H7RYcTNiewbUh1glyOlrzXqbPWf4VZ.5G5kGAxl7URMEyoEoaC8IPs_N Gf28K2GM9KHZ9retkC8umWb8nNHhoKeLU7Q2uuCmTtk8ze19JxgPSKukNCARCPkHRnLuYEbZks5r Cjg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.bf2.yahoo.com with HTTP; Sat, 30 Sep 2017 19:38:24 +0000 Date: Sat, 30 Sep 2017 19:28:14 +0000 (UTC) From: Filippo Moretti To: "current@freebsd.org" Message-ID: <541566864.1265551.1506799694265@mail.yahoo.com> In-Reply-To: <173970722.743739.1506703346081@mail.yahoo.com> References: <173970722.743739.1506703346081.ref@mail.yahoo.com> <173970722.743739.1506703346081@mail.yahoo.com> Subject: Fw: Problem with buildkernel MIME-Version: 1.0 X-Mailer: WebService/1.1.10653 YMailNorrin Mozilla/5.0 (X11; FreeBSD i386; rv:56.0) Gecko/20100101 Firefox/56.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 19:38:25 -0000 =20 ----- Forwarded Message ----- From: Filippo Moretti To: FreeBSD Current Sent: Friday, September= 29, 2017, 6:42:26 PM GMT+2Subject: Problem with buildkernel After buildworld of yesterday the buildkernel fails with the following mes= sage:r/src/tmp/usr/bin -c -O2 -pipe -fno-strict-aliasing -nostdinc -I. -I/u= sr/src/sys -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEA= DERS -include opt_global.h -MD -MF.depend.if_hn.o -MTif_hn.o -mno-mmx -mno= -sse -msoft-float -ffreestanding -fwrapv -fstack-protector -Wall -Wredundan= t-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer= -arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=3D__free= bsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown= -pragmas -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-p= arentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno= -error-shift-negative-value -Wno-error-address-of-packed-member -mno-aes -m= no-avx -std=3Diso9899:1999 -Werror /usr/src/sys/dev/hyperv/netvsc/if_hn.c**= * Error code 1Stop.make[2]: stopped in /usr/obj/usr/src/sys/STING_VT*** Err= or code 1Stop.make[1]: stopped in /usr/src*** Error code 1Stop.make: stoppe= d in /usr/src~ [root@sting /home/filippo]# uname -aFreeBSD sting 12.0-CURRENT FreeBSD 12.0= -CURRENT #4 r322877M: Fri Aug 25 19:11:32 CEST 2017 root@sting:/usr/obj/usr= /src/sys/STING_VT i386sincerelyFilippo =20 From owner-freebsd-current@freebsd.org Sat Sep 30 16:29:15 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C82DFE2A3BF; Sat, 30 Sep 2017 16:29:15 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88EAA74116; Sat, 30 Sep 2017 16:29:15 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: by mail-oi0-x234.google.com with SMTP id i128so3585351oih.6; Sat, 30 Sep 2017 09:29:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=dmNsDEqskX9LqUrCuqu0iMRchm6i+eB1MJ1+9yvZ7tA=; b=i7M2WCPKfGiXWQ6CTv5DyewkUMXadcSlOB1/znTfpBnIyxWrlwBbfFSetq1qEBOzpB vLJJyMbIKtVItDmOlUD4GIro20sStDEXm00QPcNtwASm6v9NKeAT/UQg8zQiy2rKkk35 MUUpzfmKcPv9ja9nl0Y+BjH2gbwvWFhFmjiRNYvCSFkykg8tUMnAGHW14vXMAJd9Ecrq h9ikSQtsUXiRt3N6ntdScY6nAkqrPDgs74zSYNZtuAFKtpf5a4pG1lK3DuFidPgb5ua/ XJScKd8BuY3kUhirsFBNC9OaV/KKE7Z3EaRyWMv9wW+tl+fjra54z8o/I0UWwhFR/kU8 MQYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=dmNsDEqskX9LqUrCuqu0iMRchm6i+eB1MJ1+9yvZ7tA=; b=A136FOfqRAIkwwqrmgC7m1J3/k7RMzFdQmSyUin1bvJ8UUgfLcg9o20RqsHxDAhqns GnoOQejZUwHUDn4YbK7bIwJkIMY8xhMsvt/LhlYvFKc0BohgxPdpXUtJ2njPe7XFwvvr Usxq2ySoPvOkqj0cym8g6/seM0N8c3bM6MFXnNDBiLWH2FEosG5DrxoPSDobqXK4xgV2 S9snyEYaSog2Lj6euUdPvvlehlj30EF154UVqRVpcZpwAunrEHrk1/+gxQCQeiptHRHX gSUJWXwT2pYGyIr/NvH8Q09yRIyyAsEx4OEhJl/vvcn9zAcDkq8TqM/12h74JbRHtX4y YwBg== X-Gm-Message-State: AMCzsaXqE+R/YFsfSsugmQ1CltvBuculNwhuNAgkKttYIi0F1StYxr7H xdSCQshqFj/8ImWSzpXJz8JBNd40Cu2D+qhEzPE= X-Google-Smtp-Source: AOwi7QBJNpIHS2dUO4vyiWz6z6PGeXP/xOqmHFXukDmLOUGJn8ZofL1sMJyfrDgrdEtTDEB6sg8q7VNqvUv6PGRmZXQ= X-Received: by 10.202.205.86 with SMTP id d83mr3798717oig.181.1506788954782; Sat, 30 Sep 2017 09:29:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.36.228 with HTTP; Sat, 30 Sep 2017 09:29:14 -0700 (PDT) In-Reply-To: <2C4ECAC8-1F4E-4C7B-8FE6-18B649F204AB@me.com> References: <2C4ECAC8-1F4E-4C7B-8FE6-18B649F204AB@me.com> From: Elena Mihailescu Date: Sat, 30 Sep 2017 19:29:14 +0300 Message-ID: Subject: Re: Cannot install FreeBSD 12.0 on HP Compaq DC5850 To: Toomas Soome Cc: Tommi Pernila , freebsd-current , freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Sat, 30 Sep 2017 19:45:40 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 16:29:15 -0000 > Two things to test - check for bios update and UEFI boot (with decure boo= t disabled). Also make sure any non-essential devices are disconnected - th= is hung smells like it is getting stuck on disk device probing, and one cas= e I know I can happen was about SD card reader attached. > I already updated the BIOS and I couldn't find any UEFI boot/ Secure boot features available on my computer (in the BIOS settings). I don't have any other devices connected than those with which the computer arrived. Thanks, Elena From owner-freebsd-current@freebsd.org Sat Sep 30 19:58:22 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 825D1E2DF8A; Sat, 30 Sep 2017 19:58:22 +0000 (UTC) (envelope-from tsoome@me.com) Received: from st13p35im-asmtp002.me.com (st13p35im-asmtp002.me.com [17.164.199.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 507A17E393; Sat, 30 Sep 2017 19:58:22 +0000 (UTC) (envelope-from tsoome@me.com) Received: from process-dkim-sign-daemon.st13p35im-asmtp002.me.com by st13p35im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) id <0OX300500Z6HRO00@st13p35im-asmtp002.me.com>; Sat, 30 Sep 2017 19:58:14 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=04042017; t=1506801494; bh=cOVesOWCUwReUTRfAlNnvHji3x0YnUzXNiQrbHLRAeY=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=lrHJ8Oe14pgS43x5tHms+pLz53SA7bYyFoSMaDdjekAinwkn2XE+nt4iaLXrugh7P 39q6VmIjR8K/KJt2klcCuTViEsPI6W9GO+oL5aIqgh1JXHpok2UoMzjOPs7zoSOdLw y/12rsXSeE1IgsdF2HSUQxWkNrus/9xZFQaHOf3ynuYl8fIne73Pa2Qw+oBJVgBXLd xx2O+tcB/9Szxw5Rn0FJHBlYfU9VtMzKDDRW6XJ851wzfMYyk5fkgmGk7Qc/0IrBgT 8vhCA02Ivk2dr1Q7y8vQKozEJBOXHPrq8TFTPNASEUnRPLOgC5isGg8FXzg1B5sXcb xkQuLKw0UTiUw== Received: from icloud.com ([127.0.0.1]) by st13p35im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) with ESMTPSA id <0OX300KAAZGZ1V10@st13p35im-asmtp002.me.com>; Sat, 30 Sep 2017 19:58:14 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-09-30_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1015 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709300299 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 11.0 \(3445.1.6\)) Subject: Re: Cannot install FreeBSD 12.0 on HP Compaq DC5850 From: Toomas Soome In-reply-to: Date: Sat, 30 Sep 2017 22:58:11 +0300 Cc: Tommi Pernila , freebsd-current , freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Content-transfer-encoding: quoted-printable Message-id: References: <2C4ECAC8-1F4E-4C7B-8FE6-18B649F204AB@me.com> To: Elena Mihailescu X-Mailer: Apple Mail (2.3445.1.6) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 19:58:22 -0000 > On 30 Sep 2017, at 19:29, Elena Mihailescu = wrote: >=20 >> Two things to test - check for bios update and UEFI boot (with decure = boot disabled). Also make sure any non-essential devices are = disconnected - this hung smells like it is getting stuck on disk device = probing, and one case I know I can happen was about SD card reader = attached. >>=20 >=20 > I already updated the BIOS and I couldn't find any UEFI boot/ Secure > boot features available on my computer (in the BIOS settings). I don't > have any other devices connected than those with which the computer > arrived. >=20 > Thanks, > Elena Ok, the error is happening really early as you only get BTX messages and = none from the actual loader itself. Can you test usb boot and see if it = will get further? Also testing/debugging with usb image will be much = easier as we can update loader on usb=E2=80=A6.=20 rgds, toomas= From owner-freebsd-current@freebsd.org Sat Sep 30 20:25:04 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5073BE2E8C7 for ; Sat, 30 Sep 2017 20:25:04 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0088.outbound.protection.outlook.com [104.47.32.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4F7F7F4B7; Sat, 30 Sep 2017 20:25:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM (52.132.78.18) by YQXPR0101MB2278.CANPRD01.PROD.OUTLOOK.COM (52.132.80.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Sat, 30 Sep 2017 20:25:02 +0000 Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5]) by YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5%13]) with mapi id 15.20.0077.011; Sat, 30 Sep 2017 20:25:02 +0000 From: Rick Macklem To: Michael Butler , freebsd-current CC: "rmacklem@freebsd.org" Subject: Re: Can't NFS mount ZFS volume Thread-Topic: Can't NFS mount ZFS volume Thread-Index: AQHTOYjN+X59uTi4TECKw4yDH2UgS6LN4AWa Date: Sat, 30 Sep 2017 20:25:02 +0000 Message-ID: References: <4cbb6150-0fcc-5393-846f-309e19cfb0ea@protected-networks.net> In-Reply-To: <4cbb6150-0fcc-5393-846f-309e19cfb0ea@protected-networks.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQXPR0101MB2278; 6:MpmNjbSKSqTWWOK7eTkX/pEe9yiDjE3RHtc4fidwEi8SRejulLAp7jp9dykvceVMZz+7hx2Dq+JIy1UPyXZ7shA5dstEYmgq3oLojgf/VRUZHZBHG0D9UnpMcwoiOf5v6BiMn4h25qzV3lofn9WCH9Kl5U480YKmx6xu3NVhnMvD8I5+eHnD5rVaXFkwBMyRTMnrZXmKyu/hi0N0nemrzlRe4id/W8NJTKRCQZEC71becwq5Vys+D0G5OOr+3Qt7wEvxk3CW7Rqzd8uiZS3U0kRH0O9XHYPwGcGbYNzMjgpt2wsyeBShb5xbQZSq2Gl7VdvntpIAujnJ8RsjvuatOA==; 5:qZnFJbyXjSbSO4ogQD/Gymo5qmOFVOq0A469qd9fw1Lp9i58bIbIjneZQMxeDrBScPAq9AuwU0hvKj4Lx9ukRKMMuuH5lYd7dwWvDg6ktFjZtQexNo3vgFpxELSw2vhOV+sLJ68fVnNnzskgxC/Fag==; 24:mOZWEzJ7JE+BAEHaVa8Nc+wVILGND/2JsonzvGlgiSvZKV96bKFsPS7Lz75UENBgqcxm6yUU5NPDt8mabcO7x966S5wBvKUVBQFhWlpRkCc=; 7:5lSARwQSkeCtE5++PSi8ktW88kj1aoxqz9+Kg8VEve+KJ3Wq+fMWHIfO4zAj/pJSeYM4nuz7dv1Jpn9fnPW3Byw3bPE0kxbWLTbq3ZiOH9pkhhX5Bgv15FS2JyGfZ+kqJzZu3NX00qDZLBXpDvnzTn3SWD7Kt79iIOzRS1eNcI+s8kYkFnrBMjsGXYcNG1wbXY9U12n5UW15rM0TLYZCyEIEzsylY/tN/f5kFTDaCZY= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: bb16c4c1-93af-441c-71c1-08d50841542d x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:YQXPR0101MB2278; x-ms-traffictypediagnostic: YQXPR0101MB2278: x-exchange-antispam-report-test: UriScan:(158342451672863)(265634631926514); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6041248)(20161123558100)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123564025)(20161123555025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:YQXPR0101MB2278; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:YQXPR0101MB2278; x-forefront-prvs: 0446F0FCE1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(39860400002)(376002)(377454003)(189002)(199003)(53936002)(97736004)(189998001)(105586002)(7696004)(53546010)(305945005)(786003)(74316002)(101416001)(575784001)(76176999)(54356999)(5660300001)(50986999)(2900100001)(2950100002)(106356001)(9686003)(6506006)(316002)(8936002)(3280700002)(4326008)(6246003)(5250100002)(2906002)(25786009)(3660700001)(68736007)(102836003)(74482002)(14454004)(33656002)(81156014)(6436002)(86362001)(55016002)(229853002)(81166006)(478600001)(110136005)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR0101MB2278; H:YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2017 20:25:02.0172 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB2278 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 20:25:04 -0000 I have only done two NFS commits within that range. 1 - A trivial one that adds two new arguments always specified as 0, which has no change in semantics. 2 - One that only affects NFSv4 during dismount, so it shouldn't affect an NFSv3 mount. Some things to try: - get rid of rpc.statd and rpc.lockd and do the mount with the "nolockd" option - capture packets during the mount and look at them in wireshark, to see what is going on the wire. - see if there are any patches applied to your net interface driver during that commit rev. range - if you have any kind of firewall setup, get rid of that and see if that helps Good luck with it, rick ________________________________________ From: Michael Butler Sent: Friday, September 29, 2017 9:09:35 PM To: freebsd-current Cc: rmacklem@freebsd.org Subject: Can't NFS mount ZFS volume Both client and server have been upgraded from SVN r324033 to r324089 Now I can't mount a ZFS dataset over NFS :-( imb@toshi:/home/imb> sudo /sbin/mount -t nfs vm01:/usr/local/exports/ /mnt imb@toshi:/home/imb> mount /dev/ada0s3a on / (ufs, local, soft-updates) devfs on /dev (devfs, local, multilabel) procfs on /proc (procfs, local) linprocfs on /usr/compat/linux/proc (linprocfs, local) linsysfs on /usr/compat/linux/sys (linsysfs, local) vm01:/usr/local/exports on /mnt (nfs) imb@toshi:/home/imb> sudo ls -l /mnt ls: /mnt: Input/output error The server shows the mount as being registered .. imb@vm01:/home/imb> showmount -a All mount points on localhost: toshi.auburn.protected-networks.net:/usr/local/exports It takes some time to complete an umount request but it does complete, Any thoughts? imb From owner-freebsd-current@freebsd.org Sat Sep 30 21:39:21 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02599E2FE21 for ; Sat, 30 Sep 2017 21:39:21 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from mail.protected-networks.net (mail.protected-networks.net [202.12.127.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protected-networks.net", Issuer "Protected Networks CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C535181BF6; Sat, 30 Sep 2017 21:39:20 +0000 (UTC) (envelope-from imb@protected-networks.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= protected-networks.net; h=content-transfer-encoding :content-language:content-type:content-type:in-reply-to :mime-version:user-agent:date:date:message-id:from:from :references:subject:subject; s=201508; t=1506807552; bh=j6BM5Cxe Ywz/lg2RBRJv9Sv0rKh5OdHXF17oGe9N4h4=; b=p2Kf3llXKPMkvknE2rRWHapy IbKja/LJ6wwwrhetgkau1uDgVGY68HA+QgXVMayOK+3PNjtpGASpT+R6vhMAv1bn 3PvFJeRMhbw4dCAf8ybSGxMo9tjF3YakOz0KwfYbo+kp2AmawvIwRcGuQmqBdLHQ OR26ILvwVx123dyuVFo= Received: from toshi.auburn.protected-networks.net (toshi.auburn.protected-networks.net [192.168.1.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: imb@mail.protected-networks.net) by mail.protected-networks.net (Postfix) with ESMTPSA id 557EB3D79; Sat, 30 Sep 2017 17:39:12 -0400 (EDT) Subject: Re: Can't NFS mount ZFS volume To: Rick Macklem , freebsd-current Cc: "rmacklem@freebsd.org" References: <4cbb6150-0fcc-5393-846f-309e19cfb0ea@protected-networks.net> From: Michael Butler Message-ID: <65feb444-3265-b9bc-3d4d-d57b125513d3@protected-networks.net> Date: Sat, 30 Sep 2017 17:39:10 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 21:39:21 -0000 I have no idea why but using .. sudo /sbin/mount vm01:/usr/local/exports/ /mnt .. instead of .. sudo /sbin/mount -t nfs vm01:/usr/local/exports/ /mnt .. works; sorry for the noise, imb On 09/30/17 16:25, Rick Macklem wrote: > I have only done two NFS commits within that range. > 1 - A trivial one that adds two new arguments always specified as 0, > which has no change in semantics. > 2 - One that only affects NFSv4 during dismount, so it shouldn't affect > an NFSv3 mount. > > Some things to try: > - get rid of rpc.statd and rpc.lockd and do the mount with the "nolockd" > option > - capture packets during the mount and look at them in wireshark, to > see what is going on the wire. > - see if there are any patches applied to your net interface driver during > that commit rev. range > - if you have any kind of firewall setup, get rid of that and see if that > helps > > Good luck with it, rick > ________________________________________ > From: Michael Butler > Sent: Friday, September 29, 2017 9:09:35 PM > To: freebsd-current > Cc: rmacklem@freebsd.org > Subject: Can't NFS mount ZFS volume > > Both client and server have been upgraded from SVN r324033 to r324089 > > Now I can't mount a ZFS dataset over NFS :-( > > imb@toshi:/home/imb> sudo /sbin/mount -t nfs vm01:/usr/local/exports/ /mnt > > imb@toshi:/home/imb> mount > /dev/ada0s3a on / (ufs, local, soft-updates) > devfs on /dev (devfs, local, multilabel) > procfs on /proc (procfs, local) > linprocfs on /usr/compat/linux/proc (linprocfs, local) > linsysfs on /usr/compat/linux/sys (linsysfs, local) > vm01:/usr/local/exports on /mnt (nfs) > > imb@toshi:/home/imb> sudo ls -l /mnt > ls: /mnt: Input/output error > > The server shows the mount as being registered .. > > imb@vm01:/home/imb> showmount -a > All mount points on localhost: > toshi.auburn.protected-networks.net:/usr/local/exports > > It takes some time to complete an umount request but it does complete, > > Any thoughts? > > imb >