From owner-freebsd-hackers@freebsd.org Sun Jun 11 03:39:54 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6C03BFD300 for ; Sun, 11 Jun 2017 03:39:54 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 882A271D83 for ; Sun, 11 Jun 2017 03:39:54 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from T530-Allan.HML3.ScaleEngine.net (unknown [38.64.177.99]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 2D8C913333 for ; Sun, 11 Jun 2017 03:39:52 +0000 (UTC) Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: freebsd-hackers@freebsd.org References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> <20170610163642.GA18123@zxy.spb.ru> From: Allan Jude Message-ID: Date: Sat, 10 Jun 2017 23:39:34 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <20170610163642.GA18123@zxy.spb.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-CA Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2017 03:39:54 -0000 On 06/10/2017 12:36, Slawa Olhovchenkov wrote: > On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > >> Gents, >> >> I'm experiencing an issue where iterating over a PostgreSQL table of ~21.5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds on Intel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back from Jan 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm using PostgreSQL 9.5.7. >> >> I've tried: >> >> * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 2017. >> >> * Tested on half a dozen machines with different models of SSDs: >> >> o Intel 510s (120GB) in ZFS mirrored pair >> >> o Intel 520s (120GB) in ZFS mirrored pair >> >> o Intel 540s (120GB) in ZFS mirrored pair >> >> o Samsung 850 Pros (256GB) in ZFS mirrored pair >> >> * Using bonnie++ to remove Postgres from the equation and performance does indeed drop. >> >> * Rebooting server and immediately re-running test and performance is back to original. >> >> * Tried using Karl Denninger's patch from PR187594 (which took some work to find a kernel that the FreeBSD10 patch would both apply and compile cleanly against). >> >> * Tried disabling ZFS lz4 compression. >> >> * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL 9.1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime and test took ~80 seconds after which I rebooted and re-ran test and was still at ~80 seconds (older processor and memory in this system). >> >> I realize that there's a whole lot of info I'm not including (dmesg, zfs-stats -a, gstat, et cetera): I'm hoping some enlightened individual will be able to point me to a solution with only the above to go on. > > Just a random guess: can you try r307264 (I am mean regression in > r307266)? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > This sounds a bit like an issue I investigated for a customer a few months ago. Look at gstat -d (includes DELETE operations like TRIM) If you see a lot of that happening, but try: vfs.zfs.trim.enabled=0 in /boot/loader.conf and see if your issues go away. the FreeBSD TRIM code for ZFS basicallys waits until the sector has been free for a while (to avoid doing a TRIM on a block we'll immediately reuse), so your benchmark will run file for a little while, then suddenly the TRIM will kick in. For postgres, fio, bonnie++ etc, make sure the ZFS dataset you are storing the data on / benchmarking has a recordsize that matches the workload. If you are doing a write-only benchmark, and you see lots of reads in gstat, you know you are having to do read/modify/write's, and that is why your performance is so bad. -- Allan Jude From owner-freebsd-hackers@freebsd.org Sun Jun 11 16:33:25 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 782C1D8A542 for ; Sun, 11 Jun 2017 16:33:25 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0065.outbound.protection.outlook.com [104.47.41.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F10DC6EF99 for ; Sun, 11 Jun 2017 16:33:24 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hC2OgNrG1nEfZB+bQ76YlLKTiTNvamG7iunJJuZ+6YI=; b=qG2Bq3Tjhi8tjAPJJS7YECKZ19Z+OdbWXXc2UnfhFoHLlEzy4tVRSaaFdTHDZYPdzHerEfRQA98pUbLOoYrzN0iyvJsifyduPjDi9PWmVz+gBwyMseMQWU5FrkxKXD4d+yJmFWPMaOYVWLA1yNmVirMF7KCWjWY8A2hRrRzJCVU= Received: from CY1PR03CA0009.namprd03.prod.outlook.com (10.174.128.19) by BY2PR0301MB0743.namprd03.prod.outlook.com (10.160.63.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sun, 11 Jun 2017 16:33:22 +0000 Received: from BN1BFFO11FD027.protection.gbl (2a01:111:f400:7c10::1:144) by CY1PR03CA0009.outlook.office365.com (2603:10b6:600::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Sun, 11 Jun 2017 16:33:21 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; zxy.spb.ru; dkim=none (message not signed) header.d=none;zxy.spb.ru; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BN1BFFO11FD027.mail.protection.outlook.com (10.58.144.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Sun, 11 Jun 2017 16:33:20 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sun, 11 Jun 2017 16:33:19 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Sun, 11 Jun 2017 16:33:19 +0000 From: "Caza, Aaron" To: Slawa Olhovchenkov CC: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLiAalX9ncyh3crQla1B5/716RidAABhQ0AADIocOA= Date: Sun, 11 Jun 2017 16:33:19 +0000 Message-ID: <29f90e91bc05472d986b711cffc98c80@DM2PR58MB013.032d.mgd.msft.net> References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> <20170610163642.GA18123@zxy.spb.ru> In-Reply-To: <20170610163642.GA18123@zxy.spb.ru> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.110.68] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB013:|BY2PR0301MB0743: X-MS-Office365-Filtering-Correlation-Id: 536d7d9f-3ac4-4791-d0ec-08d4b0e7929c Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB013.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39840400002)(39860400002)(39400400002)(39410400002)(39850400002)(39450400003)(2980300002)(438002)(377454003)(51914003)(189002)(199003)(13464003)(24454002)(9170700003)(66066001)(33646002)(22756006)(7696004)(8746002)(356003)(8936002)(24736003)(47776003)(108616004)(2900100001)(54356999)(76176999)(50986999)(229853002)(305945005)(7736002)(5890100001)(189998001)(5660300001)(4326008)(86146001)(38730400002)(110136004)(55016002)(42882006)(2906002)(6916009)(2950100002)(72206003)(3846002)(53546009)(102836003)(53936002)(86362001)(6246003)(23726003)(81166006)(46406003)(97756001)(478600001)(8676002)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR0301MB0743; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD027; 1:Jam9KOKL4Lyyg7e9mjykuQczog9G2Cc7fKZDWctJsYl4r5BdHw+P+hZZI1vXxD54xPh2PlJX3tSIxk3ts5T4uXA0IpfzYfHoJNCIClfqxbeTo16SaeIZ8NEg8xilZszUX51IqvIuKoQzFXAV2NZDJCVWyO4iA2oVTOrvwX6mqHQnG3f5E/bvrnbLD3ZO/WvAbZOZDwLYCLvGaj18uR+Nk+As0g6xlhwQb4//cjw2boHkQdpmIgPmh5hir8e44RzUe5kCPZMEuOI++sqWXXf9InsSyRc6U/x8igiL4CRvc6C7GGA6v0tjN2vgNqBwjQTMT536XAPH1NkHUbFlCtMyXDTiW6hCyW5pf0poqbel1sGQ+S7FCTJf5SQGeJqNSnRQef0eblE1AZPZA60YfUdHGvzyjG90ShAxfz064bGKmQrn9KqdGLUSUO0SdsmCFMsBEsjg9jzuo1UZQsj41UHwkS/0wJ5fG9/UUOIWywL9W5emuI+xCXdK6fOHoHdpLxsTTuQhqVQ2Cj0edUKBtPP6Dw== X-CrossPremisesHeadersPromoted: BN1BFFO11FD027.protection.gbl X-CrossPremisesHeadersFiltered: BN1BFFO11FD027.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:BY2PR0301MB0743; X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 3:ivXPuGqgTswtVz1qnLlrleP/4FV5tEyoSVLH/p7fLEGeMx8MkXUzFJfOXwEfUfdwNoaTenW3M0WFG1c4m2Vg33DIWa5FSB5RzEQRv80Mgx1AAHz8LcIEYfkIMpNeSO2LC1dWyAIpd+nAZDdRvxWGDgfqZ1QFSz9Q6p7GBvtf2nlg7eBqMYEcghOOjltV3GBeik0DA8ovTTYkZ8l22NUXVj/9C20Ef6MNrh5e5XviiM+13t2grImTjiCtkqY35orLCeIzmEjVzWCXDj3TqkVUNSHnqlakGbc/g2TSN1Re1a/TpuhwMS4KT14ftpD0sTLy/1A6bdiIQP6QR7a+SGb5LgY1UWD3mt1BNGUxs96Ro/ylJ8oIPqz4v6Z7SkxJL7tmkMKItus4eq1uuDTjph4TRCFtn6H2o41KBaqZ0pup7Txj2TrcMwkVGvg+4eHdVjwRbJwd6y63b2gEx3FuPcQbIfK6IOZY/18FcdAPSsOdLvaVhqbf72eWBOKcvCRe0qAvNW74n0Esz96+/ub+wvxwew== X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 25:LA/4n91DDAAraFqTymeVJ36R4xmcZQgTt8tR+bOW9nHoqIc2ZMyu6DFK0q+bpgV3hnqBOT+q1jbRr0c8a6+MsbGzjh8HBCfBDqrm5g80KeW3frLyIWrY8R1zI2T5cPTB6cDOAA5qwC5FBz1ACJynxLR5gnSGHldVkwPZj9dk2e/i3eyHS1QXCgJzmalki9OCHbWwJBND35G2ZRdgQIR7H4EsQnsN2Z5iPz20Uf54PXC2RJQmTXemFBc5bq6Tx6eMygDY9IyIO+xTa8xR1SD3/ELlyJoexU+bSixa/cQGajLqVbfp/BWZLJcRwJ0br4lulChHRlGhhY2vVT3YfVHZMYfPxIMqmzRbw+rfXbOknTfe6hnW6pDqEq323Jo1QsHsAnKi9ZS31FHWwZUq9IhrakvuZKXu45t3+krPHwyIGOIR7J+DwQeMrS8WRAFK9dSMffe/3YiLultk3eEe/+DskbzYQHdsSeL9tIgDW+Vzq40=; 31:iulUbXbROdlAZbeFqJCM9ogu2nvM8gp4s50JDH94V9SblWZLICPxuxQGqoECp5ZqUGr53rBalZu2U0sWVDa3cfGSmCN1WLIc8A9iZioDBLwdT1m6EkdgQXcpBphHX3exbz4sAaK133kuJd02EFI4q0FGIt8XK/PIlNuwAYUwMb5gngcmiV/5XR32IyGlr53hxd2Jm17DI4KvS4iJ4zHX/XXUaIW7a2A7Pvx7BzhoMx9fiCw6I0ZpyHqTrujHFCMrkzKDRcs45A1dfFroX402Mw== X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 20:oz8aCkgU7RjF8dsv/IV1yqZDwaWpubIGcTsi8sDuokI7jVoBI/yx6dA/d/ggdVF4I/BewjtK0kqPxilU7rJTQBX4r5JEkL6VJUeOSgwORfgPzRpV8gcIAIWCd5g0pS1ORn0KkVYUTat7zRHnlW+kMfW+rztcjW5Kc8xRvRGyZ4negwrpZCAVMcrjIPdAEzu/H3UKt6sGXQghfty4UMVHTbI8IgoQSm8M23XvjdLmhFOgXsA5qADSytzABw0mJaOgmFf2tQYD0NM3/ggBiLUPJC7r0e67UjdW1VXel5MVMgnsCO7gOKUdAip2eKtycvTwM1NZdNgZ0xpID1qDwgwhdHRot471JhSu90NjAKw5K42tVdgUPIDDPnFeM7HlTdt0MldzFe7EL8wJXhzJXaD0eSVaAPCLceJSG9orCj2BlXf89os1kTmnyVtMEgJP003Yn9B47PmSxtIZKcu/dLzoI4N/fwUrCxDsoRYIlQq2S8Cfz39UKBUiQuAKHS6epZ/j X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(93006095)(93004095)(3002001)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123562025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY2PR0301MB0743; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2PR0301MB0743; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0743; 4:uxWKH4/oydCLtFFDlfyVXqluc1DMj5yCIFw4QyHZ?= =?us-ascii?Q?dTfRmqx4xsyN7IpLcb7c0gC2gTDItoJe9wYd72XJ3TDv47R1LvX4cQOzKL5h?= =?us-ascii?Q?LHwVaEvr6FCuPX20YhjBe3ZEX3xcBMAeDqzOb+n7XldnXrFsdManjO13G9sE?= =?us-ascii?Q?QDP+Md7PY9IfW9nDZXVLkhMfxc0hoaeJQKBNq1g2Z45AZEFagEdc9JYc6/kB?= =?us-ascii?Q?a9zaw0rQcz0v7V/TpEt6AYKpJQEmnzauNqMVI6De4tVi3reOOymf/ByxuN+C?= =?us-ascii?Q?JEBcb2iE0uxB3BHHt0Lq/K/VHB0mF8LNAQunaES5jVaADoMIZCl/xd2pS8am?= =?us-ascii?Q?zmrUWH4yfLXazAT8uTcWWSd/SqCryH85WVPbX7w+pLLFraiqAT4lFPMcvzkQ?= =?us-ascii?Q?1SyMy/j6eBpOv+0XANiuHjtJc28r+swa612NoRvl4uTBCX2gf5xidv4qTOnq?= =?us-ascii?Q?Zv17odGP60H6xGShcLsDEu8bAHFSi+C2AfU+YlgKs9y8G9g29WcHnTbzKRK7?= =?us-ascii?Q?0CFOBY4C2ayX90qeZ52fdBT4OPKKPED9PzotSpgl1eMnjKu08umpAf3Lt1Jb?= =?us-ascii?Q?AjXyimhQYLgS2QaAw4y7PLU/4PUP/+JuMH/q0UJDzpOM7O/df9o2ayt6CH+E?= =?us-ascii?Q?t7o3xm8GhX31rdPNqf/8QBvYdVllSwEwYi3H96poU5bcNGbN8X0FSwqjgaDo?= =?us-ascii?Q?JSTCmUGrbUy+WIKBK1ym/skEVLSRt2e0KScLh+F3UoaLpcj3rrWk/sgAJZTq?= =?us-ascii?Q?CsQ27NBD1SbZqzuq8Gzyo7N/TEYdDe+rCyj32QxIgMNXcMV3faXuJNYFH9YG?= =?us-ascii?Q?I7P9gqh+iqwhG8RVOhdS4xWcf/Ldaj6HwgABssgJIA8u0zWVJYN5gr3cM2En?= =?us-ascii?Q?a4Z1LoBqHKpruKATLn/7vIsbgtnn6pJbuO/C1Wqg2ynNz80H6KcoA0y/kQIH?= =?us-ascii?Q?Vo/GwyV8aKdkq3527/YhuHJByoNgOtkvTlODtW8HGKINeb8ZtFdO9mKL5HGb?= =?us-ascii?Q?uKSGlqvQ+TOyH5EQecEJv9qCzX0JdmrGbodz5IaSCw/u4MHiEG2A6buCsvvo?= =?us-ascii?Q?nq6J5iJEAKhRGCqZ8b2R++DVB3Tpv99EafJyBGPlZ+e1MEvo9DZjmfebDsE/?= =?us-ascii?Q?Xb2nw8styr0rgt067Qx6tax4JM/qHbXeKrDElNqW8cV80YuAX4SoUgZg0ff4?= =?us-ascii?Q?XbGSut7HINahiavYsG/+ov70bMpPCexyZ3merrzgw1y6hUIbh3ylUwS7VflC?= =?us-ascii?Q?mDhlDV4kXygIjkgYvVCyx8ZnuFXNWyYvvUz/0AQ8Mm8FQ1rudAQDTQQJASI0?= =?us-ascii?Q?Tmb1CM/EITrx0FfYXVnKLis=3D?= X-Forefront-PRVS: 03355EE97E X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0743; 23:i9dYIXXELw2moJsIKvcD/wvWcqDyaHGZrTnKNmE?= =?us-ascii?Q?oNNxWm5inZJk8Eeml/3enLG4Ru3sbdxabxozkFBb0g8cjQwAvR0f5b3SZdKf?= =?us-ascii?Q?Vd16QT4oF0k54KwKvKUsZhBwfN8voxL0cI+H9zXvjmfoxIB2qKVvI71sGM4B?= =?us-ascii?Q?vEYW2frZSY2F0GLBp42/ErGtremeW5oeNeUhxBSLi9v8R0kf7zP/5vCwiPTh?= =?us-ascii?Q?ROOhzQsB22g/HwCJ7C4Puf1X0ymGfFusT9z3ZiLSKtwntJMf/ynvLsDo1wtI?= =?us-ascii?Q?UuaApdrZ0YfGExtxzFZSDwN58SO7dtmAQSj/wr614zhnsZnmZiv/s69lPy9t?= =?us-ascii?Q?sDPk48UMYzFOOU97fL/zx6evmSTKbxagtSzOCGxaPaw0rk2pVyD2ln5zejmj?= =?us-ascii?Q?i1NcaIELYo98okOO3Y9KJVsi6q6o29T9UP2inUtVbe0Wcw5RY/2s1EJGhdRa?= =?us-ascii?Q?AGMb3TfWjE77CupKsjrkCXkZvSHKS0U3p2Yqzz6ey4NqgALIoZgkc1G1DQDi?= =?us-ascii?Q?eRh98IJrQJDD6SpK/ZZZuLpfv2+mB51aW3sCx0cDeh6Z47SBfq9hCtLkJTam?= =?us-ascii?Q?v4vsUAXf3kSKl7O5dZv5/bnHB2m2NdIMrXSXsdm7wjMQ7/jPRfMnfvp3DAu3?= =?us-ascii?Q?Bh5cpRN9hVnOPm0XazSlHN2tVqE+mDVanfWXgSJ9yZMTy0NIzhDB0/0hxfLM?= =?us-ascii?Q?m+PX0pUqakQgIEvjcXoy3CpS1MirlAsxT6X5vRdpycOvQz7w6tfaAqu63liN?= =?us-ascii?Q?ehJh/THf89NKvNAqkERpDzJ8wszviWTYED7XLGGycIpOw3VRlWhFM67Bm/lb?= =?us-ascii?Q?TMYD3F/rh45bpT3apf8DHg7AuOnS4yyfm1YGm0QwkUrU1SDi0IjuM5CIy5Pg?= =?us-ascii?Q?fCUHO6AYFnxZqd6QTO+7ArAvmCrPxGtAU1Xvt+CPADhGsGhqYzvMMNyFIvCx?= =?us-ascii?Q?BHuBmQLQh4Y7stcAt0MvRRkhbbEIyOFjqONk48O/lgOoMKD25horjhuByVBC?= =?us-ascii?Q?BHoG7sIdEw5U5pmz1H1OatvGoDl027Iq0YITsc8LquTDSgNJyc7nqKwN3FMx?= =?us-ascii?Q?eKw7mNwQC+pwNZNPC/wwmtpj2T9DhGOFp2XU3OG3j5uB7QjnEodLDTEuRb+F?= =?us-ascii?Q?Huosh2/nwxT3gkrTZl2O3zyiLRZHUX2CRCOHeYc/LFKGB+DecYXmA4TiUyl0?= =?us-ascii?Q?MF5qIhbc/PRV5Z5YIV9TjueIz/Vk2zljEjlIPcMoDjpXmH/0MkKyvJnSlBNQ?= =?us-ascii?Q?K8YG+4A5YqrUp1UVaun+Ww+1KkeT4tw26V+JnOPUFji7VNEnNo21rur9+smO?= =?us-ascii?Q?9HTaTplKu2PtM0/Brj7mBg8r963/RQRCGSjd+jaJ7R3aROSJb10FKKklmd+I?= =?us-ascii?Q?BUbGA37aeIP7wPeCqjdtGAb1/57NV4bqNKetvJRNpjAaXepUd6rKy2AHhoa+?= =?us-ascii?Q?JQ/ez75hnJw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 6:NCkiXzc70PjR8Wa5gbYGj1jJ036UQlrc6W9skg+VTJcDguR7KO7NcCa3ciO1otIfkjEtghVj1Zkly+QDT4+7K2z6baoack7yyUyfecOcq0+vhkk2CiqUAJyDJmR1w040gvj15ApDPZge3N04j+H/MGF5SP5XqKOhirqLDWbaU4HUpXtKjmGFUl3zX/5qMrWikGdcK4UXTEScII9X01IOUND8D3kLK0QJdHKINqhYQBcBiR72D3jVs52q1L5hfd9h2NeF6vvQ8ntLS1aBd8D64xd34YhRQY3BxoLVAR/ei0wf4SsWcKgNTzPDCgAzwqolosizsxpn5ES9ni4jiM6Jh4WT9YjJIO8uRfsDkdbRzd+nxS2S2AZjN1cSO8GiJR+KllCMWXn55BzGXBc61orvx8seMcJiPDdFrrxhfHaZqGH7qh2lK2cPEaHeea93KUFv5FMlqW3s8iut+JlLEwEmR6UEsIrcS3iTL0S+F4108reMhaIZA8uz5QeS4/piF1uL6Gsr3AE9vFwkqHVrjqI7GttOEleL2YnDpR0rMPjVMMk= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 5:VH5XluJZpzvxo8w5vNpgkaBk8G86SEsuvpvC2hAm5Uhrod5yNPbNYyHNBEqAol03tyjC+Csw1lkY4yxIeSeYJFGDug58iBaBsr5Sxlao7/KUqJvbPBqXftuwZTF9tYN00iyOSAlq7PmJMGmZALO3aUtDZxKvcjEKiBIDItMk9JzwGxqmJiWtBeLXE8CEzNGPfxgC94dNbUb/f8TSNb5ZeuvpjmC/K8FNn7nqy4aQTHuW6s2uDA3C3tBs76fzp0dOgjw6pXqQwI4NdzHe/Wa1VI1XzXQuYhb9xVkt6Eml9JiXWBoaVcO7cVvD7EvIvKFwN/xN/05xDhm9rxGmDJS00OiwE7llrHTUbIcJ9GSLgLG96wFqQOuZRJ4WW6BAjoj8hvktC1APiIZ/7sdnEDZL0dpv7+6Ff3mW2X6qTdXPOoWRXiVaCJ9cdecZk1CooIblkHEDmzomLLTDIoTyH1D2b37NKvSW4UKXD6r+R9nRcd/DxtCrY6adrhDLPaluriAU; 24:26ycGo2PqwS+22ghaIz1y+JCPgZYussnxzQIdIjaEEz/0EgGjLteRLF38Wa9lfKLVN6kfYeMGFTWQ1L3/tOKgwRcAcFVn/LR7IZ9S8r4fSw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 7:rYWLxHVDwYer2af+MTmzB8p+rAk3yn90hAjCgJ2ygNgNaTH2Jf78mA7JkphEWx8PM5jpwUtjilLpGhbJtotHe71532nte/ohygtZyjuTiyZ8bMjww0TNrxVzk6L+UpX8PuDWH/d4G7j9DSg6E3tHHJeRxhLGstTHqgg6tYMhrB78y64b6Q7JgUFoDJN8Uh0WzPyhCnSXsGIaaLL+2uI/VeedbJKPARLRislu00n/3O+dMyx/5GIILw6Z9wjaTzzZtUbV7nJ/0mAf5zj8PrkNq4bdUHJkbcIYqdlOkFEUJMyxppiFVhUZKIzn56FHtMTIvLPZYkSvLsuFS1OqG0QaPw== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2017 16:33:20.8987 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0301MB0743 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2017 16:33:25 -0000 Thanks for the response. I happened across r307264; however, my understand= ing is that it applies to the FreeBSD 11 and I'm using FreeBSD 10. Note that I have experienced the same performance degradation on a FreeBSD1= 1.0-RELEASE-p1 amd64 system using 2x Intel 520 SSDs in ZFS mirrored pair wi= th server an Intel Core i5-2500 with 16GB ram and running PostgreSQL 9.6.1.= There's no memory pressure in any of the systems I've tested on. -----Original Message----- From: Slawa Olhovchenkov [mailto:slw@zxy.spb.ru] Sent: Saturday, June 10, 2017 10:37 AM To: Caza, Aaron Cc: freebsd-hackers@freebsd.org Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performan= ce drop < 24 hours On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > Gents, > > I'm experiencing an issue where iterating over a PostgreSQL table of ~21.= 5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds on I= ntel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back from Ja= n 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm using Po= stgreSQL 9.5.7. > > I've tried: > > * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 2017. > > * Tested on half a dozen machines with different models of SSDs: > > o Intel 510s (120GB) in ZFS mirrored pair > > o Intel 520s (120GB) in ZFS mirrored pair > > o Intel 540s (120GB) in ZFS mirrored pair > > o Samsung 850 Pros (256GB) in ZFS mirrored pair > > * Using bonnie++ to remove Postgres from the equation and performan= ce does indeed drop. > > * Rebooting server and immediately re-running test and performance = is back to original. > > * Tried using Karl Denninger's patch from PR187594 (which took some= work to find a kernel that the FreeBSD10 patch would both apply and compil= e cleanly against). > > * Tried disabling ZFS lz4 compression. > > * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL 9= .1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime an= d test took ~80 seconds after which I rebooted and re-ran test and was stil= l at ~80 seconds (older processor and memory in this system). > > I realize that there's a whole lot of info I'm not including (dmesg, zfs-= stats -a, gstat, et cetera): I'm hoping some enlightened individual will be= able to point me to a solution with only the above to go on. Just a random guess: can you try r307264 (I am mean regression in r307266)? This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Sun Jun 11 16:51:18 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CE13D8A8B7 for ; Sun, 11 Jun 2017 16:51:18 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0079.outbound.protection.outlook.com [104.47.42.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3BC286F431; Sun, 11 Jun 2017 16:51:17 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=f10qQohDYboXn0G1k/EqtP3PxWMKvTnPc9VsMUJrtG0=; b=vbcZVgOCYJ4ttD08fG+rGGNfcUrzY3uSZnvSdZ6RJUKyqN01Ppq4bxeUjj32GLDtnNtos0H5eE+k3Zz9TVge9bVzSUDRLIoLfscRqYEMaMkZv7YgsEJqcqv9fwmfpuuPaFh8kSVHIG8OdmeKPTrMiSpAzsGAOfRRhQ0GEv4ehL0= Received: from CY1PR03CA0023.namprd03.prod.outlook.com (10.174.128.33) by BY2PR0301MB0743.namprd03.prod.outlook.com (10.160.63.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sun, 11 Jun 2017 16:51:16 +0000 Received: from BY2FFO11FD033.protection.gbl (2a01:111:f400:7c0c::101) by CY1PR03CA0023.outlook.office365.com (2603:10b6:600::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Sun, 11 Jun 2017 16:51:16 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BY2FFO11FD033.mail.protection.outlook.com (10.1.14.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Sun, 11 Jun 2017 16:51:15 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sun, 11 Jun 2017 16:51:14 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Sun, 11 Jun 2017 16:51:14 +0000 From: "Caza, Aaron" To: Allan Jude , "freebsd-hackers@freebsd.org" Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLi0u6ENydglNrDQ4GNB7pMA3Leng== Date: Sun, 11 Jun 2017 16:51:13 +0000 Message-ID: Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.110.68] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB013:|BY2PR0301MB0743: X-MS-Office365-Filtering-Correlation-Id: 7e34c11b-4227-4cbf-4cf9-08d4b0ea1307 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB013.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39450400003)(39850400002)(39410400002)(39400400002)(39840400002)(39860400002)(2980300002)(438002)(13464003)(24454002)(377454003)(199003)(189002)(9170700003)(966005)(55016002)(86146001)(38730400002)(72206003)(2906002)(42882006)(189998001)(551544002)(5660300001)(478600001)(8676002)(97756001)(9686003)(6306002)(450100002)(53936002)(23726003)(86362001)(6246003)(102836003)(53546009)(3846002)(81166006)(46406003)(8746002)(356003)(8936002)(108616004)(47776003)(24736003)(66066001)(7696004)(33646002)(22756006)(5890100001)(2501003)(54356999)(50986999)(2900100001)(305945005)(7736002)(229853002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR0301MB0743; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; A:0; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD033; 1:lP2lrgj9AAO4oUdTb37sudYm/WHbUloo2KxNFYDUtlKNUKZdHDqnLecDZBdTZMernUAcegCihWCYvCqAlez0oTwZqINkj5C/z9lPlJkXJifRDDbcYanMoGIqFKxpjJVkrDdD4kUw3JFmkjYqXqNOUOsbddPUQUYVC5P/Rrd5BhrsgOp75lw70tKHtQCyy9x9daxAQ4DTembAuHXpF9nWnHcoUN6J+33PfU+BuFCLvKy37uJ7Jn058oZ7EKq757n6kSNX13jOcW6v/YmCVvyf/M9WxSCHc69FBCYSJt3WxGSSx0kVeHz0YB27X4JBvHqWVxDtR0Tc9sGn4VH1KK7I1mbSHInI+4cgl6lYIPlQF9cNSLN4lSeLM+5CWLDIoWZee4gqkjfsIcROVND2ZXieifTaIahnsx+vvEqoCt6o8HCv+3vzHLERl6PFhIhbIfPu+RTyd6X7JyQY8mJlXJG2TJ+0K0x1c+fA/37zkovmKBp/IzIuT8VTYofMX+wnHz7hotPo5ECnxvcizuBirXA91w== X-CrossPremisesHeadersPromoted: BY2FFO11FD033.protection.gbl X-CrossPremisesHeadersFiltered: BY2FFO11FD033.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:BY2PR0301MB0743; X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 3:8jR6RJB0ugLz1LnafpxQ6KIOpGmk4AegnguSsa2n0kndoC9YGhj47heGkg+mgVSDZTe9+CQyM0mhBUU4mm4e1lbUpngbYTK+VFcW1r0D7JqYGZKhc+q2+TbmBZsefCQSnp2ybkOkJtMogx9x89XR1CoH1aTuH7dHhE/Z10yDBz7atM6zygIMykBKILvsWbwztvet7h/HsiDYSz67dr8WpObkO329LKbc3Pvy4Uee2AziSbY/rMdU80wtVShjiDJCRInQCmEMq39KAgd3aLjmP3xx9nLvqZpYAF4VgZ2ciKoSn9DtkPDev1dKmWuxkinZlpu++USiKKJVmSZfkwA5CU04nQiKNE8C6pTeRn7Isdg5LjykqtrvsQVS9/d6hFnuACbG+XV/lmE74rT7nkuKOVFX5BrwlDU9cAqKMXV5zQCIlgezDAnrqmYjd3DUK6QmUxILu2R8jz+LV7DOgYkVygMq3LpH8eBsCevgIf0c00s= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 25:XgdUfeERWxrCHURXjTS7vEjxtUq5zYzJyLDKJ7E+xBZ3H4EP91uJXapaYmPd/ofi/YQJagCh65G8kDlpRWk1cyXaa33duU7Fkeqro/+Jm8AqaFvz3V+yBu50F1KvpVNb5sdn2Y8F28PdTL11Ipk6RvdSU+FtdQf026HpBEXkkSRnsTACAWXRe5N3ImeqROSTc5LK0vDC4tUsr7etB5t5pbiGLyFipJg3+pPH8tMNikhNNC1crbkm5KZaP7XBjLl/83RgnQeBgzjLX7ktClCzIxsrBBRK7hzbieQJMQFGwQr1m+4tZnTaUs3jpUK46hOnrXF81pw+SwQfMhkk8bjULgOBYxdAaq+KqkZMQU6y60e+TKQhFL8bJ84m591duS8JGi5HA3lCK0eAfmys5P5lWtdM/nR0Sy0Od78m56a3VL0v5vv9Aj85z9y/xAxI8ZNmh8dfG6ptSTp26qeEjbMLJbX+0cC7zmaBEveiBzcbarM=; 31:Xl5TQD0Rfay72gQoS9os461jO/TzGcVNjOv0TttfKrPVXt9UwjFe8gaGY3mYbKZ2e9VhbGApHxK4tCJAWLPN2gVDRtQKxrhILpcCQT0dEyTq5B6AFbA10dG1AwbVMroT68LFSnSZOT3p28TXMTGevOzKhKQZodCDsDxCyava/x9OcGhmPhMqW7HXTbe6C9R3EHJ0Z+P8IF9m1PQHrgqM0B2cNFA6mgk2KQVhwyNiAgMYac5kVO+duwAtSa1a8rgORzziDFEccUmhZLTXGhmkMg== X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 20:xYEdzOpFlYeIOG9XFP1vIlzxHB2v3RejCmp0324qiAlpUh1CpA64Z1umS9GDMWSneqM3LIk/tzBov/9kp7NH6kpFOmvM8DdwCbGoUK4GLuAnpRwsOqmGq6lGJ3GTcyS11EJpTO6A/F64Pjb0pscm77eXwHTA9HApk2yNSVYPHisxVS+/gwoC5UVoZCOFFKzUJ6zeVbfTk8mfBX/3iydAuT9o/FoeeFGW3bgwlbj+s+KpMHS1RswMf3KU3Yhv/u6S0pixzeNzuMacbssm/p/woFh0W0T+C8UNIMrbTO4IbStIJPv0t09GDpy/LLvDUbWK7fBfxmkhAbZDmGLulcbYJ93GW7QIPcoxSv8esFzxIhVnntU1BDjrYh2zEyeUz6jhZToEUOcsJt5Lj+teBAI2asNbk7v7Kw1zJvcsNUMHoDvOLbf3dL4c3D+5Yv5vBIETyz+duQL0qzv1RNAmX3g6ZuMhj8kickw56kZAwNtnfvQTSq5YlBGVKGVQUb835uOX X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705)(75325880899374); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(93006095)(93004095)(3002001)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123562025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY2PR0301MB0743; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2PR0301MB0743; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0743; 4:sDJ14HVy6YwIlm7QMNh05GG4eMXVzicK9PyizGRs?= =?us-ascii?Q?IlSvMOWbMtRUcXRtzfrtzFMnj+4TtgEYnWAke89iigZaxgJCjPo17q6dBjCr?= =?us-ascii?Q?lcsim0KkyFrV9J5pBHnQiK5EcQjtPAr4ay2sO917dIYjqkx6M6KUqXZ62fuZ?= =?us-ascii?Q?gnQXlfchkY7z0Ke724NAl5+Ez0JxjaCcPRLwOUFLIdrXUvzyv+ZvHRdN3laH?= =?us-ascii?Q?G+4I0gRtGhOKlwQO7o1dH4uyojOmG/piNlFoCD+avMWLObMADq7mmWL4ozLY?= =?us-ascii?Q?bJXxZMRvMAXaS5byplaDI9r2TFKeHlkoTIzZUu41x1YGndTDsNIlmCFSRpjD?= =?us-ascii?Q?i33uH4ZH3SVRmxvuQ3wG/nOmstA4Wyb/a+tPz34KccgEtX9Yd2YSL23GAG1F?= =?us-ascii?Q?k2WVyCmiG9vDHaHfg2bB7kU30bXntZr7B2Lr5p+z82C/caqoRjI0acRvDVQi?= =?us-ascii?Q?F7crd2EG2D5MujQMqrFDADnnJ+63gr6FfUHLueo494VRBCf25+0XRwmTXcwm?= =?us-ascii?Q?ifRJRbxdIHQwIJJqmMG5tggnujt96SEuZLCs1+w/payLQoKRg8Xv8uEpL5pQ?= =?us-ascii?Q?xML9Pagfxd99GiVfltsV+p0KKJTwX5B0nYa0o186Ox0qvdCkDdYrKYwH3Jd0?= =?us-ascii?Q?i7rcUMgRfBSElWPwoQIC1cFku6/q2ALdWYZDLsV/Q9lB8vZwk6VzVgJh3o4n?= =?us-ascii?Q?w3ZQEJwCRgopDC5DvsE6vWrZCLDR2EE7RMJORtKT14y60GbX4r4k7l4F51Qn?= =?us-ascii?Q?VFmBKy4ZuLPPJL+GK36OTQccn3Y5ODegBtgoXriHhvShIqcTj+m3ADnQee1e?= =?us-ascii?Q?/NGIFFjDxhZzayM6l84y5tYjbV2FUuRTkJw3sfUSkRtcSTRR/ECBT/LywWCD?= =?us-ascii?Q?F/4NiqhPyoVZQYiQ0zHKnuF36SdT5sYkAzHuPIZz+kUuXwa41LYAqZBYCjU0?= =?us-ascii?Q?3aiBWfBCZibeYqKADqHUNQMSQsTYjg78/Uh3b6cvMfa28BJoywLs0tzF4liL?= =?us-ascii?Q?s5sF9Jnxw4wyKfratj4kT4/n5FlsehnyD+AOUBal6b2bO1xtdDnggRb94I7u?= =?us-ascii?Q?WPUxMsL408oAvPmKgCaCOyUD/YXOERmEfjGi9i7D4dx3DKavOOAEw9idL5x+?= =?us-ascii?Q?0acOEGSM68ek8bl6m7bT044jNtt39yOUGny9VVfL6RX2Lm8/vJUehPkMra3M?= =?us-ascii?Q?tzce+Hl7gzG8jxSaC4EA76s4r5/GDO2GhLuww0vpmK6Y0sMJ+KRMWzxBLUFY?= =?us-ascii?Q?lMKFlCm2Xd6kNk1578eKIcfh8u9oLlKB5ciTtxTAjX0mW9AR18zbxFH5X0O5?= =?us-ascii?Q?vT83UJ/7qLSyHKL1DTeVLIXCBgHntoDs9QsMFGzwW2vF?= X-Forefront-PRVS: 03355EE97E X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0743; 23:G7Nz2h/OKiiZ/XAjDfgOScNMLBDoRYLxCuPLbYD?= =?us-ascii?Q?wunwM4giXcYMXpOY/wOiuZfB0V8cKg9R0qd73JTz3o93SXpslJcLsY70P0W2?= =?us-ascii?Q?K8Iesx+bo1HaDdW/2gu+NEVyefMNOk8R2/Yk+ApS/k4Pdk5yNIvzYEr9mUXT?= =?us-ascii?Q?4bilvEMVsbaOfa5U5/emQBm/1UrrwotFqwR+JDeXIxXh2Q2edIlvGT2CQFKD?= =?us-ascii?Q?+BcA8oKP0KL3sgjQxTIpnFVgVmZelCdfn8GBEioOA2NTOwE5xN+Etx0NZAC4?= =?us-ascii?Q?1QARnV/CSkj7J2MwgkB3nFxjs5m8sSz1JUzi72/DqWfgWnvR1d4xnPuMdkqn?= =?us-ascii?Q?Cnbv7AWvJsAuH8oKqckI+L991m1HzeCPUFNpcG63mP+Fnyrr7ni9QQki/lO0?= =?us-ascii?Q?1YFuPUD6WfxUPX09HV7AlNt7ALq0hIhRSUmqZaawLZl4yDMWKzBstVWmkxv/?= =?us-ascii?Q?7/BS7oh/Kg/zS/IJFkS+9DR9SUDL1yO2JhXASr5cwmZulzhZoDF4ksnAGEeb?= =?us-ascii?Q?S05UidUkZSD6TntKyj4sLNgw8I9fB3RF2l7QdFDgBpYv0R3dHmpgCU0Zrg43?= =?us-ascii?Q?Ke6XS5H6jGzUVb6pS22NspgciZaMUrGaOXQKTiS0MTmMQrZQ+EvtUjpmjrCH?= =?us-ascii?Q?OC71G4/RRpx888YqmYt0qWxVIrT1lLWEGhaNFj+V5kh0VmCrPZuuKsW3vYN6?= =?us-ascii?Q?OyrdUdwoKK1+jUu5DStiD61TqqHj7XTZf/K/z1IFL8TP0sPKlmsv5J/mU3AU?= =?us-ascii?Q?uBnqwiG8trHa2G4Hb5aESZ46xm07WtwqMMPUc+BipPvvf43ShthW4BZouI1K?= =?us-ascii?Q?F5uaqtBS47yzet+gL1oJojioskzTDkehbU18i7AAhcZYMnGt7xMl1Dd12vzr?= =?us-ascii?Q?TU+NMVnMvFVpkarjfXdDgfP4QDbhInc5uBz1ldaJQirir7pX8a8S9+mww5MI?= =?us-ascii?Q?8BkJVVpOZ71DN4+LJ5+nuAC5JFqq+4JSnxPovbOp4MkyJAz9WMh5yuR7rUHj?= =?us-ascii?Q?d3oOxWrtqRuFM3FTKi679sp8SaKPjcuc/WZpYAkc3Mb/U7Qo1D1cbbB5VZxA?= =?us-ascii?Q?AiA2rSCWyCJHjTtWfrGmx7iAfSkKWQKiXPptzR9l6KiuO2LMvABDX8/Bn86K?= =?us-ascii?Q?1D3NfLmelVECOufCgBpLI4rmDJauR/MBwGrbK9dE8LEUsa8qX1gipAndnm36?= =?us-ascii?Q?MixtlYxhbSMmGhDJRcm0Ts5gYLNb/UUEMZKsgmQU+7QMTrskB7H6jQyOx9Se?= =?us-ascii?Q?744c1b/hEHCjD//ijrTBTT+yooLb2beiZK8Tic5sBGNA++R+2x3CLSOyFVHM?= =?us-ascii?Q?FvY4cX9oBE+nGF6urpxY6KGbapuGbN61WWuGpCHnruW0jMRiLaGsFBS7Or6X?= =?us-ascii?Q?yoPyV6djaLtdkMVBtyPOHCN8f8ktluTzlJRvoRe3EYvfDVdvd?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 6:4SX7C7HX0+7VjFJUt05++8IdkQm5IWd9xHqTCPOa5YQL6tC72NH7cx048exlgJ1e8aPFSA222SwJHiyPaFxbJMqkPfhwEA8btO22krZW4AFDNHonC0N0f5SFKgIDBW18ATJ32JkPM1Q0DCxz/JVzEtSfAu9ecV1zgugb4NPekJa32h2eW0rTjLBVBa5424hbfoJrMdy9vh5qgq2kf6s/Gf0pQJJ4EOxsE0Pi0k6pXVw+JSIDfIyZncbNo7D0Em2f2WNod4Fv7a0yhBH4iMBhhMFX3dn5mK43Kgqsin/yq1sIgUJVr8N5sKXvcWcqKV7G6ZnowvotYRbNnmpGyJtZ6ejywQRvA1s8Okp5wHMthd220uHndmpmIqEOVONYrGVscwXUT4AEWS8jHQLQF+erUJJaECTukZoCaZrLMU+Sqdxhqr+PwCYyvOYpwoqLDi3G7EqglloepeYbu3+nH6/FXi9BzjPumCcCPV4p4GQ0ejR2gya6dKpEHPa2/jYZj5CjaYyWibwhwUVRNgBfqoGongOr8iG3fp9bY6hk2tE3SfM= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 5:9O1GM1YSknxnxdhsF9/EwZzCPLFTY0fI2Nyo6x+2q1GulUuUVKzt1mSigzOtnd6GVA6u/kmZ5BKlAzipGu+kKoyTk7O8GUzmxCcEr3fMDrw7nbQaPbjO9tU2rIEZyieycutj2jD4Hk9UtpQvHcp3j+UOqL+kOt8r4zhcPoAlgu2njxEpiGjicbcG4Xe6Ye15Cqwi/JJL44uJEXF5StkmS88jdtfAEhcdJPdg59qn6qPF0OcJtsJZ2P5AZgpbzhDUIZtPV8DzCqEIZgOj2sdfbxxSp000blFeHvelvL5e+XYASt1viXUbKgJa5ualViBfDse5D3OglFId7m+CITHz3ciV8ICoIiehlH20kvugUTQJO+hsKLMXCebwK+ASOug8/SLOqDBdFNWEr/g262vMQo+ryCQ03ba/PVpmYEeZaY2BPP3RbtieTA6dTDqZVMvUTNXS1V9YZRfGsk+QC8qmtde8NcO/pQmX/PPCsCqrJ6990KVHi06vIP4Ng9RHkOqg; 24:932RW/7TjqG80W/hwbzx5vfx26qPbwFWHhgfcNNph3QAxSQNCStM6jvIvGIGrzThFlJR24S7Swjsb+GaSo72+aTgYLyqjPzax/BELifHxmc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0743; 7:PTwCoUS1GHaOR8MMlGhYVsf1h2hsTe8LG0ZrHKMr48Xo+kmaAu2o+Dxs4ngm+s7C9SFgNHnvKxMmc5jkfxXCZw3YLGxLUwLV0/52BzN3SyES05NFV2XGLf4rsFmWEYB675ss4w3Ee2rN9XZVa54th639bxqUNRO+9XpnAzsAz7O8BmMzJWh3C6Z1qdBuOEBYNP6h9EzcBGebkuoLp/Wv5ZCbeHOBsIPI2+YNZOIIZEfXsvhqh3nyQ6kJHMGu9ez2WamfoiNlJh64IzrHRvVlbuZZz398A+xGlYTBCBgQn+qN6mJXicL6oIDO9QMRrjshyAn1TpkrmAEiW+zIQ9MPag== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2017 16:51:15.3227 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0301MB0743 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2017 16:51:18 -0000 Thanks Allan for the suggestions. I tried gstat -d but deletes (d/s) doesn= 't seem to be it as it stays at 0 despite vfs.zfs.trim.enabled=3D1. This is most likely due to the "layering" I use as, for historical reasons,= I have GEOM ELI set up to essentially emulate 4k sectors regardless of the= underlying media. I do my own alignment and partition sizing as well as h= ave the ZFS record size set to 8k for Postgres. In gstat, the SSDs %busy is 90-100% on startup after reboot. Once the perf= ormance degradation hits (<24 hours later), I'm seeing %busy at ~10%. #!/bin/sh psql --username=3Dtest --password=3Dsupersecret -h /db -d test << EOL \timing on select count(*) from test; \q EOL Sample run of above script after reboot (before degradation hits) (Samsung = 850 Pros in ZFS mirror): Timing is on. count ---------- 21568508 (1 row) Time: 57029.262 ms Sample run of above script after degradation (Samsung 850 Pros in ZFS mirro= r): Timing is on. count ---------- 21568508 (1 row) Time: 583595.239 ms (Uptime ~1 day in this particular case.) Any other suggestions? Regards, A -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freeb= sd.org] On Behalf Of Allan Jude Sent: Saturday, June 10, 2017 9:40 PM To: freebsd-hackers@freebsd.org Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performan= ce drop < 24 hours On 06/10/2017 12:36, Slawa Olhovchenkov wrote: > On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > >> Gents, >> >> I'm experiencing an issue where iterating over a PostgreSQL table of ~21= .5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds on = Intel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back from J= an 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm using P= ostgreSQL 9.5.7. >> >> I've tried: >> >> * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 2017= . >> >> * Tested on half a dozen machines with different models of SSDs: >> >> o Intel 510s (120GB) in ZFS mirrored pair >> >> o Intel 520s (120GB) in ZFS mirrored pair >> >> o Intel 540s (120GB) in ZFS mirrored pair >> >> o Samsung 850 Pros (256GB) in ZFS mirrored pair >> >> * Using bonnie++ to remove Postgres from the equation and performa= nce does indeed drop. >> >> * Rebooting server and immediately re-running test and performance= is back to original. >> >> * Tried using Karl Denninger's patch from PR187594 (which took som= e work to find a kernel that the FreeBSD10 patch would both apply and compi= le cleanly against). >> >> * Tried disabling ZFS lz4 compression. >> >> * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL = 9.1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime a= nd test took ~80 seconds after which I rebooted and re-ran test and was sti= ll at ~80 seconds (older processor and memory in this system). >> >> I realize that there's a whole lot of info I'm not including (dmesg, zfs= -stats -a, gstat, et cetera): I'm hoping some enlightened individual will b= e able to point me to a solution with only the above to go on. > > Just a random guess: can you try r307264 (I am mean regression in > r307266)? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > This sounds a bit like an issue I investigated for a customer a few months = ago. Look at gstat -d (includes DELETE operations like TRIM) If you see a lot of that happening, but try: vfs.zfs.trim.enabled=3D0 in /b= oot/loader.conf and see if your issues go away. the FreeBSD TRIM code for ZFS basicallys waits until the sector has been fr= ee for a while (to avoid doing a TRIM on a block we'll immediately reuse), = so your benchmark will run file for a little while, then suddenly the TRIM = will kick in. For postgres, fio, bonnie++ etc, make sure the ZFS dataset you are storing = the data on / benchmarking has a recordsize that matches the workload. If you are doing a write-only benchmark, and you see lots of reads in gstat= , you know you are having to do read/modify/write's, and that is why your p= erformance is so bad. -- Allan Jude _______________________________________________ freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/mailman/= listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Sun Jun 11 23:09:36 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF17DBEF18C for ; Sun, 11 Jun 2017 23:09:36 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-211-155.reflexion.net [208.70.211.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 955C07A158 for ; Sun, 11 Jun 2017 23:09:35 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 26194 invoked from network); 11 Jun 2017 23:02:55 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 11 Jun 2017 23:02:55 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Sun, 11 Jun 2017 19:02:55 -0400 (EDT) Received: (qmail 13073 invoked from network); 11 Jun 2017 23:02:54 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 11 Jun 2017 23:02:54 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 33979EC8074; Sun, 11 Jun 2017 16:02:54 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: A different 32-bit powerpc head -r317820 panic on old PowerMac G5: dual backtraces from "timeout stopping cpus" (dump failed though): any comments? Date: Sun, 11 Jun 2017 16:02:53 -0700 References: <1F1E52BD-375E-47CC-BF06-ECB1092121B4@dsl-only.net> To: Justin Hibbits , Nathan Whitehorn , FreeBSD PowerPC ML , freebsd-hackers@freebsd.org In-Reply-To: <1F1E52BD-375E-47CC-BF06-ECB1092121B4@dsl-only.net> Message-Id: <29CCA1EC-242D-42E7-97E9-6F2F67178DF3@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2017 23:09:37 -0000 On 2017-Jun-6, at 11:09 AM, Mark Millard wrote: > . . . > FYI: I'm currently doing an approximate > binary search for localizing part of the panic problem. This effort failed. More after the reminder of the technique as it was when I started to try this. > This is based on the classic panics that are instead > from jumping to a non-code area. . . > > At a given point in my other experiments I was > getting: > > srr0=0x90a0f0 etext+0xb8fc > > Adding (unused) code somewhat before that etext > (so increasing etext) got: > > srr0=0x90a0f0 etext+0xb8a8 > (The additional code was larger than I now use.) > > But instead adding some code earlier (by around > 0x100000 in this example) got: > > srr0=0x90a110 etext+0xb8fc > > So comparing to the starting conditions in > each case: > > The bad-address accessed in one case stayed > constant but the etext offset decreased: in essence > the only thing that happened is etext increased > (matching the offset decrease). > > In the other case the etext offset stayed constant > but the bad-address and etext increased by the > same amount. > > . . . > > Currently I'm adding code by adding: > > void HACKISH_EXTRA_CODE(void) {} > > to one .c file from /usr/src/sys/. . . based which > file gets to within a ballpark of a more accurate > binary search position. (Large binary search > jumps currently: I'm not being picky about where > in the .c the addition is made yet.) The reason for the failure is that the behavioral changes and failure modes changed depending where HACKISH_EXTRA_CODE was added (over a very wide span of addresses for where the code was tried). Overall I was unable to have a criteria for picking between larger addresses and smaller addresses in the search in a way that targeted getting near a boundary having two specific, distinct behaviors on each side of the boundary. Also adding code to panic instead of accessing or changing inappropriate memory for failures seen in some failures again changed the behavior observed, no longer accessing or corrupting the same way. So for the binary search I had to revert such extra problem-detection code. Very memory-layout dependent. At this point I'm not hopeful of providing any better evidence than I have in my various prior list messages. I doubt anyone can pick anything out based on just those from the last several weeks. At most if something is noticed the reports might be able to be checked for "would this now identified code-problem have possibly contributed to those reports?". (Even that use seems unlikely.) === Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sun Jun 11 23:50:26 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82C78BEF8AC for ; Sun, 11 Jun 2017 23:50:26 +0000 (UTC) (envelope-from amutu@amutu.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 303547ADB5 for ; Sun, 11 Jun 2017 23:50:26 +0000 (UTC) (envelope-from amutu@amutu.com) Received: by mail-oi0-x22a.google.com with SMTP id s3so46826868oia.0 for ; Sun, 11 Jun 2017 16:50:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amutu-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dLDQqbnbFFlgHlKkpD7vKzj3jHHzqMXibcwl+YU7nAM=; b=i7eQ+XVFnRsvpMmP5P/WgOHyKkKG0DyItIf4zXXvQbi8jSTpuptP+fvDYByB6rW0PQ VrJdW9l78vf0I+9v+XeAGIJM2CpK/DspfL1f70ZU6YKNgu8WC+Kl0lMgWqWVNwVKwvHr C+cAPYdFO1xEiCoXWM3D4J9+wWrbha6dFZLCs/FXuSgizFdr+XsSqnJ609XO41RXFoU0 EQZAaoWkqZdpFg9oSaStHV2wciziu01Bom+7pEISJ5KDomTMkjpM/SdjlKnBNNoYlu1P X2ugPwcSXZfugBS/btqFzZ90B8VRqE/VRVNb6Mx3Yh7qpfWHdj3QnhTHBq+GRS066rg1 pyPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dLDQqbnbFFlgHlKkpD7vKzj3jHHzqMXibcwl+YU7nAM=; b=aLtJ1rjaUSUuF9I4vWq+aphq/9XrhLyZ/PsYqS/ZdJA5P5dA78BNacY7A3/5WRATpU tlh95ywYVUKtdzEaMrtsnQMmcdTYXVAxa2UFQkMXPxIPxPdkGl8qKAJgYfyGut1a27v1 d+OzHiqIeAMxFzQ48vW6MY+wn8X7ow9Or1dlWvZPIs1R+UlzXBKMS+3yJOr2IZyWXpdL qQ+sVyt+ZvCVamjLdgFQa5ZXyTAKUJ0YLPg4cisMYzqflg/RYvIhVv39CuzGQqwx0jOP fC6Wbea8nkHeMeTrZwUCfwgRqGcX54k904FfiWMpSvlhjeF7XB8CJ7RaXpaNwxx6me9k zBPg== X-Gm-Message-State: AODbwcDGjZqRjol/vHqN8UU4EBggIT1d2nnvE5wQVzqxKUoaZ5Kjc4ui RXY1CxC4UbFDLOBsdeq7xQ== X-Received: by 10.202.188.139 with SMTP id m133mr28912287oif.12.1497225025335; Sun, 11 Jun 2017 16:50:25 -0700 (PDT) Received: from mail-ot0-f181.google.com (mail-ot0-f181.google.com. [74.125.82.181]) by smtp.gmail.com with ESMTPSA id d27sm4134130ote.41.2017.06.11.16.50.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Jun 2017 16:50:24 -0700 (PDT) Received: by mail-ot0-f181.google.com with SMTP id i31so59013904ota.3; Sun, 11 Jun 2017 16:50:24 -0700 (PDT) X-Received: by 10.157.9.35 with SMTP id 32mr5154049otp.118.1497225024430; Sun, 11 Jun 2017 16:50:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.133.136 with HTTP; Sun, 11 Jun 2017 16:50:23 -0700 (PDT) Received: by 10.74.133.136 with HTTP; Sun, 11 Jun 2017 16:50:23 -0700 (PDT) In-Reply-To: References: From: Jov Date: Mon, 12 Jun 2017 07:50:23 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: "Caza, Aaron" Cc: freebsd-hackers@freebsd.org, Allan Jude Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jun 2017 23:50:26 -0000 To exclude the fs problem=EF=BC=8CI will do a dd test on the pgdata data se= t after the performance drop,if the read and/or write utility can reach 100% or performance expected then I will say the problem is not fs or os. For pg,what's your output of explain analyze before and after performance drop? 2017=E5=B9=B46=E6=9C=8812=E6=97=A5 12:51 AM=EF=BC=8C"Caza, Aaron" =E5=86=99=E9=81=93=EF=BC=9A > Thanks Allan for the suggestions. I tried gstat -d but deletes (d/s) > doesn't seem to be it as it stays at 0 despite vfs.zfs.trim.enabled=3D1. > > This is most likely due to the "layering" I use as, for historical > reasons, I have GEOM ELI set up to essentially emulate 4k sectors > regardless of the underlying media. I do my own alignment and partition > sizing as well as have the ZFS record size set to 8k for Postgres. > > In gstat, the SSDs %busy is 90-100% on startup after reboot. Once the > performance degradation hits (<24 hours later), I'm seeing %busy at ~10%. > > #!/bin/sh > psql --username=3Dtest --password=3Dsupersecret -h /db -d test << EOL > \timing on > select count(*) from test; > \q > EOL > > Sample run of above script after reboot (before degradation hits) (Samsun= g > 850 Pros in ZFS mirror): > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 57029.262 ms > > Sample run of above script after degradation (Samsung 850 Pros in ZFS > mirror): > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 583595.239 ms > (Uptime ~1 day in this particular case.) > > > Any other suggestions? > > Regards, > A > > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@ > freebsd.org] On Behalf Of Allan Jude > Sent: Saturday, June 10, 2017 9:40 PM > To: freebsd-hackers@freebsd.org > Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD > performance drop < 24 hours > > On 06/10/2017 12:36, Slawa Olhovchenkov wrote: > > On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > > > >> Gents, > >> > >> I'm experiencing an issue where iterating over a PostgreSQL table of > ~21.5 million rows (select count(*)) goes from ~35 seconds to ~635 second= s > on Intel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back > from Jan 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm > using PostgreSQL 9.5.7. > >> > >> I've tried: > >> > >> * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, > 2017. > >> > >> * Tested on half a dozen machines with different models of SSDs: > >> > >> o Intel 510s (120GB) in ZFS mirrored pair > >> > >> o Intel 520s (120GB) in ZFS mirrored pair > >> > >> o Intel 540s (120GB) in ZFS mirrored pair > >> > >> o Samsung 850 Pros (256GB) in ZFS mirrored pair > >> > >> * Using bonnie++ to remove Postgres from the equation and > performance does indeed drop. > >> > >> * Rebooting server and immediately re-running test and > performance is back to original. > >> > >> * Tried using Karl Denninger's patch from PR187594 (which took > some work to find a kernel that the FreeBSD10 patch would both apply and > compile cleanly against). > >> > >> * Tried disabling ZFS lz4 compression. > >> > >> * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQ= L > 9.1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime > and test took ~80 seconds after which I rebooted and re-ran test and was > still at ~80 seconds (older processor and memory in this system). > >> > >> I realize that there's a whole lot of info I'm not including (dmesg, > zfs-stats -a, gstat, et cetera): I'm hoping some enlightened individual > will be able to point me to a solution with only the above to go on. > > > > Just a random guess: can you try r307264 (I am mean regression in > > r307266)? > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@ > freebsd.org" > > > > This sounds a bit like an issue I investigated for a customer a few month= s > ago. > > Look at gstat -d (includes DELETE operations like TRIM) > > If you see a lot of that happening, but try: vfs.zfs.trim.enabled=3D0 in > /boot/loader.conf and see if your issues go away. > > the FreeBSD TRIM code for ZFS basicallys waits until the sector has been > free for a while (to avoid doing a TRIM on a block we'll immediately > reuse), so your benchmark will run file for a little while, then suddenly > the TRIM will kick in. > > For postgres, fio, bonnie++ etc, make sure the ZFS dataset you are storin= g > the data on / benchmarking has a recordsize that matches the workload. > > If you are doing a write-only benchmark, and you see lots of reads in > gstat, you know you are having to do read/modify/write's, and that is why > your performance is so bad. > > > -- > Allan Jude > _______________________________________________ > freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/ > mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > > This message may contain confidential and privileged information. If it > has been sent to you in error, please reply to advise the sender of the > error and then immediately delete it. If you are not the intended > recipient, do not read, copy, disclose or otherwise use this message. The > sender disclaims any liability for such unauthorized use. PLEASE NOTE tha= t > all incoming e-mails sent to Weatherford e-mail accounts will be archived > and may be scanned by us and/or by external service providers to detect a= nd > prevent threats to our systems, investigate illegal or inappropriate > behavior, and/or eliminate unsolicited promotional e-mails (spam). This > process could result in deletion of a legitimate e-mail before it is read > by its intended recipient at our organization. Moreover, based on the > scanning results, the full text of e-mails and attachments may be made > available to Weatherford security and other personnel for review and > appropriate action. If you have any concerns about this process, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@freebsd.org Mon Jun 12 04:50:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 648C3BF3CD2 for ; Mon, 12 Jun 2017 04:50:49 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0082.outbound.protection.outlook.com [104.47.42.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0777181CBA for ; Mon, 12 Jun 2017 04:50:48 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+1o9cNHc2M39CkIHo6nS+camV33ww8ZKs3UeQ64hBRA=; b=P+GvyI56MZdE6HJRmHMA9zgxabaYDGDvQjYjaPH3yB2hleDHtpsdiAVlqJ8FeMoJNdyslrwCyCn4j6pEGpIgKK2enzwU4eBPgDzz6vMBTjU1nYYqzHElSf72W4T/PZ0G8WLcnyZC3tUqMlE00M0SnmJjLTxRJ54QUZaK10wRyVc= Received: from DM5PR03CA0013.namprd03.prod.outlook.com (10.175.104.23) by CY1PR0301MB0745.namprd03.prod.outlook.com (10.160.159.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Mon, 12 Jun 2017 04:50:46 +0000 Received: from BL2FFO11OLC002.protection.gbl (2a01:111:f400:7c09::177) by DM5PR03CA0013.outlook.office365.com (2603:10b6:3:118::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Mon, 12 Jun 2017 04:50:46 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; amutu.com; dkim=none (message not signed) header.d=none;amutu.com; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BL2FFO11OLC002.mail.protection.outlook.com (10.173.161.186) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Mon, 12 Jun 2017 04:50:46 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB014.032d.mgd.msft.net (141.251.110.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Mon, 12 Jun 2017 04:50:44 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Mon, 12 Jun 2017 04:50:44 +0000 From: "Caza, Aaron" To: Jov CC: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLi0u6ENydglNrDQ4GNB7pMA3LengAOo8uAAAioTWA= Date: Mon, 12 Jun 2017 04:50:44 +0000 Message-ID: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> References: In-Reply-To: Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.136.196] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB014:|CY1PR0301MB0745: X-MS-Office365-Filtering-Correlation-Id: 28a670c2-d4f4-4d61-3dd6-08d4b14e96d2 MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB014.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(39840400002)(39860400002)(39400400002)(39450400003)(39410400002)(39850400002)(2980300002)(438002)(377454003)(24454002)(199003)(13464003)(189002)(9170700003)(53546009)(54356999)(24736003)(76176999)(50986999)(55016002)(5660300001)(7696004)(8676002)(81166006)(8936002)(108616004)(86146001)(42882006)(22756006)(551544002)(72206003)(2950100002)(2900100001)(6916009)(84326002)(5890100001)(478600001)(229853002)(86362001)(38730400002)(356003)(110136004)(4326008)(790700001)(102836003)(3846002)(966005)(53936002)(54896002)(236005)(9686003)(6306002)(512874002)(2906002)(7906003)(606005)(14971765001)(189998001)(33646002)(7736002)(6246003)(66066001)(53946003); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0301MB0745; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; A:0; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11OLC002; 1:D+gXJE1CCJFouXkQlLTJndembJLPufYd7vcVehcWe2mez5ZQooa7p4cNOUFy+JscTcaFvtKC0IncZ/vzpY3haIjo89SfmieWDi8/XgR//H42cRRH3qiLmUVy8jHPurGn2r7wns+DZrh69i0JbtPZl5/Fl56fxiuYqHewIddSZBGfYEE6QAto67vxCU/e/rQYj/Kyo8x3DB4UHX+GOxbqh9BsS8eLk/lBkRjOju3pGb0untwz8hGx+uKW5FP797BOmbGIaa1i2wE+5nKBaqMOzUZ3u03JeoIKNyALJ002bgp4gGDb3jrwFPigBmO92LkT4N+vMjgMENXh7ZE4FOHSM/NJkKtsyLDRtAe6tHNU7SiIEMXgp8rJ9sLS9I7eDNWLBJwpgg7lRrPsuTRswx02+zWhpd9UqIuPf1qtoOKQRZOAs7NIb7BWXKa/c7p4xXZxsx7QBQC/32/KgUbdCnGY59CKdrHUqsnPb4gSMDc5NzE6BhjVQvvVn1mjflvHlSKkMkOj/6TgUYVnphJC4JFkEA== X-CrossPremisesHeadersPromoted: BL2FFO11OLC002.protection.gbl X-CrossPremisesHeadersFiltered: BL2FFO11OLC002.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:CY1PR0301MB0745; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 3:NNtXsQ7VImfcnSoE2UC9V/4KhE+mrrfa3Wnz6ZyY0CFI7KyxEs3gvRZMfWzIZ3Y+IPkt0T2PLQ8yNk57UF1QuhMo2nOs/jMmEqZijlEx9PhrjwSuMM1rFKCa5xcrA7ydEmqutAsPGaPJSGI6xkRkiZnCRRwSEqYp/uleaMGhxCxv8uPHiUQzl8hzkNpfzxTXxdTETQEl1bu4/VvQt3Vck12XhMgz/8z24Zy51vzR59/nT+KvYGBlWbvsU3wESeFBfQ8BI+f0pKN5Iar0SdY7/i7dLI9d2Qd4+8pJlvsvRPdyBhNtDGLGRh8AH+iiojjZeSfhQVA37QgNi7dxbRNX2RiUSDzH6+Q9kxRrKeFHeR8GrkqSWj9FGCl0rhFUm1biJiBx1q6FV7YXocuLv7JESJwpuGFWqT9+LSR3nG3MpwOqhCCSokXIkcRdstXkqOf2wOqdGflH47F1fQAL4XwNNvXZkoT4ZjAfs5v5Vt9eNEmw1UpW5zeZjMdleX/CHRTce9StbrDjgsSnBn4Ttu382w== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 25:4W1rT1saFSrt6yjFDcvb7mW5uYDBZ5PJRFXi04m3bhcDaMmvd4tUezfCCtOrHMUGjSe+r48VqaUuaDP+awqwm/2aX1R5KwbG1yt/gGATWG/OhaaXL7PZLWD7nkIG5qdlNQfj+h1KCTxgsm6UAfkcXVVSNkzgG+9HuzPrqh/FGWlbPaFn76ublEjE8KQbMYRjbWEZjUo6k7eVWwqEMzLiWTvrvDe/+cdNevSRp3mZv69mxcXV0/vyEasaYfnhEJXOOnUzJGzNg4GaGKeSO+ZVyLxgdF+s5OcbOtIbraWabIYQdMdRC3M05vSAy83YPhy1ZiZHyFkJrjtdsPszlVWh0CgjYIw2INJ7chQG9HgJdMsyF6su/G28tnAdeK8FSc9IhJaNmq0EBxFgMW2MxUY1SpEgFhiTU9nfiJXuvdDGBA9mxb6navpJbIsJkevAlmRsP1UDq3SR2E1+t6gNczxLT37V7zMyfKI4NbRp7e05f/M=; 31:n3c1PcVi+aweoChWxflfQFL5UY9eEMmm9jhfM2Tkwse26oicdpnS2nvjxy0USpsIr7xVProev6phMUV0n0NHoBIC3nmzUG0DNjIPr3MiWiU3fXHolhmXXXnh0rx3MzvC0RNpe72TdxQAIqRx65miBpDKnMFAgvqOll7gFxulzRWtmHgJjKz7gr+Wvo1olJ7sdLr8lAP17tds9WSnwtYO1gQk18g3JkR2VYmBYUgj7oPy1Q/Y3RH+lGQHsgP/aARQGHxbttwb15wlC4sTsip0ZQ== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 20:JzGvBuODemxgVYQQ23v4JdGPHi66EBMZGncrUZuTunrFm2q2cnEPGbxfWzlGrGUPJjHzPY24xMBGWqZzohu1jXicnbeYE5TrArwvGUJ08Xq8m5Qve4VRLd1vuIkIORWrMUNLM5hu6bZ0yWyx3Jq7r+CKpCcls9uGs6fj9vbiNzbkREBce8/M+WB3LyuNpk7OfkuBsnY+JckwPinnHCuXndGbUcnYeKDKmNdQOprSKI38SSsXGQeAbZIFv5dSIAYIdQ6TcEgky/uhZclmOXcS9PeGBdapFdhPxxKO9yTp33zLSlWkyon8hggbM8WfRxvCQvzoxN70NXmwiDQX3XLodDG0BfnkofiAeSbdkb5in5m655f0X1hxSWKT22kAGa4e5oXHTGJwiLGmV/jDb0U7o2meJ8fktt9UGJqWqpCeASE6hWiAkb3YHVrQv/+BSZJi0VBZfEIhygXahYZDpoTtj1PWWZr2EVSdnTv1ECmMhXhQXwweWoirXsK4vrp0Hmr4 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(84480959824636)(192374486261705)(75325880899374)(21748063052155); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93004095)(6055026)(6041248)(20161123560025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0301MB0745; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0301MB0745; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB0745; 4:Npm0V1THC6dW1dd1r4Gmys5hrzT5cWuKMkwpVAjy?= =?us-ascii?Q?AvfG2HdZlApPTT31HXHOpOdH9hzr12aHfl94iaiG+FyFCCFwXrd7HnSlIPbE?= =?us-ascii?Q?5IKCSCsT1SMaCOsAKuMbVswRvr56xDHLJW4U3/md2hT7BlgCen/NG4HIC4ro?= =?us-ascii?Q?jjy8vsMQVxPtnSdZm9GM1Q69GyJV7WpaFIoLQwU+uqVAr00VLfVN5XhzLTUe?= =?us-ascii?Q?8ZDjsorzUs18WKITggoYdbGB3js4pGbHzC/7vu1dsOIYo7y/izBr/kVMdsfq?= =?us-ascii?Q?VqZ6c5teY5eim0lCIekvGLuZkagwTjpxwsLObmtbVHA0IbLLwsliK5hTA7AO?= =?us-ascii?Q?wJinIW97NJPu11r/TOqaGa7kdoT+PdzdV0AvIjHIfrArHQzJ9g8/irjjHfdp?= =?us-ascii?Q?1uH4WSsVhnwZI2P+bLRNATLx87Oyvtsj/O9vTGEYRs/hPvK5ZMxf/m41knC0?= =?us-ascii?Q?3yR+xeDJ3sbRm2DkBtPwANqShrTHu+Ul8WARNMvS5zL0Hms7jhzW1ED4PSq/?= =?us-ascii?Q?FglzT89FSA8UxDH1d1MGKBRBG6FMFkttubYtv0NMUeeZmxtLLk95Xfma58mB?= =?us-ascii?Q?yylDzIPP4fIMiXQIkXV1j+On2066AKPnk1yaQf0I3BsvZAqhzWtVc2AvCwdm?= =?us-ascii?Q?pMuLxx2Cy0MoWsrttEAsWW/z4nG/eIEA4I9zaBwOexAuDPZCQwUDlU+dQPj2?= =?us-ascii?Q?3KzlAPv1wXmxOPu5197t+CclaVYumN7suBxWirXBHMFDzK9vnXXU/HztLw/N?= =?us-ascii?Q?8KIxS8mZ47G30fi5UWV7zI3ky7jYubqSVk9CCqHduvJtMf3EFv2z0jMxnKqx?= =?us-ascii?Q?H8JZGrwAt26yxZIM/Y2LLuk7YXP0ZoI71jTmK8XEv72L9/EKmfR+ckFk7RXN?= =?us-ascii?Q?UpEHA9fkO2qZVM/8aL8D/FPE2/OgDCH9SnarmmrjOlvQmYpfWPkbhtMDhE12?= =?us-ascii?Q?DGKxb+eyH9twP8YnlCF0BON+5ntqhi+TFrpNGmhe6BuJ+nsN9NfchD5hYYPC?= =?us-ascii?Q?wZhMbrHkf2zpJZ6Nl+8w1ypf6IZ1vQ4ruSAK8MMtxVoQ17HZBQlDg16wOcoT?= =?us-ascii?Q?/6ovJuYAeFKsW76vwcA/6+bCPSe3lX4gWOZ24bFY70dAfvhRH0k9IC1EzsV4?= =?us-ascii?Q?VucaHUoBGCeMgyysJNyugqqckJsjo1LWrXSoV9Nn8EF0tcgQVrlmn5/YJXlN?= =?us-ascii?Q?EUG9dEP0aewRi7MZ1K/truUrI5x0udibIdPXaHrkW6qM6MD0hLLj4elfa4f6?= =?us-ascii?Q?DmQff5aZly0deCBuglodHPQ8jnCFTcZvO0WcJJCSt6RA/lziKn+lGUbmBJrY?= =?us-ascii?Q?D52OPSFEB7p8K8AIDPCQfik8/EZTfkuITjua/t/SEOpZEIp1H1G/Gt/qMxcx?= =?us-ascii?Q?BH9Bu+gHKyplkdVEISLkP2gX8baqfJOYOCLo7/D88YTdWY7NZeHiFiiuX7xN?= =?us-ascii?Q?Wdh83q0I8A=3D=3D?= X-Forefront-PRVS: 03361FCC43 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB0745; 23:PkPTNV6CwTQGKyJ96Ih6lFr94DdvmyIAvXiYKLx?= =?us-ascii?Q?SgayJx/w9uPkelVEPrD5dsDeR+Hs8h6WJV5LFbbGLUMnuwl7K06U9mRto1x8?= =?us-ascii?Q?X8NwwNocOCvWsP3e9LRZkUlH+6M9NxcHAOyfPTwJ0OmX3zcc2KlB0ezcun6Q?= =?us-ascii?Q?zLzX1VGZ2Bf8K+i9Y5drptxK913nlPJTJQphUlHM+QFlYzP0Hsnkbu0nTXSJ?= =?us-ascii?Q?I747gMBeWBDcs096ioLLLt92PfdHFZnp/cqmS9BDxMwfqk5tjNrfuxduI0UG?= =?us-ascii?Q?BLWDPNF3Q9p8mvHyvQWVBUUiRhRw7WfvXiGnE2Y9/zy3ipx6Lo57Rd1YkyZv?= =?us-ascii?Q?p12VeHCmTWbtcQ22U3+h7mGTFE2gR1p845Dc0RYS+qlH3ZHM1pmCKejoAzf+?= =?us-ascii?Q?YCyD8jO06KQHYjN8XrjsZGEqjPggZyFMHpWn7Z9lGgz7cqBw8oJ8IQXM9Mga?= =?us-ascii?Q?dOAbbrIb7eC+2V+IJ1JaZ/x8PDXwGeSYaY9LKuTFhM0rO5XN9U75C9FcnH5v?= =?us-ascii?Q?Fx8qVr0fbirBZf3zIcYMqg50mzSwYSBnquNMtI0HkMnVMml/5Sg4LijJZup6?= =?us-ascii?Q?jfPvsOQhRQSqHwNL/ONL0vFCidJyI/OBPPq4G48D2lOGTVP1IwBoIaTsPm0Y?= =?us-ascii?Q?AGN9P3ikPJ7IIz/vey3zRLYWtc9by2i14rhpeQ6nkVomuYI8nZrIxyOMZyyz?= =?us-ascii?Q?vL7aGkgSVkFxMYd5JeZqidfv0VNgjPx8Zv2kraLnoHPmIr225QgLn27yHlU1?= =?us-ascii?Q?HkWK8+EOrCZqemUw16hj1NZakeIv3kmRnIP4zrAuB8ICew7RMnfog+D8y5iz?= =?us-ascii?Q?ZgeuYkqz//F67AsTDyHwYqJ0Um4s9wom5sOTMhb6sgDjIt5zTJIw8VPm4C4C?= =?us-ascii?Q?dB2SSlVnIVDrcRNgbohdA5didvwbT+dA8lDToVlPXF4q2kz2/XoFSszQjfF9?= =?us-ascii?Q?BN5rcqu1a29uNGFaNIXMOtnOkfV3G3dDAaSZhP6PJF4lYYp9WA4fM+G3/BoY?= =?us-ascii?Q?ulawr2Qcko5uq/Bw4vFhtINSFO50CFZR0Mkjsp9fRRVxNizGw6Hbk3n/HwkS?= =?us-ascii?Q?PWbm/tl01Z/j3zdRkFzcXsHIhL4G5qqctIkuxpTfGL39Nb4LlNP84CtIYPx5?= =?us-ascii?Q?si+c5QHtFQSJKtCw/O/XROTmdhVqYJ92n3iIkY78bsKIPVoOSoDjxKTEAq/y?= =?us-ascii?Q?SIPntD/jHGdWvofjGEIZc6fKumOMvt7/KnfuwK4WZqDOhbn6WLzt6rnSX9Dg?= =?us-ascii?Q?R+Jr73DBXAYiRBpnSAkZbnNn9se/1D5SamfyS3npYmR4Fb7WRRgutt+RRxKj?= =?us-ascii?Q?dcVx/CWmkHxlLCSHv61LDNolF/icX+py86ihT8Z2O0Gkj/Qy+iTqLZiBwyeY?= =?us-ascii?Q?2KCG+dReIvsp7mmy7+EvbHB52X/aCdhaOdaRlvc9NAg74ASoE+bVUSXSErV8?= =?us-ascii?Q?yhCMKfOiJO+UAvr21RAE0hjf7mT5kZAy/3CYH2NDwBxwk1rhc8FII4nIq3Uo?= =?us-ascii?Q?WjoExlN1LoxMZT5RqW06u+J4fmdxBPfK3lLY=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 6:IT+VDPyRQohI/Q0RUyQ/mtzKqD7onaSGdA9PPWOklBHkIuhFglu5siPw4K0e+QzZnQ9JfL9zP73fNcoCVwEjL16pAl9a/gE19r7VvSNftpxMuojb4+wLvzXdHjzwhTUnNie27VxWWhRTbgoRNbTLQ7NJW03QgMfs8jEv5SRMv6TOGp/VSgxKwGwPtAX2SNYa3nZ/ffO4w5wvLr3LoIzBmVW3PSJotyQRWhv+BWYe+qQC5S4c1ul4abVailNdxlYKvGqTfzRc6N7rYP6JUB7RNojqAb2yCF77xpSKxY5JnMBp/WIo+psNQNODojFOwbmB/HUyP38WmcmpLNapqb8S+pSaHJsQTtiruYDyHBuW6zVBF13VFhDayYkYm5VtYX2fvcxgNQV/WiBb3r5wp68LOiC7QPbAA2wmKXxdFnJvzoIcdzWwQxefyQcyD+y25PqjeFXoRH1I7RMVJeVKQjGYJV9m5ovsAUQpRQ8aoz7hCqSW9SIux6sMezta3oXnwtYtpF53FF4ZAVgjQ89dZuR2jTzAeVBva4PCLggOb+iLQfI= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 5:d/yk7B7pRL5ujFwEgPiRLjLZ30P2EPoLdP5o4+gucDIdBfUJR50HSVpcNYyHR4k6ZOwzFiv1FDf62CAYk+E9JwRulH4KM4Zi4yI7yuO7OUtuHvaHzaAA3+WmXbOvBFsGTZNS9h8W68Thl9Yl3TgJJlBY4RtOf501Eycqq+1TE0GX/PyjWPkJFe4Sf1xtYWmWwUXTX3GmHoywmY3RrPOda7MYoRL/KivVigkIHR5+Wz0qwhtdk3kZNtErOipSusq23ngWhQKZy+9uRjBI9QKZl2bPX/v2YF8/to6Sgu52jjiFhPPST2apAE01qUE7q3QipKU3yImQuHxegB/NAEIgF3mvMD/cSB4zKeoqIu1GP+bVAuInd5+FwmpGK9xMZmuMw4Jr9dqM7F7Gpl7CV5qTRLOZa7TWUiZMkNcHJkRQhohhYFxnvSHJPkSLxcwXBG0e4gQFL9Ikmepf7irxTboE1rrQ/pg+bvb1lqJnl+zDblcZrEwqtfb14ibBoMEGnAAf; 24:R+GDDj3Tg1gtXmrlPxkmiwkzv85/rT+MML+JqKvcrgZychGbHVwbTFVyrcc/0Z2w8FcOrbvBgcGX7FWZLSid3dCZij0fMXWLJtXhdAOhOng= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 7:zmNRoV+pp16dFg/tjGMfxhTvZSFmruI4httHBVeFLUD+0BGtj6l8E262EjAlwtR6tE0oPomOQqI8vO560KfhOS8iFsRoIlSFhiMpB9bSfvGBLnb+yG+O9A6AkE8uH7JQHqomT6CghE1KZm2+DRTNNw5qfNqN5iN6JX07E3urWmTKSh0H15WUbCB79llULAw/urT3lpZ1rnkUxK2ssTdEasQPSG/rINshT9i159KKM1VeXlB1aS4uZ3ee1vzdAVdB93A3kVG6NTt8iMsNBbxHVHrCE6HzDCl/4J7UUB4+E6HiprkKWGMpQi+TE88MKf36qP7y8m/Hd5K+EGVAuZ2tPg== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2017 04:50:46.0760 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0301MB0745 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2017 04:50:49 -0000 VGhhbmtzLCBKb3YsIGZvciB5b3Ugc3VnZ2VzdGlvbnMuICBQZXIgeW91ciBlLW1haWwgSSBhZGRl ZCDigJxleHBsYWluIGFuYWx5emXigJ0gdG8gdGhlIHNjcmlwdDoNCg0KIyEvYmluL3NoDQpwc3Fs IC0tdXNlcm5hbWU9dGVzdCAtLXBhc3N3b3JkPXN1cGVyc2VjcmV0IC1oIC9kYiAtZCB0ZXN0IDw8 IEVPTA0KXHRpbWluZyBvbg0KZXhwbGFpbiBhbmFseXplIHNlbGVjdCBjb3VudCgqKSBmcm9tIHRl c3Q7DQpccQ0KRU9MDQoNClNhbXBsZSBydW4gb2YgYWJvdmUgc2NyaXB0IGJlZm9yZSBkZWdyYWRh dGlvbjoNClRpbWluZyBpcyBvbi4NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBRVUVSWSBQTEFODQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0N CkFnZ3JlZ2F0ZSAgKGNvc3Q9MzM1MDgyMi4zNS4uMzM1MDgyMi4zNiByb3dzPTEgd2lkdGg9MCkg KGFjdHVhbCB0aW1lPTYwMjM0LjU1Ni4uNjAyMzQuNTU2IHJvd3M9MSBsb29wcz0xKQ0KICAgLT4g IFNlcSBTY2FuIG9uIHRlc3QgIChjb3N0PTAuMDAuLjMyOTY5MDEuMDggcm93cz0yMTU2ODUwOCB3 aWR0aD0wKSAoYWN0dWFsIHRpbWU9MS4xMjYuLjU3MDIxLjQ3MCByb3dzPTIxNTY4NTA4IGxvb3Bz PTEpDQpQbGFubmluZyB0aW1lOiA0Ljk2OCBtcw0KRXhlY3V0aW9uIHRpbWU6IDYwMjM0LjY0OSBt cw0KKDQgcm93cykNCg0KVGltZTogNjAyNDguNTAzIG1zDQp0ZXN0JCB1cHRpbWUNCjEwOjMzUE0g IHVwIDcgbWlucywgMyB1c2VycywgbG9hZCBhdmVyYWdlczogMS42OCwgMS43OSwgMC45NA0KDQoN ClNhbXBsZSBydW4gb2YgYWJvdmUgc2NyaXB0IGFmdGVyIGRlZ3JhZGF0aW9uICh+MTEuMzMgaG91 cnMgdXB0aW1lKToNClRpbWluZyBpcyBvbi4NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBRVUVSWSBQTEFODQotLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tDQpBZ2dyZWdhdGUgIChjb3N0PTMzNTA4MjIuMzUuLjMzNTA4MjIuMzYgcm93cz0xIHdp ZHRoPTApIChhY3R1YWwgdGltZT00ODU2NjkuMzYxLi40ODU2NjkuMzYxIHJvd3M9MSBsb29wcz0x KQ0KICAgLT4gIFNlcSBTY2FuIG9uIHRlc3QgIChjb3N0PTAuMDAuLjMyOTY5MDEuMDggcm93cz0y MTU2ODUwOCB3aWR0aD0wKSAoYWN0dWFsIHRpbWU9MC4wMDguLjQ4MzI0MS4yNTMgcm93cz0yMTU2 ODUwOCBsb29wcz0xKQ0KUGxhbm5pbmcgdGltZTogMC41MjkgbXMNCkV4ZWN1dGlvbiB0aW1lOiA0 ODU2NjkuNDExIG1zDQooNCByb3dzKQ0KDQpUaW1lOiA0ODU2NzAuNDMyIG1zDQp0ZXN0JCB1cHRp bWUNCjk6NTlQTSAgdXAgMTE6MjEsIDIgdXNlcnMsIGxvYWQgYXZlcmFnZXM6IDEuMTEsIDIuMTMs IDIuMTQNCg0KDQpSZWdhcmRpbmcgZGTigJlpbmcgdGhlIHBnZGF0YSBkaXJlY3RvcnksIHRoYXQg ZGlkbuKAmXQgd29yayBmb3IgbWUgYXMgUG9zdGdyZXMgc3BsaXRzIHRoZSBkYXRhYmFzZSB1cCBp bnRvIG11bHRpcGxlIDJHQiBmaWxlcyDigJMgZGTigJlpbmcgb2YgYSAyR0IgZmlsZSBvbiBhIHN5 c3RlbSB3aXRoIDhHQiByYW0gZG9lc27igJl0IHNlZW0gcmVwcmVzZW50YXRpdmUuICBJIG9wdGVk IHRvIGNyZWF0ZSBhIDE2R0IgIGZpbGUgKGRkIGlmPS9kZXYvcmFuZG9tIG9mPS90ZXN0ZGIvdGVz dCBicz0xbSBjb3VudD0xNjAwMCkgb24gdGhlIHBlcnRpbmVudCBaRlMgZmlsZSBzeXN0ZW0gdGhl biBwZXJmb3JtZWQgZGQgb3BlcmF0aW9uIG9uIHRoYXQ6DQoNClNhbXBsZSBvZiBydW4gYWZ0ZXIg ZGVncmFkYXRpb24gKH4xMS42NiBob3VycyB1cHRpbWUpOg0KMTYwMDArMCByZWNvcmRzIGluDQox NjAwMCswIHJlY29yZHMgb3V0DQoxNjc3NzIxNjAwMCBieXRlcyB0cmFuc2ZlcnJlZCBpbiAyNzQu ODQxNzkyIHNlY3MgKDYxMDQzMTc2IGJ5dGVzL3NlYykNCnRlc3QkIHVwdGltZQ0KMTA6MjVQTSAg dXAgMTE6NDYsIDIgdXNlcnMsIGxvYWQgYXZlcmFnZXM6IDEuMDAsIDEuMjgsIDEuNTkNCg0KDQpB ZnRlciByZWJvb3RpbmcsIHdlIGNhbiBzZWUgKk1VQ0gqIGJlZm9yZSBwZXJmb3JtYW5jZToNCnRl c3QkIGRkIGlmPS90ZXN0ZGIvdGVzdCBvZj0vZGV2L251bGwgYnM9MW0NCjE2MDAwKzAgcmVjb3Jk cyBpbg0KMTYwMDArMCByZWNvcmRzIG91dA0KMTY3NzcyMTYwMDAgYnl0ZXMgdHJhbnNmZXJyZWQg aW4gMTkuNDU2MDQzIHNlY3MgKDg2MjMxMzg4MyBieXRlcy9zZWMpDQp0ZXN0JCBkZCBpZj0vdGVz dGRiL3Rlc3Qgb2Y9L2Rldi9udWxsIGJzPTFtDQoxNjAwMCswIHJlY29yZHMgaW4NCjE2MDAwKzAg cmVjb3JkcyBvdXQNCjE2Nzc3MjE2MDAwIGJ5dGVzIHRyYW5zZmVycmVkIGluIDE5LjM3NTMyMSBz ZWNzICg4NjU5MDY0NzMgYnl0ZXMvc2VjKQ0KdGVzdCQgZGQgaWY9L3Rlc3RkYi90ZXN0IG9mPS9k ZXYvbnVsbCBicz0xbQ0KMTYwMDArMCByZWNvcmRzIGluDQoxNjAwMCswIHJlY29yZHMgb3V0DQox Njc3NzIxNjAwMCBieXRlcyB0cmFuc2ZlcnJlZCBpbiAxOS4xNzM0NTggc2VjcyAoODc1MDIyOTY4 IGJ5dGVzL3NlYykNCnRlc3QkIHVwdGltZQ0KMTA6MzBQTSAgdXAgNCBtaW5zLCAzIHVzZXJzLCBs b2FkIGF2ZXJhZ2VzOiAzLjUyLCAxLjYyLCAwLjY5DQoNClRoZXNlIHRlc3RzIHdlcmUgY29uZHVj dGVkIHdpdGggdGhlIHByZXZpb3VzbHkgbWVudGlvbmVkIFNhbXN1bmcgODUwIFBybyAyNTZHQiBT U0RzIChJbnRlbCBYZW9uIEUzMTI0MCB3aXRoIDhHQiByYW0pLiAgVGhlcmXigJlzIGVzc2VudGlh bGx5IG5vdGhpbmcgZWxzZSBydW5uaW5nIG9uIHRoaXMgc3lzdGVtICg5OS41LTEwMCUgaWRsZSkg YW5kIG5vIG90aGVyIGRpc2sgYWN0aXZpdHkuDQoNClJlZ2FyZHMsDQpBDQoNCkZyb206IEpvdiBb bWFpbHRvOmFtdXR1QGFtdXR1LmNvbV0NClNlbnQ6IFN1bmRheSwgSnVuZSAxMSwgMjAxNyA1OjUw IFBNDQpUbzogQ2F6YSwgQWFyb24NCkNjOiBmcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmc7IEFs bGFuIEp1ZGUNClN1YmplY3Q6IFtFWFRFUk5BTF0gUmU6IEZyZWVCU0QxMCBTdGFibGUgKyBaRlMg KyBQb3N0Z3JlU1FMICsgU1NEIHBlcmZvcm1hbmNlIGRyb3AgPCAyNCBob3Vycw0KDQpUbyBleGNs dWRlIHRoZSBmcyBwcm9ibGVt77yMSSB3aWxsIGRvIGEgZGQgdGVzdCBvbiB0aGUgcGdkYXRhIGRh dGEgc2V0IGFmdGVyIHRoZSBwZXJmb3JtYW5jZSBkcm9wLGlmIHRoZSByZWFkIGFuZC9vciB3cml0 ZSB1dGlsaXR5IGNhbiByZWFjaCAxMDAlIG9yIHBlcmZvcm1hbmNlIGV4cGVjdGVkIHRoZW4gSSB3 aWxsIHNheSB0aGUgcHJvYmxlbSBpcyBub3QgZnMgb3Igb3MuDQoNCkZvciBwZyx3aGF0J3MgeW91 ciBvdXRwdXQgb2YgZXhwbGFpbiBhbmFseXplIGJlZm9yZSBhbmQgYWZ0ZXIgcGVyZm9ybWFuY2Ug ZHJvcD8NCg0KMjAxN+W5tDbmnIgxMuaXpSAxMjo1MSBBTe+8jCJDYXphLCBBYXJvbiIgPEFhcm9u LkNhemFAY2Eud2VhdGhlcmZvcmQuY29tPG1haWx0bzpBYXJvbi5DYXphQGNhLndlYXRoZXJmb3Jk LmNvbT4+5YaZ6YGT77yaDQpUaGFua3MgQWxsYW4gZm9yIHRoZSBzdWdnZXN0aW9ucy4gIEkgdHJp ZWQgZ3N0YXQgLWQgYnV0IGRlbGV0ZXMgKGQvcykgZG9lc24ndCBzZWVtIHRvIGJlIGl0IGFzIGl0 IHN0YXlzIGF0IDAgZGVzcGl0ZSB2ZnMuemZzLnRyaW0uZW5hYmxlZD0xLg0KDQpUaGlzIGlzIG1v c3QgbGlrZWx5IGR1ZSB0byB0aGUgImxheWVyaW5nIiBJIHVzZSBhcywgZm9yIGhpc3RvcmljYWwg cmVhc29ucywgSSBoYXZlIEdFT00gRUxJIHNldCB1cCB0byBlc3NlbnRpYWxseSBlbXVsYXRlIDRr IHNlY3RvcnMgcmVnYXJkbGVzcyBvZiB0aGUgdW5kZXJseWluZyBtZWRpYS4gIEkgZG8gbXkgb3du IGFsaWdubWVudCBhbmQgcGFydGl0aW9uIHNpemluZyBhcyB3ZWxsIGFzIGhhdmUgdGhlIFpGUyBy ZWNvcmQgc2l6ZSBzZXQgdG8gOGsgZm9yIFBvc3RncmVzLg0KDQpJbiBnc3RhdCwgdGhlIFNTRHMg JWJ1c3kgaXMgOTAtMTAwJSBvbiBzdGFydHVwIGFmdGVyIHJlYm9vdC4gIE9uY2UgdGhlIHBlcmZv cm1hbmNlIGRlZ3JhZGF0aW9uIGhpdHMgKDwyNCBob3VycyBsYXRlciksIEknbSBzZWVpbmcgJWJ1 c3kgYXQgfjEwJS4NCg0KIyEvYmluL3NoDQpwc3FsIC0tdXNlcm5hbWU9dGVzdCAtLXBhc3N3b3Jk PXN1cGVyc2VjcmV0IC1oIC9kYiAtZCB0ZXN0IDw8IEVPTA0KXHRpbWluZyBvbg0Kc2VsZWN0IGNv dW50KCopIGZyb20gdGVzdDsNClxxDQpFT0wNCg0KU2FtcGxlIHJ1biBvZiBhYm92ZSBzY3JpcHQg YWZ0ZXIgcmVib290IChiZWZvcmUgZGVncmFkYXRpb24gaGl0cykgKFNhbXN1bmcgODUwIFByb3Mg aW4gWkZTIG1pcnJvcik6DQpUaW1pbmcgaXMgb24uDQogIGNvdW50DQotLS0tLS0tLS0tDQogMjE1 Njg1MDgNCigxIHJvdykNCg0KVGltZTogNTcwMjkuMjYyIG1zDQoNClNhbXBsZSBydW4gb2YgYWJv dmUgc2NyaXB0IGFmdGVyIGRlZ3JhZGF0aW9uIChTYW1zdW5nIDg1MCBQcm9zIGluIFpGUyBtaXJy b3IpOg0KVGltaW5nIGlzIG9uLg0KICBjb3VudA0KLS0tLS0tLS0tLQ0KIDIxNTY4NTA4DQooMSBy b3cpDQoNClRpbWU6IDU4MzU5NS4yMzkgbXMNCihVcHRpbWUgfjEgZGF5IGluIHRoaXMgcGFydGlj dWxhciBjYXNlLikNCg0KDQpBbnkgb3RoZXIgc3VnZ2VzdGlvbnM/DQoNClJlZ2FyZHMsDQpBDQoN Ci0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBvd25lci1mcmVlYnNkLWhhY2tlcnNA ZnJlZWJzZC5vcmc8bWFpbHRvOm93bmVyLWZyZWVic2QtaGFja2Vyc0BmcmVlYnNkLm9yZz4gW21h aWx0bzpvd25lci1mcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmc8bWFpbHRvOm93bmVyLWZyZWVi c2QtaGFja2Vyc0BmcmVlYnNkLm9yZz5dIE9uIEJlaGFsZiBPZiBBbGxhbiBKdWRlDQpTZW50OiBT YXR1cmRheSwgSnVuZSAxMCwgMjAxNyA5OjQwIFBNDQpUbzogZnJlZWJzZC1oYWNrZXJzQGZyZWVi c2Qub3JnPG1haWx0bzpmcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmc+DQpTdWJqZWN0OiBbRVhU RVJOQUxdIFJlOiBGcmVlQlNEMTAgU3RhYmxlICsgWkZTICsgUG9zdGdyZVNRTCArIFNTRCBwZXJm b3JtYW5jZSBkcm9wIDwgMjQgaG91cnMNCg0KT24gMDYvMTAvMjAxNyAxMjozNiwgU2xhd2EgT2xo b3ZjaGVua292IHdyb3RlOg0KPiBPbiBTYXQsIEp1biAxMCwgMjAxNyBhdCAwNDoyNTo1OVBNICsw MDAwLCBDYXphLCBBYXJvbiB3cm90ZToNCj4NCj4+IEdlbnRzLA0KPj4NCj4+IEknbSBleHBlcmll bmNpbmcgYW4gaXNzdWUgd2hlcmUgaXRlcmF0aW5nIG92ZXIgYSBQb3N0Z3JlU1FMIHRhYmxlIG9m IH4yMS41IG1pbGxpb24gcm93cyAoc2VsZWN0IGNvdW50KCopKSBnb2VzIGZyb20gfjM1IHNlY29u ZHMgdG8gfjYzNSBzZWNvbmRzIG9uIEludGVsIDU0MCBTU0RzLiAgVGhpcyBpcyB1c2luZyBhIEZy ZWVCU0QgMTAgYW1kNjQgc3RhYmxlIGtlcm5lbCBiYWNrIGZyb20gSmFuIDIwMTcuICBTU0RzIGFy ZSBiYXNpY2FsbHkgMiBkcml2ZXMgaW4gYSBaRlMgbWlycm9yZWQgenBvb2wuICBJJ20gdXNpbmcg UG9zdGdyZVNRTCA5LjUuNy4NCj4+DQo+PiBJJ3ZlIHRyaWVkOg0KPj4NCj4+ICogICAgICAgVXNp bmcgdGhlIEZyZWVCU0QxMCBhbWQ2NCBzdGFibGUga2VybmVsIHNuYXBzaG90IG9mIE1heSAyNSwg MjAxNy4NCj4+DQo+PiAqICAgICAgIFRlc3RlZCBvbiBoYWxmIGEgZG96ZW4gbWFjaGluZXMgd2l0 aCBkaWZmZXJlbnQgbW9kZWxzIG9mIFNTRHM6DQo+Pg0KPj4gbyAgIEludGVsIDUxMHMgKDEyMEdC KSBpbiBaRlMgbWlycm9yZWQgcGFpcg0KPj4NCj4+IG8gICBJbnRlbCA1MjBzICgxMjBHQikgaW4g WkZTIG1pcnJvcmVkIHBhaXINCj4+DQo+PiBvICAgSW50ZWwgNTQwcyAoMTIwR0IpIGluIFpGUyBt aXJyb3JlZCBwYWlyDQo+Pg0KPj4gbyAgIFNhbXN1bmcgODUwIFByb3MgKDI1NkdCKSBpbiBaRlMg bWlycm9yZWQgcGFpcg0KPj4NCj4+ICogICAgICAgVXNpbmcgYm9ubmllKysgdG8gcmVtb3ZlIFBv c3RncmVzIGZyb20gdGhlIGVxdWF0aW9uIGFuZCBwZXJmb3JtYW5jZSBkb2VzIGluZGVlZCBkcm9w Lg0KPj4NCj4+ICogICAgICAgUmVib290aW5nIHNlcnZlciBhbmQgaW1tZWRpYXRlbHkgcmUtcnVu bmluZyB0ZXN0IGFuZCBwZXJmb3JtYW5jZSBpcyBiYWNrIHRvIG9yaWdpbmFsLg0KPj4NCj4+ICog ICAgICAgVHJpZWQgdXNpbmcgS2FybCBEZW5uaW5nZXIncyBwYXRjaCBmcm9tIFBSMTg3NTk0ICh3 aGljaCB0b29rIHNvbWUgd29yayB0byBmaW5kIGEga2VybmVsIHRoYXQgdGhlIEZyZWVCU0QxMCBw YXRjaCB3b3VsZCBib3RoIGFwcGx5IGFuZCBjb21waWxlIGNsZWFubHkgYWdhaW5zdCkuDQo+Pg0K Pj4gKiAgICAgICBUcmllZCBkaXNhYmxpbmcgWkZTIGx6NCBjb21wcmVzc2lvbi4NCj4+DQo+PiAq ICAgICAgIFJhbiB0aGUgc2FtZSB0ZXN0IG9uIGEgRnJlZUJTRDkuMCBhbWQ2NCBzeXN0ZW0gdXNp bmcgUG9zdGdyZVNRTCA5LjEuMyB3aXRoIDIgSW50ZWwgNTIwcyBpbiBaRlMgbWlycm9yZWQgcGFp ci4gIFN5c3RlbSBoYWQgMTY1IGRheXMgdXB0aW1lIGFuZCB0ZXN0IHRvb2sgfjgwIHNlY29uZHMg YWZ0ZXIgd2hpY2ggSSByZWJvb3RlZCBhbmQgcmUtcmFuIHRlc3QgYW5kIHdhcyBzdGlsbCBhdCB+ ODAgc2Vjb25kcyAob2xkZXIgcHJvY2Vzc29yIGFuZCBtZW1vcnkgaW4gdGhpcyBzeXN0ZW0pLg0K Pj4NCj4+IEkgcmVhbGl6ZSB0aGF0IHRoZXJlJ3MgYSB3aG9sZSBsb3Qgb2YgaW5mbyBJJ20gbm90 IGluY2x1ZGluZyAoZG1lc2csIHpmcy1zdGF0cyAtYSwgZ3N0YXQsIGV0IGNldGVyYSk6IEknbSBo b3Bpbmcgc29tZSBlbmxpZ2h0ZW5lZCBpbmRpdmlkdWFsIHdpbGwgYmUgYWJsZSB0byBwb2ludCBt ZSB0byBhIHNvbHV0aW9uIHdpdGggb25seSB0aGUgYWJvdmUgdG8gZ28gb24uDQo+DQo+IEp1c3Qg YSByYW5kb20gZ3Vlc3M6IGNhbiB5b3UgdHJ5IHIzMDcyNjQgKEkgYW0gbWVhbiByZWdyZXNzaW9u IGluDQo+IHIzMDcyNjYpPw0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXw0KPiBmcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmc8bWFpbHRvOmZyZWVic2Qt aGFja2Vyc0BmcmVlYnNkLm9yZz4gbWFpbGluZyBsaXN0DQo+IGh0dHBzOi8vbGlzdHMuZnJlZWJz ZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWhhY2tlcnMNCj4gVG8gdW5zdWJzY3JpYmUs IHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtaGFja2Vycy11bnN1YnNjcmliZUBmcmVlYnNkLm9y ZzxtYWlsdG86ZnJlZWJzZC1oYWNrZXJzLXVuc3Vic2NyaWJlQGZyZWVic2Qub3JnPiINCj4NCg0K VGhpcyBzb3VuZHMgYSBiaXQgbGlrZSBhbiBpc3N1ZSBJIGludmVzdGlnYXRlZCBmb3IgYSBjdXN0 b21lciBhIGZldyBtb250aHMgYWdvLg0KDQpMb29rIGF0IGdzdGF0IC1kIChpbmNsdWRlcyBERUxF VEUgb3BlcmF0aW9ucyBsaWtlIFRSSU0pDQoNCklmIHlvdSBzZWUgYSBsb3Qgb2YgdGhhdCBoYXBw ZW5pbmcsIGJ1dCB0cnk6IHZmcy56ZnMudHJpbS5lbmFibGVkPTAgaW4gL2Jvb3QvbG9hZGVyLmNv bmYgYW5kIHNlZSBpZiB5b3VyIGlzc3VlcyBnbyBhd2F5Lg0KDQp0aGUgRnJlZUJTRCBUUklNIGNv ZGUgZm9yIFpGUyBiYXNpY2FsbHlzIHdhaXRzIHVudGlsIHRoZSBzZWN0b3IgaGFzIGJlZW4gZnJl ZSBmb3IgYSB3aGlsZSAodG8gYXZvaWQgZG9pbmcgYSBUUklNIG9uIGEgYmxvY2sgd2UnbGwgaW1t ZWRpYXRlbHkgcmV1c2UpLCBzbyB5b3VyIGJlbmNobWFyayB3aWxsIHJ1biBmaWxlIGZvciBhIGxp dHRsZSB3aGlsZSwgdGhlbiBzdWRkZW5seSB0aGUgVFJJTSB3aWxsIGtpY2sgaW4uDQoNCkZvciBw b3N0Z3JlcywgZmlvLCBib25uaWUrKyBldGMsIG1ha2Ugc3VyZSB0aGUgWkZTIGRhdGFzZXQgeW91 IGFyZSBzdG9yaW5nIHRoZSBkYXRhIG9uIC8gYmVuY2htYXJraW5nIGhhcyBhIHJlY29yZHNpemUg dGhhdCBtYXRjaGVzIHRoZSB3b3JrbG9hZC4NCg0KSWYgeW91IGFyZSBkb2luZyBhIHdyaXRlLW9u bHkgYmVuY2htYXJrLCBhbmQgeW91IHNlZSBsb3RzIG9mIHJlYWRzIGluIGdzdGF0LCB5b3Uga25v dyB5b3UgYXJlIGhhdmluZyB0byBkbyByZWFkL21vZGlmeS93cml0ZSdzLCBhbmQgdGhhdCBpcyB3 aHkgeW91ciBwZXJmb3JtYW5jZSBpcyBzbyBiYWQuDQoNCg0KLS0NCkFsbGFuIEp1ZGUNCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpmcmVlYnNkLWhhY2tl cnNAZnJlZWJzZC5vcmc8bWFpbHRvOmZyZWVic2QtaGFja2Vyc0BmcmVlYnNkLm9yZz4gbWFpbGlu ZyBsaXN0IGh0dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNk LWhhY2tlcnMNClRvIHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvICJmcmVlYnNkLWhhY2tl cnMtdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmc8bWFpbHRvOmZyZWVic2QtaGFja2Vycy11bnN1YnNj cmliZUBmcmVlYnNkLm9yZz4iDQoNClRoaXMgbWVzc2FnZSBtYXkgY29udGFpbiBjb25maWRlbnRp YWwgYW5kIHByaXZpbGVnZWQgaW5mb3JtYXRpb24uIElmIGl0IGhhcyBiZWVuIHNlbnQgdG8geW91 IGluIGVycm9yLCBwbGVhc2UgcmVwbHkgdG8gYWR2aXNlIHRoZSBzZW5kZXIgb2YgdGhlIGVycm9y IGFuZCB0aGVuIGltbWVkaWF0ZWx5IGRlbGV0ZSBpdC4gSWYgeW91IGFyZSBub3QgdGhlIGludGVu ZGVkIHJlY2lwaWVudCwgZG8gbm90IHJlYWQsIGNvcHksIGRpc2Nsb3NlIG9yIG90aGVyd2lzZSB1 c2UgdGhpcyBtZXNzYWdlLiBUaGUgc2VuZGVyIGRpc2NsYWltcyBhbnkgbGlhYmlsaXR5IGZvciBz dWNoIHVuYXV0aG9yaXplZCB1c2UuIFBMRUFTRSBOT1RFIHRoYXQgYWxsIGluY29taW5nIGUtbWFp bHMgc2VudCB0byBXZWF0aGVyZm9yZCBlLW1haWwgYWNjb3VudHMgd2lsbCBiZSBhcmNoaXZlZCBh bmQgbWF5IGJlIHNjYW5uZWQgYnkgdXMgYW5kL29yIGJ5IGV4dGVybmFsIHNlcnZpY2UgcHJvdmlk ZXJzIHRvIGRldGVjdCBhbmQgcHJldmVudCB0aHJlYXRzIHRvIG91ciBzeXN0ZW1zLCBpbnZlc3Rp Z2F0ZSBpbGxlZ2FsIG9yIGluYXBwcm9wcmlhdGUgYmVoYXZpb3IsIGFuZC9vciBlbGltaW5hdGUg dW5zb2xpY2l0ZWQgcHJvbW90aW9uYWwgZS1tYWlscyAoc3BhbSkuIFRoaXMgcHJvY2VzcyBjb3Vs ZCByZXN1bHQgaW4gZGVsZXRpb24gb2YgYSBsZWdpdGltYXRlIGUtbWFpbCBiZWZvcmUgaXQgaXMg cmVhZCBieSBpdHMgaW50ZW5kZWQgcmVjaXBpZW50IGF0IG91ciBvcmdhbml6YXRpb24uIE1vcmVv dmVyLCBiYXNlZCBvbiB0aGUgc2Nhbm5pbmcgcmVzdWx0cywgdGhlIGZ1bGwgdGV4dCBvZiBlLW1h aWxzIGFuZCBhdHRhY2htZW50cyBtYXkgYmUgbWFkZSBhdmFpbGFibGUgdG8gV2VhdGhlcmZvcmQg c2VjdXJpdHkgYW5kIG90aGVyIHBlcnNvbm5lbCBmb3IgcmV2aWV3IGFuZCBhcHByb3ByaWF0ZSBh Y3Rpb24uIElmIHlvdSBoYXZlIGFueSBjb25jZXJucyBhYm91dCB0aGlzIHByb2Nlc3MsDQogIHBs ZWFzZSBjb250YWN0IHVzIGF0IGRhdGFwcml2YWN5QHdlYXRoZXJmb3JkLmNvbTxtYWlsdG86ZGF0 YXByaXZhY3lAd2VhdGhlcmZvcmQuY29tPi4NCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fDQpmcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmc8bWFpbHRvOmZy ZWVic2QtaGFja2Vyc0BmcmVlYnNkLm9yZz4gbWFpbGluZyBsaXN0DQpodHRwczovL2xpc3RzLmZy ZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJzZC1oYWNrZXJzDQpUbyB1bnN1YnNjcmli ZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJzZC1oYWNrZXJzLXVuc3Vic2NyaWJlQGZyZWVic2Qu b3JnPG1haWx0bzpmcmVlYnNkLWhhY2tlcnMtdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmc+Ig0K From owner-freebsd-hackers@freebsd.org Mon Jun 12 06:13:53 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53598BF4D99 for ; Mon, 12 Jun 2017 06:13:53 +0000 (UTC) (envelope-from amutu@amutu.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0871B83B6C for ; Mon, 12 Jun 2017 06:13:52 +0000 (UTC) (envelope-from amutu@amutu.com) Received: by mail-oi0-x231.google.com with SMTP id k145so40004908oih.3 for ; Sun, 11 Jun 2017 23:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amutu-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=EMY7hwj8VduAIfatVYPv/s+Tmyp3dT5M+0FOgsPaaV4=; b=d+XOO0Lab2LqlhmRaq6YJDQUpp1WCckXj92gbILuMp7zVbq7/72ima+FEuTHpMX9Ww YP3b5sDvLPs0hNfABAY2l77gEtWPkCAhWNea26HxOHthIbzEuEYHQIMxdr/xMPwRe1i3 IyU1/X1APiUsIfZPw0iQjHU9RZSqXcr74RvgeY3cdLQkCYTemPVTIQf5wrTFD4GlLmKn MovwjRb8YF+P9nByBVJ2G05+DfAC1tDIrcyLQnfYMw4L4fCTz9O8v0lT0/zTPzCB4k2D tSLVWgGunaIUFwXX3pv/+8XUCzEfNyuxJSZ7s0je5IAvMVF0OzEytxlFZlbjvRQ2z+P0 UOwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=EMY7hwj8VduAIfatVYPv/s+Tmyp3dT5M+0FOgsPaaV4=; b=QNiHnWkMMzinOMm3HZUwBTZZtZV91igPRk6+f/2Ui7whHSZMolNpXJeGj7MvnMH8fn Bn1eYGKsuIxvr684PmDiKDS0J7ZXoocz3+wJbx3rt5BMV4ZcZ8YhZDVKbCjayTbicbxx IpG1YvqdV8D0cmkmg8Mkvrt4hgkkE8mOv4QFJlHxq/d0kcMg3gbjYOjv70D4iYl6nrq5 ozGNOUoAby43klZ5xSozvnd3/VRB72SCqOhkzWhZbaVRE6eOtDKEgbH0Y+8Fpe9NyYuh UwKlbe9dmElZP++7lQfnwTsyJe0fa9VZ1G+THU3sZHqlvG4jHkHViapYlV8C06dfTHEn 7rCg== X-Gm-Message-State: AODbwcCd1qpqIbiILCBzttUb8ZhCWoUq8RPnMpQvfHREkjSfz1Unx6lx 4mIOJi2lUSQQKslFL1ZBZg== X-Received: by 10.202.69.6 with SMTP id s6mr27176694oia.193.1497248031982; Sun, 11 Jun 2017 23:13:51 -0700 (PDT) Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com. [209.85.218.43]) by smtp.gmail.com with ESMTPSA id c42sm4496772otc.58.2017.06.11.23.13.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Jun 2017 23:13:51 -0700 (PDT) Received: by mail-oi0-f43.google.com with SMTP id s64so6241453oif.1; Sun, 11 Jun 2017 23:13:51 -0700 (PDT) X-Received: by 10.202.227.3 with SMTP id a3mr1852330oih.52.1497248031209; Sun, 11 Jun 2017 23:13:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.133.136 with HTTP; Sun, 11 Jun 2017 23:13:30 -0700 (PDT) In-Reply-To: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> References: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> From: Jov Date: Mon, 12 Jun 2017 14:13:30 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: "Caza, Aaron" Cc: "freebsd-hackers@freebsd.org" , freebsd-fs Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2017 06:13:53 -0000 >From the output of explain analyze of PG, the problem can be excluded from the database.I am not a fs expert, I CCed freebsd-fs@freebsd.org.It may be helpful if you provide more info such as sysctl -a | grep zfs after degradation. 2017-06-12 12:50 GMT+08:00 Caza, Aaron : > Thanks, Jov, for you suggestions. Per your e-mail I added =E2=80=9Cexpla= in > analyze=E2=80=9D to the script: > > > > #!/bin/sh > psql --username=3Dtest --password=3Dsupersecret -h /db -d test << EOL > \timing on > explain analyze select count(*) from test; > \q > EOL > > Sample run of above script before degradation: > > Timing is on. > > QUERY > PLAN > > ------------------------------------------------------------ > ------------------------------------------------------------------------ > > Aggregate (cost=3D3350822.35..3350822.36 rows=3D1 width=3D0) (actual > time=3D60234.556..60234.556 rows=3D1 loops=3D1) > > -> Seq Scan on test (cost=3D0.00..3296901.08 rows=3D21568508 width= =3D0) > (actual time=3D1.126..57021.470 rows=3D21568508 loops=3D1) > > Planning time: 4.968 ms > > Execution time: 60234.649 ms > > (4 rows) > > > > Time: 60248.503 ms > > test$ uptime > > 10:33PM up 7 mins, 3 users, load averages: 1.68, 1.79, 0.94 > > > > > > Sample run of above script after degradation (~11.33 hours uptime): > > Timing is on. > > QUERY > PLAN > > ------------------------------------------------------------ > ------------------------------------------------------------------------- > > Aggregate (cost=3D3350822.35..3350822.36 rows=3D1 width=3D0) (actual > time=3D485669.361..485669.361 rows=3D1 loops=3D1) > > -> Seq Scan on test (cost=3D0.00..3296901.08 rows=3D21568508 width= =3D0) > (actual time=3D0.008..483241.253 rows=3D21568508 loops=3D1) > > Planning time: 0.529 ms > > Execution time: 485669.411 ms > > (4 rows) > > > > Time: 485670.432 ms > > test$ uptime > > 9:59PM up 11:21, 2 users, load averages: 1.11, 2.13, 2.14 > > > > > > Regarding dd=E2=80=99ing the pgdata directory, that didn=E2=80=99t work f= or me as Postgres > splits the database up into multiple 2GB files =E2=80=93 dd=E2=80=99ing o= f a 2GB file on a > system with 8GB ram doesn=E2=80=99t seem representative. I opted to crea= te a 16GB > file (dd if=3D/dev/random of=3D/testdb/test bs=3D1m count=3D16000) on th= e > pertinent ZFS file system then performed dd operation on that: > > > > Sample of run after degradation (~11.66 hours uptime): > > 16000+0 records in > > 16000+0 records out > > 16777216000 bytes transferred in 274.841792 secs (61043176 bytes/sec) > > test$ uptime > > 10:25PM up 11:46, 2 users, load averages: 1.00, 1.28, 1.59 > > > > > > After rebooting, we can see **MUCH** before performance: > > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > 16000+0 records in > > 16000+0 records out > > 16777216000 bytes transferred in 19.456043 secs (862313883 bytes/sec) > > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > 16000+0 records in > > 16000+0 records out > > 16777216000 bytes transferred in 19.375321 secs (865906473 bytes/sec) > > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > 16000+0 records in > > 16000+0 records out > > 16777216000 bytes transferred in 19.173458 secs (875022968 bytes/sec) > > test$ uptime > > 10:30PM up 4 mins, 3 users, load averages: 3.52, 1.62, 0.69 > > > > These tests were conducted with the previously mentioned Samsung 850 Pro > 256GB SSDs (Intel Xeon E31240 with 8GB ram). There=E2=80=99s essentially= nothing > else running on this system (99.5-100% idle) and no other disk activity. > > > > Regards, > > A > > > > *From:* Jov [mailto:amutu@amutu.com] > *Sent:* Sunday, June 11, 2017 5:50 PM > *To:* Caza, Aaron > *Cc:* freebsd-hackers@freebsd.org; Allan Jude > > *Subject:* [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD > performance drop < 24 hours > > > > To exclude the fs problem=EF=BC=8CI will do a dd test on the pgdata data = set > after the performance drop,if the read and/or write utility can reach 100= % > or performance expected then I will say the problem is not fs or os. > > > > For pg,what's your output of explain analyze before and after performance > drop? > > > > 2017=E5=B9=B46=E6=9C=8812=E6=97=A5 12:51 AM=EF=BC=8C"Caza, Aaron" =E5=86=99=E9=81=93=EF=BC=9A > > Thanks Allan for the suggestions. I tried gstat -d but deletes (d/s) > doesn't seem to be it as it stays at 0 despite vfs.zfs.trim.enabled=3D1. > > This is most likely due to the "layering" I use as, for historical > reasons, I have GEOM ELI set up to essentially emulate 4k sectors > regardless of the underlying media. I do my own alignment and partition > sizing as well as have the ZFS record size set to 8k for Postgres. > > In gstat, the SSDs %busy is 90-100% on startup after reboot. Once the > performance degradation hits (<24 hours later), I'm seeing %busy at ~10%. > > #!/bin/sh > psql --username=3Dtest --password=3Dsupersecret -h /db -d test << EOL > \timing on > select count(*) from test; > \q > EOL > > Sample run of above script after reboot (before degradation hits) (Samsun= g > 850 Pros in ZFS mirror): > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 57029.262 ms > > Sample run of above script after degradation (Samsung 850 Pros in ZFS > mirror): > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 583595.239 ms > (Uptime ~1 day in this particular case.) > > > Any other suggestions? > > Regards, > A > > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@ > freebsd.org] On Behalf Of Allan Jude > Sent: Saturday, June 10, 2017 9:40 PM > To: freebsd-hackers@freebsd.org > Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD > performance drop < 24 hours > > On 06/10/2017 12:36, Slawa Olhovchenkov wrote: > > On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > > > >> Gents, > >> > >> I'm experiencing an issue where iterating over a PostgreSQL table of > ~21.5 million rows (select count(*)) goes from ~35 seconds to ~635 second= s > on Intel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back > from Jan 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm > using PostgreSQL 9.5.7. > >> > >> I've tried: > >> > >> * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, > 2017. > >> > >> * Tested on half a dozen machines with different models of SSDs: > >> > >> o Intel 510s (120GB) in ZFS mirrored pair > >> > >> o Intel 520s (120GB) in ZFS mirrored pair > >> > >> o Intel 540s (120GB) in ZFS mirrored pair > >> > >> o Samsung 850 Pros (256GB) in ZFS mirrored pair > >> > >> * Using bonnie++ to remove Postgres from the equation and > performance does indeed drop. > >> > >> * Rebooting server and immediately re-running test and > performance is back to original. > >> > >> * Tried using Karl Denninger's patch from PR187594 (which took > some work to find a kernel that the FreeBSD10 patch would both apply and > compile cleanly against). > >> > >> * Tried disabling ZFS lz4 compression. > >> > >> * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQ= L > 9.1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime > and test took ~80 seconds after which I rebooted and re-ran test and was > still at ~80 seconds (older processor and memory in this system). > >> > >> I realize that there's a whole lot of info I'm not including (dmesg, > zfs-stats -a, gstat, et cetera): I'm hoping some enlightened individual > will be able to point me to a solution with only the above to go on. > > > > Just a random guess: can you try r307264 (I am mean regression in > > r307266)? > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@ > freebsd.org" > > > > This sounds a bit like an issue I investigated for a customer a few month= s > ago. > > Look at gstat -d (includes DELETE operations like TRIM) > > If you see a lot of that happening, but try: vfs.zfs.trim.enabled=3D0 in > /boot/loader.conf and see if your issues go away. > > the FreeBSD TRIM code for ZFS basicallys waits until the sector has been > free for a while (to avoid doing a TRIM on a block we'll immediately > reuse), so your benchmark will run file for a little while, then suddenly > the TRIM will kick in. > > For postgres, fio, bonnie++ etc, make sure the ZFS dataset you are storin= g > the data on / benchmarking has a recordsize that matches the workload. > > If you are doing a write-only benchmark, and you see lots of reads in > gstat, you know you are having to do read/modify/write's, and that is why > your performance is so bad. > > > -- > Allan Jude > _______________________________________________ > freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/ > mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > > This message may contain confidential and privileged information. If it > has been sent to you in error, please reply to advise the sender of the > error and then immediately delete it. If you are not the intended > recipient, do not read, copy, disclose or otherwise use this message. The > sender disclaims any liability for such unauthorized use. PLEASE NOTE tha= t > all incoming e-mails sent to Weatherford e-mail accounts will be archived > and may be scanned by us and/or by external service providers to detect a= nd > prevent threats to our systems, investigate illegal or inappropriate > behavior, and/or eliminate unsolicited promotional e-mails (spam). This > process could result in deletion of a legitimate e-mail before it is read > by its intended recipient at our organization. Moreover, based on the > scanning results, the full text of e-mails and attachments may be made > available to Weatherford security and other personnel for review and > appropriate action. If you have any concerns about this process, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@freebsd.org Mon Jun 12 17:48:01 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAF51C092C5 for ; Mon, 12 Jun 2017 17:48:01 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8183D77DB7 for ; Mon, 12 Jun 2017 17:48:01 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: by mail-it0-x231.google.com with SMTP id m62so27066281itc.0 for ; Mon, 12 Jun 2017 10:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+hXlkvoE3bOefpBCUsW6Ghh6ZV6FQ8PkY4jOW6qMCzk=; b=LtmTOsopwOGjyrdkaXMQdRlGvhPFEEZwBGB7AXtQMgVJaJtXvccLuyMKU6OoPdoDLf 5GhUJ4zNsxeBeStwooMQk9YqWB3BvmLFWyLGEnWR/OF2i4XcjV1KtwS5dfqWlIHumFNq N9Cqf2aVbo7nK7K+aWbVEzD5VEJXkiAO/NG0BVOaRScjjnUc5Uur2KzIcRLiZi2xVSAU WVax0BRCGkii+Js//anzRmw6iXpAV2RRP4JpmhxTTaC2v0q2+mu1fb7lBL7ca8IA72lR IYocvXyX29ELzR0akkAZUVEjLWJfbt3Gb7wV5e34riKPsk3a360kFFEJu1hglA+AJyKu kNrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+hXlkvoE3bOefpBCUsW6Ghh6ZV6FQ8PkY4jOW6qMCzk=; b=PJDsAAFLlctmk61s9pexKRawPMDbaU17ZUlQFbsI7GEHUDe5BBNXCbLJ0j4TaYXRzf 4D3xzPqOCXn4lFBR8HBbFVHP9osXWOAfqSAwGT/ZJoCuW7FrrqhNWu6S2g8pteuRC0e8 Eg1m46ukVyxLUjK0UKr98CxyLazd1vooqXMfB4uLIr9GXXlRgyURK1bmSifqxK7vcil2 USiBgaL8BQyA65k1rdSNOH1KIvIW3q7AZq6Mmls7zxO3yLKsg5fQXA0FQYHN//SN21pl yJPqr/MF/58+f12FsDNSCodySx9Lb9SgEKgvWhxGyU60FM7qWt2JHcO0drgbMUZ+ZBCZ A5/w== X-Gm-Message-State: AODbwcDTYRcud6TWiHfm09D8hx4Ztq1kUPUzYNY5I4p93r8Q4jw4UcuO Y4otK1t9sK1M3uFNk0K2sQFKHfkq7w== X-Received: by 10.36.69.19 with SMTP id y19mr12825988ita.26.1497289680806; Mon, 12 Jun 2017 10:48:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.75.132 with HTTP; Mon, 12 Jun 2017 10:48:00 -0700 (PDT) Received: by 10.79.75.132 with HTTP; Mon, 12 Jun 2017 10:48:00 -0700 (PDT) In-Reply-To: <1738C902-7A9C-4711-A2F7-D302249C9AF8@dsl-only.net> References: <1738C902-7A9C-4711-A2F7-D302249C9AF8@dsl-only.net> From: "rollingbits (Lucas)" Date: Mon, 12 Jun 2017 14:48:00 -0300 Message-ID: Subject: Re: 32-bit powerpc FreeBSD head -r317820 panic example: Example showing register r10 having been trashed during set_affinity's time frame To: Mark Millard Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2017 17:48:01 -0000 On Jun 7, 2017 1:50 AM, "Mark Millard" wrote: Summary of the analysis below: (...) I think you are approaching your problem from the wrong side but I also don't have any other clues. --=20 rollingbits -- =F0=9F=93=A7 rollingbits@gmail.com =F0=9F=93=A7 rollingbits@= terra.com.br =F0=9F=93=A7 rollingbits@yahoo.com =F0=9F=93=A7 rollingbits@globo.com From owner-freebsd-hackers@freebsd.org Tue Jun 13 06:24:41 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEF47D8F7BA for ; Tue, 13 Jun 2017 06:24:41 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0068.outbound.protection.outlook.com [104.47.41.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44E1B6F920 for ; Tue, 13 Jun 2017 06:24:40 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=iImpU1JFa7ieZIf+U3ATfCU7JnNpz0nmS14OijDDI4c=; b=U1FBZQWW2nYlt2UjgsIcoJ3A2aaH3M7w/zc8FdwtRmh7Dcz/9vFGRMrVkQSuZMBi+64Gj8/7hSneRwYQruo+0EAtkg2DiyHYDScUSRYrFpBjOeOTOFybZwa44ZwNQPJU2uryfxzTTTYrZMTmZ3hHBkE6kzxgbtnP69jgeSZSHPg= Received: from BN6PR03CA0018.namprd03.prod.outlook.com (10.168.230.156) by BY2PR0301MB0742.namprd03.prod.outlook.com (10.160.63.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 06:24:37 +0000 Received: from BL2FFO11FD009.protection.gbl (2a01:111:f400:7c09::177) by BN6PR03CA0018.outlook.office365.com (2603:10b6:404:23::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Tue, 13 Jun 2017 06:24:36 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; zxy.spb.ru; dkim=none (message not signed) header.d=none;zxy.spb.ru; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BL2FFO11FD009.mail.protection.outlook.com (10.173.161.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Tue, 13 Jun 2017 06:24:36 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB014.032d.mgd.msft.net (141.251.110.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 06:24:35 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Tue, 13 Jun 2017 06:24:34 +0000 From: "Caza, Aaron" To: Slawa Olhovchenkov CC: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLiAalX9ncyh3crQla1B5/716RidAABhQ0AADIocOAATt+h8A== Date: Tue, 13 Jun 2017 06:24:34 +0000 Message-ID: References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> <20170610163642.GA18123@zxy.spb.ru> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.110.133] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB014:|BY2PR0301MB0742: X-MS-Office365-Filtering-Correlation-Id: 3d32a4ef-644f-4d54-ea42-08d4b224dd0b Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB014.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39400400002)(39450400003)(39840400002)(39860400002)(39850400002)(39410400002)(2980300002)(438002)(43784003)(199003)(189002)(377454003)(13464003)(24454002)(51914003)(9170700003)(305945005)(55016002)(2900100001)(6916009)(189998001)(6246003)(478600001)(47776003)(53936002)(8676002)(72206003)(356003)(42882006)(8936002)(86362001)(81166006)(66066001)(8746002)(22756006)(50986999)(54356999)(7736002)(3846002)(23726003)(102836003)(9686003)(97756001)(76176999)(24736003)(2906002)(46406003)(108616004)(5660300001)(33646002)(7696004)(53546009)(110136004)(229853002)(86146001)(4326008)(38730400002)(5890100001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR0301MB0742; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; A:0; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD009; 1:Uo1FEGkKMMPy5/jCED+H3eRlmlDdKDNB6rfr4f2iwuVPdfzfIrferv4VLOVfwrOhGbJqEWb+6nAu6/47jgbe+OPhCHMg3oN1Pmifh9v1jfsmS+W6/BI/mBXXeua5zJE8/Q5KtGE1I6xQp8xTRlt09wFeHg70pwmbuJh8tulgmloO5Qa7Kmfs1VgaNSnibyK0h459hmcELIfEqvgJLWHV8uJrONznVhiZwrCoaLN/SV3YFu4EC+XK8oie0T+vv/kk75llsHZWjfieccaYlYCKv3kVuqrbwPDdMel8qE26Htc24pJ42EzU4Y4pEanovuW4t8No449K3xFsVhJe9DYT3mNtiJfqWMQAqTzTYOvnkZpzHomON7I7zyzAIZ2q97HSg+Jxf0c3GqbCUqKCB+U7K3aAudqyvfQW08ggyZQ8lGNxph4sqgS3qX6MfspUhU8ByvHehlbnpHheg88L7l8nx1z9NLatZj8qBX0r3beobLWPMNeUnhyt/6k06CCW2uN6gCmRLItnv5leUerNQjJbqw== X-CrossPremisesHeadersPromoted: BL2FFO11FD009.protection.gbl X-CrossPremisesHeadersFiltered: BL2FFO11FD009.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:BY2PR0301MB0742; X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 3:OFJ5Nb+iJn8nTtC3pu3BY68GsFJinl7EQ54B3pin/xVPnJtMy4Nf4tQRl8tsjbCP641pi9Pmz4iakpsoGPTrVOPnCK3hILa7kLeS6QBnyWjCSkQShGU/TOk0lTfzCncUvavnYEk0PLQrgVKu/w8E/HlXM0amvaPB9XOgfuDf9rpSYQAadlCBp6wtZ8uDY/mKWfdQmRo/iwQpO9FtedRU7N7B2uEBGKyLmH5X9jlWbk/u4RTt0xIRVqW9uyWGS0JF9puKmKcrOCVLHniiQsv4en/Eiqq6Yi6FNONtbrVMgtkzM7kzWcdpWdPqbORQWHwW0XCK3OGCA8U1J1D4lre6ZgZynfKdTluJPit1HXep2X7fXXBFAtXofFHUTC3Qt2feNSs45/ey1OhR24F1zDVU7NWlOVO6XOXZgpnnbfTF3sk3B5BlBF23Sam2CFLb0ItG3VysGRuY7WUCf1agCWufse6xP5ZNi0i0no8wzr15Zv4= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 25:eoQbmX15lldkkjQFSSd4dsUoHK+RR8NP9kmetbcR8rrwSwVrP32y7Qkcu4aKXBSLZx1OC9b+oXSgaLToavJUEANC0ZgCGzipDPW/8FJcBILajv47/s9G3MHmEukW9DE2cZ6w5hA6fKWAokaCjLCM1dOaxSucCJlIyYqG+zPUr83nc4jpCJGDRFv7oxhUjsejQchzY9+ELhY4FJmEOhjIYnXKAxekWBAyjcY5nsR1O/+z1n60BPab477S0mem6ocv3ouBKfpZW84WpWikDuERujEyyKs18oRim/qZUU/EdsFw48MIkd143azTXhYmgxkl+elgeD1KDjxD/LC0PYkkjZEKmq91HHg3KrRwRxtVzgN/EkmPM2GNrh8lrT6PLOU+tTZd3G2nIIQdorG5GX6+Xdoa9hsiBWLVcrP8mMpmKPfx5kT+0rI+bAN13Ge66RfE9YhdRt3joIeZHFKPt9+Gy3OQ79+jAq5cTweZIGFOc9U=; 31:18htVaOK6tHZb9eC/+2j72NBdzBETRDXtNYI6JuJyA+QiDBaRHs4G36mtSVabVx1qpeiYLV2NlHI6zddE462hQSRkVUSR3cbkpL095vi99neJbnDtMGU35egYEvEv2qalHAoj3jgg5atm6i1eb1hwnwORzamPnOIby/KviQnvR207FjPqPLu3GNWTOWq2Cg3BIiYV9KKza6Eu3N9FhiHJGYRxC0XhmaQmxh7pDliEeZpN3wrirxtfL0TedtanoQCXj+E2Qqyzy2OyhVrnQirGg== X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 20:QOmc+AkX2OA/3pOgYNeVqr0CT0abO8mehEfHkCH+jGr6YAkdT9Y1ggB3wmKCuhojnvrcT/4keISw7LCGXT1II0n4CccfeDB08VazRkwGd3R9KyH/WtBslvV5WZ6enCc4OoVX3uqzu0jPTsT/zJAUngjVHQizCEW4cra5nhazJm6YcGnWk7h+QPi+2nXF6/n1Vvmx2o616MOO0xN374G3O9hpDqywavIGHDZtbBMHbHutpU2CBsdrvrjvYVVOkFktipYYvlIwFb5aX371ynbrqmnVOgtiPzlXuZerG+7aV8jmKqnbvfB7dFrXv4kl1ddd/JDxHZqmB+jWKrd0KmJQeo9QC8TD3qA0cXZlk7QAJXXS0CrBx1IKo/P5OhrffR90YuP3/DN05eDu1aJZaeSjdySBtGUc6YGOO3yJTjqBbeRSh6WDcKXGZvFLnts8dWoVBNwThLvIuqCel7SKND6Ij0Y5By7+4XEP4xqIVvKLDEGwwWCbqDfamyo5zVmbNRSb X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(5005006)(13016025)(8121501046)(93006095)(93004095)(10201501046)(3002001)(100000703101)(100105400095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123558100)(20161123562025)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY2PR0301MB0742; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2PR0301MB0742; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0742; 4:t7X5TsA07pDeTz9cXvxSe9XNZrMQRFNZm/cXgDW9?= =?us-ascii?Q?t8ESZhcp5/prNwv+XJLTxV9TitOvp0oim/kDoDJ2TK2+2mqF7/g2RyduRKs/?= =?us-ascii?Q?mtMhA7QW0cYAHVGfXDjydRNQNuZ6Md3P5CGHJcSO5XoXRSe0NSqyZ44wQrzP?= =?us-ascii?Q?Q7ZauF4UQP93ADq/oXENN3pqhx4GrcoosdJiC7ysW40OSw2ARg08tAhZk06+?= =?us-ascii?Q?uZkKyo0rFuAMVFw5CSVQdKvF6A8K0YdkE0wpGRvqfKFyCnan+lLCpaf+HPSW?= =?us-ascii?Q?NXJkO/Flnughtfz1jkI5CnWwLgeKMSsiWJA0ezQ6hwqLZ3BGUKgb8N9h1rbO?= =?us-ascii?Q?JJ7hJ/DMLQ4nyzR9K97byOkb3U+DB7UXyTx9iKKGnb7Mr0fEMrjihwVNCkTw?= =?us-ascii?Q?ewbQt9pXC3wd7NrWNSccdxm0qLi4SpXq7uUnWPzuQ1IWQ87hpTH14FxWA09B?= =?us-ascii?Q?81+q2O53/JXctMFdBOIHM34kBJC61KDehVdTQXhlxc1FPsNnG3cxeX9A1AoG?= =?us-ascii?Q?PrhKcDboK51HkkVsw0ECq11iGwvTKPva+SPbteFYdQ1MrWQNhidwicp3eDXg?= =?us-ascii?Q?NNxspUkbceJjCwu1aZFifPTJrBwhgfDAsoV3OXMrwf9ldZTr4BzJFB9aNZWK?= =?us-ascii?Q?tONhSgaJdkD48MbRVN7Pr6UAyiAIxxRkRmhciIF6ARXzFsw2RLDaEYoM/wQm?= =?us-ascii?Q?gFoikBK8ciwNmZkfGZmpPqqdI5jIP+4E5ANsIGgJUiAjaOHjNjfAY27klh1P?= =?us-ascii?Q?UdH3oovPeCNuPEyXfnut1YE2PE8YOjEr73tyVuLRJD/WUaH5fLIoiDthe+Go?= =?us-ascii?Q?LxZWSpi4iWiIM1/4TMkAN5ABA4AIZi3ij5PE0RK528JA7N5GHMSOVaQK2iop?= =?us-ascii?Q?BtUvlCpw2PJw2uVRM7zkqsN461WwvfY6Ofz2B/vYHWQLu1YhYJ0H5U7vAD09?= =?us-ascii?Q?k1K1DEBA7USxrXMtm3O0hvUT7iYk1yD/sNglFYdtcqintSkwwKxeY2LY2P/g?= =?us-ascii?Q?UjgkUh4W4zGSKtH1a9gT3p8yKKqz6NSNXtlDN826VJWkLw+cn3oKVsJpHrkp?= =?us-ascii?Q?M9XWAHxBb3rBYpwhiPhX7EKNYLJcBeyW4Y0W5lGTqvPXwXZbmX+40VeQ8Kxa?= =?us-ascii?Q?cUcluBfyigmOlYQm88vAwQmUgks7D3qq3xDUqUeZPq9W9JQ7LBeO7NKJCgz0?= =?us-ascii?Q?2bjO1FOz296THDm3He4dvgznyoqeL/sXWMD4eOeV8Ahkgs2uWgTFCTIbq0i3?= =?us-ascii?Q?izGQmbSE2kMYuRVWcTaZRTtS/Z2BXbKloTtwVmI1Tk/zSAvxoZ2iyGDovPfB?= =?us-ascii?Q?3bMDwd/vGvDtGbCNT+gYcvs=3D?= X-Forefront-PRVS: 0337AFFE9A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0301MB0742; 23:7fhr7HatSj/pwHtFOTtTkI7FDpb0MLCfi3zdWN1?= =?us-ascii?Q?Znn1cDn9KNcrUQk9kGT96YWkkOSSs0FbF4wIsFPpFdFhCAbhypiUVFWhADTW?= =?us-ascii?Q?NjNQ3Cf0JhB6p4pDc2HAWONzoVR3qI0Yvfnj2D/PMzFXLav2o+MdiwI/I7/F?= =?us-ascii?Q?+f4PE1ZLQxM0POFLCZc+Hp4BxaBeaGpze0DzXBPt0nIQjYrLV8X1gd2D6ZWT?= =?us-ascii?Q?xEIgl6VxKXyS5KDoS2okviGYUWsPtn9C56+Dn3Nr3cnloSO5pZel1sYZHDvl?= =?us-ascii?Q?9ffnlCl3L5IwdmZcG88qq1To9gIVY58oY6VINAhfJdntW7cgsE+grZRh7HHk?= =?us-ascii?Q?eegan7jYBoW3+8jzbk3EJbG7SfvpHZnAmZkHmIZK0E39wQ4P6U/TiJ6n9wje?= =?us-ascii?Q?quyZ+B3n0O1q5zX+zvKNqjgMf4gG08JP8ObnfNMrFjUzRyxqiBKKHh2zi70B?= =?us-ascii?Q?OTs/+xKjFPgCBYpUAgvA7sIOkbNTsU33cFpHh2wjczpUVHMIInWPGLOeqDt+?= =?us-ascii?Q?AUEIJhX808xLpV6lOKuN3CW3ZdhS2xfqcH8eYrogL5SbdSwppVQ90/rIZhNG?= =?us-ascii?Q?flVbUcgvUiYmfZHOjuWrAMsEQvVCJE/nGEpVr6LNjq+wnwXpUPbQt2Gi2TTo?= =?us-ascii?Q?NqpTHGq0oOKPc/DT52QabAp6d9Z/rHoL0hKBXM6OPDyi+FcOaLXyMvKY+B8D?= =?us-ascii?Q?qR1IKVmcq1mo6wMSkNmBGXG4eWbO7SKF+Ga8AZE6+gBQ88Srje/2wllvGXmW?= =?us-ascii?Q?a8VpHrhOWlEcsIELCEizAPh625GBPCIFInnFjjeDUsVc8ypcJqnQC0bn5yTf?= =?us-ascii?Q?+t8Etpk0Z5yPiyckMVqIfYn/ING7KLStte6N0O7GGFEd39cOpV2dpjI4jxNB?= =?us-ascii?Q?y+PxV2HSmLmZIKgU7L33ve5zweKimvc2NNekOB3B3iqbJXsEIQqca58Dw1kq?= =?us-ascii?Q?sfqzzbx7DYvQfOB6MYTZNhQNNHmIQbAVu4rUvbf1NcERj3Ddi7T4hdPPFyWI?= =?us-ascii?Q?GuIrvUdHuwBof/pz2KUjVC/ekbF3o0qDpBZMCsZjFI/IVuA33h/DuhyJLFtA?= =?us-ascii?Q?JAFfeJy8sEPoImgvMmR4gc9gZu8P4Z5AGgeh3q//uh8fr//zi10cvmd4Ey+6?= =?us-ascii?Q?9jfVA1g3cNJ3jg5IXsasItrB7Nzynf1PtwRhvCbwW6Mbr3KemcHgsSSuSKql?= =?us-ascii?Q?G9AoEh4XawSC2k2JoG+Vu5932u0Xb4b2mxNPh0+gmMlshz8acJtLhkZG2ynA?= =?us-ascii?Q?oGuTFNjAWN8Nas//+Xq/rhsHH+FlC3I3xa7p7Zvv1W5QXIBXvaAUyi3y73m+?= =?us-ascii?Q?VsMbwZEXZHZOnwxv25L/CD3ZgDxBhiPC4fWEUDLCOPKMfHVTUnCVzPB5FHOU?= =?us-ascii?Q?K/vTYch2kqu/iXLTbFskHKL8xC0dP1oxv9Ty0KwrIU+a6O8rQuINvzxfKjMp?= =?us-ascii?Q?iNYn60FATXw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 6:w+Q58aQQ421QNQAVWn3oN08OdfLznlBxXzX/uokncT3R0FIOlOT6xJES0PJewqcX9nfHySz43ITutwE22WHu+sPgo4ecG7kXvUuHucRtIJEbGQh6hZk5v9bnqXlCvuF6tg9m4qqZ4HZ9cnetk60O8/TQ/oYvt28NWGT7yzN13CoFNtqfLrFAnhhua/itSN8wiwKF/QH9A1sZi+ZWT+vTQoRHSLnE5hro/jL+8c0CFiNJ4DPumY2dGylcTyMGbOGxB+RXn8V+7w7fMMiq1YG50CbwxTJHWCMXRE0PRKyciUZSrgEl90UiPUOxbwG+OxBvFLyNLyJ6iTuLbAAXmfl2I4G7KEHKbW4QzcwRzRaxroKEXiuCh1ZutM+mnstHhiPe6xMOk53c+p2NqzQ/vRZMGRO+PlFo+zl/11uYy+/823UfhdmJ8n7aYaYKfrRugkBn/2Al8wfLdFlzm1JdIu7gGFfeRLVcyg0/UfCpytdsMYmZb1/hgsnNA9e4rO2mnX+x6ISSr9UDATiXBw8jTFjEJM2RUXoJ2QlR+nGn6RWy33s= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 5:Xb7OuB7G6yJx1C69+JxuiCUbA8qH3pPe2qeeBcN44RmRtpyzwrI0SZ/pButzRY95oDAGZWHJhN2LaIec380970as8GkTGoErCmu+avDHVFrGdGFQ2gXhA4x9b5shbRcOX3J2nEoNJwriDhH0Ivm0BDIKv0jo6IVVge+C4WN8ij0K8g8RGdmnOEdkl1Vp0sNv6C2Rm3RoEjF5QTODHkMTCbe8bXRuXs/5VMAt8G7r0pW5U0+RAoC3YXSCaknN/1RBDxYvLNV5N9uIzXlzs9t9cJBkGjxftGhEdqoyf8hcYr9LC5RN0bRRnXp/eWjCi7Xx9tllmzhNfUxUosODANSBb/a1ZuLAIXDPz+KCgckdzi5iNoznbKMHMSWJGPeQOHoDp3ovGTDMPr4PHXxX1e8wWSrBPnehwIukqJ9T+eYx/tQg+aYlpfUX6TDFSxneEVSBc0oj81V+XzXn0I06J/I7Z14bAMjhGlVxABY1D0xNhud9q1Ci3SFiYPPILZdDD1wi; 24:o9lzbAc3pl9sLB82iOwOpeLO1QDGvX3Qp+z+p4BuSY39iVfmXchEgoraZrBu+s9ZNRHAHAK8mp7ePwurGJDRc7zn6AzMs5mRwc/twFin4dg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY2PR0301MB0742; 7:8hQRGs295ZNn/7ZVLQS/V4BLC3A023OQfXZdDtKRqFVheCx5Y1zrL0rjpfcIr6d2WiC85gpoCND7Plj9gMXRfLH+b6GC3KAW/D2qsadPKHRMm07CxzyduwiKEs3LDqUQxetPuz3VKbZ0YsMQVknOCb5wT9Q/ljNIYbfOtLyGVOvM8N/ElWon1dBMEzGYKbPz6PSDVNlvE/uvZ/RdHDX8qJEepyisyIO4hqqKcdBrstklxFVJDpPKHHwaqTmKSRuSvGAjNRCu/mYxCzl91pp+1Znj2F7C9CnJBNWOKfDEfc9J9XjfXfmVVOAwrDVAkPtFBkW9r7m2eCgETeCt6OxVDA== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2017 06:24:36.2414 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0301MB0742 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 06:24:41 -0000 Slawa - thanks again for your suggestion but, unfortunately, same behavior: (Server is Intel Core i5-2500K w 16GB ram and 2x Intel 520 120GB SSDs): PostgreSQL test results immediately after reboot: Timing is on. count ---------- 21568508 (1 row) Time: 53855.000 ms test$ uptime 7:51PM up 2 mins, 1 user, load averages: 2.46, 0.88, 0.34 test$ uname -a FreeBSD xyz.com 10.3-STABLE FreeBSD 10.3-STABLE #0 r307264M: Mon Jun 12 17:= 48:24 MDT 2017 aaronc@WFT:XYZ amd64 PostgreSQL test results after ~5 hours: Timing is on. count ---------- 21568508 (1 row) Time: 737626.512 ms test$ uname -a FreeBSD xyz.com 10.3-STABLE FreeBSD 10.3-STABLE #0 r307264M: Mon Jun 12 17:= 48:24 MDT 2017 aaronc@WFT:XYZ amd64 test$ uptime 1:06AM up 5:16, 1 user, load averages: 2.21, 2.59, 2.04 Perhaps if I was on stable/11 this would be of some benefit. Regards, A -----Original Message----- From: Caza, Aaron Sent: Sunday, June 11, 2017 10:33 AM To: 'Slawa Olhovchenkov' Cc: freebsd-hackers@freebsd.org Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD perfo= rmance drop < 24 hours Thanks for the response. I happened across r307264; however, my understand= ing is that it applies to the FreeBSD 11 and I'm using FreeBSD 10. Note that I have experienced the same performance degradation on a FreeBSD1= 1.0-RELEASE-p1 amd64 system using 2x Intel 520 SSDs in ZFS mirrored pair wi= th server an Intel Core i5-2500 with 16GB ram and running PostgreSQL 9.6.1.= There's no memory pressure in any of the systems I've tested on. -----Original Message----- From: Slawa Olhovchenkov [mailto:slw@zxy.spb.ru] Sent: Saturday, June 10, 2017 10:37 AM To: Caza, Aaron Cc: freebsd-hackers@freebsd.org Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performan= ce drop < 24 hours On Sat, Jun 10, 2017 at 04:25:59PM +0000, Caza, Aaron wrote: > Gents, > > I'm experiencing an issue where iterating over a PostgreSQL table of ~21.= 5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds on I= ntel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back from Ja= n 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm using Po= stgreSQL 9.5.7. > > I've tried: > > * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 2017. > > * Tested on half a dozen machines with different models of SSDs: > > o Intel 510s (120GB) in ZFS mirrored pair > > o Intel 520s (120GB) in ZFS mirrored pair > > o Intel 540s (120GB) in ZFS mirrored pair > > o Samsung 850 Pros (256GB) in ZFS mirrored pair > > * Using bonnie++ to remove Postgres from the equation and performan= ce does indeed drop. > > * Rebooting server and immediately re-running test and performance = is back to original. > > * Tried using Karl Denninger's patch from PR187594 (which took some= work to find a kernel that the FreeBSD10 patch would both apply and compil= e cleanly against). > > * Tried disabling ZFS lz4 compression. > > * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL 9= .1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime an= d test took ~80 seconds after which I rebooted and re-ran test and was stil= l at ~80 seconds (older processor and memory in this system). > > I realize that there's a whole lot of info I'm not including (dmesg, zfs-= stats -a, gstat, et cetera): I'm hoping some enlightened individual will be= able to point me to a solution with only the above to go on. Just a random guess: can you try r307264 (I am mean regression in r307266)? This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Tue Jun 13 12:47:19 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36912BF3710 for ; Tue, 13 Jun 2017 12:47:19 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EBD367A926 for ; Tue, 13 Jun 2017 12:47:18 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1dKlDw-0004Wd-Bz; Tue, 13 Jun 2017 15:47:08 +0300 Date: Tue, 13 Jun 2017 15:47:08 +0300 From: Slawa Olhovchenkov To: "Caza, Aaron" Cc: "freebsd-hackers@freebsd.org" Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Message-ID: <20170613124708.GB18123@zxy.spb.ru> References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> <20170610163642.GA18123@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 12:47:19 -0000 On Tue, Jun 13, 2017 at 06:24:34AM +0000, Caza, Aaron wrote: > Slawa - thanks again for your suggestion but, unfortunately, same behavior: np, thanks for check! > (Server is Intel Core i5-2500K w 16GB ram and 2x Intel 520 120GB SSDs): > > PostgreSQL test results immediately after reboot: > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 53855.000 ms > test$ uptime > 7:51PM up 2 mins, 1 user, load averages: 2.46, 0.88, 0.34 > test$ uname -a > FreeBSD xyz.com 10.3-STABLE FreeBSD 10.3-STABLE #0 r307264M: Mon Jun 12 17:48:24 MDT 2017 aaronc@WFT:XYZ amd64 > > PostgreSQL test results after ~5 hours: > Timing is on. > count > ---------- > 21568508 > (1 row) > > Time: 737626.512 ms > test$ uname -a > FreeBSD xyz.com 10.3-STABLE FreeBSD 10.3-STABLE #0 r307264M: Mon Jun 12 17:48:24 MDT 2017 aaronc@WFT:XYZ amd64 > test$ uptime > 1:06AM up 5:16, 1 user, load averages: 2.21, 2.59, 2.04 > > Perhaps if I was on stable/11 this would be of some benefit. > Thanks for the response. I happened across r307264; however, my understanding is that it applies to the FreeBSD 11 and I'm using FreeBSD 10. This commit (ZFS ARC comression support) was MFC on stable/11 and stable/10, so you don't need check stable/11. From owner-freebsd-hackers@freebsd.org Tue Jun 13 13:37:14 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A9F6BF4B29 for ; Tue, 13 Jun 2017 13:37:14 +0000 (UTC) (envelope-from trafdev@mail.ru) Received: from fallback.mail.ru (fallback12.m.smailru.net [94.100.179.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A38B37C2E7 for ; Tue, 13 Jun 2017 13:37:12 +0000 (UTC) (envelope-from trafdev@mail.ru) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject; bh=/ZF66YNLFNMc9f/t6ni4mDIRtexLtAUiQPaSiSSwM9k=; b=iaENALc4r+h99lPPKNP+fDRMcLYvWb0AzFxMi9ctOjlBv0Z19F6nwmwoXURwa12FFizlNwTchpYZISmsbZy4geRJY9N+f4S6R3gO/C2tDUt6H4+j6QqHxzDUiaI7J2/YNbBknjzWfo6pMrTTTOVJGpGSHW9RA3z9eBZMPfHxj64=; Received: from [10.161.22.24] (port=59686 helo=smtp54.i.mail.ru) by fallback12.m.smailru.net with esmtp (envelope-from ) id 1dKm0K-0003cu-AC for freebsd-hackers@freebsd.org; Tue, 13 Jun 2017 16:37:08 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject; bh=/ZF66YNLFNMc9f/t6ni4mDIRtexLtAUiQPaSiSSwM9k=; b=iaENALc4r+h99lPPKNP+fDRMcLYvWb0AzFxMi9ctOjlBv0Z19F6nwmwoXURwa12FFizlNwTchpYZISmsbZy4geRJY9N+f4S6R3gO/C2tDUt6H4+j6QqHxzDUiaI7J2/YNbBknjzWfo6pMrTTTOVJGpGSHW9RA3z9eBZMPfHxj64=; Received: from c-50-168-21-188.hsd1.ca.comcast.net ([50.168.21.188]:10352 helo=[192.168.1.116]) by smtp54.i.mail.ru with esmtpa (envelope-from ) id 1dKm0B-0006LC-7N for freebsd-hackers@freebsd.org; Tue, 13 Jun 2017 16:36:59 +0300 Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: freebsd-hackers@freebsd.org References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> From: trafdev Message-ID: Date: Tue, 13 Jun 2017 06:36:53 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> Content-Language: en-US X-7FA49CB5: 0D63561A33F958A5C1514F08AEFCFB8DB56E0D7C1CB992536A0769601CF48B36725E5C173C3A84C32F3DB049E8B62580B05529FD6524BE1057739F23D657EF2BB5C8C57E37DE458B4C7702A67D5C3316FA3894348FB808DBEB6346B700B4D54FE5BFE6E7EFDEDCD789D4C264860C145E X-Mailru-Sender: 48346E3B4B381038050A507F55EACB66BF5CF714AC9CA338153D1631923CDB726C816BAD93A5906EADF500B581EA1E492D063C67CFD4E849954DEF1BE664373F92CFA38D0130A48967EA787935ED9F1B X-Mras: OK X-7FA49CB5: 0D63561A33F958A54E59700E7CA88202274F91C3957E8F6A4F42B403FB85EA03462275124DF8B9C99B0B8D173C204012BD9CCCA9EDD067B1EDA766A37F9254B7 X-Mailru-Sender: A5480F10D64C90054EBC784E05B22290D6BB3225CB142CB91A6B38F4995D801CC59A7EFB72EB47B2601D6E3165EAD397C77752E0C033A69E7A8C3768F8E4DEB7AD818CEE1F0DEEE73453F38A29522196 X-Mras: OK Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 13:37:14 -0000 > Tested on half a dozen machines with different models of SSDs Do they all share same MB models? I have a similar setup (OVH Enterprise SP-128-S dedicated server with=20 128GB RAM, 480GB SSD in ZFS mirror and an original manually installed=20 FreeBSD 10.3 image): robert@sqldb:~ % uname -a FreeBSD xxx.xxx.xxx 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug=20 11 18:38:15 UTC 2016=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 robert@sqldb:~ % uptime 6:27AM up 95 days, 9:41, 1 user, load averages: 3.29, 4.26, 5.28 zfs partition created with: zfs create -o recordsize=3D128k -o primarycache=3Dall zroot/ara/sqldb/pgs= ql custom param in sysctl.conf: vfs.zfs.metaslab.lba_weighting_enabled=3D0 robert@sqldb:~ % sudo dd if=3D/dev/urandom of=3D/ara/sqldb/pgsql/test.bin= =20 bs=3D1M count=3D16000 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 283.185773 secs (59244558 bytes/sec) robert@sqldb:~ % dd if=3D/ara/sqldb/pgsql/test.bin of=3D/dev/null bs=3D1m= 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 33.517116 secs (500556670 bytes/sec) robert@sqldb:~ % sudo diskinfo -c -t -v ada0 ada0 512 # sectorsize 480103981056 # mediasize in bytes (447G) 937703088 # mediasize in sectors 4096 # stripesize 0 # stripeoffset 930261 # Cylinders according to firmware. 16 # Heads according to firmware. 63 # Sectors according to firmware. PHWA629405UP480FGN # Disk ident. I/O command overhead: time to read 10MB block 0.285341 sec =3D 0.014 msec/secto= r time to read 20480 sectors 2.641372 sec =3D 0.129 msec/secto= r calculated command overhead =3D 0.115 msec/sector Seek times: Full stroke: 250 iter in 0.016943 sec =3D 0.068 msec Half stroke: 250 iter in 0.016189 sec =3D 0.065 msec Quarter stroke: 500 iter in 0.022226 sec =3D 0.044 msec Short forward: 400 iter in 0.018208 sec =3D 0.046 msec Short backward: 400 iter in 0.019637 sec =3D 0.049 msec Seq outer: 2048 iter in 0.066197 sec =3D 0.032 msec Seq inner: 2048 iter in 0.054291 sec =3D 0.027 msec Transfer rates: outside: 102400 kbytes in 0.671285 sec =3D 152543 kbytes/s= ec middle: 102400 kbytes in 0.640391 sec =3D 159902 kbytes/s= ec inside: 102400 kbytes in 0.328650 sec =3D 311578 kbytes/s= ec On 06/10/17 09:25, Caza, Aaron wrote: > Gents, > > I'm experiencing an issue where iterating over a PostgreSQL table of ~2= 1.5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds = on Intel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back f= rom Jan 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm = using PostgreSQL 9.5.7. > > I've tried: > > * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 201= 7. > > * Tested on half a dozen machines with different models of SSDs: > > o Intel 510s (120GB) in ZFS mirrored pair > > o Intel 520s (120GB) in ZFS mirrored pair > > o Intel 540s (120GB) in ZFS mirrored pair > > o Samsung 850 Pros (256GB) in ZFS mirrored pair > > * Using bonnie++ to remove Postgres from the equation and perform= ance does indeed drop. > > * Rebooting server and immediately re-running test and performanc= e is back to original. > > * Tried using Karl Denninger's patch from PR187594 (which took so= me work to find a kernel that the FreeBSD10 patch would both apply and co= mpile cleanly against). > > * Tried disabling ZFS lz4 compression. > > * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL= 9.1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptim= e and test took ~80 seconds after which I rebooted and re-ran test and wa= s still at ~80 seconds (older processor and memory in this system). > > I realize that there's a whole lot of info I'm not including (dmesg, zf= s-stats -a, gstat, et cetera): I'm hoping some enlightened individual wil= l be able to point me to a solution with only the above to go on. > > Cheers, > Aaron > This message may contain confidential and privileged information. If it= has been sent to you in error, please reply to advise the sender of the = error and then immediately delete it. If you are not the intended recipie= nt, do not read, copy, disclose or otherwise use this message. The sender= disclaims any liability for such unauthorized use. PLEASE NOTE that all = incoming e-mails sent to Weatherford e-mail accounts will be archived and= may be scanned by us and/or by external service providers to detect and = prevent threats to our systems, investigate illegal or inappropriate beha= vior, and/or eliminate unsolicited promotional e-mails (spam). This proce= ss could result in deletion of a legitimate e-mail before it is read by i= ts intended recipient at our organization. Moreover, based on the scannin= g results, the full text of e-mails and attachments may be made available= to Weatherford security and other personnel for review and appropriate a= ction. If you have any concerns about this process, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" > From owner-freebsd-hackers@freebsd.org Tue Jun 13 18:36:52 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B2FBBFB89D for ; Tue, 13 Jun 2017 18:36:52 +0000 (UTC) (envelope-from emorrasg@yahoo.es) Received: from nm36-vm6.bullet.mail.ir2.yahoo.com (nm36-vm6.bullet.mail.ir2.yahoo.com [212.82.97.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C8B62200 for ; Tue, 13 Jun 2017 18:36:51 +0000 (UTC) (envelope-from emorrasg@yahoo.es) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s2048; t=1497378810; bh=MyEif2JyOucv24Uel358K1ZYRvEnFn4un6nrXgQaTKI=; h=Date:From:To:Subject:In-Reply-To:References:From:Subject; b=EJokprq9ANV30eLdaXg/iACohdPnVR6QrN+UBkaDrrS0Poh/BXO99/qispCigJfHvly6n/d4oa7+buFZ7wuReylVzY9HXvK+WO/iCNW2galmrXFC8ADYb04pU+mmbNb73/apv0zFI4RCbUHNQEzGiFwhNRoYYHzaT+vAsQNPvXDyUsmH5aBXT65sh50f5G9VArTcbXN3FN/fBLJWL/68UfnZGKyO0XJoVutT41Rz0O9Ax93VfoCGXgNF3/iPcyW5zJajh/HIdPnDPXWMDHGq+MycmWK0B0c+9X/nHsjuw0GyN7O9rqTc6SeIz6YMmNA15OLxJnW231xIFaGvESPmpg== Received: from [212.82.98.57] by nm36.bullet.mail.ir2.yahoo.com with NNFMP; 13 Jun 2017 18:33:30 -0000 Received: from [46.228.39.91] by tm10.bullet.mail.ir2.yahoo.com with NNFMP; 13 Jun 2017 18:33:30 -0000 Received: from [127.0.0.1] by smtp128.mail.ir2.yahoo.com with NNFMP; 13 Jun 2017 18:33:30 -0000 X-Yahoo-Newman-Id: 600039.53384.bm@smtp128.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 23DDvNwVM1m9wiq.yajxkov15nX1XvHYSjHR77Bk8HIP2fC N4dvq7S8EzRcdAl1xvwO.JgR7RxQXm.Bk9JywIgRxkkWSelSMxE47v2rfcW. Yv8itzZjpHU3zNUWc2ckS8U.35thXEzDu4ueupoLRIP0Gp5tf4_MCJ3beUKe Fx5Xp9yPxN8xP0..7GPL1nCrOt91dv57egU5lg1Hihx...CLbxEPALilKIsc 1hCPAV2aaAXZFVQSKKaUcBWXdTvS9WAcA_RRfIZ93ceksIUiE9OYpVXFr193 YyMBn9alTKtzyub8NaZpr_rQxlyIuCGIoHU5Vzl_kb1V.iJMVsOWqoqTPVt8 2E_V1KAeGA3fdAE7ADytsyz203O7NAvDJ27MxXSEdM0aM5vWxFJ.KQ1q8siU tJJRgtBMwPLYrbEihOa0tXu44dJaYhhFz7APtoiDOyzktcNyGwNTrskTHJoA So48W1xIHeYlYAPcwNEweHBWxFszrwlHnqJozRMSk4GWvEpNWyT7vPCTGaaF mKSlyIt8eu7w_vd9NNEWsGLDclYQoDY9V1xMrOmEjnEUrfVVMFXIKFhYRyJ3 MnM8B X-Yahoo-SMTP: mX392iiswBAeJNdO_s.EW62LZDJR Date: Tue, 13 Jun 2017 20:36:15 +0200 From: Eduardo Morras To: freebsd-hackers@freebsd.org Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Message-Id: <20170613203615.ae6df0a1c94bffe1dc6745fc@yahoo.es> In-Reply-To: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> References: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 18:36:52 -0000 On Mon, 12 Jun 2017 04:50:44 +0000 "Caza, Aaron" wrote: > Thanks, Jov, for you suggestions. Per your e-mail I added ?explain > analyze? to the script: Can you check that autovacuum is not involved in a lock storm? Autovacuums workers start to work but a higher level lock is acquired by other process/transaction stopping autovacuum workers continuously, making autovacuum useless (but trying continually to clean).=20 Also, you're comparing Postgresql 9.1 with 9.5, are you using the same conf file? Did you upgrade it (conf file) to 9.5? --- --- Eduardo Morras From owner-freebsd-hackers@freebsd.org Tue Jun 13 18:48:00 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 316F7BFBDFB for ; Tue, 13 Jun 2017 18:48:00 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0065.outbound.protection.outlook.com [104.47.32.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACBD92A7F for ; Tue, 13 Jun 2017 18:47:59 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=EM8YnyFUbJ2b54sEbid6XmF4zyEyHeqgum6myc11b7g=; b=QASJrYaFcxGtd2AhBf0qAefXmZXUz0iCuEQkN4Sjo1NJ4croNqX1DaPucmP2CpP4zipnOJHaMskyXxBLEq//F5XnM4JaSSoEt1z2Xnmd2KkoMF2ctesa33dTjN2iVw0CihdhsBnzhlXvx0v7AavPVtuJ2x/HHJVu2CIXPlAlkso= Received: from BN6PR03CA0062.namprd03.prod.outlook.com (10.173.137.24) by BN3PR0301MB1234.namprd03.prod.outlook.com (10.161.207.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 18:47:57 +0000 Received: from BN1BFFO11FD042.protection.gbl (2a01:111:f400:7c10::1:136) by BN6PR03CA0062.outlook.office365.com (2603:10b6:404:4c::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Tue, 13 Jun 2017 18:47:57 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BN1BFFO11FD042.mail.protection.outlook.com (10.58.144.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Tue, 13 Jun 2017 18:47:57 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB015.032d.mgd.msft.net (141.251.110.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 18:47:56 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Tue, 13 Jun 2017 18:47:56 +0000 From: "Caza, Aaron" To: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLiAalX9ncyh3crQla1B5/716RidACSHS2AAAqINMA= Date: Tue, 13 Jun 2017 18:47:56 +0000 Message-ID: References: <79528bf7a85a47079756dc508130360b@DM2PR58MB013.032d.mgd.msft.net> In-Reply-To: Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.205.133] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB015:|BN3PR0301MB1234: X-MS-Office365-Filtering-Correlation-Id: 244e9ea7-986d-4828-ccf3-08d4b28cb55e Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB015.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39850400002)(39450400003)(39840400002)(39860400002)(39410400002)(39400400002)(2980300002)(438002)(189002)(377454003)(60444003)(24454002)(13464003)(199003)(86362001)(24736003)(2900100001)(38730400002)(9686003)(66066001)(55016002)(110136004)(6246003)(6306002)(7736002)(356003)(305945005)(53546009)(47776003)(5640700003)(108616004)(53936002)(68736007)(97756001)(46406003)(2501003)(50466002)(2351001)(5890100001)(2906002)(6116002)(102836003)(3846002)(23726003)(50986999)(189998001)(33646002)(97736004)(5660300001)(8936002)(69596002)(106466001)(81156014)(7696004)(72206003)(6916009)(229853002)(966005)(22756006)(76176999)(8676002)(478600001)(54356999)(8746002)(42882006)(81166006)(86146001)(2950100002)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0301MB1234; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD042; 1:VzsEmOBtRy9s8qJwoR9jhGvvPxFBoGJFsyw1rLoO2kHV9PrYCfRrxICdc+/gRMQrFV0cVWRYadO0maS1BY1mntIs4qkOE2OeDTOVUvw1MfpI3n9LUxn6DNApNkqWjAlo+dns6H+mes2002Q9djCEtG/hKuQ40H7DeAKmMxOiT6Z+21z+s/Ls/fKVXhj4GZMLcJjyjjQR5Gh+QoYLsJLIjKiePTTkNtV/teVojaCRjjL0U07gCYHZvXvrxCviKDW+eCE+0vz9aUcxQNfClBHoXvKbSKRdBmlo9y0KjFKNn+bk0RtSud8R3JUKTgwGW4qEWCNMHeY2dGIZzLVcQa18QB1LQMpZTgE3F5mai3UaWR61iVf0VV5EElJEkxDHrSGNFn9LSwTM65COi2rRTRx8fr7VndYJKVf7/90X9SVFPoGqMlH7MiMgCeYv4moJetkK0p6Ul7B5XtwkN7m1z5e52bRjyJwwnmLsj6WZE9FbtLABDQxvS9RY/1+eqab7/yd7sUvrs2lew79N5kKM5bgiGNJfy6aC1aLIOG3OSMAQqUo= X-CrossPremisesHeadersPromoted: BN1BFFO11FD042.protection.gbl X-CrossPremisesHeadersFiltered: BN1BFFO11FD042.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:BN3PR0301MB1234; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 3:V7DfPlW8Tli/22GYlpJXT7o8KXHT2lKbXPiJpuDh83fMNmmn3f9H/hiYNp0L5CRephCut+iVoZTXl4Y8KRR1HMgDPldV+9AxU2wTfXtSU4T/m8u0k/I5ssgK7LdvheKCxXA8HlmtmhdajroTMz/Qh2jVxWYXesSBwff5+nkY8ke6TWaBB77iU/hF9/7cdEsH2Om/EpVJ83mpqcUaoDRSfmbDmcmpJC+z5bGBfm3Q6Tmf3dwObiq7ihGD+rq9Tuk3TMzeYseWMU1CSJu7hYJnLJP7hvqEiws6cR/Kl6S6h/aSm1Mgxh79BN/LoSRBWvvZusV99TouU6JJ1GsAws/ZTDpwHgjgXbTCfd+CBptBj0EgnMz1RP15cm/rlR9F6f1/RI8pg1YCJJbJxm0qtnrlEAnSei9L+RHqi3s860vee3pR6FtcPFHUpzkl69Mmmzcv8gPXNqlr5JouejnH6C+/maIpJeR8dPbs/AV9wJ7gmNo= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 25:dt+JUOvpXw+7k74qsv8wD3EmXAS1uHgvnLVc4rJe21nL5dAxmQF1/ThAa2T4gA2LfKSF8soGR2dfee2XqKchwlIZOLPUNtIulQjh0gmNZJf72m913X4hzyFfkxp4Ok/bsk654S6aeetce4p3uIfeos91XidfTfhtRrFzXsoruR11tW5/Nh9wUb3rC21SKMlcv/QH1EQ0yQFm3iL5rayjTm5ePWhHqN7tQRvBNb9HQvMf90SSq3A5E+1+Wc21zCPn1+U9xcPpn9mpiuac8I/i2shQgZxqmVXmQDeh7DzUXmPcJ/ii42zE+R6ultX8yveGggv+plo8LWTbgwBx33D+dVBf7Qh5TxAvRQIPHwBDLQPw+sVMp8aH0cYZOwQizadM6aQnRLBo9uuf8R+oiafybIJpyCV7MxjlqufRH3eQf/LVY6dEa9gc1jWSfe9tM12WI1z8249OjZ5C5L6P4lIw1uht0J3jMo0oGfBlY4EhOFk=; 31:15itu96UAHUcvyZ1hCmMeOftGavW/fu17cRfuOb+CPM7/tZVG+0p4ht3B9JtlgniUO9KY8zZewcqD0wBCvt2WKO+DiAoI8lPKc9xZFWZ/4OaHTNRAWS2uylspk7vPv0blgJPtmcV2AVnkCmd2GABpQguDYtguaetfPujESnUb2MzDK89Im60JLMT8CVJneZMK17ty/pphnn9LbcC06/dar6KFw3KSuwTRLlVSkK4oEkAjBpck/QNNv2eK+36lDUOfGHYkl2txevIhyqX2MIlTFzJCoMQgYZZPlqQRm+AKUk= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 20:IO2tJa4FwFZtOgPXuBQBQ4kSSzYWY+hBzmIUmRMAOyXghWAOHb7vwZd7l4xkQfZ+8gERqlLkZXp/cHu4FqhX6qfOvUYfRv31K4rxZnnWRnQmGum3eFot/aH6oVJ+81cIhRQTNVdrimNjBNlrSyfqy/ZfZlJp4WBGL+OzB5kWCqg2XNP+nTjZfoLTsOrgi4QaOKnWR9uGy80p9gi8ehNzLDAMWCh8zeCjTUDHt4Ncmtol/guyUI2xoj3gABt5/zaFzcE2i8fBw0jGGp369nU7bR+1W7E9NTJSSnPLC15fjeE5wjal0Om6jZEXlgLPFq6wnWppEWh2luknFvgvmKtVEgVbW5LfXlfsQVwv0QLHoZprXqUBxY21VM3VrY+pA/X8PqE4Zssd91ssRmt/BalTVWozVcV7mb/Ireha6Yv5goGuMXs3NTRGBEnkF1XZ0BIgPzg0o7IrGAliwNFJwaEBYLkyydVJT8RDZcG6FdSfxK9NQLy4ohAJdQy63sZjXu/c X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705)(75325880899374)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93004095)(6055026)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN3PR0301MB1234; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN3PR0301MB1234; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0301MB1234; 4:BMBWy91SsZMrTdL3e/54920gLu9fcuXUWAQ1RR2P?= =?us-ascii?Q?1yTwyGci5kbF50O0CmUOL4yqDkmYvOpbb8JeFEQobhQXIUEFpJh90lxHaxoV?= =?us-ascii?Q?hPiggV93EUgnYJD7auXoxLkK/JD25yj2oWCmisR309Qs0++pFFqzpPwipwLv?= =?us-ascii?Q?pNQIyW8pZdS9H8FzbdeoXlg7jxPcHU88HJ7tkoMYlcLV8Icc0GoSK9AVo6C3?= =?us-ascii?Q?1Ymyywd8UbH5zHG5KDb/2HAEfs3W4Xry62iCA3h3p3S8KnlIHu98zhr3oQQX?= =?us-ascii?Q?O7wByOJsNKCaIxKe5Of0qu3v5P3jXooe8KBukZs4cxncxjkw1dFjM0RSDU7A?= =?us-ascii?Q?G0uYpwNrTRljYfZDMxQkXGJktyC+WCZUXAOkofU77K7u0/+cu06jWpxLC9/L?= =?us-ascii?Q?DmhO5Y+7xLtQfH7vNdouantz2yNwBQf4ZFzm+1Ie9bVDX3Xw6xYMi6GFf8iD?= =?us-ascii?Q?H436Fm2wy2s6vITVWbut+byL5vxFQQRKS6itzmiRnVAxb+9DkvAYGJPc1K+Z?= =?us-ascii?Q?1oVTcTQgOMdi7Lznbz+R4r3aA7EgIneNmvAzeSy+IL9J7U5lLLc8XfqZz2Ms?= =?us-ascii?Q?LDRI4c+Dxy19BaA/ui2uvEhF9fUWN3d1gQe+3gVSpf0bsSnAmDSA6inGRJ8e?= =?us-ascii?Q?o/w1fsSgympbvSMam7TvNRa/vb3x4uFEn3hRw/a14TMFRF96Af9tum0RJtyy?= =?us-ascii?Q?FWrndBPYVxTRKbjSHlCPTSlKk9w6TPck4zo2QzXl2Ym12ANZrtmW4TPFmieI?= =?us-ascii?Q?HkhOF26l8sifyEqyfDYuwBpFcGXKyyS4mYBLIwzvHUx/DzJ12TgfE8dA+Vd1?= =?us-ascii?Q?5LV8zLVt0IEemNnzBPsfS3mlKiPNrDNcLRe36ABhJI3nxkOVww9LYcBE/5B0?= =?us-ascii?Q?DtVNb1alGhjDfxA2wroEQ0ogKZKJ7HmzNRr6OPkPKLzfDQayLVqBVvdAAXEM?= =?us-ascii?Q?RvnAzsMGib01OCkMypJJwYjF++Xluqj+up/HOyjim4gbL3hp2UTrcpMDDHzS?= =?us-ascii?Q?UHHoKgWkjpC1u/rhvd7rGKe1D573xJy4fTYTJ0wEaXM7I09H4gV0ky7WYbbd?= =?us-ascii?Q?scbqgcvEMurtSnr4PHSj8jIKh2YoTuaeFQ8Fk2SgrT+klpjPfLe1WpSMgbMb?= =?us-ascii?Q?U2+oAMKxE1yc1MY1i0xDaaQDD/msHo0uSXh0d669sNJKyCWLKoSNum0klVOL?= =?us-ascii?Q?Uxqkaz1ssOGkThPafYejYZuH2HQrGCT4itM+0kVWOf5/j5zf9XtEH+wfkXsM?= =?us-ascii?Q?Ptam2rTpOlm5y50lQ49rEb8sBNSZmCgHr4GwJXnNjpiMno5imjy47LAwiws2?= =?us-ascii?Q?9LhaMG6Xt9hLWq41k/B2yqyQrCjaypYBB1keIHOdnh6HLV5Cmd90WGLAOkm5?= =?us-ascii?Q?ozgI0wFotuiWylglNOlH9TIi24A=3D?= X-Forefront-PRVS: 0337AFFE9A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0301MB1234; 23:/HIGmTXugiLNbc1m42bHb86Cy6xQgfxLp4Ng54j?= =?us-ascii?Q?qDIGRYzmR6PgzYM/8KzCNeyIlD2c+s0VpxxxWVrU+G7u6dOh1ecQeadOFC3R?= =?us-ascii?Q?rAbyxwr+dnW/+HwU59eheBOvJf0xq0ZKfJDNrllwtgY9Q9aO0ZTZ3rt1E+NK?= =?us-ascii?Q?/oGvYEO2Em6uv2ILF1nyrZEs2AsxRnvxGkOc+rrfzLkFuu+wJyoPzKoRewiI?= =?us-ascii?Q?I7o0775hnolhKzSzC17J2zpBc6Or3IOnmobjXb4cCPJqSRoa7WradWfZojD8?= =?us-ascii?Q?J37hQHGMGTGZeClzgVcAVPIKDEYAywBcU/GIKqympGLEerVSfdhO+Qqb9/gJ?= =?us-ascii?Q?sAi74xvsaSYf410/WMebDzPP493knFbpjaMviaJWl4NdcFokj21nwJvxCRdV?= =?us-ascii?Q?QlYqYX9PBF20bmJzMdnpKr3meSO6cAqj9C/oCVwt3ixU5ULgXn4yGNLznVUM?= =?us-ascii?Q?SjCkrt/b0MCjY3TPE4jjrG/OlCY2rfBxmXmrKqkw2uE1rIMnzWDqhaN1BhCu?= =?us-ascii?Q?S4qL3QeDe2nSYKG9VBPzw+W2Vaz/0gH0vOcrx5haoH58tSuro88KiCDSTYAq?= =?us-ascii?Q?dLFKJ3qqQof7DNS7cZvE54YXA1JmmIma52FdC6hJMMHEaS+Is8eEFPdCJqXO?= =?us-ascii?Q?cOooSPXvUjMsdLbrIEAlBfTxQKeo9XXrxnqmMQ7DzyGUolK8ig3tFPrFWyjl?= =?us-ascii?Q?cJirlk2C9rhI60dHo0waBWps7UzIWSiDEYy4w06fmmgbeo3WVa5sYaZQ+MkF?= =?us-ascii?Q?mRNNqGmY3rRceFG9UziLW9Kn+i6eC38ZfT1ev3RXdwF+zLyxQtreugtdvt6h?= =?us-ascii?Q?Qi9HHblVK0LCpXpBmZz/snyUpWHIY+/x8W4C+2w7p1pegLoLTPsqvJ+6yuac?= =?us-ascii?Q?a6aLfHR9wSgovwOrg5Z0k88nhPHIQeXgZkfYsDQscKtDVNgt/8u7xavhJEBW?= =?us-ascii?Q?LOqyDcVfBw1rYfyHQsCC0x2AYpbr5g+DeK/EuOJ76NIdCCTO3TfY0vYCiwUm?= =?us-ascii?Q?vIiswXRh9bP71FNn9e2dUi+vSh9PlJu9BMDuPwK5dybJ4L4fGqSlq9UvTEq/?= =?us-ascii?Q?2VJ1AWCG/4sjBAdUlBSfeRDBtYSR9nr48FEj1S7WLgqcFKP8f/pvqHDLDnSa?= =?us-ascii?Q?COEN/gjIR4jJV0lBwQPCG1Ey3gHWNHgh2KYPDX1/iIQ6AWIbeeGR4W/ZapJh?= =?us-ascii?Q?RfQqZTo16fNnC3zGiQbdZHU2sAKSrKLZDmrDFM4UjV2wNM60aoo0qnmBavtZ?= =?us-ascii?Q?E0D6BW/uNGj2usYPmWbP1K/JBolN2zuIPlDlaQvd4t7CeUVtv5OvpxcGC2vF?= =?us-ascii?Q?SVdaOIho8Jgd66vZGYxELt556zAzwLDfbuX569vpV+rH8jsrfxBLBP/gtXp3?= =?us-ascii?Q?kCuKiYoxbO2LwLO3q3o1wcm7YYTM629HHwrtbpCh0up0xc8JBt5WFqtFZtJV?= =?us-ascii?Q?OIgH9gxJkuJfXLatEh2I/V/4ckudE51wxP5xed8My03Ah3RbKKmHRXvTyXyD?= =?us-ascii?Q?6GG8EcFbVagQFYC1x4j6T7k0suPFGHppJVAIUSXhQK7gbSnkmimFytYC3BAB?= =?us-ascii?Q?cHuNniTYLEUPXdFoVI5UaTM9A8OAY3pFi5eUEvzG0u2aF/bRyAyuv4JfM59D?= =?us-ascii?Q?tJccyQ1wLUBeX8wO/AnRQ6Itq2mdK6i92rdlc2lyaXC8=3D?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 6:7/H4Pu/6tqkPpoabhKW0UhB1H2AbqJvo6TXj02S5VAdX95dXRxjoNt3iZyz07B15TJBqzRIFWItkKyJUQTsTl0jWMn1p6BhOehuys/E4TZrAgdPf7xQzxJoLnfPQjW3/ifoRFy5EvfLtb0Aq2YSK1xvoiyFtvtuPBXcNehQn0vthi12OK8SFXrW/hUzkBdBZYkoTCRD/o+XXk8EpE5KAayhw7qWiP+PbGl4bSHpGYD0OpaObupAYeD8y+1d5CfMWJSTcfbPOSI8/gGMyXd00gkFyWJL+msgNsADM0vWcL79u1Eg9UiIyOABbVqj72tVhcMSxmz3RWz0n3bUKnYYS/x94Q/DMBp7StF5sPx96pOJxcudliaiad++ndoJmF+ynaLTskoAzOPba0+2DOkYF/GYetsxD8pLzUWPAPLbDuA3mzpLmM2RUOguX1g89Z0d91ej2UFVE3k4HFzuatoIL6cCzbLCjSn01+1cVhawi/D35ppKmrCK5iCOu7k/GehPCXlRfgPFdemzjQPOd5AejiFRreiM87qzniYNVcitEbo8= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 5:A/WS80EuhEJhouQaj5k5kdNTeBlhgfhmvLybXBgpK8oWpe+iPS3hDgu8TBB9wk1Vwc1C6z2Mejf/mLu5ZXbvmGGmlrpfWmJc40gVhEZSJtxQEEErdloBiqe7vfmk3YX7iXiZ4RIVvIjN0RfhWID3g4HzWga/XWBeQ1qt6s/Nj/zls0iL7+TPdWQnnKcgD0mssf+zPs7XNJzK85Y+gU9YfiqYs03Gf+mp82ED8H6qfutXA2Ki+jB/0CJ0ti+zoRZyy3X2tHxKCSl4CPQGzVRgqPDwOR06TSACuG1ROT9BCLJ2wDYZe13Od2a5v/wf8Ib6/yvpWTclj3qspd21y3lXVhg9qGqJjMVO9U7YXmmpuqC9rtHcd06VwM5/DNtwfLYfPUv7y4tjf46pJV5ILCqdvlTb+cEJLRzxzmZOODk1RW/mFgEmoFkdBabwBYddCQ7f0zq2YLu0JIRPo4G7XSCDHzKmA1kIYVS9myb3kJ/cfn8ov07CFBmWVeoBRNnvb9rv; 24:0sk7aHoR5ghBj2XS0dPWIQBtFDsTS6jLpE4LqmEcjwDqEp4AffRk1KtJXL0S5i0O4AmJNOLu+DV1wHMbkSr4d5gM3lBjJ7AfT8bDS1PMULc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN3PR0301MB1234; 7:chHmQT9F9c4DhLFJH6WaMkDE81UUqxYfW3PNHfcPRFR4Efxu6+ddNKOHhVVLfTTuA8+PVlZojdZVxArcU8gAUoBn1rTUfOTKXUzTSx75ev7QKhwiP1JSxKCYC3VtGLvF4fT5nUbArIiGe+h4V3vZGL6rgnMdlhZKkcoC6lFdMCKmIWKdpNWe2E1LBorheSOmpwc1++XzMNhf0J8r4d/1vnJ+PUeA8P0+YqtnfadOLmzu3SgMfRw+Sl+VEuZS0HKSFkeEdTZPWwhL/oCZGw2o7GKECMztlK+3kYF1PvTQ/d9MYLpXHtIe0vjzSLzSsLqh5R1h/gTJJ6NGrueHSFA73w== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2017 18:47:57.3357 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0301MB1234 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 18:48:00 -0000 In response to the below, all the MB models I'm currently testing on are di= fferent. Thanks for sharing your results - you're running the FreeBSD10.3-RELEASE-p7= kernel back from Aug 2016. All the 10.3 kernels I've tested with so far a= re stable/10 from Jan 2017 or later so I might have to try going back earli= er. Currently trying a checkout of base/releng/10.3. Unfortunately, it so= metimes takes a while before the degradation hits which slows down testing.= Also testing a stable/11 r307264 per prior suggestion from Slawa. -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freeb= sd.org] On Behalf Of trafdev via freebsd-hackers Sent: Tuesday, June 13, 2017 7:37 AM To: freebsd-hackers@freebsd.org Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performan= ce drop < 24 hours > Tested on half a dozen machines with different models of SSDs Do they all share same MB models? I have a similar setup (OVH Enterprise SP-128-S dedicated server with 128GB= RAM, 480GB SSD in ZFS mirror and an original manually installed FreeBSD 10= .3 image): robert@sqldb:~ % uname -a FreeBSD xxx.xxx.xxx 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 robert@sqldb:~ % uptime 6:27AM up 95 days, 9:41, 1 user, load averages: 3.29, 4.26, 5.28 zfs partition created with: zfs create -o recordsize=3D128k -o primarycache=3Dall zroot/ara/sqldb/pgsql custom param in sysctl.conf: vfs.zfs.metaslab.lba_weighting_enabled=3D0 robert@sqldb:~ % sudo dd if=3D/dev/urandom of=3D/ara/sqldb/pgsql/test.bin b= s=3D1M count=3D16000 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 283.185773 secs (59244558 bytes/sec) robert@sqldb:~ % dd if=3D/ara/sqldb/pgsql/test.bin of=3D/dev/null bs=3D1m 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 33.517116 secs (500556670 bytes/sec) robert@sqldb:~ % sudo diskinfo -c -t -v ada0 ada0 512 # sectorsize 480103981056 # mediasize in bytes (447G) 937703088 # mediasize in sectors 4096 # stripesize 0 # stripeoffset 930261 # Cylinders according to firmware. 16 # Heads according to firmware. 63 # Sectors according to firmware. PHWA629405UP480FGN # Disk ident. I/O command overhead: time to read 10MB block 0.285341 sec =3D 0.014 msec/sector time to read 20480 sectors 2.641372 sec =3D 0.129 msec/sector calculated command overhead =3D 0.115 msec/sector Seek times: Full stroke: 250 iter in 0.016943 sec =3D 0.068 msec Half stroke: 250 iter in 0.016189 sec =3D 0.065 msec Quarter stroke: 500 iter in 0.022226 sec =3D 0.044 msec Short forward: 400 iter in 0.018208 sec =3D 0.046 msec Short backward: 400 iter in 0.019637 sec =3D 0.049 msec Seq outer: 2048 iter in 0.066197 sec =3D 0.032 msec Seq inner: 2048 iter in 0.054291 sec =3D 0.027 msec Transfer rates: outside: 102400 kbytes in 0.671285 sec =3D 152543 kbytes/sec middle: 102400 kbytes in 0.640391 sec =3D 159902 kbytes/sec inside: 102400 kbytes in 0.328650 sec =3D 311578 kbytes/sec On 06/10/17 09:25, Caza, Aaron wrote: > Gents, > > I'm experiencing an issue where iterating over a PostgreSQL table of ~21.= 5 million rows (select count(*)) goes from ~35 seconds to ~635 seconds on I= ntel 540 SSDs. This is using a FreeBSD 10 amd64 stable kernel back from Ja= n 2017. SSDs are basically 2 drives in a ZFS mirrored zpool. I'm using Po= stgreSQL 9.5.7. > > I've tried: > > * Using the FreeBSD10 amd64 stable kernel snapshot of May 25, 2017. > > * Tested on half a dozen machines with different models of SSDs: > > o Intel 510s (120GB) in ZFS mirrored pair > > o Intel 520s (120GB) in ZFS mirrored pair > > o Intel 540s (120GB) in ZFS mirrored pair > > o Samsung 850 Pros (256GB) in ZFS mirrored pair > > * Using bonnie++ to remove Postgres from the equation and performan= ce does indeed drop. > > * Rebooting server and immediately re-running test and performance = is back to original. > > * Tried using Karl Denninger's patch from PR187594 (which took some= work to find a kernel that the FreeBSD10 patch would both apply and compil= e cleanly against). > > * Tried disabling ZFS lz4 compression. > > * Ran the same test on a FreeBSD9.0 amd64 system using PostgreSQL 9= .1.3 with 2 Intel 520s in ZFS mirrored pair. System had 165 days uptime an= d test took ~80 seconds after which I rebooted and re-ran test and was stil= l at ~80 seconds (older processor and memory in this system). > > I realize that there's a whole lot of info I'm not including (dmesg, zfs-= stats -a, gstat, et cetera): I'm hoping some enlightened individual will be= able to point me to a solution with only the above to go on. > > Cheers, > Aaron > This message may contain confidential and privileged information. If=20 > it has been sent to you in error, please reply to advise the sender of=20 > the error and then immediately delete it. If you are not the intended=20 > recipient, do not read, copy, disclose or otherwise use this message.=20 > The sender disclaims any liability for such unauthorized use. PLEASE=20 > NOTE that all incoming e-mails sent to Weatherford e-mail accounts=20 > will be archived and may be scanned by us and/or by external service=20 > providers to detect and prevent threats to our systems, investigate=20 > illegal or inappropriate behavior, and/or eliminate unsolicited=20 > promotional e-mails (spam). This process could result in deletion of a=20 > legitimate e-mail before it is read by its intended recipient at our=20 > organization. Moreover, based on the scanning results, the full text=20 > of e-mails and attachments may be made available to Weatherford=20 > security and other personnel for review and appropriate action. If you=20 > have any concerns about this proces s, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > _______________________________________________ freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/mailman/= listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Jun 13 18:51:35 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A9AABFC109 for ; Tue, 13 Jun 2017 18:51:35 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0073.outbound.protection.outlook.com [104.47.42.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD1DC3119 for ; Tue, 13 Jun 2017 18:51:33 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=U74AtwvIh3h804F6gdetFSb23pHm3f58EsiX+MxKFFc=; b=fLepUfMujMhCzvKXX+3pnSw6LSUruNpRLlX0EBENas3fE5FCnNsjYv8yvEBCoiTrJdC3rcePMrIsVhVDY+ANc7di+6BdTjlUM1qOXsu17YGZA9WmJZirWXzEoGaAVTvMUntEW90GOkVWFH2829RBNgrkU7pSX6/67yKj6EIjSj0= Received: from CY4PR03CA0081.namprd03.prod.outlook.com (10.171.242.150) by SN2PR03MB045.namprd03.prod.outlook.com (10.255.175.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 18:51:32 +0000 Received: from BL2FFO11OLC005.protection.gbl (2a01:111:f400:7c09::171) by CY4PR03CA0081.outlook.office365.com (2603:10b6:910:4d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Tue, 13 Jun 2017 18:51:31 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BL2FFO11OLC005.mail.protection.outlook.com (10.173.160.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Tue, 13 Jun 2017 18:51:31 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB014.032d.mgd.msft.net (141.251.110.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Tue, 13 Jun 2017 18:51:30 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Tue, 13 Jun 2017 18:51:30 +0000 From: "Caza, Aaron" To: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLi0u6ENydglNrDQ4GNB7pMA3LengAOo8uAAAioTWAAUPRVgAAAaP7w Date: Tue, 13 Jun 2017 18:51:30 +0000 Message-ID: <12e95f4e7e814093b1e877d722316ae0@DM2PR58MB013.032d.mgd.msft.net> References: <4410c303314a4d11832a8d248e0b53e1@DM2PR58MB013.032d.mgd.msft.net> <20170613203615.ae6df0a1c94bffe1dc6745fc@yahoo.es> In-Reply-To: <20170613203615.ae6df0a1c94bffe1dc6745fc@yahoo.es> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.205.133] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB014:|SN2PR03MB045: X-MS-Office365-Filtering-Correlation-Id: 0e27fb74-88cf-4592-cd55-08d4b28d3548 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB014.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39450400003)(39840400002)(39410400002)(39860400002)(39850400002)(39400400002)(2980300002)(438002)(13464003)(189002)(24454002)(377454003)(199003)(2906002)(97756001)(305945005)(2501003)(66066001)(108616004)(47776003)(229853002)(24736003)(5890100001)(69596002)(86146001)(33646002)(68736007)(22756006)(106466001)(86362001)(7736002)(2351001)(97736004)(53936002)(2900100001)(6246003)(53546009)(110136004)(38730400002)(55016002)(6306002)(5640700003)(5660300001)(9686003)(189998001)(2950100002)(93886004)(72206003)(966005)(6916009)(42882006)(356003)(7696004)(50466002)(478600001)(46406003)(8746002)(50986999)(8936002)(81156014)(54356999)(76176999)(23726003)(8676002)(102836003)(3846002)(6116002)(81166006)(551934003); DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR03MB045; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11OLC005; 1:eytmsH7zyEot1nkqx+CPEyLku4bBnAGVH6GLKFW3O52oOcaY3AF9IIjbLipMpiuD7joHAjQfbt5s1Cl2WzvlmXr8JuEKOV81n2gKivhm1Jcydjx2CgXvcMdYsBpQxKuCK8pxCdFVJedc2zMcoRVMYVBfLhGC12ujnnrDkWlZJoGEz6f1I9JYsNp27HJz274tOQg49aF4H92YVhwSvpHAxqZapqc69MotYJ81Q+x1g/KSXodCM3ut1CkBw4xNXDtbuvMdBUdNJdjeHvOsHnmsiknnTZd1ctrVfBPXRG0fSVGyiniofIQJAVkcpnqLVloOXXSadvEXgO1e+E3s+wOyVpKkxOCd6Ff37iBz+Q8rJlDf+NOtTyi4oEL5yYXKcYaSPURb14w1ZvJhqaqWt/IXOpb8fwXTASpQWlNmsxRGmcJDjN1DZsbhnrTCbKChLukKQGiOJ9S7JgmOMVE+AwHDAMmuBPNS2qnZDhTATYNJaSXJHQOAVNGwfQNjyscRcbjjs9rvJ73CWe7PRYk5QdGwiUBZi7wKRRu4fbPaYcjOO6U= X-CrossPremisesHeadersPromoted: BL2FFO11OLC005.protection.gbl X-CrossPremisesHeadersFiltered: BL2FFO11OLC005.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:SN2PR03MB045; X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 3:vM+og5kiMHah38ofoZAuvTiBKjM8SBNHq2zaNLq9+E6PMNHrmbysLilQ1J6srXDMGqaRsRaCrmgkoIMcu5PpYdSG1XUSPNVDzO/jqxPtWRTBDM/kl4ijrueMe26YV2CSi6mFdd2oF8KzSMM+hBOuiA6cASHf75I5MVByAYEmPca38r3qDy9lRf3C1UB//8p4K9JjPVJmH3WoiQ1Oe6T4Wo+s/fiGMX2pqDrYueFRU5S0VheLmT1Yvyafbj6pZLn06QBJNXBtUfZgis+6XQcNUv2a/rqYGPuAD45sF6EcJWVeP0/c4DZSeExdNKHQWR+saaiP0W0uY/ciaefdXmA14EfvfV86xQCNOinEWndJ6un/RTcCbSXCtGi/2EKaRuP5RbsYlk6hB+N2kqKnl+gsgUDDy2WNRKAsRwnjvM//Rud+Y7lTkwIbanvTFfx7WQiReLEtNLSwNKnL6IRbPzK3uc1n5L5p+qHq/wrNCf9jxlGbd81FoLeJc1fDm+InFuY4/NPaCNIg4x8NaCXREJM1og== X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 25:X+WXtxzPe74xIbjAGVNFAjRmiDg2KBPaZmBh0oG1qavQHCAPI4/mxay3Wi0r6bFxHFu4n0kEK9odYoUELIZJQ7pqlGy6UpqcYw9bETzGLowBvPYYgknoB4ZjRqM+Z41Ege2V990+bJgn9PIvQl+PBNVBTjUwFaSFDnf8CZGpS80X6CFxmGnpMdVX3y+P4jOeciIcTEHOsxdxcjLn3GTB5A2Xz3GuGEkeH5A/K42y8n+xHWRHbQ9Klqn+tIPVavhMWCToI74Jo1x8hFwuY4Bh+/TGMBbtErRTmXO6cTjFi5GiHwOA8DtaeygjLWhvCONa/HMSqKuICb6wWp9vpckO5Brzbz1pXH74hp4WPxO0KphCjAmlWy8WfxLAznkmchlSjGUwIpCj7FND2ojNC4+FUkF4RzMwbww+T1m1X2ZCv3qvFREaOY4VYnWZ/g9kaJvw3CBMr4DMvuSonLLcZ4h45vTeodGL67sAYqlZUGpMhz8=; 31:xlJXE5z9PePryPj4BNVk+OgvTEpxYEIL1pMgFJCGkUquGJgKHUrftWff4U0/AfTSzEsRvt/Y1t+EEZE78HX7oW9H0ENNHlXTn9cOwQa6orYsh3xhDSaeodWBGYRVAmp5f4mRwy28yVMH/M+U/oxq2CCHRezDOKv4WPy4TQKtljpf2QIy8kXBI7e5jT87rHqAFP+pqBbTCaY6FERmnwtasO98qkjzlqWyV0Z+1tXoR6FSSFimHQRLMCVdPV6bPNU+68fQZZihoRtFfp5piiwzjQ== X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 20:b2Crprij1JXo+4mGdzLw78YC62/aXGD/MlTdrzAnVNGtzVs2TbwRy4W0Y4lo25oDXTeU3f3xefUD+/tDIKLChUh8mpJRgHAAWXvHnwdSeNLTDLviSMRmeOFdiT0VBlgMW/n9Zr3608D0pPHW3QABs2antZrV88hWNFnnyCtOwr6P82XR9cG7BD6b/IpIePcRxUFMUP1zVNsYtv1UiyxFd0L54bNObeRjYhMcNc2i8pLZWYuVzWH8+u3budozmA8R3WKlqHWnfoYG8TqzDiOzuxQQGr+uIUv6T64IX3a1Qosk9ga7hOGaeLutnjUf45VVgIWOSJF0LSjGLFRN/zKZbIGTcPJa2rr3XpjdXYzTCmnn+wIJpXNirCSCVuP8g0F23YkJoqIXFoB38xyAuVm7Y6siFAy3UMl5pEgHUgtkAgQoEFajH9xXi3jnqUxUK/egY95boF+Fgt1dbxmLa93UecXc98t97ctZGbZ1xt+05MXKbhhmUTnjrAkt/XJJ4Jyp X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(84480959824636)(192374486261705)(75325880899374); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(8121501046)(5005006)(13016025)(13018025)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93004095)(6055026)(6041248)(20161123555025)(20161123562025)(20161123560025)(20161123564025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN2PR03MB045; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN2PR03MB045; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN2PR03MB045; 4:SHG34wifH9brYJjaCtwjRmsTs73LYjMay4jFjAB2/tW?= =?us-ascii?Q?H7v3LcGI1cqJqOzVXQoYnypB/LD8ZHHqY/OocnnK9yXWgwMVRdnEpdB1I7RX?= =?us-ascii?Q?Q/0cB9pSzAbcTqBMLbxN7to29Aq1FwqU6t9YnRDPd7+umBkpVZibUbDPt8r6?= =?us-ascii?Q?313CW9Y8jqqKd2uHtbg7fU2a3uaiECLedy23URRBJg3rKuBjtn19hOlwXASg?= =?us-ascii?Q?pbIdb6ZFwc1A3OhlIw+3e3GhBJ4dFxu4ZIvIunGI6FnWx1fQgLZDOjW/jeBS?= =?us-ascii?Q?b49se++1Vg0xgQFwDsbJzhu81X+oYpt24w2Se79p/yx1y9h7qqdbfb7Pz6pB?= =?us-ascii?Q?Kz3pu4cmuDweeDdu6wxCUpt3OvoCUi4m9udmKiJlnNSbjyM3QG8NewyfX73f?= =?us-ascii?Q?GvxIKUQINjjY2wikGQWFkJuMzYLykL+5UoJ4054FF8QutdnjvlF+8W9gDMkA?= =?us-ascii?Q?LP/gfphEHAFsHBy4tZGIfh6B+uK9PechqGEzMkk3lGbyFCnkjJf+EFWxkRwQ?= =?us-ascii?Q?wE8FaKa3F45uuSx8sajYGhutlN20WPqnDOKVEq6ZSKfv51Ulc5Sm3P9dHN5Z?= =?us-ascii?Q?4G7IAnGCf4K564+1DIrHYXhC2Tmk2itH8JjoO6haenLoP9DXXp+N4ACWlsa6?= =?us-ascii?Q?RFfOXBuufQpxNwLdkNjPxhapDb9ZVUjk4VJIayNBeXZI2fDGFG+jL0jLzCtb?= =?us-ascii?Q?ZmxWeVRa0anV2sKAGMbosMUPFwsVaut+sfGrL0fQKzFXJ2ms5GMGj06EdYiP?= =?us-ascii?Q?UhtXVL2apXqLhcGUT4G9AE9JkX7I58fr88alQe3ZRVGJe43Gd5p94lh6lB5b?= =?us-ascii?Q?v/aIJrdB0VtK4ojFGv8dPNR0PXK3lWUmgWL64wNATN/M3Pkow83ZOjcQJhhf?= =?us-ascii?Q?dz2hQBK8mCJPnPVGPOrYH2fx3FGGGduHQHmcR3ocHnV5lKwU2ypUxiNsF+V6?= =?us-ascii?Q?oIVJZyFh8tvr1IoFj5Wl8e47IiAqnCnejXJUyvwzVYQHGKuYFUHIpm0TKD0N?= =?us-ascii?Q?EPqf9jYHbtgLEnmgY2v2yoyS/8GtARYISqVmpEBwbIbQvK/JKTDa05a/oqMd?= =?us-ascii?Q?9ko32hga3SPr5Rk8V9u6/KCaLRMknfrO3FrQ/9nLU5RJrIPajQ7GELxIAD2U?= =?us-ascii?Q?sJ2Ymdv/yJBdc+d4rPyrV/qg/JgP3Xkpulns5/pdoWfKXkUR/vmcmXsUtAhd?= =?us-ascii?Q?RR7AOCqE5OArTFsYQ8Se4yWOqfY1uecx4IbHfgF9SYh4fHO3sA2L/w1IGVOV?= =?us-ascii?Q?8Hgx6xl72hgvbMjzWaJlbeoeAOgbmGZ2rQlINudQ3OTWCMZA1yWQfCh0Ucbw?= =?us-ascii?Q?qPtEaQWlZhtLRqbASyL6Ktd3cqXYxnst4B60xIA/L?= X-Forefront-PRVS: 0337AFFE9A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN2PR03MB045; 23:9qJvKgCA+LguJ5F+4ddzokY8i8AdH9egOifnh4v5bb?= =?us-ascii?Q?3JQrfL9oOhCj/ioTfvqV0obh6Xmsg9p8PcyIKtADSfarDB+xG0rOXOR/z3UC?= =?us-ascii?Q?2yQ8vHNkuFLL80+U2A1K/jbyEbFRIP5r13GrdyMznmZHivKh5mX+Mzn1PJqu?= =?us-ascii?Q?s2ZUVLDn9RHR5GPTfDofT1cH0j3lag4v1HvhhQFq9Zt3lUpJ74RwCNUYWI9C?= =?us-ascii?Q?qeHDd7sfoxp5k5jvyOwIvtbGABAOXDB9UqD51JboUEbL9AGxZIZnPRO0ZQqb?= =?us-ascii?Q?eaCM05YiVGHqPwLsOzgyXrP5T6MMU7ql+8/VXN130nxocH97nrOx9Q4d5V2d?= =?us-ascii?Q?74CNq9XLSx1ee3vmVp2t/1q96H9r4otpvViIKRHD1I/qgMU+KfCxBO7MPAZ3?= =?us-ascii?Q?ojg0xuGx3vzTVp6HVxoqrCRXj36/ODQeOH7SJnK6sQuSAdUID7bp0y/ASgza?= =?us-ascii?Q?OQIGoVSAv0WvhuIOrhTauVyI9BaTyXIql+jcZ+l5StyqjT662VACRQCfTk32?= =?us-ascii?Q?k6Mlzog0Ha/9AwT7E8VPtT9moNn5GsUyW6oVdtZAkn2wBmacy10XVBxncDKG?= =?us-ascii?Q?rg9eznu56VaNR/3W8dGrjosrOSt4uO979k7S3oUDZ4BA9CwV2EW2f8OUDzOj?= =?us-ascii?Q?HYJGuRSssdTthwzMx9KNrAqB7IuVm7ZkgDLrY1XrjxQ/SnxltAo7BDOTPDy/?= =?us-ascii?Q?n9TbDM8FgixvkyB2j9GGM/29INeoEyT4kT2mcjBR9kPW56FQZJALaaQVZWcM?= =?us-ascii?Q?Ja9CAEAIU0C3mFUXr8bkLZnszspyiqAReunO3jdAVjAn6+Rb1v7PGC4IRrRd?= =?us-ascii?Q?PD6NlbWJzD0fuCm41MQgXEuqLkmCq0DUA3eMGhwaMzb21lXv2kJeuW0NGnfO?= =?us-ascii?Q?GLi2G86W1l5ACtoDTDNaYB+XHno2Mv9QRYZFUq/7Jp2o8kXycsX8Rm/Z45Jx?= =?us-ascii?Q?KOdY/G3Rf84FCTZOK9no7gvWh2Gu+q6wnwSCaz+RFG5N6V5O5yUA4gnWiQ+X?= =?us-ascii?Q?eB3OmSI72PW1/J73ij4ny7EeVaKD+o9aojQrJ6UPbI5ezyYTufy6OeVWYDZF?= =?us-ascii?Q?qU6iVYy39gHzy77W3wYAK/xTbEs3neMrbF5U0ioI2/KkwJL4YmmY4sSMfOoF?= =?us-ascii?Q?XCiNRTswIHT0TBgimtHF6k8kCewOMIAxPnmyuFp+G7ETkROpMZyQ6BZjg+Kw?= =?us-ascii?Q?LtpAAwvKVXe75IuoZMinbwEdwMlg3pG9ibOOIuYNswQg0qYQFf2LVVEYVZ9y?= =?us-ascii?Q?aGsZHp2XQocfWb0cYcGWxeFFn1MFvAQRqpGXJzwuWc47T3yMk76Q/XfvHYIA?= =?us-ascii?Q?uHxIAm8CJ7avLalcjkNrSdf33GXAkqDS2t/PsvgpLtBjTxVxJtTwBdgSNIXO?= =?us-ascii?Q?oV7R0T72rZvxksR/R0K8E9Iap/O3mX5tkXxtC7TvyQBz0QshtCrAyKe90y4X?= =?us-ascii?Q?kvlwdx1FISBh6indqkFcjEoli7WzP9Z/atTYvoif+gu25YO8ORHhdQUxpUgA?= =?us-ascii?Q?vT1zq7AX6XAAH8Y1VBH++YgsA28xBdsLgcRq+0jMlZsSS4JSmdsOQ5Rf16JS?= =?us-ascii?Q?B3H3eBs0o8/LDoLUUpamj5D1/Q55RUdtrzbwrrK/wycwP90qkMSWtBeAg0Di?= =?us-ascii?Q?BXFJVEvBvYP4M4ulY8cdssZreP1QTxcby5jzRcSpM3UsVOTjyS3R7E//NfHL?= =?us-ascii?Q?9z?= X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 6:C6GScrix7Gn0G2EE16SDvMPbptM54gGThCYsLQf6w9lsN0r45nYNNZ1i/6Xj5EXjuqX8V/VxOT6FL+kvGXKc+PxOUNBymGz/f2Bwey2C5fDAu7UN/P3jRgBj9guUN/LxnxWbWOQIDSVexWmGpafhbdooH6D2KJKBcd2r07xS/Sq0iAwztUVfnBuFykWw2/MM50l9O4UUyMFqFff1sYIkHmZbi2vfC3s76gXYPYLCzbqPcwHHADq5LN1ImyJ/J8NjP7g3U2ATvdOs7H8i7LjvvdURdZfX+J5E4UolHaWp6oewMK5vjeer0hRln3/Jw/osmblZhPVa8S2WcB4wqjMlOfaTKQOemuFFqJBPgTHkV1r9cm5B1T6VGnz8GwskoCrga9nEZdryamHQbyZfOjZ95ELX1kPZyIZwFwXSjcaRC3jeCO5Cw3CCNx4101abo5zYH8MtFBHVzaYtFaKlyvwavRO3cRFFajffTdG36eiHwVdPmhEE0EOsth5oHCyc670rgqA9NlkB9Ahf0gI+2D6YS1tGNDt3dW/S77/AlwqkCVw= X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 5:x/twqxzb8CrsgCKhGUk+PGvCft1Ggre0s5fe1k6jDmZA1F9hJasW/Fk9zfXcfKGM5+P6/Dla7bXl/zaGSPFWhFERVWaDEtt6nAN5MJcIYc875AojzvB6GLiGly9XplQtfgyRa+Da8idJddvKv/IW68CnLb6u5LBN2xzS0PKThRsCrk8huKAnD6uIgxv9j0SFGQs0i7BDQfo8c/MtdKXMBwD8I8idgJ+IzPBWVRE+uaKTSEwQC4H7QSAAWaYeuI4sWtm8tVU6fvAvShUn9LiVXb5ZdtHBPKSq+aiXDQTxueumiOkpOXd/l6XCig3n30Vi2yfWqrPOAxYg1E4F8DjWFaDSAPxYdRHZjKQ9yDtSUOOHGoTr26LtCzjPeN0MNavSW/hcv3feAzL37GTjcU2fBhl5uKSh4fh+FOE1QebIV/2e95LA1yGMbvtdCIxGmxiPvlqN/xlKJMJQXqzgZZofBa+eg5uFws62GQLxlUxrkxyObfMwYM/6Kwtq9PIxM9gI; 24:VSgGE3wFuXurI68vs8n1mgRWvQmIXpNb/ESim7E7jhedkwRm+H9PzJgvCTYgJm47GuMCJZZ1P7VIzhpDb+/z5ukotKjXaLAIhVsRzYWtG9A= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB045; 7:egK7DMtIXzGtjKQu37SNJVUeI6nFCb2xu214d0SzptZlDj+yLL5TmitqWkCD50Iu5SO80KnDCVeoLD21SneSJZwUJC7EXu4Ry3s0CqzVnHPoByyJMAoLITua27zoEepshuFDJaniFJXkSqDzdDGB7u1imuAqB9jf3zOaBL64pB/y2m46y7JzahlKkS2drw1LfvCkEXBoZrSt6hiBe+M0Fv30Ylgsc/kGb6cUhTV1KIwT177vkDb/WQUzr0c5vfr0w6r2VYOsOtzH+oCEholo7qT7Vb8xjjZITMSnt8cKA3EaUMUhOjRRTuQzM7ZHRQwpDywEi2uYqZ05At+A3cOkeQ== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2017 18:51:31.9243 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR03MB045 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2017 18:51:35 -0000 In prior testing I completely removed PostgreSQL from the equation by creat= ing a 16GB random file on the pertinent ZFS filesystem and reading after re= boot and later after the performance degradation hits. Read performance go= es from ~865MB/s with 95-100% busy in gstat down to ~61MB/s with ~10% busy. It's not Postgres. -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freeb= sd.org] On Behalf Of Eduardo Morras via freebsd-hackers Sent: Tuesday, June 13, 2017 12:36 PM To: freebsd-hackers@freebsd.org Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD perfo= rmance drop < 24 hours On Mon, 12 Jun 2017 04:50:44 +0000 "Caza, Aaron" wrote: > Thanks, Jov, for you suggestions. Per your e-mail I added ?explain > analyze? to the script: Can you check that autovacuum is not involved in a lock storm? Autovacuums workers start to work but a higher level lock is acquired by ot= her process/transaction stopping autovacuum workers continuously, making au= tovacuum useless (but trying continually to clean). Also, you're comparing Postgresql 9.1 with 9.5, are you using the same conf= file? Did you upgrade it (conf file) to 9.5? --- --- Eduardo Morras _______________________________________________ freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/mailman/= listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Wed Jun 14 01:22:50 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95FC0C316FE for ; Wed, 14 Jun 2017 01:22:50 +0000 (UTC) (envelope-from baijiaju1990@163.com) Received: from m12-11.163.com (m12-11.163.com [220.181.12.11]) by mx1.freebsd.org (Postfix) with ESMTP id E1204738FE for ; Wed, 14 Jun 2017 01:22:49 +0000 (UTC) (envelope-from baijiaju1990@163.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=Message-ID:Date:From:MIME-Version:Subject; bh=FCpqg Q6/YH51f4vEb4KhA/0F3cbFe44Ny+1OZ9HhXeE=; b=jVk8yxM9pM+cquyVTKg+m XlpH40/7KhPcxrYBpd4EGtLWtWKCL6CsUL1Ihb9oR+ZIETPRAd81dLw0ik5+hi1/ dGZyAQdAnaoPPtrP78RawN/FdYk0mL6fEvQGzsvuS4Bs9TiHDZjcQxO7fbz51hWR UDLF9DybDd8R2RleXt3ykE= Received: from [166.111.70.19] (unknown [166.111.70.19]) by smtp7 (Coremail) with SMTP id C8CowACHjxdWjEBZoRdjLQ--.44985S2; Wed, 14 Jun 2017 09:07:34 +0800 (CST) Message-ID: <59408D05.7070407@163.com> Date: Wed, 14 Jun 2017 09:10:29 +0800 From: Jia-Ju Bai User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120410 Thunderbird/11.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: A question about in lock usage in FreeBSD Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-CM-TRANSID: C8CowACHjxdWjEBZoRdjLQ--.44985S2 X-Coremail-Antispam: 1Uf129KBjvdXoWrZF4fXr4kGF1kury8WF47XFb_yoWfJrg_Ca y0kFWDCa1agr4Sqa1jgF4fCr4jgF48ury5AFyrAF13Jw1fJa97WF1kua4xZF4fJF45Aw4D tr98Aa4a9rya9jkaLaAFLSUrUUUUUb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUvcSsGvfC2KfnxnUUI43ZEXa7IU5bID7UUUUU== X-Originating-IP: [166.111.70.19] X-CM-SenderInfo: xedlyx5dmximizq6il2tof0z/1tbiYxb2elaDtYC0QwAAsu X-Mailman-Approved-At: Wed, 14 Jun 2017 01:39:22 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 01:22:50 -0000 Hi, I am a freshman in developing FreeBSD drivers, and I have a question in lock usage in FreeBSD. The kernel provides some kinds of locks in developing drivers, such as "mutex lock", "mutex spin lock", "rw lock" and "sx lock". I want to know which locks should be held when the thread can sleep. From my knowledge of FreeBSD document, I make the following points: (1) "rw lock" and "sx lock" can be used in this situation; (2) "mutex lock" and "mutex spin lock" are forbidden in this situation. If my points are right, I will make another point: mutex_lock will let the thread sleep when the lock is requested, so the mutex_lock can not be called in nested style (namely it is unsafe that mutex_lock is called again when a "mutex lock" is held). Are my points are right? I am looking forward to useful opinions and answers :) Thanks in advance :) Jia-Ju Bai From owner-freebsd-hackers@freebsd.org Wed Jun 14 05:05:24 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02C92C79F22 for ; Wed, 14 Jun 2017 05:05:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81E0479CEB for ; Wed, 14 Jun 2017 05:05:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5E55EPa050495 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 14 Jun 2017 08:05:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5E55EPa050495 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5E55DiT050488; Wed, 14 Jun 2017 08:05:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Jun 2017 08:05:13 +0300 From: Konstantin Belousov To: Jia-Ju Bai Cc: freebsd-hackers@freebsd.org Subject: Re: A question about in lock usage in FreeBSD Message-ID: <20170614050513.GI2088@kib.kiev.ua> References: <59408D05.7070407@163.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <59408D05.7070407@163.com> User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 05:05:24 -0000 On Wed, Jun 14, 2017 at 09:10:29AM +0800, Jia-Ju Bai wrote: > Hi, > > I am a freshman in developing FreeBSD drivers, and I have a question in > lock usage in FreeBSD. > > The kernel provides some kinds of locks in developing drivers, such as > "mutex lock", "mutex spin lock", "rw lock" and "sx lock". I want to know > which locks should be held when the thread can sleep. Replace should with can in the question. > From my knowledge of FreeBSD document, I make the following points: > (1) "rw lock" and "sx lock" can be used in this situation; You cannot own rw lock while trying to sleep. > (2) "mutex lock" and "mutex spin lock" are forbidden in this situation. > > If my points are right, I will make another point: > mutex_lock will let the thread sleep when the lock is requested, so the > mutex_lock can not be called in nested style (namely it is unsafe that > mutex_lock is called again when a "mutex lock" is held). No, mutex does not put the thread which cannot obtain the lock immediately, to sleep. The thread is blocked on turnstile. > > Are my points are right? > I am looking forward to useful opinions and answers :) Read locking(9). From owner-freebsd-hackers@freebsd.org Wed Jun 14 08:01:55 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B0F6D88610 for ; Wed, 14 Jun 2017 08:01:55 +0000 (UTC) (envelope-from baijiaju1990@163.com) Received: from m12-17.163.com (m12-17.163.com [220.181.12.17]) by mx1.freebsd.org (Postfix) with ESMTP id E4C837E916 for ; Wed, 14 Jun 2017 08:01:54 +0000 (UTC) (envelope-from baijiaju1990@163.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=Message-ID:Date:From:MIME-Version:Subject; bh=wvz85 0iao2G+3Q/WDIDDk5H3Nc0WTVbwzh7BUYYCyhY=; b=W+0PqNpIGt2DJrcqTQybe 191AejjqhUc3RtCf7oMhavXvuWvBIgfXJVQd9qlB7KOKQSEdhkUbxBzQu1XY6JVf E67EKOq4IJD4ad20ch/SEent8n7IPMwXAmYBT1umIl11EFrCR62if5SM6FbOWSRr zFj1MeCI8rz9Tb5EoRakjE= Received: from [166.111.70.19] (unknown [166.111.70.19]) by smtp13 (Coremail) with SMTP id EcCowADX7i7X6UBZzdtdKw--.55961S2; Wed, 14 Jun 2017 15:46:32 +0800 (CST) Message-ID: <5940EA87.4030002@163.com> Date: Wed, 14 Jun 2017 15:49:27 +0800 From: Jia-Ju Bai User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120410 Thunderbird/11.0.1 MIME-Version: 1.0 To: kostikbel@gmail.com CC: freebsd-hackers@freebsd.org Subject: Re: Re: A question about in lock usage in FreeBSD X-CM-TRANSID: EcCowADX7i7X6UBZzdtdKw--.55961S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7CFW7ZFy3uF4rCF43tF4ruFg_yoW8Xr13pF Z3Gas8tFs5Zw48Zwnrt3yfur40y3y7GFW5Xry3WF1xZryj9an3KrW7KFZ8urWfWr1xXa4v vw45XF97Aa90vaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jkKsUUUUUU= X-Originating-IP: [166.111.70.19] X-CM-SenderInfo: xedlyx5dmximizq6il2tof0z/1tbiThj2elUCyYSXHAAAsL Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 08:01:55 -0000 > On Wed, Jun 14, 2017 at 09:10:29AM +0800, Jia-Ju Bai wrote: > >/ Hi, /> >/ /> >/ I am a freshman in developing FreeBSD drivers, and I have a question in /> >/ lock usage in FreeBSD. /> >/ /> >/ The kernel provides some kinds of locks in developing drivers, such as /> >/ "mutex lock", "mutex spin lock", "rw lock" and "sx lock". I want to know /> >/ which locks should be held when the thread can sleep. /> Replace should with can in the question. > >/ From my knowledge of FreeBSD document, I make the following points: /> >/ (1) "rw lock" and "sx lock" can be used in this situation; /> You cannot own rw lock while trying to sleep. > >/ (2) "mutex lock" and "mutex spin lock" are forbidden in this situation. /> >/ /> >/ If my points are right, I will make another point: /> >/ mutex_lock will let the thread sleep when the lock is requested, so the /> >/ mutex_lock can not be called in nested style (namely it is unsafe that /> >/ mutex_lock is called again when a "mutex lock" is held). /> No, mutex does not put the thread which cannot obtain the lock immediately, > to sleep. The thread is blocked on turnstile. > > >/ /> >/ Are my points are right? /> >/ I am looking forward to useful opinions and answers :) /> Read locking(9). Thank you very much, your reply is very helpful to me :) According to your advice, I think I get the answer from locking(9). Namely, it is unsafe that the thread calls the function which may sleep when the thread holds mutexes, reader/writer locks or read-mostly locks. Is it right? Thanks, Jia-Ju Bai From owner-freebsd-hackers@freebsd.org Wed Jun 14 14:45:30 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96D36D8F932 for ; Wed, 14 Jun 2017 14:45:30 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B57D668E5 for ; Wed, 14 Jun 2017 14:45:30 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from lm0.local (rrcs-96-10-213-43.midsouth.biz.rr.com [96.10.213.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id 28C27239591 for ; Wed, 14 Jun 2017 10:36:28 -0400 (EDT) To: freebsd-hackers@freebsd.org From: Nikolai Lifanov Subject: crash under load in GENERIC Message-ID: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> Date: Wed, 14 Jun 2017 10:36:20 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="pDEDVNaN1PC5TF40wwpEemff9GFSOgd06" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 14:45:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --pDEDVNaN1PC5TF40wwpEemff9GFSOgd06 Content-Type: multipart/mixed; boundary="S7FMxSxI1HdrLLWSCun9spE1GhnUKn3BP"; protected-headers="v1" From: Nikolai Lifanov To: freebsd-hackers@freebsd.org Message-ID: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> Subject: crash under load in GENERIC --S7FMxSxI1HdrLLWSCun9spE1GhnUKn3BP Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi! I haven't nailed it to a specific commit, but some time after head-319667, my package builder box crashes when running Poudriere. The earliest version that I can confirm crashes is head-319735. I have full crash summary and core dumps if these are helpful. __curthread () at ./machine/pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:232 #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xffffffff80a705f6 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xffffffff80a70bf6 in vpanic (fmt=3D, ap=3D0xfffffe20249e4360) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xffffffff80a70c43 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xffffffff80efe960 in trap_fatal (frame=3D0xfffffe20249e4600, eva=3D1= 024) at /usr/src/sys/amd64/amd64/trap.c:801 #6 0xffffffff80efe9d2 in trap_pfault (frame=3D0xfffffe20249e4600, usermo= de=3D0) at /usr/src/sys/amd64/amd64/trap.c:657 #7 0xffffffff80efe168 in trap (frame=3D0xfffffe20249e4600) at /usr/src/sys/amd64/amd64/trap.c:421 #8 #9 _sx_xlock_hard (sx=3D0xfffff8022c4b4258, x=3D, tid=3D, opts=3D, file=3D= , line=3D) at /usr/src/sys/kern/kern_sx.c:576 #10 0xffffffff80a78cac in _sx_xlock (sx=3D0xfffff8022c4b4258, opts=3D64, file=3D0xffffffff81142b62 "/usr/src/sys/kern/uipc_sockbuf.c", line=3D= 283) at /usr/src/sys/kern/kern_sx.c:308 #11 0xffffffff80b04aff in sosend_generic (so=3D0xffffffff81142b62, addr=3D= 0x0, uio=3D, top=3D0x0, control=3D0x0, flags=3D283, td=3D) at /usr/src/sys/kern/uipc_socket.c:1464 #12 0xffffffff80ae1292 in soo_write (fp=3D, uio=3D0xfffffe20249e4940, active_cred=3D, flags=3D, td=3D) at /usr/src/sys/kern/sys_socket.c:146 #13 0xffffffff80ad8407 in fo_write (fp=3D, uio=3D0xfffffe20249e4940, active_cred=3D0x0, flags=3D0, td=3D) at /usr/src/sys/sys/file.h:307 #14 dofilewrite (td=3D0xfffff803d3dc4560, fd=3D9, fp=3D0xfffff805a8f320a0= , auio=3D0xfffffe20249e4940, offset=3D, flags=3D0) at /usr/src/sys/kern/sys_generic.c:592 #15 0xffffffff80ad7ff8 in kern_writev (td=3D0xfffff803d3dc4560, fd=3D9, auio=3D0xfffffe20249e4940) at /usr/src/sys/kern/sys_generic.c:506 #16 0xffffffff80ad7f86 in sys_write (td=3D0xfffff8022c4b4258, uap=3D) at /usr/src/sys/kern/sys_generic.c:420 #17 0xffffffff80eff399 in syscallenter (td=3D0xfffff803d3dc4560) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:132 #18 amd64_syscall (td=3D0xfffff803d3dc4560, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:903 #19 #20 0x0000000800ba2fca in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffe498 (kgdb) - Nikolai --S7FMxSxI1HdrLLWSCun9spE1GhnUKn3BP-- --pDEDVNaN1PC5TF40wwpEemff9GFSOgd06 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJIBAEBCAAyFiEE5oT6TcuaWvG5gtjzZ6sv56ecR0UFAllBSeQUHGxpZmFub3ZA ZnJlZWJzZC5vcmcACgkQZ6sv56ecR0V3oA/8DVs2Fs3NjPYudCOVw8/dqM4EJhWs vaSO4iVpo1qcemM8V34Kx89s/FeqIzHnpgId25EaXvrBvFe2VobDCs59hwX9jlsF eO4aAY78ExOpK1bgMGo7GrdxyreRejVWyXCM43ryFOk30jO3W02t4pl+pX4bMgH5 RLzCBhKfhih8f5t5v2wPv6DWqI7cScIxDMKIfgrHbxKyXd3eWWBPr75t7xWxOgiv 9LZcKVPA4LFNSYLNG1Jo5kuJ1sc+oj2l7vCTq1o1h80SetoBncDwWheOB24kFQmG f9zLe0NqaVygGEjj45CJ7N0bBNlQWrUCqF7zQJ6N8PFIF8z8BWTSAzhZgsev38r2 HQY04lMP7gon/cMCSwEl/y81Gspmpy4PmQxdr+nvW5FKdAW4gNB3NUM8ZiZhk47f A/huFNpZO+nbTFjZjxZcugBOXw4l/H6oVE+E9TxFzR1cIUWKNAObJtaPzZ4HW2f1 +VEY1M6JlAxZJcOxnoSCgTtPm6GPJR5US0Dc4OAAu1GFQzpHYEcH6Kg+Udg4gMQs YlXtDMfnsa6eHtQ9Tz8yDMM+QED2Mxm9/y1lZ0Q07TYngAxc9boIcYL2tagiE8Du i2Tal+GppX1ekNcxi97EzYiWdDBXkqjcOzHJCAkdNB8xLznuNwTYkVG8Ag5TA6vE 5uYg/1MzpZsKI1I= =l3nY -----END PGP SIGNATURE----- --pDEDVNaN1PC5TF40wwpEemff9GFSOgd06-- From owner-freebsd-hackers@freebsd.org Wed Jun 14 15:01:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB23BD8FCCA for ; Wed, 14 Jun 2017 15:01:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6ECE9671FA; Wed, 14 Jun 2017 15:01:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5EF1iMg083840 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 14 Jun 2017 18:01:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5EF1iMg083840 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5EF1ixi083839; Wed, 14 Jun 2017 18:01:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Jun 2017 18:01:44 +0300 From: Konstantin Belousov To: Nikolai Lifanov Cc: freebsd-hackers@freebsd.org Subject: Re: crash under load in GENERIC Message-ID: <20170614150144.GP2088@kib.kiev.ua> References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 15:01:50 -0000 On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: > Hi! > > I haven't nailed it to a specific commit, but some time after > head-319667, my package builder box crashes when running Poudriere. > The earliest version that I can confirm crashes is head-319735. Do you see crashes past r319754 ? If yes, try reverting r319722+r319754. From owner-freebsd-hackers@freebsd.org Wed Jun 14 16:16:17 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59C6FBEE0B2 for ; Wed, 14 Jun 2017 16:16:17 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FDB06ED17 for ; Wed, 14 Jun 2017 16:16:16 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-lf0-f51.google.com with SMTP id m77so5058110lfe.0 for ; Wed, 14 Jun 2017 09:16:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=WTAKBsgVe2KqK1OWbCLBrI/y0pUh6WkIE/jSTPsS/I0=; b=kLEJb10pCiMX5o3a45Aye/xFWdKEVDQrFuiq1TylcnLHOe7n0qXtIfi4ES4fSbUnBW +9W+1IlkPGl3DlL60HgtAdKAXWOwrYJpXY0f89iONh1QAC7D8gPbVr2vgq1ybWYUJthd uvVDnjTFhVzKyZWmfwQrkN3Q9rAIg2HhfHkpK01ByC2sYZTM8Jk8YAjKdLBK5pCbP2Lh +/TZT+Vs3jQgtseGCjD7YcOtMP4imb5q3lIpA36CNkxrKWfxfWBvvw6PRNMtcAJetmAS ZqkEtInv2MRkKTQpfgqyw49GAzSrzegtAKl8IRQO6hjIoK6cV4UhaZKCm2yIn772yF39 0dfA== X-Gm-Message-State: AKS2vOz37TQIM/ePSRrcY3laIm3WvkvxVEPetXk4dC2hCWlW7ytrxSoF cQUYbupKiRJbJ7ULvBA= X-Received: by 10.46.84.73 with SMTP id y9mr261790ljd.130.1497455122733; Wed, 14 Jun 2017 08:45:22 -0700 (PDT) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com. [209.85.215.51]) by smtp.gmail.com with ESMTPSA id e28sm80417ljb.58.2017.06.14.08.45.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Jun 2017 08:45:22 -0700 (PDT) Received: by mail-lf0-f51.google.com with SMTP id v20so4387158lfa.1 for ; Wed, 14 Jun 2017 08:45:22 -0700 (PDT) X-Received: by 10.80.180.188 with SMTP id w57mr567753edd.42.1497455122359; Wed, 14 Jun 2017 08:45:22 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.164.154 with HTTP; Wed, 14 Jun 2017 08:45:21 -0700 (PDT) In-Reply-To: <5940EA87.4030002@163.com> References: <5940EA87.4030002@163.com> From: Conrad Meyer Date: Wed, 14 Jun 2017 08:45:21 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Re: A question about in lock usage in FreeBSD To: Jia-Ju Bai Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 16:16:17 -0000 On Wed, Jun 14, 2017 at 12:49 AM, Jia-Ju Bai wrote: > Namely, it is unsafe that the thread calls the function which may sleep when > the thread holds mutexes, reader/writer locks or read-mostly locks. > Is it right? Correct. sxlocks ("shared-exclusive," although I tend to think of the "s" as standing for "sleepable") can be held calling functions which may sleep. Best, Conrad From owner-freebsd-hackers@freebsd.org Wed Jun 14 16:16:53 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A852CBEE16F for ; Wed, 14 Jun 2017 16:16:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D4DF6EE2B; Wed, 14 Jun 2017 16:16:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5EGGm0a000583 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 14 Jun 2017 19:16:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5EGGm0a000583 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5EGGmgX000582; Wed, 14 Jun 2017 19:16:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Jun 2017 19:16:48 +0300 From: Konstantin Belousov To: Conrad Meyer Cc: "freebsd-hackers@freebsd.org" Subject: Re: Re: A question about in lock usage in FreeBSD Message-ID: <20170614161648.GQ2088@kib.kiev.ua> References: <5940EA87.4030002@163.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 16:16:53 -0000 On Wed, Jun 14, 2017 at 08:45:21AM -0700, Conrad Meyer wrote: > On Wed, Jun 14, 2017 at 12:49 AM, Jia-Ju Bai wrote: > > Namely, it is unsafe that the thread calls the function which may sleep when > > the thread holds mutexes, reader/writer locks or read-mostly locks. > > Is it right? > > Correct. sxlocks ("shared-exclusive," although I tend to think of the > "s" as standing for "sleepable") can be held calling functions which > may sleep. Note that 'can' there refers only to the ability to make calls into the locking subsystem without making it panic or triggering witness warnings. The concrete sleepable locks itself might be unsuitable to held during specific sleep, or cause some side effects if owned while sleeping that make the system unusable. For instance, we used to own vnode and buffer locks while filesystems copied data in/out in the VOP_READ and VOP_WRITE. It was tolerated by locking subsystem but was the source of large series of hard to tackle deadlocks. In other words, the permit to own the sleepable locks while sleeping is only about an absence of the tiranic overseer who disallows that, and not a guarantee that the permit is handy to use. From owner-freebsd-hackers@freebsd.org Wed Jun 14 17:11:20 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60EFABEFCCF for ; Wed, 14 Jun 2017 17:11:20 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0075.outbound.protection.outlook.com [104.47.40.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8A9671C7D for ; Wed, 14 Jun 2017 17:11:19 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=BgO/ydwdQ3l+7OidZa4X/P9MKJm2s9buG014crvVSDI=; b=aca6s2AibCjHZRphi8vRYh3OWGEXUKGLfYqeWNv/4qLw+TBArptrKprqJj7hHFvTIUojiqw8I+jH2ro4ZHknSP56AXrI8roxhJJHkGN8uE42Wt6rAIAF7fKHbceiXbQ6Cen7lxycU1yXrEJLl38khSPc/AlCDCNgKkDoEzI5MzI= Received: from DM5PR03CA0032.namprd03.prod.outlook.com (10.174.189.149) by CY1PR0301MB1243.namprd03.prod.outlook.com (10.161.212.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Wed, 14 Jun 2017 17:11:18 +0000 Received: from BY2FFO11FD026.protection.gbl (2a01:111:f400:7c0c::133) by DM5PR03CA0032.outlook.office365.com (2603:10b6:4:3b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Wed, 14 Jun 2017 17:11:17 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BY2FFO11FD026.mail.protection.outlook.com (10.1.15.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Wed, 14 Jun 2017 17:11:17 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Wed, 14 Jun 2017 17:11:16 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Wed, 14 Jun 2017 17:11:15 +0000 From: "Caza, Aaron" To: "freebsd-hackers@freebsd.org" Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLlL01Bja9RgR6rT826WHBpLYHdbg== Date: Wed, 14 Jun 2017 17:11:15 +0000 Message-ID: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.205.69] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB013:|CY1PR0301MB1243: X-MS-Office365-Filtering-Correlation-Id: dff4c4ff-ff91-4911-687c-08d4b3485eb2 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB013.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39450400003)(39410400002)(39840400002)(39860400002)(39400400002)(39850400002)(2980300002)(438002)(189002)(199003)(9170700003)(24736003)(2501003)(2351001)(66066001)(189998001)(356003)(97756001)(7736002)(5890100001)(229853002)(5660300001)(8676002)(86362001)(8746002)(2900100001)(8936002)(47776003)(46406003)(305945005)(106466001)(81166006)(6916009)(2906002)(7696004)(86146001)(6116002)(102836003)(3846002)(108616004)(6246003)(42882006)(72206003)(54356999)(50986999)(22756006)(50466002)(23726003)(478600001)(55016002)(110136004)(5640700003)(53936002)(33646002)(9686003)(38730400002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0301MB1243; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD026; 1:j8oCGxMygF6ju/Gtq+clkNvX36+kUA4uXvXgfe/14KFwWOme4u09TF/PYVkxQOut21jABvU+O8S6PYX8Bha6aXgCYlXJ67U197dsX9Y8F7ycHF5qPlyc1q09/cWsICmqoYleAKKYEG7ECRQd8208+w+M0s6mX6amFnI3iBHsFLkQmSeEk2bRT/99vdrpyLVKkwOZkP7QYZH0NTGl7zJdS9lwn94VxIJ5GYbgiSS36d+aP/MIXQuU4mPXTUaiEi8fpl5Jj8uj3yIH/kuzAJlbGnKUW7ZBQ6HOIkiTpAiyYu2CCmV1XQLGtMxVoX3/WdB0P6rfwDu+xNnk98oAAjjv09ZkNiwgHPxiJ3f2wAae9DJMeEM8RsoPypUfDUvvPdfJgdtkLqfCWPoX1LmjDqMUQZMXLZYXKkTmSzXQ8O5MZCEcQY9Oa2OxAxA+17DOQgrBSOaA8TNdYM0zWSQ/75N9qnXxZkTFJtWMvX7MJLA+n3l8OpZ/mEOhfLtYrbfkuZyqnuwYtDNiywUfEuDzQyif3yu5Tv9wJO0w8sSTKUZqME8= X-CrossPremisesHeadersPromoted: BY2FFO11FD026.protection.gbl X-CrossPremisesHeadersFiltered: BY2FFO11FD026.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:CY1PR0301MB1243; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 3:A2WK6cKMCGlLwfc8FeCAkr3rjlfWmEwqE59KlhMQ2ZK1oJo5nLPExxYhnsJ5aX2pzOoUpMCAIsZ+XpxKSrYTdU334suYqmPJo8rY6viqJcUTkrmg4Q19BqLK6/KTYrEHa7wzGs010e4zt4C0FLMJWczPo8Epa56fYYCC5BBr4J+Qa1rUoh8VdX2r1TCO+NdvqaN6uTvP3HjjkkMfNitIWJO5tPf0r5Gzmuebic5X1YrT/BjMO+M2z0quXcRzjWrR1ysjBPF+UDRapmnV3QhgHnnVMZKYR+gOTOsiUwmnIrc7JgsCq2Y3EStbklGKBCY6SYAQ1X+4YL+6wCfX0xjN2r9FMp5Y/8p3+ffexskx9kBfbBC+hayFexHWJ/qHn4qpKojOuIAROs5vQr6rB7d8KJkMa7Clb+ns4DnStjCMMYPq000V9PgtOKEqNqentwBf1SoXiO4YrdF6RYSUuVbKxxa8jqsaCVPBN2Ij9LHlyx4= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 25:gvxuM8j1W1VxzGvKrp7gMjQRAJWn5vQn1yLO5H/BaioL/uXSvf2u8PBCLaUU2aWrj+Ij2TKHFv+3z49fZpBCJvh3MXfoNmREliIMpYcKf9kslc4D2pW6dT6W1+c0G4ch0wp6WrL7k0s1QBk2J1MLeEcvCAIJ762/Ut8NMf1rGCs3P7oD2V7W2OJZ+3CUOar5s+0V8EYao+UQb3R4jJNhgjICqOhV7pCgMjwxtYHqkVXdn5GM1P7VFIyyT2pWvv8SWAANtuKNfAZbXjfQVq0CUeiuPw2yWD/K6FgFvJPRW8STRy95+/adHO2MBM/gZzExeCjAWQwcWzU216AV24GwGbWqp4SceVsF28+gfifoLy92xYAvlQmi9KGdQQYUdk33dKK8o+nl1r/0mqP/PHnkLTWW2nHDGrVtMkOe6eyaSlrwSJeuOUEvJ80w2SuoDWL34Ib1FeOXuvy9iNt+qVOOC9BA0QRz2P5AD3bhD203yv8=; 31:erY0FOYcPcud6js+zGE1FR3w5QaUZEyJKJzqNNqFARZC3CWWy9eSGJ/RcSAD/wueZhnO2jlduYBksknIiyp1isEP1V3vu1VDkcuxMzOWDe1KLwxkzeRksB2PhaB+nh04JdqeCyON/kYFvFTzbjuKSo8lF5Qina1lCAyU1/YyhUJAjpqUV6lW8St6ydU64QfBVRbS7/WRRHa9Yke1qkOGtoLM2jsEbtgku7xNa6hCSO0VvGT6lbF0A5Shvlov2u0hkA52wgvF27OIZVmpb+4UgEZkS6Wsk6Na9Zo+2BFy4/E= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 20:fKsW1D/5H6ri1myaRmGfjm0nf8IThr8dhhRBpXhIsy0dGCb91E6pnUU8OVCuZVKPzJzUpqkBnvWHsIEGE4CgRFr/P/+awA0b+OpnjWueVCSQPLQ1awXKm35K0L5Q1C1Qtm+lG4HRepwzziS83JmQa4kESHcGk1Qsq5UvjYvmTeejS4Sc5EGbejGfffJg5rTekmprjTfn2/dDOowz/ewx7OGqqknmYSjqGJyOKqkOUq2VrUcbA9rBvn74zu6LPH8hZxaosVAVV8E5gau+sF003EhXYbCuk7/m/RnXIHMu+N36Ut/9zhcLWqC0XkP4o1z2enDNfi0fhToK02vJKXS5sPhdiclGmuU7URyRamrV5/XaQ647aW3jb2TnoiJ1RisayukDB4DqsrAy43hlC+EvoOlsA0zvz+vctovXLYhHodsfqQXLuXwAD2TnG3vqzbevEH3CNrRPWvuaRaQNLl9kP8E8BDnX1SjHzKURQYr0c2d7xiPiy+9Dc9R2fMCWLraP X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(8121501046)(5005006)(13016025)(13018025)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93004095)(6055026)(6041248)(20161123562025)(20161123560025)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0301MB1243; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0301MB1243; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB1243; 4:7GjVRxF4qi/52MvhpCXqmgslMO1oi/n1iAlQuJwt?= =?us-ascii?Q?X1190Klgvvj7jr93hsYGP5jKQyxKmG6XpWkIgQyjt+YSdkIYESR3iYciaYrj?= =?us-ascii?Q?lPvORYp0LjP6LPCwlBk89AU5ibE/OODeX/2vF1zRnX5u/pRJp1oR/jgOVTKq?= =?us-ascii?Q?BXo3UdWmhjtZWr+KgFZQQqlh2RCQk63SOMX+05b9lrj5Cy8JhQJE4q71JZdl?= =?us-ascii?Q?O9T0+m1eeOj6M7J8bdlBCIO3no7DD05ykI0O29jZzz6teSlUF8JnTU6oTznY?= =?us-ascii?Q?MqqBvlXurwTL5yvA8Rhf8AErKSsnqe9v4p/mbPnHF22I+8uvEfPkXHYJ7Qfh?= =?us-ascii?Q?AYWviKGgH476bmb79iv+MQaAvy+YsZ7DyaBUf+Jd09sDHkNsORlCao6NbEZq?= =?us-ascii?Q?Iw+pyLxOb0kD0p9nwGBhw+D9XAb9of8LEEMPZ9r7+qH/tUwjQ3BuX4SnE5HD?= =?us-ascii?Q?SMilP20naX4LhDluwjM1A2xHEFuXSVQqx4CJ12QXKLE0oa+MP935nCVscvcK?= =?us-ascii?Q?EAFQJ9gph7dPpU0prh34p30xh+gUql8kGipU6/jB4QsnE0jwqufgCau/e4Qi?= =?us-ascii?Q?+zFrmrjELXGWc69KqPfU50DSmAjpmrK2kiqz4+pPG5GpKiWzmcDnFA5kWubS?= =?us-ascii?Q?by8OGsLcfOJkaVsDzfsDQo1YzfMLx5z0hje+BeGlLnlWKcNCO0x+djsAbuye?= =?us-ascii?Q?jtdvSu1EoTygXc5KAgqOv7d6aM6WGR7CwPcbvFSRaw8+m8b9T8OI3K1P1OEC?= =?us-ascii?Q?CO28EZZ3w9EtBBEWanyIPWZkIRlswuNsnXGFco2iilLSATNHdhn+QhjUPyD2?= =?us-ascii?Q?e7r6Ihwo9QBcbx8sVnSyBvcoC+D9txfcAN8QenlKPtnZjvAAcrgw6J/ZpODe?= =?us-ascii?Q?++4BiXYUN8CnyPG4xrZsO0B+6dqmPSckg3fvLiFzem27yym+wBBqhEQ6Ekqf?= =?us-ascii?Q?UXz4/uu4rtuAmW5NczJaVISJYG6aduAqh13rqG/96r4GSYhTrhzaOHrj0HMI?= =?us-ascii?Q?yVxKq6isRS07gusz9w40q4wSRZpc17+7ZJc6gCr3Svno8nSr2tHohGT8Qkgt?= =?us-ascii?Q?RAbwsJ7hrolgNnHchjj93oiwwdbz+vDTu2IwWqNsohLCiqnbLOrgk/VpWmvY?= =?us-ascii?Q?xgmOIcIZ2A4RfEt9Aod9LaSBKX5DfWVU5qqMrxPekE6+VT7CHBcmYAA7Fab6?= =?us-ascii?Q?eDLbRumM/LHrooFhmO9Ei/ZnqnJ422w7adAdAq+fjiIFz97hiNzZhXJJ6wiL?= =?us-ascii?Q?15mi4B4WG6+/esLwv++UXmkgN4ZjmXd8coYTfofJ?= X-Forefront-PRVS: 033857D0BD X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB1243; 23:2iz4TIAH7VKIYE6xXZKnCKmtdTFINGFkvuHeU1n?= =?us-ascii?Q?YDICxJij4Wy/YRAPfC9a61iw+kazSIEhoIOOXPC6B3+S1aU4PN2TVV0XCbLV?= =?us-ascii?Q?syAIONxMl9qyh+KlkiRchKbU754RE4ePKs4iZ4TcH2G9UaxhsD006gZkaLab?= =?us-ascii?Q?+IHB0EMp8JE40HqVwG3acuRB+EccPG/IcKgqrgYW33JxZkuyXREeZiEWlyFL?= =?us-ascii?Q?UhLAhAMovV0qgckqkDgJKtbu7D64Fsv9kI2pdmOM2prYfV+TRKOJcs7tJqma?= =?us-ascii?Q?jOJAT9EtsUmvGvH65rcP49rQSyjhcybDJ9QK+vKUFq+gKTF+9PSiBUdXlofu?= =?us-ascii?Q?BLeyM6P6ohsru52d5bmleCLS9fS5JxRwP55hIkJFf9VtHz2G5Hdz4O2745m9?= =?us-ascii?Q?2wuLmvSTT8pWxW51Xqh5/b94UEneP9LGw1CimmQRX6H0WVxFe8YeY5nPNoFB?= =?us-ascii?Q?S6JBP2GCkJqMDDhTAC2aAQvqD/urLHq4Rxfa7O97ZMEQMriIhH2pr+jI3dfV?= =?us-ascii?Q?Dr5DDdzLBOsmA/fzBmHavTaI/o6v7rlSwu6weH4uIaKgCryYCRXbfylC21zD?= =?us-ascii?Q?K2xssa/KiWlv/+z5TPuU7mib5RYEqZAJ0dr3nKL8uL27hJLkH/U9k81iEsZI?= =?us-ascii?Q?Vjtvk07ylC1NSNrWBi2B+yJhJCweRunaCHWF4wAI+KcMi1Gzi1DuOFaCx4rQ?= =?us-ascii?Q?BdBuUlH6vq4OigJAhI6b15FL1iq5cxerbO1BbC/pNKjUCEDzVbvouCw/Kq6H?= =?us-ascii?Q?9BdwjQbKPu86lrsTuKH//6qW7NcQAN7uU81a8WobePEpxsaHeivb6ORHlMfg?= =?us-ascii?Q?1bXPRyx+gCbfotVKyadOkIMPFD/5b8m0VmJdsErj47NA+8ifbTDMPgbDO/fY?= =?us-ascii?Q?eRi5tAwcqqAXdRcycd0ks6ICF+F5zuZSPzA2Hw5DrhvxV27rasldN97Vh4mP?= =?us-ascii?Q?dRw20yz2e5zwasUFVT+u5ylbnH5fHC84Y36ivVXY13E3pEdbw1Rk0QXhDPOb?= =?us-ascii?Q?2LhOWLby7xap6bYVQ2PjoGUgLvBkh7dE0rh7HZte1J/vOsp4sCLe0WkmXmrX?= =?us-ascii?Q?z14VrR9C7jXv0YFUQ6SGmFVwoAMkvXzEnTj3csPrUxh0q0o26rUWJg2BiJm0?= =?us-ascii?Q?Vg1WDcm2BOT9l7uOQRfUMnNEVDK2tAx42FGE3xBrl63jwP+x+h9L2cC3JJE+?= =?us-ascii?Q?TiI1ketlS4EH8KrD7iVrGODMESqgaEqq+2pE0IKAEl4Ep1u/GWPqrIIQt9KU?= =?us-ascii?Q?/5z7ISu4El+FSd4oEhTgv1RUKAZe1F5L11lSWHO/0uC0c6Wi5F5QZtrt2KIF?= =?us-ascii?Q?y5UHY6JD+mtfTCAV/qJf0UaHLrcTfFik2bblCPt890+sbYPjEkPBlP95kIt3?= =?us-ascii?Q?VzKRaBfGPexoBHAx8VrB5Z6oqO1w=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 6:J+Zf/b4q9M1WuDEtS/GUc7J1GKLPQr+YEGhN1LWOl1iS70vrImF1nfZRR9GAz5aruLnW/JLCtfGxPyRhlFiXTXBojrmg23tPOo76xzgfq0AACmwe8BVpPm2JSlx3b8rT6mOqkEF70+hwYAA2qB3tYBXlN2Xj6TwbiBHYDgXxuV/4ks2HHDSAt5Qy4npri23aLrNzijqTndR9Buw3vpoC9BI0nyxDyOYHlqvj5opTN0b9BQHmWDCh2U1u4X4aa7tIr5mzloVI3MDM8dnJDOtSNmXsiIOHRIa7nzVgjLFa0jy4ujScMSJ61M9ARxyEdqbdkhmTsnpHjz5J/8zgH3wQcg0LHp05yBikG0zF0RrgbLY11pcXsPDNmOomS7yw14snQzvY4/+Yspp9B9+pXR3W7az2DUFxT5QPKyWhMVk8guJyVecI1Lphj1HyiiwNC71F+mg+bZ0MLlBytjewJh4sxhGMuQzEm461Uxg1k1QkPIkA5Oh2Xh/0edDNqS/aAGLq7c42gaCPkItP3jdJXUOe7lKQS/nYdnLmoilV4QUD9tc= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 5:NJZ6ZbyEpz/IDrQyZ1wnzsXiSLLp3VBiGe4jqV2ytKWoia1cfJuD8xMZceuqdIE/m1xeLmUaij1803mQL8WqhbLuEvXtcvMssZ4UMXvO9bsxHtQORcueYjmRFPmlIYR73MrJ7lSrGm4kphORFV6Y8L2fwF0v0JVe53QWjajcl0HdjJFSdsaebTMKd+C/InD/hqZspsXo9Z7qUoEpbKQOf9xMSODQjZxwVTJesxBjIgZW8lOJln9xoqMBRKBy28OBk/0ib/V8hppqfh+750J0guXxdu6zf6lMQr9EKlC1fFrcBkK0999+QThjU2qUzWcryAYRy9QucxOHwfAWs23utsebRfwpg4FJs2N1dZoFPjAhJ/JmenlaVvQ7NbSlbdaW8Uw+I2xN69QwMHZHe+ZE5xfKjKfFdkeb3gVuI7rYQmgIDyJPgZLujHDbIquYj23+aNXIw6eWXgShpMMw+nMHoekyaNXc6wF5sqTwpn9EvHpyKuKad3wGPo2hJBV8oRcb; 24:I5Vv1aY7rlmS6AOXmI4p3V4wIKXyHQfr3b/No9+YZleeISQM6OQxTg6MNQZZARFgdspT1BMn0racaMjpJS2Fo5oXaFccC60aJGuJzeMqzQM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB1243; 7:DvQ4COIcsdDcUeKSZ5OiuO9vpRcgPWQh9Yv0JL2hR+3YKiexlbqD7vMUvU4rV9z+4axXyjQX4g97HPtbe0ZRW6Cv8WHQo9IMzWzs6www6FgdeWXgTT0GI3BJVLGQHBRXp4VY1+2133wuu1/VsPl7rZ5R9w+RAjD4Zgmmj/vywMAvB9sFnd8up71bRlA4fax2qLkhVxktvC1bHU4kBBiEgI1dT2/rI3omEvlAuwchNp8enIEhK/eeAP9N2Aqx9DWzs32B1OMgpDipiyTSiKt2ftDmzfWyKIch6KmF/ZNXK+1KnsAJsQwK5plxfGF+rxBS5pCT9QIEM8TIlRC1paef7Q== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2017 17:11:17.2914 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0301MB1243 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 17:11:20 -0000 Further to this: I've now tested FreeBSD 10.3-RELEASE-p19 and FreeBSD 11.0-STABLE #0 r307264= M, both of which suffer the same degraded performance. test$ uname -a FreeBSD xyz.com 10.3-RELEASE-p19 FreeBSD 10.3-RELEASE-p19 #0 r319904M: Tue = Jun 13 12:38:29 MDT 2017 aaronc@WFT:XYZ amd64 test$ uptime 10:15AM up 21:09, 2 users, load averages: 1.00, 1.14, 1.30 test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 200.379127 secs (83727363 bytes/sec) After reboot: test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 23.213040 secs (722749623 bytes/sec) Same Intel Xeon E31240 with 8GB ram and 2x Samsung 850 Pro 256GB SSDs as be= fore. This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Wed Jun 14 17:20:22 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0CD9BEFEF4 for ; Wed, 14 Jun 2017 17:20:22 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C0D4720B8 for ; Wed, 14 Jun 2017 17:20:22 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id C7C8313D72 for ; Wed, 14 Jun 2017 17:20:20 +0000 (UTC) Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: freebsd-hackers@freebsd.org References: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> From: Allan Jude Message-ID: <42e7a692-974b-5b3f-a0a9-017850511e19@freebsd.org> Date: Wed, 14 Jun 2017 13:20:17 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6vke97k0VGGJ73PVLCFaUL9kC9UxJ5DhJ" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 17:20:22 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --6vke97k0VGGJ73PVLCFaUL9kC9UxJ5DhJ Content-Type: multipart/mixed; boundary="H5c9v3bQIxfLRrbnbmqa2qC1WPamANHLR"; protected-headers="v1" From: Allan Jude To: freebsd-hackers@freebsd.org Message-ID: <42e7a692-974b-5b3f-a0a9-017850511e19@freebsd.org> Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours References: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> In-Reply-To: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> --H5c9v3bQIxfLRrbnbmqa2qC1WPamANHLR Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2017-06-14 13:11, Caza, Aaron wrote: > Further to this: >=20 > I've now tested FreeBSD 10.3-RELEASE-p19 and FreeBSD 11.0-STABLE #0 r30= 7264M, both of which suffer the same degraded performance. >=20 > test$ uname -a > FreeBSD xyz.com 10.3-RELEASE-p19 FreeBSD 10.3-RELEASE-p19 #0 r319904M: = Tue Jun 13 12:38:29 MDT 2017 aaronc@WFT:XYZ amd64 > test$ uptime > 10:15AM up 21:09, 2 users, load averages: 1.00, 1.14, 1.30 > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > 16000+0 records in > 16000+0 records out > 16777216000 bytes transferred in 200.379127 secs (83727363 bytes/sec) >=20 > After reboot: > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > 16000+0 records in > 16000+0 records out > 16777216000 bytes transferred in 23.213040 secs (722749623 bytes/sec) >=20 > Same Intel Xeon E31240 with 8GB ram and 2x Samsung 850 Pro 256GB SSDs a= s before. >=20 > This message may contain confidential and privileged information. If it= has been sent to you in error, please reply to advise the sender of the = error and then immediately delete it. If you are not the intended recipie= nt, do not read, copy, disclose or otherwise use this message. The sender= disclaims any liability for such unauthorized use. PLEASE NOTE that all = incoming e-mails sent to Weatherford e-mail accounts will be archived and= may be scanned by us and/or by external service providers to detect and = prevent threats to our systems, investigate illegal or inappropriate beha= vior, and/or eliminate unsolicited promotional e-mails (spam). This proce= ss could result in deletion of a legitimate e-mail before it is read by i= ts intended recipient at our organization. Moreover, based on the scannin= g results, the full text of e-mails and attachments may be made available= to Weatherford security and other personnel for review and appropriate a= ction. If you have any concerns about this process, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 Can you do the same test, but grab the memory lines from top(1) before and after each of those two runs. I am guessing the ARC is being squeezed out by PostgreSQL, because you have so little RAM. --=20 Allan Jude --H5c9v3bQIxfLRrbnbmqa2qC1WPamANHLR-- --6vke97k0VGGJ73PVLCFaUL9kC9UxJ5DhJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJZQXBUAAoJEBmVNT4SmAt+5mgP+gNMFFe1O735f7BUOyEYasMc Ixde4g0Q4lBXvIx2O6CaHTP9C4zWDS955S8UAXNmJ4CY5JrlQ9o1veJ8aTlj3geu uo5tHrm36W4ZbNKnTUbn+oqGtbccT1O2rhJJ9UmS5cfOp5nd8/iRIV6/iVZRJCDG qKhIFvDr+cDth2xkVLMmvUYqoNtQuD4lF9PTVkuOCNjP9zSICj5dtjL9et1Tujsf NgKBvnpNevJ3h1nU+Rh4Wev0mhECBWsrL1SkWitfIZwPH6Mhh16rJnw8uxn48Nn7 Egyb6/PXWs6Bg7IflACptzfGiec54D9RLsjD08WoRhcCu7Bodoth4We3KR+oBIDv yhMNK2GYmAFT42aaq8AP4oEHUzYoBWuRFdysVbDCsuVq8uiLjPXACaA9ZDAGgUvP o2AtVu9JZdEj2D0WqYAB6RCNWqWKnKyCuGs7/c5xcLat4ey3D14Z4hGO5XM20sLf XgPv0brB2XweujSH0dQkBRtkGosNsQAE85gjF1/X5/+qSrUzRdhrLnjqrl2zsLNg YdEfyGdmmmANXKl8+c+QAvri0yYbk/w7gsAAy/O1G4eeLPFukLcYosgpXjhqnEVx biGRpg7j3PwluQQduugoXMLdgrHZ4jH0Q7Sm2gsjlbV8zeWjhH6y7WDc4Y/v7D2o b10A9VoZaSu35PayCAA1 =rnGR -----END PGP SIGNATURE----- --6vke97k0VGGJ73PVLCFaUL9kC9UxJ5DhJ-- From owner-freebsd-hackers@freebsd.org Wed Jun 14 17:56:53 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4F16BF09A5 for ; Wed, 14 Jun 2017 17:56:53 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC8EB7331F for ; Wed, 14 Jun 2017 17:56:53 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from lm0.local (rrcs-96-10-213-43.midsouth.biz.rr.com [96.10.213.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id CD886239431; Wed, 14 Jun 2017 13:56:51 -0400 (EDT) Subject: Re: crash under load in GENERIC To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170614150144.GP2088@kib.kiev.ua> From: Nikolai Lifanov Message-ID: Date: Wed, 14 Jun 2017 13:56:42 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <20170614150144.GP2088@kib.kiev.ua> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="DlMXeKuEVaRmBOW85Kh7iftj2s8xHmsfj" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 17:56:53 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DlMXeKuEVaRmBOW85Kh7iftj2s8xHmsfj Content-Type: multipart/mixed; boundary="V75GkJ5P9X1nKSG7vgrUX4WQsNagrpVF7"; protected-headers="v1" From: Nikolai Lifanov To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Message-ID: Subject: Re: crash under load in GENERIC References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170614150144.GP2088@kib.kiev.ua> In-Reply-To: <20170614150144.GP2088@kib.kiev.ua> --V75GkJ5P9X1nKSG7vgrUX4WQsNagrpVF7 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 6/14/17 11:01 AM, Konstantin Belousov wrote: > On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: >> Hi! >> >> I haven't nailed it to a specific commit, but some time after >> head-319667, my package builder box crashes when running Poudriere. >> The earliest version that I can confirm crashes is head-319735. > Do you see crashes past r319754 ? If yes, try reverting r319722+r319754= =2E >=20 I do see crashes past r319754. I'm trying this with r319722 and r319754 reverted, but it will take a couple of hours to get a package repo to the point during which it crashes. - Nikolai --V75GkJ5P9X1nKSG7vgrUX4WQsNagrpVF7-- --DlMXeKuEVaRmBOW85Kh7iftj2s8xHmsfj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJIBAEBCAAyFiEE5oT6TcuaWvG5gtjzZ6sv56ecR0UFAllBeNoUHGxpZmFub3ZA ZnJlZWJzZC5vcmcACgkQZ6sv56ecR0XO8w//UADekTzvX9XxMuvYu+1KxAbrAAZA AqnqACZ4YK0b7xMTgxIj5qumX3JuZHNwDkuyMnb7RubEszzz1ciycQ2EahbG1wSR KNek1inh1xiAYrbA0JHUj3CWPUWV/1mFps+n86mm3+Wts1GandJjyBTDtpewgRh1 SxNZPoYC10rATdB4k9iXkqobkqJhLZ+LDVrsH8aHKu+5b2edsuog/UIVvQoVrt5Q rWEdJZ++/jxE0CNpl/fpGgpFLb0YgTdH9vESA4A3JpK8FVIRpG1rQYhYwZuWbVwX rpaKs2X1vzAcnzLUsu3YmShPFCpY9khFKTF5mUcoebStw2MkHmDIdBKsYZRm7ZlA NVzjxVTwo3jii5UG96ggAUoJwPW7W28lcJme/XS+hySHRB/dh7C5Vok6y9N+qyKM ZRIYGOqr/6qFAt4X0kpAvrdz6+gdE6CXul3V0JnOiQuUgKHswrcm7oJ+O717YYj4 yMfLmtU7CCHIcgvSvdY0fS+lCF201xnIvbc3aUU/qG7EYWjfgTIRmwY330OrG2qL AiWmTbYsjB5phFdRnP1nI3ysyYflc0T2CvmX4i/Uwe+xuBTSe7OlkFHwJhNCeWIq MM5iBiNb1xKXORXOSyr/pu6BukUj3teybCa84s3e9cPB3Ddl/EtTpOysPIl8LaM4 cDBw7C1yj+ZJc8k= =OuTR -----END PGP SIGNATURE----- --DlMXeKuEVaRmBOW85Kh7iftj2s8xHmsfj-- From owner-freebsd-hackers@freebsd.org Wed Jun 14 18:10:31 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78D9EBF0BD3 for ; Wed, 14 Jun 2017 18:10:31 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0080.outbound.protection.outlook.com [104.47.42.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A14E737A6; Wed, 14 Jun 2017 18:10:30 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=or327Q7torjcNZRQRxQ0spQxVmkw9ZqMt8+hTO+uiks=; b=DMzj60KJ2XNUCRXzqnXXUI0avdmGuLFUJd+hCIRFDx//4MqZDDpDjLZSghQsEqnlACJb+exri/IMUAYm9jqgOshFqfV5HTsTSGrOADM4yP3a5AJTEQcdLSCVgSKsOnLKsziEVbdGY5T7sTkzz1v7EByJw9QjYC3EI4VD/8Jh7Hw= Received: from DM5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:3:118::27) by DM2PR0301MB1245.namprd03.prod.outlook.com (2a01:111:e400:5020::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Wed, 14 Jun 2017 18:10:29 +0000 Received: from BL2FFO11FD021.protection.gbl (2a01:111:f400:7c09::120) by DM5PR03CA0017.outlook.office365.com (2603:10b6:3:118::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Wed, 14 Jun 2017 18:10:29 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BL2FFO11FD021.mail.protection.outlook.com (10.173.161.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Wed, 14 Jun 2017 18:10:29 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Wed, 14 Jun 2017 18:10:28 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.017; Wed, 14 Jun 2017 18:10:28 +0000 From: "Caza, Aaron" To: Allan Jude , "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLlL01Bja9RgR6rT826WHBpLYHdbgAAzCeAAABmhaA= Date: Wed, 14 Jun 2017 18:10:27 +0000 Message-ID: References: <5fb864560929410bafc6a233d374c620@DM2PR58MB013.032d.mgd.msft.net> <42e7a692-974b-5b3f-a0a9-017850511e19@freebsd.org> In-Reply-To: <42e7a692-974b-5b3f-a0a9-017850511e19@freebsd.org> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.205.69] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB013:|DM2PR0301MB1245: X-MS-Office365-Filtering-Correlation-Id: ad408a4c-ce58-4c8d-c688-08d4b350a3b8 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB013.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39860400002)(39840400002)(39850400002)(39450400003)(39400400002)(39410400002)(2980300002)(438002)(377454003)(199003)(24454002)(377424004)(189002)(13464003)(9170700003)(229853002)(3846002)(8936002)(5660300001)(86362001)(106466001)(5890100001)(6116002)(2900100001)(102836003)(50466002)(305945005)(24736003)(7736002)(2501003)(478600001)(23676002)(50986999)(33646002)(54356999)(86146001)(2906002)(8676002)(7696004)(6246003)(9686003)(108616004)(76176999)(966005)(38730400002)(6306002)(55016002)(53936002)(22756006)(53546009)(66066001)(2950100002)(42882006)(47776003)(81166006)(72206003)(450100002)(189998001)(356003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0301MB1245; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; A:0; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD021; 1:QdjPyYo+exsbJaIFOCWB6ksUSefoUPtEynDj62M2TSyPD24vkDT/PG5IOCAUgkZsesnc46rs47LssHX1vB+2W/oMbNI6Ytz6i1KsLr0SFn0eENqpiHw1DH4g2ByqviNGbLQ/7hRLa5th+bGX+Y1AhmSbkt9BUdRVSRFGpMndr8Jhl8mhfQKWO90z8n6Hh0lcSOk6QvU0G/8CywuAJDkCWaLgU0ssrslOgizjmN5zn4nJ2EOFxcsNO8HI5pDwWA5o8P7/qRYWpuUPsbF6nA7BFILVQ7m/CbMjBsGdhx6tHRe2Ri2P4xR9G1cELItt0LgMe9C672YNFRxhhRPIJDmlzEP7KGKMgQtU1QI9Ega+x3q3WSgeurKyWoshq2yF1WYmNtSiKTKmdjVIJKHGkluKSHHtiUCiPPdsac4pK+Pun5vSy6OZlrC+5oCUydbUseIYYIgTyPJMN45jZLrGUVAPRKbpRC1Mnep0V/N60bjns+c4BZ3KbhgZElPq1qVLzJAKRKsdzE8k2K74+C+rdCj1VERFVk1VCjbylyZyRyfx3jc= X-CrossPremisesHeadersPromoted: BL2FFO11FD021.protection.gbl X-CrossPremisesHeadersFiltered: BL2FFO11FD021.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:DM2PR0301MB1245; X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 3:bf7dvM2pVqFRkCvqeruO81VcxUMNRkv0r5OHKUcwGtNuIcDyC/udGbrYRgAvC6GVi0ZjKtdHW9E+/mwksI8Lc3L8mwlArEJf2HXKTamBs2FnHK6IZRSchvoQ6yHZgvClGpcPTYH1zp7KggZdNHr8CuRzlDsHzhd5SYyAFZWK+MNsn3dcMGTmndE9Vi+mSnt2s8MgKkWiNF6+/wNcGaXpb9POKDvYq0AqbGNnyKdsmL9L0S9r8Pc/0+el2w/rMcmniabBwIlZ8gLtAe3Lkozka0XBC/AXHNl/v2vBB6cCk2pzqf3D8luSsbnLA8Dx5am7/MhBSISyxa2SgP6OBbCuKDn1qVRfhhEzJyIhzYBTH6dZheEwe2mTzFgk1BHpE9BKy4gThTtbljK1Yds5VrHTbuTYxwAMO4i/6eSQwZQbKuxFS9TmrQT1F/lxLTOWS9P5B5W7k7YU0VqgimOmXJuCZN0eGyLoKU+ZIJHANzNRveU= X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 25:EMszWf4MB+L1jk+NXdanRfcFOUgT1xilCd/5SbjSd4KdK2vaCaCkaAeX9Zb9+GiwPTWhfeJVK8Jcqbg37zfC316/+D8zjih9k+xzKu72gaRq1urMSjjzuOnVgiZstdnnKyjefZ/AFZ5rZs2/bnhj7GxRmxJDQr6Bpdzp7ASWrYD67llMev+cWljSdmMcteQjbwfGrEiz6KK2FDxcmfacPEVX+dD1EMLrLGQlNTljLTR01sJ2cH/6xn4geD3MfGhWk40mwkmrR+avG3OIo3EveuZOXhzB9xD1FkRvgM9mEm+K4jYL18l/6hoPX0pQVuLn+JPlZ/uO37bsws/GDzbFldhsBh8ecKwVE/i6cZBjzzYG0au0CW7xFLaMeRRq5HZj6j8QGnsje0+1GIOEaHnfr+MPvHLNHJFItz50tscKq34dLZL1HqmJDeUNeQoUZNWgEJfx5IDKnuaa2yb7WAp2bGdQCPR0IphRFSUSdk3kUTE=; 31:1kuCM5gbd7J8+1fK5ffKZJvH9bP1v7Hrgb2ryEsRr3cx8pWUrN5RaWgmlmparkM8vqIwJrJU7YRwOqpQFRui+uKKf7DTyhmtwleGOpMeSu+rIGcKb0rdsa6uaUwPPv/hgwjZBoE6WJXhlhVVGy+xHRkw2raDz6+Ombi5h24gbyafGsOYCidMScDHrmt73V0HJceG8LoGB5yTvBya+RbM6c0Ovh/PigjiHR8xS9jpAgC5P55eUhM6yz0TGrZ/vOEBTCn3Olai2iwVdVIhcR0P6g== X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 20:kjTv0Whos5YDQcmyaUnTLleARpUZGo9rIhvxfpNxLc1AguBl2gyyh6rm3XrB5mH4tfEJuarauaBtL1ykKcCf2l/SxtTyzXjp0bnIDO9xihWE2dJ3KnnmJg22K7uQHe9+TLX6Ae0sW+myOoiKExmv9Ff2BMUn/E/I2ZHcSGeyCZs03fJehmHNij4KIfCamg2FWvbh1L3C7z7wMaiV3iVqvoDPjYCSWIjYT4Jkn/RtTve69P2eiJpdzghf5mIQsEHRmE2uQPTWkElNZ6odE1m3Rx7w6Drh7klv36wBsQlRS7RwRWmGhCozZHCiUOgCc5q79jzhVhCoQnYNW+/qz74CZxDx7ayB8tPGGJOH6QhPT7gH+/Gmr6Tr4S1EZgwUyKoJHDSJfSdChdpD5odw3AbpONNuti6m2h1ZRMBy1IB0ha7y0qLwaMf6T7JdZNf5lDi2V/O2MtgqvHyULsrv5RA4jfZ/AMlsLCAtxvThdq1WbPEanIKAZmR6cYuYSaY+F1/7 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705)(75325880899374); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(93006095)(93004095)(100000703101)(100105400095)(10201501046)(3002001)(6055026)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123564025)(20161123560025)(20161123562025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM2PR0301MB1245; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM2PR0301MB1245; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTJQUjAzMDFNQjEyNDU7NDpPeUk1YzhZQk43aFYxOTVEa214aFp3eXhN?= =?utf-8?B?cG05ZFRSckpnRmpXalhUc3h0QjFac1ZRVWdVWDVHRC9aTzlMZFFBeGZJUzhL?= =?utf-8?B?SHJCRjlYd1U3T0xjaXhhL0NmMHRUZFYvbWY4YlFYcHU3UVBiNHJ5OVlHNGxT?= =?utf-8?B?Mkwyckd2Wm81dTlxdXo0VmIyeHgxK3V2TFVmN0VlMFJUK3N4T0lrSkFuTk9K?= =?utf-8?B?ck5scG8yV1hNZmhZb0o1S0d4bEFHSWkzc1BxNTBiWDIrQVRKMVpDTGUzM2h6?= =?utf-8?B?M1pkY0VYMXVUVkgzYm40QjVTM29KOXpvbVZxQXlqV25OSWhZUjVHci9VaXpG?= =?utf-8?B?dzYwWDAyNFpxeDBCaVB5YVhweE10NlNkMXF3cFQwUGsrdUw2ejJadW5ST3lW?= =?utf-8?B?VW9uaTZiS2RmME5GWVZxNVpXMERsVHpGM0tHUnc3dXZoMkliVEl1VVpCSU9B?= =?utf-8?B?aTVtVllkeDV3ZzVmZmlBRFdFcEVscHpRazdLeW5DQmtMVGpxSkpqcVpOZkNV?= =?utf-8?B?OGZXc21Dd2VDa1hZNmxSVDJrTjhReXdTbnFUczR6WFMwdkFZeUc1NlNBaTN5?= =?utf-8?B?U29XSjI5cGZvV04vb3FUNG5YZWU0RXdBWHBkRk83NmdoM0tQNlg5WG9xRllr?= =?utf-8?B?WE5HTitlYWxIU01HSUZEdlhxOVduZ2R6dzdZbjEyRW9SM2EvS3BFM0lvWFlr?= =?utf-8?B?Q2tlRUY1T3g5cUZPcUo2dFBJTUNSNHpQcCthTVJUYjgwTXJUMENFekU3dllX?= =?utf-8?B?aDUxanBzWDlFZGtkUXArbWNVYjBlK3pRVFAyTHVYRC9HQlVTMmpxczdrVmV2?= =?utf-8?B?MExvTWNuZUNjVDlMaHZQUkRwam9FMEEzSWFDeWMwYS9lWkNhNjRpN2FFMTdV?= =?utf-8?B?ajE5RDIvK1BoQmczQ3JSUysvcVZHZ0o1cjdDSjJnRkovM1VBOTB1ZGVQdUVs?= =?utf-8?B?YktQSlVTZzVITFhmTW5jL3FxTTNiTzh6OTgvdG5qb3N6VTlxTE9TWmZvcVp5?= =?utf-8?B?eCtHQUc1WmdIUEtPbFRoQ3NKYzNhUm1pYTlrTDVIbUZFM2tCaEZNS0pDUDg4?= =?utf-8?B?WXFjY0pZTWdWM1JNWTl5Q1RQdzBRUDNDeXZ4WDFZdXUzWHRUQ0ZqSk1LL0RM?= =?utf-8?B?b1NQOHFXajJ4ZWlwQ3JXd0hsS2s2aHl0Wlp4bUlFUE5DZjBDMjBadzM4a3FU?= =?utf-8?B?T00wT2ltMDhYVFo3TzJUdk1LTjJ3RmJ4WDBMblkyRGl4WTR0WE1KZnVKbFVY?= =?utf-8?B?UTBEM0k0RXhVUysvdy9kV0xLaVlzMXNiMDhZYWdELzF1WjZEd1MwNXJCMGQw?= =?utf-8?B?QTgxd0FmaVJIMS9YcE5SUTBZam5YUkFyK0VBTFR1QkVtL0U5RUZqWkdaY1Bw?= =?utf-8?B?c0h6S0t1VlVUaDM1VVhVcXFldmdFK3VZUXJpaUpZeEVPRFhKdEU5V2xrYURR?= =?utf-8?B?R2ZBdjJGdHNXSUZlRExEdDVVQ2pEZVdON0dUK3VGR3V4bUgwZGFiV0Z2dHFR?= =?utf-8?B?cEQ4eS9DQ1pXT0RreFl6cExKc0R4L1g1UGRBckhoTEtKMDd4VXduZjMxYkNu?= =?utf-8?B?K2RudDdQL0trUXpiN29UNlZSVFVRS2ljNEJZN2ZvVk5PMzQwZEFzUmRtbGJl?= =?utf-8?B?NVpjZmVHOS9EbU1CWWh2cDlGYURGVW1JRFFQYW0vUE1NOFlNN0RCd1I2SkVw?= =?utf-8?B?MFZnUzNCejkwdC9HQ2NQUU83L3ZPY2lUY0NRbDBaaVArVStuQkZhRFVaMDNE?= =?utf-8?B?WkV2QWxnak1Xa0NMVDF4U21wRGtOUE5lcVZRUDg3eEdHQlkvUWd6OTl6eWtW?= =?utf-8?Q?QmfpYdA88GEmY+?= X-Forefront-PRVS: 033857D0BD X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTJQUjAzMDFNQjEyNDU7MjM6b1duVXo5RTBING11NG5idDFhTm41V0hQ?= =?utf-8?B?ZWhnVER4UWRHU3FYUkRKbUtRbDRRbThXWElUWkhhR3hubkJwazl4SFZGM2Q4?= =?utf-8?B?R3dJencrZStBSUpNNWxITzY4Y2FOTjBpVy9td3JnU0ozY2V5YlNnZHQ3L3R4?= =?utf-8?B?ZWVKWkZlQVlpcUQrRWYwMzZqSnRkanVzOUUvWGJ4QyswTXdNYTJPYTVJT1U4?= =?utf-8?B?SXZOTmhLdjVuWWIrWFVzdXYzRGFOaEhGZk1FM2p4K2tRbEpRSk0vVnJMNFln?= =?utf-8?B?dGc1MmZYQU9aUzdvdlpBYVl3d1NTSmpPVVpsbllaalhPd0V1SC9RZVdZKzcz?= =?utf-8?B?RTBuMUc1eklMVlg0TEE5djgrMVQ3QWpxYWs2cWlNd1M2QUtRdTJGcnNvMWsy?= =?utf-8?B?bWRGVW5KQUZkUDNOSGNSUWRvN3B4UHoyeXltWVZrM01BWXFKeEZLYTBHSUtI?= =?utf-8?B?ZTdBTGVucGs3cVhwakZtVW5iSzQ0OGZxWEZGWjJJZG5TR0cyV1pYVmtSRTlq?= =?utf-8?B?NUFZdzJqaDRpNjAvQTZCVnZZc1kwcyt2NUxhOHdDdkZuMHZreVc4WmpqOGgv?= =?utf-8?B?aU5Xc2R4b0w1SktnaGFxSUJwNlBXemNVenRIWUVQTVltUGIvam1KQVA4R09M?= =?utf-8?B?UmdXcUdNUkNRUEdJSkRLRk95bTgvdmJPMW5pc2VwRE9FbWtva2R4a0U0TnNm?= =?utf-8?B?U0w0VFNybWVEY2lMR0N3MHQ2bVlSekxmcXJmYk1lUWJhZ1VhRU05dWt3Qmo3?= =?utf-8?B?QVhRNW9lck11cm8wZ3g1LzV0bWU4UWZKOEhxYmZRVFZJMXVJZmJKZ1QybElz?= =?utf-8?B?SXVLalR6K2F6QXlsMDZmeVRrZmZTTVdVdUdjRTVPVkREa3R1VU5RM1loVUlN?= =?utf-8?B?R0d6NTdCaFpMYXNXbzVHdnJETEhVbWpkL3FENW9xSlhiQm5hUzQ2T0NQajZt?= =?utf-8?B?TEg0cktOL3RHajdlWUxMbU5NTFZEOWcrNlpMOEpCa2xFOVBJQnZzOUxUYWhL?= =?utf-8?B?OFZYZi9zRGNXY3dWRkFNUmNkdUxGNGhDSVdmWDkzZEd1cUdHWGlKNkdEcHp6?= =?utf-8?B?alN5VHNoK2M1ekFOQ2dyYmlGS1BHVDFYY3YxbjQrbUVCNTZYbEtzd0NIQ2ky?= =?utf-8?B?dmozWVo5cVowUngzUHdxTTNUREMydTM0YjdqNi9UMWphSFpkUVlqTEhDSEww?= =?utf-8?B?enZMOUgvUGZSKzdhK2xnYWJJcGMwaWtQUkJnbFh4cGlNeXovaU5RcVVaTndP?= =?utf-8?B?Qm9iUDYyTURpb0xNZFI1L3JYMjVJR1pRdGoyekF1dUdvQXhGRHhoR1FOdGJX?= =?utf-8?B?NDR6WDRISUN2R0xUcCtVVldZOVNsb1FrSm9VNWI3dk8yc0RUc3RNQTFyek1L?= =?utf-8?B?Ni9UczZXbWYrZlhnQTJhbzJ6eGtNQkw3UWYvM21XUE5PMjdoYlFlWjI3QnpW?= =?utf-8?B?MTJzVnpRcUNEVTNESzF4TlZUSXBaR2VwSTUvMUllN2tVcDZWNGFBNTdNWEtE?= =?utf-8?B?cmFEdk9rbG5UcUk4bCsxZzRZZEZXL2hhb3daRy92YkFhT04yWlpILzNXM2Zl?= =?utf-8?B?TGcyWGpCa04xWUFvN3N0ditYSG5IbG8vUnYrMGs0eDlucldSV0dTSm8ycUVD?= =?utf-8?B?em1FNi9FWlZtU2w2VC95VzQxaGdXSnR1a1krT2VsR0VVZlFaQzdvVGhxaHlP?= =?utf-8?B?d0hXQktPTmUvSlFEL1ZPcnFrNHJNRHFYOW9GUzBpZGpQMFByM3c4ZGFWeHFl?= =?utf-8?B?Y2JBV2FFeTRtMzJJeFU2VUZrUXJXTjgxVXpEYXVpano1QkJjK2xHckZlYytF?= =?utf-8?B?Sm50NiswVGprektuNXYyUGRlM1ZYVU1kSHArazFaMWJaM2luSEg2R3Y5Tm0r?= =?utf-8?B?N0JNZGZPS0syRmNsRWlPMXNUdURRVmJwUUFWaDBHaEJ6VFB4WXFXYm1rVE52?= =?utf-8?Q?flJuA6MYsxk0VUNFtqXmYSrjXE71UluU=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 6:rFOSorajWXmtZrRdQMKd7TSCwxRj8wKXtyeJrnktfzke2Gka0tmAAdY+LSfQ0sWSaTe4s+1nwIo482WuodittzWH1hJnSsb0cCRzi+Be51mw8C/3rMmaESJG1so8rNz35d9szug0GlT+QvSET0QyjQWTOTkzkF24cgbIPR/BqM/ipKZ4YZaHm2n72yK4MJ+23aiq56bsynN2yPUeBw7swC9XTa0O6MRFdFQGFLATYlyAy41G05yv/m4w1cUBw2j3esiHPeK0kWpumhekHJkFf8PSVzAEsgy+fa0igZVi0EW5L3jMblkX1k/edmor4/e/z7Xz1BFQcgSmA55JGJZRs7GF3sy92E81ceGUkDzke9QmQPZKpzPEtVnpOhNGzcB1MdbnRMidvWISgV8u0lEqNLL/OkNOZ72TD9k6y4onH8iH39DV7XLMspKJ3C3XRF2Ni+DeC3FH0MaxA+mqEIwvsICh7MWICho3weVC8TKSgY3j+V2jrvfV9LfVAGbcQRGFC9sSa+e9iJrUQac8p5Hwzsc7plV2VpkY3kQrDmGuWtk= X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 5:pgvyUw79yP0/SYbhDkSXRMLzivytPsFY/zdmIkcblGmNvF2mVS82PymsrTH0j97EgeOOq31R0yL0ytR8J/3qL1R/mWbCPj/i1orq6uHp8gSNpH+MP+o5Cyp5O4qg36qUqQ7JDHDdeRrWIc8uNHOJo2I0usQSoa0LdPW86tLRYa1BabVQ2sAIcTEkx6AwTbVhu5dLMUo+uugmoHG42w3aLoJFLmqRuZdw4fDE4TaWKBNxPOaaLRtJoZ6WhdMD/RV0gSqDwRX676T20pzkcz7Ga8cmB0iVzrX7fPyZDTB3PnWh6mqRhTXpfkwDGyB0wgVlBRPhE0HAGsUcGFq+U0f4JWKLRUfNsVIpbiNvoxpQCg+NMIQLWO8+arAn+cBbn5RK7WoWxwv3NzKx2pIHfp3NeoN4UMtbMS6ODHOgX0mE/C4q2UtSRGcs/zTjhTnMGm3EW0cbHYuFdF/LsYPAbXN954KLXsGurbd9Kkej7BU7+IutfLhS4P3DKiFqv6Jrx9Se; 24:oQW1hjC2h7ouh7DW5CXv9R5JYdxrgp6OCagbrmK1bGigGRiHUQ8k1BCCvu3VeLvPIrbC5lXm5XByKS5Hlm1hA00jZOavFf19LKlAcBpTmoo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1245; 7:oCsd2cJoDYYJtQSLFOVCDhHeuXwNHjH9OT+cpMeaD1jqIpaF04RboxR4VYxSPGLDuxQhsPo9Km/Uga7exmT+zbdvBYGrvxUvRSRrBfkuNOAINfXyErVPGj3kDud1amCCaGgX5rh4LC4mrcIAWUDoVtTnIMD4dYLTSYgWAJtCMvXtrN+0KcT6U6t2zoMGzkRgKjsc4wEyqnFFnOI9by2+CgynYgxtUalsvLAtPO5852n+X4VK2+zWteGbUEhOVvy4yWPDQVoUD/ULGA51ihu9vjmCRe2nCZIM6lb1QIPqhEeyNJQ6Yt14On1MP/KD60N7x9SdIUC7j605vv4qHs29LA== X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2017 18:10:29.0853 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0301MB1245 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 18:10:31 -0000 PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBvd25lci1mcmVlYnNkLWhhY2tl cnNAZnJlZWJzZC5vcmcgW21haWx0bzpvd25lci1mcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmdd IE9uIEJlaGFsZiBPZiBBbGxhbiBKdWRlDQo+IFNlbnQ6IFdlZG5lc2RheSwgSnVuZSAxNCwgMjAx NyAxMToyMCBBTQ0KPiBUbzogZnJlZWJzZC1oYWNrZXJzQGZyZWVic2Qub3JnDQo+IFN1YmplY3Q6 IFtFWFRFUk5BTF0gUmU6IEZyZWVCU0QxMCBTdGFibGUgKyBaRlMgKyBQb3N0Z3JlU1FMICsgU1NE IHBlcmZvcm1hbmNlIGRyb3AgPCAyNCBob3Vycw0KPg0KPiBPbiAyMDE3LTA2LTE0IDEzOjExLCBD YXphLCBBYXJvbiB3cm90ZToNCj4gPiBGdXJ0aGVyIHRvIHRoaXM6DQo+ID4NCj4gPiBJJ3ZlIG5v dyB0ZXN0ZWQgRnJlZUJTRCAxMC4zLVJFTEVBU0UtcDE5IGFuZCBGcmVlQlNEIDExLjAtU1RBQkxF ICMwIHIzMDcyNjRNLCBib3RoIG9mIHdoaWNoIHN1ZmZlciB0aGUgc2FtZSBkZWdyYWRlZCBwZXJm b3JtYW5jZS4NCj4gPg0KPiA+IHRlc3QkIHVuYW1lIC1hDQo+ID4gRnJlZUJTRCB4eXouY29tIDEw LjMtUkVMRUFTRS1wMTkgRnJlZUJTRCAxMC4zLVJFTEVBU0UtcDE5ICMwIHIzMTk5MDRNOiBUdWUg SnVuIDEzIDEyOjM4OjI5IE1EVCAyMDE3ICAgICBhYXJvbmNAV0ZUOlhZWiAgYW1kNjQNCj4gPiB0 ZXN0JCB1cHRpbWUNCj4gPiAxMDoxNUFNICB1cCAyMTowOSwgMiB1c2VycywgbG9hZCBhdmVyYWdl czogMS4wMCwgMS4xNCwgMS4zMCB0ZXN0JCBkZA0KPiA+IGlmPS90ZXN0ZGIvdGVzdCBvZj0vZGV2 L251bGwgYnM9MW0NCj4gPiAxNjAwMCswIHJlY29yZHMgaW4NCj4gPiAxNjAwMCswIHJlY29yZHMg b3V0DQo+ID4gMTY3NzcyMTYwMDAgYnl0ZXMgdHJhbnNmZXJyZWQgaW4gMjAwLjM3OTEyNyBzZWNz ICg4MzcyNzM2MyBieXRlcy9zZWMpDQo+ID4NCj4gPiBBZnRlciByZWJvb3Q6DQo+ID4gdGVzdCQg ZGQgaWY9L3Rlc3RkYi90ZXN0IG9mPS9kZXYvbnVsbCBicz0xbQ0KPiA+IDE2MDAwKzAgcmVjb3Jk cyBpbg0KPiA+IDE2MDAwKzAgcmVjb3JkcyBvdXQNCj4gPiAxNjc3NzIxNjAwMCBieXRlcyB0cmFu c2ZlcnJlZCBpbiAyMy4yMTMwNDAgc2VjcyAoNzIyNzQ5NjIzIGJ5dGVzL3NlYykNCj4gPg0KPiA+ IFNhbWUgSW50ZWwgWGVvbiBFMzEyNDAgd2l0aCA4R0IgcmFtIGFuZCAyeCBTYW1zdW5nIDg1MCBQ cm8gMjU2R0IgU1NEcyBhcyBiZWZvcmUuDQo+ID4NCj4gPiBUaGlzIG1lc3NhZ2UgbWF5IGNvbnRh aW4gY29uZmlkZW50aWFsIGFuZCBwcml2aWxlZ2VkIGluZm9ybWF0aW9uLiBJZiBpdCBoYXMgYmVl biBzZW50IHRvIHlvdSBpbiBlcnJvciwgcGxlYXNlIHJlcGx5IHRvIGFkdmlzZSB0aGUgc2VuZGVy IG9mIHRoZSBlcnJvciBhbmQgdGhlbiBpbW1lZGlhdGVseSBkZWxldGUgaXQuIElmIHlvdSBhcmUg bm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQsIGRvIG5vdCByZWFkLCBjb3B5LCBkaXNjbG9zZSBv ciBvdGhlcndpc2UgdXNlIHRoaXMgbWVzc2FnZS4gVGhlIHNlbmRlciBkaXNjbGFpbXMgYW55IGxp YWJpbGl0eSBmb3Igc3VjaCB1bmF1dGhvcml6ZWQgdXNlLiA+ID4gUExFQVNFIE5PVEUgdGhhdCBh bGwgaW5jb21pbmcgZS1tYWlscyBzZW50IHRvIFdlYXRoZXJmb3JkIGUtbWFpbCBhY2NvdW50cyB3 aWxsIGJlIGFyY2hpdmVkIGFuZCBtYXkgYmUgc2Nhbm5lZCBieSB1cyBhbmQvb3IgYnkgZXh0ZXJu YWwgc2VydmljZSBwcm92aWRlcnMgdG8gZGV0ZWN0IGFuZCBwcmV2ZW50IHRocmVhdHMgdG8gb3Vy IHN5c3RlbXMsIGludmVzdGlnYXRlIGlsbGVnYWwgb3IgaW5hcHByb3ByaWF0ZSBiZWhhdmlvciwg YW5kL29yIGVsaW1pbmF0ZSB1bnNvbGljaXRlZCBwcm9tb3Rpb25hbCBlLW1haWxzIChzcGFtKS4g VGhpcyBwcm9jZXNzIGNvdWxkIHJlc3VsdCBpbiA+ID4gPiA+IGRlbGV0aW9uIG9mIGEgbGVnaXRp bWF0ZSBlLW1haWwgYmVmb3JlIGl0IGlzIHJlYWQgYnkgaXRzIGludGVuZGVkIHJlY2lwaWVudCBh dCBvdXIgb3JnYW5pemF0aW9uLiBNb3Jlb3ZlciwgYmFzZWQgb24gdGhlIHNjYW5uaW5nIHJlc3Vs dHMsIHRoZSBmdWxsIHRleHQgb2YgZS1tYWlscyBhbmQgYXR0YWNobWVudHMgbWF5IGJlIG1hZGUg YXZhaWxhYmxlIHRvIFdlYXRoZXJmb3JkIHNlY3VyaXR5IGFuZCBvdGhlciBwZXJzb25uZWwgZm9y IHJldmlldyBhbmQgYXBwcm9wcmlhdGUgYWN0aW9uLiBJZiB5b3UgaGF2ZSBhbnkgY29uY2VybnMg YWJvdXQgdGhpcyBwcm9jZXNzLA0KPiA+ICAgcGxlYXNlIGNvbnRhY3QgdXMgYXQgZGF0YXByaXZh Y3lAd2VhdGhlcmZvcmQuY29tLg0KPiA+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fDQo+ID4gZnJlZWJzZC1oYWNrZXJzQGZyZWVic2Qub3JnIG1haWxpbmcg bGlzdA0KPiA+IGh0dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVl YnNkLWhhY2tlcnMNCj4gPiBUbyB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJz ZC1oYWNrZXJzLXVuc3Vic2NyaWJlQGZyZWVic2Qub3JnIg0KPiA+DQo+DQo+IENhbiB5b3UgZG8g dGhlIHNhbWUgdGVzdCwgYnV0IGdyYWIgdGhlIG1lbW9yeSBsaW5lcyBmcm9tIHRvcCgxKSBiZWZv cmUgYW5kIGFmdGVyIGVhY2ggb2YgdGhvc2UgdHdvIHJ1bnMuDQo+DQo+IEkgYW0gZ3Vlc3Npbmcg dGhlIEFSQyBpcyBiZWluZyBzcXVlZXplZCBvdXQgYnkgUG9zdGdyZVNRTCwgYmVjYXVzZSB5b3Ug aGF2ZSBzbyBsaXR0bGUgUkFNLg0KPg0KPiAtLQ0KPiBBbGxhbiBKdWRlDQoNClRha2VzIGEgd2hp bGUgZm9yIHRoZSBkZWdyYWRhdGlvbiB0byBraWNrIGluIG5vdyB0aGF0IEkgcmVib290ZWQgdGhp cyBtb3JuaW5nLg0KDQpSZWdhcmRpbmcgdGhlIEFSQyBiZWluZyBzcXVlZXplZCAtIHdlbGwsIHRo YXQgZG9lc24ndCBleHBsYWluIHdoeSBnc3RhdCBzaG93cyBvbiA5NS0xMDAlIGJ1c3kgb24gdGhl IGRyaXZlcyBvbiByZWJvb3QgYnV0IG9ubHkgfjE1ICVidXN5IGFmdGVyIHRoZSBkZWdyYWRhdGlv biBoaXRzLg0KDQpJbiBmYWN0LCBBUkMgaXMgYmVpbmcgc3F1ZWV6ZWQgYWxsIHRoZSB0aW1lIGJl Y2F1c2UgSSd2ZSBsaW1pdGVkIGl0IHRvIDUwTSBpbiAvYm9vdC9sb2FkZXIuY29uZjoNCnZmcy56 ZnMuYXJjX21pbj0iNTBNIg0KdmZzLnpmcy5hcmNfbWF4PSI1ME0iDQoNCk5vdGUgdGhhdCB0aGUg RnJlZUJTRCA5LjAgc2VydmVyIHRoYXQgSSB0ZXN0ZWQgb24gYWxzbyBoYW1zdHJpbmdzIHRoZSBB UkMgdG8gNTBNIGJ1dCBkb2Vzbid0IHN1ZmZlciBhIHBlcmZvcm1hbmNlIGRlZ3JhZGF0aW9uIGhl bmNlIHdoeSBJIGhhZG4ndCBib3RoZXJlZCBtZW50aW9uaW5nIGl0IGJlZm9yZS4NCg0KVG8gcmVt b3ZlIFBvc3RncmVzIGVudGlyZWx5LCBJIHdvbid0IGV2ZW4gc3RhcnQgaXQgYW5kIHNpbXBseSB1 c2UgZGQgb24gdGhlIDE2R0IgZmlsZS4gIFRoZSBzZXJ2ZXIgaXMgZXNzZW50aWFsbHkgZG9pbmcg bm90aGluZyBhdCBhbGwuDQoNCkF0IHRoaXMgcG9pbnQsIEknbSBsb29raW5nIGF0IGdvaW5nIGJh Y2sgdG8gRnJlZUJTRCAxMC4zLVJFTEVBU0UtcDcgYXMgeWVzdGVyZGF5IGFzICd0cmFmZGV2JyBy ZXBvcnRlZCB0aGF0IGhlIGRvZXNuJ3Qgc2VlIGFueSBwZXJmb3JtYW5jZSBkcm9wIGFuZCBoZSdz IGdvdCA5NSBkYXlzIHVwdGltZS4gIEhlJ3MgYWxzbyBtZW50aW9uZWQgdmZzLnpmcy5tZXRhc2xh Yi5sYmFfd2VpZ2h0aW5nX2VuYWJsZWQ9MCBzZXR0aW5nIHdoaWNoIEkgYWxzbyBuZWVkIHRvIHRy eS4NCg0KLS0NCkFhcm9uDQoNCg0KVGhpcyBtZXNzYWdlIG1heSBjb250YWluIGNvbmZpZGVudGlh bCBhbmQgcHJpdmlsZWdlZCBpbmZvcm1hdGlvbi4gSWYgaXQgaGFzIGJlZW4gc2VudCB0byB5b3Ug aW4gZXJyb3IsIHBsZWFzZSByZXBseSB0byBhZHZpc2UgdGhlIHNlbmRlciBvZiB0aGUgZXJyb3Ig YW5kIHRoZW4gaW1tZWRpYXRlbHkgZGVsZXRlIGl0LiBJZiB5b3UgYXJlIG5vdCB0aGUgaW50ZW5k ZWQgcmVjaXBpZW50LCBkbyBub3QgcmVhZCwgY29weSwgZGlzY2xvc2Ugb3Igb3RoZXJ3aXNlIHVz ZSB0aGlzIG1lc3NhZ2UuIFRoZSBzZW5kZXIgZGlzY2xhaW1zIGFueSBsaWFiaWxpdHkgZm9yIHN1 Y2ggdW5hdXRob3JpemVkIHVzZS4gUExFQVNFIE5PVEUgdGhhdCBhbGwgaW5jb21pbmcgZS1tYWls cyBzZW50IHRvIFdlYXRoZXJmb3JkIGUtbWFpbCBhY2NvdW50cyB3aWxsIGJlIGFyY2hpdmVkIGFu ZCBtYXkgYmUgc2Nhbm5lZCBieSB1cyBhbmQvb3IgYnkgZXh0ZXJuYWwgc2VydmljZSBwcm92aWRl cnMgdG8gZGV0ZWN0IGFuZCBwcmV2ZW50IHRocmVhdHMgdG8gb3VyIHN5c3RlbXMsIGludmVzdGln YXRlIGlsbGVnYWwgb3IgaW5hcHByb3ByaWF0ZSBiZWhhdmlvciwgYW5kL29yIGVsaW1pbmF0ZSB1 bnNvbGljaXRlZCBwcm9tb3Rpb25hbCBlLW1haWxzIChzcGFtKS4gVGhpcyBwcm9jZXNzIGNvdWxk IHJlc3VsdCBpbiBkZWxldGlvbiBvZiBhIGxlZ2l0aW1hdGUgZS1tYWlsIGJlZm9yZSBpdCBpcyBy ZWFkIGJ5IGl0cyBpbnRlbmRlZCByZWNpcGllbnQgYXQgb3VyIG9yZ2FuaXphdGlvbi4gTW9yZW92 ZXIsIGJhc2VkIG9uIHRoZSBzY2FubmluZyByZXN1bHRzLCB0aGUgZnVsbCB0ZXh0IG9mIGUtbWFp bHMgYW5kIGF0dGFjaG1lbnRzIG1heSBiZSBtYWRlIGF2YWlsYWJsZSB0byBXZWF0aGVyZm9yZCBz ZWN1cml0eSBhbmQgb3RoZXIgcGVyc29ubmVsIGZvciByZXZpZXcgYW5kIGFwcHJvcHJpYXRlIGFj dGlvbi4gSWYgeW91IGhhdmUgYW55IGNvbmNlcm5zIGFib3V0IHRoaXMgcHJvY2VzcywgcGxlYXNl IGNvbnRhY3QgdXMgYXQgZGF0YXByaXZhY3lAd2VhdGhlcmZvcmQuY29tLg0K From owner-freebsd-hackers@freebsd.org Wed Jun 14 22:04:25 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59D85BF4C26 for ; Wed, 14 Jun 2017 22:04:25 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 284BF7A5AA; Wed, 14 Jun 2017 22:04:24 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v5EM4JGb063455; Wed, 14 Jun 2017 15:04:19 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v5EM4IEu063454; Wed, 14 Jun 2017 15:04:18 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201706142204.v5EM4IEu063454@pdx.rh.CN85.dnsmgr.net> Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours In-Reply-To: To: "Caza, Aaron" Date: Wed, 14 Jun 2017 15:04:18 -0700 (PDT) CC: Allan Jude , "freebsd-hackers@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Wed, 14 Jun 2017 23:37:04 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 22:04:25 -0000 > > -----Original Message----- > > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@f= reebsd.org] On Behalf Of Allan Jude > > Sent: Wednesday, June 14, 2017 11:20 AM > > To: freebsd-hackers@freebsd.org > > Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD perfo= rmance drop < 24 hours > > > > On 2017-06-14 13:11, Caza, Aaron wrote: > > > Further to this: > > > > > > I've now tested FreeBSD 10.3-RELEASE-p19 and FreeBSD 11.0-STABLE #0 r= 307264M, both of which suffer the same degraded performance. > > > > > > test$ uname -a > > > FreeBSD xyz.com 10.3-RELEASE-p19 FreeBSD 10.3-RELEASE-p19 #0 r319904M= : Tue Jun 13 12:38:29 MDT 2017 aaronc@WFT:XYZ amd64 > > > test$ uptime > > > 10:15AM up 21:09, 2 users, load averages: 1.00, 1.14, 1.30 test$ dd > > > if=3D/testdb/test of=3D/dev/null bs=3D1m > > > 16000+0 records in > > > 16000+0 records out > > > 16777216000 bytes transferred in 200.379127 secs (83727363 bytes/sec) > > > > > > After reboot: > > > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > > 16000+0 records in > > > 16000+0 records out > > > 16777216000 bytes transferred in 23.213040 secs (722749623 bytes/sec) > > > > > > Same Intel Xeon E31240 with 8GB ram and 2x Samsung 850 Pro 256GB SSDs= as before. > > > > > > This message may contain confidential and privileged information. If = it has been sent to you in error, please reply to advise the sender of the = error and then immediately delete it. If you are not the intended recipient= , do not read, copy, disclose= or otherwise use this message. The sender disclaims any liability for such= unauthorized use. > > PLEASE NOTE that all incoming e-mails sent to Weathe= rford e-mail accounts will be archived and may be scanned by us and/or by e= xternal service providers to=20= detect and prevent threats to our systems, investigate illegal or inappropr= iate behavior, and/or eliminate unsolicited promotional e-mails (spam). Thi= s process could result in > > > > deletion of a legitimate e-mail before it= is read by its intended reci= pient at our organization. Moreover, based on the scanning results, the ful= l text of e-mails and attachments may be made available to Weatherford secu= rity and other personnel for review and appropriate action. If you have any= concerns abo > ut this process, > > > please contact us at dataprivacy@weatherford.com. > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd= .org" > > > > > > > Can you do the same test, but grab the memory lines from top(1) before = and after each of those two runs. > > > > I am guessing the ARC is being squeezed out by PostgreSQL, because you = have so little RAM. > > > > -- > > Allan Jude >=20 > Takes a while for the degradation to kick in now that I rebooted this mor= ning. >=20 > Regarding the ARC being squeezed - well, that doesn't explain why gstat s= hows on 95-100% busy on the drives on reboot but only ~15 %busy after the d= egradation hits. >=20 > In fact, ARC is being squeezed all the time because I've limited it to 50= M in /boot/loader.conf: > vfs.zfs.arc_min=3D"50M" > vfs.zfs.arc_max=3D"50M" Would you passify an old fart by at least having a delta of 1 between a min= and max please? Some code may oscilate when min=3Dmax. > Note that the FreeBSD 9.0 server that I tested on also hamstrings the ARC= to 50M but doesn't suffer a performance degradation hence why I hadn't bot= hered mentioning it before. >=20 > To remove Postgres entirely, I won't even start it and simply use dd on t= he 16GB file. The server is essentially doing nothing at all. >=20 > At this point, I'm looking at going back to FreeBSD 10.3-RELEASE-p7 as ye= sterday as 'trafdev' reported that he doesn't see any performance drop and = he's got 95 days uptime. He's also mentioned vfs.zfs.metaslab.lba_weightin= g_enabled=3D0 setting which I a= lso need to try. >=20 > -- > Aaron >=20 >=20 > This message may contain confidential and privileged information. If it h= as been sent to you in error, please reply to advise the sender of the erro= r and then immediately delete it. If you are not the intended recipient, do= not read, copy, disclose or=20= otherwise use this message. The sender disclaims any liability for such una= uthorized use. PLEASE NOTE that all incoming e-mails sent to Weatherford e-= mail accounts will be archived and may be scanned by us and/or by external = service providers to detect a= nd prevent threats to our systems, investigate illegal or inappropriate beh= avior, and/or eliminate unsolicited promotional e-mails (spam). This proces= s could result in deletion of a legitimate e-mail before it is read by its = intended recipient at our org= anization. Moreover, based on the scanning results, the full text of e-mail= s and attachments may be made available to Weatherford security and other p= ersonnel for review and appropriate action. If you have any concerns about = this process, > please contact us at dataprivacy@weatherford.com. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >=20 --=20 Rod Grimes rgrimes@freebsd.= org From owner-freebsd-hackers@freebsd.org Thu Jun 15 17:37:46 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1F09D8EA4F for ; Thu, 15 Jun 2017 17:37:46 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A795F7CEB8 for ; Thu, 15 Jun 2017 17:37:46 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from lm0.local (cpe-75-177-185-136.nc.res.rr.com [75.177.185.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id 860DE239431; Thu, 15 Jun 2017 13:37:39 -0400 (EDT) Subject: Re: crash under load in GENERIC To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170614150144.GP2088@kib.kiev.ua> From: Nikolai Lifanov Message-ID: Date: Thu, 15 Jun 2017 13:37:30 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20170614150144.GP2088@kib.kiev.ua> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="nGIdqNbo19UCcxeNmc3235uhNSJgO0HAr" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jun 2017 17:37:46 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --nGIdqNbo19UCcxeNmc3235uhNSJgO0HAr Content-Type: multipart/mixed; boundary="Q3wuIWELQNpG7WflupwUMNAutEK6Bj6iq"; protected-headers="v1" From: Nikolai Lifanov To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Message-ID: Subject: Re: crash under load in GENERIC References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170614150144.GP2088@kib.kiev.ua> In-Reply-To: <20170614150144.GP2088@kib.kiev.ua> --Q3wuIWELQNpG7WflupwUMNAutEK6Bj6iq Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 6/14/17 11:01 AM, Konstantin Belousov wrote: > On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: >> Hi! >> >> I haven't nailed it to a specific commit, but some time after >> head-319667, my package builder box crashes when running Poudriere. >> The earliest version that I can confirm crashes is head-319735. > Do you see crashes past r319754 ? If yes, try reverting r319722+r319754= =2E >=20 OK, I just confirmed. I do get the crash with head-319953. I do not get the crash with head-319953 with r319722+r319754 reverted. - Nikolai --Q3wuIWELQNpG7WflupwUMNAutEK6Bj6iq-- --nGIdqNbo19UCcxeNmc3235uhNSJgO0HAr Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJIBAEBCAAyFiEE5oT6TcuaWvG5gtjzZ6sv56ecR0UFAllCxeEUHGxpZmFub3ZA ZnJlZWJzZC5vcmcACgkQZ6sv56ecR0X01BAAhjVfHTzMLTbqVWcOQTEbb4dNroKa O6qy/3xyAcPb3m77ZLen+TugN4pC9B4PdzSGtsp1dIhnk3GHP168O8Arodh+qH5j Bm38lB81Yd/3y4IvJxGcTJ9Nv9vdG8+PqVTkx7xjXXmmS9CGT7UjLz7uLo+sE2Jf LCT9JLnCssrIvpIhUiRAOAdCKEaK6O+Tl18dYBiVRBS8cUZ/jLjoivJ8vYtri7J7 jFr6Q72++e4+WqfII7KOMfcEtU+6NxvZVaYYShmjjT0VoUgjcgXFAUokEkNkufbm 8cmTOQr0uFSHBOaTEigqxM7eGyVRCf/Dge+ltONWTipNIZP6ZmE3txanhEEc644O hQkETSZ/YtRGVmiexlNkqLfyvX2tIQV57R+1LOxK/HVR/0o82gGxtDdB8EHJN7vm XvqEX6xzcruqIxSYPGJ4E8s+OtrhAvTFuHPWD2QbeSB0UVJCFmh5ckMjqaDmTdA1 7TxL5tvzPm542k+Ko5rAFrSUUTkt2N3zitOaTAMjPmGy0Qt+wr+fi8UlxeEcefx6 q34ZQ3A0/pkZ5lgTJ2cLrsl6eSqWxVpeJQKawGSW8pEd9tMQ2Puk80Wv4RtqP5UM RJbposPwzUF166fcL1UK1kD+h1VECJ603YYKp6z2fOf2EznsEThsy7sWrIrzzusM NtpqiNFAKpoRzWg= =rZkA -----END PGP SIGNATURE----- --nGIdqNbo19UCcxeNmc3235uhNSJgO0HAr-- From owner-freebsd-hackers@freebsd.org Thu Jun 15 17:51:24 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B204D8F037 for ; Thu, 15 Jun 2017 17:51:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFC4D7DC66; Thu, 15 Jun 2017 17:51:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5FHpE2a042258 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 15 Jun 2017 20:51:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5FHpE2a042258 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5FHpEE2042257; Thu, 15 Jun 2017 20:51:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 Jun 2017 20:51:14 +0300 From: Konstantin Belousov To: Nikolai Lifanov Cc: freebsd-hackers@freebsd.org, glebius@freebsd.org Subject: Re: crash under load in GENERIC Message-ID: <20170615175114.GX2088@kib.kiev.ua> References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170614150144.GP2088@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jun 2017 17:51:24 -0000 On Thu, Jun 15, 2017 at 01:37:30PM -0400, Nikolai Lifanov wrote: > On 6/14/17 11:01 AM, Konstantin Belousov wrote: > > On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: > >> Hi! > >> > >> I haven't nailed it to a specific commit, but some time after > >> head-319667, my package builder box crashes when running Poudriere. > >> The earliest version that I can confirm crashes is head-319735. > > Do you see crashes past r319754 ? If yes, try reverting r319722+r319754. > > > > OK, I just confirmed. > > I do get the crash with head-319953. I do not get the crash with > head-319953 with r319722+r319754 reverted. You should have added Gleb to your reply. I did it for you. From owner-freebsd-hackers@freebsd.org Thu Jun 15 20:12:06 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B49C5B95A6C for ; Thu, 15 Jun 2017 20:12:06 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F1C382817; Thu, 15 Jun 2017 20:12:06 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id v5FKC4Bs072785 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 15 Jun 2017 13:12:04 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id v5FKC4kQ072784; Thu, 15 Jun 2017 13:12:04 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 15 Jun 2017 13:12:04 -0700 From: Gleb Smirnoff To: Nikolai Lifanov Cc: freebsd-hackers@freebsd.org Subject: Re: crash under load in GENERIC Message-ID: <20170615201204.GR50023@FreeBSD.org> References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> User-Agent: Mutt/1.8.2 (2017-04-18) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jun 2017 20:12:06 -0000 Nikolai, On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: N> I haven't nailed it to a specific commit, but some time after N> head-319667, my package builder box crashes when running Poudriere. N> The earliest version that I can confirm crashes is head-319735. Should be fixed in r319988. -- Totus tuus, Glebius. From owner-freebsd-hackers@freebsd.org Fri Jun 16 05:06:31 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 496CDBF689E for ; Fri, 16 Jun 2017 05:06:31 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0049.outbound.protection.outlook.com [104.47.36.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5E7A70523 for ; Fri, 16 Jun 2017 05:06:30 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=/01uIStrE5la+cL0AqhUsqSHYcO2YiqF3MdqJ7mg9Ig=; b=FWjPmutoc8JOljCyROUC14RtNNSyzdlYPgFNu4OKDidvvcN7cfImUmIAPXr0GiKNYwDk5rs8sqkiRvlpuNn08LFzw2BqTtV7egiM93vTvDys5qSAIVHfb6gNjGMrmOLolCUD9zh8ZUw+Z5aWgNZFDNWNd/Tzv24pfubLaN6HwLo= Received: from DM5PR03CA0055.namprd03.prod.outlook.com (10.174.189.172) by BY1PR0301MB1239.namprd03.prod.outlook.com (10.161.203.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Fri, 16 Jun 2017 05:06:28 +0000 Received: from BN1AFFO11FD026.protection.gbl (2a01:111:f400:7c10::116) by DM5PR03CA0055.outlook.office365.com (2603:10b6:4:3b::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Fri, 16 Jun 2017 05:06:27 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.20) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.20 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.20; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.20) by BN1AFFO11FD026.mail.protection.outlook.com (10.58.52.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Fri, 16 Jun 2017 05:06:27 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net (141.251.110.81) by DM2PR58MB015.032d.mgd.msft.net (141.251.110.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Fri, 16 Jun 2017 05:06:26 +0000 Received: from DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) by DM2PR58MB013.032d.mgd.msft.net ([141.251.110.81]) with mapi id 15.01.1157.022; Fri, 16 Jun 2017 05:06:26 +0000 From: "Caza, Aaron" To: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLlL01Bja9RgR6rT826WHBpLYHdbgAAzCeAAABmhaAACYTJAABAA/BQ Date: Fri, 16 Jun 2017 05:06:26 +0000 Message-ID: References: <201706142204.v5EM4IEu063454@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201706142204.v5EM4IEu063454@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.136.196] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: DM2PR58MB015:|BY1PR0301MB1239: X-MS-Office365-Filtering-Correlation-Id: 074f86ba-b2c3-4cb7-fb53-08d4b47571bb Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: DM2PR58MB015.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.20; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39450400003)(39410400002)(39400400002)(39850400002)(39840400002)(39860400002)(2980300002)(438002)(13464003)(377424004)(189002)(199003)(377454003)(24454002)(9170700003)(86146001)(86362001)(6916009)(106466001)(46406003)(2900100001)(8746002)(5640700003)(108616004)(8936002)(72206003)(305945005)(54356999)(2351001)(81166006)(9686003)(110136004)(76176999)(229853002)(53936002)(66066001)(38730400002)(6246003)(50986999)(114624004)(47776003)(50466002)(8676002)(55016002)(2501003)(102836003)(2950100002)(3846002)(6116002)(2906002)(23726003)(33646002)(97756001)(189998001)(7696004)(22756006)(356003)(478600001)(5660300001)(24736003)(7736002)(53546009)(42882006)(5890100001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0301MB1239; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; MX:1; A:0; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD026; 1:9oeLyZDAO56jT8O0sAE5gpvVRR+joUncKMLROTXXKrmxPSAscU5NYjJCUML8FtJkz3m38xxC2PcBa4Y9xhymJhaLPiygKWknPXjrBe6D7eeiwzdaDgNHk8L8BivR9J18g/4jaLlPzC4S0f7Z8jWDevq41qPSoP3q+qN5shMOam6S4A+O5OMhD2XL1JFGzq2YrpifKi3u29RW4+lQ0wEba4mAm/muACUIMkkdF6nN+9XecJbb/eSBqPLWJKFhdCzZAjsePNVQttDWQVsdErW4rUylW/kf9v/1fHjG8ftugqfiWGJ2FgRkG0FxHYd6z/O17AvVNt8VfXzVeNiWGAju3TzPBebYe4o3oA/Ck55EI3PIVXzY0i6nU6bNjyyWPvlFTi7/7hTcZfGNnQCiP9nXy2QwZdAB/NG5k/KeLxiAhAkLENgS6elA4JkpyFQd76JFTvpcTbwNIiZxHcjKONDDAZLPGcKRKq2Huqiwv+sKbQEbLiLs0JzvFuomvUH7FdxQOZC99FMO3OKAn7CLUswUSGIJGLuLtAycS3jx5WlnJTY= X-CrossPremisesHeadersPromoted: BN1AFFO11FD026.protection.gbl X-CrossPremisesHeadersFiltered: BN1AFFO11FD026.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:BY1PR0301MB1239; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB1239; 3:AzvSFmhZDubtSs/tiHIGMZaF/RZDZG9mnZt0j5YGwnom/wJmLfuNG2qfyVI+gG7ChlbwDoxQj7QgKn1RziSoufUGaKPzEdq8vrL38yq+gKWBKlvORyIgv5/r5RC/FVRioriBRHLqbKoC67OPt90DhF1S7KJdoIWqTiBKDJ7UCoM208r47iH96/VS+FWXRzaasPXe+ct3YjkYvMPQTbPwSO7jXSXZPVQu819p91ZazrwpLk5CeLESRihkiO0t/f80TyoE1x07U2U4THdiuRVoXcMNYmqpf7y74JdkNmfCYaQ5skLcXLdMQCCZ7Ct3wSX/SmDwPhW32nQzDGDj2bejSe7DBySfZ0b2vSr0FhW8RnIKNBezgkJIN48jwwPSFC/INlS5jIZofjTIXQQneE8VZPdVtNYcPe90sd16InqCmWo3BW475JA6qInPkmVycBVgDDHw0NIlvDiiXjk1sQ20ksjla2ttVHKfqrGM5Kpt4DuQQQ8aLh/bmPkdJ9YvR2Z7kBLAK13BIAuOOfCQ2bU2zg== X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB1239; 25:3Zlg13wpLn1amGi6DtK3JeRQ47ruLayzGyQkAaHqxwmJK2rPT7rfnORFn8tfaUeb+J4jXJPSZ/Y4dgNKjnhv4uH+WZTOeYlURe2Gdk2xlgTMv4G0in6a6OS4vztKcgOTc7t6CMdpBXHBXGveQkQGddFfUFswdcQGl1AU/MsgUebxWHc8EkuWL+So7CLCXlwfvqmAsYk/XWIny7Z2/7bxmJF/53Gpyw9EWUujPLAmrpGKp7EmN4mPOuY3nDUIs0jVXj5Od/6q/m3zhYXj7FOM0A79pFHAF4bgJrnsFP085RHE4JMDgO3Bzwi6JwpBma7KYmKP3LR8oW6fNAr/NV7FvTf82XYK/nUg9O3qj/xyihN9VA63mf8NKDGK1gz+3L9+NIJWZjOf4WiUAm86e3cf1lxV+JJloJr1+4iOsvgaYHnSavBC5HDdF0bsqNlwITvOEn9qFnhz4M4mzICFg5dikrvc64EFfbvc64WCM3dAx7o=; 31:cDc4yaAhokRhpxo5aBbalDcSZe0otmlmS1Vazy7ds5kMrLY9vJ90/AYKS4tpOXB2KxZyAckwbUE1CYmveIgkx3zKJT8/qj0CCJV6C2TgdSh8et0NiQVIKmls+nh1CfrgQnlRBEC78hKamXWVCJrCgEfYZnuQdcga5LbbBOsda+Gm0i50tgXhlOqAAd4Qlg0eCXcFQGUrTw/QpEWJoFnYI1cZVrsa4887EAcny+Ay5MC+UdC8apH5CgjtHHcL6WIu6JsdAn0TbSTbJzKAuhvzUCQTSg3dX2QVi3CeeVPrkYw= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB1239; 20:8GGviNCJe9EvcsxNYwk+QTbKMMnmVED2AK2Xz7jwjMBfNvPJNK35jcDeSBiAtFSbuXAqubNOciS/Sr9J0nvvSqQgXssz09n4g48D0JnJGBtrV19ERvraXVkko5ZJAW27jGmAmc7q/1ROFI/QhW4a7TVc1kqNn+F+nT0TwLtB12WCcpQfjhkEyKhz9kkfyLi1raAkvQfkTWVtlHunfjubB/BXmS0UY0iTByujuoU8O7Gw+BGivCBntsB3nbtmMLH2owkWCrAPC8QwJFHXv+qzKvtYQsdvmbPTXAAua2KZ+TJWqRXTFeDFkhpNmPn8p5NYAy0DFzjgujkw8UCmDgr1T8JWzsuVttX2JK19KZyJFmH5SkteOvuUsvg9f0oRjMpV5ZlY1tLehiE309SYz/h8cpqriC49TRinnGPkGqCYPdp9S3JT4XDXEYDX2NxsdY8//MXv3tWu+xo9lLQaLz7G+v3LXW6qqlhVDGJn5zNuWIgZGYOypzae8yY04IKAxZJa X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(5005006)(13016025)(8121501046)(93006095)(93004095)(3002001)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123558100)(20161123560025)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY1PR0301MB1239; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY1PR0301MB1239; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0301MB1239; 4:DfGgTgb0EJqCL3sDDCqvurOYmKm7zSxJhSTC9+V8?= =?us-ascii?Q?QCh/Dma+Br7MIF4yY7vSxW5FAjhbeIX+OYQLUvp+3fmMHZp0JkFzo/7epWv2?= =?us-ascii?Q?TmM2Wsk4DAd3FckSotpM+hrbPNhARscvm7hFezw9uC8DTwKVaXfm/IrWIizD?= =?us-ascii?Q?u0wZvfZA8c5xD+kwI0MWdCAvYmccrZR3KN1S6Avp9NPL1m2BnAHgTn0QR5yK?= =?us-ascii?Q?CSfcqrbsNIee0Gpgj3gBqc3o9T/ruf0GLh+GRcPKj00cG3u1Lk6ZxgJla5h2?= =?us-ascii?Q?5zS89HEcuxuBHrnlHlG6Yd8C+p5ROlO5qrWrzIPf6t6/C2OLMpFjzsKaTdi+?= =?us-ascii?Q?hOsbQkqItRMx8sJqIrtNLbElUGUcG/iTddjnptUOHMguzLdhVbxbZn0A05nt?= =?us-ascii?Q?qpy1XYO/dudMqnftRcPEjsp+jWqjhIQxeM9L/l94Dcph4ltVN3wRhoaaQrCV?= =?us-ascii?Q?YPgfuVgVXjPjCraoPazQVnYduRQtvvCuEktQ+aN2hm6XYKCYu3Hcf/DGrvAu?= =?us-ascii?Q?8o+i3o8Ko7fu7+vtc980z+KVnSn3HoIxgx1Ce5WQ0Y4ChyNvgUc340I9Gm/w?= =?us-ascii?Q?mQPZRXQB4DmNWEBZgltmNgLzIJnhX7kTijmbyplJiwE7Sipq9TXZ6oUilUF9?= =?us-ascii?Q?Wu2Wuw5owpasNti/mFhgqzpAKSaDdtvI/dr3MS85UV0jt2xd/AxKraZCq5JN?= =?us-ascii?Q?+V+sHEG61gfTlFWZF4ohZQS3OSylRHdm9S/IVDLK503dZ5IBm7Ojis/aOvcN?= =?us-ascii?Q?ZyxWdLDLLleFUkwq3mzGQuu6ZABbOqMW3sg+Y8LXHmTNTjjrpdr19BEP7CSG?= =?us-ascii?Q?c3ENYLrq2LMGtno4Al6QtaHUqh0GllQ409aQRBzFpUvysNIl2W5COjYGbvx2?= =?us-ascii?Q?fK8TNVcc1u6jP/qEnb96+LubEezjIwcF0sVqEsFhqGoha83PXKMi8N9eQrd9?= =?us-ascii?Q?xWj6fscE+Pc5cwBG3PQgnysvV7prMK9cWx+8u8kWzcyYa3erlrE5XM+vdPqi?= =?us-ascii?Q?sa10/yu5zPF0M7uCVSHkkyZmq9FdXsAazO1kLr3kjRSLmLYWc5+WWsmxz+my?= =?us-ascii?Q?Ps5P2aUufYdxPYVAFh616rR1XeApj7wJNpnPNHDSoY6sA465ADD4TASERPRR?= =?us-ascii?Q?Bj0w9y8UjtGfU7beF9FHfCn5dO4myhL5Ti/oNBNAv8pOQBCKHQ+Cd72yNTb2?= =?us-ascii?Q?rG6Zg+5UD0aup4k/PIe0cJOXjaLYxUv+8Pa4yySbCEJbM+nGAnxkyoEpvp7V?= =?us-ascii?Q?bRnNJbvcqajgiJq3sVHbSEUFXtkwb7/MTytUW2c5f7Q/ORGi+8AtlqBxCtgA?= =?us-ascii?Q?eiKWoNL10CEDpgxm372/bdE=3D?= X-Forefront-PRVS: 0340850FCD X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0301MB1239; 23:1Yl+8LwVwgQESuk8uxSfWCp0+yzIyfxRSFuHb6h?= =?us-ascii?Q?Tzr4jwzeP2gicDflqBRhfIikCF/E/PDAswRmoXsUTrBh+HHuthcCR4oj/b6v?= =?us-ascii?Q?g39ltGTw1pkzzXAzyLTUX9N2XNvcdF7zht38MIFsWMW8e7y9XcpY3QEbzCQR?= =?us-ascii?Q?VeEjeA2GfgeA+ZqHQUb3WPS6zDEobti6S3bG0fUKH0SdQTTVNeqeJkZy2wcX?= =?us-ascii?Q?Rd0fNO1gH2yKnDwyO5aYpzYdf9/T7Xuy11YT0DCrUOLlvP30RVjos4eKqwl3?= =?us-ascii?Q?GTtV8Z6hBluytWJrfr98mGn7HjcQItscIfBgaCUrIQXYcS9UOCMDQbyyWYAz?= =?us-ascii?Q?cPjYfp75Kb5U28ihvXKAquP5m0z1t2Tu0F/hd3mHEMvhq6qoYZXTIQJMsxZx?= =?us-ascii?Q?ZDyVybLfPgFAVuE8NlGnloQkHWghD4KS+il130/VEV85gaA8rYp5HTK3jfVi?= =?us-ascii?Q?v6oiwhf4Z0sFTLk8eBpOjfq9DBxIfL+gXkQFPNSJJiDcA2SSWLzJWTpJ/rjT?= =?us-ascii?Q?cXWVf77fvWPg6uytAE2JGLGrr/SQ0Wmli6kvek06KJUyENNx0n7FJ4OYD6wp?= =?us-ascii?Q?ebD+9azgMCfHZ97cT8ED0lu83NsO5WT0m5yCIMc7VJmviaJYdiDow90WwqhA?= =?us-ascii?Q?2OTBifCOKFw2BZii/XlIRvjyx4/yV6zlGrlFixgI10YgzHqgrZdzsejn4FL9?= =?us-ascii?Q?jGX7vxqIIic1TNylfIrUPENk6ITnJDiEI707c9GxOVWSLzbZ2y6j/H8K8rJ1?= =?us-ascii?Q?M3gH4ozXzB6FvWqdXosJNQg7k4f498bLNKw5Cg20ikxP+ArAbtuY7pjzobs/?= =?us-ascii?Q?05MEYvWNUjwtSYUkFpfjQ0nJX7nS6KHOoRAvGqKA2riBgQZQk5mOTdB71L9S?= =?us-ascii?Q?XcPnE/fWCowXWEHYsrYfXX8FP5gYTmkGfatuymAcAfZLAdt9wTqZpUlIfRMX?= =?us-ascii?Q?RbuLwY7AEps3+gkiIWkm8n9Ag1DSV+xJ3dubvI2fzt3IIuWFqvvnsLNZBbPw?= =?us-ascii?Q?QAICoxtjDCDt5yIu/ULNLp+VmYo7QDk/jVsnO2IwRuLcTfF9Y+1DmvUERdhr?= =?us-ascii?Q?T/AWQavqpBvp8I31bgXR1wpatgKq4vIcdASOCXoWfnerCnlOoMd5wCBjLu2g?= =?us-ascii?Q?cD77Ej+vTMlmxSgFb8rkmPEpLOacfj0oqXF1b8s1uxxD/E8GFJIcHjIzghb3?= =?us-ascii?Q?tR2UllplNP53MmqOT8sNwYG5/3LNnRhee9RwGMsm6YT4qHZvg+tk/Prf/bma?= =?us-ascii?Q?KWD0OaG3GsUBsjmj6bHi6/EdCu5d2VN3d0B3X0QlMxs7nFQNWZXsKBJuTpBN?= =?us-ascii?Q?AQGi5KpdXVFwvZiEK4BYJbNx3CLpOyZ5ApRBxeiTUsOpFOd25pvePybXNL+o?= =?us-ascii?Q?auxFW54WkgVZFlqTvPceGVKduoscTR2c8XMng9h4ds30u0Z/oepS3uJbZTd1?= =?us-ascii?Q?dM3jVg26lhu8VGqtpbL1IL1VUOHLJCWEiSscws91dEPXhPFJ7KfUDeSLQMhA?= =?us-ascii?Q?wGIRgds9ZIfFPc6aXwNmkEh57JiqYsA/7hUei1DsdH051LK3FjxkkaJOVuBQ?= =?us-ascii?Q?lOqj7Ai2Iwz7khKqN2Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0301MB1239; 6:2AsAjIMxraDA2wBfNeNHgdGw6eMQp+K4afFwPSHV?= =?us-ascii?Q?gvfCeDIetEl+3a/VkRGy7InkjYpEpbCbHoBD31NPYW6fW1GBqvR43RaVqwj6?= =?us-ascii?Q?yOKw4o7QEZRgAkL2E8h5kom4R1Yq7HcQr55oIX8RHEBQdGtkSjtg2rqMDXLL?= =?us-ascii?Q?iB3oldGGIHB6E1lmjw8G3R6uUi5LV2FcF3aZ9cdZhn9nHQI0mhPl+l7eyZrH?= =?us-ascii?Q?dcpAfu9BI+8N1f29WgNezRkkb33llXlAxF4kxzujgQFMflTXE1EsjlIxDP6b?= =?us-ascii?Q?tIYBinkPjPa6o+qXdwVfNRIVzLXAc2nu6jFBYh/kJGlEFSIm9OaH8RcRE6cu?= =?us-ascii?Q?JbOYhP+nBhNKWTxi+DV+PkOFUMqYDpzyfOZLDFl78L5Fv5dwPhBrSVQSRnUH?= =?us-ascii?Q?VYPZGszbnaB0xXnrdjKm8fZmV+netfdCvaF0/2gNr/E6mk3azuKAz+L/rhLM?= =?us-ascii?Q?erhJrHwXBm+bwvcI8k99CPi4BfzpgpJTRBn5FU3v9Kgh/ONIKc6YGQWyDPi4?= =?us-ascii?Q?ehyBz/56SwDIr87Ifm/Wsgytj7f6NOQl6zTOVlm1sdxpwin3noocmYFYFHe6?= =?us-ascii?Q?xcCaiviRIwMZcSgGjJSt8pzOcobjdlJC4w92feX/ngAFHrLLYvNWMWueY55g?= =?us-ascii?Q?yQYAE7ifscS2PfmKDza9E6br3sAvaOJ0aSTcTSPYwJJrCJqk3zOxB9gbqsGX?= =?us-ascii?Q?wVYqlUGfJvohitDKb5/Bm7hlzo4TXSqh8A7Gt05wX4r+S6N7wfBw9DOovqhp?= =?us-ascii?Q?0mbnFPqdtC2g5WHtC4sNqyqRMZhrcJKuHaReCXWXWIoM8OUsqHOWAJ4StEU1?= =?us-ascii?Q?CQsiaFDuvWYLpEwciVLLA15C7dThRzNL4o2cX1oApACAe8ugOGmXSA6hYfy8?= =?us-ascii?Q?zet22MFCSFvxPK/WJnGAg61A2THqEri0DsC+b5vUQC6D8EK3Qisqdd9Kn8ra?= =?us-ascii?Q?8BfJcuvQTRkJhHo8fHEDC9zOejECXKh5sw7WZ6ui8b4w1NUXI363sUdJl3hB?= =?us-ascii?Q?PHZosz3o1eU22FF37MvyjoCi?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB1239; 5:XKj5TFYEl2WHJn9VGjuetkR4IxBXgjwLaLjM9YaSvsR8jfevXcgoyAIxINASzNGHBVJ72K3D0LPM9blvx6uTA/eDW5Q1dGSaSJe0DRrSKWjsJ5PD1GicjPqPYLMjBz1oFI6iwrJo6P3ikGVTP5PcN7nwbmSXGAzhX4NxiGXwffUwyK9YxGPSNeADxQrXI3uHf5NLFHDMw3xlfcTLt2KL38eKDzLFNZ0SrAhiz5VpHNWciqSrTv7bt3ifro6vcboCsaGcJjEz4XBMBI23Y0NzQ2P2hdKvVSlzc4DnrXHaBTagh3zGv1Mi9GbIivbK9dElBFrziCvpVpsUGtUUwRG8MZ+YGeQz7dWP2EUq73No4RO06D57ASQV/rhL9cfvjLot94OCxzaV3QFH1BCgF2zPjyy54ybkqBNboodQ1zzC9+kicegPKsPzS0rckQDQy74oMN+GzSg7LzrsqRP6HpU60DmlTZ7m47MCR1uZPN3tylsxbPA13naGQOE2i3yg4i7J; 24:FQUQUFZhuQdrM8uwDFZOOvDrsIdovNqX6OEGGgczSW9zTFODh5xsrY0FiPNlKXvPrcT/2OVr4erUcgc/4KOI2RARI1DvE9hofz4JztVAalo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB1239; 7:PFAVyLX5Ryj1nM71zVw9/uFDKND55dokkTPPQfhC5muelPJqmgSE7He8trPT0dR7HeQWqImHzPjNk1wEvZ6b+j0PHoeJZiEUxTGwbf8+LrsTAGo+06Ar1qsfphu/rjgKtRLxKXn7LCHmjV+SeAJbzXCt3OEfaDyfmR05z6OPDlrwcdzQTqKYQg1STtOrywgZxnEbCrTObJkpMD9T6noS53KUjYY36M2MVChnb/Wk2HtiV1CFAotplfjUf/5yu3l6IiDlan6CtWSUAJYjHBHanVtV/Nzoydlbb8f7jKiCurFADtVIKNJhXGTuiT1YEmgrFZ8JoynIKSdTDqUPcfm1Nkm4JSQzE2Bcm5OgesBWkqy88slV8MEGJxEugDn5pNzTftq+I2R87yPNeX2XSAijHz0/yDMSyzmfpCP0XE5YiS9AUi6AEFhk1rLGUKIgKzdjj5AaY2Xyl6CV1dCUCmcu7ZTA6nnUmEuLLkKBx0+KlozyANiKPyEW6W94xIZwNsi/xVuAyxufXU1ddb+Co1OSzYnmDwxrGd1dxOKcixEsvssdEMEAx96/iN1k4rnAC05YTYyme+wc5DQCZKOfG9UI1k9xEjgF/qyah+2Znp8By5i+Ut37JGesrdxUjtdOt2kcmZTHHryCTvfEOCEOrGUPcZtjkryPCtHmFq5nqWg7eujgafmzIcH9kxiHSr8yLJCwyfDIvRDvws+DGXekwP7akTiHc3hzASUlqFbuL7+zQ1+7AvaDaucP7XPHCPzkk3M5KNpE/aHOFUBmCduN32xgghZWqS3KLYJz0CltAe6FodU= X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2017 05:06:27.7620 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.20]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0301MB1239 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 05:06:31 -0000 > -----Original Message----- > From: Rodney W. Grimes [mailto:freebsd-rwg@pdx.rh.CN85.dnsmgr.net] > Sent: Wednesday, June 14, 2017 4:04 PM > To: Caza, Aaron > Cc: Allan Jude; freebsd-hackers@freebsd.org > Subject: Re: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD per= formance drop < 24 hours > > > > -----Original Message----- > > > From: owner-freebsd-hackers@freebsd.org > > > [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Allan Jude > > > Sent: Wednesday, June 14, 2017 11:20 AM > > > To: freebsd-hackers@freebsd.org > > > Subject: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD > > > performance drop < 24 hours > > > > > > On 2017-06-14 13:11, Caza, Aaron wrote: > > > > Further to this: > > > > > > > > I've now tested FreeBSD 10.3-RELEASE-p19 and FreeBSD 11.0-STABLE #0= r307264M, both of which suffer the same degraded performance. > > > > > > > > test$ uname -a > > > > FreeBSD xyz.com 10.3-RELEASE-p19 FreeBSD 10.3-RELEASE-p19 #0 r31990= 4M: Tue Jun 13 12:38:29 MDT 2017 aaronc@WFT:XYZ amd64 > > > > test$ uptime > > > > 10:15AM up 21:09, 2 users, load averages: 1.00, 1.14, 1.30 test$ > > > > dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > > > 16000+0 records in > > > > 16000+0 records out > > > > 16777216000 bytes transferred in 200.379127 secs (83727363 > > > > bytes/sec) > > > > > > > > After reboot: > > > > test$ dd if=3D/testdb/test of=3D/dev/null bs=3D1m > > > > 16000+0 records in > > > > 16000+0 records out > > > > 16777216000 bytes transferred in 23.213040 secs (722749623 > > > > bytes/sec) > > > > > > > > Same Intel Xeon E31240 with 8GB ram and 2x Samsung 850 Pro 256GB SS= Ds as before. > > > > > > > > > > Can you do the same test, but grab the memory lines from top(1) befor= e and after each of those two runs. > > > > > > I am guessing the ARC is being squeezed out by PostgreSQL, because yo= u have so little RAM. > > > > > > -- > > > Allan Jude > > > > Takes a while for the degradation to kick in now that I rebooted this m= orning. > > > > Regarding the ARC being squeezed - well, that doesn't explain why gstat= shows on 95-100% busy on the drives on reboot but only ~15 %busy after the= degradation hits. > > > > In fact, ARC is being squeezed all the time because I've limited it to = 50M in /boot/loader.conf: > > vfs.zfs.arc_min=3D"50M" > > vfs.zfs.arc_max=3D"50M" > > Would you passify an old fart by at least having a delta of 1 between a m= in and max please? > Some code may oscilate when min=3Dmax. > > Note that the FreeBSD 9.0 server that I tested on also hamstrings the A= RC to 50M but doesn't suffer a performance degradation hence why I hadn't b= othered mentioning it before. > > > > To remove Postgres entirely, I won't even start it and simply use dd on= the 16GB file. The server is essentially doing nothing at all. > > > > At this point, I'm looking at going back to FreeBSD 10.3-RELEASE-p7 as = yesterday as 'trafdev' reported that he doesn't see any performance drop an= d he's got 95 days uptime. He's also mentioned vfs.zfs.metaslab.lba_weight= ing_enabled=3D0 setting which I also need to try. > > > > -- > > Aaron > > -- > Rod Grimes rgrimes@freebs= d.org Allan, here's the top(1) output after degradation: last pid: 9403; load averages: 1.09, 1.48, 1.30 up 0+23:29:14 10:3= 0:06 21 processes: 1 running, 19 sleeping, 1 zombie CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 1528K Active, 46M Inact, 462M Wired, 4856K Cache, 624K Buf, 7376M Free ARC: 53M Total, 5K MFU, 16M MRU, 16K Anon, 1137K Header, 36M Other And, after reboot with performance back to normal: last pid: 739; load averages: 0.99, 0.95, 0.58 up 0+00:08:02 10:5= 3:21 19 processes: 1 running, 17 sleeping, 1 zombie CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 18M Active, 22M Inact, 224M Wired, 432K Buf, 7626M Free ARC: 28M Total, 3385K MFU, 24M MRU, 16K Anon, 232K Header, 1005K Other The above was done using FreeBSD 10.3-RELEASEp19 (amd64) on the aforementio= ned Xeon server with 2x Samsung 850 Pro 256GB SSDs. PostgreSQL was never s= tarted or use - simply dd'ing the 16GB test file. These results were still= using vfs.zfs.arc_min and vfs.zfs_arc_max of "50MB" though subsequent test= s will utilize 50M and 51M, respectively as suggested by Rod. Checking the FreeBSD 9.0 test server, it had vfs.zfs.arc_max=3D"50M" but di= d not have vfs.zfs.arc_min set in /boot/loader.conf; consequently, it was a= ctually higher that vfs.zfs.arc_max. I've reset to 50M and 51M and reboote= d. Time for "select count(*)" in Postgres for ~21.5 million row test table= , after 24hours, is still maintaining the ~82 seconds as it did after reboo= t. Additionally, on separate test servers I've checked the performance of both= FreeBSD 10.3-RELEASE-p6 r303605M and the FreeBSD 11.0-STABLE snapshot date= d May 10, 2017 and both exhibited the same degradation in performance. Suggestions? -- Aaron This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Fri Jun 16 07:47:20 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D005BF91D9 for ; Fri, 16 Jun 2017 07:47:20 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3AB5C748C5 for ; Fri, 16 Jun 2017 07:47:20 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from [88.198.220.132] (helo=sslproxy03.your-server.de) by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.85_2) (envelope-from ) id 1dLlZL-0004y6-Qv for freebsd-hackers@freebsd.org; Fri, 16 Jun 2017 09:21:23 +0200 Received: from [82.135.62.35] (helo=mail.embedded-brains.de) by sslproxy03.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.84_2) (envelope-from ) id 1dLlZL-0005Rn-8v for freebsd-hackers@freebsd.org; Fri, 16 Jun 2017 09:21:23 +0200 Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 8707C2A160A for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id iHb6kkgWYmRY for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 48B4A2A1677 for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id lx_fij7ZL71C for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from huber-linux.eb.localhost (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTP id 2FD8D2A004F for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) From: Sebastian Huber To: freebsd-hackers@freebsd.org Subject: [PATCH 1/2] cfi: Add intr_restore() to cfi_write_block() Date: Fri, 16 Jun 2017 09:21:21 +0200 Message-Id: <20170616072122.17504-1-sebastian.huber@embedded-brains.de> X-Mailer: git-send-email 2.12.3 X-Authenticated-Sender: smtp-embedded@poldinet.de X-Virus-Scanned: Clear (ClamAV 0.99.2/23478/Fri Jun 16 06:12:21 2017) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 07:47:20 -0000 Restore the interrupt status in case of a timeout. --- sys/dev/cfi/cfi_core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/dev/cfi/cfi_core.c b/sys/dev/cfi/cfi_core.c index 1bf687e3cee..e90db06dcef 100644 --- a/sys/dev/cfi/cfi_core.c +++ b/sys/dev/cfi/cfi_core.c @@ -647,6 +647,7 @@ cfi_write_block(struct cfi_softc *sc) cfi_write(sc, sc->sc_wrofs + i, CFI_BCS_BUF_PROG_SETUP); if (sbinuptime() > start + sc->sc_max_timeouts[CFI_TIMEOUT_BUFWRITE]) { + intr_restore(intr); error = ETIMEDOUT; goto out; } -- 2.12.3 From owner-freebsd-hackers@freebsd.org Fri Jun 16 07:47:18 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 954ADBF91D6 for ; Fri, 16 Jun 2017 07:47:18 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C571748C3 for ; Fri, 16 Jun 2017 07:47:17 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from [88.198.220.132] (helo=sslproxy03.your-server.de) by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.85_2) (envelope-from ) id 1dLlZM-0004yB-2t for freebsd-hackers@freebsd.org; Fri, 16 Jun 2017 09:21:24 +0200 Received: from [82.135.62.35] (helo=mail.embedded-brains.de) by sslproxy03.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.84_2) (envelope-from ) id 1dLlZL-0005Tl-If for freebsd-hackers@freebsd.org; Fri, 16 Jun 2017 09:21:23 +0200 Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id CCC072A1677 for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id QP28xLHmAS57 for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 55FC32A004F for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id VxKVDAx9kAFJ for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) Received: from huber-linux.eb.localhost (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTP id 3B1202A160A for ; Fri, 16 Jun 2017 09:21:34 +0200 (CEST) From: Sebastian Huber To: freebsd-hackers@freebsd.org Subject: [PATCH 2/2] cfi: Refactor cfi_write_block() Date: Fri, 16 Jun 2017 09:21:22 +0200 Message-Id: <20170616072122.17504-2-sebastian.huber@embedded-brains.de> X-Mailer: git-send-email 2.12.3 In-Reply-To: <20170616072122.17504-1-sebastian.huber@embedded-brains.de> References: <20170616072122.17504-1-sebastian.huber@embedded-brains.de> X-Authenticated-Sender: smtp-embedded@poldinet.de X-Virus-Scanned: Clear (ClamAV 0.99.2/23478/Fri Jun 16 06:12:21 2017) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 07:47:18 -0000 Split up cfi_write_block() into re-usable functions to make it easier to use this code in other systems. --- sys/dev/cfi/cfi_core.c | 199 ++++++++++++++++++++++++++++--------------------- 1 file changed, 114 insertions(+), 85 deletions(-) diff --git a/sys/dev/cfi/cfi_core.c b/sys/dev/cfi/cfi_core.c index e90db06dcef..be596a7e4ab 100644 --- a/sys/dev/cfi/cfi_core.c +++ b/sys/dev/cfi/cfi_core.c @@ -570,121 +570,134 @@ cfi_wait_ready(struct cfi_softc *sc, u_int ofs, sbintime_t start, return (error); } -int -cfi_write_block(struct cfi_softc *sc) +static void +cfi_unlock_flash(struct cfi_softc *sc, u_int ofs) { - union { - uint8_t *x8; - uint16_t *x16; - uint32_t *x32; - } ptr, cpyprt; - register_t intr; - int error, i, neederase = 0; - uint32_t st; - u_int wlen; - sbintime_t start; /* Intel flash must be unlocked before modification */ switch (sc->sc_cmdset) { case CFI_VEND_INTEL_ECS: case CFI_VEND_INTEL_SCS: - cfi_write(sc, sc->sc_wrofs, CFI_INTEL_LBS); - cfi_write(sc, sc->sc_wrofs, CFI_INTEL_UB); - cfi_write(sc, sc->sc_wrofs, CFI_BCS_READ_ARRAY); + cfi_write(sc, ofs, CFI_INTEL_LBS); + cfi_write(sc, ofs, CFI_INTEL_UB); + cfi_write(sc, ofs, CFI_BCS_READ_ARRAY); break; } +} - /* Check if an erase is required. */ - for (i = 0; i < sc->sc_wrbufsz; i++) - if ((sc->sc_wrbuf[i] & sc->sc_wrbufcpy[i]) != sc->sc_wrbuf[i]) { - neederase = 1; - break; - } +static void +cfi_relock_flash(struct cfi_softc *sc, u_int ofs) +{ - if (neederase) { - intr = intr_disable(); - start = sbinuptime(); - /* Erase the block. */ - switch (sc->sc_cmdset) { - case CFI_VEND_INTEL_ECS: - case CFI_VEND_INTEL_SCS: - cfi_write(sc, sc->sc_wrofs, CFI_BCS_BLOCK_ERASE); - cfi_write(sc, sc->sc_wrofs, CFI_BCS_CONFIRM); - break; - case CFI_VEND_AMD_SCS: - case CFI_VEND_AMD_ECS: - cfi_amd_write(sc, sc->sc_wrofs, AMD_ADDR_START, - CFI_AMD_ERASE_SECTOR); - cfi_amd_write(sc, sc->sc_wrofs, 0, CFI_AMD_BLOCK_ERASE); - break; - default: - /* Better safe than sorry... */ - intr_restore(intr); - return (ENODEV); - } + /* Relock Intel flash */ + switch (sc->sc_cmdset) { + case CFI_VEND_INTEL_ECS: + case CFI_VEND_INTEL_SCS: + cfi_write(sc, ofs, CFI_INTEL_LBS); + cfi_write(sc, ofs, CFI_INTEL_LB); + cfi_write(sc, ofs, CFI_BCS_READ_ARRAY); + break; + } +} + +static int +cfi_erase_block(struct cfi_softc *sc, u_int ofs) +{ + register_t intr; + sbintime_t start; + + intr = intr_disable(); + start = sbinuptime(); + /* Erase the block. */ + switch (sc->sc_cmdset) { + case CFI_VEND_INTEL_ECS: + case CFI_VEND_INTEL_SCS: + cfi_write(sc, ofs, CFI_BCS_BLOCK_ERASE); + cfi_write(sc, ofs, CFI_BCS_CONFIRM); + break; + case CFI_VEND_AMD_SCS: + case CFI_VEND_AMD_ECS: + cfi_amd_write(sc, ofs, AMD_ADDR_START, + CFI_AMD_ERASE_SECTOR); + cfi_amd_write(sc, ofs, 0, CFI_AMD_BLOCK_ERASE); + break; + default: + /* Better safe than sorry... */ intr_restore(intr); - error = cfi_wait_ready(sc, sc->sc_wrofs, start, - CFI_TIMEOUT_ERASE); - if (error) - goto out; - } else - error = 0; + return (ENODEV); + } + intr_restore(intr); + return (cfi_wait_ready(sc, ofs, start, CFI_TIMEOUT_ERASE)); +} + +static int +cfi_program_block(struct cfi_softc *sc, u_int ofs, const u_char *wrbuf, + const u_char *wrbufcpy, u_int wrbufsz, int neederase) +{ + union { + const uint8_t *x8; + const uint16_t *x16; + const uint32_t *x32; + } ptr, cpyprt; + register_t intr; + int error = 0; + uint32_t st; + u_int i, wlen; + sbintime_t start; /* Write the block using a multibyte write if supported. */ - ptr.x8 = sc->sc_wrbuf; - cpyprt.x8 = sc->sc_wrbufcpy; + ptr.x8 = wrbuf; + cpyprt.x8 = wrbufcpy; if (sc->sc_maxbuf > sc->sc_width) { switch (sc->sc_cmdset) { case CFI_VEND_INTEL_ECS: case CFI_VEND_INTEL_SCS: - for (i = 0; i < sc->sc_wrbufsz; i += wlen) { - wlen = MIN(sc->sc_maxbuf, sc->sc_wrbufsz - i); + for (i = 0; i < wrbufsz; i += wlen) { + wlen = MIN(sc->sc_maxbuf, wrbufsz - i); intr = intr_disable(); start = sbinuptime(); do { - cfi_write(sc, sc->sc_wrofs + i, + cfi_write(sc, ofs + i, CFI_BCS_BUF_PROG_SETUP); if (sbinuptime() > start + sc->sc_max_timeouts[CFI_TIMEOUT_BUFWRITE]) { intr_restore(intr); - error = ETIMEDOUT; - goto out; + return (ETIMEDOUT); } - st = cfi_read(sc, sc->sc_wrofs + i); + st = cfi_read(sc, ofs + i); } while (! (st & CFI_INTEL_STATUS_WSMS)); - cfi_write(sc, sc->sc_wrofs + i, + cfi_write(sc, ofs + i, (wlen / sc->sc_width) - 1); switch (sc->sc_width) { case 1: bus_space_write_region_1(sc->sc_tag, - sc->sc_handle, sc->sc_wrofs + i, + sc->sc_handle, ofs + i, ptr.x8 + i, wlen); break; case 2: bus_space_write_region_2(sc->sc_tag, - sc->sc_handle, sc->sc_wrofs + i, + sc->sc_handle, ofs + i, ptr.x16 + i / 2, wlen / 2); break; case 4: bus_space_write_region_4(sc->sc_tag, - sc->sc_handle, sc->sc_wrofs + i, + sc->sc_handle, ofs + i, ptr.x32 + i / 4, wlen / 4); break; } - cfi_write(sc, sc->sc_wrofs + i, - CFI_BCS_CONFIRM); + cfi_write(sc, ofs + i, CFI_BCS_CONFIRM); intr_restore(intr); - error = cfi_wait_ready(sc, sc->sc_wrofs + i, - start, CFI_TIMEOUT_BUFWRITE); + error = cfi_wait_ready(sc, ofs + i, start, + CFI_TIMEOUT_BUFWRITE); if (error != 0) - goto out; + return (error); } - goto out; + return (error); default: /* Fall through to single word case */ break; @@ -693,7 +706,7 @@ cfi_write_block(struct cfi_softc *sc) } /* Write the block one byte/word at a time. */ - for (i = 0; i < sc->sc_wrbufsz; i += sc->sc_width) { + for (i = 0; i < wrbufsz; i += sc->sc_width) { /* Avoid writing unless we are actually changing bits */ if (!neederase) { @@ -724,7 +737,7 @@ cfi_write_block(struct cfi_softc *sc) switch (sc->sc_cmdset) { case CFI_VEND_INTEL_ECS: case CFI_VEND_INTEL_SCS: - cfi_write(sc, sc->sc_wrofs + i, CFI_BCS_PROGRAM); + cfi_write(sc, ofs + i, CFI_BCS_PROGRAM); break; case CFI_VEND_AMD_SCS: case CFI_VEND_AMD_ECS: @@ -734,40 +747,56 @@ cfi_write_block(struct cfi_softc *sc) switch (sc->sc_width) { case 1: bus_space_write_1(sc->sc_tag, sc->sc_handle, - sc->sc_wrofs + i, *(ptr.x8 + i)); + ofs + i, *(ptr.x8 + i)); break; case 2: bus_space_write_2(sc->sc_tag, sc->sc_handle, - sc->sc_wrofs + i, *(ptr.x16 + i / 2)); + ofs + i, *(ptr.x16 + i / 2)); break; case 4: bus_space_write_4(sc->sc_tag, sc->sc_handle, - sc->sc_wrofs + i, *(ptr.x32 + i / 4)); + ofs + i, *(ptr.x32 + i / 4)); break; } - + intr_restore(intr); - error = cfi_wait_ready(sc, sc->sc_wrofs, start, - CFI_TIMEOUT_WRITE); + error = cfi_wait_ready(sc, ofs, start, CFI_TIMEOUT_WRITE); if (error) - goto out; + return (error); } /* error is 0. */ + return (error); +} - out: - cfi_reset_default(sc); +int +cfi_write_block(struct cfi_softc *sc) +{ + int error, neederase = 0; + u_int i; - /* Relock Intel flash */ - switch (sc->sc_cmdset) { - case CFI_VEND_INTEL_ECS: - case CFI_VEND_INTEL_SCS: - cfi_write(sc, sc->sc_wrofs, CFI_INTEL_LBS); - cfi_write(sc, sc->sc_wrofs, CFI_INTEL_LB); - cfi_write(sc, sc->sc_wrofs, CFI_BCS_READ_ARRAY); - break; + cfi_unlock_flash(sc, sc->sc_wrofs); + + /* Check if an erase is required. */ + for (i = 0; i < sc->sc_wrbufsz; i++) + if ((sc->sc_wrbuf[i] & sc->sc_wrbufcpy[i]) != sc->sc_wrbuf[i]) { + neederase = 1; + break; + } + + if (neederase) { + error = cfi_erase_block(sc, sc->sc_wrofs); + if (error) + goto out; } + + error = cfi_program_block(sc, sc->sc_wrofs, sc->sc_wrbuf, + sc->sc_wrbufcpy, sc->sc_wrbufsz, neederase); + + out: + cfi_reset_default(sc); + cfi_relock_flash(sc, sc->sc_wrofs); return (error); } -- 2.12.3 From owner-freebsd-hackers@freebsd.org Fri Jun 16 09:49:37 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69771BFC34B for ; Fri, 16 Jun 2017 09:49:37 +0000 (UTC) (envelope-from karnajitw@gmail.com) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2925878AA2 for ; Fri, 16 Jun 2017 09:49:37 +0000 (UTC) (envelope-from karnajitw@gmail.com) Received: by mail-it0-x229.google.com with SMTP id m62so32337596itc.0 for ; Fri, 16 Jun 2017 02:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=9HCn1ZXu1bXGetV/0N1PQcliPF0xAtoLzjmtAIMqPUM=; b=Cxksj95teiWwlXgIFDdmt6o4yLGR4+vosqjhcgzxQi4nRMoCPi7x4pDsu9B9QQl84Y nkiz+7WUsDjv2JftYxepdcskiSK7+UDI8gEIivQHO+2D1EXdDxmdxtX1rX+v69KPHXKM S/WuDukxvNAOcWhHj41pN4N0+3zmUEVLJGQnlj4kmi5fD18LXclq9kNf066q2RfR9Klk xIiqmrKLz+kLrv5gBR/aR5729INHCwYQEN22UB6Sf1DGPimjRM9DT9rWJtesNzwy2rsd hXz9q4WE/UhXLQPAftVS0duzfFeLZb1Mafs469eFodY+mmtSV05Xqki/0yCNO1KZx56N nomg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=9HCn1ZXu1bXGetV/0N1PQcliPF0xAtoLzjmtAIMqPUM=; b=LJ4D1gBX+7SgP9WloVZtR1CBF0SW9YsyxPmQ0rgE8zw2HsrIuQGpz1m9EKANr/OV7m 5q4UxaTnjBGXwJjrhC0vB6gwFAScQ3CrUjetr246HGOn9fK359IfvOckNKzOzak8hW/K gqwilhsT+p7nQ5zUCkTg5Unv1I9g4Xi7Zuhg67wNzIoDjMLj0I+aDGjaZhX6DfYjKA9r IzqmLuKb+H4U42//9LxgFDGV2jG8A2PsGRNE1N9kq50BHsBPbqY1ZRNY76GbrqJ/1XXA PplvUQGvljXTl/MmDQJ1zmuWQ7PDym9vg/qxfKwO0WyGjjl6KOh8KuhP4n1Iph7jj4yY GeaA== X-Gm-Message-State: AKS2vOzjHS2fgThEjlCMnA6KS/nrXWfaNLQsyPpjXykj96+QAnaRClsI r4nRuMXqH0lMvK64Zau1Gv9i8i9hKi1L X-Received: by 10.36.125.208 with SMTP id b199mr9488537itc.87.1497606576272; Fri, 16 Jun 2017 02:49:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.136.131 with HTTP; Fri, 16 Jun 2017 02:49:35 -0700 (PDT) From: karnajit wangkhem Date: Fri, 16 Jun 2017 15:19:35 +0530 Message-ID: Subject: Undesirable FPU tag word value after PT_SETFPREGS on i386 To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Fri, 16 Jun 2017 11:13:06 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 09:49:37 -0000 Hi All, Please help me understand why NaN is reported after PT_SETFPREGS. As a result of this, if the inferior process contains an float operations, they result in NaN. This behavior is not seen in case of 32/64 bit binary on amd64. My env kern.ostype: FreeBSD kern.osrelease: 11.0-RELEASE-p1 kern.version: FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 03:40:55 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC hw.machine_arch: i386 Result of the below program on an i386 box FPU TAG = ffff FPU TAG = fe00 -> 11 11 11 10 00 00 00 00 --------------------------------------------------------------------------------------------------------------------------- #include #include #include #include #include #include #include int main() { pid_t child_pid = fork(); if (child_pid == 0) { ptrace(PT_TRACE_ME, 0, NULL, 0); execl("/bin/ls", "ls", NULL); perror("Exec failed\n"); exit(1); } int status; int options = 0; if (waitpid(child_pid, &status, options) < 0) { perror("Failed to wait for child process!\n"); exit(1); } if (WIFSTOPPED(status)) { printf("Child has stopped...\n"); } else { perror("Child suppose to stop\n"); exit(1); } //****************************************************// struct fpreg fpregs; ptrace(PT_GETFPREGS, child_pid, (caddr_t)&fpregs, 0); printf("FPU TAG = %04x\n", fpregs.fpr_env[2]); ptrace(PT_SETFPREGS, child_pid, (caddr_t)&fpregs, 0); // <------ ptrace(PT_GETFPREGS, child_pid, (caddr_t)&fpregs, 0); printf("FPU TAG = %04x\n", fpregs.fpr_env[2]); //****************************************************// printf("Send continue to child process\n"); ptrace(PT_CONTINUE, child_pid, (caddr_t)1, 0); if (waitpid(child_pid, &status, options) < 0) { perror("Failed to wait for child process!\n"); exit(1); } if (WIFEXITED(status)) { printf("Child exited\n"); } else { perror("Child suppose to exit\n"); exit(1); } return 0; } Regards, Karan From owner-freebsd-hackers@freebsd.org Fri Jun 16 13:23:36 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F42BC08BD3 for ; Fri, 16 Jun 2017 13:23:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0EDB7EF86 for ; Fri, 16 Jun 2017 13:23:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5GDNURm010143 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 16 Jun 2017 16:23:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5GDNURm010143 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5GDNUaL010142; Fri, 16 Jun 2017 16:23:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 16 Jun 2017 16:23:29 +0300 From: Konstantin Belousov To: karnajit wangkhem Cc: freebsd-hackers@freebsd.org Subject: Re: Undesirable FPU tag word value after PT_SETFPREGS on i386 Message-ID: <20170616132329.GY2088@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 13:23:36 -0000 On Fri, Jun 16, 2017 at 03:19:35PM +0530, karnajit wangkhem wrote: > Hi All, > > Please help me understand why NaN is reported after PT_SETFPREGS. As a > result of this, if the inferior process contains an float operations, they > result in NaN. > This behavior is not seen in case of 32/64 bit binary on amd64. > > My env > kern.ostype: FreeBSD > kern.osrelease: 11.0-RELEASE-p1 > kern.version: FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 03:40:55 UTC > 2016 > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > hw.machine_arch: i386 > > Result of the below program on an i386 box > FPU TAG = ffff > FPU TAG = fe00 -> 11 11 11 10 00 00 00 00 FreeBSD 11 started translating between abridged and full FPU tags, but the translation appeared to be broken. Try this. diff --git a/sys/i386/isa/npx.c b/sys/i386/isa/npx.c index 5ca1ee507fe..de29adc4101 100644 --- a/sys/i386/isa/npx.c +++ b/sys/i386/isa/npx.c @@ -1108,7 +1108,7 @@ npx_fill_fpregs_xmm1(struct savexmm *sv_xmm, struct save87 *sv_87) sv_87->sv_ac[i] = sv_xmm->sv_fp[i].fp_acc; if ((penv_xmm->en_tw & (1 << i)) != 0) /* zero and special are set as valid */ - penv_87->en_tw &= ~(3 << i); + penv_87->en_tw &= ~(3 << i * 2); } } @@ -1143,7 +1143,7 @@ npx_set_fpregs_xmm(struct save87 *sv_87, struct savexmm *sv_xmm) penv_xmm->en_tw = 0; for (i = 0; i < 8; ++i) { sv_xmm->sv_fp[i].fp_acc = sv_87->sv_ac[i]; - if ((penv_87->en_tw && (3 << i)) != (3 << i)) + if ((penv_87->en_tw & (3 << i * 2)) != (3 << i * 2)) penv_xmm->en_tw |= 1 << i; } } From owner-freebsd-hackers@freebsd.org Fri Jun 16 13:28:39 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B44EFC08E27 for ; Fri, 16 Jun 2017 13:28:39 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C02F7F241; Fri, 16 Jun 2017 13:28:39 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from lm0.local (rrcs-96-10-213-43.midsouth.biz.rr.com [96.10.213.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id 399A7239431; Fri, 16 Jun 2017 09:28:38 -0400 (EDT) Subject: Re: crash under load in GENERIC To: Gleb Smirnoff Cc: freebsd-hackers@freebsd.org References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170615201204.GR50023@FreeBSD.org> From: Nikolai Lifanov Message-ID: <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> Date: Fri, 16 Jun 2017 09:28:30 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20170615201204.GR50023@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="DjcOaie9aRw66j7PjjI8wcsPTALrSiFDu" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 13:28:39 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DjcOaie9aRw66j7PjjI8wcsPTALrSiFDu Content-Type: multipart/mixed; boundary="gFR1Xxq5Ieh7poLEU171O93oCTQlplsHF"; protected-headers="v1" From: Nikolai Lifanov To: Gleb Smirnoff Cc: freebsd-hackers@freebsd.org Message-ID: <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> Subject: Re: crash under load in GENERIC References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170615201204.GR50023@FreeBSD.org> In-Reply-To: <20170615201204.GR50023@FreeBSD.org> --gFR1Xxq5Ieh7poLEU171O93oCTQlplsHF Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 6/15/17 4:12 PM, Gleb Smirnoff wrote: > Nikolai, >=20 > On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: > N> I haven't nailed it to a specific commit, but some time after > N> head-319667, my package builder box crashes when running Poudriere. > N> The earliest version that I can confirm crashes is head-319735. >=20 > Should be fixed in r319988. >=20 Gleb, After r319988, my system now crashes on boot. __curthread () at ./machine/pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:232 #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xffffffff80a992c7 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xffffffff80a997c6 in vpanic (fmt=3D, ap=3D0xfffffe2021edf540) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xffffffff80a995e3 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xffffffff80f2f250 in trap_fatal (frame=3D0xfffffe2021edf7e0, eva=3D1= 6) at /usr/src/sys/amd64/amd64/trap.c:801 #6 0xffffffff80f2f2a9 in trap_pfault (frame=3D0xfffffe2021edf7e0, usermo= de=3D0) at /usr/src/sys/amd64/amd64/trap.c:655 #7 0xffffffff80f2eae0 in trap (frame=3D0xfffffe2021edf7e0) at /usr/src/sys/amd64/amd64/trap.c:421 #8 #9 udp_get_inpcbinfo (protocol=3D) at /usr/src/sys/netinet/udp_var.h:157 #10 udp6_attach (so=3D0xfffff8014d648000, proto=3D17, td=3D0xfffff8013e08= 6560) at /usr/src/sys/netinet6/udp6_usrreq.c:973 #11 0xffffffff80b265d1 in socreate (dom=3D, aso=3D, type=3D, proto=3D17, cred=3D0xfffff8011ee32800, td=3D0xfffff8013e086560) at /usr/src/sys/kern/uipc_socket.c:550 #12 0xffffffff80b2e923 in kern_socket (td=3D, domain=3D28,= type=3D2, protocol=3D17) at /usr/src/sys/kern/uipc_syscalls.c:160 #13 0xffffffff80f2fd50 in syscallenter (td=3D) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:132 #14 amd64_syscall (td=3D0xfffff8013e086560, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:903 #15 #16 0x00000008003de3ba in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffe418 - Nikolai --gFR1Xxq5Ieh7poLEU171O93oCTQlplsHF-- --DjcOaie9aRw66j7PjjI8wcsPTALrSiFDu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJIBAEBCAAyFiEE5oT6TcuaWvG5gtjzZ6sv56ecR0UFAllD3QQUHGxpZmFub3ZA ZnJlZWJzZC5vcmcACgkQZ6sv56ecR0XaABAAiLU/hy1w5yGzpds4703afq5tnZ+o SXCHzRbaL4vtz8EWIIah2OThbtV5cAZ6/0JkKwAC/anlVgCC6HaSB1U0/xdUhweA 7E0Ash1rg/YogAOAfaDo9/gqktkzYz+DHzAlvyBHPDI/D8U0SKWe+BYtgM0ORt1V VAzq5FcNpETneeaH5D3x/UXDouhzFfTbzcd3whd4QfFEt8MfBa4DMubYFEOVMFR9 ZzlcWHqxBimCkmp5syQCY8OIEMZj3ZOpGAKjMGumt/bHKfO59y28tzkWm5J4cXMI EaUa92c1VzZPOmGvfXxFpehJsYvl1fiWScNx6LTY5xFWxmY2QU2mlGy+Av8QHzuR t0fjsZIXMGsbzVXK3bvMx0IAAKObpqVwT/ftK5AJUQsAlcQmttnbgpjxqeIUBx5W lGGTAwuqKhMAEfWlrGayEC3lnnu7zR6Rz96cvNMCSnpT4YJ5+kQVkzP6F62jvmBO hKhPbyHpv7nxc2s5e299eUKm1/qGQRZblEXXatFF6KbwhPKbZb3xHcm/cFZxfogz iUkg3zXNgiEJZzmI+dheK11GA6csvSprAYg5MbtrE0Jcd2cQ2drMPGF/RZh8g1xj m5BeyCwseF/FunL63LDPN8Csn+dpgXnNVSFuNvni7rS4EkwkkmMbF/GxuN3DBVnk 1jUA9Z9U9LNzH0I= =/or8 -----END PGP SIGNATURE----- --DjcOaie9aRw66j7PjjI8wcsPTALrSiFDu-- From owner-freebsd-hackers@freebsd.org Fri Jun 16 13:38:07 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FBB6C09203 for ; Fri, 16 Jun 2017 13:38:07 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 761837FA6C; Fri, 16 Jun 2017 13:38:07 +0000 (UTC) (envelope-from lifanov@FreeBSD.org) Received: from lm0.local (rrcs-96-10-213-43.midsouth.biz.rr.com [96.10.213.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id CA4BC239431; Fri, 16 Jun 2017 09:38:06 -0400 (EDT) Subject: Re: crash under load in GENERIC From: Nikolai Lifanov To: Gleb Smirnoff Cc: freebsd-hackers@freebsd.org References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170615201204.GR50023@FreeBSD.org> <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> Message-ID: Date: Fri, 16 Jun 2017 09:38:05 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="U2VXqcJM22I5wMEDXg0h0RBDms6U5n3Vb" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 13:38:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --U2VXqcJM22I5wMEDXg0h0RBDms6U5n3Vb Content-Type: multipart/mixed; boundary="EpXFa6PAME5AcioT8amS6Txfc0eaMEEuo"; protected-headers="v1" From: Nikolai Lifanov To: Gleb Smirnoff Cc: freebsd-hackers@freebsd.org Message-ID: Subject: Re: crash under load in GENERIC References: <2cbb32a6-4721-544c-ddd1-c9cfa6633a79@FreeBSD.org> <20170615201204.GR50023@FreeBSD.org> <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> In-Reply-To: <96025ad9-48da-3391-7bad-d2e72a35d13e@FreeBSD.org> --EpXFa6PAME5AcioT8amS6Txfc0eaMEEuo Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 6/16/17 9:28 AM, Nikolai Lifanov wrote: > On 6/15/17 4:12 PM, Gleb Smirnoff wrote: >> Nikolai, >> >> On Wed, Jun 14, 2017 at 10:36:20AM -0400, Nikolai Lifanov wrote: >> N> I haven't nailed it to a specific commit, but some time after >> N> head-319667, my package builder box crashes when running Poudriere.= >> N> The earliest version that I can confirm crashes is head-319735. >> >> Should be fixed in r319988. >> >=20 > Gleb, >=20 > After r319988, my system now crashes on boot. >=20 Never mind, false alarm. I nuked my obj and installed a clean environment at r319988 and now my system boots and works. - Nikolai --EpXFa6PAME5AcioT8amS6Txfc0eaMEEuo-- --U2VXqcJM22I5wMEDXg0h0RBDms6U5n3Vb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJIBAEBCAAyFiEE5oT6TcuaWvG5gtjzZ6sv56ecR0UFAllD3z0UHGxpZmFub3ZA ZnJlZWJzZC5vcmcACgkQZ6sv56ecR0XD9A/9E9fL1ySKhvN98bHPSWO8NVPK8Me7 Jqpw4Ba1v6oVXc6sXCcOFl+Wl85kWWj+t2lBGOfflNXxokAtuxlynWzPbqfHQ1Hy NPDCiBxghwr6VjmYzSNa03C1Fp+wZb4CmimsiX22d8O7hYtGQkQchbEfhBLesSCX ruj3FG0fDUNf7r+RpbnnJdlRjKClV7sxEsWiR/P/7tCAgj+eNRec+UxWjtBSdDI/ MVgHFAV2IEcILoETviziCQmBimAfB8xmJ6D6dU9Quhk76hJDlhltILwuh1Ery7zt 3Ir+oRtBDaAGDlmDmYhaplIJfnqVFSqWNvB9C1SK13+sAT9ORdfopc5pdcZeDpZn pIchBnpAEc7L9pYzWEGVbAH7PivOgF1VO2dfOUjo4EOFmpini1aJ6c28FZPi58VO G+3G/kCp4eNsboIu6GnEbdgS/KqQaTG0egszJyQ5fzBLTvs4DDGhB0r3+BtsP7FX zg7eNSMNGp5R+2/yvxenGosw0RaTojbxX58H/CSI+8+YaW9iMMR7gJlslp2wqO2c J1s9SviJRXlMYXTSPymRKorE4X62d2ZXzl8s/RBN/2Knj33kc5Cj0dA3GzJpiDGC xL4vrjPRVhnMFzQHPhnm6bLPzNVp5XXvBzjbIYd6ig0g5M5s8qGrESeIrSoeLYQy 54GKGN1ksVGU3c8= =qoaq -----END PGP SIGNATURE----- --U2VXqcJM22I5wMEDXg0h0RBDms6U5n3Vb-- From owner-freebsd-hackers@freebsd.org Fri Jun 16 16:56:48 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CC73C78410 for ; Fri, 16 Jun 2017 16:56:48 +0000 (UTC) (envelope-from aplattner@nvidia.com) Received: from hqemgate14.nvidia.com (hqemgate14.nvidia.com [216.228.121.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hqemgate14.nvidia.com", Issuer "RapidSSL SHA256 CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46AE22E1F for ; Fri, 16 Jun 2017 16:56:47 +0000 (UTC) (envelope-from aplattner@nvidia.com) Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqemgate14.nvidia.com id ; Fri, 16 Jun 2017 09:51:32 -0700 Received: from HQMAIL105.nvidia.com ([172.20.13.39]) by hqpgpgate102.nvidia.com (PGP Universal service); Fri, 16 Jun 2017 09:51:42 -0700 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Fri, 16 Jun 2017 09:51:42 -0700 Received: from HQMAIL102.nvidia.com (172.18.146.10) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Fri, 16 Jun 2017 16:51:41 +0000 Received: from tenor.nvidia.com (172.20.13.39) by HQMAIL102.nvidia.com (172.18.146.10) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Fri, 16 Jun 2017 16:51:40 +0000 To: X-Nvconfidentiality: public From: Aaron Plattner Subject: efifb framebuffer info for NVIDIA driver console restore Message-ID: Date: Fri, 16 Jun 2017 09:51:40 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [172.20.13.39] X-ClientProxiedBy: HQMAIL106.nvidia.com (172.18.146.12) To HQMAIL102.nvidia.com (172.18.146.10) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 16:56:48 -0000 Hi FreeBSD hackers, My name is Aaron Plattner. I'm a driver developer at NVIDIA working on the FreeBSD driver, among other things. I was hoping you could help me out regarding some changes I've been making to the driver. Recently, we've revamped how our driver handles switching from graphical modes to the console mode. For framebuffer consoles, the new nvidia_modeset module tries to take care of it without having to fall back to old-school VESA VBE modesets. However, in order for this to work, the driver needs to know where the framebuffer console is in physical memory, and its layout. On Linux, we get this information from the global 'screen_info' structure: void NV_API_CALL os_get_screen_info( NvU64 *pPhysicalAddress, NvU16 *pFbWidth, NvU16 *pFbHeight, NvU16 *pFbDepth, NvU16 *pFbPitch ) { // // If there is not a framebuffer console, return 0 size. // // orig_video_isVGA is set to 1 during early Linux kernel // initialization, and then will be set to a value, such as // VIDEO_TYPE_VLFB or VIDEO_TYPE_EFI if an fbdev console is used. // if (screen_info.orig_video_isVGA <= 1) { *pPhysicalAddress = 0; *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; return; } *pPhysicalAddress = screen_info.lfb_base; #if defined(VIDEO_CAPABILITY_64BIT_BASE) *pPhysicalAddress |= (NvU64)screen_info.ext_lfb_base << 32; #endif *pFbWidth = screen_info.lfb_width; *pFbHeight = screen_info.lfb_height; *pFbDepth = screen_info.lfb_depth; *pFbPitch = screen_info.lfb_linelength; } This works for both legacy boot systems with vesafb as well as UEFI systems that use Linux's efifb. On FreeBSD, I was able to find this information on my legacy system: void NV_API_CALL os_get_screen_info( NvU64 *pPhysicalAddress, NvU16 *pFbWidth, NvU16 *pFbHeight, NvU16 *pFbDepth, NvU16 *pFbPitch ) { const sc_softc_t *sc = sc_get_softc(0, SC_KERNEL_CONSOLE); if (sc) { const video_adapter_t *adp = sc->adp; if (adp) { const struct video_info *vi = &adp->va_info; if (vi && (vi->vi_flags & V_INFO_LINEAR)) { *pPhysicalAddress = vi->vi_buffer; *pFbWidth = vi->vi_width; *pFbHeight = vi->vi_height; *pFbDepth = vi->vi_depth; *pFbPitch = adp->va_line_width; return; } } } *pPhysicalAddress = 0; *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; } However, this doesn't work on UEFI systems because efifb doesn't go through the video_adapter_t stuff. Does anyone know how I can get that information from efifb, or who to talk to about adding an interface the driver can use to find it? Sincerely, Aaron From owner-freebsd-hackers@freebsd.org Fri Jun 16 17:21:42 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F4B6C78B9D for ; Fri, 16 Jun 2017 17:21:42 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: from mail-vk0-x232.google.com (mail-vk0-x232.google.com [IPv6:2607:f8b0:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26AA93D80 for ; Fri, 16 Jun 2017 17:21:42 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: by mail-vk0-x232.google.com with SMTP id 191so25685625vko.2 for ; Fri, 16 Jun 2017 10:21:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=jVGkpyTj/dgDWQwZXj0+rp8XClAeTpPC779MoCOykNM=; b=gYAQ8B+gFbz9syHIki1ZeXRCnJzKahGPQ38Dwx015IG4+UOiyqwrn2D6bQOj0sStup gYAqdoLXvjVjwBmc35wX/qO4/qO4j+3nkbxIbUz2AMQCE09B2u7E7AeSgCEeIxSQ58MH aDdpaq4z36bk+o+FskKTe2gj6auYYx1qtFn7pblkSnYdabzg1nFbpCd20555r+4hlmID YwGJik+QZQ+M/4dQwzkdQZWCzxFPPv/GIXlrTO1OYV96I8mYbDun37xk/TxHgsZAFvhJ Up9NxXgCphLPUHPvK6vrW7Z0uycs0K/Ct1omNa+LY6T/b105zRXT6Nci65hv6TX+QMU4 O19A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=jVGkpyTj/dgDWQwZXj0+rp8XClAeTpPC779MoCOykNM=; b=DdnCPUMz0m8o3nef4yym7jiuAGyf9amucbos4Q/RFFiSnu0cOFge5APTkopIE91mVP IgpLMDUdNzNqzijOYV37lB7ectaDTcsunXoyU/0LIRXDgpggz1I6v4cgHAYg8BMr85XI zpKpa+U5j0SbGN+cPyCHc4FYRpie9vLkjkAZRMcuMKOykammKqNSIGFKemu9qH4/SjLR IkDaBJppPFSXp8BEdakVSK0a6Mg8tn6O8YEygY9clj7kmE166hEQcuzc6lcfsiukwehs 1TodwI4OhY6xat1q1Ah++TcIrMfdjtME6IxsWNLfm6/z2VBA7wJLMy/QINMASM/90ea/ NIyg== X-Gm-Message-State: AKS2vOzMbktIC/4UNsCMqbUspO4HukY5vGyHgqm/5Cf2CEatSfO8JXqu DynBDPgkLZ4vv5zQ0VMIQb5GHd/TzqQ2 X-Received: by 10.31.69.138 with SMTP id s132mr7065901vka.13.1497633701251; Fri, 16 Jun 2017 10:21:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.36.195 with HTTP; Fri, 16 Jun 2017 10:21:40 -0700 (PDT) In-Reply-To: References: From: Oliver Pinter Date: Fri, 16 Jun 2017 19:21:40 +0200 Message-ID: Subject: Re: efifb framebuffer info for NVIDIA driver console restore To: Aaron Plattner Cc: "freebsd-hackers@freebsd.org" , Ed Maste , Aleksandr Rybalko Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 17:21:42 -0000 CC: Ray and Ed. On Friday, June 16, 2017, Aaron Plattner wrote: > Hi FreeBSD hackers, > > My name is Aaron Plattner. I'm a driver developer at NVIDIA working on > the FreeBSD driver, among other things. I was hoping you could help me > out regarding some changes I've been making to the driver. > > Recently, we've revamped how our driver handles switching from > graphical modes to the console mode. For framebuffer consoles, the new > nvidia_modeset module tries to take care of it without having to fall > back to old-school VESA VBE modesets. However, in order for this to > work, the driver needs to know where the framebuffer console is in > physical memory, and its layout. > > On Linux, we get this information from the global 'screen_info' > structure: > > void NV_API_CALL os_get_screen_info( > NvU64 *pPhysicalAddress, > NvU16 *pFbWidth, > NvU16 *pFbHeight, > NvU16 *pFbDepth, > NvU16 *pFbPitch > ) > { > // > // If there is not a framebuffer console, return 0 size. > // > // orig_video_isVGA is set to 1 during early Linux kernel > // initialization, and then will be set to a value, such as > // VIDEO_TYPE_VLFB or VIDEO_TYPE_EFI if an fbdev console is used. > // > if (screen_info.orig_video_isVGA <= 1) > { > *pPhysicalAddress = 0; > *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; > return; > } > > *pPhysicalAddress = screen_info.lfb_base; > #if defined(VIDEO_CAPABILITY_64BIT_BASE) > *pPhysicalAddress |= (NvU64)screen_info.ext_lfb_base << 32; > #endif > *pFbWidth = screen_info.lfb_width; > *pFbHeight = screen_info.lfb_height; > *pFbDepth = screen_info.lfb_depth; > *pFbPitch = screen_info.lfb_linelength; > } > > This works for both legacy boot systems with vesafb as well as UEFI > systems that use Linux's efifb. > > On FreeBSD, I was able to find this information on my legacy system: > > void NV_API_CALL os_get_screen_info( > NvU64 *pPhysicalAddress, > NvU16 *pFbWidth, > NvU16 *pFbHeight, > NvU16 *pFbDepth, > NvU16 *pFbPitch > ) > { > const sc_softc_t *sc = sc_get_softc(0, SC_KERNEL_CONSOLE); > > if (sc) > { > const video_adapter_t *adp = sc->adp; > > if (adp) > { > const struct video_info *vi = &adp->va_info; > > if (vi && (vi->vi_flags & V_INFO_LINEAR)) > { > *pPhysicalAddress = vi->vi_buffer; > *pFbWidth = vi->vi_width; > *pFbHeight = vi->vi_height; > *pFbDepth = vi->vi_depth; > *pFbPitch = adp->va_line_width; > return; > } > } > } > > *pPhysicalAddress = 0; > *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; > } > > However, this doesn't work on UEFI systems because efifb doesn't go > through the video_adapter_t stuff. Does anyone know how I can get that > information from efifb, or who to talk to about adding an interface > the driver can use to find it? > > Sincerely, > Aaron > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > " > From owner-freebsd-hackers@freebsd.org Fri Jun 16 17:34:23 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0D2AC790EC; Fri, 16 Jun 2017 17:34:23 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B26B664784; Fri, 16 Jun 2017 17:34:23 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v5GHYM45038833 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 16 Jun 2017 10:34:22 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v5GHYMFh038832; Fri, 16 Jun 2017 10:34:22 -0700 (PDT) (envelope-from sgk) Date: Fri, 16 Jun 2017 10:34:22 -0700 From: Steve Kargl To: Mathieu Arnold Cc: freebsd-ports@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: JAVA_VERSION??? Message-ID: <20170616173422.GA38774@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu References: <20170615234533.GA2871@troutmask.apl.washington.edu> <582b49d8-5910-35b5-40ed-307475830a64@FreeBSD.org> <20170616144318.GA11287@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 17:34:23 -0000 On Fri, Jun 16, 2017 at 05:07:30PM +0200, Mathieu Arnold wrote: > Le 16/06/2017 à 16:43, Steve Kargl a écrit : > > On Fri, Jun 16, 2017 at 02:17:03PM +0200, Mathieu Arnold wrote: > >> Le 16/06/2017 à 01:45, Steve Kargl a écrit : > >>> Is this the expected behavior? > >>> > >>> % cd ports/math/octave > >>> % make fetch > >>> octave-4.0.3_5: Makefile error: "1.7+" is not a valid value for JAVA_VERSION. It should be one or more of: 1.6 1.7 1.8 (with an optional "+" suffix.) > >>> % grep JAVA_VERSION Makefile > >>> JAVA_VERSION= 1.7+ > >> > >> Not really. What OSVERSION are you running? > >> > >> What is the content of the _JAVA_VERSION_LIST and > >> _JAVA_VERSION_LIST_REGEXP variables ? > >> > > I'm running a month old freebsd-current. > > > > % uname -a > > FreeBSD troutmask.apl.washington.edu 12.0-CURRENT FreeBSD 12.0-CURRENT > > #0 r318546: Fri May 19 12:51:04 PDT 2017 > > > > % cd /usr/ports/math/octave > > % make -d v fetch |& grep JAVA_VERSION > > $ make -V _JAVA_VERSION_LIST > 1.6 1.7 1.8 1.6+ 1.7+ 1.8+ > $ make -V _JAVA_VERSION_LIST_REGEXP > 1.6\|1.7\|1.8\|1.6+\|1.7+\|1.8+ > > You should get the same as me. I wonder if it is not a difference in how > grep behaves on 12 where it does not match 1.7+ with the regexp in there. > Yes, it appears to be a grep issue. In Mk/bsd.java.mk, I removed the @ in line 227 to see what the test did. I then try to reproduce just the part with grep (without the -q option) at a Bourne shell prompt. # echo "1.7+" 1.7+ # echo "1.7+" | /usr/bin/tr " " "\n" 1.7+ # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep "1.6\|1.7\|1.8\|1.6+\|1.7+\|1.8+" # So, nothing is returned. If I reduce the regex pattern, I see # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep "1.7+" 1.7+ -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-hackers@freebsd.org Fri Jun 16 17:41:21 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 143F0C7936F; Fri, 16 Jun 2017 17:41:21 +0000 (UTC) (envelope-from kevans91@ksu.edu) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0040.outbound.protection.outlook.com [104.47.36.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9036764B59; Fri, 16 Jun 2017 17:41:20 +0000 (UTC) (envelope-from kevans91@ksu.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ksu.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=XARcVx9J+qfEbD6JG0nSrXsZsgfqgcavvlfgU6Zo/a4=; b=S4JkJbMFXA4Fg7jpKqi1ibii7kjHOY3gYGhmWnJpcUfd9Ek0n8wHuPATVOyh0/AzUOLyj3T7Y5eaTvFkLdRe4eEn4+0i4HzVSM+HQqcnfGcYInkyR1fiADf4f69aUin1IEJnDsZEoEQVeKI9Z/SBvsO/61XFaB7LbdBXCRqnnOw= Received: from BY1PR0501CA0010.namprd05.prod.outlook.com (10.162.139.20) by CY1PR0501MB1722.namprd05.prod.outlook.com (10.163.140.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.10; Fri, 16 Jun 2017 17:41:18 +0000 Received: from SN1NAM02FT007.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e44::207) by BY1PR0501CA0010.outlook.office365.com (2a01:111:e400:4821::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6 via Frontend Transport; Fri, 16 Jun 2017 17:41:18 +0000 Authentication-Results: spf=pass (sender IP is 129.130.18.151) smtp.mailfrom=ksu.edu; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ksu.edu; Received-SPF: Pass (protection.outlook.com: domain of ksu.edu designates 129.130.18.151 as permitted sender) receiver=protection.outlook.com; client-ip=129.130.18.151; helo=ome-vm-smtp1.campus.ksu.edu; Received: from ome-vm-smtp1.campus.ksu.edu (129.130.18.151) by SN1NAM02FT007.mail.protection.outlook.com (10.152.72.88) with Microsoft SMTP Server id 15.1.1157.12 via Frontend Transport; Fri, 16 Jun 2017 17:41:18 +0000 Received: from calypso.engg.ksu.edu (calypso.engg.ksu.edu [129.130.43.181]) by ome-vm-smtp1.campus.ksu.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id v5GHfHP6002003; Fri, 16 Jun 2017 12:41:17 -0500 Received: by calypso.engg.ksu.edu (Postfix, from userid 110) id A12E23DD1; Fri, 16 Jun 2017 12:41:17 -0500 (CDT) Received: from mail-vk0-f48.google.com (mail-vk0-f48.google.com [209.85.213.48]) by calypso.engg.ksu.edu (Postfix) with ESMTPA id 7AF0B3DC5; Fri, 16 Jun 2017 12:41:15 -0500 (CDT) Received: by mail-vk0-f48.google.com with SMTP id y70so25962075vky.3; Fri, 16 Jun 2017 10:41:15 -0700 (PDT) X-Gm-Message-State: AKS2vOzatt90owajs4JU894yHIaSb0HrX72Ln3UO33+k2OE+G/uw4QZ1 Cdb3vTNb9d5Uzt8NBg2n6qfSwBWZRw== X-Received: by 10.31.69.138 with SMTP id s132mr7110034vka.13.1497634874297; Fri, 16 Jun 2017 10:41:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.86.86 with HTTP; Fri, 16 Jun 2017 10:40:53 -0700 (PDT) In-Reply-To: <20170616173422.GA38774@troutmask.apl.washington.edu> References: <20170615234533.GA2871@troutmask.apl.washington.edu> <582b49d8-5910-35b5-40ed-307475830a64@FreeBSD.org> <20170616144318.GA11287@troutmask.apl.washington.edu> <20170616173422.GA38774@troutmask.apl.washington.edu> From: Kyle Evans Date: Fri, 16 Jun 2017 12:40:53 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: JAVA_VERSION??? To: CC: Mathieu Arnold , , ports-list freebsd X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:129.130.18.151; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39400400002)(39450400003)(39840400002)(39410400002)(39850400002)(39860400002)(2980300002)(438002)(199003)(189002)(24454002)(54094003)(377454003)(54906002)(606005)(229853002)(55446002)(61726006)(236005)(9686003)(478600001)(2171002)(7116003)(512874002)(75432002)(42186005)(7906003)(305945005)(53386004)(38730400002)(2950100002)(110136004)(6246003)(6306002)(8676002)(59536001)(84326002)(63696999)(76176999)(966005)(8576002)(54356999)(498394004)(8936002)(61266001)(50986999)(356003)(93516999)(53546009)(4326008)(5660300001)(9896002)(106466001)(6862004)(189998001)(93886004)(1680700002)(2906002)(86362001)(46386002)(450100002)(2351001)(45336002)(88552002)(80390200001)(55456009); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0501MB1722; H:ome-vm-smtp1.campus.ksu.edu; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; SN1NAM02FT007; 1:3ayHzBH5nOwdyvXdoOGGApbnN/XsBWVDZMgYn9Xu2k5+uLOPRWFFduRFSTXJndJCPyxquTiJHBgxPqA+Z/t4e9T3je/O36QPXIkaxEcgUKsAeb1AGRTZ6bk38N/SACf+o7xOSzcTH7fJNvSSKF9T69lHjdCIKQWjGbmQQqJm5ACwWhXqxiNJdtMqq7NpblgNrGGLyTL6xNhH5q4f2y3DTWHA99GxDKxkQxlYLqwzLrwJEJrNRWwWYUUNw8S/K3q8n3TGIxWotcptv9tFj79QG8xXPscFhL70eNIQ4dGdzntD5VY4qRGwdHiPSyFc4IrxTf47m6BHMv9NCaVilw2w+U9aICWjA97/DwZ+luUSdXfZtTY2KchqZwOd0nhRISvOcRAIjyAqutC4HSd0JSjG/0DajRzABkoJhDDN5bmytj8n/jCR8LxaexwBg6Kurioca+Fslpf4UB1i8E9A+kwoA3AHptryLMiAF6WESSYRGJpcqPfIGOCDb/A3O5yusp2DdO0uXPZEYUUIwgU4LaNngUo/zqu0R6XEvelgRoIGh+rIkYgGtlFjq5dO/bkjqWqp X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fdf9cfaa-35ef-42a0-2a35-08d4b4dee4f7 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:CY1PR0501MB1722; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 3:ldbvt4ByF/n4tzAhkjOeXnw4G/Hbh+8Ngr6H3b4Xv6d9BnetuA/S897D0OB3gCLbArvlMQHBA34UGybvvFkbXPUPk8gRCfDmR6ivhkawMDz/pVjmpKLRiXYPTxWBuSSseXLsgVMSuWWTIiW09mP1hcYOMNzUbSVC5l4IBoJ1eBnINYE8uml/bO0QZC4ka0j88xi9bW7sQjhR6AI+C5JIQnaoPUrEHY+K9ambZbBcfPRUfpo3YIoh5Ik0uCZ4Tl9mDucbkhZXSfF4k40/ol+i9NJ4zVjFKsLrsvZKmdA4uZUQA/bMYhjwXsyW6982lo1osuRr8ioXKmY8U+Kb2GoP2b47eNUFkzI8+7+OmvNSm0gE8qH7yBmIJUbnavhH3CcXTJekf52IQ5ntVIpsDeGsFHoGMFZxz0UCz4Njraz4nJxWq1qTTQAaMBI7mc47DsUTAEHd9icScwHWmhjVzb1cHLISRIInTWrIzokpgpfgCWLNUTbFMthjKwwZ29L0Jj1c X-MS-TrafficTypeDiagnostic: CY1PR0501MB1722: X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 25:/X9jiI08xmaztthVhHNUg4kDi0xmwGR5crufUBt8LNcjp0LNsNzdA3oJlbExWLax8QgWsNumvtOofGtxFJUsrfUTUkaiZYzIh1Sx2Wny0aV0P/z7WRC6D79bVGa1ozvvfL+9oRg3st3GKUeE6wL6wuhvk+p3RPoFQucG5SlwjdOobxfNdUC4C+TK18Pzs7PC968qDBKX7VLF4Q40Tkon665aCFmohHSuSTPmx9c2Z/2KCHMgjzesERKVZ6xO7yqob5ROgv1jtQo964lmC4wfccDfu2NTppCKrPfclodGbJdcPOenVZ55/T+1JbLJy0p4e/T1diuQ9jcJgq+tuy+12AfbnQlnL3zx/HbZckCwbVReE8S5Ow/+Yt5mzyhpj6OgbIox9xBaqN15QwQf8osvqM7fd5H9WFZb8+j+QVLUeaDntjtSryaRQpFK6KvrC7Vp+tF7amdxhvtjFuBZJL9IVxODb+32bDrjl4Rxpq5XCe0=; 31:Ahwd4givZmcqvn0TpdrOT++2ceucodNOwJPzNwlQpiZwB63/PCI0MAOnoBEWOc5FG1vpTGU/eG02QlPqHR+MRttrbp9T8IR3hzNlmLyeaHmwDIN2R/xGdqgz4A6p2SWn4rIVcdkFv7Npoo6cY3LM/cLOjwYnTCUM8Ms2BcVGEuZ+nNVlHIOFp0DBBgXorKmc5Zvv36dH08l4Of5iClxurF9EOK0dKr8kXT80zQUkYf8K94AhI32sqV4/vNgPRgRYSCXw2BXW1KspkEFv77PM8g== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 20:Ex6LHom2U43P66fKIL3rUNthBzTPgWm1QUoIGrHMPddco7yjgOtqFVmd6vcKV36OaJvjG104460BcghP0zLwtQH5IIy7jKzveJHZHL2bCy20MhDTVT5Ts4rqUuOcpiOO6tsN1YTlrJ4G/OrUoJk5OHHb64h9ckP7vkM4NpnEGIVOqBVtztLS1PILI6LrG7HlzZuQwoCYgcgSdBBT3iQyuJb+/tPnDR6uGEcdtq97XYPZdp4KScWbsuvCQj+LBaD9bkd68Je2tCYNBsacDwOh2QENVBJCqNV88Id6cL57FTHnb0MsIcEXttGstzw6cVLcWMaZDUCuMhR4Qw2ROp0o2nh3EnSsMgPF+JH0cpJQg+pjCC57zUlp8NO85hOYjL1TuVVvxQBrB4zdW2mxxVo2Ki/6oktRO3g8g7urN4pqyFAhq9I/jjpm/P7lJ3fOEqxtDSNhLdMaVvnOLQtuXxVq+7c1qScRMJVhxlmwhp3hGy8LOcMIjhp4iDUBW5PpIPAu X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(127970982066743)(47954115253988); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(13018025)(8121501046)(5005006)(13016025)(93006095)(93004095)(3002001)(100000703101)(100105400095)(10201501046)(6041248)(20161123558100)(20161123562025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123560025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0501MB1722; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0501MB1722; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0501MB1722; 4:I24u9b8kDS0BZ/XkH74BM78yRt4hFdtyKk+s0w0T?= =?us-ascii?Q?0ijgt0IB8JtSB2ocya3OHM0cSDldGuZxFtQGckHKKmwltDe1gAHb9TMKdz1w?= =?us-ascii?Q?lUwPbx+7+87XZjrVHBW10sv6xYIL/gC0eD+GeUiLKrf/COaCL+Z6888Qp1rS?= =?us-ascii?Q?PBjMrNGyFq0bHZ2TXnlqDLkMcvnn8hOgQ8LPP8TEEEmO/mcwNHXItEHzvP2F?= =?us-ascii?Q?c7WTz/fgPHB4nn/A7AWhjAHbMLGkdxcWuh/wZqJ46Am3QzQSexLFG/VJb/zm?= =?us-ascii?Q?EOP9tIipzwtpsabJwbrFPS+O/YO5HhKto7PpM2wdB0fu4Ds0emDmjLzPieDm?= =?us-ascii?Q?wzZADlWIe17afNHtEUGhHLIrHc7dc+CK0pprK33BGm8No80iUlyOkfoH1Rhy?= =?us-ascii?Q?L/MjSHPBPvK02BAqSdoM++j9WDv73nnZntc0WmzhPEt8i3rGvVKxtlsiO1dM?= =?us-ascii?Q?SIKYZsYkQlESdbjjBYzP8K9Ksh6bxFac8RujyW04Otrm2WISbN3k6FMb1iOZ?= =?us-ascii?Q?8Lm7MeySxZmUXvjgmRGu1oIhstUF+5uFLz3itNKBT02MLOkmC5FdG7625tXd?= =?us-ascii?Q?cSuYgN3UzdrKnz+NQ+jer1k6jzMzRd4zQOF1+TKIv1sZK/qWzrRnsYJNtcLE?= =?us-ascii?Q?dHGyfyovFG2q9teWpZ7dY7+2VIGItngt0vnirqVUeC66Ee/zGgtjI+va/QQq?= =?us-ascii?Q?n5ffY81+pEXcM+fvSLgbFqFuEGGM29xpjj2s+evvX/MIuI0s7QpjDgZQXST1?= =?us-ascii?Q?I+GISge4Q9vp0+6YLEfTamuVZQ1foVuaek4LSQhl1JmX5FTYZC7sul/Y0078?= =?us-ascii?Q?kJNF5eZmRTCFWZ96ILspVUC4ScsmJva87RhftgCsoSCQ5N6JWACb/XEG1XJq?= =?us-ascii?Q?ly0xWya+lUknvWPKwA8I8vGTXP4s4VDcUnF6pDKIw6EMhVQficnNjlt5NfEW?= =?us-ascii?Q?nSWawzZOhjoKDNiJsYHHqA/uxN1xyQ18qB1s8xKukSLgwRYdw1GZaX6fl8QE?= =?us-ascii?Q?9/2Rhm5kviU/zMbVFLe56u/er4nmCgm7Y4yKjvDgbU1x7EE/RX3TAZPz14Uo?= =?us-ascii?Q?o914YJwcVt6DrLUI5eyF0/R9+RDMhrV2IIF9dEudaUtS61deKsJVCgx/UhC1?= =?us-ascii?Q?iL3DXxqLwkjWznLkTJ17SXv1+3LN9xA7nJaqcIrUsB5loNL/dQncHOlIA2/e?= =?us-ascii?Q?kt7UnS5DkGdZSbIcwXz6roW7CCZg2/4WJl58SDIvyulkcizU5LSlkceM5iTU?= =?us-ascii?Q?IQtWzN565Xfn277dECclxZ+pJiLYCnWR7+BPX7k7t5PmpEr0Zg80elK+X3ef?= =?us-ascii?Q?fw=3D=3D?= X-Forefront-PRVS: 0340850FCD X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0501MB1722; 23:ndtHdXUaHTsDeHnE8Zkb6EZ6jB9dj5C6kR0ZNen?= =?us-ascii?Q?EZVR2/ma76n4iCbm6xZU/bCyOxRAhgYZeqStskwrm7v0D6oLPBcMfM+LXE0q?= =?us-ascii?Q?4mmzGhdAylhWfVLp/dcIpAgRv3sn4jNwznEXhddHRoIsoZZdvbpunWRk1AFC?= =?us-ascii?Q?VSEr8nU2mxYLC65TWXlFXBM0aNZS6nVkHU3OdJPTiYKfVRkiwqkQmM/YfdKl?= =?us-ascii?Q?766L8VTTaRAiTXh9vENgM5eXGJxkhDg+H8zaVCIW2AhxSjrqLpdCejzOQcd1?= =?us-ascii?Q?WbRhwMlR9en/F4nBagk1wV2FhURt0ZTBeqwk6wIPXPMHxSqgtXzMvQWyvM4n?= =?us-ascii?Q?MVQCgwEybn+iqLrSP7VefzvN9l94RZUFMIFBmcWJ8MIxmVot7WhHAQFBnBx1?= =?us-ascii?Q?O6kds6NjDaSj53y+JLNQImOa8Zk4ACisctThiQzWDLCD6GwJdBWrvW++rQ5a?= =?us-ascii?Q?smD0bnwePvBV6/EOXqZ7WkUh/9B7kupwiEmQS0svs7dSssqPJEC6kJOwmyyp?= =?us-ascii?Q?LFYhn7MRN0NvxGVsCYnREMC/ubQu1qTUJLwLki7tyBiA7J9eZEwL8BLIk8FB?= =?us-ascii?Q?o/UQ9Aen3jNFgIjx/Birqq7B6aXtx7MFkyVjMS3634aCHuT/QPz5pCYPWkpm?= =?us-ascii?Q?DkhvUTWprxRFMBfPpgTCLkfr+Kczvu+ZGqhIDPZGBWpeM7G9JZzRn+WmH4lg?= =?us-ascii?Q?PlOd00+9rIxFMbwwdqj+s75+CmCMGDtpK71B4aOYP4jG8BtZwUxjn2a1Idu7?= =?us-ascii?Q?WtmGlOT/ZKQsYGYu1W5986qBDF6tqC/xdTMuiCiv1bapSgWrh3rf4/3oodUc?= =?us-ascii?Q?++dNtBWh0eLaE3v9XefNmg8HsAD8RVZENB4cIm16yNOmN8OIKVrZcczqJJpx?= =?us-ascii?Q?hjGMEKnRKU89Tw4Sgqb1RKAPJIYPWBeUUad7Lcz/ZftdCQcMIDyECpwO9RL/?= =?us-ascii?Q?PQkxGzW2msdKKX4fXa2Gm9lfVcrQg/PA44rjRRgYduQ8s5QoIdZi2pnrH4Vc?= =?us-ascii?Q?5QzhTvudn2UIJwMKn1B3dm2xvTqMncF3FLy9f4H9YUQIJUpbRvKM1zugiuJG?= =?us-ascii?Q?FibWPfWUBnkHRJHcc1F3QlCaxaJP8ekhiBssDQDr6oQH5MRbaYZgzimRPYHI?= =?us-ascii?Q?njkdNr5bMcGZ9F6Ljqr7KsPadvt36DgYHd9CBAE8jglCrBtVvdgZvPS9Pcd2?= =?us-ascii?Q?dDYkLbYV5tc+cxzVReRLCRbzM+fKqQ24L6mTp+hM2DnYgym8P6OuZOac29Yb?= =?us-ascii?Q?1Z/eXf1Mz7Zcptcxh0IEDruEdcttlncVXt3A4rW/hoeoM7ntfTJc+hPDWePj?= =?us-ascii?Q?il5U5t/QwR9u9IM33gQb6XffHeRbdvd392mQZQ7PDtcJftGLCAL+XnBvRXE8?= =?us-ascii?Q?AKt292J62TMEJrUj9myM5LCa6RGxvqGQ2WyFXpAd4L7pol1BPjrm5n9PaRsB?= =?us-ascii?Q?6AheMFxWfJwLxHGPTutYcP33gNB2IK2CA1Mg+0udXl7z+NAvEbQz+ljvvjOS?= =?us-ascii?Q?gEL8jUH76hTWj3R7erQkXTKxEn1+aYUzJTigVRazDNIuHbnYM9r/owMUe4KL?= =?us-ascii?Q?ts/aAszgLk9PRMRUR4Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0501MB1722; 6:UnfLkowopRfc41hJ4dCK1U/XFiRKejgMyI9dT6em?= =?us-ascii?Q?cesjeNoooZc/MKHgdMZbj6F9dDtOLU9WqY80f0tndbtH8wOSlpHh834tuZhD?= =?us-ascii?Q?pbeZECpziOANJoSlsR9IJ/WShbgssYuaQHZF109+Qyy2abPNN8nayxsogl26?= =?us-ascii?Q?2Ef20pOXSfBN4zmjJcfM6q3uC2Hcg6TwNPNoopqFkFGYYGCLjARrTRtwzgAU?= =?us-ascii?Q?dLTFMA/p7+9e4QTgYTW23Bc+XjgcpWz1Cg/hX4UsIZe03okE/Noa0GWaRs0t?= =?us-ascii?Q?iZukJuWWpfF3jU01BrHYZoX0rkqTX2OUrV4XYEB05S6eRT4rCI1Qq0nDQjvZ?= =?us-ascii?Q?YZjn9FVR+29T+jBWsQ4+QoXCfnb/Hq5qmwdhYCJIqk20aFg6eUGMJ5OKVX+w?= =?us-ascii?Q?+oNurWU0ybcwFXIdTj0kszxheLQAY8Axr3p/d2s9YqNPFYy+hhTOAeFiFglo?= =?us-ascii?Q?qK0Cbm+PcwRdqkGf1XBRZR9zKTDIaEC2qR1OFICVfA9LekqEo91pUNFvA5uY?= =?us-ascii?Q?Wb/TmyfckFAqz85iNvDUow8OkRtV4mBXiZ+8JDDxFiMEHo1DH5B4VtK2NMPe?= =?us-ascii?Q?JwlKgEeWkjELR8gli6/WxA4YcJRvYICx/To/ETKq1vkQx+vjvPYI1avZ8lxv?= =?us-ascii?Q?BRw+lP7ofnh4e5oTqvfIiONWuMPaiJGcc0NjEmVxEl7pr7ImwihhhOnnd0NY?= =?us-ascii?Q?9RpvGebHCgu5fUZj5gMsRCItNKcpZn4OPkdFEJxlnwlnVDvWClQkaPfJZLbs?= =?us-ascii?Q?j/GwxrcKQeN039uOMrgxrirVbn8c3lXoAzJuMVzkMJZFyVw7ZLNyJWR7wZNw?= =?us-ascii?Q?ycpGP4IbdbXBB+GBAmdZB/t0dsbuLz1eZfc/hBI5I5pcPG12q4KAZ8geX1qO?= =?us-ascii?Q?gv0hDiwObjSQo2yF93Ico3tCSbkfefyAkN4/GYqU7u+08z7fFWWbetv9ADxH?= =?us-ascii?Q?r0hUgtsqKV1M5B+8+DXfSQIVDDA3km/u+L7lC7JWqR3D5YGhVRXofQ/vuIbc?= =?us-ascii?Q?KCE=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 5:kUUQmrkPv0ahAWiIPI4wdedQKDUUCGBT7yDG7WKPjT18qmSXuDMs3gm4nydVLYAS+kGq0sn3j2wUxZ3w7Y6YRTKh8IrzMieCa6gs2fud3ybVjL+4hFPTjSTceSB1vjQpC3pfzJ20zszX8gs/BlhAVzU7mik2ACD5EpxVWOOVhfczmYI3LlzMIMtAII+pum7LKYDmyu91bP2fz5fgl+IOVy4zARP7S7sTZMj4teAgAwkZ8ugR5froJpiZjetHsQ6FAhD95zZs1ipNSY7Zt3EFPuF0t4OaCWRs3vydgjkTX2U/wCSAxM1dHs0L7TWOtrk38I1LzRxegrEYFh8jOaCB8lCdZaByI12+9L4ab1fP3k531eP/kluc3kkqATS9WQ7Bt2Q7WtUy/0cqJxn/wwFIFXuYrFN51kZNg9rYwogj/fEJE28RhJkmemy9xZ+PKXAGPb18hc8mkZdSQuWcmUAG2AQdXSScvTvi7mECoghetcEvNQloQqvWKOVQtuP0MEUI; 24:Q/S3XqdoUeuVbiH4+PoBTu0fqy8QHc2FAqLWjUdGZX+X5/4OkZ2u72Z+ADIssx3Q3jjDLJHykQSYoYPeq06UI9Und2CpFm8cj6nyZHx4Tv0= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 7:FZJjaR/561DDRmDwvaBrqOVZvc2Ns1f3L6aO5Jejh0ME9huBhBGBe/z8RjUuuH2BcjvLDdapy/sJuVEQ8SVHwoWThD8okQDaD4NMVby7OJfznopJ6dTRmYbC+HM0MNi8xtENUpDVctvKBXx/KJZetbRtIi7t087tPVU+rfGnIRhfAHEJxEFWGTQS+GMDfGkp4GDXQOf2NUnwLasdAjuvnyLPc7cx9QoKWuiutfu6hlQy+m5k6ahdg01szRRy1NWqLmTXDwCjPlOh/pZMVtH596wUKXX0z9e+eBq3djbUjp7NjB9SJE471KX9RzTVnxm2QyY2qzOYRP6cMVJ25GLYePiawXMYNZrGBDnJ+0YRl6nq3dpgx0bNemb2Y9HWTm6tu5GEU+YcJpi/etdsVrKpSFiDICxs83IcbdazTIMTIE2eyu4PuJnXgXnG7RitJ6uggXRH6SgHtugqv4Xs61OzX0OZT24Q5M+OHWd38JiZwKFY4MuxtSxDwWSpAp/FU06cyOwATTvnffGfSXoBvri54vY/R9wtbtW/DuK+/6mkzxqXtgW6fOGZRobRotVaG7pqha/rcd1YIUfQXAzQX2/y/ofSXlm7mSjb043CKhbxssCybEjLsxH7Jv8d6XMuAyYcqh5oDS98aYL0lTQnWGY5OB6Cb7m802bBfMUpG4lviQdHUtQXRoH9AwiFUOIW9zt3RtUzaqFqxaFoKaR1g8DzWRMOtgfAtAJTYhyP31jmkB7VJUctgREgHcFs+UAFxl9YN/N3DV4wtoLIUWetBG0NorOlQqFgpi0q4p0je/60aGA= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0501MB1722; 20:SS2pL7V8lL88GkTmkrv0O0mdgjd0NKZl6UA6ky8EuJuvOr37EJQgB0OcxSfKExPwDuOFzIJdCwc+Z9xUA0bONEYpqsCW8ID6eOROqWWcRiccasv52oeN9MAo5I5COGBsPTwpUM6U983wjlO9f+UmEhCo7c7XRwHhxcJST09J2/s= X-OriginatorOrg: ksu.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2017 17:41:18.1350 (UTC) X-MS-Exchange-CrossTenant-Id: d9a2fa71-d67d-4cb6-b541-06ccaa8013fb X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=d9a2fa71-d67d-4cb6-b541-06ccaa8013fb; Ip=[129.130.18.151]; Helo=[ome-vm-smtp1.campus.ksu.edu] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0501MB1722 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 17:41:21 -0000 On Fri, Jun 16, 2017 at 12:34 PM, Steve Kargl < sgk@troutmask.apl.washington.edu> wrote: > On Fri, Jun 16, 2017 at 05:07:30PM +0200, Mathieu Arnold wrote: > > Le 16/06/2017 =C3=A0 16:43, Steve Kargl a =C3=A9crit : > > > On Fri, Jun 16, 2017 at 02:17:03PM +0200, Mathieu Arnold wrote: > > >> Le 16/06/2017 =C3=A0 01:45, Steve Kargl a =C3=A9crit : > > >>> Is this the expected behavior? > > >>> > > >>> % cd ports/math/octave > > >>> % make fetch > > >>> octave-4.0.3_5: Makefile error: "1.7+" is not a valid value for > JAVA_VERSION. It should be one or more of: 1.6 1.7 1.8 (with an optional > "+" suffix.) > > >>> % grep JAVA_VERSION Makefile > > >>> JAVA_VERSION=3D 1.7+ > > >> > > >> Not really. What OSVERSION are you running? > > >> > > >> What is the content of the _JAVA_VERSION_LIST and > > >> _JAVA_VERSION_LIST_REGEXP variables ? > > >> > > > I'm running a month old freebsd-current. > > > > > > % uname -a > > > FreeBSD troutmask.apl.washington.edu 12.0-CURRENT FreeBSD 12.0-CURREN= T > > > #0 r318546: Fri May 19 12:51:04 PDT 2017 > > > > > > % cd /usr/ports/math/octave > > > % make -d v fetch |& grep JAVA_VERSION > > > > $ make -V _JAVA_VERSION_LIST > > 1.6 1.7 1.8 1.6+ 1.7+ 1.8+ > > $ make -V _JAVA_VERSION_LIST_REGEXP > > 1.6\|1.7\|1.8\|1.6+\|1.7+\|1.8+ > > > > You should get the same as me. I wonder if it is not a difference in ho= w > > grep behaves on 12 where it does not match 1.7+ with the regexp in ther= e. > > > > Yes, it appears to be a grep issue. In Mk/bsd.java.mk, I removed > the @ in line 227 to see what the test did. I then try to reproduce > just the part with grep (without the -q option) at a Bourne shell > prompt. > > # echo "1.7+" > 1.7+ > # echo "1.7+" | /usr/bin/tr " " "\n" > 1.7+ > # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep > "1.6\|1.7\|1.8\|1.6+\|1.7+\|1.8+" > # > > So, nothing is returned. If I reduce the regex pattern, I see > > # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep "1.7+" > 1.7+ > > Hi, Please try this patch applied to your bsd.java.mk: https://files.kyle-evans.net/freebsd/java.diff Thanks, Kyle Evans From owner-freebsd-hackers@freebsd.org Fri Jun 16 17:50:10 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7D8EC796C8; Fri, 16 Jun 2017 17:50:10 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FFF464FF3; Fri, 16 Jun 2017 17:50:10 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v5GHo9G5039900 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 16 Jun 2017 10:50:09 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v5GHo9P0039899; Fri, 16 Jun 2017 10:50:09 -0700 (PDT) (envelope-from sgk) Date: Fri, 16 Jun 2017 10:50:09 -0700 From: Steve Kargl To: Kyle Evans Cc: Mathieu Arnold , freebsd-hackers@freebsd.org, ports-list freebsd Subject: Re: JAVA_VERSION??? Message-ID: <20170616175009.GA39872@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu References: <20170615234533.GA2871@troutmask.apl.washington.edu> <582b49d8-5910-35b5-40ed-307475830a64@FreeBSD.org> <20170616144318.GA11287@troutmask.apl.washington.edu> <20170616173422.GA38774@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 17:50:10 -0000 On Fri, Jun 16, 2017 at 12:40:53PM -0500, Kyle Evans wrote: > On Fri, Jun 16, 2017 at 12:34 PM, Steve Kargl < > sgk@troutmask.apl.washington.edu> wrote: > > > Yes, it appears to be a grep issue. In Mk/bsd.java.mk, I removed > > the @ in line 227 to see what the test did. I then try to reproduce > > just the part with grep (without the -q option) at a Bourne shell > > prompt. > > > > # echo "1.7+" > > 1.7+ > > # echo "1.7+" | /usr/bin/tr " " "\n" > > 1.7+ > > # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep > > "1.6\|1.7\|1.8\|1.6+\|1.7+\|1.8+" > > # > > > > So, nothing is returned. If I reduce the regex pattern, I see > > > > # echo "1.7+" | /usr/bin/tr " " "\n" | /usr/bin/grep "1.7+" > > 1.7+ > > > > Please try this patch applied to your bsd.java.mk: > https://files.kyle-evans.net/freebsd/java.diff > > Thanks, Kyle, Thanks. That patch fixes the problem with the spurious error message. -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-hackers@freebsd.org Fri Jun 16 17:53:21 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90E48C7993E; Fri, 16 Jun 2017 17:53:21 +0000 (UTC) (envelope-from kevans91@ksu.edu) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0071.outbound.protection.outlook.com [104.47.34.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2755865491; Fri, 16 Jun 2017 17:53:20 +0000 (UTC) (envelope-from kevans91@ksu.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ksu.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=KvZsQ6u+kHbypOPbAmFuUtDs5W8CXnOaEmPfOv/DYW8=; b=OSxQ3w7pdkyGIXnt7RS7i8jDe8jotwCWOfUeyy+y1BSWbKobcsB/dk/XfwuN3mB5QwCBhB5cem9ZrHpvg+T3bY2aN8yXerX/k+1ZrYmTcbveGAWqTE4BMo4tTEW1/9IZBuOstlN7K1Tu0CiU/9Fi/YLG8FI6umpqRhmjRpWiu30= Received: from BY1PR0501CA0012.namprd05.prod.outlook.com (10.162.139.22) by BY2PR0501MB1720.namprd05.prod.outlook.com (10.163.154.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.10; Fri, 16 Jun 2017 17:53:19 +0000 Received: from CY1NAM02FT058.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e45::206) by BY1PR0501CA0012.outlook.office365.com (2a01:111:e400:4821::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1199.6 via Frontend Transport; Fri, 16 Jun 2017 17:53:19 +0000 Authentication-Results: spf=pass (sender IP is 129.130.18.151) smtp.mailfrom=ksu.edu; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ksu.edu; Received-SPF: Pass (protection.outlook.com: domain of ksu.edu designates 129.130.18.151 as permitted sender) receiver=protection.outlook.com; client-ip=129.130.18.151; helo=ome-vm-smtp2.campus.ksu.edu; Received: from ome-vm-smtp2.campus.ksu.edu (129.130.18.151) by CY1NAM02FT058.mail.protection.outlook.com (10.152.74.149) with Microsoft SMTP Server id 15.1.1157.12 via Frontend Transport; Fri, 16 Jun 2017 17:53:18 +0000 Received: from calypso.engg.ksu.edu (calypso.engg.ksu.edu [129.130.43.181]) by ome-vm-smtp2.campus.ksu.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id v5GHrIBa029371; Fri, 16 Jun 2017 12:53:18 -0500 Received: by calypso.engg.ksu.edu (Postfix, from userid 110) id 6750F3DFA; Fri, 16 Jun 2017 12:53:18 -0500 (CDT) Received: from mail-vk0-f46.google.com (mail-vk0-f46.google.com [209.85.213.46]) by calypso.engg.ksu.edu (Postfix) with ESMTPA id 3F5C620B3; Fri, 16 Jun 2017 12:53:16 -0500 (CDT) Received: by mail-vk0-f46.google.com with SMTP id p62so26154816vkp.0; Fri, 16 Jun 2017 10:53:16 -0700 (PDT) X-Gm-Message-State: AKS2vOzvd6u5VXw38b0cpP2YPHNjONEggEdSexMB+rDkB7hOVP1TTKvU mc6Admsup/CM2DUG4Uabq1so9/VmPw== X-Received: by 10.31.11.143 with SMTP id 137mr5900561vkl.131.1497635595591; Fri, 16 Jun 2017 10:53:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.86.86 with HTTP; Fri, 16 Jun 2017 10:52:55 -0700 (PDT) In-Reply-To: <20170616175009.GA39872@troutmask.apl.washington.edu> References: <20170615234533.GA2871@troutmask.apl.washington.edu> <582b49d8-5910-35b5-40ed-307475830a64@FreeBSD.org> <20170616144318.GA11287@troutmask.apl.washington.edu> <20170616173422.GA38774@troutmask.apl.washington.edu> <20170616175009.GA39872@troutmask.apl.washington.edu> From: Kyle Evans Date: Fri, 16 Jun 2017 12:52:55 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: JAVA_VERSION??? To: CC: Mathieu Arnold , , ports-list freebsd X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:129.130.18.151; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39400400002)(39410400002)(39860400002)(39850400002)(39840400002)(39450400003)(2980300002)(438002)(199003)(189002)(24454002)(377454003)(512874002)(61266001)(5660300001)(7116003)(2171002)(9896002)(86362001)(59536001)(93886004)(236005)(54906002)(606005)(450100002)(189998001)(356003)(6862004)(8576002)(38730400002)(53386004)(110136004)(6246003)(8676002)(8936002)(9686003)(6306002)(4326008)(498394004)(229853002)(50986999)(53546009)(93516999)(63696999)(54356999)(76176999)(106466001)(2351001)(7906003)(305945005)(2950100002)(45336002)(5000100001)(46386002)(42186005)(55446002)(84326002)(478600001)(61726006)(1680700002)(75432002)(2906002)(88552002)(80390200001)(55456009); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR0501MB1720; H:ome-vm-smtp2.campus.ksu.edu; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; CY1NAM02FT058; 1:WeS72qYMxH45q6yAycHzs7M+K0lFjlLjgtZaC3GylnT4xuMc8nACXTFgKfORqzrgdnPCi1qb1vdmnrcc2fJAOVAn/qCVgwva5uw9xgcQ5miF4Jkazks7Vdyk5AsdB6ydAMr8FvqIg/rG0wTKJ9YAwlu+O7VcRpQPpxSIXtZPInvF9DSqfJZY3lK9C/Hm8OY48rea+qxqRw19EGQpKhyNEqP53RgaxWI2kTxXbe9jx1fCNlk96/1Shu2Y4fxpcN6i+cyAQPMUabgkW6vNavA2G7e7sINbkFRfOvWktwOdNfZPDMALmYg3zBnogyMjEKZoLnqXH0vwz1pSQqgv8x6HSVDsIiyDt759Ge5M+rKMr6Bx/QnAAU4ZsyOzAP8UcOCSYbQ9z1ECBggn4RlrtQmYPujOztXA4Wte2HH7grNi7NTsFiiIPE7joALTs1YMwWXLAJpxYKnARxsYzUuNCBbcp/Cird/1dLBEYX+IG6vEH2VfK0OYidqxQ/rfXzr9PoCVAibr1edpGhpt9z8mSWibv4vs/lizO0zWIiPi09KEXoAB+CENiGBIa0oKau3ADUa8 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5964f092-a9c4-4802-33b5-08d4b4e09294 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500041)(300135000095)(300000501041)(300135300095)(22001)(300000502041)(300135100095)(8251501002)(2017030254075)(300000503041)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504041)(300135200095)(300000505041)(300135600095)(300000506037)(300135500095); SRVR:BY2PR0501MB1720; X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 3:ZrMOopy8orUhgIP8qooo6qvwE630X1l2AbRfZ1P0JtDwoFkqkl25sNR3n2FibqSwg8q7bvjfooNbOIGNYxywJydgvv98E3KAFbMmZvhmI+gkjb1O86EJbOehE6YotrOeXbyBy270eOLqdl2XxY0Ys78uOVIqo1YrfNi6gsiC52MqYhsFUpPPKWgCRYX3de4ormKm4xaGBCdhhfYITfZdkM4YsIMsGwkqxlo9kXmhhRD70qfY7lOwQ14jMfYnfqrw74wFk1uqYlYqL+eqxS3uP1ngy4aGs17fqjl/6kbVix7V1g5Wuz1dXeNVCmKLjG9oElkZ/Pat+fh6vJBOAcewmxZhKR3nIxqAesq6rQaiFxFJ8hj/bbe46Vx/AdADhFG+PKPwlwNlO2RnUPP1JVIKTfcXvueDNC/1gmFhE+wOOGMk6IMx+/qJ1Vc4ow0cvdIjJc+/T6FGLxMjK1XIxQ6ttDYy4FlpQ7NQk2r2qExt0wEpNMwgnCu/tGWibyxFnjkVapm31UDW8ZafBXT3w309bS9KVm5zjdjZ4b3d4hp7AfWDGQPgT0iExw80trBpBSlLffWz6UXHKSu7rsqhB7bB/FqWbHVD/Me09gNnYJUaQHxzMUZbCbyWl0MeW8bk6sV9evIGwOUPCoNODLN095tdFfgN56x+aVPNjgtW47heJn/yhDeG1VUVk1pviihigXR/9uyp1OnHwrfxz/yC38AQNIudl4lgGuUGie5BK6eb1FyvoumF2mG2aXh7JSdemjyxHzvk3rXc9IEWxxPqStJ1UJpEpGk2y4Zv+Aisc9tbV6/wjMsr0GrbXAABGO8uzB83YYjO5P4B9t7qpa7iCZ+p8ZX19zu5YBD/f1th+0TP59rR6BmCeQkiAEjoZEMxvexXyXRrOW5z82l/IKbhQGWvleo1FSTCEt47Ag7s8qejuypkSYjMyHP5UagxThgOYYD2 X-MS-TrafficTypeDiagnostic: BY2PR0501MB1720: X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 25:BFl7dYtx6MMXfUr8XLsPdXWk+4xpiXpkHAw60a3va9/rb6w2Hpy4srX4dyHNau0r7URKzhX1EiFXLnivX+bsUksCECmUzOxLAtv+og9uPq3jkCvmPS1PEs+IzR124us0FQdzMk4JsXJDLd9ZEJAtC6k8dcUZdwwnVESrmkVId38mwkNlFUTCHvNRx8l7CiemoN7N96qZatfWoNIpVqIP29dS6HvspZOQ0f5W7zN0V2uiXm2j2vRtu6TaHx1BUF1SGSx/z/PjLIKGUK1WuCo5XJ51k5EiAuWVLRREVKM5JIqP1a+VInGJ0TK7jBfycNTt4qVtuK1X09YAr9plu3uYYnodUkzswM89zcXkVyJS3Kp7O2B1OTBfXQ01egkAWMJSZKUYEZLLthd2dk2dV/ogMvw+vQCie6ufSWa9C426hOnpJCqnfPDjvwYymdGocyOzdkKgQ9PdnDU0J9MsZlqZH60OYkl3Lb9k/hzWLYeNTcT9plk4PAVqEzFJqZZeVF6EXnEQgLQsAYgX1J5fPeZfwFewgWeCu88+wHq2f0CTJW3PnxqaLokk/EMZPvCeL9nOe1sccizhWsVLALZlTiwvBWQkLJz2a8y5fu2fZ3UDdrpSShtlJImqZl+yyMns+A9nF45Is6VVD5/LN0T1ObEAvGwu9OLTvi0qYIDpHs1jj1Q2RW1KP4PDB5GgSVwdTyGPzLkJMMAovBU1BMr9gPNYYSIChslDXe4GYn5r4+LiqPKSYexP2X56UggBafnMPHiX9Om2IOIqScnXX68j0oXerTNi0pIyMzOrRIAuNftDdmyvNcgUmzN4qESTQpEE72amnyhlDiz0vupaogwLfW4Igezb7u1nGu6kmZloaxw8tjIZn/PjeK4zrr3Lmqk5hWVH6qYAVcRFrjbAXDYwWPQ9WviT4/pkTysZD3/jhmAazLM= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 31:fmQH5GpjkXH3raQW1X8sSCk52Q4PNAuKXyk7DF6YFwLeUAxXMR2Q6vzzk6x4GfCbNpTcfSpdCo/eibBTtx0s5Ya9Ri8BT4pCkrIXCZmg91Tri2lQCSRVof9VQqLhDS9RdlB01PqNTJfR2EGnyq4Rjl3ccQ0MviSJrTIN65e9N5MHZBjupwbRqBT0PpwMGHWeET+JiQzXMXK2t0/zN160NlCa+JMZdkPJkvARCidLoVlhy8nN1SpdW8EBA1D66nFLUnT2GyxaiiG1gYU+IrlRPNT9zRfd8luZJdLmN9t/OjbfK/SRcE24OA8ZepRnFiq4CFFSrrLD8GKblemimEjPv28mEa2QKOwGKhgaarzmMwvxkxOhc1lgkXwupbin6iUZpNcBCQPeUHUeUHlZgvshLrNAEXETTZBrzmmGWSE8NWFiktntUryzfuf/C1I2jmiyEX2+Ae1lk68eQnbWxc1JdkV0h5K6ollXm2ltd3PXmeD1K9N2YASfiXr79SLasEfbkh1u89AcFRCHOeCt9qVXakk8/zTbW8QroQMNoYlZjYsIeG8zsHCIPmfsxlCR1LYNxgtL1TQ6O+uIYJcGotDH46JjlaVGeC1r8oSUabRgMiZF1akYRT/k+MwOc+GnWYWVhZMzJbh2nm815bz96FOitf5iKIv1lDqkQnoMvh8BJxOuQ7YOrqSqhkmF/9PDvwV9vT2EZUv3d/xpyIkfAsEpQw== X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 20:UZUnoGwHTFwI8fWkFzW3uRip88YIi129x9xMBq7wgorDTwzakyRmt2ck4ACs2w29NK1/yj997IifZ1YX7DWKq7mkuJkbw2jdRQIiAghNIVmwHppamvOxR1d8w3LBQ6GpJJTrJstokGr4g9X5RpTOK9bEGG5GXachOR13jNgveXiT3LVeh7hW2QqtqLuD360VGTaNspjLEJ33vmPLVBcDQ1opBeUMPHHBkO5pNPloKpDq2c7e4JFboNs2fHDsC87hloMjUTMqIFKcbcIVPPSC4mqVfTgs1B11FGKwKBeKp7JEHylsK6sVfhNQ0VYiuyyknMbFlEfOIOFlXrdCcwftOnugv590ejdy4h5skkp4G2fpvbz058jiJImr2BlMEqGkpo8tPga4SCgkLCgWbaoARe1CR3wwXKbVAKUcF6OcStVfa6x8hlYxhF+qpbV8wPCu5M02YPNhqy1SaiCIfmfXXTPzyr7CSHv9PpTiuE+iLGWtgUbWrBhnRYuk1Xjj6Cyq X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(47954115253988)(254730959083279)(91638250987450); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(3002001)(100000703101)(100105400095)(10201501046)(93006095)(93004095)(6041248)(20161123560025)(20161123558100)(20161123564025)(20161123562025)(20161123555025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY2PR0501MB1720; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2PR0501MB1720; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0501MB1720; 4:3X6OxuQxevzt7e05TGrTZ+HN2gEHdmsM5iM5A9pS?= =?us-ascii?Q?zmMrWYqn87RTnqD6BQPbkxzUEq8rZESuvcfRSrRDJgQZOt5zYAks3FZ/gWjN?= =?us-ascii?Q?SCuoQcopHeqZngLgGV3oj7GAhTY4Yv7i4Rm7FlY10qO1BSBRa1nU55pLg7qG?= =?us-ascii?Q?vCVBfm9SjshPgyuP1cf2UsmOdnL4kAk5Tv+2ZenHgwmhCKh/E/n+om0J5eys?= =?us-ascii?Q?8BYpx6ZYu0Z05JgmDCqbi9pxCAYjNirPelb2Xt2ztUdjv4Lo31jfF+i544ZJ?= =?us-ascii?Q?e7a6BJ+NjZy96WzFA5GX7PMFStDpqnAHHjqn5ZWRdhxMZf/Q8tdiBvA8U4fh?= =?us-ascii?Q?K0zV4y+GL3v+wkb7h9orbhDMb8AE9jXNgY2iXTqK+lKDA2cBtym3R3qRZzFn?= =?us-ascii?Q?iBnPwk1omujssfttArsNSAqmmUoJHp8J0rJmYgQbdz5oe6MxLkHWH44JlmIf?= =?us-ascii?Q?jXfhAv3DSYAhFBNfTWJXu9yTu0SwNB3eOMczJn02LczER3zN6WbQ9BHNZOF1?= =?us-ascii?Q?tSiFLB4sU2spkUy2GftcSeQSX2ndk0ewSBXp5zqMqRCPcA5WqQNm3AYlxPOQ?= =?us-ascii?Q?e2QCca634ftQYtFcd7UqiKq+gLNLLlYEjITSQhSRULCdsFt0CBelOk5Dhfd4?= =?us-ascii?Q?9Zj5FlZOOL9u+2Ny8ok+vXGAPevXuW4c5HtaqkF/31NZyCwQIN4dbsOE79mz?= =?us-ascii?Q?xlZB8KJ5MMUTnk2digC7lpmY30P0GwrsIIlxm0N/JUayqzWwemM5QZvfpCox?= =?us-ascii?Q?S/d5CgIGjOu1bFW8NIiX+HID1ybaB4Sqcifw6ydoxN4PA0AgpYOEXP9HKDd6?= =?us-ascii?Q?7wu8HIwxQ/uc1BYIZNpcQr3DAziHr8gm1IhzDRKR2Bj+o+KM4W769mAjmnSi?= =?us-ascii?Q?HWsyjIxdrqJznPzm3nWuqU8gj1sLXXPiCIFSU3MKRXeGpvHWS3yvnid2n4/q?= =?us-ascii?Q?0T7Vv4nN1IJvEE91r5R/zbc1fi/iLwZTVJX/B4p3teIaV6wEodDFmvkUWDMy?= =?us-ascii?Q?fVilplw40WnC4+YhOIhB0kdScDMxwHdIOF5qmXuePza0TC0Y1EiNCsjtr+kT?= =?us-ascii?Q?NdIun0VtlFJDiE+9vLbjCUmsTFqrlY1DtWt7o9raBbfYk/4/XSUW0hWx5Vd9?= =?us-ascii?Q?Mzn04mNNnIoDEMQXxTezKeexEbqCHnXa2bX9oEhYkkOFJ4C7YcYvNZ0FXvEM?= =?us-ascii?Q?0FsVZySK3EpNQyimgDInuC7ktTCf7EIpPa0C0Z7w+G7oq9plGSLR5op2VJa6?= =?us-ascii?Q?R8anfOPgphd1B7NCLhtZ8LMJW3fYQy2giYHn2MY7L3Ix8UrUo6OrcbvTrET8?= =?us-ascii?Q?mKLhsjSQwxy+BI4CZAVnPpMKw/SzaPXouQAupserbagh?= X-Forefront-PRVS: 0340850FCD X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0501MB1720; 23:KQHBkQJsPJr6OHg9OrcFm8mfk4fEQ7ix3grVofa?= =?us-ascii?Q?EyU2A76ZFb3Lw2PhMS+ZwPd6MfMjqauIt25M8o2AoZmM8013hLaB3P3nozXl?= =?us-ascii?Q?lJUH9nilVrjmHq6QGzasy3vepOEI4UB5nLuQBH2A1pcNOH0/bA1aaSIawrWm?= =?us-ascii?Q?HKiyjO0lItAcwlsmywvTdEASVNj6izSKN4tSkll26IYbixGIVWOSwj3Su4IJ?= =?us-ascii?Q?+5WV+crLJvKib/HNY8dKYy8gSEEY+zTd5cQOuvzPy/+6y+1pPx6X/eas1zBE?= =?us-ascii?Q?qarpcb+6GlxkW+blb3IvgugvD+3Cc5jL8NCfa4t24OreJHnKFiiKB8W0cD4y?= =?us-ascii?Q?Z+iGoPWDjMMkhuUerAZpQejXUGL8c2rkGy1FrovWWSu9knUEzDbJlQGy58Cc?= =?us-ascii?Q?D+pxHsFc9YxZM/BLBP+fTbiNaDvNLyAhuDBOSfQAKLTlimZ1DNlhechKPiUE?= =?us-ascii?Q?tCEJ3MvsH6QdRiXbY0EZo4oxEEJoUtEOq5PoSKr+AtkEGWlDyQn8Uo1nDd7G?= =?us-ascii?Q?EmlwIdo3/XFcUSeqbUIMlB3MEl3S1TO0k+FDWD8xQcBP3ab4gzxnVWPc8fF7?= =?us-ascii?Q?E+EqAP5ftB0AQukP+DIP/8qhOcQt8jaQHiqLHBDNX0VsgaJM/l7InLZ19+iu?= =?us-ascii?Q?01Oz+XxLBKhSI3Z/3ZP19aWzh8pBZEMIu8nZWWRC3j8Dt8892uajEnZzndrN?= =?us-ascii?Q?qNACosTYeFuxFuvnS+mKHvU9XJeWeZow/DL25HnA7K1HG1QISd342w9t5F6V?= =?us-ascii?Q?J+Xr0Yh/cWOz7nDPpY/5mVTzeeXevydlpS0iNsWuU6hovqt4n2TDbVZ88spv?= =?us-ascii?Q?0MP6Nhaj9MaUwbRcfOCyK3uUWBaMguwakPCiPQj8j2bEhZ/KMJTUj7MTtT+3?= =?us-ascii?Q?ldFCpSP0tGPZu85VAv79JfOzsLJkMDlFz7fLkuCHO4A9f+9aujODYegBzpc9?= =?us-ascii?Q?yBg9rYihyFD5aMLd0+cNjJbXXxrPnPCOoblCrwK12mfNIPaprAsRKg6ApvHZ?= =?us-ascii?Q?Oi9/mhBTsj4vPkvNZkWFMJtBUXcXkND5HeWNYCIZuath9ISmY+G1amFJgp/v?= =?us-ascii?Q?wX5waLp8d15/lKANn8qx7JoAaMi3q0166lKOImMIAd9OSlKfwczCavPugSkT?= =?us-ascii?Q?tT2+YSwaxzovTeqp0iCQvVJBy6/z6sar0Ao+GsWQDfZwB07VmbY6V2Sw0M/X?= =?us-ascii?Q?dzzvxk/7iXztgX80Zy63mDTwX5Jv8BivBFBhFUF/wtbUogB+Cc7UhrrB5y1w?= =?us-ascii?Q?VTE2CZ4Bx5wAIiDn2adW49fuTG0RzB2n+TYijllbcTLw7DnkQK36QTqaQEzw?= =?us-ascii?Q?ONmT+ao4+kRk6xzJS9HobkHFz3DqICIeMpVmpDPXZ3vtdRe8joSap9xlJKM8?= =?us-ascii?Q?Txn0dkBKwU/vcRBzo0dTw+Vie7adMdWYsZEb7IJba117kRhlmkqgiMfuIkMk?= =?us-ascii?Q?UFyb6j3mwI0Kf75yXjXz+glmIPo5JUZQddW9nZlXCCyzicD8Y7ly7Dr3O6IW?= =?us-ascii?Q?gP0sWKemXThtIIgp2k4TWWcH7OLzqUg+xi9IsK9i0+IPzQbux6SGYQk84up9?= =?us-ascii?Q?INn88wbG4rrSb5d4KpQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR0501MB1720; 6:XDxtcSDAhFvNvCNpf96LtkKXSH+0H4Z+IlVcZ4oq?= =?us-ascii?Q?z1esXIY14QI8KjX7rcQHuCqtH/gndMBJ2aYV5YMbC42izZljNEns81Gb2lP0?= =?us-ascii?Q?l9PRWHXfLG9VENjxcb98qv4l+k90SQApr8MAoI0g8b40v+Z6w5h60N3XxjlF?= =?us-ascii?Q?yNXzbR241Rcy5mZBqp01pBSPUhlkpG4+Jg17oCDN0um/hsU7kAejhqDToS65?= =?us-ascii?Q?xdS0M8znpKsZhpUrYfAGtt/umBFOFCOqEPwX93dqwz7oPS0GFqYM0gn1tnWS?= =?us-ascii?Q?DOrY20AJ/3ACJFlSOZtRjiDpolLwbGrNChAij3NvAopQ2SMIIT6wwqz7U8hP?= =?us-ascii?Q?dEOu2ksfhHYuUdon4u23DVnxvwlyGHsq/u9KEK75Q0Te0hn/QSwP2a/kDsku?= =?us-ascii?Q?ILQe2sx00ce6B9QcwqawBqyqVxp6/9MA+FWYbtqxdPMmppd0G0kI//2iasXy?= =?us-ascii?Q?QjwpeFppjh1LwVO9y/lAQxaajNkGroxJnE4MhsfRopBfaDQTsYwZQrAu6Ou1?= =?us-ascii?Q?aFQM4yvjHyEokIZUF53ceUzQF/liIrUdvdncRfy4tVVuzlpKxGBzg2LK+Bxh?= =?us-ascii?Q?4TbmtitEiRGGEs2DKAoEXeCSTvuumf2o8Rvgm9rRp/ioKmc/E/88cXGrS0a0?= =?us-ascii?Q?eawbE62owueVPVLblwp9r0S3XQ2txWM42Gzo4t8iGm1HcBfSpczpm53UbUMI?= =?us-ascii?Q?ZkanOaw+bJLLn8tpgESrX0FnTmWiVINOrTiCKDyjfjmpWawWdXF+1O2FAU1k?= =?us-ascii?Q?CuiQr98GJIEJkhG5skIuLMsg+YKZA43Qxb7rCr8RN0sq/cF0jjvC4IACEcgs?= =?us-ascii?Q?FN0vVPsOaKPFpJI/aSQsMRgCs50KDm/QZm+ZGjAYwkQFIWgLQViybOCEbM5n?= =?us-ascii?Q?+DJA3soPsGYd0llwfKdoz3xEdnGtH+nZLxokpz3Wtf+wXad6/zGk8yXRjgVj?= =?us-ascii?Q?yUBXcNDAEQYkB8VR8Tn7A5vsqYIRPYU0TZ4U94WOXbTm6FzM9F2VvwoBMSDE?= =?us-ascii?Q?z2g=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 5:QkAPUHdh/PxLlz/TPzpo4k3ziWHIgBscpa4CPSyFyN2unAd1R6WKH0zecKENSESqnIrXywjBhNo9t4w5eIZM6qJ+zP1+dOFf3RLxw05kiFMGnkMFXJbNK40i+rvUpYyliieS6CFTeAyUzpe7m+vJhTHooE8M0iz7lAFjbFNr5LZ/iBJSbkT9Pmg5XL+FEOk83uKLJ3E3tKQb46IU3Sq2KJ3kGbg21Zs2MHF5p+qnKBUA7wMQ1qRkYZFgqJ6HvtAnlp9R1PX7LZFRyyHSvfISlHiqzW/pS1pYPNzIsTMohyNjaJn49bkvhn2lQsaVZ3PVaaX5ezf/cRWR0Y6OCgPtluxJM21sxYY9pZu07q0zK8fWAAn0b6ofzYGAojLBUP52ikme/Dp4k1g22IdKYm9HctSogRMaYz4BN9kVAmqfssvkoSZKDw06iDsJZfuSt4yyaBBScbVSdbOb8TEOU4uh3tINJoZSUMbwmxGUp77wtl+f9H+m9wTGOEd1IQxf56UQ; 24:L0K73VOqlbWtAdc8UxBL1Ix8ySRjMQFvd9ibvY20J16YbJeP70pM0dS23Q1qFDtLt4h9QBxdjLzCPoXBhaMf5BgujnPrs/U9Z/OCct7ciNk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 7:3h3+lBGZaoVqhVutNHzSfrgbPs8qabsNk46NM8S3pmgShxICb+Jjt2MEu9eQsoIXSVzNKuu70BgSIcL0g5x/JjHhPRHaqZyNGVPu6zto7H/+E1me8/7fObO46npHqH6wjSDY7dvd73tfc6vQ7QPSLHF1mcICipyRDYfCd1LRi+WoBkwd8D/d7as4pFOf/A1s3XmDYNzr4yIKfzGxmsZZFfr/+FOIGWMlUuljCGnzWJ2GYMHiyuOYac9/jLClkk9aHFporR1/pIZpkGQM6YOJyvWat6BaStIxVpcAXz+oFE5R/yAJQmdiVKKJVrpvic2BBWOIFIzHCAINlDZW1QzVnqrMze8OLpriSnuzMsSX3Wx6LkNkQvftCdVFVi1oONvB7Bwo7bWZ73TH5q2fZROZQnC6zqS/GFMj44sehlkEX+TsZLnx3gv6wGprK879RpZr6zyJJusTGAKMIAa9HZ7JUXdeuFfX4EI3mMrex2j+95VBGlZ3cDQ/caF10BaRV4zDUzQAnlDr5GGi+KYxrF9RqQpagMU2OhrN/MLb9o6l5Ht5bu7KLrJifGbvEiOxKGUMlE5YuUgVFWc98g20EhFotNMUoLpcs5V3NIeqmmyYztLK13XNMLxAFMYi7vD2FMe9KYz2daI+xZO4zy+d87bvOGGmWlus6MQnS2iiJoksL6h7sU0HwfgQj7hxQ3mf82uSHmgqNlqw1BNItOD/wRejbeeqB1D5TtfYaSRpiqnPJ0lPgiZOkNlaeyyasDp0iQD0/hoC0v0H63eqUIHGXF2fyD+S8CDLKcgnAQ150+/h5UE= X-Microsoft-Exchange-Diagnostics: 1; BY2PR0501MB1720; 20:ix/9hisD9eJCAbS2Yt19wz+eS9ELWdGLI05nzcjC5BwiSb4j4bcyMfVTkmbUYf/bmawPT7PeLe3i+Q+57pQJz7hIYzZJ/iuHOh1e5nNNmenNz6UUfrdDdB3cIWyjslBsa3tDXtFHadfN2EZFoKT6H00As0s2lhMChOr5MFuaC9U= X-OriginatorOrg: ksu.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2017 17:53:18.8286 (UTC) X-MS-Exchange-CrossTenant-Id: d9a2fa71-d67d-4cb6-b541-06ccaa8013fb X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=d9a2fa71-d67d-4cb6-b541-06ccaa8013fb; Ip=[129.130.18.151]; Helo=[ome-vm-smtp2.campus.ksu.edu] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0501MB1720 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 17:53:21 -0000 On Fri, Jun 16, 2017 at 12:50 PM, Steve Kargl < sgk@troutmask.apl.washington.edu> wrote: > > Kyle, > > Thanks. That patch fixes the problem with the spurious error > message. > > Excellent, I'll file a PR with this patch. You're using bsdgrep and it hasn't regrown GNU extensions yet, which bsd.java.mk is dependent upon. We've changed a lot of things to use POSIX compliant expressions, but it seems that we've not yet caught them all. Sorry for the trouble! From owner-freebsd-hackers@freebsd.org Fri Jun 16 18:02:31 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 617F5C79E73; Fri, 16 Jun 2017 18:02:31 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 433BE65C8D; Fri, 16 Jun 2017 18:02:31 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v5GI2Una040057 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 16 Jun 2017 11:02:30 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v5GI2UGA040056; Fri, 16 Jun 2017 11:02:30 -0700 (PDT) (envelope-from sgk) Date: Fri, 16 Jun 2017 11:02:30 -0700 From: Steve Kargl To: Kyle Evans Cc: Mathieu Arnold , freebsd-hackers@freebsd.org, ports-list freebsd Subject: Re: JAVA_VERSION??? Message-ID: <20170616180230.GA39983@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu References: <20170615234533.GA2871@troutmask.apl.washington.edu> <582b49d8-5910-35b5-40ed-307475830a64@FreeBSD.org> <20170616144318.GA11287@troutmask.apl.washington.edu> <20170616173422.GA38774@troutmask.apl.washington.edu> <20170616175009.GA39872@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 18:02:31 -0000 On Fri, Jun 16, 2017 at 12:52:55PM -0500, Kyle Evans wrote: > On Fri, Jun 16, 2017 at 12:50 PM, Steve Kargl < > sgk@troutmask.apl.washington.edu> wrote: > > > > > Thanks. That patch fixes the problem with the spurious error > > message. > > Excellent, I'll file a PR with this patch. You're using bsdgrep and it > hasn't regrown GNU extensions yet, which bsd.java.mk is dependent upon. > We've changed a lot of things to use POSIX compliant expressions, but it > seems that we've not yet caught them all. Sorry for the trouble! Again, thanks. It is somewhat odd that bsd.java.mk check variables for appropriate values, issues an error message, and then proceeds to build the port. If the make process aborted after the error message, someone would have noticed the issue well before me. -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-hackers@freebsd.org Fri Jun 16 19:14:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 341B2D87469; Fri, 16 Jun 2017 19:14:38 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C923682FA; Fri, 16 Jun 2017 19:14:38 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id v5GJEa9Q001246 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 16 Jun 2017 12:14:36 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id v5GJEade001245; Fri, 16 Jun 2017 12:14:36 -0700 (PDT) (envelope-from sgk) Date: Fri, 16 Jun 2017 12:14:36 -0700 From: Steve Kargl To: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: panic: handle_written_inodeblock: Invalid link count 57368 Message-ID: <20170616191436.GA1189@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 19:14:38 -0000 I grabbed a spare USB 2 TB hard drive yesterday. Put a GPT scheme on the drive and then used newfs to create 1 large UFS2 partition with softupdates and with journalling enabled. I then some 150 GB of data to drive. The drive in question is ugen0.3: at usbus0 umass0 on uhub8 umass0: on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x4000 umass0:9:0: Attached to scbus9 da0 at umass-sim0 bus 0 scbus9 target 0 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number 575835314134334136333937 da0: 40.000MB/s transfers da0: 1907697MB (3906963456 512 byte sectors) da0: quirks=0x2 I just got the indicated panic while trying to do ls in a directory with 100+ files. From the panic, I have core.txt.7, info.7, and vmcore.7. info.7 contains Dump header from device: /dev/ada0p3 Architecture: amd64 Architecture Version: 2 Dump Length: 1150738432 Blocksize: 512 Dumptime: Fri Jun 16 11:47:07 2017 Hostname: troutmask.apl.washington.edu Magic: FreeBSD Kernel Dump Version String: FreeBSD 12.0-CURRENT #0 r318546: Fri May 19 12:51:04 PDT 2017 kargl@troutmask.apl.washington.edu:/data/obj/usr/src/sys/SPEW Panic String: handle_written_inodeblock: Invalid link count 57368 for inodedep 0xfffff800a56c4000 Dump Parity: 2911432532 Bounds: 7 Dump Status: good Leading portion of core.txt.7 is Unread portion of the kernel message buffer: panic: handle_written_inodeblock: Invalid link count 57368 for inodedep 0xfffff800a56c4000 cpuid = 3 time = 1497638827 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe022e2c40b0 vpanic() at vpanic+0x19c/frame 0xfffffe022e2c4130 panic() at panic+0x43/frame 0xfffffe022e2c4190 handle_written_inodeblock() at handle_written_inodeblock+0x7c9/frame 0xfffffe022e2c41e0 softdep_disk_write_complete() at softdep_disk_write_complete+0x1b4/frame 0xfffffe022e2c4240 bufdone_finish() at bufdone_finish+0x34/frame 0xfffffe022e2c42b0 bufdone() at bufdone+0x45/frame 0xfffffe022e2c42d0 g_io_deliver() at g_io_deliver+0x234/frame 0xfffffe022e2c4330 g_io_deliver() at g_io_deliver+0x234/frame 0xfffffe022e2c4390 g_disk_done() at g_disk_done+0x10d/frame 0xfffffe022e2c43e0 dadone() at dadone+0x1e21/frame 0xfffffe022e2c4960 xpt_done_process() at xpt_done_process+0x5d6/frame 0xfffffe022e2c49a0 xpt_done_td() at xpt_done_td+0x166/frame 0xfffffe022e2c49f0 fork_exit() at fork_exit+0x75/frame 0xfffffe022e2c4a30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe022e2c4a30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 27d22h21m17s (da0:umass-sim0:0:0:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da0:umass-sim0:0:0:0): CAM status: Resource Unavailable (da0:umass-sim0:0:0:0): Error 5, Retries exhausted (da0:umass-sim0:0:0:0): Synchronize cache failed Dumping 1097 out of 8142 MB:..2%..11%..21%..31%..41%..52%..62%..72%..81%..91% Reading symbols from /boot/kernel/radeonkms.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkms.ko.debug...done. done. Reading symbols from /boot/kernel/drm2.ko...Reading symbols from /usr/lib/debug//boot/kernel/drm2.ko.debug...done. done. Reading symbols from /boot/kernel/agp.ko...Reading symbols from /usr/lib/debug//boot/kernel/agp.ko.debug...done. done. Reading symbols from /boot/kernel/radeonkmsfw_CAICOS_pfp.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_CAICOS_pfp.ko.debug...done. done. Reading symbols from /boot/kernel/radeonkmsfw_CAICOS_me.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_CAICOS_me.ko.debug...done. done. Reading symbols from /boot/kernel/radeonkmsfw_BTC_rlc.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_BTC_rlc.ko.debug...done. done. Reading symbols from /boot/kernel/radeonkmsfw_CAICOS_mc.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_CAICOS_mc.ko.debug...done. done. __curthread () at ./machine/pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:232 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xffffffff8058649b in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xffffffff80586916 in vpanic (fmt=, ap=0xfffffe022e2c4170) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xffffffff80586733 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xffffffff8079c1b9 in handle_written_inodeblock ( inodedep=0xfffff800a56c4000, bp=0xfffffe01f01b7a58, flags=) at /usr/src/sys/ufs/ffs/ffs_softdep.c:11535 #6 0xffffffff80782514 in softdep_disk_write_complete (bp=0xfffffe01f01b7a58) at /usr/src/sys/ufs/ffs/ffs_softdep.c:11043 #7 0xffffffff8061d2b4 in buf_complete (bp=0xfffffe01f01b7a58) at /usr/src/sys/sys/buf.h:422 #8 bufdone_finish (bp=0xfffffe01f01b7a58) at /usr/src/sys/kern/vfs_bio.c:4045 #9 0xffffffff8061d1b5 in bufdone (bp=0xfffffe01f01b7a58) at /usr/src/sys/kern/vfs_bio.c:4033 #10 0xffffffff8050df14 in g_io_deliver (bp=0xfffff80115d71bc0, error=) at /usr/src/sys/geom/geom_io.c:738 #11 0xffffffff8050df14 in g_io_deliver (bp=0xfffff801fd3b32f0, error=) at /usr/src/sys/geom/geom_io.c:738 #12 0xffffffff8050b77d in g_disk_done (bp=0xfffff8007613c2f0) at /usr/src/sys/geom/geom_disk.c:256 #13 0xffffffff802ef1b1 in dadone (periph=, done_ccb=0xfffff80068599800) at /usr/src/sys/cam/scsi/scsi_da.c:4216 #14 0xffffffff80292936 in xpt_done_process (ccb_h=0xfffff80068599800) at /usr/src/sys/cam/cam_xpt.c:5453 #15 0xffffffff80294ab6 in xpt_done_td ( arg=0xffffffff80bf9300 ) at /usr/src/sys/cam/cam_xpt.c:5480 #16 0xffffffff80553ff5 in fork_exit ( callout=0xffffffff80294950 , arg=0xffffffff80bf9300 , frame=0xfffffe022e2c4a40) at /usr/src/sys/kern/kern_fork.c:1038 #17 core.txt.7 and vmcore.7 can be made available for the asking. -- Steve 20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4 20161221 https://www.youtube.com/watch?v=IbCHE-hONow From owner-freebsd-hackers@freebsd.org Fri Jun 16 21:28:54 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CD02D898FB for ; Fri, 16 Jun 2017 21:28:54 +0000 (UTC) (envelope-from aplattner@nvidia.com) Received: from hqemgate14.nvidia.com (hqemgate14.nvidia.com [216.228.121.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hqemgate14.nvidia.com", Issuer "RapidSSL SHA256 CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 517BB70A68; Fri, 16 Jun 2017 21:28:53 +0000 (UTC) (envelope-from aplattner@nvidia.com) Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate14.nvidia.com id ; Fri, 16 Jun 2017 14:28:43 -0700 Received: from HQMAIL105.nvidia.com ([172.20.13.39]) by hqpgpgate101.nvidia.com (PGP Universal service); Fri, 16 Jun 2017 14:28:53 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Fri, 16 Jun 2017 14:28:53 -0700 Received: from HQMAIL102.nvidia.com (172.18.146.10) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Fri, 16 Jun 2017 21:28:52 +0000 Received: from tenor.nvidia.com (172.20.13.39) by HQMAIL102.nvidia.com (172.18.146.10) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Fri, 16 Jun 2017 21:28:52 +0000 Subject: Re: efifb framebuffer info for NVIDIA driver console restore To: CC: "freebsd-hackers@freebsd.org" References: X-Nvconfidentiality: public From: Aaron Plattner Message-ID: <533f7954-61af-4121-3801-d9e5f02e4d35@nvidia.com> Date: Fri, 16 Jun 2017 14:28:52 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [172.20.13.39] X-ClientProxiedBy: HQMAIL103.nvidia.com (172.20.187.11) To HQMAIL102.nvidia.com (172.18.146.10) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 21:28:54 -0000 Thanks, that seems to work great! Unless someone has a better idea, I'll ship something based on this with a future driver release. On 06/16/2017 11:08 AM, Conrad Meyer wrote: > Hi Aaron, > > I'm not sure it's the best way, but you could find EFI framebuffer > information in the same way the efifb vt(4) driver > (sys/dev/vt/hw/efifb/efifb.c) does, by inspecting the struct efi_fb > preload data (which is passed from the loader to the kernel). > Something like: > > caddr_t kmdp; > struct efi_fb *efifb; > > kmdp = preload_search_by_type("elf kernel"); > if (kmdp == NULL) > kmdp = preload_search_by_type("elf64 kernel"); > efifb = (struct efi_fb *)preload_search_info(kmdp, > MODINFO_METADATA | MODINFOMD_EFI_FB); > if (efifb != NULL) > /* Found framebuffer information */; > > Hope that helps, > Conrad > > On Fri, Jun 16, 2017 at 9:51 AM, Aaron Plattner wrote: >> Hi FreeBSD hackers, >> >> My name is Aaron Plattner. I'm a driver developer at NVIDIA working on >> the FreeBSD driver, among other things. I was hoping you could help me >> out regarding some changes I've been making to the driver. >> >> Recently, we've revamped how our driver handles switching from >> graphical modes to the console mode. For framebuffer consoles, the new >> nvidia_modeset module tries to take care of it without having to fall >> back to old-school VESA VBE modesets. However, in order for this to >> work, the driver needs to know where the framebuffer console is in >> physical memory, and its layout. >> >> On Linux, we get this information from the global 'screen_info' >> structure: >> >> void NV_API_CALL os_get_screen_info( >> NvU64 *pPhysicalAddress, >> NvU16 *pFbWidth, >> NvU16 *pFbHeight, >> NvU16 *pFbDepth, >> NvU16 *pFbPitch >> ) >> { >> // >> // If there is not a framebuffer console, return 0 size. >> // >> // orig_video_isVGA is set to 1 during early Linux kernel >> // initialization, and then will be set to a value, such as >> // VIDEO_TYPE_VLFB or VIDEO_TYPE_EFI if an fbdev console is used. >> // >> if (screen_info.orig_video_isVGA <= 1) >> { >> *pPhysicalAddress = 0; >> *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; >> return; >> } >> >> *pPhysicalAddress = screen_info.lfb_base; >> #if defined(VIDEO_CAPABILITY_64BIT_BASE) >> *pPhysicalAddress |= (NvU64)screen_info.ext_lfb_base << 32; >> #endif >> *pFbWidth = screen_info.lfb_width; >> *pFbHeight = screen_info.lfb_height; >> *pFbDepth = screen_info.lfb_depth; >> *pFbPitch = screen_info.lfb_linelength; >> } >> >> This works for both legacy boot systems with vesafb as well as UEFI >> systems that use Linux's efifb. >> >> On FreeBSD, I was able to find this information on my legacy system: >> >> void NV_API_CALL os_get_screen_info( >> NvU64 *pPhysicalAddress, >> NvU16 *pFbWidth, >> NvU16 *pFbHeight, >> NvU16 *pFbDepth, >> NvU16 *pFbPitch >> ) >> { >> const sc_softc_t *sc = sc_get_softc(0, SC_KERNEL_CONSOLE); >> >> if (sc) >> { >> const video_adapter_t *adp = sc->adp; >> >> if (adp) >> { >> const struct video_info *vi = &adp->va_info; >> >> if (vi && (vi->vi_flags & V_INFO_LINEAR)) >> { >> *pPhysicalAddress = vi->vi_buffer; >> *pFbWidth = vi->vi_width; >> *pFbHeight = vi->vi_height; >> *pFbDepth = vi->vi_depth; >> *pFbPitch = adp->va_line_width; >> return; >> } >> } >> } >> >> *pPhysicalAddress = 0; >> *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; >> } >> >> However, this doesn't work on UEFI systems because efifb doesn't go >> through the video_adapter_t stuff. Does anyone know how I can get that >> information from efifb, or who to talk to about adding an interface >> the driver can use to find it? >> >> Sincerely, >> Aaron >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sat Jun 17 00:01:47 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 355ECD8BE42 for ; Sat, 17 Jun 2017 00:01:47 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-16.reflexion.net [208.70.210.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D811974D8E for ; Sat, 17 Jun 2017 00:01:46 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 28712 invoked from network); 17 Jun 2017 00:05:46 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 17 Jun 2017 00:05:46 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Fri, 16 Jun 2017 20:01:45 -0400 (EDT) Received: (qmail 28514 invoked from network); 17 Jun 2017 00:01:44 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Jun 2017 00:01:44 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 258A2EC938A; Fri, 16 Jun 2017 17:01:44 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? Message-Id: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> Date: Fri, 16 Jun 2017 17:01:43 -0700 To: kib@FreeBSD.org, FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 00:01:47 -0000 buildworld via clang for powerpc64 and powerpc fails for lack of `__udivdi3' referenced in sys/boot/common/ufsread.c fsread_size code. But this lead to me looking around and I found a conceptually separate possible issue. . . sys/sys/_types.h : typedef __uint64_t __ino_t; /* inode number */ # find /usr/src/sys/ -exec grep __ino_t {} \; -print | more typedef __ino_t ino_t; /usr/src/sys/sys/stat.h typedef __ino_t ino_t; /* inode number */ /usr/src/sys/sys/types.h typedef __uint64_t __ino_t; /* inode number */ /usr/src/sys/sys/_types.h typedef __ino_t ino_t; /usr/src/sys/sys/dirent.h sys/boot/common/ufsread.c : . . . #include #include #include . . . typedef uint32_t ufs_ino_t; . . . Note the 32-bit type above. The headers included have use of the 64-bit ino_t type as well, for example: sys/ufs/ufs/diniode.h : . . . #define UFS_ROOTINO ((ino_t)2) . . . #define UFS_WINO ((ino_t)1) . . . sys/ufs/ffs/fs.h : . . . #define ino_to_cg(fs, x) (((ino_t)(x)) / (fs)->fs_ipg) #define ino_to_fsba(fs, x) = \ ((ufs2_daddr_t)(cgimin(fs, ino_to_cg(fs, (ino_t)(x))) + = \ (blkstofrags((fs), ((((ino_t)(x)) % (fs)->fs_ipg) / = INOPB(fs)))))) #define ino_to_fsbo(fs, x) (((ino_t)(x)) % INOPB(fs)) . . . I believe the powerpc64/powerpc issue gives evidence of ino_t being used in addition ot ufs_ino_t in sys/boot/common/ufsread.c 's fsread_size . Other things that look 32-bit inode-ish: (I do not claim to know that any of this matters.) sys/ufs/ufs/dir.h has: struct direct { u_int32_t d_ino; /* inode number of entry */ . . . struct dirtemplate { u_int32_t dot_ino; . . . u_int32_t dotdot_ino; . . . struct odirtemplate { u_int32_t dot_ino; . . . u_int32_t dotdot_ino; . . . sys/ufs/ffs/fs.h has: struct jrefrec { . . . uint32_t jr_ino; struct jmvrec { . . . uint32_t jm_ino; struct jblkrec { . . . uint32_t jb_ino; struct jtrncrec { . . . uint32_t jt_ino; =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sat Jun 17 01:59:15 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3DB1D8F0AC for ; Sat, 17 Jun 2017 01:59:14 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-16.reflexion.net [208.70.210.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A270378B77 for ; Sat, 17 Jun 2017 01:59:13 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 29556 invoked from network); 17 Jun 2017 02:03:08 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 17 Jun 2017 02:03:08 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Fri, 16 Jun 2017 21:59:06 -0400 (EDT) Received: (qmail 30288 invoked from network); 17 Jun 2017 01:59:06 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Jun 2017 01:59:06 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 180B7EC8714; Fri, 16 Jun 2017 18:59:06 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? Date: Fri, 16 Jun 2017 18:59:05 -0700 References: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> To: kib@FreeBSD.org, FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML In-Reply-To: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> Message-Id: <700CC284-8750-448D-83DB-19C6F5CB6AE8@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 01:59:15 -0000 Top post of context note: I should have noted up front that: /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c does: #include "ufsread.c" and that is the context of the __udivdi3 use that is rejected at link time when clang is used to buildworld for powerpc or pwoerpc64. My original note might really trace back to boot1.c needing to be different even if ufsread.c stays the same. === Mark Millard markmi at dsl-only.net On 2017-Jun-16, at 5:01 PM, Mark Millard wrote: buildworld via clang for powerpc64 and powerpc fails for lack of `__udivdi3' referenced in sys/boot/common/ufsread.c fsread_size code. But this lead to me looking around and I found a conceptually separate possible issue. . . sys/sys/_types.h : typedef __uint64_t __ino_t; /* inode number */ # find /usr/src/sys/ -exec grep __ino_t {} \; -print | more typedef __ino_t ino_t; /usr/src/sys/sys/stat.h typedef __ino_t ino_t; /* inode number */ /usr/src/sys/sys/types.h typedef __uint64_t __ino_t; /* inode number */ /usr/src/sys/sys/_types.h typedef __ino_t ino_t; /usr/src/sys/sys/dirent.h sys/boot/common/ufsread.c : . . . #include #include #include . . . typedef uint32_t ufs_ino_t; . . . Note the 32-bit type above. The headers included have use of the 64-bit ino_t type as well, for example: sys/ufs/ufs/diniode.h : . . . #define UFS_ROOTINO ((ino_t)2) . . . #define UFS_WINO ((ino_t)1) . . . sys/ufs/ffs/fs.h : . . . #define ino_to_cg(fs, x) (((ino_t)(x)) / (fs)->fs_ipg) #define ino_to_fsba(fs, x) \ ((ufs2_daddr_t)(cgimin(fs, ino_to_cg(fs, (ino_t)(x))) + \ (blkstofrags((fs), ((((ino_t)(x)) % (fs)->fs_ipg) / INOPB(fs)))))) #define ino_to_fsbo(fs, x) (((ino_t)(x)) % INOPB(fs)) . . . I believe the powerpc64/powerpc issue gives evidence of ino_t being used in addition ot ufs_ino_t in sys/boot/common/ufsread.c 's fsread_size . Other things that look 32-bit inode-ish: (I do not claim to know that any of this matters.) sys/ufs/ufs/dir.h has: struct direct { u_int32_t d_ino; /* inode number of entry */ . . . struct dirtemplate { u_int32_t dot_ino; . . . u_int32_t dotdot_ino; . . . struct odirtemplate { u_int32_t dot_ino; . . . u_int32_t dotdot_ino; . . . sys/ufs/ffs/fs.h has: struct jrefrec { . . . uint32_t jr_ino; struct jmvrec { . . . uint32_t jm_ino; struct jblkrec { . . . uint32_t jb_ino; struct jtrncrec { . . . uint32_t jt_ino; === Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sat Jun 17 02:46:14 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36902D909B1 for ; Sat, 17 Jun 2017 02:46:14 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C842F7A55E for ; Sat, 17 Jun 2017 02:46:13 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-wm0-f51.google.com with SMTP id u195so23852099wmd.1 for ; Fri, 16 Jun 2017 19:46:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=ZbBCGPHFU/GzUeEC0dXyuJhGXtkR/jHYDb3e1vqw1dg=; b=OATq9YCiMEJuf8UJgAvWtGnBhj3Pd9xH/lwXd0kyrfjP25nv8Qi+t/L2rAwKhH4ITU xFhOq3hO/csl3NMC4vEKnHGzUzKEWhLC6pYSlsF3b/9dyafWSvYJLWAPjK/WDppeQLqO Z+HV8E+Ini9CyIwZguTQy6vWdXkUrF5F6EcyZoCccywDTur7MoGl2FN0JqU1ThtKjWPo kmwj9Cqvo9D/b3ZJf4g+Yno78VJnImlnAMx07t57WtZkDCvzXVswR48QcbM+oz/E186U rR1bO918wiCJx6xmGVLl7sDSl726dQfnKB0B524xZgIvZkqFKyX/ccudPZINEnVj4qx7 4u7Q== X-Gm-Message-State: AKS2vOz8XryPq6sCCUrqBs972WUR3rbCucU+BT7rX/GqOx0XXnoiNSmE S7GxJNDVUPg8T2eLfMQ= X-Received: by 10.28.143.149 with SMTP id r143mr8271156wmd.61.1497636483831; Fri, 16 Jun 2017 11:08:03 -0700 (PDT) Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com. [74.125.82.48]) by smtp.gmail.com with ESMTPSA id e77sm2039554wma.32.2017.06.16.11.08.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Jun 2017 11:08:03 -0700 (PDT) Received: by mail-wm0-f48.google.com with SMTP id d73so29885632wma.0 for ; Fri, 16 Jun 2017 11:08:03 -0700 (PDT) X-Received: by 10.80.184.129 with SMTP id l1mr8208440ede.88.1497636483498; Fri, 16 Jun 2017 11:08:03 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.164.154 with HTTP; Fri, 16 Jun 2017 11:08:03 -0700 (PDT) In-Reply-To: References: From: Conrad Meyer Date: Fri, 16 Jun 2017 11:08:03 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: efifb framebuffer info for NVIDIA driver console restore To: Aaron Plattner Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 02:46:14 -0000 Hi Aaron, I'm not sure it's the best way, but you could find EFI framebuffer information in the same way the efifb vt(4) driver (sys/dev/vt/hw/efifb/efifb.c) does, by inspecting the struct efi_fb preload data (which is passed from the loader to the kernel). Something like: caddr_t kmdp; struct efi_fb *efifb; kmdp = preload_search_by_type("elf kernel"); if (kmdp == NULL) kmdp = preload_search_by_type("elf64 kernel"); efifb = (struct efi_fb *)preload_search_info(kmdp, MODINFO_METADATA | MODINFOMD_EFI_FB); if (efifb != NULL) /* Found framebuffer information */; Hope that helps, Conrad On Fri, Jun 16, 2017 at 9:51 AM, Aaron Plattner wrote: > Hi FreeBSD hackers, > > My name is Aaron Plattner. I'm a driver developer at NVIDIA working on > the FreeBSD driver, among other things. I was hoping you could help me > out regarding some changes I've been making to the driver. > > Recently, we've revamped how our driver handles switching from > graphical modes to the console mode. For framebuffer consoles, the new > nvidia_modeset module tries to take care of it without having to fall > back to old-school VESA VBE modesets. However, in order for this to > work, the driver needs to know where the framebuffer console is in > physical memory, and its layout. > > On Linux, we get this information from the global 'screen_info' > structure: > > void NV_API_CALL os_get_screen_info( > NvU64 *pPhysicalAddress, > NvU16 *pFbWidth, > NvU16 *pFbHeight, > NvU16 *pFbDepth, > NvU16 *pFbPitch > ) > { > // > // If there is not a framebuffer console, return 0 size. > // > // orig_video_isVGA is set to 1 during early Linux kernel > // initialization, and then will be set to a value, such as > // VIDEO_TYPE_VLFB or VIDEO_TYPE_EFI if an fbdev console is used. > // > if (screen_info.orig_video_isVGA <= 1) > { > *pPhysicalAddress = 0; > *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; > return; > } > > *pPhysicalAddress = screen_info.lfb_base; > #if defined(VIDEO_CAPABILITY_64BIT_BASE) > *pPhysicalAddress |= (NvU64)screen_info.ext_lfb_base << 32; > #endif > *pFbWidth = screen_info.lfb_width; > *pFbHeight = screen_info.lfb_height; > *pFbDepth = screen_info.lfb_depth; > *pFbPitch = screen_info.lfb_linelength; > } > > This works for both legacy boot systems with vesafb as well as UEFI > systems that use Linux's efifb. > > On FreeBSD, I was able to find this information on my legacy system: > > void NV_API_CALL os_get_screen_info( > NvU64 *pPhysicalAddress, > NvU16 *pFbWidth, > NvU16 *pFbHeight, > NvU16 *pFbDepth, > NvU16 *pFbPitch > ) > { > const sc_softc_t *sc = sc_get_softc(0, SC_KERNEL_CONSOLE); > > if (sc) > { > const video_adapter_t *adp = sc->adp; > > if (adp) > { > const struct video_info *vi = &adp->va_info; > > if (vi && (vi->vi_flags & V_INFO_LINEAR)) > { > *pPhysicalAddress = vi->vi_buffer; > *pFbWidth = vi->vi_width; > *pFbHeight = vi->vi_height; > *pFbDepth = vi->vi_depth; > *pFbPitch = adp->va_line_width; > return; > } > } > } > > *pPhysicalAddress = 0; > *pFbWidth = *pFbHeight = *pFbDepth = *pFbPitch = 0; > } > > However, this doesn't work on UEFI systems because efifb doesn't go > through the video_adapter_t stuff. Does anyone know how I can get that > information from efifb, or who to talk to about adding an interface > the driver can use to find it? > > Sincerely, > Aaron > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Sat Jun 17 02:48:59 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D2AAD90B00; Sat, 17 Jun 2017 02:48:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A87AC7A6D8; Sat, 17 Jun 2017 02:48:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5H2mptV094830 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 17 Jun 2017 05:48:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5H2mptV094830 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5H2moBR094829; Sat, 17 Jun 2017 05:48:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 17 Jun 2017 05:48:50 +0300 From: Konstantin Belousov To: Mark Millard Cc: FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML Subject: Re: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? Message-ID: <20170617024850.GB2088@kib.kiev.ua> References: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 02:48:59 -0000 On Fri, Jun 16, 2017 at 05:01:43PM -0700, Mark Millard wrote: > buildworld via clang for powerpc64 and powerpc fails for lack of > `__udivdi3' referenced in sys/boot/common/ufsread.c fsread_size > code. But this lead to me looking around and I found a conceptually > separate possible issue. . . > > sys/sys/_types.h : > > typedef __uint64_t __ino_t; /* inode number */ > > # find /usr/src/sys/ -exec grep __ino_t {} \; -print | more > typedef __ino_t ino_t; > /usr/src/sys/sys/stat.h > typedef __ino_t ino_t; /* inode number */ > /usr/src/sys/sys/types.h > typedef __uint64_t __ino_t; /* inode number */ > /usr/src/sys/sys/_types.h > typedef __ino_t ino_t; > /usr/src/sys/sys/dirent.h > > > sys/boot/common/ufsread.c : > > . . . > #include > #include > #include > . . . > typedef uint32_t ufs_ino_t; > . . . > > Note the 32-bit type above. The headers included > have use of the 64-bit ino_t type as well, for > example: > > sys/ufs/ufs/diniode.h : > > . . . > #define UFS_ROOTINO ((ino_t)2) > . . . > #define UFS_WINO ((ino_t)1) > . . . > > sys/ufs/ffs/fs.h : > > . . . > #define ino_to_cg(fs, x) (((ino_t)(x)) / (fs)->fs_ipg) > #define ino_to_fsba(fs, x) \ > ((ufs2_daddr_t)(cgimin(fs, ino_to_cg(fs, (ino_t)(x))) + \ > (blkstofrags((fs), ((((ino_t)(x)) % (fs)->fs_ipg) / INOPB(fs)))))) > #define ino_to_fsbo(fs, x) (((ino_t)(x)) % INOPB(fs)) > . . . > > > I believe the powerpc64/powerpc issue > gives evidence of ino_t being used in > addition ot ufs_ino_t in > sys/boot/common/ufsread.c 's fsread_size . > > > Other things that look 32-bit inode-ish: > (I do not claim to know that any of this > matters.) > > sys/ufs/ufs/dir.h has: > > struct direct { > u_int32_t d_ino; /* inode number of entry */ > . . . > struct dirtemplate { > u_int32_t dot_ino; > . . . > u_int32_t dotdot_ino; > . . . > > struct odirtemplate { > u_int32_t dot_ino; > . . . > u_int32_t dotdot_ino; > . . . > > > sys/ufs/ffs/fs.h has: > > struct jrefrec { > . . . > uint32_t jr_ino; > > struct jmvrec { > . . . > uint32_t jm_ino; > > struct jblkrec { > . . . > uint32_t jb_ino; > > struct jtrncrec { > . . . > uint32_t jt_ino; UFS uses 32bit inodes, changing to 64bit is both pointless currently, and causes on-disk layout incompatibilities. As a consequence, use of ino_t (64bit) or uint32_t for inode numbers are almost always interchangeable, unless used for specifying on-disk layout. UFS correctly uses (and was changed to use) uint32_t for inode numbers in the disk-layout definitions. Other places, which calculate inode numbers from inode block numbers, or do some other calculations with inodes, are fine with either width. That is, I believe that all instances which I looked at during the ino64 preparation are fine. From owner-freebsd-hackers@freebsd.org Sat Jun 17 03:54:13 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE6B7BEE23C for ; Sat, 17 Jun 2017 03:54:13 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-16.reflexion.net [208.70.210.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 777147D4C2 for ; Sat, 17 Jun 2017 03:54:12 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 26687 invoked from network); 17 Jun 2017 03:55:37 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 17 Jun 2017 03:55:37 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.1) with SMTP; Fri, 16 Jun 2017 23:54:11 -0400 (EDT) Received: (qmail 16225 invoked from network); 17 Jun 2017 03:54:11 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Jun 2017 03:54:11 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id CF334EC8A8B; Fri, 16 Jun 2017 20:54:10 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? From: Mark Millard In-Reply-To: <20170617024850.GB2088@kib.kiev.ua> Date: Fri, 16 Jun 2017 20:54:10 -0700 Cc: FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <73F88E18-37A1-47C6-8783-F51F131A9671@dsl-only.net> References: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> <20170617024850.GB2088@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 03:54:14 -0000 On 2017-Jun-16, at 7:48 PM, Konstantin Belousov = wrote: > On Fri, Jun 16, 2017 at 05:01:43PM -0700, Mark Millard wrote: >> . . . >=20 > UFS uses 32bit inodes, changing to 64bit is both pointless currently, = and > causes on-disk layout incompatibilities. >=20 > As a consequence, use of ino_t (64bit) or uint32_t for inode numbers = are > almost always interchangeable, unless used for specifying on-disk = layout. > UFS correctly uses (and was changed to use) uint32_t for inode numbers > in the disk-layout definitions. Other places, which calculate inode > numbers from inode block numbers, or do some other calculations with > inodes, are fine with either width. >=20 > That is, I believe that all instances which I looked at during the > ino64 preparation are fine. Thanks for letting me know --and good to know. I've added a note to the bugzilla report of the failed linking of boot1.elf for powerpc and powerpc64 that you have indicated that if the __udivdi3 is supplied to allow the linking to complete for builds based on clang then the result should operate okay for the mix of types. (The report is bugzilla 220024 .) =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sat Jun 17 05:16:27 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9B79BF0A9D for ; Sat, 17 Jun 2017 05:16:27 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0040.outbound.protection.outlook.com [104.47.32.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7897E7FC2D for ; Sat, 17 Jun 2017 05:16:26 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=14fOniUNBibErRY9sLzGIyl3cxxS8qyvdOCDln+Z6PY=; b=Oc7eQvsbisGa5Akb2TY3CF8d9EZmoGkidbW8Ae0/ISwcv304XE3itFi3H0ucY8XwXeQZt+UbnXd+sXY6od0Sw0ha1c36vO2Oq3Pw+6So5RSpxkxEaV1yp+VchJhebUzs0T9totPHN+PAws6m0jDvGDMI77IPTvsbf7BKWrRu5fk= Received: from BN6PR03CA0069.namprd03.prod.outlook.com (10.173.137.31) by CY4PR03MB2965.namprd03.prod.outlook.com (10.175.116.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sat, 17 Jun 2017 05:16:24 +0000 Received: from BL2FFO11FD036.protection.gbl (2a01:111:f400:7c09::147) by BN6PR03CA0069.outlook.office365.com (2603:10b6:404:4c::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Sat, 17 Jun 2017 05:16:23 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.36) smtp.mailfrom=ca.weatherford.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.36 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.36; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.36) by BL2FFO11FD036.mail.protection.outlook.com (10.173.161.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Sat, 17 Jun 2017 05:16:23 +0000 Received: from BLUPR58MB002.032d.mgd.msft.net (141.251.110.146) by BLUPR58MB002.032d.mgd.msft.net (141.251.110.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Sat, 17 Jun 2017 05:16:22 +0000 Received: from BLUPR58MB002.032d.mgd.msft.net ([141.251.110.146]) by BLUPR58MB002.032d.mgd.msft.net ([141.251.110.146]) with mapi id 15.01.1178.018; Sat, 17 Jun 2017 05:16:22 +0000 From: "Caza, Aaron" To: "freebsd-hackers@freebsd.org" Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLnJr8RDocWLK5YRfCaDeFG8ME4LA== Date: Sat, 17 Jun 2017 05:16:22 +0000 Message-ID: <4561529b83ce4270b09aa0e3b12f299f@BLUPR58MB002.032d.mgd.msft.net> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.110.69] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: BLUPR58MB002:|CY4PR03MB2965: X-MS-Office365-Filtering-Correlation-Id: 5aa1474b-0600-4f05-7613-08d4b53fff4c Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: BLUPR58MB002.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.36; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(39840400002)(39850400002)(39400400002)(39450400003)(39860400002)(39410400002)(2980300002)(438002)(40224003)(199003)(13734003)(189002)(9170700003)(53946003)(7696004)(53936002)(102836003)(47776003)(229853002)(3846002)(6246003)(9686003)(6116002)(478600001)(86146001)(575784001)(97756001)(38730400002)(55016002)(22756006)(110136004)(6916009)(46406003)(33646002)(23726003)(114624004)(2906002)(189998001)(72206003)(86362001)(8936002)(106466001)(54356999)(2900100001)(2351001)(2501003)(50466002)(50986999)(8746002)(356003)(42882006)(5890100001)(305945005)(66066001)(81166006)(5660300001)(7736002)(108616004)(24736003)(5640700003)(8676002)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR03MB2965; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:sfv; A:0; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD036; 1:XPFOh0wlaEkPq4nN7Eyur5a3oSNg8fxGd/mvfJgwM5qKj3JVf3Ste13VBpkJV09p+0VuosrVfM4rHqqt2ac8NIzragkb0Vcac7wborCPcYb0qQ6oGZEo3ZSEfxpxRzCe4ee93mQ4tWhe2Y7gBEYla2pFyRfeVB5b83pIVs4WX+HMLcBOJ9OBeKP+LnUo8UD8n+Jeoc1f5292ffVVF1tpoaxxSBUpe377CaPpkM9fJJ3XkDHYCln/3y/mKmBGnEshEPlmU2n0uq94Y45EguH+q+3K0v5Z3ivnthDVjkif25EdB/kbhdtI1WNmF03ZmRD9r17vBm6mmFhOBSdu2G2vGvifRNsxxkRCESiUnDPzgoE+ECSyBIr1NapdXbQXpQtBxtXIEWKhKcAWVs4DWBfBKz5WHkoSBlMedfMRaJEm1Eop/VaF8an6F+HwAk7VocZStYnUnUyKx/oTwlwm+U99F57clNT2kpxDTAb1IkG4ZKcNzUnuhZQLnCNKGVB5LZHZysA6J423FFV605FG1Y8p5cKhNtnyK6mo+eWkOkAxAg8= X-CrossPremisesHeadersPromoted: BL2FFO11FD036.protection.gbl X-CrossPremisesHeadersFiltered: BL2FFO11FD036.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:CY4PR03MB2965; X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2965; 3:L5GyaiXhY8N/42rfGSGJxKILvtzs4j9nIoJhSIF1KP3PRcU31sJWTbhaBIx2nFUZIY43V9xDu1kTnjFNt0xXFp55Bj8R/kPHCHOT8pmCo60h6NZpdMfDghv0UdEokUdYelaTtm9fPswkoV3kvwU8x0Swh9TPJOwGnRyLhIjitIUQYz5ArcZEg4FmbgvNpeAE6kbSdHl4tyMPusqz6eMfHhcqoMVu/PkW9MGIDDPM/yCiUeQSDWNZ461Otk2LDFzSK8z3IJ7/d2Kc6hUjwgLWEyIRI1BXUvLNh3+Y9iMQ8sqZQ6O+z4X2aincQ16ADfrQxbAR7ATzUu/vXBx0OzvhlMuAL90Jexe+jbZQaDnpL/b7J8S3VEFlFjvP6RrjWtdWSAoUJ/WE4HqEpKIEI9jOLnUm08lfVCFZbRKqhFz9WUGpAYGlwXq4n1/6W02ZRCbJsNKtzCcMtoQLWSrn/0cbVCJgBZQ7hiBK/qGRIBwBQJY= X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2965; 25:b6Grr3lWdPvPiirs9M17hedx+nn/gwipaNqzi01+qK6O2CrzzgFyp9Uwy1ns7D1BMpuGfurCr7A38pnqbwn967UikiiBr7iaaMT6bylQi0uYUz+WBt5X9UQGF3Z0yI6rad0umKrwiBY+cNOZaDqQXfNi2wt+b9QNX8zQubvvZYGB7rvhsMdCKQDjGz4zLR0NkuEo0iQcfFxZrrBPmyxn0Go5Ol/kJit9GH9izb2FsDLgm1QrCc3aZ5QbP0ll+heJDtTCqh6sz5PA4T3wuWRx4MLZ3ciL6fYM0x7jmtkO7tmtaX51httGdkgHd1xZIOaQZ7k0ApFgC89HV9EZbfJvgUp9uWMb9bSQxIKB3pmntsS4UPsthcskiP62RYjxIG7qbKAoEsYnWanhKGbX/l/AH/v0kS5rPabsYiJG39UqFwPs+gkPA6q8XjsBWvSGGLBqjLUvFfb8SUufuuwlExn2R2WPJPJHeXSiYiKiMwaBZ4g=; 31:p43KU92BZLqRVl2z1XkiinGBas8lAavWKaiEgUlv0WPSMw/xUsa3CoRYcpxvF+DScD8tc0oVCCKYvot5GulIApswC3PDzcIMJF/MMQCx5XEuOcYaq7YXZOkcrDY4iJ11TC3NNDUXb1QA34mgZEPYttPv1Jtfda2+6psEPXpOuxHPwat6EtTjlZ+9oPDjPxIO5uUrNgW99d+2se03KlocjMbdwLIF9u88zzjIHqlNeZ88pirwfVA2nWZ6NxxfN2+2 X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2965; 20:A7QPvkEA+mnkNKInIGMTFZxUbGo+g5eYjhPTn5OhM92OfVIT+NvZaShFYwcfkek6XFPbUoiR3nw5E5lMnIbTjl93FJ2gSuwV6QAH9RfcPQiKEYaQKkHN4/iDjh1qtPZGt7G1JmtdwMm6YDlZZmFO52sTeP9Yp33no8ZV1eZd6LfWX/3/sorEs/zwh0J2o8ln9PAILK90RitoA1k33N0eus1gKafNpxghHu266MMRbEzGanbZSPP9ia08WX1PCODtsMcnqwACXRS40fMAfHwdGoBZbBRa/2On1BMFYOEfK2kNsVZafpEaI/5T6fFtoz4sgizyuugUO6WaNnEYgl1taQym7X5+NbW7e/0EB3wkK3MNmmw/USQ6vqe0TBz+wUAQ6/VpVXQi8/0FNO0zOosBMzzha6sy4nc/5Ln5zyTvOLB1oT1tIcEEMTKKpIPbWL17/rc8ZxZ5kzPYZCyk5URPGHbB43BHOZ8+nZ3je+n9rythtmhi6lcHhF0DeMzFbAMf X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851)(192374486261705)(131327999870524); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(93006095)(93004095)(3002001)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(20161123562025)(20161123555025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR03MB2965; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR03MB2965; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2965; 4:XgS5dvA1gFE2Q5DzyOipOXi7ERR/VHhYIVinxnvqR5?= =?us-ascii?Q?AR9khK+Cv201CFx8DIZ4ufqJzL8Ihj/xWLi0sJigFVgjDvUJn5j7Bsl5xtJC?= =?us-ascii?Q?EG4ychU/CU6ld8cfnCWoE8czw+61Jwjzki28aqpDJzRB4IuxQLevAy5TswZu?= =?us-ascii?Q?NrFLLYQEsACDueMqBJlGtIY3oLCtvBrGUuC1RdZB16rLFgI57agXT1LE7W+K?= =?us-ascii?Q?SeOOLyLvIUbphYMSE0xhfqEHESBeLUCXX80R42bmHV/9U48sAN7KRPNKHnq3?= =?us-ascii?Q?UABp3OFuZw7cz/QZdezUaO3/w3nfmkdXuLis2lEKR6RKIwrGpwUQcj0bqx5a?= =?us-ascii?Q?UhJD09cXRuWIZasgGxEq5xfuQmonM/pGT5TOWM2V3cu3QsWxKjeixRUNdJBL?= =?us-ascii?Q?E91TEMTo6DmLMpZYuvjtP42Dion9CiR9HvykMZcVDbvAIiisQWx2/7/+WPDb?= =?us-ascii?Q?YcwVNCze5ICWdLxFBMArMK8jIB1+3FTB6wRyaVSvM4ybKfoLZ/gsl6hRVens?= =?us-ascii?Q?yEV50tYDcORmjldaKPnfxshhuo+ddj6Ss6PC5kNBN28OX+sROscFZd274QwE?= =?us-ascii?Q?WHJE77XOe53tznodAlCAMyC3lx8a/L4couEwdz+GTn8UUXXWiYrNcczeRJh9?= =?us-ascii?Q?1Avn46ZBXU/eDJgSkheox0c28J6WZ2C5djG/5IgiKJYEXUhUf4HmWOO5DUBr?= =?us-ascii?Q?evnvdlcyXZ7BKLOjU5ZEHuYbpBGPFksNZCHEDJpbuw/8l7WiM1igwPoauIom?= =?us-ascii?Q?qf1hq+YrDPlIJcGbu2z7vuRcmHudxdu3gNcl4U0yKGi6LakH639dtQtPpdPI?= =?us-ascii?Q?yCRjWEgvW+D+kLCttdFSXIa+qVmoF+IhsOGjY6xOLNCYl7loIt6ibzNc0IEQ?= =?us-ascii?Q?u2dQYtgKDwcWr/fiyG3Tta95rkZQ19exopmVOeQoekd1gUMI4NBAc3onKaXV?= =?us-ascii?Q?Ru3K1jn9GiCynqIFVddXkqacElT8NaQd7K89F38HWfmHdFwecyj+V/NiU//p?= =?us-ascii?Q?S/KNVn3cvujUxdN5A8++8D2WWoazZdVEWVRxe2mgfQrJdYwoeK0X6vBewAn5?= =?us-ascii?Q?rirwvW6kE1IyWiP6wVY6oDm3XjXS04wtNg+Vf663pp4Gr6/+k9MRKe4/82oV?= =?us-ascii?Q?wsYURs0SHFSnIRtrqcBMMFOyzU1V5eJPthLkvcMLYFJ7APEQEpJ+abn+UpHj?= =?us-ascii?Q?MMd9tbtRLU+ISrNcb0iGYxVbdcJq6kHr+yz8S14HdTh7eG6lLvoWoN4d+41p?= =?us-ascii?Q?gtKxtLkEYzqdkitUNM488mnmYMiKDqbZO+iiQM3qBiPZZxyc43duroP+2R1n?= =?us-ascii?Q?eRHmEF3uoDxKrf6a6+tc6N45vkh40/f8nKVEpk5pBB?= X-Forefront-PRVS: 034119E4F6 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2965; 23:x+gVIBp/ucSRFJwiVQx8ZfEwGlMPaNI9UdyBb/fSD?= =?us-ascii?Q?UzcOApId0azh2I8bF0X5oxQrmR6pcmjoURAtC5D/RI3XynCLaDzzAHAap4Xx?= =?us-ascii?Q?6Mlni8HCqC9/SEXAsMqjSvTRqyFgsoTLdRP+p5eq0N7v2TtoDqllU76b5dPv?= =?us-ascii?Q?JlzEuZKlo4j+X/TvAImcpzuSmbT9adrJxxGEyigEcCOwoCFZR+aBfv0Iciau?= =?us-ascii?Q?X5B/jDFamcq0Ohf0ryycNjRojykP95cCuRa9HiJElfawAzbtUqfErTJjVQHC?= =?us-ascii?Q?wTPij6Xpu9DqNfb8GEedNDH60H4Y9ylCQT5NIKrnbw9zcWOXKjGYeJFz4KT5?= =?us-ascii?Q?rnDK0sae1dg6PLu4ABNfngsZa9z/5ic7E14S3SHjwGP2UW4cxA3Y3NhCVRQR?= =?us-ascii?Q?rbho9by1btB5ipOmB94xNlaQNqswdabtB2+Ism1LusqDQOPvHQVnWew0Bhqf?= =?us-ascii?Q?DAlyOBNlL44Uj3JqtPXRXpjLEm0yKeu2bMkQ+WuF+W3tGAFHNqPTMb03a8H3?= =?us-ascii?Q?Z7EcPJlDGGwLyAAPWhRSawbX/lBgRb5p7BWtMjPiu9kSVhuCXfaO7mXmN1va?= =?us-ascii?Q?jRxG8K2NdmCAv9h1GuxEb7I0+None0MblC64kKyFgmYJ9+mmjMfRUYABJffq?= =?us-ascii?Q?o3u0exFsuN6eYchqXAzIoYde1tvZoGxjVZmiE4as5txvhVM5/b5/tjgA6x68?= =?us-ascii?Q?CT/e/cph3AaDdiHmJuMLPHCCsqZMTGCPgEhoV916Qa/0rtoK1LKnEj/GaCP3?= =?us-ascii?Q?nSBudUA4MvhaxJlANxOOFUa1YFh+VIsC54SdM6WPbgmoVbYXCgC4pUVeEqQv?= =?us-ascii?Q?a6L8eYUpPj5+KYz+uOa0IgVv34KWwadEQa0JPOlsdgBytSHEmQVzdMUXRVgT?= =?us-ascii?Q?OpSS0hE71+qajWmkZ15HVw3f3otaKW3sTMNgnU5oAEZoDjpurSOgDw4vBfGP?= =?us-ascii?Q?Vz/Qv0KiSJR7GpW+OWQzgLvdEHWOLu0GC3c2Pb3pSkcJnlQ8q8neFQhgKKUu?= =?us-ascii?Q?IWNPurE0T02X+iQuQDCeVoIbhJp5ICrE694bdQCnX2WMRg9tqQFDp/tI/cOn?= =?us-ascii?Q?VynBf2CM4XmZDd3u0fh2ABrvc+qju8xHnhbWjn3mtneESBrfEs/Tcq0evAkO?= =?us-ascii?Q?J4x5fA5xlT64uh0krdcsL2bgY6H37YPSX4lGHdLJG2y86MuQeWcgP0qBh5Ai?= =?us-ascii?Q?+nfdrrtrBo7XFy7mB+sIIFbAYW9xhka7ZcwUqYawzSjKx+9/T/4qh+S2hT9r?= =?us-ascii?Q?1KFBQS06Jkuz4NbQ4UhJu7H+r49UYKa7zti5KXHbeDO5owi4nUHCSb1EJWki?= =?us-ascii?Q?PJ2f/V2qnYzVWmJasDcfpAfaSpzxE7oq3+RcdlvcNwVenvqpAI49wnz/GJ//?= =?us-ascii?Q?4i50lkG+njlb+F/14dFLUYl4tpVQ6MpaGoPEEvWQTr9YOm5WqIeMprfmyvul?= =?us-ascii?Q?IbYs66JiDUsCrU+6uT8vlmObE7Fp/edkie52xLIgU9e0nT1lb8pifttB2p/5?= =?us-ascii?Q?ZWq9vXwDm/8mo3GmYdW3fKgdFC5hi0Hh48=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2965; 6:zVI4p86/Jjhkw/Llk7VJu+AnLg1uKLspixjN/ZQSgY?= =?us-ascii?Q?V1f/0/C2hIDxNyPOpIMm2NzYEbLR2BjK6sKj0etng/zqNB2CFt2z8q28uUwG?= =?us-ascii?Q?niaqinq2mocjYcqwYVXnAqGc/xtHQyGDJOlGCgoO1H6WWETNnOg5qyAxNGJv?= =?us-ascii?Q?DWJT+zgU0Pj5OFkWGm7D2hRSZP3oA/nVORENgOnGxn5PzMAzgsf9VFE1f6JC?= =?us-ascii?Q?n3At2X2vyNvpWZ5uks43jw0JJ6esg1MoPLcQm3y+g3hau1bRhBxkOV+pYBDB?= =?us-ascii?Q?pzaCxppiq3vWBW8XVOELfSEs+qaXTybMsqyQyS+CwR0kGKHxSeCPy/0kFLfW?= =?us-ascii?Q?4Krk13ldCLPnTPc1gucuiBazahhgPR8g3umN4LHH4wpppsTOILT2C3184eu8?= =?us-ascii?Q?TMywMrGG9j9J8kPR9EUYQxMUfRDzZbBkRU0/5nAmLAFoPFRA86UG3GA2svxK?= =?us-ascii?Q?LT62ptaWoB0psIvJ2uSwmzuVZD3ZjfId09MNg93h4nsy+s1zkgYJFttJChwW?= =?us-ascii?Q?qlZDEFa1cPxmTtZmrH+IX1C4pGLnUhTa1lsEP6nLIl4PU4NVI84EDGsXxaY4?= =?us-ascii?Q?qAVaPbV9a4r1iVchg2U5hhBXzhZs3icuzleCp6aKySzN6g6eyeRBhLSPBDqg?= =?us-ascii?Q?CcLYrJPgIVfZVHEd6dBmv9ZW+yKZUMvDAguW/vM4nDw45QMokMq1sOyY6ZV2?= =?us-ascii?Q?sIAU1RDgS/A0EfxYp22bUorkAhxfiZ2wBPZ+8gYagLU0whw7AGvmGWY0MT90?= =?us-ascii?Q?7/sNoPO76OBkH5d1l9HaG4l/vIO+9QL8vJs38+2Ln5axYVhlZK8LkFi3rWUW?= =?us-ascii?Q?z6DeatzFJ3a5LSsCGvmsbZy2ezt7m4niQpUG6TTeCAavBCI/Igb073JfJWqn?= =?us-ascii?Q?eASynUVhrmlaP6CnRBcuJDfd/nXNHrjMILy6jmCqzCkCVcmH8bQWPAB3NslR?= =?us-ascii?Q?4Le+vUum1HRUc43eag+VXk5VJTkagBKl+0DSES2wEKdqLMjNcT/G9iLhCoK4?= =?us-ascii?Q?HyDI/JNaAT1DaqhDUIc+dF?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2965; 5:qCRAO/xz64tWTDRDqpz00kYBZt4blovsiY2clZx0649pBIqYDk3xq3arESfkAgCb1JbdYGfyaszi6ybtOoiDzWqs94kSaxtv2yM+h3+6o+lqEWgyFmStx/2U0oJVuP/A0UMNGx8Z0gxQL5k2igBcbZarYApnVT+Y7yUEXu3wvFJ/D7y0hriksg7DySSWqr8yUv7HS6AayKioOkIBiOuknusO2u03uzeOBgA4sZFjlLpwXpN4AFZXondzH+3mijA50CzZ27TVF+z7YOv7hNx+l1wpT507n2wi/qOqlEQQFqtcqm1H1Zg7X6WtRjo+iLlZX59OHwsZsK3/EoHIG8c9IrhTdgQN4o0DujNt4YEkkU7bSeGatAKH3MmGaArjkFpLi8eXCy2g2+zkzGb61AvZBxcJXJ95fq2WJBJ+Rm4INqvov5YrTzeO8/GcCZaqqbGBOVZNyXlNBll9STHMnMTXMnOluIVFaKakhttDYgkG85btD+t3/LZKURCfRw9vQaqu; 24:o2DJV2PeW6oajG1EyQdatZJALBszklCon6U6Cp5FxAT42ZWTmKt0zsuc+HNBl6sJmLy0oVajh2cRJ87mIRjT/hODmdKrTA55Gbyp+JaRQvY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2965; 7:Tjk8hAqd/UkWx8JC1Qp+mtIyNPOHavbaG/29VjXC5ox+SSXgCvhzV9mPirkmUb4yAEIg1l9aElYaMKtyUOks9zhmosjMmAWU9AJGUY/PnexhDhBi+/wlBUrXDwRz/gFfVIw7vOzYAWHSydSZFn6frPcrm2R5W1BjW6ymn7NFQURVaO1a1STJaf1L1p8Fg9t67rDe7qhKkyrViCB0saWNmM9k3vGXOOmwGp5Hs8VsATdL1px4FleQpK8fjXCEXVFVJIx5/dxmBKNEpsMADBmZWyfTQSRX8b4zqMMqO+rBDS6HNHGDFKBvCp9KRkWxZwuMSnLRAjrgSPJ+CWj1g0oeRFG+OB91KF5/WHwU7ITA9OuknLykzckhUog6AVeuT0r3W1c7nPWIsdJyzU7uPGak+iTme0MCpcWMgQWPvJIosBP/vs0kWywPJw+O85AM0zO8RVgz7PtkZlEJIka2YuPPM+FXDm1rGd09sv8+DTymC56lIBtbys4phNBL9pB6YK649Kvs9phqq5dvsqrww2s/AjWYZ4l7GGevXTNLFqiI35f7JPPGPUw2fFTrJ3HheCxS+8FaFBMglBjKpy4xgD1sonPvO29DlrH7UhEbHYQa/9X51FShAqJvC3H85dvvorQCRrGpySeEwr+8xKBd0aLZ6bWRA25NWLCkmzi1jR1xhqDtkxgikyxIQUULeiiAnUbGS9mQzTudN2PhEV91BJQInDR00i9Cb/esvyZfY17zVcsdMDItN1z8szeRvURTko5uXGwMebB+cm192hA8yq2CWE8uo/qqw3RQkMs04Qzigkw= X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2017 05:16:23.6538 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.36]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR03MB2965 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 05:16:28 -0000 Regarding this issue, I've now conducted testing using merely a FreeBSD 10.= 3 Stable amd64 GENERIC kernel and using dd to read a large file. The follo= wing is a log, taken hourly, of the degradation which occurred at just over= 9 hours of uptime. As the original is quite large, I've removed some sect= ions; however, these can be supplied if desired. Supplied are the initial dmesg and zpool status, logged only on startup, fo= llowed by uptime, uname -a, and zfs-stats -a output, each of which are logg= ed hourly. Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:23:44 UTC 2017 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU E31240 @ 3.30GHz (3292.60-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x206a7 Family=3D0x6 Model=3D0x2a Steppi= ng=3D7 Features=3D0xbfebfbff Features2=3D0x1dbae3ff AMD Features=3D0x28100800 AMD Features2=3D0x1 XSAVE Features=3D0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory =3D 8589934592 (8192 MB) avail memory =3D 8219299840 (7838 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 random: initialized ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 em0: port 0xf020-0xf03f mem = 0xfba00000-0xfba1ffff,0xfba24000-0xfba24fff irq 20 at device 25.0 on pci0 em0: Using an MSI interrupt em0: Ethernet address: 00:25:90:76:6b:41 ehci0: mem 0xfba23000-0xfba233ff ir= q 16 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 pcib1: irq 17 at device 28.0 on pci0 pci1: on pcib1 pcib2: irq 17 at device 28.4 on pci0 pci2: on pcib2 em1: port 0xe000-0xe01f mem = 0xfb900000-0xfb91ffff,0xfb920000-0xfb923fff irq 16 at device 0.0 on pci2 em1: Using MSIX interrupts with 3 vectors em1: Ethernet address: 00:25:90:76:6b:40 ehci1: mem 0xfba22000-0xfba223ff ir= q 23 at device 29.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci1 pcib3: at device 30.0 on pci0 pci3: on pcib3 vgapci0: mem 0xfe000000-0xfe7fffff,0xfb800000-0xfb= 803fff,0xfb000000-0xfb7fffff irq 23 at device 3.0 on pci3 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xf070-0xf077,0xf060-= 0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfba21000-0xfba217ff = irq 19 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahciem0: on ahci0 acpi_button0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 est1: on cpu1 est2: on cpu2 est3: on cpu3 est4: on cpu4 est5: on cpu5 est6: on cpu6 est7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec md0: Preloaded image 17686528 bytes at 0xffffffff81daa1b8 random: unblocking device. usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number S39KNB0HB00482Y ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada0: Command Queueing enabled ada0: 244198MB (500118192 512 byte sectors) ada0: quirks=3D0x1<4K> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: Serial Number S39KNB0HB00473Z ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada1: Command Queueing enabled ada1: 244198MB (500118192 512 byte sectors) ada1: quirks=3D0x1<4K> ses0 at ahciem0 bus 0 scbus2 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device SMP: AP CPU #1 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #7 Launched! Timecounter "TSC-low" frequency 1646298306 Hz quality 1000 Root mount waiting for: usbus1 usbus0 uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered Root mount waiting for: usbus1 usbus0 ugen1.2: at usbus1 uhub2: on = usbus1 ugen0.2: at usbus0 uhub3: on = usbus0 Root mount waiting for: usbus1 usbus0 uhub2: 6 ports with 6 removable, self powered uhub3: 6 ports with 6 removable, self powered ugen1.3: at usbus1 Trying to mount root from ufs:/dev/md0 []... bridge0: Ethernet address: 02:5d:9c:c3:f4:00 bridge0: link state changed to UP em0: promiscuous mode enabled em1: promiscuous mode enabled em0: link state changed to UP Zpool Status: pool: wwbase state: ONLINE scan: scrub repaired 0 in 0h1m with 0 errors on Sat Jun 10 18:01:26 2017 config: NAME STATE READ WRITE CKSUM wwbase ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/83b2ce34-4b9f-11e7-8d12-002590766b41 ONLINE 0 0 0 gpt/8ec0d395-4b9f-11e7-8d12-002590766b41 ONLINE 0 0 0 errors: No known data errors ---------------------------------------------------------------------------= ---- Testing SSD performance @ Fri Jun 16 19:00:00 UTC 2017 FreeBSD xyz.test 10.3-STABLE FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:= 23:44 UTC 2017 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERI= C amd64 7:00PM up 12 mins, 1 user, load averages: 0.00, 0.04, 0.07 Starting 'dd' test of large file...please wait 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 18.844918 secs (890278004 bytes/sec) ------------------------------------------------------------------------ ZFS Subsystem ReportFri Jun 16 19:00:18 2017 ------------------------------------------------------------------------ System Information: Kernel Version:1003514 (osreldate) Hardware Platform:amd64 Processor Architecture:amd64 ZFS Storage pool Version:5000 ZFS Filesystem Version:5 FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:23:44 UTC 2017 root 7:00PM up 13 mins, 1 user, load averages: 2.27, 0.57, 0.25 ------------------------------------------------------------------------ System Memory: 0.19%14.90MiB Active,0.38%29.62MiB Inact 4.31%339.96MiB Wired,0.00%0 Cache 95.13%7.33GiB Free,0.00%4.00KiB Gap Real Installed:8.00GiB Real Available:99.18%7.93GiB Real Managed:97.11%7.71GiB Logical Total:8.00GiB Logical Used:8.02%656.83MiB Logical Free:91.98%7.36GiB Kernel Memory:102.71MiB Data:73.04%75.02MiB Text:26.96%27.69MiB Kernel Memory Map:7.71GiB Size:2.48%196.06MiB Free:97.52%7.51GiB ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count:0 ARC Misc: Deleted:2.06m Recycle Misses:0 Mutex Misses:10.06k Evict Skips:1.49m ARC Size:103.44%52.76MiB Target Size: (Adaptive)100.00%51.00MiB Min Size (Hard Limit):98.04%50.00MiB Max Size (High Water):1:151.00MiB ARC Size Breakdown: Recently Used Cache Size:89.95%47.45MiB Frequently Used Cache Size:10.05%5.30MiB ARC Hash Breakdown: Elements Max:11.92k Elements Current:73.83%8.80k Collisions:19.65k Chain Max:2 Chains:25 ------------------------------------------------------------------------ ARC Efficiency:4.06m Cache Hit Ratio:49.05%1.99m Cache Miss Ratio:50.95%2.07m Actual Hit Ratio:49.05%1.99m Data Demand Efficiency:99.91%1.94m Data Prefetch Efficiency:0.01%2.05m CACHE HITS BY CACHE LIST: Anonymously Used:0.01%159 Most Recently Used:99.82%1.99m Most Frequently Used:0.16%3.28k Most Recently Used Ghost:0.00%9 Most Frequently Used Ghost:0.00%83 CACHE HITS BY DATA TYPE: Demand Data:97.40%1.94m Prefetch Data:0.01%151 Demand Metadata:2.59%51.44k Prefetch Metadata:0.01%100 CACHE MISSES BY DATA TYPE: Demand Data:0.09%1.76k Prefetch Data:99.07%2.05m Demand Metadata:0.07%1.50k Prefetch Metadata:0.77%16.01k ------------------------------------------------------------------------ L2ARC is disabled ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency:46.70k Hit Ratio:34.40%16.06k Miss Ratio:65.60%30.64k Colinear:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 Stride:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 DMU Misc: Reclaim:0 Successes:100.00%0 Failures:100.00%0 Streams:0 +Resets:100.00%0 -Resets:100.00%0 Bogus:0 ------------------------------------------------------------------------ VDEV Cache Summary:16.52k Hit Ratio:1.65%272 Miss Ratio:93.47%15.44k Delegations:4.88%806 ------------------------------------------------------------------------ ZFS Tunables (sysctl): kern.maxusers 843 vm.kmem_size 8273297408 vm.kmem_size_scale 1 vm.kmem_size_min 0 vm.kmem_size_max 1319413950874 vfs.zfs.trim.max_interval 1 vfs.zfs.trim.timeout 30 vfs.zfs.trim.txg_delay 32 vfs.zfs.trim.enabled 0 vfs.zfs.vol.unmap_enabled 1 vfs.zfs.vol.mode 1 vfs.zfs.version.zpl 5 vfs.zfs.version.spa 5000 vfs.zfs.version.acl 1 vfs.zfs.version.ioctl 7 vfs.zfs.debug 0 vfs.zfs.super_owner 0 vfs.zfs.sync_pass_rewrite 2 vfs.zfs.sync_pass_dont_compress 5 vfs.zfs.sync_pass_deferred_free 2 vfs.zfs.zio.dva_throttle_enabled 1 vfs.zfs.zio.exclude_metadata 0 vfs.zfs.zio.use_uma 1 vfs.zfs.cache_flush_disable 0 vfs.zfs.zil_replay_disable 0 vfs.zfs.min_auto_ashift 9 vfs.zfs.max_auto_ashift 13 vfs.zfs.vdev.trim_max_pending 10000 vfs.zfs.vdev.bio_delete_disable 0 vfs.zfs.vdev.bio_flush_disable 0 vfs.zfs.vdev.queue_depth_pct 1000 vfs.zfs.vdev.write_gap_limit 4096 vfs.zfs.vdev.read_gap_limit 32768 vfs.zfs.vdev.aggregation_limit 131072 vfs.zfs.vdev.trim_max_active 64 vfs.zfs.vdev.trim_min_active 1 vfs.zfs.vdev.scrub_max_active 2 vfs.zfs.vdev.scrub_min_active 1 vfs.zfs.vdev.async_write_max_active 10 vfs.zfs.vdev.async_write_min_active 1 vfs.zfs.vdev.async_read_max_active 3 vfs.zfs.vdev.async_read_min_active 1 vfs.zfs.vdev.sync_write_max_active 10 vfs.zfs.vdev.sync_write_min_active 10 vfs.zfs.vdev.sync_read_max_active 10 vfs.zfs.vdev.sync_read_min_active 10 vfs.zfs.vdev.max_active 1000 vfs.zfs.vdev.async_write_active_max_dirty_percent60 vfs.zfs.vdev.async_write_active_min_dirty_percent30 vfs.zfs.vdev.mirror.non_rotating_seek_inc1 vfs.zfs.vdev.mirror.non_rotating_inc 0 vfs.zfs.vdev.mirror.rotating_seek_offset1048576 vfs.zfs.vdev.mirror.rotating_seek_inc 5 vfs.zfs.vdev.mirror.rotating_inc 0 vfs.zfs.vdev.trim_on_init 1 vfs.zfs.vdev.cache.bshift 16 vfs.zfs.vdev.cache.size 10485760 vfs.zfs.vdev.cache.max 16384 vfs.zfs.vdev.metaslabs_per_vdev 200 vfs.zfs.txg.timeout 5 vfs.zfs.space_map_blksz 4096 vfs.zfs.spa_min_slop 134217728 vfs.zfs.spa_slop_shift 5 vfs.zfs.spa_asize_inflation 24 vfs.zfs.deadman_enabled 1 vfs.zfs.deadman_checktime_ms 5000 vfs.zfs.deadman_synctime_ms 1000000 vfs.zfs.debug_flags 0 vfs.zfs.debugflags 0 vfs.zfs.recover 0 vfs.zfs.spa_load_verify_data 1 vfs.zfs.spa_load_verify_metadata 1 vfs.zfs.spa_load_verify_maxinflight 10000 vfs.zfs.ccw_retry_interval 300 vfs.zfs.check_hostid 1 vfs.zfs.mg_fragmentation_threshold 85 vfs.zfs.mg_noalloc_threshold 0 vfs.zfs.condense_pct 200 vfs.zfs.metaslab.bias_enabled 1 vfs.zfs.metaslab.lba_weighting_enabled 1 vfs.zfs.metaslab.fragmentation_factor_enabled1 vfs.zfs.metaslab.preload_enabled 1 vfs.zfs.metaslab.preload_limit 3 vfs.zfs.metaslab.unload_delay 8 vfs.zfs.metaslab.load_pct 50 vfs.zfs.metaslab.min_alloc_size 33554432 vfs.zfs.metaslab.df_free_pct 4 vfs.zfs.metaslab.df_alloc_threshold 131072 vfs.zfs.metaslab.debug_unload 0 vfs.zfs.metaslab.debug_load 0 vfs.zfs.metaslab.fragmentation_threshold70 vfs.zfs.metaslab.gang_bang 16777217 vfs.zfs.free_bpobj_enabled 1 vfs.zfs.free_max_blocks -1 vfs.zfs.no_scrub_prefetch 0 vfs.zfs.no_scrub_io 0 vfs.zfs.resilver_min_time_ms 3000 vfs.zfs.free_min_time_ms 1000 vfs.zfs.scan_min_time_ms 1000 vfs.zfs.scan_idle 50 vfs.zfs.scrub_delay 4 vfs.zfs.resilver_delay 2 vfs.zfs.top_maxinflight 32 vfs.zfs.zfetch.array_rd_sz 1048576 vfs.zfs.zfetch.max_distance 8388608 vfs.zfs.zfetch.min_sec_reap 2 vfs.zfs.zfetch.max_streams 8 vfs.zfs.prefetch_disable 0 vfs.zfs.delay_scale 500000 vfs.zfs.delay_min_dirty_percent 60 vfs.zfs.dirty_data_sync 67108864 vfs.zfs.dirty_data_max_percent 10 vfs.zfs.dirty_data_max_max 4294967296 vfs.zfs.dirty_data_max 851961036 vfs.zfs.max_recordsize 1048576 vfs.zfs.send_holes_without_birth_time 1 vfs.zfs.mdcomp_disable 0 vfs.zfs.nopwrite_enabled 1 vfs.zfs.dedup.prefetch 1 vfs.zfs.l2c_only_size 0 vfs.zfs.mfu_ghost_data_esize 0 vfs.zfs.mfu_ghost_metadata_esize 39650304 vfs.zfs.mfu_ghost_size 39650304 vfs.zfs.mfu_data_esize 0 vfs.zfs.mfu_metadata_esize 2785280 vfs.zfs.mfu_size 2839040 vfs.zfs.mru_ghost_data_esize 253952 vfs.zfs.mru_ghost_metadata_esize 7712768 vfs.zfs.mru_ghost_size 7966720 vfs.zfs.mru_data_esize 40481280 vfs.zfs.mru_metadata_esize 3874816 vfs.zfs.mru_size 49604608 vfs.zfs.anon_data_esize 0 vfs.zfs.anon_metadata_esize 0 vfs.zfs.anon_size 28672 vfs.zfs.l2arc_norw 1 vfs.zfs.l2arc_feed_again 1 vfs.zfs.l2arc_noprefetch 1 vfs.zfs.l2arc_feed_min_ms 200 vfs.zfs.l2arc_feed_secs 1 vfs.zfs.l2arc_headroom 2 vfs.zfs.l2arc_write_boost 8388608 vfs.zfs.l2arc_write_max 8388608 vfs.zfs.arc_meta_limit 13369344 vfs.zfs.arc_free_target 14047 vfs.zfs.compressed_arc_enabled 1 vfs.zfs.arc_shrink_shift 7 vfs.zfs.arc_average_blocksize 8192 vfs.zfs.arc_min 52428800 vfs.zfs.arc_max 53477376 ------------------------------------------------------------------------ SSD performance testing completed @ Fri Jun 16 19:00:19 UTC 2017 ---------------------------------------------------------------------------= ---- This section was removed in the interests of brevity but can be supplied is required. ---------------------------------------------------------------------------= ---- Testing SSD performance @ Sat Jun 17 03:00:00 UTC 2017 FreeBSD xyz.test 10.3-STABLE FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:= 23:44 UTC 2017 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERI= C amd64 3:00AM up 8:12, 0 users, load averages: 0.00, 0.00, 0.00 Starting 'dd' test of large file...please wait 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 18.995611 secs (883215382 bytes/sec) ------------------------------------------------------------------------ ZFS Subsystem ReportSat Jun 17 03:00:19 2017 ------------------------------------------------------------------------ System Information: Kernel Version:1003514 (osreldate) Hardware Platform:amd64 Processor Architecture:amd64 ZFS Storage pool Version:5000 ZFS Filesystem Version:5 FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:23:44 UTC 2017 root 3:00AM up 8:13, 0 users, load averages: 3.41, 0.80, 0.29 ------------------------------------------------------------------------ System Memory: 0.04%2.78MiB Active,0.55%43.71MiB Inact 4.59%361.96MiB Wired,0.01%632.00KiB Cache 94.82%7.31GiB Free,0.00%4.00KiB Gap Real Installed:8.00GiB Real Available:99.18%7.93GiB Real Managed:97.11%7.71GiB Logical Total:8.00GiB Logical Used:8.14%666.72MiB Logical Free:91.86%7.35GiB Kernel Memory:103.60MiB Data:73.27%75.91MiB Text:26.73%27.69MiB Kernel Memory Map:7.71GiB Size:2.66%210.04MiB Free:97.34%7.50GiB ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count:0 ARC Misc: Deleted:18.57m Recycle Misses:0 Mutex Misses:91.37k Evict Skips:13.33m ARC Size:110.62%56.41MiB Target Size: (Adaptive)100.00%51.00MiB Min Size (Hard Limit):98.04%50.00MiB Max Size (High Water):1:151.00MiB ARC Size Breakdown: Recently Used Cache Size:84.43%47.63MiB Frequently Used Cache Size:15.57%8.78MiB ARC Hash Breakdown: Elements Max:11.93k Elements Current:80.43%9.59k Collisions:193.31k Chain Max:3 Chains:30 ------------------------------------------------------------------------ ARC Efficiency:36.57m Cache Hit Ratio:49.18%17.98m Cache Miss Ratio:50.82%18.58m Actual Hit Ratio:49.18%17.98m Data Demand Efficiency:99.96%17.53m Data Prefetch Efficiency:0.00%18.43m CACHE HITS BY CACHE LIST: Anonymously Used:0.00%588 Most Recently Used:99.83%17.95m Most Frequently Used:0.16%29.16k Most Recently Used Ghost:0.00%56 Most Frequently Used Ghost:0.00%263 CACHE HITS BY DATA TYPE: Demand Data:97.46%17.53m Prefetch Data:0.00%633 Demand Metadata:2.54%456.26k Prefetch Metadata:0.00%275 CACHE MISSES BY DATA TYPE: Demand Data:0.04%7.74k Prefetch Data:99.15%18.43m Demand Metadata:0.03%5.77k Prefetch Metadata:0.77%143.89k ------------------------------------------------------------------------ L2ARC is disabled ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency:597.55k Hit Ratio:24.22%144.70k Miss Ratio:75.78%452.85k Colinear:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 Stride:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 DMU Misc: Reclaim:0 Successes:100.00%0 Failures:100.00%0 Streams:0 +Resets:100.00%0 -Resets:100.00%0 Bogus:0 ------------------------------------------------------------------------ VDEV Cache Summary:138.31k Hit Ratio:0.57%794 Miss Ratio:98.82%136.67k Delegations:0.61%840 ------------------------------------------------------------------------ ZFS Tunables (sysctl): kern.maxusers 843 vm.kmem_size 8273297408 vm.kmem_size_scale 1 vm.kmem_size_min 0 vm.kmem_size_max 1319413950874 vfs.zfs.trim.max_interval 1 vfs.zfs.trim.timeout 30 vfs.zfs.trim.txg_delay 32 vfs.zfs.trim.enabled 0 vfs.zfs.vol.unmap_enabled 1 vfs.zfs.vol.mode 1 vfs.zfs.version.zpl 5 vfs.zfs.version.spa 5000 vfs.zfs.version.acl 1 vfs.zfs.version.ioctl 7 vfs.zfs.debug 0 vfs.zfs.super_owner 0 vfs.zfs.sync_pass_rewrite 2 vfs.zfs.sync_pass_dont_compress 5 vfs.zfs.sync_pass_deferred_free 2 vfs.zfs.zio.dva_throttle_enabled 1 vfs.zfs.zio.exclude_metadata 0 vfs.zfs.zio.use_uma 1 vfs.zfs.cache_flush_disable 0 vfs.zfs.zil_replay_disable 0 vfs.zfs.min_auto_ashift 9 vfs.zfs.max_auto_ashift 13 vfs.zfs.vdev.trim_max_pending 10000 vfs.zfs.vdev.bio_delete_disable 0 vfs.zfs.vdev.bio_flush_disable 0 vfs.zfs.vdev.queue_depth_pct 1000 vfs.zfs.vdev.write_gap_limit 4096 vfs.zfs.vdev.read_gap_limit 32768 vfs.zfs.vdev.aggregation_limit 131072 vfs.zfs.vdev.trim_max_active 64 vfs.zfs.vdev.trim_min_active 1 vfs.zfs.vdev.scrub_max_active 2 vfs.zfs.vdev.scrub_min_active 1 vfs.zfs.vdev.async_write_max_active 10 vfs.zfs.vdev.async_write_min_active 1 vfs.zfs.vdev.async_read_max_active 3 vfs.zfs.vdev.async_read_min_active 1 vfs.zfs.vdev.sync_write_max_active 10 vfs.zfs.vdev.sync_write_min_active 10 vfs.zfs.vdev.sync_read_max_active 10 vfs.zfs.vdev.sync_read_min_active 10 vfs.zfs.vdev.max_active 1000 vfs.zfs.vdev.async_write_active_max_dirty_percent60 vfs.zfs.vdev.async_write_active_min_dirty_percent30 vfs.zfs.vdev.mirror.non_rotating_seek_inc1 vfs.zfs.vdev.mirror.non_rotating_inc 0 vfs.zfs.vdev.mirror.rotating_seek_offset1048576 vfs.zfs.vdev.mirror.rotating_seek_inc 5 vfs.zfs.vdev.mirror.rotating_inc 0 vfs.zfs.vdev.trim_on_init 1 vfs.zfs.vdev.cache.bshift 16 vfs.zfs.vdev.cache.size 10485760 vfs.zfs.vdev.cache.max 16384 vfs.zfs.vdev.metaslabs_per_vdev 200 vfs.zfs.txg.timeout 5 vfs.zfs.space_map_blksz 4096 vfs.zfs.spa_min_slop 134217728 vfs.zfs.spa_slop_shift 5 vfs.zfs.spa_asize_inflation 24 vfs.zfs.deadman_enabled 1 vfs.zfs.deadman_checktime_ms 5000 vfs.zfs.deadman_synctime_ms 1000000 vfs.zfs.debug_flags 0 vfs.zfs.debugflags 0 vfs.zfs.recover 0 vfs.zfs.spa_load_verify_data 1 vfs.zfs.spa_load_verify_metadata 1 vfs.zfs.spa_load_verify_maxinflight 10000 vfs.zfs.ccw_retry_interval 300 vfs.zfs.check_hostid 1 vfs.zfs.mg_fragmentation_threshold 85 vfs.zfs.mg_noalloc_threshold 0 vfs.zfs.condense_pct 200 vfs.zfs.metaslab.bias_enabled 1 vfs.zfs.metaslab.lba_weighting_enabled 1 vfs.zfs.metaslab.fragmentation_factor_enabled1 vfs.zfs.metaslab.preload_enabled 1 vfs.zfs.metaslab.preload_limit 3 vfs.zfs.metaslab.unload_delay 8 vfs.zfs.metaslab.load_pct 50 vfs.zfs.metaslab.min_alloc_size 33554432 vfs.zfs.metaslab.df_free_pct 4 vfs.zfs.metaslab.df_alloc_threshold 131072 vfs.zfs.metaslab.debug_unload 0 vfs.zfs.metaslab.debug_load 0 vfs.zfs.metaslab.fragmentation_threshold70 vfs.zfs.metaslab.gang_bang 16777217 vfs.zfs.free_bpobj_enabled 1 vfs.zfs.free_max_blocks -1 vfs.zfs.no_scrub_prefetch 0 vfs.zfs.no_scrub_io 0 vfs.zfs.resilver_min_time_ms 3000 vfs.zfs.free_min_time_ms 1000 vfs.zfs.scan_min_time_ms 1000 vfs.zfs.scan_idle 50 vfs.zfs.scrub_delay 4 vfs.zfs.resilver_delay 2 vfs.zfs.top_maxinflight 32 vfs.zfs.zfetch.array_rd_sz 1048576 vfs.zfs.zfetch.max_distance 8388608 vfs.zfs.zfetch.min_sec_reap 2 vfs.zfs.zfetch.max_streams 8 vfs.zfs.prefetch_disable 0 vfs.zfs.delay_scale 500000 vfs.zfs.delay_min_dirty_percent 60 vfs.zfs.dirty_data_sync 67108864 vfs.zfs.dirty_data_max_percent 10 vfs.zfs.dirty_data_max_max 4294967296 vfs.zfs.dirty_data_max 851961036 vfs.zfs.max_recordsize 1048576 vfs.zfs.send_holes_without_birth_time 1 vfs.zfs.mdcomp_disable 0 vfs.zfs.nopwrite_enabled 1 vfs.zfs.dedup.prefetch 1 vfs.zfs.l2c_only_size 0 vfs.zfs.mfu_ghost_data_esize 0 vfs.zfs.mfu_ghost_metadata_esize 41795584 vfs.zfs.mfu_ghost_size 41795584 vfs.zfs.mfu_data_esize 0 vfs.zfs.mfu_metadata_esize 2670592 vfs.zfs.mfu_size 2719744 vfs.zfs.mru_ghost_data_esize 1622016 vfs.zfs.mru_ghost_metadata_esize 6819840 vfs.zfs.mru_ghost_size 8441856 vfs.zfs.mru_data_esize 42278912 vfs.zfs.mru_metadata_esize 4341760 vfs.zfs.mru_size 52709376 vfs.zfs.anon_data_esize 0 vfs.zfs.anon_metadata_esize 0 vfs.zfs.anon_size 147456 vfs.zfs.l2arc_norw 1 vfs.zfs.l2arc_feed_again 1 vfs.zfs.l2arc_noprefetch 1 vfs.zfs.l2arc_feed_min_ms 200 vfs.zfs.l2arc_feed_secs 1 vfs.zfs.l2arc_headroom 2 vfs.zfs.l2arc_write_boost 8388608 vfs.zfs.l2arc_write_max 8388608 vfs.zfs.arc_meta_limit 13369344 vfs.zfs.arc_free_target 14047 vfs.zfs.compressed_arc_enabled 1 vfs.zfs.arc_shrink_shift 7 vfs.zfs.arc_average_blocksize 8192 vfs.zfs.arc_min 52428800 vfs.zfs.arc_max 53477376 ------------------------------------------------------------------------ SSD performance testing completed @ Sat Jun 17 03:00:19 UTC 2017 ---------------------------------------------------------------------------= ---- Testing SSD performance @ Sat Jun 17 04:00:00 UTC 2017 FreeBSD xyz.test 10.3-STABLE FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:= 23:44 UTC 2017 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERI= C amd64 4:00AM up 9:12, 0 users, load averages: 0.00, 0.00, 0.00 Starting 'dd' test of large file...please wait 16000+0 records in 16000+0 records out 16777216000 bytes transferred in 268.165167 secs (62562995 bytes/sec) ------------------------------------------------------------------------ ZFS Subsystem ReportSat Jun 17 04:04:28 2017 ------------------------------------------------------------------------ System Information: Kernel Version:1003514 (osreldate) Hardware Platform:amd64 Processor Architecture:amd64 ZFS Storage pool Version:5000 ZFS Filesystem Version:5 FreeBSD 10.3-STABLE #0 r319701: Mon Jun 12 19:23:44 UTC 2017 root 4:04AM up 9:17, 0 users, load averages: 1.05, 0.67, 0.30 ------------------------------------------------------------------------ System Memory: 0.04%2.80MiB Active,0.60%47.30MiB Inact 5.90%465.19MiB Wired,0.07%5.36MiB Cache 93.40%7.20GiB Free,0.00%4.00KiB Gap Real Installed:8.00GiB Real Available:99.18%7.93GiB Real Managed:97.11%7.71GiB Logical Total:8.00GiB Logical Used:9.40%769.96MiB Logical Free:90.60%7.25GiB Kernel Memory:108.62MiB Data:74.51%80.93MiB Text:25.49%27.69MiB Kernel Memory Map:7.71GiB Size:2.68%211.31MiB Free:97.32%7.50GiB ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count:0 ARC Misc: Deleted:20.64m Recycle Misses:0 Mutex Misses:682.38k Evict Skips:2.33b ARC Size:141.09%71.95MiB Target Size: (Adaptive)100.00%51.00MiB Min Size (Hard Limit):98.04%50.00MiB Max Size (High Water):1:151.00MiB ARC Size Breakdown: Recently Used Cache Size:66.20%47.63MiB Frequently Used Cache Size:33.80%24.32MiB ARC Hash Breakdown: Elements Max:11.93k Elements Current:41.76%4.98k Collisions:204.57k Chain Max:3 Chains:7 ------------------------------------------------------------------------ ARC Efficiency:40.75m Cache Hit Ratio:49.30%20.09m Cache Miss Ratio:50.70%20.66m Actual Hit Ratio:49.29%20.08m Data Demand Efficiency:99.96%19.58m Data Prefetch Efficiency:0.00%20.47m CACHE HITS BY CACHE LIST: Anonymously Used:0.02%3.46k Most Recently Used:99.63%20.02m Most Frequently Used:0.33%66.92k Most Recently Used Ghost:0.01%2.68k Most Frequently Used Ghost:0.00%604 CACHE HITS BY DATA TYPE: Demand Data:97.43%19.57m Prefetch Data:0.00%639 Demand Metadata:2.54%510.40k Prefetch Metadata:0.03%6.10k CACHE MISSES BY DATA TYPE: Demand Data:0.04%8.56k Prefetch Data:99.10%20.47m Demand Metadata:0.08%16.61k Prefetch Metadata:0.78%160.96k ------------------------------------------------------------------------ L2ARC is disabled ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency:775.18k Hit Ratio:20.75%160.83k Miss Ratio:79.25%614.36k Colinear:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 Stride:0 Hit Ratio:100.00%0 Miss Ratio:100.00%0 DMU Misc: Reclaim:0 Successes:100.00%0 Failures:100.00%0 Streams:0 +Resets:100.00%0 -Resets:100.00%0 Bogus:0 ------------------------------------------------------------------------ VDEV Cache Summary:156.84k Hit Ratio:1.80%2.83k Miss Ratio:97.53%152.96k Delegations:0.67%1.05k ------------------------------------------------------------------------ ZFS Tunables (sysctl): kern.maxusers 843 vm.kmem_size 8273297408 vm.kmem_size_scale 1 vm.kmem_size_min 0 vm.kmem_size_max 1319413950874 vfs.zfs.trim.max_interval 1 vfs.zfs.trim.timeout 30 vfs.zfs.trim.txg_delay 32 vfs.zfs.trim.enabled 0 vfs.zfs.vol.unmap_enabled 1 vfs.zfs.vol.mode 1 vfs.zfs.version.zpl 5 vfs.zfs.version.spa 5000 vfs.zfs.version.acl 1 vfs.zfs.version.ioctl 7 vfs.zfs.debug 0 vfs.zfs.super_owner 0 vfs.zfs.sync_pass_rewrite 2 vfs.zfs.sync_pass_dont_compress 5 vfs.zfs.sync_pass_deferred_free 2 vfs.zfs.zio.dva_throttle_enabled 1 vfs.zfs.zio.exclude_metadata 0 vfs.zfs.zio.use_uma 1 vfs.zfs.cache_flush_disable 0 vfs.zfs.zil_replay_disable 0 vfs.zfs.min_auto_ashift 9 vfs.zfs.max_auto_ashift 13 vfs.zfs.vdev.trim_max_pending 10000 vfs.zfs.vdev.bio_delete_disable 0 vfs.zfs.vdev.bio_flush_disable 0 vfs.zfs.vdev.queue_depth_pct 1000 vfs.zfs.vdev.write_gap_limit 4096 vfs.zfs.vdev.read_gap_limit 32768 vfs.zfs.vdev.aggregation_limit 131072 vfs.zfs.vdev.trim_max_active 64 vfs.zfs.vdev.trim_min_active 1 vfs.zfs.vdev.scrub_max_active 2 vfs.zfs.vdev.scrub_min_active 1 vfs.zfs.vdev.async_write_max_active 10 vfs.zfs.vdev.async_write_min_active 1 vfs.zfs.vdev.async_read_max_active 3 vfs.zfs.vdev.async_read_min_active 1 vfs.zfs.vdev.sync_write_max_active 10 vfs.zfs.vdev.sync_write_min_active 10 vfs.zfs.vdev.sync_read_max_active 10 vfs.zfs.vdev.sync_read_min_active 10 vfs.zfs.vdev.max_active 1000 vfs.zfs.vdev.async_write_active_max_dirty_percent60 vfs.zfs.vdev.async_write_active_min_dirty_percent30 vfs.zfs.vdev.mirror.non_rotating_seek_inc1 vfs.zfs.vdev.mirror.non_rotating_inc 0 vfs.zfs.vdev.mirror.rotating_seek_offset1048576 vfs.zfs.vdev.mirror.rotating_seek_inc 5 vfs.zfs.vdev.mirror.rotating_inc 0 vfs.zfs.vdev.trim_on_init 1 vfs.zfs.vdev.cache.bshift 16 vfs.zfs.vdev.cache.size 10485760 vfs.zfs.vdev.cache.max 16384 vfs.zfs.vdev.metaslabs_per_vdev 200 vfs.zfs.txg.timeout 5 vfs.zfs.space_map_blksz 4096 vfs.zfs.spa_min_slop 134217728 vfs.zfs.spa_slop_shift 5 vfs.zfs.spa_asize_inflation 24 vfs.zfs.deadman_enabled 1 vfs.zfs.deadman_checktime_ms 5000 vfs.zfs.deadman_synctime_ms 1000000 vfs.zfs.debug_flags 0 vfs.zfs.debugflags 0 vfs.zfs.recover 0 vfs.zfs.spa_load_verify_data 1 vfs.zfs.spa_load_verify_metadata 1 vfs.zfs.spa_load_verify_maxinflight 10000 vfs.zfs.ccw_retry_interval 300 vfs.zfs.check_hostid 1 vfs.zfs.mg_fragmentation_threshold 85 vfs.zfs.mg_noalloc_threshold 0 vfs.zfs.condense_pct 200 vfs.zfs.metaslab.bias_enabled 1 vfs.zfs.metaslab.lba_weighting_enabled 1 vfs.zfs.metaslab.fragmentation_factor_enabled1 vfs.zfs.metaslab.preload_enabled 1 vfs.zfs.metaslab.preload_limit 3 vfs.zfs.metaslab.unload_delay 8 vfs.zfs.metaslab.load_pct 50 vfs.zfs.metaslab.min_alloc_size 33554432 vfs.zfs.metaslab.df_free_pct 4 vfs.zfs.metaslab.df_alloc_threshold 131072 vfs.zfs.metaslab.debug_unload 0 vfs.zfs.metaslab.debug_load 0 vfs.zfs.metaslab.fragmentation_threshold70 vfs.zfs.metaslab.gang_bang 16777217 vfs.zfs.free_bpobj_enabled 1 vfs.zfs.free_max_blocks -1 vfs.zfs.no_scrub_prefetch 0 vfs.zfs.no_scrub_io 0 vfs.zfs.resilver_min_time_ms 3000 vfs.zfs.free_min_time_ms 1000 vfs.zfs.scan_min_time_ms 1000 vfs.zfs.scan_idle 50 vfs.zfs.scrub_delay 4 vfs.zfs.resilver_delay 2 vfs.zfs.top_maxinflight 32 vfs.zfs.zfetch.array_rd_sz 1048576 vfs.zfs.zfetch.max_distance 8388608 vfs.zfs.zfetch.min_sec_reap 2 vfs.zfs.zfetch.max_streams 8 vfs.zfs.prefetch_disable 0 vfs.zfs.delay_scale 500000 vfs.zfs.delay_min_dirty_percent 60 vfs.zfs.dirty_data_sync 67108864 vfs.zfs.dirty_data_max_percent 10 vfs.zfs.dirty_data_max_max 4294967296 vfs.zfs.dirty_data_max 851961036 vfs.zfs.max_recordsize 1048576 vfs.zfs.send_holes_without_birth_time 1 vfs.zfs.mdcomp_disable 0 vfs.zfs.nopwrite_enabled 1 vfs.zfs.dedup.prefetch 1 vfs.zfs.l2c_only_size 0 vfs.zfs.mfu_ghost_data_esize 1933312 vfs.zfs.mfu_ghost_metadata_esize 45613056 vfs.zfs.mfu_ghost_size 47546368 vfs.zfs.mfu_data_esize 0 vfs.zfs.mfu_metadata_esize 475136 vfs.zfs.mfu_size 1127936 vfs.zfs.mru_ghost_data_esize 4771840 vfs.zfs.mru_ghost_metadata_esize 1060864 vfs.zfs.mru_ghost_size 5832704 vfs.zfs.mru_data_esize 0 vfs.zfs.mru_metadata_esize 0 vfs.zfs.mru_size 26115584 vfs.zfs.anon_data_esize 0 vfs.zfs.anon_metadata_esize 0 vfs.zfs.anon_size 147456 vfs.zfs.l2arc_norw 1 vfs.zfs.l2arc_feed_again 1 vfs.zfs.l2arc_noprefetch 1 vfs.zfs.l2arc_feed_min_ms 200 vfs.zfs.l2arc_feed_secs 1 vfs.zfs.l2arc_headroom 2 vfs.zfs.l2arc_write_boost 8388608 vfs.zfs.l2arc_write_max 8388608 vfs.zfs.arc_meta_limit 13369344 vfs.zfs.arc_free_target 14047 vfs.zfs.compressed_arc_enabled 1 vfs.zfs.arc_shrink_shift 7 vfs.zfs.arc_average_blocksize 8192 vfs.zfs.arc_min 52428800 vfs.zfs.arc_max 53477376 ------------------------------------------------------------------------ SSD performance testing completed @ Sat Jun 17 04:04:28 UTC 2017 ---------------------------------------------------------------------------= ---- Hopefully, the above proves useful to help track down this issue. -- Aaron This message may contain confidential and privileged information. If it has= been sent to you in error, please reply to advise the sender of the error = and then immediately delete it. If you are not the intended recipient, do n= ot read, copy, disclose or otherwise use this message. The sender disclaims= any liability for such unauthorized use. PLEASE NOTE that all incoming e-m= ails sent to Weatherford e-mail accounts will be archived and may be scanne= d by us and/or by external service providers to detect and prevent threats = to our systems, investigate illegal or inappropriate behavior, and/or elimi= nate unsolicited promotional e-mails (spam). This process could result in d= eletion of a legitimate e-mail before it is read by its intended recipient = at our organization. Moreover, based on the scanning results, the full text= of e-mails and attachments may be made available to Weatherford security a= nd other personnel for review and appropriate action. If you have any conce= rns about this process, please contact us at dataprivacy@weatherford.com. From owner-freebsd-hackers@freebsd.org Sat Jun 17 06:49:22 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A27B4BF2AA0 for ; Sat, 17 Jun 2017 06:49:22 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3694581F20 for ; Sat, 17 Jun 2017 06:49:22 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id u195so25776218wmd.1 for ; Fri, 16 Jun 2017 23:49:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=VkJwrYLHm0TSnfk1rBF5uxAN1Dc8ksvjQXnMx9m/NgY=; b=Mbr8GXWT1Zu0dFPrpFu9Ve/n/7ydlzcm87onPEt9SMfql5u7yFvtqx4MUVmK6KgL03 ayNU6cczetEzMUv/Q78tmcGnFAFYNVUUNzaEJMSJkg2JI88VUtfvrbxHBdzsZmRdVv6i L2GxGJrLPyYfx7o91TcP2vF0I74uOF3baubrw8R/3wF2s5GHyeZV0vbuka9h5lxlbgQG HJUpuqQQMq7Ujk0HzA3nwMNkBUBGCCzsbne6CSAWVU61VL+NaeLceTBfgWnwWo6lBYjn dH2KI5J68Wtn8meGBiwfRKqmzm7zgTk0wOr2CeBuEpipcIWy8eNGQslJzMsjCjns6uLq Oidg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=VkJwrYLHm0TSnfk1rBF5uxAN1Dc8ksvjQXnMx9m/NgY=; b=oNONTYFT0N7RKNS5EQYRwV8tQ0TeCk690LQ+DLipScFjKfegGfOoiAXCNVQETNF682 8S9Asg/LPaIKctB+yt0o0UhkqzpdLg9WIrgbdLFAgUPpY3XkIDIf0PYR5mO0ZozPbfZo cb2GznVpGu4Zv9OYQ2KcA5mmrmSuKhYfiTPBBB744nLNHtQVa9w4s4MpRJwcQRwYIQ1z IifGwAdgDq8VgmET5ROhJDWV4vgKGk1bvh8H8spm3M0iXjurKh0lkRS5sEjQOQCa/x0E JkOfY8Nu9/NrcU6WfSeiPpKeP8vsRRtceD7LM2V9RuKzY6J5F5pk3e08Q0i37NhVruoB WO8Q== X-Gm-Message-State: AKS2vOzFJJVPzkNCIa0z8rV/9qcNEDGy9EQ4bADtK4BGKxn9WkpAwnB2 H/XJc7uuOaYPipB2icFT/v9Zl+AWME6A X-Received: by 10.80.137.136 with SMTP id g8mr9724361edg.125.1497682159225; Fri, 16 Jun 2017 23:49:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.140.162 with HTTP; Fri, 16 Jun 2017 23:49:18 -0700 (PDT) In-Reply-To: <4561529b83ce4270b09aa0e3b12f299f@BLUPR58MB002.032d.mgd.msft.net> References: <4561529b83ce4270b09aa0e3b12f299f@BLUPR58MB002.032d.mgd.msft.net> From: Adam Vande More Date: Sat, 17 Jun 2017 01:49:18 -0500 Message-ID: Subject: Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours To: "Caza, Aaron" Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 06:49:22 -0000 On Sat, Jun 17, 2017 at 12:16 AM, Caza, Aaron wrote: > > Regarding this issue, I've now conducted testing using merely a FreeBSD > 10.3 Stable amd64 GENERIC kernel and using dd to read a large file. The > following is a log, taken hourly, of the degradation which occurred at just > over 9 hours of uptime. As the original is quite large, I've removed some > sections; however, these can be supplied if desired. > Supplied are the initial dmesg and zpool status, logged only on startup, > followed by uptime, uname -a, and zfs-stats -a output, each of which are > logged hourly. > What does this say? # zpool get all From owner-freebsd-hackers@freebsd.org Sat Jun 17 07:11:47 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC6DCBF323A for ; Sat, 17 Jun 2017 07:11:47 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0044.outbound.protection.outlook.com [104.47.36.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 441D88285B for ; Sat, 17 Jun 2017 07:11:46 +0000 (UTC) (envelope-from Aaron.Caza@ca.weatherford.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weatherford.onmicrosoft.com; s=selector1-ca-weatherford-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=WTG3eGR+2M30UVBTRyxMCjM1NAFSLPp352XQrokSLUA=; b=IG/6JCRqFUz8OAzzv0mTmfFeWH9TANYRETBexbeC34xTNYjZ8gHvgOVD3jKdRH6VBx8bvatWPrxwunYpBcJ2pUEc5ObkT5EGGAQ172SU/d21505pBtgj4ea1qiWRm1kSClH5rrz1zYaFR2RwcNPOFzuklIK4mBjhpITcmeO6hf8= Received: from BN6PR03CA0021.namprd03.prod.outlook.com (10.168.230.159) by CY4PR03MB2967.namprd03.prod.outlook.com (10.175.116.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12; Sat, 17 Jun 2017 07:11:43 +0000 Received: from BY2FFO11FD016.protection.gbl (2a01:111:f400:7c0c::133) by BN6PR03CA0021.outlook.office365.com (2603:10b6:404:23::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14 via Frontend Transport; Sat, 17 Jun 2017 07:11:43 +0000 Authentication-Results: spf=pass (sender IP is 23.103.226.36) smtp.mailfrom=ca.weatherford.com; gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=bestguesspass action=none header.from=ca.weatherford.com; Received-SPF: Pass (protection.outlook.com: domain of ca.weatherford.com designates 23.103.226.36 as permitted sender) receiver=protection.outlook.com; client-ip=23.103.226.36; helo=032-smtp-out.weatherford.com; Received: from 032-smtp-out.weatherford.com (23.103.226.36) by BY2FFO11FD016.mail.protection.outlook.com (10.1.14.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Sat, 17 Jun 2017 07:11:42 +0000 Received: from BLUPR58MB002.032d.mgd.msft.net (141.251.110.146) by BLUPR58MB001.032d.mgd.msft.net (141.251.110.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1178.14; Sat, 17 Jun 2017 07:11:41 +0000 Received: from BLUPR58MB002.032d.mgd.msft.net ([141.251.110.146]) by BLUPR58MB002.032d.mgd.msft.net ([141.251.110.146]) with mapi id 15.01.1178.018; Sat, 17 Jun 2017 07:11:41 +0000 From: "Caza, Aaron" To: Adam Vande More CC: "freebsd-hackers@freebsd.org" Subject: RE: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Topic: [EXTERNAL] Re: FreeBSD10 Stable + ZFS + PostgreSQL + SSD performance drop < 24 hours Thread-Index: AdLnJr8RDocWLK5YRfCaDeFG8ME4LAADxhQAAABbDJA= Date: Sat, 17 Jun 2017 07:11:41 +0000 Message-ID: <498c08b2d4d54d5aaa54abc1abfd4046@BLUPR58MB002.032d.mgd.msft.net> References: <4561529b83ce4270b09aa0e3b12f299f@BLUPR58MB002.032d.mgd.msft.net> In-Reply-To: Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.110.132] x-ms-publictraffictype: Email X-MS-TrafficTypeDiagnostic: BLUPR58MB001:|CY4PR03MB2967: X-MS-Office365-Filtering-Correlation-Id: 25576712-5de2-4359-8989-08d4b5501b94 MIME-Version: 1.0 X-OrganizationHeadersPreserved: BLUPR58MB001.032d.mgd.msft.net X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:23.103.226.36; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10009020)(39850400002)(39400400002)(39840400002)(39860400002)(39410400002)(39450400003)(2980300002)(438002)(24454002)(199003)(42174003)(189002)(377454003)(9170700003)(478600001)(72206003)(76176999)(5890100001)(54356999)(53546009)(66066001)(4326008)(24736003)(22756006)(512874002)(2950100002)(229853002)(50986999)(42882006)(1411001)(6916009)(7736002)(189998001)(108616004)(7696004)(39060400002)(2900100001)(6246003)(2906002)(54896002)(114624004)(5660300001)(9686003)(110136004)(55016002)(236005)(38730400002)(3846002)(6306002)(102836003)(53936002)(86146001)(84326002)(790700001)(53946003)(16200700003)(8936002)(33646002)(8676002)(106466001)(81166006)(6116002)(86362001)(356003)(569005); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR03MB2967; H:032-smtp-out.weatherford.com; FPR:; SPF:Pass; MLV:ovrnspm; A:0; MX:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD016; 1:GeSKA2YQesjYbYq975tvtgXCBps80RiGJOE4zC9R4WDZXSv1J2pep0BHxdt7TfwjZ7wNb0jag05eC7MXGua5G+dSTkY4Y79fkTIaGERcQAcb5zwt2SmzheCtDqjn+RN5RP1HGt1IichN0nqfDoLEKnbDtZHb0M7I2d/jq2Ht8VouXiUOBczy7rV5i3ayDvG2z25QN9npREFb2Zof2731ylbBGS1r510GaoBTawTjzVWqkhL7NgEL3RzVU0TRX6XA3eF5LbdlKZoTMmaPEsD+U7BTpqE8U47zf168EZep9cSFHJLabSsNJzraD65pYuKlcKAPaxKe9A+d8RiQK0ljLsUtlchEMMHQ8omOB1fA7bsPgw71ofi1DiLnDJVkszWsAuVsZLwPtRBaAw6ODHNL8Oy4TtFLZp3O5wEKhSbf60K+78EzCrHXrlE3W4tXtK0PuDKN/eRXFbjMsDSKfSt260a8t16ddWRLpFagS3OEnBGfdg9o5Jr0MFcsO4j4g9J6uLjG0E9PAGyOqgW61u9JsinPIy3/6gsKUp8pavoXO04= X-CrossPremisesHeadersPromoted: BY2FFO11FD016.protection.gbl X-CrossPremisesHeadersFiltered: BY2FFO11FD016.protection.gbl X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081)(201702281549075); SRVR:CY4PR03MB2967; X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 3:gQleljXvLqGQTwQO6ZW2si8Wki427vE9XNrDR0g7DS29XlyYy7ftyTbtvS7RmAGC7uGBoA2FSxjwVQxWY+jnB1CZrONBIo8vQvzL8KUPfXi/VnJ3BFzw7XDgHj3zSMqS/ADGnLBXZ61Ezar755lgwalo41jdNTNtEbi8sMFwS91qeuGCYMKgguq9vtMfwP2dQDC9Ios20toX+57B4S2tcc92/Y5shtGDJjBhmUutS0sXDEwi9Fw9VcZurz9e9Q9UMN9KoxM4HLlTumPs3iZFFK7OZJ162H2VHxykc4WtVT18E86hcxTowCtDM47w01EWSUjx8w9PydfK5pe4A8Mbn7uIcocp/PajR7dGwdOAiEsl0WCQu29sjxlV+Si7z62YtDwPW0WsrfmTIYTEEndHHjZATQ+/UsSxuMP6c56OdueXrNB0qGF6SizjeA5jjuUTGwuYMHL47fXkFyCSkilvjiUkL9wE5vtwFRMbOC8MTNZzOnikE9QIeATP1eaGh25ifwVs2mf8e1IKUgQRZMRpyQ== X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 25:L1scTx7Lb0OtVx//DVmMAp4/hoImCaFfCrtDEez1hsK+9mTpzUKlrGGwn2DoTWUAD8C5JFIC93m9jUNn6CZNRnUmLJcHBMBxwHX41kiHJAZi+PdjL8+mjY1KSbh1Fs9mvdKaSUWuN4YK1LITeETTWs/CT81n4Q5/oKINfMbzCScv/gyb99GblcIbdy6+PBSQdVmSpuon2qP9cfvJOsMWNa44KcOWXQ1acsw5jxLOLH9QFdEHAhLKrUlpJV5X4KVfXEYVLysAOUGlCaM2X2+NMits0SFBlfHKa2rlcF+m0IE0GpF4Y0A4Qgm1CjeSQyHGua0F83WBeSYpnCD3nQuJjhM0wyqn5fAts/lVWx89RBN47wj53geScUwp+EFqj3bpM61yKAsXsybVmwH9vwVcEI07gXLbw+5WDDWtnefvpcDLKmwmlBLzr8hsb/5wI2GKzqkKYqj4KJGd1Rj/VITLXGoiJjmcTkO6Zy6MMxUsgbC+faUmFRbsFaeAzH1/olI8vGLC7VTjr4C8IX4jGC7YUSbrAiPH6jjImieGjO1J5ygF6tNkUrliWNI8T/fVOmhzdhfydqywX8eIlq9qWlcTdthOilxOTpZ8lNRdD15MfwqtxTMlQj3ZVVCFKO95+9Wg1b2vRfodKNgpwSwNfGF84Dr9ZMjWOMwt/ktZnh5AQRH6IFn4AjvHyUXtTQm4qIBU5p/qnOotM5/0k/yziqb42+fuxDcCtacCkkTC7DW0BhSLlHYYdAGNI/kbAJaZkNbpZVdVax/HIeamfMT1rvU7qIYcjDLoypAcwaW7h2nkIiuDGsx9jUVB2i7mQ2zi8Q1+q5AZ5OZ19WhvLHVZKWHar6XGPuVQcXUwa7Hl60L3q5jCq5noRz8nuo3nb5GD7APMFb6NfwUs17YwwD5YVHoDZGXqa64HxmKJ2MM+aApklf4= X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 31:XS/nx2FoSyWXBZ6DjAf4tzRoERtu3SCfUr9rqBDHahPMoFmKw+iC1DN0M2X45VK/3NZIHdlvotImiVFrHa5erdIgndgby/zhUV17U7Nazepxc8lQ0Ms595JMGXs4zrNMu/MYbc8jDRdvMwPldO+hhQuxDZDoU/5Zck/Cq53A3PLI/LKp5MTU2GeLsjXz0KChZu9CoUgueuSwncnIV2o1E2eOuQyclROvU1C2fulDOLdnPakE1FmRCz8rAo3wS1Y9nGPRr5QzUF6+5r0n5qAHTzD+xztCCg4U68Sn9gzqmDfpKGzgpab6kwK0YiJDXEzbqemxgXJVtbW3pW+VrhBgg8z7yRobuZk/0SZEMr1BlLtHM6LKTlG45hBaLSyc1b8FGoSyhM2xMDRjfA6kgClHCAqzzHdGa5EbYdoky7riLofNYO3BPMqiTD6ONcumbkccGrxSzckStyFJ4W9wH/iIq77nNnRpjvZWS99F9M6WUzGEebpcCnpxE0nNA7uQU+8Ol/Kcdz3VJnJJl/irrQ/l30r5U8lqGRmL4zFw1Olcj8KINxXscqHUVjgffNK87Rqjenf14tjnzrjL1gjNK6XSBxV3AMzsRkz7Q3ZxNTmyQakn82gjV401gplEQWtkD/Dr2OcYorqdfgEB+gfd08LWKQ== X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 20:4NQH28oGNMQOQUHj8ZoCK0OEJp0ZMbsD5iMIOa7wLf+twxAITXf3oMGF+5q8vh+p+aEup/dqlG0j+8IrY4O3i3GVEYFDhltuldyhiqE0RtNcLcyAO41uj4P0nnEvVH5Ni19rahjRLbZ4TrJQFKw38YZpS4DH7Ao7dommog0H6weBF1HwDJP5XyODp94H0SErBqCl7IJVf/Dmp9bvlYMRC7C6KGNNA91Mri4aIGbLpwnCbBtgQGGbXD+20PYFfwpAI7EP+/TD5dTw1VObLRIKIfZR/3MgQ0baMDp/i2pdP5+LOV8trXw60VC8txvXAVV5OmP6kqyaivQaPNVFJC9A38I7PINeW/rZ+AFkHzV0ZdigRI5yJI183u9T6/FSY6+bOkQPIF8aNa8+nmwl51fU7zREoOwLkPz2xOWcdNwHs5m1/crasIqsTgF0WygOmqV2XbkVT82gv6dZEQ3p240RwJRg6+gOtHOcCNP+K+x/7AVs52hdOYxhPHaSBzh37Ee3 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(84480959824636)(192374486261705)(21748063052155); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93004095)(6055026)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR03MB2967; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR03MB2967; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2967; 4:pG7PBNoMZSJCvOH2/2kHCZ39FoP3BQ8vwAyOCHA6eE?= =?us-ascii?Q?Ce2VUL7itC9CUXuQPhZJk53zLKFkwuPGWY8z5PjujfQG5AapKQ13TfiOd7G7?= =?us-ascii?Q?7+LYxPferXrDzxoZu6TYpmsSRf/FyE6XHiO7VHhPpU/bptn1pUfevMTedrhT?= =?us-ascii?Q?2797M4Y/k15AKTDHOBHWS0Ch29xJ/80yrHmdse/f1dAbTxD53yGKPMWF5xRh?= =?us-ascii?Q?etKA9b6qGyhpuc3gEQEhIYtQuIdITrH+NxaZnI3DINxFdUBDVuaQInqSDbau?= =?us-ascii?Q?xhGpdd4ecXdQZkPvhYMGwbOyjjRyveyJDZJ5ORB9SEvIJq8vltxPqlRxWivh?= =?us-ascii?Q?7vkufcuyBR7Tq9Eojp4xbXAEfqImbNA7zQgqLJ2hyYZOTLsAKOqyN3CXBHMQ?= =?us-ascii?Q?nDQRiARG13olEDc1XoCWv43v19iVXBxWqyWgHwLn3I/1WV599LSOXmmehVJq?= =?us-ascii?Q?AX90FwOuxzDDA7K13KoSLg0olMeHuN2dVTpfKdTfRTITH+25ov+pAUVGVNsT?= =?us-ascii?Q?HNpNeyaY3wUCeQylVYCpVZ8/H5eAMs932X5JdWTDQXjRzxpq/hvDhvHuWmFT?= =?us-ascii?Q?Xxt73P7Jq8+CsHgJ+cEGQewnMStmz1LDs15/VQyWGPMtJ4Ln4a05iXm2vmrM?= =?us-ascii?Q?zgE+QX7PkiKX/nPsAKJK2oCmA1hIVLrjpp21nIz18sa3BlQf4tNBpCGx5Vz4?= =?us-ascii?Q?uTSN39O4HX5g0mioE8u81ztKUNj0RuZnP0n9S3s2O08n8B8hmIPDtMK1kTdv?= =?us-ascii?Q?zFOg+kw6GIpAPyaVjXFzKXimZa+X+qGaNQSHATw12N8XTfFDR0s2cXAd4F6K?= =?us-ascii?Q?tnxDTgK6KOHemHnWA2yZB+u3shjmkrXho2DMK4q6pjcODf5816kXEqSPKHsa?= =?us-ascii?Q?i4qbzDaUmYmvKUTAfZwcgYQM0bQn/G818mj+6xLg4a+IV9WHNKmQV187C7vV?= =?us-ascii?Q?hX3PqQBelBxw5QskmG3ll3TGqeVAG8bVJNisiLRXwbNdM/RB4KSCAsa+a0xK?= =?us-ascii?Q?yxD4xhXSe/6gL1fip74T1p8H3Q0UCOplHD9rOvNFuU7S14oF2gMPLqwAGJwq?= =?us-ascii?Q?ZXEnMvEUpqD19MALgZ5bGbagSQyzMQm4E7t2yj6xfa9rYZEGP9aw5nefDpPS?= =?us-ascii?Q?LKZu+oOMginEICMZ3QfvBVBpShwcmrKsIVkpRsaE4h0UkvJeP9wDn8h4WBAT?= =?us-ascii?Q?TKCWahPPT0ech1XWUAFNw77Vyp0fUnwF5zyLaJDZxkptW7VrO/w8mwo8AKVU?= =?us-ascii?Q?FHh0ZJRoDwOQQBUAcZmgxPJlkApEfMj77BbdydMrjJy7mbAHP2V7MfseB/J+?= =?us-ascii?Q?949PQFHOzF7PP+2OMS1sFmrfuOa9pTVIa0f8MqA3gAevjny/Q0H3Kf+EKPUV?= =?us-ascii?Q?NpCQ=3D=3D?= X-Forefront-PRVS: 034119E4F6 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2967; 23:pJTuybrt04x2B0vtrO22GwguOjEvFhQOU3msi2rqg?= =?us-ascii?Q?U/H7O21rbXbzmfjcA93gvLuaBpaT/5dBYr91LMNqz6BumtPKUKdCjQBJ8Cfi?= =?us-ascii?Q?Z1TGUFRT3X1hU0lQEuh0EreZOxCg+whuH/hrb+zfH77LlpD0ia1qLD+OgQhN?= =?us-ascii?Q?XR98G3WGYtF1r/S79Hu2FjUSVqr/8+f6rRleswGqfBfWt/s3fTw1UaBaHi+C?= =?us-ascii?Q?tRVb/BG7PjI+/woCJQfvSBH1q2waHliRIKm27Xms0u3bkLEusOc+IXnJvhYv?= =?us-ascii?Q?v8RULTUGE3EbnCHIjPEunMFOAl8pzvP8X9Tp9jetmpgz+wKPiPqqcBNtJHxP?= =?us-ascii?Q?Vk3h6E8jq6ivE7ZYXS+vEwOMRVdNe5dqln4si7t3kxWkEdLDZeRckBgY2DUZ?= =?us-ascii?Q?gPYI4PCQ/4lKaTwruESKj0sIoNhcHLgqgNFXrw583XsDTUz3mtPJX26cd0Bw?= =?us-ascii?Q?83R5d9zP1FW3RpzQMVzUvjw12Wc5VcPn/nnD112FR16G4MpUJNnQkmTn/glU?= =?us-ascii?Q?kRsew7feUiGuA/tYVmaPq7KLNnTqeGNQMdcMBVpXq2oljv8SXl/6lHtUKFas?= =?us-ascii?Q?Cpp1pvtjSYFdiu3CvRt9WJioHZMaw157c8OXK1fjfn0A56JGTnFTkNcI5m6g?= =?us-ascii?Q?vW//BWPxj6w6y7MUrCbTLp7sf/u4Aor3QOroKr9PbotHHayFvV02Vdqnl/ZF?= =?us-ascii?Q?qDHwzEOeY+fiIdJ4FiutaeDTEHXZbt/LWIbcSpy46VfsuzZeRYjEOB8c3Lw0?= =?us-ascii?Q?5fP4pWlnkbsjh9YqUKnKkv4vaO8Bvd0O53jpAYukt5wOz3lL/HxZxTBuQARL?= =?us-ascii?Q?CXobI3WOrXCkxZFSEBC0+0RUFQ7cp8ic4y0nvqUCJ/0Fp50fZg5rxwc828jY?= =?us-ascii?Q?LWNU/8Ihu8zRFHajoR/0uuUpEfNon+s5YOAEYSwuUIshVS1BM6Pr1razxvvk?= =?us-ascii?Q?r2GbxiTWWanGkg/PRHLRjazqvTTwzEju/9sto5+xolYbZYPEy5X7fpVSg4lT?= =?us-ascii?Q?leel6LygSY3IlyAeg1jfuWMe1SwDunmCsyIiqid1IhSrHB+CduHEVp1NsrtD?= =?us-ascii?Q?UFde+/bLvBjssc42RLPo6XYtxymjQscoGdy8tsRdEjHn5tVzi1fo5tapZD5l?= =?us-ascii?Q?3iCRnWgzMclR4p3N+WLdrfLVfntzj8hMoj58tr7+9LGZvLrhUs0jRRPsua1n?= =?us-ascii?Q?XEQ5wdw2YDcZUaoiHv477XLHMBplg78OPZrQ9j+kyMWCcFngJSdb1xVefOGd?= =?us-ascii?Q?Pp6GsoVGRuIa/Y1JnwDYngRd5dske2k8XNrVkkZOBdL75ag0OwPcrhba5niS?= =?us-ascii?Q?f/bJfcnqjkRMgqz9qP/4pPUgqlxE8j5oedsm0w9z2sx8sDXZrmy4s1WGEnRE?= =?us-ascii?Q?Ik0bbK81xCX7VysscF+0aLnS99IRDb66D+O8Sa214gtn2Nmy7G4oHIwLtln3?= =?us-ascii?Q?15tjsLJl982pxyGeMJZQG7YybrTLgYLgigtoJh62fjC+5z2q1872bmEpuPpB?= =?us-ascii?Q?Da/DwZ6vBJ+wCaPjwvDI7WsKQAckwfbIcLBhRsdxTaJTMIFsHA2tldg7KWnZ?= =?us-ascii?Q?e6DUUjL6xJd8snvEA=3D=3D?= X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB2967; 6:XvabbjqVnKJgugW7Gfv9RE+Wgi4GIOI7O4D7NSBvkE?= =?us-ascii?Q?SUbIzaAyO921luglYakMlKvic1C0YHJ5mrtgle7s0a4PkrXSNvPupr3/nmdi?= =?us-ascii?Q?Q/xDMPL4LLHf+V3BCs1t+FwSFXWrJXOCX0DMaJ7fM4IzM559Dra1NiOKIzwJ?= =?us-ascii?Q?AfRGexdytCvAPMZbzsSsILb9MzfQ0YHR0VRjTspNHCzJAqn755AwuEh2vAUO?= =?us-ascii?Q?XcQskyoswypykshaIl99Ee2fnJyyvkxYFgU4bpiOlNAWsyVW+tO6vlBW5Av0?= =?us-ascii?Q?IHzQbb8hNEkXmGOHQ+eg+vPffATgzszwMctvke+IFmgdLzJCCQItUgjF8h3j?= =?us-ascii?Q?0Ylge2WEWko4jFlPjPLevGE91l6Ex3PAwL7UPtmgJc0Z0TZMVheRpRG+31TL?= =?us-ascii?Q?Le6VJtPWW05bEnXnM89bBUpNSHyCSjWQ7BkMO3CQ8QdwQdphb9mYqLvkhbTN?= =?us-ascii?Q?4FgLMoOgyx4DXBzz8Si8gB0chdwCPU3Sx0R4WRlkBD/IhvQDusJFTNfxbqwg?= =?us-ascii?Q?Ofg1w5/K9qlETtnGna4HCzZoyEb+5S6F6IW+xsZYvfc6d24SzkXVzvvYBXnF?= =?us-ascii?Q?RqFCRzSJGvnJYbDQtJn1sefOYCwBZqG1bFjOO69SKVqLwihpVzKmdm1I0W+Z?= =?us-ascii?Q?+6XQ4yyS2mSby68+tiDfbmTwwLl9ERhgR2br6QnetKtQ8ORoNBWM9ASE+x08?= =?us-ascii?Q?Czky4DQaz/nrvqmgjd7N8O7K5YEznNcawMNufOleL+j5mtyyDyGJXBImWiLP?= =?us-ascii?Q?8+jjSv9kpFKF6VWPAkTGeQBOjbAlpan+KrzVPTQjchGG3dCKRC4gFVT/pKVU?= =?us-ascii?Q?SMyiZ3xEMaOxx+kgs6C0WaerIddeP9CZ/7EhURkrvKgJaVFucrtxPZ7x8NCh?= =?us-ascii?Q?sbt7w67h6FCLbuFSibQv/I1W1zdA/WqDBTBs+npzdYoRm3AyVrKFbTbdQmCx?= =?us-ascii?Q?wQKoACBesnsnSSyrZa0zGV9h6FYi5VxqnNMQsV/HJpozf5Av2/hkTnYhzseR?= =?us-ascii?Q?epl28Hz+YdzGQjMxkK+bPI?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 5:9H0euAjV6yyurVXHtTZMJexDfwuucRdz6jvw5RAOLj08dQxk1pfBpI3tPeA0lILO4IsVrdz3YsHGFhnC8ooPayjzUunh08zJAfsuyKFZW3B2RcVo01BBctCM7CFSQoBrXalbxr0EQCXRCRz7/QO6OQ4S/MOMhBKvI4R6XI9q2sce81v8LFQ1J++ARr/wNlTR1FBgrcyOaEOFEn4QHl63CZTyWCcDlbwmNLLhsyImmonpBtj+bj2jXvCsfCFaskduPq9K//NrkvPTAvrOAZ+o/612p8xfX6X5MHgaIjtRoOozg1RsA12HjtwJVBsDIMUbZpJrRf56deP94Y5+0KIsXT83mozCWejf8THkCWZrLj2MXK1kgRxNGXjPXkmb7obnnt9m3+eMjowHH1iwTO0I5seCThZTXZxcINV5EVvNoyW563+6NCu4rlpFKsirFjBL+ZEXvP85+Wc4MpiRMYcUDPjLA5jOgdwzAZKOv42DxuBNwL5n1px3Xh1+w/T1Dgtv; 24:E7vopUKMPtjB+ZS/j7NXknUmsbN5XDv+lkGEXrAeLx6Dz4EDT3aUna1BIcO/vzbNgNo5N1bBVYuGB/F9fA3Jch7FTKa1/zjxPgQ6ANl9lgs= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB2967; 7:C8H99Im20IQPOCay3O3BERrzLIInwWlcDHqiGcNqwvMPIJ83SXVaBgL1t6Z8xAyDwOmXEpBm5T2MXWg7MxPE/NGc9BNZIVS8xidE1V8S19mhnVcVQvhgXA7XWlbP4K4myDSSFLkxYUuO/tuKK3m4P0B5RgNxdriavQgwrQTJr+8y1NzXVO3+zIXxGQy2bZvgfdzgga8CA9Ctw0d5A/I8UDLzxXGvDcJbhQnmimreKxSrLAdB6+9nrfZMAN4DupOnVxzVljLZIpA09GGThCVHaUNPdOMIsQZ+AEJmH7xv0nhpcBC/aw+Tl5tBxSLJGXLWN2x3D4yRR297RSgDNel06/xza0fQvNzBoXrAO4YQUGWehiTXmeD2CLaGObQOPpZc+MOE4q3QWDRo4NQu9MM71E3g05fVaJpJiP/sxArsiiBjT7EbjvupIjmF6FnEt1/kxaa8W+1jeBOXWKjScZzciiGjTa3oqS5EWNOe8rOOzdsyEWnKWqnSpkV840IEgb9KnWoTXketwYv2xcLg0RsLNffnlPU5enU3yjAvD7knGo4elfgcnczqd2zd3TeffNXOuBLR+4DdvQMRlFuGZd4EKmycyPmlT0e7c8IR/uPRbOtnMMdIUjuwH2fNvzlewBCspS9ZAScnkS80ialoYfi7D+rWli+aZVZaQQaH1v0QfT0Wcmkt/+fWJ1+QEbwF97rU7O8g12Wx95NwVka9eVA7fisPdwWfpaYzW1e/0/rIgY+T1esVQFgShSNDavhj2Nh1+jaL9LBQ0UEnFgdw3fYjfadtNqt0AM1TxQcnulyE4+E= X-OriginatorOrg: ca.weatherford.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2017 07:11:42.9253 (UTC) X-MS-Exchange-CrossTenant-Id: dd63fb60-07f6-4d96-8d40-ebeca61a524e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=dd63fb60-07f6-4d96-8d40-ebeca61a524e; Ip=[23.103.226.36]; Helo=[032-smtp-out.weatherford.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR03MB2967 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 07:11:47 -0000 DQpGcm9tOiBBZGFtIFZhbmRlIE1vcmUgW21haWx0bzphbXZhbmRlbW9yZUBnbWFpbC5jb21dDQog U2VudDogU2F0dXJkYXksIEp1bmUgMTcsIDIwMTcgMTI6NDkgQU0NClRvOiBDYXphLCBBYXJvbg0K Q2M6IGZyZWVic2QtaGFja2Vyc0BmcmVlYnNkLm9yZw0KU3ViamVjdDogW0VYVEVSTkFMXSBSZTog RnJlZUJTRDEwIFN0YWJsZSArIFpGUyArIFBvc3RncmVTUUwgKyBTU0QgcGVyZm9ybWFuY2UgZHJv cCA8IDI0IGhvdXJzDQoNCk9uIFNhdCwgSnVuIDE3LCAyMDE3IGF0IDEyOjE2IEFNLCBDYXphLCBB YXJvbiA8QWFyb24uQ2F6YUBjYS53ZWF0aGVyZm9yZC5jb208bWFpbHRvOkFhcm9uLkNhemFAY2Eu d2VhdGhlcmZvcmQuY29tPj4gd3JvdGU6DQoNClJlZ2FyZGluZyB0aGlzIGlzc3VlLCBJJ3ZlIG5v dyBjb25kdWN0ZWQgdGVzdGluZyB1c2luZyBtZXJlbHkgYSBGcmVlQlNEIDEwLjMgU3RhYmxlIGFt ZDY0IEdFTkVSSUMga2VybmVsIGFuZCB1c2luZyBkZCB0byByZWFkIGEgbGFyZ2UgZmlsZS4gIFRo ZSBmb2xsb3dpbmcgaXMgYSBsb2csIHRha2VuIGhvdXJseSwgb2YgdGhlIGRlZ3JhZGF0aW9uIHdo aWNoIG9jY3VycmVkIGF0IGp1c3Qgb3ZlciA5IGhvdXJzIG9mIHVwdGltZS4gIEFzIHRoZSBvcmln aW5hbCBpcyBxdWl0ZSBsYXJnZSwgSSd2ZSByZW1vdmVkIHNvbWUgc2VjdGlvbnM7IGhvd2V2ZXIs IHRoZXNlIGNhbiBiZSBzdXBwbGllZCBpZiBkZXNpcmVkLg0KU3VwcGxpZWQgYXJlIHRoZSBpbml0 aWFsIGRtZXNnIGFuZCB6cG9vbCBzdGF0dXMsIGxvZ2dlZCBvbmx5IG9uIHN0YXJ0dXAsIGZvbGxv d2VkIGJ5IHVwdGltZSwgdW5hbWUgLWEsIGFuZCB6ZnMtc3RhdHMgLWEgb3V0cHV0LCBlYWNoIG9m IHdoaWNoIGFyZSBsb2dnZWQgaG91cmx5Lg0KDQpXaGF0IGRvZXMgdGhpcyBzYXk/DQojIHpwb29s IGdldCBhbGwNCg0KVGhlIG91dHB1dCBvZiB6cG9vbCBnZXQgYWxsIGlzIGFzIGZvbGxvd3M6DQoN Ck5BTUUgICAgICAgICAgICBQUk9QRVJUWSAgICAgICAgICAgICAgVkFMVUUgICAgICAgICAgICAg ICAgICBTT1VSQ0UNCnd3YmFzZSAgICAgICAgICB0eXBlICAgICAgICAgICAgICAgICAgZmlsZXN5 c3RlbSAgICAgICAgICAgICAtDQp3d2Jhc2UgICAgICAgICAgY3JlYXRpb24gICAgICAgICAgICAg IFdlZCBKdW4gIDcgMTA6MzcgMjAxNyAgLQ0Kd3diYXNlICAgICAgICAgIHVzZWQgICAgICAgICAg ICAgICAgICA0Mi44RyAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZSAgICAgICAgICBhdmFpbGFi bGUgICAgICAgICAgICAgMTg2RyAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2UgICAgICAgICAg cmVmZXJlbmNlZCAgICAgICAgICAgIDEuOTZHICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlICAg ICAgICAgIGNvbXByZXNzcmF0aW8gICAgICAgICAxLjAweCAgICAgICAgICAgICAgICAgIC0NCnd3 YmFzZSAgICAgICAgICBtb3VudGVkICAgICAgICAgICAgICAgeWVzICAgICAgICAgICAgICAgICAg ICAtDQp3d2Jhc2UgICAgICAgICAgcXVvdGEgICAgICAgICAgICAgICAgIG5vbmUgICAgICAgICAg ICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIHJlc2VydmF0aW9uICAgICAgICAgICBu b25lICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICByZWNvcmRzaXpl ICAgICAgICAgICAgMTI4SyAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UgICAgICAg ICAgbW91bnRwb2ludCAgICAgICAgICAgIC96ZnNyb290ICAgICAgICAgICAgICAgbG9jYWwNCnd3 YmFzZSAgICAgICAgICBzaGFyZW5mcyAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAg ICBkZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgY2hlY2tzdW0gICAgICAgICAgICAgIG9uICAgICAg ICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIGNvbXByZXNzaW9uICAgICAg ICAgICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICBhdGlt ZSAgICAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Ug ICAgICAgICAgZGV2aWNlcyAgICAgICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVm YXVsdA0Kd3diYXNlICAgICAgICAgIGV4ZWMgICAgICAgICAgICAgICAgICBvbiAgICAgICAgICAg ICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICBzZXR1aWQgICAgICAgICAgICAgICAg b24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgcmVhZG9ubHkg ICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAg ICAgIGphaWxlZCAgICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQN Cnd3YmFzZSAgICAgICAgICBzbmFwZGlyICAgICAgICAgICAgICAgaGlkZGVuICAgICAgICAgICAg ICAgICBkZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgYWNsbW9kZSAgICAgICAgICAgICAgIGRpc2Nh cmQgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIGFjbGluaGVyaXQgICAg ICAgICAgICByZXN0cmljdGVkICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICBj YW5tb3VudCAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jh c2UgICAgICAgICAgeGF0dHIgICAgICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAg dGVtcG9yYXJ5DQp3d2Jhc2UgICAgICAgICAgY29waWVzICAgICAgICAgICAgICAgIDEgICAgICAg ICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIHZlcnNpb24gICAgICAgICAg ICAgICA1ICAgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZSAgICAgICAgICB1dGY4b25seSAg ICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2UgICAgICAgICAgbm9y bWFsaXphdGlvbiAgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlICAgICAg ICAgIGNhc2VzZW5zaXRpdml0eSAgICAgICBzZW5zaXRpdmUgICAgICAgICAgICAgIC0NCnd3YmFz ZSAgICAgICAgICB2c2NhbiAgICAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICBk ZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgbmJtYW5kICAgICAgICAgICAgICAgIG9mZiAgICAgICAg ICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIHNoYXJlc21iICAgICAgICAgICAg ICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICByZWZxdW90 YSAgICAgICAgICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UgICAg ICAgICAgcmVmcmVzZXJ2YXRpb24gICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAgZGVmYXVs dA0Kd3diYXNlICAgICAgICAgIHByaW1hcnljYWNoZSAgICAgICAgICBhbGwgICAgICAgICAgICAg ICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICBzZWNvbmRhcnljYWNoZSAgICAgICAgYWxs ICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgdXNlZGJ5c25hcHNo b3RzICAgICAgIDAgICAgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlICAgICAgICAgIHVzZWRi eWRhdGFzZXQgICAgICAgICAxLjk2RyAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZSAgICAgICAg ICB1c2VkYnljaGlsZHJlbiAgICAgICAgNDAuOEcgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Ug ICAgICAgICAgdXNlZGJ5cmVmcmVzZXJ2YXRpb24gIDAgICAgICAgICAgICAgICAgICAgICAgLQ0K d3diYXNlICAgICAgICAgIGxvZ2JpYXMgICAgICAgICAgICAgICBsYXRlbmN5ICAgICAgICAgICAg ICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICBkZWR1cCAgICAgICAgICAgICAgICAgb2ZmICAg ICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UgICAgICAgICAgbWxzbGFiZWwgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlICAgICAgICAgIHN5bmMgICAg ICAgICAgICAgICAgICBzdGFuZGFyZCAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAg ICAgICByZWZjb21wcmVzc3JhdGlvICAgICAgMS4wMHggICAgICAgICAgICAgICAgICAtDQp3d2Jh c2UgICAgICAgICAgd3JpdHRlbiAgICAgICAgICAgICAgIDEuOTZHICAgICAgICAgICAgICAgICAg LQ0Kd3diYXNlICAgICAgICAgIGxvZ2ljYWx1c2VkICAgICAgICAgICA0Mi40RyAgICAgICAgICAg ICAgICAgIC0NCnd3YmFzZSAgICAgICAgICBsb2dpY2FscmVmZXJlbmNlZCAgICAgMS44OEcgICAg ICAgICAgICAgICAgICAtDQp3d2Jhc2UgICAgICAgICAgdm9sbW9kZSAgICAgICAgICAgICAgIGRl ZmF1bHQgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIGZpbGVzeXN0ZW1f bGltaXQgICAgICBub25lICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAg ICBzbmFwc2hvdF9saW1pdCAgICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3 d2Jhc2UgICAgICAgICAgZmlsZXN5c3RlbV9jb3VudCAgICAgIG5vbmUgICAgICAgICAgICAgICAg ICAgZGVmYXVsdA0Kd3diYXNlICAgICAgICAgIHNuYXBzaG90X2NvdW50ICAgICAgICBub25lICAg ICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZSAgICAgICAgICByZWR1bmRhbnRfbWV0YWRh dGEgICAgYWxsICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAgICAgdHlw ZSAgICAgICAgICAgICAgICAgIGZpbGVzeXN0ZW0gICAgICAgICAgICAgLQ0Kd3diYXNlL3RtcCAg ICAgIGNyZWF0aW9uICAgICAgICAgICAgICBXZWQgSnVuICA3IDEwOjM3IDIwMTcgIC0NCnd3YmFz ZS90bXAgICAgICB1c2VkICAgICAgICAgICAgICAgICAgMTI4SyAgICAgICAgICAgICAgICAgICAt DQp3d2Jhc2UvdG1wICAgICAgYXZhaWxhYmxlICAgICAgICAgICAgIDUxMk0gICAgICAgICAgICAg ICAgICAgLQ0Kd3diYXNlL3RtcCAgICAgIHJlZmVyZW5jZWQgICAgICAgICAgICAxMjhLICAgICAg ICAgICAgICAgICAgIC0NCnd3YmFzZS90bXAgICAgICBjb21wcmVzc3JhdGlvICAgICAgICAgMS4w MHggICAgICAgICAgICAgICAgICAtDQp3d2Jhc2UvdG1wICAgICAgbW91bnRlZCAgICAgICAgICAg ICAgIHllcyAgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3RtcCAgICAgIHF1b3RhICAgICAg ICAgICAgICAgICA1MTJNICAgICAgICAgICAgICAgICAgIGxvY2FsDQp3d2Jhc2UvdG1wICAgICAg cmVzZXJ2YXRpb24gICAgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3di YXNlL3RtcCAgICAgIHJlY29yZHNpemUgICAgICAgICAgICAxMjhLICAgICAgICAgICAgICAgICAg IGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBtb3VudHBvaW50ICAgICAgICAgICAgL3pmc3Jvb3Qv dG1wICAgICAgICAgICBsb2NhbA0Kd3diYXNlL3RtcCAgICAgIHNoYXJlbmZzICAgICAgICAgICAg ICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBjaGVja3N1 bSAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1w ICAgICAgY29tcHJlc3Npb24gICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVs dA0Kd3diYXNlL3RtcCAgICAgIGF0aW1lICAgICAgICAgICAgICAgICBvbiAgICAgICAgICAgICAg ICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBkZXZpY2VzICAgICAgICAgICAgICAgb24g ICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAgICAgZXhlYyAgICAgICAg ICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAg IHNldHVpZCAgICAgICAgICAgICAgICBvbiAgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3 YmFzZS90bXAgICAgICByZWFkb25seSAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAg ICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAgICAgamFpbGVkICAgICAgICAgICAgICAgIG9mZiAgICAg ICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAgIHNuYXBkaXIgICAgICAgICAg ICAgICBoaWRkZW4gICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBhY2xt b2RlICAgICAgICAgICAgICAgZGlzY2FyZCAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uv dG1wICAgICAgYWNsaW5oZXJpdCAgICAgICAgICAgIHJlc3RyaWN0ZWQgICAgICAgICAgICAgZGVm YXVsdA0Kd3diYXNlL3RtcCAgICAgIGNhbm1vdW50ICAgICAgICAgICAgICBvbiAgICAgICAgICAg ICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICB4YXR0ciAgICAgICAgICAgICAgICAg b2ZmICAgICAgICAgICAgICAgICAgICB0ZW1wb3JhcnkNCnd3YmFzZS90bXAgICAgICBjb3BpZXMg ICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1w ICAgICAgdmVyc2lvbiAgICAgICAgICAgICAgIDUgICAgICAgICAgICAgICAgICAgICAgLQ0Kd3di YXNlL3RtcCAgICAgIHV0Zjhvbmx5ICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAgICAg IC0NCnd3YmFzZS90bXAgICAgICBub3JtYWxpemF0aW9uICAgICAgICAgbm9uZSAgICAgICAgICAg ICAgICAgICAtDQp3d2Jhc2UvdG1wICAgICAgY2FzZXNlbnNpdGl2aXR5ICAgICAgIHNlbnNpdGl2 ZSAgICAgICAgICAgICAgLQ0Kd3diYXNlL3RtcCAgICAgIHZzY2FuICAgICAgICAgICAgICAgICBv ZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBuYm1hbmQgICAg ICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAg ICAgc2hhcmVzbWIgICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0K d3diYXNlL3RtcCAgICAgIHJlZnF1b3RhICAgICAgICAgICAgICBub25lICAgICAgICAgICAgICAg ICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICByZWZyZXNlcnZhdGlvbiAgICAgICAgbm9uZSAg ICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAgICAgcHJpbWFyeWNhY2hlICAg ICAgICAgIGFsbCAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAgIHNl Y29uZGFyeWNhY2hlICAgICAgICBhbGwgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFz ZS90bXAgICAgICB1c2VkYnlzbmFwc2hvdHMgICAgICAgMCAgICAgICAgICAgICAgICAgICAgICAt DQp3d2Jhc2UvdG1wICAgICAgdXNlZGJ5ZGF0YXNldCAgICAgICAgIDEyOEsgICAgICAgICAgICAg ICAgICAgLQ0Kd3diYXNlL3RtcCAgICAgIHVzZWRieWNoaWxkcmVuICAgICAgICAwICAgICAgICAg ICAgICAgICAgICAgIC0NCnd3YmFzZS90bXAgICAgICB1c2VkYnlyZWZyZXNlcnZhdGlvbiAgMCAg ICAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2UvdG1wICAgICAgbG9nYmlhcyAgICAgICAgICAg ICAgIGxhdGVuY3kgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAgIGRlZHVw ICAgICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90 bXAgICAgICBtbHNsYWJlbCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtDQp3 d2Jhc2UvdG1wICAgICAgc3luYyAgICAgICAgICAgICAgICAgIHN0YW5kYXJkICAgICAgICAgICAg ICAgZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAgIHJlZmNvbXByZXNzcmF0aW8gICAgICAxLjAweCAg ICAgICAgICAgICAgICAgIC0NCnd3YmFzZS90bXAgICAgICB3cml0dGVuICAgICAgICAgICAgICAg MTI4SyAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2UvdG1wICAgICAgbG9naWNhbHVzZWQgICAg ICAgICAgIDU5SyAgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3RtcCAgICAgIGxvZ2ljYWxy ZWZlcmVuY2VkICAgICA1OUsgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS90bXAgICAgICB2 b2xtb2RlICAgICAgICAgICAgICAgZGVmYXVsdCAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jh c2UvdG1wICAgICAgZmlsZXN5c3RlbV9saW1pdCAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAg ZGVmYXVsdA0Kd3diYXNlL3RtcCAgICAgIHNuYXBzaG90X2xpbWl0ICAgICAgICBub25lICAgICAg ICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS90bXAgICAgICBmaWxlc3lzdGVtX2NvdW50ICAg ICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2UvdG1wICAgICAgc25hcHNo b3RfY291bnQgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3Rt cCAgICAgIHJlZHVuZGFudF9tZXRhZGF0YSAgICBhbGwgICAgICAgICAgICAgICAgICAgIGRlZmF1 bHQNCnd3YmFzZS93dyAgICAgICB0eXBlICAgICAgICAgICAgICAgICAgZmlsZXN5c3RlbSAgICAg ICAgICAgICAtDQp3d2Jhc2Uvd3cgICAgICAgY3JlYXRpb24gICAgICAgICAgICAgIFdlZCBKdW4g IDcgMTA6MzcgMjAxNyAgLQ0Kd3diYXNlL3d3ICAgICAgIHVzZWQgICAgICAgICAgICAgICAgICA0 MC44RyAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93dyAgICAgICBhdmFpbGFibGUgICAgICAg ICAgICAgMTg2RyAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cgICAgICAgcmVmZXJlbmNl ZCAgICAgICAgICAgIDQwLjhHICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3ICAgICAgIGNv bXByZXNzcmF0aW8gICAgICAgICAxLjAweCAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93dyAg ICAgICBtb3VudGVkICAgICAgICAgICAgICAgeWVzICAgICAgICAgICAgICAgICAgICAtDQp3d2Jh c2Uvd3cgICAgICAgcXVvdGEgICAgICAgICAgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAg ZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIHJlc2VydmF0aW9uICAgICAgICAgICBub25lICAgICAg ICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICByZWNvcmRzaXplICAgICAgICAg ICAgOEsgICAgICAgICAgICAgICAgICAgICBsb2NhbA0Kd3diYXNlL3d3ICAgICAgIG1vdW50cG9p bnQgICAgICAgICAgICAvemZzcm9vdC93dyAgICAgICAgICAgIGxvY2FsDQp3d2Jhc2Uvd3cgICAg ICAgc2hhcmVuZnMgICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0K d3diYXNlL3d3ICAgICAgIGNoZWNrc3VtICAgICAgICAgICAgICBvbiAgICAgICAgICAgICAgICAg ICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICBjb21wcmVzc2lvbiAgICAgICAgICAgb2ZmICAg ICAgICAgICAgICAgICAgICBsb2NhbA0Kd3diYXNlL3d3ICAgICAgIGF0aW1lICAgICAgICAgICAg ICAgICBvbiAgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICBkZXZp Y2VzICAgICAgICAgICAgICAgb24gICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uv d3cgICAgICAgZXhlYyAgICAgICAgICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVm YXVsdA0Kd3diYXNlL3d3ICAgICAgIHNldHVpZCAgICAgICAgICAgICAgICBvbiAgICAgICAgICAg ICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICByZWFkb25seSAgICAgICAgICAgICAg b2ZmICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3cgICAgICAgamFpbGVkICAg ICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAg ICAgIHNuYXBkaXIgICAgICAgICAgICAgICBoaWRkZW4gICAgICAgICAgICAgICAgIGRlZmF1bHQN Cnd3YmFzZS93dyAgICAgICBhY2xtb2RlICAgICAgICAgICAgICAgZGlzY2FyZCAgICAgICAgICAg ICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3cgICAgICAgYWNsaW5oZXJpdCAgICAgICAgICAgIHJlc3Ry aWN0ZWQgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIGNhbm1vdW50ICAgICAg ICAgICAgICBvbiAgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICB4 YXR0ciAgICAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICB0ZW1wb3JhcnkNCnd3 YmFzZS93dyAgICAgICBjb3BpZXMgICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAgICAgICAg ICBkZWZhdWx0DQp3d2Jhc2Uvd3cgICAgICAgdmVyc2lvbiAgICAgICAgICAgICAgIDUgICAgICAg ICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3ICAgICAgIHV0Zjhvbmx5ICAgICAgICAgICAgICBv ZmYgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93dyAgICAgICBub3JtYWxpemF0aW9uICAg ICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cgICAgICAgY2FzZXNlbnNp dGl2aXR5ICAgICAgIHNlbnNpdGl2ZSAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3ICAgICAgIHZz Y2FuICAgICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFz ZS93dyAgICAgICBuYm1hbmQgICAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICBk ZWZhdWx0DQp3d2Jhc2Uvd3cgICAgICAgc2hhcmVzbWIgICAgICAgICAgICAgIG9mZiAgICAgICAg ICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIHJlZnF1b3RhICAgICAgICAgICAg ICBub25lICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICByZWZyZXNl cnZhdGlvbiAgICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3cg ICAgICAgcHJpbWFyeWNhY2hlICAgICAgICAgIGFsbCAgICAgICAgICAgICAgICAgICAgZGVmYXVs dA0Kd3diYXNlL3d3ICAgICAgIHNlY29uZGFyeWNhY2hlICAgICAgICBhbGwgICAgICAgICAgICAg ICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICB1c2VkYnlzbmFwc2hvdHMgICAgICAgMCAg ICAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cgICAgICAgdXNlZGJ5ZGF0YXNldCAgICAg ICAgIDQwLjhHICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3ICAgICAgIHVzZWRieWNoaWxk cmVuICAgICAgICAwICAgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93dyAgICAgICB1c2Vk YnlyZWZyZXNlcnZhdGlvbiAgMCAgICAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cgICAg ICAgbG9nYmlhcyAgICAgICAgICAgICAgIGxhdGVuY3kgICAgICAgICAgICAgICAgZGVmYXVsdA0K d3diYXNlL3d3ICAgICAgIGRlZHVwICAgICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAg ICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAgICBtbHNsYWJlbCAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cgICAgICAgc3luYyAgICAgICAgICAgICAgICAg IHN0YW5kYXJkICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIHJlZmNvbXBy ZXNzcmF0aW8gICAgICAxLjAweCAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93dyAgICAgICB3 cml0dGVuICAgICAgICAgICAgICAgNDAuOEcgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3cg ICAgICAgbG9naWNhbHVzZWQgICAgICAgICAgIDQwLjVHICAgICAgICAgICAgICAgICAgLQ0Kd3di YXNlL3d3ICAgICAgIGxvZ2ljYWxyZWZlcmVuY2VkICAgICA0MC41RyAgICAgICAgICAgICAgICAg IC0NCnd3YmFzZS93dyAgICAgICB2b2xtb2RlICAgICAgICAgICAgICAgZGVmYXVsdCAgICAgICAg ICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3cgICAgICAgZmlsZXN5c3RlbV9saW1pdCAgICAgIG5v bmUgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIHNuYXBzaG90X2xp bWl0ICAgICAgICBub25lICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93dyAgICAg ICBmaWxlc3lzdGVtX2NvdW50ICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3 d2Jhc2Uvd3cgICAgICAgc25hcHNob3RfY291bnQgICAgICAgIG5vbmUgICAgICAgICAgICAgICAg ICAgZGVmYXVsdA0Kd3diYXNlL3d3ICAgICAgIHJlZHVuZGFudF9tZXRhZGF0YSAgICBhbGwgICAg ICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICB0eXBlICAgICAgICAgICAg ICAgICAgZmlsZXN5c3RlbSAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3dkZWJ1ZyAgY3JlYXRpb24g ICAgICAgICAgICAgIFdlZCBKdW4gIDcgMTA6MzcgMjAxNyAgLQ0Kd3diYXNlL3d3ZGVidWcgIHVz ZWQgICAgICAgICAgICAgICAgICAxMTZLICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93d2Rl YnVnICBhdmFpbGFibGUgICAgICAgICAgICAgNTEyTSAgICAgICAgICAgICAgICAgICAtDQp3d2Jh c2Uvd3dkZWJ1ZyAgcmVmZXJlbmNlZCAgICAgICAgICAgIDExNksgICAgICAgICAgICAgICAgICAg LQ0Kd3diYXNlL3d3ZGVidWcgIGNvbXByZXNzcmF0aW8gICAgICAgICAxLjAweCAgICAgICAgICAg ICAgICAgIC0NCnd3YmFzZS93d2RlYnVnICBtb3VudGVkICAgICAgICAgICAgICAgeWVzICAgICAg ICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3dkZWJ1ZyAgcXVvdGEgICAgICAgICAgICAgICAgIDUx Mk0gICAgICAgICAgICAgICAgICAgbG9jYWwNCnd3YmFzZS93d2RlYnVnICByZXNlcnZhdGlvbiAg ICAgICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAg cmVjb3Jkc2l6ZSAgICAgICAgICAgIDEyOEsgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3di YXNlL3d3ZGVidWcgIG1vdW50cG9pbnQgICAgICAgICAgICAvemZzcm9vdC93dy9kZWJ1ZyAgICAg IGxvY2FsDQp3d2Jhc2Uvd3dkZWJ1ZyAgc2hhcmVuZnMgICAgICAgICAgICAgIG9mZiAgICAgICAg ICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIGNoZWNrc3VtICAgICAgICAgICAg ICBvbiAgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBjb21wcmVz c2lvbiAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dk ZWJ1ZyAgYXRpbWUgICAgICAgICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVmYXVs dA0Kd3diYXNlL3d3ZGVidWcgIGRldmljZXMgICAgICAgICAgICAgICBvbiAgICAgICAgICAgICAg ICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBleGVjICAgICAgICAgICAgICAgICAgb24g ICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAgc2V0dWlkICAgICAg ICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcg IHJlYWRvbmx5ICAgICAgICAgICAgICBvZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3 YmFzZS93d2RlYnVnICBqYWlsZWQgICAgICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAg ICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAgc25hcGRpciAgICAgICAgICAgICAgIGhpZGRlbiAg ICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIGFjbG1vZGUgICAgICAgICAg ICAgICBkaXNjYXJkICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBhY2xp bmhlcml0ICAgICAgICAgICAgcmVzdHJpY3RlZCAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uv d3dkZWJ1ZyAgY2FubW91bnQgICAgICAgICAgICAgIG9uICAgICAgICAgICAgICAgICAgICAgZGVm YXVsdA0Kd3diYXNlL3d3ZGVidWcgIHhhdHRyICAgICAgICAgICAgICAgICBvZmYgICAgICAgICAg ICAgICAgICAgIHRlbXBvcmFyeQ0Kd3diYXNlL3d3ZGVidWcgIGNvcGllcyAgICAgICAgICAgICAg ICAxICAgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICB2ZXJzaW9u ICAgICAgICAgICAgICAgNSAgICAgICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3dkZWJ1ZyAg dXRmOG9ubHkgICAgICAgICAgICAgIG9mZiAgICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3 ZGVidWcgIG5vcm1hbGl6YXRpb24gICAgICAgICBub25lICAgICAgICAgICAgICAgICAgIC0NCnd3 YmFzZS93d2RlYnVnICBjYXNlc2Vuc2l0aXZpdHkgICAgICAgc2Vuc2l0aXZlICAgICAgICAgICAg ICAtDQp3d2Jhc2Uvd3dkZWJ1ZyAgdnNjYW4gICAgICAgICAgICAgICAgIG9mZiAgICAgICAgICAg ICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIG5ibWFuZCAgICAgICAgICAgICAgICBv ZmYgICAgICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBzaGFyZXNtYiAg ICAgICAgICAgICAgb2ZmICAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1 ZyAgcmVmcXVvdGEgICAgICAgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0K d3diYXNlL3d3ZGVidWcgIHJlZnJlc2VydmF0aW9uICAgICAgICBub25lICAgICAgICAgICAgICAg ICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBwcmltYXJ5Y2FjaGUgICAgICAgICAgYWxsICAg ICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAgc2Vjb25kYXJ5Y2FjaGUg ICAgICAgIGFsbCAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIHVz ZWRieXNuYXBzaG90cyAgICAgICAwICAgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93d2Rl YnVnICB1c2VkYnlkYXRhc2V0ICAgICAgICAgMTE2SyAgICAgICAgICAgICAgICAgICAtDQp3d2Jh c2Uvd3dkZWJ1ZyAgdXNlZGJ5Y2hpbGRyZW4gICAgICAgIDAgICAgICAgICAgICAgICAgICAgICAg LQ0Kd3diYXNlL3d3ZGVidWcgIHVzZWRieXJlZnJlc2VydmF0aW9uICAwICAgICAgICAgICAgICAg ICAgICAgIC0NCnd3YmFzZS93d2RlYnVnICBsb2diaWFzICAgICAgICAgICAgICAgbGF0ZW5jeSAg ICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAgZGVkdXAgICAgICAgICAgICAg ICAgIG9mZiAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIG1sc2xh YmVsICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0NCnd3YmFzZS93d2RlYnVn ICBzeW5jICAgICAgICAgICAgICAgICAgc3RhbmRhcmQgICAgICAgICAgICAgICBkZWZhdWx0DQp3 d2Jhc2Uvd3dkZWJ1ZyAgcmVmY29tcHJlc3NyYXRpbyAgICAgIDEuMDB4ICAgICAgICAgICAgICAg ICAgLQ0Kd3diYXNlL3d3ZGVidWcgIHdyaXR0ZW4gICAgICAgICAgICAgICAxMTZLICAgICAgICAg ICAgICAgICAgIC0NCnd3YmFzZS93d2RlYnVnICBsb2dpY2FsdXNlZCAgICAgICAgICAgNDUuNUsg ICAgICAgICAgICAgICAgICAtDQp3d2Jhc2Uvd3dkZWJ1ZyAgbG9naWNhbHJlZmVyZW5jZWQgICAg IDQ1LjVLICAgICAgICAgICAgICAgICAgLQ0Kd3diYXNlL3d3ZGVidWcgIHZvbG1vZGUgICAgICAg ICAgICAgICBkZWZhdWx0ICAgICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBm aWxlc3lzdGVtX2xpbWl0ICAgICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jh c2Uvd3dkZWJ1ZyAgc25hcHNob3RfbGltaXQgICAgICAgIG5vbmUgICAgICAgICAgICAgICAgICAg ZGVmYXVsdA0Kd3diYXNlL3d3ZGVidWcgIGZpbGVzeXN0ZW1fY291bnQgICAgICBub25lICAgICAg ICAgICAgICAgICAgIGRlZmF1bHQNCnd3YmFzZS93d2RlYnVnICBzbmFwc2hvdF9jb3VudCAgICAg ICAgbm9uZSAgICAgICAgICAgICAgICAgICBkZWZhdWx0DQp3d2Jhc2Uvd3dkZWJ1ZyAgcmVkdW5k YW50X21ldGFkYXRhICAgIGFsbCAgICAgICAgICAgICAgICAgICAgZGVmYXVsdA0KDQpOb3RlIHRo YXQgdGhlIOKAmGRk4oCZIHVzZWQgaW4gdGhlIHByZXZpb3VzbHkgcG9zdGVkIGxvZ2ZpbGUgaXMg b24gd3diYXNlL3d3IGhlbmNlIHRoZSByZWNvcmRzaXplIGlzIDhLIHJhdGhlciB0aGFuIHRoZSAx MjhLIGFzIGlzIHVzZWQgZWxzZXdoZXJlICg4SyBzZWxlY3Rpb24gZHVlIHRvIFBvc3RncmVTUUwg dXNhZ2UpLg0KDQoocHMuICBQbGVhc2Ugbm90ZSB0aGF0IHRoZSBhYm92ZSBpbiBubyB3YXkgcmVm bGVjdHMgdGhhdCBJIGFtIGluIGFueSB3YXkgY29uY2VybmVkIGFib3V0IHRoaXMgaXNzdWUgbm9y IHRoYXQgaXQgaXMga2VlcGluZyBtZSBhd2FrZSBhdCBuaWdodDogMToxMEFNIGhlcmUgaW4gcmFp bnkgQ2FsZ2FyeSwgQWxiZXJ0YSBDYW5hZGEpDQpUaGlzIG1lc3NhZ2UgbWF5IGNvbnRhaW4gY29u ZmlkZW50aWFsIGFuZCBwcml2aWxlZ2VkIGluZm9ybWF0aW9uLiBJZiBpdCBoYXMgYmVlbiBzZW50 IHRvIHlvdSBpbiBlcnJvciwgcGxlYXNlIHJlcGx5IHRvIGFkdmlzZSB0aGUgc2VuZGVyIG9mIHRo ZSBlcnJvciBhbmQgdGhlbiBpbW1lZGlhdGVseSBkZWxldGUgaXQuIElmIHlvdSBhcmUgbm90IHRo ZSBpbnRlbmRlZCByZWNpcGllbnQsIGRvIG5vdCByZWFkLCBjb3B5LCBkaXNjbG9zZSBvciBvdGhl cndpc2UgdXNlIHRoaXMgbWVzc2FnZS4gVGhlIHNlbmRlciBkaXNjbGFpbXMgYW55IGxpYWJpbGl0 eSBmb3Igc3VjaCB1bmF1dGhvcml6ZWQgdXNlLiBQTEVBU0UgTk9URSB0aGF0IGFsbCBpbmNvbWlu ZyBlLW1haWxzIHNlbnQgdG8gV2VhdGhlcmZvcmQgZS1tYWlsIGFjY291bnRzIHdpbGwgYmUgYXJj aGl2ZWQgYW5kIG1heSBiZSBzY2FubmVkIGJ5IHVzIGFuZC9vciBieSBleHRlcm5hbCBzZXJ2aWNl IHByb3ZpZGVycyB0byBkZXRlY3QgYW5kIHByZXZlbnQgdGhyZWF0cyB0byBvdXIgc3lzdGVtcywg aW52ZXN0aWdhdGUgaWxsZWdhbCBvciBpbmFwcHJvcHJpYXRlIGJlaGF2aW9yLCBhbmQvb3IgZWxp bWluYXRlIHVuc29saWNpdGVkIHByb21vdGlvbmFsIGUtbWFpbHMgKHNwYW0pLiBUaGlzIHByb2Nl c3MgY291bGQgcmVzdWx0IGluIGRlbGV0aW9uIG9mIGEgbGVnaXRpbWF0ZSBlLW1haWwgYmVmb3Jl IGl0IGlzIHJlYWQgYnkgaXRzIGludGVuZGVkIHJlY2lwaWVudCBhdCBvdXIgb3JnYW5pemF0aW9u LiBNb3Jlb3ZlciwgYmFzZWQgb24gdGhlIHNjYW5uaW5nIHJlc3VsdHMsIHRoZSBmdWxsIHRleHQg b2YgZS1tYWlscyBhbmQgYXR0YWNobWVudHMgbWF5IGJlIG1hZGUgYXZhaWxhYmxlIHRvIFdlYXRo ZXJmb3JkIHNlY3VyaXR5IGFuZCBvdGhlciBwZXJzb25uZWwgZm9yIHJldmlldyBhbmQgYXBwcm9w cmlhdGUgYWN0aW9uLiBJZiB5b3UgaGF2ZSBhbnkgY29uY2VybnMgYWJvdXQgdGhpcyBwcm9jZXNz LCBwbGVhc2UgY29udGFjdCB1cyBhdCBkYXRhcHJpdmFjeUB3ZWF0aGVyZm9yZC5jb20uDQo= From owner-freebsd-hackers@freebsd.org Sat Jun 17 10:24:15 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50DC8BF776F; Sat, 17 Jun 2017 10:24:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDCC764346; Sat, 17 Jun 2017 10:24:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5HAO7ZE096948 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 17 Jun 2017 13:24:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5HAO7ZE096948 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5HAO79w096947; Sat, 17 Jun 2017 13:24:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 17 Jun 2017 13:24:07 +0300 From: Konstantin Belousov To: Mark Millard Cc: FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML Subject: Re: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? Message-ID: <20170617102407.GD2088@kib.kiev.ua> References: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> <20170617024850.GB2088@kib.kiev.ua> <73F88E18-37A1-47C6-8783-F51F131A9671@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <73F88E18-37A1-47C6-8783-F51F131A9671@dsl-only.net> User-Agent: Mutt/1.8.2 (2017-04-18) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 10:24:15 -0000 On Fri, Jun 16, 2017 at 08:54:10PM -0700, Mark Millard wrote: > On 2017-Jun-16, at 7:48 PM, Konstantin Belousov wrote: > > > On Fri, Jun 16, 2017 at 05:01:43PM -0700, Mark Millard wrote: > >> . . . > > > > UFS uses 32bit inodes, changing to 64bit is both pointless currently, and > > causes on-disk layout incompatibilities. > > > > As a consequence, use of ino_t (64bit) or uint32_t for inode numbers are > > almost always interchangeable, unless used for specifying on-disk layout. > > UFS correctly uses (and was changed to use) uint32_t for inode numbers > > in the disk-layout definitions. Other places, which calculate inode > > numbers from inode block numbers, or do some other calculations with > > inodes, are fine with either width. > > > > That is, I believe that all instances which I looked at during the > > ino64 preparation are fine. > > Thanks for letting me know --and good to know. > > I've added a note to the bugzilla report of the failed > linking of boot1.elf for powerpc and powerpc64 that > you have indicated that if the __udivdi3 is supplied to > allow the linking to complete for builds based on clang > then the result should operate okay for the mix of types. > (The report is bugzilla 220024 .) I never said that. From owner-freebsd-hackers@freebsd.org Sat Jun 17 10:53:01 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6428BF7F14 for ; Sat, 17 Jun 2017 10:53:01 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-16.reflexion.net [208.70.210.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9689F651CF for ; Sat, 17 Jun 2017 10:53:01 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 24734 invoked from network); 17 Jun 2017 10:57:02 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 17 Jun 2017 10:57:02 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.1) with SMTP; Sat, 17 Jun 2017 06:53:00 -0400 (EDT) Received: (qmail 30304 invoked from network); 17 Jun 2017 10:52:59 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Jun 2017 10:52:59 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 247CAEC8714; Sat, 17 Jun 2017 03:52:59 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: INO64 in head: Does sys/boot/common/ufsread.c need its "typedef uint32_t ufs_ino_t;" replaced? From: Mark Millard In-Reply-To: <20170617102407.GD2088@kib.kiev.ua> Date: Sat, 17 Jun 2017 03:52:58 -0700 Cc: FreeBSD Current , freebsd-hackers@freebsd.org, FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <063A0C56-E9D5-4C84-AD15-B36F267F00BA@dsl-only.net> References: <3AF2C2DB-1A61-4EC3-BCB7-B05D99273561@dsl-only.net> <20170617024850.GB2088@kib.kiev.ua> <73F88E18-37A1-47C6-8783-F51F131A9671@dsl-only.net> <20170617102407.GD2088@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 10:53:02 -0000 On 2017-Jun-17, at 3:24 AM, Konstantin Belousov = wrote: > On Fri, Jun 16, 2017 at 08:54:10PM -0700, Mark Millard wrote: >> On 2017-Jun-16, at 7:48 PM, Konstantin Belousov wrote: >>=20 >>> On Fri, Jun 16, 2017 at 05:01:43PM -0700, Mark Millard wrote: >>>> . . . >>>=20 >>> UFS uses 32bit inodes, changing to 64bit is both pointless = currently, and >>> causes on-disk layout incompatibilities. >>>=20 >>> As a consequence, use of ino_t (64bit) or uint32_t for inode numbers = are >>> almost always interchangeable, unless used for specifying on-disk = layout. >>> UFS correctly uses (and was changed to use) uint32_t for inode = numbers >>> in the disk-layout definitions. Other places, which calculate inode >>> numbers from inode block numbers, or do some other calculations with >>> inodes, are fine with either width. >>>=20 >>> That is, I believe that all instances which I looked at during the >>> ino64 preparation are fine. >>=20 >> Thanks for letting me know --and good to know. >>=20 >> I've added a note to the bugzilla report of the failed >> linking of boot1.elf for powerpc and powerpc64 that >> you have indicated that if the __udivdi3 is supplied to >> allow the linking to complete for builds based on clang >> then the result should operate okay for the mix of types. >> (The report is bugzilla 220024 .) > I never said that. Sorry. I apparently read too much of my=20 overall purpose into your reply to what I asked about for if the types needed to be changed in fsread.c . I've reported the "I never said that" in 220024. I've also copied and pasted your original reply for reference. Again: Sorry to have misrepresented you. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sat Jun 17 08:53:20 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED9D8BF5702 for ; Sat, 17 Jun 2017 08:53:20 +0000 (UTC) (envelope-from karnajitw@gmail.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B5AAA6BE for ; Sat, 17 Jun 2017 08:53:20 +0000 (UTC) (envelope-from karnajitw@gmail.com) Received: by mail-io0-x22e.google.com with SMTP id t87so41966955ioe.0 for ; Sat, 17 Jun 2017 01:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fkJO6okXJTwX5kJ6UhyZHQIPer1pZlcTpSHNJAx8fT0=; b=EOZxvVx+jhpRpszHmUsiFvNtwg8HfEqYC5Ia72TCqRcgbks29kglmMWyTpcW2yJDAb 7HsIHg9Oc3m3erRJRK+W/1FtU4iEU8ymZr0LpWhxz9RsJZVB3yySetACNPVbNxqWWgGv yLyNUC6L/XYAnE7sbIGZl5RGYZDEeIWEqqq+reMWFna937YPPb/+Wq7o42iMZM23G8iu Ugqg+IUno5JPVAvM+EzObPrbYLe1g0Roa9BltpSiblJKI3Cic+QuOM5YC+5AsDD+bnbJ LnqZzgmcnii4ylq4mpG6+wQDmNp7sFqZr7kE4reHOtJE9P5Vf4EQHm1k5s2zXYrJJlig ujxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fkJO6okXJTwX5kJ6UhyZHQIPer1pZlcTpSHNJAx8fT0=; b=dWIigK22iYjuxC5d05rKjWzdLZWAZO3/DLPF/0G5juiET0Rq1JkuEXfxd0PGQPGB2d 59c4GMl9qCAx3TufatT29CvhbkWNe2mwfsw3XKqN6p1x4HfT6XPlhbMlME11+SOzRh0A mbE7oKUrzl3ZS589W5qyPMZLejxESp4lOXIcu3AZbo6ZUn0GlggMblctjooLKAMIvRtA 2yKqzpWSy4h5lnLsq4WmJukvO/SNoZpSBW36utKzxk3+Q8FbKI8iSksibyL87IPGAc6M JK9GLN7xjjyrhjS64SBv2PbN1g179yd0SB9Ctz0Ya5jlqwLhrRwr+dtB4u0WjkCeiGUt zKOg== X-Gm-Message-State: AKS2vOyKDt7chzVKBc3fKPtce+286OGvq1tn6MhONScDckt+SqFjDEJ3 y6KjZFbwlYnrl2Ukvk8/c/KalIXbbA== X-Received: by 10.107.156.17 with SMTP id f17mr1280220ioe.9.1497689599992; Sat, 17 Jun 2017 01:53:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.136.131 with HTTP; Sat, 17 Jun 2017 01:53:19 -0700 (PDT) In-Reply-To: <20170616132329.GY2088@kib.kiev.ua> References: <20170616132329.GY2088@kib.kiev.ua> From: karnajit wangkhem Date: Sat, 17 Jun 2017 14:23:19 +0530 Message-ID: Subject: Re: Undesirable FPU tag word value after PT_SETFPREGS on i386 To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sat, 17 Jun 2017 12:17:21 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 08:53:21 -0000 Thanks that worked... $ ./a.out Child has stopped... FPU TAG = ffff FPU TAG = ffff Send continue to child process a.out test.c Child exited Before: $ ./fpu Process has not terminated, but has stopped... Send continue to child process Enter d = 4.4 Process has not terminated, but has stopped... Send continue to child process d = 4.400000 : e = nan : f = nan Child exited normally Now: $ ./fpu Process has not terminated, but has stopped... Enter d = Send continue to child process 4.4 Process has not terminated, but has stopped... Send continue to child process d = 4.400000 : e = 7.900000 : f = 10.400000 Child exited normally Regards, Karan On Fri, Jun 16, 2017 at 6:53 PM, Konstantin Belousov wrote: > On Fri, Jun 16, 2017 at 03:19:35PM +0530, karnajit wangkhem wrote: > > Hi All, > > > > Please help me understand why NaN is reported after PT_SETFPREGS. As a > > result of this, if the inferior process contains an float operations, > they > > result in NaN. > > This behavior is not seen in case of 32/64 bit binary on amd64. > > > > My env > > kern.ostype: FreeBSD > > kern.osrelease: 11.0-RELEASE-p1 > > kern.version: FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 03:40:55 UTC > > 2016 > > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > > hw.machine_arch: i386 > > > > Result of the below program on an i386 box > > FPU TAG = ffff > > FPU TAG = fe00 -> 11 11 11 10 00 00 00 00 > > FreeBSD 11 started translating between abridged and full FPU tags, but > the translation appeared to be broken. Try this. > > diff --git a/sys/i386/isa/npx.c b/sys/i386/isa/npx.c > index 5ca1ee507fe..de29adc4101 100644 > --- a/sys/i386/isa/npx.c > +++ b/sys/i386/isa/npx.c > @@ -1108,7 +1108,7 @@ npx_fill_fpregs_xmm1(struct savexmm *sv_xmm, struct > save87 *sv_87) > sv_87->sv_ac[i] = sv_xmm->sv_fp[i].fp_acc; > if ((penv_xmm->en_tw & (1 << i)) != 0) > /* zero and special are set as valid */ > - penv_87->en_tw &= ~(3 << i); > + penv_87->en_tw &= ~(3 << i * 2); > } > } > > @@ -1143,7 +1143,7 @@ npx_set_fpregs_xmm(struct save87 *sv_87, struct > savexmm *sv_xmm) > penv_xmm->en_tw = 0; > for (i = 0; i < 8; ++i) { > sv_xmm->sv_fp[i].fp_acc = sv_87->sv_ac[i]; > - if ((penv_87->en_tw && (3 << i)) != (3 << i)) > + if ((penv_87->en_tw & (3 << i * 2)) != (3 << i * 2)) > penv_xmm->en_tw |= 1 << i; > } > } >