From owner-freebsd-hackers@freebsd.org Sun Aug 14 03:29:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 094F0BB8C14 for ; Sun, 14 Aug 2016 03:29:34 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from nm46-vm6.bullet.mail.bf1.yahoo.com (nm46-vm6.bullet.mail.bf1.yahoo.com [216.109.115.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B64A31CA5 for ; Sun, 14 Aug 2016 03:29:33 +0000 (UTC) (envelope-from pfg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1471145366; bh=acnsskhDu6Kyov/B9Y9YZvHv6A6PnKq9v/9GWeti5z4=; h=Subject:To:References:Cc:From:Date:In-Reply-To:From:Subject; b=pwsKOSjduQw5UWetJDHzoPfDhvJy0/sl5ILWrEGtZuHslqVAQN3GPTzz3y9XsMzSCUGcNBrrwJyL/KEVkK2p9oELwi5WYzXDX1w0rsLcPe+10tSiOvgz7tpptgEn/szV91lPmg6tjWVhmbbUkO/quZcNm4GZGk3e8OZ0ThwLfQSWuQuuYYoRaGO2gmePH/FBmDj7QRE3HCYBH71rnovkuH8l09AN89OCcqnLoThkMlq4E5hAtYY0x8Fa/yd0jvvY8tIQrlpEZP6XMDIrXC+SPno9r+eWwotC1eT6cqBhM+V2og6M0TPfLSgMjdyzeArpEPRfVEsjPM3BxcBmZT403w== Received: from [98.139.170.182] by nm46.bullet.mail.bf1.yahoo.com with NNFMP; 14 Aug 2016 03:29:26 -0000 Received: from [98.139.211.162] by tm25.bullet.mail.bf1.yahoo.com with NNFMP; 14 Aug 2016 03:29:26 -0000 Received: from [127.0.0.1] by smtp219.mail.bf1.yahoo.com with NNFMP; 14 Aug 2016 03:29:26 -0000 X-Yahoo-Newman-Id: 517861.39669.bm@smtp219.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: pYJ5SX0VM1noWlCuL81I_k42mH8og7l_tGiCnNcXbZUU_YT jMs4dN8XdmBZlkmmAJCZiqlRNzHAwAcNjXfdL5_pSp0dG.D.ivltrPjoABig RqA6mH8trQPA9ApQfIIU8qBuZnALSk0iQLd_BtW82.zca2wt_V_KIaazTLVB 2cmOGh7q2s861zc8nq8k_sKaifsxKn1zS9AVgCY7wevPVNrdxg0JfG3vYSwK 2qzddzNhGpCSXyllMNOBvT2lECPWtw_u1u4HIwa7HD1hJ.TGRr3ugs_1TPaC QulQfZYFQVu_3Xuxn8R.5hfeQKIgQ3hG7t.Qp8mpi1j3t5A_DmaZrf5IqsB. jfchYmFGSwBWbTmwB0idtAhnyvmjFua4BJJmexHOYD3M1r2uMwhy7ghsu6KT L85k2WNZyQIhNGnNMOstY_NcjZlRsGbNFCnfI6Yex9Ee.qRA_9XFSoj03rrx EBxdu9ipiwb7JVByOHTJkA1Nv4CI.rxMCE0AbiDy.hD5wT5RUfuv2ZeD6Pih 7kA2Mgn1lbDFfZBwNOFFkhmm3kOOR_HddgAEperlgpgmVHw-- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf Subject: Re: How to get better debugging for the kernel. To: Gordon Ross References: <7b58509a-556b-0784-56c5-00378a1fc5e2@FreeBSD.org> Cc: FreeBSD Hackers From: Pedro Giffuni Message-ID: <4e42609b-2d18-e90c-68fd-56b263cfc429@FreeBSD.org> Date: Sat, 13 Aug 2016 22:29:25 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Aug 2016 03:29:34 -0000 On 13/08/2016 18:49, Gordon Ross wrote: > Actually, mdb primaryliy uses Compact ANSI-C Type Format (CTF), > which is in sections added to binaries by tools that convert from > either dwarf or stabs. > > I've assumed FreeBSD already has CTF since dtrace normally uses it too... Ah yes, FreeBSD gets CTF from dwarf. Indeed we had a student that worked for two GSoC projects on CTF-related projects. I am not sure if the MDB port has advanced but you are looking for this page: https://wiki.freebsd.org/PortingMDB Enjoy, Pedro. > > On Sat, Aug 13, 2016 at 1:22 PM, Pedro Giffuni wrote: >> Gordon Ross wrote: >>> I heard a rumor someone might be working on a port of illumos "mdb". >>> Anyone know if that's true, and how far along it went? >>> >> FWIW, I heard the same rumor. >> >>> We depend heavily upon this tool for production support; >>> so much that I'm not sure how we'd live without it. >> >> There is interest, and I have seen some attempt at work on it however >> it is not something likely to bear much fruit because mdb knows nothing >> about dwarf; AFAICT it knows only about stabs, which we don't use >> at all. >> >> Also, Oracle did make some Dtrace enhancements in upstream gdb that >> would be good to port. >> >> Pedro. From owner-freebsd-hackers@freebsd.org Sun Aug 14 05:53:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 447D7BB96AA; Sun, 14 Aug 2016 05:53:35 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-yw0-x22e.google.com (mail-yw0-x22e.google.com [IPv6:2607:f8b0:4002:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 061121B7B; Sun, 14 Aug 2016 05:53:35 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-yw0-x22e.google.com with SMTP id j12so12734128ywb.2; Sat, 13 Aug 2016 22:53:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=RaM3dVYlvXRPMFZLb8yN69GZIDDEa5ha9zG8kaX9QNU=; b=gGpbo1vAycUyWOTwN02KNZ5YKpusSMB0rYfBUvAfwNy5tabl1AjFljSKeUrtCF5IAA U9ojzgVyXbxvVOC2N2pcn30N8N+fLvfKf1Uf4Tv5RWSc+s6hujaDjFO2/Gc7NnsHkgEl N+8N4fN3wda5S4dJw54f1SpRUbAK9KKG0rmorWTZqKiowx8EhGGMdLPJ8HGPHgrV2wml 9c+0IXchmshN/oJrMPkS7I6p3MS2mvFIZqHXJ4uyi+FT0Am8PNbq4YtGVZGLRx9PB1R+ b27XX5CEonddvbLoEWKbGOGcn1UyawauDXFQxRfHhZ5Y3TA0DSo3D+hAfa0Q7PYMuJGE JSWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=RaM3dVYlvXRPMFZLb8yN69GZIDDEa5ha9zG8kaX9QNU=; b=BQtdntLXEB4JvMeJ/P1GERqzamebjq/qhYnJalYGRAuyCYXP4PeS3lOQDXwZX7iKyM bEq91OD16GoiQu51oEORkfuq6ZpVcucoIsLhI5ffW4KKK7JINLzFURpqULwJ1O649h0n WeJUOJVUWQjYncc0SvjjgYZ3HDu69g1kNy7JfzWQj6BxTYubnRheiFgJsL1zN92NXZAx icIN2jw0zYUpVoWmMZVlXiCwFubC/nPOG7LsZ6dR3H8ECRLTTTVe7UaocOZwlGRfBW7A jcJNPXWvUi7sL8QOimznNtrOF8/JUUDW6uCBEzEBnTIqm8r3ikxH0QLUYMDkn+1SZ+bZ UVSA== X-Gm-Message-State: AEkooutP2hEOoCavoZ8o3dDfH+7KENe7Cap3UwX6YWkw1p3iGKZ1vhD8QfDXF9wehwyBiP3DRsWE6CA6DmogsQ== X-Received: by 10.13.195.67 with SMTP id f64mr15761801ywd.1.1471154013865; Sat, 13 Aug 2016 22:53:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.161.37 with HTTP; Sat, 13 Aug 2016 22:53:33 -0700 (PDT) From: Zaphod Beeblebrox Date: Sun, 14 Aug 2016 01:53:33 -0400 Message-ID: Subject: ZFS corrupt DVA panic: can it be fixed? To: freebsd-fs , FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Aug 2016 05:53:35 -0000 Before this problem, I had a few crashes... which may have been hardware related. The hardware is (I think) fixed, but this problem remains. My searches seem to indicate that this has happened to other people. ... I've pasted here only the first two lines of the last 3 panic's I've had. panic: dva_get_dsize_sync(): bad DVA 1573890:1587590144 #3 0xffffffff822b8b01 at dva_get_dsize_sync+0xb1 panic: dva_get_dsize_sync(): bad DVA 1573890:1587590144 #3 0xffffffff822b8b01 at dva_get_dsize_sync+0xb1 panic: dva_get_dsize_sync(): bad DVA 1573890:1587590144 #3 0xffffffff822b8b01 at dva_get_dsize_sync+0xb1 I gather that the machine runs until something causes the kernel to encounter the corrupt DVA. I gather from reading stuff that this is part of the structure that holds free space on the drive. Since the numbers are the same in each panic, I'm assuming that each panic is encountering the same one. This is also the panic that is not dumping properly to either USB or spinning disk. I have zdb -uuumcD running right now. It seems to estimate that it's going to take an awfully long time, but the estimation might be broken because it's on 159 of 171 of whatever it's reading. Now... question: is this fixable? Can I just mark off the space as unusable, maybe? Since this has happened to more than one person, I gather it's a significant hole in the claim that ZFS is crashproof (or that it doesn't need repair after crashing). Maybe this check can be added to scrub (or scrub + an option)? Or maybe when we run across it, we fix it? Does fixing it (in the theoretical sense) require knowing all the free space on the drive? Doesn't scrub do that? From owner-freebsd-hackers@freebsd.org Mon Aug 15 05:15:17 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A42E8BBA36E for ; Mon, 15 Aug 2016 05:15:17 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72B8E1ED8 for ; Mon, 15 Aug 2016 05:15:17 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: by mail-pa0-x229.google.com with SMTP id fi15so13575617pac.1 for ; Sun, 14 Aug 2016 22:15:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=jePPY2X3wk1kZkEgJhz1o5liXfmRE68TqAz4K6Juz/s=; b=BcFk3yhSVsIvYY+r7qcSbRKDL38QoNiJGOee6LnkoPiVLXLemLwgPT+17Tux6Dzvhq 30dJ+FVhhrxaafQVE01+ywcNKcTDoCHMHRB8JKDqXaqZh+iOQ+EYB11xLC2V148jTZ7X siMq4tuKBSEDToIaXa+jmxdHw45+LjRMIcJOzWY0fek0FogtPGy7ukEg84Sk9+Tk/uRS xnIJbYkI+V580bntGg0+v2T/UsDM8vbsUeY+nc9Pg5NWfpXoHeR8ymsb967Ni9Pm1WNz hcJotbgJEvALHiodI6zrHmFKf56wQFchCq7m5VD4z679A+gGWlWMfAeV4IOLuRUgR+HB PWrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jePPY2X3wk1kZkEgJhz1o5liXfmRE68TqAz4K6Juz/s=; b=U+Heayb6TaIV17XvD9gimUXFe2Z1W4MRoUvruscTR3jm/atDDn1NZSTOuIC237oZ/V U0ULS+Rpm4lYDLrFFzoq+/HudCxfEqkj6bzoTtVw8AKDNzzejBnw5ZT5Ijw9LZ3ItdlX 9haOjR1twSpHAhH04v3lgBSi2U8nwyFMm4v2qdOcLDkvDk930Ipox0sMNKiptx/OE6NP 77nvnO8Jm+Mhla+mfxvTXm1YYKfUysyV38MNvP9M89P8XLH/sAgTKiphkRWx0eJJLH+Z pitQtK7A0lX4Fkdofh9l2UUzXrwUpfgQg9XOtItl6y212vjcM5E9PhYh6hoSnoeiTXNA YNBQ== X-Gm-Message-State: AEkoouuC4mVb4q2KtHE6pWF9xBrqmMtCszL5opPwij9mbE1ghZJBJH7+t42xJTOLf7Dfzg== X-Received: by 10.66.193.65 with SMTP id hm1mr51726780pac.12.1471238116803; Sun, 14 Aug 2016 22:15:16 -0700 (PDT) Received: from splash.akips.com (CPE-120-146-191-2.static.qld.bigpond.net.au. [120.146.191.2]) by smtp.gmail.com with ESMTPSA id ao6sm29328189pac.8.2016.08.14.22.15.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Aug 2016 22:15:15 -0700 (PDT) Date: Mon, 15 Aug 2016 15:15:01 +1000 From: Paul Koch To: freebsd-hackers@freebsd.org Subject: Re: ZFS ARC and mmap/page cache coherency question Message-ID: <20160815151501.5f5b4a86@splash.akips.com> In-Reply-To: <45865ae6-18c9-ce9a-4a1e-6b2a8e44a8b2@denninger.net> References: <20160630140625.3b4aece3@splash.akips.com> <20160703123004.74a7385a@splash.akips.com> <155afb8148f.c6f5294d33485.2952538647262141073@nextbsd.org> <45865ae6-18c9-ce9a-4a1e-6b2a8e44a8b2@denninger.net> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 05:15:17 -0000 Just a followup to my original post about VM/ZFS ARC coherency. We've done a few simple changes to our application to get around the coherency issues, and observed some odd things. Description of our app: Very large scale ping/snmp poller/database (made up of 8 underlying databases - configuration, event, time-series, common strings, etc). Each database contains various sized mmap'ed files, ranging from 512 bytes to many gigabytes. All mmap'ed files are opened with the MAP_NOSYNC flag. The poller updates every page of the mmap'ed data every minute. We fsync the mmap'ed data every 10 minutes when the system is mostly idle. Everything works fine while the mmap'ed data is in both the VM and ZFS caches. Every 80 minutes we process a very large amount of cached poller data, which pushes the mmap'ed data out of the ARC. The performance of the next 10 minute fsync then falls off a cliff, causing lots of read/write contention. This is due to the lack of VM/ZFS ARC coherency. We've changed our sync algorithm to something like: 1. Exclusive lock on the entire database 2. fsync() all the small 512 byte mmap'ed files 3. Write out new copies of all the other mmap'ed files - mprotect - write - rename - munprotect 4. Release exclusive lock 5. Signal all database processes so they reopen the database. Our sync now completes in a very predictable manner and is significantly faster. But we observed some odd things: 1. The rename in step 3 above can be painfully slow for large files. Not sure what is going on, but we also noticed that deleting the same files using unlink(2) or rm(1) was also painfully slow. It is much much faster to truncate(2) the large files to zero bytes before calling rename(2) or unlink(2). Why is that ?? 2. We are using both fsync(2) and write(2) in the above sync. We observed that order was very important. If we write/rename the large mmap'ed files first and then fsync the small 512 byte files, the fsync sits in zio for some time. Doing the fsync calls first and then the large write/renames is much faster. Not sure what is going on there. Paul. -- Paul Koch | Founder | CEO AKIPS Network Monitor | akips.com Brisbane, Australia From owner-freebsd-hackers@freebsd.org Mon Aug 15 17:18:55 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EED8BBA31E for ; Mon, 15 Aug 2016 17:18:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E5F9C1922; Mon, 15 Aug 2016 17:18:54 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9171FB98E; Mon, 15 Aug 2016 13:18:53 -0400 (EDT) From: John Baldwin To: Warner Losh Cc: Andriy Gapon , "freebsd-hackers@freebsd.org" Subject: Re: on BIOS problems with disks larger than 2 TB Date: Mon, 15 Aug 2016 10:15:51 -0700 Message-ID: <3259742.YBnQU2zACB@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.3-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: References: <6cec427b-4df1-50f0-3014-a96e5f8210f5@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 15 Aug 2016 13:18:53 -0400 (EDT) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 17:18:55 -0000 On Friday, August 12, 2016 10:44:04 PM Warner Losh wrote: > On Fri, Aug 12, 2016 at 2:58 PM, Andriy Gapon wrote: > > On 12/08/2016 22:18, John Baldwin wrote: > >> Hmm, I'm not sure how easy it is to handle this case (i.e. how do you know > >> if an LBA beyond the size is really legit due to truncation vs coming from > >> corrupted metadata). Related is that tsoome's bcache stuff wants to know > >> where the end of the disk is (to avoid reading off the end), so just > >> ignoring the size is not easy. > > > > One idea that I have in mind but haven't really explored yet is for GPT > > formatted disks. Basically, if a GPT label hints that the disk size is larger > > than what BIOS reports, then we could try to read a backup label and if it > > matches what we expect, then we could adjust the size. > > > > Hmm, I think I recall that a long time ago some BIOSes used to do something > > similar with MBR :-) > > I think we should just trust the GPT bounds and ignore the actual size > of the disk. > If this is incorrect, we'll get I/O errors to indicate something is > wrong. Doesn't > matter what's wrong, at the end of the day, and many different > pathologies present > as the same error. We should ignore the total size of the disk reported by BIOS > routines because they lie to stay compatible with the long-dead hand > of the past. Well, we don't always have a GPT for one, and not all BIOS'es lie. We can always just start with what the BIOS says and override it if we find a valid GPT that indicates a larger size (which is what Andriy suggested I believe). -- John Baldwin From owner-freebsd-hackers@freebsd.org Mon Aug 15 17:19:01 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54371BBA34B for ; Mon, 15 Aug 2016 17:19:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3751919C0 for ; Mon, 15 Aug 2016 17:19:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 31890BBA348; Mon, 15 Aug 2016 17:19:01 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 311E4BBA347 for ; Mon, 15 Aug 2016 17:19:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08D4019B0 for ; Mon, 15 Aug 2016 17:19:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E9912B915; Mon, 15 Aug 2016 13:18:59 -0400 (EDT) From: John Baldwin To: David Wolfskill Cc: hackers@freebsd.org Subject: Re: "ipmi0: KCS..." whines Date: Fri, 12 Aug 2016 17:29:58 -0700 Message-ID: <6661021.NZidrlQVOE@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.3-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20160812214340.GZ1112@albert.catwhisker.org> References: <20160811175409.GW1112@albert.catwhisker.org> <2855524.PakqtZoDR6@ralph.baldwin.cx> <20160812214340.GZ1112@albert.catwhisker.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 15 Aug 2016 13:19:00 -0400 (EDT) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 17:19:01 -0000 On Friday, August 12, 2016 02:43:40 PM David Wolfskill wrote: > On Fri, Aug 12, 2016 at 11:54:38AM -0700, John Baldwin wrote: > > ... > > So the issue is probably that the BMC controller on your box is sometimes > > slow in responding. The completion code is the third byte of the reply > > we wait to read after sending a request to the BMC via KCS. However, the > > first two bytes just echo back the request ID and command we asked for, so > > it may be that the BMC echoes those back right away without waiting for > > whatever work it needs to do to handle the request to complete, but doesn't > > send the completion code (the status of the request) until the request is > > fully processed. > > > > The driver is complaining that the BMC didn't respond with the completion > > code before it's timeout expired. The default timeout is MAX_TIMEOUT in > > sys/dev/ipmi/ipmivars.h which corresponds to 6 seconds. It may be that > > occasionally some "background" task runs in the BMC OS that delays responses > > to handling commands. It could also be that whatever work the BMC has to do > > to read this specific value is actually timing out or having issues in the > > hardware, etc. > > I could easily modify the stress-test loop to run "date" after each > "ipmitool" invocation. (Pity we don't seem to have a sub-second format > in strftime().) > > So... I tried the above (interspersing "date" commands while running > "ipmitool dcmi power reading" in a loop within script(1)). I did not > get a whine at 32 repetitions; I got one at 64. > > The total elapsed time was no more than 3 seconds (last timestamp - > first timestamp difference was 2 seconds). Hmm, you might see what 'MAX_TIMEOUT' is in sys/dev/ipmi/ipmivars.h in your tree. It might also be worthwhile wrapping it in ()'s as in HEAD it is just a bare '6 * hz'. The code to wait for IBF doesn't look like it would break without the ()'s though. It was bumped from 3 seconds to 6 seconds back in 10-current in r253812, but perhaps your box has 3 seconds instead of 6? -- John Baldwin From owner-freebsd-hackers@freebsd.org Mon Aug 15 17:24:59 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCFCDBBA61B for ; Mon, 15 Aug 2016 17:24:59 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A7BD61178 for ; Mon, 15 Aug 2016 17:24:59 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id A36EBBBA61A; Mon, 15 Aug 2016 17:24:59 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A30F1BBA619 for ; Mon, 15 Aug 2016 17:24:59 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7450E1176; Mon, 15 Aug 2016 17:24:59 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id u7FHOvlr017324; Mon, 15 Aug 2016 17:24:57 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id u7FHOvEw017323; Mon, 15 Aug 2016 10:24:57 -0700 (PDT) (envelope-from david) Date: Mon, 15 Aug 2016 10:24:57 -0700 From: David Wolfskill To: John Baldwin Cc: hackers@freebsd.org Subject: Re: "ipmi0: KCS..." whines Message-ID: <20160815172457.GS5926@albert.catwhisker.org> References: <20160811175409.GW1112@albert.catwhisker.org> <2855524.PakqtZoDR6@ralph.baldwin.cx> <20160812214340.GZ1112@albert.catwhisker.org> <6661021.NZidrlQVOE@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="pd495SECmvzXpBRb" Content-Disposition: inline In-Reply-To: <6661021.NZidrlQVOE@ralph.baldwin.cx> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 17:24:59 -0000 --pd495SECmvzXpBRb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 12, 2016 at 05:29:58PM -0700, John Baldwin wrote: > .... > > The total elapsed time was no more than 3 seconds (last timestamp - > > first timestamp difference was 2 seconds). >=20 > Hmm, you might see what 'MAX_TIMEOUT' is in sys/dev/ipmi/ipmivars.h in yo= ur > tree. It might also be worthwhile wrapping it in ()'s as in HEAD it is j= ust a > bare '6 * hz'. The code to wait for IBF doesn't look like it would break= =20 > without the ()'s though. I haven't tried wrapping in parens (yet), but: g1-252(11.0-P)[3] grep MAX_TIMEOUT sys/dev/ipmi/ipmivars.h #define MAX_TIMEOUT 6 * hz > ... Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --pd495SECmvzXpBRb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJXsfrpXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4X6pYH/R2sOgq/bdPdfDEek+GnIwsr miiNmNy3T/bV8V9kdm9D1VSD9ydFqw7BaoDenWodFsdW1s0BvGdwcBfu2teQpPgs gNbClF4sSOKtncgR2QeLutTadxfwfmgR/q8Wr8bx9OTOgRBRnxp6rB5WHv9ZMFK0 5SWH0GR+t8O76iHBdjRRL3XQXpkDcxtq+Wq7XvTEd6JGOducXlexyRUi+A2reYIC +oIpB3Buf374nRc2zNkGIQesM71hTPVjRWpu5kE3VLDMUdwB9XEyzJVQxtZEw0T4 NyHSfoh3IT9PVC4uCttj1EUGOWEe4FsVcGzVx7LZnuUWsMTuB0UijgWybx1SGpM= =ntu/ -----END PGP SIGNATURE----- --pd495SECmvzXpBRb-- From owner-freebsd-hackers@freebsd.org Mon Aug 15 22:37:30 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B70FBB9586; Mon, 15 Aug 2016 22:37:30 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B438E150C; Mon, 15 Aug 2016 22:37:29 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-wm0-f65.google.com with SMTP id o80so13402728wme.0; Mon, 15 Aug 2016 15:37:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-transfer-encoding:subject:date :message-id:cc:to:mime-version; bh=js/dn0ARYNgBAO9nkckxOs9AVSSUkrrFAtu/8ngTw3U=; b=lYRsr9cQj5BZ0vrafeSQ620ypUc1pn3ZNiYNlIi7qH6Qd/NX2KGuTuOr/hpNkLdXdN X0Tx/KqDzKX22NyHoAIgXvM28HF8bkNMzXz0lNnqwm0YxlWeD9EO47ZdTmFin4BOjYOF ZcZYib7mM4IndVeWpxpFYpFsxAqeWje8FduTz4+vaC3Su7EpYUmtvxCwv+niOeU2H9RT zvyXD7FhQVDd9m0KxK623y3KJNM42cQm9LT4qRThZAbnU0kZObE1XNP7hf+4htVdZ2Yy i5LmQNsTKidXT6QTQILEHIxQ4ovzB6lZGk3snZ4DbLdgOePjhWtgNJC7TDT+o6hnyp7f Mu8Q== X-Gm-Message-State: AEkooutReWOIkKu/yRJDN3ipULfc8nWOXkbDYEmACWUAjFFm3VOLML0R6A/ocXJDMwRuNg== X-Received: by 10.28.113.151 with SMTP id d23mr18500003wmi.89.1471299522730; Mon, 15 Aug 2016 15:18:42 -0700 (PDT) Received: from maka.fritz.box (dslb-178-008-181-169.178.008.pools.vodafone-ip.de. [178.8.181.169]) by smtp.gmail.com with ESMTPSA id h7sm23547890wjd.17.2016.08.15.15.18.41 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 15 Aug 2016 15:18:41 -0700 (PDT) From: Mateusz Piotrowski <0mp@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: How to bring au_to_attr(3) back to the userland? Date: Tue, 16 Aug 2016 00:18:40 +0200 Message-Id: <83CC669E-FED9-4ABE-A5A5-376E1A743AF8@FreeBSD.org> Cc: Konrad Witaszczyk , rwatson@FreeBSD.org To: freebsd-hackers@freebsd.org, trustedbsd-audit@freebsd.org, trustedbsd-discuss@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 22:37:30 -0000 Hello, I participate in Google Summer of Code at FreeBSD this year. My project = is about converting Linux Audit logs to the BSM format (see my wiki[0]). Recently, I've come across a problem with the libbsm(3) API. I'd like to = be able to generate an attribute token. Unfortunatelly, au_to_attr which = generates those tokens is not available in the userland (I email FreeBSD-hackers at = FreeBSD about this issue[1]). Together with my mentor we came up with a few possible solutions to this = problem but we are not sure which one is the best. This is why I'd like to = dicuss the pros and cons. Solutions: 1. The first idea is to add a userland version of the au_to_attr = function. The implementation would be similar to the one of the au_to_exec_* = functions. (See sys/security/audit/bsm_token.c[2].) 2. The second idea is to bring back the vattr structure. At the moment au_to_attr has one paramter of type `struct vnode_au_info`. = Historically, au_to_attr used `struct vattr`. A possible solution is to bring vattr = to the userland and change the parameter of au_to_attr back to `struct vattr`. At the moment `struct vattr` is included in sys/vnode.h but it lacks = the interace. (I summed up everything I know on this wiki page[3].) 3. The last idea is to make `struct vnode_au_info` and `au_to_attr` = accessible from the userland (by simply unwrapping the prototypes from the = KERNEL/_KERNEL conditional compilation macros). Cheers, -Mateusz [0]: = https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools [1]: = https://lists.freebsd.org/pipermail/freebsd-hackers/2016-August/049835.htm= l [2]: = https://github.com/freebsd/freebsd/blob/af3e10e5a78d3af8cef6088748978c6c61= 2757f0/sys/security/audit/bsm_token.c#L1281-L1405 [3]: = https://github.com/0mp/freebsd/wiki/vattr(99://github.com/0mp/freebsd/wiki= /vattr(99) From owner-freebsd-hackers@freebsd.org Tue Aug 16 06:58:24 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81D95BBB0EF; Tue, 16 Aug 2016 06:58:24 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id 467FC1223; Tue, 16 Aug 2016 06:58:24 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from [10.0.1.11] (host109-151-51-95.range109-151.btcentralplus.com [109.151.51.95]) by cyrus.watson.org (Postfix) with ESMTPSA id 0267846B43; Tue, 16 Aug 2016 02:58:22 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to bring au_to_attr(3) back to the userland? From: "Robert N. M. Watson" In-Reply-To: <83CC669E-FED9-4ABE-A5A5-376E1A743AF8@FreeBSD.org> Date: Tue, 16 Aug 2016 07:58:22 +0100 Cc: freebsd-hackers@freebsd.org, trustedbsd-audit@freebsd.org, trustedbsd-discuss@freebsd.org, Konrad Witaszczyk Content-Transfer-Encoding: quoted-printable Message-Id: <09D137C4-2630-4B93-ACDC-CB3AFC86D89F@FreeBSD.org> References: <83CC669E-FED9-4ABE-A5A5-376E1A743AF8@FreeBSD.org> To: Mateusz Piotrowski <0mp@FreeBSD.org> X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 06:58:24 -0000 Hi Mateusz: I=E2=80=99m hear about your project =E2=80=94 a way to import Linux = audit records would be extremely useful to the community of BSM users = and tool writers. My memory is a bit hazy, but I believe the reason we did not stick with = Solaris=E2=80=99s API in this instance is that, in both Mac OS X and = FreeBSD, struct vattr is an in-kernel data structure that incorporates = kernel-internal data types, and is subject to (moderately frequent) = changes in a way that would disrupt the userspace ABI. In more recent = versions of Mac OS X, struct vattr has been replaced with struct = vfs_attr. The ideal choice would be to take the existing public data structure = describing file attributes (struct stat) and use that =E2=80=94 = unfortunately, not all of the fields we require (in particular the = filesystem ID) are present in the stat structure. I=E2=80=99m slightly unhappy with the name vnode_au_info, as although in = FreeBSD, Mac OS X, and Solaris, =E2=80=9Cvnode=E2=80=9D is the term = used, that=E2=80=99s not portable to Linux, where the term =E2=80=9Cinode=E2= =80=9D is used. We also want to avoid being almost but not quite API = compatible with the Solaris BSM API =E2=80=94 e.g., having an = au_to_attr(3) that takes a different pointer type is likely a recipe for = trouble. As a result, I feel a bit conflicted on the topic, but suspect the = easiest path is to rename vnode_au_info to struct au_vattr, and add a = new API au_to_vattr(3) that accepts a pointer to a new struct au_vattr = pointer. We would continue to not provide an au_to_attr symbol in the = OpenBSM libbsm. This would align us with the Solaris model, accepting = that it is a =E2=80=9Cvnode attribute token=E2=80=9D, but avoid = confusion about function signatures / prototypes / types. I think I=E2=80=99= d also re-craft struct au_vattr a bit to use the same types that are = present in the underlying token (e.g., uint32_t, =E2=80=A6) rather than = OS-local types (e.g., dev_t), in order to force consumers of the API to = cast to the BSM type, rather than having that occur transparently within = OpenBSM, which could cause information loss invisible to the caller = (they should do the information loss for us). Not sure you how you feel about that strategy? Robert > On 15 Aug 2016, at 23:18, Mateusz Piotrowski <0mp@FreeBSD.org> wrote: >=20 > Hello, >=20 > I participate in Google Summer of Code at FreeBSD this year. My = project is about > converting Linux Audit logs to the BSM format (see my wiki[0]). >=20 > Recently, I've come across a problem with the libbsm(3) API. I'd like = to be able > to generate an attribute token. Unfortunatelly, au_to_attr which = generates those > tokens is not available in the userland (I email FreeBSD-hackers at = FreeBSD > about this issue[1]). >=20 > Together with my mentor we came up with a few possible solutions to = this problem > but we are not sure which one is the best. This is why I'd like to = dicuss the > pros and cons. >=20 > Solutions: >=20 > 1. The first idea is to add a userland version of the au_to_attr = function. The > implementation would be similar to the one of the au_to_exec_* = functions. >=20 > (See sys/security/audit/bsm_token.c[2].) >=20 > 2. The second idea is to bring back the vattr structure. At the moment > au_to_attr has one paramter of type `struct vnode_au_info`. = Historically, > au_to_attr used `struct vattr`. A possible solution is to bring vattr = to the > userland and change the parameter of au_to_attr back to `struct = vattr`. >=20 > At the moment `struct vattr` is included in sys/vnode.h but it lacks = the > interace. >=20 > (I summed up everything I know on this wiki page[3].) >=20 > 3. The last idea is to make `struct vnode_au_info` and `au_to_attr` = accessible > from the userland (by simply unwrapping the prototypes from the = KERNEL/_KERNEL > conditional compilation macros). >=20 > Cheers, >=20 > -Mateusz >=20 > [0]: = https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools > [1]: = https://lists.freebsd.org/pipermail/freebsd-hackers/2016-August/049835.htm= l > [2]: = https://github.com/freebsd/freebsd/blob/af3e10e5a78d3af8cef6088748978c6c61= 2757f0/sys/security/audit/bsm_token.c#L1281-L1405 > [3]: = https://github.com/0mp/freebsd/wiki/vattr(99://github.com/0mp/freebsd/wiki= /vattr(99) >=20 From owner-freebsd-hackers@freebsd.org Tue Aug 16 23:18:21 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90640BBCBF3; Tue, 16 Aug 2016 23:18:21 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32C601C9A; Tue, 16 Aug 2016 23:18:20 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-wm0-f68.google.com with SMTP id i138so18986171wmf.3; Tue, 16 Aug 2016 16:18:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=l87CpPKi8pUuB4kp8x3duVb6TRRFyH2CiF12sRrcdFA=; b=Ufo/Pa8xbGIOjzfdu7JQeZ/glqHewuTc6F13E44MIg9WUFPKEK3jokMtBKuF5eCl11 U6o49fpyfneVUrN3Q0lfTQypoxp4/kV/J/bHd0NGwDu6bCGKtI3qIx8EjPdw9cQaB+LE MC7U79U6KVX/1y7ObrE6EH5teuORyz9I7ftlC0VMX+0UfVkDXklPlVkNqzqG8aTsWsd8 BCPdjsixCk5/xlbSXayPr47Qd5sn3deUJo73osGq1jPlfSlw4XUNA1wqm9HG5AL7r5Uq 0UwvGIgGYNT9I6EE1cINhom+jc/HBxr1q+ZxmggskxebgPVQmmbsy+EoUsIJHEFUf8Ao 7nPg== X-Gm-Message-State: AEkoouvA1sVYIROr3N9+os3XnDJuHpH22CItsojD1T+8j8YHg3t/oyo3QZc/rVaarD5eqA== X-Received: by 10.25.210.75 with SMTP id j72mr6165112lfg.4.1471389492594; Tue, 16 Aug 2016 16:18:12 -0700 (PDT) Received: from [192.168.0.15] (87-207-152-10.dynamic.chello.pl. [87.207.152.10]) by smtp.gmail.com with ESMTPSA id 206sm1145084ljj.4.2016.08.16.16.18.11 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 16 Aug 2016 16:18:11 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to bring au_to_attr(3) back to the userland? From: Mateusz Piotrowski <0mp@FreeBSD.org> In-Reply-To: <09D137C4-2630-4B93-ACDC-CB3AFC86D89F@FreeBSD.org> Date: Wed, 17 Aug 2016 01:18:10 +0200 Cc: freebsd-hackers@freebsd.org, trustedbsd-discuss@freebsd.org, trustedbsd-audit@freebsd.org, Konrad Witaszczyk Content-Transfer-Encoding: quoted-printable Message-Id: References: <83CC669E-FED9-4ABE-A5A5-376E1A743AF8@FreeBSD.org> <09D137C4-2630-4B93-ACDC-CB3AFC86D89F@FreeBSD.org> To: "Robert N. M. Watson" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 23:18:21 -0000 Hello, On 16 Aug 2016, at 08:58, Robert N. M. Watson = wrote: >> On 15 Aug 2016, at 23:18, Mateusz Piotrowski <0mp@FreeBSD.org> wrote: >>=20 >> I participate in Google Summer of Code at FreeBSD this year. My = project is about >> converting Linux Audit logs to the BSM format (see my wiki[0]). >>=20 >> Recently, I've come across a problem with the libbsm(3) API. I'd like = to be able >> to generate an attribute token. Unfortunatelly, au_to_attr which = generates those >> tokens is not available in the userland (I email FreeBSD-hackers at = FreeBSD >> about this issue[1]). >>=20 >> Together with my mentor we came up with a few possible solutions to = this problem >> but we are not sure which one is the best. This is why I'd like to = dicuss the >> pros and cons. >>=20 >> Solutions: >>=20 >> 1. The first idea is to add a userland version of the au_to_attr = function. The >> implementation would be similar to the one of the au_to_exec_* = functions. >>=20 >> (See sys/security/audit/bsm_token.c[2].) >>=20 >> 2. The second idea is to bring back the vattr structure. At the = moment >> au_to_attr has one paramter of type `struct vnode_au_info`. = Historically, >> au_to_attr used `struct vattr`. A possible solution is to bring vattr = to the >> userland and change the parameter of au_to_attr back to `struct = vattr`. >>=20 >> At the moment `struct vattr` is included in sys/vnode.h but it lacks = the >> interace. >>=20 >> (I summed up everything I know on this wiki page[3].) >>=20 >> 3. The last idea is to make `struct vnode_au_info` and `au_to_attr` = accessible >> from the userland (by simply unwrapping the prototypes from the = KERNEL/_KERNEL >> conditional compilation macros). >>=20 >> [0]: = https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools >> [1]: = https://lists.freebsd.org/pipermail/freebsd-hackers/2016-August/049835.htm= l >> [2]: = https://github.com/freebsd/freebsd/blob/af3e10e5a78d3af8cef6088748978c6c61= 2757f0/sys/security/audit/bsm_token.c#L1281-L1405 >> [3]: = https://github.com/0mp/freebsd/wiki/vattr(99://github.com/0mp/freebsd/wiki= /vattr(99) >>=20 >=20 > I=E2=80=99m hear about your project =E2=80=94 a way to import Linux = audit records would be extremely useful to the community of BSM users = and tool writers. Thank you. It is really motivating to hear that after a couple of weeks = of starting=20 implementing the conversion over and over again. > My memory is a bit hazy, but I believe the reason we did not stick = with Solaris=E2=80=99s API in this instance is that, in both Mac OS X = and FreeBSD, struct vattr is an in-kernel data structure that = incorporates kernel-internal data types, and is subject to (moderately = frequent) changes in a way that would disrupt the userspace ABI. In more = recent versions of Mac OS X, struct vattr has been replaced with struct = vfs_attr. >=20 > The ideal choice would be to take the existing public data structure = describing file attributes (struct stat) and use that =E2=80=94 = unfortunately, not all of the fields we require (in particular the = filesystem ID) are present in the stat structure. This sounds like the best temporarily idea although the "au_vattr" plan = seems to be much=20 more reasonable to me. > I=E2=80=99m slightly unhappy with the name vnode_au_info, as although = in FreeBSD, Mac OS X, and Solaris, =E2=80=9Cvnode=E2=80=9D is the term = used, that=E2=80=99s not portable to Linux, where the term =E2=80=9Cinode=E2= =80=9D is used. We also want to avoid being almost but not quite API = compatible with the Solaris BSM API =E2=80=94 e.g., having an = au_to_attr(3) that takes a different pointer type is likely a recipe for = trouble. >=20 > As a result, I feel a bit conflicted on the topic, but suspect the = easiest path is to rename vnode_au_info to struct au_vattr, and add a = new API au_to_vattr(3) that accepts a pointer to a new struct au_vattr = pointer. We would continue to not provide an au_to_attr symbol in the = OpenBSM libbsm. This would align us with the Solaris model, accepting = that it is a =E2=80=9Cvnode attribute token=E2=80=9D, but avoid = confusion about function signatures / prototypes / types. I think I=E2=80=99= d also re-craft struct au_vattr a bit to use the same types that are = present in the underlying token (e.g., uint32_t, =E2=80=A6) rather than = OS-local types (e.g., dev_t), in order to force consumers of the API to = cast to the BSM type, rather than having that occur transparently within = OpenBSM, which could cause information loss invisible to the caller = (they should do the information loss for us). >=20 > Not sure you how you feel about that strategy? To sum it up. The idea is to: 1. Rename vnode_au_info to au_vattr. 2. Keep au_to_attr away from the userland. 3. Add au_to_vattr (the parameter of which is struct au_vattr) to the = libbsm API and make it available to the userland. 4. Re-craft au_vattr to use the same types that are present in the = underlying=20 attribute token. I am not sure if I understand this properly; do we want to simply rename=20= vnode_au_info to au_vattr and make it available in the userland after a = couple=20 of modifications? If so then it sounds like a good idea to me as long as = I don't=20 break something accidentally. Wouldn't renaming and modifying struct = vnode_au_info=20 cause compatibility problems and potentially break someone's software? Apart from that it sounds like a reasonable solution. Thank you very much for the detailed introduction to the complexity of = this problem. Although the GSoC is coming to an end and I plan to focus on integrating = my work into auditdistd, I hope to apply the solutions we discuss here sometime = later. -Mateusz= From owner-freebsd-hackers@freebsd.org Wed Aug 17 06:47:46 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 810FEBBD9E3; Wed, 17 Aug 2016 06:47:46 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id 635001398; Wed, 17 Aug 2016 06:47:46 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from [10.0.1.11] (host109-151-51-95.range109-151.btcentralplus.com [109.151.51.95]) by cyrus.watson.org (Postfix) with ESMTPSA id A24BF46B89; Wed, 17 Aug 2016 02:47:39 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to bring au_to_attr(3) back to the userland? From: "Robert N. M. Watson" In-Reply-To: Date: Wed, 17 Aug 2016 07:47:38 +0100 Cc: freebsd-hackers@freebsd.org, trustedbsd-discuss@freebsd.org, trustedbsd-audit@freebsd.org, Konrad Witaszczyk Content-Transfer-Encoding: quoted-printable Message-Id: <93122C2D-A660-4A47-A780-44E8309E4377@FreeBSD.org> References: <83CC669E-FED9-4ABE-A5A5-376E1A743AF8@FreeBSD.org> <09D137C4-2630-4B93-ACDC-CB3AFC86D89F@FreeBSD.org> To: Mateusz Piotrowski <0mp@FreeBSD.org> X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 06:47:46 -0000 On 17 Aug 2016, at 00:18, Mateusz Piotrowski <0mp@FreeBSD.org> wrote: > To sum it up. The idea is to: >=20 > 1. Rename vnode_au_info to au_vattr. > 2. Keep au_to_attr away from the userland. > 3. Add au_to_vattr (the parameter of which is struct au_vattr) to the = libbsm > API and make it available to the userland. > 4. Re-craft au_vattr to use the same types that are present in the = underlying=20 > attribute token. >=20 > I am not sure if I understand this properly; do we want to simply = rename=20 > vnode_au_info to au_vattr and make it available in the userland after = a couple=20 > of modifications? If so then it sounds like a good idea to me as long = as I don't=20 > break something accidentally. Wouldn't renaming and modifying struct = vnode_au_info=20 > cause compatibility problems and potentially break someone's software? I guess you have two choices: (1) Retain existing KPIs to slightly ease merging to FreeBSD and Mac OS = X; they can adopt the new in-kernel interfaces when ready. (2) Simply remove the old KPIs and consider it a feature. The former probably does marginally ease merging the new OpenBSM version = (one fewer kernel changes for FreeBSD and Mac OS X at the point of = merge), so I see no harm in retaining it. However, as it=E2=80=99s = ifdef=E2=80=99d _KERNEL || KERNEL in the OpenBSM header, it has not been = exposed to user applications, just the kernel. Remember that changes in these structures don=E2=80=99t affect the = layout and interpretation of the tokens at all =E2=80=94 it=E2=80=99s = really just on the producer side that a KPI changes =E2=80=94 and the = information we=E2=80=99re able to expose. The existing vnode_au_info isn=E2=80=99t really an appropriate public = interface, so do make sure not to remove the ifdefs preventing its use = =E2=80=94 instead, we should focus on a new interface that is = appropriate to be public by virtue of (a) having an appropriate struct = type argument that has both the fields we require and is as = non-OS-specific as possible; (b) doesn=E2=80=99t conflict with the = current interface on FreeBSD/Mac OS X; and (c) doesn=E2=80=99t conflict = with the current interface on Solaris. > Apart from that it sounds like a reasonable solution. >=20 > Thank you very much for the detailed introduction to the complexity of = this problem. > Although the GSoC is coming to an end and I plan to focus on = integrating my work > into auditdistd, I hope to apply the solutions we discuss here = sometime later. The problems are all about compatibility, and I think we have a = reasonable path here that does what we need without too much work. Robert