From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 26 01:30:46 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EFB7106566B for ; Mon, 26 Dec 2011 01:30:46 +0000 (UTC) (envelope-from geoffrey.levand@mail.ru) Received: from fallback1.mail.ru (fallback1.mail.ru [94.100.176.18]) by mx1.freebsd.org (Postfix) with ESMTP id 864348FC0C for ; Mon, 26 Dec 2011 01:30:45 +0000 (UTC) Received: from f71.mail.ru (f71.mail.ru [94.100.178.233]) by fallback1.mail.ru (mPOP.Fallback_MX) with ESMTP id 559EF5482321 for ; Mon, 26 Dec 2011 03:16:36 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail; h=Message-Id:Content-Type:Reply-To:Date:Mime-Version:Subject:To:From; bh=yUwJ9K9TmIUi6hOfUINrb5qLgcy5bmZluG8elGt4R0M=; b=UUdmA5bMCSCkNUIq6Fo5lJYn07Ntxi54gDDmTRj/L5vXl6nfq15AG3R5zfCYGgkQhHW7tzvHDtghb7CrQXhg0iUzJGK1hYZxPqj1g9t3PHQH7RmeqMH/dFLz94vuxlbz; Received: from mail by f71.mail.ru with local id 1RexIw-0007hd-00 for freebsd-hackers@freebsd.org; Mon, 26 Dec 2011 03:16:34 +0400 Received: from [46.5.58.225] by e.mail.ru with HTTP; Mon, 26 Dec 2011 03:16:34 +0400 From: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= To: freebsd-hackers@freebsd.org Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [46.5.58.225] Date: Mon, 26 Dec 2011 03:16:34 +0400 X-Priority: Message-Id: X-Spam: Not detected X-Mras: Ok X-Mras: Ok X-Mailman-Approved-At: Mon, 26 Dec 2011 02:24:01 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: SMP: stopping/restarting CPUs on powerpc arch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 01:30:46 -0000 CkhpLAoKaSB3YXMgaW1wbGVtZW50aW5nIGFuZCB0ZXN0aW5nIGNvcmUgZHVtcCBmZWF0dXJlIG9m IEhERCBkcml2ZXIgb24gUFMzIHBvd2VycGMgYXJjaCBhbmQgCnRyaWVkIHRvIGVudGVyIEtEQiBt YW51YWxseSB3aXRoIHN5c2N0bCBkZWJ1Zy5rZGIucGFuaWM9MS4KQW5kIGknbSBnZXR0aW5nIHRo ZSBmb2xsb3dpbmcgbWVzc2FnZSBmcm9tIEtEQjogdGltZW91dCBzdG9wcGluZyBjcHVzLgoKSSB0 b29rIGEgbG9vayBhdCBnZW5lcmljX3N0b3BfY3B1cyB3aGljaCBzZW5kcyBJUEkgdG8gZXZlcnkg Q1BVLgpPbiBwb3dlcnBjLCBJUElfU1RPUCBpcyBpbXBsZW1lbnRlZCB0aGUgZm9sbG93aW5nIHdh eToKCmNwdWlkID0gUENQVV9HRVQoY3B1aWQpOwpzYXZlY3R4KCZzdG9wcGNic1tjcHVpZF0pOwpz YXZlY3R4KFBDUFVfR0VUKGN1cnBjYikpOwpDUFVfU0VUX0FUT01JQyhjcHVpZCwgJnN0b3BwZWRf Y3B1cyk7CndoaWxlICghQ1BVX0lTU0VUKGNwdWlkLCAmc3RhcnRlZF9jcHVzKSkKwqDCoMKgIGNw dV9zcGlud2FpdCgpOwpDUFVfQ0xSX0FUT01JQyhjcHVpZCwgJnN0b3BwZWRfY3B1cyk7CkNQVV9D TFJfQVRPTUlDKGNwdWlkLCAmc3RhcnRlZF9jcHVzKTsKCmdlbmVyaWNfc3RvcF9jcHVzIHdhaXRz IHVudGlsIGFsbCBDUFVzIGNsZWFyIGl0cyBJRCBpbiBzdG9wcGVkX2NwdXMgYml0bWFwLgpJZiB5 b3UgdGFrZSBhIGxvb2sgYXQgdGhlIHByZXZpb3VzIHNuaXBwZXQgb2YgY29kZSB0aGVuIHRoZSBi aXQgaXMgaW5kZWVkIHNldCBidXQgdGhlbiBjbGVhcmVkIGltbWVkaWF0ZWx5LgpBbmQgdGhhdCBp cyB3aHkgaSBnZXQgdGltZW91dCBtZXNzYWdlIGluIGtkYiBpIHRoaW5rLgpPciBkbyBpIG5vdCB1 bmRlc3RhbmQgc29tZXRoaW5nIGhlcmUgPyBDb3VsZCBzb21lb25lIHBsZWFzZSBjbGFyaWZ5LgoK SSBjb3VsZCBiZSB3cm9uZyBidXQgaSB0aGluayBpdCBzaG91bGQgYmUgbGlrZSB0aGlzOgoKY3B1 aWQgPSBQQ1BVX0dFVChjcHVpZCk7CnNhdmVjdHgoJnN0b3BwY2JzW2NwdWlkXSk7CnNhdmVjdHgo UENQVV9HRVQoY3VycGNiKSk7CkNQVV9DTFJfQVRPTUlDKGNwdWlkLCAmc3RhcnRlZF9jcHVzKTvC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA8LS0tLS0tLS0tLS0tLSBtb3ZlIGhlcmUKQ1BVX1NF VF9BVE9NSUMoY3B1aWQsICZzdG9wcGVkX2NwdXMpOwp3aGlsZSAoIUNQVV9JU1NFVChjcHVpZCwg JnN0YXJ0ZWRfY3B1cykpCsKgwqDCoCBjcHVfc3BpbndhaXQoKTsKQ1BVX0NMUl9BVE9NSUMoY3B1 aWQsICZzdG9wcGVkX2NwdXMpOwoKVGhhbmtzCgpSZWdhcmRzCgoKCg== From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 26 19:29:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D729E106566B for ; Mon, 26 Dec 2011 19:29:01 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id A13C88FC12 for ; Mon, 26 Dec 2011 19:29:01 +0000 (UTC) Received: (qmail 2314 invoked from network); 26 Dec 2011 19:02:19 -0000 Received: from unknown (HELO ?10.50.50.212?) (spawk@64.147.100.2) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 26 Dec 2011 19:02:19 -0000 Message-ID: <4EF8C4B1.5050308@acm.poly.edu> Date: Mon, 26 Dec 2011 14:02:09 -0500 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20111222 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Getting swapped-out memory per process X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 19:29:01 -0000 Hi. Is there a way, from userspace, to get the amount of memory a given process currently has swapped out? -Boris From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 26 20:12:06 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EDBF106564A for ; Mon, 26 Dec 2011 20:12:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id BCD3C8FC15 for ; Mon, 26 Dec 2011 20:12:04 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBQKBt1n095078 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 26 Dec 2011 22:11:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBQKBtZ0001508; Mon, 26 Dec 2011 22:11:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBQKBsta001507; Mon, 26 Dec 2011 22:11:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 26 Dec 2011 22:11:53 +0200 From: Kostik Belousov To: Boris Kochergin Message-ID: <20111226201153.GQ50300@deviant.kiev.zoral.com.ua> References: <4EF8C4B1.5050308@acm.poly.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uXC0L8K1G2d8ohZ2" Content-Disposition: inline In-Reply-To: <4EF8C4B1.5050308@acm.poly.edu> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Getting swapped-out memory per process X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 20:12:06 -0000 --uXC0L8K1G2d8ohZ2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 26, 2011 at 02:02:09PM -0500, Boris Kochergin wrote: > Hi. >=20 > Is there a way, from userspace, to get the amount of memory a given=20 > process currently has swapped out? The VM does not track memory 'per process'. Simplifying to the point where the statement becomes false, it assigns the memory pages to the vm objects, and allows to map objects into process address space. Pageout works on the page by page basis, regardless of the page ownership (for the normal pages, using some definition of normal). Since one page can and often is mapped into several address spaces, accounting on the per-process is meaningless or causes the stress of the imagination of somebody who defines the accounting policy. --uXC0L8K1G2d8ohZ2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk741QgACgkQC3+MBN1Mb4gZjgCePktiCpgLoDSFtaoZZdaSPwOK 1+4An1xgdW17YNzSJ/NCpZkcEj75/rpp =SL/q -----END PGP SIGNATURE----- --uXC0L8K1G2d8ohZ2-- From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 26 20:42:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79063106564A for ; Mon, 26 Dec 2011 20:42:03 +0000 (UTC) (envelope-from me@acm.jhu.edu) Received: from centaur.acm.jhu.edu (batman.acm.jhu.edu [128.220.251.35]) by mx1.freebsd.org (Postfix) with ESMTP id 4DE4C8FC12 for ; Mon, 26 Dec 2011 20:42:03 +0000 (UTC) Received: from centaur.acm.jhu.edu (localhost.localdomain [127.0.0.1]) by centaur.acm.jhu.edu (Postfix) with ESMTP id 0DF8C1F60B0C for ; Mon, 26 Dec 2011 15:24:15 -0500 (EST) Received: by centaur.acm.jhu.edu (Postfix, from userid 1003) id CF3CD1F60AD2; Mon, 26 Dec 2011 15:24:14 -0500 (EST) Date: Mon, 26 Dec 2011 15:24:14 -0500 From: Venkatesh Srinivas To: freebsd-hackers@freebsd.org Message-ID: <20111226202414.GA18713@centaur.acm.jhu.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-08-17) X-Virus-Scanned: ClamAV using ClamSMTP X-Mailman-Approved-At: Tue, 27 Dec 2011 01:37:38 +0000 Subject: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 20:42:03 -0000 Hi! I've been playing with two things in DragonFly that might be of interest here. Thing #1 := First, per-mountpoint syncer threads. Currently there is a single thread, 'syncer', which periodically calls fsync() on dirty vnodes from every mount, along with calling vfs_sync() on each filesystem itself (via syncer vnodes). My patch modifies this to create syncer threads for mounts that request it. For these mounts, vnodes are synced from their mount-specific thread rather than the global syncer. The idea is that periodic fsync/sync operations from one filesystem should not stall or delay synchronization for other ones. The patch was fairly simple: http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1efc595db0db397b4365f08b640 There's certainly more that could be done in this direction -- the current patch does preserve a global syncer ('syncer0') for unflagged filesystems and for running the rushjobs logic from speedup_syncer. And the current patch preserves the notion of syncer vnodes, which are entirely overkill when there are per-mount sync threads. But its a start and something very similar could apply to FreeBSD. Thing #2 := Currently when pagedaemon decides to launder a dirty page, it initiates I/O for the launder from within its own thread context. While the I/O is generally asynchronous, the call path to get there from pagedaemon is deep and fraught with stall points: (for vnode_pager, possible stalls annotated) pagedaemon scans -> ... vm_pageout_clean -> [block on vm_object locks, page busy] vm_pageout_flush -> vnode_pager_putpages -> vnode_generic_putpages -> _write -> [block on FS locks] b(,a,d)write -> [wait on runningbufspace] _stratgy -> Oh my... While any part of this path is stalled, pagedaemon is not continuing to do its job; this could be a problem -- so long as it is not laundering pages, we are not resolving any page shortages. Given Thing #1, we have per-mountpoint service threads; I think it'd be worth pushing out the deeper parts of this callpath into those threads. The idea is that pagedaemon would select and cluster pages as it does now, but would use the syncer threads to walk through the pager and FS layer. An added benefit of using the syncer threads is that contention between fsync/vfs_sync on an FS and pageout on that same FS would be excluded. The pagedaemon would not wait for the I/O to initiate before continuing to scan more candidates. I've not found an ideal place to break up this callchain, but either between vm_pageout_clean / vm_pageout_flush, or at the entry to the vnode_pager would be good places. In experiments, I've sent the vm_pageout_flush calls off to a convenient taskqueue, seems to work okay. But sending them to per-mount threads would be better. Any thoughts on either of these things? Hope this was interesting, --vs; From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 07:11:46 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FFCC106566B for ; Tue, 27 Dec 2011 07:11:46 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id 5CC868FC08 for ; Tue, 27 Dec 2011 07:11:46 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id pBR6eBNj063980; Mon, 26 Dec 2011 22:40:16 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201112270640.pBR6eBNj063980@gw.catspoiler.org> Date: Mon, 26 Dec 2011 22:40:11 -0800 (PST) From: Don Lewis To: vsrinivas@dragonflybsd.org In-Reply-To: <20111226202414.GA18713@centaur.acm.jhu.edu> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-hackers@FreeBSD.org Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 07:11:46 -0000 On 26 Dec, Venkatesh Srinivas wrote: > Hi! > > I've been playing with two things in DragonFly that might be of interest here. > > Thing #1 := > > First, per-mountpoint syncer threads. Currently there is a single thread, > 'syncer', which periodically calls fsync() on dirty vnodes from every mount, > along with calling vfs_sync() on each filesystem itself (via syncer vnodes). > > My patch modifies this to create syncer threads for mounts that request it. > For these mounts, vnodes are synced from their mount-specific thread rather > than the global syncer. > > The idea is that periodic fsync/sync operations from one filesystem should not > stall or delay synchronization for other ones. > > The patch was fairly simple: > http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1efc595db0db397b4365f08b640 > > There's certainly more that could be done in this direction -- the current patch > does preserve a global syncer ('syncer0') for unflagged filesystems and for > running the rushjobs logic from speedup_syncer. And the current patch preserves > the notion of syncer vnodes, which are entirely overkill when there are > per-mount sync threads. But its a start and something very similar could apply > to FreeBSD. I used to think that something like this was a good idea, but the first issue that I thought of was that this could cause excessive seeking if multiple threads were attempting to sync vnodes for multiple partitions on the same physical device at the same time. What might be better is one thread per physical device, possibly doing a simple elevator sort based on partition number and inode number on the items in each worklist bucket. It might still be possible to retain the advantages of this with a one thread per mount point implementation by adding interlocks (or even just start time offsets) so that they don't all try to run at once and fight over the head actuator. Implementing one thread per mount point does have the advantage of making it easy to observe which mount points are "busy". The next complication is that all of the different ways that we have to slice, dice, and combine storage devices (various forms of RAID, ZFS pools, etc.) make the concept of a device a lot more complicated. How do we optimize? Should we even try? One of the things that you didn't mention about syncer vnodes is all of the nastyness that goes on inside ffs_sync() and friends every time the syncer gets to the syncer vnode. That causes a big burst of I/O every 30 seconds. > Thing #2 := > > Currently when pagedaemon decides to launder a dirty page, it initiates I/O > for the launder from within its own thread context. While the I/O is generally > asynchronous, the call path to get there from pagedaemon is deep and fraught > with stall points: (for vnode_pager, possible stalls annotated) > > pagedaemon scans -> > ... > vm_pageout_clean -> [block on vm_object locks, > page busy] > vm_pageout_flush -> > vnode_pager_putpages -> > vnode_generic_putpages -> > _write -> [block on FS locks] > b(,a,d)write -> [wait on runningbufspace] > _stratgy -> > Oh my... > > While any part of this path is stalled, pagedaemon is not continuing to do its > job; this could be a problem -- so long as it is not laundering pages, we are > not resolving any page shortages. > > Given Thing #1, we have per-mountpoint service threads; I think it'd be worth > pushing out the deeper parts of this callpath into those threads. The idea is > that pagedaemon would select and cluster pages as it does now, but would use > the syncer threads to walk through the pager and FS layer. An added benefit > of using the syncer threads is that contention between fsync/vfs_sync on an > FS and pageout on that same FS would be excluded. The pagedaemon would not > wait for the I/O to initiate before continuing to scan more candidates. > > I've not found an ideal place to break up this callchain, but either between > vm_pageout_clean / vm_pageout_flush, or at the entry to the vnode_pager would > be good places. In experiments, I've sent the vm_pageout_flush calls off to > a convenient taskqueue, seems to work okay. But sending them to per-mount > threads would be better. The current implementation definitely has the flaws that you mention. I remember system deadlocks in years past. Your idea for a fix looks interesting. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 11:33:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C10F3106566B for ; Tue, 27 Dec 2011 11:33:03 +0000 (UTC) (envelope-from geoffrey.levand@mail.ru) Received: from f155.mail.ru (f155.mail.ru [217.69.128.109]) by mx1.freebsd.org (Postfix) with ESMTP id 3CEB08FC0C for ; Tue, 27 Dec 2011 11:33:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail; h=Message-Id:Content-Transfer-Encoding:Content-Type:Reply-To:In-Reply-To:References:Date:Mime-Version:Subject:Cc:To:From; bh=4dK27YPmlgYDm1IAdZt0N+f/99zUBkVT7flsVy/8f+k=; b=c0ukEXf1yvtROCZSIpEIYk5JeFna9IOjBeH4joiFd9waHtFV2QWzh2rZ5aG8PDCKH/Miy6T7bFOAdIQkudRoEuvrEUyppnMtbW122LolCGsBKQ75LV6m2qVCdGI8yMc+; Received: from mail by f155.mail.ru with local id 1RfVHB-00053s-00; Tue, 27 Dec 2011 15:33:01 +0400 Received: from [46.5.58.225] by e.mail.ru with HTTP; Tue, 27 Dec 2011 15:33:01 +0400 From: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= To: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [46.5.58.225] Date: Tue, 27 Dec 2011 15:33:01 +0400 References: In-Reply-To: X-Priority: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Message-Id: X-Spam: Not detected X-Mras: Ok X-Mailman-Approved-At: Tue, 27 Dec 2011 12:15:57 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: SMP: stopping/restarting CPUs on powerpc arch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 11:33:03 -0000 TmV2ZXJtaW5kLiBJIGNoYW5nZWQgc29tZXRoaW5nIGluIElQSSBoYW5kbGluZyBhbmQgdGhhdCBj YXVzZWQgdGhpcyBwcm9ibGVtLiBOb3cgdGhlIENQVXMgY2FuIGJlIHN0b3BwZWQgYnV0IEtEQiBq dXN0IGhhbmdzCmFmdGVyIGl0IHdhcyBlbnRlcmVkLiBUaHJlYWQgSUQgYW5kIHByb2Nlc3MgSUQg YXJlIHByaW50ZWQgb24gdGhlIGNvbnNvbGUgYnV0IHRoYXQncyBpdC4gQWZ0ZXIgdGhhdCBub3Ro aW5nIHdvcmtzIGFueW1vcmUuCgoKMjYg0LTQtdC60LDQsdGA0Y8gMjAxMSwgMDY6MjQg0L7RgiBn ZW9mZnJleSBsZXZhbmQgPGdlb2ZmcmV5LmxldmFuZEBtYWlsLnJ1PjoKPiAKPiBIaSwKPiAKPiBp IHdhcyBpbXBsZW1lbnRpbmcgYW5kIHRlc3RpbmcgY29yZSBkdW1wIGZlYXR1cmUgb2YgSEREIGRy aXZlciBvbiBQUzMgcG93ZXJwYyBhcmNoIGFuZAo+IHRyaWVkIHRvIGVudGVyIEtEQiBtYW51YWxs eSB3aXRoIHN5c2N0bCBkZWJ1Zy5rZGIucGFuaWM9MS4KPiBBbmQgaSdtIGdldHRpbmcgdGhlIGZv bGxvd2luZyBtZXNzYWdlIGZyb20gS0RCOiB0aW1lb3V0IHN0b3BwaW5nIGNwdXMuCj4gCj4gSSB0 b29rIGEgbG9vayBhdCBnZW5lcmljX3N0b3BfY3B1cyB3aGljaCBzZW5kcyBJUEkgdG8gZXZlcnkg Q1BVLgo+IE9uIHBvd2VycGMsIElQSV9TVE9QIGlzIGltcGxlbWVudGVkIHRoZSBmb2xsb3dpbmcg d2F5Ogo+IAo+IGNwdWlkID0gUENQVV9HRVQoY3B1aWQpOwo+IHNhdmVjdHgoJnN0b3BwY2JzW2Nw dWlkXSk7Cj4gc2F2ZWN0eChQQ1BVX0dFVChjdXJwY2IpKTsKPiBDUFVfU0VUX0FUT01JQyhjcHVp ZCwgJnN0b3BwZWRfY3B1cyk7Cj4gd2hpbGUgKCFDUFVfSVNTRVQoY3B1aWQsICZzdGFydGVkX2Nw dXMpKQo+IMKgwqDCoCBjcHVfc3BpbndhaXQoKTsKPiBDUFVfQ0xSX0FUT01JQyhjcHVpZCwgJnN0 b3BwZWRfY3B1cyk7Cj4gQ1BVX0NMUl9BVE9NSUMoY3B1aWQsICZzdGFydGVkX2NwdXMpOwo+IAo+ IGdlbmVyaWNfc3RvcF9jcHVzIHdhaXRzIHVudGlsIGFsbCBDUFVzIGNsZWFyIGl0cyBJRCBpbiBz dG9wcGVkX2NwdXMgYml0bWFwLgo+IElmIHlvdSB0YWtlIGEgbG9vayBhdCB0aGUgcHJldmlvdXMg c25pcHBldCBvZiBjb2RlIHRoZW4gdGhlIGJpdCBpcyBpbmRlZWQgc2V0IGJ1dCB0aGVuIGNsZWFy ZWQgaW1tZWRpYXRlbHkuCj4gQW5kIHRoYXQgaXMgd2h5IGkgZ2V0IHRpbWVvdXQgbWVzc2FnZSBp biBrZGIgaSB0aGluay4KPiBPciBkbyBpIG5vdCB1bmRlc3RhbmQgc29tZXRoaW5nIGhlcmUgPyBD b3VsZCBzb21lb25lIHBsZWFzZSBjbGFyaWZ5Lgo+IAo+IEkgY291bGQgYmUgd3JvbmcgYnV0IGkg dGhpbmsgaXQgc2hvdWxkIGJlIGxpa2UgdGhpczoKPiAKPiBjcHVpZCA9IFBDUFVfR0VUKGNwdWlk KTsKPiBzYXZlY3R4KCZzdG9wcGNic1tjcHVpZF0pOwo+IHNhdmVjdHgoUENQVV9HRVQoY3VycGNi KSk7Cj4gQ1BVX0NMUl9BVE9NSUMoY3B1aWQsICZzdGFydGVkX2NwdXMpO8KgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIDwtLS0tLS0tLS0tLS0tIG1vdmUgaGVyZQo+IENQVV9TRVRfQVRPTUlDKGNw dWlkLCAmc3RvcHBlZF9jcHVzKTsKPiB3aGlsZSAoIUNQVV9JU1NFVChjcHVpZCwgJnN0YXJ0ZWRf Y3B1cykpCj4gwqDCoMKgIGNwdV9zcGlud2FpdCgpOwo+IENQVV9DTFJfQVRPTUlDKGNwdWlkLCAm c3RvcHBlZF9jcHVzKTsKPiAKPiBUaGFua3MKPiAKPiBSZWdhcmRzCj4gCj4g From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 16:00:19 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B503D106566B for ; Tue, 27 Dec 2011 16:00:18 +0000 (UTC) (envelope-from giovanni.trematerra@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3B44B8FC08 for ; Tue, 27 Dec 2011 16:00:17 +0000 (UTC) Received: by qcse13 with SMTP id e13so10565605qcs.13 for ; Tue, 27 Dec 2011 08:00:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uywAWe5nTNLf2Pg7n4JY4rx3ym1q/TcRg+wSuHOIjLc=; b=Qo7dbDEbqSwAsKKKtzg0ZUGeYpmyXXrEMaNykIp1mqGkDAWb9EhrTUgMfjT8Em+2dq eKvziOUUbcqDUyBaO7MG9P2NXanwpGhs6uBW+kyLW4x2gJZU0sNMHztl4yVZHRUpyaZO 3cYfLlLykjqi+HtCuFN4MTXddLvZ/KhGKmv0w= MIME-Version: 1.0 Received: by 10.229.134.197 with SMTP id k5mr10163694qct.58.1325000284984; Tue, 27 Dec 2011 07:38:04 -0800 (PST) Received: by 10.229.185.82 with HTTP; Tue, 27 Dec 2011 07:38:04 -0800 (PST) In-Reply-To: <20111226202414.GA18713@centaur.acm.jhu.edu> References: <20111226202414.GA18713@centaur.acm.jhu.edu> Date: Tue, 27 Dec 2011 16:38:04 +0100 Message-ID: From: Giovanni Trematerra To: Venkatesh Srinivas Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 16:00:19 -0000 On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas wrote: > Hi! > > I've been playing with two things in DragonFly that might be of interest > here. > > Thing #1 := > > First, per-mountpoint syncer threads. Currently there is a single thread, > 'syncer', which periodically calls fsync() on dirty vnodes from every mount, > along with calling vfs_sync() on each filesystem itself (via syncer vnodes). > > My patch modifies this to create syncer threads for mounts that request it. > For these mounts, vnodes are synced from their mount-specific thread rather > than the global syncer. > > The idea is that periodic fsync/sync operations from one filesystem should > not > stall or delay synchronization for other ones. > The patch was fairly simple: > http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1efc595db0db397b4365f08b640 > There's something WIP by attilio@ on that area. you might want to take a look at http://people.freebsd.org/~attilio/syncer_alpha_15.diff I don't know what hammerfs needs but UFS/FFS and buffer cache make a good job performance-wise and so the authors are skeptical about the boost that such a change can give. We believe that brain cycles need to be spent on other pieces of the system such as ARC and ZFS. -- Gianni From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 16:34:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B16771065675 for ; Tue, 27 Dec 2011 16:34:13 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 46DDF8FC20 for ; Tue, 27 Dec 2011 16:34:12 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so21167442wgb.31 for ; Tue, 27 Dec 2011 08:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=S5YJvPz8XQuWt0ASru+cu2iJxXtznvl4Zq2LvvI9Qew=; b=LAS+5R4XKKw7paPZmymnEPZ3C0ZJuO2qQ1gYQBZU21JMYatGo9PDgpde16dZVKb4pH NYxpnKtuY+5BRLX2mzvr958N2rYZFal1rvlSec6WMjFa1/HxjfijiEpPMQCzmGD13uUG ABygNtCLKzo2OoHt0BsD8jj/BzR/dgv4YeXQo= MIME-Version: 1.0 Received: by 10.227.207.15 with SMTP id fw15mr28245083wbb.15.1325001904453; Tue, 27 Dec 2011 08:05:04 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.216.18.130 with HTTP; Tue, 27 Dec 2011 08:05:04 -0800 (PST) In-Reply-To: References: <20111226202414.GA18713@centaur.acm.jhu.edu> Date: Tue, 27 Dec 2011 17:05:04 +0100 X-Google-Sender-Auth: Nw4eTCV6-6YleDvNPiSFG4EVqQQ Message-ID: From: Attilio Rao To: Giovanni Trematerra Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Venkatesh Srinivas Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 16:34:13 -0000 2011/12/27 Giovanni Trematerra : > On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas > wrote: >> Hi! >> >> I've been playing with two things in DragonFly that might be of interest >> here. >> >> Thing #1 := >> >> First, per-mountpoint syncer threads. Currently there is a single thread, >> 'syncer', which periodically calls fsync() on dirty vnodes from every mount, >> along with calling vfs_sync() on each filesystem itself (via syncer vnodes). >> >> My patch modifies this to create syncer threads for mounts that request it. >> For these mounts, vnodes are synced from their mount-specific thread rather >> than the global syncer. >> >> The idea is that periodic fsync/sync operations from one filesystem should >> not >> stall or delay synchronization for other ones. >> The patch was fairly simple: >> http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1efc595db0db397b4365f08b640 >> > > There's something WIP by attilio@ on that area. > you might want to take a look at > http://people.freebsd.org/~attilio/syncer_alpha_15.diff > > I don't know what hammerfs needs but UFS/FFS and buffer cache make a good > job performance-wise and so the authors are skeptical about the boost that such > a change can give. We believe that brain cycles need to be spent on > other pieces of the system such as ARC and ZFS. More specifically, it is likely that focusing on UFS and buffer cache for performance is not really useful, we should drive our efforts over ARC and ZFS. Also, the real bottlenecks in our I/O paths are in GEOM single-threaded design, lack of unmapped I/O functionality, possibly lack of proritized I/O, etc. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 16:58:08 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9A78106566B; Tue, 27 Dec 2011 16:58:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 57E788FC14; Tue, 27 Dec 2011 16:58:06 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBRGvudk010298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 27 Dec 2011 18:57:56 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBRGvuxZ004948; Tue, 27 Dec 2011 18:57:56 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBRGvubR004947; Tue, 27 Dec 2011 18:57:56 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 27 Dec 2011 18:57:56 +0200 From: Kostik Belousov To: Attilio Rao Message-ID: <20111227165755.GT50300@deviant.kiev.zoral.com.ua> References: <20111226202414.GA18713@centaur.acm.jhu.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5+cpaZGbq3HGGD8f" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Giovanni Trematerra , Venkatesh Srinivas Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 16:58:08 -0000 --5+cpaZGbq3HGGD8f Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Dec 27, 2011 at 05:05:04PM +0100, Attilio Rao wrote: > 2011/12/27 Giovanni Trematerra : > > On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas > > wrote: > >> Hi! > >> > >> I've been playing with two things in DragonFly that might be of intere= st > >> here. > >> > >> Thing #1 :=3D > >> > >> First, per-mountpoint syncer threads. Currently there is a single thre= ad, > >> 'syncer', which periodically calls fsync() on dirty vnodes from every = mount, > >> along with calling vfs_sync() on each filesystem itself (via syncer vn= odes). > >> > >> My patch modifies this to create syncer threads for mounts that reques= t it. > >> For these mounts, vnodes are synced from their mount-specific thread r= ather > >> than the global syncer. > >> > >> The idea is that periodic fsync/sync operations from one filesystem sh= ould > >> not > >> stall or delay synchronization for other ones. > >> The patch was fairly simple: > >> http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1= efc595db0db397b4365f08b640 > >> > > > > There's something WIP by attilio@ on that area. > > you might want to take a look at > > http://people.freebsd.org/~attilio/syncer_alpha_15.diff > > > > I don't know what hammerfs needs but UFS/FFS and buffer cache make a go= od > > job performance-wise and so the authors are skeptical about the boost t= hat such > > a change can give. We believe that brain cycles need to be spent on > > other pieces of the system such as ARC and ZFS. >=20 > More specifically, it is likely that focusing on UFS and buffer cache > for performance is not really useful, we should drive our efforts over > ARC and ZFS. > Also, the real bottlenecks in our I/O paths are in GEOM > single-threaded design, lack of unmapped I/O functionality, possibly > lack of proritized I/O, etc. Even if not useful for performance (this is possible), the change itself is useful to provide better system behaviour in the case of failure. E.g., slowly-responding or wedged NFS server, dying disk etc would more limited impact with the patch then without it. It will not completely solve the issue, since e.g. dirty buffers amount is not limited per-mount point, only globally. But at least it covers significant part of the problem. Also, it should help with interactivity and load pikes at 30sec interval. I remember that I had no major objections when I read the patch. I personal= ly would prefer to have it committed. --5+cpaZGbq3HGGD8f Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk75+RMACgkQC3+MBN1Mb4jJzACg0h7Lp23ry6srdSQa0bgVRc7V PVgAn0WKCAn9kNrdPmkqlEtn/iK7u26U =ZvUp -----END PGP SIGNATURE----- --5+cpaZGbq3HGGD8f-- From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 16:59:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59940106566B for ; Tue, 27 Dec 2011 16:59:43 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 27A388FC16 for ; Tue, 27 Dec 2011 16:59:42 +0000 (UTC) Received: by dakp5 with SMTP id p5so11049605dak.13 for ; Tue, 27 Dec 2011 08:59:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=m8dYOfuPXeqPLxkyu/bxTeisTCG+slcvyCKey+au6Xk=; b=aYGiE20jHXML2jeD5cE3GdKqYVUtRk4zc2x9Kwim7RJeWZDaooeRi1yRPG9zg0kgLg z0nX9J80tSyTnLY8FCxoReqMVpkFmgQDFpzRn+5eOeNYyTGv00rL3ua14EBLiJ25rwyS Qx0SfpoapeQ0Ww3ricI3wDP7EnnSxC4YpoEu4= MIME-Version: 1.0 Received: by 10.68.209.68 with SMTP id mk4mr37859709pbc.88.1325005181231; Tue, 27 Dec 2011 08:59:41 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.208.167 with HTTP; Tue, 27 Dec 2011 08:59:41 -0800 (PST) In-Reply-To: References: <20111226202414.GA18713@centaur.acm.jhu.edu> Date: Tue, 27 Dec 2011 08:59:41 -0800 X-Google-Sender-Auth: baA_8D0tIZK-gwn7ljmtoqfJofM Message-ID: From: mdf@FreeBSD.org To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Giovanni Trematerra , Venkatesh Srinivas Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 16:59:43 -0000 On Tue, Dec 27, 2011 at 8:05 AM, Attilio Rao wrote: > 2011/12/27 Giovanni Trematerra : >> On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas >> wrote: >>> Hi! >>> >>> I've been playing with two things in DragonFly that might be of interest >>> here. >>> >>> Thing #1 := >>> >>> First, per-mountpoint syncer threads. Currently there is a single thread, >>> 'syncer', which periodically calls fsync() on dirty vnodes from every mount, >>> along with calling vfs_sync() on each filesystem itself (via syncer vnodes). >>> >>> My patch modifies this to create syncer threads for mounts that request it. >>> For these mounts, vnodes are synced from their mount-specific thread rather >>> than the global syncer. >>> >>> The idea is that periodic fsync/sync operations from one filesystem should >>> not >>> stall or delay synchronization for other ones. >>> The patch was fairly simple: >>> http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1efc595db0db397b4365f08b640 >>> >> >> There's something WIP by attilio@ on that area. >> you might want to take a look at >> http://people.freebsd.org/~attilio/syncer_alpha_15.diff >> >> I don't know what hammerfs needs but UFS/FFS and buffer cache make a good >> job performance-wise and so the authors are skeptical about the boost that such >> a change can give. We believe that brain cycles need to be spent on >> other pieces of the system such as ARC and ZFS. > > More specifically, it is likely that focusing on UFS and buffer cache > for performance is not really useful, we should drive our efforts over > ARC and ZFS. > Also, the real bottlenecks in our I/O paths are in GEOM > single-threaded design, lack of unmapped I/O functionality, possibly > lack of proritized I/O, etc. Indeed, Isilon (and probably other vendors as well) entirely skip VFS_SYNC when the WAIT argument is MNT_LAZY. Since we're a distributed journalled filesystem, syncing via a system thread is not a relevant operation; i.e. all writes that have exited a VOP_WRITE or similar operation are already in reasonably stable storage in a journal on the relevant nodes. However, we do then have our own threads running on each node to flush the journal regularly (in addition to when it fills up), and I don't know enough about this to know if it could be fit into the syncer thread idea or if it's too tied in somehow to our architecture. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 17:05:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92BBB106566B; Tue, 27 Dec 2011 17:05:49 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id EB9158FC0C; Tue, 27 Dec 2011 17:05:45 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so21198510wgb.31 for ; Tue, 27 Dec 2011 09:05:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=M/KufI56rlxkDHPM5O5GtePaan/LJTcvGNd/EWk48l0=; b=F1EA7nPc1v+RTRHy90Y/vwaElIs7DucxIOYlgvRg52FbZozqaBPNX7bpytUFOHD7qD Xxu9KAj+qclZWmhcLMmOuDGdcEqRlZmqJyl59dY8Z3GsBpxRNAltNkrS+LEvId2Dr7qr D5DF1Hug+Kcls0qKK3akGZp6a33UmzaHLUOAA= MIME-Version: 1.0 Received: by 10.227.207.15 with SMTP id fw15mr28428072wbb.15.1325005544447; Tue, 27 Dec 2011 09:05:44 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.216.18.130 with HTTP; Tue, 27 Dec 2011 09:05:44 -0800 (PST) In-Reply-To: References: <20111226202414.GA18713@centaur.acm.jhu.edu> Date: Tue, 27 Dec 2011 18:05:44 +0100 X-Google-Sender-Auth: utPQqJ-hIoEkTFmidEiIxdWWszo Message-ID: From: Attilio Rao To: mdf@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Giovanni Trematerra , Venkatesh Srinivas Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 17:05:49 -0000 2011/12/27 : > On Tue, Dec 27, 2011 at 8:05 AM, Attilio Rao wrote: >> 2011/12/27 Giovanni Trematerra : >>> On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas >>> wrote: >>>> Hi! >>>> >>>> I've been playing with two things in DragonFly that might be of intere= st >>>> here. >>>> >>>> Thing #1 :=3D >>>> >>>> First, per-mountpoint syncer threads. Currently there is a single thre= ad, >>>> 'syncer', which periodically calls fsync() on dirty vnodes from every = mount, >>>> along with calling vfs_sync() on each filesystem itself (via syncer vn= odes). >>>> >>>> My patch modifies this to create syncer threads for mounts that reques= t it. >>>> For these mounts, vnodes are synced from their mount-specific thread r= ather >>>> than the global syncer. >>>> >>>> The idea is that periodic fsync/sync operations from one filesystem sh= ould >>>> not >>>> stall or delay synchronization for other ones. >>>> The patch was fairly simple: >>>> http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e1= efc595db0db397b4365f08b640 >>>> >>> >>> There's something WIP by attilio@ on that area. >>> you might want to take a look at >>> http://people.freebsd.org/~attilio/syncer_alpha_15.diff >>> >>> I don't know what hammerfs needs but UFS/FFS and buffer cache make a go= od >>> job performance-wise and so the authors are skeptical about the boost t= hat such >>> a change can give. We believe that brain cycles need to be spent on >>> other pieces of the system such as ARC and ZFS. >> >> More specifically, it is likely that focusing on UFS and buffer cache >> for performance is not really useful, we should drive our efforts over >> ARC and ZFS. >> Also, the real bottlenecks in our I/O paths are in GEOM >> single-threaded design, lack of unmapped I/O functionality, possibly >> lack of proritized I/O, etc. > > Indeed, Isilon (and probably other vendors as well) entirely skip > VFS_SYNC when the WAIT argument is MNT_LAZY. =C2=A0Since we're a > distributed journalled filesystem, syncing via a system thread is not > a relevant operation; i.e. all writes that have exited a VOP_WRITE or > similar operation are already in reasonably stable storage in a > journal on the relevant nodes. > > However, we do then have our own threads running on each node to flush > the journal regularly (in addition to when it fills up), and I don't > know enough about this to know if it could be fit into the syncer > thread idea or if it's too tied in somehow to our architecture. I'm not really sure how does journaling is implemented on OneFS, but when I made this patch SU+J wasn't yet there. Also, this patch just adds the infrastructure for a multithreaded and configurable syncer, which means it still requires the UFS bits for skipping the "double-syncing" (alias the MNT_LAZY skippage you mentioned). Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 27 17:22:07 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 043591065670; Tue, 27 Dec 2011 17:22:07 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id C42AF8FC12; Tue, 27 Dec 2011 17:22:06 +0000 (UTC) Received: by dakp5 with SMTP id p5so11062580dak.13 for ; Tue, 27 Dec 2011 09:22:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=k4jsR63zRpPmq8Dg8kJwA37Czdqs4/Y1R6RauuSAkG0=; b=TnAupdvWJMsoSewOEfiVQ7r31vjRpAlpRthqn4YAsB66Puc5gqlrV2pEim7kb2iQ7R 2W0Fe/xPbtYlL+NhQ46YtYIB3VgyPiDZ2ZmfvZ8Zf0eCiPTJoWVCT0fB9lK2VEwBs7OP ugZDrk4QYtbVubFrUChbH1qkpLBhwbx8N89Oo= MIME-Version: 1.0 Received: by 10.68.196.169 with SMTP id in9mr66821744pbc.54.1325006526270; Tue, 27 Dec 2011 09:22:06 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.208.167 with HTTP; Tue, 27 Dec 2011 09:22:06 -0800 (PST) In-Reply-To: References: <20111226202414.GA18713@centaur.acm.jhu.edu> Date: Tue, 27 Dec 2011 09:22:06 -0800 X-Google-Sender-Auth: _q_1c9XGB-k320kiaTuAIqfo-5E Message-ID: From: mdf@FreeBSD.org To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Giovanni Trematerra , Venkatesh Srinivas Subject: Re: Per-mount syncer threads and fanout for pagedaemon cleaning X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 17:22:07 -0000 On Tue, Dec 27, 2011 at 9:05 AM, Attilio Rao wrote: > 2011/12/27 =A0: >> On Tue, Dec 27, 2011 at 8:05 AM, Attilio Rao wrote= : >>> 2011/12/27 Giovanni Trematerra : >>>> On Mon, Dec 26, 2011 at 9:24 PM, Venkatesh Srinivas >>>> wrote: >>>>> Hi! >>>>> >>>>> I've been playing with two things in DragonFly that might be of inter= est >>>>> here. >>>>> >>>>> Thing #1 :=3D >>>>> >>>>> First, per-mountpoint syncer threads. Currently there is a single thr= ead, >>>>> 'syncer', which periodically calls fsync() on dirty vnodes from every= mount, >>>>> along with calling vfs_sync() on each filesystem itself (via syncer v= nodes). >>>>> >>>>> My patch modifies this to create syncer threads for mounts that reque= st it. >>>>> For these mounts, vnodes are synced from their mount-specific thread = rather >>>>> than the global syncer. >>>>> >>>>> The idea is that periodic fsync/sync operations from one filesystem s= hould >>>>> not >>>>> stall or delay synchronization for other ones. >>>>> The patch was fairly simple: >>>>> http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/50e4012a4b55e= 1efc595db0db397b4365f08b640 >>>>> >>>> >>>> There's something WIP by attilio@ on that area. >>>> you might want to take a look at >>>> http://people.freebsd.org/~attilio/syncer_alpha_15.diff >>>> >>>> I don't know what hammerfs needs but UFS/FFS and buffer cache make a g= ood >>>> job performance-wise and so the authors are skeptical about the boost = that such >>>> a change can give. We believe that brain cycles need to be spent on >>>> other pieces of the system such as ARC and ZFS. >>> >>> More specifically, it is likely that focusing on UFS and buffer cache >>> for performance is not really useful, we should drive our efforts over >>> ARC and ZFS. >>> Also, the real bottlenecks in our I/O paths are in GEOM >>> single-threaded design, lack of unmapped I/O functionality, possibly >>> lack of proritized I/O, etc. >> >> Indeed, Isilon (and probably other vendors as well) entirely skip >> VFS_SYNC when the WAIT argument is MNT_LAZY. =A0Since we're a >> distributed journalled filesystem, syncing via a system thread is not >> a relevant operation; i.e. all writes that have exited a VOP_WRITE or >> similar operation are already in reasonably stable storage in a >> journal on the relevant nodes. >> >> However, we do then have our own threads running on each node to flush >> the journal regularly (in addition to when it fills up), and I don't >> know enough about this to know if it could be fit into the syncer >> thread idea or if it's too tied in somehow to our architecture. > > I'm not really sure how does journaling is implemented on OneFS, but > when I made this patch SU+J wasn't yet there. > Also, this patch just adds the infrastructure for a multithreaded and > configurable syncer, which means it still requires the UFS bits for > skipping the "double-syncing" (alias the MNT_LAZY skippage you > mentioned). Right, I don't object to any changes relating to multiple sync threads, etc., just trying to offer a vendor viewpoint. Though having one thread per mount would allow for a different sync interval for each filesystem which can be of advantage. Right after I did Isilon's last FreeBSD merge (it seems like a long time ago now), I wanted to look into what it would take to eliminate our specialed journal flush thread (i.e. tie it into VFS_SYNC), but one objection was that then the flush interval would not be configurable separately from the one for our UFS partition. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 01:49:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 933C2106564A for ; Wed, 28 Dec 2011 01:49:41 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id 5539E8FC0A for ; Wed, 28 Dec 2011 01:49:41 +0000 (UTC) Received: from pps.filterd (ltcfislmsgpa04 [127.0.0.1]) by ltcfislmsgpa04.fnfis.com (8.14.4/8.14.4) with SMTP id pBS1I7i4021773; Tue, 27 Dec 2011 19:23:31 -0600 Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa04.fnfis.com with ESMTP id 11ynvpgp5c-2 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 27 Dec 2011 19:23:31 -0600 Received: from dtwin (10.14.152.15) by smtp.fisglobal.com (10.132.206.31) with Microsoft SMTP Server (TLS) id 14.1.323.3; Tue, 27 Dec 2011 19:23:30 -0600 From: Devin Teske To: Date: Tue, 27 Dec 2011 17:23:35 -0800 Message-ID: <02aa01ccc4ff$52db7380$f8925a80$@fisglobal.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_02AB_01CCC4BC.44B881A0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AczE/1BMfk5YjHQHRf2ZyhfbAskOxQ== Content-Language: en-us X-Originating-IP: [10.14.152.15] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.5.7110, 1.0.211, 0.0.0000 definitions=2011-12-27_08:2011-12-27, 2011-12-27, 1970-01-01 signatures=0 Cc: Garrett Cooper , devin.teske@fisglobal.com Subject: [PATCH] Fix kenv(1) output in w/respect to new boot loader variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 01:49:41 -0000 ------=_NextPart_000_02AB_01CCC4BC.44B881A0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Garrett Cooper and a few others have requested that I write a patch to fix a regression w/respect to kenv(1) output in FreeBSD-9.0 and HEAD. The issue is with the new boot loader menu. It adds many loader variables including ones that contain ANSI color escapes. Obviously, these ANSI codes don't play well with serial consoles when kenv(1) is executed without arguments (reports vary as to what happens, but it's never pretty). Attached is a patch to the Forth code that clears-out the menu-associated variables before invoking the kernel. The net-effect is that kenv(1) no longer reports menu-related variables. In essence, kenv(1) output should now appear the same as on RELENG_8 (which lacks the new boot loader and didn't use any such variables). Thus, restoring serial console glory. -- Devin _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ------=_NextPart_000_02AB_01CCC4BC.44B881A0 Content-Type: text/plain; name="loader_menu.20110825000410.patch.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="loader_menu.20110825000410.patch.txt" --- sys/boot/forth/menu.4th.orig Sat May 28 01:50:38 2011=0A= +++ sys/boot/forth/menu.4th Wed Aug 24 23:46:46 2011=0A= @@ -742,11 +742,10 @@=0A= else=0A= -rot 2drop=0A= =0A= - \ disable timeout if less than zero=0A= + \ boot immediately if less than zero=0A= dup 0< if=0A= drop=0A= - 0 menu_timeout_enabled !=0A= - 0 ( assigned to menu_timeout below )=0A= + 0 boot=0A= then=0A= then=0A= then=0A= --- sys/boot/forth/menu.4th.8.orig Sat May 28 01:50:38 2011=0A= +++ sys/boot/forth/menu.4th.8 Wed Aug 24 23:45:57 2011=0A= @@ -96,11 +96,15 @@=0A= by default) unless a key is pressed.=0A= If set to=0A= .Dq Li NO=0A= -(case-insensitive) or=0A= -.Dq Li -1 ,=0A= +(case-insensitive)=0A= .Ic menu-display=0A= will wait for user input and never execute=0A= .Ic menu_timeout_command .=0A= +If set to=0A= +.Dq Li -1 ,=0A= +.Ic menu-display=0A= +will boot immediately, preventing both interruption of the autoboot = process and=0A= +escaping to the loader prompt.=0A= Default is=0A= .Dq Li 10 .=0A= See=0A= ------=_NextPart_000_02AB_01CCC4BC.44B881A0-- From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 01:58:08 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FA45106566B for ; Wed, 28 Dec 2011 01:58:08 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id 510D18FC0C for ; Wed, 28 Dec 2011 01:58:07 +0000 (UTC) Received: from pps.filterd (ltcfislmsgpa02 [127.0.0.1]) by ltcfislmsgpa02.fnfis.com (8.14.4/8.14.4) with SMTP id pBS1QFWn007848; Tue, 27 Dec 2011 19:26:15 -0600 Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa02.fnfis.com with ESMTP id 11yrf5r79s-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 27 Dec 2011 19:26:14 -0600 Received: from dtwin (10.14.152.15) by smtp.fisglobal.com (10.132.206.31) with Microsoft SMTP Server (TLS) id 14.1.323.3; Tue, 27 Dec 2011 19:26:13 -0600 From: Devin Teske To: References: In-Reply-To: Date: Tue, 27 Dec 2011 17:26:19 -0800 Message-ID: <02b401ccc4ff$b475b7e0$1d6127a0$@fisglobal.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_02B5_01CCC4BC.A6529EF0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQIIKNfbqUsqtbcxt9/FRZOCMLBqJJV5rWrg Content-Language: en-us X-Originating-IP: [10.14.152.15] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.5.7110, 1.0.211, 0.0.0000 definitions=2011-12-27_08:2011-12-27, 2011-12-27, 1970-01-01 signatures=0 Cc: Garrett Cooper Subject: RE: [PATCH] Fix kenv(1) output in w/respect to new boot loader variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 01:58:08 -0000 ------=_NextPart_000_02B5_01CCC4BC.A6529EF0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit D'Oh! Attached wrong (OLD; already applied) patch. Please find appropriate patch attached! > -----Original Message----- > From: Devin Teske [mailto:devin.teske@fisglobal.com] > Sent: Tuesday, December 27, 2011 5:24 PM > To: 'freebsd-hackers@freebsd.org' > Cc: Garrett Cooper; devin.teske@fisglobal.com > Subject: [PATCH] Fix kenv(1) output in w/respect to new boot loader variables > > Garrett Cooper and a few others have requested that I write a patch to fix a > regression w/respect to kenv(1) output in FreeBSD-9.0 and HEAD. > > The issue is with the new boot loader menu. It adds many loader variables > including ones that contain ANSI color escapes. > > Obviously, these ANSI codes don't play well with serial consoles when kenv(1) is > executed without arguments (reports vary as to what happens, but it's never > pretty). > > Attached is a patch to the Forth code that clears-out the menu-associated > variables before invoking the kernel. > > The net-effect is that kenv(1) no longer reports menu-related variables. > > In essence, kenv(1) output should now appear the same as on RELENG_8 (which > lacks the new boot loader and didn't use any such variables). Thus, restoring > serial console glory. > -- > Devin _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ------=_NextPart_000_02B5_01CCC4BC.A6529EF0 Content-Type: text/plain; name="loader_menu.20111227163902.patch.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="loader_menu.20111227163902.patch.txt" --- src/sys/boot/forth/menu.4th.8.orig Tue Dec 27 11:36:25 2011=0A= +++ src/sys/boot/forth/menu.4th.8 Tue Dec 27 11:41:08 2011=0A= @@ -24,7 +24,7 @@=0A= .\"=0A= .\" $FreeBSD: src/sys/boot/forth/menu.4th.8,v 1.2 2011/09/02 19:29:37 = jh Exp $=0A= .\"=0A= -.Dd Aug 29, 2011=0A= +.Dd Dec 27, 2011=0A= .Dt MENU.4TH 8=0A= .Os=0A= .Sh NAME=0A= @@ -69,9 +69,13 @@=0A= Calls=0A= .Ic menu-erase=0A= and then redraws the menu.=0A= +.It Ic menu-unset=0A= +Unsets the environment variables associated with individual menu items,=0A= +clearing the way for a new menu.=0A= .It Ic menu-clear=0A= -Unsets all possible environment variables used=0A= -to configure the menu and then calls=0A= +Calls=0A= +.Ic menu-unset=0A= +and then=0A= .Ic menu-erase .=0A= .El=0A= .Pp=0A= --- src/sys/boot/forth/menu.4th.orig Fri Dec 2 11:17:45 2011=0A= +++ src/sys/boot/forth/menu.4th Tue Dec 27 17:09:04 2011=0A= @@ -131,11 +131,11 @@=0A= =0A= \ Print the value of menuidx=0A= loader_color? if=0A= - ." =1B[1m"=0A= + ." =1B[1m" ( =1B[22m )=0A= then=0A= menuidx @ .=0A= loader_color? if=0A= - ." =1B[37m"=0A= + ." =1B[37m" ( =1B[39m )=0A= then=0A= =0A= \ Move the cursor forward 1 column=0A= @@ -897,22 +897,60 @@=0A= ;=0A= =0A= \ This function unsets all the possible environment variables = associated with=0A= -\ creating the interactive menu. Call this when you want to clear the = menu=0A= -\ area in preparation for another menu.=0A= +\ creating the interactive menu.=0A= \ =0A= -: menu-clear ( -- )=0A= +: menu-unset ( -- )=0A= =0A= 49 \ Iterator start (loop range 49 to 56; ASCII '1' to '8')=0A= begin=0A= - \ basename for caption variable=0A= - loader_color? if=0A= - s" ansi_caption[x]"=0A= - else=0A= - s" menu_caption[x]"=0A= - then=0A= + \ Unset variables in-order of appearance in menu.4th(8)=0A= +=0A= + s" menu_caption[x]" \ basename for caption variable=0A= -rot 2dup 13 + c! rot \ replace 'x' with current iteration=0A= unsetenv \ not erroneous to unset unknown var=0A= =0A= + s" menu_command[x]" \ command basename=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + unsetenv=0A= +=0A= + s" menu_keycode[x]" \ keycode basename=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + unsetenv=0A= +=0A= + s" ansi_caption[x]" \ ANSI caption basename=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + unsetenv=0A= +=0A= + s" toggled_text[x]" \ toggle_menuitem caption basename=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + unsetenv=0A= +=0A= + s" toggled_ansi[x]" \ toggle_menuitem ANSI caption basename=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + unsetenv=0A= +=0A= + s" menu_caption[x][y]" \ cycle_menuitem caption=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + 49 -rot=0A= + begin=0A= + 16 2over rot + c! \ replace 'y'=0A= + 2dup unsetenv=0A= +=0A= + rot 1+ dup 56 > 2swap rot=0A= + until=0A= + 2drop drop=0A= +=0A= + s" ansi_caption[x][y]" \ cycle_menuitem ANSI caption=0A= + -rot 2dup 13 + c! rot \ replace 'x'=0A= + 49 -rot=0A= + begin=0A= + 16 2over rot + c! \ replace 'y'=0A= + 2dup unsetenv=0A= +=0A= + rot 1+ dup 56 > 2swap rot=0A= + until=0A= + 2drop drop=0A= +=0A= s" 0 menukeyN !" \ basename for key association var=0A= -rot 2dup 9 + c! rot \ replace 'N' with current iteration=0A= evaluate \ assign zero (0) to key assoc. var=0A= @@ -921,6 +959,9 @@=0A= until=0A= drop \ iterator=0A= =0A= + \ unset the timeout command=0A= + s" menu_timeout_command" unsetenv=0A= +=0A= \ clear the "Reboot" menu option flag=0A= s" menu_reboot" unsetenv=0A= 0 menureboot !=0A= @@ -933,6 +974,13 @@=0A= s" menu_options" unsetenv=0A= 0 menuoptions !=0A= =0A= +;=0A= +=0A= +\ This function both unsets menu variables and visually erases the menu = area=0A= +\ in-preparation for another menu.=0A= +\ =0A= +: menu-clear ( -- )=0A= + menu-unset=0A= menu-erase=0A= ;=0A= =0A= --- src/sys/boot/forth/loader.4th.orig Sun May 29 19:03:52 2011=0A= +++ src/sys/boot/forth/loader.4th Tue Dec 27 11:24:55 2011=0A= @@ -41,12 +41,15 @@=0A= =0A= include /boot/support.4th=0A= =0A= -\ ***** boot-conf=0A= -\=0A= -\ Prepares to boot as specified by loaded configuration files.=0A= -=0A= only forth also support-functions also builtins definitions=0A= =0A= +: try-menu-unset=0A= + s" menu-unset"=0A= + ['] evaluate catch if=0A= + 2drop=0A= + then=0A= +;=0A= +=0A= : boot=0A= 0=3D if ( interpreted ) get_arguments then=0A= =0A= @@ -57,23 +60,31 @@=0A= 0 1 unload drop=0A= else=0A= s" kernelname" getenv? if ( a kernel has been loaded )=0A= + try-menu-unset=0A= 1 boot exit=0A= then=0A= load_kernel_and_modules=0A= ?dup if exit then=0A= + try-menu-unset=0A= 0 1 boot exit=0A= then=0A= else=0A= s" kernelname" getenv? if ( a kernel has been loaded )=0A= + try-menu-unset=0A= 1 boot exit=0A= then=0A= load_kernel_and_modules=0A= ?dup if exit then=0A= + try-menu-unset=0A= 0 1 boot exit=0A= then=0A= load_kernel_and_modules=0A= ?dup 0=3D if 0 1 boot then=0A= ;=0A= +=0A= +\ ***** boot-conf=0A= +\=0A= +\ Prepares to boot as specified by loaded configuration files.=0A= =0A= : boot-conf=0A= 0=3D if ( interpreted ) get_arguments then=0A= ------=_NextPart_000_02B5_01CCC4BC.A6529EF0-- From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 11:38:15 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6F521065672 for ; Wed, 28 Dec 2011 11:38:15 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id B221C8FC27 for ; Wed, 28 Dec 2011 11:38:15 +0000 (UTC) Received: by iadj38 with SMTP id j38so27192166iad.13 for ; Wed, 28 Dec 2011 03:38:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; bh=53cgtznTHDrbW+T/zhLVDMhgiEDNBB59ortz21oVKEw=; b=jfGEVGaVeJHIvq1fCtMUhA2aH8E/m1BsIt9qHFd89A2GlX8VgtTTuPYQKxNw1I9igV 0SWA+UJcfq3sBcGgcO3dCUOsg10J5EXVt/3pPdZZiYmLjT75c2XTaPZSjEVZXkDG4pEX rI5RgPdn1aN5Qbso5MO28/usZjTez1ia/eMlo= Received: by 10.50.155.195 with SMTP id vy3mr35245167igb.12.1325072295268; Wed, 28 Dec 2011 03:38:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Wed, 28 Dec 2011 03:37:44 -0800 (PST) From: Chris Rees Date: Wed, 28 Dec 2011 11:37:44 +0000 Message-ID: To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 11:38:16 -0000 Hi all, Apologies for the noob question-- I'm trying to stop audio/oss causing panics when loaded on a system that includes DEVICE sound in its kernel config (which has been GENERIC for ~6months now :)) Is there a simple way to check for existence of a driver? I could even check for /dev/sndstat, though that doesn't seem elegant to me... Chris From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 12:24:29 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C689106564A for ; Wed, 28 Dec 2011 12:24:29 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id CD1D28FC15 for ; Wed, 28 Dec 2011 12:24:28 +0000 (UTC) Received: by iadj38 with SMTP id j38so27281921iad.13 for ; Wed, 28 Dec 2011 04:24:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=VP7z7F+wXRnykDC1yOc5vDlhxRS1vfM8wr3VLuNEbgM=; b=x6HU8K9Uxpk3a1OfcL0J6tfB5+j6np101fYuRXVZ/cFrEzgtcMYfmypxZzLjdFEfqM /5MpB9SU+UEnj/Fo5AA9bTMDb3jvU9551MdseRpvekUVmWLujgth8A65oALDd+y9Ep76 Sb48dRTazuM9lGxReLMaBx3JHCJLzkpbRMygM= Received: by 10.50.77.227 with SMTP id v3mr35091467igw.8.1325075068290; Wed, 28 Dec 2011 04:24:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Wed, 28 Dec 2011 04:23:58 -0800 (PST) In-Reply-To: References: From: Chris Rees Date: Wed, 28 Dec 2011 12:23:58 +0000 Message-ID: To: "Daniel O'Connor" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 12:24:29 -0000 On 28 December 2011 12:21, Daniel O'Connor wrote: > > On 28/12/2011, at 22:07, Chris Rees wrote: >> Is there a simple way to check for existence of a driver? =A0I could >> even check for /dev/sndstat, though that doesn't seem elegant to me... > > kldstat -v, but really /dev/sndstat seems simpler and just as effective. > Cheers-- I was thinking of a kernel-level function though. cognet@ suggested using modfind("sound"), I'll go with that. Thanks! Chris From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 12:37:33 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E4B3106566C for ; Wed, 28 Dec 2011 12:37:33 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 0C1B58FC08 for ; Wed, 28 Dec 2011 12:37:32 +0000 (UTC) Received: from ur.dons.net.au (ppp121-45-104-78.lns20.adl6.internode.on.net [121.45.104.78]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id pBSCLUFj041540 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 28 Dec 2011 22:51:36 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: "Daniel O'Connor" In-Reply-To: Date: Wed, 28 Dec 2011 22:51:30 +1030 Content-Transfer-Encoding: 7bit Message-Id: References: To: Chris Rees X-Mailer: Apple Mail (2.1251.1) X-Spam-Score: 2.163 (**) BAYES_00,KHOP_DYNAMIC,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 12:37:33 -0000 On 28/12/2011, at 22:07, Chris Rees wrote: > Is there a simple way to check for existence of a driver? I could > even check for /dev/sndstat, though that doesn't seem elegant to me... kldstat -v, but really /dev/sndstat seems simpler and just as effective. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 14:36:14 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7D14106574B for ; Wed, 28 Dec 2011 14:36:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 344668FC08 for ; Wed, 28 Dec 2011 14:36:13 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBSDug7p099088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 28 Dec 2011 15:56:42 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBSDufun008251; Wed, 28 Dec 2011 15:56:41 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBSDufxu008250; Wed, 28 Dec 2011 15:56:41 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 28 Dec 2011 15:56:41 +0200 From: Kostik Belousov To: Chris Rees Message-ID: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kfDFKqcrHLBvkXCZ" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 14:36:14 -0000 --kfDFKqcrHLBvkXCZ Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: > On 28 December 2011 12:21, Daniel O'Connor wrote: > > > > On 28/12/2011, at 22:07, Chris Rees wrote: > >> Is there a simple way to check for existence of a driver? =9AI could > >> even check for /dev/sndstat, though that doesn't seem elegant to me... > > > > kldstat -v, but really /dev/sndstat seems simpler and just as effective. > > >=20 > Cheers-- I was thinking of a kernel-level function though. >=20 > cognet@ suggested using modfind("sound"), I'll go with that. Obvious question is what the panic is. Checking for modules loaded is papering over some issue. --kfDFKqcrHLBvkXCZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk77IBkACgkQC3+MBN1Mb4h/6QCgjo5nLrCT3ZHMWXokC1zEt2DE LOYAoIcgcJXyrjFDs9mhLvBzMLOuCkQM =Jg97 -----END PGP SIGNATURE----- --kfDFKqcrHLBvkXCZ-- From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 14:54:13 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D157A106566C for ; Wed, 28 Dec 2011 14:54:13 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9AEFD8FC13 for ; Wed, 28 Dec 2011 14:54:13 +0000 (UTC) Received: by iadj38 with SMTP id j38so27548971iad.13 for ; Wed, 28 Dec 2011 06:54:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=tjkRBhPUH+d3Oob87lA9lxE9xvBlRKK5DjJByg5qM0w=; b=c6HEH68CzfEH6WSEGP7Erp8/rE4nU2rvcEIClOI5HDrXU6SoTRbUosWAwzYpvChmL8 pFpT0wlwiS7UoEfXUOM9vM2z7KEObOKUXYWBYsHUz9DHI/MU9w8HrsdFJtZMThqb7AM7 l2x3tAhWp/Bvmq5ARs3S4yHUk0PPJ/gx2r5dM= Received: by 10.50.135.41 with SMTP id pp9mr36162642igb.8.1325084053262; Wed, 28 Dec 2011 06:54:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Wed, 28 Dec 2011 06:53:42 -0800 (PST) In-Reply-To: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> From: Chris Rees Date: Wed, 28 Dec 2011 14:53:42 +0000 Message-ID: To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 14:54:13 -0000 2011/12/28 Kostik Belousov : > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: >> On 28 December 2011 12:21, Daniel O'Connor wrote= : >> > >> > On 28/12/2011, at 22:07, Chris Rees wrote: >> >> Is there a simple way to check for existence of a driver? =A0I could >> >> even check for /dev/sndstat, though that doesn't seem elegant to me..= . >> > >> > kldstat -v, but really /dev/sndstat seems simpler and just as effectiv= e. >> > >> >> Cheers-- I was thinking of a kernel-level function though. >> >> cognet@ suggested using modfind("sound"), I'll go with that. > Obvious question is what the panic is. Checking for modules loaded is > papering over some issue. True, although I figured that it was a simple conflict, possibly to do with sndstat. Also, I'm getting panics with the following patch, whether sound is loaded or not :) + if (modfind("sound") >=3D 0) + { + cmn_err (CE_WARN, "A conflicting sound driver is already loaded"); + return EBUSY; + } + Is there a better way to handle such conflicts? Chris From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 28 15:00:28 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6710106566C for ; Wed, 28 Dec 2011 15:00:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8320A8FC12 for ; Wed, 28 Dec 2011 15:00:27 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBSF0NuB004249 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 28 Dec 2011 17:00:23 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBSF0NCq008490; Wed, 28 Dec 2011 17:00:23 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBSF0Nic008489; Wed, 28 Dec 2011 17:00:23 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 28 Dec 2011 17:00:23 +0200 From: Kostik Belousov To: Chris Rees Message-ID: <20111228150023.GY50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="b10Qt5W+7gVLGshn" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 15:00:28 -0000 --b10Qt5W+7gVLGshn Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: > 2011/12/28 Kostik Belousov : > > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: > >> On 28 December 2011 12:21, Daniel O'Connor wro= te: > >> > > >> > On 28/12/2011, at 22:07, Chris Rees wrote: > >> >> Is there a simple way to check for existence of a driver? =9AI could > >> >> even check for /dev/sndstat, though that doesn't seem elegant to me= ... > >> > > >> > kldstat -v, but really /dev/sndstat seems simpler and just as effect= ive. > >> > > >> > >> Cheers-- I was thinking of a kernel-level function though. > >> > >> cognet@ suggested using modfind("sound"), I'll go with that. > > Obvious question is what the panic is. Checking for modules loaded is > > papering over some issue. >=20 > True, although I figured that it was a simple conflict, possibly to do > with sndstat. >=20 > Also, I'm getting panics with the following patch, whether sound is > loaded or not :) >=20 > + if (modfind("sound") >=3D 0) > + { > + cmn_err (CE_WARN, "A conflicting sound driver is already loaded"); > + return EBUSY; > + } > + >=20 > Is there a better way to handle such conflicts? You have missed the point. There is some bug in oss driver that causing the panic. Presumed 'conflict' cannot cause the harm itself, besides not allowing second driver to attach to the same device, and should not result in panic. Trying to implement a half-measure that only covers the problem you do a mis-service. And you still did not provided the panic message. --b10Qt5W+7gVLGshn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk77LwcACgkQC3+MBN1Mb4g8JQCePyY1CYPa2sp/3ACnW3jgMmTh syQAn27MIo0wDRz7LFcDEbr7rW7OjyTO =wfmT -----END PGP SIGNATURE----- --b10Qt5W+7gVLGshn-- From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 29 11:47:29 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61E311065673 for ; Thu, 29 Dec 2011 11:47:29 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 265A68FC14 for ; Thu, 29 Dec 2011 11:47:28 +0000 (UTC) Received: by iadj38 with SMTP id j38so29705621iad.13 for ; Thu, 29 Dec 2011 03:47:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=0V/+NdJ7yGMjiFAZr0lg0QtxvTM30rK9jwUZdhg99c8=; b=X56W0jp2m9Ya+6F0hGJQeUu7lMwRUW2pJZZU7LzD4dfrpT7oG2LBR25LKptTa/jMCP f77TeqkulWproSkogy6BCVVvvNf8IY6dXW0xBdFIt6rBcGdghNb6LaCTjXh1b5PAHmyD vgQNCchVpoPo8K4BCEwGRzfwH5oLaygJKzvEU= Received: by 10.50.100.164 with SMTP id ez4mr20684307igb.12.1325159248278; Thu, 29 Dec 2011 03:47:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Thu, 29 Dec 2011 03:46:57 -0800 (PST) In-Reply-To: <20111228150023.GY50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> <20111228150023.GY50300@deviant.kiev.zoral.com.ua> From: Chris Rees Date: Thu, 29 Dec 2011 11:46:57 +0000 Message-ID: To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 11:47:29 -0000 2011/12/28 Kostik Belousov : > On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: >> 2011/12/28 Kostik Belousov : >> > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: >> >> On 28 December 2011 12:21, Daniel O'Connor wr= ote: >> >> > >> >> > On 28/12/2011, at 22:07, Chris Rees wrote: >> >> >> Is there a simple way to check for existence of a driver? =A0I cou= ld >> >> >> even check for /dev/sndstat, though that doesn't seem elegant to m= e... >> >> > >> >> > kldstat -v, but really /dev/sndstat seems simpler and just as effec= tive. >> >> > >> >> >> >> Cheers-- I was thinking of a kernel-level function though. >> >> >> >> cognet@ suggested using modfind("sound"), I'll go with that. >> > Obvious question is what the panic is. Checking for modules loaded is >> > papering over some issue. >> >> True, although I figured that it was a simple conflict, possibly to do >> with sndstat. >> >> Also, I'm getting panics with the following patch, whether sound is >> loaded or not :) >> >> + =A0if (modfind("sound") >=3D 0) >> + =A0 =A0{ >> + =A0 =A0 =A0cmn_err (CE_WARN, "A conflicting sound driver is already lo= aded"); >> + =A0 =A0 =A0return EBUSY; >> + =A0 =A0} >> + >> >> Is there a better way to handle such conflicts? > > You have missed the point. There is some bug in oss driver that causing > the panic. Presumed 'conflict' cannot cause the harm itself, besides not > allowing second driver to attach to the same device, and should not resul= t > in panic. Trying to implement a half-measure that only covers the problem > you do a mis-service. > > And you still did not provided the panic message. I'm sorry, you're right. However, your guess was in fact correct; make_dev was being called, which returned a null pointer because it failed. The patch at [1] stops the panic, however I was hoping that returning EBUSY would abort loading the module... At the moment it loads the module, and doesn't create the sndstat dev, which causes weird errors with the oss binary commands. Since this solves the panic and anyone should be able to work out from the warning message what the problem is, AND this is a port that apparently no-one else uses, should this be sufficient? BTW, it only affects FreeBSD 9+, couldn't reproduce on my 8.2 dev machine, but could once I upgraded it. Chris [1] http://www.bayofrum.net/~crees/patches/oss-patch-kernel-OS-FreeBSD-os_f= reebsd.c From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 29 12:03:39 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1E62106566B for ; Thu, 29 Dec 2011 12:03:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2C5AA8FC19 for ; Thu, 29 Dec 2011 12:03:38 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBTC3YOg054046 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 29 Dec 2011 14:03:34 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBTC3X3i017165; Thu, 29 Dec 2011 14:03:33 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBTC3XBe017164; Thu, 29 Dec 2011 14:03:33 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Dec 2011 14:03:33 +0200 From: Kostik Belousov To: Chris Rees Message-ID: <20111229120333.GI50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> <20111228150023.GY50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DBl6kzebMedsdQd1" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 12:03:39 -0000 --DBl6kzebMedsdQd1 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 29, 2011 at 11:46:57AM +0000, Chris Rees wrote: > 2011/12/28 Kostik Belousov : > > On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: > >> 2011/12/28 Kostik Belousov : > >> > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: > >> >> On 28 December 2011 12:21, Daniel O'Connor = wrote: > >> >> > > >> >> > On 28/12/2011, at 22:07, Chris Rees wrote: > >> >> >> Is there a simple way to check for existence of a driver? =9AI c= ould > >> >> >> even check for /dev/sndstat, though that doesn't seem elegant to= me... > >> >> > > >> >> > kldstat -v, but really /dev/sndstat seems simpler and just as eff= ective. > >> >> > > >> >> > >> >> Cheers-- I was thinking of a kernel-level function though. > >> >> > >> >> cognet@ suggested using modfind("sound"), I'll go with that. > >> > Obvious question is what the panic is. Checking for modules loaded is > >> > papering over some issue. > >> > >> True, although I figured that it was a simple conflict, possibly to do > >> with sndstat. > >> > >> Also, I'm getting panics with the following patch, whether sound is > >> loaded or not :) > >> > >> + =9Aif (modfind("sound") >=3D 0) > >> + =9A =9A{ > >> + =9A =9A =9Acmn_err (CE_WARN, "A conflicting sound driver is already = loaded"); > >> + =9A =9A =9Areturn EBUSY; > >> + =9A =9A} > >> + > >> > >> Is there a better way to handle such conflicts? > > > > You have missed the point. There is some bug in oss driver that causing > > the panic. Presumed 'conflict' cannot cause the harm itself, besides not > > allowing second driver to attach to the same device, and should not res= ult > > in panic. Trying to implement a half-measure that only covers the probl= em > > you do a mis-service. > > > > And you still did not provided the panic message. >=20 > I'm sorry, you're right. However, your guess was in fact correct; > make_dev was being called, which returned a null pointer because it > failed. >=20 > The patch at [1] stops the panic, however I was hoping that returning > EBUSY would abort loading the module... At the moment it loads the > module, and doesn't create the sndstat dev, which causes weird errors > with the oss binary commands. >=20 > Since this solves the panic and anyone should be able to work out from > the warning message what the problem is, AND this is a port that > apparently no-one else uses, should this be sufficient? >=20 > BTW, it only affects FreeBSD 9+, couldn't reproduce on my 8.2 dev > machine, but could once I upgraded it. On 8.2, there is no check in the devfs for duplicated cdev names, AFAIR. So you get absolutely undeterministic behaviour which driver is referenced by devfs node. > Chris >=20 > [1] http://www.bayofrum.net/~crees/patches/oss-patch-kernel-OS-FreeBSD-os= _freebsd.c I highly recommend to return error in case of any make_dev_p(9) failure, and not only EEXIST. --DBl6kzebMedsdQd1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk78VxQACgkQC3+MBN1Mb4hDRwCcD1N/Z24y1mPuAnUxAW/XGNPL MBUAoImdr6Z8q8zV/9W0E09grmJENPAZ =gpKx -----END PGP SIGNATURE----- --DBl6kzebMedsdQd1-- From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 29 12:53:51 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23CBB1065673 for ; Thu, 29 Dec 2011 12:53:51 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D9FD18FC0A for ; Thu, 29 Dec 2011 12:53:50 +0000 (UTC) Received: by iadj38 with SMTP id j38so29815513iad.13 for ; Thu, 29 Dec 2011 04:53:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=z1xGZSqre8MqsVsCgK51z4sVAxQoCTtg6dIceBn01No=; b=fyI6dD3zZH6sGybMHRr2PSOsGKChDQX+QR29qGeV0OQ/s4K4hGZ0fZ9HzsiLIuF8zb /fIm3zEUyrvgW3DCx8wAUOQbavSwOv1irY4b8MazrjWS0t7ypvU7iDDy0taxo5kIU3pf AzwW0xH10MOb8/U/ubTqbKYwrUDBo9/0BAs5o= Received: by 10.43.51.69 with SMTP id vh5mr41754508icb.32.1325163230287; Thu, 29 Dec 2011 04:53:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Thu, 29 Dec 2011 04:53:19 -0800 (PST) In-Reply-To: <20111229120333.GI50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> <20111228150023.GY50300@deviant.kiev.zoral.com.ua> <20111229120333.GI50300@deviant.kiev.zoral.com.ua> From: Chris Rees Date: Thu, 29 Dec 2011 12:53:19 +0000 Message-ID: To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 12:53:51 -0000 2011/12/29 Kostik Belousov : > On Thu, Dec 29, 2011 at 11:46:57AM +0000, Chris Rees wrote: >> 2011/12/28 Kostik Belousov : >> > On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: >> >> 2011/12/28 Kostik Belousov : >> >> > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: >> >> >> On 28 December 2011 12:21, Daniel O'Connor = wrote: >> >> >> > >> >> >> > On 28/12/2011, at 22:07, Chris Rees wrote: >> >> >> >> Is there a simple way to check for existence of a driver? =A0I = could >> >> >> >> even check for /dev/sndstat, though that doesn't seem elegant t= o me... >> >> >> > >> >> >> > kldstat -v, but really /dev/sndstat seems simpler and just as ef= fective. >> >> >> > >> >> >> >> >> >> Cheers-- I was thinking of a kernel-level function though. >> >> >> >> >> >> cognet@ suggested using modfind("sound"), I'll go with that. >> >> > Obvious question is what the panic is. Checking for modules loaded = is >> >> > papering over some issue. >> >> >> >> True, although I figured that it was a simple conflict, possibly to d= o >> >> with sndstat. >> >> >> >> Also, I'm getting panics with the following patch, whether sound is >> >> loaded or not :) >> >> >> >> + =A0if (modfind("sound") >=3D 0) >> >> + =A0 =A0{ >> >> + =A0 =A0 =A0cmn_err (CE_WARN, "A conflicting sound driver is already= loaded"); >> >> + =A0 =A0 =A0return EBUSY; >> >> + =A0 =A0} >> >> + >> >> >> >> Is there a better way to handle such conflicts? >> > >> > You have missed the point. There is some bug in oss driver that causin= g >> > the panic. Presumed 'conflict' cannot cause the harm itself, besides n= ot >> > allowing second driver to attach to the same device, and should not re= sult >> > in panic. Trying to implement a half-measure that only covers the prob= lem >> > you do a mis-service. >> > >> > And you still did not provided the panic message. >> >> I'm sorry, you're right. =A0However, your guess was in fact correct; >> make_dev was being called, which returned a null pointer because it >> failed. >> >> The patch at [1] stops the panic, however I was hoping that returning >> EBUSY would abort loading the module... At the moment it loads the >> module, and doesn't create the sndstat dev, which causes weird errors >> with the oss binary commands. >> >> Since this solves the panic and anyone should be able to work out from >> the warning message what the problem is, AND this is a port that >> apparently no-one else uses, should this be sufficient? >> >> BTW, it only affects FreeBSD 9+, couldn't reproduce on my 8.2 dev >> machine, but could once I upgraded it. > On 8.2, there is no check in the devfs for duplicated cdev names, AFAIR. > So you get absolutely undeterministic behaviour which driver is reference= d > by devfs node. > >> Chris >> >> [1] http://www.bayofrum.net/~crees/patches/oss-patch-kernel-OS-FreeBSD-o= s_freebsd.c > > I highly recommend to return error in case of any make_dev_p(9) failure, = and > not only EEXIST. That'd be great-- but I can't work out how to do it :( Do I need to return a different value? Chris From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 29 13:35:22 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8439F106566C for ; Thu, 29 Dec 2011 13:35:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1C82B8FC1A for ; Thu, 29 Dec 2011 13:35:21 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBTDZGxF073328 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 29 Dec 2011 15:35:16 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBTDZGMj017407; Thu, 29 Dec 2011 15:35:16 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBTDZGAF017406; Thu, 29 Dec 2011 15:35:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Dec 2011 15:35:16 +0200 From: Kostik Belousov To: Chris Rees Message-ID: <20111229133516.GJ50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> <20111228150023.GY50300@deviant.kiev.zoral.com.ua> <20111229120333.GI50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="e/SfQGHLOMWAfGD8" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 13:35:22 -0000 --e/SfQGHLOMWAfGD8 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 29, 2011 at 12:53:19PM +0000, Chris Rees wrote: > 2011/12/29 Kostik Belousov : > > On Thu, Dec 29, 2011 at 11:46:57AM +0000, Chris Rees wrote: > >> 2011/12/28 Kostik Belousov : > >> > On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: > >> >> 2011/12/28 Kostik Belousov : > >> >> > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: > >> >> >> On 28 December 2011 12:21, Daniel O'Connor wrote: > >> >> >> > > >> >> >> > On 28/12/2011, at 22:07, Chris Rees wrote: > >> >> >> >> Is there a simple way to check for existence of a driver? =9A= I could > >> >> >> >> even check for /dev/sndstat, though that doesn't seem elegant= to me... > >> >> >> > > >> >> >> > kldstat -v, but really /dev/sndstat seems simpler and just as = effective. > >> >> >> > > >> >> >> > >> >> >> Cheers-- I was thinking of a kernel-level function though. > >> >> >> > >> >> >> cognet@ suggested using modfind("sound"), I'll go with that. > >> >> > Obvious question is what the panic is. Checking for modules loade= d is > >> >> > papering over some issue. > >> >> > >> >> True, although I figured that it was a simple conflict, possibly to= do > >> >> with sndstat. > >> >> > >> >> Also, I'm getting panics with the following patch, whether sound is > >> >> loaded or not :) > >> >> > >> >> + =9Aif (modfind("sound") >=3D 0) > >> >> + =9A =9A{ > >> >> + =9A =9A =9Acmn_err (CE_WARN, "A conflicting sound driver is alrea= dy loaded"); > >> >> + =9A =9A =9Areturn EBUSY; > >> >> + =9A =9A} > >> >> + > >> >> > >> >> Is there a better way to handle such conflicts? > >> > > >> > You have missed the point. There is some bug in oss driver that caus= ing > >> > the panic. Presumed 'conflict' cannot cause the harm itself, besides= not > >> > allowing second driver to attach to the same device, and should not = result > >> > in panic. Trying to implement a half-measure that only covers the pr= oblem > >> > you do a mis-service. > >> > > >> > And you still did not provided the panic message. > >> > >> I'm sorry, you're right. =9AHowever, your guess was in fact correct; > >> make_dev was being called, which returned a null pointer because it > >> failed. > >> > >> The patch at [1] stops the panic, however I was hoping that returning > >> EBUSY would abort loading the module... At the moment it loads the > >> module, and doesn't create the sndstat dev, which causes weird errors > >> with the oss binary commands. > >> > >> Since this solves the panic and anyone should be able to work out from > >> the warning message what the problem is, AND this is a port that > >> apparently no-one else uses, should this be sufficient? > >> > >> BTW, it only affects FreeBSD 9+, couldn't reproduce on my 8.2 dev > >> machine, but could once I upgraded it. > > On 8.2, there is no check in the devfs for duplicated cdev names, AFAIR. > > So you get absolutely undeterministic behaviour which driver is referen= ced > > by devfs node. > > > >> Chris > >> > >> [1] http://www.bayofrum.net/~crees/patches/oss-patch-kernel-OS-FreeBSD= -os_freebsd.c > > > > I highly recommend to return error in case of any make_dev_p(9) failure= , and > > not only EEXIST. >=20 > That'd be great-- but I can't work out how to do it :( >=20 > Do I need to return a different value? error =3D make_dev_p(); if (error !=3D 0) { printf("Error creating device node /dev/%s: %d\n", name, error); return (error); } Or I do not understand your question. --e/SfQGHLOMWAfGD8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk78bJMACgkQC3+MBN1Mb4jGewCg468OpxW9oKFYmDrlkfpOQ2y+ Rc0An2tDvAM3wUqfZLXP1M6zYDXFUGiD =U2gS -----END PGP SIGNATURE----- --e/SfQGHLOMWAfGD8-- From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 29 13:50:46 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21F9C1065675 for ; Thu, 29 Dec 2011 13:50:46 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D9C508FC14 for ; Thu, 29 Dec 2011 13:50:45 +0000 (UTC) Received: by iadj38 with SMTP id j38so29907306iad.13 for ; Thu, 29 Dec 2011 05:50:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=DRG86oExDruJssVMvAUyLUCqi0Ps2rrbVwF6scJx4Ik=; b=SgSYxWcZpyq5TEY8aMdBaNAE1vqjHgofD7AoLBA/hChZlVKjV4SZiN4H+xFNMsEah7 RfcUvmpohu7S06Da8CT8OIpZ4DMzMNJ0Cj431x5+72Jne67QJksDduToIOvU88/TEMZv VoWGmVVdw0ACmOrs3P1+oZIFqJJ1M1TKFWLzo= Received: by 10.50.155.195 with SMTP id vy3mr40683300igb.12.1325166645321; Thu, 29 Dec 2011 05:50:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.207.7 with HTTP; Thu, 29 Dec 2011 05:50:14 -0800 (PST) In-Reply-To: <20111229133516.GJ50300@deviant.kiev.zoral.com.ua> References: <20111228135641.GV50300@deviant.kiev.zoral.com.ua> <20111228150023.GY50300@deviant.kiev.zoral.com.ua> <20111229120333.GI50300@deviant.kiev.zoral.com.ua> <20111229133516.GJ50300@deviant.kiev.zoral.com.ua> From: Chris Rees Date: Thu, 29 Dec 2011 13:50:14 +0000 Message-ID: To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Checking for other kernel modules on load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 13:50:46 -0000 2011/12/29 Kostik Belousov : > On Thu, Dec 29, 2011 at 12:53:19PM +0000, Chris Rees wrote: >> 2011/12/29 Kostik Belousov : >> > On Thu, Dec 29, 2011 at 11:46:57AM +0000, Chris Rees wrote: >> >> 2011/12/28 Kostik Belousov : >> >> > On Wed, Dec 28, 2011 at 02:53:42PM +0000, Chris Rees wrote: >> >> >> 2011/12/28 Kostik Belousov : >> >> >> > On Wed, Dec 28, 2011 at 12:23:58PM +0000, Chris Rees wrote: >> >> >> >> On 28 December 2011 12:21, Daniel O'Connor wrote: >> >> >> >> > >> >> >> >> > On 28/12/2011, at 22:07, Chris Rees wrote: >> >> >> >> >> Is there a simple way to check for existence of a driver? = =A0I could >> >> >> >> >> even check for /dev/sndstat, though that doesn't seem elegan= t to me... >> >> >> >> > >> >> >> >> > kldstat -v, but really /dev/sndstat seems simpler and just as= effective. >> >> >> >> > >> >> >> >> >> >> >> >> Cheers-- I was thinking of a kernel-level function though. >> >> >> >> >> >> >> >> cognet@ suggested using modfind("sound"), I'll go with that. >> >> >> > Obvious question is what the panic is. Checking for modules load= ed is >> >> >> > papering over some issue. >> >> >> >> >> >> True, although I figured that it was a simple conflict, possibly t= o do >> >> >> with sndstat. >> >> >> >> >> >> Also, I'm getting panics with the following patch, whether sound i= s >> >> >> loaded or not :) >> >> >> >> >> >> + =A0if (modfind("sound") >=3D 0) >> >> >> + =A0 =A0{ >> >> >> + =A0 =A0 =A0cmn_err (CE_WARN, "A conflicting sound driver is alre= ady loaded"); >> >> >> + =A0 =A0 =A0return EBUSY; >> >> >> + =A0 =A0} >> >> >> + >> >> >> >> >> >> Is there a better way to handle such conflicts? >> >> > >> >> > You have missed the point. There is some bug in oss driver that cau= sing >> >> > the panic. Presumed 'conflict' cannot cause the harm itself, beside= s not >> >> > allowing second driver to attach to the same device, and should not= result >> >> > in panic. Trying to implement a half-measure that only covers the p= roblem >> >> > you do a mis-service. >> >> > >> >> > And you still did not provided the panic message. >> >> >> >> I'm sorry, you're right. =A0However, your guess was in fact correct; >> >> make_dev was being called, which returned a null pointer because it >> >> failed. >> >> >> >> The patch at [1] stops the panic, however I was hoping that returning >> >> EBUSY would abort loading the module... At the moment it loads the >> >> module, and doesn't create the sndstat dev, which causes weird errors >> >> with the oss binary commands. >> >> >> >> Since this solves the panic and anyone should be able to work out fro= m >> >> the warning message what the problem is, AND this is a port that >> >> apparently no-one else uses, should this be sufficient? >> >> >> >> BTW, it only affects FreeBSD 9+, couldn't reproduce on my 8.2 dev >> >> machine, but could once I upgraded it. >> > On 8.2, there is no check in the devfs for duplicated cdev names, AFAI= R. >> > So you get absolutely undeterministic behaviour which driver is refere= nced >> > by devfs node. >> > >> >> Chris >> >> >> >> [1] http://www.bayofrum.net/~crees/patches/oss-patch-kernel-OS-FreeBS= D-os_freebsd.c >> > >> > I highly recommend to return error in case of any make_dev_p(9) failur= e, and >> > not only EEXIST. >> >> That'd be great-- but I can't work out how to do it :( >> >> Do I need to return a different value? > > =A0 =A0 =A0 =A0error =3D make_dev_p(); > =A0 =A0 =A0 =A0if (error !=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0printf("Error creating device node /dev/%s= : %d\n", name, error); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (error); > =A0 =A0 =A0 =A0} > No, that's what I wanted. Thank you. Chris From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 30 06:27:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F150106564A for ; Fri, 30 Dec 2011 06:27:09 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0ACD58FC15 for ; Fri, 30 Dec 2011 06:27:08 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so15149059obb.13 for ; Thu, 29 Dec 2011 22:27:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ch+kjrPGCQTLH4cjM54zM7H4aJZikBW0OUAqEd+aArc=; b=SB8EXESK8GfBAakWvckU03CQimPGOY7z74fYvy3YDdJcwu9gQ2kHDnmyqJitlSuyCq 28uzyAmA7Iki99gl3RDCPGHJqJnJB+RJRc2bBx0/uUeOb817n3nH6V+2zswK+nndWCGG 6h76w0vU9n117KYC4wT7SNceYGyEBeUuOYYcU= MIME-Version: 1.0 Received: by 10.182.15.104 with SMTP id w8mr33491323obc.20.1325226428411; Thu, 29 Dec 2011 22:27:08 -0800 (PST) Received: by 10.182.171.67 with HTTP; Thu, 29 Dec 2011 22:27:08 -0800 (PST) In-Reply-To: <02b401ccc4ff$b475b7e0$1d6127a0$@fisglobal.com> References: <02b401ccc4ff$b475b7e0$1d6127a0$@fisglobal.com> Date: Fri, 30 Dec 2011 09:27:08 +0300 Message-ID: From: Sergey Kandaurov To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , freebsd-hackers@freebsd.org Subject: Re: [PATCH] Fix kenv(1) output in w/respect to new boot loader variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 06:27:11 -0000 On 28 December 2011 05:26, Devin Teske wrote: > D'Oh! Attached wrong (OLD; already applied) patch. > > Please find appropriate patch attached! Hi. I committed your patch to head as svn r228985. Thank you! > >> -----Original Message----- >> From: Devin Teske [mailto:devin.teske@fisglobal.com] >> Sent: Tuesday, December 27, 2011 5:24 PM >> To: 'freebsd-hackers@freebsd.org' >> Cc: Garrett Cooper; devin.teske@fisglobal.com >> Subject: [PATCH] Fix kenv(1) output in w/respect to new boot loader vari= ables >> >> Garrett Cooper and a few others have requested that I write a patch to f= ix a >> regression w/respect to kenv(1) output in FreeBSD-9.0 and HEAD. >> >> The issue is with the new boot loader menu. It adds many loader variable= s >> including ones that contain ANSI color escapes. >> >> Obviously, these ANSI codes don't play well with serial consoles when ke= nv(1) > is >> executed without arguments (reports vary as to what happens, but it's ne= ver >> pretty). >> >> Attached is a patch to the Forth code that clears-out the menu-associate= d >> variables before invoking the kernel. >> >> The net-effect is that kenv(1) no longer reports menu-related variables. >> >> In essence, kenv(1) output should now appear the same as on RELENG_8 (wh= ich >> lacks the new boot loader and didn't use any such variables). Thus, rest= oring >> serial console glory. >> -- >> Devin > > _____________ > The information contained in this message is proprietary and/or confident= ial. If you are not the intended recipient, please: (i) delete the message = and all copies; (ii) do not disclose, distribute or use the message in any = manner; and (iii) notify the sender immediately. In addition, please be awa= re that any message addressed to our domain is subject to archiving and rev= iew by persons other than the intended recipient. Thank you. Great! --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 30 17:53:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 061F6106566B for ; Fri, 30 Dec 2011 17:53:11 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D4CF18FC12 for ; Fri, 30 Dec 2011 17:53:10 +0000 (UTC) Received: by pbcc3 with SMTP id c3so12003121pbc.13 for ; Fri, 30 Dec 2011 09:53:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; bh=YtCVTkO39bvxEoKMrg5P7bPEQF9cidxEx3/GDLYBt3Y=; b=pVtkoHM5MSPcZx77yfcBVX3vavxbyZqJ5B2UlLoGkudCvBoSj2f4AdX7+ww5r4nJg8 nA439DZx+Np2xYXRCVZHqKPOKbr9TgHUWbdSh7qAALhPCBZGIQNcASUZBODgmxifWwh7 SWbD6zOEkJ2FGu94Nv4XasWYMRQ7t5Km8dBMM= Received: by 10.68.192.74 with SMTP id he10mr11988705pbc.24.1325267590285; Fri, 30 Dec 2011 09:53:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.143.77.16 with HTTP; Fri, 30 Dec 2011 09:52:39 -0800 (PST) From: arrowdodger <6yearold@gmail.com> Date: Fri, 30 Dec 2011 20:52:39 +0300 Message-ID: To: freebsd-hackers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Using symbolic execution for analyzing scheduler performance? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 17:53:11 -0000 Hi. First, let me put a little disclaimer: I have absolutely no CS education and eny degree in science, no idea on how OS kernels and CPU schedulers are implemented and working. Moreover, i haven't even know math at the level needed to talk about what i'm proposing. What i'm going to propose may be just nonsence. I've assumed that: - Scheduler in FreeBSD is just a bunch of code, which implemens some interface. - This implementation is self-contained - it doesn't call any other kernel functions and do not depend on other state except itself. - OS kernel calls scheduler functions in some defined order. I've these assumptions are true, it may be possible to compile scheduler code as userland code and link it with sort-of driver, which would call scheduler functions in same way as real kernel does. So we get a statically-linked executable, which would emulate working kernel for the scheduler. Now we will be using KLEE [1] - a virtual machine for symbolic execution. It uses SAT solvers to reason about veriables values. In our driver code we insert calls to klee_assert() after every call to scheduler function to make KLEE dump current symbolic restrictions on scheduler's internal state values. Finally, we mark all data, describing scheduler state as symbolic and run program on KLEE. As result, we get (i hope so) a set of all possible states in which scheduler can ever be in form of KLEE test file (.ktest). A test is represented by descriptions of what value each variable can have in the current context. So, any of generated states is not intersecting with each other. Now it's possible to concretize symbolic values for each test and save it as normal executable. You may think of it as a model of how our scheduler is functioning. Now we can symbolically execute these binaries again, but for now marking as symbolic all "external" data from scheduler point of view. This way we can track and debug scheduler decisions in any circumstance. In other words: 1. All possible scheduler states are being found. 2. Identical states are being thrown away. // done by KLEE 3. For each state model scheduler behavior for every input (and skipping modellings, which yields same results). I'm not sure if it can help to solve current ULE problems, but it should really help debugging scheduler during development. What do you think? Does it make any sence, or i should just return under my rock? PS: Sorry for my english, i hope you understood what i've been trying to say. [1] http://klee.llvm.org/ From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 31 00:31:27 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFC63106564A for ; Sat, 31 Dec 2011 00:31:27 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id CDDDC8FC0A for ; Sat, 31 Dec 2011 00:31:27 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id pBV0VPdk068664 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 30 Dec 2011 16:31:26 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4EFE5806.3090000@freebsd.org> Date: Fri, 30 Dec 2011 16:32:06 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: arrowdodger <6yearold@gmail.com> References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers Subject: Re: Using symbolic execution for analyzing scheduler performance? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 00:31:28 -0000 On 12/30/11 9:52 AM, arrowdodger wrote: > Hi. First, let me put a little disclaimer: > I have absolutely no CS education and any degree in science, no idea on how > OS kernels and CPU schedulers are implemented and working. Moreover, i > haven't even know math at the level needed to talk about what i'm > proposing. What i'm going to propose may be just nonsence. > > I've assumed that: > - Scheduler in FreeBSD is just a bunch of code, which implemens some > interface. yes, though it is very complicated interface, and some of it is not explicit. > - This implementation is self-contained - it doesn't call any other kernel > functions and do not depend on other state except itself. That is not really true. There are all sorts of interactions. Not all of them are as simple as a call. > - OS kernel calls scheduler functions in some defined order. The OS doesn't really call the scheduler in that way. all sorts of threads of execution jump into the scheduler from all sorts of places and the internal state of the scheduler changes with these calls. Sometimes those calls never return, and sometimes they return a long time later.. As I said, it's a very complicated interface. > If these assumptions are true, it may be possible to compile scheduler > code as userland code and link it with sort-of driver, which would call > scheduler functions in same way as real kernel does. So we get a > statically-linked executable, which would emulate working kernel for the > scheduler. yes that would certainly be true from some perspective. But it would be quite a bit of work. > Now we will be using KLEE [1] - a virtual machine for symbolic execution. > It uses SAT solvers to reason about veriables values. > In our driver code we insert calls to klee_assert() after every call to > scheduler function to make KLEE dump current symbolic restrictions on > scheduler's internal state values. Finally, we mark all data, describing > scheduler state as symbolic and run program on KLEE. > As result, we get (i hope so) a set of all possible states in which > scheduler can ever be in form of KLEE test file (.ktest). A test is > represented by descriptions of what value each variable can have in the > current context. So, any of generated states is not intersecting with each > other. It is a nice goal but I can't see it happening.. I think that the scheduler is probably a bit too complicated.. the automatic testing and verification woudl be nice but my head hurts thinking about it :-) > Now it's possible to concretize symbolic values for each test and save it > as normal executable. You may think of it as a model of how our scheduler > is functioning. Now we can symbolically execute these binaries again, but > for now marking as symbolic all "external" data from scheduler point of > view. This way we can track and debug scheduler decisions in any > circumstance. > > In other words: > 1. All possible scheduler states are being found. > 2. Identical states are being thrown away. // done by KLEE > 3. For each state model scheduler behavior for every input (and skipping > modellings, which yields same results). > > I'm not sure if it can help to solve current ULE problems, but it should > really help debugging scheduler during development. I don't think that the scheduler needs debugging so much as the specification of it needs thinking.. The scheduler has teh problem of trying to make decisions about what to do in the future from imperfect knowledge of the past, and no knowledge of the future. > What do you think? Does it make any sence, or i should just return under my > rock? no, come out from your rock.. If you are interested in the scheduler, by all means go and read it and try and understand it, and then come back to us if you do have ideas. I don't think your idea is really bad but I think you underestimate the difficulty of the task. > PS: Sorry for my english, i hope you understood what i've been trying to > say. your english is fine.. what do you normally speak? (and you are not really 6 years old, are you? :-) > [1] http://klee.llvm.org/ > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 31 07:34:34 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00AB5106564A for ; Sat, 31 Dec 2011 07:34:34 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id C51778FC0A for ; Sat, 31 Dec 2011 07:34:33 +0000 (UTC) Received: by pbcc3 with SMTP id c3so12310192pbc.13 for ; Fri, 30 Dec 2011 23:34:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=VQqDthIErpdzRwjs9LvMLzgmA07marpRg8VOftj+848=; b=BM4S+8YepBgwjYKCy+if0rIab97uOL0HjMG9esGC33nSHorXRKPzqsG9hP4wy4bqWf jP/YbICjs/6OiZ7x+RendFDFlEnVsQrck7DLvUveJmPW5hprzjdjfzEW2lSp5aECw8bu nLRQQKUneYnqXXrnhDNXrHOkmGvFl1WX7EDq8= Received: by 10.68.213.68 with SMTP id nq4mr13897195pbc.126.1325316873239; Fri, 30 Dec 2011 23:34:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.143.77.16 with HTTP; Fri, 30 Dec 2011 23:34:02 -0800 (PST) In-Reply-To: <4EFE5806.3090000@freebsd.org> References: <4EFE5806.3090000@freebsd.org> From: arrowdodger <6yearold@gmail.com> Date: Sat, 31 Dec 2011 10:34:02 +0300 Message-ID: To: Julian Elischer Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers Subject: Re: Using symbolic execution for analyzing scheduler performance? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 07:34:34 -0000 On Sat, Dec 31, 2011 at 4:32 AM, Julian Elischer wrote: > On 12/30/11 9:52 AM, arrowdodger wrote: > >> - OS kernel calls scheduler functions in some defined order. >> > > The OS doesn't really call the scheduler in that way. > all sorts of threads of execution jump into the scheduler from all sorts > of places and > the internal state of the scheduler changes with these calls. Sometimes > those calls > never return, and sometimes they return a long time later.. As I said, > it's a very > complicated interface. > Oh, threads. Yes, this imposes unimaginable complexity on what i'm proposing. > What do you think? Does it make any sence, or i should just return under >> my >> rock? >> > > no, come out from your rock.. If you are interested in the scheduler, by > all means > go and read it and try and understand it, and then come back to us if you > do have ideas. > Yeah, i think it's what i should've done in first place, before dumping by brain to ML. Okay, i will try to get an idea of how schedulers work in detail and after that will try to find parts of it, which can be automatically verified. BTW, is there any documentation on how write schedulers for FreeBSD, or at least, ULE specification? I don't think your idea is really bad but I think you underestimate the > difficulty of the task. > > PS: Sorry for my english, i hope you understood what i've been trying to >> say. >> > your english is fine.. what do you normally speak? > Russian. (and you are not really 6 years old, are you? :-) > Yeah, i've grown up a little, since then. Thanks for your insight! From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 31 21:52:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 010BD106564A for ; Sat, 31 Dec 2011 21:52:33 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id CA5898FC08 for ; Sat, 31 Dec 2011 21:52:32 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id pBVLqTuF073591 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 31 Dec 2011 13:52:31 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4EFF8449.1060800@freebsd.org> Date: Sat, 31 Dec 2011 13:53:13 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: arrowdodger <6yearold@gmail.com> References: <4EFE5806.3090000@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers Subject: Re: Using symbolic execution for analyzing scheduler performance? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 21:52:33 -0000 On 12/30/11 11:34 PM, arrowdodger wrote: > On Sat, Dec 31, 2011 at 4:32 AM, Julian Elischer > wrote: > > On 12/30/11 9:52 AM, arrowdodger wrote: > > - OS kernel calls scheduler functions in some defined order. > > > The OS doesn't really call the scheduler in that way. > all sorts of threads of execution jump into the scheduler from > all sorts of places and > the internal state of the scheduler changes with these calls. > Sometimes those calls > never return, and sometimes they return a long time later.. As > I said, it's a very > complicated interface. > > > Oh, threads. Yes, this imposes unimaginable complexity on what i'm > proposing. > > > What do you think? Does it make any sence, or i should just > return under my > rock? > > > no, come out from your rock.. If you are interested in the > scheduler, by all means > go and read it and try and understand it, and then come back to > us if you do have ideas. > > > Yeah, i think it's what i should've done in first place, before > dumping by brain to ML. > Okay, i will try to get an idea of how schedulers work in detail and > after that will try to find parts of it, which can be automatically > verified. > > BTW, is there any documentation on how write schedulers for FreeBSD, > or at least, ULE specification? there is a paper that was presented somewhere on ULE, but the best source of information is of course the code.. start by reading the 4bsd scheduler.. as it's simpler.. > > I don't think your idea is really bad but I think you > underestimate the difficulty of the task. > > PS: Sorry for my english, i hope you understood what i've > been trying to > say. > > your english is fine.. what do you normally speak? > > > Russian. there are a lot of russian developers so you should be able to find someone if you need explanations in Russian.. > > (and you are not really 6 years old, are you? :-) > > > Yeah, i've grown up a little, since then. > > Thanks for your insight!