From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 24 10:11:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3FB7ADD4; Sun, 24 Feb 2013 10:11:11 +0000 (UTC) (envelope-from phileas-fogg@mail.ru) Received: from smtp12.mail.ru (smtp12.mail.ru [94.100.176.89]) by mx1.freebsd.org (Postfix) with ESMTP id DB1561B29; Sun, 24 Feb 2013 10:11:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References:Subject:CC:To:MIME-Version:From:Date:Message-ID; bh=+J/q639eVpFWdrBkZGO+F+Be0R0UDM3XQZ+4a8iGNI8=; b=vDb+nISxc64bPvrpoh6W1pcSXgFg4w+nTSkCutmwh+3Ut3QNZwosDNvXSgk4OYKaFUawopNK3MRHcGPcLJz93FZZ7A4qnBoa4ymeas9GWxhYIdYKJL4/AymTVSnvTp7L; Received: from [109.192.50.197] (port=1060 helo=[192.168.1.76]) by smtp12.mail.ru with esmtpa (envelope-from ) id 1U9YY0-0005mN-Qq; Sun, 24 Feb 2013 14:11:09 +0400 Message-ID: <5129F54D.9040503@mail.ru> Date: Sun, 24 Feb 2013 12:11:09 +0100 From: Phileas Fogg User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20100101 Firefox/19.0 SeaMonkey/2.16 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Scrolling in framebuffer syscons References: <511F879C.3030801@mail.ru> <51217854.8000508@freebsd.org> <5123D941.2050906@mail.ru> <51293505.1030706@freebsd.org> In-Reply-To: <51293505.1030706@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: Oleksandr Tymoshenko , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2013 10:11:11 -0000 Julian Elischer wrote: > > not sure if it's relevent, but remember that updating the screen mor ethan 50 > times a second is pointless. > I'm not sure if the curent video console does it but having the final copy only > done on 50Hz ticks can save a lot of copying. > Right, but then you have to handle VSYNC hardware interrupts and copy framebuffer data from system memory to video memory. regards From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 25 01:18:57 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 469CDFA6; Mon, 25 Feb 2013 01:18:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 8C67323D; Mon, 25 Feb 2013 01:18:56 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hi8so2609003wib.13 for ; Sun, 24 Feb 2013 17:18:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=jj5pFUEuQcqQZoqdDb/eSAox1Y7fg5ISE8bVtG3pmsA=; b=iZQWRo62s/6hTaOM9n4rPu3c2KkBSqpKgCmnHFxk3SW1Fdh5PppUIYGF0unZdz6ED8 ZFVdj7SoM2cJtZ5DBsKxo0QkmkC6iZqzxXZsVGpbxCtMatopGIFSXZZMIos8jTtQ6EH/ Idvr+xXGC97Pgj67wikKOMsj0NdPcIXi7t1VEilLm4ik0QKcS84MzpBOWGPbaZtqd8wl WCzjNj3WCNokFFl6Nxlx8yBT5JmAtaCr9f41OV2P/p+D5i9rPurf54V8X12ToiBMYRrc uYsIqW7c15Yjl3D1kt6+9U687agqwzKZ1TkAcTQD82hOvFBB+9RPcp46y8kVEmlJqUim sAtQ== MIME-Version: 1.0 X-Received: by 10.180.24.229 with SMTP id x5mr8994657wif.17.1361755129242; Sun, 24 Feb 2013 17:18:49 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.74.194 with HTTP; Sun, 24 Feb 2013 17:18:49 -0800 (PST) In-Reply-To: <5129F54D.9040503@mail.ru> References: <511F879C.3030801@mail.ru> <51217854.8000508@freebsd.org> <5123D941.2050906@mail.ru> <51293505.1030706@freebsd.org> <5129F54D.9040503@mail.ru> Date: Sun, 24 Feb 2013 17:18:49 -0800 X-Google-Sender-Auth: W84RScLeNXm_wf4ZY0ya-eC2oXQ Message-ID: Subject: Re: Scrolling in framebuffer syscons From: Adrian Chadd To: Phileas Fogg Content-Type: text/plain; charset=ISO-8859-1 Cc: Oleksandr Tymoshenko , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2013 01:18:57 -0000 On 24 February 2013 03:11, Phileas Fogg wrote: > Julian Elischer wrote: > >> >> not sure if it's relevent, but remember that updating the screen mor ethan >> 50 >> times a second is pointless. >> I'm not sure if the curent video console does it but having the final copy >> only >> done on 50Hz ticks can save a lot of copying. >> > > Right, but then you have to handle VSYNC hardware interrupts and copy > framebuffer data from system memory to video memory. Which you likely should do anyway, right? if you're going to be doing any kind of large-scale screen updating. Adrian Adrian From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 25 08:34:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 506E2820 for ; Mon, 25 Feb 2013 08:34:55 +0000 (UTC) (envelope-from jean-sebastien.pedron@dumbbell.fr) Received: from mail.made4.biz (unknown [IPv6:2001:41d0:1:7018::1:3]) by mx1.freebsd.org (Postfix) with ESMTP id E964E6A1 for ; Mon, 25 Feb 2013 08:34:54 +0000 (UTC) Received: from [2001:1b48:10b:cafe:225:64ff:febe:589f] (helo=viking.yzserv.com) by mail.made4.biz with esmtpsa (TLSv1:DHE-RSA-CAMELLIA256-SHA:256) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1U9tWM-00071A-71 for freebsd-hackers@freebsd.org; Mon, 25 Feb 2013 09:34:53 +0100 Message-ID: <512B2226.4070402@dumbbell.fr> Date: Mon, 25 Feb 2013 09:34:46 +0100 From: =?ISO-8859-1?Q?Jean-S=E9bastien_P=E9dron?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Building a program to be used only during make buildkernel References: <5127746A.6060702@dumbbell.fr> <20130222165548.GM2598@kib.kiev.ua> In-Reply-To: <20130222165548.GM2598@kib.kiev.ua> X-Enigmail-Version: 1.5.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="----enig2NSFMRSTKONEULXPKWPUT" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2013 08:34:55 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) ------enig2NSFMRSTKONEULXPKWPUT Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 22.02.2013 17:55, Konstantin Belousov wrote: > On Fri, Feb 22, 2013 at 02:36:42PM +0100, Jean-S?bastien P?dron wrote: >> Hello, >> >> During the build of the "radeon" DRM driver in Linux, a C program is >> first built to generate headers included by the driver. The program is= >> not meant to be installed. >> >> In FreeBSD, how would one build and use such a program during the buil= d >> of a kernel module? And what if the driver is built into the kernel in= stead? >=20 > Look at the genasssym program, used by the kernel and several ABI modul= es > for exactly this use. Thanks for the hint! I looked at sys/modules/linux/Makefile. If I understand it correctly, there's no special magic in there, just regular Makefile targets/rules, right? Here's what I did: ---8<--- =2EPATH: ${.CURDIR}/../../../dev/drm2/radeon (...) MKREGTABLE=3D ./mkregtable REG_SRCS_DIR=3D ${.CURDIR}/../../../dev/drm2/radeon/reg_srcs ${MKREGTABLE}: mkregtable.c @ ${CC} -o $@ @/dev/drm2/radeon/mkregtable.c rn50_reg_safe.h: ${REG_SRCS_DIR}/rn50 ${MKREGTABLE} ${MKREGTABLE} ${REG_SRCS_DIR}/rn50 > $@ (...) CLEANFILES=3D \ rn50_reg_safe.h \ ... \ ${MKREGTABLE} ---8<--- It's working for me, but I don't know if it's correct. In particular: o I'm not sure "${MKREGTABLE}" should depend on "@". One target in modules/linux/Makefile has it, and I blindly copied it... o I needed to use the full path to mkregtable.c (@/dev/drm2/radeon/mkregtable.c). I thought I could use ${.IMPSRC} or "mkregtable.c" and the source would have been found magically. I haven't tried to build the module into the kernel yet. --=20 Jean-S=E9bastien P=E9dron ------enig2NSFMRSTKONEULXPKWPUT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlErIioACgkQa+xGJsFYOlOjPwCfeaF11XPAjd4B/vHYZRS3Nklm fQAAnjNUL0Kr13oM6KWTIEf7gpShelI4 =ZQxt -----END PGP SIGNATURE----- ------enig2NSFMRSTKONEULXPKWPUT-- From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 25 09:53:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 70370994; Mon, 25 Feb 2013 09:53:16 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 845C2924; Mon, 25 Feb 2013 09:53:14 +0000 (UTC) Received: from terran (unknown [192.168.99.1]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id 6EF50C4946; Mon, 25 Feb 2013 11:53:07 +0200 (EET) Date: Mon, 25 Feb 2013 11:53:40 +0200 From: Aleksandr Rybalko To: Julian Elischer Subject: Re: Scrolling in framebuffer syscons Message-Id: <20130225115340.28ce773b2a7f305cc519de11@ddteam.net> In-Reply-To: <51293505.1030706@freebsd.org> References: <511F879C.3030801@mail.ru> <51217854.8000508@freebsd.org> <5123D941.2050906@mail.ru> <51293505.1030706@freebsd.org> X-Mailer: Sylpheed 3.2.0 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Phileas Fogg , Oleksandr Tymoshenko , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2013 09:53:16 -0000 On Sat, 23 Feb 2013 13:30:45 -0800 Julian Elischer wrote: > On 2/19/13 11:57 AM, Phileas Fogg wrote: > > Oleksandr Tymoshenko wrote: > >> On 2/16/2013 5:20 AM, Phileas Fogg wrote: > >>> Hi, > >>> > >>> i have a question about how the scrolling in a framebuffer syscons > >>> works. > >>> I'm trying to speed up the syscons on the PS3 console which is a > >>> simple > >>> framebuffer syscons. > >>> It uses the renderer _gfbrndrsw_ (see dev/syscons/scgfbrndr.c) to > >>> draw into > >>> the framebuffer of the PS3 console. > >>> The _gfb_draw_ function implements a simple scrolling that moves > >>> data from > >>> bottom to top with _vidd_copy_. > >>> And that's where i have a problem because _vidd_copy_ calls the > >>> function > >>> _ps3fb_copy_ (see powerpc/ps3/ps3_syscons.c). > >>> But the function _ps3fb_copy_ is NOT implemented yet. So, the > >>> question is how > >>> does the scrolling work then ? > >>> I took a look at other syscons implementation based on a > >>> framebuffer, and > >>> almost all of them do NOT implement _vidd_copy_ > >>> function, e.g. XBOX syscons. > >> > >> I think driver re-renders whole screen character by character. > >> That's why no > >> copy operation is invoked. > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to > >> "freebsd-hackers-unsubscribe@freebsd.org" > > > > You are right, syscons is using the teken terminal emulator which > > implements scrolling in software. The vidd_copy callback is never > > called. To optimize the PS3 syscons driver, i have to speed up > > vidd_puts and vidd_putc callbacks. > > not sure if it's relevent, but remember that updating the screen mor > ethan 50 times a second is pointless. > I'm not sure if the curent video console does it but having the final > copy only done on 50Hz ticks can save a lot of copying. Completely irrelevant :) syscons draw to framebuffer only when he need to do so. Framebuffer can reside in graphic card memory or in main memory. In first case it is served(GPU fetch data and put it to DAC pixel by pixel) by graphic card internally, in second case it served with graphic hardware own DMA or host DMA. It is possible to do it by software, but I don't remember such driver in our tree and it is too heavy. Another one possible case - asynchronous displays, which have internal memory (internal framebuffer). > > > > > regards > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to > > "freebsd-hackers-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Aleksandr Rybalko From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 25 12:16:15 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 54284D1F for ; Mon, 25 Feb 2013 12:16:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id A53B96E0 for ; Mon, 25 Feb 2013 12:16:14 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r1PCG5Ys061463; Mon, 25 Feb 2013 14:16:05 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r1PCG5Ys061463 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r1PCG5Jw061462; Mon, 25 Feb 2013 14:16:05 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 25 Feb 2013 14:16:05 +0200 From: Konstantin Belousov To: Jean-S?bastien P?dron Subject: Re: Building a program to be used only during make buildkernel Message-ID: <20130225121605.GY2454@kib.kiev.ua> References: <5127746A.6060702@dumbbell.fr> <20130222165548.GM2598@kib.kiev.ua> <512B2226.4070402@dumbbell.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DWqF7Vcvgq9cBwZ0" Content-Disposition: inline In-Reply-To: <512B2226.4070402@dumbbell.fr> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2013 12:16:15 -0000 --DWqF7Vcvgq9cBwZ0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Feb 25, 2013 at 09:34:46AM +0100, Jean-S?bastien P?dron wrote: > I haven't tried to build the module into the kernel yet. IMO this is not a target for now. The KMS currently is not integrated with syscons at all, so startup of a KMS driver causes blank screen and lost console. The vidconsole is functional, but it has no use for the user who cannot see the output. This is the state of i915 driver, which does not provide a files glue for static linking into the kernel, for the described reason. Solving integration issues with syscons is rather pointless, both due to the baroque interface of the syscons, and due to the supposed replacement of syscons, available in the base/user/ed/newcons. --DWqF7Vcvgq9cBwZ0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRK1YEAAoJEJDCuSvBvK1BdzkP/RdH0bcGrqeC08yFwQzBpQXE J+PrOJcl9qAZNkA+M5IZCkjMPLLtGgpxOymiAMJZffaVfrYfK64V0KxvBMefzLcH fYM2gpoD1ZmX3CmFxVDn6hSbN8kC5m6lHsfHbUwsJGbsdvZArV7SkkYUPcEkIISq HTy8Cg0r1007L46kHJlBxP0D1XT2uA1UroZnzNX+JZ+hRYuVrmgmDgaltRkLsV3K uqPz7cWsuzijz766iYOroHqIZD4/UITIhPxr4xmmS9Dw0c4DlRiKsFqhh4ExNnRR 4OdrqmOc/aVKKw6tcNyzKpzYElN18sU5ZAKKwj5ZdYjrc6P/kdP7GyUGt/IqGjob LxLfoIC/uLumjsXlwVZmudS+817coIUDoAPc7mT00zvnvy7JA4EzAugGBD2xFR2Y ksVVmlGY/o45wW7m1hoXFP/Ow0CenGGftrcTD8KG5bNlKAILZ51zVtdmhYhxXOPE CVYf5XbM5S91So96YEu42WJxl3sT6TNuLhfMNMqiMtsUJeLFT588pC86fMN+7gqC z1UVXmhX25aRvC26Ke1u3neiRjcwARVB1EoBVwGS1Rx7Vw3MrRVnEU0ArBKO+Cjh uH60YW8PSoXp4tKv2fZcucZSvb3mr3pGHE7cYtK/Sz9NTuOO5VspM403LTgubVF2 AVJlD+G5veLi+2gNPsWp =e681 -----END PGP SIGNATURE----- --DWqF7Vcvgq9cBwZ0-- From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 15:19:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 59D39439 for ; Tue, 26 Feb 2013 15:19:02 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id C8B339DF for ; Tue, 26 Feb 2013 15:19:01 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r1QFIj0m027045; Tue, 26 Feb 2013 16:18:45 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r1QFIjQf027042; Tue, 26 Feb 2013 16:18:45 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 26 Feb 2013 16:18:44 +0100 (CET) From: Wojciech Puchar To: Vincent Hoffman Subject: Re: attaching iscsi (or ggate) disk before mounting root In-Reply-To: <51292900.6010400@unsane.co.uk> Message-ID: References: <51292900.6010400@unsane.co.uk> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 26 Feb 2013 16:18:45 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 15:19:02 -0000 thank you very much. i solved that problem other way, finally ending in NFS root. but i still need raw block device as swap device, which will (rarely) be in active use. What do you recommend - iscsi or geom_gate. the latter is 100 times more simple, and i like simple solutions. but geom_gate use userland process for I/O. Does it make a risk of deadlock? is iscsi deadlock free too? i mean iscsi in kernel needs to allocate memory to perform swapout, but it have to wait for free memory, but memory will not be freed until swapout is done. From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 20:52:43 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 59768C5D; Tue, 26 Feb 2013 20:52:43 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-lb0-f176.google.com (mail-lb0-f176.google.com [209.85.217.176]) by mx1.freebsd.org (Postfix) with ESMTP id B2C371EEE; Tue, 26 Feb 2013 20:52:42 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id s4so3389420lbc.7 for ; Tue, 26 Feb 2013 12:52:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=E5cHE9QK6vU7Mk/y/md6C5PLkNxsD9U8RLWGNRdUD0U=; b=csQ4k6m1EJZ2MEiQxVu220b2bCBY6bbZPnO8AUW1YF6E1KYg9WlqrJXx6RipqWzn8H hvnZb2vv/qKKuvfnva7c4gn8ZbuOwRu3dKpEUg4QnGd0Z2tjjOblDgw3ueHuw6uwcFVs Okbt668EVCiXZq3pMG2MZBO31Y6IWXoRdzI9goQBRMe24HyalqlXI1a42a6FZt0y26Ml NElz6sdP6z33LwX79IxhFeUBo3dn1d57rRuuESyQyukm3sEObtfOsB6X2AqV3CZuKZQ7 SnSx/P6BtC5YjEcaczvzdyM/ku9QVoXiV+fGH0qOLLXsr5/0PLcAaXCPiCyKIdmIZctR LdTg== X-Received: by 10.112.49.99 with SMTP id t3mr1132732lbn.108.1361911955946; Tue, 26 Feb 2013 12:52:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.152.20.138 with HTTP; Tue, 26 Feb 2013 12:52:15 -0800 (PST) In-Reply-To: References: <20130220154855.GF2598@kib.kiev.ua> <51253759.70508@coosemans.org> <20130221154433.GY2598@kib.kiev.ua> From: Damjan Jovanovic Date: Tue, 26 Feb 2013 22:52:15 +0200 Message-ID: Subject: Re: [patch] Wine DLL base address patches To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-emulation@freebsd.org, Tijl Coosemans X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 20:52:43 -0000 On Fri, Feb 22, 2013 at 5:19 AM, Damjan Jovanovic wrote: > On Thu, Feb 21, 2013 at 5:44 PM, Konstantin Belousov > wrote: >> On Thu, Feb 21, 2013 at 12:57:45AM +0200, Damjan Jovanovic wrote: >>> On Wed, Feb 20, 2013 at 10:51 PM, Tijl Coosemans wrote: >>> > On 20-02-2013 16:48, Konstantin Belousov wrote: >>> >> On Wed, Feb 20, 2013 at 05:29:01PM +0200, Damjan Jovanovic wrote: >>> >>> Hi >>> >>> >>> >>> Wine needs some of its libraries to be loaded at specific base >>> >>> addresses (https://wiki.freebsd.org/Wine), something FreeBSD currently >>> >>> lacks. >>> >>> >>> >>> I've written a patch to the dynamic loader (/libexec/ld-elf.so.1) that >>> >>> loads libraries at their preferred base addresses >>> >>> (http://www.freebsd.org/cgi/query-pr.cgi?pr=176216), as well as a port >>> >>> of Prelink to FreeBSD which Wine uses to set base addresses >>> >>> (http://www.freebsd.org/cgi/query-pr.cgi?pr=176283). Both work :-), >>> >>> the changed dynamic loader doesn't show any problems in a few days of >>> >>> testing, and prelink works with the --reloc-only option as used by >>> >>> Wine. >>> >>> >>> >>> Please review/test/comment/commit. >>> >> >>> >> Unfortunately, it is not safe. MAP_FIXED overrides any previous mappings >>> >> which could exist at the specified address. >>> > >>> > I've simplified the rtld patch to a single line. The second patch makes >>> > Wine use -Ttext-segment linker flag instead of prelink. This requires >>> > binutils from ports, but it's easier than porting prelink. >>> > >>> >>> All of that occurred to me as well. >>> >>> The problem with that one-line rtld patch is that loading an >>> application will now fail if any of its libraries cannot be loaded at >>> their requested address. >> But this is intended behaviour. Also, the default virtaddr base for the >> shared libraries is 0, so the existing binaries should be not affected. > > In that case, and since failing to load a library only causes the > process to exit when starting up and not when it calls dlopen(), I > approve. > >>> >>> The problem with -Ttext-segment (and isn't it just -Ttext?) is that it >>> doesn't seem to work: the base_vaddr seen by rtld will remain 0, and >>> the address listed in /proc/.../map is different from what it should >>> be. Also run "readelf -l" on a library compiled that way and compare >>> with the output of one run through "prelink --reloc-only", you'll see >>> the lowest VirtAddr and PhysAddr in LOAD headers change only with >>> prelink. I really ported prelink because there was no other choice. >> The -Ttext-segment does work. As indicated by Tijl, you need recent >> binutils. I just verified that ld 2.32.1 obeys -Ttext-segment. >> >> You can also take a look at the default linker script to see how >> -Ttext-segment is used, look for SEGMENT_START("text-segment"). >> > > My apologies: I confused -Ttext which is documented but doesn't work, > with -Ttext-segment which is undocumented in FreeBSD 9.1 and might > work. I would test it further, but -CURRENT doesn't installworld > (ERROR: Required auditdistd user is missing, see /usr/src/UPDATING.) > and I am away until next week. > > Prelink is now in Ports. What I'd recommend is checking if the > binaries are the same, and if not, doing a diff between "readelf -a" > outputs of the prelinked binary vs -Ttext-segmented binary. Also run > this a few times and make sure the address is what's expected: > > #include > #include > int main(int argc, char **argv) > { > printf("%p\n", LoadLibrary("KERNEL32")); > return 0; > } > > mingw32-gcc hello.c -o hello.exe > wine hello.exe With binutils 2.23.1 (in ports), comparing the output of "ld -Ttext-segment=0x7b800000" and "prelink --reloc-only 0x7b800000" using diffs of "readelf -a" outputs gives this: - 11: 000000007b800000 0 OBJECT LOCAL DEFAULT 6 _GLOBAL_OFFSET_TABLE_ + 11: 0000000000000000 0 OBJECT LOCAL DEFAULT 6 _GLOBAL_OFFSET_TABLE_ in other words, prelink also shifts the global offset table to the requested base address, ld does not. I don't think this matters since it's only ELF segments that get loaded - sections are irrelevant. "objdump -s" finds no differences. So I am happy with all of Tijl's patches, please commit them. From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 21:04:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 81CAE79; Tue, 26 Feb 2013 21:04:29 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) by mx1.freebsd.org (Postfix) with ESMTP id B46E11FED; Tue, 26 Feb 2013 21:04:28 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id ek20so4412639lab.30 for ; Tue, 26 Feb 2013 13:04:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=S1fp+DhwxGD3yGlw5/SqvdzS7+Np6kgwtMH1VixTGWY=; b=Ei8eSb9GUoRBqYo90dyt6qVaIyUiFFdca4LQlsJ889oSYu/bZWPX4sFVfzMSM5wE3p ohHA2Cale/nEzWt1FsRE1wpyicEbsWKIrJdIgQmjjivwIr8IQ8uBpO7AL9RvF9AENQ0H t2rHuEOZh70nBnduIlDUim4IlMiIWLJMhwMqQwYQziGcmow+aXqvL8zkUsvH+YM2YOuE 3/jn6XcA21ht6PvciOJ1WWLyyyTCmRW68iiEKFSz94/pkcYrn9JUA47AlbR7vaUK532n i48sNyCNd3HBg1A6BNzt3+WQPv7kExuY6EGJovxbPOeWX28HJXAplKArZzRGcDRTpwQK qShQ== X-Received: by 10.112.37.194 with SMTP id a2mr1162947lbk.40.1361912667364; Tue, 26 Feb 2013 13:04:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.152.20.138 with HTTP; Tue, 26 Feb 2013 13:04:07 -0800 (PST) In-Reply-To: <20130221213528.GD92116@felucia.tataz.chchile.org> References: <20130221213528.GD92116@felucia.tataz.chchile.org> From: Damjan Jovanovic Date: Tue, 26 Feb 2013 23:04:07 +0200 Message-ID: Subject: Re: [patch] Wine DLL base address patches To: Damjan Jovanovic , freebsd-emulation@freebsd.org, freebsd-hackers@freebsd.org, tijl@coosemans.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 21:04:29 -0000 On Thu, Feb 21, 2013 at 11:35 PM, Jeremie Le Hen wrote: > Hi Damjan, > > On Wed, Feb 20, 2013 at 05:29:01PM +0200, Damjan Jovanovic wrote: >> >> Wine needs some of its libraries to be loaded at specific base >> addresses (https://wiki.freebsd.org/Wine), something FreeBSD currently >> lacks. >> >> I've written a patch to the dynamic loader (/libexec/ld-elf.so.1) that >> loads libraries at their preferred base addresses >> (http://www.freebsd.org/cgi/query-pr.cgi?pr=176216), as well as a port >> of Prelink to FreeBSD which Wine uses to set base addresses >> (http://www.freebsd.org/cgi/query-pr.cgi?pr=176283). Both work :-), >> the changed dynamic loader doesn't show any problems in a few days of >> testing, and prelink works with the --reloc-only option as used by >> Wine. > > Thanks for this work. > > Out of curiosity, did you try to run prelink on the whole base system? > If yes, did you make any measurement of the performance improvement? > > Thank you! > -- > Jeremie Le Hen > > Scientists say the world is made up of Protons, Neutrons and Electrons. > They forgot to mention Morons. I didn't, please let us know if you do. IMO it would be a lot better to add -Bdirect support like Solaris has (http://sourceware.org/ml/binutils/2005-10/msg00436.html), both for better performance and because it works around ELF's moronic idea of resolving symbols by searching for them in all libraries in the order they were loaded, which easily causes memory corruption and crashes if eg. 2 versions of a library are loaded into the same process. From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 22:09:27 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1F093FB1; Tue, 26 Feb 2013 22:09:27 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id D71B4341; Tue, 26 Feb 2013 22:09:26 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0F391B95E; Tue, 26 Feb 2013 17:09:26 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: intr_event_handle never sends EOI if we fail to schedule ithread Date: Tue, 26 Feb 2013 15:25:48 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201302261525.48603.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 26 Feb 2013 17:09:26 -0500 (EST) Cc: FreeBSD Hackers , Ryan Stone X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 22:09:27 -0000 On Thursday, February 21, 2013 5:10:44 pm Ryan Stone wrote: > I recently saw an issue where all interrupts from a particular interrupt > vector were never raised. After investigating it appears that I ran into > the bug fixed in r239095[1], where an interrupt handler that uses an > ithread was added to the list of interrupt handlers for a particular event > before the ithread was allocated. If an interrupt for this event > (presumably for a different device sharing the same interrupt line) comes > in after the handler is added to the list but before the ithread has been > allocated we hit the following code: > > if (thread) { > if (ie->ie_pre_ithread != NULL) > ie->ie_pre_ithread(ie->ie_source); > } else { > if (ie->ie_post_filter != NULL) > ie->ie_post_filter(ie->ie_source); > } > > /* Schedule the ithread if needed. */ > if (thread) { > error = intr_event_schedule_thread(ie); > #ifndef XEN > KASSERT(error == 0, ("bad stray interrupt")); > #else > if (error != 0) > log(LOG_WARNING, "bad stray interrupt"); > #endif > } > > thread is true, so we will not run ie_post_filter (which would send the > EOI). However because the ithread has not been allocated > intr_event_schedule_thread will return an error. If INVARIANTS is not > defined we skip the KASSERT and return. > > Now, r239095 fixes this scenario, but I think that we should call > ie_post_filter whenever intr_event_schedule_thread fails to ensure that we > don't block an interrupt vector indefinitely. Any comments? Actually, I think you want to call post_ithread as you've already called pre_ithread? Also, pre_ithread should already EOI the interrupt, the problem is that it leaves it masked, and you need to invoke post_ithread to unmask it. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 22:09:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E0FB6FB5; Tue, 26 Feb 2013 22:09:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id C0824344; Tue, 26 Feb 2013 22:09:29 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 1EEDCB96C; Tue, 26 Feb 2013 17:09:29 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: rtprio_thread trouble Date: Tue, 26 Feb 2013 15:29:24 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <1361559960.1185.81.camel@revolution.hippie.lan> In-Reply-To: <1361559960.1185.81.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201302261529.24862.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 26 Feb 2013 17:09:29 -0500 (EST) Cc: Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 22:09:29 -0000 On Friday, February 22, 2013 2:06:00 pm Ian Lepore wrote: > I ran into some trouble with rtprio_thread() today. I have a worker > thread that I want to run at idle priority most of the time, but if it > falls too far behind I'd like to bump it back up to regular timeshare > priority until it catches up. In a worst case, the system is > continuously busy and something scheduled at idle priority is never > going to run (for some definition of 'never'). > > What I found is that in this worst case, even after my main thread has > used rtprio_thread() to change the worker thread back to > RTP_PRIO_NORMAL, the worker thread never gets scheduled. This is with > the 4BSD scheduler but it appears that the same would be the case with > ULE, based on code inspection. I find that this fixes it for 4BSD, and > I think the same would be true for ULE... > > --- a/sys/kern/sched_4bsd.c Wed Feb 13 12:54:36 2013 -0700 > +++ b/sys/kern/sched_4bsd.c Fri Feb 22 11:55:35 2013 -0700 > @@ -881,6 +881,9 @@ sched_user_prio(struct thread *td, u_cha > return; > oldprio = td->td_user_pri; > td->td_user_pri = prio; > + if (td->td_flags & TDF_BORROWING && td->td_priority <= prio) > + return; > + sched_priority(td, prio); > } > > void > > But I'm not sure if this would have any negative side effects, > especially since in the ULE case there's a comment on this function that > specifically notes that it changes the the user priority without > changing the current priority (but it doesn't say why that matters). > > Is this a reasonable way to fix this problem, or is there a better way? This will lose the "priority boost" afforded to interactive threads when they sleep in the kernel in the 4BSD scheduler. You aren't supposed to drop the user priority to loose this boost until userret(). You could perhaps try only altering the priority if the new user pri is lower than your current priority (and then you don't have to check TDF_BORROWING I believe): if (prio < td->td_priority) sched_priority(td, prio); -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 26 22:09:32 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 022F4FB7; Tue, 26 Feb 2013 22:09:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id D488F346; Tue, 26 Feb 2013 22:09:31 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3603DB976; Tue, 26 Feb 2013 17:09:31 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: why no per-thread scheduling niceness? Date: Tue, 26 Feb 2013 15:30:37 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <1361560374.1185.85.camel@revolution.hippie.lan> In-Reply-To: <1361560374.1185.85.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201302261530.37936.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 26 Feb 2013 17:09:31 -0500 (EST) Cc: Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 22:09:32 -0000 On Friday, February 22, 2013 2:12:54 pm Ian Lepore wrote: > I'm curious why the concept of scheduling niceness applies only to an > entire process, and it's not possible to have nice threads within a > process. Is there any fundamental reason why it couldn't be supported > with some extra bookkeeping to track niceness per thread? Only that the existing 'nice' command only works on processes and nice is traditionally a process concept. Also see things like renice. Individual threads can already alter their priority somewhat (e.g. to set an individual thread to an idle or real-time priority). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 27 06:24:58 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CA284556; Wed, 27 Feb 2013 06:24:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 26F08C03; Wed, 27 Feb 2013 06:24:57 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r1R6Op64009944; Wed, 27 Feb 2013 08:24:51 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r1R6Op64009944 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r1R6OoK8009943; Wed, 27 Feb 2013 08:24:50 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 27 Feb 2013 08:24:50 +0200 From: Konstantin Belousov To: Damjan Jovanovic Subject: Re: [patch] Wine DLL base address patches Message-ID: <20130227062450.GX2454@kib.kiev.ua> References: <20130220154855.GF2598@kib.kiev.ua> <51253759.70508@coosemans.org> <20130221154433.GY2598@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Pui5YDBJbCQuJ1A1" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org, freebsd-emulation@freebsd.org, Tijl Coosemans X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2013 06:24:58 -0000 --Pui5YDBJbCQuJ1A1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 26, 2013 at 10:52:15PM +0200, Damjan Jovanovic wrote: > On Fri, Feb 22, 2013 at 5:19 AM, Damjan Jovanovic = wrote: > > On Thu, Feb 21, 2013 at 5:44 PM, Konstantin Belousov > > wrote: > >> On Thu, Feb 21, 2013 at 12:57:45AM +0200, Damjan Jovanovic wrote: > >>> On Wed, Feb 20, 2013 at 10:51 PM, Tijl Coosemans = wrote: > >>> > On 20-02-2013 16:48, Konstantin Belousov wrote: > >>> >> On Wed, Feb 20, 2013 at 05:29:01PM +0200, Damjan Jovanovic wrote: > >>> >>> Hi > >>> >>> > >>> >>> Wine needs some of its libraries to be loaded at specific base > >>> >>> addresses (https://wiki.freebsd.org/Wine), something FreeBSD curr= ently > >>> >>> lacks. > >>> >>> > >>> >>> I've written a patch to the dynamic loader (/libexec/ld-elf.so.1)= that > >>> >>> loads libraries at their preferred base addresses > >>> >>> (http://www.freebsd.org/cgi/query-pr.cgi?pr=3D176216), as well as= a port > >>> >>> of Prelink to FreeBSD which Wine uses to set base addresses > >>> >>> (http://www.freebsd.org/cgi/query-pr.cgi?pr=3D176283). Both work = :-), > >>> >>> the changed dynamic loader doesn't show any problems in a few day= s of > >>> >>> testing, and prelink works with the --reloc-only option as used by > >>> >>> Wine. > >>> >>> > >>> >>> Please review/test/comment/commit. > >>> >> > >>> >> Unfortunately, it is not safe. MAP_FIXED overrides any previous ma= ppings > >>> >> which could exist at the specified address. > >>> > > >>> > I've simplified the rtld patch to a single line. The second patch m= akes > >>> > Wine use -Ttext-segment linker flag instead of prelink. This requir= es > >>> > binutils from ports, but it's easier than porting prelink. > >>> > > >>> > >>> All of that occurred to me as well. > >>> > >>> The problem with that one-line rtld patch is that loading an > >>> application will now fail if any of its libraries cannot be loaded at > >>> their requested address. > >> But this is intended behaviour. Also, the default virtaddr base for the > >> shared libraries is 0, so the existing binaries should be not affected. > > > > In that case, and since failing to load a library only causes the > > process to exit when starting up and not when it calls dlopen(), I > > approve. > > > >>> > >>> The problem with -Ttext-segment (and isn't it just -Ttext?) is that it > >>> doesn't seem to work: the base_vaddr seen by rtld will remain 0, and > >>> the address listed in /proc/.../map is different from what it should > >>> be. Also run "readelf -l" on a library compiled that way and compare > >>> with the output of one run through "prelink --reloc-only", you'll see > >>> the lowest VirtAddr and PhysAddr in LOAD headers change only with > >>> prelink. I really ported prelink because there was no other choice. > >> The -Ttext-segment does work. As indicated by Tijl, you need recent > >> binutils. I just verified that ld 2.32.1 obeys -Ttext-segment. > >> > >> You can also take a look at the default linker script to see how > >> -Ttext-segment is used, look for SEGMENT_START("text-segment"). > >> > > > > My apologies: I confused -Ttext which is documented but doesn't work, > > with -Ttext-segment which is undocumented in FreeBSD 9.1 and might > > work. I would test it further, but -CURRENT doesn't installworld > > (ERROR: Required auditdistd user is missing, see /usr/src/UPDATING.) > > and I am away until next week. > > > > Prelink is now in Ports. What I'd recommend is checking if the > > binaries are the same, and if not, doing a diff between "readelf -a" > > outputs of the prelinked binary vs -Ttext-segmented binary. Also run > > this a few times and make sure the address is what's expected: > > > > #include > > #include > > int main(int argc, char **argv) > > { > > printf("%p\n", LoadLibrary("KERNEL32")); > > return 0; > > } > > > > mingw32-gcc hello.c -o hello.exe > > wine hello.exe >=20 >=20 > With binutils 2.23.1 (in ports), comparing the output of "ld > -Ttext-segment=3D0x7b800000" and "prelink --reloc-only 0x7b800000" using > diffs of "readelf -a" outputs gives this: > - 11: 000000007b800000 0 OBJECT LOCAL DEFAULT 6 > _GLOBAL_OFFSET_TABLE_ > + 11: 0000000000000000 0 OBJECT LOCAL DEFAULT 6 > _GLOBAL_OFFSET_TABLE_ > in other words, prelink also shifts the global offset table to the > requested base address, ld does not. I don't think this matters since > it's only ELF segments that get loaded - sections are irrelevant. I suspect that it is sort of bug in ld. On the other hand, _G_O_T_ symbol should be not used for real relocations, because both i386 and amd64 define specific relocations which allow to reference the begining of the GOT. The symbol is mostly a symbolic way to generate the relocations. So indeed, this should be fine. >=20 > "objdump -s" finds no differences. >=20 > So I am happy with all of Tijl's patches, please commit them. I expect Tijl to do it himself. --Pui5YDBJbCQuJ1A1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRLaayAAoJEJDCuSvBvK1ByIgP/i4ZOBfVNJmTgruq/BUTrUs9 QnK1Xy0AnX7U1QupTnnbm6ehK0PKCDix2FPvRdLHZnLkwtJwVs4hbVHpqyyqVkU3 tujfI8IC2gVOpDQdNz/BzTTmZs2M7hjifty+lXQorJs1msGrYjmf6xyPNoZq7+X0 uWmkwIEW7PQQrGBrxyON2sagThasn/GFEEPzRQc3VApDp/XHPzO3eoKButvoaWR4 EtcpkuacpENcpyjASpzttogTCFIrrH6+2l7wod0ZF2g8yDo6Zx+d7U2tkhCaaNXW /fiBSTrErElNMJ5Z1m4V5nuARjZJg4jO0Q+ln+u7S9ncOAqb3WD6q/+VHXOs+5Mj 5Bi/CqgOBWka2LRI1vJc/Q+ao1ZxogWc3pX2TOZcPn46aDcDKuZiqfj/fdgLDBZA 3u6ueEv2pO1bEBvVCzy2UjB7moKADZFw4Rw9dRZZ4Mv+NFju7KMwSojc4GqXL0/0 xohx0/nlfYC/hGZK8hJJYWcyphmXi4420k1eLPRp8g33GpiMO1yvH4rlV603rWt5 oF5+eYFmIDr8712CEupoOGdkNCA5kooPiHd0Bx8ZxkVqiQzgtwV3wWdFt3iattm6 +a3dFNDb6GxJ+xbvGlBpjqo7zUnuAazytijwoI/gXt6em0I9rBQgfsDD7/p73Fp1 yt3AVTXjNyAUwOZ0xwGx =hCIn -----END PGP SIGNATURE----- --Pui5YDBJbCQuJ1A1-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 27 21:31:44 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 87792D5E for ; Wed, 27 Feb 2013 21:31:44 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id 5166C11E for ; Wed, 27 Feb 2013 21:31:44 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 54664358C61 for ; Wed, 27 Feb 2013 22:31:42 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 3F0D52848C; Wed, 27 Feb 2013 22:31:42 +0100 (CET) Date: Wed, 27 Feb 2013 22:31:42 +0100 From: Jilles Tjoelker To: freebsd-hackers@freebsd.org Subject: [patch] statfs does not detect -t nullfs -o union as a union mount Message-ID: <20130227213141.GA18210@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2013 21:31:44 -0000 While testing recent changes to opendir(), I noticed that fstatfs() does not return the MNT_UNION flag for a -t nullfs -o union mount. As a result, opendir()/readdir() return files that exist in both top and bottom directories twice (at least . and ..). Other -o union mounts and -t unionfs mounts work correctly in this regard. The below patch passes through just the MNT_UNION flag of the nullfs mount itself. Perhaps more flags should be passed through. commit fce32a779af4eb977c9b96feb6e4f811d89f2881 Author: Jilles Tjoelker Date: Sat Feb 23 22:22:39 2013 +0100 nullfs: Preserve the MNT_UNION flag of the nullfs mount itself. This is needed so that opendir() can properly detect a union mount like mount -t nullfs -o union dir1 dir2. diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index 3724e0a..ff06f57 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -313,7 +313,7 @@ nullfs_statfs(mp, sbp) /* now copy across the "interesting" information and fake the rest */ sbp->f_type = mstat.f_type; - sbp->f_flags = mstat.f_flags; + sbp->f_flags = (sbp->f_flags & MNT_UNION) | mstat.f_flags; sbp->f_bsize = mstat.f_bsize; sbp->f_iosize = mstat.f_iosize; sbp->f_blocks = mstat.f_blocks; -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 27 22:21:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C965958C for ; Wed, 27 Feb 2013 22:21:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 6CC69703 for ; Wed, 27 Feb 2013 22:21:48 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r1RMLiOj096638; Thu, 28 Feb 2013 00:21:44 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.0 kib.kiev.ua r1RMLiOj096638 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r1RMLi3e096637; Thu, 28 Feb 2013 00:21:44 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Feb 2013 00:21:44 +0200 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: [patch] statfs does not detect -t nullfs -o union as a union mount Message-ID: <20130227222144.GG2454@kib.kiev.ua> References: <20130227213141.GA18210@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6DRyk0WLoccTprpY" Content-Disposition: inline In-Reply-To: <20130227213141.GA18210@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2013 22:21:48 -0000 --6DRyk0WLoccTprpY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 27, 2013 at 10:31:42PM +0100, Jilles Tjoelker wrote: > While testing recent changes to opendir(), I noticed that fstatfs() does > not return the MNT_UNION flag for a -t nullfs -o union mount. As a > result, opendir()/readdir() return files that exist in both top and > bottom directories twice (at least . and ..). Other -o union mounts and > -t unionfs mounts work correctly in this regard. >=20 > The below patch passes through just the MNT_UNION flag of the nullfs > mount itself. Perhaps more flags should be passed through. >=20 > commit fce32a779af4eb977c9b96feb6e4f811d89f2881 > Author: Jilles Tjoelker > Date: Sat Feb 23 22:22:39 2013 +0100 >=20 > nullfs: Preserve the MNT_UNION flag of the nullfs mount itself. > =20 > This is needed so that opendir() can properly detect a union mount li= ke > mount -t nullfs -o union dir1 dir2. >=20 > diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c > index 3724e0a..ff06f57 100644 > --- a/sys/fs/nullfs/null_vfsops.c > +++ b/sys/fs/nullfs/null_vfsops.c > @@ -313,7 +313,7 @@ nullfs_statfs(mp, sbp) > =20 > /* now copy across the "interesting" information and fake the rest */ > sbp->f_type =3D mstat.f_type; > - sbp->f_flags =3D mstat.f_flags; > + sbp->f_flags =3D (sbp->f_flags & MNT_UNION) | mstat.f_flags; > sbp->f_bsize =3D mstat.f_bsize; > sbp->f_iosize =3D mstat.f_iosize; > sbp->f_blocks =3D mstat.f_blocks; Would it make sense to preserve more flags from the upper mount ? I see a use for MNT_NOEXEC as well, at least. --6DRyk0WLoccTprpY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRLob3AAoJEJDCuSvBvK1BgIAQAKDwSxbw59Wuz5dnwHYm9Shs Aj8eFjdkItqRh3iU4apkyjSBt+eQh1teowtPphftGcfoUZZ8ZhLAS/a9023niRtB t/9pfe1zckgG89ZZKOO9mQP1dj9TYwILAhuDh7/W0Ly7GXVNzi0uhDc8ZET5Ickg aGd1/ALv+uEcVpdqv1IeiReUTGOTXh4p4UvMLxLQj8NUuUJL62Zhg/tTGaQ8FgjD kAmC8SRbs78ZEayKi+wqh0k1Z5sNPWAsl11wLqgJfJSM7fVDQVIaaK0P4LOmkvvo i5v35SNE7EU56WulLXn/yFoBWYeS+/uxVUNhga8aW633ZP/Q+MpUErkufLkGfbfz NzZSyskWDK6MvzlFqzFJoDPnBtplrvYbbl+pYVlXTrooGwcQoWi3lEdVeaAtWAMN gcnuKxUDUonKb/alFvR6cm/YU2Q8ruF12taZvtrrsPtHZI9hPL6aSCbbkJrVN2P4 /VMWT3/Kn77Rx8GkyZOga3LoL+neDFdQwLPh1qQV+NKfR/4UXCDjRhE5vawJLxSh p2VVgj///OWrADYS03BP7h17ZtCw5RHWoBvLZ9HQ8wMTtZS74zKCuhdLP1k/IOjm jvO3QiNtlWLR0GlA1GpYy9v5VBQ2L1/Cymf/d02q8HsqXOh+nNrX5W7YhU3XqXX4 yvgapzlIY2W5sLCRpzR6 =zAH7 -----END PGP SIGNATURE----- --6DRyk0WLoccTprpY-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 28 21:11:09 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 92137A86; Thu, 28 Feb 2013 21:11:09 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) by mx1.freebsd.org (Postfix) with ESMTP id 5DDCB23A; Thu, 28 Feb 2013 21:11:08 +0000 (UTC) Received: from c-24-8-232-202.hsd1.co.comcast.net ([24.8.232.202] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1UB9dO-0007n7-OQ; Thu, 28 Feb 2013 19:59:18 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r1SJxGjU085690; Thu, 28 Feb 2013 12:59:16 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.232.202 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18T80ULGcz2T7nsdkwef8bU Subject: Re: rtprio_thread trouble From: Ian Lepore To: John Baldwin In-Reply-To: <201302261529.24862.jhb@freebsd.org> References: <1361559960.1185.81.camel@revolution.hippie.lan> <201302261529.24862.jhb@freebsd.org> Content-Type: multipart/mixed; boundary="=-fruE5NV5DEzFjKJ3O5EF" Date: Thu, 28 Feb 2013 12:59:16 -0700 Message-ID: <1362081556.1195.62.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2013 21:11:09 -0000 --=-fruE5NV5DEzFjKJ3O5EF Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Tue, 2013-02-26 at 15:29 -0500, John Baldwin wrote: > On Friday, February 22, 2013 2:06:00 pm Ian Lepore wrote: > > I ran into some trouble with rtprio_thread() today. I have a worker > > thread that I want to run at idle priority most of the time, but if it > > falls too far behind I'd like to bump it back up to regular timeshare > > priority until it catches up. In a worst case, the system is > > continuously busy and something scheduled at idle priority is never > > going to run (for some definition of 'never'). > > > > What I found is that in this worst case, even after my main thread has > > used rtprio_thread() to change the worker thread back to > > RTP_PRIO_NORMAL, the worker thread never gets scheduled. This is with > > the 4BSD scheduler but it appears that the same would be the case with > > ULE, based on code inspection. I find that this fixes it for 4BSD, and > > I think the same would be true for ULE... > > > > --- a/sys/kern/sched_4bsd.c Wed Feb 13 12:54:36 2013 -0700 > > +++ b/sys/kern/sched_4bsd.c Fri Feb 22 11:55:35 2013 -0700 > > @@ -881,6 +881,9 @@ sched_user_prio(struct thread *td, u_cha > > return; > > oldprio = td->td_user_pri; > > td->td_user_pri = prio; > > + if (td->td_flags & TDF_BORROWING && td->td_priority <= prio) > > + return; > > + sched_priority(td, prio); > > } > > > > void > > > > But I'm not sure if this would have any negative side effects, > > especially since in the ULE case there's a comment on this function that > > specifically notes that it changes the the user priority without > > changing the current priority (but it doesn't say why that matters). > > > > Is this a reasonable way to fix this problem, or is there a better way? > > This will lose the "priority boost" afforded to interactive threads when they > sleep in the kernel in the 4BSD scheduler. You aren't supposed to drop the > user priority to loose this boost until userret(). You could perhaps try > only altering the priority if the new user pri is lower than your current > priority (and then you don't have to check TDF_BORROWING I believe): > > if (prio < td->td_priority) > sched_priority(td, prio); > That's just the sort of insight I was looking for, thanks. That made me look at the code more and think harder about the problem I'm trying to solve, and I concluded that doing it within the scheduler is all wrong. That led me to look elsewhere, and I discovered the change you made in r228207, which does almost what I want, but your change does it only for realtime priorities, and I need a similar effect for idle priorities. What I came up with is a bit different than yours (attached below) and I'd like your thoughts on it. I start with the same test as yours: if sched_user_prio() didn't actually change the user priority (due to borrowing), do nothing. Then mine differs: call sched_prio() to effect the change only if either the old or the new priority class is not timeshare. My reasoning for the second half of the test is that if it's a change in timeshare priority then the scheduler is going to adjust that priority in a way that completely wipes out the requested change anyway, so what's the point? (If that's not true, then allowing a thread to change its own timeshare priority would subvert the scheduler's adjustments and let a cpu-bound thread monopolize the cpu; if allowed at all, that should require priveleges.) On the other hand, if either the old or new priority class is not timeshare, then the scheduler doesn't make automatic adjustments, so we should honor the request and make the priority change right away. The reason the old class gets caught up in this is the very reason I'm wanting to make a change: when thread A changes the priority of its child thread B from idle back to timeshare, thread B never actually gets moved to a timeshare-range run queue unless there are some idle cycles available to allow it to first get scheduled again as an idle thread. Finally, my change doesn't consider the td == curthread situation at all, because I don't see how that's germane. This is the thing I'm least sure of -- I don't at all understand why the old code (even before your changes) had that test. The old code had that flagged as "XXX dubious" (a comment a bit too cryptic to be useful). -- Ian --=-fruE5NV5DEzFjKJ3O5EF Content-Disposition: inline; filename="rtprio_thread.diff" Content-Type: text/x-patch; name="rtprio_thread.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: sys/kern/kern_resource.c =================================================================== --- sys/kern/kern_resource.c (revision 247421) +++ sys/kern/kern_resource.c (working copy) @@ -469,8 +469,7 @@ sys_rtprio(td, uap) int rtp_to_pri(struct rtprio *rtp, struct thread *td) { - u_char newpri; - u_char oldpri; + u_char newpri, oldpri, oldclass; switch (RTP_PRIO_BASE(rtp->type)) { case RTP_PRIO_REALTIME: @@ -493,11 +492,12 @@ rtp_to_pri(struct rtprio *rtp, struct thread *td) } thread_lock(td); + oldclass = td->td_pri_class; sched_class(td, rtp->type); /* XXX fix */ oldpri = td->td_user_pri; sched_user_prio(td, newpri); - if (td->td_user_pri != oldpri && (td == curthread || - td->td_priority == oldpri || td->td_user_pri <= PRI_MAX_REALTIME)) + if (td->td_user_pri != oldpri && (oldclass != RTP_PRIO_NORMAL || + td->td_pri_class != RTP_PRIO_NORMAL)) sched_prio(td, td->td_user_pri); if (TD_ON_UPILOCK(td) && oldpri != newpri) { critical_enter(); --=-fruE5NV5DEzFjKJ3O5EF-- From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 01:52:29 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5545E737; Fri, 1 Mar 2013 01:52:29 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from flat.berklix.org (flat.berklix.org [83.236.223.115]) by mx1.freebsd.org (Postfix) with ESMTP id 7CB8BE7F; Fri, 1 Mar 2013 01:52:28 +0000 (UTC) Received: from mart.js.berklix.net (pD9FBE93D.dip.t-dialin.net [217.251.233.61]) (authenticated bits=128) by flat.berklix.org (8.14.5/8.14.5) with ESMTP id r210jnTm038059; Fri, 1 Mar 2013 01:45:50 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id r210jrr4085349; Fri, 1 Mar 2013 01:45:53 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id r210j3IF038754; Fri, 1 Mar 2013 01:45:53 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201303010045.r210j3IF038754@fire.js.berklix.net> To: hackers@freebsd.org Subject: sbin/fsck_msdosfs does not handle media with 2K blocks From: "Julian H. Stacey" Organization: http://berklix.com BSD Linux Unix Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com/~jhs/cv/ Date: Fri, 01 Mar 2013 01:45:03 +0100 Sender: jhs@berklix.com Cc: Tom Rhodes , paulp@uts.amdahl.com, rnordier@freebsd.org, gdt@NetBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 01:52:29 -0000 Hi hackers@freebsd.org cc: gdt@NetBSD, paulp@uts.amdahl.com (< grep @ src/sbin/fsck_msdosfs/*.c) cc: rnordier@FreeBSD.org (< man newfs_msdos) cc: Tom Rhodes (< man 5 msdosfs> sbin/fsck_msdosfs (FreeBSD 9.1-RELEASE) fails on 2K block media, media in this case, an mp3 player: http://www.berklix.com/~jhs/txt/sigmatel/ sysctl kern.geom.debugflags=16 kern.geom.debugflags: 16 -> 16 /sbin/fsck_msdosfs /dev/da0s1 ** /dev/da0s1 could not read boot block (Invalid argument) devd.conf: "vendor" "0x066f"; "product" "0x8000"; "devclass" "0x00"; "devsubclass" "0x00"; "intclass" "0x08"; "intsubclass" "0x06"; "intprotocol" "0x50" ; "release" "0x1001"; /sys/dev/usb/usbdevs: vendor SIGMATEL 0x066f Sigmatel dd if=/dev/da0 bs=2k of=da0 # 242304+0 records 496238592 bytes dd if=/dev/da0s1 bs=2k of=da0s1 # 242256+0 records 496140288 bytes echo "2048 242304 * p" | dc # 496238592 echo "2048 242256 * p" | dc # 496140288 fdisk /dev/da0 cylinders=118 heads=64 sectors/track=32 (2048 blks/cyl) Media sector size is 2048 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 6 (0x06),(Primary DOS, 16 bit FAT (>= 32MB)) start 48, size 242256 (473 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 630/ head 7/ sector 48 ( I think this unit's FS got corrupted. Possibly as part of media error ? This unit (when not USB connected, but powered on battery as a player, flashed its LCD display on & off, (ame warning as to also indicate a low battery) whereas other units of same type with same battery powered up OK, & battery is OK, so player is probably also seeing a bad FS. ) I tried this: sysctl kern.geom.debugflags=16 dd if=/dev/da0s1 bs=2k of=image mdconfig -a -t vnode -f image /sbin/fsck_msdosfs -y /dev/md2 ** /dev/md2 ** Phase 1 - Read and Compare FATs ** Phase 2 - Check Cluster Chains ** Phase 3 - Checking Directories /audio has entries after end of directory Extend? yes /audio has entries after end of directory Extend? yes long filename record cluster start != 0 Invalid long filename entry for volume label Remove? yes ** Phase 4 - Checking for Lost Files 139 files, 60280 free (7535 clusters) ***** FILE SYSTEM WAS MODIFIED ***** ... Repeating fsck above, shows the same errors above ... newfs_msdos -F 16 -S 2048 /dev/md2 newfs_msdos: Cannot get number of sectors per track, Operation not supported newfs_msdos: Cannot get number of heads, Operation not supported newfs_msdos: trim 21 sectors to adjust to a multiple of 63 /dev/md2: 242192 sectors in 15137 FAT16 clusters (32768 bytes/cluster) BytesPerSec=2048 SecPerClust=16 ResSectors=1 FATs=2 RootDirEnts=512 Media=0xf0 FATsecs=15 SecPerTrack=63 Heads=16 HiddenSecs=0 HugeSectors=242235 /sbin/fsck_msdosfs -y /dev/md2 ** /dev/md2 ** Phase 1 - Read and Compare FATs ** Phase 2 - Check Cluster Chains ** Phase 3 - Checking Directories ** Phase 4 - Checking for Lost Files 1 files, 484384 free (15137 clusters) mount -t msdosfs /dev/md2 /mnt cp SETTINGS.DAT /mnt umount /mnt mdconfig -d -u 2 dd if=image bs=2k of=/dev/da0s1 dd: /dev/da0s1: Input/output error 80+0 records in 79+0 records out 161792 bytes transferred in 32.715037 secs (4945 bytes/sec) ... Remove & Reinsert & waited for devd & unmounted after auto-mount... Repeated dd & same error. ... Remove & Reinsert & wait for devd automount ls /usb/sigmatel ... it has the new empty FS with SETTINGS.DAT ... but still the LCD flashes fdisk -B /dev/da0 fdisk: /boot/mbr: length must be a multiple of sector size cat /boot/mbr /boot/mbr /boot/mbr /boot/mbr > /boot/mbr2k fdisk -B -b /boot/mbr2k /dev/da0 Should we write new partition table? [n] y ... still the LCD flashes fdisk -i /dev/da0 fdisk -u /dev/da0 ( Not Tried: newfs_msdos with -s 512 to see if the Sigmatel chipset will play it, but not much point as fdisk says 2K ... ) So it seems this particular unit has a problem (that other units of same manufacture here do not have). I can't think what else to try ? It seems I discovered a limitation in FreeBSD (while doing the above): We should hack various sources to [also?] allow R/W of 2K media blocks , eg inc. here: /usr/src/sbin/fsck_msdosfs/ dosfs.h: #define DOSBOOTBLOCKSIZE 512 boot.c: readboot(int dosfs, struct bootblock *boot) { u_char block[DOSBOOTBLOCKSIZE]; if (read(dosfs, block, sizeof block) != sizeof block) { perror("could not read boot block"); Who might best hack those sources ? Maybe someone since 1997 Wolfgang Solfrank & 1995 Martin Husemann be interested? Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Reply below not above, like a play script. Indent old text with "> ". Send plain text. No quoted-printable, HTML, base64, multipart/alternative. From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 13:50:31 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2E657D7A for ; Fri, 1 Mar 2013 13:50:31 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 75CC0E62 for ; Fri, 1 Mar 2013 13:50:30 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA09155 for ; Fri, 01 Mar 2013 15:50:28 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <5130B224.8000600@FreeBSD.org> Date: Fri, 01 Mar 2013 15:50:28 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers Subject: memory allocation in spinlock context X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 13:50:31 -0000 I am trying to understand if it is possible to allow memory allocations (M_NOWAIT, of course) in a spinlock context. I do not see any obvious architectural obstacles. But the fact that all of the uma locks, system map lock, object locks, page queue locks and so on are regular mutexes makes it impossible to allocate memory without violating the fundamental lock ordering rules. Could all of the above mentioned locks potentially be converted to spin mutexes? (And that seems to be a large nasty change) Are there any alternative possibilities? BTW, currently we have at least one place where a memory allocation of this kind is done stealthily (and thus dangerously?). ACPI resume code must execute AcpiLeaveSleepStatePrep with interrupts disabled and ACPICA code performs memory allocations in that code path. Since the interrupts are disabled by means of intr_disable(), witness(9) and similar are completely oblivious of the fact. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 14:22:20 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A74FF4B5 for ; Fri, 1 Mar 2013 14:22:20 +0000 (UTC) (envelope-from mjacob@freebsd.org) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 69E6BF88 for ; Fri, 1 Mar 2013 14:22:19 +0000 (UTC) Received: from [192.168.135.7] (76-14-49-207.sf-cable.astound.net [76.14.49.207]) (authenticated bits=0) by ns1.feral.com (8.14.6/8.14.4) with ESMTP id r21EMCMP087292 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 1 Mar 2013 06:22:12 -0800 (PST) (envelope-from mjacob@freebsd.org) Message-ID: <5130B990.2010001@freebsd.org> Date: Fri, 01 Mar 2013 06:22:08 -0800 From: Matthew Jacob Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: memory allocation in spinlock context References: <5130B224.8000600@FreeBSD.org> In-Reply-To: <5130B224.8000600@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (ns1.feral.com [192.67.166.1]); Fri, 01 Mar 2013 06:22:12 -0800 (PST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: mjacob@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 14:22:20 -0000 On 3/1/2013 5:50 AM, Andriy Gapon wrote: > I am trying to understand if it is possible to allow memory allocations (M_NOWAIT, > of course) in a spinlock context. > There are mechanisms to do just this- essentially by creating private pools that are organized in a way to allow for spinlock (and thus possible interrupt level) safe allocations (and failure if the pool is empty). Are you trying to make a general mechanism for this? From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 14:36:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DC9E0932; Fri, 1 Mar 2013 14:36:06 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-ve0-f178.google.com (mail-ve0-f178.google.com [209.85.128.178]) by mx1.freebsd.org (Postfix) with ESMTP id 7F5C28E; Fri, 1 Mar 2013 14:36:06 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id db10so2907879veb.23 for ; Fri, 01 Mar 2013 06:36:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=JnLxKJIIldMy8zxEUIzhiAGGrGK/gPTyFat7CbXnGD0=; b=EbH2wfVdkKRYY6JrcO8Ogbb9jUVwnrD9Zj6X7PEGEP/3YWKmgq7VCiCf33L+/Qjr7d yZLey5iShIOk00xqYMKmjHntfDvyo2zDaGvRdD/E1KlTp7Q7oNqqGBJy/lVeH8uFFuLk jXM9rsyBupF4jiMZrnFsYwgl7fEAu7rTgSl7lF3tDTjEOEqNyvqFZ9Hoophd6orcpbjt d7wnhVCOLDQu90hHQ5ADWmZ+jyqmJnue7/a7ekozs0oxtUuhA/BB535K6cX7qYSLhr34 KRmXd2o2g05ErkETLoFNjwy4tL+AX2NWGaqi7HbI8RUrTSWczJfq4GYClsBQE5n0aUk5 no2Q== MIME-Version: 1.0 X-Received: by 10.59.11.67 with SMTP id eg3mr4236056ved.31.1362148565554; Fri, 01 Mar 2013 06:36:05 -0800 (PST) Sender: davide.italiano@gmail.com Received: by 10.220.34.203 with HTTP; Fri, 1 Mar 2013 06:36:05 -0800 (PST) In-Reply-To: <5130B224.8000600@FreeBSD.org> References: <5130B224.8000600@FreeBSD.org> Date: Fri, 1 Mar 2013 15:36:05 +0100 X-Google-Sender-Auth: uP12gW5Mwo7ZR-N4fkjntBgum54 Message-ID: Subject: Re: memory allocation in spinlock context From: Davide Italiano To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 14:36:06 -0000 On Fri, Mar 1, 2013 at 2:50 PM, Andriy Gapon wrote: > > I am trying to understand if it is possible to allow memory allocations (M_NOWAIT, > of course) in a spinlock context. > I do not see any obvious architectural obstacles. > But the fact that all of the uma locks, system map lock, object locks, page queue > locks and so on are regular mutexes makes it impossible to allocate memory without > violating the fundamental lock ordering rules. > > Could all of the above mentioned locks potentially be converted to spin mutexes? > (And that seems to be a large nasty change) AFAIK they're suitable for particular uses and not in general. For example if the critical section is short, spinning rather than sleeping could avoid a potential context switches, increasing performances. OTOH has the disadvantage of wasting time that could be used in other activities. So, IMHO, if such a change need to be adopted, ti should be pondered/profiled more than a bit, and I doubt it could be used for the wide class of locks you mentioned. > Are there any alternative possibilities? > Is there anything that prevent you to drop the lock, perform the allocation in a reliable fashion (M_WAITOK) and try to reacquire the lock later on? > BTW, currently we have at least one place where a memory allocation of this kind > is done stealthily (and thus dangerously?). ACPI resume code must execute > AcpiLeaveSleepStatePrep with interrupts disabled and ACPICA code performs memory > allocations in that code path. Since the interrupts are disabled by means of > intr_disable(), witness(9) and similar are completely oblivious of the fact. > > -- > Andriy Gapon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Thanks, -- Davide From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 15:48:44 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 002994E2; Fri, 1 Mar 2013 15:48:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0392F684; Fri, 1 Mar 2013 15:48:42 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA09987; Fri, 01 Mar 2013 17:48:41 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <5130CDD8.2050107@FreeBSD.org> Date: Fri, 01 Mar 2013 17:48:40 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: mjacob@FreeBSD.org Subject: Re: memory allocation in spinlock context References: <5130B224.8000600@FreeBSD.org> <5130B990.2010001@freebsd.org> In-Reply-To: <5130B990.2010001@freebsd.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 15:48:44 -0000 on 01/03/2013 16:22 Matthew Jacob said the following: > On 3/1/2013 5:50 AM, Andriy Gapon wrote: >> I am trying to understand if it is possible to allow memory allocations (M_NOWAIT, >> of course) in a spinlock context. >> > There are mechanisms to do just this- essentially by creating private pools that > are organized in a way to allow for spinlock (and thus possible interrupt level) > safe allocations (and failure if the pool is empty). Are you trying to make a > general mechanism for this? I am just pondering what would it take to develop such a general mechanism. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 17:13:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B8765568 for ; Fri, 1 Mar 2013 17:13:42 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) by mx1.freebsd.org (Postfix) with ESMTP id 8BA24A72 for ; Fri, 1 Mar 2013 17:13:42 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id r21HDfHt043068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 1 Mar 2013 09:13:41 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id r21HDfIs043067 for freebsd-hackers@freebsd.org; Fri, 1 Mar 2013 09:13:41 -0800 (PST) (envelope-from jmg) Date: Fri, 1 Mar 2013 09:13:41 -0800 From: John-Mark Gurney To: freebsd-hackers@freebsd.org Subject: Re: looking for someone to fix humanize_number (test cases included) Message-ID: <20130301171341.GJ55866@funkthat.com> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20121225172037.GA56225@volcano.org> <20121225182355.GA56732@volcano.org> <20121225194626.GA57447@volcano.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20121225194626.GA57447@volcano.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 01 Mar 2013 09:13:41 -0800 (PST) X-Mailman-Approved-At: Fri, 01 Mar 2013 17:34:19 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 17:13:42 -0000 Clifton Royston wrote this message on Tue, Dec 25, 2012 at 09:46 -1000: > On Tue, Dec 25, 2012 at 08:23:55AM -1000, Clifton Royston wrote: > > On Tue, Dec 25, 2012 at 07:20:37AM -1000, Clifton Royston wrote: > > > On Mon, Dec 24, 2012 at 12:00:01PM +0000, freebsd-hackers-request@freebsd.org wrote: > > > > From: John-Mark Gurney > > > > To: hackers@FreeBSD.org > > > > Subject: looking for someone to fix humanize_number (test cases > > > > included) > > > > > > > > I'm looking for a person who is interested in fixing up humanize_number. > > ... > > > > So I decided to write a test program to test the output, and now I'm even > > > > more surprised by the output... Neither 7.2-R nor 10-current give what > > > > I expect are the correct results... > > ... > > > > I am bemused. > > I correct myself: the function works fine, and there are no bugs I > could find, though it's clear the man page could emphasize the correct > usage a bit more. > > I had to read the source several times and start on debugging it > before I understood the correct usage of the flag values with the scale > and flags parameters, despite the man page stating: > > The following flags may be passed in scale: > > HN_AUTOSCALE Format the buffer using the lowest multiplier pos- > sible. > HN_GETSCALE Return the prefix index number (the number of > times number must be divided to fit) instead of > formatting it to the buffer. > > The following flags may be passed in flags: > > HN_DECIMAL If the final result is less than 10, display it > using one digit. > ... > HN_DIVISOR_1000 Divide number with 1000 instead of 1024. > > That is, certain flags must be passed in flags and others must only > be passed in scale - a bit counter-intuitive. Also, scale == 0 is > clearly not interpreted as AUTOSCALE, but I am not yet clear how it is > being handled - it seems somewhat like AUTOSCALE but not identical. > > When the test program constant table is updated to pass the scale > flags as specified, as well as fixing the bugs mentioned in the > previous emails, it all passes except for the one (intentional?) > inconsistency that "k" is used in place of "K" if HN_DECIMAL is > enabled. > > The bug in the transfer speed results which prompted this inquiry > suggests that perhaps some clients of humanize_number in the codebase > are also passing the scale parameters incorrectly. I would propose > accepting HN_AUTOSCALE and HN_GETSCALE in the flags field (they don't > overlap with other values) while continuing to accept them in the scale > field for backwards compatibility. Trivial diff below. Sorry I didn't get back to this, but now I have a few minutes... > + getscale = (flags | scale) & HN_GETSCALE; This isn't good: #define HN_IEC_PREFIXES 0x10 #define HN_GETSCALE 0x10 If someone sets HN_IEC_PREFIXES, they'll acidentally enable _GETSCALE.. We could do something anoying by changing the value of _GETSCALE, and then leaving some legacy code to accept the old _GETSCALE on the scale input... This would let new code work, but would break new code on old libraries... So, I don't see an easy way to fix this... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 1 19:30:30 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 36568573; Fri, 1 Mar 2013 19:30:30 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2697E1164; Fri, 1 Mar 2013 19:30:30 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id CFD431A3C3D; Fri, 1 Mar 2013 11:30:24 -0800 (PST) Message-ID: <513101CD.9060702@mu.org> Date: Fri, 01 Mar 2013 11:30:21 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130216 Thunderbird/17.0.3 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: memory allocation in spinlock context References: <5130B224.8000600@FreeBSD.org> In-Reply-To: <5130B224.8000600@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 19:30:30 -0000 On 3/1/13 5:50 AM, Andriy Gapon wrote: > I am trying to understand if it is possible to allow memory allocations (M_NOWAIT, > of course) in a spinlock context. > I do not see any obvious architectural obstacles. > But the fact that all of the uma locks, system map lock, object locks, page queue > locks and so on are regular mutexes makes it impossible to allocate memory without > violating the fundamental lock ordering rules. > > Could all of the above mentioned locks potentially be converted to spin mutexes? > (And that seems to be a large nasty change) > Are there any alternative possibilities? > > BTW, currently we have at least one place where a memory allocation of this kind > is done stealthily (and thus dangerously?). ACPI resume code must execute > AcpiLeaveSleepStatePrep with interrupts disabled and ACPICA code performs memory > allocations in that code path. Since the interrupts are disabled by means of > intr_disable(), witness(9) and similar are completely oblivious of the fact. > Typically the need for such a facility means that the locks are being held for too long. I think someone has suggested using a private allocator carving out of a pre-allocated space. Depending on the subsystem you are allocating for this may work for you. I am looking to do this for the kernel gzip routines so that we can do compressed kernel dumps as soon as I verify the bounds of the gzip allocations. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 00:00:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AB4281A5 for ; Sat, 2 Mar 2013 00:00:48 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id 77DD11D97 for ; Sat, 2 Mar 2013 00:00:48 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 630211203CA; Sat, 2 Mar 2013 01:00:33 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 3B4EE2848C; Sat, 2 Mar 2013 01:00:33 +0100 (CET) Date: Sat, 2 Mar 2013 01:00:33 +0100 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: [patch] statfs does not detect -t nullfs -o union as a union mount Message-ID: <20130302000032.GB49921@stack.nl> References: <20130227213141.GA18210@stack.nl> <20130227222144.GG2454@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130227222144.GG2454@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 00:00:48 -0000 On Thu, Feb 28, 2013 at 12:21:44AM +0200, Konstantin Belousov wrote: > On Wed, Feb 27, 2013 at 10:31:42PM +0100, Jilles Tjoelker wrote: > > While testing recent changes to opendir(), I noticed that fstatfs() does > > not return the MNT_UNION flag for a -t nullfs -o union mount. As a > > result, opendir()/readdir() return files that exist in both top and > > bottom directories twice (at least . and ..). Other -o union mounts and > > -t unionfs mounts work correctly in this regard. > > The below patch passes through just the MNT_UNION flag of the nullfs > > mount itself. Perhaps more flags should be passed through. > > commit fce32a779af4eb977c9b96feb6e4f811d89f2881 > > Author: Jilles Tjoelker > > Date: Sat Feb 23 22:22:39 2013 +0100 > > > > nullfs: Preserve the MNT_UNION flag of the nullfs mount itself. > > > > This is needed so that opendir() can properly detect a union mount like > > mount -t nullfs -o union dir1 dir2. > > > > diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c > > index 3724e0a..ff06f57 100644 > > --- a/sys/fs/nullfs/null_vfsops.c > > +++ b/sys/fs/nullfs/null_vfsops.c > > @@ -313,7 +313,7 @@ nullfs_statfs(mp, sbp) > > > > /* now copy across the "interesting" information and fake the rest */ > > sbp->f_type = mstat.f_type; > > - sbp->f_flags = mstat.f_flags; > > + sbp->f_flags = (sbp->f_flags & MNT_UNION) | mstat.f_flags; > > sbp->f_bsize = mstat.f_bsize; > > sbp->f_iosize = mstat.f_iosize; > > sbp->f_blocks = mstat.f_blocks; > Would it make sense to preserve more flags from the upper mount ? > I see a use for MNT_NOEXEC as well, at least. Yes, preserving MNT_NOEXEC will make -t nullfs -o noexec work better, in particular rtld's check for libraries loaded via environment variables. In the same way MNT_RDONLY, MNT_NOSUID and MNT_NOSYMFOLLOW could be preserved. On the other hand, MNT_ROOTFS should probably not be passed through from the underlying filesystem. This would give sbp->f_flags = (sbp->f_flags & (MNT_RDONLY | MNT_NOEXEC | MNT_NOSUID | MNT_UNION | MNT_NOSYMFOLLOW) | (mstat.f_flags & ~MNT_ROOTFS); -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 04:13:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 630BBE3 for ; Sat, 2 Mar 2013 04:13:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id B467A86E for ; Sat, 2 Mar 2013 04:13:07 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r224D4KH040200; Sat, 2 Mar 2013 06:13:04 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.0 kib.kiev.ua r224D4KH040200 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r224D4iA040199; Sat, 2 Mar 2013 06:13:04 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 2 Mar 2013 06:13:04 +0200 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: [patch] statfs does not detect -t nullfs -o union as a union mount Message-ID: <20130302041304.GI2930@kib.kiev.ua> References: <20130227213141.GA18210@stack.nl> <20130227222144.GG2454@kib.kiev.ua> <20130302000032.GB49921@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4LwthZj+AV2mq5CX" Content-Disposition: inline In-Reply-To: <20130302000032.GB49921@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 04:13:08 -0000 --4LwthZj+AV2mq5CX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 02, 2013 at 01:00:33AM +0100, Jilles Tjoelker wrote: > On Thu, Feb 28, 2013 at 12:21:44AM +0200, Konstantin Belousov wrote: > > On Wed, Feb 27, 2013 at 10:31:42PM +0100, Jilles Tjoelker wrote: > > > While testing recent changes to opendir(), I noticed that fstatfs() d= oes > > > not return the MNT_UNION flag for a -t nullfs -o union mount. As a > > > result, opendir()/readdir() return files that exist in both top and > > > bottom directories twice (at least . and ..). Other -o union mounts a= nd > > > -t unionfs mounts work correctly in this regard. >=20 > > > The below patch passes through just the MNT_UNION flag of the nullfs > > > mount itself. Perhaps more flags should be passed through. >=20 > > > commit fce32a779af4eb977c9b96feb6e4f811d89f2881 > > > Author: Jilles Tjoelker > > > Date: Sat Feb 23 22:22:39 2013 +0100 > > >=20 > > > nullfs: Preserve the MNT_UNION flag of the nullfs mount itself. > > > =20 > > > This is needed so that opendir() can properly detect a union moun= t like > > > mount -t nullfs -o union dir1 dir2. > > >=20 > > > diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c > > > index 3724e0a..ff06f57 100644 > > > --- a/sys/fs/nullfs/null_vfsops.c > > > +++ b/sys/fs/nullfs/null_vfsops.c > > > @@ -313,7 +313,7 @@ nullfs_statfs(mp, sbp) > > > =20 > > > /* now copy across the "interesting" information and fake the rest = */ > > > sbp->f_type =3D mstat.f_type; > > > - sbp->f_flags =3D mstat.f_flags; > > > + sbp->f_flags =3D (sbp->f_flags & MNT_UNION) | mstat.f_flags; > > > sbp->f_bsize =3D mstat.f_bsize; > > > sbp->f_iosize =3D mstat.f_iosize; > > > sbp->f_blocks =3D mstat.f_blocks; >=20 > > Would it make sense to preserve more flags from the upper mount ? > > I see a use for MNT_NOEXEC as well, at least. >=20 > Yes, preserving MNT_NOEXEC will make -t nullfs -o noexec work better, in > particular rtld's check for libraries loaded via environment variables. >=20 > In the same way MNT_RDONLY, MNT_NOSUID and MNT_NOSYMFOLLOW could be > preserved. >=20 > On the other hand, MNT_ROOTFS should probably not be passed through from > the underlying filesystem. >=20 > This would give > sbp->f_flags =3D (sbp->f_flags & (MNT_RDONLY | MNT_NOEXEC | MNT_NOSUID | > MNT_UNION | MNT_NOSYMFOLLOW) | (mstat.f_flags & ~MNT_ROOTFS); I think this is fine. --4LwthZj+AV2mq5CX Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRMXxPAAoJEJDCuSvBvK1BBfsP+QERYVXfGU9X12Fv7nNZtQRC rtBkXuhqDGQ4iVjJ0JVNK6gU40X5mnIlqi3G2o+O+SeBDJYLX6phCWk4UIEg0nbj VXw5cObZsyp9iJX7B7ptGHLBV1U9hZOX8t8lAzfMWbjyjAXHRUPtr2yFTxGslOQM ecLJfcnKhy90u/QQzTGEzPLs4MnUhHZHTn70d95SViBC8Dp8YRXlgCII9mstmOBE GroumBgGVdgSvA6Wxm0uUBUHf1YaklY8GTbQg+ptp5v+MfxGOTOtGxbKT0BKyRhg f64iUlBf3FXn9PtuJOS7j3h1IeBMKOwj0Wi6QsTcFY7E5hQbidUA4qk7rVDpQEjN 2pFteBXPkCQw1FTtla8o8T465hixHWUmEKVx7QE3ajk2KzMQyyU3An8KmRWTlNw4 ZzZOKjJnO1ZZF3O9u6UYY1Kxm7HkIyBeYNdiXlTWzIa3dzY61uP/f1muaIAFMt+1 fUcFENkwQjroj8VKepGmSigWhCJ62NVWkcIjNvi7zzDrGnuqD8GB2Ck2GeMU0XP4 KGd+BJy6b3F24aifI2CEC4AZH343g3kbXI6wmbeFZ0ZYXkJuv6Aix/7PtvhMHsgA F3l8qVHQ30sBeGMXSxdzFFc+Z8+VtzbvtGF1M8BwwkjwhzC1NykZCpzLi+FIkeDh Ukb83c9OCgKu0wNceXto =riYL -----END PGP SIGNATURE----- --4LwthZj+AV2mq5CX-- From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 17:36:12 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 50629E77 for ; Sat, 2 Mar 2013 17:36:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4D1C194A for ; Sat, 2 Mar 2013 17:36:10 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA21721 for ; Sat, 02 Mar 2013 19:36:02 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UBqLp-000Aty-OM for freebsd-hackers@FreeBSD.org; Sat, 02 Mar 2013 19:36:01 +0200 Message-ID: <5132387E.8010808@FreeBSD.org> Date: Sat, 02 Mar 2013 19:35:58 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: clang generated code sometimes confuses fbt X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 17:36:12 -0000 I observe the following problem. There are two tiny wrapper functions around a larger implementation function: int bpobj_iterate(bpobj_t *bpo, bpobj_itor_t func, void *arg, dmu_tx_t *tx) { return (bpobj_iterate_impl(bpo, func, arg, tx, B_TRUE)); } int bpobj_iterate_nofree(bpobj_t *bpo, bpobj_itor_t func, void *arg, dmu_tx_t *tx) { return (bpobj_iterate_impl(bpo, func, arg, tx, B_FALSE)); } On a clang compiled system: $ dtrace -l | fgrep bpobj_iterate 1483 fbt kernel bpobj_iterate_impl entry 1484 fbt kernel bpobj_iterate_impl return On a gcc compiled system: dtrace -l | fgrep bpobj_iterate 647 fbt kernel bpobj_iterate_impl entry 648 fbt kernel bpobj_iterate_impl return 20656 fbt kernel bpobj_iterate entry 20657 fbt kernel bpobj_iterate return 28426 fbt kernel bpobj_iterate_nofree entry 28427 fbt kernel bpobj_iterate_nofree return Examination reveals why that is so. clang: Dump of assembler code for function bpobj_iterate: 0xffffffff802d5a90 : mov $0x1,%r8d 0xffffffff802d5a96 : jmp 0xffffffff802d5aa0 gcc: Dump of assembler code for function bpobj_iterate: 0xffffffff802d3f43 : push %rbp 0xffffffff802d3f44 : mov %rsp,%rbp 0xffffffff802d3f47 : mov $0x1,%r8d 0xffffffff802d3f4d : callq 0xffffffff802d3787 0xffffffff802d3f52 : pop %rbp 0xffffffff802d3f53 : retq So quite obviously fbt can not really entry/return points for the clang function. This is not a big problem on its own, of course, but here is a bad twist. On the clang system: $ ctfdump -f /boot/kernel/kernel | fgrep bpobj_iterate [8975] FUNC (bpobj_iterate) returns: 24 args: (2601, 4824, 34, 2221) [13093] FUNC (bpobj_iterate_nofree) returns: 24 args: (2601, 4824, 34, 2221) Now that's the problem: fbt sees only bpobj_iterate_impl, but CTF data is generated/present only for bpobj_iterate and bpobj_iterate_nofree. On the gcc system: ctfdump -f /boot/kernel/kernel | fgrep bpobj_iterate [323] FUNC (bpobj_iterate_impl) returns: 1 args: (5153, 5661, 6, 5078, 1350) [10439] FUNC (bpobj_iterate) returns: 1 args: (5153, 5661, 6, 5078) [14377] FUNC (bpobj_iterate_nofree) returns: 1 args: (5153, 5661, 6, 5078) To summarize: I would be glad of either clang generated code was "fbt-friendly" or if ctf information was generated for bpobj_iterate_impl. Either is perfect for me. Now, I am not quite sure why ctfconvert skips bpobj_iterate_impl in the clang-generated code. Seems like some sort of a bug in ctfconvert. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 17:52:38 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B259A9B9; Sat, 2 Mar 2013 17:52:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9978F9FF; Sat, 2 Mar 2013 17:52:37 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA21821; Sat, 02 Mar 2013 19:52:36 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UBqbr-000Avp-KX; Sat, 02 Mar 2013 19:52:35 +0200 Message-ID: <51323C62.4040506@FreeBSD.org> Date: Sat, 02 Mar 2013 19:52:34 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> In-Reply-To: <5132387E.8010808@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 17:52:38 -0000 on 02/03/2013 19:35 Andriy Gapon said the following: > Now, I am not quite sure why ctfconvert skips bpobj_iterate_impl in the > clang-generated code. Seems like some sort of a bug in ctfconvert. It seems that gcc and clang put different names for symbol of type FILE: clang: readelf -a -W /usr/obj/usr/src/sys/TRANT/bpobj.o| fgrep -w FILE 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c gcc: readelf -a -W /usr/obj/usr/src/sys/ODYSSEY/bpobj.o| fgrep -w FILE 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS bpobj.c ctfconvert seems to compare this value with "bpobj.c" and so in the clang case it doesn't recognize the static symbols. Does my analysis seem reasonable? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 20:23:13 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 50360EB4; Sat, 2 Mar 2013 20:23:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) by mx1.freebsd.org (Postfix) with ESMTP id 19465FB; Sat, 2 Mar 2013 20:23:13 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 722C55C43; Sat, 2 Mar 2013 21:23:09 +0100 (CET) Message-ID: <51325FB3.7080300@FreeBSD.org> Date: Sat, 02 Mar 2013 21:23:15 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20130117 Thunderbird/19.0 MIME-Version: 1.0 To: Andriy Gapon , freebsd-hackers@FreeBSD.org Subject: Re: clang generated code sometimes confuses fbt References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> In-Reply-To: <51323C62.4040506@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 20:23:13 -0000 On 2013-03-02 18:52, Andriy Gapon wrote: > on 02/03/2013 19:35 Andriy Gapon said the following: >> Now, I am not quite sure why ctfconvert skips bpobj_iterate_impl in the >> clang-generated code. Seems like some sort of a bug in ctfconvert. > > It seems that gcc and clang put different names for symbol of type FILE: > clang: > readelf -a -W /usr/obj/usr/src/sys/TRANT/bpobj.o| fgrep -w FILE > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c > > gcc: > readelf -a -W /usr/obj/usr/src/sys/ODYSSEY/bpobj.o| fgrep -w FILE > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS bpobj.c > > ctfconvert seems to compare this value with "bpobj.c" and so in the clang case > it doesn't recognize the static symbols. > > Does my analysis seem reasonable? Have you verified that ctfconvert does the right thing, if you modify the FILE symbol to have just the filename? Indeed, clang puts the original filename from the command line in the .file directive, while gcc explicitly removes any directory names; see contrib/gcc/toplev.c, around line 680: void output_file_directive (FILE *asm_file, const char *input_name) { int len; const char *na; if (input_name == NULL) input_name = ""; len = strlen (input_name); na = input_name + len; /* NA gets INPUT_NAME sans directory names. */ while (na > input_name) { if (IS_DIR_SEPARATOR (na[-1])) break; na--; } ... That "NA gets INPUT_NAME sans directory names" comment was inserted by rms in r279. :-) So I guess this is the way gcc has done it from the start, but there is no explanation as to why rms chose to remove those directory names. I do not see the problem, except maybe for having reproducible builds? -Dimitry From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 2 20:52:33 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5896443C; Sat, 2 Mar 2013 20:52:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id A62E61E5; Sat, 2 Mar 2013 20:52:32 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r22KqNMT031061; Sat, 2 Mar 2013 22:52:23 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.0 kib.kiev.ua r22KqNMT031061 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r22KqNqm031060; Sat, 2 Mar 2013 22:52:23 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 2 Mar 2013 22:52:23 +0200 From: Konstantin Belousov To: Dimitry Andric Subject: Re: clang generated code sometimes confuses fbt Message-ID: <20130302205223.GR2930@kib.kiev.ua> References: <5132387E.8010808@FreeBSD.org> <51323C62.4040506@FreeBSD.org> <51325FB3.7080300@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VdnGiXwuH6t1Tqzo" Content-Disposition: inline In-Reply-To: <51325FB3.7080300@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@FreeBSD.org, Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Mar 2013 20:52:33 -0000 --VdnGiXwuH6t1Tqzo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 02, 2013 at 09:23:15PM +0100, Dimitry Andric wrote: > On 2013-03-02 18:52, Andriy Gapon wrote: > > on 02/03/2013 19:35 Andriy Gapon said the following: > >> Now, I am not quite sure why ctfconvert skips bpobj_iterate_impl in the > >> clang-generated code. Seems like some sort of a bug in ctfconvert. > > > > It seems that gcc and clang put different names for symbol of type FILE: > > clang: > > readelf -a -W /usr/obj/usr/src/sys/TRANT/bpobj.o| fgrep -w FILE > > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS > > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c > > > > gcc: > > readelf -a -W /usr/obj/usr/src/sys/ODYSSEY/bpobj.o| fgrep -w FILE > > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS bpobj.c > > > > ctfconvert seems to compare this value with "bpobj.c" and so in the cla= ng case > > it doesn't recognize the static symbols. > > > > Does my analysis seem reasonable? >=20 > Have you verified that ctfconvert does the right thing, if you modify > the FILE symbol to have just the filename? >=20 > Indeed, clang puts the original filename from the command line in the > .file directive, while gcc explicitly removes any directory names; see > contrib/gcc/toplev.c, around line 680: >=20 > void > output_file_directive (FILE *asm_file, const char *input_name) > { > int len; > const char *na; >=20 > if (input_name =3D=3D NULL) > input_name =3D ""; >=20 > len =3D strlen (input_name); > na =3D input_name + len; >=20 > /* NA gets INPUT_NAME sans directory names. */ > while (na > input_name) > { > if (IS_DIR_SEPARATOR (na[-1])) > break; > na--; > } > ... >=20 > That "NA gets INPUT_NAME sans directory names" comment was inserted by > rms in r279. :-) So I guess this is the way gcc has done it from the > start, but there is no explanation as to why rms chose to remove those > directory names. I do not see the problem, except maybe for having > reproducible builds? I seems that at least gdb also depends on the stripping the path for stabs (which is not dwarf) debugging format interpretation. On the FreeBSD system, do 'info', select 'stabs' -> Stab Sections -> Elf Linker Relocation. The last paragraph documents the gdb requirements. So it seems that stripped path in the STT_FILE is the common expectation. --VdnGiXwuH6t1Tqzo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRMmaHAAoJEJDCuSvBvK1BY2sQAJrnIETDlx7M+Z80EA3swFot wCJ3vd8Y4+HfECXSGxCbPxE7OOAhmnlns5Ub5HSt3qzq+KDJ9O455E4AscOA0V/N 0OMiLW5+7abGbtPXtf6zr6at3rou/rq1NGg3gln7S+qJcb0HzhOFy8MnivY59miv 1CRVIAXgiXemmzgamDnCAsoJyG1jp1DyEvKcDkF833n9mlZqdRe2gIJZfJL/ldjB BCsipwIkOsgAZd+f3VZDMGK50cgeHNMeSFzzareDr0aqI7chIlAxnX9dQxoasnPt 5z+GD/MlLc0CJ2mHXywi2Tq1vYt6D6fjgyd22paRgvN9fRvpJYK3fMI/t8KgD0B2 I0KcRZ9dmm04ULSXbbnQFSdUyOofmRtow9a+Bwd/EnAqiz+ec03mvb72EVanP6Jo nkyOf5AS5M+WMDfIbX23JyGRuxFvaMMng/kwk65vJEEJ6vh8CfQkTkerWJCNZEOx ZjET8aWuoAlC7xmgM/S1RCicHNFuPHcFt8lEIwXE5YjtPZycUMXPXTLWgwPNTCip +sc7YcBDQ1DW8Gr0CvcKgeBDBaxUD8rWSCON+4SFVi2hggInXZCxpfsGDNGqdvs6 bP3ZyzehGFnWjUc0szNVDQQtqBLWstdV73y+yj61Yg2sEMdEjtTWtJE61R7+atke pRNNNSd+2F7h1oD79Q4z =3KDl -----END PGP SIGNATURE----- --VdnGiXwuH6t1Tqzo--